feat: accept GCP credentials Secret in storage integration clients (#4771)

Adds GCP inputs to the existing database-owned storage integration
methods. GCP takes one Secret reference; the Secret stores the complete
application credentials JSON as an opaque string. The same methods and
SQL transport handle create, describe, list and drop.

```python
info = db.create_storage_integration(
    "gcs", "gs://customer-data/raw/", storage_provider="GCP",
    gcp_credentials_json_secret=SecretReference(
        "gcp_json", database="shared", namespace_path=("credentials",),
    ),
)
print(info.configuration_instructions)
info = db.describe_storage_integration("gcs")
names = list(db.list_storage_integrations())
db.drop_storage_integration("gcs")
```

```sql
CREATE STORAGE INTEGRATION gcs
    STORAGE_PROVIDER = 'GCP'
    STORAGE_PREFIX = 'gs://customer-data/raw/'
    GCP_CREDENTIALS_JSON_SECRET = shared.credentials.gcp_json;
DESCRIBE STORAGE INTEGRATION gcs;
SHOW STORAGE INTEGRATIONS;
DROP STORAGE INTEGRATION gcs;
```

```mermaid
flowchart LR
    Python[Sync or async Python] --> SQL[Existing Flight SQL transport]
    SQL --> Config[Configuration and one Secret reference]
```

Adds Rust request construction, SQL rendering and typed metadata
decoding, plus sync/async Flight SQL regressions across the three Arrow
string encodings. Rejects missing or mixed-cloud credential options.
Local builds/tests are excluded by request; validation runs in CI.
This commit is contained in:
Jack Ye
2026-10-31 01:11:25 -07:00
committed by GitHub
parent b15fa9d6e4
commit 1fb2dd1551
7 changed files with 246 additions and 9 deletions
+38
View File
@@ -1023,3 +1023,41 @@ DROP STORAGE INTEGRATION customer_blobs;
```
Registration returns setup instructions and leaves `access_verified` false. It stores an account-qualified prefix such as `az://customer-data@customeraccount.blob.core.windows.net/raw/`.
### GCP storage integrations
Create one Secret containing the complete GCP application credentials JSON file
content, then reference its name when registering Google Cloud Storage access:
```python
from pathlib import Path
db.create_secret("gcp_credentials", Path("credentials.json").read_text())
integration = db.create_storage_integration(
"customer_gcs",
"gs://customer-data/raw/",
storage_provider="GCP",
gcp_credentials_json_secret="gcp_credentials",
)
print(integration.configuration_instructions)
```
The equivalent SQL references the existing Secret:
```sql
CREATE STORAGE INTEGRATION customer_gcs
STORAGE_PROVIDER = 'GCP'
STORAGE_PREFIX = 'gs://customer-data/raw/'
GCP_CREDENTIALS_JSON_SECRET = public.gcp_credentials;
DESCRIBE STORAGE INTEGRATION customer_gcs;
SHOW STORAGE INTEGRATIONS;
DROP STORAGE INTEGRATION customer_gcs;
```
`gcp_credentials_json_secret` names a Secret; pass the file content to
`create_secret`, not to `create_storage_integration`. Use `SecretReference` to
qualify another database or namespace. Sync and async connections accept the same
options. The service interprets the stored JSON when accessing storage; registration
does not validate its credentials format or verify access, and leaves
`access_verified` false. Integration metadata contains the Secret reference, never
its JSON content. GCP options cannot be combined with AWS or Azure options.