build_gcs_client always called `.with_service_account_key(...)`, so an
absent key (the settings UI stores "no key" as the empty JSON object `{}`)
was handed to the builder and failed to parse instead of falling through
to the object_store crate's InstanceCredentialProvider. Skip the call when
the key is blank so GCS uses the instance's ambient credentials (GKE
Workload Identity / the GCP metadata server).
"Blank" (empty/whitespace/`{}`/`null`) is centralized in a shared
`gcs_service_account_key_is_blank` predicate so the build path and the
non-super-admin connectivity-test SSRF guard (`validate_object_storage_test`)
agree on what counts as "no key" — otherwise a blank key would bypass the
guard yet still trigger the ambient-credential fallback, letting an
untrusted caller probe arbitrary buckets with the server's instance role.
Also clarify the settings UI hint that the key may be left empty for
ambient credentials, and add regression tests for the blank-key build path
and the guard.
Fixes WIN-2110
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GCS service account key JSON contains the secret private_key and was rendered in plain text in the settings editor on every page load (unlike S3 secret_key / Azure accessKey, which use password inputs). When a key is already configured, hide the editor behind an explicit "Show sensitive values" reveal; the editor (and thus the private_key) is only rendered on opt-in. bucket_config keeps the real key untouched while hidden, so saving round-trips correctly.
Fixes WIN-2106
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: ignore NotFound errors when deleting log files from object store
Periodic and manual log cleanup delete log files from instance object
storage. S3's DeleteObjects silently ignores missing keys, but GCS
returns a 404 for each individual delete, which the object_store crate's
default delete_stream surfaces as Error::NotFound. This produced noisy
error/warning logs on every cleanup cycle even though the cleanup
succeeded (DB records are removed regardless).
Treat a NotFound delete as a successful no-op in both delete handlers:
- monitor.rs: skip logging NotFound errors
- log_cleanup.rs: count NotFound as deleted instead of an error
Fixes WIN-2081
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: report 404 (already-absent) count in object store log cleanup
Track delete calls that returned 404 (object already absent) separately
from real deletes so operators can see how many of the attempted deletes
were no-ops, instead of those numbers silently folding into s3_deleted.
- monitor.rs: emit a final info summary per cleanup cycle:
"N deleted, M already absent (404), K failed" (only when work occurred)
- log_cleanup.rs: add s3_not_found to LogCleanupProgress (serde default for
backward-compatible deserialization of in-flight rows), thread it through
s3_bulk_delete and all call sites, and log a final summary on release
- openapi.yaml + generated client + ObjectStoreConfigSettings.svelte:
surface the 404 count in the manual cleanup status UI
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: import ObjectStoreError directly from object_store_reexports
The object_store_reexports module already re-exports object_store::Error
under the name ObjectStoreError, so `Error as ObjectStoreError` failed to
resolve (no `Error` in that module). This compiles only behind the
parquet feature, which the local dev `cargo watch` doesn't enable, so it
was caught by CI's full-feature check rather than locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: log cleanup scans S3 orphans and works cross-server
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: don't skip service log orphan scan when job retention is disabled
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: time-based heartbeat + flag partial folder sizes on list errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: move background_task module from common to api-settings
Only log_cleanup and storage_usage use it today, both in windmill-api-settings.
Keeping it in the consumer crate narrows the blast radius; if workers or
indexer later need cross-server lease+progress coordination they can move it
back to common then.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* backend
* iterate
* all
* all
* all
* iterate
* revert
* all
* add tracing to get of authed client
* all
* all
* lal
* all
* update
* fix
* push
* all
* all
* revert
* frontend
* fix checks
* avoid deadlock
* safer
* fix
* fix
Ruben FiszelandClaude Fable 5
Guilhem
hugocasa
Henri Courdent
Ruben Fiszel