mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-21 08:02:26 +00:00
script-editor-edit-code-debug
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 hugocasa
|
ee3d82f01f |
fix(native-triggers): serialize Google channel renewal across replicas (#9060)
* fix(native-triggers): serialize Google channel renewal across replicas `sync_all_triggers` runs every 5 minutes on every windmill-app replica with no leader election. Multiple replicas were each rotating the webhook token, creating a new Google watch channel, and racing the trigger UPDATE — leaving the loser's new token (in `token`) and channel (in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week without the silent best-effort `delete_token_by_hash` ever logging a warning. Wrap each per-trigger renewal in a transaction and acquire the row with `SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row instead of duplicating the work. The lock spans `rotate_webhook_token` → Google API call → `update_native_trigger_service_config` and is only released on commit. Re-checks `should_renew_channel` after acquiring the lock so a replica that committed seconds earlier doesn't trigger a duplicate renewal. The pattern matches existing batch-cleanup paths in `monitor.rs` (job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites. Also logs at `debug!` when `delete_token_by_hash` finds no matching row, so future investigations can distinguish "deleted" from "not found" without changing the `Ok(false)` contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address claude review: - #5: per-skip log info -> debug (expected outcome under SKIP LOCKED) - #2: warn moved out of delete_token_by_hash to the call site that knows the expected state (try_renew_channel_locked); other callers are race-prone and shouldn't warn - #3: NULL service_config now warns (anomalous case) - #4: post-Google-API DB-update + commit failures log distinctly so the channel-orphan case is grep-able Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration, mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the existing 'ephemeral-' filter excludes them from user-token email/critical-alert paths (no filter changes in 3 places). Orphans now self-clean via the existing expiry sweep in monitor.rs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address second-round review: - Claude #1 (P2): username_override_from_label now strips the 'ephemeral-' prefix for ephemeral-webhook-* labels, so created_by stays webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-... (preserves audit/job-list filter compatibility) - Codex (P2): updated renew_channel doc — labels are no longer copied; rotate mints fresh ephemeral-webhook-google-{rd5} with 14d expiration - Claude #3 (optional): test_rotate_webhook_token now asserts the rotated Google token has an ephemeral-webhook-google-* label and a populated expiration Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Reconsider the previous fixup: stripping the 'ephemeral-' prefix made created_by no longer match token.label exactly, defeating the linking purpose. Just allowlist 'ephemeral-webhook-' alongside the other recognized webhook/email/ws prefixes — created_by becomes ephemeral-webhook-google-XXXXX, matching token.label exactly. The 'ephemeral-' substring also informs operators that this is a system-managed auto-expiring token vs a user-managed webhook trigger. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
 Ruben Fiszel
|
5b3913052e |
refactor: convert read-hot globals to AtomicBool/I64 and ArcSwap (#8815)
* refactor: extract load helpers from reload_setting family Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert atomic primitive globals to AtomicBool/AtomicI64 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert CRITICAL_*/HUB_API_SECRET/INSTANCE_EVENTS_WEBHOOK/JWT_SECRET to ArcSwap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to arcswap-refactor EE branch commit * refactor: convert BASE_URL/HUB_BASE_URL/MIN_VERSION/LICENSE_KEY*/LICENSE_KEY_ID to ArcSwap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert worker hot-path globals to ArcSwap (WORKER_CONFIG et al) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to combined arcswap-urls+worker EE commit * chore: update ee-repo-ref to d8be8f88cb8898c8f6b27421989d53528223815d This commit updates the EE repository reference after PR #532 was merged in windmill-ee-private. Previous ee-repo-ref: c375aaaac9ec0fc0480993627d0defc8054c31a4 New ee-repo-ref: d8be8f88cb8898c8f6b27421989d53528223815d Automated by sync-ee-ref workflow. * fix: cleanup unused imports + fix 2 missed WORKER_CONFIG readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to ce0f8fbbbde09c4a858312d2d8716d224e99042c This commit updates the EE repository reference after PR #534 was merged in windmill-ee-private. Previous ee-repo-ref: 450b601b5aba0ca0b2045f4b5071aa8701b4bfb7 New ee-repo-ref: ce0f8fbbbde09c4a858312d2d8716d224e99042c Automated by sync-ee-ref workflow. * fix: secret_backend_integration test — BASE_URL.write().await → .store() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert APP_WORKSPACED_ROUTE to AtomicBool for symmetry with HTTP_ROUTE_WORKSPACED_ROUTE Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to e587df8 (post-#535 merge) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
f2be625348 |
feat: store hashed tokens instead of plaintext (#8217)
* feat: store hashed tokens in the token table instead of plaintext
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review issues in token hash migration
- Update all base.sql fixtures to include token_hash/token_prefix columns
- Keep plaintext token for webhook tokens (needed for URL reconstruction)
- Restore get_token_by_prefix to query DB for webhook tokens
- Fix down migration to delete NULL-token rows before restoring NOT NULL
- Update parser fixture standalone schema
- Update EE dedicated_worker_ee.rs to use token_hash/token_prefix
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: restore sqlx offline cache (only add new query files)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: keep writing plaintext token column for backward compat
Write to token column alongside token_hash until MIN_VERSION_SUPPORTS_TOKEN_HASH
(1.649.0) is reached. This ensures older workers can still authenticate
during rolling upgrades. Remove the separate UPDATE in new_webhook_token
since create_token_internal now writes plaintext directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: branch on MIN_VERSION to write plaintext token or null
Check MIN_VERSION_SUPPORTS_TOKEN_HASH at runtime: write plaintext to
token column while old workers exist, switch to NULL once all workers
are >= 1.649.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: set MIN_VERSION_SUPPORTS_TOKEN_HASH to 1.650.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use token_hash for email lookup and expiry notifications
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: rotate webhook tokens instead of recovering plaintext from DB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use token_hash for native trigger token lookups and deletes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* sqlx
* refactor: drop webhook_token_prefix from native_trigger table
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: backward compat for token rotation and make webhook_token_hash NOT NULL
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: prevent panic on short superadmin secret token prefix
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: prevent panic on short superadmin secret token prefix
Replace all `token[0..TOKEN_PREFIX_LEN]` slicing with
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent
panics when a token shorter than 10 chars is provided (e.g.
malformed Authorization header, short superadmin secret).
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
* fix: prevent panic on short token prefix slicing
Replace all `token[0..TOKEN_PREFIX_LEN]` with safe
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent panics
on malformed tokens shorter than 10 characters.
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "fix: prevent panic on short superadmin secret token prefix"
This reverts commit
|