starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-21 08:02:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
13,594 Commits 733 Branches 1,511 Tags
script-editor-edit-code-debug
Commit Graph

5 Commits

Author SHA1 Message Date
hugocasa ba4b368706 fix: prevent variable push from corrupting is_secret variables (#9705) 
* fix: prevent variable push from corrupting is_secret variables

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(cli): unit-test looksLikeWorkspaceCiphertext shape detection

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): scope is_secret downgrade to single-file push, not sync push

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): warn when variable push stores a secret value as already-encrypted

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): route workspace-resolution and auth diagnostics to stderr

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(cli): rephrase comments to describe current behavior, not history

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-23 12:45:36 +02:00
Ruben Fiszel 0ba128afe7 fix(security): scope variable and resource value caches by caller identity (#9427) 
The variable and resource value caches (backing
`GET /api/w/{w}/variables/get_value/{path}?allow_cache=true` and
`.../resources/get_value_interpolated/{path}?allow_cache=true`) are consulted
before the per-folder RLS query and store the already-decrypted value. The
resource cache was keyed only by `workspace:path` with no caller identity, so a
cache entry warmed by a privileged peer using `allow_cache=true` could be
returned to a caller with no access to the resource's folder on a cache hit
within the 30s TTL — leaking another folder's decrypted secrets.

Scope both caches to the caller's full authorization identity. The key is now
`auth_identity(authed):workspace:path`, where `auth_identity` is a SHA-256 of the
caller's effective authorization context (email, username, is_admin, is_operator,
sorted groups, sorted folders, sorted scopes) — mirroring
`job_read_access_cache_key`. Email alone is insufficient: the same email can
resolve to different effective permissions via job/owner-scoped tokens, so a
lower-privilege context must not reuse a higher-privilege context's entry.

Job-context resource interpolation is handled correctly: only `$WM_*` contextual
variables are resolved (and only when a `job_id` is present). The interpolation
reports whether the value contains a `$WM_*` placeholder
(`transform_json_value_tracked` + an `AtomicBool`). A value containing one is
job-dependent — even on a no-job read where it's left unresolved — and is never
cached (so a later job read never gets a stale placeholder or another job's
context). Any value without a `$WM_*` placeholder is job-independent and cached
under the identity key, shared across job contexts, so reads carrying a `job_id`
still hit the cache.

BEHAVIOR CHANGE: custom workspace environment variables are no longer interpolated
into resource values via `$NAME` (this was undocumented and prevented caching of
any `$`-prefixed value). Custom envs remain available to scripts/workers as before.
Built-in `$WM_*` contextual variables in resource values are unchanged.

The variable cache previously wrote with an identity-scoped key but read with the
unscoped key, so it never hit (a latent functional bug that happened to be safe).
Aligning the read path enables the cache and makes it identity-scoped by
construction. Secret variables are cached too, but the entry carries the
`is_secret` flag so a cache hit re-runs the per-read side effects a secret read
performs — the EE `variables.decrypt_secret` audit and running-job secret
registration (factored into `audit_decrypt_secret`, shared by both paths).

The unused `invalidate_{variable,resource}_cache` helpers can no longer target
identity-scoped entries; documented the constraint and refreshed the stale
key-format docs on the cache statics.

Tests:
- integration regression for both caches: a folder-scoped user warms the cache via
  allow_cache=true, then a user without folder access is denied (401) and never
  receives the cached value.
- integration regression that variables (secret included) are served from cache.
- integration regression for job context: plain and non-`$WM_` `$`-string resources
  stay cached and are served under a job_id, while a `$WM_*` resource (warmed without
  a job_id) is not cached.
- unit tests for `auth_identity`.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-03 10:31:45 +00:00
hugocasa d6c642b170 feat: add Azure Event Grid triggers (#8888) 
* feat: add Azure Event Grid triggers (EE)

Introduces a new enterprise trigger kind `azure` that supports three
modes via a single unified trigger type:
- basic_push: Azure Event Grid basic — custom topics, system topics
  (Storage, Resource Manager, Key Vault, etc.), domains (push only)
- namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push)
- namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token
  ack/reject for dead-lettering)

Auth uses a Service Principal resource (tenant_id, client_id,
client_secret, subscription_id). Subscriptions are created in
CloudEvents 1.0 schema so the push webhook handler and the pull listener
share one payload parser.

Backend
- New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from
  windmill-ee-private)
- Migration `azure_trigger` table with CHECK constraints enforcing
  mode/columns coherence
- `TriggerKind::Azure`, `JobTriggerKind::Azure`,
  `DeployedObject::AzureTrigger` variants
- Push route `/api/azure/w/{workspace}/*path` handles classic
  Event Grid SubscriptionValidation handshake and CloudEvents 1.0
  abuse-protection OPTIONS handshake
- Optional inbound JWT validation (audience check only for v1)
- Feature flag `azure_trigger` propagated through windmill-api,
  windmill-store (resource helper), and added to ee_core

Frontend
- `triggers/azure/` editor with mode toggle (basic/namespace-push/
  namespace-pull) and per-mode config (topic ARM id / namespace +
  topic name / subscription / filters / push auth / pull options)
- Registered in icon map, display names, save functions, badge,
  wrapper, editor, add-trigger menu

OpenAPI
- `AzureTrigger`, `AzureTriggerData`, `AzureMode`,
  `AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection`
  schemas; `/azure_triggers/*` endpoints; client regenerated

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8

This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private.

Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9

New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8

Automated by sync-ee-ref workflow.

* feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity

Frontend:
- Split mode selector into Namespace/Basic + Pull/Push
- ARM resource dropdowns (namespaces, Basic topics, namespace topics)
  populated from the service principal; cascade with stale-selection
  reset on SP / edition change
- Remove stale authenticate toggle + audience input (server-managed
  push_auth_config has replaced them)
- Azure listing page: "Create from template" button; "Also delete Azure
  subscription" toggle in the delete modal; simplified trigger label
  falling back to path
- AzureCapture.svelte: "Test subscription name" with -wm-capture suffix
- CompareWorkspaces.svelte: wire Azure for fork/compare
- Drop Trigger-deployed/event-loss warning (capture subscription is
  isolated with -wm-capture)

Backend:
- Shared-secret push auth (see EE crate for detail)
- JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)]
  so clients/CLI/exports never see it
- Drop redundant enabled column; mode supersedes
- Azure capture infra: AzureTriggerConfig + set_azure_trigger_config +
  azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on
  capture subscriptions so they bound storage after tab close
- Granular ACLs, users offboarding, trash, git-sync deployed-object:
  all include azure_trigger

CLI:
- Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath,
  trigger commands (get/update/create/list/template), sync delete
  switch + regex; e2e test for `trigger new --kind azure`
- system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger;
  auto-generated/* regenerated

Skill:
- .claude/skills/adding-a-trigger/ checklist covering every file that
  needs editing when wiring a new trigger type (learned from this PR)

ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts

- frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger'
  to the Kind type so the listing page's "Permissions" action compiles
  (ts2345 — caught by npm_check on CI, missed by fast-check locally).
- system_prompts/auto-generated/: regenerate to drop the stale
  delivery_config / AzureDeliveryConfig fields from the Azure schema
  (check-freshness on CI).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(azure-trigger): use workspace constant_time_eq crate

Drop hand-rolled constant-time compare in favour of the workspace
constant_time_eq crate (same one used by http_trigger_auth).

ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): pass placeholder + disabled via inputProps

`TextInput`'s `placeholder` and `disabled` go through its `inputProps`
prop — CI's `npm run check` caught the stale top-level passing that
`npm run check:fast` missed. Align with the DefaultEmailConfigSection
pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213

The hub deploy of the azure-aware sync-script is version 28213, not
28214. Backend was pinning a non-existent hub script, which broke the
git_sync_e2e suite (every deploy's sync step 404'd).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): add azure_triggers to token scope selector + skill

- windmill-api/src/token.rs: `build_trigger_scope_domains` was missing
  `("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope
  selector didn't surface azure_triggers:read/write. Backend already had
  `ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it.
- .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related
  files under the hardcoded-arrays section so future triggers don't miss
  the UI surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(adding-a-trigger-skill): clarify token.rs scope effect

Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS
just means the scope works via API/CLI but has no UI checkbox.

* docs(adding-a-trigger-skill): trim token.rs bullet

* fix(azure-trigger): regen openapi-deref + swap textarea for TextInput

- Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the
  12 azure_triggers paths + schemas. These files are served by the
  runtime (include_str! in windmill-api/src/lib.rs) to external SDK
  consumers; without this regen the new endpoints wouldn't be advertised.
- Replace the raw <textarea> for event type filters with the
  design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw
  HTML elements).

Addresses cubic + claude PR review items.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-04-23 16:30:18 +00:00
Ruben Fiszel c57c769dea feat: add CI test scripts with auto-trigger on deploy (#8736) 
* feat: add CI test scripts with auto-trigger on deploy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fix annotation parser early return and handle renames correctly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move CI test results to top of script/flow detail pages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve CI test results spacing, icon, and remove pass label

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: support one-line annotation and use script/path format

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move CI test trigger logic to EE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move CI badge next to New badge and add deduplicated CI summary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add CI test e2e tests and fix nullable column annotations

Add integration tests for CI test annotation parsing (creates/removes
ci_test_reference rows) and the CI test results API (single + batch
endpoints). Add backend test for auto-trigger on deploy (private+python).

Fix sqlx LEFT JOIN LATERAL nullable column annotations in
get_ci_test_results and get_ci_test_results_batch queries — sqlx
cannot infer nullability from LATERAL subqueries, causing runtime
decode errors when no matching job exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix build/sqlx

* fix

* feat: CI test improvements and templates

- Fix windmill-dep-map/private feature propagation in worker, api-scripts,
  and api-flows Cargo.toml so CI test triggers actually fire in EE mode
- Clone ci_test_reference rows during workspace fork
- Add polling to CiTestResults component (refetch every 3s while running)
- Add running state and auto-refresh to ForkWorkspaceBanner CI summary
- Add yellow "CI test" badge on script list rows and detail page
- Fix Library badge border color (remove indigo border override)
- Add CI Test TypeScript and CI Test Python templates in ScriptBuilder
- Update sqlx offline cache
- Add debug tracing for CI test trigger in worker_lockfiles

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add missing children prop to WorkspaceDeployLayout

Fixes svelte-fast-check type error when passing named snippets as
children content inside the component tag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback

- Remove empty wrapper divs around CiTestResults, move mb-4 into component
- Add batch endpoint size cap (max 200 items)
- Add ON DELETE CASCADE to ci_test_reference workspace FK (new migration)
- Downgrade CI test trigger logs from info to debug
- Fix false-positive polling: only treat status='running' as running,
  not null status (CiTestResults, CompareWorkspaces, ForkWorkspaceBanner)
- Fix test numbering in integration tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to latest EE commit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to d9d68c2406df0b59f413ea0b2cb24780a9817d04

This commit updates the EE repository reference after PR #516 was merged in windmill-ee-private.

Previous ee-repo-ref: d7ccd9b86da99ec056a0e8708e3637d64290387a

New ee-repo-ref: d9d68c2406df0b59f413ea0b2cb24780a9817d04

Automated by sync-ee-ref workflow.

* fix: treat queued jobs (job_id set, null status) as running

Jobs that have been pushed but not yet picked up by a worker have a
job_id but null status. Treat these as 'running' to avoid showing
misleading 'pass' badges or '0 passing'. Tests that were never
triggered (no job_id, null status) remain neutral/hidden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-04-09 17:21:36 +00:00
Ruben Fiszel 9ff8a85af6 refactor: extract windmill-api into subcrates for parallel compilation (#7845) 
* refactor: extract windmill-api into 4 subcrates (api-auth, store, api-sse, api-jobs)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: eliminate refresh_token OnceLock bridge in windmill-store

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: eliminate FromRequestParts OnceLock bridge in windmill-api-auth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: wire subcrates into workspace and clean up unused re-exports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve cargo check --all-features errors in subcrate wiring

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* all

* chore: update ee-repo-ref for warning fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract windmill-trigger crate and expand windmill-api-jobs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-kafka crate from windmill-api

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-postgres crate from windmill-api

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-websocket and windmill-trigger-mqtt crates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-nats, sqs, gcp, and email crates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-http crate from windmill-api

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: move token creation and permission helpers to windmill-api-auth

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-native-triggers crate from windmill-api

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* all

* refactor: extract windmill-api-embeddings crate and fix CI warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve type mismatch in oauth2_oss and remaining warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use correct HTTP_CLIENT config in embeddings crate (30s timeout, cert override)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* fix: gate oauth_refresh_ee on oauth2 feature to fix warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-07 22:12:55 +00:00