Merge branch 'main' into workflow-execution-time-display

2026-08-20 16:02:19 +00:00 · 2026-06-09 08:14:18 +02:00
parent dbc80e671c 92c21bbe65
commit 370c7439fe
175 changed files with 10873 additions and 3464 deletions
@@ -106,3 +106,4 @@ $NAV --root backend callees "X"                           # what does X call?
 - Search for existing code to reuse before writing new code
 - Follow established patterns in the codebase
 - Keep changes focused — don't refactor beyond what's asked
+- **Never attribute work to a specific customer, account, or "requested by a customer" in repo-tracked content** (PR descriptions, commit messages, code comments, docs). Describe changes by their technical motivation instead.
@@ -1,5 +1,54 @@
 # Changelog

+## [1.720.0](https://github.com/windmill-labs/windmill/compare/v1.719.0...v1.720.0) (2026-06-08)
+
+
+### Features
+
+* allow private MCP server URLs ([#9470](https://github.com/windmill-labs/windmill/issues/9470)) ([3bc5800](https://github.com/windmill-labs/windmill/commit/3bc5800197db383ae6f708415701a4bdbe2e3345))
+* **api:** add endpoint to update token label ([#9474](https://github.com/windmill-labs/windmill/issues/9474)) ([e8e0701](https://github.com/windmill-labs/windmill/commit/e8e0701a360d0614c4c5a74f6410ba6ac0638caa))
+* **frontend:** use unified drill picker for AI chat @-mention dropdown ([#9159](https://github.com/windmill-labs/windmill/issues/9159)) ([64b089c](https://github.com/windmill-labs/windmill/commit/64b089cd23cca4601abb09f092a32becb80d9394))
+
+
+### Bug Fixes
+
+* center auth0/okta icons and respect currentColor ([#9457](https://github.com/windmill-labs/windmill/issues/9457)) ([5d0ef7d](https://github.com/windmill-labs/windmill/commit/5d0ef7dfd91b3021d125a1b34f81f0788f173786))
+* **forks:** keep trigger/schedule operational state owned by the parent - WIN-2019 ([#9476](https://github.com/windmill-labs/windmill/issues/9476)) ([192574a](https://github.com/windmill-labs/windmill/commit/192574ab8f98d9521a232fc8a4935d407b00cb3a))
+* **frontend:** respect forced column order for numeric column names ([#9463](https://github.com/windmill-labs/windmill/issues/9463)) ([44f5dd6](https://github.com/windmill-labs/windmill/commit/44f5dd6636d4b23aa55383b8b8abe4c2f73bc88d))
+* **frontend:** use ban icon for canceled jobs instead of hourglass ([#9478](https://github.com/windmill-labs/windmill/issues/9478)) ([fa86c62](https://github.com/windmill-labs/windmill/commit/fa86c62b6600e7d47dadf4706d7002706333d919))
+* gate native integration pickers behind non-operator check ([#9465](https://github.com/windmill-labs/windmill/issues/9465)) ([6156e23](https://github.com/windmill-labs/windmill/commit/6156e2372a785ccd0c6f29cb74e90bee69e76483))
+* **oauth:** persist refreshed token through configured secret backend ([#9471](https://github.com/windmill-labs/windmill/issues/9471)) ([76c0d97](https://github.com/windmill-labs/windmill/commit/76c0d970a18bf28ddc48dd746570e73486542606))
+* refresh session editor preview on breadcrumb target switch ([#9475](https://github.com/windmill-labs/windmill/issues/9475)) ([6d522b3](https://github.com/windmill-labs/windmill/commit/6d522b3989ace1f214bd049d910bc0d2a2a6893e))
+
+## [1.719.0](https://github.com/windmill-labs/windmill/compare/v1.718.0...v1.719.0) (2026-06-06)
+
+
+### Features
+
+* **otel:** connect jobs to the inbound distributed trace ([#9456](https://github.com/windmill-labs/windmill/issues/9456)) ([fad1a54](https://github.com/windmill-labs/windmill/commit/fad1a549d95c00d0746a48163c4f95fc69733e1a))
+
+
+### Bug Fixes
+
+* authenticate slack callback payload with per-workspace hmac ([#9461](https://github.com/windmill-labs/windmill/issues/9461)) ([fbdf81b](https://github.com/windmill-labs/windmill/commit/fbdf81ba5f77d282c025360ecee14138dd4cb4a2))
+* prevent token label collision bypassing job read access control ([#9462](https://github.com/windmill-labs/windmill/issues/9462)) ([e1e7af6](https://github.com/windmill-labs/windmill/commit/e1e7af6a25a44eb06b67332ce1efeae2a21e0c6d))
+* **python:** escape reserved-keyword step ids in wrapper codegen ([#9460](https://github.com/windmill-labs/windmill/issues/9460)) ([6a15a9b](https://github.com/windmill-labs/windmill/commit/6a15a9b152ad20be4b5c3de6000516da231e41e0)), closes [#8893](https://github.com/windmill-labs/windmill/issues/8893)
+
+## [1.718.0](https://github.com/windmill-labs/windmill/compare/v1.717.1...v1.718.0) (2026-06-05)
+
+
+### Features
+
+* **flows:** opt-in to include the stopping step's result in early-stop errors ([#9446](https://github.com/windmill-labs/windmill/issues/9446)) ([f2f0812](https://github.com/windmill-labs/windmill/commit/f2f0812a04c9256cfc8eba5e0dcf38d71d971410))
+* make C# dotnet target framework configurable via DOTNET_TARGET_FRAMEWORK ([#9454](https://github.com/windmill-labs/windmill/issues/9454)) ([9a609bf](https://github.com/windmill-labs/windmill/commit/9a609bf08ac1b6157dbdfb827fc01e771d71262e))
+* sandboxed daemonless container runtime via '# sandbox &lt;image&gt;' ([#9453](https://github.com/windmill-labs/windmill/issues/9453)) ([1727271](https://github.com/windmill-labs/windmill/commit/1727271e197b34026efeaf1b6561bb404a440baa))
+* **sandbox:** pull/extract images with crane instead of podman ([#9455](https://github.com/windmill-labs/windmill/issues/9455)) ([7590b28](https://github.com/windmill-labs/windmill/commit/7590b281085afd1fc2774e8fb37a4c0af3aedbad))
+
+
+### Bug Fixes
+
+* distinguish canceled jobs in runs ([#9452](https://github.com/windmill-labs/windmill/issues/9452)) ([9067787](https://github.com/windmill-labs/windmill/commit/90677872f6185eb0c81e0e84a426a54653818457))
+
 ## [1.717.1](https://github.com/windmill-labs/windmill/compare/v1.717.0...v1.717.1) (2026-06-04)


@@ -307,6 +307,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends libprotobuf32 l
    && apt-get clean && rm -rf /var/lib/apt/lists/*
 COPY --from=nsjail /nsjail/nsjail /bin/nsjail

+# crane: pulls + flattens images for the sandboxed container runtime (`# sandbox <image>`).
+# Single static binary — no daemon/store/root needed. See docs/docker-v2-runtime.md.
+ARG CRANE_VERSION=v0.20.6
+RUN arch="$(dpkg --print-architecture)"; \
+    case "$arch" in amd64) crane_arch=x86_64 ;; arm64) crane_arch=arm64 ;; *) echo >&2 "error: unsupported arch '$arch' for crane"; exit 1 ;; esac; \
+    wget -O /tmp/crane.tgz "https://github.com/google/go-containerregistry/releases/download/${CRANE_VERSION}/go-containerregistry_Linux_${crane_arch}.tar.gz" \
+    && tar -xzf /tmp/crane.tgz -C /usr/local/bin crane \
+    && rm /tmp/crane.tgz \
+    && chmod +x /usr/local/bin/crane
+
 WORKDIR ${APP}

 RUN ln -s ${APP}/windmill /usr/local/bin/windmill
@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT path FROM flow WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "path",
+        "type_info": "Varchar"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "TextArray"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "0a568f630e069118fe302099a709e89cc4a702158899f7fdc66d0922e8fb9b29"
+}
@@ -0,0 +1,20 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT has_changes FROM workspace_diff\n         WHERE path = 'f/shared/renamed_away' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "has_changes",
+        "type_info": "Bool"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      true
+    ]
+  },
+  "hash": "4c81384b579bad74b64c72ca053839f316fe5412f99f3d3bbbdc0c65f55ab794"
+}
@@ -0,0 +1,12 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n         VALUES ('wm-fork-test-workspace', 'f/shared/renamed_away', 67890, 'def main(): return 1', '', '', 'python3', 'test@windmill.dev', NOW(), true, false, false, false)",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": []
+  },
+  "hash": "a60306f2bae0702363787c4cf7c1267af8e42a3b6aa382f1dd483bec7219c67c"
+}
@@ -0,0 +1,24 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE token SET label = $1\n           WHERE email = $2 AND token_prefix = $3\n             AND (label IS NULL OR (\n                 label <> 'session'\n                 AND lower(label) NOT LIKE 'ephemeral%'\n                 AND label <> 'debugger-token'\n                 AND label NOT LIKE 'mcp-oauth-%'\n             ))\n           RETURNING token_prefix",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "token_prefix",
+        "type_info": "Varchar"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Varchar",
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "c0da3f1f2c55900dbdf92b16ebbfdb7b4cc11a648460f175e4f57d080a0005a5"
+}
@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT DISTINCT path FROM script WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "path",
+        "type_info": "Varchar"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "TextArray"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "c4966cf071a8504f578eed5518134db7b001d740d524075f19b91bae6fdb41b9"
+}
@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT permissioned_as_email FROM v2_job WHERE id = $1 AND workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "permissioned_as_email",
+        "type_info": "Varchar"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Uuid",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "c533691be8136c5ed6835c2fcdb016c257a5e4ee271d221fa961c24bd119d98e"
+}
@@ -0,0 +1,12 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "INSERT INTO workspace_diff\n         (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork)\n         VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/renamed_away', 'script', 1, 0, true, false, true)",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": []
+  },
+  "hash": "e9c2e8c50fc45576453885340800c3a44de6e32651a9a71c1aef49da2696a04f"
+}
@@ -1,6 +1,6 @@
 [package]
 name = "windmill"
-version = "1.717.1"
+version = "1.720.0"
 authors.workspace = true
 edition.workspace = true

@@ -87,7 +87,7 @@ members = [
 exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]

 [workspace.package]
-version = "1.717.1"
+version = "1.720.0"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]
 edition = "2021"

@@ -0,0 +1,172 @@
+# Threat Model: Windmill Backend
+
+## 1. System context
+
+Windmill is an open-source (AGPLv3) developer platform for internal tools,
+workflows, background jobs, API integrations, and UIs — a self-hostable
+alternative to Retool / Pipedream / Airplane. The backend is a Rust workspace
+(~60 crates: `windmill-api`, `windmill-worker`, `windmill-queue`,
+`windmill-common`, a family of `windmill-trigger-*` crates, `windmill-mcp`,
+`windmill-sandbox`, etc.) fronting a PostgreSQL database. A Svelte 5 frontend
+(not in scope here, but referenced where stored-XSS threats originate) is
+served by the same instance. The product ships in a Community Edition (CE,
+public Docker images) and an Enterprise Edition (EE, `*_ee.rs` files gated by
+`enterprise`/`private`/`license` cargo features).
+
+The defining characteristic for threat modeling is that **Windmill executes
+arbitrary user-supplied code** (Python, TypeScript via Bun/Deno, Go, Bash,
+SQL, GraphQL, PowerShell, Rust, …) on its workers, and **stores the
+credentials to every system its users connect to** (databases, cloud
+accounts, SaaS APIs, OAuth tokens). It is therefore simultaneously an
+arbitrary-code-execution engine and a credential vault — compromising one
+instance can pivot into an organization's entire connected estate. Crucially,
+the owner confirms `nsjail` is **off by default everywhere** (`ENABLE_NSJAIL`
+is opt-in) and network isolation (`clone_newnet`) is separately gated: the
+*only* job isolation present in a default install is PID-namespace `unshare`.
+Filesystem and outbound-network isolation are therefore absent unless an
+operator deliberately enables them, which makes "weak-by-default isolation" a
+more accurate frame than "sandbox escape" for typical deployments. Cross-tenant
+separation is enforced in software via workspace IDs, token scopes, folder
+ACLs, and Postgres row-level security; on the managed offering, sensitive
+customers can opt into dedicated DB / worker / namespace infrastructure, but
+the shared tier relies entirely on that software boundary. Administrators are 
+strongly encouraged to use nsjail sandboxing and are reminded that if they don't, 
+their security model is that they trust their developers that write code ran on windmill 
+to not do anything TOO malicious on the workers. When the default
+database secret backend is used, only per-workspace secret *variables* are
+encrypted at rest — instance-level `global_settings` (OAuth client secrets,
+SMTP, object-store keys, license) are stored plaintext, so a database read
+yields the instance-wide credential set. Internet-facing instances are
+typically exposed directly with no built-in rate limiting or WAF.
+
+It is deployed self-hosted (Docker Compose, Kubernetes/Helm, bare metal), on
+cloud providers, and as a Windmill-Labs-managed multi-tenant service. The API
+server is internet-facing in most deployments; workers pull jobs from the
+Postgres queue. The large public attack surface (a sprawling authenticated
+HTTP API, unauthenticated public-app and webhook/trigger endpoints, outbound
+HTTP from user code and proxies) combined with the high-value assets makes
+authorization-enforcement bugs, SSRF, SQL injection, and sandbox escape the
+dominant risk categories — a pattern strongly confirmed by the project's
+published advisory history (73 GHSA advisories, several rated 9.9 critical).
+
+## 2. Assets
+
+| asset | description | sensitivity |
+|---|---|---|
+| Workspace encryption keys | Per-workspace key (`workspace_key`) used to encrypt secret variables (MagicCrypt256); decrypts all secrets in the workspace | critical |
+| Secret variables | User secrets stored encrypted in `variable` (is_secret) | critical |
+| Resource credentials | DB passwords, cloud creds, API keys, connection strings in `resource` JSONB | critical |
+| OAuth / external-account tokens | Refresh/access tokens in `account`, MCP OAuth tables | critical |
+| User password hashes | Argon2 hashes in `password` table | critical |
+| API tokens & session cookies | Bearer tokens / cookies in `token`; superadmin & scoped tokens | critical |
+| Instance global settings | License key, JWT secret, SUPERADMIN_SECRET, SMTP, object-store + secret-backend (Vault/KMS/SM) creds in `global_settings` | critical |
+| Worker host & process integrity | The host that runs untrusted user code | critical |
+| Cross-tenant / cross-workspace isolation | The software boundary separating workspaces, folders, and tenants | critical |
+| Downstream connected systems | Windmill is a credential vault: stored creds reach external DBs, cloud accounts, SaaS | critical |
+| Script / flow / app source | Customer IP & business logic in `script`, `flow`, `app`, `raw_app` | high |
+| Job arguments, results & logs | `queue`/`completed_job` args+result, `job_logs`; routinely contain secrets | high |
+| Object store / S3 data | Files uploaded/produced by jobs | high |
+| Audit logs | `audit`/`audit_partitioned` action trail | high |
+| Service availability | API server + worker fleet uptime | high |
+| PII | User emails, group membership | medium |
+
+## 3. Entry points & trust boundaries
+
+| entry_point | description | trust_boundary | reachable_assets |
+|---|---|---|---|
+| EP1 Authenticated job-execution API | `jobs/run/preview`, `run/h/{hash}`, `run_flow/run_script` — runs user code on workers | authenticated user → arbitrary code on worker | Worker host, downstream systems, isolation, job args/results/logs |
+| EP2 Unauthenticated public endpoints | `apps_u/*`, `jobs_u/getupdate*`, `scripts_u`, `settings_u`, `resources_u` (`public_app_layer.rs`) | unauth HTTP → app logic & job data | Job results, scripts, secrets, PII |
+| EP3 HTTP-trigger & webhook ingestion | `/api/r/*`, GCP/Azure push, Slack callback, `capture_u/*` | untrusted webhook → job queue | Job execution integrity, worker host |
+| EP4 Message-queue / native triggers | kafka, postgres, mqtt, websocket, nats, sqs, email triggers | external broker/message → job queue | Job execution integrity, availability |
+| EP5 HTTP API authorization layer | Token/scope/RLS/folder-ACL enforcement across all workspaced routes (`windmill-api-auth`) | scoped token / low-priv user → other users' & workspaces' data | Scripts, job data, secrets, isolation |
+| EP6 AI proxy & MCP endpoints | `ai/proxy/*`, `mcp` — resolve `$var:`/resources, proxy to LLM APIs, `X-Resource-Path` | authenticated user → outbound HTTP + secret resolution | Secrets, resource creds, internal network, downstream |
+| EP7 Outbound HTTP from executors/resources | GraphQL/HTTP/Postgres executors, webhook delivery, `test_object_storage_config`, git clone, npm tarball fetch | user-controlled URL → server-side request | Cloud metadata, internal network, downstream creds |
+| EP8 SQL query builders & contextual-var substitution | App DB query builder (`whereClause`/`tags`), Postgres-trigger `where_clause`, `%%WM_*%%` interpolation, `WM_INTERNAL_DB` | user input → raw SQL | Database, connected DBs |
+| EP9 Worker sandbox | nsjail / unshare / dind / rootless podman isolating user code | user code → host & cross-tenant filesystem/network | Worker host, isolation, downstream |
+| EP10 Worker code generation / wrappers | Entrypoint override, env-var names, workspace env interpolated into generated wrapper code | user-controlled identifier → executable code | Worker host, isolation |
+| EP11 OAuth / OIDC / SAML / MCP-OAuth / logout | Login callbacks, MCP OAuth client registration, logout `rd` redirect | untrusted IdP / redirect input → session | Session tokens, accounts |
+| EP12 Stored-content rendering | App builder HTML component, markdown, S3 download response headers | stored user content → admin browser (same origin) | Admin session, account takeover |
+| EP13 Log/file reading & export endpoints | `service_logs`, `jobs_u/getupdate` log file read (symlinks), workspace/tarball export | authed/unauth request → arbitrary file or admin-only config | Arbitrary files, global settings |
+| EP14 Secret-value & resource-value caches | In-memory caches in `windmill-store` keyed (historically un-keyed) by path | cache lookup crossing identity/folder boundary | Secret variables, resource creds |
+| EP15 Deployment & runtime config | docker-compose defaults: dind, debugger (`REQUIRE_SIGNED_DEBUG_REQUESTS=false`), CORS `Any`, default admin/`changeme`, exposed Postgres, `SUPERADMIN_SECRET`, `ENABLE_NSJAIL=false`, privileged containers | operator/infra default → full instance | All assets |
+| EP16 Supply chain | Cached hub scripts, GitHub workflow actions, vendored deps, Docker base image | build/update-time input → host & build integrity | Worker host, build integrity |
+| EP17 Token lifecycle | Token create/rescope/refresh, script-issued JWTs | scoped caller → broader privilege | Tokens, accounts, isolation |
+
+## 4. Threats
+
+| id | threat | actor | surface | asset | impact | likelihood | status | controls | evidence |
+|---|---|---|---|---|---|---|---|---|---|
+| T1 | SQL injection in app/internal query builders and trigger clauses compromises the metadata DB and connected databases | remote_auth | EP8 | Database, downstream connected systems | critical | almost_certain | partially_mitigated | sqlx parameterized queries elsewhere; query-builder safety reviews | GHSA-225c-j3xq-g6x6, GHSA-78p7-jc72-gv66, GHSA-hvc7-f67h-jx3g, GHSA-wrrg-f89m-f84q, GHSA-79vf-3qwm-2w64, GHSA-55p6-fxj4-v983, GHSA-5g4v-49rj-r52r, GHSA-x6cq-7xr8-53x3, 2cf4bb180b |
+| T2 | Server-side request forgery via proxies/executors reaches cloud metadata, internal network, and downstream credentials | remote_auth | EP6, EP7 | Cloud metadata, internal network, downstream connected systems, resource creds | critical | almost_certain | partially_mitigated | SSRF URL validation + redirect-following disabled added piecemeal; MCP private URL access requires the instance-wide `ALLOW_PRIVATE_MCP_SERVER_URLS` opt-in; outbound network isolation (`clone_newnet`) is opt-in and off by default | GHSA-3ggp-h37f-5qfw, GHSA-98qq-g8rh-xhff, GHSA-hfw8-27mx-63jm, GHSA-3r59-qvvc-774j, GHSA-4pj9-w5jc-g8w7, GHSA-8hh3-jf25-78j5, GHSA-3pjm-4w7f-3r2w, GHSA-f44c-x9hq-h68r, GHSA-j4h4-f8fj-3m3c, 4b06881918, 96a8eb63d4, dbd3942ef3 |
+| T3 | Broken authorization / IDOR lets a scoped token or low-privilege member read scripts, job data, and secrets across folders and workspaces | remote_auth | EP5, EP2, EP1 | Scripts, job data, secrets, isolation | critical | almost_certain | partially_mitigated | RLS, token scopes, folder ACLs, view-token HMAC (added incrementally); on managed, sensitive tenants can opt into dedicated DB/worker/namespace, but the shared tier IS the software boundary | GHSA-qfg7-x243-5hg4, GHSA-8x8x-88qc-qp4r, GHSA-2ppx-66jv-wpw5, GHSA-x3x7-g97v-mp59, GHSA-j276-g4h8-g6h5, GHSA-8mv7-hmrg-96xv, GHSA-x2wf-f962-7frq, GHSA-qc7c-gcw6-h4xp, GHSA-vxc5-w28p-m9xw, GHSA-2g34-wfvr-5qqj, GHSA-w7p6-wpxm-pp66, 7edf3f0212, 89a7a37776, ab11c7747a, 664edcdfb7 |
+| T4 | Remote code execution by injecting attacker-controlled identifiers into generated worker wrappers | remote_auth | EP10 | Worker host, isolation, downstream | critical | likely | partially_mitigated | entrypoint/env-var-name validation added | GHSA-wxjq-w5pj-jqhx, GHSA-5f5q-2vg2-r2x4, GHSA-8q8j-mm3g-5c2q (CVE-2026-33881), bf93657fee, bd05bcadde, 22ec4da5f0 |
+| T5 | Worker compromise & cross-tenant access via weak-by-default isolation (nsjail off by default → user code runs with only PID-ns `unshare`); sandbox escape where nsjail/dind/podman is enabled | remote_auth | EP9, EP15 | Worker host, isolation, downstream | critical | likely | unmitigated | nsjail off by default everywhere (`DISABLE_NSJAIL=true`); shipped compose gives PID-ns `unshare` only (`FAVOR_UNSHARE_PID=true`), bare installs get no isolation. Where nsjail enabled: read-only remounts, jail-tmp refusal, podman socket gating | GHSA-6qr8-xhg4-453q, GHSA-3vpp-vf62-wqp6, f8467f38c8, df5aec0f5d, f1b6746e0e |
+| T6 | Disclosure of secrets, resource credentials, and workspace encryption keys across the authorization boundary (AI proxy, MCP, caches, export); database read additionally yields plaintext instance-level `global_settings` secrets | remote_auth | EP6, EP14, EP13 | Secret variables, encryption keys, resource creds, global settings | critical | likely | partially_mitigated | RLS on `$var:`, cache scoping by caller, admin checks on export; per-workspace secret *variables* encrypted at rest, but `global_settings` is plaintext under the default DB secret backend | GHSA-jwg4-v3cj-rvfm, GHSA-8m2p-2crh-9h3w, GHSA-6635-6fch-v8px, GHSA-437f-725p-7w84, GHSA-f27g-j463-q85w (CVE-2026-26964), GHSA-j679-v6vj-jfxc, GHSA-6vrr-fq33-qpfp, 0ba128afe7, 7836a4e733, ff8e39c69b |
+| T7 | Full instance compromise from insecure deployment defaults (dind control, default admin/`changeme`, exposed Postgres, publicly readable SUPERADMIN_SECRET) | remote_unauth | EP15 | All assets | critical | likely | partially_mitigated | first-time-setup warning on default admin; docs recommend hardening | GHSA-3vpp-vf62-wqp6, GHSA-24fr-44f8-fqwg (CVE-2026-29059), GHSA-6q36-5p3h-766j |
+| T8 | Unauthenticated RCE via the Debugger WebSocket in the default `windmill_extra` configuration | remote_unauth | EP15 | Worker host, all assets | critical | possible | unmitigated | `REQUIRE_SIGNED_DEBUG_REQUESTS` exists but defaults to false | GHSA-725h-99vx-9xr4 |
+| T9 | Supply-chain compromise via cached hub scripts, GitHub workflow command injection, or vulnerable base-image deps | supply_chain | EP16 | Worker host, build integrity | critical | possible | partially_mitigated | hub-script re-pin to patched versions; HUB_BASE_URL override | GHSA-w2m9-q5f7-3gpq, edf340c4d4, GHSA-8rq7-w7g6-8wvr, GHSA-vch9-39v5-4wg7 (CVE-2024-37371) |
+| T10 | Unauthenticated disclosure of job results, args, logs, and admin config via missing-authz public endpoints | remote_unauth | EP2, EP13 | Job results/args/logs, global settings, scripts | high | likely | partially_mitigated | anonymous-job checks, log-endpoint authz hardening | GHSA-qfg7-x243-5hg4, GHSA-v448-fmm4-52fp, 108a88a180, bb90f4ce83 |
+| T11 | Stored XSS leading to admin/account takeover via app HTML component, markdown, or S3 download content-type | remote_auth | EP12 | Admin session, accounts | high | likely | partially_mitigated | DOMPurify markdown sanitization, `X-Content-Type-Options: nosniff` + CSP sandbox on downloads | GHSA-9c5c-hh3c-r9mc, GHSA-qxj7-hpx3-r892, GHSA-cf2x-rg8c-v63v, bb78b1c06d, 625b67dff0 |
+| T12 | Webhook authentication bypass / signature replay forges trigger invocations and approvals | remote_unauth | EP3 | Job execution integrity, approvals | high | likely | partially_mitigated | HMAC verification on some triggers; signing-oracle fix | GHSA-jw8c-h45c-xpjw, GHSA-hh9x-rcf8-xjr2, GHSA-q9g3-q6fj-hc2x, GHSA-8jc4-wj2p-2vmp, ab2a15b2a8 |
+| T13 | Path traversal / arbitrary file read via log-reading and MCP path endpoints (incl. symlink following) | remote_auth | EP13 | Arbitrary files on server, global settings | high | likely | partially_mitigated | traversal checks + no-symlink-follow added | GHSA-4hrf-mgvv-xp9x, bb90f4ce83, df451aa64f, ad5ec293b5, 5f2d3e6812 |
+| T14 | Privilege escalation via token rescope/refresh, script-issued JWTs, or operator-permission gaps | remote_auth | EP17, EP5 | Tokens, isolation, accounts | high | likely | partially_mitigated | monotonic-privilege enforcement on token lifecycle; SECURITY DEFINER triggers | GHSA-p62p-67xp-v775, GHSA-vv9w-wx3c-q3x2, 2ddf93de96, 865ab70c89, 33fb08cf3d |
+| T15 | Credential leakage via worker `/proc` environment and unmasked secrets in job logs | remote_auth | EP9, EP1 | DB creds, secrets, downstream | high | likely | partially_mitigated | Aho-Corasick secret masking in logs | GHSA-pmp9-9924-f9cx, 0885d8c986 |
+| T16 | Denial of service via resource exhaustion: unbounded uploads, runaway jobs, queue flooding, or trigger-message storms | remote_auth | EP1, EP3, EP4 | Service availability, worker fleet | high | likely | risk_accepted | Per-job rlimits/timeouts exist; instance-wide DoS by an authenticated tenant is largely accepted on shared self-host (operator's job to add global quotas). Hard requirement only for managed multi-tenant | |
+| T17 | Account/credential theft via unauthenticated MCP-OAuth client registration and open redirect on logout | remote_unauth | EP11 | Accounts, session tokens | high | possible | partially_mitigated | redirect-URI handling / registration hardening | GHSA-q9xg-f2v2-695g, GHSA-53xj-pvqf-wpm9, GHSA-rr8j-ffc4-pf7h, GHSA-6c5w-777m-8rv5 |
+| T18 | Account takeover via missing rate limiting / brute force on auth endpoints | remote_unauth | EP11 | Accounts | medium | likely | unmitigated | none built-in; owner confirms instances are typically exposed directly with no app-level rate limiting or WAF | GHSA-cmv6-m7wc-c87p |
+| T19 | Enterprise license bypass and account impersonation | remote_auth | EP5 | Global settings, accounts | medium | possible | unmitigated | license validation gated by `license` feature | GHSA-48j5-p323-4mpx, GHSA-pv35-65rq-w29h, GHSA-2qx7-634r-qj6r |
+| T20 | Trigger spoofing: an actor with broker/queue access injects messages that execute jobs without app-level auth | adjacent_network | EP4 | Job execution integrity, downstream | medium | possible | risk_accepted | Owner confirms trust is delegated to broker ACLs by design; no app-level message authenticity check. Anyone able to publish to a subscribed topic/queue can cause job execution | |
+| T21 | Data-in-transit interception/tampering from TLS-disabled defaults (DB `sslmode=disable`, HTTP-only Caddy) | adjacent_network | EP15 | DB creds, secrets, session tokens | medium | possible | unmitigated | docs recommend TLS; not default | |
+| T22 | Repudiation / incident blind spots from gaps in audit coverage of sensitive actions | remote_auth | EP5 | Audit logs | medium | possible | partially_mitigated | `windmill-audit` records many actions | |
+
+## 5. Deprioritized
+
+| threat | reason |
+|---|---|
+| Physical access to the host / cold-boot key extraction | Out of scope; deployment-environment responsibility, not addressable in this codebase |
+| Memory-safety RCE in the Rust backend itself | Rust's safety model makes this rare; no evidence in history. Note: `unsafe` FFI (duckdb) is a narrow exception folded into supply-chain/T9 |
+| Client-side-only nuisance bugs (CSS, layout) with no security impact | No asset compromised |
+| Insider with legitimate superadmin / DB-root access | Trusted role; mitigations are operational (least privilege, audit), not technical controls in scope |
+| Spoofing of a fully-trusted upstream IdP that has itself been compromised | Out of model; Windmill trusts the configured IdP by design |
+| Instance-wide DoS by an authenticated tenant on shared self-host (T16) | Risk accepted (owner): per-job rlimits/timeouts are in place; global concurrency/queue quotas are the operator's responsibility on self-host. Remains a hard requirement for the managed multi-tenant fleet |
+| Job execution triggered by an actor with legitimate broker/queue publish access (T20) | Risk accepted (owner): trigger authenticity is delegated to broker ACLs by design; consuming from a configured source and acting on its messages is the intended behavior |
+
+## 6. Open questions
+
+Facts that drove the score changes above. Two were confirmed in code during
+the interview (`[Code-verified]`); the rest remain `[Owner-states]` pending a
+check.
+
+- [Code-verified] nsjail is off by default in every configuration: `DISABLE_NSJAIL` defaults to `true` (`windmill-worker/src/worker.rs:346`), and `is_sandboxing_enabled()` requires `DISABLE_NSJAIL=false` or the `job_isolation` global setting = `nsjail_sandboxing` (`worker.rs:890`). PID-ns `unshare` is also off at the code level (`is_unshare_enabled()`, `worker.rs:903`); the shipped `docker-compose.yml` sets `FAVOR_UNSHARE_PID=true` (line 91), so the official compose gives PID-ns unshare only, nsjail off — a bare install gets no isolation at all. No separate `clone_newnet` flag exists; network isolation is an nsjail feature, so outbound network from user code is unrestricted by default. Affects: T2 controls/likelihood, T5 status (unmitigated), T8.
+- [Code-verified] `global_settings` is plaintext at rest under the default DB backend: `set_value_in_global_settings` stores the raw JSON value with no encryption (`windmill-common/src/global_settings.rs:259`); the encrypting secret backend (`secret_backend/database.rs:66`) only encrypts per-workspace `variable` rows with `is_secret=true`. Instance-level SMTP/OAuth/AI/object-store secrets are therefore plaintext. Affects: T6 impact/controls, T7.
+- [Owner-states] Internet-facing instances are typically exposed directly with no built-in rate limiting / WAF. Affects: T16, T18 likelihood. Verify by: confirm absence of a rate-limit layer in `windmill-api/src/lib.rs` middleware stack.
+- [Owner-states] Managed offering provides an optional dedicated DB/worker/namespace tier for sensitive tenants; the shared tier relies solely on the software authz boundary. Affects: T3 controls. Verify by: deployment topology (not in this repo) — out-of-tree.
+- [Owner-states] Per-job rlimits/timeouts exist; instance-wide DoS by an authed tenant is risk-accepted on shared self-host. Affects: T16 status. Verify by: locate the rlimit/timeout enforcement in the worker execution path and confirm there is no global queue/concurrency cap.
+- [Owner-states] Message-queue trigger authenticity is delegated to broker ACLs only. Affects: T20 status. Verify by: review `windmill-trigger-{kafka,sqs,nats,mqtt,postgres}` consume paths for any payload authentication.
+
+## 7. Provenance
+
+- mode: bootstrap-then-interview
+- date: 2026-06-05
+- target: /home/rfiszel/windmill/backend @ 819ba5e150
+- inputs: git-log mined + GitHub security advisories (gh api, 73 advisories) + CHANGELOG; seed: THREAT_MODEL.md (bootstrap pass)
+- owner: Ruben Fiszel (Windmill core dev)
+
+## 8. Recommended mitigations
+
+| mitigation | threat_ids | closes_class | effort |
+|---|---|---|---|
+| Centralize a single audited query-builder that forbids string-interpolated SQL; ban `format!`-built queries via lint/CI | T1 | yes | M |
+| Route all outbound requests through one SSRF-guarded HTTP client (allowlist/denylist of private+metadata ranges, redirects disabled, re-validated per hop) | T2 | yes | M |
+| Enforce authorization centrally in middleware (scope + RLS + folder ACL) with deny-by-default and a per-route coverage test, instead of per-handler checks | T3, T10, T14, T22 | yes | L |
+| Treat all user-supplied identifiers as data: pass via argv/env/structured params, never splice into generated wrapper source; validate against strict allowlists at the boundary | T4 | yes | M |
+| Make `nsjail` + network-namespace isolation default-on / fail-closed (flip `ENABLE_NSJAIL` and `clone_newnet` defaults) and remove privileged/dind defaults from shipped compose; default-deny debugger | T2, T5, T7, T8 | partial | L |
+| Encrypt `global_settings` at rest under the workspace/instance key even on the default DB secret backend, so a DB read no longer yields plaintext instance-wide credentials | T6, T7 | partial | M |
+| Ship hardened defaults: random per-install secrets, no default admin password, Postgres not exposed, CORS locked to configured origin, TLS-on | T7, T18, T21 | partial | M |
+| Resolve secrets/resources only with the caller's identity and scope every cache entry by (caller, scope); apply uniformly to AI proxy, MCP, and exports | T6 | yes | M |
+| Output-encode/sanitize all stored content at render and force `nosniff` + restrictive CSP on every user-content response | T11 | yes | M |
+| Verify webhook authenticity uniformly (constant-time HMAC + timestamp/nonce anti-replay) in a shared trigger-auth helper | T12 | yes | S |
+| Canonicalize + confine all file-path inputs to a base dir and never follow symlinks in log/file readers | T13 | yes | S |
+| Mask secrets at the log sink and keep secrets out of worker process env (`/proc`) — pass via files/pipes scrubbed after use | T15 | partial | M |
+| Add global rate limiting and per-tenant resource/queue quotas at the edge | T16, T18 | partial | M |
+| Pin and integrity-verify hub scripts and CI actions; SBOM + automated base-image CVE scanning in release | T9 | partial | M |
@@ -1 +1 @@
-3742e0659c5e97aab03b9efeea14cd94a3ac658a
+481ea7f28dc5af6b72390c82f494f34cb9809546
@@ -154,29 +154,34 @@
  "zoho": {
    "auth_url": "https://accounts.zoho.com/oauth/v2/auth",
    "token_url": "https://accounts.zoho.com/oauth/v2/token",
-    "scopes": [
-      "ZohoAssist.sessionapi.ALL"
-    ],
+    "scopes": ["ZohoAssist.sessionapi.ALL"],
    "extra_params": {
      "access_type": "offline"
    }
  },
-  "snowflake_oauth": {},
+  "snowflake_oauth": {
+    "connect_config_template": {
+      "display_name": "Snowflake",
+      "label": "Snowflake Account Identifier",
+      "placeholder": "<orgname>-<account_name>",
+      "help_url": "https://docs.snowflake.com/en/user-guide/admin-account-identifier#using-an-account-name-as-an-identifier",
+      "auth_url": "https://{instance}.snowflakecomputing.com/oauth/authorize",
+      "token_url": "https://{instance}.snowflakecomputing.com/oauth/token-request",
+      "req_body_auth": false,
+      "extra_params_key": "account_identifier",
+      "resource_mapping": { "account_identifier": "{instance}" }
+    }
+  },
  "apify": {
    "auth_url": "https://console.apify.com/authorize/oauth",
    "token_url": "https://console-backend.apify.com/oauth/apps/token",
-    "scopes": [
-      "profile",
-      "full_api_access"
-    ],
+    "scopes": ["profile", "full_api_access"],
    "extra_params": {}
  },
  "docusign": {
    "auth_url": "https://account.docusign.com/oauth/auth",
    "token_url": "https://account.docusign.com/oauth/token",
-    "scopes": [
-      "signature"
-    ],
+    "scopes": ["signature"],
    "sandbox": {
      "auth_url": "https://account-d.docusign.com/oauth/auth",
      "token_url": "https://account-d.docusign.com/oauth/token"
@@ -185,14 +190,25 @@
  "salesforce": {
    "auth_url": "https://login.salesforce.com/services/oauth2/authorize",
    "token_url": "https://login.salesforce.com/services/oauth2/token",
-    "scopes": [
-      "api",
-      "refresh_token",
-      "offline_access"
-    ],
+    "scopes": ["api", "refresh_token", "offline_access"],
    "sandbox": {
      "auth_url": "https://test.salesforce.com/services/oauth2/authorize",
      "token_url": "https://test.salesforce.com/services/oauth2/token"
    }
+  },
+  "servicenow": {
+    "connect_config_template": {
+      "display_name": "ServiceNow",
+      "label": "ServiceNow Instance",
+      "placeholder": "<instance> (e.g. dev12345)",
+      "help_url": "https://www.servicenow.com/docs/bundle/zurich-platform-security/page/administer/security/concept/c_OAuthApplications.html",
+      "auth_url": "https://{instance}.service-now.com/oauth_auth.do",
+      "token_url": "https://{instance}.service-now.com/oauth_token.do",
+      "req_body_auth": true,
+      "strip_suffix": ".service-now.com",
+      "resource_mapping": {
+        "instance_url": "https://{instance}.service-now.com"
+      }
+    }
  }
 }
@@ -6183,7 +6183,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"

 [[package]]
 name = "windmill-common"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "aho-corasick",
 "anyhow",
@@ -6263,7 +6263,7 @@ dependencies = [

 [[package]]
 name = "windmill-macros"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "proc-macro2",
 "quote",
@@ -6275,7 +6275,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "convert_case",
 "serde",
@@ -6284,7 +6284,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-bash"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "lazy_static",
@@ -6296,7 +6296,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-csharp"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "serde_json",
@@ -6308,7 +6308,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-go"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "gosyn",
@@ -6320,7 +6320,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-graphql"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "lazy_static",
@@ -6332,7 +6332,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-java"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "serde_json",
@@ -6344,7 +6344,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-nu"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "nu-parser",
@@ -6355,7 +6355,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-php"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "itertools 0.14.0",
@@ -6366,7 +6366,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-py"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "itertools 0.14.0",
@@ -6378,7 +6378,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-py-asset"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "rustpython-ast",
@@ -6389,7 +6389,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-py-imports"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "async-recursion",
@@ -6411,7 +6411,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-r"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "serde_json",
@@ -6423,7 +6423,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-ruby"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "lazy_static",
@@ -6437,7 +6437,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-rust"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "convert_case",
@@ -6454,7 +6454,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-sql"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "lazy_static",
@@ -6467,7 +6467,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-sql-asset"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "serde",
@@ -6479,7 +6479,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-ts"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "lazy_static",
@@ -6497,7 +6497,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-ts-asset"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "serde-wasm-bindgen",
@@ -6513,7 +6513,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-wac"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "rustpython-ast",
@@ -6529,7 +6529,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-wasm"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "getrandom 0.2.17",
@@ -6561,7 +6561,7 @@ dependencies = [

 [[package]]
 name = "windmill-parser-yaml"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "serde",
@@ -6572,7 +6572,7 @@ dependencies = [

 [[package]]
 name = "windmill-types"
-version = "1.717.1"
+version = "1.720.0"
 dependencies = [
 "anyhow",
 "bitflags",
@@ -12,7 +12,7 @@ resolver = "2"
 members = ["."]

 [workspace.package]
-version = "1.717.1"
+version = "1.720.0"
 edition = "2021"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]

@@ -57,11 +57,14 @@ use windmill_common::{
        PREVIEW_TAGS_OVERRIDE_SETTING, REQUEST_SIZE_LIMIT_SETTING,
        REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RESTART_COORDINATION_SETTING,
        RETENTION_PERIOD_SECS_SETTING, RUBY_REPOS_SETTING, SAML_METADATA_SETTING,
-        SCIM_TOKEN_SETTING, SMTP_SETTING, STORE_AUDIT_LOGS_S3_SETTING, TEAMS_SETTING,
-        TIMEOUT_WAIT_RESULT_SETTING, UV_EXCLUDE_NEWER_SETTING, UV_INDEX_STRATEGY_SETTING,
-        UV_PYTHON_INSTALL_MIRROR_SETTING, WORKSPACE_FAIRNESS_DURATION_SECS_SETTING,
-        WORKSPACE_FAIRNESS_ENABLED_SETTING, WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING,
-        WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING, WORKSPACE_REGISTRIES_SETTING,
+        SANDBOX_IMAGE_CACHE_MAX_MB_SETTING, SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING,
+        SANDBOX_IMAGE_MAX_SIZE_MB_SETTING, SANDBOX_IMAGE_PULL_POLICY_SETTING,
+        SANDBOX_REGISTRY_AUTH_SETTING, SCIM_TOKEN_SETTING, SMTP_SETTING,
+        STORE_AUDIT_LOGS_S3_SETTING, TEAMS_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
+        UV_EXCLUDE_NEWER_SETTING, UV_INDEX_STRATEGY_SETTING, UV_PYTHON_INSTALL_MIRROR_SETTING,
+        WORKSPACE_FAIRNESS_DURATION_SECS_SETTING, WORKSPACE_FAIRNESS_ENABLED_SETTING,
+        WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING, WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING,
+        WORKSPACE_REGISTRIES_SETTING,
    },
    scripts::ScriptLang,
    stats_oss::schedule_stats,
@@ -134,8 +137,11 @@ use crate::monitor::{
    reload_job_default_timeout_setting, reload_job_isolation_setting, reload_jwt_secret_setting,
    reload_license_key, reload_npm_config_registry_setting, reload_nsjail_tmp_backing_setting,
    reload_nsjail_tmpfs_size_setting, reload_otel_tracing_proxy_setting,
-    reload_pip_index_url_setting, reload_retention_period_setting, reload_scim_token_setting,
-    reload_smtp_config, reload_store_audit_logs_s3_setting, reload_uv_exclude_newer_setting,
+    reload_pip_index_url_setting, reload_retention_period_setting,
+    reload_sandbox_image_cache_max_setting, reload_sandbox_image_default_registry_setting,
+    reload_sandbox_image_max_size_setting, reload_sandbox_image_pull_policy_setting,
+    reload_sandbox_registry_auth_setting, reload_scim_token_setting, reload_smtp_config,
+    reload_store_audit_logs_s3_setting, reload_uv_exclude_newer_setting,
    reload_uv_index_strategy_setting, reload_uv_python_install_mirror_setting,
    reload_worker_config, MonitorIteration,
 };
@@ -1827,6 +1833,19 @@ async fn process_notify_event(
                JOB_ISOLATION_SETTING => reload_job_isolation_setting(conn).await,
                NSJAIL_TMPFS_SIZE_MB_SETTING => reload_nsjail_tmpfs_size_setting(conn).await,
                NSJAIL_TMP_BACKING_SETTING => reload_nsjail_tmp_backing_setting(conn).await,
+                SANDBOX_IMAGE_MAX_SIZE_MB_SETTING => {
+                    reload_sandbox_image_max_size_setting(conn).await
+                }
+                SANDBOX_IMAGE_CACHE_MAX_MB_SETTING => {
+                    reload_sandbox_image_cache_max_setting(conn).await
+                }
+                SANDBOX_IMAGE_PULL_POLICY_SETTING => {
+                    reload_sandbox_image_pull_policy_setting(conn).await
+                }
+                SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING => {
+                    reload_sandbox_image_default_registry_setting(conn).await
+                }
+                SANDBOX_REGISTRY_AUTH_SETTING => reload_sandbox_registry_auth_setting(conn).await,
                #[cfg(feature = "parquet")]
                OBJECT_STORE_CONFIG_SETTING => {
                    if !disable_s3_store {
@@ -66,7 +66,9 @@ use windmill_common::{
        OTEL_SETTING, OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING,
        POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING, PREVIEW_TAGS_OVERRIDE_SETTING,
        REQUEST_SIZE_LIMIT_SETTING, REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING,
-        RETENTION_PERIOD_SECS_SETTING, SAML_METADATA_SETTING, SCIM_TOKEN_SETTING,
+        RETENTION_PERIOD_SECS_SETTING, SAML_METADATA_SETTING, SANDBOX_IMAGE_CACHE_MAX_MB_SETTING,
+        SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING, SANDBOX_IMAGE_MAX_SIZE_MB_SETTING,
+        SANDBOX_IMAGE_PULL_POLICY_SETTING, SANDBOX_REGISTRY_AUTH_SETTING, SCIM_TOKEN_SETTING,
        STORE_AUDIT_LOGS_S3_SETTING, TIMEOUT_WAIT_RESULT_SETTING, UV_EXCLUDE_NEWER_SETTING,
        UV_INDEX_STRATEGY_SETTING, UV_PYTHON_INSTALL_MIRROR_SETTING,
        WORKSPACE_FAIRNESS_DURATION_SECS_SETTING, WORKSPACE_FAIRNESS_ENABLED_SETTING,
@@ -112,8 +114,10 @@ use windmill_worker::{
    JOB_DEFAULT_TIMEOUT, JOB_ISOLATION, KEEP_JOB_DIR, MAVEN_REPOS, MAVEN_SETTINGS_XML,
    NO_DEFAULT_MAVEN, NPMRC, NPM_CONFIG_REGISTRY, NSJAIL_AVAILABLE, NSJAIL_TMPFS_SIZE_MB,
    NSJAIL_TMP_BACKING, NUGET_CONFIG, OTEL_TRACING_PROXY_SETTINGS, PIP_EXTRA_INDEX_URL,
-    PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, UNSHARE_PATH, UV_EXCLUDE_NEWER,
-    UV_INDEX_STRATEGY, UV_PYTHON_INSTALL_MIRROR, WORKSPACE_REGISTRIES,
+    PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, SANDBOX_IMAGE_CACHE_MAX_MB,
+    SANDBOX_IMAGE_DEFAULT_REGISTRY, SANDBOX_IMAGE_MAX_SIZE_MB, SANDBOX_IMAGE_PULL_POLICY,
+    SANDBOX_REGISTRY_AUTH, UNSHARE_PATH, UV_EXCLUDE_NEWER, UV_INDEX_STRATEGY,
+    UV_PYTHON_INSTALL_MIRROR, WORKSPACE_REGISTRIES,
 };

 #[cfg(feature = "parquet")]
@@ -407,6 +411,11 @@ pub async fn initial_load(
        reload_job_isolation_setting(&conn).await;
        reload_nsjail_tmpfs_size_setting(&conn).await;
        reload_nsjail_tmp_backing_setting(&conn).await;
+        reload_sandbox_image_max_size_setting(&conn).await;
+        reload_sandbox_image_cache_max_setting(&conn).await;
+        reload_sandbox_image_pull_policy_setting(&conn).await;
+        reload_sandbox_image_default_registry_setting(&conn).await;
+        reload_sandbox_registry_auth_setting(&conn).await;
        reload_extra_pip_index_url_setting(&conn).await;
        reload_pip_index_url_setting(&conn).await;
        reload_uv_index_strategy_setting(&conn).await;
@@ -1095,24 +1104,8 @@ struct TokenRow {
    workspace_id: Option<String>,
 }

-/// When updating this filter, also update:
-/// - `register_token_expiry_notification` in windmill-api-auth/src/lib.rs
-/// - `isUserToken` in frontend/src/lib/components/settings/TokensTable.svelte
-fn is_user_token(label: Option<&str>) -> bool {
-    match label {
-        None => true,
-        Some(l) => {
-            l != "session"
-                && !l.starts_with("ephemeral")
-                && !l.starts_with("Ephemeral")
-                && l != "debugger-token"
-                && !l.starts_with("mcp-oauth-")
-        }
-    }
-}
-
 async fn report_token_expiration(db: &DB, token: &TokenRow, expired: bool) {
-    if !is_user_token(token.label.as_deref()) {
+    if !windmill_common::auth::is_user_token(token.label.as_deref()) {
        return;
    }
    let prefix = token.token_prefix.as_deref().unwrap_or("??????????");
@@ -2045,6 +2038,66 @@ pub async fn reload_nsjail_tmp_backing_setting(conn: &Connection) {
    .await;
 }

+pub async fn reload_sandbox_image_max_size_setting(conn: &Connection) {
+    reload_option_setting_with_tracing(
+        conn,
+        SANDBOX_IMAGE_MAX_SIZE_MB_SETTING,
+        "SANDBOX_IMAGE_MAX_SIZE_MB",
+        SANDBOX_IMAGE_MAX_SIZE_MB.clone(),
+    )
+    .await;
+}
+
+pub async fn reload_sandbox_image_cache_max_setting(conn: &Connection) {
+    reload_option_setting_with_tracing(
+        conn,
+        SANDBOX_IMAGE_CACHE_MAX_MB_SETTING,
+        "SANDBOX_IMAGE_CACHE_MAX_MB",
+        SANDBOX_IMAGE_CACHE_MAX_MB.clone(),
+    )
+    .await;
+}
+
+pub async fn reload_sandbox_image_pull_policy_setting(conn: &Connection) {
+    reload_option_setting_with_tracing(
+        conn,
+        SANDBOX_IMAGE_PULL_POLICY_SETTING,
+        "SANDBOX_IMAGE_PULL_POLICY",
+        SANDBOX_IMAGE_PULL_POLICY.clone(),
+    )
+    .await;
+}
+
+pub async fn reload_sandbox_image_default_registry_setting(conn: &Connection) {
+    reload_option_setting_with_tracing(
+        conn,
+        SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING,
+        "SANDBOX_IMAGE_DEFAULT_REGISTRY",
+        SANDBOX_IMAGE_DEFAULT_REGISTRY.clone(),
+    )
+    .await;
+}
+
+pub async fn reload_sandbox_registry_auth_setting(conn: &Connection) {
+    // Secret-aware: the value is a raw docker/podman auth.json with credentials, so
+    // it must never be logged. Load directly (the generic reload_option_setting path
+    // logs the value via load_option_setting_value) and only log a redacted message.
+    let q =
+        match load_value_from_global_settings_with_conn(conn, SANDBOX_REGISTRY_AUTH_SETTING, true)
+            .await
+        {
+            Ok(q) => q,
+            Err(e) => {
+                tracing::error!("Error reloading setting SANDBOX_REGISTRY_AUTH: {e:?}");
+                return;
+            }
+        };
+    let value = q.and_then(|q| serde_json::from_value::<String>(q).ok());
+    let configured = value.as_ref().is_some_and(|v| !v.trim().is_empty());
+    *SANDBOX_REGISTRY_AUTH.write().await = value;
+    tracing::info!("Loaded setting SANDBOX_REGISTRY_AUTH (redacted), configured={configured}");
+}
+
 pub async fn reload_job_isolation_setting(conn: &Connection) {
    let value =
        match load_value_from_global_settings_with_conn(conn, JOB_ISOLATION_SETTING, true).await {
@@ -2916,6 +2916,7 @@ export function main() {
            expr: "flow_env.STOP === true".to_string(),
            skip_if_stopped: true,
            error_message: None,
+            error_include_result: false,
        });
        m
    };
@@ -2966,6 +2967,92 @@ export function main() {
    Ok(())
 }

+// stop_after_if with `error_message` + `error_include_result` should fail the
+// flow but preserve the stopping step's own result inside the raised error
+// object, i.e. `{ "error": { .., "result": <step result> } }`. With the flag off
+// (the default) the error object carries no `result`. Regression for the
+// early-stop branch in `update_flow_status_after_job_completion_internal`.
+#[cfg(feature = "deno_core")]
+#[sqlx::test(fixtures("base"))]
+async fn test_stop_after_if_error_include_result(db: Pool<Postgres>) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+
+    let make_flow = |include_result: bool| {
+        let mut m = flow_module(
+            "step",
+            FlowModuleValue::RawScript {
+                input_transforms: Default::default(),
+                language: ScriptLang::Deno,
+                content: r#"
+export function main() {
+    return { userErrors: ["email taken"], ok: false };
+}
+"#
+                .to_string(),
+                path: None,
+                lock: None,
+                tag: None,
+                concurrency_settings: Default::default(),
+                is_trigger: None,
+                assets: None,
+            },
+        );
+        m.stop_after_if = Some(windmill_common::flows::StopAfterIf {
+            expr: "true".to_string(),
+            skip_if_stopped: false,
+            error_message: Some("API returned userErrors".to_string()),
+            error_include_result: include_result,
+        });
+        FlowValue { modules: vec![m], same_worker: false, ..Default::default() }
+    };
+
+    // include_result = true: result preserves both the error and the step output
+    let job = RunJob::from(JobPayload::RawFlow {
+        value: make_flow(true),
+        path: None,
+        restarted_from: None,
+    })
+    .run_until_complete(&db, false, server.addr.port())
+    .await;
+    assert!(
+        !job.success,
+        "flow with raised early-stop error should fail"
+    );
+    let result = job.json_result().unwrap();
+    assert_eq!(
+        result["error"]["name"], "EarlyStopError",
+        "expected EarlyStopError; got {result:?}"
+    );
+    assert_eq!(result["error"]["message"], "API returned userErrors");
+    assert_eq!(
+        result["error"]["result"],
+        json!({ "userErrors": ["email taken"], "ok": false }),
+        "step result should be preserved under `error.result`; got {result:?}"
+    );
+
+    // include_result = false (default behavior): result is the bare error object
+    let job = RunJob::from(JobPayload::RawFlow {
+        value: make_flow(false),
+        path: None,
+        restarted_from: None,
+    })
+    .run_until_complete(&db, false, server.addr.port())
+    .await;
+    assert!(
+        !job.success,
+        "flow with raised early-stop error should fail"
+    );
+    let result = job.json_result().unwrap();
+    assert_eq!(result["error"]["name"], "EarlyStopError");
+    assert!(
+        result["error"].get("result").is_none(),
+        "without the flag the error must not embed the step result; got {result:?}"
+    );
+
+    Ok(())
+}
+
 // retry_if predicate sees flow_env. Regression for the two evaluate_retry
 // call sites in `update_flow_status_after_job_completion_internal` (lines
 // 1194 and 1576) which used to pass `None` for flow_env.
@@ -3093,6 +3180,7 @@ export function main(i: number) {
            expr: "flow_env.STOP === true".to_string(),
            skip_if_stopped: true,
            error_message: None,
+            error_include_result: false,
        });
        m
    };
@@ -3143,3 +3231,84 @@ export function main() {

    Ok(())
 }
+
+// stop_after_all_iters_if with `error_message` + `error_include_result` fails the
+// flow and embeds the loop's aggregated iteration results under `error.result`.
+// Covers the loop/branch-all path where `nresult` is already populated with the
+// aggregated results (distinct from the per-step fallback to `result`).
+#[cfg(feature = "deno_core")]
+#[sqlx::test(fixtures("base"))]
+async fn test_stop_after_all_iters_if_error_includes_result(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+
+    let inner = flow_module(
+        "iter_step",
+        FlowModuleValue::RawScript {
+            input_transforms: [js_input("i", "flow_input.iter.value")].into(),
+            language: ScriptLang::Deno,
+            content: r#"
+export function main(i: number) {
+    return { iter: i };
+}
+"#
+            .to_string(),
+            path: None,
+            lock: None,
+            tag: None,
+            concurrency_settings: Default::default(),
+            is_trigger: None,
+            assets: None,
+        },
+    );
+
+    let loop_module = {
+        let mut m = flow_module(
+            "loop",
+            FlowModuleValue::ForloopFlow {
+                iterator: InputTransform::Javascript { expr: "[1, 2, 3]".to_string() },
+                modules: vec![inner],
+                modules_node: None,
+                skip_failures: false,
+                parallel: false,
+                parallelism: None,
+                squash: None,
+            },
+        );
+        m.stop_after_all_iters_if = Some(windmill_common::flows::StopAfterIf {
+            expr: "true".to_string(),
+            skip_if_stopped: false,
+            error_message: Some("loop failed".to_string()),
+            error_include_result: true,
+        });
+        m
+    };
+
+    let flow = FlowValue { modules: vec![loop_module], same_worker: false, ..Default::default() };
+
+    let job = RunJob::from(JobPayload::RawFlow { value: flow, path: None, restarted_from: None })
+        .run_until_complete(&db, false, server.addr.port())
+        .await;
+
+    assert!(
+        !job.success,
+        "loop with a raised early-stop error should fail"
+    );
+    let result = job.json_result().unwrap();
+    assert_eq!(result["error"]["name"], "EarlyStopError", "got {result:?}");
+    assert_eq!(result["error"]["message"], "loop failed");
+    // error.result holds the aggregated iteration results (one per iteration)
+    let iters = result["error"]["result"].as_array().unwrap_or_else(|| {
+        panic!("error.result should be an array of iteration results; got {result:?}")
+    });
+    let iter_values: Vec<_> = iters.iter().map(|r| r["iter"].clone()).collect();
+    assert_eq!(
+        iter_values,
+        vec![json!(1), json!(2), json!(3)],
+        "error.result should contain each iteration's output; got {result:?}"
+    );
+
+    Ok(())
+}
@@ -507,3 +507,123 @@ async fn test_root_job_span_attributes_values() {
    assert_eq!(get_attr("workspace_id"), "test-workspace");
    assert_eq!(get_attr("script_path"), "f/test/script");
 }
+
+// ═══════════════════════════════════════════════════════════════════════
+// INBOUND TRACE CONTEXT (W3C traceparent → span link)
+// ═══════════════════════════════════════════════════════════════════════
+
+const SAMPLE_TRACEPARENT: &str = "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01";
+
+fn sample_trace_id() -> opentelemetry::trace::TraceId {
+    opentelemetry::trace::TraceId::from_hex("0af7651916cd43dd8448eb211c80319c").unwrap()
+}
+
+fn sample_span_id() -> opentelemetry::trace::SpanId {
+    opentelemetry::trace::SpanId::from_hex("b7ad6b7169203331").unwrap()
+}
+
+#[test]
+fn test_span_cx_from_traceparent_valid() {
+    let cx = span_cx_from_traceparent(SAMPLE_TRACEPARENT).expect("valid traceparent");
+    assert_eq!(cx.trace_id(), sample_trace_id());
+    assert_eq!(cx.span_id(), sample_span_id());
+    assert!(cx.is_remote());
+    assert!(cx.is_sampled());
+}
+
+#[test]
+fn test_span_cx_from_traceparent_unsampled_flag() {
+    let cx = span_cx_from_traceparent("00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-00")
+        .expect("valid traceparent");
+    assert!(!cx.is_sampled());
+}
+
+#[test]
+fn test_span_cx_from_traceparent_malformed() {
+    for bad in [
+        "",
+        "garbage",
+        "00-tooshort-b7ad6b7169203331-01",
+        // missing flags field
+        "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331",
+        // trailing extra field
+        "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01-extra",
+        // all-zero trace id / span id are invalid per the spec
+        "00-00000000000000000000000000000000-b7ad6b7169203331-01",
+        "00-0af7651916cd43dd8448eb211c80319c-0000000000000000-01",
+        // non-hex
+        "00-zzf7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01",
+    ] {
+        assert!(
+            span_cx_from_traceparent(bad).is_none(),
+            "expected None for {bad:?}"
+        );
+    }
+}
+
+fn job_with_traceparent(tp: Option<&str>) -> windmill_queue::MiniPulledJob {
+    let mut job = make_test_job(uuid::Uuid::new_v4(), None);
+    if let Some(tp) = tp {
+        let mut args = std::collections::HashMap::new();
+        args.insert(
+            windmill_common::jobs::WM_TRACEPARENT.to_string(),
+            windmill_common::worker::to_raw_value(&tp),
+        );
+        job.args = Some(sqlx::types::Json(args));
+    }
+    job
+}
+
+#[test]
+fn test_inbound_span_cx_from_job_present() {
+    let job = job_with_traceparent(Some(SAMPLE_TRACEPARENT));
+    let cx = windmill_worker::otel_ee::inbound_span_cx_from_job(&job).expect("link expected");
+    assert_eq!(cx.trace_id(), sample_trace_id());
+    assert_eq!(cx.span_id(), sample_span_id());
+}
+
+#[test]
+fn test_inbound_span_cx_from_job_absent_or_malformed() {
+    // No reserved key (e.g. a flow step or internally-created job) → no link.
+    assert!(
+        windmill_worker::otel_ee::inbound_span_cx_from_job(&job_with_traceparent(None)).is_none()
+    );
+    // Malformed header is ignored rather than producing a bogus link.
+    assert!(
+        windmill_worker::otel_ee::inbound_span_cx_from_job(&job_with_traceparent(Some("garbage")))
+            .is_none()
+    );
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_root_job_span_relocated_to_inbound_trace() {
+    let state = ensure_setup().await;
+    state.span_exporter.reset();
+
+    let job = job_with_traceparent(Some(SAMPLE_TRACEPARENT));
+    let job_id = job.id;
+    windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
+
+    let spans = state.span_exporter.get_finished_spans().unwrap();
+    let span = spans
+        .iter()
+        .find(|s| s.name == "full_job")
+        .expect("full_job span not found");
+
+    // Relocated into the inbound trace, keeping the job-UUID-derived span id and
+    // parented on the inbound caller span.
+    assert_eq!(span.span_context.trace_id(), sample_trace_id());
+    let expected_span_id =
+        opentelemetry::trace::SpanId::from_bytes(job_id.as_u64_pair().1.to_be_bytes());
+    assert_eq!(span.span_context.span_id(), expected_span_id);
+    assert_eq!(span.parent_span_id, sample_span_id());
+
+    // Linked back to the UUID-derived context so trace-by-job-id still resolves.
+    assert_eq!(span.links.links.len(), 1);
+    let expected_uuid_trace =
+        opentelemetry::trace::TraceId::from_bytes(job_id.as_u128().to_be_bytes());
+    assert_eq!(
+        span.links.links[0].span_context.trace_id(),
+        expected_uuid_trace
+    );
+}
@@ -871,9 +871,6 @@ pub async fn create_token_internal(
 /// Insert a pending expiry notification row for user tokens that have an expiration.
 /// Stores the token_hash so the join in check_expiring_tokens works even when
 /// the plaintext token column is NULL (after hash migration).
-/// When updating this filter, also update:
-/// - `is_user_token` in src/monitor.rs
-/// - `isUserToken` in frontend/src/lib/components/settings/TokensTable.svelte
 pub async fn register_token_expiry_notification(
    tx: &mut sqlx::PgConnection,
    token_hash: &str,
@@ -881,14 +878,8 @@ pub async fn register_token_expiry_notification(
    expiration: Option<chrono::DateTime<chrono::Utc>>,
 ) {
    let Some(expiration) = expiration else { return };
-    if label == Some("session")
-        || label.is_some_and(|l| {
-            l.starts_with("ephemeral")
-                || l.starts_with("Ephemeral")
-                || l == "debugger-token"
-                || l.starts_with("mcp-oauth-")
-        })
-    {
+    // System tokens don't get expiry notifications.
+    if !windmill_common::auth::is_user_token(label) {
        return;
    }
    if let Err(e) = sqlx::query!(
@@ -35,6 +35,9 @@ anyhow.workspace = true
 uuid.workspace = true
 futures.workspace = true
 rand.workspace = true
+hmac.workspace = true
+sha2.workspace = true
+hex.workspace = true
 rumqttc.workspace = true
 rdkafka.workspace = true
 async-nats.workspace = true
@@ -17,8 +17,8 @@ use windmill_native_triggers::{
    decrypt_oauth_data, delete_native_trigger, delete_workspace_integration,
    get_workspace_integration,
    google::{parse_stop_channel_params, should_renew_channel},
-    store_native_trigger, store_workspace_integration, NativeTriggerConfig, OAuthConfig,
-    ServiceName,
+    require_native_integration_use, store_native_trigger, store_workspace_integration,
+    NativeTriggerConfig, OAuthConfig, ServiceName,
 };

 // ============================================================================
@@ -329,6 +329,26 @@ async fn test_token_update_persists(db: Pool<Postgres>) -> anyhow::Result<()> {
 // 3. Channel Expiration Renewal — should_renew_channel
 // ============================================================================

+#[test]
+fn test_require_native_integration_use_blocks_operators() {
+    // Regression: the integration *use* routes (calendar/drive/repo/event pickers)
+    // must reject read-only operators, who cannot create native triggers and so
+    // must not be able to drive the admin-configured integration's upstream API.
+    let mut operator = test_authed();
+    operator.is_admin = false;
+    operator.is_operator = true;
+    assert!(require_native_integration_use(&operator).is_err());
+
+    // A regular non-admin author (the population that configures triggers) is allowed.
+    let mut author = test_authed();
+    author.is_admin = false;
+    author.is_operator = false;
+    assert!(require_native_integration_use(&author).is_ok());
+
+    // Admins are allowed.
+    assert!(require_native_integration_use(&test_authed()).is_ok());
+}
+
 #[test]
 fn test_should_renew_drive_channel_expired() {
    let config = json!({
@@ -0,0 +1,230 @@
+//! Regression tests for GHSA-vm75-gmpw-rvp9: the unauthenticated `/api/slack` callback must
+//! not be drivable into decrypting arbitrary workspace variables.
+//!
+//! The OpenModal branch reaches `get_slack_token` (a privileged, RLS-bypassing variable
+//! decryption). It is now gated by a per-workspace HMAC over (w_id, job_id, path) — the same
+//! workspace key used to sign resume URLs. Without a valid signature the request is rejected
+//! with 401 before any decryption, even when `SLACK_SIGNING_SECRET` is unset (the default).
+
+use hmac::{Hmac, Mac};
+use serde_json::json;
+use sha2::Sha256;
+use sqlx::{Pool, Postgres};
+use uuid::Uuid;
+use windmill_test_utils::*;
+
+fn client() -> reqwest::Client {
+    reqwest::Client::new()
+}
+
+/// Re-implementation of the server's `sign_slack_payload` for the positive-control test.
+/// The fixture sets `workspace_key.key = 'test-key'` for `test-workspace`.
+fn sign(w_id: &str, parts: &[&[u8]]) -> String {
+    let mut mac = Hmac::<Sha256>::new_from_slice(b"test-key").unwrap();
+    mac.update(b"slack_payload_v1\0"); // SLACK_PAYLOAD_HMAC_DOMAIN
+    mac.update(w_id.as_bytes());
+    for p in parts {
+        mac.update(b"\0");
+        mac.update(p);
+    }
+    hex::encode(mac.finalize().into_bytes())
+}
+
+/// POST an `open_modal` block action to the unauthenticated `/api/slack` callback.
+async fn post_open_modal(port: u16, value: serde_json::Value) -> reqwest::Response {
+    let payload = json!({
+        "type": "block_actions",
+        "trigger_id": "trigger-123",
+        "container": { "message_ts": "0", "channel_id": "C1" },
+        "actions": [ { "action_id": "open_modal", "value": value.to_string() } ],
+    });
+    client()
+        .post(format!("http://localhost:{port}/api/slack"))
+        .form(&[("payload", payload.to_string())])
+        .send()
+        .await
+        .unwrap()
+}
+
+/// POST a `view_submission` to the unauthenticated `/api/slack` callback with the given
+/// private_metadata.
+async fn post_view_submission(port: u16, private_metadata: serde_json::Value) -> reqwest::Response {
+    let payload = json!({
+        "type": "view_submission",
+        "view": {
+            "state": { "values": {} },
+            "private_metadata": private_metadata.to_string(),
+        },
+    });
+    client()
+        .post(format!("http://localhost:{port}/api/slack"))
+        .form(&[("payload", payload.to_string())])
+        .send()
+        .await
+        .unwrap()
+}
+
+/// A submission with an unsigned (or tampered) `private_metadata` must be rejected with 401
+/// BEFORE the resume/cancel action runs — the signature gate is checked first. The resume_url
+/// here is well-formed (so it parses) but never acted upon.
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_view_submission_without_signature_is_rejected(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let job_id = Uuid::new_v4();
+    let resume_url = format!("/api/w/test-workspace/jobs_u/resume/{job_id}/1/deadbeef");
+
+    let resp = post_view_submission(
+        port,
+        json!({
+            "resume_url": resume_url,
+            "resource_path": "u/admin/secret",
+            "container": { "message_ts": "0", "channel_id": "C1" },
+            "hide_cancel": false,
+        }),
+    )
+    .await;
+    assert_eq!(
+        resp.status(),
+        401,
+        "unsigned submission must be rejected before the resume action"
+    );
+
+    Ok(())
+}
+
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_open_modal_without_signature_is_rejected(db: Pool<Postgres>) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let job_id = Uuid::new_v4();
+
+    // No signature → must be rejected with 401 before any variable lookup. Before the fix
+    // this reached `get_slack_token` and forced decryption of `u/admin/secret`.
+    let resp = post_open_modal(
+        port,
+        json!({
+            "w_id": "test-workspace",
+            "job_id": job_id.to_string(),
+            "path": "u/admin/secret",
+            "flow_step_id": "a",
+        }),
+    )
+    .await;
+    assert_eq!(
+        resp.status(),
+        401,
+        "unsigned OpenModal callback must be rejected"
+    );
+
+    Ok(())
+}
+
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_open_modal_with_wrong_signature_is_rejected(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let job_id = Uuid::new_v4();
+
+    let resp = post_open_modal(
+        port,
+        json!({
+            "w_id": "test-workspace",
+            "job_id": job_id.to_string(),
+            "path": "u/admin/secret",
+            "flow_step_id": "a",
+            "signature": "deadbeef",
+        }),
+    )
+    .await;
+    assert_eq!(
+        resp.status(),
+        401,
+        "OpenModal callback with an invalid signature must be rejected"
+    );
+
+    Ok(())
+}
+
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_open_modal_with_tampered_path_is_rejected(db: Pool<Postgres>) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let job_id = Uuid::new_v4();
+
+    // A signature legitimately minted for one path cannot be reused to decrypt another: the
+    // path is bound into the HMAC.
+    let signature = sign(
+        "test-workspace",
+        &[job_id.to_string().as_bytes(), b"u/admin/legit_resource"],
+    );
+    let resp = post_open_modal(
+        port,
+        json!({
+            "w_id": "test-workspace",
+            "job_id": job_id.to_string(),
+            "path": "u/admin/some_other_secret",
+            "flow_step_id": "a",
+            "signature": signature,
+        }),
+    )
+    .await;
+    assert_eq!(
+        resp.status(),
+        401,
+        "a signature bound to a different path must not authorize decryption"
+    );
+
+    Ok(())
+}
+
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_open_modal_with_valid_signature_passes_the_gate(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let job_id = Uuid::new_v4();
+    let path = "u/admin/nonexistent_resource";
+
+    // A correctly signed payload passes the authorization gate and proceeds to resolve the
+    // slack resource. The resource does not exist, so the handler returns a generic 400
+    // ("Invalid Slack callback request") rather than 401 — proving the gate accepted the
+    // signature (so the fix does not simply reject everything) without echoing the path.
+    let signature = sign(
+        "test-workspace",
+        &[job_id.to_string().as_bytes(), path.as_bytes()],
+    );
+    let resp = post_open_modal(
+        port,
+        json!({
+            "w_id": "test-workspace",
+            "job_id": job_id.to_string(),
+            "path": path,
+            "flow_step_id": "a",
+            "signature": signature,
+        }),
+    )
+    .await;
+    let status = resp.status();
+    let body = resp.text().await?;
+    assert_eq!(
+        status, 400,
+        "validly signed callback should pass the gate and 400 on the missing resource, got {status}: {body}"
+    );
+    assert!(
+        !body.contains("nonexistent_resource"),
+        "error must not echo the probed path: {body}"
+    );
+
+    Ok(())
+}
@@ -0,0 +1,192 @@
+//! Regression tests for GHSA-8x8x-88qc-qp4r: token label collision bypassing job read
+//! access control (IDOR).
+//!
+//! `username_override` is derived from a fully user-controlled token label, so a bare
+//! `username_override == created_by` match in `require_job_read_access` is forgeable. The fix
+//! binds that fast path to a non-forgeable attribute — the job's `permissioned_as_email` (the
+//! token owner's email) must equal the caller's email. This:
+//! - denies a colliding-label token created by a different principal, while
+//! - still allowing a principal to re-read its own labeled-token jobs (incl. when RLS would
+//!   otherwise hide them), and
+//! - leaving user-facing webhook/http/email trigger token creation untouched (those labels
+//!   are created through the public token API by design).
+
+use serde_json::json;
+use sqlx::{Pool, Postgres};
+use uuid::Uuid;
+use windmill_test_utils::*;
+
+fn client() -> reqwest::Client {
+    reqwest::Client::new()
+}
+
+fn bearer(builder: reqwest::RequestBuilder, token: &str) -> reqwest::RequestBuilder {
+    builder.header("Authorization", format!("Bearer {token}"))
+}
+
+async fn create_token_with_label(port: u16, caller_token: &str, label: &str) -> reqwest::Response {
+    bearer(
+        client().post(format!("http://localhost:{port}/api/users/tokens/create")),
+        caller_token,
+    )
+    .json(&json!({ "label": label }))
+    .send()
+    .await
+    .unwrap()
+}
+
+/// Insert a completed job with a labeled-token `created_by`, running as `permissioned_as`
+/// (email `permissioned_as_email`) with the given `runnable_path` (which governs RLS).
+async fn insert_labeled_job(
+    db: &Pool<Postgres>,
+    created_by: &str,
+    runnable_path: &str,
+    permissioned_as: &str,
+    permissioned_as_email: &str,
+) -> Uuid {
+    let id = Uuid::new_v4();
+    sqlx::query(
+        "INSERT INTO v2_job (id, workspace_id, created_by, permissioned_as, permissioned_as_email, runnable_path, kind, tag, args, visible_to_owner)
+         VALUES ($1, 'test-workspace', $2, $3, $4, $5, 'script', 'deno', '{}'::jsonb, true)",
+    )
+    .bind(id)
+    .bind(created_by)
+    .bind(permissioned_as)
+    .bind(permissioned_as_email)
+    .bind(runnable_path)
+    .execute(db)
+    .await
+    .unwrap();
+    sqlx::query(
+        "INSERT INTO v2_job_completed (id, workspace_id, duration_ms, result, status)
+         VALUES ($1, 'test-workspace', 100, '{\"secret\":\"super-secret-value\"}'::jsonb, 'success')",
+    )
+    .bind(id)
+    .execute(db)
+    .await
+    .unwrap();
+    id
+}
+
+/// The core IDOR: an operator who mints a token whose label collides with another
+/// principal's labeled-token identity must NOT be able to read that principal's job — the
+/// `permissioned_as_email` of that job is the victim's, not the attacker's.
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_label_collision_does_not_grant_job_read(db: Pool<Postgres>) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let base = format!("http://localhost:{port}/api/w/test-workspace/jobs");
+
+    // A job submitted with a token labeled "collide", running as the admin (test-user).
+    let job_id = insert_labeled_job(
+        &db,
+        "label-collide",
+        "u/test-user/secret_script",
+        "u/test-user",
+        "test@windmill.dev",
+    )
+    .await;
+
+    // Sanity: the admin can read it, so the job exists and is otherwise readable.
+    let resp = bearer(
+        client().get(format!("{base}/completed/get/{job_id}")),
+        "SECRET_TOKEN",
+    )
+    .send()
+    .await?;
+    assert_eq!(resp.status(), 200, "admin must still read the job");
+
+    // The attacker (a different member, test-user-2) mints a colliding-label token.
+    let resp = create_token_with_label(port, "SECRET_TOKEN_2", "collide").await;
+    assert_eq!(resp.status(), 201);
+    let attacker_token = resp.text().await?;
+
+    // Reading the admin's job with the colliding token must be denied. Before the fix the
+    // `username_override == created_by` fast path returned the full result here.
+    let resp = bearer(
+        client().get(format!("{base}/completed/get/{job_id}")),
+        &attacker_token,
+    )
+    .send()
+    .await?;
+    assert!(
+        !resp.status().is_success(),
+        "colliding-label token must not read another principal's job (got {})",
+        resp.status()
+    );
+    let body = resp.text().await?;
+    assert!(
+        !body.contains("super-secret-value"),
+        "job result must not leak to the colliding-label token"
+    );
+
+    Ok(())
+}
+
+/// The fix must not regress the legitimate case: a principal re-reading its own
+/// labeled-token job is granted via the email-bound fast path, even when RLS would hide the
+/// job (the runnable lives in another user's space the caller has no RLS path to).
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_legit_labeled_self_read_still_works(db: Pool<Postgres>) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let base = format!("http://localhost:{port}/api/w/test-workspace/jobs");
+
+    // Created by test-user-2's labeled token, running as test-user-2, but the runnable lives
+    // under u/test-user so RLS alone would not reveal it to test-user-2 — the grant must come
+    // from the email-bound fast path.
+    let job_id = insert_labeled_job(
+        &db,
+        "label-mine",
+        "u/test-user/shared_script",
+        "u/test-user-2",
+        "test2@windmill.dev",
+    )
+    .await;
+
+    let resp = create_token_with_label(port, "SECRET_TOKEN_2", "mine").await;
+    assert_eq!(resp.status(), 201);
+    let token = resp.text().await?;
+
+    let resp = bearer(
+        client().get(format!("{base}/completed/get/{job_id}")),
+        &token,
+    )
+    .send()
+    .await?;
+    assert_eq!(
+        resp.status(),
+        200,
+        "owner must still read their own labeled-token job via the email-bound fast path"
+    );
+
+    Ok(())
+}
+
+/// P1 regression guard: the user-facing token API must keep accepting the labels that the
+/// webhook / http-route / email trigger panels mint (e.g. `webhook-<user>-<rand>`). The fix
+/// must not reserve those prefixes.
+#[sqlx::test(migrations = "../migrations", fixtures("base"))]
+async fn test_trigger_token_labels_still_creatable(db: Pool<Postgres>) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+
+    for label in [
+        "webhook-test-user-2-ab12",
+        "http-test-user-2-cd34",
+        "email-test-user-2-ef56",
+        "my-ci-token",
+    ] {
+        let resp = create_token_with_label(port, "SECRET_TOKEN_2", label).await;
+        assert_eq!(
+            resp.status(),
+            201,
+            "creating a token with label {label:?} must succeed"
+        );
+    }
+
+    Ok(())
+}
@@ -634,6 +634,69 @@ async fn test_compare_workspaces_comprehensive(db: Pool<Postgres>) -> anyhow::Re
        "Non-existent item should be deleted from workspace_diff"
    );

+    // ==============================================================
+    // Stale Archived Cache Test (regression)
+    // ==============================================================
+    //
+    // Unlike the lazy_test above (has_changes = NULL → always re-evaluated), a
+    // cached `has_changes = true` row is trusted without re-running the per-kind
+    // comparison. It can go stale: after a rename the old path keeps only
+    // archived versions, and for lock-gen languages the `has_changes = NULL`
+    // reset is deferred to the dependency job — so until that runs the archived
+    // old path lingers as a live "ahead" change carrying `exists_in_fork = true`.
+    // The visibility check treats archived as non-existent and finds nothing, so
+    // even this superadmin used to get `all_ahead_items_visible = false`. The fix
+    // re-validates such rows and drops the archived (== non-existent) item.
+    sqlx::query!(
+        "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)
+         VALUES ('wm-fork-test-workspace', 'f/shared/renamed_away', 67890, 'def main(): return 1', '', '', 'python3', 'test@windmill.dev', NOW(), true, false, false, false)"
+    )
+    .execute(&db)
+    .await?;
+    sqlx::query!(
+        "INSERT INTO workspace_diff
+         (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork)
+         VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/renamed_away', 'script', 1, 0, true, false, true)"
+    )
+    .execute(&db)
+    .await?;
+
+    let comparison3: serde_json::Value = client
+        .client()
+        .get(&format!(
+            "{base_url}/w/test-workspace/workspaces/compare/wm-fork-test-workspace"
+        ))
+        .send()
+        .await?
+        .json()
+        .await?;
+
+    // The archived item must be dropped (not surfaced) and must not trip the
+    // "changes not visible to your user" warning for a superadmin.
+    assert_eq!(
+        comparison3["all_ahead_items_visible"].as_bool(),
+        Some(true),
+        "archived (renamed-away) item must not trip the 'changes not visible' warning: {comparison3}"
+    );
+    assert!(
+        !comparison3["diffs"]
+            .as_array()
+            .unwrap()
+            .iter()
+            .any(|d| d["path"] == "f/shared/renamed_away"),
+        "archived item should be dropped, not surfaced as a diff: {comparison3}"
+    );
+    let stale_archived = sqlx::query!(
+        "SELECT has_changes FROM workspace_diff
+         WHERE path = 'f/shared/renamed_away' AND kind = 'script' AND source_workspace_id = 'test-workspace'"
+    )
+    .fetch_optional(&db)
+    .await?;
+    assert!(
+        stale_archived.is_none(),
+        "stale archived diff row should be re-evaluated and deleted"
+    );
+
    Ok(())
 }

@@ -191,6 +191,7 @@ async fn get_concurrent_intervals(
            script_path_exact: None,
            script_hash: None,
            created_by: None,
+            status: None,
            success: None,
            running: None,
            parent_job: None,
@@ -430,7 +430,15 @@ pub fn filter_list_completed_query(
            sqlb.and_where_in("created_by", &quoted);
        }
    }
-    if let Some(r) = &lq.success {
+    if let Some(status) = &lq.status {
+        let status = match status {
+            windmill_common::jobs::JobStatus::Success => "success",
+            windmill_common::jobs::JobStatus::Failure => "failure",
+            windmill_common::jobs::JobStatus::Canceled => "canceled",
+            windmill_common::jobs::JobStatus::Skipped => "skipped",
+        };
+        sqlb.and_where_eq("v2_job_completed.status", quote(status));
+    } else if let Some(r) = &lq.success {
        if *r {
            sqlb.and_where_eq("status", "'success'")
                .or_where_eq("status", "'skipped'");
@@ -572,6 +580,11 @@ pub fn list_completed_jobs_query(
            if lq.completed_before.is_some()
                || lq.completed_after.is_some()
                || lq.success == Some(false)
+                || matches!(
+                    lq.status,
+                    Some(windmill_common::jobs::JobStatus::Failure)
+                        | Some(windmill_common::jobs::JobStatus::Canceled)
+                )
            {
                "v2_job_completed.completed_at"
            } else {
@@ -653,6 +666,7 @@ mod tests {
            created_after_queue: None,
            completed_after: None,
            completed_before: None,
+            status: None,
            success: None,
            running: None,
            parent_job: None,
@@ -928,6 +942,23 @@ mod tests {
        assert!(sql.contains("'failure'"));
    }

+    #[test]
+    fn test_completed_filter_status_canceled() {
+        let lq = ListCompletedQuery {
+            status: Some(windmill_common::jobs::JobStatus::Canceled),
+            ..empty_completed_query()
+        };
+        let sqlb = filter_list_completed_query(
+            SqlBuilder::select_from("v2_job_completed").clone(),
+            &lq,
+            "ws",
+            false,
+        );
+        let sql = build_sql(sqlb);
+        assert!(sql.contains("v2_job_completed.status"));
+        assert!(sql.contains("'canceled'"));
+    }
+
    #[test]
    fn test_completed_order_by_completed_at() {
        let lq = ListCompletedQuery {
@@ -939,6 +970,25 @@ mod tests {
        assert!(sql.contains("completed_at"));
    }

+    #[test]
+    fn test_completed_order_by_completed_at_status_failure_canceled() {
+        // status=failure|canceled must order by v2_job_completed.completed_at so the
+        // partial index ix_v2_job_completed_failure_workspace serves both filtering
+        // and ordering in a single scan.
+        for status in [
+            windmill_common::jobs::JobStatus::Failure,
+            windmill_common::jobs::JobStatus::Canceled,
+        ] {
+            let lq = ListCompletedQuery { status: Some(status), ..empty_completed_query() };
+            let sqlb = list_completed_jobs_query("ws", Some(10), 0, &lq, &["id"], false, None);
+            let sql = build_sql(sqlb);
+            assert!(
+                sql.contains("ORDER BY v2_job_completed.completed_at"),
+                "expected order by completed_at, got: {sql}"
+            );
+        }
+    }
+
    #[test]
    fn test_completed_filter_label() {
        let lq = ListCompletedQuery {
@@ -19,7 +19,7 @@ use std::collections::HashMap;
 use uuid::Uuid;
 use windmill_common::{
    error,
-    jobs::{CompletedJob, JobKind, JobTriggerKind, QueuedJob},
+    jobs::{CompletedJob, JobKind, JobStatus, JobTriggerKind, QueuedJob},
    scripts::{ScriptHash, ScriptLang},
    utils::now_from_db,
    DB,
@@ -142,6 +142,7 @@ pub struct ListCompletedQuery {
    pub created_after_queue: Option<chrono::DateTime<chrono::Utc>>,
    pub completed_after: Option<chrono::DateTime<chrono::Utc>>,
    pub completed_before: Option<chrono::DateTime<chrono::Utc>>,
+    pub status: Option<JobStatus>,
    pub success: Option<bool>,
    pub running: Option<bool>,
    pub parent_job: Option<String>,
@@ -680,6 +681,7 @@ mod tests {
            created_after_queue: None,
            completed_after: None,
            completed_before: None,
+            status: None,
            success: None,
            running: Some(true),
            parent_job: None,
@@ -752,6 +754,7 @@ mod tests {
            created_after_queue: Some(specific_time),
            completed_after: None,
            completed_before: None,
+            status: None,
            success: None,
            running: None,
            parent_job: None,
@@ -242,6 +242,23 @@ async fn create_schedule(

    let mut tx: Transaction<'_, Postgres> = user_db.begin(&authed).await?;

+    // A git-sync/merge/create write into a fork never sets operational state:
+    // force `enabled = false` so a cloned / synced / merged / UI-created schedule
+    // can't fire alongside the parent's. The fork owner re-enables locally via
+    // `setenabled`. Schedule analog of the trigger rule in
+    // `windmill-trigger::handler::workspace_is_fork`; the read half (parent-value
+    // substitution on fork export) lives in `workspaces_export.rs`. Read fork-ness
+    // on the non-RLS `db` pool (like the other two sites) so the determination is
+    // complete regardless of the caller's folder perms.
+    let target_is_fork: bool = sqlx::query_scalar!(
+        "SELECT parent_workspace_id IS NOT NULL FROM workspace WHERE id = $1",
+        w_id
+    )
+    .fetch_optional(&db)
+    .await?
+    .flatten()
+    .unwrap_or(false);
+
    // Check schedule for error
    ScheduleType::from_str(&ns.schedule, ns.cron_version.as_deref(), true)?;

@@ -341,7 +358,12 @@ async fn create_schedule(
        // flows (CLI merge, UI merge, `wmill push` of a fork tarball) — which
        // either send the source's actual flag (create case) or omit `enabled`
        // entirely (update case, where `EditSchedule` lacks the field).
-        ns.enabled.unwrap_or(true),
+        // A write into a fork always lands disabled regardless of the request.
+        if target_is_fork {
+            false
+        } else {
+            ns.enabled.unwrap_or(true)
+        },
        resolved_email,
        resolved_permissioned_as,
        ns.on_failure,
@@ -413,7 +435,7 @@ async fn create_schedule(
        .await?;
    }

-    if ns.enabled.unwrap_or(true) {
+    if !target_is_fork && ns.enabled.unwrap_or(true) {
        tx = push_scheduled_job(&db, tx, &schedule, Some(&authed.clone().into()), None).await?
    }
    tx.commit().await?;
@@ -139,6 +139,10 @@ pub fn global_service() -> Router {
            "/tokens/update_scopes/{token_prefix}",
            post(update_token_scopes),
        )
+        .route(
+            "/tokens/update_label/{token_prefix}",
+            post(update_token_label),
+        )
        .route("/tokens/list", get(list_tokens))
        .route("/tokens/impersonate", post(impersonate))
        .route("/usage", get(get_usage))
@@ -2408,6 +2412,89 @@ async fn update_token_scopes(
    Ok(format!("updated scopes for token {prefix}"))
 }

+#[derive(Deserialize)]
+struct UpdateTokenLabelRequest {
+    label: Option<String>,
+}
+
+async fn update_token_label(
+    Extension(db): Extension<DB>,
+    authed: ApiAuthed,
+    Path(token_prefix): Path<String>,
+    Json(req): Json<UpdateTokenLabelRequest>,
+) -> Result<String> {
+    // The new label must not collide with a system-token namespace (`session`,
+    // `ephemeral*`, `debugger-token`, `mcp-oauth-*`): those labels are
+    // load-bearing, and a user-set collision would orphan the token — hidden
+    // from the UI (`isUserToken`) and rejected by the editability guard below —
+    // while it still authenticates. (`is_user_token(None)` is true, so clearing
+    // the label is allowed.)
+    if !windmill_common::auth::is_user_token(req.label.as_deref()) {
+        return Err(Error::BadRequest(
+            "label collides with a reserved system-token namespace".to_string(),
+        ));
+    }
+
+    // Matches the `token.label VARCHAR(1000)` column — reject overlong labels with
+    // a 400 rather than letting Postgres raise a 500.
+    const MAX_TOKEN_LABEL_LEN: usize = 1000;
+    if req
+        .label
+        .as_deref()
+        .is_some_and(|l| l.chars().count() > MAX_TOKEN_LABEL_LEN)
+    {
+        return Err(Error::BadRequest(format!(
+            "label must be at most {MAX_TOKEN_LABEL_LEN} characters"
+        )));
+    }
+
+    let mut tx = db.begin().await?;
+
+    // Only user-created tokens may be relabeled — system tokens carry the
+    // load-bearing labels described above. This SQL mirrors the canonical
+    // `windmill_common::auth::is_user_token`; keep the two in sync (note the
+    // case-insensitive `ephemeral` match).
+    let updated: Option<String> = sqlx::query_scalar!(
+        "UPDATE token SET label = $1
+           WHERE email = $2 AND token_prefix = $3
+             AND (label IS NULL OR (
+                 label <> 'session'
+                 AND lower(label) NOT LIKE 'ephemeral%'
+                 AND label <> 'debugger-token'
+                 AND label NOT LIKE 'mcp-oauth-%'
+             ))
+           RETURNING token_prefix",
+        req.label.as_deref(),
+        &authed.email,
+        &token_prefix,
+    )
+    .fetch_optional(&mut *tx)
+    .await?;
+
+    let prefix = updated.ok_or_else(|| {
+        Error::NotFound(format!(
+            "token {token_prefix} not found, not owned by user, or not editable"
+        ))
+    })?;
+
+    audit_log(
+        &mut *tx,
+        &authed,
+        "users.token.update_label",
+        ActionKind::Update,
+        &"global",
+        Some(&prefix),
+        Some([("label", req.label.as_deref().unwrap_or(""))].into()),
+    )
+    .await?;
+
+    tx.commit().await?;
+
+    windmill_api_auth::invalidate_token_from_cache(&prefix);
+
+    Ok(format!("updated label for token {prefix}"))
+}
+
 async fn leave_workspace(
    Extension(db): Extension<DB>,
    Path(w_id): Path<String>,
@@ -6290,13 +6290,67 @@ async fn compare_workspaces(
    .fetch_all(&db)
    .await?;

+    // A cached `has_changes = true` row is trusted without re-running the
+    // per-kind comparison, but that verdict can go stale: an item archived or
+    // deleted after it was cached still carries `exists_in_*=true` here. The
+    // common offender is the old path after a rename — for lock-gen languages
+    // (Python/TS/…) the `has_changes=NULL` reset is deferred to the dependency
+    // job, so until that runs (or if it fails) the archived old path looks like
+    // a live ahead change. Treat archived as non-existent: re-validate such rows
+    // against the live tables and, if the item no longer exists on a side the
+    // cache claims, re-evaluate it below so it gets corrected or removed.
+    //
+    // Only scripts/flows can hit this (they have `archived`; other kinds reset
+    // synchronously on delete). Probe both sides in one batched query per kind
+    // (mirroring `query_visible_items`) rather than per row, to keep the hot
+    // compare path off an O(number of cached diffs) sequence of round trips.
+    let (live_source, live_fork) = {
+        let mut cached_source: HashMap<&str, Vec<&str>> = HashMap::new();
+        let mut cached_fork: HashMap<&str, Vec<&str>> = HashMap::new();
+        for item in &diff_items {
+            if item.has_changes == Some(true) && (item.kind == "script" || item.kind == "flow") {
+                if item.exists_in_source.unwrap_or(false) {
+                    cached_source
+                        .entry(item.kind.as_str())
+                        .or_default()
+                        .push(item.path.as_str());
+                }
+                if item.exists_in_fork.unwrap_or(false) {
+                    cached_fork
+                        .entry(item.kind.as_str())
+                        .or_default()
+                        .push(item.path.as_str());
+                }
+            }
+        }
+        (
+            existing_runnables(&db, &source_workspace_id, &cached_source).await?,
+            existing_runnables(&db, &fork_workspace_id, &cached_fork).await?,
+        )
+    };
+
    let mut confirmed_diffs = vec![];
    for item in diff_items {
        if let Some(has_changes) = item.has_changes {
-            if has_changes {
-                confirmed_diffs.push(item);
+            if !has_changes {
+                // Defensive: rows that compared equal are normally deleted, so
+                // this is rarely hit. Not a diff — skip.
+                continue;
            }
-            continue;
+            // Stale only applies to script/flow (others aren't in the probed
+            // sets); a row whose claimed-existing side has no live version is
+            // stale and falls through to re-evaluation.
+            let key = (item.kind.clone(), item.path.clone());
+            let fork_stale = item.exists_in_fork.unwrap_or(false) && !live_fork.contains(&key);
+            let source_stale =
+                item.exists_in_source.unwrap_or(false) && !live_source.contains(&key);
+            let probed = item.kind == "script" || item.kind == "flow";
+            if !(probed && (fork_stale || source_stale)) {
+                // Cache is still valid (or not a probed kind) — trust it.
+                confirmed_diffs.push(item);
+                continue;
+            }
+            // Stale cache: fall through to re-evaluate (and correct/delete) below.
        }

        let item_comparison = match item.kind.as_str() {
@@ -6721,6 +6775,50 @@ async fn query_visible_items<'c>(
    Ok(visible)
 }

+/// Batched existence probe used to detect stale `workspace_diff` cache rows.
+///
+/// Given candidate paths grouped by kind, returns the set of `(kind, path)`
+/// that currently have a *deployable* (non-archived) version in the workspace,
+/// mirroring the existence semantics of `compare_two_scripts` /
+/// `compare_two_flows`. Only scripts and flows are probed — they're the only
+/// kinds with `archived`, and the only ones whose diff-row reset can lag behind
+/// the actual change (deferred dependency job for lock-gen languages); other
+/// kinds reset synchronously on delete, so their cache is trusted (and they're
+/// never passed in). One query per kind keeps the compare path off a per-row
+/// sequence of round trips. Runs on `&db` (no RLS) — this is a pure existence
+/// check; authorization stays in `filter_visible_diffs` / `query_visible_items`.
+async fn existing_runnables(
+    db: &DB,
+    workspace_id: &str,
+    items_by_kind: &HashMap<&str, Vec<&str>>,
+) -> Result<HashSet<(String, String)>> {
+    let mut existing = HashSet::new();
+    for (kind, paths) in items_by_kind {
+        let paths_vec: Vec<String> = paths.iter().map(|s| s.to_string()).collect();
+        let found: Vec<String> = match *kind {
+            "script" => sqlx::query_scalar!(
+                "SELECT DISTINCT path FROM script WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
+                workspace_id,
+                &paths_vec
+            )
+            .fetch_all(db)
+            .await?,
+            "flow" => sqlx::query_scalar!(
+                "SELECT path FROM flow WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
+                workspace_id,
+                &paths_vec
+            )
+            .fetch_all(db)
+            .await?,
+            _ => vec![],
+        };
+        for path in found {
+            existing.insert((kind.to_string(), path));
+        }
+    }
+    Ok(existing)
+}
+
 #[derive(Debug)]
 struct ItemComparison {
    has_changes: bool,
@@ -1,7 +1,7 @@
 openapi: "3.0.3"

 info:
-  version: 1.717.1
+  version: 1.720.0
  title: Windmill API

  contact:
@@ -5120,6 +5120,37 @@ paths:
              schema:
                type: string

+  /users/tokens/update_label/{token_prefix}:
+    post:
+      summary: update label of an existing token (owner only)
+      operationId: updateTokenLabel
+      tags:
+        - user
+      parameters:
+        - name: token_prefix
+          in: path
+          required: true
+          schema:
+            type: string
+      requestBody:
+        description: new label (null or omitted = no label)
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                label:
+                  type: string
+                  nullable: true
+      responses:
+        "200":
+          description: label updated
+          content:
+            text/plain:
+              schema:
+                type: string
+
  /users/tokens/list:
    get:
      summary: list token
@@ -11821,6 +11852,16 @@ paths:
          in: query
          schema:
            type: boolean
+        - name: status
+          description: filter on the exact completed job status. Unlike `success=true` (which also matches `skipped`), `status=success` matches only `success`.
+          in: query
+          schema:
+            type: string
+            enum:
+              - success
+              - failure
+              - canceled
+              - skipped
        - name: all_workspaces
          description: get jobs from all workspaces (only valid if request come from the `admins` workspace)
          in: query
@@ -12056,6 +12097,16 @@ paths:
        - $ref: "#/components/parameters/StartedBefore"
        - $ref: "#/components/parameters/StartedAfter"
        - $ref: "#/components/parameters/Success"
+        - name: status
+          description: filter on the exact completed job status. Unlike `success=true` (which also matches `skipped`), `status=success` matches only `success`.
+          in: query
+          schema:
+            type: string
+            enum:
+              - success
+              - failure
+              - canceled
+              - skipped
        - $ref: "#/components/parameters/JobKinds"
        - $ref: "#/components/parameters/ArgsFilter"
        - $ref: "#/components/parameters/ResultFilter"
@@ -12263,6 +12314,16 @@ paths:
          in: query
          schema:
            type: boolean
+        - name: status
+          description: filter on the exact completed job status. Unlike `success=true` (which also matches `skipped`), `status=success` matches only `success`.
+          in: query
+          schema:
+            type: string
+            enum:
+              - success
+              - failure
+              - canceled
+              - skipped
        - name: all_workspaces
          description: get jobs from all workspaces (only valid if request come from the `admins` workspace)
          in: query
@@ -19772,6 +19833,16 @@ paths:
          in: query
          schema:
            type: boolean
+        - name: status
+          description: filter on the exact completed job status. Unlike `success=true` (which also matches `skipped`), `status=success` matches only `success`.
+          in: query
+          schema:
+            type: string
+            enum:
+              - success
+              - failure
+              - canceled
+              - skipped
        - name: all_workspaces
          description: get jobs from all workspaces (only valid if request come from the `admins` workspace)
          in: query
@@ -13,9 +13,10 @@ use serde_json::value::RawValue;
 use sqlx::types::JsonRawValue;
 use windmill_common::{
    error::Error,
+    jobs::WM_TRACEPARENT,
    triggers::{RunnableFormat, RunnableFormatVersion, TriggerKind},
    worker::to_raw_value,
-    DB,
+    DB, OTEL_TRACING_ENABLED,
 };
 use windmill_queue::PushArgsOwned;

@@ -280,6 +281,13 @@ impl WebhookArgs {
        self,
        runnable_format: RunnableFormat,
    ) -> Result<PushArgsOwned, Error> {
+        // Capture the inbound W3C `traceparent` before `self.metadata` is
+        // consumed below. Read back at root-job completion to link the job's
+        // OTLP span to the originating distributed trace. Deliberately bypasses
+        // the header whitelist, and is gated to OTel-enabled instances so others
+        // don't get a stray `_wm_traceparent` arg key.
+        let trace_context = inbound_traceparent(&self.metadata.headers);
+
        let headers = build_headers(
            &self.metadata.headers,
            self.metadata.query_include_header,
@@ -292,7 +300,7 @@ impl WebhookArgs {
            runnable_format.has_preprocessor,
        );

-        match runnable_format {
+        let mut push_args = match runnable_format {
            RunnableFormat { has_preprocessor: true, version: RunnableFormatVersion::V2 } => {
                let mut args = HashMap::new();

@@ -307,7 +315,7 @@ impl WebhookArgs {
                    }),
                );

-                Ok(PushArgsOwned { args, extra: None })
+                PushArgsOwned { args, extra: None }
            }
            RunnableFormat { has_preprocessor, .. } => {
                let mut extra = HashMap::new();
@@ -343,16 +351,40 @@ impl WebhookArgs {
                        if query_wrap_body {
                            body = HashMap::from([("body".to_string(), to_raw_value(&body))]);
                        }
-                        Ok(PushArgsOwned { args: body, extra })
+                        PushArgsOwned { args: body, extra }
                    }
                    Body::NoHashMap(args) => {
                        let mut hm = HashMap::new();
                        hm.insert("body".to_string(), args);
-                        Ok(PushArgsOwned { args: hm, extra })
+                        PushArgsOwned { args: hm, extra }
                    }
                }
            }
+        };
+
+        // `_wm_traceparent` is Windmill-controlled: strip any caller-supplied
+        // value (e.g. smuggled through the request body) so only the header we
+        // captured above can become the job's inbound trace context. Then stash
+        // the captured value as a reserved arg key — it rides the `args` jsonb
+        // like `_ENTRYPOINT_OVERRIDE`; normal scripts never see it (args are
+        // bound by declared parameter name).
+        push_args.args.remove(WM_TRACEPARENT);
+        if let Some(ref mut extra) = push_args.extra {
+            extra.remove(WM_TRACEPARENT);
        }
+        if let Some(trace_context) = trace_context {
+            let raw = to_raw_value(&trace_context);
+            match push_args.extra {
+                Some(ref mut extra) => {
+                    extra.insert(WM_TRACEPARENT.to_string(), raw);
+                }
+                None => {
+                    push_args.args.insert(WM_TRACEPARENT.to_string(), raw);
+                }
+            }
+        }
+
+        Ok(push_args)
    }
 }

@@ -487,6 +519,23 @@ lazy_static::lazy_static! {
        .collect()).unwrap_or_default();
 }

+/// Extract the inbound W3C `traceparent` header so the enqueued job can be
+/// linked back to the originating distributed trace. Returns `None` when OTel
+/// tracing is disabled (so non-tracing instances don't accumulate a stray
+/// reserved arg key) or when no `traceparent` header is present. The W3C format
+/// is not validated here — it is checked later at use time
+/// (`valid_w3c_traceparent` for the env, EE `span_cx_from_traceparent` for the
+/// span).
+fn inbound_traceparent(headers: &HeaderMap) -> Option<String> {
+    if !OTEL_TRACING_ENABLED.load(std::sync::atomic::Ordering::Relaxed) {
+        return None;
+    }
+    headers
+        .get("traceparent")
+        .and_then(|v| v.to_str().ok())
+        .map(|s| s.to_string())
+}
+
 pub fn build_headers(
    headers: &HeaderMap,
    include_header: Option<String>,
@@ -1001,15 +1001,40 @@ async fn require_job_read_access(
    // Fast path: you can always read a job you launched. This is also load-bearing
    // for apps — a component job runs as the app policy's `permissioned_as`, but its
    // `created_by` is the launching viewer, so the RLS probe below would hide it.
-    if created_by == authed.username
-        || authed
-            .username_override
-            .as_deref()
-            .is_some_and(|u| u == created_by)
-    {
+    if created_by == authed.username {
        return Ok(());
    }

+    // `username_override` is derived from the token *label* (`username_override_from_label`),
+    // which is fully user-controlled with no uniqueness/ownership check (webhook-/http-/
+    // email-/ws- trigger tokens, `ephemeral-script-end-user-*`, and the generic `label-*`
+    // all flow through it). A bare `username_override == created_by` match is therefore
+    // forgeable: any member can mint a token with a colliding label and read another
+    // principal's jobs (IDOR — results/args/logs with resolved secrets). Bind the grant to
+    // a non-forgeable attribute instead: the job must actually run as the caller's own
+    // identity, i.e. its `permissioned_as_email` (the token owner's email, never set from
+    // the label) equals `authed.email`. This still admits every legitimate same-owner
+    // re-read (trigger tokens reading their own webhook/http/email jobs, the
+    // ephemeral-script-end-user worker token, generic labeled tokens) while denying
+    // cross-principal collisions. The DB hit only happens when an override is present and
+    // matches, so the common session/token path stays query-free.
+    if authed
+        .username_override
+        .as_deref()
+        .is_some_and(|u| u == created_by)
+    {
+        let job_email = sqlx::query_scalar!(
+            "SELECT permissioned_as_email FROM v2_job WHERE id = $1 AND workspace_id = $2",
+            job_id,
+            w_id,
+        )
+        .fetch_optional(db)
+        .await?;
+        if job_email.as_deref() == Some(authed.email.as_str()) {
+            return Ok(());
+        }
+    }
+
    // Share read link: a valid view token minted by someone with read access grants
    // this authenticated member read of the shared job and its flow subtree.
    if let Some(token) = view_token {
@@ -1211,6 +1236,59 @@ async fn require_job_update_read_access(
    require_job_read_access(db, user_db, authed, w_id, job_id, &created_by, view_token).await
 }

+/// Whether a validated approval token should grant the job-read bypass. The token alone
+/// is sufficient unless the current approval step has `user_auth_required`, in which case
+/// only an authorized approver may read the job (and thus its args/flow inputs).
+async fn approval_token_grants_view(
+    db: &DB,
+    w_id: &str,
+    flow_id: Uuid,
+    opt_authed: &Option<ApiAuthed>,
+) -> error::Result<bool> {
+    #[derive(sqlx::FromRow)]
+    struct FlowAuthRow {
+        script_path: Option<String>,
+        email: String,
+        flow_status: Option<serde_json::Value>,
+    }
+    let row = sqlx::query_as::<_, FlowAuthRow>(
+        "SELECT j.runnable_path as script_path, j.permissioned_as_email as email, s.flow_status
+         FROM v2_job j
+         LEFT JOIN v2_job_status s ON s.id = j.id
+         WHERE j.id = $1 AND j.workspace_id = $2",
+    )
+    .bind(flow_id)
+    .bind(w_id)
+    .fetch_optional(db)
+    .await?;
+    let Some(row) = row else {
+        return Ok(false);
+    };
+
+    // approval_conditions are stored at the top of flow_status for both classic and WAC flows.
+    let approval_conditions = row
+        .flow_status
+        .as_ref()
+        .and_then(|v| v.get("approval_conditions").cloned())
+        .and_then(|v| serde_json::from_value::<ApprovalConditions>(v).ok());
+
+    let user_auth_required = approval_conditions
+        .as_ref()
+        .map(|ac| ac.user_auth_required)
+        .unwrap_or(false);
+
+    if !user_auth_required {
+        return Ok(true);
+    }
+
+    Ok(can_approve_step(
+        opt_authed,
+        &approval_conditions,
+        row.script_path.as_deref(),
+        row.email.as_str(),
+    ))
+}
+
 async fn get_job(
    OptViewToken(view_token): OptViewToken,
    OptAuthed(opt_authed): OptAuthed,
@@ -1229,17 +1307,31 @@ async fn get_job(
    // so the approval page can render job metadata without login. The approval
    // URL usually carries the flow id directly — try that first and only
    // resolve the parent flow if the direct check fails.
-    let has_valid_approval_token = if let Some(ref token) = approval_token {
+    let approved_flow_id: Option<Uuid> = if let Some(ref token) = approval_token {
        if validate_approval_token(&db, token, id, &w_id).await.is_ok() {
-            true
+            Some(id)
        } else if let Ok(flow_id) = get_flow_id_for_job(&db, id).await {
-            flow_id != id
+            if flow_id != id
                && validate_approval_token(&db, token, flow_id, &w_id)
                    .await
                    .is_ok()
+            {
+                Some(flow_id)
+            } else {
+                None
+            }
        } else {
-            false
+            None
        }
+    } else {
+        None
+    };
+
+    // A valid token grants the read bypass, but only to an authorized approver when the
+    // current approval step requires auth — otherwise the token (which lives in the shareable
+    // approval URL) would leak the flow inputs to anyone holding the link.
+    let has_valid_approval_token = if let Some(flow_id) = approved_flow_id {
+        approval_token_grants_view(&db, &w_id, flow_id, &opt_authed).await?
    } else {
        false
    };
@@ -2550,15 +2642,19 @@ async fn list_filtered_job_uuids(
        false,
        get_scope_tags(&authed),
    );
-    let sqlb2 = list_queue_jobs_query(
-        w_id.as_str(),
-        &lq.into(),
-        &["v2_job.id"],
-        Pagination { page: None, per_page: None },
-        false,
-        get_scope_tags(&authed),
-    );
-    let query = sqlb.union_all(sqlb2.subquery()?).subquery()?;
+    let query = if lq.status.is_some() {
+        sqlb.subquery()?
+    } else {
+        let sqlb2 = list_queue_jobs_query(
+            w_id.as_str(),
+            &lq.into(),
+            &["v2_job.id"],
+            Pagination { page: None, per_page: None },
+            false,
+            get_scope_tags(&authed),
+        );
+        sqlb.union_all(sqlb2.subquery()?).subquery()?
+    };
    let ids = sqlx::query_scalar(query.as_str()).fetch_all(&db).await?;
    Ok(Json(ids))
 }
@@ -2716,9 +2812,9 @@ async fn list_jobs(
        tracing::warn!("offset is not 0, but is ignored for list_jobs. Use created_before or completed_before instead.");
    }

-    if lq.success.is_some() && lq.running.is_some_and(|x| x) {
+    if (lq.success.is_some() || lq.status.is_some()) && lq.running.is_some_and(|x| x) {
        return Err(error::Error::BadRequest(
-            "cannot specify both success and running".to_string(),
+            "cannot specify success/status with running".to_string(),
        ));
    }

@@ -2771,6 +2867,7 @@ async fn list_jobs(
    };

    let sql = if lq.success.is_none()
+        && lq.status.is_none()
        && lq.label.is_none()
        && lq.result.is_none()
        && !lq.is_skipped.unwrap_or(false)
@@ -2796,7 +2893,7 @@ async fn list_jobs(
    } else {
        if sqlc.is_none() {
            return Err(error::Error::BadRequest(
-                "cannot specify success, label, created_or_started_before, or starte
+                "cannot specify success, status, label, created_or_started_before, or starte
                    d_before with running"
                    .to_string(),
            ));
@@ -3102,6 +3199,43 @@ struct ApprovalInfo {
    approvers: Vec<Approval>,
 }

+/// Whether `opt_authed` is allowed to approve — and therefore view — this approval step.
+/// Mirrors the authorization performed at the resume boundary: workspace admins and owners
+/// of the runnable always qualify; otherwise the approval conditions (user_auth_required /
+/// user_groups_required / self_approval_disabled) decide. When the step does not require auth,
+/// an anonymous (token-only) caller qualifies.
+fn can_approve_step(
+    opt_authed: &Option<ApiAuthed>,
+    approval_conditions: &Option<ApprovalConditions>,
+    script_path: Option<&str>,
+    trigger_email: &str,
+) -> bool {
+    match opt_authed {
+        Some(authed) => {
+            if authed.is_admin {
+                return true;
+            }
+            let is_owner = script_path
+                .map(|p| require_owner_of_path(authed, p).is_ok())
+                .unwrap_or(false);
+            if is_owner {
+                return true;
+            }
+            conditionally_require_authed_user(
+                Some(authed.clone()),
+                approval_conditions.clone(),
+                trigger_email,
+            )
+            .is_ok()
+        }
+        // Not logged in — only acceptable when the step does not require auth.
+        None => !approval_conditions
+            .as_ref()
+            .map(|ac| ac.user_auth_required)
+            .unwrap_or(false),
+    }
+}
+
 async fn get_approval_info(
    OptAuthed(opt_authed): OptAuthed,
    Extension(db): Extension<DB>,
@@ -3260,32 +3394,33 @@ async fn get_approval_info(
        .map(|ac| ac.user_auth_required)
        .unwrap_or(false);

-    // Determine if current user can approve
-    let can_approve = if let Some(ref authed) = opt_authed {
-        if authed.is_admin {
-            true
-        } else {
-            let is_owner = row
-                .script_path
-                .as_deref()
-                .map(|p| require_owner_of_path(authed, p).is_ok())
-                .unwrap_or(false);
-            if is_owner {
-                true
-            } else {
-                let trigger_email = row.email.as_str();
-                conditionally_require_authed_user(
-                    Some(authed.clone()),
-                    approval_conditions.clone(),
-                    trigger_email,
-                )
-                .is_ok()
-            }
-        }
-    } else {
-        // Not logged in — can approve only if no auth required
-        !user_auth_required
-    };
+    // Determine if current user can approve this step.
+    let can_approve = can_approve_step(
+        &opt_authed,
+        &approval_conditions,
+        row.script_path.as_deref(),
+        row.email.as_str(),
+    );
+
+    // When the step requires auth, the approval details must be revealed only to authorized
+    // approvers — a valid token alone is not sufficient. Return a stripped response that lets
+    // the frontend render the sign-in / not-authorized state without leaking the form,
+    // description, prefilled args, or other approvers' identities.
+    let can_view = !user_auth_required || can_approve;
+    if !can_view {
+        return Ok(Json(ApprovalInfo {
+            flow_id: row.id,
+            form_schema: None,
+            description: None,
+            default_args: None,
+            enums: None,
+            approval_conditions,
+            can_approve: false,
+            user_auth_required,
+            hide_cancel: None,
+            approvers: vec![],
+        }));
+    }

    // Get existing approvers
    let approvers: Vec<Approval> = sqlx::query_as::<_, (i32, Option<String>)>(
@@ -9073,3 +9208,90 @@ async fn get_otel_traces(

    Ok(Json(traces))
 }
+
+#[cfg(test)]
+mod approval_view_gate_tests {
+    use super::*;
+
+    fn authed(username: &str, is_admin: bool, groups: Vec<String>) -> ApiAuthed {
+        ApiAuthed {
+            email: format!("{username}@example.com"),
+            username: username.to_string(),
+            is_admin,
+            is_operator: false,
+            groups,
+            folders: vec![],
+            scopes: None,
+            username_override: None,
+            token_prefix: None,
+            read_only: false,
+        }
+    }
+
+    fn conds(user_auth_required: bool, groups: Vec<String>) -> ApprovalConditions {
+        ApprovalConditions {
+            user_auth_required,
+            user_groups_required: groups,
+            self_approval_disabled: false,
+        }
+    }
+
+    // Mirrors the view gate applied in get_approval_info and get_job:
+    // details are revealed only when the step doesn't require auth OR the caller
+    // is an authorized approver.
+    fn can_view(
+        opt_authed: &Option<ApiAuthed>,
+        approval_conditions: &Option<ApprovalConditions>,
+        script_path: Option<&str>,
+        trigger_email: &str,
+    ) -> bool {
+        let user_auth_required = approval_conditions
+            .as_ref()
+            .map(|c| c.user_auth_required)
+            .unwrap_or(false);
+        !user_auth_required
+            || can_approve_step(opt_authed, approval_conditions, script_path, trigger_email)
+    }
+
+    #[test]
+    fn anonymous_cannot_view_when_auth_required() {
+        // The regression: an unauthenticated holder of the approval token must see nothing.
+        let c = Some(conds(true, vec![]));
+        assert!(!can_view(&None, &c, Some("f/team/flow"), "trigger@example.com"));
+    }
+
+    #[test]
+    fn anonymous_can_view_when_no_auth_required() {
+        // Unchanged behaviour: token alone is sufficient when auth isn't required.
+        let c = Some(conds(false, vec![]));
+        assert!(can_view(&None, &c, Some("f/team/flow"), "trigger@example.com"));
+        // No approval conditions at all also allows token-only view.
+        assert!(can_view(&None, &None, Some("f/team/flow"), "trigger@example.com"));
+    }
+
+    #[test]
+    fn admin_can_view_when_auth_required() {
+        let c = Some(conds(true, vec!["approvers".to_string()]));
+        let a = Some(authed("alice", true, vec![]));
+        assert!(can_view(&a, &c, Some("f/team/flow"), "trigger@example.com"));
+    }
+
+    #[test]
+    fn owner_can_view_when_auth_required() {
+        let c = Some(conds(true, vec!["approvers".to_string()]));
+        let a = Some(authed("bob", false, vec![]));
+        // bob owns u/bob/flow regardless of group membership.
+        assert!(can_view(&a, &c, Some("u/bob/flow"), "trigger@example.com"));
+    }
+
+    #[cfg(feature = "enterprise")]
+    #[test]
+    fn group_membership_decides_view_when_auth_required() {
+        let c = Some(conds(true, vec!["approvers".to_string()]));
+        let member = Some(authed("carol", false, vec!["approvers".to_string()]));
+        let outsider = Some(authed("dave", false, vec!["other".to_string()]));
+        // Use a non-owned folder path so ownership doesn't short-circuit the check.
+        assert!(can_view(&member, &c, Some("f/team/flow"), "trigger@example.com"));
+        assert!(!can_view(&outsider, &c, Some("f/team/flow"), "trigger@example.com"));
+    }
+}
@@ -3,15 +3,17 @@ use axum::{
    Extension,
 };
 use bytes::Bytes;
+use hmac::{Hmac, Mac};
 use http::HeaderMap;
 use hyper::StatusCode;
 use reqwest::Client;
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
+use sha2::Sha256;
 use sqlx::types::Uuid;
 use std::collections::HashMap;
-use windmill_common::error::Error;
-use windmill_common::variables::get_secret_value_as_admin;
+use windmill_common::error::{to_anyhow, Error};
+use windmill_common::variables::{get_secret_value_as_admin, get_workspace_key};

 use crate::db::{ApiAuthed, DB};
 use crate::jobs::{QueryApprover, ResumeUrls};
@@ -111,6 +113,9 @@ struct ModalActionValue {
    dynamic_enums_json: Option<String>,
    resume_button_text: Option<String>,
    cancel_button_text: Option<String>,
+    // HMAC over (w_id, job_id, path) keyed on the workspace key; minted by
+    // `send_slack_message`, required by the OpenModal callback branch.
+    signature: Option<String>,
 }

 #[derive(Deserialize, Debug)]
@@ -119,8 +124,16 @@ struct PrivateMetadata {
    resource_path: String,
    container: Container,
    hide_cancel: Option<bool>,
+    // HMAC over (w_id, resource_path) keyed on the workspace key; minted when the modal is
+    // built, required by `handle_submission` before the resource_path is decrypted.
+    signature: Option<String>,
 }

+// Opportunistic transport-level check: when `SLACK_SIGNING_SECRET` is configured we verify
+// the Slack request signature (which also defeats replay). It is NOT the primary defense:
+// the secret is unset in the default deployment, so authorization of the sensitive actions
+// is instead anchored on a per-workspace HMAC over the callback payload itself (see
+// `verify_slack_payload`), which holds even when this check is a no-op.
 #[cfg(feature = "oauth2")]
 fn verify_slack_callback_signature(headers: &HeaderMap, body: &str) -> Result<(), Error> {
    if let Some(sv) = crate::SLACK_SIGNING_SECRET.as_ref() {
@@ -143,6 +156,66 @@ fn verify_slack_callback_signature(_headers: &HeaderMap, _body: &str) -> Result<
    Ok(())
 }

+/// HMAC keyed on the per-workspace encryption key (the same trust anchor as resume-URL
+/// signatures). Used to authenticate the `/api/slack` callback payload itself so the
+/// unauthenticated route cannot be driven into decrypting arbitrary workspace variables,
+/// regardless of whether `SLACK_SIGNING_SECRET` is configured.
+type SlackPayloadHmac = Hmac<Sha256>;
+
+/// Domain-separation tag prepended to every Slack-payload MAC. The workspace key is also used
+/// for resume-secret signatures (`create_signature` in `jobs.rs`), and those secrets are
+/// distributed to approvers in resume URLs — so a fixed, scheme-specific prefix makes the two
+/// MAC families non-interchangeable by construction rather than relying on their byte layouts
+/// happening to differ. Bump the version suffix if the signed layout ever changes.
+const SLACK_PAYLOAD_HMAC_DOMAIN: &[u8] = b"slack_payload_v1\0";
+
+/// Sign the security-sensitive fields of a Slack callback payload with the workspace key.
+/// Parts are joined with a `\0` delimiter (absent from paths/UUIDs) so distinct field tuples
+/// cannot collide into the same MAC.
+async fn sign_slack_payload(db: &DB, w_id: &str, parts: &[&[u8]]) -> Result<String, Error> {
+    let key = get_workspace_key(w_id, db).await?;
+    let mut mac = SlackPayloadHmac::new_from_slice(key.as_bytes()).map_err(to_anyhow)?;
+    mac.update(SLACK_PAYLOAD_HMAC_DOMAIN);
+    mac.update(w_id.as_bytes());
+    for part in parts {
+        mac.update(b"\0");
+        mac.update(part);
+    }
+    Ok(hex::encode(mac.finalize().into_bytes()))
+}
+
+/// Verify a signature produced by [`sign_slack_payload`] in constant time. A missing or
+/// malformed signature is rejected: an attacker cannot forge one without the workspace key.
+async fn verify_slack_payload(
+    db: &DB,
+    w_id: &str,
+    parts: &[&[u8]],
+    signature: Option<&str>,
+) -> Result<(), Error> {
+    let signature = signature.ok_or_else(|| {
+        Error::NotAuthorized("Slack callback rejected: missing payload signature".to_string())
+    })?;
+    let provided = hex::decode(signature).map_err(|_| {
+        Error::NotAuthorized("Slack callback rejected: malformed payload signature".to_string())
+    })?;
+    // Map a missing workspace key (e.g. non-existent workspace) to the same generic 401 as a
+    // bad signature, so an unauthenticated caller cannot use the status code (500 vs 401) as a
+    // workspace-existence oracle.
+    let key = get_workspace_key(w_id, db).await.map_err(|_| {
+        Error::NotAuthorized("Slack callback rejected: invalid payload signature".to_string())
+    })?;
+    let mut mac = SlackPayloadHmac::new_from_slice(key.as_bytes()).map_err(to_anyhow)?;
+    mac.update(SLACK_PAYLOAD_HMAC_DOMAIN);
+    mac.update(w_id.as_bytes());
+    for part in parts {
+        mac.update(b"\0");
+        mac.update(part);
+    }
+    mac.verify_slice(&provided).map_err(|_| {
+        Error::NotAuthorized("Slack callback rejected: invalid payload signature".to_string())
+    })
+}
+
 pub async fn slack_app_callback_handler(
    authed: Option<ApiAuthed>,
    opt_tokened: OptTokened,
@@ -188,7 +261,30 @@ pub async fn slack_app_callback_handler(
                            let job_id = Uuid::parse_str(&parsed_value.job_id)?;
                            let flow_step_id = parsed_value.flow_step_id.as_deref();

-                            let slack_token = get_slack_token(&db, path, w_id).await?;
+                            // Authorize the request before any privileged read: the button
+                            // payload was minted by `send_slack_message` with an HMAC over
+                            // (w_id, job_id, path) keyed on the workspace key. Without a valid
+                            // signature an unauthenticated caller cannot reach the decryption
+                            // below for an arbitrary variable, even when SLACK_SIGNING_SECRET
+                            // is unset.
+                            verify_slack_payload(
+                                &db,
+                                w_id,
+                                &[parsed_value.job_id.as_bytes(), path.as_bytes()],
+                                parsed_value.signature.as_deref(),
+                            )
+                            .await?;
+
+                            // Map any lookup/decryption failure to a generic error: the
+                            // raw error echoes the probed `path`/`w_id` back, which would be
+                            // a cross-workspace existence oracle. Log the detail server-side.
+                            let slack_token =
+                                get_slack_token(&db, path, w_id).await.map_err(|e| {
+                                    tracing::warn!(
+                                        "Failed to resolve slack token for {w_id}/{path}: {e:#}"
+                                    );
+                                    Error::BadRequest("Invalid Slack callback request".to_string())
+                                })?;
                            let client = Client::new();
                            let container = payload.container.ok_or_else(|| {
                                Error::BadRequest("No container found.".to_string())
@@ -281,6 +377,7 @@ pub async fn request_slack_approval(

    send_slack_message(
        &client,
+        &db,
        slack_token.as_str(),
        channel_id.as_str(),
        &w_id,
@@ -334,12 +431,21 @@ async fn handle_submission(
    let resource_path = private_metadata.resource_path;
    let container: Container = private_metadata.container;
    let hide_cancel = private_metadata.hide_cancel;
+    let signature = private_metadata.signature;

    // If hide_cancel is true, we don't need to extract information from the private_metadata
    if hide_cancel.unwrap_or(false) && action == "cancel" {
        return Ok(());
    }

+    let w_id = extract_w_id_from_resume_url(&resume_url)?;
+    // Authorize the submission BEFORE taking any action. `resource_path` comes from the
+    // (client-held) modal metadata and is not covered by the resume-URL signature, so a
+    // tampered/unsigned submission must be rejected up front — otherwise it could still drive
+    // the resume/cancel and reach the decryption below with a swapped path. Require the
+    // workspace-keyed HMAC minted when the modal was built.
+    verify_slack_payload(&db, w_id, &[resource_path.as_bytes()], signature.as_deref()).await?;
+
    // Use the common handler to process the resume/cancel action
    handle_resume_action(
        authed,
@@ -351,8 +457,12 @@ async fn handle_submission(
    )
    .await?;

-    let w_id = extract_w_id_from_resume_url(&resume_url)?;
-    let slack_token = get_slack_token(&db, &resource_path, w_id).await?;
+    let slack_token = get_slack_token(&db, &resource_path, w_id)
+        .await
+        .map_err(|e| {
+            tracing::warn!("Failed to resolve slack token for {w_id}/{resource_path}: {e:#}");
+            Error::BadRequest("Invalid Slack callback request".to_string())
+        })?;
    update_original_slack_message(action, slack_token, container).await?;
    Ok(())
 }
@@ -780,6 +890,7 @@ async fn get_slack_token(db: &DB, slack_resource_path: &str, w_id: &str) -> anyh
 // Sends a Slack message with a button that opens a modal
 async fn send_slack_message(
    client: &Client,
+    db: &DB,
    bot_token: &str,
    channel_id: &str,
    w_id: &str,
@@ -827,6 +938,18 @@ async fn send_slack_message(
        value["cancel_button_text"] = serde_json::json!(cancel_button_text);
    }

+    // Authenticate the button payload so the unauthenticated callback cannot be driven into
+    // decrypting an arbitrary variable: bind (w_id, job_id, path) with the workspace key.
+    // `job_id` is signed over its string form to match how it is parsed back on callback.
+    let signature = sign_slack_payload(
+        db,
+        w_id,
+        &[job_id.to_string().as_bytes(), resource_path.as_bytes()],
+    )
+    .await
+    .map_err(|e| Box::new(e) as Box<dyn std::error::Error>)?;
+    value["signature"] = serde_json::json!(signature);
+
    let payload = serde_json::json!({
        "channel": channel_id,
        "text": "A flow has been suspended. Please approve or reject the flow.",
@@ -893,6 +1016,12 @@ async fn get_modal_blocks(
    resume_button_text: Option<&str>,
    cancel_button_text: Option<&str>,
 ) -> Result<axum::Json<serde_json::Value>, Error> {
+    // Bind the resource_path embedded in the modal's private_metadata to the workspace key so
+    // it cannot be tampered with on the way back in `handle_submission`. Computed before `db`
+    // is moved into `get_approval_form_details`.
+    let private_metadata_signature =
+        sign_slack_payload(&db, w_id, &[resource_path.as_bytes()]).await?;
+
    let approval_details = crate::approvals::get_approval_form_details(
        db,
        w_id,
@@ -947,6 +1076,7 @@ async fn get_modal_blocks(
        container,
        resume_button_text,
        cancel_button_text,
+        &private_metadata_signature,
    )))
 }

@@ -959,6 +1089,7 @@ fn construct_payload(
    container: Container,
    resume_button_text: Option<&str>,
    cancel_button_text: Option<&str>,
+    signature: &str,
 ) -> serde_json::Value {
    let mut view = serde_json::json!({
        "type": "modal",
@@ -973,7 +1104,7 @@ fn construct_payload(
            "type": "plain_text",
            "text": resume_button_text.unwrap_or("Resume Workflow")
        },
-        "private_metadata": serde_json::json!({ "resume_url": resume_url, "resource_path": resource_path, "container": container, "hide_cancel": hide_cancel }).to_string(),
+        "private_metadata": serde_json::json!({ "resume_url": resume_url, "resource_path": resource_path, "container": container, "hide_cancel": hide_cancel, "signature": signature }).to_string(),
    });

    if !hide_cancel {
@@ -125,10 +125,18 @@ pub fn is_none_or_false(val: &Option<bool>) -> bool {
    }
 }

-/// Returns the keys to strip from trigger/schedule serialization when the
-/// source workspace is a fork. Stripping these keys avoids propagating
-/// fork-local operational state (enabled flag, runtime listener identifiers)
-/// back to the parent workspace through the git-sync round-trip.
+/// A fork's git-sync export rewrites each trigger's `mode` (and each schedule's
+/// `enabled`) to the *parent* workspace's value, instead of emitting the fork's
+/// own (clone-disabled / locally-toggled) state. This keeps the fork's synced
+/// file byte-identical to the parent on the operational-state field, so a
+/// normal-git PR merge has nothing to resolve — no dropped `mode:` line, no
+/// flipped parent trigger. Fork-only paths (absent from the parent) keep the
+/// fork's own value: there's no parent state to defer to, so the trigger lands
+/// with whatever the fork creator set. The write half of the same rule lives in
+/// `windmill-trigger::handler::workspace_is_fork`.
+///
+/// Maps trigger `path` → parent `mode` (as the lowercase enum text that matches
+/// `TriggerMode`'s serde representation). Empty when not a fork.
 #[cfg(any(
    feature = "http_trigger",
    feature = "websocket",
@@ -148,20 +156,78 @@ pub fn is_none_or_false(val: &Option<bool>) -> bool {
        feature = "private"
    )
 ))]
-fn fork_trigger_ignore_keys(is_fork: bool) -> Option<Vec<&'static str>> {
-    if is_fork {
-        Some(vec!["mode", "enabled"])
-    } else {
-        None
-    }
+async fn fork_parent_trigger_modes(
+    db: &DB,
+    table_name: &str,
+    parent_workspace_id: Option<&str>,
+) -> Result<HashMap<String, String>> {
+    let Some(parent) = parent_workspace_id else {
+        return Ok(HashMap::new());
+    };
+    // Read the parent's rows on the non-RLS pool (like `workspace_is_fork`): the
+    // substitution must be complete regardless of the exporter's folder perms,
+    // otherwise a parent path the exporter can't read would fall back to the
+    // fork's own value and silently re-introduce the divergence we're fixing.
+    // No leak: only values for paths the fork already has (it's a clone) are used.
+    // SAFETY: `table_name` is a compile-time `TriggerCrud::TABLE_NAME` constant.
+    let rows: Vec<(String, String)> = sqlx::query_as(&format!(
+        "SELECT path, mode::text FROM {} WHERE workspace_id = $1",
+        table_name
+    ))
+    .bind(parent)
+    .fetch_all(db)
+    .await?;
+    Ok(rows.into_iter().collect())
 }

-fn fork_schedule_ignore_keys(is_fork: bool) -> Option<Vec<&'static str>> {
-    if is_fork {
-        Some(vec!["enabled"])
-    } else {
-        None
-    }
+/// Build the `{ "mode": <parent value> }` override for a single trigger, or
+/// `None` (keep the fork's own value) when the path is fork-only.
+#[cfg(any(
+    feature = "http_trigger",
+    feature = "websocket",
+    feature = "postgres_trigger",
+    feature = "mqtt_trigger",
+    feature = "native_trigger",
+    all(
+        feature = "enterprise",
+        any(
+            feature = "kafka",
+            feature = "sqs_trigger",
+            feature = "gcp_trigger",
+            feature = "azure_trigger",
+            feature = "nats",
+            feature = "smtp",
+        ),
+        feature = "private"
+    )
+))]
+fn trigger_mode_override(
+    parent_modes: &HashMap<String, String>,
+    path: &str,
+) -> Option<serde_json::Map<String, Value>> {
+    parent_modes.get(path).map(|mode| {
+        let mut o = serde_json::Map::new();
+        o.insert("mode".to_string(), Value::String(mode.clone()));
+        o
+    })
+}
+
+/// Schedule analog of [`fork_parent_trigger_modes`]: maps schedule `path` →
+/// parent `enabled`. Empty when not a fork.
+async fn fork_parent_schedule_enabled(
+    db: &DB,
+    parent_workspace_id: Option<&str>,
+) -> Result<HashMap<String, bool>> {
+    let Some(parent) = parent_workspace_id else {
+        return Ok(HashMap::new());
+    };
+    // Non-RLS pool, same rationale as `fork_parent_trigger_modes`.
+    let rows: Vec<(String, bool)> =
+        sqlx::query_as("SELECT path, enabled FROM schedule WHERE workspace_id = $1")
+            .bind(parent)
+            .fetch_all(db)
+            .await?;
+    Ok(rows.into_iter().collect())
 }

 enum ArchiveImpl {
@@ -260,6 +326,25 @@ pub fn to_string_without_metadata<T>(
    extra_perms: ExtraPermsBehavior,
    ignore_keys: Option<Vec<&str>>,
 ) -> Result<String>
+where
+    T: ?Sized + Serialize,
+{
+    to_string_without_metadata_inner(value, extra_perms, ignore_keys, None)
+}
+
+/// Like [`to_string_without_metadata`] but additionally lets the caller
+/// override top-level keys after stripping. Used for fork trigger/schedule
+/// exports, where `mode`/`enabled` is rewritten to the *parent* workspace's
+/// value so the fork's synced file is byte-identical to the parent on those
+/// fields — a clean 3-way git merge instead of a dropped line. See the write
+/// half of the rule in `windmill-trigger::handler::workspace_is_fork`.
+#[inline]
+pub fn to_string_without_metadata_inner<T>(
+    value: &T,
+    extra_perms: ExtraPermsBehavior,
+    ignore_keys: Option<Vec<&str>>,
+    overrides: Option<&serde_json::Map<String, Value>>,
+) -> Result<String>
 where
    T: ?Sized + Serialize,
 {
@@ -324,6 +409,12 @@ where
                obj.remove("default_permissioned_as");
            }

+            if let Some(overrides) = overrides {
+                for (k, v) in overrides {
+                    obj.insert(k.clone(), v.clone());
+                }
+            }
+
            serde_json::to_string_pretty(&obj).ok()
        })
        .flatten()
@@ -504,18 +595,18 @@ pub(crate) async fn tarball_workspace(

    let mut tx = user_db.begin(&authed).await?;

-    // Source-of-truth check for fork-ness: the workspace's parent_workspace_id
-    // column. The wm-fork-* prefix is a creation-time naming convention that
-    // could in principle drift (rename, manual SQL); the column is the
-    // contract that matches what the conflict-warning gates read.
-    let is_fork: bool = sqlx::query_scalar!(
-        "SELECT parent_workspace_id IS NOT NULL FROM workspace WHERE id = $1",
-        &w_id
+    // Source-of-truth for fork-ness: the workspace's parent_workspace_id column.
+    // The wm-fork-* prefix is a creation-time naming convention that could in
+    // principle drift (rename, manual SQL); the column is the contract that
+    // matches what the conflict-warning gates read. The id is also the workspace
+    // whose trigger `mode` / schedule `enabled` a fork export defers to.
+    let parent_workspace_id: Option<String> = sqlx::query_scalar::<_, Option<String>>(
+        "SELECT parent_workspace_id FROM workspace WHERE id = $1",
    )
+    .bind(&w_id)
    .fetch_optional(&mut *tx)
    .await?
-    .flatten()
-    .unwrap_or(false);
+    .flatten();

    let tmp_dir = TempDir::new_in(&*WINDMILL_DIR)?;

@@ -799,12 +890,21 @@ pub(crate) async fn tarball_workspace(
        .fetch_all(&mut *tx)
        .await?;

-        let schedule_ignore_keys = fork_schedule_ignore_keys(is_fork);
+        // For a fork, defer each schedule's `enabled` to the parent so the
+        // synced file matches the parent and the merge doesn't flip it.
+        let parent_enabled =
+            fork_parent_schedule_enabled(&db, parent_workspace_id.as_deref()).await?;
        for schedule in schedules {
-            let app_str = &to_string_without_metadata(
+            let enabled_override = parent_enabled.get(&schedule.path).map(|enabled| {
+                let mut o = serde_json::Map::new();
+                o.insert("enabled".to_string(), Value::Bool(*enabled));
+                o
+            });
+            let app_str = &to_string_without_metadata_inner(
                &schedule,
                ExtraPermsBehavior::Drop,
-                schedule_ignore_keys.clone(),
+                None,
+                enabled_override.as_ref(),
            )
            .unwrap();
            archive
@@ -814,38 +914,25 @@ pub(crate) async fn tarball_workspace(
    }

    if include_triggers.unwrap_or(false) {
-        #[cfg(any(
-            feature = "http_trigger",
-            feature = "websocket",
-            feature = "postgres_trigger",
-            feature = "mqtt_trigger",
-            feature = "native_trigger",
-            all(
-                feature = "enterprise",
-                any(
-                    feature = "kafka",
-                    feature = "sqs_trigger",
-                    feature = "gcp_trigger",
-                    feature = "azure_trigger",
-                    feature = "nats",
-                    feature = "smtp",
-                ),
-                feature = "private"
-            )
-        ))]
-        let trigger_ignore_keys = fork_trigger_ignore_keys(is_fork);
-
        #[cfg(feature = "http_trigger")]
        {
            use crate::triggers::http::HttpTrigger;
            let handler = HttpTrigger;
            let http_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <HttpTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in http_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -862,12 +949,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::websocket::WebsocketTrigger;
            let handler = WebsocketTrigger;
            let websocket_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <WebsocketTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in websocket_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -884,12 +979,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::kafka::KafkaTrigger;
            let handler = KafkaTrigger;
            let kafka_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <KafkaTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in kafka_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -906,12 +1009,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::sqs::SqsTrigger;
            let handler = SqsTrigger;
            let sqs_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <SqsTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in sqs_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -928,12 +1039,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::gcp::GcpTrigger;
            let handler = GcpTrigger;
            let gcp_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <GcpTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in gcp_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -950,12 +1069,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::azure::AzureTrigger;
            let handler = AzureTrigger;
            let azure_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <AzureTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in azure_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -972,12 +1099,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::nats::NatsTrigger;
            let handler = NatsTrigger;
            let nats_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <NatsTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in nats_triggers {
-                let trigger_str: &String = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str: &String = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -994,12 +1129,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::postgres::PostgresTrigger;
            let handler = PostgresTrigger;
            let postgres_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <PostgresTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in postgres_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -1016,12 +1159,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::mqtt::MqttTrigger;
            let handler = MqttTrigger;
            let mqtt_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <MqttTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in mqtt_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -1038,12 +1189,20 @@ pub(crate) async fn tarball_workspace(
            use crate::triggers::email::EmailTrigger;
            let handler = EmailTrigger;
            let email_triggers = handler.list_triggers(&mut *tx, &w_id, None).await?;
+            let parent_modes = fork_parent_trigger_modes(
+                &db,
+                <EmailTrigger as TriggerCrud>::TABLE_NAME,
+                parent_workspace_id.as_deref(),
+            )
+            .await?;

            for trigger in email_triggers {
-                let trigger_str = &to_string_without_metadata(
+                let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
+                let trigger_str = &to_string_without_metadata_inner(
                    &trigger,
                    ExtraPermsBehavior::Drop,
-                    trigger_ignore_keys.clone(),
+                    None,
+                    mode_override.as_ref(),
                )
                .unwrap();
                archive
@@ -1065,10 +1224,12 @@ pub(crate) async fn tarball_workspace(
                    list_native_triggers(&mut *tx, &w_id, service_name, None, None, None, None)
                        .await?;

-                let mut native_ignore_keys = vec!["webhook_token_hash"];
-                if let Some(ref extra) = trigger_ignore_keys {
-                    native_ignore_keys.extend_from_slice(extra);
-                }
+                // Native triggers (Nextcloud, Google Drive, GitHub) are never
+                // cloned into a fork — a fork only has one if its owner created
+                // it there, so it's always "fork-only" and keeps its own mode.
+                // No parent-value substitution applies; we only strip the
+                // webhook token hash.
+                let native_ignore_keys = vec!["webhook_token_hash"];

                for trigger in native_triggers {
                    let trigger_str = &to_string_without_metadata(
@@ -1359,3 +1520,67 @@ pub(crate) async fn tarball_workspace(
    ];
    Ok((headers, body))
 }
+
+#[cfg(test)]
+mod fork_export_tests {
+    use super::*;
+    use serde_json::json;
+
+    /// A fork export rewrites `mode` to the parent's value: the serialized file
+    /// carries the parent's state (`enabled`), not the fork's clone-disabled DB
+    /// value — so a normal-git merge sees no change on that line.
+    #[test]
+    fn override_substitutes_parent_mode() {
+        let fork_trigger = json!({
+            "path": "f/triggers/x",
+            "script_path": "f/scripts/x",
+            "mode": "disabled", // fork's local (clone-disabled) state
+            "is_flow": false,
+        });
+        let mut overrides = serde_json::Map::new();
+        overrides.insert("mode".to_string(), Value::String("enabled".to_string()));
+
+        let out = to_string_without_metadata_inner(
+            &fork_trigger,
+            ExtraPermsBehavior::Drop,
+            None,
+            Some(&overrides),
+        )
+        .unwrap();
+        let parsed: Value = serde_json::from_str(&out).unwrap();
+
+        assert_eq!(parsed["mode"], json!("enabled"), "parent mode substituted");
+        // `path` is in the metadata strip list, so it should be removed.
+        assert!(parsed.get("path").is_none());
+    }
+
+    /// A fork-only trigger (no parent counterpart, so no override) keeps the
+    /// fork creator's chosen state.
+    #[test]
+    fn no_override_keeps_fork_value() {
+        let fork_only = json!({ "mode": "enabled", "script_path": "f/scripts/x" });
+        let out =
+            to_string_without_metadata_inner(&fork_only, ExtraPermsBehavior::Drop, None, None)
+                .unwrap();
+        let parsed: Value = serde_json::from_str(&out).unwrap();
+        assert_eq!(parsed["mode"], json!("enabled"));
+    }
+
+    /// `trigger_mode_override` builds an override only when the parent has the
+    /// path; fork-only paths return `None` (keep the fork's own value).
+    #[cfg(feature = "http_trigger")]
+    #[test]
+    fn trigger_mode_override_defers_to_parent_or_self() {
+        let mut parent_modes = HashMap::new();
+        parent_modes.insert("f/triggers/shared".to_string(), "enabled".to_string());
+
+        let shared = trigger_mode_override(&parent_modes, "f/triggers/shared");
+        assert_eq!(
+            shared.as_ref().and_then(|o| o.get("mode")),
+            Some(&Value::String("enabled".to_string())),
+        );
+
+        // Fork-only path: no parent entry → no override → keep fork's own value.
+        assert!(trigger_mode_override(&parent_modes, "f/triggers/fork_only").is_none());
+    }
+}
@@ -18,6 +18,31 @@ use crate::{
    DB,
 };

+/// Whether `label` denotes a user-created token rather than a system token
+/// (`session`, `ephemeral*`, `debugger-token`, `mcp-oauth-*`). System-token
+/// labels are load-bearing — session cleanup, super_admin propagation, expiry
+/// notifications and username overrides all key off them — so they must not be
+/// user-editable. `None` (no label) is treated as a user token.
+///
+/// This is the canonical copy. When updating it, also update its mirrors:
+/// - the `update_token_label` editability guard (SQL `WHERE`) in
+///   windmill-api-users/src/users.rs
+/// - `isUserToken` in frontend/src/lib/components/settings/TokensTable.svelte
+pub fn is_user_token(label: Option<&str>) -> bool {
+    match label {
+        None => true,
+        Some(l) => {
+            // `ephemeral` is matched case-insensitively to agree exactly with the
+            // frontend mirror (`label.toLowerCase().startsWith('ephemeral')`) and
+            // the SQL `lower(label) NOT LIKE 'ephemeral%'` guard.
+            l != "session"
+                && !l.to_lowercase().starts_with("ephemeral")
+                && l != "debugger-token"
+                && !l.starts_with("mcp-oauth-")
+        }
+    }
+}
+
 /// Hash a raw token using SHA-256 (hex-encoded, 64 chars).
 /// Used to store and look up tokens without keeping plaintext in the DB.
 pub fn hash_token(token: &str) -> String {
@@ -641,3 +666,35 @@ pub mod aws {
        Ok(assume_role_with_web_identity_fluent_builder)
    }
 }
+
+#[cfg(test)]
+mod tests {
+    use super::is_user_token;
+
+    #[test]
+    fn user_tokens_are_editable() {
+        assert!(is_user_token(None)); // no label
+        assert!(is_user_token(Some("")));
+        assert!(is_user_token(Some("my-ci-token")));
+        assert!(is_user_token(Some("webhook-foo"))); // username-override prefix, not a system kind here
+    }
+
+    #[test]
+    fn system_tokens_are_not_editable() {
+        assert!(!is_user_token(Some("session")));
+        assert!(!is_user_token(Some("ephemeral-script")));
+        assert!(!is_user_token(Some("ephemeral-webhook-x")));
+        assert!(!is_user_token(Some("Ephemeral lsp token")));
+        assert!(!is_user_token(Some("debugger-token")));
+        assert!(!is_user_token(Some("mcp-oauth-client")));
+    }
+
+    #[test]
+    fn ephemeral_match_is_case_insensitive() {
+        // Must agree with the frontend mirror (`toLowerCase().startsWith('ephemeral')`)
+        // so a token can't be relabeled to a casing the backend allows but the UI hides.
+        assert!(!is_user_token(Some("Ephemeral-test")));
+        assert!(!is_user_token(Some("ePhemeral-test")));
+        assert!(!is_user_token(Some("EPHEMERAL-test")));
+    }
+}
@@ -58,6 +58,11 @@ pub const NSJAIL_TMPFS_SIZE_MB_SETTING: &str = "nsjail_tmpfs_size_mb";
 pub const NSJAIL_TMP_BACKING_SETTING: &str = "nsjail_tmp_backing";
 pub const NSJAIL_TMP_BACKING_DISK: &str = "disk";
 pub const NSJAIL_TMP_BACKING_TMPFS: &str = "tmpfs";
+pub const SANDBOX_IMAGE_MAX_SIZE_MB_SETTING: &str = "sandbox_image_max_size_mb";
+pub const SANDBOX_IMAGE_CACHE_MAX_MB_SETTING: &str = "sandbox_image_cache_max_mb";
+pub const SANDBOX_IMAGE_PULL_POLICY_SETTING: &str = "sandbox_image_pull_policy";
+pub const SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING: &str = "sandbox_image_default_registry";
+pub const SANDBOX_REGISTRY_AUTH_SETTING: &str = "sandbox_registry_auth";
 pub const OBJECT_STORE_CONFIG_SETTING: &str = "object_store_cache_config";
 pub const HUB_API_SECRET_SETTING: &str = "hub_api_secret";

@@ -976,6 +976,7 @@ const SENSITIVE_SETTINGS: &[&str] = &[
    "ruby_repos",
    "powershell_repo_pat",
    "workspace_registries",
+    "sandbox_registry_auth",
 ];

 /// Object-valued settings that contain sensitive sub-fields.
@@ -35,6 +35,11 @@ pub struct LogContext {
    pub uri: Option<String>,
    pub trace_id: Option<String>,

+    // Inbound W3C `traceparent` captured at enqueue (reserved `_wm_traceparent`
+    // arg). Carried here so the worker's OTLP span and the script's injected
+    // TRACEPARENT env can relocate into the originating distributed trace.
+    pub inbound_traceparent: Option<String>,
+
    // Auth (windmill-api-auth/src/auth.rs)
    pub email: Option<String>,
    pub username: Option<String>,
@@ -2,6 +2,8 @@ use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};

 use crate::error::Error;

+pub const ALLOW_PRIVATE_MCP_SERVER_URLS_ENV: &str = "ALLOW_PRIVATE_MCP_SERVER_URLS";
+
 /// Why a URL failed SSRF validation.
 ///
 /// The distinction matters for callers that gate private endpoints behind a
@@ -116,6 +118,49 @@ pub async fn validate_url_for_ssrf(url: &str) -> Result<(), SsrfValidationError>
    Ok(())
 }

+pub fn allow_private_mcp_server_urls() -> bool {
+    std::env::var(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV)
+        .ok()
+        .is_some_and(|v| v == "true" || v == "1")
+}
+
+pub async fn validate_mcp_server_url(url: &str) -> Result<(), SsrfValidationError> {
+    let parsed =
+        url::Url::parse(url).map_err(|e| SsrfValidationError::InvalidUrl(e.to_string()))?;
+
+    match parsed.scheme() {
+        "http" | "https" => {}
+        scheme => return Err(SsrfValidationError::DisallowedScheme(scheme.to_string())),
+    }
+
+    parsed.host_str().ok_or(SsrfValidationError::MissingHost)?;
+
+    if allow_private_mcp_server_urls() {
+        return Ok(());
+    }
+
+    validate_url_for_ssrf(url).await
+}
+
+pub async fn validate_mcp_server_url_for_bad_request(url: &str, label: &str) -> Result<(), Error> {
+    validate_mcp_server_url(url).await.map_err(|e| {
+        Error::BadRequest(format!(
+            "{label} is not allowed: {}",
+            mcp_ssrf_error_message(&e)
+        ))
+    })
+}
+
+pub fn mcp_ssrf_error_message(e: &SsrfValidationError) -> String {
+    match e {
+        SsrfValidationError::Private { .. } => format!(
+            "{e}. If you need to use private/internal MCP server URLs, \
+             set the {ALLOW_PRIVATE_MCP_SERVER_URLS_ENV}=true environment variable"
+        ),
+        _ => e.to_string(),
+    }
+}
+
 fn is_private_ip(ip: &IpAddr) -> bool {
    match ip {
        IpAddr::V4(ipv4) => is_private_ipv4(ipv4),
@@ -152,6 +197,32 @@ fn is_private_ipv6(ip: &Ipv6Addr) -> bool {
 mod tests {
    use super::*;

+    static TEST_ENV_LOCK: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(());
+
+    struct PrivateMcpServerUrlsEnvGuard {
+        previous: Option<String>,
+    }
+
+    impl PrivateMcpServerUrlsEnvGuard {
+        fn set(value: Option<&str>) -> Self {
+            let previous = std::env::var(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV).ok();
+            match value {
+                Some(value) => std::env::set_var(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV, value),
+                None => std::env::remove_var(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV),
+            }
+            Self { previous }
+        }
+    }
+
+    impl Drop for PrivateMcpServerUrlsEnvGuard {
+        fn drop(&mut self) {
+            match &self.previous {
+                Some(value) => std::env::set_var(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV, value),
+                None => std::env::remove_var(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV),
+            }
+        }
+    }
+
    #[test]
    fn test_private_ipv4() {
        assert!(is_private_ipv4(&"127.0.0.1".parse().unwrap()));
@@ -227,4 +298,66 @@ mod tests {
            Err(SsrfValidationError::Private { resolved: false })
        ));
    }
+
+    #[tokio::test]
+    async fn validate_mcp_server_url_blocks_private_by_default() {
+        let _lock = TEST_ENV_LOCK.lock().await;
+        let _guard = PrivateMcpServerUrlsEnvGuard::set(None);
+
+        assert!(matches!(
+            validate_mcp_server_url("http://127.0.0.1/foo").await,
+            Err(SsrfValidationError::Private { resolved: false })
+        ));
+    }
+
+    #[tokio::test]
+    async fn validate_mcp_server_url_allows_private_when_env_is_enabled() {
+        let _lock = TEST_ENV_LOCK.lock().await;
+        let _guard = PrivateMcpServerUrlsEnvGuard::set(Some("true"));
+
+        assert!(validate_mcp_server_url("http://127.0.0.1/foo")
+            .await
+            .is_ok());
+    }
+
+    #[tokio::test]
+    async fn validate_mcp_server_url_allows_private_when_env_is_one() {
+        let _lock = TEST_ENV_LOCK.lock().await;
+        let _guard = PrivateMcpServerUrlsEnvGuard::set(Some("1"));
+
+        assert!(validate_mcp_server_url("http://10.0.0.1/foo").await.is_ok());
+    }
+
+    #[tokio::test]
+    async fn validate_mcp_server_url_keeps_syntax_guards_when_private_urls_are_allowed() {
+        let _lock = TEST_ENV_LOCK.lock().await;
+        let _guard = PrivateMcpServerUrlsEnvGuard::set(Some("true"));
+
+        assert!(matches!(
+            validate_mcp_server_url("localhost:11434/v1").await,
+            Err(SsrfValidationError::DisallowedScheme(_))
+        ));
+        assert!(matches!(
+            validate_mcp_server_url("file:///tmp/socket").await,
+            Err(SsrfValidationError::DisallowedScheme(_))
+        ));
+    }
+
+    #[tokio::test]
+    async fn private_mcp_error_message_includes_env_hint_only_for_private_urls() {
+        let _lock = TEST_ENV_LOCK.lock().await;
+        let _guard = PrivateMcpServerUrlsEnvGuard::set(None);
+
+        let private_error = validate_mcp_server_url("http://127.0.0.1/foo")
+            .await
+            .unwrap_err();
+        assert!(
+            mcp_ssrf_error_message(&private_error).contains("ALLOW_PRIVATE_MCP_SERVER_URLS=true")
+        );
+
+        let invalid_error = validate_mcp_server_url("localhost:11434/v1")
+            .await
+            .unwrap_err();
+        assert!(!mcp_ssrf_error_message(&invalid_error).contains(ALLOW_PRIVATE_MCP_SERVER_URLS_ENV));
+    }
 }
@@ -859,6 +859,37 @@ pub struct BashAnnotations {
    pub sandbox: bool,
 }

+impl BashAnnotations {
+    /// If the script declares `# sandbox <image>` (an image ref after the sandbox
+    /// annotation), returns that image ref. This selects the daemonless, sandboxed
+    /// container runtime: extract the image's rootfs and run it inside the job's
+    /// nsjail sandbox.
+    ///
+    /// A bare `# sandbox` (no image argument) returns `None` and keeps the plain
+    /// nsjail-sandboxed-bash behavior (the `sandbox` boolean modifier). `# docker`
+    /// is unaffected and keeps the legacy v1 (dind/daemon) path.
+    pub fn sandbox_image(code: &str) -> Option<String> {
+        for line in code.lines() {
+            let line = line.trim();
+            if line.is_empty() {
+                continue;
+            }
+            // Mirror the annotation parser: stop at the first non-comment line.
+            if !line.starts_with('#') {
+                break;
+            }
+            let mut tokens = line[1..].split_whitespace();
+            if tokens.next() == Some("sandbox") {
+                // `# sandbox <image>` -> container; bare `# sandbox` -> nsjail bash.
+                if let Some(image) = tokens.next() {
+                    return Some(image.to_string());
+                }
+            }
+        }
+        None
+    }
+}
+
 #[derive(Debug, Clone, Copy, PartialEq)]
 pub enum SqlResultCollectionStrategy {
    LastStatementAllRows,
@@ -2224,6 +2255,34 @@ mod tests {
    use super::*;
    use std::collections::HashMap;

+    #[test]
+    fn test_bash_sandbox_image_annotation() {
+        // `# sandbox <image>` selects the container runtime and returns the image.
+        assert_eq!(
+            BashAnnotations::sandbox_image("# sandbox alpine:latest\necho hi"),
+            Some("alpine:latest".to_string())
+        );
+        // Extra whitespace and a leading non-spaced `#` still work.
+        assert_eq!(
+            BashAnnotations::sandbox_image("#sandbox   python:3.12-slim\n"),
+            Some("python:3.12-slim".to_string())
+        );
+        // A bare `# sandbox` (no image) keeps the nsjail-bash modifier -> None.
+        assert_eq!(BashAnnotations::sandbox_image("# sandbox\necho hi"), None);
+        // `sandbox` must be its own token, not a prefix.
+        assert_eq!(BashAnnotations::sandbox_image("# sandboxed foo"), None);
+        // Stops at the first non-comment line (image declared too late is ignored).
+        assert_eq!(
+            BashAnnotations::sandbox_image("echo hi\n# sandbox alpine"),
+            None
+        );
+        // `# docker` is a different annotation -> not a sandbox image.
+        assert_eq!(
+            BashAnnotations::sandbox_image("# docker alpine\necho hi"),
+            None
+        );
+    }
+
    #[test]
    fn test_mixed_tags() {
        let input = vec![
@@ -43,9 +43,14 @@ impl McpClient {
        // The resource URL is author-controlled and we send a (potentially
        // secret) bearer token to it, so it must be validated against SSRF
        // before we connect (e.g. cloud metadata endpoints, internal services).
-        windmill_common::ssrf::validate_url_for_ssrf(&resource.url)
+        windmill_common::ssrf::validate_mcp_server_url(&resource.url)
            .await
-            .map_err(|e| anyhow::anyhow!("MCP server URL is not allowed: {}", e))?;
+            .map_err(|e| {
+                anyhow::anyhow!(
+                    "MCP server URL is not allowed: {}",
+                    windmill_common::ssrf::mcp_ssrf_error_message(&e)
+                )
+            })?;

        // Build custom reqwest client with headers if provided
        let mut headers = HeaderMap::new();
@@ -230,6 +235,33 @@ impl McpClient {
 mod tests {
    use super::*;

+    struct PrivateMcpServerUrlsEnvGuard {
+        previous: Option<String>,
+    }
+
+    impl PrivateMcpServerUrlsEnvGuard {
+        fn unset() -> Self {
+            let previous =
+                std::env::var(windmill_common::ssrf::ALLOW_PRIVATE_MCP_SERVER_URLS_ENV).ok();
+            std::env::remove_var(windmill_common::ssrf::ALLOW_PRIVATE_MCP_SERVER_URLS_ENV);
+            Self { previous }
+        }
+    }
+
+    impl Drop for PrivateMcpServerUrlsEnvGuard {
+        fn drop(&mut self) {
+            match &self.previous {
+                Some(value) => std::env::set_var(
+                    windmill_common::ssrf::ALLOW_PRIVATE_MCP_SERVER_URLS_ENV,
+                    value,
+                ),
+                None => {
+                    std::env::remove_var(windmill_common::ssrf::ALLOW_PRIVATE_MCP_SERVER_URLS_ENV)
+                }
+            }
+        }
+    }
+
    /// Regression test: `from_resource` must refuse to connect to a URL that
    /// targets a private/internal address (here the AWS
    /// instance-metadata endpoint), so a resource author cannot use the MCP
@@ -237,6 +269,8 @@ mod tests {
    /// before any connection attempt, so this fails fast without network access.
    #[tokio::test]
    async fn from_resource_rejects_ssrf_url() {
+        let _guard = PrivateMcpServerUrlsEnvGuard::unset();
+
        let resource = McpResource {
            name: "evil".to_string(),
            url: "http://169.254.169.254".to_string(),
@@ -17,7 +17,7 @@ use windmill_common::db::DB;
 use windmill_common::error;
 use windmill_common::variables::{build_crypt, decrypt, encrypt};

-use crate::oauth::AuthorizationManager;
+use crate::oauth::{no_redirect_http_client, AuthorizationManager};

 /// MCP client credentials returned by [`get_or_refresh_mcp_client`].
 pub struct McpClientCredentials {
@@ -77,7 +77,14 @@ async fn register_client(
    redirect_uri: &str,
    client_name: &str,
 ) -> Result<DcrResponse, error::Error> {
-    let client = reqwest::Client::new();
+    windmill_common::ssrf::validate_mcp_server_url_for_bad_request(
+        registration_endpoint,
+        "MCP server registration endpoint URL",
+    )
+    .await?;
+
+    let client = no_redirect_http_client()
+        .map_err(|e| error::Error::BadRequest(format!("Failed to build DCR client: {e}")))?;
    let request = DcrRequest {
        client_name: client_name.to_string(),
        redirect_uris: vec![redirect_uri.to_string()],
@@ -121,6 +128,12 @@ pub async fn get_or_refresh_mcp_client(
    let base_url = (**windmill_common::BASE_URL.load()).clone();
    let redirect_uri = format!("{}/api/mcp/oauth/callback", base_url);

+    windmill_common::ssrf::validate_mcp_server_url_for_bad_request(
+        mcp_server_url,
+        "MCP server URL",
+    )
+    .await?;
+
    let cached_client: Option<McpOAuthClient> =
        sqlx::query_as("SELECT mcp_server_url, client_id, client_secret, client_secret_expires_at, token_endpoint FROM mcp_oauth_client WHERE mcp_server_url = $1")
            .bind(mcp_server_url)
@@ -131,6 +144,11 @@ pub async fn get_or_refresh_mcp_client(
    if let Some(client) = cached_client {
        if !client.is_expired() {
            tracing::debug!("Using cached MCP client for {}", mcp_server_url);
+            windmill_common::ssrf::validate_mcp_server_url_for_bad_request(
+                &client.token_endpoint,
+                "MCP server token endpoint URL",
+            )
+            .await?;
            let decrypted_secret = if let Some(ref encrypted_secret) = client.client_secret {
                Some(decrypt_client_secret(db, encrypted_secret).await?)
            } else {
@@ -145,17 +163,27 @@ pub async fn get_or_refresh_mcp_client(
        tracing::debug!("Cached MCP client expired, re-registering");
    }

-    windmill_common::ssrf::validate_url_for_ssrf(mcp_server_url).await?;
-
-    let manager = AuthorizationManager::new(mcp_server_url)
+    let mut manager = AuthorizationManager::new(mcp_server_url)
        .await
        .map_err(|e| error::Error::BadRequest(format!("Failed to create auth manager: {e}")))?;
+    let discovery_client = no_redirect_http_client().map_err(|e| {
+        error::Error::BadRequest(format!("Failed to build MCP OAuth discovery client: {e}"))
+    })?;
+    manager
+        .with_client(discovery_client)
+        .map_err(|e| error::Error::BadRequest(format!("Failed to configure auth manager: {e}")))?;

    let metadata = manager
        .discover_metadata()
        .await
        .map_err(|e| error::Error::BadRequest(format!("OAuth discovery failed: {e}")))?;

+    windmill_common::ssrf::validate_mcp_server_url_for_bad_request(
+        &metadata.token_endpoint,
+        "MCP server token endpoint URL",
+    )
+    .await?;
+
    let supports_dynamic_registration = metadata.registration_endpoint.is_some();

    let (client_id, client_secret, expires_at) = if supports_dynamic_registration {
@@ -38,11 +38,70 @@ pub mod client_registration;
 pub mod oauth {
    //! Re-exports of rmcp auth and oauth2 types for MCP OAuth implementations

+    use std::time::Duration;
+
    pub use rmcp::transport::auth::AuthorizationManager;

+    const DEFAULT_OAUTH_HTTP_TIMEOUT: Duration = Duration::from_secs(30);
+
+    pub fn no_redirect_http_client() -> Result<reqwest::Client, reqwest::Error> {
+        no_redirect_http_client_with_timeout(DEFAULT_OAUTH_HTTP_TIMEOUT)
+    }
+
+    pub(crate) fn no_redirect_http_client_with_timeout(
+        timeout: Duration,
+    ) -> Result<reqwest::Client, reqwest::Error> {
+        reqwest::Client::builder()
+            .timeout(timeout)
+            .redirect(reqwest::redirect::Policy::none())
+            .build()
+    }
+
    // Re-export oauth2 types needed for MCP OAuth flow
    pub use oauth2::{
        basic::BasicClient, AuthUrl, ClientId, ClientSecret, CsrfToken, PkceCodeChallenge,
        RedirectUrl, Scope, TokenUrl,
    };
+
+    #[cfg(test)]
+    mod tests {
+        use super::*;
+        use std::{
+            io::Read,
+            net::TcpListener,
+            thread,
+            time::{Duration, Instant},
+        };
+
+        #[tokio::test]
+        async fn no_redirect_http_client_times_out_stalled_responses() {
+            let listener = TcpListener::bind("127.0.0.1:0").unwrap();
+            let addr = listener.local_addr().unwrap();
+
+            let handle = thread::spawn(move || {
+                if let Ok((mut stream, _)) = listener.accept() {
+                    let _ = stream.set_read_timeout(Some(Duration::from_millis(200)));
+                    let mut buffer = [0; 1024];
+                    let _ = stream.read(&mut buffer);
+                    thread::sleep(Duration::from_millis(300));
+                }
+            });
+
+            let client = no_redirect_http_client_with_timeout(Duration::from_millis(50)).unwrap();
+            let started = Instant::now();
+            let err = client
+                .get(format!("http://{addr}/stall"))
+                .send()
+                .await
+                .expect_err("stalled response should time out");
+
+            assert!(err.is_timeout(), "expected timeout error, got: {err}");
+            assert!(
+                started.elapsed() < Duration::from_secs(2),
+                "stalled request should fail promptly"
+            );
+
+            handle.join().unwrap();
+        }
+    }
 }
@@ -2,9 +2,10 @@ use std::sync::Arc;

 use axum::{extract::Path, routing::get, Extension, Json, Router};
 use http::Method;
+use windmill_api_auth::ApiAuthed;
 use windmill_common::{error::JsonResult, DB};

-use crate::{get_workspace_integration, External, ServiceName};
+use crate::{get_workspace_integration, require_native_integration_use, External, ServiceName};

 use super::{GitHub, GithubApiRepoResponse, GithubRepoEntry};

@@ -12,10 +13,12 @@ const PER_PAGE: usize = 100;
 const MAX_PAGES: usize = 10;

 async fn list_repos(
+    authed: ApiAuthed,
    Extension(handler): Extension<Arc<GitHub>>,
    Extension(db): Extension<DB>,
    Path(workspace_id): Path<String>,
 ) -> JsonResult<Vec<GithubRepoEntry>> {
+    require_native_integration_use(&authed)?;
    get_workspace_integration(&db, &workspace_id, ServiceName::Github).await?;

    let mut all_entries = Vec::new();
@@ -7,9 +7,10 @@ use axum::{
 };
 use http::Method;
 use serde::{Deserialize, Serialize};
+use windmill_api_auth::ApiAuthed;
 use windmill_common::{error::JsonResult, DB};

-use crate::{get_workspace_integration, External, ServiceName};
+use crate::{get_workspace_integration, require_native_integration_use, External, ServiceName};

 use super::Google;

@@ -84,10 +85,12 @@ pub struct DriveFilesQuery {
 }

 async fn list_calendars(
+    authed: ApiAuthed,
    Extension(handler): Extension<Arc<Google>>,
    Extension(db): Extension<DB>,
    Path(workspace_id): Path<String>,
 ) -> JsonResult<Vec<GoogleCalendarEntry>> {
+    require_native_integration_use(&authed)?;
    get_workspace_integration(&db, &workspace_id, ServiceName::Google).await?;

    let url = format!(
@@ -113,11 +116,13 @@ async fn list_calendars(
 }

 async fn list_drive_files(
+    authed: ApiAuthed,
    Extension(handler): Extension<Arc<Google>>,
    Extension(db): Extension<DB>,
    Path(workspace_id): Path<String>,
    Query(query): Query<DriveFilesQuery>,
 ) -> JsonResult<GoogleDriveFilesResponse> {
+    require_native_integration_use(&authed)?;
    get_workspace_integration(&db, &workspace_id, ServiceName::Google).await?;

    let drive_query = if query.shared_with_me {
@@ -186,10 +191,12 @@ struct SharedDriveApiEntry {
 }

 async fn list_shared_drives(
+    authed: ApiAuthed,
    Extension(handler): Extension<Arc<Google>>,
    Extension(db): Extension<DB>,
    Path(workspace_id): Path<String>,
 ) -> JsonResult<Vec<SharedDriveEntry>> {
+    require_native_integration_use(&authed)?;
    get_workspace_integration(&db, &workspace_id, ServiceName::Google).await?;

    let url = format!(
@@ -1226,6 +1226,20 @@ pub async fn store_workspace_integration(
    Ok(())
 }

+/// Authorization gate for the integration *use* routes (calendar/drive/repo/event
+/// pickers). A workspace admin configures the integration, but any member who can
+/// create a native trigger needs the pickers to configure one. Operators are
+/// read-only and cannot create triggers, so they must not be able to drive the
+/// admin-configured integration's upstream API and enumerate its data.
+pub fn require_native_integration_use(authed: &ApiAuthed) -> Result<()> {
+    if authed.is_operator {
+        return Err(Error::NotAuthorized(
+            "Operators cannot use workspace integrations".to_string(),
+        ));
+    }
+    Ok(())
+}
+
 pub async fn get_workspace_integration<'c, E: sqlx::Executor<'c, Database = Postgres>>(
    db: E,
    workspace_id: &str,
@@ -7,17 +7,21 @@ use windmill_common::{
    DB,
 };

+use windmill_api_auth::ApiAuthed;
+
 use crate::{
    get_workspace_integration,
    nextcloud::{NextCloudEventType, OcsResponse},
-    External, ServiceName,
+    require_native_integration_use, External, ServiceName,
 };

 async fn list_available_events<T: External>(
+    authed: ApiAuthed,
    Extension(handler): Extension<Arc<T>>,
    Extension(db): Extension<DB>,
    Path(workspace_id): Path<String>,
 ) -> JsonResult<Vec<NextCloudEventType>> {
+    require_native_integration_use(&authed)?;
    let integration = get_workspace_integration(&db, &workspace_id, ServiceName::Nextcloud).await?;

    let base_url = integration
@@ -26,7 +26,6 @@ use windmill_common::error::{self, to_anyhow, Error};
 use windmill_common::more_serde::maybe_number_opt;
 use windmill_common::oauth2::*;
 use windmill_common::utils::now_from_db;
-use windmill_common::variables::{build_crypt, encrypt};
 use windmill_common::BASE_URL;

 pub type DB = sqlx::Pool<sqlx::Postgres>;
@@ -92,6 +91,12 @@ pub struct OAuthConfig {
    /// entry, `build_oauth_clients` registers a second client under that key.
    #[serde(skip_serializing_if = "Option::is_none")]
    pub sandbox: Option<OAuthSandboxOverride>,
+    /// Frontend-only metadata for per-instance OAuth providers (Snowflake,
+    /// ServiceNow, …) whose authorize/token URLs are derived from an
+    /// admin-entered instance name. Ignored by the backend, which only ever
+    /// sees the resulting concrete `connect_config`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub connect_config_template: Option<ConnectConfigTemplate>,
 }

 /// URL overrides for an OAuth provider's sandbox environment. Inherits
@@ -106,6 +111,43 @@ pub struct OAuthSandboxOverride {
    pub userinfo_url: Option<String>,
 }

+/// Frontend metadata for a per-instance OAuth provider. The instance-settings
+/// UI renders one generic instance-name input and substitutes `{instance}` into
+/// `auth_url`/`token_url` to build the per-client `connect_config`. Adding a new
+/// per-instance provider needs only a registry entry carrying this template —
+/// no frontend code change. The backend never reads it.
+#[derive(Clone, Debug, Serialize, Deserialize)]
+pub struct ConnectConfigTemplate {
+    /// Properly-cased provider name for the settings dropdown (e.g. "ServiceNow");
+    /// the UI falls back to a capitalized registry key when absent.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub display_name: Option<String>,
+    pub label: String,
+    pub placeholder: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub help_url: Option<String>,
+    pub auth_url: String,
+    pub token_url: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub req_body_auth: Option<bool>,
+    /// Key under `connect_config.extra_params` where the instance name is
+    /// stored (defaults to `instance`). Snowflake uses `account_identifier` for
+    /// backward compatibility with previously-saved configs.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub extra_params_key: Option<String>,
+    /// Optional host suffix stripped from the input before substitution (e.g.
+    /// `.service-now.com`), so the admin can paste a full host or a bare name.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub strip_suffix: Option<String>,
+    /// Maps OAuth-connected resource arg fields to value templates substituting
+    /// `{instance}` (e.g. ServiceNow's `instance_url` ->
+    /// `https://{instance}.service-now.com`). Applied by the resource-connect
+    /// flow so the created resource carries the instance-specific fields the
+    /// scripts need (ServiceNow's token response omits the host).
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub resource_mapping: Option<HashMap<String, String>>,
+}
+
 impl OAuthConfig {
    /// Returns a copy of this config with sandbox URL overrides applied and
    /// the nested `sandbox` field cleared. Returns `None` if no overrides are
@@ -482,11 +524,14 @@ pub struct OAuthAccountInfo {
    pub scopes: Option<Vec<String>>,
 }

-/// Refresh an OAuth token and update the database.
+/// Refresh an OAuth token and update the `account` row.
 /// Fetches the account from DB, then delegates to `refresh_token_for_account`.
+///
+/// Returns the freshly minted access token. Persisting it to the secret variable
+/// backing the resource is the caller's responsibility (it must route through the
+/// configured secret backend — see `store_oauth_token_value` in `windmill-store`).
 pub async fn refresh_token<'c>(
    mut tx: Transaction<'c, Postgres>,
-    path: &str,
    w_id: &str,
    id: i32,
    db: &DB,
@@ -506,7 +551,6 @@ pub async fn refresh_token<'c>(

    refresh_token_for_account(
        tx,
-        path,
        w_id,
        id,
        db,
@@ -519,9 +563,14 @@ pub async fn refresh_token<'c>(
 }

 /// Refresh an OAuth token given pre-fetched account info (no additional SELECT).
+///
+/// Exchanges the refresh token, updates the `account` row (`refresh_token`,
+/// `expires_at`, `refresh_error`) and returns the new access token. It does NOT
+/// persist the token to the secret variable — the caller must do that through the
+/// configured secret backend (`store_oauth_token_value`), otherwise an external
+/// secret backend would keep serving the stale connect-time token.
 pub async fn refresh_token_for_account<'c>(
    mut tx: Transaction<'c, Postgres>,
-    path: &str,
    w_id: &str,
    id: i32,
    db: &DB,
@@ -633,17 +682,6 @@ pub async fn refresh_token_for_account<'c>(
    tx.commit().await?;

    let token_str = token.access_token.to_string();
-    let mc = build_crypt(db, w_id).await?;
-    let encrypted_token = encrypt(&mc, token_str.as_str());
-
-    sqlx::query!(
-        "UPDATE variable SET value = $1 WHERE workspace_id = $2 AND path = $3",
-        encrypted_token,
-        w_id,
-        path
-    )
-    .execute(db)
-    .await?;

    tracing::info!(
        grant_type = %account.grant_type,
@@ -817,6 +855,7 @@ mod tests {
                token_url: Some("https://account-d.example.com/oauth/token".to_string()),
                userinfo_url: None,
            }),
+            connect_config_template: None,
        }
    }

@@ -5292,6 +5292,7 @@ async fn push_inner<'c, 'd>(
                        expr: skip_handler.stop_condition,
                        skip_if_stopped: true,
                        error_message: Some(skip_handler.stop_message),
+                        error_include_result: false,
                    }),
                    ..Default::default()
                });
@@ -14,3 +14,6 @@ pub mod resources;
 pub mod secret_backend_ext;
 pub mod var_resource_cache;
 pub mod variables;
+
+#[cfg(all(test, feature = "oauth2", feature = "private", feature = "enterprise"))]
+mod oauth_refresh_secret_backend_tests;
@@ -24,9 +24,8 @@ pub async fn _refresh_token<'c>(
    id: i32,
    db: &DB,
 ) -> error::Result<String> {
-    windmill_oauth::refresh_token(
+    let token = windmill_oauth::refresh_token(
        tx,
-        path,
        w_id,
        id,
        db,
@@ -34,5 +33,12 @@ pub async fn _refresh_token<'c>(
        &windmill_oauth::OAUTH_HTTP_CLIENT,
        include_str!("../../oauth_connect.json"),
    )
-    .await
+    .await?;
+
+    // Persist the refreshed token through the configured secret backend so an
+    // external backend (Vault / Azure KV / AWS Secrets Manager) is updated too,
+    // not just the in-DB variable mirror.
+    crate::secret_backend_ext::store_oauth_token_value(db, w_id, path, &token).await?;
+
+    Ok(token)
 }
@@ -0,0 +1,337 @@
+//! E2E regression tests for OAuth token refresh persistence through the
+//! configured secret backend.
+//!
+//! Regression for windmill#9471 / windmill-ee-private#607: the lazy on-fetch
+//! OAuth refresh used to persist the freshly minted token with a raw
+//! `UPDATE variable SET value = <db-encrypted>`, bypassing the secret-backend
+//! abstraction. With an external backend (AWS Secrets Manager / Azure Key
+//! Vault / Vault) reads resolve through the backend and ignore `variable.value`
+//! entirely, so the external store stayed frozen at its connect-time token and
+//! every read that did not itself trigger a mint served a stale/expired token.
+//!
+//! These tests exercise the persistence step (`store_oauth_token_value`) — the
+//! exact code path that was fixed — against both the database backend and an
+//! external (AWS Secrets Manager via LocalStack) backend, plus the self-healing
+//! reset on a failed persist. They are opt-in (they mutate the shared
+//! `global_settings.secret_backend` row and workspace/variable/account rows on a
+//! real DB) and skip unless `RUN_SECRET_BACKEND_E2E=1` is set.
+//!
+//! ## Run
+//!
+//! Database-backend case (needs a migrated DB). Note `RUN_SECRET_BACKEND_E2E=1`
+//! is required or every test skips:
+//!
+//! ```bash
+//! RUN_SECRET_BACKEND_E2E=1 \
+//! DATABASE_URL=postgres://postgres:changeme@127.0.0.1:5432/windmill \
+//!   cargo test -p windmill-store --features private,enterprise,oauth2 \
+//!   oauth_refresh_secret_backend_tests -- --nocapture --test-threads=1
+//! ```
+//!
+//! External-backend cases additionally need LocalStack `secretsmanager` and
+//! `RUN_AWS_SM_TESTS=1`:
+//!
+//! ```bash
+//! docker run -d -e SERVICES=secretsmanager localstack/localstack:3.8
+//! RUN_SECRET_BACKEND_E2E=1 RUN_AWS_SM_TESTS=1 AWS_SM_ENDPOINT=http://<localstack-ip>:4566 \
+//! DATABASE_URL=postgres://postgres:changeme@127.0.0.1:5432/windmill \
+//!   cargo test -p windmill-store --features private,enterprise,oauth2 \
+//!   oauth_refresh_secret_backend_tests -- --nocapture --test-threads=1
+//! ```
+
+use crate::secret_backend_ext::{get_secret_value, store_oauth_token_value, store_secret_value};
+use sqlx::postgres::PgPoolOptions;
+use sqlx::{Pool, Postgres};
+
+// global_settings holds a single `secret_backend` row shared across tests;
+// serialize the test bodies so concurrent runs don't clobber each other's
+// configured backend. (Also run with --test-threads=1 for good measure.)
+static SERIAL: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(());
+
+fn env_flag(name: &str) -> bool {
+    std::env::var(name)
+        .map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
+        .unwrap_or(false)
+}
+
+// Opt-in gate so the suite never runs (and mutates shared DB state) as part of a
+// normal `cargo test` invocation.
+fn run_e2e() -> bool {
+    env_flag("RUN_SECRET_BACKEND_E2E")
+}
+
+fn run_aws_sm() -> bool {
+    env_flag("RUN_AWS_SM_TESTS")
+}
+
+/// Restore the default (database) backend so we don't leave the instance
+/// pointed at a test backend for any concurrently-running suite.
+async fn reset_backend(db: &Pool<Postgres>) {
+    set_backend(db, serde_json::json!({ "type": "Database" })).await;
+}
+
+fn aws_sm_endpoint() -> String {
+    std::env::var("AWS_SM_ENDPOINT").unwrap_or_else(|_| "http://localhost:4566".to_string())
+}
+
+async fn db() -> Pool<Postgres> {
+    let url = std::env::var("DATABASE_URL")
+        .expect("DATABASE_URL must point at a migrated windmill database");
+    PgPoolOptions::new()
+        .max_connections(5)
+        .connect(&url)
+        .await
+        .expect("connect to DATABASE_URL")
+}
+
+/// Fresh workspace + key + clean variable/account rows for `w_id`.
+async fn setup_workspace(db: &Pool<Postgres>, w_id: &str) {
+    sqlx::query("DELETE FROM variable WHERE workspace_id = $1")
+        .bind(w_id)
+        .execute(db)
+        .await
+        .unwrap();
+    sqlx::query("DELETE FROM account WHERE workspace_id = $1")
+        .bind(w_id)
+        .execute(db)
+        .await
+        .unwrap();
+    sqlx::query("DELETE FROM workspace_key WHERE workspace_id = $1")
+        .bind(w_id)
+        .execute(db)
+        .await
+        .unwrap();
+    sqlx::query("DELETE FROM workspace WHERE id = $1")
+        .bind(w_id)
+        .execute(db)
+        .await
+        .unwrap();
+
+    sqlx::query("INSERT INTO workspace (id, name, owner) VALUES ($1, $1, 'admin@windmill.dev')")
+        .bind(w_id)
+        .execute(db)
+        .await
+        .unwrap();
+    sqlx::query(
+        "INSERT INTO workspace_key (workspace_id, kind, key) VALUES ($1, 'cloud', 'e2ekey')",
+    )
+    .bind(w_id)
+    .execute(db)
+    .await
+    .unwrap();
+}
+
+async fn set_backend(db: &Pool<Postgres>, config: serde_json::Value) {
+    sqlx::query(
+        "INSERT INTO global_settings (name, value) VALUES ('secret_backend', $1) \
+         ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
+    )
+    .bind(config)
+    .execute(db)
+    .await
+    .unwrap();
+}
+
+fn aws_sm_config(endpoint: &str) -> serde_json::Value {
+    serde_json::json!({
+        "type": "AwsSecretsManager",
+        "region": "us-east-1",
+        "access_key_id": "test",
+        "secret_access_key": "test",
+        "endpoint_url": endpoint,
+        "prefix": "windmill-e2e/"
+    })
+}
+
+/// Simulate `Connect`: store the initial token through the backend and create
+/// the linked secret variable + account (expired, with a refresh token).
+async fn simulate_connect(db: &Pool<Postgres>, w_id: &str, path: &str, initial_token: &str) -> i32 {
+    let stored = store_secret_value(db, w_id, path, initial_token)
+        .await
+        .expect("store initial token");
+
+    let account_id: i32 = sqlx::query_scalar(
+        "INSERT INTO account (workspace_id, expires_at, refresh_token, client, grant_type) \
+         VALUES ($1, now() - interval '1 hour', 'rt_dummy', 'gdrive', 'authorization_code') \
+         RETURNING id",
+    )
+    .bind(w_id)
+    .fetch_one(db)
+    .await
+    .unwrap();
+
+    sqlx::query(
+        "INSERT INTO variable (workspace_id, path, value, is_secret, is_oauth, account, expires_at) \
+         VALUES ($1, $2, $3, true, true, $4, now() - interval '1 hour')",
+    )
+    .bind(w_id)
+    .bind(path)
+    .bind(&stored)
+    .bind(account_id)
+    .execute(db)
+    .await
+    .unwrap();
+
+    account_id
+}
+
+async fn variable_value(db: &Pool<Postgres>, w_id: &str, path: &str) -> String {
+    sqlx::query_scalar("SELECT value FROM variable WHERE workspace_id = $1 AND path = $2")
+        .bind(w_id)
+        .bind(path)
+        .fetch_one(db)
+        .await
+        .unwrap()
+}
+
+async fn account_expires_in_past(db: &Pool<Postgres>, w_id: &str, account_id: i32) -> bool {
+    sqlx::query_scalar("SELECT expires_at < now() FROM account WHERE workspace_id = $1 AND id = $2")
+        .bind(w_id)
+        .bind(account_id)
+        .fetch_one(db)
+        .await
+        .unwrap()
+}
+
+/// Database backend (the "without external storage" case): refresh must
+/// re-encrypt the new token into `variable.value`; reads serve the new token.
+#[tokio::test]
+async fn database_backend_persists_refreshed_token() {
+    if !run_e2e() {
+        println!(
+            "Skipping database_backend_persists_refreshed_token: set RUN_SECRET_BACKEND_E2E=1"
+        );
+        return;
+    }
+    let _guard = SERIAL.lock().await;
+    let db = db().await;
+    let w_id = "wm_e2e_db";
+    let path = "f/google/gdrive";
+
+    set_backend(&db, serde_json::json!({ "type": "Database" })).await;
+    setup_workspace(&db, w_id).await;
+    let _ = simulate_connect(&db, w_id, path, "OLD_TOKEN").await;
+
+    // Connect-time token is served.
+    let v = variable_value(&db, w_id, path).await;
+    assert_eq!(
+        get_secret_value(&db, w_id, path, &v).await.unwrap(),
+        "OLD_TOKEN"
+    );
+
+    // Refresh persists the new token.
+    store_oauth_token_value(&db, w_id, path, "NEW_TOKEN")
+        .await
+        .unwrap();
+
+    let v = variable_value(&db, w_id, path).await;
+    assert_eq!(
+        get_secret_value(&db, w_id, path, &v).await.unwrap(),
+        "NEW_TOKEN",
+        "database backend should serve the refreshed token"
+    );
+    println!("  ✓ database backend serves refreshed token");
+    reset_backend(&db).await;
+}
+
+/// External backend (the "with external storage" case): refresh must write
+/// the new token to AWS Secrets Manager. Before the fix the external store
+/// stayed frozen and reads served the stale connect-time token.
+#[tokio::test]
+async fn external_backend_persists_refreshed_token() {
+    if !run_e2e() || !run_aws_sm() {
+        println!("Skipping external_backend_persists_refreshed_token: set RUN_SECRET_BACKEND_E2E=1 and RUN_AWS_SM_TESTS=1");
+        return;
+    }
+    let _guard = SERIAL.lock().await;
+    let db = db().await;
+    let w_id = "wm_e2e_awssm";
+    let path = "f/google/gsheets";
+
+    set_backend(&db, aws_sm_config(&aws_sm_endpoint())).await;
+    setup_workspace(&db, w_id).await;
+    let _ = simulate_connect(&db, w_id, path, "OLD_TOKEN").await;
+
+    // Connect-time token is served from the external store.
+    let marker = variable_value(&db, w_id, path).await;
+    assert!(
+        marker.starts_with("$aws_sm:"),
+        "external backend should store a marker in variable.value, got {marker}"
+    );
+    assert_eq!(
+        get_secret_value(&db, w_id, path, &marker).await.unwrap(),
+        "OLD_TOKEN"
+    );
+
+    // Demonstrate the original bug shape: a raw DB write to variable.value is
+    // futile because reads resolve through the backend and ignore it.
+    sqlx::query(
+        "UPDATE variable SET value = 'ignored_db_blob' WHERE workspace_id = $1 AND path = $2",
+    )
+    .bind(w_id)
+    .bind(path)
+    .execute(&db)
+    .await
+    .unwrap();
+    assert_eq!(
+        get_secret_value(&db, w_id, path, "ignored_db_blob")
+            .await
+            .unwrap(),
+        "OLD_TOKEN",
+        "reads ignore variable.value for external backends — a raw UPDATE can't refresh the served token"
+    );
+
+    // The fix: persist through the backend.
+    store_oauth_token_value(&db, w_id, path, "NEW_TOKEN")
+        .await
+        .unwrap();
+
+    let marker = variable_value(&db, w_id, path).await;
+    assert_eq!(
+        get_secret_value(&db, w_id, path, &marker).await.unwrap(),
+        "NEW_TOKEN",
+        "external backend should serve the refreshed token written back to AWS SM"
+    );
+    println!("  ✓ external (AWS SM) backend serves refreshed token written back to the store");
+    reset_backend(&db).await;
+}
+
+/// If persisting the refreshed token fails (e.g. transient external-backend
+/// error) after the account was committed fresh, the account expiry must be
+/// reset to the past so the next fetch retries instead of serving a stale
+/// token for the whole token lifetime.
+#[tokio::test]
+async fn failed_persist_resets_account_expiry() {
+    if !run_e2e() || !run_aws_sm() {
+        println!("Skipping failed_persist_resets_account_expiry: set RUN_SECRET_BACKEND_E2E=1 and RUN_AWS_SM_TESTS=1");
+        return;
+    }
+    let _guard = SERIAL.lock().await;
+    let db = db().await;
+    let w_id = "wm_e2e_selfheal";
+    let path = "f/google/gdrive";
+
+    // Working backend first to seed the variable + a *fresh* account.
+    set_backend(&db, aws_sm_config(&aws_sm_endpoint())).await;
+    setup_workspace(&db, w_id).await;
+    let account_id = simulate_connect(&db, w_id, path, "OLD_TOKEN").await;
+    sqlx::query("UPDATE account SET expires_at = now() + interval '1 hour' WHERE workspace_id = $1 AND id = $2")
+        .bind(w_id)
+        .bind(account_id)
+        .execute(&db)
+        .await
+        .unwrap();
+    assert!(!account_expires_in_past(&db, w_id, account_id).await);
+
+    // Point the backend at an unreachable endpoint so the persist fails.
+    set_backend(&db, aws_sm_config("http://127.0.0.1:1")).await;
+
+    let res = store_oauth_token_value(&db, w_id, path, "NEW_TOKEN").await;
+    assert!(res.is_err(), "persist to unreachable backend should fail");
+    assert!(
+        account_expires_in_past(&db, w_id, account_id).await,
+        "a failed persist must reset account.expires_at to the past so refresh retries"
+    );
+    println!("  ✓ failed persist reset account expiry (self-healing)");
+    reset_backend(&db).await;
+}
@@ -26,7 +26,10 @@ use windmill_common::{
 #[cfg(all(feature = "private", feature = "enterprise"))]
 use windmill_common::{
    global_settings::{load_value_from_global_settings, SECRET_BACKEND_SETTING},
-    secret_backend::{AwsSecretsManagerBackend, AwsSecretsManagerSettings, AzureKeyVaultBackend, AzureKeyVaultSettings, SecretBackendConfig, VaultBackend, VaultSettings},
+    secret_backend::{
+        AwsSecretsManagerBackend, AwsSecretsManagerSettings, AzureKeyVaultBackend,
+        AzureKeyVaultSettings, SecretBackendConfig, VaultBackend, VaultSettings,
+    },
 };

 #[cfg(all(feature = "private", feature = "enterprise"))]
@@ -225,7 +228,12 @@ pub async fn is_vault_backend_configured(db: &DB) -> Result<bool> {
        None => SecretBackendConfig::default(),
    };

-    Ok(matches!(config, SecretBackendConfig::HashiCorpVault(_) | SecretBackendConfig::AzureKeyVault(_) | SecretBackendConfig::AwsSecretsManager(_)))
+    Ok(matches!(
+        config,
+        SecretBackendConfig::HashiCorpVault(_)
+            | SecretBackendConfig::AzureKeyVault(_)
+            | SecretBackendConfig::AwsSecretsManager(_)
+    ))
 }

 /// Get a secret value using the configured backend
@@ -252,12 +260,8 @@ pub async fn get_secret_value(
            // Fetch from Vault directly
            backend.get_secret(workspace_id, path).await
        }
-        "azure_key_vault" => {
-            backend.get_secret(workspace_id, path).await
-        }
-        "aws_secrets_manager" => {
-            backend.get_secret(workspace_id, path).await
-        }
+        "azure_key_vault" => backend.get_secret(workspace_id, path).await,
+        "aws_secrets_manager" => backend.get_secret(workspace_id, path).await,
        _ => Err(Error::internal_err(format!(
            "Unknown backend: {}",
            backend.backend_name()
@@ -303,6 +307,93 @@ pub async fn store_secret_value(
    }
 }

+/// Persist a freshly minted OAuth access token to the secret variable backing
+/// a resource, routing through the configured secret backend.
+///
+/// This is the write counterpart of the lazy on-fetch OAuth refresh: it stores
+/// the token via [`store_secret_value`] (which writes to the external backend —
+/// AWS Secrets Manager / Azure Key Vault / Vault — when one is configured, or
+/// encrypts for the database backend) and updates `variable.value` with the
+/// returned value (the encrypted blob for the DB backend, or a `$...:` marker
+/// for an external backend). Using a raw `UPDATE variable SET value = <encrypted>`
+/// here instead would leave the external store frozen at its connect-time token
+/// while reads (which resolve through the backend) keep serving the stale value.
+///
+/// The caller has already committed the `account` row as fresh (advanced
+/// `expires_at`) by the time we get here. If persisting the token fails — most
+/// likely a transient error talking to an external backend — that would leave
+/// the account marked fresh while the served secret is stale, so the on-fetch
+/// refresh gate (`now() > expires_at`) would skip refresh and keep serving the
+/// stale token for the whole token lifetime. To avoid that we reset `expires_at`
+/// to the past (and record `refresh_error`) on failure — looking the account up
+/// via `variable.account` — so the very next fetch retries the refresh instead.
+///
+/// Authorization contract: this performs NO access control. It writes the
+/// caller-supplied token into the secret variable at `path` and may mutate the
+/// linked `account` row, so callers MUST have already authorized the operation
+/// against `workspace_id`/`path` (the OAuth refresh adapters only run after the
+/// read path has resolved and gated the variable). It is therefore kept
+/// `pub(crate)` and intended solely for the in-crate refresh adapters.
+#[cfg(feature = "oauth2")]
+pub(crate) async fn store_oauth_token_value(
+    db: &DB,
+    workspace_id: &str,
+    path: &str,
+    token: &str,
+) -> Result<()> {
+    let persist = async {
+        let value = store_secret_value(db, workspace_id, path, token).await?;
+        sqlx::query("UPDATE variable SET value = $1 WHERE workspace_id = $2 AND path = $3")
+            .bind(value)
+            .bind(workspace_id)
+            .bind(path)
+            .execute(db)
+            .await?;
+        Ok::<(), Error>(())
+    }
+    .await;
+
+    if let Err(e) = persist {
+        // Mark the account expired again so the next fetch re-runs the refresh
+        // instead of serving the now-stale token until it naturally expires. The
+        // account id is the one linked from the variable being refreshed.
+        let account_id: Option<i32> = sqlx::query_scalar::<_, Option<i32>>(
+            "SELECT account FROM variable WHERE workspace_id = $1 AND path = $2",
+        )
+        .bind(workspace_id)
+        .bind(path)
+        .fetch_optional(db)
+        .await
+        .ok()
+        .flatten()
+        .flatten();
+
+        if let Some(account_id) = account_id {
+            if let Err(reset_err) = sqlx::query(
+                "UPDATE account SET expires_at = now() - interval '1 minute', refresh_error = $1 \
+                 WHERE workspace_id = $2 AND id = $3",
+            )
+            .bind(format!(
+                "OAuth token was refreshed but persisting it to the secret backend failed: {e}"
+            ))
+            .bind(workspace_id)
+            .bind(account_id)
+            .execute(db)
+            .await
+            {
+                tracing::error!(
+                    workspace_id = %workspace_id,
+                    account_id = %account_id,
+                    "failed to reset account expiry after token persistence error: {reset_err}"
+                );
+            }
+        }
+        return Err(e);
+    }
+
+    Ok(())
+}
+
 /// Delete a secret from the configured backend (if using Vault)
 ///
 /// For database backend: no-op (DB delete is handled separately)
@@ -32,6 +32,27 @@ use std::sync::Arc;
 use windmill_audit::{audit_oss::audit_log, ActionKind};
 use windmill_git_sync::handle_deployment_metadata;

+/// True when the workspace is a fork (`parent_workspace_id IS NOT NULL`).
+///
+/// Operational state (`mode`) belongs to the parent workspace: a git-sync /
+/// merge / clone / UI-create write into a fork must never set it. On create we
+/// force `disabled` so a fork trigger can't compete with the parent's listener;
+/// on update we preserve the fork's existing value. `setmode` is the intended
+/// explicit mutator of a fork's mode (and carries its own conflict warning) —
+/// runtime error handling may still auto-disable an errored trigger, which is
+/// orthogonal to this rule. This is the write half whose read half lives in
+/// `workspaces_export.rs` (parent-value substitution on fork export), and it is
+/// the single authority shared by both the git-sync round-trip and the in-app
+/// compare-workspaces merge.
+async fn workspace_is_fork(db: &DB, workspace_id: &str) -> Result<bool> {
+    let is_fork: Option<bool> =
+        sqlx::query_scalar("SELECT parent_workspace_id IS NOT NULL FROM workspace WHERE id = $1")
+            .bind(workspace_id)
+            .fetch_optional(db)
+            .await?;
+    Ok(is_fork.unwrap_or(false))
+}
+
 #[async_trait]
 pub trait TriggerCrud: Send + Sync + 'static {
    type Trigger: Serialize
@@ -440,6 +461,13 @@ async fn create_trigger<T: TriggerCrud>(

    let mut tx = user_db.begin(&authed).await?;

+    // Writing into a fork never sets operational state: force `disabled` so a
+    // cloned / synced / merged / UI-created trigger can't compete with the
+    // parent's listener. The fork owner re-enables locally via `setmode`.
+    if workspace_is_fork(&db, &workspace_id).await? {
+        new_trigger.base.set_mode(TriggerMode::Disabled);
+    }
+
    let new_path = new_trigger.base.path.clone();
    let labels = new_trigger.base.labels.clone();

@@ -591,11 +619,16 @@ async fn update_trigger<T: TriggerCrud>(

    let mut tx = user_db.begin(&authed).await?;

-    // When the request omits `mode`/`enabled`, preserve the existing DB value
-    // instead of falling back to the BaseTriggerData default (Enabled). This
-    // keeps fork→parent git-sync round-trips from flipping the parent's
-    // operational state — see fork_trigger_ignore_keys in workspaces_export.rs.
-    if edit_trigger.base.is_mode_unspecified() {
+    // Preserve the existing DB `mode` instead of writing the incoming value
+    // when either:
+    //   * the target is a fork — a fork's operational state is fork-local and
+    //     is never set through a git-sync/merge write (only via `setmode`); or
+    //   * the request omits `mode`/`enabled` (legacy clients / YAML round-trip),
+    //     where falling back to the BaseTriggerData default (Enabled) would flip
+    //     the parent on a fork→parent merge.
+    // Read half of the rule: parent-value substitution on fork export in
+    // workspaces_export.rs.
+    if workspace_is_fork(&db, &workspace_id).await? || edit_trigger.base.is_mode_unspecified() {
        let existing_mode: Option<TriggerMode> = sqlx::query_scalar(&format!(
            "SELECT mode FROM {} WHERE workspace_id = $1 AND path = $2",
            T::TABLE_NAME
@@ -315,6 +315,11 @@ pub struct StopAfterIf {
    pub expr: String,
    pub skip_if_stopped: bool,
    pub error_message: Option<String>,
+    /// When stopping with an error (`error_message` set), embed the stopping
+    /// step's own result inside the raised error object (as `error.result`)
+    /// instead of discarding it. The top-level result stays `{ "error": .. }`.
+    #[serde(default, skip_serializing_if = "is_false")]
+    pub error_include_result: bool,
 }

 #[derive(Deserialize, Serialize, Debug, Clone, Default, PartialEq)]
@@ -557,6 +557,13 @@ pub struct OnBehalfOf {

 pub const ENTRYPOINT_OVERRIDE: &str = "_ENTRYPOINT_OVERRIDE";

+/// Reserved job-arg key holding the inbound W3C `traceparent` captured from the
+/// request that enqueued the job (run endpoints). It rides the `args` jsonb like
+/// [`ENTRYPOINT_OVERRIDE`]; normal scripts never see it because args are bound by
+/// declared parameter name. Read back at root-job completion to link the job's
+/// OTLP span to the originating distributed trace (EE/OTel only).
+pub const WM_TRACEPARENT: &str = "_wm_traceparent";
+
 /// The entrypoint override (`_ENTRYPOINT_OVERRIDE` job arg ->
 /// `v2_job.script_entrypoint_override`) is interpolated verbatim into
 /// generated worker wrappers in a code position (e.g. the NativeTS
@@ -0,0 +1,103 @@
+name: "docker v2 run"
+
+mode: ONCE
+hostname: "container"
+log_level: ERROR
+time_limit: {TIMEOUT}
+
+disable_rl: true
+
+cwd: {WORKDIR}
+
+clone_newnet: false
+clone_newuser: {CLONE_NEWUSER}
+
+skip_setsid: true
+keep_caps: false
+# keep_env forwards nsjail's OWN process env (only windmill-trusted keys: reserved
+# vars + proxy) to the child. The image's attacker-controlled Env is delivered via
+# the envar directives below — NEVER nsjail's process env, so a hostile image cannot
+# set LD_PRELOAD/LD_LIBRARY_PATH/LD_AUDIT on the nsjail binary itself.
+keep_env: true
+mount_proc: true
+
+# Image Env (+ PATH/HOME fallbacks), proto-escaped. Applied to the child only.
+{ENVARS}
+
+# Map uid/gid 0 inside the jail to the (single) worker user outside. The image's
+# rootfs is extracted as the worker user, so a root process inside the container
+# owns the rootfs and runs like a normal "root in container" — without any subuid
+# range. Multi-uid images are a later enhancement (newuidmap range).
+uidmap {
+    inside_id: "0"
+    outside_id: ""
+    count: 1
+}
+gidmap {
+    inside_id: "0"
+    outside_id: ""
+    count: 1
+}
+
+# The image's root filesystem, bound one top-level entry at a time. Binding the
+# whole rootfs at "/" trips nsjail's read-only remount of its base root in a
+# rootless userns ("mount(... MS_REMOUNT|MS_BIND|MS_RDONLY): Operation not
+# permitted"); per-entry binds sit as rw submounts under nsjail's own tmpfs root
+# and avoid it. Generated from the extracted rootfs.
+{ROOTFS_MOUNTS}
+
+# Pseudo-filesystems the image expects. /tmp honors the same instance settings as
+# every other nsjail job (nsjail_tmp_backing tmpfs/disk, nsjail_tmpfs_size_mb);
+# /dev gets the standard nodes; /proc comes from mount_proc (the jail's own pid ns).
+{TMP_MOUNT_BLOCK}
+
+mount {
+    src: "/dev/null"
+    dst: "/dev/null"
+    is_bind: true
+    rw: true
+}
+
+mount {
+    src: "/dev/zero"
+    dst: "/dev/zero"
+    is_bind: true
+    rw: true
+}
+
+mount {
+    src: "/dev/random"
+    dst: "/dev/random"
+    is_bind: true
+}
+
+mount {
+    src: "/dev/urandom"
+    dst: "/dev/urandom"
+    is_bind: true
+}
+
+# Host DNS config layered over the image's /etc so name resolution works on the
+# job's network (mandatory:false: some minimal images have no /etc files to shadow).
+mount {
+    src: "/etc/resolv.conf"
+    dst: "/etc/resolv.conf"
+    is_bind: true
+    mandatory: false
+}
+
+mount {
+    src: "/etc/hosts"
+    dst: "/etc/hosts"
+    is_bind: true
+    mandatory: false
+}
+
+# `# volume` mounts (and the same-worker /tmp/shared folder). Placed after the
+# rootfs binds and the tmpfs /tmp so a volume target overrides any colliding image
+# path and isn't shadowed by the tmpfs. Empty when there are no volumes.
+{SHARED_MOUNT}
+
+iface_no_lo: true
+
+#{DEV}
@@ -40,9 +40,9 @@ use crate::handle_child::run_future_with_polling_update_job_poller;

 use crate::{
    common::{
-        build_args_map, build_command_with_isolation, get_reserved_variables, read_file,
-        read_file_content, resolve_nsjail_timeout, resolve_nsjail_tmp_mount_block, start_child_process,
-        OccupancyMetrics, DEV_CONF_NSJAIL,
+        build_args_map, build_command_with_isolation, get_reserved_variables, raw_to_string,
+        read_file, read_file_content, resolve_nsjail_timeout, resolve_nsjail_tmp_mount_block,
+        start_child_process, OccupancyMetrics, DEV_CONF_NSJAIL,
    },
    get_proxy_envs_for_lang,
    handle_child::handle_child,
@@ -57,14 +57,6 @@ lazy_static::lazy_static! {
    pub static ref ANSI_ESCAPE_RE: Regex = Regex::new(r"\x1b\[[0-9;]*m").unwrap();
 }

-fn raw_to_string(x: &str) -> String {
-    match serde_json::from_str::<serde_json::Value>(x) {
-        Ok(serde_json::Value::String(x)) => x,
-        Ok(x) => serde_json::to_string(&x).unwrap_or_else(|_| String::new()),
-        _ => String::new(),
-    }
-}
-
 #[tracing::instrument(level = "trace", skip_all)]
 pub async fn handle_bash_job(
    mem_peak: &mut i32,
@@ -84,6 +76,28 @@ pub async fn handle_bash_job(
 ) -> Result<Box<RawValue>, Error> {
    let annotation = windmill_common::worker::BashAnnotations::parse(&content);

+    // `# sandbox <image>` selects the daemonless, nsjail-sandboxed container runtime
+    // (extract the image's rootfs + run it inside the job's sandbox). A bare
+    // `# sandbox` keeps the plain nsjail-bash modifier; `# docker` keeps v1 (dind).
+    if let Some(image) = windmill_common::worker::BashAnnotations::sandbox_image(content) {
+        return crate::docker_v2::handle_docker_v2_job(
+            &image,
+            mem_peak,
+            canceled_by,
+            job,
+            conn,
+            client,
+            parent_runnable_path,
+            content,
+            job_dir,
+            shared_mount,
+            base_internal_url,
+            worker_name,
+            occupancy_metrics,
+        )
+        .await;
+    }
+
    // Check if sandbox annotation is used but nsjail is not available
    if annotation.sandbox && NSJAIL_AVAILABLE.is_none() {
        return Err(Error::ExecutionErr(
@@ -68,6 +68,16 @@ mount {
 #[cfg(not(debug_assertions))]
 pub const DEV_CONF_NSJAIL: &str = "";

+/// Turn a JSON value into the string a shell/CLI arg should receive: a JSON string
+/// becomes its inner value, anything else is re-serialized compactly.
+pub(crate) fn raw_to_string(x: &str) -> String {
+    match serde_json::from_str::<serde_json::Value>(x) {
+        Ok(serde_json::Value::String(x)) => x,
+        Ok(x) => serde_json::to_string(&x).unwrap_or_else(|_| String::new()),
+        _ => String::new(),
+    }
+}
+
 pub async fn build_args_map<'a>(
    job: &'a MiniPulledJob,
    client: &AuthedClient,
@@ -56,9 +56,13 @@ const DOTNET_ROOT_DEFAULT: &str = "C:\\Program Files\\dotnet";
 #[cfg(unix)]
 const DOTNET_ROOT_DEFAULT: &str = "/usr/share/dotnet";

+#[cfg(feature = "csharp")]
+const DOTNET_TARGET_FRAMEWORK_DEFAULT: &str = "net9.0";
+
 #[cfg(feature = "csharp")]
 lazy_static::lazy_static! {
    static ref DOTNET_ROOT: String = std::env::var("DOTNET_ROOT").unwrap_or_else(|_| DOTNET_ROOT_DEFAULT.to_string());
+    static ref DOTNET_TARGET_FRAMEWORK: String = std::env::var("DOTNET_TARGET_FRAMEWORK").unwrap_or_else(|_| DOTNET_TARGET_FRAMEWORK_DEFAULT.to_string());
 }

 #[cfg(feature = "csharp")]
@@ -212,6 +216,7 @@ fn gen_cs_proj(
        )
    };

+    let target_framework = DOTNET_TARGET_FRAMEWORK.as_str();
    write_file(
        job_dir,
        "Main.csproj",
@@ -219,7 +224,7 @@ fn gen_cs_proj(
            r#"<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
-    <TargetFramework>net9.0</TargetFramework>
+    <TargetFramework>{target_framework}</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <StartupObject>WindmillScriptCSharpInternal.Wrapper</StartupObject>
    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
@@ -510,9 +515,10 @@ pub async fn handle_csharp_job(

    let ws_suffix = crate::workspace_registry_cache_suffix(&job.workspace_id).await;
    let mut hash = calculate_hash(&format!(
-        "{}{}",
+        "{}{}{}",
        inner_content,
-        requirements_o.unwrap_or(&String::new())
+        requirements_o.unwrap_or(&String::new()),
+        DOTNET_TARGET_FRAMEWORK.as_str()
    ));
    hash.push_str(&ws_suffix);
    let bin_path = format!("{}/{hash}", *CSHARP_CACHE_DIR);
@@ -0,0 +1,892 @@
+//! Sandboxed container runtime: run a container as a sandboxed subprogram of the job.
+//!
+//! Unlike the legacy `# docker` (dind/daemon) path, this has no daemon and no Docker
+//! API. It splits *pull* from *run*:
+//!
+//! 1. **pull/extract** (`crane`, no daemon/store/root): materialize the image's root
+//!    filesystem into `{job_dir}/rootfs` and read its OCI config
+//!    (Env/Cmd/Entrypoint/WorkingDir), via a digest-keyed rootfs cache.
+//! 2. **run** (the job's own nsjail sandbox): execute the image command with the
+//!    extracted rootfs bound in as the new root, so the container inherits exactly
+//!    the job's confinement (filesystem mask, pid namespace, network, uid) and can't
+//!    escape past what the job itself can reach.
+//!
+//! Selected by `# sandbox <image>` (a bare `# sandbox` keeps plain nsjail-bash;
+//! `# docker` keeps the v1 daemon path). The script body runs inside the image via
+//! `/bin/sh`; an empty body runs the image's ENTRYPOINT/CMD.
+
+use std::process::Stdio;
+
+use serde::Deserialize;
+use serde_json::{json, value::RawValue};
+use sqlx::types::Json;
+use tokio::process::Command;
+
+use windmill_common::{client::AuthedClient, scripts::ScriptLang};
+use windmill_common::{
+    error::Error,
+    worker::{to_raw_value, write_file, Connection},
+};
+
+use windmill_queue::{append_logs, CanceledBy, MiniPulledJob};
+
+use crate::{
+    common::{
+        build_args_map, get_reserved_variables, raw_to_string, resolve_nsjail_timeout,
+        resolve_nsjail_tmp_mount_block, start_child_process, OccupancyMetrics, DEV_CONF_NSJAIL,
+    },
+    get_proxy_envs_for_lang,
+    handle_child::handle_child,
+    DISABLE_NUSER, NSJAIL_AVAILABLE, NSJAIL_PATH, SANDBOX_IMAGE_CACHE_MAX_MB,
+    SANDBOX_IMAGE_DEFAULT_REGISTRY, SANDBOX_IMAGE_MAX_SIZE_MB, SANDBOX_IMAGE_PULL_POLICY,
+    SANDBOX_REGISTRY_AUTH,
+};
+
+const NSJAIL_CONFIG_RUN_DOCKER_CONTENT: &str = include_str!("../nsjail/run.docker.config.proto");
+
+const DEFAULT_PATH: &str = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin";
+
+lazy_static::lazy_static! {
+    /// `crane` (google/go-containerregistry) — pulls + flattens an image to a rootfs
+    /// without a daemon, store, root, or privileged container. We never *run* the
+    /// image via crane (nsjail does the run), so a full container engine is overkill.
+    pub static ref CRANE_PATH: String =
+        std::env::var("CRANE_PATH").unwrap_or_else(|_| "crane".to_string());
+
+    /// `linux/<arch>` for the worker, pinned on every crane call so multi-arch images
+    /// resolve deterministically (and `crane manifest` returns a real manifest, not an
+    /// index).
+    static ref CRANE_PLATFORM: String = format!("linux/{}", match std::env::consts::ARCH {
+        "x86_64" => "amd64",
+        "aarch64" => "arm64",
+        other => other,
+    });
+
+    /// Content-addressed cache of flattened rootfs tars, keyed by image digest. crane
+    /// has no persistent store, so this is what gives cross-job dedup (and, since it's
+    /// digest-keyed, automatic freshness when a moving tag changes).
+    static ref ROOTFS_CACHE_DIR: String =
+        format!("{}sandbox_rootfs", *windmill_common::worker::ROOT_CACHE_DIR);
+}
+
+/// Guards against overlapping cache-eviction passes across concurrent jobs.
+static EVICTION_RUNNING: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
+
+/// `sandbox_image_pull_policy` instance setting. With the digest-keyed cache, `newer`
+/// (default) re-resolves the digest each job (cheap manifest fetch) so moving tags
+/// like `:latest` stay fresh while unchanged digests reuse the cache. `missing` skips
+/// the registry when a digest is already cached for the ref; `never` only uses the
+/// cache (errors if absent); `always` == `newer` here.
+async fn pull_policy() -> String {
+    let p = SANDBOX_IMAGE_PULL_POLICY.read().await.clone();
+    match p.as_deref() {
+        Some(p @ ("missing" | "newer" | "always" | "never")) => p.to_string(),
+        _ => "newer".to_string(),
+    }
+}
+
+/// `sandbox_image_max_size_mb` instance setting; 0 (or unset/non-positive) = no limit.
+async fn max_image_size_mb() -> u64 {
+    SANDBOX_IMAGE_MAX_SIZE_MB.read().await.unwrap_or(0).max(0) as u64
+}
+
+/// `sandbox_image_cache_max_mb` instance setting; 0 (or unset/non-positive) = unbounded.
+async fn image_cache_max_mb() -> u64 {
+    SANDBOX_IMAGE_CACHE_MAX_MB.read().await.unwrap_or(0).max(0) as u64
+}
+
+/// A ref is registry-qualified if the component before the first `/` looks like a
+/// host (contains `.` or `:`, or is `localhost`). Bare repos (`alpine`,
+/// `alpine:latest`, `myorg/img`) are unqualified and resolve against docker.io —
+/// or the configured default registry.
+fn registry_qualified(image: &str) -> bool {
+    match image.split_once('/') {
+        None => false,
+        Some((first, _)) => first.contains('.') || first.contains(':') || first == "localhost",
+    }
+}
+
+/// Prepend the `sandbox_image_default_registry` instance setting to unqualified image
+/// refs (fully-qualified refs are left untouched).
+async fn resolve_image_ref(image: &str) -> String {
+    let registry = SANDBOX_IMAGE_DEFAULT_REGISTRY.read().await.clone();
+    match registry {
+        Some(registry) if !registry.trim().is_empty() && !registry_qualified(image) => {
+            format!("{}/{}", registry.trim().trim_end_matches('/'), image)
+        }
+        _ => image.to_string(),
+    }
+}
+
+/// If the `sandbox_registry_auth` instance setting holds a docker `auth.json` blob,
+/// write it to a per-job `DOCKER_CONFIG` dir (`{job_dir}/.docker/config.json`, 0600,
+/// removed with the job) and return the dir to pass to crane via `DOCKER_CONFIG`.
+/// Returns `None` when unset. (docker `config.json` and podman `auth.json` share the
+/// `{"auths": {...}}` schema, so the same blob works.)
+async fn write_auth_dir(job_dir: &str) -> Result<Option<String>, Error> {
+    let auth = SANDBOX_REGISTRY_AUTH.read().await.clone();
+    let Some(auth) = auth.filter(|a| !a.trim().is_empty()) else {
+        return Ok(None);
+    };
+    let dir = format!("{job_dir}/.docker");
+    tokio::fs::create_dir_all(&dir).await?;
+    let path = format!("{dir}/config.json");
+    // Create 0600 from the start (registry credentials) — no world-readable window.
+    #[cfg(unix)]
+    {
+        use tokio::io::AsyncWriteExt;
+        let mut f = tokio::fs::OpenOptions::new()
+            .write(true)
+            .create(true)
+            .truncate(true)
+            .mode(0o600)
+            .open(&path)
+            .await?;
+        f.write_all(auth.as_bytes()).await?;
+    }
+    #[cfg(not(unix))]
+    tokio::fs::write(&path, auth).await?;
+    Ok(Some(dir))
+}
+
+/// The subset of an image's OCI config we apply to the run.
+#[derive(Deserialize, Default, Debug)]
+struct OciConfig {
+    #[serde(default, rename = "Env")]
+    env: Option<Vec<String>>,
+    #[serde(default, rename = "Cmd")]
+    cmd: Option<Vec<String>>,
+    #[serde(default, rename = "Entrypoint")]
+    entrypoint: Option<Vec<String>>,
+    #[serde(default, rename = "WorkingDir")]
+    working_dir: Option<String>,
+}
+
+/// Quote a string as a protobuf-text-format string literal for safe inclusion in
+/// the nsjail config. Image-controlled values (mount srcs/dsts, symlink targets,
+/// WorkingDir) flow into the config, so they MUST be escaped — an unescaped `"` or
+/// newline would otherwise let a hostile image config inject arbitrary nsjail
+/// directives and break out of the sandbox. Every byte is emitted as a printable
+/// ASCII char or a valid protobuf escape (`\"`, `\\`, `\n`/`\r`/`\t`, or 3-digit
+/// octal `\NNN` for control/non-ASCII bytes), so the result always parses.
+fn proto_str(s: &str) -> String {
+    let mut out = String::with_capacity(s.len() + 2);
+    out.push('"');
+    for &b in s.as_bytes() {
+        match b {
+            b'"' => out.push_str("\\\""),
+            b'\\' => out.push_str("\\\\"),
+            b'\n' => out.push_str("\\n"),
+            b'\r' => out.push_str("\\r"),
+            b'\t' => out.push_str("\\t"),
+            0x20..=0x7e => out.push(b as char),
+            _ => out.push_str(&format!("\\{b:03o}")),
+        }
+    }
+    out.push('"');
+    out
+}
+
+/// Render container env vars as nsjail `envar:` directives (one per line). Each
+/// `KEY=VALUE` is proto-escaped, so image-controlled keys/values can neither break
+/// the config nor reach nsjail's own process environment.
+fn render_envars(env: &[(String, String)]) -> String {
+    env.iter()
+        .map(|(k, v)| format!("envar: {}", proto_str(&format!("{k}={v}"))))
+        .collect::<Vec<_>>()
+        .join("\n")
+}
+
+/// Run `crane` with the optional per-job `DOCKER_CONFIG` auth dir.
+async fn crane(args: &[&str], auth_dir: Option<&str>) -> Result<std::process::Output, Error> {
+    let mut cmd = Command::new(CRANE_PATH.as_str());
+    cmd.args(args);
+    if let Some(dir) = auth_dir {
+        cmd.env("DOCKER_CONFIG", dir);
+    }
+    cmd.output()
+        .await
+        .map_err(|e| Error::ExecutionErr(format!("failed to run crane {}: {e}", args.join(" "))))
+}
+
+/// `crane config` output: the image config (Env/Cmd/Entrypoint/WorkingDir) is nested
+/// under the top-level `config` key.
+#[derive(Deserialize, Default)]
+struct CraneConfig {
+    #[serde(default)]
+    config: OciConfig,
+}
+
+/// Filesystem-safe cache key for a digest (`sha256:ab..` -> `sha256_ab..`).
+fn digest_key(digest: &str) -> String {
+    digest.replace([':', '/'], "_")
+}
+
+/// Filesystem-safe, collision-resistant key for an image ref (the ref->digest file).
+fn ref_key(image: &str) -> String {
+    use std::hash::{Hash, Hasher};
+    let mut h = std::collections::hash_map::DefaultHasher::new();
+    image.hash(&mut h);
+    let safe: String = image
+        .chars()
+        .map(|c| {
+            if c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-') {
+                c
+            } else {
+                '_'
+            }
+        })
+        .collect();
+    let safe = &safe[safe.len().saturating_sub(80)..];
+    format!("{safe}_{:016x}", h.finish())
+}
+
+/// Resolve the image ref to a content digest, honoring the pull policy + a ref->digest
+/// cache. `missing`/`never` reuse a cached digest without hitting the registry (`never`
+/// errors if absent); `newer`/`always` always re-resolve via `crane digest`.
+async fn resolve_digest(
+    image: &str,
+    policy: &str,
+    auth_dir: Option<&str>,
+) -> Result<String, Error> {
+    let refs_dir = format!("{}/refs", *ROOTFS_CACHE_DIR);
+    let ref_file = format!("{refs_dir}/{}", ref_key(image));
+
+    if matches!(policy, "missing" | "never") {
+        if let Ok(d) = tokio::fs::read_to_string(&ref_file).await {
+            let d = d.trim().to_string();
+            if !d.is_empty()
+                && tokio::fs::metadata(format!("{}/{}.tar", *ROOTFS_CACHE_DIR, digest_key(&d)))
+                    .await
+                    .is_ok()
+            {
+                return Ok(d);
+            }
+        }
+        if policy == "never" {
+            return Err(Error::ExecutionErr(format!(
+                "image {image} is not in the sandbox cache and SANDBOX_IMAGE_PULL_POLICY=never"
+            )));
+        }
+    }
+
+    let out = crane(&["digest", "--platform", &CRANE_PLATFORM, image], auth_dir).await?;
+    if !out.status.success() {
+        return Err(Error::ExecutionErr(format!(
+            "failed to resolve image {image}: {}",
+            String::from_utf8_lossy(&out.stderr)
+        )));
+    }
+    let digest = String::from_utf8_lossy(&out.stdout).trim().to_string();
+    let _ = tokio::fs::create_dir_all(&refs_dir).await;
+    // tmp+rename so a concurrent `missing`/`never` reader never sees a torn ref file.
+    let ref_tmp = format!("{ref_file}.tmp.{}", digest_key(&digest));
+    if tokio::fs::write(&ref_tmp, &digest).await.is_ok() {
+        let _ = tokio::fs::rename(&ref_tmp, &ref_file).await;
+    }
+    Ok(digest)
+}
+
+/// Pull (if not cached) and unpack `image` into `{job_dir}/rootfs`, returning its OCI
+/// config. Uses `crane export`/`config` (no daemon/store/root) with a content-addressed
+/// rootfs+config cache keyed by digest for cross-job dedup.
+async fn extract_image(image: &str, job_dir: &str) -> Result<OciConfig, Error> {
+    let rootfs = format!("{job_dir}/rootfs");
+    tokio::fs::create_dir_all(&rootfs).await?;
+    tokio::fs::create_dir_all(&*ROOTFS_CACHE_DIR).await?;
+
+    let auth_dir = write_auth_dir(job_dir).await?;
+    let auth = auth_dir.as_deref();
+    let digest = resolve_digest(image, &pull_policy().await, auth).await?;
+    // Pin every subsequent fetch to the resolved digest, not the (mutable) tag, so the
+    // content can't diverge from the digest we cache under if the tag moves mid-fetch.
+    let pinned = format!("{}@{digest}", image.split('@').next().unwrap_or(image));
+    let key = digest_key(&digest);
+    let tar = format!("{}/{key}.tar", *ROOTFS_CACHE_DIR);
+    let cfg = format!("{}/{key}.json", *ROOTFS_CACHE_DIR);
+    let size_file = format!("{}/{key}.size", *ROOTFS_CACHE_DIR);
+    let token = std::path::Path::new(job_dir)
+        .file_name()
+        .map(|x| x.to_string_lossy().into_owned())
+        .unwrap_or_default();
+
+    // Enforce the size cap on EVERY job (not just cache misses), using a cached size so
+    // a cache reuse needs no registry call — lowering the limit rejects cached images too.
+    enforce_image_size_limit(&pinned, &size_file, auth).await?;
+
+    // Materialize the flattened rootfs. The cache tar can be evicted concurrently, so up
+    // to two attempts: hardlink the cache tar into the job dir (pins the inode against
+    // eviction) before extracting; if it vanished first, re-fetch.
+    let job_tar = format!("{job_dir}/rootfs.tar");
+    for attempt in 0..2 {
+        if tokio::fs::metadata(&tar).await.is_err() {
+            fetch_into_cache(&pinned, &tar, &cfg, &token, auth).await?;
+        }
+        let config = read_oci_config(&cfg).await;
+        let _ = tokio::fs::remove_file(&job_tar).await;
+        // Stage the cache tar into the job dir so concurrent eviction can't unlink it out
+        // from under `tar -xf`. Prefer a hardlink (free), but the cache volume and the job
+        // dir are usually on *different* filesystems in the shipped deployments (the cache
+        // is its own volume/PVC) — there `hard_link` returns EXDEV, so fall back to a copy.
+        // `copy` reads through the source inode, so an eviction mid-copy still completes.
+        let staged = match tokio::fs::hard_link(&tar, &job_tar).await {
+            Ok(()) => Ok(()),
+            Err(e) if e.kind() == std::io::ErrorKind::NotFound => Err(e), // vanished — re-fetch
+            Err(_) => tokio::fs::copy(&tar, &job_tar).await.map(|_| ()),
+        };
+        match staged {
+            Ok(()) => {}
+            Err(e) if e.kind() == std::io::ErrorKind::NotFound && attempt == 0 => {
+                continue; // evicted between the check and the staging — re-fetch
+            }
+            Err(e) => return Err(Error::ExecutionErr(format!("failed to stage rootfs: {e}"))),
+        }
+        // Extract as the worker user (rootfs is worker-owned → uid 0 inside the jail).
+        let untar = Command::new("tar")
+            .args(["-xf", &job_tar, "-C", &rootfs])
+            .output()
+            .await
+            .map_err(|e| Error::ExecutionErr(format!("failed to run tar: {e}")))?;
+        let _ = tokio::fs::remove_file(&job_tar).await;
+        if !untar.status.success() {
+            return Err(Error::ExecutionErr(format!(
+                "failed to unpack image {image}: {}",
+                String::from_utf8_lossy(&untar.stderr)
+            )));
+        }
+        return Ok(config);
+    }
+    Err(Error::ExecutionErr(format!(
+        "failed to materialize rootfs for {image} (cache evicted twice)"
+    )))
+}
+
+/// Fetch + flatten `pinned` (a `name@digest` ref) into the cache: export the rootfs tar
+/// and write the OCI config sidecar, both via tmp+rename so concurrent readers never see
+/// a torn file. The tar is published last (a present tar implies a present config).
+async fn fetch_into_cache(
+    pinned: &str,
+    tar: &str,
+    cfg: &str,
+    token: &str,
+    auth: Option<&str>,
+) -> Result<(), Error> {
+    let tar_tmp = format!("{tar}.tmp.{token}");
+    let cfg_tmp = format!("{cfg}.tmp.{token}");
+    let exported = crane(
+        &["export", "--platform", &CRANE_PLATFORM, pinned, &tar_tmp],
+        auth,
+    )
+    .await?;
+    if !exported.status.success() {
+        let _ = tokio::fs::remove_file(&tar_tmp).await;
+        return Err(Error::ExecutionErr(format!(
+            "failed to export image {pinned}: {}",
+            String::from_utf8_lossy(&exported.stderr)
+        )));
+    }
+    let config = crane(&["config", "--platform", &CRANE_PLATFORM, pinned], auth).await?;
+    if !config.status.success() {
+        let _ = tokio::fs::remove_file(&tar_tmp).await;
+        return Err(Error::ExecutionErr(format!(
+            "failed to read image {pinned} config: {}",
+            String::from_utf8_lossy(&config.stderr)
+        )));
+    }
+    let _ = tokio::fs::write(&cfg_tmp, &config.stdout).await;
+    let _ = tokio::fs::rename(&cfg_tmp, cfg).await;
+    tokio::fs::rename(&tar_tmp, tar).await?;
+    Ok(())
+}
+
+/// Read the cached OCI config (Env/Cmd/Entrypoint/WorkingDir); tolerate a missing or torn
+/// sidecar by falling back to defaults (the run still works off the body + image FS).
+async fn read_oci_config(cfg: &str) -> OciConfig {
+    match tokio::fs::read(cfg).await {
+        Ok(bytes) => serde_json::from_slice::<CraneConfig>(&bytes)
+            .map(|c| c.config)
+            .unwrap_or_default(),
+        Err(_) => OciConfig::default(),
+    }
+}
+
+/// Manifest descriptor (`crane manifest`), for the pre-download size guard.
+#[derive(Deserialize, Default)]
+struct CraneDescriptor {
+    #[serde(default)]
+    size: u64,
+}
+#[derive(Deserialize, Default)]
+struct CraneManifest {
+    #[serde(default)]
+    layers: Vec<CraneDescriptor>,
+    #[serde(default)]
+    config: CraneDescriptor,
+}
+
+/// Reject the image if its compressed download size exceeds `SANDBOX_IMAGE_MAX_SIZE_MB`.
+/// Runs on EVERY job (so lowering the limit rejects already-cached images too); the size
+/// is read from a `{digest}.size` sidecar when present (no registry call on cache reuse)
+/// and otherwise fetched once via `crane manifest` (before any layer download) and cached.
+/// No-op when the limit is 0 (unset).
+async fn enforce_image_size_limit(
+    pinned: &str,
+    size_file: &str,
+    auth_dir: Option<&str>,
+) -> Result<(), Error> {
+    let max = max_image_size_mb().await;
+    if max == 0 {
+        return Ok(());
+    }
+    let bytes = match tokio::fs::read_to_string(size_file)
+        .await
+        .ok()
+        .and_then(|s| s.trim().parse::<u64>().ok())
+    {
+        Some(b) => b,
+        None => {
+            let out = crane(
+                &["manifest", "--platform", &CRANE_PLATFORM, pinned],
+                auth_dir,
+            )
+            .await?;
+            if !out.status.success() {
+                // Don't silently bypass the guard — surface it so an operator can see the
+                // size limit isn't being enforced for this image.
+                tracing::warn!(
+                    "sandbox image size guard: `crane manifest {pinned}` failed, not enforcing \
+                    SANDBOX_IMAGE_MAX_SIZE_MB: {}",
+                    String::from_utf8_lossy(&out.stderr)
+                );
+                return Ok(());
+            }
+            let manifest: CraneManifest = match serde_json::from_slice(&out.stdout) {
+                Ok(m) => m,
+                Err(e) => {
+                    tracing::warn!(
+                        "sandbox image size guard: cannot parse `crane manifest` json: {e}"
+                    );
+                    return Ok(());
+                }
+            };
+            let b = manifest.config.size + manifest.layers.iter().map(|l| l.size).sum::<u64>();
+            let _ = tokio::fs::write(size_file, b.to_string()).await;
+            b
+        }
+    };
+    let mb = bytes / 1_000_000;
+    if mb > max {
+        return Err(Error::ExecutionErr(format!(
+            "image {pinned} is {mb} MB (compressed), over the SANDBOX_IMAGE_MAX_SIZE_MB limit of {max} MB"
+        )));
+    }
+    Ok(())
+}
+
+/// Best-effort eviction: while the cached rootfs tars exceed `SANDBOX_IMAGE_CACHE_MAX_MB`,
+/// remove the oldest by mtime (creation order — tars are write-once, cache hits don't
+/// touch mtime). No-op when the limit is 0 (unset). Skipped if another pass is already
+/// running. The per-job extracted rootfs lives in the job dir (cleaned with the job), so
+/// only the content-addressed tar+config+size cache is pruned. Also sweeps orphaned
+/// `*.tmp.*` files left by a crashed mid-export.
+async fn enforce_image_cache_limit() {
+    use std::sync::atomic::Ordering;
+    let max_mb = image_cache_max_mb().await;
+    if max_mb == 0 {
+        return;
+    }
+    if EVICTION_RUNNING
+        .compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
+        .is_err()
+    {
+        return;
+    }
+    // Reset the guard on every exit path (incl. an early `break` or a panic), so a
+    // stuck flag can never permanently disable eviction until a worker restart.
+    struct ResetOnDrop;
+    impl Drop for ResetOnDrop {
+        fn drop(&mut self) {
+            EVICTION_RUNNING.store(false, std::sync::atomic::Ordering::SeqCst);
+        }
+    }
+    let _reset = ResetOnDrop;
+    let max_bytes = max_mb.saturating_mul(1_000_000);
+
+    // (path, size, mtime) for every cached rootfs tar; also sweep orphaned tmp files.
+    async fn list_tars() -> Vec<(std::path::PathBuf, u64, std::time::SystemTime)> {
+        let mut out = Vec::new();
+        let Ok(mut rd) = tokio::fs::read_dir(&*ROOTFS_CACHE_DIR).await else {
+            return out;
+        };
+        while let Ok(Some(e)) = rd.next_entry().await {
+            let p = e.path();
+            let name = e.file_name();
+            let name = name.to_string_lossy();
+            // Reclaim leftover `*.tmp.<token>` files from a crashed mid-export.
+            if name.contains(".tmp.") {
+                let _ = tokio::fs::remove_file(&p).await;
+                continue;
+            }
+            if p.extension().and_then(|x| x.to_str()) != Some("tar") {
+                continue;
+            }
+            if let Ok(m) = e.metadata().await {
+                let mtime = m.modified().unwrap_or(std::time::UNIX_EPOCH);
+                out.push((p, m.len(), mtime));
+            }
+        }
+        out
+    }
+
+    loop {
+        let mut tars = list_tars().await;
+        let total: u64 = tars.iter().map(|(_, s, _)| *s).sum();
+        if total <= max_bytes || tars.is_empty() {
+            break;
+        }
+        tars.sort_by_key(|(_, _, mtime)| *mtime);
+        let victim = tars[0].0.clone();
+        if tokio::fs::remove_file(&victim).await.is_err() {
+            break; // can't reclaim — stop rather than spin on the same victim
+        }
+        // Drop the sibling config + size sidecars too.
+        let _ = tokio::fs::remove_file(victim.with_extension("json")).await;
+        let _ = tokio::fs::remove_file(victim.with_extension("size")).await;
+        tracing::info!("sandbox image cache eviction: removed {}", victim.display());
+    }
+    // `_reset` drops here and clears EVICTION_RUNNING.
+}
+
+/// Build the nsjail mount block that binds each top-level entry of the rootfs in
+/// place. Binding the whole rootfs at `/` trips nsjail's read-only remount of its
+/// base root in a rootless userns; per-entry binds avoid it. `proc`, `dev`, `tmp`
+/// and `sys` are skipped — the profile provides them.
+async fn generate_rootfs_mounts(rootfs: &str) -> Result<String, Error> {
+    let mut block = String::new();
+    let mut entries = tokio::fs::read_dir(rootfs).await?;
+    while let Some(entry) = entries.next_entry().await? {
+        let name = entry.file_name();
+        let name = name.to_string_lossy();
+        if matches!(name.as_ref(), "proc" | "dev" | "tmp" | "sys") {
+            continue;
+        }
+        let src = proto_str(&format!("{rootfs}/{name}"));
+        let dst = proto_str(&format!("/{name}"));
+        let file_type = entry.file_type().await?;
+        if file_type.is_symlink() {
+            // Recreate top-level symlinks (e.g. usr-merged /bin -> usr/bin) as
+            // symlinks in the jail. The target is image-controlled but only ever
+            // *resolved inside the jail* (against the bound rootfs dirs / jail
+            // pseudo-fs) — there is no host `/` in the jail for it to point at — and
+            // it is escaped via proto_str, so it can neither escape nor inject config.
+            let target = tokio::fs::read_link(entry.path())
+                .await
+                .map(|p| p.to_string_lossy().into_owned())
+                .unwrap_or_default();
+            block.push_str(&format!(
+                "mount {{\n  src: {}\n  dst: {dst}\n  is_symlink: true\n  mandatory: false\n}}\n",
+                proto_str(&target),
+            ));
+        } else {
+            block.push_str(&format!(
+                "mount {{\n  src: {src}\n  dst: {dst}\n  is_bind: true\n  rw: true\n  mandatory: false\n}}\n",
+            ));
+        }
+    }
+    Ok(block)
+}
+
+#[tracing::instrument(level = "trace", skip_all)]
+pub async fn handle_docker_v2_job(
+    image: &str,
+    mem_peak: &mut i32,
+    canceled_by: &mut Option<CanceledBy>,
+    job: &MiniPulledJob,
+    conn: &Connection,
+    client: &AuthedClient,
+    parent_runnable_path: Option<String>,
+    content: &str,
+    job_dir: &str,
+    shared_mount: &str,
+    base_internal_url: &str,
+    worker_name: &str,
+    occupancy_metrics: &mut OccupancyMetrics,
+) -> Result<Box<RawValue>, Error> {
+    // The sandboxed container runtime *is* nsjail, so it requires nsjail. (`# docker`
+    // keeps the v1 dind path for non-sandboxed workers.)
+    if NSJAIL_AVAILABLE.is_none() {
+        return Err(Error::ExecutionErr(format!(
+            "`# sandbox {image}` runs the image inside nsjail, which is not available on \
+            this worker. Install nsjail, or use a bare `# docker` (dind) instead."
+        )));
+    }
+
+    // Apply the default-registry instance setting to unqualified refs.
+    let resolved_image = resolve_image_ref(image).await;
+    let image = resolved_image.as_str();
+
+    append_logs(
+        &job.id,
+        &job.workspace_id,
+        format!("\n\n--- SANDBOXED CONTAINER (nsjail) ---\nextracting image {image}...\n"),
+        conn,
+    )
+    .await;
+
+    let config = extract_image(image, job_dir).await?;
+    let rootfs = format!("{job_dir}/rootfs");
+
+    // Best-effort: keep the cached rootfs tars under their size cap (overlaps the run).
+    tokio::spawn(enforce_image_cache_limit());
+
+    // Resolve the script args from the bash signature, like the bash executor.
+    let args = build_args_map(job, client, conn).await?.map(Json);
+    let job_args = if args.is_some() {
+        args.as_ref()
+    } else {
+        job.args.as_ref()
+    };
+    let args_owned = windmill_parser_bash::parse_bash_sig(content)?
+        .args
+        .iter()
+        .map(|arg| {
+            job_args
+                .and_then(|x| x.get(&arg.name).map(|x| raw_to_string(x.get())))
+                .unwrap_or_else(String::new)
+        })
+        .collect::<Vec<String>>();
+
+    // The body is everything that isn't a leading `#` annotation/comment line. With
+    // a body we run it via the image's `/bin/sh`; without one we run the image's
+    // ENTRYPOINT + CMD.
+    let has_body = content
+        .lines()
+        .any(|l| !l.trim().is_empty() && !l.trim_start().starts_with('#'));
+
+    let cmd_args: Vec<String> = if has_body {
+        // Pass the body straight to `sh -c` rather than writing a script file into
+        // the image-controlled rootfs: a malicious image could plant that path as a
+        // symlink to a host file and capture the worker's write before nsjail starts
+        // (sandbox-boundary bypass). `sh -c <body> sh <args...>` binds args as $1.. .
+        let mut v = vec![
+            "/bin/sh".to_string(),
+            "-c".to_string(),
+            format!("set -e\n{content}"),
+            "sh".to_string(),
+        ];
+        v.extend(args_owned.iter().cloned());
+        v
+    } else {
+        let mut v = config.entrypoint.clone().unwrap_or_default();
+        v.extend(config.cmd.clone().unwrap_or_default());
+        if v.is_empty() {
+            return Err(Error::ExecutionErr(format!(
+                "image {image} has no ENTRYPOINT/CMD and the script body is empty — \
+                nothing to run"
+            )));
+        }
+        v.extend(args_owned.iter().cloned());
+        v
+    };
+
+    let working_dir = config
+        .working_dir
+        .as_deref()
+        .filter(|w| !w.is_empty())
+        .unwrap_or("/");
+
+    // The image's OCI Env is attacker-controlled (BOTH keys and values), so it must
+    // NOT enter the nsjail launcher's own process env: a hostile image could set
+    // LD_PRELOAD / LD_LIBRARY_PATH / LD_AUDIT and have the dynamic loader run code in
+    // the nsjail binary as the worker — outside the jail — before it sandboxes.
+    // Deliver it to the *child only* via proto-escaped `envar:` directives.
+    let mut container_env: Vec<(String, String)> = Vec::new();
+    for kv in config.env.unwrap_or_default() {
+        if let Some((k, v)) = kv.split_once('=') {
+            container_env.push((k.to_string(), v.to_string()));
+        }
+    }
+    if !container_env.iter().any(|(k, _)| k == "PATH") {
+        container_env.push(("PATH".to_string(), DEFAULT_PATH.to_string()));
+    }
+    if !container_env.iter().any(|(k, _)| k == "HOME") {
+        container_env.push(("HOME".to_string(), "/root".to_string()));
+    }
+    let envars = render_envars(&container_env);
+
+    // Render the nsjail profile: dynamic per-entry rootfs binds + image WorkingDir.
+    let nsjail_timeout = resolve_nsjail_timeout(conn, &job.workspace_id, job.id, job.timeout).await;
+    let rootfs_mounts = generate_rootfs_mounts(&rootfs).await?;
+    write_file(
+        job_dir,
+        "run.docker.config.proto",
+        &NSJAIL_CONFIG_RUN_DOCKER_CONTENT
+            .replace("{TIMEOUT}", &nsjail_timeout)
+            .replace("{CLONE_NEWUSER}", &(!*DISABLE_NUSER).to_string())
+            // proto_str-quoted: WorkingDir is image-controlled, must not break out
+            // of the `cwd:` string and inject nsjail directives.
+            .replace("{WORKDIR}", &proto_str(working_dir))
+            .replace("{ROOTFS_MOUNTS}", &rootfs_mounts)
+            .replace(
+                "{TMP_MOUNT_BLOCK}",
+                &resolve_nsjail_tmp_mount_block(job_dir).await,
+            )
+            // `# volume` mounts + same-worker shared folder (empty if none).
+            .replace("{SHARED_MOUNT}", shared_mount)
+            // Image env as `envar:` directives (child-only), so it never touches
+            // nsjail's process env.
+            .replace("{ENVARS}", &envars)
+            .replace("#{DEV}", DEV_CONF_NSJAIL),
+    )?;
+
+    // nsjail's OWN process env: only windmill-trusted keys (reserved vars so
+    // `wmill`/API calls work, + proxy). `keep_env: true` forwards these to the
+    // child. The image env is NOT here — see container_env above.
+    let mut reserved_variables =
+        get_reserved_variables(job, &client.token, conn, parent_runnable_path).await?;
+    reserved_variables.insert("RUST_LOG".to_string(), "info".to_string());
+    reserved_variables.insert(
+        "BASE_INTERNAL_URL".to_string(),
+        base_internal_url.to_string(),
+    );
+
+    let proxy_envs = get_proxy_envs_for_lang(
+        &ScriptLang::Bash,
+        job.kind,
+        &job.id,
+        &job.workspace_id,
+        conn,
+    )
+    .await?;
+
+    let mut nsjail_run_args = vec!["--config", "run.docker.config.proto", "--"];
+    nsjail_run_args.extend(cmd_args.iter().map(|s| s.as_str()));
+
+    let mut nsjail_cmd = Command::new(NSJAIL_PATH.as_str());
+    nsjail_cmd
+        .current_dir(job_dir)
+        .env_clear()
+        .envs(reserved_variables)
+        .envs(proxy_envs)
+        .args(nsjail_run_args)
+        .stdin(Stdio::null())
+        .stdout(Stdio::piped())
+        .stderr(Stdio::piped());
+    let child = start_child_process(nsjail_cmd, NSJAIL_PATH.as_str(), false).await?;
+
+    handle_child(
+        &job.id,
+        conn,
+        mem_peak,
+        canceled_by,
+        child,
+        true,
+        worker_name,
+        &job.workspace_id,
+        "sandboxed container run",
+        job.timeout,
+        true,
+        &mut Some(occupancy_metrics),
+        None,
+        None,
+    )
+    .await?;
+
+    Ok(to_raw_value(&json!(format!(
+        "sandboxed container ({image}) completed successfully"
+    ))))
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{digest_key, proto_str, ref_key, registry_qualified, render_envars};
+
+    #[test]
+    fn digest_key_is_filesystem_safe() {
+        assert_eq!(digest_key("sha256:4d889c14e7d5"), "sha256_4d889c14e7d5");
+        // No `:` or `/` survives (both would break the cache filename).
+        let k = digest_key("sha256:ab/cd:ef");
+        assert!(!k.contains(':') && !k.contains('/'));
+    }
+
+    #[test]
+    fn ref_key_is_safe_and_stable() {
+        // Deterministic for a given ref...
+        assert_eq!(ref_key("ghcr.io/o/i:tag"), ref_key("ghcr.io/o/i:tag"));
+        // ...distinguishes different refs...
+        assert_ne!(ref_key("alpine:latest"), ref_key("alpine:edge"));
+        // ...and is filesystem-safe (no `/` or `:`), incl. for multibyte refs (no panic
+        // on the trailing-80 byte slice since every char maps to single-byte ASCII).
+        for r in [
+            "alpine",
+            "ghcr.io/o/i:tag",
+            "localhost:5000/r@sha256:ab",
+            "rég/imagé:tag",
+        ] {
+            let k = ref_key(r);
+            assert!(!k.contains('/') && !k.contains(':'));
+        }
+    }
+
+    #[test]
+    fn render_envars_emits_proto_directives() {
+        // Image-controlled env (incl. loader vars) is rendered as `envar:` directives
+        // — i.e. delivered to the child via the config, NOT nsjail's process env, so
+        // it can never set LD_PRELOAD/etc. on the nsjail binary itself.
+        let env = vec![
+            ("PATH".to_string(), "/usr/bin".to_string()),
+            ("LD_PRELOAD".to_string(), "rootfs/evil.so".to_string()),
+        ];
+        let out = render_envars(&env);
+        assert_eq!(
+            out,
+            "envar: \"PATH=/usr/bin\"\nenvar: \"LD_PRELOAD=rootfs/evil.so\""
+        );
+        // A value trying to inject extra directives is escaped, not interpreted.
+        let evil = vec![("X".to_string(), "v\"\nclone_newuser: false".to_string())];
+        let line = render_envars(&evil);
+        assert!(line.starts_with("envar: \""));
+        assert!(!line.contains("\nclone_newuser"));
+        assert!(line.contains("\\n"));
+    }
+
+    #[test]
+    fn proto_str_escapes_injection() {
+        // Normal paths are just wrapped in quotes.
+        assert_eq!(proto_str("/app"), "\"/app\"");
+        // A `"` is escaped so it cannot close the surrounding string and inject
+        // subsequent nsjail directives — this is what the WorkingDir / mount-src
+        // sandboxing fixes depend on.
+        let malicious = "/x\"\nmount { src: \"/\" dst: \"/host\" is_bind: true }\n#";
+        let escaped = proto_str(malicious);
+        assert!(escaped.starts_with('"') && escaped.ends_with('"'));
+        // No raw quote or newline survives inside the rendered literal.
+        let inner = &escaped[1..escaped.len() - 1];
+        assert!(!inner.contains('\n'));
+        assert!(!inner.contains("\"") || inner.contains("\\\""));
+        assert!(escaped.contains("\\\"")); // the inner quote is backslash-escaped
+        assert!(escaped.contains("\\n")); // the newline is escaped
+                                          // Control and non-ASCII bytes render as valid 3-digit octal escapes (never
+                                          // a raw byte or an invalid `\u{..}` that nsjail's parser would reject).
+        assert_eq!(proto_str("a\u{1b}b"), "\"a\\033b\""); // ESC (0x1b)
+        assert_eq!(proto_str("é"), "\"\\303\\251\""); // UTF-8 bytes 0xc3 0xa9
+    }
+
+    #[test]
+    fn registry_qualified_classifies_refs() {
+        // Unqualified: bare repos (with/without tag) and docker.io org/repo.
+        for img in ["alpine", "alpine:latest", "myorg/img", "myorg/img:1.2"] {
+            assert!(!registry_qualified(img), "{img} should be unqualified");
+        }
+        // Qualified: the first path component is a host (has `.`/`:`) or localhost.
+        for img in [
+            "ghcr.io/org/img",
+            "registry.example.com/img:tag",
+            "localhost:5000/img",
+            "localhost/img",
+            "host:5000/a/b",
+        ] {
+            assert!(registry_qualified(img), "{img} should be qualified");
+        }
+    }
+}
@@ -31,6 +31,7 @@ mod csharp_executor;
 mod dedicated_worker_ee;
 mod dedicated_worker_oss;
 mod deno_executor;
+mod docker_v2;
 #[cfg(feature = "duckdb")]
 mod duckdb_executor;
 mod global_cache;
@@ -7,3 +7,6 @@ use windmill_queue::MiniPulledJob;

 #[cfg(not(feature = "private"))]
 pub fn add_root_flow_job_to_otlp(_queued_job: &MiniPulledJob, _success: bool) {}
+
+#[cfg(not(feature = "private"))]
+pub fn set_job_span_parent(_span: &tracing::Span, _job: &MiniPulledJob, _rj: &uuid::Uuid) {}
@@ -597,11 +597,57 @@ async fn postinstall(
    Ok(())
 }

+/// Python hard keywords cannot be used as a bare name in `import <name>` /
+/// `from <pkg> import <name>`. A flow inline step whose id (or a folder on its
+/// path) is such a keyword — e.g. a step id `in` — otherwise generates
+/// `from pkg import in as inner_script`, a SyntaxError. Prefix these with `_`,
+/// mirroring the existing digit-leading guard.
+fn is_python_keyword(s: &str) -> bool {
+    matches!(
+        s,
+        "False"
+            | "None"
+            | "True"
+            | "and"
+            | "as"
+            | "assert"
+            | "async"
+            | "await"
+            | "break"
+            | "class"
+            | "continue"
+            | "def"
+            | "del"
+            | "elif"
+            | "else"
+            | "except"
+            | "finally"
+            | "for"
+            | "from"
+            | "global"
+            | "if"
+            | "import"
+            | "in"
+            | "is"
+            | "lambda"
+            | "nonlocal"
+            | "not"
+            | "or"
+            | "pass"
+            | "raise"
+            | "return"
+            | "try"
+            | "while"
+            | "with"
+            | "yield"
+    )
+}
+
 /// Compute the directory (relative to job_dir) where Python writes the main script.
 /// Module files must be placed in this same directory for relative imports to work.
 pub fn compute_python_module_dir(script_path: &str) -> String {
    let script_path_splitted = script_path.split("/").map(|x| {
-        if x.starts_with(|x: char| x.is_ascii_digit()) {
+        if x.starts_with(|x: char| x.is_ascii_digit()) || is_python_keyword(x) {
            format!("_{}", x)
        } else {
            x.to_string()
@@ -1236,6 +1282,12 @@ pub fn compute_py_codegen(content: &str, script_path: &str) -> PyScriptCodegen {
        .replace("-", "_")
        .replace(" ", "_")
        .to_lowercase();
+    // `last` is lowercased above, so this catches a keyword id in any case.
+    let last = if is_python_keyword(&last) {
+        format!("_{last}")
+    } else {
+        last
+    };

    let sig = windmill_parser_py::parse_python_signature(content, None, false).unwrap_or_default();
    let pre_sig = windmill_parser_py::parse_python_signature(
@@ -1605,6 +1657,12 @@ async fn prepare_wrapper(
        .replace("-", "_")
        .replace(" ", "_")
        .to_lowercase();
+    // `last` is lowercased above, so this catches a keyword id in any case.
+    let last = if is_python_keyword(&last) {
+        format!("_{last}")
+    } else {
+        last
+    };
    let module_dir = format!("{}/{}", job_dir, dirs);
    tokio::fs::create_dir_all(format!("{module_dir}/")).await?;

@@ -3286,6 +3344,13 @@ mod tests {
        assert_eq!(compute_python_module_dir("u/@admin/script"), "u/.admin");
    }

+    #[test]
+    fn test_compute_python_module_dir_keyword_segment() {
+        // A folder whose name is a Python keyword would otherwise produce an
+        // invalid `from f.in.x import ...`; it is underscore-prefixed.
+        assert_eq!(compute_python_module_dir("f/in/script"), "f/_in");
+    }
+
    #[test]
    fn test_compute_py_codegen_basic_args() {
        let code = "def main(x: str, y: int):\n    return x\n";
@@ -3297,6 +3362,22 @@ mod tests {
        assert_eq!(cg.module_name, "script");
    }

+    #[test]
+    fn test_compute_py_codegen_keyword_step_id() {
+        // Regression for a flow inline step whose auto-assigned id is a Python
+        // keyword (e.g. `in`): the generated wrapper must not emit
+        // `from pkg import in as inner_script` (SyntaxError). The module name is
+        // underscore-prefixed, matching the digit-leading guard.
+        let code = "def main():\n    return 1\n";
+        let cg = compute_py_codegen(code, "u/admin/myflow/in");
+        assert_eq!(cg.module_name, "_in");
+        assert_eq!(cg.module_dir_dot, "u.admin.myflow");
+
+        // Non-keyword ids are unaffected.
+        let cg2 = compute_py_codegen(code, "u/admin/myflow/step");
+        assert_eq!(cg2.module_name, "step");
+    }
+
    #[test]
    fn test_compute_py_codegen_with_datetime_and_bytes() {
        let code = "import datetime\n\ndef main(name: str, created_at: datetime.datetime, file: bytes):\n    return name\n";
@@ -185,6 +185,9 @@ async fn process_jc(
            span.record("labels", labels.join(","));
        }
    }
+    // The secondary `job_postprocessing` span stays on the UUID-derived context
+    // (MiniCompletedJob carries no args, so the inbound traceparent isn't
+    // available here); the primary job span is relocated in `create_span_with_name`.
    windmill_common::otel_oss::set_span_parent(&span, &rj);

    if let Some(lg) = jc.job.script_lang.as_ref() {
@@ -694,6 +694,27 @@ lazy_static::lazy_static! {
    /// RAM-backed tmpfs sized by `nsjail_tmpfs_size_mb`.
    pub static ref NSJAIL_TMP_BACKING: Arc<RwLock<Option<String>>> = Arc::new(RwLock::new(None));

+    /// Reject a `# sandbox <image>` whose compressed download size exceeds this many
+    /// MB, before download. `None`/non-positive = no limit. (`sandbox_image_max_size_mb`.)
+    pub static ref SANDBOX_IMAGE_MAX_SIZE_MB: Arc<RwLock<Option<i64>>> = Arc::new(RwLock::new(None));
+
+    /// Best-effort cap (MB) on the worker's cached rootfs tars; oldest evicted after a
+    /// run when exceeded. `None`/non-positive = unbounded. (`sandbox_image_cache_max_mb`.)
+    pub static ref SANDBOX_IMAGE_CACHE_MAX_MB: Arc<RwLock<Option<i64>>> = Arc::new(RwLock::new(None));
+
+    /// Sandbox image pull policy (`missing`/`newer`/`always`/`never`). `None`/unrecognized
+    /// falls back to `newer`. (`sandbox_image_pull_policy`.)
+    pub static ref SANDBOX_IMAGE_PULL_POLICY: Arc<RwLock<Option<String>>> = Arc::new(RwLock::new(None));
+
+    /// If set, unqualified sandbox image refs (e.g. `alpine`) are pulled from this
+    /// registry instead of docker.io. Fully-qualified refs are unaffected.
+    /// (`sandbox_image_default_registry`.)
+    pub static ref SANDBOX_IMAGE_DEFAULT_REGISTRY: Arc<RwLock<Option<String>>> = Arc::new(RwLock::new(None));
+
+    /// Optional docker `auth.json` blob for private registries, written to a per-job
+    /// `DOCKER_CONFIG` dir for crane. (`sandbox_registry_auth`.)
+    pub static ref SANDBOX_REGISTRY_AUTH: Arc<RwLock<Option<String>>> = Arc::new(RwLock::new(None));
+
    /// Optional mirror URL for `uv python install`. Wires to the `UV_PYTHON_INSTALL_MIRROR`
    /// env var when forwarded to uv. Can be set via the `UV_PYTHON_INSTALL_MIRROR` env var
    /// or the `uv_python_install_mirror` instance setting.
@@ -919,14 +940,50 @@ pub async fn is_otel_tracing_proxy_enabled_for_lang(lang: &ScriptLang) -> bool {
    }
 }

+/// Strict check that a string is a well-formed W3C `traceparent`
+/// (`version-traceid-spanid-flags`, lowercase hex, non-zero ids, version != ff).
+/// Used before forwarding an inbound header value verbatim to a job subprocess,
+/// so we don't hand downstream OTel parsers something they'll reject.
+#[cfg(all(feature = "private", feature = "enterprise"))]
+fn valid_w3c_traceparent(tp: &str) -> bool {
+    let p: Vec<&str> = tp.split('-').collect();
+    p.len() == 4
+        && p[0].len() == 2
+        && p[1].len() == 32
+        && p[2].len() == 16
+        && p[3].len() == 2
+        // version "ff" is reserved/invalid per the W3C spec
+        && p[0] != "ff"
+        && p[1] != "00000000000000000000000000000000"
+        && p[2] != "0000000000000000"
+        // W3C mandates lowercase hex
+        && p
+            .iter()
+            .all(|s| s.bytes().all(|b| matches!(b, b'0'..=b'9' | b'a'..=b'f')))
+}
+
 /// Get OTEL trace context environment variables for a job (TRACEPARENT, OTEL_TRACE_ID, OTEL_SPAN_ID).
 /// Returns an empty vec when OTEL tracing is not enabled or on non-enterprise builds.
+///
+/// When the request that enqueued the job carried a valid inbound `traceparent`
+/// (propagated via the job's [`LogContext`](windmill_common::log_context::LogContext)),
+/// it is forwarded verbatim so the script's spans join the originating
+/// distributed trace. Otherwise the trace context is derived from the job UUID.
 pub fn get_otel_context_envs(job_id: &uuid::Uuid) -> Vec<(&'static str, String)> {
    #[cfg(all(feature = "private", feature = "enterprise"))]
    if windmill_common::OTEL_TRACING_ENABLED.load(std::sync::atomic::Ordering::Relaxed) {
-        let trace_id = format!("{:032x}", job_id.as_u128());
-        let span_id = format!("{:016x}", job_id.as_u64_pair().1);
-        let traceparent = format!("00-{}-{}-01", trace_id, span_id);
+        let inbound = windmill_common::log_context::current_log_context()
+            .and_then(|c| c.inbound_traceparent.clone())
+            .filter(|tp| valid_w3c_traceparent(tp));
+        let (traceparent, trace_id, span_id) = if let Some(tp) = inbound {
+            let trace_id = tp[3..35].to_string();
+            let span_id = tp[36..52].to_string();
+            (tp, trace_id, span_id)
+        } else {
+            let trace_id = format!("{:032x}", job_id.as_u128());
+            let span_id = format!("{:016x}", job_id.as_u64_pair().1);
+            (format!("00-{}-{}-01", trace_id, span_id), trace_id, span_id)
+        };
        return vec![
            ("TRACEPARENT", traceparent),
            ("OTEL_TRACE_ID", trace_id),
@@ -1466,7 +1523,10 @@ pub fn create_span_with_name(
        span.record("script_hash", script_hash.to_string().as_str());
    }

-    windmill_common::otel_oss::set_span_parent(&span, &rj);
+    // Parent the job span on the inbound distributed trace when the request that
+    // enqueued it (or its flow root) carried a W3C `traceparent`; otherwise on
+    // the UUID-derived context. See `otel_ee::set_job_span_parent`.
+    crate::otel_oss::set_job_span_parent(&span, arc_job, &rj);
    span
 }

@@ -1567,10 +1627,21 @@ pub fn log_context_for_job(
        trigger_kind: arc_job.trigger_kind.as_ref().map(|k| k.to_string()),
        trigger: arc_job.trigger.clone(),
        hostname: hostname.map(|h| h.to_string()),
+        inbound_traceparent: job_inbound_traceparent(arc_job),
        ..existing
    }
 }

+/// Extract the inbound W3C `traceparent` captured at enqueue from a job's args
+/// (reserved `_wm_traceparent` key). Present only on directly-triggered jobs
+/// (and flow steps that inherited it).
+pub(crate) fn job_inbound_traceparent(job: &MiniPulledJob) -> Option<String> {
+    job.args
+        .as_ref()
+        .and_then(|a| a.get(windmill_common::jobs::WM_TRACEPARENT))
+        .and_then(|raw| serde_json::from_str::<String>(raw.get()).ok())
+}
+
 pub async fn handle_all_job_kind_error(
    conn: &Connection,
    authed_client: &AuthedClient,
@@ -311,13 +311,14 @@ struct RecoveryObject {
    recover: Option<bool>,
 }

-fn get_stop_after_if_data(stop_after_if: Option<&StopAfterIf>) -> (bool, Option<String>) {
+/// Returns `(skip_if_stopped, error_message, include_step_result)`.
+fn get_stop_after_if_data(stop_after_if: Option<&StopAfterIf>) -> (bool, Option<String>, bool) {
    if let Some(stop_after_if) = stop_after_if {
        // skip_if_stopped and error_message are mutually exclusive:
        // skip_if_stopped=true means clean stop (mark remaining as skipped),
        // error_message means stop with error. skip_if_stopped takes precedence.
        if stop_after_if.skip_if_stopped {
-            return (true, None);
+            return (true, None, false);
        }
        let err_msg = stop_after_if.error_message.as_ref().and_then(|message| {
            if message.is_empty() {
@@ -326,9 +327,9 @@ fn get_stop_after_if_data(stop_after_if: Option<&StopAfterIf>) -> (bool, Option<
                Some(message.clone())
            }
        });
-        return (false, err_msg);
+        return (false, err_msg, stop_after_if.error_include_result);
    }
-    return (false, None);
+    return (false, None, false);
 }

 async fn get_id_ctx_for_expr(
@@ -358,6 +359,7 @@ async fn evaluate_stop_after_all_iters_if(
    stop_early: &mut bool,
    skip_if_stop_early: &mut bool,
    stop_early_err_msg: &mut Option<String>,
+    stop_early_include_result: &mut bool,
    nresult: &mut Option<Arc<Box<RawValue>>>,
    args: HashMap<String, Box<RawValue>>,
    flow_env: Option<&HashMap<String, Box<RawValue>>>,
@@ -394,8 +396,11 @@ async fn evaluate_stop_after_all_iters_if(

    if stop_early_after_all_iters {
        *stop_early = true;
-        (*skip_if_stop_early, *stop_early_err_msg) =
-            get_stop_after_if_data(Some(stop_after_all_iters_if));
+        (
+            *skip_if_stop_early,
+            *stop_early_err_msg,
+            *stop_early_include_result,
+        ) = get_stop_after_if_data(Some(stop_after_all_iters_if));
    }
    Ok(())
 }
@@ -655,19 +660,24 @@ pub async fn update_flow_status_after_job_completion_internal(
            false
        };

-        let (mut stop_early, mut stop_early_err_msg, mut skip_if_stop_early, continue_on_error) =
-            if stop_early_override.is_some()
-                && !is_flow_stop_early_override
-                && !parallel_loop
-                && !parallel_branchall
-            {
-                // we ignore stop_early_override (stop_early in children) if module is parallel or is a flow step
-                let se = stop_early_override.as_ref().unwrap();
-                (true, None, *se, false)
-            } else if is_failure_step || module_step.is_preprocessor_step() {
-                (false, None, false, false)
-            } else if let Some(current_module) = current_module {
-                let stop_early = success
+        let (
+            mut stop_early,
+            mut stop_early_err_msg,
+            mut skip_if_stop_early,
+            mut stop_early_include_result,
+            continue_on_error,
+        ) = if stop_early_override.is_some()
+            && !is_flow_stop_early_override
+            && !parallel_loop
+            && !parallel_branchall
+        {
+            // we ignore stop_early_override (stop_early in children) if module is parallel or is a flow step
+            let se = stop_early_override.as_ref().unwrap();
+            (true, None, *se, false, false)
+        } else if is_failure_step || module_step.is_preprocessor_step() {
+            (false, None, false, false, false)
+        } else if let Some(current_module) = current_module {
+            let stop_early = success
                    && !is_branch_all // we don't support stop_early per branch
                    && !parallel_loop // we don't support anymore stop_early per iteration when parallel for loop (removed from frontend)
                    && !is_identity_job // don't evaluate stop_after_if for skipped (identity) steps
@@ -717,22 +727,23 @@ pub async fn update_flow_status_after_job_completion_internal(
                    } else {
                        false
                    };
-                let (skip_if_stopped, stop_early_err_msg) = if stop_early {
-                    get_stop_after_if_data(current_module.stop_after_if.as_ref())
-                } else {
-                    (false, None)
-                };
-
-                (
-                    stop_early,
-                    stop_early_err_msg,
-                    skip_if_stopped,
-                    current_module.continue_on_error.unwrap_or(false),
-                )
+            let (skip_if_stopped, stop_early_err_msg, include_result) = if stop_early {
+                get_stop_after_if_data(current_module.stop_after_if.as_ref())
            } else {
-                (false, None, false, false)
+                (false, None, false)
            };

+            (
+                stop_early,
+                stop_early_err_msg,
+                skip_if_stopped,
+                include_result,
+                current_module.continue_on_error.unwrap_or(false),
+            )
+        } else {
+            (false, None, false, false, false)
+        };
+
        let skip_seq_branch_failure = match module_status {
            FlowStatusModule::InProgress {
                branchall: Some(BranchAllStatus { branch, .. }),
@@ -974,6 +985,7 @@ pub async fn update_flow_status_after_job_completion_internal(
                            &mut stop_early,
                            &mut skip_if_stop_early,
                            &mut stop_early_err_msg,
+                            &mut stop_early_include_result,
                            &mut nresult,
                            args,
                            resolved_flow_env.as_deref(),
@@ -1173,6 +1185,7 @@ pub async fn update_flow_status_after_job_completion_internal(
                    stop_early = false;
                    stop_early_err_msg = None;
                    skip_if_stop_early = false;
+                    stop_early_include_result = false;
                }

                if is_loop || (is_branch_all && !stop_early) {
@@ -1194,6 +1207,7 @@ pub async fn update_flow_status_after_job_completion_internal(
                            &mut stop_early,
                            &mut skip_if_stop_early,
                            &mut stop_early_err_msg,
+                            &mut stop_early_include_result,
                            &mut nresult,
                            args,
                            resolved_flow_env.as_deref(),
@@ -1310,12 +1324,22 @@ pub async fn update_flow_status_after_job_completion_internal(
        };

        if stop_early && stop_early_err_msg.is_some() {
-            nresult = Some(Arc::new(to_raw_value(&serde_json::json! ({
-                "error": {
-                    "name": "EarlyStopError",
-                    "message": stop_early_err_msg.as_ref().unwrap(),
-                }
-            }))));
+            let mut error = serde_json::json!({
+                "name": "EarlyStopError",
+                "message": stop_early_err_msg.as_ref().unwrap(),
+            });
+            if stop_early_include_result {
+                // Embed the stopping step's own result inside the error object instead
+                // of discarding it, keeping the top-level result shape `{ "error": .. }`
+                // unchanged. `nresult` is already set for loops/branchall (aggregated
+                // iteration results), otherwise fall back to the step result.
+                let step_result = nresult.clone().unwrap_or_else(|| result.clone());
+                error["result"] =
+                    serde_json::to_value(&step_result).unwrap_or(serde_json::Value::Null);
+            }
+            nresult = Some(Arc::new(to_raw_value(
+                &serde_json::json!({ "error": error }),
+            )));
        }

        let step_counter = if inc_step_counter {
@@ -4174,6 +4198,21 @@ async fn push_next_flow_job(
            }
        }

+        // Propagate the inbound W3C traceparent captured at enqueue to each step
+        // so the whole flow shares the originating distributed trace (the trace
+        // identity is otherwise derived from the root job UUID). Observability
+        // only — no security impact — so unlike _TEMP_SCRIPT_REFS it is not
+        // gated to previews.
+        if let Some(traceparent) = arc_flow_job_args
+            .as_ref()
+            .get(windmill_common::jobs::WM_TRACEPARENT)
+        {
+            push_args.extra.get_or_insert_with(HashMap::new).insert(
+                windmill_common::jobs::WM_TRACEPARENT.to_string(),
+                traceparent.clone(),
+            );
+        }
+
        tracing::debug!(id = %flow_job.id, root_id = %job_root, "computed args for job {i} of {len}");

        let value_with_parallel = module.get_value_with_parallel()?;
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
 import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
 import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";

-export const VERSION = "v1.717.1";
+export const VERSION = "v1.720.0";

 export async function login(email: string, password: string): Promise<string> {
  return await windmill.UserService.login({
@@ -10,4 +10,4 @@ export const WM_FORK_PREFIX = "wm-fork";
 // (e.g. utils.ts) can read it without importing main.ts and creating a circular
 // dependency (main → workspace → utils → main) that triggers a TDZ.
 // Re-exported from main.ts for backwards compatibility.
-export const VERSION = "1.717.1";
+export const VERSION = "1.720.0";
@@ -49,25 +49,6 @@ services:

    logging: *default-logging

-  # Docker-in-Docker sidecar: provides an isolated Docker daemon so user scripts
-  # can run containers without accessing the host Docker socket.
-  dind:
-    image: docker:dind
-    privileged: true
-    restart: unless-stopped
-    environment:
-      DOCKER_TLS_CERTDIR: ""
-    volumes:
-      - dind-data:/var/lib/docker
-    expose:
-      - 2375
-    healthcheck:
-      test: ["CMD", "docker", "info"]
-      interval: 10s
-      timeout: 5s
-      retries: 5
-    logging: *default-logging
-
  windmill_worker:
    image: ${WM_IMAGE}
    pull_policy: always
@@ -89,22 +70,19 @@ services:
      # If running with non-root/non-windmill UID (e.g., user: "1001:1001"),
      # add: - HOME=/tmp
      - FAVOR_UNSHARE_PID=true
-      # Connect to the dind sidecar instead of the host Docker socket
-      - DOCKER_HOST=tcp://dind:2375
    depends_on:
      db:
        condition: service_healthy
-      dind:
-        condition: service_healthy
    # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
    volumes:
      - worker_dependency_cache:/tmp/windmill/cache
      - worker_logs:/tmp/windmill/logs
-      ## WARNING: mounting the host Docker socket grants user scripts full access to
-      ## the host Docker daemon, enabling host filesystem access and privilege escalation.
-      ## Only use this if you fully trust all users who can run scripts.
-      ## To use it, remove the DOCKER_HOST env var and dind depends_on above,
-      ## and uncomment the line below:
+      ## Sandboxed containers (`# sandbox <image>`) run daemonless via crane + nsjail
+      ## inside the worker itself — no Docker socket or dind sidecar required.
+      ## For the legacy full-compat docker (a bare `# docker`, trusted users only),
+      ## mount the host Docker socket by uncommenting the line below. WARNING: this
+      ## grants user scripts full access to the host Docker daemon (host filesystem
+      ## access and privilege escalation) — only use it if you fully trust all users.
      # - /var/run/docker.sock:/var/run/docker.sock

    logging: *default-logging
@@ -237,4 +215,3 @@ volumes:
  windmill_index: null
  lsp_cache: null
  caddy_data: null
-  dind-data: null
@@ -90,6 +90,16 @@ RUN apt-get update \
    && apt-get clean && rm -rf /var/lib/apt/lists/*
 COPY --from=nsjail /nsjail/nsjail /bin/nsjail

+# crane: pulls + flattens images for the sandboxed container runtime (`# sandbox <image>`).
+# Single static binary — no daemon/store/root needed. See docs/docker-v2-runtime.md.
+ARG CRANE_VERSION=v0.20.6
+RUN arch="$(dpkg --print-architecture)"; \
+    case "$arch" in amd64) crane_arch=x86_64 ;; arm64) crane_arch=arm64 ;; *) echo >&2 "error: unsupported arch '$arch' for crane"; exit 1 ;; esac; \
+    wget -O /tmp/crane.tgz "https://github.com/google/go-containerregistry/releases/download/${CRANE_VERSION}/go-containerregistry_Linux_${crane_arch}.tar.gz" \
+    && tar -xzf /tmp/crane.tgz -C /usr/local/bin crane \
+    && rm /tmp/crane.tgz \
+    && chmod +x /usr/local/bin/crane
+
 WORKDIR ${APP}

 COPY --from=ghcr.io/windmill-labs/windmill:dev --chmod=755  ${APP}/windmill ${APP}/windmill
@@ -90,6 +90,16 @@ RUN apt-get update \
    && apt-get clean && rm -rf /var/lib/apt/lists/*
 COPY --from=nsjail /nsjail/nsjail /bin/nsjail

+# crane: pulls + flattens images for the sandboxed container runtime (`# sandbox <image>`).
+# Single static binary — no daemon/store/root needed. See docs/docker-v2-runtime.md.
+ARG CRANE_VERSION=v0.20.6
+RUN arch="$(dpkg --print-architecture)"; \
+    case "$arch" in amd64) crane_arch=x86_64 ;; arm64) crane_arch=arm64 ;; *) echo >&2 "error: unsupported arch '$arch' for crane"; exit 1 ;; esac; \
+    wget -O /tmp/crane.tgz "https://github.com/google/go-containerregistry/releases/download/${CRANE_VERSION}/go-containerregistry_Linux_${crane_arch}.tar.gz" \
+    && tar -xzf /tmp/crane.tgz -C /usr/local/bin crane \
+    && rm /tmp/crane.tgz \
+    && chmod +x /usr/local/bin/crane
+
 WORKDIR ${APP}

 COPY --from=ghcr.io/windmill-labs/windmill-ee:dev --chmod=755  ${APP}/windmill ${APP}/windmill
@@ -0,0 +1,110 @@
+# Sandboxed container runtime (daemonless docker)
+
+Windmill bash scripts can run a container image. There are **two** runtimes:
+
+| | legacy `# docker` | sandboxed `# sandbox <image>` |
+|---|---|---|
+| selected by | bare `# docker` | `# sandbox <image>` |
+| runtime | dind / Docker daemon (bollard, `dind` feature) | daemonless: extract rootfs + nsjail-run |
+| boundary | separate (daemon outside the jail) | the job's own nsjail sandbox |
+| nsjail | not provided (trusted-tenant) | **required** — this *is* the sandbox |
+| safety | trusted-tenant | sandboxed (untrusted-capable) |
+| compat | full `docker run`/`-d`/API | run-a-command subset |
+
+The three bash annotations are distinct and don't overload each other:
+
+- `# docker` → legacy daemon docker (unchanged).
+- `# sandbox` → run the bash script under nsjail.
+- `# sandbox <image>` → run that image's command under nsjail (this runtime).
+
+## Using it
+
+Put the image ref on a `# sandbox` annotation line; the rest of the script runs
+**inside** that image:
+
+```bash
+# sandbox python:3.12-slim
+name="$1"          # windmill args bind positionally, like any bash script
+python3 -c "import sys; print('hello', sys.argv[1])" "$name"
+```
+
+- The body runs via the image's `/bin/sh -c` (so the image needs a shell).
+- An **empty** body runs the image's `ENTRYPOINT` + `CMD`.
+- Windmill args (declared `x="$1"`, …) are appended to the command.
+- The image's `Env`, `WorkingDir` are applied; the windmill reserved variables
+  (`WM_TOKEN`, `BASE_INTERNAL_URL`, …) are injected so `wmill`/API calls work.
+
+## How it works
+
+1. **Pull/extract** ([`crane`](https://github.com/google/go-containerregistry), no
+   daemon/store/root): `crane export <image>` streams the image's flattened root
+   filesystem to a tar (layers + whiteouts applied, like `docker export`) and
+   `crane config` reads its OCI config. The tar + config are cached
+   content-addressed by digest (`crane digest`) so unchanged digests reuse the
+   cache; `tar -x` materializes the per-job `{job_dir}/rootfs`. crane is a single
+   ~25 MB static binary — we never *run* the image with it (nsjail does), so a full
+   container engine like podman isn't needed.
+2. **Run** (the job's nsjail sandbox): nsjail binds each top-level entry of the
+   rootfs in place (binding the whole rootfs at `/` trips nsjail's read-only
+   remount of its base root in a rootless userns), mounts the standard
+   pseudo-filesystems (`/proc` from the jail's pid namespace, a tmpfs `/tmp`,
+   `/dev` nodes), maps uid/gid 0 inside → the worker user outside, and runs the
+   command. The container *is* the jail.
+
+```
+# sandbox <image>  ─▶  crane export → digest-keyed rootfs cache → tar -x → {job_dir}/rootfs  ─▶  nsjail (chroot rootfs)
+                       crane config (OCI config) ───────────────────────────────────────────▶  Env / Cmd / WorkingDir
+```
+
+Because the run is just the job's own nsjail with the image's filesystem as root,
+the container inherits exactly the job's confinement:
+
+- **Filesystem**: only the rootfs + the job's mounts are visible — no host `/`,
+  no other job dirs, no dep cache. There is nothing to bind-mount escape to.
+- **/proc**: the jail's own pid namespace — the worker and other jobs aren't
+  visible.
+- **uid**: a single-uid jail — an escape lands as the unprivileged worker user.
+- **network**: the job's network (same as any bash job).
+
+## Image storage, freshness & limits
+
+- **Where pulls live:** a content-addressed cache of flattened rootfs tars (+ OCI
+  config sidecars) keyed by image digest, under `{ROOT_CACHE_DIR}/sandbox_rootfs`
+  (persistent, dedups pulls across jobs). The per-job extracted rootfs lives in
+  `{job_dir}/rootfs` and is removed with the job.
+- **Freshness (`SANDBOX_IMAGE_PULL_POLICY`, default `newer`):** the cache is keyed
+  by digest, so a moving tag whose digest changed re-pulls automatically. `newer`
+  (default) / `always` re-resolve the digest each job (one cheap `crane digest`
+  manifest fetch); `missing` reuses a cached digest for the ref without hitting the
+  registry; `never` only uses the cache (errors if absent). Pinning a digest
+  (`img@sha256:…`) is immutable and never stale.
+- **Per-image size cap (`SANDBOX_IMAGE_MAX_SIZE_MB`, default 0 = off):** images
+  whose *compressed download* size (`crane manifest`) exceeds the cap are rejected
+  **before any layer is downloaded**.
+- **Cache size cap (`SANDBOX_IMAGE_CACHE_MAX_MB`, default 0 = off):** best-effort
+  eviction — after a run, the oldest cached rootfs tars (by creation time) are
+  removed until the cache is back under the cap.
+
+## Requirements
+
+- [`crane`](https://github.com/google/go-containerregistry) and `tar` on the worker
+  for image pull/extract (a single static binary — no daemon, root, or privileged).
+- `nsjail` on the worker — **required**. If nsjail is absent, a `# sandbox <image>`
+  job errors clearly (use a bare `# docker` + a daemon instead).
+
+## Limitations (by design — daemonless, run-to-completion)
+
+- No `docker run -d` + later `exec`/`attach`/`logs -f`, no `docker build`,
+  `compose`, swarm, healthchecks.
+- No arbitrary `-v` host bind mounts, `--privileged`, `--cap-add`, `--device`,
+  host namespace sharing.
+- Images that drop to a non-root uid or chown to arbitrary uids inside need a
+  subuid **range** in the jail (single-uid only today — follow-up: `newuidmap`
+  range mapping).
+- The script result is a completion message; capture output via stdout/logs.
+
+## Follow-ups
+
+- Subuid-range nsjail variant for multi-uid images.
+- Per-container isolated networking (slirp/pasta).
+- Support under the non-nsjail `unshare` isolation mode.
@@ -51,27 +51,53 @@ parent and incoming mail would be delivered arbitrarily. The clone filter
 copies email triggers only when `workspaced_local_part IS TRUE` (or
 `CLOUD_HOSTED`, since cloud scopes email lookup by `workspace_id` natively).

-## Merge-direction filter (always on)
+## Operational state is owned by the parent

-Whenever the source workspace has `parent_workspace_id IS NOT NULL` (i.e.
-it's a fork), the tarball export at `/api/w/{workspace}/workspaces/tarball`
-strips fork-local fields:
+The rule that makes both the normal-git PR merge and the in-app merge behave:

- `mode` from every `*_trigger` row
- `enabled` from every `schedule` row
+> **A trigger's `mode` (and a schedule's `enabled`) belongs to the parent
+> workspace. Git-sync *reads* the parent's value into a fork's synced file and
+> *never writes* a fork's value back. No git-sync / merge / create / update
+> write sets a fork's operational state; the `setmode` / `setenabled` endpoint
+> is the intended explicit mutator.**

-The fork-detection key is the column, not the `wm-fork-*` naming convention,
-so it stays consistent with the conflict-warning gates in `set_trigger_mode`
-and `set_schedule_enabled` and survives any future ID rename.
+(Runtime error handling can still auto-disable an errored trigger or schedule —
+that's orthogonal to this rule, which governs git-sync/merge/create/update
+writes.)

-The trigger update handler complements this: when an incoming `update_trigger`
-request omits both `mode` and `enabled`, the existing DB value is preserved
-instead of falling back to the BaseTriggerData default of `Enabled`. This
-means the fork→parent merge cannot flip the parent's operational state, even
-if the fork has an explicit (locally-disabled) state for that path.
+This is enforced in two halves, keyed off `parent_workspace_id IS NOT NULL`
+(the column, not the `wm-fork-*` naming convention — it stays consistent with
+the conflict-warning gates and survives any future ID rename):

-The schedule `EditSchedule` payload already lacks an `enabled` field, so its
-update path is naturally safe.
+**Read half — parent-value substitution on export.** When the source workspace
+is a fork, the tarball export at `/api/w/{workspace}/workspaces/tarball`
+rewrites each trigger's `mode` (and each schedule's `enabled`) to the
+*parent's* value for the same path, looked up at export time. A fork-only path
+(absent from the parent) keeps the fork's own value — there's no parent state
+to defer to, so it lands with whatever the fork creator set.
+
+The earlier design *stripped* these fields instead. That broke a normal-git PR
+merge: the parent branch (and the merge base) carries the line, the fork branch
+dropped it, so the 3-way merge either silently deleted `mode`/`enabled` from
+the parent — corrupting the source of truth — or conflicted outright when the
+parent had also edited it. Substituting the parent's value makes the fork's
+file byte-identical to the parent on that field, so the merge has nothing to
+resolve.
+
+**Write half — fork writes never set operational state.** A write into a fork
+(git-sync push, merge deploy, clone, or a plain UI create) must not set the
+state, otherwise pulling the substituted parent value straight back into the
+fork would re-enable it. So `create_trigger`/`create_schedule` force `disabled`
+for a fork target, and `update_trigger` preserves the fork's existing `mode`
+(`workspace_is_fork` in `windmill-trigger/src/handler.rs`; schedule `enabled` is
+naturally preserved because `EditSchedule` has no `enabled` field). The same
+handlers serve both merge paths, so the two can't diverge. The fork owner
+re-enables locally via `setmode`/`setenabled` (which carry the conflict
+warning below).
+
+For a non-fork target the incoming value is applied as given — so a fork→parent
+merge of an existing trigger writes the parent's own (substituted) value (a
+no-op), and a fork-only trigger lands with the fork creator's chosen state.

 ## Conflict warning on enable

@@ -161,22 +187,23 @@ but `tally_deployed_object_changes` still records mutations against them; the
 deploy will fail at the workspace-collision check if the user tries to
 deploy a non-workspaced row to a fork.

-Operational state (`mode` for triggers, `enabled` for schedules) is handled
-asymmetrically between update and create:
+Operational state (`mode` for triggers, `enabled` for schedules) follows the
+same "owned by the parent" rule as the git round-trip (see *Operational state
+is owned by the parent* above) — the two paths share the backend `create`/
+`update` handlers, so they can't diverge.

- **Update**: the merge deploy strips `mode`/`enabled` so the target's existing
-  state is preserved. Triggers rely on the backend's `is_mode_unspecified()`
-  safeguard in `update_trigger`; schedules rely on `EditSchedule` lacking the
-  `enabled` field. Both deploy paths perform the strip
-  (`stripOperationalState` in `utils_deployable.ts`, `preparePayload` in
-  `merge.ts`). This matches the YAML round-trip's `fork_trigger_ignore_keys`
-  / `fork_schedule_ignore_keys`.
- **Create**: the source's `mode`/`enabled` is passed through. There's no
-  target row to preserve, so a fork-only trigger or schedule lands with the
-  state the fork creator chose. When the source omits the flag entirely (e.g.
-  legacy clients), the backend defaults to `enabled` for both kinds —
-  `BaseTriggerData::mode()` returns `Enabled` and the schedule insert defaults
-  to `true` to match.
+- **Update**: the merge deploy strips `mode`/`enabled`
+  (`stripOperationalStateOnUpdate` in the shared `windmill-utils-internal`
+  package, `cli/windmill-utils-internal/src/deploy.ts`), so the target's
+  existing value is preserved — equivalent to substituting the target's value.
+  For a fork target the backend preserves it regardless (`workspace_is_fork`);
+  for a parent target the `is_mode_unspecified()` safeguard does. Schedules also
+  rely on `EditSchedule` lacking the `enabled` field.
+- **Create**: the source's `mode`/`enabled` is passed through. Into a **parent**
+  there's no row to preserve, so a fork-only trigger/schedule lands with the
+  state the fork creator chose (omitting the flag defaults to `enabled`:
+  `BaseTriggerData::mode()` → `Enabled`, schedule insert → `true`). Into a
+  **fork** the backend forces `disabled` — a fork write never enables anything.

 ## Future work — runtime listener suffix

@@ -1,12 +1,12 @@
 {
 	"name": "@windmill-labs/components",
-	"version": "1.717.1",
+	"version": "1.720.0",
 	"lockfileVersion": 3,
 	"requires": true,
 	"packages": {
 		"": {
 			"name": "@windmill-labs/components",
-			"version": "1.717.1",
+			"version": "1.720.0",
 			"hasInstallScript": true,
 			"license": "AGPL-3.0",
 			"dependencies": {
@@ -1,6 +1,6 @@
 {
 	"name": "@windmill-labs/components",
-	"version": "1.717.1",
+	"version": "1.720.0",
 	"scripts": {
 		"dev": "vite dev",
 		"dev:ui-builder": "mv static/ui_builder static/ui_builder.dev-disabled 2>/dev/null || true ; trap 'mv static/ui_builder.dev-disabled static/ui_builder 2>/dev/null || true' EXIT ; vite dev",
@@ -13,6 +13,7 @@
 		type ResourceType
 	} from '$lib/gen'
 	import { emptyString, truncateRev, urlize } from '$lib/utils'
+	import oauthConnectRegistry from '$oauth_connect_registry'
 	import { createEventDispatcher, onDestroy } from 'svelte'
 	import Path from './Path.svelte'
 	import { Button, Skeleton } from './common'
@@ -489,12 +490,25 @@
 				throw Error(`Resource at path ${path} already exists. Delete it or pick another path`)
 			}

-			if (resourceType == 'snowflake_oauth') {
-				const account_identifier = extra_params.find(([key, _]) => key == 'account_identifier')
-				if (account_identifier) {
-					args['account_identifier'] = account_identifier[1]
+			// Per-instance OAuth providers (Snowflake, ServiceNow, …): copy the
+			// admin-configured instance from the OAuth client's extra_params into the
+			// resource args, per the registry template's resource_mapping (e.g.
+			// ServiceNow -> instance_url: https://{instance}.service-now.com). Generic
+			// so a new per-instance provider needs only a registry entry.
+			const connectTemplate = (oauthConnectRegistry as Record<string, any>)[resourceType]
+				?.connect_config_template
+			if (connectTemplate?.resource_mapping) {
+				const instanceKey = connectTemplate.extra_params_key ?? 'instance'
+				const found = extra_params.find(([key, _]) => key === instanceKey)
+				if (found) {
+					for (const [argField, valueTemplate] of Object.entries(
+						connectTemplate.resource_mapping as Record<string, string>
+					)) {
+						args[argField] = valueTemplate.replaceAll('{instance}', found[1])
+					}
 				}
-			} else if (resourceType === 'quickbooks' && responseExtra['realmId']) {
+			}
+			if (resourceType === 'quickbooks' && responseExtra['realmId']) {
 				args['realmId'] = responseExtra['realmId']
 			}

@@ -29,7 +29,10 @@
 	import oauthConnectRegistry from '$oauth_connect_registry'

 	interface Props {
-		snowflakeAccountIdentifier?: string
+		// Per-instance OAuth providers (Snowflake, ServiceNow, …): instance name
+		// keyed by provider, used to build their per-instance connect_config URLs.
+		// Required (and always bound by InstanceSettings) so it is never undefined.
+		instanceInputs: Record<string, string>
 		oauths?: Record<string, any>
 		requirePreexistingUserForOauth?: boolean
 		baseUrl?: string
@@ -39,7 +42,7 @@
 	}

 	let {
-		snowflakeAccountIdentifier = $bindable(),
+		instanceInputs = $bindable(),
 		oauths = $bindable(),
 		requirePreexistingUserForOauth = $bindable(),
 		baseUrl,
@@ -49,9 +52,6 @@
 	}: Props = $props()

 	$effect(() => {
-		if (snowflakeAccountIdentifier == undefined) {
-			snowflakeAccountIdentifier = ''
-		}
 		if (oauths == undefined) {
 			oauths = {}
 		}
@@ -79,7 +79,6 @@
 		'visma',
 		'sage_intacct',
 		'spotify',
-		'snowflake_oauth',
 		'teams',
 		'zoho',
 		'xero',
@@ -96,9 +95,20 @@
 	const windmillBuiltinsWithSandbox = Object.entries(oauthConnectRegistry)
 		.filter(([, cfg]) => cfg && typeof cfg === 'object' && 'sandbox' in cfg)
 		.map(([name]) => name)
+	// Per-instance providers (Snowflake, ServiceNow, …): registry entries that
+	// carry a `connect_config_template`. Derived from the registry so adding a
+	// new one needs only a JSON entry — they get a builtin tile + the generic
+	// instance-name input below, with no frontend change.
+	const connectConfigTemplates: Record<string, any> = Object.fromEntries(
+		Object.entries(oauthConnectRegistry)
+			.filter(([, cfg]) => cfg && typeof cfg === 'object' && 'connect_config_template' in cfg)
+			.map(([name, cfg]) => [name, (cfg as any).connect_config_template])
+	)
+	const windmillBuiltinsTemplated = Object.keys(connectConfigTemplates)
 	const windmillBuiltins = [
 		...windmillBuiltinsBase,
-		...windmillBuiltinsWithSandbox.map((n) => `${n}_sandbox`)
+		...windmillBuiltinsWithSandbox.map((n) => `${n}_sandbox`),
+		...windmillBuiltinsTemplated
 	]

 	let showCustomOAuthForm = $state(false)
@@ -238,6 +248,20 @@
 			}
 		})

+		// Add per-instance providers (registry entries with a connect_config_template)
+		windmillBuiltinsTemplated.forEach((name) => {
+			if (!oauths || !oauths[name]) {
+				const icon = getOAuthProviderIcon(name)
+				items.push({
+					// Prefer the template's display_name (properly cased, e.g. "ServiceNow")
+					// over capitalize(name) which yields "Servicenow"/"Snowflake_oauth".
+					displayName: connectConfigTemplates[name]?.display_name ?? capitalize(name),
+					action: () => createOAuthClient(name),
+					icon: icon
+				})
+			}
+		})
+
 		// Add custom option
 		items.push({
 			displayName: `Custom OAuth client ${!$enterpriseLicense ? '(requires ee)' : ''}`,
@@ -486,19 +510,23 @@
 								{:else if !windmillBuiltins.includes(k) && k != 'slack'}
 									<CustomOauth bind:connect_config={oauths[k]['connect_config']} />
 								{/if}
-								{#if k == 'snowflake_oauth'}
+								{#if connectConfigTemplates[k]}
+									{@const tmpl = connectConfigTemplates[k]}
 									<label class="block pb-2">
-										<span class="text-primary font-semibold text-xs flex gap-2 items-center"
-											><a
-												href="https://docs.snowflake.com/en/user-guide/admin-account-identifier#using-an-account-name-as-an-identifier"
-												target="_blank">Snowflake Account Identifier</a
-											><ExternalLink size={12} /></span
-										>
+										<span class="text-primary font-semibold text-xs flex gap-2 items-center">
+											{#if tmpl.help_url}
+												<a href={tmpl.help_url} target="_blank">{tmpl.label}</a><ExternalLink
+													size={12}
+												/>
+											{:else}
+												{tmpl.label}
+											{/if}
+										</span>
 										<input
 											type="text"
-											placeholder="<orgname>-<account_name>"
+											placeholder={tmpl.placeholder}
 											required={true}
-											bind:value={snowflakeAccountIdentifier}
+											bind:value={instanceInputs[k]}
 										/>
 									</label>
 								{/if}
@@ -0,0 +1,399 @@
+<script lang="ts">
+	import WorkspaceDeployLayout from './WorkspaceDeployLayout.svelte'
+	import DiffDrawer from './DiffDrawer.svelte'
+	import WorkspaceDeployItemSummary from './WorkspaceDeployItemSummary.svelte'
+	import { Badge } from './common'
+	import Button from './common/button/Button.svelte'
+	import ConfirmationModal from './common/confirmationModal/ConfirmationModal.svelte'
+	import { ArrowRight, DiffIcon, GitFork, Pencil, Undo2 } from 'lucide-svelte'
+	import { untrack } from 'svelte'
+	import CompareModeToggle, { type CompareMode } from './CompareModeToggle.svelte'
+	import { editUrlFor } from './sessions/forkEditUrl'
+	import { AppService, FlowService, ScriptService, type WorkspaceItemDiff } from '$lib/gen'
+	import { sendUserToast } from '$lib/toast'
+	import { getDraftDiffValues, deployDraft, discardDraft } from '$lib/utils_draft_deploy'
+	import { type DraftItem } from '$lib/workspaceDrafts.svelte'
+
+	interface Props {
+		currentWorkspaceId: string
+		/** The Draft Items to review, owned by the page's Workspace Drafts resource
+		 * and passed down so we don't mount a second resource (which would double
+		 * the list fetches). Deploy/discard here invalidate that resource, so the
+		 * list refetches upstream and the new `draftItems` flow back in. */
+		draftItems: DraftItem[]
+		/** True while the page's Workspace Drafts resource is loading. */
+		draftsLoading?: boolean
+		/** Fork context drives the merged toggle: only a fork offers the
+		 * deploy_to/update directions, so the toggle is hidden otherwise. */
+		isFork?: boolean
+		parentWorkspaceId?: string
+		deployCount?: number
+		updateCount?: number
+		draftCount?: number
+		/** Selecting deploy_to/update asks the page to swap to CompareWorkspaces. */
+		onModeSelected?: (v: CompareMode) => void
+		/** Fired after a deploy/discard so the page can refresh the *fork*
+		 * comparison (ahead/behind). The Draft Count refreshes itself — deploy/
+		 * discard invalidate the Workspace Drafts resource. */
+		onChanged?: () => void
+	}
+
+	let {
+		currentWorkspaceId,
+		draftItems,
+		draftsLoading = false,
+		isFork = false,
+		parentWorkspaceId,
+		deployCount = 0,
+		updateCount = 0,
+		draftCount = 0,
+		onModeSelected,
+		onChanged
+	}: Props = $props()
+
+	type Row = {
+		kind: DraftItem['kind']
+		path: string
+		summary?: string
+		draft_only: boolean
+		raw_app: boolean
+		key: string
+	}
+	function getItemKey(kind: string, path: string): string {
+		return `${kind}:${path}`
+	}
+
+	// The list (and the Draft Count) come from the shared Workspace Drafts module,
+	// owned by the page and passed in via `draftItems`; deploy/discard invalidate
+	// that resource, so the list refetches and deployed items drop off without a
+	// manual reload here.
+	const items: Row[] = $derived(draftItems.map((d) => ({ ...d, key: getItemKey(d.kind, d.path) })))
+
+	// The Draft Items list only carries the *deployed* summary, so the draft's
+	// (new) display name isn't known yet. Fetch each item's draft blob once and
+	// cache both names — mirrors CompareWorkspaces' fetchSummaries (eager on load,
+	// keyed by row key) so the rename rendering is shared and consistent. Only
+	// non-`draft_only` items can show a rename: a `draft_only` item has no deployed
+	// side to diff the name against. Raw apps live on a separate route and aren't
+	// fetchable here, so they're skipped (no rename shown, same as before).
+	const summaryCache = $state<
+		Record<string, { deployed?: string; draft?: string; loading?: boolean }>
+	>({})
+
+	async function fetchDraftSummary(item: Row) {
+		if (summaryCache[item.key]) return
+		summaryCache[item.key] = { loading: true }
+		try {
+			const r = (await (item.kind === 'script'
+				? ScriptService.getScriptByPathWithDraft({ workspace: currentWorkspaceId, path: item.path })
+				: item.kind === 'flow'
+					? FlowService.getFlowByPathWithDraft({ workspace: currentWorkspaceId, path: item.path })
+					: AppService.getAppByPathWithDraft({
+							workspace: currentWorkspaceId,
+							path: item.path
+						}))) as any
+			summaryCache[item.key] = {
+				deployed: r.summary,
+				draft: (r.draft as any)?.summary,
+				loading: false
+			}
+		} catch (error) {
+			console.error(`Failed to fetch draft summary for ${item.kind}:${item.path}`, error)
+			summaryCache[item.key] = { loading: false }
+		}
+	}
+
+	$effect(() => {
+		const current = items
+		untrack(() => {
+			for (const item of current) {
+				if (!item.draft_only && !item.raw_app && !summaryCache[item.key]) {
+					void fetchDraftSummary(item)
+				}
+			}
+		})
+	})
+
+	let selectedItems = $state<string[]>([])
+	let deploying = $state(false)
+	// Select all on the first non-empty load (deploy-all is the common intent);
+	// only once, so a refetch after a deploy doesn't re-select the leftovers.
+	let hasAutoSelected = $state(false)
+
+	const deploymentStatus: Record<
+		string,
+		{ status: 'loading' | 'deployed' | 'failed'; error?: string }
+	> = $state({})
+
+	// Prune transient deploy status for items no longer in the list (a deployed
+	// item drops off after the resource refetches). Keeps the map from growing
+	// unbounded and avoids a stale 'deployed' entry suppressing a row if the same
+	// kind:path is re-drafted within this mount.
+	$effect(() => {
+		const live = new Set(items.map((i) => i.key))
+		untrack(() => {
+			for (const key of Object.keys(deploymentStatus)) {
+				if (!live.has(key)) delete deploymentStatus[key]
+			}
+		})
+	})
+
+	$effect(() => {
+		if (!hasAutoSelected && items.length > 0) {
+			selectedItems = items
+				.filter((i) => deploymentStatus[i.key]?.status !== 'deployed')
+				.map((i) => i.key)
+			hasAutoSelected = true
+		}
+	})
+
+	// Selected items still in the live list and deployable. Derived (not a pruning
+	// effect) so the "Deploy N drafts" button stays reactive to the Workspace
+	// Drafts resource: deploy/discard drop items, and stale keys left in
+	// selectedItems are simply ignored here (and by deploySelected).
+	let selectedCount = $derived(
+		items.filter(
+			(i) => selectedItems.includes(i.key) && deploymentStatus[i.key]?.status !== 'deployed'
+		).length
+	)
+
+	let allSelected = $derived(
+		items.length > 0 &&
+			items
+				.filter((i) => deploymentStatus[i.key]?.status !== 'deployed')
+				.every((i) => selectedItems.includes(i.key))
+	)
+
+	function toggleItem(item: { key: string }) {
+		if (selectedItems.includes(item.key)) {
+			selectedItems = selectedItems.filter((k) => k !== item.key)
+		} else {
+			selectedItems = [...selectedItems, item.key]
+		}
+	}
+
+	function selectAll() {
+		selectedItems = items
+			.filter((i) => deploymentStatus[i.key]?.status !== 'deployed')
+			.map((i) => i.key)
+	}
+
+	function deselectAll() {
+		selectedItems = []
+	}
+
+	// --- Diff ---
+	let diffDrawer: DiffDrawer | undefined = $state(undefined)
+	let isFlow = $state(false)
+	// Monotonic token so that two quick "Show diff" clicks don't race: a slower
+	// earlier fetch must not overwrite a faster later one in the (single) drawer.
+	let diffRequestId = 0
+
+	async function showDiff(item: Row) {
+		if (!diffDrawer) return
+		const reqId = ++diffRequestId
+		isFlow = item.kind === 'flow'
+		diffDrawer.openDrawer()
+		const { deployed, draft } = await getDraftDiffValues(
+			item.kind,
+			item.path,
+			currentWorkspaceId,
+			item.draft_only
+		)
+		// A newer Show-diff click superseded this one — drop the stale result.
+		if (reqId !== diffRequestId) return
+		diffDrawer.setDiff({
+			mode: 'simple',
+			original: deployed as any,
+			current: draft as any,
+			title: 'Deployed → Draft'
+		})
+	}
+
+	// --- Deploy ---
+	async function deploySelected() {
+		deploying = true
+		// Snapshot the items to deploy: deployDraft invalidates the Workspace Drafts
+		// resource, so `items` can change mid-loop — iterate a stable copy.
+		const toDeploy = items.filter((i) => selectedItems.includes(i.key))
+		let deployedAny = false
+		for (const item of toDeploy) {
+			deploymentStatus[item.key] = { status: 'loading' }
+			const res = await deployDraft(
+				item.kind,
+				item.path,
+				currentWorkspaceId,
+				item.draft_only,
+				item.raw_app
+			)
+			if (res.success) {
+				deploymentStatus[item.key] = { status: 'deployed' }
+				deployedAny = true
+			} else {
+				deploymentStatus[item.key] = { status: 'failed', error: res.error }
+				sendUserToast(`Failed to deploy ${item.path}: ${res.error}`, true)
+			}
+		}
+		deploying = false
+		selectedItems = []
+		// The Draft list refetches itself (deployDraft invalidated it). Deploying
+		// also changes the fork comparison (ahead/behind) — ask the page to refresh
+		// that.
+		if (deployedAny) onChanged?.()
+	}
+
+	// --- Discard ---
+	let discardTarget = $state<Row | undefined>(undefined)
+
+	async function confirmDiscard() {
+		const item = discardTarget
+		discardTarget = undefined
+		if (!item) return
+		const res = await discardDraft(item.kind, item.path, currentWorkspaceId, item.draft_only)
+		if (res.success) {
+			sendUserToast(item.draft_only ? `Deleted ${item.path}` : `Discarded draft of ${item.path}`)
+			// discardDraft invalidated the Draft list; refresh the fork comparison.
+			onChanged?.()
+		} else {
+			sendUserToast(`Failed to discard ${item.path}: ${res.error}`, true)
+		}
+	}
+
+	// Editor URL for a draft item, scoped to the current workspace. Raw apps live
+	// under a different editor route, so map their kind accordingly.
+	function draftEditUrl(d: Row): string | undefined {
+		return editUrlFor(
+			{ kind: d.raw_app ? 'raw_app' : d.kind, path: d.path } as unknown as WorkspaceItemDiff,
+			currentWorkspaceId
+		)
+	}
+</script>
+
+<div class="flex flex-col gap-4">
+	<div class="bg-surface-tertiary p-4 rounded-md border">
+		<WorkspaceDeployLayout
+			{items}
+			{selectedItems}
+			{deploymentStatus}
+			{allSelected}
+			selectablePredicate={(item) => deploymentStatus[item.key]?.status !== 'deployed'}
+			onToggleItem={toggleItem}
+			onSelectAll={selectAll}
+			onDeselectAll={deselectAll}
+			emptyMessage={draftsLoading ? 'Loading drafts…' : 'No drafts in this workspace'}
+		>
+			{#snippet header()}
+				{#if isFork}
+					<div class="flex flex-wrap gap-1 items-center bg-surface-tertiary pb-4">
+						<CompareModeToggle
+							selected="draft"
+							{isFork}
+							{parentWorkspaceId}
+							{deployCount}
+							{updateCount}
+							{draftCount}
+							disabled={deploying}
+							onSelected={(v) => onModeSelected?.(v)}
+						/>
+						<!-- Direction badge, mirroring the fork compare header: make it explicit
+						     that deploying a draft promotes it *within this fork* (deployed↔draft),
+						     not up to the parent. -->
+						<div class="flex-1 flex gap-1 items-center">
+							<Badge color="transparent" class="ml-5 font-semibold">
+								<span class="text-secondary">deploy:</span>
+								<Pencil size={14} />
+								<span class="text-emphasis">draft</span>
+							</Badge>
+							<ArrowRight size={16} />
+							<Badge color="transparent" class="font-semibold" title={currentWorkspaceId}>
+								<span class="text-secondary">into:</span>
+								<GitFork size={14} />
+								<span class="text-emphasis">{currentWorkspaceId}</span>
+							</Badge>
+						</div>
+					</div>
+				{/if}
+			{/snippet}
+
+			{#snippet itemSummary(item)}
+				{@const draftItem = item as unknown as Row}
+				{@const editUrl = draftEditUrl(draftItem)}
+				{@const cache = summaryCache[draftItem.key]}
+				{@const oldSummary = cache?.deployed ?? draftItem.summary}
+				{@const newSummary = cache?.draft ?? draftItem.summary}
+				<WorkspaceDeployItemSummary
+					path={draftItem.path}
+					{editUrl}
+					{oldSummary}
+					{newSummary}
+					renamed={!draftItem.draft_only &&
+						oldSummary != null &&
+						newSummary != null &&
+						oldSummary !== newSummary}
+				/>
+			{/snippet}
+
+			{#snippet itemActions(item)}
+				{@const draftItem = item as unknown as Row}
+				{#if draftItem.draft_only}
+					<Badge color="indigo" size="xs">New</Badge>
+				{/if}
+				{#if deploymentStatus[draftItem.key]?.status !== 'deployed'}
+					<Button
+						unifiedSize="xs"
+						variant="subtle"
+						startIcon={{ icon: DiffIcon }}
+						onClick={() => showDiff(draftItem)}
+					>
+						Show diff
+					</Button>
+					<Button
+						unifiedSize="xs"
+						variant="subtle"
+						destructive
+						startIcon={{ icon: Undo2 }}
+						onClick={() => (discardTarget = draftItem)}
+					>
+						Discard draft
+					</Button>
+				{/if}
+			{/snippet}
+
+			{#snippet footer()}
+				<div class="flex items-center justify-end">
+					<Button
+						variant="accent"
+						disabled={selectedCount === 0 || deploying}
+						loading={deploying}
+						onClick={deploySelected}
+					>
+						Deploy {selectedCount} draft{selectedCount !== 1 ? 's' : ''}
+					</Button>
+				</div>
+			{/snippet}
+		</WorkspaceDeployLayout>
+	</div>
+
+	<DiffDrawer bind:this={diffDrawer} {isFlow} />
+</div>
+
+<ConfirmationModal
+	open={discardTarget !== undefined}
+	title={discardTarget?.draft_only ? 'Delete item' : 'Discard draft'}
+	confirmationText={discardTarget?.draft_only ? 'Delete' : 'Discard'}
+	onConfirmed={confirmDiscard}
+	onCanceled={() => (discardTarget = undefined)}
+>
+	{#if discardTarget?.draft_only}
+		<p>
+			<span class="font-mono font-medium text-primary">{discardTarget?.path}</span> exists only as a
+			draft. Discarding it will permanently delete the item. This cannot be undone.
+		</p>
+	{:else}
+		<p>
+			Discard the draft of
+			<span class="font-mono font-medium text-primary">{discardTarget?.path}</span>? The deployed
+			version is unaffected.
+		</p>
+	{/if}
+</ConfirmationModal>
@@ -0,0 +1,61 @@
+<script lang="ts" module>
+	// The merged compare control: fork direction (deploy_to / update) and the
+	// deployed↔draft comparison live in one toggle. `deploy_to`/`update` only
+	// apply in a fork; a non-fork workspace has draft as the sole option (the
+	// caller hides the toggle entirely in that case).
+	export type CompareMode = 'deploy_to' | 'update' | 'draft'
+</script>
+
+<script lang="ts">
+	import ToggleButtonGroup from './common/toggleButton-v2/ToggleButtonGroup.svelte'
+	import ToggleButton from './common/toggleButton-v2/ToggleButton.svelte'
+	import { ArrowUp, ArrowDown, Pencil } from 'lucide-svelte'
+
+	interface Props {
+		selected: CompareMode
+		isFork: boolean
+		parentWorkspaceId?: string
+		deployCount?: number
+		updateCount?: number
+		draftCount?: number
+		disabled?: boolean
+		onSelected: (v: CompareMode) => void
+	}
+
+	let {
+		selected,
+		isFork,
+		parentWorkspaceId,
+		deployCount = 0,
+		updateCount = 0,
+		draftCount = 0,
+		disabled = false,
+		onSelected
+	}: Props = $props()
+
+	// Append a count suffix only when there is something to act on, mirroring the
+	// draft toggle (no "(0)" noise).
+	function withCount(label: string, count: number): string {
+		return count > 0 ? `${label} (${count})` : label
+	}
+</script>
+
+<ToggleButtonGroup {disabled} {selected} onSelected={(v) => onSelected(v as CompareMode)} noWFull>
+	{#snippet children({ item })}
+		{#if isFork}
+			<ToggleButton
+				value="deploy_to"
+				label={withCount(`Deploy to ${parentWorkspaceId}`, deployCount)}
+				icon={ArrowUp}
+				{item}
+			/>
+			<ToggleButton
+				value="update"
+				label={withCount('Update current', updateCount)}
+				icon={ArrowDown}
+				{item}
+			/>
+		{/if}
+		<ToggleButton value="draft" label={`Deploy draft (${draftCount})`} icon={Pencil} {item} />
+	{/snippet}
+</ToggleButtonGroup>
@@ -1,10 +1,8 @@
 <script lang="ts">
 	import {
 		AlertTriangle,
-		ArrowDown,
 		ArrowDownRight,
 		ArrowRight,
-		ArrowUp,
 		ArrowUpRight,
 		Building,
 		CircleCheck,
@@ -29,7 +27,9 @@
 		type WorkspaceItemDiff
 	} from '$lib/gen'
 	import Button from './common/button/Button.svelte'
+	import ConfirmationModal from './common/confirmationModal/ConfirmationModal.svelte'
 	import DiffDrawer from './DiffDrawer.svelte'
+	import WorkspaceDeployItemSummary from './WorkspaceDeployItemSummary.svelte'
 	import ParentWorkspaceProtectionAlert from './ParentWorkspaceProtectionAlert.svelte'
 	import { userWorkspaces, workspaceStore } from '$lib/stores'

@@ -54,22 +54,58 @@
 	import DeploymentRequestPanel from './deploymentRequest/DeploymentRequestPanel.svelte'
 	import { userStore } from '$lib/stores'
 	import { base } from '$lib/base'
-	import ToggleButtonGroup from './common/toggleButton-v2/ToggleButtonGroup.svelte'
-	import ToggleButton from './common/toggleButton-v2/ToggleButton.svelte'
+	import CompareModeToggle, { type CompareMode } from './CompareModeToggle.svelte'
+	import { editUrlFor } from './sessions/forkEditUrl'
 	import DatatableSchemaDiff from './DatatableSchemaDiff.svelte'

 	interface Props {
 		currentWorkspaceId: string
 		parentWorkspaceId: string
 		comparison: WorkspaceComparison | undefined
+		/** Initial merge direction; lets the page restore the chosen direction when
+		 * switching back from draft mode (deploy_to → true, update → false). */
+		initialMergeIntoParent?: boolean
+		/** Per-direction counts for the merged toggle badges (the page owns them). */
+		deployCount?: number
+		updateCount?: number
+		/** Draft count for the merged toggle's badge (the page owns it). */
+		draftCount?: number
+		/** Keys (`kind:path`) of fork items that are deployed *and* have a pending
+		 * draft (has_draft). Such rows get a "+Draft" warning badge and are left
+		 * out of the default selection — deploying/updating moves the deployed
+		 * version, not the draft. The page derives this from the fork drafts. */
+		draftKeys?: Set<string>
+		/** Selecting `draft` asks the page to swap us out for CompareDrafts;
+		 * deploy_to/update are handled internally but reported so the page can
+		 * remember the direction. */
+		onModeSelected?: (v: CompareMode) => void
+		/** Fired after a deploy/update so the page re-fetches the comparison and
+		 * draft count, keeping the toggle badges in sync with the new state. */
+		onChanged?: () => void
 	}

-	let { currentWorkspaceId, parentWorkspaceId, comparison }: Props = $props()
+	let {
+		currentWorkspaceId,
+		parentWorkspaceId,
+		comparison,
+		initialMergeIntoParent = true,
+		deployCount = 0,
+		updateCount = 0,
+		draftCount = 0,
+		draftKeys = new Set<string>(),
+		onModeSelected,
+		onChanged
+	}: Props = $props()
+
+	// A fork row has a pending draft when its key is in the page-provided set.
+	function hasDraft(diff: WorkspaceItemDiff): boolean {
+		return draftKeys.has(getItemKey(diff))
+	}

 	let currentWorkspaceInfo = $derived($userWorkspaces.find((w) => w.id == currentWorkspaceId))
 	let parentWorkspaceInfo = $derived($userWorkspaces.find((w) => w.id == parentWorkspaceId))

-	let mergeIntoParent = $state(true)
+	let mergeIntoParent = $state(initialMergeIntoParent)
 	let deploying = $state(false)
 	let hasAutoSelected = $state(false)
 	let canDeployToParent = $state(true)
@@ -89,6 +125,32 @@

 	let selectedItems = $state<string[]>([])

+	// Selected items that carry a pending draft. They're opt-in (excluded from the
+	// default selection), so a non-empty list means the user explicitly picked an
+	// item whose draft won't be included — confirm before deploying.
+	let selectedDraftKeys = $derived(selectedItems.filter((k) => draftKeys.has(k)))
+	let draftConfirmOpen = $state(false)
+
+	function requestDeploy() {
+		if (selectedDraftKeys.length > 0) {
+			draftConfirmOpen = true
+		} else {
+			deployChanges()
+		}
+	}
+
+	// Nothing actionable in the current direction (no items ahead to deploy, or
+	// none behind to update). When so we show a message instead of a table of
+	// greyed, non-actionable rows.
+	let nothingToAct = $derived(selectableDiffs.length === 0)
+	let emptyDeployMessage = $derived(
+		(comparison?.diffs.length ?? 0) === 0
+			? 'No changes between this fork and its parent.'
+			: mergeIntoParent
+				? `Nothing to deploy — ${parentWorkspaceId} already has every change from this fork.`
+				: `Nothing to update — this fork is up to date with ${parentWorkspaceId}.`
+	)
+
 	let conflictingDiffs = $derived(
 		comparison?.diffs.filter((diff) => diff.ahead > 0 && diff.behind > 0) ?? []
 	)
@@ -419,6 +481,10 @@
 				console.error('Failed to close open deployment request after merge', e)
 			}
 		}
+
+		// Deployed items are now in sync and should drop off the comparison; ask
+		// the page to re-fetch so the list and toggle badges reflect the new state.
+		onChanged?.()
 	}

 	function toggleKey(key: string) {
@@ -434,7 +500,9 @@
 		// parent (kafka group_id, postgres replication slot, schedule firing time)
 		// and pushing them by default would surprise users running a routine "Deploy
 		// to parent" flow. The user picks them à la carte by clicking the row.
-		const filtered = selectableDiffs.filter((d) => !isTriggerOrScheduleKind(d.kind))
+		// Items with a pending draft are also left out by default: the deployed
+		// version (not the draft) is what moves, so we make the user opt in.
+		const filtered = selectableDiffs.filter((d) => !isTriggerOrScheduleKind(d.kind) && !hasDraft(d))
 		const conflictSafe = mergeIntoParent
 			? filtered
 			: filtered.filter((d) => !(d.ahead > 0 && d.behind > 0))
@@ -449,6 +517,15 @@
 		selectDefault()
 	}

+	// Merged toggle: deploy_to/update flip the direction in place; draft asks the
+	// page to swap us out for CompareDrafts. Either way report it up so the page
+	// remembers the chosen direction across mode switches.
+	function onToggleMode(v: CompareMode) {
+		onModeSelected?.(v)
+		if (v === 'draft') return
+		toggleDeploymentDirection(v)
+	}
+
 	// Fetch user permissions for both workspaces
 	$effect(() => {
 		;[currentWorkspaceId, parentWorkspaceId]
@@ -601,7 +678,7 @@
 	<div class="flex flex-col gap-4">
 		<div class="bg-surface-tertiary p-4 rounded-md border">
 			<WorkspaceDeployLayout
-				items={deployableItems}
+				items={nothingToAct ? [] : deployableItems}
 				{selectedItems}
 				{deploymentStatus}
 				selectablePredicate={(item) => selectableDiffs.some((d) => getItemKey(d) === item.key)}
@@ -609,28 +686,22 @@
 				onToggleItem={(item) => toggleKey(item.key)}
 				onSelectAll={selectAll}
 				onDeselectAll={deselectAll}
-				emptyMessage="No comparison data available"
+				emptyMessage={emptyDeployMessage}
 			>
 				{#snippet header()}
 					<div class="flex items-center justify-between bg-surface-tertiary">
-						<div class="flex flex-col gap-2 w-full pb-4 border-b">
+						<div class="flex flex-col gap-2 w-full pb-4">
 							<div class="flex flex-wrap gap-1 items-center">
-								<ToggleButtonGroup
+								<CompareModeToggle
+									selected={mergeIntoParent ? 'deploy_to' : 'update'}
+									isFork={true}
+									{parentWorkspaceId}
+									{deployCount}
+									{updateCount}
+									{draftCount}
 									disabled={deploying}
-									selected="deploy_to"
-									onSelected={toggleDeploymentDirection}
-									noWFull
-								>
-									{#snippet children({ item })}
-										<ToggleButton
-											value="deploy_to"
-											label="Deploy to {parentWorkspaceId}"
-											icon={ArrowUp}
-											{item}
-										/>
-										<ToggleButton value="update" label="Update current" icon={ArrowDown} {item} />
-									{/snippet}
-								</ToggleButtonGroup>
+									onSelected={onToggleMode}
+								/>
 								{#if currentWorkspaceInfo && parentWorkspaceInfo}
 									<div class="flex-1 flex gap-1 items-center">
 										<Badge
@@ -719,6 +790,24 @@
 				{/if}

 				{#snippet alerts()}
+					{#if draftCount > 0}
+						<Alert title="Undeployed drafts" type="warning" size="xs" class="my-2">
+							<div class="flex items-center gap-2 flex-wrap">
+								<span>
+									{#if mergeIntoParent}
+										This workspace has {draftCount} undeployed draft{draftCount !== 1 ? 's' : ''}.
+										Only deployed versions in this fork can be sent to {parentWorkspaceId} — deploy
+										{draftCount !== 1 ? 'them' : 'it'} first, otherwise those changes won't be included.
+									{:else}
+										This workspace has {draftCount} undeployed draft{draftCount !== 1 ? 's' : ''}.
+									{/if}
+								</span>
+								<Button variant="subtle" unifiedSize="xs" onclick={() => onModeSelected?.('draft')}>
+									See drafts
+								</Button>
+							</div>
+						</Alert>
+					{/if}
 					{#if mergeIntoParent}
 						<ParentWorkspaceProtectionAlert
 							{parentWorkspaceId}
@@ -780,6 +869,13 @@
 				{#snippet itemSummary(item)}
 					{@const diff = item.diff as WorkspaceItemDiff}
 					{@const key = item.key}
+					<!-- Point the edit link at the workspace the item actually lives in:
+					     a parent-only row (deleted/absent in the fork) would 404 if linked
+					     into the fork, so link it into the parent instead. -->
+					{@const editUrl = editUrlFor(
+						diff,
+						diff.exists_in_fork ? currentWorkspaceId : parentWorkspaceId
+					)}
 					{#if isTriggerOrScheduleKind(diff.kind)}
 						<span class="text-emphasis">
 							{KIND_DISPLAY_NAMES[diff.kind as string] ?? diff.kind}
@@ -798,14 +894,13 @@
 							(diff.exists_in_fork && !diff.exists_in_source) ||
 							(!diff.exists_in_fork && diff.exists_in_source)
 						)}
-						{#if oldSummary != newSummary && isSelectable && existsInBothWorkspaces}
-							<span class="line-through text-secondary">{oldSummary || diff.path}</span>
-							{newSummary || diff.path}
-						{:else if !existsInBothWorkspaces}
-							{newSummary || oldSummary || diff.path}
-						{:else}
-							{newSummary || diff.path}
-						{/if}
+						<WorkspaceDeployItemSummary
+							path={diff.path}
+							{editUrl}
+							{oldSummary}
+							{newSummary}
+							renamed={oldSummary != newSummary && isSelectable && existsInBothWorkspaces}
+						/>
 					{/if}
 				{/snippet}

@@ -836,6 +931,21 @@
 					{#if diff.kind === 'raw_app'}
 						<Badge small icon={{ icon: FileJson }}>Raw</Badge>
 					{/if}
+					{#if hasDraft(diff)}
+						<!-- This deployed fork item also has a pending draft. Deploying/updating
+						     moves the deployed version, not the draft — so we warn (yellow, ahead
+						     of the New/status badges) and leave it out of the default selection
+						     (see selectDefault). -->
+						<Badge
+							title={mergeIntoParent
+								? 'This item has a draft — deploying sends the deployed version, not the draft.'
+								: 'This item has a draft — updating replaces the deployed version your draft is based on.'}
+							color="yellow"
+							size="xs"
+						>
+							<AlertTriangle class="w-3 h-3 inline mr-0.5" />+Draft
+						</Badge>
+					{/if}
 					<!-- Status badges -->
 					{#if !diff.exists_in_fork && diff.exists_in_source && diff.ahead == 0 && diff.behind > 0}
 						<Badge
@@ -898,11 +1008,11 @@
 						</div>
 						<div class:invisible={!existsInBothWorkspaces}>
 							<Button
-								size="xs"
+								unifiedSize="xs"
 								variant="subtle"
-								onclick={() => showDiff(diff.kind as Kind, diff.path)}
+								startIcon={{ icon: DiffIcon }}
+								onClick={() => showDiff(diff.kind as Kind, diff.path)}
 							>
-								<DiffIcon class="w-3 h-3" />
 								Show diff
 							</Button>
 						</div>
@@ -910,63 +1020,65 @@
 				{/snippet}

 				{#snippet footer()}
-					<div class="flex items-center justify-between">
-						<div></div>
+					{#if !nothingToAct}
+						<div class="flex items-center justify-between">
+							<div></div>

-						<div class="flex flex-col items-end gap-2">
-							{#if comparison.all_behind_items_visible && comparison.all_ahead_items_visible}
-								<div class="flex items-center gap-2">
-									{#if mergeIntoParent && !hasOpenDeploymentRequest && !deploymentRequestPanel?.isDialogOpen()}
-										<Button
-											variant="default"
-											startIcon={{ icon: UserPlus }}
-											on:click={() => deploymentRequestPanel?.openRequestDialog()}
-										>
-											Request deployment
-										</Button>
-									{/if}
-									<Button
-										variant="accent"
-										disabled={selectedItems.length === 0 ||
-											deploying ||
-											(hasBehindChanges && !allowBehindChangesOverride) ||
-											(mergeIntoParent && !canDeployToParent) ||
-											hasUnselectedOnBehalfOf}
-										loading={deploying}
-										on:click={deployChanges}
-									>
-										{mergeIntoParent ? 'Deploy' : 'Update'}
-										{selectedItems.length} Item{selectedItems.length !== 1 ? 's' : ''}
-										{#if selectedConflicts != 0}
-											({selectedConflicts} conflicts)
+							<div class="flex flex-col items-end gap-2">
+								{#if comparison.all_behind_items_visible && comparison.all_ahead_items_visible}
+									<div class="flex items-center gap-2">
+										{#if mergeIntoParent && !hasOpenDeploymentRequest && !deploymentRequestPanel?.isDialogOpen()}
+											<Button
+												variant="default"
+												startIcon={{ icon: UserPlus }}
+												on:click={() => deploymentRequestPanel?.openRequestDialog()}
+											>
+												Request deployment
+											</Button>
 										{/if}
-									</Button>
-								</div>
-								{#if !(mergeIntoParent && !canDeployToParent) && hasUnselectedOnBehalfOf}
-									<span class="text-xs text-yellow-600">
-										You must set the "on behalf of" user for all items before deploying
-										<Tooltip class="text-yellow-600">
-											The "run on behalf of" field defines which user's permissions will be applied
-											during execution. Make sure this is set to an appropriate user before
-											deploying.
-										</Tooltip>
-									</span>
+										<Button
+											variant="accent"
+											disabled={selectedItems.length === 0 ||
+												deploying ||
+												(hasBehindChanges && !allowBehindChangesOverride) ||
+												(mergeIntoParent && !canDeployToParent) ||
+												hasUnselectedOnBehalfOf}
+											loading={deploying}
+											on:click={requestDeploy}
+										>
+											{mergeIntoParent ? 'Deploy' : 'Update'}
+											{selectedItems.length} Item{selectedItems.length !== 1 ? 's' : ''}
+											{#if selectedConflicts != 0}
+												({selectedConflicts} conflicts)
+											{/if}
+										</Button>
+									</div>
+									{#if !(mergeIntoParent && !canDeployToParent) && hasUnselectedOnBehalfOf}
+										<span class="text-xs text-yellow-600">
+											You must set the "on behalf of" user for all items before deploying
+											<Tooltip class="text-yellow-600">
+												The "run on behalf of" field defines which user's permissions will be
+												applied during execution. Make sure this is set to an appropriate user
+												before deploying.
+											</Tooltip>
+										</span>
+									{/if}
 								{/if}
-							{/if}

-							{#if deploymentErrorMessage != ''}
-								<Alert
-									title="Cannot {mergeIntoParent ? 'deploy these changes' : 'update these items'}"
-									type="error"
-									class="my-2 max-w-80"
-								>
-									<span>
-										{deploymentErrorMessage}
-									</span>
-								</Alert>
-							{/if}
+								{#if deploymentErrorMessage != ''}
+									<Alert
+										title="Cannot {mergeIntoParent ? 'deploy these changes' : 'update these items'}"
+										type="error"
+										class="my-2 max-w-80"
+									>
+										<span>
+											{deploymentErrorMessage}
+										</span>
+									</Alert>
+								{/if}
+							</div>
 						</div>
-					</div>
+					{/if}
 				{/snippet}
 			</WorkspaceDeployLayout>

@@ -988,6 +1100,35 @@
 	</div>

 	<DiffDrawer bind:this={diffDrawer} {isFlow} />
+
+	<ConfirmationModal
+		open={draftConfirmOpen}
+		title={mergeIntoParent ? 'Deploy items with a draft?' : 'Update items with a draft?'}
+		confirmationText={mergeIntoParent ? 'Deploy anyway' : 'Update anyway'}
+		onConfirmed={() => {
+			draftConfirmOpen = false
+			deployChanges()
+		}}
+		onCanceled={() => (draftConfirmOpen = false)}
+	>
+		<div class="flex flex-col gap-2">
+			<p>
+				{selectedDraftKeys.length} selected item{selectedDraftKeys.length !== 1 ? 's' : ''}
+				{selectedDraftKeys.length !== 1 ? 'have' : 'has'} an undeployed draft.
+				{#if mergeIntoParent}
+					Deploying sends the deployed version, not the draft — those draft changes won't be
+					included.
+				{:else}
+					Updating replaces the deployed version your draft is based on.
+				{/if}
+			</p>
+			<ul class="list-disc pl-5 text-sm font-mono text-secondary">
+				{#each selectedDraftKeys as k (k)}
+					<li>{k.split(':').slice(1).join(':')}</li>
+				{/each}
+			</ul>
+		</div>
+	</ConfirmationModal>
 {:else}
 	<div class="flex items-center justify-center h-full">
 		<div class="text-gray-500">No comparison data available</div>
@@ -410,6 +410,23 @@
 		return json
 	}

+	// The explicit column order from a leading header row (see
+	// handleArrayOfObjectsHeaders). Returned as an array so the order survives:
+	// baking it into object keys loses integer-like names like "1234", which JS
+	// enumerates first in ascending numeric order.
+	function getForcedColumnOrder(json: any): string[] | undefined {
+		if (
+			Array.isArray(json) &&
+			json.length > 0 &&
+			Array.isArray(json[0]) &&
+			json[0].length > 0 &&
+			json[0].every((item) => typeof item === 'string')
+		) {
+			return json[0]
+		}
+		return undefined
+	}
+
 	type InputObject = { [key: string]: number[] }

 	function objectOfArraysToObjects(input: InputObject): any[] {
@@ -643,6 +660,7 @@
 							? 'absolute inset-0 [&>div]:h-full [&>div]:min-h-[10rem]'
 							: ''}
 						objects={handleArrayOfObjectsHeaders(data)}
+						headerOrder={getForcedColumnOrder(data)}
 					/>
 				{:else if !forceJson && resultKind === 'html'}
 					<div class="h-full">
@@ -0,0 +1,532 @@
+<!--
+@component
+Generic drill-through picker. Renders a tree of branches and leaves; one
+level is shown at a time. The host supplies the tree shape — workspace
+items, chat context elements, etc. all map to the same component.
+
+  - **Root** (no scope): the tree's top-level entries.
+  - **Branch** (scope = `[...keys]`): the children of the branch resolved
+    by walking the tree along the scope chain.
+
+Clicking a row drills *down*; the chevron-left in the header walks one
+level *up*. Filter (internal or `externalFilter`) is global across all
+leaves and ignores the current scope.
+-->
+<script lang="ts" generics="L">
+	import { ChevronLeft, ChevronRight, Loader2 } from 'lucide-svelte'
+	import TextInput from '$lib/components/text_input/TextInput.svelte'
+	import SearchItems from '$lib/components/SearchItems.svelte'
+	import { generateRandomString } from '$lib/utils'
+	import { onMount, untrack, type Snippet } from 'svelte'
+	import {
+		collectLeavesGrouped,
+		leafHaystack,
+		resolveScope,
+		scopeChain,
+		type DrillBranch,
+		type DrillLeaf,
+		type DrillNode
+	} from './drillPicker'
+
+	interface Props {
+		tree: DrillNode<L>[]
+		onPick: (leaf: DrillLeaf<L>) => void
+		/** Drill path to land on initially. Empty = root. */
+		initialScope?: string[]
+		/** Composite key of the row to highlight initially. */
+		initialHighlight?: string
+		/** When set (any string, incl. ''), the host owns search input.
+		 * The internal search field is hidden and the host is expected to
+		 * forward keydown events via `handleKeydown`. */
+		externalFilter?: string
+		autoFocus?: boolean
+		/** Drop the outer fixed width / max height wrapper. */
+		flush?: boolean
+		/** Custom renderer for the icon column of a leaf. */
+		leafIcon?: Snippet<[DrillLeaf<L>]>
+		/** Custom renderer for the icon column of a branch (used inside
+		 * the entry row AND as the leading icon in the breadcrumb header). */
+		branchIcon?: Snippet<[DrillBranch<L>]>
+		/** Override for a leaf row's secondary text in the **drilled view**
+		 * (search results always show `leaf.secondary` to keep absolute
+		 * paths visible globally). Returns `undefined` to defer to
+		 * `leaf.secondary`. Used by the workspace adapter to render
+		 * scope-relative paths once the user has drilled into a folder. */
+		leafSecondary?: (leaf: DrillLeaf<L>, scope: string[]) => string | undefined
+		/** Fires whenever the scope changes. Lets the host trigger lazy
+		 * data loading for the branch the user drilled into. */
+		onScopeChange?: (scope: string[]) => void
+		/** Fires whenever the EFFECTIVE filter changes (internal OR external).
+		 * Lets the host trigger global preloads when the user starts searching
+		 * — needed in internal-filter mode where the host can't observe the
+		 * picker's own search box otherwise. */
+		onFilterChange?: (filter: string) => void
+	}
+
+	let {
+		tree,
+		onPick,
+		initialScope,
+		initialHighlight,
+		externalFilter,
+		autoFocus = true,
+		flush = false,
+		leafIcon,
+		branchIcon,
+		leafSecondary,
+		onScopeChange,
+		onFilterChange
+	}: Props = $props()
+
+	let searchInput: TextInput | undefined = $state()
+	let pickerRoot: HTMLElement | undefined = $state()
+	const instanceId = generateRandomString(8)
+	const listboxId = `dpkr-list-${instanceId}`
+	const idFor = (key: string) => `dpkr-${instanceId}-${key.replace(/[^a-zA-Z0-9-]/g, '_')}`
+
+	export function focus() {
+		searchInput?.focus()
+	}
+
+	// Sibling-popover open: melt-ui's `openFocus` runs once during the
+	// close→open transition; the picker may not be mounted yet. Retry
+	// after settle. Skipped when `autoFocus` is false (host keeps focus)
+	// or when the search input is not rendered (external filter mode).
+	onMount(() => {
+		if (!autoFocus || externalFilter !== undefined) return
+		const t = setTimeout(focus, 50)
+		return () => clearTimeout(t)
+	})
+
+	let scope = $state<string[]>(untrack(() => initialScope ?? []))
+	let internalFilter = $state('')
+	const filter = $derived(externalFilter ?? internalFilter)
+	const isSearching = $derived(filter.trim() !== '')
+
+	$effect(() => {
+		void scope
+		onScopeChange?.(scope)
+	})
+
+	$effect(() => {
+		void filter
+		onFilterChange?.(filter)
+	})
+
+	/** Tracks whether the last user action was mouse movement (true) or
+	 * keyboard nav (false). When false, row `mouseenter` events are
+	 * ignored — prevents the cursor from stealing the keyboard-driven
+	 * highlight as rows shift under it during scope changes. Re-enabled
+	 * on `mousemove`. Starts `false` so the synthetic `mouseenter` fired
+	 * when the popover mounts under a stationary cursor doesn't clobber
+	 * `initialHighlight`. */
+	let mouseActive = $state(false)
+
+	const currentBranch = $derived(resolveScope(tree, scope))
+	const entries = $derived<DrillNode<L>[]>(
+		scope.length === 0 ? tree : (currentBranch?.children ?? [])
+	)
+
+	// Flat leaf pool for global search. Skips branches flagged
+	// `omitFromSearch` (e.g. workspace cross-kind 'all' branch).
+	const searchPool = $derived(collectLeavesGrouped(tree))
+	type SearchEntry = { leaf: DrillLeaf<L>; group: DrillBranch<L> | null; _key: string }
+	const searchItems = $derived<SearchEntry[]>(
+		searchPool.map(({ leaf, group }) => ({ leaf, group, _key: leaf.key }))
+	)
+	let searchedItems: (SearchEntry & { marked: string })[] | undefined = $state(undefined)
+
+	// Group filtered results by their nearest-branch ancestor for display.
+	const searchResultsByGroup = $derived.by(() => {
+		const groups = new Map<
+			string,
+			{ group: DrillBranch<L> | null; items: (SearchEntry & { marked: string })[] }
+		>()
+		if (!searchedItems) return [] as { group: DrillBranch<L> | null; items: SearchEntry[] }[]
+		for (const r of searchedItems) {
+			const gkey = r.group?.key ?? '__none'
+			const existing = groups.get(gkey)
+			if (existing) existing.items.push(r)
+			else groups.set(gkey, { group: r.group, items: [r] })
+		}
+		return Array.from(groups.values())
+	})
+
+	type Entry =
+		| { type: 'branch'; key: string; node: DrillBranch<L> }
+		| { type: 'leaf'; key: string; node: DrillLeaf<L> }
+
+	const entryList = $derived<Entry[]>(
+		entries.map((n) =>
+			n.type === 'branch'
+				? { type: 'branch' as const, key: n.key, node: n }
+				: { type: 'leaf' as const, key: n.key, node: n }
+		)
+	)
+
+	const navKeys = $derived(
+		isSearching
+			? (searchedItems ?? ([] as typeof searchItems)).map((r) => r.leaf.key)
+			: entryList.map((e) => e.key)
+	)
+
+	let highlightedKey = $state<string | undefined>(untrack(() => initialHighlight))
+	let highlightedId = $derived(highlightedKey ? idFor(highlightedKey) : undefined)
+
+	$effect(() => {
+		if (navKeys.length === 0) return
+		if (!highlightedKey || !navKeys.includes(highlightedKey)) {
+			highlightedKey = navKeys[0]
+		}
+	})
+
+	$effect(() => {
+		if (highlightedKey && navKeys.includes(highlightedKey)) {
+			requestAnimationFrame(scrollHighlightIntoView)
+		}
+	})
+
+	function scrollHighlightIntoView() {
+		if (!pickerRoot || !highlightedKey) return
+		const el = pickerRoot.querySelector<HTMLElement>(
+			`[data-nav-key="${CSS.escape(highlightedKey)}"]`
+		)
+		el?.scrollIntoView({ block: 'nearest', behavior: 'smooth' })
+	}
+
+	function moveHighlight(delta: 1 | -1) {
+		if (navKeys.length === 0) return
+		const cur = navKeys.indexOf(highlightedKey ?? '')
+		const next = cur < 0 ? 0 : (cur + delta + navKeys.length) % navKeys.length
+		highlightedKey = navKeys[next]
+		mouseActive = false
+		requestAnimationFrame(scrollHighlightIntoView)
+	}
+
+	function setHoverHighlight(key: string) {
+		// Ignored until the user actually moves the mouse. Prevents the
+		// cursor (parked over a row) from clobbering keyboard-driven
+		// selection when the layout shifts beneath it.
+		if (mouseActive) highlightedKey = key
+	}
+
+	function pick(leaf: DrillLeaf<L>) {
+		if (leaf.current || leaf.disabled) return
+		onPick(leaf)
+	}
+
+	function activate(key: string | undefined) {
+		if (!key) return
+		if (isSearching) {
+			const found = (searchedItems ?? []).find((r) => r.leaf.key === key)
+			if (found) pick(found.leaf)
+			return
+		}
+		const entry = entryList.find((e) => e.key === key)
+		if (!entry) return
+		drill(entry)
+	}
+
+	function drill(entry: Entry) {
+		if (entry.type === 'branch') {
+			scope = [...scope, entry.key]
+		} else {
+			pick(entry.node)
+		}
+	}
+
+	function goUp() {
+		if (scope.length === 0) return
+		const leaving = scope[scope.length - 1]
+		scope = scope.slice(0, -1)
+		highlightedKey = leaving
+	}
+
+	function handleSearchKeydown(e: KeyboardEvent) {
+		if (e.key === 'ArrowDown') {
+			e.preventDefault()
+			e.stopPropagation()
+			moveHighlight(1)
+		} else if (e.key === 'ArrowUp') {
+			e.preventDefault()
+			e.stopPropagation()
+			moveHighlight(-1)
+		} else if (e.key === 'Enter' || e.key === 'Tab') {
+			// Tab mirrors Enter so the inline `@<word>` mention completes
+			// without losing focus to the next form control (matches the
+			// previous `AvailableContextList` behavior). Guard on a
+			// highlighted row so an unrelated Tab in an empty picker still
+			// falls through to natural focus movement.
+			if (e.key === 'Tab' && !highlightedKey) return
+			e.preventDefault()
+			e.stopPropagation()
+			mouseActive = false
+			activate(highlightedKey)
+		} else if (
+			(e.key === 'ArrowLeft' || e.key === 'Backspace') &&
+			filter === '' &&
+			scope.length > 0
+		) {
+			// Walk up the tree. Only when search is empty — otherwise these
+			// keys would hijack cursor movement / character deletion.
+			e.preventDefault()
+			e.stopPropagation()
+			mouseActive = false
+			goUp()
+		} else if (e.key === 'ArrowRight' && filter === '' && !isSearching) {
+			// Drill into the highlighted branch. Leaves are reserved for
+			// Enter (more deliberate, since picking navigates away).
+			const entry = entryList.find((en) => en.key === highlightedKey)
+			if (entry && entry.type === 'branch') {
+				e.preventDefault()
+				e.stopPropagation()
+				mouseActive = false
+				drill(entry)
+			}
+		}
+	}
+
+	export function handleKeydown(e: KeyboardEvent) {
+		handleSearchKeydown(e)
+	}
+
+	export function pickHighlighted() {
+		activate(highlightedKey)
+	}
+
+	// Breadcrumb header — labels of branches along the scope chain.
+	const headerChain = $derived(scopeChain(tree, scope))
+	const headerSegments = $derived(headerChain.map((b) => b.label))
+	const headerLabel = $derived(headerSegments.join(' › '))
+
+	/** Number of intermediate segments to hide behind a `…`. Always keep
+	 * the first segment (kind / category) and the deepest. Bumped up by
+	 * the measurement effect below. */
+	let hiddenCount = $state(0)
+
+	const headerLabelDisplay = $derived.by(() => {
+		if (headerSegments.length <= 2 || hiddenCount === 0) return headerLabel
+		return [headerSegments[0], '…', ...headerSegments.slice(1 + hiddenCount)].join(' › ')
+	})
+
+	let breadcrumbSpan: HTMLElement | undefined = $state()
+	let lastSegmentsKey = ''
+
+	/** Measurement loop: each pass reads `scrollWidth > clientWidth` on
+	 * the truncated span; if overflowing and there's still an intermediate
+	 * segment to drop, increment `hiddenCount`. Mutating `hiddenCount`
+	 * re-renders and re-fires this effect, so the loop self-terminates
+	 * either when the text fits or when only [first, …, leaf] remain
+	 * (`truncate-start` then polishes any final overflow). When the
+	 * breadcrumb itself changes (new scope), reset to 0 first. */
+	$effect(() => {
+		const key = headerSegments.join('|')
+		const segmentsChanged = key !== lastSegmentsKey
+		if (segmentsChanged) {
+			lastSegmentsKey = key
+			if (hiddenCount !== 0) {
+				hiddenCount = 0
+				return
+			}
+		}
+		void hiddenCount
+		if (!breadcrumbSpan) return
+		const maxHide = Math.max(0, headerSegments.length - 2)
+		if (hiddenCount >= maxHide) return
+		queueMicrotask(() => {
+			if (!breadcrumbSpan) return
+			if (breadcrumbSpan.scrollWidth > breadcrumbSpan.clientWidth + 1) {
+				hiddenCount = hiddenCount + 1
+			}
+		})
+	})
+
+	const branchLoading = $derived(currentBranch?.loading ?? false)
+</script>
+
+<SearchItems
+	{filter}
+	items={isSearching ? searchItems : []}
+	bind:filteredItems={searchedItems}
+	f={(x: SearchEntry) => leafHaystack(x.leaf)}
+	opts={{}}
+/>
+
+{#snippet defaultLeafIcon(leaf: DrillLeaf<L>)}
+	{#if leafIcon}
+		{@render leafIcon(leaf)}
+	{:else if leaf.icon}
+		{@const Icon = leaf.icon}
+		<Icon size={12} class="shrink-0" />
+	{/if}
+{/snippet}
+
+{#snippet defaultBranchIcon(branch: DrillBranch<L>)}
+	{#if branchIcon}
+		{@render branchIcon(branch)}
+	{:else if branch.icon}
+		{@const Icon = branch.icon}
+		<Icon size={12} class="shrink-0 text-tertiary" />
+	{/if}
+{/snippet}
+
+{#snippet leafRow(leaf: DrillLeaf<L>, secondary: string | undefined, baseClass: string)}
+	{@const key = leaf.key}
+	{@const isHl = key === highlightedKey}
+	{@const isCur = !!leaf.current}
+	<button
+		type="button"
+		id={idFor(key)}
+		role="option"
+		aria-selected={isHl}
+		data-nav-key={key}
+		aria-current={isCur ? 'true' : undefined}
+		class="w-full text-left flex items-center gap-2 px-3 transition-colors {baseClass} {isHl
+			? 'bg-surface-hover'
+			: ''} {isCur ? 'cursor-default text-emphasis font-medium' : ''} {leaf.disabled
+			? 'opacity-50 cursor-not-allowed'
+			: ''}"
+		disabled={leaf.disabled}
+		onmousedown={(e) => e.preventDefault()}
+		onclick={() => pick(leaf)}
+		onmouseenter={() => setHoverHighlight(key)}
+	>
+		{@render defaultLeafIcon(leaf)}
+		<div class="min-w-0 flex-1">
+			{#if leaf.secondary}
+				<div class="text-xs text-primary font-normal truncate">{leaf.label}</div>
+				<div class="text-2xs text-hint font-normal font-mono truncate">
+					{secondary ?? leaf.secondary}
+				</div>
+			{:else}
+				<div class="text-xs text-primary font-normal font-mono truncate">
+					{secondary ?? leaf.label}
+				</div>
+			{/if}
+		</div>
+	</button>
+{/snippet}
+
+<!-- svelte-ignore a11y_no_static_element_interactions -->
+<div
+	bind:this={pickerRoot}
+	class={flush
+		? 'flex flex-col w-full h-full'
+		: 'flex flex-col w-[min(420px,calc(100vw-20px))] max-h-[60vh]'}
+	onkeydown={handleSearchKeydown}
+	onmousemove={() => (mouseActive = true)}
+>
+	{#if externalFilter === undefined}
+		<div class="px-3 py-2 border-b border-gray-200 dark:border-gray-700">
+			<TextInput
+				bind:this={searchInput}
+				bind:value={internalFilter}
+				size="sm"
+				inputProps={{
+					placeholder: 'Search by name or summary...',
+					'data-workspace-picker-search': '',
+					role: 'combobox',
+					'aria-controls': listboxId,
+					'aria-expanded': 'true',
+					'aria-autocomplete': 'list',
+					'aria-activedescendant': highlightedId
+				}}
+			/>
+		</div>
+	{/if}
+
+	{#if scope.length > 0 && !isSearching}
+		<button
+			type="button"
+			class="flex items-center gap-1.5 w-full text-left px-3 py-1 text-xs font-medium font-mono text-secondary bg-surface-secondary/20 hover:bg-surface-hover transition-colors"
+			onmousedown={(e) => e.preventDefault()}
+			onclick={goUp}
+			title={headerLabel}
+		>
+			<ChevronLeft size={12} class="shrink-0 text-secondary" />
+			{#if headerChain[0]}
+				{@render defaultBranchIcon(headerChain[0])}
+			{/if}
+			<span bind:this={breadcrumbSpan} class="flex-1 min-w-0 truncate truncate-start">
+				{headerLabelDisplay}
+			</span>
+		</button>
+	{/if}
+
+	<div class="flex-1 overflow-y-auto" role="listbox" id={listboxId}>
+		{#if isSearching}
+			{@const total = (searchedItems ?? []).length}
+			{#if !searchedItems}
+				<div role="status" class="px-3 py-2 text-xs text-tertiary flex items-center gap-2">
+					<Loader2 size={14} class="animate-spin" /> Searching…
+				</div>
+			{:else if total === 0}
+				<div role="status" class="px-3 py-2 text-xs text-tertiary">No matches</div>
+			{:else}
+				{#each searchResultsByGroup as { group, items } (group?.key ?? '__none')}
+					{#if group}
+						<div class="px-3 pt-3 pb-1 text-2xs uppercase tracking-wide text-tertiary font-medium">
+							{group.label}
+						</div>
+					{/if}
+					<ul class="pb-1">
+						{#each items as r (r.leaf.key)}
+							<li>{@render leafRow(r.leaf, r.leaf.secondary ?? r.leaf.label, 'py-1.5')}</li>
+						{/each}
+					</ul>
+				{/each}
+			{/if}
+		{:else if branchLoading && entryList.length === 0}
+			<div role="status" class="px-3 py-2 text-xs text-tertiary flex items-center gap-2">
+				<Loader2 size={14} class="animate-spin" /> Loading…
+			</div>
+		{:else if entryList.length === 0}
+			<div role="status" class="px-3 py-2 text-xs text-tertiary">Empty</div>
+		{:else}
+			<div class="flex flex-col py-1">
+				{#each entryList as entry (entry.key)}
+					{@const isHl = entry.key === highlightedKey}
+					{#if entry.type === 'leaf'}
+						{@render leafRow(
+							entry.node,
+							leafSecondary?.(entry.node, scope) ?? entry.node.secondary,
+							'py-1.5'
+						)}
+					{:else}
+						<button
+							type="button"
+							id={idFor(entry.key)}
+							role="option"
+							aria-selected={isHl}
+							data-nav-key={entry.key}
+							class="flex items-center gap-1.5 w-full text-left px-3 py-1.5 text-xs font-medium font-mono text-emphasis transition-colors {isHl
+								? 'bg-surface-hover'
+								: ''}"
+							onmousedown={(e) => e.preventDefault()}
+							onclick={() => drill(entry)}
+							onmouseenter={() => setHoverHighlight(entry.key)}
+						>
+							{@render defaultBranchIcon(entry.node)}
+							<span class="flex-1 truncate">{entry.node.label}</span>
+							{#if entry.node.loading}
+								<Loader2 size={12} class="animate-spin text-tertiary" />
+							{/if}
+							<ChevronRight size={10} class="shrink-0 text-secondary" />
+						</button>
+					{/if}
+				{/each}
+			</div>
+		{/if}
+	</div>
+</div>
+
+<style>
+	/* Path truncates from the start (left ellipsis) so the deepest
+	 * (rightmost) folder stays visible. `unicode-bidi: plaintext` keeps
+	 * each path segment laid out per its own direction. */
+	.truncate-start {
+		direction: rtl;
+		text-align: left;
+		unicode-bidi: plaintext;
+	}
+</style>
@@ -0,0 +1,34 @@
+<script lang="ts">
+	import { ExternalLink } from 'lucide-svelte'
+	import type { Snippet } from 'svelte'
+
+	// Shared "open in a new tab" link used by the compare/diff row titles
+	// (CompareDrafts, CompareWorkspaces, WorkspaceDiffDrawer). Wraps the common
+	// boilerplate — target/rel, the click-through stopPropagation (so following
+	// the link doesn't toggle row selection), and the hover-revealed external
+	// icon. Each caller supplies its own title text, extra classes, and inner
+	// label content via the `children` snippet.
+	let {
+		href,
+		title,
+		class: klass = '',
+		children
+	}: {
+		href: string
+		title?: string
+		class?: string
+		children: Snippet
+	} = $props()
+</script>
+
+<a
+	{href}
+	{title}
+	target="_blank"
+	rel="noopener noreferrer"
+	onclick={(e) => e.stopPropagation()}
+	class="group inline-flex items-center gap-1 max-w-full hover:underline {klass}"
+>
+	{@render children()}
+	<ExternalLink class="w-3 h-3 shrink-0 opacity-0 group-hover:opacity-60 transition-opacity" />
+</a>
@@ -133,7 +133,8 @@
 		onSaveDraftError,
 		onSaveDraftOnlyAtNewPath,
 		onHistoryRestore,
-		onNavigate
+		onNavigate,
+		onTestJob
 	}: FlowBuilderProps = $props()

 	let initialPathStore = writable(initialPath)
@@ -735,7 +736,18 @@
 		flowStore.val = redo(history)
 	}

+	let flowBuilderRoot: HTMLDivElement | undefined = $state()
+
 	function onKeyDown(event: KeyboardEvent) {
+		// Defer to anything that has explicitly grabbed focus — menus, modals,
+		// drawers etc. live outside the flow root. Flow nodes aren't focusable,
+		// so the unfocused default (activeElement === body) means "flow is the
+		// canvas" and we should react.
+		const active = document.activeElement
+		if (active && active !== document.body && !flowBuilderRoot?.contains(active)) {
+			return
+		}
+
 		let classes = event.target?.['className']
 		if (
 			(typeof classes === 'string' && classes.includes('inputarea')) ||
@@ -1175,7 +1187,7 @@
 		<ScriptEditorDrawer bind:this={$scriptEditorDrawer} />
 		<FlowEditorDrawer bind:this={$flowEditorDrawer} />

-		<div class="flex flex-col flex-1 h-screen">
+		<div bind:this={flowBuilderRoot} class="flex flex-col flex-1 h-screen">
 			<!-- Nav between steps-->
 			<div
 				bind:clientWidth={topbarWidth}
@@ -1256,11 +1268,14 @@
 					bind:localModuleStates
 					bind:this={flowPreviewButtons}
 					{loading}
-					onRunPreview={() => {
+					onRunPreview={(jobId) => {
 						stepsInputArgs.resetManuallyEditedArgs()
 						modulesTestStates.hideJobsInGraph()
 						localModuleStates = {}
 						showJobStatus = true
+						if (jobId) {
+							onTestJob?.({ jobId })
+						}
 					}}
 				/>
 			{/snippet}
@@ -59,7 +59,7 @@
 		scrollTop?: number
 		localModuleStates?: Record<string, GraphModuleState>
 		localDurationStatuses?: Record<string, DurationStatus>
-		onRunPreview?: () => void
+		onRunPreview?: (jobId?: string) => void
 		render?: boolean
 		onJobDone?: () => void
 		upToId?: string | undefined
@@ -200,7 +200,7 @@
 				savedArgs = $state.snapshot(previewArgs.val)
 				inputSelected = undefined
 			}
-			onRunPreview?.()
+			onRunPreview?.(newJobId)
 		} catch (e) {
 			sendUserToast('Could not run preview', true, undefined, e.toString())
 			isRunning = false
@@ -6,6 +6,7 @@
 	import { AlertTriangle, GitFork, CircleCheck, CircleX, Loader2 } from 'lucide-svelte'
 	import { goto } from '$app/navigation'
 	import { onMount, untrack } from 'svelte'
+	import { useWorkspaceDrafts } from '$lib/workspaceDrafts.svelte'

 	let loading = $state(false)
 	let comparison: WorkspaceComparison | undefined = $state(undefined)
@@ -16,6 +17,23 @@
 	let parentWorkspaceId = $derived(currentWorkspaceData?.parent_workspace_id)
 	let parentWorkspaceData = $derived($userWorkspaces.find((w) => w.id === parentWorkspaceId))

+	// Drafts in this fork. When the fork is otherwise in sync with its parent, a
+	// user with only pending drafts should still get the draft CTA (mirrors the
+	// non-fork WorkspaceDraftsBanner). Pass undefined when not a fork so it doesn't
+	// fetch.
+	const drafts = useWorkspaceDrafts(() => (isFork ? ($workspaceStore ?? undefined) : undefined))
+	const draftCount = $derived(drafts.count)
+
+	// Fork is fully in sync with its parent (comparison ran, no ahead/behind diffs).
+	// Typed helper avoids the $state `never`-inference quirk on `comparison` in $derived.
+	function isUpToDate(c: WorkspaceComparison | undefined): boolean {
+		return !!c && !c.skipped_comparison && c.summary.total_diffs === 0
+	}
+	let upToDate = $derived(isUpToDate(comparison))
+	// Up to date with the parent but local drafts are pending — show the draft
+	// state (same text + CTA as the draft banner) instead of "Everything is up to date".
+	let showDraftsOnly = $derived(upToDate && draftCount > 0)
+
 	$effect(() => {
 		;[$workspaceStore, parentWorkspaceId]
 		untrack(() => {
@@ -68,6 +86,14 @@
 		}
 	}

+	function openDraftCompare() {
+		if ($workspaceStore) {
+			goto('/forks/compare?workspace_id=' + encodeURIComponent($workspaceStore) + '&mode=draft', {
+				replaceState: true
+			})
+		}
+	}
+
 	let ciTestPassing = $state(0)
 	let ciTestFailing = $state(0)
 	let ciTestRunning = $state(0)
@@ -270,6 +296,10 @@
 									This fork was created before the addition of certain windmill features, and
 									therefore the changes with its parent workspace cannot be displayed.</span
 								>
+							{:else if showDraftsOnly}
+								<span class="text-blue-700 dark:text-blue-100">
+									This workspace has {draftCount} draft{draftCount !== 1 ? 's' : ''}
+								</span>
 							{:else}
 								<span class="text-blue-600 dark:text-blue-200"> Everything is up to date </span>
 							{/if}
@@ -278,8 +308,14 @@
 				</div>

 				<div class="flex items-center gap-2">
-					<Button size="xs" color="blue" on:click={openComparisonDrawer}>
-						{#if (comparison?.summary.total_ahead ?? 0) > 0}
+					<Button
+						variant="default"
+						unifiedSize="sm"
+						onclick={showDraftsOnly ? openDraftCompare : openComparisonDrawer}
+					>
+						{#if showDraftsOnly}
+							Review & deploy drafts
+						{:else if (comparison?.summary.total_ahead ?? 0) > 0}
 							Review & Deploy Changes
 						{:else}
 							Review & Update fork
@@ -13,6 +13,7 @@
 	import { createEventDispatcher } from 'svelte'
 	import { setLicense } from '$lib/enterpriseUtils'
 	import AuthSettings from './AuthSettings.svelte'
+	import oauthConnectRegistry from '$oauth_connect_registry'
 	import InstanceSetting from './InstanceSetting.svelte'
 	import { writable, type Writable } from 'svelte/store'
 	import { ExternalLink, Loader2 } from 'lucide-svelte'
@@ -54,7 +55,9 @@

 	let initialValues: Record<string, any> = $state({})
 	let baseUrlIsFallback = $state(false)
-	let snowflakeAccountIdentifier = $state('')
+	// Per-instance OAuth providers (Snowflake, ServiceNow, …): instance name
+	// keyed by provider, used to build their per-instance connect_config URLs.
+	let instanceInputs: Record<string, string> = $state({})
 	let version: string = $state('')
 	let loading = $state(true)

@@ -147,12 +150,8 @@
 		$values = nvalues
 		loading = false

-		// populate snowflake account identifier from db
-		const account_identifier =
-			oauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier
-		if (account_identifier) {
-			snowflakeAccountIdentifier = account_identifier
-		}
+		// populate per-instance OAuth provider inputs (snowflake, servicenow, …) from db
+		loadInstanceInputs(oauths)
 	}

 	export async function saveSettings() {
@@ -162,13 +161,7 @@
 			}
 		}

-		if (
-			oauths?.snowflake_oauth &&
-			oauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier !==
-				snowflakeAccountIdentifier
-		) {
-			setupSnowflakeUrls()
-		}
+		setupTemplatedOauthUrls()

 		// Remove empty or invalid entries for critical error channels
 		$values.critical_error_channels = $values.critical_error_channels.filter((entry: any) => {
@@ -283,19 +276,54 @@
 		}
 	}

-	function setupSnowflakeUrls() {
-		// strip all whitespaces from account identifier
-		snowflakeAccountIdentifier = snowflakeAccountIdentifier.replace(/\s/g, '')
+	// Per-instance OAuth providers (Snowflake, ServiceNow, …) keyed by name ->
+	// their registry connect_config_template. Adding a new one needs only a
+	// registry entry — no code here.
+	const connectConfigTemplates: Record<string, any> = Object.fromEntries(
+		Object.entries(oauthConnectRegistry)
+			.filter(([, cfg]) => cfg && typeof cfg === 'object' && 'connect_config_template' in cfg)
+			.map(([name, cfg]) => [name, (cfg as any).connect_config_template])
+	)

-		const connect_config = {
-			scopes: [],
-			auth_url: `https://${snowflakeAccountIdentifier}.snowflakecomputing.com/oauth/authorize`,
-			token_url: `https://${snowflakeAccountIdentifier}.snowflakecomputing.com/oauth/token-request`,
-			req_body_auth: false,
-			extra_params: { account_identifier: snowflakeAccountIdentifier },
-			extra_params_callback: {}
+	function normalizeInstanceInput(tmpl: any, raw: string): string {
+		let v = (raw ?? '').replace(/\s/g, '')
+		if (tmpl.strip_suffix) {
+			// accept a full host/URL or a bare name -> reduce to the bare instance
+			v = v.replace(/^https?:\/\//, '').replace(/\/.*$/, '')
+			if (v.endsWith(tmpl.strip_suffix)) {
+				v = v.slice(0, -tmpl.strip_suffix.length)
+			}
+		}
+		return v
+	}
+
+	// Build each per-instance provider's connect_config from the admin-entered
+	// instance name + its registry template (substituting {instance} into the
+	// URLs). Replaces the old per-provider setup functions.
+	function setupTemplatedOauthUrls() {
+		for (const [name, tmpl] of Object.entries(connectConfigTemplates)) {
+			if (!oauths?.[name]) continue
+			const key = tmpl.extra_params_key ?? 'instance'
+			const v = normalizeInstanceInput(tmpl, instanceInputs[name] ?? '')
+			instanceInputs[name] = v
+			if (oauths[name].connect_config?.extra_params?.[key] === v) continue
+			oauths[name].connect_config = {
+				scopes: [],
+				auth_url: tmpl.auth_url.replaceAll('{instance}', v),
+				token_url: tmpl.token_url.replaceAll('{instance}', v),
+				req_body_auth: tmpl.req_body_auth ?? false,
+				extra_params: { [key]: v },
+				extra_params_callback: {}
+			}
+		}
+	}
+
+	// Recover the instance-name inputs from a saved oauths config (for load/discard).
+	function loadInstanceInputs(savedOauths: Record<string, any>) {
+		for (const [name, tmpl] of Object.entries(connectConfigTemplates)) {
+			const key = tmpl.extra_params_key ?? 'instance'
+			instanceInputs[name] = savedOauths?.[name]?.connect_config?.extra_params?.[key] ?? ''
 		}
-		oauths['snowflake_oauth'].connect_config = connect_config
 	}

 	let sendingStats = $state(false)
@@ -510,9 +538,7 @@
 		if (category === 'Auth/OAuth/SAML') {
 			oauths = JSON.parse(JSON.stringify(initialOauths))
 			requirePreexistingUserForOauth = initialRequirePreexistingUserForOauth
-			const account_identifier =
-				initialOauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier
-			snowflakeAccountIdentifier = account_identifier ?? ''
+			loadInstanceInputs(initialOauths)
 		} else if (category === 'Registries') {
 			const v = initialValues['workspace_registries']
 			$values['workspace_registries'] = v !== undefined ? JSON.parse(JSON.stringify(v)) : undefined
@@ -524,9 +550,7 @@
 		$values = JSON.parse(JSON.stringify(initialValues))
 		oauths = JSON.parse(JSON.stringify(initialOauths))
 		requirePreexistingUserForOauth = initialRequirePreexistingUserForOauth
-		const account_identifier =
-			initialOauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier
-		snowflakeAccountIdentifier = account_identifier ?? ''
+		loadInstanceInputs(initialOauths)
 		if (yamlMode) {
 			syncFormToYaml()
 		}
@@ -535,13 +559,7 @@
 	export async function saveCategorySettings(category: string) {
 		// Category-specific pre-processing
 		if (category === 'Auth/OAuth/SAML') {
-			if (
-				oauths?.snowflake_oauth &&
-				oauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier !==
-					snowflakeAccountIdentifier
-			) {
-				setupSnowflakeUrls()
-			}
+			setupTemplatedOauthUrls()
 		}

 		if (category === 'Alerts' && $values?.critical_error_channels) {
@@ -1116,7 +1134,7 @@
 		{:else if category == 'Auth/OAuth/SAML'}
 			<AuthSettings
 				bind:oauths
-				bind:snowflakeAccountIdentifier
+				bind:instanceInputs
 				bind:requirePreexistingUserForOauth
 				baseUrl={$values?.base_url}
 				bind:tab={authSubTab}
@@ -259,12 +259,11 @@
 			createdBy: filters.val.user || undefined,
 			scriptPathStart: filters.val.folder ? `f/${filters.val.folder}/` : undefined,
 			jobKinds: jobKinds == '' ? undefined : jobKinds,
-			success:
-				filters.val.status == 'success'
-					? true
-					: filters.val.status == 'failure'
-						? false
-						: undefined,
+			success: filters.val.status == 'success' ? true : undefined,
+			status:
+				filters.val.status == 'failure' || filters.val.status == 'canceled'
+					? filters.val.status
+					: undefined,
 			running:
 				filters.val.status == 'running' || filters.val.status == 'suspended'
 					? true
@@ -677,6 +676,14 @@
 							}}
 							{item}
 						/>
+						<ToggleButton
+							value={'canceled'}
+							tooltip="Canceled"
+							class="whitespace-nowrap"
+							icon={Hourglass}
+							selectedColor="gray"
+							{item}
+						/>
 						{#if filters.val.status == 'waiting'}
 							<ToggleButton
 								value={'waiting'}
@@ -134,6 +134,7 @@
 		onSaveDraftError,
 		onSaveDraft,
 		onNavigate,
+		onTestJob,
 		disableAi,
 		initialTestPanelCollapsed = false,
 		initialPathChosen = false
@@ -1007,21 +1008,6 @@

 	function onScriptLanguageTrigger(lang: 'docker' | 'bunnative' | ScriptLang) {
 		if (lang == 'docker') {
-			if (isCloudHosted()) {
-				sendUserToast(
-					'You cannot use Docker scripts on the multi-tenant platform. Use a dedicated instance or self-host windmill instead.',
-					true,
-					[
-						{
-							label: 'Learn more',
-							callback: () => {
-								window.open('https://www.windmill.dev/docs/advanced/docker', '_blank')
-							}
-						}
-					]
-				)
-				return
-			}
 			template = 'docker'
 		} else if (lang == 'bunnative') {
 			template = 'bunnative'
@@ -1580,7 +1566,7 @@
 													if (script.timeout && script.timeout != undefined) {
 														script.timeout = undefined
 													} else {
-														script.timeout = 300
+														script.timeout = customUi?.defaultTimeout ?? 300
 													}
 												}}
 												options={{
@@ -2099,6 +2085,7 @@
 			{disableAi}
 			bind:selectedTab={selectedInputTab}
 			{customUi}
+			{onTestJob}
 			collabMode
 			edit={initialPath != ''}
 			on:format={() => {
@@ -160,6 +160,9 @@
 		modules?: { [key: string]: ScriptModule } | null
 		editorBarRight?: import('svelte').Snippet
 		enablePreprocessorSnippet?: boolean
+		// Fired whenever a test run is started from this editor, with the
+		// preview job id. Used by whitelabel embedders to track test jobs.
+		onTestJob?: (e: { jobId: string }) => void
 		// When true the right-hand test/run pane mounts collapsed. The user
 		// can still expand it via `toggleTestPanel`. Defaults to false so the
 		// regular /scripts/edit route keeps its current open-by-default UX;
@@ -199,6 +202,7 @@
 		modules = $bindable(undefined),
 		editorBarRight,
 		enablePreprocessorSnippet = false,
+		onTestJob,
 		initialTestPanelCollapsed = false
 	}: Props = $props()

@@ -729,6 +733,9 @@
 			undefined,
 			activeModuleTab !== null ? undefined : modules
 		)
+		if (job) {
+			onTestJob?.({ jobId: job })
+		}
 		logPanel?.setFocusToLogs()
 		return job
 	}
@@ -1357,9 +1364,7 @@
 	// width (Svelte wires a ResizeObserver for bind:clientWidth).
 	let splitContainerWidth = $state(0)
 	const TEST_PANE_MIN_PX = 400
-	const testPaneMinPercent = $derived(
-		paneMinPercent(splitContainerWidth, TEST_PANE_MIN_PX)
-	)
+	const testPaneMinPercent = $derived(paneMinPercent(splitContainerWidth, TEST_PANE_MIN_PX))

 	// Raw user-controlled test size (what the splitter wrote, or what the
 	// toggle set). The size we actually pass to <Pane> is clamped to the
@@ -0,0 +1,43 @@
+<script lang="ts">
+	import ExternalEditLink from './ExternalEditLink.svelte'
+
+	interface Props {
+		/** Path of the item, used as the fallback label and link title. */
+		path: string
+		/** Editor URL; when set the summary becomes a new-tab link. */
+		editUrl?: string
+		/** Deployed/source-side display name (struck through when renamed). */
+		oldSummary?: string
+		/** Draft/target-side display name (the surviving name when renamed). */
+		newSummary?: string
+		/** Render `~~oldSummary~~ newSummary`. The caller decides this from its
+		 * own concepts (fork: exists-in-both & selectable; draft: !draft_only),
+		 * keeping page-specific logic out of this presentational component. */
+		renamed: boolean
+	}
+
+	let { path, editUrl, oldSummary, newSummary, renamed }: Props = $props()
+</script>
+
+{#snippet label()}
+	{#if renamed}
+		<!-- Two names side by side: don't truncate, mirror the fork compare page. -->
+		<span class="line-through text-secondary">{oldSummary || path}</span>
+		{newSummary || path}
+	{:else}
+		<span class="truncate">{newSummary || oldSummary || path}</span>
+	{/if}
+{/snippet}
+
+{#if editUrl}
+	<!-- Truncate the single-name case; let a rename pair render full-width. -->
+	<ExternalEditLink
+		href={editUrl}
+		title="Open {path} in a new tab"
+		class={renamed ? 'text-emphasis' : 'text-emphasis truncate'}
+	>
+		{@render label()}
+	</ExternalEditLink>
+{:else}
+	{@render label()}
+{/if}
@@ -55,6 +55,12 @@

 	let selectableItems = $derived(items.filter(selectablePredicate))
 	let hasSelectableItems = $derived(selectableItems.length > 0)
+
+	// Plain row click and the checkbox both toggle this row in/out — multi-select
+	// is the default, no modifier needed.
+	function handleSelect(item: DeployableItem) {
+		onToggleItem?.(item)
+	}
 </script>

 <div class="flex flex-col h-full">
@@ -73,9 +79,10 @@
 	{#if items.length > 0}
 		<!-- Select all row -->
 		<div class="px-4 py-2 flex items-center justify-between">
-			<div
+			<label
 				class="flex items-center gap-2 text-secondary text-xs"
 				class:opacity-50={!hasSelectableItems}
+				class:cursor-pointer={hasSelectableItems}
 			>
 				<input
 					type="checkbox"
@@ -84,7 +91,7 @@
 					onchange={allSelected ? onDeselectAll : onSelectAll}
 					class="rounded max-w-4 w-full"
 				/> Select all
-			</div>
+			</label>
 		</div>

 		<!-- Items list -->
@@ -98,10 +105,11 @@

 					<Row
 						isSelectable={isSelectable && !isDeployed}
+						selectOnRowClick={true}
 						alignWithSelectable={true}
 						disabled={!isSelectable}
 						selected={isSelected && !isDeployed}
-						onSelect={() => onToggleItem?.(item)}
+						onSelect={() => handleSelect(item)}
 						path={item.kind !== 'resource' &&
 						item.kind !== 'variable' &&
 						item.kind !== 'resource_type'
@@ -0,0 +1,48 @@
+<script lang="ts">
+	import { workspaceStore } from '$lib/stores'
+	import { Button } from './common'
+	import { Pencil } from 'lucide-svelte'
+	import { goto } from '$app/navigation'
+	import { useWorkspaceDrafts } from '$lib/workspaceDrafts.svelte'
+
+	// Surfaces pending drafts (scripts/flows/apps) for the current workspace and
+	// links to the compare page in draft mode. Mutually exclusive with
+	// ForkWorkspaceBanner: that one self-gates on `isFork`, this one on `!isFork`,
+	// so a fork workspace never shows both. In a fork, drafts are discovered via
+	// the on-page "Deployed ↔ draft (N)" toggle badge instead.
+	let isFork = $derived($workspaceStore?.startsWith('wm-fork-') ?? false)
+
+	// Count comes from the shared Workspace Drafts resource (count ≡ the draft
+	// list; refreshes itself on deploy/discard). Pass undefined in a fork or with
+	// no workspace so it doesn't fetch and the banner stays hidden.
+	const drafts = useWorkspaceDrafts(() => (!isFork ? ($workspaceStore ?? undefined) : undefined))
+	const draftCount = $derived(drafts.count)
+
+	function openDraftCompare() {
+		if ($workspaceStore) {
+			goto('/forks/compare?workspace_id=' + encodeURIComponent($workspaceStore) + '&mode=draft', {
+				replaceState: true
+			})
+		}
+	}
+</script>
+
+{#if !isFork && draftCount > 0}
+	<div class="w-full bg-blue-50 dark:bg-blue-900 text-xs rounded-b-md max-w-7xl mx-auto">
+		<div class="px-4 py-2">
+			<div class="flex items-center justify-between">
+				<div class="flex items-center gap-3">
+					<Pencil class="w-4 h-4 text-accent" />
+					<span class="text-sm font-medium text-blue-900 dark:text-blue-100">
+						This workspace has {draftCount} draft{draftCount !== 1 ? 's' : ''}
+					</span>
+				</div>
+				<!-- Same button as the sibling ForkWorkspaceBanner CTA (they sit on the
+				     same home page), kept visually identical on purpose. -->
+				<Button variant="default" unifiedSize="sm" onclick={openDraftCompare}>
+					Review & deploy drafts
+				</Button>
+			</div>
+		</div>
+	</div>
+{/if}
@@ -1,44 +1,30 @@
 <!--
@component
-Drill-through workspace item picker. One level is shown at a time:
+Workspace drill picker — adapter over the generic `DrillPicker`. Preserves
+the workspace-specific public API (kinds, scope = `{ kind, dir? }`,
+currentItem, leaf/branch icons) so callers (BreadcrumbSegment, EditorHeader)
+don't need to know about the generic tree model underneath.

-  - **Root** (no scope): All + kinds (Flows / Scripts / Apps). "All" is a
-    cross-kind row — drilling in shows folders/items merged across every kind.
-  - **Kind** (`{ kind }`): top-level scopes for that kind (e.g. `f/demo`, `u/alice`).
-    `kind: 'all'` is the cross-kind variant — folders contain items from
-    every kind, leaves still belong to a real kind.
-  - **Dir** (`{ kind, dir }`): immediate children of `dir` — subdirs + leaves.
-
-Clicking a row drills *down*; the chevron-left in the header walks one level
-*up*. Search is global across all kinds and ignores the current scope.
+Surfaces AI-created localStorage drafts (via `listGlobalDrafts`) as extra
+items alongside the backend-loaded list, so chat-scaffolded scripts/flows/
+apps that haven't been deployed yet are still navigable. Gated on
+`isGlobalAiEnabled()` — without sessions, the only UserDrafts present are
+standalone editor autosaves and surfacing those in the breadcrumb picker
+would be surprising.
 -->
 <script lang="ts">
 	import { workspaceStore } from '$lib/stores'
-	import { ChevronLeft, ChevronRight, Folder, Layers, Loader2, User } from 'lucide-svelte'
-	import TextInput from '$lib/components/text_input/TextInput.svelte'
 	import RowIcon from '$lib/components/common/table/RowIcon.svelte'
-	import WorkspaceItemRow from '$lib/components/WorkspaceItemRow.svelte'
-	import SearchItems from '$lib/components/SearchItems.svelte'
-	import { onMount, untrack } from 'svelte'
-	import { generateRandomString } from '$lib/utils'
-	import {
-		dirKey,
-		getCachedItems,
-		KIND_LABEL,
-		KIND_LABEL_LOWER,
-		kindKey,
-		leafKeyFor,
-		loadKind,
-		type WorkspaceItem,
-		type WorkspaceItemKind
-	} from './workspacePicker'
+	import { untrack } from 'svelte'
+	import { type WorkspaceItem, type WorkspaceItemKind } from './workspacePicker'
+	import { useWorkspaceItemsLoader } from './workspaceItemsLoader.svelte'
+	import DrillPicker from './DrillPicker.svelte'
+	import type { DrillBranch, DrillLeaf } from './drillPicker'
+	import { buildWorkspaceTree, legacyScopeToPath, relativizeWorkspacePath } from './workspaceTree'
 	import { listGlobalDrafts } from '$lib/components/copilot/chat/global/userDraftAdapter'
 	import { isGlobalAiEnabled } from '$lib/components/copilot/chat/global/gate'

 	type Kind = WorkspaceItemKind
-	type Item = WorkspaceItem
-	/** `'all'` is a virtual cross-kind scope — items still belong to a real
-	 * kind, but folders and the root row group items from every kind. */
 	type ScopeKind = Kind | 'all'

 	export type Scope = { kind: ScopeKind; dir?: string } | undefined
@@ -46,14 +32,12 @@ Clicking a row drills *down*; the chevron-left in the header walks one level
 	interface Props {
 		onPick: (item: WorkspaceItem) => void
 		kinds?: Kind[]
-		/** Where the picker lands when first opened. `undefined` = root (kinds list). */
 		initialScope?: Scope
-		/** Composite key of the row to highlight (e.g. `dir:flow:f/demo`). */
 		initialHighlight?: string
-		/** Currently-edited item — gets `aria-current` and a no-op click. If
-		 * `savedPath` differs from `path` (draft rename), the saved entry is
-		 * suppressed so only the live one shows. */
 		currentItem?: WorkspaceItem & { savedPath?: string }
+		externalFilter?: string
+		autoFocus?: boolean
+		flush?: boolean
 	}

 	let {
@@ -61,112 +45,39 @@ Clicking a row drills *down*; the chevron-left in the header walks one level
 		kinds = ['flow', 'script', 'app'],
 		initialScope,
 		initialHighlight,
-		currentItem
+		currentItem,
+		externalFilter,
+		autoFocus = true,
+		flush = false
 	}: Props = $props()

-	let searchInput: TextInput | undefined = $state()
-	let pickerRoot: HTMLElement | undefined = $state()
-	const instanceId = generateRandomString(8)
-	const listboxId = `pkr-list-${instanceId}`
-	const idFor = (key: string) => `pkr-${instanceId}-${key.replace(/[^a-zA-Z0-9-]/g, '_')}`
+	let inner = $state<DrillPicker<WorkspaceItem> | undefined>(undefined)

 	export function focus() {
-		searchInput?.focus()
+		inner?.focus()
+	}
+	export function handleKeydown(e: KeyboardEvent) {
+		inner?.handleKeydown(e)
+	}
+	export function pickHighlighted() {
+		inner?.pickHighlighted()
 	}

-	// Sibling-popover open: melt-ui's `openFocus` runs once during the close→open
-	// transition; the picker may not be mounted yet. Retry after settle.
-	// Also kicks off the initial scope's fetch — drill/goUp do the same from
-	// their respective branches, so `ensureLoaded` is always a callback
-	// reaction to user navigation, never a reactive consequence.
-	onMount(() => {
-		const t = setTimeout(focus, 50)
-		const initial = untrack(() => scope)
-		if (initial) {
-			if (initial.kind === 'all') for (const k of kinds) ensureLoaded(k)
-			else ensureLoaded(initial.kind)
-		}
-		return () => clearTimeout(t)
-	})
-
-	const leafKey = (it: Item) => leafKeyFor(it.kind, it.path)
-
-	let scope = $state<Scope>(untrack(() => initialScope))
-	let filter = $state('')
-
-	/**
-	 * Canonical entry point for changing the picker's scope. Triggers the
-	 * fetch for the kind(s) the new scope needs at the same point in time.
-	 * Replaces the older "react to `scope` change via `$effect`" wiring,
-	 * which had a subtle bug: `ensureLoaded` reads `loaded[kind]`, so the
-	 * effect ended up subscribed to the signal it fills — every fetch
-	 * result re-fired it. With explicit callbacks the fetch is tied to
-	 * the user's action, never to a reactive consequence of that action.
-	 */
-	function setScope(next: Scope) {
-		scope = next
-		if (!next) return
-		if (next.kind === 'all') for (const k of kinds) ensureLoaded(k)
-		else ensureLoaded(next.kind)
-	}
-
-	/** Tracks whether the last user action was mouse movement (true) or
-	 * keyboard nav (false). When false, row `mouseenter` events are ignored
-	 * — prevents the cursor from stealing the keyboard-driven highlight as
-	 * rows shift under it during scope changes. Re-enabled on `mousemove`.
-	 * Starts `false` so the synthetic `mouseenter` fired when the popover
-	 * mounts under a stationary cursor doesn't clobber `initialHighlight`. */
-	let mouseActive = $state(false)
-
-	// Seed from the last fetched snapshot so kinds already fetched in this
-	// session render on the first frame. Each entry is replaced once
-	// `loadKind` returns fresh data — stale-while-revalidate, so deploys and
-	// AI-created drafts surface on the next open without explicit cache
-	// busting.
-	let loaded = $state<Partial<Record<Kind, Item[]>>>(
-		(() => {
-			if (!$workspaceStore) return {}
-			const out: Partial<Record<Kind, Item[]>> = {}
-			for (const k of kinds) {
-				const cached = getCachedItems($workspaceStore, k)
-				if (cached) out[k] = cached
-			}
-			return out
-		})()
+	const loader = useWorkspaceItemsLoader(
+		() => $workspaceStore,
+		() => kinds
 	)
-	let loadingKind = $state<Partial<Record<Kind, boolean>>>({})

-	async function ensureLoaded(kind: Kind) {
-		if (!$workspaceStore) return
-		// Always re-fetch. If we have nothing cached, show a spinner; if we do,
-		// keep displaying it and quietly swap to fresh data when it lands.
-		// `loaded[kind]` is read inside `untrack(...)` because this function is
-		// reachable from the search `$effect` below — without the untrack,
-		// that effect would subscribe to the signal `ensureLoaded` fills, and
-		// each `loaded[kind] = items` (proxy `set` notifies even when the ref
-		// is unchanged from cache) would refire it → runaway loop. Drill
-		// navigation goes through `setScope` directly so it isn't affected.
-		if (!untrack(() => loaded[kind])) loadingKind[kind] = true
-		try {
-			const items = await loadKind($workspaceStore, kind)
-			loaded[kind] = items
-		} finally {
-			loadingKind[kind] = false
-		}
-	}
-
-	// Chat tools and session editor previews write drafts through
-	// `UserDraft` (workspace-scoped, localStorage-backed). Merge those into
-	// the picker so users can navigate to in-flight items that haven't been
-	// deployed yet. Filter to kinds the picker actually displays.
-	//
-	// Gated on the same dev flag as the rest of the sessions feature: without
-	// it there are no sessions, so the only UserDrafts present are the
-	// standalone editors' autosaves — surfacing those in the breadcrumb picker
-	// would be surprising (they'd appear as navigable items that 404 on the
-	// backend draft fetch). When the flag is off this is a no-op.
+	// Chat tools and session editor previews write drafts through `UserDraft`
+	// (workspace-scoped, localStorage-backed). Merge those into the picker so
+	// users can navigate to in-flight items that haven't been deployed yet.
+	// Filter to kinds the picker actually displays. Gated on the global-AI
+	// flag — without sessions, the only UserDrafts present are the standalone
+	// editors' autosaves and surfacing those in the breadcrumb picker would
+	// be surprising (they'd appear as navigable items that 404 on the backend
+	// draft fetch).
 	const KIND_TO_DRAFT_TYPE = { flow: 'flow', script: 'script', app: 'app' } as const
-	function aiDraftsForKind(k: Kind): Item[] {
+	function aiDraftsForKind(k: Kind): WorkspaceItem[] {
 		if (!isGlobalAiEnabled()) return []
 		if (!$workspaceStore) return []
 		const targetType = KIND_TO_DRAFT_TYPE[k]
@@ -181,573 +92,58 @@ Clicking a row drills *down*; the chevron-left in the header walks one level
 			}))
 	}

-	// Searching is global → load every kind.
-	$effect(() => {
-		if (filter.trim() !== '') for (const k of kinds) ensureLoaded(k)
-	})
-
-	type DirNode = {
-		fullPath: string
-		name: string
-		isScope: boolean
-		children: DirNode[]
-		leaves: Item[]
-	}
-
-	/** Merge AI-created in-memory drafts into a kind's list. The AI may have
-	 * scaffolded a script/flow/app via chat tools without the user saving
-	 * yet — those drafts should be navigable from the picker. Existing items
-	 * (same path) win to keep the backend's metadata (summary etc.). */
-	function withAiDrafts(items: Item[], k: Kind): Item[] {
-		const ai = aiDraftsForKind(k)
-		if (ai.length === 0) return items
-		const known = new Set(items.map((it) => it.path))
-		return items.concat(ai.filter((d) => !known.has(d.path)))
-	}
-
-	/** Inject the currently-edited item into a kind's list at its live path,
-	 * dropping the saved entry when a draft rename is in progress. Other kinds
-	 * pass through untouched. */
-	function withCurrent(items: Item[], k: Kind): Item[] {
-		if (!currentItem || currentItem.kind !== k) return items
-		const drafted =
-			currentItem.savedPath && currentItem.savedPath !== currentItem.path
-				? items.filter((it) => it.path !== currentItem.savedPath)
-				: items
-		if (drafted.some((it) => it.path === currentItem.path)) return drafted
-		return [
-			...drafted,
-			{
-				path: currentItem.path,
-				summary: currentItem.summary,
-				kind: k,
-				raw_app: currentItem.raw_app
-			}
-		]
-	}
-
-	function buildTreeFromItems(items: Item[]): DirNode[] {
-		const scopeRoots = new Map<string, DirNode>()
-		for (const it of items) {
-			const parts = it.path.split('/')
-			if (parts.length < 3) continue
-			const scopeFp = parts.slice(0, 2).join('/')
-			let node = scopeRoots.get(scopeFp)
-			if (!node) {
-				node = { fullPath: scopeFp, name: scopeFp, isScope: true, children: [], leaves: [] }
-				scopeRoots.set(scopeFp, node)
-			}
-			const slug = parts.slice(2)
-			let cur = node
-			for (let i = 0; i < slug.length - 1; i++) {
-				const seg = slug[i]
-				const fullPath = cur.fullPath + '/' + seg
-				let next = cur.children.find((c) => c.name === seg)
-				if (!next) {
-					next = { fullPath, name: seg, isScope: false, children: [], leaves: [] }
-					cur.children.push(next)
-				}
-				cur = next
-			}
-			cur.leaves.push(it)
-		}
-		const scopes = Array.from(scopeRoots.values()).sort((a, b) => {
-			const af = a.fullPath.startsWith('f/') ? 0 : 1
-			const bf = b.fullPath.startsWith('f/') ? 0 : 1
-			if (af !== bf) return af - bf
-			return a.fullPath.localeCompare(b.fullPath)
-		})
-		const sortNode = (n: DirNode) => {
-			n.children.sort((a, b) => a.name.localeCompare(b.name))
-			n.leaves.sort((a, b) => a.path.localeCompare(b.path))
-			n.children.forEach(sortNode)
-		}
-		scopes.forEach(sortNode)
-		return scopes
-	}
-
-	/** Per-kind tree deriveds. Each only re-evaluates `buildTreeFromItems`
-	 * when its own `loaded[k]` changes or when the user is mid-rename on
-	 * that kind — typing in a flow's path edit leaves script/app trees
-	 * cached. */
-	function buildIfActive(k: Kind, list: Item[] | undefined): DirNode[] {
-		if (!kinds.includes(k)) return []
-		const items = withAiDrafts(withCurrent(list ?? [], k), k)
-		if (items.length === 0) return []
-		return buildTreeFromItems(items)
-	}
-
-	const flowTree = $derived(buildIfActive('flow', loaded.flow))
-	const scriptTree = $derived(buildIfActive('script', loaded.script))
-	const appTree = $derived(buildIfActive('app', loaded.app))
-	/** Cross-kind tree: every loaded item from every active kind, merged into
-	 * one folder hierarchy. Each leaf still carries its real kind, so the row
-	 * icon and `editPathFor` routing still work; folders contain a mix. */
-	const allTree = $derived.by(() => {
-		const merged = kinds.flatMap((k) => withAiDrafts(withCurrent(loaded[k] ?? [], k), k))
-		return merged.length === 0 ? [] : buildTreeFromItems(merged)
-	})
-
-	function treeFor(k: ScopeKind): DirNode[] {
-		if (k === 'all') return allTree
-		if (k === 'flow') return flowTree
-		if (k === 'script') return scriptTree
-		return appTree
-	}
-
-	function findDirInList(list: DirNode[], fullPath: string): DirNode | undefined {
-		for (const n of list) {
-			if (n.fullPath === fullPath) return n
-			const sub = findDirInList(n.children, fullPath)
-			if (sub) return sub
-		}
-		return undefined
-	}
-
-	function parentDirPath(p: string): string | undefined {
-		const parts = p.split('/')
-		if (parts.length <= 2) return undefined
-		return parts.slice(0, -1).join('/')
-	}
-
-	type Entry =
-		| { type: 'kind'; key: string; kind: ScopeKind }
-		| { type: 'dir'; key: string; kind: ScopeKind; node: DirNode }
-		| { type: 'leaf'; key: string; item: Item }
-
-	type DisplayItem = Item & { marked?: string }
-	type SearchInput = Item & { _key: string }
-
-	let allItems = $derived<SearchInput[]>(
-		kinds.flatMap((k) =>
-			withAiDrafts(withCurrent(loaded[k] ?? [], k), k).map((it) => ({
-				...it,
-				_key: `${k}:${it.path}`
-			}))
-		)
+	const extraItemsByKind = $derived<Partial<Record<Kind, WorkspaceItem[]>>>(
+		Object.fromEntries(kinds.map((k) => [k, aiDraftsForKind(k)]))
 	)

-	let searchedItems: DisplayItem[] | undefined = $state(undefined)
-
-	let isSearching = $derived(filter.trim() !== '')
-
-	let searchResultsByKind = $derived.by(() => {
-		const out: Record<Kind, DisplayItem[]> = { flow: [], script: [], app: [] }
-		if (!searchedItems) return out
-		for (const it of searchedItems) out[it.kind].push(it)
-		return out
-	})
-
-	/** Rows currently shown — drives both rendering and keyboard nav. Not used
-	 * while `isSearching` (search renders its own grouped layout). */
-	let entries = $derived.by<Entry[]>(() => {
-		const s = scope
-		if (!s) {
-			const kindEntries = kinds.map((k) => ({
-				type: 'kind' as const,
-				key: kindKey(k),
-				kind: k
-			}))
-			// "All" only makes sense across multiple kinds — with a single kind
-			// it would duplicate that kind's own root row.
-			if (kinds.length <= 1) return kindEntries
-			return [
-				{ type: 'kind' as const, key: kindKey('all'), kind: 'all' as ScopeKind },
-				...kindEntries
-			]
-		}
-		const tree = treeFor(s.kind)
-		if (!s.dir) {
-			return tree.map((node) => ({
-				type: 'dir',
-				key: dirKey(s.kind, node.fullPath),
-				kind: s.kind,
-				node
-			}))
-		}
-		const node = findDirInList(tree, s.dir)
-		if (!node) return []
-		return [
-			...node.children.map(
-				(c): Entry => ({
-					type: 'dir',
-					key: dirKey(s.kind, c.fullPath),
-					kind: s.kind,
-					node: c
-				})
-			),
-			...node.leaves.map((l): Entry => ({ type: 'leaf', key: leafKey(l), item: l }))
-		]
-	})
-
-	let navKeys = $derived.by(() => {
-		if (isSearching) {
-			return kinds.flatMap((k) => searchResultsByKind[k].map((it) => leafKey(it)))
-		}
-		return entries.map((e) => e.key)
-	})
-
-	let highlightedKey = $state<string | undefined>(untrack(() => initialHighlight))
-	let highlightedId = $derived(highlightedKey ? idFor(highlightedKey) : undefined)
-
-	$effect(() => {
-		if (navKeys.length === 0) return
-		if (!highlightedKey || !navKeys.includes(highlightedKey)) {
-			highlightedKey = navKeys[0]
-		}
-	})
-
-	$effect(() => {
-		if (highlightedKey && navKeys.includes(highlightedKey)) {
-			requestAnimationFrame(scrollHighlightIntoView)
-		}
-	})
-
-	function scrollHighlightIntoView() {
-		if (!pickerRoot || !highlightedKey) return
-		const el = pickerRoot.querySelector<HTMLElement>(
-			`[data-nav-key="${CSS.escape(highlightedKey)}"]`
-		)
-		el?.scrollIntoView({ block: 'nearest', behavior: 'smooth' })
-	}
-
-	function moveHighlight(delta: 1 | -1) {
-		if (navKeys.length === 0) return
-		const cur = navKeys.indexOf(highlightedKey ?? '')
-		const next = cur < 0 ? 0 : (cur + delta + navKeys.length) % navKeys.length
-		highlightedKey = navKeys[next]
-		mouseActive = false
-		requestAnimationFrame(scrollHighlightIntoView)
-	}
-
-	function setHoverHighlight(key: string) {
-		// Ignored until the user actually moves the mouse. Prevents the cursor
-		// (parked over a row) from clobbering keyboard-driven selection when
-		// the layout shifts beneath it.
-		if (mouseActive) highlightedKey = key
-	}
-
-	function isCurrent(it: Item): boolean {
-		return !!currentItem && currentItem.kind === it.kind && currentItem.path === it.path
-	}
-
-	function pick(it: Item) {
-		if (isCurrent(it)) return
-		onPick({ path: it.path, summary: it.summary, kind: it.kind, raw_app: it.raw_app })
-	}
-
-	function activate(key: string | undefined) {
-		if (!key) return
-		if (isSearching) {
-			const flat = kinds.flatMap((k) => searchResultsByKind[k])
-			const it = flat.find((x) => leafKey(x) === key)
-			if (it) pick(it)
-			return
-		}
-		const entry = entries.find((e) => e.key === key)
-		if (!entry) return
-		drill(entry)
-	}
-
-	function drill(entry: Entry) {
-		if (entry.type === 'kind') {
-			setScope({ kind: entry.kind })
-		} else if (entry.type === 'dir') {
-			setScope({ kind: entry.kind, dir: entry.node.fullPath })
-		} else {
-			pick(entry.item)
-		}
-	}
-
-	function goUp() {
-		if (!scope) return
-		// Highlight the row in the parent view that represents the scope we
-		// just left, so the user sees where they came from.
-		if (!scope.dir) {
-			const leaving = kindKey(scope.kind)
-			setScope(undefined)
-			highlightedKey = leaving
-			return
-		}
-		const leaving = dirKey(scope.kind, scope.dir)
-		const parent = parentDirPath(scope.dir)
-		setScope(parent ? { kind: scope.kind, dir: parent } : { kind: scope.kind })
-		highlightedKey = leaving
-	}
-
-	function handleSearchKeydown(e: KeyboardEvent) {
-		if (e.key === 'ArrowDown') {
-			e.preventDefault()
-			e.stopPropagation()
-			moveHighlight(1)
-		} else if (e.key === 'ArrowUp') {
-			e.preventDefault()
-			e.stopPropagation()
-			moveHighlight(-1)
-		} else if (e.key === 'Enter') {
-			e.preventDefault()
-			e.stopPropagation()
-			mouseActive = false
-			activate(highlightedKey)
-		} else if ((e.key === 'ArrowLeft' || e.key === 'Backspace') && filter === '' && scope) {
-			// Walk up the tree. Only when search is empty — otherwise these
-			// keys would hijack cursor movement / character deletion.
-			e.preventDefault()
-			e.stopPropagation()
-			mouseActive = false
-			goUp()
-		} else if (e.key === 'ArrowRight' && filter === '' && !isSearching) {
-			// Drill into the highlighted folder/kind. Leaves are reserved for
-			// Enter (more deliberate, since picking navigates away).
-			const entry = entries.find((en) => en.key === highlightedKey)
-			if (entry && entry.type !== 'leaf') {
-				e.preventDefault()
-				e.stopPropagation()
-				mouseActive = false
-				drill(entry)
-			}
-		}
-	}
-
-	/** Breadcrumb segments for the header — kind name first, then the scope
-	 * (`f/<folder>` or `u/<user>`) as one chunk, then any nested subdirs. */
-	let headerSegments = $derived.by<string[]>(() => {
-		if (!scope) return []
-		const out = [scope.kind === 'all' ? 'all' : KIND_LABEL_LOWER[scope.kind]]
-		if (scope.dir) {
-			const parts = scope.dir.split('/')
-			out.push(parts.slice(0, 2).join('/'))
-			for (let i = 2; i < parts.length; i++) out.push(parts[i])
-		}
-		return out
-	})
-
-	/** Full breadcrumb (used for the hover tooltip). */
-	let headerLabel = $derived(headerSegments.join(' › '))
-
-	/** Number of intermediate segments to hide behind a `…`. The collapsed
-	 * window is segments[2 .. 2 + hiddenCount); we always keep the first
-	 * two (kind, top-level scope) and the deepest segment. Bumped up by an
-	 * effect that measures actual overflow — see below. */
-	let hiddenCount = $state(0)
-
-	/** Breadcrumb shown to the user. Hides intermediate segments first, then
-	 * relies on `truncate-start` for any remaining overflow on the deepest
-	 * segment. Hover reveals the full path via `title`. */
-	let headerLabelDisplay = $derived.by(() => {
-		if (headerSegments.length <= 3 || hiddenCount === 0) return headerLabel
-		return [
-			headerSegments[0],
-			headerSegments[1],
-			'…',
-			...headerSegments.slice(2 + hiddenCount)
-		].join(' › ')
-	})
-
-	let breadcrumbSpan: HTMLElement | undefined = $state()
-	let lastSegmentsKey = ''
-
-	/** Measurement loop: each pass reads `scrollWidth > clientWidth` on the
-	 * truncated span; if overflowing and there's still an intermediate segment
-	 * to drop, increment `hiddenCount`. Mutating `hiddenCount` re-renders and
-	 * re-fires this effect, so the loop self-terminates either when the text
-	 * fits or when only [kind, scope, …, leaf] remain (`truncate-start` then
-	 * polishes any final overflow). When the breadcrumb itself changes (new
-	 * scope), reset to 0 first so a shorter path can re-expand. */
-	$effect(() => {
-		const key = headerSegments.join('|')
-		const segmentsChanged = key !== lastSegmentsKey
-		if (segmentsChanged) {
-			lastSegmentsKey = key
-			if (hiddenCount !== 0) {
-				hiddenCount = 0
-				return
-			}
-		}
-		// Track hiddenCount so each collapse step re-measures.
-		void hiddenCount
-		if (!breadcrumbSpan) return
-		const maxHide = Math.max(0, headerSegments.length - 3)
-		if (hiddenCount >= maxHide) return
-		queueMicrotask(() => {
-			if (!breadcrumbSpan) return
-			if (breadcrumbSpan.scrollWidth > breadcrumbSpan.clientWidth + 1) {
-				hiddenCount = hiddenCount + 1
-			}
+	const tree = $derived(
+		buildWorkspaceTree({
+			loaded: loader.loaded,
+			kinds,
+			currentItem,
+			loadingKind: loader.loadingKind,
+			extraItemsByKind
 		})
-	})
+	)

-	let scopeLoading = $derived.by(() => {
-		if (!scope) return false
-		if (scope.kind === 'all') {
-			return kinds.some((k) => !loaded[k] && !!loadingKind[k])
-		}
-		return !loaded[scope.kind] && !!loadingKind[scope.kind]
-	})
+	// Mount-time only: callers (BreadcrumbSegment, EditorHeader) snapshot the
+	// scope when the popover opens, so re-evaluating on prop changes would
+	// fight the user's drilling.
+	const computedInitialScope = untrack(() => legacyScopeToPath(initialScope, kinds))
 </script>

-<SearchItems
-	{filter}
-	items={isSearching ? allItems : []}
-	bind:filteredItems={searchedItems}
-	f={(x: SearchInput) => (x.summary ? `${x.summary} (${x.path})` : x.path)}
-	opts={{}}
-/>
-
-{#snippet leafRow(it: Item, secondary: string, baseClass: string)}
-	{@const key = leafKey(it)}
-	<WorkspaceItemRow
-		kind={it.kind}
-		summary={it.summary}
-		{secondary}
-		highlighted={key === highlightedKey}
-		current={isCurrent(it)}
-		id={idFor(key)}
-		navKey={key}
-		{baseClass}
-		onclick={() => pick(it)}
-		onmouseenter={() => setHoverHighlight(key)}
-	/>
+{#snippet leafIcon(leaf: DrillLeaf<WorkspaceItem>)}
+	<RowIcon kind={leaf.data.kind} size={12} />
 {/snippet}

-<!-- svelte-ignore a11y_no_static_element_interactions -->
-<div
-	bind:this={pickerRoot}
-	class="flex flex-col w-[420px] max-h-[60vh]"
-	onkeydown={handleSearchKeydown}
-	onmousemove={() => (mouseActive = true)}
->
-	<div class="px-3 py-2 border-b border-gray-200 dark:border-gray-700">
-		<TextInput
-			bind:this={searchInput}
-			bind:value={filter}
-			size="sm"
-			inputProps={{
-				placeholder: 'Search by name or summary...',
-				'data-workspace-picker-search': '',
-				role: 'combobox',
-				'aria-controls': listboxId,
-				'aria-expanded': 'true',
-				'aria-autocomplete': 'list',
-				'aria-activedescendant': highlightedId
-			}}
-		/>
-	</div>
-
-	{#if scope}
-		{@const s = scope}
-		<button
-			type="button"
-			class="flex items-center gap-1.5 w-full text-left px-3 py-1 text-xs font-medium font-mono text-secondary bg-surface-secondary/20 hover:bg-surface-hover transition-colors"
-			onmousedown={(e) => e.preventDefault()}
-			onclick={goUp}
-			title={headerLabel}
-		>
-			<ChevronLeft size={12} class="shrink-0 text-secondary" />
-			{#if s.kind === 'all'}
-				<Layers size={12} class="shrink-0 text-tertiary" />
-			{:else}
-				<RowIcon kind={s.kind} size={12} />
-			{/if}
-			<span bind:this={breadcrumbSpan} class="flex-1 min-w-0 truncate truncate-start"
-				>{headerLabelDisplay}</span
-			>
-		</button>
+{#snippet branchIcon(branch: DrillBranch<WorkspaceItem>)}
+	{#if branch.key === 'kind:flow' || branch.key === 'kind:script' || branch.key === 'kind:app'}
+		{@const k = branch.key.slice(5) as Kind}
+		<RowIcon kind={k} size={12} />
+	{:else if branch.icon}
+		{@const Icon = branch.icon}
+		<Icon size={12} class="shrink-0 text-tertiary" />
 	{/if}
+{/snippet}

-	<div class="flex-1 overflow-y-auto" role="listbox" id={listboxId}>
-		{#if isSearching}
-			{@const total = (searchedItems ?? []).length}
-			{@const anyKindLoading = kinds.some((k) => loadingKind[k])}
-			{#if !searchedItems || anyKindLoading}
-				<!-- "Searching…" while any active kind is still loading, otherwise
-				     `SearchItems` would briefly write `filteredItems=[]` from the
-				     partial set and flash "No matches" before results trickle in. -->
-				<div role="status" class="px-3 py-2 text-xs text-tertiary flex items-center gap-2">
-					<Loader2 size={14} class="animate-spin" /> Searching…
-				</div>
-			{:else if total === 0}
-				<div role="status" class="px-3 py-2 text-xs text-tertiary">No matches</div>
-			{:else}
-				{#each kinds as k (k)}
-					{@const results = searchResultsByKind[k]}
-					{#if results.length > 0}
-						<div class="px-3 pt-3 pb-1 text-2xs uppercase tracking-wide text-tertiary font-medium">
-							{KIND_LABEL[k]}
-						</div>
-						<ul class="pb-1">
-							{#each results as it (leafKey(it))}
-								<li>{@render leafRow(it, it.path, 'py-1.5')}</li>
-							{/each}
-						</ul>
-					{/if}
-				{/each}
-			{/if}
-		{:else if scopeLoading && entries.length === 0}
-			<div role="status" class="px-3 py-2 text-xs text-tertiary flex items-center gap-2">
-				<Loader2 size={14} class="animate-spin" /> Loading…
-			</div>
-		{:else if entries.length === 0}
-			<div role="status" class="px-3 py-2 text-xs text-tertiary">Empty</div>
-		{:else}
-			<div class="flex flex-col py-1">
-				{#each entries as entry (entry.key)}
-					{@const isHl = entry.key === highlightedKey}
-					{#if entry.type === 'leaf'}
-						{@render leafRow(
-							entry.item,
-							scope?.dir ? entry.item.path.slice(scope.dir.length + 1) : entry.item.path,
-							'py-1.5'
-						)}
-					{:else}
-						<button
-							type="button"
-							id={idFor(entry.key)}
-							role="option"
-							aria-selected={isHl}
-							data-nav-key={entry.key}
-							class="flex items-center gap-1.5 w-full text-left px-3 py-1.5 text-xs font-medium font-mono text-emphasis transition-colors {isHl
-								? 'bg-surface-hover'
-								: ''}"
-							onmousedown={(e) => e.preventDefault()}
-							onclick={() => drill(entry)}
-							onmouseenter={() => setHoverHighlight(entry.key)}
-						>
-							{#if entry.type === 'kind'}
-								{#if entry.kind === 'all'}
-									<Layers size={12} class="shrink-0 text-tertiary" />
-									<span class="flex-1">All</span>
-								{:else}
-									<RowIcon kind={entry.kind} size={12} />
-									<span class="flex-1">{KIND_LABEL[entry.kind]}</span>
-								{/if}
-							{:else if entry.node.isScope && entry.node.fullPath.startsWith('u/')}
-								<User size={12} class="shrink-0 text-tertiary" />
-								<span class="flex-1 truncate">{entry.node.name}</span>
-							{:else}
-								<Folder size={12} class="shrink-0 text-tertiary" />
-								<span class="flex-1 truncate">{entry.node.name}</span>
-							{/if}
-							{#if entry.type === 'kind' && entry.kind !== 'all' && loadingKind[entry.kind]}
-								<Loader2 size={12} class="animate-spin text-tertiary" />
-							{/if}
-							<ChevronRight size={10} class="shrink-0 text-secondary" />
-						</button>
-					{/if}
-				{/each}
-			</div>
-		{/if}
-	</div>
-</div>
-
-<style>
-	/* Path truncates from the start (left ellipsis) so the deepest (rightmost)
-	 * folder stays visible. `unicode-bidi: plaintext` keeps each path segment
-	 * laid out per its own direction — defends against any future RTL char
-	 * appearing in a workspace path. */
-	.truncate-start {
-		direction: rtl;
-		text-align: left;
-		unicode-bidi: plaintext;
-	}
-</style>
+<DrillPicker
+	bind:this={inner}
+	{tree}
+	onPick={(leaf) => onPick(leaf.data)}
+	initialScope={computedInitialScope}
+	{initialHighlight}
+	{externalFilter}
+	{autoFocus}
+	{flush}
+	{leafIcon}
+	{branchIcon}
+	leafSecondary={(leaf, scope) => relativizeWorkspacePath(leaf.data.path, scope)}
+	onScopeChange={(scope) => {
+		if (scope.length > 0) loader.ensureForScopeSegment(scope[0])
+		// Single-kind layout has no kind branch at root — `buildWorkspaceTree`
+		// collapses to the kind's children. The picker mounts with scope=[],
+		// so without this fallback nothing fires until the user searches.
+		else if (kinds.length === 1) loader.ensureLoaded(kinds[0])
+	}}
+	onFilterChange={loader.onFilterChange}
+/>
--- a/Show More
+++ b/Show More