Addresses local-review findings before PR:
- P1: record_and_check_join_slot was a non-atomic check-then-act on a
pooled connection; concurrent completion of a subscriber's last two
partition-bearing inputs on different workers could double-dispatch.
Now one transaction guarded by a tx-scoped advisory lock keyed on
(workspace, subscriber, partition) so the gate fires exactly once.
- P2: the preprocessed-args overwrite in result_processor replaced args
wholesale, dropping a partition resolved by resolve_partition_for_job;
the UPDATE now preserves an existing persisted partition key.
- P2: gate resolve_partition_for_job on a cheap code.contains check so
non-pipeline script jobs skip the annotation scan on the hot path.
- P2: remove 40 scratch screenshot PNGs, a flicker-debug script and a
local scheduler lock accidentally committed; gitignore the lock.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage E3. fetch_subscribers now also returns debounce_s; push_subscriber
builds real DebouncingSettings (delay + a (subscriber, partition) key,
so distinct partitions never collapse and latest-in-window falls out)
instead of ::default() when the edge opted in. Default stays no-debounce
(fan-out — the prior deliberate behaviour, now overridable rather than
reversed). Wiring test asserts the dispatched job carries the configured
window/key and an undebounced edge carries none.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage E1+E2. Parser: script-level // debounce <dur> + per-// on
debounce=<dur> override (edge wins, else script default, else none =
fan-out, unchanged); TriggerSpec::Asset carries the per-edge override;
split_trailing_kv_opts separates the ref from trailing key=val opts.
Schema/deploy: reversible migration adds script_trigger.debounce_s;
parse_duration_secs (bare int or <n>s|m|h|d, fail-safe on garbage)
resolves the effective per-edge window at deploy and writes it per row.
No reader yet (dispatch wiring is E3) so runtime is unchanged. New unit
tests for the parser directive and duration parsing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage D: a // trigger all subscriber no longer fires on any input. New
join_pending_inputs slot table keyed (workspace, subscriber, partition);
fetch_subscribers now returns join_all and the dispatch loop records each
partition-bearing input arrival, pushing the subscriber once only when
every partition-bearing input it declares is present for that partition.
Per-partition slots, cleared on fire (re-accumulate, no double-fire),
skew-immune (unlike debounce). Case-3 guard: an unpartitioned producer or
a reference (non-{partition}) input never fires a partitioned join.
Integration test covers wait/fire/isolation/no-double-fire.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage C: in handle_code_execution_job, once the script content is loaded,
parse the // partitioned annotation (free here) and resolve the concrete
partition once — schedule fire-time (scheduled_for anchor, not wall-clock)
for time kinds, triggering payload for dynamic. The value is injected
into the in-memory args the body sees (via a shadowed job clone) and
persisted back to v2_job.args so dispatch_asset_triggers propagates the
same value down the cascade. Already-set (explicit/backfill/cascade)
partitions are never re-resolved (run identity immutable); unresolvable
partitioned runs fail with a clear error. Integration test exercises the
full worker loop + cascade propagation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stage A: JoinMode{Any(default),All} + `// trigger any|all` directive in
parse_pipeline_annotations; TriggerSpec::is_partition_bearing() (path
contains {partition}); join_mode threaded through all 4 asset-parser
crates (ts/py/sql/yaml). Stage B: reversible migration adds
script_trigger.join_all; insert_script_trigger writes it; deploy path
sets it from the parsed annotation. No reader yet (AND-join dispatch is
the next stage) so runtime behaviour is unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AssetGraphDetailsPane keeps the compact (hideArgs) preview but, via a new
previewPanel.argsAboveLogs flag, renders a compact SchemaForm between the
floating Test button and the logs/result panel when the script declares
inputs (e.g. a partitioned script needing a `partition` arg). The preview
pane also grows ~18pts so the args form doesn't shrink logs/result.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
windmill-common/partition: pure resolver — time kinds (tz/format/start
anchor) + dynamic $.a.b JSONPath; 9 unit tests. asset_dispatch:
read the producer's resolved partition and thread it into every
cascaded subscriber's args + trigger.partition, so a chain resolves
once at the top. No migration (cascade needs no spec lookup). Stage
1+3 of pipeline partition runtime; run-start resolution is Stage 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If the poll never sampled a launched run's in-flight window, the
catch-up pulse re-flashed its edges one tick after it correctly
stopped (the page already animated it zero-latency via activeRunnable).
arm(launchedId) records the launched runnable id; catch-up skips it.
Cascade hops (other ids) still pulse. launchedIds cleared on stop.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The catch-up pulse re-added a completed job to the active set if its
start was within the (lagging) lookback window — even one we'd already
animated the whole time it ran — keeping its edges lit ~a poll
interval past completion (~5s after a 3.5s test). Track job ids seen
in-flight and skip the pulse for them; it still fires for hops whose
whole lifetime fell between two polls. Bound the set in lockstep with
eventsById; cleared on dispose.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
prune_runtime_assets deleted by (workspace_id, path, kind) tuple, so
trimming surplus usage_kind='job' rows for an s3 path also wiped the
static usage_kind='script'/'flow' producer rows for the same path —
silently breaking the asset-trigger cascade (fetch_producer_writes
found no writes; downstream never dispatched; required band-aid
re-syncs). Delete the surplus job rows by id instead; the inner query
is already scoped to usage_kind='job'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Controls -> top-right horizontal, no lock toggle; MiniMap !mb-10 so
it clears the activity bar; hide the per-script Save button when the
script is already at its latest save point (drafts still show Create).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the ~230-line graphWithDraft precedence/merge (base < session-
inferred < draft-seeded < open-script-live, +read/write/annotation
overlays, +dedup) out of the 1648-line route into a pure, testable
resolveGraph() module; the route's graphWithDraft is now a thin
$derived. Behaviour extracted verbatim. 10 unit tests cover the
precedence matrix. Phase 1 of the state/render split.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sugiyama-waypoint routing looked worse than the original; revert
AssetGraphEdge/assetGraphLayout to the pre-routing bezier logic (same
as the flow editor's BaseEdge) and drop the now-unused route plumbing
from the canvas. Add a small 'Parsing assets…' hint shown while the
load-time prefetch sweep is still inferring folder scripts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lockfile churn from local wasm-pack (asset target) + npm operations
during the asset-graph work. No source/dependency-intent change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors merged PR #9181 so feat/asset-graph-view is self-contained
(local origin/main is stale and lacks it). Object/{ s3, storage }
form of writeS3File/loadS3File is now detected, not only the bare
s3:// string literal.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renaming e.g. duckdb read_parquet('s3://...') / loadS3File now updates
the asset->reader edge live instead of only after Save re-derives the
persisted asset rows.
- extractReads() (+ shared refsByAccess) mirroring extractWrites
- inferredReadsByPath sticky cache, filled by handleAssetsChange and
the load prefetch alongside writes
- replace the write-only overlay loop with one overlayLineage(map,
access) helper invoked for both 'w' and 'r' (net DRY)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- skip reactive ids/states/events reassignment when unchanged, so an
idle poll tick no longer re-runs the full sugiyama layout every 3-6s
- bound countedJobIds (rebuilt from eventsById in lockstep with prune)
- extract shared extractWrites() helper, replacing 4 copy-pasted
write-asset filter/map blocks in the pipeline page
- compute activeRunnable node-id once, reuse for the active-edge set
and the optimistic badge (flattened ternary); trim narrating docs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On pipeline load, eagerly infer body assets for every persisted folder
script and seed the existing inferredWritesByPath overlay, instead of
only filling it when a node is selected. Scripts whose persisted asset
rows are missing (e.g. object-form writeS3File) now have their edges
from first paint, so clicking a node no longer re-layouts the graph.
One-shot per (workspace, base-graph) load, untracked map reads,
generation-cancelled, pool-capped fetches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- collapsible folder activity log (PipelineEventLog): live job feed,
polls only while open/active, slow idle cadence, capped + pruned
- composable: observe mode + events list + run-count anchored to
graph-open time (pre-existing history excluded)
- optimistic node badge: launched script shows running instantly via
the zero-latency activeRunnable hint, keeps the polled run count
- activity pane height capped (min(18rem,40vh)) then scrolls
- route asset-graph edges through sugiyama-computed waypoints so they
go around nodes instead of under them; bezier fallback for
adjacent-layer / draft-overlay edges
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- folder-scoped queue poll lights up the downstream asset-trigger
cascade (not just the launched script); zero requests at rest,
catch-up for fast hops, auto-disarm when idle
- per-runnable node badge: last-run status + session run count
- animate unsaved/live-parsed edges (was unconditionally suppressed)
- background-pane click no longer clears selection
- run-bridge guarded so node selection/save no longer triggers a test
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: aggregate wait time should target the true root job, not flow_innermost_root_job
* refactor: reuse get_root_job_id helper for wait-time aggregation
Instead of duplicating the root_job → flow_innermost_root_job →
parent_job fallback chain inline, call the existing get_root_job_id()
helper (the same one used by push_next_flow_job) and filter out the
self-id case so standalone scripts still skip aggregate insertion.
Behaviorally identical, single source of truth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [ee] fix(nativets): pass tracing-enabled OtelConfig to deno_telemetry::init
* chore: update ee-repo-ref to 19a76a09ffb43649ee19e62d07e8b8a42d78757b
This commit updates the EE repository reference after PR #573 was merged in windmill-ee-private.
Previous ee-repo-ref: daf7ce31f192d64ca6b5602af626ab6453d2aff4
New ee-repo-ref: 19a76a09ffb43649ee19e62d07e8b8a42d78757b
Automated by sync-ee-ref workflow.
---------
Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat: include service accounts in instance settings users list
Service accounts (workspace-scoped, no password row) now appear in the
superadmin users list with a Bot icon, workspace badge, and a link to
manage them in the workspace settings. Role is locked to Operator.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update sqlx offline cache
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: use composite key for users each block
Service accounts can share emails across workspaces, so key by email + workspace_id to avoid Svelte each_key_duplicate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(python): preserve strings containing Infinity/NaN in result JSON
* test(python): add sanity checks for Infinity/NaN in results
* refactor(python): use string-aware regex callback for single-pass cleanup
* refactor(python): compact regex + handle backslash-escape parity
* perf(python): short-circuit cleanup when no Infinity/NaN/NUL in result
* feat: read-only flag on API tokens, orthogonal to scopes
Add a per-token `read_only` boolean set at creation time. When true, the
token can only call HTTP methods classified as Read (GET/HEAD/OPTIONS).
Mutating methods and job-run actions are rejected with 403, regardless of
which scopes are attached. Surfaced as a prominent toggle in the standard
token-creation flow and a discreet `2xs` toggle in MCP mode (where users
often want write access, so we don't bias them toward enabling it).
MCP enforcement: read-only tokens hide all script/flow/hub tools from
`list_tools` and only see endpoint tools whose method is GET, and the
runner rejects `call_tool` on anything mutating.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: review fixes for read-only token flag
- Exempt /api/mcp/* and /mcp/* paths from the read-only middleware check.
MCP transport runs over POST (streamable HTTP / SSE), so otherwise the
middleware would 403 every MCP request before the runner could enforce
read-only at the tool-call level.
- Tighten is_endpoint_read_only to GET only, matching the read_only_hint
that create_endpoint_annotations actually emits.
- Add unit test for check_read_only_for_route covering GET/HEAD/OPTIONS,
mutating methods, and run paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump ee-repo-ref to read-only-trigger-toggle
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(frontend): make read-only toggle discreet in both modes
Match the MCP-mode treatment in standard mode: text-tertiary, 2xs, shared
"Read-only" label. The tooltip switches per mode so the explanation still
fits the context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(frontend): gate read-only toggle behind Limit token permissions
The read-only toggle now only shows when the user has limited the token's
scopes (standard mode) or in MCP mode (which always picks an MCP scope).
Turning the limit off also resets read-only so it doesn't silently stick.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(frontend): hide incompatible MCP tools when read-only is on
When the read-only toggle is on in MCP mode:
- Endpoint badges and the custom-mode endpoint MultiSelect filter to GET.
- Already-selected non-GET endpoints are pruned from the scope.
- The scripts/flows preview is replaced with a note explaining they're
hidden (the runner already rejects script/flow runs for read-only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(frontend): place read-only toggle at top of limited scope area
The previous gate required at least one scope to be picked before the
read-only toggle appeared, which made it look missing while the user was
still building their scope list. Move the toggle inside ScopesPicker:
- Standard mode: sits directly under the "Limit token permissions" toggle
whenever Limit is on, before the scope selector.
- MCP mode: sits at the top of the MCP scope block.
readOnly is now $bindable on ScopesPicker so CreateToken still owns the
value. The auto-reset on un-limit moves into ScopesPicker too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(frontend): nest read-only toggle inside the scope list card
Place the read-only toggle at the top of the scope list (between the
Selected Scopes summary and the bordered domain list) via a new optional
topSlot snippet on ScopeSelector. Keeps ScopeSelector decoupled from
read-only specifics; ScopesPicker fills the slot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 9bc8160be50b3e57a60daf4e1b71c389a6e02b8a
This commit updates the EE repository reference after PR #571 was merged in windmill-ee-private.
Previous ee-repo-ref: f53d26e6685dfd60bfa67686fbd7358169cfd130
New ee-repo-ref: 9bc8160be50b3e57a60daf4e1b71c389a6e02b8a
Automated by sync-ee-ref workflow.
* fix: address CI review for read-only token flag
- P1 (Codex): narrow the MCP middleware exemption from "any /api/mcp/*"
to just the streamable HTTP transport endpoints
(/api/mcp/gateway, /api/mcp/w/{ws}/{mcp,sse,list_tools}). Without this,
a read-only token could POST /api/mcp/gateway/oauth/server/approve and
mint a follow-on non-read-only MCP token via the OAuth code/token
exchange.
- P2 (Claude/cubic): fix test comment/assertion mismatch — the run-path
assertion now exercises GET (which is what the RUN_PATH_ACTIONS
elevation comment describes) in addition to POST. Add a regression
assertion for /api/mcp/gateway/oauth/server/approve.
- P2 (cubic): short-circuit script/flow/hub-script/resource fetches in
MCP list_tools when read_only is on — they would only be discarded
below, so skipping the DB and resource fan-out is pure win.
- P2 (cubic): when scopes are pre-supplied via the CreateToken prop, the
ScopesPicker isn't rendered, which previously hid the read-only
toggle entirely. Render it next to the pre-supplied scopes display.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
- exists_script_by_path now filters archived = false, matching the
conflict check in create_script_internal. Previously the frontend
blocked creating a new script at a path occupied only by archived
scripts, even though renaming to that same path was allowed.
- Hide the Delete entry in the script details "..." menu unless the
user is admin. The backend delete_script_by_hash already requires
admin, so non-admins would always see an error after clicking.
* perf(dynselect): only retrigger when helper-script args actually change
Parse the inline helper's signature with the existing WASM parser and
restrict the form-arg diff to keys the helper actually consumes. Typing
into unrelated fields no longer queues a dynselect job every second.
Falls back to the previous full-args comparison when the helper is
deployed or parsing fails.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dynselect): avoid double helper-script fetch on mount
usePromise defaults to loadInit=true, so refresh() ran before the
JobLoader child was bound (firing a no-op pending promise) and the
$effect then fired a second refresh once the bind:this resolved.
Disable loadInit so the effect owns the single first call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(dynselect): use parser directly instead of inferArgs
inferArgs mutates a Schema object we never use and goes through a
shared cache; when fed an empty schema for non-main entrypoints the
caller cannot reliably read back the resulting properties. Add
parseEntrypointArgs that just runs the parser and returns the
parameter name Set (or undefined when unknown / unsupported / has
rest args / function not found). DynamicInput uses that and keeps
the previous params in flight while the next parse is computing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(dynselect): support deployed helpers in smart retrigger
Add getHelperEntrypointArgs which dispatches on HelperScript.source:
inline parses immediately; deployed fetches the script (or the flow's
inline dyn-select code) once and caches per (workspace, kind, path,
entrypoint). Without this the /scripts/get/* run view fell back to
the full-args comparison and still retriggered on unrelated fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(dynselect): zero-arg helpers report empty deps, not unknown
Codex review flagged that a valid zero-parameter entrypoint was being
treated as "couldn't determine signature" and falling back to the
full-args comparison. Distinguish "function found with no params" from
"function not found" via the parser's auto_kind field — only the
latter sets it, so empty args + auto_kind=null means a real zero-arg
helper and we return an empty Set (no retrigger on unrelated fields).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(bun): pass --preserve-symlinks on unbundled execution
Bun 1.2/1.3 moved its global package cache to a content-addressed
layout and the installer now creates a single directory symlink from
node_modules/<pkg> to the cache entry. Without --preserve-symlinks,
Bun resolves modules from each file's realpath, so any require/import
inside an installed package walks up from cache_nomount/bun/... and
never finds the sibling deps living under <job_dir>/node_modules/.
This manifested as e.g.
ENOENT while resolving package 'zod/v3' from
'/tmp/windmill/cache_nomount/bun/@langchain/core@1.1.44@@@1/dist/...'
on //nobundling scripts that pull @langchain/core, even though zod is
correctly installed alongside it in node_modules.
The bundled execution path already had --preserve-symlinks since #4132
(needed because we symlink the cached bundle file into the job dir).
The unbundled path didn't, because at the time Bun installed via per-
file hardlinks and the realpath of node_modules entries was the job
dir itself. The Bun installer's layout change made the flag necessary
on the unbundled path as well.
Add the flag to all three unbundled `bun run` invocations:
- nsjail unbundled path
- non-nsjail unbundled path
- dedicated worker (always unbundled)
This also fixes a latent bug on the first run of any bun script that
imports a package whose internals reference siblings (the build_cache
path runs unbundled this round while it builds the bundle for next
time).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(bun): regression test for nobundling + transitive require resolution
Adds an integration test that mirrors the original failure: a //nobundling
script importing @langchain/core, which (in its CJS internals) does
require('zod/v3'). Before --preserve-symlinks was added to the unbundled
bun run invocations, this failed with:
ENOENT while resolving package 'zod/v3' from
'.../cache_nomount/bun/@langchain/core@<ver>@@@1/dist/runnables/base.js'
The test covers the non-nsjail unbundled path. Reproducibility of the
pre-fix failure depends on Bun's installer choosing the directory-symlink
layout for the node_modules entry (the default on Bun 1.2/1.3+ with the
new content-addressed global cache that produced the user's error).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruben FiszelandClaude Opus 4.7
centdix