* feat: record and replay raw app sessions step by step
* fix: address review findings on raw app session recorder
* fix: stamp replay target before pruning the snapshot clone
* fix: redact step metadata, lock down replayed frames, fix control pre-state
* feat: add a checkpoint timeline to the app recording player
* fix: parser-based replay CSP, fold label clicks, drop stale frame indices
* fix: scrub redacted attributes, keep scroll, neutralize replay navigation
* fix: bound replay payloads, strip namespaced nav links, keep control pre-frames
* fix: strip SMIL navigation, redact metadata sources, capture pre-edit on beforeinput
* fix: redact template content, drop shadow templates, make replays inert
* test: pin snapshot redaction and replay sanitization with DOM tests
* fix: allow-list no-record attributes and cover a marked document root
* fix: classify input types positively so pickers get pre-change frames
* fix: one step per control interaction and bound step metadata
* fix: keep button inputs recordable and coalesce only continuous controls
* fix: no frames for coalesced repeats and drop inline styles when redacting
* fix: fold only the label's own click and keep marked stylesheets out
* fix: keep label-forwarded and radio-group pre-frames, fold submitter clicks
* fix: bound key pre-frames to their gesture and clear ancestor pointer frames
* fix: age-bound pre-frames and treat a radio group as one target
* fix: consume pre-frames per interaction and coalesce on the browser repeat flag
* fix: spend only the pre-frame a step actually used
* fix: settle a step from its successor's pre-state and drop stale pointer frames
* fix: bound remote frame payloads and snapshot stylesheets as rendered
* fix: let a control change spend its own frame and dedupe Enter activations
* fix: record Escape on controls and drop disabled stylesheets
* feat: collapse the replay step list by default behind a toggle
* fix: neutralize disabled sheets in place and fold Enter submissions
* fix: withhold redacted control state, fold key repeats, validate remote metadata
* fix: drop noscript markup and fold implicit form submissions
* fix: mask a select whose chosen option is redacted
* fix: mask redacted select choices before the clone diverges
* fix: run clone-paired passes before removals and fold only Enter submissions
* feat: record a raw app demo from the publish flow instead of the viewer
* fix: wait for in-flight runnable jobs before settling a step
* feat: record from the editor menu and replay publicly at /replay
* feat: export the app recording player and its loader for the hub
* feat: publish from folders only, drop iframe sharing
* fix: observe runnable responses where they land and mount the hub recording route
* fix: respect the app's sandbox opt-in when recording a session
* fix: let stop wait for the runnable the last step is still running
* fix: filter redacted class/id to styled tokens and gate publish on admin
* fix: drop marked sheets from the token vocabulary and bound the replay error
* test: pin the remote app-recording validator
* fix: carry in-flight runnables across a reload and fold held keys into one step
* fix: bind runnable responses off the request and honor base in the replay handoff
* fix: close the settling step when a new fill starts and always re-read stylesheets
* fix: empty the no-record marker so it carries nothing of its own
* fix: decode css escapes so utility classes survive redaction
* fix: read keyDriven from the frame the change starts from
* docs: condense recorder comments to the invariant each protects
* fix: rewrite only real url() tokens and accept leading css escapes
* feat: play flow, script and pipeline recordings on the public /replay page (#10327)
* feat: play flow, script and pipeline recordings on the public /replay page
* fix: render a recorded approval result inert while replaying
* fix: bound an asset sample's cell product and validate recording headers
* fix: make a replayed approval step inert and bound nested recording structures
* fix: stop recorded markup from fetching and bound flow/script render trees
* fix: gate recorded markdown at its renderer and close remaining render-budget gaps
* fix: replace per-key render caps with one structural budget per recorded value
* fix: bound component fan-out and text alongside the structural budget
* fix: make component fan-out cumulative and cap the parsed data-test checklist
* fix: bound the whole recording, graph contents, metadata strings and timer bursts
* fix: keep the published loader path, charge object keys, refuse huge serialized fan-out
* fix: cap flat maps a renderer turns into rows (args, schema properties)
* fix: refuse structure hidden past the depth ceiling and bound errored samples
* fix: count array-shaped argument collections against the row cap
* feat: paint canvas pixels into the snapshot
* fix: budget canvas encoding per snapshot and bound the unknown-kind error
* fix: cap flow graph overlay fan-out and condense budget comments
* docs: teach the raw-app prompt about data-wm-no-record
* feat(frontend): add data-pipeline run recorder and interactive player
Adds a recorder/player for data pipelines, mirroring the existing flow and
script recorders. Arm "Record" on a pipeline, run it, and the resulting
cascade is captured into a downloadable JSON that the /replay player can
rerun fully offline.
Because a pipeline run is a cascade of independent jobs (not a single root
SSE job like flows), the recording captures three things: the resolved
asset graph, the per-node cascade status timeline (from the orchestrator's
onUpdate), and each node's job stream (opened via getupdate_sse on launch).
The player renders the graph read-only, animates the recorded node
transitions in real time, and lets you click any node to inspect its
recorded args, logs and result — reusing the same JobLoader replay path
the flow/script players use (setActiveReplay + isReplay gating), so no
network calls are made during replay.
- recording/types.ts: PipelineRecording, PipelineTimelineFrame, RecordedNodeState
- recording/pipelineRecording.svelte.ts: createPipelineRecording() store
- recording/PipelineRecordingReplay.svelte: the player component
- replay/+page.svelte: dispatch type === 'pipeline'
- pipeline/[folder]/+page.svelte: Record toggle + Download recording; capture
the whole-pipeline / bounded cascade run
Fixes WIN-2156
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(frontend): capture DuckLake/datatable data samples in pipeline recordings
Follow-up to the pipeline recorder/player: asset nodes are now inspectable
offline in the player, showing what each table held after the recorded run.
At record finalization, for each ducklake/datatable asset in the pipeline the
recorder samples the table (up to 100 rows + columns + row count) reusing the
exact live-preview query path (loadAllTablesMetaData + getRows), so a replayed
sample matches what the asset-detail pane would have shown. Captures are
best-effort and per-asset — a missing/unconfigured table is stored as an error
marker, never thrown, so the recording still completes.
The player renders the sample as a read-only typed grid when an asset node is
clicked (script nodes keep their logs/result/args detail).
- recording/types.ts: PipelineAssetSample + assetSamples on PipelineRecording
- recording/pipelineAssetSample.ts: capturePipelineAssetSample() helper
- recording/pipelineRecording.svelte.ts: recordAssetSample() + assetSamples
- recording/PipelineRecordingReplay.svelte: asset-node data-sample panel
- pipeline/[folder]/+page.svelte: sample each asset in finalizePipelineRecording
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* recorder
* feat(hub): record data pipelines in deploy-to-hub with interactive player
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub): match editor cascade timeout, warn on cycles, reset badge on re-run
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): address review — finalize race, stale replay timers, /replay redirect, bounded sampling, jobs validation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): structural recording validation, guard-clear + SSE cleanup on throw paths
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): validate nested graph arrays and timeline frame statuses
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub): scope recording to bundle membership, fail cyclic runs, validate recording elements
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub): prune recorded graph + asset samples to bundle membership
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): validate graph.triggers array and per-job initial_job/events shapes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): guard non-object payloads, event elements, and asset-sample/code maps
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): render error boundary + validate trigger_kind and non-empty sample error
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): validate event.data and recorded-job shapes for all replay types
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): make the replay event timer crash-proof against malformed events
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): await replay completion and boundary-wrap all three players
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(recording): guard flow Play handler, cap ?src= download size, trim comment
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(parsers): keep s3 asset path suffix verbatim to preserve storage distinction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01An2pTqSmqJd2XwnagvX4kM
* package json
* fix(pipelines): preserve named storage in generated TS/Python S3 URIs
The TS/Python templates emitted `s3:///${s3Key(path)}`, stripping the
leading slash and pinning the URI to default storage. For a named-storage
asset path (`secondary/key`) that produced `s3:///secondary/key`, which
resolves to the default storage with key `secondary/key`, dropping the
named-storage dependency and reading/writing the wrong object.
Emit the path verbatim after `s3://` (matching the DuckDB template) so a
named-storage input/output keeps its storage; identical to the previous
output for default-storage paths. Removes the now-unused `s3Key` helper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(cli): align bun.lock parser versions with frontend
The PR bumped windmill-parser-wasm-asset (1.749.0→1.753.0) and
windmill-parser-wasm-regex (1.692.0→1.764.0) in package.json and the npm
package-lock.json for both cli and frontend, but cli/bun.lock was left
pinned to the old versions. Sync it so the CLI's wasm asset parser (used
by localGraph inference) matches the frontend and deploy-time parser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* feat(ai-chat): add image attachments and agent raw-app screenshots
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(ai-chat): generalise take_screenshot fidelity caveat
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): keep compaction boundary on a displayed user message
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(raw-apps): count line boxes by vertical overlap, not rect count
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): enforce vision gating and bound image attachments
Refuse images on known text-only models instead of warning and sending them
anyway; cap input bytes before decode; keep clipboard text when it accompanies
a bitmap; don't queue a message whose images can't ride the plain-text queue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* perf(ai-chat): trim take_screenshot schema and shrink the card's copy
Move the fidelity caveat from the tool def onto the tool result: the def is
re-sent every global iteration (~258 tok), while the caveat only matters once a
capture exists. Keep a downscaled copy in displayMessages when it is actually
smaller — those are never compacted and are re-cloned on every saveChat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): carry attached images through the message queue
Enter during a streaming turn queued the text and silently dropped the images,
so the auto-send was not the message the user submitted. The queue now holds
both, moved together via takeQueue/clearQueue/restoreQueue so none of the three
flush sites, the dequeue-to-composer path, or the two conversation-switch drops
can leak one without the other.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): gate screenshots on vision, narrow when the tool fires
take_screenshot buffered an image unconditionally, so a text-only model got an
image_url and rejected the turn; the attach-time check never covered it, nor a
model switched after attaching. Gate before capture and again at send.
Only reach for the tool when the user raises how the app looks, rather than
after every UI edit.
A collapsed preview keeps the iframe mounted at zero width, passing the ready
checks and then failing inside the rasteriser as '[object Event]'. Name it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): hold sending while attachments decode
addImages read the free-slot count before its await and appended after it, so a
send during the ~50-800ms decode cleared images while the closure still wrote to
them, landing the picture on the following message; two drops also claimed the
same slots and could pass the cap. Reserve slots up front, block sending until
they resolve, and show a placeholder so the held send is explained.
Keep only a bounded copy in the transcript: displayMessages are never compacted
and are re-cloned on every save. Measured 6.1x smaller per attachment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): route screenshots to the visible tab, resend full-res on retry
Every mounted raw-app editor claimed the runtime's single screenshot slot, so
take_screenshot could capture a background tab's app; ownership now follows the
visible tab and only the owner releases it.
restartGeneration resent displayMessages' images, which became a 384px thumbnail
when the transcript copy was bounded — retries downgraded the model's own input.
Recover the sent parts from the API message instead.
Move modelSupportsVision to modelConfig: it was untestable behind lib.ts's monaco
import chain, and the denylist missed bundled text-only defaults (Groq/Together
Llama 3.3, Foundry Phi-4 and Mistral-Large). Llama 3.2 and Phi-4 split by variant,
so both are matched narrowly. Pinned against the shipped defaultModels.
Decode attachments one at a time and derive the preview from the bounded copy: a
12MP bitmap is ~48MB and the batch was held live at once, decoded twice each.
The attach tooltip claimed nothing is uploaded, which is untrue for images.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): keep images out of text-only turns and bound the queue
The vision gate only dropped the current turn's images, so history's image parts
still went out after a switch to a text-only model and failed the request; strip
the outbound copy instead, leaving history intact for a switch back.
queueMessage had no cap, and each queued send clears the composer for another
eight, so repeated sends stacked an unbounded batch into one message.
Editing a message resent displayMessages' bounded copy, downgrading the model's
own input; retries recovered the full-size one but then re-persisted it at full
resolution. storedImages pairs the API message with its transcript entry so both
paths resend the original and re-persist the bounded copy.
Reserve image slots before awaiting text attachments: the gap left sending
enabled with an image pending, measured ~90ms for a 40-file drop, now ~8ms
regardless of batch size.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): treat deepseek-v4 as text-only
deepseek-v4-pro ships as a bundled default and the gate let images through to
it, so an attachment would fail the turn. DeepSeek's vision line is deepseek-vl.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): drop a rejected image instead of wedging the conversation
A provider that refuses an image leaves it in history, so every later turn
resends it and fails identically: the chat is stuck until the user edits the
message or starts over, and Retry re-sends the same image. The vision gate only
knows the models we ship, so this is the net for the rest. Strip the parts on an
image-related rejection and say so; unrelated failures keep the image.
Verified at the wire that no provider rejects a base64 data URL: anthropic
(source.base64), openai/gpt-4o (input_image), googleai and aws_bedrock/claude
(image_url passthrough) all 200 and read the image.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): match text-only models exactly, from provider API docs
The gate guessed by substring over model names, which answers the wrong
question. What matters is whether a provider's API accepts image parts, not
whether the model can see: DeepSeek V4 ships vision in its chat product that its
API has no content type for, and o3-mini gained vision in ChatGPT the API never
exposed. Neither is inferable from a name.
Substrings also block working models. 'mistral-large' matches Mistral Large 3,
which takes images; 'phi-4' matches Phi-4-multimodal, which does too. A wrong
entry blocks with no override, while a missing one costs a turn and recovers via
the rejection path, so the list is now exact ids only, each backed by a provider
doc.
Verdicts verified against provider API docs rather than recall. Live-checked
where a doc was contradicted: Bedrock's compatibility matrix claims no Anthropic
model is served over chat completions, but it serves images fine.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): stop retry resurrecting a rejected image
The rejection fallback strips the image from history but leaves the bubble's
thumbnail so the user can still see what they sent. storedImages fell back to
that thumbnail when the API message had no parts, so Retry re-attached the very
image the provider had just refused and failed identically — the conversation
stayed wedged through the one control offered to escape it.
Found by retrying in the UI; unit tests, wire tests and four review passes all
missed it, since it only exists between two separate fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): harden image rejection recovery and drop-path attachment
* fix(ai-chat): fix image drop race, mid-turn vision gate, retry aliasing
* fix(ai-chat): key vision denylist by provider, flatten alpha before jpeg
* feat(ai-chat): offer take_screenshot on chromium only, ask for one elsewhere
* feat(ai-chat): image-only sends and click-to-expand image previews
* fix(ai-chat): capture screenshots at 2x and expand tool images full-res
* feat(frontend): expandable image previews in composer and result views
* fix(ai-chat): image-only send edge cases from review round
* fix(ai-chat): keep image-only drafts on rollback, track failing model id
* fix(ai-chat): gate rejection recovery on the failing iteration's model
* refactor(ai-chat): record iteration model via onBeforeIteration, trim tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): restore composer draft when beforeSend preflight fails
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): bound cumulative outbound image bytes per request
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): make the image byte bound part-granular so over-cap turns keep a subset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): evict newest-first within a message in the image byte bound
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): prune over-cap images from stored history, not just requests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): bound history at every save boundary, keep thumbnail pairing across eviction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): slot-align storedImages so the bubble expands the right image after eviction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): match rejection keywords as whole words so provisioning errors keep images
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): match input_image rejections, restore images refused by non-GLOBAL modes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): defer non-GLOBAL image refusal restore past the composer clear
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): persist full tool screenshots for post-reload expansion
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai-chat): persist chat images out-of-band via blob-store refs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): scope image blobs per chat and stop cap-eviction rotation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): keep blob-cap chronology across drop-oldest compaction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai-chat): derive blob eviction from the saved record, not write times
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): serialize chat history DB writes per manager
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): pin queued history writes to the enqueue-time user database
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): delete stale image blobs only after the chat record commits
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): don't double-restore a queued image-only draft on vision refusal
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): label image-only chats and evicted image-only bubbles
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): keep the in-memory chat mirror hydrated for DB-less sessions
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): converge the chat mirror to refs after a successful DB commit
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): guard mirror convergence against rewinding newer saves
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): invalidate pending convergences on identity re-init, keep retry image names
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): bound the screenshot raster before rasterization
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai-chat): drop the no-IndexedDB in-memory image fallback
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: parse all names in grouped go param declarations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump windmill-parser-wasm-go to 1.761.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(frontend): sanitize html and svg result rendering
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): add dompurify to lockfile root deps
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): keep sanitizing rich results on public app surfaces
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(frontend): gate risky app markup on unsandboxed public surfaces
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): derive app markup isolation from the real origin
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(frontend): use the design-system danger alert for the markup gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add datatable_migrations table
* feat: add route to run datatable migrations
* feat: sync datatable migrations as .up.sql/.down.sql files
* feat: add datatable migrate up/down commands and post-push run prompt
* feat: add datatable migrate new command to scaffold migrations
* feat: add datatable migrations management UI
* feat: prompt to create migration on DDL in datatable SQL editors
* feat: support running a single specific datatable migration
* feat: view migration content, run single migration, fix stacked modal
* feat: per-row revert button with out-of-order warning
* fix: avoid migrations list flicker on refresh after an action
* feat: generate initial datatable migration via pg_dump
* fix: surface datatable migration API error details in toasts
* fix: revert created migration if create-and-run fails to run
* fix: include postgres error detail in migration run/rollback failures
* feat: sync datatable migrations as files via the workspace export
* refactor: move datatable migrations to migrations/datatable/ path
* fix: drop redundant datatable_migration label in sync output
* fix: exclude datatable migration sql files from script metadata generation
* feat: run datatable migrations as user-permissioned labeled jobs
* feat: reject invalid datatable migrations on sync push
* feat: datatable migrate up/down default to all datatables, --datatable to target one
* fix: surface postgres error detail when datatable migrations fail to run
* chore: regenerate CLI docs for datatable migrate commands
* feat: default new datatable migration to a BEGIN/END transaction template
* fix: validate datatable migration name and datatable at the API boundary
* fix: ensure detected DDL ends with semicolon when wrapped in transaction
* fix: re-prompt instead of stripping DDL when new-migration modal is cancelled
* feat: refresh datatable schema after running a migration from the SQL REPL
* feat: record db manager DDL on data tables as migrations
* feat: make datatable migrations opt-in per data table
* fix: make migration view editor read-only so its code can scroll
* fix: don't re-prompt DDL guard when creating a migration without running
* feat: generate down migrations for db manager DDL (postgres)
* fix: correct down migration for db manager alters (no double-wrap, serial)
* feat: explain migrations purpose with a tooltip in the migrations modal
* compare paeg
* feat: add datatable_migration kind to workspace diff pipeline
* chore: point ee-repo-ref at datatable_migration git-sync companion
* fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests
* feat: deploy and run datatable migrations on workspace merge
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Refactor + handle datatable setting delete/rename
* refactor: move datatable migration rename/delete cascade into module
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(db-manager): add Migrations button to top bar, make Refresh icon-only
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* BEGIN/END placeholder in down migration
* feat: autofocus migration name input and flag it red when empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* border nits
* refresh db manager schema on migrations
* BEGIN/END scaffold in CLI
* feat(cli): push local datatable migrations before running on migrate up
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: flag invalid migration name with red border, not just empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop random slug from auto-generated migration names
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: offer revert-and-delete when deleting an installed migration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: record fork merge as a migration when target datatable opts in
* nit
* clone migrations on fork
* windmill-utils-internal
* fix(datatable-migrations): serialize run/rollback with a per-db advisory lock
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db-manager): fail closed when migrations-status check errors on DDL apply
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: fix generate_initial migration ordering comment to match code
* chore(datatable-migrations): remove unused update_datatable_migrations endpoint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: run DDL migration guard on the script editor Test button
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* split
* ee-repo-ref
* chore(frontend): sync package-lock with package.json (@emnapi deps)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(datatable-migrations): never resolve instance credentials into migration job args
datatable_database_arg eagerly resolved instance data-table credentials
(including the shared instance-wide Postgres password) and passed them as the
migration job's plaintext `database` arg, landing in v2_job.args. Since the
run route has no admin gate, a non-admin could run a migration and read
args.database to recover the password, granting cross-workspace psql access to
all instance data-table DBs.
Pass a `datatable://<name>` reference for both resource-backed and instance
data tables instead; the pg executor already resolves it to real credentials
server-side at run time, so nothing sensitive is ever stored in the job args.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* fix: handle dollar-quoting and comments when splitting SQL statements
* feat: deploy datatable migrations on merge with explicit opt-in error
* fix(frontend): sync package-lock with npm 11 peer-dep resolution
npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from
lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as
peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly
1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the
highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree
needs both versions; the committed lock only had 1.10.0.
Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for
the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit npm publish
* fix: fail closed on migrations-status error in fork schema merge
* nit CI emnapi/core version
* prevent initial_datatable_migration if migrations already exist
* fix(datatable-migrations): validate persisted data table names as path segments
edit_datatable_config only validated rename segments, not the actual
settings.datatables keys, so a data table could be saved directly under a name
like '..' or one containing '/'. Since new tables default to
migrations_enabled = true, generate_initial_datatable_migration would then
insert a migration row and the sync export would build
migrations/datatable/<name>/... paths from that name, producing malformed or
directory-escaping export paths.
Validate every persisted data table name in edit_datatable_config (alongside
the existing rename checks) and add validate_datatable_path_segment to
generate_initial_datatable_migration for defense in depth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: scope datatable _wm_migrations by data table and cascade renames/deletes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(system_prompts): resolve nested local command groups in CLI docs generator
The CLI docs generator anchored on the first `new Command()` in a file and
never resolved locally-defined command groups passed as
`.command("name", localCmd)`. For datatable this flattened the nested
`migrate` group: it emitted `datatable new/up/down` plus a bare
`datatable migrate`, and mislabeled the datatable command with the migrate
group's description. jobs was broken the same way (its description was pull's,
and pull/push rendered empty).
Anchor block extraction on the `export default`ed command, recurse into
locally-defined `const x = new Command()` groups mounted as subcommands, and
render nested sub-subcommands. Regenerated docs now show
`datatable migrate new/up/down` and `jobs pull/push` with their real
options.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop unreleased _wm_migrations legacy-upgrade handling
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: return datatable migration SQL from getItemValue for the diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* nit
* fix: handle datatable migration renames on push and dedupe timestamps
* fix: reject rewriting an already-applied datatable migration on upsert
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries
Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi
declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved
lockfile entries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): datatable migrate up/down default to main datatable, not all
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: fail closed when applied status unreadable on datatable migration rewrite
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: surface full error detail in Database Manager DDL/query errors
* "See migration" button in the toast
* feat: add Enter shortcut to Create-a-migration in the DDL guard
* fix(frontend): warn before running a newly-created datatable migration out of order
The row-level Run action warns when earlier migrations are still pending, but
the create-and-run paths ran a just-created migration with `only` directly,
applying it ahead of older pending migrations without that confirmation.
Reuse the same "Run migration out of order" confirmation across all
create-and-run paths via a shared helper (datatableMigrationUtils):
- NewDataTableMigrationModal "Create and run" (and the DDL guard path)
- DatatableSchemaDiff fork→parent merge
- dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a
MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a
silent cancel
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: keep renamed datatable migrations visible in compare view
* fix: record per-migration deployment on datatable migrations disable
* fix(cli): run deployed datatable migrations after workspace merge
The merge command upserted datatable_migration definitions into the target
workspace and reported the item as successfully deployed, but never ran the
migrations. For forked datatables backed by separate databases, this left the
target schema unchanged until someone manually ran `wmill datatable migrate up`,
while the CLI reported a successful merge.
Collect the datatable migrations deployed (not deleted) into the target and,
after the deploy loop, offer to run them via the existing offerToRunNewMigrations
helper — the same post-deploy run prompt the push/sync path uses (interactive
only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export
parseDatatableMigrationDeployPath so the merge path can parse the deployed items.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(backend): serialize datatable migration edits/deletes with the run lock
A migration run snapshots a migration's code_up from datatable_migrations and
only records its version in the data table's _wm_migrations after the job
succeeds. upsert_datatable_migration checked _wm_migrations before allowing an
edit but took no lock, so a concurrent edit could read "not applied yet",
rewrite code_up/code_down, and then the in-flight run would record the version
for the old SQL — leaving _wm_migrations pointing at SQL that was never applied
(migrate up then skips it; rollback runs a down that doesn't match).
Serialize definition rewrites and deletes with the same per-database advisory
lock the run/rollback paths use:
- Factor the connect+advisory-lock into lock_datatable_migration_runs and the
applied-versions read into read_applied_versions_on_client.
- run_datatable_migrations now snapshots the definitions AFTER taking the lock,
so code_up can't change between snapshot and version-record.
- upsert (when changing an existing def) and delete take the lock across the
applied-check and the write; delete now rejects deleting an already-applied
migration (would orphan its _wm_migrations record), symmetric with upsert.
Both fail closed if the data table database is unreachable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): stack the out-of-order migration confirm above the DB editor preview
Creating a table on a migrations-enabled data table opened the DB table editor's
"Confirm running the following" preview modal, whose confirm triggers applyDdl,
which then asks for out-of-order confirmation. Both are ConfirmationModals with a
hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before
the editor), so it rendered behind the still-open preview modal.
Add an optional zIndexClass prop to ConfirmationModal (default z-[9999],
backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it
stacks on top.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c
This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private.
Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4
New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix(cli): emit HD-1 test_edges + HD-2 scd2 _current write in --local pipeline graph
Close the remaining local-vs-deployed graph parity gaps in `wmill pipeline
show <folder> --local` so it matches the deployed graph (backend
`asset_graph`, windmill-api-assets):
- HD-1 `test_edges`: synthesize ordering-only producer → tested-script edges
from parsed `// data_test` annotations. A `relationships` test references
its `to_path` asset; a custom `// data_test <script>` resolves best-effort
against that script's parsed reads. Each referenced asset is resolved to its
in-pipeline producer via the write edges; self-edges and producer-less
(external) assets are dropped — mirroring the backend set semantics.
Routed through the asset node in boundedCascade's lineage DAG (asset →
tested script) so a cold/bounded cascade orders the referenced dimension
first, matching the frontend.
- HD-2 scd2 `<dim>_current` companion write: a managed `// materialize …
history` (scd2 && !manual) also produces a `<dim>_current` view. Register it
as a second write edge and mark the asset `derived_from` its base dimension,
so a consumer reading only the view links back to the producer instead of
orphaning. Gated exactly like the backend `MaterializeSpec::write_targets` /
`scd2_current_target`.
The pinned `windmill-parser-wasm-asset` (1.740.0) predates the `scd2`
materialize flag, so `buildLocalPipelineGraph` takes an injectable parser and
the HD-2 test injects one that re-adds `scd2` for a `history` materialize —
exercising the already-shipped companion-write branch until a wasm carrying
`scd2` is republished (cf. #9926).
Extends cli/test/pipeline_local_graph_unit.test.ts with HD-1 (relationships,
no-producer, self-test, custom) and HD-2 coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(cli): pin windmill-parser-wasm-asset 1.749.0, drop HD-2 test parser seam
Now that windmill-parser-wasm-asset 1.749.0 (which serializes the `scd2`
materialize flag) is published, bump the CLI pin and retire the temporary
injection seam:
- Remove the `infer?` parameter from `buildLocalPipelineGraph`; it always uses
the wasm-backed `inferScriptAssets` again.
- The HD-2 `<dim>_current` companion-write test drives the real wasm directly
(drops the `inferWithScd2` wrapper that re-added `scd2` against the pinned
1.740.0 build).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): pin windmill-parser-wasm-asset 1.749.0 to match CLI
Restore the CLI↔frontend lockstep on the asset parser wasm broken by the
previous commit: every other windmill-parser-wasm-* package is pinned to the
same version in both cli/package.json and frontend/package.json, so keep the
asset parser aligned too. The frontend derives materialize/scd2 from its own
TS annotation parser (`parsePipelineAnnotations`), so this bump only affects
body asset inference in the live graph — moving it in step with the CLI
`--local` graph and the deployed backend parser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Ruben Fiszel
Diego Imbert
hugocasa