* fix(parsers): keep s3 asset path suffix verbatim to preserve storage distinction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01An2pTqSmqJd2XwnagvX4kM
* package json
* fix(pipelines): preserve named storage in generated TS/Python S3 URIs
The TS/Python templates emitted `s3:///${s3Key(path)}`, stripping the
leading slash and pinning the URI to default storage. For a named-storage
asset path (`secondary/key`) that produced `s3:///secondary/key`, which
resolves to the default storage with key `secondary/key`, dropping the
named-storage dependency and reading/writing the wrong object.
Emit the path verbatim after `s3://` (matching the DuckDB template) so a
named-storage input/output keeps its storage; identical to the previous
output for default-storage paths. Removes the now-unused `s3Key` helper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(cli): align bun.lock parser versions with frontend
The PR bumped windmill-parser-wasm-asset (1.749.0→1.753.0) and
windmill-parser-wasm-regex (1.692.0→1.764.0) in package.json and the npm
package-lock.json for both cli and frontend, but cli/bun.lock was left
pinned to the old versions. Sync it so the CLI's wasm asset parser (used
by localGraph inference) matches the frontend and deploy-time parser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* feat(ai-chat): add image attachments and agent raw-app screenshots
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(ai-chat): generalise take_screenshot fidelity caveat
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): keep compaction boundary on a displayed user message
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(raw-apps): count line boxes by vertical overlap, not rect count
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): enforce vision gating and bound image attachments
Refuse images on known text-only models instead of warning and sending them
anyway; cap input bytes before decode; keep clipboard text when it accompanies
a bitmap; don't queue a message whose images can't ride the plain-text queue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* perf(ai-chat): trim take_screenshot schema and shrink the card's copy
Move the fidelity caveat from the tool def onto the tool result: the def is
re-sent every global iteration (~258 tok), while the caveat only matters once a
capture exists. Keep a downscaled copy in displayMessages when it is actually
smaller — those are never compacted and are re-cloned on every saveChat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): carry attached images through the message queue
Enter during a streaming turn queued the text and silently dropped the images,
so the auto-send was not the message the user submitted. The queue now holds
both, moved together via takeQueue/clearQueue/restoreQueue so none of the three
flush sites, the dequeue-to-composer path, or the two conversation-switch drops
can leak one without the other.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): gate screenshots on vision, narrow when the tool fires
take_screenshot buffered an image unconditionally, so a text-only model got an
image_url and rejected the turn; the attach-time check never covered it, nor a
model switched after attaching. Gate before capture and again at send.
Only reach for the tool when the user raises how the app looks, rather than
after every UI edit.
A collapsed preview keeps the iframe mounted at zero width, passing the ready
checks and then failing inside the rasteriser as '[object Event]'. Name it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): hold sending while attachments decode
addImages read the free-slot count before its await and appended after it, so a
send during the ~50-800ms decode cleared images while the closure still wrote to
them, landing the picture on the following message; two drops also claimed the
same slots and could pass the cap. Reserve slots up front, block sending until
they resolve, and show a placeholder so the held send is explained.
Keep only a bounded copy in the transcript: displayMessages are never compacted
and are re-cloned on every save. Measured 6.1x smaller per attachment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): route screenshots to the visible tab, resend full-res on retry
Every mounted raw-app editor claimed the runtime's single screenshot slot, so
take_screenshot could capture a background tab's app; ownership now follows the
visible tab and only the owner releases it.
restartGeneration resent displayMessages' images, which became a 384px thumbnail
when the transcript copy was bounded — retries downgraded the model's own input.
Recover the sent parts from the API message instead.
Move modelSupportsVision to modelConfig: it was untestable behind lib.ts's monaco
import chain, and the denylist missed bundled text-only defaults (Groq/Together
Llama 3.3, Foundry Phi-4 and Mistral-Large). Llama 3.2 and Phi-4 split by variant,
so both are matched narrowly. Pinned against the shipped defaultModels.
Decode attachments one at a time and derive the preview from the bounded copy: a
12MP bitmap is ~48MB and the batch was held live at once, decoded twice each.
The attach tooltip claimed nothing is uploaded, which is untrue for images.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): keep images out of text-only turns and bound the queue
The vision gate only dropped the current turn's images, so history's image parts
still went out after a switch to a text-only model and failed the request; strip
the outbound copy instead, leaving history intact for a switch back.
queueMessage had no cap, and each queued send clears the composer for another
eight, so repeated sends stacked an unbounded batch into one message.
Editing a message resent displayMessages' bounded copy, downgrading the model's
own input; retries recovered the full-size one but then re-persisted it at full
resolution. storedImages pairs the API message with its transcript entry so both
paths resend the original and re-persist the bounded copy.
Reserve image slots before awaiting text attachments: the gap left sending
enabled with an image pending, measured ~90ms for a 40-file drop, now ~8ms
regardless of batch size.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): treat deepseek-v4 as text-only
deepseek-v4-pro ships as a bundled default and the gate let images through to
it, so an attachment would fail the turn. DeepSeek's vision line is deepseek-vl.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): drop a rejected image instead of wedging the conversation
A provider that refuses an image leaves it in history, so every later turn
resends it and fails identically: the chat is stuck until the user edits the
message or starts over, and Retry re-sends the same image. The vision gate only
knows the models we ship, so this is the net for the rest. Strip the parts on an
image-related rejection and say so; unrelated failures keep the image.
Verified at the wire that no provider rejects a base64 data URL: anthropic
(source.base64), openai/gpt-4o (input_image), googleai and aws_bedrock/claude
(image_url passthrough) all 200 and read the image.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): match text-only models exactly, from provider API docs
The gate guessed by substring over model names, which answers the wrong
question. What matters is whether a provider's API accepts image parts, not
whether the model can see: DeepSeek V4 ships vision in its chat product that its
API has no content type for, and o3-mini gained vision in ChatGPT the API never
exposed. Neither is inferable from a name.
Substrings also block working models. 'mistral-large' matches Mistral Large 3,
which takes images; 'phi-4' matches Phi-4-multimodal, which does too. A wrong
entry blocks with no override, while a missing one costs a turn and recovers via
the rejection path, so the list is now exact ids only, each backed by a provider
doc.
Verdicts verified against provider API docs rather than recall. Live-checked
where a doc was contradicted: Bedrock's compatibility matrix claims no Anthropic
model is served over chat completions, but it serves images fine.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): stop retry resurrecting a rejected image
The rejection fallback strips the image from history but leaves the bubble's
thumbnail so the user can still see what they sent. storedImages fell back to
that thumbnail when the API message had no parts, so Retry re-attached the very
image the provider had just refused and failed identically — the conversation
stayed wedged through the one control offered to escape it.
Found by retrying in the UI; unit tests, wire tests and four review passes all
missed it, since it only exists between two separate fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ai-chat): harden image rejection recovery and drop-path attachment
* fix(ai-chat): fix image drop race, mid-turn vision gate, retry aliasing
* fix(ai-chat): key vision denylist by provider, flatten alpha before jpeg
* feat(ai-chat): offer take_screenshot on chromium only, ask for one elsewhere
* feat(ai-chat): image-only sends and click-to-expand image previews
* fix(ai-chat): capture screenshots at 2x and expand tool images full-res
* feat(frontend): expandable image previews in composer and result views
* fix(ai-chat): image-only send edge cases from review round
* fix(ai-chat): keep image-only drafts on rollback, track failing model id
* fix(ai-chat): gate rejection recovery on the failing iteration's model
* refactor(ai-chat): record iteration model via onBeforeIteration, trim tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): restore composer draft when beforeSend preflight fails
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): bound cumulative outbound image bytes per request
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): make the image byte bound part-granular so over-cap turns keep a subset
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): evict newest-first within a message in the image byte bound
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): prune over-cap images from stored history, not just requests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): bound history at every save boundary, keep thumbnail pairing across eviction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): slot-align storedImages so the bubble expands the right image after eviction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): match rejection keywords as whole words so provisioning errors keep images
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): match input_image rejections, restore images refused by non-GLOBAL modes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): defer non-GLOBAL image refusal restore past the composer clear
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): persist full tool screenshots for post-reload expansion
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai-chat): persist chat images out-of-band via blob-store refs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): scope image blobs per chat and stop cap-eviction rotation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): keep blob-cap chronology across drop-oldest compaction
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai-chat): derive blob eviction from the saved record, not write times
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): serialize chat history DB writes per manager
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): pin queued history writes to the enqueue-time user database
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): delete stale image blobs only after the chat record commits
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): don't double-restore a queued image-only draft on vision refusal
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): label image-only chats and evicted image-only bubbles
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): keep the in-memory chat mirror hydrated for DB-less sessions
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): converge the chat mirror to refs after a successful DB commit
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): guard mirror convergence against rewinding newer saves
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): invalidate pending convergences on identity re-init, keep retry image names
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai-chat): bound the screenshot raster before rasterization
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai-chat): drop the no-IndexedDB in-memory image fallback
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: parse all names in grouped go param declarations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump windmill-parser-wasm-go to 1.761.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(frontend): sanitize html and svg result rendering
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): add dompurify to lockfile root deps
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): keep sanitizing rich results on public app surfaces
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(frontend): gate risky app markup on unsandboxed public surfaces
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): derive app markup isolation from the real origin
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(frontend): use the design-system danger alert for the markup gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add datatable_migrations table
* feat: add route to run datatable migrations
* feat: sync datatable migrations as .up.sql/.down.sql files
* feat: add datatable migrate up/down commands and post-push run prompt
* feat: add datatable migrate new command to scaffold migrations
* feat: add datatable migrations management UI
* feat: prompt to create migration on DDL in datatable SQL editors
* feat: support running a single specific datatable migration
* feat: view migration content, run single migration, fix stacked modal
* feat: per-row revert button with out-of-order warning
* fix: avoid migrations list flicker on refresh after an action
* feat: generate initial datatable migration via pg_dump
* fix: surface datatable migration API error details in toasts
* fix: revert created migration if create-and-run fails to run
* fix: include postgres error detail in migration run/rollback failures
* feat: sync datatable migrations as files via the workspace export
* refactor: move datatable migrations to migrations/datatable/ path
* fix: drop redundant datatable_migration label in sync output
* fix: exclude datatable migration sql files from script metadata generation
* feat: run datatable migrations as user-permissioned labeled jobs
* feat: reject invalid datatable migrations on sync push
* feat: datatable migrate up/down default to all datatables, --datatable to target one
* fix: surface postgres error detail when datatable migrations fail to run
* chore: regenerate CLI docs for datatable migrate commands
* feat: default new datatable migration to a BEGIN/END transaction template
* fix: validate datatable migration name and datatable at the API boundary
* fix: ensure detected DDL ends with semicolon when wrapped in transaction
* fix: re-prompt instead of stripping DDL when new-migration modal is cancelled
* feat: refresh datatable schema after running a migration from the SQL REPL
* feat: record db manager DDL on data tables as migrations
* feat: make datatable migrations opt-in per data table
* fix: make migration view editor read-only so its code can scroll
* fix: don't re-prompt DDL guard when creating a migration without running
* feat: generate down migrations for db manager DDL (postgres)
* fix: correct down migration for db manager alters (no double-wrap, serial)
* feat: explain migrations purpose with a tooltip in the migrations modal
* compare paeg
* feat: add datatable_migration kind to workspace diff pipeline
* chore: point ee-repo-ref at datatable_migration git-sync companion
* fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests
* feat: deploy and run datatable migrations on workspace merge
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Refactor + handle datatable setting delete/rename
* refactor: move datatable migration rename/delete cascade into module
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(db-manager): add Migrations button to top bar, make Refresh icon-only
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* BEGIN/END placeholder in down migration
* feat: autofocus migration name input and flag it red when empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* border nits
* refresh db manager schema on migrations
* BEGIN/END scaffold in CLI
* feat(cli): push local datatable migrations before running on migrate up
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: flag invalid migration name with red border, not just empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop random slug from auto-generated migration names
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: offer revert-and-delete when deleting an installed migration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: record fork merge as a migration when target datatable opts in
* nit
* clone migrations on fork
* windmill-utils-internal
* fix(datatable-migrations): serialize run/rollback with a per-db advisory lock
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db-manager): fail closed when migrations-status check errors on DDL apply
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: fix generate_initial migration ordering comment to match code
* chore(datatable-migrations): remove unused update_datatable_migrations endpoint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: run DDL migration guard on the script editor Test button
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* split
* ee-repo-ref
* chore(frontend): sync package-lock with package.json (@emnapi deps)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(datatable-migrations): never resolve instance credentials into migration job args
datatable_database_arg eagerly resolved instance data-table credentials
(including the shared instance-wide Postgres password) and passed them as the
migration job's plaintext `database` arg, landing in v2_job.args. Since the
run route has no admin gate, a non-admin could run a migration and read
args.database to recover the password, granting cross-workspace psql access to
all instance data-table DBs.
Pass a `datatable://<name>` reference for both resource-backed and instance
data tables instead; the pg executor already resolves it to real credentials
server-side at run time, so nothing sensitive is ever stored in the job args.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* fix: handle dollar-quoting and comments when splitting SQL statements
* feat: deploy datatable migrations on merge with explicit opt-in error
* fix(frontend): sync package-lock with npm 11 peer-dep resolution
npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from
lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as
peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly
1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the
highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree
needs both versions; the committed lock only had 1.10.0.
Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for
the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit npm publish
* fix: fail closed on migrations-status error in fork schema merge
* nit CI emnapi/core version
* prevent initial_datatable_migration if migrations already exist
* fix(datatable-migrations): validate persisted data table names as path segments
edit_datatable_config only validated rename segments, not the actual
settings.datatables keys, so a data table could be saved directly under a name
like '..' or one containing '/'. Since new tables default to
migrations_enabled = true, generate_initial_datatable_migration would then
insert a migration row and the sync export would build
migrations/datatable/<name>/... paths from that name, producing malformed or
directory-escaping export paths.
Validate every persisted data table name in edit_datatable_config (alongside
the existing rename checks) and add validate_datatable_path_segment to
generate_initial_datatable_migration for defense in depth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: scope datatable _wm_migrations by data table and cascade renames/deletes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(system_prompts): resolve nested local command groups in CLI docs generator
The CLI docs generator anchored on the first `new Command()` in a file and
never resolved locally-defined command groups passed as
`.command("name", localCmd)`. For datatable this flattened the nested
`migrate` group: it emitted `datatable new/up/down` plus a bare
`datatable migrate`, and mislabeled the datatable command with the migrate
group's description. jobs was broken the same way (its description was pull's,
and pull/push rendered empty).
Anchor block extraction on the `export default`ed command, recurse into
locally-defined `const x = new Command()` groups mounted as subcommands, and
render nested sub-subcommands. Regenerated docs now show
`datatable migrate new/up/down` and `jobs pull/push` with their real
options.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop unreleased _wm_migrations legacy-upgrade handling
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: return datatable migration SQL from getItemValue for the diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* nit
* fix: handle datatable migration renames on push and dedupe timestamps
* fix: reject rewriting an already-applied datatable migration on upsert
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries
Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi
declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved
lockfile entries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): datatable migrate up/down default to main datatable, not all
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: fail closed when applied status unreadable on datatable migration rewrite
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: surface full error detail in Database Manager DDL/query errors
* "See migration" button in the toast
* feat: add Enter shortcut to Create-a-migration in the DDL guard
* fix(frontend): warn before running a newly-created datatable migration out of order
The row-level Run action warns when earlier migrations are still pending, but
the create-and-run paths ran a just-created migration with `only` directly,
applying it ahead of older pending migrations without that confirmation.
Reuse the same "Run migration out of order" confirmation across all
create-and-run paths via a shared helper (datatableMigrationUtils):
- NewDataTableMigrationModal "Create and run" (and the DDL guard path)
- DatatableSchemaDiff fork→parent merge
- dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a
MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a
silent cancel
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: keep renamed datatable migrations visible in compare view
* fix: record per-migration deployment on datatable migrations disable
* fix(cli): run deployed datatable migrations after workspace merge
The merge command upserted datatable_migration definitions into the target
workspace and reported the item as successfully deployed, but never ran the
migrations. For forked datatables backed by separate databases, this left the
target schema unchanged until someone manually ran `wmill datatable migrate up`,
while the CLI reported a successful merge.
Collect the datatable migrations deployed (not deleted) into the target and,
after the deploy loop, offer to run them via the existing offerToRunNewMigrations
helper — the same post-deploy run prompt the push/sync path uses (interactive
only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export
parseDatatableMigrationDeployPath so the merge path can parse the deployed items.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(backend): serialize datatable migration edits/deletes with the run lock
A migration run snapshots a migration's code_up from datatable_migrations and
only records its version in the data table's _wm_migrations after the job
succeeds. upsert_datatable_migration checked _wm_migrations before allowing an
edit but took no lock, so a concurrent edit could read "not applied yet",
rewrite code_up/code_down, and then the in-flight run would record the version
for the old SQL — leaving _wm_migrations pointing at SQL that was never applied
(migrate up then skips it; rollback runs a down that doesn't match).
Serialize definition rewrites and deletes with the same per-database advisory
lock the run/rollback paths use:
- Factor the connect+advisory-lock into lock_datatable_migration_runs and the
applied-versions read into read_applied_versions_on_client.
- run_datatable_migrations now snapshots the definitions AFTER taking the lock,
so code_up can't change between snapshot and version-record.
- upsert (when changing an existing def) and delete take the lock across the
applied-check and the write; delete now rejects deleting an already-applied
migration (would orphan its _wm_migrations record), symmetric with upsert.
Both fail closed if the data table database is unreachable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): stack the out-of-order migration confirm above the DB editor preview
Creating a table on a migrations-enabled data table opened the DB table editor's
"Confirm running the following" preview modal, whose confirm triggers applyDdl,
which then asks for out-of-order confirmation. Both are ConfirmationModals with a
hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before
the editor), so it rendered behind the still-open preview modal.
Add an optional zIndexClass prop to ConfirmationModal (default z-[9999],
backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it
stacks on top.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c
This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private.
Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4
New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix(cli): emit HD-1 test_edges + HD-2 scd2 _current write in --local pipeline graph
Close the remaining local-vs-deployed graph parity gaps in `wmill pipeline
show <folder> --local` so it matches the deployed graph (backend
`asset_graph`, windmill-api-assets):
- HD-1 `test_edges`: synthesize ordering-only producer → tested-script edges
from parsed `// data_test` annotations. A `relationships` test references
its `to_path` asset; a custom `// data_test <script>` resolves best-effort
against that script's parsed reads. Each referenced asset is resolved to its
in-pipeline producer via the write edges; self-edges and producer-less
(external) assets are dropped — mirroring the backend set semantics.
Routed through the asset node in boundedCascade's lineage DAG (asset →
tested script) so a cold/bounded cascade orders the referenced dimension
first, matching the frontend.
- HD-2 scd2 `<dim>_current` companion write: a managed `// materialize …
history` (scd2 && !manual) also produces a `<dim>_current` view. Register it
as a second write edge and mark the asset `derived_from` its base dimension,
so a consumer reading only the view links back to the producer instead of
orphaning. Gated exactly like the backend `MaterializeSpec::write_targets` /
`scd2_current_target`.
The pinned `windmill-parser-wasm-asset` (1.740.0) predates the `scd2`
materialize flag, so `buildLocalPipelineGraph` takes an injectable parser and
the HD-2 test injects one that re-adds `scd2` for a `history` materialize —
exercising the already-shipped companion-write branch until a wasm carrying
`scd2` is republished (cf. #9926).
Extends cli/test/pipeline_local_graph_unit.test.ts with HD-1 (relationships,
no-producer, self-test, custom) and HD-2 coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(cli): pin windmill-parser-wasm-asset 1.749.0, drop HD-2 test parser seam
Now that windmill-parser-wasm-asset 1.749.0 (which serializes the `scd2`
materialize flag) is published, bump the CLI pin and retire the temporary
injection seam:
- Remove the `infer?` parameter from `buildLocalPipelineGraph`; it always uses
the wasm-backed `inferScriptAssets` again.
- The HD-2 `<dim>_current` companion-write test drives the real wasm directly
(drops the `inferWithScd2` wrapper that re-added `scd2` against the pinned
1.740.0 build).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): pin windmill-parser-wasm-asset 1.749.0 to match CLI
Restore the CLI↔frontend lockstep on the asset parser wasm broken by the
previous commit: every other windmill-parser-wasm-* package is pinned to the
same version in both cli/package.json and frontend/package.json, so keep the
asset parser aligned too. The frontend derives materialize/scd2 from its own
TS annotation parser (`parsePipelineAnnotations`), so this bump only affects
body asset inference in the live graph — moving it in step with the CLI
`--local` graph and the deployed backend parser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: column-level lineage for ducklake pipelines via // column annotation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat: auto-derive column lineage from DuckDB SQL AST (annotation as override)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: clarify column-lineage inference is server-side; drafts use annotations
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(frontend): surface inferred column lineage in live pipeline drafts
Threads the DuckDB SQL-AST column lineage (from the WASM asset parser) through
ScriptEditor -> details pane -> page -> resolveGraph, merged with // column
annotations (annotation wins) so the live preview matches the deployed graph.
Takes effect once windmill-parser-wasm-asset is republished with the inference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(frontend): bump windmill-parser-wasm-asset to 1.740.0 for SQL column-lineage inference
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: column-lineage inference now runs live (WASM) too, merged with annotations
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(frontend): transitive column-lineage trace (impact analysis)
Stitches every producer's column_lineage into a pipeline-wide column graph
(columnLineageGraph.ts) and replaces the single-hop diagram with an
interactive ColumnLineageTrace: select an asset to see its columns' full
upstream/downstream lineage across scripts; click any column to highlight its
complete transitive impact set (forward + backward) and dim the rest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: address CI review on column lineage (parse-fallback, node-id, perf, leak)
- backend: DuckDB SQL parse failure now falls back to `// column` annotation
lineage instead of dropping it (Codex P1)
- columnLineageGraph: collision-proof JSON node ids; deterministic first-write
output anchoring when a producer has multiple ducklake writes (cubic P2 ×2)
- pipeline page: gate buildColumnGraph to a ducklake-asset selection so it
doesn't rebuild on every editor keystroke (cubic P2)
- ScriptEditor: clear inferredColumnLineage on parse error so it can't leak
across a script switch (cubic P2)
- AssetGraphEdge: widen badge stacking offset 12px->18px to fully clear (cubic P3)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: resolve JOIN inputs + anchor column lineage to // materialize target
Addresses the second Codex review pass (two P1s):
- SQL inference now walks JOINed tables: build_from_maps maps every FROM entry
AND its joins into the alias map, and single-table attribution requires no
joins. `SELECT o.x, c.y FROM a o JOIN b c` now resolves c.y (was dropped).
- The column graph anchors a producer's lineage to its declared // materialize
target (surfaced on the runnable node) instead of guessing a ducklake
write-edge, which is unordered for deployed graphs and ambiguous for
multi-output scripts. Falls back to a write-edge when no materialize target.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: gate column-lineage badge to the // materialize target write-edge
The canvas badge keyed on `e.asset_kind === 'ducklake'`, so a multi-output
producer showed the same column mapping on every ducklake write-edge. Use the
same materialize-target anchor as buildColumnGraph: the badge lands only on the
declared output's edge, falling back to the ducklake write-edge when there's no
materialize annotation. (Codex P1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: build column trace from displayGraph so View hides draft lineage
The transitive column trace was built from graphWithDraft regardless of mode, so
in View with drafts hidden it could surface draft `// column` lineage the
deployed canvas doesn't show. Build it from `displayGraph` (the graph the canvas
actually renders) so the trace matches: draft overlays in edit / show-drafts,
deployed-only in plain View. (Codex P2)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: don't infer column lineage for local/temp staging CTAS
A CTAS into a local/temp staging table isn't the materialized output, but its
projection was inferred and (flat) column_lineage anchored to the script's
// materialize target — so staging columns showed up as the final asset's. Gate
inference to the actual output: a top-level managed-materialize SELECT, or a
CTAS/CREATE VIEW whose target resolves to a real asset. (Codex P1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: scope inferred column lineage to one output asset
Inference accumulated columns from every output-producing query into one flat
list, all anchored (frontend) to the script's // materialize target — so an
auxiliary CTAS into a different asset showed its columns on the materialized
one. Tag each inferred entry with its output asset and, in parse_assets, scope
the list to the // materialize target (keeping untagged top-level-SELECT
entries); with no declared target, drop inference when entries span multiple
output assets rather than attribute them to an arbitrary one. Parser-internal —
no wire change. (Codex P1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: treat CREATE TEMP TABLE/VIEW as local even under an active USE
A one-part temp name under `USE dl` resolved to an asset (ducklake://…/tmp)
before being registered local, so a final SELECT reading it invented
`final.total <- warehouse/tmp.amt` (a phantom DuckLake column) and recorded a
phantom asset. track_table_definition now registers any temporary table/view as
local up front, bypassing active-asset resolution; CreateTable/CreateView pass
their `temporary` flag. (Codex P1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat: render mermaid diagrams in chat code blocks
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: guard mermaid render against out-of-order async and transient streaming failures
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: only show mermaid diagram while it matches current source
Addresses Codex review: keeping the last good SVG through parse failures left a stale, mismatched diagram on screen when the source changed to something invalid. Tie the rendered SVG to the source that produced it and only display it while it still matches the current code, falling back to the raw source otherwise.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ruben Fiszel
Diego Imbert
hugocasa