mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-27 16:02:43 +00:00
af019f26faf4a84a94b5cd91c76a9bd24fcf918b
8542
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Diego ImbertandClaude Opus 4.8
|
af019f26fa |
feat(hub-projects): generate and apply datatable migrations on project publish/install (#9977)
* feat: add datatable_migrations table
* feat: add route to run datatable migrations
* feat: sync datatable migrations as .up.sql/.down.sql files
* feat: add datatable migrate up/down commands and post-push run prompt
* feat: add datatable migrate new command to scaffold migrations
* feat: add datatable migrations management UI
* feat: prompt to create migration on DDL in datatable SQL editors
* feat: support running a single specific datatable migration
* feat: view migration content, run single migration, fix stacked modal
* feat: per-row revert button with out-of-order warning
* fix: avoid migrations list flicker on refresh after an action
* feat: generate initial datatable migration via pg_dump
* fix: surface datatable migration API error details in toasts
* fix: revert created migration if create-and-run fails to run
* fix: include postgres error detail in migration run/rollback failures
* feat: sync datatable migrations as files via the workspace export
* refactor: move datatable migrations to migrations/datatable/ path
* fix: drop redundant datatable_migration label in sync output
* fix: exclude datatable migration sql files from script metadata generation
* feat: run datatable migrations as user-permissioned labeled jobs
* feat: reject invalid datatable migrations on sync push
* feat: datatable migrate up/down default to all datatables, --datatable to target one
* fix: surface postgres error detail when datatable migrations fail to run
* chore: regenerate CLI docs for datatable migrate commands
* feat: default new datatable migration to a BEGIN/END transaction template
* fix: validate datatable migration name and datatable at the API boundary
* fix: ensure detected DDL ends with semicolon when wrapped in transaction
* fix: re-prompt instead of stripping DDL when new-migration modal is cancelled
* feat: refresh datatable schema after running a migration from the SQL REPL
* feat: record db manager DDL on data tables as migrations
* feat: make datatable migrations opt-in per data table
* fix: make migration view editor read-only so its code can scroll
* fix: don't re-prompt DDL guard when creating a migration without running
* feat: generate down migrations for db manager DDL (postgres)
* fix: correct down migration for db manager alters (no double-wrap, serial)
* feat: explain migrations purpose with a tooltip in the migrations modal
* compare paeg
* feat: add datatable_migration kind to workspace diff pipeline
* chore: point ee-repo-ref at datatable_migration git-sync companion
* fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests
* feat: deploy and run datatable migrations on workspace merge
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Refactor + handle datatable setting delete/rename
* refactor: move datatable migration rename/delete cascade into module
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(db-manager): add Migrations button to top bar, make Refresh icon-only
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* BEGIN/END placeholder in down migration
* feat: autofocus migration name input and flag it red when empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* border nits
* refresh db manager schema on migrations
* BEGIN/END scaffold in CLI
* feat(cli): push local datatable migrations before running on migrate up
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: flag invalid migration name with red border, not just empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop random slug from auto-generated migration names
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: offer revert-and-delete when deleting an installed migration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: record fork merge as a migration when target datatable opts in
* nit
* clone migrations on fork
* windmill-utils-internal
* fix(datatable-migrations): serialize run/rollback with a per-db advisory lock
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db-manager): fail closed when migrations-status check errors on DDL apply
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: fix generate_initial migration ordering comment to match code
* chore(datatable-migrations): remove unused update_datatable_migrations endpoint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: run DDL migration guard on the script editor Test button
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* split
* ee-repo-ref
* chore(frontend): sync package-lock with package.json (@emnapi deps)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(datatable-migrations): never resolve instance credentials into migration job args
datatable_database_arg eagerly resolved instance data-table credentials
(including the shared instance-wide Postgres password) and passed them as the
migration job's plaintext `database` arg, landing in v2_job.args. Since the
run route has no admin gate, a non-admin could run a migration and read
args.database to recover the password, granting cross-workspace psql access to
all instance data-table DBs.
Pass a `datatable://<name>` reference for both resource-backed and instance
data tables instead; the pg executor already resolves it to real credentials
server-side at run time, so nothing sensitive is ever stored in the job args.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* fix: handle dollar-quoting and comments when splitting SQL statements
* feat: deploy datatable migrations on merge with explicit opt-in error
* fix(frontend): sync package-lock with npm 11 peer-dep resolution
npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from
lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as
peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly
1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the
highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree
needs both versions; the committed lock only had 1.10.0.
Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for
the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit npm publish
* fix: fail closed on migrations-status error in fork schema merge
* nit CI emnapi/core version
* prevent initial_datatable_migration if migrations already exist
* fix(datatable-migrations): validate persisted data table names as path segments
edit_datatable_config only validated rename segments, not the actual
settings.datatables keys, so a data table could be saved directly under a name
like '..' or one containing '/'. Since new tables default to
migrations_enabled = true, generate_initial_datatable_migration would then
insert a migration row and the sync export would build
migrations/datatable/<name>/... paths from that name, producing malformed or
directory-escaping export paths.
Validate every persisted data table name in edit_datatable_config (alongside
the existing rename checks) and add validate_datatable_path_segment to
generate_initial_datatable_migration for defense in depth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: scope datatable _wm_migrations by data table and cascade renames/deletes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(system_prompts): resolve nested local command groups in CLI docs generator
The CLI docs generator anchored on the first `new Command()` in a file and
never resolved locally-defined command groups passed as
`.command("name", localCmd)`. For datatable this flattened the nested
`migrate` group: it emitted `datatable new/up/down` plus a bare
`datatable migrate`, and mislabeled the datatable command with the migrate
group's description. jobs was broken the same way (its description was pull's,
and pull/push rendered empty).
Anchor block extraction on the `export default`ed command, recurse into
locally-defined `const x = new Command()` groups mounted as subcommands, and
render nested sub-subcommands. Regenerated docs now show
`datatable migrate new/up/down` and `jobs pull/push` with their real
options.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop unreleased _wm_migrations legacy-upgrade handling
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: return datatable migration SQL from getItemValue for the diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* nit
* fix: handle datatable migration renames on push and dedupe timestamps
* fix: reject rewriting an already-applied datatable migration on upsert
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries
Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi
declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved
lockfile entries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): datatable migrate up/down default to main datatable, not all
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: fail closed when applied status unreadable on datatable migration rewrite
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: surface full error detail in Database Manager DDL/query errors
* "See migration" button in the toast
* feat: add Enter shortcut to Create-a-migration in the DDL guard
* fix(frontend): warn before running a newly-created datatable migration out of order
The row-level Run action warns when earlier migrations are still pending, but
the create-and-run paths ran a just-created migration with `only` directly,
applying it ahead of older pending migrations without that confirmation.
Reuse the same "Run migration out of order" confirmation across all
create-and-run paths via a shared helper (datatableMigrationUtils):
- NewDataTableMigrationModal "Create and run" (and the DDL guard path)
- DatatableSchemaDiff fork→parent merge
- dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a
MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a
silent cancel
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: keep renamed datatable migrations visible in compare view
* fix: record per-migration deployment on datatable migrations disable
* fix(cli): run deployed datatable migrations after workspace merge
The merge command upserted datatable_migration definitions into the target
workspace and reported the item as successfully deployed, but never ran the
migrations. For forked datatables backed by separate databases, this left the
target schema unchanged until someone manually ran `wmill datatable migrate up`,
while the CLI reported a successful merge.
Collect the datatable migrations deployed (not deleted) into the target and,
after the deploy loop, offer to run them via the existing offerToRunNewMigrations
helper — the same post-deploy run prompt the push/sync path uses (interactive
only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export
parseDatatableMigrationDeployPath so the merge path can parse the deployed items.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(backend): serialize datatable migration edits/deletes with the run lock
A migration run snapshots a migration's code_up from datatable_migrations and
only records its version in the data table's _wm_migrations after the job
succeeds. upsert_datatable_migration checked _wm_migrations before allowing an
edit but took no lock, so a concurrent edit could read "not applied yet",
rewrite code_up/code_down, and then the in-flight run would record the version
for the old SQL — leaving _wm_migrations pointing at SQL that was never applied
(migrate up then skips it; rollback runs a down that doesn't match).
Serialize definition rewrites and deletes with the same per-database advisory
lock the run/rollback paths use:
- Factor the connect+advisory-lock into lock_datatable_migration_runs and the
applied-versions read into read_applied_versions_on_client.
- run_datatable_migrations now snapshots the definitions AFTER taking the lock,
so code_up can't change between snapshot and version-record.
- upsert (when changing an existing def) and delete take the lock across the
applied-check and the write; delete now rejects deleting an already-applied
migration (would orphan its _wm_migrations record), symmetric with upsert.
Both fail closed if the data table database is unreachable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): stack the out-of-order migration confirm above the DB editor preview
Creating a table on a migrations-enabled data table opened the DB table editor's
"Confirm running the following" preview modal, whose confirm triggers applyDdl,
which then asks for out-of-order confirmation. Both are ConfirmationModals with a
hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before
the editor), so it rendered behind the still-open preview modal.
Add an optional zIndexClass prop to ConfirmationModal (default z-[9999],
backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it
stacks on top.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(hub-projects): generate and apply datatable migrations for projects
Detect datatable assets in a project's scripts/flows/raw apps when
publishing to the Hub, generate a best-effort CREATE TABLE migration per
data table from the source workspace's live schema, and let the publisher
edit/toggle them in the bundle drawer. On import, offer to run the shipped
migrations: recorded (datatable_migrations + _wm_migrations) when the
target data table opted into migrations, otherwise as a one-off preview
job. Missing target data tables are surfaced and skipped.
- backend: POST /hub/migrations proxy forwarding to the Hub
- frontend publish: projectMigrations.ts detection + generation, new
"Data table migrations" section in DeployToHub
- frontend import: run/skip modal + missing-datatable confirmation
- extract pure SQL-gen from DatatableSchemaDiff.svelte into
datatableSchemaSql.ts so plain .ts modules can import it
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(hub-projects): close datatable migration table set over foreign keys
Pull a referenced table's FK targets into the generated migration
transitively, so it creates every table it references (ordered by FK
dependency), and drop any FK whose target still isn't in the set so the
generated SQL always runs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(hub-projects): show Data table dependencies in the publish view
Detect data table usage off the predeploy bundle preview and surface it as
a "Data table dependencies" summary right after "Resource dependencies",
mirroring how resource types and triggers are shown. The editable
migration itself stays in the bundle drawer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(hub-projects): explain un-generated migrations with SQL comments
When a table can't be found in the schema, a data table is referenced as a
whole, or the schema can't be loaded, write a `--` comment describing the
problem into the migration instead of leaving it blank. Partial migrations
keep the CREATE TABLEs that did generate and comment the rest; comment-only
migrations stay disabled. The bundle drawer now always shows the SQL box so
those comments are visible and editable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(hub-projects): review/edit migrations on import + rollback down migration
Replace the plain "run migrations?" confirmation with a review drawer that
previews each runnable migration, lets the user edit the SQL and toggle
which to run, before the import proceeds. When recording an imported
migration, also record a down migration (DROP TABLE of the created tables,
in reverse order) derived from the up SQL, so it can be rolled back; the
derived rollback is previewed in both the publish bundle drawer and the
import review drawer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* feat(hub-projects): editable Up/Down Monaco editor for migrations
Replace the plain textarea with a Monaco SQL editor split into Up/Down
tabs. The down migration is now generated once as best-effort (DROP TABLE
in reverse creation order) and is fully editable — no longer parsed back
out of the up SQL. The down is threaded through publish → Hub → import
(new project_migration.sql_down) and recorded as code_down when an imported
migration is applied.
- projectMigrations: GeneratedMigration.sql_down generated from the table set
- MigrationSqlEditor.svelte: shared Up/Down tabbed Monaco editor (re-keyed on
regeneration since Monaco ignores external code changes)
- DeployToHub + install review drawer use it; sql_down pushed/applied
- backend: PublishMigrationBody carries sql_down
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub-projects): generate CREATE TABLE IF NOT EXISTS for project migrations
The FK closure pulls a referenced table's parents into the same transaction
(e.g. `orders` drags in `customers`); those shared parents often already
exist in the target, so a plain CREATE TABLE aborted the whole migration on
the first collision. Emit CREATE TABLE IF NOT EXISTS for project migrations
(via a new opt-in flag on generateMigrationSql, leaving the schema-diff
behavior unchanged) so a pre-existing parent is skipped instead of failing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub-projects): key FK ordering by schema-qualified table name
orderByFkDependency keyed its dependency graph by bare table name (and
resolved FK targets with .split('.').pop()), so two same-named tables in
different schemas collapsed and one was dropped from the ordered set and
never created. Key by schema.table like the rest of the pipeline, resolving
FK targets through resolveTable. Also let resolveTable fall back to the bare
table name when a schema-qualified ref's schema doesn't match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub-projects): comment out generated down-migration DROP statements
The generated down migration listed DROP TABLE for every table in the FK
closure, including shared parent tables that may have pre-existed in the
target — a rollback could drop a table the project never created (data
loss). Emit all DROP statements commented out with a note, so nothing is
dropped by default; the publisher uncomments the tables this migration
actually owns.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub-projects): disable Import button during migration review
planMigrations awaits the review / missing-datatable modals before setting
installing = true, so the Import button stayed enabled during review and a
second click launched a concurrent install() (second review drawer,
duplicated item creation). Track a planningMigrations flag, disable the
button on it, and early-return install() if already installing or planning.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub-projects): toast when migration generation fails
regenerateMigrations cleared the drafts on error, showing "No data table
usage detected" — indistinguishable from a genuine schema-load failure. Add
a toast on the catch so the publisher can tell the two apart.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(hub-projects): honor cancel on the missing-data-table warning
planMigrations awaited missingDatatableModal.ask() but ignored its boolean,
so cancelling the "some data tables are missing" warning still proceeded
with the import — the cancel affordance did nothing. Show the warning first
and abort the whole import when the user cancels (planMigrations returns
null; install() early-returns), so they can create the data table(s) and
re-run. Confirming still imports without the missing migrations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(hub-projects): detect data tables from low-code app DB-table config
Low-code apps don't carry a persisted asset list, but the DB-table
component declares its data table and table explicitly: a `oneOf` `type`
config with `selected === 'datatable'` holding `datatable://<name>` and the
table. Walk the app value for those configs so an app that reads a data
table is picked up by the Data table dependencies detection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Revert "feat(hub-projects): detect data tables from low-code app DB-table config"
This reverts commit
|
||
 Diego Imbert
|
bca0b366fb | Merge remote-tracking branch 'origin/main' into tl/workspace-to-hub | ||
|
e47aedac0a |
feat: add SQL migrations for data tables (#9693)
* feat: add datatable_migrations table * feat: add route to run datatable migrations * feat: sync datatable migrations as .up.sql/.down.sql files * feat: add datatable migrate up/down commands and post-push run prompt * feat: add datatable migrate new command to scaffold migrations * feat: add datatable migrations management UI * feat: prompt to create migration on DDL in datatable SQL editors * feat: support running a single specific datatable migration * feat: view migration content, run single migration, fix stacked modal * feat: per-row revert button with out-of-order warning * fix: avoid migrations list flicker on refresh after an action * feat: generate initial datatable migration via pg_dump * fix: surface datatable migration API error details in toasts * fix: revert created migration if create-and-run fails to run * fix: include postgres error detail in migration run/rollback failures * feat: sync datatable migrations as files via the workspace export * refactor: move datatable migrations to migrations/datatable/ path * fix: drop redundant datatable_migration label in sync output * fix: exclude datatable migration sql files from script metadata generation * feat: run datatable migrations as user-permissioned labeled jobs * feat: reject invalid datatable migrations on sync push * feat: datatable migrate up/down default to all datatables, --datatable to target one * fix: surface postgres error detail when datatable migrations fail to run * chore: regenerate CLI docs for datatable migrate commands * feat: default new datatable migration to a BEGIN/END transaction template * fix: validate datatable migration name and datatable at the API boundary * fix: ensure detected DDL ends with semicolon when wrapped in transaction * fix: re-prompt instead of stripping DDL when new-migration modal is cancelled * feat: refresh datatable schema after running a migration from the SQL REPL * feat: record db manager DDL on data tables as migrations * feat: make datatable migrations opt-in per data table * fix: make migration view editor read-only so its code can scroll * fix: don't re-prompt DDL guard when creating a migration without running * feat: generate down migrations for db manager DDL (postgres) * fix: correct down migration for db manager alters (no double-wrap, serial) * feat: explain migrations purpose with a tooltip in the migrations modal * compare paeg * feat: add datatable_migration kind to workspace diff pipeline * chore: point ee-repo-ref at datatable_migration git-sync companion * fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests * feat: deploy and run datatable migrations on workspace merge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Refactor + handle datatable setting delete/rename * refactor: move datatable migration rename/delete cascade into module Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(db-manager): add Migrations button to top bar, make Refresh icon-only Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * BEGIN/END placeholder in down migration * feat: autofocus migration name input and flag it red when empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * border nits * refresh db manager schema on migrations * BEGIN/END scaffold in CLI * feat(cli): push local datatable migrations before running on migrate up Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: flag invalid migration name with red border, not just empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop random slug from auto-generated migration names Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: offer revert-and-delete when deleting an installed migration Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: record fork merge as a migration when target datatable opts in * nit * clone migrations on fork * windmill-utils-internal * fix(datatable-migrations): serialize run/rollback with a per-db advisory lock Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db-manager): fail closed when migrations-status check errors on DDL apply Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix generate_initial migration ordering comment to match code * chore(datatable-migrations): remove unused update_datatable_migrations endpoint Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: run DDL migration guard on the script editor Test button Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * split * ee-repo-ref * chore(frontend): sync package-lock with package.json (@emnapi deps) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(datatable-migrations): never resolve instance credentials into migration job args datatable_database_arg eagerly resolved instance data-table credentials (including the shared instance-wide Postgres password) and passed them as the migration job's plaintext `database` arg, landing in v2_job.args. Since the run route has no admin gate, a non-admin could run a migration and read args.database to recover the password, granting cross-workspace psql access to all instance data-table DBs. Pass a `datatable://<name>` reference for both resource-backed and instance data tables instead; the pg executor already resolves it to real credentials server-side at run time, so nothing sensitive is ever stored in the job args. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit * fix: handle dollar-quoting and comments when splitting SQL statements * feat: deploy datatable migrations on merge with explicit opt-in error * fix(frontend): sync package-lock with npm 11 peer-dep resolution npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly 1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree needs both versions; the committed lock only had 1.10.0. Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit npm publish * fix: fail closed on migrations-status error in fork schema merge * nit CI emnapi/core version * prevent initial_datatable_migration if migrations already exist * fix(datatable-migrations): validate persisted data table names as path segments edit_datatable_config only validated rename segments, not the actual settings.datatables keys, so a data table could be saved directly under a name like '..' or one containing '/'. Since new tables default to migrations_enabled = true, generate_initial_datatable_migration would then insert a migration row and the sync export would build migrations/datatable/<name>/... paths from that name, producing malformed or directory-escaping export paths. Validate every persisted data table name in edit_datatable_config (alongside the existing rename checks) and add validate_datatable_path_segment to generate_initial_datatable_migration for defense in depth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope datatable _wm_migrations by data table and cascade renames/deletes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(system_prompts): resolve nested local command groups in CLI docs generator The CLI docs generator anchored on the first `new Command()` in a file and never resolved locally-defined command groups passed as `.command("name", localCmd)`. For datatable this flattened the nested `migrate` group: it emitted `datatable new/up/down` plus a bare `datatable migrate`, and mislabeled the datatable command with the migrate group's description. jobs was broken the same way (its description was pull's, and pull/push rendered empty). Anchor block extraction on the `export default`ed command, recurse into locally-defined `const x = new Command()` groups mounted as subcommands, and render nested sub-subcommands. Regenerated docs now show `datatable migrate new/up/down` and `jobs pull/push` with their real options. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop unreleased _wm_migrations legacy-upgrade handling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: return datatable migration SQL from getItemValue for the diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit * nit * fix: handle datatable migration renames on push and dedupe timestamps * fix: reject rewriting an already-applied datatable migration on upsert Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved lockfile entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): datatable migrate up/down default to main datatable, not all Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: fail closed when applied status unreadable on datatable migration rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: surface full error detail in Database Manager DDL/query errors * "See migration" button in the toast * feat: add Enter shortcut to Create-a-migration in the DDL guard * fix(frontend): warn before running a newly-created datatable migration out of order The row-level Run action warns when earlier migrations are still pending, but the create-and-run paths ran a just-created migration with `only` directly, applying it ahead of older pending migrations without that confirmation. Reuse the same "Run migration out of order" confirmation across all create-and-run paths via a shared helper (datatableMigrationUtils): - NewDataTableMigrationModal "Create and run" (and the DDL guard path) - DatatableSchemaDiff fork→parent merge - dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a silent cancel Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep renamed datatable migrations visible in compare view * fix: record per-migration deployment on datatable migrations disable * fix(cli): run deployed datatable migrations after workspace merge The merge command upserted datatable_migration definitions into the target workspace and reported the item as successfully deployed, but never ran the migrations. For forked datatables backed by separate databases, this left the target schema unchanged until someone manually ran `wmill datatable migrate up`, while the CLI reported a successful merge. Collect the datatable migrations deployed (not deleted) into the target and, after the deploy loop, offer to run them via the existing offerToRunNewMigrations helper — the same post-deploy run prompt the push/sync path uses (interactive only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export parseDatatableMigrationDeployPath so the merge path can parse the deployed items. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): serialize datatable migration edits/deletes with the run lock A migration run snapshots a migration's code_up from datatable_migrations and only records its version in the data table's _wm_migrations after the job succeeds. upsert_datatable_migration checked _wm_migrations before allowing an edit but took no lock, so a concurrent edit could read "not applied yet", rewrite code_up/code_down, and then the in-flight run would record the version for the old SQL — leaving _wm_migrations pointing at SQL that was never applied (migrate up then skips it; rollback runs a down that doesn't match). Serialize definition rewrites and deletes with the same per-database advisory lock the run/rollback paths use: - Factor the connect+advisory-lock into lock_datatable_migration_runs and the applied-versions read into read_applied_versions_on_client. - run_datatable_migrations now snapshots the definitions AFTER taking the lock, so code_up can't change between snapshot and version-record. - upsert (when changing an existing def) and delete take the lock across the applied-check and the write; delete now rejects deleting an already-applied migration (would orphan its _wm_migrations record), symmetric with upsert. Both fail closed if the data table database is unreachable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): stack the out-of-order migration confirm above the DB editor preview Creating a table on a migrations-enabled data table opened the DB table editor's "Confirm running the following" preview modal, whose confirm triggers applyDdl, which then asks for out-of-order confirmation. Both are ConfirmationModals with a hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before the editor), so it rendered behind the still-open preview modal. Add an optional zIndexClass prop to ConfirmationModal (default z-[9999], backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it stacks on top. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private. Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4 New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
 GuilhemandClaude Opus 4.8
|
8df613b4d2 |
feat(raw-apps): runtime-error overlay + AI import-React instruction (#9966)
* feat(raw-apps): render runtime-error overlay + instruct AI to import React Render the `runtimeError` message the raw-app preview frame now posts as a prominent overlay, so an uncaught exception that blanks the app is visible instead of silent. Cleared on the next successful build (via a shared `feedPreviewIframe` helper so every preview-feed path resets it). Add an AI app-generation instruction to begin React files with `import React from 'react'`: raw apps bundle with the classic JSX transform, so a missing import compiles fine but throws "React is not defined" at runtime. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(raw-apps): add the import-React rule to the shared raw-app prompt The global AI chat and the raw-app CLI skill draw their raw-app authoring reference from system_prompts/base/raw-app.md — a separate surface from the app chat's inline prompt (core.ts). Add the same "always begin JSX files with `import React`" rule there (esbuild's classic transform needs React in scope, or JSX throws "React is not defined" at runtime) and regenerate the derived prompt files. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(raw-apps): bump ui_builder tarball to f8cecf9 (runtime-error overlay) Pins the ui_builder artifact to windmill-code-ui-builder#15, which pushes uncaught runtime errors from the preview iframe to the parent so the raw-app editor can render them in the error overlay. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
 Ruben FiszelandClaude Opus 4.8
|
51e1eba1cc |
icon-only muted-read badge + lighter DuckDB template (#9972)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Ruben Fiszelandrubenfiszel
|
8dd5e48a68 |
chore(main): release 1.751.0 (#9965)
* chore(main): release 1.751.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Guilhem
|
f3da86512a |
theme-aware prose palette for markdown in dark mode (#9971)
* fix(frontend): theme-aware prose palette for markdown in dark mode * chore(frontend): add markdown example to kitchen_sink showcase |
||
|
GuilhemandClaude Fable 5
|
2e14302e4a |
feat(frontend): custom skills — detail modal, batch manage, shared validation (#9847)
* feat(frontend): simplify custom skills workspace settings UI Collapse the "Custom skills" AI settings section into a single block. When no skills exist, show two side-by-side zones: a drag-and-drop folder dropzone (reusing FileInput) and a paste textarea whose add button appears only once content is entered. When skills exist, an "+ Add skills" dropdown offers "Import a folder of skills" (native picker) and "Paste a skill" (modal), above the skills list. Folder ingestion is shared by both the picker and the dropzone via processFolderFiles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): custom skills detail modal + shared zod validation Rework the Custom skills settings: header Add-skills dropdown, per-row ellipsis menu (edit/delete), a Show more detail modal with a view/edit toggle (rendered markdown in read mode), accent Save gated by dirty detection and inline validation, and a folder-import conflict modal with per-skill overwrite toggles. Extract skill parsing/validation into a shared Zod-backed aiSkills module used by both the modal and the importer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): use Button for Show more; surface empty-body validation Address review: swap the raw <button> Show-more affordance for the design-system Button (per frontend component standards), and render the Save/inline-error block whenever editing an existing skill so clearing the body surfaces "body is required" instead of hiding both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(frontend): unit-test aiSkills; use themed border token Add aiSkills.test.ts covering parseSkillMd (BOM, CRLF, malformed YAML), validateSkill (code-point vs byte limits, name pattern), parseAndValidateSkill (nameOverride precedence) and buildSkillMd round-trip. Replace the hardcoded gray borders with the themed border-border-light token. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): cap custom skills list height and scroll Constrain the skills list to max-h-96 with overflow-y-auto so a large number of skills scrolls within the section instead of pushing the page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): manage-mode batch delete for custom skills Add a "Manage skills" button (shown only with more than one skill) that enters a multi-select mode: a checkbox per row plus a sticky select-all (tri-state) header, and a batch Delete gated on the selection with a confirmation. Manage mode auto-exits when the list drops to one skill. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): destructive delete, subtle manage button, Esc exits manage mode Batch Delete uses the destructive accent variant, Manage skills uses the subtle variant, and Escape leaves manage mode (mirroring Done) unless a modal or menu is open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): add neutral 'info' type to ConfirmationModal; use for skill import ConfirmationModal only had 'danger' and 'reload' semantics, so a constructive confirmation like importing skills defaulted to danger (red warning + destructive button). Add a neutral 'info' type (blue Info icon, non-destructive accent confirm) and use it for the Import skills modal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 hugocasaandClaude Opus 4.8
|
fd8e64d11f |
feat: add cosmetic dev/staging label for dev workspaces (#9959)
* feat: add cosmetic dev/staging label for dev workspaces Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: prefill dev fork name and use a link to switch its label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: reword the dev/staging label link copy Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: preview the dev/staging label as a badge in the switch link Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: show the dev/staging badge in the session diff drawer header Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 4.8
|
6587019d26 |
fix: critical alerts modal mute toggles no longer close popover or fail to save (#9969)
* fix: mute toggles in critical alerts modal no longer close popover or fail to save Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: mark popover content root as dropdown-portal so padding clicks don't close modal Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: derive no-channels warning from mute state so it survives modal reopen Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 4.8
|
45946d1185 |
fix(assets): responsive layout for small screens (#9961)
* fix(assets): handle card/header overflow on small screens Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(assets): keep filter row label on one line with min spacing from refresh Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(assets): wrap card header actions below title instead of collapsing docs to icon Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(assets): widen card basis to 340px so cards wrap sooner and header stays one row Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Guilhem
|
9821596251 | fix(frontend): theme-aware code block background in prose markdown (#9968) | ||
|
Ruben FiszelandClaude Opus 4.8
|
3dcd3949a1 |
feat(pipelines): auto-derive cascade edges from ducklake/s3 reads (+ muted-read badge) (#9963)
* feat(pipelines): auto-derive cascade trigger edges from ducklake/s3 reads Within a `// pipeline`, a read of a ducklake table or s3 object now auto-wires its cascade trigger edge straight from the FROM clause, so `// on <asset>` is only needed for edges inference can't see (dynamic SQL) or to carry per-edge opts. Two opt-outs: `// mute <asset>` suppresses a single derived edge (a lookup / SCD input read every run but not cascaded on), and `// mute all` opts the script out of derivation entirely (back to explicit-`// on`-only). Explicit `// on` still wins the dedup. Scoped to ducklake + s3 reads; resource/datatable/volume stay explicit. Read-write (RW) and write inputs are excluded so a self-referential merge can't loop-trigger itself; ambiguous (None) access is skipped. - parser: `mute` / `mute_all` in PipelineAnnotations (Rust + TS mirror) - deploy: derive_pipeline_asset_trigger_refs → script_trigger rows - frontend: resolveGraph mirrors derivation for the live edit-mode canvas - tests: shared parity corpus + derive-helper units + resolveGraph overlays Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipelines): mark auto-derived cascade edges with a persisted derived flag + "auto" badge Persist script_trigger.derived (deploy: true for ducklake/s3-read derivation, false for explicit // on) and return it from the asset-graph endpoint so the canvas renders a Sparkles "auto" badge on auto-wired edges — the inference is now visible on both the deployed graph and the live edit canvas, not just implied. Dispatch (fetch_subscribers) ignores the flag, so a derived edge fires identically to an explicit // on. Also copy derived in the workspace-clone trigger copy, and backfill muteAssets/muteAll into two empty PipelineAnnotations literals the base commit left stale (check:fast). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): derive cascade edge from effective (alt-fallback) asset access derive_pipeline_asset_trigger_refs gated on the raw parser access_type, but the persisted asset.usage_access_type and the frontend canvas both use access_type.or(alt_access_type). An ambiguous parse with a manual read override was persisted/drawn as a read yet derived no edge, so the auto edge silently vanished on deploy. Gate on the effective access type for parity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipelines): badge muted reads instead of auto-derived edges Auto-derivation is the default now, so badging every derived cascade edge is noise. Drop the "auto" badge and the persisted `script_trigger.derived` flag (migration + insert param + graph field + clone copy) that only powered it, and instead badge the exception: a ducklake/s3 asset a script reads but does NOT cascade — `// mute <asset>` / `// mute all`. `computeMutedReadKeys` marks a read-only ('r') supported read with no cascade trigger and no self-write; the canvas renders a bell-off "muted" badge on that read edge. Also fixes two review parity nits: - TS `// on` parser now strips trailing `key=value` opts (e.g. `debounce=60s`) like the Rust `split_trailing_kv_opts`, so the ref dedups against inference. - A `// materialize` producer reading its own target is upgraded to `rw` (deploy) / excluded via the materialize write refs (canvas), so it neither self-cascades nor shows as a muted read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): drop redundant // on for auto-derived reads; gate muted badge to pipeline scripts - Templates no longer scaffold `// on <asset>` for a ducklake/s3 input the body reads — the read auto-wires the cascade now that derivation is the default. Kept for datatable/resource (not auto-derived) and native triggers. The discoverability hint now mentions `// mute` (the newly relevant annotation). - computeMutedReadKeys only badges reads by `// pipeline` scripts. A plain script or flow reading a ducklake/s3 asset never had an auto trigger to suppress, so it must render as ordinary lineage, not "muted" (Codex review). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): only drop template // on when the body actually reads the input The redundant-`// on` removal assumed the generated body reads the ducklake/s3 input, but postgres/bash/generic bodies (and `data_upload`, which reads the picker file) ignore `input` — dropping `// on` there left the asset-created script with no cascade at all. Gate the drop on READS_INPUT_LANGS (bun/deno/python/duckdb) so non-reading templates keep the explicit trigger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasaandClaude Opus 4.8
|
cc2f638de6 |
fix(ai): centralize Anthropic Messages API routing across completion paths (#9960)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
2217c23dd3 | Merge remote-tracking branch 'origin/main' into tl/workspace-to-hub | ||
|
Ruben Fiszelandrubenfiszel
|
5fe7e1f3e8 |
chore(main): release 1.750.0 (#9952)
* chore(main): release 1.750.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
|
056ebdb035 |
fix: read chat drafts via own-draft route so drawer-kind drafts deploy (#9913)
* fix: read chat drafts via own-draft route so drawer-kind drafts deploy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover trigger and resource chat-draft read/deploy regressions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover non-secret variable chat-draft read/deploy regression Completes the drawer-kind matrix from the review notes on #9913: schedule, trigger, and resource already had full write→read→deploy regressions; this adds the variable one (non-secret — the secret flow deploys through the ephemeral in-memory value and is pinned by the existing ephemeral tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai_evals): mock getOwnDraft so eval draft hydration stays in-memory The frontend eval adapter intercepts DraftService for benchmark workspaces, but only updateDraft/getDraftForUser/listDrafts. Global eval output collection hydrates draft values through getGlobalDraft, which reads via getOwnDraft — so draft-producing global cases fell through to the real generated client instead of the in-memory benchmark store. Adds a getBenchmarkOwnDraft helper (null on miss, mirroring the 200/null route semantics), wires it into the adapter mock, and pins it in mockBackendDrafts.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
hugocasaandClaude Opus 4.8
|
ea19cc9dc4 |
fix(ai): test key routes Azure Foundry Claude models via Anthropic Messages API (#9956)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
43044c2e28 |
feat(pipelines): wm_partition macro for grain-agnostic partition filters (#9950)
* wip: partial work before earlyoom-recovery relaunch
* fix(pipelines): scaffold the strftime {partition} filter idiom (frontend-only)
The DuckDB materialize scaffold and the AI pipeline prompt now teach the
grain-agnostic `WHERE strftime(<ts_col>, '<fmt>') = {partition}` filter instead
of the naive `= TIMESTAMP {partition}` cast. `{partition}` substitutes to the
partition IDENTITY string (`2026-07-05T23`, `2026-W27`, `2026-07`), which is not
a valid DuckDB TIMESTAMP literal for any non-daily grain — so the naive form
raises a `Conversion Error` for hourly/weekly/monthly (only daily parses).
Adds a frontend unit test asserting the hourly scaffold emits the strftime
idiom (`%Y-%m-%dT%H`) for every grain and never scaffolds the naive TIMESTAMP
cast as executable SQL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): scope strftime partition idiom to time grains
Review nit: `dynamic` partitioning's identity is a caller-supplied key, not a
timestamp, so `strftime` doesn't apply. Scope the scaffold + AI prompt claim to
time grains and add a `dynamic` example that filters on the user's own key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): wm_partition macro for grain-agnostic partition filters
The materialize runtime now injects a `wm_partition(ts)` temp macro as the first
setup statement of a time-partitioned script, so filtering the source to the
active slice is one grain-agnostic line — `WHERE wm_partition(<ts_col>) =
{partition}` — instead of a hand-written `strftime` format the author must keep
in lockstep with the resolver, or the `= TIMESTAMP {partition}` cast that only
parses for daily and Conversion-Errors for hourly/weekly/monthly.
The macro's format comes from `PartitionKind::default_time_format` in
windmill-parser, the same source the EE resolver reads to stamp the `{partition}`
identity, so the two can't drift. `dynamic` partitions get no macro (their
identity is a caller-supplied key → `WHERE <key_col> = {partition}`).
Replaces the earlier 9-line strftime comment block in the scaffold with the
single macro line; AI pipeline prompt and design doc updated to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(pipelines): verify wm_partition strftime parity vs chrono through real DuckDB
Runs the bundled DuckDB engine in-memory and asserts strftime renders every
grain format (daily/hourly/weekly `%G-W%V`/monthly) byte-for-byte identically to
chrono — the engine the resolver uses to stamp the `{partition}` identity —
across ISO-week year boundaries (2027-01-01 → 2026-W53 etc.). Also proves the
injected `wm_partition` macro buckets the whole slice and that the naive
`TIMESTAMP '<weekly|monthly identity>'` cast Conversion-Errors.
Closes the one cross-engine assumption the pure-Rust/frontend tests couldn't
reach (flagged by CI review for weekly ISO-week rendering).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 0de2412ff0734b11e12ba378c9bcc373ff9ae800
This commit updates the EE repository reference after PR #649 was merged in windmill-ee-private.
Previous ee-repo-ref: ad6c6685689d7741058e7d2c9ecbe95d982e6268
New ee-repo-ref: 0de2412ff0734b11e12ba378c9bcc373ff9ae800
Automated by sync-ee-ref workflow.
* fix(pipelines): classify CREATE TEMP MACRO as a DuckDB prepare-path setup statement
The FFI prepare/diagnostics pass only EXECUTES statements recognized by
is_setup_statement (ATTACH/USE/INSTALL/…); everything else is merely prepared.
`CREATE [OR REPLACE] TEMP MACRO` wasn't recognized, so on a `-- prepare` run of a
partitioned materialize the injected `wm_partition` macro was never created on
the connection, and the later generated `CREATE TABLE … SELECT … WHERE
wm_partition(...)` failed to bind ("function does not exist"). The same latent
gap affected the workspace-macro splicer, which injects TEMP MACRO blocks too.
Classify CREATE [OR REPLACE] TEMP|TEMPORARY MACRO as setup so it's executed
before dependent blocks and excluded from the PrepareQueryResult count
(persistent CREATE MACRO stays a user statement). Adds a prepare-path test that
fails without the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Ruben FiszelandClaude Opus 4.8
|
a1c5b7aa3e |
feat(pipelines): require data uploads before running a pipeline (#9953)
* feat(pipelines): require data uploads before running a pipeline Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): require every S3Object filled for data-upload readiness Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipelines): Run pipeline defaults schedule-triggered scripts to their schedule args Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: correct why only schedules default their args in Run pipeline Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): close cascade double-start race and gate data-upload readiness on full-schema validity Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: expose ScriptEditor validity via callback, not banned bindable-with-default Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
4b1f2207b8 |
ci: replace expiring-PAT org membership gate with author_association (#9957)
* ci for broken links + fix broken links * ci: replace expiring-PAT org membership gate with author_association The shared check-org-membership.yml reusable workflow authenticated to the GitHub API with the ORG_ACCESS_TOKEN PAT to confirm org membership. That PAT expired ~1 year after issuance, so the API could no longer see private org members and check-membership emitted is_member=false — silently skipping every auto-review, command-triggered review, /ai, /plan, and git-command job while still reporting success. Gate on the event payload's author_association (OWNER/MEMBER/COLLABORATOR) instead, which comes from the built-in GITHUB_TOKEN and never expires. The trusted internal bot and existing draft/fork/command guards are preserved; the workflow_call paths stay open as trusted upstream. Deletes the now-unused reusable workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 4.8
|
a6c0b3756b |
feat: chat-scoped session changes bar + unified diff drawer (#9762)
* feat(frontend): chat-scoped unified session changes bar Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(frontend): drop diff-baseline toggle, show natural per-row diffs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): carry Draft marker to expanded raw-app file rows Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): show raw-app Draft badge once at tree root, not per file Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(frontend): reuse shared DraftBadge in session diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): show draft-author avatars in session diff badge, icon-only in sidebar Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): keep badge pill around avatar in icon-only DraftBadge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): small draft marker = indigo pen + avatar; correct itemKind label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): drop package-lock churn from merge (match origin/main) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): dedup diff-button count for legacy fork sessions; test mask helper Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): collapsible session diff panel + per-row open-diff action Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): shared sessionDeployModel for review & deploy (S1) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): model-driven session review drawer, deploy inert (S2) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): wire session deploy + on-behalf/conflict gating (S3) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): behind banner + Update fork + deployment request (S4) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): session changes dock opens drawer by filter (S5) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): session review UI polish (badge, dock, In parent, tree width) - draft rows show only the avatar DraftBadge, not a duplicate state pill - drop redundant dock Review button (same as "N to review") - rename Done -> In parent with a "deployed in parent workspace" tooltip - widen the file tree Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): existence-gate In parent rows; Badge filters; badge hover - drop discarded mask-only items from the In-parent segment (existence check) - use the Badge component for the drawer filter segments and the changes dock - soften the blue Badge hover (blue-50 base was jumping to blue-200) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): polish session diff drawer (layout, badges, actions) - remove empty fork-banner gap; uniform sidebar tree padding - full-bleed diff list: drop card borders/side padding, separators between items - clamp tree x-overflow; right-align tree badges (min-w-0 on the row button) - brand-compliant selected filter badges; smaller draft badge - hide per-row open-diff button when the panel is open - rename "Delete draft" to "Discard draft" (destructive); remove header Review button - larger sm deploy/discard action buttons; remove per-item diff-content collapse Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): session Edits dock — deploy gating + change-op tracking Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): session bar per-status badges; drop change-op tracking Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): unwrap raw apps into per-file tree in session diff sidebar Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): dot-parcours pipeline (badge-derived, melt tooltip) + discard confirm Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): behind-only session item reads as deployed, not bare Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(frontend): even sidebar tree margins; gutter-aware right padding Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): pass chat id as from_session; wire deploying flag Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): session diff drawer scroll-to-flush, ordering, spacer, deploy gating Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): preserve chat mask on compact; guard stale existence checks; clear poll timers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): refresh bar after drawer deploys; conflict hint over chip; plain conflict badge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(frontend): diff drawer card layout with flash ring and aligned insets Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): hide stale deployed chip once row status badge reads deployed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(frontend): session dock to two states; drop parent deploy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): staged deploy animation in session edits drawer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): stale-draft warning in session edits drawer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): keep chat mask honest on deploy and discard Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): map trigger_email deploy kind; serialize mask persists Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): reset mask on new chat; close review-flagged races Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(frontend): rename session drawer title to Edited during session Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): keep mask persist queue alive after a failed save Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): sync session chatId on chat rotation; gate deploy on canWrite Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(frontend): keep compare handoff for deletion-only session edits Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): hold deploy success beat across re-keyed rows Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
ad6f23d6bf |
fix(cli): HD-1 test_edges + HD-2 scd2 _current write in --local pipeline graph (#9947)
* fix(cli): emit HD-1 test_edges + HD-2 scd2 _current write in --local pipeline graph Close the remaining local-vs-deployed graph parity gaps in `wmill pipeline show <folder> --local` so it matches the deployed graph (backend `asset_graph`, windmill-api-assets): - HD-1 `test_edges`: synthesize ordering-only producer → tested-script edges from parsed `// data_test` annotations. A `relationships` test references its `to_path` asset; a custom `// data_test <script>` resolves best-effort against that script's parsed reads. Each referenced asset is resolved to its in-pipeline producer via the write edges; self-edges and producer-less (external) assets are dropped — mirroring the backend set semantics. Routed through the asset node in boundedCascade's lineage DAG (asset → tested script) so a cold/bounded cascade orders the referenced dimension first, matching the frontend. - HD-2 scd2 `<dim>_current` companion write: a managed `// materialize … history` (scd2 && !manual) also produces a `<dim>_current` view. Register it as a second write edge and mark the asset `derived_from` its base dimension, so a consumer reading only the view links back to the producer instead of orphaning. Gated exactly like the backend `MaterializeSpec::write_targets` / `scd2_current_target`. The pinned `windmill-parser-wasm-asset` (1.740.0) predates the `scd2` materialize flag, so `buildLocalPipelineGraph` takes an injectable parser and the HD-2 test injects one that re-adds `scd2` for a `history` materialize — exercising the already-shipped companion-write branch until a wasm carrying `scd2` is republished (cf. #9926). Extends cli/test/pipeline_local_graph_unit.test.ts with HD-1 (relationships, no-producer, self-test, custom) and HD-2 coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): pin windmill-parser-wasm-asset 1.749.0, drop HD-2 test parser seam Now that windmill-parser-wasm-asset 1.749.0 (which serializes the `scd2` materialize flag) is published, bump the CLI pin and retire the temporary injection seam: - Remove the `infer?` parameter from `buildLocalPipelineGraph`; it always uses the wasm-backed `inferScriptAssets` again. - The HD-2 `<dim>_current` companion-write test drives the real wasm directly (drops the `inferWithScd2` wrapper that re-added `scd2` against the pinned 1.740.0 build). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): pin windmill-parser-wasm-asset 1.749.0 to match CLI Restore the CLI↔frontend lockstep on the asset parser wasm broken by the previous commit: every other windmill-parser-wasm-* package is pinned to the same version in both cli/package.json and frontend/package.json, so keep the asset parser aligned too. The frontend derives materialize/scd2 from its own TS annotation parser (`parsePipelineAnnotations`), so this bump only affects body asset inference in the live graph — moving it in step with the CLI `--local` graph and the deployed backend parser. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
6eabb96ae7 |
fix(pipelines): make node & pipeline-level run affordances always visible (#9948)
* fix(pipelines): make node & pipeline-level run affordances always visible Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): don't leave node Run disabled by stale form validity The always-visible node Run button read `isValid` directly, but that flag is only meaningful while PipelineRunForm is mounted to set it. On a same-path re-resolve (the component is keyed on script.path) from an input-carrying script to an input-less one, the form unmounts leaving `isValid=false`, which wrongly kept the empty-args Run button disabled. Gate validity through a `runValid` derived that is true whenever no form is rendered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): run with {} when node has no form, not stale args The always-visible node Run button dispatched $state.snapshot(args) unconditionally. `args` persists across a same-path re-resolve (component keyed on script.path), so a script that once had inputs/partition args and is re-resolved as input-less would run with the stale hidden args instead of {}. Send {} whenever no form is rendered, matching the no-form run intent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
f9840cd477 |
add missing amber-950 shade so dark pipeline callouts render (#9951)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben Fiszelandrubenfiszel
|
891b32195a |
chore(main): release 1.749.0 (#9938)
* chore(main): release 1.749.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
b13113964a |
fix(pipelines): canonicalize S3 asset keys so SDK writes and DuckDB reads connect (#9939)
* fix(pipelines): canonicalize S3 asset keys so SDK writes and DuckDB reads connect
The SDK object forms — TS `writeS3File({s3:"exports/x"})` and Python
`write_s3_file(S3Object(s3="exports/x"))` — resolve to the URI `s3:///exports/x`
(empty default storage), whose parsed asset path was `/exports/x` (leading
slash). DuckDB `read_csv('s3://exports/x')` and the `// on s3://exports/x`
trigger form yielded the bare `exports/x`. The same object thus produced two
asset identities, so a DuckDB consumer never connected to a TS/Python producer
in the pipeline graph.
`parse_asset_syntax` (shared by the native backend parsers and the wasm parser
that drives `frontend/src/lib/infer.ts` and the CLI `localGraph`) now strips a
single leading slash from S3 paths, so `s3:///key`, `s3://storage/key`, DuckDB
`s3://…`, and `// on` all canonicalize to one key. Both deploy-time inference
and editor/CLI inference agree, and the producer's write edge and the
consumer's read/trigger edge share a node.
Only one leading slash is stripped, so `s3:///` triple-slash default-storage
keys collapse to the bare key while Hive-partition keys
(`s3://bucket/y=2024/f.parquet`) and explicit-storage `s3://storage/key` paths
are untouched. Non-S3 asset kinds (res://, ducklake://, …) keep their paths
verbatim.
Note: existing deployed pipelines that recorded `/key` paths need a redeploy to
pick up the canonical `key`; the fix is forward-consistent for anything parsed
after this change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(pipelines): mark S3 asset-path normalization (item 6) resolved
The open-issues list still flagged the SDK-form leading-slash vs bare-URI
no-slash mismatch as "Still open", contradicting the fix in this PR. Mark it
resolved to match the updated Language-coverage prose.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs+test(pipelines): disclose S3 explicit-storage vs default-storage-nested-key aliasing
Collapsing to one canonical key means `s3://storage/key` (explicit storage) and
`s3:///storage/key` (default-storage nested key) now alias to the same node
`storage/key`, though they name different objects. Low-probability (needs a
storage config named to match a default-storage prefix) and inherent to a
best-effort lineage graph that doesn't split the first segment as a storage
name, but previously undisclosed. Document the tradeoff and pin the intended
aliasing with a test so it's intentional, not a latent surprise.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): mirror S3 leading-slash strip in frontend live-preview parser
The pipeline graph live preview parses `// on` annotations client-side via the
hand-written `parsePipelineAnnotations.ts` (a TS mirror of the Rust annotation
scanner), NOT the wasm parser. Its `parseAssetSyntax` still returned the raw
suffix, so `// on s3:///exports/x` yielded `/exports/x` while the deploy-time
and wasm parsers now canonicalize to `exports/x`. `resolveGraph` synthesizes
trigger edges from that path, so the browser preview could still render
disconnected `/exports/x` and `exports/x` nodes for the exact triple-slash case
this PR fixes at deploy time.
Mirror the S3-only single-leading-slash strip in the TS parser and extend the
shared parity fixture corpus (run by both the Rust and TS parity suites) with
the triple-slash trigger case, so Rust/TS drift on this is now caught.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): seed slashless S3 template asset paths to match canonical key
`autoOutputAsset` seeded new S3 template outputs with a leading slash
(`/pipelines/…`), which the old parser required to match `s3:///key` writes.
This PR made `parse_asset_syntax` strip that slash, so the seeded draft asset
(stored as `outputAssets`, used by `resolveGraph` for inactive-draft node
identity) no longer matched the body-inferred identity `pipelines/…` — the live
preview could render a duplicate `/pipelines/…` node and a phantom post-deploy
drift warning.
Seed the canonical slashless key instead, and switch the DuckDB body's S3 URIs
from `s3://${path}` to `s3:///${path}` so the generated runtime URI stays the
triple-slash default-storage form byte-for-byte (the SDK sites already build
`s3:///` + bare key). Add a pure-logic parity test asserting, for every
language and S3 output kind, that the seeded asset path is slashless and that
every S3 URI the generated body emits is triple-slash and canonicalizes back to
that seeded path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): canonicalize S3 keys in CLI + frontend bounded-cascade resolvers
Two more hand-written S3-URI sites returned the raw suffix, so `s3:///exports/x`
stayed `/exports/x` while native/wasm parsers now canonicalize to `exports/x`:
- `cli/src/commands/pipeline/localGraph.ts` — the no-wasm fallback `// on`
scanner (go/bash/ruby). A fallback consumer's `// on s3:///x` would not
connect to a wasm-inferred `x` producer in `wmill pipeline show/run --local`.
- `boundedCascade.ts` `assetUriToNodeId` (duplicated in the CLI and the frontend
AssetGraph engines, kept in sync) — `--to s3:///exports/x` / a cascade bound
token would not resolve against the canonical graph node `s3object:exports/x`.
`resolveToken` delegates here, so it is covered too.
Mirror the S3-only single-leading-slash strip in all three, and add `s3:///`
tests to the CLI local-graph fallback suite and both bounded-cascade suites
(explicit-storage and Hive-partition keys asserted untouched).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(pipelines): phrase S3 template test comment as a current invariant
Describe the slashless-seed requirement as the invariant it is, not as change
history, per the AGENTS.md "describe the code as it is" rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): strip all leading slashes from S3 keys so trigger refs round-trip
`parse_asset_syntax` stripped only one leading slash, so `S3Object(s3="/x")` —
which resolves to the quad-slash URI `s3:////x` — parsed to path `/x`. But
`trigger_spec_to_row` rebuilds a stored trigger ref as `s3://<path>` =
`s3:///x`, which `parse_asset_trigger_ref` then parses back to `x`. The
producer recorded `/x` while its consumer trigger resolved to `x` → a broken
edge. The same asymmetry affects every `s3://`+path reconstruction site
(backend refs, frontend `assetUri`, page refs) whenever a path starts with `/`.
Strip ALL leading slashes so a canonical S3 path never starts with `/`; naive
`prefix + path` reconstruction then round-trips everywhere. Applied uniformly
across all six S3-URI sites (Rust `parse_asset_syntax`, the TS live-preview
parser, template `s3Key`, and the frontend+CLI `assetUriToNodeId` and CLI
fallback scanner). The pathological leading-slash key collapses to the bare key
— acceptable for a best-effort lineage graph that never split storage anyway.
Tests: a windmill-common round-trip test (parse → trigger_spec_to_row →
parse_asset_trigger_ref) over every URI form incl. the quad-slash case; a
`s3:////x` shared parity fixture (Rust + TS); and quad-slash assertions in the
Rust parser test and both bounded-cascade suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(pipelines): align S3 template parity helper with strip-all canonicalization
The template seed/body parity test's `canonicalS3Key` helper (and its comment)
still stripped a single leading slash, so it no longer mirrored the parser it
claims to pin. Strip all leading slashes to match `parse_asset_syntax` and the
frontend/CLI mirrors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Ruben FiszelandClaude Opus 4.8
|
2d3a773441 |
feat(pipelines): mid-DAG selective execution (dbt model+) for pipeline runs (#9945)
* feat(pipelines): mid-DAG selective execution (dbt `model+`) for pipeline runs
Relax the root-only constraint on bounded-cascade starts so `--from` can name
any node in a pipeline — not just a schedule/manual root. A mid-DAG start runs
that node plus its transitive downstream and never re-runs upstream, giving
dbt's most common gesture (`dbt run --select model+`) a direct form:
wmill pipeline run f/orders --from fct_orders_daily
Previously this errored with "Starts must be schedule-triggered or manual
roots". The bounded-run engine already computed downstream/path-between sets
generically; only the eligibility gate was root-only.
- Shared engine (`boundedCascade.ts`, CLI + frontend mirror): add
`validFromStarts` — every autorun-able script (roots AND mid-DAG asset
subscribers / pure readers), excluding only event/input-only handlers
(kafka/mqtt/…/webhook/data_upload) that can't run with empty args.
- CLI: `--from` accepts any `validFromStarts` node; asset `--from` and
non-autorun handlers still rejected (the latter runnable via `--upload`). An
explicit mid-DAG start is protected from the barrier cut. Help text + regenerated
system_prompts describe the new surface.
- Frontend graph UI parity: any node with downstream now offers "Run + downstream…"
(was roots-only). With no end picked the bounded-run bar runs the full downstream
closure (`model+`); picking end(s) still bounds the path-between set.
- Unit tests for the new selection semantics in both engines.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): address CI review — scheduled-root --from regression + pick-mode barrier parity
Codex review findings on #9945:
- P1: explicit `--from` rejected a scheduled root that also carries a secondary
non-autorun trigger (e.g. `// on schedule` + `// on data_upload`), even though
it stays a valid IMPLICIT start. `validFromStarts` excluded anything in
`nonAutorunTriggerScripts`; now it unions in `validStarts` (which lets the
schedule identity win over the secondary trigger), so a scheduled root is
`--from`-eligible in both CLI and the graph UI. Regression tests added in both
engines.
- P2: bounded-pick mode built `eligible` (pickable end bounds) from raw
`descendants`, so an event handler — or a node only reachable through one —
could be clicked as an end yet be silently dropped from the barrier-cut run.
`eligible` is now the barrier-cut closure, so those nodes are dimmed and
non-pickable. The highlighted `bounded` ring now also reflects the actual
(barrier-cut) run set, including the no-ends "Run + downstream" case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): frontend barrier set must exclude all valid roots, not just the picked start
Codex review follow-up: the frontend `boundReachable` barrier set only protected
the picked start (`id !== boundPickStart`), while the CLI protects every valid
root (`!starts.has(id)`). So a scheduled root that also carries an event trigger,
reached downstream from another start, was wrongly treated as a barrier — the UI
dimmed/skipped it and its downstream, diverging from the CLI run set.
Exclude `validStarts` from the barrier set too (a scheduled/manual root runs on
its own identity even with a secondary event trigger). Regression test asserts a
scheduled-event root and its downstream stay reachable from an upstream start,
and that the naive (start-only) barrier set would have dropped them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): frontend must exclude webhook/data_upload as mid-DAG autorun starts
Codex review follow-up: the frontend `validFromStarts` only excluded
`EVENT_TRIGGER_KINDS`, so a mid-DAG `webhook`/`data_upload` subscriber was added
by the new eligibility loop — the UI would offer "Run + downstream" and launch it
with empty args (no uploaded S3Object / webhook body). The CLI mirror already
excludes these input-only kinds.
Add a frontend `NON_AUTORUN_TRIGGER_KINDS` (event kinds + webhook + data_upload),
mirroring the CLI, and use it in both `validFromStarts` (exclude such mid-DAG
handlers from starts) and `nonAutorunTriggerScripts` (cut them as barriers).
When the marker is visible (editor overlay / draft) these are now handled
exactly as the CLI does; the deployed-graph blind spot (no webhook/data_upload
rows) remains the documented pre-existing `validStarts` limitation.
Regression test: a `data_upload`/`webhook` mid-DAG subscriber is not an eligible
start and is barrier-cut (with its exclusive downstream) when running from an
upstream root.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Ruben FiszelandClaude Opus 4.8
|
475b072987 |
feat(pipelines): partition run-arg picker + first-run setup signpost (#9943)
* feat(pipelines): partition run-arg picker + first-run setup signpost Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): honor partition start= anchor in picker default and per-input upstream hint Addresses CI review (Codex): - P1: never seed/offer a pre-start bucket — the worker takes an explicit partition arg verbatim, so seeding today's bucket before start= would materialize early. defaultBucket now clamps to the start bucket and drops pre-start recent-missing chips; a hint explains the start anchor. - P2: upstream-missing hint checks each partitioned ducklake input separately instead of unioning, so a fan-in where one input has the bucket no longer masks another that lacks it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): reseed partition picker on header change and fail safe on bad tz/start Addresses second CI review (Codex): - P1: run form is now keyed on the parsed partition spec as well as the schema, so editing the // partitioned header (same schema) remounts the picker and reseeds/re-strips instead of keeping a stale bucket that re-bypasses the start anchor. - P1: malformed metadata is fail-safe in parity with the backend — invalid tz= no longer throws in Intl (falls back to UTC for display), invalid start= (e.g. 2026-02-31) is rejected via round-trip check, and neither auto-seeds an explicit partition (which would bypass the worker's own tz/start validation). A warning hint points at the header. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
6ae8dd37b1 |
fix(pipelines): pipeline-level run control, tables label, data-test rollback + fork badges (#9944)
* fix(pipelines): pipeline-level run control, tables label, data-test rollback + fork badges - Add always-visible "Run pipeline" header control (edit mode) that runs every script in dependency order via the bounded-cascade engine, so a run no longer requires hovering a node's play button. - Header summary counts ducklake/datatable assets as "tables" (and s3object as "files") instead of the raw kind, collapsing shared nouns. - Surface a data-test outcome badge on guarded asset nodes: EE shows a rolled-back (previous version left live) state, CE shows published-despite- failure — driven by the producer's last run state and the edition. - Make the fork data-environment marker a prominent labeled chip (⑂ fork / ↗ parent) instead of a bare icon. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): address CI review — scope Run pipeline to members, anchor guard badge, spin loader - Run pipeline now filters to `in_pipeline` script runnables, so it never launches dependency-only endpoints the graph shows for context (macro libraries, custom data-test scripts, out-of-folder producers). - Data-test guard badge only attaches to the producer's declared `// materialize` target, so a multi-output producer no longer badges its other ducklake writes. - Spin the Loader2 icon in the "Run pipeline" button while a run is in progress (startIcon classes), matching every other loading affordance. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): data-test badge copy speaks to write policy, not failure cause producerFailed is a generic job-failure signal, so the failed-state tooltip no longer claims the run "failed its data tests" (it could be a runtime/worker error). It now states the edition's behavior on any failed materialize: EE rolls back (previous version left live), CE may leave a failing write live. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipelines): Run pipeline keeps independent branches running after a failure runSelection used a single global fail-fast flag, so once any node failed it refused to schedule *any* newly-ready node — a failure in one branch could strand an unrelated healthy branch as 'skipped' depending on job timing. Now a failure poisons only its transitive descendants; independent branches finish. Add regression tests: independent-branch-survives-failure and join-node-skipped -when-one-upstream-fails. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
574d3ac9ff |
fix(pipelines): link SCD2 <dim>_current view to its producer across all graph surfaces (#9933)
* fix(pipelines): link SCD2 <dim>_current view to its producer across all graph surfaces An SCD2 producer (`// materialize … history`) creates the base table AND a `<dim>_current` view at runtime. The deploy path already registered both writes, but the CLI `--local` graph and the frontend live-editor graph only emitted the base write, so a consumer reading only `<dim>_current` orphaned there. Centralize the companion derivation in `MaterializeSpec::write_targets` / `scd2_current_target` (+ TS `scd2CurrentTargetPath` mirror), emit the `_current` write in every surface, and mark the companion node `derived_from` the base so the canvas renders it as a derived "current view" instead of an unrelated table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): keep scd2 _current write edge when editing a saved producer Addresses Codex CI review (P1): opening a deployed scd2 materialize producer for editing dropped its persisted `<dim>_current` write edge. `liveRefKeys` (the set of asset keys a saved-script edit preserves against stale-filtering) only added the base materialize target, so the companion `_current` write was judged stale and filtered — orphaning consumers of only the view mid-edit. Add `scd2CurrentTargetPath(m)` to `liveRefKeys` too; covered by a new saved-edit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
46be39dfb7 |
fix(pipelines): order data_test relationships refs before the tested script in a cascade (#9934)
* fix(pipelines): order data_test relationships refs before the tested script in a cascade Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): key custom-test reads by (usage_kind, path) to avoid same-path flow collisions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
0ad174fa49 |
feat(pipelines): self-teaching custom data_test errors + scaffold (#9937)
Custom `// data_test <path>` scripts must be a single SELECT reading the freshly-materialized target via the internal `_wm_target.<table>` alias — neither was documented or scaffolded. Make the codegen errors name the exact violation (multi-statement, non-SELECT, wrong alias, empty) and append a copyable `SELECT * FROM _wm_target.<table> WHERE <condition>` example. Add a DuckDB-only 'Data test' pipeline output kind that scaffolds that starter body. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
22452ce540 |
fix(object-storage): remove 20-file bucket-browser listing cap in CE (#9935)
* fix(object-storage): remove 20-file bucket-browser cap in CE The Community Edition build rejected the object-storage `list_stored_files` endpoint with an error once a workspace bucket held more than 20 objects, making the bucket browser unusable on larger buckets. The listing already collects up to `max_keys` objects, so the hard cap was purely a gate. Drops the CE listing cap (in the EE-symlinked `job_helpers_ee.rs`, tracked in the companion windmill-ee-private PR) and removes the now-inaccurate sentence from the workspace object-storage settings banner. The 10 GiB total-storage write quota remains as the intentional CE limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 4bd7b73dcef1b77dc2866bc58e0c86962559495c This commit updates the EE repository reference after PR #648 was merged in windmill-ee-private. Previous ee-repo-ref: aa14d0724216030948c2f575bcc19c0e6e0476a7 New ee-repo-ref: 4bd7b73dcef1b77dc2866bc58e0c86962559495c Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszelandrubenfiszel
|
799b9e3b7c |
chore(main): release 1.748.0 (#9914)
* chore(main): release 1.748.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
5769b6036c |
fix(pipelines): activity-axis label clarity + select failed node on cascade failure (#9931)
* fix(pipelines): clarify activity-window axis label + select failed node on cascade failure Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): clear active draft so failed-node focus is not masked PipelineGraphEditor gives an open draft (activeDraftPath) priority over selection via openScriptPath, so the cascade-failure focus set only selection and stayed masked while a draft pane was open. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Fable 5
|
39eb9de1bc |
feat(pipelines): fork data environments for ducklake materialization (dev data) (#9915)
* feat(pipelines): fork-scoped ducklake namespaces with read-defer to parent Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): fork graph indicator + fork ducklake namespace cleanup endpoint Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): fork_views-keyed view transition, fork lineage clone, design doc Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): review hardening - fork DATA_PATH last-wins, registry cache TTL, defer tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): per-lake isolated/shared choice at fork creation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): chain-aware defer discovery + per-location fork namespace registry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): lake-scoped fork schemas, catalog identity in registry, chain-aware graph chips Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): cleanup deletes fork data from the registered storage identity Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): collapse fork data-path segment to one component (slash-safe ids) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): per-catalog ancestor checks, ancestor extra_args passthrough, test compile fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): invalidate fork ancestor-chain cache on lineage mutations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): sweep descendant ancestor-chain caches on delete/reparent Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): run fork ducklake cleanup inline in delete_workspace Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): resolve fork cleanup credentials pre-commit, destroy post-commit Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): shared dev-workspace authz gate for namespace drop, invalidatable registration cache, segment-boundary delete filter - extract require_prod_admin_for_dev_workspace, used by both delete_workspace and drop_forked_ducklake_namespaces so the gates cannot drift - key FORK_DUCKLAKE_REGISTERED per workspace and invalidate it in cleanup_fork_ducklake_namespaces so a same-id fork recreated within the TTL re-registers its namespaces - filter listed object locations to the segment boundary before deletion Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): keep orphaned wm-fork-* workspaces ducklake-isolated parent_workspace_id is ON DELETE SET NULL, so a fork can outlive its parent with an empty ancestor chain while its cloned config still points at the shared lake. Key the isolation gate on the wm-fork- prefix as well as the chain (mirroring workspace_is_fork): orphaned forks get the write redirect, registration and cleanup with zero ancestors (no defer), and keep their 'fork' graph chips. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): attach orphaned wm-fork-* ancestors at their fork namespace Chain position alone classified the last ancestor as a root, but an orphaned wm-fork-* ancestor (its own parent deleted, SET NULL) ends the chain the same way while its data lives in its fork namespace — its descendants' defer views bound the dead root's lake instead. Key the root-vs-fork decision on the wm-fork- prefix too, matching the resolution gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): never inherit shared lake opt-out; durable cleanup ledger for failed fork deletions - fork creation strips cloned fork_behavior stamps before applying the request's shared_ducklakes list: sharing is a per-creation choice, a fork of a shared fork defaults back to isolated - fork_ducklake_namespace loses its ON DELETE CASCADE FK: rows are the durable cleanup ledger and outlive the workspace when physical cleanup fails post-commit; fork creation retries leftover rows for the reused id and refuses to create while a metadata schema still cannot be dropped (data-file leftovers alone are inert once the schema is gone and are swept by the next successful same-prefix cleanup) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): make orphaned-namespace cleanup retries independent of deleted fork resources - ledger rows gain a schema_dropped phase flag: set when the schema drop succeeded but data cleanup failed, so later retries skip the schema phase and need no catalog credentials at all; registration resets it on re-attach (ON CONFLICT DO UPDATE) since attaching recreates the schema - retry-path $res: resolution falls back to the workspace being forked (the deleted fork's resources were clones of a parent's); live paths (delete_workspace prepare, drop endpoint) pass no fallback Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): fork tables from failed-after-commit runs stay fork-owned in defer and graph A failed materialization must not disguise a physically existing fork table as deferred: CREATE VIEW IF NOT EXISTS silently yields to the table, so reads hit fork data while the graph claims parent defer. - record_mat upsert preserves the last committed snapshot_id on failure - defer discovery and graph chips treat fork rows with a committed snapshot as fork-owned even when status is failed - inspect_fork_catalog also lists live fork tables (same round trip) and the defer list is filtered against them — covers rows recorded before this fix and tables created by raw SQL - drop stale FK-cascade wording in the design doc and sidebar comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): fork-mode ducklake settings — per-lake isolated/shared chips + banner, fork_behavior round-trip The workspace-settings ducklake editor had no fork awareness: no reminder of each lake's isolated/shared choice and no warning about what edits mean in a fork. It also rebuilt each lake explicitly on save, silently dropping fork_behavior — any settings save in a shared fork flipped the lake back to isolated. - fork detection mirrors the backend gate (parent link or wm-fork- prefix) - info banner explaining isolated vs shared semantics in a fork - per-lake chip (emerald 'isolated' / amber 'shared with parent') with tooltips, matching the pipeline graph chip colors - fork_behavior added to DucklakeSettingsType and preserved through convertDucklakeSettingsToBackend Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d4b4374de8 |
feat(pipelines): capture violating-row samples for data tests (#9919)
* feat(pipelines): capture violating-row samples for data tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): byte-accurate sample cap and leaf-level payload sanitize Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: bump ee-repo-ref to WAP guard probe adaptation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: WAP failures are counts-only — samples exist only on commit-then-test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: qualify where sample row data appears — job result and failed-job log line Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: error handlers receive the full result incl. samples, like any failed job Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 80d309edebb899e36a3bdcdf4ea73c4db070534d This commit updates the EE repository reference after PR #646 was merged in windmill-ee-private. Previous ee-repo-ref: 16e916bf11f26381920560b55771fce693e668c6 New ee-repo-ref: 80d309edebb899e36a3bdcdf4ea73c4db070534d Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Fable 5
|
5ad2de91a2 |
feat(sdk): enforce s3:// URIs for string S3 params + ingestion (EL) docs (#9912)
* feat(pipelines): ingestion (EL) templates + docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): review nits — draft collision guard, template-mode selection reset, invariant test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): lead the insert menu with ingestion templates Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(pipelines): ingestion story as docs-only — drop editor template UI The insert-menu template section mixed two selection grammars in one popover and confused more than it helped. The three E2E-verified example pipelines now live verbatim in docs/pipeline-ingestion.md; the Python bare-string S3 key fix in pipelineTemplates.ts stays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sdk): bare string S3 keys in py/ts clients + asset parsers A plain string passed where an S3Object is expected is now a bare key in the default storage — previously the py client silently degraded it to s3="" (auto-generated key) and both asset parsers canonicalized it without the leading slash, splitting lineage. parseS3Object moves to s3Types.ts so it is unit-testable without the generated services. The pipeline template fix from the earlier commit is superseded (bare strings are the supported spelling again); docs examples flipped to bare keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(sdk): enforce s3:// URIs for string S3Object params Bare strings now raise/throw with a hint pointing at the s3:///<key> spelling instead of being treated as keys (previous commit) or silently degrading to an empty key (original behavior). One string spelling everywhere: SDK calls, // on annotations, and DuckDB SQL all use s3:///<key>. TS regains the s3://-template-literal type; the asset parsers record no asset for a bare string (the call can only error); templates emit the URI form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(pipelines): move ingestion (EL) guide to windmilldocs, keep design constraints User-facing how-to (engine choice, cursor recipes, schema drift, worked examples) moves to windmilldocs core_concepts/63_pipelines (windmilldocs#1462); the repo keeps only the design constraints future feature work must not break, as a section of ducklake-materialization.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate system prompts after parse_s3_object docstring change Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): reject empty-key s3 URIs; align asset parsers with the runtime rule Addresses CI review: s3:/// and s3://bucket/ now raise (an empty key would fall back to the auto-generated-key path the strict contract exists to prevent); the asset parsers' string branch applies the same valid-URI-with-non-empty-key rule so no R/W edge is recorded for a call that can only error (the generic URI-literal scan still records ambiguous access-None assets, by design); comments rephrased as current constraints per AGENTS.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
33521505db |
feat(ducklake): scheduled lake maintenance (expiry, compaction, orphan cleanup) (#9916)
* feat(ducklake): scheduled lake maintenance (snapshot expiry, compaction, orphan cleanup) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): review fixes — starts_with not LIKE, CE license-lapse escape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ducklake): auth-contract docs + _unchecked rename per codex review Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ducklake): move maintenance payload construction into EE module Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): fall through to script resolution for non-managed reserved-prefix schedules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ducklake): document accepted pre-existing-schedule limitation on the reserved prefix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): CE save-off clears the managed schedule row and queued occurrence Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 2fab310d4f50ed7c34857d69c9b854f4491bf217 This commit updates the EE repository reference after PR #645 was merged in windmill-ee-private. Previous ee-repo-ref: fff1fd830a36beba732486f05941ec243cf6b640 New ee-repo-ref: 2fab310d4f50ed7c34857d69c9b854f4491bf217 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasaandClaude Opus 4.8
|
a368d49bd8 |
feat(ai-agent): support reasoning effort in AI agent workflow steps (#9886)
* feat(ai-agent): support reasoning effort in AI agent workflow steps Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): round-trip native Anthropic thinking blocks and fix DeepSeek/Mistral reasoning Address review: native Anthropic now captures the signed thinking block during streaming and replays it before tool_use across iterations (prevents a 400 on multi-turn tool use). DeepSeek 'off' sends thinking:{type:disabled} instead of the rejected reasoning_effort:none, and Mistral drops temperature when reasoning is on. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-agent): move reasoning effort into the provider/model selector Store reasoning_effort on ProviderConfig (next to the model) instead of a separate flow arg, and render the selector inside AIProviderPicker under the model dropdown. Add an explicit 'off' option on models that disable reasoning by omission (e.g. Claude), so reasoning can always be turned off from the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-agent): use DropdownV2 for reasoning effort, matching copilot chat Replace the Select combobox with the same DropdownV2 action-menu the copilot chat reasoning selector uses. Each option carries an action instead of a bound value, so click selection is unambiguous and there is no typeahead/sentinel-value mismatch on the off/default entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(ai-agent): regenerate system prompts for ProviderConfig.reasoning_effort Refresh system_prompts/auto-generated and cli skills.gen after adding reasoning_effort to the OpenFlow ProviderConfig schema (check-freshness). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): clear stale reasoning effort on model change; dedup bedrock reasoning folding Address cubic review: (P1) the reasoning picker now clears the stored effort when the newly selected model doesn't accept it (e.g. carrying 'xhigh' from Opus onto a model that tops out at 'high'), not only when the model can't reason at all. (P3) the proxy's accumulate_reasoning_delta now delegates to the shared bedrock_stream_event_to_reasoning_delta so worker and proxy folding can't drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-agent): stream reasoning summary and show a thinking affordance in flow chat Add StreamingEvent::ReasoningTokenDelta, emitted from every worker reasoning path (Anthropic native thinking deltas, Bedrock, Gemini thought parts, OpenAI-compatible reasoning_content, OpenAI Responses reasoning_summary_text with summary:auto). The flow chat parses it and renders a collapsible 'Thinking' affordance on the assistant message (thinking tokens bill regardless of display, so surfacing the summary is billing-neutral). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): persist streamed reasoning onto the finished chat message Reasoning isn't stored server-side, so the completion re-poll (which swaps temp messages for the persisted ones) was dropping the streamed thinking summary. Carry it onto the final assistant message so the 'Thought process' affordance survives the run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-agent): match flow-chat thinking box to the copilot chat reasoning UI Replace the plain <details> thinking affordance with the same bordered, collapsible reasoning box the copilot chat uses (chevron + Brain/spinner + 'Thinking' header, markdown body, expand-while-streaming/collapse-on-answer). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): attribute streamed reasoning per turn by content; drop duplicated comment Address review: the completion-poll carry-over now matches each temp assistant turn's thinking summary to its persisted message by content, so a multi-turn response (reasoning -> tool call -> final answer) no longer misattributes an earlier turn's thinking to the final answer or drops intermediate turns. Also removes a leftover duplicated comment block above the AIReasoningEffortPicker effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): address review round 2 (carry-over edges, off-token validity, aria, test) cubic round 2: (1) reasoning carry-over now consumes temp turns in order verifying content, so identical/empty-content multi-turn responses attribute thinking correctly and reasoning-only turns aren't dropped; (2) the picker's stale-value check only accepts the off token when the model can actually disable reasoning; (3) add aria-expanded to the Thinking toggle; (4) add a test for the failed tool_result path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): add bottom margin below the flow-chat thinking box Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): don't request OpenAI reasoning summary, matching the copilot chat OpenAI gates reasoning summaries behind org verification, so requesting summary: auto would 400 for unverified orgs. The copilot chat requests effort only and never asks for a summary; align the worker with it (reasoning: { effort }) and drop the now-unreachable summary-delta parsing. OpenAI reasoning no longer streams a summary in flow chat (consistent with the copilot); Anthropic/Bedrock/Gemini/DeepSeek reasoning display is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): scope reasoning carry-over to newly persisted messages cubic round 3: matching by content over the full history could attach a new turn's reasoning to an older message with identical text. Restrict eligible targets to the messages just fetched for this response (via afterSeq), so historical turns are never touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): carry reasoning newest-first instead of gating on the final poll cubic round 4: gating the carry-over on the final poll's filteredResponse dropped reasoning for messages already fetched by an earlier streaming poll (their id is excluded by afterSeq). Walk persisted newest-first and consume the newest matching pending summary, stopping once summaries run out. This response's turns are always at the end, so they claim their own reasoning (P1) before older history is reached (P2), regardless of which poll persisted them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-agent): drop flow-chat reasoning display, keep backend + effort picker The chat-side thinking box relied on non-deterministically matching streamed (ephemeral) reasoning back onto persisted messages, which kept spawning edge cases. Remove the flow-chat display entirely (ChatMessage box, FlowChatManager carry-over/threading, parseStreamDeltas reasoning) and keep the sound backend: per-provider reasoning-effort requests, thinking-block round-trips for tool calls, and ReasoningTokenDelta streaming. A display can be built on top later, deterministically (e.g. once the stream carries the persisted message id). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): include reasoning_effort in default-config compare; document reasoning_token_delta Codex/Pi nits: isSameAsStoredConfig now compares reasoning_effort so the 'use as personal default' toggle reflects effort-only changes; openflow streaming-events doc lists the reasoning_token_delta event (regenerated auto prompts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Fable 5
|
42e11c6570 |
feat(pipelines): schema contracts — save-time consumer checks vs captured schemas (#9917)
* feat(pipelines): schema contracts — save-time consumer checks vs captured schemas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: move schemaContractContext above schemaCanEvolve doc comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: emit scd2/on_schema_change in CLI local graph, address review notes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: gate editor _current ignore-suppression on scd2, matching backend Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
5d7fb6deca |
feat(pipelines): asset freshness — fresh/stale badge (CE) + watchdog (EE) (#9909)
* feat(pipelines): passive asset freshness tracking on the graph Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(pipelines): drop dead freshness-enforcement stub, document query ordering Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): freshness watchdog (EE) — auto re-run stale producers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): watchdog review fixes — archived workspaces, badge kind parity, scan index Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): CI review — no singlestepflow in freshness, +N parity, completion-time fallback Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): CI review — history completedAt, freshness/asset trigger UI metadata Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 6f5fe0f7f56696fbef5a8349da38496c32e71666 This commit updates the EE repository reference after PR #643 was merged in windmill-ee-private. Previous ee-repo-ref: 1f13380354bf591ae25a2c20d36917534bcc5459 New ee-repo-ref: 6f5fe0f7f56696fbef5a8349da38496c32e71666 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Fable 5
|
af36498432 |
feat(pipelines): record upstream snapshot ids on cascade-dispatched jobs (#9910)
* feat(pipelines): record upstream snapshot ids on cascade-dispatched jobs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: batch upstream-snapshot lookup and memoize per subscriber Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszelandrubenfiszel
|
df6e511763 |
chore(main): release 1.747.0 (#9901)
* chore(main): release 1.747.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
GuilhemandClaude Fable 5
|
4ba17d0f9c |
fix: prevent truncated tool call args from bricking AI chat sessions (#9902)
* fix(frontend): prevent truncated ai chat tool-call arguments from bricking the session Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: heal empty tool-call arguments when replaying chat history Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: state sanitize invariant without drafting history Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
hugocasaandClaude Opus 4.8
|
d600c7ecfe |
fix(ai): route Azure Foundry Claude models via Anthropic Messages API (#9908)
* fix(ai): route Azure Foundry Claude models via Anthropic Messages API Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai): keep explicit Azure OpenAI deployment base URLs intact build_azure_openai_url only appends /openai/v1 for a bare resource root; any base with an explicit path (e.g. .../openai/deployments/<id>) is preserved. Adds a regression test and a unit test for usesAnthropicMessagesApi. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): enable Claude extended thinking on Azure Foundry Route azure_foundry+Claude through the Anthropic reasoning branch (adaptive thinking + output_config.effort) instead of the gpt/o gate, and recognize claude-sonnet-5. Live-verified: sonnet-5 and opus-4-8 on Foundry accept the low/medium/high/xhigh/max ladder and render summarized thinking. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Fable 5
|
ae85d27437 |
feat(frontend): add federatedTokenFile field to instance object storage Azure config (#9904)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
bcb4dd8d27 | docs link | ||
|
Ruben Fiszelandrubenfiszel
|
fad5419b9d |
chore(main): release 1.746.0 (#9872)
* chore(main): release 1.746.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |