Add a "Windmill CI tests" GitHub check run on any PR against the tracked
branch, so a workspace-fork customer can mark it a required status check and
have Windmill's own CI tests (the `// test:` annotation) block the PR —
replacing the GitHub Action that polls the CI-test results API.
Driven by the pull_request webhook (same event as the Phase 4 diff check): a
new git_sync_ci_test_check table tracks one check per (fork workspace, PR head
commit); a per-ci_test-job completion hook (result_processor) and the git-sync
poller sweep (monitor) conclude it. Reuses the Checks: write grant from #9552;
token repos keep the documented Action. See docs Phase 7.
EE logic lives in windmill-ee-private (git_sync_ee.rs); see ee-repo-ref.txt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
promotion_open_prs/fork_open_prs are rejected on save without an Enterprise
plan (like auto_pull), and the deploy completion hook re-checks the plan
before opening PRs so flags stored while licensed stop driving GitHub calls
after a lapse.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
A git resource without an explicit branch polled as the bare "HEAD" ref,
which the fork/dev-label fan-out cannot scope (wm-fork/<branch>/*), so fork
sync silently never ran on polling-only repos. Resolve the remote's default
branch name with `ls-remote --symref HEAD` (one call for name + head sha);
"HEAD" only remains when resolution fails. The polling e2e test now uses a
branch-less resource to cover this shape.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
The full read-modify-write raced the poller's concurrent last_synced_sha /
last_pull_status writes on the same column; mirror the EE status writer and
update only the matching repository element's open_pr_error key.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- addSyncRepository skips the auto_pull/fork_open_prs defaults on fork/dev
workspaces where the backend rejects them (saving a new sync repo from an
EE fork 400'd deterministically)
- change_workspace_id strips webhook id/secret/error from the copied git_sync
and deletes the stale GitHub hooks post-commit: they deliver to the old
(archived) workspace URL, so the new workspace would report a live webhook
while polling at the relaxed interval; next save re-registers cleanly
- EE: PR diff checks for contributor-fork PRs clone the synthetic
pull/<n>/head ref (head.ref doesn't exist in the base repo)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- detach_dev_workspace clears parent_workspace_id for prefix-less (attached
standalone) workspaces so they stop classifying as forks and deploying to
wm-fork/** branches; wm-fork- re-designated forks keep their parent; cache
invalidations mirror attach
- dev-workspace create/attach reject an environment label that equals a
git-sync repository's tracked branch (prod's or the candidate's): deploys
would target the very branch the repo syncs from, and the CLI guard would
fail every push job after the fact
- CLI unit tests: prefix-less fork beats wm_deploy derivation; isForkWorkspace
parent-id argument
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(db-health): add connection sizing guidance
The Database Connections panel showed current/max connections but gave no
guidance on how to size max_connections for the deployment. Derive an estimate
from the live worker fleet: each worker instance shares a pool sized
DEFAULT_MAX_CONNECTIONS_WORKER + (workers - 1), and each server opens up to
DEFAULT_MAX_CONNECTIONS_SERVER (both overridable via DATABASE_CONNECTIONS).
The endpoint now returns live worker/instance counts, the default per-server
and per-worker pool sizes, the estimated peak worker connections, the reserved
superuser connections, and a recommended max_connections floor (workers + one
server + 25% headroom). Servers do not ping worker_ping, so the recommendation
assumes one server and exposes the per-server increment. The panel renders this
as a sizing breakdown and warns when max_connections is below the recommended
floor.
Fixes WIN-2147
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(db-health): single source for pool-size constants + sizing tests
Address review: db_connect.rs kept its own copies of DEFAULT_MAX_CONNECTIONS_*
that duplicate the windmill_common constants the sizing guidance reads, so
tuning the runtime pool size would silently leave the guidance stale. Re-export
the windmill_common constants from db_connect.rs so there is one source of truth.
Add unit tests for compute_connection_sizing covering the zero-fleet, single
worker, multi-instance, and reserved-clamp cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(db-health): 20% headroom and 200-connection minimum floor
Lower the sizing headroom from 25% to 20% and never recommend below 200
connections (postgres defaults to 100; cheap headroom for growth/bursts/psql).
Update the guidance message and unit tests accordingly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db-health): honor DATABASE_CONNECTIONS in sizing recommendation
Address Codex P1: the runtime caps every process's pool at DATABASE_CONNECTIONS
when set (db_connect.rs), but the sizing guidance always used the default 50/5
pools. For a tuned deployment this under-estimated worker demand and could hide
a genuine under-provisioning (e.g. DATABASE_CONNECTIONS=100 with 5 instances is
500 worker connections, not 25).
compute_connection_sizing now takes the effective DATABASE_CONNECTIONS override
(read the same way db_connect.rs reads it): when set, each worker instance and
server pool is that value and the worker estimate is override * instances. The
response exposes server_pool_size / worker_pool_size (effective) and
database_connections_override; the panel renders both pool rows and labels them
(default) vs (DATABASE_CONNECTIONS), and the message states which source is used.
Adds a unit test for the override path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db-health): exclude agent workers from connection sizing
Agent workers reach the API over HTTP (MODE=agent, Connection::Http) and hold
no postgres pool, but their pings still land in worker_ping (written server-side
by /api/agent_workers/update_ping). Counting them inflated the connection
estimate. Filter the fleet query by the worker-name prefixes: DB-connected
workers use "wk-" (WORKER_NAME_PREFIX), agent workers use "ag-"
(AGENT_WORKER_NAME_PREFIX). Only wk- workers/instances feed the estimate; ag-
workers are counted separately and surfaced as context ("N agent workers
excluded — they use HTTP, not postgres connections"). Adds a unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- edit_git_sync_repository re-checks the runtime Enterprise gate against the
EFFECTIVE repo state after preservation: the older-client arm copies the
existing auto_pull back, which the request-side check never saw
- attach_dev_workspace now mirrors the fork-creation copy on the attached
workspace's own git sync: promotion repos dropped, auto_pull/fork PRs/PR
error stripped, and any managed webhook deleted after commit (the attached
workspace is parent-managed and must not keep pulling its old tracked branch)
- integration test: attaching an auto-pull-enabled workspace strips it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- Fork sync toggle renamed and kept in the pull section; the fork PR toggle
moves to the push section with a note that push settings apply to forks
- Fork/dev workspaces' push section names their actual branch instead of the
tracked-branch line; promotion repos hide the pull direction (promotion
pushes deploy branches on top of a sync-mode setup)
- Promotion mode line describes the wm_deploy/** branch + merge-to-promote
flow; workflow-fallback hints lead with the how-to and link to the docs;
test connection button demoted from accent per brand guidelines
- New server-owned open_pr_error on repo settings: the deploy completion hook
records why a PR couldn't be opened (e.g. app permission not yet approved)
and clears it on the next success; shown as a warning under the PR toggles
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
A PR that only touches files outside the repository's include paths gets
"In sync", which reads as a wrong verdict; the check summary (and managed
comment) now name the filters, e.g. "Only files matching this repository's
sync filters deploy on merge: `f/**` (excluding `f/pat/**`)."
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- CE: the auto-pull and fork-PR toggles are disabled with an EE badge, and
new sync repos only default them on when licensed (basic git sync is
available on CE since #8493, but auto-pull is EE and the backend rejects it)
- The pull modal passes clone_ref for wm-fork- forks (wm-fork/<tracked>/<id>)
so a manual pull fetches the fork branch instead of the tracked branch head
- PR-on-deploy skips no-op pushes: when the push script reports pushed=false
(e.g. the deploy was caused by an auto-pull), the completion hook no longer
ensures a PR, so closed PRs aren't recreated by the sync loop
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- The PR diff completion hook maintains one managed comment on the PR
(Cloudflare deploy-preview style: workspace, status, commit, collapsible
change list), upserted per synchronize via a hidden marker. The check run
stays for required-check gating.
- A settings difference in the diff summary is worded by cause: the PR
changes wmill.yaml, vs pre-existing drift between the repo's wmill.yaml
and the workspace, vs undetermined (neutral wording).
- Deploy-status check titles name the target workspace ("Deployed 2
change(s) to staging"), since GitHub shows a head commit's checks on any
PR containing it and a bare "Deployed" read as if the PR had deployed.
- Token-based repos see a hint pointing at the open-pr-on-commit /
open-pr-on-fork-commit workflows where the app-only PR toggles would be;
an API-set toggle on a non-app repo now logs a warning naming the
fallback; the design doc lists app-only features and their degradation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat: add parent_workspace claim to OIDC job tokens for fork workspaces
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: rename claim to fork_parent_workspace for clarity
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: update ee-repo-ref to e2df172596e00877068d4b0a98afaef62fe429d1
This commit updates the EE repository reference after PR #651 was merged in windmill-ee-private.
Previous ee-repo-ref: f73001ac6c038694cfc2604233a59be1c0daa40b
New ee-repo-ref: e2df172596e00877068d4b0a98afaef62fe429d1
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
A fork of a dev workspace now roots its wm-fork/** branch on the dev's
environment-label branch (the content it diverged from) and its PR merges
back into that branch: the backend passes parent_dev_workspace_label with
the deploy (parent row joined in both enqueue paths), the CLI gains
--parent-dev-workspace-label and checks it before the wm-fork- prefix
fallback when rooting a fork-of-a-fork branch, and the PR completion hook
uses it as the PR base.
Fork sync routing covers the whole live descendant chain of the
webhook/poller workspace (recursive, depth-capped) instead of direct
children only, and fork_open_prs is resolved at the root ancestor — only
the root can hold auto-pull config, so grandchild forks sync through it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
A dev workspace's git branch is its environment label verbatim (dev/
staging, default dev) — a first-class env branch like the documented
push-on-merge-staging layout — instead of the wm-fork/** form. The label
rides the deploy job args (backend → hub script → CLI
--dev-workspace-label), the PR completion hook derives the same head, the
webhook/poller route label branches into the matching dev-workspace child
(poller lists them alongside wm-fork/* via extra ls-remote refs / per-label
API lookups), and manual pulls from the UI pass clone_ref accordingly. The
CLI refuses to deploy when the label branch equals the checked-out tracked
branch, which would otherwise commit fork content straight to it.
Because the branch is keyed on the label, the label is now immutable after
creation: set at create/attach only, the set_dev_workspace_label endpoint
is removed and the settings tab shows it read-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
Fork sync (push-on-merge-to-forks parity): a parent-level
auto_pull.sync_forks toggle routes changes on each fork's wm-fork/** branch
into that fork workspace, via the parent's existing webhook and one extra
fork-heads listing per poll tick (git ls-remote pattern for token repos,
git/matching-refs for app-backed). Fork state is a server-written
status-only auto_pull blob on the fork's own repo entry; the fork's card
shows a read-only "managed in the parent workspace" line with its branch
and last pull status. Dev workspaces (prefix-less ids) use the same branch
parsing (unit-tested in windmill-common).
PR-on-deploy: opening PRs for Windmill-pushed branches moves into the
deploy pipeline, per repo toggle (promotion_open_prs on the promotion
repo; parent-level fork_open_prs for fork deploys). The push job carries a
marker and the job-completion hook derives the pushed branch (helper
unit-tested against the CLI formula) and opens the PR outbound, so it
works without inbound webhooks; the webhook-side wm_deploy PR arm is
removed. The documented open-pr-* GitHub Actions remain valid alternatives
(PR creation is idempotent).
Fork guards: promotion mode, enabled auto-pull, and fork_open_prs are
rejected on fork workspaces (they are parent-managed; a fork's deploys
always target its wm-fork/** branch) and the promotion card is hidden in a
fork's settings. Enabling auto-pull now also requires EE, and the
post-commit webhook reconcile persists the normalized delivery mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat: add datatable_migrations table
* feat: add route to run datatable migrations
* feat: sync datatable migrations as .up.sql/.down.sql files
* feat: add datatable migrate up/down commands and post-push run prompt
* feat: add datatable migrate new command to scaffold migrations
* feat: add datatable migrations management UI
* feat: prompt to create migration on DDL in datatable SQL editors
* feat: support running a single specific datatable migration
* feat: view migration content, run single migration, fix stacked modal
* feat: per-row revert button with out-of-order warning
* fix: avoid migrations list flicker on refresh after an action
* feat: generate initial datatable migration via pg_dump
* fix: surface datatable migration API error details in toasts
* fix: revert created migration if create-and-run fails to run
* fix: include postgres error detail in migration run/rollback failures
* feat: sync datatable migrations as files via the workspace export
* refactor: move datatable migrations to migrations/datatable/ path
* fix: drop redundant datatable_migration label in sync output
* fix: exclude datatable migration sql files from script metadata generation
* feat: run datatable migrations as user-permissioned labeled jobs
* feat: reject invalid datatable migrations on sync push
* feat: datatable migrate up/down default to all datatables, --datatable to target one
* fix: surface postgres error detail when datatable migrations fail to run
* chore: regenerate CLI docs for datatable migrate commands
* feat: default new datatable migration to a BEGIN/END transaction template
* fix: validate datatable migration name and datatable at the API boundary
* fix: ensure detected DDL ends with semicolon when wrapped in transaction
* fix: re-prompt instead of stripping DDL when new-migration modal is cancelled
* feat: refresh datatable schema after running a migration from the SQL REPL
* feat: record db manager DDL on data tables as migrations
* feat: make datatable migrations opt-in per data table
* fix: make migration view editor read-only so its code can scroll
* fix: don't re-prompt DDL guard when creating a migration without running
* feat: generate down migrations for db manager DDL (postgres)
* fix: correct down migration for db manager alters (no double-wrap, serial)
* feat: explain migrations purpose with a tooltip in the migrations modal
* compare paeg
* feat: add datatable_migration kind to workspace diff pipeline
* chore: point ee-repo-ref at datatable_migration git-sync companion
* fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests
* feat: deploy and run datatable migrations on workspace merge
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Refactor + handle datatable setting delete/rename
* refactor: move datatable migration rename/delete cascade into module
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(db-manager): add Migrations button to top bar, make Refresh icon-only
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* BEGIN/END placeholder in down migration
* feat: autofocus migration name input and flag it red when empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* border nits
* refresh db manager schema on migrations
* BEGIN/END scaffold in CLI
* feat(cli): push local datatable migrations before running on migrate up
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: flag invalid migration name with red border, not just empty
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop random slug from auto-generated migration names
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: offer revert-and-delete when deleting an installed migration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: record fork merge as a migration when target datatable opts in
* nit
* clone migrations on fork
* windmill-utils-internal
* fix(datatable-migrations): serialize run/rollback with a per-db advisory lock
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(db-manager): fail closed when migrations-status check errors on DDL apply
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: fix generate_initial migration ordering comment to match code
* chore(datatable-migrations): remove unused update_datatable_migrations endpoint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: run DDL migration guard on the script editor Test button
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* split
* ee-repo-ref
* chore(frontend): sync package-lock with package.json (@emnapi deps)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(datatable-migrations): never resolve instance credentials into migration job args
datatable_database_arg eagerly resolved instance data-table credentials
(including the shared instance-wide Postgres password) and passed them as the
migration job's plaintext `database` arg, landing in v2_job.args. Since the
run route has no admin gate, a non-admin could run a migration and read
args.database to recover the password, granting cross-workspace psql access to
all instance data-table DBs.
Pass a `datatable://<name>` reference for both resource-backed and instance
data tables instead; the pg executor already resolves it to real credentials
server-side at run time, so nothing sensitive is ever stored in the job args.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* fix: handle dollar-quoting and comments when splitting SQL statements
* feat: deploy datatable migrations on merge with explicit opt-in error
* fix(frontend): sync package-lock with npm 11 peer-dep resolution
npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from
lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as
peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly
1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the
highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree
needs both versions; the committed lock only had 1.10.0.
Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for
the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit npm publish
* fix: fail closed on migrations-status error in fork schema merge
* nit CI emnapi/core version
* prevent initial_datatable_migration if migrations already exist
* fix(datatable-migrations): validate persisted data table names as path segments
edit_datatable_config only validated rename segments, not the actual
settings.datatables keys, so a data table could be saved directly under a name
like '..' or one containing '/'. Since new tables default to
migrations_enabled = true, generate_initial_datatable_migration would then
insert a migration row and the sync export would build
migrations/datatable/<name>/... paths from that name, producing malformed or
directory-escaping export paths.
Validate every persisted data table name in edit_datatable_config (alongside
the existing rename checks) and add validate_datatable_path_segment to
generate_initial_datatable_migration for defense in depth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: scope datatable _wm_migrations by data table and cascade renames/deletes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(system_prompts): resolve nested local command groups in CLI docs generator
The CLI docs generator anchored on the first `new Command()` in a file and
never resolved locally-defined command groups passed as
`.command("name", localCmd)`. For datatable this flattened the nested
`migrate` group: it emitted `datatable new/up/down` plus a bare
`datatable migrate`, and mislabeled the datatable command with the migrate
group's description. jobs was broken the same way (its description was pull's,
and pull/push rendered empty).
Anchor block extraction on the `export default`ed command, recurse into
locally-defined `const x = new Command()` groups mounted as subcommands, and
render nested sub-subcommands. Regenerated docs now show
`datatable migrate new/up/down` and `jobs pull/push` with their real
options.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: drop unreleased _wm_migrations legacy-upgrade handling
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: return datatable migration SQL from getItemValue for the diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* nit
* nit
* fix: handle datatable migration renames on push and dedupe timestamps
* fix: reject rewriting an already-applied datatable migration on upsert
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries
Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi
declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved
lockfile entries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): datatable migrate up/down default to main datatable, not all
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: fail closed when applied status unreadable on datatable migration rewrite
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: surface full error detail in Database Manager DDL/query errors
* "See migration" button in the toast
* feat: add Enter shortcut to Create-a-migration in the DDL guard
* fix(frontend): warn before running a newly-created datatable migration out of order
The row-level Run action warns when earlier migrations are still pending, but
the create-and-run paths ran a just-created migration with `only` directly,
applying it ahead of older pending migrations without that confirmation.
Reuse the same "Run migration out of order" confirmation across all
create-and-run paths via a shared helper (datatableMigrationUtils):
- NewDataTableMigrationModal "Create and run" (and the DDL guard path)
- DatatableSchemaDiff fork→parent merge
- dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a
MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a
silent cancel
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: keep renamed datatable migrations visible in compare view
* fix: record per-migration deployment on datatable migrations disable
* fix(cli): run deployed datatable migrations after workspace merge
The merge command upserted datatable_migration definitions into the target
workspace and reported the item as successfully deployed, but never ran the
migrations. For forked datatables backed by separate databases, this left the
target schema unchanged until someone manually ran `wmill datatable migrate up`,
while the CLI reported a successful merge.
Collect the datatable migrations deployed (not deleted) into the target and,
after the deploy loop, offer to run them via the existing offerToRunNewMigrations
helper — the same post-deploy run prompt the push/sync path uses (interactive
only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export
parseDatatableMigrationDeployPath so the merge path can parse the deployed items.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(backend): serialize datatable migration edits/deletes with the run lock
A migration run snapshots a migration's code_up from datatable_migrations and
only records its version in the data table's _wm_migrations after the job
succeeds. upsert_datatable_migration checked _wm_migrations before allowing an
edit but took no lock, so a concurrent edit could read "not applied yet",
rewrite code_up/code_down, and then the in-flight run would record the version
for the old SQL — leaving _wm_migrations pointing at SQL that was never applied
(migrate up then skips it; rollback runs a down that doesn't match).
Serialize definition rewrites and deletes with the same per-database advisory
lock the run/rollback paths use:
- Factor the connect+advisory-lock into lock_datatable_migration_runs and the
applied-versions read into read_applied_versions_on_client.
- run_datatable_migrations now snapshots the definitions AFTER taking the lock,
so code_up can't change between snapshot and version-record.
- upsert (when changing an existing def) and delete take the lock across the
applied-check and the write; delete now rejects deleting an already-applied
migration (would orphan its _wm_migrations record), symmetric with upsert.
Both fail closed if the data table database is unreachable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): stack the out-of-order migration confirm above the DB editor preview
Creating a table on a migrations-enabled data table opened the DB table editor's
"Confirm running the following" preview modal, whose confirm triggers applyDdl,
which then asks for out-of-order confirmation. Both are ConfirmationModals with a
hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before
the editor), so it rendered behind the still-open preview modal.
Add an optional zIndexClass prop to ConfirmationModal (default z-[9999],
backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it
stacks on top.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c
This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private.
Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4
New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
The post-commit webhook reconcile in edit_git_sync_repository and
edit_git_sync_config wrote the whole pre-reconcile git_sync snapshot back
after the main save committed. A concurrent git-sync edit or poller status
write that landed in the gap could then be dropped by the stale snapshot.
Re-read the current row and merge only the reconciled webhook id/secret/error
for the repos the reconcile actually changed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
Both write endpoints (edit_git_sync_repository, edit_git_sync_config)
persisted caller-supplied auto_pull.webhook_id / webhook_secret /
webhook_error / last_synced_sha / last_pull_status when adding a repo or
newly enabling auto-pull, letting a client inject a webhook id/secret or
fake sync state. Strip those server-owned fields from the request up front;
existing repos re-derive them from the DB (carried over), new ones start
clean and the server (re)creates the webhook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
The poller scanned workspace_settings directly, so an archived (soft-deleted)
or renamed-away workspace — whose settings row persists — kept polling and
could enqueue a pull into a dead workspace. Join workspace and require
NOT deleted. The EE webhook receiver gets the same filter (ee ref bumped).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- Export (P1): strip the server-owned auto_pull state (webhook secret/id/error
+ synced sha + last pull status) from git_sync before it is written into an
export's settings.json for both settings formats. The HMAC webhook secret
must never leave the server (matching the GET-settings redaction), and a
re-imported workspace must not inherit another install's hook/sync state.
- Docs: the webhook receiver is a single per-workspace endpoint
/api/w/{workspace}/github_app/webhook (host-aware for managed + self-managed);
update the stale push_webhook/{id} and instance-global /api/github_app/webhook
references.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- edit_git_sync_config: also delete a repo's old webhook when the save drops
the repo OR clears its auto_pull. Webhook fields are only preserved onto a
Some auto_pull, so a save that present-but-clears a repo would otherwise
orphan its hook.
- GitSyncRepositoryCard: isFork now uses parent_workspace_id OR the wm-fork-
prefix (was AND), matching the backend/CLI rule, so prefix-less dev
workspaces are detected as forks and don't show the parent fork-PR toggle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
edit_git_sync_config preserved server-owned webhook fields but never
created or deleted the managed GitHub webhook, so enabling auto-pull
through the whole-config endpoint only polled, and disabling or removing
a repo left an orphan hook still delivering. Mirror the per-repository
endpoint: after the commit is durable, reconcile every saved repo's
webhook (sync_repo_webhook) and delete the hooks of repos the save
removed, including the clear-whole-config case. Addresses the Codex nit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
Removes the "Keep forks in sync with the tracked branch" toggle and its
fan-out. Pulling the tracked branch straight into every fork was the
inconsistent piece; the consistent model is per-fork branch sync (each
fork tracks its own wm-fork/** branch), which is a separate follow-up.
fork_open_prs is kept. Also tightens the fork toggle-section spacing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
migrate() and fix_flow_versioning_migration re-acquired a second connection from the pool while already holding one (the migrator's checked-out, advisory-locked connection). That deadlocks any backend limited to one connection at a time — connection-constrained managed Postgres, PgBouncer transaction pooling, or an embedded single-connection dev database. Route those housekeeping queries onto the already-held connection via a new CustomMigrator::connection() accessor. Fewer connections during migration and, for fix_flow_versioning, the existence check and write now run on the same advisory-locked connection. Default multi-connection behavior is unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add cosmetic dev/staging label for dev workspaces
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: prefill dev fork name and use a link to switch its label
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style: reword the dev/staging label link copy
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style: preview the dev/staging label as a badge in the switch link
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: show the dev/staging badge in the session diff drawer header
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): auto-derive cascade trigger edges from ducklake/s3 reads
Within a `// pipeline`, a read of a ducklake table or s3 object now
auto-wires its cascade trigger edge straight from the FROM clause, so
`// on <asset>` is only needed for edges inference can't see (dynamic SQL)
or to carry per-edge opts. Two opt-outs: `// mute <asset>` suppresses a
single derived edge (a lookup / SCD input read every run but not cascaded
on), and `// mute all` opts the script out of derivation entirely (back to
explicit-`// on`-only). Explicit `// on` still wins the dedup.
Scoped to ducklake + s3 reads; resource/datatable/volume stay explicit.
Read-write (RW) and write inputs are excluded so a self-referential
merge can't loop-trigger itself; ambiguous (None) access is skipped.
- parser: `mute` / `mute_all` in PipelineAnnotations (Rust + TS mirror)
- deploy: derive_pipeline_asset_trigger_refs → script_trigger rows
- frontend: resolveGraph mirrors derivation for the live edit-mode canvas
- tests: shared parity corpus + derive-helper units + resolveGraph overlays
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): mark auto-derived cascade edges with a persisted derived flag + "auto" badge
Persist script_trigger.derived (deploy: true for ducklake/s3-read derivation,
false for explicit // on) and return it from the asset-graph endpoint so the
canvas renders a Sparkles "auto" badge on auto-wired edges — the inference is
now visible on both the deployed graph and the live edit canvas, not just
implied. Dispatch (fetch_subscribers) ignores the flag, so a derived edge fires
identically to an explicit // on. Also copy derived in the workspace-clone
trigger copy, and backfill muteAssets/muteAll into two empty PipelineAnnotations
literals the base commit left stale (check:fast).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): derive cascade edge from effective (alt-fallback) asset access
derive_pipeline_asset_trigger_refs gated on the raw parser access_type, but the
persisted asset.usage_access_type and the frontend canvas both use
access_type.or(alt_access_type). An ambiguous parse with a manual read override
was persisted/drawn as a read yet derived no edge, so the auto edge silently
vanished on deploy. Gate on the effective access type for parity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): badge muted reads instead of auto-derived edges
Auto-derivation is the default now, so badging every derived cascade edge is
noise. Drop the "auto" badge and the persisted `script_trigger.derived` flag
(migration + insert param + graph field + clone copy) that only powered it, and
instead badge the exception: a ducklake/s3 asset a script reads but does NOT
cascade — `// mute <asset>` / `// mute all`. `computeMutedReadKeys` marks a
read-only ('r') supported read with no cascade trigger and no self-write; the
canvas renders a bell-off "muted" badge on that read edge.
Also fixes two review parity nits:
- TS `// on` parser now strips trailing `key=value` opts (e.g. `debounce=60s`)
like the Rust `split_trailing_kv_opts`, so the ref dedups against inference.
- A `// materialize` producer reading its own target is upgraded to `rw`
(deploy) / excluded via the materialize write refs (canvas), so it neither
self-cascades nor shows as a muted read.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): drop redundant // on for auto-derived reads; gate muted badge to pipeline scripts
- Templates no longer scaffold `// on <asset>` for a ducklake/s3 input the body
reads — the read auto-wires the cascade now that derivation is the default.
Kept for datatable/resource (not auto-derived) and native triggers. The
discoverability hint now mentions `// mute` (the newly relevant annotation).
- computeMutedReadKeys only badges reads by `// pipeline` scripts. A plain
script or flow reading a ducklake/s3 asset never had an auto trigger to
suppress, so it must render as ordinary lineage, not "muted" (Codex review).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): only drop template // on when the body actually reads the input
The redundant-`// on` removal assumed the generated body reads the ducklake/s3
input, but postgres/bash/generic bodies (and `data_upload`, which reads the
picker file) ignore `input` — dropping `// on` there left the asset-created
script with no cascade at all. Gate the drop on READS_INPUT_LANGS
(bun/deno/python/duckdb) so non-reading templates keep the explicit trigger.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(auth): add runtime NO_AUTH mode for authentication bypass
Adds a runtime `NO_AUTH` env flag that makes every request resolve as the
`admin@windmill.dev` superadmin with no login required, so self-hosted
deployments can front Windmill with their own authenticating gateway
without building a dedicated `oss` (compile-time `no_auth`) binary.
- `NO_AUTH` is honored in any build but is force-disabled when
`CLOUD_HOSTED` is set, so the managed cloud always enforces real auth.
- The existing compile-time `no_auth` feature keeps its always-on behavior
(`cfg!(feature = "no_auth") || *NO_AUTH`), so `oss` builds are unchanged.
- `Tokened` now yields a synthetic token in no-auth mode so handlers that
require it (e.g. global_whoami, called by the frontend on load) resolve.
- A loud startup banner warns when the mode is on; `HIDE_NO_AUTH_BANNER`
silences it once the operator has deliberately deployed behind a gateway.
Fixes WIN-2131
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(auth): dismissable NO_AUTH warning banner via global setting
Replaces the HIDE_NO_AUTH_BANNER env flag with a UI warning banner that
can be permanently dismissed for all users from within the running
instance (not exposed in instance settings).
- New `no_auth_banner_dismissed` global setting, only ever written by
dismissing the banner itself.
- `GET /api/settings/no_auth_banner` returns whether to show the banner
(true only when NO_AUTH is active and it hasn't been dismissed).
- NoAuthBanner.svelte renders a top-of-app warning in NO_AUTH mode; its
dismiss button opens a confirmation modal, then writes the global
setting via the existing setGlobal endpoint so it stays hidden for
everyone.
- The server still logs the startup NO_AUTH warning unconditionally.
Fixes WIN-2131
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(auth): resolve NO_AUTH in AuthCache so all_runnables works
Codex/Pi review flagged that `/api/users/all_runnables` still failed in
NO_AUTH mode: `get_all_runnables` extracts `Tokened` and re-validates the
request token per workspace via `AuthCache::get_authed`, which rejected the
fabricated `"no_auth"` token (no matching DB row) with a 400.
Short-circuit `AuthCache::get_opt_job_authed` (the resolver behind
`get_authed`) to the admin superadmin in no-auth mode, so any direct cache
caller resolves without a real token. Single-source the mode check and the
synthetic identity via `is_no_auth()` / `no_auth_admin_authed()` and reuse
them across the extractor, resolver, and login paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* revert(auth): drop the NO_AUTH dismissable UI banner
The in-app banner added a GET /api/settings/no_auth_banner request to every
instance load for little benefit. The startup log warning already surfaces
that auth is bypassed to operators, so drop the banner, its endpoint, and the
no_auth_banner_dismissed global setting entirely.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codex nits:
- edit_git_sync_config saved the client config verbatim, so the webhook_secret
redacted from the GET response would be dropped (breaking delivery). Preserve
server-owned auto-pull state (webhook id/secret, synced sha, last status) per
repo from the existing settings, matching edit_git_sync_repository.
- addSyncRepository (the visible add path) didn't set the auto_pull default, so
new sync repos added from the UI came up with auto-deploy off. Match
addRepository's default (webhook + polling fallback).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
Codex nit: edit_git_sync_repository ran sync_repo_webhook before the transaction
committed. The rollback only covered created hooks, but sync_repo_webhook also
deletes a hook on disable/switch-to-polling — a commit failure then left the DB
with a webhook_id whose hook was already gone (and it wouldn't be recreated).
Save + commit first, then reconcile the webhook against the durable config and
persist any hook id/secret change (best-effort). Bump EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
Codex re-review nits:
- delete_git_sync_repository deleted the webhook before the settings transaction
committed; a failed save would then leave the repo pointing at a hook that no
longer exists (sync_repo_webhook treats a set webhook_id as live and won't
recreate it). Capture the hook id, commit the DB removal, then delete the hook.
- Reword a fork-copy comment to drop drafting-history wording per AGENTS.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
- Deleting a git-sync repository now tears down its managed GitHub webhook
(deletion bypassed the sync_repo_webhook lifecycle, orphaning the hook so
GitHub kept delivering to the instance).
- Worker completion hook rolls back the optimistic auto-pull sha on job failure
(OSS side of the EE change) + caches the new marker query. Bump EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* wip: partial work before earlyoom-recovery relaunch
* fix(pipelines): scaffold the strftime {partition} filter idiom (frontend-only)
The DuckDB materialize scaffold and the AI pipeline prompt now teach the
grain-agnostic `WHERE strftime(<ts_col>, '<fmt>') = {partition}` filter instead
of the naive `= TIMESTAMP {partition}` cast. `{partition}` substitutes to the
partition IDENTITY string (`2026-07-05T23`, `2026-W27`, `2026-07`), which is not
a valid DuckDB TIMESTAMP literal for any non-daily grain — so the naive form
raises a `Conversion Error` for hourly/weekly/monthly (only daily parses).
Adds a frontend unit test asserting the hourly scaffold emits the strftime
idiom (`%Y-%m-%dT%H`) for every grain and never scaffolds the naive TIMESTAMP
cast as executable SQL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): scope strftime partition idiom to time grains
Review nit: `dynamic` partitioning's identity is a caller-supplied key, not a
timestamp, so `strftime` doesn't apply. Scope the scaffold + AI prompt claim to
time grains and add a `dynamic` example that filters on the user's own key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): wm_partition macro for grain-agnostic partition filters
The materialize runtime now injects a `wm_partition(ts)` temp macro as the first
setup statement of a time-partitioned script, so filtering the source to the
active slice is one grain-agnostic line — `WHERE wm_partition(<ts_col>) =
{partition}` — instead of a hand-written `strftime` format the author must keep
in lockstep with the resolver, or the `= TIMESTAMP {partition}` cast that only
parses for daily and Conversion-Errors for hourly/weekly/monthly.
The macro's format comes from `PartitionKind::default_time_format` in
windmill-parser, the same source the EE resolver reads to stamp the `{partition}`
identity, so the two can't drift. `dynamic` partitions get no macro (their
identity is a caller-supplied key → `WHERE <key_col> = {partition}`).
Replaces the earlier 9-line strftime comment block in the scaffold with the
single macro line; AI pipeline prompt and design doc updated to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(pipelines): verify wm_partition strftime parity vs chrono through real DuckDB
Runs the bundled DuckDB engine in-memory and asserts strftime renders every
grain format (daily/hourly/weekly `%G-W%V`/monthly) byte-for-byte identically to
chrono — the engine the resolver uses to stamp the `{partition}` identity —
across ISO-week year boundaries (2027-01-01 → 2026-W53 etc.). Also proves the
injected `wm_partition` macro buckets the whole slice and that the naive
`TIMESTAMP '<weekly|monthly identity>'` cast Conversion-Errors.
Closes the one cross-engine assumption the pure-Rust/frontend tests couldn't
reach (flagged by CI review for weekly ISO-week rendering).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 0de2412ff0734b11e12ba378c9bcc373ff9ae800
This commit updates the EE repository reference after PR #649 was merged in windmill-ee-private.
Previous ee-repo-ref: ad6c6685689d7741058e7d2c9ecbe95d982e6268
New ee-repo-ref: 0de2412ff0734b11e12ba378c9bcc373ff9ae800
Automated by sync-ee-ref workflow.
* fix(pipelines): classify CREATE TEMP MACRO as a DuckDB prepare-path setup statement
The FFI prepare/diagnostics pass only EXECUTES statements recognized by
is_setup_statement (ATTACH/USE/INSTALL/…); everything else is merely prepared.
`CREATE [OR REPLACE] TEMP MACRO` wasn't recognized, so on a `-- prepare` run of a
partitioned materialize the injected `wm_partition` macro was never created on
the connection, and the later generated `CREATE TABLE … SELECT … WHERE
wm_partition(...)` failed to bind ("function does not exist"). The same latent
gap affected the workspace-macro splicer, which injects TEMP MACRO blocks too.
Classify CREATE [OR REPLACE] TEMP|TEMPORARY MACRO as setup so it's executed
before dependent blocks and excluded from the PrepareQueryResult count
(persistent CREATE MACRO stays a user statement). Adds a prepare-path test that
fails without the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
hugocasaandClaude Opus 4.8
Ruben Fiszelandrubenfiszel
Diego Imbert