feat: add SQL migrations for data tables (#9693)

* feat: add datatable_migrations table

* feat: add route to run datatable migrations

* feat: sync datatable migrations as .up.sql/.down.sql files

* feat: add datatable migrate up/down commands and post-push run prompt

* feat: add datatable migrate new command to scaffold migrations

* feat: add datatable migrations management UI

* feat: prompt to create migration on DDL in datatable SQL editors

* feat: support running a single specific datatable migration

* feat: view migration content, run single migration, fix stacked modal

* feat: per-row revert button with out-of-order warning

* fix: avoid migrations list flicker on refresh after an action

* feat: generate initial datatable migration via pg_dump

* fix: surface datatable migration API error details in toasts

* fix: revert created migration if create-and-run fails to run

* fix: include postgres error detail in migration run/rollback failures

* feat: sync datatable migrations as files via the workspace export

* refactor: move datatable migrations to migrations/datatable/ path

* fix: drop redundant datatable_migration label in sync output

* fix: exclude datatable migration sql files from script metadata generation

* feat: run datatable migrations as user-permissioned labeled jobs

* feat: reject invalid datatable migrations on sync push

* feat: datatable migrate up/down default to all datatables, --datatable to target one

* fix: surface postgres error detail when datatable migrations fail to run

* chore: regenerate CLI docs for datatable migrate commands

* feat: default new datatable migration to a BEGIN/END transaction template

* fix: validate datatable migration name and datatable at the API boundary

* fix: ensure detected DDL ends with semicolon when wrapped in transaction

* fix: re-prompt instead of stripping DDL when new-migration modal is cancelled

* feat: refresh datatable schema after running a migration from the SQL REPL

* feat: record db manager DDL on data tables as migrations

* feat: make datatable migrations opt-in per data table

* fix: make migration view editor read-only so its code can scroll

* fix: don't re-prompt DDL guard when creating a migration without running

* feat: generate down migrations for db manager DDL (postgres)

* fix: correct down migration for db manager alters (no double-wrap, serial)

* feat: explain migrations purpose with a tooltip in the migrations modal

* compare paeg

* feat: add datatable_migration kind to workspace diff pipeline

* chore: point ee-repo-ref at datatable_migration git-sync companion

* fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests

* feat: deploy and run datatable migrations on workspace merge

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Refactor + handle datatable setting delete/rename

* refactor: move datatable migration rename/delete cascade into module

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(db-manager): add Migrations button to top bar, make Refresh icon-only

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* BEGIN/END placeholder in down migration

* feat: autofocus migration name input and flag it red when empty

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* border nits

* refresh db manager schema on migrations

* BEGIN/END scaffold in CLI

* feat(cli): push local datatable migrations before running on migrate up

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: flag invalid migration name with red border, not just empty

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: drop random slug from auto-generated migration names

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: offer revert-and-delete when deleting an installed migration

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: record fork merge as a migration when target datatable opts in

* nit

* clone migrations on fork

* windmill-utils-internal

* fix(datatable-migrations): serialize run/rollback with a per-db advisory lock

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(db-manager): fail closed when migrations-status check errors on DDL apply

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: fix generate_initial migration ordering comment to match code

* chore(datatable-migrations): remove unused update_datatable_migrations endpoint

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: run DDL migration guard on the script editor Test button

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* split

* ee-repo-ref

* chore(frontend): sync package-lock with package.json (@emnapi deps)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(datatable-migrations): never resolve instance credentials into migration job args

datatable_database_arg eagerly resolved instance data-table credentials
(including the shared instance-wide Postgres password) and passed them as the
migration job's plaintext `database` arg, landing in v2_job.args. Since the
run route has no admin gate, a non-admin could run a migration and read
args.database to recover the password, granting cross-workspace psql access to
all instance data-table DBs.

Pass a `datatable://<name>` reference for both resource-backed and instance
data tables instead; the pg executor already resolves it to real credentials
server-side at run time, so nothing sensitive is ever stored in the job args.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* nit

* fix: handle dollar-quoting and comments when splitting SQL statements

* feat: deploy datatable migrations on merge with explicit opt-in error

* fix(frontend): sync package-lock with npm 11 peer-dep resolution

npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from
lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as
peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly
1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the
highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree
needs both versions; the committed lock only had 1.10.0.

Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for
the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* nit npm publish

* fix: fail closed on migrations-status error in fork schema merge

* nit CI emnapi/core version

* prevent initial_datatable_migration if migrations already exist

* fix(datatable-migrations): validate persisted data table names as path segments

edit_datatable_config only validated rename segments, not the actual
settings.datatables keys, so a data table could be saved directly under a name
like '..' or one containing '/'. Since new tables default to
migrations_enabled = true, generate_initial_datatable_migration would then
insert a migration row and the sync export would build
migrations/datatable/<name>/... paths from that name, producing malformed or
directory-escaping export paths.

Validate every persisted data table name in edit_datatable_config (alongside
the existing rename checks) and add validate_datatable_path_segment to
generate_initial_datatable_migration for defense in depth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: scope datatable _wm_migrations by data table and cascade renames/deletes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(system_prompts): resolve nested local command groups in CLI docs generator

The CLI docs generator anchored on the first `new Command()` in a file and
never resolved locally-defined command groups passed as
`.command("name", localCmd)`. For datatable this flattened the nested
`migrate` group: it emitted `datatable new/up/down` plus a bare
`datatable migrate`, and mislabeled the datatable command with the migrate
group's description. jobs was broken the same way (its description was pull's,
and pull/push rendered empty).

Anchor block extraction on the `export default`ed command, recurse into
locally-defined `const x = new Command()` groups mounted as subcommands, and
render nested sub-subcommands. Regenerated docs now show
`datatable migrate new/up/down` and `jobs pull/push` with their real
options.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: drop unreleased _wm_migrations legacy-upgrade handling

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: return datatable migration SQL from getItemValue for the diff drawer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* nit

* nit

* fix: handle datatable migration renames on push and dedupe timestamps

* fix: reject rewriting an already-applied datatable migration on upsert

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries

Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi
declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved
lockfile entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): datatable migrate up/down default to main datatable, not all

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: fail closed when applied status unreadable on datatable migration rewrite

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: surface full error detail in Database Manager DDL/query errors

* "See migration" button in the toast

* feat: add Enter shortcut to Create-a-migration in the DDL guard

* fix(frontend): warn before running a newly-created datatable migration out of order

The row-level Run action warns when earlier migrations are still pending, but
the create-and-run paths ran a just-created migration with `only` directly,
applying it ahead of older pending migrations without that confirmation.

Reuse the same "Run migration out of order" confirmation across all
create-and-run paths via a shared helper (datatableMigrationUtils):
- NewDataTableMigrationModal "Create and run" (and the DDL guard path)
- DatatableSchemaDiff fork→parent merge
- dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a
  MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a
  silent cancel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: keep renamed datatable migrations visible in compare view

* fix: record per-migration deployment on datatable migrations disable

* fix(cli): run deployed datatable migrations after workspace merge

The merge command upserted datatable_migration definitions into the target
workspace and reported the item as successfully deployed, but never ran the
migrations. For forked datatables backed by separate databases, this left the
target schema unchanged until someone manually ran `wmill datatable migrate up`,
while the CLI reported a successful merge.

Collect the datatable migrations deployed (not deleted) into the target and,
after the deploy loop, offer to run them via the existing offerToRunNewMigrations
helper — the same post-deploy run prompt the push/sync path uses (interactive
only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export
parseDatatableMigrationDeployPath so the merge path can parse the deployed items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(backend): serialize datatable migration edits/deletes with the run lock

A migration run snapshots a migration's code_up from datatable_migrations and
only records its version in the data table's _wm_migrations after the job
succeeds. upsert_datatable_migration checked _wm_migrations before allowing an
edit but took no lock, so a concurrent edit could read "not applied yet",
rewrite code_up/code_down, and then the in-flight run would record the version
for the old SQL — leaving _wm_migrations pointing at SQL that was never applied
(migrate up then skips it; rollback runs a down that doesn't match).

Serialize definition rewrites and deletes with the same per-database advisory
lock the run/rollback paths use:
- Factor the connect+advisory-lock into lock_datatable_migration_runs and the
  applied-versions read into read_applied_versions_on_client.
- run_datatable_migrations now snapshots the definitions AFTER taking the lock,
  so code_up can't change between snapshot and version-record.
- upsert (when changing an existing def) and delete take the lock across the
  applied-check and the write; delete now rejects deleting an already-applied
  migration (would orphan its _wm_migrations record), symmetric with upsert.
  Both fail closed if the data table database is unreachable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): stack the out-of-order migration confirm above the DB editor preview

Creating a table on a migrations-enabled data table opened the DB table editor's
"Confirm running the following" preview modal, whose confirm triggers applyDdl,
which then asks for out-of-order confirmation. Both are ConfirmationModals with a
hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before
the editor), so it rendered behind the still-open preview modal.

Add an optional zIndexClass prop to ConfirmationModal (default z-[9999],
backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it
stacks on top.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c

This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private.

Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4

New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

cli/src/commands/datatable/datatable.ts

@@ -9,6 +9,13 @@ import { GlobalOptions } from "../../types.ts";
 import { runCatalogQuery } from "../../utils/catalog.ts";
 import { psql as psqlDatatable } from "./psql.ts";
 import { serve as serveDatatable } from "./serve.ts";
+import {
+  createMigration,
+  pushLocalMigrations,
+  rollbackMigrations,
+  runMigrations,
+  validateLocalMigrations,
+} from "../datatable_migrations.ts";
 
 const DEFAULT_DATATABLE_NAME = "main";
 
@@ -41,6 +48,69 @@ async function run(
   await runCatalogQuery(opts, "datatable", name, sql);
 }
 
+function migrateNew(
+  opts: GlobalOptions & { datatable?: string },
+  name: string,
+) {
+  createMigration(opts.datatable ?? DEFAULT_DATATABLE_NAME, name);
+}
+
+async function migrateUp(opts: GlobalOptions & { datatable?: string }) {
+  const workspace = await resolveWorkspace(opts);
+  await requireLogin(opts);
+  const dt = opts.datatable ?? DEFAULT_DATATABLE_NAME;
+  // Reject malformed local migrations (duplicate timestamps, orphan downs) before
+  // pushing — the same check `wmill sync push` runs — so a duplicate timestamp
+  // can't silently overwrite one migration on upsert.
+  const errors = validateLocalMigrations(new Set([dt]));
+  if (errors.length > 0) {
+    log.error(
+      "Invalid datatable migrations, aborting:\n" +
+        errors.map((e) => `  - ${e}`).join("\n"),
+    );
+    process.exit(1);
+  }
+  // Push any locally-created/edited migration files first (without running
+  // them), so `migrate up` works even before a `wmill sync push`.
+  await pushLocalMigrations(workspace.workspaceId, dt);
+  await runMigrations(workspace.workspaceId, dt);
+}
+
+async function migrateDown(opts: GlobalOptions & { datatable?: string }) {
+  const workspace = await resolveWorkspace(opts);
+  await requireLogin(opts);
+  const dt = opts.datatable ?? DEFAULT_DATATABLE_NAME;
+  await rollbackMigrations(workspace.workspaceId, dt);
+}
+
+const migrateCommand = new Command()
+  .description("manage datatable migrations")
+  .command("new", "scaffold a new migration (.up.sql / .down.sql files)")
+  .arguments("<name:string>")
+  .option(
+    "-d --datatable <datatable:string>",
+    "Target datatable (default: main)",
+  )
+  .action(migrateNew as any)
+  .command(
+    "up",
+    "apply all pending migrations to the main datatable (or one via --datatable)",
+  )
+  .option(
+    "-d --datatable <datatable:string>",
+    "Target datatable (default: main)",
+  )
+  .action(migrateUp as any)
+  .command(
+    "down",
+    "roll back the most recent migration on the main datatable (or one via --datatable)",
+  )
+  .option(
+    "-d --datatable <datatable:string>",
+    "Target datatable (default: main)",
+  )
+  .action(migrateDown as any);
+
 async function create(
   opts: GlobalOptions & { resource?: string; force?: boolean },
   name?: string,
@@ -124,6 +194,7 @@ const command = new Command()
     "Output only the final result as JSON. Useful for scripting.",
   )
   .action(run as any)
+  .command("migrate", migrateCommand)
   .command(
     "create",
     "register a datatable database in the workspace (default: instance-backed 'main') so scripts can use datatable://<name>",

cli/src/commands/datatable_migrations.ts

@@ -0,0 +1,340 @@
+import * as fs from "node:fs";
+import * as path from "node:path";
+import * as log from "../core/log.ts";
+import { colors } from "@cliffy/ansi/colors";
+import * as wmill from "../../gen/services.gen.ts";
+import { readTextFile } from "../utils/utils.ts";
+import { Confirm } from "@cliffy/prompt/confirm";
+
+// Migrations live under <cwd>/migrations/datatable/<datatable>/, one folder per
+// target data table, as `<timestamp>_<name>.up.sql` (and optional `.down.sql`).
+// They are synced as ordinary workspace files (see the workspace tarball export
+// and the `datatable_migration` handling in sync.ts); this module only holds the
+// `wmill datatable migrate` command helpers and the per-file push primitive.
+const MIGRATIONS_DIR = path.join("migrations", "datatable");
+
+// Migration names map directly onto file names and the DB `name` column.
+const MIGRATION_NAME_RE = /^[a-zA-Z0-9_-]+$/;
+
+/** Current UTC time as a YYYYMMDDHHMMSS migration version. */
+function migrationTimestamp(): string {
+  const d = new Date();
+  const p = (n: number) => String(n).padStart(2, "0");
+  return (
+    `${d.getUTCFullYear()}${p(d.getUTCMonth() + 1)}${p(d.getUTCDate())}` +
+    `${p(d.getUTCHours())}${p(d.getUTCMinutes())}${p(d.getUTCSeconds())}`
+  );
+}
+
+/**
+ * A migration version unique within a data table folder: the current UTC
+ * timestamp bumped past any existing version, so two migrations scaffolded in
+ * the same second don't collide on the `(datatable, timestamp)` identity used to
+ * upsert them.
+ */
+function nextMigrationTimestamp(dir: string): string {
+  const now = Number(migrationTimestamp());
+  let max = 0;
+  if (fs.existsSync(dir)) {
+    for (const file of fs.readdirSync(dir)) {
+      const m = file.match(/^(\d+)_.*\.(up|down)\.sql$/);
+      if (m) max = Math.max(max, Number(m[1]));
+    }
+  }
+  return String(max >= now ? max + 1 : now);
+}
+
+/**
+ * Scaffold a new migration under migrations/datatable/<datatable>/ as empty
+ * `<timestamp>_<name>.up.sql` and `.down.sql` files. Purely local — no network.
+ */
+export function createMigration(datatable: string, name: string): void {
+  if (!MIGRATION_NAME_RE.test(name)) {
+    throw new Error(
+      `Invalid migration name '${name}': use only letters, digits, '_' and '-'`,
+    );
+  }
+  const dir = path.join(process.cwd(), MIGRATIONS_DIR, datatable);
+  fs.mkdirSync(dir, { recursive: true });
+
+  const timestamp = nextMigrationTimestamp(dir);
+  const base = `${timestamp}_${name}`;
+  const up = path.join(dir, `${base}.up.sql`);
+  const down = path.join(dir, `${base}.down.sql`);
+  // Frame the body in an explicit transaction so it applies atomically, matching
+  // the template the UI's "New migration" modal seeds.
+  const template = (direction: string) =>
+    `-- ${direction} migration: ${name}\nBEGIN;\n\n-- Add your migration here\n\nEND;\n`;
+  fs.writeFileSync(up, template("up"), "utf-8");
+  fs.writeFileSync(down, template("down"), "utf-8");
+
+  log.info(
+    colors.green(`Created migration ${base} in ${MIGRATIONS_DIR}/${datatable}/`),
+  );
+  for (const f of [up, down]) {
+    log.info(colors.gray(`  ${path.relative(process.cwd(), f)}`));
+  }
+}
+
+/**
+ * Apply the workspace's pending migrations to a data table (forwards migrations
+ * recorded in `_wm_migrations`). Mirrors `wmill datatable migrate up`.
+ */
+export async function runMigrations(
+  workspace: string,
+  datatableName: string,
+): Promise<void> {
+  const result = await wmill.runDatatableMigrations({
+    workspace,
+    datatableName,
+  });
+  const applied = result.applied ?? [];
+  if (applied.length === 0) {
+    log.info(colors.gray(`No pending migrations to run on '${datatableName}'`));
+    return;
+  }
+  log.info(
+    colors.green(`Applied ${applied.length} migration(s) to '${datatableName}':`),
+  );
+  for (const m of applied) {
+    log.info(colors.gray(`  ${m.version} ${m.name}`));
+  }
+}
+
+/**
+ * Roll back the most recently applied migration on a data table (one step).
+ * Mirrors `wmill datatable migrate down`.
+ */
+export async function rollbackMigrations(
+  workspace: string,
+  datatableName: string,
+): Promise<void> {
+  const result = await wmill.rollbackDatatableMigrations({
+    workspace,
+    datatableName,
+  });
+  const rolledBack = result.rolled_back ?? [];
+  if (rolledBack.length === 0) {
+    log.info(
+      colors.gray(`No applied migrations to roll back on '${datatableName}'`),
+    );
+    return;
+  }
+  for (const m of rolledBack) {
+    log.info(
+      colors.green(`Rolled back migration ${m.version} ${m.name} on '${datatableName}'`),
+    );
+  }
+}
+
+/**
+ * Validate the on-disk migration files for the given data tables (or all of
+ * them when `datatables` is omitted). Returns a list of human-readable problems;
+ * an empty list means the migrations are well-formed. Two states are invalid:
+ *  - two up (or two down) files sharing the same timestamp, which collide on the
+ *    `(datatable, timestamp)` identity used to upsert; and
+ *  - a `.down.sql` with no matching `.up.sql` (an up file is mandatory).
+ */
+export function validateLocalMigrations(datatables?: Set<string>): string[] {
+  const errors: string[] = [];
+  const root = path.join(process.cwd(), MIGRATIONS_DIR);
+  if (!fs.existsSync(root)) return errors;
+
+  for (const datatable of fs.readdirSync(root)) {
+    if (datatables && !datatables.has(datatable)) continue;
+    const dtDir = path.join(root, datatable);
+    if (!fs.statSync(dtDir).isDirectory()) continue;
+
+    const upNamesByTs = new Map<number, string[]>();
+    const downNamesByTs = new Map<number, string[]>();
+    const upBases = new Set<string>();
+    const downBases: { ts: number; name: string }[] = [];
+
+    for (const file of fs.readdirSync(dtDir)) {
+      const m = file.match(/^(\d+)_(.*)\.(up|down)\.sql$/);
+      if (!m) continue;
+      const ts = Number(m[1]);
+      const name = m[2];
+      if (m[3] === "up") {
+        (upNamesByTs.get(ts) ?? upNamesByTs.set(ts, []).get(ts)!).push(name);
+        upBases.add(`${ts}_${name}`);
+      } else {
+        (downNamesByTs.get(ts) ?? downNamesByTs.set(ts, []).get(ts)!).push(name);
+        downBases.push({ ts, name });
+      }
+    }
+
+    for (const [ts, names] of upNamesByTs) {
+      if (names.length > 1) {
+        errors.push(
+          `${datatable}: ${names.length} up migrations share timestamp ${ts} (${names.join(", ")})`,
+        );
+      }
+    }
+    for (const [ts, names] of downNamesByTs) {
+      if (names.length > 1) {
+        errors.push(
+          `${datatable}: ${names.length} down migrations share timestamp ${ts} (${names.join(", ")})`,
+        );
+      }
+    }
+    for (const d of downBases) {
+      if (!upBases.has(`${d.ts}_${d.name}`)) {
+        errors.push(
+          `${datatable}: ${d.ts}_${d.name}.down.sql has no matching ${d.ts}_${d.name}.up.sql`,
+        );
+      }
+    }
+  }
+
+  return errors;
+}
+
+/**
+ * Sync a single migration to the workspace based on the current on-disk state of
+ * its `<datatable>/<timestamp>_<name>.up.sql` file: upsert it when the up file
+ * exists, otherwise delete it. Called by `wmill sync push` for each changed
+ * `datatable_migration` file.
+ */
+export async function pushMigrationFromDisk(
+  workspace: string,
+  m: { datatable: string; timestamp: number },
+): Promise<void> {
+  const dir = path.join(process.cwd(), MIGRATIONS_DIR, m.datatable);
+  // Find the up file for this timestamp regardless of its name segment. A rename
+  // (`123_old.up.sql` -> `123_new.up.sql`) keeps the (datatable, timestamp)
+  // identity but changes the name; the diff sorter may process the deleted old
+  // path before the added new one, so keying off the passed name would delete
+  // the record. Scanning by timestamp upserts the surviving file instead.
+  const upFile = fs.existsSync(dir)
+    ? fs.readdirSync(dir).find((f) => {
+        const parsed = f.match(/^(\d+)_(.*)\.up\.sql$/);
+        return parsed !== null && Number(parsed[1]) === m.timestamp;
+      })
+    : undefined;
+
+  if (upFile === undefined) {
+    log.info(colors.red(`Deleting datatable_migration ${m.datatable}/${m.timestamp}`));
+    await wmill.deleteDatatableMigration({
+      workspace,
+      datatableName: m.datatable,
+      timestamp: m.timestamp,
+    });
+    return;
+  }
+
+  const name = upFile.match(/^(\d+)_(.*)\.up\.sql$/)![2];
+  const base = `${m.timestamp}_${name}`;
+  const code_up = await readTextFile(path.join(dir, upFile));
+  const downPath = path.join(dir, `${base}.down.sql`);
+  const code_down = fs.existsSync(downPath) ? await readTextFile(downPath) : undefined;
+
+  log.info(colors.green(`Pushing datatable_migration ${m.datatable}/${base}`));
+  await wmill.upsertDatatableMigration({
+    workspace,
+    datatableName: m.datatable,
+    requestBody: {
+      timestamp: m.timestamp,
+      name,
+      code_up,
+      ...(code_down !== undefined ? { code_down } : {}),
+    },
+  });
+}
+
+/**
+ * Upsert the on-disk migrations of a data table to the workspace, so a freshly
+ * created migration file works with `wmill datatable migrate up` even without a
+ * prior `wmill sync push`. Pushes only migrations that are new or edited
+ * (compared against the workspace's current definitions); it never deletes
+ * remote migrations absent on disk and never touches other item kinds.
+ */
+export async function pushLocalMigrations(
+  workspace: string,
+  datatableName: string,
+): Promise<void> {
+  const dir = path.join(process.cwd(), MIGRATIONS_DIR, datatableName);
+  if (!fs.existsSync(dir)) return;
+
+  // Local migrations are identified by their `.up.sql` file (the up file is
+  // mandatory); this deliberately ignores files that were only deleted locally.
+  const local: { timestamp: number; name: string }[] = [];
+  for (const file of fs.readdirSync(dir)) {
+    const m = file.match(/^(\d+)_(.*)\.up\.sql$/);
+    if (m) local.push({ timestamp: Number(m[1]), name: m[2] });
+  }
+  if (local.length === 0) return;
+
+  const remote = await wmill.listDatatableMigrations({ workspace });
+  const remoteByTs = new Map(
+    remote
+      .filter((r) => r.datatable === datatableName)
+      .map((r) => [r.timestamp, r] as const),
+  );
+
+  for (const { timestamp, name } of local) {
+    const base = `${timestamp}_${name}`;
+    const code_up = await readTextFile(path.join(dir, `${base}.up.sql`));
+    const downPath = path.join(dir, `${base}.down.sql`);
+    const code_down = fs.existsSync(downPath)
+      ? await readTextFile(downPath)
+      : undefined;
+
+    const r = remoteByTs.get(timestamp);
+    const unchanged =
+      r !== undefined &&
+      r.name === name &&
+      r.code_up === code_up &&
+      (r.code_down ?? undefined) === code_down;
+    if (unchanged) continue;
+
+    log.info(colors.green(`Pushing datatable_migration ${datatableName}/${base}`));
+    await wmill.upsertDatatableMigration({
+      workspace,
+      datatableName,
+      requestBody: {
+        timestamp,
+        name,
+        code_up,
+        ...(code_down !== undefined ? { code_down } : {}),
+      },
+    });
+  }
+}
+
+/**
+ * After a push that introduced new migrations, list them and (interactively)
+ * offer to run them, equivalent to `wmill datatable migrate up` on each affected
+ * data table.
+ */
+export async function offerToRunNewMigrations(
+  workspace: string,
+  newMigrations: { datatable: string; timestamp: number; name: string }[],
+  opts?: { yes?: boolean; jsonOutput?: boolean },
+): Promise<void> {
+  if (newMigrations.length === 0) return;
+
+  log.info(colors.green("New migrations were pushed:"));
+  for (const m of newMigrations) {
+    log.info(colors.gray(`  ${m.datatable}: ${m.timestamp} ${m.name}`));
+  }
+
+  // Running migrations mutates the data tables, so skip the prompt in
+  // non-interactive contexts (--yes, --json, no TTY).
+  const interactive = !opts?.jsonOutput && !opts?.yes && !!process.stdin.isTTY;
+  if (!interactive) {
+    return;
+  }
+
+  const shouldRun = await Confirm.prompt({
+    message: "New migrations were pushed, run them?",
+    default: false,
+  });
+  if (!shouldRun) {
+    return;
+  }
+
+  for (const datatable of new Set(newMigrations.map((m) => m.datatable))) {
+    await runMigrations(workspace, datatable);
+  }
+}

cli/src/commands/generate-metadata/generate-metadata.ts

@@ -2,7 +2,7 @@ import { Command } from "@cliffy/command";
 import { Confirm } from "@cliffy/prompt/confirm";
 import { colors } from "@cliffy/ansi/colors";
 import { sep as SEP } from "node:path";
-import { GlobalOptions } from "../../types.ts";
+import { GlobalOptions, isDatatableMigrationPath } from "../../types.ts";
 import { SyncOptions, mergeConfigWithConfigFile } from "../../core/conf.ts";
 import { resolveWorkspace } from "../../core/context.ts";
 import { requireLogin } from "../../core/auth.ts";
@@ -54,6 +54,8 @@ async function walkLocalScripts(
       (!isD && !exts.some((ext) => p.endsWith(ext))) ||
       ignore(p, isD) ||
       isFolderResourcePathAnyFormat(p) ||
+      // Datatable migration `.sql` files aren't Windmill scripts.
+      isDatatableMigrationPath(p) ||
       (isScriptModulePath(p) && !isModuleEntryPoint(p)),
     false,
     {},
@@ -221,6 +223,8 @@ function categorizeLocalFiles(
     } else if (
       exts.some((ext) => p.endsWith(ext)) &&
       !isFolderResourcePathAnyFormat(p) &&
+      // Datatable migration `.sql` files aren't Windmill scripts.
+      !isDatatableMigrationPath(p) &&
       !(isScriptModulePath(p) && !isModuleEntryPoint(p))
     ) {
       scripts.push(p);

cli/src/commands/sync/sync.ts

@@ -23,10 +23,17 @@ import {
   showDiff,
   extractNativeTriggerInfo,
   redactEncryptionKey,
+  isDatatableMigrationPath,
+  parseDatatableMigrationPath,
 } from "../../types.ts";
 import { downloadZip } from "./pull.ts";
 import { runLint, printReport, checkMissingLocks } from "../lint/lint.ts";
 import { pullSharedUi, pushSharedUi } from "../shared_ui.ts";
+import {
+  pushMigrationFromDisk,
+  offerToRunNewMigrations,
+  validateLocalMigrations,
+} from "../datatable_migrations.ts";
 
 import {
   exts,
@@ -2477,7 +2484,8 @@ const isNotWmillFile = (p: string, isDirectory: boolean) => {
       !p.startsWith("g" + SEP) &&
       !p.startsWith("users" + SEP) &&
       !p.startsWith("groups" + SEP) &&
-      !p.startsWith("dependencies" + SEP)
+      !p.startsWith("dependencies" + SEP) &&
+      !p.startsWith("migrations" + SEP)
     );
   }
 
@@ -2488,6 +2496,11 @@ const isNotWmillFile = (p: string, isDirectory: boolean) => {
 
   try {
     const typ = getTypeStrFromPath(p);
+    // Datatable migrations live under migrations/datatable/<datatable>/, outside
+    // the u/f/g namespaces, but are valid wmill files.
+    if (typ == "datatable_migration") {
+      return false;
+    }
     if (
       typ == "resource-type" ||
       typ == "settings" ||
@@ -2519,7 +2532,8 @@ export const isWhitelisted = (p: string) => {
     p == "ui" ||
     p == "users" ||
     p == "groups" ||
-    p == "dependencies"
+    p == "dependencies" ||
+    p == "migrations"
   );
 };
 
@@ -2614,6 +2628,11 @@ interface ChangeTracker {
 }
 
 async function addToChangedIfNotExists(p: string, tracker: ChangeTracker) {
+  // Datatable migration .sql files are not scripts; they're synced via the
+  // dedicated datatable_migration handler in the push loop.
+  if (isDatatableMigrationPath(p)) {
+    return;
+  }
   const isScript = exts.some((e) => p.endsWith(e)) && !isFileResource(p) && !isFilesetResource(p);
   if (isScript) {
     if (isFlowPath(p)) {
@@ -3119,7 +3138,7 @@ export async function pull(
           change.path.endsWith(".json")
         ) {
           log.info(
-            `Editing ${getTypeStrFromPath(change.path)} ${targetPath}${
+            `Editing ${changeTypeLabel(change.path)}${targetPath}${
               targetPath !== change.path
                 ? colors.gray(` (workspace-specific override for ${change.path})`)
                 : ""
@@ -3137,7 +3156,7 @@ export async function pull(
         if (opts.stateful) {
           await mkdir(path.dirname(stateTarget), { recursive: true });
           log.info(
-            `Adding ${getTypeStrFromPath(change.path)} ${targetPath}${
+            `Adding ${changeTypeLabel(change.path)}${targetPath}${
               targetPath !== change.path
                 ? colors.gray(` (workspace-specific override for ${change.path})`)
                 : ""
@@ -3146,7 +3165,7 @@ export async function pull(
         }
         await writeFile(target, change.content, "utf-8");
         log.info(
-          `Writing ${getTypeStrFromPath(change.path)} ${targetPath}${
+          `Writing ${changeTypeLabel(change.path)}${targetPath}${
             targetPath !== change.path
               ? colors.gray(` (workspace-specific override for ${change.path})`)
               : ""
@@ -3158,7 +3177,7 @@ export async function pull(
       } else if (change.name === "deleted") {
         try {
           log.info(
-            `Deleting ${getTypeStrFromPath(change.path)} ${change.path}`,
+            `Deleting ${changeTypeLabel(change.path)}${change.path}`,
           );
           await rm(target);
           if (opts.stateful) {
@@ -3350,6 +3369,9 @@ export async function pull(
     log.warn(`Failed to pull shared UI folder: ${e}`);
   }
 
+  // Datatable migrations are part of the workspace export now, so they flow
+  // through the normal diff/apply above as `datatable_migration` items.
+
   // Git-sync deployment-callback mode stops here: branch checkout + pull have
   // happened, but commit + push are the caller's job. The hub script does
   // them in-process with `set_gpg_signing_secret` so the agent's pre-warmed
@@ -3425,6 +3447,14 @@ export async function gitDeploy(
   } as any);
 }
 
+// Display label for a change's type, with a trailing space. Datatable migrations
+// are self-describing via their `migrations/datatable/...` path, so they get no
+// label prefix.
+function changeTypeLabel(p: string): string {
+  const t = getTypeStrFromPath(p);
+  return t === "datatable_migration" ? "" : `${t} `;
+}
+
 function prettyChanges(
   changes: Change[],
   specificItems?: SpecificItemsConfig,
@@ -3456,7 +3486,7 @@ function prettyChanges(
     if (change.name === "added") {
       log.info(
         colors.green(
-          `+ ${getTypeStrFromPath(change.path)} ` +
+          `+ ${changeTypeLabel(change.path)}` +
             displayPath +
             colors.gray(wsNote),
         ) + extraNote,
@@ -3464,7 +3494,7 @@ function prettyChanges(
     } else if (change.name === "deleted") {
       log.info(
         colors.red(
-          `- ${getTypeStrFromPath(change.path)} ` +
+          `- ${changeTypeLabel(change.path)}` +
             displayPath +
             colors.gray(wsNote),
         ),
@@ -3473,7 +3503,7 @@ function prettyChanges(
       const changeType = getTypeStrFromPath(change.path);
       log.info(
         colors.yellow(
-          `~ ${changeType} ` +
+          `~ ${changeTypeLabel(change.path)}` +
             displayPath +
             colors.gray(wsNote) +
             (change.codebase ? ` (codebase changed)` : ""),
@@ -4221,6 +4251,24 @@ export async function push(
       ));
     }
 
+    // Reject malformed datatable migrations (duplicate timestamps, orphan downs)
+    // before touching the remote, scanning only the data tables in this push.
+    const migrationDatatables = new Set(
+      changes
+        .map((c) => parseDatatableMigrationPath(c.path)?.datatable)
+        .filter((d): d is string => !!d),
+    );
+    if (migrationDatatables.size > 0) {
+      const migrationErrors = validateLocalMigrations(migrationDatatables);
+      if (migrationErrors.length > 0) {
+        log.error(
+          "Invalid datatable migrations, aborting push:\n" +
+            migrationErrors.map((e) => `  - ${e}`).join("\n"),
+        );
+        process.exit(1);
+      }
+    }
+
     if (
       !opts.yes &&
       !(await Confirm.prompt({
@@ -4293,6 +4341,21 @@ export async function push(
     // Cache git branch at the start to avoid repeated execSync calls per change
     const cachedWsNameForPush = wsNameForFiles || (isGitRepository() ? getCurrentGitBranch() : null);
 
+    // Datatable migrations are two files (.up.sql/.down.sql) for one record, so
+    // dedupe upsert/delete by (datatable, version) across the whole push.
+    const pushedMigrationKeys = new Set<string>();
+    // Migrations newly added by this push (an added .up.sql) — offered to run once
+    // the push has completed.
+    const newDatatableMigrations = changes
+      .filter((c) => c.name === "added")
+      .map((c) => parseDatatableMigrationPath(c.path))
+      .filter((p) => !!p && p.kind === "up")
+      .map((p) => ({
+        datatable: p!.datatable,
+        timestamp: p!.timestamp,
+        name: p!.name,
+      }));
+
     while (queue.length > 0 || pool.size > 0) {
       // Fill the pool until we reach the effective parallelism limit.
       // During the folder-meta phase this is 1 (sequential) so no item change
@@ -4320,6 +4383,20 @@ export async function push(
           }
 
           for await (const change of changes) {
+            // A datatable migration is one record across two files; upsert/delete
+            // it from disk once (deduped), regardless of which file changed.
+            if (isDatatableMigrationPath(change.path)) {
+              const parsed = parseDatatableMigrationPath(change.path);
+              if (parsed) {
+                const key = `${parsed.datatable}\0${parsed.timestamp}`;
+                if (!pushedMigrationKeys.has(key)) {
+                  pushedMigrationKeys.add(key);
+                  await pushMigrationFromDisk(workspace.workspaceId, parsed);
+                }
+              }
+              continue;
+            }
+
             let stateTarget = undefined;
             if (stateful) {
               try {
@@ -4974,6 +5051,16 @@ export async function push(
     } catch (e) {
       log.warn(`Failed to push shared UI folder: ${e}`);
     }
+    try {
+      await offerToRunNewMigrations(workspace.workspaceId, newDatatableMigrations, {
+        yes: opts.yes,
+        jsonOutput: opts.jsonOutput,
+      });
+    } catch (e: any) {
+      log.warn(
+        `Failed to run new datatable migrations: ${e?.body ?? e?.message ?? e}`,
+      );
+    }
     const lockJobs = await checkServerLockJobs(
       workspace.workspaceId,
       pushStartedAt,
@@ -5039,6 +5126,7 @@ export async function push(
     } catch (e) {
       log.warn(`Failed to push shared UI folder: ${e}`);
     }
+    // No changes pushed, so no new datatable migrations to run.
     if (opts.jsonOutput) {
       console.log(
         JSON.stringify(

cli/src/commands/workspace/merge.ts

@@ -11,10 +11,12 @@ import {
   deleteItemInWorkspace,
   getOnBehalfOf,
   isTriggerOrScheduleKind,
+  parseDatatableMigrationDeployPath,
   type DeployKind,
   type DeployProvider,
   type TriggerDeployKind,
 } from "../../../windmill-utils-internal/src/deploy.ts";
+import { offerToRunNewMigrations } from "../datatable_migrations.ts";
 
 // ---------------------------------------------------------------------------
 // Provider adapter — wraps CLI's standalone API functions
@@ -85,6 +87,10 @@ const provider: DeployProvider = {
   createSchedule: wmill.createSchedule,
   updateSchedule: wmill.updateSchedule,
   deleteSchedule: wmill.deleteSchedule,
+  // Datatable migrations
+  listDatatableMigrations: wmill.listDatatableMigrations,
+  upsertDatatableMigration: wmill.upsertDatatableMigration,
+  deleteDatatableMigration: wmill.deleteDatatableMigration,
 };
 
 /**
@@ -530,6 +536,14 @@ async function mergeWorkspaces(
   // 10. Deploy
   let successCount = 0;
   let failCount = 0;
+  // Datatable migrations deployed (not deleted) into the target. Deploying a
+  // migration only upserts its definition — the target schema is unchanged until
+  // the migration is run — so offer to run them afterwards (like the push path).
+  const deployedMigrations: {
+    datatable: string;
+    timestamp: number;
+    name: string;
+  }[] = [];
 
   for (const diff of sorted) {
     const label = `${diff.kind}:${diff.path}`;
@@ -573,6 +587,12 @@ async function mergeWorkspaces(
     if (result.success) {
       log.info(colors.green(`  ✓ ${label}`));
       successCount++;
+      if (
+        !itemDeletedInSource &&
+        (diff.kind as DeployKind) === "datatable_migration"
+      ) {
+        deployedMigrations.push(parseDatatableMigrationDeployPath(diff.path));
+      }
     } else {
       log.info(colors.red(`  ✗ ${label}: ${result.error}`));
       failCount++;
@@ -606,6 +626,18 @@ async function mergeWorkspaces(
       )
     );
   }
+
+  // 13. Deployed migration definitions don't touch the target schema until run;
+  // offer to run them on the target now (interactive only, like the push path).
+  if (deployedMigrations.length > 0) {
+    try {
+      await offerToRunNewMigrations(workspaceTo, deployedMigrations, {
+        yes: opts.yes,
+      });
+    } catch (e) {
+      log.warn(colors.yellow(`Failed to run deployed migrations: ${e}`));
+    }
+  }
 }
 
 export { mergeWorkspaces };

cli/src/guidance/skills.gen.ts

@@ -6593,6 +6593,13 @@ datatable related commands
 - \`datatable run <sql:string>\` - run a SQL query on a datatable
   - \`-n --name <name:string>\` - Datatable name (default: main)
   - \`-s --silent\` - Output only the final result as JSON. Useful for scripting.
+- \`datatable migrate\` - manage datatable migrations
+  - \`datatable migrate new <name:string>\` - scaffold a new migration (.up.sql / .down.sql files)
+    - \`-d --datatable <datatable:string>\` - Target datatable (default: main)
+  - \`datatable migrate up\` - apply all pending migrations to the main datatable (or one via --datatable)
+    - \`-d --datatable <datatable:string>\` - Target datatable (default: main)
+  - \`datatable migrate down\` - roll back the most recent migration on the main datatable (or one via --datatable)
+    - \`-d --datatable <datatable:string>\` - Target datatable (default: main)
 - \`datatable create [name:string]\` - register a datatable database in the workspace (default: instance-backed 'main') so scripts can use datatable://<name>
   - \`--resource <resource:string>\` - Back the datatable with an existing postgresql resource path instead of the instance database
   - \`--force\` - Allow adding to a workspace that already has datatables (fork metadata on existing ones is not preserved)
@@ -6861,19 +6868,18 @@ Manage jobs (list, inspect, cancel)
 
 ### jobs
 
-Pull completed and queued jobs from workspace
-
-**Arguments:** \`[workspace:string]\`
-
-**Options:**
-- \`-c, --completed-output <file:string>\` - Completed jobs output file (default: completed_jobs.json)
-- \`-q, --queued-output <file:string>\` - Queued jobs output file (default: queued_jobs.json)
-- \`--skip-worker-check\` - Skip checking for active workers before export
+Manage jobs (import/export)
 
 **Subcommands:**
 
-- \`jobs pull\`
-- \`jobs push\`
+- \`jobs pull [workspace:string]\` - Pull completed and queued jobs from workspace
+  - \`-c, --completed-output <file:string>\` - Completed jobs output file (default: completed_jobs.json)
+  - \`-q, --queued-output <file:string>\` - Queued jobs output file (default: queued_jobs.json)
+  - \`--skip-worker-check\` - Skip checking for active workers before export
+- \`jobs push [workspace:string]\` - Push completed and queued jobs to workspace
+  - \`-c, --completed-file <file:string>\` - Completed jobs input file (default: completed_jobs.json)
+  - \`-q, --queued-file <file:string>\` - Queued jobs input file (default: queued_jobs.json)
+  - \`--skip-worker-check\` - Skip checking for active workers before import
 
 ### lint
 

cli/src/types.ts

@@ -274,10 +274,10 @@ export function parseFromPath(p: string, content: string): any {
   return isWorkspaceDependencies(p)
     ? content
     : p.endsWith(".yaml")
-    ? yamlParseContent(p, content)
-    : p.endsWith(".json")
-    ? JSON.parse(content)
-    : content;
+      ? yamlParseContent(p, content)
+      : p.endsWith(".json")
+        ? JSON.parse(content)
+        : content;
 }
 export function parseFromFile(p: string): any {
   if (p.endsWith(".json")) {
@@ -288,9 +288,38 @@ export function parseFromFile(p: string): any {
     throw new Error("Could not read file " + p);
   }
 }
+/**
+ * Parse a `migrations/datatable/<datatable>/<timestamp>_<name>.(up|down).sql`
+ * path into its parts. Returns undefined for any other path.
+ */
+export function parseDatatableMigrationPath(p: string):
+  | { datatable: string; timestamp: number; name: string; kind: "up" | "down" }
+  | undefined {
+  const parts = p.split("/");
+  if (
+    parts[0] !== "migrations" ||
+    parts[1] !== "datatable" ||
+    parts.length !== 4
+  )
+    return undefined;
+  const m = parts[3].match(/^(\d+)_(.*)\.(up|down)\.sql$/);
+  if (!m) return undefined;
+  return {
+    datatable: parts[2],
+    timestamp: Number(m[1]),
+    name: m[2],
+    kind: m[3] as "up" | "down",
+  };
+}
+
+export function isDatatableMigrationPath(p: string): boolean {
+  return parseDatatableMigrationPath(p) !== undefined;
+}
+
 export function getTypeStrFromPath(
   p: string
 ):
+  | "datatable_migration"
   | "script"
   | "variable"
   | "flow"
@@ -316,6 +345,9 @@ export function getTypeStrFromPath(
   | "settings"
   | "encryption_key"
   | "workspace_dependencies" {
+  if (isDatatableMigrationPath(p)) {
+    return "datatable_migration";
+  }
   if (isScriptModulePath(p)) {
     return "script";
   }