docs(datatables): describe the external instance cluster and how to run one locally

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.release-please-manifest.json

@@ -1,3 +1,3 @@
 {
-  ".": "1.814.0"
+  ".": "1.813.0"
 }

AGENTS.md

@@ -29,6 +29,10 @@ Open-source platform for internal tools, workflows, API integrations, background
 - **Agent workers**: `docs/agent-worker-e2e.md` — building and running one locally. An agent
   reaches the DB only through the API, so `Connection::Http` paths are never taken by a plain
   `cargo run`; a normal build cannot start one at all.
+- **External instance data tables**: `docs/external-instance-datatables.md` — the cluster Windmill
+  administers behind `external_instance` data tables and Ducklake catalogs: its invariants (one
+  lifecycle lock, managed-object markers, the setup gate, per-cluster roles, fork copy ownership)
+  and how to run one locally
 - **Enterprise**: `docs/enterprise.md` — EE file conventions and PR workflow
 - **Auth surface**: `docs/auth-surface.md` — credential precedence, session/cache invalidation
   scope, which token labels email their owner at expiry, how OAuth login matches `login_type`, and

CHANGELOG.md

@@ -1,31 +1,5 @@
 # Changelog
 
-## [1.814.0](https://github.com/windmill-labs/windmill/compare/v1.813.0...v1.814.0) (2026-09-17)
-
-
-### Features
-
-* **ai-chat:** add list_workers and list_data_metrics global tools ([#11143](https://github.com/windmill-labs/windmill/issues/11143)) ([e954d33](https://github.com/windmill-labs/windmill/commit/e954d33613e4ff5027667eb8f646615d9bbd499d))
-* **ai-chat:** merge get_job_logs and get_flow_run_details into get_run ([#11172](https://github.com/windmill-labs/windmill/issues/11172)) ([5bb37ca](https://github.com/windmill-labs/windmill/commit/5bb37ca3388666fba72c55534e37f37bb3e9299e))
-* allow git sync auto-pull, promotion and PRs on Pro licenses ([#11173](https://github.com/windmill-labs/windmill/issues/11173)) ([02e47de](https://github.com/windmill-labs/windmill/commit/02e47de8b4c4f3f54753aabf8c67bc8e71ffb957))
-* badge chat-input flows on the home list ([#11164](https://github.com/windmill-labs/windmill/issues/11164)) ([3d08197](https://github.com/windmill-labs/windmill/commit/3d0819718221f885b61e73d02b43dcc853c7d02a))
-* collect flow conversations and agent memory once their last message goes ([#11178](https://github.com/windmill-labs/windmill/issues/11178)) ([23c24a9](https://github.com/windmill-labs/windmill/commit/23c24a9688d4c8c462f53221334d538280f16bca))
-* flow chat model picker on a shared model-settings component ([#11187](https://github.com/windmill-labs/windmill/issues/11187)) ([189793c](https://github.com/windmill-labs/windmill/commit/189793c2e4db7f1c853695ebcc895c1ec82ed19f))
-* keep flow inputs and seed the agent when chat mode is enabled ([#11177](https://github.com/windmill-labs/windmill/issues/11177)) ([68f2248](https://github.com/windmill-labs/windmill/commit/68f2248018fc218a090bf939e1eb22ff97d5bc22))
-* let plan mode search and read connected mcp servers ([#11205](https://github.com/windmill-labs/windmill/issues/11205)) ([5371519](https://github.com/windmill-labs/windmill/commit/5371519f0f5ce7750982dcdb374dca72115902e7))
-* let test_run_flow name the conversation of a chat-mode test run ([#11198](https://github.com/windmill-labs/windmill/issues/11198)) ([6e1ef93](https://github.com/windmill-labs/windmill/commit/6e1ef93f329cb396ffc3df3304d592e8fa0e0e71))
-* managed memory with an inherited or custom memory id per step ([#11118](https://github.com/windmill-labs/windmill/issues/11118)) ([c297ed0](https://github.com/windmill-labs/windmill/commit/c297ed0052d998fb8f063faa2a36c6eb03e327be))
-* render the flow chat through the shared session chat components ([#11175](https://github.com/windmill-labs/windmill/issues/11175)) ([a9ec0ae](https://github.com/windmill-labs/windmill/commit/a9ec0aec3ac0c6b0f7919d0eb2168816923826d7))
-* show flow step detail inside the graph tab on narrow detail layouts ([#11168](https://github.com/windmill-labs/windmill/issues/11168)) ([64dffe6](https://github.com/windmill-labs/windmill/commit/64dffe6106ad6a55b61a423c855a4b5b0cef533e))
-* store mcp tool call, result and reasoning on flow conversation rows ([#11176](https://github.com/windmill-labs/windmill/issues/11176)) ([a571117](https://github.com/windmill-labs/windmill/commit/a571117f3fd2cef14c920770645c60ee358fdfdd))
-* tell test flow conversations from deployed ones and rename a chat ([#11179](https://github.com/windmill-labs/windmill/issues/11179)) ([4eab995](https://github.com/windmill-labs/windmill/commit/4eab995cf7cf091a5e4640da4cb77e0921bb7fdf))
-
-
-### Bug Fixes
-
-* disable a schedule whose cron has no run left instead of panicking ([#11195](https://github.com/windmill-labs/windmill/issues/11195)) ([381d447](https://github.com/windmill-labs/windmill/commit/381d4470ef699ea82283742132e56556b95d2bd2))
-* skip expiry notifications for app embed and SDK tokens ([#11169](https://github.com/windmill-labs/windmill/issues/11169)) ([9d348f8](https://github.com/windmill-labs/windmill/commit/9d348f84c7830f36b6153472556fd70e3d84cd24))
-
 ## [1.813.0](https://github.com/windmill-labs/windmill/compare/v1.812.0...v1.813.0) (2026-09-16)
 
 

ai_evals/README.md

@@ -175,10 +175,6 @@ the decrypted value, exactly as against a real backend. The chat's read path pas
 Seed a recognizable secret (the existing fixture uses `sk_live_do_not_leak_me`) and
 assert it via `valueExcludes` to catch a leak.
 
-`toolExpect.toolCallArgs` entries support `sharedByAtLeast: <n>`: at least `n` recorded
-calls to that tool must carry the same non-blank string in the field. Use it for calls that
-have to share an identifier, like two test runs of one chat conversation.
-
 `toolExpect.toolCallArgs` entries additionally support `fieldMustBeAbsent: true`: no
 recorded call to that tool may pass the field at all (an explicit `null` counts as
 passing it). Use it for partial-update tools, where supplying a field the model could

ai_evals/adapters/frontend/backendPreview.ts

@@ -1,5 +1,5 @@
+import { randomUUID } from 'node:crypto'
 import type { BackendValidationSettings } from '../../core/backendValidation'
-import { buildWorkspaceId } from './workspaceId'
 
 interface CompletedJobResultMaybe {
 	completed: boolean
@@ -24,6 +24,7 @@ export interface CompletedPreviewJob {
 const tokenCache = new Map<string, Promise<string>>()
 const sharedWorkspaceQueue = new Map<string, Promise<void>>()
 const managedSharedWorkspacePrefixes = ['f/evals/']
+const DEFAULT_WORKSPACE_PREFIX = 'ai-evals'
 
 export class BackendPreviewClient {
 	constructor(private readonly settings: BackendValidationSettings) {}
@@ -440,6 +441,16 @@ async function withSharedWorkspaceLock<T>(workspaceId: string, body: () => Promi
 	}
 }
 
+function buildWorkspaceId(caseId: string, attempt: number): string {
+	const caseSlug = caseId
+		.toLowerCase()
+		.replace(/[^a-z0-9-]+/g, '-')
+		.replace(/^-+|-+$/g, '')
+		.slice(0, 30)
+	const suffix = randomUUID().slice(0, 8)
+	return `${DEFAULT_WORKSPACE_PREFIX}-${caseSlug || 'case'}-a${attempt}-${suffix}`
+}
+
 function extractFolderName(path: string): string | null {
 	if (!path.startsWith('f/')) {
 		return null

ai_evals/adapters/frontend/mockBackend.ts

@@ -11,7 +11,6 @@ import type {
 	Script
 } from '../../../frontend/src/lib/gen'
 import type {
-	DataMetric,
 	DataTableTables,
 	DataTableTableSchema,
 	EndpointTool,
@@ -49,16 +48,12 @@ export interface BenchmarkWorkspaceFlow {
 export interface BenchmarkWorkspaceApp {
 	path: string
 	summary: string
-	/** Defaults to true. Set false for a drag-and-drop app, which the chat can list
-	 * and read but has no tool to edit — its value is a grid, not files. */
-	rawApp?: boolean
 	value: {
-		files?: Record<string, string>
-		runnables?: Record<string, unknown>
+		files: Record<string, string>
+		runnables: Record<string, unknown>
 		data?: unknown
 		policy?: unknown
 		custom_path?: unknown
-		[key: string]: unknown
 	}
 }
 
@@ -117,9 +112,6 @@ export interface BenchmarkWorkspaceRunnables {
 	aiProviders?: BenchmarkWorkspaceAiProvider[]
 	resources?: BenchmarkWorkspaceResource[]
 	datatables?: BenchmarkDatatableSeed[]
-	/** DuckLake catalog names, as `list_ducklakes` reports them. */
-	ducklakes?: string[]
-	dataMetrics?: DataMetric[]
 	jobs?: BenchmarkWorkspaceJob[]
 }
 
@@ -681,27 +673,6 @@ export function listBenchmarkDatatables(workspace: string): DataTableTables[] |
 	}))
 }
 
-// ============= DuckLake catalogs and declared metrics =============
-
-/** Seeded DuckLake names, or `null` for a non-benchmark workspace. */
-export function listBenchmarkDucklakes(workspace: string): string[] | null {
-	const runnables = benchmarkWorkspaceRunnables.get(workspace)
-	return runnables ? (runnables.ducklakes ?? []) : null
-}
-
-/**
- * Seeded metric declarations, or `null` for a non-benchmark workspace.
- *
- * The `table` / `path_prefix` filters are ignored: which rows a filter selects is
- * `canonical_table_path`'s business and is pinned by `ducklakeTools.test.ts`.
- * Re-deriving it here would give the eval its own copy of that spec to drift from,
- * and the case this serves measures whether the model reaches for the tool at all.
- */
-export function listBenchmarkDataMetrics(workspace: string): DataMetric[] | null {
-	const runnables = benchmarkWorkspaceRunnables.get(workspace)
-	return runnables ? (runnables.dataMetrics ?? []) : null
-}
-
 export function getBenchmarkDatatableSchema(input: {
 	workspace: string
 	datatableName: string
@@ -869,29 +840,6 @@ export function runBenchmarkFlowByPath(input: {
 	})
 }
 
-/**
- * Mirror `JobService.runFlowPreview` for benchmark workspaces, including the server's
- * refusal of a chat-enabled flow run that names no conversation (`memory_id`).
- */
-export function runBenchmarkFlowPreview(input: {
-	workspace: string
-	memoryId?: string
-	requestBody?: { path?: string; value?: { chat_input_enabled?: boolean }; args?: unknown }
-}): string {
-	if (input.requestBody?.value?.chat_input_enabled && !input.memoryId) {
-		throw new Error('Bad request: memory_id is required for chat-enabled flows')
-	}
-	const args = (input.requestBody?.args ?? {}) as Record<string, unknown>
-	return createBenchmarkCompletedJob({
-		workspace: input.workspace,
-		jobKind: 'flowpreview',
-		success: true,
-		args,
-		result: { path: input.requestBody?.path, args, mocked: true },
-		logs: 'Mock benchmark flow preview completed successfully.'
-	})
-}
-
 export function previewBenchmarkSchedule(input: {
 	requestBody?: Record<string, unknown>
 }): Record<string, unknown> {
@@ -998,7 +946,7 @@ function buildBenchmarkListableApp(app: BenchmarkWorkspaceApp): ListableApp {
 		extra_perms: {},
 		edited_at: BENCHMARK_TIMESTAMP,
 		execution_mode: 'viewer',
-		raw_app: app.rawApp ?? true
+		raw_app: true
 	}
 }
 
@@ -1016,7 +964,7 @@ function buildBenchmarkApp(app: BenchmarkWorkspaceApp): AppWithLastVersion {
 		execution_mode: 'viewer',
 		extra_perms: {},
 		custom_path: app.value.custom_path as string | undefined,
-		raw_app: app.rawApp ?? true
+		raw_app: true
 	}
 }
 

ai_evals/adapters/frontend/vitestAdapter.test.ts

@@ -76,9 +76,7 @@ vi.mock('$lib/gen', async () => {
 		listBenchmarkPlainResources,
 		listBenchmarkApps,
 		listBenchmarkDatatables,
-		listBenchmarkDataMetrics,
 		listBenchmarkDrafts,
-		listBenchmarkDucklakes,
 		listBenchmarkFlows,
 		listBenchmarkJobs,
 		listBenchmarkScripts,
@@ -89,7 +87,6 @@ vi.mock('$lib/gen', async () => {
 		previewBenchmarkSchedule,
 		runBenchmarkDatatableSql,
 		runBenchmarkFlowByPath,
-		runBenchmarkFlowPreview,
 		runBenchmarkScriptByPath,
 		runBenchmarkScriptPreview,
 		updateBenchmarkDraft,
@@ -296,14 +293,6 @@ vi.mock('$lib/gen', async () => {
 							args: data.requestBody
 						})
 					: actual.JobService.runScriptByPath(data),
-			runFlowPreview: async (data: {
-				workspace: string
-				memoryId?: string
-				requestBody?: { path?: string; value?: { chat_input_enabled?: boolean }; args?: unknown }
-			}) =>
-				hasBenchmarkWorkspace(data.workspace)
-					? runBenchmarkFlowPreview(data)
-					: actual.JobService.runFlowPreview(data as any),
 			runFlowByPath: async (data: {
 				workspace: string
 				path: string
@@ -352,10 +341,6 @@ vi.mock('$lib/gen', async () => {
 				hasBenchmarkWorkspace(data.workspace)
 					? (listBenchmarkDatatables(data.workspace) ?? [])
 					: actual.WorkspaceService.listDataTableTables(data),
-			listDucklakes: async (data: { workspace: string }) =>
-				hasBenchmarkWorkspace(data.workspace)
-					? (listBenchmarkDucklakes(data.workspace) ?? [])
-					: actual.WorkspaceService.listDucklakes(data),
 			getDataTableTableSchema: async (data: {
 				workspace: string
 				datatableName: string
@@ -371,12 +356,6 @@ vi.mock('$lib/gen', async () => {
 						})
 					: actual.WorkspaceService.getDataTableTableSchema(data)
 		}),
-		DataMetricService: wrapService(actual.DataMetricService, {
-			listDataMetrics: async (data: { workspace: string }) =>
-				hasBenchmarkWorkspace(data.workspace)
-					? { metrics: listBenchmarkDataMetrics(data.workspace) ?? [] }
-					: actual.DataMetricService.listDataMetrics(data)
-		}),
 		ScheduleService: wrapService(actual.ScheduleService, {
 			existsSchedule: async (data: { workspace: string; path: string }) =>
 				hasBenchmarkWorkspace(data.workspace) ? false : actual.ScheduleService.existsSchedule(data),

ai_evals/adapters/frontend/windmillBackend.ts

@@ -1,8 +1,9 @@
+import { randomUUID } from "node:crypto";
 import type { WindmillBackendSettings } from "../../core/windmillBackendSettings";
-import { buildWorkspaceId } from "./workspaceId";
 
 const tokenCache = new Map<string, Promise<string>>();
 const sharedWorkspaceQueue = new Map<string, Promise<void>>();
+const DEFAULT_WORKSPACE_PREFIX = "ai-evals";
 
 export class WindmillBackendClient {
   constructor(private readonly settings: WindmillBackendSettings) {}
@@ -178,6 +179,16 @@ async function withSharedWorkspaceLock<T>(
   }
 }
 
+function buildWorkspaceId(caseId: string, attempt: number): string {
+  const caseSlug = caseId
+    .toLowerCase()
+    .replace(/[^a-z0-9-]+/g, "-")
+    .replace(/^-+|-+$/g, "")
+    .slice(0, 30);
+  const suffix = randomUUID().slice(0, 8);
+  return `${DEFAULT_WORKSPACE_PREFIX}-${caseSlug || "case"}-a${attempt}-${suffix}`;
+}
+
 async function expectOk(response: Response, context: string): Promise<void> {
   if (response.ok) {
     return;

ai_evals/adapters/frontend/workspaceId.test.ts

@@ -1,21 +0,0 @@
-import { describe, expect, it } from "bun:test";
-import { buildWorkspaceId } from "./workspaceId";
-
-describe("buildWorkspaceId", () => {
-  // `workspace.proper_id` rejects `--`, which a case id can carry itself and
-  // which truncating a slug on a hyphen produces once the suffix adds its own.
-  // One id per shape: cut landing on a hyphen, cut landing mid-word, no cut, and
-  // a doubled hyphen no cut ever reaches.
-  it("stays within the id length cap and the proper_id format", () => {
-    for (const caseId of [
-      "global-test6-secret-variable-draft",
-      "global-test23-datatable-query-select",
-      "short",
-      "global--test-foo",
-    ]) {
-      const id = buildWorkspaceId(caseId, 1);
-      expect(id.length).toBeLessThanOrEqual(50);
-      expect(id).toMatch(/^\w+(-\w+)*$/);
-    }
-  });
-});

ai_evals/adapters/frontend/workspaceId.ts

@@ -1,22 +0,0 @@
-import { randomUUID } from "node:crypto";
-
-const DEFAULT_WORKSPACE_PREFIX = "ai-evals";
-
-// A workspace id must be at most 50 characters AND match `^\w+(-\w+)*$`
-// (`workspace.proper_id`), so the case slug yields to the random suffix that
-// makes the id unique, and no hyphen may end up doubled — neither one already in
-// the case id nor one a truncation leaves for the suffix to follow.
-const MAX_WORKSPACE_ID_LENGTH = 50;
-
-export function buildWorkspaceId(caseId: string, attempt: number): string {
-  const caseSlug = caseId
-    .toLowerCase()
-    .replace(/[^a-z0-9-]+/g, "-")
-    .replace(/-{2,}/g, "-")
-    .replace(/^-+|-+$/g, "");
-  const suffix = `-a${attempt}-${randomUUID().slice(0, 8)}`;
-  const head = `${DEFAULT_WORKSPACE_PREFIX}-${caseSlug || "case"}`;
-  return `${head
-    .slice(0, MAX_WORKSPACE_ID_LENGTH - suffix.length)
-    .replace(/-+$/, "")}${suffix}`;
-}

ai_evals/cases/global.yaml

@@ -1919,11 +1919,10 @@
     - when the lookup fails, tells the user instead of inventing table names
     - does not write scripts or resources to answer a read-only question
 
-# --- Dedicated tools preferred over the API catalog ---
-# The harness serves worker/queue reads itself (benchmark fetch handlers in
-# adapters/frontend), so these cases do not require an mcp-enabled eval backend.
-# The stale `api-catalog` in the id below is kept so results stay comparable
-# across benchmark runs.
+# --- API catalog (search_api_endpoints / call_api_get / call_api_endpoint) ---
+# The harness serves the catalog and the executed calls itself (mock
+# listMcpTools + benchmark fetch handlers in adapters/frontend), so these cases
+# do not require an mcp-enabled eval backend.
 
 - id: global-test30-api-catalog-workers
   prompt: |-
@@ -1935,42 +1934,23 @@
     draftCountExactly: 0
   toolExpect:
     requiredToolsUsed:
-      - list_workers
-    forbiddenToolsUsed:
+      - search_api_endpoints
       - call_api_get
+    forbiddenToolsUsed:
       - call_api_endpoint
       - write_script
       - deploy_workspace_item
+    toolCallArgs:
+      - tool: call_api_get
+        field: name
+        stringIncludesAnyOf:
+          - listWorkers
   # Read-only workspace inspection produces no draft; validate via tool use.
   skipJudge: true
   judgeChecklist:
-    - reads worker state through list_workers instead of guessing or fabricating
+    - discovers the workers endpoint through the API catalog instead of guessing or fabricating
     - reports worker status from the returned data
 
-- id: global-test37-ducklake-declared-measure
-  prompt: |-
-    We track orders in the main ducklake. Write me a duckdb script that reports total
-    revenue by month. Keep it as a draft, don't deploy it.
-  initial: ai_evals/fixtures/frontend/global/initial/ducklake_orders_metrics.json
-  runtime:
-    maxTurns: 8
-  validate:
-    draftCountExactly: 1
-  toolExpect:
-    requiredToolsUsed:
-      - list_data_metrics
-    forbiddenToolsUsed:
-      - deploy_workspace_item
-      - delete_workspace_item
-  # The judge runs: the point is not that the tool was called but that the number it
-  # describes is the declared one. `revenue` excludes test rows, so an aggregate that
-  # reproduces it without the filter is plausible, runnable and wrong.
-  judgeChecklist:
-    - totals revenue with the declared sum over the amount column rather than an invented aggregate over a guessed column
-    - excludes test orders from the total, as the declared revenue measure does
-    - groups by month using the declared order_month expression over order_date
-    - does not introduce column names absent from the declarations
-
 - id: global-test31-draft-test-run-not-deployed
   prompt: |-
     Update `f/evals/global/format_greeting` so the provided name is uppercased in the greeting, then run it with name "ada" to check it works.
@@ -2160,32 +2140,6 @@
     - creates an AI draft of f/evals/global/process_invoice applying 8% tax
     - does not deploy or save the draft
 
-- id: global-test38-chat-flow-follow-up-same-conversation
-  prompt: |-
-    I want to check that my support chat flow `f/evals/global/support_chat` remembers what was said.
-    Test it: first send "My name is Ada", then send "What is my name?" as a follow-up in the same chat.
-  initial: ai_evals/fixtures/frontend/global/initial/support_chat_flow.json
-  runtime:
-    maxTurns: 8
-  validate:
-    draftCountExactly: 0
-  toolExpect:
-    requiredToolsUsed:
-      - test_run_flow
-    # A chat flow's memory lives in its conversation, so a follow-up only reaches the first
-    # turn's history when both test runs name the same conversation.
-    toolCallArgs:
-      - tool: test_run_flow
-        field: memory_id
-        sharedByAtLeast: 2
-    forbiddenToolsUsed:
-      - run_flow
-      - deploy_workspace_item
-  # The judge cannot observe runs; what this case guards is the conversation the runs share.
-  skipJudge: true
-  judgeChecklist:
-    - test-runs the chat flow twice, the second message as a follow-up in the first run's conversation
-
 - id: global-undo-created-draft
   prompt: |-
     Create a draft Postgres resource at `u/admin/scratch_db` for host db.example.com port 5432, database `orders`, user `app`, and tell me what fields it ended up with.
@@ -2611,40 +2565,3 @@
   judgeChecklist:
     - runs the existing script rather than rewriting it
     - passes the GitHub resource as the bare string $res:f/evals/global/github_main
-
-- id: global-drag-and-drop-app-not-editable
-  prompt: |-
-    Add a refresh button to the ops console app, and the same to the sales board app.
-  initial: ai_evals/fixtures/frontend/global/initial/apps_code_and_drag_and_drop.json
-  runtime:
-    maxTurns: 12
-  validate:
-    # One request, two apps, only one of them editable: the code app must come back with a
-    # draft and the drag-and-drop one must not. Refusing both, or editing both, fails here —
-    # which is what makes this a test of the distinction rather than of caution.
-    draftCountExactly: 1
-    requiredDrafts:
-      - type: app
-        path: f/evals/global/ops_console
-    forbiddenDrafts:
-      - type: app
-        path: f/evals/global/sales_board
-  toolExpect:
-    # Deliberately not constraining write_app_file/patch_app_file by argument: an entry there
-    # fails when its tool was never called, so naming both would fail on whichever the model
-    # did not pick. The draft assertions above cover the same ground, tool-agnostically.
-    forbiddenToolsUsed:
-      - init_app
-      - deploy_workspace_item
-      - delete_app_file
-      - delete_app_runnable
-  assistantExpect:
-    # A refusal leaves no draft for the judge to read, so the explanation is checked here.
-    # Only the app kind: substring tests cannot see paraphrase, and every wording of "I can't
-    # edit it" defeats a fixed list.
-    requiredMentionsAnyOf:
-      - - drag-and-drop
-        - drag and drop
-        - low-code
-        - no-code
-  skipJudge: true

ai_evals/core/types.ts

@@ -182,13 +182,6 @@ export interface ToolCallArgumentRule {
    * the point is that the model filled it in at all rather than what it said.
    */
   nonEmpty?: boolean;
-  /**
-   * Existential over calls: at least this many recorded calls to `tool` carry the
-   * same non-blank string in `field`. Use when calls have to share an identifier —
-   * e.g. test runs that continue one conversation — while a retry with a rejected
-   * value in between is still acceptable.
-   */
-  sharedByAtLeast?: number;
   /**
    * Universal over calls: no recorded call to `tool` may pass `field` at all.
    * For partial-update tools, where supplying a field the model could not have

ai_evals/core/validators.test.ts

@@ -396,32 +396,6 @@ describe("validateToolExpectations", () => {
     expect(nonEmptyCheck?.details).toContain("blank on 1 of 2");
   });
 
-  it("requires sharedByAtLeast calls to carry one value, not merely a value each", () => {
-    const run = (ids: (string | undefined)[]) =>
-      validateToolExpectations({
-        run: {
-          success: true,
-          actual: {},
-          assistantMessageCount: 1,
-          toolCallCount: ids.length,
-          toolsUsed: ["test_run_flow"],
-          toolCallDetails: ids.map((memory_id) => ({
-            name: "test_run_flow",
-            arguments: { path: "f/chat", memory_id },
-          })),
-          skillsInvoked: [],
-        },
-        toolExpect: {
-          toolCallArgs: [{ tool: "test_run_flow", field: "memory_id", sharedByAtLeast: 2 }],
-        },
-      }).find((c) => c.name.includes("is shared by at least 2 calls"))?.passed;
-
-    expect(run(["a", "b"])).toBe(false);
-    expect(run(["a"])).toBe(false);
-    expect(run([undefined, undefined])).toBe(false);
-    expect(run(["rejected", "a", "a"])).toBe(true);
-  });
-
   it("passes nonEmpty when every call filled the field", () => {
     const checks = validateToolExpectations({
       run: {

ai_evals/core/validators.ts

@@ -320,23 +320,6 @@ export function validateToolExpectations(input: {
       );
     }
 
-    if (rule.sharedByAtLeast !== undefined) {
-      const counts = new Map<string, number>();
-      for (const value of values) {
-        if (typeof value === "string" && value.trim().length > 0) {
-          counts.set(value, (counts.get(value) ?? 0) + 1);
-        }
-      }
-      const mostShared = Math.max(0, ...counts.values());
-      checks.push(
-        check(
-          `${rule.tool}.${rule.field} is shared by at least ${rule.sharedByAtLeast} calls`,
-          mostShared >= rule.sharedByAtLeast,
-          `most calls sharing one value: ${mostShared}; values: ${summarizeToolValues(values)}`
-        )
-      );
-    }
-
     if (rule.fieldMustBeAbsent) {
       // Anything other than `undefined` was supplied — an explicit `null` is the
       // model passing the field, not omitting it.

ai_evals/fixtures/frontend/global/initial/apps_code_and_drag_and_drop.json

@@ -1,30 +0,0 @@
-{
-  "user": {
-    "username": "admin",
-    "is_admin": true,
-    "folders": ["evals"],
-    "folders_read": ["evals"]
-  },
-  "workspace": {
-    "apps": [
-      {
-        "path": "f/evals/global/sales_board",
-        "summary": "Sales board",
-        "rawApp": false,
-        "value": {
-          "grid": []
-        }
-      },
-      {
-        "path": "f/evals/global/ops_console",
-        "summary": "Ops console",
-        "value": {
-          "files": {
-            "/App.tsx": "export default function App() {\n\treturn <div>Ops console</div>\n}\n"
-          },
-          "runnables": {}
-        }
-      }
-    ]
-  }
-}

ai_evals/fixtures/frontend/global/initial/ducklake_orders_metrics.json

@@ -1,36 +0,0 @@
-{
-  "workspace": {
-    "ducklakes": ["main"],
-    "dataMetrics": [
-      {
-        "script_path": "f/analytics/orders_pipeline",
-        "table_path": "main/main.orders",
-        "kind": "measure",
-        "name": "revenue",
-        "expr": "sum(amount)",
-        "filter": "not is_test"
-      },
-      {
-        "script_path": "f/analytics/orders_pipeline",
-        "table_path": "main/main.orders",
-        "kind": "measure",
-        "name": "order_count",
-        "expr": "count(*)"
-      },
-      {
-        "script_path": "f/analytics/orders_pipeline",
-        "table_path": "main/main.orders",
-        "kind": "dimension",
-        "name": "order_month",
-        "expr": "date_trunc('month', order_date)"
-      },
-      {
-        "script_path": "f/analytics/orders_pipeline",
-        "table_path": "main/main.orders",
-        "kind": "dimension",
-        "name": "region",
-        "expr": "region"
-      }
-    ]
-  }
-}

ai_evals/fixtures/frontend/global/initial/support_chat_flow.json

@@ -1,59 +0,0 @@
-{
-  "workspace": {
-    "flows": [
-      {
-        "path": "f/evals/global/support_chat",
-        "summary": "Support chat",
-        "description": "Answers customer questions in a chat, remembering earlier messages.",
-        "schema": {
-          "$schema": "https://json-schema.org/draft/2020-12/schema",
-          "type": "object",
-          "properties": {
-            "user_message": {
-              "type": "string",
-              "description": "Message from user"
-            }
-          },
-          "required": ["user_message"]
-        },
-        "value": {
-          "chat_input_enabled": true,
-          "modules": [
-            {
-              "id": "assistant",
-              "summary": "Support assistant",
-              "value": {
-                "type": "aiagent",
-                "tools": [],
-                "input_transforms": {
-                  "provider": {
-                    "type": "static",
-                    "value": {
-                      "kind": "anthropic",
-                      "model": "claude-haiku-4-5-20251001",
-                      "resource": "$res:f/evals/ai/anthropic"
-                    }
-                  },
-                  "user_message": {
-                    "type": "javascript",
-                    "expr": "flow_input.user_message"
-                  },
-                  "system_prompt": {
-                    "type": "static",
-                    "value": "You are a friendly support assistant. Keep answers short."
-                  },
-                  "memory": {
-                    "type": "static",
-                    "value": { "kind": "auto", "context_length": 10 }
-                  },
-                  "streaming": { "type": "static", "value": true },
-                  "output_type": { "type": "static", "value": "text" }
-                }
-              }
-            }
-          ]
-        }
-      }
-    ]
-  }
-}

backend/.sqlx/query-1c3473a0f9f6b6148b2c975f9f05bdefedf8a51c4e6ddf0eca367b9cc778d051.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments\n             FROM flow_conversation_message\n             WHERE conversation_id = $1\n               AND created_seq > $2\n             ORDER BY created_seq ASC\n             LIMIT $3\n             ",
+  "query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success\n             FROM (\n                SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success\n                FROM flow_conversation_message\n                WHERE conversation_id = $1\n                ORDER BY created_seq DESC\n                LIMIT $2 OFFSET $3\n             ) AS messages\n             ORDER BY created_seq ASC\n             ",
   "describe": {
     "columns": [
       {
@@ -58,26 +58,6 @@
         "ordinal": 8,
         "name": "success",
         "type_info": "Bool"
-      },
-      {
-        "ordinal": 9,
-        "name": "tool_arguments",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 10,
-        "name": "tool_result",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 11,
-        "name": "reasoning",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 12,
-        "name": "attachments",
-        "type_info": "Jsonb"
       }
     ],
     "parameters": {
@@ -96,12 +76,8 @@
       false,
       false,
       true,
-      false,
-      true,
-      true,
-      true,
-      true
+      false
     ]
   },
-  "hash": "d6fa78c43b6c5f8040d7bccb29ad8627be1dac6fbe0097735a52f47c173f51c9"
+  "hash": "1c3473a0f9f6b6148b2c975f9f05bdefedf8a51c4e6ddf0eca367b9cc778d051"
 }

backend/.sqlx/query-255ba68caa78d0e814ea817693a319f769907025f9c7d1150d45e8ecb1bff4ab.json

@@ -0,0 +1,15 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE workspace_settings SET auto_invite = $1 WHERE workspace_id = $2",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Jsonb",
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "255ba68caa78d0e814ea817693a319f769907025f9c7d1150d45e8ecb1bff4ab"
+}

backend/.sqlx/query-5b9c9eb64051f291fed4be9bc0b0cc0aef2e7bde732899976eddac36a2da7658.json

@@ -1,24 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "UPDATE flow_conversation SET title = $1, updated_at = updated_at\n         WHERE id = $2 AND workspace_id = $3\n         RETURNING id",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "id",
-        "type_info": "Uuid"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Varchar",
-        "Uuid",
-        "Text"
-      ]
-    },
-    "nullable": [
-      false
-    ]
-  },
-  "hash": "5b9c9eb64051f291fed4be9bc0b0cc0aef2e7bde732899976eddac36a2da7658"
-}

backend/.sqlx/query-6f32c1feed096ff706ae359ad6a3ca33b3f82ca38289dfa4a69aa95041027d57.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test\n         FROM flow_conversation\n         WHERE id = $1 AND workspace_id = $2",
+  "query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by\n         FROM flow_conversation\n         WHERE id = $1 AND workspace_id = $2\n         FOR UPDATE",
   "describe": {
     "columns": [
       {
@@ -37,11 +37,6 @@
         "ordinal": 6,
         "name": "created_by",
         "type_info": "Varchar"
-      },
-      {
-        "ordinal": 7,
-        "name": "is_test",
-        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -57,9 +52,8 @@
       true,
       false,
       false,
-      false,
       false
     ]
   },
-  "hash": "48c8522a4fed219c5011f4ba63c81cfe028a8b2a32bd790840cef65c452a8c31"
+  "hash": "6f32c1feed096ff706ae359ad6a3ca33b3f82ca38289dfa4a69aa95041027d57"
 }

backend/.sqlx/query-71ee2cb6661cca1fa4d8874a7f6d368347c59f36fd87df6dc7996152ccb84af0.json

@@ -1,38 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT id, name, enabled, pwd FROM datatable_role",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "id",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "name",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 2,
-        "name": "enabled",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 3,
-        "name": "pwd",
-        "type_info": "Text"
-      }
-    ],
-    "parameters": {
-      "Left": []
-    },
-    "nullable": [
-      false,
-      false,
-      false,
-      true
-    ]
-  },
-  "hash": "71ee2cb6661cca1fa4d8874a7f6d368347c59f36fd87df6dc7996152ccb84af0"
-}

backend/.sqlx/query-75e6b5cd52d63ac094c90abd1524bc7a919692a96b076b8aaf24d812723531c8.json

@@ -1,23 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT EXISTS(SELECT 1 FROM usr WHERE email = $1 AND is_service_account IS true\n            AND ($2::varchar IS NULL OR workspace_id = $2))",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "exists",
-        "type_info": "Bool"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Varchar"
-      ]
-    },
-    "nullable": [
-      null
-    ]
-  },
-  "hash": "75e6b5cd52d63ac094c90abd1524bc7a919692a96b076b8aaf24d812723531c8"
-}

backend/.sqlx/query-79799b5a2e499df6c28e286c42b9ad2db940c2455ab19cc95e5198baf96d5629.json

@@ -1,29 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "\n        SELECT ws.workspace_id AS \"workspace_id!\", dt.key AS \"datatable!\"\n        FROM workspace_settings ws\n        JOIN workspace w ON w.id = ws.workspace_id AND w.deleted = false\n        CROSS JOIN LATERAL jsonb_each(COALESCE(ws.datatable->'datatables', '{}'::jsonb)) dt\n        WHERE ws.workspace_id <> $1\n          AND dt.value->'database'->>'resource_type' = 'instance'\n          AND dt.value->'database'->>'resource_path' = $2\n        ORDER BY ws.workspace_id, dt.key\n        ",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "workspace_id!",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "datatable!",
-        "type_info": "Text"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      false,
-      null
-    ]
-  },
-  "hash": "79799b5a2e499df6c28e286c42b9ad2db940c2455ab19cc95e5198baf96d5629"
-}

backend/.sqlx/query-86af9d51a158ea5cb6161461ecddf2a63695f8cbf8af648da5a0a77a5b9d02ba.json

@@ -1,17 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "INSERT INTO datatable_role (id, name, enabled, pwd) VALUES ($1, $2, $3, $4)",
-  "describe": {
-    "columns": [],
-    "parameters": {
-      "Left": [
-        "Varchar",
-        "Varchar",
-        "Bool",
-        "Text"
-      ]
-    },
-    "nullable": []
-  },
-  "hash": "86af9d51a158ea5cb6161461ecddf2a63695f8cbf8af648da5a0a77a5b9d02ba"
-}

backend/.sqlx/query-88a134e4ca82d5ce0334977c7713021ae3e99a5a61ea1c944c1df1368746dfa5.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT workspace.id, workspace.name, usr.username, workspace_settings.color, workspace.parent_workspace_id,\n                workspace.is_dev_workspace, workspace.dev_workspace_label,\n                workspace.owner AS \"created_by?\",\n                CASE WHEN usr.operator THEN workspace_settings.operator_settings ELSE NULL END as operator_settings,\n                usr.disabled, usr.is_service_account\n         FROM workspace\n         JOIN usr ON usr.workspace_id = workspace.id\n         JOIN workspace_settings ON workspace_settings.workspace_id = workspace.id\n         WHERE usr.email = $1 AND workspace.deleted = false",
+  "query": "SELECT workspace.id, workspace.name, usr.username, workspace_settings.color, workspace.parent_workspace_id,\n                workspace.is_dev_workspace, workspace.dev_workspace_label,\n                workspace.owner AS \"created_by?\",\n                CASE WHEN usr.operator THEN workspace_settings.operator_settings ELSE NULL END as operator_settings,\n                usr.disabled\n         FROM workspace\n         JOIN usr ON usr.workspace_id = workspace.id\n         JOIN workspace_settings ON workspace_settings.workspace_id = workspace.id\n         WHERE usr.email = $1 AND workspace.deleted = false",
   "describe": {
     "columns": [
       {
@@ -52,11 +52,6 @@
         "ordinal": 9,
         "name": "disabled",
         "type_info": "Bool"
-      },
-      {
-        "ordinal": 10,
-        "name": "is_service_account",
-        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -74,9 +69,8 @@
       true,
       false,
       null,
-      false,
       false
     ]
   },
-  "hash": "7b33adb5cf051bc123340982b97a80a96ee43e1c91d9907d50cc2ee14111623e"
+  "hash": "88a134e4ca82d5ce0334977c7713021ae3e99a5a61ea1c944c1df1368746dfa5"
 }

backend/.sqlx/query-94fd0a57cfc9341b2e9deae60506c6c06aa6934b87200da14231f12f65149cd3.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "DELETE FROM token_expiry_notification n\n         USING token t\n         WHERE n.token_hash = t.token_hash\n           AND n.expiration > now()\n           AND n.expiration <= now() + make_interval(days => $1)\n         RETURNING t.token_prefix, t.label, t.email, t.workspace_id",
+  "query": "DELETE FROM token_expiry_notification n\n         USING token t\n         WHERE n.token_hash = t.token_hash\n           AND n.expiration > now()\n           AND n.expiration <= now() + interval '7 days'\n         RETURNING t.token_prefix, t.label, t.email, t.workspace_id",
   "describe": {
     "columns": [
       {
@@ -25,9 +25,7 @@
       }
     ],
     "parameters": {
-      "Left": [
-        "Int4"
-      ]
+      "Left": []
     },
     "nullable": [
       false,
@@ -36,5 +34,5 @@
       true
     ]
   },
-  "hash": "6c57c46c5a0462f379ed6a22ae97e363009177fdc45d795447a920f455f193cc"
+  "hash": "94fd0a57cfc9341b2e9deae60506c6c06aa6934b87200da14231f12f65149cd3"
 }

backend/.sqlx/query-a1bcf143135ecb9f32709a56a353d3246dc238920c3e155caa9cf767246fa6fb.json

@@ -1,15 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "UPDATE workspace_settings SET auto_invite = (COALESCE(auto_invite, '{}'::jsonb) - 'domain') || $1::jsonb WHERE workspace_id = $2",
-  "describe": {
-    "columns": [],
-    "parameters": {
-      "Left": [
-        "Jsonb",
-        "Text"
-      ]
-    },
-    "nullable": []
-  },
-  "hash": "a1bcf143135ecb9f32709a56a353d3246dc238920c3e155caa9cf767246fa6fb"
-}

backend/.sqlx/query-b1a9a433e577133869c067b2ce383fc6ce4e9df307feb5fd3edc0d1276d61ff1.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success, tool_arguments, tool_result, reasoning, attachments)\n         VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)",
+  "query": "INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success)\n         VALUES ($1, $2, $3, $4, $5, $6)",
   "describe": {
     "columns": [],
     "parameters": {
@@ -21,14 +21,10 @@
         "Text",
         "Uuid",
         "Varchar",
-        "Bool",
-        "Text",
-        "Text",
-        "Text",
-        "Jsonb"
+        "Bool"
       ]
     },
     "nullable": []
   },
-  "hash": "12329c3359a7944ab5fa3aa27ddca1b26f340ccf574b9fa07641fe88b2d2987c"
+  "hash": "b1a9a433e577133869c067b2ce383fc6ce4e9df307feb5fd3edc0d1276d61ff1"
 }

backend/.sqlx/query-b9842d2d8abf382bd82d8fa1de012373638be391f884f81dc387ffc465badac6.json

@@ -1,20 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT jsonb_object_keys(value->'databases') FROM global_settings\n         WHERE name = 'custom_instance_pg_databases'",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "jsonb_object_keys",
-        "type_info": "Text"
-      }
-    ],
-    "parameters": {
-      "Left": []
-    },
-    "nullable": [
-      null
-    ]
-  },
-  "hash": "b9842d2d8abf382bd82d8fa1de012373638be391f884f81dc387ffc465badac6"
-}

backend/.sqlx/query-c1e3ed3ecc3bcb98f60ba8196d33fee4a74f61b061e5025ecb75882208b3ba8f.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title, is_test)\n         VALUES ($1, $2, $3, $4, $5, $6)\n         ON CONFLICT (id) DO NOTHING\n         RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test",
+  "query": "INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title)\n         VALUES ($1, $2, $3, $4, $5)\n         ON CONFLICT (id) DO NOTHING\n         RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by",
   "describe": {
     "columns": [
       {
@@ -37,11 +37,6 @@
         "ordinal": 6,
         "name": "created_by",
         "type_info": "Varchar"
-      },
-      {
-        "ordinal": 7,
-        "name": "is_test",
-        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -50,8 +45,7 @@
         "Varchar",
         "Varchar",
         "Varchar",
-        "Varchar",
-        "Bool"
+        "Varchar"
       ]
     },
     "nullable": [
@@ -61,9 +55,8 @@
       true,
       false,
       false,
-      false,
       false
     ]
   },
-  "hash": "6d259b8cce5da5fecefe4ce322789b6b2cc43b51f2056c677d58f39c31fb26cb"
+  "hash": "c1e3ed3ecc3bcb98f60ba8196d33fee4a74f61b061e5025ecb75882208b3ba8f"
 }

backend/.sqlx/query-c383cc023714b361d10c10e8fef1fc148ab1da942951ee9ffdddaecee76a6be9.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test\n         FROM flow_conversation\n         WHERE id = $1 AND workspace_id = $2\n         FOR UPDATE",
+  "query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by\n         FROM flow_conversation\n         WHERE id = $1 AND workspace_id = $2",
   "describe": {
     "columns": [
       {
@@ -37,11 +37,6 @@
         "ordinal": 6,
         "name": "created_by",
         "type_info": "Varchar"
-      },
-      {
-        "ordinal": 7,
-        "name": "is_test",
-        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -57,9 +52,8 @@
       true,
       false,
       false,
-      false,
       false
     ]
   },
-  "hash": "dd84f9dfb238d18cb74f9e43228345427131bf8008eba6920021d6a449791534"
+  "hash": "c383cc023714b361d10c10e8fef1fc148ab1da942951ee9ffdddaecee76a6be9"
 }

backend/.sqlx/query-d48ca62c86b1af7a9dd2450c1c28dc45020a2a553d8874c49f9eafedea5a9d40.json

@@ -1,24 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT dt.key AS \"datatable!\"\n           FROM workspace_settings ws\n           CROSS JOIN LATERAL jsonb_each(COALESCE(ws.datatable->'datatables', '{}'::jsonb)) dt\n           WHERE ws.workspace_id = $1\n             AND dt.key <> $2\n             AND NOT dt.value ? 'permissions'\n             AND dt.value->'database'->>'resource_type' = 'instance'\n             AND dt.value->'database'->>'resource_path' = $3\n           ORDER BY dt.key",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "datatable!",
-        "type_info": "Text"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      null
-    ]
-  },
-  "hash": "d48ca62c86b1af7a9dd2450c1c28dc45020a2a553d8874c49f9eafedea5a9d40"
-}

backend/.sqlx/query-d631a26e5531589ff37e677f91a4f1d9f850e3e46c17130dd580426cda7a9f65.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "UPDATE token SET label = $1\n           WHERE email = $2 AND token_prefix = $3\n             AND (label IS NULL OR (\n                 label <> 'session'\n                 AND label <> 'guest_session'\n                 AND lower(label) NOT LIKE 'ephemeral%'\n                 AND label <> 'debugger-token'\n                 AND label NOT LIKE 'mcp-oauth-%'\n                 AND NOT starts_with(label, 'embed_app:')\n                 AND NOT starts_with(label, 'sdk_app:')\n                 AND NOT starts_with(label, 'impersonation:')\n                 AND NOT starts_with(label, 'cli-login:')\n             ))\n           RETURNING token_prefix",
+  "query": "UPDATE token SET label = $1\n           WHERE email = $2 AND token_prefix = $3\n             AND (label IS NULL OR (\n                 label <> 'session'\n                 AND label <> 'guest_session'\n                 AND lower(label) NOT LIKE 'ephemeral%'\n                 AND label <> 'debugger-token'\n                 AND label NOT LIKE 'mcp-oauth-%'\n                 AND NOT starts_with(label, 'embed_app:')\n                 AND NOT starts_with(label, 'sdk_app:')\n                 AND NOT starts_with(label, 'impersonation:')\n             ))\n           RETURNING token_prefix",
   "describe": {
     "columns": [
       {
@@ -20,5 +20,5 @@
       false
     ]
   },
-  "hash": "383c80239525d9c4ee90e2f5cb6e46c8a0983c52ad15db083f84a9e344dc4c99"
+  "hash": "d631a26e5531589ff37e677f91a4f1d9f850e3e46c17130dd580426cda7a9f65"
 }

backend/.sqlx/query-dd89d652154748d6d7e625e31778f6885d0ee62d29a4b8894a4b459dd215a103.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "\n        SELECT\n            j.args as \"args: Json<HashMap<String, Box<RawValue>>>\",\n            js.flow_status as \"flow_status: Json<windmill_common::flow_status::FlowStatus>\",\n            j.runnable_path\n        FROM v2_job_status js\n        INNER JOIN v2_job j ON j.id = js.id\n        WHERE js.id = $1\n        ",
+  "query": "\n        SELECT\n            j.args as \"args: Json<HashMap<String, Box<RawValue>>>\",\n            js.flow_status as \"flow_status: Json<windmill_common::flow_status::FlowStatus>\"\n        FROM v2_job_status js\n        INNER JOIN v2_job j ON j.id = js.id\n        WHERE js.id = $1\n        ",
   "describe": {
     "columns": [
       {
@@ -12,11 +12,6 @@
         "ordinal": 1,
         "name": "flow_status: Json<windmill_common::flow_status::FlowStatus>",
         "type_info": "Jsonb"
-      },
-      {
-        "ordinal": 2,
-        "name": "runnable_path",
-        "type_info": "Varchar"
       }
     ],
     "parameters": {
@@ -25,10 +20,9 @@
       ]
     },
     "nullable": [
-      true,
       true,
       true
     ]
   },
-  "hash": "9008f9abb70a9a07e38acb20bea6a710d0efd77dac4aedeb88d72240e816530b"
+  "hash": "dd89d652154748d6d7e625e31778f6885d0ee62d29a4b8894a4b459dd215a103"
 }

backend/.sqlx/query-e8802be9203c1e88a06e337260ccca029380139f89a01a89033e36a6ed9ac082.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments\n             FROM (\n                SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments\n                FROM flow_conversation_message\n                WHERE conversation_id = $1\n                ORDER BY created_seq DESC\n                LIMIT $2 OFFSET $3\n             ) AS messages\n             ORDER BY created_seq ASC\n             ",
+  "query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success\n             FROM flow_conversation_message\n             WHERE conversation_id = $1\n               AND created_seq > $2\n             ORDER BY created_seq ASC\n             LIMIT $3\n             ",
   "describe": {
     "columns": [
       {
@@ -58,26 +58,6 @@
         "ordinal": 8,
         "name": "success",
         "type_info": "Bool"
-      },
-      {
-        "ordinal": 9,
-        "name": "tool_arguments",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 10,
-        "name": "tool_result",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 11,
-        "name": "reasoning",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 12,
-        "name": "attachments",
-        "type_info": "Jsonb"
       }
     ],
     "parameters": {
@@ -96,12 +76,8 @@
       false,
       false,
       true,
-      false,
-      true,
-      true,
-      true,
-      true
+      false
     ]
   },
-  "hash": "a4a823f70b3dbe6aaf4a61c98345e94c5042fd5e6351fea139a66ecb1fb812ab"
+  "hash": "e8802be9203c1e88a06e337260ccca029380139f89a01a89033e36a6ed9ac082"
 }

backend/Cargo.lock

@@ -728,7 +728,7 @@ dependencies = [
  "futures-lite 2.6.1",
  "parking",
  "polling 3.11.0",
- "rustix 1.1.5",
+ "rustix 1.1.4",
  "slab",
  "windows-sys 0.61.2",
 ]
@@ -873,7 +873,7 @@ checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -1976,7 +1976,7 @@ dependencies = [
  "prettyplease 0.3.0",
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -2000,7 +2000,7 @@ dependencies = [
  "proc-macro-crate",
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -2144,7 +2144,7 @@ checksum = "6a1f896587b6f2c069c73d2f0913e2d590c3990285cd2f0b6aa02b786b4c679c"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -2164,9 +2164,9 @@ dependencies = [
 
 [[package]]
 name = "bytes-str"
-version = "0.2.9"
+version = "0.2.8"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4dde6d05e75a31ec9610eb6446a6f0a10dd30ff5100d720fee4c7c7a9008b5ba"
+checksum = "577d2bf5650f8554d5a372af5ac93535110a0fc75b3e702bb853369febf227c2"
 dependencies = [
  "bytes",
  "serde",
@@ -2338,9 +2338,9 @@ dependencies = [
 
 [[package]]
 name = "cfg-if"
-version = "1.0.5"
+version = "1.0.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
+checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
 
 [[package]]
 name = "cfg_aliases"
@@ -2454,7 +2454,7 @@ dependencies = [
  "heck",
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -3047,7 +3047,7 @@ dependencies = [
  "proc-macro2",
  "quote",
  "strsim 0.11.1",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -3102,7 +3102,7 @@ checksum = "2ac7135c3ef02b2f7833bbeb1be5ba7f966dcde8a87c6b87f65a778d71a02785"
 dependencies = [
  "darling_core 0.24.1",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -4580,7 +4580,7 @@ checksum = "e01a3366d27ee9890022452ee61b2b63a67e6f13f58900b651ff5665f0bb1fab"
 dependencies = [
  "libc",
  "option-ext",
- "redox_users 0.5.3",
+ "redox_users 0.5.2",
  "windows-sys 0.61.2",
 ]
 
@@ -4603,7 +4603,7 @@ checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -4850,7 +4850,7 @@ checksum = "a65863d15a4ce2888bd2f0f543cc963d3879c3a022c8ee43f6141d479a3ac815"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -5198,7 +5198,7 @@ version = "0.13.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "8640e34b88f7652208ce9e88b1a37a2ae95227d84abec377ccd3c5cfeb141ed4"
 dependencies = [
- "rustix 1.1.5",
+ "rustix 1.1.4",
  "windows-sys 0.59.0",
 ]
 
@@ -5319,7 +5319,7 @@ checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -9438,7 +9438,7 @@ dependencies = [
  "concurrent-queue",
  "hermit-abi 0.5.3",
  "pin-project-lite",
- "rustix 1.1.5",
+ "rustix 1.1.4",
  "windows-sys 0.61.2",
 ]
 
@@ -9574,7 +9574,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "2bfe0f4c752e450fc2faf62654f1c134747922825d5b04ca717b8874f41a40c0"
 dependencies = [
  "proc-macro2",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -10235,10 +10235,11 @@ dependencies = [
 
 [[package]]
 name = "redox_users"
-version = "0.5.3"
+version = "0.5.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "60dc65c0ff1a7ae1294b0c67b9f14baf70b644404010370171787bfac1038fc0"
+checksum = "a4e608c6638b9c18977b00b475ac1f28d14e84b27d8d42f70e0bf1e3dec127ac"
 dependencies = [
+ "getrandom 0.2.17",
  "libredox",
  "thiserror 2.0.20",
 ]
@@ -10260,7 +10261,7 @@ checksum = "92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -10579,7 +10580,7 @@ dependencies = [
  "proc-macro2",
  "quote",
  "serde_json",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -10797,9 +10798,9 @@ dependencies = [
 
 [[package]]
 name = "rustix"
-version = "1.1.5"
+version = "1.1.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
 dependencies = [
  "bitflags 2.13.2",
  "errno",
@@ -11216,7 +11217,7 @@ dependencies = [
  "proc-macro2",
  "quote",
  "serde_derive_internals 0.30.0",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -11415,7 +11416,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -11437,7 +11438,7 @@ checksum = "f852137cce035d6a4df67ccce505ff6b3e9fd3a10e3e52b24dc71e650bb1a9bd"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -11491,7 +11492,7 @@ checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -11559,7 +11560,7 @@ dependencies = [
  "darling 0.24.1",
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -12707,9 +12708,9 @@ dependencies = [
 
 [[package]]
 name = "syn"
-version = "3.0.6"
+version = "3.0.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
+checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -12744,7 +12745,7 @@ checksum = "901704edd0dfe137f1987838ee4f259e4e063c31371bdb423f7ae38ec6f77f02"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -13035,7 +13036,7 @@ dependencies = [
  "fastrand 2.5.0",
  "getrandom 0.4.3",
  "once_cell",
- "rustix 1.1.5",
+ "rustix 1.1.4",
  "windows-sys 0.61.2",
 ]
 
@@ -13054,7 +13055,7 @@ version = "0.4.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874"
 dependencies = [
- "rustix 1.1.5",
+ "rustix 1.1.4",
  "windows-sys 0.61.2",
 ]
 
@@ -13114,7 +13115,7 @@ checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -14074,7 +14075,7 @@ checksum = "f153acc4e99a5f2a5aefa09fb078be54e26271b2813f6041200b224c098d8328"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -14178,9 +14179,9 @@ checksum = "81b79ad29b5e19de4260020f8919b443b2ef0277d242ce532ec7b7a2cc8b6007"
 
 [[package]]
 name = "unicode-ident"
-version = "1.0.26"
+version = "1.0.24"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
+checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
 
 [[package]]
 name = "unicode-normalization"
@@ -14545,7 +14546,7 @@ dependencies = [
  "bumpalo",
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
  "wasm-bindgen-shared",
 ]
 
@@ -14588,7 +14589,7 @@ checksum = "8c89dcab8b516b6b603baca9d550b7282d68fcc7f367e3956cff7ebf406a3f12"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]
@@ -14793,7 +14794,7 @@ dependencies = [
 
 [[package]]
 name = "windmill"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-nats",
@@ -14881,7 +14882,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-ai"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "async-stream",
  "async-trait",
@@ -14895,7 +14896,6 @@ dependencies = [
  "eventsource-stream",
  "futures",
  "http 1.5.0",
- "indexmap 2.14.2",
  "lazy_static",
  "mime_guess",
  "reqwest 0.13.5",
@@ -14915,7 +14915,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-alerting"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -14928,7 +14928,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "argon2",
@@ -15068,7 +15068,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-agent-workers"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15091,7 +15091,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-assets"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15108,7 +15108,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-auth"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "axum 0.8.9",
@@ -15134,7 +15134,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-client"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "reqwest 0.12.28",
  "serde",
@@ -15144,7 +15144,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-configs"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15161,7 +15161,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-debug"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "base64 0.22.1",
@@ -15183,7 +15183,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-embeddings"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "axum 0.8.9",
@@ -15206,7 +15206,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-flow-conversations"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15222,7 +15222,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-flows"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15244,7 +15244,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-groups"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15266,7 +15266,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-inputs"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15280,7 +15280,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-integration-tests"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-nats",
@@ -15315,7 +15315,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-jobs"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "axum 0.8.9",
@@ -15340,7 +15340,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-npm-proxy"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15368,7 +15368,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-openapi"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "axum 0.8.9",
@@ -15390,7 +15390,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-schedule"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15410,7 +15410,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-scripts"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15448,7 +15448,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-settings"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "axum 0.8.9",
@@ -15477,7 +15477,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-sse"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "lazy_static",
  "serde",
@@ -15489,7 +15489,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-users"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "argon2",
  "axum 0.8.9",
@@ -15513,7 +15513,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-workers"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15527,7 +15527,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-api-workspaces"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "axum 0.8.9",
  "chrono",
@@ -15562,7 +15562,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-audit"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "chrono",
  "lazy_static",
@@ -15576,7 +15576,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-autoscaling"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "axum 0.8.9",
@@ -15595,7 +15595,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-common"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "aes-gcm",
  "aho-corasick",
@@ -15653,6 +15653,7 @@ dependencies = [
  "pin-project-lite",
  "pkcs1",
  "postgres-native-tls 0.5.3",
+ "postgres-protocol",
  "prometheus",
  "quick_cache",
  "rand 0.9.0",
@@ -15666,7 +15667,6 @@ dependencies = [
  "serde",
  "serde_json",
  "serde_yml",
- "sha1",
  "sha2 0.10.9",
  "size",
  "spki",
@@ -15702,7 +15702,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-dep-map"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "chrono",
  "futures",
@@ -15722,7 +15722,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-git-sync"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "regex",
  "serde",
@@ -15739,7 +15739,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-indexer"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "astral-tokio-tar",
@@ -15766,7 +15766,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-jseval"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "futures",
@@ -15783,7 +15783,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-macros"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "itertools 0.14.0",
  "lazy_static",
@@ -15799,7 +15799,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-mcp"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -15820,7 +15820,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-native-triggers"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -15851,7 +15851,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-oauth"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "arc-swap",
@@ -15876,7 +15876,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-object-store"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-stream",
@@ -15911,7 +15911,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-operator"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "futures",
@@ -15929,7 +15929,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "convert_case 0.6.0",
  "serde",
@@ -15938,7 +15938,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-bash"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -15950,7 +15950,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-csharp"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde_json",
@@ -15962,7 +15962,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-go"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "gosyn",
@@ -15974,7 +15974,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-graphql"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -15986,7 +15986,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-java"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde_json",
@@ -15998,7 +15998,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-nu"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "nu-parser",
@@ -16009,7 +16009,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-php"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "itertools 0.14.0",
@@ -16020,7 +16020,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-py"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "itertools 0.14.0",
@@ -16032,7 +16032,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-py-asset"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "rustpython-ast",
@@ -16043,7 +16043,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-py-imports"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-recursion",
@@ -16065,7 +16065,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-r"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde_json",
@@ -16077,7 +16077,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-ruby"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -16091,7 +16091,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-rust"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "convert_case 0.6.0",
@@ -16108,7 +16108,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-sql"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -16121,7 +16121,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-sql-asset"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde",
@@ -16133,7 +16133,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-ts"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -16151,7 +16151,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-ts-asset"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde-wasm-bindgen",
@@ -16167,7 +16167,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-wac"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "rustpython-ast",
@@ -16183,7 +16183,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-yaml"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -16197,7 +16197,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-queue"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-recursion",
@@ -16236,7 +16236,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-runtime-nativets"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "const_format",
@@ -16276,7 +16276,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-sql-datatype-parser-wasm"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "getrandom 0.3.4",
  "wasm-bindgen",
@@ -16287,7 +16287,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-store"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-recursion",
@@ -16322,7 +16322,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-test-utils"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16346,7 +16346,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16379,7 +16379,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-amqp"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16406,7 +16406,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-azure"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16439,7 +16439,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-email"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16459,7 +16459,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-gcp"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16493,7 +16493,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-http"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16529,7 +16529,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-kafka"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16552,7 +16552,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-mqtt"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16576,7 +16576,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-nats"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-nats",
@@ -16600,7 +16600,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-postgres"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16635,7 +16635,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-sqs"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16663,7 +16663,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-trigger-websocket"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-trait",
@@ -16688,7 +16688,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-types"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "bitflags 2.13.2",
@@ -16707,7 +16707,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-worker"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-once-cell",
@@ -16825,7 +16825,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-worker-volumes"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "bytes",
  "futures",
@@ -17458,7 +17458,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
 dependencies = [
  "libc",
- "rustix 1.1.5",
+ "rustix 1.1.4",
 ]
 
 [[package]]
@@ -17519,7 +17519,7 @@ checksum = "33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
  "synstructure 0.14.0",
 ]
 
@@ -17560,7 +17560,7 @@ checksum = "f75b4683f6c7f45248d4d64056a24298c6281e0993356d7d1b4a1a962ef10d4a"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
  "synstructure 0.14.0",
 ]
 
@@ -17616,7 +17616,7 @@ checksum = "34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da"
 dependencies = [
  "proc-macro2",
  "quote",
- "syn 3.0.6",
+ "syn 3.0.5",
 ]
 
 [[package]]

backend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "windmill"
-version = "1.814.0"
+version = "1.813.0"
 authors.workspace = true
 edition.workspace = true
 
@@ -88,7 +88,7 @@ members = [
 exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
 
 [workspace.package]
-version = "1.814.0"
+version = "1.813.0"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]
 edition = "2021"
 
@@ -624,6 +624,7 @@ wasm-bindgen-test = "^0"
 convert_case = "0.6.0"
 getrandom = "0.2"
 tokio-postgres = {version = "^0.7", features = ["array-impls", "with-serde_json-1", "with-chrono-0_4", "with-uuid-1", "with-bit-vec-0_6"]}
+postgres-protocol = "0.6"
 rust-postgres = { package = "tokio-postgres", git = "https://github.com/MaterializeInc/rust-postgres", rev = "78c1222577bb091d69bc22b1bc7ad01c14675abe"}
 rust-postgres-native-tls = { package = "postgres-native-tls", git = "https://github.com/MaterializeInc/rust-postgres", features = ["runtime"], rev = "78c1222577bb091d69bc22b1bc7ad01c14675abe" }
 bit-vec = "=0.6.3"

backend/ee-repo-ref.txt

@@ -1 +1 @@
-f2fced19fcae81de7f6dac545010ce404c052e1b
+151be033ea2bf2f768f381a8cd3b888bdd366cf9

backend/migrations/20260916144653_flow_conversation_message_tool_call.down.sql

@@ -1,3 +0,0 @@
-ALTER TABLE flow_conversation_message DROP COLUMN tool_arguments;
-ALTER TABLE flow_conversation_message DROP COLUMN tool_result;
-ALTER TABLE flow_conversation_message DROP COLUMN reasoning;

backend/migrations/20260916144653_flow_conversation_message_tool_call.up.sql

@@ -1,12 +0,0 @@
--- A chat is rebuilt from its rows without reading jobs, so every tool row carries its call:
--- the arguments the model wrote and the text the model got back, or what the call failed
--- with. A script or flow tool's job holds the args its input transforms produced, not the
--- model's; an MCP tool runs inside the agent's job, whose result lists every call of the
--- turn with nothing tying one to a row. A provider-native web search carries only its
--- citations, the provider never returning the query.
-ALTER TABLE flow_conversation_message ADD COLUMN tool_arguments TEXT;
-ALTER TABLE flow_conversation_message ADD COLUMN tool_result TEXT;
-
--- The thinking behind this row. The agent job keeps the turn's thinking as one string;
--- the rows keep it per iteration, next to the answer or tool call it led to.
-ALTER TABLE flow_conversation_message ADD COLUMN reasoning TEXT;

backend/migrations/20260916155738_flow_conversation_is_test.down.sql

@@ -1 +0,0 @@
-ALTER TABLE flow_conversation DROP COLUMN is_test;

backend/migrations/20260916155738_flow_conversation_is_test.up.sql

@@ -1,26 +0,0 @@
--- A chat run from the flow editor's test panel is stored exactly like one from the
--- deployed flow, so the two were indistinguishable once written. Marking them lets the
--- lists tell a trial apart from a real conversation.
-ALTER TABLE flow_conversation ADD COLUMN is_test BOOLEAN NOT NULL DEFAULT false;
-
--- Existing rows: a conversation whose messages came from a flowpreview run was a test.
--- Derived once here because the job is purged on retention, after which the origin of an
--- old conversation is unknowable.
---
--- Walked to the root job rather than matched directly: an existing message row never holds
--- the flow job itself. The rows point at the step that produced them — the AI agent's job
--- for an answer, the tool's own job for a tool call — whose kind is never 'flowpreview'.
---
--- `root_job` first, matching `get_root_job_id` (windmill-worker/src/common.rs): only it
--- reaches the top of the run. `flow_innermost_root_job` stops at the closest flow scope by
--- design, so an agent inside a subflow would land on that subflow's 'flow' row and the
--- conversation would read as deployed.
-UPDATE flow_conversation c
-SET is_test = true
-WHERE EXISTS (
-    SELECT 1 FROM flow_conversation_message m
-    JOIN v2_job j ON j.id = m.job_id
-    JOIN v2_job root
-      ON root.id = coalesce(j.root_job, j.flow_innermost_root_job, j.parent_job, j.id)
-    WHERE m.conversation_id = c.id AND root.kind = 'flowpreview'
-);

backend/migrations/20260917074040_flow_conversation_message_attachments.down.sql

@@ -1 +0,0 @@
-ALTER TABLE flow_conversation_message DROP COLUMN attachments;

backend/migrations/20260917074040_flow_conversation_message_attachments.up.sql

@@ -1,4 +0,0 @@
--- The files a user message carried, as object-storage references: `[{input, s3, storage?,
--- filename?}]`. Only references, never file bytes and never a presigned URL, so a
--- transcript can show a message's files without reading its run's args.
-ALTER TABLE flow_conversation_message ADD COLUMN attachments JSONB;

backend/migrations/20260917133824_datatable_role_cluster.down.sql

@@ -0,0 +1,22 @@
+-- Roles on the external cluster are live logins there; dropping the column would forget them.
+LOCK TABLE datatable_role;
+DO $$
+BEGIN
+    IF EXISTS (SELECT 1 FROM datatable_role WHERE cluster <> 'instance') THEN
+        RAISE EXCEPTION 'datatable_role holds roles on the external instance cluster. Delete them in instance settings first.';
+    END IF;
+    -- Before this, only data tables on Windmill's own cluster could be under roles, and a role
+    -- block left with just `admin` survives deleting every external role.
+    IF EXISTS (
+        SELECT 1 FROM workspace_settings ws,
+            jsonb_each(CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
+                THEN ws.datatable->'datatables' ELSE '{}'::jsonb END) dt
+        WHERE dt.value->'database'->>'resource_type' = 'external_instance'
+          AND dt.value ? 'permissions'
+    ) THEN
+        RAISE EXCEPTION 'external instance data tables are still under roles. Turn their roles off first.';
+    END IF;
+END $$;
+ALTER TABLE datatable_role DROP CONSTRAINT datatable_role_cluster_name_key;
+ALTER TABLE datatable_role ADD CONSTRAINT datatable_role_name_key UNIQUE (name);
+ALTER TABLE datatable_role DROP COLUMN cluster;

backend/migrations/20260917133824_datatable_role_cluster.up.sql

@@ -0,0 +1,8 @@
+-- A data table role is a Postgres login on one cluster: Windmill's own ('instance'), or the external
+-- instance cluster ('external_instance'). Role names are the cluster's own key, so they are unique
+-- per cluster rather than across the instance.
+ALTER TABLE datatable_role
+    ADD COLUMN cluster VARCHAR(20) NOT NULL DEFAULT 'instance'
+        CHECK (cluster IN ('instance', 'external_instance'));
+ALTER TABLE datatable_role DROP CONSTRAINT datatable_role_name_key;
+ALTER TABLE datatable_role ADD CONSTRAINT datatable_role_cluster_name_key UNIQUE (cluster, name);

backend/parsers/windmill-parser-wasm/Cargo.lock

@@ -6191,7 +6191,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
 
 [[package]]
 name = "windmill-common"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "aho-corasick",
  "anyhow",
@@ -6274,7 +6274,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-macros"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "proc-macro2",
  "quote",
@@ -6286,7 +6286,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "convert_case",
  "serde",
@@ -6295,7 +6295,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-bash"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -6307,7 +6307,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-csharp"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde_json",
@@ -6319,7 +6319,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-go"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "gosyn",
@@ -6331,7 +6331,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-graphql"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -6343,7 +6343,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-java"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde_json",
@@ -6355,7 +6355,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-nu"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "nu-parser",
@@ -6366,7 +6366,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-php"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "itertools 0.14.0",
@@ -6377,7 +6377,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-py"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "itertools 0.14.0",
@@ -6389,7 +6389,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-py-asset"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "rustpython-ast",
@@ -6400,7 +6400,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-py-imports"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "async-recursion",
@@ -6422,7 +6422,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-r"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde_json",
@@ -6434,7 +6434,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-ruby"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -6448,7 +6448,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-rust"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "convert_case",
@@ -6465,7 +6465,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-sql"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -6478,7 +6478,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-sql-asset"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde",
@@ -6490,7 +6490,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-ts"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -6508,7 +6508,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-ts-asset"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "serde-wasm-bindgen",
@@ -6524,7 +6524,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-wac"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "rustpython-ast",
@@ -6540,7 +6540,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-wasm"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "getrandom 0.2.17",
@@ -6572,7 +6572,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-parser-yaml"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "lazy_static",
@@ -6586,7 +6586,7 @@ dependencies = [
 
 [[package]]
 name = "windmill-types"
-version = "1.814.0"
+version = "1.813.0"
 dependencies = [
  "anyhow",
  "bitflags",

backend/parsers/windmill-parser-wasm/Cargo.toml

@@ -12,7 +12,7 @@ resolver = "2"
 members = ["."]
 
 [workspace.package]
-version = "1.814.0"
+version = "1.813.0"
 edition = "2021"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]
 

backend/src/monitor.rs

@@ -45,10 +45,7 @@ use windmill_common::otel_oss::{
 use windmill_common::{
     agent_workers::DECODED_AGENT_TOKEN,
     apps::APP_WORKSPACED_ROUTE,
-    auth::{
-        create_token_for_owner, ephemeral_script_token_label, job_token_expiry_secs,
-        TOKEN_EXPIRY_WARNING_DAYS,
-    },
+    auth::{create_token_for_owner, ephemeral_script_token_label, job_token_expiry_secs},
     ee_oss::CriticalErrorChannel,
     email_oss::send_email_if_possible,
     error,
@@ -2201,7 +2198,7 @@ async fn cleanup_scheduled_job_deletions(db: &Pool<Postgres>) {
 }
 
 pub async fn check_expiring_tokens(db: &DB) {
-    // Find tokens expiring within the warning window that still have a pending notification row.
+    // Find tokens expiring within 7 days that still have a pending notification row.
     // The notification table stores token_hash (not plaintext) so the join works
     // even after the hash migration makes token.token nullable.
     let expiring_tokens_r = sqlx::query_as!(
@@ -2210,9 +2207,8 @@ pub async fn check_expiring_tokens(db: &DB) {
          USING token t
          WHERE n.token_hash = t.token_hash
            AND n.expiration > now()
-           AND n.expiration <= now() + make_interval(days => $1)
+           AND n.expiration <= now() + interval '7 days'
          RETURNING t.token_prefix, t.label, t.email, t.workspace_id",
-        TOKEN_EXPIRY_WARNING_DAYS,
     )
     .fetch_all(db)
     .await;

backend/summarized_schema.txt

@@ -99,9 +99,9 @@ email_trigger: path(char), local_part(char), workspaced_local_part(bool), script
 favorite: usr(char), workspace_id(char), path(char), favorite_kind(favorite_kind)
 flow: workspace_id(char), path(char), summary(text), description(text), value(jsonb), edited_by(char), edited_at(ts), archived(bool), schema(json), extra_perms(jsonb), dependency_job(uuid), draft_only(bool), tag(char), ws_error_handler_muted(bool), dedicated_worker(bool), timeout(int), visible_to_runner_only(bool), concurrency_key(char), versions(bigint[]), on_behalf_of(varchar), on_behalf_of_email(text), lock_error_logs(text), labels(text[])
   FK: (workspace_id) -> workspace(id)
-flow_conversation: id(uuid), workspace_id(char), flow_path(char), title(char), created_at(ts), updated_at(ts), created_by(char), is_test(bool)
+flow_conversation: id(uuid), workspace_id(char), flow_path(char), title(char), created_at(ts), updated_at(ts), created_by(char)
   FK: (workspace_id) -> workspace(id)
-flow_conversation_message: id(uuid), conversation_id(uuid), message_type(message_type), content(text), job_id(uuid), created_at(ts), created_seq(int8), step_name(char), success(bool), tool_arguments(text), tool_result(text), reasoning(text), attachments(jsonb)
+flow_conversation_message: id(uuid), conversation_id(uuid), message_type(message_type), content(text), job_id(uuid), created_at(ts), created_seq(int8), step_name(char), success(bool)
   FK: (conversation_id) -> flow_conversation(id) | (job_id) -> v2_job(id)
 flow_iterator_data: job_id(uuid), itered(jsonb)
 flow_node: id(bigint), workspace_id(char), hash(bigint), path(char), lock(text), code(text), flow(jsonb), hash_v2(char(64))

backend/tests/flow_engine_parity.rs

@@ -3312,60 +3312,3 @@ export function main(i: number) {
 
     Ok(())
 }
-
-// A `$flow_expr[...]` step tag is resolved from the flow's state before the step is pushed, and
-// one that cannot be resolved fails the step instead of queueing it on a tag no worker serves.
-#[cfg(feature = "deno_core")]
-#[sqlx::test(fixtures("base"))]
-async fn test_flow_expr_step_tag(db: Pool<Postgres>) -> anyhow::Result<()> {
-    initialize_tracing().await;
-    let server = ApiServer::start(db.clone()).await?;
-
-    let step = |id: &str, tag: Option<&str>| {
-        flow_module(
-            id,
-            FlowModuleValue::RawScript {
-                input_transforms: Default::default(),
-                language: ScriptLang::Deno,
-                content: "export function main() { return { lang: 'bun' } }".to_string(),
-                path: None,
-                lock: None,
-                tag: tag.map(str::to_string),
-                concurrency_settings: Default::default(),
-                is_trigger: None,
-                assets: None,
-            },
-        )
-    };
-    let flow = FlowValue {
-        modules: vec![
-            step("a", None),
-            step("b", Some("$flow_expr[results.a.lang]")),
-            step("c", Some("nobody-serves-$flow_expr[a.lang]")),
-        ],
-        same_worker: false,
-        ..Default::default()
-    };
-
-    let job = RunJob::from(JobPayload::RawFlow { value: flow, path: None, restarted_from: None })
-        .run_until_complete(&db, false, server.addr.port())
-        .await;
-
-    let b_tag = sqlx::query_scalar::<_, String>(
-        "SELECT tag FROM v2_job WHERE parent_job = $1 AND flow_step_id = 'b'",
-    )
-    .bind(job.id)
-    .fetch_one(&db)
-    .await?;
-    assert_eq!(b_tag, "bun");
-
-    assert!(!job.success);
-    let result = job.json_result().unwrap();
-    let message = result["error"]["message"].as_str().unwrap_or_default();
-    assert!(
-        message.contains("Could not resolve the step tag `nobody-serves-$flow_expr[a.lang]`"),
-        "got {result:?}"
-    );
-
-    Ok(())
-}

backend/tests/instance_config.rs

@@ -1530,31 +1530,6 @@ async fn declarative_sync_rejects_an_unusable_instance_banner(db: Pool<Postgres>
     );
 }
 
-#[sqlx::test(fixtures("base"))]
-async fn declarative_sync_rejects_a_malformed_max_token_expiration(db: Pool<Postgres>) {
-    clear_settings_and_configs(&db).await;
-    let before = count_global_settings(&db).await;
-
-    let mut desired = BTreeMap::new();
-    desired.insert(
-        "max_token_expiration_days".to_string(),
-        serde_json::json!("7.0"),
-    );
-
-    let err = windmill_common::instance_config::sync_global_settings_declarative(
-        &db,
-        &BTreeMap::new(),
-        &desired,
-    )
-    .await
-    .expect_err("a ceiling the token routes cannot read must fail the sync");
-    assert!(
-        err.to_string().contains("max_token_expiration_days"),
-        "the error should name the offending setting, got: {err}"
-    );
-    assert_eq!(count_global_settings(&db).await, before);
-}
-
 #[sqlx::test(fixtures("base"))]
 async fn declarative_sync_rejects_an_unusable_default_allowed_origins(db: Pool<Postgres>) {
     // The declarative writers (the sync-config CLI, the operator's ConfigMap

backend/tests/list_jobs.rs

@@ -1199,63 +1199,3 @@ async fn test_wm_labels_from_result_merged_with_static_labels(
 
     Ok(())
 }
-
-/// `tag` lives only on `v2_job`, which count_jobs joins only when `tags` is set.
-#[sqlx::test(fixtures("base"))]
-async fn test_count_completed_jobs_tags_filter(db: Pool<Postgres>) -> anyhow::Result<()> {
-    initialize_tracing().await;
-
-    let server = ApiServer::start(db.clone()).await?;
-    let port = server.addr.port();
-    let client = windmill_api_client::create_client(
-        &format!("http://localhost:{port}"),
-        "SECRET_TOKEN".to_string(),
-    );
-
-    for (ws, tag, status) in [
-        ("test-workspace", "deno", "success"),
-        ("test-workspace", "deno", "failure"),
-        ("test-workspace", "python3", "success"),
-        ("other-workspace", "deno", "success"),
-    ] {
-        let id = uuid::Uuid::new_v4();
-        sqlx::query("INSERT INTO v2_job (id, workspace_id, tag) VALUES ($1, $2, $3)")
-            .bind(id)
-            .bind(ws)
-            .bind(tag)
-            .execute(&db)
-            .await?;
-        sqlx::query(
-            "INSERT INTO v2_job_completed (id, workspace_id, status, duration_ms) VALUES ($1, $2, $3::job_status, 0)",
-        )
-        .bind(id)
-        .bind(ws)
-        .bind(status)
-        .execute(&db)
-        .await?;
-    }
-
-    for (query, expected) in [
-        ("", 3),
-        ("tags=deno", 2),
-        ("tags=deno&success=true", 1),
-        ("tags=deno,python3&completed_after_s_ago=3600", 3),
-    ] {
-        let response = client
-            .client()
-            .get(format!(
-                "{}/w/test-workspace/jobs/completed/count_jobs?{query}",
-                client.baseurl()
-            ))
-            .send()
-            .await?;
-        assert!(
-            response.status().is_success(),
-            "{query}: {}",
-            response.text().await?
-        );
-        assert_eq!(response.json::<i64>().await?, expected, "{query}");
-    }
-
-    Ok(())
-}

backend/tests/v2_job_delete_orphans.rs

@@ -258,7 +258,6 @@ async fn test_new_turns_wait_for_conversation_cleanup_and_recreate(
                     "test-user",
                     "hi again",
                     conv_id,
-                    false,
                 )
                 .await?;
                 windmill_common::flow_conversations::add_message_to_conversation_tx(
@@ -269,7 +268,6 @@ async fn test_new_turns_wait_for_conversation_cleanup_and_recreate(
                     windmill_common::flow_conversations::MessageType::User,
                     None,
                     true,
-                    None,
                 )
                 .await?;
                 tx.commit().await?;

backend/windmill-ai/Cargo.toml

@@ -23,7 +23,6 @@ async-trait.workspace = true
 async-stream.workspace = true
 base64.workspace = true
 bytes.workspace = true
-indexmap.workspace = true
 eventsource-stream.workspace = true
 futures.workspace = true
 http.workspace = true

backend/windmill-ai/src/providers/bedrock.rs

@@ -1074,8 +1074,7 @@ impl BedrockQueryBuilder {
 
         let mut accumulated_text = String::new();
         let mut events_str = String::new();
-        let mut accumulated_tool_calls: indexmap::IndexMap<String, StreamingToolCall> =
-            indexmap::IndexMap::new();
+        let mut accumulated_tool_calls: HashMap<String, StreamingToolCall> = HashMap::new();
         let mut current_tool_use_id: Option<String> = None;
         let mut usage: Option<TokenUsage> = None;
         // Claude reasoning block for the turn (only populated when thinking is on),
@@ -1264,10 +1263,7 @@ mod tests {
         // recovers the uncached share by subtracting the details back out.
         assert_eq!(usage["usage"]["prompt_tokens"], 1010);
         assert_eq!(usage["usage"]["completion_tokens"], 7);
-        assert_eq!(
-            usage["usage"]["prompt_tokens_details"]["cached_tokens"],
-            900
-        );
+        assert_eq!(usage["usage"]["prompt_tokens_details"]["cached_tokens"], 900);
         assert_eq!(
             usage["usage"]["prompt_tokens_details"]["cache_write_tokens"],
             100

backend/windmill-ai/src/sse.rs

@@ -1,7 +1,6 @@
 use std::collections::HashMap;
 
 use eventsource_stream::Eventsource;
-use indexmap::IndexMap;
 use reqwest::Response;
 use serde::Deserialize;
 use tokio_stream::StreamExt;
@@ -138,9 +137,7 @@ pub struct OpenAISSEParser {
     pub accumulated_content: String,
     /// The thinking streamed before the answer, kept so it can be stored with it.
     pub accumulated_reasoning: String,
-    // Insertion-ordered in every parser: tool calls run and are persisted in the order the
-    // stream showed them, and a chat attaches a round's thinking to its first call.
-    pub accumulated_tool_calls: IndexMap<i64, OpenAIToolCall>,
+    pub accumulated_tool_calls: HashMap<i64, OpenAIToolCall>,
     pub events_str: String,
     pub stream_event_processor: Box<dyn StreamEventSink>,
     /// Token usage from final chunk (when stream_options.include_usage is true)
@@ -152,7 +149,7 @@ impl OpenAISSEParser {
         Self {
             accumulated_content: String::new(),
             accumulated_reasoning: String::new(),
-            accumulated_tool_calls: IndexMap::new(),
+            accumulated_tool_calls: HashMap::new(),
             events_str: String::new(),
             stream_event_processor,
             usage: None,
@@ -362,7 +359,7 @@ pub struct AnthropicSSEParser {
     pub accumulated_content: String,
     /// The thinking streamed before the answer, kept so it can be stored with it.
     pub accumulated_reasoning: String,
-    pub accumulated_tool_calls: IndexMap<i64, OpenAIToolCall>,
+    pub accumulated_tool_calls: HashMap<i64, OpenAIToolCall>,
     pub events_str: String,
     pub stream_event_processor: Box<dyn StreamEventSink>,
     /// Track content block types by index
@@ -385,7 +382,7 @@ impl AnthropicSSEParser {
         Self {
             accumulated_content: String::new(),
             accumulated_reasoning: String::new(),
-            accumulated_tool_calls: IndexMap::new(),
+            accumulated_tool_calls: HashMap::new(),
             events_str: String::new(),
             stream_event_processor,
             content_blocks: HashMap::new(),
@@ -604,7 +601,7 @@ pub struct GeminiSSEParser {
     pub accumulated_content: String,
     /// The thinking streamed before the answer, kept so it can be stored with it.
     pub accumulated_reasoning: String,
-    pub accumulated_tool_calls: IndexMap<i64, OpenAIToolCall>,
+    pub accumulated_tool_calls: HashMap<i64, OpenAIToolCall>,
     pub events_str: String,
     pub stream_event_processor: Box<dyn StreamEventSink>,
     tool_call_index: i64,
@@ -618,7 +615,7 @@ impl GeminiSSEParser {
         Self {
             accumulated_content: String::new(),
             accumulated_reasoning: String::new(),
-            accumulated_tool_calls: IndexMap::new(),
+            accumulated_tool_calls: HashMap::new(),
             events_str: String::new(),
             stream_event_processor,
             tool_call_index: 0,
@@ -836,7 +833,7 @@ pub struct OpenAIResponsesSSEParser {
     pub accumulated_content: String,
     /// The reasoning summary streamed before the answer, kept so it can be stored with it.
     pub accumulated_reasoning: String,
-    pub accumulated_tool_calls: IndexMap<String, OpenAIToolCall>,
+    pub accumulated_tool_calls: HashMap<String, OpenAIToolCall>,
     /// Maps item_id -> (name, call_id) for function calls
     tool_call_metadata: HashMap<String, (String, String)>,
     /// Maps item_id -> accumulated arguments
@@ -858,7 +855,7 @@ impl OpenAIResponsesSSEParser {
         Self {
             accumulated_content: String::new(),
             accumulated_reasoning: String::new(),
-            accumulated_tool_calls: IndexMap::new(),
+            accumulated_tool_calls: HashMap::new(),
             tool_call_metadata: HashMap::new(),
             tool_call_arguments: HashMap::new(),
             events_str: String::new(),

backend/windmill-ai/src/types.rs

@@ -78,50 +78,17 @@ impl Default for OutputType {
 #[serde(tag = "kind", rename_all = "lowercase")]
 pub enum Memory {
     Off,
-    Window {
-        #[serde(default, deserialize_with = "deserialize_null_as_zero")]
-        context_length: usize,
-    },
-    /// Written before `window`. Its `memory_id` stays a fallback behind the run's memory id.
     Auto {
-        #[serde(default, deserialize_with = "deserialize_null_as_zero")]
+        #[serde(default)]
         context_length: usize,
-        #[serde(default, deserialize_with = "deserialize_blank_as_none")]
+        #[serde(default)]
         memory_id: Option<Uuid>,
     },
-    /// Written before a step had history inputs of its own, and read on its own where it remains.
     Manual {
         messages: Vec<OpenAIMessage>,
     },
 }
 
-// An editor form can leave `""` in a legacy baked id it never filled; it means no id rather than
-// failing every run of the step.
-fn deserialize_blank_as_none<'de, D: serde::Deserializer<'de>>(
-    deserializer: D,
-) -> Result<Option<Uuid>, D::Error> {
-    match <Option<String> as serde::Deserialize>::deserialize(deserializer)? {
-        Some(id) if !id.trim().is_empty() => Uuid::parse_str(id.trim())
-            .map(Some)
-            .map_err(serde::de::Error::custom),
-        _ => Ok(None),
-    }
-}
-
-// A count the editor's number field was cleared of is stored as `null`, which `default` does not
-// cover; it reads as 0, memory off, rather than failing every run of the step.
-fn deserialize_null_as_zero<'de, D: serde::Deserializer<'de>>(
-    deserializer: D,
-) -> Result<usize, D::Error> {
-    <Option<usize> as serde::Deserialize>::deserialize(deserializer).map(Option::unwrap_or_default)
-}
-
-fn deserialize_present<'de, D: serde::Deserializer<'de>>(
-    deserializer: D,
-) -> Result<Option<serde_json::Value>, D::Error> {
-    <serde_json::Value as serde::Deserialize>::deserialize(deserializer).map(Some)
-}
-
 #[derive(Deserialize)]
 struct AIAgentArgsRaw {
     provider: ProviderWithResource,
@@ -136,12 +103,6 @@ struct AIAgentArgsRaw {
     streaming: Option<bool>,
     max_iterations: Option<usize>,
     memory: Option<Memory>,
-    // A null must stay distinguishable from an absent key: a step whose own memory id evaluates to
-    // nothing runs stateless instead of falling back to the run's memory id.
-    #[serde(default, deserialize_with = "deserialize_present")]
-    memory_id: Option<serde_json::Value>,
-    #[serde(default)]
-    previous_messages: Option<Vec<OpenAIMessage>>,
     enabled_tools: Option<Vec<String>>,
     // Legacy field for backward compatibility
     messages_context_length: Option<usize>,
@@ -163,10 +124,6 @@ pub struct AIAgentArgs {
     pub streaming: Option<bool>,
     pub max_iterations: Option<usize>,
     pub memory: Option<Memory>,
-    /// Memory id set on the step, overriding the run's. Empty when its expression produced none.
-    pub memory_id: Option<String>,
-    /// History supplied by the flow, replayed without reading or writing memory.
-    pub previous_messages: Option<Vec<OpenAIMessage>>,
     /// Which of the agent's tools this run may call; `narrow_roster` holds what the names are and
     /// what `None` means.
     pub enabled_tools: Option<Vec<String>>,
@@ -182,17 +139,12 @@ impl From<AIAgentArgsRaw> for AIAgentArgs {
         });
 
         // Backward compatibility: if context_length is 0, use off mode
-        let memory = memory.map(|memory| match memory {
-            Memory::Auto { context_length: 0, .. } | Memory::Window { context_length: 0 } => {
+        let memory = memory.map(|memory| {
+            if let Memory::Auto { context_length: 0, .. } = memory {
                 Memory::Off
+            } else {
+                memory
             }
-            memory => memory,
-        });
-
-        let memory_id = raw.memory_id.map(|value| match value {
-            serde_json::Value::Null => String::new(),
-            serde_json::Value::String(s) => s.trim().to_string(),
-            value => value.to_string(),
         });
 
         AIAgentArgs {
@@ -207,8 +159,6 @@ impl From<AIAgentArgsRaw> for AIAgentArgs {
             streaming: raw.streaming,
             max_iterations: raw.max_iterations,
             memory,
-            memory_id,
-            previous_messages: raw.previous_messages,
             enabled_tools: raw.enabled_tools,
             credentials_check: raw.credentials_check.unwrap_or(false),
         }

backend/windmill-api-auth/src/lib.rs

@@ -1249,10 +1249,6 @@ pub async fn register_token_expiry_notification(
     if !windmill_common::auth::is_user_token(label) {
         return;
     }
-    let warning_days = windmill_common::auth::TOKEN_EXPIRY_WARNING_DAYS;
-    if expiration <= chrono::Utc::now() + chrono::Duration::days(warning_days.into()) {
-        return;
-    }
     if let Err(e) = sqlx::query!(
         "INSERT INTO token_expiry_notification (token_hash, expiration) VALUES ($1, $2) ON CONFLICT DO NOTHING",
         token_hash,

backend/windmill-api-flow-conversations/src/lib.rs

@@ -1,6 +1,6 @@
 use axum::{
     extract::{Path, Query},
-    routing::{delete, get, post},
+    routing::{delete, get},
     Extension, Json, Router,
 };
 use chrono::{DateTime, Utc};
@@ -15,14 +15,13 @@ use windmill_common::{
     db::{UserDB, DB},
     error::{JsonResult, Result},
     flow_conversations::MessageType,
-    utils::{not_found_if_none, paginate, truncate_with_ellipsis, Pagination},
+    utils::{not_found_if_none, paginate, Pagination},
 };
 
 pub fn workspaced_service() -> Router {
     Router::new()
         .route("/list", get(list_conversations))
         .route("/delete/{conversation_id}", delete(delete_conversation))
-        .route("/update/{conversation_id}", post(update_conversation))
         .route("/{conversation_id}/messages", get(list_messages))
 }
 
@@ -37,37 +36,11 @@ pub struct FlowConversationMessage {
     pub created_seq: i64,
     pub step_name: Option<String>,
     pub success: bool,
-    /// On a tool row, the arguments the model wrote. For a Windmill tool these exclude the
-    /// inputs its step wires in. Null for a web search, whose query the provider does not
-    /// return.
-    pub tool_arguments: Option<String>,
-    /// On a tool row, the text the model got back, or what the call failed with; a web
-    /// search's citations.
-    pub tool_result: Option<String>,
-    /// On an answer, the thinking that produced it; on a tool row, the thinking that led to
-    /// the call. The agent job keeps the turn's thinking as one string.
-    pub reasoning: Option<String>,
-    /// The files a user message carried, as object-storage references
-    /// (`[{input, s3, storage?, filename?}]`).
-    pub attachments: Option<sqlx::types::JsonValue>,
-}
-
-/// Which conversations a listing holds. A test chat was started from the editor's test
-/// panel; a deployed one from the flow itself.
-#[derive(Deserialize, Default, Clone, Copy)]
-#[serde(rename_all = "lowercase")]
-pub enum ConversationKind {
-    Test,
-    /// The default: a deployed flow's chat should not surface someone's trial runs.
-    #[default]
-    Deployed,
-    All,
 }
 
 #[derive(Deserialize)]
 pub struct ListConversationsQuery {
     pub flow_path: Option<String>,
-    pub kind: Option<ConversationKind>,
 }
 
 #[derive(Deserialize)]
@@ -94,7 +67,6 @@ async fn list_conversations(
         "created_at",
         "updated_at",
         "created_by",
-        "is_test",
     ])
     .and_where_eq("workspace_id", "?".bind(&w_id));
 
@@ -102,16 +74,6 @@ async fn list_conversations(
         sqlb.and_where_eq("flow_path", "?".bind(flow_path));
     }
 
-    match query.kind.unwrap_or_default() {
-        ConversationKind::Test => {
-            sqlb.and_where_eq("is_test", "true");
-        }
-        ConversationKind::Deployed => {
-            sqlb.and_where_eq("is_test", "false");
-        }
-        ConversationKind::All => {}
-    }
-
     sqlb.order_by("updated_at", true)
         .limit(per_page as i64)
         .offset(offset as i64);
@@ -139,7 +101,7 @@ async fn delete_conversation(
     // Verify the conversation exists and belongs to the user
     let conversation = sqlx::query_as!(
         FlowConversation,
-        "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test
+        "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by
          FROM flow_conversation
          WHERE id = $1 AND workspace_id = $2",
         conversation_id,
@@ -186,50 +148,6 @@ async fn delete_conversation(
     Ok(format!("Conversation {} deleted", conversation_id))
 }
 
-#[derive(Deserialize)]
-pub struct UpdateConversation {
-    pub title: String,
-}
-
-async fn update_conversation(
-    authed: ApiAuthed,
-    Extension(user_db): Extension<UserDB>,
-    Path((w_id, conversation_id)): Path<(String, Uuid)>,
-    Json(update): Json<UpdateConversation>,
-) -> Result<String> {
-    // Postgres refuses a NUL in a text column, so it must not reach the query as a 500.
-    if update.title.contains('\0') {
-        return Err(windmill_common::error::Error::BadRequest(
-            "title cannot contain a NUL character".to_string(),
-        ));
-    }
-    // The column is VARCHAR(255) and the helper appends an ellipsis to what it cuts, so the
-    // bound it takes is three short of the column's. A longer title would otherwise reach
-    // Postgres as a 22001 and come back a 500.
-    let title = truncate_with_ellipsis(update.title.trim(), 252);
-
-    let mut tx = user_db.clone().begin(&authed).await?;
-
-    // `updated_at` is kept: the list is ordered by it, and a rename must not move the
-    // chat to the top the way a new turn does.
-    let updated = sqlx::query_scalar!(
-        "UPDATE flow_conversation SET title = $1, updated_at = updated_at
-         WHERE id = $2 AND workspace_id = $3
-         RETURNING id",
-        title,
-        conversation_id,
-        &w_id
-    )
-    .fetch_optional(&mut *tx)
-    .await?;
-
-    not_found_if_none(updated, "Conversation", conversation_id.to_string())?;
-
-    tx.commit().await?;
-
-    Ok(format!("Conversation {} updated", conversation_id))
-}
-
 async fn list_messages(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
@@ -260,7 +178,7 @@ async fn list_messages(
     let messages = if let Some(after_seq) = query.after_seq {
         sqlx::query_as!(
             FlowConversationMessage,
-            r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments
+            r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success
              FROM flow_conversation_message
              WHERE conversation_id = $1
                AND created_seq > $2
@@ -277,9 +195,9 @@ async fn list_messages(
         // Fetch messages for this conversation, oldest first, but reverse the order of the messages for easy rendering on the frontend
         sqlx::query_as!(
             FlowConversationMessage,
-            r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments
+            r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success
              FROM (
-                SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments
+                SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success
                 FROM flow_conversation_message
                 WHERE conversation_id = $1
                 ORDER BY created_seq DESC

backend/windmill-api-integration-tests/tests/datatable_acl.rs

@@ -0,0 +1,108 @@
+//! Who may read and change a data table's grants and owners. On the Enterprise Edition: its
+//! administrators, from the workspace that governs it. Without it: nobody. Each refusal is decided
+//! before anything connects to the data table, so the fixture's database never has to exist.
+
+use serde_json::{json, Value};
+use sqlx::{Pool, Postgres};
+
+use windmill_test_utils::*;
+
+fn grant_select_on_public() -> Value {
+    json!({
+        "target": {"kind": "schema", "schema": "public"},
+        "change": {"type": "grant", "role": "analytics", "privileges": ["SELECT"],
+                   "scope": "all_tables"},
+        "statements": [r#"GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO "analytics""#]
+    })
+}
+
+async fn post_acl(
+    port: u16,
+    w_id: &str,
+    action: &str,
+    token: &str,
+) -> anyhow::Result<reqwest::Response> {
+    Ok(reqwest::Client::new()
+        .post(format!(
+            "http://localhost:{port}/api/w/{w_id}/workspaces/datatable_acl/main/{action}"
+        ))
+        .header("Authorization", format!("Bearer {token}"))
+        .json(&grant_select_on_public())
+        .send()
+        .await?)
+}
+
+/// A fork reaches the data table through a pointer: it may use it, never change what each role may
+/// touch on it — not even as an admin of the fork.
+#[cfg(all(feature = "private", feature = "enterprise"))]
+#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
+async fn a_fork_cannot_change_access_on_the_data_table_it_points_at(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    for action in ["plan", "apply"] {
+        let resp = post_acl(port, "wm-fork-dt", action, "SECRET_TOKEN_2").await?;
+        assert_eq!(resp.status(), 401, "{action}: {}", resp.text().await?);
+    }
+    Ok(())
+}
+
+#[cfg(all(feature = "private", feature = "enterprise"))]
+#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
+async fn a_member_who_is_not_an_admin_cannot_change_access(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    for action in ["plan", "apply"] {
+        let resp = post_acl(port, "test-workspace", action, "SECRET_TOKEN_2").await?;
+        assert_eq!(resp.status(), 401, "{action}: {}", resp.text().await?);
+    }
+    Ok(())
+}
+
+/// Not even reading, and not even on a data table that is not under roles — which any member
+/// reaches, so only the edition stands between them and the instance's credentials.
+#[cfg(not(all(feature = "private", feature = "enterprise")))]
+#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
+async fn only_the_enterprise_edition_has_the_access_editor(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    sqlx::query(
+        "UPDATE workspace_settings
+         SET datatable = datatable #- '{datatables,main,permissions}'
+         WHERE workspace_id = 'test-workspace'",
+    )
+    .execute(&db)
+    .await?;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+
+    let read = reqwest::Client::new()
+        .get(format!(
+            "http://localhost:{port}/api/w/test-workspace/workspaces/datatable_acl/main?kind=database"
+        ))
+        .header("Authorization", "Bearer SECRET_TOKEN")
+        .send()
+        .await?;
+    let mut responses = vec![("read", read)];
+    for action in ["plan", "apply"] {
+        responses.push((
+            action,
+            post_acl(port, "test-workspace", action, "SECRET_TOKEN").await?,
+        ));
+    }
+    for (action, resp) in responses {
+        assert_eq!(resp.status(), 400, "{action}");
+        let body = resp.text().await?;
+        assert!(
+            body.contains("Data table roles are a Windmill Enterprise Edition feature"),
+            "{action}: {body}"
+        );
+    }
+    Ok(())
+}

backend/windmill-api-integration-tests/tests/datatable_roles.rs

@@ -391,7 +391,11 @@ async fn concurrent_role_creations_both_survive(db: Pool<Postgres>) -> anyhow::R
         assert_eq!(a.0, 200, "{}", a.1);
         assert_eq!(b.0, 200, "{}", b.1);
 
-        let catalog = windmill_common::datatable_roles::read_role_catalog(&db).await?;
+        let catalog = windmill_common::datatable_roles::read_role_catalog(
+            &db,
+            windmill_common::datatable_roles::DatatableRoleCluster::Instance,
+        )
+        .await?;
         let recorded: Vec<&str> = catalog.values().map(|r| r.name.as_str()).collect();
         for name in &names {
             assert!(
@@ -460,7 +464,11 @@ async fn a_role_delete_that_fails_part_way_leaves_the_role_disabled(
         let body = resp.text().await?;
         assert_eq!(status, 400, "{body}");
 
-        let catalog = windmill_common::datatable_roles::read_role_catalog(&db).await?;
+        let catalog = windmill_common::datatable_roles::read_role_catalog(
+            &db,
+            windmill_common::datatable_roles::DatatableRoleCluster::Instance,
+        )
+        .await?;
         let role = catalog
             .get(&id)
             .expect("a failed delete keeps the entry to retry");
@@ -912,6 +920,18 @@ async fn a_stored_name_containing_a_question_mark_resolves_as_itself(
         resolve("main?dt").await.is_err(),
         "an unknown parameter was ignored"
     );
+
+    sqlx::query(
+        "UPDATE workspace_settings
+         SET datatable = jsonb_set(datatable, '{datatables,main?role=analytics}', datatable->'datatables'->'main')
+         WHERE workspace_id = 'test-workspace'",
+    )
+    .execute(&db)
+    .await?;
+    assert!(
+        resolve("main?role=analytics").await.is_err(),
+        "a reference naming both a stored data table and a role on another resolved to one of them"
+    );
     Ok(())
 }
 
@@ -1009,6 +1029,70 @@ async fn an_entry_without_roles_cannot_newly_reach_a_database_under_roles(
     Ok(())
 }
 
+/// Browsing names the role it connects as, and a role the caller may not use is refused rather
+/// than quietly listed as the default. The refusal is decided before connecting, so the fixture's
+/// database never has to exist.
+#[cfg(all(feature = "private", feature = "enterprise"))]
+#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
+async fn browsing_as_a_role_the_caller_may_not_use_is_refused(
+    db: Pool<Postgres>,
+) -> anyhow::Result<()> {
+    initialize_tracing().await;
+    let server = ApiServer::start(db.clone()).await?;
+    let port = server.addr.port();
+    let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
+
+    // `test-user-2` is a tenant of `analytics` only.
+    let resp = authed(
+        client().get(format!(
+            "{base}/list_datatable_tables?role_for=main&role=admin"
+        )),
+        "SECRET_TOKEN_2",
+    )
+    .send()
+    .await?;
+    assert_eq!(resp.status(), 200);
+    let body: Value = resp.json().await?;
+    let entry = body
+        .as_array()
+        .and_then(|a| a.iter().find(|e| e["datatable_name"] == "main"))
+        .expect("main is listed");
+    assert_eq!(entry["usable_roles"], json!(["analytics"]), "{entry}");
+    assert_eq!(entry["default_role"], "analytics", "{entry}");
+    assert_eq!(entry["permissioned"], true, "{entry}");
+    assert_eq!(entry["instance"], true, "{entry}");
+    let error = entry["error"].as_str().unwrap_or_default();
+    assert!(
+        error.contains("Not allowed to use role 'admin'"),
+        "listed as another role than the one asked for: {entry}"
+    );
+
+    let resp = authed(
+        client().get(format!(
+            "{base}/get_datatable_table_schema?datatable_name=main&schema_name=public&table_name=t&role=admin"
+        )),
+        "SECRET_TOKEN_2",
+    )
+    .send()
+    .await?;
+    let status = resp.status();
+    let text = resp.text().await?;
+    assert!(
+        text.contains("Not allowed to use role 'admin'"),
+        "{status}: {text}"
+    );
+
+    // A role means nothing without the data table it belongs to.
+    let resp = authed(
+        client().get(format!("{base}/list_datatable_tables?role=analytics")),
+        "SECRET_TOKEN_2",
+    )
+    .send()
+    .await?;
+    assert_eq!(resp.status(), 400, "{}", resp.text().await?);
+    Ok(())
+}
+
 #[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
 async fn an_alias_saved_elsewhere_waits_for_roles_going_on_for_its_database(
     db: Pool<Postgres>,

backend/windmill-api-integration-tests/tests/fork_clone_on_behalf_of.rs

@@ -311,121 +311,3 @@ async fn test_fork_keeps_only_resolvable_on_behalf_of(db: Pool<Postgres>) -> any
 
     Ok(())
 }
-
-/// Apps, schedules, triggers and their drafts cannot drop an identity the way scripts and flows
-/// do, so one naming nobody in the fork goes to its creator while one that still resolves stays.
-/// Forked as an admin, whose app policies the clone otherwise keeps.
-#[sqlx::test(migrations = "../migrations", fixtures("base"))]
-async fn test_fork_repoints_unresolvable_identities(db: Pool<Postgres>) -> anyhow::Result<()> {
-    initialize_tracing().await;
-
-    let server = ApiServer::start(db.clone()).await?;
-    let base_url = format!("http://localhost:{}/api", server.addr.port());
-
-    let stranger = json!({
-        "on_behalf_of": "u/test-user-2",
-        "on_behalf_of_email": "test2@windmill.dev",
-        "execution_mode": "publisher",
-    });
-    sqlx::query(
-        "INSERT INTO app (workspace_id, path, summary, policy, versions)
-         VALUES ('test-workspace', 'u/test-user/stranger', '', $1, '{}'),
-                ('test-workspace', 'u/test-user/group', '', $2, '{}')",
-    )
-    .bind(&stranger)
-    .bind(json!({
-        "on_behalf_of": "g/all",
-        "on_behalf_of_email": "group-all@windmill.dev",
-        "execution_mode": "publisher",
-    }))
-    .execute(&db)
-    .await?;
-    // The clone re-aggregates `versions` from `app_version`, and the column is NOT NULL.
-    sqlx::query(
-        "WITH v AS (
-            INSERT INTO app_version (app_id, value, created_by)
-            SELECT id, '{}'::json, 'test-user' FROM app WHERE workspace_id = 'test-workspace'
-            RETURNING id, app_id
-         )
-         UPDATE app SET versions = ARRAY[v.id] FROM v WHERE app.id = v.app_id",
-    )
-    .execute(&db)
-    .await?;
-    sqlx::query(
-        "INSERT INTO draft (workspace_id, path, typ, value, created_at, email)
-         VALUES ('test-workspace', 'u/test-user/stranger', 'raw_app', $1::json, NOW(), 'test@windmill.dev'),
-                ('test-workspace', 'u/test-user/stranger', 'trigger_websocket', $2::json, NOW(), 'test@windmill.dev'),
-                ('test-workspace', 'u/test-user/nul', 'raw_app', $3::json, NOW(), 'test@windmill.dev')",
-    )
-    .bind(json!({ "policy": stranger }))
-    .bind(json!({ "permissioned_as": "u/test-user-2" }))
-    // Saved before drafts were stripped of NULs: any jsonb parse of it raises, so it must be
-    // skipped rather than abort the fork. Built from parts because a NUL escape can't sit in source.
-    .bind(format!(
-        r#"{{"policy":{{"on_behalf_of":"u/test-user-2"}},"files":{{"f":"a{}u0000"}}}}"#,
-        "\\"
-    ))
-    .execute(&db)
-    .await?;
-    sqlx::query(
-        "INSERT INTO schedule (workspace_id, path, edited_by, schedule, script_path, email, permissioned_as, enabled)
-         VALUES ('test-workspace', 'u/test-user/stranger', 'test-user', '0 0 * * * *', 'u/test-user/s', 'test2@windmill.dev', 'u/test-user-2', false)",
-    )
-    .execute(&db)
-    .await?;
-    sqlx::query(
-        "INSERT INTO websocket_trigger (workspace_id, path, url, script_path, is_flow, edited_by, permissioned_as, mode)
-         VALUES ('test-workspace', 'u/test-user/stranger', 'ws://localhost', 'u/test-user/s', false, 'test-user', 'u/test-user-2', 'disabled')",
-    )
-    .execute(&db)
-    .await?;
-
-    let resp = reqwest::Client::new()
-        .post(format!(
-            "{base_url}/w/test-workspace/workspaces/create_fork"
-        ))
-        .header("Authorization", "Bearer SECRET_TOKEN")
-        .json(&json!({ "id": "wm-fork-repoint", "name": "Fork", "color": "#0000ff" }))
-        .send()
-        .await?;
-    assert!(
-        resp.status().is_success(),
-        "creating the fork: {}",
-        resp.text().await?
-    );
-
-    let text = |sql: &'static str| sqlx::query_scalar::<_, String>(sql).fetch_one(&db);
-    assert_eq!(
-        text("SELECT (policy->>'on_behalf_of') || ' ' || (policy->>'on_behalf_of_email') FROM app WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/stranger'").await?,
-        "u/test-user test@windmill.dev"
-    );
-    assert_eq!(
-        text("SELECT policy->>'on_behalf_of' FROM app WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/group'").await?,
-        "g/all"
-    );
-    assert_eq!(
-        text("SELECT value->'policy'->>'on_behalf_of' FROM draft WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/stranger' AND typ = 'raw_app'").await?,
-        "u/test-user"
-    );
-    assert_eq!(
-        text("SELECT CASE WHEN strpos(value::text, 'u/test-user-2') > 0 THEN 'kept' ELSE 'rewritten' END FROM draft WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/nul'").await?,
-        "kept"
-    );
-    assert_eq!(
-        text("SELECT value->>'permissioned_as' FROM draft WHERE workspace_id = 'wm-fork-repoint' AND typ = 'trigger_websocket'").await?,
-        "u/test-user"
-    );
-    assert_eq!(
-        text("SELECT permissioned_as || ' ' || email FROM schedule WHERE workspace_id = 'wm-fork-repoint'").await?,
-        "u/test-user test@windmill.dev"
-    );
-    assert_eq!(
-        text(
-            "SELECT permissioned_as FROM websocket_trigger WHERE workspace_id = 'wm-fork-repoint'"
-        )
-        .await?,
-        "u/test-user"
-    );
-
-    Ok(())
-}

backend/windmill-api-integration-tests/tests/login_link.rs

@@ -115,60 +115,6 @@ async fn login_link_is_single_use_and_same_origin(db: Pool<Postgres>) -> anyhow:
     Ok(())
 }
 
-#[sqlx::test(migrations = "../migrations", fixtures("base"))]
-async fn confirmed_login_link_is_spent_by_the_click_not_the_page(
-    db: Pool<Postgres>,
-) -> anyhow::Result<()> {
-    initialize_tracing().await;
-    let server = ApiServer::start(db.clone()).await?;
-    let port = server.addr.port();
-    let base = format!("http://localhost:{port}/api");
-
-    let resp = client()
-        .post(format!("{base}/users/login_links"))
-        .header("Authorization", "Bearer SECRET_TOKEN")
-        .json(&json!({"email": "test2@windmill.dev", "confirm": true}))
-        .send()
-        .await?;
-    assert_eq!(resp.status(), 201);
-    let link = resp.json::<serde_json::Value>().await?;
-    // The URL handed out is the frontend page, not the API path that signs in on a GET.
-    let token = link["url"]
-        .as_str()
-        .unwrap()
-        .split_once("/user/login_link?token=")
-        .expect("confirmation page url")
-        .1
-        .to_string();
-
-    let confirm = || {
-        client()
-            .post(format!("{base}/auth/login_link/{token}"))
-            .send()
-    };
-    let resp = confirm().await?;
-    assert_eq!(resp.status(), 200);
-    assert!(resp
-        .headers()
-        .get_all("set-cookie")
-        .iter()
-        .any(|c| c.to_str().unwrap().starts_with("token=")));
-    assert_eq!(
-        resp.json::<serde_json::Value>().await?["location"],
-        "/user/workspaces"
-    );
-
-    let resp = confirm().await?;
-    assert_eq!(resp.status(), 200);
-    assert!(resp.headers().get("set-cookie").is_none());
-    assert_eq!(
-        resp.json::<serde_json::Value>().await?["location"],
-        "/user/login_link_expired?reason=used"
-    );
-
-    Ok(())
-}
-
 #[sqlx::test(migrations = "../migrations", fixtures("base"))]
 async fn login_link_mint_can_require_a_login_type(db: Pool<Postgres>) -> anyhow::Result<()> {
     initialize_tracing().await;

backend/windmill-api-integration-tests/tests/max_token_expiration.rs

@@ -1,239 +0,0 @@
-//! `max_token_expiration_days`: the instance-wide ceiling on how far ahead a token a caller
-//! picks the lifetime of may expire, and the service-account exemption.
-
-use serde_json::json;
-use sqlx::types::chrono::{DateTime, Utc};
-use sqlx::{Pool, Postgres};
-use windmill_test_utils::*;
-
-const DAY: u64 = 24 * 60 * 60;
-
-fn client() -> reqwest::Client {
-    reqwest::Client::new()
-}
-
-fn from_now(secs: u64) -> DateTime<Utc> {
-    Utc::now() + std::time::Duration::from_secs(secs)
-}
-
-async fn set_max(db: &Pool<Postgres>, value: serde_json::Value) {
-    sqlx::query(
-        "INSERT INTO global_settings (name, value) VALUES ('max_token_expiration_days', $1)
-         ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
-    )
-    .bind(value)
-    .execute(db)
-    .await
-    .unwrap();
-}
-
-/// Mints as `test2@windmill.dev`, a plain member of `test-workspace`.
-async fn create_token(port: u16, body: serde_json::Value) -> reqwest::Response {
-    client()
-        .post(format!("http://localhost:{port}/api/users/tokens/create"))
-        .header("Authorization", "Bearer SECRET_TOKEN_2")
-        .json(&body)
-        .send()
-        .await
-        .unwrap()
-}
-
-async fn stored_expiration(db: &Pool<Postgres>, label: &str) -> Option<DateTime<Utc>> {
-    sqlx::query_scalar::<_, Option<DateTime<Utc>>>("SELECT expiration FROM token WHERE label = $1")
-        .bind(label)
-        .fetch_one(db)
-        .await
-        .unwrap()
-}
-
-#[sqlx::test(migrations = "../migrations", fixtures("base"))]
-async fn test_max_token_expiration_days_shortens_user_tokens(
-    db: Pool<Postgres>,
-) -> anyhow::Result<()> {
-    initialize_tracing().await;
-    let server = ApiServer::start(db.clone()).await?;
-    let port = server.addr.port();
-
-    let resp = create_token(port, json!({ "label": "unset" })).await;
-    assert_eq!(resp.status(), 201);
-    assert_eq!(
-        stored_expiration(&db, "unset").await,
-        None,
-        "with no setting a token may still have no expiration"
-    );
-
-    // Refused at write time: the token routes can only read a value they cannot parse as no
-    // ceiling at all.
-    let resp = client()
-        .post(format!(
-            "http://localhost:{port}/api/settings/global/max_token_expiration_days"
-        ))
-        .header("Authorization", "Bearer SECRET_TOKEN")
-        .json(&json!({ "value": 7.5 }))
-        .send()
-        .await?;
-    assert_eq!(resp.status(), 400);
-
-    set_max(&db, json!(7)).await;
-
-    // The token form reads the ceiling as whoever is creating the token, usually not a
-    // superadmin, so it can offer only expirations the server would keep.
-    let resp = client()
-        .get(format!(
-            "http://localhost:{port}/api/settings/global/max_token_expiration_days"
-        ))
-        .header("Authorization", "Bearer SECRET_TOKEN_2")
-        .send()
-        .await?;
-    assert_eq!(resp.status(), 200);
-    assert_eq!(resp.text().await?, "7");
-
-    let resp = create_token(port, json!({ "label": "none asked" })).await;
-    assert_eq!(resp.status(), 201);
-    let expiration = stored_expiration(&db, "none asked")
-        .await
-        .expect("a token asking for no expiration gets the ceiling");
-    assert!(
-        expiration > from_now(6 * DAY) && expiration <= from_now(7 * DAY),
-        "expected the 7 day ceiling, got {expiration}"
-    );
-
-    let resp = create_token(
-        port,
-        json!({ "label": "past the ceiling", "expiration": from_now(30 * DAY) }),
-    )
-    .await;
-    assert_eq!(resp.status(), 201);
-    let expiration = stored_expiration(&db, "past the ceiling").await.unwrap();
-    assert!(
-        expiration > from_now(6 * DAY) && expiration <= from_now(7 * DAY),
-        "expected an expiration past the ceiling to be shortened to it, got {expiration}"
-    );
-
-    let resp = create_token(
-        port,
-        json!({ "label": "within", "expiration": from_now(3 * DAY) }),
-    )
-    .await;
-    assert_eq!(resp.status(), 201);
-    let expiration = stored_expiration(&db, "within").await.unwrap();
-    assert!(
-        expiration <= from_now(3 * DAY),
-        "an expiration within the ceiling must be kept, got {expiration}"
-    );
-
-    // The settings UI stores an integer, but the YAML instance config and config sync can write
-    // the same whole number as a string or as `5.0`. Reading either as "unset" would silently
-    // drop the ceiling, while the token form (`parseMaxTokenExpirationDays`) still showed it.
-    for (stored, label) in [
-        (json!("5"), "string setting"),
-        (json!(5.0), "float setting"),
-    ] {
-        set_max(&db, stored).await;
-        let resp = create_token(port, json!({ "label": label })).await;
-        assert_eq!(resp.status(), 201);
-        let expiration = stored_expiration(&db, label).await;
-        assert!(
-            expiration.is_some_and(|e| e > from_now(4 * DAY) && e <= from_now(5 * DAY)),
-            "{label}: expected the 5 day ceiling, got {expiration:?}"
-        );
-    }
-
-    // A superadmin impersonating a user picks the lifetime too, so the ceiling applies there;
-    // left out, it would be the one way to mint a token that never expires.
-    let resp = client()
-        .post(format!(
-            "http://localhost:{port}/api/users/tokens/impersonate"
-        ))
-        .header("Authorization", "Bearer SECRET_TOKEN")
-        .json(&json!({ "label": "impersonated", "impersonate_email": "test3@windmill.dev" }))
-        .send()
-        .await?;
-    assert_eq!(resp.status(), 201);
-    assert!(
-        stored_expiration(&db, "impersonated").await.is_some(),
-        "an impersonation token asking for no expiration gets the ceiling"
-    );
-
-    Ok(())
-}
-
-#[sqlx::test(migrations = "../migrations", fixtures("base"))]
-async fn test_service_accounts_are_exempt(db: Pool<Postgres>) -> anyhow::Result<()> {
-    initialize_tracing().await;
-    let server = ApiServer::start(db.clone()).await?;
-    let port = server.addr.port();
-
-    set_max(&db, json!(7)).await;
-    // The same email is a service account in one workspace and an ordinary user in another.
-    sqlx::query(
-        "UPDATE usr SET is_service_account = true
-         WHERE email = 'test2@windmill.dev' AND workspace_id = 'test-workspace'",
-    )
-    .execute(&db)
-    .await?;
-    sqlx::query("INSERT INTO workspace (id, name, owner) VALUES ('other', 'other', 'test-user')")
-        .execute(&db)
-        .await?;
-    sqlx::query("INSERT INTO workspace_settings (workspace_id) VALUES ('other')")
-        .execute(&db)
-        .await?;
-    sqlx::query(
-        "INSERT INTO usr (workspace_id, email, username, is_admin, role)
-         VALUES ('other', 'test2@windmill.dev', 'test-user-2', false, 'User')",
-    )
-    .execute(&db)
-    .await?;
-
-    // The token form decides the exemption from this list, so it must carry each membership's flag.
-    let memberships: serde_json::Value = client()
-        .get(format!("http://localhost:{port}/api/workspaces/users"))
-        .header("Authorization", "Bearer SECRET_TOKEN_2")
-        .send()
-        .await?
-        .json()
-        .await?;
-    for (workspace, is_service_account) in [("test-workspace", true), ("other", false)] {
-        let membership = memberships["workspaces"]
-            .as_array()
-            .and_then(|ws| ws.iter().find(|w| w["id"] == workspace))
-            .unwrap_or_else(|| panic!("{workspace} missing from {memberships}"));
-        assert_eq!(membership["is_service_account"], json!(is_service_account));
-    }
-
-    for (label, workspace_id, exempt) in [
-        ("own workspace", Some("test-workspace"), true),
-        ("other workspace", Some("other"), false),
-        // A workspace-less token has no workspace to match, so a service account anywhere counts.
-        ("global", None, true),
-    ] {
-        let resp = create_token(
-            port,
-            json!({ "label": label, "workspace_id": workspace_id }),
-        )
-        .await;
-        assert_eq!(resp.status(), 201);
-        assert_eq!(
-            stored_expiration(&db, label).await.is_none(),
-            exempt,
-            "{label}: expected exempt = {exempt}"
-        );
-    }
-
-    // Impersonation checks the impersonated account, not the superadmin minting the token.
-    let resp = client()
-        .post(format!(
-            "http://localhost:{port}/api/users/tokens/impersonate"
-        ))
-        .header("Authorization", "Bearer SECRET_TOKEN")
-        .json(&json!({ "label": "impersonated service account", "impersonate_email": "test2@windmill.dev" }))
-        .send()
-        .await?;
-    assert_eq!(resp.status(), 201);
-    assert_eq!(
-        stored_expiration(&db, "impersonated service account").await,
-        None
-    );
-
-    Ok(())
-}

backend/windmill-api-integration-tests/tests/token_expiry_warning.rs

@@ -1,49 +0,0 @@
-//! Which user tokens get an "expiring soon" warning queued when they are created.
-
-use serde_json::json;
-use sqlx::types::chrono::Utc;
-use sqlx::{Pool, Postgres};
-use windmill_test_utils::*;
-
-const DAY: u64 = 24 * 60 * 60;
-
-async fn warning_queued(db: &Pool<Postgres>, label: &str) -> bool {
-    sqlx::query_scalar::<_, bool>(
-        "SELECT EXISTS(SELECT 1 FROM token_expiry_notification n
-            JOIN token t ON t.token_hash = n.token_hash WHERE t.label = $1)",
-    )
-    .bind(label)
-    .fetch_one(db)
-    .await
-    .unwrap()
-}
-
-#[sqlx::test(migrations = "../migrations", fixtures("base"))]
-async fn test_short_lived_tokens_get_no_expiry_warning(db: Pool<Postgres>) -> anyhow::Result<()> {
-    initialize_tracing().await;
-    let server = ApiServer::start(db.clone()).await?;
-    let port = server.addr.port();
-
-    for (label, lifetime) in [("short", DAY), ("long", 30 * DAY)] {
-        let resp = reqwest::Client::new()
-            .post(format!("http://localhost:{port}/api/users/tokens/create"))
-            .header("Authorization", "Bearer SECRET_TOKEN_2")
-            .json(&json!({
-                "label": label,
-                "expiration": Utc::now() + std::time::Duration::from_secs(lifetime),
-            }))
-            .send()
-            .await?;
-        assert_eq!(resp.status(), 201);
-    }
-
-    assert!(
-        !warning_queued(&db, "short").await,
-        "a token whose whole lifetime fits in the warning window must not be warned about"
-    );
-    assert!(
-        warning_queued(&db, "long").await,
-        "a longer-lived token still gets its warning"
-    );
-    Ok(())
-}

backend/windmill-api-integration-tests/tests/workspaces.rs

@@ -1166,47 +1166,3 @@ async fn test_create_service_account_drops_orphaned_group_memberships(
 
     Ok(())
 }
-
-#[cfg(feature = "private")]
-#[sqlx::test(migrations = "../migrations", fixtures("base"))]
-async fn test_edit_auto_invite_preserves_instance_groups(db: Pool<Postgres>) -> anyhow::Result<()> {
-    initialize_tracing().await;
-
-    sqlx::query(
-        r#"UPDATE workspace_settings
-           SET auto_invite = '{"instance_groups": ["eng"], "instance_groups_roles": {"eng": "developer"}}'
-           WHERE workspace_id = 'test-workspace'"#,
-    )
-    .execute(&db)
-    .await?;
-
-    let server = ApiServer::start(db.clone()).await?;
-    let port = server.addr.port();
-    let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
-
-    // enable, then disable
-    for body in [
-        json!({"operator": false, "invite_all": true, "auto_add": false}),
-        json!({}),
-    ] {
-        let resp = authed(client().post(format!("{base}/edit_auto_invite")))
-            .json(&body)
-            .send()
-            .await?;
-        assert_eq!(resp.status(), 200, "{body}: {}", resp.text().await?);
-
-        let auto_invite: serde_json::Value = sqlx::query_scalar(
-            "SELECT auto_invite FROM workspace_settings WHERE workspace_id = 'test-workspace'",
-        )
-        .fetch_one(&db)
-        .await?;
-        assert_eq!(auto_invite["instance_groups"], json!(["eng"]), "{body}");
-        assert_eq!(
-            auto_invite["instance_groups_roles"],
-            json!({"eng": "developer"}),
-            "{body}"
-        );
-    }
-
-    Ok(())
-}

backend/windmill-api-jobs/src/execution.rs

@@ -26,9 +26,7 @@ use windmill_api_auth::{check_scopes, get_scope_tags, ApiAuthed};
 use windmill_common::{
     db::{UserDB, UserDbWithAuthed},
     error::{self, Error},
-    flow_conversations::{
-        add_message_to_conversation_tx, message_attachments, MessageExtras, MessageType,
-    },
+    flow_conversations::{add_message_to_conversation_tx, MessageType},
     get_latest_flow_version_info_for_path,
     jobs::{
         check_tag_available_for_workspace_internal, format_result, script_path_to_payload,
@@ -655,11 +653,9 @@ pub async fn set_flow_memory_id(
 pub async fn process_flow_run_query_params(
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
     job_id: Uuid,
-    w_id: &str,
-    flow_path: &str,
     run_query: &RunJobQuery,
 ) -> error::Result<()> {
-    if let Some(memory_id) = run_query.memory_key(w_id, flow_path) {
+    if let Some(memory_id) = run_query.memory_id {
         set_flow_memory_id(tx, job_id, memory_id).await?;
     }
     Ok(())
@@ -673,13 +669,10 @@ pub async fn handle_chat_conversation_messages(
     run_query: &RunJobQuery,
     user_message_raw: Option<&Box<serde_json::value::RawValue>>,
     job_id: Uuid,
-    is_test: bool,
-    // The run's args, for the files the message carried.
-    args: &HashMap<String, Box<serde_json::value::RawValue>>,
 ) -> error::Result<()> {
     // Names the query parameter rather than the field: it is not a flow argument, and
     // supplying it as one is the first thing tried on reading `memory_id is required`.
-    let memory_id = run_query.memory_key(w_id, flow_path).ok_or_else(|| {
+    let memory_id = run_query.memory_id.ok_or_else(|| {
         windmill_common::error::Error::BadRequest(
             "memory_id is required for chat-enabled flows. Pass it as the `memory_id` query \
              parameter, not as a flow argument: it names the conversation the turn belongs to, \
@@ -708,12 +701,11 @@ pub async fn handle_chat_conversation_messages(
         &authed.username,
         &user_message,
         memory_id,
-        is_test,
     )
     .await?;
 
-    // The run this message started. The row keeps the files the message carried as
-    // references; its args are the only record of every other flow input, and nothing
+    // The run this message started. Its args are the only record of what the message
+    // carried besides its text — attachments and every other flow input — and nothing
     // written later points at them: an assistant row holds the AI agent step's job.
     add_message_to_conversation_tx(
         tx,
@@ -723,7 +715,6 @@ pub async fn handle_chat_conversation_messages(
         MessageType::User,
         None,
         true,
-        Some(&MessageExtras { attachments: message_attachments(args), ..Default::default() }),
     )
     .await?;
 
@@ -831,7 +822,7 @@ pub async fn run_flow<'c>(
     .await?;
 
     // Set memory_id if provided (for agent memory)
-    if let Some(memory_id) = run_query.memory_key(w_id, flow_path) {
+    if let Some(memory_id) = run_query.memory_id {
         set_flow_memory_id(&mut tx, uuid, memory_id).await?;
     }
 
@@ -845,8 +836,6 @@ pub async fn run_flow<'c>(
             &run_query,
             args.args.get("user_message"),
             uuid,
-            false,
-            &args.args,
         )
         .await?;
     }

backend/windmill-api-jobs/src/types.rs

@@ -47,25 +47,13 @@ pub struct RunJobQuery {
     pub cache_ignore_s3_path: Option<bool>,
     pub skip_preprocessor: Option<bool>,
     pub poll_delay_ms: Option<u64>,
-    /// Any string; see [`RunJobQuery::memory_key`].
-    pub memory_id: Option<String>,
+    pub memory_id: Option<Uuid>,
     pub trigger_external_id: Option<String>,
     pub service_name: Option<String>,
     pub suspended_mode: Option<bool>,
 }
 
 impl RunJobQuery {
-    /// The memory id as stored in `flow_status.memory_id`: a uuid is kept, any other string hashed
-    /// within the workspace and the flow being run.
-    pub fn memory_key(&self, workspace_id: &str, flow_path: &str) -> Option<Uuid> {
-        self.memory_id
-            .as_deref()
-            .filter(|memory_id| !memory_id.trim().is_empty())
-            .map(|memory_id| {
-                windmill_common::flow_conversations::memory_key(workspace_id, flow_path, memory_id)
-            })
-    }
-
     pub async fn get_scheduled_for(
         &self,
         db: &DB,

backend/windmill-api-settings/src/lib.rs

@@ -60,14 +60,14 @@ use windmill_common::{
     global_settings::{
         AI_CONFIG_SETTING, APP_WORKSPACED_ROUTE_SETTING, AUTOMATE_USERNAME_CREATION_SETTING,
         CRITICAL_ALERT_MUTE_UI_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING,
-        DISABLE_HUB_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
+        DISABLE_HUB_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS, EXTERNAL_INSTANCE_PG_SETTING,
         GITHUB_APP_WEBHOOK_BASE_URL_SETTING, HTTP_ROUTE_DEFAULT_ALLOWED_ORIGINS_SETTING,
         HTTP_ROUTE_WORKSPACED_ROUTE_SETTING, HUB_ACCESSIBLE_URL_SETTING, HUB_BASE_URL_SETTING,
         INSTANCE_BANNER_SETTING, MAX_RETENTION_OVERRIDE_WORKSPACES,
-        MAX_TOKEN_EXPIRATION_DAYS_SETTING, RETENTION_PERIOD_SECS_OVERRIDES_SETTING,
-        RUFF_CONFIG_SETTING, UNIQUE_ID_SETTING, WORKSPACE_FAIRNESS_DURATION_SECS_SETTING,
-        WORKSPACE_FAIRNESS_ENABLED_SETTING, WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING,
-        WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING, WS_BASE_URL_SETTING,
+        RETENTION_PERIOD_SECS_OVERRIDES_SETTING, RUFF_CONFIG_SETTING, UNIQUE_ID_SETTING,
+        WORKSPACE_FAIRNESS_DURATION_SECS_SETTING, WORKSPACE_FAIRNESS_ENABLED_SETTING,
+        WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING, WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING,
+        WS_BASE_URL_SETTING,
     },
     instance_config::{self, ApplyMode, InstanceConfig},
     server::Smtp,
@@ -168,6 +168,22 @@ pub fn global_service() -> Router {
             "/refresh_custom_instance_user_pwd",
             post(refresh_custom_instance_user_pwd),
         )
+        .route(
+            "/external_instance_pg/status",
+            get(get_external_instance_pg_status),
+        )
+        .route(
+            "/external_instance_pg/setup",
+            post(setup_external_instance_pg),
+        )
+        .route(
+            "/external_instance_pg/databases",
+            get(list_external_instance_pg_databases),
+        )
+        .route(
+            "/external_instance_pg/databases/{name}",
+            post(create_external_instance_pg_database).delete(drop_external_instance_pg_database),
+        )
         .route(
             "/setup_custom_instance_pg_database/{name}",
             post(setup_custom_instance_pg_database),
@@ -877,6 +893,14 @@ pub async fn set_global_setting_internal(
         )));
     }
 
+    if key == EXTERNAL_INSTANCE_PG_SETTING {
+        return windmill_common::external_instance_pg::write_external_instance_pg_setting(
+            db,
+            Some(&value),
+        )
+        .await;
+    }
+
     run_setting_pre_write_hook(db, &key, &value).await?;
 
     match value {
@@ -1195,12 +1219,6 @@ async fn run_setting_pre_write_hook(
                 }
             }
         }
-        MAX_TOKEN_EXPIRATION_DAYS_SETTING => {
-            windmill_common::global_settings::parse_max_token_expiration_days(Some(value))
-                .map_err(|e| {
-                    error::Error::BadRequest(format!("{MAX_TOKEN_EXPIRATION_DAYS_SETTING}: {e}"))
-                })?;
-        }
         INSTANCE_BANNER_SETTING => {
             match value {
                 // Clearing (delete row) is handled by the caller; allow it through.
@@ -1264,7 +1282,7 @@ async fn set_instance_config(
     let desired_map = desired.global_settings.to_settings_map();
     if !desired_map.is_empty() {
         let current_map = current.global_settings.to_settings_map();
-        let settings_diff =
+        let mut settings_diff =
             instance_config::diff_global_settings(&current_map, &desired_map, ApplyMode::Merge);
         let ai_config_changed = settings_diff
             .upserts
@@ -1293,8 +1311,15 @@ async fn set_instance_config(
         }
 
         for (key, value) in &settings_diff.upserts {
-            run_setting_pre_write_hook(&db, key, value).await?;
+            if key != EXTERNAL_INSTANCE_PG_SETTING {
+                run_setting_pre_write_hook(&db, key, value).await?;
+            }
         }
+        windmill_common::external_instance_pg::write_external_instance_pg_from_diff(
+            &db,
+            &mut settings_diff,
+        )
+        .await?;
 
         instance_config::apply_settings_diff(&db, &settings_diff)
             .await
@@ -1362,8 +1387,6 @@ pub async fn get_global_setting(
         && key != HTTP_ROUTE_DEFAULT_ALLOWED_ORIGINS_SETTING
         && key != WS_BASE_URL_SETTING
         && key != INSTANCE_BANNER_SETTING
-        // The token form reads it to stop offering expirations the server would shorten.
-        && key != MAX_TOKEN_EXPIRATION_DAYS_SETTING
     {
         require_super_admin(&db, &authed).await?;
     }
@@ -1655,6 +1678,8 @@ struct CustomInstanceDb {
     tag: Option<String>,
     #[serde(default, skip_serializing_if = "Vec::is_empty")]
     used_by_workspaces: Vec<String>,
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    workspace_id: Option<String>,
 }
 
 #[derive(Deserialize, Debug, Serialize, Default)]
@@ -1751,6 +1776,135 @@ async fn refresh_custom_instance_user_pwd(
     Ok(Json(()))
 }
 
+async fn get_external_instance_pg_status(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+) -> JsonResult<windmill_common::external_instance_pg::ExternalInstancePgStatus> {
+    require_super_admin(&db, &authed).await?;
+    Ok(Json(
+        windmill_common::external_instance_pg::external_instance_pg_status(&db).await?,
+    ))
+}
+
+#[derive(Deserialize)]
+struct SetupExternalInstancePgBody {
+    #[serde(default)]
+    rotate_passwords: bool,
+}
+
+async fn setup_external_instance_pg(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+    Json(body): Json<SetupExternalInstancePgBody>,
+) -> JsonResult<windmill_common::external_instance_pg::ExternalInstancePgSetupReport> {
+    require_super_admin(&db, &authed).await?;
+    let report = windmill_common::external_instance_pg::setup_external_instance_pg_unchecked(
+        &db,
+        body.rotate_passwords,
+    )
+    .await?;
+    let rotated = body.rotate_passwords.to_string();
+    let success = report.success.to_string();
+    windmill_audit::audit_oss::audit_log(
+        &db,
+        &authed,
+        "settings.setup_external_instance_pg",
+        windmill_audit::ActionKind::Update,
+        "global",
+        Some(&authed.email),
+        Some(
+            [
+                ("rotate_passwords", rotated.as_str()),
+                ("success", success.as_str()),
+            ]
+            .into(),
+        ),
+    )
+    .await?;
+    Ok(Json(report))
+}
+
+#[derive(Serialize)]
+struct ExternalInstancePgDatabase {
+    #[serde(flatten)]
+    status: windmill_common::instance_config::CustomInstanceDb,
+    used_by_workspaces: Vec<String>,
+}
+
+async fn list_external_instance_pg_databases(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+) -> JsonResult<std::collections::BTreeMap<String, ExternalInstancePgDatabase>> {
+    require_super_admin(&db, &authed).await?;
+    let databases = windmill_common::external_instance_pg::external_instance_databases(&db).await?;
+    let mut usages =
+        windmill_common::external_instance_pg::external_instance_database_usages(&db).await?;
+    Ok(Json(
+        databases
+            .into_iter()
+            .map(|(name, status)| {
+                let used_by_workspaces = usages.remove(&name).unwrap_or_default();
+                (
+                    name,
+                    ExternalInstancePgDatabase {
+                        status,
+                        used_by_workspaces: used_by_workspaces.into_iter().collect(),
+                    },
+                )
+            })
+            .collect(),
+    ))
+}
+
+async fn create_external_instance_pg_database(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+    Path(dbname): Path<String>,
+    Json(body): Json<SetupCustomInstanceDbBody>,
+) -> JsonResult<()> {
+    require_super_admin(&db, &authed).await?;
+    let tag = body.tag.as_deref().unwrap_or("datatable");
+    windmill_common::external_instance_pg::create_external_instance_database_unchecked(
+        &db, &dbname, tag, None,
+    )
+    .await?;
+    windmill_audit::audit_oss::audit_log(
+        &db,
+        &authed,
+        "settings.create_external_instance_pg_database",
+        windmill_audit::ActionKind::Create,
+        "global",
+        Some(&authed.email),
+        Some([("dbname", dbname.as_str()), ("tag", tag)].into()),
+    )
+    .await?;
+    Ok(Json(()))
+}
+
+async fn drop_external_instance_pg_database(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+    Path(dbname): Path<String>,
+) -> JsonResult<()> {
+    require_super_admin(&db, &authed).await?;
+    // A data table naming a dropped database fails on every job, far from the drop that caused it.
+    windmill_common::external_instance_pg::drop_external_instance_database_unchecked(
+        &db, &dbname, None,
+    )
+    .await?;
+    windmill_audit::audit_oss::audit_log(
+        &db,
+        &authed,
+        "settings.drop_external_instance_pg_database",
+        windmill_audit::ActionKind::Delete,
+        "global",
+        Some(&authed.email),
+        Some([("dbname", dbname.as_str())].into()),
+    )
+    .await?;
+    Ok(Json(()))
+}
+
 #[derive(Deserialize)]
 struct SetupCustomInstanceDbBody {
     tag: Option<String>,
@@ -1762,12 +1916,31 @@ async fn setup_custom_instance_pg_database(
     Path(dbname): Path<String>,
     Json(body): Json<SetupCustomInstanceDbBody>,
 ) -> JsonResult<CustomInstanceDb> {
+    // Before anything is recorded: the status written below replaces the registry entry, and with it
+    // the workspace a fork copy is reserved for.
+    require_super_admin(&db, &authed).await?;
+    // A re-run keeps the fork reservation: without it, the workspace the copy was made for could no
+    // longer import into it or finish its fork.
+    let workspace_id = sqlx::query_scalar::<_, Option<String>>(
+        "SELECT value->'databases'->$1->>'workspace_id' FROM global_settings
+         WHERE name = 'custom_instance_pg_databases'",
+    )
+    .bind(&dbname)
+    .fetch_optional(&db)
+    .await?
+    .flatten();
     let mut logs = CustomInstanceDbLogs::default();
     let result = setup_custom_instance_pg_database_inner(authed, &db, &dbname, &mut logs).await;
     let success = result.is_ok();
     let error = result.err().map(|e| e.to_string());
-    let status =
-        CustomInstanceDb { logs, success, error, tag: body.tag, used_by_workspaces: vec![] };
+    let status = CustomInstanceDb {
+        logs,
+        success,
+        error,
+        tag: body.tag,
+        used_by_workspaces: vec![],
+        workspace_id,
+    };
     let status_json = serde_json::to_value(&status).map_err(to_anyhow)?;
     // Save that the database was setup successfully
     sqlx::query!(

backend/windmill-api-users/src/users.rs

@@ -47,10 +47,7 @@ use windmill_audit::audit_oss::audit_log;
 use windmill_audit::ActionKind;
 use windmill_common::audit::AuditAuthor;
 use windmill_common::auth::{hash_token, safe_token_prefix, TOKEN_PREFIX_LEN};
-use windmill_common::global_settings::{
-    load_value_from_global_settings, parse_max_token_expiration_days,
-    AUTOMATE_USERNAME_CREATION_SETTING, MAX_TOKEN_EXPIRATION_DAYS_SETTING,
-};
+use windmill_common::global_settings::AUTOMATE_USERNAME_CREATION_SETTING;
 use windmill_common::oauth2::InstanceEvent;
 use windmill_common::per_minute_counter::PerMinuteCounter;
 use windmill_common::users::truncate_token;
@@ -171,10 +168,7 @@ pub fn make_unauthed_service() -> Router {
         .route("/logout", post(logout).get(logout))
         .route("/is_first_time_setup", get(is_first_time_setup))
         .route("/request_password_reset", post(request_password_reset))
-        .route(
-            "/login_link/{token}",
-            get(consume_login_link).post(confirm_login_link),
-        )
+        .route("/login_link/{token}", get(consume_login_link))
         .route("/is_smtp_configured", get(is_smtp_configured))
         .route(
             "/is_password_login_disabled",
@@ -3098,68 +3092,11 @@ pub async fn create_guest_session_token<'c>(
 
 // create_token_internal is re-exported from windmill-api-auth above
 
-/// Applies the instance-wide ceiling on how long a token a caller picks the lifetime of may
-/// live (`create_token`, and `impersonate` for superadmins), returning the expiration to store:
-/// the requested one while it fits, the ceiling otherwise, and the ceiling as well when none was
-/// requested. Only the stored expiration is capped: tokens already stored when the setting is
-/// turned on or lowered keep theirs, since the auth lookup never reads the setting.
-///
-/// It shortens rather than refuses because most callers do not comply on their own. The CLI
-/// authorization page, `wmill user create-token` and the editor's language-server token each
-/// pick a lifetime, often none at all, without reading the setting (and CLIs already installed
-/// never will), so refusing would break logging in and the editor instead of the long-lived
-/// tokens the setting is aimed at.
-///
-/// Read from `global_settings` on each call rather than cached: token creation is rare
-/// enough that the round trip costs nothing, and the ceiling is then never served stale.
-///
-/// A token owned by a service account is exempt: in the workspace the token names, or in any
-/// workspace for a workspace-less token, which has none to match. Service accounts are the
-/// identity automation that needs a long-lived credential runs as. The cost is that any
-/// workspace admin can create and impersonate one to hold an uncapped token, so the ceiling
-/// bounds personal tokens rather than what an admin can obtain.
-async fn cap_token_expiration(
-    db: &DB,
-    owner_email: &str,
-    workspace_id: Option<&str>,
-    requested: Option<chrono::DateTime<chrono::Utc>>,
-) -> Result<Option<chrono::DateTime<chrono::Utc>>> {
-    let value = load_value_from_global_settings(db, MAX_TOKEN_EXPIRATION_DAYS_SETTING).await?;
-    let max_days = match parse_max_token_expiration_days(value.as_ref()) {
-        Ok(Some(max_days)) => max_days,
-        Ok(None) => return Ok(requested),
-        // Both write paths reject this, so only a row written around them gets here.
-        Err(e) => {
-            tracing::warn!("ignoring {MAX_TOKEN_EXPIRATION_DAYS_SETTING}: {e}");
-            return Ok(requested);
-        }
-    };
-    let max = chrono::Utc::now() + chrono::Duration::days(max_days);
-
-    let is_service_account = sqlx::query_scalar!(
-        "SELECT EXISTS(SELECT 1 FROM usr WHERE email = $1 AND is_service_account IS true
-            AND ($2::varchar IS NULL OR workspace_id = $2))",
-        owner_email,
-        workspace_id,
-    )
-    .fetch_one(db)
-    .await?
-    .unwrap_or(false);
-    if is_service_account {
-        return Ok(requested);
-    }
-
-    Ok(Some(match requested {
-        Some(expiration) if expiration < max => expiration,
-        _ => max,
-    }))
-}
-
 async fn create_token(
     Extension(db): Extension<DB>,
     authed: ApiAuthed,
     OptJobAuthed { job_id, .. }: OptJobAuthed,
-    Json(mut token_config): Json<NewToken>,
+    Json(token_config): Json<NewToken>,
 ) -> Result<(StatusCode, String)> {
     forbid_elevated_job_token(&db, &authed.email, job_id).await?;
     check_token_create_rate_limit(&authed.username)?;
@@ -3181,14 +3118,6 @@ async fn create_token(
 
     windmill_api_auth::ensure_scopes_within_caller(&authed, token_config.scopes.as_deref())?;
 
-    token_config.expiration = cap_token_expiration(
-        &db,
-        &authed.email,
-        token_config.workspace_id.as_deref(),
-        token_config.expiration,
-    )
-    .await?;
-
     let mut tx = db.begin().await?;
 
     let token = create_token_internal(&mut *tx, &db, &authed, token_config).await?;
@@ -3245,7 +3174,6 @@ async fn impersonate(
     .fetch_optional(&db)
     .await?
     .unwrap_or(false);
-    let expiration = cap_token_expiration(&db, &impersonated, None, new_token.expiration).await?;
     let mut tx = db.begin().await?;
 
     sqlx::query!(
@@ -3257,7 +3185,7 @@ async fn impersonate(
         plaintext as Option<&str>,
         impersonated,
         new_token.label,
-        expiration,
+        new_token.expiration,
         is_super_admin
     )
     .execute(&mut *tx)
@@ -3267,7 +3195,7 @@ async fn impersonate(
         &mut *tx,
         &t_hash,
         new_token.label.as_deref(),
-        expiration,
+        new_token.expiration,
     )
     .await;
 
@@ -3287,12 +3215,9 @@ async fn impersonate(
 }
 
 const LOGIN_LINK_DEFAULT_TTL_S: u32 = 600;
-// Long enough for a link sent by email to still work when it is read. `require_login_type` is
-// only checked at mint, so a much longer cap would need re-checking it when the link is opened.
-const LOGIN_LINK_MAX_TTL_S: u32 = 7200;
+const LOGIN_LINK_MAX_TTL_S: u32 = 900;
 const LOGIN_LINK_DEFAULT_RD: &str = "/user/workspaces";
 const LOGIN_LINK_EXPIRED_PAGE: &str = "/user/login_link_expired";
-const LOGIN_LINK_CONFIRM_PAGE: &str = "/user/login_link";
 
 #[derive(Deserialize)]
 pub struct NewLoginLink {
@@ -3303,9 +3228,6 @@ pub struct NewLoginLink {
     /// account it created can require `pending_oauth`, so the link stops working once the
     /// owner has set a password or signed in with a provider.
     pub require_login_type: Option<String>,
-    /// Hand out a page that signs in only when its button is clicked. Mail scanners open links
-    /// on delivery, and opening the plain link spends it, so a link sent by email sets this.
-    pub confirm: Option<bool>,
 }
 
 #[derive(Serialize)]
@@ -3456,12 +3378,11 @@ async fn create_login_link(
     .await?;
     tx.commit().await?;
 
-    let base_url = (**BASE_URL.load()).clone();
-    let url = if nl.confirm.unwrap_or(false) {
-        format!("{base_url}{LOGIN_LINK_CONFIRM_PAGE}?token={token}")
-    } else {
-        format!("{base_url}/api/auth/login_link/{token}")
-    };
+    let url = format!(
+        "{}/api/auth/login_link/{}",
+        (**BASE_URL.load()).clone(),
+        token
+    );
     Ok((StatusCode::CREATED, Json(LoginLink { url, expires_at })))
 }
 
@@ -3707,45 +3628,19 @@ async fn consume_login_link(
     Path(token): Path<String>,
     Query(query): Query<LoginLinkQuery>,
 ) -> Result<Response> {
-    let location = redeem_login_link(&headers, cookies, &db, &token, query.rd).await?;
-    Ok(login_link_redirect(location))
-}
-
-#[derive(Serialize)]
-struct LoginLinkLocation {
-    location: String,
-}
-
-/// The confirmation page's click. It answers with where to go rather than redirecting, and the
-/// page navigates there itself.
-async fn confirm_login_link(
-    headers: axum::http::HeaderMap,
-    cookies: Cookies,
-    Extension(db): Extension<DB>,
-    Path(token): Path<String>,
-) -> JsonResult<LoginLinkLocation> {
-    let location = redeem_login_link(&headers, cookies, &db, &token, None).await?;
-    Ok(Json(LoginLinkLocation { location }))
-}
-
-/// Spends the link and sets the session cookie, returning the post-login destination; or
-/// returns the explanation page, with no session, when the link cannot be used.
-async fn redeem_login_link(
-    headers: &axum::http::HeaderMap,
-    cookies: Cookies,
-    db: &DB,
-    token: &str,
-    requested_rd: Option<String>,
-) -> Result<String> {
-    let bounce = |reason: &str| Ok(format!("{LOGIN_LINK_EXPIRED_PAGE}?reason={reason}"));
+    let bounce = |reason: &str| {
+        Ok(login_link_redirect(format!(
+            "{LOGIN_LINK_EXPIRED_PAGE}?reason={reason}"
+        )))
+    };
     if token.len() != 32 {
         return bounce("invalid");
     }
-    let t_hash = hash_token(token);
+    let t_hash = hash_token(&token);
     // The account is unknown until the row is read, so only the global and per-IP tiers
     // apply here; a 32-char random token leaves nothing for the per-account tier to guard.
     windmill_common::login_rate_limit::check_and_increment_login_attempt(
-        headers,
+        &headers,
         &t_hash[..TOKEN_PREFIX_LEN],
     )?;
 
@@ -3812,10 +3707,11 @@ async fn redeem_login_link(
     .await?;
     tx.commit().await?;
 
-    Ok(link
+    let rd = link
         .rd
-        .or_else(|| same_origin_rd(requested_rd))
-        .unwrap_or_else(|| LOGIN_LINK_DEFAULT_RD.to_string()))
+        .or_else(|| same_origin_rd(query.rd))
+        .unwrap_or_else(|| LOGIN_LINK_DEFAULT_RD.to_string());
+    Ok(login_link_redirect(rd))
 }
 
 #[derive(Deserialize)]
@@ -4049,7 +3945,6 @@ async fn update_token_label(
                  AND NOT starts_with(label, 'embed_app:')
                  AND NOT starts_with(label, 'sdk_app:')
                  AND NOT starts_with(label, 'impersonation:')
-                 AND NOT starts_with(label, 'cli-login:')
              ))
            RETURNING token_prefix",
         req.label.as_deref(),

backend/windmill-api-workspaces/src/datatable_acl_oss.rs

@@ -0,0 +1,44 @@
+/*
+ * Author: Ruben Fiszel
+ * Copyright: Windmill Labs, Inc 2022
+ * This file and its contents are licensed under the AGPLv3 License.
+ * Please see the included NOTICE for copyright information and
+ * LICENSE-AGPL for a copy of the license.
+ */
+
+//! Where the ACL planner comes from: the enterprise one, or a refusal.
+//!
+//! Data table roles are an Enterprise Edition feature, and so is everything here — reading who
+//! owns what included. `private` alone is not that edition — community builds carry it — so the
+//! planner is behind `enterprise` as well.
+
+#[cfg(all(feature = "private", feature = "enterprise"))]
+pub(crate) use crate::datatable_acl_ee::plan_statements;
+
+#[cfg(all(feature = "private", feature = "enterprise"))]
+pub(crate) fn ensure_datatable_acl_available() -> windmill_common::error::Result<()> {
+    Ok(())
+}
+
+#[cfg(not(all(feature = "private", feature = "enterprise")))]
+use {
+    crate::datatable_acl::{AclChange, AclPlan, AclTarget, CatalogFacts},
+    windmill_common::{datatable_roles_oss::datatable_roles_unavailable, error::Result},
+};
+
+/// Checked first by every ACL route, before anything is read or connected to.
+#[cfg(not(all(feature = "private", feature = "enterprise")))]
+pub(crate) fn ensure_datatable_acl_available() -> Result<()> {
+    Err(datatable_roles_unavailable())
+}
+
+#[cfg(not(all(feature = "private", feature = "enterprise")))]
+pub(crate) fn plan_statements(
+    _target: &AclTarget,
+    _change: &AclChange,
+    _dbname: &str,
+    _pg_role: &str,
+    _facts: &CatalogFacts,
+) -> Result<AclPlan> {
+    Err(datatable_roles_unavailable())
+}

backend/windmill-api-workspaces/src/datatable_migrations.rs

@@ -11,7 +11,7 @@
 //! to keep that file focused on core workspace configuration.
 
 use crate::workspaces::{
-    is_instance_datatable, pg_dump_database, strip_unreplayable_dump_lines, ItemComparison,
+    managed_datatable_kind, pg_dump_database, strip_unreplayable_dump_lines, ItemComparison,
     PgDumpOptions,
 };
 
@@ -1556,7 +1556,9 @@ async fn generate_initial_datatable_migration(
     // without what a replay elsewhere cannot run: the replaying user owns none of this
     // database's objects, and the grants Windmill plants in an instance database (`ALTER
     // DEFAULT PRIVILEGES FOR ROLE ...`) fail even replaying onto the same server.
-    let no_acl = is_instance_datatable(&db, &w_id, &datatable_name).await?;
+    let no_acl = managed_datatable_kind(&db, &w_id, &datatable_name)
+        .await?
+        .is_some();
     let dump_file = pg_dump_database(
         &pg_db,
         PgDumpOptions {

backend/windmill-api-workspaces/src/datatable_permissions.rs

@@ -65,3 +65,15 @@ pub(crate) async fn ensure_reaches_datatable(
 ) -> Result<()> {
     roles::ensure_reaches_datatable(db, w_id, datatable_name, authed).await
 }
+
+/// [`ensure_reaches_datatable`] against an entry already resolved, for a caller that goes on to
+/// connect from that same entry.
+pub(crate) async fn ensure_reaches_governing_datatable(
+    db: &DB,
+    w_id: &str,
+    datatable_name: &str,
+    governing: &GoverningDatatable,
+    authed: &ApiAuthed,
+) -> Result<()> {
+    roles::ensure_reaches_governing_datatable(db, w_id, datatable_name, governing, authed).await
+}

backend/windmill-api-workspaces/src/datatable_permissions_oss.rs

@@ -12,8 +12,9 @@
 
 #[cfg(all(feature = "private", feature = "enterprise"))]
 pub(crate) use crate::datatable_permissions_ee::{
-    ensure_governs_datatable, ensure_reaches_datatable, get_datatable_permissions,
-    list_usable_datatable_roles, set_datatable_permissions,
+    ensure_governs_datatable, ensure_reaches_datatable, ensure_reaches_governing_datatable,
+    get_datatable_permissions, list_usable_datatable_roles, set_datatable_permissions,
+    usable_datatable_roles,
 };
 
 #[cfg(not(all(feature = "private", feature = "enterprise")))]
@@ -56,6 +57,20 @@ mod ce {
         }
     }
 
+    pub(crate) async fn ensure_reaches_governing_datatable(
+        _db: &DB,
+        _w_id: &str,
+        _datatable_name: &str,
+        governing: &GoverningDatatable,
+        _authed: &ApiAuthed,
+    ) -> Result<()> {
+        if governing.datatable.permissions.is_none() {
+            Ok(())
+        } else {
+            Err(unavailable())
+        }
+    }
+
     // The routes stay registered so the API has one shape; each answers after authentication,
     // before anything is read.
 
@@ -70,4 +85,28 @@ mod ce {
     pub(crate) async fn list_usable_datatable_roles(_authed: ApiAuthed) -> Result<String> {
         Err(unavailable())
     }
+
+    pub(crate) struct UsableDatatableRoles {
+        pub(crate) permissioned: bool,
+        pub(crate) roles: Vec<String>,
+        pub(crate) default_role: String,
+    }
+
+    /// A data table not under roles is used as `admin`, as before roles existed. One under roles
+    /// is refused: no role of it can be connected as.
+    pub(crate) async fn usable_datatable_roles(
+        _db: &DB,
+        _authed: &ApiAuthed,
+        _w_id: &str,
+        governing: &GoverningDatatable,
+    ) -> Result<UsableDatatableRoles> {
+        if governing.datatable.permissions.is_some() {
+            return Err(unavailable());
+        }
+        Ok(UsableDatatableRoles {
+            permissioned: false,
+            roles: vec![],
+            default_role: windmill_common::datatable_roles::ADMIN_DATATABLE_ROLE.to_string(),
+        })
+    }
 }

backend/windmill-api-workspaces/src/lib.rs

@@ -1,6 +1,8 @@
 #[cfg(feature = "parquet")]
 pub mod ai_session_backups;
 pub mod data_metrics;
+pub mod datatable_acl;
+pub mod datatable_acl_oss;
 pub mod datatable_migrations;
 pub mod datatable_permissions;
 pub mod datatable_permissions_oss;
@@ -12,5 +14,8 @@ pub mod workspaces_oss;
 #[cfg(feature = "private")]
 pub mod workspaces_ee;
 
+#[cfg(all(feature = "private", feature = "enterprise"))]
+pub mod datatable_acl_ee;
+
 #[cfg(all(feature = "private", feature = "enterprise"))]
 pub mod datatable_permissions_ee;

backend/windmill-api-workspaces/src/workspaces.rs

@@ -144,6 +144,7 @@ pub fn workspaced_service() -> Router {
         )
         .merge(crate::datatable_migrations::routes())
         .merge(crate::datatable_permissions::routes())
+        .merge(crate::datatable_acl::routes())
         .route("/git_sync_enabled", get(get_git_sync_enabled))
         .route("/git_sync_deploy_mode", get(get_git_sync_deploy_mode))
         .route("/edit_git_sync_config", post(edit_git_sync_config))
@@ -555,7 +556,6 @@ struct UserWorkspace {
     /// screen off this.
     pub created_by: Option<String>,
     pub disabled: bool,
-    pub is_service_account: bool,
 }
 
 #[derive(Deserialize)]
@@ -2263,6 +2263,26 @@ struct DataTableTables {
     schemas: TableListMap,
     #[serde(skip_serializing_if = "Option::is_none")]
     error: Option<String>,
+    /// On a database Windmill manages, on its own cluster or the external one: the only kinds that
+    /// can be under roles or have their access edited.
+    instance: bool,
+    permissioned: bool,
+    /// The roles this caller may connect as, by name; empty when not under roles.
+    usable_roles: Vec<String>,
+    default_role: String,
+    /// What the role the listing connected as may create.
+    can_create_schema: bool,
+    creatable_schemas: Vec<String>,
+}
+
+#[derive(Deserialize)]
+struct ListDataTableTablesQuery {
+    /// List only this data table: each entry opens a connection to its database.
+    datatable_name: Option<String>,
+    /// The data table `role` applies to. Every other one is listed as its default role, since a
+    /// role name means nothing outside the data table it belongs to.
+    role_for: Option<String>,
+    role: Option<String>,
 }
 
 #[derive(Deserialize)]
@@ -2270,6 +2290,7 @@ struct GetDataTableSchemaQuery {
     datatable_name: String,
     schema_name: String,
     table_name: String,
+    role: Option<String>,
 }
 
 #[derive(Serialize, Debug)]
@@ -2437,25 +2458,89 @@ async fn list_datatable_tables(
     authed: ApiAuthed,
     Extension(db): Extension<DB>,
     Path(w_id): Path<String>,
+    Query(query): Query<ListDataTableTablesQuery>,
 ) -> JsonResult<Vec<DataTableTables>> {
-    let datatable_names = list_datatable_names(&db, &w_id).await?;
+    if query.role.is_some() && query.role_for.is_none() {
+        return Err(Error::BadRequest(
+            "`role` needs `role_for`, the data table it is a role of".to_string(),
+        ));
+    }
+    if let (Some(only), Some(role_for)) =
+        (query.datatable_name.as_deref(), query.role_for.as_deref())
+    {
+        if only != role_for {
+            return Err(Error::BadRequest(format!(
+                "`role_for` names '{role_for}', which `datatable_name` leaves out of the listing"
+            )));
+        }
+    }
+    let mut datatable_names = list_datatable_names(&db, &w_id).await?;
+    for named in [query.role_for.as_deref(), query.datatable_name.as_deref()]
+        .into_iter()
+        .flatten()
+    {
+        if !datatable_names.iter().any(|n| n == named) {
+            return Err(Error::NotFound(format!(
+                "No data table named '{named}' in this workspace"
+            )));
+        }
+    }
+    if let Some(only) = query.datatable_name.as_deref() {
+        datatable_names.retain(|n| n == only);
+    }
     let mut results = Vec::new();
 
     for datatable_name in datatable_names {
-        let tables = match get_datatable_tables(&db, &authed, &w_id, &datatable_name).await {
-            Ok(schemas) => DataTableTables { datatable_name, schemas, error: None },
-            Err(e) => DataTableTables {
-                datatable_name,
-                schemas: HashMap::new(),
-                error: Some(e.to_string()),
-            },
-        };
-        results.push(tables);
+        let role = query
+            .role
+            .as_deref()
+            .filter(|_| query.role_for.as_deref() == Some(datatable_name.as_str()));
+        results.push(list_one_datatable_tables(&db, &authed, &w_id, datatable_name, role).await);
     }
 
     Ok(Json(results))
 }
 
+async fn list_one_datatable_tables(
+    db: &DB,
+    authed: &ApiAuthed,
+    w_id: &str,
+    datatable_name: String,
+    role: Option<&str>,
+) -> DataTableTables {
+    let mut entry = DataTableTables {
+        datatable_name,
+        schemas: HashMap::new(),
+        error: None,
+        instance: false,
+        permissioned: false,
+        usable_roles: vec![],
+        default_role: windmill_common::datatable_roles::ADMIN_DATATABLE_ROLE.to_string(),
+        can_create_schema: false,
+        creatable_schemas: vec![],
+    };
+    let result: Result<()> = async {
+        let governing = resolve_governing_datatable(db, w_id, &entry.datatable_name).await?;
+        entry.instance = governing.role_cluster().is_some();
+        let usable =
+            crate::datatable_permissions_oss::usable_datatable_roles(db, authed, w_id, &governing)
+                .await?;
+        entry.permissioned = usable.permissioned;
+        entry.usable_roles = usable.roles;
+        entry.default_role = usable.default_role;
+        let listing = get_datatable_tables(db, authed, w_id, &entry.datatable_name, role).await?;
+        entry.schemas = listing.schemas;
+        entry.can_create_schema = listing.can_create_schema;
+        entry.creatable_schemas = listing.creatable_schemas;
+        Ok(())
+    }
+    .await;
+    if let Err(e) = result {
+        entry.error = Some(e.to_string());
+    }
+    entry
+}
+
 async fn get_datatable_table_schema(
     authed: ApiAuthed,
     Extension(db): Extension<DB>,
@@ -2469,6 +2554,7 @@ async fn get_datatable_table_schema(
         &query.datatable_name,
         &query.schema_name,
         &query.table_name,
+        query.role.as_deref(),
     )
     .await?;
 
@@ -2507,14 +2593,13 @@ async fn resolve_datatable_pg_as_caller(
     authed: &ApiAuthed,
     w_id: &str,
     datatable_name: &str,
+    role: Option<&str>,
 ) -> Result<PgDatabase> {
     let db_resource = get_datatable_resource_from_db(
         db,
         w_id,
         datatable_name,
-        // The data table's default role. Browsing has no way to name another one yet; when the
-        // database manager grows a role picker it passes the pick through here.
-        None,
+        role,
         DatatableAccess::Authed(authed.to_authed_ref()),
     )
     .await?;
@@ -2528,7 +2613,7 @@ async fn get_datatable_schema(
     w_id: &str,
     datatable_name: &str,
 ) -> Result<SchemaMap> {
-    let pg_db = resolve_datatable_pg_as_caller(db, authed, w_id, datatable_name).await?;
+    let pg_db = resolve_datatable_pg_as_caller(db, authed, w_id, datatable_name, None).await?;
 
     // Connect to the datatable database
     let (client, connection) = pg_db.connect(Some(db)).await?;
@@ -2616,13 +2701,20 @@ async fn get_datatable_schema(
     Ok(schema_map)
 }
 
+struct DatatableTableListing {
+    schemas: TableListMap,
+    can_create_schema: bool,
+    creatable_schemas: Vec<String>,
+}
+
 async fn get_datatable_tables(
     db: &DB,
     authed: &ApiAuthed,
     w_id: &str,
     datatable_name: &str,
-) -> Result<TableListMap> {
-    let pg_db = resolve_datatable_pg_as_caller(db, authed, w_id, datatable_name).await?;
+    role: Option<&str>,
+) -> Result<DatatableTableListing> {
+    let pg_db = resolve_datatable_pg_as_caller(db, authed, w_id, datatable_name, role).await?;
     let (client, connection) = pg_db.connect(Some(db)).await?;
 
     tokio::spawn(async move {
@@ -2634,7 +2726,7 @@ async fn get_datatable_tables(
     let schema_rows = client
         .query(
             r#"
-            SELECT nspname::text AS schema_name
+            SELECT nspname::text AS schema_name, has_schema_privilege(oid, 'CREATE') AS can_create
             FROM pg_namespace
             WHERE nspname NOT IN ('information_schema', 'pg_toast', 'pg_catalog')
               AND nspname NOT LIKE 'pg_%'
@@ -2648,11 +2740,29 @@ async fn get_datatable_tables(
             Error::internal_err(format!("Failed to query schemas: {}", pg_error_message(&e)))
         })?;
 
+    let can_create_schema: bool = client
+        .query_one(
+            "SELECT has_database_privilege(current_database(), 'CREATE')",
+            &[],
+        )
+        .await
+        .map_err(|e| {
+            Error::internal_err(format!(
+                "Failed to read database privileges: {}",
+                pg_error_message(&e)
+            ))
+        })?
+        .get(0);
+
     let mut table_map: TableListMap = HashMap::new();
+    let mut creatable_schemas = Vec::new();
     let schema_names: Vec<String> = schema_rows
         .iter()
         .map(|row| {
             let name: String = row.get(0);
+            if row.get::<_, bool>(1) {
+                creatable_schemas.push(name.clone());
+            }
             table_map.entry(name.clone()).or_default();
             name
         })
@@ -2682,7 +2792,7 @@ async fn get_datatable_tables(
         table_map.entry(table_schema).or_default().push(table_name);
     }
 
-    Ok(table_map)
+    Ok(DatatableTableListing { schemas: table_map, can_create_schema, creatable_schemas })
 }
 
 async fn get_datatable_table_columns(
@@ -2692,6 +2802,7 @@ async fn get_datatable_table_columns(
     datatable_name: &str,
     schema_name: &str,
     table_name: &str,
+    role: Option<&str>,
 ) -> Result<ColumnMap> {
     if is_system_pg_schema(schema_name) {
         return Err(Error::BadRequest(format!(
@@ -2700,7 +2811,7 @@ async fn get_datatable_table_columns(
         )));
     }
 
-    let pg_db = resolve_datatable_pg_as_caller(db, authed, w_id, datatable_name).await?;
+    let pg_db = resolve_datatable_pg_as_caller(db, authed, w_id, datatable_name, role).await?;
     let (client, connection) = pg_db.connect(Some(db)).await?;
 
     tokio::spawn(async move {
@@ -2968,23 +3079,32 @@ pub(crate) async fn resolve_pg_source_checked(
         .map_err(|e| Error::internal_err(format!("Failed to parse database credentials: {}", e)))
 }
 
-/// Whether the data table `name` is backed by the Windmill instance's own PostgreSQL
-/// rather than a user resource.
-pub(crate) async fn is_instance_datatable(db: &DB, w_id: &str, name: &str) -> Result<bool> {
+/// The kind of the database backing the data table `name` when Windmill manages it (on its own
+/// cluster or the external one), `None` when it is a user resource.
+pub(crate) async fn managed_datatable_kind(
+    db: &DB,
+    w_id: &str,
+    name: &str,
+) -> Result<Option<DataTableCatalogResourceType>> {
     // Resolved rather than read: a pointer entry owns no database of its own, so only the entry it
-    // lands on can answer. A name that resolves to nothing keeps the historical `false`.
+    // lands on can answer. A name that resolves to nothing keeps the historical `None`.
     Ok(resolve_governing_datatable(db, w_id, name)
         .await
         .ok()
         .and_then(|g| g.datatable.database)
-        .is_some_and(|d| d.resource_type == DataTableCatalogResourceType::Instance))
+        .map(|d| d.resource_type)
+        .filter(|kind| kind.is_windmill_managed()))
 }
 
 /// Same, for the `datatable://<name>` / `$res:<path>` form the import endpoints take.
-async fn is_instance_datatable_source(db: &DB, w_id: &str, source: &str) -> Result<bool> {
+async fn managed_datatable_source_kind(
+    db: &DB,
+    w_id: &str,
+    source: &str,
+) -> Result<Option<DataTableCatalogResourceType>> {
     match source.strip_prefix("datatable://") {
-        Some(name) => is_instance_datatable(db, w_id, name).await,
-        None => Ok(false),
+        Some(name) => managed_datatable_kind(db, w_id, name).await,
+        None => Ok(None),
     }
 }
 
@@ -3083,10 +3203,7 @@ pub(crate) async fn pg_dump_database(
     if let Some(ref password) = pg_db.password {
         cmd.env("PGPASSWORD", password);
     }
-
-    if let Some(ref sslmode) = pg_db.sslmode {
-        cmd.env("PGSSLMODE", sslmode);
-    }
+    let _root_cert = apply_pg_tls_env(&mut cmd, pg_db)?;
 
     let output = cmd
         .output()
@@ -3204,7 +3321,7 @@ async fn comment_out_unsupported_settings(
 
 /// A psql invocation against `pg_db`, carrying the connection settings the CLI reads
 /// from the environment.
-fn psql_command(pg_db: &PgDatabase) -> tokio::process::Command {
+fn psql_command(pg_db: &PgDatabase) -> Result<(tokio::process::Command, Option<DumpFile>)> {
     let mut cmd = tokio::process::Command::new("psql");
     cmd.arg("--host")
         .arg(&pg_db.host)
@@ -3221,10 +3338,88 @@ fn psql_command(pg_db: &PgDatabase) -> tokio::process::Command {
     if let Some(ref password) = pg_db.password {
         cmd.env("PGPASSWORD", password);
     }
+    let root_cert = apply_pg_tls_env(&mut cmd, pg_db)?;
+    Ok((cmd, root_cert))
+}
+
+/// Give libpq the TLS settings `PgDatabase::connect` applies. The returned file holds the root
+/// certificate `PGSSLROOTCERT` names, so it must outlive the command.
+fn apply_pg_tls_env(
+    cmd: &mut tokio::process::Command,
+    pg_db: &PgDatabase,
+) -> Result<Option<DumpFile>> {
     if let Some(ref sslmode) = pg_db.sslmode {
         cmd.env("PGSSLMODE", sslmode);
     }
-    cmd
+    if let Some(pem) = pg_db
+        .root_certificate_pem
+        .as_deref()
+        .filter(|p| !p.is_empty())
+    {
+        let file = DumpFile::new()?;
+        std::fs::write(&file.path, pem)
+            .map_err(|e| Error::internal_err(format!("Failed to write root certificate: {e}")))?;
+        cmd.env("PGSSLROOTCERT", &file.path);
+        return Ok(Some(file));
+    }
+    // Only a connection that asked to be verified against the system trust store. Without a file,
+    // libpq's own default would look for `~/.postgresql/root.crt` and refuse a verify-* mode. libpq
+    // takes the special `system` value with verify-full only, so verify-ca needs the bundle itself.
+    if pg_db.accept_invalid_certs == Some(false) {
+        match pg_db.sslmode.as_deref() {
+            Some("verify-full") => {
+                cmd.env("PGSSLROOTCERT", "system");
+            }
+            Some("verify-ca") => {
+                if let Some(bundle) = windmill_common::system_ca_bundle() {
+                    cmd.env("PGSSLROOTCERT", bundle);
+                }
+            }
+            _ => {}
+        }
+    }
+    Ok(None)
+}
+
+
+#[cfg(test)]
+mod pg_tls_env_tests {
+    use super::apply_pg_tls_env;
+    use windmill_common::PgDatabase;
+
+    fn root_cert_env(sslmode: &str) -> Option<std::ffi::OsString> {
+        let pg_db = PgDatabase {
+            host: "db".to_string(),
+            user: None,
+            password: None,
+            port: None,
+            sslmode: Some(sslmode.to_string()),
+            dbname: "d".to_string(),
+            root_certificate_pem: None,
+            accept_invalid_certs: Some(false),
+            use_iam_auth: None,
+            region: None,
+        };
+        let mut cmd = tokio::process::Command::new("psql");
+        apply_pg_tls_env(&mut cmd, &pg_db).unwrap();
+        cmd.as_std()
+            .get_envs()
+            .find(|(k, _)| *k == "PGSSLROOTCERT")
+            .and_then(|(_, v)| v.map(|v| v.to_os_string()))
+    }
+
+    #[test]
+    fn system_roots_only_through_verify_full() {
+        assert_eq!(
+            root_cert_env("verify-full").as_deref(),
+            Some("system".as_ref())
+        );
+        // libpq refuses `sslrootcert=system` with verify-ca, which would fail every dump and restore.
+        assert_ne!(
+            root_cert_env("verify-ca").as_deref(),
+            Some("system".as_ref())
+        );
+    }
 }
 
 /// GUC names the server backing `pg_db` knows about.
@@ -3234,7 +3429,8 @@ fn psql_command(pg_db: &PgDatabase) -> tokio::process::Command {
 /// and an unset mode, where `PgDatabase::connect` would hand a TLS-only server a
 /// plaintext socket and fail before the import ever starts.
 async fn server_setting_names(pg_db: &PgDatabase) -> Result<HashSet<String>> {
-    let output = psql_command(pg_db)
+    let (mut cmd, _root_cert) = psql_command(pg_db)?;
+    let output = cmd
         .arg("--tuples-only")
         .arg("--no-align")
         .arg("--command")
@@ -3268,7 +3464,8 @@ async fn pg_import_dump(target_db: &PgDatabase, dump_file: &DumpFile) -> Result<
     let supported_settings = server_setting_names(target_db).await?;
     comment_out_unsupported_settings(dump_file, &supported_settings).await?;
 
-    let output = psql_command(target_db)
+    let (mut cmd, _root_cert) = psql_command(target_db)?;
+    let output = cmd
         .arg("--set")
         .arg("ON_ERROR_STOP=1")
         .arg("--single-transaction")
@@ -3326,9 +3523,23 @@ async fn create_pg_database(
         }
     }
 
-    if is_instance_datatable_source(&db, &w_id, &req.source).await? {
-        windmill_common::create_custom_instance_database(&db, &req.target_dbname, "datatable")
-            .await?;
+    let source_kind = managed_datatable_source_kind(&db, &w_id, &req.source).await?;
+    if source_kind == Some(DataTableCatalogResourceType::ExternalInstance) {
+        windmill_common::external_instance_pg::create_external_instance_database_unchecked(
+            &db,
+            &req.target_dbname,
+            "datatable",
+            Some(&w_id),
+        )
+        .await?;
+    } else if source_kind == Some(DataTableCatalogResourceType::Instance) {
+        windmill_common::create_custom_instance_database(
+            &db,
+            &req.target_dbname,
+            "datatable",
+            Some(&w_id),
+        )
+        .await?;
     } else {
         let source_pg =
             resolve_pg_source_checked(&db, &user_db, &authed, &w_id, &req.source).await?;
@@ -3429,12 +3640,12 @@ async fn ensure_datatable_is_clonable(
     }
     // The copy has to name a database of its own. A resource-backed entry reached through a
     // pointer names one this workspace does not own, so there is nothing here to repoint.
-    let is_instance = governing
+    let is_managed = governing
         .datatable
         .database
         .as_ref()
-        .is_some_and(|d| d.resource_type == DataTableCatalogResourceType::Instance);
-    if governing.workspace_id != w_id && !is_instance {
+        .is_some_and(|d| d.resource_type.is_windmill_managed());
+    if governing.workspace_id != w_id && !is_managed {
         return Err(Error::BadRequest(format!(
             "Data table '{name}' points at a resource-backed data table in another workspace \
              and cannot be copied; fork it from the workspace that owns it."
@@ -3482,6 +3693,15 @@ async fn import_pg_database(
                         .to_string(),
                 ));
             }
+            if let Some(kind) = managed_datatable_source_kind(&db, &w_id, &req.target).await? {
+                windmill_common::ensure_fork_database_available_to(
+                    &db,
+                    kind,
+                    override_dbname,
+                    &w_id,
+                )
+                .await?;
+            }
         }
         target_pg.dbname = override_dbname.clone();
     }
@@ -3491,8 +3711,12 @@ async fn import_pg_database(
     // what it creates it owns. Grants do, except around an instance data table — Windmill
     // plants `custom_instance_user` grants in one, which nothing else can replay. Elsewhere
     // the ACLs are user intent (`REVOKE ... FROM PUBLIC`) and dropping them widens access.
-    let no_acl = is_instance_datatable_source(&db, &w_id, &req.target).await?
-        || is_instance_datatable_source(&db, &w_id, &req.source).await?;
+    let no_acl = managed_datatable_source_kind(&db, &w_id, &req.target)
+        .await?
+        .is_some()
+        || managed_datatable_source_kind(&db, &w_id, &req.source)
+            .await?
+            .is_some();
 
     let dump_file = pg_dump_database(
         &source_pg,
@@ -3605,22 +3829,37 @@ async fn edit_ducklake_config(
     let old_ducklakes: HashMap<String, Ducklake> =
         serde_json::from_value(old_ducklakes).unwrap_or_default();
 
-    // Check that non-superadmins are not abusing Instance databases
-    if !is_superadmin {
-        for (name, dl) in new_config.settings.ducklakes.iter() {
-            if dl.catalog.resource_type == DucklakeCatalogResourceType::Instance {
-                let old_dl = old_ducklakes.get(name);
-                if old_dl.is_none()
-                    || old_dl.unwrap().catalog.resource_type
-                        != DucklakeCatalogResourceType::Instance
-                    || old_dl.unwrap().catalog.resource_path != dl.catalog.resource_path
-                {
-                    return Err(Error::BadRequest(
-                        "Only superadmins can create or modify ducklakes with Instance databases"
-                            .to_string(),
-                    ));
-                }
-            }
+    // Check that non-superadmins are not abusing Instance databases. An unchanged catalog is left
+    // alone either way, so a downgraded instance can still save lakes that already name an
+    // external instance database.
+    for (name, dl) in new_config.settings.ducklakes.iter() {
+        let kind = &dl.catalog.resource_type;
+        if !matches!(
+            kind,
+            DucklakeCatalogResourceType::Instance | DucklakeCatalogResourceType::ExternalInstance
+        ) {
+            continue;
+        }
+        let unchanged = old_ducklakes.get(name).is_some_and(|old| {
+            &old.catalog.resource_type == kind
+                && old.catalog.resource_path == dl.catalog.resource_path
+        });
+        if unchanged {
+            continue;
+        }
+        if *kind == DucklakeCatalogResourceType::ExternalInstance {
+            windmill_common::external_instance_pg::ensure_external_instance_available()?;
+            windmill_common::external_instance_pg::ensure_external_instance_database_registered(
+                &mut tx,
+                &dl.catalog.resource_path,
+            )
+            .await?;
+        }
+        if !is_superadmin {
+            return Err(Error::BadRequest(
+                "Only superadmins can create or modify ducklakes with Instance databases"
+                    .to_string(),
+            ));
         }
     }
 
@@ -3792,6 +4031,7 @@ async fn edit_datatable_config(
                 // so these line up with the `datatable_configured` adoption counts.
                 created_substrates.push(match dt.database.as_ref().map(|d| d.resource_type) {
                     Some(DataTableCatalogResourceType::Instance) => "instance",
+                    Some(DataTableCatalogResourceType::ExternalInstance) => "external_instance",
                     Some(DataTableCatalogResourceType::Postgresql) => "postgresql",
                     None => "reference",
                 });
@@ -3811,18 +4051,20 @@ async fn edit_datatable_config(
             None => None,
         };
         // Carrying the block onto a resource-backed entry would produce a data table the chokepoint
-        // refuses on every job — a save that succeeds and breaks everything afterwards. Refuse it
-        // instead: turning roles off first is one step, and it keeps discarding an access decision
-        // something somebody chose rather than a side effect of moving a database.
+        // refuses on every job — a save that succeeds and breaks everything afterwards — and onto
+        // the other managed cluster, one whose role ids name nothing in that cluster's catalog.
+        // Refuse it instead: turning roles off first is one step, and it keeps discarding an access
+        // decision something somebody chose rather than a side effect of moving a database.
+        let old_kind = old.and_then(|old| old.database.as_ref()).map(|d| d.resource_type);
         if dt.permissions.is_some()
             && dt
                 .database
                 .as_ref()
-                .is_some_and(|d| d.resource_type != DataTableCatalogResourceType::Instance)
+                .is_some_and(|d| Some(d.resource_type) != old_kind)
         {
             return Err(Error::BadRequest(format!(
-                "Data table '{name}' is under roles, which only a data table on the instance \
-                 database can be. Turn its roles off before moving it to a PostgreSQL resource."
+                "Data table '{name}' is under roles, which belong to the cluster its database is \
+                 on. Turn its roles off before moving it to another kind of database."
             )));
         }
         // A pointer names no database of its own, so the form's empty `database` is correct there.
@@ -3847,26 +4089,39 @@ async fn edit_datatable_config(
     // Check that non-superadmins are not abusing Instance databases, which reach a database this
     // workspace does not own. Pointing an entry at another workspace's data table is not checked
     // here because it cannot be requested at all: `reference` is overwritten from the stored entry
-    // above, for every caller.
-    if !is_superadmin {
-        for (name, dt) in new_config.settings.datatables.iter() {
-            let old_dt = old_datatables.get(name);
-            if dt
-                .database
-                .as_ref()
-                .is_some_and(|d| d.resource_type == DataTableCatalogResourceType::Instance)
-            {
-                let unchanged = old_dt.and_then(|o| o.database.as_ref()).is_some_and(|o| {
-                    o.resource_type == DataTableCatalogResourceType::Instance
-                        && Some(&o.resource_path) == dt.database.as_ref().map(|d| &d.resource_path)
-                });
-                if !unchanged {
-                    return Err(Error::BadRequest(
-                        "Only superadmins can create or modify data tables with Instance databases"
-                            .to_string(),
-                    ));
-                }
-            }
+    // above, for every caller. An unchanged entry is left alone either way, so a downgraded
+    // instance can still save settings that already name an external instance database.
+    for (name, dt) in new_config.settings.datatables.iter() {
+        let Some(database) = dt
+            .database
+            .as_ref()
+            .filter(|d| d.resource_type.is_windmill_managed())
+        else {
+            continue;
+        };
+        let unchanged = old_datatables
+            .get(name)
+            .and_then(|o| o.database.as_ref())
+            .is_some_and(|o| {
+                o.resource_type == database.resource_type
+                    && o.resource_path == database.resource_path
+            });
+        if unchanged {
+            continue;
+        }
+        if database.resource_type == DataTableCatalogResourceType::ExternalInstance {
+            windmill_common::external_instance_pg::ensure_external_instance_available()?;
+            windmill_common::external_instance_pg::ensure_external_instance_database_registered(
+                &mut tx,
+                &database.resource_path,
+            )
+            .await?;
+        }
+        if !is_superadmin {
+            return Err(Error::BadRequest(
+                "Only superadmins can create or modify data tables with Instance databases"
+                    .to_string(),
+            ));
         }
     }
 
@@ -3886,7 +4141,7 @@ async fn edit_datatable_config(
     // entry through a declared rename alone, and a settings sync never declares one, so an entry
     // without roles that newly points at such a database — a name added, or an existing one
     // repointed — would answer everyone there as `admin`. That holds whichever workspace governs it.
-    let newly_pointed: Vec<(&String, &str)> = new_config
+    let newly_pointed: Vec<(&String, DataTableCatalogResourceType, &str)> = new_config
         .settings
         .datatables
         .iter()
@@ -3895,7 +4150,7 @@ async fn edit_datatable_config(
             let db = dt
                 .database
                 .as_ref()
-                .filter(|d| d.resource_type == DataTableCatalogResourceType::Instance)?;
+                .filter(|d| d.resource_type.is_windmill_managed())?;
             let lookup = rename_src
                 .get(name.as_str())
                 .copied()
@@ -3907,38 +4162,44 @@ async fn edit_datatable_config(
                     old_db.resource_type != db.resource_type
                         || old_db.resource_path != db.resource_path
                 });
-            repointed.then_some((name, db.resource_path.as_str()))
+            repointed.then_some((name, db.resource_type, db.resource_path.as_str()))
         })
         .collect();
     // Another workspace turning roles on for the same database holds only its own settings row, so
     // without this the scan below could read past its uncommitted write.
     windmill_common::datatable_roles::lock_instance_databases_governance(
         &mut *tx,
-        newly_pointed.iter().map(|(_, dbname)| *dbname),
+        newly_pointed.iter().map(|(_, _, dbname)| *dbname),
     )
     .await?;
-    let governed_elsewhere: Vec<String> = if newly_pointed.is_empty() {
+    let governed_elsewhere: Vec<(String, String)> = if newly_pointed.is_empty() {
         vec![]
     } else {
-        sqlx::query_scalar(
-            "SELECT DISTINCT dt.value->'database'->>'resource_path' FROM workspace_settings ws
+        sqlx::query_as(
+            "SELECT DISTINCT dt.value->'database'->>'resource_type',
+                    dt.value->'database'->>'resource_path'
+             FROM workspace_settings ws
              CROSS JOIN LATERAL jsonb_each(COALESCE(ws.datatable->'datatables', '{}'::jsonb)) dt
              WHERE ws.workspace_id <> $1 AND dt.value ? 'permissions'
-               AND dt.value->'database'->>'resource_type' = 'instance'",
+               AND dt.value->'database'->>'resource_type' IN ('instance', 'external_instance')",
         )
         .bind(&w_id)
         .fetch_all(&mut *tx)
         .await?
     };
-    for (name, dbname) in newly_pointed {
+    for (name, kind, dbname) in newly_pointed {
         let governed_here = old_datatables.values().any(|old| {
             old.permissions.is_some()
-                && old.database.as_ref().is_some_and(|d| {
-                    d.resource_type == DataTableCatalogResourceType::Instance
-                        && d.resource_path == dbname
-                })
+                && old
+                    .database
+                    .as_ref()
+                    .is_some_and(|d| d.resource_type == kind && d.resource_path == dbname)
         });
-        if governed_here || governed_elsewhere.iter().any(|g| g == dbname) {
+        if governed_here
+            || governed_elsewhere
+                .iter()
+                .any(|(k, p)| k == kind.as_ref() && p == dbname)
+        {
             return Err(Error::BadRequest(format!(
                 "Data table '{name}' would point at database '{dbname}', which a data table under \
                  roles uses, without carrying those roles: everyone reaching '{name}' would connect \
@@ -5891,7 +6152,7 @@ async fn user_workspaces(
                 workspace.is_dev_workspace, workspace.dev_workspace_label,
                 workspace.owner AS \"created_by?\",
                 CASE WHEN usr.operator THEN workspace_settings.operator_settings ELSE NULL END as operator_settings,
-                usr.disabled, usr.is_service_account
+                usr.disabled
          FROM workspace
          JOIN usr ON usr.workspace_id = workspace.id
          JOIN workspace_settings ON workspace_settings.workspace_id = workspace.id
@@ -7277,127 +7538,6 @@ async fn clear_orphaned_compat_address(
     Ok(())
 }
 
-/// SQL boolean: the principal the `principal` expression yields resolves in the workspace bound as
-/// `$1`. The same predicate `clone_scripts` and `clone_flows` inline, whose `query!` macros cannot
-/// take a composed string, so keep the three in step.
-fn principal_resolves_sql(principal: &str) -> String {
-    format!(
-        "CASE WHEN {principal} LIKE 'u/%' THEN EXISTS (
-                SELECT 1 FROM usr u WHERE u.workspace_id = $1
-                  AND u.username = substring({principal} from 3)
-                UNION ALL
-                SELECT 1 FROM password pw WHERE pw.super_admin
-                  AND (pw.username = substring({principal} from 3)
-                       OR pw.email = substring({principal} from 3)))
-              WHEN {principal} LIKE 'g/%' THEN EXISTS (
-                SELECT 1 FROM group_ g WHERE g.workspace_id = $1
-                  AND g.name = substring({principal} from 3))
-              ELSE EXISTS (
-                SELECT 1 FROM usr u WHERE u.workspace_id = $1 AND u.username = {principal}
-                UNION ALL
-                SELECT 1 FROM password pw WHERE pw.email = {principal} AND pw.super_admin)
-         END"
-    )
-}
-
-/// Re-point the identities a fork clones verbatim at its creator when they name nobody in the fork,
-/// once its membership is final so copied members keep theirs. Unlike scripts and flows these
-/// cannot drop the identity: an app deploy rejects a preserved one that does not resolve, and
-/// publisher apps, schedules and triggers need one to run.
-async fn repoint_unresolvable_cloned_identities(
-    tx: &mut Transaction<'_, Postgres>,
-    target_workspace_id: &str,
-    authed: &ApiAuthed,
-) -> Result<()> {
-    let principal = username_to_permissioned_as(&authed.username);
-
-    sqlx::query(&format!(
-        "UPDATE app SET policy = policy
-             || jsonb_build_object('on_behalf_of', $2::text, 'on_behalf_of_email', $3::text)
-         WHERE workspace_id = $1 AND policy->>'on_behalf_of' IS NOT NULL
-           AND NOT ({})",
-        principal_resolves_sql("(policy->>'on_behalf_of')")
-    ))
-    .bind(target_workspace_id)
-    .bind(&principal)
-    .bind(&authed.email)
-    .execute(&mut **tx)
-    .await?;
-
-    // A draft holding a genuine NUL escape (the rule of `json_text_has_nul_escape`) is left as it
-    // is, since parsing it would abort the fork. The check must stay in a CASE: json `->>` raises on
-    // a NUL anywhere in the value, and Postgres reorders plain AND conditions.
-    let nul_escape = r"(^|[^\\])(\\\\)*\\u0000";
-    sqlx::query(&format!(
-        "UPDATE draft SET value = to_json(jsonb_set(jsonb_set(to_jsonb(value),
-             ARRAY['policy', 'on_behalf_of'], to_jsonb($2::text)),
-             ARRAY['policy', 'on_behalf_of_email'], to_jsonb($3::text)))
-         WHERE workspace_id = $1 AND typ IN ('app', 'raw_app')
-           AND CASE WHEN value::text ~ $4 THEN false
-                    ELSE value->'policy'->>'on_behalf_of' IS NOT NULL AND NOT ({}) END",
-        principal_resolves_sql("(value->'policy'->>'on_behalf_of')")
-    ))
-    .bind(target_workspace_id)
-    .bind(&principal)
-    .bind(&authed.email)
-    .bind(nul_escape)
-    .execute(&mut **tx)
-    .await?;
-
-    sqlx::query(&format!(
-        "UPDATE draft SET value = to_json(jsonb_set(to_jsonb(value),
-             ARRAY['permissioned_as'], to_jsonb($2::text)))
-         WHERE workspace_id = $1 AND starts_with(typ::text, 'trigger_')
-           AND CASE WHEN value::text ~ $3 THEN false
-                    ELSE value->>'permissioned_as' IS NOT NULL AND NOT ({}) END",
-        principal_resolves_sql("(value->>'permissioned_as')")
-    ))
-    .bind(target_workspace_id)
-    .bind(&principal)
-    .bind(nul_escape)
-    .execute(&mut **tx)
-    .await?;
-
-    let column_resolves = principal_resolves_sql("permissioned_as");
-
-    // SAFETY: every table name is a literal from this list, never user input.
-    for table in [
-        "http_trigger",
-        "websocket_trigger",
-        "kafka_trigger",
-        "nats_trigger",
-        "postgres_trigger",
-        "mqtt_trigger",
-        "amqp_trigger",
-        "sqs_trigger",
-        "gcp_trigger",
-        "azure_trigger",
-        "email_trigger",
-    ] {
-        sqlx::query(&format!(
-            "UPDATE {table} SET permissioned_as = $2
-             WHERE workspace_id = $1 AND NOT ({column_resolves})"
-        ))
-        .bind(target_workspace_id)
-        .bind(&principal)
-        .execute(&mut **tx)
-        .await?;
-    }
-
-    // `email` is still written for workers that predate `permissioned_as`.
-    sqlx::query(&format!(
-        "UPDATE schedule SET permissioned_as = $2, email = $3
-         WHERE workspace_id = $1 AND NOT ({column_resolves})"
-    ))
-    .bind(target_workspace_id)
-    .bind(&principal)
-    .bind(&authed.email)
-    .execute(&mut **tx)
-    .await?;
-
-    Ok(())
-}
-
 /// Carries over the recorded principal under the rule spelled out on [`clone_scripts`].
 async fn clone_flows(
     tx: &mut Transaction<'_, Postgres>,
@@ -8073,13 +8213,14 @@ async fn point_kept_datatables_at_parent(
         if dt.reference.is_some() {
             continue;
         }
-        // Only instance databases. A resource-backed data table names a resource, and the settings
-        // clone gave the fork its own copy of that resource in its own workspace — pointing at the
-        // parent's entry would silently move the fork onto the parent's resource instead.
+        // Only instance databases, on either cluster. A resource-backed data table names a
+        // resource, and the settings clone gave the fork its own copy of that resource in its own
+        // workspace — pointing at the parent's entry would silently move the fork onto the
+        // parent's resource instead.
         if dt
             .database
             .as_ref()
-            .is_none_or(|d| d.resource_type != DataTableCatalogResourceType::Instance)
+            .is_none_or(|d| !d.resource_type.is_windmill_managed())
         {
             continue;
         }
@@ -8209,11 +8350,30 @@ async fn apply_forked_datatable(
         })?,
     };
 
-    if database.resource_type == DataTableCatalogResourceType::Instance {
+    if database.resource_type == DataTableCatalogResourceType::ExternalInstance {
+        windmill_common::external_instance_pg::ensure_external_instance_database_registered(
+            tx,
+            &fdt.new_dbname,
+        )
+        .await?;
+    }
+    if database.resource_type.is_windmill_managed()
+        && !windmill_api_auth::is_super_admin_authed(db, authed).await?
+    {
+        windmill_common::ensure_fork_database_available_to(
+            db,
+            database.resource_type,
+            &fdt.new_dbname,
+            parent_w_id,
+        )
+        .await?;
+    }
+    if database.resource_type.is_windmill_managed() {
         // The whole `database` object, not just its `resource_path`: a pointer entry has none to
-        // patch. `reference` goes with it — exactly one of the two may be set.
+        // patch. `reference` goes with it — exactly one of the two may be set. The copy was created
+        // on the same cluster as its source, so it keeps the source's kind.
         let new_database = serde_json::json!({
-            "resource_type": "instance",
+            "resource_type": database.resource_type,
             "resource_path": &fdt.new_dbname,
         });
         sqlx::query!(
@@ -8602,6 +8762,11 @@ async fn create_workspace_fork(
     }
 
     let mut tx: Transaction<'_, Postgres> = db.begin().await?;
+    // Before the settings clone reads the parent's data tables: a pointer this fork ends up with
+    // must not be written after cleanup of the parent decided that nothing points at its copies.
+    // Also before the external cluster's lifecycle lock, which finalizing an external copy takes:
+    // fork cleanup takes the two in this order.
+    windmill_common::workspaces::lock_fork_datatables(&mut tx, &parent_workspace_id).await?;
 
     if nw.is_dev_workspace {
         // The checks above ran outside a transaction, so the parent's eligibility and the chain's
@@ -8714,8 +8879,6 @@ async fn create_workspace_fork(
     // re-enables in the fork, with parent-conflict warnings on enable.
     clone_triggers_and_schedules(&mut tx, &parent_workspace_id, &forked_id).await?;
 
-    repoint_unresolvable_cloned_identities(&mut tx, &forked_id, &authed).await?;
-
     // Update forked datatable settings to point to new databases
     for fdt in &nw.forked_datatables {
         apply_forked_datatable(&db, &mut tx, &authed, &parent_workspace_id, &forked_id, fdt)

backend/windmill-api-workspaces/src/workspaces_extra.rs

@@ -110,6 +110,21 @@ pub(crate) async fn change_workspace_id(
     .execute(&mut *tx)
     .await?;
 
+    // A fork copy reserved for the old id would otherwise be unreachable: its creator cannot
+    // import into it or finish its fork under the new id, and nothing else would ever drop it.
+    sqlx::query(
+        r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}', (
+               SELECT COALESCE(jsonb_object_agg(k, CASE WHEN v->>'workspace_id' = $1
+                   THEN jsonb_set(v, '{workspace_id}', to_jsonb($2::text)) ELSE v END), '{}'::jsonb)
+               FROM jsonb_each(COALESCE(value->'databases', '{}'::jsonb)) AS e(k, v)
+           ))
+           WHERE name = 'custom_instance_pg_databases'"#,
+    )
+    .bind(&old_id)
+    .bind(&rw.new_id)
+    .execute(&mut *tx)
+    .await?;
+
     // Duplicate workspace settings (keep copy in old workspace for reference)
     info!("Duplicating workspace_settings table");
     sqlx::query!(
@@ -1409,9 +1424,7 @@ pub async fn drop_forked_datatable_databases(
             _ => continue,
         };
 
-        if database.resource_type
-            == windmill_common::workspaces::DataTableCatalogResourceType::Instance
-        {
+        if database.resource_type.is_windmill_managed() {
             let db_to_drop = &database.resource_path;
             if !db_to_drop.starts_with("wm_fork_") {
                 errors.push(format!(
@@ -1420,7 +1433,42 @@ pub async fn drop_forked_datatable_databases(
                 ));
                 continue;
             }
-            if let Err(e) = windmill_common::drop_custom_instance_database(&db, db_to_drop).await {
+            // The fork's own entry is what is going away; anything else still reaching the copy,
+            // a child fork's pointer at this entry included, keeps it. The lock keeps a child fork
+            // from gaining such a pointer before the drop.
+            let dropped = async {
+                let mut tx = db.begin().await?;
+                windmill_common::workspaces::lock_fork_datatables(&mut tx, &w_id).await?;
+                if database.resource_type
+                    == windmill_common::workspaces::DataTableCatalogResourceType::ExternalInstance
+                {
+                    windmill_common::external_instance_pg::drop_external_instance_database_unchecked(
+                        &db,
+                        db_to_drop,
+                        Some((&w_id, dt_name)),
+                    )
+                    .await?;
+                } else {
+                    let uses = windmill_common::workspaces::managed_database_uses(
+                        &mut tx,
+                        windmill_common::workspaces::DataTableCatalogResourceType::Instance,
+                        db_to_drop,
+                        Some((&w_id, dt_name)),
+                    )
+                    .await?;
+                    if !uses.is_empty() {
+                        return Err(Error::BadRequest(format!(
+                            "it is still used by {}",
+                            uses.join(", ")
+                        )));
+                    }
+                    windmill_common::drop_custom_instance_database(&db, db_to_drop).await?;
+                }
+                tx.commit().await?;
+                Ok::<_, Error>(())
+            }
+            .await;
+            if let Err(e) = dropped {
                 errors.push(format!(
                     "Could not drop instance database '{}' for datatable://{}: {}",
                     db_to_drop, dt_name, e
@@ -1787,7 +1835,17 @@ async fn resolve_fork_catalog_pg(
             "ducklake://{ducklake_name}: malformed registry catalog identity `{catalog}`"
         ))
     })?;
-    let catalog_resource = if resource_type == "instance" {
+    let catalog_resource = if resource_type == "external_instance" {
+        serde_json::to_value(
+            windmill_common::external_instance_pg::external_instance_connection_unchecked(
+                db,
+                resource_path,
+                false,
+            )
+            .await?,
+        )
+        .map_err(|e| Error::internal_err(format!("serializing pg creds: {e}")))?
+    } else if resource_type == "instance" {
         let mut pg_creds = windmill_common::PgDatabase::parse_uri(
             &windmill_common::get_database_url().await?.as_str().await,
         )?;

backend/windmill-api/openapi.yaml

@@ -1,7 +1,7 @@
 openapi: "3.0.3"
 
 info:
-  version: 1.814.0
+  version: 1.813.0
   title: Windmill API
 
   contact:
@@ -529,30 +529,6 @@ paths:
       responses:
         "302":
           description: redirected to the post-login destination, or to /user/login_link_expired when the link is used, expired or unknown
-    post:
-      security: []
-      summary: consume a single-use login link from its confirmation page, set the session cookie and answer where to go
-      operationId: confirmLoginLink
-      tags:
-        - user
-      parameters:
-        - name: token
-          in: path
-          required: true
-          schema:
-            type: string
-      responses:
-        "200":
-          description: the post-login destination, or /user/login_link_expired when the link is used, expired or unknown
-          content:
-            application/json:
-              schema:
-                type: object
-                required:
-                  - location
-                properties:
-                  location:
-                    type: string
 
   /auth/reset_password:
     post:
@@ -1596,6 +1572,104 @@ paths:
               schema:
                 type: object
 
+  /settings/external_instance_pg/status:
+    get:
+      summary: Returns whether the external instance cluster is configured and how its last setup went
+      operationId: getExternalInstancePgStatus
+      tags:
+        - setting
+      responses:
+        "200":
+          description: external instance cluster status
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ExternalInstancePgStatus"
+
+  /settings/external_instance_pg/setup:
+    post:
+      summary: Sets up the external instance cluster with its saved admin login, optionally rotating the passwords Windmill manages on it (enterprise edition only)
+      operationId: setupExternalInstancePg
+      tags:
+        - setting
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                rotate_passwords:
+                  type: boolean
+      responses:
+        "200":
+          description: the setup report, also stored as the last setup
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ExternalInstancePgSetupReport"
+
+  /settings/external_instance_pg/databases:
+    get:
+      summary: Lists the databases Windmill created on the external instance cluster, with the workspaces whose data tables, Ducklake catalogs or pending fork cleanups use each
+      operationId: listExternalInstancePgDatabases
+      tags:
+        - setting
+      responses:
+        "200":
+          description: databases by name
+          content:
+            application/json:
+              schema:
+                type: object
+                additionalProperties:
+                  $ref: "#/components/schemas/CustomInstanceDb"
+
+  /settings/external_instance_pg/databases/{name}:
+    post:
+      summary: Creates a database on the external instance cluster (enterprise edition only)
+      operationId: createExternalInstancePgDatabase
+      tags:
+        - setting
+      parameters:
+        - name: name
+          in: path
+          required: true
+          schema:
+            type: string
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                tag:
+                  $ref: "#/components/schemas/CustomInstanceDbTag"
+      responses:
+        "200":
+          description: database created
+          content:
+            application/json:
+              schema: {}
+    delete:
+      summary: Drops a database Windmill created on the external instance cluster, refused while a data table, Ducklake catalog or pending fork cleanup uses it
+      operationId: dropExternalInstancePgDatabase
+      tags:
+        - setting
+      parameters:
+        - name: name
+          in: path
+          required: true
+          schema:
+            type: string
+      responses:
+        "200":
+          description: database dropped
+          content:
+            application/json:
+              schema: {}
+
   /settings/list_custom_instance_pg_databases:
     post:
       summary: Returns the set-up statuses of custom instance pg databases
@@ -1614,13 +1688,19 @@ paths:
 
   /settings/datatable_roles:
     get:
-      summary: list the instance's data table roles
+      summary: list the data table roles of one Windmill-managed Postgres cluster
       operationId: listInstanceDatatableRoles
       tags:
         - setting
+      parameters:
+        - in: query
+          name: cluster
+          required: false
+          schema:
+            $ref: "#/components/schemas/DatatableRoleCluster"
       responses:
         "200":
-          description: the instance role catalog
+          description: the cluster's role catalog
           content:
             application/json:
               schema:
@@ -1628,7 +1708,7 @@ paths:
                 items:
                   $ref: "#/components/schemas/InstanceDatatableRole"
     post:
-      summary: create a data table role on the instance's Postgres cluster
+      summary: create a data table role on a Windmill-managed Postgres cluster
       operationId: createInstanceDatatableRole
       tags:
         - setting
@@ -1642,6 +1722,8 @@ paths:
               properties:
                 name:
                   type: string
+                cluster:
+                  $ref: "#/components/schemas/DatatableRoleCluster"
       responses:
         "200":
           description: the created role
@@ -5254,7 +5336,7 @@ paths:
                       type: string
                     resource_type:
                       type: string
-                      enum: [postgres, instance]
+                      enum: [postgres, instance, external_instance]
                     resource_path:
                       type: string
                     governing_workspace_id:
@@ -5318,6 +5400,97 @@ paths:
               schema:
                 type: string
 
+  /w/{workspace}/workspaces/datatable_acl/{datatable_name}:
+    get:
+      summary: read the owner and grants of an instance data table's database, schema or table
+      operationId: getDatatableAcl
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - name: datatable_name
+          in: path
+          required: true
+          schema:
+            type: string
+        - name: kind
+          in: query
+          required: true
+          schema:
+            type: string
+            enum: [database, schema, table]
+        - name: schema
+          in: query
+          required: false
+          schema:
+            type: string
+        - name: table
+          in: query
+          required: false
+          schema:
+            type: string
+      responses:
+        "200":
+          description: owner and grants
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/DatatableAclInfo"
+
+  /w/{workspace}/workspaces/datatable_acl/{datatable_name}/plan:
+    post:
+      summary: preview the SQL an ownership or grant change would run (data table administrators only)
+      operationId: planDatatableAcl
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - name: datatable_name
+          in: path
+          required: true
+          schema:
+            type: string
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/AclChangeRequest"
+      responses:
+        "200":
+          description: statements that would run, in a single transaction
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/AclPlan"
+
+  /w/{workspace}/workspaces/datatable_acl/{datatable_name}/apply:
+    post:
+      summary: run an ownership or grant change exactly as planned (data table administrators only)
+      operationId: applyDatatableAcl
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - name: datatable_name
+          in: path
+          required: true
+          schema:
+            type: string
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/AclChangeRequest"
+      responses:
+        "200":
+          description: change applied
+          content:
+            text/plain:
+              schema:
+                type: string
+
   /w/{workspace}/workspaces/datatable_usable_roles/{datatable_name}:
     get:
       summary: list the data table roles the caller may connect as
@@ -5421,6 +5594,21 @@ paths:
         - workspace
       parameters:
         - $ref: "#/components/parameters/WorkspaceId"
+        - name: datatable_name
+          in: query
+          description: list only this data table; each listed data table opens a connection to its database
+          schema:
+            type: string
+        - name: role_for
+          in: query
+          description: the data table `role` applies to; every other one is listed as its default role
+          schema:
+            type: string
+        - name: role
+          in: query
+          description: the role to list `role_for` as; refused, in that entry's `error`, if the caller may not use it
+          schema:
+            type: string
       responses:
         "200":
           description: table metadata of all datatables
@@ -5454,6 +5642,11 @@ paths:
           required: true
           schema:
             type: string
+        - name: role
+          in: query
+          description: the data table role to read the table as; defaults to the data table's default role
+          schema:
+            type: string
       responses:
         "200":
           description: schema of one datatable table
@@ -6777,7 +6970,7 @@ paths:
                   type: string
                 expires_in_s:
                   type: integer
-                  description: link lifetime in seconds, at most 7200 (default 600)
+                  description: link lifetime in seconds, at most 900 (default 600)
                 rd:
                   type: string
                   description: same-origin path the browser lands on after login (default /user/workspaces)
@@ -6787,12 +6980,6 @@ paths:
                     mint only while the account still has this login type (for example
                     pending_oauth), so a link stops working once the owner has set a password
                     or signed in with a provider
-                confirm:
-                  type: boolean
-                  description: >-
-                    return a /user/login_link page that signs in only when its button is
-                    clicked, instead of a link spent by opening it; set it for links sent by
-                    email, which mail scanners open on delivery (default false)
       responses:
         "201":
           description: login link minted
@@ -11486,10 +11673,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
 
       requestBody:
         description: script args
@@ -11526,10 +11714,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
 
       requestBody:
         description: script args
@@ -11566,10 +11755,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
 
       responses:
         "200":
@@ -11592,10 +11782,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
         - name: poll_delay_ms
           description: delay between polling for job updates in milliseconds
           in: query
@@ -11633,10 +11824,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
         - name: poll_delay_ms
           description: delay between polling for job updates in milliseconds
           in: query
@@ -11672,10 +11864,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
         - name: poll_delay_ms
           description: delay between polling for job updates in milliseconds
           in: query
@@ -11719,10 +11912,11 @@ paths:
         - $ref: "#/components/parameters/NewJobId"
         - $ref: "#/components/parameters/SkipPreprocessor"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
         - name: poll_delay_ms
           description: delay between polling for job updates in milliseconds
           in: query
@@ -12676,15 +12870,6 @@ paths:
           in: query
           schema:
             type: string
-        - name: kind
-          description: which conversations to list - the flow editor's test chats, the deployed flow's own (the default), or both
-          in: query
-          schema:
-            type: string
-            enum:
-              - test
-              - deployed
-              - all
       responses:
         "200":
           description: flow conversations list
@@ -12695,40 +12880,6 @@ paths:
                 items:
                   $ref: "#/components/schemas/FlowConversation"
 
-  /w/{workspace}/flow_conversations/update/{conversation_id}:
-    post:
-      summary: rename flow conversation
-      operationId: updateFlowConversation
-      tags:
-        - flow_conversations
-      parameters:
-        - $ref: "#/components/parameters/WorkspaceId"
-        - name: conversation_id
-          description: conversation id
-          in: path
-          required: true
-          schema:
-            type: string
-            format: uuid
-      requestBody:
-        required: true
-        content:
-          application/json:
-            schema:
-              type: object
-              required: [title]
-              properties:
-                title:
-                  type: string
-                  description: the chat's name
-      responses:
-        "200":
-          description: flow conversation updated
-          content:
-            text/plain:
-              schema:
-                type: string
-
   /w/{workspace}/flow_conversations/delete/{conversation_id}:
     delete:
       summary: delete flow conversation
@@ -15123,10 +15274,11 @@ paths:
           schema:
             type: boolean
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
       requestBody:
         description: flow args
         required: true
@@ -15180,10 +15332,11 @@ paths:
           schema:
             type: boolean
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
       requestBody:
         description: flow args
         required: true
@@ -15671,10 +15824,11 @@ paths:
             type: boolean
         - $ref: "#/components/parameters/NewJobId"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
 
       requestBody:
         description: preview
@@ -15702,10 +15856,11 @@ paths:
       parameters:
         - $ref: "#/components/parameters/WorkspaceId"
         - name: memory_id
-          description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
+          description: memory ID for chat-enabled flows
           in: query
           schema:
             type: string
+            format: uuid
 
       requestBody:
         description: preview
@@ -28552,7 +28707,7 @@ components:
     FlowConversation:
       type: object
       required:
-        [id, workspace_id, flow_path, created_at, updated_at, created_by, is_test]
+        [id, workspace_id, flow_path, created_at, updated_at, created_by]
       properties:
         id:
           type: string
@@ -28579,9 +28734,6 @@ components:
         created_by:
           type: string
           description: Username who created the conversation
-        is_test:
-          type: boolean
-          description: Started from the flow editor's test panel rather than a deployed run
 
     FlowConversationMessage:
       type: object
@@ -28621,50 +28773,6 @@ components:
         success:
           type: boolean
           description: Whether the message is a success
-        tool_arguments:
-          type: string
-          nullable: true
-          description: >-
-            On a tool row, the arguments the model wrote for the call. For a script, flow or
-            AI agent tool these exclude the inputs its step wires in, which only the tool's
-            job holds. Null for a provider-native web search, whose query the provider does
-            not return.
-        tool_result:
-          type: string
-          nullable: true
-          description: >-
-            On a tool row, the text the model got back from the call, or what the call
-            failed with — the row's own text names the tool rather than the reason. For a
-            provider-native web search, its citations.
-        reasoning:
-          type: string
-          nullable: true
-          description: >-
-            On an answer, the thinking that produced it; on a tool row, the thinking that
-            led to the call. Each round's thinking is on one row. The agent job's result
-            keeps the turn's thinking as a single string.
-        attachments:
-          type: array
-          nullable: true
-          description: >-
-            The files a user message carried, as object-storage references: every flow
-            input other than user_message that held one or a list of them, at most 20. Never
-            file bytes or a presigned URL.
-          items:
-            type: object
-            required: [input, s3]
-            properties:
-              input:
-                type: string
-                description: The flow input that held the file
-              s3:
-                type: string
-                description: The file's key in object storage
-              storage:
-                type: string
-                description: The secondary storage holding the file, absent for the primary one
-              filename:
-                type: string
 
     EndpointTool:
       type: object
@@ -33619,9 +33727,55 @@ components:
         - ducklake
         - datatable
 
+    ExternalInstancePgSetupStep:
+      type: object
+      required: [name, status, message]
+      properties:
+        name:
+          type: string
+        status:
+          type: string
+          enum: [ok, warning, error]
+        message:
+          type: string
+
+    ExternalInstancePgSetupReport:
+      type: object
+      required: [success, finished_at, steps]
+      properties:
+        success:
+          type: boolean
+          description: no step failed; warnings leave it true
+        finished_at:
+          type: string
+          format: date-time
+        steps:
+          type: array
+          items:
+            $ref: "#/components/schemas/ExternalInstancePgSetupStep"
+
+    ExternalInstancePgStatus:
+      type: object
+      required: [configured, database_count]
+      properties:
+        configured:
+          type: boolean
+        database_count:
+          type: integer
+        last_setup:
+          $ref: "#/components/schemas/ExternalInstancePgSetupReport"
+
+    DatatableRoleCluster:
+      type: string
+      description: >-
+        The Windmill-managed Postgres cluster a data table role is a login on: Windmill's own
+        (behind `instance` data tables) or the external instance cluster (behind
+        `external_instance` ones). Defaults to `instance`.
+      enum: [instance, external_instance]
+
     InstanceDatatableRole:
       type: object
-      required: [id, name, enabled]
+      required: [id, name, enabled, cluster]
       properties:
         id:
           type: string
@@ -33629,6 +33783,8 @@ components:
           type: string
         enabled:
           type: boolean
+        cluster:
+          $ref: "#/components/schemas/DatatableRoleCluster"
 
     DatatableRoleTenants:
       type: object
@@ -33650,8 +33806,10 @@ components:
         supported:
           type: boolean
           description: >-
-            Whether this data table can be put under roles at all. Only one backed by the
-            instance database can: a role is a login on that cluster.
+            Whether this data table can be put under roles at all. Only one on a database Windmill
+            manages can: a role is a login on that database's cluster.
+        cluster:
+          $ref: "#/components/schemas/DatatableRoleCluster"
         permissioned:
           type: boolean
         default_role:
@@ -33679,6 +33837,255 @@ components:
               datatable:
                 type: string
 
+    AclTarget:
+      description: what access is read or changed on
+      oneOf:
+        - $ref: "#/components/schemas/AclTargetDatabase"
+        - $ref: "#/components/schemas/AclTargetSchema"
+        - $ref: "#/components/schemas/AclTargetTable"
+      discriminator:
+        propertyName: kind
+        mapping:
+          database: "#/components/schemas/AclTargetDatabase"
+          schema: "#/components/schemas/AclTargetSchema"
+          table: "#/components/schemas/AclTargetTable"
+
+    AclTargetDatabase:
+      type: object
+      required: [kind]
+      properties:
+        kind:
+          type: string
+          enum: [database]
+
+    AclTargetSchema:
+      type: object
+      required: [kind, schema]
+      properties:
+        kind:
+          type: string
+          enum: [schema]
+        schema:
+          type: string
+
+    AclTargetTable:
+      type: object
+      required: [kind, schema, table]
+      properties:
+        kind:
+          type: string
+          enum: [table]
+        schema:
+          type: string
+        table:
+          type: string
+
+    AclChange:
+      description: one change to plan or apply
+      oneOf:
+        - $ref: "#/components/schemas/AclChangeSetOwner"
+        - $ref: "#/components/schemas/AclChangeGrant"
+        - $ref: "#/components/schemas/AclChangeRevoke"
+      discriminator:
+        propertyName: type
+        mapping:
+          set_owner: "#/components/schemas/AclChangeSetOwner"
+          grant: "#/components/schemas/AclChangeGrant"
+          revoke: "#/components/schemas/AclChangeRevoke"
+
+    AclChangeSetOwner:
+      type: object
+      description: >-
+        hands the target to role — for a schema, with everything already in it but an extension's
+        members, which stay with the extension
+      required: [type, role]
+      properties:
+        type:
+          type: string
+          enum: [set_owner]
+        role:
+          type: string
+          description: a data table role of the instance, or admin
+
+    AclChangeGrant:
+      type: object
+      required: [type, role, privileges, scope]
+      properties:
+        type:
+          type: string
+          enum: [grant]
+        role:
+          type: string
+          description: a data table role of the instance, or admin
+        privileges:
+          type: array
+          items:
+            type: string
+        scope:
+          $ref: "#/components/schemas/AclGrantScope"
+
+    AclChangeRevoke:
+      type: object
+      required: [type, role, privileges, scope]
+      properties:
+        type:
+          type: string
+          enum: [revoke]
+        role:
+          type: string
+          description: a data table role of the instance, other than admin
+        privileges:
+          type: array
+          items:
+            type: string
+        scope:
+          $ref: "#/components/schemas/AclGrantScope"
+        objects:
+          type: array
+          description: >-
+            objects inside the target the revoke covers, empty for the target itself. Only with the
+            target scope; a revoke on all objects of a kind is refused, since it cannot say which
+            grants it takes back.
+          items:
+            $ref: "#/components/schemas/AclObject"
+
+    AclGrantScope:
+      type: string
+      enum:
+        [
+          target,
+          all_tables,
+          all_sequences,
+          all_functions,
+          future_tables,
+          future_sequences,
+          future_functions,
+        ]
+
+    AclChangeRequest:
+      type: object
+      required: [target, change]
+      properties:
+        target:
+          $ref: "#/components/schemas/AclTarget"
+        change:
+          $ref: "#/components/schemas/AclChange"
+        statements:
+          type: array
+          description: >-
+            The statements the plan showed. Required to apply, which plans again and refuses if
+            the result differs.
+          items:
+            type: string
+
+    AclPlan:
+      type: object
+      required: [statements, warnings]
+      properties:
+        statements:
+          type: array
+          items:
+            type: string
+        warnings:
+          type: array
+          items:
+            type: string
+
+    AclObject:
+      type: object
+      required: [name, kind]
+      properties:
+        name:
+          type: string
+        kind:
+          type: string
+          description: >-
+            TABLE, SEQUENCE, FUNCTION, PROCEDURE or TYPE — what the object is. A revoke turns it
+            into the keyword it takes, ROUTINE for both routine kinds; a type's grants are read
+            only.
+        args:
+          type: string
+          description: identity arguments of a routine, which is what tells two of the same name apart
+
+    AclGrant:
+      type: object
+      required: [grantee, privileges, sources]
+      properties:
+        grantee:
+          type: string
+        privileges:
+          type: array
+          items:
+            type: string
+        object:
+          $ref: "#/components/schemas/AclObject"
+        future:
+          type: string
+          description: >-
+            set for a default privilege, naming the kind of object it covers (TABLES, SEQUENCES,
+            FUNCTIONS, TYPES, or SCHEMAS). On a schema, the defaults set in that schema; on the
+            database, the ones set database-wide, which apply in every schema and which no schema's
+            own defaults take back.
+        sources:
+          type: array
+          description: >-
+            the roles the grant comes from, each once — who granted it, or for a default privilege
+            the role whose future objects it covers. A revoke of some of the grant's privileges
+            takes them back from every source that gave them.
+          items:
+            $ref: "#/components/schemas/AclSource"
+
+    AclSource:
+      type: object
+      required: [role, privileges, reachable]
+      properties:
+        role:
+          type: string
+        privileges:
+          type: array
+          description: >-
+            what role gave of the grant's privileges. A revoke is held back only by a source out of
+            reach that gave some of what it takes back.
+          items:
+            type: string
+        reachable:
+          type: boolean
+          description: >-
+            whether the data table's connection can take back what role gave. On an object that is
+            the owner, when the connection acts for the owner, and otherwise the connection itself;
+            for a default privilege, a creating role the connection acts for. What a source out of
+            reach gave is not revocable from here; privileges only other sources gave still are.
+
+    DatatableAclInfo:
+      type: object
+      required: [owner, roles, editable, supports_maintain, dbname, grants, children]
+      properties:
+        owner:
+          type: string
+        roles:
+          type: array
+          description: the roles a change may name; empty unless the caller may change anything
+          items:
+            type: string
+        editable:
+          type: boolean
+          description: whether the caller may plan and apply changes
+        supports_maintain:
+          type: boolean
+          description: whether the server is Postgres 17+, which added the MAINTAIN table privilege
+        dbname:
+          type: string
+          description: the database the target lives in
+        grants:
+          type: array
+          items:
+            $ref: "#/components/schemas/AclGrant"
+        children:
+          type: array
+          description: a database's schemas, or a schema's tables
+          items:
+            type: string
+
     CustomInstanceDb:
       type: object
       required:
@@ -33703,6 +34110,9 @@ components:
           items:
             type: string
           description: Workspaces that reference this database via a ducklake catalog or datatable database with resource_type 'instance'. Computed at request time, not persisted.
+        workspace_id:
+          type: string
+          description: The workspace a member created this database for as a fork copy. Only that workspace can import into it or point a fork at it.
 
     NewSqsTrigger:
       type: object
@@ -34761,9 +35171,6 @@ components:
                 nullable: true
               disabled:
                 type: boolean
-              is_service_account:
-                type: boolean
-                description: Whether this membership is a service account.
             required:
               - id
               - name
@@ -35679,6 +36086,7 @@ components:
                       - postgresql
                       - mysql
                       - instance
+                      - external_instance
                   resource_path:
                     type: string
                 required:
@@ -35742,6 +36150,7 @@ components:
                     enum:
                       - postgresql
                       - instance
+                      - external_instance
                   resource_path:
                     type: string
                 required:
@@ -35825,7 +36234,17 @@ components:
 
     DataTableTables:
       type: object
-      required: [datatable_name, schemas]
+      required:
+        [
+          datatable_name,
+          schemas,
+          instance,
+          permissioned,
+          usable_roles,
+          default_role,
+          can_create_schema,
+          creatable_schemas,
+        ]
       properties:
         datatable_name:
           type: string
@@ -35838,6 +36257,26 @@ components:
               type: string
         error:
           type: string
+        instance:
+          type: boolean
+          description: on the instance database, the only kind that can be under roles or have its access edited
+        permissioned:
+          type: boolean
+        usable_roles:
+          type: array
+          description: the roles the caller may connect as, by name; empty when not under roles
+          items:
+            type: string
+        default_role:
+          type: string
+        can_create_schema:
+          type: boolean
+          description: whether the role the listing connected as may create schemas
+        creatable_schemas:
+          type: array
+          description: the schemas the role the listing connected as may create in
+          items:
+            type: string
 
     DataTableTableSchema:
       type: object

backend/windmill-api/src/apps.rs

@@ -4310,11 +4310,11 @@ async fn execute_component(
         }
     }
 
-    let flow_path = payload
+    let is_flow = payload
         .path
-        .as_deref()
-        .and_then(|path| path.strip_prefix("flow/"))
-        .map(str::to_string);
+        .as_ref()
+        .map(|p| p.starts_with("flow/"))
+        .unwrap_or(false);
 
     // Tag for inline-script jobs is read from the deployed policy in run mode;
     // only preview mode (editor) honors the client-supplied tag. This applies to
@@ -4444,9 +4444,8 @@ async fn execute_component(
 
     // Apply runnable query parameters if provided
     if let Some(ref run_query) = payload.run_query_params {
-        if let Some(flow_path) = flow_path.as_deref() {
-            crate::jobs::process_flow_run_query_params(&mut tx, uuid, &w_id, flow_path, run_query)
-                .await?;
+        if is_flow {
+            crate::jobs::process_flow_run_query_params(&mut tx, uuid, run_query).await?;
         }
     }
 

backend/windmill-api/src/jobs.rs

@@ -4369,12 +4369,12 @@ async fn count_completed_jobs_detail(
     Query(query): Query<CountCompletedJobsQuery>,
 ) -> error::JsonResult<i64> {
     let mut sqlb = SqlBuilder::select_from("v2_job_completed");
+    //FOR RLS
+    sqlb.join("v2_job USING (id)");
     sqlb.field("COUNT(*) as count");
 
-    // Filtering on v2_job.workspace_id instead would keep the planner off
-    // ix_job_workspace_id_completed_at_all and scan the whole retention window.
     if !(w_id == "admins" && query.all_workspaces.unwrap_or(false)) {
-        sqlb.and_where_eq("v2_job_completed.workspace_id", "?".bind(&w_id));
+        sqlb.and_where_eq("v2_job.workspace_id", "?".bind(&w_id));
     }
 
     if let Some(after_s_ago) = query.completed_after_s_ago {
@@ -4393,7 +4393,6 @@ async fn count_completed_jobs_detail(
     }
 
     if let Some(tags) = query.tags {
-        sqlb.join("v2_job USING (id)");
         sqlb.and_where_in(
             "v2_job.tag",
             &tags.split(",").map(|t| quote(t)).collect::<Vec<_>>(),
@@ -4401,19 +4400,7 @@ async fn count_completed_jobs_detail(
     }
 
     let sql = sqlb.sql()?;
-    let mut tx = db.begin().await?;
-    set_list_jobs_statement_timeout(&mut tx).await?;
-    let stats = sqlx::query_scalar::<_, i64>(&sql)
-        .fetch_one(&mut *tx)
-        .await
-        .map_err(|e| {
-            list_jobs_timeout_error(
-                e,
-                "Counting completed jobs",
-                "Lower completed_after_s_ago or narrow the filters.",
-            )
-        })?;
-    tx.commit().await?;
+    let stats = sqlx::query_scalar::<_, i64>(&sql).fetch_one(&db).await?;
 
     Ok(Json(stats))
 }
@@ -4442,33 +4429,6 @@ lazy_static::lazy_static! {
             .unwrap_or(30);
 }
 
-/// A client that gives up does not cancel its query, so without this bound every retry of a
-/// slow filter stacks another scan running until the connection-wide 5min timeout.
-async fn set_list_jobs_statement_timeout(tx: &mut Transaction<'_, Postgres>) -> error::Result<()> {
-    let timeout_secs = *LIST_JOBS_STATEMENT_TIMEOUT_SECS;
-    if timeout_secs > 0 {
-        sqlx::query(&format!("SET LOCAL statement_timeout = '{timeout_secs}s'"))
-            .execute(&mut **tx)
-            .await?;
-    }
-    Ok(())
-}
-
-fn list_jobs_timeout_error(e: sqlx::Error, action: &str, hint: &str) -> Error {
-    let timeout_secs = *LIST_JOBS_STATEMENT_TIMEOUT_SECS;
-    match e {
-        sqlx::Error::Database(ref db_err)
-            if timeout_secs > 0 && db_err.code().as_deref() == Some("57014") =>
-        {
-            Error::Generic(
-                StatusCode::BAD_REQUEST,
-                format!("{action} took more than {timeout_secs}s and was stopped. {hint}"),
-            )
-        }
-        e => e.into(),
-    }
-}
-
 async fn list_jobs(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
@@ -4585,14 +4545,32 @@ async fn list_jobs(
     };
     // tracing::info!("sql: {}", &sql);
     let mut tx: Transaction<'_, Postgres> = user_db.begin(&authed).await?;
-    set_list_jobs_statement_timeout(&mut tx).await?;
+
+    // A client that gives up does not cancel its query, so without this bound every retry of a
+    // slow filter stacks another scan running until the connection-wide 5min timeout.
+    let timeout_secs = *LIST_JOBS_STATEMENT_TIMEOUT_SECS;
+    if timeout_secs > 0 {
+        sqlx::query(&format!("SET LOCAL statement_timeout = '{timeout_secs}s'"))
+            .execute(&mut *tx)
+            .await?;
+    }
 
     let jobs: Vec<UnifiedJob> = sqlx::query_as(&sql)
         .fetch_all(&mut *tx)
         .warn_after_seconds_with_sql(5, format!("list_jobs: {}", sql))
         .await
-        .map_err(|e| {
-            list_jobs_timeout_error(e, "Listing jobs", "Set a start date or narrow the filters.")
+        .map_err(|e| match e {
+            sqlx::Error::Database(ref db_err)
+                if timeout_secs > 0 && db_err.code().as_deref() == Some("57014") =>
+            {
+                Error::Generic(
+                    StatusCode::BAD_REQUEST,
+                    format!(
+                        "Listing jobs took more than {timeout_secs}s and was stopped. Set a start date or narrow the filters."
+                    ),
+                )
+            }
+            e => e.into(),
         })?;
     tx.commit().await?;
 
@@ -8723,7 +8701,12 @@ fn register_potential_assets_on_inline_execution(
             .as_ref()
             .and_then(|args| args.get("database"))
             .map(|v| v.get().trim_matches('"'))
-            .and_then(|dt| dt.strip_prefix("datatable://"));
+            .and_then(|dt| dt.strip_prefix("datatable://"))
+            // `?role=` picks the connection, not the data table. Anything else after a `?` may be
+            // part of a name stored before names were restricted, so it stays.
+            .map(|dt| {
+                windmill_common::workspaces::parse_datatable_ref(dt).map_or(dt, |(name, _)| name)
+            });
         if let Some(datatable) = datatable {
             let re = regex::Regex::new(r#"SET search_path TO "([^"]+)";"#).unwrap();
             let (schema, content) = if let Some(captures) = re.captures(&preview.content) {
@@ -9563,7 +9546,7 @@ async fn run_preview_flow_job(
     .await?;
 
     // Set memory_id if provided (for agent memory)
-    if let Some(memory_id) = run_query.memory_key(&w_id, &flow_path) {
+    if let Some(memory_id) = run_query.memory_id {
         set_flow_memory_id(&mut tx, uuid, memory_id).await?;
     }
 
@@ -9577,9 +9560,6 @@ async fn run_preview_flow_job(
             &run_query,
             user_message.as_ref(),
             uuid,
-            // Run from the editor's test panel: a trial, not a real conversation.
-            true,
-            &flow_args,
         )
         .await?;
     }

backend/windmill-common/Cargo.toml

@@ -33,7 +33,6 @@ path = "src/lib.rs"
 tar.workspace = true
 hmac.workspace = true
 sha2.workspace = true
-sha1.workspace = true
 thiserror.workspace = true
 anyhow.workspace = true
 serde.workspace = true
@@ -76,6 +75,7 @@ bitflags.workspace = true
 once_cell.workspace = true
 phf.workspace = true
 tokio-postgres.workspace = true
+postgres-protocol.workspace = true
 postgres-native-tls.workspace = true
 native-tls.workspace = true
 

backend/windmill-common/src/auth.rs

@@ -20,7 +20,7 @@ use crate::{
 
 /// Whether `label` denotes a user-created token rather than a system token
 /// (`session`, `guest_session`, `ephemeral*`, `debugger-token`, `mcp-oauth-*`,
-/// `embed_app:*`, `sdk_app:*`, `impersonation:*`, `cli-login:*`). System-token labels are load-bearing —
+/// `embed_app:*`, `sdk_app:*`, `impersonation:*`). System-token labels are load-bearing —
 /// session cleanup, super_admin propagation, expiry notifications and username overrides
 /// all key off them — so they must not be user-editable. `None` (no label) is treated as
 /// a user token.
@@ -47,17 +47,10 @@ pub fn is_user_token(label: Option<&str>) -> bool {
                 && !l.starts_with(APP_EMBED_TOKEN_LABEL_PREFIX)
                 && !l.starts_with(RAW_APP_SDK_TOKEN_LABEL_PREFIX)
                 && !l.starts_with("impersonation:")
-                && !l.starts_with(CLI_LOGIN_TOKEN_LABEL_PREFIX)
         }
     }
 }
 
-/// How far ahead of a user token's expiration its owner is warned (`check_expiring_tokens` in
-/// the monitor). A token whose whole lifetime fits in this window gets no warning at all: it
-/// would arrive minutes after creation, about a lifetime its creator just picked. Its
-/// "expired and deleted" notice still goes out.
-pub const TOKEN_EXPIRY_WARNING_DAYS: i32 = 7;
-
 /// Label prefix, followed by the app path, of the token an app viewer's sandboxed iframe
 /// runs with. Reserved in [`is_user_token`], whose SQL and frontend mirrors spell it out.
 pub const APP_EMBED_TOKEN_LABEL_PREFIX: &str = "embed_app:";
@@ -66,13 +59,6 @@ pub const APP_EMBED_TOKEN_LABEL_PREFIX: &str = "embed_app:";
 /// frontend SDK. Reserved in [`is_user_token`], whose SQL and frontend mirrors spell it out.
 pub const RAW_APP_SDK_TOKEN_LABEL_PREFIX: &str = "sdk_app:";
 
-/// Label prefix, followed by the username, of the token the CLI authorization page mints for
-/// `wmill` logins. Reserved in [`is_user_token`], whose SQL and frontend mirrors spell it out:
-/// the CLI signs in again on its own once that token expires, so an expiry email for it asks
-/// the user to do nothing. Not in [`is_server_minted_label`], since the page mints it through
-/// `/users/tokens/create`.
-pub const CLI_LOGIN_TOKEN_LABEL_PREFIX: &str = "cli-login:";
-
 /// Whether `label` belongs to a namespace only the server mints, and which therefore must be
 /// rejected by `create_token`. Narrower than [`is_user_token`], which also drives label
 /// editability and expiry notifications and can afford to reserve more: `Ephemeral lsp token`,
@@ -489,16 +475,14 @@ async fn fetch_authed_from_permissioned_as_inner(
     // principal — a username freed and reassigned while its previous holder keeps a privileged
     // account — would mix one account's role with another's instance privileges.
     let member = match permissioned_as.split_once('/') {
-        Some(("u", name)) => {
-            sqlx::query!(
-                "SELECT is_admin, operator, email FROM usr where username = $1 AND \
+        Some(("u", name)) => sqlx::query!(
+            "SELECT is_admin, operator, email FROM usr where username = $1 AND \
                                          workspace_id = $2 AND disabled = false",
-                name,
-                &w_id
-            )
-            .fetch_optional(&mut *conn)
-            .await?
-        }
+            name,
+            &w_id
+        )
+        .fetch_optional(&mut *conn)
+        .await?,
         _ => None,
     };
     let resolved_email;
@@ -996,7 +980,6 @@ mod tests {
         assert!(!is_user_token(Some("embed_app:f/team/dashboard")));
         assert!(!is_user_token(Some("sdk_app:u/admin/raw app")));
         assert!(!is_user_token(Some("impersonation:admin@windmill.dev")));
-        assert!(!is_user_token(Some("cli-login:admin")));
     }
 
     #[test]

backend/windmill-common/src/datatable_roles.rs

@@ -6,22 +6,67 @@
  * LICENSE-AGPL for a copy of the license.
  */
 
-//! The instance's data table role catalog.
+//! The instance's data table role catalogs.
 //!
-//! A data table role is a real Postgres login role on the Windmill cluster, named exactly as the
-//! user named it, shared by every instance database. Windmill decides who may ask for a role (the
-//! per-data-table tenant lists in [`crate::workspaces`]); Postgres decides what the role may then
-//! touch. The catalog here is only the first half's vocabulary plus the cluster provisioning.
+//! A data table role is a real Postgres login role on one cluster — Windmill's own, or the external
+//! instance cluster — named exactly as the user named it, shared by every database Windmill manages
+//! on that cluster. Each cluster has its own catalog: a role exists where it was created and nowhere
+//! else. Windmill decides who may ask for a role (the per-data-table tenant lists in
+//! [`crate::workspaces`]); Postgres decides what the role may then touch. The catalog here is only
+//! the first half's vocabulary plus the cluster provisioning.
 //!
 //! Entries are keyed by a generated id so a rename moves nothing else: tenants name the id.
 
 use std::collections::BTreeMap;
 
+use serde::{Deserialize, Serialize};
+
 use crate::{
     error::{Error, Result},
+    workspaces::DataTableCatalogResourceType,
     DB,
 };
 
+/// The cluster a role catalog belongs to.
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "snake_case")]
+pub enum DatatableRoleCluster {
+    /// Windmill's own Postgres, behind `instance` data tables.
+    #[default]
+    Instance,
+    /// The external instance cluster, behind `external_instance` data tables.
+    ExternalInstance,
+}
+
+impl DatatableRoleCluster {
+    pub fn as_str(self) -> &'static str {
+        match self {
+            Self::Instance => "instance",
+            Self::ExternalInstance => "external_instance",
+        }
+    }
+
+    pub fn parse(value: &str) -> Result<Self> {
+        match value {
+            "instance" => Ok(Self::Instance),
+            "external_instance" => Ok(Self::ExternalInstance),
+            other => Err(Error::BadRequest(format!(
+                "Unknown data table role cluster '{other}': expected instance or external_instance"
+            ))),
+        }
+    }
+
+    /// The cluster whose roles a data table on `kind` can use. `None` for a resource-backed one,
+    /// which is never under roles.
+    pub fn of(kind: DataTableCatalogResourceType) -> Option<Self> {
+        match kind {
+            DataTableCatalogResourceType::Instance => Some(Self::Instance),
+            DataTableCatalogResourceType::ExternalInstance => Some(Self::ExternalInstance),
+            DataTableCatalogResourceType::Postgresql => None,
+        }
+    }
+}
+
 /// The connection every data table resolved to before roles existed (`custom_instance_user`). It
 /// owns every pre-existing object, so it is a reserved name rather than a catalog entry: never
 /// created, renamed or dropped.
@@ -164,28 +209,42 @@ pub async fn lock_instance_databases_governance<'a>(
 /// need the names — but callers MUST NOT let `pwd` reach a response, a log line, an audit record
 /// or an export. Nothing about who may call it: the credential is the whole risk, and `Debug` is
 /// hand-written to redact it for the same reason.
-pub async fn read_role_catalog(db: &DB) -> Result<DatatableRoleCatalog> {
-    crate::datatable_roles_oss::read_role_catalog(db).await
+pub async fn read_role_catalog(
+    db: &DB,
+    cluster: DatatableRoleCluster,
+) -> Result<DatatableRoleCatalog> {
+    crate::datatable_roles_oss::read_role_catalog(db, cluster).await
 }
 
 /// As [`read_role_catalog`], reading inside the caller's transaction so the value is the one
 /// [`lock_role_catalog`] is protecting. Same disclosure contract.
 pub async fn read_role_catalog_tx(
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    cluster: DatatableRoleCluster,
 ) -> Result<DatatableRoleCatalog> {
-    crate::datatable_roles_oss::read_role_catalog_tx(tx).await
+    crate::datatable_roles_oss::read_role_catalog_tx(tx, cluster).await
 }
 
-/// Record a role, in the caller's transaction so it commits with the `CREATE ROLE` it describes.
+/// The cluster a role belongs to, or `None` if no role has this id.
+pub async fn role_cluster(
+    tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    id: &str,
+) -> Result<Option<DatatableRoleCluster>> {
+    crate::datatable_roles_oss::role_cluster(tx, id).await
+}
+
+/// Record a role, in the caller's transaction. On Windmill's own cluster that commits it with the
+/// `CREATE ROLE` it describes; on the external cluster the role already exists by then.
 ///
 /// Authorization: writes a generated Postgres credential. Callers MUST restrict this to superadmin
 /// paths and MUST hold [`lock_role_catalog`] on `tx`.
 pub async fn insert_role_catalog_entry(
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
     id: &str,
+    cluster: DatatableRoleCluster,
     role: &InstanceDatatableRole,
 ) -> Result<()> {
-    crate::datatable_roles_oss::insert_role_catalog_entry(tx, id, role).await
+    crate::datatable_roles_oss::insert_role_catalog_entry(tx, id, cluster, role).await
 }
 
 /// Update a role's recorded name, login flag and password. Same contract as
@@ -215,7 +274,7 @@ pub fn role_id_by_name<'a>(catalog: &'a DatatableRoleCatalog, name: &str) -> Res
         .find(|(_, role)| role.name == name)
         .ok_or_else(|| {
             Error::NotFound(format!(
-                "'{name}' is not a data table role of this instance. Defined roles: {}.",
+                "'{name}' is not a data table role of this database's cluster. Defined roles: {}.",
                 catalog
                     .values()
                     .map(|r| r.name.as_str())
@@ -231,70 +290,90 @@ pub fn role_id_by_name<'a>(catalog: &'a DatatableRoleCatalog, name: &str) -> Res
     Ok(entry.0.as_str())
 }
 
-/// Every instance database the registry knows about. Role provisioning has to reach all of them:
-/// a role that cannot `CONNECT` to a database is refused by Postgres before any grant matters.
+/// Every database Windmill manages on `cluster`. Role provisioning has to reach all of them: a role
+/// that cannot `CONNECT` to a database is refused by Postgres before any grant matters.
 ///
-/// Authorization: checks nothing, and names every instance database across all workspaces. Callers
+/// Authorization: checks nothing, and names every managed database across all workspaces. Callers
 /// MUST be superadmin-gated or keep the names server-side; never return them to a workspace caller.
-pub async fn registered_instance_databases(db: &DB) -> Result<Vec<String>> {
-    crate::datatable_roles_oss::registered_instance_databases(db).await
+pub async fn registered_instance_databases(
+    db: &DB,
+    cluster: DatatableRoleCluster,
+) -> Result<Vec<String>> {
+    crate::datatable_roles_oss::registered_instance_databases(db, cluster).await
 }
 
-/// `CONNECT` on `dbname` for every enabled role, and none for `PUBLIC`. Run at role creation, at
-/// database creation, and lazily whenever an instance data table is administered, so a database
-/// provisioned before a role existed is repaired rather than left silently unreachable.
+/// `CONNECT` on `dbname` for every enabled role of `cluster`, and none for `PUBLIC`. Run at role
+/// creation, at database creation, and lazily whenever a managed data table is administered, so a
+/// database provisioned before a role existed is repaired rather than left silently unreachable.
 ///
 /// Authorization: rewrites a database's ACL with the server's own credentials and checks nothing.
 /// Callers MUST have authorized administration of `dbname` — superadmin, or an admin of the
 /// workspace governing a data table on it.
-pub async fn converge_connect_grants(db: &DB, dbname: &str) -> Result<()> {
-    crate::datatable_roles_oss::converge_connect_grants(db, dbname).await
+pub async fn converge_connect_grants(
+    db: &DB,
+    cluster: DatatableRoleCluster,
+    dbname: &str,
+) -> Result<()> {
+    crate::datatable_roles_oss::converge_connect_grants(db, cluster, dbname).await
 }
 
-/// As [`converge_connect_grants`], with a catalog the caller already read. Same contract.
+/// As [`converge_connect_grants`], with the catalog of `cluster` the caller already read. Same
+/// contract.
 pub async fn converge_connect_grants_with(
     db: &DB,
+    cluster: DatatableRoleCluster,
     dbname: &str,
     catalog: &DatatableRoleCatalog,
 ) -> Result<()> {
-    crate::datatable_roles_oss::converge_connect_grants_with(db, dbname, catalog).await
+    crate::datatable_roles_oss::converge_connect_grants_with(db, cluster, dbname, catalog).await
 }
 
-/// `CREATE ROLE <name> LOGIN PASSWORD ...; GRANT <name> TO custom_instance_user`, and `CONNECT` on
-/// every registered database. No privileges beyond that — an admin grants them through SQL or the
-/// ACL editor.
+/// `CREATE ROLE <name> LOGIN PASSWORD ...; GRANT <name> TO custom_instance_user` on `cluster`. No
+/// privileges beyond that — an admin grants them through SQL or the ACL editor.
+///
+/// On Windmill's own cluster the DDL runs on `tx`, so it commits with the catalog row. The external
+/// cluster is another server: the role is created there before `tx` commits, and callers MUST drop
+/// it again ([`drop_datatable_role`]) if `tx` then fails to commit.
 ///
 /// Authorization: creates a cluster-wide Postgres login. Callers MUST restrict this to superadmin
-/// paths, and MUST hold [`lock_role_catalog`] on the same transaction.
-pub async fn create_instance_role(
+/// paths, and MUST hold [`lock_role_catalog`] on `tx`.
+pub async fn create_datatable_role(
+    db: &DB,
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    cluster: DatatableRoleCluster,
     name: &str,
     password: &str,
 ) -> Result<()> {
-    crate::datatable_roles_oss::create_instance_role(tx, name, password).await
+    crate::datatable_roles_oss::create_datatable_role(db, tx, cluster, name, password).await
 }
 
 /// Authorization: alters a cluster-wide Postgres login. Callers MUST restrict this to superadmin
-/// paths, and MUST hold [`lock_role_catalog`] on the same transaction.
-pub async fn set_instance_role_login(
+/// paths, and MUST hold [`lock_role_catalog`] on `tx`.
+pub async fn set_datatable_role_login(
+    db: &DB,
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    cluster: DatatableRoleCluster,
     name: &str,
     enabled: bool,
 ) -> Result<()> {
-    crate::datatable_roles_oss::set_instance_role_login(tx, name, enabled).await
+    crate::datatable_roles_oss::set_datatable_role_login(db, tx, cluster, name, enabled).await
 }
 
-/// A rename discards an md5-hashed password, so the caller has to hand over a fresh one.
+/// A rename discards an md5-hashed password, so the caller has to hand over a fresh one. On the
+/// external cluster the rename lands before `tx` commits, and callers MUST rename it back if `tx`
+/// then fails to commit.
 ///
 /// Authorization: renames a cluster-wide Postgres login. Callers MUST restrict this to superadmin
-/// paths, and MUST hold [`lock_role_catalog`] on the same transaction.
-pub async fn rename_instance_role(
+/// paths, and MUST hold [`lock_role_catalog`] on `tx`.
+pub async fn rename_datatable_role(
+    db: &DB,
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    cluster: DatatableRoleCluster,
     from: &str,
     to: &str,
     password: &str,
 ) -> Result<()> {
-    crate::datatable_roles_oss::rename_instance_role(tx, from, to, password).await
+    crate::datatable_roles_oss::rename_datatable_role(db, tx, cluster, from, to, password).await
 }
 
 /// A role owning anything in any database blocks its own `DROP ROLE`, and both its objects and the
@@ -302,8 +381,9 @@ pub async fn rename_instance_role(
 /// registry. An unreachable database aborts the whole delete: dropping the role while one database
 /// still holds objects owned by it leaves those objects owned by a numeric OID nobody can name.
 ///
-/// Each pass runs as the instance's own Postgres user rather than `custom_instance_user`, which
-/// owns the databases and can therefore revoke a grant whoever made it. `custom_instance_user`
+/// Each pass runs as the cluster's administrator rather than `custom_instance_user`: on Windmill's
+/// own cluster the instance's Postgres user, on the external one its configured admin login. Both
+/// own the databases and can therefore revoke a grant whoever made it. `custom_instance_user`
 /// could only undo what it granted itself, so a privilege planted by an operator in psql — the
 /// ordinary way privileges reach a role — would survive and block the drop.
 ///
@@ -311,16 +391,19 @@ pub async fn rename_instance_role(
 /// MUST restrict this to superadmin paths, and MUST hold [`lock_role_catalog`] on `tx`.
 ///
 /// The per-database passes open their own connections and cannot join `tx`; the lock is what keeps
-/// a concurrent mutation out while they run. Only the final `DROP ROLE` is on `tx`, so it commits
-/// or rolls back with the catalog write that forgets the role. Those passes commit as they go, so
-/// callers MUST have disabled the role in an earlier committed transaction: a failure part-way
-/// then leaves a disabled role to retry, not an enabled one already stripped in some databases.
-pub async fn drop_instance_role(
+/// a concurrent mutation out while they run. On Windmill's own cluster only the final `DROP ROLE`
+/// is on `tx`, so it commits or rolls back with the catalog write that forgets the role; on the
+/// external cluster it runs there, and tolerates a role already gone so a retry after a failed
+/// commit can finish. The passes commit as they go, so callers MUST have disabled the role in an
+/// earlier committed transaction: a failure part-way then leaves a disabled role to retry, not an
+/// enabled one already stripped in some databases.
+pub async fn drop_datatable_role(
     db: &DB,
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    cluster: DatatableRoleCluster,
     name: &str,
 ) -> Result<()> {
-    crate::datatable_roles_oss::drop_instance_role(db, tx, name).await
+    crate::datatable_roles_oss::drop_datatable_role(db, tx, cluster, name).await
 }
 
 #[cfg(test)]

backend/windmill-common/src/datatable_roles_oss.rs

@@ -15,7 +15,8 @@
 
 use crate::error::Error;
 
-/// What every roles path answers without the Enterprise Edition.
+/// What every roles path answers without the Enterprise Edition. The frontend matches this exact
+/// sentence (`datatableUsableRoles.ts`) to read the refusal as "not under roles": reword both.
 pub fn datatable_roles_unavailable() -> Error {
     Error::BadRequest("Data table roles are a Windmill Enterprise Edition feature".to_string())
 }
@@ -23,12 +24,12 @@ pub fn datatable_roles_unavailable() -> Error {
 #[cfg(all(feature = "private", feature = "enterprise"))]
 pub(crate) use crate::datatable_roles_ee::{
     can_use_datatable_role, can_use_datatable_role_in_governing_workspace, converge_connect_grants,
-    converge_connect_grants_with, create_instance_role, delete_role_catalog_entry,
-    drop_instance_role, ensure_can_use_datatable_role, ensure_datatable_admin_access,
+    converge_connect_grants_with, create_datatable_role, delete_role_catalog_entry,
+    drop_datatable_role, ensure_can_use_datatable_role, ensure_datatable_admin_access,
     ensure_instance_db_grant_options_unchecked, forget_datatable_role_everywhere,
     insert_role_catalog_entry, read_role_catalog, read_role_catalog_tx,
-    registered_instance_databases, rename_instance_role, resolve_datatable_role_connection,
-    set_instance_role_login, update_role_catalog_entry,
+    registered_instance_databases, rename_datatable_role, resolve_datatable_role_connection,
+    role_cluster, set_datatable_role_login, update_role_catalog_entry,
 };
 
 #[cfg(not(all(feature = "private", feature = "enterprise")))]
@@ -38,7 +39,7 @@ pub(crate) use ce::*;
 mod ce {
     use super::datatable_roles_unavailable as unavailable;
     use crate::{
-        datatable_roles::{DatatableRoleCatalog, InstanceDatatableRole},
+        datatable_roles::{DatatableRoleCatalog, DatatableRoleCluster, InstanceDatatableRole},
         db::AuthedRef,
         error::Result,
         workspaces::{
@@ -49,17 +50,31 @@ mod ce {
 
     type Tx<'a> = sqlx::Transaction<'a, sqlx::Postgres>;
 
-    pub(crate) async fn read_role_catalog(_db: &DB) -> Result<DatatableRoleCatalog> {
+    pub(crate) async fn read_role_catalog(
+        _db: &DB,
+        _cluster: DatatableRoleCluster,
+    ) -> Result<DatatableRoleCatalog> {
         Err(unavailable())
     }
 
-    pub(crate) async fn read_role_catalog_tx(_tx: &mut Tx<'_>) -> Result<DatatableRoleCatalog> {
+    pub(crate) async fn read_role_catalog_tx(
+        _tx: &mut Tx<'_>,
+        _cluster: DatatableRoleCluster,
+    ) -> Result<DatatableRoleCatalog> {
+        Err(unavailable())
+    }
+
+    pub(crate) async fn role_cluster(
+        _tx: &mut Tx<'_>,
+        _id: &str,
+    ) -> Result<Option<DatatableRoleCluster>> {
         Err(unavailable())
     }
 
     pub(crate) async fn insert_role_catalog_entry(
         _tx: &mut Tx<'_>,
         _id: &str,
+        _cluster: DatatableRoleCluster,
         _role: &InstanceDatatableRole,
     ) -> Result<()> {
         Err(unavailable())
@@ -77,43 +92,57 @@ mod ce {
         Err(unavailable())
     }
 
-    pub(crate) async fn registered_instance_databases(_db: &DB) -> Result<Vec<String>> {
+    pub(crate) async fn registered_instance_databases(
+        _db: &DB,
+        _cluster: DatatableRoleCluster,
+    ) -> Result<Vec<String>> {
         Err(unavailable())
     }
 
-    /// Nothing to converge: with no roles to admit, an instance database keeps the `CONNECT`
-    /// grants it was created with, `PUBLIC`'s included, as it did before roles existed.
-    pub(crate) async fn converge_connect_grants(_db: &DB, _dbname: &str) -> Result<()> {
+    /// Nothing to converge: with no roles to admit, a managed database keeps the `CONNECT` grants
+    /// it was created with, as it did before roles existed.
+    pub(crate) async fn converge_connect_grants(
+        _db: &DB,
+        _cluster: DatatableRoleCluster,
+        _dbname: &str,
+    ) -> Result<()> {
         Ok(())
     }
 
     /// As [`converge_connect_grants`].
     pub(crate) async fn converge_connect_grants_with(
         _db: &DB,
+        _cluster: DatatableRoleCluster,
         _dbname: &str,
         _catalog: &DatatableRoleCatalog,
     ) -> Result<()> {
         Ok(())
     }
 
-    pub(crate) async fn create_instance_role(
+    pub(crate) async fn create_datatable_role(
+        _db: &DB,
         _tx: &mut Tx<'_>,
+        _cluster: DatatableRoleCluster,
         _name: &str,
         _password: &str,
     ) -> Result<()> {
         Err(unavailable())
     }
 
-    pub(crate) async fn set_instance_role_login(
+    pub(crate) async fn set_datatable_role_login(
+        _db: &DB,
         _tx: &mut Tx<'_>,
+        _cluster: DatatableRoleCluster,
         _name: &str,
         _enabled: bool,
     ) -> Result<()> {
         Err(unavailable())
     }
 
-    pub(crate) async fn rename_instance_role(
+    pub(crate) async fn rename_datatable_role(
+        _db: &DB,
         _tx: &mut Tx<'_>,
+        _cluster: DatatableRoleCluster,
         _from: &str,
         _to: &str,
         _password: &str,
@@ -121,12 +150,18 @@ mod ce {
         Err(unavailable())
     }
 
-    pub(crate) async fn drop_instance_role(_db: &DB, _tx: &mut Tx<'_>, _name: &str) -> Result<()> {
+    pub(crate) async fn drop_datatable_role(
+        _db: &DB,
+        _tx: &mut Tx<'_>,
+        _cluster: DatatableRoleCluster,
+        _name: &str,
+    ) -> Result<()> {
         Err(unavailable())
     }
 
     pub(crate) async fn ensure_instance_db_grant_options_unchecked(
         _db: &DB,
+        _cluster: DatatableRoleCluster,
         _dbname: &str,
     ) -> Result<()> {
         Err(unavailable())

backend/windmill-common/src/external_instance_pg.rs

@@ -0,0 +1,439 @@
+/*
+ * Author: Ruben Fiszel
+ * Copyright: Windmill Labs, Inc 2022
+ * This file and its contents are licensed under the AGPLv3 License.
+ * Please see the included NOTICE for copyright information and
+ * LICENSE-AGPL for a copy of the license.
+ */
+
+//! The external Postgres cluster behind `external_instance` data tables and Ducklake catalogs.
+//!
+//! Windmill administers that cluster itself, logged in as the user in
+//! [`EXTERNAL_INSTANCE_PG_SETTING`]. It creates `custom_instance_user` and
+//! `custom_instance_replication_user` there, with passwords it generates and keeps in
+//! [`EXTERNAL_INSTANCE_PG_STATE_SETTING`]. They share their names with the roles on Windmill's own
+//! cluster, but they are different roles with different passwords.
+//!
+//! The cluster may hold data Windmill did not create. Two Windmill instances sharing one is not
+//! supported: each would keep resetting the passwords the other depends on.
+
+use std::collections::{BTreeMap, BTreeSet};
+
+use serde::{Deserialize, Serialize};
+
+use crate::{
+    error::{Error, Result},
+    global_settings::{EXTERNAL_INSTANCE_PG_SETTING, EXTERNAL_INSTANCE_PG_STATE_SETTING},
+    instance_config::{CustomInstanceDb, ExternalInstancePg},
+    DB,
+};
+
+/// What Windmill keeps about the external cluster. Server-managed and hidden: never part of the
+/// instance config, never readable by an agent worker. No `Debug`: it carries live passwords.
+#[derive(Serialize, Deserialize, Clone, Default)]
+pub struct ExternalInstancePgState {
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub user_pwd: Option<String>,
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub replication_pwd: Option<String>,
+    /// The databases Windmill created on the cluster. It only ever drops one of these.
+    #[serde(default)]
+    pub databases: BTreeMap<String, CustomInstanceDb>,
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub last_setup: Option<ExternalInstancePgSetupReport>,
+    /// The cluster ([`external_instance_pg_address`]) the last successful setup converged. Databases
+    /// are only created on a cluster setup succeeded on: the passwords above exist as soon as setup
+    /// first runs, whether or not the cluster accepted them.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub set_up_for: Option<String>,
+}
+
+/// What identifies the cluster a configuration points at. Other fields (admin login, sslmode) can
+/// change without it becoming another cluster.
+pub fn external_instance_pg_address(config: &ExternalInstancePg) -> String {
+    format!(
+        "{}:{}",
+        config.host.trim().to_lowercase(),
+        config.port.unwrap_or(5432)
+    )
+}
+
+#[derive(Serialize, Deserialize, Clone, Debug)]
+pub struct ExternalInstancePgSetupReport {
+    /// No step failed. Warnings leave it true.
+    pub success: bool,
+    pub finished_at: chrono::DateTime<chrono::Utc>,
+    pub steps: Vec<ExternalInstancePgSetupStep>,
+}
+
+#[derive(Serialize, Deserialize, Clone, Debug)]
+pub struct ExternalInstancePgSetupStep {
+    pub name: String,
+    pub status: SetupStepStatus,
+    pub message: String,
+}
+
+#[derive(Serialize, Deserialize, Clone, Copy, Debug, PartialEq, Eq)]
+#[serde(rename_all = "lowercase")]
+pub enum SetupStepStatus {
+    Ok,
+    Warning,
+    Error,
+}
+
+/// The status the settings page shows without running anything.
+#[derive(Serialize, Debug)]
+pub struct ExternalInstancePgStatus {
+    pub configured: bool,
+    pub database_count: usize,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub last_setup: Option<ExternalInstancePgSetupReport>,
+}
+
+/// Authorization: returns the cluster's admin password and checks nothing. Callers MUST be
+/// superadmin or an internal server path.
+pub(crate) async fn read_external_instance_pg_config<'c>(
+    executor: impl sqlx::PgExecutor<'c>,
+) -> Result<Option<ExternalInstancePg>> {
+    let value = sqlx::query_scalar!(
+        "SELECT value FROM global_settings WHERE name = $1",
+        EXTERNAL_INSTANCE_PG_SETTING
+    )
+    .fetch_optional(executor)
+    .await?;
+    value
+        .map(|v| {
+            serde_json::from_value(v).map_err(|e| {
+                Error::internal_err(format!("reading {EXTERNAL_INSTANCE_PG_SETTING}: {e}"))
+            })
+        })
+        .transpose()
+}
+
+/// Authorization: returns the passwords Windmill generated on the cluster and checks nothing.
+/// Callers MUST be superadmin or an internal server path.
+pub(crate) async fn read_external_instance_pg_state<'c>(
+    executor: impl sqlx::PgExecutor<'c>,
+) -> Result<ExternalInstancePgState> {
+    let value = sqlx::query_scalar!(
+        "SELECT value FROM global_settings WHERE name = $1",
+        EXTERNAL_INSTANCE_PG_STATE_SETTING
+    )
+    .fetch_optional(executor)
+    .await?;
+    match value {
+        None => Ok(ExternalInstancePgState::default()),
+        Some(v) => serde_json::from_value(v).map_err(|e| {
+            Error::internal_err(format!("reading {EXTERNAL_INSTANCE_PG_STATE_SETTING}: {e}"))
+        }),
+    }
+}
+
+pub async fn external_instance_pg_status(db: &DB) -> Result<ExternalInstancePgStatus> {
+    let configured = read_external_instance_pg_config(db).await?.is_some();
+    let state = read_external_instance_pg_state(db).await?;
+    Ok(ExternalInstancePgStatus {
+        configured,
+        database_count: state.databases.len(),
+        last_setup: state.last_setup,
+    })
+}
+
+/// The databases Windmill created on the external cluster, without the passwords kept beside them.
+///
+/// Authorization: names every database across all workspaces, and the workspace each fork copy is
+/// reserved for, and checks nothing. Callers MUST be superadmin or an internal authorization or
+/// lifecycle path that does not return the names to a workspace caller.
+pub async fn external_instance_databases(db: &DB) -> Result<BTreeMap<String, CustomInstanceDb>> {
+    Ok(read_external_instance_pg_state(db).await?.databases)
+}
+
+/// The workspaces whose data tables or Ducklake catalogs name each database on the external cluster,
+/// and the forks whose Ducklake metadata schemas there are still waiting to be dropped: those rows
+/// outlive a settings change, and cleanup cannot drop a schema in a database that is gone. A row
+/// whose schema is already dropped only waits on object storage, which needs no database.
+///
+/// Authorization: reads every workspace's settings and checks nothing. Callers MUST be superadmin
+/// or an internal lifecycle path.
+pub async fn external_instance_database_usages<'c>(
+    db: impl sqlx::PgExecutor<'c>,
+) -> Result<BTreeMap<String, BTreeSet<String>>> {
+    let rows = sqlx::query_as::<_, (String, String)>(
+        "SELECT ws.workspace_id, entry->'database'->>'resource_path'
+         FROM workspace_settings ws
+         CROSS JOIN LATERAL jsonb_each(
+             CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
+                 THEN ws.datatable->'datatables'
+                 ELSE '{}'::jsonb END
+         ) AS dt(k, entry)
+         WHERE entry->'database'->>'resource_type' = 'external_instance'
+           AND entry->'database'->>'resource_path' IS NOT NULL
+         UNION ALL
+         SELECT ws.workspace_id, entry->'catalog'->>'resource_path'
+         FROM workspace_settings ws
+         CROSS JOIN LATERAL jsonb_each(
+             CASE WHEN jsonb_typeof(ws.ducklake->'ducklakes') = 'object'
+                 THEN ws.ducklake->'ducklakes'
+                 ELSE '{}'::jsonb END
+         ) AS dl(k, entry)
+         WHERE entry->'catalog'->>'resource_type' = 'external_instance'
+           AND entry->'catalog'->>'resource_path' IS NOT NULL
+         UNION ALL
+         SELECT workspace_id, substring(catalog FROM length('external_instance:') + 1)
+         FROM fork_ducklake_namespace
+         WHERE catalog LIKE 'external\\_instance:%' AND NOT schema_dropped",
+    )
+    .fetch_all(db)
+    .await?;
+    let mut usages: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
+    for (workspace_id, dbname) in rows {
+        usages.entry(dbname).or_default().insert(workspace_id);
+    }
+    Ok(usages)
+}
+
+/// Refuse to unset the cluster while Windmill still has databases or data table roles on it, or a
+/// workspace still points at one: every data table there would stop resolving, and every role
+/// would be a login nothing can drop any more. Allowed on every edition, so a
+/// downgraded instance can still clear a setting it no longer uses.
+pub async fn ensure_external_instance_pg_removable(db: &DB) -> Result<()> {
+    ensure_external_instance_pg_unused(db, &format!("removing {EXTERNAL_INSTANCE_PG_SETTING}"))
+        .await
+}
+
+/// Refuse while Windmill has databases or data table roles on the cluster, or a workspace points
+/// at one of its databases. `before` finishes the sentence saying what to do first.
+async fn ensure_external_instance_pg_unused(db: &DB, before: &str) -> Result<()> {
+    let state = read_external_instance_pg_state(db).await?;
+    let usages = external_instance_database_usages(db).await?;
+    let roles = sqlx::query_scalar::<_, String>(
+        "SELECT name FROM datatable_role WHERE cluster = 'external_instance' ORDER BY name",
+    )
+    .fetch_all(db)
+    .await?;
+    if state.databases.is_empty() && usages.is_empty() && roles.is_empty() {
+        return Ok(());
+    }
+    let mut held = vec![];
+    if !(state.databases.is_empty() && usages.is_empty()) {
+        let names = state
+            .databases
+            .keys()
+            .chain(usages.keys())
+            .collect::<BTreeSet<_>>()
+            .into_iter()
+            .cloned()
+            .collect::<Vec<_>>()
+            .join(", ");
+        held.push(format!("databases in use ({names})"));
+    }
+    if !roles.is_empty() {
+        held.push(format!("data table roles ({})", roles.join(", ")));
+    }
+    Err(Error::BadRequest(format!(
+        "The external instance cluster still holds {}. Drop them and repoint the data tables and \
+         Ducklake catalogs using them before {before}.",
+        held.join(" and ")
+    )))
+}
+
+/// Refuse a workspace setting that newly names an `external_instance` database on an edition
+/// without them.
+pub fn ensure_external_instance_available() -> Result<()> {
+    crate::external_instance_pg_oss::ensure_external_instance_available()
+}
+
+/// The connection an `external_instance` database resolves to: `custom_instance_user`, or the
+/// replication user, on the external cluster.
+///
+/// Authorization: returns live credentials and checks nothing. Callers MUST have authorized access
+/// to the data table that names `dbname`.
+pub async fn external_instance_connection_unchecked(
+    db: &DB,
+    dbname: &str,
+    replication: bool,
+) -> Result<crate::PgDatabase> {
+    crate::external_instance_pg_oss::external_instance_connection_unchecked(db, dbname, replication)
+        .await
+}
+
+/// Create `dbname` on the external cluster and register it. Refuses a name already taken there,
+/// whoever took it.
+///
+/// Authorization: checks nothing. Callers MUST be superadmin, or be cloning a data table they may
+/// fork into a `wm_fork_` database.
+pub async fn create_external_instance_database_unchecked(
+    db: &DB,
+    dbname: &str,
+    tag: &str,
+    for_workspace: Option<&str>,
+) -> Result<()> {
+    crate::external_instance_pg_oss::create_external_instance_database_unchecked(
+        db,
+        dbname,
+        tag,
+        for_workspace,
+    )
+    .await
+}
+
+/// Drop `dbname` from the external cluster: only a database Windmill registered creating, and still
+/// carries the mark it set there. Refused while anything uses it
+/// ([`crate::workspaces::managed_database_uses`]), except the `exempt` data table entry: the fork
+/// copy being cleaned up.
+///
+/// Authorization: checks nothing. Callers MUST be superadmin, or be deleting the fork that owns
+/// this `wm_fork_` database.
+pub async fn drop_external_instance_database_unchecked(
+    db: &DB,
+    dbname: &str,
+    exempt: Option<(&str, &str)>,
+) -> Result<()> {
+    crate::external_instance_pg_oss::drop_external_instance_database_unchecked(db, dbname, exempt)
+    .await
+}
+
+/// Serializes everything that changes which databases exist on the external cluster, or which data
+/// tables name them: setup, creates, drops, and data table saves. Held until `tx` ends.
+pub async fn lock_external_instance_pg_state(
+    tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+) -> Result<()> {
+    sqlx::query("SELECT pg_advisory_xact_lock(hashtext($1))")
+        .bind(EXTERNAL_INSTANCE_PG_STATE_SETTING)
+        .execute(&mut **tx)
+        .await?;
+    Ok(())
+}
+
+/// Refuse a data table naming `dbname` unless Windmill created it on the external cluster. Takes
+/// the lock drops take, so none can remove the database before `tx`, which saves the data table,
+/// commits.
+pub async fn ensure_external_instance_database_registered(
+    tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    dbname: &str,
+) -> Result<()> {
+    lock_external_instance_pg_state(tx).await?;
+    if read_external_instance_pg_state(&mut **tx)
+        .await?
+        .databases
+        .contains_key(dbname)
+    {
+        return Ok(());
+    }
+    Err(Error::BadRequest(format!(
+        "Windmill did not create a database named '{dbname}' on the external instance cluster. \
+         Create it from the instance settings first."
+    )))
+}
+
+/// Write [`EXTERNAL_INSTANCE_PG_SETTING`]: `None`, null or an empty string unsets it. Every writer
+/// of global settings goes through this for that key — the per-key and bulk endpoints as well as
+/// the declarative sync — instead of writing the row itself.
+///
+/// The checks and the write share one transaction holding [`lock_external_instance_pg_state`]. A
+/// check taken outside it could pass while a database create still reads the old cluster, which
+/// would then register a database there after the setting names another one.
+///
+/// Authorization: checks nothing. Callers MUST be superadmin, or the declarative instance config
+/// sync, which applies what the operator deployed.
+pub async fn write_external_instance_pg_setting(
+    db: &DB,
+    value: Option<&serde_json::Value>,
+) -> Result<()> {
+    let value = match value {
+        None | Some(serde_json::Value::Null) => None,
+        Some(serde_json::Value::String(s)) if s.trim().is_empty() => None,
+        Some(value) => Some(value),
+    };
+    let mut tx = db.begin().await?;
+    lock_external_instance_pg_state(&mut tx).await?;
+    match value {
+        None => {
+            ensure_external_instance_pg_removable(db).await?;
+            sqlx::query("DELETE FROM global_settings WHERE name = $1")
+                .bind(EXTERNAL_INSTANCE_PG_SETTING)
+                .execute(&mut *tx)
+                .await?;
+        }
+        Some(value) => {
+            crate::external_instance_pg_oss::validate_external_instance_pg_setting(value)?;
+            ensure_external_instance_pg_not_repointed(db, value).await?;
+            sqlx::query(
+                "INSERT INTO global_settings (name, value) VALUES ($1, $2)
+                 ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value, updated_at = now()",
+            )
+            .bind(EXTERNAL_INSTANCE_PG_SETTING)
+            .bind(value)
+            .execute(&mut *tx)
+            .await?;
+        }
+    }
+    tx.commit().await?;
+    tracing::info!(
+        "{} global setting {EXTERNAL_INSTANCE_PG_SETTING}",
+        if value.is_some() { "Set" } else { "Unset" }
+    );
+    Ok(())
+}
+
+/// [`write_external_instance_pg_setting`] for a settings diff: writes the key if the diff touches
+/// it, and takes it out of the diff so the generic apply does not write it again.
+///
+/// Authorization: checks nothing. Callers MUST be superadmin, or the declarative instance config
+/// sync, which applies what the operator deployed.
+pub async fn write_external_instance_pg_from_diff(
+    db: &DB,
+    diff: &mut crate::instance_config::SettingsDiff,
+) -> Result<()> {
+    if let Some(value) = diff.upserts.remove(EXTERNAL_INSTANCE_PG_SETTING) {
+        write_external_instance_pg_setting(db, Some(&value)).await?;
+    }
+    if let Some(i) = diff
+        .deletes
+        .iter()
+        .position(|k| k == EXTERNAL_INSTANCE_PG_SETTING)
+    {
+        diff.deletes.remove(i);
+        write_external_instance_pg_setting(db, None).await?;
+    }
+    Ok(())
+}
+
+/// Refuse pointing the setting at another host or port while databases or data table roles live on
+/// the current one. Data tables name databases, and the role catalog names logins, not clusters, so
+/// both would silently resolve to whatever the new cluster holds under the same names. Other fields
+/// (admin login, sslmode) may change freely.
+async fn ensure_external_instance_pg_not_repointed(
+    db: &DB,
+    value: &serde_json::Value,
+) -> Result<()> {
+    let Some(current) = read_external_instance_pg_config(db).await? else {
+        return Ok(());
+    };
+    let Ok(desired) = serde_json::from_value::<ExternalInstancePg>(value.clone()) else {
+        return Ok(());
+    };
+    if external_instance_pg_address(&current) == external_instance_pg_address(&desired) {
+        return Ok(());
+    }
+    ensure_external_instance_pg_unused(
+        db,
+        &format!("pointing {EXTERNAL_INSTANCE_PG_SETTING} at another cluster"),
+    )
+    .await
+}
+
+/// Converge the external cluster on the configured login: check what it can do, create or update
+/// Windmill's two roles with the stored passwords, and report anything that would get in the way.
+/// With `rotate_passwords`, generate new passwords first. Safe to run again; running it again is
+/// how a failed rotation is repaired.
+///
+/// Authorization: administers the external cluster with its admin credentials and checks nothing.
+/// Callers MUST be superadmin.
+pub async fn setup_external_instance_pg_unchecked(
+    db: &DB,
+    rotate_passwords: bool,
+) -> Result<ExternalInstancePgSetupReport> {
+    crate::external_instance_pg_oss::setup_external_instance_pg_unchecked(db, rotate_passwords)
+        .await
+}

backend/windmill-common/src/external_instance_pg_oss.rs

@@ -0,0 +1,81 @@
+/*
+ * Author: Ruben Fiszel
+ * Copyright: Windmill Labs, Inc 2022
+ * This file and its contents are licensed under the AGPLv3 License.
+ * Please see the included NOTICE for copyright information and
+ * LICENSE-AGPL for a copy of the license.
+ */
+
+//! Where the external instance cluster comes from: the enterprise implementation, or a refusal.
+//! `private` alone is not that edition: community builds carry it.
+
+use crate::error::Error;
+
+pub fn external_instance_pg_unavailable() -> Error {
+    Error::BadRequest(
+        "External instance databases are a Windmill Enterprise Edition feature".to_string(),
+    )
+}
+
+#[cfg(all(feature = "private", feature = "enterprise"))]
+pub(crate) use crate::external_instance_pg_ee::{
+    create_external_instance_database_unchecked, drop_external_instance_database_unchecked,
+    external_instance_connection_unchecked, setup_external_instance_pg_unchecked,
+    validate_external_instance_pg_setting,
+};
+
+#[cfg(all(feature = "private", feature = "enterprise"))]
+pub(crate) fn ensure_external_instance_available() -> crate::error::Result<()> {
+    Ok(())
+}
+
+#[cfg(not(all(feature = "private", feature = "enterprise")))]
+pub(crate) use ce::*;
+
+#[cfg(not(all(feature = "private", feature = "enterprise")))]
+mod ce {
+    use super::external_instance_pg_unavailable as unavailable;
+    use crate::{
+        error::Result, external_instance_pg::ExternalInstancePgSetupReport, PgDatabase, DB,
+    };
+
+    pub(crate) fn validate_external_instance_pg_setting(_value: &serde_json::Value) -> Result<()> {
+        Err(unavailable())
+    }
+
+    pub(crate) fn ensure_external_instance_available() -> Result<()> {
+        Err(unavailable())
+    }
+
+    pub(crate) async fn setup_external_instance_pg_unchecked(
+        _db: &DB,
+        _rotate_passwords: bool,
+    ) -> Result<ExternalInstancePgSetupReport> {
+        Err(unavailable())
+    }
+
+    pub(crate) async fn external_instance_connection_unchecked(
+        _db: &DB,
+        _dbname: &str,
+        _replication: bool,
+    ) -> Result<PgDatabase> {
+        Err(unavailable())
+    }
+
+    pub(crate) async fn create_external_instance_database_unchecked(
+        _db: &DB,
+        _dbname: &str,
+        _tag: &str,
+        _for_workspace: Option<&str>,
+    ) -> Result<()> {
+        Err(unavailable())
+    }
+
+    pub(crate) async fn drop_external_instance_database_unchecked(
+        _db: &DB,
+        _dbname: &str,
+        _exempt: Option<(&str, &str)>,
+    ) -> Result<()> {
+        Err(unavailable())
+    }
+}

backend/windmill-common/src/flow_conversations.rs

@@ -1,39 +1,12 @@
-use std::collections::HashMap;
-
 use chrono::{DateTime, Utc};
 use serde::{Deserialize, Serialize};
-use serde_json::value::RawValue;
 use sqlx::{self, FromRow};
 use uuid::Uuid;
-use windmill_types::s3::S3Object;
 
 use crate::db::DB;
 use crate::error::Result;
 use crate::utils::truncate_with_ellipsis;
 
-/// Changing it detaches every memory stored under a string memory id.
-const MEMORY_ID_NAMESPACE: Uuid = Uuid::from_u128(0x6f1c2d4e_8a3b_5c7d_9e0f_1a2b3c4d5e6f);
-
-/// Memory is stored and carried in `flow_status.memory_id` as a uuid, which names the same memory
-/// wherever it is passed, as a chat conversation id must. Any other string names a memory through a
-/// name-based (v5) uuid scoped to its workspace and flow, so the same key in two flows or two
-/// workspaces names two memories, and chat conversation ids stay unique across workspaces.
-pub fn memory_key(workspace_id: &str, flow_path: &str, memory_id: &str) -> Uuid {
-    let memory_id = memory_id.trim();
-    Uuid::parse_str(memory_id).unwrap_or_else(|_| {
-        use sha1::{Digest, Sha1};
-        let mut hasher = Sha1::new();
-        hasher.update(MEMORY_ID_NAMESPACE.as_bytes());
-        for part in [workspace_id, flow_path, memory_id] {
-            hasher.update(part.as_bytes());
-            hasher.update([0u8]);
-        }
-        let mut bytes = [0u8; 16];
-        bytes.copy_from_slice(&hasher.finalize()[..16]);
-        uuid::Builder::from_sha1_bytes(bytes).into_uuid()
-    })
-}
-
 #[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, sqlx::Type)]
 #[sqlx(type_name = "MESSAGE_TYPE", rename_all = "lowercase")]
 #[serde(rename_all = "lowercase")]
@@ -53,12 +26,8 @@ pub struct FlowConversation {
     pub created_at: DateTime<Utc>,
     pub updated_at: DateTime<Utc>,
     pub created_by: String,
-    /// Started from the flow editor's test panel rather than a deployed run.
-    pub is_test: bool,
 }
 
-/// `is_test` is written on insert. An existing conversation of the other kind refuses the
-/// turn, so preview and deployed runs never share one.
 pub async fn get_or_create_conversation_with_id(
     tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
     w_id: &str,
@@ -66,10 +35,9 @@ pub async fn get_or_create_conversation_with_id(
     username: &str,
     title: &str,
     conversation_id: Uuid,
-    is_test: bool,
 ) -> Result<FlowConversation> {
     if let Some(existing) = lock_conversation(tx, w_id, conversation_id).await? {
-        return same_kind(existing, is_test);
+        return Ok(existing);
     }
 
     // Truncate title to 25 characters max
@@ -79,16 +47,15 @@ pub async fn get_or_create_conversation_with_id(
     // wins, the others wait on it, do nothing, and read the row it created.
     let created = sqlx::query_as!(
         FlowConversation,
-        "INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title, is_test)
-         VALUES ($1, $2, $3, $4, $5, $6)
+        "INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title)
+         VALUES ($1, $2, $3, $4, $5)
          ON CONFLICT (id) DO NOTHING
-         RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test",
+         RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by",
         conversation_id,
         w_id,
         flow_path,
         username,
-        title,
-        is_test
+        title
     )
     .fetch_optional(&mut **tx)
     .await?;
@@ -96,29 +63,13 @@ pub async fn get_or_create_conversation_with_id(
         return Ok(conversation);
     }
 
-    // The concurrent first turn that won the insert may have been of the other kind.
-    let existing = lock_conversation(tx, w_id, conversation_id)
+    lock_conversation(tx, w_id, conversation_id)
         .await?
         .ok_or_else(|| {
             crate::error::Error::BadRequest(format!(
                 "conversation {conversation_id} belongs to another workspace"
             ))
-        })?;
-    same_kind(existing, is_test)
-}
-
-/// `memory_id` is the caller's to choose, so a preview run could name a deployed
-/// conversation and the reverse. A conversation's kind is fixed at creation and nothing
-/// would show the mixing afterwards, so the turn is refused before it starts.
-fn same_kind(existing: FlowConversation, is_test: bool) -> Result<FlowConversation> {
-    if existing.is_test == is_test {
-        return Ok(existing);
-    }
-    Err(crate::error::Error::BadRequest(if existing.is_test {
-        "this conversation was started from the flow editor's test panel; start a new conversation to run the deployed flow".to_string()
-    } else {
-        "this conversation belongs to the deployed flow; start a new conversation to test from the flow editor".to_string()
-    }))
+        })
 }
 
 /// Locked, so a turn orders against retention collecting the conversation
@@ -132,7 +83,7 @@ async fn lock_conversation(
 ) -> Result<Option<FlowConversation>> {
     Ok(sqlx::query_as!(
         FlowConversation,
-        "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test
+        "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by
          FROM flow_conversation
          WHERE id = $1 AND workspace_id = $2
          FOR UPDATE",
@@ -143,65 +94,6 @@ async fn lock_conversation(
     .await?)
 }
 
-/// What a row carries beyond its text. A chat is rebuilt from its rows alone, without
-/// reading jobs, so every tool row carries the model's call and what the model got back: a
-/// Windmill tool's job holds the args its input transforms produced rather than the model's,
-/// and an MCP tool's call sits among every call of the turn in the agent's job. That job's
-/// `reasoning` is one string for the whole turn, where the rows keep it per iteration.
-#[derive(Debug, Clone, Default)]
-pub struct MessageExtras {
-    pub tool_arguments: Option<String>,
-    pub tool_result: Option<String>,
-    pub reasoning: Option<String>,
-    /// The files a user message carried; see `message_attachments`.
-    pub attachments: Vec<MessageAttachment>,
-}
-
-/// The most files a user message keeps references to; the rest are dropped.
-pub const MAX_MESSAGE_ATTACHMENTS: usize = 20;
-
-/// A file a user message carried, as the object-storage reference its run received.
-#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
-pub struct MessageAttachment {
-    /// The flow input that held it.
-    pub input: String,
-    pub s3: String,
-    #[serde(skip_serializing_if = "Option::is_none")]
-    pub storage: Option<String>,
-    #[serde(skip_serializing_if = "Option::is_none")]
-    pub filename: Option<String>,
-}
-
-/// The files a run's args carry for its user message: every top-level input other than
-/// `user_message` whose value is an object-storage reference or a list of them, in input
-/// name order, capped at `MAX_MESSAGE_ATTACHMENTS`. Only the reference is kept: `presigned`
-/// grants access to the file, and any other value may be the file's bytes.
-pub fn message_attachments(args: &HashMap<String, Box<RawValue>>) -> Vec<MessageAttachment> {
-    let mut inputs: Vec<_> = args
-        .iter()
-        .filter(|(name, _)| name.as_str() != "user_message")
-        .collect();
-    inputs.sort_by(|a, b| a.0.cmp(b.0));
-    inputs
-        .into_iter()
-        .flat_map(|(name, value)| {
-            serde_json::from_str::<S3Object>(value.get())
-                .map(|object| vec![object])
-                .or_else(|_| serde_json::from_str::<Vec<S3Object>>(value.get()))
-                .unwrap_or_default()
-                .into_iter()
-                .filter(|object| !object.s3.is_empty())
-                .map(move |object| MessageAttachment {
-                    input: name.clone(),
-                    s3: object.s3,
-                    storage: object.storage,
-                    filename: object.filename,
-                })
-        })
-        .take(MAX_MESSAGE_ATTACHMENTS)
-        .collect()
-}
-
 /// Add a message to a conversation using an existing transaction
 /// If the conversation doesn't exist, logs a warning and returns Ok (no error thrown)
 /// This allows memory_id to be used for agent memory without requiring a conversation
@@ -213,7 +105,6 @@ pub async fn add_message_to_conversation_tx(
     message_type: MessageType,
     step_name: Option<&str>,
     success: bool,
-    extras: Option<&MessageExtras>,
 ) -> Result<()> {
     // Check if conversation exists first
     let conversation_exists = sqlx::query!(
@@ -234,21 +125,14 @@ pub async fn add_message_to_conversation_tx(
 
     // Insert the message
     sqlx::query!(
-        "INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success, tool_arguments, tool_result, reasoning, attachments)
-         VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)",
+        "INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success)
+         VALUES ($1, $2, $3, $4, $5, $6)",
         conversation_id,
         message_type as MessageType,
         content,
         job_id,
         step_name,
-        success,
-        extras.and_then(|e| e.tool_arguments.as_deref()),
-        extras.and_then(|e| e.tool_result.as_deref()),
-        extras.and_then(|e| e.reasoning.as_deref()),
-        extras
-            .map(|e| &e.attachments)
-            .filter(|attachments| !attachments.is_empty())
-            .map(sqlx::types::Json) as Option<sqlx::types::Json<&Vec<MessageAttachment>>>
+        success
     )
     .execute(&mut **tx)
     .await?;
@@ -280,67 +164,3 @@ pub async fn delete_conversation_memory(
 
     Ok(())
 }
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use serde_json::{json, value::to_raw_value};
-
-    fn args(values: serde_json::Value) -> HashMap<String, Box<RawValue>> {
-        values
-            .as_object()
-            .unwrap()
-            .iter()
-            .map(|(name, value)| (name.clone(), to_raw_value(value).unwrap()))
-            .collect()
-    }
-
-    #[test]
-    fn keeps_only_object_storage_references() {
-        let attachments = message_attachments(&args(json!({
-            "user_message": { "s3": "not/an/attachment.png" },
-            "avatar": { "s3": "u/a.png", "storage": "secondary", "presigned": "https://signed" },
-            "files": [
-                { "s3": "u/b.pdf", "filename": "b.pdf" },
-                { "s3": "" }
-            ],
-            "photo": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==",
-            "count": 3
-        })));
-        assert_eq!(
-            serde_json::to_value(&attachments).unwrap(),
-            json!([
-                { "input": "avatar", "s3": "u/a.png", "storage": "secondary" },
-                { "input": "files", "s3": "u/b.pdf", "filename": "b.pdf" }
-            ])
-        );
-    }
-
-    #[test]
-    fn caps_the_references_of_one_message() {
-        let files: Vec<_> = (0..25)
-            .map(|i| json!({ "s3": format!("u/{i}.png") }))
-            .collect();
-        let attachments = message_attachments(&args(json!({ "files": files })));
-        assert_eq!(attachments.len(), MAX_MESSAGE_ATTACHMENTS);
-        assert_eq!(attachments.last().unwrap().s3, "u/19.png");
-    }
-
-    /// A string names a memory only within its workspace and flow; a uuid is used as is.
-    #[test]
-    fn memory_key_scopes_strings_but_not_uuids() {
-        let key = memory_key("ws", "f/support/triage", " customer-1 ");
-        assert_eq!(key, memory_key("ws", "f/support/triage", "customer-1"));
-        assert_ne!(
-            key,
-            memory_key("other_ws", "f/support/triage", "customer-1")
-        );
-        assert_ne!(key, memory_key("ws", "f/sales/triage", "customer-1"));
-        let conversation = Uuid::from_u128(7).to_string();
-        assert_eq!(memory_key("ws", "f/a", &conversation), Uuid::from_u128(7));
-        assert_eq!(
-            memory_key("other_ws", "f/b", &conversation),
-            Uuid::from_u128(7)
-        );
-    }
-}

backend/windmill-common/src/global_settings.rs

@@ -57,6 +57,8 @@ pub const SAML_METADATA_SETTING: &str = "saml_metadata";
 pub const SMTP_SETTING: &str = "smtp_settings";
 pub const TEAMS_SETTING: &str = "teams";
 pub const INDEXER_SETTING: &str = "indexer_settings";
+pub const EXTERNAL_INSTANCE_PG_SETTING: &str = "external_instance_pg";
+pub const EXTERNAL_INSTANCE_PG_STATE_SETTING: &str = "external_instance_pg_state";
 pub const TIMEOUT_WAIT_RESULT_SETTING: &str = "timeout_wait_result";
 
 pub const UNIQUE_ID_SETTING: &str = "uid";
@@ -102,45 +104,6 @@ pub const HUB_API_SECRET_SETTING: &str = "hub_api_secret";
 pub const AUTOMATE_USERNAME_CREATION_SETTING: &str = "automate_username_creation";
 pub const DISABLE_WORKSPACE_INVITE_EMAILS_SETTING: &str = "disable_workspace_invite_emails";
 pub const DISABLE_PASSWORD_LOGIN_SETTING: &str = "disable_password_login";
-/// Ceiling, in days, on how far ahead a token minted through `POST /users/tokens/create` or
-/// `POST /users/tokens/impersonate` may expire; a request asking for more, or for no
-/// expiration at all, is shortened to it rather than refused. On those routes only: server-side
-/// mints (webhook tokens, app embed tokens, sessions) choose a lifetime the caller never picks
-/// and go straight to `create_token_internal`. Read and validated by
-/// [`parse_max_token_expiration_days`].
-pub const MAX_TOKEN_EXPIRATION_DAYS_SETTING: &str = "max_token_expiration_days";
-/// Largest `max_token_expiration_days` read as a ceiling, about 2,700 years. The token form
-/// applies the same bound (`frontend/src/lib/tokenExpiration.ts`) so that it and the server
-/// agree on whether a ceiling exists.
-pub const MAX_TOKEN_EXPIRATION_DAYS_BOUND: i64 = 1_000_000;
-
-/// Reads a stored `max_token_expiration_days`: `Ok(None)` when unset or cleared (null or an
-/// empty string), the ceiling for a whole number of days within
-/// `1..=MAX_TOKEN_EXPIRATION_DAYS_BOUND` stored as an integer, an integral float or a string of
-/// digits, and an error for anything else.
-///
-/// The settings API and config sync both reject the error at write time: the token routes can
-/// only read an unparseable value as no ceiling, so accepting a typo would silently turn the
-/// policy off. `parseMaxTokenExpirationDays` in the frontend must accept exactly the same values.
-pub fn parse_max_token_expiration_days(
-    value: Option<&serde_json::Value>,
-) -> Result<Option<i64>, String> {
-    let days = match value {
-        None | Some(serde_json::Value::Null) => return Ok(None),
-        Some(serde_json::Value::String(s)) if s.trim().is_empty() => return Ok(None),
-        Some(serde_json::Value::Number(n)) => n
-            .as_i64()
-            .or_else(|| n.as_f64().filter(|f| f.fract() == 0.0).map(|f| f as i64)),
-        Some(serde_json::Value::String(s)) => s.trim().parse::<i64>().ok(),
-        Some(_) => None,
-    };
-    match days {
-        Some(days) if (1..=MAX_TOKEN_EXPIRATION_DAYS_BOUND).contains(&days) => Ok(Some(days)),
-        _ => Err(format!(
-            "must be a whole number of days from 1 to {MAX_TOKEN_EXPIRATION_DAYS_BOUND}, or empty for no limit"
-        )),
-    }
-}
 pub const AUTO_LOGIN_PROVIDER_SETTING: &str = "auto_login_provider";
 /// Name of the SAML attribute or OIDC userinfo claim carrying the user's IdP groups. Unset or
 /// empty leaves instance-group membership entirely to SCIM.
@@ -394,6 +357,9 @@ pub const AGENT_WORKER_BLOCKED_SETTINGS: &[&str] = &[
     // resolve datatable connections through the dedicated datatable endpoints, never these.
     "custom_instance_pg_databases",
     "custom_instance_replication_pwd",
+    // The external cluster's admin login, and the passwords Windmill generated on it.
+    EXTERNAL_INSTANCE_PG_SETTING,
+    EXTERNAL_INSTANCE_PG_STATE_SETTING,
 ];
 
 /// Whether an agent worker may read the given global setting over HTTP.
@@ -850,55 +816,6 @@ pub fn workspace_integration_auth_endpoint(client_name: &str, base_url: &str) ->
 mod tests {
     use super::*;
 
-    // `frontend/src/lib/tokenExpiration.test.ts` holds the same table for the token form's
-    // parser; the two must stay in step.
-    #[test]
-    fn max_token_expiration_days_accepts_only_whole_days_within_the_bound() {
-        use serde_json::json;
-        for (stored, days) in [
-            (json!(7), 7),
-            (json!(7.0), 7),
-            (json!("7"), 7),
-            (json!(" 30 "), 30),
-            (json!("+7"), 7),
-            (
-                json!(MAX_TOKEN_EXPIRATION_DAYS_BOUND),
-                MAX_TOKEN_EXPIRATION_DAYS_BOUND,
-            ),
-        ] {
-            assert_eq!(
-                parse_max_token_expiration_days(Some(&stored)),
-                Ok(Some(days)),
-                "{stored}"
-            );
-        }
-        for cleared in [json!(null), json!(""), json!("  ")] {
-            assert_eq!(
-                parse_max_token_expiration_days(Some(&cleared)),
-                Ok(None),
-                "{cleared}"
-            );
-        }
-        assert_eq!(parse_max_token_expiration_days(None), Ok(None));
-        for bad in [
-            json!(7.5),
-            json!(0),
-            json!(-3),
-            json!("7.0"),
-            json!("1e1"),
-            json!("0x7"),
-            json!(MAX_TOKEN_EXPIRATION_DAYS_BOUND + 1),
-            json!("99999999999999999999"),
-            json!(true),
-            json!([7]),
-        ] {
-            assert!(
-                parse_max_token_expiration_days(Some(&bad)).is_err(),
-                "{bad} must be rejected"
-            );
-        }
-    }
-
     #[test]
     fn webhook_base_url_errors_never_echo_credentials() {
         // These strings reach sync-config output and operator logs, so no branch may

backend/windmill-common/src/instance_config.rs

@@ -350,6 +350,8 @@ pub struct GlobalSettings {
     pub ducklake_settings: Option<DucklakeSettings>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub custom_instance_pg_databases: Option<CustomInstancePgDatabases>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub external_instance_pg: Option<ExternalInstancePg>,
 
     // Opaque settings (EE-private structs or no clear schema)
     #[serde(skip_serializing_if = "Option::is_none")]
@@ -809,6 +811,9 @@ pub struct CustomInstanceDb {
     pub error: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub tag: Option<String>,
+    /// The workspace a member created this fork copy for. Absent when a superadmin created it.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub workspace_id: Option<String>,
 }
 
 /// Setup log entries for a custom instance database.
@@ -833,6 +838,36 @@ pub struct CustomInstanceDbLogs {
     pub replication_user_error: Option<String>,
 }
 
+// ---------------------------------------------------------------------------
+// External instance PG cluster
+// ---------------------------------------------------------------------------
+
+/// The external Postgres cluster Windmill manages for `external_instance` data tables and Ducklake
+/// catalogs. `user` logs in as the cluster's administrator: it needs `CREATEDB` and `CREATEROLE`.
+/// `dbname` is only where that login connects to run cluster-wide statements.
+///
+/// Every field defaults rather than being required: this deserializes as part of the whole
+/// instance config, and one malformed row must not make every other setting unreadable. The
+/// write path and every use reject an incomplete value instead.
+#[derive(Deserialize, Serialize, Clone, Debug, Default)]
+#[cfg_attr(feature = "instance_config_schema", derive(schemars::JsonSchema))]
+pub struct ExternalInstancePg {
+    #[serde(default)]
+    pub host: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub port: Option<u16>,
+    #[serde(default)]
+    pub user: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub password: Option<StringOrSecretRef>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub dbname: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub sslmode: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub root_certificate_pem: Option<String>,
+}
+
 // ---------------------------------------------------------------------------
 // Autoscaling (worker config)
 // ---------------------------------------------------------------------------
@@ -969,6 +1004,7 @@ pub const PROTECTED_SETTINGS: &[&str] = &[
     "ducklake_settings",
     "custom_instance_pg_databases",
     "custom_instance_replication_pwd",
+    "external_instance_pg_state",
     "uid",
     "rsa_keys",
     "jwt_secret",
@@ -994,6 +1030,8 @@ pub const HIDDEN_SETTINGS: &[&str] = &[
     // Server-only (written by setup/refresh via direct SQL), never operator-authored —
     // hidden so the config machinery can't read, rewrite, or drop it.
     "custom_instance_replication_pwd",
+    // Same for the passwords and database registry Windmill keeps for the external cluster.
+    "external_instance_pg_state",
 ];
 
 /// Top-level settings whose entire value is sensitive and must be fully redacted in logs.
@@ -1005,6 +1043,7 @@ const SENSITIVE_SETTINGS: &[&str] = &[
     "license_key",
     "ducklake_user_pg_pwd",
     "custom_instance_replication_pwd",
+    "external_instance_pg_state",
     "pip_index_url",
     "pip_extra_index_url",
     "npm_config_registry",
@@ -1030,6 +1069,7 @@ const NESTED_SENSITIVE_FIELDS: &[(&str, &[&str])] = &[
         &["secret_key", "serviceAccountKey"],
     ),
     ("custom_instance_pg_databases", &["user_pwd"]),
+    ("external_instance_pg", &["password"]),
 ];
 
 fn redact_json_value(value: &serde_json::Value) -> serde_json::Value {
@@ -1358,11 +1398,8 @@ pub async fn sync_global_settings_declarative(
     crate::global_settings::parse_allowed_origins_setting(desired.get(origins_key))
         .map_err(|e| anyhow::anyhow!("{origins_key}: {e}"))?;
 
-    let max_expiration_key = crate::global_settings::MAX_TOKEN_EXPIRATION_DAYS_SETTING;
-    crate::global_settings::parse_max_token_expiration_days(desired.get(max_expiration_key))
-        .map_err(|e| anyhow::anyhow!("{max_expiration_key}: {e}"))?;
-
-    let diff = diff_global_settings(current, desired, ApplyMode::Replace);
+    let mut diff = diff_global_settings(current, desired, ApplyMode::Replace);
+    crate::external_instance_pg::write_external_instance_pg_from_diff(db, &mut diff).await?;
     apply_settings_diff(db, &diff).await?;
 
     Ok(())
@@ -1495,6 +1532,10 @@ pub fn resolve_env_refs(settings: &mut GlobalSettings) -> Result<(), String> {
         resolve_env_option(&mut pg.user_pwd)?;
     }
 
+    if let Some(pg) = &mut settings.external_instance_pg {
+        resolve_env_option(&mut pg.password)?;
+    }
+
     Ok(())
 }
 
@@ -2464,39 +2505,33 @@ mod tests {
     }
 
     #[test]
-    fn custom_instance_replication_pwd_is_isolated_from_config() {
-        // The replication-role password is server-only: written by setup/refresh via direct
-        // SQL, never operator-authored. It must stay out of the declarative config surface
-        // (hidden on read) and be undeletable, so config sync can't read, rewrite, or drop it.
-        assert!(HIDDEN_SETTINGS.contains(&"custom_instance_replication_pwd"));
-        assert!(PROTECTED_SETTINGS.contains(&"custom_instance_replication_pwd"));
-        assert!(SENSITIVE_SETTINGS.contains(&"custom_instance_replication_pwd"));
+    fn server_generated_db_passwords_are_isolated_from_config() {
+        // These hold passwords the server generates: written by setup/refresh via direct SQL,
+        // never operator-authored. They must stay out of the declarative config surface
+        // (hidden on read) and be undeletable, so config sync can't read, rewrite, or drop them.
+        for key in [
+            "custom_instance_replication_pwd",
+            "external_instance_pg_state",
+        ] {
+            assert!(HIDDEN_SETTINGS.contains(&key), "{key}");
+            assert!(PROTECTED_SETTINGS.contains(&key), "{key}");
+            assert!(SENSITIVE_SETTINGS.contains(&key), "{key}");
 
-        // A stray desired value (e.g. flattened into `extra`) is ignored, not upserted.
-        let mut desired = BTreeMap::new();
-        desired.insert(
-            "custom_instance_replication_pwd".to_string(),
-            serde_json::json!("attacker-set"),
-        );
-        let diff = diff_global_settings(&BTreeMap::new(), &desired, ApplyMode::Merge);
-        assert!(
-            diff.upserts.is_empty(),
-            "hidden setting must not be upserted"
-        );
+            // A stray desired value (e.g. flattened into `extra`) is ignored, not upserted.
+            let mut desired = BTreeMap::new();
+            desired.insert(key.to_string(), serde_json::json!("attacker-set"));
+            let diff = diff_global_settings(&BTreeMap::new(), &desired, ApplyMode::Merge);
+            assert!(diff.upserts.is_empty(), "{key} must not be upserted");
 
-        // A current value is never deleted by a Replace that omits it.
-        let mut current = BTreeMap::new();
-        current.insert(
-            "custom_instance_replication_pwd".to_string(),
-            serde_json::json!("live"),
-        );
-        let diff = diff_global_settings(&current, &BTreeMap::new(), ApplyMode::Replace);
-        assert!(
-            !diff
-                .deletes
-                .contains(&"custom_instance_replication_pwd".to_string()),
-            "hidden setting must not be deleted"
-        );
+            // A current value is never deleted by a Replace that omits it.
+            let mut current = BTreeMap::new();
+            current.insert(key.to_string(), serde_json::json!("live"));
+            let diff = diff_global_settings(&current, &BTreeMap::new(), ApplyMode::Replace);
+            assert!(
+                !diff.deletes.contains(&key.to_string()),
+                "{key} must not be deleted"
+            );
+        }
     }
 
     #[test]

backend/windmill-common/src/lib.rs

@@ -58,6 +58,10 @@ pub mod ee_oss;
 pub mod email_ee;
 pub mod email_oss;
 pub mod error;
+pub mod external_instance_pg;
+#[cfg(all(feature = "private", feature = "enterprise"))]
+mod external_instance_pg_ee;
+pub mod external_instance_pg_oss;
 pub mod external_ip;
 #[cfg(feature = "private")]
 pub mod feature_usage_ee;
@@ -1024,6 +1028,20 @@ impl Future for TokioPgConnection {
     }
 }
 
+impl TokioPgConnection {
+    /// Drive the connection and hand back what the server sends outside of a query's response —
+    /// notices above all, which driving it as a future silently discards.
+    pub fn poll_message(
+        &mut self,
+        cx: &mut core::task::Context<'_>,
+    ) -> core::task::Poll<Option<Result<tokio_postgres::AsyncMessage, tokio_postgres::Error>>> {
+        match self {
+            TokioPgConnection::Tls(conn) => conn.poll_message(cx),
+            TokioPgConnection::NoTls(conn) => conn.poll_message(cx),
+        }
+    }
+}
+
 impl PgDatabase {
     /// The role the connection logs in as, whichever way it authenticates.
     pub fn login_name(&self) -> &str {
@@ -1068,7 +1086,13 @@ impl PgDatabase {
                 if err_str.contains("password authentication failed for user")
                     && err_str.contains("custom_instance_user")
                 {
-                    if let Some(db) = main_db {
+                    // The external instance cluster has a `custom_instance_user` of its own, whose
+                    // password setup manages. Rotating the local one would break every instance
+                    // data table and fix nothing.
+                    let local = PgDatabase::parse_uri(&get_database_url().await?.as_str().await)?;
+                    let on_local_cluster = local.host == self.host
+                        && local.port.unwrap_or(5432) == self.port.unwrap_or(5432);
+                    if let Some(db) = main_db.filter(|_| on_local_cluster) {
                         tracing::warn!(
                             "custom_instance_user password auth failed, refreshing and retrying..."
                         );
@@ -1539,26 +1563,30 @@ pub(crate) fn instance_db_grants(dbname: &str) -> String {
     )
 }
 
-/// Re-apply [`instance_db_grants`] to an instance database provisioned before data table roles
-/// existed, whose grants carry no grant option. Connects as the instance's own Postgres user —
-/// the database and `public` schema owner — since only it can hand out an option it holds.
+/// Re-apply [`instance_db_grants`] to a managed database provisioned before data table roles
+/// existed, whose grants carry no grant option. Connects as the cluster's administrator — the
+/// database and `public` schema owner — since only it can hand out an option it holds.
 ///
-/// Authorization: reaches an instance database with the server's own credentials and checks
+/// Authorization: reaches a managed database with the server's own credentials and checks
 /// nothing. Callers MUST have authorized administration of `dbname` — superadmin, or an admin of
 /// the workspace governing a data table on it.
 pub async fn ensure_instance_db_grant_options_unchecked(
     db: &DB,
+    cluster: crate::datatable_roles::DatatableRoleCluster,
     dbname: &str,
 ) -> error::Result<()> {
-    crate::datatable_roles_oss::ensure_instance_db_grant_options_unchecked(db, dbname).await
+    crate::datatable_roles_oss::ensure_instance_db_grant_options_unchecked(db, cluster, dbname)
+        .await
 }
 
 /// Create a custom instance database: CREATE DATABASE, grant permissions, register in global_settings.
-/// The `tag` is stored in global_settings metadata (e.g. "datatable" or "ducklake").
+/// The `tag` is stored in global_settings metadata (e.g. "datatable" or "ducklake"). `for_workspace`
+/// is the workspace a member creates a fork copy for; see [`ensure_fork_database_available_to`].
 pub async fn create_custom_instance_database(
     db: &DB,
     dbname: &str,
     tag: &str,
+    for_workspace: Option<&str>,
 ) -> error::Result<()> {
     let dbname = dbname.trim();
     validate_dbname(dbname)?;
@@ -1612,7 +1640,8 @@ pub async fn create_custom_instance_database(
         },
         "success": true,
         "error": null,
-        "tag": tag
+        "tag": tag,
+        "workspace_id": for_workspace,
     });
     sqlx::query!(
         r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}', (COALESCE(value->'databases', '{}'::jsonb) || to_jsonb($1::json))) WHERE name = 'custom_instance_pg_databases'"#,
@@ -1624,7 +1653,13 @@ pub async fn create_custom_instance_database(
     // A data table role can only reach a database it may CONNECT to, and PUBLIC's default CONNECT
     // would otherwise let every role in regardless of what this instance defines. Best-effort: a
     // failure here leaves the database usable as `admin`, and the next role change repairs it.
-    if let Err(e) = crate::datatable_roles::converge_connect_grants(db, dbname).await {
+    if let Err(e) = crate::datatable_roles::converge_connect_grants(
+        db,
+        crate::datatable_roles::DatatableRoleCluster::Instance,
+        dbname,
+    )
+    .await
+    {
         tracing::warn!("Could not set CONNECT grants on instance database '{dbname}': {e}");
     }
 
@@ -1632,6 +1667,71 @@ pub async fn create_custom_instance_database(
     Ok(())
 }
 
+/// The system's CA bundle file, for libpq clients that cannot take `sslrootcert=system`: that value
+/// needs libpq 16, and verify-full only.
+pub fn system_ca_bundle() -> Option<std::path::PathBuf> {
+    std::env::var_os("SSL_CERT_FILE")
+        .map(std::path::PathBuf::from)
+        .into_iter()
+        .chain(
+            [
+                "/etc/ssl/certs/ca-certificates.crt",
+                "/etc/pki/tls/certs/ca-bundle.crt",
+                "/etc/ssl/cert.pem",
+                "/etc/ssl/ca-bundle.pem",
+            ]
+            .map(std::path::PathBuf::from),
+        )
+        .find(|path| path.is_file())
+}
+
+/// Refuse a workspace member writing a fork copy into, or pointing a fork at, the managed database
+/// `dbname` of `kind`, unless `w_id` created it for that ([`create_custom_instance_database`], or
+/// its external instance counterpart) and nothing uses it yet. The `wm_fork_` prefix is no
+/// authorization: every database of a cluster answers to the same `custom_instance_user`, so a name
+/// is all it takes to reach another workspace's copy.
+///
+/// Authorization: reads the global registries and every workspace's settings, and names other
+/// workspaces in its refusal. Callers MUST have authorized `w_id` for the caller first — a member
+/// of it forking or importing there — and MUST NOT call it on a workspace the caller is not in.
+pub async fn ensure_fork_database_available_to(
+    db: &DB,
+    kind: workspaces::DataTableCatalogResourceType,
+    dbname: &str,
+    w_id: &str,
+) -> error::Result<()> {
+    let created_for = match kind {
+        workspaces::DataTableCatalogResourceType::ExternalInstance => {
+            external_instance_pg::external_instance_databases(db)
+                .await?
+                .remove(dbname)
+                .and_then(|entry| entry.workspace_id)
+        }
+        _ => sqlx::query_scalar::<_, Option<String>>(
+            "SELECT value->'databases'->$1->>'workspace_id' FROM global_settings
+             WHERE name = 'custom_instance_pg_databases'",
+        )
+        .bind(dbname)
+        .fetch_optional(db)
+        .await?
+        .flatten(),
+    };
+    if created_for.as_deref() != Some(w_id) {
+        return Err(Error::BadRequest(format!(
+            "Database '{dbname}' was not created for a fork of workspace '{w_id}'"
+        )));
+    }
+    let uses =
+        workspaces::managed_database_uses(&mut *db.acquire().await?, kind, dbname, None).await?;
+    if !uses.is_empty() {
+        return Err(Error::BadRequest(format!(
+            "Database '{dbname}' is already in use: {}",
+            uses.join(", ")
+        )));
+    }
+    Ok(())
+}
+
 /// Connection options parsed from a database URL.
 ///
 /// The only place a database URL becomes `PgConnectOptions`. Providers that mint the password

backend/windmill-common/src/query_builders.rs

@@ -329,6 +329,7 @@ pub fn try_expand_internal_db_query(
         "ALTER_TABLE" => expand_alter_table(json_str, db_type).map(ExpandedQuery::sql),
         "CREATE_SCHEMA" => expand_create_schema(json_str, db_type).map(ExpandedQuery::sql),
         "DROP_SCHEMA" => expand_drop_schema(json_str, db_type).map(ExpandedQuery::sql),
+        "RENAME_SCHEMA" => expand_rename_schema(json_str, db_type).map(ExpandedQuery::sql),
         // Metadata queries
         "LOAD_TABLE_METADATA" => expand_load_table_metadata(json_str, db_type),
         "FOREIGN_KEYS" => expand_foreign_keys(json_str, db_type).map(ExpandedQuery::sql),
@@ -1716,6 +1717,13 @@ struct DropSchemaPayload {
     ducklake: Option<String>,
 }
 
+#[derive(Deserialize)]
+struct RenameSchemaPayload {
+    schema: String,
+    new_schema: String,
+    ducklake: Option<String>,
+}
+
 #[derive(Debug, Clone, Deserialize)]
 struct TableEditorColumn {
     name: String,
@@ -2004,6 +2012,23 @@ fn expand_drop_schema(json_str: &str, db_type: DbType) -> Result<String, String>
     Ok(maybe_wrap_ducklake(query, p.ducklake.as_deref()))
 }
 
+fn expand_rename_schema(json_str: &str, db_type: DbType) -> Result<String, String> {
+    let p: RenameSchemaPayload = serde_json::from_str(json_str)
+        .map_err(|e| format!("Invalid RENAME_SCHEMA payload: {}", e))?;
+    if !matches!(db_type, DbType::Postgresql | DbType::Snowflake) || p.ducklake.is_some() {
+        return Err(format!(
+            "Renaming a schema is not supported on {:?}",
+            db_type
+        ));
+    }
+    let query = format!(
+        "ALTER SCHEMA {} RENAME TO {};",
+        qi(&p.schema, db_type),
+        qi(&p.new_schema, db_type)
+    );
+    Ok(query)
+}
+
 fn expand_create_table(json_str: &str, db_type: DbType) -> Result<String, String> {
     let p: CreateTablePayload = serde_json::from_str(json_str)
         .map_err(|e| format!("Invalid CREATE_TABLE payload: {}", e))?;
@@ -2598,7 +2623,9 @@ WHERE table_catalog = current_database()",
                 )
             } else {
                 (
-                    "\nWHERE c.relkind = 'r' AND a.attnum > 0 AND NOT a.attisdropped\n    AND ns.nspname != 'pg_catalog' AND ns.nspname != 'information_schema'".to_string(),
+                    // pg_catalog is readable by everyone: without the privilege check this lists
+                    // tables of schemas the connection's role cannot even enter.
+                    "\nWHERE c.relkind = 'r' AND a.attnum > 0 AND NOT a.attisdropped\n    AND ns.nspname != 'pg_catalog' AND ns.nspname != 'information_schema'\n    AND has_schema_privilege(ns.oid, 'USAGE')".to_string(),
                     ",\n    ns.nspname AS schema_name,\n    c.relname AS table_name".to_string(),
                     "\nJOIN pg_catalog.pg_class c ON a.attrelid = c.oid\nJOIN pg_catalog.pg_namespace ns ON c.relnamespace = ns.oid".to_string(),
                     "ns.nspname, c.relname, a.attnum".to_string(),
@@ -4101,6 +4128,13 @@ mod tests {
         assert_eq!(sql, "DROP SCHEMA \"old_schema\" CASCADE;");
     }
 
+    #[test]
+    fn test_expand_rename_schema() {
+        let marker = r#"-- WM_INTERNAL_DB_RENAME_SCHEMA {"schema":"old","new_schema":"new"}"#;
+        let sql = expand_code(marker, &ScriptLang::Postgresql);
+        assert_eq!(sql, "ALTER SCHEMA \"old\" RENAME TO \"new\";");
+    }
+
     #[test]
     fn test_expand_create_schema_with_ducklake() {
         let marker = r#"-- WM_INTERNAL_DB_CREATE_SCHEMA {"schema":"s","ducklake":"lake"}"#;
@@ -4468,6 +4502,7 @@ mod tests {
         assert!(sql.contains("schema_name"));
         assert!(sql.contains("table_name"));
         assert!(sql.contains("c.relkind = 'r'"));
+        assert!(sql.contains("has_schema_privilege(ns.oid, 'USAGE')"));
     }
 
     #[test]

backend/windmill-common/src/utils.rs

@@ -976,33 +976,18 @@ fn six_fields_hint(schedule_str: &str, version: Option<&str>, seconds_required:
 }
 
 impl ScheduleType {
-    /// `NotFound` means the expression has no run left (an expired year, an impossible
-    /// date), and schedule pushes disable the schedule on it. Every other error must stay
-    /// transient: croner fails across a DST jump longer than an hour (Antarctica/Troll)
-    /// and succeeds again once the jump has passed.
     pub fn find_next(
         &self,
         starting_from: &chrono::DateTime<chrono_tz::Tz>,
-    ) -> Result<chrono::DateTime<chrono_tz::Tz>> {
-        let no_run_left = || {
-            Error::NotFound(format!(
-                "cron: the schedule has no run left after {}",
-                starting_from.format("%Y-%m-%d %H:%M:%S %Z")
-            ))
-        };
+    ) -> chrono::DateTime<chrono_tz::Tz> {
         match self {
             ScheduleType::Croner(croner_schedule) => croner_schedule
                 .find_next_occurrence(starting_from, false)
-                .map_err(|e| match e {
-                    croner::errors::CronError::TimeSearchLimitExceeded => no_run_left(),
-                    e => Error::internal_err(format!(
-                        "cron: could not compute the run after {}: {e}",
-                        starting_from.format("%Y-%m-%d %H:%M:%S %Z")
-                    )),
-                }),
-            ScheduleType::Cron(schedule) => {
-                schedule.after(starting_from).next().ok_or_else(no_run_left)
-            }
+                .expect("cron: a schedule should have a next event"),
+            ScheduleType::Cron(schedule) => schedule
+                .after(starting_from)
+                .next()
+                .expect("cron: a schedule should have a next event"),
         }
     }
 
@@ -1724,22 +1709,6 @@ mod tests {
         assert!(!err.contains("6 fields"), "{err}");
     }
 
-    #[test]
-    fn find_next_reports_only_a_cron_with_no_run_left_as_not_found() {
-        use chrono::TimeZone;
-        let troll: chrono_tz::Tz = "Antarctica/Troll".parse().unwrap();
-        // Troll's clocks jump from 01:00 to 03:00 on the last Sunday of March.
-        let before_jump = troll.with_ymd_and_hms(2027, 3, 28, 0, 30, 0).unwrap();
-
-        let expired = ScheduleType::from_str("0 0 9 1 1 * 2026", Some("v1"), true).unwrap();
-        let err = expired.find_next(&before_jump).unwrap_err();
-        assert!(matches!(err, Error::NotFound(_)), "{err}");
-
-        let across_jump = ScheduleType::from_str("0 30 1 * * *", Some("v2"), true).unwrap();
-        let err = across_jump.find_next(&before_jump).unwrap_err();
-        assert!(!matches!(err, Error::NotFound(_)), "{err}");
-    }
-
     /// A worker that restarts must land on the exact same name to reclaim its `worker_ping`
     /// row, while still never colliding with the other workers of its own process. The
     /// suffix must also stay a single `-` segment, which is what the interactive shell tag

backend/windmill-common/src/workspaces.rs

@@ -1397,6 +1397,18 @@ pub enum DataTableCatalogResourceType {
     #[strum(serialize = "postgres")]
     Postgresql,
     Instance,
+    /// On the external instance cluster ([`crate::external_instance_pg`]). Enterprise Edition.
+    #[serde(rename = "external_instance")]
+    #[strum(serialize = "external_instance")]
+    ExternalInstance,
+}
+
+impl DataTableCatalogResourceType {
+    /// A database Windmill created and administers, on its own cluster or the external one, as
+    /// opposed to one a user brought as a resource.
+    pub fn is_windmill_managed(self) -> bool {
+        matches!(self, Self::Instance | Self::ExternalInstance)
+    }
 }
 
 /// Build a self-teaching error for an unresolved `datatable://<name>` reference.
@@ -1471,14 +1483,82 @@ pub struct GoverningDatatable {
     pub datatable: DataTable,
 }
 
+/// Everything still using the Windmill-managed database `dbname`, one description per use: data
+/// table entries naming it, fork entries pointing at those, Ducklake catalogs on it, and fork
+/// Ducklake metadata schemas there that cleanup has not dropped yet. `exempt` is the one data table
+/// entry, `(workspace_id, name)`, the caller is about to stop using it through; pointers at that
+/// entry still count, since dropping the database would leave them resolving to nothing.
+///
+/// Authorization: reads every workspace's settings and checks nothing. Callers MUST only turn the
+/// answer into a refusal for someone allowed to administer `dbname`.
+pub async fn managed_database_uses(
+    conn: &mut sqlx::PgConnection,
+    kind: DataTableCatalogResourceType,
+    dbname: &str,
+    exempt: Option<(&str, &str)>,
+) -> Result<Vec<String>> {
+    let (exempt_workspace, exempt_name) = exempt.unzip();
+    Ok(sqlx::query_scalar::<_, String>(
+        "WITH entries AS (
+             SELECT ws.workspace_id::text AS workspace_id, dt.key AS name, dt.value
+             FROM workspace_settings ws
+             CROSS JOIN LATERAL jsonb_each(
+                 CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
+                     THEN ws.datatable->'datatables' ELSE '{}'::jsonb END) dt
+         ), naming AS (
+             SELECT workspace_id, name FROM entries
+             WHERE value->'database'->>'resource_type' = $1
+               AND value->'database'->>'resource_path' = $2
+         )
+         SELECT format('data table ''%s'' in workspace ''%s''', name, workspace_id) FROM naming
+         WHERE $3::text IS NULL OR NOT (workspace_id = $3 AND name = $4)
+         UNION ALL
+         SELECT format('data table ''%s'' in workspace ''%s'', which points at the one in ''%s''',
+                       e.name, e.workspace_id, n.workspace_id)
+         FROM entries e JOIN naming n
+           ON e.value->'reference'->>'workspace_id' = n.workspace_id
+          AND e.value->'reference'->>'datatable' = n.name
+         UNION ALL
+         SELECT format('Ducklake ''%s'' in workspace ''%s''', dl.key, ws.workspace_id)
+         FROM workspace_settings ws
+         CROSS JOIN LATERAL jsonb_each(
+             CASE WHEN jsonb_typeof(ws.ducklake->'ducklakes') = 'object'
+                 THEN ws.ducklake->'ducklakes' ELSE '{}'::jsonb END) dl
+         WHERE dl.value->'catalog'->>'resource_type' = $1
+           AND dl.value->'catalog'->>'resource_path' = $2
+         UNION ALL
+         SELECT format('the Ducklake namespace of fork ''%s'', not cleaned up yet', workspace_id)
+         FROM fork_ducklake_namespace
+         WHERE catalog = $1 || ':' || $2 AND NOT schema_dropped
+         ORDER BY 1",
+    )
+    .bind(kind.as_ref())
+    .bind(dbname)
+    .bind(exempt_workspace)
+    .bind(exempt_name)
+    .fetch_all(&mut *conn)
+    .await?)
+}
+
+/// Held by fork cleanup of `w_id`'s data tables and by forking `w_id`, which can hand the new fork
+/// pointers at them, so a pointer cannot appear between cleanup's check and its drop.
+pub async fn lock_fork_datatables(conn: &mut sqlx::PgConnection, w_id: &str) -> Result<()> {
+    sqlx::query("SELECT pg_advisory_xact_lock(hashtext('fork_datatables:' || $1))")
+        .bind(w_id)
+        .execute(&mut *conn)
+        .await?;
+    Ok(())
+}
+
 impl GoverningDatatable {
-    /// Backed by the Windmill instance's own Postgres, which is the only substrate data table
-    /// roles apply to.
-    pub fn is_instance(&self) -> bool {
+    /// The Windmill-managed cluster whose data table roles this entry can use. `None` for a
+    /// resource-backed one: roles are logins Windmill creates, and it creates none on a host a
+    /// workspace admin chose.
+    pub fn role_cluster(&self) -> Option<crate::datatable_roles::DatatableRoleCluster> {
         self.datatable
             .database
             .as_ref()
-            .is_some_and(|d| d.resource_type == DataTableCatalogResourceType::Instance)
+            .and_then(|d| crate::datatable_roles::DatatableRoleCluster::of(d.resource_type))
     }
 }
 
@@ -1613,7 +1693,8 @@ pub async fn resolve_workspace_governing_datatables(
 }
 
 /// Build the `admin` connection for a governing entry: `custom_instance_user` for an instance
-/// database, the user's own resource for a BYO-postgres one.
+/// database, on Windmill's cluster or the external one; the user's own resource for a BYO-postgres
+/// one.
 async fn resolve_datatable_connection_unchecked(
     db: &DB,
     governing: &GoverningDatatable,
@@ -1624,7 +1705,16 @@ async fn resolve_datatable_connection_unchecked(
         .database
         .as_ref()
         .expect("a governing entry owns a database");
-    if database.resource_type == DataTableCatalogResourceType::Instance {
+    if database.resource_type == DataTableCatalogResourceType::ExternalInstance {
+        let pg_creds = crate::external_instance_pg::external_instance_connection_unchecked(
+            db,
+            &database.resource_path,
+            replication,
+        )
+        .await?;
+        serde_json::to_value(&pg_creds)
+            .map_err(|e| Error::internal_err(format!("Error serializing pg creds: {}", e)))
+    } else if database.resource_type == DataTableCatalogResourceType::Instance {
         let mut pg_creds = PgDatabase::parse_uri(&get_database_url().await?.as_str().await)?;
         pg_creds.dbname = database.resource_path.clone();
         if replication {
@@ -1967,6 +2057,8 @@ pub fn strip_datatable_permissions(
 /// As [`parse_datatable_ref`], except that an entry whose stored name itself contains `?` — which
 /// names could before they were restricted — resolves by that exact name, without a role. It is
 /// looked up first, so `sales?role=x` never reaches a different entry than the one stored so.
+/// When `sales` is stored too, the reference means either one, and is refused rather than
+/// resolved to whichever is looked up first.
 ///
 /// Authorization: checks nothing, and its answer reveals whether `w_id` stores that exact name.
 /// Callers MUST already act for `w_id` — a job of it, or a caller authenticated into it — and
@@ -1977,16 +2069,26 @@ pub async fn parse_datatable_ref_for(
     reference: &str,
 ) -> Result<(String, Option<String>)> {
     if reference.contains('?') {
-        let exists = sqlx::query_scalar::<_, Option<bool>>(
-            "SELECT (datatable->'datatables') ? $2 FROM workspace_settings WHERE workspace_id = $1",
+        let role_target = parse_datatable_ref(reference)
+            .ok()
+            .and_then(|(name, role)| role.map(|_| name));
+        let (exists, target_exists) = sqlx::query_as::<_, (Option<bool>, Option<bool>)>(
+            "SELECT (datatable->'datatables') ? $2, (datatable->'datatables') ? $3
+             FROM workspace_settings WHERE workspace_id = $1",
         )
         .bind(w_id)
         .bind(reference)
+        .bind(role_target)
         .fetch_optional(db)
         .await?
-        .flatten()
-        .unwrap_or(false);
-        if exists {
+        .unwrap_or((None, None));
+        if exists.unwrap_or(false) {
+            if let (Some(name), Some(true)) = (role_target, target_exists) {
+                return Err(Error::BadRequest(format!(
+                    "Data table reference '{reference}' names both the data table '{reference}' \
+                     and a role on the data table '{name}'. Rename '{reference}' to use either."
+                )));
+            }
             return Ok((reference.to_string(), None));
         }
     }
@@ -2156,6 +2258,10 @@ pub enum DucklakeCatalogResourceType {
     Postgresql,
     Mysql,
     Instance,
+    /// On the external instance cluster ([`crate::external_instance_pg`]). Enterprise Edition.
+    #[serde(rename = "external_instance")]
+    #[strum(serialize = "external_instance")]
+    ExternalInstance,
 }
 
 #[derive(Deserialize, Serialize)]
@@ -2683,7 +2789,16 @@ async fn ducklake_conn_data(
     let ducklake = serde_json::from_value::<Ducklake>(ducklake)?;
 
     let catalog_resource =
-        if ducklake.catalog.resource_type == DucklakeCatalogResourceType::Instance {
+        if ducklake.catalog.resource_type == DucklakeCatalogResourceType::ExternalInstance {
+            let pg_creds = crate::external_instance_pg::external_instance_connection_unchecked(
+                db,
+                &ducklake.catalog.resource_path,
+                false,
+            )
+            .await?;
+            serde_json::to_value(&pg_creds)
+                .map_err(|e| Error::internal_err(format!("Error serializing pg creds: {}", e)))?
+        } else if ducklake.catalog.resource_type == DucklakeCatalogResourceType::Instance {
             let mut pg_creds = PgDatabase::parse_uri(&get_database_url().await?.as_str().await)?;
             pg_creds.dbname = ducklake.catalog.resource_path.clone();
             pg_creds.user = Some("custom_instance_user".to_string());
@@ -3064,6 +3179,14 @@ async fn register_fork_ducklake_namespace(
     {
         return Ok(());
     }
+    let mut tx = db.begin().await?;
+    // A row naming an external database counts as a use of it. Written under the lock a drop takes,
+    // and only while the database is still registered, so a drop cannot slip in between the
+    // settings this attach resolved and the row that protects the database.
+    if let Some(dbname) = catalog.strip_prefix("external_instance:") {
+        crate::external_instance_pg::ensure_external_instance_database_registered(&mut tx, dbname)
+            .await?;
+    }
     sqlx::query!(
         "INSERT INTO fork_ducklake_namespace
            (workspace_id, ducklake_name, metadata_schema, catalog, storage, storage_ref, data_path)
@@ -3078,9 +3201,10 @@ async fn register_fork_ducklake_namespace(
         &storage_ref,
         data_path,
     )
-    .execute(db)
+    .execute(&mut *tx)
     .await
     .map_err(|e| Error::internal_err(format!("registering fork ducklake namespace: {e:#}")))?;
+    tx.commit().await?;
     let mut locations = FORK_DUCKLAKE_REGISTERED
         .get(w_id)
         .filter(|(_, exp)| *exp > now)

backend/windmill-queue/src/jobs.rs

@@ -4650,30 +4650,6 @@ pub fn tag_reads_args(tag: &str) -> bool {
     RE_ARG_TAG.is_match(tag)
 }
 
-/// Whether the tag reads the flow's state (`$flow_expr[results.a.foo]`), which only the flow
-/// runtime can resolve, right before pushing the step. A malformed placeholder counts too, so it
-/// is rejected or dropped instead of queueing the job on its literal text.
-pub fn tag_reads_flow_expr(tag: &str) -> bool {
-    tag.contains("$flow_expr[")
-}
-
-/// Renders the value at the dotted `path` below `root` as a dynamic tag component, shared by
-/// `$args[...]` and `$flow_expr[...]`: its JSON text with surrounding quotes trimmed, and empty
-/// once a segment is missing. Only object keys are followed, never array indexes.
-pub fn render_tag_path(root: Option<&RawValue>, path: &str) -> String {
-    let mut value = root.map(|x| x.get()).unwrap_or_default().to_string();
-    for part in path.split('.').filter(|p| !p.is_empty()) {
-        match serde_json::from_str::<serde_json::Value>(&value) {
-            Ok(obj) => value = obj.get(part).map(|v| v.to_string()).unwrap_or_default(),
-            Err(_) => {
-                value = String::new();
-                break;
-            }
-        }
-    }
-    value.trim_matches('"').to_string()
-}
-
 pub fn interpolate_args(x: String, args: &PushArgs, workspace_id: &str) -> String {
     // Save this value to avoid parsing twice
     let workspaced = x.as_str().replace("$workspace", workspace_id).to_string();
@@ -4681,12 +4657,40 @@ pub fn interpolate_args(x: String, args: &PushArgs, workspace_id: &str) -> Strin
         let mut interpolated = workspaced.clone();
         for cap in RE_ARG_TAG.captures_iter(&workspaced) {
             let arg_name = cap.get(1).unwrap().as_str();
-            let (root, rest) = arg_name.split_once('.').unwrap_or((arg_name, ""));
-            let root_value = args
-                .args
-                .get(root)
-                .or(args.extra.as_ref().and_then(|x| x.get(root)));
-            let arg_value = render_tag_path(root_value.map(|x| &**x), rest);
+            let arg_value = if arg_name.contains('.') {
+                let parts: Vec<&str> = arg_name.split('.').collect();
+                let root = parts[0];
+                let mut value = args
+                    .args
+                    .get(root)
+                    .or(args.extra.as_ref().and_then(|x| x.get(root)))
+                    .map(|x| x.get())
+                    .unwrap_or_default()
+                    .to_string();
+
+                for part in parts.iter().skip(1) {
+                    if let Ok(obj) = serde_json::from_str::<serde_json::Value>(&value) {
+                        value = obj
+                            .get(part)
+                            .and_then(|v| Some(v.to_string()))
+                            .unwrap_or_default()
+                            .as_str()
+                            .to_string();
+                    } else {
+                        value = "".to_string(); // Invalid JSON or missing field
+                        break;
+                    }
+                }
+                value.trim_matches('"').to_string()
+            } else {
+                args.args
+                    .get(arg_name)
+                    .or(args.extra.as_ref().and_then(|x| x.get(arg_name)))
+                    .map(|x| x.get())
+                    .unwrap_or_default()
+                    .trim_matches('"')
+                    .to_string()
+            };
             interpolated =
                 interpolated.replace(format!("$args[{}]", arg_name).as_str(), &arg_value);
         }
@@ -5265,8 +5269,6 @@ pub fn empty_result() -> Box<RawValue> {
 
 lazy_static::lazy_static! {
     pub static ref RE_ARG_TAG: Regex = Regex::new(r#"\$args\[((?:\w+\.)*\w+)\]"#).unwrap();
-    pub static ref RE_FLOW_EXPR_TAG: Regex =
-        Regex::new(r#"\$flow_expr\[((?:\w+\.)*\w+)\]"#).unwrap();
 }
 
 #[cfg(feature = "cloud")]
@@ -6535,10 +6537,7 @@ async fn push_inner<'c, 'd>(
         );
         windmill_common::worker::dedicated_worker_tag(workspace_id, &full_path)
     } else {
-        // The flow runtime resolves a step's `$flow_expr[...]` before pushing it, so one still here
-        // was pushed with no flow state to read (a step test, a dependency job) and would name a
-        // queue no worker serves: the job runs on its default tag instead.
-        if tag == Some("".to_string()) || tag.as_deref().is_some_and(tag_reads_flow_expr) {
+        if tag == Some("".to_string()) {
             tag = None;
         }
 
@@ -7923,43 +7922,3 @@ mod result_metadata_tests {
         assert_eq!(meta.wm_failure.as_deref(), Some("boom"));
     }
 }
-
-#[cfg(test)]
-mod render_tag_path_tests {
-    use super::{interpolate_args, render_tag_path, PushArgs};
-    use serde_json::value::RawValue;
-    use std::collections::HashMap;
-
-    fn render(root: &str, path: &str) -> String {
-        render_tag_path(
-            Some(&RawValue::from_string(root.to_string()).unwrap()),
-            path,
-        )
-    }
-
-    // Existing `$args[...]` tags route on exactly these renderings.
-    #[test]
-    fn renders_like_args_tags() {
-        assert_eq!(render(r#""eu""#, ""), "eu");
-        assert_eq!(render(r#"{"a": {"b": "eu"}}"#, "a.b"), "eu");
-        assert_eq!(render(r#"{"n": 4}"#, "n"), "4");
-        assert_eq!(render("null", ""), "null");
-        assert_eq!(render(r#"{"a": 1}"#, "b.c"), "");
-        assert_eq!(render(r#"{"a": ["eu"]}"#, "a.0"), "");
-        assert_eq!(render_tag_path(None, "a"), "");
-
-        let args = HashMap::from([("cfg".to_string(), raw(r#"{"lang": "eu"}"#))]);
-        let push_args = PushArgs {
-            args: &args,
-            extra: Some(HashMap::from([("e".to_string(), raw(r#""x""#))])),
-        };
-        assert_eq!(
-            interpolate_args("w-$args[cfg.lang]-$args[e]".to_string(), &push_args, "ws"),
-            "w-eu-x"
-        );
-    }
-
-    fn raw(json: &str) -> Box<RawValue> {
-        RawValue::from_string(json.to_string()).unwrap()
-    }
-}

backend/windmill-queue/src/schedule.rs

@@ -166,10 +166,13 @@ pub async fn push_scheduled_job<'c>(
         }
     };
 
-    let next = sched.find_next(&starting_from)?;
+    let next = sched.find_next(&starting_from);
+    // println!("next event ({:?}): {}", tz, next);
+    // println!("next event(UTC): {}", next.with_timezone(&chrono::Utc));
 
     // Scheduled events must be stored in the database in UTC
     let next = next.with_timezone(&chrono::Utc);
+    // panic!("next: {}", next);
     let already_exists: bool = sqlx::query_scalar!(
         // Query plan:
         // - use of the `ix_v2_job_root_by_path` index; hence the `parent_job IS NULL` clause.

backend/windmill-queue/tests/schedule_push.rs

@@ -921,47 +921,6 @@ mod schedule_push {
         Ok(())
     }
 
-    // -----------------------------------------------------------------------
-    // try_schedule_next_job: a cron with no run left disables the schedule
-    // -----------------------------------------------------------------------
-
-    #[sqlx::test(migrations = "../migrations", fixtures("base", "schedule_push"))]
-    async fn test_cron_with_no_run_left_disables_schedule(
-        db: Pool<Postgres>,
-    ) -> anyhow::Result<()> {
-        sqlx::query(
-            "INSERT INTO schedule (workspace_id, path, edited_by, edited_at, schedule, timezone, enabled, script_path, is_flow, email, extra_perms, ws_error_handler_muted, no_flow_overlap, permissioned_as, cron_version)
-             VALUES ('test-workspace', 'f/system/test_schedule', 'test-user', now(), '0 0 9 1 1 * 2020', 'UTC', true, 'f/system/test_script', false, 'test@windmill.dev', '{}', false, false, 'u/test-user', 'v1')"
-        )
-        .execute(&db)
-        .await?;
-
-        let schedule = make_schedule(|s| {
-            s.schedule = "0 0 9 1 1 * 2020".to_string();
-            s.cron_version = Some("v1".to_string());
-        });
-        let job = make_completed_job(&schedule);
-
-        let tx = db.begin().await?;
-        let (tx, err) =
-            try_schedule_next_job(&db, tx, &job, &schedule, &schedule.script_path).await;
-        assert!(err.is_none(), "completion must go through, got: {err:?}");
-        tx.commit().await?;
-
-        assert_eq!(count_queued_jobs(&db).await, 0);
-        let (enabled, error): (bool, Option<String>) = sqlx::query_as(
-            "SELECT enabled, error FROM schedule WHERE workspace_id = 'test-workspace' AND path = 'f/system/test_schedule'",
-        )
-        .fetch_one(&db)
-        .await?;
-        assert!(!enabled, "schedule with no run left must be disabled");
-        assert!(
-            error.as_deref().is_some_and(|e| e.contains("no run left")),
-            "error should say why, got: {error:?}"
-        );
-        Ok(())
-    }
-
     // -----------------------------------------------------------------------
     // try_schedule_next_job: disabled schedule leaves no side effects
     // -----------------------------------------------------------------------

backend/windmill-types/src/flows.rs

@@ -1100,8 +1100,7 @@ pub enum FlowModuleValue {
         omit_output_from_conversation: bool,
         /// When set, the agent brain config (provider/model/system prompt/etc.) and tools are
         /// resolved at runtime from this `ai_agent` resource path (hybrid linking). The module's
-        /// `input_transforms` then only carry the flow-local inputs: user_message,
-        /// user_attachments, enabled_tools and the history inputs memory_id and previous_messages.
+        /// `input_transforms` then only carry the flow-local inputs (user_message/user_attachments).
         #[serde(default, skip_serializing_if = "Option::is_none")]
         agent: Option<String>,
         /// Binds an agent's tools to *this* flow's context, keyed by tool id then input key, without