mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-20 00:02:28 +00:00
Compare commits
30
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7577c55b13 | ||
|
|
51224582c1 | ||
|
|
2661dadba6 | ||
|
|
57211d6c95 | ||
|
|
0dbf5eef74 | ||
|
|
8e780a7923 | ||
|
|
3fdb3047a7 | ||
|
|
5dfb2f30f7 | ||
|
|
4a0d7c0c29 | ||
|
|
d62807e9ed | ||
|
|
696b4ab5f8 | ||
|
|
48f026eb0f | ||
|
|
0e659898f3 | ||
|
|
a296ca016a | ||
|
|
947e44fe49 | ||
|
|
489493dba2 | ||
|
|
c55ff3141c | ||
|
|
5eec203099 | ||
|
|
0b3882f2b2 | ||
|
|
58731b1828 | ||
|
|
5624d350d5 | ||
|
|
53d62dd399 | ||
|
|
e17b76c255 | ||
|
|
cfada61f41 | ||
|
|
0406f133c5 | ||
|
|
aebae79397 | ||
|
|
b62d8b246a | ||
|
|
5bc2f13565 | ||
|
|
0c53990d18 | ||
|
|
67f488c676 |
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "1.814.0"
|
||||
".": "1.813.0"
|
||||
}
|
||||
|
||||
@@ -1,31 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [1.814.0](https://github.com/windmill-labs/windmill/compare/v1.813.0...v1.814.0) (2026-09-17)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **ai-chat:** add list_workers and list_data_metrics global tools ([#11143](https://github.com/windmill-labs/windmill/issues/11143)) ([e954d33](https://github.com/windmill-labs/windmill/commit/e954d33613e4ff5027667eb8f646615d9bbd499d))
|
||||
* **ai-chat:** merge get_job_logs and get_flow_run_details into get_run ([#11172](https://github.com/windmill-labs/windmill/issues/11172)) ([5bb37ca](https://github.com/windmill-labs/windmill/commit/5bb37ca3388666fba72c55534e37f37bb3e9299e))
|
||||
* allow git sync auto-pull, promotion and PRs on Pro licenses ([#11173](https://github.com/windmill-labs/windmill/issues/11173)) ([02e47de](https://github.com/windmill-labs/windmill/commit/02e47de8b4c4f3f54753aabf8c67bc8e71ffb957))
|
||||
* badge chat-input flows on the home list ([#11164](https://github.com/windmill-labs/windmill/issues/11164)) ([3d08197](https://github.com/windmill-labs/windmill/commit/3d0819718221f885b61e73d02b43dcc853c7d02a))
|
||||
* collect flow conversations and agent memory once their last message goes ([#11178](https://github.com/windmill-labs/windmill/issues/11178)) ([23c24a9](https://github.com/windmill-labs/windmill/commit/23c24a9688d4c8c462f53221334d538280f16bca))
|
||||
* flow chat model picker on a shared model-settings component ([#11187](https://github.com/windmill-labs/windmill/issues/11187)) ([189793c](https://github.com/windmill-labs/windmill/commit/189793c2e4db7f1c853695ebcc895c1ec82ed19f))
|
||||
* keep flow inputs and seed the agent when chat mode is enabled ([#11177](https://github.com/windmill-labs/windmill/issues/11177)) ([68f2248](https://github.com/windmill-labs/windmill/commit/68f2248018fc218a090bf939e1eb22ff97d5bc22))
|
||||
* let plan mode search and read connected mcp servers ([#11205](https://github.com/windmill-labs/windmill/issues/11205)) ([5371519](https://github.com/windmill-labs/windmill/commit/5371519f0f5ce7750982dcdb374dca72115902e7))
|
||||
* let test_run_flow name the conversation of a chat-mode test run ([#11198](https://github.com/windmill-labs/windmill/issues/11198)) ([6e1ef93](https://github.com/windmill-labs/windmill/commit/6e1ef93f329cb396ffc3df3304d592e8fa0e0e71))
|
||||
* managed memory with an inherited or custom memory id per step ([#11118](https://github.com/windmill-labs/windmill/issues/11118)) ([c297ed0](https://github.com/windmill-labs/windmill/commit/c297ed0052d998fb8f063faa2a36c6eb03e327be))
|
||||
* render the flow chat through the shared session chat components ([#11175](https://github.com/windmill-labs/windmill/issues/11175)) ([a9ec0ae](https://github.com/windmill-labs/windmill/commit/a9ec0aec3ac0c6b0f7919d0eb2168816923826d7))
|
||||
* show flow step detail inside the graph tab on narrow detail layouts ([#11168](https://github.com/windmill-labs/windmill/issues/11168)) ([64dffe6](https://github.com/windmill-labs/windmill/commit/64dffe6106ad6a55b61a423c855a4b5b0cef533e))
|
||||
* store mcp tool call, result and reasoning on flow conversation rows ([#11176](https://github.com/windmill-labs/windmill/issues/11176)) ([a571117](https://github.com/windmill-labs/windmill/commit/a571117f3fd2cef14c920770645c60ee358fdfdd))
|
||||
* tell test flow conversations from deployed ones and rename a chat ([#11179](https://github.com/windmill-labs/windmill/issues/11179)) ([4eab995](https://github.com/windmill-labs/windmill/commit/4eab995cf7cf091a5e4640da4cb77e0921bb7fdf))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* disable a schedule whose cron has no run left instead of panicking ([#11195](https://github.com/windmill-labs/windmill/issues/11195)) ([381d447](https://github.com/windmill-labs/windmill/commit/381d4470ef699ea82283742132e56556b95d2bd2))
|
||||
* skip expiry notifications for app embed and SDK tokens ([#11169](https://github.com/windmill-labs/windmill/issues/11169)) ([9d348f8](https://github.com/windmill-labs/windmill/commit/9d348f84c7830f36b6153472556fd70e3d84cd24))
|
||||
|
||||
## [1.813.0](https://github.com/windmill-labs/windmill/compare/v1.812.0...v1.813.0) (2026-09-16)
|
||||
|
||||
|
||||
|
||||
@@ -175,10 +175,6 @@ the decrypted value, exactly as against a real backend. The chat's read path pas
|
||||
Seed a recognizable secret (the existing fixture uses `sk_live_do_not_leak_me`) and
|
||||
assert it via `valueExcludes` to catch a leak.
|
||||
|
||||
`toolExpect.toolCallArgs` entries support `sharedByAtLeast: <n>`: at least `n` recorded
|
||||
calls to that tool must carry the same non-blank string in the field. Use it for calls that
|
||||
have to share an identifier, like two test runs of one chat conversation.
|
||||
|
||||
`toolExpect.toolCallArgs` entries additionally support `fieldMustBeAbsent: true`: no
|
||||
recorded call to that tool may pass the field at all (an explicit `null` counts as
|
||||
passing it). Use it for partial-update tools, where supplying a field the model could
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import type { BackendValidationSettings } from '../../core/backendValidation'
|
||||
import { buildWorkspaceId } from './workspaceId'
|
||||
|
||||
interface CompletedJobResultMaybe {
|
||||
completed: boolean
|
||||
@@ -24,6 +24,7 @@ export interface CompletedPreviewJob {
|
||||
const tokenCache = new Map<string, Promise<string>>()
|
||||
const sharedWorkspaceQueue = new Map<string, Promise<void>>()
|
||||
const managedSharedWorkspacePrefixes = ['f/evals/']
|
||||
const DEFAULT_WORKSPACE_PREFIX = 'ai-evals'
|
||||
|
||||
export class BackendPreviewClient {
|
||||
constructor(private readonly settings: BackendValidationSettings) {}
|
||||
@@ -440,6 +441,16 @@ async function withSharedWorkspaceLock<T>(workspaceId: string, body: () => Promi
|
||||
}
|
||||
}
|
||||
|
||||
function buildWorkspaceId(caseId: string, attempt: number): string {
|
||||
const caseSlug = caseId
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9-]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
.slice(0, 30)
|
||||
const suffix = randomUUID().slice(0, 8)
|
||||
return `${DEFAULT_WORKSPACE_PREFIX}-${caseSlug || 'case'}-a${attempt}-${suffix}`
|
||||
}
|
||||
|
||||
function extractFolderName(path: string): string | null {
|
||||
if (!path.startsWith('f/')) {
|
||||
return null
|
||||
|
||||
@@ -11,7 +11,6 @@ import type {
|
||||
Script
|
||||
} from '../../../frontend/src/lib/gen'
|
||||
import type {
|
||||
DataMetric,
|
||||
DataTableTables,
|
||||
DataTableTableSchema,
|
||||
EndpointTool,
|
||||
@@ -113,9 +112,6 @@ export interface BenchmarkWorkspaceRunnables {
|
||||
aiProviders?: BenchmarkWorkspaceAiProvider[]
|
||||
resources?: BenchmarkWorkspaceResource[]
|
||||
datatables?: BenchmarkDatatableSeed[]
|
||||
/** DuckLake catalog names, as `list_ducklakes` reports them. */
|
||||
ducklakes?: string[]
|
||||
dataMetrics?: DataMetric[]
|
||||
jobs?: BenchmarkWorkspaceJob[]
|
||||
}
|
||||
|
||||
@@ -677,27 +673,6 @@ export function listBenchmarkDatatables(workspace: string): DataTableTables[] |
|
||||
}))
|
||||
}
|
||||
|
||||
// ============= DuckLake catalogs and declared metrics =============
|
||||
|
||||
/** Seeded DuckLake names, or `null` for a non-benchmark workspace. */
|
||||
export function listBenchmarkDucklakes(workspace: string): string[] | null {
|
||||
const runnables = benchmarkWorkspaceRunnables.get(workspace)
|
||||
return runnables ? (runnables.ducklakes ?? []) : null
|
||||
}
|
||||
|
||||
/**
|
||||
* Seeded metric declarations, or `null` for a non-benchmark workspace.
|
||||
*
|
||||
* The `table` / `path_prefix` filters are ignored: which rows a filter selects is
|
||||
* `canonical_table_path`'s business and is pinned by `ducklakeTools.test.ts`.
|
||||
* Re-deriving it here would give the eval its own copy of that spec to drift from,
|
||||
* and the case this serves measures whether the model reaches for the tool at all.
|
||||
*/
|
||||
export function listBenchmarkDataMetrics(workspace: string): DataMetric[] | null {
|
||||
const runnables = benchmarkWorkspaceRunnables.get(workspace)
|
||||
return runnables ? (runnables.dataMetrics ?? []) : null
|
||||
}
|
||||
|
||||
export function getBenchmarkDatatableSchema(input: {
|
||||
workspace: string
|
||||
datatableName: string
|
||||
@@ -865,29 +840,6 @@ export function runBenchmarkFlowByPath(input: {
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Mirror `JobService.runFlowPreview` for benchmark workspaces, including the server's
|
||||
* refusal of a chat-enabled flow run that names no conversation (`memory_id`).
|
||||
*/
|
||||
export function runBenchmarkFlowPreview(input: {
|
||||
workspace: string
|
||||
memoryId?: string
|
||||
requestBody?: { path?: string; value?: { chat_input_enabled?: boolean }; args?: unknown }
|
||||
}): string {
|
||||
if (input.requestBody?.value?.chat_input_enabled && !input.memoryId) {
|
||||
throw new Error('Bad request: memory_id is required for chat-enabled flows')
|
||||
}
|
||||
const args = (input.requestBody?.args ?? {}) as Record<string, unknown>
|
||||
return createBenchmarkCompletedJob({
|
||||
workspace: input.workspace,
|
||||
jobKind: 'flowpreview',
|
||||
success: true,
|
||||
args,
|
||||
result: { path: input.requestBody?.path, args, mocked: true },
|
||||
logs: 'Mock benchmark flow preview completed successfully.'
|
||||
})
|
||||
}
|
||||
|
||||
export function previewBenchmarkSchedule(input: {
|
||||
requestBody?: Record<string, unknown>
|
||||
}): Record<string, unknown> {
|
||||
|
||||
@@ -76,9 +76,7 @@ vi.mock('$lib/gen', async () => {
|
||||
listBenchmarkPlainResources,
|
||||
listBenchmarkApps,
|
||||
listBenchmarkDatatables,
|
||||
listBenchmarkDataMetrics,
|
||||
listBenchmarkDrafts,
|
||||
listBenchmarkDucklakes,
|
||||
listBenchmarkFlows,
|
||||
listBenchmarkJobs,
|
||||
listBenchmarkScripts,
|
||||
@@ -89,7 +87,6 @@ vi.mock('$lib/gen', async () => {
|
||||
previewBenchmarkSchedule,
|
||||
runBenchmarkDatatableSql,
|
||||
runBenchmarkFlowByPath,
|
||||
runBenchmarkFlowPreview,
|
||||
runBenchmarkScriptByPath,
|
||||
runBenchmarkScriptPreview,
|
||||
updateBenchmarkDraft,
|
||||
@@ -296,14 +293,6 @@ vi.mock('$lib/gen', async () => {
|
||||
args: data.requestBody
|
||||
})
|
||||
: actual.JobService.runScriptByPath(data),
|
||||
runFlowPreview: async (data: {
|
||||
workspace: string
|
||||
memoryId?: string
|
||||
requestBody?: { path?: string; value?: { chat_input_enabled?: boolean }; args?: unknown }
|
||||
}) =>
|
||||
hasBenchmarkWorkspace(data.workspace)
|
||||
? runBenchmarkFlowPreview(data)
|
||||
: actual.JobService.runFlowPreview(data as any),
|
||||
runFlowByPath: async (data: {
|
||||
workspace: string
|
||||
path: string
|
||||
@@ -352,10 +341,6 @@ vi.mock('$lib/gen', async () => {
|
||||
hasBenchmarkWorkspace(data.workspace)
|
||||
? (listBenchmarkDatatables(data.workspace) ?? [])
|
||||
: actual.WorkspaceService.listDataTableTables(data),
|
||||
listDucklakes: async (data: { workspace: string }) =>
|
||||
hasBenchmarkWorkspace(data.workspace)
|
||||
? (listBenchmarkDucklakes(data.workspace) ?? [])
|
||||
: actual.WorkspaceService.listDucklakes(data),
|
||||
getDataTableTableSchema: async (data: {
|
||||
workspace: string
|
||||
datatableName: string
|
||||
@@ -371,12 +356,6 @@ vi.mock('$lib/gen', async () => {
|
||||
})
|
||||
: actual.WorkspaceService.getDataTableTableSchema(data)
|
||||
}),
|
||||
DataMetricService: wrapService(actual.DataMetricService, {
|
||||
listDataMetrics: async (data: { workspace: string }) =>
|
||||
hasBenchmarkWorkspace(data.workspace)
|
||||
? { metrics: listBenchmarkDataMetrics(data.workspace) ?? [] }
|
||||
: actual.DataMetricService.listDataMetrics(data)
|
||||
}),
|
||||
ScheduleService: wrapService(actual.ScheduleService, {
|
||||
existsSchedule: async (data: { workspace: string; path: string }) =>
|
||||
hasBenchmarkWorkspace(data.workspace) ? false : actual.ScheduleService.existsSchedule(data),
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type { WindmillBackendSettings } from "../../core/windmillBackendSettings";
|
||||
import { buildWorkspaceId } from "./workspaceId";
|
||||
|
||||
const tokenCache = new Map<string, Promise<string>>();
|
||||
const sharedWorkspaceQueue = new Map<string, Promise<void>>();
|
||||
const DEFAULT_WORKSPACE_PREFIX = "ai-evals";
|
||||
|
||||
export class WindmillBackendClient {
|
||||
constructor(private readonly settings: WindmillBackendSettings) {}
|
||||
@@ -178,6 +179,16 @@ async function withSharedWorkspaceLock<T>(
|
||||
}
|
||||
}
|
||||
|
||||
function buildWorkspaceId(caseId: string, attempt: number): string {
|
||||
const caseSlug = caseId
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9-]+/g, "-")
|
||||
.replace(/^-+|-+$/g, "")
|
||||
.slice(0, 30);
|
||||
const suffix = randomUUID().slice(0, 8);
|
||||
return `${DEFAULT_WORKSPACE_PREFIX}-${caseSlug || "case"}-a${attempt}-${suffix}`;
|
||||
}
|
||||
|
||||
async function expectOk(response: Response, context: string): Promise<void> {
|
||||
if (response.ok) {
|
||||
return;
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
import { describe, expect, it } from "bun:test";
|
||||
import { buildWorkspaceId } from "./workspaceId";
|
||||
|
||||
describe("buildWorkspaceId", () => {
|
||||
// `workspace.proper_id` rejects `--`, which a case id can carry itself and
|
||||
// which truncating a slug on a hyphen produces once the suffix adds its own.
|
||||
// One id per shape: cut landing on a hyphen, cut landing mid-word, no cut, and
|
||||
// a doubled hyphen no cut ever reaches.
|
||||
it("stays within the id length cap and the proper_id format", () => {
|
||||
for (const caseId of [
|
||||
"global-test6-secret-variable-draft",
|
||||
"global-test23-datatable-query-select",
|
||||
"short",
|
||||
"global--test-foo",
|
||||
]) {
|
||||
const id = buildWorkspaceId(caseId, 1);
|
||||
expect(id.length).toBeLessThanOrEqual(50);
|
||||
expect(id).toMatch(/^\w+(-\w+)*$/);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,22 +0,0 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
|
||||
const DEFAULT_WORKSPACE_PREFIX = "ai-evals";
|
||||
|
||||
// A workspace id must be at most 50 characters AND match `^\w+(-\w+)*$`
|
||||
// (`workspace.proper_id`), so the case slug yields to the random suffix that
|
||||
// makes the id unique, and no hyphen may end up doubled — neither one already in
|
||||
// the case id nor one a truncation leaves for the suffix to follow.
|
||||
const MAX_WORKSPACE_ID_LENGTH = 50;
|
||||
|
||||
export function buildWorkspaceId(caseId: string, attempt: number): string {
|
||||
const caseSlug = caseId
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9-]+/g, "-")
|
||||
.replace(/-{2,}/g, "-")
|
||||
.replace(/^-+|-+$/g, "");
|
||||
const suffix = `-a${attempt}-${randomUUID().slice(0, 8)}`;
|
||||
const head = `${DEFAULT_WORKSPACE_PREFIX}-${caseSlug || "case"}`;
|
||||
return `${head
|
||||
.slice(0, MAX_WORKSPACE_ID_LENGTH - suffix.length)
|
||||
.replace(/-+$/, "")}${suffix}`;
|
||||
}
|
||||
+12
-58
@@ -1919,11 +1919,10 @@
|
||||
- when the lookup fails, tells the user instead of inventing table names
|
||||
- does not write scripts or resources to answer a read-only question
|
||||
|
||||
# --- Dedicated tools preferred over the API catalog ---
|
||||
# The harness serves worker/queue reads itself (benchmark fetch handlers in
|
||||
# adapters/frontend), so these cases do not require an mcp-enabled eval backend.
|
||||
# The stale `api-catalog` in the id below is kept so results stay comparable
|
||||
# across benchmark runs.
|
||||
# --- API catalog (search_api_endpoints / call_api_get / call_api_endpoint) ---
|
||||
# The harness serves the catalog and the executed calls itself (mock
|
||||
# listMcpTools + benchmark fetch handlers in adapters/frontend), so these cases
|
||||
# do not require an mcp-enabled eval backend.
|
||||
|
||||
- id: global-test30-api-catalog-workers
|
||||
prompt: |-
|
||||
@@ -1935,42 +1934,23 @@
|
||||
draftCountExactly: 0
|
||||
toolExpect:
|
||||
requiredToolsUsed:
|
||||
- list_workers
|
||||
forbiddenToolsUsed:
|
||||
- search_api_endpoints
|
||||
- call_api_get
|
||||
forbiddenToolsUsed:
|
||||
- call_api_endpoint
|
||||
- write_script
|
||||
- deploy_workspace_item
|
||||
toolCallArgs:
|
||||
- tool: call_api_get
|
||||
field: name
|
||||
stringIncludesAnyOf:
|
||||
- listWorkers
|
||||
# Read-only workspace inspection produces no draft; validate via tool use.
|
||||
skipJudge: true
|
||||
judgeChecklist:
|
||||
- reads worker state through list_workers instead of guessing or fabricating
|
||||
- discovers the workers endpoint through the API catalog instead of guessing or fabricating
|
||||
- reports worker status from the returned data
|
||||
|
||||
- id: global-test37-ducklake-declared-measure
|
||||
prompt: |-
|
||||
We track orders in the main ducklake. Write me a duckdb script that reports total
|
||||
revenue by month. Keep it as a draft, don't deploy it.
|
||||
initial: ai_evals/fixtures/frontend/global/initial/ducklake_orders_metrics.json
|
||||
runtime:
|
||||
maxTurns: 8
|
||||
validate:
|
||||
draftCountExactly: 1
|
||||
toolExpect:
|
||||
requiredToolsUsed:
|
||||
- list_data_metrics
|
||||
forbiddenToolsUsed:
|
||||
- deploy_workspace_item
|
||||
- delete_workspace_item
|
||||
# The judge runs: the point is not that the tool was called but that the number it
|
||||
# describes is the declared one. `revenue` excludes test rows, so an aggregate that
|
||||
# reproduces it without the filter is plausible, runnable and wrong.
|
||||
judgeChecklist:
|
||||
- totals revenue with the declared sum over the amount column rather than an invented aggregate over a guessed column
|
||||
- excludes test orders from the total, as the declared revenue measure does
|
||||
- groups by month using the declared order_month expression over order_date
|
||||
- does not introduce column names absent from the declarations
|
||||
|
||||
- id: global-test31-draft-test-run-not-deployed
|
||||
prompt: |-
|
||||
Update `f/evals/global/format_greeting` so the provided name is uppercased in the greeting, then run it with name "ada" to check it works.
|
||||
@@ -2160,32 +2140,6 @@
|
||||
- creates an AI draft of f/evals/global/process_invoice applying 8% tax
|
||||
- does not deploy or save the draft
|
||||
|
||||
- id: global-test38-chat-flow-follow-up-same-conversation
|
||||
prompt: |-
|
||||
I want to check that my support chat flow `f/evals/global/support_chat` remembers what was said.
|
||||
Test it: first send "My name is Ada", then send "What is my name?" as a follow-up in the same chat.
|
||||
initial: ai_evals/fixtures/frontend/global/initial/support_chat_flow.json
|
||||
runtime:
|
||||
maxTurns: 8
|
||||
validate:
|
||||
draftCountExactly: 0
|
||||
toolExpect:
|
||||
requiredToolsUsed:
|
||||
- test_run_flow
|
||||
# A chat flow's memory lives in its conversation, so a follow-up only reaches the first
|
||||
# turn's history when both test runs name the same conversation.
|
||||
toolCallArgs:
|
||||
- tool: test_run_flow
|
||||
field: memory_id
|
||||
sharedByAtLeast: 2
|
||||
forbiddenToolsUsed:
|
||||
- run_flow
|
||||
- deploy_workspace_item
|
||||
# The judge cannot observe runs; what this case guards is the conversation the runs share.
|
||||
skipJudge: true
|
||||
judgeChecklist:
|
||||
- test-runs the chat flow twice, the second message as a follow-up in the first run's conversation
|
||||
|
||||
- id: global-undo-created-draft
|
||||
prompt: |-
|
||||
Create a draft Postgres resource at `u/admin/scratch_db` for host db.example.com port 5432, database `orders`, user `app`, and tell me what fields it ended up with.
|
||||
|
||||
@@ -182,13 +182,6 @@ export interface ToolCallArgumentRule {
|
||||
* the point is that the model filled it in at all rather than what it said.
|
||||
*/
|
||||
nonEmpty?: boolean;
|
||||
/**
|
||||
* Existential over calls: at least this many recorded calls to `tool` carry the
|
||||
* same non-blank string in `field`. Use when calls have to share an identifier —
|
||||
* e.g. test runs that continue one conversation — while a retry with a rejected
|
||||
* value in between is still acceptable.
|
||||
*/
|
||||
sharedByAtLeast?: number;
|
||||
/**
|
||||
* Universal over calls: no recorded call to `tool` may pass `field` at all.
|
||||
* For partial-update tools, where supplying a field the model could not have
|
||||
|
||||
@@ -396,32 +396,6 @@ describe("validateToolExpectations", () => {
|
||||
expect(nonEmptyCheck?.details).toContain("blank on 1 of 2");
|
||||
});
|
||||
|
||||
it("requires sharedByAtLeast calls to carry one value, not merely a value each", () => {
|
||||
const run = (ids: (string | undefined)[]) =>
|
||||
validateToolExpectations({
|
||||
run: {
|
||||
success: true,
|
||||
actual: {},
|
||||
assistantMessageCount: 1,
|
||||
toolCallCount: ids.length,
|
||||
toolsUsed: ["test_run_flow"],
|
||||
toolCallDetails: ids.map((memory_id) => ({
|
||||
name: "test_run_flow",
|
||||
arguments: { path: "f/chat", memory_id },
|
||||
})),
|
||||
skillsInvoked: [],
|
||||
},
|
||||
toolExpect: {
|
||||
toolCallArgs: [{ tool: "test_run_flow", field: "memory_id", sharedByAtLeast: 2 }],
|
||||
},
|
||||
}).find((c) => c.name.includes("is shared by at least 2 calls"))?.passed;
|
||||
|
||||
expect(run(["a", "b"])).toBe(false);
|
||||
expect(run(["a"])).toBe(false);
|
||||
expect(run([undefined, undefined])).toBe(false);
|
||||
expect(run(["rejected", "a", "a"])).toBe(true);
|
||||
});
|
||||
|
||||
it("passes nonEmpty when every call filled the field", () => {
|
||||
const checks = validateToolExpectations({
|
||||
run: {
|
||||
|
||||
@@ -320,23 +320,6 @@ export function validateToolExpectations(input: {
|
||||
);
|
||||
}
|
||||
|
||||
if (rule.sharedByAtLeast !== undefined) {
|
||||
const counts = new Map<string, number>();
|
||||
for (const value of values) {
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
counts.set(value, (counts.get(value) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
const mostShared = Math.max(0, ...counts.values());
|
||||
checks.push(
|
||||
check(
|
||||
`${rule.tool}.${rule.field} is shared by at least ${rule.sharedByAtLeast} calls`,
|
||||
mostShared >= rule.sharedByAtLeast,
|
||||
`most calls sharing one value: ${mostShared}; values: ${summarizeToolValues(values)}`
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
if (rule.fieldMustBeAbsent) {
|
||||
// Anything other than `undefined` was supplied — an explicit `null` is the
|
||||
// model passing the field, not omitting it.
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
{
|
||||
"workspace": {
|
||||
"ducklakes": ["main"],
|
||||
"dataMetrics": [
|
||||
{
|
||||
"script_path": "f/analytics/orders_pipeline",
|
||||
"table_path": "main/main.orders",
|
||||
"kind": "measure",
|
||||
"name": "revenue",
|
||||
"expr": "sum(amount)",
|
||||
"filter": "not is_test"
|
||||
},
|
||||
{
|
||||
"script_path": "f/analytics/orders_pipeline",
|
||||
"table_path": "main/main.orders",
|
||||
"kind": "measure",
|
||||
"name": "order_count",
|
||||
"expr": "count(*)"
|
||||
},
|
||||
{
|
||||
"script_path": "f/analytics/orders_pipeline",
|
||||
"table_path": "main/main.orders",
|
||||
"kind": "dimension",
|
||||
"name": "order_month",
|
||||
"expr": "date_trunc('month', order_date)"
|
||||
},
|
||||
{
|
||||
"script_path": "f/analytics/orders_pipeline",
|
||||
"table_path": "main/main.orders",
|
||||
"kind": "dimension",
|
||||
"name": "region",
|
||||
"expr": "region"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
{
|
||||
"workspace": {
|
||||
"flows": [
|
||||
{
|
||||
"path": "f/evals/global/support_chat",
|
||||
"summary": "Support chat",
|
||||
"description": "Answers customer questions in a chat, remembering earlier messages.",
|
||||
"schema": {
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"user_message": {
|
||||
"type": "string",
|
||||
"description": "Message from user"
|
||||
}
|
||||
},
|
||||
"required": ["user_message"]
|
||||
},
|
||||
"value": {
|
||||
"chat_input_enabled": true,
|
||||
"modules": [
|
||||
{
|
||||
"id": "assistant",
|
||||
"summary": "Support assistant",
|
||||
"value": {
|
||||
"type": "aiagent",
|
||||
"tools": [],
|
||||
"input_transforms": {
|
||||
"provider": {
|
||||
"type": "static",
|
||||
"value": {
|
||||
"kind": "anthropic",
|
||||
"model": "claude-haiku-4-5-20251001",
|
||||
"resource": "$res:f/evals/ai/anthropic"
|
||||
}
|
||||
},
|
||||
"user_message": {
|
||||
"type": "javascript",
|
||||
"expr": "flow_input.user_message"
|
||||
},
|
||||
"system_prompt": {
|
||||
"type": "static",
|
||||
"value": "You are a friendly support assistant. Keep answers short."
|
||||
},
|
||||
"memory": {
|
||||
"type": "static",
|
||||
"value": { "kind": "auto", "context_length": 10 }
|
||||
},
|
||||
"streaming": { "type": "static", "value": true },
|
||||
"output_type": { "type": "static", "value": "text" }
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
+3
-27
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments\n FROM flow_conversation_message\n WHERE conversation_id = $1\n AND created_seq > $2\n ORDER BY created_seq ASC\n LIMIT $3\n ",
|
||||
"query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success\n FROM (\n SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success\n FROM flow_conversation_message\n WHERE conversation_id = $1\n ORDER BY created_seq DESC\n LIMIT $2 OFFSET $3\n ) AS messages\n ORDER BY created_seq ASC\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -58,26 +58,6 @@
|
||||
"ordinal": 8,
|
||||
"name": "success",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 9,
|
||||
"name": "tool_arguments",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 10,
|
||||
"name": "tool_result",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 11,
|
||||
"name": "reasoning",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 12,
|
||||
"name": "attachments",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -96,12 +76,8 @@
|
||||
false,
|
||||
false,
|
||||
true,
|
||||
false,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "d6fa78c43b6c5f8040d7bccb29ad8627be1dac6fbe0097735a52f47c173f51c9"
|
||||
"hash": "1c3473a0f9f6b6148b2c975f9f05bdefedf8a51c4e6ddf0eca367b9cc778d051"
|
||||
}
|
||||
-24
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE flow_conversation SET title = $1, updated_at = updated_at\n WHERE id = $2 AND workspace_id = $3\n RETURNING id",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "id",
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Uuid",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "5b9c9eb64051f291fed4be9bc0b0cc0aef2e7bde732899976eddac36a2da7658"
|
||||
}
|
||||
+2
-8
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test\n FROM flow_conversation\n WHERE id = $1 AND workspace_id = $2",
|
||||
"query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by\n FROM flow_conversation\n WHERE id = $1 AND workspace_id = $2\n FOR UPDATE",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -37,11 +37,6 @@
|
||||
"ordinal": 6,
|
||||
"name": "created_by",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 7,
|
||||
"name": "is_test",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -57,9 +52,8 @@
|
||||
true,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "48c8522a4fed219c5011f4ba63c81cfe028a8b2a32bd790840cef65c452a8c31"
|
||||
"hash": "6f32c1feed096ff706ae359ad6a3ca33b3f82ca38289dfa4a69aa95041027d57"
|
||||
}
|
||||
-38
@@ -1,38 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT id, name, enabled, pwd FROM datatable_role",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "name",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "enabled",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "pwd",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "71ee2cb6661cca1fa4d8874a7f6d368347c59f36fd87df6dc7996152ccb84af0"
|
||||
}
|
||||
-29
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT ws.workspace_id AS \"workspace_id!\", dt.key AS \"datatable!\"\n FROM workspace_settings ws\n JOIN workspace w ON w.id = ws.workspace_id AND w.deleted = false\n CROSS JOIN LATERAL jsonb_each(COALESCE(ws.datatable->'datatables', '{}'::jsonb)) dt\n WHERE ws.workspace_id <> $1\n AND dt.value->'database'->>'resource_type' = 'instance'\n AND dt.value->'database'->>'resource_path' = $2\n ORDER BY ws.workspace_id, dt.key\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "workspace_id!",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "datatable!",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "79799b5a2e499df6c28e286c42b9ad2db940c2455ab19cc95e5198baf96d5629"
|
||||
}
|
||||
-17
@@ -1,17 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO datatable_role (id, name, enabled, pwd) VALUES ($1, $2, $3, $4)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "86af9d51a158ea5cb6161461ecddf2a63695f8cbf8af648da5a0a77a5b9d02ba"
|
||||
}
|
||||
+3
-7
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success, tool_arguments, tool_result, reasoning, attachments)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)",
|
||||
"query": "INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success)\n VALUES ($1, $2, $3, $4, $5, $6)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -21,14 +21,10 @@
|
||||
"Text",
|
||||
"Uuid",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Text",
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb"
|
||||
"Bool"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "12329c3359a7944ab5fa3aa27ddca1b26f340ccf574b9fa07641fe88b2d2987c"
|
||||
"hash": "b1a9a433e577133869c067b2ce383fc6ce4e9df307feb5fd3edc0d1276d61ff1"
|
||||
}
|
||||
-20
@@ -1,20 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT jsonb_object_keys(value->'databases') FROM global_settings\n WHERE name = 'custom_instance_pg_databases'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "jsonb_object_keys",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "b9842d2d8abf382bd82d8fa1de012373638be391f884f81dc387ffc465badac6"
|
||||
}
|
||||
+3
-10
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title, is_test)\n VALUES ($1, $2, $3, $4, $5, $6)\n ON CONFLICT (id) DO NOTHING\n RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test",
|
||||
"query": "INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (id) DO NOTHING\n RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -37,11 +37,6 @@
|
||||
"ordinal": 6,
|
||||
"name": "created_by",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 7,
|
||||
"name": "is_test",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -50,8 +45,7 @@
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool"
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
@@ -61,9 +55,8 @@
|
||||
true,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "6d259b8cce5da5fecefe4ce322789b6b2cc43b51f2056c677d58f39c31fb26cb"
|
||||
"hash": "c1e3ed3ecc3bcb98f60ba8196d33fee4a74f61b061e5025ecb75882208b3ba8f"
|
||||
}
|
||||
+2
-8
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test\n FROM flow_conversation\n WHERE id = $1 AND workspace_id = $2\n FOR UPDATE",
|
||||
"query": "SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by\n FROM flow_conversation\n WHERE id = $1 AND workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -37,11 +37,6 @@
|
||||
"ordinal": 6,
|
||||
"name": "created_by",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 7,
|
||||
"name": "is_test",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -57,9 +52,8 @@
|
||||
true,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "dd84f9dfb238d18cb74f9e43228345427131bf8008eba6920021d6a449791534"
|
||||
"hash": "c383cc023714b361d10c10e8fef1fc148ab1da942951ee9ffdddaecee76a6be9"
|
||||
}
|
||||
-24
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT dt.key AS \"datatable!\"\n FROM workspace_settings ws\n CROSS JOIN LATERAL jsonb_each(COALESCE(ws.datatable->'datatables', '{}'::jsonb)) dt\n WHERE ws.workspace_id = $1\n AND dt.key <> $2\n AND NOT dt.value ? 'permissions'\n AND dt.value->'database'->>'resource_type' = 'instance'\n AND dt.value->'database'->>'resource_path' = $3\n ORDER BY dt.key",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "datatable!",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "d48ca62c86b1af7a9dd2450c1c28dc45020a2a553d8874c49f9eafedea5a9d40"
|
||||
}
|
||||
+2
-8
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n j.args as \"args: Json<HashMap<String, Box<RawValue>>>\",\n js.flow_status as \"flow_status: Json<windmill_common::flow_status::FlowStatus>\",\n j.runnable_path\n FROM v2_job_status js\n INNER JOIN v2_job j ON j.id = js.id\n WHERE js.id = $1\n ",
|
||||
"query": "\n SELECT\n j.args as \"args: Json<HashMap<String, Box<RawValue>>>\",\n js.flow_status as \"flow_status: Json<windmill_common::flow_status::FlowStatus>\"\n FROM v2_job_status js\n INNER JOIN v2_job j ON j.id = js.id\n WHERE js.id = $1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -12,11 +12,6 @@
|
||||
"ordinal": 1,
|
||||
"name": "flow_status: Json<windmill_common::flow_status::FlowStatus>",
|
||||
"type_info": "Jsonb"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "runnable_path",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -25,10 +20,9 @@
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true,
|
||||
true,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "9008f9abb70a9a07e38acb20bea6a710d0efd77dac4aedeb88d72240e816530b"
|
||||
"hash": "dd89d652154748d6d7e625e31778f6885d0ee62d29a4b8894a4b459dd215a103"
|
||||
}
|
||||
+3
-27
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments\n FROM (\n SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments\n FROM flow_conversation_message\n WHERE conversation_id = $1\n ORDER BY created_seq DESC\n LIMIT $2 OFFSET $3\n ) AS messages\n ORDER BY created_seq ASC\n ",
|
||||
"query": "SELECT id, conversation_id, message_type as \"message_type: MessageType\", content, job_id, created_at, created_seq, step_name, success\n FROM flow_conversation_message\n WHERE conversation_id = $1\n AND created_seq > $2\n ORDER BY created_seq ASC\n LIMIT $3\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -58,26 +58,6 @@
|
||||
"ordinal": 8,
|
||||
"name": "success",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 9,
|
||||
"name": "tool_arguments",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 10,
|
||||
"name": "tool_result",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 11,
|
||||
"name": "reasoning",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 12,
|
||||
"name": "attachments",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -96,12 +76,8 @@
|
||||
false,
|
||||
false,
|
||||
true,
|
||||
false,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
true
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "a4a823f70b3dbe6aaf4a61c98345e94c5042fd5e6351fea139a66ecb1fb812ab"
|
||||
"hash": "e8802be9203c1e88a06e337260ccca029380139f89a01a89033e36a6ed9ac082"
|
||||
}
|
||||
Generated
+125
-126
@@ -728,7 +728,7 @@ dependencies = [
|
||||
"futures-lite 2.6.1",
|
||||
"parking",
|
||||
"polling 3.11.0",
|
||||
"rustix 1.1.5",
|
||||
"rustix 1.1.4",
|
||||
"slab",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
@@ -873,7 +873,7 @@ checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1976,7 +1976,7 @@ dependencies = [
|
||||
"prettyplease 0.3.0",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2000,7 +2000,7 @@ dependencies = [
|
||||
"proc-macro-crate",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2144,7 +2144,7 @@ checksum = "6a1f896587b6f2c069c73d2f0913e2d590c3990285cd2f0b6aa02b786b4c679c"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2164,9 +2164,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "bytes-str"
|
||||
version = "0.2.9"
|
||||
version = "0.2.8"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4dde6d05e75a31ec9610eb6446a6f0a10dd30ff5100d720fee4c7c7a9008b5ba"
|
||||
checksum = "577d2bf5650f8554d5a372af5ac93535110a0fc75b3e702bb853369febf227c2"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"serde",
|
||||
@@ -2338,9 +2338,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "cfg-if"
|
||||
version = "1.0.5"
|
||||
version = "1.0.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
|
||||
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
|
||||
|
||||
[[package]]
|
||||
name = "cfg_aliases"
|
||||
@@ -2454,7 +2454,7 @@ dependencies = [
|
||||
"heck",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3047,7 +3047,7 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"strsim 0.11.1",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3102,7 +3102,7 @@ checksum = "2ac7135c3ef02b2f7833bbeb1be5ba7f966dcde8a87c6b87f65a778d71a02785"
|
||||
dependencies = [
|
||||
"darling_core 0.24.1",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4580,7 +4580,7 @@ checksum = "e01a3366d27ee9890022452ee61b2b63a67e6f13f58900b651ff5665f0bb1fab"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"option-ext",
|
||||
"redox_users 0.5.3",
|
||||
"redox_users 0.5.2",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
@@ -4603,7 +4603,7 @@ checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4850,7 +4850,7 @@ checksum = "a65863d15a4ce2888bd2f0f543cc963d3879c3a022c8ee43f6141d479a3ac815"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -5198,7 +5198,7 @@ version = "0.13.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8640e34b88f7652208ce9e88b1a37a2ae95227d84abec377ccd3c5cfeb141ed4"
|
||||
dependencies = [
|
||||
"rustix 1.1.5",
|
||||
"rustix 1.1.4",
|
||||
"windows-sys 0.59.0",
|
||||
]
|
||||
|
||||
@@ -5319,7 +5319,7 @@ checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -9438,7 +9438,7 @@ dependencies = [
|
||||
"concurrent-queue",
|
||||
"hermit-abi 0.5.3",
|
||||
"pin-project-lite",
|
||||
"rustix 1.1.5",
|
||||
"rustix 1.1.4",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
@@ -9574,7 +9574,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2bfe0f4c752e450fc2faf62654f1c134747922825d5b04ca717b8874f41a40c0"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -10235,10 +10235,11 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "redox_users"
|
||||
version = "0.5.3"
|
||||
version = "0.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "60dc65c0ff1a7ae1294b0c67b9f14baf70b644404010370171787bfac1038fc0"
|
||||
checksum = "a4e608c6638b9c18977b00b475ac1f28d14e84b27d8d42f70e0bf1e3dec127ac"
|
||||
dependencies = [
|
||||
"getrandom 0.2.17",
|
||||
"libredox",
|
||||
"thiserror 2.0.20",
|
||||
]
|
||||
@@ -10260,7 +10261,7 @@ checksum = "92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -10579,7 +10580,7 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"serde_json",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -10797,9 +10798,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "rustix"
|
||||
version = "1.1.5"
|
||||
version = "1.1.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
|
||||
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"errno",
|
||||
@@ -11216,7 +11217,7 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"serde_derive_internals 0.30.0",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -11415,7 +11416,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -11437,7 +11438,7 @@ checksum = "f852137cce035d6a4df67ccce505ff6b3e9fd3a10e3e52b24dc71e650bb1a9bd"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -11491,7 +11492,7 @@ checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -11559,7 +11560,7 @@ dependencies = [
|
||||
"darling 0.24.1",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -12707,9 +12708,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "syn"
|
||||
version = "3.0.6"
|
||||
version = "3.0.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
|
||||
checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -12744,7 +12745,7 @@ checksum = "901704edd0dfe137f1987838ee4f259e4e063c31371bdb423f7ae38ec6f77f02"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -13035,7 +13036,7 @@ dependencies = [
|
||||
"fastrand 2.5.0",
|
||||
"getrandom 0.4.3",
|
||||
"once_cell",
|
||||
"rustix 1.1.5",
|
||||
"rustix 1.1.4",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
@@ -13054,7 +13055,7 @@ version = "0.4.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874"
|
||||
dependencies = [
|
||||
"rustix 1.1.5",
|
||||
"rustix 1.1.4",
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
@@ -13114,7 +13115,7 @@ checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -14074,7 +14075,7 @@ checksum = "f153acc4e99a5f2a5aefa09fb078be54e26271b2813f6041200b224c098d8328"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -14178,9 +14179,9 @@ checksum = "81b79ad29b5e19de4260020f8919b443b2ef0277d242ce532ec7b7a2cc8b6007"
|
||||
|
||||
[[package]]
|
||||
name = "unicode-ident"
|
||||
version = "1.0.26"
|
||||
version = "1.0.24"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
|
||||
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
|
||||
|
||||
[[package]]
|
||||
name = "unicode-normalization"
|
||||
@@ -14545,7 +14546,7 @@ dependencies = [
|
||||
"bumpalo",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
"wasm-bindgen-shared",
|
||||
]
|
||||
|
||||
@@ -14588,7 +14589,7 @@ checksum = "8c89dcab8b516b6b603baca9d550b7282d68fcc7f367e3956cff7ebf406a3f12"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -14793,7 +14794,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-nats",
|
||||
@@ -14881,7 +14882,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-ai"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"async-stream",
|
||||
"async-trait",
|
||||
@@ -14895,7 +14896,6 @@ dependencies = [
|
||||
"eventsource-stream",
|
||||
"futures",
|
||||
"http 1.5.0",
|
||||
"indexmap 2.14.2",
|
||||
"lazy_static",
|
||||
"mime_guess",
|
||||
"reqwest 0.13.5",
|
||||
@@ -14915,7 +14915,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-alerting"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -14928,7 +14928,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"argon2",
|
||||
@@ -15068,7 +15068,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-agent-workers"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15091,7 +15091,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-assets"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15108,7 +15108,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-auth"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum 0.8.9",
|
||||
@@ -15134,7 +15134,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-client"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"reqwest 0.12.28",
|
||||
"serde",
|
||||
@@ -15144,7 +15144,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-configs"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15161,7 +15161,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-debug"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"base64 0.22.1",
|
||||
@@ -15183,7 +15183,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-embeddings"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum 0.8.9",
|
||||
@@ -15206,7 +15206,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-flow-conversations"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15222,7 +15222,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-flows"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15244,7 +15244,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-groups"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15266,7 +15266,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-inputs"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15280,7 +15280,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-integration-tests"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-nats",
|
||||
@@ -15315,7 +15315,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-jobs"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum 0.8.9",
|
||||
@@ -15340,7 +15340,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-npm-proxy"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15368,7 +15368,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-openapi"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum 0.8.9",
|
||||
@@ -15390,7 +15390,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-schedule"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15410,7 +15410,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-scripts"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15448,7 +15448,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-settings"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum 0.8.9",
|
||||
@@ -15477,7 +15477,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-sse"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"lazy_static",
|
||||
"serde",
|
||||
@@ -15489,7 +15489,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-users"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"argon2",
|
||||
"axum 0.8.9",
|
||||
@@ -15513,7 +15513,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-workers"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15527,7 +15527,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-workspaces"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"axum 0.8.9",
|
||||
"chrono",
|
||||
@@ -15562,7 +15562,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-audit"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"lazy_static",
|
||||
@@ -15576,7 +15576,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-autoscaling"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum 0.8.9",
|
||||
@@ -15595,7 +15595,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-common"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"aes-gcm",
|
||||
"aho-corasick",
|
||||
@@ -15667,7 +15667,6 @@ dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
"serde_yml",
|
||||
"sha1",
|
||||
"sha2 0.10.9",
|
||||
"size",
|
||||
"spki",
|
||||
@@ -15703,7 +15702,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-dep-map"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"futures",
|
||||
@@ -15723,7 +15722,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-git-sync"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"regex",
|
||||
"serde",
|
||||
@@ -15740,7 +15739,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-indexer"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"astral-tokio-tar",
|
||||
@@ -15767,7 +15766,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-jseval"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"futures",
|
||||
@@ -15784,7 +15783,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-macros"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"itertools 0.14.0",
|
||||
"lazy_static",
|
||||
@@ -15800,7 +15799,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-mcp"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -15821,7 +15820,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-native-triggers"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -15852,7 +15851,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-oauth"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"arc-swap",
|
||||
@@ -15877,7 +15876,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-object-store"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-stream",
|
||||
@@ -15912,7 +15911,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-operator"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"futures",
|
||||
@@ -15930,7 +15929,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"convert_case 0.6.0",
|
||||
"serde",
|
||||
@@ -15939,7 +15938,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-bash"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -15951,7 +15950,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-csharp"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde_json",
|
||||
@@ -15963,7 +15962,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-go"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"gosyn",
|
||||
@@ -15975,7 +15974,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-graphql"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -15987,7 +15986,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-java"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde_json",
|
||||
@@ -15999,7 +15998,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-nu"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"nu-parser",
|
||||
@@ -16010,7 +16009,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-php"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools 0.14.0",
|
||||
@@ -16021,7 +16020,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools 0.14.0",
|
||||
@@ -16033,7 +16032,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py-asset"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"rustpython-ast",
|
||||
@@ -16044,7 +16043,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py-imports"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-recursion",
|
||||
@@ -16066,7 +16065,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-r"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde_json",
|
||||
@@ -16078,7 +16077,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ruby"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -16092,7 +16091,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-rust"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"convert_case 0.6.0",
|
||||
@@ -16109,7 +16108,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-sql"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -16122,7 +16121,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-sql-asset"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde",
|
||||
@@ -16134,7 +16133,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ts"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -16152,7 +16151,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ts-asset"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde-wasm-bindgen",
|
||||
@@ -16168,7 +16167,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-wac"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"rustpython-ast",
|
||||
@@ -16184,7 +16183,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-yaml"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -16198,7 +16197,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-queue"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-recursion",
|
||||
@@ -16237,7 +16236,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-runtime-nativets"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"const_format",
|
||||
@@ -16277,7 +16276,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-sql-datatype-parser-wasm"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"getrandom 0.3.4",
|
||||
"wasm-bindgen",
|
||||
@@ -16288,7 +16287,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-store"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-recursion",
|
||||
@@ -16323,7 +16322,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-test-utils"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16347,7 +16346,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16380,7 +16379,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-amqp"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16407,7 +16406,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-azure"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16440,7 +16439,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-email"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16460,7 +16459,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-gcp"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16494,7 +16493,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-http"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16530,7 +16529,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-kafka"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16553,7 +16552,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-mqtt"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16577,7 +16576,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-nats"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-nats",
|
||||
@@ -16601,7 +16600,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-postgres"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16636,7 +16635,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-sqs"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16664,7 +16663,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-trigger-websocket"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -16689,7 +16688,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-types"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"bitflags 2.13.2",
|
||||
@@ -16708,7 +16707,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-worker"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-once-cell",
|
||||
@@ -16826,7 +16825,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-worker-volumes"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures",
|
||||
@@ -17459,7 +17458,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
|
||||
dependencies = [
|
||||
"libc",
|
||||
"rustix 1.1.5",
|
||||
"rustix 1.1.4",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -17520,7 +17519,7 @@ checksum = "33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
"synstructure 0.14.0",
|
||||
]
|
||||
|
||||
@@ -17561,7 +17560,7 @@ checksum = "f75b4683f6c7f45248d4d64056a24298c6281e0993356d7d1b4a1a962ef10d4a"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
"synstructure 0.14.0",
|
||||
]
|
||||
|
||||
@@ -17617,7 +17616,7 @@ checksum = "34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "windmill"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
authors.workspace = true
|
||||
edition.workspace = true
|
||||
|
||||
@@ -88,7 +88,7 @@ members = [
|
||||
exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
|
||||
edition = "2021"
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
220301c27272b6c9e38401884c45cd959bd02a09
|
||||
151be033ea2bf2f768f381a8cd3b888bdd366cf9
|
||||
@@ -1,3 +0,0 @@
|
||||
ALTER TABLE flow_conversation_message DROP COLUMN tool_arguments;
|
||||
ALTER TABLE flow_conversation_message DROP COLUMN tool_result;
|
||||
ALTER TABLE flow_conversation_message DROP COLUMN reasoning;
|
||||
@@ -1,12 +0,0 @@
|
||||
-- A chat is rebuilt from its rows without reading jobs, so every tool row carries its call:
|
||||
-- the arguments the model wrote and the text the model got back, or what the call failed
|
||||
-- with. A script or flow tool's job holds the args its input transforms produced, not the
|
||||
-- model's; an MCP tool runs inside the agent's job, whose result lists every call of the
|
||||
-- turn with nothing tying one to a row. A provider-native web search carries only its
|
||||
-- citations, the provider never returning the query.
|
||||
ALTER TABLE flow_conversation_message ADD COLUMN tool_arguments TEXT;
|
||||
ALTER TABLE flow_conversation_message ADD COLUMN tool_result TEXT;
|
||||
|
||||
-- The thinking behind this row. The agent job keeps the turn's thinking as one string;
|
||||
-- the rows keep it per iteration, next to the answer or tool call it led to.
|
||||
ALTER TABLE flow_conversation_message ADD COLUMN reasoning TEXT;
|
||||
@@ -1 +0,0 @@
|
||||
ALTER TABLE flow_conversation DROP COLUMN is_test;
|
||||
@@ -1,26 +0,0 @@
|
||||
-- A chat run from the flow editor's test panel is stored exactly like one from the
|
||||
-- deployed flow, so the two were indistinguishable once written. Marking them lets the
|
||||
-- lists tell a trial apart from a real conversation.
|
||||
ALTER TABLE flow_conversation ADD COLUMN is_test BOOLEAN NOT NULL DEFAULT false;
|
||||
|
||||
-- Existing rows: a conversation whose messages came from a flowpreview run was a test.
|
||||
-- Derived once here because the job is purged on retention, after which the origin of an
|
||||
-- old conversation is unknowable.
|
||||
--
|
||||
-- Walked to the root job rather than matched directly: an existing message row never holds
|
||||
-- the flow job itself. The rows point at the step that produced them — the AI agent's job
|
||||
-- for an answer, the tool's own job for a tool call — whose kind is never 'flowpreview'.
|
||||
--
|
||||
-- `root_job` first, matching `get_root_job_id` (windmill-worker/src/common.rs): only it
|
||||
-- reaches the top of the run. `flow_innermost_root_job` stops at the closest flow scope by
|
||||
-- design, so an agent inside a subflow would land on that subflow's 'flow' row and the
|
||||
-- conversation would read as deployed.
|
||||
UPDATE flow_conversation c
|
||||
SET is_test = true
|
||||
WHERE EXISTS (
|
||||
SELECT 1 FROM flow_conversation_message m
|
||||
JOIN v2_job j ON j.id = m.job_id
|
||||
JOIN v2_job root
|
||||
ON root.id = coalesce(j.root_job, j.flow_innermost_root_job, j.parent_job, j.id)
|
||||
WHERE m.conversation_id = c.id AND root.kind = 'flowpreview'
|
||||
);
|
||||
@@ -1 +0,0 @@
|
||||
ALTER TABLE flow_conversation_message DROP COLUMN attachments;
|
||||
@@ -1,4 +0,0 @@
|
||||
-- The files a user message carried, as object-storage references: `[{input, s3, storage?,
|
||||
-- filename?}]`. Only references, never file bytes and never a presigned URL, so a
|
||||
-- transcript can show a message's files without reading its run's args.
|
||||
ALTER TABLE flow_conversation_message ADD COLUMN attachments JSONB;
|
||||
@@ -0,0 +1,22 @@
|
||||
-- Roles on the external cluster are live logins there; dropping the column would forget them.
|
||||
LOCK TABLE datatable_role;
|
||||
DO $$
|
||||
BEGIN
|
||||
IF EXISTS (SELECT 1 FROM datatable_role WHERE cluster <> 'instance') THEN
|
||||
RAISE EXCEPTION 'datatable_role holds roles on the external instance cluster. Delete them in instance settings first.';
|
||||
END IF;
|
||||
-- Before this, only data tables on Windmill's own cluster could be under roles, and a role
|
||||
-- block left with just `admin` survives deleting every external role.
|
||||
IF EXISTS (
|
||||
SELECT 1 FROM workspace_settings ws,
|
||||
jsonb_each(CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
|
||||
THEN ws.datatable->'datatables' ELSE '{}'::jsonb END) dt
|
||||
WHERE dt.value->'database'->>'resource_type' = 'external_instance'
|
||||
AND dt.value ? 'permissions'
|
||||
) THEN
|
||||
RAISE EXCEPTION 'external instance data tables are still under roles. Turn their roles off first.';
|
||||
END IF;
|
||||
END $$;
|
||||
ALTER TABLE datatable_role DROP CONSTRAINT datatable_role_cluster_name_key;
|
||||
ALTER TABLE datatable_role ADD CONSTRAINT datatable_role_name_key UNIQUE (name);
|
||||
ALTER TABLE datatable_role DROP COLUMN cluster;
|
||||
@@ -0,0 +1,8 @@
|
||||
-- A data table role is a Postgres login on one cluster: Windmill's own ('instance'), or the external
|
||||
-- instance cluster ('external_instance'). Role names are the cluster's own key, so they are unique
|
||||
-- per cluster rather than across the instance.
|
||||
ALTER TABLE datatable_role
|
||||
ADD COLUMN cluster VARCHAR(20) NOT NULL DEFAULT 'instance'
|
||||
CHECK (cluster IN ('instance', 'external_instance'));
|
||||
ALTER TABLE datatable_role DROP CONSTRAINT datatable_role_name_key;
|
||||
ALTER TABLE datatable_role ADD CONSTRAINT datatable_role_cluster_name_key UNIQUE (cluster, name);
|
||||
+24
-24
@@ -6191,7 +6191,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
|
||||
|
||||
[[package]]
|
||||
name = "windmill-common"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"aho-corasick",
|
||||
"anyhow",
|
||||
@@ -6274,7 +6274,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-macros"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -6286,7 +6286,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"convert_case",
|
||||
"serde",
|
||||
@@ -6295,7 +6295,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-bash"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -6307,7 +6307,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-csharp"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde_json",
|
||||
@@ -6319,7 +6319,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-go"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"gosyn",
|
||||
@@ -6331,7 +6331,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-graphql"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -6343,7 +6343,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-java"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde_json",
|
||||
@@ -6355,7 +6355,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-nu"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"nu-parser",
|
||||
@@ -6366,7 +6366,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-php"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools 0.14.0",
|
||||
@@ -6377,7 +6377,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools 0.14.0",
|
||||
@@ -6389,7 +6389,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py-asset"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"rustpython-ast",
|
||||
@@ -6400,7 +6400,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py-imports"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-recursion",
|
||||
@@ -6422,7 +6422,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-r"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde_json",
|
||||
@@ -6434,7 +6434,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ruby"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -6448,7 +6448,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-rust"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"convert_case",
|
||||
@@ -6465,7 +6465,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-sql"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -6478,7 +6478,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-sql-asset"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde",
|
||||
@@ -6490,7 +6490,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ts"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -6508,7 +6508,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ts-asset"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"serde-wasm-bindgen",
|
||||
@@ -6524,7 +6524,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-wac"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"rustpython-ast",
|
||||
@@ -6540,7 +6540,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-wasm"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"getrandom 0.2.17",
|
||||
@@ -6572,7 +6572,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-yaml"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"lazy_static",
|
||||
@@ -6586,7 +6586,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-types"
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"bitflags",
|
||||
|
||||
@@ -12,7 +12,7 @@ resolver = "2"
|
||||
members = ["."]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.814.0"
|
||||
version = "1.813.0"
|
||||
edition = "2021"
|
||||
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
|
||||
|
||||
|
||||
@@ -99,9 +99,9 @@ email_trigger: path(char), local_part(char), workspaced_local_part(bool), script
|
||||
favorite: usr(char), workspace_id(char), path(char), favorite_kind(favorite_kind)
|
||||
flow: workspace_id(char), path(char), summary(text), description(text), value(jsonb), edited_by(char), edited_at(ts), archived(bool), schema(json), extra_perms(jsonb), dependency_job(uuid), draft_only(bool), tag(char), ws_error_handler_muted(bool), dedicated_worker(bool), timeout(int), visible_to_runner_only(bool), concurrency_key(char), versions(bigint[]), on_behalf_of(varchar), on_behalf_of_email(text), lock_error_logs(text), labels(text[])
|
||||
FK: (workspace_id) -> workspace(id)
|
||||
flow_conversation: id(uuid), workspace_id(char), flow_path(char), title(char), created_at(ts), updated_at(ts), created_by(char), is_test(bool)
|
||||
flow_conversation: id(uuid), workspace_id(char), flow_path(char), title(char), created_at(ts), updated_at(ts), created_by(char)
|
||||
FK: (workspace_id) -> workspace(id)
|
||||
flow_conversation_message: id(uuid), conversation_id(uuid), message_type(message_type), content(text), job_id(uuid), created_at(ts), created_seq(int8), step_name(char), success(bool), tool_arguments(text), tool_result(text), reasoning(text), attachments(jsonb)
|
||||
flow_conversation_message: id(uuid), conversation_id(uuid), message_type(message_type), content(text), job_id(uuid), created_at(ts), created_seq(int8), step_name(char), success(bool)
|
||||
FK: (conversation_id) -> flow_conversation(id) | (job_id) -> v2_job(id)
|
||||
flow_iterator_data: job_id(uuid), itered(jsonb)
|
||||
flow_node: id(bigint), workspace_id(char), hash(bigint), path(char), lock(text), code(text), flow(jsonb), hash_v2(char(64))
|
||||
|
||||
@@ -1199,63 +1199,3 @@ async fn test_wm_labels_from_result_merged_with_static_labels(
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `tag` lives only on `v2_job`, which count_jobs joins only when `tags` is set.
|
||||
#[sqlx::test(fixtures("base"))]
|
||||
async fn test_count_completed_jobs_tags_filter(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
let client = windmill_api_client::create_client(
|
||||
&format!("http://localhost:{port}"),
|
||||
"SECRET_TOKEN".to_string(),
|
||||
);
|
||||
|
||||
for (ws, tag, status) in [
|
||||
("test-workspace", "deno", "success"),
|
||||
("test-workspace", "deno", "failure"),
|
||||
("test-workspace", "python3", "success"),
|
||||
("other-workspace", "deno", "success"),
|
||||
] {
|
||||
let id = uuid::Uuid::new_v4();
|
||||
sqlx::query("INSERT INTO v2_job (id, workspace_id, tag) VALUES ($1, $2, $3)")
|
||||
.bind(id)
|
||||
.bind(ws)
|
||||
.bind(tag)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job_completed (id, workspace_id, status, duration_ms) VALUES ($1, $2, $3::job_status, 0)",
|
||||
)
|
||||
.bind(id)
|
||||
.bind(ws)
|
||||
.bind(status)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
}
|
||||
|
||||
for (query, expected) in [
|
||||
("", 3),
|
||||
("tags=deno", 2),
|
||||
("tags=deno&success=true", 1),
|
||||
("tags=deno,python3&completed_after_s_ago=3600", 3),
|
||||
] {
|
||||
let response = client
|
||||
.client()
|
||||
.get(format!(
|
||||
"{}/w/test-workspace/jobs/completed/count_jobs?{query}",
|
||||
client.baseurl()
|
||||
))
|
||||
.send()
|
||||
.await?;
|
||||
assert!(
|
||||
response.status().is_success(),
|
||||
"{query}: {}",
|
||||
response.text().await?
|
||||
);
|
||||
assert_eq!(response.json::<i64>().await?, expected, "{query}");
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -258,7 +258,6 @@ async fn test_new_turns_wait_for_conversation_cleanup_and_recreate(
|
||||
"test-user",
|
||||
"hi again",
|
||||
conv_id,
|
||||
false,
|
||||
)
|
||||
.await?;
|
||||
windmill_common::flow_conversations::add_message_to_conversation_tx(
|
||||
@@ -269,7 +268,6 @@ async fn test_new_turns_wait_for_conversation_cleanup_and_recreate(
|
||||
windmill_common::flow_conversations::MessageType::User,
|
||||
None,
|
||||
true,
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
@@ -23,7 +23,6 @@ async-trait.workspace = true
|
||||
async-stream.workspace = true
|
||||
base64.workspace = true
|
||||
bytes.workspace = true
|
||||
indexmap.workspace = true
|
||||
eventsource-stream.workspace = true
|
||||
futures.workspace = true
|
||||
http.workspace = true
|
||||
|
||||
@@ -1074,8 +1074,7 @@ impl BedrockQueryBuilder {
|
||||
|
||||
let mut accumulated_text = String::new();
|
||||
let mut events_str = String::new();
|
||||
let mut accumulated_tool_calls: indexmap::IndexMap<String, StreamingToolCall> =
|
||||
indexmap::IndexMap::new();
|
||||
let mut accumulated_tool_calls: HashMap<String, StreamingToolCall> = HashMap::new();
|
||||
let mut current_tool_use_id: Option<String> = None;
|
||||
let mut usage: Option<TokenUsage> = None;
|
||||
// Claude reasoning block for the turn (only populated when thinking is on),
|
||||
@@ -1264,10 +1263,7 @@ mod tests {
|
||||
// recovers the uncached share by subtracting the details back out.
|
||||
assert_eq!(usage["usage"]["prompt_tokens"], 1010);
|
||||
assert_eq!(usage["usage"]["completion_tokens"], 7);
|
||||
assert_eq!(
|
||||
usage["usage"]["prompt_tokens_details"]["cached_tokens"],
|
||||
900
|
||||
);
|
||||
assert_eq!(usage["usage"]["prompt_tokens_details"]["cached_tokens"], 900);
|
||||
assert_eq!(
|
||||
usage["usage"]["prompt_tokens_details"]["cache_write_tokens"],
|
||||
100
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
use std::collections::HashMap;
|
||||
|
||||
use eventsource_stream::Eventsource;
|
||||
use indexmap::IndexMap;
|
||||
use reqwest::Response;
|
||||
use serde::Deserialize;
|
||||
use tokio_stream::StreamExt;
|
||||
@@ -138,9 +137,7 @@ pub struct OpenAISSEParser {
|
||||
pub accumulated_content: String,
|
||||
/// The thinking streamed before the answer, kept so it can be stored with it.
|
||||
pub accumulated_reasoning: String,
|
||||
// Insertion-ordered in every parser: tool calls run and are persisted in the order the
|
||||
// stream showed them, and a chat attaches a round's thinking to its first call.
|
||||
pub accumulated_tool_calls: IndexMap<i64, OpenAIToolCall>,
|
||||
pub accumulated_tool_calls: HashMap<i64, OpenAIToolCall>,
|
||||
pub events_str: String,
|
||||
pub stream_event_processor: Box<dyn StreamEventSink>,
|
||||
/// Token usage from final chunk (when stream_options.include_usage is true)
|
||||
@@ -152,7 +149,7 @@ impl OpenAISSEParser {
|
||||
Self {
|
||||
accumulated_content: String::new(),
|
||||
accumulated_reasoning: String::new(),
|
||||
accumulated_tool_calls: IndexMap::new(),
|
||||
accumulated_tool_calls: HashMap::new(),
|
||||
events_str: String::new(),
|
||||
stream_event_processor,
|
||||
usage: None,
|
||||
@@ -362,7 +359,7 @@ pub struct AnthropicSSEParser {
|
||||
pub accumulated_content: String,
|
||||
/// The thinking streamed before the answer, kept so it can be stored with it.
|
||||
pub accumulated_reasoning: String,
|
||||
pub accumulated_tool_calls: IndexMap<i64, OpenAIToolCall>,
|
||||
pub accumulated_tool_calls: HashMap<i64, OpenAIToolCall>,
|
||||
pub events_str: String,
|
||||
pub stream_event_processor: Box<dyn StreamEventSink>,
|
||||
/// Track content block types by index
|
||||
@@ -385,7 +382,7 @@ impl AnthropicSSEParser {
|
||||
Self {
|
||||
accumulated_content: String::new(),
|
||||
accumulated_reasoning: String::new(),
|
||||
accumulated_tool_calls: IndexMap::new(),
|
||||
accumulated_tool_calls: HashMap::new(),
|
||||
events_str: String::new(),
|
||||
stream_event_processor,
|
||||
content_blocks: HashMap::new(),
|
||||
@@ -604,7 +601,7 @@ pub struct GeminiSSEParser {
|
||||
pub accumulated_content: String,
|
||||
/// The thinking streamed before the answer, kept so it can be stored with it.
|
||||
pub accumulated_reasoning: String,
|
||||
pub accumulated_tool_calls: IndexMap<i64, OpenAIToolCall>,
|
||||
pub accumulated_tool_calls: HashMap<i64, OpenAIToolCall>,
|
||||
pub events_str: String,
|
||||
pub stream_event_processor: Box<dyn StreamEventSink>,
|
||||
tool_call_index: i64,
|
||||
@@ -618,7 +615,7 @@ impl GeminiSSEParser {
|
||||
Self {
|
||||
accumulated_content: String::new(),
|
||||
accumulated_reasoning: String::new(),
|
||||
accumulated_tool_calls: IndexMap::new(),
|
||||
accumulated_tool_calls: HashMap::new(),
|
||||
events_str: String::new(),
|
||||
stream_event_processor,
|
||||
tool_call_index: 0,
|
||||
@@ -836,7 +833,7 @@ pub struct OpenAIResponsesSSEParser {
|
||||
pub accumulated_content: String,
|
||||
/// The reasoning summary streamed before the answer, kept so it can be stored with it.
|
||||
pub accumulated_reasoning: String,
|
||||
pub accumulated_tool_calls: IndexMap<String, OpenAIToolCall>,
|
||||
pub accumulated_tool_calls: HashMap<String, OpenAIToolCall>,
|
||||
/// Maps item_id -> (name, call_id) for function calls
|
||||
tool_call_metadata: HashMap<String, (String, String)>,
|
||||
/// Maps item_id -> accumulated arguments
|
||||
@@ -858,7 +855,7 @@ impl OpenAIResponsesSSEParser {
|
||||
Self {
|
||||
accumulated_content: String::new(),
|
||||
accumulated_reasoning: String::new(),
|
||||
accumulated_tool_calls: IndexMap::new(),
|
||||
accumulated_tool_calls: HashMap::new(),
|
||||
tool_call_metadata: HashMap::new(),
|
||||
tool_call_arguments: HashMap::new(),
|
||||
events_str: String::new(),
|
||||
|
||||
@@ -78,50 +78,17 @@ impl Default for OutputType {
|
||||
#[serde(tag = "kind", rename_all = "lowercase")]
|
||||
pub enum Memory {
|
||||
Off,
|
||||
Window {
|
||||
#[serde(default, deserialize_with = "deserialize_null_as_zero")]
|
||||
context_length: usize,
|
||||
},
|
||||
/// Written before `window`. Its `memory_id` stays a fallback behind the run's memory id.
|
||||
Auto {
|
||||
#[serde(default, deserialize_with = "deserialize_null_as_zero")]
|
||||
#[serde(default)]
|
||||
context_length: usize,
|
||||
#[serde(default, deserialize_with = "deserialize_blank_as_none")]
|
||||
#[serde(default)]
|
||||
memory_id: Option<Uuid>,
|
||||
},
|
||||
/// Written before a step had history inputs of its own, and read on its own where it remains.
|
||||
Manual {
|
||||
messages: Vec<OpenAIMessage>,
|
||||
},
|
||||
}
|
||||
|
||||
// An editor form can leave `""` in a legacy baked id it never filled; it means no id rather than
|
||||
// failing every run of the step.
|
||||
fn deserialize_blank_as_none<'de, D: serde::Deserializer<'de>>(
|
||||
deserializer: D,
|
||||
) -> Result<Option<Uuid>, D::Error> {
|
||||
match <Option<String> as serde::Deserialize>::deserialize(deserializer)? {
|
||||
Some(id) if !id.trim().is_empty() => Uuid::parse_str(id.trim())
|
||||
.map(Some)
|
||||
.map_err(serde::de::Error::custom),
|
||||
_ => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
// A count the editor's number field was cleared of is stored as `null`, which `default` does not
|
||||
// cover; it reads as 0, memory off, rather than failing every run of the step.
|
||||
fn deserialize_null_as_zero<'de, D: serde::Deserializer<'de>>(
|
||||
deserializer: D,
|
||||
) -> Result<usize, D::Error> {
|
||||
<Option<usize> as serde::Deserialize>::deserialize(deserializer).map(Option::unwrap_or_default)
|
||||
}
|
||||
|
||||
fn deserialize_present<'de, D: serde::Deserializer<'de>>(
|
||||
deserializer: D,
|
||||
) -> Result<Option<serde_json::Value>, D::Error> {
|
||||
<serde_json::Value as serde::Deserialize>::deserialize(deserializer).map(Some)
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct AIAgentArgsRaw {
|
||||
provider: ProviderWithResource,
|
||||
@@ -136,12 +103,6 @@ struct AIAgentArgsRaw {
|
||||
streaming: Option<bool>,
|
||||
max_iterations: Option<usize>,
|
||||
memory: Option<Memory>,
|
||||
// A null must stay distinguishable from an absent key: a step whose own memory id evaluates to
|
||||
// nothing runs stateless instead of falling back to the run's memory id.
|
||||
#[serde(default, deserialize_with = "deserialize_present")]
|
||||
memory_id: Option<serde_json::Value>,
|
||||
#[serde(default)]
|
||||
previous_messages: Option<Vec<OpenAIMessage>>,
|
||||
enabled_tools: Option<Vec<String>>,
|
||||
// Legacy field for backward compatibility
|
||||
messages_context_length: Option<usize>,
|
||||
@@ -163,10 +124,6 @@ pub struct AIAgentArgs {
|
||||
pub streaming: Option<bool>,
|
||||
pub max_iterations: Option<usize>,
|
||||
pub memory: Option<Memory>,
|
||||
/// Memory id set on the step, overriding the run's. Empty when its expression produced none.
|
||||
pub memory_id: Option<String>,
|
||||
/// History supplied by the flow, replayed without reading or writing memory.
|
||||
pub previous_messages: Option<Vec<OpenAIMessage>>,
|
||||
/// Which of the agent's tools this run may call; `narrow_roster` holds what the names are and
|
||||
/// what `None` means.
|
||||
pub enabled_tools: Option<Vec<String>>,
|
||||
@@ -182,17 +139,12 @@ impl From<AIAgentArgsRaw> for AIAgentArgs {
|
||||
});
|
||||
|
||||
// Backward compatibility: if context_length is 0, use off mode
|
||||
let memory = memory.map(|memory| match memory {
|
||||
Memory::Auto { context_length: 0, .. } | Memory::Window { context_length: 0 } => {
|
||||
let memory = memory.map(|memory| {
|
||||
if let Memory::Auto { context_length: 0, .. } = memory {
|
||||
Memory::Off
|
||||
} else {
|
||||
memory
|
||||
}
|
||||
memory => memory,
|
||||
});
|
||||
|
||||
let memory_id = raw.memory_id.map(|value| match value {
|
||||
serde_json::Value::Null => String::new(),
|
||||
serde_json::Value::String(s) => s.trim().to_string(),
|
||||
value => value.to_string(),
|
||||
});
|
||||
|
||||
AIAgentArgs {
|
||||
@@ -207,8 +159,6 @@ impl From<AIAgentArgsRaw> for AIAgentArgs {
|
||||
streaming: raw.streaming,
|
||||
max_iterations: raw.max_iterations,
|
||||
memory,
|
||||
memory_id,
|
||||
previous_messages: raw.previous_messages,
|
||||
enabled_tools: raw.enabled_tools,
|
||||
credentials_check: raw.credentials_check.unwrap_or(false),
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
use axum::{
|
||||
extract::{Path, Query},
|
||||
routing::{delete, get, post},
|
||||
routing::{delete, get},
|
||||
Extension, Json, Router,
|
||||
};
|
||||
use chrono::{DateTime, Utc};
|
||||
@@ -15,14 +15,13 @@ use windmill_common::{
|
||||
db::{UserDB, DB},
|
||||
error::{JsonResult, Result},
|
||||
flow_conversations::MessageType,
|
||||
utils::{not_found_if_none, paginate, truncate_with_ellipsis, Pagination},
|
||||
utils::{not_found_if_none, paginate, Pagination},
|
||||
};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
.route("/list", get(list_conversations))
|
||||
.route("/delete/{conversation_id}", delete(delete_conversation))
|
||||
.route("/update/{conversation_id}", post(update_conversation))
|
||||
.route("/{conversation_id}/messages", get(list_messages))
|
||||
}
|
||||
|
||||
@@ -37,37 +36,11 @@ pub struct FlowConversationMessage {
|
||||
pub created_seq: i64,
|
||||
pub step_name: Option<String>,
|
||||
pub success: bool,
|
||||
/// On a tool row, the arguments the model wrote. For a Windmill tool these exclude the
|
||||
/// inputs its step wires in. Null for a web search, whose query the provider does not
|
||||
/// return.
|
||||
pub tool_arguments: Option<String>,
|
||||
/// On a tool row, the text the model got back, or what the call failed with; a web
|
||||
/// search's citations.
|
||||
pub tool_result: Option<String>,
|
||||
/// On an answer, the thinking that produced it; on a tool row, the thinking that led to
|
||||
/// the call. The agent job keeps the turn's thinking as one string.
|
||||
pub reasoning: Option<String>,
|
||||
/// The files a user message carried, as object-storage references
|
||||
/// (`[{input, s3, storage?, filename?}]`).
|
||||
pub attachments: Option<sqlx::types::JsonValue>,
|
||||
}
|
||||
|
||||
/// Which conversations a listing holds. A test chat was started from the editor's test
|
||||
/// panel; a deployed one from the flow itself.
|
||||
#[derive(Deserialize, Default, Clone, Copy)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum ConversationKind {
|
||||
Test,
|
||||
/// The default: a deployed flow's chat should not surface someone's trial runs.
|
||||
#[default]
|
||||
Deployed,
|
||||
All,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub struct ListConversationsQuery {
|
||||
pub flow_path: Option<String>,
|
||||
pub kind: Option<ConversationKind>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -94,7 +67,6 @@ async fn list_conversations(
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"created_by",
|
||||
"is_test",
|
||||
])
|
||||
.and_where_eq("workspace_id", "?".bind(&w_id));
|
||||
|
||||
@@ -102,16 +74,6 @@ async fn list_conversations(
|
||||
sqlb.and_where_eq("flow_path", "?".bind(flow_path));
|
||||
}
|
||||
|
||||
match query.kind.unwrap_or_default() {
|
||||
ConversationKind::Test => {
|
||||
sqlb.and_where_eq("is_test", "true");
|
||||
}
|
||||
ConversationKind::Deployed => {
|
||||
sqlb.and_where_eq("is_test", "false");
|
||||
}
|
||||
ConversationKind::All => {}
|
||||
}
|
||||
|
||||
sqlb.order_by("updated_at", true)
|
||||
.limit(per_page as i64)
|
||||
.offset(offset as i64);
|
||||
@@ -139,7 +101,7 @@ async fn delete_conversation(
|
||||
// Verify the conversation exists and belongs to the user
|
||||
let conversation = sqlx::query_as!(
|
||||
FlowConversation,
|
||||
"SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test
|
||||
"SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by
|
||||
FROM flow_conversation
|
||||
WHERE id = $1 AND workspace_id = $2",
|
||||
conversation_id,
|
||||
@@ -186,50 +148,6 @@ async fn delete_conversation(
|
||||
Ok(format!("Conversation {} deleted", conversation_id))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub struct UpdateConversation {
|
||||
pub title: String,
|
||||
}
|
||||
|
||||
async fn update_conversation(
|
||||
authed: ApiAuthed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path((w_id, conversation_id)): Path<(String, Uuid)>,
|
||||
Json(update): Json<UpdateConversation>,
|
||||
) -> Result<String> {
|
||||
// Postgres refuses a NUL in a text column, so it must not reach the query as a 500.
|
||||
if update.title.contains('\0') {
|
||||
return Err(windmill_common::error::Error::BadRequest(
|
||||
"title cannot contain a NUL character".to_string(),
|
||||
));
|
||||
}
|
||||
// The column is VARCHAR(255) and the helper appends an ellipsis to what it cuts, so the
|
||||
// bound it takes is three short of the column's. A longer title would otherwise reach
|
||||
// Postgres as a 22001 and come back a 500.
|
||||
let title = truncate_with_ellipsis(update.title.trim(), 252);
|
||||
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
|
||||
// `updated_at` is kept: the list is ordered by it, and a rename must not move the
|
||||
// chat to the top the way a new turn does.
|
||||
let updated = sqlx::query_scalar!(
|
||||
"UPDATE flow_conversation SET title = $1, updated_at = updated_at
|
||||
WHERE id = $2 AND workspace_id = $3
|
||||
RETURNING id",
|
||||
title,
|
||||
conversation_id,
|
||||
&w_id
|
||||
)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?;
|
||||
|
||||
not_found_if_none(updated, "Conversation", conversation_id.to_string())?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(format!("Conversation {} updated", conversation_id))
|
||||
}
|
||||
|
||||
async fn list_messages(
|
||||
authed: ApiAuthed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
@@ -260,7 +178,7 @@ async fn list_messages(
|
||||
let messages = if let Some(after_seq) = query.after_seq {
|
||||
sqlx::query_as!(
|
||||
FlowConversationMessage,
|
||||
r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments
|
||||
r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success
|
||||
FROM flow_conversation_message
|
||||
WHERE conversation_id = $1
|
||||
AND created_seq > $2
|
||||
@@ -277,9 +195,9 @@ async fn list_messages(
|
||||
// Fetch messages for this conversation, oldest first, but reverse the order of the messages for easy rendering on the frontend
|
||||
sqlx::query_as!(
|
||||
FlowConversationMessage,
|
||||
r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments
|
||||
r#"SELECT id, conversation_id, message_type as "message_type: MessageType", content, job_id, created_at, created_seq, step_name, success
|
||||
FROM (
|
||||
SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success, tool_arguments, tool_result, reasoning, attachments
|
||||
SELECT id, conversation_id, message_type, content, job_id, created_at, created_seq, step_name, success
|
||||
FROM flow_conversation_message
|
||||
WHERE conversation_id = $1
|
||||
ORDER BY created_seq DESC
|
||||
|
||||
@@ -391,7 +391,11 @@ async fn concurrent_role_creations_both_survive(db: Pool<Postgres>) -> anyhow::R
|
||||
assert_eq!(a.0, 200, "{}", a.1);
|
||||
assert_eq!(b.0, 200, "{}", b.1);
|
||||
|
||||
let catalog = windmill_common::datatable_roles::read_role_catalog(&db).await?;
|
||||
let catalog = windmill_common::datatable_roles::read_role_catalog(
|
||||
&db,
|
||||
windmill_common::datatable_roles::DatatableRoleCluster::Instance,
|
||||
)
|
||||
.await?;
|
||||
let recorded: Vec<&str> = catalog.values().map(|r| r.name.as_str()).collect();
|
||||
for name in &names {
|
||||
assert!(
|
||||
@@ -460,7 +464,11 @@ async fn a_role_delete_that_fails_part_way_leaves_the_role_disabled(
|
||||
let body = resp.text().await?;
|
||||
assert_eq!(status, 400, "{body}");
|
||||
|
||||
let catalog = windmill_common::datatable_roles::read_role_catalog(&db).await?;
|
||||
let catalog = windmill_common::datatable_roles::read_role_catalog(
|
||||
&db,
|
||||
windmill_common::datatable_roles::DatatableRoleCluster::Instance,
|
||||
)
|
||||
.await?;
|
||||
let role = catalog
|
||||
.get(&id)
|
||||
.expect("a failed delete keeps the entry to retry");
|
||||
|
||||
@@ -26,9 +26,7 @@ use windmill_api_auth::{check_scopes, get_scope_tags, ApiAuthed};
|
||||
use windmill_common::{
|
||||
db::{UserDB, UserDbWithAuthed},
|
||||
error::{self, Error},
|
||||
flow_conversations::{
|
||||
add_message_to_conversation_tx, message_attachments, MessageExtras, MessageType,
|
||||
},
|
||||
flow_conversations::{add_message_to_conversation_tx, MessageType},
|
||||
get_latest_flow_version_info_for_path,
|
||||
jobs::{
|
||||
check_tag_available_for_workspace_internal, format_result, script_path_to_payload,
|
||||
@@ -655,11 +653,9 @@ pub async fn set_flow_memory_id(
|
||||
pub async fn process_flow_run_query_params(
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
job_id: Uuid,
|
||||
w_id: &str,
|
||||
flow_path: &str,
|
||||
run_query: &RunJobQuery,
|
||||
) -> error::Result<()> {
|
||||
if let Some(memory_id) = run_query.memory_key(w_id, flow_path) {
|
||||
if let Some(memory_id) = run_query.memory_id {
|
||||
set_flow_memory_id(tx, job_id, memory_id).await?;
|
||||
}
|
||||
Ok(())
|
||||
@@ -673,13 +669,10 @@ pub async fn handle_chat_conversation_messages(
|
||||
run_query: &RunJobQuery,
|
||||
user_message_raw: Option<&Box<serde_json::value::RawValue>>,
|
||||
job_id: Uuid,
|
||||
is_test: bool,
|
||||
// The run's args, for the files the message carried.
|
||||
args: &HashMap<String, Box<serde_json::value::RawValue>>,
|
||||
) -> error::Result<()> {
|
||||
// Names the query parameter rather than the field: it is not a flow argument, and
|
||||
// supplying it as one is the first thing tried on reading `memory_id is required`.
|
||||
let memory_id = run_query.memory_key(w_id, flow_path).ok_or_else(|| {
|
||||
let memory_id = run_query.memory_id.ok_or_else(|| {
|
||||
windmill_common::error::Error::BadRequest(
|
||||
"memory_id is required for chat-enabled flows. Pass it as the `memory_id` query \
|
||||
parameter, not as a flow argument: it names the conversation the turn belongs to, \
|
||||
@@ -708,12 +701,11 @@ pub async fn handle_chat_conversation_messages(
|
||||
&authed.username,
|
||||
&user_message,
|
||||
memory_id,
|
||||
is_test,
|
||||
)
|
||||
.await?;
|
||||
|
||||
// The run this message started. The row keeps the files the message carried as
|
||||
// references; its args are the only record of every other flow input, and nothing
|
||||
// The run this message started. Its args are the only record of what the message
|
||||
// carried besides its text — attachments and every other flow input — and nothing
|
||||
// written later points at them: an assistant row holds the AI agent step's job.
|
||||
add_message_to_conversation_tx(
|
||||
tx,
|
||||
@@ -723,7 +715,6 @@ pub async fn handle_chat_conversation_messages(
|
||||
MessageType::User,
|
||||
None,
|
||||
true,
|
||||
Some(&MessageExtras { attachments: message_attachments(args), ..Default::default() }),
|
||||
)
|
||||
.await?;
|
||||
|
||||
@@ -831,7 +822,7 @@ pub async fn run_flow<'c>(
|
||||
.await?;
|
||||
|
||||
// Set memory_id if provided (for agent memory)
|
||||
if let Some(memory_id) = run_query.memory_key(w_id, flow_path) {
|
||||
if let Some(memory_id) = run_query.memory_id {
|
||||
set_flow_memory_id(&mut tx, uuid, memory_id).await?;
|
||||
}
|
||||
|
||||
@@ -845,8 +836,6 @@ pub async fn run_flow<'c>(
|
||||
&run_query,
|
||||
args.args.get("user_message"),
|
||||
uuid,
|
||||
false,
|
||||
&args.args,
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
|
||||
@@ -47,25 +47,13 @@ pub struct RunJobQuery {
|
||||
pub cache_ignore_s3_path: Option<bool>,
|
||||
pub skip_preprocessor: Option<bool>,
|
||||
pub poll_delay_ms: Option<u64>,
|
||||
/// Any string; see [`RunJobQuery::memory_key`].
|
||||
pub memory_id: Option<String>,
|
||||
pub memory_id: Option<Uuid>,
|
||||
pub trigger_external_id: Option<String>,
|
||||
pub service_name: Option<String>,
|
||||
pub suspended_mode: Option<bool>,
|
||||
}
|
||||
|
||||
impl RunJobQuery {
|
||||
/// The memory id as stored in `flow_status.memory_id`: a uuid is kept, any other string hashed
|
||||
/// within the workspace and the flow being run.
|
||||
pub fn memory_key(&self, workspace_id: &str, flow_path: &str) -> Option<Uuid> {
|
||||
self.memory_id
|
||||
.as_deref()
|
||||
.filter(|memory_id| !memory_id.trim().is_empty())
|
||||
.map(|memory_id| {
|
||||
windmill_common::flow_conversations::memory_key(workspace_id, flow_path, memory_id)
|
||||
})
|
||||
}
|
||||
|
||||
pub async fn get_scheduled_for(
|
||||
&self,
|
||||
db: &DB,
|
||||
|
||||
@@ -42,6 +42,7 @@ use axum::{
|
||||
routing::{get, post},
|
||||
Json, Router,
|
||||
};
|
||||
use serde_json::json;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use windmill_ai::ai_cache::bump_instance_ai_config_revision;
|
||||
@@ -1719,15 +1720,7 @@ async fn list_custom_instance_pg_databases(
|
||||
))
|
||||
})?;
|
||||
|
||||
if !windmill_api_auth::is_super_admin_authed(&db, &authed).await? {
|
||||
// Which workspace reserved a fork copy is nobody else's business: it would enumerate every
|
||||
// pending fork on the instance.
|
||||
for entry in result.values_mut() {
|
||||
entry.workspace_id = None;
|
||||
}
|
||||
return Ok(Json(result));
|
||||
}
|
||||
{
|
||||
if windmill_api_auth::is_super_admin_authed(&db, &authed).await? {
|
||||
// Enrich each database with the list of workspaces referencing it through
|
||||
// either a ducklake catalog or a datatable database whose resource_type is
|
||||
// 'instance'. Not stored in DB to avoid drift.
|
||||
@@ -1926,6 +1919,16 @@ async fn setup_custom_instance_pg_database(
|
||||
// Before anything is recorded: the status written below replaces the registry entry, and with it
|
||||
// the workspace a fork copy is reserved for.
|
||||
require_super_admin(&db, &authed).await?;
|
||||
// A re-run keeps the fork reservation: without it, the workspace the copy was made for could no
|
||||
// longer import into it or finish its fork.
|
||||
let workspace_id = sqlx::query_scalar::<_, Option<String>>(
|
||||
"SELECT value->'databases'->$1->>'workspace_id' FROM global_settings
|
||||
WHERE name = 'custom_instance_pg_databases'",
|
||||
)
|
||||
.bind(&dbname)
|
||||
.fetch_optional(&db)
|
||||
.await?
|
||||
.flatten();
|
||||
let mut logs = CustomInstanceDbLogs::default();
|
||||
let result = setup_custom_instance_pg_database_inner(authed, &db, &dbname, &mut logs).await;
|
||||
let success = result.is_ok();
|
||||
@@ -1936,25 +1939,14 @@ async fn setup_custom_instance_pg_database(
|
||||
error,
|
||||
tag: body.tag,
|
||||
used_by_workspaces: vec![],
|
||||
workspace_id: None,
|
||||
workspace_id,
|
||||
};
|
||||
let status_json = serde_json::to_value(&status).map_err(to_anyhow)?;
|
||||
// The fork reservation is carried over inside the write, from whatever the row holds then: a
|
||||
// rename migrating it while the setup above ran would otherwise be overwritten with the value
|
||||
// this request started from, stranding the copy under the archived workspace.
|
||||
let saved = sqlx::query_scalar::<_, serde_json::Value>(
|
||||
r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}',
|
||||
COALESCE(value->'databases', '{}'::jsonb)
|
||||
|| jsonb_build_object($1::text, $2::jsonb || jsonb_build_object(
|
||||
'workspace_id', value->'databases'->$1::text->'workspace_id')))
|
||||
WHERE name = 'custom_instance_pg_databases'
|
||||
RETURNING value->'databases'->$1::text"#,
|
||||
)
|
||||
.bind(&dbname)
|
||||
.bind(&status_json)
|
||||
.fetch_one(&db)
|
||||
.await?;
|
||||
let status: CustomInstanceDb = serde_json::from_value(saved).map_err(to_anyhow)?;
|
||||
// Save that the database was setup successfully
|
||||
sqlx::query!(
|
||||
r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}', (COALESCE(value->'databases', '{}'::jsonb) || to_jsonb($1::json))) WHERE name = 'custom_instance_pg_databases'"#,
|
||||
json!({ dbname: status_json })
|
||||
).execute(&db).await?;
|
||||
|
||||
Ok(Json(status))
|
||||
}
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
//! Ownership and grants on the objects of an instance data table.
|
||||
//! Ownership and grants on the objects of a data table on a cluster Windmill manages.
|
||||
//!
|
||||
//! [`datatable_permissions`](crate::datatable_permissions) decides who may connect as which role;
|
||||
//! this decides what each role may then touch. Every change is a real `GRANT`, `REVOKE`,
|
||||
@@ -33,7 +33,7 @@ use windmill_audit::audit_oss::audit_log;
|
||||
use windmill_audit::ActionKind;
|
||||
use windmill_common::datatable_roles::{
|
||||
lock_role_catalog, quote_ident, read_role_catalog, read_role_catalog_tx, DatatableRoleCatalog,
|
||||
ADMIN_DATATABLE_ROLE, CUSTOM_INSTANCE_USER,
|
||||
DatatableRoleCluster, ADMIN_DATATABLE_ROLE, CUSTOM_INSTANCE_USER,
|
||||
};
|
||||
use windmill_common::error::{pg_error_message, Error, JsonResult, Result};
|
||||
use windmill_common::workspaces::{resolve_governing_datatable, DataTable, GoverningDatatable};
|
||||
@@ -295,22 +295,22 @@ fn role_names(catalog: &DatatableRoleCatalog) -> Vec<String> {
|
||||
names
|
||||
}
|
||||
|
||||
fn ensure_instance(governing: &GoverningDatatable) -> Result<()> {
|
||||
if governing.is_instance() {
|
||||
return Ok(());
|
||||
}
|
||||
Err(Error::BadRequest(format!(
|
||||
"Data table '{}' is backed by a Postgres resource, so its access is managed on that \
|
||||
server directly. Only a data table on the Windmill instance's own database has data \
|
||||
table roles to grant to.",
|
||||
governing.name
|
||||
)))
|
||||
/// The cluster whose roles the data table's grants name.
|
||||
fn ensure_managed(governing: &GoverningDatatable) -> Result<DatatableRoleCluster> {
|
||||
governing.role_cluster().ok_or_else(|| {
|
||||
Error::BadRequest(format!(
|
||||
"Data table '{}' is backed by a Postgres resource, so its access is managed on that \
|
||||
server directly. Only a data table on a database Windmill manages has data table \
|
||||
roles to grant to.",
|
||||
governing.name
|
||||
))
|
||||
})
|
||||
}
|
||||
|
||||
/// The data table's `admin` connection, and the notices Postgres sends on it.
|
||||
///
|
||||
/// Authorization: connects as `custom_instance_user` with the instance's own credentials and checks
|
||||
/// nothing. Callers MUST have authorized the request first — a request about to be refused must
|
||||
/// Authorization: connects as `custom_instance_user` with the cluster's stored credentials and
|
||||
/// checks nothing. Callers MUST have authorized the request first — a request about to be refused must
|
||||
/// not get as far as this connection.
|
||||
async fn connect_as_admin_unchecked(
|
||||
db: &DB,
|
||||
@@ -320,21 +320,33 @@ async fn connect_as_admin_unchecked(
|
||||
mpsc::UnboundedReceiver<DbError>,
|
||||
String,
|
||||
)> {
|
||||
ensure_instance(governing)?;
|
||||
let cluster = ensure_managed(governing)?;
|
||||
// Built from the authorized entry, never by resolving the settings again: a save in between
|
||||
// could point the entry at a resource on another server and back, and this connection would
|
||||
// then alter a database the later checks of the entry never see.
|
||||
let mut pg = PgDatabase::parse_uri(&windmill_common::get_database_url().await?.as_str().await)?;
|
||||
pg.dbname = governing
|
||||
let dbname = governing
|
||||
.datatable
|
||||
.database
|
||||
.as_ref()
|
||||
.expect("a governing entry owns a database")
|
||||
.resource_path
|
||||
.clone();
|
||||
pg.user = Some(CUSTOM_INSTANCE_USER.to_string());
|
||||
pg.password = Some(windmill_common::utils::get_custom_pg_instance_password(db).await?);
|
||||
let dbname = pg.dbname.clone();
|
||||
let pg = match cluster {
|
||||
DatatableRoleCluster::Instance => {
|
||||
let mut pg =
|
||||
PgDatabase::parse_uri(&windmill_common::get_database_url().await?.as_str().await)?;
|
||||
pg.dbname = dbname.clone();
|
||||
pg.user = Some(CUSTOM_INSTANCE_USER.to_string());
|
||||
pg.password = Some(windmill_common::utils::get_custom_pg_instance_password(db).await?);
|
||||
pg
|
||||
}
|
||||
DatatableRoleCluster::ExternalInstance => {
|
||||
windmill_common::external_instance_pg::external_instance_connection_unchecked(
|
||||
db, &dbname, false,
|
||||
)
|
||||
.await?
|
||||
}
|
||||
};
|
||||
let (client, mut connection) = pg.connect(Some(db)).await?;
|
||||
// Unbounded: the driver must never wait on the receiver, which only drains once the statement
|
||||
// the driver is carrying has completed.
|
||||
@@ -1020,12 +1032,12 @@ async fn get_datatable_acl(
|
||||
let target: AclTarget = query.try_into()?;
|
||||
let governing = resolve_governing_datatable(&db, &w_id, &datatable_name).await?;
|
||||
ensure_reaches_governing_datatable(&db, &w_id, &datatable_name, &governing, &authed).await?;
|
||||
ensure_instance(&governing)?;
|
||||
let cluster = ensure_managed(&governing)?;
|
||||
let editable = ensure_governs_datatable(&db, &authed, &w_id, &governing)
|
||||
.await
|
||||
.is_ok();
|
||||
let roles = if editable {
|
||||
role_names(&read_role_catalog(&db).await?)
|
||||
role_names(&read_role_catalog(&db, cluster).await?)
|
||||
} else {
|
||||
vec![]
|
||||
};
|
||||
@@ -1310,7 +1322,7 @@ async fn authorize_acl_change(
|
||||
) -> Result<GoverningDatatable> {
|
||||
let governing = resolve_governing_datatable(db, w_id, datatable_name).await?;
|
||||
ensure_governs_datatable(db, authed, w_id, &governing).await?;
|
||||
ensure_instance(&governing)?;
|
||||
ensure_managed(&governing)?;
|
||||
Ok(governing)
|
||||
}
|
||||
|
||||
@@ -1320,8 +1332,14 @@ static APPLY_SLOT: tokio::sync::Semaphore = tokio::sync::Semaphore::const_new(1)
|
||||
/// provisioned before data table roles gave `custom_instance_user` none. Adds that option to its
|
||||
/// database and `public` privileges, and nothing else: default privileges are left alone, since a
|
||||
/// schema's change of owner is planned against them. Best-effort, as a grant it fails to enable is
|
||||
/// refused when it runs.
|
||||
async fn ensure_grant_options(client: &tokio_postgres::Client, db: &DB, dbname: &str) {
|
||||
/// refused when it runs. An external instance database was created with the options, so one
|
||||
/// missing there is someone's deliberate revoke and is left alone.
|
||||
async fn ensure_grant_options(
|
||||
client: &tokio_postgres::Client,
|
||||
db: &DB,
|
||||
cluster: DatatableRoleCluster,
|
||||
dbname: &str,
|
||||
) {
|
||||
let held = client
|
||||
.query_one(
|
||||
"SELECT has_database_privilege(current_database(), 'CONNECT WITH GRANT OPTION')
|
||||
@@ -1333,7 +1351,7 @@ async fn ensure_grant_options(client: &tokio_postgres::Client, db: &DB, dbname:
|
||||
)
|
||||
.await
|
||||
.is_ok_and(|row| row.get::<_, bool>(0));
|
||||
if held {
|
||||
if held || cluster != DatatableRoleCluster::Instance {
|
||||
return;
|
||||
}
|
||||
if let Err(e) = grant_options_as_server(db, dbname).await {
|
||||
@@ -1559,7 +1577,8 @@ async fn plan_datatable_acl(
|
||||
) -> JsonResult<AclPlan> {
|
||||
crate::datatable_acl_oss::ensure_datatable_acl_available()?;
|
||||
let governing = authorize_acl_change(&db, &authed, &w_id, &datatable_name).await?;
|
||||
let catalog = read_role_catalog(&db).await?;
|
||||
let cluster = ensure_managed(&governing)?;
|
||||
let catalog = read_role_catalog(&db, cluster).await?;
|
||||
let (client, _notices, dbname) = connect_as_admin_unchecked(&db, &governing).await?;
|
||||
Ok(Json(
|
||||
build_plan(&client, &dbname, &catalog, &req.target, &req.change).await?,
|
||||
@@ -1583,6 +1602,7 @@ async fn apply_datatable_acl(
|
||||
// connection could wait forever on a pool that concurrent applies, queued on the same locks,
|
||||
// have exhausted.
|
||||
let governing = authorize_acl_change(&db, &authed, &w_id, &datatable_name).await?;
|
||||
let cluster = ensure_managed(&governing)?;
|
||||
// Applies queue on an instance-wide lock while each holds a direct connection to the instance's
|
||||
// Postgres; unbounded, the queue alone could exhaust its connection limit. One at a time per
|
||||
// server, and the ones waiting hold no connection at all.
|
||||
@@ -1591,7 +1611,7 @@ async fn apply_datatable_acl(
|
||||
.await
|
||||
.map_err(|e| Error::internal_err(format!("ACL apply slot closed: {e}")))?;
|
||||
let (mut client, mut notices, dbname) = connect_as_admin_unchecked(&db, &governing).await?;
|
||||
ensure_grant_options(&client, &db, &dbname).await;
|
||||
ensure_grant_options(&client, &db, cluster, &dbname).await;
|
||||
|
||||
// Held until the change is committed: a role renamed or dropped meanwhile would change what
|
||||
// the plan names, and a settings save could move the entry onto another database. Taken in the
|
||||
@@ -1606,7 +1626,7 @@ async fn apply_datatable_acl(
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?
|
||||
.flatten();
|
||||
let catalog = read_role_catalog_tx(&mut tx).await?;
|
||||
let catalog = read_role_catalog_tx(&mut tx, cluster).await?;
|
||||
|
||||
let plan = build_plan(&client, &dbname, &catalog, &req.target, &req.change).await?;
|
||||
if !entry_unchanged(&governing, entry_now) || &plan.statements != confirmed {
|
||||
|
||||
@@ -2263,7 +2263,8 @@ struct DataTableTables {
|
||||
schemas: TableListMap,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
error: Option<String>,
|
||||
/// On the instance database: the only kind that can be under roles or have its access edited.
|
||||
/// On a database Windmill manages, on its own cluster or the external one: the only kinds that
|
||||
/// can be under roles or have their access edited.
|
||||
instance: bool,
|
||||
permissioned: bool,
|
||||
/// The roles this caller may connect as, by name; empty when not under roles.
|
||||
@@ -2520,7 +2521,7 @@ async fn list_one_datatable_tables(
|
||||
};
|
||||
let result: Result<()> = async {
|
||||
let governing = resolve_governing_datatable(db, w_id, &entry.datatable_name).await?;
|
||||
entry.instance = governing.is_instance();
|
||||
entry.instance = governing.role_cluster().is_some();
|
||||
let usable =
|
||||
crate::datatable_permissions_oss::usable_datatable_roles(db, authed, w_id, &governing)
|
||||
.await?;
|
||||
@@ -3370,7 +3371,7 @@ fn apply_pg_tls_env(
|
||||
cmd.env("PGSSLROOTCERT", "system");
|
||||
}
|
||||
Some("verify-ca") => {
|
||||
if let Some(bundle) = system_ca_bundle() {
|
||||
if let Some(bundle) = windmill_common::system_ca_bundle() {
|
||||
cmd.env("PGSSLROOTCERT", bundle);
|
||||
}
|
||||
}
|
||||
@@ -3380,21 +3381,6 @@ fn apply_pg_tls_env(
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
fn system_ca_bundle() -> Option<std::path::PathBuf> {
|
||||
std::env::var_os("SSL_CERT_FILE")
|
||||
.map(std::path::PathBuf::from)
|
||||
.into_iter()
|
||||
.chain(
|
||||
[
|
||||
"/etc/ssl/certs/ca-certificates.crt",
|
||||
"/etc/pki/tls/certs/ca-bundle.crt",
|
||||
"/etc/ssl/cert.pem",
|
||||
"/etc/ssl/ca-bundle.pem",
|
||||
]
|
||||
.map(std::path::PathBuf::from),
|
||||
)
|
||||
.find(|path| path.is_file())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod pg_tls_env_tests {
|
||||
@@ -3695,7 +3681,6 @@ async fn import_pg_database(
|
||||
}
|
||||
|
||||
let schema_only = req.fork_behavior == DataTableForkBehavior::SchemaOnly;
|
||||
let mut fork_lock: Option<Transaction<'_, Postgres>> = None;
|
||||
let source_pg = resolve_pg_source_checked(&db, &user_db, &authed, &w_id, &req.source).await?;
|
||||
let mut target_pg =
|
||||
resolve_pg_source_checked(&db, &user_db, &authed, &w_id, &req.target).await?;
|
||||
@@ -3709,10 +3694,6 @@ async fn import_pg_database(
|
||||
));
|
||||
}
|
||||
if let Some(kind) = managed_datatable_source_kind(&db, &w_id, &req.target).await? {
|
||||
// Held until the restore is done, as fork finalization takes it: a fork must not
|
||||
// commit this database while `psql` is still filling it.
|
||||
let mut tx = db.begin().await?;
|
||||
windmill_common::workspaces::lock_fork_datatables(&mut tx, &w_id).await?;
|
||||
windmill_common::ensure_fork_database_available_to(
|
||||
&db,
|
||||
kind,
|
||||
@@ -3720,7 +3701,6 @@ async fn import_pg_database(
|
||||
&w_id,
|
||||
)
|
||||
.await?;
|
||||
fork_lock = Some(tx);
|
||||
}
|
||||
}
|
||||
target_pg.dbname = override_dbname.clone();
|
||||
@@ -3744,9 +3724,6 @@ async fn import_pg_database(
|
||||
)
|
||||
.await?;
|
||||
pg_import_dump(&target_pg, &dump_file).await?;
|
||||
if let Some(tx) = fork_lock {
|
||||
tx.commit().await?;
|
||||
}
|
||||
|
||||
Ok(format!(
|
||||
"Imported from '{}' into '{}'",
|
||||
@@ -3852,22 +3829,37 @@ async fn edit_ducklake_config(
|
||||
let old_ducklakes: HashMap<String, Ducklake> =
|
||||
serde_json::from_value(old_ducklakes).unwrap_or_default();
|
||||
|
||||
// Check that non-superadmins are not abusing Instance databases
|
||||
if !is_superadmin {
|
||||
for (name, dl) in new_config.settings.ducklakes.iter() {
|
||||
if dl.catalog.resource_type == DucklakeCatalogResourceType::Instance {
|
||||
let old_dl = old_ducklakes.get(name);
|
||||
if old_dl.is_none()
|
||||
|| old_dl.unwrap().catalog.resource_type
|
||||
!= DucklakeCatalogResourceType::Instance
|
||||
|| old_dl.unwrap().catalog.resource_path != dl.catalog.resource_path
|
||||
{
|
||||
return Err(Error::BadRequest(
|
||||
"Only superadmins can create or modify ducklakes with Instance databases"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
// Check that non-superadmins are not abusing Instance databases. An unchanged catalog is left
|
||||
// alone either way, so a downgraded instance can still save lakes that already name an
|
||||
// external instance database.
|
||||
for (name, dl) in new_config.settings.ducklakes.iter() {
|
||||
let kind = &dl.catalog.resource_type;
|
||||
if !matches!(
|
||||
kind,
|
||||
DucklakeCatalogResourceType::Instance | DucklakeCatalogResourceType::ExternalInstance
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
let unchanged = old_ducklakes.get(name).is_some_and(|old| {
|
||||
&old.catalog.resource_type == kind
|
||||
&& old.catalog.resource_path == dl.catalog.resource_path
|
||||
});
|
||||
if unchanged {
|
||||
continue;
|
||||
}
|
||||
if *kind == DucklakeCatalogResourceType::ExternalInstance {
|
||||
windmill_common::external_instance_pg::ensure_external_instance_available()?;
|
||||
windmill_common::external_instance_pg::ensure_external_instance_database_registered(
|
||||
&mut tx,
|
||||
&dl.catalog.resource_path,
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
if !is_superadmin {
|
||||
return Err(Error::BadRequest(
|
||||
"Only superadmins can create or modify ducklakes with Instance databases"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3926,8 +3918,6 @@ async fn edit_datatable_config(
|
||||
let is_superadmin = require_super_admin(&db, &authed).await.is_ok();
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
// Ahead of the settings row, as fork cleanup of this workspace takes the two.
|
||||
windmill_common::workspaces::lock_fork_datatables(&mut tx, &w_id).await?;
|
||||
|
||||
// Read under the row lock this transaction will write with. `permissions`, `reference` and
|
||||
// `forked_from` are carried across from what this read returns, so a permissions save
|
||||
@@ -4061,18 +4051,20 @@ async fn edit_datatable_config(
|
||||
None => None,
|
||||
};
|
||||
// Carrying the block onto a resource-backed entry would produce a data table the chokepoint
|
||||
// refuses on every job — a save that succeeds and breaks everything afterwards. Refuse it
|
||||
// instead: turning roles off first is one step, and it keeps discarding an access decision
|
||||
// something somebody chose rather than a side effect of moving a database.
|
||||
// refuses on every job — a save that succeeds and breaks everything afterwards — and onto
|
||||
// the other managed cluster, one whose role ids name nothing in that cluster's catalog.
|
||||
// Refuse it instead: turning roles off first is one step, and it keeps discarding an access
|
||||
// decision something somebody chose rather than a side effect of moving a database.
|
||||
let old_kind = old.and_then(|old| old.database.as_ref()).map(|d| d.resource_type);
|
||||
if dt.permissions.is_some()
|
||||
&& dt
|
||||
.database
|
||||
.as_ref()
|
||||
.is_some_and(|d| d.resource_type != DataTableCatalogResourceType::Instance)
|
||||
.is_some_and(|d| Some(d.resource_type) != old_kind)
|
||||
{
|
||||
return Err(Error::BadRequest(format!(
|
||||
"Data table '{name}' is under roles, which only a data table on the instance \
|
||||
database can be. Turn its roles off before moving it to a PostgreSQL resource."
|
||||
"Data table '{name}' is under roles, which belong to the cluster its database is \
|
||||
on. Turn its roles off before moving it to another kind of database."
|
||||
)));
|
||||
}
|
||||
// A pointer names no database of its own, so the form's empty `database` is correct there.
|
||||
@@ -4117,14 +4109,6 @@ async fn edit_datatable_config(
|
||||
if unchanged {
|
||||
continue;
|
||||
}
|
||||
// Before the registration check, whose refusal would otherwise tell a workspace admin
|
||||
// which databases exist on the cluster.
|
||||
if !is_superadmin {
|
||||
return Err(Error::BadRequest(
|
||||
"Only superadmins can create or modify data tables with Instance databases"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
if database.resource_type == DataTableCatalogResourceType::ExternalInstance {
|
||||
windmill_common::external_instance_pg::ensure_external_instance_available()?;
|
||||
windmill_common::external_instance_pg::ensure_external_instance_database_registered(
|
||||
@@ -4133,6 +4117,12 @@ async fn edit_datatable_config(
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
if !is_superadmin {
|
||||
return Err(Error::BadRequest(
|
||||
"Only superadmins can create or modify data tables with Instance databases"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
// Worked out from the locked entries rather than taken from `deleted_datatables`: a settings
|
||||
@@ -4151,7 +4141,7 @@ async fn edit_datatable_config(
|
||||
// entry through a declared rename alone, and a settings sync never declares one, so an entry
|
||||
// without roles that newly points at such a database — a name added, or an existing one
|
||||
// repointed — would answer everyone there as `admin`. That holds whichever workspace governs it.
|
||||
let newly_pointed: Vec<(&String, &str)> = new_config
|
||||
let newly_pointed: Vec<(&String, DataTableCatalogResourceType, &str)> = new_config
|
||||
.settings
|
||||
.datatables
|
||||
.iter()
|
||||
@@ -4160,7 +4150,7 @@ async fn edit_datatable_config(
|
||||
let db = dt
|
||||
.database
|
||||
.as_ref()
|
||||
.filter(|d| d.resource_type == DataTableCatalogResourceType::Instance)?;
|
||||
.filter(|d| d.resource_type.is_windmill_managed())?;
|
||||
let lookup = rename_src
|
||||
.get(name.as_str())
|
||||
.copied()
|
||||
@@ -4172,66 +4162,44 @@ async fn edit_datatable_config(
|
||||
old_db.resource_type != db.resource_type
|
||||
|| old_db.resource_path != db.resource_path
|
||||
});
|
||||
repointed.then_some((name, db.resource_path.as_str()))
|
||||
repointed.then_some((name, db.resource_type, db.resource_path.as_str()))
|
||||
})
|
||||
.collect();
|
||||
// Another workspace turning roles on for the same database holds only its own settings row, so
|
||||
// without this the scan below could read past its uncommitted write. Every managed database
|
||||
// this save newly names is locked, not just the ones the scan is about: fork cleanup takes the
|
||||
// same lock to decide nothing uses the database it is dropping.
|
||||
let newly_named: std::collections::BTreeSet<&str> = new_config
|
||||
.settings
|
||||
.datatables
|
||||
.iter()
|
||||
.filter_map(|(name, dt)| {
|
||||
let db = dt
|
||||
.database
|
||||
.as_ref()
|
||||
.filter(|d| d.resource_type == DataTableCatalogResourceType::Instance)?;
|
||||
let lookup = rename_src
|
||||
.get(name.as_str())
|
||||
.copied()
|
||||
.unwrap_or(name.as_str());
|
||||
old_datatables
|
||||
.get(lookup)
|
||||
.and_then(|old| old.database.as_ref())
|
||||
.is_none_or(|old_db| {
|
||||
old_db.resource_type != db.resource_type
|
||||
|| old_db.resource_path != db.resource_path
|
||||
})
|
||||
.then_some(db.resource_path.as_str())
|
||||
})
|
||||
.collect();
|
||||
// without this the scan below could read past its uncommitted write.
|
||||
windmill_common::datatable_roles::lock_instance_databases_governance(
|
||||
&mut *tx,
|
||||
newly_pointed
|
||||
.iter()
|
||||
.map(|(_, dbname)| *dbname)
|
||||
.chain(newly_named.iter().copied()),
|
||||
newly_pointed.iter().map(|(_, _, dbname)| *dbname),
|
||||
)
|
||||
.await?;
|
||||
let governed_elsewhere: Vec<String> = if newly_pointed.is_empty() {
|
||||
let governed_elsewhere: Vec<(String, String)> = if newly_pointed.is_empty() {
|
||||
vec![]
|
||||
} else {
|
||||
sqlx::query_scalar(
|
||||
"SELECT DISTINCT dt.value->'database'->>'resource_path' FROM workspace_settings ws
|
||||
sqlx::query_as(
|
||||
"SELECT DISTINCT dt.value->'database'->>'resource_type',
|
||||
dt.value->'database'->>'resource_path'
|
||||
FROM workspace_settings ws
|
||||
CROSS JOIN LATERAL jsonb_each(COALESCE(ws.datatable->'datatables', '{}'::jsonb)) dt
|
||||
WHERE ws.workspace_id <> $1 AND dt.value ? 'permissions'
|
||||
AND dt.value->'database'->>'resource_type' = 'instance'",
|
||||
AND dt.value->'database'->>'resource_type' IN ('instance', 'external_instance')",
|
||||
)
|
||||
.bind(&w_id)
|
||||
.fetch_all(&mut *tx)
|
||||
.await?
|
||||
};
|
||||
for (name, dbname) in newly_pointed {
|
||||
for (name, kind, dbname) in newly_pointed {
|
||||
let governed_here = old_datatables.values().any(|old| {
|
||||
old.permissions.is_some()
|
||||
&& old.database.as_ref().is_some_and(|d| {
|
||||
d.resource_type == DataTableCatalogResourceType::Instance
|
||||
&& d.resource_path == dbname
|
||||
})
|
||||
&& old
|
||||
.database
|
||||
.as_ref()
|
||||
.is_some_and(|d| d.resource_type == kind && d.resource_path == dbname)
|
||||
});
|
||||
if governed_here || governed_elsewhere.iter().any(|g| g == dbname) {
|
||||
if governed_here
|
||||
|| governed_elsewhere
|
||||
.iter()
|
||||
.any(|(k, p)| k == kind.as_ref() && p == dbname)
|
||||
{
|
||||
return Err(Error::BadRequest(format!(
|
||||
"Data table '{name}' would point at database '{dbname}', which a data table under \
|
||||
roles uses, without carrying those roles: everyone reaching '{name}' would connect \
|
||||
|
||||
@@ -1438,19 +1438,7 @@ pub async fn drop_forked_datatable_databases(
|
||||
// from gaining such a pointer before the drop.
|
||||
let dropped = async {
|
||||
let mut tx = db.begin().await?;
|
||||
// The three locks a settings save takes, in its order: this workspace's data
|
||||
// tables, its settings row, and the database itself. Without them a save could
|
||||
// rename this entry, or point another one here, either side of the check below.
|
||||
windmill_common::workspaces::lock_fork_datatables(&mut tx, &w_id).await?;
|
||||
sqlx::query("SELECT 1 FROM workspace_settings WHERE workspace_id = $1 FOR UPDATE")
|
||||
.bind(&w_id)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?;
|
||||
windmill_common::datatable_roles::lock_instance_databases_governance(
|
||||
&mut tx,
|
||||
[db_to_drop.as_str()],
|
||||
)
|
||||
.await?;
|
||||
if database.resource_type
|
||||
== windmill_common::workspaces::DataTableCatalogResourceType::ExternalInstance
|
||||
{
|
||||
@@ -1476,16 +1464,6 @@ pub async fn drop_forked_datatable_databases(
|
||||
}
|
||||
windmill_common::drop_custom_instance_database(&db, db_to_drop).await?;
|
||||
}
|
||||
// The entry goes with the database: a fork this one is cloned into afterwards must
|
||||
// not inherit a pointer at a data table whose database is gone.
|
||||
sqlx::query(
|
||||
"UPDATE workspace_settings SET datatable = datatable #- ARRAY['datatables', $2]
|
||||
WHERE workspace_id = $1",
|
||||
)
|
||||
.bind(&w_id)
|
||||
.bind(dt_name)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
Ok::<_, Error>(())
|
||||
}
|
||||
@@ -1857,7 +1835,17 @@ async fn resolve_fork_catalog_pg(
|
||||
"ducklake://{ducklake_name}: malformed registry catalog identity `{catalog}`"
|
||||
))
|
||||
})?;
|
||||
let catalog_resource = if resource_type == "instance" {
|
||||
let catalog_resource = if resource_type == "external_instance" {
|
||||
serde_json::to_value(
|
||||
windmill_common::external_instance_pg::external_instance_connection_unchecked(
|
||||
db,
|
||||
resource_path,
|
||||
false,
|
||||
)
|
||||
.await?,
|
||||
)
|
||||
.map_err(|e| Error::internal_err(format!("serializing pg creds: {e}")))?
|
||||
} else if resource_type == "instance" {
|
||||
let mut pg_creds = windmill_common::PgDatabase::parse_uri(
|
||||
&windmill_common::get_database_url().await?.as_str().await,
|
||||
)?;
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: "3.0.3"
|
||||
|
||||
info:
|
||||
version: 1.814.0
|
||||
version: 1.813.0
|
||||
title: Windmill API
|
||||
|
||||
contact:
|
||||
@@ -1611,7 +1611,7 @@ paths:
|
||||
|
||||
/settings/external_instance_pg/databases:
|
||||
get:
|
||||
summary: Lists the databases Windmill created on the external instance cluster, with the workspaces whose data tables use each
|
||||
summary: Lists the databases Windmill created on the external instance cluster, with the workspaces whose data tables, Ducklake catalogs or pending fork cleanups use each
|
||||
operationId: listExternalInstancePgDatabases
|
||||
tags:
|
||||
- setting
|
||||
@@ -1653,7 +1653,7 @@ paths:
|
||||
application/json:
|
||||
schema: {}
|
||||
delete:
|
||||
summary: Drops a database Windmill created on the external instance cluster, refused while a data table uses it
|
||||
summary: Drops a database Windmill created on the external instance cluster, refused while a data table, Ducklake catalog or pending fork cleanup uses it
|
||||
operationId: dropExternalInstancePgDatabase
|
||||
tags:
|
||||
- setting
|
||||
@@ -1688,13 +1688,19 @@ paths:
|
||||
|
||||
/settings/datatable_roles:
|
||||
get:
|
||||
summary: list the instance's data table roles
|
||||
summary: list the data table roles of one Windmill-managed Postgres cluster
|
||||
operationId: listInstanceDatatableRoles
|
||||
tags:
|
||||
- setting
|
||||
parameters:
|
||||
- in: query
|
||||
name: cluster
|
||||
required: false
|
||||
schema:
|
||||
$ref: "#/components/schemas/DatatableRoleCluster"
|
||||
responses:
|
||||
"200":
|
||||
description: the instance role catalog
|
||||
description: the cluster's role catalog
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
@@ -1702,7 +1708,7 @@ paths:
|
||||
items:
|
||||
$ref: "#/components/schemas/InstanceDatatableRole"
|
||||
post:
|
||||
summary: create a data table role on the instance's Postgres cluster
|
||||
summary: create a data table role on a Windmill-managed Postgres cluster
|
||||
operationId: createInstanceDatatableRole
|
||||
tags:
|
||||
- setting
|
||||
@@ -1716,6 +1722,8 @@ paths:
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
cluster:
|
||||
$ref: "#/components/schemas/DatatableRoleCluster"
|
||||
responses:
|
||||
"200":
|
||||
description: the created role
|
||||
@@ -11665,10 +11673,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
requestBody:
|
||||
description: script args
|
||||
@@ -11705,10 +11714,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
requestBody:
|
||||
description: script args
|
||||
@@ -11745,10 +11755,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
responses:
|
||||
"200":
|
||||
@@ -11771,10 +11782,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- name: poll_delay_ms
|
||||
description: delay between polling for job updates in milliseconds
|
||||
in: query
|
||||
@@ -11812,10 +11824,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- name: poll_delay_ms
|
||||
description: delay between polling for job updates in milliseconds
|
||||
in: query
|
||||
@@ -11851,10 +11864,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- name: poll_delay_ms
|
||||
description: delay between polling for job updates in milliseconds
|
||||
in: query
|
||||
@@ -11898,10 +11912,11 @@ paths:
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- $ref: "#/components/parameters/SkipPreprocessor"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- name: poll_delay_ms
|
||||
description: delay between polling for job updates in milliseconds
|
||||
in: query
|
||||
@@ -12855,15 +12870,6 @@ paths:
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
- name: kind
|
||||
description: which conversations to list - the flow editor's test chats, the deployed flow's own (the default), or both
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
enum:
|
||||
- test
|
||||
- deployed
|
||||
- all
|
||||
responses:
|
||||
"200":
|
||||
description: flow conversations list
|
||||
@@ -12874,40 +12880,6 @@ paths:
|
||||
items:
|
||||
$ref: "#/components/schemas/FlowConversation"
|
||||
|
||||
/w/{workspace}/flow_conversations/update/{conversation_id}:
|
||||
post:
|
||||
summary: rename flow conversation
|
||||
operationId: updateFlowConversation
|
||||
tags:
|
||||
- flow_conversations
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- name: conversation_id
|
||||
description: conversation id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [title]
|
||||
properties:
|
||||
title:
|
||||
type: string
|
||||
description: the chat's name
|
||||
responses:
|
||||
"200":
|
||||
description: flow conversation updated
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/flow_conversations/delete/{conversation_id}:
|
||||
delete:
|
||||
summary: delete flow conversation
|
||||
@@ -15302,10 +15274,11 @@ paths:
|
||||
schema:
|
||||
type: boolean
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
requestBody:
|
||||
description: flow args
|
||||
required: true
|
||||
@@ -15359,10 +15332,11 @@ paths:
|
||||
schema:
|
||||
type: boolean
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
requestBody:
|
||||
description: flow args
|
||||
required: true
|
||||
@@ -15850,10 +15824,11 @@ paths:
|
||||
type: boolean
|
||||
- $ref: "#/components/parameters/NewJobId"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
requestBody:
|
||||
description: preview
|
||||
@@ -15881,10 +15856,11 @@ paths:
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- name: memory_id
|
||||
description: Memory id for the flow's AI agent steps. A uuid is used as is; any other string is hashed within the workspace and flow, so the same string always names the same memory of that flow.
|
||||
description: memory ID for chat-enabled flows
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
requestBody:
|
||||
description: preview
|
||||
@@ -28731,7 +28707,7 @@ components:
|
||||
FlowConversation:
|
||||
type: object
|
||||
required:
|
||||
[id, workspace_id, flow_path, created_at, updated_at, created_by, is_test]
|
||||
[id, workspace_id, flow_path, created_at, updated_at, created_by]
|
||||
properties:
|
||||
id:
|
||||
type: string
|
||||
@@ -28758,9 +28734,6 @@ components:
|
||||
created_by:
|
||||
type: string
|
||||
description: Username who created the conversation
|
||||
is_test:
|
||||
type: boolean
|
||||
description: Started from the flow editor's test panel rather than a deployed run
|
||||
|
||||
FlowConversationMessage:
|
||||
type: object
|
||||
@@ -28800,50 +28773,6 @@ components:
|
||||
success:
|
||||
type: boolean
|
||||
description: Whether the message is a success
|
||||
tool_arguments:
|
||||
type: string
|
||||
nullable: true
|
||||
description: >-
|
||||
On a tool row, the arguments the model wrote for the call. For a script, flow or
|
||||
AI agent tool these exclude the inputs its step wires in, which only the tool's
|
||||
job holds. Null for a provider-native web search, whose query the provider does
|
||||
not return.
|
||||
tool_result:
|
||||
type: string
|
||||
nullable: true
|
||||
description: >-
|
||||
On a tool row, the text the model got back from the call, or what the call
|
||||
failed with — the row's own text names the tool rather than the reason. For a
|
||||
provider-native web search, its citations.
|
||||
reasoning:
|
||||
type: string
|
||||
nullable: true
|
||||
description: >-
|
||||
On an answer, the thinking that produced it; on a tool row, the thinking that
|
||||
led to the call. Each round's thinking is on one row. The agent job's result
|
||||
keeps the turn's thinking as a single string.
|
||||
attachments:
|
||||
type: array
|
||||
nullable: true
|
||||
description: >-
|
||||
The files a user message carried, as object-storage references: every flow
|
||||
input other than user_message that held one or a list of them, at most 20. Never
|
||||
file bytes or a presigned URL.
|
||||
items:
|
||||
type: object
|
||||
required: [input, s3]
|
||||
properties:
|
||||
input:
|
||||
type: string
|
||||
description: The flow input that held the file
|
||||
s3:
|
||||
type: string
|
||||
description: The file's key in object storage
|
||||
storage:
|
||||
type: string
|
||||
description: The secondary storage holding the file, absent for the primary one
|
||||
filename:
|
||||
type: string
|
||||
|
||||
EndpointTool:
|
||||
type: object
|
||||
@@ -33836,9 +33765,17 @@ components:
|
||||
last_setup:
|
||||
$ref: "#/components/schemas/ExternalInstancePgSetupReport"
|
||||
|
||||
DatatableRoleCluster:
|
||||
type: string
|
||||
description: >-
|
||||
The Windmill-managed Postgres cluster a data table role is a login on: Windmill's own
|
||||
(behind `instance` data tables) or the external instance cluster (behind
|
||||
`external_instance` ones). Defaults to `instance`.
|
||||
enum: [instance, external_instance]
|
||||
|
||||
InstanceDatatableRole:
|
||||
type: object
|
||||
required: [id, name, enabled]
|
||||
required: [id, name, enabled, cluster]
|
||||
properties:
|
||||
id:
|
||||
type: string
|
||||
@@ -33846,6 +33783,8 @@ components:
|
||||
type: string
|
||||
enabled:
|
||||
type: boolean
|
||||
cluster:
|
||||
$ref: "#/components/schemas/DatatableRoleCluster"
|
||||
|
||||
DatatableRoleTenants:
|
||||
type: object
|
||||
@@ -33867,8 +33806,10 @@ components:
|
||||
supported:
|
||||
type: boolean
|
||||
description: >-
|
||||
Whether this data table can be put under roles at all. Only one backed by the
|
||||
instance database can: a role is a login on that cluster.
|
||||
Whether this data table can be put under roles at all. Only one on a database Windmill
|
||||
manages can: a role is a login on that database's cluster.
|
||||
cluster:
|
||||
$ref: "#/components/schemas/DatatableRoleCluster"
|
||||
permissioned:
|
||||
type: boolean
|
||||
default_role:
|
||||
@@ -34168,7 +34109,7 @@ components:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: Workspaces that reference this database through a ducklake catalog or a datatable database of the kind being listed — 'instance' for the instance databases endpoint, 'external_instance' for the external cluster one. Computed at request time, not persisted, and only returned to superadmins.
|
||||
description: Workspaces that reference this database via a ducklake catalog or datatable database with resource_type 'instance'. Computed at request time, not persisted.
|
||||
workspace_id:
|
||||
type: string
|
||||
description: The workspace a member created this database for as a fork copy. Only that workspace can import into it or point a fork at it.
|
||||
@@ -36145,6 +36086,7 @@ components:
|
||||
- postgresql
|
||||
- mysql
|
||||
- instance
|
||||
- external_instance
|
||||
resource_path:
|
||||
type: string
|
||||
required:
|
||||
|
||||
@@ -4310,11 +4310,11 @@ async fn execute_component(
|
||||
}
|
||||
}
|
||||
|
||||
let flow_path = payload
|
||||
let is_flow = payload
|
||||
.path
|
||||
.as_deref()
|
||||
.and_then(|path| path.strip_prefix("flow/"))
|
||||
.map(str::to_string);
|
||||
.as_ref()
|
||||
.map(|p| p.starts_with("flow/"))
|
||||
.unwrap_or(false);
|
||||
|
||||
// Tag for inline-script jobs is read from the deployed policy in run mode;
|
||||
// only preview mode (editor) honors the client-supplied tag. This applies to
|
||||
@@ -4444,9 +4444,8 @@ async fn execute_component(
|
||||
|
||||
// Apply runnable query parameters if provided
|
||||
if let Some(ref run_query) = payload.run_query_params {
|
||||
if let Some(flow_path) = flow_path.as_deref() {
|
||||
crate::jobs::process_flow_run_query_params(&mut tx, uuid, &w_id, flow_path, run_query)
|
||||
.await?;
|
||||
if is_flow {
|
||||
crate::jobs::process_flow_run_query_params(&mut tx, uuid, run_query).await?;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -4369,12 +4369,12 @@ async fn count_completed_jobs_detail(
|
||||
Query(query): Query<CountCompletedJobsQuery>,
|
||||
) -> error::JsonResult<i64> {
|
||||
let mut sqlb = SqlBuilder::select_from("v2_job_completed");
|
||||
//FOR RLS
|
||||
sqlb.join("v2_job USING (id)");
|
||||
sqlb.field("COUNT(*) as count");
|
||||
|
||||
// Filtering on v2_job.workspace_id instead would keep the planner off
|
||||
// ix_job_workspace_id_completed_at_all and scan the whole retention window.
|
||||
if !(w_id == "admins" && query.all_workspaces.unwrap_or(false)) {
|
||||
sqlb.and_where_eq("v2_job_completed.workspace_id", "?".bind(&w_id));
|
||||
sqlb.and_where_eq("v2_job.workspace_id", "?".bind(&w_id));
|
||||
}
|
||||
|
||||
if let Some(after_s_ago) = query.completed_after_s_ago {
|
||||
@@ -4393,7 +4393,6 @@ async fn count_completed_jobs_detail(
|
||||
}
|
||||
|
||||
if let Some(tags) = query.tags {
|
||||
sqlb.join("v2_job USING (id)");
|
||||
sqlb.and_where_in(
|
||||
"v2_job.tag",
|
||||
&tags.split(",").map(|t| quote(t)).collect::<Vec<_>>(),
|
||||
@@ -4401,19 +4400,7 @@ async fn count_completed_jobs_detail(
|
||||
}
|
||||
|
||||
let sql = sqlb.sql()?;
|
||||
let mut tx = db.begin().await?;
|
||||
set_list_jobs_statement_timeout(&mut tx).await?;
|
||||
let stats = sqlx::query_scalar::<_, i64>(&sql)
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
list_jobs_timeout_error(
|
||||
e,
|
||||
"Counting completed jobs",
|
||||
"Lower completed_after_s_ago or narrow the filters.",
|
||||
)
|
||||
})?;
|
||||
tx.commit().await?;
|
||||
let stats = sqlx::query_scalar::<_, i64>(&sql).fetch_one(&db).await?;
|
||||
|
||||
Ok(Json(stats))
|
||||
}
|
||||
@@ -4442,33 +4429,6 @@ lazy_static::lazy_static! {
|
||||
.unwrap_or(30);
|
||||
}
|
||||
|
||||
/// A client that gives up does not cancel its query, so without this bound every retry of a
|
||||
/// slow filter stacks another scan running until the connection-wide 5min timeout.
|
||||
async fn set_list_jobs_statement_timeout(tx: &mut Transaction<'_, Postgres>) -> error::Result<()> {
|
||||
let timeout_secs = *LIST_JOBS_STATEMENT_TIMEOUT_SECS;
|
||||
if timeout_secs > 0 {
|
||||
sqlx::query(&format!("SET LOCAL statement_timeout = '{timeout_secs}s'"))
|
||||
.execute(&mut **tx)
|
||||
.await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn list_jobs_timeout_error(e: sqlx::Error, action: &str, hint: &str) -> Error {
|
||||
let timeout_secs = *LIST_JOBS_STATEMENT_TIMEOUT_SECS;
|
||||
match e {
|
||||
sqlx::Error::Database(ref db_err)
|
||||
if timeout_secs > 0 && db_err.code().as_deref() == Some("57014") =>
|
||||
{
|
||||
Error::Generic(
|
||||
StatusCode::BAD_REQUEST,
|
||||
format!("{action} took more than {timeout_secs}s and was stopped. {hint}"),
|
||||
)
|
||||
}
|
||||
e => e.into(),
|
||||
}
|
||||
}
|
||||
|
||||
async fn list_jobs(
|
||||
authed: ApiAuthed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
@@ -4585,14 +4545,32 @@ async fn list_jobs(
|
||||
};
|
||||
// tracing::info!("sql: {}", &sql);
|
||||
let mut tx: Transaction<'_, Postgres> = user_db.begin(&authed).await?;
|
||||
set_list_jobs_statement_timeout(&mut tx).await?;
|
||||
|
||||
// A client that gives up does not cancel its query, so without this bound every retry of a
|
||||
// slow filter stacks another scan running until the connection-wide 5min timeout.
|
||||
let timeout_secs = *LIST_JOBS_STATEMENT_TIMEOUT_SECS;
|
||||
if timeout_secs > 0 {
|
||||
sqlx::query(&format!("SET LOCAL statement_timeout = '{timeout_secs}s'"))
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
|
||||
let jobs: Vec<UnifiedJob> = sqlx::query_as(&sql)
|
||||
.fetch_all(&mut *tx)
|
||||
.warn_after_seconds_with_sql(5, format!("list_jobs: {}", sql))
|
||||
.await
|
||||
.map_err(|e| {
|
||||
list_jobs_timeout_error(e, "Listing jobs", "Set a start date or narrow the filters.")
|
||||
.map_err(|e| match e {
|
||||
sqlx::Error::Database(ref db_err)
|
||||
if timeout_secs > 0 && db_err.code().as_deref() == Some("57014") =>
|
||||
{
|
||||
Error::Generic(
|
||||
StatusCode::BAD_REQUEST,
|
||||
format!(
|
||||
"Listing jobs took more than {timeout_secs}s and was stopped. Set a start date or narrow the filters."
|
||||
),
|
||||
)
|
||||
}
|
||||
e => e.into(),
|
||||
})?;
|
||||
tx.commit().await?;
|
||||
|
||||
@@ -9568,7 +9546,7 @@ async fn run_preview_flow_job(
|
||||
.await?;
|
||||
|
||||
// Set memory_id if provided (for agent memory)
|
||||
if let Some(memory_id) = run_query.memory_key(&w_id, &flow_path) {
|
||||
if let Some(memory_id) = run_query.memory_id {
|
||||
set_flow_memory_id(&mut tx, uuid, memory_id).await?;
|
||||
}
|
||||
|
||||
@@ -9582,9 +9560,6 @@ async fn run_preview_flow_job(
|
||||
&run_query,
|
||||
user_message.as_ref(),
|
||||
uuid,
|
||||
// Run from the editor's test panel: a trial, not a real conversation.
|
||||
true,
|
||||
&flow_args,
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
|
||||
@@ -33,7 +33,6 @@ path = "src/lib.rs"
|
||||
tar.workspace = true
|
||||
hmac.workspace = true
|
||||
sha2.workspace = true
|
||||
sha1.workspace = true
|
||||
thiserror.workspace = true
|
||||
anyhow.workspace = true
|
||||
serde.workspace = true
|
||||
|
||||
@@ -6,22 +6,67 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
//! The instance's data table role catalog.
|
||||
//! The instance's data table role catalogs.
|
||||
//!
|
||||
//! A data table role is a real Postgres login role on the Windmill cluster, named exactly as the
|
||||
//! user named it, shared by every instance database. Windmill decides who may ask for a role (the
|
||||
//! per-data-table tenant lists in [`crate::workspaces`]); Postgres decides what the role may then
|
||||
//! touch. The catalog here is only the first half's vocabulary plus the cluster provisioning.
|
||||
//! A data table role is a real Postgres login role on one cluster — Windmill's own, or the external
|
||||
//! instance cluster — named exactly as the user named it, shared by every database Windmill manages
|
||||
//! on that cluster. Each cluster has its own catalog: a role exists where it was created and nowhere
|
||||
//! else. Windmill decides who may ask for a role (the per-data-table tenant lists in
|
||||
//! [`crate::workspaces`]); Postgres decides what the role may then touch. The catalog here is only
|
||||
//! the first half's vocabulary plus the cluster provisioning.
|
||||
//!
|
||||
//! Entries are keyed by a generated id so a rename moves nothing else: tenants name the id.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::{
|
||||
error::{Error, Result},
|
||||
workspaces::DataTableCatalogResourceType,
|
||||
DB,
|
||||
};
|
||||
|
||||
/// The cluster a role catalog belongs to.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum DatatableRoleCluster {
|
||||
/// Windmill's own Postgres, behind `instance` data tables.
|
||||
#[default]
|
||||
Instance,
|
||||
/// The external instance cluster, behind `external_instance` data tables.
|
||||
ExternalInstance,
|
||||
}
|
||||
|
||||
impl DatatableRoleCluster {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Instance => "instance",
|
||||
Self::ExternalInstance => "external_instance",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse(value: &str) -> Result<Self> {
|
||||
match value {
|
||||
"instance" => Ok(Self::Instance),
|
||||
"external_instance" => Ok(Self::ExternalInstance),
|
||||
other => Err(Error::BadRequest(format!(
|
||||
"Unknown data table role cluster '{other}': expected instance or external_instance"
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
/// The cluster whose roles a data table on `kind` can use. `None` for a resource-backed one,
|
||||
/// which is never under roles.
|
||||
pub fn of(kind: DataTableCatalogResourceType) -> Option<Self> {
|
||||
match kind {
|
||||
DataTableCatalogResourceType::Instance => Some(Self::Instance),
|
||||
DataTableCatalogResourceType::ExternalInstance => Some(Self::ExternalInstance),
|
||||
DataTableCatalogResourceType::Postgresql => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The connection every data table resolved to before roles existed (`custom_instance_user`). It
|
||||
/// owns every pre-existing object, so it is a reserved name rather than a catalog entry: never
|
||||
/// created, renamed or dropped.
|
||||
@@ -164,28 +209,42 @@ pub async fn lock_instance_databases_governance<'a>(
|
||||
/// need the names — but callers MUST NOT let `pwd` reach a response, a log line, an audit record
|
||||
/// or an export. Nothing about who may call it: the credential is the whole risk, and `Debug` is
|
||||
/// hand-written to redact it for the same reason.
|
||||
pub async fn read_role_catalog(db: &DB) -> Result<DatatableRoleCatalog> {
|
||||
crate::datatable_roles_oss::read_role_catalog(db).await
|
||||
pub async fn read_role_catalog(
|
||||
db: &DB,
|
||||
cluster: DatatableRoleCluster,
|
||||
) -> Result<DatatableRoleCatalog> {
|
||||
crate::datatable_roles_oss::read_role_catalog(db, cluster).await
|
||||
}
|
||||
|
||||
/// As [`read_role_catalog`], reading inside the caller's transaction so the value is the one
|
||||
/// [`lock_role_catalog`] is protecting. Same disclosure contract.
|
||||
pub async fn read_role_catalog_tx(
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
cluster: DatatableRoleCluster,
|
||||
) -> Result<DatatableRoleCatalog> {
|
||||
crate::datatable_roles_oss::read_role_catalog_tx(tx).await
|
||||
crate::datatable_roles_oss::read_role_catalog_tx(tx, cluster).await
|
||||
}
|
||||
|
||||
/// Record a role, in the caller's transaction so it commits with the `CREATE ROLE` it describes.
|
||||
/// The cluster a role belongs to, or `None` if no role has this id.
|
||||
pub async fn role_cluster(
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
id: &str,
|
||||
) -> Result<Option<DatatableRoleCluster>> {
|
||||
crate::datatable_roles_oss::role_cluster(tx, id).await
|
||||
}
|
||||
|
||||
/// Record a role, in the caller's transaction. On Windmill's own cluster that commits it with the
|
||||
/// `CREATE ROLE` it describes; on the external cluster the role already exists by then.
|
||||
///
|
||||
/// Authorization: writes a generated Postgres credential. Callers MUST restrict this to superadmin
|
||||
/// paths and MUST hold [`lock_role_catalog`] on `tx`.
|
||||
pub async fn insert_role_catalog_entry(
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
id: &str,
|
||||
cluster: DatatableRoleCluster,
|
||||
role: &InstanceDatatableRole,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::insert_role_catalog_entry(tx, id, role).await
|
||||
crate::datatable_roles_oss::insert_role_catalog_entry(tx, id, cluster, role).await
|
||||
}
|
||||
|
||||
/// Update a role's recorded name, login flag and password. Same contract as
|
||||
@@ -215,7 +274,7 @@ pub fn role_id_by_name<'a>(catalog: &'a DatatableRoleCatalog, name: &str) -> Res
|
||||
.find(|(_, role)| role.name == name)
|
||||
.ok_or_else(|| {
|
||||
Error::NotFound(format!(
|
||||
"'{name}' is not a data table role of this instance. Defined roles: {}.",
|
||||
"'{name}' is not a data table role of this database's cluster. Defined roles: {}.",
|
||||
catalog
|
||||
.values()
|
||||
.map(|r| r.name.as_str())
|
||||
@@ -231,70 +290,90 @@ pub fn role_id_by_name<'a>(catalog: &'a DatatableRoleCatalog, name: &str) -> Res
|
||||
Ok(entry.0.as_str())
|
||||
}
|
||||
|
||||
/// Every instance database the registry knows about. Role provisioning has to reach all of them:
|
||||
/// a role that cannot `CONNECT` to a database is refused by Postgres before any grant matters.
|
||||
/// Every database Windmill manages on `cluster`. Role provisioning has to reach all of them: a role
|
||||
/// that cannot `CONNECT` to a database is refused by Postgres before any grant matters.
|
||||
///
|
||||
/// Authorization: checks nothing, and names every instance database across all workspaces. Callers
|
||||
/// Authorization: checks nothing, and names every managed database across all workspaces. Callers
|
||||
/// MUST be superadmin-gated or keep the names server-side; never return them to a workspace caller.
|
||||
pub async fn registered_instance_databases(db: &DB) -> Result<Vec<String>> {
|
||||
crate::datatable_roles_oss::registered_instance_databases(db).await
|
||||
pub async fn registered_instance_databases(
|
||||
db: &DB,
|
||||
cluster: DatatableRoleCluster,
|
||||
) -> Result<Vec<String>> {
|
||||
crate::datatable_roles_oss::registered_instance_databases(db, cluster).await
|
||||
}
|
||||
|
||||
/// `CONNECT` on `dbname` for every enabled role, and none for `PUBLIC`. Run at role creation, at
|
||||
/// database creation, and lazily whenever an instance data table is administered, so a database
|
||||
/// provisioned before a role existed is repaired rather than left silently unreachable.
|
||||
/// `CONNECT` on `dbname` for every enabled role of `cluster`, and none for `PUBLIC`. Run at role
|
||||
/// creation, at database creation, and lazily whenever a managed data table is administered, so a
|
||||
/// database provisioned before a role existed is repaired rather than left silently unreachable.
|
||||
///
|
||||
/// Authorization: rewrites a database's ACL with the server's own credentials and checks nothing.
|
||||
/// Callers MUST have authorized administration of `dbname` — superadmin, or an admin of the
|
||||
/// workspace governing a data table on it.
|
||||
pub async fn converge_connect_grants(db: &DB, dbname: &str) -> Result<()> {
|
||||
crate::datatable_roles_oss::converge_connect_grants(db, dbname).await
|
||||
pub async fn converge_connect_grants(
|
||||
db: &DB,
|
||||
cluster: DatatableRoleCluster,
|
||||
dbname: &str,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::converge_connect_grants(db, cluster, dbname).await
|
||||
}
|
||||
|
||||
/// As [`converge_connect_grants`], with a catalog the caller already read. Same contract.
|
||||
/// As [`converge_connect_grants`], with the catalog of `cluster` the caller already read. Same
|
||||
/// contract.
|
||||
pub async fn converge_connect_grants_with(
|
||||
db: &DB,
|
||||
cluster: DatatableRoleCluster,
|
||||
dbname: &str,
|
||||
catalog: &DatatableRoleCatalog,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::converge_connect_grants_with(db, dbname, catalog).await
|
||||
crate::datatable_roles_oss::converge_connect_grants_with(db, cluster, dbname, catalog).await
|
||||
}
|
||||
|
||||
/// `CREATE ROLE <name> LOGIN PASSWORD ...; GRANT <name> TO custom_instance_user`, and `CONNECT` on
|
||||
/// every registered database. No privileges beyond that — an admin grants them through SQL or the
|
||||
/// ACL editor.
|
||||
/// `CREATE ROLE <name> LOGIN PASSWORD ...; GRANT <name> TO custom_instance_user` on `cluster`. No
|
||||
/// privileges beyond that — an admin grants them through SQL or the ACL editor.
|
||||
///
|
||||
/// On Windmill's own cluster the DDL runs on `tx`, so it commits with the catalog row. The external
|
||||
/// cluster is another server: the role is created there before `tx` commits, and callers MUST drop
|
||||
/// it again ([`drop_datatable_role`]) if `tx` then fails to commit.
|
||||
///
|
||||
/// Authorization: creates a cluster-wide Postgres login. Callers MUST restrict this to superadmin
|
||||
/// paths, and MUST hold [`lock_role_catalog`] on the same transaction.
|
||||
pub async fn create_instance_role(
|
||||
/// paths, and MUST hold [`lock_role_catalog`] on `tx`.
|
||||
pub async fn create_datatable_role(
|
||||
db: &DB,
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
cluster: DatatableRoleCluster,
|
||||
name: &str,
|
||||
password: &str,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::create_instance_role(tx, name, password).await
|
||||
crate::datatable_roles_oss::create_datatable_role(db, tx, cluster, name, password).await
|
||||
}
|
||||
|
||||
/// Authorization: alters a cluster-wide Postgres login. Callers MUST restrict this to superadmin
|
||||
/// paths, and MUST hold [`lock_role_catalog`] on the same transaction.
|
||||
pub async fn set_instance_role_login(
|
||||
/// paths, and MUST hold [`lock_role_catalog`] on `tx`.
|
||||
pub async fn set_datatable_role_login(
|
||||
db: &DB,
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
cluster: DatatableRoleCluster,
|
||||
name: &str,
|
||||
enabled: bool,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::set_instance_role_login(tx, name, enabled).await
|
||||
crate::datatable_roles_oss::set_datatable_role_login(db, tx, cluster, name, enabled).await
|
||||
}
|
||||
|
||||
/// A rename discards an md5-hashed password, so the caller has to hand over a fresh one.
|
||||
/// A rename discards an md5-hashed password, so the caller has to hand over a fresh one. On the
|
||||
/// external cluster the rename lands before `tx` commits, and callers MUST rename it back if `tx`
|
||||
/// then fails to commit.
|
||||
///
|
||||
/// Authorization: renames a cluster-wide Postgres login. Callers MUST restrict this to superadmin
|
||||
/// paths, and MUST hold [`lock_role_catalog`] on the same transaction.
|
||||
pub async fn rename_instance_role(
|
||||
/// paths, and MUST hold [`lock_role_catalog`] on `tx`.
|
||||
pub async fn rename_datatable_role(
|
||||
db: &DB,
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
cluster: DatatableRoleCluster,
|
||||
from: &str,
|
||||
to: &str,
|
||||
password: &str,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::rename_instance_role(tx, from, to, password).await
|
||||
crate::datatable_roles_oss::rename_datatable_role(db, tx, cluster, from, to, password).await
|
||||
}
|
||||
|
||||
/// A role owning anything in any database blocks its own `DROP ROLE`, and both its objects and the
|
||||
@@ -302,8 +381,9 @@ pub async fn rename_instance_role(
|
||||
/// registry. An unreachable database aborts the whole delete: dropping the role while one database
|
||||
/// still holds objects owned by it leaves those objects owned by a numeric OID nobody can name.
|
||||
///
|
||||
/// Each pass runs as the instance's own Postgres user rather than `custom_instance_user`, which
|
||||
/// owns the databases and can therefore revoke a grant whoever made it. `custom_instance_user`
|
||||
/// Each pass runs as the cluster's administrator rather than `custom_instance_user`: on Windmill's
|
||||
/// own cluster the instance's Postgres user, on the external one its configured admin login. Both
|
||||
/// own the databases and can therefore revoke a grant whoever made it. `custom_instance_user`
|
||||
/// could only undo what it granted itself, so a privilege planted by an operator in psql — the
|
||||
/// ordinary way privileges reach a role — would survive and block the drop.
|
||||
///
|
||||
@@ -311,16 +391,19 @@ pub async fn rename_instance_role(
|
||||
/// MUST restrict this to superadmin paths, and MUST hold [`lock_role_catalog`] on `tx`.
|
||||
///
|
||||
/// The per-database passes open their own connections and cannot join `tx`; the lock is what keeps
|
||||
/// a concurrent mutation out while they run. Only the final `DROP ROLE` is on `tx`, so it commits
|
||||
/// or rolls back with the catalog write that forgets the role. Those passes commit as they go, so
|
||||
/// callers MUST have disabled the role in an earlier committed transaction: a failure part-way
|
||||
/// then leaves a disabled role to retry, not an enabled one already stripped in some databases.
|
||||
pub async fn drop_instance_role(
|
||||
/// a concurrent mutation out while they run. On Windmill's own cluster only the final `DROP ROLE`
|
||||
/// is on `tx`, so it commits or rolls back with the catalog write that forgets the role; on the
|
||||
/// external cluster it runs there, and tolerates a role already gone so a retry after a failed
|
||||
/// commit can finish. The passes commit as they go, so callers MUST have disabled the role in an
|
||||
/// earlier committed transaction: a failure part-way then leaves a disabled role to retry, not an
|
||||
/// enabled one already stripped in some databases.
|
||||
pub async fn drop_datatable_role(
|
||||
db: &DB,
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
cluster: DatatableRoleCluster,
|
||||
name: &str,
|
||||
) -> Result<()> {
|
||||
crate::datatable_roles_oss::drop_instance_role(db, tx, name).await
|
||||
crate::datatable_roles_oss::drop_datatable_role(db, tx, cluster, name).await
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -24,12 +24,12 @@ pub fn datatable_roles_unavailable() -> Error {
|
||||
#[cfg(all(feature = "private", feature = "enterprise"))]
|
||||
pub(crate) use crate::datatable_roles_ee::{
|
||||
can_use_datatable_role, can_use_datatable_role_in_governing_workspace, converge_connect_grants,
|
||||
converge_connect_grants_with, create_instance_role, delete_role_catalog_entry,
|
||||
drop_instance_role, ensure_can_use_datatable_role, ensure_datatable_admin_access,
|
||||
converge_connect_grants_with, create_datatable_role, delete_role_catalog_entry,
|
||||
drop_datatable_role, ensure_can_use_datatable_role, ensure_datatable_admin_access,
|
||||
ensure_instance_db_grant_options_unchecked, forget_datatable_role_everywhere,
|
||||
insert_role_catalog_entry, read_role_catalog, read_role_catalog_tx,
|
||||
registered_instance_databases, rename_instance_role, resolve_datatable_role_connection,
|
||||
set_instance_role_login, update_role_catalog_entry,
|
||||
registered_instance_databases, rename_datatable_role, resolve_datatable_role_connection,
|
||||
role_cluster, set_datatable_role_login, update_role_catalog_entry,
|
||||
};
|
||||
|
||||
#[cfg(not(all(feature = "private", feature = "enterprise")))]
|
||||
@@ -39,7 +39,7 @@ pub(crate) use ce::*;
|
||||
mod ce {
|
||||
use super::datatable_roles_unavailable as unavailable;
|
||||
use crate::{
|
||||
datatable_roles::{DatatableRoleCatalog, InstanceDatatableRole},
|
||||
datatable_roles::{DatatableRoleCatalog, DatatableRoleCluster, InstanceDatatableRole},
|
||||
db::AuthedRef,
|
||||
error::Result,
|
||||
workspaces::{
|
||||
@@ -50,17 +50,31 @@ mod ce {
|
||||
|
||||
type Tx<'a> = sqlx::Transaction<'a, sqlx::Postgres>;
|
||||
|
||||
pub(crate) async fn read_role_catalog(_db: &DB) -> Result<DatatableRoleCatalog> {
|
||||
pub(crate) async fn read_role_catalog(
|
||||
_db: &DB,
|
||||
_cluster: DatatableRoleCluster,
|
||||
) -> Result<DatatableRoleCatalog> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn read_role_catalog_tx(_tx: &mut Tx<'_>) -> Result<DatatableRoleCatalog> {
|
||||
pub(crate) async fn read_role_catalog_tx(
|
||||
_tx: &mut Tx<'_>,
|
||||
_cluster: DatatableRoleCluster,
|
||||
) -> Result<DatatableRoleCatalog> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn role_cluster(
|
||||
_tx: &mut Tx<'_>,
|
||||
_id: &str,
|
||||
) -> Result<Option<DatatableRoleCluster>> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn insert_role_catalog_entry(
|
||||
_tx: &mut Tx<'_>,
|
||||
_id: &str,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_role: &InstanceDatatableRole,
|
||||
) -> Result<()> {
|
||||
Err(unavailable())
|
||||
@@ -78,43 +92,57 @@ mod ce {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn registered_instance_databases(_db: &DB) -> Result<Vec<String>> {
|
||||
pub(crate) async fn registered_instance_databases(
|
||||
_db: &DB,
|
||||
_cluster: DatatableRoleCluster,
|
||||
) -> Result<Vec<String>> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
/// Nothing to converge: with no roles to admit, an instance database keeps the `CONNECT`
|
||||
/// grants it was created with, `PUBLIC`'s included, as it did before roles existed.
|
||||
pub(crate) async fn converge_connect_grants(_db: &DB, _dbname: &str) -> Result<()> {
|
||||
/// Nothing to converge: with no roles to admit, a managed database keeps the `CONNECT` grants
|
||||
/// it was created with, as it did before roles existed.
|
||||
pub(crate) async fn converge_connect_grants(
|
||||
_db: &DB,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_dbname: &str,
|
||||
) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// As [`converge_connect_grants`].
|
||||
pub(crate) async fn converge_connect_grants_with(
|
||||
_db: &DB,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_dbname: &str,
|
||||
_catalog: &DatatableRoleCatalog,
|
||||
) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) async fn create_instance_role(
|
||||
pub(crate) async fn create_datatable_role(
|
||||
_db: &DB,
|
||||
_tx: &mut Tx<'_>,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_name: &str,
|
||||
_password: &str,
|
||||
) -> Result<()> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn set_instance_role_login(
|
||||
pub(crate) async fn set_datatable_role_login(
|
||||
_db: &DB,
|
||||
_tx: &mut Tx<'_>,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_name: &str,
|
||||
_enabled: bool,
|
||||
) -> Result<()> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn rename_instance_role(
|
||||
pub(crate) async fn rename_datatable_role(
|
||||
_db: &DB,
|
||||
_tx: &mut Tx<'_>,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_from: &str,
|
||||
_to: &str,
|
||||
_password: &str,
|
||||
@@ -122,12 +150,18 @@ mod ce {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn drop_instance_role(_db: &DB, _tx: &mut Tx<'_>, _name: &str) -> Result<()> {
|
||||
pub(crate) async fn drop_datatable_role(
|
||||
_db: &DB,
|
||||
_tx: &mut Tx<'_>,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_name: &str,
|
||||
) -> Result<()> {
|
||||
Err(unavailable())
|
||||
}
|
||||
|
||||
pub(crate) async fn ensure_instance_db_grant_options_unchecked(
|
||||
_db: &DB,
|
||||
_cluster: DatatableRoleCluster,
|
||||
_dbname: &str,
|
||||
) -> Result<()> {
|
||||
Err(unavailable())
|
||||
|
||||
@@ -148,7 +148,10 @@ pub async fn external_instance_databases(db: &DB) -> Result<BTreeMap<String, Cus
|
||||
Ok(read_external_instance_pg_state(db).await?.databases)
|
||||
}
|
||||
|
||||
/// The workspaces whose data tables name each database on the external cluster.
|
||||
/// The workspaces whose data tables or Ducklake catalogs name each database on the external cluster,
|
||||
/// and the forks whose Ducklake metadata schemas there are still waiting to be dropped: those rows
|
||||
/// outlive a settings change, and cleanup cannot drop a schema in a database that is gone. A row
|
||||
/// whose schema is already dropped only waits on object storage, which needs no database.
|
||||
///
|
||||
/// Authorization: reads every workspace's settings and checks nothing. Callers MUST be superadmin
|
||||
/// or an internal lifecycle path.
|
||||
@@ -164,7 +167,21 @@ pub async fn external_instance_database_usages<'c>(
|
||||
ELSE '{}'::jsonb END
|
||||
) AS dt(k, entry)
|
||||
WHERE entry->'database'->>'resource_type' = 'external_instance'
|
||||
AND entry->'database'->>'resource_path' IS NOT NULL",
|
||||
AND entry->'database'->>'resource_path' IS NOT NULL
|
||||
UNION ALL
|
||||
SELECT ws.workspace_id, entry->'catalog'->>'resource_path'
|
||||
FROM workspace_settings ws
|
||||
CROSS JOIN LATERAL jsonb_each(
|
||||
CASE WHEN jsonb_typeof(ws.ducklake->'ducklakes') = 'object'
|
||||
THEN ws.ducklake->'ducklakes'
|
||||
ELSE '{}'::jsonb END
|
||||
) AS dl(k, entry)
|
||||
WHERE entry->'catalog'->>'resource_type' = 'external_instance'
|
||||
AND entry->'catalog'->>'resource_path' IS NOT NULL
|
||||
UNION ALL
|
||||
SELECT workspace_id, substring(catalog FROM length('external_instance:') + 1)
|
||||
FROM fork_ducklake_namespace
|
||||
WHERE catalog LIKE 'external\\_instance:%' AND NOT schema_dropped",
|
||||
)
|
||||
.fetch_all(db)
|
||||
.await?;
|
||||
@@ -175,27 +192,48 @@ pub async fn external_instance_database_usages<'c>(
|
||||
Ok(usages)
|
||||
}
|
||||
|
||||
/// Refuse to unset the cluster while Windmill still has databases on it, or a workspace still
|
||||
/// points at one: every data table there would stop resolving. Allowed on every edition, so a
|
||||
/// Refuse to unset the cluster while Windmill still has databases or data table roles on it, or a
|
||||
/// workspace still points at one: every data table there would stop resolving, and every role
|
||||
/// would be a login nothing can drop any more. Allowed on every edition, so a
|
||||
/// downgraded instance can still clear a setting it no longer uses.
|
||||
pub async fn ensure_external_instance_pg_removable(conn: &mut sqlx::PgConnection) -> Result<()> {
|
||||
let state = read_external_instance_pg_state(&mut *conn).await?;
|
||||
let usages = external_instance_database_usages(&mut *conn).await?;
|
||||
if state.databases.is_empty() && usages.is_empty() {
|
||||
pub async fn ensure_external_instance_pg_removable(db: &DB) -> Result<()> {
|
||||
ensure_external_instance_pg_unused(db, &format!("removing {EXTERNAL_INSTANCE_PG_SETTING}"))
|
||||
.await
|
||||
}
|
||||
|
||||
/// Refuse while Windmill has databases or data table roles on the cluster, or a workspace points
|
||||
/// at one of its databases. `before` finishes the sentence saying what to do first.
|
||||
async fn ensure_external_instance_pg_unused(db: &DB, before: &str) -> Result<()> {
|
||||
let state = read_external_instance_pg_state(db).await?;
|
||||
let usages = external_instance_database_usages(db).await?;
|
||||
let roles = sqlx::query_scalar::<_, String>(
|
||||
"SELECT name FROM datatable_role WHERE cluster = 'external_instance' ORDER BY name",
|
||||
)
|
||||
.fetch_all(db)
|
||||
.await?;
|
||||
if state.databases.is_empty() && usages.is_empty() && roles.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let names = state
|
||||
.databases
|
||||
.keys()
|
||||
.chain(usages.keys())
|
||||
.collect::<BTreeSet<_>>()
|
||||
.into_iter()
|
||||
.cloned()
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
let mut held = vec![];
|
||||
if !(state.databases.is_empty() && usages.is_empty()) {
|
||||
let names = state
|
||||
.databases
|
||||
.keys()
|
||||
.chain(usages.keys())
|
||||
.collect::<BTreeSet<_>>()
|
||||
.into_iter()
|
||||
.cloned()
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
held.push(format!("databases in use ({names})"));
|
||||
}
|
||||
if !roles.is_empty() {
|
||||
held.push(format!("data table roles ({})", roles.join(", ")));
|
||||
}
|
||||
Err(Error::BadRequest(format!(
|
||||
"The external instance cluster still holds databases in use ({names}). Drop them and \
|
||||
repoint the data tables using them before removing {EXTERNAL_INSTANCE_PG_SETTING}."
|
||||
"The external instance cluster still holds {}. Drop them and repoint the data tables and \
|
||||
Ducklake catalogs using them before {before}.",
|
||||
held.join(" and ")
|
||||
)))
|
||||
}
|
||||
|
||||
@@ -270,9 +308,6 @@ pub async fn lock_external_instance_pg_state(
|
||||
/// Refuse a data table naming `dbname` unless Windmill created it on the external cluster. Takes
|
||||
/// the lock drops take, so none can remove the database before `tx`, which saves the data table,
|
||||
/// commits.
|
||||
///
|
||||
/// Authorization: its refusal says whether Windmill created a database of that name, which is
|
||||
/// instance-wide knowledge. Callers MUST have authorized the caller as superadmin first.
|
||||
pub async fn ensure_external_instance_database_registered(
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
dbname: &str,
|
||||
@@ -312,12 +347,9 @@ pub async fn write_external_instance_pg_setting(
|
||||
};
|
||||
let mut tx = db.begin().await?;
|
||||
lock_external_instance_pg_state(&mut tx).await?;
|
||||
// Every check runs on this transaction's own connection: it holds the advisory lock, and
|
||||
// taking a second connection from the pool while other writers queue on that lock is how a
|
||||
// small pool deadlocks.
|
||||
match value {
|
||||
None => {
|
||||
ensure_external_instance_pg_removable(&mut tx).await?;
|
||||
ensure_external_instance_pg_removable(db).await?;
|
||||
sqlx::query("DELETE FROM global_settings WHERE name = $1")
|
||||
.bind(EXTERNAL_INSTANCE_PG_SETTING)
|
||||
.execute(&mut *tx)
|
||||
@@ -325,7 +357,7 @@ pub async fn write_external_instance_pg_setting(
|
||||
}
|
||||
Some(value) => {
|
||||
crate::external_instance_pg_oss::validate_external_instance_pg_setting(value)?;
|
||||
ensure_external_instance_pg_not_repointed(&mut tx, value).await?;
|
||||
ensure_external_instance_pg_not_repointed(db, value).await?;
|
||||
sqlx::query(
|
||||
"INSERT INTO global_settings (name, value) VALUES ($1, $2)
|
||||
ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value, updated_at = now()",
|
||||
@@ -367,14 +399,15 @@ pub async fn write_external_instance_pg_from_diff(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Refuse pointing the setting at another host or port while databases live on the current one.
|
||||
/// Data tables name databases, not clusters, so they would silently resolve to whatever the new
|
||||
/// cluster holds under the same names. Other fields (admin login, sslmode) may change freely.
|
||||
/// Refuse pointing the setting at another host or port while databases or data table roles live on
|
||||
/// the current one. Data tables name databases, and the role catalog names logins, not clusters, so
|
||||
/// both would silently resolve to whatever the new cluster holds under the same names. Other fields
|
||||
/// (admin login, sslmode) may change freely.
|
||||
async fn ensure_external_instance_pg_not_repointed(
|
||||
conn: &mut sqlx::PgConnection,
|
||||
db: &DB,
|
||||
value: &serde_json::Value,
|
||||
) -> Result<()> {
|
||||
let Some(current) = read_external_instance_pg_config(&mut *conn).await? else {
|
||||
let Some(current) = read_external_instance_pg_config(db).await? else {
|
||||
return Ok(());
|
||||
};
|
||||
let Ok(desired) = serde_json::from_value::<ExternalInstancePg>(value.clone()) else {
|
||||
@@ -383,17 +416,11 @@ async fn ensure_external_instance_pg_not_repointed(
|
||||
if external_instance_pg_address(¤t) == external_instance_pg_address(&desired) {
|
||||
return Ok(());
|
||||
}
|
||||
let state = read_external_instance_pg_state(&mut *conn).await?;
|
||||
let usages = external_instance_database_usages(&mut *conn).await?;
|
||||
if state.databases.is_empty() && usages.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
Err(Error::BadRequest(format!(
|
||||
"The external instance cluster at {}:{} still holds databases in use. Drop them and repoint \
|
||||
what uses them before pointing {EXTERNAL_INSTANCE_PG_SETTING} at another cluster.",
|
||||
current.host.trim(),
|
||||
current.port.unwrap_or(5432)
|
||||
)))
|
||||
ensure_external_instance_pg_unused(
|
||||
db,
|
||||
&format!("pointing {EXTERNAL_INSTANCE_PG_SETTING} at another cluster"),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// Converge the external cluster on the configured login: check what it can do, create or update
|
||||
|
||||
@@ -1,39 +1,12 @@
|
||||
use std::collections::HashMap;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::value::RawValue;
|
||||
use sqlx::{self, FromRow};
|
||||
use uuid::Uuid;
|
||||
use windmill_types::s3::S3Object;
|
||||
|
||||
use crate::db::DB;
|
||||
use crate::error::Result;
|
||||
use crate::utils::truncate_with_ellipsis;
|
||||
|
||||
/// Changing it detaches every memory stored under a string memory id.
|
||||
const MEMORY_ID_NAMESPACE: Uuid = Uuid::from_u128(0x6f1c2d4e_8a3b_5c7d_9e0f_1a2b3c4d5e6f);
|
||||
|
||||
/// Memory is stored and carried in `flow_status.memory_id` as a uuid, which names the same memory
|
||||
/// wherever it is passed, as a chat conversation id must. Any other string names a memory through a
|
||||
/// name-based (v5) uuid scoped to its workspace and flow, so the same key in two flows or two
|
||||
/// workspaces names two memories, and chat conversation ids stay unique across workspaces.
|
||||
pub fn memory_key(workspace_id: &str, flow_path: &str, memory_id: &str) -> Uuid {
|
||||
let memory_id = memory_id.trim();
|
||||
Uuid::parse_str(memory_id).unwrap_or_else(|_| {
|
||||
use sha1::{Digest, Sha1};
|
||||
let mut hasher = Sha1::new();
|
||||
hasher.update(MEMORY_ID_NAMESPACE.as_bytes());
|
||||
for part in [workspace_id, flow_path, memory_id] {
|
||||
hasher.update(part.as_bytes());
|
||||
hasher.update([0u8]);
|
||||
}
|
||||
let mut bytes = [0u8; 16];
|
||||
bytes.copy_from_slice(&hasher.finalize()[..16]);
|
||||
uuid::Builder::from_sha1_bytes(bytes).into_uuid()
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq, sqlx::Type)]
|
||||
#[sqlx(type_name = "MESSAGE_TYPE", rename_all = "lowercase")]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
@@ -53,12 +26,8 @@ pub struct FlowConversation {
|
||||
pub created_at: DateTime<Utc>,
|
||||
pub updated_at: DateTime<Utc>,
|
||||
pub created_by: String,
|
||||
/// Started from the flow editor's test panel rather than a deployed run.
|
||||
pub is_test: bool,
|
||||
}
|
||||
|
||||
/// `is_test` is written on insert. An existing conversation of the other kind refuses the
|
||||
/// turn, so preview and deployed runs never share one.
|
||||
pub async fn get_or_create_conversation_with_id(
|
||||
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
|
||||
w_id: &str,
|
||||
@@ -66,10 +35,9 @@ pub async fn get_or_create_conversation_with_id(
|
||||
username: &str,
|
||||
title: &str,
|
||||
conversation_id: Uuid,
|
||||
is_test: bool,
|
||||
) -> Result<FlowConversation> {
|
||||
if let Some(existing) = lock_conversation(tx, w_id, conversation_id).await? {
|
||||
return same_kind(existing, is_test);
|
||||
return Ok(existing);
|
||||
}
|
||||
|
||||
// Truncate title to 25 characters max
|
||||
@@ -79,16 +47,15 @@ pub async fn get_or_create_conversation_with_id(
|
||||
// wins, the others wait on it, do nothing, and read the row it created.
|
||||
let created = sqlx::query_as!(
|
||||
FlowConversation,
|
||||
"INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title, is_test)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
"INSERT INTO flow_conversation (id, workspace_id, flow_path, created_by, title)
|
||||
VALUES ($1, $2, $3, $4, $5)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test",
|
||||
RETURNING id, workspace_id, flow_path, title, created_at, updated_at, created_by",
|
||||
conversation_id,
|
||||
w_id,
|
||||
flow_path,
|
||||
username,
|
||||
title,
|
||||
is_test
|
||||
title
|
||||
)
|
||||
.fetch_optional(&mut **tx)
|
||||
.await?;
|
||||
@@ -96,29 +63,13 @@ pub async fn get_or_create_conversation_with_id(
|
||||
return Ok(conversation);
|
||||
}
|
||||
|
||||
// The concurrent first turn that won the insert may have been of the other kind.
|
||||
let existing = lock_conversation(tx, w_id, conversation_id)
|
||||
lock_conversation(tx, w_id, conversation_id)
|
||||
.await?
|
||||
.ok_or_else(|| {
|
||||
crate::error::Error::BadRequest(format!(
|
||||
"conversation {conversation_id} belongs to another workspace"
|
||||
))
|
||||
})?;
|
||||
same_kind(existing, is_test)
|
||||
}
|
||||
|
||||
/// `memory_id` is the caller's to choose, so a preview run could name a deployed
|
||||
/// conversation and the reverse. A conversation's kind is fixed at creation and nothing
|
||||
/// would show the mixing afterwards, so the turn is refused before it starts.
|
||||
fn same_kind(existing: FlowConversation, is_test: bool) -> Result<FlowConversation> {
|
||||
if existing.is_test == is_test {
|
||||
return Ok(existing);
|
||||
}
|
||||
Err(crate::error::Error::BadRequest(if existing.is_test {
|
||||
"this conversation was started from the flow editor's test panel; start a new conversation to run the deployed flow".to_string()
|
||||
} else {
|
||||
"this conversation belongs to the deployed flow; start a new conversation to test from the flow editor".to_string()
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
/// Locked, so a turn orders against retention collecting the conversation
|
||||
@@ -132,7 +83,7 @@ async fn lock_conversation(
|
||||
) -> Result<Option<FlowConversation>> {
|
||||
Ok(sqlx::query_as!(
|
||||
FlowConversation,
|
||||
"SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by, is_test
|
||||
"SELECT id, workspace_id, flow_path, title, created_at, updated_at, created_by
|
||||
FROM flow_conversation
|
||||
WHERE id = $1 AND workspace_id = $2
|
||||
FOR UPDATE",
|
||||
@@ -143,65 +94,6 @@ async fn lock_conversation(
|
||||
.await?)
|
||||
}
|
||||
|
||||
/// What a row carries beyond its text. A chat is rebuilt from its rows alone, without
|
||||
/// reading jobs, so every tool row carries the model's call and what the model got back: a
|
||||
/// Windmill tool's job holds the args its input transforms produced rather than the model's,
|
||||
/// and an MCP tool's call sits among every call of the turn in the agent's job. That job's
|
||||
/// `reasoning` is one string for the whole turn, where the rows keep it per iteration.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct MessageExtras {
|
||||
pub tool_arguments: Option<String>,
|
||||
pub tool_result: Option<String>,
|
||||
pub reasoning: Option<String>,
|
||||
/// The files a user message carried; see `message_attachments`.
|
||||
pub attachments: Vec<MessageAttachment>,
|
||||
}
|
||||
|
||||
/// The most files a user message keeps references to; the rest are dropped.
|
||||
pub const MAX_MESSAGE_ATTACHMENTS: usize = 20;
|
||||
|
||||
/// A file a user message carried, as the object-storage reference its run received.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct MessageAttachment {
|
||||
/// The flow input that held it.
|
||||
pub input: String,
|
||||
pub s3: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub storage: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub filename: Option<String>,
|
||||
}
|
||||
|
||||
/// The files a run's args carry for its user message: every top-level input other than
|
||||
/// `user_message` whose value is an object-storage reference or a list of them, in input
|
||||
/// name order, capped at `MAX_MESSAGE_ATTACHMENTS`. Only the reference is kept: `presigned`
|
||||
/// grants access to the file, and any other value may be the file's bytes.
|
||||
pub fn message_attachments(args: &HashMap<String, Box<RawValue>>) -> Vec<MessageAttachment> {
|
||||
let mut inputs: Vec<_> = args
|
||||
.iter()
|
||||
.filter(|(name, _)| name.as_str() != "user_message")
|
||||
.collect();
|
||||
inputs.sort_by(|a, b| a.0.cmp(b.0));
|
||||
inputs
|
||||
.into_iter()
|
||||
.flat_map(|(name, value)| {
|
||||
serde_json::from_str::<S3Object>(value.get())
|
||||
.map(|object| vec![object])
|
||||
.or_else(|_| serde_json::from_str::<Vec<S3Object>>(value.get()))
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.filter(|object| !object.s3.is_empty())
|
||||
.map(move |object| MessageAttachment {
|
||||
input: name.clone(),
|
||||
s3: object.s3,
|
||||
storage: object.storage,
|
||||
filename: object.filename,
|
||||
})
|
||||
})
|
||||
.take(MAX_MESSAGE_ATTACHMENTS)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Add a message to a conversation using an existing transaction
|
||||
/// If the conversation doesn't exist, logs a warning and returns Ok (no error thrown)
|
||||
/// This allows memory_id to be used for agent memory without requiring a conversation
|
||||
@@ -213,7 +105,6 @@ pub async fn add_message_to_conversation_tx(
|
||||
message_type: MessageType,
|
||||
step_name: Option<&str>,
|
||||
success: bool,
|
||||
extras: Option<&MessageExtras>,
|
||||
) -> Result<()> {
|
||||
// Check if conversation exists first
|
||||
let conversation_exists = sqlx::query!(
|
||||
@@ -234,21 +125,14 @@ pub async fn add_message_to_conversation_tx(
|
||||
|
||||
// Insert the message
|
||||
sqlx::query!(
|
||||
"INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success, tool_arguments, tool_result, reasoning, attachments)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)",
|
||||
"INSERT INTO flow_conversation_message (conversation_id, message_type, content, job_id, step_name, success)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)",
|
||||
conversation_id,
|
||||
message_type as MessageType,
|
||||
content,
|
||||
job_id,
|
||||
step_name,
|
||||
success,
|
||||
extras.and_then(|e| e.tool_arguments.as_deref()),
|
||||
extras.and_then(|e| e.tool_result.as_deref()),
|
||||
extras.and_then(|e| e.reasoning.as_deref()),
|
||||
extras
|
||||
.map(|e| &e.attachments)
|
||||
.filter(|attachments| !attachments.is_empty())
|
||||
.map(sqlx::types::Json) as Option<sqlx::types::Json<&Vec<MessageAttachment>>>
|
||||
success
|
||||
)
|
||||
.execute(&mut **tx)
|
||||
.await?;
|
||||
@@ -280,67 +164,3 @@ pub async fn delete_conversation_memory(
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::{json, value::to_raw_value};
|
||||
|
||||
fn args(values: serde_json::Value) -> HashMap<String, Box<RawValue>> {
|
||||
values
|
||||
.as_object()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.map(|(name, value)| (name.clone(), to_raw_value(value).unwrap()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn keeps_only_object_storage_references() {
|
||||
let attachments = message_attachments(&args(json!({
|
||||
"user_message": { "s3": "not/an/attachment.png" },
|
||||
"avatar": { "s3": "u/a.png", "storage": "secondary", "presigned": "https://signed" },
|
||||
"files": [
|
||||
{ "s3": "u/b.pdf", "filename": "b.pdf" },
|
||||
{ "s3": "" }
|
||||
],
|
||||
"photo": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==",
|
||||
"count": 3
|
||||
})));
|
||||
assert_eq!(
|
||||
serde_json::to_value(&attachments).unwrap(),
|
||||
json!([
|
||||
{ "input": "avatar", "s3": "u/a.png", "storage": "secondary" },
|
||||
{ "input": "files", "s3": "u/b.pdf", "filename": "b.pdf" }
|
||||
])
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn caps_the_references_of_one_message() {
|
||||
let files: Vec<_> = (0..25)
|
||||
.map(|i| json!({ "s3": format!("u/{i}.png") }))
|
||||
.collect();
|
||||
let attachments = message_attachments(&args(json!({ "files": files })));
|
||||
assert_eq!(attachments.len(), MAX_MESSAGE_ATTACHMENTS);
|
||||
assert_eq!(attachments.last().unwrap().s3, "u/19.png");
|
||||
}
|
||||
|
||||
/// A string names a memory only within its workspace and flow; a uuid is used as is.
|
||||
#[test]
|
||||
fn memory_key_scopes_strings_but_not_uuids() {
|
||||
let key = memory_key("ws", "f/support/triage", " customer-1 ");
|
||||
assert_eq!(key, memory_key("ws", "f/support/triage", "customer-1"));
|
||||
assert_ne!(
|
||||
key,
|
||||
memory_key("other_ws", "f/support/triage", "customer-1")
|
||||
);
|
||||
assert_ne!(key, memory_key("ws", "f/sales/triage", "customer-1"));
|
||||
let conversation = Uuid::from_u128(7).to_string();
|
||||
assert_eq!(memory_key("ws", "f/a", &conversation), Uuid::from_u128(7));
|
||||
assert_eq!(
|
||||
memory_key("other_ws", "f/b", &conversation),
|
||||
Uuid::from_u128(7)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1563,18 +1563,20 @@ pub(crate) fn instance_db_grants(dbname: &str) -> String {
|
||||
)
|
||||
}
|
||||
|
||||
/// Re-apply [`instance_db_grants`] to an instance database provisioned before data table roles
|
||||
/// existed, whose grants carry no grant option. Connects as the instance's own Postgres user —
|
||||
/// the database and `public` schema owner — since only it can hand out an option it holds.
|
||||
/// Re-apply [`instance_db_grants`] to a managed database provisioned before data table roles
|
||||
/// existed, whose grants carry no grant option. Connects as the cluster's administrator — the
|
||||
/// database and `public` schema owner — since only it can hand out an option it holds.
|
||||
///
|
||||
/// Authorization: reaches an instance database with the server's own credentials and checks
|
||||
/// Authorization: reaches a managed database with the server's own credentials and checks
|
||||
/// nothing. Callers MUST have authorized administration of `dbname` — superadmin, or an admin of
|
||||
/// the workspace governing a data table on it.
|
||||
pub async fn ensure_instance_db_grant_options_unchecked(
|
||||
db: &DB,
|
||||
cluster: crate::datatable_roles::DatatableRoleCluster,
|
||||
dbname: &str,
|
||||
) -> error::Result<()> {
|
||||
crate::datatable_roles_oss::ensure_instance_db_grant_options_unchecked(db, dbname).await
|
||||
crate::datatable_roles_oss::ensure_instance_db_grant_options_unchecked(db, cluster, dbname)
|
||||
.await
|
||||
}
|
||||
|
||||
/// Create a custom instance database: CREATE DATABASE, grant permissions, register in global_settings.
|
||||
@@ -1651,7 +1653,13 @@ pub async fn create_custom_instance_database(
|
||||
// A data table role can only reach a database it may CONNECT to, and PUBLIC's default CONNECT
|
||||
// would otherwise let every role in regardless of what this instance defines. Best-effort: a
|
||||
// failure here leaves the database usable as `admin`, and the next role change repairs it.
|
||||
if let Err(e) = crate::datatable_roles::converge_connect_grants(db, dbname).await {
|
||||
if let Err(e) = crate::datatable_roles::converge_connect_grants(
|
||||
db,
|
||||
crate::datatable_roles::DatatableRoleCluster::Instance,
|
||||
dbname,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!("Could not set CONNECT grants on instance database '{dbname}': {e}");
|
||||
}
|
||||
|
||||
@@ -1659,6 +1667,24 @@ pub async fn create_custom_instance_database(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The system's CA bundle file, for libpq clients that cannot take `sslrootcert=system`: that value
|
||||
/// needs libpq 16, and verify-full only.
|
||||
pub fn system_ca_bundle() -> Option<std::path::PathBuf> {
|
||||
std::env::var_os("SSL_CERT_FILE")
|
||||
.map(std::path::PathBuf::from)
|
||||
.into_iter()
|
||||
.chain(
|
||||
[
|
||||
"/etc/ssl/certs/ca-certificates.crt",
|
||||
"/etc/pki/tls/certs/ca-bundle.crt",
|
||||
"/etc/ssl/cert.pem",
|
||||
"/etc/ssl/ca-bundle.pem",
|
||||
]
|
||||
.map(std::path::PathBuf::from),
|
||||
)
|
||||
.find(|path| path.is_file())
|
||||
}
|
||||
|
||||
/// Refuse a workspace member writing a fork copy into, or pointing a fork at, the managed database
|
||||
/// `dbname` of `kind`, unless `w_id` created it for that ([`create_custom_instance_database`], or
|
||||
/// its external instance counterpart) and nothing uses it yet. The `wm_fork_` prefix is no
|
||||
|
||||
@@ -976,33 +976,18 @@ fn six_fields_hint(schedule_str: &str, version: Option<&str>, seconds_required:
|
||||
}
|
||||
|
||||
impl ScheduleType {
|
||||
/// `NotFound` means the expression has no run left (an expired year, an impossible
|
||||
/// date), and schedule pushes disable the schedule on it. Every other error must stay
|
||||
/// transient: croner fails across a DST jump longer than an hour (Antarctica/Troll)
|
||||
/// and succeeds again once the jump has passed.
|
||||
pub fn find_next(
|
||||
&self,
|
||||
starting_from: &chrono::DateTime<chrono_tz::Tz>,
|
||||
) -> Result<chrono::DateTime<chrono_tz::Tz>> {
|
||||
let no_run_left = || {
|
||||
Error::NotFound(format!(
|
||||
"cron: the schedule has no run left after {}",
|
||||
starting_from.format("%Y-%m-%d %H:%M:%S %Z")
|
||||
))
|
||||
};
|
||||
) -> chrono::DateTime<chrono_tz::Tz> {
|
||||
match self {
|
||||
ScheduleType::Croner(croner_schedule) => croner_schedule
|
||||
.find_next_occurrence(starting_from, false)
|
||||
.map_err(|e| match e {
|
||||
croner::errors::CronError::TimeSearchLimitExceeded => no_run_left(),
|
||||
e => Error::internal_err(format!(
|
||||
"cron: could not compute the run after {}: {e}",
|
||||
starting_from.format("%Y-%m-%d %H:%M:%S %Z")
|
||||
)),
|
||||
}),
|
||||
ScheduleType::Cron(schedule) => {
|
||||
schedule.after(starting_from).next().ok_or_else(no_run_left)
|
||||
}
|
||||
.expect("cron: a schedule should have a next event"),
|
||||
ScheduleType::Cron(schedule) => schedule
|
||||
.after(starting_from)
|
||||
.next()
|
||||
.expect("cron: a schedule should have a next event"),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1724,22 +1709,6 @@ mod tests {
|
||||
assert!(!err.contains("6 fields"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn find_next_reports_only_a_cron_with_no_run_left_as_not_found() {
|
||||
use chrono::TimeZone;
|
||||
let troll: chrono_tz::Tz = "Antarctica/Troll".parse().unwrap();
|
||||
// Troll's clocks jump from 01:00 to 03:00 on the last Sunday of March.
|
||||
let before_jump = troll.with_ymd_and_hms(2027, 3, 28, 0, 30, 0).unwrap();
|
||||
|
||||
let expired = ScheduleType::from_str("0 0 9 1 1 * 2026", Some("v1"), true).unwrap();
|
||||
let err = expired.find_next(&before_jump).unwrap_err();
|
||||
assert!(matches!(err, Error::NotFound(_)), "{err}");
|
||||
|
||||
let across_jump = ScheduleType::from_str("0 30 1 * * *", Some("v2"), true).unwrap();
|
||||
let err = across_jump.find_next(&before_jump).unwrap_err();
|
||||
assert!(!matches!(err, Error::NotFound(_)), "{err}");
|
||||
}
|
||||
|
||||
/// A worker that restarts must land on the exact same name to reclaim its `worker_ping`
|
||||
/// row, while still never colliding with the other workers of its own process. The
|
||||
/// suffix must also stay a single `-` segment, which is what the interactive shell tag
|
||||
|
||||
@@ -1551,13 +1551,14 @@ pub async fn lock_fork_datatables(conn: &mut sqlx::PgConnection, w_id: &str) ->
|
||||
}
|
||||
|
||||
impl GoverningDatatable {
|
||||
/// Backed by the Windmill instance's own Postgres, which is the only substrate data table
|
||||
/// roles apply to.
|
||||
pub fn is_instance(&self) -> bool {
|
||||
/// The Windmill-managed cluster whose data table roles this entry can use. `None` for a
|
||||
/// resource-backed one: roles are logins Windmill creates, and it creates none on a host a
|
||||
/// workspace admin chose.
|
||||
pub fn role_cluster(&self) -> Option<crate::datatable_roles::DatatableRoleCluster> {
|
||||
self.datatable
|
||||
.database
|
||||
.as_ref()
|
||||
.is_some_and(|d| d.resource_type == DataTableCatalogResourceType::Instance)
|
||||
.and_then(|d| crate::datatable_roles::DatatableRoleCluster::of(d.resource_type))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2257,6 +2258,10 @@ pub enum DucklakeCatalogResourceType {
|
||||
Postgresql,
|
||||
Mysql,
|
||||
Instance,
|
||||
/// On the external instance cluster ([`crate::external_instance_pg`]). Enterprise Edition.
|
||||
#[serde(rename = "external_instance")]
|
||||
#[strum(serialize = "external_instance")]
|
||||
ExternalInstance,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Serialize)]
|
||||
@@ -2784,7 +2789,16 @@ async fn ducklake_conn_data(
|
||||
let ducklake = serde_json::from_value::<Ducklake>(ducklake)?;
|
||||
|
||||
let catalog_resource =
|
||||
if ducklake.catalog.resource_type == DucklakeCatalogResourceType::Instance {
|
||||
if ducklake.catalog.resource_type == DucklakeCatalogResourceType::ExternalInstance {
|
||||
let pg_creds = crate::external_instance_pg::external_instance_connection_unchecked(
|
||||
db,
|
||||
&ducklake.catalog.resource_path,
|
||||
false,
|
||||
)
|
||||
.await?;
|
||||
serde_json::to_value(&pg_creds)
|
||||
.map_err(|e| Error::internal_err(format!("Error serializing pg creds: {}", e)))?
|
||||
} else if ducklake.catalog.resource_type == DucklakeCatalogResourceType::Instance {
|
||||
let mut pg_creds = PgDatabase::parse_uri(&get_database_url().await?.as_str().await)?;
|
||||
pg_creds.dbname = ducklake.catalog.resource_path.clone();
|
||||
pg_creds.user = Some("custom_instance_user".to_string());
|
||||
@@ -3165,6 +3179,14 @@ async fn register_fork_ducklake_namespace(
|
||||
{
|
||||
return Ok(());
|
||||
}
|
||||
let mut tx = db.begin().await?;
|
||||
// A row naming an external database counts as a use of it. Written under the lock a drop takes,
|
||||
// and only while the database is still registered, so a drop cannot slip in between the
|
||||
// settings this attach resolved and the row that protects the database.
|
||||
if let Some(dbname) = catalog.strip_prefix("external_instance:") {
|
||||
crate::external_instance_pg::ensure_external_instance_database_registered(&mut tx, dbname)
|
||||
.await?;
|
||||
}
|
||||
sqlx::query!(
|
||||
"INSERT INTO fork_ducklake_namespace
|
||||
(workspace_id, ducklake_name, metadata_schema, catalog, storage, storage_ref, data_path)
|
||||
@@ -3179,9 +3201,10 @@ async fn register_fork_ducklake_namespace(
|
||||
&storage_ref,
|
||||
data_path,
|
||||
)
|
||||
.execute(db)
|
||||
.execute(&mut *tx)
|
||||
.await
|
||||
.map_err(|e| Error::internal_err(format!("registering fork ducklake namespace: {e:#}")))?;
|
||||
tx.commit().await?;
|
||||
let mut locations = FORK_DUCKLAKE_REGISTERED
|
||||
.get(w_id)
|
||||
.filter(|(_, exp)| *exp > now)
|
||||
|
||||
@@ -166,10 +166,13 @@ pub async fn push_scheduled_job<'c>(
|
||||
}
|
||||
};
|
||||
|
||||
let next = sched.find_next(&starting_from)?;
|
||||
let next = sched.find_next(&starting_from);
|
||||
// println!("next event ({:?}): {}", tz, next);
|
||||
// println!("next event(UTC): {}", next.with_timezone(&chrono::Utc));
|
||||
|
||||
// Scheduled events must be stored in the database in UTC
|
||||
let next = next.with_timezone(&chrono::Utc);
|
||||
// panic!("next: {}", next);
|
||||
let already_exists: bool = sqlx::query_scalar!(
|
||||
// Query plan:
|
||||
// - use of the `ix_v2_job_root_by_path` index; hence the `parent_job IS NULL` clause.
|
||||
|
||||
@@ -921,47 +921,6 @@ mod schedule_push {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// try_schedule_next_job: a cron with no run left disables the schedule
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "schedule_push"))]
|
||||
async fn test_cron_with_no_run_left_disables_schedule(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
sqlx::query(
|
||||
"INSERT INTO schedule (workspace_id, path, edited_by, edited_at, schedule, timezone, enabled, script_path, is_flow, email, extra_perms, ws_error_handler_muted, no_flow_overlap, permissioned_as, cron_version)
|
||||
VALUES ('test-workspace', 'f/system/test_schedule', 'test-user', now(), '0 0 9 1 1 * 2020', 'UTC', true, 'f/system/test_script', false, 'test@windmill.dev', '{}', false, false, 'u/test-user', 'v1')"
|
||||
)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
let schedule = make_schedule(|s| {
|
||||
s.schedule = "0 0 9 1 1 * 2020".to_string();
|
||||
s.cron_version = Some("v1".to_string());
|
||||
});
|
||||
let job = make_completed_job(&schedule);
|
||||
|
||||
let tx = db.begin().await?;
|
||||
let (tx, err) =
|
||||
try_schedule_next_job(&db, tx, &job, &schedule, &schedule.script_path).await;
|
||||
assert!(err.is_none(), "completion must go through, got: {err:?}");
|
||||
tx.commit().await?;
|
||||
|
||||
assert_eq!(count_queued_jobs(&db).await, 0);
|
||||
let (enabled, error): (bool, Option<String>) = sqlx::query_as(
|
||||
"SELECT enabled, error FROM schedule WHERE workspace_id = 'test-workspace' AND path = 'f/system/test_schedule'",
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await?;
|
||||
assert!(!enabled, "schedule with no run left must be disabled");
|
||||
assert!(
|
||||
error.as_deref().is_some_and(|e| e.contains("no run left")),
|
||||
"error should say why, got: {error:?}"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// try_schedule_next_job: disabled schedule leaves no side effects
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
@@ -1100,8 +1100,7 @@ pub enum FlowModuleValue {
|
||||
omit_output_from_conversation: bool,
|
||||
/// When set, the agent brain config (provider/model/system prompt/etc.) and tools are
|
||||
/// resolved at runtime from this `ai_agent` resource path (hybrid linking). The module's
|
||||
/// `input_transforms` then only carry the flow-local inputs: user_message,
|
||||
/// user_attachments, enabled_tools and the history inputs memory_id and previous_messages.
|
||||
/// `input_transforms` then only carry the flow-local inputs (user_message/user_attachments).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
agent: Option<String>,
|
||||
/// Binds an agent's tools to *this* flow's context, keyed by tool id then input key, without
|
||||
|
||||
@@ -33,7 +33,7 @@ use windmill_common::{
|
||||
client::AuthedClient,
|
||||
db::DB,
|
||||
error::Error,
|
||||
flow_conversations::{MessageExtras, MessageType},
|
||||
flow_conversations::MessageType,
|
||||
flow_status::AgentAction,
|
||||
flows::FlowModuleValue,
|
||||
worker::{to_raw_value, Connection},
|
||||
@@ -74,9 +74,6 @@ pub struct ToolExecutionContext<'a> {
|
||||
pub stream_event_processor: Option<&'a StreamEventProcessor>,
|
||||
pub flow_context: &'a mut FlowContext,
|
||||
pub omit_output_from_conversation: bool,
|
||||
/// The thinking that led to this round's calls, stored on the first tool row written.
|
||||
/// None when the round wrote text, whose row carries it.
|
||||
pub reasoning: Option<String>,
|
||||
pub previous_result: &'a Option<Box<RawValue>>,
|
||||
pub id_context: &'a Option<crate::js_eval::IdContext>,
|
||||
|
||||
@@ -238,24 +235,9 @@ async fn execute_mcp_tool_call(
|
||||
update_flow_status_module_with_actions_success(ctx.db, parent_job, true).await?;
|
||||
}
|
||||
|
||||
// An MCP tool runs inside the agent's job, whose result holds every call of the
|
||||
// turn and nothing tying one of them to this row: same job id for all of them,
|
||||
// no call id on the row. Kept here so the card shows this call — and the row
|
||||
// names that job, so retention sweeps it with every other row of the turn.
|
||||
// Add tool message to conversation if chat_input_enabled
|
||||
let content = format!("Used {} tool", tool_call.function.name);
|
||||
let agent_job_id = ctx.job.id;
|
||||
add_tool_message_to_chat(
|
||||
ctx,
|
||||
Some(agent_job_id),
|
||||
&content,
|
||||
true,
|
||||
Some(MessageExtras {
|
||||
tool_arguments: Some(tool_call.function.arguments.clone()),
|
||||
tool_result: Some(result_str),
|
||||
..Default::default()
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
add_tool_message_to_chat(ctx, None, &content, true).await;
|
||||
}
|
||||
Err(e) => {
|
||||
let error_msg = format!("MCP tool error: {}", e);
|
||||
@@ -289,23 +271,8 @@ async fn execute_mcp_tool_call(
|
||||
update_flow_status_module_with_actions_success(ctx.db, parent_job, false).await?;
|
||||
}
|
||||
|
||||
// Add tool message to conversation if chat_input_enabled. The row is worded from
|
||||
// the tool, like every other tool row, and the error it failed with is its result
|
||||
// — the one field a call that produced nothing else still has something to put in.
|
||||
let agent_job_id = ctx.job.id;
|
||||
let content = format!("Error executing {}", tool_name);
|
||||
add_tool_message_to_chat(
|
||||
ctx,
|
||||
Some(agent_job_id),
|
||||
&content,
|
||||
false,
|
||||
Some(MessageExtras {
|
||||
tool_arguments: Some(tool_call.function.arguments.clone()),
|
||||
tool_result: Some(error_msg.clone()),
|
||||
..Default::default()
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
// Add tool message to conversation if chat_input_enabled
|
||||
add_tool_message_to_chat(ctx, None, &error_msg, false).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -713,8 +680,8 @@ async fn handle_tool_execution_error(
|
||||
update_flow_status_module_with_actions_success(ctx.db, parent_job, false).await?;
|
||||
}
|
||||
|
||||
let (content, extras) = windmill_tool_row(tool_call, false, &error_message);
|
||||
add_tool_message_to_chat(ctx, Some(job_id), &content, false, Some(extras)).await;
|
||||
// Add tool message to conversation if chat_input_enabled (error case)
|
||||
add_tool_message_to_chat(ctx, Some(job_id), &error_message, false).await;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -815,17 +782,13 @@ async fn handle_tool_execution_success(
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
let (content, extras) = windmill_tool_row(tool_call, success, &tool_result);
|
||||
|
||||
// The job ran; whether it ran successfully is `success`, and the row stored below is
|
||||
// worded from it. The stream has to carry the same value, or the card the reader watches
|
||||
// and the row that replaces it describe the same call differently.
|
||||
// Stream tool result (success case)
|
||||
if let Some(stream_event_processor) = ctx.stream_event_processor {
|
||||
let tool_result_event = StreamingEvent::ToolResult {
|
||||
call_id: tool_call.id.clone(),
|
||||
function_name: tool_call.function.name.clone(),
|
||||
result: tool_result,
|
||||
success,
|
||||
success: true,
|
||||
};
|
||||
stream_event_processor
|
||||
.send(tool_result_event, final_events_str)
|
||||
@@ -836,56 +799,28 @@ async fn handle_tool_execution_success(
|
||||
update_flow_status_module_with_actions_success(ctx.db, parent_job, success).await?;
|
||||
}
|
||||
|
||||
add_tool_message_to_chat(ctx, Some(job_id), &content, success, Some(extras)).await;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A Windmill tool's conversation row: worded from the tool, carrying the model's call and
|
||||
/// the exact text the model got back, the same text agent memory keeps for that tool
|
||||
/// message, so a card needs no job fetch. The call is the model's arguments, not the job's
|
||||
/// args: the step's input transforms add inputs the model never wrote.
|
||||
fn windmill_tool_row(
|
||||
tool_call: &OpenAIToolCall,
|
||||
success: bool,
|
||||
sent_to_model: &str,
|
||||
) -> (String, MessageExtras) {
|
||||
// Add tool message to conversation if chat_input_enabled
|
||||
let content = if success {
|
||||
format!("Used {} tool", tool_call.function.name)
|
||||
} else {
|
||||
format!("Error executing {}", tool_call.function.name)
|
||||
};
|
||||
let extras = MessageExtras {
|
||||
tool_arguments: Some(tool_call.function.arguments.clone()),
|
||||
tool_result: Some(sent_to_model.to_string()),
|
||||
..Default::default()
|
||||
};
|
||||
(content, extras)
|
||||
|
||||
add_tool_message_to_chat(ctx, Some(job_id), &content, success).await;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Add tool message to conversation if chat is enabled
|
||||
async fn add_tool_message_to_chat(
|
||||
ctx: &mut ToolExecutionContext<'_>,
|
||||
// The job this row belongs to: the tool's own where it has one, else the agent's, which
|
||||
// is the job it ran inside. Every row names one so that retention collects the whole
|
||||
// turn — `delete_jobs` removes messages by `job_id = ANY(..)` (there is no FK on the
|
||||
// column; `drop_v2_job_side_table_cascades` dropped it), and a row naming no job would
|
||||
// survive every purge and leave a conversation that can never become empty.
|
||||
tool_job_id: Option<Uuid>,
|
||||
content: &str,
|
||||
success: bool,
|
||||
// The model's call and what it got back; every tool row carries both.
|
||||
extras: Option<MessageExtras>,
|
||||
) {
|
||||
if ctx.omit_output_from_conversation {
|
||||
return;
|
||||
}
|
||||
let extras = match ctx.reasoning.take() {
|
||||
Some(reasoning) => {
|
||||
Some(MessageExtras { reasoning: Some(reasoning), ..extras.unwrap_or_default() })
|
||||
}
|
||||
None => extras,
|
||||
};
|
||||
|
||||
let chat_enabled = ctx
|
||||
.flow_context
|
||||
@@ -900,74 +835,41 @@ async fn add_tool_message_to_chat(
|
||||
.as_ref()
|
||||
.and_then(|fs| fs.memory_id)
|
||||
{
|
||||
let db_clone = ctx.db.clone();
|
||||
let effective_step_id = ctx
|
||||
.flow_step_id_override
|
||||
.or(ctx.job.flow_step_id.as_deref());
|
||||
let step_name = get_step_name_from_flow(ctx.summary.as_deref(), effective_step_id);
|
||||
let content = content.to_string();
|
||||
|
||||
// Awaited, not spawned: `created_seq` is the transcript's order, so a round's rows
|
||||
// must commit in the order of its calls. Calls run one after another; running them
|
||||
// in parallel would need their rows written in call order all the same.
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
ctx.db,
|
||||
&memory_id,
|
||||
tool_job_id,
|
||||
content,
|
||||
MessageType::Tool,
|
||||
&step_name,
|
||||
success,
|
||||
extras.as_ref(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add tool message to conversation {}: {}",
|
||||
memory_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
// Spawn task because we do not need to wait for the result
|
||||
tokio::spawn(async move {
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
&db_clone,
|
||||
&memory_id,
|
||||
tool_job_id,
|
||||
&content,
|
||||
MessageType::Tool,
|
||||
&step_name,
|
||||
success,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add tool message to conversation {}: {}",
|
||||
memory_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{extract_ai_agent_output, windmill_tool_row};
|
||||
use super::extract_ai_agent_output;
|
||||
use serde_json::value::RawValue;
|
||||
use windmill_ai::ai_types::{OpenAIFunction, OpenAIToolCall};
|
||||
|
||||
#[test]
|
||||
fn a_windmill_tool_row_carries_the_models_call_and_what_it_got_back() {
|
||||
let tool_call = OpenAIToolCall {
|
||||
id: "call_1".to_string(),
|
||||
function: OpenAIFunction {
|
||||
name: "get_price".to_string(),
|
||||
arguments: r#"{"item":"widget"}"#.to_string(),
|
||||
},
|
||||
r#type: "function".to_string(),
|
||||
extra_content: None,
|
||||
};
|
||||
|
||||
let (content, extras) = windmill_tool_row(&tool_call, true, r#"{"price":42}"#);
|
||||
assert_eq!(content, "Used get_price tool");
|
||||
assert_eq!(
|
||||
extras.tool_arguments.as_deref(),
|
||||
Some(r#"{"item":"widget"}"#)
|
||||
);
|
||||
assert_eq!(extras.tool_result.as_deref(), Some(r#"{"price":42}"#));
|
||||
|
||||
let (content, extras) =
|
||||
windmill_tool_row(&tool_call, false, "Error running tool: ExecutionErr: boom");
|
||||
assert_eq!(content, "Error executing get_price");
|
||||
assert_eq!(
|
||||
extras.tool_arguments.as_deref(),
|
||||
Some(r#"{"item":"widget"}"#)
|
||||
);
|
||||
assert_eq!(
|
||||
extras.tool_result.as_deref(),
|
||||
Some("Error running tool: ExecutionErr: boom")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extracts_only_the_output_of_an_agent_result() {
|
||||
|
||||
@@ -13,7 +13,7 @@ use windmill_common::flows::FlowModuleValue;
|
||||
use windmill_common::{
|
||||
db::DB,
|
||||
error::Error,
|
||||
flow_conversations::{add_message_to_conversation_tx, MessageExtras, MessageType},
|
||||
flow_conversations::{add_message_to_conversation_tx, MessageType},
|
||||
flow_status::AgentAction,
|
||||
flows::{InputTransform, Step},
|
||||
jobs::JobKind,
|
||||
@@ -154,8 +154,6 @@ pub async fn get_flow_job_runnable_and_raw_flow(
|
||||
pub struct FlowContext {
|
||||
pub flow_inputs: Option<HashMap<String, Box<RawValue>>>,
|
||||
pub flow_status: Option<windmill_common::flow_status::FlowStatus>,
|
||||
/// Path of the flow the run started from, which scopes a string memory id.
|
||||
pub flow_path: Option<String>,
|
||||
}
|
||||
|
||||
/// Get flow context (chat settings + args + flow_status) from root flow's job data
|
||||
@@ -173,8 +171,7 @@ pub async fn get_flow_context(db: &DB, job: &MiniPulledJob) -> FlowContext {
|
||||
r#"
|
||||
SELECT
|
||||
j.args as "args: Json<HashMap<String, Box<RawValue>>>",
|
||||
js.flow_status as "flow_status: Json<windmill_common::flow_status::FlowStatus>",
|
||||
j.runnable_path
|
||||
js.flow_status as "flow_status: Json<windmill_common::flow_status::FlowStatus>"
|
||||
FROM v2_job_status js
|
||||
INNER JOIN v2_job j ON j.id = js.id
|
||||
WHERE js.id = $1
|
||||
@@ -187,7 +184,6 @@ pub async fn get_flow_context(db: &DB, job: &MiniPulledJob) -> FlowContext {
|
||||
Ok(Some(row)) => FlowContext {
|
||||
flow_inputs: row.args.map(|j| j.0),
|
||||
flow_status: row.flow_status.map(|j| j.0),
|
||||
flow_path: row.runnable_path,
|
||||
},
|
||||
Ok(None) => {
|
||||
tracing::warn!(
|
||||
@@ -213,7 +209,6 @@ pub async fn add_message_to_conversation(
|
||||
message_type: MessageType,
|
||||
step_name: &Option<String>,
|
||||
success: bool,
|
||||
extras: Option<&MessageExtras>,
|
||||
) -> Result<(), Error> {
|
||||
let mut tx = db.begin().await?;
|
||||
add_message_to_conversation_tx(
|
||||
@@ -224,7 +219,6 @@ pub async fn add_message_to_conversation(
|
||||
message_type,
|
||||
step_name.as_deref(),
|
||||
success,
|
||||
extras,
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
@@ -44,7 +44,7 @@ use windmill_common::{
|
||||
client::AuthedClient,
|
||||
db::DB,
|
||||
error::{self, Error},
|
||||
flow_conversations::{memory_key, MessageExtras, MessageType},
|
||||
flow_conversations::MessageType,
|
||||
flow_status::AgentAction,
|
||||
flows::{AgentTool, FlowModule, FlowModuleValue, InputTransform, ToolValue},
|
||||
get_latest_hash_for_path,
|
||||
@@ -111,148 +111,6 @@ fn prepare_auto_memory_messages_for_persistence(
|
||||
non_system_messages[start_idx..].to_vec()
|
||||
}
|
||||
|
||||
/// The inputs a linked step supplies for itself; the resource holds the rest of the brain.
|
||||
const FLOW_LOCAL_AGENT_KEYS: [&str; 5] = [
|
||||
"user_message",
|
||||
"user_attachments",
|
||||
"enabled_tools",
|
||||
"memory_id",
|
||||
"previous_messages",
|
||||
];
|
||||
|
||||
/// The flow-local inputs that name a conversation, which a saved agent never carries.
|
||||
const STEP_HISTORY_KEYS: [&str; 2] = ["memory_id", "previous_messages"];
|
||||
|
||||
/// Where one agent invocation's history comes from.
|
||||
#[derive(Debug)]
|
||||
enum HistorySource<'a> {
|
||||
/// Supplied by the flow and replayed as is: memory is neither read nor written.
|
||||
Messages(&'a [OpenAIMessage]),
|
||||
Window {
|
||||
memory_id: Uuid,
|
||||
context_length: usize,
|
||||
},
|
||||
Stateless,
|
||||
}
|
||||
|
||||
/// A step's memory id counts only as the step authored it. A static empty value is a form
|
||||
/// placeholder, so it reads as unset rather than as an expression that evaluated to nothing, which
|
||||
/// runs without memory; an AI-filled value would let the model choose which memory the agent reads.
|
||||
fn keep_authored_memory_id(
|
||||
args: &mut AIAgentArgs,
|
||||
step_input_transforms: &HashMap<String, InputTransform>,
|
||||
) {
|
||||
match step_input_transforms.get("memory_id") {
|
||||
Some(InputTransform::Javascript { .. }) => {}
|
||||
Some(InputTransform::Static { .. }) if args.memory_id.as_deref() != Some("") => {}
|
||||
_ => args.memory_id = None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Reconciles the step's history inputs, the agent's memory policy and the run's memory id. A step
|
||||
/// holds one of two shapes: an older `auto` or `manual` memory, read as the editor that wrote it
|
||||
/// meant it, or the current setting plus the step's own history inputs. Also returns lines for the
|
||||
/// job log: an input that went unused, or a policy that remembers ending up stateless.
|
||||
fn resolve_history_source<'a>(
|
||||
args: &'a AIAgentArgs,
|
||||
run_memory_id: Option<Uuid>,
|
||||
workspace_id: &str,
|
||||
flow_path: &str,
|
||||
) -> (HistorySource<'a>, Vec<&'static str>) {
|
||||
let mut notes = Vec::new();
|
||||
let no_memory_id = "No memory id was passed to this run, so the agent runs without memory.";
|
||||
match &args.memory {
|
||||
// The step's own history inputs came after these, so a step that still holds one reads it
|
||||
// alone: what it did before the editor offered them is what it keeps doing.
|
||||
Some(Memory::Manual { messages }) => {
|
||||
note_unread_step_inputs(&mut notes, args);
|
||||
(HistorySource::Messages(messages), notes)
|
||||
}
|
||||
Some(Memory::Auto { context_length, memory_id }) => {
|
||||
note_unread_step_inputs(&mut notes, args);
|
||||
// An id baked in at save time only ever applied when the run carried none.
|
||||
match run_memory_id.or(*memory_id) {
|
||||
Some(memory_id) => (
|
||||
HistorySource::Window { memory_id, context_length: *context_length },
|
||||
notes,
|
||||
),
|
||||
None => {
|
||||
notes.push(no_memory_id);
|
||||
(HistorySource::Stateless, notes)
|
||||
}
|
||||
}
|
||||
}
|
||||
Some(Memory::Window { context_length }) => {
|
||||
if args
|
||||
.previous_messages
|
||||
.as_ref()
|
||||
.is_some_and(|messages| !messages.is_empty())
|
||||
{
|
||||
notes.push("Managed memory is on, so this step's previous messages are ignored.");
|
||||
}
|
||||
let memory_id = match args.memory_id.as_deref() {
|
||||
Some("") => {
|
||||
notes.push(
|
||||
"This step's memory id evaluated to an empty value, so the agent runs without memory.",
|
||||
);
|
||||
return (HistorySource::Stateless, notes);
|
||||
}
|
||||
Some(step_memory_id) => memory_key(workspace_id, flow_path, step_memory_id),
|
||||
None => match run_memory_id {
|
||||
Some(memory_id) => memory_id,
|
||||
None => {
|
||||
notes.push(no_memory_id);
|
||||
return (HistorySource::Stateless, notes);
|
||||
}
|
||||
},
|
||||
};
|
||||
(
|
||||
HistorySource::Window { memory_id, context_length: *context_length },
|
||||
notes,
|
||||
)
|
||||
}
|
||||
Some(Memory::Off) | None => {
|
||||
if args.memory_id.as_deref().is_some_and(|id| !id.is_empty()) {
|
||||
notes.push("Managed memory is off, so this step's memory id is ignored.");
|
||||
}
|
||||
match &args.previous_messages {
|
||||
Some(messages) => (HistorySource::Messages(messages), notes),
|
||||
None => (HistorySource::Stateless, notes),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// An older memory setting reads neither history input, which is only visible in the job log: the
|
||||
/// editor offers them on a step that has been moved to the current settings.
|
||||
fn note_unread_step_inputs(notes: &mut Vec<&'static str>, args: &AIAgentArgs) {
|
||||
if args.memory_id.as_deref().is_some_and(|id| !id.is_empty()) {
|
||||
notes.push("This step uses an older memory setting, so its memory id is not read.");
|
||||
}
|
||||
if args
|
||||
.previous_messages
|
||||
.as_ref()
|
||||
.is_some_and(|messages| !messages.is_empty())
|
||||
{
|
||||
notes
|
||||
.push("This step uses an older memory setting, so its previous messages are not read.");
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a request has something to ask the model. Only text output sends previous messages, so
|
||||
/// an image prompt comes from the user message alone. An empty list is no conversation, except
|
||||
/// under a legacy `manual` memory, which ran on whatever list it held.
|
||||
fn has_prompt(
|
||||
history: &HistorySource,
|
||||
has_user_message: bool,
|
||||
is_text_output: bool,
|
||||
legacy_list: bool,
|
||||
) -> bool {
|
||||
has_user_message
|
||||
|| (is_text_output
|
||||
&& (legacy_list || matches!(history, HistorySource::Messages(m) if !m.is_empty())))
|
||||
}
|
||||
|
||||
fn find_module_by_id(
|
||||
modules: &Vec<FlowModule>,
|
||||
target_id: &str,
|
||||
@@ -278,16 +136,14 @@ async fn find_ai_agent_tool_module_in_parent_agent(
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
let FlowModuleValue::AIAgent { tools, agent, tool_inputs, .. } =
|
||||
parent_agent_module.get_value()?
|
||||
else {
|
||||
let FlowModuleValue::AIAgent { tools, agent, .. } = parent_agent_module.get_value()? else {
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
// A linked parent carries no tools on the module (they live in the resource, resolved only in
|
||||
// the main execution branch). Resolve them from the resource here too, so a nested agent tool
|
||||
// of a saved+linked agent can still be located when it runs as its own job.
|
||||
let mut tools = if let Some(agent_ref) = agent.as_deref() {
|
||||
let tools = if let Some(agent_ref) = agent.as_deref() {
|
||||
let agent_path = agent_ref
|
||||
.trim_start_matches("$res:")
|
||||
.trim_start_matches("res://");
|
||||
@@ -314,9 +170,6 @@ async fn find_ai_agent_tool_module_in_parent_agent(
|
||||
} else {
|
||||
tools
|
||||
};
|
||||
// The nested job reads its history inputs from the tool's transforms, which must carry the
|
||||
// host flow's bindings as the parent evaluated them.
|
||||
overlay_tool_inputs(&mut tools, &tool_inputs);
|
||||
|
||||
for tool in tools {
|
||||
if tool.id == tool_module_id {
|
||||
@@ -603,7 +456,6 @@ pub async fn handle_ai_agent_job(
|
||||
omit_output_from_conversation,
|
||||
agent,
|
||||
tool_inputs,
|
||||
input_transforms: step_input_transforms,
|
||||
..
|
||||
} = module.get_value()?
|
||||
else {
|
||||
@@ -614,11 +466,9 @@ pub async fn handle_ai_agent_job(
|
||||
|
||||
// A linked step takes its brain and tools from the resource and keeps only its own flow-local
|
||||
// inputs. The brain and the roster stay rigid; what the step binds to this flow is the message
|
||||
// it asks, which of those tools this use may call, the conversation it is part of (its memory
|
||||
// id and previous messages), and the tools' own inputs — the last overlaid from `tool_inputs`
|
||||
// below.
|
||||
let (mut args, tools): (AIAgentArgs, Vec<AgentTool>) = if let Some(agent_ref) = agent.as_deref()
|
||||
{
|
||||
// it asks, which of those tools this use may call, the conversation it is part of, and the
|
||||
// tools' own inputs — the last overlaid from `tool_inputs` below.
|
||||
let (args, tools): (AIAgentArgs, Vec<AgentTool>) = if let Some(agent_ref) = agent.as_deref() {
|
||||
let agent_path = agent_ref
|
||||
.trim_start_matches("$res:")
|
||||
.trim_start_matches("res://");
|
||||
@@ -650,12 +500,6 @@ pub async fn handle_ai_agent_job(
|
||||
None => Vec::new(),
|
||||
};
|
||||
overlay_tool_inputs(&mut tools, &tool_inputs);
|
||||
// The resource is not validated against a schema, so a history input it happens to carry
|
||||
// is dropped before interpolation, where a bad `$res:` in it would fail the step. The
|
||||
// other flow-local keys stay: a resource's own user message is the step's fallback.
|
||||
for key in STEP_HISTORY_KEYS {
|
||||
config.remove(key);
|
||||
}
|
||||
let brain = transform_json_value(
|
||||
"ai_agent",
|
||||
client,
|
||||
@@ -677,7 +521,7 @@ pub async fn handle_ai_agent_job(
|
||||
// Only after interpolating the resource: these are caller-controlled and already resolved by
|
||||
// build_args_map, so passing them through it again would expand contextual values —
|
||||
// `$WM_TOKEN` in a user message would reach the model provider.
|
||||
for key in FLOW_LOCAL_AGENT_KEYS {
|
||||
for key in ["user_message", "user_attachments", "enabled_tools"] {
|
||||
if let Some(v) = local_args.get(key) {
|
||||
brain.insert(
|
||||
key.to_string(),
|
||||
@@ -702,8 +546,6 @@ pub async fn handle_ai_agent_job(
|
||||
(args, tools)
|
||||
};
|
||||
|
||||
keep_authored_memory_id(&mut args, &step_input_transforms);
|
||||
|
||||
// Nesting is capped at flow → agent → nested agent. When this job is itself a nested tool,
|
||||
// a linked resource's tool set may still contain AIAgent tools (the editor can't constrain a
|
||||
// shared resource); don't advertise them — invoking one would only fail the depth check as a
|
||||
@@ -1168,18 +1010,8 @@ pub async fn run_agent(
|
||||
// Fetch flow context for input transforms context, chat and memory
|
||||
let mut flow_context = get_flow_context(db, job).await;
|
||||
|
||||
// The run's memory id is also the chat conversation id, which a step's own memory id never
|
||||
// replaces.
|
||||
let conversation_id = flow_context
|
||||
.flow_status
|
||||
.as_ref()
|
||||
.and_then(|fs| fs.memory_id);
|
||||
let (history, history_notes) = resolve_history_source(
|
||||
args,
|
||||
conversation_id,
|
||||
&job.workspace_id,
|
||||
flow_context.flow_path.as_deref().unwrap_or_default(),
|
||||
);
|
||||
// Determine if we're using manual messages (which bypasses memory)
|
||||
let use_manual_messages = matches!(args.memory, Some(Memory::Manual { .. }));
|
||||
|
||||
// Check if user_message is provided and non-empty
|
||||
let has_user_message = args
|
||||
@@ -1188,63 +1020,63 @@ pub async fn run_agent(
|
||||
.map(|m| !m.is_empty())
|
||||
.unwrap_or(false);
|
||||
|
||||
// Validate: at least one of memory with manual messages or user_message must be provided
|
||||
if !use_manual_messages && !has_user_message {
|
||||
return Err(Error::internal_err(
|
||||
"Either 'memory' with manual messages or 'user_message' must be provided".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let is_text_output = output_type == &OutputType::Text;
|
||||
|
||||
if is_text_output {
|
||||
for note in &history_notes {
|
||||
append_logs(&job.id, &job.workspace_id, format!("{note}\n"), conn).await;
|
||||
}
|
||||
} else if !matches!(args.memory, None | Some(Memory::Off))
|
||||
|| args.memory_id.is_some()
|
||||
|| args.previous_messages.is_some()
|
||||
{
|
||||
append_logs(
|
||||
&job.id,
|
||||
&job.workspace_id,
|
||||
"Image output sends no history, so memory and previous messages are not read.\n",
|
||||
conn,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
// A `manual` memory sent whatever list it held, an empty one included, so a step that still has
|
||||
// one keeps running without a user message.
|
||||
let legacy_list = matches!(args.memory, Some(Memory::Manual { .. }));
|
||||
if !has_prompt(&history, has_user_message, is_text_output, legacy_list) {
|
||||
let missing = if !is_text_output {
|
||||
"'user_message' must be provided for image output"
|
||||
} else if matches!(
|
||||
args.memory,
|
||||
Some(Memory::Window { .. } | Memory::Auto { .. })
|
||||
) {
|
||||
"'user_message' must be provided while managed memory is on"
|
||||
} else {
|
||||
"Either 'previous_messages' or 'user_message' must be provided"
|
||||
};
|
||||
return Err(Error::internal_err(missing.to_string()));
|
||||
}
|
||||
// Flow-level memory_id (from chat mode) takes precedence over step-level memory_id
|
||||
let memory_id = flow_context
|
||||
.flow_status
|
||||
.as_ref()
|
||||
.and_then(|fs| fs.memory_id)
|
||||
.or_else(|| {
|
||||
// Extract memory_id from Memory::Auto if present
|
||||
match &args.memory {
|
||||
Some(Memory::Auto { memory_id, .. }) => *memory_id,
|
||||
_ => None,
|
||||
}
|
||||
});
|
||||
|
||||
// Load messages based on history mode
|
||||
if matches!(output_type, OutputType::Text) {
|
||||
match &history {
|
||||
HistorySource::Messages(provided) => messages.extend(provided.iter().cloned()),
|
||||
HistorySource::Window { memory_id, context_length } => {
|
||||
match &args.memory {
|
||||
Some(Memory::Manual { messages: manual_messages }) => {
|
||||
// Use explicitly provided messages (bypass memory)
|
||||
if !manual_messages.is_empty() {
|
||||
messages.extend(manual_messages.clone());
|
||||
}
|
||||
}
|
||||
Some(Memory::Auto { context_length, .. }) => {
|
||||
// Auto mode: load from memory
|
||||
if let Some(step_id) = effective_flow_step_id {
|
||||
match read_from_memory(db, &job.workspace_id, *memory_id, step_id).await {
|
||||
Ok(Some(loaded_messages)) => {
|
||||
let messages_to_load = prepare_auto_memory_messages_for_request(
|
||||
&loaded_messages,
|
||||
*context_length,
|
||||
);
|
||||
messages.extend(messages_to_load);
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => {
|
||||
tracing::error!("Failed to read memory for step {}: {}", step_id, e);
|
||||
if let Some(memory_id) = memory_id {
|
||||
// Read messages from memory
|
||||
match read_from_memory(db, &job.workspace_id, memory_id, step_id).await {
|
||||
Ok(Some(loaded_messages)) => {
|
||||
let messages_to_load = prepare_auto_memory_messages_for_request(
|
||||
&loaded_messages,
|
||||
*context_length,
|
||||
);
|
||||
messages.extend(messages_to_load);
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => {
|
||||
tracing::error!(
|
||||
"Failed to read memory for step {}: {}",
|
||||
step_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
HistorySource::Stateless => {}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1689,35 +1521,30 @@ pub async fn run_agent(
|
||||
..Default::default()
|
||||
});
|
||||
if persist_output_to_conversation {
|
||||
if let Some(conversation_id) = conversation_id {
|
||||
// The search ran inside the provider's call, so this job's args
|
||||
// describe the agent, not the search: its sources reach the row
|
||||
// only if they are written here.
|
||||
let extras = (!annotations.is_empty()).then(|| MessageExtras {
|
||||
tool_result: serde_json::to_string(&annotations).ok(),
|
||||
..Default::default()
|
||||
if let Some(memory_id) = memory_id {
|
||||
let agent_job_id = job.id;
|
||||
let db_clone = db.clone();
|
||||
let message_content = "Used websearch tool successfully".to_string();
|
||||
let step_name = step_name.clone();
|
||||
tokio::spawn(async move {
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
&db_clone,
|
||||
&memory_id,
|
||||
Some(agent_job_id),
|
||||
&message_content,
|
||||
MessageType::Tool,
|
||||
&step_name,
|
||||
true,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add websearch tool message to conversation {}: {}",
|
||||
memory_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
});
|
||||
// Awaited like every row of the loop, so rows commit in turn order.
|
||||
// Worded like every other tool row, so a reader recovers the tool
|
||||
// name from the sentence.
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
db,
|
||||
&conversation_id,
|
||||
Some(job.id),
|
||||
"Used websearch tool",
|
||||
MessageType::Tool,
|
||||
&step_name,
|
||||
true,
|
||||
extras.as_ref(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add websearch tool message to conversation {}: {}",
|
||||
conversation_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1746,30 +1573,32 @@ pub async fn run_agent(
|
||||
|
||||
// Add assistant message to conversation if chat_input_enabled
|
||||
if persist_output_to_conversation && !response_content.is_empty() {
|
||||
if let Some(conversation_id) = conversation_id {
|
||||
// This iteration's thinking goes on the answer's row; the job
|
||||
// result only keeps the turn's thinking as one string.
|
||||
let extras = response_reasoning.clone().map(|reasoning| {
|
||||
MessageExtras { reasoning: Some(reasoning), ..Default::default() }
|
||||
if let Some(memory_id) = memory_id {
|
||||
let agent_job_id = job.id;
|
||||
let db_clone = db.clone();
|
||||
let message_content = response_content.clone();
|
||||
let step_name = step_name.clone();
|
||||
|
||||
// Spawn task because we do not need to wait for the result
|
||||
tokio::spawn(async move {
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
&db_clone,
|
||||
&memory_id,
|
||||
Some(agent_job_id),
|
||||
&message_content,
|
||||
MessageType::Assistant,
|
||||
&step_name,
|
||||
true,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add assistant message to conversation {}: {}",
|
||||
memory_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
});
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
db,
|
||||
&conversation_id,
|
||||
Some(job.id),
|
||||
response_content,
|
||||
MessageType::Assistant,
|
||||
&step_name,
|
||||
true,
|
||||
extras.as_ref(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add assistant message to conversation {}: {}",
|
||||
conversation_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1808,18 +1637,6 @@ pub async fn run_agent(
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
// A round's thinking is stored on one row, the first the round writes, which is
|
||||
// where the stream shows it: its text row when it wrote text, else the row of
|
||||
// its first call — the answer row below when that call is the structured-output
|
||||
// tool. Two rows carrying it would show it twice after a reload.
|
||||
let call_reasoning = response_reasoning
|
||||
.clone()
|
||||
.filter(|_| response_content.as_deref().unwrap_or("").is_empty());
|
||||
let structured_output_first = structured_output_tool_name
|
||||
.as_ref()
|
||||
.zip(tool_calls.first())
|
||||
.map_or(false, |(name, tc)| tc.function.name == *name);
|
||||
|
||||
// Handle tool calls using extracted tools module
|
||||
let tool_execution_ctx = ToolExecutionContext {
|
||||
db,
|
||||
@@ -1838,11 +1655,6 @@ pub async fn run_agent(
|
||||
stream_event_processor: stream_event_processor.as_ref(),
|
||||
flow_context: &mut flow_context,
|
||||
omit_output_from_conversation,
|
||||
reasoning: if structured_output_first {
|
||||
None
|
||||
} else {
|
||||
call_reasoning.clone()
|
||||
},
|
||||
previous_result: &previous_result,
|
||||
id_context: &id_context,
|
||||
tool_abort_handles: tool_abort_handles.clone(),
|
||||
@@ -1861,41 +1673,6 @@ pub async fn run_agent(
|
||||
.await?;
|
||||
|
||||
messages.extend(tool_messages);
|
||||
|
||||
// A structured answer is the arguments of the structured-output tool call,
|
||||
// on which the loop ends without a text iteration, so its row is written here.
|
||||
if tool_used_structured_output && persist_output_to_conversation {
|
||||
if let (Some(conversation_id), Some(OpenAIContent::Text(answer))) =
|
||||
(conversation_id, tool_content.as_ref())
|
||||
{
|
||||
let extras = call_reasoning
|
||||
.clone()
|
||||
.filter(|_| structured_output_first)
|
||||
.map(|reasoning| MessageExtras {
|
||||
reasoning: Some(reasoning),
|
||||
..Default::default()
|
||||
});
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
db,
|
||||
&conversation_id,
|
||||
Some(job.id),
|
||||
answer,
|
||||
MessageType::Assistant,
|
||||
&step_name,
|
||||
true,
|
||||
extras.as_ref(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add structured answer to conversation {}: {}",
|
||||
conversation_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(tc) = tool_content {
|
||||
content = Some(tc);
|
||||
}
|
||||
@@ -1915,7 +1692,10 @@ pub async fn run_agent(
|
||||
|
||||
// Add assistant message to conversation if chat_input_enabled
|
||||
if persist_output_to_conversation {
|
||||
if let Some(conversation_id) = conversation_id {
|
||||
if let Some(memory_id) = memory_id {
|
||||
let agent_job_id = job.id;
|
||||
let db_clone = db.clone();
|
||||
|
||||
// Create extended version with type discriminator for conversation storage
|
||||
// This avoids conflicts with outputs that are of the same format as S3 objects
|
||||
let s3_with_type = S3ObjectWithType {
|
||||
@@ -1926,24 +1706,26 @@ pub async fn run_agent(
|
||||
let message_content = serde_json::to_string(&s3_with_type)
|
||||
.unwrap_or_else(|_| content.get().to_string());
|
||||
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
db,
|
||||
&conversation_id,
|
||||
Some(job.id),
|
||||
&message_content,
|
||||
MessageType::Assistant,
|
||||
&step_name,
|
||||
true,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add assistant message to conversation {}: {}",
|
||||
conversation_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
// Spawn task because we do not need to wait for the result
|
||||
tokio::spawn(async move {
|
||||
if let Err(e) = add_message_to_conversation(
|
||||
&db_clone,
|
||||
&memory_id,
|
||||
Some(agent_job_id),
|
||||
&message_content,
|
||||
MessageType::Assistant,
|
||||
&step_name,
|
||||
true,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"Failed to add assistant message to conversation {}: {}",
|
||||
memory_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1996,10 +1778,13 @@ pub async fn run_agent(
|
||||
}
|
||||
}
|
||||
|
||||
// final_messages holds the complete history: what was loaded plus this run's messages
|
||||
if matches!(output_type, OutputType::Text) {
|
||||
if let HistorySource::Window { memory_id, context_length } = &history {
|
||||
// Persist complete conversation to memory at the end (only if in auto mode with context length)
|
||||
// Skip memory persistence if using manual messages (bypass memory entirely)
|
||||
// final_messages contains the complete history (old messages + new ones)
|
||||
if matches!(output_type, OutputType::Text) && !use_manual_messages {
|
||||
if let Some(Memory::Auto { context_length, .. }) = &args.memory {
|
||||
if let Some(step_id) = effective_flow_step_id {
|
||||
// Extract OpenAIMessages from final_messages
|
||||
let all_messages: Vec<OpenAIMessage> =
|
||||
final_messages.iter().map(|m| m.message.clone()).collect();
|
||||
|
||||
@@ -2009,21 +1794,23 @@ pub async fn run_agent(
|
||||
*context_length,
|
||||
);
|
||||
|
||||
if let Err(e) = write_to_memory(
|
||||
db,
|
||||
&job.workspace_id,
|
||||
*memory_id,
|
||||
step_id,
|
||||
&messages_to_persist,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::error!(
|
||||
"Failed to persist {} messages to memory for step {}: {}",
|
||||
messages_to_persist.len(),
|
||||
if let Some(memory_id) = memory_id {
|
||||
if let Err(e) = write_to_memory(
|
||||
db,
|
||||
&job.workspace_id,
|
||||
memory_id,
|
||||
step_id,
|
||||
e
|
||||
);
|
||||
&messages_to_persist,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::error!(
|
||||
"Failed to persist {} messages to memory for step {}: {}",
|
||||
messages_to_persist.len(),
|
||||
step_id,
|
||||
e
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -2083,228 +1870,6 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, PartialEq)]
|
||||
enum Resolved {
|
||||
Messages(usize),
|
||||
Window(Uuid, usize),
|
||||
Stateless { noted: bool },
|
||||
}
|
||||
|
||||
/// Every memory shape a worker may still read, resolved against a run with or without a
|
||||
/// memory id. The hashed id is pinned: changing it detaches memories stored under string ids.
|
||||
#[test]
|
||||
fn history_source_resolves_every_memory_shape() {
|
||||
use serde_json::json;
|
||||
let run = Uuid::from_u128(1);
|
||||
let baked = Uuid::from_u128(2);
|
||||
let cust_1 = Uuid::parse_str("0168fcea-ffa7-5c15-bdb0-7709bb5f540d").unwrap();
|
||||
let window = json!({ "kind": "window", "context_length": 10 });
|
||||
let message = json!([{ "role": "user", "content": "earlier" }]);
|
||||
let two_messages = json!([
|
||||
{ "role": "user", "content": "earlier" },
|
||||
{ "role": "assistant", "content": "reply" }
|
||||
]);
|
||||
let cases = [
|
||||
(
|
||||
"absent memory is off",
|
||||
json!({}),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: false },
|
||||
),
|
||||
(
|
||||
"legacy off",
|
||||
json!({ "memory": { "kind": "off" } }),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: false },
|
||||
),
|
||||
(
|
||||
"legacy auto prefers the run's id",
|
||||
json!({ "memory": { "kind": "auto", "context_length": 4, "memory_id": baked } }),
|
||||
Some(run),
|
||||
Resolved::Window(run, 4),
|
||||
),
|
||||
(
|
||||
"legacy auto falls back to its baked id",
|
||||
json!({ "memory": { "kind": "auto", "context_length": 4, "memory_id": baked } }),
|
||||
None,
|
||||
Resolved::Window(baked, 4),
|
||||
),
|
||||
(
|
||||
"legacy auto with an empty baked id uses the run's",
|
||||
json!({ "memory": { "kind": "auto", "context_length": 4, "memory_id": "" } }),
|
||||
Some(run),
|
||||
Resolved::Window(run, 4),
|
||||
),
|
||||
(
|
||||
"legacy auto with an empty baked id and no run id is stateless",
|
||||
json!({ "memory": { "kind": "auto", "context_length": 4, "memory_id": " " } }),
|
||||
None,
|
||||
Resolved::Stateless { noted: true },
|
||||
),
|
||||
(
|
||||
"legacy auto without a length is off",
|
||||
json!({ "memory": { "kind": "auto", "memory_id": baked } }),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: false },
|
||||
),
|
||||
(
|
||||
"a cleared count is off",
|
||||
json!({ "memory": { "kind": "window", "context_length": null } }),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: false },
|
||||
),
|
||||
(
|
||||
"legacy manual replays its messages",
|
||||
json!({ "memory": { "kind": "manual", "messages": message } }),
|
||||
Some(run),
|
||||
Resolved::Messages(1),
|
||||
),
|
||||
(
|
||||
"window keeps the run's memory",
|
||||
json!({ "memory": window }),
|
||||
Some(run),
|
||||
Resolved::Window(run, 10),
|
||||
),
|
||||
(
|
||||
"window without a memory id is stateless",
|
||||
json!({ "memory": window }),
|
||||
None,
|
||||
Resolved::Stateless { noted: true },
|
||||
),
|
||||
(
|
||||
"a step memory id overrides the run's",
|
||||
json!({ "memory": window, "memory_id": "cust_1" }),
|
||||
Some(run),
|
||||
Resolved::Window(cust_1, 10),
|
||||
),
|
||||
(
|
||||
"a uuid step memory id is used as is",
|
||||
json!({ "memory": window, "memory_id": baked.to_string() }),
|
||||
Some(run),
|
||||
Resolved::Window(baked, 10),
|
||||
),
|
||||
(
|
||||
"a step memory id evaluating to null is stateless",
|
||||
json!({ "memory": window, "memory_id": null }),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: true },
|
||||
),
|
||||
(
|
||||
"an off policy ignores the step memory id, and says so",
|
||||
json!({ "memory": { "kind": "off" }, "memory_id": "cust_1" }),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: true },
|
||||
),
|
||||
(
|
||||
"managed memory ignores the step's previous messages",
|
||||
json!({ "memory": window, "memory_id": "cust_1", "previous_messages": message }),
|
||||
Some(run),
|
||||
Resolved::Window(cust_1, 10),
|
||||
),
|
||||
(
|
||||
"memory that is off sends the step's previous messages",
|
||||
json!({ "previous_messages": message }),
|
||||
Some(run),
|
||||
Resolved::Messages(1),
|
||||
),
|
||||
(
|
||||
"a previous messages expression that evaluated to null is no history",
|
||||
json!({ "previous_messages": null }),
|
||||
Some(run),
|
||||
Resolved::Stateless { noted: false },
|
||||
),
|
||||
(
|
||||
"a legacy manual list ignores the step's previous messages",
|
||||
json!({ "memory": { "kind": "manual", "messages": message }, "previous_messages": two_messages }),
|
||||
Some(run),
|
||||
Resolved::Messages(1),
|
||||
),
|
||||
(
|
||||
"legacy auto ignores a step memory id",
|
||||
json!({ "memory": { "kind": "auto", "context_length": 4, "memory_id": baked }, "memory_id": "cust_1" }),
|
||||
None,
|
||||
Resolved::Window(baked, 4),
|
||||
),
|
||||
];
|
||||
for (name, history, run_memory_id, expected) in cases {
|
||||
let mut raw = json!({ "provider": { "kind": "openai", "resource": {}, "model": "m" } });
|
||||
raw.as_object_mut()
|
||||
.unwrap()
|
||||
.extend(history.as_object().unwrap().clone());
|
||||
let args: AIAgentArgs = serde_json::from_value(raw).unwrap();
|
||||
let resolved = match resolve_history_source(&args, run_memory_id, "ws", "f/flow") {
|
||||
(HistorySource::Messages(m), _) => Resolved::Messages(m.len()),
|
||||
(HistorySource::Window { memory_id, context_length }, _) => {
|
||||
Resolved::Window(memory_id, context_length)
|
||||
}
|
||||
(HistorySource::Stateless, notes) => {
|
||||
Resolved::Stateless { noted: !notes.is_empty() }
|
||||
}
|
||||
};
|
||||
assert_eq!(resolved, expected, "{name}");
|
||||
}
|
||||
}
|
||||
|
||||
/// A placeholder the form seeds must not read as a memory id that evaluated to nothing, which
|
||||
/// would turn memory off for the step.
|
||||
#[test]
|
||||
fn only_an_expression_can_set_an_empty_step_memory_id() {
|
||||
let transforms = |memory_id: &str| -> HashMap<String, InputTransform> {
|
||||
HashMap::from([(
|
||||
"memory_id".to_string(),
|
||||
serde_json::from_str(memory_id).unwrap(),
|
||||
)])
|
||||
};
|
||||
let args = || -> AIAgentArgs {
|
||||
serde_json::from_value(serde_json::json!({
|
||||
"provider": { "kind": "openai", "resource": {}, "model": "m" },
|
||||
"memory_id": null,
|
||||
}))
|
||||
.unwrap()
|
||||
};
|
||||
for (transform, expected) in [
|
||||
(r#"{ "type": "static" }"#, None),
|
||||
(r#"{ "type": "static", "value": "" }"#, None),
|
||||
(r#"{ "type": "ai" }"#, None),
|
||||
(
|
||||
r#"{ "type": "javascript", "expr": "flow_input.customer_id" }"#,
|
||||
Some(""),
|
||||
),
|
||||
] {
|
||||
let mut args = args();
|
||||
keep_authored_memory_id(&mut args, &transforms(transform));
|
||||
assert_eq!(args.memory_id.as_deref(), expected, "{transform}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Only text output sends previous messages, so they never stand in for an image prompt.
|
||||
#[test]
|
||||
fn previous_messages_never_stand_in_for_an_image_prompt() {
|
||||
let args: AIAgentArgs = serde_json::from_value(serde_json::json!({
|
||||
"provider": { "kind": "openai", "resource": {}, "model": "m" },
|
||||
"previous_messages": [{ "role": "user", "content": "earlier" }],
|
||||
}))
|
||||
.unwrap();
|
||||
let (history, _) = resolve_history_source(&args, None, "ws", "f/flow");
|
||||
assert!(has_prompt(&history, false, true, false));
|
||||
assert!(!has_prompt(&history, false, false, false));
|
||||
assert!(has_prompt(&history, true, false, false));
|
||||
assert!(!has_prompt(
|
||||
&HistorySource::Messages(&[]),
|
||||
false,
|
||||
true,
|
||||
false
|
||||
));
|
||||
// A legacy `manual` memory ran on an empty list alone, and still does for text output.
|
||||
assert!(has_prompt(&HistorySource::Messages(&[]), false, true, true));
|
||||
assert!(!has_prompt(
|
||||
&HistorySource::Messages(&[]),
|
||||
false,
|
||||
false,
|
||||
true
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reasoning_keeps_every_iteration_in_order() {
|
||||
let mut acc = String::new();
|
||||
|
||||
@@ -2240,11 +2240,99 @@ fn parse_attach_db_resource<'a>(query: &'a str) -> Option<ParsedAttachDbResource
|
||||
None
|
||||
}
|
||||
|
||||
/// The verification a DuckDB postgres attach keeps, as its libpq `sslmode` and `sslrootcert`.
|
||||
///
|
||||
/// Attaches have always turned verify-ca and verify-full into `require`, which resources rely on.
|
||||
/// A connection that explicitly refuses invalid certificates — the external instance cluster's —
|
||||
/// keeps its mode instead: under `require` its shared password would go to whichever server
|
||||
/// answers. DuckDB's libpq takes one root file, so it gets the system bundle plus the configured
|
||||
/// certificate.
|
||||
fn pg_attach_verification(res: &PgDatabase) -> Result<Option<(&str, std::path::PathBuf)>> {
|
||||
let mode = match res.sslmode.as_deref() {
|
||||
Some(mode @ ("verify-ca" | "verify-full")) if res.accept_invalid_certs == Some(false) => mode,
|
||||
_ => return Ok(None),
|
||||
};
|
||||
let bundle = windmill_common::system_ca_bundle()
|
||||
.map(std::fs::read_to_string)
|
||||
.transpose()
|
||||
.map_err(|e| Error::ExecutionErr(format!("Failed to read the system CA bundle: {e}")))?
|
||||
.unwrap_or_default();
|
||||
let pem = res.root_certificate_pem.as_deref().unwrap_or_default();
|
||||
if bundle.is_empty() && pem.is_empty() {
|
||||
return Err(Error::ExecutionErr(format!(
|
||||
"sslmode {mode} needs a root certificate, and this worker has no system CA bundle"
|
||||
)));
|
||||
}
|
||||
let roots = format!("{bundle}\n{pem}\n");
|
||||
use sha2::Digest;
|
||||
let dir = std::env::temp_dir().join("windmill-pg-roots");
|
||||
let path = dir.join(format!(
|
||||
"{}.pem",
|
||||
hex::encode(&sha2::Sha256::digest(roots.as_bytes())[..8])
|
||||
));
|
||||
let write_err = |e: std::io::Error| {
|
||||
Error::ExecutionErr(format!("Failed to write root certificates: {e}"))
|
||||
};
|
||||
if path.is_file() {
|
||||
// Marks it recently used, so pruning takes the others first.
|
||||
let _ = std::fs::File::options()
|
||||
.append(true)
|
||||
.open(&path)
|
||||
.and_then(|f| f.set_modified(std::time::SystemTime::now()));
|
||||
} else {
|
||||
std::fs::create_dir_all(&dir).map_err(write_err)?;
|
||||
// Renamed into place: a job attaching concurrently must never read a half-written file.
|
||||
let partial = path.with_extension(format!("{}.partial", Uuid::new_v4()));
|
||||
std::fs::write(&partial, &roots)
|
||||
.and_then(|()| std::fs::rename(&partial, &path))
|
||||
.map_err(write_err)?;
|
||||
prune_pg_roots(&dir, &path);
|
||||
}
|
||||
Ok(Some((mode, path)))
|
||||
}
|
||||
|
||||
/// Root files outlive the job: a resource's certificate is workspace-controlled, so each distinct
|
||||
/// one would otherwise add a file forever. Keeps the most recently used ones.
|
||||
const PG_ROOTS_KEPT: usize = 32;
|
||||
|
||||
fn prune_pg_roots(dir: &std::path::Path, keep: &std::path::Path) {
|
||||
let Ok(entries) = std::fs::read_dir(dir) else {
|
||||
return;
|
||||
};
|
||||
let mut files: Vec<(std::time::SystemTime, std::path::PathBuf)> = entries
|
||||
.filter_map(|e| e.ok())
|
||||
.map(|e| e.path())
|
||||
.filter(|p| p.extension().is_some_and(|x| x == "pem") && p != keep)
|
||||
.filter_map(|p| Some((std::fs::metadata(&p).ok()?.modified().ok()?, p)))
|
||||
.collect();
|
||||
if files.len() < PG_ROOTS_KEPT {
|
||||
return;
|
||||
}
|
||||
files.sort();
|
||||
for (_, p) in &files[..=files.len() - PG_ROOTS_KEPT] {
|
||||
let _ = std::fs::remove_file(p);
|
||||
}
|
||||
}
|
||||
|
||||
fn pg_attach_uri(res: &PgDatabase) -> Result<String> {
|
||||
let uri = res.to_uri();
|
||||
let Some((mode, roots)) = pg_attach_verification(res)? else {
|
||||
return Ok(uri);
|
||||
};
|
||||
let base = uri.strip_suffix("?sslmode=require").ok_or_else(|| {
|
||||
Error::internal_err("unexpected sslmode in a postgres connection URI".to_string())
|
||||
})?;
|
||||
Ok(format!(
|
||||
"{base}?sslmode={mode}&sslrootcert={}",
|
||||
urlencoding::encode(&roots.to_string_lossy())
|
||||
))
|
||||
}
|
||||
|
||||
fn format_attach_db_conn_str(db_resource: Value, db_type: &str) -> Result<String> {
|
||||
let s = match db_type.to_lowercase().as_str() {
|
||||
"postgres" | "postgresql" => {
|
||||
let res: PgDatabase = serde_json::from_value(db_resource)?;
|
||||
res.to_uri()
|
||||
pg_attach_uri(&res)?
|
||||
}
|
||||
#[cfg(feature = "mysql")]
|
||||
"mysql" => {
|
||||
@@ -2391,7 +2479,9 @@ async fn transform_attach_ducklake(
|
||||
format!(", {}", user_extra_args)
|
||||
};
|
||||
let db_type = match ducklake.catalog.resource_type {
|
||||
DucklakeCatalogResourceType::Instance => "postgres",
|
||||
DucklakeCatalogResourceType::Instance | DucklakeCatalogResourceType::ExternalInstance => {
|
||||
"postgres"
|
||||
}
|
||||
_ => ducklake.catalog.resource_type.as_ref(),
|
||||
};
|
||||
|
||||
@@ -2507,7 +2597,8 @@ fn fork_defer_statements(
|
||||
hidden_passwords.lock().unwrap().push(pwd.to_string());
|
||||
}
|
||||
let db_type = match a.catalog.resource_type {
|
||||
DucklakeCatalogResourceType::Instance => "postgres",
|
||||
DucklakeCatalogResourceType::Instance
|
||||
| DucklakeCatalogResourceType::ExternalInstance => "postgres",
|
||||
_ => a.catalog.resource_type.as_ref(),
|
||||
};
|
||||
stmts.push(get_attach_db_install_str(db_type)?.to_string());
|
||||
@@ -2700,10 +2791,21 @@ fn pg_secret_attach_statements(db_resource: Value, alias_name: &str) -> Result<V
|
||||
let esc = |s: &str| s.replace('\'', "''");
|
||||
// The postgres secret type has no sslmode parameter, so it goes in the ATTACH
|
||||
// string; only the libpq values PgDatabase::to_uri collapses to are forwarded.
|
||||
let sslmode = match res.sslmode.as_deref() {
|
||||
Some("disable") => "disable",
|
||||
Some("require") | Some("verify-ca") | Some("verify-full") => "require",
|
||||
_ => "prefer",
|
||||
let sslmode = match pg_attach_verification(&res)? {
|
||||
// A libpq keyword/value string: the path is quoted for libpq, then for the DuckDB literal.
|
||||
Some((mode, roots)) => format!(
|
||||
"{mode} sslrootcert=''{}''",
|
||||
roots
|
||||
.to_string_lossy()
|
||||
.replace('\\', "\\\\")
|
||||
.replace('\'', "\\''")
|
||||
),
|
||||
None => match res.sslmode.as_deref() {
|
||||
Some("disable") => "disable",
|
||||
Some("require") | Some("verify-ca") | Some("verify-full") => "require",
|
||||
_ => "prefer",
|
||||
}
|
||||
.to_string(),
|
||||
};
|
||||
let secret_name = datatable_secret_name(alias_name);
|
||||
Ok(vec![
|
||||
@@ -2796,6 +2898,46 @@ pub struct Arg {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn pg_attach_keeps_verification_only_when_required() {
|
||||
let pg = |sslmode: &str, accept_invalid_certs: Option<bool>| PgDatabase {
|
||||
host: "db.internal".to_string(),
|
||||
user: Some("custom_instance_user".to_string()),
|
||||
password: Some("pw".to_string()),
|
||||
port: None,
|
||||
sslmode: Some(sslmode.to_string()),
|
||||
dbname: "dt".to_string(),
|
||||
root_certificate_pem: Some("-----BEGIN CERTIFICATE-----test".to_string()),
|
||||
accept_invalid_certs,
|
||||
use_iam_auth: None,
|
||||
region: None,
|
||||
};
|
||||
let uri = pg_attach_uri(&pg("verify-full", Some(false))).unwrap();
|
||||
assert!(uri.contains("?sslmode=verify-full&sslrootcert="), "{uri}");
|
||||
let root = urlencoding::decode(uri.split("sslrootcert=").nth(1).unwrap()).unwrap();
|
||||
let roots = std::fs::read_to_string(root.as_ref()).unwrap();
|
||||
for i in 0..(PG_ROOTS_KEPT + 5) {
|
||||
let mut other = pg("verify-full", Some(false));
|
||||
other.root_certificate_pem = Some(format!("-----BEGIN CERTIFICATE-----{i}"));
|
||||
pg_attach_uri(&other).unwrap();
|
||||
}
|
||||
let kept = std::fs::read_dir(std::env::temp_dir().join("windmill-pg-roots"))
|
||||
.unwrap()
|
||||
.filter(|e| e.as_ref().unwrap().path().extension().is_some_and(|x| x == "pem"))
|
||||
.count();
|
||||
assert!(kept <= PG_ROOTS_KEPT, "{kept} root files kept");
|
||||
assert!(roots.contains("-----BEGIN CERTIFICATE-----test"));
|
||||
let external = serde_json::to_value(pg("verify-full", Some(false))).unwrap();
|
||||
let attach = &pg_secret_attach_statements(external, "dt").unwrap()[3];
|
||||
assert!(
|
||||
attach.starts_with(&format!("ATTACH 'sslmode=verify-full sslrootcert=''{}''", root)),
|
||||
"{attach}"
|
||||
);
|
||||
// A resource that never opted in keeps the historical downgrade.
|
||||
assert!(pg_attach_uri(&pg("verify-full", None)).unwrap().ends_with("?sslmode=require"));
|
||||
assert!(pg_attach_uri(&pg("require", Some(false))).unwrap().ends_with("?sslmode=require"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn attach_datatable_parses_name_and_role() {
|
||||
let reference_of = |q: &str| parse_attach_datatable(q).unwrap().reference;
|
||||
|
||||
@@ -2236,7 +2236,6 @@ async fn add_tool_message_to_conversation(
|
||||
MessageType::Assistant,
|
||||
None,
|
||||
success,
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
|
||||
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
|
||||
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
|
||||
|
||||
export const VERSION = "v1.814.0";
|
||||
export const VERSION = "v1.813.0";
|
||||
|
||||
export async function login(email: string, password: string): Promise<string> {
|
||||
return await windmill.UserService.login({
|
||||
|
||||
+6
-37
@@ -56,10 +56,8 @@ not a replacement of the previous answer.
|
||||
|
||||
The transport also carries the history helpers: `transport.loadMessages(id)` returns
|
||||
`UIMessage`s for `useChat({ messages })` or `setMessages`, `transport.listConversations()`
|
||||
and `transport.deleteConversation(id)`. A loaded user message lists the files it carried
|
||||
in `metadata.attachments`; `WindmillChatApi.attachmentUrl` gives each one's download URL.
|
||||
Sending attachments is not supported: `sendMessage` with `files` is refused with an
|
||||
explanatory error.
|
||||
and `transport.deleteConversation(id)`. Attachments are not supported: `sendMessage` with
|
||||
`files` is refused with an explanatory error.
|
||||
|
||||
## assistant-ui
|
||||
|
||||
@@ -183,7 +181,7 @@ await chat.sendMessage('Hello')
|
||||
| `workspace` | Detected inside a raw app. |
|
||||
| `token` | A token, or a function returning one (called before every request, so it can fetch a short-lived token from your backend). Omit it inside a raw app. |
|
||||
| `history` | `'server'`, `'local'` or `'none'`, see [History](#history). Defaults to `'server'` with a viewer session and `'local'` with an explicit `token`. |
|
||||
| `inputs` | Extra flow inputs sent with every message. `sendMessage(text, { inputs })` adds per-message ones, and `{ attachments, attachmentsInput }` files, see [Attachments](#attachments). |
|
||||
| `inputs` | Extra flow inputs sent with every message. `sendMessage(text, { inputs })` adds per-message ones. |
|
||||
| `storageKey` | Namespace for `local` history, e.g. the signed-in user's id. Local history is per browser and per flow; without it, users sharing a browser share it. |
|
||||
| `fetch`, `storage` | Replacements for the globals, for tests and unusual runtimes. |
|
||||
| `pageSize` | Messages and conversations per page of server history. Default 50. |
|
||||
@@ -226,41 +224,12 @@ A turn goes `submitted` (the flow is queued) → `streaming` (the answer is arri
|
||||
answer, an `assistant` message with `success: false`. `status: 'error'` (with `error`
|
||||
set) means the turn could not run or be followed at all, such as a refused request.
|
||||
|
||||
Methods: `sendMessage(text, { inputs?, attachments?, attachmentsInput? })`, `stop()`,
|
||||
`newConversation()`, `selectConversation(id)`, `loadConversations({ page?, perPage?, kind? })`,
|
||||
`deleteConversation(id)`, `renameConversation(id, title)`, `loadOlderMessages()`,
|
||||
`destroy()`. `kind` lists the flow editor's test chats (`'test'`), the deployed flow's
|
||||
own (`'deployed'`, the server's default) or both (`'all'`); each `Conversation` carries
|
||||
`isTest`. A rename keeps the conversation's place in the list. Switching conversations
|
||||
Methods: `sendMessage(text, { inputs? })`, `stop()`, `newConversation()`,
|
||||
`selectConversation(id)`, `loadConversations({ page?, perPage? })`,
|
||||
`deleteConversation(id)`, `loadOlderMessages()`, `destroy()`. Switching conversations
|
||||
stops following the current answer; the flow keeps running and, with server history,
|
||||
its answer is there when you come back.
|
||||
|
||||
## Attachments
|
||||
|
||||
A flow whose AI agent step reads `user_attachments` from an `s3object[]` (or a single
|
||||
`s3object`) flow input takes files with a message:
|
||||
|
||||
```ts
|
||||
await chat.sendMessage('What does this contract say?', {
|
||||
attachments: [{ name: file.name, data: file }], // a Blob/File, or a `data:` URL
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
```
|
||||
|
||||
Each file is uploaded to the workspace's object storage under
|
||||
`windmill_uploads/chat/<turn>/<index>/<name>` and handed to the input as `{ s3, filename }`
|
||||
objects (the object for a single-file input). Once the uploads return, the pending user
|
||||
message lists them in `attachments`, as `{ input, s3, filename }` references. The name's
|
||||
extension is corrected to the file's media type for PNG, JPEG and PDF, because the worker
|
||||
reads the type off the key.
|
||||
Files need message text to go with them. A failed upload rejects `sendMessage` before any
|
||||
run starts, and `stop()` during the upload aborts it; both leave the transcript as it was.
|
||||
The chat never deletes uploads, so files of a send that did not run stay in storage. The
|
||||
workspace needs object storage set up. With Enterprise advanced storage permissions, the
|
||||
user needs read and write on `windmill_uploads/*`, which the default rules grant. The upload goes through
|
||||
`job_helpers`, so a restricted token needs `job_helpers:write`; a sandboxed raw app cannot
|
||||
request that scope today, so attachments are not available there yet.
|
||||
|
||||
## History
|
||||
|
||||
Windmill stores every conversation of a chat-mode flow, and each Windmill user sees
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "windmill-chat",
|
||||
"version": "1.814.0",
|
||||
"version": "1.813.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "windmill-chat",
|
||||
"version": "1.814.0",
|
||||
"version": "1.813.0",
|
||||
"license": "Apache-2.0",
|
||||
"devDependencies": {
|
||||
"@ai-sdk/react": "^4.0.102",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "windmill-chat",
|
||||
"description": "Build chat interfaces on Windmill flows deployed in chat mode, from any frontend or raw app",
|
||||
"version": "1.814.0",
|
||||
"version": "1.813.0",
|
||||
"author": "Ruben Fiszel",
|
||||
"license": "Apache-2.0",
|
||||
"homepage": "https://github.com/windmill-labs/windmill/tree/main/chat-sdk#readme",
|
||||
|
||||
+10
-35
@@ -1,10 +1,5 @@
|
||||
import type { ChatTransport, UIMessage, UIMessageChunk, UIMessagePart } from 'ai'
|
||||
import {
|
||||
WindmillApiError,
|
||||
WindmillChatApi,
|
||||
type FlowConversationMessage,
|
||||
type WindmillChatApiOptions
|
||||
} from './api'
|
||||
import { WindmillApiError, WindmillChatApi, type WindmillChatApiOptions } from './api'
|
||||
import { followJob } from './follow'
|
||||
import type { AgentStreamEvent } from './stream'
|
||||
import type { ChatMessage, Conversation } from './types'
|
||||
@@ -106,9 +101,7 @@ export function createWindmillChatTransport<UI_MESSAGE extends UIMessage = UIMes
|
||||
stepName: row.step_name ?? undefined,
|
||||
pending: false,
|
||||
seq: row.created_seq,
|
||||
reasoning: row.reasoning ?? undefined,
|
||||
attachments: row.attachments ?? undefined,
|
||||
tool: toolFromRow(row)
|
||||
tool: toolFromRowContent(row.message_type, row.content, row.success ?? true)
|
||||
}))
|
||||
) as UI_MESSAGE[]
|
||||
},
|
||||
@@ -130,20 +123,10 @@ export function createWindmillChatTransport<UI_MESSAGE extends UIMessage = UIMes
|
||||
}
|
||||
}
|
||||
|
||||
/** The call a stored tool row carries: its tool, named by the sentence the worker words
|
||||
* every tool row from, and the model's arguments and what it got back — for a failed
|
||||
* tool, the result is what it failed with. */
|
||||
function toolFromRow(row: FlowConversationMessage): ChatMessage['tool'] {
|
||||
if (row.message_type !== 'tool') return undefined
|
||||
const name =
|
||||
/^Used (.+) tool$/.exec(row.content)?.[1] ?? /^Error executing (.+)$/.exec(row.content)?.[1]
|
||||
if (!name) return undefined
|
||||
return {
|
||||
name,
|
||||
status: (row.success ?? true) ? 'success' : 'error',
|
||||
arguments: row.tool_arguments ?? undefined,
|
||||
result: row.tool_result ?? undefined
|
||||
}
|
||||
function toolFromRowContent(role: string, content: string, success: boolean): ChatMessage['tool'] {
|
||||
if (role !== 'tool') return undefined
|
||||
const name = /^Used (.+) tool$/.exec(content)?.[1] ?? /^Error executing (.+)$/.exec(content)?.[1]
|
||||
return name ? { name, status: success ? 'success' : 'error' } : undefined
|
||||
}
|
||||
|
||||
/** Streams a job's answer as AI SDK chunks; resumes from `entry.offset` when the job is already running. */
|
||||
@@ -309,21 +292,13 @@ class PartWriter {
|
||||
|
||||
/**
|
||||
* `ChatMessage`s (Windmill's role-per-row model) as `UIMessage`s: an assistant
|
||||
* turn becomes one message whose parts carry its text, reasoning and tool calls. A
|
||||
* user message's attachments ride in `metadata.attachments`, as references for
|
||||
* `WindmillChatApi.attachmentUrl`: a `file` part would need a URL the browser can
|
||||
* load unauthenticated.
|
||||
* turn becomes one message whose parts carry its text, reasoning and tool calls.
|
||||
*/
|
||||
export function toUIMessages(messages: ChatMessage[]): UIMessage[] {
|
||||
const out: UIMessage[] = []
|
||||
for (const m of messages) {
|
||||
if (m.role === 'user' || m.role === 'system') {
|
||||
out.push({
|
||||
id: m.id,
|
||||
role: m.role,
|
||||
...(m.attachments?.length ? { metadata: { attachments: m.attachments } } : {}),
|
||||
parts: [{ type: 'text', text: m.content }]
|
||||
})
|
||||
out.push({ id: m.id, role: m.role, parts: [{ type: 'text', text: m.content }] })
|
||||
continue
|
||||
}
|
||||
let target = out[out.length - 1]
|
||||
@@ -331,18 +306,18 @@ export function toUIMessages(messages: ChatMessage[]): UIMessage[] {
|
||||
target = { id: m.id, role: 'assistant', parts: [] }
|
||||
out.push(target)
|
||||
}
|
||||
if (m.reasoning) target.parts.push({ type: 'reasoning', text: m.reasoning, state: 'done' })
|
||||
if (m.role === 'tool') {
|
||||
const toolCallId = m.tool?.callId ?? m.id
|
||||
const toolName = m.tool?.name ?? 'tool'
|
||||
const input = parseJsonOr(m.tool?.arguments)
|
||||
target.parts.push(
|
||||
m.success
|
||||
? { type: 'dynamic-tool', toolName, toolCallId, state: 'output-available', input, output: m.tool?.result !== undefined ? parseJsonOr(m.tool.result) : m.content }
|
||||
? { type: 'dynamic-tool', toolName, toolCallId, state: 'output-available', input, output: parseJsonOr(m.tool?.result) ?? m.content }
|
||||
: { type: 'dynamic-tool', toolName, toolCallId, state: 'output-error', input, errorText: m.tool?.result ?? m.content }
|
||||
)
|
||||
continue
|
||||
}
|
||||
if (m.reasoning) target.parts.push({ type: 'reasoning', text: m.reasoning, state: 'done' })
|
||||
if (m.content) target.parts.push({ type: 'text', text: m.content, state: 'done' })
|
||||
}
|
||||
return out
|
||||
|
||||
+4
-67
@@ -1,4 +1,4 @@
|
||||
import type { ChatAttachment, FetchLike, TokenSource } from './types'
|
||||
import type { FetchLike, TokenSource } from './types'
|
||||
|
||||
export interface WindmillChatApiOptions {
|
||||
baseUrl: string
|
||||
@@ -28,16 +28,8 @@ export interface FlowConversation {
|
||||
created_at: string
|
||||
updated_at: string
|
||||
created_by: string
|
||||
/** Started from the flow editor's test panel rather than a deployed run. */
|
||||
is_test: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Which conversations a listing holds: the flow editor's test chats, the deployed flow's
|
||||
* own (the server's default), or both.
|
||||
*/
|
||||
export type ConversationKind = 'test' | 'deployed' | 'all'
|
||||
|
||||
export interface FlowConversationMessage {
|
||||
id: string
|
||||
conversation_id: string
|
||||
@@ -48,14 +40,6 @@ export interface FlowConversationMessage {
|
||||
created_seq: number
|
||||
step_name?: string | null
|
||||
success?: boolean
|
||||
/** On a tool row, the arguments the model wrote, without the inputs a step wires in; null for a web search. */
|
||||
tool_arguments?: string | null
|
||||
/** On a tool row, the text the model got back or what the call failed with; a web search's citations. */
|
||||
tool_result?: string | null
|
||||
/** On an answer, the thinking that produced it; on a tool row, the thinking that led to the call. */
|
||||
reasoning?: string | null
|
||||
/** The files a user message carried, as object-storage references. */
|
||||
attachments?: ChatAttachment[] | null
|
||||
}
|
||||
|
||||
export type JobUpdateEvent =
|
||||
@@ -174,17 +158,6 @@ export class WindmillChatApi {
|
||||
return (await res.json()) as FlowJobStatus
|
||||
}
|
||||
|
||||
/**
|
||||
* Where a message's attachment downloads from. The endpoint authenticates like every other
|
||||
* request: a consumer holding a token must fetch it with that token, not put the URL in an
|
||||
* `img src`, which would send only the Windmill session cookie.
|
||||
*/
|
||||
attachmentUrl(attachment: ChatAttachment): string {
|
||||
const query = new URLSearchParams({ file_key: attachment.s3 })
|
||||
if (attachment.storage) query.set('storage', attachment.storage)
|
||||
return `${this.#baseUrl}/api/w/${encodeURIComponent(this.#workspace)}/job_helpers/download_s3_file?${query}`
|
||||
}
|
||||
|
||||
async cancelJob(jobId: string, reason = 'Stopped from the chat'): Promise<void> {
|
||||
await this.#request(`jobs_u/queue/cancel/${encodeURIComponent(jobId)}`, {
|
||||
method: 'POST',
|
||||
@@ -194,25 +167,15 @@ export class WindmillChatApi {
|
||||
|
||||
async listConversations(
|
||||
flowPath: string,
|
||||
options: { page?: number; perPage?: number; kind?: ConversationKind; signal?: AbortSignal } = {}
|
||||
options: { page?: number; perPage?: number; signal?: AbortSignal } = {}
|
||||
): Promise<FlowConversation[]> {
|
||||
const extra: Record<string, string> = { flow_path: flowPath }
|
||||
if (options.kind !== undefined) extra.kind = options.kind
|
||||
const res = await this.#request('flow_conversations/list', {
|
||||
query: pagination(options, extra),
|
||||
query: pagination(options, { flow_path: flowPath }),
|
||||
signal: options.signal
|
||||
})
|
||||
return (await res.json()) as FlowConversation[]
|
||||
}
|
||||
|
||||
/** Sets a conversation's title. Its place in the list is kept: only a turn moves one. */
|
||||
async renameConversation(conversationId: string, title: string): Promise<void> {
|
||||
await this.#request(`flow_conversations/update/${encodeURIComponent(conversationId)}`, {
|
||||
method: 'POST',
|
||||
body: { title }
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Without `afterSeq`: one page counted from the newest message, returned oldest first.
|
||||
* With `afterSeq`: the messages created after that cursor, oldest first.
|
||||
@@ -230,27 +193,6 @@ export class WindmillChatApi {
|
||||
return (await res.json()) as FlowConversationMessage[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Puts bytes in the workspace's object storage under `fileKey` and returns the key they were
|
||||
* stored under (the server may rewrite it). Needs the workspace to have object storage set.
|
||||
*/
|
||||
async uploadFile(
|
||||
fileKey: string,
|
||||
body: Blob,
|
||||
options: { contentType?: string; signal?: AbortSignal } = {}
|
||||
): Promise<{ file_key: string }> {
|
||||
const query: Record<string, string> = { file_key: fileKey }
|
||||
if (options.contentType) query.content_type = options.contentType
|
||||
const res = await this.#request('job_helpers/upload_s3_file', {
|
||||
method: 'POST',
|
||||
query,
|
||||
raw: body,
|
||||
contentType: options.contentType || 'application/octet-stream',
|
||||
signal: options.signal
|
||||
})
|
||||
return (await res.json()) as { file_key: string }
|
||||
}
|
||||
|
||||
async deleteConversation(conversationId: string): Promise<void> {
|
||||
await this.#request(`flow_conversations/delete/${encodeURIComponent(conversationId)}`, {
|
||||
method: 'DELETE'
|
||||
@@ -262,11 +204,7 @@ export class WindmillChatApi {
|
||||
init: {
|
||||
method?: string
|
||||
query?: Record<string, string>
|
||||
/** JSON-encoded. */
|
||||
body?: unknown
|
||||
/** Sent as is, under `contentType`. */
|
||||
raw?: Blob
|
||||
contentType?: string
|
||||
accept?: string
|
||||
signal?: AbortSignal
|
||||
} = {}
|
||||
@@ -277,14 +215,13 @@ export class WindmillChatApi {
|
||||
const headers: Record<string, string> = {}
|
||||
if (init.accept) headers['Accept'] = init.accept
|
||||
if (init.body !== undefined) headers['Content-Type'] = 'application/json'
|
||||
else if (init.raw !== undefined) headers['Content-Type'] = init.contentType ?? 'application/octet-stream'
|
||||
const token = typeof this.#token === 'function' ? await this.#token() : this.#token
|
||||
if (token) headers['Authorization'] = `Bearer ${token}`
|
||||
|
||||
const res = await this.#fetch(url.toString(), {
|
||||
method: init.method ?? 'GET',
|
||||
headers,
|
||||
body: init.body === undefined ? init.raw : JSON.stringify(init.body),
|
||||
body: init.body === undefined ? undefined : JSON.stringify(init.body),
|
||||
// A token must not be paired with ambient cookies; without one, the cookie is
|
||||
// the credential and only rides same-origin requests.
|
||||
credentials: token ? 'omit' : 'same-origin',
|
||||
|
||||
@@ -53,8 +53,7 @@ export function useWindmillRuntime(options: WindmillRuntimeOptions): AssistantRu
|
||||
threads: chat.conversations.map((c) => ({ status: 'regular' as const, id: c.id, title: c.title })),
|
||||
onSwitchToNewThread: () => chat.newConversation(),
|
||||
onSwitchToThread: (id) => chat.selectConversation(id),
|
||||
onDelete: (id) => chat.deleteConversation(id),
|
||||
onRename: (id, title) => chat.renameConversation(id, title)
|
||||
onDelete: (id) => chat.deleteConversation(id)
|
||||
}
|
||||
}
|
||||
: undefined
|
||||
@@ -92,7 +91,6 @@ export function toThreadMessage(turn: WindmillTurn): ThreadMessageLike {
|
||||
}
|
||||
const content: ThreadContentPart[] = []
|
||||
for (const m of turn.messages) {
|
||||
if (m.reasoning) content.push({ type: 'reasoning', text: m.reasoning })
|
||||
if (m.role === 'tool') {
|
||||
const args = parseJsonOr(m.tool?.arguments)
|
||||
content.push({
|
||||
@@ -101,12 +99,12 @@ export function toThreadMessage(turn: WindmillTurn): ThreadMessageLike {
|
||||
toolName: m.tool?.name ?? 'tool',
|
||||
args: (isJsonObject(args) ? args : args === undefined ? {} : { input: args }) as ToolCallArgs,
|
||||
argsText: m.tool?.arguments ?? '',
|
||||
result:
|
||||
m.tool?.status === 'running' ? undefined : m.tool?.result !== undefined ? parseJsonOr(m.tool.result) : m.content,
|
||||
result: m.tool?.status === 'running' ? undefined : (parseJsonOr(m.tool?.result) ?? m.content),
|
||||
isError: m.tool?.status === 'error'
|
||||
})
|
||||
continue
|
||||
}
|
||||
if (m.reasoning) content.push({ type: 'reasoning', text: m.reasoning })
|
||||
if (m.content) content.push({ type: 'text', text: m.content })
|
||||
}
|
||||
const last = turn.messages[turn.messages.length - 1]
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
import type { WindmillChatApi } from './api'
|
||||
import type { AttachmentUpload } from './types'
|
||||
import { abortError, isAbortError } from './utils'
|
||||
|
||||
/**
|
||||
* Where a chat's uploads live in the workspace's object storage. Under `windmill_uploads/`
|
||||
* because the default Enterprise storage permissions grant every user write and read there
|
||||
* and deny any other top-level prefix: a key outside it is refused for non-admins, both on
|
||||
* upload and when the agent's job reads the file back.
|
||||
*/
|
||||
export const CHAT_UPLOADS_PREFIX = 'windmill_uploads/chat'
|
||||
|
||||
/** What an AI agent step reads out of `user_attachments`. */
|
||||
export interface UploadedAttachment {
|
||||
s3: string
|
||||
filename: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The extension each type must be stored under. The worker reads an attachment's media type
|
||||
* from the key's extension only (`mime_guess` in `windmill-ai/src/image_handler.rs`, falling
|
||||
* back to `image/png`), never from the stored content type, so the extension must be true.
|
||||
*/
|
||||
const EXTENSION_BY_MEDIA_TYPE: Record<string, string> = {
|
||||
'image/png': 'png',
|
||||
'image/jpeg': 'jpg',
|
||||
'application/pdf': 'pdf'
|
||||
}
|
||||
|
||||
/**
|
||||
* The name an attachment is stored under: the picked name with the extension its media type
|
||||
* needs, e.g. a `photo.webp` re-encoded to PNG becomes `photo.png`. Other types keep their name.
|
||||
*/
|
||||
export function storedAttachmentName(filename: string, mediaType: string): string {
|
||||
const extension = EXTENSION_BY_MEDIA_TYPE[mediaType]
|
||||
if (!extension) return filename
|
||||
const stem = filename.replace(/\.[^./]+$/, '')
|
||||
return `${stem || filename}.${extension}`
|
||||
}
|
||||
|
||||
/** The bytes of an attachment as a Blob carrying its media type. */
|
||||
export function attachmentBlob(attachment: AttachmentUpload): Blob {
|
||||
const data =
|
||||
typeof attachment.data === 'string'
|
||||
? dataUrlToBlob(attachment.data, 'application/octet-stream')
|
||||
: attachment.data
|
||||
return attachment.mediaType && attachment.mediaType !== data.type
|
||||
? new Blob([data], { type: attachment.mediaType })
|
||||
: data
|
||||
}
|
||||
|
||||
function dataUrlToBlob(dataUrl: string, fallbackType: string): Blob {
|
||||
const comma = dataUrl.indexOf(',')
|
||||
if (!dataUrl.startsWith('data:') || comma === -1) {
|
||||
throw new Error('windmill-chat: an attachment given as a string must be a data: URL')
|
||||
}
|
||||
const header = dataUrl.slice(5, comma)
|
||||
const isBase64 = header.endsWith(';base64')
|
||||
const mediaType = (isBase64 ? header.slice(0, -';base64'.length) : header) || fallbackType
|
||||
const payload = dataUrl.slice(comma + 1)
|
||||
if (!isBase64) return new Blob([decodeURIComponent(payload)], { type: mediaType })
|
||||
const binary = atob(payload)
|
||||
const bytes = new Uint8Array(binary.length)
|
||||
for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i)
|
||||
return new Blob([bytes], { type: mediaType })
|
||||
}
|
||||
|
||||
/**
|
||||
* Put each attachment in the workspace's object storage and hand back what the agent reads.
|
||||
* The key's turn prefix and per-file index keep two files with the same name, in this turn or
|
||||
* an earlier one, from overwriting each other; the name stays the last segment.
|
||||
*/
|
||||
export async function uploadAttachments(
|
||||
api: WindmillChatApi,
|
||||
attachments: AttachmentUpload[],
|
||||
turnId: string,
|
||||
signal?: AbortSignal
|
||||
): Promise<UploadedAttachment[]> {
|
||||
const prefix = `${CHAT_UPLOADS_PREFIX}/${turnId}`
|
||||
// One failed upload aborts the rest. Nothing already stored is deleted: the chat never
|
||||
// removes objects from the workspace's storage, so a send that does not run leaves them.
|
||||
if (signal?.aborted) throw abortError()
|
||||
const batch = new AbortController()
|
||||
const abortBatch = () => batch.abort()
|
||||
signal?.addEventListener('abort', abortBatch, { once: true })
|
||||
try {
|
||||
const results = await Promise.allSettled(
|
||||
attachments.map(async (attachment, index) => {
|
||||
try {
|
||||
const blob = attachmentBlob(attachment)
|
||||
const filename = storedAttachmentName(
|
||||
attachment.name || `attachment-${index + 1}`,
|
||||
blob.type
|
||||
)
|
||||
const { file_key } = await api.uploadFile(`${prefix}/${index}/${filename}`, blob, {
|
||||
contentType: blob.type,
|
||||
signal: batch.signal
|
||||
})
|
||||
return { s3: file_key, filename }
|
||||
} catch (e) {
|
||||
batch.abort()
|
||||
throw e
|
||||
}
|
||||
})
|
||||
)
|
||||
const reasons = results.flatMap((r) => (r.status === 'rejected' ? [r.reason] : []))
|
||||
// A stop that lands once every upload has answered still withdraws the batch.
|
||||
if (reasons.length === 0 && !signal?.aborted) {
|
||||
return results.flatMap((r) => (r.status === 'fulfilled' ? [r.value] : []))
|
||||
}
|
||||
// The failure that started it, not the aborts it caused in the other uploads.
|
||||
throw reasons.find((reason) => !isAbortError(reason)) ?? reasons[0] ?? abortError()
|
||||
} finally {
|
||||
signal?.removeEventListener('abort', abortBatch)
|
||||
}
|
||||
}
|
||||
+20
-184
@@ -1,14 +1,12 @@
|
||||
import {
|
||||
WindmillApiError,
|
||||
WindmillChatApi,
|
||||
type ConversationKind,
|
||||
type FlowConversation,
|
||||
type FlowConversationMessage
|
||||
} from './api'
|
||||
import { resolveConfig, type ResolvedConfig } from './config'
|
||||
import { followJob } from './follow'
|
||||
import { createLocalHistory, type LocalHistory } from './history'
|
||||
import { uploadAttachments } from './attachments'
|
||||
import type { AgentStreamEvent } from './stream'
|
||||
import type {
|
||||
Chat,
|
||||
@@ -16,15 +14,12 @@ import type {
|
||||
ChatOptions,
|
||||
ChatState,
|
||||
Conversation,
|
||||
SendMessageOptions,
|
||||
ToolInvocation
|
||||
} from './types'
|
||||
import {
|
||||
conversationTitle,
|
||||
truncateTitle,
|
||||
errorResultMessage,
|
||||
extractChatAnswer,
|
||||
abortError,
|
||||
isAbortError,
|
||||
isErrorResult,
|
||||
now,
|
||||
@@ -44,12 +39,6 @@ interface Turn {
|
||||
conversationId: string
|
||||
/** Id of the turn's user message; the answer is whatever follows it. */
|
||||
userMessageId: string
|
||||
/** The turn opened the conversation; withdrawing it closes the conversation again. */
|
||||
isNew: boolean
|
||||
/** The run was asked for. Before that, a failure or a stop withdraws the turn instead of failing it. */
|
||||
started: boolean
|
||||
/** Both `stop()` and the send's own rejection withdraw; only the first may. */
|
||||
withdrawn: boolean
|
||||
jobId?: string
|
||||
/** The flow job and its step jobs; a persisted answer carries one of them as `job_id`. */
|
||||
jobIds?: Set<string>
|
||||
@@ -70,8 +59,6 @@ class ChatImpl implements Chat {
|
||||
#state: ChatState
|
||||
#turn: Turn | undefined
|
||||
#page = 1
|
||||
/** The kind the caller last listed, so the refresh after a new turn lists the same rows. */
|
||||
#conversationKind: ConversationKind | undefined
|
||||
#persistTimer: ReturnType<typeof setTimeout> | undefined
|
||||
|
||||
constructor(options: ChatOptions) {
|
||||
@@ -110,34 +97,21 @@ class ChatImpl implements Chat {
|
||||
}
|
||||
}
|
||||
|
||||
sendMessage = async (text: string, options: SendMessageOptions = {}): Promise<void> => {
|
||||
sendMessage = async (
|
||||
text: string,
|
||||
options: { inputs?: Record<string, unknown> } = {}
|
||||
): Promise<void> => {
|
||||
const content = text.trim()
|
||||
if (!content) {
|
||||
// A run needs a message; files alone would otherwise be dropped without a word.
|
||||
if (options.attachments?.length) throw new Error('windmill-chat: attachments need a message to go with them')
|
||||
return
|
||||
}
|
||||
if (!content) return
|
||||
if (this.#turn) {
|
||||
throw new Error('windmill-chat: a message is already being answered; call stop() first')
|
||||
}
|
||||
const attachments = options.attachments ?? []
|
||||
const attachmentsInput = options.attachmentsInput
|
||||
if (attachments.length > 0 && !attachmentsInput) {
|
||||
throw new Error('windmill-chat: attachments need `attachmentsInput`, the flow input that takes them')
|
||||
}
|
||||
if (attachmentsInput && !attachmentsInput.multiple && attachments.length > 1) {
|
||||
// Uploading all of them would run with the first and leave the rest stranded in storage.
|
||||
throw new Error(`windmill-chat: \`${attachmentsInput.name}\` holds one file; got ${attachments.length}`)
|
||||
}
|
||||
const isNew = this.#state.conversationId === undefined
|
||||
const conversationId = this.#state.conversationId ?? randomId()
|
||||
const turn: Turn = {
|
||||
controller: new AbortController(),
|
||||
conversationId,
|
||||
userMessageId: `pending-${randomId()}`,
|
||||
isNew,
|
||||
started: false,
|
||||
withdrawn: false,
|
||||
streamedText: false
|
||||
}
|
||||
this.#turn = turn
|
||||
@@ -149,6 +123,7 @@ class ChatImpl implements Chat {
|
||||
const touched = { ...conversation, updatedAt: timestamp }
|
||||
this.#set({
|
||||
conversationId,
|
||||
conversations: [touched, ...this.#state.conversations.filter((c) => c.id !== conversationId)],
|
||||
messages: [
|
||||
...this.#state.messages,
|
||||
{ id: turn.userMessageId, role: 'user', content, success: true, createdAt: timestamp, pending: true }
|
||||
@@ -156,32 +131,10 @@ class ChatImpl implements Chat {
|
||||
status: 'submitted',
|
||||
error: undefined
|
||||
})
|
||||
this.#rememberConversation()
|
||||
|
||||
try {
|
||||
const args: Record<string, unknown> = { ...this.#config.inputs, ...options.inputs, user_message: content }
|
||||
if (attachmentsInput && attachments.length > 0) {
|
||||
// Uploaded with the turn already shown as submitted: the message is in the transcript
|
||||
// and `stop()` can abort the upload, while a second send is refused as usual.
|
||||
const uploaded = await uploadAttachments(this.#api, attachments, randomId(), turn.controller.signal)
|
||||
args[attachmentsInput.name] = attachmentsInput.multiple ? uploaded : uploaded[0]
|
||||
// Shown on the pending message until its server row replaces it, carrying its own.
|
||||
const carried = uploaded.map((u) => ({ input: attachmentsInput.name, s3: u.s3, filename: u.filename }))
|
||||
if (this.#turnActive(turn)) {
|
||||
this.#set({
|
||||
messages: this.#state.messages.map((m) => (m.id === turn.userMessageId ? { ...m, attachments: carried } : m))
|
||||
})
|
||||
}
|
||||
}
|
||||
// Nothing may start once stop() or a conversation switch has withdrawn the turn, including
|
||||
// a stop from a subscriber told of the attachments just above.
|
||||
if (turn.controller.signal.aborted) throw abortError()
|
||||
turn.started = true
|
||||
// Listed only once the run is asked for: a send that never runs (an upload that failed
|
||||
// or was stopped) then has no conversation entry to take back.
|
||||
this.#set({
|
||||
conversations: [touched, ...this.#state.conversations.filter((c) => c.id !== conversationId)]
|
||||
})
|
||||
this.#rememberConversation()
|
||||
const args = { ...this.#config.inputs, ...options.inputs, user_message: content }
|
||||
const context = { memoryId: conversationId, conversationId, signal: turn.controller.signal }
|
||||
turn.jobId = this.#config.run
|
||||
? await this.#config.run(args, context)
|
||||
@@ -195,13 +148,6 @@ class ChatImpl implements Chat {
|
||||
}
|
||||
await this.#finishTurn(turn, result, isNew)
|
||||
} catch (e) {
|
||||
if (!turn.started) {
|
||||
// Nothing ran: the message is withdrawn rather than shown as a failed turn, and the
|
||||
// caller gets the reason (an upload that failed, or the AbortError of a stop()).
|
||||
if (this.#turn === turn) this.#turn = undefined
|
||||
this.#withdrawTurn(turn)
|
||||
throw e
|
||||
}
|
||||
// stop() and a conversation switch abort the turn and settle the state themselves.
|
||||
if (turn.controller.signal.aborted || isAbortError(e)) return
|
||||
this.#failTurn(turn, e)
|
||||
@@ -214,12 +160,6 @@ class ChatImpl implements Chat {
|
||||
const turn = this.#turn
|
||||
if (!turn) return
|
||||
this.#detachTurn()
|
||||
if (!turn.started) {
|
||||
// Still uploading its attachments: there is no run to cancel, and the message the
|
||||
// reader took back must not stay in the transcript as sent.
|
||||
this.#withdrawTurn(turn)
|
||||
return
|
||||
}
|
||||
if (this.#state.conversationId === turn.conversationId) {
|
||||
this.#set({ messages: finalized(this.#state.messages), status: 'idle' })
|
||||
this.#persistLocal()
|
||||
@@ -289,21 +229,15 @@ class ChatImpl implements Chat {
|
||||
}
|
||||
|
||||
loadConversations = async (
|
||||
options: { page?: number; perPage?: number; kind?: ConversationKind } = {}
|
||||
options: { page?: number; perPage?: number } = {}
|
||||
): Promise<Conversation[]> => {
|
||||
const page = options.page ?? 1
|
||||
// A different kind is a different listing: its first rows replace the held ones, on
|
||||
// whichever page they were asked for.
|
||||
const kindChanged = 'kind' in options && options.kind !== this.#conversationKind
|
||||
if ('kind' in options) this.#conversationKind = options.kind
|
||||
const kind = this.#conversationKind
|
||||
let conversations: Conversation[]
|
||||
if (this.#state.history === 'server') {
|
||||
try {
|
||||
const rows = await this.#api.listConversations(this.#config.flowPath, {
|
||||
page,
|
||||
perPage: options.perPage ?? this.#config.pageSize,
|
||||
kind
|
||||
perPage: options.perPage ?? this.#config.pageSize
|
||||
})
|
||||
conversations = rows.map(fromConversation)
|
||||
} catch (e) {
|
||||
@@ -313,13 +247,10 @@ class ChatImpl implements Chat {
|
||||
} else {
|
||||
conversations = this.#state.history === 'local' ? this.#local.listConversations() : []
|
||||
}
|
||||
// Another kind was asked for while this list was on its way: its rows are not the
|
||||
// listing any more, whichever response lands last.
|
||||
if (kind !== this.#conversationKind) return conversations
|
||||
const known = new Set(this.#state.conversations.map((c) => c.id))
|
||||
this.#set({
|
||||
conversations:
|
||||
page === 1 || kindChanged
|
||||
page === 1
|
||||
? conversations
|
||||
: [...this.#state.conversations, ...conversations.filter((c) => !known.has(c.id))]
|
||||
})
|
||||
@@ -342,24 +273,6 @@ class ChatImpl implements Chat {
|
||||
this.#set({ conversations: this.#state.conversations.filter((c) => c.id !== conversationId) })
|
||||
}
|
||||
|
||||
renameConversation = async (conversationId: string, title: string): Promise<void> => {
|
||||
// Cut here as the server cuts, so the title shown is the one stored.
|
||||
const trimmed = truncateTitle(title.trim())
|
||||
if (!trimmed) return
|
||||
if (this.#state.history === 'server') {
|
||||
await this.#api.renameConversation(conversationId, trimmed)
|
||||
} else if (this.#state.history === 'local') {
|
||||
this.#local.renameConversation(conversationId, trimmed)
|
||||
}
|
||||
// Patched in place: the server keeps `updated_at` on a rename, so the list order the
|
||||
// next load returns is the one shown now.
|
||||
this.#set({
|
||||
conversations: this.#state.conversations.map((c) =>
|
||||
c.id === conversationId ? { ...c, title: trimmed } : c
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
loadOlderMessages = async (): Promise<void> => {
|
||||
const conversationId = this.#state.conversationId
|
||||
if (
|
||||
@@ -431,21 +344,16 @@ class ChatImpl implements Chat {
|
||||
tool: { ...existing.tool!, ...toolPatch }
|
||||
}
|
||||
} else {
|
||||
// Thinking that produced no text led to this call, and is stored on its row.
|
||||
const a = turn.assistantId ? messages.findIndex((m) => m.id === turn.assistantId) : -1
|
||||
const reasoning = a >= 0 && messages[a].content === '' ? messages.splice(a, 1)[0].reasoning : undefined
|
||||
messages.push({
|
||||
id: `pending-${randomId()}`,
|
||||
role: 'tool',
|
||||
content: content ?? '',
|
||||
reasoning,
|
||||
success: success ?? true,
|
||||
createdAt: now(),
|
||||
pending: true,
|
||||
tool: { callId, name, status: 'running', ...toolPatch }
|
||||
})
|
||||
}
|
||||
turn.assistantId = undefined
|
||||
}
|
||||
const appendAssistant = (text: string, reasoning: string) => {
|
||||
const i = turn.assistantId
|
||||
@@ -480,14 +388,17 @@ class ChatImpl implements Chat {
|
||||
case 'reasoning_token_delta':
|
||||
appendAssistant('', event.content)
|
||||
break
|
||||
// A call completes the round's text: text after it is a new message.
|
||||
case 'tool_call':
|
||||
// The round's text is complete; text after the tool result is a new message.
|
||||
turn.assistantId = undefined
|
||||
upsertTool(event.call_id, event.function_name, { status: 'running' })
|
||||
break
|
||||
case 'tool_call_arguments':
|
||||
turn.assistantId = undefined
|
||||
upsertTool(event.call_id, event.function_name, { arguments: event.arguments })
|
||||
break
|
||||
case 'tool_execution':
|
||||
turn.assistantId = undefined
|
||||
upsertTool(event.call_id, event.function_name, { status: 'running' })
|
||||
break
|
||||
case 'tool_result':
|
||||
@@ -634,36 +545,6 @@ class ChatImpl implements Chat {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Take back the user message of a turn that never ran. A conversation it would have opened
|
||||
* was never listed (see `sendMessage`), so only the message goes, and, while it is the turn
|
||||
* on screen, the busy status. A switch away mid-upload has already written the message to
|
||||
* local history, so it is removed there too.
|
||||
*/
|
||||
#withdrawTurn(turn: Turn): void {
|
||||
if (turn.withdrawn) return
|
||||
turn.withdrawn = true
|
||||
const id = turn.conversationId
|
||||
const withoutTurn = (messages: ChatMessage[]) => messages.filter((m) => m.id !== turn.userMessageId)
|
||||
if (this.#state.conversationId === id) {
|
||||
const messages = withoutTurn(this.#state.messages)
|
||||
// A turn started since, such as a resend right after Stop, owns the status.
|
||||
const newerTurn = this.#turn !== undefined && this.#turn !== turn
|
||||
if (newerTurn) {
|
||||
this.#set({ messages })
|
||||
} else {
|
||||
const unopened = turn.isNew && messages.length === 0
|
||||
this.#set({ messages, status: 'idle', error: undefined, ...(unopened ? { conversationId: undefined } : {}) })
|
||||
}
|
||||
this.#persistLocal()
|
||||
}
|
||||
if (this.#state.history === 'local' && this.#state.conversationId !== id) {
|
||||
const stored = withoutTurn(this.#local.getMessages(id))
|
||||
if (stored.length > 0) this.#local.saveMessages(id, stored)
|
||||
else if (!this.#state.conversations.some((c) => c.id === id)) this.#local.deleteConversation(id)
|
||||
}
|
||||
}
|
||||
|
||||
#failTurn(turn: Turn, e: unknown): void {
|
||||
if (!this.#turnActive(turn)) return
|
||||
const error = toError(e)
|
||||
@@ -731,54 +612,22 @@ class ChatImpl implements Chat {
|
||||
for (const row of rows.map(fromRow)) {
|
||||
if (known.has(row.id)) continue
|
||||
known.add(row.id)
|
||||
let i = messages.findIndex(
|
||||
const i = messages.findIndex(
|
||||
(m) =>
|
||||
m.seq === undefined &&
|
||||
m.role === row.role &&
|
||||
(m.content === row.content || (row.tool !== undefined && m.tool?.name === row.tool.name))
|
||||
)
|
||||
// A structured answer streams as the call of the structured-output tool, whose
|
||||
// arguments are the answer's text: its row replaces that call. Only past the newest
|
||||
// user message, where a stopped turn's identical call cannot be.
|
||||
if (i < 0 && row.role === 'assistant') {
|
||||
let j = messages.length - 1
|
||||
while (j >= 0 && messages[j].role !== 'user') {
|
||||
const m = messages[j]
|
||||
if (m.seq === undefined && m.role === 'tool' && m.tool?.arguments === row.content) i = j
|
||||
j--
|
||||
}
|
||||
}
|
||||
if (i >= 0) {
|
||||
const m = messages[i]
|
||||
messages[i] = {
|
||||
...row,
|
||||
id: m.id,
|
||||
reasoning: m.reasoning ?? row.reasoning,
|
||||
// The stream's call wins where it has a value; a stream cut short leaves gaps the row fills.
|
||||
tool:
|
||||
row.role === 'tool' && m.tool
|
||||
? {
|
||||
...m.tool,
|
||||
arguments: m.tool.arguments ?? row.tool?.arguments,
|
||||
result: m.tool.result ?? row.tool?.result,
|
||||
status: row.tool?.status ?? m.tool.status
|
||||
}
|
||||
: row.tool
|
||||
tool: m.tool ? { ...m.tool, status: row.tool?.status ?? m.tool.status } : row.tool
|
||||
}
|
||||
} else {
|
||||
// A tool row nothing streamed, such as a provider-native web search, goes where a
|
||||
// reload puts it: after the last message of a lower `seq`, before the streamed answer.
|
||||
// Any other row closes the turn, a failure included, and stays last: above a streamed
|
||||
// message that never got a row, it would hide the turn's failure.
|
||||
let at = messages.length
|
||||
for (let j = messages.length - 1; row.role === 'tool' && j >= 0; j--) {
|
||||
const seq = messages[j].seq
|
||||
if (seq !== undefined && seq < row.seq!) {
|
||||
at = j + 1
|
||||
break
|
||||
}
|
||||
}
|
||||
messages.splice(at, 0, row)
|
||||
messages.push(row)
|
||||
}
|
||||
}
|
||||
this.#set({ messages })
|
||||
@@ -876,19 +725,7 @@ function fromRow(row: FlowConversationMessage): ChatMessage {
|
||||
stepName: row.step_name ?? undefined,
|
||||
pending: false,
|
||||
seq: row.created_seq,
|
||||
reasoning: row.reasoning ?? undefined,
|
||||
attachments: row.attachments ?? undefined,
|
||||
// The call the row carries: the model's arguments and what the model got back. For a
|
||||
// failed tool the result is what it failed with, and the row's text names the tool
|
||||
// rather than the reason.
|
||||
tool: toolName
|
||||
? {
|
||||
name: toolName,
|
||||
status: success ? 'success' : 'error',
|
||||
arguments: row.tool_arguments ?? undefined,
|
||||
result: row.tool_result ?? undefined
|
||||
}
|
||||
: undefined
|
||||
tool: toolName ? { name: toolName, status: success ? 'success' : 'error' } : undefined
|
||||
}
|
||||
}
|
||||
|
||||
@@ -897,8 +734,7 @@ function fromConversation(row: FlowConversation): Conversation {
|
||||
id: row.id,
|
||||
title: row.title ?? undefined,
|
||||
createdAt: row.created_at,
|
||||
updatedAt: row.updated_at,
|
||||
isTest: row.is_test
|
||||
updatedAt: row.updated_at
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -4,8 +4,6 @@ export interface LocalHistory {
|
||||
listConversations(): Conversation[]
|
||||
getMessages(conversationId: string): ChatMessage[]
|
||||
upsertConversation(conversation: Conversation): void
|
||||
/** Changes a stored conversation's title in place; unlike `upsertConversation`, its position is kept. */
|
||||
renameConversation(conversationId: string, title: string): void
|
||||
saveMessages(conversationId: string, messages: ChatMessage[]): void
|
||||
deleteConversation(conversationId: string): void
|
||||
}
|
||||
@@ -57,11 +55,6 @@ export function createLocalHistory(storage: StorageLike | undefined, key: string
|
||||
}
|
||||
write(s)
|
||||
},
|
||||
renameConversation(id, title) {
|
||||
const s = read()
|
||||
s.conversations = s.conversations.map((c) => (c.id === id ? { ...c, title } : c))
|
||||
write(s)
|
||||
},
|
||||
saveMessages(id, messages) {
|
||||
const s = read()
|
||||
s.messages[id] = messages.map((m) => ({ ...m, pending: false }))
|
||||
|
||||
@@ -5,7 +5,6 @@ export {
|
||||
WindmillApiError,
|
||||
readServerSentEvents,
|
||||
type WindmillChatApiOptions,
|
||||
type ConversationKind,
|
||||
type FlowConversation,
|
||||
type FlowConversationMessage,
|
||||
type JobUpdateEvent,
|
||||
@@ -14,12 +13,8 @@ export {
|
||||
export { parseStreamEvents, createStreamEventParser, type AgentStreamEvent } from './stream'
|
||||
export { followJob, type FollowEvent } from './follow'
|
||||
export { extractChatAnswer, conversationIdFor } from './utils'
|
||||
export { storedAttachmentName, uploadAttachments, CHAT_UPLOADS_PREFIX, type UploadedAttachment } from './attachments'
|
||||
export type {
|
||||
AttachmentsInput,
|
||||
AttachmentUpload,
|
||||
Chat,
|
||||
ChatAttachment,
|
||||
ChatMessage,
|
||||
ChatOptions,
|
||||
ChatRole,
|
||||
@@ -28,7 +23,6 @@ export type {
|
||||
Conversation,
|
||||
FetchLike,
|
||||
HistoryMode,
|
||||
SendMessageOptions,
|
||||
StorageLike,
|
||||
TokenSource,
|
||||
ToolInvocation
|
||||
|
||||
@@ -11,7 +11,6 @@ export type UseWindmillChat = ChatState &
|
||||
| 'selectConversation'
|
||||
| 'loadConversations'
|
||||
| 'deleteConversation'
|
||||
| 'renameConversation'
|
||||
| 'loadOlderMessages'
|
||||
> & { chat: Chat }
|
||||
|
||||
@@ -66,7 +65,6 @@ export function useWindmillChat(options: ChatOptions): UseWindmillChat {
|
||||
selectConversation: chat.selectConversation,
|
||||
loadConversations: chat.loadConversations,
|
||||
deleteConversation: chat.deleteConversation,
|
||||
renameConversation: chat.renameConversation,
|
||||
loadOlderMessages: chat.loadOlderMessages
|
||||
}),
|
||||
[state, chat]
|
||||
|
||||
+4
-56
@@ -26,23 +26,19 @@ export interface ToolInvocation {
|
||||
status: 'running' | 'success' | 'error'
|
||||
}
|
||||
|
||||
export interface ChatAttachment { input: string; s3: string; storage?: string; filename?: string }
|
||||
|
||||
export interface ChatMessage {
|
||||
id: string
|
||||
role: ChatRole
|
||||
content: string
|
||||
/** The model's reasoning summary, when the provider streams one. */
|
||||
reasoning?: string
|
||||
/** The call on a `tool` message, from the live stream or its stored row; `callId` is only known from the stream. */
|
||||
/** Set on `tool` messages that came from the live stream. */
|
||||
tool?: ToolInvocation
|
||||
success: boolean
|
||||
createdAt: string
|
||||
jobId?: string
|
||||
/** The flow step that produced the message. */
|
||||
stepName?: string
|
||||
/** The files a user message carried, as object-storage references. */
|
||||
attachments?: ChatAttachment[]
|
||||
/** True while the message is optimistic or still streaming. */
|
||||
pending: boolean
|
||||
/** Id of the persisted row once the server has it; `id` itself never changes, so list keys stay stable. */
|
||||
@@ -56,11 +52,6 @@ export interface Conversation {
|
||||
title: string | undefined
|
||||
createdAt: string
|
||||
updatedAt: string
|
||||
/**
|
||||
* Started from the flow editor's test panel rather than a deployed run. Known once the
|
||||
* server has listed the conversation; unset for one only this client has seen.
|
||||
*/
|
||||
isTest?: boolean
|
||||
}
|
||||
|
||||
export interface ChatState {
|
||||
@@ -127,61 +118,18 @@ export interface ChatOptions {
|
||||
onError?: (error: Error, turn: { conversationId: string; jobId?: string }) => void
|
||||
}
|
||||
|
||||
/** A file sent with a message. It is uploaded to the workspace's object storage before the run starts. */
|
||||
export interface AttachmentUpload {
|
||||
/** Kept as the last segment of the stored key, its extension corrected to the media type for PNG, JPEG and PDF. */
|
||||
name: string
|
||||
/** The bytes: a Blob, or a `data:` URL of them. */
|
||||
data: Blob | string
|
||||
/** The file's media type. Defaults to the Blob's own type, or the data URL's. */
|
||||
mediaType?: string
|
||||
}
|
||||
|
||||
/** The flow input the uploaded attachments are handed to: an `s3object` (`multiple: false`) or an `s3object[]`. */
|
||||
export interface AttachmentsInput {
|
||||
name: string
|
||||
multiple: boolean
|
||||
}
|
||||
|
||||
export interface SendMessageOptions {
|
||||
/** Extra flow inputs for this message, on top of `ChatOptions.inputs`. */
|
||||
inputs?: Record<string, unknown>
|
||||
/**
|
||||
* Files to upload and hand to the flow as `{ s3, filename }` objects in `attachmentsInput`,
|
||||
* the way an AI agent step reads `user_attachments`. A failed upload rejects `sendMessage`
|
||||
* and the run never starts; `stop()` during the upload does the same with an `AbortError`.
|
||||
*/
|
||||
attachments?: AttachmentUpload[]
|
||||
/** Required with `attachments`, which also need message text. With `multiple: false`, more than one attachment is refused before anything uploads. */
|
||||
attachmentsInput?: AttachmentsInput
|
||||
}
|
||||
|
||||
export interface Chat {
|
||||
getState(): ChatState
|
||||
/** Calls `listener` now and on every change; returns the unsubscribe function (Svelte store contract). */
|
||||
subscribe(listener: (state: ChatState) => void): () => void
|
||||
/**
|
||||
* Sends a message in the current conversation, starting one when there is none. Resolves
|
||||
* when the answer is complete. Rejects when the message could not be sent at all — a turn
|
||||
* already running, an attachment that failed to upload — without touching the transcript.
|
||||
*/
|
||||
sendMessage(text: string, options?: SendMessageOptions): Promise<void>
|
||||
/** Sends a message in the current conversation, starting one when there is none. Resolves when the answer is complete. */
|
||||
sendMessage(text: string, options?: { inputs?: Record<string, unknown> }): Promise<void>
|
||||
/** Stops following the answer and asks Windmill to cancel the run. */
|
||||
stop(): Promise<void>
|
||||
newConversation(): void
|
||||
selectConversation(conversationId: string): Promise<void>
|
||||
/**
|
||||
* `kind` narrows server history to the flow editor's test chats, the deployed flow's
|
||||
* own (the server's default), or both. Local history has no test chats and ignores it.
|
||||
*/
|
||||
loadConversations(options?: {
|
||||
page?: number
|
||||
perPage?: number
|
||||
kind?: 'test' | 'deployed' | 'all'
|
||||
}): Promise<Conversation[]>
|
||||
loadConversations(options?: { page?: number; perPage?: number }): Promise<Conversation[]>
|
||||
deleteConversation(conversationId: string): Promise<void>
|
||||
/** Sets a conversation's title. The list keeps its order: only a turn moves a conversation. */
|
||||
renameConversation(conversationId: string, title: string): Promise<void>
|
||||
loadOlderMessages(): Promise<void>
|
||||
/** Stops background work (stream, polling) and writes local history out. The chat stays usable. */
|
||||
destroy(): void
|
||||
|
||||
@@ -69,12 +69,6 @@ export function conversationTitle(firstMessage: string): string {
|
||||
return chars.length > 25 ? `${chars.slice(0, 25).join('')}...` : firstMessage
|
||||
}
|
||||
|
||||
/** The server's bound on a typed title: 252 characters plus an ellipsis fits its 255-char column. */
|
||||
export function truncateTitle(title: string): string {
|
||||
const chars = Array.from(title)
|
||||
return chars.length > 252 ? `${chars.slice(0, 252).join('')}...` : title
|
||||
}
|
||||
|
||||
export function sleep(ms: number, signal?: AbortSignal): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (signal?.aborted) return reject(abortError())
|
||||
|
||||
@@ -2,7 +2,7 @@ import { describe, expect, test } from 'bun:test'
|
||||
import type { UIMessage, UIMessageChunk } from 'ai'
|
||||
import { createWindmillChatTransport, toUIMessages } from '../src/ai-sdk'
|
||||
import type { ChatMessage } from '../src/types'
|
||||
import { fetchMock, json, messageRow, ndjson, sse, text, type Route } from './support'
|
||||
import { fetchMock, json, ndjson, sse, text, type Route } from './support'
|
||||
|
||||
const FLOW = 'f/chat/agent'
|
||||
const run: Route = (c) =>
|
||||
@@ -141,26 +141,6 @@ describe('createWindmillChatTransport', () => {
|
||||
expect(second[1]).toMatchObject({ errorText: 'ExecutionErr: boom' })
|
||||
})
|
||||
|
||||
test('loads the attachments of a user row, the call an MCP tool row carries and the reasoning behind an answer', async () => {
|
||||
const { fetch } = fetchMock((c) =>
|
||||
c.method === 'GET' && c.url.pathname.endsWith('/messages')
|
||||
? json([
|
||||
messageRow(1, 'user', 'hi', { attachments: [{ input: 'files', s3: 'chat/a.png', filename: 'a.png' }] }),
|
||||
messageRow(2, 'tool', 'Used lookup tool', { job_id: 'agent-job', reasoning: 'why', tool_arguments: '{"q":1}', tool_result: '42' }),
|
||||
messageRow(3, 'assistant', 'The answer is 42', { reasoning: 'hmm' })
|
||||
])
|
||||
: undefined
|
||||
)
|
||||
const transport = createWindmillChatTransport({ baseUrl: 'http://wm.test', workspace: 'ws', flowPath: FLOW, fetch })
|
||||
const ui = await transport.loadMessages('c')
|
||||
expect(ui.map((m) => m.parts.map((p) => p.type))).toEqual([['text'], ['reasoning', 'dynamic-tool', 'reasoning', 'text']])
|
||||
expect(ui[0].metadata).toEqual({ attachments: [{ input: 'files', s3: 'chat/a.png', filename: 'a.png' }] })
|
||||
expect(ui[1].metadata).toBeUndefined()
|
||||
expect(ui[1].parts[0]).toMatchObject({ type: 'reasoning', text: 'why' })
|
||||
expect(ui[1].parts[1]).toMatchObject({ toolName: 'lookup', state: 'output-available', input: { q: 1 }, output: 42 })
|
||||
expect(ui[1].parts[2]).toMatchObject({ type: 'reasoning', text: 'hmm' })
|
||||
})
|
||||
|
||||
test('refuses attachments with a clear error', async () => {
|
||||
const transport = createWindmillChatTransport({ baseUrl: 'http://wm.test', workspace: 'ws', flowPath: FLOW, fetch: fetchMock().fetch })
|
||||
await expect(
|
||||
@@ -195,11 +175,4 @@ describe('toUIMessages', () => {
|
||||
expect(ui[1].parts[0]).toMatchObject({ toolCallId: 'c1', toolName: 'lookup', state: 'output-available', input: { q: 1 }, output: 42 })
|
||||
expect(ui[3].parts[0]).toMatchObject({ state: 'output-error', errorText: 'Error executing lookup' })
|
||||
})
|
||||
|
||||
test('keeps a stored JSON null result rather than the row text', () => {
|
||||
const ui = toUIMessages([
|
||||
{ success: true, createdAt: '2026-01-01T00:00:00Z', pending: false, id: 't1', role: 'tool', content: 'Used notify tool', tool: { callId: 'c1', name: 'notify', status: 'success', arguments: '{}', result: 'null' } }
|
||||
])
|
||||
expect(ui[0].parts[0]).toMatchObject({ type: 'dynamic-tool', state: 'output-available', output: null })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,14 +0,0 @@
|
||||
import { describe, expect, test } from 'bun:test'
|
||||
import { WindmillChatApi } from '../src/api'
|
||||
|
||||
describe('WindmillChatApi.attachmentUrl', () => {
|
||||
test('points at the workspace download endpoint, with the storage only when there is one', () => {
|
||||
const api = new WindmillChatApi({ baseUrl: 'https://wm.test/api/', workspace: 'my ws' })
|
||||
expect(api.attachmentUrl({ input: 'files', s3: 'chat/a b&c.png', storage: 'secondary' })).toBe(
|
||||
'https://wm.test/api/w/my%20ws/job_helpers/download_s3_file?file_key=chat%2Fa+b%26c.png&storage=secondary'
|
||||
)
|
||||
expect(api.attachmentUrl({ input: 'files', s3: 'chat/a.png' })).toBe(
|
||||
'https://wm.test/api/w/my%20ws/job_helpers/download_s3_file?file_key=chat%2Fa.png'
|
||||
)
|
||||
})
|
||||
})
|
||||
@@ -31,13 +31,6 @@ describe('assistant-ui conversion', () => {
|
||||
expect(toThreadMessage(turns[0])).toMatchObject({ role: 'user', content: [{ type: 'text', text: 'hi' }] })
|
||||
})
|
||||
|
||||
test('keeps a stored JSON null result rather than the row text', () => {
|
||||
const turn = groupTurns([
|
||||
{ ...base, id: 't1', role: 'tool', content: 'Used notify tool', tool: { callId: 'c1', name: 'notify', status: 'success', arguments: '{}', result: 'null' } }
|
||||
])[0]
|
||||
expect(toThreadMessage(turn).content).toMatchObject([{ type: 'tool-call', toolName: 'notify', result: null }])
|
||||
})
|
||||
|
||||
test('marks a failed answer as incomplete', () => {
|
||||
const [turn] = groupTurns([{ ...base, id: 'a', role: 'assistant', content: 'boom', success: false }])
|
||||
expect(toThreadMessage(turn).status).toEqual({ type: 'incomplete', reason: 'error', error: 'boom' })
|
||||
|
||||
@@ -1,444 +0,0 @@
|
||||
import { describe, expect, test } from 'bun:test'
|
||||
import { WindmillChatApi } from '../src/api'
|
||||
import { storedAttachmentName, uploadAttachments } from '../src/attachments'
|
||||
import { createChat } from '../src/chat'
|
||||
import type { ChatOptions } from '../src/types'
|
||||
import { abortError } from '../src/utils'
|
||||
import { fetchMock, json, memoryStorage, sse, text, type RecordedCall, type Route } from './support'
|
||||
|
||||
const BASE = 'http://wm.test'
|
||||
const FLOW = 'f/chat/agent'
|
||||
const UPLOAD_PATH = '/api/w/ws/job_helpers/upload_s3_file'
|
||||
|
||||
const run: Route = (c) =>
|
||||
c.method === 'POST' && c.url.pathname === `/api/w/ws/jobs/run/f/${FLOW}`
|
||||
? text('job-1')
|
||||
: undefined
|
||||
|
||||
/** Stores under the key it was asked to, like the server with a `file_key`. */
|
||||
const upload: Route = (c) =>
|
||||
c.url.pathname === UPLOAD_PATH
|
||||
? json({ file_key: c.url.searchParams.get('file_key') })
|
||||
: undefined
|
||||
|
||||
const answer: Route = (c) =>
|
||||
c.url.pathname === '/api/w/ws/jobs_u/getupdate_sse/job-1'
|
||||
? sse([
|
||||
{
|
||||
type: 'update',
|
||||
completed: true,
|
||||
only_result: { output: 'ok', messages: [] }
|
||||
}
|
||||
])
|
||||
: undefined
|
||||
|
||||
function options(fetch: ChatOptions['fetch']): ChatOptions {
|
||||
return {
|
||||
flowPath: FLOW,
|
||||
baseUrl: BASE,
|
||||
workspace: 'ws',
|
||||
token: 'tok',
|
||||
fetch,
|
||||
storage: memoryStorage()
|
||||
}
|
||||
}
|
||||
|
||||
const uploads = (calls: RecordedCall[]) => calls.filter((c) => c.url.pathname === UPLOAD_PATH)
|
||||
const runs = (calls: RecordedCall[]) =>
|
||||
calls.filter((c) => c.url.pathname.startsWith('/api/w/ws/jobs/run/'))
|
||||
|
||||
const png = new Blob([new Uint8Array([0x89, 0x50, 0x4e, 0x47])], {
|
||||
type: 'image/png'
|
||||
})
|
||||
const pdf = new Blob(['%PDF-1.7'], { type: 'application/pdf' })
|
||||
|
||||
describe('storedAttachmentName', () => {
|
||||
// The worker reads the media type from the key's extension, so it has to match the bytes.
|
||||
test('renames a re-encoded image and gives a bare name its extension', () => {
|
||||
expect(storedAttachmentName('photo.webp', 'image/png')).toBe('photo.png')
|
||||
expect(storedAttachmentName('holiday.png', 'image/jpeg')).toBe('holiday.jpg')
|
||||
expect(storedAttachmentName('contract', 'application/pdf')).toBe('contract.pdf')
|
||||
expect(storedAttachmentName('report.2026.final.webp', 'image/png')).toBe(
|
||||
'report.2026.final.png'
|
||||
)
|
||||
})
|
||||
|
||||
test('leaves a type it does not know alone', () => {
|
||||
expect(storedAttachmentName('notes.csv', 'text/csv')).toBe('notes.csv')
|
||||
})
|
||||
})
|
||||
|
||||
describe('sendMessage with attachments', () => {
|
||||
test('uploads each file under the turn prefix and hands the list to the input', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
|
||||
await chat.sendMessage('read these', {
|
||||
inputs: { locale: 'fr' },
|
||||
attachments: [
|
||||
{ name: 'photo.webp', data: png },
|
||||
{ name: 'contract', data: pdf },
|
||||
// A data URL is decoded to its bytes; the mediaType names what they are.
|
||||
{
|
||||
name: 'photo.webp',
|
||||
data: `data:image/png;base64,${btoa('\x89PNG')}`
|
||||
}
|
||||
],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
|
||||
const keys = uploads(calls).map((c) => c.url.searchParams.get('file_key')!)
|
||||
expect(keys).toHaveLength(3)
|
||||
const prefix = keys[0].split('/').slice(0, 3).join('/')
|
||||
expect(prefix).toMatch(/^windmill_uploads\/chat\/[0-9a-f-]{36}$/)
|
||||
expect(keys).toEqual([
|
||||
`${prefix}/0/photo.png`,
|
||||
`${prefix}/1/contract.pdf`,
|
||||
`${prefix}/2/photo.png`
|
||||
])
|
||||
expect(uploads(calls).map((c) => c.url.searchParams.get('content_type'))).toEqual([
|
||||
'image/png',
|
||||
'application/pdf',
|
||||
'image/png'
|
||||
])
|
||||
expect(uploads(calls).map((c) => c.headers['content-type'])).toEqual([
|
||||
'image/png',
|
||||
'application/pdf',
|
||||
'image/png'
|
||||
])
|
||||
expect(new Uint8Array(await uploads(calls)[2].raw!.arrayBuffer())).toEqual(
|
||||
new Uint8Array([0x89, 0x50, 0x4e, 0x47])
|
||||
)
|
||||
|
||||
expect(runs(calls)[0].body).toEqual({
|
||||
locale: 'fr',
|
||||
user_message: 'read these',
|
||||
files: [
|
||||
{ s3: `${prefix}/0/photo.png`, filename: 'photo.png' },
|
||||
{ s3: `${prefix}/1/contract.pdf`, filename: 'contract.pdf' },
|
||||
{ s3: `${prefix}/2/photo.png`, filename: 'photo.png' }
|
||||
]
|
||||
})
|
||||
expect(chat.getState().status).toBe('idle')
|
||||
})
|
||||
|
||||
test('hands a single object to an input that holds one file', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
await chat.sendMessage('read this', {
|
||||
attachments: [{ name: 'contract.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'file', multiple: false }
|
||||
})
|
||||
const body = runs(calls)[0].body as Record<string, unknown>
|
||||
expect(body.file).toEqual({
|
||||
s3: expect.stringMatching(/\/0\/contract\.pdf$/),
|
||||
filename: 'contract.pdf'
|
||||
})
|
||||
})
|
||||
|
||||
test('refuses several files for an input that holds one, before uploading any', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
await expect(
|
||||
chat.sendMessage('read these', {
|
||||
attachments: [
|
||||
{ name: 'a.pdf', data: pdf },
|
||||
{ name: 'b.png', data: png }
|
||||
],
|
||||
attachmentsInput: { name: 'file', multiple: false }
|
||||
})
|
||||
).rejects.toThrow('holds one file')
|
||||
expect(calls).toHaveLength(0)
|
||||
expect(chat.getState().messages).toEqual([])
|
||||
})
|
||||
|
||||
test('refuses attachments without message text, before uploading', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
await expect(
|
||||
chat.sendMessage(' ', {
|
||||
attachments: [{ name: 'a.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
).rejects.toThrow('need a message')
|
||||
expect(calls).toHaveLength(0)
|
||||
})
|
||||
|
||||
test('refuses attachments without an input to put them in', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
await expect(
|
||||
chat.sendMessage('hi', { attachments: [{ name: 'a.pdf', data: pdf }] })
|
||||
).rejects.toThrow('attachmentsInput')
|
||||
expect(calls).toHaveLength(0)
|
||||
})
|
||||
|
||||
test('a failed upload rejects without a run, and withdraws the message', async () => {
|
||||
const { fetch, calls } = fetchMock(
|
||||
(c) => (c.url.pathname === UPLOAD_PATH ? text('no object storage', 500) : undefined),
|
||||
run,
|
||||
answer
|
||||
)
|
||||
const chat = createChat(options(fetch))
|
||||
const statuses: string[] = []
|
||||
chat.subscribe((s) => statuses.push(s.status))
|
||||
|
||||
await expect(
|
||||
chat.sendMessage('read this', {
|
||||
attachments: [{ name: 'contract.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
).rejects.toThrow('no object storage')
|
||||
|
||||
expect(runs(calls)).toHaveLength(0)
|
||||
// Shown as submitted while uploading, then withdrawn whole: no message, no conversation.
|
||||
expect(statuses).toContain('submitted')
|
||||
const state = chat.getState()
|
||||
expect(state.status).toBe('idle')
|
||||
expect(state.messages).toEqual([])
|
||||
expect(state.conversationId).toBeUndefined()
|
||||
expect(state.conversations).toEqual([])
|
||||
// The chat is free for the next message.
|
||||
await chat.sendMessage('plain')
|
||||
expect(runs(calls)).toHaveLength(1)
|
||||
})
|
||||
|
||||
test('stop() during the upload aborts it and withdraws the message', async () => {
|
||||
const { fetch, calls } = fetchMock(
|
||||
(c) =>
|
||||
c.url.pathname === UPLOAD_PATH
|
||||
? new Promise((_, reject) =>
|
||||
c.signal!.addEventListener('abort', () => reject(abortError()))
|
||||
)
|
||||
: undefined,
|
||||
run,
|
||||
answer
|
||||
)
|
||||
const chat = createChat(options(fetch))
|
||||
const sending = chat.sendMessage('read this', {
|
||||
attachments: [{ name: 'contract.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
await new Promise((r) => setTimeout(r, 0))
|
||||
expect(chat.getState().status).toBe('submitted')
|
||||
await chat.stop()
|
||||
await expect(sending).rejects.toMatchObject({ name: 'AbortError' })
|
||||
expect(runs(calls)).toHaveLength(0)
|
||||
expect(chat.getState()).toMatchObject({
|
||||
status: 'idle',
|
||||
messages: [],
|
||||
conversationId: undefined
|
||||
})
|
||||
})
|
||||
|
||||
test('a switch away mid-upload leaves no conversation behind', async () => {
|
||||
const storage = memoryStorage()
|
||||
const { fetch, calls } = fetchMock(
|
||||
(c) =>
|
||||
c.url.pathname === UPLOAD_PATH
|
||||
? new Promise((_, reject) =>
|
||||
c.signal!.addEventListener('abort', () => reject(abortError()))
|
||||
)
|
||||
: undefined,
|
||||
run,
|
||||
answer
|
||||
)
|
||||
const chat = createChat({ ...options(fetch), storage })
|
||||
const sending = chat.sendMessage('never runs', {
|
||||
attachments: [{ name: 'contract.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
await new Promise((r) => setTimeout(r, 0))
|
||||
const opened = chat.getState().conversationId!
|
||||
chat.newConversation()
|
||||
await expect(sending).rejects.toMatchObject({ name: 'AbortError' })
|
||||
|
||||
expect(runs(calls)).toHaveLength(0)
|
||||
expect(chat.getState().conversations.map((c) => c.id)).not.toContain(opened)
|
||||
const reloaded = createChat({ ...options(fetch), storage })
|
||||
expect((await reloaded.loadConversations()).map((c) => c.id)).not.toContain(opened)
|
||||
})
|
||||
|
||||
test('a failed upload aborts the rest of its batch and deletes nothing', async () => {
|
||||
let first: (r: Response) => void = () => {}
|
||||
const { fetch, calls } = fetchMock(
|
||||
(c) => {
|
||||
if (c.url.pathname !== UPLOAD_PATH) return undefined
|
||||
const key = c.url.searchParams.get('file_key')!
|
||||
// The first file lands after the second has already failed.
|
||||
if (key.includes('/0/')) return new Promise<Response>((resolve) => (first = resolve))
|
||||
setTimeout(() => first(json({ file_key: keys()[0] })), 5)
|
||||
return text('quota exceeded', 507)
|
||||
},
|
||||
run,
|
||||
answer
|
||||
)
|
||||
const keys = () => uploads(calls).map((c) => c.url.searchParams.get('file_key')!)
|
||||
const chat = createChat(options(fetch))
|
||||
await expect(
|
||||
chat.sendMessage('read these', {
|
||||
attachments: [
|
||||
{ name: 'a.pdf', data: pdf },
|
||||
{ name: 'b.png', data: png }
|
||||
],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
).rejects.toThrow('quota exceeded')
|
||||
// The upload still in flight when the other failed was told to stop.
|
||||
expect(uploads(calls)[0].signal?.aborted).toBe(true)
|
||||
expect(calls.filter((c) => c.method === 'DELETE')).toEqual([])
|
||||
expect(runs(calls)).toHaveLength(0)
|
||||
})
|
||||
|
||||
test('a send made right after stop() is not reset by the stopped upload', async () => {
|
||||
let releaseRun: (r: Response) => void = () => {}
|
||||
const { fetch, calls } = fetchMock(
|
||||
(c) =>
|
||||
c.url.pathname === UPLOAD_PATH
|
||||
? new Promise((_, reject) =>
|
||||
c.signal!.addEventListener('abort', () => reject(abortError()))
|
||||
)
|
||||
: undefined,
|
||||
(c) =>
|
||||
c.method === 'POST' && c.url.pathname === `/api/w/ws/jobs/run/f/${FLOW}`
|
||||
? new Promise<Response>((resolve) => (releaseRun = resolve))
|
||||
: undefined,
|
||||
answer
|
||||
)
|
||||
const chat = createChat(options(fetch))
|
||||
// An existing conversation, so the stopped turn and the next one share it.
|
||||
const first = chat.sendMessage('first')
|
||||
await new Promise((r) => setTimeout(r, 0))
|
||||
releaseRun(text('job-1'))
|
||||
await first
|
||||
const stopped = chat.sendMessage('with a file', {
|
||||
attachments: [{ name: 'a.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
await new Promise((r) => setTimeout(r, 0))
|
||||
void chat.stop()
|
||||
const next = chat.sendMessage('right after')
|
||||
await expect(stopped).rejects.toMatchObject({ name: 'AbortError' })
|
||||
await new Promise((r) => setTimeout(r, 0))
|
||||
expect(chat.getState().status).toBe('submitted')
|
||||
expect(chat.getState().messages.map((m) => m.content)).toContain('right after')
|
||||
expect(chat.getState().messages.map((m) => m.content)).not.toContain('with a file')
|
||||
releaseRun(text('job-1'))
|
||||
await next
|
||||
expect(runs(calls)).toHaveLength(2)
|
||||
})
|
||||
|
||||
test('a conversation is listed only once its run starts', async () => {
|
||||
let failUpload: (r: Response) => void = () => {}
|
||||
const { fetch } = fetchMock(
|
||||
(c) =>
|
||||
c.url.pathname === UPLOAD_PATH
|
||||
? new Promise<Response>((resolve) => (failUpload = resolve))
|
||||
: undefined,
|
||||
run,
|
||||
answer
|
||||
)
|
||||
const chat = createChat(options(fetch))
|
||||
const sending = chat.sendMessage('read this', {
|
||||
attachments: [{ name: 'a.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
await new Promise((r) => setTimeout(r, 0))
|
||||
expect(chat.getState()).toMatchObject({ status: 'submitted', conversations: [] })
|
||||
failUpload(text('boom', 500))
|
||||
await expect(sending).rejects.toThrow('boom')
|
||||
expect(chat.getState().conversations).toEqual([])
|
||||
})
|
||||
|
||||
test('an already aborted signal uploads nothing', async () => {
|
||||
const { fetch, calls } = fetchMock(upload)
|
||||
const api = new WindmillChatApi({ baseUrl: BASE, workspace: 'ws', token: 'tok', fetch })
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
await expect(
|
||||
uploadAttachments(api, [{ name: 'a.pdf', data: pdf }], 'turn', controller.signal)
|
||||
).rejects.toMatchObject({ name: 'AbortError' })
|
||||
expect(calls).toHaveLength(0)
|
||||
})
|
||||
|
||||
test('the pending user message carries its uploaded files before the run returns', async () => {
|
||||
let releaseRun: (r: Response) => void = () => {}
|
||||
const { fetch, calls } = fetchMock(
|
||||
upload,
|
||||
(c) =>
|
||||
c.method === 'POST' && c.url.pathname === `/api/w/ws/jobs/run/f/${FLOW}`
|
||||
? new Promise<Response>((resolve) => (releaseRun = resolve))
|
||||
: undefined,
|
||||
answer
|
||||
)
|
||||
const chat = createChat(options(fetch))
|
||||
const sending = chat.sendMessage('read these', {
|
||||
attachments: [
|
||||
{ name: 'photo.webp', data: png },
|
||||
{ name: 'contract', data: pdf }
|
||||
],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
while (runs(calls).length === 0) await new Promise((r) => setTimeout(r, 1))
|
||||
const keys = uploads(calls).map((c) => c.url.searchParams.get('file_key')!)
|
||||
const pending = chat.getState().messages.find((m) => m.role === 'user')!
|
||||
expect(pending.pending).toBe(true)
|
||||
expect(pending.attachments).toEqual([
|
||||
{ input: 'files', s3: keys[0], filename: 'photo.png' },
|
||||
{ input: 'files', s3: keys[1], filename: 'contract.pdf' }
|
||||
])
|
||||
releaseRun(text('job-1'))
|
||||
await sending
|
||||
})
|
||||
|
||||
test('an explicit mediaType wins over the type a data URL declares', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
await chat.sendMessage('read this', {
|
||||
attachments: [
|
||||
{
|
||||
name: 'contract',
|
||||
data: `data:application/octet-stream;base64,${btoa('%PDF')}`,
|
||||
mediaType: 'application/pdf'
|
||||
}
|
||||
],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
const call = uploads(calls)[0]
|
||||
expect(call.url.searchParams.get('file_key')).toMatch(/\/0\/contract\.pdf$/)
|
||||
expect(call.url.searchParams.get('content_type')).toBe('application/pdf')
|
||||
})
|
||||
|
||||
test('stop() after the uploads land but before the run starts runs nothing', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
const sending = chat.sendMessage('read this', {
|
||||
attachments: [{ name: 'contract.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
// The upload responds at once; Stop lands before the send resumes after it.
|
||||
while (uploads(calls).length === 0) await Promise.resolve()
|
||||
await chat.stop()
|
||||
await expect(sending).rejects.toMatchObject({ name: 'AbortError' })
|
||||
expect(runs(calls)).toHaveLength(0)
|
||||
expect(calls.filter((c) => c.method === 'DELETE')).toEqual([])
|
||||
expect(chat.getState()).toMatchObject({ status: 'idle', messages: [], conversations: [] })
|
||||
})
|
||||
|
||||
test('a subscriber stopping when the attachments appear runs nothing', async () => {
|
||||
const { fetch, calls } = fetchMock(upload, run, answer)
|
||||
const chat = createChat(options(fetch))
|
||||
chat.subscribe((s) => {
|
||||
if (s.messages.some((m) => m.attachments)) void chat.stop()
|
||||
})
|
||||
await expect(
|
||||
chat.sendMessage('read this', {
|
||||
attachments: [{ name: 'contract.pdf', data: pdf }],
|
||||
attachmentsInput: { name: 'files', multiple: true }
|
||||
})
|
||||
).rejects.toMatchObject({ name: 'AbortError' })
|
||||
expect(runs(calls)).toHaveLength(0)
|
||||
expect(calls.filter((c) => c.method === 'DELETE')).toEqual([])
|
||||
expect(chat.getState()).toMatchObject({ status: 'idle', messages: [], conversations: [] })
|
||||
})
|
||||
})
|
||||
@@ -327,113 +327,6 @@ describe('createChat with server history', () => {
|
||||
expect(messagesCall.headers.authorization).toBeUndefined()
|
||||
})
|
||||
|
||||
test('a persisted row brings back its attachments, reasoning and the call an MCP tool row carries', async () => {
|
||||
const { fetch } = fetchMock(
|
||||
(c) =>
|
||||
c.method === 'GET' && c.url.pathname === '/api/w/ws/flow_conversations/conv-1/messages'
|
||||
? json([
|
||||
messageRow(1, 'user', 'hi', {
|
||||
attachments: [{ input: 'files', s3: 'chat/a.png', storage: 'secondary', filename: 'a.png' }]
|
||||
}),
|
||||
messageRow(2, 'tool', 'Used lookup tool', {
|
||||
job_id: 'agent-job',
|
||||
tool_arguments: '{"q":1}',
|
||||
tool_result: '42'
|
||||
}),
|
||||
messageRow(3, 'tool', 'Error executing lookup', {
|
||||
job_id: 'agent-job',
|
||||
success: false,
|
||||
tool_arguments: '{"q":2}',
|
||||
tool_result: 'MCP tool error: boom'
|
||||
}),
|
||||
messageRow(4, 'assistant', 'The answer is 42', { reasoning: 'hmm' }),
|
||||
messageRow(5, 'assistant', 'Hello'),
|
||||
messageRow(6, 'tool', 'Used get_price tool', {
|
||||
job_id: 'script-tool-job',
|
||||
tool_arguments: '{"item":"widget"}',
|
||||
tool_result: '{"price":42}'
|
||||
})
|
||||
])
|
||||
: undefined
|
||||
)
|
||||
const chat = createChat(options({ history: 'server' }, fetch))
|
||||
await chat.selectConversation('conv-1')
|
||||
|
||||
const [user, used, failed, answer, plain, scriptTool] = chat.getState().messages
|
||||
expect(scriptTool).toMatchObject({ jobId: 'script-tool-job', tool: { name: 'get_price', status: 'success', arguments: '{"item":"widget"}', result: '{"price":42}' } })
|
||||
expect(user.attachments).toEqual([{ input: 'files', s3: 'chat/a.png', storage: 'secondary', filename: 'a.png' }])
|
||||
expect(answer.attachments).toBeUndefined()
|
||||
expect(used.tool).toEqual({ name: 'lookup', status: 'success', arguments: '{"q":1}', result: '42' })
|
||||
expect(failed.tool).toEqual({ name: 'lookup', status: 'error', arguments: '{"q":2}', result: 'MCP tool error: boom' })
|
||||
expect(answer.reasoning).toBe('hmm')
|
||||
expect(plain.reasoning).toBeUndefined()
|
||||
})
|
||||
|
||||
test('a row nothing streamed, like a web search, lands before the answer as on reload', async () => {
|
||||
const { fetch } = fetchMock(
|
||||
run,
|
||||
(c) =>
|
||||
c.url.pathname === streamPath
|
||||
? sse([
|
||||
{
|
||||
type: 'update',
|
||||
new_result_stream: ndjson({ type: 'token_delta', content: 'Rust.' }),
|
||||
stream_offset: 1,
|
||||
completed: true,
|
||||
only_result: { output: 'Rust.', messages: [] }
|
||||
}
|
||||
])
|
||||
: undefined,
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/messages')
|
||||
? json([
|
||||
messageRow(91, 'user', 'hi'),
|
||||
messageRow(92, 'tool', 'Used websearch tool', { job_id: 'step-1', tool_result: '[{"url":"https://example.com"}]' }),
|
||||
messageRow(93, 'assistant', 'Rust.', { job_id: 'step-1' })
|
||||
])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([]) : undefined)
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.sendMessage('hi')
|
||||
expect(chat.getState().messages.map((m) => [m.role, m.content, m.seq])).toEqual([
|
||||
['user', 'hi', 91],
|
||||
['tool', 'Used websearch tool', 92],
|
||||
['assistant', 'Rust.', 93]
|
||||
])
|
||||
})
|
||||
|
||||
test('a failure row stays after streamed text that never got a row', async () => {
|
||||
const { fetch } = fetchMock(
|
||||
run,
|
||||
(c) =>
|
||||
c.url.pathname === streamPath
|
||||
? sse([
|
||||
{
|
||||
type: 'update',
|
||||
new_result_stream: ndjson({ type: 'token_delta', content: 'Let me look' }),
|
||||
stream_offset: 1,
|
||||
completed: true,
|
||||
only_result: { error: { name: 'ExecutionErr', message: 'boom' } }
|
||||
}
|
||||
])
|
||||
: undefined,
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/messages')
|
||||
? json([messageRow(91, 'user', 'hi'), messageRow(92, 'assistant', 'boom', { job_id: 'step-1', success: false })])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([]) : undefined)
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.sendMessage('hi')
|
||||
const messages = chat.getState().messages
|
||||
expect(messages.map((m) => [m.content, m.success])).toEqual([
|
||||
['hi', true],
|
||||
['Let me look', true],
|
||||
['boom', false]
|
||||
])
|
||||
})
|
||||
|
||||
test('keeps the streamed answer until its row lands, even when a tool row lands first', async () => {
|
||||
let messageFetches = 0
|
||||
const { fetch } = fetchMock(
|
||||
@@ -521,112 +414,6 @@ describe('createChat with server history', () => {
|
||||
expect(chat.getState().conversations.map((c) => c.id)).toEqual(['c1', 'c2'])
|
||||
})
|
||||
|
||||
test('lists one kind of conversation and carries which kind each one is', async () => {
|
||||
const row = (id: string, is_test: boolean) => ({
|
||||
id,
|
||||
workspace_id: 'ws',
|
||||
flow_path: FLOW,
|
||||
title: id,
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
updated_at: '2026-01-01T00:00:00Z',
|
||||
created_by: 'admin',
|
||||
is_test
|
||||
})
|
||||
const { fetch, calls } = fetchMock((c) =>
|
||||
c.url.pathname === '/api/w/ws/flow_conversations/list'
|
||||
? json(c.url.searchParams.get('kind') === 'test' ? [row('t1', true)] : [row('d1', false)])
|
||||
: undefined
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.loadConversations()
|
||||
expect(calls[0].url.searchParams.has('kind')).toBe(false)
|
||||
expect(chat.getState().conversations.map((c) => [c.id, c.isTest])).toEqual([['d1', false]])
|
||||
await chat.loadConversations({ kind: 'test' })
|
||||
expect(calls[1].url.searchParams.get('kind')).toBe('test')
|
||||
expect(chat.getState().conversations.map((c) => [c.id, c.isTest])).toEqual([['t1', true]])
|
||||
})
|
||||
|
||||
test('a list for a kind no longer asked for does not replace the newer one', async () => {
|
||||
const row = (id: string, is_test: boolean) => ({
|
||||
id,
|
||||
workspace_id: 'ws',
|
||||
flow_path: FLOW,
|
||||
title: id,
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
updated_at: '2026-01-01T00:00:00Z',
|
||||
created_by: 'admin',
|
||||
is_test
|
||||
})
|
||||
const { fetch } = fetchMock((c) => {
|
||||
if (c.url.pathname !== '/api/w/ws/flow_conversations/list') return undefined
|
||||
if (c.url.searchParams.get('kind') === 'test') {
|
||||
return new Promise<Response>((r) => setTimeout(() => r(json([row('t1', true)])), 50))
|
||||
}
|
||||
return json([row('d1', false)])
|
||||
})
|
||||
const chat = createChat(options({}, fetch))
|
||||
const slow = chat.loadConversations({ kind: 'test' })
|
||||
await chat.loadConversations({ kind: 'deployed' })
|
||||
await slow
|
||||
expect(chat.getState().conversations.map((c) => c.id)).toEqual(['d1'])
|
||||
// Another kind asked for on a later page starts its own listing rather than appending.
|
||||
await chat.loadConversations({ page: 2, kind: 'test' })
|
||||
expect(chat.getState().conversations.map((c) => c.id)).toEqual(['t1'])
|
||||
})
|
||||
|
||||
test('the refresh after a new turn lists the kind last asked for', async () => {
|
||||
const { fetch, calls } = fetchMock(
|
||||
run,
|
||||
(c) =>
|
||||
c.url.pathname === streamPath
|
||||
? sse([{ type: 'update', completed: true, only_result: { output: 'Hello', messages: [] } }])
|
||||
: undefined,
|
||||
(c) =>
|
||||
c.method === 'GET' && c.url.pathname.endsWith('/messages')
|
||||
? json([messageRow(11, 'user', 'hi'), messageRow(12, 'assistant', 'Hello', { job_id: 'agent-job' })])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([]) : undefined)
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.loadConversations({ kind: 'test' })
|
||||
await chat.sendMessage('hi')
|
||||
const lists = calls.filter((c) => c.url.pathname === '/api/w/ws/flow_conversations/list')
|
||||
expect(lists.length).toBeGreaterThan(1)
|
||||
expect(lists.every((c) => c.url.searchParams.get('kind') === 'test')).toBe(true)
|
||||
})
|
||||
|
||||
test('renaming a conversation keeps its place in the list', async () => {
|
||||
const row = (id: string) => ({
|
||||
id,
|
||||
workspace_id: 'ws',
|
||||
flow_path: FLOW,
|
||||
title: id,
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
updated_at: '2026-01-01T00:00:00Z',
|
||||
created_by: 'admin',
|
||||
is_test: false
|
||||
})
|
||||
const { fetch, calls } = fetchMock(
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([row('c1'), row('c2')]) : undefined),
|
||||
(c) =>
|
||||
c.method === 'POST' && c.url.pathname === '/api/w/ws/flow_conversations/update/c2'
|
||||
? text('Conversation c2 updated')
|
||||
: undefined
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.loadConversations()
|
||||
await chat.renameConversation('c2', ' Budget review ')
|
||||
expect(calls[1].body).toEqual({ title: 'Budget review' })
|
||||
expect(chat.getState().conversations.map((c) => [c.id, c.title])).toEqual([
|
||||
['c1', 'c1'],
|
||||
['c2', 'Budget review']
|
||||
])
|
||||
// Cut as the server cuts, so what is shown is what is stored.
|
||||
await chat.renameConversation('c2', 'x'.repeat(300))
|
||||
expect(chat.getState().conversations[1].title).toBe('x'.repeat(252) + '...')
|
||||
expect(calls[2].body).toEqual({ title: 'x'.repeat(252) + '...' })
|
||||
})
|
||||
|
||||
test('a turn started right after stop() is not touched by the stop sync', async () => {
|
||||
let jobs = 0
|
||||
const { fetch } = fetchMock(
|
||||
@@ -796,120 +583,6 @@ describe('createChat with server history', () => {
|
||||
expect(chat.getState().messages.map((m) => m.content)).toEqual(['hi', 'Let me check', 'Used search tool', 'Final answer'])
|
||||
})
|
||||
|
||||
test('thinking that led to a tool call rides on the call, live and once its row lands', async () => {
|
||||
const { fetch } = fetchMock(
|
||||
run,
|
||||
(c) =>
|
||||
c.url.pathname === streamPath
|
||||
? sse([
|
||||
{
|
||||
type: 'update',
|
||||
// No `tool_call_arguments`: a stream cut short leaves the call without them.
|
||||
new_result_stream: ndjson(
|
||||
{ type: 'reasoning_token_delta', content: 'r1' },
|
||||
{ type: 'tool_call', call_id: 'c1', function_name: 'lookup' },
|
||||
{ type: 'tool_result', call_id: 'c1', function_name: 'lookup', result: '1', success: true },
|
||||
{ type: 'token_delta', content: 'Final' }
|
||||
),
|
||||
stream_offset: 4,
|
||||
completed: true,
|
||||
only_result: { output: 'Final', messages: [] }
|
||||
}
|
||||
])
|
||||
: undefined,
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/messages')
|
||||
? json([
|
||||
messageRow(71, 'user', 'hi'),
|
||||
messageRow(72, 'tool', 'Used lookup tool', { job_id: 'step-1', reasoning: 'r1', tool_arguments: '{"q":1}', tool_result: '1' }),
|
||||
messageRow(73, 'assistant', 'Final', { job_id: 'step-1' })
|
||||
])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([]) : undefined)
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.sendMessage('hi')
|
||||
const messages = chat.getState().messages
|
||||
expect(messages.map((m) => [m.role, m.content, m.reasoning, m.seq])).toEqual([
|
||||
['user', 'hi', undefined, 71],
|
||||
['tool', 'Used lookup tool', 'r1', 72],
|
||||
['assistant', 'Final', undefined, 73]
|
||||
])
|
||||
expect(messages[1].tool).toMatchObject({ callId: 'c1', arguments: '{"q":1}', result: '1', status: 'success' })
|
||||
})
|
||||
|
||||
test('a structured answer row replaces the call it streamed as', async () => {
|
||||
const { fetch } = fetchMock(
|
||||
run,
|
||||
(c) =>
|
||||
c.url.pathname === streamPath
|
||||
? sse([
|
||||
{
|
||||
type: 'update',
|
||||
new_result_stream: ndjson(
|
||||
{ type: 'reasoning_token_delta', content: 'hmm' },
|
||||
{ type: 'tool_call', call_id: 'c9', function_name: 'structured_output' },
|
||||
{ type: 'tool_call_arguments', call_id: 'c9', function_name: 'structured_output', arguments: '{"n": 1}' },
|
||||
{ type: 'tool_execution', call_id: 'c9', function_name: 'structured_output' }
|
||||
),
|
||||
stream_offset: 4,
|
||||
completed: true,
|
||||
only_result: { output: { n: 1 }, messages: [] }
|
||||
}
|
||||
])
|
||||
: undefined,
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/messages')
|
||||
? json([messageRow(75, 'user', 'hi'), messageRow(76, 'assistant', '{"n": 1}', { job_id: 'step-1', reasoning: 'hmm' })])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([]) : undefined)
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
await chat.sendMessage('hi')
|
||||
expect(chat.getState().messages.map((m) => [m.role, m.content, m.reasoning, m.tool])).toEqual([
|
||||
['user', 'hi', undefined, undefined],
|
||||
['assistant', '{"n": 1}', 'hmm', undefined]
|
||||
])
|
||||
})
|
||||
|
||||
test('a structured answer row leaves a stopped turn its identical call', async () => {
|
||||
const call = (id: string) =>
|
||||
ndjson(
|
||||
{ type: 'tool_call', call_id: id, function_name: 'structured_output' },
|
||||
{ type: 'tool_call_arguments', call_id: id, function_name: 'structured_output', arguments: '{"ok": true}' }
|
||||
)
|
||||
let jobs = 0
|
||||
const { fetch } = fetchMock(
|
||||
(c) => (c.method === 'POST' && c.url.pathname.includes('/jobs/run/f/') ? text(`job-${++jobs}`) : undefined),
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/getupdate_sse/job-1') ? sse([{ type: 'update', new_result_stream: call('c1'), stream_offset: 2 }]) : undefined,
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/getupdate_sse/job-2')
|
||||
? sse([{ type: 'update', new_result_stream: call('c2'), stream_offset: 2, completed: true, only_result: { output: { ok: true }, messages: [] } }])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname.includes('/queue/cancel/') ? text('ok') : undefined),
|
||||
(c) =>
|
||||
c.url.pathname.endsWith('/messages')
|
||||
? json([messageRow(41, 'user', 'first'), messageRow(42, 'user', 'again'), messageRow(43, 'assistant', '{"ok": true}', { job_id: 'step-2' })])
|
||||
: undefined,
|
||||
(c) => (c.url.pathname === '/api/w/ws/flow_conversations/list' ? json([]) : undefined)
|
||||
)
|
||||
const chat = createChat(options({}, fetch))
|
||||
const first = chat.sendMessage('first')
|
||||
await new Promise((r) => setTimeout(r, 50))
|
||||
const stopped = chat.stop()
|
||||
await first
|
||||
const second = chat.sendMessage('again')
|
||||
await stopped
|
||||
await second
|
||||
expect(chat.getState().messages.map((m) => [m.role, m.content, m.tool?.callId])).toEqual([
|
||||
['user', 'first', undefined],
|
||||
['tool', '', 'c1'],
|
||||
['user', 'again', undefined],
|
||||
['assistant', '{"ok": true}', undefined]
|
||||
])
|
||||
})
|
||||
|
||||
test('the stream asks for a server poll interval only when one is set', async () => {
|
||||
const answer: Route = (c) =>
|
||||
c.url.pathname === streamPath ? sse([{ type: 'update', completed: true, only_result: 'ok' }]) : undefined
|
||||
@@ -1053,27 +726,6 @@ describe('createChat with server history', () => {
|
||||
expect((await again.loadConversations()).map((c) => c.id)).toEqual([newer, older])
|
||||
})
|
||||
|
||||
test('renaming a local conversation persists the title without reordering history', async () => {
|
||||
const storage = memoryStorage()
|
||||
const { fetch, calls } = fetchMock(run, (c) =>
|
||||
c.url.pathname === streamPath ? sse([{ type: 'update', completed: true, only_result: 'ok' }]) : undefined
|
||||
)
|
||||
const chat = createChat(options({ token: 'tok', storage }, fetch))
|
||||
await chat.sendMessage('older')
|
||||
const older = chat.getState().conversationId!
|
||||
chat.newConversation()
|
||||
await chat.sendMessage('newer')
|
||||
const newer = chat.getState().conversationId!
|
||||
const before = calls.length
|
||||
await chat.renameConversation(older, 'Renamed')
|
||||
expect(calls.length).toBe(before)
|
||||
const again = createChat(options({ token: 'tok', storage }, fetch))
|
||||
expect((await again.loadConversations()).map((c) => [c.id, c.title])).toEqual([
|
||||
[newer, 'newer'],
|
||||
[older, 'Renamed']
|
||||
])
|
||||
})
|
||||
|
||||
test('destroying the chat mid-turn leaves it idle', async () => {
|
||||
const { fetch } = fetchMock(run, (c) =>
|
||||
c.url.pathname === streamPath
|
||||
|
||||
@@ -5,9 +5,6 @@ export interface RecordedCall {
|
||||
url: URL
|
||||
headers: Record<string, string>
|
||||
body: unknown
|
||||
/** A body sent as is rather than as JSON (an upload). */
|
||||
raw?: Blob
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
export type Route = (call: RecordedCall) => Response | Promise<Response> | undefined
|
||||
@@ -23,9 +20,7 @@ export function fetchMock(...routes: Route[]): { fetch: FetchLike; calls: Record
|
||||
headers: Object.fromEntries(
|
||||
Object.entries((init?.headers as Record<string, string>) ?? {}).map(([k, v]) => [k.toLowerCase(), v])
|
||||
),
|
||||
body: typeof init?.body === 'string' ? JSON.parse(init.body) : undefined,
|
||||
raw: init?.body instanceof Blob ? init.body : undefined,
|
||||
signal: init?.signal ?? undefined
|
||||
body: typeof init?.body === 'string' ? JSON.parse(init.body) : undefined
|
||||
}
|
||||
calls.push(call)
|
||||
for (const route of routes) {
|
||||
|
||||
@@ -111,6 +111,8 @@ const migrateCommand = new Command()
|
||||
)
|
||||
.action(migrateDown as any);
|
||||
|
||||
type DataTableResourceType = "postgresql" | "instance" | "external_instance";
|
||||
|
||||
async function create(
|
||||
opts: GlobalOptions & { resource?: string; force?: boolean },
|
||||
name?: string,
|
||||
@@ -139,12 +141,12 @@ async function create(
|
||||
|
||||
const datatables: Record<
|
||||
string,
|
||||
{ database: { resource_type: "postgresql" | "instance"; resource_path?: string } }
|
||||
{ database: { resource_type: DataTableResourceType; resource_path?: string } }
|
||||
> = {};
|
||||
for (const d of existing) {
|
||||
datatables[d.name] = {
|
||||
database: {
|
||||
resource_type: d.resource_type as "postgresql" | "instance",
|
||||
resource_type: d.resource_type as DataTableResourceType,
|
||||
resource_path: d.resource_path ?? undefined,
|
||||
},
|
||||
};
|
||||
|
||||
@@ -10,4 +10,4 @@ export const WM_FORK_PREFIX = "wm-fork";
|
||||
// (e.g. utils.ts) can read it without importing main.ts and creating a circular
|
||||
// dependency (main → workspace → utils → main) that triggers a TDZ.
|
||||
// Re-exported from main.ts for backwards compatibility.
|
||||
export const VERSION = "1.814.0";
|
||||
export const VERSION = "1.813.0";
|
||||
|
||||
Generated
+2
-2
File diff suppressed because one or more lines are too long
@@ -16,12 +16,11 @@ every workspace via the standard cached-resource-type sync, like other built-in
|
||||
- The brain config and tools are resolved at runtime from the resource
|
||||
(`windmill-worker/src/ai_executor.rs`): the brain is interpolated, so a nested provider `$res:`
|
||||
credential resolves automatically.
|
||||
- The step keeps only the flow-local inputs (`user_message`, `user_attachments`, `enabled_tools`,
|
||||
and the history inputs `memory_id` and `previous_messages`) in its own `input_transforms`; the
|
||||
brain and tools stay in the resource (read-only in the step). `enabled_tools` says which of the
|
||||
roster this step may call, narrowing one use of a shared agent without touching the agent: an
|
||||
absent field carries every tool, a list carries the ones it names, and an empty list carries
|
||||
none.
|
||||
- The step keeps only the flow-local inputs (`user_message`, `user_attachments`, `enabled_tools`)
|
||||
in its own `input_transforms`; the brain and tools stay in the resource (read-only in the step).
|
||||
`enabled_tools` says which of the roster this step may call, narrowing one use of a shared agent
|
||||
without touching the agent: an absent field carries every tool, a list carries the ones it names,
|
||||
and an empty list carries none.
|
||||
- The agent carries its tools' default input bindings verbatim as authored (static, AI-filled,
|
||||
or flow expressions), so saving round-trips losslessly. Each host flow overrides what it
|
||||
needs: `tool_inputs` stores per-tool overrides (a diff from the resource tool's own
|
||||
@@ -37,58 +36,6 @@ agent step); below the step's inputs, each tool gets a section with the standard
|
||||
input editors (prop picker included) and a read-only view of its code — edits persist into
|
||||
`tool_inputs`.
|
||||
|
||||
## Memory
|
||||
|
||||
Memory is split between three owners, so a saved agent carries whether it remembers and never
|
||||
which memory it is:
|
||||
|
||||
- **Agent: managed memory.** `memory` is a brain key, so it moves with a saved agent.
|
||||
`{ kind: window, context_length }` has Windmill store the conversation and replay its last N
|
||||
messages; `{ kind: off }` keeps none. An absent `memory` means off, the default: the editor turns
|
||||
it on when chat input is enabled. `auto` and `manual` are the older spellings and are still read.
|
||||
- **Run: memory id.** `flow_status.memory_id`, set when the run is queued: the chat conversation
|
||||
id, an app chat session id, or the `memory_id` run parameter. Any string is accepted, and one
|
||||
that is not a uuid is hashed to a v5 uuid scoped to the workspace and the flow the run started
|
||||
from (`memory_key` in `windmill-common/src/flow_conversations.rs`), so the same key in two flows
|
||||
names two memories. A uuid is used as is. Nothing is generated at save time, so schedules,
|
||||
webhooks, evals and plain runs pass no id and run stateless.
|
||||
- **Step: history inputs.** Flow-local, so they stay on a linked step. Each is read in one memory
|
||||
state only, and the editor offers it only there, the memory id behind a *Custom* toggle that
|
||||
writes the key only once it is on. With managed memory on, `memory_id` overrides the run's id,
|
||||
hashed the same way: a fixed value is one memory shared by every run, an expression such as
|
||||
`flow_input.customer_id` one memory per key, and an expression that evaluates to nothing runs
|
||||
stateless rather than falling back to the run's id. With memory off, `previous_messages` supplies
|
||||
the history itself. An older `auto` or `manual` memory reads neither, so the editor offers them
|
||||
only once the step is moved to the current settings, which the alert's button does. The editor
|
||||
never seeds a placeholder for either, because a present key is the step's choice, and a static
|
||||
empty value reads as unset.
|
||||
|
||||
The worker reconciles them once per agent invocation, nested agent tools included, in
|
||||
`resolve_history_source` (`windmill-worker/src/ai_executor.rs`):
|
||||
|
||||
1. A legacy `auto` or `manual` memory: read as the editor that wrote it ran it. `manual` replays
|
||||
its list; `auto` uses the run's memory id, else the id baked into it, else runs stateless.
|
||||
Neither history input is read. An `auto` without a count, or with 0, is off and read as such.
|
||||
2. Managed memory: the memory id is the step's, else the run's. With no memory id the agent runs
|
||||
stateless, and a step `previous_messages` is ignored.
|
||||
3. Memory off: the history is `previous_messages`, else nothing. Memory is neither read nor
|
||||
written, and a step `memory_id` is ignored.
|
||||
|
||||
Each ignored input and each stateless fallback is written to the job log.
|
||||
|
||||
Memory is stored per (memory id, step id), in `ai_agent_memory` or S3 at
|
||||
`memory/{workspace}/{memory id}/{step}.json`. The chat transcript (`flow_conversation_message`)
|
||||
always follows the run's id, even when a step sets its own. Nothing expires stored memory: deleting
|
||||
a chat conversation deletes its memory, and a memory named by a string id stays until it is
|
||||
overwritten.
|
||||
|
||||
Compatibility runs one way. New workers read every older shape. The editor rewrites a legacy step
|
||||
only when the author changes it, so a flow nobody edits keeps running on older workers, while a
|
||||
step saved with `window` or a history input needs a worker that knows them. An id an older editor
|
||||
baked into `memory` stays a fallback behind the run's id until the author chooses *Keep as memory
|
||||
id* or *Use the run's memory id*. In a chat flow it is dropped on save, since the conversation id
|
||||
always took precedence there.
|
||||
|
||||
## Drafts
|
||||
|
||||
The agent editor edits the resource through a **per-user resource draft** (`draft` table,
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "@windmill-labs/components",
|
||||
"version": "1.814.0",
|
||||
"version": "1.813.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "@windmill-labs/components",
|
||||
"version": "1.814.0",
|
||||
"version": "1.813.0",
|
||||
"hasInstallScript": true,
|
||||
"license": "AGPL-3.0",
|
||||
"dependencies": {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user