starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-19 16:02:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into change-68b704f7

Browse Source 
# Conflicts:
#	backend/ee-repo-ref.txt
#	cli/src/commands/datatable/datatable.ts
#	cli/src/guidance/skills.gen.ts
#	system_prompts/auto-generated/cli/cli-commands.md
#	system_prompts/auto-generated/prompts.ts
#	system_prompts/auto-generated/skills/cli-commands/SKILL.md
This commit is contained in:
Diego Imbert
2026-06-20 18:24:10 +02:00
parent 8484c58559 346cc30e2d
commit cdc7750248
488 changed files with 45695 additions and 3016 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+2 -1
View File
@@ -33,4 +33,5 @@ backend/chrome_profiler.json
.fast-check/
__pycache__/
.playwright-mcp/
.codex
.codex
.claude/scheduled_tasks.lock
CHANGELOG.md
+134
View File
@@ -1,5 +1,139 @@
# Changelog
## [1.734.0](https://github.com/windmill-labs/windmill/compare/v1.733.1...v1.734.0) (2026-06-20)
### Features
* ducklake materialization for data pipelines ([#9689](https://github.com/windmill-labs/windmill/issues/9689)) ([3ebf243](https://github.com/windmill-labs/windmill/commit/3ebf24359d66048d6361ce65cd879cdc04b737ed))
### Bug Fixes
* **frontend:** clear branch step state when switching outer loop iterations ([#9650](https://github.com/windmill-labs/windmill/issues/9650)) ([09a8004](https://github.com/windmill-labs/windmill/commit/09a80040ca268a4379d5302e3401435ba93247e0))
## [1.733.1](https://github.com/windmill-labs/windmill/compare/v1.733.0...v1.733.1) (2026-06-19)
### Bug Fixes
* **backend:** validate ansible vault_id entries before config generation ([#9681](https://github.com/windmill-labs/windmill/issues/9681)) ([c1f31c0](https://github.com/windmill-labs/windmill/commit/c1f31c0e4777bf0cfed0dd7f03249e9a61cd8cb9))
* **frontend:** group live pipeline runs in the activity panel ([#9684](https://github.com/windmill-labs/windmill/issues/9684)) ([1be4df9](https://github.com/windmill-labs/windmill/commit/1be4df9acb935250d4cc12e83cf67e366d870d5a))
* require super admin for object storage config test endpoint ([#9683](https://github.com/windmill-labs/windmill/issues/9683)) ([fb44fe7](https://github.com/windmill-labs/windmill/commit/fb44fe7af2b8ebe8ef64ffb0e5acce8580bf4200))
* validate websocket trigger urls and gate trigger test route ([#9682](https://github.com/windmill-labs/windmill/issues/9682)) ([c39ee07](https://github.com/windmill-labs/windmill/commit/c39ee07c0bcd2249dd19ffa5cd988125eefc6c9f))
## [1.733.0](https://github.com/windmill-labs/windmill/compare/v1.732.0...v1.733.0) (2026-06-19)
### Features
* **ai-chat:** cap read_app_file + search_app grep tool to bound context in large raw apps ([#9653](https://github.com/windmill-labs/windmill/issues/9653)) ([4296a6a](https://github.com/windmill-labs/windmill/commit/4296a6ae1f73564de4df54fe1df0a03c1df05dfd))
* **python, windows:** enable S3 to cache wheels ([#5199](https://github.com/windmill-labs/windmill/issues/5199)) ([ab3bc97](https://github.com/windmill-labs/windmill/commit/ab3bc97cd92b6480327029bcf018280442462af7))
### Bug Fixes
* allow users to always discard their own drafts without write permission ([#9659](https://github.com/windmill-labs/windmill/issues/9659)) ([6833a55](https://github.com/windmill-labs/windmill/commit/6833a554aeddb3e63173d3c3140b490c0bf2822b))
* **backend:** clean up unique_ext_jwt_token on workspace deletion ([#9676](https://github.com/windmill-labs/windmill/issues/9676)) ([9add719](https://github.com/windmill-labs/windmill/commit/9add719d936cdcfb2c4062629e3e1f792694dafe))
* **backend:** strip NUL bytes from draft values on write ([#9673](https://github.com/windmill-labs/windmill/issues/9673)) ([924f9c7](https://github.com/windmill-labs/windmill/commit/924f9c7e8d8863d9af40aee246a519b4be0e1ea2))
* **python:** split PIP_TRUSTED_HOST by whitespace to support multiple hosts ([#9675](https://github.com/windmill-labs/windmill/issues/9675)) ([cafb473](https://github.com/windmill-labs/windmill/commit/cafb473494d9cff3a8b2aeaf9f18b015f966e7b3))
## [1.732.0](https://github.com/windmill-labs/windmill/compare/v1.731.0...v1.732.0) (2026-06-19)
### Features
* **ansible:** add AI chat and editor bar buttons for ansible ([#9671](https://github.com/windmill-labs/windmill/issues/9671)) ([017c3d3](https://github.com/windmill-labs/windmill/commit/017c3d3343c2577501103be4b2dd8dac9727d80d))
### Bug Fixes
* **ai:** emit token usage in gemini proxy streaming translation ([#9669](https://github.com/windmill-labs/windmill/issues/9669)) ([0cc2257](https://github.com/windmill-labs/windmill/commit/0cc2257596a3965cf6db21a0090edcce6e1b8419))
* **backend:** grant script_trigger access to windmill roles ([#9674](https://github.com/windmill-labs/windmill/issues/9674)) ([3361736](https://github.com/windmill-labs/windmill/commit/33617367d09537667d2ab3f91135c736194b9e7e))
* **frontend:** ignore hash/assets in script diffs and drafts (WIN-2071) ([#9664](https://github.com/windmill-labs/windmill/issues/9664)) ([3371265](https://github.com/windmill-labs/windmill/commit/33712653821e83f2562dd5f271dbec0188d5d2f8))
## [1.731.0](https://github.com/windmill-labs/windmill/compare/v1.730.0...v1.731.0) (2026-06-19)
### Features
* **backend:** auto-reconnect postgres trigger listener with backoff (WIN-2073) ([#9666](https://github.com/windmill-labs/windmill/issues/9666)) ([a425431](https://github.com/windmill-labs/windmill/commit/a425431e9067bcf85474fdc7b7ef7f73e41b9071))
### Bug Fixes
* **backend:** grant notify_event access to windmill roles ([#9665](https://github.com/windmill-labs/windmill/issues/9665)) ([a682d02](https://github.com/windmill-labs/windmill/commit/a682d02311a2110bfc0d5e0a5b52e96147fe0dd7))
* **mcp:** repair invalid type keywords in tool JSON schemas ([#9667](https://github.com/windmill-labs/windmill/issues/9667)) ([c30bdec](https://github.com/windmill-labs/windmill/commit/c30bdecea77ff9b4d74d52961f3101201099b683))
* trigger flow error handler on unrecoverable (OOM/zombie) step failures ([#9662](https://github.com/windmill-labs/windmill/issues/9662)) ([7e4df02](https://github.com/windmill-labs/windmill/commit/7e4df02bd60c4d6ee8c92d3dfd19f4e587ff9632))
## [1.730.0](https://github.com/windmill-labs/windmill/compare/v1.729.0...v1.730.0) (2026-06-18)
### Features
* **ai-chat:** summary-based conversation compaction ([#9645](https://github.com/windmill-labs/windmill/issues/9645)) ([5d553b8](https://github.com/windmill-labs/windmill/commit/5d553b81c06664aab61131a93b198575c088d12d))
* Data Pipelines alpha ([#9193](https://github.com/windmill-labs/windmill/issues/9193)) ([7155a0b](https://github.com/windmill-labs/windmill/commit/7155a0bb96cf30bd878272a0f4c3c3b02341b261))
### Bug Fixes
* **ai-chat:** stop echoing app draft value in global chat write tool results ([#9658](https://github.com/windmill-labs/windmill/issues/9658)) ([2fed808](https://github.com/windmill-labs/windmill/commit/2fed808b9e716d9a44b34c7a073ec0d37374be05))
* **backend:** include raw_app drafts in list_apps draft_users ([#9647](https://github.com/windmill-labs/windmill/issues/9647)) ([19bc005](https://github.com/windmill-labs/windmill/commit/19bc0052f1069d732231950a0ec958f675d57417))
* **frontend:** keep ?new_draft flag until first save is confirmed ([#9656](https://github.com/windmill-labs/windmill/issues/9656)) ([9b6b7c3](https://github.com/windmill-labs/windmill/commit/9b6b7c3862d9988e5e91eaab2b967a23f41cdc0d))
* **frontend:** re-key raw-app autosave on post-deploy navigation ([#9646](https://github.com/windmill-labs/windmill/issues/9646)) ([1058bde](https://github.com/windmill-labs/windmill/commit/1058bdeccdc4c403ef4599db0ee74a65a66c715f))
* gate agent-worker global setting reads with a blocklist ([#9623](https://github.com/windmill-labs/windmill/issues/9623)) ([fdd82f0](https://github.com/windmill-labs/windmill/commit/fdd82f0c48f29805cd9e219649f27fba45c7fd92))
* **workspaces:** add instance setting to disable workspace invite/add emails ([#9643](https://github.com/windmill-labs/windmill/issues/9643)) ([796230d](https://github.com/windmill-labs/windmill/commit/796230d90a7e6d1debc15e139ab708881e527862))
## [1.729.0](https://github.com/windmill-labs/windmill/compare/v1.728.1...v1.729.0) (2026-06-18)
### Features
* add ducklake schema support to the database manager ([#9633](https://github.com/windmill-labs/windmill/issues/9633)) ([3eeccaf](https://github.com/windmill-labs/windmill/commit/3eeccaf9682b7803fdf5be8dcbc4d243e0ba2e49))
* **ai-chat:** self-hosted docs tools via windmill.dev llms.txt + ask benchmark ([#9578](https://github.com/windmill-labs/windmill/issues/9578)) ([f4425fc](https://github.com/windmill-labs/windmill/commit/f4425fca9fb0d02b845bd72888ade54905c5a30b))
* **frontend:** View Diff and in-place Load for other users' drafts ([#9621](https://github.com/windmill-labs/windmill/issues/9621)) ([5508f1d](https://github.com/windmill-labs/windmill/commit/5508f1da9cd04c2583eb3f7ee6bce19d067f2227))
* per-user draft review & deploy page (gating, badges, rename, raw-app deploy fixes) ([#9625](https://github.com/windmill-labs/windmill/issues/9625)) ([e09cd58](https://github.com/windmill-labs/windmill/commit/e09cd5862cb636e143027fe8d9a5be9c7097b031))
* queue messages typed while ai chat is streaming ([#9525](https://github.com/windmill-labs/windmill/issues/9525)) ([51bd869](https://github.com/windmill-labs/windmill/commit/51bd8692a482850f7ac8b04dd16db5876336b5b9))
* zero-setup oauth client credentials for registry providers ([#9559](https://github.com/windmill-labs/windmill/issues/9559)) ([e26a923](https://github.com/windmill-labs/windmill/commit/e26a9239a62a25abf90ef06ade4dde7f36e791bb))
### Bug Fixes
* **ai_evals:** adapt global eval harness to DB-backed user drafts ([#9641](https://github.com/windmill-labs/windmill/issues/9641)) ([e87ff79](https://github.com/windmill-labs/windmill/commit/e87ff79ecf6a6e0958916ed1b3756fb3addf719f))
* **drafts:** preserve original timestamp when migrating localStorage drafts ([#9638](https://github.com/windmill-labs/windmill/issues/9638)) ([8021775](https://github.com/windmill-labs/windmill/commit/8021775f5f961ef6fd01b022639b85855326a1da))
* **frontend:** don't save drafts on leave when auto-save is off, warn instead ([#9630](https://github.com/windmill-labs/windmill/issues/9630)) ([2523465](https://github.com/windmill-labs/windmill/commit/252346500945a9571af744c839ac0c7d6870504f))
* **frontend:** render Modal2 dialogs above the AI chat panel ([#9636](https://github.com/windmill-labs/windmill/issues/9636)) ([b67c8cf](https://github.com/windmill-labs/windmill/commit/b67c8cf42b477575fc1bc448058ec0d3b7e54fee))
* **frontend:** show AI sessions when AI unconfigured, with disabled chat ([#9644](https://github.com/windmill-labs/windmill/issues/9644)) ([ba69d81](https://github.com/windmill-labs/windmill/commit/ba69d8147b615e160cf3d2885fc65a0777b78b71))
* **git-sync:** bump default sync script to hub/28719 (windmill-cli 1.728.1) for WAC modules ([#9649](https://github.com/windmill-labs/windmill/issues/9649)) ([3c0e38b](https://github.com/windmill-labs/windmill/commit/3c0e38b5890d77983cb5cf5f422a62a73e7a4f22))
## [1.728.1](https://github.com/windmill-labs/windmill/compare/v1.728.0...v1.728.1) (2026-06-17)
### Bug Fixes
* **backend:** purge workspace_diff cache on workspace delete ([#9627](https://github.com/windmill-labs/windmill/issues/9627)) ([8a3f69d](https://github.com/windmill-labs/windmill/commit/8a3f69dda8f2088fb859ed8ed6e54458940423d0))
* **cli:** fall back to esbuild-wasm on native host/binary mismatch ([#9629](https://github.com/windmill-labs/windmill/issues/9629)) ([86d1d16](https://github.com/windmill-labs/windmill/commit/86d1d160f0d3bd9faabdafada07e2956dd98445d))
* **frontend:** persist session-editor draft path/summary edits + per-line diff tooltips ([#9622](https://github.com/windmill-labs/windmill/issues/9622)) ([e4bfeb2](https://github.com/windmill-labs/windmill/commit/e4bfeb29bc4e89669863b5f6396904a331167658))
## [1.728.0](https://github.com/windmill-labs/windmill/compare/v1.727.0...v1.728.0) (2026-06-16)
### Features
* **frontend:** adapt AI-chat/sessions drafts to DB-backed model ([#9601](https://github.com/windmill-labs/windmill/issues/9601)) ([611c70a](https://github.com/windmill-labs/windmill/commit/611c70acd211cf4b8f8308da4a264c670a2f5f43))
* **frontend:** consolidate draft-migration errors into a single toast + modal ([#9612](https://github.com/windmill-labs/windmill/issues/9612)) ([bc0d5bf](https://github.com/windmill-labs/windmill/commit/bc0d5bf241df3633921bd9d43d171e91034fbfcf))
* **frontend:** dedup user drafts against the deployed baseline ([#9618](https://github.com/windmill-labs/windmill/issues/9618)) ([a2ce446](https://github.com/windmill-labs/windmill/commit/a2ce44645fdbfa98bf250fac2d15d2b5b26c4b47))
### Bug Fixes
* **frontend:** reset deleteWorkspaceForkModal on confirm in SidebarContent ([#9619](https://github.com/windmill-labs/windmill/issues/9619)) ([7cb5c6e](https://github.com/windmill-labs/windmill/commit/7cb5c6e749b2020dee5ee1499f0dc69c5109a6d8))
* **frontend:** session Drafts drawer uses raw_app kind for the raw-app diff ([#9617](https://github.com/windmill-labs/windmill/issues/9617)) ([46288b6](https://github.com/windmill-labs/windmill/commit/46288b6143efae4dfdf6fe068b97a1e8831fce6a))
* **nativets:** respect custom CA certs in in-process fetch runtime ([#9615](https://github.com/windmill-labs/windmill/issues/9615)) ([41562c7](https://github.com/windmill-labs/windmill/commit/41562c7d7c708d7d056d9b3d0c39b994a6f4a016))
* **ResourceForm:** initialize JSON editor when resource type schema is unavailable ([#9611](https://github.com/windmill-labs/windmill/issues/9611)) ([5a24057](https://github.com/windmill-labs/windmill/commit/5a2405743b4622fc1021109114d007057abd5dfd))
* show folder labels in the folder list table ([#9620](https://github.com/windmill-labs/windmill/issues/9620)) ([651fa13](https://github.com/windmill-labs/windmill/commit/651fa13ee80ff76e5a53ef1ed545b03ce6792294))
* show last updated date per user in other-users-drafts modal ([#9614](https://github.com/windmill-labs/windmill/issues/9614)) ([f6104ce](https://github.com/windmill-labs/windmill/commit/f6104ce05c4005ffb9fe8112782d1ef6d3065300))
## [1.727.0](https://github.com/windmill-labs/windmill/compare/v1.726.1...v1.727.0) (2026-06-16)
ai_evals/adapters/cli/runtime.ts
+11
View File
@@ -16,6 +16,9 @@ export interface PromptRunResult {
output: string;
durationMs: number;
tokenUsage: BenchmarkTokenUsage | null;
// Input tokens on the last assistant turn. The SDK `result` message reports
// usage cumulatively, so the final context size comes from per-turn usage.
finalContextTokens: number | null;
trace: CliTrace;
}
@@ -144,6 +147,7 @@ export async function runPromptAndCapture(
let output = "";
let assistantMessageCount = 0;
let tokenUsage: BenchmarkTokenUsage | null = null;
let finalContextTokens: number | null = null;
const startedAt = Date.now();
const stubBinDir = join(cwd, WMILL_STUB_DIR_NAME);
const wmillLogPath = join(cwd, WMILL_LOG_FILE_NAME);
@@ -166,6 +170,12 @@ export async function runPromptAndCapture(
for await (const message of query({ prompt, options })) {
if (message.type === "assistant") {
assistantMessageCount += 1;
const turnContext = anthropicUsageToBenchmarkTokenUsage(
message.message?.usage
)?.prompt;
if (turnContext && turnContext > 0) {
finalContextTokens = turnContext;
}
const content = message.message?.content;
if (Array.isArray(content)) {
for (const block of content) {
@@ -210,6 +220,7 @@ export async function runPromptAndCapture(
output,
durationMs: Date.now() - startedAt,
tokenUsage,
finalContextTokens,
trace: {
toolsUsed,
skillsInvoked,
ai_evals/adapters/frontend/benchmarkRunner.ts
+6 -1
View File
@@ -96,7 +96,12 @@ async function getModeRunner(
}
function parseMode(value: string | undefined): FrontendBenchmarkMode {
if (value === "flow" || value === "app" || value === "script" || value === "global") {
if (
value === "flow" ||
value === "app" ||
value === "script" ||
value === "global"
) {
return value;
}
throw new Error(`Unsupported frontend benchmark mode: ${String(value)}`);
ai_evals/adapters/frontend/core/app/appEvalRunner.ts
+2
View File
@@ -38,6 +38,7 @@ export interface AppEvalResult {
toolCallCount: number;
toolsUsed: string[];
tokenUsage: TokenUsage;
finalContextTokens: number | null;
}
export interface AppEvalOptions {
@@ -113,6 +114,7 @@ export async function runAppEval(
toolCallCount: rawResult.toolCallsCount,
toolsUsed: rawResult.toolsCalled,
tokenUsage: rawResult.tokenUsage,
finalContextTokens: rawResult.finalContextTokens,
};
} finally {
await cleanup();
ai_evals/adapters/frontend/core/flow/flowEvalRunner.ts
+2
View File
@@ -39,6 +39,7 @@ export interface FlowEvalResult {
toolsUsed: string[];
toolCallDetails: ToolCallDetail[];
tokenUsage: TokenUsage;
finalContextTokens: number | null;
}
export interface FlowEvalOptions {
@@ -113,6 +114,7 @@ export async function runFlowEval(
toolsUsed: rawResult.toolsCalled,
toolCallDetails: rawResult.toolCallDetails,
tokenUsage: rawResult.tokenUsage,
finalContextTokens: rawResult.finalContextTokens,
};
} finally {
await cleanup();
ai_evals/adapters/frontend/core/global/globalEvalRunner.ts
+61 -22
View File
@@ -9,6 +9,7 @@ import {
} from "../../../../../frontend/src/lib/components/copilot/chat/global/core";
import {
clearGlobalDrafts,
getGlobalDraft,
listGlobalDrafts,
} from "../../../../../frontend/src/lib/components/copilot/chat/global/userDraftAdapter";
import type { Tool as ProductionTool } from "../../../../../frontend/src/lib/components/copilot/chat/shared";
@@ -18,6 +19,7 @@ import type { GlobalDraftState } from "../../../../core/validators";
import type { WindmillBackendSettings } from "../../../../core/windmillBackendSettings";
import {
registerBenchmarkWorkspaceRunnables,
seedBenchmarkDraft,
unregisterBenchmarkWorkspaceRunnables,
type BenchmarkWorkspaceRunnables,
} from "../../mockBackend";
@@ -30,6 +32,10 @@ const MUTATING_GLOBAL_TOOLS = new Set([
]);
const DISABLE_ACTIVE_EDITOR_CONTEXT_ENV =
"WMILL_AI_EVAL_DISABLE_ACTIVE_EDITOR_CONTEXT";
// A/B gate for the search_app read tool: set to "1" to run the baseline arm
// (toolset without search_app) so its token cost can be compared against the arm
// that offers it.
const DISABLE_SEARCH_APP_ENV = "WMILL_AI_EVAL_DISABLE_SEARCH_APP";
const LIVE_EDITOR_ITEM_KINDS = {
script: "script",
@@ -53,6 +59,7 @@ export interface GlobalEvalResult {
toolsUsed: string[];
toolCallDetails: ToolCallDetail[];
tokenUsage: TokenUsage;
finalContextTokens: number | null;
}
export interface GlobalEvalOptions {
@@ -94,7 +101,7 @@ export async function runGlobalEval(
tools: getGlobalEvalTools(),
helpers: {},
apiKey,
getOutput: () => ({ drafts: listGlobalDrafts(workspaceRoot) }),
getOutput: () => collectGlobalDraftState(workspaceRoot),
onAssistantMessageStart: options.runContext?.onAssistantMessageStart,
onAssistantToken: options.runContext?.onAssistantChunk,
onAssistantMessageEnd: options.runContext?.onAssistantMessageEnd,
@@ -119,6 +126,7 @@ export async function runGlobalEval(
toolsUsed: rawResult.toolsCalled,
toolCallDetails: rawResult.toolCallDetails,
tokenUsage: rawResult.tokenUsage,
finalContextTokens: rawResult.finalContextTokens,
};
} finally {
clearGlobalDrafts(workspaceRoot);
@@ -130,6 +138,32 @@ export async function runGlobalEval(
}
}
// Build the harness output from the DB-backed drafts. `listGlobalDrafts` returns
// metadata-only rows for backend drafts (the model's `write_script` etc. persist
// straight to the backend with no in-tab editor cell), so re-read each such row
// with `getGlobalDraft` to attach the full value the validators assert on. A row
// that already carries a value (the production in-tab cell overlay) is kept as-is.
async function collectGlobalDraftState(
workspace: string,
): Promise<GlobalDraftState> {
const items = await listGlobalDrafts(workspace);
const drafts = await Promise.all(
items.map(async (item) => {
if (item.value !== undefined) {
return item;
}
const full = await getGlobalDraft(
workspace,
item.type,
item.path,
item.triggerKind,
);
return full ?? item;
}),
);
return { drafts: drafts as GlobalDraftState["drafts"] };
}
function seedLiveEditorDrafts(
workspace: string,
fixtures: GlobalLiveEditorDraftFixture[],
@@ -138,7 +172,9 @@ function seedLiveEditorDrafts(
const itemKind = LIVE_EDITOR_ITEM_KINDS[fixture.type];
const storagePath = fixture.storagePath ?? fixture.effectivePath ?? "";
if (fixture.value !== undefined) {
UserDraft.save(itemKind, storagePath, fixture.value, { workspace });
// Seed as a backend draft row, not an in-tab cell: a cell would shadow the
// model's DB-backed edit when the output is read back via listGlobalDrafts.
seedBenchmarkDraft(workspace, itemKind, storagePath, fixture.value);
}
UserDraft.setLiveEditorDraft({
workspace,
@@ -161,25 +197,28 @@ function clearLiveEditorDrafts(
}
function getGlobalEvalTools(): ProductionTool<{}>[] {
return (globalTools as ProductionTool<{}>[]).map((tool) => {
if (!MUTATING_GLOBAL_TOOLS.has(tool.def.function.name)) {
return tool;
}
const disableSearchApp = process.env[DISABLE_SEARCH_APP_ENV] === "1";
return (globalTools as ProductionTool<{}>[])
.filter((tool) => !(disableSearchApp && tool.def.function.name === "search_app"))
.map((tool) => {
if (!MUTATING_GLOBAL_TOOLS.has(tool.def.function.name)) {
return tool;
}
return {
...tool,
requiresConfirmation: false,
validateBeforeConfirmation: undefined,
fn: async () =>
JSON.stringify(
{
success: false,
error:
"This mutating workspace tool is disabled during ai_evals global mode.",
},
null,
2,
),
};
});
return {
...tool,
requiresConfirmation: false,
validateBeforeConfirmation: undefined,
fn: async () =>
JSON.stringify(
{
success: false,
error:
"This mutating workspace tool is disabled during ai_evals global mode.",
},
null,
2,
),
};
});
}
ai_evals/adapters/frontend/core/script/scriptEvalRunner.ts
+2
View File
@@ -25,6 +25,7 @@ export interface ScriptEvalResult {
toolsUsed: string[];
toolCallDetails: ToolCallDetail[];
tokenUsage: TokenUsage;
finalContextTokens: number | null;
}
export interface ScriptEvalOptions {
@@ -111,6 +112,7 @@ export async function runScriptEval(
toolsUsed: rawResult.toolsCalled,
toolCallDetails: rawResult.toolCallDetails,
tokenUsage: rawResult.tokenUsage,
finalContextTokens: rawResult.finalContextTokens,
};
} finally {
await cleanup();
ai_evals/adapters/frontend/core/shared/baseEvalRunner.ts
+9 -5
View File
@@ -38,8 +38,9 @@ export interface RunEvalParams<THelpers, TOutput> {
helpers: THelpers;
/** API key for the provider */
apiKey: string;
/** Function to get the current output state */
getOutput: () => TOutput;
/** Function to get the current output state. May be async — global mode reads
* DB-backed drafts back through the (mocked) backend to build its output. */
getOutput: () => TOutput | Promise<TOutput>;
/** Model and Windmill backend configuration */
options: EvalRunnerOptions;
onAssistantMessageStart?: () => void;
@@ -154,9 +155,10 @@ export async function runEval<THelpers, TOutput>(
if (result.hitMaxIterations) {
return {
success: false,
output: getOutput(),
output: (await getOutput()) as TOutput,
error: `Reached max turns (${maxIterations})`,
tokenUsage: result.tokenUsage,
finalContextTokens: result.lastIterationUsage?.prompt ?? null,
toolCallsCount,
toolsCalled,
toolCallDetails,
@@ -170,8 +172,9 @@ export async function runEval<THelpers, TOutput>(
return {
success: true,
output: getOutput(),
output: (await getOutput()) as TOutput,
tokenUsage: result.tokenUsage,
finalContextTokens: result.lastIterationUsage?.prompt ?? null,
toolCallsCount,
toolsCalled,
toolCallDetails,
@@ -191,9 +194,10 @@ export async function runEval<THelpers, TOutput>(
return {
success: false,
output: getOutput(),
output: (await getOutput()) as TOutput,
error: errorMessage,
tokenUsage: { prompt: 0, completion: 0, total: 0 },
finalContextTokens: null,
toolCallsCount,
toolsCalled,
toolCallDetails,
ai_evals/adapters/frontend/core/shared/types.ts
+2
View File
@@ -28,6 +28,8 @@ export interface RawEvalResult<TOutput> {
output: TOutput;
error?: string;
tokenUsage: TokenUsage;
/** Input tokens on the last model request of the loop (see BenchmarkAttemptResult.finalContextTokens). */
finalContextTokens: number | null;
toolCallsCount: number;
toolsCalled: string[];
toolCallDetails: ToolCallDetail[];
ai_evals/adapters/frontend/mockBackend.ts
+182 -2
View File
@@ -1,9 +1,20 @@
import { randomUUID } from 'node:crypto'
import type { CompletedJob, Flow, Job, Script } from '../../../frontend/src/lib/gen'
import type {
AppWithLastVersion,
CompletedJob,
Flow,
Job,
ListableApp,
Script
} from '../../../frontend/src/lib/gen'
import type {
DataTableTables,
DataTableTableSchema,
ScriptLang
GetDraftForUserResponse,
ListDraftsResponse,
ScriptLang,
UpdateDraftResponse,
UserDraftItemKind
} from '../../../frontend/src/lib/gen/types.gen'
import { buildScriptLintResult } from './core/script/preview'
import { applyDatatableSql, type BenchmarkDatatableSeed } from './datatableSqlEngine'
@@ -29,6 +40,18 @@ export interface BenchmarkWorkspaceFlow {
value: Flow['value']
}
export interface BenchmarkWorkspaceApp {
path: string
summary: string
value: {
files: Record<string, string>
runnables: Record<string, unknown>
data?: unknown
policy?: unknown
custom_path?: unknown
}
}
export interface BenchmarkWorkspaceJob {
/** Stable id so a case prompt can reference a specific run (e.g. for get_job_logs). */
id?: string
@@ -43,6 +66,7 @@ export interface BenchmarkWorkspaceJob {
export interface BenchmarkWorkspaceRunnables {
scripts?: BenchmarkWorkspaceScript[]
flows?: BenchmarkWorkspaceFlow[]
apps?: BenchmarkWorkspaceApp[]
datatables?: BenchmarkDatatableSeed[]
jobs?: BenchmarkWorkspaceJob[]
}
@@ -63,6 +87,7 @@ export function resetBenchmarkMockBackend(): void {
benchmarkWorkspaces.clear()
benchmarkWorkspaceRunnables.clear()
benchmarkJobs.clear()
benchmarkDrafts.clear()
}
export function registerBenchmarkWorkspace(workspace: string): void {
@@ -74,6 +99,8 @@ export function registerBenchmarkWorkspaceRunnables(
runnables: BenchmarkWorkspaceRunnables
): void {
benchmarkWorkspaces.add(workspace)
// Fresh case: drop any drafts left from a prior run on this workspace id.
clearBenchmarkDrafts(workspace)
// Datatables are mutated in place by exec_datatable_sql (a write must be visible
// to later reads), so store an isolated deep copy — never mutate the caller's seed.
benchmarkWorkspaceRunnables.set(workspace, {
@@ -98,6 +125,7 @@ export function registerBenchmarkWorkspaceRunnables(
export function unregisterBenchmarkWorkspace(workspace: string): void {
benchmarkWorkspaces.delete(workspace)
benchmarkWorkspaceRunnables.delete(workspace)
clearBenchmarkDrafts(workspace)
for (const [jobId, entry] of benchmarkJobs.entries()) {
if (entry.workspace === workspace) {
benchmarkJobs.delete(jobId)
@@ -153,6 +181,22 @@ export function getBenchmarkFlowByPath(workspace: string, path: string): Flow |
return flow ? buildBenchmarkFlow(flow) : null
}
export function listBenchmarkApps(workspace: string): ListableApp[] | null {
const runnables = benchmarkWorkspaceRunnables.get(workspace)
if (!runnables) {
return null
}
return (runnables.apps ?? []).map(buildBenchmarkListableApp)
}
export function getBenchmarkAppByPath(workspace: string, path: string): AppWithLastVersion | null {
const app = benchmarkWorkspaceRunnables
.get(workspace)
?.apps?.find((entry) => entry.path === path)
return app ? buildBenchmarkApp(app) : null
}
export function createBenchmarkCompletedJob(input: {
workspace: string
jobKind: CompletedJob['job_kind']
@@ -238,6 +282,110 @@ export function getBenchmarkJobLogs(workspace: string, jobId: string): string {
return job.logs ?? ''
}
// ============= Drafts (per-user, DB-backed in production) =============
/**
* In-memory stand-in for the per-user draft backend (`DraftService`). The global
* AI chat now persists and reads drafts through the backend DB instead of an
* in-tab `UserDraft` cell, so the eval mocks the three draft endpoints it
* exercises (`updateDraft` / `getDraftForUser` / `listDrafts`) and keeps the
* saved values here, keyed by workspace + draft kind + storage path. Mirrors the
* semantics of the production unit test's mock in
* `frontend/src/lib/components/copilot/chat/global/core.test.ts`.
*/
const benchmarkDrafts = new Map<
string,
{ workspace: string; kind: UserDraftItemKind; path: string; value: unknown }
>()
// Fixed timestamp so artifacts stay deterministic. No eval simulates a
// concurrent writer, so every save is accepted and the conflict branch is
// never taken — the syncer just records this as its `last_sync` baseline.
const BENCHMARK_DRAFT_TIMESTAMP = '1970-01-01T00:00:00.000Z'
function benchmarkDraftKey(workspace: string, kind: string, path: string): string {
return `${workspace}::${kind}::${path}`
}
export function clearBenchmarkDrafts(workspace: string): void {
for (const [key, entry] of benchmarkDrafts.entries()) {
if (entry.workspace === workspace) {
benchmarkDrafts.delete(key)
}
}
}
/**
* Seed a draft straight into the store — used by the eval's live-editor draft
* fixtures, which model "the user already has this draft open/saved". Writing it
* here (instead of through `UserDraft.save`) keeps it a backend draft row with no
* shadowing in-tab cell, so a model edit that persists to the backend is what the
* output read-back captures — not the stale seed.
*/
export function seedBenchmarkDraft(
workspace: string,
kind: UserDraftItemKind,
path: string,
value: unknown
): void {
benchmarkDrafts.set(benchmarkDraftKey(workspace, kind, path), {
workspace,
kind,
path,
value
})
}
/** Mirror `DraftService.updateDraft`: a `null`/omitted value deletes the row. */
export function updateBenchmarkDraft(input: {
workspace: string
kind: UserDraftItemKind
path: string
requestBody?: { value?: unknown }
}): UpdateDraftResponse {
const key = benchmarkDraftKey(input.workspace, input.kind, input.path)
const value = input.requestBody?.value
if (value == null) {
benchmarkDrafts.delete(key)
} else {
benchmarkDrafts.set(key, {
workspace: input.workspace,
kind: input.kind,
path: input.path,
value
})
}
return { status: 'saved', current_timestamp: BENCHMARK_DRAFT_TIMESTAMP }
}
/** Mirror `DraftService.getDraftForUser`: 404-shaped throw when absent so the
* adapter's narrowed catch treats it as "no draft" instead of re-throwing. */
export function getBenchmarkDraftForUser(input: {
workspace: string
kind: UserDraftItemKind
path: string
}): GetDraftForUserResponse {
const entry = benchmarkDrafts.get(benchmarkDraftKey(input.workspace, input.kind, input.path))
if (!entry) {
throw Object.assign(new Error(`no draft for "${input.path}"`), { status: 404 })
}
return { value: entry.value, created_at: BENCHMARK_DRAFT_TIMESTAMP }
}
/** Mirror `DraftService.listDrafts`: metadata rows (no value) for a workspace. */
export function listBenchmarkDrafts(workspace: string): ListDraftsResponse {
return [...benchmarkDrafts.values()]
.filter((entry) => entry.workspace === workspace)
.map((entry) => ({
kind: entry.kind,
path: entry.path,
summary: (entry.value as { summary?: string } | null)?.summary,
draft_only: true,
legacy_draft: false,
created_at: BENCHMARK_DRAFT_TIMESTAMP
}))
}
// ============= Datatables (best-effort in-memory SQL) =============
/**
@@ -492,3 +640,35 @@ function buildBenchmarkFlow(flow: BenchmarkWorkspaceFlow): Flow {
extra_perms: {}
} as Flow
}
function buildBenchmarkListableApp(app: BenchmarkWorkspaceApp): ListableApp {
return {
id: 0,
workspace_id: 'benchmark',
path: app.path,
summary: app.summary,
version: 1,
extra_perms: {},
edited_at: BENCHMARK_TIMESTAMP,
execution_mode: 'viewer',
raw_app: true
}
}
function buildBenchmarkApp(app: BenchmarkWorkspaceApp): AppWithLastVersion {
return {
id: 0,
workspace_id: 'benchmark',
path: app.path,
summary: app.summary,
versions: [1],
created_by: 'benchmark',
created_at: BENCHMARK_TIMESTAMP,
value: app.value,
policy: (app.value.policy ?? {}) as AppWithLastVersion['policy'],
execution_mode: 'viewer',
extra_perms: {},
custom_path: app.value.custom_path as string | undefined,
raw_app: true
}
}
ai_evals/adapters/frontend/mockBackendDrafts.test.ts
+94
View File
@@ -0,0 +1,94 @@
import { afterEach, beforeEach, describe, expect, it } from 'bun:test'
import {
clearBenchmarkDrafts,
getBenchmarkDraftForUser,
listBenchmarkDrafts,
resetBenchmarkMockBackend,
seedBenchmarkDraft,
updateBenchmarkDraft
} from './mockBackend'
const WORKSPACE = 'benchmark-drafts-ws'
// Drives the in-memory stand-in for the per-user draft backend (`DraftService`)
// that the global AI-chat eval round-trips its drafts through. Mirrors the
// production-unit-test mock in
// `frontend/src/lib/components/copilot/chat/global/core.test.ts`.
describe('mockBackend drafts', () => {
beforeEach(() => resetBenchmarkMockBackend())
afterEach(() => resetBenchmarkMockBackend())
it('round-trips a saved draft through update / get / list', () => {
const value = { summary: 'Greet a user', content: 'export async function main() {}' }
const res = updateBenchmarkDraft({
workspace: WORKSPACE,
kind: 'script',
path: 'f/evals/greet',
requestBody: { value }
})
expect(res.status).toBe('saved')
expect(getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'script', path: 'f/evals/greet' }).value).toEqual(
value
)
const rows = listBenchmarkDrafts(WORKSPACE)
expect(rows).toHaveLength(1)
expect(rows[0]).toMatchObject({ kind: 'script', path: 'f/evals/greet', summary: 'Greet a user', draft_only: true })
})
it('treats a null value as a delete', () => {
updateBenchmarkDraft({
workspace: WORKSPACE,
kind: 'variable',
path: 'f/evals/token',
requestBody: { value: { summary: 'token' } }
})
updateBenchmarkDraft({
workspace: WORKSPACE,
kind: 'variable',
path: 'f/evals/token',
requestBody: { value: null }
})
expect(listBenchmarkDrafts(WORKSPACE)).toHaveLength(0)
expect(() => getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'variable', path: 'f/evals/token' })).toThrow()
})
it('throws a 404-shaped error when no draft exists', () => {
try {
getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'script', path: 'f/evals/missing' })
throw new Error('expected a throw')
} catch (e) {
expect((e as { status?: number }).status).toBe(404)
}
})
it('seeds a draft as a backend row that a later edit overwrites', () => {
seedBenchmarkDraft(WORKSPACE, 'script', 'f/evals/current', { content: 'seed' })
expect(getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'script', path: 'f/evals/current' }).value).toEqual({
content: 'seed'
})
// A model edit persists the same path and must win over the seed.
updateBenchmarkDraft({
workspace: WORKSPACE,
kind: 'script',
path: 'f/evals/current',
requestBody: { value: { content: 'edited' } }
})
expect(getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'script', path: 'f/evals/current' }).value).toEqual({
content: 'edited'
})
})
it('clears only the targeted workspace', () => {
seedBenchmarkDraft(WORKSPACE, 'script', 'f/a', { content: 'a' })
seedBenchmarkDraft('other-ws', 'script', 'f/b', { content: 'b' })
clearBenchmarkDrafts(WORKSPACE)
expect(listBenchmarkDrafts(WORKSPACE)).toHaveLength(0)
expect(listBenchmarkDrafts('other-ws')).toHaveLength(1)
})
})
ai_evals/adapters/frontend/vitestAdapter.test.ts
+38 -5
View File
@@ -33,15 +33,19 @@ vi.mock('$lib/components/vscode', () => ({}))
vi.mock('$lib/gen', async () => {
const actual = await vi.importActual<any>('$lib/gen')
const {
getBenchmarkAppByPath,
getBenchmarkCompletedJob,
getBenchmarkCompletedJobResultMaybe,
getBenchmarkDatatableSchema,
getBenchmarkDraftForUser,
getBenchmarkFlowByPath,
getBenchmarkJobLogs,
getBenchmarkScriptByHash,
getBenchmarkScriptByPath,
hasBenchmarkWorkspace,
listBenchmarkApps,
listBenchmarkDatatables,
listBenchmarkDrafts,
listBenchmarkFlows,
listBenchmarkJobs,
listBenchmarkScripts,
@@ -50,7 +54,8 @@ vi.mock('$lib/gen', async () => {
previewBenchmarkSchedule,
runBenchmarkDatatableSql,
runBenchmarkFlowByPath,
runBenchmarkScriptPreview
runBenchmarkScriptPreview,
updateBenchmarkDraft
} = await import('./mockBackend')
function wrapService<T extends object>(target: T, overrides: Record<string, unknown>): T {
@@ -66,6 +71,25 @@ vi.mock('$lib/gen', async () => {
return {
...actual,
DraftService: wrapService(actual.DraftService, {
updateDraft: async (data: {
workspace: string
kind: any
path: string
requestBody?: { value?: unknown }
}) =>
hasBenchmarkWorkspace(data.workspace)
? updateBenchmarkDraft(data)
: actual.DraftService.updateDraft(data),
getDraftForUser: async (data: { workspace: string; kind: any; path: string }) =>
hasBenchmarkWorkspace(data.workspace)
? getBenchmarkDraftForUser(data)
: actual.DraftService.getDraftForUser(data),
listDrafts: async (data: { workspace: string }) =>
hasBenchmarkWorkspace(data.workspace)
? listBenchmarkDrafts(data.workspace)
: actual.DraftService.listDrafts(data)
}),
ScriptService: wrapService(actual.ScriptService, {
listScripts: async (data: { workspace: string }) =>
hasBenchmarkWorkspace(data.workspace)
@@ -277,12 +301,20 @@ vi.mock('$lib/gen', async () => {
}),
AppService: wrapService(actual.AppService, {
existsApp: async (data: { workspace: string; path: string }) =>
hasBenchmarkWorkspace(data.workspace) ? false : actual.AppService.existsApp(data),
hasBenchmarkWorkspace(data.workspace)
? Boolean(getBenchmarkAppByPath(data.workspace, data.path))
: actual.AppService.existsApp(data),
listApps: async (data: { workspace: string }) =>
hasBenchmarkWorkspace(data.workspace) ? [] : actual.AppService.listApps(data),
hasBenchmarkWorkspace(data.workspace)
? (listBenchmarkApps(data.workspace) ?? [])
: actual.AppService.listApps(data),
getAppByPath: async (data: { workspace: string; path: string }) => {
if (hasBenchmarkWorkspace(data.workspace)) {
throw new Error(`App "${data.path}" not found in benchmark workspace`)
const app = getBenchmarkAppByPath(data.workspace, data.path)
if (!app) {
throw new Error(`App "${data.path}" not found in benchmark workspace`)
}
return app
}
return actual.AppService.getAppByPath(data)
}
@@ -434,5 +466,6 @@ benchmarkIt(
resetBenchmarkMockBackend()
}
},
600_000
// Full-suite runs (30+ cases at concurrency 2-3) routinely exceed 10 minutes.
7_200_000
)
ai_evals/cases/global.yaml
+243
View File
@@ -870,3 +870,246 @@
judgeChecklist:
- fetches the logs for the requested job id
- explains the failure from the returned logs (connection refused to the upstream API)
# --- Documentation search (search_docs) ---
# Pure product-knowledge questions: the assistant should consult the docs via
# search_docs and answer conversationally, not draft or mutate anything. No
# draft is produced, so the global judge is skipped and we validate tool use.
- id: global-docs-ai-agent-step
prompt: |-
Does Windmill support a flow step where an LLM decides which of my scripts to call based on the input?
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- search_docs
forbiddenToolsUsed:
- write_script
- write_flow
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
- id: global-docs-retry-step
prompt: |-
How does automatic retry work for a flow step that calls a flaky API?
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- search_docs
forbiddenToolsUsed:
- write_script
- write_flow
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
- id: global-docs-key-value-store
prompt: |-
Can I use a Redis-style key-value store from my Windmill scripts, and how?
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- search_docs
forbiddenToolsUsed:
- write_script
- write_flow
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
- id: global-docs-cron-schedule-format
prompt: |-
How do Windmill's cron schedules work, and what format does the schedule expression use?
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- search_docs
forbiddenToolsUsed:
- write_script
- write_flow
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
# --- Raw app on a large project (context-usage benchmark) ---
# These cases run against the deliberately large `analytics_dashboard` raw-app
# fixture (~20 frontend files incl. a 5k-line data module, plus backend runnables).
# They exist to measure how much context the global chat consumes when working in a
# big raw app: test29 is a read-heavy debugging hunt, test30 is a small edit baseline.
# tokenUsage is recorded per run, so the same cases re-run after a read-tool change
# (the read_app_file cap + offset/limit paging) quantify the optimization. skipJudge:
# the judge only sees the drafts artifact and cannot run the app, so we validate
# deterministically.
- id: global-test29-raw-app-debug-large
prompt: |-
The analytics dashboard app at `f/evals/global/analytics_dashboard` has a bug:
the Revenue Summary tile shows a total that is lower than the per-order line
totals and the per-region breakdown. Track down what is computing revenue
incorrectly and fix it. Keep the change as an AI draft only; do not deploy or
save it.
initial: ai_evals/fixtures/frontend/global/initial/analytics_dashboard
runtime:
maxTurns: 20
validate:
draftCountExactly: 1
requiredDrafts:
- type: app
path: f/evals/global/analytics_dashboard
valueIncludes:
- "return order.unitPrice * order.quantity"
toolExpect:
requiredToolsAnyOf:
# Inspecting the app's files is satisfied by either reading them directly
# or grepping for the revenue calculation.
- [read_app_file, search_app]
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- inspects the dashboard app's files to locate the revenue calculation
- fixes the per-order revenue so it multiplies unit price by quantity
- leaves the result as an AI draft and does not deploy or save it
- id: global-test30-raw-app-small-edit-large
prompt: |-
In the dashboard app at `f/evals/global/analytics_dashboard`, change the main
page heading from "Operations Console" to "Revenue Overview". Leave everything
else unchanged. Keep it as an AI draft only; do not deploy or save it.
initial: ai_evals/fixtures/frontend/global/initial/analytics_dashboard
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: app
path: f/evals/global/analytics_dashboard
valueIncludes:
- "Revenue Overview"
toolExpect:
requiredToolsAnyOf:
# Inspecting the app's files is satisfied by reading them directly or
# grepping for the target with search_app.
- [read_app_file, search_app]
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- renames the main page heading to Revenue Overview
- does not change other dashboard behavior
- leaves the result as an AI draft only
- id: global-test31-raw-app-debug-inspect-data
prompt: |-
The raw app dashboard at `f/evals/global/analytics_dashboard` is reporting
revenue totals that look too low. Inspect the app's files — both the sample
order data module and the revenue calculation — to work out whether the bug is
in the data or in the calculation, then fix the actual cause. Keep the change as
an AI draft only; do not deploy or save it.
initial: ai_evals/fixtures/frontend/global/initial/analytics_dashboard
runtime:
maxTurns: 22
validate:
draftCountExactly: 1
requiredDrafts:
- type: app
path: f/evals/global/analytics_dashboard
valueIncludes:
- "return order.unitPrice * order.quantity"
toolExpect:
requiredToolsAnyOf:
# Inspecting the app's files is satisfied by reading them directly or
# grepping for the target with search_app.
- [read_app_file, search_app]
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- inspects both the sample order data module and the revenue aggregation logic
- identifies the per-order revenue bug and fixes it to multiply unit price by quantity
- leaves the result as an AI draft only
- id: global-test32-raw-app-cross-file-consistency
prompt: |-
The raw app dashboard at `f/evals/global/analytics_dashboard` shows revenue
totals that disagree between the Revenue Summary tile, the orders table, and the
regional breakdown. Investigate how each of those computes revenue, work out
which calculation is wrong, and fix it. Keep the change as an AI draft only; do
not deploy or save it.
# Cross-file investigation: forces the model through several overlapping files
# (the summary's aggregation helper, the orders table, the regional breakdown) —
# a realistic multi-file read load that exercises the read_app_file cap.
initial: ai_evals/fixtures/frontend/global/initial/analytics_dashboard
runtime:
maxTurns: 24
validate:
draftCountExactly: 1
requiredDrafts:
- type: app
path: f/evals/global/analytics_dashboard
valueIncludes:
- "return order.unitPrice * order.quantity"
toolExpect:
requiredToolsAnyOf:
# Inspecting the app's files is satisfied by reading them directly or
# grepping for the target with search_app.
- [read_app_file, search_app]
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- inspects the revenue calculation behind the summary tile, the orders table, and the regional breakdown
- identifies that the per-order revenue helper omits quantity and fixes it to multiply unit price by quantity
- leaves the result as an AI draft only
- id: global-test33-raw-app-rename-across-files
prompt: |-
In the dashboard app at `f/evals/global/analytics_dashboard`, rename the
`formatCurrency` helper to `formatMoney` everywhere it is defined, imported, and
called. Leave the separate `formatCurrencyPrecise` helper exactly as it is. Keep
the change as an AI draft only; do not deploy or save it.
# Find-all-usages rename: formatCurrency is defined once and called in 6 places
# spread across 4 component files (and imported in 4). Locating every usage is the
# exact task search_app is meant to make cheap — one grep returns all file:line
# rows instead of reading each component whole. valueExcludes "formatCurrency("
# asserts the definition and all call sites were renamed while tolerating the
# preserved formatCurrencyPrecise (which is never followed by "(").
initial: ai_evals/fixtures/frontend/global/initial/analytics_dashboard
runtime:
maxTurns: 22
validate:
draftCountExactly: 1
requiredDrafts:
- type: app
path: f/evals/global/analytics_dashboard
valueIncludes:
- "export function formatMoney"
- "formatMoney("
valueExcludes:
- "formatCurrency("
toolExpect:
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- renames the formatCurrency definition, imports, and all call sites to formatMoney
- leaves the unrelated formatCurrencyPrecise helper unchanged
- leaves the result as an AI draft only
ai_evals/core/cases.test.ts
+15
View File
@@ -246,6 +246,21 @@ describe("loadCases", () => {
});
});
it("loads global docs-search cases as tool-use checks", async () => {
const globalCases = await loadCases("global");
const docsCases = globalCases.filter((entry) =>
entry.id.startsWith("global-docs-"),
);
expect(docsCases.length).toBeGreaterThanOrEqual(3);
// Each docs case verifies the assistant reaches for search_docs and does not
// draft anything; with no draft, the global judge is skipped.
for (const entry of docsCases) {
expect(entry.skipJudge).toBe(true);
expect(entry.toolExpect?.requiredToolsUsed).toContain("search_docs");
}
});
it("loads tool expectations for workspace mutation cases", async () => {
const scriptCases = await loadCases("script");
const caseEntry = scriptCases.find(
ai_evals/core/results.test.ts
+65
View File
@@ -92,6 +92,71 @@ describe("benchmark results", () => {
expect(summary).toContain("Average duration (all attempts): 550ms");
});
it("aggregates final context size over passed attempts only", () => {
const result = buildRunResult({
mode: "global",
runs: 1,
runModel: "model-under-test",
judgeModel: "judge-model",
caseResults: [
caseResult([
{
attempt: 1,
passed: true,
durationMs: 1000,
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["edit_script"],
skillsInvoked: [],
checks: [{ name: "edited", passed: true }],
judgeScore: 100,
judgeSummary: "ok",
error: null,
tokenUsage: { prompt: 12000, completion: 200, total: 12200 },
finalContextTokens: 5000,
},
{
attempt: 2,
passed: true,
durationMs: 1100,
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["edit_script"],
skillsInvoked: [],
checks: [{ name: "edited", passed: true }],
judgeScore: 100,
judgeSummary: "ok",
error: null,
tokenUsage: { prompt: 18000, completion: 300, total: 18300 },
finalContextTokens: 7000,
},
{
attempt: 3,
passed: false,
durationMs: 100,
assistantMessageCount: 1,
toolCallCount: 0,
toolsUsed: [],
skillsInvoked: [],
checks: [{ name: "edited", passed: false }],
judgeScore: 10,
judgeSummary: "missed",
error: "failed",
tokenUsage: { prompt: 20000, completion: 100, total: 20100 },
finalContextTokens: 9000,
},
]),
],
});
// Final context size stays below cumulative prompt and ignores the failed attempt.
expect(result.averageFinalContextTokensPassed).toBe(6000);
expect(result.maxFinalContextTokensPassed).toBe(7000);
expect(formatRunSummary(result)).toContain(
"Final context size (passed): 6000 tokens (max 7000)",
);
});
it("reports passed averages as unavailable when no attempt passes", () => {
const result = buildRunResult({
mode: "global",
ai_evals/core/results.ts
+34
View File
@@ -16,6 +16,9 @@ type AttemptAggregate = {
durationTotal: number;
tokenUsageAttemptCount: number;
tokenUsageTotal: BenchmarkTokenUsage | null;
finalContextAttemptCount: number;
finalContextTotal: number;
finalContextMax: number | null;
};
export async function writeRunResult(
@@ -117,6 +120,8 @@ export function buildRunResult(input: {
passedAttemptAggregate,
passedAttempts,
),
averageFinalContextTokensPassed: averageFinalContext(passedAttemptAggregate),
maxFinalContextTokensPassed: passedAttemptAggregate.finalContextMax,
cases: input.caseResults,
};
}
@@ -133,6 +138,11 @@ export function formatRunSummary(result: BenchmarkRunResult): string {
`Average tokens (passed): ${formatTokenUsage(result.averageTokenUsagePerPassedAttempt)}`,
);
}
if (result.averageFinalContextTokensPassed != null) {
lines.push(
`Final context size (passed): ${Math.round(result.averageFinalContextTokensPassed)} tokens (max ${Math.round(result.maxFinalContextTokensPassed ?? 0)})`,
);
}
if (result.passedAttempts < result.attemptCount) {
lines.push(
`Average duration (all attempts): ${Math.round(result.averageDurationMs)}ms`,
@@ -181,10 +191,21 @@ function aggregateAttempts(attempts: BenchmarkAttemptResult[]): AttemptAggregate
durationTotal: 0,
tokenUsageAttemptCount: 0,
tokenUsageTotal: null,
finalContextAttemptCount: 0,
finalContextTotal: 0,
finalContextMax: null,
};
for (const attempt of attempts) {
aggregate.durationTotal += attempt.durationMs;
if (typeof attempt.finalContextTokens === "number") {
aggregate.finalContextAttemptCount += 1;
aggregate.finalContextTotal += attempt.finalContextTokens;
aggregate.finalContextMax = Math.max(
aggregate.finalContextMax ?? 0,
attempt.finalContextTokens,
);
}
if (!attempt.tokenUsage) {
continue;
}
@@ -204,6 +225,12 @@ function averageDuration(aggregate: AttemptAggregate): number | null {
: aggregate.durationTotal / aggregate.attemptCount;
}
function averageFinalContext(aggregate: AttemptAggregate): number | null {
return aggregate.finalContextAttemptCount === 0
? null
: aggregate.finalContextTotal / aggregate.finalContextAttemptCount;
}
function averageTokenUsage(
aggregate: AttemptAggregate,
denominator: number,
@@ -318,6 +345,9 @@ function toHistoryRecord(result: BenchmarkRunResult) {
averageTokenUsagePerAttempt: result.averageTokenUsagePerAttempt ?? null,
averageTokenUsagePerPassedAttempt:
result.averageTokenUsagePerPassedAttempt ?? null,
averageFinalContextTokensPassed:
result.averageFinalContextTokensPassed ?? null,
maxFinalContextTokensPassed: result.maxFinalContextTokensPassed ?? null,
failedCaseIds: Array.from(
new Set(
result.cases
@@ -361,6 +391,10 @@ function toHistoryRecord(result: BenchmarkRunResult) {
passedAttemptAggregate,
passedAttempts,
),
averageFinalContextTokensPassed: averageFinalContext(
passedAttemptAggregate,
),
maxFinalContextTokensPassed: passedAttemptAggregate.finalContextMax,
};
}),
};
ai_evals/core/runSuite.ts
+5 -1
View File
@@ -225,7 +225,9 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
checklist: input.evalCase.judgeChecklist,
initial,
expected: input.modeRunner.mode === "cli" ? undefined : expected,
actual: run.actual,
actual: input.modeRunner.prepareJudgeActual
? input.modeRunner.prepareJudgeActual(run.actual)
: run.actual,
model: input.judgeModel,
});
@@ -255,6 +257,7 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
judgeSummary,
error: run.error ?? null,
tokenUsage: run.tokenUsage ?? null,
finalContextTokens: run.finalContextTokens ?? null,
artifactsPath: null,
artifactFiles,
};
@@ -291,6 +294,7 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
judgeSummary: null,
error: message,
tokenUsage: null,
finalContextTokens: null,
};
if (surface) {
input.onProgress?.({
ai_evals/core/types.ts
+26 -1
View File
@@ -168,11 +168,21 @@ export interface ToolCallArgumentRule {
export interface ToolValidationSpec {
requiredToolsUsed?: string[];
/**
* Each inner array is an alternatives group: the check passes when at least
* one tool in the group was used. Use when several tools satisfy the same
* intent so a model that picks any valid path passes — e.g. inspecting an
* app's files via either `read_app_file` or `search_app`.
*/
requiredToolsAnyOf?: string[][];
forbiddenToolsUsed?: string[];
toolCallArgs?: ToolCallArgumentRule[];
}
export type EvalValidationSpec = FlowValidationSpec | AppValidationSpec | GlobalValidationSpec;
export type EvalValidationSpec =
| FlowValidationSpec
| AppValidationSpec
| GlobalValidationSpec;
export interface EvalCase {
id: string;
@@ -249,6 +259,12 @@ export interface ModeRunOutput<TActual> {
toolCallDetails?: ToolCallDetail[];
skillsInvoked: string[];
tokenUsage?: BenchmarkTokenUsage | null;
/**
* Total input tokens occupying the context window on the LAST model request
* of the agentic loop (input + cache-creation + cache-read). Complements the
* cumulative `tokenUsage.prompt`, which sums every iteration's input.
*/
finalContextTokens?: number | null;
}
export interface ModeRunContext {
@@ -294,6 +310,12 @@ export interface ModeRunner<TInitial, TExpected, TActual> {
context: ModeRunContext;
}): Promise<BackendValidationResult | null>;
buildArtifacts?(actual: TActual): BenchmarkArtifactFile[];
/**
* Optional transform applied to `actual` before it is handed to the LLM judge.
* Use it to strip fields the judge must stay blind to (e.g. which docs-tool
* arm produced an answer). When omitted, the judge receives `actual` as-is.
*/
prepareJudgeActual?(actual: TActual): unknown;
}
export interface BenchmarkAttemptResult {
@@ -310,6 +332,7 @@ export interface BenchmarkAttemptResult {
judgeSummary: string | null;
error: string | null;
tokenUsage?: BenchmarkTokenUsage | null;
finalContextTokens?: number | null;
artifactsPath?: string | null;
artifactFiles?: BenchmarkArtifactFile[];
}
@@ -340,6 +363,8 @@ export interface BenchmarkRunResult {
totalPassedTokenUsage?: BenchmarkTokenUsage | null;
averageTokenUsagePerAttempt?: BenchmarkTokenUsage | null;
averageTokenUsagePerPassedAttempt?: BenchmarkTokenUsage | null;
averageFinalContextTokensPassed?: number | null;
maxFinalContextTokensPassed?: number | null;
artifactsPath?: string | null;
cases: BenchmarkCaseResult[];
}
ai_evals/core/validators.test.ts
+43
View File
@@ -245,6 +245,49 @@ describe("validateToolExpectations", () => {
'accepted substrings: insert into, update; values: "DROP TABLE orders"',
});
});
it("passes requiredToolsAnyOf when any alternative in the group is used", () => {
const checks = validateToolExpectations({
run: {
success: true,
actual: {},
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["search_app", "patch_app_file"],
skillsInvoked: [],
},
toolExpect: {
requiredToolsAnyOf: [["read_app_file", "search_app"]],
},
});
expect(checks).toContainEqual({
name: "uses one of read_app_file, search_app",
passed: true,
});
});
it("fails requiredToolsAnyOf when no alternative in the group is used", () => {
const checks = validateToolExpectations({
run: {
success: true,
actual: {},
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["patch_app_file"],
skillsInvoked: [],
},
toolExpect: {
requiredToolsAnyOf: [["read_app_file", "search_app"]],
},
});
expect(checks).toContainEqual({
name: "uses one of read_app_file, search_app",
passed: false,
details: "tools used: patch_app_file",
});
});
});
describe("validateGlobalState", () => {
ai_evals/core/validators.ts
+10
View File
@@ -169,6 +169,16 @@ export function validateToolExpectations(input: {
);
}
for (const group of expect.requiredToolsAnyOf ?? []) {
checks.push(
check(
`uses one of ${group.join(", ")}`,
group.some((toolName) => input.run.toolsUsed.includes(toolName)),
`tools used: ${input.run.toolsUsed.join(", ") || "none"}`
)
);
}
for (const toolName of expect.forbiddenToolsUsed ?? []) {
checks.push(
check(
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/computeSummary/main.ts
+68
View File
@@ -0,0 +1,68 @@
type OrderStatus = 'paid' | 'shipped' | 'delivered' | 'pending' | 'refunded' | 'cancelled'
interface Order {
id: string
region: string
quantity: number
unitPrice: number
status: OrderStatus
placedAt: string
}
// Server-side revenue rollup. Mirrors the client aggregation but is computed
// from the authoritative mocked order book so it can be used to cross-check
// the dashboard and to back the export.
const orders: Order[] = [
{ id: 'ORD-10001', region: 'North America', quantity: 3, unitPrice: 1195, status: 'delivered', placedAt: '2024-05-02' },
{ id: 'ORD-10002', region: 'EMEA', quantity: 5, unitPrice: 880, status: 'shipped', placedAt: '2024-05-03' },
{ id: 'ORD-10003', region: 'APAC', quantity: 2, unitPrice: 640, status: 'paid', placedAt: '2024-05-05' },
{ id: 'ORD-10004', region: 'LATAM', quantity: 7, unitPrice: 315, status: 'delivered', placedAt: '2024-05-07' },
{ id: 'ORD-10005', region: 'North America', quantity: 4, unitPrice: 150, status: 'refunded', placedAt: '2024-05-09' },
{ id: 'ORD-10006', region: 'EMEA', quantity: 6, unitPrice: 220, status: 'shipped', placedAt: '2024-05-12' },
{ id: 'ORD-10007', region: 'APAC', quantity: 1, unitPrice: 980, status: 'pending', placedAt: '2024-05-15' },
{ id: 'ORD-10008', region: 'North America', quantity: 8, unitPrice: 1100, status: 'delivered', placedAt: '2024-05-18' },
{ id: 'ORD-10009', region: 'EMEA', quantity: 2, unitPrice: 860, status: 'cancelled', placedAt: '2024-05-22' },
{ id: 'ORD-10010', region: 'LATAM', quantity: 9, unitPrice: 290, status: 'paid', placedAt: '2024-05-26' }
]
const REVENUE_STATUSES: OrderStatus[] = ['paid', 'shipped', 'delivered']
export async function main({
from,
to,
region
}: {
from: string
to: string
region: string
}): Promise<{
totalRevenue: number
netRevenue: number
totalOrders: number
averageOrderValue: number
unitsSold: number
refundedRevenue: number
currency: string
}> {
let scoped = orders.filter((order) => order.placedAt >= from && order.placedAt <= to)
if (region && region !== 'all') {
scoped = scoped.filter((order) => order.region === region)
}
const booked = scoped.filter((order) => REVENUE_STATUSES.includes(order.status))
const totalRevenue = booked.reduce((acc, order) => acc + order.unitPrice * order.quantity, 0)
const unitsSold = booked.reduce((acc, order) => acc + order.quantity, 0)
const refundedRevenue = scoped
.filter((order) => order.status === 'refunded')
.reduce((acc, order) => acc + order.unitPrice * order.quantity, 0)
return {
totalRevenue,
netRevenue: totalRevenue - refundedRevenue,
totalOrders: booked.length,
averageOrderValue: booked.length === 0 ? 0 : Math.round(totalRevenue / booked.length),
unitsSold,
refundedRevenue,
currency: 'USD'
}
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/computeSummary/meta.json
+4
View File
@@ -0,0 +1,4 @@
{
"name": "Compute Summary",
"language": "bun"
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/exportReport/main.ts
+51
View File
@@ -0,0 +1,51 @@
// Builds a downloadable report for the current dashboard view. Returns a data
// URL the browser can open directly so the export works without object storage.
export async function main({
from,
to,
region,
format
}: {
from: string
to: string
region: string
format: 'csv' | 'json'
}): Promise<{ url: string; rows: number; filename: string }> {
const summary = {
from,
to,
region: region || 'all',
generatedAt: new Date().toISOString(),
rows: [
{ region: 'North America', revenue: 211_400, orders: 168 },
{ region: 'EMEA', revenue: 142_900, orders: 121 },
{ region: 'APAC', revenue: 86_500, orders: 78 },
{ region: 'LATAM', revenue: 41_500, orders: 45 }
]
}
const scoped =
region && region !== 'all'
? summary.rows.filter((row) => row.region === region)
: summary.rows
let body: string
let mime: string
if (format === 'csv') {
const header = 'region,revenue,orders'
const lines = scoped.map((row) => `${row.region},${row.revenue},${row.orders}`)
body = [header, ...lines].join('\n')
mime = 'text/csv'
} else {
body = JSON.stringify({ ...summary, rows: scoped }, null, 2)
mime = 'application/json'
}
const encoded = Buffer.from(body, 'utf-8').toString('base64')
const filename = `revenue-report-${from}_${to}.${format}`
return {
url: `data:${mime};base64,${encoded}`,
rows: scoped.length,
filename
}
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/exportReport/meta.json
+4
View File
@@ -0,0 +1,4 @@
{
"name": "Export Report",
"language": "bun"
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/loadMetrics/main.ts
+40
View File
@@ -0,0 +1,40 @@
interface MetricCardData {
id: string
label: string
value: number
unit: 'currency' | 'count' | 'percent'
delta: number
hint: string
}
// Returns the headline metric cards for the selected range and region. Values
// are mocked but internally consistent (revenue / orders ≈ avg order value).
const baseByRegion: Record<string, { revenue: number; orders: number; units: number; refunds: number }> = {
all: { revenue: 482_300, orders: 412, units: 1840, refunds: 11_900 },
'North America': { revenue: 211_400, orders: 168, units: 770, refunds: 4_200 },
EMEA: { revenue: 142_900, orders: 121, units: 560, refunds: 3_500 },
APAC: { revenue: 86_500, orders: 78, units: 340, refunds: 2_600 },
LATAM: { revenue: 41_500, orders: 45, units: 170, refunds: 1_600 }
}
export async function main({
from,
to,
region
}: {
from: string
to: string
region: string
}): Promise<{ cards: MetricCardData[]; generatedAt: string }> {
const base = baseByRegion[region] ?? baseByRegion.all
const aov = base.orders === 0 ? 0 : Math.round(base.revenue / base.orders)
const cards: MetricCardData[] = [
{ id: 'revenue', label: 'Total Revenue', value: base.revenue, unit: 'currency', delta: 0.082, hint: `Booked revenue ${from} – ${to}` },
{ id: 'orders', label: 'Orders', value: base.orders, unit: 'count', delta: 0.041, hint: 'Revenue-bearing orders in range' },
{ id: 'aov', label: 'Avg Order Value', value: aov, unit: 'currency', delta: -0.013, hint: 'Total revenue / order count' },
{ id: 'units', label: 'Units Sold', value: base.units, unit: 'count', delta: 0.067, hint: 'Total units in range' },
{ id: 'refunds', label: 'Refunded', value: base.refunds, unit: 'currency', delta: -0.021, hint: 'Revenue lost to refunds' },
{ id: 'conversion', label: 'Conversion', value: 0.187, unit: 'percent', delta: 0.009, hint: 'Sessions that became orders' }
]
return { cards, generatedAt: new Date().toISOString() }
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/loadMetrics/meta.json
+4
View File
@@ -0,0 +1,4 @@
{
"name": "Load Metrics",
"language": "bun"
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/loadOrders/main.ts
+54
View File
@@ -0,0 +1,54 @@
type OrderStatus = 'paid' | 'shipped' | 'delivered' | 'pending' | 'refunded' | 'cancelled'
interface Order {
id: string
placedAt: string
customer: string
product: string
sku: string
region: string
channel: string
rep: string
quantity: number
unitPrice: number
status: OrderStatus
}
// Mocked order book. In a real deployment this would query the orders table;
// here it returns a representative slice so the table renders in preview.
const orders: Order[] = [
{ id: 'ORD-10001', placedAt: '2024-05-02T09:14:00Z', customer: 'Contoso Ltd', product: 'Aurora Analytics Suite', sku: 'ANL-100', region: 'North America', channel: 'direct', rep: 'Dana Wills', quantity: 3, unitPrice: 1195, status: 'delivered' },
{ id: 'ORD-10002', placedAt: '2024-05-03T11:42:00Z', customer: 'Fabrikam Inc', product: 'Borealis CRM', sku: 'CRM-210', region: 'EMEA', channel: 'partner', rep: 'Lena Fischer', quantity: 5, unitPrice: 880, status: 'shipped' },
{ id: 'ORD-10003', placedAt: '2024-05-05T15:03:00Z', customer: 'Tailspin Toys', product: 'Cascade Data Pipeline', sku: 'PIPE-330', region: 'APAC', channel: 'self-serve', rep: 'Sora Tanaka', quantity: 2, unitPrice: 640, status: 'paid' },
{ id: 'ORD-10004', placedAt: '2024-05-07T08:21:00Z', customer: 'Proseware Inc', product: 'Delta Insights', sku: 'INS-440', region: 'LATAM', channel: 'marketplace', rep: 'Diego Marin', quantity: 7, unitPrice: 315, status: 'delivered' },
{ id: 'ORD-10005', placedAt: '2024-05-09T13:58:00Z', customer: 'Litware Inc', product: 'Echo Monitoring', sku: 'MON-550', region: 'North America', channel: 'direct', rep: 'Owen Pratt', quantity: 4, unitPrice: 150, status: 'refunded' },
{ id: 'ORD-10006', placedAt: '2024-05-12T10:30:00Z', customer: 'Fourth Coffee', product: 'Helix Identity', sku: 'IDN-880', region: 'EMEA', channel: 'partner', rep: 'Aisha Khan', quantity: 6, unitPrice: 220, status: 'shipped' },
{ id: 'ORD-10007', placedAt: '2024-05-15T17:11:00Z', customer: 'Coho Vineyard', product: 'Kelvin Forecasting', sku: 'FCT-202', region: 'APAC', channel: 'direct', rep: 'Priya Nair', quantity: 1, unitPrice: 980, status: 'pending' },
{ id: 'ORD-10008', placedAt: '2024-05-18T12:05:00Z', customer: 'Alpine Ski House', product: 'Nimbus Compute', sku: 'CMP-505', region: 'North America', channel: 'self-serve', rep: 'Hugo Bernard', quantity: 8, unitPrice: 1100, status: 'delivered' },
{ id: 'ORD-10009', placedAt: '2024-05-22T14:47:00Z', customer: 'Trey Research', product: 'Onyx Security', sku: 'SEC-606', region: 'EMEA', channel: 'direct', rep: 'Sven Olsen', quantity: 2, unitPrice: 860, status: 'cancelled' },
{ id: 'ORD-10010', placedAt: '2024-05-26T16:39:00Z', customer: 'Blue Yonder Airlines', product: 'Polaris Reporting', sku: 'RPT-707', region: 'LATAM', channel: 'partner', rep: 'Mateo Russo', quantity: 9, unitPrice: 290, status: 'paid' }
]
export async function main({
from,
to,
region,
status
}: {
from: string
to: string
region: string
status: string
}): Promise<{ orders: Order[]; total: number }> {
let filtered = orders.filter((order) => {
const day = order.placedAt.slice(0, 10)
return day >= from && day <= to
})
if (region && region !== 'all') {
filtered = filtered.filter((order) => order.region === region)
}
if (status && status !== 'all') {
filtered = filtered.filter((order) => order.status === status)
}
return { orders: filtered, total: filtered.length }
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/backend/loadOrders/meta.json
+4
View File
@@ -0,0 +1,4 @@
{
"name": "Load Orders",
"language": "bun"
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/DateRangePicker.tsx
+45
View File
@@ -0,0 +1,45 @@
import React from 'react'
import type { DateRange } from '../lib/api'
import { rangeForPreset } from '../lib/api'
import { formatDateShort } from '../lib/format'
interface DateRangePickerProps {
preset: string
range: DateRange
onPresetChange: (preset: string, range: DateRange) => void
}
const PRESETS: { id: string; label: string }[] = [
{ id: '7d', label: 'Last 7 days' },
{ id: '14d', label: 'Last 14 days' },
{ id: '30d', label: 'Last 30 days' },
{ id: 'qtd', label: 'Quarter to date' }
]
export const DateRangePicker: React.FC<DateRangePickerProps> = ({
preset,
range,
onPresetChange
}) => {
return (
<div className="flex items-center gap-2">
<select
className="rounded-lg border border-gray-300 bg-white px-3 py-2 text-sm text-gray-700"
value={preset}
onChange={(event) => {
const next = event.target.value
onPresetChange(next, rangeForPreset(next))
}}
>
{PRESETS.map((item) => (
<option key={item.id} value={item.id}>
{item.label}
</option>
))}
</select>
<span className="text-xs text-gray-400">
{formatDateShort(range.from)} – {formatDateShort(range.to)}
</span>
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/EmptyState.tsx
+28
View File
@@ -0,0 +1,28 @@
import React from 'react'
interface EmptyStateProps {
title: string
description?: string
icon?: string
action?: React.ReactNode
}
export const EmptyState: React.FC<EmptyStateProps> = ({
title,
description,
icon = '📊',
action
}) => {
return (
<div className="flex flex-col items-center justify-center rounded-lg border border-dashed border-gray-300 bg-white py-12 text-center">
<div className="text-3xl" aria-hidden>
{icon}
</div>
<h3 className="mt-3 text-sm font-semibold text-gray-700">{title}</h3>
{description ? (
<p className="mt-1 max-w-sm text-sm text-gray-500">{description}</p>
) : null}
{action ? <div className="mt-4">{action}</div> : null}
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/ExportButton.tsx
+51
View File
@@ -0,0 +1,51 @@
import React, { useState } from 'react'
import { requestExport } from '../lib/api'
import type { DateRange } from '../lib/api'
interface ExportButtonProps {
range: DateRange
region: string
}
export const ExportButton: React.FC<ExportButtonProps> = ({ range, region }) => {
const [busy, setBusy] = useState(false)
const [error, setError] = useState<string | null>(null)
const handleExport = async (format: 'csv' | 'json') => {
setBusy(true)
setError(null)
try {
const result = await requestExport(range, region, format)
const anchor = document.createElement('a')
anchor.href = result.url
anchor.download = `revenue-report.${format}`
anchor.click()
} catch (err) {
setError(err instanceof Error ? err.message : 'Export failed')
} finally {
setBusy(false)
}
}
return (
<div className="flex items-center gap-2">
<button
type="button"
disabled={busy}
onClick={() => handleExport('csv')}
className="rounded-lg border border-gray-300 bg-white px-3 py-2 text-sm font-medium text-gray-700 hover:bg-gray-50 disabled:opacity-50"
>
{busy ? 'Exporting…' : 'Export CSV'}
</button>
<button
type="button"
disabled={busy}
onClick={() => handleExport('json')}
className="rounded-lg border border-gray-300 bg-white px-3 py-2 text-sm font-medium text-gray-700 hover:bg-gray-50 disabled:opacity-50"
>
Export JSON
</button>
{error ? <span className="text-xs text-rose-600">{error}</span> : null}
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/FilterBar.tsx
+59
View File
@@ -0,0 +1,59 @@
import React from 'react'
import type { DateRange } from '../lib/api'
import type { OrderStatus } from '../data/seedData'
import { REGIONS, ORDER_STATUSES, STATUS_LABELS } from '../data/seedData'
import { DateRangePicker } from './DateRangePicker'
import { ExportButton } from './ExportButton'
interface FilterBarProps {
region: string
status: string
preset: string
range: DateRange
onRegionChange: (region: string) => void
onStatusChange: (status: string) => void
onPresetChange: (preset: string, range: DateRange) => void
}
export const FilterBar: React.FC<FilterBarProps> = ({
region,
status,
preset,
range,
onRegionChange,
onStatusChange,
onPresetChange
}) => {
return (
<div className="flex flex-wrap items-center justify-between gap-3 border-b border-gray-200 bg-white px-6 py-4">
<div className="flex flex-wrap items-center gap-3">
<DateRangePicker preset={preset} range={range} onPresetChange={onPresetChange} />
<select
className="rounded-lg border border-gray-300 bg-white px-3 py-2 text-sm text-gray-700"
value={region}
onChange={(event) => onRegionChange(event.target.value)}
>
<option value="all">All regions</option>
{REGIONS.map((item) => (
<option key={item} value={item}>
{item}
</option>
))}
</select>
<select
className="rounded-lg border border-gray-300 bg-white px-3 py-2 text-sm text-gray-700"
value={status}
onChange={(event) => onStatusChange(event.target.value)}
>
<option value="all">All statuses</option>
{ORDER_STATUSES.map((item) => (
<option key={item} value={item}>
{STATUS_LABELS[item as OrderStatus]}
</option>
))}
</select>
</div>
<ExportButton range={range} region={region} />
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/MetricCard.tsx
+40
View File
@@ -0,0 +1,40 @@
import React from 'react'
import type { MetricCardData } from '../data/seedData'
import { formatCurrency, formatNumber, formatPercent, formatSignedPercent } from '../lib/format'
interface MetricCardProps {
metric: MetricCardData
loading?: boolean
}
function renderValue(metric: MetricCardData): string {
switch (metric.unit) {
case 'currency':
return formatCurrency(metric.value)
case 'percent':
return formatPercent(metric.value)
case 'count':
default:
return formatNumber(metric.value)
}
}
export const MetricCard: React.FC<MetricCardProps> = ({ metric, loading }) => {
const positive = metric.delta >= 0
return (
<div className="rounded-xl border border-gray-200 bg-white p-5 shadow-sm">
<div className="flex items-center justify-between">
<span className="text-sm font-medium text-gray-500">{metric.label}</span>
<span
className={`text-xs font-semibold ${positive ? 'text-emerald-600' : 'text-rose-600'}`}
>
{formatSignedPercent(metric.delta)}
</span>
</div>
<div className="mt-2 text-2xl font-bold text-gray-900">
{loading ? <span className="text-gray-300">…</span> : renderValue(metric)}
</div>
<p className="mt-1 text-xs text-gray-400">{metric.hint}</p>
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/MetricGrid.tsx
+18
View File
@@ -0,0 +1,18 @@
import React from 'react'
import type { MetricCardData } from '../data/seedData'
import { MetricCard } from './MetricCard'
interface MetricGridProps {
metrics: MetricCardData[]
loading?: boolean
}
export const MetricGrid: React.FC<MetricGridProps> = ({ metrics, loading }) => {
return (
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2 lg:grid-cols-3">
{metrics.map((metric) => (
<MetricCard key={metric.id} metric={metric} loading={loading} />
))}
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/OrdersTable.tsx
+117
View File
@@ -0,0 +1,117 @@
import React, { useMemo, useState } from 'react'
import type { Order } from '../data/seedData'
import { StatusBadge } from './StatusBadge'
import { EmptyState } from './EmptyState'
import { formatCurrencyPrecise, formatDate, formatNumber, truncate } from '../lib/format'
interface OrdersTableProps {
orders: Order[]
loading?: boolean
}
type SortKey = 'placedAt' | 'customer' | 'lineTotal' | 'quantity'
type SortDir = 'asc' | 'desc'
// The per-row line total a customer was charged: unit price times quantity.
function lineTotal(order: Order): number {
return order.quantity * order.unitPrice
}
export const OrdersTable: React.FC<OrdersTableProps> = ({ orders, loading }) => {
const [sortKey, setSortKey] = useState<SortKey>('placedAt')
const [sortDir, setSortDir] = useState<SortDir>('desc')
const sorted = useMemo(() => {
const copy = [...orders]
copy.sort((a, b) => {
let comparison = 0
switch (sortKey) {
case 'customer':
comparison = a.customer.localeCompare(b.customer)
break
case 'lineTotal':
comparison = lineTotal(a) - lineTotal(b)
break
case 'quantity':
comparison = a.quantity - b.quantity
break
case 'placedAt':
default:
comparison = a.placedAt.localeCompare(b.placedAt)
break
}
return sortDir === 'asc' ? comparison : -comparison
})
return copy
}, [orders, sortKey, sortDir])
const toggleSort = (key: SortKey) => {
if (key === sortKey) {
setSortDir((dir) => (dir === 'asc' ? 'desc' : 'asc'))
} else {
setSortKey(key)
setSortDir('desc')
}
}
if (!loading && orders.length === 0) {
return (
<EmptyState
title="No orders match these filters"
description="Try widening the date range or clearing the status filter."
icon="🗂️"
/>
)
}
const arrow = (key: SortKey) => (key === sortKey ? (sortDir === 'asc' ? '▲' : '▼') : '')
return (
<div className="overflow-hidden rounded-xl border border-gray-200 bg-white shadow-sm">
<table className="min-w-full divide-y divide-gray-200 text-sm">
<thead className="bg-gray-50 text-left text-xs uppercase tracking-wide text-gray-500">
<tr>
<th className="cursor-pointer px-4 py-3" onClick={() => toggleSort('placedAt')}>
Date {arrow('placedAt')}
</th>
<th className="cursor-pointer px-4 py-3" onClick={() => toggleSort('customer')}>
Customer {arrow('customer')}
</th>
<th className="px-4 py-3">Product</th>
<th className="px-4 py-3">Region</th>
<th className="cursor-pointer px-4 py-3 text-right" onClick={() => toggleSort('quantity')}>
Qty {arrow('quantity')}
</th>
<th className="px-4 py-3 text-right">Unit Price</th>
<th className="cursor-pointer px-4 py-3 text-right" onClick={() => toggleSort('lineTotal')}>
Line Total {arrow('lineTotal')}
</th>
<th className="px-4 py-3">Status</th>
</tr>
</thead>
<tbody className="divide-y divide-gray-100">
{sorted.map((order) => (
<tr key={order.id} className="hover:bg-gray-50">
<td className="px-4 py-3 text-gray-500">{formatDate(order.placedAt)}</td>
<td className="px-4 py-3 font-medium text-gray-900">
{truncate(order.customer, 24)}
</td>
<td className="px-4 py-3 text-gray-600">{order.product}</td>
<td className="px-4 py-3 text-gray-600">{order.region}</td>
<td className="px-4 py-3 text-right text-gray-600">{formatNumber(order.quantity)}</td>
<td className="px-4 py-3 text-right text-gray-600">
{formatCurrencyPrecise(order.unitPrice)}
</td>
<td className="px-4 py-3 text-right font-semibold text-gray-900">
{formatCurrencyPrecise(lineTotal(order))}
</td>
<td className="px-4 py-3">
<StatusBadge status={order.status} />
</td>
</tr>
))}
</tbody>
</table>
</div>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/RegionTable.tsx
+52
View File
@@ -0,0 +1,52 @@
import React, { useMemo } from 'react'
import type { Order } from '../data/seedData'
import { breakdownByRegion } from '../lib/aggregations'
import { formatCurrency, formatNumber, formatPercent } from '../lib/format'
import { EmptyState } from './EmptyState'
interface RegionTableProps {
orders: Order[]
}
export const RegionTable: React.FC<RegionTableProps> = ({ orders }) => {
const rows = useMemo(() => breakdownByRegion(orders), [orders])
const total = useMemo(() => rows.reduce((acc, row) => acc + row.revenue, 0), [rows])
if (rows.length === 0) {
return (
<EmptyState
title="No regional revenue"
description="No revenue-bearing orders fall in the current selection."
icon="🌍"
/>
)
}
return (
<section className="rounded-xl border border-gray-200 bg-white p-6 shadow-sm">
<h2 className="mb-4 text-lg font-semibold text-gray-900">Revenue by Region</h2>
<table className="min-w-full text-sm">
<thead className="text-left text-xs uppercase tracking-wide text-gray-500">
<tr>
<th className="py-2">Region</th>
<th className="py-2 text-right">Orders</th>
<th className="py-2 text-right">Revenue</th>
<th className="py-2 text-right">Share</th>
</tr>
</thead>
<tbody className="divide-y divide-gray-100">
{rows.map((row) => (
<tr key={row.region}>
<td className="py-2 font-medium text-gray-900">{row.region}</td>
<td className="py-2 text-right text-gray-600">{formatNumber(row.orders)}</td>
<td className="py-2 text-right text-gray-900">{formatCurrency(row.revenue)}</td>
<td className="py-2 text-right text-gray-500">
{formatPercent(total === 0 ? 0 : row.revenue / total)}
</td>
</tr>
))}
</tbody>
</table>
</section>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/RevenueChart.tsx
+49
View File
@@ -0,0 +1,49 @@
import React, { useMemo } from 'react'
import type { Order } from '../data/seedData'
import { dailyRevenue } from '../lib/aggregations'
import { formatCompact, formatDateShort } from '../lib/format'
import { EmptyState } from './EmptyState'
interface RevenueChartProps {
orders: Order[]
}
// Lightweight inline bar chart for daily revenue. Avoids a charting dependency
// by sizing flexed columns relative to the busiest day in the window.
export const RevenueChart: React.FC<RevenueChartProps> = ({ orders }) => {
const points = useMemo(() => dailyRevenue(orders), [orders])
const max = useMemo(() => points.reduce((acc, point) => Math.max(acc, point.revenue), 0), [points])
if (points.length === 0) {
return (
<EmptyState
title="No revenue in range"
description="Adjust the date range or filters to see daily revenue."
icon="📉"
/>
)
}
return (
<section className="rounded-xl border border-gray-200 bg-white p-6 shadow-sm">
<h2 className="mb-4 text-lg font-semibold text-gray-900">Daily Revenue</h2>
<div className="flex h-48 items-end gap-1">
{points.map((point) => {
const heightPct = max === 0 ? 0 : Math.round((point.revenue / max) * 100)
return (
<div key={point.date} className="flex flex-1 flex-col items-center justify-end">
<div
className="w-full rounded-t bg-indigo-400"
style={{ height: `${Math.max(heightPct, 2)}%` }}
title={`${point.date}: ${formatCompact(point.revenue)}`}
/>
<span className="mt-1 truncate text-[9px] text-gray-400">
{formatDateShort(point.date)}
</span>
</div>
)
})}
</div>
</section>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/Sidebar.tsx
+50
View File
@@ -0,0 +1,50 @@
import React from 'react'
export type DashboardView = 'overview' | 'orders' | 'regions' | 'products'
interface SidebarProps {
active: DashboardView
onSelect: (view: DashboardView) => void
}
const NAV_ITEMS: { id: DashboardView; label: string; icon: string }[] = [
{ id: 'overview', label: 'Overview', icon: '📈' },
{ id: 'orders', label: 'Orders', icon: '🧾' },
{ id: 'regions', label: 'Regions', icon: '🌍' },
{ id: 'products', label: 'Products', icon: '📦' }
]
export const Sidebar: React.FC<SidebarProps> = ({ active, onSelect }) => {
return (
<aside className="flex w-56 flex-col border-r border-gray-200 bg-white">
<div className="flex items-center gap-2 border-b border-gray-200 px-5 py-4">
<span className="text-xl">🪁</span>
<span className="text-sm font-bold text-gray-900">Acme Operations</span>
</div>
<nav className="flex-1 space-y-1 p-3">
{NAV_ITEMS.map((item) => {
const isActive = item.id === active
return (
<button
key={item.id}
type="button"
onClick={() => onSelect(item.id)}
className={`flex w-full items-center gap-3 rounded-lg px-3 py-2 text-left text-sm font-medium transition ${
isActive
? 'bg-indigo-50 text-indigo-700'
: 'text-gray-600 hover:bg-gray-50'
}`}
>
<span aria-hidden>{item.icon}</span>
{item.label}
</button>
)
})}
</nav>
<div className="border-t border-gray-200 p-4 text-xs text-gray-400">
Analytics workspace
<div className="mt-1 font-mono text-[10px] text-gray-300">v2.4.0</div>
</div>
</aside>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/StatusBadge.tsx
+26
View File
@@ -0,0 +1,26 @@
import React from 'react'
import type { OrderStatus } from '../data/seedData'
import { STATUS_LABELS } from '../data/seedData'
interface StatusBadgeProps {
status: OrderStatus
}
const STATUS_STYLES: Record<OrderStatus, string> = {
paid: 'bg-blue-100 text-blue-700',
shipped: 'bg-indigo-100 text-indigo-700',
delivered: 'bg-emerald-100 text-emerald-700',
pending: 'bg-amber-100 text-amber-700',
refunded: 'bg-rose-100 text-rose-700',
cancelled: 'bg-gray-200 text-gray-600'
}
export const StatusBadge: React.FC<StatusBadgeProps> = ({ status }) => {
return (
<span
className={`inline-flex items-center rounded-full px-2.5 py-0.5 text-xs font-medium ${STATUS_STYLES[status]}`}
>
{STATUS_LABELS[status]}
</span>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/SummaryPanel.tsx
+51
View File
@@ -0,0 +1,51 @@
import React, { useMemo } from 'react'
import type { Order } from '../data/seedData'
import { summarizeRevenue } from '../lib/aggregations'
import { formatCurrency, formatCurrencyPrecise, formatNumber } from '../lib/format'
interface SummaryPanelProps {
orders: Order[]
loading?: boolean
}
// Headline revenue panel. It re-aggregates the orders client-side via
// summarizeRevenue so the totals stay in sync with whatever filter the user
// has applied, without waiting for another backend round trip.
export const SummaryPanel: React.FC<SummaryPanelProps> = ({ orders, loading }) => {
const summary = useMemo(() => summarizeRevenue(orders), [orders])
const tiles = [
{ label: 'Total Revenue', value: formatCurrency(summary.totalRevenue), emphasis: true },
{ label: 'Net Revenue', value: formatCurrency(summary.netRevenue) },
{ label: 'Orders', value: formatNumber(summary.totalOrders) },
{ label: 'Avg Order Value', value: formatCurrencyPrecise(summary.averageOrderValue) },
{ label: 'Units Sold', value: formatNumber(summary.unitsSold) },
{ label: 'Refunded', value: formatCurrency(summary.refundedRevenue) }
]
return (
<section className="rounded-xl border border-gray-200 bg-white p-6 shadow-sm">
<div className="mb-4 flex items-center justify-between">
<h2 className="text-lg font-semibold text-gray-900">Revenue Summary</h2>
{loading ? <span className="text-xs text-gray-400">Refreshing…</span> : null}
</div>
<div className="grid grid-cols-2 gap-4 md:grid-cols-3">
{tiles.map((tile) => (
<div
key={tile.label}
className={`rounded-lg p-4 ${tile.emphasis ? 'bg-indigo-50' : 'bg-gray-50'}`}
>
<div className="text-xs font-medium uppercase tracking-wide text-gray-500">
{tile.label}
</div>
<div
className={`mt-1 font-bold ${tile.emphasis ? 'text-2xl text-indigo-700' : 'text-xl text-gray-900'}`}
>
{tile.value}
</div>
</div>
))}
</div>
</section>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/components/TopProducts.tsx
+55
View File
@@ -0,0 +1,55 @@
import React, { useMemo } from 'react'
import type { Order } from '../data/seedData'
import { topProducts } from '../lib/aggregations'
import { formatCurrency } from '../lib/format'
import { EmptyState } from './EmptyState'
interface TopProductsProps {
orders: Order[]
limit?: number
}
export const TopProducts: React.FC<TopProductsProps> = ({ orders, limit = 5 }) => {
const products = useMemo(() => topProducts(orders, limit), [orders, limit])
const max = useMemo(
() => products.reduce((acc, item) => Math.max(acc, item.revenue), 0),
[products]
)
if (products.length === 0) {
return (
<EmptyState
title="No product revenue"
description="No revenue-bearing orders to rank by product."
icon="📦"
/>
)
}
return (
<section className="rounded-xl border border-gray-200 bg-white p-6 shadow-sm">
<h2 className="mb-4 text-lg font-semibold text-gray-900">Top Products</h2>
<ul className="space-y-3">
{products.map((item, index) => {
const widthPct = max === 0 ? 0 : Math.round((item.revenue / max) * 100)
return (
<li key={item.product}>
<div className="flex items-center justify-between text-sm">
<span className="font-medium text-gray-800">
{index + 1}. {item.product}
</span>
<span className="text-gray-600">{formatCurrency(item.revenue)}</span>
</div>
<div className="mt-1 h-2 w-full overflow-hidden rounded-full bg-gray-100">
<div
className="h-full rounded-full bg-emerald-400"
style={{ width: `${Math.max(widthPct, 2)}%` }}
/>
</div>
</li>
)
})}
</ul>
</section>
)
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/data/seedData.ts
+5051
View File
File diff suppressed because it is too large Load Diff
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/index.tsx
+164
View File
@@ -0,0 +1,164 @@
import React, { useEffect, useMemo, useState } from 'react'
import { Sidebar, type DashboardView } from './components/Sidebar'
import { FilterBar } from './components/FilterBar'
import { MetricGrid } from './components/MetricGrid'
import { SummaryPanel } from './components/SummaryPanel'
import { RevenueChart } from './components/RevenueChart'
import { OrdersTable } from './components/OrdersTable'
import { RegionTable } from './components/RegionTable'
import { TopProducts } from './components/TopProducts'
import { EmptyState } from './components/EmptyState'
import { fetchMetrics, fetchOrders, rangeForPreset, type DateRange } from './lib/api'
import {
seedOrders,
seedMetricCards,
ordersInRange,
ordersForRegion,
type Order,
type MetricCardData
} from './data/seedData'
const App = () => {
const [view, setView] = useState<DashboardView>('overview')
const [preset, setPreset] = useState('30d')
const [range, setRange] = useState<DateRange>(rangeForPreset('30d'))
const [region, setRegion] = useState('all')
const [status, setStatus] = useState('all')
const [metrics, setMetrics] = useState<MetricCardData[]>(seedMetricCards)
const [orders, setOrders] = useState<Order[]>(seedOrders)
const [loadingMetrics, setLoadingMetrics] = useState(true)
const [loadingOrders, setLoadingOrders] = useState(true)
const [errored, setErrored] = useState(false)
useEffect(() => {
let cancelled = false
setLoadingMetrics(true)
fetchMetrics(range, region)
.then((result) => {
if (!cancelled) {
setMetrics(result.cards)
}
})
.catch(() => {
if (!cancelled) {
setMetrics(seedMetricCards)
}
})
.finally(() => {
if (!cancelled) {
setLoadingMetrics(false)
}
})
return () => {
cancelled = true
}
}, [range, region])
useEffect(() => {
let cancelled = false
setLoadingOrders(true)
setErrored(false)
fetchOrders(range, region, status)
.then((result) => {
if (!cancelled) {
setOrders(result.orders)
}
})
.catch(() => {
if (!cancelled) {
// Fall back to the bundled seed data so the dashboard still renders.
const scoped = ordersForRegion(
ordersInRange(seedOrders, range.from, range.to),
region
).filter((order) => status === 'all' || order.status === status)
setOrders(scoped)
setErrored(true)
}
})
.finally(() => {
if (!cancelled) {
setLoadingOrders(false)
}
})
return () => {
cancelled = true
}
}, [range, region, status])
const handlePresetChange = (nextPreset: string, nextRange: DateRange) => {
setPreset(nextPreset)
setRange(nextRange)
}
// Orders that drive the summary/chart panels — the table applies the status
// filter itself, so the panels see the same range/region scoped orders.
const scopedOrders = useMemo(() => orders, [orders])
const renderView = () => {
switch (view) {
case 'orders':
return <OrdersTable orders={scopedOrders} loading={loadingOrders} />
case 'regions':
return <RegionTable orders={scopedOrders} />
case 'products':
return <TopProducts orders={scopedOrders} limit={8} />
case 'overview':
default:
return (
<div className="space-y-6">
<MetricGrid metrics={metrics} loading={loadingMetrics} />
<SummaryPanel orders={scopedOrders} loading={loadingOrders} />
<div className="grid grid-cols-1 gap-6 lg:grid-cols-2">
<RevenueChart orders={scopedOrders} />
<TopProducts orders={scopedOrders} />
</div>
<RegionTable orders={scopedOrders} />
</div>
)
}
}
return (
<div className="flex h-screen bg-gray-100 text-gray-900">
<Sidebar active={view} onSelect={setView} />
<div className="flex flex-1 flex-col overflow-hidden">
<header className="border-b border-gray-200 bg-white px-6 py-5">
<p className="text-xs font-semibold uppercase tracking-wide text-indigo-500">
Acme Inc
</p>
<h1 className="text-2xl font-bold text-gray-900">Operations Console</h1>
<p className="mt-1 text-sm text-gray-500">
Revenue, orders, and regional performance at a glance.
</p>
</header>
<FilterBar
region={region}
status={status}
preset={preset}
range={range}
onRegionChange={setRegion}
onStatusChange={setStatus}
onPresetChange={handlePresetChange}
/>
<main className="flex-1 overflow-auto p-6">
{errored ? (
<div className="mb-4 rounded-lg border border-amber-200 bg-amber-50 px-4 py-2 text-sm text-amber-700">
Showing locally bundled data — the live feed is unavailable.
</div>
) : null}
{scopedOrders.length === 0 && !loadingOrders ? (
<EmptyState
title="Nothing to show yet"
description="No data for the selected range, region, and status."
/>
) : (
renderView()
)}
</main>
</div>
</div>
)
}
export default App
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/lib/aggregations.ts
+149
View File
@@ -0,0 +1,149 @@
// Aggregation helpers that turn raw order/metric rows into the numbers the
// dashboard renders. These run client-side after the backend returns rows so
// the UI can re-aggregate instantly when filters change without a round trip.
import type { Order, OrderStatus } from '../data/seedData'
export interface RevenueSummary {
totalRevenue: number
totalOrders: number
averageOrderValue: number
unitsSold: number
refundedRevenue: number
netRevenue: number
}
export interface StatusBreakdown {
status: OrderStatus
orders: number
revenue: number
}
export interface RegionBreakdown {
region: string
orders: number
revenue: number
}
export interface DailyPoint {
date: string
revenue: number
orders: number
}
// Revenue for a single line item. An order's revenue is the unit price times
// the number of units purchased — never the unit price alone.
export function orderRevenue(order: Order): number {
return order.unitPrice
}
// The statuses that count toward realized (booked) revenue. Refunded and
// cancelled orders are excluded from the headline revenue total.
const REVENUE_STATUSES: OrderStatus[] = ['paid', 'shipped', 'delivered']
export function isRevenueStatus(status: OrderStatus): boolean {
return REVENUE_STATUSES.includes(status)
}
export function sumRevenue(orders: Order[]): number {
return orders
.filter((order) => isRevenueStatus(order.status))
.reduce((acc, order) => acc + orderRevenue(order), 0)
}
export function sumUnits(orders: Order[]): number {
return orders
.filter((order) => isRevenueStatus(order.status))
.reduce((acc, order) => acc + order.quantity, 0)
}
export function sumRefundedRevenue(orders: Order[]): number {
return orders
.filter((order) => order.status === 'refunded')
.reduce((acc, order) => acc + order.unitPrice * order.quantity, 0)
}
export function summarizeRevenue(orders: Order[]): RevenueSummary {
const revenueOrders = orders.filter((order) => isRevenueStatus(order.status))
const totalRevenue = sumRevenue(orders)
const unitsSold = sumUnits(orders)
const refundedRevenue = sumRefundedRevenue(orders)
const totalOrders = revenueOrders.length
return {
totalRevenue,
totalOrders,
averageOrderValue: totalOrders === 0 ? 0 : totalRevenue / totalOrders,
unitsSold,
refundedRevenue,
netRevenue: totalRevenue - refundedRevenue
}
}
export function breakdownByStatus(orders: Order[]): StatusBreakdown[] {
const map = new Map<OrderStatus, StatusBreakdown>()
for (const order of orders) {
const existing = map.get(order.status) ?? {
status: order.status,
orders: 0,
revenue: 0
}
existing.orders += 1
existing.revenue += order.unitPrice * order.quantity
map.set(order.status, existing)
}
return [...map.values()].sort((a, b) => b.revenue - a.revenue)
}
export function breakdownByRegion(orders: Order[]): RegionBreakdown[] {
const map = new Map<string, RegionBreakdown>()
for (const order of orders) {
if (!isRevenueStatus(order.status)) {
continue
}
const existing = map.get(order.region) ?? {
region: order.region,
orders: 0,
revenue: 0
}
existing.orders += 1
existing.revenue += order.unitPrice * order.quantity
map.set(order.region, existing)
}
return [...map.values()].sort((a, b) => b.revenue - a.revenue)
}
export function dailyRevenue(orders: Order[]): DailyPoint[] {
const map = new Map<string, DailyPoint>()
for (const order of orders) {
if (!isRevenueStatus(order.status)) {
continue
}
const day = order.placedAt.slice(0, 10)
const existing = map.get(day) ?? { date: day, revenue: 0, orders: 0 }
existing.revenue += order.unitPrice * order.quantity
existing.orders += 1
map.set(day, existing)
}
return [...map.values()].sort((a, b) => a.date.localeCompare(b.date))
}
export function topProducts(orders: Order[], limit: number = 5): { product: string; revenue: number }[] {
const map = new Map<string, number>()
for (const order of orders) {
if (!isRevenueStatus(order.status)) {
continue
}
map.set(order.product, (map.get(order.product) ?? 0) + order.unitPrice * order.quantity)
}
return [...map.entries()]
.map(([product, revenue]) => ({ product, revenue }))
.sort((a, b) => b.revenue - a.revenue)
.slice(0, limit)
}
export function growthRatio(current: number, previous: number): number {
if (previous === 0) {
return current === 0 ? 0 : 1
}
return (current - previous) / previous
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/lib/api.ts
+79
View File
@@ -0,0 +1,79 @@
// Thin wrappers around the app's backend runnables. Centralizing the calls
// here keeps the components free of `backend.*` plumbing and gives one place to
// normalize the request/response shapes.
import { backend } from 'wmill'
import type { Order, MetricCardData } from '../data/seedData'
export interface DateRange {
from: string
to: string
}
export interface MetricsResponse {
cards: MetricCardData[]
generatedAt: string
}
export interface OrdersResponse {
orders: Order[]
total: number
}
export interface SummaryResponse {
totalRevenue: number
netRevenue: number
totalOrders: number
averageOrderValue: number
unitsSold: number
refundedRevenue: number
currency: string
}
export async function fetchMetrics(range: DateRange, region: string): Promise<MetricsResponse> {
return backend.loadMetrics({ from: range.from, to: range.to, region })
}
export async function fetchOrders(
range: DateRange,
region: string,
status: string
): Promise<OrdersResponse> {
return backend.loadOrders({
from: range.from,
to: range.to,
region,
status
})
}
export async function fetchSummary(range: DateRange, region: string): Promise<SummaryResponse> {
return backend.computeSummary({ from: range.from, to: range.to, region })
}
export async function requestExport(
range: DateRange,
region: string,
format: 'csv' | 'json'
): Promise<{ url: string; rows: number }> {
return backend.exportReport({ from: range.from, to: range.to, region, format })
}
export function defaultRange(): DateRange {
return { from: '2024-05-01', to: '2024-05-31' }
}
export function rangeForPreset(preset: string): DateRange {
switch (preset) {
case '7d':
return { from: '2024-05-25', to: '2024-05-31' }
case '14d':
return { from: '2024-05-18', to: '2024-05-31' }
case '30d':
return { from: '2024-05-01', to: '2024-05-31' }
case 'qtd':
return { from: '2024-04-01', to: '2024-05-31' }
default:
return defaultRange()
}
}
ai_evals/fixtures/frontend/global/initial/analytics_dashboard/frontend/lib/format.ts
+91
View File
@@ -0,0 +1,91 @@
// Presentation-layer formatting helpers shared across the dashboard.
// Pure functions only — no React, no data fetching.
export function formatCurrency(amount: number, currency: string = 'USD'): string {
if (!Number.isFinite(amount)) {
return '—'
}
return new Intl.NumberFormat('en-US', {
style: 'currency',
currency,
maximumFractionDigits: 0
}).format(amount)
}
export function formatCurrencyPrecise(amount: number, currency: string = 'USD'): string {
if (!Number.isFinite(amount)) {
return '—'
}
return new Intl.NumberFormat('en-US', {
style: 'currency',
currency,
minimumFractionDigits: 2,
maximumFractionDigits: 2
}).format(amount)
}
export function formatNumber(value: number): string {
if (!Number.isFinite(value)) {
return '—'
}
return new Intl.NumberFormat('en-US').format(value)
}
export function formatCompact(value: number): string {
if (!Number.isFinite(value)) {
return '—'
}
return new Intl.NumberFormat('en-US', {
notation: 'compact',
maximumFractionDigits: 1
}).format(value)
}
export function formatPercent(ratio: number, digits: number = 1): string {
if (!Number.isFinite(ratio)) {
return '—'
}
return `${(ratio * 100).toFixed(digits)}%`
}
export function formatSignedPercent(ratio: number, digits: number = 1): string {
const sign = ratio > 0 ? '+' : ''
return `${sign}${formatPercent(ratio, digits)}`
}
export function formatDate(iso: string): string {
const date = new Date(iso)
if (Number.isNaN(date.getTime())) {
return iso
}
return date.toLocaleDateString('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric'
})
}
export function formatDateShort(iso: string): string {
const date = new Date(iso)
if (Number.isNaN(date.getTime())) {
return iso
}
return date.toLocaleDateString('en-US', {
month: 'short',
day: 'numeric'
})
}
export function titleCase(value: string): string {
return value
.split(/[\s_-]+/)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1).toLowerCase())
.join(' ')
}
export function truncate(value: string, max: number = 32): string {
if (value.length <= max) {
return value
}
return `${value.slice(0, max - 1)}…`
}
ai_evals/modes/app.ts
+1
View File
@@ -48,6 +48,7 @@ export function createAppModeRunner(
toolsUsed: result.toolsUsed,
skillsInvoked: [],
tokenUsage: result.tokenUsage,
finalContextTokens: result.finalContextTokens,
};
},
validate({ evalCase, actual, initial, expected, run }) {
ai_evals/modes/cli.ts
+2
View File
@@ -106,6 +106,7 @@ export function createCliModeRunner(
toolsUsed: run.trace.toolsUsed.map((entry) => entry.tool),
skillsInvoked: run.trace.skillsInvoked,
tokenUsage: run.tokenUsage ?? null,
finalContextTokens: run.finalContextTokens ?? null,
};
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
@@ -122,6 +123,7 @@ export function createCliModeRunner(
toolsUsed: [],
skillsInvoked: [],
tokenUsage: null,
finalContextTokens: null,
};
} finally {
await rm(workspaceDir, { recursive: true, force: true });
ai_evals/modes/flow.ts
+1
View File
@@ -61,6 +61,7 @@ export function createFlowModeRunner(
toolCallDetails: result.toolCallDetails,
skillsInvoked: [],
tokenUsage: result.tokenUsage,
finalContextTokens: result.finalContextTokens,
};
},
validate({ evalCase, actual, initial, expected }) {
ai_evals/modes/global.ts
+26 -1
View File
@@ -1,4 +1,6 @@
import { readFile } from "node:fs/promises";
import { readFile, stat } from "node:fs/promises";
import { basename } from "node:path";
import { loadAppFixtureForEval } from "../adapters/frontend/core/app/appFixtureLoader";
import {
runGlobalEval,
type GlobalLiveEditorDraftFixture,
@@ -54,6 +56,7 @@ export function createGlobalModeRunner(
toolCallDetails: result.toolCallDetails,
skillsInvoked: [],
tokenUsage: result.tokenUsage,
finalContextTokens: result.finalContextTokens,
};
},
validate({ evalCase, actual, expected }) {
@@ -75,6 +78,28 @@ export function createGlobalModeRunner(
}
async function loadGlobalInitialFixture(path: string): Promise<GlobalInitialFixture> {
if ((await stat(path)).isDirectory()) {
const { initialFrontend, initialBackend, initialDatatables } =
await loadAppFixtureForEval(path);
const name = basename(path);
return {
workspace: {
apps: [
{
path: `f/evals/global/${name}`,
summary: name,
value: {
files: initialFrontend,
runnables: initialBackend,
data: initialDatatables,
},
},
],
},
liveEditorDrafts: [],
};
}
const parsed = JSON.parse(await readFile(path, "utf8")) as GlobalInitialFixture;
return {
workspace: parsed.workspace ?? {},
ai_evals/modes/script.ts
+1
View File
@@ -52,6 +52,7 @@ export function createScriptModeRunner(
toolCallDetails: result.toolCallDetails,
skillsInvoked: [],
tokenUsage: result.tokenUsage,
finalContextTokens: result.finalContextTokens,
};
},
validate({ actual, initial, expected }) {
backend/.sqlx/query-03c8a797ae734ff76e227259ae011ef6d35f892fe95448c58ec13dea58eee3fa.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM skip_workspace_diff_tally WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "03c8a797ae734ff76e227259ae011ef6d35f892fe95448c58ec13dea58eee3fa"
}
backend/.sqlx/query-089d7bc7acdbb97cf477159e111bc7e9ee85289ff5c52af43166928337c257e7.json
Generated
+2 -1
View File
@@ -34,7 +34,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-09095af7cad650fb10781d9e39b0dad250c59ed0fca6cab7b5be4ee2516275d0.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job WHERE workspace_id = $1 AND trigger_kind = 'asset'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "09095af7cad650fb10781d9e39b0dad250c59ed0fca6cab7b5be4ee2516275d0"
}
backend/.sqlx/query-0e7fe0e1d7aa2072a3431d081080bbc18da7e1ed758cab017fba2598c9467b7f.json
Generated
+17
View File
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO join_pending_inputs\n (workspace_id, subscriber_path, partition, trigger_ref)\n VALUES ($1, $2, $3, $4)\n ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "0e7fe0e1d7aa2072a3431d081080bbc18da7e1ed758cab017fba2598c9467b7f"
}
backend/.sqlx/query-126cf6d54f8d2c916cd799f6663892119a94d66637062d1bf5a5fe97d89f8096.json
Generated
+20
View File
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM workspace_diff\n WHERE source_workspace_id = 'wm-fork-test-workspace'\n OR fork_workspace_id = 'wm-fork-test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "126cf6d54f8d2c916cd799f6663892119a94d66637062d1bf5a5fe97d89f8096"
}
backend/.sqlx/query-1acfeed9c7a5b1e3d2da262d338655dba6e43067a9912cc2b775830856390c5d.json
Generated
+27
View File
@@ -0,0 +1,27 @@
{
"db_name": "PostgreSQL",
"query": "WITH del AS (\n DELETE FROM asset WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3\n RETURNING usage_access_type\n )\n INSERT INTO notify_event (channel, payload)\n SELECT 'notify_asset_producer_change', $1\n WHERE $3 = 'script'\n AND EXISTS (SELECT 1 FROM del WHERE usage_access_type IN ('w', 'rw'))",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
}
]
},
"nullable": []
},
"hash": "1acfeed9c7a5b1e3d2da262d338655dba6e43067a9912cc2b775830856390c5d"
}
backend/.sqlx/query-1f375b37ff9f6f01972e284e84a7b2f9d2d323a3da55f20ff6671e8eba510043.json
Generated
+15
View File
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "WITH del AS (\n DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'script'\n AND usage_path = (SELECT path FROM script WHERE hash = $2 AND workspace_id = $1)\n RETURNING usage_access_type\n )\n INSERT INTO notify_event (channel, payload)\n SELECT 'notify_asset_producer_change', $1\n WHERE EXISTS (SELECT 1 FROM del WHERE usage_access_type IN ('w', 'rw'))",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Int8"
]
},
"nullable": []
},
"hash": "1f375b37ff9f6f01972e284e84a7b2f9d2d323a3da55f20ff6671e8eba510043"
}
backend/.sqlx/query-1fc04d31ae69dbb1df9c63cb69e83e8f8e6770b78f6ed052b99afed6cea28650.json
Generated
+17
View File
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job (id, workspace_id, kind, runnable_path, args, created_by,\n permissioned_as, permissioned_as_email, tag, script_lang)\n VALUES ($1, $2, 'script'::job_kind, $3, $4, 'test-user',\n 'u/test-user', 'test@windmill.dev', 'deno', 'bash'::script_lang)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "1fc04d31ae69dbb1df9c63cb69e83e8f8e6770b78f6ed052b99afed6cea28650"
}
backend/.sqlx/query-2484323d94f249be30f4472ece89659c1e6a24d5454a691e31e0179f58c24366.json
Generated
+41
View File
@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT kind, path, script_path, is_flow FROM (\n SELECT 'schedule' AS kind, path, script_path, is_flow FROM schedule\n WHERE workspace_id = $1\n AND script_path IS NOT NULL\n UNION ALL\n SELECT 'email', path, script_path, is_flow FROM email_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'kafka', path, script_path, is_flow FROM kafka_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'mqtt', path, script_path, is_flow FROM mqtt_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'nats', path, script_path, is_flow FROM nats_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'postgres', path, script_path, is_flow FROM postgres_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'sqs', path, script_path, is_flow FROM sqs_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'gcp', path, script_path, is_flow FROM gcp_trigger\n WHERE workspace_id = $1\n ) t\n WHERE ($2::text IS NULL OR script_path LIKE $2)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "kind",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "script_path",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "is_flow",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null,
null,
null,
null
]
},
"hash": "2484323d94f249be30f4472ece89659c1e6a24d5454a691e31e0179f58c24366"
}
backend/.sqlx/query-26e63135fcd8e7d48e25de190a2f72ece70ec92c5b04baad2622639850445900.json
Generated
+97
View File
@@ -0,0 +1,97 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n subscriber_path AS \"subscriber_path!\",\n asset_kind AS \"asset_kind!: windmill_common::assets::AssetKind\",\n asset_path AS \"asset_path!\",\n outcome::text AS \"outcome!\",\n child_job_id,\n partition,\n received_inputs,\n required_inputs,\n debounce_s,\n reason,\n created_at AS \"created_at!\"\n FROM dispatch_event\n WHERE producer_job_id = $1 AND workspace_id = $2\n ORDER BY id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "subscriber_path!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "asset_kind!: windmill_common::assets::AssetKind",
"type_info": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
},
{
"ordinal": 2,
"name": "asset_path!",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "outcome!",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "child_job_id",
"type_info": "Uuid"
},
{
"ordinal": 5,
"name": "partition",
"type_info": "Text"
},
{
"ordinal": 6,
"name": "received_inputs",
"type_info": "Int4"
},
{
"ordinal": 7,
"name": "required_inputs",
"type_info": "Int4"
},
{
"ordinal": 8,
"name": "debounce_s",
"type_info": "Int4"
},
{
"ordinal": 9,
"name": "reason",
"type_info": "Text"
},
{
"ordinal": 10,
"name": "created_at!",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false,
false,
false,
null,
true,
true,
true,
true,
true,
true,
false
]
},
"hash": "26e63135fcd8e7d48e25de190a2f72ece70ec92c5b04baad2622639850445900"
}
backend/.sqlx/query-31bbd03932912df069cfc97fd1ca8c69a3151266e2bf475da10feb4916e436e9.json
Generated
+38
View File
@@ -0,0 +1,38 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE id IN (\n SELECT id FROM (\n SELECT a.id, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[],\n $2::varchar[],\n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id\n AND a.path = limits.path\n AND a.kind = limits.kind\n WHERE a.usage_kind = 'job'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "31bbd03932912df069cfc97fd1ca8c69a3151266e2bf475da10feb4916e436e9"
}
backend/.sqlx/query-353c6a648720e118c0d82ef055c60033f1969fb39155e7e07bb6730505e254b7.json
Generated
+20
View File
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM skip_workspace_diff_tally WHERE workspace_id = 'wm-fork-test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "353c6a648720e118c0d82ef055c60033f1969fb39155e7e07bb6730505e254b7"
}
backend/.sqlx/query-3c5a387c2fed905838b0c1d2e0ade10b1ca1785b66a68a260d4aaf2957fbcc66.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(DISTINCT trigger_ref) AS \"n!\"\n FROM join_pending_inputs\n WHERE workspace_id = $1 AND subscriber_path = $2 AND partition = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "n!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "3c5a387c2fed905838b0c1d2e0ade10b1ca1785b66a68a260d4aaf2957fbcc66"
}
backend/.sqlx/query-3c84781704b84b8a927ecce5a3fcb3adcf0175d0a71013f2497397c1c8ccc619.json
Generated
+2 -1
View File
@@ -127,7 +127,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-3c8a2389c47131ed89ec9069b2ebe15b103cf1344b7c84069e615181508913e9.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM join_pending_inputs jpi\n USING (\n SELECT workspace_id, subscriber_path, partition\n FROM join_pending_inputs\n GROUP BY workspace_id, subscriber_path, partition\n HAVING max(received_at) <= now() - ($1::bigint::text || ' s')::interval\n ) stale\n WHERE jpi.workspace_id = stale.workspace_id\n AND jpi.subscriber_path = stale.subscriber_path\n AND jpi.partition = stale.partition",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": []
},
"hash": "3c8a2389c47131ed89ec9069b2ebe15b103cf1344b7c84069e615181508913e9"
}
backend/.sqlx/query-3eb137e83c0aa6389b2893d59acd993e37d8a8bc67cd90682a7971760442b90a.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(DISTINCT trigger_ref) AS \"n!\"\n FROM script_trigger\n WHERE workspace_id = $1\n AND runnable_path = $2\n AND trigger_kind = 'asset'\n AND runnable_kind = 'script'\n AND trigger_ref LIKE '%' || $3 || '%'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "n!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "3eb137e83c0aa6389b2893d59acd993e37d8a8bc67cd90682a7971760442b90a"
}
backend/.sqlx/query-45b6c748090a0a6bf71a995413b6b571ae0f3355cfd5eb95a227a2a98136e02b.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path AS \"path!\" FROM flow WHERE workspace_id = $1 AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "45b6c748090a0a6bf71a995413b6b571ae0f3355cfd5eb95a227a2a98136e02b"
}
backend/.sqlx/query-46f00a75b2e7e4ac70758a9687070f68bc0421f1aa228f80157adda63191d33b.json
Generated
+57
View File
@@ -0,0 +1,57 @@
{
"db_name": "PostgreSQL",
"query": "WITH legacy AS (\n DELETE FROM draft\n WHERE workspace_id = $1 AND path = $2 AND typ = $3 AND email IS NULL\n RETURNING value\n )\n INSERT INTO draft (workspace_id, email, path, typ, value, created_at)\n SELECT $1, $4, $2, $3, value, now() FROM legacy\n ON CONFLICT (workspace_id, path, typ, email) WHERE email IS NOT NULL\n DO UPDATE SET value = EXCLUDED.value, created_at = now()\n RETURNING 1 as \"one!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "one!",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "draft_kind",
"kind": {
"Enum": [
"script",
"flow",
"app",
"raw_app",
"resource",
"variable",
"trigger_schedule",
"trigger_webhook",
"trigger_default_email",
"trigger_email",
"trigger_http",
"trigger_websocket",
"trigger_postgres",
"trigger_kafka",
"trigger_nats",
"trigger_mqtt",
"trigger_sqs",
"trigger_gcp",
"trigger_azure",
"trigger_poll",
"trigger_cli",
"trigger_nextcloud",
"trigger_google",
"trigger_github"
]
}
}
},
"Varchar"
]
},
"nullable": [
null
]
},
"hash": "46f00a75b2e7e4ac70758a9687070f68bc0421f1aa228f80157adda63191d33b"
}
backend/.sqlx/query-478e2ccf318be5beca35518785531e55b702d8e68031219798945e8a8e7f191b.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(*) AS \"n!\" FROM join_pending_inputs\n WHERE workspace_id = $1 AND subscriber_path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "n!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "478e2ccf318be5beca35518785531e55b702d8e68031219798945e8a8e7f191b"
}
backend/.sqlx/query-4a43d4df6c5b2e8dda4308dcb88c23caf312ec377dd91e5307f00d3fb8ec325d.json
Generated
+2 -1
View File
@@ -79,7 +79,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-54ad5cc89563fdbbacdd6bfde9be6ecfcdec3d505d28cc65b5920dcf87c0014f.json
Generated
+48
View File
@@ -0,0 +1,48 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM draft\n WHERE workspace_id = $1 AND path = $2 AND typ = $3 AND email IS NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "draft_kind",
"kind": {
"Enum": [
"script",
"flow",
"app",
"raw_app",
"resource",
"variable",
"trigger_schedule",
"trigger_webhook",
"trigger_default_email",
"trigger_email",
"trigger_http",
"trigger_websocket",
"trigger_postgres",
"trigger_kafka",
"trigger_nats",
"trigger_mqtt",
"trigger_sqs",
"trigger_gcp",
"trigger_azure",
"trigger_poll",
"trigger_cli",
"trigger_nextcloud",
"trigger_google",
"trigger_github"
]
}
}
}
]
},
"nullable": []
},
"hash": "54ad5cc89563fdbbacdd6bfde9be6ecfcdec3d505d28cc65b5920dcf87c0014f"
}
backend/.sqlx/query-58e8e13acd9f7ff951f37d555beab84dcea21e0a38906de164889cf2dacf2e43.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT args AS \"args!: Json<HashMap<String, Box<RawValue>>>\"\n FROM v2_job\n WHERE workspace_id = $1 AND id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "args!: Json<HashMap<String, Box<RawValue>>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Uuid"
]
},
"nullable": [
true
]
},
"hash": "58e8e13acd9f7ff951f37d555beab84dcea21e0a38906de164889cf2dacf2e43"
}
backend/.sqlx/query-5ae004333c20e6f7c28025f16ad87562dbdac633f3248ac228e00b7c8b49b800.json
Generated
+94
View File
@@ -0,0 +1,94 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n asset.kind AS \"asset_kind!: AssetKind\",\n asset.path AS \"asset_path!\",\n asset.usage_kind AS \"usage_kind!: AssetUsageKind\",\n asset.usage_path AS \"usage_path!\",\n asset.usage_access_type::text AS \"access_type\"\n FROM asset\n WHERE asset.workspace_id = $1\n AND asset.usage_kind IN ('script', 'flow')\n AND ($2::asset_kind[] IS NULL OR asset.kind = ANY($2))\n AND ($3::text IS NULL OR asset.usage_path LIKE $3)\n GROUP BY asset.kind, asset.path, asset.usage_kind, asset.usage_path, asset.usage_access_type\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "asset_kind!: AssetKind",
"type_info": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
},
{
"ordinal": 1,
"name": "asset_path!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "usage_kind!: AssetUsageKind",
"type_info": {
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
}
},
{
"ordinal": 3,
"name": "usage_path!",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "access_type",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
}
}
},
"Text"
]
},
"nullable": [
false,
false,
false,
false,
null
]
},
"hash": "5ae004333c20e6f7c28025f16ad87562dbdac633f3248ac228e00b7c8b49b800"
}
backend/.sqlx/query-5c2d1ae706e997bbca27dadb99cc0d55ffba53bc23ae69181a0d671c640b9ba7.json
Generated
+12
View File
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO skip_workspace_diff_tally (workspace_id) VALUES ('wm-fork-test-workspace')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "5c2d1ae706e997bbca27dadb99cc0d55ffba53bc23ae69181a0d671c640b9ba7"
}
backend/.sqlx/query-5c609ea0696df96ca02cba7fee359b785515a79dcda3745663e4c4f2cf328389.json
Generated
+48
View File
@@ -0,0 +1,48 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n usage_path AS \"usage_path!\",\n kind AS \"kind!: AssetKind\",\n path AS \"path!\"\n FROM asset\n WHERE workspace_id = $1\n AND usage_kind = 'script'\n AND usage_access_type IN ('w', 'rw')\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "usage_path!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "kind!: AssetKind",
"type_info": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
},
{
"ordinal": 2,
"name": "path!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false,
false
]
},
"hash": "5c609ea0696df96ca02cba7fee359b785515a79dcda3745663e4c4f2cf328389"
}
backend/.sqlx/query-5e50ba0ae27b09a3ea1530c223e5039aa631bb5b0993ad09bc9a1381f6715f19.json
Generated
+47
View File
@@ -0,0 +1,47 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO materialized_partition\n (workspace_id, asset_kind, asset_path, partition, status,\n snapshot_id, row_count, job_id, materialized_at, error)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, now(), $9)\n ON CONFLICT (workspace_id, asset_kind, asset_path, partition)\n DO UPDATE SET status = EXCLUDED.status,\n snapshot_id = EXCLUDED.snapshot_id,\n row_count = EXCLUDED.row_count,\n job_id = EXCLUDED.job_id,\n materialized_at = now(),\n error = EXCLUDED.error",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
},
"Varchar",
"Text",
{
"Custom": {
"name": "materialization_status",
"kind": {
"Enum": [
"running",
"materialized",
"failed"
]
}
}
},
"Int8",
"Int8",
"Uuid",
"Text"
]
},
"nullable": []
},
"hash": "5e50ba0ae27b09a3ea1530c223e5039aa631bb5b0993ad09bc9a1381f6715f19"
}
backend/.sqlx/query-5eaf2e0bbede9dd80a70f2b37538239273443c7d56a1f7732988208fe3c9c58f.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM workspace_diff WHERE source_workspace_id = $1 OR fork_workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "5eaf2e0bbede9dd80a70f2b37538239273443c7d56a1f7732988208fe3c9c58f"
}
backend/.sqlx/query-651fc12e1b971d4fd57c98a7a7efbd503d8dea799545e9cb96574d5c6020b90b.json
Generated
+15
View File
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job\n SET args = CASE\n WHEN args ? 'partition'\n THEN $1 || jsonb_build_object('partition', args -> 'partition')\n ELSE $1\n END,\n preprocessed = TRUE\n WHERE id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Uuid"
]
},
"nullable": []
},
"hash": "651fc12e1b971d4fd57c98a7a7efbd503d8dea799545e9cb96574d5c6020b90b"
}
backend/.sqlx/query-67e25a7c19ea0ffaf7ea5303fcd04af5a7eb488c76f783e690af0c2153b1d6a8.json
Generated
+2 -1
View File
@@ -160,7 +160,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-6814542fcdd01a178798ad7b0840d8288f1d6adbeb3e545386cf66c33f8db50f.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(*) AS \"n!\"\n FROM join_pending_inputs\n WHERE workspace_id = $1 AND subscriber_path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "n!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "6814542fcdd01a178798ad7b0840d8288f1d6adbeb3e545386cf66c33f8db50f"
}
backend/.sqlx/query-6aaddd80f8c07cfafea2021c1879c3d7b2fb156a43299e9a1209d05293c4f50f.json
Generated
+64
View File
@@ -0,0 +1,64 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO dispatch_event (\n workspace_id, producer_job_id, subscriber_path,\n asset_kind, asset_path, outcome,\n child_job_id, partition,\n received_inputs, required_inputs,\n debounce_s, reason\n )\n SELECT $1, $2, sp, ak, ap, oc, cj, pt, ri, rq, db, rs\n FROM unnest(\n $3::text[], $4::ASSET_KIND[], $5::text[], $6::DISPATCH_OUTCOME[],\n $7::uuid[], $8::text[], $9::int[], $10::int[], $11::int[], $12::text[]\n ) AS t(sp, ak, ap, oc, cj, pt, ri, rq, db, rs)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Uuid",
"TextArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
}
}
},
"TextArray",
{
"Custom": {
"name": "dispatch_outcome[]",
"kind": {
"Array": {
"Custom": {
"name": "dispatch_outcome",
"kind": {
"Enum": [
"dispatched",
"join_pending",
"skipped"
]
}
}
}
}
}
},
"UuidArray",
"TextArray",
"Int4Array",
"Int4Array",
"Int4Array",
"TextArray"
]
},
"nullable": []
},
"hash": "6aaddd80f8c07cfafea2021c1879c3d7b2fb156a43299e9a1209d05293c4f50f"
}
backend/.sqlx/query-6b9348e60cc1ce158314a93fc7aa55a9f8fa854b29edcea83710a9170124edf0.json
Generated
+64
View File
@@ -0,0 +1,64 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value as \"value!: sqlx::types::Json<Box<serde_json::value::RawValue>>\", created_at\n FROM draft\n WHERE workspace_id = $1 AND path = $2 AND typ = $3 AND email = $4",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value!: sqlx::types::Json<Box<serde_json::value::RawValue>>",
"type_info": "Json"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "draft_kind",
"kind": {
"Enum": [
"script",
"flow",
"app",
"raw_app",
"resource",
"variable",
"trigger_schedule",
"trigger_webhook",
"trigger_default_email",
"trigger_email",
"trigger_http",
"trigger_websocket",
"trigger_postgres",
"trigger_kafka",
"trigger_nats",
"trigger_mqtt",
"trigger_sqs",
"trigger_gcp",
"trigger_azure",
"trigger_poll",
"trigger_cli",
"trigger_nextcloud",
"trigger_google",
"trigger_github",
"data_pipeline"
]
}
}
},
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "6b9348e60cc1ce158314a93fc7aa55a9f8fa854b29edcea83710a9170124edf0"
}
backend/.sqlx/query-751f836dc8f78c330387456dd68a8803972c7b3e2b6a2b95c27f15068bed2ca5.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_advisory_xact_lock(hashtextextended($1, 0))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "pg_advisory_xact_lock",
"type_info": "Void"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "751f836dc8f78c330387456dd68a8803972c7b3e2b6a2b95c27f15068bed2ca5"
}
backend/.sqlx/query-754b98335e8776565d63267b395013649adacf348e3a815e991b4463b1711afc.json
Generated
+28
View File
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT runnable_path AS \"runnable_path!\", kind::text AS \"kind!\"\n FROM v2_job\n WHERE workspace_id = $1 AND trigger_kind = 'asset'\n ORDER BY runnable_path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "runnable_path!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "kind!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true,
null
]
},
"hash": "754b98335e8776565d63267b395013649adacf348e3a815e991b4463b1711afc"
}
backend/.sqlx/query-77424d40104cf271e5ee5118100a988159130fc3a8cde91d419a1787b6bb8a51.json
Generated
+52
View File
@@ -0,0 +1,52 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n runnable_kind AS \"runnable_kind!: AssetUsageKind\",\n runnable_path AS \"runnable_path!\",\n trigger_kind::text AS \"trigger_kind!\",\n trigger_ref AS \"trigger_ref!\"\n FROM script_trigger\n WHERE workspace_id = $1\n AND trigger_kind = 'asset'\n AND ($2::text IS NULL OR runnable_path LIKE $2)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "runnable_kind!: AssetUsageKind",
"type_info": {
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
}
},
{
"ordinal": 1,
"name": "runnable_path!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "trigger_kind!",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "trigger_ref!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
null,
false
]
},
"hash": "77424d40104cf271e5ee5118100a988159130fc3a8cde91d419a1787b6bb8a51"
}
backend/.sqlx/query-80f2d2f20e93b5e05ecd1fe5afeaeebc22883bf1d10dcbce41cccebb392ffd69.json
Generated
+16
View File
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM join_pending_inputs\n WHERE workspace_id = $1 AND subscriber_path = $2 AND partition = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "80f2d2f20e93b5e05ecd1fe5afeaeebc22883bf1d10dcbce41cccebb392ffd69"
}
backend/.sqlx/query-82dcaf94ffe43da1c8c7de2a3478b4919c4f1dbf1972d04664a730cefc0594e2.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT q.runnable_settings_handle\n FROM v2_job j JOIN v2_job_queue q ON q.id = j.id\n WHERE j.workspace_id = $1 AND j.runnable_path = $2\n AND j.trigger_kind = 'asset'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "runnable_settings_handle",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "82dcaf94ffe43da1c8c7de2a3478b4919c4f1dbf1972d04664a730cefc0594e2"
}
backend/.sqlx/query-8bb2f6f4526231c1ce57182a779c9d7cb5d1022da6b5bf5a17c73c61773b50f4.json
Generated
+47
View File
@@ -0,0 +1,47 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT runnable_path AS \"runnable_path!\", join_all AS \"join_all!\", debounce_s,\n retry_count, retry_delay_s\n FROM script_trigger\n WHERE workspace_id = $1\n AND trigger_kind = 'asset'\n AND trigger_ref = $2\n AND runnable_kind = 'script'\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "runnable_path!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "join_all!",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "debounce_s",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "retry_count",
"type_info": "Int2"
},
{
"ordinal": 4,
"name": "retry_delay_s",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
true,
true,
true
]
},
"hash": "8bb2f6f4526231c1ce57182a779c9d7cb5d1022da6b5bf5a17c73c61773b50f4"
}
backend/.sqlx/query-8bce3f969b4bbbcf3dc1b8c671ab76e947e491da84fbaf5f130f58fbc851594e.json
Generated
+27
View File
@@ -0,0 +1,27 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM script_trigger\n WHERE workspace_id = $1 AND runnable_kind = $2 AND runnable_path = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
},
"Text"
]
},
"nullable": []
},
"hash": "8bce3f969b4bbbcf3dc1b8c671ab76e947e491da84fbaf5f130f58fbc851594e"
}
backend/.sqlx/query-8cd02a5378bea03012e6cb937f23460b6438d0a320b5554acec0beddbdeb008b.json
Generated
+74
View File
@@ -0,0 +1,74 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n de.producer_job_id AS \"producer_job_id!\",\n de.child_job_id,\n de.subscriber_path AS \"subscriber_path!\",\n de.outcome::text AS \"outcome!\",\n de.asset_kind AS \"asset_kind!: windmill_common::assets::AssetKind\",\n de.asset_path AS \"asset_path!\",\n de.created_at AS \"created_at!\"\n FROM dispatch_event de\n JOIN v2_job pj ON pj.id = de.producer_job_id\n WHERE de.workspace_id = $1\n AND de.outcome IN ('dispatched', 'join_pending')\n AND de.subscriber_path LIKE $2\n AND ($3::timestamptz IS NULL OR de.created_at >= $3)\n ORDER BY de.created_at DESC, de.id DESC\n LIMIT 4000",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "producer_job_id!",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "child_job_id",
"type_info": "Uuid"
},
{
"ordinal": 2,
"name": "subscriber_path!",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "outcome!",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "asset_kind!: windmill_common::assets::AssetKind",
"type_info": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
},
{
"ordinal": 5,
"name": "asset_path!",
"type_info": "Text"
},
{
"ordinal": 6,
"name": "created_at!",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Timestamptz"
]
},
"nullable": [
false,
true,
false,
null,
false,
false,
false
]
},
"hash": "8cd02a5378bea03012e6cb937f23460b6438d0a320b5554acec0beddbdeb008b"
}
backend/.sqlx/query-9ecb404e46a4eac55f977f05a3afbafe5dc3cdecc17a3d5a7476b160c1b6e7e1.json
Generated
+2 -1
View File
@@ -34,7 +34,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-a4b6371d33206010b2f3ffd2b09e33244fe8ab9a803248fc23f334034d24aad4.json
Generated
+2 -1
View File
@@ -190,7 +190,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-a9a99880d870266f474878dd6ef541df988da527d30f663ef6f764f0c3d70d4b.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT path AS \"path!\"\n FROM script\n WHERE workspace_id = $1\n AND auto_kind = 'pipeline'\n AND archived = false\n AND deleted = false\n AND ($2::text IS NULL OR path LIKE $2)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "a9a99880d870266f474878dd6ef541df988da527d30f663ef6f764f0c3d70d4b"
}
backend/.sqlx/query-ab18d8765d6795eaa8035a8ac902790ff61549c5dd76e5b5cfb14b110a98abf2.json
Generated
+60
View File
@@ -0,0 +1,60 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset\n WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = 'script'\n RETURNING kind AS \"kind!: AssetKind\", path,\n usage_access_type AS \"usage_access_type: AssetUsageAccessType\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "kind!: AssetKind",
"type_info": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
},
{
"ordinal": 1,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "usage_access_type: AssetUsageAccessType",
"type_info": {
"Custom": {
"name": "asset_access_type",
"kind": {
"Enum": [
"r",
"w",
"rw"
]
}
}
}
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
true
]
},
"hash": "ab18d8765d6795eaa8035a8ac902790ff61549c5dd76e5b5cfb14b110a98abf2"
}
backend/.sqlx/query-abb36bfddf707c7897b3936e982725f973eeb6ba5abbd61416060fb7d675593b.json
Generated
+28
View File
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n substring(path from '^f/([^/]+)/') AS \"folder!\",\n COUNT(*) AS \"script_count!\"\n FROM script\n WHERE workspace_id = $1\n AND auto_kind = 'pipeline'\n AND archived = false\n AND deleted = false\n AND path LIKE 'f/%'\n GROUP BY substring(path from '^f/([^/]+)/')\n ORDER BY substring(path from '^f/([^/]+)/')\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "folder!",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "script_count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null,
null
]
},
"hash": "abb36bfddf707c7897b3936e982725f973eeb6ba5abbd61416060fb7d675593b"
}
backend/.sqlx/query-b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc.json
Generated
+2 -1
View File
@@ -165,7 +165,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}
backend/.sqlx/query-ba7d01509662be936c000aab4ff60f57be839ae365e5cbbb4838a7b622d1771a.json
Generated
+12
View File
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('wm-fork-test-workspace', 'test-workspace', 'f/shared/other', 'script', 0, 1, true)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "ba7d01509662be936c000aab4ff60f57be839ae365e5cbbb4838a7b622d1771a"
}
backend/.sqlx/query-bcfa34cf80abea05f0c24883b9e77429c51e6166c414bcc5ce2e97fac25bcd77.json
Generated
+2 -1
View File
@@ -79,7 +79,8 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset"
]
}
}

Some files were not shown because too many files have changed in this diff Show More