Compare commits

..
Author SHA1 Message Date
Ruben FiszelandClaude Opus 4.6 315d4fd2ff fix: correct migration comment to list actual sources (ui, cli, merge)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 12:33:19 +00:00
Ruben FiszelandClaude Opus 4.6 cca4666eb4 feat: set deploy source to 'merge' during fork merge UI flow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:28:44 +00:00
Ruben FiszelandClaude Opus 4.6 d0f80c7af2 fix: make deploy source optional, UI sends explicit header
Default source is now None (direct API call), always blocked by
DisableDirectDeployment. UI, CLI, and merge explicitly identify
themselves via X-Windmill-Deploy-Source header.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:03:18 +00:00
Ruben FiszelandClaude Opus 4.6 523aa34fcb feat: add granular deployment source rules for protection rulesets
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 14:43:03 +00:00
Ruben Fiszel a3f24aeff8 sqlx 2026-04-15 15:14:44 +00:00
centdixandClaude Opus 4.5 f1e84cb088 chore: add backend preview validation to ai evals (#8827)
* feat: add backend preview validation to ai evals

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: refresh shared preview workspace assets

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: harden shared backend preview validation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-15 15:11:25 +00:00
Ruben Fiszel 3aa279cfd7 nit tx commit cj 2026-04-15 12:05:11 +00:00
centdixandClaude Opus 4.5 5c179e5448 fix: preserve gemini thought signatures in ai chat (#8837)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-15 11:49:57 +00:00
Diego ImbertandClaude Opus 4.6 12d0a3de08 fix: parse assets on inline script module creation to avoid false toast (#8835)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 11:01:07 +00:00
Ruben Fiszelandrubenfiszel a98f5b9dfd chore(main): release 1.684.1 (#8834)
* chore(main): release 1.684.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-14 21:48:03 +00:00
Ruben Fiszel 75e204dad1 pin tree-sitter 2026-04-14 21:36:00 +00:00
Ruben FiszelandClaude Opus 4.5 6158ff2ebe fix: stop escalating missing email recipients to critical alert (#8833)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-14 21:21:50 +00:00
Ruben Fiszelandrubenfiszel 8ee14644f4 chore(main): release 1.684.0 (#8831)
* chore(main): release 1.684.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-14 20:58:06 +00:00
Ruben Fiszel f273341759 remove axios deps 2026-04-14 20:50:20 +00:00
hugocasaandClaude Opus 4.6 64ba3a632e feat: cascade trigger script_path on runnable rename + fix trigger permissioned_as (#8823)
* feat: cascade trigger script_path updates on script/flow rename + fix trigger permissioned_as

Backend: When a script or flow path is renamed, automatically update script_path
across all trigger tables (http, email, kafka, websocket, postgres, mqtt, nats,
sqs, gcp, native). Long-running triggers get server_id reset to force restart.
Native triggers additionally get async webhook URL re-registration with external
services (Google, Nextcloud) via token rotation + handler.update().

Frontend: Fix permissioned_as handling across all trigger/schedule editors:
- Allow setting permissioned_as on trigger creation (not just edit) for admins
- Fix hasChanged detection for permissioned_as changes
- Fix FolderEditor group selector showing usernames instead of group names

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: rename script_rename -> runnable_rename for consistency

"Runnable" is the correct term for both scripts and flows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove native trigger re-registration from runnable rename

Keep it simple — only update script_path in the DB for non-native triggers.
Native triggers require external service re-registration (token rotation +
webhook URL update) which adds significant complexity; defer to a future PR.

sqlx files for the updated CTE query need regenerating.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* sqlx

* refactor: call update_triggers_script_path directly, remove windmill-trigger wrapper

No need for the extra module/dep — the common function is called directly
from scripts.rs and flows.rs with inline error mapping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: reject empty principal in folder default permissioned_as validation

`u/` and `g/` (no name after prefix) were passing validation. Use regex
to require at least one character after the prefix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prevent async folder-default load from overwriting user's permissioned_as choice

Split the initialization effect into two: one that resets on trigger switch
(tracks permissionedAs), and one that handles folder default loading (tracks
folderDefault.value). The second effect is guarded by a userHasSelected flag
set in handleSelect, so a late-arriving folder default doesn't wipe the
user's explicit selection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* lock

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:42:13 +00:00
Ruben FiszelandClaude Opus 4.6 aebf758412 fix: allow dedicated flow substeps to inherit parent tag (#8832)
Flow substeps that inherit the parent flow's tag were re-validated
against CUSTOM_TAGS, which rejected dedicated flow tags
(`{workspace_id}:flow/{path}`) since they are never user-registered.
The parent flow's tag was already validated at push time, so skip the
redundant check when the substep simply inherits it.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:38:37 +00:00
hugocasaandClaude Opus 4.6 91064ce857 feat(frontend): improve permissions drawer UX and auto-share resource variables (#8824)
* feat: improve permissions drawer UX and auto-share resource variables

- When sharing a resource, automatically detect linked variables ($var: refs)
  and offer to apply the same permission changes via a toggle (on by default)
- Rename "Share" to "Permissions" across all dropdown menus (resources, variables,
  scripts, flows, apps, schedules, triggers)
- Replace Share icon with Shield icon for consistency
- Show default permissions (folder/user/group) as a separate section in the drawer
- Move item path into drawer title ("Permissions for {path}")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: guard async results against stale drawer state and null-safe extra_perms

- Add path staleness check in loadLinkedVarPaths and loadDefaultPerms
  to prevent late async responses from overwriting state when the drawer
  was reopened for a different item
- Use ?? {} fallback for folder.extra_perms which can be undefined

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:30:36 +00:00
Ruben Fiszel 2c1fe88fed fix ws_specific grant 2026-04-14 20:25:32 +00:00
Diego ImbertandClaude Opus 4.5 7fe639d91e fix: hide serial types in column type dropdown for existing columns (#8828)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-14 20:17:51 +00:00
Diego Imbert 06fe809ecc fix: DB Manager delete/update for timestamp and serial types (#8830)
* Fix time(stamp)(tz) comparisons in pg_executor

* fix serial bug

* UPDATE and DELETE use primary key only
2026-04-14 20:17:39 +00:00
Diego Imbert 5069a3b2e3 Better S3 error context (#8829) 2026-04-14 20:17:28 +00:00
Diego ImbertandClaude Opus 4.6 e1dbce02c2 fix: compute wall-clock duration for flow job groups in CLI (#8826)
The total duration of a for-loop/branchall group was computed as the
naive sum of all iteration durations. This is wrong for parallel
execution and doesn't account for orchestration overhead. Instead,
compute actual wall-clock time as max(completed_at) - min(started_at).

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:16:50 +00:00
Ruben Fiszelandrubenfiszel 6bb80ff28b chore(main): release 1.683.2 (#8820)
* chore(main): release 1.683.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-14 00:23:20 +00:00
5b3913052e refactor: convert read-hot globals to AtomicBool/I64 and ArcSwap (#8815)
* refactor: extract load helpers from reload_setting family

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert atomic primitive globals to AtomicBool/AtomicI64

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert CRITICAL_*/HUB_API_SECRET/INSTANCE_EVENTS_WEBHOOK/JWT_SECRET to ArcSwap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pin ee-repo-ref to arcswap-refactor EE branch commit

* refactor: convert BASE_URL/HUB_BASE_URL/MIN_VERSION/LICENSE_KEY*/LICENSE_KEY_ID to ArcSwap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert worker hot-path globals to ArcSwap (WORKER_CONFIG et al)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pin ee-repo-ref to combined arcswap-urls+worker EE commit

* chore: update ee-repo-ref to d8be8f88cb8898c8f6b27421989d53528223815d

This commit updates the EE repository reference after PR #532 was merged in windmill-ee-private.

Previous ee-repo-ref: c375aaaac9ec0fc0480993627d0defc8054c31a4

New ee-repo-ref: d8be8f88cb8898c8f6b27421989d53528223815d

Automated by sync-ee-ref workflow.

* fix: cleanup unused imports + fix 2 missed WORKER_CONFIG readers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to ce0f8fbbbde09c4a858312d2d8716d224e99042c

This commit updates the EE repository reference after PR #534 was merged in windmill-ee-private.

Previous ee-repo-ref: 450b601b5aba0ca0b2045f4b5071aa8701b4bfb7

New ee-repo-ref: ce0f8fbbbde09c4a858312d2d8716d224e99042c

Automated by sync-ee-ref workflow.

* fix: secret_backend_integration test — BASE_URL.write().await → .store()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert APP_WORKSPACED_ROUTE to AtomicBool for symmetry with HTTP_ROUTE_WORKSPACED_ROUTE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to e587df8 (post-#535 merge)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-14 00:04:10 +00:00
Ruben FiszelandClaude Opus 4.6 4dc54ca3aa fix: persist indexer max_index_time_window_secs setting (#8821)
* fix: persist indexer max_index_time_window_secs setting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: toggle UX for indexer time window cap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 23:56:38 +00:00
Ruben FiszelandClaude Opus 4.6 89c8e4bb96 fix: detect WAC v2 Python workflows that only use step() (no @task) (#8819)
is_wac_v2_py required both @workflow and @task, so a workflow using
only inline step() calls fell through to the regular Python path and
returned the raw coroutine object instead of its awaited result. Match
the TS detector and accept @workflow alone.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 23:22:30 +00:00
Ruben Fiszelandrubenfiszel eb85da932a chore(main): release 1.683.1 (#8817)
* chore(main): release 1.683.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-13 22:49:27 +00:00
Ruben FiszelandClaude Opus 4.6 f7f26b3224 fix: use OpenAPI 3.0 nullable pattern for getOpenDeploymentRequest (#8816)
The response schema used `oneOf: [$ref, {type: null}]` which is
OpenAPI 3.1 syntax, but the spec is declared as 3.0.3. Both
oapi-codegen (Go) and openapi-python-client rejected it, breaking
the client release jobs. Switched to the standard 3.0 pattern
(`nullable: true` + `allOf: [$ref]`), matching existing usage at
openapi.yaml:21410.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:41:59 +00:00
1964 changed files with 3008 additions and 48690 deletions
+38
View File
@@ -1,5 +1,43 @@
# Changelog
## [1.684.1](https://github.com/windmill-labs/windmill/compare/v1.684.0...v1.684.1) (2026-04-14)
### Bug Fixes
* stop escalating missing email recipients to critical alert ([#8833](https://github.com/windmill-labs/windmill/issues/8833)) ([6158ff2](https://github.com/windmill-labs/windmill/commit/6158ff2ebe29d6a9a7ff4d524e152bb2f7c24dfc))
## [1.684.0](https://github.com/windmill-labs/windmill/compare/v1.683.2...v1.684.0) (2026-04-14)
### Features
* cascade trigger script_path on runnable rename + fix trigger permissioned_as ([#8823](https://github.com/windmill-labs/windmill/issues/8823)) ([64ba3a6](https://github.com/windmill-labs/windmill/commit/64ba3a632eee041d09093e89961f63f2a090fcad))
* **frontend:** improve permissions drawer UX and auto-share resource variables ([#8824](https://github.com/windmill-labs/windmill/issues/8824)) ([91064ce](https://github.com/windmill-labs/windmill/commit/91064ce85712b85e32e3f8cff2a0794cd5597ed6))
### Bug Fixes
* allow dedicated flow substeps to inherit parent tag ([#8832](https://github.com/windmill-labs/windmill/issues/8832)) ([aebf758](https://github.com/windmill-labs/windmill/commit/aebf758412383dd65e0bf6c72de8f2668561cd88))
* compute wall-clock duration for flow job groups in CLI ([#8826](https://github.com/windmill-labs/windmill/issues/8826)) ([e1dbce0](https://github.com/windmill-labs/windmill/commit/e1dbce02c22bcaa3d7d447ee54db69373bc1cf7b))
* DB Manager delete/update for timestamp and serial types ([#8830](https://github.com/windmill-labs/windmill/issues/8830)) ([06fe809](https://github.com/windmill-labs/windmill/commit/06fe809ecc3c6b37af7582175f9dd90c2c2a8f98))
* hide serial types in column type dropdown for existing columns ([#8828](https://github.com/windmill-labs/windmill/issues/8828)) ([7fe639d](https://github.com/windmill-labs/windmill/commit/7fe639d91e93a6b3069e0d87b57c232d67c8ad65))
## [1.683.2](https://github.com/windmill-labs/windmill/compare/v1.683.1...v1.683.2) (2026-04-14)
### Bug Fixes
* detect WAC v2 Python workflows that only use step() (no [@task](https://github.com/task)) ([#8819](https://github.com/windmill-labs/windmill/issues/8819)) ([89c8e4b](https://github.com/windmill-labs/windmill/commit/89c8e4bb9680c179bf44a66a22dcf047334944ae))
* persist indexer max_index_time_window_secs setting ([#8821](https://github.com/windmill-labs/windmill/issues/8821)) ([4dc54ca](https://github.com/windmill-labs/windmill/commit/4dc54ca3aa14beab175da59eb8b9072918301b43))
## [1.683.1](https://github.com/windmill-labs/windmill/compare/v1.683.0...v1.683.1) (2026-04-13)
### Bug Fixes
* use OpenAPI 3.0 nullable pattern for getOpenDeploymentRequest ([#8816](https://github.com/windmill-labs/windmill/issues/8816)) ([f7f26b3](https://github.com/windmill-labs/windmill/commit/f7f26b32244536b6efb7c1b5aafd4a7644dcb42f))
## [1.683.0](https://github.com/windmill-labs/windmill/compare/v1.682.0...v1.683.0) (2026-04-13)
+21
View File
@@ -55,6 +55,7 @@ bun run cli -- run flow flow-test4-order-processing-loop --model opus
bun run cli -- run flow flow-test0-sum-two-numbers --models haiku,opus,4o
bun run cli -- run flow flow-test0-sum-two-numbers --runs 3 --verbose
bun run cli -- run flow --record
WMILL_AI_EVAL_BACKEND_URL=http://127.0.0.1:8000 bun run cli -- run flow --backend-validation preview
bun run cli -- run cli bun-hello-script
```
@@ -72,6 +73,7 @@ Public CLI surface:
- `--models <a,b,c>`: run the same cases sequentially against several model aliases
- `--verbose`: stream assistant output for frontend runs
- `--record`: append a compact tracked summary line to `ai_evals/history/<mode>.jsonl` for full-suite runs only
- `--backend-validation <mode>`: optional backend smoke validation (`off` or `preview`) for `script` and `flow` evals
## Models
@@ -114,6 +116,7 @@ Optional fields:
- `initial`: starting state fixture
- `expected`: expected artifact fixture
- `validate`: extra deterministic validation rules
- `runtime.backendPreview`: optional real backend preview config for smoke validation
For `flow` mode, `validate` can express requirements such as:
@@ -125,6 +128,23 @@ For `flow` mode, an `initial` fixture can also include a benchmark workspace cat
existing scripts and flows. That lets the real `search_workspace` and
`get_runnable_details` tools discover reusable workspace runnables during evals.
If `--backend-validation preview` is enabled:
- `script` evals run a real backend script preview in an isolated temp workspace
- `flow` evals run a real backend flow preview only for cases that define `runtime.backendPreview`
- `flow` cases with `initial.workspace` fixtures seed those scripts and flows into the preview workspace before preview
- when `WMILL_AI_EVAL_BACKEND_WORKSPACE` is set, `ai_evals` treats that workspace as a dedicated test workspace, clears managed eval assets under `f/evals/*` before each preview run, and then reseeds the current case fixtures
Supported backend validation env vars:
- `WMILL_AI_EVAL_BACKEND_VALIDATION=preview`
- `WMILL_AI_EVAL_BACKEND_URL=http://127.0.0.1:8000`
- `WMILL_AI_EVAL_BACKEND_EMAIL=admin@windmill.dev`
- `WMILL_AI_EVAL_BACKEND_PASSWORD=changeme`
- `WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests` to reuse an existing workspace on CE installs with low workspace limits
- `WMILL_AI_EVAL_KEEP_WORKSPACES=1`
- `WMILL_AI_EVAL_WORKSPACE_PREFIX=ai-evals`
## Results And Artifacts
Every run writes:
@@ -158,6 +178,7 @@ Typical artifacts by mode:
- `script`: `script.json` plus the generated script file
- `app`: `app.json` plus frontend/backend files
- `cli`: `assistant-output.txt` plus generated workspace files
- backend-validated attempts also include `backend-preview.json`
## Layout
@@ -0,0 +1,246 @@
import { afterEach, describe, expect, it } from 'bun:test'
import type { BackendValidationSettings } from '../../core/backendValidation'
import { BackendPreviewClient } from './backendPreview'
const ORIGINAL_FETCH = globalThis.fetch
afterEach(() => {
globalThis.fetch = ORIGINAL_FETCH
})
describe('BackendPreviewClient', () => {
it('updates an existing seeded script on path conflict and waits for deployment', async () => {
const requests: Array<{ url: string; init?: RequestInit }> = []
globalThis.fetch = mockFetch(
requests,
textResponse(200, 'token'),
textResponse(200, ''),
textResponse(400, 'Path conflict for f/evals/add_two_numbers with non-archived hash 123'),
jsonResponse(200, { hash: '123' }),
textResponse(200, '456'),
jsonResponse(200, { lock: 'script.lock', lock_error_logs: null })
)
const client = new BackendPreviewClient(
buildSettings({ baseUrl: 'http://backend.test/script-upsert' })
)
await client.createScript({
workspaceId: 'test',
path: 'f/evals/add_two_numbers',
summary: 'Add two numbers',
content: 'export async function main(a: number, b: number) { return a + b }',
language: 'bun'
})
expect(requests.map((entry) => entry.url)).toEqual([
'http://backend.test/script-upsert/api/auth/login',
'http://backend.test/script-upsert/api/w/test/folders/create',
'http://backend.test/script-upsert/api/w/test/scripts/create',
'http://backend.test/script-upsert/api/w/test/scripts/get/p/f/evals/add_two_numbers',
'http://backend.test/script-upsert/api/w/test/scripts/create',
'http://backend.test/script-upsert/api/w/test/scripts/deployment_status/h/456'
])
const updateRequest = requests[4]
expect(updateRequest.init?.method).toBe('POST')
expect(JSON.parse(String(updateRequest.init?.body))).toMatchObject({
path: 'f/evals/add_two_numbers',
parent_hash: '123',
language: 'bun'
})
})
it('updates an existing seeded flow on create conflict', async () => {
const requests: Array<{ url: string; init?: RequestInit }> = []
globalThis.fetch = mockFetch(
requests,
textResponse(200, 'token'),
textResponse(200, ''),
textResponse(400, 'Flow f/evals/add_numbers_flow already exists'),
textResponse(200, '')
)
const client = new BackendPreviewClient(
buildSettings({ baseUrl: 'http://backend.test/flow-upsert' })
)
await client.createFlow({
workspaceId: 'test',
path: 'f/evals/add_numbers_flow',
summary: 'Add numbers',
value: { modules: [] }
})
expect(requests.map((entry) => entry.url)).toEqual([
'http://backend.test/flow-upsert/api/auth/login',
'http://backend.test/flow-upsert/api/w/test/folders/create',
'http://backend.test/flow-upsert/api/w/test/flows/create',
'http://backend.test/flow-upsert/api/w/test/flows/update/f/evals/add_numbers_flow'
])
const updateRequest = requests[3]
expect(updateRequest.init?.method).toBe('POST')
expect(JSON.parse(String(updateRequest.init?.body))).toMatchObject({
path: 'f/evals/add_numbers_flow',
value: { modules: [] }
})
})
it('serializes shared-workspace validations inside the overridden workspace', async () => {
globalThis.fetch = async (input) => {
const url = String(input)
if (url.endsWith('/api/auth/login')) {
return textResponse(200, 'token')
}
if (url.endsWith('/api/workspaces/exists')) {
return textResponse(200, 'true')
}
if (url.endsWith('/api/w/shared-preview/flows/list_paths')) {
return jsonResponse(200, [])
}
if (url.endsWith('/api/w/shared-preview/scripts/list_paths')) {
return jsonResponse(200, [])
}
throw new Error(`Unexpected fetch: ${url}`)
}
const client = new BackendPreviewClient(
buildSettings({
baseUrl: 'http://backend.test/shared-lock',
workspaceOverride: 'shared-preview'
})
)
const order: string[] = []
let releaseFirst: (() => void) | undefined
let notifyFirstStart: (() => void) | undefined
const firstStarted = new Promise<void>((resolve) => {
notifyFirstStart = resolve
})
const first = client.withWorkspace('flow-test1', 1, async () => {
order.push('first:start')
notifyFirstStart?.()
await new Promise<void>((resolve) => {
releaseFirst = resolve
})
order.push('first:end')
})
const second = client.withWorkspace('flow-test2', 1, async () => {
order.push('second:start')
order.push('second:end')
})
await firstStarted
expect(order).toEqual(['first:start'])
releaseFirst?.()
await Promise.all([first, second])
expect(order).toEqual(['first:start', 'first:end', 'second:start', 'second:end'])
})
it('clears managed shared-workspace assets before preview runs', async () => {
const requests: Array<{ url: string; init?: RequestInit }> = []
globalThis.fetch = mockFetch(
requests,
textResponse(200, 'token'),
textResponse(200, 'true'),
jsonResponse(200, ['f/evals/old_subflow', 'u/admin/keep_flow']),
textResponse(200, ''),
jsonResponse(200, ['f/evals/old_script', 'f/shared/keep_script']),
textResponse(200, '')
)
const client = new BackendPreviewClient(
buildSettings({
baseUrl: 'http://backend.test/shared-cleanup',
workspaceOverride: 'shared-preview'
})
)
await client.withWorkspace('flow-test1', 1, async () => undefined)
expect(requests.map((entry) => entry.url)).toEqual([
'http://backend.test/shared-cleanup/api/auth/login',
'http://backend.test/shared-cleanup/api/workspaces/exists',
'http://backend.test/shared-cleanup/api/w/shared-preview/flows/list_paths',
'http://backend.test/shared-cleanup/api/w/shared-preview/flows/delete/f/evals/old_subflow',
'http://backend.test/shared-cleanup/api/w/shared-preview/scripts/list_paths',
'http://backend.test/shared-cleanup/api/w/shared-preview/scripts/delete/p/f/evals/old_script'
])
})
it('retries login after a cached login failure', async () => {
const requests: Array<{ url: string; init?: RequestInit }> = []
globalThis.fetch = mockFetch(
requests,
textResponse(503, 'backend starting'),
textResponse(200, 'token'),
textResponse(200, 'true'),
jsonResponse(200, []),
jsonResponse(200, [])
)
const client = new BackendPreviewClient(
buildSettings({
baseUrl: 'http://backend.test/login-retry',
workspaceOverride: 'shared-preview'
})
)
await expect(client.withWorkspace('flow-test1', 1, async () => undefined)).rejects.toThrow(
'login for backend validation failed'
)
await expect(client.withWorkspace('flow-test1', 1, async () => 'ok')).resolves.toBe('ok')
expect(
requests.filter((entry) => entry.url === 'http://backend.test/login-retry/api/auth/login')
).toHaveLength(2)
})
})
function buildSettings(
overrides: Partial<BackendValidationSettings> = {}
): BackendValidationSettings {
return {
mode: 'preview',
baseUrl: 'http://backend.test/default',
email: 'admin@windmill.dev',
password: 'changeme',
keepWorkspaces: true,
workspacePrefix: 'ai-evals',
pollIntervalMs: 1,
maxWaitMs: 50,
...overrides
}
}
function mockFetch(
requests: Array<{ url: string; init?: RequestInit }>,
...responses: Response[]
): typeof fetch {
const queue = [...responses]
return async (input, init) => {
const url = String(input)
requests.push({ url, init })
const next = queue.shift()
if (!next) {
throw new Error(`Unexpected fetch: ${url}`)
}
return next
}
}
function jsonResponse(status: number, body: unknown): Response {
return new Response(JSON.stringify(body), {
status,
headers: { 'Content-Type': 'application/json' }
})
}
function textResponse(status: number, body: string): Response {
return new Response(body, { status })
}
@@ -0,0 +1,502 @@
import { randomUUID } from 'node:crypto'
import type { BackendValidationSettings } from '../../core/backendValidation'
interface CompletedJobResultMaybe {
completed: boolean
result: unknown
success?: boolean
started?: boolean
}
interface ScriptDeploymentStatus {
lock?: unknown
lock_error_logs?: string | null
}
export interface CompletedPreviewJob {
id: string
success: boolean
result: unknown
logs?: string | null
raw: Record<string, unknown>
}
const tokenCache = new Map<string, Promise<string>>()
const sharedWorkspaceQueue = new Map<string, Promise<void>>()
const managedSharedWorkspacePrefixes = ['f/evals/']
export class BackendPreviewClient {
constructor(private readonly settings: BackendValidationSettings) {}
async withWorkspace<T>(
caseId: string,
attempt: number,
body: (workspaceId: string) => Promise<T>
): Promise<T> {
const workspaceId =
this.settings.workspaceOverride ??
buildWorkspaceId(this.settings.workspacePrefix, caseId, attempt)
const run = async () => {
await this.ensureWorkspace(workspaceId)
if (this.settings.workspaceOverride) {
await this.clearManagedSharedWorkspaceAssets(workspaceId)
}
try {
return await body(workspaceId)
} finally {
if (!this.settings.keepWorkspaces && !this.settings.workspaceOverride) {
await this.deleteWorkspace(workspaceId).catch(() => undefined)
}
}
}
if (this.settings.workspaceOverride) {
return await withSharedWorkspaceLock(workspaceId, run)
}
return await run()
}
async createScript(input: {
workspaceId: string
path: string
summary: string
description?: string
schema?: Record<string, unknown>
content: string
language: string
}): Promise<void> {
await this.ensureFolderForPath(input.workspaceId, input.path)
const payload = {
path: input.path,
summary: input.summary,
description: input.description ?? '',
content: input.content,
schema: input.schema ?? { type: 'object', properties: {}, required: [] },
is_template: false,
language: input.language,
kind: 'script'
}
const response = await this.request(`/w/${encodeURIComponent(input.workspaceId)}/scripts/create`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
})
if (response.ok) {
await this.waitForScriptDeployment(input.workspaceId, input.path, (await response.text()).trim())
return
}
const message = await response.text()
if (!isConflictMessage(message)) {
throw new Error(`create script ${input.path} failed: ${response.status} ${response.statusText} - ${message}`)
}
const currentScript = await this.getScriptByPath(input.workspaceId, input.path)
const currentHash = readStringField(currentScript, 'hash', `script ${input.path}`)
const updateResponse = await this.request(
`/w/${encodeURIComponent(input.workspaceId)}/scripts/create`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
...payload,
parent_hash: currentHash
})
}
)
await expectOk(updateResponse, `update script ${input.path}`)
await this.waitForScriptDeployment(input.workspaceId, input.path, (await updateResponse.text()).trim())
}
async createFlow(input: {
workspaceId: string
path: string
summary: string
description?: string
schema?: Record<string, unknown>
value: Record<string, unknown>
}): Promise<void> {
await this.ensureFolderForPath(input.workspaceId, input.path)
const payload = {
path: input.path,
summary: input.summary,
description: input.description ?? '',
schema: input.schema ?? { type: 'object', properties: {}, required: [] },
value: input.value
}
const response = await this.request(`/w/${encodeURIComponent(input.workspaceId)}/flows/create`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
})
if (response.ok) {
return
}
const message = await response.text()
if (!isConflictMessage(message)) {
throw new Error(`create flow ${input.path} failed: ${response.status} ${response.statusText} - ${message}`)
}
const updateResponse = await this.request(
`/w/${encodeURIComponent(input.workspaceId)}/flows/update/${input.path}`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
}
)
await expectOk(updateResponse, `update flow ${input.path}`)
}
async runScriptPreview(input: {
workspaceId: string
content: string
args: Record<string, unknown>
language: string
path?: string
timeoutSeconds?: number
}): Promise<CompletedPreviewJob> {
const response = await this.request(
withQuery(`/w/${encodeURIComponent(input.workspaceId)}/jobs/run/preview`, {
timeout: input.timeoutSeconds
}),
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
content: input.content,
args: input.args,
language: input.language,
path: input.path
})
}
)
await expectOk(response, 'start script preview')
const jobId = (await response.text()).trim()
return await this.waitForCompletedJob(input.workspaceId, jobId)
}
async runFlowPreview(input: {
workspaceId: string
value: Record<string, unknown>
args: Record<string, unknown>
timeoutSeconds?: number
path?: string
}): Promise<CompletedPreviewJob> {
const response = await this.request(
withQuery(`/w/${encodeURIComponent(input.workspaceId)}/jobs/run/preview_flow`, {
timeout: input.timeoutSeconds
}),
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
value: input.value,
args: input.args,
path: input.path
})
}
)
await expectOk(response, 'start flow preview')
const jobId = (await response.text()).trim()
return await this.waitForCompletedJob(input.workspaceId, jobId)
}
private async ensureWorkspace(workspaceId: string): Promise<void> {
const existsResponse = await this.request('/workspaces/exists', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ id: workspaceId })
})
await expectOk(existsResponse, `check workspace ${workspaceId}`)
if ((await existsResponse.text()).trim() === 'true') {
return
}
const createResponse = await this.request('/workspaces/create', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ id: workspaceId, name: workspaceId })
})
try {
await expectOk(createResponse, `create workspace ${workspaceId}`)
} catch (error) {
const message = error instanceof Error ? error.message : String(error)
if (message.includes('maximum number of workspaces')) {
throw new Error(
`${message}. Reuse an existing workspace with WMILL_AI_EVAL_BACKEND_WORKSPACE=<workspace-id>.`
)
}
throw error
}
}
private async deleteWorkspace(workspaceId: string): Promise<void> {
const response = await this.request(`/workspaces/delete/${encodeURIComponent(workspaceId)}`, {
method: 'DELETE'
})
await expectOk(response, `delete workspace ${workspaceId}`)
}
private async ensureFolderForPath(workspaceId: string, path: string): Promise<void> {
const folderName = extractFolderName(path)
if (!folderName) {
return
}
const response = await this.request(`/w/${encodeURIComponent(workspaceId)}/folders/create`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name: folderName })
})
if (response.ok) {
return
}
const message = await response.text()
if (!message.toLowerCase().includes('already exists')) {
throw new Error(`Failed to create folder ${folderName}: ${message}`)
}
}
private async waitForCompletedJob(
workspaceId: string,
jobId: string
): Promise<CompletedPreviewJob> {
const deadline = Date.now() + this.settings.maxWaitMs
while (Date.now() < deadline) {
const maybeResponse = await this.request(
`/w/${encodeURIComponent(workspaceId)}/jobs_u/completed/get_result_maybe/${encodeURIComponent(jobId)}?get_started=false`
)
await expectOk(maybeResponse, `poll job ${jobId}`)
const maybeResult = (await maybeResponse.json()) as CompletedJobResultMaybe
if (maybeResult.completed) {
const completedResponse = await this.request(
`/w/${encodeURIComponent(workspaceId)}/jobs_u/completed/get/${encodeURIComponent(jobId)}`
)
await expectOk(completedResponse, `get completed job ${jobId}`)
const completedJob = (await completedResponse.json()) as Record<string, unknown>
return {
id: jobId,
success: Boolean(maybeResult.success),
result: maybeResult.result,
logs:
typeof completedJob.logs === 'string' || completedJob.logs === null
? (completedJob.logs as string | null)
: null,
raw: completedJob
}
}
await new Promise((resolve) => setTimeout(resolve, this.settings.pollIntervalMs))
}
throw new Error(`Timed out waiting for preview job ${jobId} to complete`)
}
private async getScriptByPath(workspaceId: string, path: string): Promise<Record<string, unknown>> {
const response = await this.request(`/w/${encodeURIComponent(workspaceId)}/scripts/get/p/${path}`)
await expectOk(response, `get script ${path}`)
return (await response.json()) as Record<string, unknown>
}
private async clearManagedSharedWorkspaceAssets(workspaceId: string): Promise<void> {
const flowPaths = await this.listFlowPaths(workspaceId)
for (const path of flowPaths.filter(isManagedSharedWorkspacePath)) {
await this.deleteFlowByPath(workspaceId, path)
}
const scriptPaths = await this.listScriptPaths(workspaceId)
for (const path of scriptPaths.filter(isManagedSharedWorkspacePath)) {
await this.deleteScriptByPath(workspaceId, path)
}
}
private async listFlowPaths(workspaceId: string): Promise<string[]> {
const response = await this.request(`/w/${encodeURIComponent(workspaceId)}/flows/list_paths`)
await expectOk(response, `list flows in workspace ${workspaceId}`)
return await response.json()
}
private async listScriptPaths(workspaceId: string): Promise<string[]> {
const response = await this.request(`/w/${encodeURIComponent(workspaceId)}/scripts/list_paths`)
await expectOk(response, `list scripts in workspace ${workspaceId}`)
return await response.json()
}
private async deleteFlowByPath(workspaceId: string, path: string): Promise<void> {
const response = await this.request(`/w/${encodeURIComponent(workspaceId)}/flows/delete/${path}`, {
method: 'DELETE'
})
await expectOk(response, `delete flow ${path}`)
}
private async deleteScriptByPath(workspaceId: string, path: string): Promise<void> {
const response = await this.request(`/w/${encodeURIComponent(workspaceId)}/scripts/delete/p/${path}`, {
method: 'POST'
})
await expectOk(response, `delete script ${path}`)
}
private async waitForScriptDeployment(
workspaceId: string,
path: string,
hash: string
): Promise<void> {
const deadline = Date.now() + this.settings.maxWaitMs
while (Date.now() < deadline) {
const response = await this.request(
`/w/${encodeURIComponent(workspaceId)}/scripts/deployment_status/h/${encodeURIComponent(hash)}`
)
await expectOk(response, `check deployment status for script ${path}`)
const deployment = (await response.json()) as ScriptDeploymentStatus
if (deployment.lock != null) {
return
}
if (deployment.lock_error_logs) {
throw new Error(`Script deployment failed for ${path}: ${deployment.lock_error_logs}`)
}
await new Promise((resolve) => setTimeout(resolve, this.settings.pollIntervalMs))
}
throw new Error(`Timed out waiting for script ${path} (${hash}) to deploy`)
}
private async request(path: string, init?: RequestInit): Promise<Response> {
const token = await this.getToken()
return await fetch(`${this.settings.baseUrl}/api${path}`, {
...init,
headers: {
Authorization: `Bearer ${token}`,
...(init?.headers ?? {})
}
})
}
private async getToken(): Promise<string> {
const cacheKey = `${this.settings.baseUrl}|${this.settings.email}`
let tokenPromise = tokenCache.get(cacheKey)
if (!tokenPromise) {
tokenPromise = this.login().catch((error) => {
if (tokenCache.get(cacheKey) === tokenPromise) {
tokenCache.delete(cacheKey)
}
throw error
})
tokenCache.set(cacheKey, tokenPromise)
}
return await tokenPromise
}
private async login(): Promise<string> {
const response = await fetch(`${this.settings.baseUrl}/api/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email: this.settings.email,
password: this.settings.password
})
})
await expectOk(response, 'login for backend validation')
return (await response.text()).trim()
}
}
async function withSharedWorkspaceLock<T>(workspaceId: string, body: () => Promise<T>): Promise<T> {
const previous = sharedWorkspaceQueue.get(workspaceId) ?? Promise.resolve()
let releaseCurrent: (() => void) | undefined
const current = new Promise<void>((resolve) => {
releaseCurrent = resolve
})
const tail = previous.catch(() => undefined).then(() => current)
sharedWorkspaceQueue.set(workspaceId, tail)
await previous.catch(() => undefined)
try {
return await body()
} finally {
releaseCurrent?.()
if (sharedWorkspaceQueue.get(workspaceId) === tail) {
sharedWorkspaceQueue.delete(workspaceId)
}
}
}
function buildWorkspaceId(prefix: string, caseId: string, attempt: number): string {
const caseSlug = caseId
.toLowerCase()
.replace(/[^a-z0-9-]+/g, '-')
.replace(/^-+|-+$/g, '')
.slice(0, 30)
const suffix = randomUUID().slice(0, 8)
return `${prefix}-${caseSlug || 'case'}-a${attempt}-${suffix}`
}
function extractFolderName(path: string): string | null {
if (!path.startsWith('f/')) {
return null
}
const segments = path.split('/').slice(1, -1)
return segments.length > 0 ? segments.join('/') : null
}
function withQuery(
path: string,
params: Record<string, string | number | undefined>
): string {
const query = new URLSearchParams()
for (const [key, value] of Object.entries(params)) {
if (value === undefined) {
continue
}
query.set(key, String(value))
}
const suffix = query.toString()
return suffix ? `${path}?${suffix}` : path
}
async function expectOk(response: Response, context: string): Promise<void> {
if (response.ok) {
return
}
throw new Error(`${context} failed: ${response.status} ${response.statusText} - ${await response.text()}`)
}
function readStringField(
value: Record<string, unknown>,
field: string,
context: string
): string {
const candidate = value[field]
if (typeof candidate === 'string' && candidate.length > 0) {
return candidate
}
throw new Error(`${context} is missing string field ${field}`)
}
function isConflictMessage(message: string): boolean {
const normalized = message.toLowerCase()
return normalized.includes('already exists') || normalized.includes('path conflict')
}
function isManagedSharedWorkspacePath(path: string): boolean {
return managedSharedWorkspacePrefixes.some((prefix) => path.startsWith(prefix))
}
+10 -4
View File
@@ -1,4 +1,5 @@
import { loadSelectedCases } from "../../core/cases";
import { resolveBackendValidationSettings } from "../../core/backendValidation";
import {
formatRunModelLabel,
getFrontendEvalModel,
@@ -22,9 +23,13 @@ export async function runFrontendBenchmarkFromEnv(): Promise<BenchmarkRunResult>
const emitProgress = process.env.WMILL_FRONTEND_AI_EVAL_PROGRESS === "1";
const verbose = process.env.WMILL_FRONTEND_AI_EVAL_VERBOSE === "1";
const model = resolveEvalModel(mode, process.env.WMILL_FRONTEND_AI_EVAL_MODEL);
const backendValidation = resolveBackendValidationSettings({
evalMode: mode,
requestedMode: process.env.WMILL_FRONTEND_AI_EVAL_BACKEND_VALIDATION,
});
const selectedCases = await loadSelectedCases(mode, caseIds);
const modeRunner = getModeRunner(mode, getFrontendEvalModel(model));
const modeRunner = getModeRunner(mode, getFrontendEvalModel(model), backendValidation);
const runModel = formatRunModelLabel(mode, model);
const caseResults = await runSuite({
modeRunner,
@@ -48,15 +53,16 @@ export async function runFrontendBenchmarkFromEnv(): Promise<BenchmarkRunResult>
function getModeRunner(
mode: FrontendBenchmarkMode,
model: ReturnType<typeof getFrontendEvalModel>
model: ReturnType<typeof getFrontendEvalModel>,
backendValidation: ReturnType<typeof resolveBackendValidationSettings>
): ModeRunner<any, any, any> {
switch (mode) {
case "flow":
return createFlowModeRunner(model);
return createFlowModeRunner(model, backendValidation);
case "app":
return createAppModeRunner(model);
case "script":
return createScriptModeRunner(model);
return createScriptModeRunner(model, backendValidation);
}
}
+3 -1
View File
@@ -22,6 +22,7 @@ export async function runFrontendBenchmarkAdapter(input: {
runs: number
model?: string
verbose?: boolean
backendValidation?: string
}): Promise<BenchmarkRunResult> {
const tempDir = await mkdtemp(path.join(tmpdir(), 'wmill-frontend-benchmark-'))
const outputPath = path.join(tempDir, 'result.json')
@@ -48,7 +49,8 @@ export async function runFrontendBenchmarkAdapter(input: {
WMILL_FRONTEND_AI_EVAL_RUNS: String(input.runs),
WMILL_FRONTEND_AI_EVAL_MODEL: input.model ?? "",
WMILL_FRONTEND_AI_EVAL_PROGRESS: '1',
WMILL_FRONTEND_AI_EVAL_VERBOSE: input.verbose ? '1' : '0'
WMILL_FRONTEND_AI_EVAL_VERBOSE: input.verbose ? '1' : '0',
WMILL_FRONTEND_AI_EVAL_BACKEND_VALIDATION: input.backendValidation ?? ''
}
}
)
+15
View File
@@ -3,6 +3,11 @@
Create a flow that takes two numbers, `a` and `b`, and returns their sum.
Keep it simple and use a single step named `sum_numbers`.
expected: ai_evals/fixtures/frontend/flow/expected/test0_sum_two_numbers.json
runtime:
backendPreview:
args:
a: 4
b: 5
judgeChecklist:
- "the flow takes `a` and `b` as inputs"
- "the main step is named `sum_numbers`"
@@ -15,6 +20,11 @@
The flow should take `a` and `b` as inputs and use a single step named `sum_numbers`.
initial: ai_evals/fixtures/frontend/flow/initial/test1_reuse_existing_script_initial.json
expected: ai_evals/fixtures/frontend/flow/expected/test1_reuse_existing_script.json
runtime:
backendPreview:
args:
a: 2
b: 3
judgeChecklist:
- "the flow takes `a` and `b` as inputs"
- "the main step is named `sum_numbers`"
@@ -27,6 +37,11 @@
Use a single step named `call_add_numbers`.
initial: ai_evals/fixtures/frontend/flow/initial/test2_call_existing_subflow_initial.json
expected: ai_evals/fixtures/frontend/flow/expected/test2_call_existing_subflow.json
runtime:
backendPreview:
args:
a: 7
b: 8
judgeChecklist:
- "the parent flow takes `a` and `b` as inputs"
- "the main step is named `call_add_numbers`"
+19
View File
@@ -2,6 +2,10 @@
import { Command, InvalidArgumentError } from "commander";
import { loadCases, loadSelectedCases } from "../core/cases";
import {
BACKEND_VALIDATION_MODES,
parseBackendValidationMode,
} from "../core/backendValidation";
import {
EVAL_MODELS,
type EvalModelSpec,
@@ -43,6 +47,7 @@ async function main() {
" bun run cli -- run flow --models haiku,opus,4o",
" bun run cli -- run flow flow-test0-sum-two-numbers --verbose",
" bun run cli -- run flow --record",
" bun run cli -- run flow --backend-validation preview",
" bun run cli -- run flow flow-test5-simple-modification --runs 3",
" bun run cli -- run cli bun-hello-script",
"",
@@ -77,6 +82,10 @@ async function main() {
.option("--models <names>", "comma-separated model aliases to run sequentially")
.option("--verbose", "stream assistant output during frontend runs")
.option("--record", "append a compact summary line to ai_evals/history/<mode>.jsonl")
.option(
"--backend-validation <mode>",
`backend smoke validation (${BACKEND_VALIDATION_MODES.join(", ")})`
)
.action(
async (
mode: EvalMode,
@@ -88,6 +97,7 @@ async function main() {
models?: string;
verbose?: boolean;
record?: boolean;
backendValidation?: string;
}
) => {
await handleRun({
@@ -99,6 +109,7 @@ async function main() {
models: options.models,
verbose: options.verbose ?? false,
record: options.record ?? false,
backendValidation: options.backendValidation,
});
}
);
@@ -143,6 +154,7 @@ async function handleRun(input: {
models?: string;
verbose: boolean;
record: boolean;
backendValidation?: string;
}) {
if (input.record && input.caseIds.length > 0) {
throw new Error("--record only supports full-suite runs; omit case ids to record history");
@@ -153,9 +165,15 @@ async function handleRun(input: {
const selectedCases = await loadSelectedCases(input.mode, input.caseIds);
const models = resolveRequestedModels(input.mode, input.model, input.models);
const backendValidation = parseBackendValidationMode(
input.backendValidation ?? process.env.WMILL_AI_EVAL_BACKEND_VALIDATION
);
if (input.outputPath && models.length > 1) {
throw new Error("--output only supports a single model run");
}
if (backendValidation !== "off" && input.mode !== "flow" && input.mode !== "script") {
throw new Error("--backend-validation currently supports only flow and script modes");
}
const summaries: Array<{ label: string; passRate: number; averageDurationMs: number }> = [];
@@ -177,6 +195,7 @@ async function handleRun(input: {
runs: input.runs,
model: model.id,
verbose: input.verbose,
backendValidation,
});
const resolvedOutputPath =
+36
View File
@@ -0,0 +1,36 @@
import { describe, expect, it } from "bun:test";
import {
parseBackendValidationMode,
resolveBackendValidationSettings,
} from "./backendValidation";
describe("parseBackendValidationMode", () => {
it("defaults to off", () => {
expect(parseBackendValidationMode(undefined)).toBe("off");
expect(parseBackendValidationMode("0")).toBe("off");
expect(parseBackendValidationMode("false")).toBe("off");
});
it("accepts preview aliases", () => {
expect(parseBackendValidationMode("preview")).toBe("preview");
expect(parseBackendValidationMode("1")).toBe("preview");
expect(parseBackendValidationMode("true")).toBe("preview");
});
it("rejects unknown modes", () => {
expect(() => parseBackendValidationMode("maybe")).toThrow(
"Unsupported backend validation mode: maybe"
);
});
});
describe("resolveBackendValidationSettings", () => {
it("rejects unsupported eval modes", () => {
expect(() =>
resolveBackendValidationSettings({
evalMode: "app",
requestedMode: "preview",
})
).toThrow('Backend validation mode "preview" is only supported for flow and script evals');
});
});
+104
View File
@@ -0,0 +1,104 @@
import type { EvalMode } from "./types";
export const BACKEND_VALIDATION_MODES = ["off", "preview"] as const;
export type BackendValidationMode = (typeof BACKEND_VALIDATION_MODES)[number];
export interface BackendValidationSettings {
mode: BackendValidationMode;
baseUrl: string;
email: string;
password: string;
keepWorkspaces: boolean;
workspaceOverride?: string;
workspacePrefix: string;
pollIntervalMs: number;
maxWaitMs: number;
}
export function parseBackendValidationMode(value?: string | null): BackendValidationMode {
const normalized = value?.trim().toLowerCase();
if (!normalized || normalized === "off" || normalized === "false" || normalized === "0") {
return "off";
}
if (normalized === "preview" || normalized === "true" || normalized === "1") {
return "preview";
}
throw new Error(
`Unsupported backend validation mode: ${value}. Use one of: ${BACKEND_VALIDATION_MODES.join(", ")}`
);
}
export function resolveBackendValidationSettings(input: {
evalMode: EvalMode;
requestedMode?: string | null;
}): BackendValidationSettings {
const mode = parseBackendValidationMode(
input.requestedMode ?? process.env.WMILL_AI_EVAL_BACKEND_VALIDATION
);
if (mode !== "off" && input.evalMode !== "flow" && input.evalMode !== "script") {
throw new Error(
`Backend validation mode "${mode}" is only supported for flow and script evals`
);
}
return {
mode,
baseUrl: normalizeBaseUrl(
process.env.WMILL_AI_EVAL_BACKEND_URL ??
process.env.WINDMILL_URL ??
process.env.WINDMILL_BASE_URL ??
process.env.REMOTE ??
"http://127.0.0.1:8000"
),
email: process.env.WMILL_AI_EVAL_BACKEND_EMAIL ?? "admin@windmill.dev",
password: process.env.WMILL_AI_EVAL_BACKEND_PASSWORD ?? "changeme",
keepWorkspaces: isTruthy(process.env.WMILL_AI_EVAL_KEEP_WORKSPACES),
workspaceOverride: sanitizeOptionalWorkspaceId(process.env.WMILL_AI_EVAL_BACKEND_WORKSPACE),
workspacePrefix: sanitizeWorkspacePrefix(
process.env.WMILL_AI_EVAL_WORKSPACE_PREFIX ?? "ai-evals"
),
pollIntervalMs: parsePositiveInteger(
process.env.WMILL_AI_EVAL_BACKEND_POLL_INTERVAL_MS,
2000
),
maxWaitMs: parsePositiveInteger(process.env.WMILL_AI_EVAL_BACKEND_MAX_WAIT_MS, 120000),
};
}
function normalizeBaseUrl(value: string): string {
return value.replace(/\/+$/, "");
}
function sanitizeWorkspacePrefix(value: string): string {
const sanitized = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9-]+/g, "-")
.replace(/^-+|-+$/g, "");
return sanitized.length > 0 ? sanitized : "ai-evals";
}
function sanitizeOptionalWorkspaceId(value: string | undefined): string | undefined {
const trimmed = value?.trim();
return trimmed ? trimmed : undefined;
}
function isTruthy(value: string | undefined): boolean {
if (!value) {
return false;
}
return ["1", "true", "yes", "on"].includes(value.trim().toLowerCase());
}
function parsePositiveInteger(value: string | undefined, fallback: number): number {
if (!value) {
return fallback;
}
const parsed = Number(value);
return Number.isInteger(parsed) && parsed > 0 ? parsed : fallback;
}
+18
View File
@@ -0,0 +1,18 @@
import { describe, expect, it } from "bun:test";
import { loadCases } from "./cases";
describe("loadCases", () => {
it("loads backend preview runtime config for opt-in flow cases", async () => {
const flowCases = await loadCases("flow");
const caseEntry = flowCases.find((entry) => entry.id === "flow-test1-reuse-existing-script");
expect(caseEntry?.runtime).toEqual({
backendPreview: {
args: {
a: 2,
b: 3,
},
},
});
});
});
+3 -1
View File
@@ -2,7 +2,7 @@ import { readFile } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { parse } from "yaml";
import type { EvalCase, EvalMode, FlowValidationSpec } from "./types";
import type { EvalCase, EvalCaseRuntimeSpec, EvalMode, FlowValidationSpec } from "./types";
const REPO_ROOT = fileURLToPath(new URL("../../", import.meta.url));
const CASES_DIR = path.join(REPO_ROOT, "ai_evals", "cases");
@@ -14,6 +14,7 @@ interface RawEvalCase {
expected?: string;
validate?: FlowValidationSpec;
judgeChecklist?: string[];
runtime?: EvalCaseRuntimeSpec;
}
export function getRepoRoot(): string {
@@ -40,6 +41,7 @@ export async function loadCases(mode: EvalMode): Promise<EvalCase[]> {
expectedPath: resolveFixturePath(entry.expected),
validate: entry.validate,
judgeChecklist: entry.judgeChecklist,
runtime: entry.runtime,
}));
}
+38 -1
View File
@@ -155,6 +155,44 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
run,
}),
];
const artifactFiles = input.modeRunner.buildArtifacts?.(run.actual) ?? [];
if (run.success && input.modeRunner.backendValidate) {
try {
const backendValidation = await input.modeRunner.backendValidate({
evalCase: input.evalCase,
prompt: input.evalCase.prompt,
initial,
expected,
actual: run.actual,
run,
context: {
caseId: input.evalCase.id,
caseNumber: input.caseIndex + 1,
totalCases: input.totalCases,
attempt,
runs: input.runs,
verbose: input.verbose,
onAssistantMessageStart: undefined,
onAssistantChunk: undefined,
onAssistantMessageEnd: undefined,
},
});
if (backendValidation) {
checks.push(...backendValidation.checks);
artifactFiles.push(...(backendValidation.artifactFiles ?? []));
}
} catch (error) {
checks.push(
buildCheck(
"backend validation succeeded",
false,
error instanceof Error ? error.message : String(error)
)
);
}
}
let judgeScore: number | null = null;
let judgeSummary: string | null = null;
@@ -182,7 +220,6 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
);
}
const artifactFiles = input.modeRunner.buildArtifacts?.(run.actual) ?? [];
const attemptResult: BenchmarkAttemptResult = {
attempt,
passed: checks.every((check) => check.passed),
+24
View File
@@ -2,6 +2,15 @@ export const EVAL_MODES = ["cli", "flow", "script", "app"] as const;
export type EvalMode = (typeof EVAL_MODES)[number];
export interface EvalCaseRuntimeBackendPreview {
args?: Record<string, unknown>;
timeoutSeconds?: number;
}
export interface EvalCaseRuntimeSpec {
backendPreview?: EvalCaseRuntimeBackendPreview;
}
export interface FlowValidationSpec {
schemaRequiredPaths?: string[];
schemaAnyOf?: Array<{
@@ -23,6 +32,7 @@ export interface EvalCase {
expectedPath?: string;
validate?: FlowValidationSpec;
judgeChecklist?: string[];
runtime?: EvalCaseRuntimeSpec;
}
export interface BenchmarkCheck {
@@ -43,6 +53,11 @@ export interface BenchmarkArtifactFile {
content: string;
}
export interface BackendValidationResult {
checks: BenchmarkCheck[];
artifactFiles?: BenchmarkArtifactFile[];
}
export interface BenchmarkTokenUsage {
prompt: number;
completion: number;
@@ -91,6 +106,15 @@ export interface ModeRunner<TInitial, TExpected, TActual> {
actual: TActual;
run: ModeRunOutput<TActual>;
}): BenchmarkCheck[];
backendValidate?(input: {
evalCase: EvalCase;
prompt: string;
initial: TInitial | undefined;
expected: TExpected | undefined;
actual: TActual;
run: ModeRunOutput<TActual>;
context: ModeRunContext;
}): Promise<BackendValidationResult | null>;
buildArtifacts?(actual: TActual): BenchmarkArtifactFile[];
}
@@ -5,7 +5,7 @@
"id": "sum_numbers",
"value": {
"type": "script",
"path": "f/evals/add_two_numbers.ts",
"path": "f/evals/add_two_numbers",
"input_transforms": {
"a": {
"type": "javascript",
@@ -2,7 +2,7 @@
"workspace": {
"scripts": [
{
"path": "f/evals/add_two_numbers.ts",
"path": "f/evals/add_two_numbers",
"summary": "Add two numbers",
"description": "Returns the sum of two numeric inputs.",
"language": "bun",
+92 -1
View File
@@ -1,4 +1,5 @@
import { readJsonFile } from "../core/files";
import type { BackendValidationSettings } from "../core/backendValidation";
import type { FrontendEvalModelConfig } from "../core/models";
import { validateFlowState, type FlowState } from "../core/validators";
import type { BenchmarkArtifactFile, ModeRunner } from "../core/types";
@@ -7,6 +8,7 @@ import {
type FlowFixture,
} from "../adapters/frontend/core/flow/flowEvalRunner";
import type { FlowWorkspaceFixtures } from "../adapters/frontend/core/flow/fileHelpers";
import { BackendPreviewClient } from "../adapters/frontend/backendPreview";
import { DEFAULT_FRONTEND_EVAL_MODEL, getFrontendApiKey } from "./frontendCommon";
interface FlowInitialFixture {
@@ -15,7 +17,8 @@ interface FlowInitialFixture {
}
export function createFlowModeRunner(
modelConfig: FrontendEvalModelConfig = DEFAULT_FRONTEND_EVAL_MODEL
modelConfig: FrontendEvalModelConfig = DEFAULT_FRONTEND_EVAL_MODEL,
backendValidation?: BackendValidationSettings
): ModeRunner<FlowInitialFixture, FlowState, FlowState> {
return {
mode: "flow",
@@ -61,6 +64,65 @@ export function createFlowModeRunner(
validate: evalCase.validate,
});
},
async backendValidate({ evalCase, initial, actual, context }) {
if (backendValidation?.mode !== "preview" || !evalCase.runtime?.backendPreview) {
return null;
}
if (!actual.value) {
return {
checks: [
{
name: "backend flow preview succeeded",
passed: false,
details: "Generated flow is missing value.modules",
},
],
};
}
const previewClient = new BackendPreviewClient(backendValidation);
return await previewClient.withWorkspace(evalCase.id, context.attempt, async (workspaceId) => {
await seedWorkspaceFixtures(previewClient, workspaceId, initial?.workspace);
const completedJob = await previewClient.runFlowPreview({
workspaceId,
value: actual.value as Record<string, unknown>,
args: evalCase.runtime?.backendPreview?.args ?? {},
timeoutSeconds: evalCase.runtime?.backendPreview?.timeoutSeconds,
});
return {
checks: [
{
name: "backend flow preview succeeded",
passed: completedJob.success,
details: completedJob.success
? `workspace=${workspaceId}`
: `workspace=${workspaceId}; job=${completedJob.id}`,
},
],
artifactFiles: [
{
path: "backend-preview.json",
content:
JSON.stringify(
{
workspaceId,
jobId: completedJob.id,
success: completedJob.success,
result: completedJob.result,
logs: completedJob.logs,
completedJob: completedJob.raw,
},
null,
2
) + "\n",
},
],
};
});
},
buildArtifacts(actual): BenchmarkArtifactFile[] {
return [
{
@@ -102,3 +164,32 @@ function normalizeFlowStateFixture(value: unknown): FlowState {
function isObject(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
async function seedWorkspaceFixtures(
previewClient: BackendPreviewClient,
workspaceId: string,
fixtures?: FlowWorkspaceFixtures
): Promise<void> {
for (const script of fixtures?.scripts ?? []) {
await previewClient.createScript({
workspaceId,
path: script.path,
summary: script.summary,
description: script.description,
schema: script.schema,
content: script.content,
language: script.language,
});
}
for (const flow of fixtures?.flows ?? []) {
await previewClient.createFlow({
workspaceId,
path: flow.path,
summary: flow.summary,
description: flow.description,
schema: flow.schema,
value: flow.value as Record<string, unknown>,
});
}
}
+63 -1
View File
@@ -1,13 +1,16 @@
import { readJsonFile } from "../core/files";
import type { BackendValidationSettings } from "../core/backendValidation";
import type { FrontendEvalModelConfig } from "../core/models";
import { validateScriptState } from "../core/validators";
import type { BenchmarkArtifactFile, ModeRunner } from "../core/types";
import { BackendPreviewClient } from "../adapters/frontend/backendPreview";
import { runScriptEval } from "../adapters/frontend/core/script/scriptEvalRunner";
import type { ScriptEvalState } from "../adapters/frontend/core/script/fileHelpers";
import { DEFAULT_FRONTEND_EVAL_MODEL, getFrontendApiKey } from "./frontendCommon";
export function createScriptModeRunner(
modelConfig: FrontendEvalModelConfig = DEFAULT_FRONTEND_EVAL_MODEL
modelConfig: FrontendEvalModelConfig = DEFAULT_FRONTEND_EVAL_MODEL,
backendValidation?: BackendValidationSettings
): ModeRunner<ScriptEvalState, ScriptEvalState, ScriptEvalState> {
return {
mode: "script",
@@ -45,6 +48,57 @@ export function createScriptModeRunner(
validate({ actual, initial, expected }) {
return validateScriptState({ actual, initial, expected });
},
async backendValidate({ evalCase, initial, actual, context }) {
if (backendValidation?.mode !== "preview") {
return null;
}
const previewClient = new BackendPreviewClient(backendValidation);
return await previewClient.withWorkspace(evalCase.id, context.attempt, async (workspaceId) => {
const completedJob = await previewClient.runScriptPreview({
workspaceId,
content: actual.code,
args:
(evalCase.runtime?.backendPreview?.args as Record<string, unknown> | undefined) ??
actual.args ??
initial?.args ??
{},
language: normalizePreviewLanguage(actual.lang),
path: toPreviewScriptPath(actual.path),
timeoutSeconds: evalCase.runtime?.backendPreview?.timeoutSeconds,
});
return {
checks: [
{
name: "backend script preview succeeded",
passed: completedJob.success,
details: completedJob.success
? `workspace=${workspaceId}`
: `workspace=${workspaceId}; job=${completedJob.id}`,
},
],
artifactFiles: [
{
path: "backend-preview.json",
content:
JSON.stringify(
{
workspaceId,
jobId: completedJob.id,
success: completedJob.success,
result: completedJob.result,
logs: completedJob.logs,
completedJob: completedJob.raw,
},
null,
2
) + "\n",
},
],
};
});
},
buildArtifacts(actual): BenchmarkArtifactFile[] {
return [
{
@@ -59,3 +113,11 @@ export function createScriptModeRunner(
},
};
}
function normalizePreviewLanguage(language: ScriptEvalState["lang"]): string {
return language === "bunnative" ? "bun" : language;
}
function toPreviewScriptPath(filePath: string): string {
return filePath.replace(/\.[^.\/]+$/, "");
}
@@ -1,44 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n oauth_data as \"oauth_data: sqlx::types::Json<WorkspaceOAuthConfig>\",\n service_name as \"service_name!: ServiceName\",\n resource_path\n FROM\n workspace_integrations\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "oauth_data: sqlx::types::Json<WorkspaceOAuthConfig>",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "service_name!: ServiceName",
"type_info": {
"Custom": {
"name": "native_trigger_service",
"kind": {
"Enum": [
"nextcloud",
"google"
]
}
}
}
},
{
"ordinal": 2,
"name": "resource_path",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true,
false,
true
]
},
"hash": "0010ef26da16facd1c2c832601ac687c4c27de46a90f45496b8446af1a9d0578"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by FROM v2_job WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "created_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "002d68d7c4437522a6dae95af007a356217bbae06b8453f0c32046f0cbf20dcb"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE alerts SET acknowledged_workspace = true, acknowledged = true WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "00588a40dde5189ac1c61505f17acb0f4c244c60477427505bf5bd1b104d3bf9"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT email FROM password WHERE email = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "005b9255699e73600c579f74b529caf531b2312b6e405b4d35efd2f7ca663143"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET disabled = $1 WHERE username = $2 AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "006f03e979abdf8055b1c598bc9806337216a6abf74db4eb64b0acb918a0de08"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_status\n SET flow_status = JSONB_SET(flow_status, ARRAY['cleanup_module', 'flow_jobs_to_clean'], COALESCE(flow_status->'cleanup_module'->'flow_jobs_to_clean', '[]'::jsonb) || $1)\n WHERE id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Uuid"
]
},
"nullable": []
},
"hash": "007fa93171b244490b94464938b9f95aca4e91bccde6da93cb151799b3398049"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\" FROM v2_job WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "args: sqlx::types::Json<HashMap<String, Box<RawValue>>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
true
]
},
"hash": "0084c1246d1391d106da2e67a394eafc6695257632406ed9a2111dba1dd106c7"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT runnable_path FROM v2_job WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "runnable_path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
true
]
},
"hash": "00b6340396f5121aff49323d12a7e3244ebb2cb864828637e7ff7476a4a4939b"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO dependency_map (workspace_id, importer_path, importer_kind, imported_path, importer_node_id)\n SELECT $1, importer_path, importer_kind, imported_path, importer_node_id\n FROM dependency_map\n WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "00b9f392a5cc07bd4ed14e3b69f96408e219d70015dd2f419fc87a440f070c64"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE background_task_state\n SET value = $1, updated_at = NOW()\n WHERE name = $2 AND owner = $3 AND running = true",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "00bdac005e3fb2167c90c26b96ab267c19d4b6578736c852114df96ed5575952"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM usr WHERE workspace_id = $1 AND email = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "00be497354f5375e9ccffb998d126a853da91d607ff9e57e10d0e5481e4d3848"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = now() + interval '60 seconds'\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3 AND lease_until > now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "00bf3dbd9d3f51dd7fdefcbd654d55e0379cc84188954037165cbe2d198ef71f"
}
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT f.path\n FROM workspace_runnable_dependencies wru \n JOIN flow f\n ON wru.flow_path = f.path AND wru.workspace_id = f.workspace_id\n WHERE wru.runnable_path = $1 AND wru.runnable_is_flow = $2 AND wru.workspace_id = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Bool",
"Text"
]
},
"nullable": [
false
]
},
"hash": "00c0ae12b19ba495f307f0ce6b4833947c5b3fe45826fc5468e326d171d95236"
}
@@ -1,58 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n workspace.id AS \"id!\",\n workspace.name AS \"name!\",\n workspace.owner AS \"owner!\",\n workspace.deleted AS \"deleted!\",\n workspace.premium AS \"premium!\",\n workspace_settings.color AS \"color\",\n workspace.parent_workspace_id AS \"parent_workspace_id\"\n FROM workspace\n LEFT JOIN workspace_settings ON workspace.id = workspace_settings.workspace_id\n WHERE workspace.id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "owner!",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "deleted!",
"type_info": "Bool"
},
{
"ordinal": 4,
"name": "premium!",
"type_info": "Bool"
},
{
"ordinal": 5,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "parent_workspace_id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false,
false,
false,
false,
true,
true
]
},
"hash": "00c1dd0cfaf15aafdcfcabc1f123cebdf8d777f48e148bcb171fa15e8bf6f098"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE resource SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "00f68ea1ad2d5ab045b4a20ce3f4dd7850041e396e14890e37e1d0db276d3694"
}
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT data FROM app_bundles WHERE app_version_id = $1 AND file_type = $2 AND w_id = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "data",
"type_info": "Bytea"
}
],
"parameters": {
"Left": [
"Int8",
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "01050e7057f3d1971ad9e47ac83bf6a3c3c9f41689c3607f0b264437ae6b3324"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow SET archived = $1 WHERE path = $2 AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "01576057b71e32e20a1702b2c89c380eee422060a65b475d7cda8b199d70842e"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT group_ FROM usr_to_group WHERE usr = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "group_",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "015a8551c646f9b027fc23752c5c5c81e520e3ca97dd1cd1e4ebfe3e46c4ad11"
}
@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_runnable_dependencies (app_path, runnable_path, runnable_is_flow, workspace_id) VALUES ($1, $2, $3, $4) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Bool",
"Varchar"
]
},
"nullable": []
},
"hash": "01755585cd3f6e100a66da331720286cbc09d4abf2926146b24a8c95cf21e5c8"
}
@@ -1,28 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n id As \"id!\",\n flow_status->'restarted_from'->'flow_job_id' AS \"restarted_from: Json<Uuid>\"\n FROM v2_job_status\n WHERE COALESCE((SELECT flow_innermost_root_job FROM v2_job WHERE id = $1), $1) = id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "restarted_from: Json<Uuid>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false,
null
]
},
"hash": "019100d178129340a7c35d60ab61f983c8a9cb810db4369554bf26c6b0d6003d"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT app.id FROM app\n WHERE app.path = $1 AND app.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "019258392434b3c8dfabfe53d61ad766626fe4ad67f101c1a58c9c9524531621"
}
@@ -1,12 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version (id, app_id, value, created_by, created_at)\n VALUES (3001, 3001, '{\"grid\": []}', 'admin', NOW())",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "01c040b04b487e86b7f4ff38b0faacf6af2c284ae446860113c82bc4e1da08ab"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value FROM variable WHERE path = $1 AND workspace_id = $2 AND is_secret = true",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "020c031c3de6c85577e30421ada9d39a5a47ca1b6cf3dbfd6988aa0694d7364c"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job WHERE workspace_id = $1 AND id = ANY($2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"UuidArray"
]
},
"nullable": []
},
"hash": "023555d33652d40fa381b1baaae6b319c4bac92cb2d90bb4ffd08e25f4a4d18b"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM global_settings WHERE name = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "025e5bf6aef56e9832239489daf21b4970d84529cf7ed7928ce2dcfec8b007f0"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO websocket_trigger (\n path, url, script_path, is_flow, workspace_id,\n edited_by, permissioned_as, server_id, error\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "02748cae17e8966dbd57a33017ccb747c84fcc12fbfd93c6c749570b94d35696"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE asset SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "029b81eb00250eacded407b12bcfbab2b3f35354bdb9ef6e30281a4ff6235060"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT flow_version.value AS \"value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>\"\n FROM flow\n LEFT JOIN flow_version\n ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n WHERE flow.path = $1 AND flow.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "02bf9763298f301d4fc75490c070a0663142d4d23a2df007361622b94d4783e1"
}
@@ -1,77 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n v2_job.permissioned_as_email,\n v2_job.created_by,\n v2_job.parent_job,\n v2_job.permissioned_as,\n v2_job.runnable_path,\n CASE WHEN v2_job.trigger_kind = 'schedule'::job_trigger_kind THEN v2_job.trigger END AS schedule_path,\n v2_job.flow_step_id,\n v2_job.flow_innermost_root_job,\n v2_job.root_job,\n v2_job_queue.scheduled_for AS \"scheduled_for: chrono::DateTime<chrono::Utc>\"\n FROM v2_job INNER JOIN v2_job_queue ON v2_job.id = v2_job_queue.id\n WHERE v2_job.id = $1 AND v2_job.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 3,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "schedule_path",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 8,
"name": "root_job",
"type_info": "Uuid"
},
{
"ordinal": 9,
"name": "scheduled_for: chrono::DateTime<chrono::Utc>",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false,
false,
true,
false,
true,
null,
true,
true,
true,
false
]
},
"hash": "02c945b5f18a56a826721f6884846d79167747742de236ce57f395561685adc0"
}
@@ -1,32 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO websocket_trigger (\n path, url, script_path, is_flow, workspace_id,\n edited_by, permissioned_as, mode\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8::trigger_mode)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Varchar",
{
"Custom": {
"name": "trigger_mode",
"kind": {
"Enum": [
"enabled",
"disabled",
"suspended"
]
}
}
}
]
},
"nullable": []
},
"hash": "02e04f9ebc0e14f98f290bf2dc3eb00bc613ba7d29f8dd5ff31a4acd0ef3adfd"
}
@@ -1,38 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE (workspace_id, path, kind) IN (\n SELECT workspace_id, path, kind FROM (\n SELECT a.workspace_id, a.path, a.kind, a.usage_kind, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.usage_kind = 'job'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "02e526146f3584cd599dec708e1be48db3b0cd1c74adbfa2e4039377daa016f0"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT jsonb_object_keys(ws.ducklake->'ducklakes') AS ducklake_name\n FROM workspace_settings ws\n WHERE ws.workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "ducklake_name",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "02ecdcc882931d5cbb2243e32805c8a1291a5106fff46ceba85fa27d50a0354c"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job_status WHERE id = ANY($1)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"UuidArray"
]
},
"nullable": []
},
"hash": "02fdd7b94e6b6c9bb7985dfeb2082655d08946206dcfb25158c10f78619cf7fc"
}
@@ -1,19 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_diff SET has_changes = true, exists_in_source = $5, exists_in_fork = $6\n WHERE path = $3 AND kind = $4 AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text",
"Bool",
"Bool"
]
},
"nullable": []
},
"hash": "034a8519198daf30e0eb8a74ed92f896c83bb39e1cb52fe3c29c1a224c3859c2"
}
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT path FROM resource WHERE value::text LIKE $1 AND NOT path LIKE $2 AND workspace_id = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "0359f73f7dd278cacd98a785d7ce13f7964fae84d98decf06a88248b740477db"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO job_delete_schedule (job_id, workspace_id, delete_at) VALUES ($1, $2, now() + make_interval(secs => $3::double precision)) ON CONFLICT (job_id) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Float8"
]
},
"nullable": []
},
"hash": "035e29e775bfc5b236100135e1d94a4baf2b617b86f0c3c74ba9a00b859993f6"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE schedule SET enabled = false WHERE workspace_id = $1 AND enabled = true RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "03669873e4e3b22c737d5170821f677925474aad885bf1c0780bdb978225517e"
}
@@ -1,41 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, topic, partition, \"offset\" FROM kafka_pending_commits\n WHERE workspace_id = $1 AND kafka_trigger_path = $2\n ORDER BY id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "topic",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "partition",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "offset",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false
]
},
"hash": "038d2fde90fa9e99e30d15161777fa3ab402e33edfca46daa95b52e525424586"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET auto_invite = COALESCE(auto_invite, '{}'::jsonb)\n || jsonb_build_object('instance_groups', $2::jsonb, 'instance_groups_roles', $3::jsonb)\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "03caaec9f41be87eac32e163d7a829e58ae993932e9bbad5aebab4770cc44d61"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET username = $1 WHERE email = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "03cb31686d34035f2dc64d4dff93265af0e44a7e2dabdf28fd9c9327b5b53118"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE app SET policy = jsonb_set(policy, ARRAY['on_behalf_of'], to_jsonb('u/' || $1)) WHERE policy->>'on_behalf_of' = ('u/' || $2) AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "03d4ec7a5feda9001e98c2e55f0788399ee41c9b128099937abb483f1d89a2c6"
}
@@ -1,65 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT * from resource_type WHERE name = $1 AND (workspace_id = $2 OR workspace_id = 'admins')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "schema",
"type_info": "Jsonb"
},
{
"ordinal": 3,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "edited_at",
"type_info": "Timestamptz"
},
{
"ordinal": 5,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "format_extension",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "is_fileset",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
true,
true,
true,
true,
true,
false
]
},
"hash": "03d63d2e64b012f624d2731b5bcb8849c74a9474777be61edf0ed43ddda07ef3"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM usr_to_group WHERE usr = $1 AND workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "03e9c9d8dd74c75b4608b49cad79f0a9572c7e8b48d8e373c0342b75b843d123"
}
@@ -1,20 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM password WHERE email = 'test2@windmill.dev')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "04362999299bd8026dd8d5ae6f2ce462f1dd21f0a2d5af64afd6207313258e6e"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO tutorial_progress (email, progress, skipped_all) VALUES ($2, $1::bigint::bit(64), $3) ON CONFLICT (email) DO UPDATE SET progress = EXCLUDED.progress, skipped_all = EXCLUDED.skipped_all",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Varchar",
"Bool"
]
},
"nullable": []
},
"hash": "04362a55081f7a98bca8fe4db0669939da8944711037957664cc2989b239c9d1"
}
@@ -1,18 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO alerts (alert_type, message, acknowledged, acknowledged_workspace, workspace_id, resource)\n VALUES ('critical_error', $1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Bool",
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "044e2b428ee6e2dd4543c87ad8835e239cf7567d18b8b3fa6608ea3a9d206ca7"
}
@@ -1,20 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM usr WHERE workspace_id = 'ws-with-auto-add' AND email = 'bob@example.com')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "045c5b21422113ce8592d1b645d4e513e0eff982c80d4ce490ee5381627f8d16"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE ci_test_reference SET tested_item_path = $1 WHERE tested_item_path = $2 AND workspace_id = $3 AND tested_item_kind = 'resource'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "04b56007b5d33d32a2e3afcaddd094c6c8d7434a31dcdeeb2875bd89cd6bd328"
}
@@ -1,20 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM v2_job",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "04ce5c530c80ae6f911dfe0dc9ed7d1a2e10342bbbc7f8486df0b73f5657a493"
}
@@ -1,28 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT healthy, created_at FROM healthchecks WHERE check_type = 'min_alive_workers_' || $1 ORDER BY created_at DESC LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "healthy",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "050fb876e10ad13654dbbde4532f408ff1ac92ed0f5d31a3ef6c58313e1f8671"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE schedule SET on_success = NULL, on_success_extra_args = NULL WHERE workspace_id = $1 RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "051acd49542618e3be5654dce94567a02e87be23a4c0863e3b2d53b5c2cb6749"
}
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT path FROM script WHERE content LIKE $1 AND NOT path LIKE $2 AND workspace_id = $3 AND NOT archived AND NOT deleted",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "056318f055f6ff2466952d387a3fea11132a8edf371f4cac0001804bee170c01"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM\n capture\n WHERE\n workspace_id = $1\n AND created_at <= (\n SELECT\n created_at\n FROM\n capture\n WHERE\n workspace_id = $1\n ORDER BY\n created_at DESC\n OFFSET $2\n LIMIT 1\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Int8"
]
},
"nullable": []
},
"hash": "0574df3e18f626dd8b3f83fbff8b0ee99cf8483a8fe66fa9311cb96e3f5a0ee2"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT status = 'success' OR status = 'skipped' AS \"success!\" FROM v2_job_completed WHERE id = ANY($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "success!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"UuidArray"
]
},
"nullable": [
null
]
},
"hash": "05c65ba8a56b3b5f8bd37c30c0c6707522e01c4a05104969889b7bb41d6aa509"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM resource WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "05e05a9b979941c7a11cd881da652f459e4a0444d63a96deba4a879fbe1124ff"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name FROM group_ WHERE workspace_id = $1 UNION SELECT name FROM instance_group ORDER BY name asc",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "05f4663a0f58736e92fe7cbbef3c99a03bc74ab3be1bacdbbf3910a76a1beacc"
}
@@ -1,26 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT workspace_id, path FROM variable WHERE is_secret = true",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false
]
},
"hash": "0600f2a9179f83502c6b13e8e4284f85ca82636f274f5dce47da5a8320a60088"
}
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH updated AS (\n UPDATE script SET path = REGEXP_REPLACE(path, 'u/' || $2 || '/(.*)', $1 || '/\\1')\n WHERE path LIKE ('u/' || $2 || '/%') AND workspace_id = $3\n RETURNING 1\n ) SELECT COUNT(*) FROM updated",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "0607aa96fee42b302c0f2ae2b5538c1d5118cbae680c1f58731d90f57f3fd76b"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO kafka_trigger (\n path, kafka_resource_path, topics, group_id, script_path,\n is_flow, workspace_id, edited_by, permissioned_as\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"VarcharArray",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "066c9690d1606bf889879b7e3c686529c37db0d5f18c83706bfbc63c8c3e4315"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, tag)\n SELECT unnest($1::uuid[]), 'test-workspace', now(), 'flow'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"UuidArray"
]
},
"nullable": []
},
"hash": "0681b850c033619e1b9498376263681f875a5aba22170ca50ec8b578f7fa478b"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT ai_config FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "ai_config",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true
]
},
"hash": "0689cdc6c7676f5e1984792a0e0b172ea9a70835bfba6cef56142556197e9767"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_status\n SET flow_status = flow_status - 'retry'\n WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": []
},
"hash": "06db0e720dd59a7c52c0a98ea7b316237eb1a547678858c1a1e45985035b3468"
}
@@ -1,59 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n workspace.id AS \"id!\",\n workspace.name AS \"name!\",\n workspace.owner AS \"owner!\",\n workspace.deleted AS \"deleted!\",\n workspace.premium AS \"premium!\",\n workspace_settings.color AS \"color\",\n workspace.parent_workspace_id AS \"parent_workspace_id\"\n FROM workspace\n LEFT JOIN workspace_settings ON workspace.id = workspace_settings.workspace_id\n LIMIT $1 OFFSET $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "owner!",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "deleted!",
"type_info": "Bool"
},
{
"ordinal": 4,
"name": "premium!",
"type_info": "Bool"
},
{
"ordinal": 5,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "parent_workspace_id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
false,
false,
true,
true
]
},
"hash": "07168aaf14cb6beff0ad4274b441f7f387f5055c47f493271d26731336257384"
}
@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, deployment_msg) VALUES ($1, $2, $3, $4) ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL\n DO UPDATE SET deployment_msg = EXCLUDED.deployment_msg",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "07335b75233811352fb898cf3d6c8fe7fd014adbf40cc4bc8c041f5864423367"
}
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM app WHERE policy->>'on_behalf_of' = $1 AND NOT path LIKE $2 AND workspace_id = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "0738680c7e9acedb6b8038f4e2f78289ab0268bf24c1d55d4c48abf77b697b65"
}
@@ -1,12 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO email_to_igroup (email, igroup) VALUES ('alice@example.com', 'admins') ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "077467cd813d5af161cb1cc232724f26984822d4c28ba36c0a9331273b10edc0"
}
@@ -1,20 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT setting::bigint as \"max!\" FROM pg_settings WHERE name = 'max_connections'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "max!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "07770a002a49428c4f956cfc7262d6b6792ae5b97ed90b0ee07d17480b2dffe2"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO email_to_igroup (email, igroup) VALUES ($1, $2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "07834003a631acac03a52a128d3dfce162abdef0d94e5020b84fed424dfd1ae9"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM resource WHERE path = $1 AND workspace_id = $2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "0784bb86a503f02b9ef247a2b83a82ddfa49632552b223a9f4536a449b0a1eb8"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE http_trigger SET script_path = $1 WHERE workspace_id = $2 AND path = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "079b7f09da952cee6fa594c91a932781327f2b10a6dc4c6c82414ef0a610a10c"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM flow WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "07b984faa14193f529ae589608289a9ffb0f3d4a0e48054393e1bb8b70b4e5e0"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE input SET created_by = $1 WHERE created_by = $2 AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "07ce3bce5f71b13d8aaf51f7d8aef8e32ff9e3e9cf58daab1c02276a7f0bd163"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow_version SET path = REGEXP_REPLACE(path,'u/' || $2 || '/(.*)','u/' || $1 || '/\\1') WHERE path LIKE ('u/' || $2 || '/%') AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "07d03985bb2c58d52c1ffd6ab5a6d37457e7520642a5e70bb4000e4923720957"
}
@@ -1,19 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_status SET flow_status = \n CASE WHEN flow_status->'modules'->$1::int->'flow_jobs_duration' IS NOT NULL THEN\n JSONB_SET(JSONB_SET(JSONB_SET(\n flow_status,\n ARRAY['modules', $1::TEXT, 'flow_jobs_success', $3::TEXT],\n $4\n ),\n ARRAY['modules', $1::TEXT, 'flow_jobs_duration', 'duration_ms', $3::TEXT], $5),\n ARRAY['modules', $1::TEXT, 'flow_jobs_duration', 'started_at', $3::TEXT], $6)\n ELSE\n JSONB_SET(flow_status, ARRAY['modules', $1::TEXT, 'flow_jobs_success', $3::TEXT], $4)\n END\n WHERE id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int4",
"Uuid",
"Text",
"Jsonb",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "081f838b3dbe81631d17e7ca0751db725a7f92d4e43a86bcfa06a4ac7c70ac8f"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE schedule SET ws_error_handler_muted = false, on_failure = NULL, on_failure_extra_args = NULL, on_failure_times = NULL, on_failure_exact = NULL WHERE workspace_id = $1 RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "083218b582d927d98879b9cd8193d8294c1a0e6e50d5de5d5fe1c181391a38d8"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT large_file_storage->>'volume_storage' FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "083d69abc8a662bb364cf43b8ffc6e9b159a54c179cecb108068597536835f7e"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM trashbin WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": []
},
"hash": "08522e494e34f4ecae21460262bf0ed3c5a197dd744c87cb760aaf47001febbd"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM flow_conversation WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "08574e8e5dc165041750880fb02e7ffea83ae94a670b598b6dada0b3d0914629"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM flow_conversation WHERE id = $1) as \"exists!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "086fdf726b88e9f4fd9750bf9dd7f49c589465194548d88e5ae30872846b70a9"
}
@@ -1,48 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM v2_job\n WHERE workspace_id = $1\n AND (kind = 'unassigned_script'::JOB_KIND OR kind = 'unassigned_flow'::JOB_KIND OR kind = 'unassigned_singlestepflow'::JOB_KIND)\n AND trigger_kind = $2\n AND trigger = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "job_trigger_kind",
"kind": {
"Enum": [
"webhook",
"http",
"websocket",
"kafka",
"email",
"nats",
"schedule",
"app",
"ui",
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud",
"google",
"ci_test"
]
}
}
},
"Text"
]
},
"nullable": [
false
]
},
"hash": "089d7bc7acdbb97cf477159e111bc7e9ee85289ff5c52af43166928337c257e7"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO global_settings (name, value) VALUES ($1, $2) ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value, updated_at = now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "08c1121171b98889f188ea6b33b1861f3483fa70b5d58dd2838a5cb6dabe9cc1"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM app WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "08c827d9b2de0b77ce0ea2653760751615112c501b35e931ed817dbefd7c6bdb"
}

Some files were not shown because too many files have changed in this diff Show More