Merge remote-tracking branch 'origin/main' into free-token-limit

This commit is contained in:
Diego Imbert
2026-07-09 09:35:16 +02:00
1201 changed files with 109546 additions and 16084 deletions
+40
View File
@@ -0,0 +1,40 @@
---
name: ai-chat
description: Guidance for improving the Windmill AI chat (copilot), especially global mode — tools, prompts, and context-window discipline. Use when editing chat tools, system prompts, or tool-result shapes under frontend/src/lib/components/copilot/chat, or when changing how the chat manages its context window.
---
## Always benchmark before and after
No context or behavior change ships without an `ai_evals` A/B on the affected mode.
Add or adjust cases for exactly what you changed — see the `ai-evals` skill for
authoring and the full run reference.
Run the affected mode **before** your change and **after**, same model(s), same cases.
## Measure the window first, and cumulative second
Optimize **`finalContextTokens`** (window occupancy — what drives overflow and
compaction), then cumulative prompt tokens.
## Context discipline
The dominant fixed cost is per-iteration overhead: the system prompt **plus every
tool schema** is re-sent on every loop iteration. So:
- **Every tool and every parameter is a permanent tax.** Justify each one and measure
it; an extra "locate" round-trip can cost more than the reads it saves. Strip dead
params rather than leaving them in the schema.
- **Tool results return the minimum.** Never echo content the model already has. The
canonical mistake: a write tool that returns the whole edited artifact right after
the model authored it — return `{ success, message }` instead. When you touch a
*shared* write helper (e.g. `finishAppDraftWrite` in `global/core.ts`), re-check
this invariant for **all** the write tools routing through it — the echo has
regressed before via a shared refactor.
## Prompts and tool descriptions are part of the surface
The system prompt and tool descriptions steer behavior as much as the tools
themselves, and are benchmarkable the same way. A description that advertises
truncation makes the model self-limit; the path-conventions block changes where
drafts land. Treat prompt/description edits as real changes and A/B them — a
pure-prompt change is a legitimate, measurable improvement.
+87
View File
@@ -0,0 +1,87 @@
---
name: ai-evals
description: Author and run black-box benchmark cases for the Windmill AI generation modes (flow/app/script/cli/global) in ai_evals/. Use when adding or changing eval cases, or when running before/after benchmarks for AI chat / copilot changes.
---
# AI evals — authoring and running benchmark cases
`ai_evals/` is a black-box benchmark runner for the Windmill AI generation modes:
`flow`, `app`, `script`, `cli`, `global`. It always tests the **current** production
prompts, tools, and guidance in this checkout. Each attempt runs the real production
path, deterministic validation, then LLM judging.
The goal is to test current production guidance with realistic user requests — **not**
to pin one exact implementation shape.
## Running benchmarks
```bash
cd ai_evals
bun install # first time; frontend modes also need `cd frontend && bun install`
bun run cli -- models # list model aliases
bun run cli -- cases global # list cases for a mode
bun run cli -- run global global-test1-script-create --model sonnet
```
Frontend modes (`flow`/`script`/`app`/`global`) route model calls through a Windmill
backend's `/api/w/<ws>/ai/proxy`, so you need **any** reachable backend:
```bash
WMILL_AI_EVAL_BACKEND_URL=http://127.0.0.1:<port> WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests \
bun run cli -- run global <caseIds...> --models sonnet,gpt-5.5,gemini-3.1-pro-preview
```
- **Reuse an existing workspace.** CE builds cap workspaces, so temp-workspace
creation 400s ("reached workspace limit"). Always set
`WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests` (or any existing workspace) to
reuse one. The only side effect of a run is upserting an `f/evals/ai/<provider>`
resource there.
- Provider keys live in `ai_evals/.env` and are auto-loaded by bun. The judge is a
separate Anthropic call (default `claude-sonnet-4-6`) regardless of the model under
test.
## Authoring core rules
1. Write prompts like a real user request.
2. Prefer behavior, inputs, constraints, and outcomes over internal implementation.
3. Keep deterministic validation narrow and hard.
4. Put semantic expectations in `judgeChecklist`.
5. Use `expected` fixtures only when exact structure really matters.
### Prompt writing
Prompts should sound like something a user would naturally ask. Do not write prompts
as if the user knows Windmill internals unless the case explicitly tests a power-user
workflow.
Good:
- "Create a flow that routes support requests based on customer tier."
- "Add a reset button that sets the counter back to 0."
- "Create a flow that reuses the existing greeting script instead of duplicating the logic."
Bad:
- "Use `branchone` with 3 branches and a default branch."
- "Create a `rawscript` step with this exact topology."
- "This is a benchmark harness."
### Deterministic validation
Use deterministic checks only for hard failures: missing required files; unexpected
extra files when the prompt says not to create them; syntax errors; unresolved flow
refs; missing required special modules or suspend config; obvious corruption.
Do **not** encode one preferred implementation. Bad hard checks: exact step topology
for a creation flow; exact branch structure when the prompt only asked for routing;
exact input shape when multiple reasonable shapes are acceptable.
### Judge checklist
Every non-trivial case should have a `judgeChecklist` capturing user-visible behavior
that must be present, important constraints, and key completion criteria — not
low-level implementation details unless truly required.
Good: "the flow calculates the order total with 8% tax"; "the flow reuses the existing
workspace script instead of rewriting the logic". Bad: "uses `branchone`"; "contains a
`rawscript` node".
See `ai_evals/README.md` for the full case format, fields, and fixture details.
+1
View File
@@ -0,0 +1 @@
../../../.agents/skills/ai-chat/SKILL.md
+1
View File
@@ -0,0 +1 @@
../../../.agents/skills/ai-evals/SKILL.md
+1 -1
View File
@@ -28,7 +28,7 @@ ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.25/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.11.24/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
ENV TZ=Etc/UTC
+126
View File
@@ -0,0 +1,126 @@
// Extracts every windmill.dev/docs link referenced in the frontend source and
// verifies none of them 404. Run: `node .github/scripts/check-docs-links.mjs`.
// Used by the check-docs-links GitHub workflow (release / manual trigger only).
import { readdir, readFile } from 'node:fs/promises'
import { join, extname } from 'node:path'
const ROOT = 'frontend/src'
const EXTS = new Set(['.ts', '.js', '.svelte', '.mjs', '.cjs'])
const DOCS_RE = /https?:\/\/(?:www\.)?windmill\.dev\/docs\/[^\s"'`)>\]}]*/g
// `const someBaseUrl = 'https://www.windmill.dev/docs/...'` used later as `${someBaseUrl}/foo`
const BASE_RE = /(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*=\s*['"`](https?:\/\/(?:www\.)?windmill\.dev\/docs\/[^'"`]+)['"`]/g
const CONCURRENCY = 24
const TIMEOUT_MS = 20000
const RETRIES = 2
async function walk(dir) {
const out = []
for (const entry of await readdir(dir, { withFileTypes: true })) {
const p = join(dir, entry.name)
if (entry.isDirectory()) {
if (entry.name === 'node_modules' || entry.name === '.svelte-kit') continue
out.push(...(await walk(p)))
} else if (EXTS.has(extname(entry.name))) {
out.push(p)
}
}
return out
}
// url (no fragment) -> Set of source files it appears in
const urls = new Map()
const unresolved = []
function record(url, file) {
const clean = url
.replace(/\\.*$/, '') // cut at an escape sequence embedded in a string literal (e.g. \n)
.replace(/#.*$/, '') // drop anchor fragment — irrelevant to a 404 check
.replace(/[.,;:'")\]]+$/, '')
if (!clean) return
// A `{`/`${` means the URL is built from an unresolved template/interpolation var.
if (clean.includes('{')) {
unresolved.push(`${clean} (${file})`)
return
}
if (!urls.has(clean)) urls.set(clean, new Set())
urls.get(clean).add(file)
}
for (const file of await walk(ROOT)) {
let content = await readFile(file, 'utf8')
// Inline file-local base-url constants so `${base}/page` template literals resolve.
const bases = []
for (const m of content.matchAll(BASE_RE)) bases.push({ name: m[1], value: m[2], decl: m[0] })
for (const { name, value } of bases) {
content = content.replaceAll('${' + name + '}', value)
}
// Blank each base declaration so a prefix-only base (no index page of its own,
// e.g. .../app_configuration_settings) isn't checked as a standalone link.
// A genuinely bare `${base}` usage was already inlined above, so it's still covered.
for (const { decl } of bases) content = content.replace(decl, '')
for (const m of content.matchAll(DOCS_RE)) record(m[0], file)
}
const allUrls = [...urls.keys()].sort()
console.log(`Found ${allUrls.length} distinct docs links across ${ROOT}`)
if (unresolved.length) {
console.log(`\n⚠️ ${unresolved.length} link(s) built from an unrecognized base URL — skipped (register the base const so they can be checked):`)
for (const u of [...new Set(unresolved)].sort()) console.log(` ${u}`)
}
async function check(url) {
for (let attempt = 0; attempt <= RETRIES; attempt++) {
const ctrl = new AbortController()
const timer = setTimeout(() => ctrl.abort(), TIMEOUT_MS)
try {
let res = await fetch(url, {
method: 'HEAD',
redirect: 'follow',
signal: ctrl.signal,
headers: { 'user-agent': 'windmill-docs-link-check' }
})
// Some hosts reject HEAD — fall back to GET.
if (res.status === 405 || res.status === 501) {
res = await fetch(url, {
method: 'GET',
redirect: 'follow',
signal: ctrl.signal,
headers: { 'user-agent': 'windmill-docs-link-check' }
})
}
clearTimeout(timer)
return { url, status: res.status, ok: res.status < 400 }
} catch (err) {
clearTimeout(timer)
if (attempt === RETRIES) return { url, status: 0, ok: false, error: String(err?.message || err) }
await new Promise((r) => setTimeout(r, 500 * (attempt + 1)))
}
}
}
// Simple concurrency pool.
const results = []
let idx = 0
async function worker() {
while (idx < allUrls.length) {
const url = allUrls[idx++]
results.push(await check(url))
}
}
await Promise.all(Array.from({ length: CONCURRENCY }, worker))
const failures = results.filter((r) => !r.ok)
if (failures.length === 0) {
console.log(`\n✅ All ${allUrls.length} docs links are reachable.`)
process.exit(0)
}
console.log(`\n${failures.length} broken docs link(s):`)
for (const f of failures.sort((a, b) => a.url.localeCompare(b.url))) {
console.log(`\n ${f.url}`)
console.log(` status: ${f.error ? `error (${f.error})` : f.status}`)
for (const file of urls.get(f.url)) console.log(`${file}`)
}
process.exit(1)
+132
View File
@@ -0,0 +1,132 @@
name: AI Agent Integration Tests
# Exercises the AI agent flow path (preview_flow with `aiagent` modules) against
# real LLM providers. Runs only when AI-agent backend code or the tests change,
# because each run makes real (paid) LLM calls. To avoid spending on every commit,
# the PR side triggers only when a PR is marked ready for review (out of draft) —
# not on `synchronize` — plus push to main and manual dispatch.
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "integration_tests/ai_agent_tests/**"
- "backend/windmill-ai/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-worker/src/ai_executor.rs"
- "backend/windmill-worker/src/ai/**"
- "backend/windmill-worker/src/memory_common.rs"
- "backend/windmill-common/src/flow_conversations.rs"
- ".github/workflows/ai-agent-tests.yml"
pull_request:
types: [opened, reopened, ready_for_review]
paths:
- "integration_tests/ai_agent_tests/**"
- "backend/windmill-ai/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-worker/src/ai_executor.rs"
- "backend/windmill-worker/src/ai/**"
- "backend/windmill-worker/src/memory_common.rs"
- "backend/windmill-common/src/flow_conversations.rs"
- ".github/workflows/ai-agent-tests.yml"
concurrency:
group: ai-agent-tests-${{ github.ref }}
cancel-in-progress: true
jobs:
ai_agent_e2e:
# Skip draft PRs; the `opened`/`reopened` types would otherwise fire while
# still a draft. `ready_for_review` always arrives non-draft.
if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
runs-on: ubicloud-standard-16
services:
postgres:
image: postgres:16
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.93.0
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.10
- uses: actions/setup-node@v4
with:
node-version: "20"
- uses: actions/setup-python@v5
with:
python-version: "3.11"
# CE build (no enterprise/license needed for AI agents). `quickjs` powers
# flow input-transform JS eval; `mcp` is required by the deepwiki MCP tool
# test. Bun tool scripts run via the always-on worker (BUN_PATH).
- name: Build Windmill
working-directory: ./backend
env:
SQLX_OFFLINE: true
CARGO_BUILD_JOBS: 12
RUSTFLAGS: ""
run: cargo build --features quickjs,mcp
- name: Start Windmill
working-directory: ./backend
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
BUN_PATH: bun
NODE_BIN_PATH: node
RUST_LOG: info
run: |
mkdir -p ../integration_tests/logs
./target/debug/windmill > ../integration_tests/logs/windmill.log 2>&1 &
echo "Waiting for Windmill to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
echo "Windmill is ready"
break
fi
sleep 2
done
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; tail -50 ../integration_tests/logs/windmill.log; exit 1; }
- name: Run AI agent integration tests
timeout-minutes: 20
working-directory: ./integration_tests/ai_agent_tests
env:
WINDMILL_URL: http://localhost:8000
# Only the providers we have org secrets for. Other providers
# (Azure, Bedrock, OpenRouter) are skipped by conftest when their
# keys are absent — see skip_provider_without_credentials.
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GOOGLE_AI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
run: |
python -m venv .venv
.venv/bin/pip install -r requirements.txt
# The S3/vision-attachment tests need MinIO large-file storage and
# image-capable provider setup; out of scope for this cost-controlled
# smoke. Add MinIO secrets + a storage service to enable them.
.venv/bin/python -m pytest -v \
--ignore=test_user_attachments.py \
--ignore=test_user_images.py \
--ignore=test_image_output.py
- name: Archive Windmill logs
uses: actions/upload-artifact@v4
if: always()
with:
name: ai-agent-tests-windmill-logs
path: integration_tests/logs
+166
View File
@@ -0,0 +1,166 @@
name: AI Evals (global mode)
# Smoke-tests the production global AI chat proxy/frontend execution path via
# the ai_evals harness, one case across one cheap model per provider. Runs only
# when the eval harness or the global chat code change, since each run makes real
# (paid) LLM calls. The backend is built from source purely as the AI proxy the
# harness routes model calls through; the global tools/drafts run in-process in
# the Vitest bridge against production frontend code. To avoid spending on every
# commit, the PR side triggers only when a PR is marked ready for review (out of
# draft) — not on `synchronize` — plus push to main and manual dispatch.
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "ai_evals/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-ai/**"
- "frontend/src/lib/components/copilot/**"
# The eval harness runs production frontend code in-process; these are the
# AI/draft-specific deps outside copilot/ that the global smoke exercises.
- "frontend/src/lib/userDraft.svelte.ts"
- "frontend/src/lib/userDraftDbSyncer.svelte.ts"
- "frontend/src/lib/infer.ts"
- ".github/workflows/ai-evals-test.yml"
pull_request:
types: [opened, reopened, ready_for_review]
paths:
- "ai_evals/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-ai/**"
- "frontend/src/lib/components/copilot/**"
# The eval harness runs production frontend code in-process; these are the
# AI/draft-specific deps outside copilot/ that the global smoke exercises.
- "frontend/src/lib/userDraft.svelte.ts"
- "frontend/src/lib/userDraftDbSyncer.svelte.ts"
- "frontend/src/lib/infer.ts"
- ".github/workflows/ai-evals-test.yml"
concurrency:
group: ai-evals-test-${{ github.ref }}
cancel-in-progress: true
jobs:
ai_evals_global:
# Provider secrets are unavailable to forked and Dependabot PRs.
if: >-
github.event_name != 'pull_request' ||
(
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository &&
github.event.pull_request.user.login != 'dependabot[bot]'
)
runs-on: ubicloud-standard-16
services:
postgres:
image: postgres:16
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.93.0
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.10
- uses: actions/setup-node@v4
with:
# Node 22.19+ is required by the frontend's undici 8.x, which the
# Vitest bridge loads; Node 20 fails with markAsUncloneable.
node-version: "22"
# CE build used only as the AI proxy (login, workspace, provider resource,
# /ai/proxy). No worker execution or MCP needed — global tools/drafts run
# in the Vitest bridge. quickjs matches the standard CE feature set.
- name: Build Windmill (AI proxy)
working-directory: ./backend
env:
SQLX_OFFLINE: true
CARGO_BUILD_JOBS: 12
RUSTFLAGS: ""
run: cargo build --features quickjs
- name: Start Windmill
working-directory: ./backend
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
RUST_LOG: info
run: |
mkdir -p ../ai_evals/logs
./target/debug/windmill > ../ai_evals/logs/windmill.log 2>&1 &
echo "Waiting for Windmill to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
echo "Windmill is ready"
break
fi
sleep 2
done
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; tail -50 ../ai_evals/logs/windmill.log; exit 1; }
- name: Install frontend deps + generate client
working-directory: ./frontend
run: |
npm ci
npm run generate-backend-client
- name: Run global AI evals
timeout-minutes: 20
working-directory: ./ai_evals
env:
WMILL_AI_EVAL_BACKEND_URL: http://localhost:8000
WMILL_AI_EVAL_BACKEND_WORKSPACE: integration-tests
# Anthropic backs the haiku model. Google AI uses GEMINI_API_KEY.
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
run: |
bun install
mkdir -p results
# One cheap model per provider (anthropic/openai/googleai/deepseek).
fail=0
for m in haiku 4o gemini-3-flash-preview deepseek-v4-flash; do
echo "::group::global-test1-script-create ($m)"
if ! bun run cli -- run global global-test1-script-create \
--model "$m" --execution-only --output "$PWD/results/ci-$m.json"; then
echo "$m: harness/proxy errored"
fail=1
echo "::endgroup::"
continue
fi
# The CLI exits 0 when the harness records failed attempts, so gate
# on execution-only pass counts while ignoring model output quality.
if jq -e \
'.attemptCount > 0 and .passedAttempts == .attemptCount' \
"results/ci-$m.json" > /dev/null; then
echo "$m: OK — proxy/frontend execution completed"
else
echo "$m: FAILED proxy/frontend execution"
jq -c '.cases[0].attempts[0].checks' "results/ci-$m.json" || true
fail=1
fi
echo "::endgroup::"
done
[ "$fail" = 0 ] || { echo "ai_evals global smoke failed"; exit 1; }
- name: Archive logs and results
uses: actions/upload-artifact@v4
if: always()
with:
name: ai-evals-global-logs
path: |
ai_evals/logs
ai_evals/results
+4 -2
View File
@@ -58,7 +58,9 @@ jobs:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
# Pin to the Deno version shipped in the runtime image (Dockerfile) so CI
# tests what production runs, instead of floating on the latest v2.x.
deno-version: 2.2.1
- uses: actions/setup-go@v2
with:
@@ -74,7 +76,7 @@ jobs:
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.9.25"
version: "0.11.24"
- uses: shivammathur/setup-php@v2
with:
+4 -2
View File
@@ -50,7 +50,9 @@ jobs:
dotnet-version: "9.0.x"
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
# Pin to the Deno version shipped in the runtime image (Dockerfile) so CI
# tests what production runs, instead of floating on the latest v2.x.
deno-version: 2.2.1
- uses: actions/setup-go@v2
with:
go-version: 1.21.5
@@ -62,7 +64,7 @@ jobs:
node-version: "20"
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.9.25"
version: "0.11.24"
- uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
+23
View File
@@ -0,0 +1,23 @@
name: Check frontend docs links
on:
push:
tags:
- "v*"
workflow_dispatch:
jobs:
check-docs-links:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: |
frontend/src
.github/scripts
- uses: actions/setup-node@v4
with:
node-version: "22.x"
- name: Verify docs links are not 404
run: node .github/scripts/check-docs-links.mjs
@@ -1,83 +0,0 @@
name: Check Organization Membership
on:
workflow_call:
inputs:
commenter:
required: false
type: string
default: ''
description: 'The username to check. Auto-detected from the event context if not provided.'
organization:
required: false
type: string
default: 'windmill-labs'
description: 'The organization to check membership for'
trusted_bot:
required: false
type: string
default: 'windmill-internal-app[bot]'
description: 'The trusted bot username to allow'
secrets:
access_token:
required: true
description: 'The access token to use for org membership check'
outputs:
is_member:
description: 'Whether the user is an organization member or trusted bot'
value: ${{ jobs.check-membership.outputs.is_member }}
jobs:
check-membership:
runs-on: ubicloud-standard-2
outputs:
is_member: ${{ steps.check-membership.outputs.is_member }}
steps:
- name: Determine commenter
id: determine-commenter
run: |
COMMENTER="${{ inputs.commenter }}"
if [[ -z "$COMMENTER" ]]; then
if [[ "${{ github.event_name }}" == "issue_comment" || \
"${{ github.event_name }}" == "pull_request_review_comment" ]]; then
COMMENTER="${{ github.event.comment.user.login }}"
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
COMMENTER="${{ github.event.review.user.login }}"
else
COMMENTER="${{ github.event.issue.user.login }}"
fi
fi
echo "commenter=$COMMENTER" >> $GITHUB_OUTPUT
- name: Check organization membership
id: check-membership
env:
ORG_ACCESS_TOKEN: ${{ secrets.access_token }}
COMMENTER: ${{ steps.determine-commenter.outputs.commenter }}
ORG: ${{ inputs.organization }}
TRUSTED_BOT: ${{ inputs.trusted_bot }}
run: |
# 1. Allow the trusted bot straight away
if [[ "$COMMENTER" == "$TRUSTED_BOT" ]]; then
echo "is_member=true" >> $GITHUB_OUTPUT
exit 0
fi
# 2. Disallow other bots
if [[ "${COMMENTER}" =~ \[bot\]$ ]]; then
echo "is_member=false" >> $GITHUB_OUTPUT
exit 0
fi
# 3. Otherwise check if the user is a member of the organization
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $ORG_ACCESS_TOKEN" \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
if [ "$STATUS" -eq 204 ]; then
echo "is_member=true" >> $GITHUB_OUTPUT
else
echo "is_member=false" >> $GITHUB_OUTPUT
fi
+66
View File
@@ -0,0 +1,66 @@
name: Check Write Access
# Authorizes a user to trigger privileged command workflows (/review, /ai, /plan,
# /updatesqlx, ...). The webhook author_association reports PRIVATE org members as
# CONTRIBUTOR/NONE (only public members show as MEMBER), so command jobs can't gate on
# it alone. This mints the internal GitHub App token — which can see private members —
# and confirms the user is a member or has write access to the repo. The app token is
# minted fresh per run, so unlike the old ORG_ACCESS_TOKEN PAT it never expires.
on:
workflow_call:
inputs:
username:
required: true
type: string
description: 'The user whose access to verify'
trusted_bot:
required: false
type: string
default: 'windmill-internal-app[bot]'
description: 'A bot login that is always authorized'
outputs:
authorized:
description: 'true if the user is the trusted bot, an org member, or has repo write access'
value: ${{ jobs.check.outputs.authorized }}
jobs:
check:
runs-on: ubuntu-latest
outputs:
authorized: ${{ steps.check.outputs.authorized }}
steps:
- name: Mint internal app token
id: app
uses: actions/create-github-app-token@v2
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
owner: ${{ github.repository_owner }}
- name: Resolve authorization
id: check
env:
GH_TOKEN: ${{ steps.app.outputs.token }}
USERNAME: ${{ inputs.username }}
TRUSTED_BOT: ${{ inputs.trusted_bot }}
REPO: ${{ github.repository }}
run: |
if [ "$USERNAME" = "$TRUSTED_BOT" ]; then
echo "authorized=true" >> "$GITHUB_OUTPUT"
exit 0
fi
ORG="${REPO%%/*}"
# Org membership resolves private members too (204 = member, 404 = not).
if gh api "orgs/$ORG/members/$USERNAME" --silent 2>/dev/null; then
echo "authorized=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Fallback: effective repo permission (also covers outside collaborators).
PERM=$(gh api "repos/$REPO/collaborators/$USERNAME/permission" --jq '.permission' 2>/dev/null || echo none)
if [ "$PERM" = "admin" ] || [ "$PERM" = "write" ]; then
echo "authorized=true" >> "$GITHUB_OUTPUT"
else
echo "authorized=false" >> "$GITHUB_OUTPUT"
echo "$USERNAME is neither the trusted bot, an org member, nor a repo writer."
fi
+10 -6
View File
@@ -11,20 +11,24 @@ on:
types: [submitted]
jobs:
check-membership:
# author_association misses private org members; check-access resolves them via the
# internal app token. Both are OR'd below so public members still pass instantly.
check-access:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/plan')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/plan')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/plan')) ||
(github.event_name == 'issues' && contains(github.event.issue.body, '/plan'))
uses: ./.github/workflows/check-org-membership.yml
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
uses: ./.github/workflows/check-write-access.yml
with:
username: ${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}
secrets: inherit
claude-plan-action:
needs: check-membership
needs: [check-access]
if: |
needs.check-membership.outputs.is_member == 'true'
needs.check-access.outputs.authorized == 'true' ||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association)
runs-on: ubicloud-standard-4
timeout-minutes: 20
permissions:
+48 -6
View File
@@ -11,20 +11,24 @@ on:
types: [submitted]
jobs:
check-membership:
# author_association misses private org members; check-access resolves them via the
# internal app token. Both are OR'd below so public members still pass instantly.
check-access:
if: |
(github.event_name == 'issue_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
(github.event_name == 'pull_request_review_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
(github.event_name == 'pull_request_review' && startsWith(github.event.review.body, '/ai') && !startsWith(github.event.review.body, '/ai-fast')) ||
(github.event_name == 'issues' && startsWith(github.event.issue.body, '/ai') && !startsWith(github.event.issue.body, '/ai-fast'))
uses: ./.github/workflows/check-org-membership.yml
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
uses: ./.github/workflows/check-write-access.yml
with:
username: ${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}
secrets: inherit
claude-code-action:
needs: check-membership
needs: [check-access]
if: |
needs.check-membership.outputs.is_member == 'true'
needs.check-access.outputs.authorized == 'true' ||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association)
runs-on: ubicloud-standard-8
permissions:
contents: write
@@ -37,6 +41,44 @@ jobs:
with:
fetch-depth: 1
# Make the EE source (the *_ee.rs files in the companion repo) available so the
# reviewer can see EE-only code (e.g. windmill-queue/src/jobs_ee.rs), not just the
# CE surface. The EE ref is read from the PR head's backend/ee-repo-ref.txt (via the
# API, so it reflects the PR's EE pin regardless of which ref is checked out here).
- name: Check EE access
id: ee
env:
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
run: |
if [ -z "$EE_TOKEN" ] || [ -z "$PR_NUMBER" ]; then
echo "available=false" >> "$GITHUB_OUTPUT"
exit 0
fi
HEAD_SHA=$(gh api "repos/${{ github.repository }}/pulls/$PR_NUMBER" --jq .head.sha)
REF=$(gh api "repos/${{ github.repository }}/contents/backend/ee-repo-ref.txt?ref=$HEAD_SHA" --jq .content | base64 -d | tr -d '[:space:]')
if [ -z "$REF" ]; then
echo "available=false" >> "$GITHUB_OUTPUT"
else
echo "available=true" >> "$GITHUB_OUTPUT"
echo "ee_repo_ref=$REF" >> "$GITHUB_OUTPUT"
fi
- name: Checkout EE repository
if: steps.ee.outputs.available == 'true'
uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ steps.ee.outputs.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 1
- name: Substitute EE code
if: steps.ee.outputs.available == 'true'
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Run Claude PR Action
uses: anthropics/claude-code-action@v1
with:
+8 -16
View File
@@ -32,27 +32,19 @@ concurrency:
cancel-in-progress: true
jobs:
check-membership:
if: github.event_name == 'pull_request'
uses: ./.github/workflows/check-org-membership.yml
with:
commenter: ${{ github.event.pull_request.user.login }}
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
codex-review:
needs: check-membership
runs-on: ubicloud-standard-2
timeout-minutes: 30
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
# access to this repo, so fork==false already enforces write access. Do NOT re-add
# an author_association gate: the pull_request webhook payload reports private org
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
# skips auto-review for every private member.
if: |
always() &&
github.event_name == 'workflow_call' ||
(
needs.check-membership.result == 'skipped' ||
(needs.check-membership.result == 'success' && needs.check-membership.outputs.is_member == 'true')
) &&
(
github.event_name == 'workflow_call' ||
(github.event.pull_request.draft == false && github.event.pull_request.head.repo.fork == false)
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.fork == false
)
permissions:
contents: read
+29 -19
View File
@@ -5,21 +5,22 @@ on:
types: [created]
jobs:
check-membership:
if: >-
github.event.issue.pull_request && (
startsWith(github.event.comment.body, '/updatesqlx') ||
startsWith(github.event.comment.body, '/demo') ||
startsWith(github.event.comment.body, '/eeref') ||
startsWith(github.event.comment.body, '/docs')
)
uses: ./.github/workflows/check-org-membership.yml
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
# /command comments can come from anyone; author_association misses private org
# members, so check-access resolves them via the internal app token. Runs once and is
# OR'd into each job's guard (public members still pass on author_association alone).
check-access:
if: github.event.issue.pull_request != null && startsWith(github.event.comment.body, '/')
uses: ./.github/workflows/check-write-access.yml
with:
username: ${{ github.event.comment.user.login }}
secrets: inherit
update-sqlx:
needs: check-membership
if: needs.check-membership.outputs.is_member == 'true' && startsWith(github.event.comment.body, '/updatesqlx')
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/updatesqlx')
runs-on: ubicloud-standard-8
permissions:
contents: write
@@ -147,8 +148,11 @@ jobs:
})
demo:
needs: check-membership
if: needs.check-membership.outputs.is_member == 'true' && startsWith(github.event.comment.body, '/demo')
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/demo')
runs-on: ubicloud-standard-2
permissions:
contents: read
@@ -227,8 +231,11 @@ jobs:
fi
update-ee-ref:
needs: check-membership
if: needs.check-membership.outputs.is_member == 'true' && startsWith(github.event.comment.body, '/eeref')
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/eeref')
runs-on: ubicloud-standard-2
permissions:
contents: write
@@ -313,8 +320,11 @@ jobs:
})
update-docs:
needs: check-membership
if: needs.check-membership.outputs.is_member == 'true' && startsWith(github.event.comment.body, '/docs')
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/docs')
runs-on: ubicloud-standard-2
permissions:
contents: read
+8 -16
View File
@@ -30,27 +30,19 @@ concurrency:
cancel-in-progress: true
jobs:
check-membership:
if: github.event_name == 'pull_request'
uses: ./.github/workflows/check-org-membership.yml
with:
commenter: ${{ github.event.pull_request.user.login }}
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
pi-review:
needs: check-membership
runs-on: ubicloud-standard-2
timeout-minutes: 30
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
# access to this repo, so fork==false already enforces write access. Do NOT re-add
# an author_association gate: the pull_request webhook payload reports private org
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
# skips auto-review for every private member.
if: |
always() &&
github.event_name == 'workflow_call' ||
(
needs.check-membership.result == 'skipped' ||
(needs.check-membership.result == 'success' && needs.check-membership.outputs.is_member == 'true')
) &&
(
github.event_name == 'workflow_call' ||
(github.event.pull_request.draft == false && github.event.pull_request.head.repo.fork == false)
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.fork == false
)
permissions:
contents: read
+8 -16
View File
@@ -30,26 +30,18 @@ concurrency:
cancel-in-progress: true
jobs:
check-membership:
if: github.event_name == 'pull_request'
uses: ./.github/workflows/check-org-membership.yml
with:
commenter: ${{ github.event.pull_request.user.login }}
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
auto-review:
needs: check-membership
runs-on: ubuntu-latest
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
# access to this repo, so fork==false already enforces write access. Do NOT re-add
# an author_association gate: the pull_request webhook payload reports private org
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
# skips auto-review for every private member.
if: |
always() &&
github.event_name == 'workflow_call' ||
(
needs.check-membership.result == 'skipped' ||
(needs.check-membership.result == 'success' && needs.check-membership.outputs.is_member == 'true')
) &&
(
github.event_name == 'workflow_call' ||
(github.event.pull_request.draft == false || github.event.pull_request.ready_for_review == true)
(github.event.pull_request.draft == false || github.event.pull_request.ready_for_review == true) &&
github.event.pull_request.head.repo.fork == false
)
permissions:
contents: read
+30 -13
View File
@@ -42,16 +42,24 @@ jobs:
;;
esac
check-membership:
needs: parse
# author_association misses private org members; check-access resolves them via the
# internal app token. Both are OR'd so public members still pass instantly.
check-access:
needs: [parse]
if: needs.parse.outputs.command != ''
uses: ./.github/workflows/check-org-membership.yml
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
uses: ./.github/workflows/check-write-access.yml
with:
username: ${{ github.event.comment.user.login }}
secrets: inherit
acknowledge:
needs: [parse, check-membership]
if: needs.parse.outputs.command != '' && needs.check-membership.outputs.is_member == 'true'
needs: [parse, check-access]
if: |
needs.parse.outputs.command != '' &&
(
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
needs.check-access.outputs.authorized == 'true'
)
runs-on: ubuntu-latest
permissions:
issues: write
@@ -68,9 +76,12 @@ jobs:
-f content=eyes >/dev/null
claude:
needs: [parse, check-membership]
needs: [parse, check-access]
if: |
needs.check-membership.outputs.is_member == 'true' &&
(
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
needs.check-access.outputs.authorized == 'true'
) &&
(needs.parse.outputs.command == 'review' || needs.parse.outputs.command == 'claude')
permissions:
contents: read
@@ -86,9 +97,12 @@ jobs:
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
codex:
needs: [parse, check-membership]
needs: [parse, check-access]
if: |
needs.check-membership.outputs.is_member == 'true' &&
(
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
needs.check-access.outputs.authorized == 'true'
) &&
(needs.parse.outputs.command == 'review' || needs.parse.outputs.command == 'codex')
permissions:
contents: read
@@ -105,9 +119,12 @@ jobs:
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
pi:
needs: [parse, check-membership]
needs: [parse, check-access]
if: |
needs.check-membership.outputs.is_member == 'true' &&
(
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
needs.check-access.outputs.authorized == 'true'
) &&
(needs.parse.outputs.command == 'review' || needs.parse.outputs.command == 'pi')
permissions:
contents: read
@@ -0,0 +1,52 @@
name: Refresh docs snapshot
# The backend embeds a vendored docs snapshot (backend/windmill-api/docs_snapshot/*.gz)
# so in-product docs search works with no runtime egress. This job re-fetches it from
# windmill.dev on a schedule and opens a PR when it changed, keeping the embedded docs
# fresh independently of the release cadence (the binary embeds whatever is on the
# source tree at build time, so a merged refresh rides into the next release build).
on:
schedule:
- cron: "0 6 * * 1" # Mondays 06:00 UTC
workflow_dispatch:
jobs:
refresh:
runs-on: ubicloud
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/create-github-app-token@v2
id: app
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
- uses: actions/checkout@v4
with:
token: ${{ steps.app.outputs.token }}
- name: Fetch + re-gzip docs snapshot
run: cd backend/windmill-api/docs_snapshot && ./fetch.sh
- name: Sanity-check the fetched corpus
# curl -f in fetch.sh rejects HTTP errors, but not a valid-but-garbage 200
# (truncated file, error page). Guard against embedding a broken snapshot.
run: |
cd backend/windmill-api/docs_snapshot
test "$(wc -c < llms-full.txt.gz)" -gt 100000
test "$(wc -c < llms.txt.gz)" -gt 1000
pages=$(gzip -dc llms-full.txt.gz | grep -c '^Source:' || true)
echo "pages in snapshot: $pages"
test "${pages:-0}" -ge 200
- uses: peter-evans/create-pull-request@v6
with:
token: ${{ steps.app.outputs.token }}
branch: chore/refresh-docs-snapshot
add-paths: backend/windmill-api/docs_snapshot/*.gz
commit-message: "chore: refresh vendored docs snapshot"
title: "chore: refresh vendored docs snapshot"
body: |
Automated refresh of the embedded docs snapshot
(`backend/windmill-api/docs_snapshot/*.gz`) from windmill.dev.
Review the diff for unexpected churn (a bad upstream docs deploy would
show up as a large drop in pages or content) before merging.
+2
View File
@@ -24,6 +24,8 @@ Open-source platform for internal tools, workflows, API integrations, background
## Dev Environment
- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
- **DuckDB local jobs**: before running DuckDB scripts locally, build the FFI shared library with `cd backend/windmill-duckdb-ffi-internal && ./build_dev.sh`. Re-run it after clean builds or when `backend/target/debug/libwindmill_duckdb_ffi_internal.*` is missing. The bundled DuckDB compile (~2min) is cached in a per-user dir shared across worktrees, so a fresh worktree reuses it and the build is near-instant.
- **Data pipelines (DuckLake) from source**: a plain `cargo run` (even `--features quickjs`) advertises a `duckdb` worker tag but **cannot** execute DuckDB scripts and has **no** working S3 proxy (DuckLake writes 404). Build CE DuckLake with `cargo run --features quickjs,duckdb,parquet,private` (add `,python` for Python scripts, `,enterprise,license` for EE) **and** build the FFI (bullet above). See `backend/CLAUDE.md` → "Running data pipelines (DuckLake) from source" for the exact feature sets and the two feature-gate gotchas.
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/` (port 3000+)
- **DB**: `psql postgres://postgres:changeme@localhost:5432/windmill`
- **Login**: `admin@windmill.dev` / `changeme`
+431
View File
@@ -1,5 +1,436 @@
# Changelog
## [1.753.0](https://github.com/windmill-labs/windmill/compare/v1.752.0...v1.753.0) (2026-07-08)
### Features
* AI chat background jobs tray with detach, approval and preview ([#9982](https://github.com/windmill-labs/windmill/issues/9982)) ([286da00](https://github.com/windmill-labs/windmill/commit/286da005ef2faad1d193640f74f8e96999358707))
* condensed top bar for session preview editors ([#10011](https://github.com/windmill-labs/windmill/issues/10011)) ([b847ca2](https://github.com/windmill-labs/windmill/commit/b847ca2bc7f06f494aa802d4350d6f032ba2bb58))
* **db-health:** add connection sizing guidance ([#10014](https://github.com/windmill-labs/windmill/issues/10014)) ([f28ea9c](https://github.com/windmill-labs/windmill/commit/f28ea9cb991bbda32ed1b7a37a5f1b3552a589a8))
* **sessions:** scoped preview refresh + multi-target live editors + pipeline preview ([#10006](https://github.com/windmill-labs/windmill/issues/10006)) ([32c398f](https://github.com/windmill-labs/windmill/commit/32c398f27de8cd5b1478ef60d247c13705b6b50f))
* shared tab system, universal markdown code blocks, subtle scrollbars ([#10003](https://github.com/windmill-labs/windmill/issues/10003)) ([a00ee51](https://github.com/windmill-labs/windmill/commit/a00ee5196b2c013e9672ab029f5477079ac5da21))
### Bug Fixes
* bump bundled Go CLIs to patched versions to clear image CVEs ([#9996](https://github.com/windmill-labs/windmill/issues/9996)) ([d467161](https://github.com/windmill-labs/windmill/commit/d467161117444d7d9b18def627e90d9622512e02))
* name the offending item when a fork fails on a NUL escape ([#10013](https://github.com/windmill-labs/windmill/issues/10013)) ([99d0047](https://github.com/windmill-labs/windmill/commit/99d00475156def6faad255c4e728923253f9169f))
* preserve worker group tag override on 'Run again' ([#10004](https://github.com/windmill-labs/windmill/issues/10004)) ([c4cb2f3](https://github.com/windmill-labs/windmill/commit/c4cb2f373b6361f0f3ce6b1c8e32a4c010207760))
* replicate external secret backend secrets when forking a workspace ([#10007](https://github.com/windmill-labs/windmill/issues/10007)) ([f65fe7b](https://github.com/windmill-labs/windmill/commit/f65fe7bf585d353f7d88746e947d68e2f351e516))
* session preview editors and picker dropdown overflow ([#10010](https://github.com/windmill-labs/windmill/issues/10010)) ([fb12b23](https://github.com/windmill-labs/windmill/commit/fb12b23e0169ba2cdcf454a27dcf814a2caf26b3))
## [1.752.0](https://github.com/windmill-labs/windmill/compare/v1.751.0...v1.752.0) (2026-07-07)
### Features
* add fork_parent_workspace claim to OIDC tokens for fork workspaces ([#9987](https://github.com/windmill-labs/windmill/issues/9987)) ([7efeae2](https://github.com/windmill-labs/windmill/commit/7efeae26d821b10667b6e3edd220468f6ae48936))
* add SQL migrations for data tables ([#9693](https://github.com/windmill-labs/windmill/issues/9693)) ([e47aeda](https://github.com/windmill-labs/windmill/commit/e47aedac0a4af40dd697d5fc4d54dd3c8efe9ab8))
* **cli:** clarify fork-branch workspace auto-targeting in output ([#9988](https://github.com/windmill-labs/windmill/issues/9988)) ([88c2d0e](https://github.com/windmill-labs/windmill/commit/88c2d0e8e32c218787c01daed80ef41efc39dd11))
* open runs/schedules pages from AI chat in session preview tabs ([#9976](https://github.com/windmill-labs/windmill/issues/9976)) ([4bb82ad](https://github.com/windmill-labs/windmill/commit/4bb82ad6cdb62eae7b69b1054714333e54558632))
* **raw-apps:** runtime-error overlay + AI import-React instruction ([#9966](https://github.com/windmill-labs/windmill/issues/9966)) ([8df613b](https://github.com/windmill-labs/windmill/commit/8df613b4d2f88765f49cc988a894ca323c4ec4f7))
* **sessions:** v2 unified sidebar with family/fork scoping and preview router ([#9816](https://github.com/windmill-labs/windmill/issues/9816)) ([9503190](https://github.com/windmill-labs/windmill/commit/95031903ebe223dc03b49a6bcd3e4ee67cefc4bb))
* smooth bursty AI chat streaming with a typewriter reveal ([#9991](https://github.com/windmill-labs/windmill/issues/9991)) ([a6276b5](https://github.com/windmill-labs/windmill/commit/a6276b590082d06480434a8ea002c335ea1cfb59))
* update base image to debian 13 (trixie) ([#9973](https://github.com/windmill-labs/windmill/issues/9973)) ([c5c1ead](https://github.com/windmill-labs/windmill/commit/c5c1eadeb18e509a98d1e787206c0438417683fc))
### Bug Fixes
* **ai-agent:** align agent_actions_success with agent_actions for mcp and websearch ([#9983](https://github.com/windmill-labs/windmill/issues/9983)) ([87f8d46](https://github.com/windmill-labs/windmill/commit/87f8d46aafffd5e88a336192c51e0c95ff2e6f18))
* **ai:** flow writer builds approval steps as scripts, not identity ([#9985](https://github.com/windmill-labs/windmill/issues/9985)) ([6b01caa](https://github.com/windmill-labs/windmill/commit/6b01caaf26a4f0a08f643db4e22a70e27d0dc554))
* clear old path asset usage when renaming a script ([#9979](https://github.com/windmill-labs/windmill/issues/9979)) ([927b8d0](https://github.com/windmill-labs/windmill/commit/927b8d064f693384978184992b8f8a1cd708e711))
* **cli:** auto-derive cascade triggers in --local pipeline graph ([#9978](https://github.com/windmill-labs/windmill/issues/9978)) ([edfe7b4](https://github.com/windmill-labs/windmill/commit/edfe7b415af6670c5855b7a0b52db4c1f7781964))
* **pipelines:** live materialize/dataset editing — stale graph, phantom drafts, stale Save-all deploys ([#9990](https://github.com/windmill-labs/windmill/issues/9990)) ([f7efb64](https://github.com/windmill-labs/windmill/commit/f7efb646bf1f2e132d1e3ff031b142383ae01c5e))
* **sessions:** auto-rename regression + preview-panel and fork nits ([#9993](https://github.com/windmill-labs/windmill/issues/9993)) ([804178f](https://github.com/windmill-labs/windmill/commit/804178f5e1c904c3f8e35e2b660f33c78964c6eb))
* **sessions:** scope fork session Edits to session-edited items only ([#9989](https://github.com/windmill-labs/windmill/issues/9989)) ([7046dc6](https://github.com/windmill-labs/windmill/commit/7046dc6dfb474ef49313377855bb2bd60294e25a))
## [1.751.0](https://github.com/windmill-labs/windmill/compare/v1.750.0...v1.751.0) (2026-07-06)
### Features
* add cosmetic dev/staging label for dev workspaces ([#9959](https://github.com/windmill-labs/windmill/issues/9959)) ([fd8e64d](https://github.com/windmill-labs/windmill/commit/fd8e64d11fea3ffdb7858100c67cb2e9ca841ed6))
* **auth:** add runtime NO_AUTH mode for authentication bypass ([#9962](https://github.com/windmill-labs/windmill/issues/9962)) ([91e1b08](https://github.com/windmill-labs/windmill/commit/91e1b087a206efb7189824b4184e1f3f4cda7211))
* **frontend:** custom skills — detail modal, batch manage, shared validation ([#9847](https://github.com/windmill-labs/windmill/issues/9847)) ([2e14302](https://github.com/windmill-labs/windmill/commit/2e14302e4abbad595584806bff12548d520fcb58))
* **pipelines:** auto-derive cascade edges from ducklake/s3 reads (+ muted-read badge) ([#9963](https://github.com/windmill-labs/windmill/issues/9963)) ([3dcd394](https://github.com/windmill-labs/windmill/commit/3dcd3949a14199b106506994ea31ca3de7e636b3))
### Bug Fixes
* **ai:** centralize Anthropic Messages API routing across completion paths ([#9960](https://github.com/windmill-labs/windmill/issues/9960)) ([cc2f638](https://github.com/windmill-labs/windmill/commit/cc2f638de6cebeffb9fee1d4835a0cfd565af86c))
* **assets:** responsive layout for small screens ([#9961](https://github.com/windmill-labs/windmill/issues/9961)) ([45946d1](https://github.com/windmill-labs/windmill/commit/45946d1185c0bd07948d4d8454880c2801571f9d))
* **cli:** quote non-identifier property names in resource-type namespace ([#9964](https://github.com/windmill-labs/windmill/issues/9964)) ([dc6b997](https://github.com/windmill-labs/windmill/commit/dc6b99775b550e7433fee8a159c30eaf296500c5))
* critical alerts modal mute toggles no longer close popover or fail to save ([#9969](https://github.com/windmill-labs/windmill/issues/9969)) ([6587019](https://github.com/windmill-labs/windmill/commit/6587019d263374ee5707d258f5d8eec7e73c690d))
* **frontend:** theme-aware code block background in prose markdown ([#9968](https://github.com/windmill-labs/windmill/issues/9968)) ([9821596](https://github.com/windmill-labs/windmill/commit/9821596251cff698958ffbfbd11fffa6a7988c6c))
## [1.750.0](https://github.com/windmill-labs/windmill/compare/v1.749.0...v1.750.0) (2026-07-06)
### Features
* chat-scoped session changes bar + unified diff drawer ([#9762](https://github.com/windmill-labs/windmill/issues/9762)) ([a6c0b37](https://github.com/windmill-labs/windmill/commit/a6c0b3756be78ca3fadc7bad6bae98c0887fd538))
* **pipelines:** require data uploads before running a pipeline ([#9953](https://github.com/windmill-labs/windmill/issues/9953)) ([a1c5b7a](https://github.com/windmill-labs/windmill/commit/a1c5b7aa3ed2841f09f4f148ded5c5b5ef10fd3d))
* **pipelines:** wm_partition macro for grain-agnostic partition filters ([#9950](https://github.com/windmill-labs/windmill/issues/9950)) ([43044c2](https://github.com/windmill-labs/windmill/commit/43044c2e28139b1dbde6844c781a821f8de68f58))
### Bug Fixes
* **ai:** test key routes Azure Foundry Claude models via Anthropic Messages API ([#9956](https://github.com/windmill-labs/windmill/issues/9956)) ([ea19cc9](https://github.com/windmill-labs/windmill/commit/ea19cc9dc459bd259e27f7fcc29601a010c5f8f0))
* **cli:** HD-1 test_edges + HD-2 scd2 _current write in --local pipeline graph ([#9947](https://github.com/windmill-labs/windmill/issues/9947)) ([ad6f23d](https://github.com/windmill-labs/windmill/commit/ad6f23d6bfcf1056bcb6d8c6b552114e88177328))
* **pipelines:** make node & pipeline-level run affordances always visible ([#9948](https://github.com/windmill-labs/windmill/issues/9948)) ([6eabb96](https://github.com/windmill-labs/windmill/commit/6eabb96ae78fb966f9916f907bb693d569b04c0b))
* read chat drafts via own-draft route so drawer-kind drafts deploy ([#9913](https://github.com/windmill-labs/windmill/issues/9913)) ([056ebdb](https://github.com/windmill-labs/windmill/commit/056ebdb03543a93094c80ca354c117236cd8d6c8))
* resolve extensionless bun relative imports on windows loader ([#9949](https://github.com/windmill-labs/windmill/issues/9949)) ([bf96621](https://github.com/windmill-labs/windmill/commit/bf9662172ad7e0ff53d39adc338fd7886672c8f9))
## [1.749.0](https://github.com/windmill-labs/windmill/compare/v1.748.0...v1.749.0) (2026-07-05)
### Features
* **pipelines:** mid-DAG selective execution (dbt `model+`) for pipeline runs ([#9945](https://github.com/windmill-labs/windmill/issues/9945)) ([2d3a773](https://github.com/windmill-labs/windmill/commit/2d3a77344104a587548f23f1b614ceffd52a5778))
* **pipelines:** partition run-arg picker + first-run setup signpost ([#9943](https://github.com/windmill-labs/windmill/issues/9943)) ([475b072](https://github.com/windmill-labs/windmill/commit/475b072987b33d50111f5251a5f69f4245f930ae))
* **pipelines:** self-teaching custom data_test errors + scaffold ([#9937](https://github.com/windmill-labs/windmill/issues/9937)) ([0ad174f](https://github.com/windmill-labs/windmill/commit/0ad174fa490e17eeb26280b5bdfd62956dfed9ff))
### Bug Fixes
* **cli:** macro-library parity in --local pipeline graph + read-only run --dry-run ([#9942](https://github.com/windmill-labs/windmill/issues/9942)) ([e3f4303](https://github.com/windmill-labs/windmill/commit/e3f43033cafcdb5df253aeb55ce93e599b2584d2))
* **datatable:** self-teaching error for unresolved datatable:// references ([#9941](https://github.com/windmill-labs/windmill/issues/9941)) ([55451db](https://github.com/windmill-labs/windmill/commit/55451db009e3060c21948ece2c97e102a3c9b171))
* **object-storage:** remove 20-file bucket-browser listing cap in CE ([#9935](https://github.com/windmill-labs/windmill/issues/9935)) ([22452ce](https://github.com/windmill-labs/windmill/commit/22452ce54034a9bea8f7d48946818fd148b938c0))
* **pipelines:** dedup guard for keyed merge + deploy-time SCD2 validation ([#9936](https://github.com/windmill-labs/windmill/issues/9936)) ([52ce805](https://github.com/windmill-labs/windmill/commit/52ce805f619747af4f998cde7819a164c754205a))
* **pipelines:** link SCD2 &lt;dim&gt;_current view to its producer across all graph surfaces ([#9933](https://github.com/windmill-labs/windmill/issues/9933)) ([574d3ac](https://github.com/windmill-labs/windmill/commit/574d3ac9ff5015b5d3f53040c9d4dfbfd161a076))
* **pipelines:** order data_test relationships refs before the tested script in a cascade ([#9934](https://github.com/windmill-labs/windmill/issues/9934)) ([46be39d](https://github.com/windmill-labs/windmill/commit/46be39dfb7fbfb2b70e61819d6065b45810c41c9))
* **pipelines:** pipeline-level run control, tables label, data-test rollback + fork badges ([#9944](https://github.com/windmill-labs/windmill/issues/9944)) ([6ae8dd3](https://github.com/windmill-labs/windmill/commit/6ae8dd37b1de930ab17344cebf7c28385c6cfdba))
* rebuild windows bun loader main.ts filter from forward-slash cdir ([#9946](https://github.com/windmill-labs/windmill/issues/9946)) ([a582e04](https://github.com/windmill-labs/windmill/commit/a582e04bf40cf685f88bceaf88e3d24bde3d420a))
## [1.748.0](https://github.com/windmill-labs/windmill/compare/v1.747.0...v1.748.0) (2026-07-05)
### Features
* **ai-agent:** support reasoning effort in AI agent workflow steps ([#9886](https://github.com/windmill-labs/windmill/issues/9886)) ([a368d49](https://github.com/windmill-labs/windmill/commit/a368d49bd8786a2dca6771f2051f1d44d1b2363d))
* **ducklake:** scheduled lake maintenance (expiry, compaction, orphan cleanup) ([#9916](https://github.com/windmill-labs/windmill/issues/9916)) ([3352150](https://github.com/windmill-labs/windmill/commit/33521505dbc34f22b575d21fda1cc76d698a8840))
* **pipelines:** asset freshness — fresh/stale badge (CE) + watchdog (EE) ([#9909](https://github.com/windmill-labs/windmill/issues/9909)) ([5d7fb6d](https://github.com/windmill-labs/windmill/commit/5d7fb6deca3e02e89d77e5d3856483beb8b8bfeb))
* **pipelines:** capture violating-row samples for data tests ([#9919](https://github.com/windmill-labs/windmill/issues/9919)) ([d4b4374](https://github.com/windmill-labs/windmill/commit/d4b4374de8f8a7875b050c16d1236fcd0355812b))
* **pipelines:** fork data environments for ducklake materialization (dev data) ([#9915](https://github.com/windmill-labs/windmill/issues/9915)) ([39eb9de](https://github.com/windmill-labs/windmill/commit/39eb9de1bce400109c130a081807e40e995ae068))
* **pipelines:** on_schema_change write guardrails + data_test deploy validation ([#9930](https://github.com/windmill-labs/windmill/issues/9930)) ([377c02e](https://github.com/windmill-labs/windmill/commit/377c02ec47389e64b7ef5cbbae0de05df648266d))
* **pipelines:** record upstream snapshot ids on cascade-dispatched jobs ([#9910](https://github.com/windmill-labs/windmill/issues/9910)) ([af36498](https://github.com/windmill-labs/windmill/commit/af36498432e643108308e1c03b5d986d0f0f8888))
* **pipelines:** schema contracts — save-time consumer checks vs captured schemas ([#9917](https://github.com/windmill-labs/windmill/issues/9917)) ([42e11c6](https://github.com/windmill-labs/windmill/commit/42e11c6570b62ffaa86598438fa8ddf462c4035f))
* **pipeline:** write-audit-publish for materialization data tests ([#9911](https://github.com/windmill-labs/windmill/issues/9911)) ([dce247c](https://github.com/windmill-labs/windmill/commit/dce247c6d2678a2c95bd728027e17ae3965638e2))
* **sdk:** enforce s3:// URIs for string S3 params + ingestion (EL) docs ([#9912](https://github.com/windmill-labs/windmill/issues/9912)) ([5ad2de9](https://github.com/windmill-labs/windmill/commit/5ad2de91a26b312bf27124ceca16ef331621bde8))
### Bug Fixes
* **cli:** pipeline + workspace UX batch (init/bind stub, run errors, macro libs, lock-job report, upgrade errors) ([#9929](https://github.com/windmill-labs/windmill/issues/9929)) ([28a6b08](https://github.com/windmill-labs/windmill/commit/28a6b086c842105298f236baa0a61868f71a5eb1))
* **cli:** publish all windmill-parser-wasm-* deps so local pipeline graph keeps write edges ([#9926](https://github.com/windmill-labs/windmill/issues/9926)) ([744a759](https://github.com/windmill-labs/windmill/commit/744a7597edaf3ca9a7fd2b21a34fb33457913a64))
* **pipelines:** activity-axis label clarity + select failed node on cascade failure ([#9931](https://github.com/windmill-labs/windmill/issues/9931)) ([5769b60](https://github.com/windmill-labs/windmill/commit/5769b6036cf14b0cb424c5b3d9d878c600a5652e))
## [1.747.0](https://github.com/windmill-labs/windmill/compare/v1.746.0...v1.747.0) (2026-07-03)
### Features
* **frontend:** add federatedTokenFile field to instance object storage Azure config ([#9904](https://github.com/windmill-labs/windmill/issues/9904)) ([ae85d27](https://github.com/windmill-labs/windmill/commit/ae85d274371a24c5badb6081f00deeb409123252))
### Bug Fixes
* **ai:** route Azure Foundry Claude models via Anthropic Messages API ([#9908](https://github.com/windmill-labs/windmill/issues/9908)) ([d600c7e](https://github.com/windmill-labs/windmill/commit/d600c7ecfe305533798e82e8d05e5f2f297f9b54))
* **forks:** clone only the current raw-app bundle, via server-side copy ([#9899](https://github.com/windmill-labs/windmill/issues/9899)) ([5c521d8](https://github.com/windmill-labs/windmill/commit/5c521d808a2b5d6d6bb7cf3da17fb2addc53fdf4))
* **kafka:** set https.ca.location=probe for OAUTHBEARER OIDC token endpoint ([#9897](https://github.com/windmill-labs/windmill/issues/9897)) ([1b6065f](https://github.com/windmill-labs/windmill/commit/1b6065fa9201fd548c4b2ef199f1009200645929))
* prevent truncated tool call args from bricking AI chat sessions ([#9902](https://github.com/windmill-labs/windmill/issues/9902)) ([4ba17d0](https://github.com/windmill-labs/windmill/commit/4ba17d0f9cd70489f89c84f982a0c8f0062fed1a))
* strip NUL characters from app values at save time ([#9903](https://github.com/windmill-labs/windmill/issues/9903)) ([3ec1f16](https://github.com/windmill-labs/windmill/commit/3ec1f164be9c8c6c40e003188ce593a963c65a43))
## [1.746.0](https://github.com/windmill-labs/windmill/compare/v1.745.0...v1.746.0) (2026-07-02)
### Features
* **ai:** add Azure AI Foundry as a native AI provider ([#9879](https://github.com/windmill-labs/windmill/issues/9879)) ([d9b080f](https://github.com/windmill-labs/windmill/commit/d9b080f57fa0be144cefa773d39742c45b40f043))
* **frontend:** group compare & deploy items by folder ([#9880](https://github.com/windmill-labs/windmill/issues/9880)) ([7b04820](https://github.com/windmill-labs/windmill/commit/7b04820f8ef8c7f02f79dd4239a877f667d23e6a))
* **frontend:** pipelines index page and sql editor hint ([#9881](https://github.com/windmill-labs/windmill/issues/9881)) ([20351a6](https://github.com/windmill-labs/windmill/commit/20351a6b4c262184c5f815eeb5de007ab1eaf4a0))
* **pipeline:** backfill a range of partitions from the asset drawer ([#9885](https://github.com/windmill-labs/windmill/issues/9885)) ([53bbb92](https://github.com/windmill-labs/windmill/commit/53bbb92953178eb6d0017818ef870f3cb2399dfd))
* **pipelines:** workspace duckdb macro libraries (// macros / // use) ([#9890](https://github.com/windmill-labs/windmill/issues/9890)) ([84141ad](https://github.com/windmill-labs/windmill/commit/84141add1ddf35c2573e3c213366ce7c5f1f2258))
* **s3:** replace CE 50MB upload cap with 10GiB workspace storage quota ([#9874](https://github.com/windmill-labs/windmill/issues/9874)) ([af01e90](https://github.com/windmill-labs/windmill/commit/af01e90b5c65d1b1cfacf4433f8cff7effe73768))
* support workspace forks on cloud using parent workspace limits ([#9864](https://github.com/windmill-labs/windmill/issues/9864)) ([7c7d747](https://github.com/windmill-labs/windmill/commit/7c7d7474cc86a4052272032f281cc4d7a85db37b))
### Bug Fixes
* **duckdb:** auto-declare partition arg for `// partitioned` scripts ([#9878](https://github.com/windmill-labs/windmill/issues/9878)) ([b883adb](https://github.com/windmill-labs/windmill/commit/b883adbc0011073da592dc5b39e1b79db492c83c))
* **frontend:** home New submenus fall back below, hugging the right edge ([#9894](https://github.com/windmill-labs/windmill/issues/9894)) ([186ac49](https://github.com/windmill-labs/windmill/commit/186ac4933b79aed57fce23ebcf3b525fcfd1c474))
* **frontend:** show inline workspace name editor on general settings (Fixes GIT-911) ([#9892](https://github.com/windmill-labs/windmill/issues/9892)) ([a49c087](https://github.com/windmill-labs/windmill/commit/a49c0871d7ab2aaf78a7713b8a786ead937434da))
* **frontend:** stack cron field and cron builder button on narrow screens ([#9871](https://github.com/windmill-labs/windmill/issues/9871)) ([7989795](https://github.com/windmill-labs/windmill/commit/79897950e7646b00d92a28a009174d91c705b251))
* invalidate bun bundle cache on transitive relative-import changes ([#9891](https://github.com/windmill-labs/windmill/issues/9891)) ([d15033c](https://github.com/windmill-labs/windmill/commit/d15033cde6a474b548ebbaf18ff02223fc21f701))
* make SMTP username and password optional in frontend validation ([#9895](https://github.com/windmill-labs/windmill/issues/9895)) ([37bb574](https://github.com/windmill-labs/windmill/commit/37bb57474e8336823bb31527f2a708ef41cd39c4))
* **parsers:** infer py s3 assets from S3Object constructor and dict forms ([#9877](https://github.com/windmill-labs/windmill/issues/9877)) ([659642e](https://github.com/windmill-labs/windmill/commit/659642e4889361f86e8addb038cda62fc3471006))
* pipeline dogfooding fixes — SCD2 data-test scope, --partition, s3object upload binding ([#9875](https://github.com/windmill-labs/windmill/issues/9875)) ([d65f58c](https://github.com/windmill-labs/windmill/commit/d65f58c388d88fff71cda22dfa21aecdae70c450))
* polish pipeline graph view (layout, viewport, minimap, lineage, timestamps) ([#9883](https://github.com/windmill-labs/windmill/issues/9883)) ([b92a86b](https://github.com/windmill-labs/windmill/commit/b92a86b8b3a60b877540c3a7f0ffefe36ccbb053))
* stale AI chat context picker after workspace item changes ([#9893](https://github.com/windmill-labs/windmill/issues/9893)) ([5af91a6](https://github.com/windmill-labs/windmill/commit/5af91a677cad88faccba702e3556fc4fb7b6e640))
* **triggers:** retry transient websocket connect failures before disabling ([#9887](https://github.com/windmill-labs/windmill/issues/9887)) ([7894507](https://github.com/windmill-labs/windmill/commit/789450731b0a3c8dffa336f7bfc3f3de528c09fb))
## [1.745.0](https://github.com/windmill-labs/windmill/compare/v1.744.0...v1.745.0) (2026-07-01)
### Features
* **forks:** partial-visibility deploy + surface hidden items ([#9868](https://github.com/windmill-labs/windmill/issues/9868)) ([20cd1a0](https://github.com/windmill-labs/windmill/commit/20cd1a02d582c0715bedacce52cc5c1e1e8d70ca))
* **frontend:** add zoom and download to Mermaid graphs ([#9859](https://github.com/windmill-labs/windmill/issues/9859)) ([289017b](https://github.com/windmill-labs/windmill/commit/289017bcb28c049c8258b2ffd7da0ec3e6ef120b))
* use derived username instead of email for non-member superadmins ([#9857](https://github.com/windmill-labs/windmill/issues/9857)) ([76a9523](https://github.com/windmill-labs/windmill/commit/76a95230095ca3f43c9dc9eecde0e9de6520242f))
### Bug Fixes
* **cli:** correct misleading delete-fork command description ([#9870](https://github.com/windmill-labs/windmill/issues/9870)) ([a73b14d](https://github.com/windmill-labs/windmill/commit/a73b14d902d759226d0af2f2faf9bdd6588e358c))
* **folders:** allow dots and at-signs in folder owner validation ([#9856](https://github.com/windmill-labs/windmill/issues/9856)) ([383c705](https://github.com/windmill-labs/windmill/commit/383c70523bf81c5c07784a4379ef6b1c93ff86e5))
* **forks:** require admin of both sides for the compare visibility guard ([#9869](https://github.com/windmill-labs/windmill/issues/9869)) ([7363d2c](https://github.com/windmill-labs/windmill/commit/7363d2c217cb04391f03b2f9958be70a9d0b5325))
* **forks:** reset diff tally on trigger delete + guard compare visibility for admins ([#9866](https://github.com/windmill-labs/windmill/issues/9866)) ([6a6f129](https://github.com/windmill-labs/windmill/commit/6a6f12960e29c314d11ad541519c71412f42567b))
* **jobs:** give flow dynselect a path and its worker tag, like scripts ([#9867](https://github.com/windmill-labs/windmill/issues/9867)) ([1a9debb](https://github.com/windmill-labs/windmill/commit/1a9debb689f756f38db085d1360c8fe7691ada48))
* **offboarding:** make global reassignment per-workspace and optional ([#9863](https://github.com/windmill-labs/windmill/issues/9863)) ([3586164](https://github.com/windmill-labs/windmill/commit/35861641f807a02b5c205608fb592e20ee7cad7f))
## [1.744.0](https://github.com/windmill-labs/windmill/compare/v1.743.0...v1.744.0) (2026-07-01)
### Features
* add copy-to-clipboard button to rendered Mermaid diagrams in AI chat ([#9838](https://github.com/windmill-labs/windmill/issues/9838)) ([a27e814](https://github.com/windmill-labs/windmill/commit/a27e814a03c615259381eaf684aa90d56569b0af))
* add dev workspaces paired with a lockable prod workspace ([#9793](https://github.com/windmill-labs/windmill/issues/9793)) ([b4b0c6a](https://github.com/windmill-labs/windmill/commit/b4b0c6a93e52152251fadefe319773faf42549b2))
* **ansible:** support repo-provided ansible.cfg in delegate_to_git_repo ([#9851](https://github.com/windmill-labs/windmill/issues/9851)) ([68bf0da](https://github.com/windmill-labs/windmill/commit/68bf0daf5815307cda6ce23214dd5159b6aa33b4))
* **licensing:** enforce offline license seat cap ([#9845](https://github.com/windmill-labs/windmill/issues/9845)) ([83f3d7f](https://github.com/windmill-labs/windmill/commit/83f3d7f910b331c09f60cc9ff556728afa3dec07))
* **object-store:** make GCS service account key optional for Workload Identity ([#9842](https://github.com/windmill-labs/windmill/issues/9842)) ([83ed011](https://github.com/windmill-labs/windmill/commit/83ed011e264f20ffa66a7bf933f2fe3615cf6b67))
* **pipeline:** local development for data pipelines (CLI --local + pipeline dev preview) ([#9840](https://github.com/windmill-labs/windmill/issues/9840)) ([74f579e](https://github.com/windmill-labs/windmill/commit/74f579e6d9ef08e74460f904a4c22ed9d6a3b5b0))
* **pipelines:** add managed SCD2 history materialize strategy ([#9850](https://github.com/windmill-labs/windmill/issues/9850)) ([5a66127](https://github.com/windmill-labs/windmill/commit/5a661279a3690e2393b9b16996f5d1a5a509259c))
### Bug Fixes
* **ai-chat:** replay anthropic turns verbatim to keep thinking valid ([#9843](https://github.com/windmill-labs/windmill/issues/9843)) ([a37a144](https://github.com/windmill-labs/windmill/commit/a37a144e81cf6b3de935688a617e9d0e1756004a))
* grant dispatch_event table to windmill roles ([#9852](https://github.com/windmill-labs/windmill/issues/9852)) ([f05b50d](https://github.com/windmill-labs/windmill/commit/f05b50d29ac2fdbb808a97057fb92c8e425b4a2f))
* grant workspace_diff, materialized_partition, debounce_stale_data to windmill roles ([#9853](https://github.com/windmill-labs/windmill/issues/9853)) ([293647d](https://github.com/windmill-labs/windmill/commit/293647de4c13cb8468cbd81ff1924cba90e164b4))
* honor verify-ca/verify-full sslmode for postgres connections ([#9835](https://github.com/windmill-labs/windmill/issues/9835)) ([bf6be96](https://github.com/windmill-labs/windmill/commit/bf6be967fa8c74e1299cf63f813c1cfa34b97f3e))
* **mcp:** stop double-escaping string query params in build_query_string ([#9855](https://github.com/windmill-labs/windmill/issues/9855)) ([1c46f89](https://github.com/windmill-labs/windmill/commit/1c46f899ca03edf62053f4f14d65b4eabff4255d))
* **s3_proxy:** preserve URL-encoding on Hive-partition proxy writes ([#9848](https://github.com/windmill-labs/windmill/issues/9848)) ([6b79bdd](https://github.com/windmill-labs/windmill/commit/6b79bddd42fe55f891c17cb71a7e36ee31337bac))
* validate workspace name length (max 50 chars) on create and fork ([#9854](https://github.com/windmill-labs/windmill/issues/9854)) ([b52972d](https://github.com/windmill-labs/windmill/commit/b52972d0de89004e98d18241d238ca028e4eecba))
## [1.743.0](https://github.com/windmill-labs/windmill/compare/v1.742.0...v1.743.0) (2026-06-29)
### Features
* **home:** redesign create-new popover and home header ([#9827](https://github.com/windmill-labs/windmill/issues/9827)) ([2493eaf](https://github.com/windmill-labs/windmill/commit/2493eaf031f30072637a297398674e761f039005))
* **pipeline:** AI-chat data-pipeline editor (route + in-session) + home surfacing ([#9805](https://github.com/windmill-labs/windmill/issues/9805)) ([c910278](https://github.com/windmill-labs/windmill/commit/c91027824be1f1f49cdd14148baf6aad092a1dd0))
### Bug Fixes
* **gcp:** require token verification for authenticated push delivery ([#9834](https://github.com/windmill-labs/windmill/issues/9834)) ([9b65161](https://github.com/windmill-labs/windmill/commit/9b65161c643bf3f120d2ebd82f786c17233a971b))
## [1.742.0](https://github.com/windmill-labs/windmill/compare/v1.741.0...v1.742.0) (2026-06-28)
### Features
* **apps:** add labels input to app editor deploy drawer ([#9828](https://github.com/windmill-labs/windmill/issues/9828)) ([da45e69](https://github.com/windmill-labs/windmill/commit/da45e699c8aefeede172c90769ef4f4b182fec0c))
* column-level lineage for DuckLake pipelines (SQL-AST inferred + traceable) ([#9814](https://github.com/windmill-labs/windmill/issues/9814)) ([003a262](https://github.com/windmill-labs/windmill/commit/003a262a4e9d6c2a63ada01aa8429aea1fbb6031))
### Bug Fixes
* **audit:** don't read pg_authid from an elevated context in S3 export migration ([#9832](https://github.com/windmill-labs/windmill/issues/9832)) ([75ba81b](https://github.com/windmill-labs/windmill/commit/75ba81b2d27fb0722095780312064cb93d20287e))
* close unauthenticated DAP debugger program-mode launch bypass ([#9829](https://github.com/windmill-labs/windmill/issues/9829)) ([c0768de](https://github.com/windmill-labs/windmill/commit/c0768de0acdf63eaba5fb97d04bfc64f2f03b93d))
* redeploy older app version from deployment history ([#9826](https://github.com/windmill-labs/windmill/issues/9826)) ([c479afa](https://github.com/windmill-labs/windmill/commit/c479afab8ebceccbee050e923dc5c27a6712ea62))
## [1.741.0](https://github.com/windmill-labs/windmill/compare/v1.740.0...v1.741.0) (2026-06-26)
### Features
* **ai-chat:** add create_folder tool to global chat ([#9819](https://github.com/windmill-labs/windmill/issues/9819)) ([44c25de](https://github.com/windmill-labs/windmill/commit/44c25de418612ab98341adb15d5671222b54367e))
* **ai-chat:** hint /compact in context usage tooltip ([#9777](https://github.com/windmill-labs/windmill/issues/9777)) ([aadfb62](https://github.com/windmill-labs/windmill/commit/aadfb620c0b7dcd7e94367b761875e14ef9abe69))
* **ai-chat:** let global chat edit the user's personal instructions ([#9771](https://github.com/windmill-labs/windmill/issues/9771)) ([3be2752](https://github.com/windmill-labs/windmill/commit/3be27521b05de33e48582e80c6651071f889f048))
* **ai-chat:** surface raw apps in the @-mention context picker ([#9800](https://github.com/windmill-labs/windmill/issues/9800)) ([1602244](https://github.com/windmill-labs/windmill/commit/16022447c7b445be753b9545b10b4c67da0893d5))
* capture managed-materialize output schema as asset metadata ([#2](https://github.com/windmill-labs/windmill/issues/2)a) ([#9812](https://github.com/windmill-labs/windmill/issues/9812)) ([ade74b2](https://github.com/windmill-labs/windmill/commit/ade74b297f6a03441e700a20ffc2d7291c8a85fd))
* **sdk:** allow overriding worker tag when running jobs (WIN-2105) ([#9807](https://github.com/windmill-labs/windmill/issues/9807)) ([52fc7bf](https://github.com/windmill-labs/windmill/commit/52fc7bf94cf3f87f68d9dba9884944d87e7d5d57))
### Bug Fixes
* apply step timeout to 'Test this step' preview ([#9810](https://github.com/windmill-labs/windmill/issues/9810)) ([d04062b](https://github.com/windmill-labs/windmill/commit/d04062bff58c9c4c79ce542a4321e71bcbcf0e98))
* **flows:** reject corrupt step paths at deploy + atomic cache writes ([#9751](https://github.com/windmill-labs/windmill/issues/9751)) ([#9813](https://github.com/windmill-labs/windmill/issues/9813)) ([3cda447](https://github.com/windmill-labs/windmill/commit/3cda44762148bcd2ee5c0ea821db884950376ead))
* **frontend:** clarify instance data table unavailable on cloud ([#9806](https://github.com/windmill-labs/windmill/issues/9806)) ([c3e8c78](https://github.com/windmill-labs/windmill/commit/c3e8c789ac05c9c28991d9ab6f2358f61fa87971))
* hide GCS service account key behind a reveal in object storage settings ([#9815](https://github.com/windmill-labs/windmill/issues/9815)) ([0ec5061](https://github.com/windmill-labs/windmill/commit/0ec5061270749ed078e01f5a4bc7397a1755ca32))
* ping job during volume setup to prevent false zombie restarts ([#9803](https://github.com/windmill-labs/windmill/issues/9803)) ([43bb676](https://github.com/windmill-labs/windmill/commit/43bb676dc5652cb06fe1414b8d3aacf295bae36b))
* skipped suspend step no longer parks the flow forever ([#9821](https://github.com/windmill-labs/windmill/issues/9821)) ([40110bc](https://github.com/windmill-labs/windmill/commit/40110bc7158bc42c3d84bd4637a12b82fcd72a9a))
### Performance Improvements
* **audit:** re-anchor S3 audit export on enable + opt-in backfill ([#9818](https://github.com/windmill-labs/windmill/issues/9818)) ([577ceee](https://github.com/windmill-labs/windmill/commit/577ceeee8679f054c6898d1a7889df30ab830f8f))
## [1.740.0](https://github.com/windmill-labs/windmill/compare/v1.739.0...v1.740.0) (2026-06-25)
### Features
* **api:** add structured endpoint for flow logs ([#9797](https://github.com/windmill-labs/windmill/issues/9797)) ([ba768fe](https://github.com/windmill-labs/windmill/commit/ba768fee888682cb50142d6e76c0422c40307f46))
* bounded-cascade selective execution for pipelines (UI + CLI) ([#9695](https://github.com/windmill-labs/windmill/issues/9695)) ([248540a](https://github.com/windmill-labs/windmill/commit/248540ac4d6e4ee9ee7c3e6f2cc822c63cc6426e))
* data tests for ducklake pipeline materialization ([#9708](https://github.com/windmill-labs/windmill/issues/9708)) ([f6998ec](https://github.com/windmill-labs/windmill/commit/f6998ec54cba2507703790bf33427e7567d42c4b))
* detect and guard against deploying stale drafts ([#9768](https://github.com/windmill-labs/windmill/issues/9768)) ([d865518](https://github.com/windmill-labs/windmill/commit/d8655189347f58df9d17e83dc55798baf7964279))
* ducklake time-travel UX (snapshot history + AT VERSION reads) ([#9709](https://github.com/windmill-labs/windmill/issues/9709)) ([d131d75](https://github.com/windmill-labs/windmill/commit/d131d754e1fc9674abf5de383d2bc93596df9bd1))
* self-host docs search for chat, mcp, cli; drop inkeep ([#9772](https://github.com/windmill-labs/windmill/issues/9772)) ([9d61e4e](https://github.com/windmill-labs/windmill/commit/9d61e4e59e4101de84217f7c7846f1aa94e84d89))
### Bug Fixes
* allow hyphens in postgresql database name validation ([#9782](https://github.com/windmill-labs/windmill/issues/9782)) ([170cd79](https://github.com/windmill-labs/windmill/commit/170cd79aaf92152fc3c0f675f155853c7f0e5b25))
* **debounce:** never supersede a running debounce survivor ([#9780](https://github.com/windmill-labs/windmill/issues/9780)) ([5549bdc](https://github.com/windmill-labs/windmill/commit/5549bdc67a5559a764616c44b1018543bc0568fe))
* decrypt secret variables via external backend in common resolvers ([#9784](https://github.com/windmill-labs/windmill/issues/9784)) ([cd42c6c](https://github.com/windmill-labs/windmill/commit/cd42c6ca18261328055554788932c3fe876a4a5b))
* enforce containment of python module dir for preview jobs ([#9704](https://github.com/windmill-labs/windmill/issues/9704)) ([88fca6a](https://github.com/windmill-labs/windmill/commit/88fca6a8c130b9e3b0f0cd410e422d4e074fc11f))
* **frontend:** apply script editor timeout to preview/Test runs ([#9794](https://github.com/windmill-labs/windmill/issues/9794)) ([6664ce6](https://github.com/windmill-labs/windmill/commit/6664ce6dc0c5fbc283303148de06d6bb85e4acf7))
* **frontend:** nested-loop "Test this step" resolves iter to innermost loop ([#9778](https://github.com/windmill-labs/windmill/issues/9778)) ([74ebfc6](https://github.com/windmill-labs/windmill/commit/74ebfc67f069047875db738926865bd4bd6fe9e9))
* opt out of Deno minimum-dependency-age for private npm registries ([#9802](https://github.com/windmill-labs/windmill/issues/9802)) ([b28f974](https://github.com/windmill-labs/windmill/commit/b28f974e5069f635419d9ea56fad6a0e417894e8))
* pass SSL cert env vars to `uv python install` ([#9790](https://github.com/windmill-labs/windmill/issues/9790)) ([962758c](https://github.com/windmill-labs/windmill/commit/962758c02de5f6d962c681fe9c39769b99429e8d))
* **python:** re-verify wheel RECORD on local cache reuse (once per worker) ([#9775](https://github.com/windmill-labs/windmill/issues/9775)) ([6c71c33](https://github.com/windmill-labs/windmill/commit/6c71c33470e3ea547f3b994db829eb4d04882443))
* **python:** serialize concurrent installs into shared wheel cache dir ([#9787](https://github.com/windmill-labs/windmill/issues/9787)) ([11d83ab](https://github.com/windmill-labs/windmill/commit/11d83ab1ec559be5d3263010228e6db65358e04b))
* re-pin stale-draft fork base when restoring an app deployment ([#9792](https://github.com/windmill-labs/windmill/issues/9792)) ([b9711e5](https://github.com/windmill-labs/windmill/commit/b9711e5ace8585315a1c2b85bb25ac8dd7832d6f))
* restore libargon2-1 for PHP runtime in server image ([#9795](https://github.com/windmill-labs/windmill/issues/9795)) ([e9cb806](https://github.com/windmill-labs/windmill/commit/e9cb80639b2dec63ede69fc3a4e3720bb1a3c319))
* use transaction for parallel_monitor_lock DELETE in last-iteration path ([#9789](https://github.com/windmill-labs/windmill/issues/9789)) ([754cae9](https://github.com/windmill-labs/windmill/commit/754cae956ac8d431ddeb055453835e249cdd07b7))
### Performance Improvements
* drop v2_job side-table ON DELETE CASCADE FKs to speed retention deletes ([#9786](https://github.com/windmill-labs/windmill/issues/9786)) ([aa098c7](https://github.com/windmill-labs/windmill/commit/aa098c70c0271b2b1917749d1f607c0559cf04de))
* eliminate dual-connection DB pool contention across worker, queue, and api ([#9798](https://github.com/windmill-labs/windmill/issues/9798)) ([0dbd9c1](https://github.com/windmill-labs/windmill/commit/0dbd9c1231b00d4693af68835fe1d9e7c8869b43))
## [1.739.0](https://github.com/windmill-labs/windmill/compare/v1.738.0...v1.739.0) (2026-06-24)
### Features
* add /compact session chat command ([#9764](https://github.com/windmill-labs/windmill/issues/9764)) ([83cc553](https://github.com/windmill-labs/windmill/commit/83cc5533ee92e59356a117eefeaf42dad23287f6))
* add session chat slash commands ([#9748](https://github.com/windmill-labs/windmill/issues/9748)) ([24b95e9](https://github.com/windmill-labs/windmill/commit/24b95e9fe12ba4abdfe1ff6e9f9fe42cb2ded011))
* **ai-chat:** add /clear session command to start a fresh conversation ([#9769](https://github.com/windmill-labs/windmill/issues/9769)) ([3fafac2](https://github.com/windmill-labs/windmill/commit/3fafac275d2100a6f89924040650cf959945d209))
* **ai-chat:** context usage gauge + unified model settings menu ([#9763](https://github.com/windmill-labs/windmill/issues/9763)) ([2e020b2](https://github.com/windmill-labs/windmill/commit/2e020b2ccc7a649a5923bff72a98f07d4fc85381))
* **apps:** show raw-app fork diffs as per-file tree items ([#9491](https://github.com/windmill-labs/windmill/issues/9491)) ([e98df38](https://github.com/windmill-labs/windmill/commit/e98df38ac43823ee85209a4b09cd70690469302d))
* **frontend:** add filter submenu to collapsed AI sessions popover ([#9757](https://github.com/windmill-labs/windmill/issues/9757)) ([3d48ba7](https://github.com/windmill-labs/windmill/commit/3d48ba7738c3d3356539b5fc44a871f6b7f9d548))
* **frontend:** restore raw app 'open preview in separate window' ([#9765](https://github.com/windmill-labs/windmill/issues/9765)) ([a116715](https://github.com/windmill-labs/windmill/commit/a116715c418c39d48a91e6c0b4484a31537dff38))
* **frontend:** show approval wait as a distinct segment in flow timeline ([#9756](https://github.com/windmill-labs/windmill/issues/9756)) ([2a70ccc](https://github.com/windmill-labs/windmill/commit/2a70ccc38675c7c2353807a4f85764a8a35224e2))
* scope AI sessions per workspace root with lifecycle reconcile ([#9734](https://github.com/windmill-labs/windmill/issues/9734)) ([42c5e7a](https://github.com/windmill-labs/windmill/commit/42c5e7a3fc9b74256de8806ec0d7b62e8bbf029c))
### Bug Fixes
* **ai-chat:** strip unclosed &lt;summary&gt; tag leaking into compaction summary ([#9750](https://github.com/windmill-labs/windmill/issues/9750)) ([250a05f](https://github.com/windmill-labs/windmill/commit/250a05f544ae397bb91af5fc83bf408cfe1c554d))
* **apps:** realign legacy raw-app drafts to raw_app draft kind ([#9761](https://github.com/windmill-labs/windmill/issues/9761)) ([288318a](https://github.com/windmill-labs/windmill/commit/288318ac269714fc03b15622dbb86b1c28268a36))
* **backend:** resolve folder_labels search_path on non-public (PG_SCHEMA) schemas ([#9758](https://github.com/windmill-labs/windmill/issues/9758)) ([f582878](https://github.com/windmill-labs/windmill/commit/f5828780fd6a8be070b2933ebd41ee6dff98a9e1))
* forbid superadmin job tokens from global user and token management ([#9715](https://github.com/windmill-labs/windmill/issues/9715)) ([043c2c0](https://github.com/windmill-labs/windmill/commit/043c2c05b7678c49faca0ccb28e5f6393567ba4d))
* **frontend:** highlight the runtime-chosen branch in flow graph viewer ([#9755](https://github.com/windmill-labs/windmill/issues/9755)) ([de6192b](https://github.com/windmill-labs/windmill/commit/de6192bec1695883a07452f7db2fb51c94dbfd43))
* **frontend:** keep #content portal target present on AI-session route ([#9754](https://github.com/windmill-labs/windmill/issues/9754)) ([5e09c50](https://github.com/windmill-labs/windmill/commit/5e09c501713ebbe05b28ce0084eca641f0dbe95c))
* **frontend:** show AI skills settings only when global mode enabled ([#9747](https://github.com/windmill-labs/windmill/issues/9747)) ([c017f7f](https://github.com/windmill-labs/windmill/commit/c017f7f8919a51292ddf01574961d1774bc1ba23))
* **frontend:** stop flow step id generation from being poisoned by non-canonical keys ([#9766](https://github.com/windmill-labs/windmill/issues/9766)) ([4dbf873](https://github.com/windmill-labs/windmill/commit/4dbf8737238ccc4dc2c67365e6d43f04f46c75b5))
* persist on-behalf-of user across app deploy paths ([#9773](https://github.com/windmill-labs/windmill/issues/9773)) ([f99781c](https://github.com/windmill-labs/windmill/commit/f99781ca5f77248206c951935cc44acfa5f072eb))
* reject symlink traversal in job-dir path validation ([#9713](https://github.com/windmill-labs/windmill/issues/9713)) ([b5bd824](https://github.com/windmill-labs/windmill/commit/b5bd8245d81b84fc14d3ea955bf1e66ac576bf37))
### Performance Improvements
* **audit:** adaptive timestamp floor for S3 audit-log export ([#9752](https://github.com/windmill-labs/windmill/issues/9752)) ([55bed4a](https://github.com/windmill-labs/windmill/commit/55bed4abcfce2a611b16054573980d2eb613ccb3))
* **monitor:** vacuum job_perms/job_result_stream right after each orphan sweep ([#9753](https://github.com/windmill-labs/windmill/issues/9753)) ([8912e21](https://github.com/windmill-labs/windmill/commit/8912e21d1571e57b5cf21b7d4d9520e20a28e70d))
## [1.738.0](https://github.com/windmill-labs/windmill/compare/v1.737.0...v1.738.0) (2026-06-23)
### Features
* add resource and infrastructure telemetry ([#9737](https://github.com/windmill-labs/windmill/issues/9737)) ([9793d01](https://github.com/windmill-labs/windmill/commit/9793d01575415963a89609a1baf2cd64f0d050cc))
* render mermaid diagrams in chat code blocks ([#9738](https://github.com/windmill-labs/windmill/issues/9738)) ([cfb9f1d](https://github.com/windmill-labs/windmill/commit/cfb9f1dbc23110ecf8f91bb3c8c81fc6e35dc09b))
### Bug Fixes
* **ai-chat:** Fix incorrect editor edits from ai chat [#1](https://github.com/windmill-labs/windmill/issues/1) ([#9741](https://github.com/windmill-labs/windmill/issues/9741)) ([fc797a3](https://github.com/windmill-labs/windmill/commit/fc797a35fe7885630c81453df0fc94769e73873a))
* allow object storage test for non-super-admins, harden on cloud ([#9739](https://github.com/windmill-labs/windmill/issues/9739)) ([24446e8](https://github.com/windmill-labs/windmill/commit/24446e80093ade349f7fbf65063d2d1cb5551c1e))
* **frontend:** debounce external code→Monaco sync in Editor ([#9743](https://github.com/windmill-labs/windmill/issues/9743)) ([29c67ce](https://github.com/windmill-labs/windmill/commit/29c67ced97bf2919584986f9d9eceb4337c34ad9))
* **frontend:** preserve editor content when closing instance settings drawer ([#9740](https://github.com/windmill-labs/windmill/issues/9740)) ([11d0e65](https://github.com/windmill-labs/windmill/commit/11d0e65f3af9a048bc1921bbdd3d676a07483a57))
* pipeline annotation false-positives from body comments ([#9736](https://github.com/windmill-labs/windmill/issues/9736)) ([984ea72](https://github.com/windmill-labs/windmill/commit/984ea728d98649b66b1cae899bdab9af3176caa7))
* preserve fork parent linkage on workspace id change ([#9716](https://github.com/windmill-labs/windmill/issues/9716)) ([cbf54d4](https://github.com/windmill-labs/windmill/commit/cbf54d4eb432638e27f67c4c8b879cbcc0291da3))
* prevent variable push from corrupting is_secret variables ([#9705](https://github.com/windmill-labs/windmill/issues/9705)) ([ba4b368](https://github.com/windmill-labs/windmill/commit/ba4b368706e95e22f346a10e5fe145b0795ac3f6))
### Performance Improvements
* **monitor:** skip protected prefix in retention delete via cross-batch watermark (WIN-2088) ([#9744](https://github.com/windmill-labs/windmill/issues/9744)) ([e90b2be](https://github.com/windmill-labs/windmill/commit/e90b2be8fade1eb78cd685890291f5a4553a6a10))
## [1.737.0](https://github.com/windmill-labs/windmill/compare/v1.736.0...v1.737.0) (2026-06-23)
### Features
* **apps:** opt-in sandbox isolation for published & raw apps (alpha) ([#9420](https://github.com/windmill-labs/windmill/issues/9420)) ([2879cbb](https://github.com/windmill-labs/windmill/commit/2879cbb65a4122c86b4a472206d74d9009b07904))
### Bug Fixes
* allow SQL args in managed // materialize scripts ([#9733](https://github.com/windmill-labs/windmill/issues/9733)) ([fa35968](https://github.com/windmill-labs/windmill/commit/fa3596885bf2d7ee8859f295e820ee362c756911))
* bound orphan-cleanup drain rate with capped multi-batch loop ([#9730](https://github.com/windmill-labs/windmill/issues/9730)) ([31d9215](https://github.com/windmill-labs/windmill/commit/31d9215e5a61f19662cc87be8147007e1d47ebb6))
* **ext-jwt:** reject external JWT auth for non-existent workspaces ([#9723](https://github.com/windmill-labs/windmill/issues/9723)) ([c644311](https://github.com/windmill-labs/windmill/commit/c644311eca4bcaf4b68058cf1d5d79d4078aee1a))
* optimize cleanup_job_perms_orphaned and job_result_stream cleanup queries ([#9727](https://github.com/windmill-labs/windmill/issues/9727)) ([6d94865](https://github.com/windmill-labs/windmill/commit/6d9486510933af9109f52011d93b13847dbdbb39))
* prevent silent audit-partition outage via monitor watchdog + alert ([#9729](https://github.com/windmill-labs/windmill/issues/9729)) ([8dea383](https://github.com/windmill-labs/windmill/commit/8dea38383f884f59b2956c39f1424005a21265bd))
### Performance Improvements
* **monitor:** hash active-root exclusion in retention delete (WIN-2088) ([#9732](https://github.com/windmill-labs/windmill/issues/9732)) ([75bafab](https://github.com/windmill-labs/windmill/commit/75bafabeeec76cf6da33eef41f588e37071df011))
## [1.736.0](https://github.com/windmill-labs/windmill/compare/v1.735.0...v1.736.0) (2026-06-23)
### Features
* **ai-chat:** workspace AI chat skills (SKILL.md upload + read_skill tool) ([#9648](https://github.com/windmill-labs/windmill/issues/9648)) ([6f4017d](https://github.com/windmill-labs/windmill/commit/6f4017d694494a158ebd0579c93336c378cba0fd))
### Bug Fixes
* **drafts:** stop mis-filing workspace-blind legacy drafts on migration ([#9725](https://github.com/windmill-labs/windmill/issues/9725)) ([3bf5b72](https://github.com/windmill-labs/windmill/commit/3bf5b72afab3241ea41a261a40c2434764bdaf72))
* **frontend:** destroy old WebsocketProvider on workspace switch in MultiplayerMenu ([#9719](https://github.com/windmill-labs/windmill/issues/9719)) ([6e96f90](https://github.com/windmill-labs/windmill/commit/6e96f90065dfe2f6ccc5eb4f85f4facd3515c70c))
* **frontend:** ensure type:object in test_run_flow tool schema for Anthropic ([#9721](https://github.com/windmill-labs/windmill/issues/9721)) ([d5cb944](https://github.com/windmill-labs/windmill/commit/d5cb944cf92f074b2ee42c876595eacdfa2f4d76))
* **health:** detect read-only replica via pg_is_in_recovery() ([#9722](https://github.com/windmill-labs/windmill/issues/9722)) ([e16061d](https://github.com/windmill-labs/windmill/commit/e16061df06babeae935a9396de5bdcd46e8119a9))
* re-enforce scoped API token boundaries across handlers ([#9712](https://github.com/windmill-labs/windmill/issues/9712)) ([e19594d](https://github.com/windmill-labs/windmill/commit/e19594df2ad015a0336ade95e04562f5562ec3f6))
## [1.735.0](https://github.com/windmill-labs/windmill/compare/v1.734.0...v1.735.0) (2026-06-22)
### Features
* clarify session draft bar tracks all workspace draft changes ([#9714](https://github.com/windmill-labs/windmill/issues/9714)) ([ed016a5](https://github.com/windmill-labs/windmill/commit/ed016a5edb4527877bf7e6bf92feafc2710669c2))
* **copilot:** improve global-mode path selection + add path-selection evals ([#9698](https://github.com/windmill-labs/windmill/issues/9698)) ([74a2329](https://github.com/windmill-labs/windmill/commit/74a2329d2e8395141807c43acef07ef132490039))
* link files & folders to the global AI chat ([#9520](https://github.com/windmill-labs/windmill/issues/9520)) ([84cc043](https://github.com/windmill-labs/windmill/commit/84cc043406d63a1e1472165cf24ce8c09905fc5f))
* scope default instance db name to workspace (dt_/dl_) ([#9699](https://github.com/windmill-labs/windmill/issues/9699)) ([4a8a724](https://github.com/windmill-labs/windmill/commit/4a8a724895dcecb835e1eb1e4fd7d1bbc8b3e0fb))
### Bug Fixes
* enforce job_dir containment when writing module files ([#9703](https://github.com/windmill-labs/windmill/issues/9703)) ([e403f92](https://github.com/windmill-labs/windmill/commit/e403f92d7e84cebc78709dce1a0928048ba2506d))
* **frontend:** deploy full script/flow draft from AI chat via shared module ([#9642](https://github.com/windmill-labs/windmill/issues/9642)) ([23bf6bf](https://github.com/windmill-labs/windmill/commit/23bf6bf3da01d552d2dfab6dbcfd30f758ed34d2))
* **frontend:** strip raw-app post-deploy diff noise (raw_app/lock/data) ([#9706](https://github.com/windmill-labs/windmill/issues/9706)) ([e20a277](https://github.com/windmill-labs/windmill/commit/e20a27745a08d552e6d2c5a8bbaf08ccfe89c68f))
* ignore NotFound errors when deleting log files from object store ([#9707](https://github.com/windmill-labs/windmill/issues/9707)) ([8a0b0ab](https://github.com/windmill-labs/windmill/commit/8a0b0abead71320c4f69eb3007739a19f76d4126))
* **oauth:** restore bring-your-own CC token URL override ([#9711](https://github.com/windmill-labs/windmill/issues/9711)) ([ef4962e](https://github.com/windmill-labs/windmill/commit/ef4962e52aba0bc79bf72523de9101853c660654))
* sanitize git credentials from ansible executor errors and logs ([#9697](https://github.com/windmill-labs/windmill/issues/9697)) ([ace7b68](https://github.com/windmill-labs/windmill/commit/ace7b68a28b00d715298ffcb6ae907c1974a74b8))
## [1.734.0](https://github.com/windmill-labs/windmill/compare/v1.733.1...v1.734.0) (2026-06-20)
### Features
* ducklake materialization for data pipelines ([#9689](https://github.com/windmill-labs/windmill/issues/9689)) ([3ebf243](https://github.com/windmill-labs/windmill/commit/3ebf24359d66048d6361ce65cd879cdc04b737ed))
### Bug Fixes
* **frontend:** clear branch step state when switching outer loop iterations ([#9650](https://github.com/windmill-labs/windmill/issues/9650)) ([09a8004](https://github.com/windmill-labs/windmill/commit/09a80040ca268a4379d5302e3401435ba93247e0))
## [1.733.1](https://github.com/windmill-labs/windmill/compare/v1.733.0...v1.733.1) (2026-06-19)
+24 -23
View File
@@ -1,7 +1,7 @@
ARG DEBIAN_IMAGE=debian:bookworm-slim
ARG RUST_IMAGE=rust:1.93-slim-bookworm
ARG DEBIAN_IMAGE=debian:trixie-slim
ARG RUST_IMAGE=rust:1.93-slim-trixie
FROM debian:bookworm-slim AS nsjail
FROM debian:trixie-slim AS nsjail
WORKDIR /nsjail
@@ -9,12 +9,12 @@ RUN apt-get -y update \
&& apt-get install -y \
bison=2:3.8.* \
flex=2.6.* \
g++=4:12.2.* \
gcc=4:12.2.* \
git=1:2.39.* \
g++=4:14.2.* \
gcc=4:14.2.* \
git=1:2.47.* \
libprotobuf-dev=3.21.* \
libnl-route-3-dev=3.7.* \
make=4.3-4.1 \
make=4.4.* \
pkg-config=1.8.* \
protobuf-compiler=3.21.*
@@ -44,7 +44,7 @@ FROM rust_base AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
RUN apt-get update && apt-get install -y clang=1:19.0* libclang-dev=1:19.0* cmake=3.31.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
@@ -98,7 +98,7 @@ ARG features=""
COPY --from=planner /windmill/recipe.json recipe.json
RUN apt-get update && apt-get install -y libxml2-dev=2.9.* libxmlsec1-dev=1.2.* libkrb5-dev libsasl2-dev libcurl4-openssl-dev clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
RUN apt-get update && apt-get install -y libxml2-dev=2.12.* libxmlsec1-dev=1.2.* libkrb5-dev libsasl2-dev libcurl4-openssl-dev clang=1:19.0* libclang-dev=1:19.0* cmake=3.31.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
@@ -135,9 +135,8 @@ FROM ${DEBIAN_IMAGE}
ARG TARGETPLATFORM
ARG POWERSHELL_VERSION=7.5.0
ARG POWERSHELL_DEB_VERSION=7.5.0-1
ARG KUBECTL_VERSION=1.28.7
ARG HELM_VERSION=3.14.3
ARG KUBECTL_VERSION=1.36.2
ARG HELM_VERSION=3.21.2
# NOTE: If changing, also change go version in workspace dependencies template at WorkspaceDependenciesEditor.svelte
ARG GO_VERSION=1.26.0
ARG APP=/usr/src/app
@@ -163,14 +162,14 @@ ENV PATH /usr/local/bin:/root/.local/bin:/tmp/.local/bin:$PATH
RUN apt-get update \
&& apt-get install -y --no-install-recommends netbase tzdata ca-certificates wget curl jq unzip build-essential unixodbc xmlsec1 software-properties-common tini gnupg lsb-release \
&& apt-get install -y --no-install-recommends netbase tzdata ca-certificates wget curl jq unzip build-essential unixodbc xmlsec1 tini gnupg libargon2-1 \
&& if echo "$features" | grep -q "ee"; then apt-get install -y --no-install-recommends libsasl2-modules-gssapi-mit krb5-user; fi \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install latest PostgreSQL client (pg_dump) from official PostgreSQL apt repository
RUN curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor -o /usr/share/keyrings/postgresql-archive-keyring.gpg \
&& echo "deb [signed-by=/usr/share/keyrings/postgresql-archive-keyring.gpg] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list \
&& echo "deb [signed-by=/usr/share/keyrings/postgresql-archive-keyring.gpg] https://apt.postgresql.org/pub/repos/apt $(. /etc/os-release; echo "$VERSION_CODENAME")-pgdg main" > /etc/apt/sources.list.d/pgdg.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends postgresql-client \
&& apt-get clean \
@@ -183,12 +182,14 @@ RUN if [ "$WITH_GIT" = "true" ]; then \
&& rm -rf /var/lib/apt/lists/*; \
else echo 'Building the image without git'; fi;
# PowerShell ships as a tarball: the upstream .deb depends on libicu<=74 which no longer exists in trixie
RUN if [ "$WITH_POWERSHELL" = "true" ]; then \
if [ "$TARGETPLATFORM" = "linux/amd64" ]; then apt-get update -y && apt install libicu-dev -y && wget -O 'pwsh.deb' "https://github.com/PowerShell/PowerShell/releases/download/v${POWERSHELL_VERSION}/powershell_${POWERSHELL_DEB_VERSION}.deb_amd64.deb" && apt-get clean \
&& rm -rf /var/lib/apt/lists/* && \
dpkg --install 'pwsh.deb' && \
rm 'pwsh.deb'; \
elif [ "$TARGETPLATFORM" = "linux/arm64" ]; then apt-get update -y && apt install libicu-dev -y && wget -O powershell.tar.gz "https://github.com/PowerShell/PowerShell/releases/download/v${POWERSHELL_VERSION}/powershell-${POWERSHELL_VERSION}-linux-arm64.tar.gz" && apt-get clean \
case "$TARGETPLATFORM" in \
"linux/amd64") pwsh_arch=x64 ;; \
"linux/arm64") pwsh_arch=arm64 ;; \
*) pwsh_arch="" ;; \
esac; \
if [ -n "$pwsh_arch" ]; then apt-get update -y && apt install libicu76 -y && wget -O powershell.tar.gz "https://github.com/PowerShell/PowerShell/releases/download/v${POWERSHELL_VERSION}/powershell-${POWERSHELL_VERSION}-linux-${pwsh_arch}.tar.gz" && apt-get clean \
&& rm -rf /var/lib/apt/lists/* && \
mkdir -p /opt/microsoft/powershell/7 && \
tar zxf powershell.tar.gz -C /opt/microsoft/powershell/7 && \
@@ -233,7 +234,7 @@ ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# Install UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.25/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.11.24/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
# Preinstall python runtimes to temp build location (will copy with world-writable perms later)
# --compile-bytecode precompiles the stdlib to .pyc so jobs don't recompile it on every run
@@ -292,7 +293,7 @@ RUN bun install -g windmill-cli \
RUN curl -fsSL https://claude.ai/install.sh | bash \
&& cp /root/.local/share/claude/versions/* /usr/bin/claude
COPY --from=php:8.3.30-cli-bookworm /usr/local/bin/php /usr/bin/php
COPY --from=php:8.3.30-cli-trixie /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.9.5 /usr/bin/composer /usr/bin/composer
# add the docker client to call docker from a worker if enabled
@@ -303,13 +304,13 @@ ENV CARGO_HOME="/tmp/windmill/cache/cargo"
ENV LD_LIBRARY_PATH="."
# nsjail runtime deps and binary
RUN apt-get update && apt-get install -y --no-install-recommends libprotobuf32 libnl-route-3-200 libnl-3-200 \
RUN apt-get update && apt-get install -y --no-install-recommends libprotobuf32t64 libnl-route-3-200 libnl-3-200 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
COPY --from=nsjail /nsjail/nsjail /bin/nsjail
# crane: pulls + flattens images for the sandboxed container runtime (`# sandbox <image>`).
# Single static binary — no daemon/store/root needed. See docs/docker-v2-runtime.md.
ARG CRANE_VERSION=v0.20.6
ARG CRANE_VERSION=v0.21.7
RUN arch="$(dpkg --print-architecture)"; \
case "$arch" in amd64) crane_arch=x86_64 ;; arm64) crane_arch=arm64 ;; *) echo >&2 "error: unsupported arch '$arch' for crane"; exit 1 ;; esac; \
wget -O /tmp/crane.tgz "https://github.com/google/go-containerregistry/releases/download/${CRANE_VERSION}/go-containerregistry_Linux_${crane_arch}.tar.gz" \
+10 -204
View File
@@ -1,208 +1,14 @@
# AI Evals Authoring Guide
# AI Evals
This folder contains black-box benchmark cases for:
Black-box benchmark cases for the Windmill AI generation modes (`flow`, `app`,
`script`, `cli`, `global`).
- `flow`
- `app`
- `script`
- `cli`
- `global`
**Authoring and running cases is documented in the `ai-evals` skill** — load it
before adding/changing a case or running a benchmark. Claude Code reads
`.claude/skills/ai-evals/SKILL.md`; Codex and Pi read
`.agents/skills/ai-evals/SKILL.md` (same canonical file). Invoke with `/ai-evals` in
Claude Code, `$ai-evals` in Codex, or `pi --skill ai-evals`.
The goal is to test the current production prompts and guidance with realistic user requests, not to test one exact implementation shape.
For AI chat / copilot changes that these evals measure, see the `ai-chat` skill.
## Core rules
1. Write prompts like a real user request.
2. Prefer behavior, inputs, constraints, and outcomes over internal implementation details.
3. Keep deterministic validation narrow and hard.
4. Put semantic expectations in `judgeChecklist`.
5. Use `expected` fixtures only when exact structure really matters.
## Prompt writing
Prompts should sound like something a user would naturally ask.
Good:
- "Create a flow that routes support requests based on customer tier."
- "Add a reset button that sets the counter back to 0."
- "Create a flow that reuses the existing greeting script instead of duplicating the logic."
Bad:
- "Use `branchone` with 3 branches and a default branch."
- "Create a `rawscript` step with this exact topology."
- "This is a benchmark harness."
Do not write prompts as if the user knows Windmill internals unless the case is explicitly testing a power-user workflow.
## Flow-specific rules
This is the main principle you asked for:
- flow prompts should read like requests from a user who does not know the product internals
- the user should ask for behavior, not for `branchone`, `branchall`, `rawscript`, `preprocessor_module`, `failure_module`, exact graph topology, or other internal constructs
That means:
- creation cases should describe the business behavior and expected result
- modification cases may mention existing step names, because the user can see the current flow
- only mention special Windmill constructs when the case is explicitly about those constructs
Examples:
- acceptable creation prompt:
"Create a purchase approval flow that pauses for approval and asks the approver for a comment."
- avoid:
"Create a suspend step with one required event and a resume form."
For flow cases, do not fail a case just because the model chose a different valid topology.
## App-specific rules
App prompts should focus on user-visible behavior:
- what the UI should let the user do
- what should persist
- what backend behavior is needed
Avoid prompting in terms of React structure, component names, or implementation unless the case is specifically about editing an existing app.
## CLI-specific rules
CLI prompts can be more explicit about paths and file names because real CLI users often do specify them.
Still, avoid benchmark phrasing. The prompt should read like a repo task, not a harness instruction.
When relevant, ask the assistant to tell the user which `wmill` commands to run next. That is part of the benchmarked behavior.
## Global-specific rules
Global prompts should exercise workspace-level drafting behavior:
- inspecting existing scripts, flows, apps, schedules, triggers, resources, and variables when relevant
- writing AI drafts rather than saving or deploying by default
- producing coherent multi-artifact changes when the request crosses artifact boundaries
Keep deterministic validation focused on the draft contract: required draft type/path, required content snippets, forbidden draft paths, and forbidden mutating tools such as deploy/delete unless the case explicitly asks for them.
Datatable cases should set `skipJudge: true` and validate through tool-use
(`requiredToolsUsed` / `forbiddenToolsUsed`) and SQL-argument assertions
(`toolCallArgs` with `stringIncludesAnyOf`, e.g. `['select']`, `['create table']`,
`['update', 'insert into']`). Two reasons the judge is unreliable here:
- `list_datatables`, `get_datatable_table_schema`, and `exec_datatable_sql`
produce no drafts, and the global judge only sees the drafts artifact — it
scores a no-draft conversational answer as empty (same as the
`askUserQuestion` cases).
- Even a case that *does* produce a draft (a script reading the data table via
`wmill.datatable()` at runtime) is mis-judged: the judge has no datatable SDK
reference and penalizes correct `wmill.datatable()` usage as wrong. Verify the
SDK call deterministically instead — `requiredDrafts.valueIncludes: ['wmill.datatable(']`
plus forbidding `exec_datatable_sql` (keeping chat-time SQL distinct from
runtime SDK use).
`stringIncludesAnyOf` is existential over calls (at least one matching call), so a
mutation case still passes when the model mixes its UPDATE/INSERT with
verification SELECTs. The in-memory engine (`datatableSqlEngine.ts`) is stateful
within a case — writes persist, so a model that re-queries to verify its
CREATE/UPDATE sees the change and does not loop. But the engine is best-effort
(SELECT returns all rows of the referenced/first table with no WHERE/projection),
so still never assert specific returned row values. Seed data via
`workspace.datatables` in the `initial` fixture (see README).
## Deterministic validation
Use deterministic validation only for hard failures such as:
- missing required files
- unexpected extra files when the prompt says not to create them
- syntax errors
- unresolved flow refs
- missing required special modules or suspend config
- obvious artifact corruption
Do not use deterministic validation to enforce one preferred implementation for broad creation tasks.
Examples of bad hard checks:
- exact step topology for a creation flow
- exact branch structure when the prompt only asked for routing behavior
- exact input shape when multiple reasonable shapes are acceptable
## Judge checklist
Every non-trivial case should have a `judgeChecklist`.
The checklist should capture:
- the user-visible behavior that must be present
- important constraints
- key completion criteria
The checklist should not duplicate low-level implementation details unless they are truly required by the task.
Good checklist items:
- "the flow calculates the order total with 8% tax"
- "the app persists recipes appropriately for a raw Windmill app"
- "the flow reuses the existing workspace script instead of rewriting the logic"
Bad checklist items:
- "uses `branchone`"
- "contains a `rawscript` node"
## When to use `expected`
Use `expected` fixtures when the case is structure-sensitive, for example:
- exact file creation
- exact script content
- modification cases where a specific file must change in a specific way
- cases where preserving an existing structure is part of the requirement
Do not use a full `expected` artifact as the semantic oracle for broad creation tasks when multiple valid outputs should pass.
## When to use `initial`
Use `initial` when the benchmark is about:
- editing an existing artifact
- reusing existing workspace assets
- preserving existing behavior while adding a change
If the case is greenfield, prefer no `initial`.
## Case design ladder
Prefer suites that get gradually harder:
1. trivial create case
2. realistic create case
3. reuse-existing-assets case
4. modification case
5. refactor case
6. edge-case or niche product behavior
The last cases in a suite should cover unusual or product-specific behavior.
## Anti-patterns
Avoid these:
- benchmark framing in prompts
- over-specified internal topology for creation tasks
- judge checklists that just restate implementation details
- deterministic validation that encodes one preferred solution
- fixtures that are so minimal or brittle that they create false negatives
## Before adding a case
Ask:
1. Would a real user plausibly write this prompt?
2. If the model solves it in a different valid way, would the case still pass?
3. Are the hard deterministic checks only catching objectively broken output?
4. Does the `judgeChecklist` describe the real success criteria?
5. If this case fails, will the reason be understandable from the saved artifacts?
The full case format, fields, and fixture details remain in `ai_evals/README.md`.
+3 -1
View File
@@ -75,6 +75,8 @@ Public CLI surface:
- `--model <alias>`: choose the model under test
- `--models <a,b,c>`: run the same cases sequentially against several model aliases
- `--verbose`: stream assistant output for frontend runs
- `--skip-judge`: skip LLM judge scoring for the run
- `--execution-only`: only require the model/proxy/frontend loop to complete; skip validators, tool expectations, backend artifact validation, and judge scoring
- `--record`: append a compact tracked summary line to `ai_evals/history/<mode>.jsonl` for full-suite runs only
- `--backend-validation <mode>`: optional backend smoke validation (`off` or `preview`) for `script` and `flow` evals
@@ -99,7 +101,7 @@ Notes:
- the command also prints accepted alias spellings such as `gpt-4o`, `gpt-55`, `claude-opus-4.6`, and `claude-haiku-4.5`
- frontend modes (`flow`, `script`, `app`, `global`) can use Anthropic, OpenAI, Gemini, and DeepSeek-backed aliases
- `cli` mode always uses the Anthropic agent SDK, so only Anthropic aliases are valid there
- the judge model is separate and currently defaults to `claude-sonnet-4-6`
- the judge model is separate and currently defaults to `claude-sonnet-4-6`; use `--skip-judge` for deterministic-only runs
## Case Format
@@ -25,6 +25,12 @@ export async function runFrontendBenchmarkFromEnv(): Promise<BenchmarkRunResult>
);
const emitProgress = process.env.WMILL_FRONTEND_AI_EVAL_PROGRESS === "1";
const verbose = process.env.WMILL_FRONTEND_AI_EVAL_VERBOSE === "1";
const executionOnly =
process.env.WMILL_FRONTEND_AI_EVAL_EXECUTION_ONLY === "1";
const judgeModel =
process.env.WMILL_FRONTEND_AI_EVAL_SKIP_JUDGE === "1" || executionOnly
? null
: DEFAULT_JUDGE_MODEL;
const model = resolveEvalModel(
mode,
process.env.WMILL_FRONTEND_AI_EVAL_MODEL,
@@ -48,7 +54,8 @@ export async function runFrontendBenchmarkFromEnv(): Promise<BenchmarkRunResult>
cases: selectedCases,
runs,
runModel,
judgeModel: DEFAULT_JUDGE_MODEL,
judgeModel,
executionOnly,
concurrency: verbose ? 1 : undefined,
verbose,
onProgress: emitProgress
@@ -60,7 +67,7 @@ export async function runFrontendBenchmarkFromEnv(): Promise<BenchmarkRunResult>
mode,
runs,
runModel,
judgeModel: DEFAULT_JUDGE_MODEL,
judgeModel,
caseResults,
});
}
@@ -50,6 +50,20 @@ export interface GlobalLiveEditorDraftFixture {
value?: unknown;
}
// Identity the global system prompt builds paths from. Production reads
// `userStore` (whoami) to fill `u/{username}/...`; the eval harness never logs
// in, so without this the prompt sees an empty username (`u//...`) and no
// path-selection case is meaningful. Seeded per-case via the initial fixture and
// passed straight to `prepareGlobalSystemMessage` (no global-store mutation).
export interface GlobalUserFixture {
username: string;
is_admin?: boolean;
/** Folders the user can write to (the writable set whoami returns). */
folders?: string[];
/** Folders the user can read; read-only folders = folders_read \ folders. */
folders_read?: string[];
}
export interface GlobalEvalResult {
success: boolean;
state: GlobalDraftState;
@@ -65,6 +79,7 @@ export interface GlobalEvalResult {
export interface GlobalEvalOptions {
workspaceFixtures?: BenchmarkWorkspaceRunnables;
liveEditorDrafts?: GlobalLiveEditorDraftFixture[];
user?: GlobalUserFixture;
model?: string;
maxIterations?: number;
provider?: AIProvider;
@@ -90,9 +105,11 @@ export async function runGlobalEval(
const model = options.model ?? "claude-haiku-4-5-20251001";
const injectActiveEditorContext =
process.env[DISABLE_ACTIVE_EDITOR_CONTEXT_ENV] !== "1";
// Pass the seeded identity straight to the prompt builder rather than mutating
// the process-global `userStore`, so concurrent cases never race on it.
const rawResult = await runEval({
userPrompt,
systemMessage: prepareGlobalSystemMessage(),
systemMessage: prepareGlobalSystemMessage(undefined, { user: options.user }),
userMessage: prepareGlobalUserMessage(
userPrompt,
[],
+24 -2
View File
@@ -11,6 +11,7 @@ import type {
DataTableTables,
DataTableTableSchema,
GetDraftForUserResponse,
GetOwnDraftResponse,
ListDraftsResponse,
ScriptLang,
UpdateDraftResponse,
@@ -90,6 +91,13 @@ export function resetBenchmarkMockBackend(): void {
benchmarkDrafts.clear()
}
// Stand-in for FolderService.createFolder so the global create_folder tool runs in
// memory instead of mutating the real backend. Folders aren't otherwise modelled
// (no folder-listing in evals), so this just echoes the created name.
export function createBenchmarkFolder(_workspace: string, name: string): string {
return name
}
export function registerBenchmarkWorkspace(workspace: string): void {
benchmarkWorkspaces.add(workspace)
}
@@ -287,8 +295,8 @@ export function getBenchmarkJobLogs(workspace: string, jobId: string): string {
/**
* In-memory stand-in for the per-user draft backend (`DraftService`). The global
* AI chat now persists and reads drafts through the backend DB instead of an
* in-tab `UserDraft` cell, so the eval mocks the three draft endpoints it
* exercises (`updateDraft` / `getDraftForUser` / `listDrafts`) and keeps the
* in-tab `UserDraft` cell, so the eval mocks the draft endpoints it exercises
* (`updateDraft` / `getOwnDraft` / `getDraftForUser` / `listDrafts`) and keeps the
* saved values here, keyed by workspace + draft kind + storage path. Mirrors the
* semantics of the production unit test's mock in
* `frontend/src/lib/components/copilot/chat/global/core.test.ts`.
@@ -372,6 +380,20 @@ export function getBenchmarkDraftForUser(input: {
return { value: entry.value, created_at: BENCHMARK_DRAFT_TIMESTAMP }
}
/** Mirror `DraftService.getOwnDraft`: `null` (200) when absent — unlike
* `getDraftForUser`, absence is not an error on this route. */
export function getBenchmarkOwnDraft(input: {
workspace: string
kind: UserDraftItemKind
path: string
}): GetOwnDraftResponse {
const entry = benchmarkDrafts.get(benchmarkDraftKey(input.workspace, input.kind, input.path))
if (!entry) {
return null
}
return { value: entry.value, created_at: BENCHMARK_DRAFT_TIMESTAMP }
}
/** Mirror `DraftService.listDrafts`: metadata rows (no value) for a workspace. */
export function listBenchmarkDrafts(workspace: string): ListDraftsResponse {
return [...benchmarkDrafts.values()]
@@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, it } from 'bun:test'
import {
clearBenchmarkDrafts,
getBenchmarkDraftForUser,
getBenchmarkOwnDraft,
listBenchmarkDrafts,
resetBenchmarkMockBackend,
seedBenchmarkDraft,
@@ -55,6 +56,27 @@ describe('mockBackend drafts', () => {
expect(() => getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'variable', path: 'f/evals/token' })).toThrow()
})
it('returns null from getOwnDraft when no draft exists', () => {
expect(
getBenchmarkOwnDraft({ workspace: WORKSPACE, kind: 'trigger_schedule', path: 'u/evals/missing' })
).toBeNull()
})
// The global chat hydrates drawer-kind drafts (schedule/trigger/resource/variable)
// through getOwnDraft — getDraftForUser rejects those kinds as private.
it('hydrates a saved drawer-kind draft through getOwnDraft', () => {
const value = { path: 'u/evals/nightly', schedule: '0 0 9 * * *' }
updateBenchmarkDraft({
workspace: WORKSPACE,
kind: 'trigger_schedule',
path: 'u/evals/nightly',
requestBody: { value }
})
expect(
getBenchmarkOwnDraft({ workspace: WORKSPACE, kind: 'trigger_schedule', path: 'u/evals/nightly' })?.value
).toEqual(value)
})
it('throws a 404-shaped error when no draft exists', () => {
try {
getBenchmarkDraftForUser({ workspace: WORKSPACE, kind: 'script', path: 'f/evals/missing' })
+5
View File
@@ -24,6 +24,8 @@ export async function runFrontendBenchmarkAdapter(input: {
runs: number;
model?: string;
verbose?: boolean;
skipJudge?: boolean;
executionOnly?: boolean;
backendValidation?: string;
}): Promise<BenchmarkRunResult> {
const tempDir = await mkdtemp(
@@ -40,6 +42,9 @@ export async function runFrontendBenchmarkAdapter(input: {
WMILL_FRONTEND_AI_EVAL_MODEL: input.model ?? "",
WMILL_FRONTEND_AI_EVAL_PROGRESS: "1",
WMILL_FRONTEND_AI_EVAL_VERBOSE: input.verbose ? "1" : "0",
WMILL_FRONTEND_AI_EVAL_SKIP_JUDGE:
input.skipJudge || input.executionOnly ? "1" : "0",
WMILL_FRONTEND_AI_EVAL_EXECUTION_ONLY: input.executionOnly ? "1" : "0",
WMILL_FRONTEND_AI_EVAL_BACKEND_VALIDATION: input.backendValidation ?? "",
};
@@ -40,6 +40,7 @@ vi.mock('$lib/gen', async () => {
getBenchmarkDraftForUser,
getBenchmarkFlowByPath,
getBenchmarkJobLogs,
getBenchmarkOwnDraft,
getBenchmarkScriptByHash,
getBenchmarkScriptByPath,
hasBenchmarkWorkspace,
@@ -49,6 +50,7 @@ vi.mock('$lib/gen', async () => {
listBenchmarkFlows,
listBenchmarkJobs,
listBenchmarkScripts,
createBenchmarkFolder,
createBenchmarkHttpTrigger,
createBenchmarkSchedule,
previewBenchmarkSchedule,
@@ -85,11 +87,21 @@ vi.mock('$lib/gen', async () => {
hasBenchmarkWorkspace(data.workspace)
? getBenchmarkDraftForUser(data)
: actual.DraftService.getDraftForUser(data),
getOwnDraft: async (data: { workspace: string; kind: any; path: string }) =>
hasBenchmarkWorkspace(data.workspace)
? getBenchmarkOwnDraft(data)
: actual.DraftService.getOwnDraft(data),
listDrafts: async (data: { workspace: string }) =>
hasBenchmarkWorkspace(data.workspace)
? listBenchmarkDrafts(data.workspace)
: actual.DraftService.listDrafts(data)
}),
FolderService: wrapService(actual.FolderService, {
createFolder: async (data: { workspace: string; requestBody: { name: string } }) =>
hasBenchmarkWorkspace(data.workspace)
? createBenchmarkFolder(data.workspace, data.requestBody.name)
: actual.FolderService.createFolder(data)
}),
ScriptService: wrapService(actual.ScriptService, {
listScripts: async (data: { workspace: string }) =>
hasBenchmarkWorkspace(data.workspace)
+3
View File
@@ -359,6 +359,8 @@
- request_approval
- finalize_purchase
topLevelStepTypes:
- id: request_approval
type: [rawscript, script]
- id: finalize_purchase
type: rawscript
schemaRequiredPaths:
@@ -373,6 +375,7 @@
judgeChecklist:
- "the flow includes an approval step named `request_approval`"
- "`request_approval` pauses the flow and asks the approver for a comment"
- "`request_approval` is a real script step that generates approval/resume URLs (e.g. via `getResumeUrls`) so approvers receive an actionable link, not a no-op passthrough (identity) step"
- one approval is enough to continue
- "the flow includes a final step named `finalize_purchase`"
- "`finalize_purchase` returns an approved status object after approval"
+397 -1
View File
@@ -3,8 +3,9 @@
Create a draft Bun script at `f/evals/global/greet_user`.
It should take a string `name` input and return `Hello, ${name}!`.
Leave it as an AI draft only; do not deploy or save it.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_evals_folder.json
runtime:
maxTurns: 8
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
@@ -871,6 +872,207 @@
- fetches the logs for the requested job id
- explains the failure from the returned logs (connection refused to the upstream API)
# --- Page navigation (open_page) ---
# The assistant should take the user to a Windmill page (Runs/Schedules) with the
# right filters via open_page, rather than describing where to click or dumping the
# data. No draft is produced, so the global judge is skipped and we validate the
# tool call and its arguments.
- id: global-openpage1-runs-failed-of-script
prompt: |-
Take me to the failed runs of the script at f/evals/global/greet_user so I can see what's going wrong.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- open_page
forbiddenToolsUsed:
- write_script
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: open_page
field: page
stringIncludesAnyOf:
- runs
- tool: open_page
field: status
stringIncludesAnyOf:
- failure
- tool: open_page
field: path
stringIncludesAnyOf:
- f/evals/global/greet_user
skipJudge: true
judgeChecklist:
- opens the Runs page filtered to the failed runs of f/evals/global/greet_user
- applies both the failure status and the script path as filters
- does not write, deploy, or delete anything
- id: global-openpage2-runs-of-schedule
prompt: |-
Open the runs page filtered to the jobs triggered by the schedule f/evals/global/nightly_digest.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- open_page
forbiddenToolsUsed:
- write_script
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: open_page
field: page
stringIncludesAnyOf:
- runs
- tool: open_page
field: schedule_path
stringIncludesAnyOf:
- f/evals/global/nightly_digest
skipJudge: true
judgeChecklist:
- opens the Runs page filtered to jobs triggered by the f/evals/global/nightly_digest schedule
- passes the schedule path as the filter
- does not write, deploy, or delete anything
- id: global-openpage3-open-schedule
prompt: |-
Open the schedule f/evals/global/nightly_digest so I can review and edit it.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- open_page
forbiddenToolsUsed:
- write_schedule
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: open_page
field: page
stringIncludesAnyOf:
- schedules
- tool: open_page
field: open
stringIncludesAnyOf:
- f/evals/global/nightly_digest
skipJudge: true
judgeChecklist:
- opens the Schedules page and targets the f/evals/global/nightly_digest schedule for editing
- passes the schedule path so the editor opens on it
- does not write, deploy, or delete anything
- id: global-openpage4-workspace-settings-tab
prompt: |-
Take me to the Git sync configuration for this workspace.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- open_page
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: open_page
field: page
stringIncludesAnyOf:
- workspace_settings
- tool: open_page
field: tab
stringIncludesAnyOf:
- git_sync
skipJudge: true
judgeChecklist:
- opens the Workspace settings page on the git_sync tab
- does not write, deploy, or delete anything
- id: global-openpage5-audit-logs-user
prompt: |-
Open the audit logs filtered to actions performed by the user admin.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- open_page
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: open_page
field: page
stringIncludesAnyOf:
- audit_logs
- tool: open_page
field: username
stringIncludesAnyOf:
- admin
skipJudge: true
judgeChecklist:
- opens the Audit logs page filtered to the admin user
- does not write, deploy, or delete anything
- id: global-openpage6-triggers-kind
prompt: |-
Take me to the Kafka triggers for this workspace.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- open_page
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: open_page
field: page
stringIncludesAnyOf:
- triggers
- tool: open_page
field: trigger_kind
stringIncludesAnyOf:
- kafka
skipJudge: true
judgeChecklist:
- opens the Kafka triggers page
- does not write, deploy, or delete anything
- id: global-closepage1-close-runs-tab
prompt: |-
You just opened the runs page for me in the side panel. Close that tab, I'm done looking at it.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- close_page
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: close_page
field: match
stringIncludesAnyOf:
- runs
skipJudge: true
judgeChecklist:
- closes the runs preview tab in the side panel
- does not write, deploy, or delete anything
# --- Documentation search (search_docs) ---
# Pure product-knowledge questions: the assistant should consult the docs via
# search_docs and answer conversationally, not draft or mutate anything. No
@@ -1113,3 +1315,197 @@
- renames the formatCurrency definition, imports, and all call sites to formatMoney
- leaves the unrelated formatCurrencyPrecise helper unchanged
- leaves the result as an AI draft only
# --- Path selection (u/<user> vs f/<folder>) ---
# These cases assert how the assistant picks a workspace path when the user gives
# none: a bare name defaults to the personal scope `u/<user>/`, an existing folder
# whose purpose matches is used, a non-admin targets a writable folder and never a
# read-only one, and shared intent with no matching folder asks rather than invents.
# Each depends on the seeded `user` fixture (username / is_admin / folders /
# folders_read) so the prompt's folder guidance and `u/{username}` are well-formed.
- id: global-path1-bare-name-defaults-to-personal
prompt: |-
Stage a quick draft helper that takes a string and returns it trimmed of
leading and trailing whitespace. Just keep it as a draft.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_empty.json
runtime:
maxTurns: 8
validate:
draftCountExactly: 1
requiredDrafts:
- type: script
pathStartsWith: u/admin/
toolExpect:
requiredToolsUsed:
- write_script
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- stages a single script draft for a trim helper
- defaults the path to the current user's personal scope (u/admin/...) since no path or folder was given
- does not invent an f/<folder> path
- leaves the result as a draft only
- id: global-path2-match-existing-folder
prompt: |-
Draft a flow for the marketing team's weekly campaign report.
It should take a week number and return a short summary string.
Keep it as a draft only.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_folders.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: flow
pathStartsWith: f/marketing/
toolExpect:
requiredToolsUsed:
- write_flow
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- drafts a flow for the marketing campaign report
- places it in the existing marketing folder (f/marketing/...) rather than the personal scope or an invented folder
- leaves the result as a draft only
- id: global-path3-shared-intent-unknown-folder-asks
prompt: |-
Put together a draft onboarding checklist flow for the People Ops team to use
when a new hire joins. Keep it as a draft.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_folders.json
runtime:
maxTurns: 8
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- askUserQuestion
forbiddenToolsUsed:
- write_flow
- write_script
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- recognizes the request implies shared/team work but names no existing folder (none of marketing/data_engineering/shared_utils fit People Ops)
- asks which folder to use instead of guessing or inventing one
- does not create a draft until the folder is known
- id: global-path4-nonadmin-avoids-readonly-folder
prompt: |-
Draft a small flow that returns today's date as an ISO string, and stage it in
one of our shared team folders. Keep it as a draft.
initial: ai_evals/fixtures/frontend/global/initial/user_bob_nonadmin_teams.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: flow
pathStartsWith: f/team_a/
forbiddenDrafts:
- type: flow
pathStartsWith: f/team_b/
toolExpect:
requiredToolsUsed:
- write_flow
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- drafts a flow that returns the current date as an ISO string
- places it in team_a (writable by this non-admin user) and not team_b (read-only)
- leaves the result as a draft only
- id: global-test-pipeline-create-node
prompt: |-
Set up the first step of a data pipeline at `f/evals/global/orders_ingest`.
On a schedule, it should pull raw orders and land them in a managed DuckLake
table so later steps can build on it. Keep it as an AI draft only — don't
deploy or save it.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_evals_folder.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: script
path: f/evals/global/orders_ingest
valueIncludes:
- pipeline
forbiddenDrafts:
- type: flow
pathStartsWith: f/evals/global/
toolExpect:
requiredToolsUsed:
- write_script
forbiddenToolsUsed:
- write_flow
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- builds a data pipeline node as a script (not a flow)
- marks the script as a pipeline member with the pipeline annotation in the script's comment syntax (`-- pipeline` for a DuckDB/SQL node, not `// pipeline`)
- declares a schedule trigger and writes its output to a managed DuckLake table
- leaves the result as an AI draft and does not deploy or save it
- id: global-test-pipeline-two-node-chain
prompt: |-
Build a small data pipeline in the `f/evals/global` folder: one step that
ingests orders into a DuckLake table, and a second step that reads that table
and writes a daily order-count rollup table. Wire the second step to run off
the first step's output. Keep everything as drafts — don't deploy.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_evals_folder.json
runtime:
maxTurns: 14
validate:
draftCountAtLeast: 2
forbiddenDrafts:
- type: flow
pathStartsWith: f/evals/global/
toolExpect:
requiredToolsUsed:
- write_script
forbiddenToolsUsed:
- write_flow
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- creates two data pipeline nodes as scripts (not a flow) in f/evals/global
- both scripts carry the pipeline annotation in their comment syntax (`-- pipeline` for DuckDB/SQL nodes, not `// pipeline`)
- the first ingests orders into a DuckLake table
- the second reads that same table and writes a daily rollup, wired to the first step's output asset
- leaves both as AI drafts without deploying
- id: global-path5-create-folder-then-draft
prompt: |-
Create a new shared folder called "analytics" for our data work, then draft a
script in it that returns the current timestamp as an ISO string. Keep the
script as a draft.
initial: ai_evals/fixtures/frontend/global/initial/user_admin_empty.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: script
pathStartsWith: f/analytics/
toolExpect:
requiredToolsUsed:
- create_folder
- write_script
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- creates a new shared folder named "analytics" via create_folder
- drafts a script placed in that folder (f/analytics/...) returning an ISO timestamp
- leaves the script as a draft only
+25 -3
View File
@@ -25,7 +25,9 @@ import {
import { runSuite } from "../core/runSuite";
import { EVAL_MODES, type EvalMode } from "../core/types";
import { DEFAULT_JUDGE_MODEL } from "../core/judge";
import { createCliModeRunner } from "../modes/cli";
// createCliModeRunner is imported lazily in runCliBenchmark so the non-cli modes
// (global/flow/script/app) don't pull in the wmill CLI toolchain and its JSR deps
// (e.g. @cliffy/*) just to load this entrypoint.
import { runFrontendBenchmarkAdapter } from "../adapters/frontend/runtime";
import { resolveWindmillBackendSettings } from "../core/windmillBackendSettings";
import { assertWindmillBackendReachable } from "../adapters/frontend/windmillBackend";
@@ -97,6 +99,11 @@ async function main() {
"comma-separated model aliases to run sequentially",
)
.option("--verbose", "stream assistant output during frontend runs")
.option("--skip-judge", "skip LLM judge scoring for this run")
.option(
"--execution-only",
"only require the model/proxy/frontend loop to complete",
)
.option(
"--record",
"append a compact summary line to ai_evals/history/<mode>.jsonl",
@@ -115,6 +122,8 @@ async function main() {
model?: string;
models?: string;
verbose?: boolean;
skipJudge?: boolean;
executionOnly?: boolean;
record?: boolean;
backendValidation?: string;
},
@@ -127,6 +136,8 @@ async function main() {
model: options.model,
models: options.models,
verbose: options.verbose ?? false,
skipJudge: options.skipJudge ?? false,
executionOnly: options.executionOnly ?? false,
record: options.record ?? false,
backendValidation: options.backendValidation,
});
@@ -175,6 +186,8 @@ async function handleRun(input: {
model?: string;
models?: string;
verbose: boolean;
skipJudge: boolean;
executionOnly: boolean;
record: boolean;
backendValidation?: string;
}) {
@@ -230,6 +243,8 @@ async function handleRun(input: {
input.runs,
getCliEvalModel(model),
runModel,
input.skipJudge,
input.executionOnly,
)
: await runFrontendBenchmarkAdapter({
mode: input.mode,
@@ -237,6 +252,8 @@ async function handleRun(input: {
runs: input.runs,
model: model.id,
verbose: input.verbose,
skipJudge: input.skipJudge,
executionOnly: input.executionOnly,
backendValidation,
});
@@ -278,20 +295,25 @@ async function runCliBenchmark(
runs: number,
model: ReturnType<typeof getCliEvalModel>,
runModel: string,
skipJudge: boolean,
executionOnly: boolean,
) {
const { createCliModeRunner } = await import("../modes/cli");
const judgeModel = skipJudge || executionOnly ? null : DEFAULT_JUDGE_MODEL;
const caseResults = await runSuite({
modeRunner: createCliModeRunner(model),
cases,
runs,
runModel,
judgeModel: DEFAULT_JUDGE_MODEL,
judgeModel,
executionOnly,
});
return buildRunResult({
mode: "cli",
runs,
runModel,
judgeModel: DEFAULT_JUDGE_MODEL,
judgeModel,
caseResults,
});
}
+3
View File
@@ -212,6 +212,9 @@ describe("loadCases", () => {
},
],
});
expect(caseEntry?.initialPath).toContain(
"ai_evals/fixtures/frontend/global/initial/user_admin_evals_folder.json"
);
expect(caseEntry?.toolExpect).toMatchObject({
requiredToolsUsed: ["write_script"],
forbiddenToolsUsed: ["deploy_workspace_item", "delete_workspace_item"],
+102
View File
@@ -0,0 +1,102 @@
import { describe, expect, it } from "bun:test";
import { runSuite } from "./runSuite";
import type { ModeRunner } from "./types";
const modeRunner: ModeRunner<undefined, undefined, { ok: boolean }> = {
mode: "global",
concurrency: 1,
loadInitial: async () => undefined,
loadExpected: async () => undefined,
run: async () => ({
success: true,
actual: { ok: true },
assistantMessageCount: 1,
toolCallCount: 0,
toolsUsed: [],
skillsInvoked: [],
tokenUsage: null,
}),
validate: () => [],
};
describe("runSuite", () => {
it("skips judge checks when the run disables judge scoring", async () => {
const [caseResult] = await runSuite({
modeRunner,
cases: [
{
id: "case-1",
prompt: "Create a draft script",
judgeChecklist: ["the output satisfies the prompt"],
},
],
runs: 1,
runModel: "model-under-test",
judgeModel: null,
});
const [attempt] = caseResult.attempts;
expect(attempt.passed).toBe(true);
expect(attempt.judgeScore).toBeNull();
expect(attempt.judgeSummary).toBeNull();
expect(attempt.checks.map((check) => check.name)).toEqual([
"run succeeded",
]);
});
it("only requires run success when execution-only is enabled", async () => {
let loadExpectedCalls = 0;
let validateCalls = 0;
let backendValidateCalls = 0;
const executionOnlyRunner: ModeRunner<
undefined,
undefined,
{ ok: boolean }
> = {
...modeRunner,
loadExpected: async () => {
loadExpectedCalls++;
return undefined;
},
validate: () => {
validateCalls++;
return [{ name: "validator failed", passed: false }];
},
backendValidate: async () => {
backendValidateCalls++;
return {
checks: [{ name: "backend validation failed", passed: false }],
};
},
};
const [caseResult] = await runSuite({
modeRunner: executionOnlyRunner,
cases: [
{
id: "case-1",
prompt: "Create a draft script",
expectedPath: "fixtures/expected.json",
toolExpect: { requiredToolsUsed: ["write_script"] },
judgeChecklist: ["the output satisfies the prompt"],
},
],
runs: 1,
runModel: "model-under-test",
judgeModel: "judge-model",
executionOnly: true,
});
const [attempt] = caseResult.attempts;
expect(attempt.passed).toBe(true);
expect(attempt.judgeScore).toBeNull();
expect(attempt.judgeSummary).toBeNull();
expect(attempt.checks.map((check) => check.name)).toEqual([
"run succeeded",
]);
expect(loadExpectedCalls).toBe(0);
expect(validateCalls).toBe(0);
expect(backendValidateCalls).toBe(0);
});
});
+36 -17
View File
@@ -15,11 +15,13 @@ export async function runSuite<TInitial, TExpected, TActual>(input: {
runs: number;
runModel: string | null;
judgeModel?: string | null;
executionOnly?: boolean;
concurrency?: number;
verbose?: boolean;
onProgress?: (event: FrontendBenchmarkProgressEvent) => void;
}): Promise<BenchmarkCaseResult[]> {
const judgeModel = input.judgeModel ?? DEFAULT_JUDGE_MODEL;
const judgeModel =
input.judgeModel === undefined ? DEFAULT_JUDGE_MODEL : input.judgeModel;
const concurrency = Math.max(1, input.concurrency ?? input.modeRunner.concurrency);
const results = new Array<BenchmarkCaseResult>(input.cases.length);
let cursor = 0;
@@ -52,6 +54,7 @@ export async function runSuite<TInitial, TExpected, TActual>(input: {
runs: input.runs,
judgeModel,
judgeThreshold: input.modeRunner.judgeThreshold ?? 80,
executionOnly: input.executionOnly ?? false,
modeRunner: input.modeRunner,
totalCases: input.cases.length,
verbose: input.verbose ?? false,
@@ -72,8 +75,9 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
caseIndex: number;
evalCase: EvalCase;
runs: number;
judgeModel: string;
judgeModel: string | null;
judgeThreshold: number;
executionOnly: boolean;
modeRunner: ModeRunner<TInitial, TExpected, TActual>;
totalCases: number;
verbose: boolean;
@@ -99,7 +103,9 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
try {
const initial = await input.modeRunner.loadInitial(input.evalCase.initialPath);
const expected = await input.modeRunner.loadExpected(input.evalCase.expectedPath);
const expected = input.executionOnly
? undefined
: await input.modeRunner.loadExpected(input.evalCase.expectedPath);
const run = await input.modeRunner.run(input.evalCase.prompt, initial, {
evalCase: input.evalCase,
caseId: input.evalCase.id,
@@ -162,22 +168,30 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
});
const checks: BenchmarkCheck[] = [
buildCheck("run succeeded", run.success, run.error),
...input.modeRunner.validate({
evalCase: input.evalCase,
prompt: input.evalCase.prompt,
initial,
expected,
actual: run.actual,
run,
}),
...validateToolExpectations({
run,
toolExpect: input.evalCase.toolExpect,
}),
];
if (!input.executionOnly) {
checks.push(
...input.modeRunner.validate({
evalCase: input.evalCase,
prompt: input.evalCase.prompt,
initial,
expected,
actual: run.actual,
run,
}),
...validateToolExpectations({
run,
toolExpect: input.evalCase.toolExpect,
})
);
}
const artifactFiles = input.modeRunner.buildArtifacts?.(run.actual) ?? [];
if (run.success && input.modeRunner.backendValidate) {
if (
run.success &&
!input.executionOnly &&
input.modeRunner.backendValidate
) {
try {
const backendValidation = await input.modeRunner.backendValidate({
evalCase: input.evalCase,
@@ -218,7 +232,12 @@ async function runCaseAttempts<TInitial, TExpected, TActual>(input: {
let judgeScore: number | null = null;
let judgeSummary: string | null = null;
if (run.success && !input.evalCase.skipJudge) {
if (
run.success &&
!input.executionOnly &&
input.judgeModel !== null &&
!input.evalCase.skipJudge
) {
const judge = await judgeOutput({
mode: input.modeRunner.mode,
prompt: input.evalCase.prompt,
+1 -1
View File
@@ -47,7 +47,7 @@ export interface FlowValidationSpec {
}>;
topLevelStepTypes?: Array<{
id: string;
type: string;
type: string | string[];
}>;
moduleRules?: Array<{
id: string;
+5 -2
View File
@@ -1378,11 +1378,14 @@ function validateFlowRequirements(
continue;
}
const allowedTypes = Array.isArray(requiredStep.type)
? requiredStep.type
: [requiredStep.type];
checks.push(
check(
`${requiredStep.id} type matches required`,
getModuleType(module) === requiredStep.type,
`expected ${requiredStep.type}, got ${getModuleType(module) ?? "(missing)"}`
allowedTypes.includes(getModuleType(module) ?? ""),
`expected ${allowedTypes.join(" or ")}, got ${getModuleType(module) ?? "(missing)"}`
)
);
}
@@ -0,0 +1,6 @@
{
"user": {
"username": "admin",
"is_admin": true
}
}
@@ -0,0 +1,8 @@
{
"user": {
"username": "admin",
"is_admin": true,
"folders": ["evals"],
"folders_read": ["evals"]
}
}
@@ -0,0 +1,8 @@
{
"user": {
"username": "admin",
"is_admin": true,
"folders": ["marketing", "data_engineering", "shared_utils"],
"folders_read": ["marketing", "data_engineering", "shared_utils"]
}
}
@@ -0,0 +1,8 @@
{
"user": {
"username": "bob",
"is_admin": false,
"folders": ["team_a"],
"folders_read": ["team_a", "team_b"]
}
}
+4
View File
@@ -4,6 +4,7 @@ import { loadAppFixtureForEval } from "../adapters/frontend/core/app/appFixtureL
import {
runGlobalEval,
type GlobalLiveEditorDraftFixture,
type GlobalUserFixture,
} from "../adapters/frontend/core/global/globalEvalRunner";
import type { BenchmarkWorkspaceRunnables } from "../adapters/frontend/mockBackend";
import type { FrontendEvalModelConfig } from "../core/models";
@@ -15,6 +16,7 @@ import { getFrontendApiKey } from "./frontendCommon";
export interface GlobalInitialFixture {
workspace?: BenchmarkWorkspaceRunnables;
liveEditorDrafts?: GlobalLiveEditorDraftFixture[];
user?: GlobalUserFixture;
}
export function createGlobalModeRunner(
@@ -38,6 +40,7 @@ export function createGlobalModeRunner(
{
workspaceFixtures: initial?.workspace,
liveEditorDrafts: initial?.liveEditorDrafts,
user: initial?.user,
maxIterations: context.evalCase?.runtime?.maxTurns,
provider: modelConfig.provider,
model: modelConfig.model,
@@ -104,6 +107,7 @@ async function loadGlobalInitialFixture(path: string): Promise<GlobalInitialFixt
return {
workspace: parsed.workspace ?? {},
liveEditorDrafts: parsed.liveEditorDrafts ?? [],
user: parsed.user,
};
}
@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n COUNT(*)::bigint AS \"total!\",\n COUNT(*) FILTER (WHERE name = ANY($2::text[]))::bigint AS \"replacing!\"\n FROM ai_skill\n WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "total!",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "replacing!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
null,
null
]
},
"hash": "002a606e71364b0581dbc496bf4337f276861dc71d2e277a7aef711543eb14d7"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM datatable_migrations WHERE workspace_id = $1 AND datatable = ANY($2::text[])",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": []
},
"hash": "00e9fc3fed9379264881c58df9404ab3569a0611b33c261602d62e9c847c583e"
}
@@ -0,0 +1,33 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE materialized_asset_schema\n SET snapshot_id = $5, job_id = $6, captured_at = now()\n WHERE workspace_id = $1 AND asset_kind = $2 AND asset_path = $3\n AND version = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
},
"Text",
"Int8",
"Int8",
"Uuid"
]
},
"nullable": []
},
"hash": "01732ca02b1888145c48c4e51e5b5829657224a743af9c0b2d5a140ad70e13dd"
}
@@ -1,38 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE (workspace_id, path, kind) IN (\n SELECT workspace_id, path, kind FROM (\n SELECT a.workspace_id, a.path, a.kind, a.usage_kind, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.usage_kind = 'job'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "02e526146f3584cd599dec708e1be48db3b0cd1c74adbfa2e4039377daa016f0"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET deploy_to = $1 WHERE deploy_to = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "0621faf69b1ef866a95f6310c9651875df409a2d2d72ada629bd71e8abdbbf8b"
}
@@ -46,11 +46,11 @@
]
},
"nullable": [
false,
false,
false,
false,
false,
true,
true,
true,
true,
true,
true,
true
]
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT SUM(pg_database_size(datname))::BIGINT AS \"v!\" FROM pg_database",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "v!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "074dd26f6427f4ff97e92c35163ad042144e656adcca56a4936a2eb196d3f48c"
}
@@ -35,7 +35,8 @@
"ci_test",
"github",
"azure",
"asset"
"asset",
"freshness"
]
}
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT args->>$2 FROM v2_job WHERE id = ANY($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"UuidArray",
"Text"
]
},
"nullable": [
null
]
},
"hash": "0d6c3399bc637e2c599534c2cafacf623cd6e5fde2d075a4f1f0d4852efcedca"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM pipeline_freshness_state WHERE workspace_id = $1 AND script_path = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "0e22295e40bfc173d85f0a962d491f3afd87eeea5e4e8d935b1b86eb02f061e7"
}
@@ -0,0 +1,77 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT ON (mp.asset_kind, mp.asset_path)\n mp.asset_kind AS \"asset_kind: AssetKind\", mp.asset_path,\n mp.snapshot_id AS \"snapshot_id!\", mp.partition\n FROM materialized_partition mp\n JOIN unnest($2::ASSET_KIND[], $3::text[]) AS u(kind, path)\n ON mp.asset_kind = u.kind AND mp.asset_path = u.path\n WHERE mp.workspace_id = $1\n AND mp.status = 'materialized' AND mp.snapshot_id IS NOT NULL\n ORDER BY mp.asset_kind, mp.asset_path, mp.snapshot_id DESC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "asset_kind: AssetKind",
"type_info": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
},
{
"ordinal": 1,
"name": "asset_path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "snapshot_id!",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "partition",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
}
}
}
},
"TextArray"
]
},
"nullable": [
false,
false,
true,
false
]
},
"hash": "0ee4a4dc8c6f09e7027bd99f98d8d1d346f9d67056ef272314dd9b80bee9f285"
}
@@ -0,0 +1,31 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n schedule.path, t.jobs FROM schedule,\n LATERAL(SELECT ARRAY(\n SELECT json_build_object('id', id, 'success', status = 'success', 'duration_ms', duration_ms)\n FROM v2_job_completed c JOIN v2_job j USING (id)\n WHERE trigger_kind = 'schedule'\n AND trigger = schedule.path\n AND c.workspace_id = $1\n AND j.workspace_id = $1\n AND parent_job IS NULL AND runnable_path = schedule.script_path\n AND status <> 'skipped'\n ORDER BY completed_at DESC\n LIMIT 20\n ) AS jobs) t\n WHERE workspace_id = $1 AND NOT starts_with(schedule.path, $4)\n ORDER BY edited_at DESC\n LIMIT $2 OFFSET $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "jobs",
"type_info": "JsonArray"
}
],
"parameters": {
"Left": [
"Text",
"Int8",
"Int8",
"Text"
]
},
"nullable": [
false,
null
]
},
"hash": "0f141ca6a58901dee1ddcf25694d76a4e5702c0324a27ef3a5740ef5d98d8946"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted, extra_perms)\n VALUES ('test-workspace', 'f/restricted/item', 314159, 'def main(): return 1', '', '', 'python3', 'test-user', NOW(), false, false, false, false, '{}'::jsonb)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "0ff0b7abad7717d7de398f629853b22cb831e91026dcd59d362a6d4382dc240b"
}
@@ -0,0 +1,58 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT language AS \"language!: _\", COUNT(*)::BIGINT AS \"count!\"\n FROM script\n WHERE archived = false AND deleted = false AND kind = 'script'\n AND (auto_kind IS NULL OR auto_kind <> 'wac')\n GROUP BY language\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "language!: _",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby",
"rlang"
]
}
}
}
},
{
"ordinal": 1,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
null
]
},
"hash": "11813108dbf6b104eba968c3609c74ac5a589542d3e76b1e756a82fb19d49ee8"
}
@@ -0,0 +1,46 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name, params, body, is_table_macro, provider_path FROM macro_definition WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "params",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "body",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "is_table_macro",
"type_info": "Bool"
},
{
"ordinal": 4,
"name": "provider_path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false,
false,
false,
false
]
},
"hash": "12dec9fa086fd231cdba2ecaa4585b62e55f43d448332ff88d8f036443b1edd7"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO schedule (\n workspace_id, path, edited_by, edited_at, schedule, enabled, script_path,\n args, extra_perms, is_flow, email, error, timezone, on_failure,\n on_recovery, on_failure_times, on_failure_exact, on_failure_extra_args,\n on_recovery_times, on_recovery_extra_args, ws_error_handler_muted, retry,\n summary, no_flow_overlap, tag, paused_until, on_success, on_success_extra_args,\n cron_version, description, dynamic_skip, permissioned_as, labels\n )\n SELECT\n $1, path, edited_by, edited_at, schedule, FALSE, script_path,\n args, extra_perms, is_flow, email, error, timezone, on_failure,\n on_recovery, on_failure_times, on_failure_exact, on_failure_extra_args,\n on_recovery_times, on_recovery_extra_args, ws_error_handler_muted, retry,\n summary, no_flow_overlap, tag, paused_until, on_success, on_success_extra_args,\n cron_version, description, dynamic_skip, permissioned_as, labels\n FROM schedule WHERE workspace_id = $2 AND path NOT LIKE $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "142ae6939440654da21ebb06acf838563f28e832e37e5c165329441d8471acdb"
}
@@ -1,26 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH completed AS (\n INSERT INTO v2_job_completed\n (workspace_id, id, started_at, duration_ms, result,\n flow_status, workflow_as_code_status, status, worker)\n SELECT\n q.workspace_id, q.id, q.started_at,\n (EXTRACT('epoch' FROM now()) - EXTRACT('epoch' FROM COALESCE(q.started_at, now()))) * 1000,\n CASE WHEN q.running\n THEN $3::text::jsonb\n ELSE $4::text::jsonb\n END,\n s.flow_status,\n s.workflow_as_code_status,\n 'skipped'::job_status,\n q.worker\n FROM v2_job_queue q\n LEFT JOIN v2_job_status s ON s.id = q.id\n WHERE q.id = $1\n ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, result = EXCLUDED.result\n RETURNING 1 AS x\n ), _deleted AS (\n DELETE FROM v2_job_queue WHERE id = $1\n ), _logged AS (\n INSERT INTO job_logs (logs, job_id, workspace_id)\n VALUES ($5, $1, $2)\n ON CONFLICT (job_id) DO UPDATE SET logs = concat(job_logs.logs, EXCLUDED.logs)\n )\n SELECT x FROM completed\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "x",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "1437b432d2c23e30eb05443e83069cdb049f65ec299b0778ce14677728cf6346"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM datatable_migrations WHERE workspace_id = $1 AND datatable = $2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "15c0584e9eb078442f6928adb894324c47e010e3554c14e755a7e045453745bb"
}
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, name FROM workspace WHERE parent_workspace_id = $1 AND is_dev_workspace AND deleted = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "165abf847773c7fe718fa6c832b259bf2dc531bc468d2ab27f76c9c653be0b9a"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT ws.datatable->'datatables' AS datatable_name\n FROM workspace_settings ws\n WHERE ws.workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "datatable_name",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "16a67b92dbd32024838983184e6974f2ce577b78decd6b821096c9f2f252ae8b"
}
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT (parent_workspace_id IS NOT NULL) AS \"has_parent!\", is_dev_workspace\n FROM workspace WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_parent!",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "is_dev_workspace",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null,
false
]
},
"hash": "16c7838ecfcea5fd231f2a4766f691a9a11ca7bb9797b81444419d6a72883531"
}
@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT kind::text as \"kind!\", parent_job, runnable_path\n FROM v2_job WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "kind!",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 2,
"name": "runnable_path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
null,
true,
true
]
},
"hash": "19f1cb1c7a1974920549917a6392ff0d56f31ca5662062f4a71fed0ccf859cc4"
}
@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_multipart_inflight\n (workspace_id, upload_id, storage, inflight_bytes, target_existing_size)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, upload_id)\n DO UPDATE SET inflight_bytes = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Int8",
"Int8"
]
},
"nullable": []
},
"hash": "1b05728b33decc39766ccacca50464b5ff9f34e43fbcc38154939461aadfca9f"
}
@@ -1,27 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow",
"job"
]
}
}
}
]
},
"nullable": []
},
"hash": "1c5caaaa86e3488549cad179e992172315be5d53dcc266d713da01fd27f310b6"
}
@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO workspace_protection_rule (workspace_id, name, rules, bypass_groups, bypass_users)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, name)\n DO UPDATE SET rules = EXCLUDED.rules,\n bypass_groups = EXCLUDED.bypass_groups,\n bypass_users = EXCLUDED.bypass_users\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int4",
"TextArray",
"TextArray"
]
},
"nullable": []
},
"hash": "1cd7c77e7a6a5c13c4ca521098bb07c1d805d21899fe0ebac22132b248ffd242"
}
@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT consumer_path AS \"consumer_path!\", macro_name AS \"macro_name!\"\n FROM macro_usage\n WHERE workspace_id = $1\n AND ($2::text IS NULL OR consumer_path LIKE $2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "consumer_path!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "macro_name!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "21242859ee6b72466c6bfb9d826a7159de4488adccb9349cddfbdd572b4005e9"
}
@@ -38,7 +38,9 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset",
"freshness"
]
}
}
@@ -75,7 +77,9 @@
"google",
"ci_test",
"github",
"azure"
"azure",
"asset",
"freshness"
]
}
}
@@ -1,12 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n kind AS \"kind!: AssetKind\",\n path AS \"path!\"\n FROM asset\n WHERE workspace_id = $1\n AND usage_kind = 'script'\n AND usage_path = $2\n AND usage_access_type IN ('w', 'rw')\n ",
"query": "SELECT version, columns AS \"columns: Json<Vec<SchemaColumn>>\"\n FROM materialized_asset_schema\n WHERE workspace_id = $1 AND asset_kind = $2 AND asset_path = $3\n ORDER BY version DESC\n LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "kind!: AssetKind",
"type_info": {
"name": "version",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "columns: Json<Vec<SchemaColumn>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
@@ -20,17 +31,7 @@
]
}
}
}
},
{
"ordinal": 1,
"name": "path!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
},
"Text"
]
},
@@ -39,5 +40,5 @@
false
]
},
"hash": "de06f44bad94710f14e9be4c0a6e6080e3c4faae5052500b93cc24b6fe556f2b"
"hash": "231475dc825518aa88f562698f8061d2562c3ac4af8d088f9eea9f0bc11d5fe7"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM macro_definition WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "246e302feda95892d5eb1a43a29d4d4739fd663cf34f00d947f24b47c481f035"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM native_retry_attempt nra WHERE NOT EXISTS (SELECT 1 FROM v2_job WHERE id = nra.job_id)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "255310c81beab0bc4cff13e966c84af7833264f208a94d20a5c8f216040c43cd"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script_trigger (workspace_id, runnable_kind, runnable_path, trigger_kind, trigger_ref, join_all, debounce_s, retry_count, retry_delay_s)\n SELECT $2, runnable_kind, runnable_path, trigger_kind, trigger_ref, join_all, debounce_s, retry_count, retry_delay_s\n FROM script_trigger WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Varchar"
]
},
"nullable": []
},
"hash": "256a302afef987857016f2fa636d15474599734bc46ede6930650e2cba5d8a46"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM job_perms\n WHERE ctid IN (\n SELECT jp.ctid FROM job_perms jp\n WHERE NOT EXISTS (SELECT 1 FROM v2_job_queue q WHERE q.id = jp.job_id)\n LIMIT 100000\n )",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "25ecae25ebc03d6296b0e72482a9201c2bffb0f0f7419b0b598b2786bdb326ab"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH RECURSIVE chain AS (\n SELECT id, parent_workspace_id, 0 AS depth\n FROM workspace WHERE id = $1\n UNION ALL\n SELECT w.id, w.parent_workspace_id, chain.depth + 1\n FROM workspace w\n JOIN chain ON w.id = chain.parent_workspace_id\n WHERE chain.depth < 20\n )\n SELECT COALESCE(MAX(depth), 0)::bigint AS \"depth!\" FROM chain\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "depth!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "27a131537cee699ae53088cfd710b70eb81b4e6b8f79d57db9d1d12d7f3ffde2"
}
@@ -1,12 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "SELECT 1",
"query": "SELECT NOT pg_is_in_recovery()",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
"type_info": "Bool"
}
],
"parameters": {
@@ -16,5 +16,5 @@
null
]
},
"hash": "e004ebd5b5532a4b85984a62f8ad48a81aa3460c1ca07701f386135d72cdecf5"
"hash": "282b56cfb8504312ac586cd4c1f3f914cf1651c08b8edd1b06d9a6454ed779bf"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM schedule WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": []
},
"hash": "28ec31481bdaae8b0512d21d074da5d913caa8d935b33ad8f21336dedefd1c23"
}
@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace (id, name, owner, deleted, premium, parent_workspace_id, is_dev_workspace)\n SELECT $1, $2, owner, false, premium,\n CASE WHEN $4 THEN parent_workspace_id ELSE NULL END, $5\n FROM workspace WHERE id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text",
"Bool",
"Bool"
]
},
"nullable": []
},
"hash": "29eb2c40e13d6e1ff7c37a05ab107829242f015a331bf15600986be3963878ee"
}
@@ -0,0 +1,62 @@
{
"db_name": "PostgreSQL",
"query": "SELECT version, columns AS \"columns: Json<Vec<SchemaColumn>>\",\n snapshot_id, job_id, captured_at\n FROM materialized_asset_schema\n WHERE workspace_id = $1 AND asset_kind = $2 AND asset_path = $3\n ORDER BY version DESC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "version",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "columns: Json<Vec<SchemaColumn>>",
"type_info": "Jsonb"
},
{
"ordinal": 2,
"name": "snapshot_id",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "job_id",
"type_info": "Uuid"
},
{
"ordinal": 4,
"name": "captured_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable",
"volume"
]
}
}
},
"Text"
]
},
"nullable": [
false,
false,
true,
true,
false
]
},
"hash": "2a28f13c1f06a77bc9a164393cccf0d8d2a3d7e8552d8ad09512d2d6bf27c3fb"
}
@@ -42,7 +42,8 @@
"trigger_cli",
"trigger_nextcloud",
"trigger_google",
"trigger_github"
"trigger_github",
"data_pipeline"
]
}
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE datatable_migrations SET datatable = $3 WHERE workspace_id = $1 AND datatable = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Varchar"
]
},
"nullable": []
},
"hash": "2c543583bcf7bcaf2f422638ce0741a2e193f18ffa11d08e8ca49cef5c3b850c"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH RECURSIVE tree AS (\n SELECT id, deleted, 0 AS depth FROM workspace WHERE id = $1\n UNION ALL\n SELECT w.id, w.deleted, tree.depth + 1 FROM workspace w\n JOIN tree ON w.parent_workspace_id = tree.id\n WHERE tree.depth < 20\n )\n SELECT COALESCE(MAX(depth) FILTER (WHERE NOT deleted), 0)::bigint AS \"height!\" FROM tree\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "height!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "2eb746c4cc5c65e277d4c3a12e0bc9b6a776a4941e68d8e2b31904c677493e2d"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork FROM workspace_diff\n WHERE source_workspace_id = $1 AND fork_workspace_id = $2",
"query": "SELECT path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork FROM workspace_diff\n WHERE source_workspace_id = $1 AND fork_workspace_id = $2\n AND NOT EXISTS (\n SELECT 1 FROM ws_specific ws\n WHERE ws.path = workspace_diff.path\n AND ws.item_kind = workspace_diff.kind\n AND ws.workspace_id IN (workspace_diff.source_workspace_id, workspace_diff.fork_workspace_id)\n )",
"describe": {
"columns": [
{
@@ -55,5 +55,5 @@
true
]
},
"hash": "0b8e5fe95f4a2855678ca041b50405b698a368626da42dd9f4ce9d0681d016a1"
"hash": "2ec9f88ad80d192a2066764222fdfed7c553de4df8e876ff8589738dea93d0cf"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO ws_specific (workspace_id, item_kind, path)\n SELECT $1::varchar, 'resource', $2::varchar\n WHERE EXISTS (SELECT 1 FROM resource WHERE workspace_id = $1::varchar AND path = $2::varchar)\n ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "3020d5477b4822f1b0e3b2e4f2947e24754b919f7ee1aa2e7c1cb8c36e9e94b1"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork)\n VALUES ('test-workspace', 'wm-fork-guard-test', 'f/restricted/item', 'script', 1, 0, true, true, true)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "3211c7631f46975fc0127f106e84676237a3b37ffa59ff6c9134df79eed3b680"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT email FROM usr WHERE workspace_id = $1 AND username = $2",
"query": "SELECT path FROM schedule WHERE workspace_id = $1 AND path LIKE $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"name": "path",
"type_info": "Varchar"
}
],
@@ -19,5 +19,5 @@
false
]
},
"hash": "dadf78bae0299b24f6798e7a2cc86f0ecfcf63daa0b185994a410eb2fe41fad9"
"hash": "333a703a53cc760842b047ad420efef25885a9bca64bb2ba4bb173bc19dcbdba"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM native_retry_attempt WHERE job_id = $1) AS \"exists!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "3419f7b1ec6dad074f1688cc790a42db9eb56c32047ab28457e04776c1bff76a"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT (parent_workspace_id IS NOT NULL) AS \"is_fork!\" FROM workspace WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "is_fork!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "34b94da001dafbbe66b3b945e71e33128bbc0dcdba9850ccbe4dc278836513bd"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM skip_workspace_diff_tally WHERE workspace_id IN ('test-workspace', 'wm-fork-guard-test')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "3535965aed37a1cc853b229e062d4aac2dbb2ef7596d819db222fb87187d9c4b"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT target_existing_size FROM workspace_multipart_inflight\n WHERE workspace_id = $1 AND upload_id = $2 LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "target_existing_size",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "3571fb1e1aee51850d5789e28025a055b6972eb00e4dc1818f7a68bcf04c1ba7"
}
@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT path AS \"asset_path!\", usage_path AS \"producer_path!\"\n FROM asset\n WHERE workspace_id = $1 AND kind = 'ducklake' AND path = ANY($2)\n AND usage_kind = 'script' AND usage_access_type IN ('w', 'rw')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "asset_path!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "producer_path!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false,
false
]
},
"hash": "36da136cf9b9554702e3b3ec364368a6862e35350ae43383fb2335e2c82a9419"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO macro_usage (workspace_id, consumer_path, macro_name)\n SELECT $2, consumer_path, macro_name\n FROM macro_usage WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Varchar"
]
},
"nullable": []
},
"hash": "36fb69f46ccfb559cd5a25db5fb35bd5a777effb1e61f530f3c1868b407e74e4"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT trigger_ref AS \"trigger_ref!\"\n FROM script_trigger\n WHERE workspace_id = $1\n AND runnable_path = $2\n AND trigger_kind = 'asset'\n AND runnable_kind = 'script'\n ORDER BY trigger_ref",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "trigger_ref!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "3703847d0cef9df3a480af8b6a5c21b6af282c68f06ce037a11f703284d8980d"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM macro_usage WHERE workspace_id = $1 AND (consumer_path = $2 OR macro_name IN (SELECT name FROM macro_definition WHERE workspace_id = $1 AND provider_path = $2))",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "3740ea8dfc666c8b098e55784af185331f8b03eec083fa80c007f6f23019a5d2"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO native_retry_attempt (job_id, attempt) VALUES ($1, $2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Int4"
]
},
"nullable": []
},
"hash": "39870bcb46af48191794e77d9205c6fb9518738e14ca13796395df05c7ab1c91"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job WHERE id = $1",
"query": "UPDATE v2_job_queue SET running = true WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
@@ -10,5 +10,5 @@
},
"nullable": []
},
"hash": "5190528997a879981a87420ddf3d28c978c8a5876f5c1ac1613391e86ffb550f"
"hash": "3a03aa24f77e5729c54fd896281da45d4873c38ef68f9ac480b220405fe1ade1"
}
@@ -0,0 +1,30 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n (SELECT COALESCE(SUM(bytes), 0) FROM workspace_storage_usage WHERE workspace_id = $1)::bigint as \"committed!\",\n (SELECT COALESCE(SUM(GREATEST(inflight_bytes - target_existing_size, 0)), 0)\n FROM workspace_multipart_inflight\n WHERE workspace_id = $1 AND upload_id <> $2 AND created_at > now() - ($3::text)::interval)::bigint as \"other_reserved!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "committed!",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "other_reserved!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null,
null
]
},
"hash": "3a20f8f159b7185940639716b2ae0d5d1c3769d095352002daf8c11d10eac57b"
}

Some files were not shown because too many files have changed in this diff Show More