Compare commits

..
Author SHA1 Message Date
claude[bot]andDiego Imbert 065b0efa85 Replace all structuredClone patterns with clone() utility
- Replace structuredClone($state.snapshot(x)) with clone(x)
- Replace structuredClone(stateSnapshot(x)) with clone(x)
- Replace structuredClone(x) with clone(x)
- Add clone import to 40+ files
- Remove unused stateSnapshot imports
- Fix syntax errors and import conflicts

Total: 101 structuredClone patterns replaced across codebase

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2025-06-26 13:52:00 +00:00
Diego Imbert 83e5bfbc17 clone impl 2025-06-26 11:36:36 +02:00
8117 changed files with 211335 additions and 1241436 deletions
-63
View File
@@ -1,63 +0,0 @@
# Vendored skills
These five skills are copied from an external repository, not written here:
- `grill-me`, `grilling`
- `improve-codebase-architecture`, `codebase-design`, `domain-modeling`
Source: https://github.com/mattpocock/skills
Pinned at commit `84fdeffd12f2ee307994d1eb6feb48173b6e0502`.
They form one dependency closure — `grill-me` is a stub that runs `grilling`, and
`improve-codebase-architecture` draws its vocabulary from `codebase-design` and its
CONTEXT.md upkeep from `domain-modeling`. Removing any one breaks the others.
Local changes on top of upstream, kept to the minimum so a refresh stays a diff:
- Flattened the upstream `skills/engineering/` and `skills/productivity/` split, since this
repo's skills are flat.
- Replaced each SKILL.md's markdown links to its own bundled files with plain repo-root paths
in prose (`.agents/skills/<skill>/FILE.md`). Upstream's sibling-relative links break when the
file is read through the `.claude/skills/<skill>/SKILL.md` symlink, which mirrors only
SKILL.md — and a repo-root *link* is equally wrong, since a markdown target resolves relative
to the file containing it. Companion files keep their sibling-relative links; they are only
ever read at their real path, never through the symlink.
- Dropped the upstream `agents/openai.yaml` files — Codex packaging metadata for that repo's
own plugin distribution, unused here.
- **Removed every ADR path.** Upstream, `domain-modeling` offers to write Architecture Decision
Records into `docs/adr/` and `improve-codebase-architecture` reads and cites them. This repo has
not adopted ADRs, and a skill that offers to create them is how the practice arrives by side
effect rather than by decision. Deleted `domain-modeling/ADR-FORMAT.md`, its "Offer ADRs
sparingly" section, and the `docs/adr/` entries in its file-structure diagrams; dropped the ADR
clauses from `improve-codebase-architecture` (intro, explore step, "ADR conflicts", the
offer-an-ADR bullet in the grilling loop) and the ADR callout row in `HTML-REPORT.md`. Also cut
"record an architectural decision" from `domain-modeling`'s description, since that phrase is an
invocation trigger. What remains is CONTEXT.md and ubiquitous-language work only.
To refresh, diff against the same paths at a newer commit and re-apply these four changes. The
ADR removal is the one that needs judgement: if the team later adopts ADRs, take upstream's
version of those sections back rather than rewriting them here.
## License
MIT License
Copyright (c) 2026 Matt Pocock
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-282
View File
@@ -1,282 +0,0 @@
---
name: adding-a-trigger
description: Checklist for adding a new TriggerCrud-based trigger type to Windmill (Azure, GCP, Kafka, etc.). Use when wiring a new trigger kind across backend, frontend, CLI, and capture infrastructure.
---
# Skill: Adding a New Trigger Type
Use this skill when adding a trigger kind that implements `TriggerCrud` (Kafka, GCP, Azure, MQTT, SQS, NATS, Postgres, Email…). For native triggers (Nextcloud, Google Drive — things wired through `windmill-native-triggers`), use the `native-trigger` skill instead.
The goal of this doc is to enumerate every file that needs to change. Missing any one of them leads to silent regressions: sync drops the trigger, capture button does nothing, workspace forks lose it, sidebar counters undercount. Follow the checklist top-to-bottom — each section is independent enough to be validated on its own.
Throughout this doc, substitute `{kind}` for the new trigger kind (`azure`, `kafka`, …), `{Kind}` for PascalCase (`Azure`, `Kafka`), `{KIND}` for SCREAMING (`AZURE`, `KAFKA`).
## Reference implementations
- **GCP** — closest analogue to Azure. Has push + pull, OIDC auth, ARM-like resource paths, capture handler. Grep for `gcp_trigger` / `GcpTrigger`.
- **Kafka** — simpler (pull-only, streaming). Good for trivial integrations.
- **Azure** — most recently added (2026). Shared-secret push auth, Event Grid namespaces + basic topics, ARM resource discovery, Namespace-pull data-plane. Grep for `azure_trigger` / `AzureTrigger`.
## 1. Database migration
Create a migration: `cargo sqlx migrate add -r add_{kind}_trigger` from `backend/`. Never write timestamps manually.
The `up.sql` usually defines:
- An optional enum type (e.g. `AZURE_MODE`) if the trigger has sub-kinds
- The `{kind}_trigger` table with at minimum these columns (mirrored from kafka/gcp):
- primary: `(workspace_id, path)`
- `script_path`, `is_flow`, `enabled`, `mode`, `permissioned_as`, `edited_by`, `email`
- `edited_at`, `error`, `server_id`, `last_server_ping`
- `error_handler_path`, `error_handler_args jsonb`, `retry jsonb`
- trigger-specific fields
- Indexes on foreign keys + any frequently-filtered columns
- Foreign key to `workspace`
- The RLS policies (`see_own`, `see_member`, `see_folder_extra_perms_user_*`, `see_extra_perms_user_*`, `see_extra_perms_groups_*`), copied from an existing trigger table
**RLS: wrap every session GUC read in a scalar sub-select.** Write the session
reads as `(select current_setting('session.user'))`,
`= any((select regexp_split_to_array(current_setting('session.groups'), ','))::text[])`,
`?| (select regexp_split_to_array(current_setting('session.pgroups'), ','))::text[]`,
`? (select concat('u/', current_setting('session.user')))`, etc. — not the bare
`current_setting(...)`. The GUCs are set with `SET LOCAL`, so the sub-select
hoists them to a one-time InitPlan instead of re-evaluating per scanned row.
Put the `::text[]` cast **outside** the sub-select for the array cases: in an
`= any (...)` context, casting inside — `= any((select ...::text[]))` — makes
Postgres parse the operand as a row-returning subquery and fails at CREATE with
`operator does not exist: text = text[]`. The outside cast keeps it in
array-operand form. See migration `20260714230440_wrap_session_gucs_in_rls_policies`
for the canonical wrapped forms.
Down migration drops the table and any enum types.
## 2. Backend crate (`windmill-trigger-{kind}`)
Create a new crate under `backend/windmill-trigger-{kind}/` with:
- `Cargo.toml`: features `enterprise`, `private` if EE, standard deps
- `src/lib.rs`: `pub use mod_ee::*;` behind `#[cfg(all(feature = "enterprise", feature = "private"))]`
- `src/mod_ee.rs`: core types + helpers
- `src/handler_ee.rs`: `TriggerCrud` impl + route handlers
- `src/listener_ee.rs`: (only if streaming/pull-based) `Listener` trait impl
Required in `mod_ee.rs`:
- `{Kind}Config` struct (persisted shape, `FromRow`)
- `{Kind}ConfigRequest` struct (what API receives — usually similar to Config but with validation fields)
- `{Kind}Trigger` unit struct (implements the traits)
- `impl TriggerJobArgs for {Kind}Trigger` — sets `TRIGGER_KIND`, `Payload`, `v1_payload_fn`
Required in `handler_ee.rs`:
- `#[async_trait] impl TriggerCrud for {Kind}Trigger` with:
- `type Trigger = Trigger<{Kind}Config>`
- `type TriggerConfigRequest = {Kind}ConfigRequest`
- `const ROUTE_PREFIX: &'static str = "/{kind}_triggers";`
- `const TABLE_NAME`, `ADDITIONAL_SELECT_FIELDS`
- `get_deployed_object`, `validate_config`, `create_trigger`, `update_trigger`, `delete_trigger`, `test_connection`
- `additional_routes` (optional — mount extra endpoints for things like ARM resource listing, topic discovery)
Register the crate in `backend/Cargo.toml` as a workspace member and as a dep of `windmill-api` behind the feature flag.
## 3. Wire into `windmill-api` (feature-gated everywhere)
**`backend/windmill-api/src/triggers/handler.rs`** — mount the trigger crate:
```rust
#[cfg(all(feature = "enterprise", feature = "{kind}_trigger", feature = "private"))]
{
use crate::triggers::{kind}::{Kind}Trigger;
router = router.nest({Kind}Trigger::ROUTE_PREFIX, complete_trigger_routes({Kind}Trigger));
}
```
**`backend/windmill-api/src/triggers/{kind}/mod.rs`** — re-export the crate:
```rust
pub use windmill_trigger_{kind}::*;
```
**`backend/windmill-api/src/lib.rs`** — if the trigger receives inbound pushes, add a webhook route:
```rust
.nest("/{kind}/w/{workspace_id}", {
#[cfg(all(feature = "enterprise", feature = "{kind}_trigger", feature = "private"))]
{ triggers::{kind}::handler_oss::{kind}_push_route_handler() }
#[cfg(not(...))]
{ Router::new() }
})
```
## 4. `TriggerKind` enum (`backend/windmill-types/src/triggers.rs`)
Already has slots for most triggers but verify your variant exists:
- Add `{Kind}` to the `TriggerKind` enum
- Add match arm in `to_key()`
- Add match arm in `from_str`
- Add match arm in `JobTriggerKind` (if jobs need kind tagging)
## 5. OpenAPI (`backend/windmill-api/openapi.yaml`)
This file is huge and the single most-forgotten place. Add:
- `/w/{workspace}/{kind}_triggers/create` + `/update/{path}` + `/delete/{path}` + `/get/{path}` + `/list` + `/exists/{path}` + `/setmode/{path}` + `/test` paths (mirror gcp section)
- Any `additional_routes` your handler exposes (resource discovery, etc.)
- Schemas: `{Kind}Trigger`, `{Kind}TriggerData`, `{Kind}Mode` (if enum), `{Kind}DeliveryConfig`, helper request/response types
- Add `{kind}` to `CaptureTriggerKind` enum
- Add `{kind}_used: boolean` to the `UsedTriggers` response schema
Regenerate frontend client: `npm run generate-backend-client` from `frontend/`.
## 6. `UsedTriggers` + workspace export
**`backend/windmill-api-workspaces/src/workspaces.rs`** — add `{kind}_used: bool` to the `UsedTriggers` struct and add an `EXISTS(SELECT 1 FROM {kind}_trigger …)` to the `get_used_triggers` query.
**`backend/windmill-api/src/workspaces_export.rs`** — add export block mirroring gcp's (export lists all triggers, serializes them to YAML/JSON). The block re-uses the `trigger_ignore_keys` variable so the new kind automatically participates in fork-export stripping (`mode` field is omitted when the source workspace is a fork — keeps fork→parent merges from flipping the parent's enabled state).
**Fork cloning (`clone_triggers_and_schedules` in workspaces.rs)** — add an `INSERT INTO {kind}_trigger ... SELECT ...` block that copies all rows from the parent workspace, forcing `mode = 'disabled'::TRIGGER_MODE`. Always runs at fork creation; forgetting this means users can't carry `{kind}` triggers into their forks.
## 6.5 Hardcoded trigger-kind arrays (silent-failure hotspots)
Several files keep **hardcoded arrays** of trigger kind strings. Miss one and ACL checks / user offboarding / trash drop your kind:
- **`backend/windmill-api-groups/src/granular_acls.rs`** — `KINDS: [&str; N]`. **Increment N** (the compile error is cryptic otherwise). Controls which kinds accept granular ACL operations.
- **`backend/windmill-api-users/src/users.rs`** (`extra_perms_tables`) — which tables get `extra_perms` entries cleaned when a user is deleted.
- **`backend/windmill-api/src/offboarding.rs`** — three separate arrays (enumeration, fork-copy, and delete paths). **All three** need the new kind.
- **`backend/windmill-api/src/trash.rs`** — `valid_tables` for the trash / restore API.
- **`backend/windmill-git-sync/src/lib.rs`** — add a test assertion for `DeployedObject::{Kind}Trigger.get_kind() == "{kind}_trigger"` (the `get_kind` match arm itself lives in the enum impl — already required by the Rust compiler).
- **`backend/windmill-api-auth/src/scopes.rs`** — add the `{Kind}Triggers` variant to `ScopeDomain` enum + `as_str` match + `from_str` match. Required for the OAuth/token system to recognise `{kind}_triggers:read|write` scopes.
- **`backend/windmill-api/src/token.rs`** (`build_trigger_scope_domains``TRIGGER_DOMAINS`) — add `("{kind}_triggers", "{Kind display name}")` so the CreateToken UI's scope selector surfaces the `read` / `write` checkboxes.
**OpenAPI enums** to extend (do NOT forget — generated client will allow it but server rejects as 400):
- `CaptureTriggerKind` enum
- Three `kind` enums under `/w/{workspace}/acls/{get,add,remove}/{kind}/{path}` (yes, same list repeated three times)
After editing any of these, run a full `cargo check` with your feature flag + `gcp_trigger` + other core flags — the `KINDS: [&str; N]` length mismatch only surfaces when the crate compiles.
## 7. Capture infrastructure (`backend/windmill-api/src/capture.rs`)
If the trigger supports push delivery, it also needs a capture endpoint so users can test it:
- `{Kind}TriggerConfig` struct (gated by feature flags)
- `TriggerConfig::{Kind}` variant
- `set_{kind}_trigger_config` function (creates the subscription/equivalent pointing at the capture URL — use your `manage_{kind}_subscription` helper with `trigger_mode=false`)
- Both real + no-op versions behind feature gates
- `TriggerKind::{Kind} => set_{kind}_trigger_config(...)` arm in `set_config`
- `{kind}_payload` async handler — validates auth (if any), processes payload, calls `insert_capture_payload`
- Route: `.route("/{kind}/{runnable_kind}/{*path}", post({kind}_payload))` inside `workspaced_unauthed_service` — and expand the surrounding `#[cfg(any(...))]` to include your feature flag
## 8. CLI (`cli/`) — easy to miss, breaks sync silently
Check all of these:
**`cli/src/types.ts`:**
- Add `"{kind}"` to `TRIGGER_TYPES` array
- Add `"{kind}_trigger"` to `getTypeStrFromPath` return union
- Add match case in `getTypeStrFromPath`'s `typeEnding ===` chain
- Add `pushTrigger("{kind}", ...)` branch in `pushObj`
**`cli/src/commands/trigger/trigger.ts`:**
- Import `{Kind}Trigger` type
- Add `{kind}: {Kind}Trigger` to the `Trigger` type map
- Add `{kind}: wmill.get{Kind}Trigger`, `update{Kind}Trigger`, `create{Kind}Trigger` to each function map
- Add `{kind}: { ... }` template to `triggerTemplates`
- Add `list{Kind}Triggers` call + spread in the `list` aggregation
- Update `--kind` option descriptions to mention the new kind
**`cli/src/commands/sync/sync.ts`:**
- Add `path.endsWith(".{kind}_trigger" + ext)` in the file-type filter
- Add `typ == "{kind}_trigger"` in `getTypeOrder`
- Add `"{kind}_trigger"` to the delete-suffix regex (~line 3092)
- Add a `case "{kind}_trigger"` in the delete switch
**`cli/src/guidance/skills.ts`** — **DO NOT EDIT DIRECTLY**. It's auto-generated by `system_prompts/generate.py`. Instead:
- Edit `system_prompts/utils.py` → append `('{Kind}Trigger', '{kind}_trigger')` to the `SCHEMA_MAPPINGS['triggers']` list (this is the master list — the one in `generate.py` is duplicated and `utils.py` wins)
- Then run `python3 system_prompts/generate.py` — it regenerates `cli/src/guidance/skills.ts` with the schema extracted from `backend/windmill-api/openapi.yaml`
- Commit the regenerated file
## 9. Frontend — editor + drawer
Under `frontend/src/lib/components/triggers/{kind}/`:
- `{Kind}TriggerPanel.svelte` — the tile shown in the triggers listing
- `{Kind}TriggerEditor.svelte` — outer drawer wrapper
- `{Kind}TriggerEditorInner.svelte` — state + business logic; must expose:
- `openEdit(path, isFlow, defaultValues?)` method
- `isEditor` prop, `onConfigChange` + `onCaptureConfigChange` callbacks
- `get{Kind}Config()` + `get{Kind}CaptureConfig()` helpers
- `captureConfig = $derived.by(untrack(() => isEditor) ? get{Kind}CaptureConfig : () => ({}))`
- `$effect(() => { const args = [captureConfig, isValid] as const; untrack(() => onCaptureConfigChange?.(...args)) })`
- `{Kind}TriggerEditorConfigSection.svelte` — form fields; use design-system components (`TextInput`, `Select`, `Toggle`, `ToggleButtonGroup`), never raw `<input>`
- `{Kind}Capture.svelte` — capture panel; wraps `CaptureSection` with `captureType="{kind}"`
- `utils.ts``requestBody` builders and any trigger-type-specific helpers
## 10. Frontend — global integration
Easy to miss:
- **`frontend/src/lib/components/triggers.ts`** — add `'{kind}'` to the `TriggerKind` union
- **`frontend/src/lib/components/triggers/CaptureWrapper.svelte`**:
- Import `{Kind}Capture`
- Add to `isStreamingCapture()` array (streaming = pull-style; push-style is typically `false`)
- Add `{:else if captureType === '{kind}'}` branch with the `<{Kind}Capture>` render
- **`frontend/src/lib/components/sidebar/SidebarContent.svelte`** — import the icon, add the nav entry
- **`frontend/src/lib/components/sidebar/OperatorMenu.svelte`** — add the operator-mode entry
- **`frontend/src/routes/(root)/(logged)/+layout.svelte`** — destructure `{kind}_used` from `/get_used_triggers` response, push `'{kind}'` into `usedKinds`
- **`frontend/src/lib/components/search/GlobalSearchModal.svelte`** — import icon, add "Go to {Kind} ..." entry
- **`frontend/src/lib/components/offboarding-utils.ts`** — add mappings `{kind}_trigger: '{kind}_triggers'` and `{kind}_trigger: '{kind} trigger'`
- **`frontend/src/lib/components/icons/{Kind}Icon.svelte`** — single-path SVG, `fill={color ?? 'currentColor'}`, `size` prop default 16 (match existing icons — don't hardcode colors, don't use `width`/`height` props)
- **`frontend/src/routes/(root)/(logged)/{kind}_triggers/+page.svelte`** — listing page (mirror `gcp_triggers/+page.svelte` for push+pull, `kafka_triggers` for pure streaming)
- **`frontend/src/lib/components/CompareWorkspaces.svelte`** — workspace fork / compare tool. Needs: service import, editor import, `{kind}Editor` `$state`, `case '{kind}'` in `openTriggerDetails()`, entry in `triggerServices` object (list/delete/normalize), and `<{Kind}TriggerEditor bind:this={{kind}Editor} />` in the template
## 10.5 AI system prompts (`system_prompts/`)
- **`system_prompts/utils.py`** — append `('{Kind}Trigger', '{kind}_trigger')` to `SCHEMA_MAPPINGS['triggers']` (master list used by code generation + CLI skills)
- **`system_prompts/generate.py`** — also has a duplicated `schema_types` list (~line 903) for the AI `triggers` skill content. Add `('{Kind}Trigger', '{kind}_trigger')` there too
- **`system_prompts/generate.py`** `schema_names` (~line 1192) — add `'{Kind}Trigger'` (add `'New{Kind}Trigger'` only if the OpenAPI declares one; GCP and Azure don't)
- Run `python3 system_prompts/generate.py` — this rewrites `cli/src/guidance/skills.ts` and all `auto-generated/` docs. Commit the regenerated files
## 11. Validation
Run all of these before declaring done:
```bash
# Backend
cd backend
cargo check --features enterprise,{kind}_trigger,private # minimal
cargo check --features enterprise,azure_trigger,private,gcp_trigger,http_trigger,mqtt_trigger,postgres_trigger,sqs_trigger,kafka,nats,smtp,websocket # full
# SQLx offline data (never run `cargo sqlx prepare` directly — use the wrapper)
./update_sqlx.sh
# Frontend
cd frontend
npm run generate-backend-client
npm run check:fast
```
Smoke test in the UI: create a trigger, save, check it appears in sidebar + search, delete, re-create via CLI `wmill sync`.
## 12. Common pitfalls
- **Forgetting feature gates in `workspaced_unauthed_service()`** — the surrounding `#[cfg(any(...))]` expression must include your feature flag, not just the inner `#[cfg]` on the route
- **`.route(path, ...).route(path, ...)` with same path and different methods** — older axum replaced; use `.route(path, post(h1).options(h2))` to chain methods on the same `MethodRouter`
- **`on:event` directives** — legacy Svelte 4, no-op in runes mode. Use callback props (`onSelected`, `onConfigChange`)
- **`$bindable(default_value)` on optional props** — banned by project CLAUDE.md. Use `$bindable()` + `$derived(prop ?? default)` instead
- **CORS layer intercepting OPTIONS** — tower-http CorsLayer short-circuits OPTIONS before reaching your handler. For server-to-server webhook endpoints, drop the CORS layer entirely (CORS is browser-only)
- **DeliveryAttributeMappings / custom headers for auth** — prefer HMAC or sha256-hashed shared secrets over opaque JWTs when the provider doesn't support signed tokens natively. Store only the hash; regenerate secret on every save
- **ARM / API resource-listing cascades** — if the trigger's resource type is deep (Azure: subscription → RG → namespace → topic), offer dropdowns in the UI populated from the provider's APIs using the user's credential resource
- **Clearing stale selections on dependency change** — when a dropdown's underlying data reloads (e.g., user changes SP or edition), clear selections that no longer match the new list
- **Workspace-scoped tag compatibility** — if the trigger has tags, verify forked workspaces handle them (see commit `0773b5bc85` for a historical fix)
## 13. EE file split
If the trigger is enterprise-only, the code lives in `windmill-ee-private__worktrees/.../windmill-trigger-{kind}/src/*_ee.rs` and is symlinked into the OSS tree. The `windmill-ee-private__worktrees/` directory holds the real files; changes propagate via symlinks. See `docs/enterprise.md` for the workflow.
## 14. Final checklist before PR
- [ ] Migration up/down tested (revert + re-apply)
- [ ] `./update_sqlx.sh` committed the updated `.sqlx/` offline data
- [ ] `cargo check` passes with your feature flag + with all trigger features
- [ ] `npm run check:fast` passes
- [ ] Trigger visible in sidebar with correct icon weight (not oversized/colored — use `currentColor`)
- [ ] Create, edit, delete flow all work in the UI
- [ ] Capture button works (if push-capable)
- [ ] Trigger appears in `/get_used_triggers` → sidebar pulse
- [ ] `wmill sync pull` + `wmill sync push` both round-trip the trigger
- [ ] `wmill trigger list` includes it
- [ ] OpenAPI schemas are complete (no `null` in generated types)
-40
View File
@@ -1,40 +0,0 @@
---
name: ai-chat
description: Guidance for improving the Windmill AI chat (copilot), especially global mode — tools, prompts, and context-window discipline. Use when editing chat tools, system prompts, or tool-result shapes under frontend/src/lib/components/copilot/chat, or when changing how the chat manages its context window.
---
## Always benchmark before and after
No context or behavior change ships without an `ai_evals` A/B on the affected mode.
Add or adjust cases for exactly what you changed — see the `ai-evals` skill for
authoring and the full run reference.
Run the affected mode **before** your change and **after**, same model(s), same cases.
## Measure the window first, and cumulative second
Optimize **`finalContextTokens`** (window occupancy — what drives overflow and
compaction), then cumulative prompt tokens.
## Context discipline
The dominant fixed cost is per-iteration overhead: the system prompt **plus every
tool schema** is re-sent on every loop iteration. So:
- **Every tool and every parameter is a permanent tax.** Justify each one and measure
it; an extra "locate" round-trip can cost more than the reads it saves. Strip dead
params rather than leaving them in the schema.
- **Tool results return the minimum.** Never echo content the model already has. The
canonical mistake: a write tool that returns the whole edited artifact right after
the model authored it — return `{ success, message }` instead. When you touch a
*shared* write helper (e.g. `finishAppDraftWrite` in `global/core.ts`), re-check
this invariant for **all** the write tools routing through it — the echo has
regressed before via a shared refactor.
## Prompts and tool descriptions are part of the surface
The system prompt and tool descriptions steer behavior as much as the tools
themselves, and are benchmarkable the same way. A description that advertises
truncation makes the model self-limit; the path-conventions block changes where
drafts land. Treat prompt/description edits as real changes and A/B them — a
pure-prompt change is a legitimate, measurable improvement.
-87
View File
@@ -1,87 +0,0 @@
---
name: ai-evals
description: Author and run black-box benchmark cases for the Windmill AI generation modes (flow/app/script/cli/global) in ai_evals/. Use when adding or changing eval cases, or when running before/after benchmarks for AI chat / copilot changes.
---
# AI evals — authoring and running benchmark cases
`ai_evals/` is a black-box benchmark runner for the Windmill AI generation modes:
`flow`, `app`, `script`, `cli`, `global`. It always tests the **current** production
prompts, tools, and guidance in this checkout. Each attempt runs the real production
path, deterministic validation, then LLM judging.
The goal is to test current production guidance with realistic user requests — **not**
to pin one exact implementation shape.
## Running benchmarks
```bash
cd ai_evals
bun install # first time; frontend modes also need `cd frontend && bun install`
bun run cli -- models # list model aliases
bun run cli -- cases global # list cases for a mode
bun run cli -- run global global-test1-script-create --model sonnet
```
Frontend modes (`flow`/`script`/`app`/`global`) route model calls through a Windmill
backend's `/api/w/<ws>/ai/proxy`, so you need **any** reachable backend:
```bash
WMILL_AI_EVAL_BACKEND_URL=http://127.0.0.1:<port> WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests \
bun run cli -- run global <caseIds...> --models sonnet,gpt-5.5,gemini-3.1-pro-preview
```
- **Reuse an existing workspace.** CE builds cap workspaces, so temp-workspace
creation 400s ("reached workspace limit"). Always set
`WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests` (or any existing workspace) to
reuse one. The only side effect of a run is upserting an `f/evals/ai/<provider>`
resource there.
- Provider keys live in `ai_evals/.env` and are auto-loaded by bun. The judge is a
separate Anthropic call (default `claude-sonnet-4-6`) regardless of the model under
test.
## Authoring core rules
1. Write prompts like a real user request.
2. Prefer behavior, inputs, constraints, and outcomes over internal implementation.
3. Keep deterministic validation narrow and hard.
4. Put semantic expectations in `judgeChecklist`.
5. Use `expected` fixtures only when exact structure really matters.
### Prompt writing
Prompts should sound like something a user would naturally ask. Do not write prompts
as if the user knows Windmill internals unless the case explicitly tests a power-user
workflow.
Good:
- "Create a flow that routes support requests based on customer tier."
- "Add a reset button that sets the counter back to 0."
- "Create a flow that reuses the existing greeting script instead of duplicating the logic."
Bad:
- "Use `branchone` with 3 branches and a default branch."
- "Create a `rawscript` step with this exact topology."
- "This is a benchmark harness."
### Deterministic validation
Use deterministic checks only for hard failures: missing required files; unexpected
extra files when the prompt says not to create them; syntax errors; unresolved flow
refs; missing required special modules or suspend config; obvious corruption.
Do **not** encode one preferred implementation. Bad hard checks: exact step topology
for a creation flow; exact branch structure when the prompt only asked for routing;
exact input shape when multiple reasonable shapes are acceptable.
### Judge checklist
Every non-trivial case should have a `judgeChecklist` capturing user-visible behavior
that must be present, important constraints, and key completion criteria — not
low-level implementation details unless truly required.
Good: "the flow calculates the order total with 8% tax"; "the flow reuses the existing
workspace script instead of rewriting the logic". Bad: "uses `branchone`"; "contains a
`rawscript` node".
See `ai_evals/README.md` for the full case format, fields, and fixture details.
@@ -1,37 +0,0 @@
# Deepening
How to deepen a cluster of shallow modules safely, given its dependencies. Assumes the vocabulary in [SKILL.md](SKILL.md) — **module**, **interface**, **seam**, **adapter**.
## Dependency categories
When assessing a candidate for deepening, classify its dependencies. The category determines how the deepened module is tested across its seam.
### 1. In-process
Pure computation, in-memory state, no I/O. Always deepenable — merge the modules and test through the new interface directly. No adapter needed.
### 2. Local-substitutable
Dependencies that have local test stand-ins (PGLite for Postgres, in-memory filesystem). Deepenable if the stand-in exists. The deepened module is tested with the stand-in running in the test suite. The seam is internal; no port at the module's external interface.
### 3. Remote but owned (Ports & Adapters)
Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Tests use an in-memory adapter. Production uses an HTTP/gRPC/queue adapter.
Recommendation shape: *"Define a port at the seam, implement an HTTP adapter for production and an in-memory adapter for testing, so the logic sits in one deep module even though it's deployed across a network."*
### 4. True external (Mock)
Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port; tests provide a mock adapter.
## Seam discipline
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified (typically production + test). A single-adapter seam is just indirection.
- **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them.
## Testing strategy: replace, don't layer
- Old unit tests on shallow modules become waste once tests at the deepened module's interface exist — delete them.
- Write new tests at the deepened module's interface. The **interface is the test surface**.
- Tests assert on observable outcomes through the interface, not internal state.
- Tests should survive internal refactors — they describe behaviour, not implementation. If a test has to change when the implementation changes, it's testing past the interface.
@@ -1,44 +0,0 @@
# Design It Twice
When the user wants to explore alternative interfaces for a chosen deepening candidate, use this parallel sub-agent pattern. Based on "Design It Twice" (Ousterhout) — your first idea is unlikely to be the best.
Uses the vocabulary in [SKILL.md](SKILL.md) — **module**, **interface**, **seam**, **adapter**, **leverage**.
## Process
### 1. Frame the problem space
Before spawning sub-agents, write a user-facing explanation of the problem space for the chosen candidate:
- The constraints any new interface would need to satisfy
- The dependencies it would rely on, and which category they fall into (see [DEEPENING.md](DEEPENING.md))
- A rough illustrative code sketch to ground the constraints — not a proposal, just a way to make the constraints concrete
Show this to the user, then immediately proceed to Step 2. The user reads and thinks while the sub-agents work in parallel.
### 2. Spawn sub-agents
Spawn 3+ sub-agents in parallel. Each must produce a **radically different** interface for the deepened module.
Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint:
- Agent 1: "Minimize the interface — aim for 13 entry points max. Maximise leverage per entry point."
- Agent 2: "Maximise flexibility — support many use cases and extension."
- Agent 3: "Optimise for the most common caller — make the default case trivial."
- Agent 4 (if applicable): "Design around ports & adapters for cross-seam dependencies."
Include both [SKILL.md](SKILL.md) vocabulary and CONTEXT.md vocabulary in the brief so each sub-agent names things consistently with the architecture language and the project's domain language.
Each sub-agent outputs:
1. Interface (types, methods, params — plus invariants, ordering, error modes)
2. Usage example showing how callers use it
3. What the implementation hides behind the seam
4. Dependency strategy and adapters (see [DEEPENING.md](DEEPENING.md))
5. Trade-offs — where leverage is high, where it's thin
### 3. Present and compare
Present designs sequentially so the user can absorb each one, then compare them in prose. Contrast by **depth** (leverage at the interface), **locality** (where change concentrates), and **seam placement**.
After comparing, give your own recommendation: which design you think is strongest and why. If elements from different designs would combine well, propose a hybrid. Be opinionated — the user wants a strong read, not a menu.
-114
View File
@@ -1,114 +0,0 @@
---
name: codebase-design
description: Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary.
---
# Codebase Design
Design **deep modules**: a lot of behaviour behind a small interface, placed at a clean seam, testable through that interface. Use this language and these principles wherever code is being designed or restructured. The aim is leverage for callers, locality for maintainers, and testability for everyone.
## Glossary
Use these terms exactly — don't substitute "component," "service," "API," or "boundary." Consistent language is the whole point.
**Module** — anything with an interface and an implementation. Deliberately scale-agnostic: a function, class, package, or tier-spanning slice. _Avoid_: unit, component, service.
**Interface** — everything a caller must know to use the module correctly: the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics. _Avoid_: API, signature (too narrow — they refer only to the type-level surface).
**Implementation** — what's inside a module, its body of code. Distinct from **Adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Reach for "adapter" when the seam is the topic; "implementation" otherwise.
**Depth** — leverage at the interface: the amount of behaviour a caller (or test) can exercise per unit of interface they have to learn. A module is **deep** when a large amount of behaviour sits behind a small interface, **shallow** when the interface is nearly as complex as the implementation.
**Seam** _(Michael Feathers)_ — a place where you can alter behaviour without editing in that place; the *location* at which a module's interface lives. Where to put the seam is its own design decision, distinct from what goes behind it. _Avoid_: boundary (overloaded with DDD's bounded context).
**Adapter** — a concrete thing that satisfies an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside).
**Leverage** — what callers get from depth: more capability per unit of interface they learn. One implementation pays back across N call sites and M tests.
**Locality** — what maintainers get from depth: change, bugs, knowledge, and verification concentrate in one place rather than spreading across callers. Fix once, fixed everywhere.
## Deep vs shallow
**Deep module** = small interface + lots of implementation:
```
┌─────────────────────┐
│ Small Interface │ ← Few methods, simple params
├─────────────────────┤
│ │
│ Deep Implementation│ ← Complex logic hidden
│ │
└─────────────────────┘
```
**Shallow module** = large interface + little implementation (avoid):
```
┌─────────────────────────────────┐
│ Large Interface │ ← Many methods, complex params
├─────────────────────────────────┤
│ Thin Implementation │ ← Just passes through
└─────────────────────────────────┘
```
When designing an interface, ask:
- Can I reduce the number of methods?
- Can I simplify the parameters?
- Can I hide more complexity inside?
## Principles
- **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, mockable, swappable parts — they just aren't part of the interface. A module can have **internal seams** (private to its implementation, used by its own tests) as well as the **external seam** at its interface.
- **The deletion test.** Imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep.
- **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape.
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it.
## Designing for testability
Good interfaces make testing natural:
1. **Accept dependencies, don't create them.**
```typescript
// Testable
function processOrder(order, paymentGateway) {}
// Hard to test
function processOrder(order) {
const gateway = new StripeGateway();
}
```
2. **Return results, don't produce side effects.**
```typescript
// Testable
function calculateDiscount(cart): Discount {}
// Hard to test
function applyDiscount(cart): void {
cart.total -= discount;
}
```
3. **Small surface area.** Fewer methods = fewer tests needed. Fewer params = simpler test setup.
## Relationships
- A **Module** has exactly one **Interface** (the surface it presents to callers and tests).
- **Depth** is a property of a **Module**, measured against its **Interface**.
- A **Seam** is where a **Module**'s **Interface** lives.
- An **Adapter** sits at a **Seam** and satisfies the **Interface**.
- **Depth** produces **Leverage** for callers and **Locality** for maintainers.
## Rejected framings
- **Depth as ratio of implementation-lines to interface-lines** (Ousterhout): rewards padding the implementation. We use depth-as-leverage instead.
- **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow — interface here includes every fact a caller must know.
- **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**.
## Going deeper
- **Deepening a cluster given its dependencies** — see `.agents/skills/codebase-design/DEEPENING.md` (path from the repo root): dependency categories, seam discipline, and replace-don't-layer testing.
- **Exploring alternative interfaces** — see `.agents/skills/codebase-design/DESIGN-IT-TWICE.md` (path from the repo root): spin up parallel sub-agents to design the interface several radically different ways, then compare on depth, locality, and seam placement.
-58
View File
@@ -1,58 +0,0 @@
---
name: commit
user_invocable: true
description: Create a git commit with conventional commit format. MUST use anytime you want to commit changes.
---
# Git Commit Skill
Create a focused, single-line commit following conventional commit conventions.
## Instructions
1. **Analyze changes**: Run `git status` and `git diff` to understand what was modified
2. **Stage only modified files**: Add files individually by name. NEVER use `git add -A` or `git add .`
3. **Write commit message**: Follow the conventional commit format as a single line
## Conventional Commit Format
```
<type>: <description>
```
### Types
- `feat`: New feature or capability
- `fix`: Bug fix
- `refactor`: Code change that neither fixes a bug nor adds a feature
- `docs`: Documentation only changes
- `style`: Formatting, missing semicolons, etc (no code change)
- `test`: Adding or correcting tests
- `chore`: Maintenance tasks, dependency updates, etc
- `perf`: Performance improvement
### Rules
- Message MUST be a single line (no multi-line messages)
- Description should be lowercase, imperative mood ("add" not "added")
- No period at the end
- Keep under 72 characters total
### Examples
```
feat: add token usage tracking for AI providers
fix: resolve null pointer in job executor
refactor: extract common validation logic
docs: update API endpoint documentation
chore: upgrade sqlx to 0.7
```
## Execution Steps
1. Run `git status` to see all changes
2. Run `git diff` to understand the changes in detail
3. Run `git log --oneline -5` to see recent commit style
4. Stage ONLY the modified/relevant files: `git add <file1> <file2> ...`
5. Create the commit with conventional format:
```bash
git commit -m "<type>: <description>"
```
6. Run `git status` to verify the commit succeeded
@@ -1,60 +0,0 @@
# CONTEXT.md Format
## Structure
```md
# {Context Name}
{One or two sentence description of what this context is and why it exists.}
## Language
**Order**:
{A one or two sentence description of the term}
_Avoid_: Purchase, transaction
**Invoice**:
A request for payment sent to a customer after delivery.
_Avoid_: Bill, payment request
**Customer**:
A person or organization that places orders.
_Avoid_: Client, buyer, account
```
## Rules
- **Be opinionated.** When multiple words exist for the same concept, pick the best one and list the others under `_Avoid_`.
- **Keep definitions tight.** One or two sentences max. Define what it IS, not what it does.
- **Only include terms specific to this project's context.** General programming concepts (timeouts, error types, utility patterns) don't belong even if the project uses them extensively. Before adding a term, ask: is this a concept unique to this context, or a general programming concept? Only the former belongs.
- **Group terms under subheadings** when natural clusters emerge. If all terms belong to a single cohesive area, a flat list is fine.
## Single vs multi-context repos
**Single context (most repos):** One `CONTEXT.md` at the repo root.
**Multiple contexts:** A `CONTEXT-MAP.md` at the repo root lists the contexts, where they live, and how they relate to each other:
```md
# Context Map
## Contexts
- [Ordering](./src/ordering/CONTEXT.md) — receives and tracks customer orders
- [Billing](./src/billing/CONTEXT.md) — generates invoices and processes payments
- [Fulfillment](./src/fulfillment/CONTEXT.md) — manages warehouse picking and shipping
## Relationships
- **Ordering → Fulfillment**: Ordering emits `OrderPlaced` events; Fulfillment consumes them to start picking
- **Fulfillment → Billing**: Fulfillment emits `ShipmentDispatched` events; Billing consumes them to generate invoices
- **Ordering ↔ Billing**: Shared types for `CustomerId` and `Money`
```
The skill infers which structure applies:
- If `CONTEXT-MAP.md` exists, read it to find contexts
- If only a root `CONTEXT.md` exists, single context
- If neither exists, create a root `CONTEXT.md` lazily when the first term is resolved
When multiple contexts exist, infer which one the current topic relates to. If unclear, ask.
-57
View File
@@ -1,57 +0,0 @@
---
name: domain-modeling
description: Build and sharpen a project's domain model. Use when the user wants to pin down domain terminology or a ubiquitous language, or when another skill needs to maintain the domain model.
---
# Domain Modeling
Actively build and sharpen the project's domain model as you design. This is the *active* discipline — challenging terms, inventing edge-case scenarios, and writing the glossary and decisions down the moment they crystallise. (Merely *reading* `CONTEXT.md` for vocabulary is not this skill — that's a one-line habit any skill can do. This skill is for when you're changing the model, not just consuming it.)
## File structure
Most repos have a single context:
```
/
├── CONTEXT.md
└── src/
```
If a `CONTEXT-MAP.md` exists at the root, the repo has multiple contexts. The map points to where each one lives:
```
/
├── CONTEXT-MAP.md
└── src/
├── ordering/
│ └── CONTEXT.md
└── billing/
└── CONTEXT.md
```
Create files lazily — only when you have something to write. If no `CONTEXT.md` exists, create one when the first term is resolved.
## During the session
### Challenge against the glossary
When the user uses a term that conflicts with the existing language in `CONTEXT.md`, call it out immediately. "Your glossary defines 'cancellation' as X, but you seem to mean Y — which is it?"
### Sharpen fuzzy language
When the user uses vague or overloaded terms, propose a precise canonical term. "You're saying 'account' — do you mean the Customer or the User? Those are different things."
### Discuss concrete scenarios
When domain relationships are being discussed, stress-test them with specific scenarios. Invent scenarios that probe edge cases and force the user to be precise about the boundaries between concepts.
### Cross-reference with code
When the user states how something works, check whether the code agrees. If you find a contradiction, surface it: "Your code cancels entire Orders, but you just said partial cancellation is possible — which is right?"
### Update CONTEXT.md inline
When a term is resolved, update `CONTEXT.md` right there. Don't batch these up — capture them as they happen. Use the format in `.agents/skills/domain-modeling/CONTEXT-FORMAT.md` (path from the repo root).
`CONTEXT.md` should be totally devoid of implementation details. Do not treat `CONTEXT.md` as a spec, a scratch pad, or a repository for implementation decisions. It is a glossary and nothing else.
-7
View File
@@ -1,7 +0,0 @@
---
name: grill-me
description: A relentless interview to sharpen a plan or design.
disable-model-invocation: true
---
Run a `/grilling` session.
-22
View File
@@ -1,22 +0,0 @@
---
name: grilling
description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases.
---
Interview the user relentlessly until you reach a shared understanding. Map this as a **design tree**: every decision branches into the decisions that hang off it.
Work the tree in **rounds**. The **frontier** is every decision whose prerequisites are already settled — the questions you can ask _now_ without guessing at answers you haven't heard yet. Ask the whole frontier in one round: number each question and give your recommended answer. Then wait for the user's answers before the next round.
Each question should be formatted like so:
```
❓ **Q1** - **<question title>**: <question body, might be multiple paragraphs, including multiple choices>
➡️ <your recommended answer>
```
Each round the user answers reshapes the tree — settled decisions push the frontier outward and unblock questions that depended on them. Recompute the frontier and ask the next round. A question whose answer depends on another question still open in this round belongs to a _later_ round, not this one.
Finding _facts_ is your job, never the user's. When a frontier question needs a fact from the environment (filesystem, tools, etc.), dispatch a sub-agent to find it — don't ask the user for anything you could look up yourself. Don't block on it: a running exploration is an unsettled prerequisite, so only the questions downstream of it wait for the sub-agent to report — ask the rest of the frontier now. The _decisions_ are the user's — put each to them and wait.
The session is done when the frontier is empty: every branch of the design tree visited, nothing left silently assumed. Do not act on it until the user confirms you have reached a shared understanding.
@@ -1,122 +0,0 @@
# HTML Report Format
The architectural review is rendered as a single self-contained HTML file in the OS temp directory. Tailwind and Mermaid both come from CDNs. Mermaid handles graph-shaped diagrams reliably; hand-built divs and inline SVG handle the more editorial visuals (mass diagrams, cross-sections). Mix the two — don't lean on Mermaid for everything, it'll start to look generic.
## Scaffold
```html
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>Architecture review — {{repo name}}</title>
<script src="https://cdn.tailwindcss.com"></script>
<script type="module">
import mermaid from "https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs";
mermaid.initialize({ startOnLoad: true, theme: "neutral", securityLevel: "loose" });
</script>
<style>
/* small custom layer for things Tailwind doesn't cover cleanly:
dashed seam lines, hand-drawn-feeling arrow heads, etc. */
.seam { stroke-dasharray: 4 4; }
.leak { stroke: #dc2626; }
.deep { background: linear-gradient(135deg, #0f172a, #1e293b); }
</style>
</head>
<body class="bg-stone-50 text-slate-900 font-sans">
<main class="max-w-5xl mx-auto px-6 py-12 space-y-12">
<header>...</header>
<section id="candidates" class="space-y-10">...</section>
<section id="top-recommendation">...</section>
</main>
</body>
</html>
```
## Header
Repo name, date, and a compact legend: solid box = module, dashed line = seam, red arrow = leakage, thick dark box = deep module. No introduction paragraph — straight into the candidates.
## Candidate card
The diagrams carry the weight. Prose is sparse, plain, and uses the glossary terms (from the `/codebase-design` skill) without ceremony.
Each candidate is one `<article>`:
- **Title** — short, names the deepening (e.g. "Collapse the Order intake pipeline").
- **Badge row** — recommendation strength (`Strong` = emerald, `Worth exploring` = amber, `Speculative` = slate), plus a tag for the dependency category (`in-process`, `local-substitutable`, `ports & adapters`, `mock`).
- **Files** — monospaced list, `font-mono text-sm`.
- **Before / After diagram** — the centrepiece. Two columns, side by side. See patterns below.
- **Problem** — one sentence. What hurts.
- **Solution** — one sentence. What changes.
- **Wins** — bullets, ≤6 words each. e.g. "Tests hit one interface", "Pricing logic stops leaking", "Delete 4 shallow wrappers".
No paragraphs of explanation. If the diagram needs a paragraph to be understood, redraw the diagram.
## Diagram patterns
Pick the pattern that fits the candidate. Mix them. Don't make every diagram look the same — variety is part of the point.
### Mermaid graph (the workhorse for dependencies / call flow)
Use a Mermaid `flowchart` or `graph` when the point is "X calls Y calls Z, and look at the mess." Wrap it in a Tailwind-styled card so it doesn't feel parachuted in. Style with classDef to colour leakage edges red and the deep module dark. Sequence diagrams work well for "before: 6 round-trips; after: 1."
```html
<div class="rounded-lg border border-slate-200 bg-white p-4">
<pre class="mermaid">
flowchart LR
A[OrderHandler] --> B[OrderValidator]
B --> C[OrderRepo]
C -.leak.-> D[PricingClient]
classDef leak stroke:#dc2626,stroke-width:2px;
class C,D leak
</pre>
</div>
```
### Hand-built boxes-and-arrows (when Mermaid's layout fights you)
Modules as `<div>`s with borders and labels. Arrows as inline SVG `<line>` or `<path>` elements positioned absolutely over a relative container. Reach for this when you want the "after" diagram to feel like one thick-bordered deep module with greyed-out internals — Mermaid won't render that with the right weight.
### Cross-section (good for layered shallowness)
Stack horizontal bands (`h-12 border-l-4`) to show layers a call passes through. Before: 6 thin layers each doing nothing. After: 1 thick band labelled with the consolidated responsibility.
### Mass diagram (good for "interface as wide as implementation")
Two rectangles per module — one for interface surface area, one for implementation. Before: interface rectangle is nearly as tall as the implementation rectangle (shallow). After: interface rectangle is short, implementation rectangle is tall (deep).
### Call-graph collapse
Before: a tree of function calls rendered as nested boxes. After: the same tree collapsed into one box, with the now-internal calls shown faded inside it.
## Style guidance
- Lean editorial, not corporate-dashboard. Generous whitespace. Serif optional for headings (`font-serif` works well with stone/slate).
- Colour sparingly: one accent (emerald or indigo) plus red for leakage and amber for warnings.
- Keep diagrams ~320px tall so before/after sits comfortably side by side without scrolling.
- Use `text-xs uppercase tracking-wider` for module labels inside diagrams — they should read as schematic, not as UI.
- The only scripts are the Tailwind CDN and the Mermaid ESM import. The report is otherwise static — no app code, no interactivity beyond Mermaid's own rendering.
## Top recommendation section
One larger card. Candidate name, one sentence on why, anchor link to its card. That's it.
## Tone
Plain English, concise — but the architectural nouns and verbs come straight from the `/codebase-design` skill. Concision is not an excuse to drift.
**Use exactly:** module, interface, implementation, depth, deep, shallow, seam, adapter, leverage, locality.
**Never substitute:** component, service, unit (for module) · API, signature (for interface) · boundary (for seam) · layer, wrapper (for module, when you mean module).
**Phrasings that fit the style:**
- "Order intake module is shallow — interface nearly matches the implementation."
- "Pricing leaks across the seam."
- "Deepen: one interface, one place to test."
- "Two adapters justify the seam: HTTP in prod, in-memory in tests."
**Wins bullets** name the gain in glossary terms: *"locality: bugs concentrate in one module"*, *"leverage: one interface, N call sites"*, *"interface shrinks; implementation absorbs the wrappers"*. Don't write *"easier to maintain"* or *"cleaner code"* — those terms aren't in the glossary and don't earn their place.
No hedging, no throat-clearing, no "it's worth noting that…". If a sentence could be a bullet, make it a bullet. If a bullet could be cut, cut it. If a term isn't in the `/codebase-design` glossary, reach for one that is before inventing a new one.
@@ -1,68 +0,0 @@
---
name: improve-codebase-architecture
description: Scan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.
disable-model-invocation: true
---
# Improve Codebase Architecture
Surface architectural friction and propose **deepening opportunities** — refactors that turn shallow modules into deep ones. The aim is testability and AI-navigability.
This command is _informed_ by the project's domain model and built on a shared design vocabulary:
- Run the `/codebase-design` skill for the architecture vocabulary (**module**, **interface**, **depth**, **seam**, **adapter**, **leverage**, **locality**) and its principles (the deletion test, "the interface is the test surface", "one adapter = hypothetical seam, two = real"). Use these terms exactly in every suggestion — don't drift into "component," "service," "API," or "boundary."
- The domain language in `CONTEXT.md` gives names to good seams.
## Process
### 1. Explore
**Scope before you scan — YAGNI.** Deepening a module pays off by making future changes to it easier, so put extra weight on the parts of the codebase that have recently changed. Decide *where* to look before you look:
- If the user named a direction — a module, a subsystem, a pain point — take it, and skip the inference below.
- Otherwise, walk back a good stretch of the commit history (`git log --oneline`) to find the codebase's hot spots — the files and areas that keep coming up — and let those paths pull your attention first. If the changes are scattered with no clear hot spot, widen the net.
Read the project's domain glossary (`CONTEXT.md`) first.
Then spawn a sub-agent to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction:
- Where does understanding one concept require bouncing between many small modules?
- Where are modules **shallow** — interface nearly as complex as the implementation?
- Where have pure functions been extracted just for testability, but the real bugs hide in how they're called (no **locality**)?
- Where do tightly-coupled modules leak across their seams?
- Which parts of the codebase are untested, or hard to test through their current interface?
Apply the **deletion test** to anything you suspect is shallow: would deleting it concentrate complexity, or just move it? A "yes, concentrates" is the signal you want.
### 2. Present candidates as an HTML report
Write a self-contained HTML file to the OS temp directory so nothing lands in the repo. Resolve the temp dir from `$TMPDIR`, falling back to `/tmp` (or `%TEMP%` on Windows), and write to `<tmpdir>/architecture-review-<timestamp>.html` so each run gets a fresh file. Open it for the user — `xdg-open <path>` on Linux, `open <path>` on macOS, `start <path>` on Windows — and tell them the absolute path.
The report uses **Tailwind via CDN** for layout and styling, and **Mermaid via CDN** for diagrams where a graph/flow/sequence reliably communicates the structure. Mix Mermaid with hand-crafted CSS/SVG visuals — use Mermaid when relationships are graph-shaped (call graphs, dependencies, sequences), and hand-built divs/SVG when you want something more editorial (mass diagrams, cross-sections, collapse animations). Each candidate gets a **before/after visualisation**. Be visual.
For each candidate, render a card with:
- **Files** — which files/modules are involved
- **Problem** — why the current architecture is causing friction
- **Solution** — plain English description of what would change
- **Benefits** — explained in terms of locality and leverage, and how tests would improve
- **Before / After diagram** — side-by-side, custom-drawn, illustrating the shallowness and the deepening
- **Recommendation strength** — one of `Strong`, `Worth exploring`, `Speculative`, rendered as a badge
End the report with a **Top recommendation** section: which candidate you'd tackle first and why.
**Use CONTEXT.md vocabulary for the domain, and the `/codebase-design` vocabulary for the architecture.** If `CONTEXT.md` defines "Order," talk about "the Order intake module" — not "the FooBarHandler," and not "the Order service."
See `.agents/skills/improve-codebase-architecture/HTML-REPORT.md` (path from the repo root) for the full HTML scaffold, diagram patterns, and styling guidance.
Do NOT propose interfaces yet. After the file is written, ask the user: "Which of these would you like to explore?"
### 3. Grilling loop
Once the user picks a candidate, run the `/grilling` skill to walk the decision tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive.
Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go:
- **Naming a deepened module after a concept not in `CONTEXT.md`?** Add the term to `CONTEXT.md`. Create the file lazily if it doesn't exist.
- **Sharpening a fuzzy term during the conversation?** Update `CONTEXT.md` right there.
- **Want to explore alternative interfaces for the deepened module?** Run the `/codebase-design` skill and use its design-it-twice parallel sub-agent pattern.
@@ -1,51 +0,0 @@
---
name: local-review-codex
description: Run the CI Codex PR review locally against this branch's unpushed work (committed + uncommitted) before pushing. Same policy and reasoning effort as the codex-pr-review GitHub action, on a newer model.
---
# Local Codex Review (pre-push)
Runs the exact same review Codex performs in CI (`.github/workflows/codex-pr-review.yml`),
but locally and scoped to work you have not pushed yet — so you catch what CI would flag
before the PR exists. Use this before `git push` on a non-trivial change.
**Correspondence with CI** — identical:
- Policy: `REVIEW.md` (severity triage, public-surface checklist, AGENTS.md compliance, test coverage).
- Reasoning effort: `model_reasoning_effort="xhigh"`.
- Output: markdown starting with `## Codex Review`, findings tagged P0 / P1 / P2 with file:line.
**Differences from CI** — local-only:
- Model is `gpt-6-astra`; CI stays on `gpt-5.6-sol`. Not an oversight to reconcile: `gpt-6-astra` is confirmed on the ChatGPT auth `codex login` uses locally, while CI authenticates with `OPENAI_API_KEY` (`codex-pr-review.yml` prefers it over `CODEX_AUTH_JSON`) and that tier is unverified for the model. Move CI once API access is confirmed, or once CI switches to `CODEX_AUTH_JSON`.
- Scope is the current branch vs `main` at the merge-base, **including uncommitted changes** (CI reviews a pushed PR diff).
- Sandbox is `read-only` (CI uses `danger-full-access` on an ephemeral runner). Codex reads the diff and files but cannot modify your working tree.
- Fresh context is inherent: `codex exec` is a separate cold process, so it does not anchor on the current chat session — the same reason `local-review` insists on a subagent.
## Prerequisites
- `codex` CLI **>= 0.153.4** installed and authed via `codex login` (an `OPENAI_API_KEY` in the environment takes priority and may not reach `gpt-6-astra` — see the model note above). Older CLIs reject the model with "requires a newer version of Codex"; `run.sh` checks the version up front. Upgrade with `npm install --global @openai/codex@0.153.4` (may need `sudo` for a global install). This matches the pin in `.github/workflows/codex-pr-review.yml` — the CLI version is the same on both sides, only the model differs.
- `git fetch` the base ref if it's stale, so the merge-base is accurate.
## Run
```bash
bash .agents/skills/local-review-codex/run.sh # review vs main (default)
bash .agents/skills/local-review-codex/run.sh <base> # review vs a different base ref
```
Invoke with `bash` (or run the executable directly) — the script needs Bash for
`set -o pipefail`; `sh` is Dash on Debian/Ubuntu and would fail. If `main` isn't a
local branch (e.g. a fresh single-branch checkout), the runner falls back to
`origin/main` automatically.
The script computes `BASE_SHA = git merge-base HEAD <base>`, feeds Codex `REVIEW.md` plus a
diff context pointing at `git diff <BASE_SHA>` (which folds in uncommitted edits), and prints
the review. It writes only temp files — nothing lands in the working tree.
## Relaying the result
Print the Codex output verbatim. Do not re-summarize or filter it — the value of a cold Codex
pass is surfacing what the current session would rationalize away. Then decide with the user
whether to address findings before pushing.
For a Claude-native review instead, use `local-review` (branch-diff-reviewer subagent). This
skill is the Codex counterpart; run both for independent perspectives.
-114
View File
@@ -1,114 +0,0 @@
#!/usr/bin/env bash
# Local Codex review — mirrors the .github/workflows/codex-pr-review.yml CI job,
# but scoped to this branch's unpushed work (committed + uncommitted) so you can
# review before pushing. Same policy (REVIEW.md) and reasoning effort (xhigh) as CI.
#
# The model deliberately differs from CI: gpt-6-astra is confirmed available on the
# ChatGPT auth `codex login` uses here, but CI authenticates with OPENAI_API_KEY and
# that tier is unverified for it, so codex-pr-review.yml stays on gpt-5.6-sol.
#
# Usage: run.sh [BASE_REF] (BASE_REF defaults to "main")
set -euo pipefail
MODEL="gpt-6-astra"
CODEX_MIN="0.153.4"
BASE_REF="${1:-main}"
REPO_ROOT="$(git rev-parse --show-toplevel)"
cd "$REPO_ROOT"
if ! command -v codex >/dev/null 2>&1; then
echo "codex CLI not found. Install with: npm install --global @openai/codex@$CODEX_MIN" >&2
exit 1
fi
# Older CLIs reject the model with an error that never names the CLI version as the
# cause, so check it up front rather than letting the exec fail opaquely. The `|| true`
# keeps an unrecognised --version format from aborting under `set -e`: an unparseable
# version means "cannot tell", which must fall through to the exec, not kill the review.
CODEX_VER="$(codex --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1 || true)"
if [ -n "$CODEX_VER" ] && [ "$(printf '%s\n%s\n' "$CODEX_MIN" "$CODEX_VER" | sort -V | head -1)" != "$CODEX_MIN" ]; then
echo "codex $CODEX_VER is too old for $MODEL (need >= $CODEX_MIN). Upgrade with: npm install --global @openai/codex@$CODEX_MIN" >&2
exit 1
fi
# codex prefers OPENAI_API_KEY over the ChatGPT credentials `codex login` stores, and
# that tier is not confirmed for $MODEL — the resulting failure names the model, not the
# auth that selected it.
if [ -n "${OPENAI_API_KEY:-}" ]; then
echo "warning: OPENAI_API_KEY is set and takes priority over 'codex login' credentials; $MODEL may be unavailable on that tier." >&2
fi
# Resolve the base to a concrete commit, preferring a local ref but falling back to
# the remote-tracking ref — checkouts (CI, single-branch clones) often have only
# origin/main, not a local main.
if git rev-parse --verify --quiet "${BASE_REF}^{commit}" >/dev/null; then
BASE_COMMITISH="$BASE_REF"
elif git rev-parse --verify --quiet "origin/${BASE_REF}^{commit}" >/dev/null; then
BASE_COMMITISH="origin/${BASE_REF}"
else
echo "Base ref '$BASE_REF' not found as '$BASE_REF' or 'origin/$BASE_REF'. Try: git fetch origin $BASE_REF" >&2
exit 1
fi
# Diff from the merge-base so only this branch's changes are reviewed. Using the
# base SHA with a single-ref `git diff` also folds in uncommitted working-tree edits,
# but `git diff` never sees untracked files — those are gathered separately below so
# brand-new files (a whole new module, a new skill dir) are not silently skipped.
BASE_SHA="$(git merge-base HEAD "$BASE_COMMITISH")"
HEAD_SHA="$(git rev-parse HEAD)"
UNTRACKED="$(git ls-files --others --exclude-standard)"
if [ "$BASE_SHA" = "$HEAD_SHA" ] && git diff --quiet "$BASE_SHA" && [ -z "$UNTRACKED" ]; then
echo "No changes vs $BASE_REF — nothing to review." >&2
exit 0
fi
PROMPT="$(mktemp)"
OUT="$(mktemp)"
trap 'rm -f "$PROMPT" "$OUT"' EXIT
# REVIEW.md is the shared policy CI feeds Codex. Append the local output-format
# and diff context inline (CI reads these from a generated context file; inlining
# keeps the working tree clean — no scratch files land in the repo).
cat REVIEW.md > "$PROMPT"
cat >> "$PROMPT" <<EOF
# Codex output format
- This is a pre-push LOCAL review of unpushed work; there is no PR yet.
- Inspect the changes by running the diff commands in the review context below.
- Untracked files do NOT appear in \`git diff\`. Review every untracked path listed below by reading it directly (\`cat\`) — treat its entire contents as newly added.
- Return markdown starting with \`## Codex Review\`.
- Tag each finding with a severity (P0 / P1 / P2), file path, and line number when known confidently.
# Review context
Local review (pre-push): current branch vs $BASE_REF
Base SHA: $BASE_SHA
Head SHA: $HEAD_SHA (plus any uncommitted working-tree changes)
Changed commits command:
git log --oneline $BASE_SHA..HEAD
Changed files command:
git diff --stat $BASE_SHA
Full review diff command (tracked changes, includes uncommitted edits):
git diff --unified=0 $BASE_SHA
Untracked files (NOT in the diff above — read each one directly, it is entirely new):
$(if [ -n "$UNTRACKED" ]; then printf '%s\n' "$UNTRACKED"; else echo "(none)"; fi)
EOF
codex exec \
-C "$REPO_ROOT" \
-m "$MODEL" \
-c 'model_reasoning_effort="xhigh"' \
-s read-only \
-o "$OUT" \
- < "$PROMPT"
echo
echo "===== Codex review ====="
cat "$OUT"
-98
View File
@@ -1,98 +0,0 @@
---
name: local-review
description: Code review the current PR (or branch diff against main) for bugs, security, and AGENTS.md compliance. MUST use when asked to review code.
---
# Local Code Review
Run the same review locally that the GitHub auto-review actions run on PRs (Claude / Codex / Pi). The review policy lives in `REVIEW.md`.
**Why a subagent**: the review MUST run in a fresh context — not inline in the current session. If the user has been iterating on the diff, the main session has absorbed their reasoning and rationalizations, so it anchors and misses things CI catches. A subagent starts cold, like CI does.
## Steps
1. **Determine the PR scope** (cheap, do this in the main session):
- If an argument is provided, treat it as a PR number or branch.
- Otherwise, detect from the current branch vs `main`.
- Confirm the PR/branch exists (`gh pr view <n>` or `git rev-parse <branch>`).
2. **Delegate the review to a fresh-context subagent** with a self-contained prompt. The prompt MUST include:
- The PR number or branch name to review.
- The instruction to read `REVIEW.md` first for the policy, then `AGENTS.md` files in directories touched by the diff.
- The exact output format (see below).
- Whether `--comment` was requested (so the subagent emits inline-comment payloads if needed).
- Any "Additional reviewer instructions" the user provided.
- **Claude Code**: use the `Agent` tool with `subagent_type: branch-diff-reviewer` (read-only tools, purpose-built for this). If unavailable, fall back to `general-purpose`.
- **Codex / Pi**: if the CLI exposes a fresh-session subagent mechanism, use it. Otherwise tell the user to run the skill in a fresh CLI session and stop — running inline in the current session defeats the purpose.
3. **Receive the findings** from the subagent and relay them to the user verbatim. Do not re-summarize, re-judge, or filter — the whole point of fresh context is to surface what the main session would dismiss.
4. **Post comments if `--comment` was requested**: use the `gh` commands below with the subagent's output as the body. The main session does the posting because the subagent is read-only.
## Subagent prompt template
```
Review <PR #N | branch X> against main per the policy in REVIEW.md.
Steps:
1. Read REVIEW.md (repo root) for the full policy: severity triage, public-surface
checklist, AGENTS.md compliance, test coverage assessment.
2. Read AGENTS.md (repo root) and any AGENTS.md in directories touched by the diff.
3. Get the diff: `gh pr diff <N>` (if PR) or `git diff main...<branch>`.
4. Get context: `gh pr view <N>` (if PR) or `git log main..<branch> --oneline`.
5. Read changed files only when the diff alone is insufficient to validate a finding.
6. Self-validate each finding: "is this definitely a real issue a senior engineer
would flag?" Discard if uncertain.
7. Output findings in the exact format below. Do not modify any files.
<paste output format from below>
<if --comment requested:>
Additionally emit a JSON array of inline comments suitable for the GitHub reviews
API, one per finding that maps to a specific line:
[{"path": "...", "line": N, "side": "RIGHT", "body": "[P1] ..."}, ...]
```
## Output format
```
## Code review
<verdict line per REVIEW.md>
Found N issues:
1. [P0|P1|P2] <description>
<file_path:line_number>
2. [P0|P1|P2] <description>
<file_path:line_number>
```
End with a `Test coverage` section per the shared policy.
If no issues are found:
```
## Code review
Good to merge.
No issues found. Checked for bugs, security, and AGENTS.md compliance.
```
## Posting comments (`--comment`)
For a top-level PR comment:
```bash
gh pr review --comment --body "<summary from subagent>"
```
For inline comments on specific lines (using the JSON the subagent emitted):
```bash
gh api repos/{owner}/{repo}/pulls/{pr}/reviews \
-f body="<summary>" -f event="COMMENT" -f comments="<json from subagent>"
```
-793
View File
@@ -1,793 +0,0 @@
---
name: native-trigger
description: Guidance for adding native trigger services to Windmill. Use when implementing or modifying native trigger integrations across the backend and frontend.
---
# Skill: Adding Native Trigger Services
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.
## Architecture Overview
The native trigger system consists of:
1. **Database Layer** - PostgreSQL tables and enum types
2. **Backend Rust Implementation** - Core trait, handlers, and service modules in the `windmill-native-triggers` crate
3. **Frontend Svelte Components** - Configuration forms and UI components
### Key Files
| Component | Path |
|-----------|------|
| Core module with `External` trait | `backend/windmill-native-triggers/src/lib.rs` |
| Generic CRUD handlers | `backend/windmill-native-triggers/src/handler.rs` |
| Background sync logic | `backend/windmill-native-triggers/src/sync.rs` |
| OAuth/workspace integration | `backend/windmill-native-triggers/src/workspace_integrations.rs` |
| Re-export shim (windmill-api) | `backend/windmill-api/src/native_triggers/mod.rs` |
| TriggerKind enum | `backend/windmill-common/src/triggers.rs` |
| JobTriggerKind enum | `backend/windmill-common/src/jobs.rs` |
| Frontend service registry | `frontend/src/lib/components/triggers/native/utils.ts` |
| Frontend trigger utilities | `frontend/src/lib/components/triggers/utils.ts` |
| Trigger badges (icons + counts) | `frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte` |
| Workspace integrations UI | `frontend/src/lib/components/workspaceSettings/WorkspaceIntegrations.svelte` |
| OAuth config form component | `frontend/src/lib/components/workspaceSettings/OAuthClientConfig.svelte` |
| OpenAPI spec | `backend/windmill-api/openapi.yaml` |
| Reference: Nextcloud module | `backend/windmill-native-triggers/src/nextcloud/` |
| Reference: Google module | `backend/windmill-native-triggers/src/google/` |
### Crate Structure
The native trigger code lives in the `windmill-native-triggers` crate (`backend/windmill-native-triggers/`). The `windmill-api` crate re-exports everything via a shim:
```rust
// backend/windmill-api/src/native_triggers/mod.rs
pub use windmill_native_triggers::*;
```
All new service modules go in `backend/windmill-native-triggers/src/`.
---
## Core Concepts
### The `External` Trait
Every native trigger service implements the `External` trait defined in `lib.rs`:
```rust
#[async_trait]
pub trait External: Send + Sync + 'static {
// Associated types:
type ServiceConfig: Debug + DeserializeOwned + Serialize + Send + Sync;
type TriggerData: Debug + Serialize + Send + Sync;
type OAuthData: DeserializeOwned + Serialize + Clone + Send + Sync;
type CreateResponse: DeserializeOwned + Send + Sync;
// Constants:
const SUPPORT_WEBHOOK: bool;
const SERVICE_NAME: ServiceName;
const DISPLAY_NAME: &'static str;
const TOKEN_ENDPOINT: &'static str;
const REFRESH_ENDPOINT: &'static str;
const AUTH_ENDPOINT: &'static str;
// Required methods:
async fn create(&self, w_id, oauth_data, webhook_token, data, db, tx) -> Result<Self::CreateResponse>;
async fn update(&self, w_id, oauth_data, external_id, webhook_token, data, db, tx) -> Result<serde_json::Value>;
async fn get(&self, w_id, oauth_data, external_id, db, tx) -> Result<Self::TriggerData>;
async fn delete(&self, w_id, oauth_data, external_id, db, tx) -> Result<()>;
async fn exists(&self, w_id, oauth_data, external_id, db, tx) -> Result<bool>;
async fn maintain_triggers(&self, db, workspace_id, triggers, oauth_data, synced, errors);
fn external_id_and_metadata_from_response(&self, resp) -> (String, Option<serde_json::Value>);
// Methods with defaults:
async fn prepare_webhook(&self, db, w_id, headers, body, script_path, is_flow) -> Result<PushArgsOwned>;
fn service_config_from_create_response(&self, data, resp) -> Option<serde_json::Value>;
fn additional_routes(&self) -> axum::Router;
async fn http_client_request<T, B>(&self, url, method, workspace_id, tx, db, headers, body) -> Result<T>;
}
```
Key design points:
- **`update()` returns `serde_json::Value`** - the resolved service_config to store. Each service is responsible for building the final config.
- **`maintain_triggers()`** - periodic background maintenance. Each service implements its own strategy (Nextcloud: reconcile with external state; Google: renew expiring channels).
- **No `list_all()` in the trait** - services that need it (Nextcloud) implement it privately; services that don't (Google) use different maintenance strategies.
- **No `get_external_id_from_trigger_data()` or `extract_service_config_from_trigger_data()`** - removed in favor of the `maintain_triggers` pattern.
### Create Lifecycle: Two Paths
The `create_native_trigger` handler in `handler.rs` supports two creation flows, controlled by `service_config_from_create_response()`:
**Path A: Short (Google pattern)** - `service_config_from_create_response()` returns `Some(config)`:
1. `create()` registers on external service
2. `external_id_and_metadata_from_response()` extracts the ID
3. `service_config_from_create_response()` builds the config directly from input data + response metadata
4. Stores trigger in DB -- done, no extra round-trip
Use this when the external_id is known before the create call (e.g., Google generates the channel_id as a UUID upfront and includes it in the webhook URL).
**Path B: Long (Nextcloud pattern)** - `service_config_from_create_response()` returns `None` (default):
1. `create()` registers on external service (webhook URL has no external_id yet)
2. `external_id_and_metadata_from_response()` extracts the ID
3. `update()` is called to fix the webhook URL with the now-known external_id
4. `update()` returns the resolved service_config
5. Stores trigger in DB
Use this when the external_id is assigned by the remote service and the webhook URL needs to be corrected after creation.
### OAuth Token Storage (Three-Table Pattern)
OAuth tokens are stored across three tables, NOT in `workspace_integrations.oauth_data` directly:
| Table | What's Stored |
|-------|---------------|
| `workspace_integrations` | `oauth_data` JSON with `base_url`, `client_id`, `client_secret`, `instance_shared` flag; `resource_path` pointing to the variable |
| `variable` | Encrypted `access_token` (at the path stored in `resource_path`), linked to `account` via `account` column |
| `account` | `refresh_token`, keyed by `workspace_id` + `client` (service name) + `is_workspace_integration = true` |
The `decrypt_oauth_data()` function in `lib.rs` assembles these into a unified struct:
```rust
pub struct OAuthConfig {
pub base_url: String,
pub access_token: String, // decrypted from variable
pub refresh_token: Option<String>, // from account table
pub client_id: String, // from oauth_data or instance settings
pub client_secret: String, // from oauth_data or instance settings
}
```
Instance-level sharing: when `oauth_data.instance_shared == true`, `client_id` and `client_secret` are read from global settings instead of workspace_integrations.
### URL Resolution
The `resolve_endpoint()` helper handles both absolute and relative OAuth URLs:
```rust
pub fn resolve_endpoint(base_url: &str, endpoint: &str) -> String {
if endpoint.starts_with("http://") || endpoint.starts_with("https://") {
endpoint.to_string() // Google: absolute URLs
} else {
format!("{}{}", base_url, endpoint) // Nextcloud: relative paths
}
}
```
### ServiceName Methods
`ServiceName` is the central registry enum. Each variant must implement these match arms:
| Method | Purpose |
|--------|---------|
| `as_str()` | Lowercase identifier (e.g., `"google"`) |
| `as_trigger_kind()` | Maps to `TriggerKind` enum |
| `as_job_trigger_kind()` | Maps to `JobTriggerKind` enum |
| `token_endpoint()` | OAuth token endpoint (relative or absolute) |
| `auth_endpoint()` | OAuth authorization endpoint |
| `oauth_scopes()` | Space-separated OAuth scopes |
| `resource_type()` | Resource type for token storage (e.g., `"gworkspace"`) |
| `extra_auth_params()` | Extra OAuth params (e.g., Google needs `access_type=offline`, `prompt=consent`) |
| `integration_service()` | Maps to the workspace integration service (usually `*self`) |
| `TryFrom<String>` | Parse from string |
| `Display` | Delegates to `as_str()` |
---
## Step-by-Step Implementation Guide
### Step 1: Database Migration
Create a new migration file: `backend/migrations/YYYYMMDDHHMMSS_newservice_trigger.up.sql`
```sql
-- Add the service to the native_trigger_service enum
ALTER TYPE native_trigger_service ADD VALUE IF NOT EXISTS 'newservice';
-- Add to TRIGGER_KIND enum (used for trigger tracking)
ALTER TYPE TRIGGER_KIND ADD VALUE IF NOT EXISTS 'newservice';
-- Add to job_trigger_kind enum (used for job tracking)
ALTER TYPE job_trigger_kind ADD VALUE IF NOT EXISTS 'newservice';
```
Also create the corresponding down migration.
### Step 2: Update windmill-common Enums
#### `backend/windmill-common/src/triggers.rs`
Add variant to `TriggerKind` enum, and update `to_key()` and `fmt()` implementations.
#### `backend/windmill-common/src/jobs.rs`
Add variant to `JobTriggerKind` enum and update the `Display` implementation.
### Step 3: Backend Service Module
Create a new directory: `backend/windmill-native-triggers/src/newservice/`
#### `mod.rs` - Type Definitions
```rust
use serde::{Deserialize, Serialize};
pub mod external;
// pub mod routes; // Only if you need additional service-specific routes
/// OAuth data deserialized from the three-table pattern.
/// The actual structure is built by decrypt_oauth_data() from variable + account + workspace_integrations.
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct NewServiceOAuthData {
pub base_url: String, // from workspace_integrations.oauth_data
pub access_token: String, // decrypted from variable table
pub refresh_token: Option<String>, // from account table
// Note: client_id and client_secret are in OAuthConfig, not here
// unless the service needs them at runtime for API calls
}
/// Configuration provided by user when creating/updating a trigger.
/// Stored as JSON in native_trigger.service_config.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct NewServiceConfig {
// Service-specific configuration fields
pub folder_path: String,
pub file_filter: Option<String>,
}
/// Data retrieved from the external service about a trigger.
/// Returned by the get() method and shown in the UI.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct NewServiceTriggerData {
pub folder_path: String,
pub file_filter: Option<String>,
// Fields that shouldn't affect service_config comparison should use #[serde(skip_serializing)]
}
/// Response from external service when creating a trigger/webhook.
#[derive(Debug, Deserialize)]
pub struct CreateTriggerResponse {
pub id: String,
}
/// Handler struct (stateless, used for routing)
#[derive(Copy, Clone)]
pub struct NewService;
```
#### `external.rs` - External Trait Implementation
```rust
use async_trait::async_trait;
use reqwest::Method;
use sqlx::PgConnection;
use std::collections::HashMap;
use windmill_common::{
error::{Error, Result},
BASE_URL, DB,
};
use crate::{
generate_webhook_service_url, External, NativeTrigger, NativeTriggerData, ServiceName,
sync::{SyncError, TriggerSyncInfo},
};
use super::{NewService, NewServiceConfig, NewServiceOAuthData, NewServiceTriggerData, CreateTriggerResponse};
#[async_trait]
impl External for NewService {
type ServiceConfig = NewServiceConfig;
type TriggerData = NewServiceTriggerData;
type OAuthData = NewServiceOAuthData;
type CreateResponse = CreateTriggerResponse;
const SERVICE_NAME: ServiceName = ServiceName::NewService;
const DISPLAY_NAME: &'static str = "New Service";
const SUPPORT_WEBHOOK: bool = true;
const TOKEN_ENDPOINT: &'static str = "/oauth/token";
const REFRESH_ENDPOINT: &'static str = "/oauth/token";
const AUTH_ENDPOINT: &'static str = "/oauth/authorize";
async fn create(
&self,
w_id: &str,
oauth_data: &Self::OAuthData,
webhook_token: &str,
data: &NativeTriggerData<Self::ServiceConfig>,
db: &DB,
tx: &mut PgConnection,
) -> Result<Self::CreateResponse> {
let base_url = &*BASE_URL.read().await;
// external_id is None during create (we get it from the response)
let webhook_url = generate_webhook_service_url(
base_url, w_id, &data.script_path, data.is_flow,
None, Self::SERVICE_NAME, webhook_token,
);
let url = format!("{}/api/webhooks/create", oauth_data.base_url);
let payload = serde_json::json!({
"callback_url": webhook_url,
"folder_path": data.service_config.folder_path,
});
let response: CreateTriggerResponse = self
.http_client_request(&url, Method::POST, w_id, tx, db, None, Some(&payload))
.await?;
Ok(response)
}
/// Update returns the resolved service_config as JSON.
/// For services using the update+get pattern, call self.get() and serialize.
async fn update(
&self,
w_id: &str,
oauth_data: &Self::OAuthData,
external_id: &str,
webhook_token: &str,
data: &NativeTriggerData<Self::ServiceConfig>,
db: &DB,
tx: &mut PgConnection,
) -> Result<serde_json::Value> {
let base_url = &*BASE_URL.read().await;
let webhook_url = generate_webhook_service_url(
base_url, w_id, &data.script_path, data.is_flow,
Some(external_id), Self::SERVICE_NAME, webhook_token,
);
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
let payload = serde_json::json!({
"callback_url": webhook_url,
"folder_path": data.service_config.folder_path,
});
let _: serde_json::Value = self
.http_client_request(&url, Method::PUT, w_id, tx, db, None, Some(&payload))
.await?;
// Fetch back the updated state to get the resolved config
let trigger_data = self.get(w_id, oauth_data, external_id, db, tx).await?;
serde_json::to_value(&trigger_data)
.map_err(|e| Error::InternalErr(format!("Failed to serialize trigger data: {}", e)))
}
async fn get(
&self,
w_id: &str,
oauth_data: &Self::OAuthData,
external_id: &str,
db: &DB,
tx: &mut PgConnection,
) -> Result<Self::TriggerData> {
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
self.http_client_request::<_, ()>(&url, Method::GET, w_id, tx, db, None, None).await
}
async fn delete(
&self,
w_id: &str,
oauth_data: &Self::OAuthData,
external_id: &str,
db: &DB,
tx: &mut PgConnection,
) -> Result<()> {
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
let _: serde_json::Value = self
.http_client_request::<_, ()>(&url, Method::DELETE, w_id, tx, db, None, None)
.await
.or_else(|e| match &e {
Error::InternalErr(msg) if msg.contains("404") => Ok(serde_json::Value::Null),
_ => Err(e),
})?;
Ok(())
}
async fn exists(
&self,
w_id: &str,
oauth_data: &Self::OAuthData,
external_id: &str,
db: &DB,
tx: &mut PgConnection,
) -> Result<bool> {
match self.get(w_id, oauth_data, external_id, db, tx).await {
Ok(_) => Ok(true),
Err(Error::NotFound(_)) => Ok(false),
Err(e) => Err(e),
}
}
/// Background maintenance. Choose the right pattern for your service:
/// - For services with queryable external state: use reconcile_with_external_state()
/// - For channel-based services with expiration: implement renewal logic
async fn maintain_triggers(
&self,
db: &DB,
workspace_id: &str,
triggers: &[NativeTrigger],
oauth_data: &Self::OAuthData,
synced: &mut Vec<TriggerSyncInfo>,
errors: &mut Vec<SyncError>,
) {
// Option A: Reconcile with external state (Nextcloud pattern)
// Fetch all triggers from external service and compare with DB
let external_triggers = match self.list_all(workspace_id, oauth_data, db).await {
Ok(triggers) => triggers,
Err(e) => {
errors.push(SyncError {
resource_path: format!("workspace:{}", workspace_id),
error_message: format!("Failed to list triggers: {}", e),
error_type: "api_error".to_string(),
});
return;
}
};
// Convert to (external_id, config_json) pairs
let external_pairs: Vec<(String, serde_json::Value)> = external_triggers
.into_iter()
.map(|t| (t.id.clone(), serde_json::to_value(&t).unwrap_or_default()))
.collect();
crate::sync::reconcile_with_external_state(
db, workspace_id, Self::SERVICE_NAME, triggers, &external_pairs, synced, errors,
).await;
}
fn external_id_and_metadata_from_response(
&self,
resp: &Self::CreateResponse,
) -> (String, Option<serde_json::Value>) {
(resp.id.clone(), None)
}
// service_config_from_create_response: NOT overridden (returns None).
// This means the handler uses the update+get pattern after create.
// Override and return Some(...) to skip the update+get cycle (Google pattern).
}
impl NewService {
/// Private helper to list all triggers from the external service.
async fn list_all(
&self,
w_id: &str,
oauth_data: &<Self as External>::OAuthData,
db: &DB,
) -> Result<Vec<<Self as External>::TriggerData>> {
// Implementation depends on the external service's API
todo!()
}
}
```
### Step 4: Update lib.rs Registry
In `backend/windmill-native-triggers/src/lib.rs`:
```rust
// Service modules - add new services here:
#[cfg(feature = "native_trigger")]
pub mod newservice; // <-- Add this
// ServiceName enum - add variant:
pub enum ServiceName {
Nextcloud,
Google,
NewService, // <-- Add this
}
// Then add match arms in ALL ServiceName methods:
// as_str(), as_trigger_kind(), as_job_trigger_kind(), token_endpoint(),
// auth_endpoint(), oauth_scopes(), resource_type(), extra_auth_params(),
// integration_service(), TryFrom<String>, Display
```
### Step 5: Update handler.rs Routes
In `backend/windmill-native-triggers/src/handler.rs`:
```rust
pub fn generate_native_trigger_routers() -> Router {
// ...
#[cfg(feature = "native_trigger")]
{
use crate::newservice::NewService;
return router
.nest("/nextcloud", service_routes(NextCloud))
.nest("/google", service_routes(Google))
.nest("/newservice", service_routes(NewService)); // <-- Add this
}
// ...
}
```
### Step 6: Update sync.rs
In `backend/windmill-native-triggers/src/sync.rs`:
```rust
pub async fn sync_all_triggers(db: &DB) -> Result<BackgroundSyncResult> {
// ...
#[cfg(feature = "native_trigger")]
{
use crate::newservice::NewService;
// ... existing service syncs ...
// New service sync
let (service_name, result) = sync_service_triggers(db, NewService).await;
total_synced += result.synced_triggers.len();
total_errors += result.errors.len();
service_results.insert(service_name, result);
}
// ...
}
```
### Step 7: Frontend Service Registry
In `frontend/src/lib/components/triggers/native/utils.ts`:
Add to `NATIVE_TRIGGER_SERVICES`, `getTriggerIconName()`, and `getServiceIcon()`.
### Step 8: Frontend Trigger Form Component
Create: `frontend/src/lib/components/triggers/native/services/newservice/NewServiceTriggerForm.svelte`
### Step 9: Frontend Icon Component
Create: `frontend/src/lib/components/icons/NewServiceIcon.svelte`
### Step 10: Update NativeTriggerEditor
Check `frontend/src/lib/components/triggers/native/NativeTriggerEditor.svelte` to ensure it dynamically loads form components based on service name.
### Step 11: Workspace Integration UI
Add your service to the `supportedServices` map in `frontend/src/lib/components/workspaceSettings/WorkspaceIntegrations.svelte`:
```typescript
const supportedServices: Record<string, ServiceConfig> = {
// ... existing services ...
newservice: {
name: 'newservice',
displayName: 'New Service',
description: 'Connect to New Service for triggers',
icon: NewServiceIcon,
docsUrl: 'https://www.windmill.dev/docs/integrations/newservice',
requiresBaseUrl: false, // false for cloud services, true for self-hosted
setupInstructions: [
'Step 1: Create an OAuth app on the service',
'Step 2: Configure the redirect URI shown below',
'Step 3: Enter the client credentials below'
]
}
}
```
### Step 12: Update `frontend/src/lib/components/triggers/utils.ts`
Update ALL of these maps/functions:
1. `triggerIconMap` - import and add icon
2. `triggerDisplayNamesMap` - add display name
3. `triggerTypeOrder` in `sortTriggers()` - add type
4. `getLightConfig()` - add case for your service
5. `getTriggerLabel()` - add case for your service
6. `jobTriggerKinds` - add to array
7. `countPropertyMap` - add count property
8. `triggerSaveFunctions` - add save function
### Step 13: Update TriggersBadge Component
In `frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte`:
1. Import the icon
2. Add to `baseConfig` with `countKey` (the dynamic `availableNativeServices` loop does NOT set `countKey`)
3. Add to the `allTypes` array
### Step 14: Update TriggersWrapper.svelte
In `frontend/src/lib/components/triggers/TriggersWrapper.svelte`:
Add a `{:else if selectedTrigger.type === 'yourservice'}` case that renders `<NativeTriggersPanel service="yourservice" ...>` with the same props pattern as the existing native trigger cases (e.g., `nextcloud`).
### Step 15: Update AddTriggersButton.svelte
In `frontend/src/lib/components/triggers/AddTriggersButton.svelte`:
1. Add `yourserviceAvailable` state variable
2. Add `setYourserviceState()` async function using `isServiceAvailable('yourservice', $workspaceStore!)`
3. Call it at module level
4. Add a dropdown entry to `addTriggerItems` with `hidden: !yourserviceAvailable`
### Step 16: Update TriggersEditor.svelte Delete Handling
In `frontend/src/lib/components/triggers/TriggersEditor.svelte`:
Add your service to the `nativeTriggerServices` map in `deleteDeployedTrigger()`. Native triggers use `NativeTriggerService.deleteNativeTrigger({ workspace, serviceName, externalId })` instead of the standard `path`-based delete.
### Step 17: Update `getUsedTriggers` for Sidebar Visibility
The sidebar (`frontend/src/lib/components/sidebar/SidebarContent.svelte`) shows native-trigger links only if `$usedTriggerKinds` includes the service — without this, your trigger page will never appear in the nav bar even when triggers exist.
1. **Backend** — add `{service}_used: bool` to the `UsedTriggers` struct and SELECT in `backend/windmill-api-workspaces/src/workspaces.rs::get_used_triggers()`:
```rust
EXISTS(SELECT 1 FROM native_trigger WHERE workspace_id = $1 AND service_name = '{service}'::native_trigger_service) AS "{service}_used!"
```
2. **OpenAPI** — add `{service}_used: boolean` to the response schema for `GET /w/{workspace}/workspaces/used_triggers` (under both `properties` and `required`).
3. **Layout** — in `frontend/src/routes/(root)/(logged)/+layout.svelte::loadUsedTriggerKinds()`, destructure `{service}_used` and push `'{service}'` to `usedKinds`.
### Step 18: Update OpenAPI Spec and Regenerate Types
Add to `JobTriggerKind` enum in `backend/windmill-api/openapi.yaml`, then:
```bash
cd frontend && npm run generate-backend-client
```
---
## Special Patterns
### Unified Service with `trigger_type` (Google Pattern)
When a single service handles multiple trigger types (e.g., Google Drive + Calendar share OAuth and API patterns), use a single `ServiceName` variant with a discriminator field:
```rust
pub enum GoogleTriggerType { Drive, Calendar }
pub struct GoogleServiceConfig {
pub trigger_type: GoogleTriggerType,
// Drive-specific fields (only used when trigger_type = Drive)
pub resource_id: Option<String>,
pub resource_name: Option<String>,
// Calendar-specific fields (only used when trigger_type = Calendar)
pub calendar_id: Option<String>,
pub calendar_name: Option<String>,
// Metadata set after creation
pub google_resource_id: Option<String>,
pub expiration: Option<String>,
}
```
Branch in trait methods based on `trigger_type`. Frontend uses a `ToggleButtonGroup` to switch between types. This keeps the codebase simpler (one service, one OAuth flow, one set of routes).
See `backend/windmill-native-triggers/src/google/` for the reference implementation.
### Skipping update+get After Create (Google Pattern)
Override `service_config_from_create_response()` to return `Some(config)` when the external_id is known before the create call:
```rust
fn service_config_from_create_response(
&self,
data: &NativeTriggerData<Self::ServiceConfig>,
resp: &Self::CreateResponse,
) -> Option<serde_json::Value> {
// Clone input config, add metadata from response
let mut config = data.service_config.clone();
config.google_resource_id = Some(resp.resource_id.clone());
config.expiration = Some(resp.expiration.clone());
Some(serde_json::to_value(&config).unwrap())
}
```
### Services with Absolute OAuth Endpoints (Google)
Unlike self-hosted services where OAuth endpoints are relative paths appended to `base_url`, services like Google have absolute URLs:
```rust
// Nextcloud: relative paths
ServiceName::Nextcloud => "/apps/oauth2/api/v1/token",
// Google: absolute URLs
ServiceName::Google => "https://oauth2.googleapis.com/token",
```
The `resolve_endpoint()` function handles both. For services with absolute endpoints:
- `base_url` can be empty
- `requiresBaseUrl: false` in the frontend workspace integration config
- Add `extra_auth_params()` if needed (Google requires `access_type=offline` and `prompt=consent`)
### Channel-Based Push Notifications with Renewal (Google Pattern)
For services using expiring watch channels instead of persistent webhooks:
1. Store expiration in `service_config` (as part of `ServiceConfig`)
2. In `maintain_triggers()`, implement renewal logic instead of using `reconcile_with_external_state()`:
```rust
async fn maintain_triggers(&self, db, workspace_id, triggers, oauth_data, synced, errors) {
for trigger in triggers {
if should_renew_channel(trigger) {
self.renew_channel(db, trigger, oauth_data).await;
}
}
}
```
3. Renewal: best-effort stop old channel, create new one with same external_id, update service_config with new expiration
4. Google example: Drive channels expire in 24h (renew when <1h left), Calendar channels expire in 7 days (renew when <1 day left)
### reconcile_with_external_state (Nextcloud Pattern)
The reusable function in `sync.rs` compares external triggers with DB state:
- Triggers missing externally: sets error "Trigger no longer exists on external service"
- Triggers present externally: clears errors, updates service_config if it differs
Usage in `maintain_triggers()`:
```rust
let external_pairs: Vec<(String, serde_json::Value)> = /* fetch from external */;
crate::sync::reconcile_with_external_state(
db, workspace_id, Self::SERVICE_NAME, triggers, &external_pairs, synced, errors,
).await;
```
### Webhook Payload Processing
Override `prepare_webhook()` to parse service-specific payloads into script/flow args:
```rust
async fn prepare_webhook(&self, db, w_id, headers, body, script_path, is_flow) -> Result<PushArgsOwned> {
let mut args = HashMap::new();
args.insert("event_type".to_string(), Box::new(headers.get("x-event-type").cloned()) as _);
args.insert("payload".to_string(), Box::new(serde_json::from_str::<serde_json::Value>(&body)?) as _);
Ok(PushArgsOwned { extra: None, args })
}
```
Then register in `prepare_native_trigger_args()` in `lib.rs`:
```rust
pub async fn prepare_native_trigger_args(service_name, db, w_id, headers, body) -> Result<Option<PushArgsOwned>> {
match service_name {
ServiceName::Google => { /* ... */ Ok(Some(args)) }
ServiceName::NewService => { /* ... */ Ok(Some(args)) }
ServiceName::Nextcloud => Ok(None), // Uses default body parsing
}
}
```
### Instance-Level OAuth Credentials
When `workspace_integrations.oauth_data.instance_shared == true`, `decrypt_oauth_data()` reads `client_id` and `client_secret` from instance-level global settings instead of workspace-level. This allows admins to share OAuth app credentials across workspaces.
The frontend handles this via the `generate_instance_connect_url` endpoint in `workspace_integrations.rs`.
---
## Testing Checklist
- [ ] Database migration runs successfully
- [ ] `cargo check -p windmill-native-triggers --features native_trigger` passes
- [ ] `npx svelte-check --threshold error` passes (in frontend/)
- [ ] Service appears in workspace integrations list
- [ ] OAuth flow completes successfully
- [ ] Can create a new trigger
- [ ] Can view trigger details
- [ ] Can update trigger configuration
- [ ] Can delete trigger
- [ ] Webhook receives and processes payloads
- [ ] Background sync works correctly (reconciliation or channel renewal)
- [ ] Error handling works (expired tokens, service unavailable)
---
## Reference Implementations
### Nextcloud (Self-Hosted, Update+Get Pattern)
| File | Purpose |
|------|---------|
| `nextcloud/mod.rs` | Types: NextCloudOAuthData, NextcloudServiceConfig, NextCloudTriggerData |
| `nextcloud/external.rs` | External trait: uses update+get pattern, reconcile_with_external_state for sync |
| `nextcloud/routes.rs` | Additional route: `GET /events` |
Key patterns: relative OAuth endpoints, base_url required, list_all + reconcile for sync, update returns JSON from get().
### Google (Cloud, Unified Service, Short Create)
| File | Purpose |
|------|---------|
| `google/mod.rs` | Types: GoogleServiceConfig with trigger_type discriminator, GoogleTriggerType enum |
| `google/external.rs` | External trait: overrides service_config_from_create_response, channel renewal for sync |
| `google/routes.rs` | Additional routes: `GET /calendars`, `GET /drive/files`, `GET /drive/shared_drives` |
Key patterns: absolute OAuth endpoints, empty base_url, trigger_type for Drive/Calendar, expiring watch channels with renewal, service_config_from_create_response skips update+get, get() reconstructs data from stored service_config (no external "get channel" API).
-257
View File
@@ -1,257 +0,0 @@
---
name: pr
user_invocable: true
description: Open a draft pull request on GitHub and drive CI review rounds until it is ready. MUST use when you want to create/open a PR.
---
# Pull Request Skill
Create a draft pull request with a clear title and explicit description of changes, then drive it through CI review rounds to ready.
## Instructions
1. **Analyze branch changes**: Understand all commits since diverging from main
2. **Push to remote**: Ensure all commits are pushed
3. **Create draft PR**: Always open as draft for review before merging
4. **Drive review rounds**: trigger CI reviews on the draft and only flip to ready once every verdict is a go (see "Review rounds" below)
## PR Title Format
Follow conventional commit format for the PR title:
```
<type>: <description>
```
### Types
- `feat`: New feature or capability
- `fix`: Bug fix
- `refactor`: Code restructuring
- `docs`: Documentation changes
- `chore`: Maintenance tasks
- `perf`: Performance improvements
### Title Rules
- Keep under 70 characters
- Use lowercase, imperative mood
- No period at the end
- If `*_ee.rs` files were modified, prefix with `[ee]`: `[ee] <type>: <description>`
## PR Body Format
The body MUST be explicit about what changed. Structure:
```markdown
## Summary
<Clear description of what this PR does and why>
## Changes
- <Specific change 1>
- <Specific change 2>
- <Specific change 3>
## Test plan
- [ ] <How to verify change 1>
- [ ] <How to verify change 2>
```
The harness/tooling that invoked the skill may add its own attribution trailer; the skill itself does not prescribe one.
## Screenshots (required for frontend changes)
If `git diff main...HEAD --name-only` matches `^frontend/`, the PR body **must** include
screenshots of the affected UI. Skip only when there is no visible UI effect (types,
tests, build config) — and say so in the body.
1. Verify the change in the browser (frontend/AGENTS.md → "Verifying Frontend Changes").
2. Screenshot each affected page with `mcp__playwright__browser_take_screenshot` (save to a file).
3. Host each image and get its Markdown embed by pushing to the public
`windmill-labs/agent-screenshots-internal` repo. **Pipe base64 through stdin**
passing it as `-f content=…` fails with `argument list too long` on real images:
```bash
REPO=windmill-labs/agent-screenshots-internal
IMG=screenshot.png # repeat per page
DEST="shots/$(git branch --show-current)/$(date +%s)-$(basename "$IMG")"
base64 -w0 "$IMG" | jq -Rs --arg m "add $DEST" '{message:$m, content:.}' \
| gh api -X PUT "repos/$REPO/contents/$DEST" --input - >/dev/null
echo "![$(basename "$IMG" .png)](https://raw.githubusercontent.com/$REPO/main/$DEST)"
```
Derive `$DEST` from the file name (as above) so distinct pages never collide — a
fixed name would make same-second uploads reuse one path, and the second `PUT`
then 422s (the Contents API needs the existing file's `sha` to overwrite).
4. Put the printed `![]()` lines under a `## Screenshots` heading in the PR body.
Requires `gh` (`repo` scope), `jq`, `base64` — all in the devShell. The host repo is
public (so the raw URLs render for reviewers without a token) and its history is
permanent — **never screenshot pages that show secrets or sensitive values** (workspace
variables, resource values, instance settings, OAuth/SMTP config); deleting the file
can't undo an accidental capture. (GitHub's drag-and-drop uploader needs a browser
session and can't be driven from a token.)
If `gh` can't push to the host repo (e.g. a CI token scoped only to `windmill`), do
**not** fail the PR or skip silently — hand the upload to the user, who has push access,
and continue once they confirm it's done.
## Execution Steps
1. Run `git status` to check for uncommitted changes
2. Run `git log main..HEAD --oneline` to see all commits in this branch
3. Run `git diff main...HEAD` to see the full diff against main
4. **Review the diff before creating the PR — run both reviews, do not skip:**
- **`local-review`** — Claude-native branch-diff-reviewer (`/local-review` in Claude Code, `$local-review` in Codex, `pi --skill local-review` / `/skill:local-review` in Pi).
- **`local-review-codex`** — cold Codex pass, the same review CI runs, for an independent perspective the Claude pass misses (`/local-review-codex` in Claude Code, or `bash .agents/skills/local-review-codex/run.sh`). If the `codex` CLI is missing or older than the version pinned in that skill, note it in your summary and continue — never block the PR on codex being unavailable.
Run both — they catch different things. If either surfaces issues, fix them and commit before proceeding.
5. **Screenshots for frontend changes**: if `git diff main...HEAD --name-only` matches `^frontend/`, capture and embed screenshots of the affected UI per "Screenshots" above before writing the PR body (skip only if there is no visible UI effect).
6. Check if remote branch exists and is up to date:
```bash
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
```
7. Push to remote if needed: `git push -u origin HEAD`
8. Create draft PR using gh CLI:
```bash
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
## Summary
<description>
## Changes
- <change 1>
- <change 2>
## Test plan
- [ ] <test 1>
- [ ] <test 2>
EOF
)"
```
9. Return the PR URL to the user
10. Drive the PR through CI review rounds to ready (see "Review rounds" below)
## Review rounds (draft → ready)
A PR leaves draft **only after a clean CI review round**. Never run `gh pr ready` before that.
This is the rule in every mode, autonomous included. A clean round is necessary but not always
sufficient — see "Flip, or ask first" below. The one standing exception is an explicit request to
leave that PR in draft (usually so it can be tested first) — honour it for that PR, and don't
carry it over to the next one.
1. **Trigger a round and wait for it**: launch the waiter as a background Bash task (a round takes 1030 min; you are woken when it exits — do not stop the session or poll in the foreground while it runs):
```bash
bash .agents/skills/pr/review-round.sh <PR_NUMBER>
```
It comments `/review` on the PR — which runs the Codex, Claude and Pi CI reviewers even on a draft — waits for the spawned `PR Review Commands` workflow run(s) to complete, then prints one verdict line per reviewer and saves the full review comments to files.
`/review` (and `/codex`) are **idempotent per head SHA**: if a running or successful review already covers the current head, they skip that agent and post nothing new — the waiter reads the existing verdict for that head, so a skipped agent is *not* a missing one. A cancelled/failed head run is re-run in place; a fresh run is launched only when nothing covers the head. So an unchanged-head re-review is a near no-op, not a new round — push a commit to get genuinely fresh reviews.
2. **Judge the round.** Codex is mandatory; Claude, Pi and cubic count whenever they posted. Every review starts with one of the three `REVIEW.md` verdicts:
- Codex verdict missing → the round is void: the waiter warns only when the head has no green Codex run (cancelled/failed/absent — not merely skipped-because-already-reviewed). Comment `/codex` on the PR, which re-runs the interrupted run in place (or launches one if none exists), wait the same way, and judge again.
- Any **"Should address issues before merging"** → fix the P0/P1 findings (and the nits while you're there), commit, push, and start a new round (step 1).
- Only **"Mergeable, but should ideally address nits"** and/or **"Good to merge"** → fix the nits too; a nit that is wrong or genuinely not worth fixing may instead be dismissed by replying to the review comment with your reasoning. Push nit-only fixes without starting another full round.
3. **Flip to ready with the marker comment.** The review workflows skip the redundant `ready_for_review`-triggered round when the PR author has posted a marker naming the current head SHA **and** the PR's latest Codex review *posted before the marker* has a non-blocking verdict (reviewer evidence — a bare marker with no round behind it, or one whose last pre-marker Codex verdict is "Should address issues", skips nothing). Keep the prefix exact and use the full 40-char SHA of the head you are flipping:
- every verdict was "Good to merge" (head unchanged since the round):
`✅ Review round clean @ <head-sha>`
- nit-only round, nits fixed or dismissed afterwards (head may have moved past the reviewed SHA — say so):
`✅ Review round clean @ <head-sha> — nit-only verdicts at <round-sha>; nits addressed in <commit sha(s)> / dismissed in review replies`
```bash
gh pr comment <PR_NUMBER> --body "✅ Review round clean @ $(git rev-parse HEAD)"
gh pr ready <PR_NUMBER>
```
If any P0/P1 finding is unaddressed or the head moved for reasons other than nit fixes, do **not** post the marker or flip — run another round instead.
### A round that never starts is usually a conflict
The review workflows don't run on a PR that cannot merge, so a round that produces no verdict is
more often a conflict with `main` than a CI outage. Check before assuming anything is broken:
```bash
gh pr view <PR_NUMBER> --json mergeable,mergeStateStatus
```
Resolve by **merging, not rebasing** — a rebase rewrites the head SHA that round verdicts and the
clean-round marker are keyed to, invalidating work you have already paid for:
```bash
git fetch origin main
git merge origin/main
```
**If that merge changed `backend/ee-repo-ref.txt`, move the EE worktree to match.** The file pins
the EE commit CE builds against, so a merge that advances it leaves the EE checkout behind what CE
now expects, and `cargo check --features private` compiles a tree neither you nor CI intends:
```bash
git -C <ee-worktree> merge "$(tr -d '[:space:]' < backend/ee-repo-ref.txt)"
```
Push both, then start a fresh round — the head moved, so the earlier verdicts no longer apply.
### Flip, or ask first
A clean round earns the flip; it does not always earn it *unattended*. Judge the blast radius from
the diff first — `git diff --name-only main...HEAD` answers most of these.
**Ask before flipping** when the change:
- touches `*_ee.rs` (it spans the EE repo through symlinks and has a companion PR)
- adds a migration under `backend/migrations/`
- changes `openapi.yaml`, `openflow.openapi.yaml`, or the generated client
- touches auth, permission, or token paths
- changes shared worker infrastructure — the job poller, `handle_child`, an executor
- trips `REVIEW.md`'s "Checklist for new public surfaces"
**Flip without asking** when it is self-contained: a single-file fix, test-only, docs-only, one
call site, no new public surface.
Unattended (webmux oneshot) there is nobody to ask, so the judgement holds and the action
degrades: flip the self-contained ones, and leave the rest at a clean draft with a line in the PR
description saying why — `left in draft: adds a migration, wants a human look before ready`.
Don't flip a wide-blast-radius change just because the round came back clean, and don't ask a
question nobody will read.
`AGENTS.local.md` (gitignored, so it may not exist) carries a "PR ready calibration" section
recording how past ambiguous calls went. Read it before deciding; when a call is still genuinely
ambiguous, ask, then append the answer there so the next one is less ambiguous.
### When rounds stop converging
Three or more rounds without a clean verdict usually means the change's shape is wrong, not that
there is an endless supply of independent bugs. The tells:
- findings keep landing in the same files round after round
- fixing one finding creates the next
- the findings are about coupling, duplication, or state threaded through many places, rather
than logic errors
When that pattern holds, stop running rounds — each one costs a CI cycle and is not going to
converge. Say plainly that the remaining findings look structural rather than incidental, and
name the module or seam they cluster around. With a user present, suggest they run
`/improve-codebase-architecture` over that area: it is slash-only so you cannot invoke it
yourself, and reshaping the code is a scope change they should choose. Unattended, put the
diagnosis in the PR description and stop there rather than grinding out more rounds.
## EE Companion PR (when `*_ee.rs` files were modified)
The `*_ee.rs` files in the windmill repo are **symlinks** to `windmill-ee-private` — changes won't appear in `git diff` of the windmill repo. Instead, check the EE repo for uncommitted or unpushed changes.
Follow the full EE PR workflow in `docs/enterprise.md`. The key PR-specific details:
1. Find the EE repo/worktree: see "Finding the EE Repo" in `docs/enterprise.md`
2. Check for changes: `git -C <ee-path> status --short`
- If there are no changes in the EE repo, skip this entire section
3. Follow steps 15 from the "EE PR Workflow" in `docs/enterprise.md`
4. Create the companion PR (title does NOT get the `[ee]` prefix):
```bash
gh pr create --draft --repo windmill-labs/windmill-ee-private --title "<type>: <description>" --body "$(cat <<'EOF'
Companion PR for windmill-labs/windmill#<PR_NUMBER>
EOF
)"
```
5. Commit `ee-repo-ref.txt` and push the updated windmill branch
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env bash
# Trigger a CI review round on a PR and wait for it to finish.
#
# Usage: bash .agents/skills/pr/review-round.sh [PR_NUMBER]
# PR_NUMBER defaults to the current branch's PR.
#
# Comments `/review` on the PR (works on drafts), waits for the spawned
# "PR Review Commands" workflow run(s) to complete, then prints one verdict
# line per reviewer and saves each full review comment to a file. A round
# takes 10-30 minutes: run this in the background and act on its output when
# it exits, per the pr skill ("Review rounds").
set -euo pipefail
REPO=${REPO:-$(gh repo view --json nameWithOwner --jq .nameWithOwner)}
PR=${1:-$(gh pr view --json number --jq .number)}
# Timestamp of the trigger comment, straight from GitHub, so local clock skew
# can't make the run/comment filters below miss part of the round.
TRIGGER_TIME=$(gh api "repos/$REPO/issues/$PR/comments" -f body='/review' --jq .created_at)
echo "Review round triggered on $REPO#$PR at $TRIGGER_TIME"
# Retry wrapper for one-off gh/API hiccups: a 45-minute wait must not die on
# a single transient failure.
retry() {
local attempt
for attempt in 1 2 3; do
if "$@"; then return 0; fi
sleep 10
done
return 1
}
# The /review comment spawns one "PR Review Commands" run holding the
# claude/codex/pi jobs. Runs aren't linked to a PR, so wait on every run of
# that workflow created after the trigger: a concurrent round on another PR
# can only delay the answer, never truncate it. Every issue comment on any PR
# spawns a fast-completing parse run of the same workflow, so the round's own
# run may briefly lag the listing while unrelated runs already show completed:
# require the all-completed state to hold past a floor and across two
# consecutive polls before trusting it.
DEADLINE=$(( $(date +%s) + 45 * 60 ))
NO_RUN_DEADLINE=$(( $(date +%s) + 5 * 60 ))
MIN_WAIT_UNTIL=$(( $(date +%s) + 3 * 60 ))
STABLE=0
FAILURES=0
while :; do
if RUNS=$(gh run list --repo "$REPO" --workflow=pr-review-commands.yml \
--created ">=$TRIGGER_TIME" --limit 100 --json status); then
FAILURES=0
else
FAILURES=$(( FAILURES + 1 ))
if [ "$FAILURES" -ge 5 ]; then
echo "ERROR: listing workflow runs failed $FAILURES times in a row; aborting the wait." >&2
exit 1
fi
echo "WARNING: listing workflow runs failed (attempt $FAILURES/5); retrying in 60s." >&2
sleep 60
continue
fi
TOTAL=$(jq length <<<"$RUNS")
PENDING=$(jq '[.[] | select(.status != "completed")] | length' <<<"$RUNS")
NOW=$(date +%s)
if [ "$TOTAL" -gt 0 ] && [ "$PENDING" -eq 0 ] && [ "$NOW" -gt "$MIN_WAIT_UNTIL" ]; then
STABLE=$(( STABLE + 1 ))
if [ "$STABLE" -ge 2 ]; then
break
fi
else
STABLE=0
fi
if [ "$TOTAL" -eq 0 ] && [ "$NOW" -gt "$NO_RUN_DEADLINE" ]; then
echo "ERROR: no 'PR Review Commands' run appeared within 5 minutes of the /review comment; check that the comment author has write access and the workflow is enabled." >&2
exit 1
fi
if [ "$NOW" -gt "$DEADLINE" ]; then
echo "WARNING: review round still pending after 45 minutes; reporting whatever has been posted so far." >&2
break
fi
sleep 60
done
# Head SHA at trigger time. `/review` is idempotent per head: it skips an agent a
# running/successful review already covers, re-runs a cancelled/failed one in place on a
# separate head-tied run, and launches fresh only when nothing covers the head. Verdict
# reading below therefore keys off the head, not just the trigger timestamp.
HEAD_SHA=$(retry gh api "repos/$REPO/pulls/$PR" --jq .head.sha)
echo "Reviewing head $HEAD_SHA"
# Newest non-skipped run of <workflow> tied to the head ("status conclusion"), or empty
# when none exists. A re-run-in-place or an already-covering review resolves on such a
# head-tied run — separate from the pr-review-commands run waited on above (a fresh
# launch instead runs inside it, and posts after the trigger). A `skipped` run is the
# draft/fork gate and produced no review, so it is ignored.
head_run_state() {
gh run list --repo "$REPO" --workflow "$1" --commit "$HEAD_SHA" --limit 20 \
--json databaseId,status,conclusion \
--jq '[.[] | select(.conclusion != "skipped")] | sort_by(.databaseId) | last | if . then "\(.status) \(.conclusion // "-")" else empty end' 2>/dev/null || true
}
# A re-run-in-place review lands on a head-tied run that finishes after the fast
# pr-review-commands run, so let those settle before reading verdicts.
for wf in codex-pr-review.yml pi-pr-review.yml pr-ready-review.yml; do
while :; do
case "$(head_run_state "$wf")" in
""|"completed "*) break ;;
*) if [ "$(date +%s)" -gt "$DEADLINE" ]; then break; fi; sleep 30 ;;
esac
done
done
OUT_DIR=$(mktemp -d -t review-round-XXXXXX)
COMMENTS_RAW=$(retry gh api "repos/$REPO/issues/$PR/comments?per_page=100" --paginate)
# Two views: comments from THIS round (after the trigger) and the full history. A fresh
# launch posts after the trigger; an idempotent skip leaves the covering verdict in the
# earlier run's comment, so fall back to history when that agent's head run is green.
jq -s --arg t "$TRIGGER_TIME" '[.[][] | select(.created_at > $t)]' \
<<<"$COMMENTS_RAW" > "$OUT_DIR/comments.json"
jq -s '[.[][]]' <<<"$COMMENTS_RAW" > "$OUT_DIR/comments-all.json"
# cubic posts through the PR reviews API, not issue comments.
REVIEWS_RAW=$(retry gh api "repos/$REPO/pulls/$PR/reviews?per_page=100" --paginate)
jq -s --arg t "$TRIGGER_TIME" '[.[][] | select((.submitted_at // "") > $t)]' \
<<<"$REVIEWS_RAW" > "$OUT_DIR/pr-reviews.json"
VERDICT_RE='(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)'
body_by_header() { # <file> <header-substring>
jq -r --arg h "$2" '[.[] | select(.body // "" | contains($h))] | last | .body // empty' "$1"
}
body_by_login() { # <file> <login>
jq -r --arg l "$2" '[.[] | select(.user.login == $l)] | last | .body // empty' "$1"
}
head_ok() { [ "$(head_run_state "$1")" = "completed success" ]; }
# Latest verdict for a reviewer: prefer this round's comment; if none and the reviewer's
# head run succeeded (an idempotent /review skipped re-reviewing an already-green head),
# fall back to the covering comment from the full history.
verdict_body() { # <header|login> <value> <workflow>
local body
body=$("body_by_$1" "$OUT_DIR/comments.json" "$2")
if [ -z "$body" ] && head_ok "$3"; then
body=$("body_by_$1" "$OUT_DIR/comments-all.json" "$2")
fi
printf '%s' "$body"
}
report() { # <reviewer-name> <comment-body>
local name=$1 body=$2 verdict
if [ -z "$body" ]; then
echo "$name: (no review posted for this head)"
return
fi
printf '%s\n' "$body" > "$OUT_DIR/$name.md"
verdict=$(printf '%s\n' "$body" | grep -m1 -oE "${VERDICT_RE}.*" | sed 's/\*\*//g' || true)
echo "$name: ${verdict:-(review posted but no verdict line; read $OUT_DIR/$name.md)}"
}
echo
echo "=== Review round verdicts for $REPO#$PR (head $HEAD_SHA) ==="
CODEX_BODY=$(verdict_body header '## Codex Review' codex-pr-review.yml)
report codex "$CODEX_BODY"
report claude "$(verdict_body login 'claude[bot]' pr-ready-review.yml)"
report pi "$(verdict_body header '## Pi Review' pi-pr-review.yml)"
CUBIC_BODY=$(jq -r '[.[] | select(.user.login | test("^cubic(-dev-ai)?(\\[bot\\])?$"; "i"))] | last | .body // empty' \
"$OUT_DIR/pr-reviews.json")
if [ -z "$CUBIC_BODY" ]; then
CUBIC_BODY=$(jq -r '[.[] | select(.user.login | test("^cubic(-dev-ai)?(\\[bot\\])?$"; "i"))] | last | .body // empty' \
"$OUT_DIR/comments.json")
fi
report cubic "$CUBIC_BODY"
echo
echo "Full round output: $OUT_DIR (comments.json, pr-reviews.json, one .md per reviewer)"
if [ -z "$CODEX_BODY" ]; then
echo "WARNING: no Codex verdict for $HEAD_SHA - its head run is not green (cancelled/failed/absent, not merely skipped-because-already-reviewed). Re-trigger with a '/codex' PR comment (re-runs the interrupted run in place, or launches one) and wait again." >&2
fi
-38
View File
@@ -1,38 +0,0 @@
---
name: refine
user_invocable: true
description: End-of-session reflection. Reviews friction encountered during the session and proposes updates to docs/ to capture lessons learned.
---
# Refine Skill
Reflect on the current session and update documentation with lessons learned.
## Instructions
1. **Identify friction**: Review what happened in this session:
- Run `git diff main...HEAD --stat` to see what files were touched
- Think about: what was slow, what failed, what required multiple attempts, what information was missing or hard to find
2. **Read current docs**: Read the docs that were relevant to this session:
- `docs/validation.md`
- `docs/enterprise.md`
- Any skills that were invoked
3. **Propose updates**: For each piece of friction, decide if it warrants a doc update:
- **Missing knowledge**: Information you had to discover that should be documented
- **Wrong guidance**: Instructions that led you astray
- **Missing validation rule**: A check that should be in the validation matrix
- **New pattern**: A codebase pattern worth capturing for next time
4. **Apply updates**: Edit the relevant `docs/` files. Keep changes minimal and specific — add only what would have saved time this session.
5. **Report**: Summarize what was added/changed and why.
## Rules
- Only add knowledge confirmed by this session — no speculative additions
- Keep docs concise — add a line or two, not a paragraph
- If a whole new doc is needed, create it in `docs/` and add a pointer in `CLAUDE.md`
- Don't update skills unless a coding pattern was genuinely wrong
- Don't add things Claude already knows — only Windmill-specific knowledge
-114
View File
@@ -1,114 +0,0 @@
---
name: rust-backend
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
---
# Windmill Rust Patterns
Apply these Windmill-specific patterns when writing Rust code in `backend/`.
## Error Handling
Use `Error` from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>`:
```rust
use windmill_common::error::{Error, Result};
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
sqlx::query_as!(Job, "SELECT id, workspace_id FROM v2_job WHERE id = $1", id)
.fetch_optional(db)
.await?
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
}
```
Never panic in library code. Reserve `.unwrap()` for compile-time guarantees.
## SQLx Patterns
**Never use `SELECT *`** — always list columns explicitly. Critical for backwards compatibility when workers lag behind API version:
```rust
// Correct
sqlx::query_as!(Job, "SELECT id, workspace_id, path FROM v2_job WHERE id = $1", id)
// Wrong — breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", id)
```
Use batch operations to avoid N+1:
```rust
// Preferred — single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
Use transactions for multi-step operations. Parameterize all queries.
## JSON Handling
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when storing/passing JSON without inspection:
```rust
pub struct Job {
pub args: Option<Box<serde_json::value::RawValue>>,
}
```
Only use `serde_json::Value` when you need to inspect or modify the JSON.
## Serde Optimizations
```rust
#[derive(Serialize, Deserialize)]
pub struct Job {
#[serde(skip_serializing_if = "Option::is_none")]
pub parent_job: Option<Uuid>,
#[serde(skip_serializing_if = "Vec::is_empty")]
pub tags: Vec<String>,
#[serde(default)]
pub priority: i32,
}
```
## Async & Concurrency
Never block the async runtime. Use `spawn_blocking` for CPU-intensive work:
```rust
let result = tokio::task::spawn_blocking(move || expensive_computation(&data)).await?;
```
**Mutex selection**: Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) for data protection. Only use `tokio::sync::Mutex` when holding locks across `.await` points.
Use `tokio::sync::mpsc` (bounded) for channels. Avoid `std::thread::sleep` in async contexts.
## Module Structure & Visibility
- Use `pub(crate)` instead of `pub` when possible
- Place new code in the appropriate crate based on functionality
- API endpoints go in `windmill-api/src/` organized by domain
- Shared functionality goes in `windmill-common/src/`
## Code Navigation
Always use rust-analyzer LSP for go-to-definition, find-references, and type info. Do not guess at module paths.
## Feature Telemetry
`FEATURE_USAGE_KINDS` in `windmill-api-workspaces/src/workspaces.rs` is an allowlist: a
`(feature, kind)` pair missing from it is dropped by `valid_feature_usage_event` with a bare
`continue` — no error, and the route still returns 204. Adding a counter on the frontend without
registering it here records nothing. See `docs/feature-telemetry.md`.
## Axum Handlers
Destructure extractors directly in function signatures:
```rust
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> { ... }
```
-136
View File
@@ -1,136 +0,0 @@
---
name: svelte-frontend
description: Svelte coding guidelines for the Windmill frontend. MUST use when writing or modifying code in the frontend directory.
---
# Windmill Svelte Patterns
Apply these Windmill-specific patterns when writing Svelte code in `frontend/`. For general Svelte 5 syntax (runes, snippets, event handling), use the Svelte MCP server.
## Before writing any UI (MUST)
Do both of these before the first line of markup — not after, and not only when something
looks unfamiliar.
**1. Find the component that already exists.** `frontend/src/lib/components/common/index.ts`
is the design-system barrel — 28 lines, read it in full. It exports far more than the three
documented below: `Alert`, `Badge`, `Breadcrumb`, `Drawer`/`DrawerContent`, `Menu`/`MenuItem`,
`Tabs`/`Tab`/`TabContent`, `Skeleton`, `FileInput`, `RadioCard`, `Section`, `Kbd`, `ActionRow`,
`ClearableInput`, `CopyButton`, `SecondsInput`, `UndoRedo`, `Url`.
The barrel is not the full picture either: `common/` has 34 subdirectories and only 23 exports,
so `modal/`, `popup/`, `stepper/`, `tooltip/`, `checkbox/`, `table/`, `contextmenu/`,
`confirmationModal/`, `calendarPicker/`, `fileUpload/`, `toggleButton-v2/` and more exist but
must be imported by path. Selects, text inputs and melt-based primitives sit next to `common/`
in `components/select/`, `components/text_input/`, `components/meltComponents/`.
The tree holds 1,600+ components — grep `frontend/src/lib/components` for the thing you're about
to build; it almost certainly exists. Building a new one is the last resort, not the first move.
**2. Read the guideline for what you're building.** `frontend/brand-guidelines.md` is the
authority on how it should look and read. Don't load all 34k chars — jump to the section:
| Building | Section to read |
|---|---|
| Any new screen or component | `# Components` (Core Rules, Quick Reference) |
| Buttons, CTAs | `## Buttons` — hierarchy matters, only one Accent per view |
| Colors, surfaces, borders | `# Color system` (Quick Reference, Do's and Don'ts) |
| Text, labels, headings | `# Typography` — note `## Text Casing`, sentence case throughout |
| Spacing, grids, page structure | `# Spacing & Layout`; `# Layout``## Form` for forms |
| Shadows, overlays, depth | `# Elevation` |
| Icons | `# Iconography` |
| Wording of any UI copy | `# Voice & Communication`, `# Tone of Voice` |
Get the line range with `grep -n '^#' frontend/brand-guidelines.md`, then read just that span.
## Windmill UI Components (MUST use)
Always use Windmill's design-system components. Never use raw HTML elements. The three below
are the ones you'll reach for most often — they are examples, not the catalog. For anything
else, go back to the barrel and grep.
### Buttons — `<Button>`
```svelte
<script>
import { Button } from '$lib/components/common'
import { ChevronLeft } from 'lucide-svelte'
</script>
<Button variant="default" onclick={handleClick}>Label</Button>
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prev} />
```
Props: `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`, `unifiedSize?: '2xs' | 'xs' | 'sm' | 'md' | 'lg'`, `startIcon?: { icon: SvelteComponent }`, `iconOnly?: boolean`, `disabled?: boolean`
**`size` on `<Button>` is banned** — it, `spacingSize` and `extendedSize` are the legacy sizing
system (`xs3`/`xs2`/`xs`/…, marked `@deprecated` in `Button.svelte`). Size every button with
`unifiedSize`, the small ones included: `2xs` and `xs` are `h-5`, `sm` is `h-7`, `md` is `h-8`,
`lg` is `h-10`. Existing `size="xs2"` call sites are legacy, not a precedent to copy. Same for
`variant`: `contained`/`border`/`divider` are deprecated — use the four listed above.
### Text inputs — `<TextInput>`
```svelte
<script>
import { TextInput } from '$lib/components/common'
</script>
<TextInput bind:value={val} placeholder="Enter value" />
```
Props: `value?: string | number` (bindable), `placeholder?: string`, `disabled?: boolean`, `error?: string | boolean`, `size?: 'sm' | 'md' | 'lg'`
### Selects — `<Select>`
```svelte
<script>
import Select from '$lib/components/select/Select.svelte'
</script>
<Select items={[{ label: 'Jan', value: 1 }]} bind:value={selected} />
```
Props: `items?: Array<{ label?: string; value: any }>`, `value` (bindable), `placeholder?: string`, `clearable?: boolean`, `size?: 'sm' | 'md' | 'lg'`
### Icons — `lucide-svelte`
Never write inline SVGs. Import from `lucide-svelte`:
```svelte
<script>
import { ChevronLeft, X } from 'lucide-svelte'
</script>
<ChevronLeft size={16} />
```
## Form Components
Form components (TextInput, Toggle, Select, etc.) should use the unified size system when placed together.
## Styling
- Use Tailwind CSS for all styling — no custom CSS
- Use Windmill's theming classes for colors/surfaces (see `frontend/brand-guidelines.md`)
- Read component props JSDoc before using them
## Feature Telemetry
New user-facing UX is the main source of `feature_usage` counters — propose them in the plan, not
as a separate question, and read `docs/feature-telemetry.md` first. `logFeatureUsage()` from
`$lib/utils/featureUsage` is only half the change: the `(feature, kind)` pair must also be
registered in the backend allowlist or every event is silently discarded, and the disclosure copy
in `InstanceSettings.svelte` must name what you added.
## Svelte MCP Server
Use the Svelte MCP tools when working on Svelte code:
1. **list-sections**: Call first to discover available docs
2. **get-documentation**: Fetch relevant sections based on use_cases
3. **svelte-autofixer**: MUST use on all Svelte code before finalizing — keep calling until no issues
4. **playground-link**: Only after user confirms and code was NOT written to project files
## Verifying in the Browser
After changing Svelte code, use the **Playwright MCP** (`mcp__playwright__*`) to drive the running frontend and confirm the change works. See frontend/AGENTS.md → "Verifying Frontend Changes" for the full flow. Use `playwright` (headless) on devboxes; `playwright-headed` when a display is available.
-154
View File
@@ -1,154 +0,0 @@
---
name: update-sqlx
description: How to safely update SQLx offline query cache. MUST use when SQL queries change.
---
# SQLx Offline Query Cache
Windmill uses `SQLX_OFFLINE=true` in CI, which requires all `sqlx::query!` / `sqlx::query_as!` macros to have matching cached query data in `backend/.sqlx/`.
## When to Run
Run after **adding or editing** a SQL query in Rust source. Without it, CI fails with:
```
error: `SQLX_OFFLINE=true` but there is no cached data for this query
```
**Do NOT run it when a change only *removes* queries.** The cache is already complete for
CI; all that is left are orphaned entries, which are cosmetic and never break a build.
Running `prepare` to tidy them risks destroying the cache for no gain. Delete them
offline instead: for each `.sqlx/query-*.json`, normalize its `query` field (strip `\`
line-continuations, collapse whitespace) and check whether it still appears in any `.rs`
file. That detector reports ~48 false positives in a CE checkout — EE queries live in
`*_ee.rs` symlinks it cannot read — so **filter to the tables your change touched** and
delete only those.
## Before You Run Anything
1. **Back the cache up.** `prepare` deletes `.sqlx/` *before* regenerating, so any compile
failure leaves it gutted (observed: 2350 → 142 entries).
```bash
bash .agents/skills/update-sqlx/sqlx-cache.sh backup
```
Its state is per-worktree, so a sibling worktree running `prepare` at the same time
cannot overwrite your backup.
2. **Point `DATABASE_URL` at THIS worktree's database.** `prepare` compiles every
`sqlx::query!` against the **live** database. Another worktree's DB lacks your
migrations, so every new-table query fails and takes the cache down with it. The
symptom is `relation "<your_new_table>" does not exist` — that is a wrong
`DATABASE_URL`, not a broken query. See AGENTS.md → "Per-worktree ports and database".
## Queries Inside Tests Need `--all-targets`, Which Fails In A CE Checkout
`prepare` only caches queries in code it compiles, and `--workspace` alone does **not**
compile test targets. A `sqlx::query!` inside `tests/*.rs` therefore gets no entry, and CI
fails on the test target with the usual "no cached data" error even though the lib built
clean. `SQLX_OFFLINE=true cargo check --workspace --all-targets` is what reproduces it.
Adding `--all-targets` caches them — and, in a CE checkout, **aborts partway through**:
`backend/tests/otel.rs` imports `windmill_common::otel_ee`, which exists only behind the
`private` feature, so the compile dies after `prepare` has already emptied `.sqlx/`.
Observed: 2435 → 4 entries, `error: cargo check failed with status: exit status: 101`.
Do not fight it — the abort is a pre-existing EE gap, not something your change caused.
Take the entries you need and put the backup back:
```bash
bash .agents/skills/update-sqlx/sqlx-cache.sh backup
cd backend
DATABASE_URL=<this worktree's db> \
cargo sqlx prepare --workspace -- --workspace --features all_sqlx_features --all-targets
# expected to fail; it still wrote the entries it got to before dying
cd ..
bash .agents/skills/update-sqlx/sqlx-cache.sh newq # prints each added query
bash .agents/skills/update-sqlx/sqlx-cache.sh restore # backup back, added entries grafted on
```
**Read what `newq` prints before running `restore`** — it shows each added entry's `query`
field, and every one should be yours. The set is small (one per new test query); anything
else in there means the run got further than you think.
Then verify both targets, since the lib passing says nothing about the tests:
```bash
SQLX_OFFLINE=true cargo check --workspace --features all_sqlx_features # lib
SQLX_OFFLINE=true cargo check -p <your-crate> --all-targets # tests
```
## The Problem
`cargo sqlx prepare --workspace` **deletes all existing cache files** and regenerates only the ones found in the current compilation. If you don't compile with every feature flag (especially `private` for EE files), you will **silently delete EE query caches**, breaking CI for enterprise tests.
The standard `./update_sqlx.sh` script tries to compile with all features, but it often fails locally because the EE symlinks can be out of sync with `main`.
## Safe Procedure
Always preserve the existing EE caches from `origin/main`. Use this workflow:
```bash
cd backend
# 1. Restore the full cache from main (includes EE caches)
git checkout origin/main -- .sqlx/
# 2. Run prepare with OSS features (what compiles locally)
# This regenerates OSS caches to match your code changes.
cargo sqlx prepare --workspace -- --workspace --features all_sqlx_features
# 3. Restore any EE caches that were deleted in step 2.
# These are files present in origin/main but missing after prepare.
git ls-tree origin/main backend/.sqlx/ \
| awk '{print $4}' | sed 's|backend/\.sqlx/||' | sort > /tmp/main_files.txt
find backend/.sqlx -name "*.json" -printf '%P\n' | sort > /tmp/current_files.txt
comm -23 /tmp/main_files.txt /tmp/current_files.txt > /tmp/missing_files.txt
while read f; do
git show "origin/main:backend/.sqlx/$f" > "backend/.sqlx/$f"
done < /tmp/missing_files.txt
# 4. Verify nothing was lost from main
find backend/.sqlx -name "*.json" -printf '%P\n' | sort > /tmp/current_files.txt
comm -23 /tmp/main_files.txt /tmp/current_files.txt | wc -l
# Should output: 0
```
## If EE Compiles Locally
If your EE repo happens to be in sync, you can use the full script (faster):
```bash
cd backend
./update_sqlx.sh
```
But if it fails with EE compilation errors, use the safe procedure above.
## What NOT to Do
- **Never** run `cargo sqlx prepare --workspace` with only OSS features and commit the result — it will delete EE caches.
- **Never** set `SQLX_OFFLINE=true` for local `cargo sqlx prepare` — use a live database per CLAUDE.md. (CI runs with `SQLX_OFFLINE=true`, which is why the cache must be complete.)
- **Never** run `prepare` without a `.sqlx` backup, or against a `DATABASE_URL` you have not confirmed belongs to this worktree.
- **Never** run `prepare` at all for a removal-only change.
- **Never** skip the verification step (step 4 above).
- **Never** leave a `--all-targets` run's output in place after it aborts — it is a
near-empty cache. Restore the backup and graft on only the entries you verified.
Step 4 compares against `origin/main` because step 1 restored from it, so the two agree.
If you did **not** run step 1 — auditing a branch's cache on its own, say — compare
against `git merge-base HEAD origin/main` instead: `origin/main` advances, so its newer
entries would read as losses on your branch.
## Verification
After committing, the diff against `origin/main` should show:
- A few **new** cache files (for your changed queries)
- A few **deleted** cache files (for old queries that no longer exist)
- **Zero** net deletions from the EE cache set
```bash
git diff origin/main --stat backend/.sqlx/
```
-88
View File
@@ -1,88 +0,0 @@
#!/usr/bin/env bash
# Backup / inspect / restore the SQLx offline cache around `cargo sqlx prepare`.
#
# `prepare` empties backend/.sqlx before regenerating, so any compile failure leaves the
# cache gutted (observed: 2350 -> 142 entries). A `--all-targets` run in a CE checkout
# aborts that way every time. State lives in a per-worktree directory, so sibling
# worktrees running this concurrently cannot overwrite each other's backup.
#
# sqlx-cache.sh backup snapshot backend/.sqlx
# sqlx-cache.sh newq show the entries prepare added since the snapshot, and stage them
# sqlx-cache.sh restore put the snapshot back, grafting the staged entries on top
#
# Inspect what `newq` prints before running `restore` — an entry you don't recognise means
# the run got further than you think.
set -euo pipefail
repo_root="$(git rev-parse --show-toplevel)"
cache="$repo_root/backend/.sqlx"
state="${TMPDIR:-/tmp}/wm-sqlx-cache/$(basename "$repo_root")"
backup="$state/backup"
added="$state/added"
# `find -printf` is GNU-only; a glob loop stays portable to a macOS checkout and, unlike
# `ls *.json`, does not fail the script under `set -e` when the cache is empty — which is
# exactly the state a failed `prepare` leaves behind.
list_entries() {
local f
for f in "$1"/*.json; do
[ -e "$f" ] || continue
basename "$f"
done | sort
}
show_query() {
if command -v jq >/dev/null 2>&1; then
jq -r '.query' "$1" 2>/dev/null | head -6
else
sed -n 's/^ *"query": "\(.*\)",*$/\1/p' "$1" | head -6
fi
}
case "${1:-}" in
backup)
[[ -d $cache ]] || { echo "no cache at $cache" >&2; exit 1; }
rm -rf "$state"
mkdir -p "$state"
cp -r "$cache" "$backup"
list_entries "$backup" > "$state/before.txt"
echo "backed up $(wc -l < "$state/before.txt" | tr -d ' ') entries to $backup"
;;
newq)
[[ -d $backup ]] || { echo "no backup — run '$0 backup' first" >&2; exit 1; }
list_entries "$cache" > "$state/after.txt"
comm -13 "$state/before.txt" "$state/after.txt" > "$state/new.txt"
rm -rf "$added"
mkdir -p "$added"
n=0
while read -r f; do
[[ -n $f ]] || continue
cp "$cache/$f" "$added/$f"
n=$((n + 1))
echo "--- $f"
show_query "$cache/$f"
done < "$state/new.txt"
echo "$n entries added since the backup, staged in $added"
;;
restore)
[[ -d $backup ]] || { echo "no backup — nothing to restore" >&2; exit 1; }
[[ -d $added ]] || { echo "run '$0 newq' first so the added entries are staged" >&2; exit 1; }
rm -rf "$cache"
cp -r "$backup" "$cache"
n=0
for f in "$added"/*.json; do
[[ -e $f ]] || continue
cp "$f" "$cache/"
n=$((n + 1))
done
echo "restored $(list_entries "$cache" | wc -l | tr -d ' ') entries ($n grafted from this run)"
;;
*)
sed -n '2,14p' "$0" | sed 's/^# \{0,1\}//'
exit 1
;;
esac
-127
View File
@@ -1,127 +0,0 @@
---
name: branch-diff-reviewer
description: Use this agent when you want a comprehensive code review of changes in the current branch compared to main. This includes reviewing for bugs, optimization opportunities, code style issues, potential mistakes, and adherence to project conventions. The agent should be invoked after completing a feature branch or before creating a pull request.\n\nExamples:\n\n<example>\nContext: User has finished implementing a new feature and wants feedback before merging.\nuser: "I've finished the new kafka trigger implementation, can you review my changes?"\nassistant: "I'll use the branch-diff-reviewer agent to analyze your changes against the main branch and provide comprehensive feedback."\n<commentary>\nSince the user wants a review of their branch changes, use the Task tool to launch the branch-diff-reviewer agent to compare the current branch against main and provide detailed feedback.\n</commentary>\n</example>\n\n<example>\nContext: User wants to check their code quality before submitting a PR.\nuser: "Review my branch before I create a PR"\nassistant: "Let me launch the branch-diff-reviewer agent to examine all your changes and identify any issues or improvements."\n<commentary>\nThe user is preparing for a PR, so use the branch-diff-reviewer agent to provide a thorough review of all branch differences.\n</commentary>\n</example>\n\n<example>\nContext: User is unsure if their implementation follows project patterns.\nuser: "Does my implementation look correct? I'm not sure if I followed the existing patterns"\nassistant: "I'll use the branch-diff-reviewer agent to compare your changes against main and check for pattern consistency, potential issues, and optimization opportunities."\n<commentary>\nThe user needs validation of their implementation against project standards. Launch the branch-diff-reviewer agent to analyze the diff and provide feedback on patterns, correctness, and improvements.\n</commentary>\n</example>
tools: Glob, Grep, Read, WebFetch, TodoWrite, WebSearch, ListMcpResourcesTool, ReadMcpResourceTool, mcp__svelte__get-documentation, mcp__svelte__list-sections, mcp__svelte__playground-link, mcp__svelte__svelte-autofixer, mcp__ide__getDiagnostics, mcp__ide__executeCode, Bash, Skill
model: inherit
---
You are an elite code reviewer with deep expertise in software engineering best practices, performance optimization, and security. Your role is to provide thorough, actionable feedback on code changes between the current branch and main.
## Your Review Process
1. **First, gather the diff**: Use git commands to obtain the complete diff between the current branch and main:
- Run `git diff main...HEAD` to see all changes
- Run `git log main..HEAD --oneline` to understand the commit history
- Identify all modified, added, and deleted files
2. **Analyze each changed file** in the context of:
- The project's established patterns (check CLAUDE.md and related documentation)
- The file's purpose and its role in the broader codebase
- Dependencies and how changes might affect other parts of the system
## Review Categories
For each significant change, evaluate and report on:
### 🐛 Bugs & Correctness
- Logic errors or edge cases not handled
- Null/undefined handling issues
- Race conditions in async code
- Incorrect error handling
- Type mismatches or unsafe casts
### ⚡ Performance
- Inefficient algorithms or data structures
- N+1 query problems in database code
- Unnecessary re-renders in frontend code
- Missing indexes for database queries
- Blocking operations in async contexts
- Memory leaks or excessive allocations
- For Rust: Check for unnecessary clones, inefficient serde usage, blocking in async
- For Svelte: Check for inefficient reactivity, missing keys in loops, excessive effects
### 🔒 Security
- SQL injection vulnerabilities
- Missing input validation
- Exposed sensitive data
- Authentication/authorization gaps
- Unsafe deserialization
### 📐 Code Quality & Style
- Adherence to project conventions (CLAUDE.md guidelines)
- Code duplication that should be refactored
- Unclear or misleading naming
- Missing or inadequate documentation
- Overly complex logic that could be simplified
- Dead code or unused imports
### 🏗️ Architecture & Design
- Proper separation of concerns
- Appropriate use of existing utilities vs. new code
- Consistency with established patterns
- Proper error propagation
- API design issues
### 🧪 Testing Considerations
- Suggest test cases for new functionality
- Identify untested edge cases
- Note if changes break existing test assumptions
## Project-Specific Rules
### For Rust (Backend)
- Verify `SELECT` statements list explicit columns (never `SELECT *` in worker code)
- Check for proper use of `sqlx` with parameterized queries
- Ensure errors use the custom `Error` enum from `windmill-common::error`
- Verify async code doesn't block the tokio runtime
- Check serde attributes for optimal serialization
- Ensure openapi.yaml is updated for API changes
### For Svelte (Frontend)
- For Svelte 5 files: Verify proper use of Runes (`$state`, `$derived`, `$effect`)
- Check for `key` attributes in `{#each}` blocks
- Ensure event handlers use the new syntax (`onclick` not `on:click`) in Svelte 5
- Verify snippets are used instead of slots in Svelte 5
- Check for proper props declaration with `$props()`
## Output Format
Structure your review as follows:
```
## Summary
[Brief overview of the changes and overall assessment]
## Critical Issues 🚨
[Issues that must be fixed before merging]
## Recommendations 💡
[Improvements that would significantly enhance the code]
## Minor Suggestions 📝
[Nice-to-haves and style improvements]
## Positive Observations ✅
[Well-done aspects worth acknowledging]
## File-by-File Details
[Detailed feedback organized by file]
```
For each issue, provide:
1. **Location**: File path and line number(s)
2. **Issue**: Clear description of the problem
3. **Impact**: Why this matters
4. **Suggestion**: Concrete fix or improvement with code example when helpful
## Behavioral Guidelines
- Be thorough but prioritize: focus most on critical issues
- Be constructive: every criticism should come with a suggestion
- Be specific: vague feedback is not actionable
- Acknowledge good work: positive reinforcement matters
- Consider context: understand why decisions might have been made
- Ask clarifying questions if the intent of changes is unclear
- Reference project documentation when pointing out convention violations
Begin by fetching the diff and then proceed with your comprehensive review.
-76
View File
@@ -1,76 +0,0 @@
---
name: openapi-sync
description: Use this agent when backend API endpoints are added, modified, or removed, or when the Flow structure changes and the OpenAPI specification files need to be updated. This includes changes to route handlers in windmill-api, modifications to request/response schemas, changes to authentication requirements, or updates to the Flow data structures that affect the API.\n\nExamples:\n\n<example>\nContext: User has just added a new API endpoint for managing workspace templates.\nuser: "Add a new endpoint POST /api/w/{workspace}/templates to create workspace templates"\nassistant: "I've created the new endpoint handler in windmill-api. Now let me use the openapi-sync agent to update the OpenAPI specification files."\n<commentary>\nSince a new API endpoint was added, use the openapi-sync agent to ensure the openapi.yaml and openflow.openapi.yaml files are updated with the new endpoint definition.\n</commentary>\n</example>\n\n<example>\nContext: User has modified the response schema for an existing endpoint.\nuser: "Update the GET /api/w/{workspace}/flows endpoint to also return a 'versions' array in the response"\nassistant: "I've updated the flow listing endpoint to include the versions array. Now let me use the openapi-sync agent to update the OpenAPI specification."\n<commentary>\nSince the response schema of an existing endpoint was modified, use the openapi-sync agent to update the corresponding schema in the OpenAPI files.\n</commentary>\n</example>\n\n<example>\nContext: User has made changes to the Flow structure in the codebase.\nuser: "Add a new 'retry_policy' field to the Flow value structure"\nassistant: "I've added the retry_policy field to the Flow struct. Now let me use the openapi-sync agent to update the OpenAPI specification to reflect this schema change."\n<commentary>\nSince the Flow structure was modified, use the openapi-sync agent to ensure the flow-related schemas in openapi.yaml and openflow.openapi.yaml are updated.\n</commentary>\n</example>
model: inherit
---
You are an expert API documentation engineer specializing in OpenAPI specifications for the Windmill platform. Your primary responsibility is to maintain synchronization between the Rust backend API implementation and the OpenAPI specification files.
## Your Core Responsibilities
1. **Update OpenAPI Specifications**: When API endpoints are added, modified, or removed in the windmill-api crate, you must update:
- `backend/windmill-api/openapi.yaml` - The main OpenAPI specification
- `backend/windmill-api/openflow.openapi.yaml` - Flow-specific OpenAPI definitions (if flow-related changes)
2. **Maintain Schema Accuracy**: Ensure all request/response schemas accurately reflect the Rust structs used in the API handlers.
3. **Document Comprehensively**: Include proper descriptions, examples, and parameter documentation.
## Key Files to Reference
- **API Route Definitions**: Look in `backend/windmill-api/src/` for route handlers organized by domain
- **Data Structures**: Check `backend/windmill-common/src/` for shared structs and types
- **Database Schema**: Reference `backend/summarized_schema.txt` for understanding data models
- **Existing OpenAPI Files**: Always review the current state of `openapi.yaml` and `openflow.openapi.yaml` before making changes
## Workflow
1. **Identify Changes**: Determine what API changes were made by examining:
- New or modified route handlers in windmill-api
- Changes to request/response structs
- Modifications to the Flow structure or related types
2. **Analyze the Implementation**: For each endpoint, identify:
- HTTP method and path
- Path parameters, query parameters, and request body schema
- Response schema(s) and status codes
- Authentication requirements
- Any tags or groupings
3. **Update OpenAPI Files**:
- Add or modify path definitions with accurate operation IDs
- Update or create schema definitions in the components section
- Ensure $ref references are correct
- Maintain consistent naming conventions with existing patterns
4. **Validate Changes**: Ensure the YAML syntax is valid and follows OpenAPI 3.0 specification.
## OpenAPI Conventions for Windmill
- **Operation IDs**: Use camelCase, descriptive names (e.g., `createScript`, `listFlows`, `updateWorkspaceSettings`)
- **Tags**: Group endpoints by domain (e.g., `scripts`, `flows`, `workspaces`, `users`)
- **Schema Naming**: Use PascalCase for schema names matching Rust struct names
- **Path Parameters**: Use `{workspace}` for workspace_id, maintain consistency with existing patterns
- **Security**: Most endpoints require Bearer token authentication - include appropriate security requirements
## Schema Mapping from Rust to OpenAPI
- `String` / `&str``type: string`
- `i32`, `i64``type: integer` (with appropriate format)
- `f32`, `f64``type: number`
- `bool``type: boolean`
- `Vec<T>``type: array` with `items`
- `Option<T>` → property is not in `required` array
- `HashMap<K, V>``type: object` with `additionalProperties`
- Enums → `type: string` with `enum` array
- Custom structs → `$ref` to schema definition
## Important Notes
- Always preserve existing documentation and descriptions when updating
- Maintain backward compatibility warnings in descriptions when applicable
- Include example values where they aid understanding
- For Flow-related changes, update BOTH openapi.yaml AND openflow.openapi.yaml as needed
- Follow the existing indentation and formatting style in the YAML files
When you complete updates, summarize what changes were made to which files and highlight any schema additions or modifications that downstream consumers should be aware of.
-331
View File
@@ -1,331 +0,0 @@
#!/usr/bin/env bash
# PreToolUse allowance for scratch file ops: auto-allow `mkdir` / `cp` / `mv` / `touch` /
# `chmod` whose every path operand resolves inside one of the roots `path_class` recognizes —
# under /tmp, inside a git working tree under $HOME, or in an MCP browser cache — and
# `tar` / `unzip` confined to /tmp.
# Anything else makes no decision (exit 0) and falls back to the normal permission flow, except
# for `mv` and `chmod`: those get an explicit `ask`, the only prompt they get (see
# lib-guarded-verb.sh).
#
# The command is read one segment at a time, so chaining and line breaks carry no weight of
# their own: `cd /tmp/scratch && mv /tmp/a /tmp/b` is proved on the operands of the `mv`. A
# decision covers the whole command line, so `allow` is emitted only when every segment is one
# of these verbs proved here or a `cd` that resolved, AND exactly one of them writes (see the
# gate at the foot of this file — an earlier write can change what a later operand means). A
# line that mixes a proven op with some other command makes no decision instead and leaves that
# line to the normal permission flow, rather than waving an unexamined command through with it.
#
# This is a hook rather than an allow rule because permission rules match a command prefix, so
# they can only constrain the FIRST operand. `cp /tmp/x ~/.zshrc` matches a `cp /tmp/` prefix,
# and requiring every operand is the point.
#
# One operation may not straddle two roots, sources included, and a sibling checkout is a
# different root — `path_class` names the git tree, not just its kind. A copy out of a checkout
# into /tmp would be a read-exfiltration path around the `Read(**/secrets/**)` / `Read(**/*.pem)`
# deny rules, since the content lands where `Read(/tmp/**)` allows it to be read back, and one
# out of a repo the Read tool is not confined to would do the same for that repo. Keeping every
# operand of one operation inside a single root closes both without restating those rules here.
# The checkout root itself is what makes an in-repo `mv` or `chmod` auto-allowable: deleting a
# file there has never prompted, and moving or chmod-ing one is not the graver act.
#
# Deny-by-default tokenizing, in the same spirit as guard-rm-outside-tmp.sh: every path token
# must consist only of alphanumerics and `. _ / -`, the one exception being the leading `~/` or
# `$HOME/` that `expand_home_prefix` rewrites first. That set contains none of the characters
# bash uses for quoting, expansion, or command separation ($ ` ~ { } ( ) ' " \ ; & | < >), nor
# any glob character, so all of those forms fail by construction. `canon_path` then resolves
# `..` and existing symlinks, so `/tmp/link` pointing at /etc/passwd is caught.
#
# `tar` and `unzip` keep the stricter rule — /tmp only, and absolute operands only — because
# their positional grammar makes a bare word ambiguous: `tar P -xf ...` is --absolute-names,
# not a file named P, and resolving it as a path would put an option in a root and allow it.
# The other five take relative operands, resolved against the working directory that `cd`
# tracking maintains, since for those a bare word really is a path (a GNU option starts with
# `-`, and the option allowlist below rejects the ones that would change symlink handling).
#
# `tar` and `unzip` get their own parser: their write destination arrives as a flag VALUE
# (`-C`, `-d`) rather than a positional, and a bundle like `-xzf` consumes the token after it.
# Flags are an allowlist, not a denylist, so `-P` / `--absolute-names` — which turn off tar's
# refusal to extract `..` and absolute member paths — defer rather than needing enumeration.
# Extraction additionally requires an explicit destination under /tmp, or a working directory
# already under /tmp, since otherwise members land in the project checkout.
#
# Residual risk accepted: an archive whose members include a symlink pointing out of /tmp
# followed by a write through it can still escape, because tar applies member symlinks as it
# extracts. The archive itself must be under /tmp to get here, so this is a hazard only for
# archives fetched from an untrusted source into the scratch dir.
#
# Assumes `jq`. Path canonicalization goes through `canon_path`, which covers both the Linux dev
# env and macOS; with neither backend available it proves nothing and every op falls back.
set -uo pipefail
. "${BASH_SOURCE[0]%/*}/lib-guarded-verb.sh"
input=$(cat)
command -v jq >/dev/null 2>&1 || exit 0
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // empty' 2>/dev/null)
[ -z "$cmd" ] && exit 0
cwd=$(printf '%s' "$input" | jq -r '.cwd // empty' 2>/dev/null)
# Every bail-out below goes through `defer`: `mv` and `chmod` prompt from here, since no rule
# covers them, while the other verbs stay silent and leave the decision to the normal flow.
guarded=0
for verb in mv chmod; do
runs_verb "$verb" "$cmd" && { guarded=1; break; }
done
defer() {
[ "$guarded" = 1 ] && decide ask "$1"
exit 0
}
has_substitution "$cmd" && defer "command substitution in the command line"
# 0 iff the token is a literal path this hook may reason about. A glob never auto-allows: bash
# expands it only after the hook has decided, so realpath sees the unexpanded pattern —
# `/tmp/link*` canonicalizes to itself and passes, then expands onto a symlink whose target is
# outside, and `cp` and `chmod` follow a command-line symlink, so that is a write to the target.
# (guard-rm-outside-tmp.sh can allow globs because `rm` unlinks the symlink rather than following
# it.) The charset holds none of the characters bash uses for quoting, expansion or separation.
literal_path() {
case "$1" in *[*?[]*) return 1 ;; esac
[ -z "$(printf '%s' "$1" | tr -d 'A-Za-z0-9._/-')" ]
}
# Prints the root class of a path token, then the path it resolved to on a second line,
# resolving a relative one against the tracked working directory. Fails, printing nothing,
# when the token is unsafe to reason about or lands outside every root.
operand_class() {
local t canon alt cls alt_cls=""
t=$(expand_home_prefix "$1")
literal_path "$t" || return 1
case "$t" in
/*) canon=$(canon_path "$t") ;;
*) # A `cd` may fail at runtime and leave the command where it started, so a relative
# operand has to land in the same root either way.
[ -n "$seg_cwd" ] || return 1
canon=$(canon_path "$seg_cwd/$t")
if [ -n "$alt_cwd" ]; then
alt=$(canon_path "$alt_cwd/$t")
[ -n "$alt" ] || return 1
alt_cls=$(path_class "$alt") || return 1
fi
;;
esac
[ -n "$canon" ] || return 1
cls=$(path_class "$canon") || return 1
[ -n "$alt_cls" ] && [ "$alt_cls" != "$cls" ] && return 1
# Class and resolved path together: a caller runs this in a command substitution, so a global
# set here would be set in that subshell and lost.
printf '%s\n%s' "$cls" "$canon"
}
# 0 iff the token is charset-safe and resolves to a path strictly inside /tmp. The archive
# parser's stricter check; everything else goes through operand_class.
under_tmp() {
local t canon
t=$(expand_home_prefix "$1")
literal_path "$t" || return 1
case "$t" in /*) ;; *) return 1 ;; esac
canon=$(canon_path "$t")
[ -n "$canon" ] || return 1
# /tmp itself is never a target — only paths strictly inside it.
case "$canon" in "$TMP_ROOT"/?*) return 0 ;; esac
return 1
}
# Proves one `tar` / `unzip` segment ($1 = the verb), whose tokens are in SEG_TOKS.
check_archive_segment() {
local verb="$1" ok_flags val_flags t flags val
local saw_archive=0 saw_dest=0 extracting=0 listing=0 end_opts=0 i=1
case "$verb" in
tar) ok_flags='xctzjJavfC'; val_flags='fC' ;;
unzip) ok_flags='oqnljvd'; val_flags='d' ;;
esac
while [ "$i" -lt "${#SEG_TOKS[@]}" ]; do
t="${SEG_TOKS[$i]}"
i=$((i + 1))
if [ "$end_opts" = 0 ]; then
[ "$t" = "--" ] && { end_opts=1; continue; }
case "$t" in
-?*)
flags="${t#-}"
# Allowlist: a long option, -P/--absolute-names, --transform, -I and friends all
# leave a residue here and defer rather than being enumerated as denials.
[ -n "$(printf '%s' "$flags" | tr -d "$ok_flags")" ] && defer "unrecognized option \`$t\`"
case "$flags" in *x*) extracting=1 ;; esac
case "$verb$flags" in unzip*[lv]*) listing=1 ;; esac
# A flag consuming the next token must be alone in its bundle's final position
# (`-xzf a.tar`), else the token it eats is ambiguous.
case "${flags%?}" in *[$val_flags]*) defer "ambiguous option bundle \`$t\`" ;; esac
case "${flags: -1}" in
[$val_flags])
val="${SEG_TOKS[$i]:-}"
i=$((i + 1))
[ -n "$val" ] || defer "option \`$t\` has no value"
under_tmp "$val" || defer "\`$val\` is outside /tmp"
case "${flags: -1}" in
f) saw_archive=1 ;;
C | d) saw_dest=1 ;;
esac
;;
esac
continue
;;
esac
fi
# Positional. For tar these are sources (create) or member names (extract); for unzip the
# first is the archive. Requiring every one under /tmp is conservative for member names,
# which are not filesystem paths — those defer rather than being wrongly allowed.
under_tmp "$t" || defer "\`$t\` is outside /tmp"
[ "$verb" = "unzip" ] && saw_archive=1
done
# tar without -f reads a tape/stdin; unzip needs an archive
[ "$saw_archive" = 1 ] || defer "no archive operand"
# Writes land relative to the working directory unless a destination was given. `unzip -l`
# and `-v` only list, so they need no destination.
if [ "$extracting" = 1 ] || { [ "$verb" = "unzip" ] && [ "$listing" = 0 ]; }; then
# An extraction with no destination lands in the working directory. Word splitting cannot
# tell a `cd` inside a quoted string from one the shell runs, and believing a false one
# would put an archive's members in the checkout, so once any `cd` is in the line only an
# explicit destination will do.
[ "$saw_dest" = 1 ] \
|| { [ "$saw_cd" = 0 ] && [ -n "$seg_cwd" ] && under_tmp "$seg_cwd"; } \
|| defer "extraction target is outside /tmp"
fi
}
# Proves one `mkdir` / `cp` / `mv` / `touch` / `chmod` segment ($1 = the verb), whose tokens
# are in SEG_TOKS.
check_fileops_segment() {
local verb="$1" takes_mode ok_opts t cls resolved dest seen_class=""
local path_operand=0 seen_mode=0 end_opts=0 i=1 rel_operand=0
local -a ops=()
# Options are an allowlist per command, so anything that changes how symlinks are followed
# defers instead of needing enumeration. `cp -L` / `-H` matter most: they dereference while
# recursing, which copies the CONTENT of a symlink target from outside /tmp into a scratch
# dir that `Read(/tmp/**)` then exposes. Plain `-r` and `-a` (which implies `-d`) recreate
# such a symlink as a symlink instead, so no outside content is materialized.
case "$verb" in
mkdir) takes_mode=0; ok_opts='pv' ;;
cp) takes_mode=0; ok_opts='rRvfnpa' ;;
mv) takes_mode=0; ok_opts='vfn' ;;
touch) takes_mode=0; ok_opts='acmv' ;;
chmod) takes_mode=1; ok_opts='Rvfc' ;; # chmod's first operand is a mode, not a path
esac
while [ "$i" -lt "${#SEG_TOKS[@]}" ]; do
t="${SEG_TOKS[$i]}"
i=$((i + 1))
if [ "$end_opts" = 0 ]; then
[ "$t" = "--" ] && { end_opts=1; continue; }
# Checked at any position, not just before the first operand: GNU utils permute, so
# `cp /tmp/tree -RL /tmp/out` still enables dereferencing recursion.
case "$t" in
-?*)
# Allowlist: long options and the dereferencing flags leave a residue and defer.
[ -n "$(printf '%s' "${t#-}" | tr -d "$ok_opts")" ] && defer "unrecognized option \`$t\`"
continue
;;
esac
fi
# chmod: consume the mode operand without a path check. Octal, or symbolic clauses.
if [ "$takes_mode" = 1 ] && [ "$seen_mode" = 0 ]; then
case "$t" in
[0-7] | [0-7][0-7] | [0-7][0-7][0-7] | [0-7][0-7][0-7][0-7]) ;;
*) printf '%s' "$t" | grep -Eq '^[ugoa]*[+=-][rwxXst]*(,[ugoa]*[+=-][rwxXst]*)*$' || defer "unrecognized mode \`$t\`" ;;
esac
seen_mode=1
continue
fi
resolved=$(operand_class "$t") || defer "\`$t\` is outside /tmp and the MCP caches, and not inside a git checkout in \$HOME"
cls="${resolved%%$'\n'*}"
# Every operand of one operation stays in one root: see the exfiltration note above.
[ -n "$seen_class" ] && [ "$cls" != "$seen_class" ] && defer "\`$t\` puts this $verb across two roots"
seen_class="$cls"
ops+=("${resolved#*$'\n'}")
# Against the expanded token, since `~/a` is cwd-independent and only reads as relative
# before `expand_home_prefix` has run.
case "$(expand_home_prefix "$t")" in /*) ;; *) rel_operand=1 ;; esac
path_operand=1
done
[ "$path_operand" = 1 ] || defer "no path operand"
# In directory form the command writes a path it does not name: `cp x dir` writes `dir/x`,
# and `cp` follows that child when it is a symlink — this checkout is full of them, every
# `*_ee.rs` pointing into the sibling EE repo. Deriving that child would mean reproducing
# which name the tool picks (the operand as written, not as resolved — a symlinked source
# keeps its own name) and how deep `-r` recurses. The form is left unproved instead.
case "$verb" in
cp | mv)
[ "${#ops[@]}" -ge 2 ] || return 0
# Whether the destination is an existing directory is itself a question about which of
# the two candidate working directories the command ran in, and only one of them is in
# `ops`. A `cd` that fails at runtime would otherwise let the form through: the
# destination resolved against the directory the command never reached is some path that
# does not exist, while the one it actually ran in is a directory full of symlinks.
[ -n "$alt_cwd" ] && [ "$rel_operand" = 1 ] \
&& defer "a relative operand after a \`cd\` lands in one of two directories"
# Index arithmetic rather than `${ops[-1]}`: macOS ships bash 3.2, where a negative
# subscript is a fatal error and would abort the guard mid-decision.
dest="${ops[$((${#ops[@]} - 1))]}"
[ -d "$dest" ] \
&& defer "\`$dest\` already exists as a directory, so this $verb writes a path it does not name"
;;
esac
}
split_segments "$cmd"
seg_cwd="${cwd:-$PWD}"
alt_cwd="" # where a `cd` that failed would have left the command
saw_cd=0 # a `cd` moved the working directory somewhere
proved=0 # how many ops came out inside a single root
only_ours=1 # ... and nothing else shares the command line
for seg in "${SEGMENTS[@]}"; do
segment_tokens "$seg"
case "${SEG_TOKS[0]:-}" in
"") continue ;;
mkdir | cp | mv | touch | chmod)
check_fileops_segment "${SEG_TOKS[0]}"
proved=$((proved + 1))
continue
;;
tar | unzip)
check_archive_segment "${SEG_TOKS[0]}"
proved=$((proved + 1))
continue
;;
cd)
# A `cd` writes nothing, so it never blocks an allow; it only moves where a later relative
# operand points, to one of the two candidates `apply_cd` describes.
if [ "$saw_cd" = 0 ] && new_cwd=$(apply_cd "$seg_cwd" "${SEG_TOKS[@]:1}"); then
alt_cwd="$seg_cwd"
seg_cwd="$new_cwd"
else
# Not the harmless segment an allow assumes: whatever this guard could not account for
# may be a redirect, and a redirect writes. Leave the line to the normal flow.
seg_cwd="" alt_cwd=""
only_ours=0
fi
saw_cd=1
continue
;;
esac
# Some other command shares the line. If an `mv` or `chmod` runs inside it after all — behind
# a wrapper, an env prefix or a path — this hook cannot say what it writes to.
for verb in mv chmod; do
segment_runs_verb "$verb" "$seg" && defer "$verb is not the leading command word in \`$seg\`"
done
only_ours=0
done
# Exactly one write per line. Each segment is proved against the filesystem as it stands now,
# and an earlier write can change what a later operand means: `cp -r /tmp/tree /tmp/live` that
# recreates a symlink out of /tmp turns `/tmp/live/link` — a path under /tmp when this ran —
# into a write through that symlink. Deletes compose safely and guard-rm-outside-tmp.sh allows
# several, because `rm` unlinks a symlink rather than following it.
[ "$proved" -ge 1 ] || exit 0
[ "$only_ours" = 1 ] && [ "$proved" = 1 ] && decide allow "every path operand is inside a single root"
exit 0
-22
View File
@@ -1,22 +0,0 @@
#!/bin/bash
# Format backend Rust files with rustfmt after Claude edits them
# Get the file path from the tool result (passed via stdin as JSON)
INPUT=$(cat)
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
# Exit if no file path
if [ -z "$FILE_PATH" ]; then
exit 0
fi
# Check if the file is in the backend directory and is a Rust file
if [[ "$FILE_PATH" == *"/backend/"* ]] && [[ "$FILE_PATH" =~ \.rs$ ]]; then
cd "$CLAUDE_PROJECT_DIR/backend" || exit 0
# Run rustfmt, surface errors as context but don't block Claude
if rustfmt --config-path rustfmt.toml "$FILE_PATH" 2>&1; then
echo "Formatted $(basename "$FILE_PATH")"
fi
fi
exit 0
-29
View File
@@ -1,29 +0,0 @@
#!/bin/bash
# Format frontend files with prettier after Claude edits them
# Get the file path from the tool result (passed via stdin as JSON)
INPUT=$(cat)
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
# Exit if no file path
if [ -z "$FILE_PATH" ]; then
exit 0
fi
# Only the frontend app itself, i.e. a "frontend" directory sitting at a repo root.
# A bare */frontend/* substring also matches ai_evals/adapters/frontend and the
# ai_evals app fixtures, which no prettier config governs — prettier then falls back
# to its defaults and rewrites the whole file. Anchoring to $CLAUDE_PROJECT_DIR
# instead would skip worktrees edited from a session rooted elsewhere.
if [[ "$FILE_PATH" == *"/frontend/"* ]] && [[ -e "${FILE_PATH%%/frontend/*}/.git" ]]; then
# Check if it's a formattable file type
if [[ "$FILE_PATH" =~ \.(ts|js|svelte|json|css|html|md)$ ]]; then
cd "$CLAUDE_PROJECT_DIR/frontend" || exit 0
# Run prettier, surface errors as context but don't block Claude
if ./node_modules/.bin/prettier --plugin prettier-plugin-svelte --write "$FILE_PATH" 2>&1; then
echo "Formatted $(basename "$FILE_PATH")"
fi
fi
fi
exit 0
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# PreToolUse hook: block destructive git operations when on the main branch.
# Non-git tool calls and read-only git commands pass through silently.
set -euo pipefail
input="$(cat)"
tool_name="$(echo "$input" | jq -r '.tool_name // empty')"
# Only care about Bash tool calls
[[ "$tool_name" == "Bash" ]] || exit 0
command="$(echo "$input" | jq -r '.tool_input.command // empty')"
# Only care about git write commands
if [[ "$command" =~ ^git\ (push|reset|revert|checkout|merge|rebase|commit|add) ]]; then
branch="$(git rev-parse --abbrev-ref HEAD 2>/dev/null || true)"
if [[ "$branch" == "main" ]]; then
echo "BLOCK: You are on the main branch. Create or switch to a feature branch first." >&2
exit 2
fi
fi
# Block force-push targeting main from any branch.
if [[ "$command" =~ ^git[[:space:]]+push([[:space:]]|$) ]]; then
has_force=false
if [[ "$command" =~ (--force([[:space:]]|=|$)|--force-with-lease|[[:space:]]-f([[:space:]]|$)) ]]; then
has_force=true
fi
# `+ref` refspec syntax is also a force push.
if [[ "$command" =~ [[:space:]]\+[A-Za-z] ]]; then
has_force=true
fi
if $has_force && [[ "$command" =~ (^|[[:space:]:])\+?main([[:space:]]|$) ]]; then
echo "BLOCK: Force-push to main is not allowed via Claude. Run it yourself if you really mean to." >&2
exit 2
fi
fi
-154
View File
@@ -1,154 +0,0 @@
#!/usr/bin/env bash
# PreToolUse guard for `rm`: auto-allow deletes whose every operand is a whitelisted target —
# under /tmp, inside a git working tree located in $HOME (a version-controlled project dir), or
# in one of the browser-automation caches the MCP servers rebuild on demand.
# Any other command that runs `rm` gets an explicit `ask`, which is the ordinary permission
# prompt and the only one `rm` gets (see lib-guarded-verb.sh); a command that runs no `rm` at
# all makes no decision (exit 0).
#
# The command is read one segment at a time, so chaining and line breaks carry no weight of
# their own: `rm -f /tmp/a && rm -rf /tmp/b` is two deletes, each proved on its own operands.
# A decision covers the whole command line, so `allow` is emitted only when every segment is
# an `rm` this guard proved or a `cd` it could resolve. A line that mixes a proven `rm` with
# some other command makes no decision instead and leaves that line to the normal permission
# flow: the delete is not what needed a prompt, and waving the rest of the line through with
# it would turn a trailing `rm -f /tmp/x` into a way to auto-approve anything.
#
# Deny-by-default: every token must consist only of a safe character set (alphanumerics,
# `. _ / -` and glob chars `* ? [ ]`), the one exception being the leading `~/` or `$HOME/` that
# `expand_home_prefix` rewrites first. That set contains none of the characters bash uses for
# quoting, expansion, or command separation ($ ` ~ { } ( ) ' " \ ; & | < >), so those forms
# fail by construction rather than needing to be enumerated. `canon_path` then resolves `..`
# and existing symlinks (so a symlink out of the allowed roots is caught), and a wildcard in a
# non-final path segment is refused because it can expand through a symlink realpath can't see.
#
# Which targets those roots cover, and the tradeoff they rest on, is `path_class` in
# lib-guarded-verb.sh. Globs auto-allow only under /tmp and the MCP caches — elsewhere their
# expansion could reach `.git` or a dotfile the literal checks never see. Relative operands resolve
# against the working directory the command runs from, which a `cd` in an earlier segment
# moves; once a `cd` is one this guard cannot resolve, that directory is unknown and a
# relative operand can no longer be proved.
#
# Assumes `jq`. Path canonicalization goes through `canon_path`, which covers both the Linux dev
# env and macOS; with neither backend available it proves nothing and every delete prompts.
set -uo pipefail
. "${BASH_SOURCE[0]%/*}/lib-guarded-verb.sh"
input=$(cat)
command -v jq >/dev/null 2>&1 || exit 0
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // empty' 2>/dev/null)
[ -z "$cmd" ] && exit 0
cwd=$(printf '%s' "$input" | jq -r '.cwd // empty' 2>/dev/null)
# Every bail-out below goes through `defer`, so the forms this guard refuses to reason about —
# wrapped, quoted, expanded — still reach the user as a prompt whenever an `rm` runs among them.
runs_verb rm "$cmd" && guarded=1 || guarded=0
defer() {
[ "$guarded" = 1 ] && decide ask "$1"
exit 0
}
has_substitution "$cmd" && defer "command substitution in the command line"
# Proves one `rm` segment, whose tokens are in SEG_TOKS with `rm` at index 0, resolving relative
# operands against $seg_cwd. Returns only once every operand is an auto-allowable target;
# anything it cannot prove defers instead.
check_rm_segment() {
local i=1 t p canon candidates had_operand=0 end_opts=0
while [ "$i" -lt "${#SEG_TOKS[@]}" ]; do
t="${SEG_TOKS[$i]}"
i=$((i + 1))
# Messages keep the token as written; everything downstream reasons about the expansion.
p=$(expand_home_prefix "$t")
# Whitelist every token (flags included, so an operator hidden in a flag like `-rf;rm`
# can't slip past): any character outside the safe set makes it unsafe to reason about.
[ -n "$(printf '%s' "$p" | tr -d 'A-Za-z0-9._/*?[]-')" ] && defer "unsafe characters in \`$t\`"
# A glob in an option-looking token (`-[-]`) can expand to `--` and turn a later `-name`
# into an operand — never a real option, so defer.
case "$t" in -*[*?[]*) defer "glob inside the option \`$t\`" ;; esac
if [ "$end_opts" = 0 ]; then
[ "$t" = "--" ] && { end_opts=1; continue; }
# Skip real options only before the first operand. A bare `-` is a filename, and under
# POSIXLY_CORRECT GNU rm stops option parsing at the first operand, so a later `-name`
# is a filename too — validate it rather than skipping it.
if [ "$had_operand" = 0 ]; then
case "$t" in -?*) continue ;; esac
fi
fi
had_operand=1
# No wildcard in a non-final path segment (`a/*/b`): it can expand through a symlink
# realpath can't see. A slashless glob (`*.rs`) is a final-segment match — fine.
case "$p" in */*) case "${p%/*}" in *[*?[]*) defer "glob in a non-final segment of \`$t\`" ;; esac ;; esac
# A relative operand has as many candidate paths as the command has candidate working
# directories, and every one of them has to be auto-allowable: a `cd` that fails at runtime
# leaves the delete running in the directory it started in.
case "$p" in
/*) candidates=$(canon_path "$p") ;;
*) [ -n "$seg_cwd" ] || defer "\`$t\` is relative to a working directory this guard cannot pin down"
candidates=$(canon_path "$seg_cwd/$p")
[ -n "$alt_cwd" ] && candidates="$candidates
$(canon_path "$alt_cwd/$p")"
;;
esac
while IFS= read -r canon; do
[ -n "$canon" ] || defer "cannot resolve \`$t\`"
# A glob may auto-allow only in a root where everything is deletable — /tmp and the MCP
# caches, both of which `rm -rf <root>` already clears wholesale, so matching inside one
# grants nothing more. In a checkout the expansion could reach `.git`, a dotfile like
# `.*`, or a nested checkout root that the literal-path checks never see, so require
# literal operands there.
case "$p" in
*[*?[]*)
case "$(path_class "$canon")" in
tmp | mcp-cache) ;;
*) defer "glob \`$t\` is outside /tmp and the MCP caches" ;;
esac
;;
esac
path_class "$canon" >/dev/null || defer "\`$canon\` is outside /tmp and the MCP caches, and not inside a git checkout in \$HOME"
done <<< "$candidates"
done
[ "$had_operand" = 1 ] || defer "no operand"
}
split_segments "$cmd"
seg_cwd="${cwd:-$PWD}"
alt_cwd="" # where a `cd` that failed would have left the command
saw_cd=0
proved=0 # at least one `rm` segment came out auto-allowable
only_ours=1 # ... and nothing else shares the command line
for seg in "${SEGMENTS[@]}"; do
segment_tokens "$seg"
case "${SEG_TOKS[0]:-}" in
"") continue ;;
rm)
check_rm_segment
proved=1
continue
;;
cd)
# A `cd` writes nothing, so it never blocks an allow; it only moves where a later relative
# operand points, to one of the two candidates `apply_cd` describes.
if [ "$saw_cd" = 0 ] && new_cwd=$(apply_cd "$seg_cwd" "${SEG_TOKS[@]:1}"); then
alt_cwd="$seg_cwd"
seg_cwd="$new_cwd"
else
# Not the harmless segment an allow assumes: whatever this guard could not account for
# may be a redirect, and a redirect writes. Leave the line to the normal flow.
seg_cwd="" alt_cwd=""
only_ours=0
fi
saw_cd=1
continue
;;
esac
# Some other command shares the line. If an `rm` runs inside it after all — behind a wrapper,
# an env prefix or a path — this guard cannot say what it deletes.
segment_runs_verb rm "$seg" && defer "rm is not the leading command word in \`$seg\`"
only_ours=0
done
[ "$proved" = 1 ] || exit 0
[ "$only_ours" = 1 ] && decide allow 'rm operands are under /tmp, in an MCP cache, or inside a git checkout in $HOME'
exit 0
-331
View File
@@ -1,331 +0,0 @@
#!/usr/bin/env bash
# Sourced by the PreToolUse guards; not a hook itself.
#
# A permission rule beats a hook: an `ask` rule prompts whatever a PreToolUse hook returns, which
# makes the hook's `allow` dead weight. So settings.json carries no `ask` rule for `rm`, `mv` or
# `chmod`, and the guards own both halves — `allow` what they can prove safe, `ask` for the rest.
# Removing a guard's `ask` path therefore removes that verb's prompt entirely.
#
# `set -f` is global to the sourcing script so that the unquoted word split in runs_verb cannot
# expand a glob operand against the filesystem. Neither guard relies on pathname expansion.
set -f
# Canonical absolute path: `..` and existing symlinks resolved, missing trailing components
# allowed. Resolving symlinks is the load-bearing half — a lexical normalizer would collapse
# `/tmp/link/..` without seeing where `link` points, and let an operand out of its root.
# GNU `realpath -m` is exactly this; BSD realpath on macOS has no `-m` and exits on it, which
# would leave every operand unresolvable and every delete prompting, so fall back to python3's
# os.path.realpath, which has the same semantics. Trying rather than probing keeps the cost off
# the Bash calls that never reach a path check — most of them. With neither available this
# prints nothing, and every caller treats that as "cannot prove".
canon_path() {
local out
out=$(realpath -m -- "$1" 2>/dev/null) && [ -n "$out" ] && { printf '%s' "$out"; return; }
python3 -c 'import os,sys;sys.stdout.write(os.path.realpath(sys.argv[1]))' "$1" 2>/dev/null
}
# The roots every class is anchored to, in the form a canonicalized operand comes back in. On
# macOS /tmp is a symlink to /private/tmp, so a resolved scratch path never starts with `/tmp`
# and matching the literal would put every scratch path outside every class. Both exist, so
# `cd -P` resolves them without the process canon_path would spawn on every sourcing.
TMP_ROOT=$(cd -P -- /tmp 2>/dev/null && pwd)
[ -n "$TMP_ROOT" ] || TMP_ROOT=/tmp
HOME_ROOT=""
[ -n "${HOME:-}" ] && HOME_ROOT=$(cd -P -- "$HOME" 2>/dev/null && pwd)
# Prints <token> ($1) with a leading `~/`, `$HOME/` or `${HOME}/` — and those three words on
# their own — replaced by the home directory, so the ordinary spelling of a path outside every
# checkout can still be proved. Only that prefix and only those spellings: `~user/` names another
# account, and any other `$` is an expansion nothing here can evaluate, so both stay in the token
# and fail the caller's charset check. A quoted token keeps its quotes and fails there too.
expand_home_prefix() {
[ -n "$HOME_ROOT" ] || { printf '%s' "$1"; return; }
case "$1" in
'~' | '$HOME' | '${HOME}') printf '%s' "$HOME_ROOT" ;;
'~/'*) printf '%s/%s' "$HOME_ROOT" "${1#'~/'}" ;;
'$HOME/'*) printf '%s/%s' "$HOME_ROOT" "${1#'$HOME/'}" ;;
'${HOME}/'*) printf '%s/%s' "$HOME_ROOT" "${1#'${HOME}/'}" ;;
*) printf '%s' "$1" ;;
esac
}
# 0 iff <text> ($1) starts with a command that only reads its input. An allowlist, because the
# opposite — naming the shells to avoid — would have to be complete: an unlisted one (`ash`,
# `rbash`, `busybox sh`) executes the body while the guard calls it data. Unrecognized here only
# costs a prompt. Text with no command word in it is not evidence of a reader either.
reads_only() {
local w
for w in $1; do
w="${w//[\"\'\\]/}"
w="${w%%<<*}" # a redirect needs no space: `cat<<EOF`
case "$w" in "" | -* | *=* | [0-9]* | '>'* | '<'*) continue ;; esac
case "${w##*/}" in
cat | tee | head | tail | grep | sed | awk | sort | uniq | wc | cut | diff | tr \
| jq | yq | gh | git | base64 | column | envsubst | python | python3 | node \
| psql | mysql | sqlite3 | wmill) return 0 ;;
esac
return 1
done
return 1
}
# A heredoc body is data rather than commands only when its delimiter is quoted and nothing
# executes it; a rule doesn't match a verb inside such a body, and a PR body would otherwise
# prompt for every `rm` in its text. Dropping one needs all of that, a delimiter that could
# really open a heredoc, and a terminator line — failing any part, nothing is dropped.
strip_heredoc_bodies() {
local -a lines=()
local line delim rest after trimmed piped quoted i j n
while IFS= read -r line; do lines+=("$line"); done <<< "$1"
n=${#lines[@]}
i=0
while [ "$i" -lt "$n" ]; do
line="${lines[$i]}"
printf '%s\n' "$line"
i=$((i + 1))
# A `#` opens a comment, and a comment opens no heredoc — including mid-line, as in
# `echo hi # cat <<EOF`. Cutting there also discards a `#` that is really part of a word or
# a string, which at worst leaves a real body to be scanned: an extra prompt, never a lost one.
line="${line%%'#'*}"
case "$line" in *'<<'*) ;; *) continue ;; esac
rest="${line#*<<}"
rest="${rest#-}" # <<- strips leading tabs from the body
rest="${rest#"${rest%%[![:space:]]*}"}"
delim="${rest%%[[:space:]]*}"
# Whatever follows the delimiter word decides whether this line could open a heredoc at
# all. Only a redirect or a pipe can (`cat <<EOF > f`); prose after it means the `<<` sits
# inside a string (`echo "cat <<EOF and more"`), and dropping down to a line that happens
# to match would discard the real commands in between. A quote anywhere in the remainder
# says the same thing, since `echo "cat <<EOF > f"` ends its redirect-looking text with the
# closing quote. That also refuses `cat <<EOF > "f"`, a real heredoc, which only over-prompts.
after="${rest#"$delim"}"
after="${after#"${after%%[![:space:]]*}"}"
case "$after" in
*[\"\'\\]*) continue ;;
"" | '>'* | '<'* | '|'* | [0-9]'>'* | [0-9]'<'*) ;;
*) continue ;;
esac
# A real delimiter is a bare word or one wholly quoted (`<<'EOF'`, `<<\EOF`); a stray quote
# left in it means the `<<` was quoted prose.
quoted=0
case "$delim" in
\'*\' | \"*\") delim="${delim:1:${#delim}-2}" quoted=1 ;;
\\?*) delim="${delim#\\}" quoted=1 ;;
esac
case "$delim" in
[A-Za-z_]*) ;;
*) continue ;;
esac
case "$delim" in *[!A-Za-z0-9_]*) continue ;; esac
# Only a quoted delimiter makes the body inert. Unquoted, the shell expands it before the
# consumer ever sees it, so a `$(rm -rf ~)` written in the body runs whatever reads it.
[ "$quoted" = 1 ] || continue
# Two commands can see this body: the one the `<<` belongs to, and anything it is then piped
# into. The first is whatever was started last before the `<<`, so splitting the text there
# on separators and substitution openers and taking the final piece finds `cat` in
# `--title "fix(agents): …" --body "$(cat <<`, without the title's parenthesis standing in
# for it. A line continuation (`bash \` then `<<'EOF'`) leaves that piece empty, which is
# not evidence of a reader and so keeps the body.
reads_only "$(printf '%s' "${line%%<<*}" | tr ';&|()`' '\n' | grep -v '^[[:space:]]*$' | tail -1)" || continue
piped="$after"
while :; do
case "$piped" in *'|'*) ;; *) break ;; esac
piped="${piped#*|}"
reads_only "${piped%%|*}" || continue 2
done
j="$i"
while [ "$j" -lt "$n" ]; do
trimmed="${lines[$j]#"${lines[$j]%%[![:space:]]*}"}"
[ "$trimmed" = "$delim" ] && break
j=$((j + 1))
done
[ "$j" -lt "$n" ] && i=$((j + 1))
done
}
# 0 iff <verb> ($1) runs as a command word in <segment> ($2), which must already be one
# segment (no separator left in it). Wrapper, env-prefix and `/bin/<verb>` forms all count.
segment_runs_verb() {
local verb="$1" w wrapped=0
for w in $2; do
# The shell strips quotes and backslashes before it looks up the command, so `'rm'` and
# `r\m` run rm and have to compare equal to it.
w="${w//[\"\'\\]/}"
case "$w" in
"$verb" | */"$verb") return 0 ;;
*=*) ;; # leading env assignment
-* | *'>'* | *'<'*) ;; # a flag, or a leading redirect
[0-9]*) [ "$wrapped" = 1 ] || break ;; # a wrapper's duration, not `1:` in prose
'!' | '{' | '}' | if | then | elif | else | while | until | do) ;; # never the command
timeout | time | nice | nohup | stdbuf | command | builtin | noglob | xargs | sudo | env)
wrapped=1 ;;
# A wrapper's option value is indistinguishable from a command name (`stdbuf -o L rm`),
# so past a wrapper the scan runs to the end of the segment instead of stopping at the
# first ordinary word. Before one, that word is the command and the verb cannot follow
# it. Nothing bounds the scan: a wrapper takes unboundedly many operands
# (`env -u A -u B ...`), and any cutoff — a word count, or stopping at the first quoted
# word — drops the prompt for a real `sudo -u 'root' rm`. Prose after a wrapper is the
# price, and it only over-prompts.
*) [ "$wrapped" = 1 ] || break ;;
esac
done
return 1
}
# Splits <command> ($1) into its command segments, into the global array SEGMENTS. Every guard
# reasons one segment at a time, so `a && b` is two commands here rather than one unparsable
# blob, and a newline is a separator like any other.
#
# The split set carries more than `; & |` and newlines: `$(`, backticks and `( )` open a nested
# command, and a separator that only ended statements would read `echo $(rm -rf ~)` as an
# `echo`. Braces are handled as words rather than separators, since splitting on them cuts
# `xargs -I {} … rm` in half and strands the `rm` in a segment that no longer knows a wrapper
# preceded it.
#
# `tr` and not `${1//[...]}`: a `}` inside the bracket expression closes the expansion itself,
# which silently leaves the command unsplit and every separator unseen.
split_segments() {
local seg
SEGMENTS=()
while IFS= read -r seg; do SEGMENTS+=("$seg"); done <<< "$(strip_heredoc_bodies "$1" | tr ';&|()`' '\n')"
}
# 0 iff <command> ($1) carries a command substitution outside a heredoc body. A substitution is
# concatenated into the word it sits in, and splitting on its opener cuts that word in half:
# `/tmp/a/`printf ../../etc`` would be proved as `/tmp/a/`, with the traversal validated as an
# unrelated segment. Nothing here can evaluate it, so a guard proves nothing about such a
# command. Heredoc bodies are excepted — those are data the split has already dropped.
has_substitution() {
case "$(strip_heredoc_bodies "$1")" in
*'$('* | *'`'*) return 0 ;;
esac
return 1
}
# Reads <segment> ($1) into the global array SEG_TOKS, dropping the shell keywords that can
# precede a command word so that `then rm -rf x` is analyzed as the `rm` it runs. Word
# splitting only: quotes are left in the token and fail the guards' charset check downstream,
# which is what keeps `rm -rf "$HOME/x"` unprovable.
segment_tokens() {
SEG_TOKS=()
read -r -a SEG_TOKS <<< "$1"
while [ "${#SEG_TOKS[@]}" -gt 0 ]; do
case "${SEG_TOKS[0]}" in
'!' | '{' | '}' | if | then | elif | else | while | until | do) SEG_TOKS=("${SEG_TOKS[@]:1}") ;;
*) break ;;
esac
done
}
# Prints the directory a `cd` lands in, given the current one ($1) and the tokens after the
# `cd` ($2...). Fails, printing nothing, when the destination cannot be resolved — a variable,
# `-`, an option, a relative path, no operand at all (`cd` alone is $HOME), or more than one.
#
# Resolving says nothing about whether the `cd` will SUCCEED: the destination may not exist, and
# `;` runs the next command anyway, leaving it in the directory it started in. So a caller may
# never treat this as the working directory outright — it is one of two candidates, and a
# relative operand has to be provable against the one the command started in as well. That also
# makes a `cd` word splitting invented out of quoted text harmless: it can only add a candidate,
# never drop one. Past the first `cd` the branching outruns two candidates, so a caller that
# sees a second gives up on relative operands entirely.
apply_cd() {
local cwd="$1" t
shift
[ "$#" -eq 1 ] || return 1
t=$(expand_home_prefix "$1")
[ -n "$(printf '%s' "$t" | tr -d 'A-Za-z0-9._/-')" ] && return 1
# Absolute only. A relative destination is not `$cwd/$t`: the shell searches $CDPATH first,
# so `cd ssh` may land in /etc/ssh, and this cannot see the caller's $CDPATH to rule it out.
case "$t" in /*) ;; *) return 1 ;; esac
canon_path "$t"
}
# Prints the class of a canonical path and returns 0: `tmp` for one strictly under /tmp,
# `mcp-cache` for one in a browser-automation cache the MCP servers rebuild on demand, or
# `repo:<root>` for one strictly inside the git working tree at <root>, itself under $HOME.
# Fails, printing nothing, for anything else — those are the only roots the guards are willing
# to touch unprompted. The root is part of the class so that a caller pairing two operands can
# tell one checkout from another: sibling repos are separate permission boundaries, not one.
#
# The `repo` class trades on "this is a project under version control" being lower-stakes than
# the same act elsewhere — NOT on full recoverability: committed content is restorable via git,
# but untracked / .gitignore'd / uncommitted content, and an independent nested repo's history
# under a recursively-deleted parent, are NOT. Accepted as a deliberate convenience tradeoff.
#
# The walk stops at $HOME, so a dotfiles repo at ~ can't put all of $HOME in a class, and
# top-level ~ files stay out of one. A working tree's own root folder counts only when it is a
# linked worktree, whose `.git` is a pointer file so the history lives in the main repo and
# survives; a primary checkout's `.git` is a directory holding the history itself, so losing it
# is unrecoverable.
#
# Some paths are in no class in any root, /tmp included. Git history, and the agent's own guards
# and settings, because removing those is what removes the prompt on everything else. And every
# path `.claude/settings.json` refuses to read — `.env`, `secrets/`, `*.pem`, `*.key`,
# `credentials.json`, `.secret*` — because a `cp` or `mv` that is auto-allowed on both ends
# would rename one out of those globs and hand back through `Read` exactly what they deny.
path_class() {
local canon="$1" d root="" folded
# Matched against a lowercased copy: APFS is case-insensitive by default, so `.GIT` and `.git`
# are one directory, and a case-sensitive list would leave the history — and these guards' own
# settings — one keystroke from an auto-allowed delete. On a case-sensitive volume a genuinely
# distinct `.GIT/` over-matches, which costs a prompt and nothing else. `tr` and not `${x,,}`:
# macOS ships bash 3.2, which has no case-folding expansion.
folded=$(printf '%s' "$canon" | tr 'A-Z' 'a-z')
case "$folded" in
*"/.git" | *"/.git/"* | *"/.claude" | *"/.claude/"*) return 1 ;;
*"/.env" | *"/.env."*) return 1 ;;
*"/secrets" | *"/secrets/"*) return 1 ;;
*.pem | *.key | *"/credentials.json") return 1 ;;
*"/.secret"* | *.secret | *.secrets) return 1 ;;
esac
case "$canon" in "$TMP_ROOT"/?*) printf 'tmp'; return 0 ;; esac
[ -n "$HOME_ROOT" ] || return 1
# The Playwright MCP servers download browsers into `ms-playwright` and open a throwaway
# profile per session under `ms-playwright-mcp`; nothing prunes either, so they grow without
# bound (10G here) and clearing one costs a re-download and nothing else. They sit outside
# every checkout, where no other class reaches them. Matched including the root itself,
# unlike the repo class, because wiping the whole directory is the point.
# Each root is named exactly and then again with `/*`, rather than one trailing `*`: a case
# pattern's `*` spans the `-` as well, which would put a sibling somebody created themselves —
# `ms-playwright-mcp-backup` — in a class that auto-allows deleting it.
case "$canon" in
"$HOME_ROOT"/Library/Caches/ms-playwright | "$HOME_ROOT"/Library/Caches/ms-playwright/* \
| "$HOME_ROOT"/Library/Caches/ms-playwright-mcp | "$HOME_ROOT"/Library/Caches/ms-playwright-mcp/* \
| "$HOME_ROOT"/.cache/ms-playwright | "$HOME_ROOT"/.cache/ms-playwright/* \
| "$HOME_ROOT"/.cache/ms-playwright-mcp | "$HOME_ROOT"/.cache/ms-playwright-mcp/*)
printf 'mcp-cache'
return 0
;;
esac
case "$canon" in "$HOME_ROOT"/?*) ;; *) return 1 ;; esac
d="$canon"
while [ "$d" != "/" ] && [ "$d" != "$HOME_ROOT" ]; do
[ -e "$d/.git" ] && { root="$d"; break; }
d=$(dirname "$d")
done
[ -n "$root" ] || return 1 # not inside a git working tree under $HOME
if [ "$canon" = "$root" ]; then
[ -f "$root/.git" ] || return 1
fi
printf 'repo:%s' "$root"
}
# 0 iff <verb> ($1) runs as a command word anywhere in <command> ($2). Mirrors how a Bash
# permission rule matches, so that owning the prompt here doesn't narrow what used to prompt:
# a guard consults this before it starts proving segments, and every bail-out it then takes
# is a prompt for exactly the commands a rule would have caught.
runs_verb() {
local verb="$1" seg
split_segments "$2"
for seg in "${SEGMENTS[@]}"; do
segment_runs_verb "$verb" "$seg" && return 0
done
return 1
}
# Emit a PreToolUse decision and exit. `ask` is the ordinary permission prompt.
decide() {
jq -nc --arg d "$1" --arg r "$2" \
'{hookSpecificOutput:{hookEventName:"PreToolUse",permissionDecision:$d,permissionDecisionReason:$r}}'
exit 0
}
-25
View File
@@ -1,25 +0,0 @@
#!/bin/bash
# Notify user when Claude requires input (works on macOS and Linux)
# Check if we're in an SSH session
if [[ -n "$SSH_CLIENT" || -n "$SSH_TTY" || -n "$SSH_CONNECTION" ]]; then
# SSH session - use terminal bell
# If using VSCode, enable audible terminal bell for SSH sessions:
# Add the following to .vscode/settings.json:
# "accessibility.signals.terminalBell": {
# "sound": "on"
# },
# "terminal.integrated.enableVisualBell": true
printf '\a'
else
# Local session - use native notifications
if [[ "$OSTYPE" == "darwin"* ]]; then
osascript -e 'display notification "Claude is waiting for your input" with title "Claude Code" sound name "Glass"' 2>/dev/null || printf '\a'
elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
notify-send "Claude Code" "Claude is waiting for your input" 2>/dev/null || printf '\a'
else
printf '\a'
fi
fi
exit 0
-240
View File
@@ -1,240 +0,0 @@
#!/usr/bin/env bash
# Decision table for the two scratch-dir PreToolUse guards. Run: bash .claude/hooks/test-hooks.sh
#
# What this pins is the `ask` column: a matcher change that turns one into a no-decision drops
# that command's only prompt (see lib-guarded-verb.sh). The wrapper, nested-command and quoted
# rows are the ones that catch it.
#
# The `allow` column carries its own weight, because a decision covers the whole command line:
# `allow` may only appear where every segment was proved here, and a line that also runs
# something unexamined has to come out `none` so the normal permission flow still sees it.
set -uo pipefail
H="$(cd "${BASH_SOURCE[0]%/*}" && pwd)"
CWD="$(git -C "$H" rev-parse --show-toplevel)"
OUT="$HOME/not-a-git-tree" # never written to; only the guards' path checks look at it
fails=0
# A tree's own root is auto-allowable only when it is a LINKED worktree, whose `.git` is a
# pointer file so the history lives in the main repo and survives; a primary checkout's `.git`
# is the history itself. The suite runs from either kind, so the rows that name the root follow
# the one it is run in — which is also what pins both halves of that rule.
if [ -f "$CWD/.git" ]; then
ROOT_SOLO=allow ROOT_CHAINED=none # linked worktree
else
ROOT_SOLO=ask ROOT_CHAINED=ask # primary checkout
fi
run() { # run <hook> <allow|ask|none> <command>
local hook="$1" want="$2" cmd="$3" out got
out=$(jq -nc --arg c "$cmd" --arg w "$CWD" \
'{tool_name:"Bash",tool_input:{command:$c},cwd:$w}' | "$H/$hook" 2>&1)
if [ -z "$out" ]; then
got=none
else
got=$(printf '%s' "$out" | jq -r '.hookSpecificOutput.permissionDecision // "PARSE-ERROR"' 2>/dev/null || echo PARSE-ERROR)
fi
local shown="${cmd//$'\n'/ ⏎ }"
if [ "$got" = "$want" ]; then
printf ' ok %-5s %s\n' "$got" "$shown"
else
printf 'FAIL want=%-5s got=%-5s %s\n %s\n' "$want" "$got" "$shown" "$out"
fails=$((fails + 1))
fi
}
echo "== guard-rm-outside-tmp.sh =="
G=guard-rm-outside-tmp.sh
run $G allow "rm -rf /tmp/scratch/x"
run $G allow "rm -rf /tmp/scratch/*"
run $G allow "rm -rf $CWD/frontend/scratch"
run $G ask "rm -rf /tmp"
run $G ask "rm -rf $OUT"
run $G ask "rm -rf $CWD/.git"
run $G ask "rm -rf $CWD/.claude/hooks" # the guards may not delete themselves
run $G ask "rm $CWD/.claude/settings.json"
run $G ask "rm $CWD/.claude/settings.local.json"
run $G ask "rm -rf $CWD/backend/.env"
run $G ask "rm -rf $CWD/.env.local"
run $G $ROOT_SOLO "rm -rf $CWD"
run $G ask "rm -rf $CWD/*"
run $G ask "rm -rf /etc/passwd"
# The MCP caches are the one allowed root outside /tmp and the checkouts, and `~/` and `$HOME/`
# the one expansion the charset check tolerates — so the row that matters is the one proving the
# prefix does not carry anything else along with it.
run $G allow "rm -rf ~/Library/Caches/ms-playwright-mcp"
run $G allow "rm -rf ~/.cache/ms-playwright-mcp" # the Linux spelling of the same root
run $G allow 'rm -rf $HOME/Library/Caches/ms-playwright-mcp/mcp-chrome-*'
run $G ask "rm -rf ~/.cache/ms-playwright-mcp-backup" # a sibling, not the cache
run $G ask "rm -rf ~/not-a-git-tree"
# The exclusion list is the whole protection for these paths — the `repo:` class allows deletes
# everywhere else in a checkout — and macOS resolves `.GIT` to `.git`, so the fold is what keeps
# the list from failing open there. Pattern-matched, so the row holds on either platform.
run $G ask "rm -rf $CWD/.GIT"
run $G ask "rm $CWD/.CLAUDE/settings.json"
run $G ask "rm -rf $CWD/backend/.ENV"
run $G ask 'rm -rf "$HOME/x"'
run $G ask "rm -rf /tmp/../$OUT"
run $G none "ls /tmp && rm -rf /tmp/x" # proved delete, unexamined neighbour
run $G ask 'echo $(rm -rf /etc)'
run $G ask 'echo `rm -rf /etc`'
run $G ask "{ rm -rf /etc; }"
run $G allow "{ rm -rf /tmp/scratch/x; }" # the keyword drops, the delete still proves
run $G ask "find . -name x | xargs rm"
run $G ask "timeout 5 rm -rf /tmp/x"
run $G ask "stdbuf -o L rm -rf /etc"
run $G ask "FOO=bar rm -rf /tmp/x"
run $G ask "/bin/rm -rf /tmp/x"
run $G ask "'rm' -rf /etc"
run $G ask 'r\m -rf /etc'
run $G ask "! rm -rf /etc"
run $G ask "if true; then rm -rf /etc; fi"
run $G ask ">/dev/null rm -rf $OUT"
# Data that merely mentions a verb is not a command. Both of these prompted in the field.
run $G none "$(printf 'gh pr create --body "$(cat <<%sEOF%s\ndrop `rm` and `mv` from the ask list\nrm is now guarded here\nEOF\n)"' "'" "'")"
run $G none "$(printf 'claude -p "run these in order:\n1: rm -rf /tmp/a\n2: mv /tmp/b /tmp/c"')"
# A wrapper's own flags and assignments are unbounded, so they may not be charged against the
# scan that looks past it — these run rm and must prompt.
run $G ask "env -i HOME=/tmp PATH=/usr/bin LANG=C USER=root SHELL=/bin/sh rm -rf /etc"
run $G ask "sudo -E -H -u root FOO=1 BAR=2 rm -rf $OUT"
run $G ask "xargs -a f -d d -E e -I {} -L 1 -n 1 rm /etc"
run $G ask "env -u A -u B -u C -u D -u E -u F -u G rm -rf /etc"
run $G ask "sudo -u 'root' rm -rf /etc"
run $G ask "$(printf 'echo hi # cat <<EOF\nrm -rf /etc\nEOF')"
# A `<<` inside a quoted string or a comment opens no heredoc, so the command under it is real.
run $G ask "$(printf 'echo "cat <<EOF"\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'echo "cat <<EOF and more"\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'echo "cat <<EOF "\nrm -rf /etc\nEOF')"
run $G ask "$(printf '# usage: cat <<EOF\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'echo "cat <<EOF > f"\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'echo "cat <<true > /tmp/a"\nrm -rf /etc\ntrue')"
run $G ask "$(printf "echo 'cat <<EOF | tee'\nrm -rf /etc\nEOF")"
# A body fed to a shell is executed, so it is commands and not data.
run $G ask "$(printf 'bash <<EOF\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'cat <<EOF | bash\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'ssh host <<EOF\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'bash<<%sEOF%s\nrm -rf /etc\nEOF' "'" "'")"
run $G ask "$(printf '/bin/sh <<EOF\nrm -rf /etc\nEOF')"
run $G ask "$(printf 'cat <<%sEOF%s|bash\nrm -rf /etc\nEOF' "'" "'")"
run $G ask "$(printf 'out=$(bash <<%sEOF%s\nrm -rf /etc\nEOF\n)' "'" "'")"
run $G ask "$(printf 'bash \\\n <<%sEOF%s\nrm -rf /etc\nEOF' "'" "'")"
run $G ask "$(printf 'ash <<%sEOF%s\nrm -rf /etc\nEOF' "'" "'")"
run $G ask "$(printf 'busybox sh <<%sEOF%s\nrm -rf /etc\nEOF' "'" "'")"
run $G ask "$(printf 'sudo -s <<%sEOF%s\nrm -rf /etc\nEOF' "'" "'")"
run $G ask "$(printf '(bash <<%sEOF%s)\nrm -rf /etc\nEOF' "'" "'")"
# A redirect or pipe after the delimiter is still a real heredoc.
run $G none "$(printf 'cat <<%sEOF%s > /tmp/a\nrm -rf /etc\nEOF' "'" "'")"
run $G none "$(printf 'cat <<%sEOF%s 2>&1 | tee /tmp/a\nrm -rf /etc\nEOF' "'" "'")"
# An unquoted body is expanded before its consumer sees it, so it is code.
run $G ask "$(printf 'cat <<EOF > /tmp/a\n$(rm -rf /etc)\nEOF')"
run $G ask "$(printf 'cat <<EOF > /tmp/a\nrm -rf /etc\nEOF')"
# ... but a real command after a heredoc still is one.
run $G ask "$(printf 'cat <<EOF > /tmp/s.sh\nhello\nEOF\nrm -rf %s' "$OUT")"
run $G ask "$(printf 'echo "a << b"\nrm -rf %s' "$OUT")"
run $G none "git rm frontend/foo.ts"
run $G none 'echo $(ls /tmp)'
run $G none 'grep -rn "rm" backend/'
run $G none "cargo build --release"
# Chaining and line breaks are not themselves a reason to prompt: each segment is proved on its
# own operands, and a `cd` moves where a relative one points.
run $G allow "rm -f /tmp/a; rm -rf /tmp/b"
run $G allow "$(printf 'rm -f /tmp/a\nrm -rf %s/frontend/scratch' "$CWD")"
run $G allow "cd /tmp/scratch && rm -rf sub"
run $G none "mkdir -p /tmp/x && rm -rf /tmp/x"
run $G ask "$(printf 'ls /tmp\nrm -rf /etc')"
# A `cd` this guard can resolve is where the relative operand lands; one it cannot leaves the
# working directory unknown, and an unknown one proves nothing.
run $G ask "cd /etc && rm -rf foo"
run $G ask 'cd "$D" && rm -rf foo'
run $G ask "cd $CWD && rm -rf .git"
run $G ask "cd /etc && cd /tmp/scratch && rm -rf sub" # a cd out is not walked back
# A `cd` can fail at runtime, and `;` runs the delete from where the command started, so a
# relative operand is proved from both directories.
run $G ask "cd /tmp/does-not-exist; rm -rf .git"
run $G ask "cd /tmp/does-not-exist; rm -rf backend/.env"
run $G ask "cd /tmp/a && cd /tmp/b && rm -rf sub"
run $G ask "rm -rf /tmp/clone/.git" # history is never in a class
run $G ask "rm -rf /tmp/scratch/id_rsa.key"
run $G none "cd /tmp >$OUT; rm -f /tmp/a"
# A substitution is concatenated into its word, so splitting on it would prove only the literal
# half; a relative `cd` is not $cwd/$t either, since the shell searches $CDPATH first.
run $G ask 'rm -rf /tmp/a/`printf ../../etc`'
run $G ask 'rm -rf /tmp/a/$(printf ../../etc)'
run $G ask "cd ssh && rm -rf moduli"
echo
echo "== allow-fileops-in-tmp.sh =="
A=allow-fileops-in-tmp.sh
run $A allow "mv /tmp/a /tmp/b"
run $A allow "chmod 755 /tmp/a"
run $A allow "cp -r /tmp/a /tmp/b"
run $A allow "tar -xzf /tmp/a.tar.gz -C /tmp/out"
run $A ask "mv /tmp/a $OUT"
run $A ask "mv $CWD/AGENTS.md /tmp/a"
run $A $ROOT_SOLO "chmod -R 777 $CWD"
run $A none "ls && mv /tmp/a /tmp/b" # proved move, unexamined neighbour
run $A ask 'echo $(mv /tmp/a /etc)'
run $A ask "timeout --signal KILL 5 mv /tmp/a /etc"
run $A ask "time -f FORMAT chmod 777 $OUT"
run $A ask "'mv' /tmp/a /etc"
run $A ask 'ch\mod 777 /etc'
run $A none "$(printf 'claude -p "run these in order:\n1: rm -rf /tmp/a\n2: mv /tmp/b /tmp/c"')"
run $A ask "env -i A=1 B=2 C=3 D=4 E=5 F=6 mv /tmp/a /etc"
run $A none "cp $CWD/AGENTS.md /tmp/a"
run $A none "tar -xzf /tmp/a.tar.gz -C $OUT"
run $A none "cargo build"
run $A ask "chmod -R 777 $CWD/.GIT"
run $A allow "chmod -R 755 ~/Library/Caches/ms-playwright-mcp"
run $A ask "chmod -R 777 ~/Library/Caches/ms-playwright-mcp-backup"
# The home prefix reaches this guard through `operand_class`, not the rm guard's own resolver.
case "$CWD" in
"$HOME"/*) run $A allow "mv ~${CWD#"$HOME"}/frontend/a.ts ~${CWD#"$HOME"}/frontend/b.ts" ;;
esac
run $A none "mkdir -p /tmp/x; mv /tmp/a /tmp/x; chmod 755 /tmp/x" # one write per line
run $A none "$(printf 'mv /tmp/a /tmp/b\nchmod 755 /tmp/b')"
run $A ask "ls && mv /tmp/a /etc"
run $A $ROOT_CHAINED "$(printf 'mkdir -p /tmp/x\nchmod -R 777 %s' "$CWD")"
run $A allow "cd /tmp/x && tar -xzf /tmp/a.tar.gz -C /tmp/out"
# The checkout is a root of its own, so an in-repo move or chmod is as auto-allowable as the
# in-repo delete already was — but one operation may not straddle it and /tmp.
run $A allow "chmod +x scripts/worktree-env"
run $A allow "mv backend/.sqlx backend/.sqlx.bad"
run $A allow "mv $CWD/frontend/a.ts $CWD/frontend/b.ts"
run $A ask "mv /tmp/a $CWD/frontend/a.ts"
run $A ask "chmod -R 777 $CWD/.git"
run $A ask "mv $CWD/backend/.env $CWD/backend/.env.bak"
run $A ask "mv $CWD/AGENTS.md $OUT"
run $A ask "cd /etc && mv a b"
# An auto-allowed rename may not carry a path out of the `Read` deny globs.
run $A ask "mv backend/server.pem backend/server.txt"
run $A none "cp backend/secrets/token frontend/token.txt" # cp has no prompt of its own,
# so what matters is it is not allowed
run $A ask "mv $CWD/backend/credentials.json /tmp/x"
run $A ask "cd /tmp/does-not-exist; mv .claude/settings.json settings.bak"
# A segment this hook cannot read whole may carry a redirect, and an earlier write can change
# what a later operand resolves to — neither may ride along on an allow.
run $A none "cd /tmp >$OUT; mv /tmp/a /tmp/b"
run $A none "cp -r /tmp/tree /tmp/live; cp /tmp/payload /tmp/live/link"
run $A ask 'mv /tmp/a/`printf ../../etc/x` /tmp/b'
# A sibling checkout is a different root: its files are outside what the Read tool is confined
# to, and copying them in would hand back what that confinement withholds.
EE="$(dirname "$CWD")/windmill-ee-private" # a sibling checkout; absent elsewhere, still not a root
run $A ask "mv $EE/backend/x.rs $CWD/backend/x.rs"
run $A none "cp $EE/README.md $CWD/README.copy"
# Directory form writes a path the command does not name — DEST/basename(SRC) — and `cp`
# follows that child when it is a symlink, as every `*_ee.rs` in this checkout is.
run $A ask "mv frontend/apps_ee.rs backend/windmill-api/src"
run $A none "cp frontend/apps_ee.rs backend/windmill-api/src"
run $A none "cp frontend/a.ts backend"
run $A ask "mv /tmp/a $CWD/backend"
# ... and a `cd` that fails at runtime may not hide that form: the destination is a directory
# in the directory the command actually ran in, whichever of the two that turns out to be.
run $A none "cd $CWD/AGENTS.md; cp frontend/apps_ee.rs backend/windmill-api/src"
run $A ask "cd $CWD/AGENTS.md; mv frontend/apps_ee.rs backend/windmill-api/src"
run $A none "cd /tmp/x && tar -xzf /tmp/a.tar.gz" # no -C, and the cwd is now two candidates
run $A allow "cp frontend/a.ts backend/a.ts" # ... naming the destination proves fine
echo
[ "$fails" = 0 ] && echo "ALL PASS" || { echo "$fails FAILURES"; exit 1; }
-4
View File
@@ -1,4 +0,0 @@
# Claude output format
- Use inline comments at the relevant lines for specific issues.
- Use a top-level comment for the summary, severity-tagged finding list, AGENTS.md compliance check, and the test-coverage assessment.
-150
View File
@@ -1,150 +0,0 @@
{
"permissions": {
"additionalDirectories": [
"../windmill-ee-private"
],
"allow": [
"Bash(ls:*)",
"Bash(grep:*)",
"Bash(cat:*)",
"Bash(head:*)",
"Bash(tail:*)",
"Bash(less:*)",
"Bash(more:*)",
"Bash(find:*)",
"Bash(wc:*)",
"Bash(diff:*)",
"Bash(file:*)",
"Bash(stat:*)",
"Bash(tree:*)",
"Bash(pwd)",
"Bash(which:*)",
"Bash(whereis:*)",
"Bash(echo:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git branch:*)",
"Bash(git show:*)",
"Bash(git blame:*)",
"Bash(cargo check:*)",
"Bash(cargo build --release:*)",
"Bash(sh wm-ts-nav/nav:*)",
"Bash(wm-ts-nav/nav:*)",
"Bash(./wm-ts-nav/nav:*)",
"Bash(wm-ts-nav/target/release/wm-ts-nav:*)",
"Bash(./wm-ts-nav/target/release/wm-ts-nav:*)",
"mcp__ide__getDiagnostics",
"Bash(npm run generate-backend-client:*)",
"Bash(npm run check:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)",
"Bash(git add:*)",
"Bash(git commit:*)",
"Read(/tmp/**)",
"Write(/tmp/**)",
"Edit(/tmp/**)",
"mcp__claude_ai_Gmail__search_threads",
"mcp__claude_ai_Gmail__get_thread",
"mcp__claude_ai_Gmail__get_message",
"mcp__claude_ai_Gmail__list_labels",
"mcp__claude_ai_Gmail__list_drafts"
],
"deny": [
"Edit(.env)",
"Edit(.env.*)",
"Edit(**/.env)",
"Edit(**/.env.*)",
"Edit(**/secrets/**)",
"Edit(**/*.pem)",
"Edit(**/*.key)",
"Edit(**/credentials.json)",
"Edit(**/.secret*)",
"Edit(**/.secrets*)",
"Edit(**/*.secret)",
"Edit(**/*.secrets)"
],
"ask": [
"Bash(rmdir:*)",
"Bash(chown:*)",
"Bash(truncate:*)",
"Bash(shred:*)",
"Bash(unlink:*)",
"mcp__claude_ai_Stripe",
"mcp__claude_ai_Gmail__create_draft",
"mcp__claude_ai_Gmail__update_draft",
"mcp__claude_ai_Gmail__create_label",
"mcp__claude_ai_Gmail__label_message",
"mcp__claude_ai_Gmail__label_thread",
"mcp__claude_ai_Gmail__unlabel_message",
"mcp__claude_ai_Gmail__unlabel_thread",
"mcp__claude_ai_Gmail__apply_sensitive_message_label",
"mcp__claude_ai_Gmail__apply_sensitive_thread_label",
"mcp__claude_ai_Google_Calendar",
"mcp__claude_ai_Google_Drive",
"mcp__claude_ai_Slack",
"mcp__claude_ai_Linear"
]
},
"enableAllProjectMcpServers": true,
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/guard-main-branch.sh",
"timeout": 5
},
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/guard-rm-outside-tmp.sh",
"timeout": 5
},
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/allow-fileops-in-tmp.sh",
"timeout": 5
}
]
}
],
"PostToolUse": [
{
"matcher": "Edit|Write",
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-frontend.sh",
"timeout": 30
},
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-backend.sh",
"timeout": 30
}
]
}
],
"Notification": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/notify-user.sh",
"timeout": 10
}
]
}
]
},
"enabledPlugins": {
"typescript-lsp@claude-plugins-official": true,
"code-review@claude-plugins-official": true
}
}
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/adding-a-trigger/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/ai-chat/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/ai-evals/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/codebase-design/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/commit/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/domain-modeling/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/grill-me/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/grilling/SKILL.md
@@ -1 +0,0 @@
../../../.agents/skills/improve-codebase-architecture/SKILL.md
@@ -1 +0,0 @@
../../../.agents/skills/local-review-codex/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/local-review/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/native-trigger/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/pr/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/refine/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/rust-backend/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/svelte-frontend/SKILL.md
-1
View File
@@ -1 +0,0 @@
../../../.agents/skills/update-sqlx/SKILL.md
-4
View File
@@ -1,4 +0,0 @@
#:schema https://developers.openai.com/codex/config-schema.json
[mcp_servers.svelte]
url = "https://mcp.svelte.dev/mcp"
-1
View File
@@ -3,4 +3,3 @@ frontend/build/
frontend/.svelte-kit/
backend/target/
backend/windmill-duckdb-ffi-internal/target/
-6
View File
@@ -1,7 +1 @@
use flake
# Per-worktree overrides (ports, DATABASE_URL, etc.) written by webmux/workmux
# post-create hooks. Must come after `use flake` so they take precedence over
# the flake's defaults.
# shellcheck source=/dev/null
[ -f .env.local ] && source .env.local
-3
View File
@@ -1,3 +0,0 @@
# Files a generator owns. Collapsed in review diffs and left out of language
# stats: reviewing them means reviewing the generator instead.
*.gen.ts linguist-generated=true
+3 -3
View File
@@ -1,4 +1,4 @@
* @rubenfiszel @hugocasa @alpetric
* @rubenfiszel @HugoCasa @alpetric
/community/ @rubenfiszel @hugocasa @alpetric
/frontend/ @rubenfiszel @hugocasa @alpetric
/community/ @rubenfiszel @HugoCasa @alpetric
/frontend/ @rubenfiszel @HugoCasa @alpetric
+4 -12
View File
@@ -28,7 +28,7 @@ ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.11.24/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.4.18/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
ENV TZ=Etc/UTC
@@ -42,11 +42,7 @@ RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VER
RUN /usr/local/bin/python3 -m pip install pip-tools
# Bun
COPY --from=oven/bun:1.4.0 /usr/local/bin/bun /usr/bin/bun
# Install windmill CLI
RUN bun install -g windmill-cli \
&& ln -s $(bun pm bin -g)/wmill /usr/bin/wmill
COPY --from=oven/bun:1.2.4 /usr/local/bin/bun /usr/bin/bun
ARG TARGETPLATFORM
@@ -61,12 +57,8 @@ RUN apt-get update \
RUN rustup component add rustfmt
# C#
RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh \
&& chmod +x dotnet-install.sh \
&& ./dotnet-install.sh --channel 9.0 --install-dir /usr/share/dotnet \
&& ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet \
&& rm dotnet-install.sh
COPY --from=bitnami/dotnet-sdk:9.0.101-debian-12-r0 /opt/bitnami/dotnet-sdk /opt/dotnet-sdk
RUN ln -s /opt/dotnet-sdk/bin/dotnet /usr/bin/dotnet
# Nushell
COPY --from=ghcr.io/nushell/nushell:0.101.0-bookworm /usr/bin/nu /usr/bin/nu
-14
View File
@@ -1,14 +0,0 @@
<!--
We are not seeking outside contribution at this time. Small, trivially-verified PRs that fix a
problem are still welcome; low-value PRs (e.g. typo fixes) and PRs longer than a dozen or so lines
will be closed with a reference to CONTRIBUTING.md.
For a bigger idea, please open a feature request instead:
https://github.com/windmill-labs/windmill/issues/new?template=feature_request.md
Read https://github.com/windmill-labs/windmill/blob/main/CONTRIBUTING.md before submitting.
-->
## What does this PR do?
## Related issue
@@ -1,56 +0,0 @@
name: Sign image and attach provenance
description: >
Keyless-signs a pushed image digest with cosign (index and per-arch
manifests) and records SLSA provenance as a GitHub artifact attestation
pushed to the registry. SBOMs are not generated here: the build step embeds
them as BuildKit attestation manifests (depot `sbom: true`), which the index
signature then covers. The calling job must already be logged in to the
registry and must have id-token: write, attestations: write and
packages: write permissions (write-all covers all three).
inputs:
image:
description: "Fully-qualified image name without tag, e.g. ghcr.io/windmill-labs/windmill"
required: true
digest:
description: "Pushed manifest digest (sha256:...) from build-push-action"
required: true
runs:
using: composite
steps:
- name: Preflight
shell: bash
env:
DIGEST: ${{ inputs.digest }}
run: |
if [ -z "${ACTIONS_ID_TOKEN_REQUEST_URL:-}" ]; then
echo "::error::No OIDC token available; the calling job needs id-token: write"
exit 1
fi
case "$DIGEST" in
sha256:*) ;;
*)
echo "::error::digest '$DIGEST' is not a sha256: digest"
exit 1
;;
esac
# cosign v2 writes the classic sha256-<digest>.sig tag format that the
# installed base of cosign clients can verify; v3's bundle format cannot be
# verified by v2 clients yet, so stay on v2 until v3 verification is common.
- uses: sigstore/cosign-installer@v4.1.2
with:
cosign-release: "v2.6.5"
- name: Cosign keyless sign (index + per-arch manifests)
shell: bash
env:
IMAGE: ${{ inputs.image }}
DIGEST: ${{ inputs.digest }}
run: cosign sign --yes --recursive "${IMAGE}@${DIGEST}"
- name: SLSA provenance (GitHub artifact attestation)
uses: actions/attest-build-provenance@v4
with:
subject-name: ${{ inputs.image }}
subject-digest: ${{ inputs.digest }}
push-to-registry: true
+4 -7
View File
@@ -7,7 +7,7 @@ VERSION=$1
echo "Updating versions to: $VERSION"
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/src/core/constants.ts
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/main.ts
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
@@ -15,15 +15,12 @@ sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescrip
sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i '' -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -E "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
# windmill-parser-wasm is its own workspace (excluded from the backend workspace
# because of nightly-only cargo-features), so its version lives in
# [workspace.package] and its Cargo.lock is not regenerated by the backend step.
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.toml
sed -i '' -E "s/(name = \"windmill[^\"]*\"\nversion = )\"[^\"]*\"/\\1\"$VERSION\"/g" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.lock
cd ${root_dirpath}/frontend && npm i --package-lock-only
+4 -12
View File
@@ -7,29 +7,21 @@ VERSION=$1
echo "Updating versions to: $VERSION"
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/src/core/constants.ts
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/main.ts
sed -i -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-client/package.json
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-client/jsr.json
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/windmill-yaml-validator/package.json
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -zE "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
# windmill-parser-wasm is its own workspace (excluded from the backend workspace
# because of nightly-only cargo-features), so its version lives in
# [workspace.package] and its Cargo.lock is not regenerated by the backend step.
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.toml
sed -i -zE "s/(name = \"windmill[^\"]*\"\nversion = )\"[^\"]*\"/\\1\"$VERSION\"/g" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.lock
cd ${root_dirpath}/frontend && npm i --package-lock-only --ignore-scripts
# The CLI installs this package on every `bun install`, which would otherwise rewrite the
# lockfile's version and leave a dirty tree.
cd ${root_dirpath}/windmill-yaml-validator && npm i --package-lock-only --ignore-scripts
-5
View File
@@ -1,5 +0,0 @@
# Codex output format
- Read the review context file whose absolute path is given at the end of these instructions; it holds the PR metadata and the diff commands.
- Return a markdown PR comment starting with `## Codex Review`.
- Tag each finding with a severity (P0 / P1 / P2), file path, and line number when known confidently.
+6
View File
@@ -31,3 +31,9 @@ updates:
directory: "/python-client/wmill"
schedule:
interval: "weekly"
# Maintain dependencies for wmill_pg python client
- package-ecosystem: "pip"
directory: "/python-client/wmill_pg"
schedule:
interval: "weekly"
-17
View File
@@ -1,17 +0,0 @@
# Pi output format
- Read the review context file whose absolute path is given at the end of these instructions; it holds the PR metadata and the diff (or the git commands to produce it).
- Return a markdown PR comment starting with `## Pi Review`.
- Tag each finding with a severity (P0 / P1 / P2), file path, and line number when known confidently.
- Output ONLY the final review markdown — no preamble, no thinking, no tool transcripts.
# Before you settle on a verdict
`REVIEW.md` tells you to discard findings you are not confident in. That rule exists to suppress noise, not to license a quick approval. Review in two passes:
1. Enumerate every candidate defect you notice, without judging any of them yet.
2. Take each candidate and try to prove it is real: read the surrounding code, check the caller, check the error path. Keep it, or dismiss it for a specific reason.
A "Good to merge" verdict must be accompanied by a "Considered and dismissed" section listing each candidate from pass 1 with the concrete reason it is not a finding. If that section would be empty, pass 1 was skipped: go back and do it.
Facts cut both ways. If you notice that a cached value can be multiple megabytes, that a lock is held across an await, or that a new parameter is caller-controlled, that observation is a candidate for pass 2 even when the surrounding code looks deliberate. Do not narrate such a fact as evidence that the code is fine without first checking whether it is a bug.
-160
View File
@@ -1,160 +0,0 @@
// Extracts every windmill.dev/docs link referenced in the frontend source and
// verifies none of them 404. Run: `node .github/scripts/check-docs-links.mjs`.
// Used by the check-docs-links GitHub workflow (release / manual trigger only).
import { readdir, readFile } from 'node:fs/promises'
import { join, extname } from 'node:path'
const ROOT = 'frontend/src'
const EXTS = new Set(['.ts', '.js', '.svelte', '.mjs', '.cjs'])
const DOCS_RE = /https?:\/\/(?:www\.)?windmill\.dev\/docs\/[^\s"'`)>\]}]*/g
// `const someBaseUrl = 'https://www.windmill.dev/docs/...'` used later as `${someBaseUrl}/foo`
const BASE_RE = /(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*=\s*['"`](https?:\/\/(?:www\.)?windmill\.dev\/docs\/[^'"`]+)['"`]/g
const CONCURRENCY = 24
const TIMEOUT_MS = 20000
const RETRIES = 2
// Links whose target page is written but not yet deployed on windmill.dev: the app
// link is already the final slug, so a 404 is expected until the docs side ships.
// The value is why the entry exists, for whoever has to judge whether it still should.
const PENDING_DEPLOY = new Map()
async function walk(dir) {
const out = []
for (const entry of await readdir(dir, { withFileTypes: true })) {
const p = join(dir, entry.name)
if (entry.isDirectory()) {
if (entry.name === 'node_modules' || entry.name === '.svelte-kit') continue
out.push(...(await walk(p)))
} else if (EXTS.has(extname(entry.name))) {
out.push(p)
}
}
return out
}
// url (no fragment) -> Set of source files it appears in
const urls = new Map()
const unresolved = []
function record(url, file) {
const clean = url
.replace(/\\.*$/, '') // cut at an escape sequence embedded in a string literal (e.g. \n)
.replace(/#.*$/, '') // drop anchor fragment — irrelevant to a 404 check
.replace(/[.,;:'")\]]+$/, '')
if (!clean) return
// A `{`/`${` means the URL is built from an unresolved template/interpolation var.
if (clean.includes('{')) {
unresolved.push(`${clean} (${file})`)
return
}
if (!urls.has(clean)) urls.set(clean, new Set())
urls.get(clean).add(file)
}
for (const file of await walk(ROOT)) {
let content = await readFile(file, 'utf8')
// Inline file-local base-url constants so `${base}/page` template literals resolve.
const bases = []
for (const m of content.matchAll(BASE_RE)) bases.push({ name: m[1], value: m[2], decl: m[0] })
for (const { name, value } of bases) {
content = content.replaceAll('${' + name + '}', value)
}
// Blank each base declaration so a prefix-only base (no index page of its own,
// e.g. .../app_configuration_settings) isn't checked as a standalone link.
// A genuinely bare `${base}` usage was already inlined above, so it's still covered.
for (const { decl } of bases) content = content.replace(decl, '')
for (const m of content.matchAll(DOCS_RE)) record(m[0], file)
}
const allUrls = [...urls.keys()].sort()
console.log(`Found ${allUrls.length} distinct docs links across ${ROOT}`)
if (unresolved.length) {
console.log(`\n⚠️ ${unresolved.length} link(s) built from an unrecognized base URL — skipped (register the base const so they can be checked):`)
for (const u of [...new Set(unresolved)].sort()) console.log(` ${u}`)
}
async function check(url) {
for (let attempt = 0; attempt <= RETRIES; attempt++) {
const ctrl = new AbortController()
const timer = setTimeout(() => ctrl.abort(), TIMEOUT_MS)
try {
let res = await fetch(url, {
method: 'HEAD',
redirect: 'follow',
signal: ctrl.signal,
headers: { 'user-agent': 'windmill-docs-link-check' }
})
// Some hosts reject HEAD — fall back to GET.
if (res.status === 405 || res.status === 501) {
res = await fetch(url, {
method: 'GET',
redirect: 'follow',
signal: ctrl.signal,
headers: { 'user-agent': 'windmill-docs-link-check' }
})
}
clearTimeout(timer)
return { url, status: res.status, ok: res.status < 400 }
} catch (err) {
clearTimeout(timer)
if (attempt === RETRIES) return { url, status: 0, ok: false, error: String(err?.message || err) }
await new Promise((r) => setTimeout(r, 500 * (attempt + 1)))
}
}
}
// Simple concurrency pool.
const results = []
let idx = 0
async function worker() {
while (idx < allUrls.length) {
const url = allUrls[idx++]
results.push(await check(url))
}
}
await Promise.all(Array.from({ length: CONCURRENCY }, worker))
// An entry claims one thing — the page is not published yet — and 404 is the only
// answer that means it. A timeout, 403 or 5xx on the same URL is a real fault, and
// suppressing it would also read as "still waiting" and defer the staleness check.
const isPendingDeploy = (r) => PENDING_DEPLOY.has(r.url) && r.status === 404
const pending = results.filter((r) => PENDING_DEPLOY.has(r.url))
const waiting = results.filter(isPendingDeploy)
if (waiting.length) {
console.log(`\n${waiting.length} link(s) waiting on a docs deploy:`)
for (const p of waiting.sort((a, b) => a.url.localeCompare(b.url))) {
console.log(` ${p.url}\n ${PENDING_DEPLOY.get(p.url)} — not live yet (${p.status})`)
}
}
// An entry that outlived its reason exempts a URL from the check forever, so a stale
// one has to fail the job: a line in a green log is not read at release time.
const stale = [
...pending.filter((p) => p.ok).map((p) => [p.url, 'the page is live']),
...[...PENDING_DEPLOY.keys()].filter((u) => !urls.has(u)).map((u) => [u, 'nothing references it'])
]
const failures = results.filter((r) => !r.ok && !isPendingDeploy(r))
if (failures.length === 0 && stale.length === 0) {
console.log(`\n✅ No broken docs links (${allUrls.length} checked).`)
process.exit(0)
}
if (failures.length) {
console.log(`\n${failures.length} broken docs link(s):`)
for (const f of failures.sort((a, b) => a.url.localeCompare(b.url))) {
console.log(`\n ${f.url}`)
console.log(` status: ${f.error ? `error (${f.error})` : f.status}`)
for (const file of urls.get(f.url)) console.log(`${file}`)
}
}
if (stale.length) {
console.log(`\n${stale.length} PENDING_DEPLOY entr(ies) to delete from this script:`)
for (const [url, why] of stale.sort((a, b) => a[0].localeCompare(b[0]))) {
console.log(`\n ${url}\n ${why}`)
}
}
process.exit(1)
-132
View File
@@ -1,132 +0,0 @@
name: AI Agent Integration Tests
# Exercises the AI agent flow path (preview_flow with `aiagent` modules) against
# real LLM providers. Runs only when AI-agent backend code or the tests change,
# because each run makes real (paid) LLM calls. To avoid spending on every commit,
# the PR side triggers only when a PR is marked ready for review (out of draft) —
# not on `synchronize` — plus push to main and manual dispatch.
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "integration_tests/ai_agent_tests/**"
- "backend/windmill-ai/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-worker/src/ai_executor.rs"
- "backend/windmill-worker/src/ai/**"
- "backend/windmill-worker/src/memory_common.rs"
- "backend/windmill-common/src/flow_conversations.rs"
- ".github/workflows/ai-agent-tests.yml"
pull_request:
types: [opened, reopened, ready_for_review]
paths:
- "integration_tests/ai_agent_tests/**"
- "backend/windmill-ai/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-worker/src/ai_executor.rs"
- "backend/windmill-worker/src/ai/**"
- "backend/windmill-worker/src/memory_common.rs"
- "backend/windmill-common/src/flow_conversations.rs"
- ".github/workflows/ai-agent-tests.yml"
concurrency:
group: ai-agent-tests-${{ github.ref }}
cancel-in-progress: true
jobs:
ai_agent_e2e:
# Skip draft PRs; the `opened`/`reopened` types would otherwise fire while
# still a draft. `ready_for_review` always arrives non-draft.
if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
runs-on: ubicloud-standard-16
services:
postgres:
image: postgres:16
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.0
- uses: actions/setup-node@v4
with:
node-version: "20"
- uses: actions/setup-python@v5
with:
python-version: "3.11"
# CE build (no enterprise/license needed for AI agents). `quickjs` powers
# flow input-transform JS eval; `mcp` is required by the deepwiki MCP tool
# test. Bun tool scripts run via the always-on worker (BUN_PATH).
- name: Build Windmill
working-directory: ./backend
env:
SQLX_OFFLINE: true
CARGO_BUILD_JOBS: 12
RUSTFLAGS: ""
run: cargo build --features quickjs,mcp
- name: Start Windmill
working-directory: ./backend
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
BUN_PATH: bun
NODE_BIN_PATH: node
RUST_LOG: info
run: |
mkdir -p ../integration_tests/logs
./target/debug/windmill > ../integration_tests/logs/windmill.log 2>&1 &
echo "Waiting for Windmill to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
echo "Windmill is ready"
break
fi
sleep 2
done
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; tail -50 ../integration_tests/logs/windmill.log; exit 1; }
- name: Run AI agent integration tests
timeout-minutes: 20
working-directory: ./integration_tests/ai_agent_tests
env:
WINDMILL_URL: http://localhost:8000
# Only the providers we have org secrets for. Other providers
# (Azure, Bedrock, OpenRouter) are skipped by conftest when their
# keys are absent — see skip_provider_without_credentials.
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GOOGLE_AI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
run: |
python -m venv .venv
.venv/bin/pip install -r requirements.txt
# The S3/vision-attachment tests need MinIO large-file storage and
# image-capable provider setup; out of scope for this cost-controlled
# smoke. Add MinIO secrets + a storage service to enable them.
.venv/bin/python -m pytest -v \
--ignore=test_user_attachments.py \
--ignore=test_user_images.py \
--ignore=test_image_output.py
- name: Archive Windmill logs
uses: actions/upload-artifact@v4
if: always()
with:
name: ai-agent-tests-windmill-logs
path: integration_tests/logs
-181
View File
@@ -1,181 +0,0 @@
name: AI Evals (global mode)
# Smoke-tests the production global AI chat proxy/frontend execution path via
# the ai_evals harness, one case across one cheap model per provider. Runs only
# when the eval harness or the global chat code change, since each run makes real
# (paid) LLM calls. The backend is built from source purely as the AI proxy the
# harness routes model calls through; the global tools/drafts run in-process in
# the Vitest bridge against production frontend code. To avoid spending on every
# commit, the PR side triggers only when a PR is marked ready for review (out of
# draft) — not on `synchronize` — plus push to main and manual dispatch.
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "ai_evals/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-ai/**"
- "frontend/src/lib/components/copilot/**"
# The eval harness runs production frontend code in-process; these are the
# AI/draft-specific deps outside copilot/ that the global smoke exercises.
- "frontend/src/lib/userDraft.svelte.ts"
- "frontend/src/lib/userDraftDbSyncer.svelte.ts"
- "frontend/src/lib/infer.ts"
- "frontend/src/lib/components/sessions/**"
- ".github/workflows/ai-evals-test.yml"
pull_request:
types: [opened, reopened, ready_for_review]
paths:
- "ai_evals/**"
- "backend/windmill-api/src/ai.rs"
- "backend/windmill-ai/**"
- "frontend/src/lib/components/copilot/**"
# The eval harness runs production frontend code in-process; these are the
# AI/draft-specific deps outside copilot/ that the global smoke exercises.
- "frontend/src/lib/userDraft.svelte.ts"
- "frontend/src/lib/userDraftDbSyncer.svelte.ts"
- "frontend/src/lib/infer.ts"
- "frontend/src/lib/components/sessions/**"
- ".github/workflows/ai-evals-test.yml"
concurrency:
group: ai-evals-test-${{ github.ref }}
cancel-in-progress: true
jobs:
ai_evals_global:
# Provider secrets are unavailable to forked and Dependabot PRs.
if: >-
github.event_name != 'pull_request' ||
(
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.full_name == github.repository &&
github.event.pull_request.user.login != 'dependabot[bot]'
)
runs-on: ubicloud-standard-16
services:
postgres:
image: postgres:16
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.0
- uses: actions/setup-node@v4
with:
# Node 24 ships npm 11, which frontend/package-lock.json is authored
# with; npm 10 rejects it ("Missing: picomatch@4.0.5 from lock file").
# Node must also stay >= 22.19 for the frontend's undici 8.x, which the
# Vitest bridge loads; Node 20 fails with markAsUncloneable.
node-version: "24"
# CE build used only as the AI proxy (login, workspace, provider resource,
# /ai/proxy). No worker execution or MCP needed — global tools/drafts run
# in the Vitest bridge. quickjs matches the standard CE feature set.
- name: Build Windmill (AI proxy)
working-directory: ./backend
env:
SQLX_OFFLINE: true
CARGO_BUILD_JOBS: 12
RUSTFLAGS: ""
run: cargo build --features quickjs
- name: Start Windmill
working-directory: ./backend
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
RUST_LOG: info
run: |
mkdir -p ../ai_evals/logs
./target/debug/windmill > ../ai_evals/logs/windmill.log 2>&1 &
echo "Waiting for Windmill to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
echo "Windmill is ready"
break
fi
sleep 2
done
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; tail -50 ../ai_evals/logs/windmill.log; exit 1; }
- name: Install frontend deps + generate client
working-directory: ./frontend
run: |
npm ci
npm run generate-backend-client
- name: Run harness unit tests
working-directory: ./ai_evals
run: |
bun install
bun test adapters/
# Harness code that reaches into the frontend module graph; bun cannot load it.
- name: Run harness unit tests (frontend graph)
working-directory: ./ai_evals
run: bun run test:frontend-graph
- name: Run global AI evals
timeout-minutes: 20
working-directory: ./ai_evals
env:
WMILL_AI_EVAL_BACKEND_URL: http://localhost:8000
WMILL_AI_EVAL_BACKEND_WORKSPACE: integration-tests
# Anthropic backs the haiku model. Google AI uses GEMINI_API_KEY.
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
run: |
bun install
mkdir -p results
# One cheap model per provider (anthropic/openai/googleai/deepseek).
fail=0
for m in haiku 4o gemini-3-flash-preview deepseek-v4-flash; do
echo "::group::global-test1-script-create ($m)"
if ! bun run cli -- run global global-test1-script-create \
--model "$m" --execution-only --output "$PWD/results/ci-$m.json"; then
echo "$m: harness/proxy errored"
fail=1
echo "::endgroup::"
continue
fi
# The CLI exits 0 when the harness records failed attempts, so gate
# on execution-only pass counts while ignoring model output quality.
if jq -e \
'.attemptCount > 0 and .passedAttempts == .attemptCount' \
"results/ci-$m.json" > /dev/null; then
echo "$m: OK — proxy/frontend execution completed"
else
echo "$m: FAILED proxy/frontend execution"
jq -c '.cases[0].attempts[0].checks' "results/ci-$m.json" || true
fail=1
fi
echo "::endgroup::"
done
[ "$fail" = 0 ] || { echo "ai_evals global smoke failed"; exit 1; }
- name: Archive logs and results
uses: actions/upload-artifact@v4
if: always()
with:
name: ai-evals-global-logs
path: |
ai_evals/logs
ai_evals/results
+25 -31
View File
@@ -17,13 +17,13 @@ jobs:
with:
fetch-depth: 0
- name: Install mold and clang
run: sudo apt-get update && sudo apt-get install -y mold clang
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
toolchain: 1.97.0
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
@@ -36,21 +36,24 @@ jobs:
with:
fetch-depth: 0
- name: install xmlsec1 and gssapi
- name: install xmlsec1
run: |
sudo apt-get update
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
sudo apt-get install -y libxml2-dev libxmlsec1-dev
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
toolchain: 1.97.0
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
run: |
mkdir -p fake_frontend_build
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features all_sqlx_features
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features $(./all_features_oss.sh)
check_ee:
runs-on: ubicloud-standard-8
@@ -75,13 +78,13 @@ jobs:
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Install mold and clang
run: sudo apt-get update && sudo apt-get install -y mold clang
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
toolchain: 1.97.0
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
@@ -106,10 +109,10 @@ jobs:
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: install xmlsec1 and gssapi
- name: install xmlsec1
run: |
sudo apt-get update
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
sudo apt-get install -y libxml2-dev libxmlsec1-dev
- name: Substitute EE code (EE logic is behind feature flag)
run: |
@@ -118,22 +121,13 @@ jobs:
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
- name: Fix stale v8 build cache
working-directory: ./backend
run: |
# Cargo cache may preserve v8 build fingerprints without the actual
# librusty_v8.a library. Since fingerprints look valid, cargo skips
# build.rs re-run, causing "could not find native static library rusty_v8".
for profile in debug release; do
if [ -d "target/$profile/.fingerprint" ] && [ ! -f "target/$profile/gn_out/obj/librusty_v8.a" ]; then
echo "Cleaning stale v8 build artifacts in target/$profile"
rm -rf "target/$profile/build/v8-"* "target/$profile/.fingerprint/v8-"*
fi
done
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
timeout-minutes: 16
working-directory: ./backend
run: |
mkdir -p fake_frontend_build
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features all_sqlx_features,private
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --all-features
-220
View File
@@ -1,220 +0,0 @@
name: Backend integration tests (Windows)
on:
workflow_dispatch:
push:
branches:
- "ci-windows-tests"
tags:
- "v*"
env:
CARGO_INCREMENTAL: 0
SQLX_OFFLINE: true
DISABLE_EMBEDDING: true
jobs:
cargo_test_windows:
runs-on: blacksmith-16vcpu-windows-2025
steps:
- uses: actions/checkout@v4
- name: Read EE repo commit hash
shell: pwsh
run: |
$ee_repo_ref = Get-Content .\backend\ee-repo-ref.txt
echo "ee_repo_ref=$ee_repo_ref" | Out-File -FilePath $env:GITHUB_ENV -Append
- name: Checkout windmill-ee-private repository
uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Substitute EE code
shell: bash
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Setup PostgreSQL
uses: ikalnytskyi/action-setup-postgres@v6
with:
username: postgres
password: changeme
database: windmill
port: 5432
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
# This action defaults RUSTFLAGS to "-D warnings"; unset it so the test
# run is not failed by cross-platform dead-code (cfg(unix)-only helpers
# are unused on Windows). Warning hygiene is enforced on the Linux CI
# and the build_windows_worker_ release build, not this test job.
rustflags: ""
- uses: actions/setup-dotnet@v4
with:
dotnet-version: "9.0.x"
- uses: denoland/setup-deno@v2
with:
# Pin to the Deno version shipped in the runtime image (Dockerfile) so CI
# tests what production runs, instead of floating on the latest v2.x.
deno-version: 2.2.1
- uses: actions/setup-go@v2
with:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.0
- uses: actions/setup-node@v4
with:
node-version: "20"
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.11.24"
- uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
tools: composer
- name: Install windmill CLI
shell: bash
run: |
cd cli
bash gen_wm_client.sh
bun install
mkdir -p "$HOME/.local/bin"
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
chmod +x "$HOME/.local/bin/wmill"
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Install OpenSSL via vcpkg
run: |
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe integrate install
- name: Free disk space (post-vcpkg)
shell: pwsh
run: |
# vcpkg leaves multi-GB of buildtrees/downloads after installing openssl;
# we only need the installed/ dir for linking.
$vcpkgRoot = $env:VCPKG_INSTALLATION_ROOT
foreach ($sub in @("buildtrees", "downloads", "packages")) {
$path = Join-Path $vcpkgRoot $sub
if (Test-Path $path) {
Write-Host "Removing $path"
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue $path
}
}
Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
- name: Get runtime paths
id: runtime-paths
shell: pwsh
run: |
echo "DENO_PATH=$($(Get-Command deno).Source)" >> $env:GITHUB_OUTPUT
echo "BUN_PATH=$($(Get-Command bun).Source)" >> $env:GITHUB_OUTPUT
echo "NODE_BIN_PATH=$($(Get-Command node).Source)" >> $env:GITHUB_OUTPUT
echo "GO_PATH=$($(Get-Command go).Source)" >> $env:GITHUB_OUTPUT
echo "UV_PATH=$($(Get-Command uv).Source)" >> $env:GITHUB_OUTPUT
echo "PHP_PATH=$($(Get-Command php).Source)" >> $env:GITHUB_OUTPUT
echo "COMPOSER_PATH=$($(Get-Command composer).Source)" >> $env:GITHUB_OUTPUT
echo "POWERSHELL_PATH=$($(Get-Command pwsh).Source)" >> $env:GITHUB_OUTPUT
echo "DOTNET_PATH=$($(Get-Command dotnet).Source)" >> $env:GITHUB_OUTPUT
- name: Build DuckDB FFI module
working-directory: backend/windmill-duckdb-ffi-internal
timeout-minutes: 30
run: |
cargo build --release -p windmill_duckdb_ffi_internal
New-Item -ItemType Directory -Path ..\target\debug -Force
Copy-Item target\release\windmill_duckdb_ffi_internal.dll ..\target\debug\
# duckdb is bundled (~2GB of build artifacts); the DLL is the only
# thing we need from this excluded-crate target dir.
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue target
Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
- name: Print runtime versions and env
shell: pwsh
run: |
deno --version
bun -v
node --version
go version
python3 --version
php --version
pwsh --version
dotnet --version
echo "TEMP=$env:TEMP"
echo "TMP=$env:TMP"
echo "USERPROFILE=$env:USERPROFILE"
echo "HOME=$env:HOME"
- name: Disk space before cargo test
shell: pwsh
run: Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
- name: cargo test
working-directory: backend
timeout-minutes: 60
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
RUST_LOG: "off"
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
# 16-vcpu runners with disabled PDB still hit LNK1180 ("insufficient
# disk space") at link time with 12 parallel link jobs: each test
# binary link spikes several hundred MB of transient I/O. Capping at
# 8 trades ~25% wall time for headroom on the ~75GB runner disk.
CARGO_BUILD_JOBS: 8
# backend/Cargo.toml keeps line tables on profile.dev for the (large)
# windmill workspace crates; that debuginfo is emitted into every
# object file and embedded in each test binary, and on windows-msvc
# also spawns the mspdbsrv.exe PDB type server. Across the worker
# crates' test build it drives the peak on the ~63GB free of the
# runner disk (LNK1180 / disk-full during linking). CI reads no
# backtraces, so drop it entirely for the dev/test profiles here:
# debug = 0 means no .pdb and no LNK1318 type-server limit.
CARGO_PROFILE_DEV_DEBUG: "0"
CARGO_PROFILE_TEST_DEBUG: "0"
# Tests' poll-time stack frames (deep nested async fn chains in
# debug builds) reach ~1.8MB. 4MB gives ~2x headroom against flaky
# overflows under parallel-test contention.
RUST_MIN_STACK: 4194304
VCPKGRS_DYNAMIC: 1
OPENSSL_DIR: ${{ env.VCPKG_INSTALLATION_ROOT }}\installed\x64-windows-static
DENO_PATH: ${{ steps.runtime-paths.outputs.DENO_PATH }}
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
GO_PATH: ${{ steps.runtime-paths.outputs.GO_PATH }}
UV_PATH: ${{ steps.runtime-paths.outputs.UV_PATH }}
PHP_PATH: ${{ steps.runtime-paths.outputs.PHP_PATH }}
COMPOSER_PATH: ${{ steps.runtime-paths.outputs.COMPOSER_PATH }}
POWERSHELL_PATH: ${{ steps.runtime-paths.outputs.POWERSHELL_PATH }}
DOTNET_PATH: ${{ steps.runtime-paths.outputs.DOTNET_PATH }}
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
# Windows ships a worker-only binary, so test the crates a worker runs
# (windmill-worker/-common/-queue) via -p, not `--all`: this skips the
# disk-heavy windmill-api test binaries (LNK1180) and the server-only
# windmill-trigger-* crates (amqp does not build on Windows). Linux CI runs the rest.
run: >
cargo test
--no-fail-fast
-p windmill-worker
-p windmill-common
-p windmill-queue
--features private,enterprise,deno_core,duckdb,python,rust,csharp,php,quickjs,parquet,mcp,scoped_cache,windmill-git-sync/private,windmill-object-store/private,windmill-object-store/enterprise
-- --nocapture --test-threads=10
+18 -248
View File
@@ -1,7 +1,6 @@
name: Backend only integration tests
on:
workflow_dispatch:
push:
branches:
- "main"
@@ -20,7 +19,7 @@ defaults:
jobs:
cargo_test:
runs-on: ubicloud-standard-16
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
@@ -29,20 +28,9 @@ jobs:
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c max_connections=500"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5 --shm-size=256mb
mysql:
image: mysql:8.0
ports:
- 3306:3306
env:
MYSQL_ROOT_PASSWORD: changeme
MYSQL_DATABASE: windmill_test
options: >-
--health-cmd "mysqladmin ping -h localhost" --health-interval 10s
--health-timeout 5s --health-retries 5
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
@@ -50,249 +38,31 @@ jobs:
dotnet-version: "9.0.x"
- uses: denoland/setup-deno@v2
with:
# Pin to the Deno version shipped in the runtime image (Dockerfile) so CI
# tests what production runs, instead of floating on the latest v2.x.
deno-version: 2.2.1
deno-version: v2.x
- uses: actions/setup-go@v2
with:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.0
- uses: actions/setup-node@v4
with:
node-version: "24"
bun-version: 1.1.43
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.11.24"
- uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
tools: composer
- uses: ruby/setup-ruby@v1
with:
ruby-version: "3.3"
bundler-cache: false
- name: Install windmill CLI from source
run: |
cd $GITHUB_WORKSPACE/cli
bash gen_wm_client.sh
bun install
mkdir -p "$HOME/.local/bin"
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
chmod +x "$HOME/.local/bin/wmill"
echo "$HOME/.local/bin" >> $GITHUB_PATH
working-directory: /
- name: Install PowerShell, mold and clang
run: |
sudo apt-get update && sudo apt-get install -y powershell mold clang libcurl4-openssl-dev
working-directory: /
version: "0.6.2"
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
- name: Fix stale v8 build cache
working-directory: ./backend
run: |
# Cargo cache may preserve v8 build fingerprints without the actual
# librusty_v8.a library. Since fingerprints look valid, cargo skips
# build.rs re-run, causing "could not find native static library rusty_v8".
for profile in debug release; do
if [ -d "target/$profile/.fingerprint" ] && [ ! -f "target/$profile/gn_out/obj/librusty_v8.a" ]; then
echo "Cleaning stale v8 build artifacts in target/$profile"
rm -rf "target/$profile/build/v8-"* "target/$profile/.fingerprint/v8-"*
fi
done
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Substitute EE code (EE logic is behind feature flag)
run: |
./substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Setup private npm registry with test package
working-directory: /tmp
run: |
set -e
# Install Verdaccio globally
npm install -g verdaccio
# Create Verdaccio config that requires authentication for @windmill-test packages
mkdir -p /tmp/verdaccio/storage
cat > /tmp/verdaccio/config.yaml << 'VERDACCIO_CONFIG'
storage: /tmp/verdaccio/storage
auth:
htpasswd:
file: /tmp/verdaccio/htpasswd
max_users: 100
uplinks:
npmjs:
url: https://registry.npmjs.org/
packages:
'@windmill-test/*':
access: $authenticated
publish: $authenticated
'@*/*':
access: $all
publish: $authenticated
proxy: npmjs
'**':
access: $all
publish: $authenticated
proxy: npmjs
server:
keepAliveTimeout: 60
middlewares:
audit:
enabled: true
log: { type: stdout, format: pretty, level: warn }
VERDACCIO_CONFIG
# Create empty htpasswd file (users will be created via API)
touch /tmp/verdaccio/htpasswd
# Start Verdaccio in background
verdaccio --config /tmp/verdaccio/config.yaml &
VERDACCIO_PID=$!
# Wait for Verdaccio to be ready
echo "Waiting for Verdaccio to start..."
for i in {1..30}; do
if curl -s http://localhost:4873/-/ping > /dev/null 2>&1; then
echo "Verdaccio is ready"
break
fi
sleep 1
done
# Login to get a token
echo "Getting auth token..."
RESPONSE=$(curl -s -X PUT \
-H "Content-Type: application/json" \
-d '{"name":"testuser","password":"testpass123"}' \
http://localhost:4873/-/user/org.couchdb.user:testuser)
echo "Auth response: $RESPONSE"
NPM_TOKEN=$(echo "$RESPONSE" | jq -r '.token')
if [ -z "$NPM_TOKEN" ] || [ "$NPM_TOKEN" = "null" ]; then
echo "Failed to get NPM token from response"
exit 1
fi
echo "NPM_TOKEN=${NPM_TOKEN}" >> $GITHUB_ENV
{
echo "TEST_NPMRC<<NPMRC_EOF"
echo "@windmill-test:registry=http://localhost:4873/"
echo "//localhost:4873/:_authToken=${NPM_TOKEN}"
echo "NPMRC_EOF"
} >> $GITHUB_ENV
echo "Got NPM token successfully: ${NPM_TOKEN:0:10}..."
# Configure npm globally with the auth token
echo "//localhost:4873/:_authToken=${NPM_TOKEN}" > ~/.npmrc
echo "Configured ~/.npmrc with auth token"
# Create a simple test package
mkdir -p /tmp/windmill-test-private-pkg
cat > /tmp/windmill-test-private-pkg/package.json << 'PKG_JSON'
{
"name": "@windmill-test/private-pkg",
"version": "1.0.0",
"main": "index.js"
}
PKG_JSON
cat > /tmp/windmill-test-private-pkg/index.js << 'PKG_JS'
module.exports.greet = (name) => `Hello from private package, ${name}!`;
PKG_JS
# Publish to Verdaccio with auth
cd /tmp/windmill-test-private-pkg
echo "Publishing package..."
npm publish --registry http://localhost:4873
echo "Package published successfully"
# Verify the package requires auth by trying anonymous access (should fail)
rm -f ~/.npmrc
echo "Testing anonymous access (should fail)..."
if npm view @windmill-test/private-pkg --registry http://localhost:4873 2>/dev/null; then
echo "ERROR: Package should require authentication but anonymous access worked"
exit 1
fi
echo "Verified: Package requires authentication for @windmill-test/private-pkg"
- name: Cache DuckDB FFI module build
uses: actions/cache@v3
with:
path: ./backend/windmill-duckdb-ffi-internal/target
key: ${{ runner.os }}-duckdb-ffi-${{ hashFiles('./backend/windmill-duckdb-ffi-internal/src/**/*.rs', './backend/windmill-duckdb-ffi-internal/Cargo.toml', './backend/windmill-duckdb-ffi-internal/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-duckdb-ffi-
workspaces: backend
- name: cargo test
timeout-minutes: 30
env:
# setup-rust-toolchain exports RUSTFLAGS=-D warnings, and the RUSTFLAGS env
# var fully REPLACES (never merges with) target.*.rustflags in
# backend/.cargo/config.toml. That silently drops the config's
# `-C link-arg=-fuse-ld=mold`, so CI links the many large integration-test
# binaries (v8 + duckdb + every language runtime, statically linked) with the
# default bfd linker. Its peak memory across ~12 parallel links OOM-kills the
# runner mid-link (SIGTERM => exit 143, before any test runs). Re-add the mold
# link arg here so CI links with mold like local dev, keeping -D warnings.
# (config.toml's `linker = "clang"` still applies; env only overrides rustflags.)
RUSTFLAGS: "-D warnings -C link-arg=-fuse-ld=mold"
SQLX_OFFLINE: true
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
DISABLE_EMBEDDING: true
RUST_LOG: "off"
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
# Cap parallel rustc/link jobs below the 16 available cores. The tail of
# the build links ~128 full-graph test binaries (one per tests/*.rs file
# across the workspace); at high parallelism enough heavy codegen+link
# units (rustc ~2.6GB, mold ~1GB each) overlap to exhaust the 64GB
# runner. Matches backend-test-windows.yml, which already uses 8.
CARGO_BUILD_JOBS: 8
# Incremental compilation is per-run dead weight in CI: rust-cache
# (cache-workspaces above) restores compiled dependency artifacts but
# never persists target/**/incremental, so there is no prior state to
# reuse in a one-shot `cargo test`. It only adds per-crate memory
# overhead and extra disk. Off here (kept on for local dev via
# .cargo/config.toml). Matches backend-test-windows.yml.
CARGO_INCREMENTAL: "0"
# backend/Cargo.toml keeps line tables on profile.dev for the (large)
# windmill workspace crates; that debug info is emitted into every
# object file and embedded in each test binary. Across the full
# --all --features build it drives the memory/disk peak when mold
# links the windmill-api-integration-tests binary, tipping the runner
# over (lost runner reported as a canceled step). CI reads no
# backtraces, so drop it entirely for the dev/test profiles here.
# (test profile inherits dev, but the workspace crates link in as
# dev-profile deps, so both must be set.) CI-only; local dev builds
# are unaffected.
CARGO_PROFILE_DEV_DEBUG: "0"
CARGO_PROFILE_TEST_DEBUG: "0"
# Tests' poll-time stack frames (deep nested async fn chains in
# debug builds) reach ~1.8MB, leaving very thin headroom on the
# default 2MB thread stack. 4MB gives ~2x buffer against flaky
# overflows under parallel-test contention.
RUST_MIN_STACK: 4194304
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
TEST_NPM_REGISTRY: "http://localhost:4873/:_authToken=${{ env.NPM_TOKEN }}"
run: |
deno --version && bun -v && node --version && go version && python3 --version && php --version && ruby --version && pwsh --version && dotnet --version
# The FFI crate is excluded from the workspace, so the `cargo test` below
# never reaches it. Pin the target dir (matching the cache step above) so
# its own tests run off this compile rather than a second bundled build.
(cd windmill-duckdb-ffi-internal && export CARGO_TARGET_DIR="$PWD/target" && ./build_dev.sh && cargo test --release -p windmill_duckdb_ffi_internal)
DENO_PATH=$(which deno) BUN_PATH=$(which bun) NODE_BIN_PATH=$(which node) GO_PATH=$(which go) UV_PATH=$(which uv) PHP_PATH=$(which php) COMPOSER_PATH=$(which composer) RUBY_PATH=$(which ruby) RUBY_BUNDLE_PATH=$(which bundle) RUBY_GEM_PATH=$(which gem) POWERSHELL_PATH=$(which pwsh) DOTNET_PATH=$(which dotnet) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp,run_inline --all -- --nocapture --test-threads=10
timeout-minutes: 16
run:
deno --version && bun -v && go version && python3 --version &&
SQLX_OFFLINE=true
DATABASE_URL=postgres://postgres:changeme@localhost:5432/windmill
DISABLE_EMBEDDING=true RUST_LOG=info
DENO_PATH=$(which deno) BUN_PATH=$(which bun) GO_PATH=$(which go)
UV_PATH=$(which uv) cargo test --features
enterprise,deno_core,license,python,rust,scoped_cache --all --
--nocapture
-44
View File
@@ -290,49 +290,6 @@ jobs:
path: |
*.json
benchmark_wac:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
--shm-size=2g
windmill:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always --health-interval 10s --health-timeout 5s
--health-retries 5 --health-cmd "curl
http://localhost:8000/api/version"
ports:
- 8000:8000
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: benchmark
timeout-minutes: 30
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
-c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_wac.json
- name: Save benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark_wac
path: |
*.json
benchmark_graphs:
runs-on: ubicloud
needs:
@@ -340,7 +297,6 @@ jobs:
- benchmark_dedicated
- benchmark_4workers
- benchmark_8workers
- benchmark_wac
steps:
- uses: denoland/setup-deno@v2
with:
@@ -5,20 +5,6 @@ env:
name: Build caddy-l4
on:
workflow_dispatch:
push:
branches:
- main
paths:
- docker/DockerfileCaddyL4
- docker/entrypoint-caddy.sh
- docker/caddy-compat-normalize.awk
- docker/caddy-l4.version
- docker/test-caddy-compat.sh
- Caddyfile
# The version check below reads the pin out of docker-compose.yml, so a
# compose-only bump has to trigger this workflow or the check never runs.
- docker-compose.yml
- .github/workflows/build-caddy-l4-image.yml
permissions: write-all
@@ -28,35 +14,6 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: depot/setup-action@v1
# docker-compose.yml pins an exact tag, and the Caddyfile it must agree
# with lives in the same checkout. Fail the build rather than publish a
# version nothing references, which is how :latest drifted from the
# Caddyfile in the first place.
- name: Resolve and check image version
id: version
run: |
set -euo pipefail
version="$(tr -d '[:space:]' < docker/caddy-l4.version)"
pinned="$(grep -oE 'caddy-l4:[^[:space:]"]+' docker-compose.yml | head -1 | cut -d: -f2-)"
caddy="$(grep -m1 -oE '^FROM caddy:[0-9]+\.[0-9]+\.[0-9]+' docker/DockerfileCaddyL4 | cut -d: -f2)"
if [ "$version" != "$pinned" ]; then
echo "docker/caddy-l4.version is '$version' but docker-compose.yml pins '$pinned'" >&2
echo "Bump both together." >&2
exit 1
fi
# Otherwise a caddy bump that forgets the version file publishes a tag
# that names the wrong caddy.
case "$version" in
"$caddy"-*) ;;
*)
echo "docker/caddy-l4.version is '$version' but the Dockerfile pins caddy '$caddy'" >&2
echo "The version must be <caddy-version>-<revision>." >&2
exit 1
;;
esac
echo "version=$version" >> "$GITHUB_OUTPUT"
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
@@ -66,22 +23,8 @@ jobs:
tags: |
type=sha
type=ref,event=branch
# Not gated on the default branch: docker-compose.yml pins this tag,
# so it has to be publishable from a branch (workflow_dispatch)
# before the pin merges, or main would reference a tag that does not
# exist yet. The version is immutable, so republishing from main is
# a no-op. Only branch pushes to main and manual dispatch run this
# workflow, so a branch cannot claim the tag by accident.
type=raw,value=${{ steps.version.outputs.version }}
type=raw,value=latest,enable={{is_default_branch}}
# The shim rewrites config a self-hoster never sees, so a silent
# regression here strands them on a restart loop or a dead :80.
- name: Test the legacy-Caddyfile compatibility shim
run: |
docker build -f docker/DockerfileCaddyL4 -t caddy-l4:ci ./docker
docker/test-caddy-compat.sh caddy-l4:ci
- name: Login to registry
uses: docker/login-action@v3
with:
-65
View File
@@ -1,65 +0,0 @@
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
name: Build windmill-extra
on:
workflow_dispatch:
inputs:
tag:
description: "Tag for the image"
required: false
default: "dev"
type: string
permissions: write-all
jobs:
sleep:
runs-on: ubicloud
steps:
- name: Sleep for 900 seconds waiting for pypi to update index
if: startsWith(github.ref, 'refs/tags/v')
run: sleep 900
shell: bash
build_extra:
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- uses: depot/setup-action@v1
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-extra
flavor: |
latest=false
tags: |
type=raw,value=${{ github.event.inputs.tag }}
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileExtra"
tags: |
${{ steps.meta.outputs.tags }}
labels: |
${{ steps.meta.outputs.labels }}
+55 -42
View File
@@ -9,7 +9,7 @@ permissions: write-all
jobs:
build_ee:
runs-on: ubicloud-standard-4
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
@@ -27,6 +27,8 @@ jobs:
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
@@ -55,63 +57,74 @@ jobs:
run: |
cp ./docker/RHEL9/Dockerfile ./Dockerfile
- name: Build and push EE (multi-arch)
- name: Build and push publicly ee amd64
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
push: true
build-args: |
features=ee_rhel
WM_BUILD_VERSION=${{ github.sha }}
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,license,otel,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp,private
secrets: |
rh_username=${{ secrets.RH_USERNAME }}
rh_password=${{ secrets.RH_PASSWORD }}
tags: |
${{ steps.meta-ee-public.outputs.tags }}
${{ steps.meta-ee-public.outputs.tags }}-amd64
labels: |
${{ steps.meta-ee-public.outputs.labels }}
${{ steps.meta-ee-public.outputs.labels }}-amd64
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Install crane
uses: imjasonh/setup-crane@v0.4
- name: Extract binaries with crane
run: |
mkdir -p extracted
# Extract arm64 binary (include deps/ for hard link resolution)
mkdir -p /tmp/arm64
crane export --platform linux/arm64 ${{ steps.meta-ee-public.outputs.tags }} - \
| tar -xf - -C /tmp/arm64 windmill/target/release/ usr/src/app/libwindmill_duckdb_ffi_internal.so
cp /tmp/arm64/windmill/target/release/windmill extracted/windmill-ee-arm64-rhel9
cp /tmp/arm64/usr/src/app/libwindmill_duckdb_ffi_internal.so extracted/libwindmill_duckdb_ffi_internal-arm64.so
rm -rf /tmp/arm64
# Extract amd64 binary
mkdir -p /tmp/amd64
crane export --platform linux/amd64 ${{ steps.meta-ee-public.outputs.tags }} - \
| tar -xf - -C /tmp/amd64 windmill/target/release/ usr/src/app/libwindmill_duckdb_ffi_internal.so
cp /tmp/amd64/windmill/target/release/windmill extracted/windmill-ee-amd64-rhel9
cp /tmp/amd64/usr/src/app/libwindmill_duckdb_ffi_internal.so extracted/libwindmill_duckdb_ffi_internal-amd64.so
rm -rf /tmp/amd64
- uses: actions/upload-artifact@v4
- name: Build and push publicly ee arm64
uses: depot/build-push-action@v1
with:
name: RHEL9-arm64 build
path: extracted/windmill-ee-arm64-rhel9
context: .
platforms: linux/arm64
push: true
build-args: |
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,license,otel,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp,private
secrets: |
rh_username=${{ secrets.RH_USERNAME }}
rh_password=${{ secrets.RH_PASSWORD }}
tags: |
${{ steps.meta-ee-public.outputs.tags }}-arm64
labels: |
${{ steps.meta-ee-public.outputs.labels }}-arm64
org.opencontainers.image.licenses=Windmill-Enterprise-License
- uses: shrink/actions-docker-extract@v3
id: extract-ee-amd64
with:
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
path: "/windmill/target/release/windmill"
# - uses: shrink/actions-docker-extract@v3
# id: extract-ee-arm64
# with:
# image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
# path: "/windmill/target/release/windmill"
- name: Rename binary with corresponding architecture
run: |
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9"
# mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9"
- uses: actions/upload-artifact@v4
with:
name: RHEL9-amd64 build
path: extracted/windmill-ee-amd64-rhel9
path: ${{ steps.extract-ee-amd64.outputs.destination
}}/windmill-ee-amd64-rhel9
- uses: actions/upload-artifact@v4
with:
name: RHEL9-arm64 dynamic libraries build
path: extracted/libwindmill_duckdb_ffi_internal-arm64.so
# - uses: actions/upload-artifact@v4
# with:
# name: RHEL9-arm64 build
# path:
# ${{ steps.extract-ee-arm64.outputs.destination
# }}/windmill-ee-arm64-rhel9
- uses: actions/upload-artifact@v4
with:
name: RHEL9-amd64 dynamic libraries build
path: extracted/libwindmill_duckdb_ffi_internal-amd64.so
# - name: Attach binary to release
# uses: softprops/action-gh-release@v2
# if: startsWith(github.ref, 'refs/tags/')
# with:
# files: |
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9
@@ -1,142 +0,0 @@
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
name: Build and publish windmill for RHEL8
on: workflow_dispatch
permissions: write-all
jobs:
build_ee:
runs-on: ubicloud-standard-4
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-rhel8
flavor: |
latest=false
tags: |
type=sha
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Substitute EE code
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Copy RHEL8 Dockerfile
run: |
cp ./docker/RHEL8/Dockerfile ./Dockerfile
- name: Build and push publicly ee amd64
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
build-args: |
features=ee_rhel
WM_BUILD_VERSION=${{ github.sha }}
secrets: |
rh_username=${{ secrets.RH_USERNAME }}
rh_password=${{ secrets.RH_PASSWORD }}
tags: |
${{ steps.meta-ee-public.outputs.tags }}-amd64
labels: |
${{ steps.meta-ee-public.outputs.labels }}-amd64
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Build and push publicly ee arm64
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/arm64
push: true
build-args: |
features=ee_rhel
WM_BUILD_VERSION=${{ github.sha }}
secrets: |
rh_username=${{ secrets.RH_USERNAME }}
rh_password=${{ secrets.RH_PASSWORD }}
tags: |
${{ steps.meta-ee-public.outputs.tags }}-arm64
labels: |
${{ steps.meta-ee-public.outputs.labels }}-arm64
org.opencontainers.image.licenses=Windmill-Enterprise-License
- uses: shrink/actions-docker-extract@v3
id: extract-ee-amd64
with:
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
path: "/windmill/target/release/windmill"
- uses: shrink/actions-docker-extract@v3
id: extract-duckdb-ffi-internal
with:
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
path: "/usr/src/app/libwindmill_duckdb_ffi_internal.so"
# - uses: shrink/actions-docker-extract@v3
# id: extract-ee-arm64
# with:
# image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
# path: "/windmill/target/release/windmill"
- name: Rename binary with corresponding architecture
run: |
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8"
# mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel8"
- uses: actions/upload-artifact@v4
with:
name: RHEL8-amd64 build
path: ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8
- uses: actions/upload-artifact@v4
with:
name: RHEL8-amd64 dynamic libraries build
path: ${{ steps.extract-duckdb-ffi-internal.outputs.destination }}/libwindmill_duckdb_ffi_internal.so
# - uses: actions/upload-artifact@v4
# with:
# name: RHEL8-arm64 build
# path:
# ${{ steps.extract-ee-arm64.outputs.destination
# }}/windmill-ee-arm64-rhel8
# - name: Attach binary to release
# uses: softprops/action-gh-release@v2
# if: startsWith(github.ref, 'refs/tags/')
# with:
# files: |
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel8
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8
-13
View File
@@ -13,13 +13,9 @@ permissions:
contents: read
id-token: write
packages: write
attestations: write
jobs:
publish_cli:
# a tag-targeted dispatch would republish the release tags unsigned,
# un-verifying the release; to republish a release, re-push its tag
if: github.event_name == 'push' || !startsWith(github.ref, 'refs/tags/')
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
@@ -46,23 +42,14 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
id: docker_build
uses: depot/build-push-action@v1
with:
file: "./docker/DockerfileCli"
platforms: linux/amd64,linux/arm64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
${{ steps.meta.outputs.tags }}
labels: |
${{ steps.meta.outputs.labels }}
org.opencontainers.image.licenses=AGPLv3
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
digest: ${{ steps.docker_build.outputs.digest }}
+12 -34
View File
@@ -11,7 +11,7 @@ env:
jobs:
cargo_build_windows:
runs-on: blacksmith-16vcpu-windows-2025
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
@@ -30,56 +30,34 @@ jobs:
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Setup Rust
uses: actions-rs/toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
toolchain: 1.85.0
override: true
- name: Substitute EE code
shell: bash
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Cargo check (fail fast on warnings)
timeout-minutes: 60
env:
RUSTFLAGS: "-D warnings"
run: |
cd backend
# Stub the openapi specs to empty: they are compiled in via an ungated
# include_str! but a worker binary never serves them, so this avoids
# embedding ~2.5MB of spec.
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
New-Item -Path . -Name "windmill-api/openapi-deref.json" -ItemType "File" -Force
cargo check --features=ee_windows
- name: Cargo build dynamic libraries windows
timeout-minutes: 180
run: |
cd backend/windmill-duckdb-ffi-internal
cargo build --release -p windmill_duckdb_ffi_internal
- name: Cargo build binary windows
timeout-minutes: 180
- name: Cargo build windows
timeout-minutes: 90
run: |
vcpkg.exe install openssl-windows:x64-windows
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe integrate install
$env:VCPKGRS_DYNAMIC=1
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
cd backend
cargo build --release --features=ee_windows
mkdir frontend/build && cd backend
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,license,http_trigger,zip,oauth2,kafka,nats,sqs_trigger,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages_windows,mcp,private
- name: Rename binary with corresponding architecture
run: |
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
- name: Upload binary artifact
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: windmill-ee-binary
path: ./backend/target/release/windmill-ee.exe
- name: Upload dynamic libraries artifact
uses: actions/upload-artifact@v4
with:
name: windmill_duckdb_ffi_internal.dll
path: ./backend/windmill-duckdb-ffi-internal/target/release/windmill_duckdb_ffi_internal.dll
-23
View File
@@ -1,23 +0,0 @@
name: Check frontend docs links
on:
push:
tags:
- "v*"
workflow_dispatch:
jobs:
check-docs-links:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: |
frontend/src
.github/scripts
- uses: actions/setup-node@v4
with:
node-version: "22.x"
- name: Verify docs links are not 404
run: node .github/scripts/check-docs-links.mjs
-19
View File
@@ -1,19 +0,0 @@
name: Check fixture is empty
on:
push:
branches: [main]
paths:
- "fixtures/**"
pull_request:
paths:
- "fixtures/**"
jobs:
check-empty-fixture:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Ensure fixtures/cli-sync/ has no committed snapshot
run: bash fixtures/check-empty.sh
@@ -0,0 +1,60 @@
name: Check Organization Membership
on:
workflow_call:
inputs:
commenter:
required: true
type: string
description: 'The username to check for organization membership'
organization:
required: false
type: string
default: 'windmill-labs'
description: 'The organization to check membership for'
trusted_bot:
required: false
type: string
default: 'windmill-internal-app[bot]'
description: 'The trusted bot username to allow'
secrets:
access_token:
required: true
description: 'The access token to use for org membership check'
outputs:
is_member:
description: 'Whether the user is an organization member or trusted bot'
value: ${{ jobs.check-membership.outputs.is_member }}
jobs:
check-membership:
runs-on: ubicloud-standard-2
outputs:
is_member: ${{ steps.check-membership.outputs.is_member }}
steps:
- name: Check organization membership
id: check-membership
env:
ORG_ACCESS_TOKEN: ${{ secrets.access_token }}
COMMENTER: ${{ inputs.commenter }}
ORG: ${{ inputs.organization }}
TRUSTED_BOT: ${{ inputs.trusted_bot }}
run: |
# 1. Allow the trusted bot straight away
if [[ "$COMMENTER" == "$TRUSTED_BOT" ]]; then
echo "is_member=true" >> $GITHUB_OUTPUT
exit 0
fi
# 2. Otherwise fall back to the org-membership check
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $ORG_ACCESS_TOKEN" \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
if [ "$STATUS" -eq 204 ]; then
echo "is_member=true" >> $GITHUB_OUTPUT
else
echo "is_member=false" >> $GITHUB_OUTPUT
fi
@@ -1,39 +0,0 @@
name: Check system prompts freshness
on:
push:
paths:
- "system_prompts/**"
- "typescript-client/**"
- "python-client/wmill/wmill/client.py"
- "openflow.openapi.yaml"
- "backend/windmill-api/openapi.yaml"
- "cli/src/main.ts"
- "cli/src/commands/**"
- "frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts"
pull_request:
paths:
- "system_prompts/**"
- "typescript-client/**"
- "python-client/wmill/wmill/client.py"
- "openflow.openapi.yaml"
- "backend/windmill-api/openapi.yaml"
- "cli/src/main.ts"
- "cli/src/commands/**"
- "frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts"
jobs:
check-freshness:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install pyyaml
- name: Check auto-generated files are up-to-date
run: bash system_prompts/check-freshness.sh
-74
View File
@@ -1,74 +0,0 @@
name: Check Write Access
# Authorizes a user to trigger privileged command workflows (/review, /ai, /plan,
# /updatesqlx, ...). The webhook author_association reports PRIVATE org members as
# CONTRIBUTOR/NONE (only public members show as MEMBER), so command jobs can't gate on
# it alone. This mints the internal GitHub App token — which can see private members —
# and confirms the user is a member or has write access to the repo. The app token is
# minted fresh per run, so unlike the old ORG_ACCESS_TOKEN PAT it never expires.
on:
workflow_call:
inputs:
username:
required: true
type: string
description: 'The user whose access to verify'
trusted_bot:
required: false
type: string
default: 'windmill-internal-app[bot]'
description: 'A bot login that is always authorized'
outputs:
authorized:
description: 'true if the user is the trusted bot, an org member, or has repo write access'
value: ${{ jobs.check.outputs.authorized }}
jobs:
check:
runs-on: ubuntu-latest
outputs:
authorized: ${{ steps.check.outputs.authorized }}
steps:
# This check is purely additive: callers OR it with author_association, so it must
# never fail the job. Failing here would block every dependent reviewer job through
# `needs`, turning an unconfigured or misconfigured app into a total review outage
# rather than a fallback to the author_association path.
- name: Mint internal app token
id: app
if: vars.INTERNAL_APP_ID != ''
continue-on-error: true
uses: actions/create-github-app-token@v2
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
owner: ${{ github.repository_owner }}
- name: Resolve authorization
id: check
env:
# Without the app token, the default token still resolves public members and
# repo collaborators; private members simply fall through to author_association.
GH_TOKEN: ${{ steps.app.outputs.token || github.token }}
USERNAME: ${{ inputs.username }}
TRUSTED_BOT: ${{ inputs.trusted_bot }}
REPO: ${{ github.repository }}
run: |
if [ "$USERNAME" = "$TRUSTED_BOT" ]; then
echo "authorized=true" >> "$GITHUB_OUTPUT"
exit 0
fi
ORG="${REPO%%/*}"
# Org membership resolves private members too (204 = member, 404 = not).
if gh api "orgs/$ORG/members/$USERNAME" --silent 2>/dev/null; then
echo "authorized=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Fallback: effective repo permission (also covers outside collaborators).
PERM=$(gh api "repos/$REPO/collaborators/$USERNAME/permission" --jq '.permission' 2>/dev/null || echo none)
if [ "$PERM" = "admin" ] || [ "$PERM" = "write" ]; then
echo "authorized=true" >> "$GITHUB_OUTPUT"
else
echo "authorized=false" >> "$GITHUB_OUTPUT"
echo "$USERNAME is neither the trusted bot, an org member, nor a repo writer."
fi
-73
View File
@@ -1,73 +0,0 @@
name: Claude Plan Assistant
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
# author_association misses private org members; check-access resolves them via the
# internal app token. Both are OR'd below so public members still pass instantly.
check-access:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/plan')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/plan')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/plan')) ||
(github.event_name == 'issues' && contains(github.event.issue.body, '/plan'))
uses: ./.github/workflows/check-write-access.yml
with:
username: ${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}
secrets: inherit
claude-plan-action:
needs: [check-access]
if: |
needs.check-access.outputs.authorized == 'true' ||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association)
runs-on: ubicloud-standard-4
timeout-minutes: 20
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Plan Action
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/plan'
claude_args: |
--model claude-opus-5
--system-prompt "# Claude Planning Mode
You are operating in PLANNING MODE ONLY. Your role is to create detailed, structured plans without making any code changes.
## Your Responsibilities:
1. **Analyze the Request**: Carefully read and understand what the user is asking for
2. **Explore the Codebase**: Understand the relevant code structure
3. **Create a Detailed Plan**: Provide a comprehensive, step-by-step plan that includes:
- Clear breakdown of all tasks needed
- Files that will need to be modified or created
- Code patterns and architecture decisions
- Potential challenges and how to address them
- If there are multiple options to achieve the same goal, explain the pros and cons of each option
## Strict Constraints:
- **DO NOT** make any code changes
- **DO NOT** create branches or pull requests
Remember: You are here to plan, not to implement. Provide thorough analysis and clear guidance for implementation."
+95 -64
View File
@@ -1,4 +1,4 @@
name: Fast Claude
name: Claude PR Assistant
on:
issue_comment:
@@ -11,29 +11,47 @@ on:
types: [submitted]
jobs:
# author_association misses private org members; check-access resolves them via the
# internal app token. Both are OR'd below so public members still pass instantly.
check-access:
determine-commenter:
if: |
(github.event_name == 'issue_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
(github.event_name == 'pull_request_review_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
(github.event_name == 'pull_request_review' && startsWith(github.event.review.body, '/ai') && !startsWith(github.event.review.body, '/ai-fast')) ||
(github.event_name == 'issues' && startsWith(github.event.issue.body, '/ai') && !startsWith(github.event.issue.body, '/ai-fast'))
uses: ./.github/workflows/check-write-access.yml
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/ai')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/ai')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/ai')) ||
(github.event_name == 'issues' && contains(github.event.issue.body, '/ai'))
runs-on: ubicloud-standard-2
outputs:
commenter: ${{ steps.determine-commenter.outputs.commenter }}
steps:
- name: Determine commenter
id: determine-commenter
run: |
# Work out who wrote the comment / review
if [[ "${{ github.event_name }}" == "issue_comment" || \
"${{ github.event_name }}" == "pull_request_review_comment" ]]; then
COMMENTER="${{ github.event.comment.user.login }}"
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
COMMENTER="${{ github.event.review.user.login }}"
else
COMMENTER="${{ github.event.issue.user.login }}"
fi
echo "commenter=$COMMENTER" >> $GITHUB_OUTPUT
check-membership:
needs: determine-commenter
uses: ./.github/workflows/check-org-membership.yml
with:
username: ${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}
secrets: inherit
commenter: ${{ needs.determine-commenter.outputs.commenter }}
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
claude-code-action:
needs: [check-access]
needs: [determine-commenter, check-membership]
if: |
needs.check-access.outputs.authorized == 'true' ||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association)
needs.check-membership.outputs.is_member == 'true'
runs-on: ubicloud-standard-8
permissions:
contents: write
pull-requests: write
issues: write
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
@@ -41,56 +59,69 @@ jobs:
with:
fetch-depth: 1
# Make the EE source (the *_ee.rs files in the companion repo) available so the
# reviewer can see EE-only code (e.g. windmill-queue/src/jobs_ee.rs), not just the
# CE surface. The EE ref is read from the PR head's backend/ee-repo-ref.txt (via the
# API, so it reflects the PR's EE pin regardless of which ref is checked out here).
- name: Check EE access
id: ee
env:
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
run: |
if [ -z "$EE_TOKEN" ] || [ -z "$PR_NUMBER" ]; then
echo "available=false" >> "$GITHUB_OUTPUT"
exit 0
fi
HEAD_SHA=$(gh api "repos/${{ github.repository }}/pulls/$PR_NUMBER" --jq .head.sha)
REF=$(gh api "repos/${{ github.repository }}/contents/backend/ee-repo-ref.txt?ref=$HEAD_SHA" --jq .content | base64 -d | tr -d '[:space:]')
if [ -z "$REF" ]; then
echo "available=false" >> "$GITHUB_OUTPUT"
else
echo "available=true" >> "$GITHUB_OUTPUT"
echo "ee_repo_ref=$REF" >> "$GITHUB_OUTPUT"
fi
- name: Checkout EE repository
if: steps.ee.outputs.available == 'true'
uses: actions/checkout@v4
- uses: actions/cache@v3
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ steps.ee.outputs.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 1
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Substitute EE code
if: steps.ee.outputs.available == 'true'
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Run npm install and generate-backend-client
working-directory: ./frontend
run: |
# add a build directory for cargo check
mkdir -p build
npm install
npm run generate-backend-client
- name: install xmlsec1
run: |
sudo apt-get update
sudo apt-get install -y libxml2-dev libxmlsec1-dev
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
run: |
SQLX_OFFLINE=true cargo check --features $(./all_features_oss.sh)
- name: Run Claude PR Action
uses: anthropics/claude-code-action@v1
uses: anthropics/claude-code-action@beta
env:
SQLX_OFFLINE: true
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: "windmill-internal-app[bot]"
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
timeout_minutes: "60"
allowed_tools: "mcp__github__create_pull_request,Bash"
custom_instructions: |
## IMPORTANT INSTRUCTIONS
- Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main.
## Code Quality Requirements
After making any code changes, you MUST run the appropriate validation commands:
**Frontend Changes:**
- Run: `npm run check` in the frontend directory
- Fix all warnings and errors before proceeding
**Backend Changes:**
- Run: `cargo check --features $(./all_features_oss.sh)` in the backend directory
- Fix all warnings and errors before proceeding
**Pull Request Creation:**
- DO NOT FORGET TO OPEN A DRAFT PR AFTER YOU ARE DONE if you made changes after a request from a git issue.
## Available Tools
- mcp__github__create_pull_request: Create PRs from branches
- Bash: Full access to run validation commands and git operations
trigger_phrase: "/ai"
settings: |
{
"env": {
"SQLX_OFFLINE": "true"
}
}
claude_args: |
--allowedTools "Bash,WebFetch,WebSearch"
--model claude-opus-5
-201
View File
@@ -1,201 +0,0 @@
name: CLI Tests
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "cli/**"
- "windmill-yaml-validator/**"
- "backend/migrations/**"
- ".github/workflows/cli-tests.yml"
# The bundles cli/ vendors from the frontend: their drift guards live in
# cli/test but the edits that break them land here. The policy bundle
# inlines its imports too, so those sources belong in the filter.
- "frontend/src/lib/components/raw_apps/**"
- "frontend/src/lib/components/recording/**"
- "frontend/src/lib/components/apps/editor/commonAppUtils.ts"
- "frontend/src/lib/components/apps/inputType.ts"
pull_request:
branches: [main]
paths:
- "cli/**"
- "windmill-yaml-validator/**"
- "backend/migrations/**"
- ".github/workflows/cli-tests.yml"
# The bundles cli/ vendors from the frontend: their drift guards live in
# cli/test but the edits that break them land here. The policy bundle
# inlines its imports too, so those sources belong in the filter.
- "frontend/src/lib/components/raw_apps/**"
- "frontend/src/lib/components/recording/**"
- "frontend/src/lib/components/apps/editor/commonAppUtils.ts"
- "frontend/src/lib/components/apps/inputType.ts"
env:
CARGO_TERM_COLOR: always
SQLX_OFFLINE: true
jobs:
build-check:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Generate Windmill client
working-directory: cli
run: ./gen_wm_client.sh
- name: Run CLI build
working-directory: cli
run: ./build.sh
test-linux:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: changeme
POSTGRES_DB: windmill
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: true
cache-workspaces: backend
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Symlink Bun to /usr/bin/bun
run: sudo ln -sf $(which bun) /usr/bin/bun
- name: Symlink Node to /usr/bin/node
run: sudo ln -sf $(which node) /usr/bin/node
- name: Install dependencies
working-directory: cli
run: bun install
- name: Generate Windmill clients
working-directory: cli
run: |
./gen_wm_client.sh
./windmill-utils-internal/gen_wm_client.sh
- name: Run CLI tests
working-directory: cli
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432
CI_MINIMAL_FEATURES: "true"
run: bun test --timeout 120000 test/
test-windows:
runs-on: blacksmith-16vcpu-windows-2025
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup PostgreSQL
uses: ikalnytskyi/action-setup-postgres@v6
with:
username: postgres
password: changeme
database: windmill
port: 5432
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: true
cache-workspaces: backend
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Get Bun and Node paths
id: runtime-paths
shell: pwsh
run: |
$bunPath = (Get-Command bun).Source
$nodePath = (Get-Command node).Source
echo "BUN_PATH=$bunPath" >> $env:GITHUB_OUTPUT
echo "NODE_BIN_PATH=$nodePath" >> $env:GITHUB_OUTPUT
- name: Install dependencies
working-directory: cli
run: bun install
- name: Generate Windmill clients
working-directory: cli
shell: bash
run: |
./gen_wm_client.sh
./windmill-utils-internal/gen_wm_client.sh
- name: Run CLI tests
working-directory: cli
shell: pwsh
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432
CI_MINIMAL_FEATURES: "true"
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
run: bun test --timeout 120000 test/
# Combined summary job for branch protection
test-summary:
runs-on: ubuntu-latest
needs: [build-check, test-linux, test-windows]
if: always()
steps:
- name: Check test results
run: |
if [ "${{ needs.build-check.result }}" != "success" ]; then
echo "Build check failed"
exit 1
fi
if [ "${{ needs.test-linux.result }}" != "success" ] || [ "${{ needs.test-windows.result }}" != "success" ]; then
echo "Some tests failed"
exit 1
fi
echo "All checks passed"
-421
View File
@@ -1,421 +0,0 @@
name: Codex Auto Review
on:
pull_request:
types: [ready_for_review, opened, synchronize]
workflow_call:
inputs:
pr_number:
description: 'PR number to review'
required: true
type: number
extra_prompt:
description: 'Additional reviewer instructions appended to the standard review prompt'
required: false
type: string
default: ''
triggered_by:
description: 'GitHub username that triggered this review (for audit only)'
required: false
type: string
default: ''
secrets:
OPENAI_API_KEY:
required: false
CODEX_AUTH_JSON:
required: false
WINDMILL_EE_PRIVATE_ACCESS:
required: false
concurrency:
group: codex-review-${{ inputs.pr_number || github.event.pull_request.number }}
cancel-in-progress: true
jobs:
codex-review:
runs-on: ubicloud-standard-2
timeout-minutes: 30
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
# access to this repo, so fork==false already enforces write access. Do NOT re-add
# an author_association gate: the pull_request webhook payload reports private org
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
# skips auto-review for every private member.
if: |
github.event_name == 'workflow_call' ||
(
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.fork == false
)
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Check Codex configuration
id: codex_config
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
run: |
if [ -n "$OPENAI_API_KEY" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
echo "auth_mode=api_key" >> "$GITHUB_OUTPUT"
elif [ -n "$CODEX_AUTH_JSON" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
echo "auth_mode=oauth_json" >> "$GITHUB_OUTPUT"
else
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "Codex auth is not configured; set OPENAI_API_KEY or CODEX_AUTH_JSON to enable Codex review."
fi
- name: Resolve PR metadata
if: steps.codex_config.outputs.enabled == 'true'
id: pr
env:
GH_TOKEN: ${{ github.token }}
INPUT_PR_NUMBER: ${{ inputs.pr_number }}
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
EVENT_BASE_SHA: ${{ github.event.pull_request.base.sha }}
EVENT_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
EVENT_TITLE: ${{ github.event.pull_request.title }}
EVENT_BODY: ${{ github.event.pull_request.body }}
EVENT_FORK: ${{ github.event.pull_request.head.repo.fork }}
EVENT_AUTHOR: ${{ github.event.pull_request.user.login }}
EVENT_ACTION: ${{ github.event.action }}
run: |
if [ -n "$INPUT_PR_NUMBER" ]; then
PR_JSON=$(gh pr view "$INPUT_PR_NUMBER" --repo "${{ github.repository }}" \
--json number,baseRefName,baseRefOid,headRefOid,title,body,isCrossRepository,author)
PR_NUMBER=$(echo "$PR_JSON" | jq -r '.number')
BASE_REF=$(echo "$PR_JSON" | jq -r '.baseRefName')
BASE_SHA=$(echo "$PR_JSON" | jq -r '.baseRefOid')
HEAD_SHA=$(echo "$PR_JSON" | jq -r '.headRefOid')
PR_TITLE=$(echo "$PR_JSON" | jq -r '.title')
PR_BODY=$(echo "$PR_JSON" | jq -r '.body // ""')
IS_FORK=$(echo "$PR_JSON" | jq -r '.isCrossRepository')
PR_AUTHOR=$(echo "$PR_JSON" | jq -r '.author.login // ""')
else
PR_NUMBER="$EVENT_PR_NUMBER"
BASE_REF="$EVENT_BASE_REF"
BASE_SHA="$EVENT_BASE_SHA"
HEAD_SHA="$EVENT_HEAD_SHA"
PR_TITLE="$EVENT_TITLE"
PR_BODY="$EVENT_BODY"
IS_FORK="$EVENT_FORK"
PR_AUTHOR="$EVENT_AUTHOR"
fi
# Fork PRs run untrusted code with secrets present, so the automatic
# pull_request trigger never reviews them. A non-empty INPUT_PR_NUMBER
# means we arrived via workflow_call (a maintainer /codex comment gated
# by check-write-access), so allow forks only on that path.
if [ "$IS_FORK" = "true" ] && [ -z "$INPUT_PR_NUMBER" ]; then
echo "Skipping Codex review for fork PR (automatic trigger)."
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# An agent-driven PR flips to ready only after a clean /review round on
# a draft, marked by an author comment naming the head SHA (pr skill,
# "Review rounds"). Re-reviewing that same head on ready_for_review is
# redundant. The marker alone is author attestation, so also require
# reviewer evidence: a Codex review (posted by github-actions[bot], not
# forgeable by the author) that predates the marker and carries a
# non-blocking verdict. Comment-triggered and synchronize runs never
# skip. Keep the three copies of this check in sync (pr-ready-review /
# codex-pr-review / pi-pr-review); a shared local action would need the
# repo checked out before the check, which the fork paths here
# deliberately avoid.
if [ "$EVENT_ACTION" = "ready_for_review" ] && [ -z "$INPUT_PR_NUMBER" ]; then
# Fetch failures fail open (no skip): an API hiccup must run the
# review, never skip it or fail the job.
COMMENTS=$(gh api "repos/${{ github.repository }}/issues/$PR_NUMBER/comments?per_page=100" --paginate | jq -s '[.[][]]') || COMMENTS='[]'
MARKER_TIME=$(jq -r --arg author "$PR_AUTHOR" --arg marker "✅ Review round clean @ $HEAD_SHA" \
'[.[] | select(.user.login == $author) | select(.body | contains($marker)) | .created_at] | min // empty' <<<"$COMMENTS")
CODEX_VERDICT=''
if [ -n "$MARKER_TIME" ]; then
# Only Codex evidence that predates the marker counts: the ready-
# triggered Codex run itself posts after the flip and must not
# vouch for a sibling reviewer's skip.
CODEX_VERDICT=$(jq -r --arg mt "$MARKER_TIME" \
'[.[] | select(.user.login == "github-actions[bot]") | select(.body | contains("## Codex Review")) | select(.created_at < $mt)] | last | .body // ""' <<<"$COMMENTS" \
| grep -m1 -oE '(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)' || true)
fi
if [ -n "$MARKER_TIME" ] && [ -n "$CODEX_VERDICT" ] && [ "$CODEX_VERDICT" != "Should address issues before merging" ]; then
echo "Clean review round marker found for $HEAD_SHA with pre-marker non-blocking Codex verdict; skipping redundant review."
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
fi
# PR title/body are attacker-controlled free text. Use an unguessable
# per-run delimiter so a fork can't embed a fixed heredoc terminator to
# inject extra outputs — e.g. is_fork=false (last-write-wins), which
# would re-enable the EE checkout and trusted-path settings for forks.
RAND=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')
TITLE_EOF="TITLE_EOF_${RAND}"
BODY_EOF="BODY_EOF_${RAND}"
{
echo "skip=false"
echo "is_fork=$IS_FORK"
echo "pr_number=$PR_NUMBER"
echo "base_ref=$BASE_REF"
echo "base_sha=$BASE_SHA"
echo "head_sha=$HEAD_SHA"
echo "pr_author=$PR_AUTHOR"
echo "title<<$TITLE_EOF"
printf '%s\n' "$PR_TITLE"
echo "$TITLE_EOF"
echo "body<<$BODY_EOF"
printf '%s\n' "$PR_BODY"
echo "$BODY_EOF"
} >> "$GITHUB_OUTPUT"
- name: Checkout repository
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
uses: actions/checkout@v5
with:
ref: refs/pull/${{ steps.pr.outputs.pr_number }}/merge
fetch-depth: 1
# Don't persist github.token in .git/config: the review agent can read
# the checkout, and on the fork path that token (issue/PR write) would
# otherwise be exfiltratable. All later git ops target the public origin
# and need no auth; EE checkout and gh use their own explicit tokens.
persist-credentials: false
# Never expose the EE private-repo token to untrusted fork code. Skipping
# this step leaves steps.ee.outputs.available empty, so the EE checkout and
# substitution steps below are skipped too.
- name: Check EE access
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true' && steps.pr.outputs.is_fork != 'true'
id: ee
env:
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
run: |
if [ -n "$EE_TOKEN" ]; then
echo "available=true" >> "$GITHUB_OUTPUT"
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_OUTPUT"
else
echo "available=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout EE repository
if: steps.ee.outputs.available == 'true'
uses: actions/checkout@v5
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ steps.ee.outputs.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 1
- name: Substitute EE code
if: steps.ee.outputs.available == 'true'
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Set up Node.js
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Codex CLI
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
run: npm install --global @openai/codex@0.153.4
- name: Configure Codex auth
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
run: |
CODEX_HOME="$HOME/.codex"
echo "CODEX_HOME=$CODEX_HOME" >> "$GITHUB_ENV"
mkdir -p "$CODEX_HOME"
chmod 700 "$CODEX_HOME"
cat > "$CODEX_HOME/config.toml" <<'EOF'
cli_auth_credentials_store = "file"
EOF
if [ -n "$OPENAI_API_KEY" ]; then
printf '%s' "$OPENAI_API_KEY" | codex login --with-api-key
else
printf '%s' "$CODEX_AUTH_JSON" > "$CODEX_HOME/auth.json"
chmod 600 "$CODEX_HOME/auth.json"
node -e 'JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"))' "$CODEX_HOME/auth.json"
fi
- name: Pre-fetch base and head refs for the PR
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
run: |
git fetch --no-tags origin \
"$PR_BASE_REF" \
"+refs/pull/$PR_NUMBER/head"
- name: Fetch prior PR discussion
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
run: |
# Write outside the checkout: on the fork path the merge tree is
# attacker-controlled, and a committed symlink at this path would
# redirect the write.
gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" \
--jq '[.[] | {user: .user.login, created_at: .created_at, body: (.body | .[:4000])}] | sort_by(.created_at) | .[-20:]' \
> "$RUNNER_TEMP/prior-comments.json" || echo "[]" > "$RUNNER_TEMP/prior-comments.json"
- name: Write Codex review context
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
PR_REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
PR_TITLE: ${{ steps.pr.outputs.title }}
PR_BODY: ${{ steps.pr.outputs.body }}
PR_AUTHOR: ${{ steps.pr.outputs.pr_author }}
EXTRA_PROMPT: ${{ inputs.extra_prompt }}
run: |
node <<'NODE'
const fs = require('fs');
const tmp = process.env.RUNNER_TEMP;
const lines = [
`Repository: ${process.env.PR_REPOSITORY}`,
`PR number: ${process.env.PR_NUMBER}`,
];
if (process.env.PR_AUTHOR) {
lines.push(`PR AUTHOR: ${process.env.PR_AUTHOR}`);
}
lines.push(
`Base SHA: ${process.env.PR_BASE_SHA}`,
`Head SHA: ${process.env.PR_HEAD_SHA}`,
'',
'PR title:',
process.env.PR_TITLE || '(empty)',
'',
'PR body:',
process.env.PR_BODY || '(empty)',
'',
'Changed commits command:',
`git log --oneline ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
'',
'Changed files command:',
`git diff --stat ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
'',
'Full review diff command:',
`git diff --unified=0 ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`
);
if (process.env.EXTRA_PROMPT && process.env.EXTRA_PROMPT.trim()) {
lines.push('', 'Additional reviewer instructions:', process.env.EXTRA_PROMPT.trim());
}
if (fs.existsSync(`${tmp}/prior-comments.json`)) {
try {
const comments = JSON.parse(fs.readFileSync(`${tmp}/prior-comments.json`, 'utf8'));
if (Array.isArray(comments) && comments.length > 0) {
lines.push(
'',
'Prior PR discussion (most recent up to 20 comments):',
'',
'If you have already reviewed this PR (look for your own earlier "## Codex Review" comment), focus on what changed since then per the diff and respect any decisions the human made in replies. Do not re-flag findings the human already pushed back on.',
''
);
for (const c of comments) {
lines.push(`### @${c.user} (${c.created_at})`, '', c.body, '', '---', '');
}
}
} catch (_) {}
}
fs.writeFileSync(`${tmp}/pr-review-context.md`, `${lines.join('\n')}\n`);
NODE
- name: Run Codex review
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
PR_IS_FORK: ${{ steps.pr.outputs.is_fork }}
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
run: |
if [ "$PR_IS_FORK" = "true" ]; then
# Fork code is untrusted. Read the review policy/prompt from the base
# ref (git show) rather than the attacker-controlled merge checkout,
# so a malicious fork can't rewrite the reviewer's own instructions,
# and run in a network-disabled sandbox to block secret exfiltration.
git show "origin/$PR_BASE_REF:REVIEW.md" > /tmp/codex-prompt.md
git show "origin/$PR_BASE_REF:.github/codex/pr-review.prompt.md" >> /tmp/codex-prompt.md
SANDBOX_MODE=workspace-write
else
cat REVIEW.md .github/codex/pr-review.prompt.md > /tmp/codex-prompt.md
SANDBOX_MODE=danger-full-access
fi
# The context file lives in RUNNER_TEMP (outside the attacker-controlled
# checkout); tell the agent its absolute path.
printf '\nReview context file (absolute path): %s\n' "$RUNNER_TEMP/pr-review-context.md" >> /tmp/codex-prompt.md
# Write the final message outside the checkout too: a fork could commit
# codex-final-message.md as a symlink and redirect this write to overwrite
# e.g. a GitHub Action's index.js, which then runs with our credentials.
codex exec \
-C "$GITHUB_WORKSPACE" \
-m gpt-5.6-sol \
-c 'model_reasoning_effort="xhigh"' \
-s "$SANDBOX_MODE" \
-o "$RUNNER_TEMP/codex-final-message.md" \
- < /tmp/codex-prompt.md
- name: Post Codex review comment
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
uses: actions/github-script@v7
env:
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
GH_JOB_TOKEN: ${{ github.token }}
with:
github-token: ${{ github.token }}
script: |
const fs = require('fs');
const path = `${process.env.RUNNER_TEMP}/codex-final-message.md`;
if (!fs.existsSync(path)) {
core.info('Codex did not produce a final message; skipping PR comment.');
return;
}
let body = fs.readFileSync(path, 'utf8').trim();
if (!body) {
core.info('Codex final message was empty; skipping PR comment.');
return;
}
// Defense-in-depth for fork reviews: the model call needs the provider
// credential in the env, and the posted comment bypasses Actions log
// masking. Strip any credential (API key, raw auth JSON, nested
// tokens) that leaked into the review text before posting.
const secrets = [];
const addSecret = (v, min) => {
if (typeof v === 'string' && v.length >= min) secrets.push(v);
};
addSecret(process.env.OPENAI_API_KEY, 8);
addSecret(process.env.CODEX_AUTH_JSON, 8);
addSecret(process.env.GH_JOB_TOKEN, 8);
if (process.env.CODEX_AUTH_JSON) {
try {
const collect = (o) => {
if (typeof o === 'string') addSecret(o, 20);
else if (Array.isArray(o)) o.forEach(collect);
else if (o && typeof o === 'object') Object.values(o).forEach(collect);
};
collect(JSON.parse(process.env.CODEX_AUTH_JSON));
} catch (_) {}
}
for (const s of [...new Set(secrets)].sort((a, b) => b.length - a.length)) {
body = body.split(s).join('[REDACTED]');
}
body = body.trim();
if (!body) {
core.info('Codex final message was empty after redaction; skipping PR comment.');
return;
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: Number(process.env.PR_NUMBER),
body,
});
+39
View File
@@ -0,0 +1,39 @@
on:
issue_comment:
types: [created]
jobs:
check-membership:
if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') }}
uses: ./.github/workflows/check-org-membership.yml
with:
commenter: ${{ github.event.comment.user.login }}
secrets:
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
generate-token:
needs: check-membership
if: ${{ needs.check-membership.outputs.is_member == 'true' }}
runs-on: ubicloud-standard-2
outputs:
app_token: ${{ steps.app.outputs.token }}
steps:
- name: Generate an installation token
id: app
uses: actions/create-github-app-token@v2
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
owner: windmill-labs
trigger-docs:
needs: [generate-token, check-membership]
if: ${{ needs.check-membership.outputs.is_member == 'true' }}
uses: windmill-labs/windmilldocs/.github/workflows/create-docs.yml@main
with:
pr_number: ${{ github.event.issue.number }}
repo: ${{ github.event.repository.name }}
comment_text: ${{ github.event.comment.body }}
secrets:
DOCS_TOKEN: ${{ needs.generate-token.outputs.app_token }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
@@ -6,10 +6,6 @@ on:
- opened
- ready_for_review
- closed
issue_comment:
types:
- created
- edited
jobs:
notify_discord_when_pr_opened:
@@ -37,22 +33,3 @@ jobs:
PR_NUMBER: ${{ github.event.pull_request.number }}
secrets:
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
notify_discord_on_comment:
if: >
github.event_name == 'issue_comment'
&& github.event.issue.pull_request
&& github.event.comment.user.login != 'cloudflare-workers-and-pages[bot]'
&& github.event.comment.user.login != 'ellipsis-dev[bot]'
uses: ./.github/workflows/shareable-discord-notification.yml
with:
PR_STATUS: "comment"
PR_NUMBER: ${{ github.event.issue.number }}
COMMENT_BODY: ${{ github.event.comment.body }}
COMMENT_AUTHOR: ${{ github.event.comment.user.login }}
COMMENT_URL: ${{ github.event.comment.html_url }}
COMMENT_IS_EDIT: ${{ github.event.action == 'edited' }}
DISCORD_CHANNEL_ID: "1372204995868491786"
DISCORD_GUILD_ID: "930051556043276338"
secrets:
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
+1 -2
View File
@@ -67,8 +67,7 @@ jobs:
platforms: linux/amd64,linux/arm64
push: true
build-args: |
features=ce_rpi
WM_BUILD_VERSION=${{ github.sha }}
features=embedding,parquet,openidconnect,license,http_trigger,zip,oauth2,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:dev
${{ steps.meta-public.outputs.tags }}
+107 -170
View File
@@ -24,8 +24,8 @@ on:
description: "Tag the image"
required: true
default: "test"
slim:
description: "Build slim image (true, false)"
nsjail:
description: "Build nsjail image (true, false)"
required: false
default: false
type: boolean
@@ -86,32 +86,22 @@ jobs:
type=semver,pattern={{major}}.{{minor}}
- name: Build and push publicly
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
build-args: |
features=ce
WM_BUILD_VERSION=${{ github.sha }}
features=embedding,parquet,openidconnect,jemalloc,license,http_trigger,zip,oauth2,dind,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,agent_worker_server,all_languages,deno_core,mcp,private
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
${{ steps.meta-public.outputs.tags }}
labels: |
${{ steps.meta-public.outputs.labels }}
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
digest: ${{ steps.docker_build.outputs.digest }}
build_ee:
runs-on: ubicloud
if: (github.event_name != 'workflow_dispatch') || github.event.inputs.ee
if: (github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.nsjail)
steps:
- uses: actions/checkout@v4
with:
@@ -158,16 +148,13 @@ jobs:
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Build and push publicly ee
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
build-args: |
features=ee
WM_BUILD_VERSION=${{ github.sha }}
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,agent_worker_server,tantivy,license,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,otel,dind,postgres_trigger,mqtt_trigger,gcp_trigger,websocket,smtp,static_frontend,all_languages,private,deno_core,mcp
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
${{ steps.meta-ee-public.outputs.tags }}
@@ -175,12 +162,38 @@ jobs:
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee
digest: ${{ steps.docker_build.outputs.digest }}
# disabled until we make it 100% reliable and add more meaningful tests
# playwright:
# runs-on: [self-hosted, new]
# needs: [build]
# services:
# postgres:
# image: postgres
# env:
# POSTGRES_DB: windmill
# POSTGRES_USER: admin
# POSTGRES_PASSWORD: changeme
# ports:
# - 5432:5432
# options: >-
# --health-cmd pg_isready
# --health-interval 10s
# --health-timeout 5s
# --health-retries 5
# steps:
# - uses: actions/checkout@v4
# - name: "Docker"
# run: echo "::set-output name=id::$(docker run --network=host --rm -d -p 8000:8000 --privileged -it -e DATABASE_URL=postgres://admin:changeme@localhost:5432/windmill -e BASE_INTERNAL_URL=http://localhost:8000 ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest)"
# id: docker-container
# - uses: actions/setup-node@v3
# with:
# node-version: 16
# - name: "Playwright run"
# timeout-minutes: 2
# run: cd frontend && npm ci @playwright/test && npx playwright install && export BASE_URL=http://localhost:8000 && npm run test
# - name: "Clean up"
# run: docker kill ${{ steps.docker-container.outputs.id }}
# if: always()
attach_amd64_binary_to_release:
needs: [build, build_ee]
@@ -207,12 +220,6 @@ jobs:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
path: "/usr/src/app/windmill"
- uses: shrink/actions-docker-extract@v3
id: extract-duckdb-ffi-internal
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
path: "/usr/src/app/libwindmill_duckdb_ffi_internal.so"
- uses: shrink/actions-docker-extract@v3
id: extract-ee
with:
@@ -230,61 +237,6 @@ jobs:
files: |
${{ steps.extract.outputs.destination }}/*
${{ steps.extract-ee.outputs.destination }}/*
${{ steps.extract-duckdb-ffi-internal.outputs.destination }}/*
attach_ee_debug_to_release:
needs: [build_ee]
runs-on: ubicloud
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
strategy:
matrix:
platform: [linux/amd64, linux/arm64]
include:
- platform: linux/amd64
arch: amd64
- platform: linux/arm64
arch: arm64
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
- name: Substitute EE code
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- uses: depot/setup-action@v1
- name: Extract EE debug info from builder stage (depot cache hit)
uses: depot/build-push-action@v1
with:
context: .
platforms: ${{ matrix.platform }}
target: debuginfo
build-args: |
features=ee
WM_BUILD_VERSION=${{ github.sha }}
outputs: type=local,dest=./debuginfo
- name: Rename debug file with corresponding architecture
run: |
mv ./debuginfo/windmill.debug ./debuginfo/windmill-ee-${{ matrix.arch }}.debug
- name: Attach debug file to release
uses: softprops/action-gh-release@v2
with:
files: ./debuginfo/windmill-ee-${{ matrix.arch }}.debug
# attach_arm64_binary_to_release:
# needs: [build, build_ee]
@@ -376,21 +328,6 @@ jobs:
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:main
- uses: sigstore/cosign-installer@v4.1.2
if: startsWith(github.ref, 'refs/tags/v')
with:
cosign-release: "v2.6.5"
# end-to-end release guard: the version tag pushed by this run must
# verify against this exact run's identity (the mutable :latest/:dev
# tags race with concurrent main builds, so they are not asserted here)
- name: Verify release image is signed
if: startsWith(github.ref, 'refs/tags/v')
run: |
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity "https://github.com/windmill-labs/windmill/.github/workflows/docker-image.yml@${GITHUB_REF}" \
"${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${GITHUB_REF_NAME#v}"
tag_latest_ee:
runs-on: ubicloud
needs: [run_integration_test, build_ee]
@@ -412,21 +349,6 @@ jobs:
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:latest
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:main
- uses: sigstore/cosign-installer@v4.1.2
if: startsWith(github.ref, 'refs/tags/v')
with:
cosign-release: "v2.6.5"
# end-to-end release guard: the version tag pushed by this run must
# verify against this exact run's identity (the mutable :latest/:dev
# tags race with concurrent main builds, so they are not asserted here)
- name: Verify release ee image is signed
if: startsWith(github.ref, 'refs/tags/v')
run: |
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity "https://github.com/windmill-labs/windmill/.github/workflows/docker-image.yml@${GITHUB_REF}" \
"${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${GITHUB_REF_NAME#v}"
verify_ee_image_vulnerabilities:
runs-on: ubicloud
needs: [tag_latest_ee]
@@ -469,10 +391,67 @@ jobs:
# ignore-unchanged: true
# only-fixed: true
build_ee_nsjail:
needs: [build_ee]
runs-on: ubicloud
if: (github.event_name != 'pull_request') && ((github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.nsjail))
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.ref }}
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail
flavor: |
latest=false
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
type=ref,event=branch
type=ref,event=pr
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Update Dockerfile image reference
run: |
sed -i 's|FROM ghcr.io/windmill-labs/windmill-ee:dev|FROM ghcr.io/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}|' ./docker/DockerfileNsjail
cat ./docker/DockerfileNsjail | grep "FROM"
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileNsjail"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
publish_ecr_s3:
needs: [build_ee_full]
needs: [build_ee_nsjail]
runs-on: ubicloud-standard-2-arm
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
if: (github.event_name != 'pull_request') && (github.event_name !=
'workflow_dispatch')
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
@@ -491,18 +470,23 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Get version from tag
id: version
run: echo "VERSION=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"
- name: get git hash
if: github.event_name != 'pull_request'
id: git_hash
run: |
git_hash=$(git rev-parse --short "$GITHUB_SHA")
echo "GIT_HASH=${git_hash:0:7}" >> "$GITHUB_OUTPUT"
- uses: shrink/actions-docker-extract@v3
if: github.event_name != 'pull_request'
id: extract
with:
image: |-
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-full:${{ steps.version.outputs.VERSION }}
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail:${{ steps.git_hash.outputs.GIT_HASH }}
path: "/static_frontend/."
- uses: reggionick/s3-deploy@v4
if: github.event_name != 'pull_request'
with:
folder: ${{ steps.extract.outputs.destination }}
bucket: windmill-frontend
@@ -541,13 +525,11 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
file: "./docker/DockerfileCuda"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
@@ -555,13 +537,6 @@ jobs:
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-cuda
digest: ${{ steps.docker_build.outputs.digest }}
build_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build]
@@ -594,31 +569,21 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
file: "./docker/DockerfileSlim"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-slim
digest: ${{ steps.docker_build.outputs.digest }}
build_ee_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build_ee]
runs-on: ubicloud
if: (github.event_name != 'pull_request') && ((github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.slim))
steps:
- uses: actions/checkout@v4
with:
@@ -636,7 +601,6 @@ jobs:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
@@ -648,13 +612,11 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
file: "./docker/DockerfileSlimEe"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
@@ -662,13 +624,6 @@ jobs:
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
digest: ${{ steps.docker_build.outputs.digest }}
build_full:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build]
@@ -701,26 +656,17 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
file: "./docker/DockerfileFull"
tags: |
${{ steps.meta-public.outputs.tags }}
labels: |
${{ steps.meta-public.outputs.labels }}
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-full
digest: ${{ steps.docker_build.outputs.digest }}
build_ee_full:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build_ee]
@@ -753,23 +699,14 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
id: docker_build
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
sbom: ${{ startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push' }}
file: "./docker/DockerfileFullEe"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Sign and attest release image
if: startsWith(github.ref, 'refs/tags/v') && github.event_name == 'push'
uses: ./.github/actions/sign-attest-image
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-full
digest: ${{ steps.docker_build.outputs.digest }}
+4 -8
View File
@@ -16,15 +16,11 @@ jobs:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v5
- uses: actions/setup-node@v3
with:
node-version: 24
cache: "npm"
cache-dependency-path: "frontend/package-lock.json"
node-version: 18
- name: "npm check"
timeout-minutes: 5
env:
# svelte-check peaks past node's ~4GB default ceiling on this runner and aborts.
NODE_OPTIONS: --max-old-space-size=8192
run: cd frontend && npm ci && npm run generate-backend-client && npm run
run:
cd frontend && npm ci && npm run generate-backend-client && npm run
check
-355
View File
@@ -1,355 +0,0 @@
name: Git commands
on:
issue_comment:
types: [created]
jobs:
# /command comments can come from anyone; author_association misses private org
# members, so check-access resolves them via the internal app token. Runs once and is
# OR'd into each job's guard (public members still pass on author_association alone).
check-access:
if: github.event.issue.pull_request != null && startsWith(github.event.comment.body, '/')
uses: ./.github/workflows/check-write-access.yml
with:
username: ${{ github.event.comment.user.login }}
secrets: inherit
update-sqlx:
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/updatesqlx')
runs-on: ubicloud-standard-8
permissions:
contents: write
pull-requests: write
issues: write
services:
postgres:
image: postgres:16
env:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: windmill
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/create-github-app-token@v2
id: app
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
- name: Comment on PR - Starting
uses: actions/github-script@v6
with:
github-token: ${{ steps.app.outputs.token }}
script: |
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `Starting sqlx update...\n\n[View workflow run](${runUrl})`
})
- name: Checkout repository
uses: actions/checkout@v3
with:
token: ${{ steps.app.outputs.token }}
ref: ${{ github.event.issue.pull_request.head.ref }}
fetch-depth: 0
- name: Checkout windmill-ee-private
uses: actions/checkout@v3
with:
repository: windmill-labs/windmill-ee-private
path: windmill-ee-private
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
# Setup Rust toolchain
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
- name: Install xmlsec and gssapi build-time deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
pkg-config libxml2-dev libssl-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang \
xmlsec1 libxmlsec1-dev libxmlsec1-openssl
- name: Run update-sqlx script
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/windmill
GH_TOKEN: ${{ steps.app.outputs.token }}
run: |
set -e # Exit on any command failure
PR_NUMBER=${{ github.event.issue.number }}
# Set up error trap to comment on PR for any failure
trap 'gh pr comment $PR_NUMBER --body "❌ SQLx update failed. Please check the workflow logs for details."' ERR
BRANCH_NAME=$(gh pr view $PR_NUMBER --json headRefName --jq .headRefName)
echo "Checking out PR branch: $BRANCH_NAME"
git checkout $BRANCH_NAME
git config --local user.email "windmill-internal-app[bot]@users.noreply.github.com"
git config --local user.name "windmill-internal-app[bot]"
git config pull.rebase true
git pull origin $BRANCH_NAME
# Checkout the correct windmill-ee-private commit from ee-repo-ref.txt
if [ -f backend/ee-repo-ref.txt ]; then
EE_REF=$(cat backend/ee-repo-ref.txt | tr -d '[:space:]')
echo "Checking out windmill-ee-private at commit: $EE_REF"
cd windmill-ee-private
git fetch origin $EE_REF
git checkout $EE_REF
cd ..
else
echo "Warning: ee-repo-ref.txt not found, using default branch"
fi
mkdir -p frontend/build
cd backend
cargo install sqlx-cli --version 0.8.5
sqlx migrate run
./substitute_ee_code.sh --dir ./windmill-ee-private
./update_sqlx.sh
# Pass the branch name to the next step
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
- name: Commit changes if any
run: |
git add backend/.sqlx
git commit -m "Update SQLx metadata"
git push origin ${{ env.BRANCH_NAME }}
- name: Comment on PR - Completed
uses: actions/github-script@v6
with:
github-token: ${{ steps.app.outputs.token }}
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'Successfully ran sqlx update'
})
demo:
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/demo')
runs-on: ubicloud-standard-2
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run Claude Code for Demo Generation
uses: anthropics/claude-code-action@beta
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
timeout_minutes: "10"
allowed_tools: "Bash"
direct_prompt: |
You need to:
1. Extract the Cloudflare preview URL from the cloudflare-workers-and-pages bot comment in this PR
2. Analyze the PR changes to understand what feature was added/modified
3. Create detailed instructions to give to an AI agent that will click and interact with buttons and inputs to showcase the new feature. Only include the instructions, nothing else.
4. Create a demo.json file with a valid JSON object containing:
- instructions: the demo instructions
- url: the preview URL
5. VALIDATE the JSON file using `jq` before finishing
DO NOT COMMIT THIS FILE TO THE PR.
Example demo.json:
{
"instructions": "Click on settings, then account settings, then 'generate new token'",
"url": "https://example.pages.dev"
}
CRITICAL: After creating demo.json, you MUST:
1. Run `jq empty demo.json` to validate the JSON is properly formatted
2. If validation fails, fix the JSON and validate again
3. Only proceed once the JSON passes validation
4. Use proper JSON escaping for newlines, quotes, and special characters
Make sure to:
- Create a valid JSON object that passes `jq empty demo.json`
- Extract the correct preview URL (should be a .pages.dev domain)
- Create specific, actionable demo steps based on the actual changes in the PR
- Properly escape all strings in the JSON (use jq to create the file if needed)
- NOT COMMIT THE DEMO.JSON FILE TO THE PR
- name: Send instructions to Windmill
env:
DEMO_WEBHOOK_TOKEN: ${{ secrets.DEMO_WEBHOOK_TOKEN }}
run: |
if [[ -f "demo.json" ]]; then
echo "Found demo.json, sending to Windmill..."
cat demo.json
# Validate JSON one more time (Claude should have already done this)
if ! jq empty demo.json; then
echo "Error: demo.json is not valid JSON"
exit 1
fi
RESULT=$(curl -s \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $DEMO_WEBHOOK_TOKEN" \
-X POST \
-d @demo.json \
'https://app.windmill.dev/api/w/windmill-labs/jobs/run/f/f/ai/browserbase_demo')
echo "Windmill response:"
echo -E "$RESULT"
else
echo "Error: demo.json file not found"
exit 1
fi
update-ee-ref:
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/eeref')
runs-on: ubicloud-standard-2
permissions:
contents: write
pull-requests: write
issues: write
steps:
- uses: actions/create-github-app-token@v2
id: app
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
- name: Comment on PR - Starting
uses: actions/github-script@v6
with:
github-token: ${{ steps.app.outputs.token }}
script: |
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `Starting ee ref update...\n\n[View workflow run](${runUrl})`
})
- name: Checkout repository
uses: actions/checkout@v3
with:
token: ${{ steps.app.outputs.token }}
ref: ${{ github.event.issue.pull_request.head.ref }}
fetch-depth: 0
- name: Checkout windmill-ee-private
uses: actions/checkout@v3
with:
repository: windmill-labs/windmill-ee-private
path: windmill-ee-private
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
- name: Get last commit hash of private-repo
id: get-commit-hash
run: |
cd windmill-ee-private
COMMIT_HASH=$(git rev-parse HEAD)
echo "commit_hash=$COMMIT_HASH" >> $GITHUB_OUTPUT
echo "Latest commit hash: $COMMIT_HASH"
- name: Update ee-repo-ref.txt
env:
GH_TOKEN: ${{ steps.app.outputs.token }}
run: |
set -e # Exit on any command failure
PR_NUMBER=${{ github.event.issue.number }}
# Set up error trap to comment on PR for any failure
trap 'gh pr comment $PR_NUMBER --body "❌ EE ref update failed. Please check the workflow logs for details."' ERR
BRANCH_NAME=$(gh pr view $PR_NUMBER --json headRefName --jq .headRefName)
echo "Checking out PR branch: $BRANCH_NAME"
git checkout $BRANCH_NAME
git config --local user.email "windmill-internal-app[bot]@users.noreply.github.com"
git config --local user.name "windmill-internal-app[bot]"
git config pull.rebase true
git pull origin $BRANCH_NAME
echo "${{ steps.get-commit-hash.outputs.commit_hash }}" > backend/ee-repo-ref.txt
echo "Updated backend/ee-repo-ref.txt with commit hash: ${{ steps.get-commit-hash.outputs.commit_hash }}"
# commit and push the changes
git add backend/ee-repo-ref.txt
git commit -m "Update ee-repo-ref.txt" || echo "No changes to commit"
git push origin $BRANCH_NAME
- name: Comment on PR - Completed
uses: actions/github-script@v6
with:
github-token: ${{ steps.app.outputs.token }}
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'Successfully updated ee-repo-ref.txt'
})
update-docs:
needs: [check-access]
if: >-
github.event.issue.pull_request &&
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
startsWith(github.event.comment.body, '/docs')
runs-on: ubicloud-standard-2
permissions:
contents: read
pull-requests: read
issues: read
steps:
- uses: actions/create-github-app-token@v2
id: app
with:
app-id: ${{ vars.INTERNAL_APP_ID }}
private-key: ${{ secrets.INTERNAL_APP_KEY }}
owner: ${{ github.repository_owner }}
repositories: |
windmilldocs
- name: Trigger docs update
env:
GH_TOKEN: ${{ steps.app.outputs.token }}
COMMENT_TEXT: ${{ github.event.comment.body }}
run: |
jq -n \
--argjson pr_number ${{ github.event.issue.number }} \
--arg repo "${{ github.event.repository.name }}" \
--arg comment "$COMMENT_TEXT" \
'{event_type: "create-docs", client_payload: {pr_number: $pr_number, repo: $repo, comment_text: $comment}}' | \
gh api repos/windmill-labs/windmilldocs/dispatches \
--method POST \
--input -
-224
View File
@@ -1,224 +0,0 @@
name: Git Sync Integration Tests
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "backend/windmill-git-sync/**"
- "backend/windmill-api-integration-tests/tests/git_sync*"
- "backend/ee-repo-ref.txt"
- "backend/windmill-common/src/workspaces.rs"
- "frontend/src/lib/hubPaths.json"
- "backend/windmill-worker/src/result_processor.rs"
- "backend/windmill-api-workspaces/**"
- "cli/src/commands/sync/**"
- "cli/src/utils/git.ts"
- "integration_tests/test/git_sync_test.py"
- ".github/workflows/git-sync-test.yml"
pull_request:
types: [opened, synchronize, reopened]
paths:
- "backend/windmill-git-sync/**"
- "backend/windmill-api-integration-tests/tests/git_sync*"
- "backend/ee-repo-ref.txt"
- "backend/windmill-common/src/workspaces.rs"
- "frontend/src/lib/hubPaths.json"
- "backend/windmill-worker/src/result_processor.rs"
- "backend/windmill-api-workspaces/**"
- "cli/src/commands/sync/**"
- "cli/src/utils/git.ts"
- "integration_tests/test/git_sync_test.py"
- ".github/workflows/git-sync-test.yml"
concurrency:
group: git-sync-test-${{ github.ref }}
cancel-in-progress: true
jobs:
check-relevance:
runs-on: ubuntu-latest
outputs:
should_run: ${{ steps.check.outputs.should_run }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check if git sync related files changed
id: check
env:
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
BASE=${{ github.event.pull_request.base.sha }}
else
BASE=${{ github.event.before }}
fi
CHANGED_FILES=$(git diff --name-only "$BASE"..HEAD 2>/dev/null || echo "")
echo "Changed files:"
echo "$CHANGED_FILES"
# Direct git sync file changes — always relevant.
if echo "$CHANGED_FILES" | grep -qE '^(backend/windmill-git-sync/|backend/windmill-worker/src/result_processor\.rs|backend/windmill-api-workspaces/|backend/windmill-api-integration-tests/tests/git_sync|backend/windmill-common/src/workspaces\.rs|frontend/src/lib/hubPaths\.json|cli/src/commands/sync/|cli/src/utils/git\.ts|integration_tests/test/git_sync|\.github/workflows/git-sync-test\.yml)'; then
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Relevant: direct git sync file changes"
exit 0
fi
# If ee-repo-ref.txt changed, check if the EE diff touches windmill-git-sync/
if echo "$CHANGED_FILES" | grep -q '^backend/ee-repo-ref.txt$'; then
NEW_REF=$(cat backend/ee-repo-ref.txt)
OLD_REF=$(git show "$BASE:backend/ee-repo-ref.txt" 2>/dev/null || echo "")
if [ -n "$OLD_REF" ] && [ "$OLD_REF" != "$NEW_REF" ]; then
# Clone EE repo and check diff
git clone --bare "https://x-access-token:${WINDMILL_EE_PRIVATE_ACCESS}@github.com/windmill-labs/windmill-ee-private.git" /tmp/ee-repo 2>/dev/null
EE_CHANGED=$(git -C /tmp/ee-repo diff --name-only "$OLD_REF".."$NEW_REF" 2>/dev/null || echo "")
echo "EE changed files:"
echo "$EE_CHANGED"
if echo "$EE_CHANGED" | grep -q '^windmill-git-sync/'; then
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Relevant: EE git sync files changed"
exit 0
fi
fi
fi
echo "should_run=false" >> "$GITHUB_OUTPUT"
echo "No git sync relevant changes detected, skipping tests"
git_sync_e2e:
needs: [check-relevance]
if: needs.check-relevance.outputs.should_run == 'true'
runs-on: ubicloud-standard-16
services:
postgres:
image: postgres:14
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Substitute EE code
run: |
cd backend && ./substitute_ee_code.sh --copy --dir ./windmill-ee-private
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.97.0
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.0
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- uses: actions/setup-node@v4
with:
node-version: "20"
- name: Install wmill CLI
run: |
cd cli && bash gen_wm_client.sh && bun install
mkdir -p "$HOME/.local/bin"
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
chmod +x "$HOME/.local/bin/wmill"
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Build Windmill
working-directory: ./backend
env:
SQLX_OFFLINE: true
CARGO_BUILD_JOBS: 12
RUSTFLAGS: ""
run: |
cargo build --features enterprise,private,license,zip
- name: Start Gitea
run: |
docker run -d --name gitea \
-e GITEA__database__DB_TYPE=sqlite3 \
-e GITEA__security__INSTALL_LOCK=true \
-e GITEA__server__HTTP_PORT=3000 \
-e GITEA__server__ROOT_URL=http://localhost:3000 \
-e GITEA__service__DISABLE_REGISTRATION=false \
-p 3000:3000 \
gitea/gitea:1.22-rootless
echo "Waiting for Gitea to be ready..."
for i in $(seq 1 30); do
if curl -sf http://localhost:3000/api/v1/version > /dev/null 2>&1; then
echo "Gitea is ready"
break
fi
sleep 2
done
curl -sf http://localhost:3000/api/v1/version > /dev/null || { echo "Gitea failed to start"; exit 1; }
- name: Start Windmill
working-directory: ./backend
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
DENO_PATH: deno
BUN_PATH: bun
NODE_BIN_PATH: node
# The auto-pull poller's SSRF guard rejects localhost git remotes;
# the tests' Gitea runs on localhost.
ALLOW_LOCAL_GIT_REMOTES: "true"
run: |
./target/debug/windmill &
echo "Waiting for Windmill to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
echo "Windmill is ready"
break
fi
sleep 2
done
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; exit 1; }
- name: Run git sync E2E tests
timeout-minutes: 10
env:
GITEA_DOCKER_URL: http://localhost:3000
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
run: |
python3 -m venv .venv
.venv/bin/pip install -r integration_tests/requirements.txt
cd integration_tests && ../.venv/bin/python -m unittest -v test.git_sync_test
- name: Archive logs
uses: actions/upload-artifact@v4
if: always()
with:
name: Git Sync Integration Tests Logs
path: |
integration_tests/logs
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
go build
- name: Pushes to another repository
id: push_directory
uses: cpina/github-action-push-to-another-repository@55306faa4ed53b815ae49e564af8cfb359d32ae2 # v1.7.3
uses: cpina/github-action-push-to-another-repository@devel
env:
API_TOKEN_GITHUB: ${{ secrets.DENO_PAT }}
with:
+4 -4
View File
@@ -14,7 +14,7 @@ jobs:
with:
node-version: "20.x"
registry-url: "https://registry.npmjs.org"
- run: cd typescript-client && ./publish.sh --access public && cd ..
- run: cd typescript-client && ./publish.sh && cd ..
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
publish_cli:
@@ -25,9 +25,9 @@ jobs:
with:
node-version: "20.x"
registry-url: "https://registry.npmjs.org"
- uses: oven-sh/setup-bun@v2
- uses: denoland/setup-deno@v2
with:
bun-version: latest
- run: cd cli && ./build.sh && cd npm && npm publish --access public
deno-version: v2.x
- run: cd cli && ./build.sh && cd npm && npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
-457
View File
@@ -1,457 +0,0 @@
name: Pi Auto Review
on:
pull_request:
types: [ready_for_review, opened, synchronize]
workflow_call:
inputs:
pr_number:
description: 'PR number to review'
required: true
type: number
extra_prompt:
description: 'Additional reviewer instructions appended to the standard review prompt'
required: false
type: string
default: ''
triggered_by:
description: 'GitHub username that triggered this review (for audit only)'
required: false
type: string
default: ''
secrets:
DEEPSEEK_API_KEY:
required: false
WINDMILL_EE_PRIVATE_ACCESS:
required: false
concurrency:
group: pi-review-${{ inputs.pr_number || github.event.pull_request.number }}
cancel-in-progress: true
jobs:
pi-review:
runs-on: ubicloud-standard-2
timeout-minutes: 30
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
# access to this repo, so fork==false already enforces write access. Do NOT re-add
# an author_association gate: the pull_request webhook payload reports private org
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
# skips auto-review for every private member.
if: |
github.event_name == 'workflow_call' ||
(
github.event.pull_request.draft == false &&
github.event.pull_request.head.repo.fork == false
)
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Check Pi configuration
id: pi_config
env:
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
run: |
if [ -n "$DEEPSEEK_API_KEY" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
else
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "DEEPSEEK_API_KEY is not configured; skipping Pi review."
fi
- name: Resolve PR metadata
if: steps.pi_config.outputs.enabled == 'true'
id: pr
env:
GH_TOKEN: ${{ github.token }}
INPUT_PR_NUMBER: ${{ inputs.pr_number }}
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
EVENT_BASE_SHA: ${{ github.event.pull_request.base.sha }}
EVENT_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
EVENT_TITLE: ${{ github.event.pull_request.title }}
EVENT_BODY: ${{ github.event.pull_request.body }}
EVENT_FORK: ${{ github.event.pull_request.head.repo.fork }}
EVENT_AUTHOR: ${{ github.event.pull_request.user.login }}
EVENT_ACTION: ${{ github.event.action }}
run: |
if [ -n "$INPUT_PR_NUMBER" ]; then
PR_JSON=$(gh pr view "$INPUT_PR_NUMBER" --repo "${{ github.repository }}" \
--json number,baseRefName,baseRefOid,headRefOid,title,body,isCrossRepository,author)
PR_NUMBER=$(echo "$PR_JSON" | jq -r '.number')
BASE_REF=$(echo "$PR_JSON" | jq -r '.baseRefName')
BASE_SHA=$(echo "$PR_JSON" | jq -r '.baseRefOid')
HEAD_SHA=$(echo "$PR_JSON" | jq -r '.headRefOid')
PR_TITLE=$(echo "$PR_JSON" | jq -r '.title')
PR_BODY=$(echo "$PR_JSON" | jq -r '.body // ""')
IS_FORK=$(echo "$PR_JSON" | jq -r '.isCrossRepository')
PR_AUTHOR=$(echo "$PR_JSON" | jq -r '.author.login // ""')
else
PR_NUMBER="$EVENT_PR_NUMBER"
BASE_REF="$EVENT_BASE_REF"
BASE_SHA="$EVENT_BASE_SHA"
HEAD_SHA="$EVENT_HEAD_SHA"
PR_TITLE="$EVENT_TITLE"
PR_BODY="$EVENT_BODY"
IS_FORK="$EVENT_FORK"
PR_AUTHOR="$EVENT_AUTHOR"
fi
# Fork PRs run untrusted code with secrets present, so the automatic
# pull_request trigger never reviews them. A non-empty INPUT_PR_NUMBER
# means we arrived via workflow_call (a maintainer /pi comment gated by
# check-write-access), so allow forks only on that path.
if [ "$IS_FORK" = "true" ] && [ -z "$INPUT_PR_NUMBER" ]; then
echo "Skipping Pi review for fork PR (automatic trigger)."
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# An agent-driven PR flips to ready only after a clean /review round on
# a draft, marked by an author comment naming the head SHA (pr skill,
# "Review rounds"). Re-reviewing that same head on ready_for_review is
# redundant. The marker alone is author attestation, so also require
# reviewer evidence: a Codex review (posted by github-actions[bot], not
# forgeable by the author) that predates the marker and carries a
# non-blocking verdict. Comment-triggered and synchronize runs never
# skip. Keep the three copies of this check in sync (pr-ready-review /
# codex-pr-review / pi-pr-review); a shared local action would need the
# repo checked out before the check, which the fork paths here
# deliberately avoid.
if [ "$EVENT_ACTION" = "ready_for_review" ] && [ -z "$INPUT_PR_NUMBER" ]; then
# Fetch failures fail open (no skip): an API hiccup must run the
# review, never skip it or fail the job.
COMMENTS=$(gh api "repos/${{ github.repository }}/issues/$PR_NUMBER/comments?per_page=100" --paginate | jq -s '[.[][]]') || COMMENTS='[]'
MARKER_TIME=$(jq -r --arg author "$PR_AUTHOR" --arg marker "✅ Review round clean @ $HEAD_SHA" \
'[.[] | select(.user.login == $author) | select(.body | contains($marker)) | .created_at] | min // empty' <<<"$COMMENTS")
CODEX_VERDICT=''
if [ -n "$MARKER_TIME" ]; then
# Only Codex evidence that predates the marker counts: the ready-
# triggered Codex run itself posts after the flip and must not
# vouch for a sibling reviewer's skip.
CODEX_VERDICT=$(jq -r --arg mt "$MARKER_TIME" \
'[.[] | select(.user.login == "github-actions[bot]") | select(.body | contains("## Codex Review")) | select(.created_at < $mt)] | last | .body // ""' <<<"$COMMENTS" \
| grep -m1 -oE '(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)' || true)
fi
if [ -n "$MARKER_TIME" ] && [ -n "$CODEX_VERDICT" ] && [ "$CODEX_VERDICT" != "Should address issues before merging" ]; then
echo "Clean review round marker found for $HEAD_SHA with pre-marker non-blocking Codex verdict; skipping redundant review."
echo "skip=true" >> "$GITHUB_OUTPUT"
exit 0
fi
fi
# PR title/body are attacker-controlled free text. Use an unguessable
# per-run delimiter so a fork can't embed a fixed heredoc terminator to
# inject extra outputs — e.g. is_fork=false (last-write-wins), which
# would re-enable the EE checkout and trusted-path settings for forks.
RAND=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')
TITLE_EOF="TITLE_EOF_${RAND}"
BODY_EOF="BODY_EOF_${RAND}"
{
echo "skip=false"
echo "is_fork=$IS_FORK"
echo "pr_number=$PR_NUMBER"
echo "base_ref=$BASE_REF"
echo "base_sha=$BASE_SHA"
echo "head_sha=$HEAD_SHA"
echo "pr_author=$PR_AUTHOR"
echo "title<<$TITLE_EOF"
printf '%s\n' "$PR_TITLE"
echo "$TITLE_EOF"
echo "body<<$BODY_EOF"
printf '%s\n' "$PR_BODY"
echo "$BODY_EOF"
} >> "$GITHUB_OUTPUT"
- name: Checkout repository
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
uses: actions/checkout@v5
with:
ref: refs/pull/${{ steps.pr.outputs.pr_number }}/merge
fetch-depth: 1
# Don't persist github.token in .git/config: the review agent can read
# the checkout, and on the fork path that token (issue/PR write) would
# otherwise be exfiltratable. All later git ops target the public origin
# and need no auth; EE checkout and gh use their own explicit tokens.
persist-credentials: false
# Never expose the EE private-repo token to untrusted fork code. Skipping
# this step leaves steps.ee.outputs.available empty, so the EE checkout and
# substitution steps below are skipped too.
- name: Check EE access
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true' && steps.pr.outputs.is_fork != 'true'
id: ee
env:
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
run: |
if [ -n "$EE_TOKEN" ]; then
echo "available=true" >> "$GITHUB_OUTPUT"
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_OUTPUT"
else
echo "available=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout EE repository
if: steps.ee.outputs.available == 'true'
uses: actions/checkout@v5
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ steps.ee.outputs.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 1
- name: Substitute EE code
if: steps.ee.outputs.available == 'true'
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Set up Node.js
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Pi CLI
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
# Pinned: this job holds DEEPSEEK_API_KEY and PR write access, and an
# unpinned reviewer also makes verdicts non-reproducible across runs.
run: npm install --global @earendil-works/pi-coding-agent@0.84.1
- name: Pre-fetch base and head refs for the PR
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
run: |
git fetch --no-tags origin \
"$PR_BASE_REF" \
"+refs/pull/$PR_NUMBER/head"
- name: Fetch prior PR discussion
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
run: |
# Write outside the checkout: on the fork path the merge tree is
# attacker-controlled, and a committed symlink at this path would
# redirect the write.
gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" \
--jq '[.[] | {user: .user.login, created_at: .created_at, body: (.body | .[:4000])}] | sort_by(.created_at) | .[-20:]' \
> "$RUNNER_TEMP/prior-comments.json" || echo "[]" > "$RUNNER_TEMP/prior-comments.json"
- name: Write Pi review context
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
PR_REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
PR_TITLE: ${{ steps.pr.outputs.title }}
PR_BODY: ${{ steps.pr.outputs.body }}
PR_AUTHOR: ${{ steps.pr.outputs.pr_author }}
EXTRA_PROMPT: ${{ inputs.extra_prompt }}
run: |
node <<'NODE'
const fs = require('fs');
const tmp = process.env.RUNNER_TEMP;
const lines = [
`Repository: ${process.env.PR_REPOSITORY}`,
`PR number: ${process.env.PR_NUMBER}`,
];
if (process.env.PR_AUTHOR) {
lines.push(`PR AUTHOR: ${process.env.PR_AUTHOR}`);
}
lines.push(
`Base SHA: ${process.env.PR_BASE_SHA}`,
`Head SHA: ${process.env.PR_HEAD_SHA}`,
'',
'PR title:',
process.env.PR_TITLE || '(empty)',
'',
'PR body:',
process.env.PR_BODY || '(empty)',
'',
'Changed commits command:',
`git log --oneline ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
'',
'Changed files command:',
`git diff --stat ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
'',
'Full review diff command:',
`git diff --unified=0 ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`
);
if (process.env.EXTRA_PROMPT && process.env.EXTRA_PROMPT.trim()) {
lines.push('', 'Additional reviewer instructions:', process.env.EXTRA_PROMPT.trim());
}
if (fs.existsSync(`${tmp}/prior-comments.json`)) {
try {
const comments = JSON.parse(fs.readFileSync(`${tmp}/prior-comments.json`, 'utf8'));
if (Array.isArray(comments) && comments.length > 0) {
lines.push(
'',
'Prior PR discussion (most recent up to 20 comments):',
'',
'If you have already reviewed this PR (look for your own earlier "## Pi Review (DeepSeek V4)" comment), focus on what changed since then per the diff and respect any decisions the human made in replies. Do not re-flag findings the human already pushed back on.',
''
);
for (const c of comments) {
lines.push(`### @${c.user} (${c.created_at})`, '', c.body, '', '---', '');
}
}
} catch (_) {}
}
fs.writeFileSync(`${tmp}/pr-review-context.md`, `${lines.join('\n')}\n`);
NODE
- name: Run Pi review
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
env:
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
PI_SKIP_VERSION_CHECK: '1'
PR_IS_FORK: ${{ steps.pr.outputs.is_fork }}
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
run: |
set -o pipefail
PI_HARDEN_FLAGS=()
# Keep generated files (final message, events, context) outside the
# checkout: on the fork path a committed symlink at any of these paths
# would redirect our write and could overwrite an action's code that
# then runs with our credentials. RUNNER_TEMP is outside the checkout.
OUT_DIR="$RUNNER_TEMP"
CTX="$RUNNER_TEMP/pr-review-context.md"
if [ "$PR_IS_FORK" = "true" ]; then
# Fork code is untrusted. Read the review policy/prompt from the base
# ref (git show) rather than the attacker-controlled merge checkout,
# so a malicious fork can't rewrite the reviewer's own instructions,
# and drop the bash tool so the agent has no shell to exfiltrate with.
git show "origin/$PR_BASE_REF:REVIEW.md" > /tmp/pi-prompt.md
git show "origin/$PR_BASE_REF:.github/pi/pr-review.prompt.md" >> /tmp/pi-prompt.md
PI_TOOLS=read,grep,find,ls
# The agent has no shell, so pre-compute the diff (base...head SHAs are
# trusted) into the context file it reads. It may still read fork files
# by absolute path for extra context — reads are safe.
{
echo ""
echo "## Pre-computed review diff (base...head)"
echo "You have no shell. The full diff is below. The repository checkout"
echo "is at $GITHUB_WORKSPACE — you may read files there by absolute path."
echo '```diff'
git -C "$GITHUB_WORKSPACE" diff --unified=0 "$PR_BASE_SHA...$PR_HEAD_SHA"
echo '```'
} >> "$CTX"
# Pi resolves ALL project config from <cwd>/.pi (settings/packages,
# extensions, skills, themes, prompts, SYSTEM.md); inside the fork
# checkout a fork could inject any to run code or rewrite our system
# prompt. Discovery is cwd-based, so run from a fresh empty dir.
PI_WORKDIR=$(mktemp -d)
cd "$PI_WORKDIR"
# Belt-and-suspenders on top of the isolated cwd: refuse discovery of
# extensions/skills/templates/themes/context-files, and PI_OFFLINE=1 to
# block any startup network op or package install. PI_OFFLINE gates only
# startup network ops, not the provider inference call.
PI_HARDEN_FLAGS=(--no-extensions --no-skills --no-prompt-templates --no-themes --no-context-files)
export PI_OFFLINE=1
else
cat REVIEW.md .github/pi/pr-review.prompt.md > /tmp/pi-prompt.md
PI_TOOLS=read,grep,find,ls,bash
fi
# The context file lives in RUNNER_TEMP (outside the checkout); tell the
# agent its absolute path.
printf '\nReview context file (absolute path): %s\n' "$CTX" >> /tmp/pi-prompt.md
# DeepSeek's reasoning_effort accepts low/high/max and silently maps both
# medium and xhigh onto high. Set the level explicitly rather than letting
# pi's default clamp onto it, so a change to either the default or the
# clamping is a visible diff here instead of a silent shift in review depth.
pi -p \
--provider deepseek \
--model deepseek-v4-pro \
--thinking high \
--tools "$PI_TOOLS" \
"${PI_HARDEN_FLAGS[@]}" \
--mode json \
< /tmp/pi-prompt.md \
| tee "$OUT_DIR/pi-events.jsonl" \
| jq -rc --unbuffered '
if .type == "agent_start" then "🤖 pi agent started"
elif .type == "turn_start" then "── turn ──"
elif .type == "message_end" then
"[\(.message.role)] " + (
(.message.content // [])
| map(
if .type == "text" then "text(\(.text | length)c)"
elif .type == "tool_use" then "🔧 \(.name) \(.input | @json | .[:160])"
elif .type == "tool_result" then "✅ result"
else .type
end
)
| join(" | ")
)
elif .type == "turn_end" then "── turn done (\((.toolResults // []) | length) tool result(s)) ──"
elif .type == "agent_end" then "🏁 pi agent done"
else empty
end
'
jq -r '
select(.type == "agent_end")
| .messages
| map(select(.role == "assistant"))
| last
| (.content[]? | select(.type == "text") | .text)
' "$OUT_DIR/pi-events.jsonl" > "$OUT_DIR/pi-final-message.md"
# The final message often opens with chatter ("Now I have all the context
# I need..."), which would land above the verdict in the posted comment.
# Keep the trim conditional: without the heading there is nothing to cut
# and the range expression would empty the file.
if grep -q '^## Pi Review' "$OUT_DIR/pi-final-message.md"; then
sed -i -n '/^## Pi Review/,$p' "$OUT_DIR/pi-final-message.md"
fi
- name: Post Pi review comment
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
uses: actions/github-script@v7
env:
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
GH_JOB_TOKEN: ${{ github.token }}
with:
github-token: ${{ github.token }}
script: |
const fs = require('fs');
const path = `${process.env.RUNNER_TEMP}/pi-final-message.md`;
if (!fs.existsSync(path)) {
core.info('Pi did not produce a final message; skipping PR comment.');
return;
}
let body = fs.readFileSync(path, 'utf8').trim();
if (!body) {
core.info('Pi final message was empty; skipping PR comment.');
return;
}
// Defense-in-depth for fork reviews: the model call needs the provider
// credential in the environment (readable via /proc/self/environ), and
// the posted comment is an exfiltration channel that bypasses GitHub
// Actions log masking. Strip the credential if it leaked into the text.
for (const s of [process.env.DEEPSEEK_API_KEY, process.env.GH_JOB_TOKEN]) {
if (typeof s === 'string' && s.length >= 8) {
body = body.split(s).join('[REDACTED]');
}
}
body = body.trim();
if (!body) {
core.info('Pi final message was empty after redaction; skipping PR comment.');
return;
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: Number(process.env.PR_NUMBER),
body,
});

Some files were not shown because too many files have changed in this diff Show More