mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-14 16:02:27 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ca72c97a32 | ||
|
|
9a5baa8e53 | ||
|
|
0e3d835609 | ||
|
|
7a8a0bb163 | ||
|
|
deff3b0045 | ||
|
|
1b885ce2c2 | ||
|
|
cec82a9cd2 | ||
|
|
8a792c9c9d | ||
|
|
0ea405415e | ||
|
|
d598731913 | ||
|
|
5a96d64183 | ||
|
|
a017459e12 | ||
|
|
eff3e1c43a | ||
|
|
844acbf00b | ||
|
|
e0040fe3b3 | ||
|
|
d0bad936f0 | ||
|
|
63340e6ce2 | ||
|
|
c839bae160 | ||
|
|
253452013c | ||
|
|
af70339d07 | ||
|
|
a01c26f0e2 | ||
|
|
b1bcdae01d | ||
|
|
b1a4f2ab50 | ||
|
|
1ad54e887e | ||
|
|
a446660345 | ||
|
|
c21182f502 | ||
|
|
54698b3eef | ||
|
|
59e8004f99 | ||
|
|
1271a65b5a | ||
|
|
414b9c338b | ||
|
|
53909750ef | ||
|
|
60e7fa764c | ||
|
|
a0c86ef4c7 | ||
|
|
5805554ac8 | ||
|
|
bd7607d162 | ||
|
|
7985a7bafa | ||
|
|
13896a5741 | ||
|
|
e881ed5e00 | ||
|
|
d6a654aaff |
@@ -1,282 +0,0 @@
|
||||
---
|
||||
name: adding-a-trigger
|
||||
description: Checklist for adding a new TriggerCrud-based trigger type to Windmill (Azure, GCP, Kafka, etc.). Use when wiring a new trigger kind across backend, frontend, CLI, and capture infrastructure.
|
||||
---
|
||||
|
||||
# Skill: Adding a New Trigger Type
|
||||
|
||||
Use this skill when adding a trigger kind that implements `TriggerCrud` (Kafka, GCP, Azure, MQTT, SQS, NATS, Postgres, Email…). For native triggers (Nextcloud, Google Drive — things wired through `windmill-native-triggers`), use the `native-trigger` skill instead.
|
||||
|
||||
The goal of this doc is to enumerate every file that needs to change. Missing any one of them leads to silent regressions: sync drops the trigger, capture button does nothing, workspace forks lose it, sidebar counters undercount. Follow the checklist top-to-bottom — each section is independent enough to be validated on its own.
|
||||
|
||||
Throughout this doc, substitute `{kind}` for the new trigger kind (`azure`, `kafka`, …), `{Kind}` for PascalCase (`Azure`, `Kafka`), `{KIND}` for SCREAMING (`AZURE`, `KAFKA`).
|
||||
|
||||
## Reference implementations
|
||||
|
||||
- **GCP** — closest analogue to Azure. Has push + pull, OIDC auth, ARM-like resource paths, capture handler. Grep for `gcp_trigger` / `GcpTrigger`.
|
||||
- **Kafka** — simpler (pull-only, streaming). Good for trivial integrations.
|
||||
- **Azure** — most recently added (2026). Shared-secret push auth, Event Grid namespaces + basic topics, ARM resource discovery, Namespace-pull data-plane. Grep for `azure_trigger` / `AzureTrigger`.
|
||||
|
||||
## 1. Database migration
|
||||
|
||||
Create a migration: `cargo sqlx migrate add -r add_{kind}_trigger` from `backend/`. Never write timestamps manually.
|
||||
|
||||
The `up.sql` usually defines:
|
||||
- An optional enum type (e.g. `AZURE_MODE`) if the trigger has sub-kinds
|
||||
- The `{kind}_trigger` table with at minimum these columns (mirrored from kafka/gcp):
|
||||
- primary: `(workspace_id, path)`
|
||||
- `script_path`, `is_flow`, `enabled`, `mode`, `permissioned_as`, `edited_by`, `email`
|
||||
- `edited_at`, `error`, `server_id`, `last_server_ping`
|
||||
- `error_handler_path`, `error_handler_args jsonb`, `retry jsonb`
|
||||
- trigger-specific fields
|
||||
- Indexes on foreign keys + any frequently-filtered columns
|
||||
- Foreign key to `workspace`
|
||||
- The RLS policies (`see_own`, `see_member`, `see_folder_extra_perms_user_*`, `see_extra_perms_user_*`, `see_extra_perms_groups_*`), copied from an existing trigger table
|
||||
|
||||
**RLS: wrap every session GUC read in a scalar sub-select.** Write the session
|
||||
reads as `(select current_setting('session.user'))`,
|
||||
`= any((select regexp_split_to_array(current_setting('session.groups'), ','))::text[])`,
|
||||
`?| (select regexp_split_to_array(current_setting('session.pgroups'), ','))::text[]`,
|
||||
`? (select concat('u/', current_setting('session.user')))`, etc. — not the bare
|
||||
`current_setting(...)`. The GUCs are set with `SET LOCAL`, so the sub-select
|
||||
hoists them to a one-time InitPlan instead of re-evaluating per scanned row.
|
||||
Put the `::text[]` cast **outside** the sub-select for the array cases: in an
|
||||
`= any (...)` context, casting inside — `= any((select ...::text[]))` — makes
|
||||
Postgres parse the operand as a row-returning subquery and fails at CREATE with
|
||||
`operator does not exist: text = text[]`. The outside cast keeps it in
|
||||
array-operand form. See migration `20260714230440_wrap_session_gucs_in_rls_policies`
|
||||
for the canonical wrapped forms.
|
||||
|
||||
Down migration drops the table and any enum types.
|
||||
|
||||
## 2. Backend crate (`windmill-trigger-{kind}`)
|
||||
|
||||
Create a new crate under `backend/windmill-trigger-{kind}/` with:
|
||||
|
||||
- `Cargo.toml`: features `enterprise`, `private` if EE, standard deps
|
||||
- `src/lib.rs`: `pub use mod_ee::*;` behind `#[cfg(all(feature = "enterprise", feature = "private"))]`
|
||||
- `src/mod_ee.rs`: core types + helpers
|
||||
- `src/handler_ee.rs`: `TriggerCrud` impl + route handlers
|
||||
- `src/listener_ee.rs`: (only if streaming/pull-based) `Listener` trait impl
|
||||
|
||||
Required in `mod_ee.rs`:
|
||||
- `{Kind}Config` struct (persisted shape, `FromRow`)
|
||||
- `{Kind}ConfigRequest` struct (what API receives — usually similar to Config but with validation fields)
|
||||
- `{Kind}Trigger` unit struct (implements the traits)
|
||||
- `impl TriggerJobArgs for {Kind}Trigger` — sets `TRIGGER_KIND`, `Payload`, `v1_payload_fn`
|
||||
|
||||
Required in `handler_ee.rs`:
|
||||
- `#[async_trait] impl TriggerCrud for {Kind}Trigger` with:
|
||||
- `type Trigger = Trigger<{Kind}Config>`
|
||||
- `type TriggerConfigRequest = {Kind}ConfigRequest`
|
||||
- `const ROUTE_PREFIX: &'static str = "/{kind}_triggers";`
|
||||
- `const TABLE_NAME`, `ADDITIONAL_SELECT_FIELDS`
|
||||
- `get_deployed_object`, `validate_config`, `create_trigger`, `update_trigger`, `delete_trigger`, `test_connection`
|
||||
- `additional_routes` (optional — mount extra endpoints for things like ARM resource listing, topic discovery)
|
||||
|
||||
Register the crate in `backend/Cargo.toml` as a workspace member and as a dep of `windmill-api` behind the feature flag.
|
||||
|
||||
## 3. Wire into `windmill-api` (feature-gated everywhere)
|
||||
|
||||
**`backend/windmill-api/src/triggers/handler.rs`** — mount the trigger crate:
|
||||
```rust
|
||||
#[cfg(all(feature = "enterprise", feature = "{kind}_trigger", feature = "private"))]
|
||||
{
|
||||
use crate::triggers::{kind}::{Kind}Trigger;
|
||||
router = router.nest({Kind}Trigger::ROUTE_PREFIX, complete_trigger_routes({Kind}Trigger));
|
||||
}
|
||||
```
|
||||
|
||||
**`backend/windmill-api/src/triggers/{kind}/mod.rs`** — re-export the crate:
|
||||
```rust
|
||||
pub use windmill_trigger_{kind}::*;
|
||||
```
|
||||
|
||||
**`backend/windmill-api/src/lib.rs`** — if the trigger receives inbound pushes, add a webhook route:
|
||||
```rust
|
||||
.nest("/{kind}/w/{workspace_id}", {
|
||||
#[cfg(all(feature = "enterprise", feature = "{kind}_trigger", feature = "private"))]
|
||||
{ triggers::{kind}::handler_oss::{kind}_push_route_handler() }
|
||||
#[cfg(not(...))]
|
||||
{ Router::new() }
|
||||
})
|
||||
```
|
||||
|
||||
## 4. `TriggerKind` enum (`backend/windmill-types/src/triggers.rs`)
|
||||
|
||||
Already has slots for most triggers but verify your variant exists:
|
||||
- Add `{Kind}` to the `TriggerKind` enum
|
||||
- Add match arm in `to_key()`
|
||||
- Add match arm in `from_str`
|
||||
- Add match arm in `JobTriggerKind` (if jobs need kind tagging)
|
||||
|
||||
## 5. OpenAPI (`backend/windmill-api/openapi.yaml`)
|
||||
|
||||
This file is huge and the single most-forgotten place. Add:
|
||||
|
||||
- `/w/{workspace}/{kind}_triggers/create` + `/update/{path}` + `/delete/{path}` + `/get/{path}` + `/list` + `/exists/{path}` + `/setmode/{path}` + `/test` paths (mirror gcp section)
|
||||
- Any `additional_routes` your handler exposes (resource discovery, etc.)
|
||||
- Schemas: `{Kind}Trigger`, `{Kind}TriggerData`, `{Kind}Mode` (if enum), `{Kind}DeliveryConfig`, helper request/response types
|
||||
- Add `{kind}` to `CaptureTriggerKind` enum
|
||||
- Add `{kind}_used: boolean` to the `UsedTriggers` response schema
|
||||
|
||||
Regenerate frontend client: `npm run generate-backend-client` from `frontend/`.
|
||||
|
||||
## 6. `UsedTriggers` + workspace export
|
||||
|
||||
**`backend/windmill-api-workspaces/src/workspaces.rs`** — add `{kind}_used: bool` to the `UsedTriggers` struct and add an `EXISTS(SELECT 1 FROM {kind}_trigger …)` to the `get_used_triggers` query.
|
||||
|
||||
**`backend/windmill-api/src/workspaces_export.rs`** — add export block mirroring gcp's (export lists all triggers, serializes them to YAML/JSON). The block re-uses the `trigger_ignore_keys` variable so the new kind automatically participates in fork-export stripping (`mode` field is omitted when the source workspace is a fork — keeps fork→parent merges from flipping the parent's enabled state).
|
||||
|
||||
**Fork cloning (`clone_triggers_and_schedules` in workspaces.rs)** — add an `INSERT INTO {kind}_trigger ... SELECT ...` block that copies all rows from the parent workspace, forcing `mode = 'disabled'::TRIGGER_MODE`. Always runs at fork creation; forgetting this means users can't carry `{kind}` triggers into their forks.
|
||||
|
||||
## 6.5 Hardcoded trigger-kind arrays (silent-failure hotspots)
|
||||
|
||||
Several files keep **hardcoded arrays** of trigger kind strings. Miss one and ACL checks / user offboarding / trash drop your kind:
|
||||
|
||||
- **`backend/windmill-api-groups/src/granular_acls.rs`** — `KINDS: [&str; N]`. **Increment N** (the compile error is cryptic otherwise). Controls which kinds accept granular ACL operations.
|
||||
- **`backend/windmill-api-users/src/users.rs`** (`extra_perms_tables`) — which tables get `extra_perms` entries cleaned when a user is deleted.
|
||||
- **`backend/windmill-api/src/offboarding.rs`** — three separate arrays (enumeration, fork-copy, and delete paths). **All three** need the new kind.
|
||||
- **`backend/windmill-api/src/trash.rs`** — `valid_tables` for the trash / restore API.
|
||||
- **`backend/windmill-git-sync/src/lib.rs`** — add a test assertion for `DeployedObject::{Kind}Trigger.get_kind() == "{kind}_trigger"` (the `get_kind` match arm itself lives in the enum impl — already required by the Rust compiler).
|
||||
- **`backend/windmill-api-auth/src/scopes.rs`** — add the `{Kind}Triggers` variant to `ScopeDomain` enum + `as_str` match + `from_str` match. Required for the OAuth/token system to recognise `{kind}_triggers:read|write` scopes.
|
||||
- **`backend/windmill-api/src/token.rs`** (`build_trigger_scope_domains` → `TRIGGER_DOMAINS`) — add `("{kind}_triggers", "{Kind display name}")` so the CreateToken UI's scope selector surfaces the `read` / `write` checkboxes.
|
||||
|
||||
**OpenAPI enums** to extend (do NOT forget — generated client will allow it but server rejects as 400):
|
||||
- `CaptureTriggerKind` enum
|
||||
- Three `kind` enums under `/w/{workspace}/acls/{get,add,remove}/{kind}/{path}` (yes, same list repeated three times)
|
||||
|
||||
After editing any of these, run a full `cargo check` with your feature flag + `gcp_trigger` + other core flags — the `KINDS: [&str; N]` length mismatch only surfaces when the crate compiles.
|
||||
|
||||
## 7. Capture infrastructure (`backend/windmill-api/src/capture.rs`)
|
||||
|
||||
If the trigger supports push delivery, it also needs a capture endpoint so users can test it:
|
||||
|
||||
- `{Kind}TriggerConfig` struct (gated by feature flags)
|
||||
- `TriggerConfig::{Kind}` variant
|
||||
- `set_{kind}_trigger_config` function (creates the subscription/equivalent pointing at the capture URL — use your `manage_{kind}_subscription` helper with `trigger_mode=false`)
|
||||
- Both real + no-op versions behind feature gates
|
||||
- `TriggerKind::{Kind} => set_{kind}_trigger_config(...)` arm in `set_config`
|
||||
- `{kind}_payload` async handler — validates auth (if any), processes payload, calls `insert_capture_payload`
|
||||
- Route: `.route("/{kind}/{runnable_kind}/{*path}", post({kind}_payload))` inside `workspaced_unauthed_service` — and expand the surrounding `#[cfg(any(...))]` to include your feature flag
|
||||
|
||||
## 8. CLI (`cli/`) — easy to miss, breaks sync silently
|
||||
|
||||
Check all of these:
|
||||
|
||||
**`cli/src/types.ts`:**
|
||||
- Add `"{kind}"` to `TRIGGER_TYPES` array
|
||||
- Add `"{kind}_trigger"` to `getTypeStrFromPath` return union
|
||||
- Add match case in `getTypeStrFromPath`'s `typeEnding ===` chain
|
||||
- Add `pushTrigger("{kind}", ...)` branch in `pushObj`
|
||||
|
||||
**`cli/src/commands/trigger/trigger.ts`:**
|
||||
- Import `{Kind}Trigger` type
|
||||
- Add `{kind}: {Kind}Trigger` to the `Trigger` type map
|
||||
- Add `{kind}: wmill.get{Kind}Trigger`, `update{Kind}Trigger`, `create{Kind}Trigger` to each function map
|
||||
- Add `{kind}: { ... }` template to `triggerTemplates`
|
||||
- Add `list{Kind}Triggers` call + spread in the `list` aggregation
|
||||
- Update `--kind` option descriptions to mention the new kind
|
||||
|
||||
**`cli/src/commands/sync/sync.ts`:**
|
||||
- Add `path.endsWith(".{kind}_trigger" + ext)` in the file-type filter
|
||||
- Add `typ == "{kind}_trigger"` in `getTypeOrder`
|
||||
- Add `"{kind}_trigger"` to the delete-suffix regex (~line 3092)
|
||||
- Add a `case "{kind}_trigger"` in the delete switch
|
||||
|
||||
**`cli/src/guidance/skills.ts`** — **DO NOT EDIT DIRECTLY**. It's auto-generated by `system_prompts/generate.py`. Instead:
|
||||
- Edit `system_prompts/utils.py` → append `('{Kind}Trigger', '{kind}_trigger')` to the `SCHEMA_MAPPINGS['triggers']` list (this is the master list — the one in `generate.py` is duplicated and `utils.py` wins)
|
||||
- Then run `python3 system_prompts/generate.py` — it regenerates `cli/src/guidance/skills.ts` with the schema extracted from `backend/windmill-api/openapi.yaml`
|
||||
- Commit the regenerated file
|
||||
|
||||
## 9. Frontend — editor + drawer
|
||||
|
||||
Under `frontend/src/lib/components/triggers/{kind}/`:
|
||||
|
||||
- `{Kind}TriggerPanel.svelte` — the tile shown in the triggers listing
|
||||
- `{Kind}TriggerEditor.svelte` — outer drawer wrapper
|
||||
- `{Kind}TriggerEditorInner.svelte` — state + business logic; must expose:
|
||||
- `openEdit(path, isFlow, defaultValues?)` method
|
||||
- `isEditor` prop, `onConfigChange` + `onCaptureConfigChange` callbacks
|
||||
- `get{Kind}Config()` + `get{Kind}CaptureConfig()` helpers
|
||||
- `captureConfig = $derived.by(untrack(() => isEditor) ? get{Kind}CaptureConfig : () => ({}))`
|
||||
- `$effect(() => { const args = [captureConfig, isValid] as const; untrack(() => onCaptureConfigChange?.(...args)) })`
|
||||
- `{Kind}TriggerEditorConfigSection.svelte` — form fields; use design-system components (`TextInput`, `Select`, `Toggle`, `ToggleButtonGroup`), never raw `<input>`
|
||||
- `{Kind}Capture.svelte` — capture panel; wraps `CaptureSection` with `captureType="{kind}"`
|
||||
- `utils.ts` — `requestBody` builders and any trigger-type-specific helpers
|
||||
|
||||
## 10. Frontend — global integration
|
||||
|
||||
Easy to miss:
|
||||
|
||||
- **`frontend/src/lib/components/triggers.ts`** — add `'{kind}'` to the `TriggerKind` union
|
||||
- **`frontend/src/lib/components/triggers/CaptureWrapper.svelte`**:
|
||||
- Import `{Kind}Capture`
|
||||
- Add to `isStreamingCapture()` array (streaming = pull-style; push-style is typically `false`)
|
||||
- Add `{:else if captureType === '{kind}'}` branch with the `<{Kind}Capture>` render
|
||||
- **`frontend/src/lib/components/sidebar/SidebarContent.svelte`** — import the icon, add the nav entry
|
||||
- **`frontend/src/lib/components/sidebar/OperatorMenu.svelte`** — add the operator-mode entry
|
||||
- **`frontend/src/routes/(root)/(logged)/+layout.svelte`** — destructure `{kind}_used` from `/get_used_triggers` response, push `'{kind}'` into `usedKinds`
|
||||
- **`frontend/src/lib/components/search/GlobalSearchModal.svelte`** — import icon, add "Go to {Kind} ..." entry
|
||||
- **`frontend/src/lib/components/offboarding-utils.ts`** — add mappings `{kind}_trigger: '{kind}_triggers'` and `{kind}_trigger: '{kind} trigger'`
|
||||
- **`frontend/src/lib/components/icons/{Kind}Icon.svelte`** — single-path SVG, `fill={color ?? 'currentColor'}`, `size` prop default 16 (match existing icons — don't hardcode colors, don't use `width`/`height` props)
|
||||
- **`frontend/src/routes/(root)/(logged)/{kind}_triggers/+page.svelte`** — listing page (mirror `gcp_triggers/+page.svelte` for push+pull, `kafka_triggers` for pure streaming)
|
||||
- **`frontend/src/lib/components/CompareWorkspaces.svelte`** — workspace fork / compare tool. Needs: service import, editor import, `{kind}Editor` `$state`, `case '{kind}'` in `openTriggerDetails()`, entry in `triggerServices` object (list/delete/normalize), and `<{Kind}TriggerEditor bind:this={{kind}Editor} />` in the template
|
||||
|
||||
## 10.5 AI system prompts (`system_prompts/`)
|
||||
|
||||
- **`system_prompts/utils.py`** — append `('{Kind}Trigger', '{kind}_trigger')` to `SCHEMA_MAPPINGS['triggers']` (master list used by code generation + CLI skills)
|
||||
- **`system_prompts/generate.py`** — also has a duplicated `schema_types` list (~line 903) for the AI `triggers` skill content. Add `('{Kind}Trigger', '{kind}_trigger')` there too
|
||||
- **`system_prompts/generate.py`** `schema_names` (~line 1192) — add `'{Kind}Trigger'` (add `'New{Kind}Trigger'` only if the OpenAPI declares one; GCP and Azure don't)
|
||||
- Run `python3 system_prompts/generate.py` — this rewrites `cli/src/guidance/skills.ts` and all `auto-generated/` docs. Commit the regenerated files
|
||||
|
||||
## 11. Validation
|
||||
|
||||
Run all of these before declaring done:
|
||||
|
||||
```bash
|
||||
# Backend
|
||||
cd backend
|
||||
cargo check --features enterprise,{kind}_trigger,private # minimal
|
||||
cargo check --features enterprise,azure_trigger,private,gcp_trigger,http_trigger,mqtt_trigger,postgres_trigger,sqs_trigger,kafka,nats,smtp,websocket # full
|
||||
|
||||
# SQLx offline data (never run `cargo sqlx prepare` directly — use the wrapper)
|
||||
./update_sqlx.sh
|
||||
|
||||
# Frontend
|
||||
cd frontend
|
||||
npm run generate-backend-client
|
||||
npm run check:fast
|
||||
```
|
||||
|
||||
Smoke test in the UI: create a trigger, save, check it appears in sidebar + search, delete, re-create via CLI `wmill sync`.
|
||||
|
||||
## 12. Common pitfalls
|
||||
|
||||
- **Forgetting feature gates in `workspaced_unauthed_service()`** — the surrounding `#[cfg(any(...))]` expression must include your feature flag, not just the inner `#[cfg]` on the route
|
||||
- **`.route(path, ...).route(path, ...)` with same path and different methods** — older axum replaced; use `.route(path, post(h1).options(h2))` to chain methods on the same `MethodRouter`
|
||||
- **`on:event` directives** — legacy Svelte 4, no-op in runes mode. Use callback props (`onSelected`, `onConfigChange`)
|
||||
- **`$bindable(default_value)` on optional props** — banned by project CLAUDE.md. Use `$bindable()` + `$derived(prop ?? default)` instead
|
||||
- **CORS layer intercepting OPTIONS** — tower-http CorsLayer short-circuits OPTIONS before reaching your handler. For server-to-server webhook endpoints, drop the CORS layer entirely (CORS is browser-only)
|
||||
- **DeliveryAttributeMappings / custom headers for auth** — prefer HMAC or sha256-hashed shared secrets over opaque JWTs when the provider doesn't support signed tokens natively. Store only the hash; regenerate secret on every save
|
||||
- **ARM / API resource-listing cascades** — if the trigger's resource type is deep (Azure: subscription → RG → namespace → topic), offer dropdowns in the UI populated from the provider's APIs using the user's credential resource
|
||||
- **Clearing stale selections on dependency change** — when a dropdown's underlying data reloads (e.g., user changes SP or edition), clear selections that no longer match the new list
|
||||
- **Workspace-scoped tag compatibility** — if the trigger has tags, verify forked workspaces handle them (see commit `0773b5bc85` for a historical fix)
|
||||
|
||||
## 13. EE file split
|
||||
|
||||
If the trigger is enterprise-only, the code lives in `windmill-ee-private__worktrees/.../windmill-trigger-{kind}/src/*_ee.rs` and is symlinked into the OSS tree. The `windmill-ee-private__worktrees/` directory holds the real files; changes propagate via symlinks. See `docs/enterprise.md` for the workflow.
|
||||
|
||||
## 14. Final checklist before PR
|
||||
|
||||
- [ ] Migration up/down tested (revert + re-apply)
|
||||
- [ ] `./update_sqlx.sh` committed the updated `.sqlx/` offline data
|
||||
- [ ] `cargo check` passes with your feature flag + with all trigger features
|
||||
- [ ] `npm run check:fast` passes
|
||||
- [ ] Trigger visible in sidebar with correct icon weight (not oversized/colored — use `currentColor`)
|
||||
- [ ] Create, edit, delete flow all work in the UI
|
||||
- [ ] Capture button works (if push-capable)
|
||||
- [ ] Trigger appears in `/get_used_triggers` → sidebar pulse
|
||||
- [ ] `wmill sync pull` + `wmill sync push` both round-trip the trigger
|
||||
- [ ] `wmill trigger list` includes it
|
||||
- [ ] OpenAPI schemas are complete (no `null` in generated types)
|
||||
@@ -1,40 +0,0 @@
|
||||
---
|
||||
name: ai-chat
|
||||
description: Guidance for improving the Windmill AI chat (copilot), especially global mode — tools, prompts, and context-window discipline. Use when editing chat tools, system prompts, or tool-result shapes under frontend/src/lib/components/copilot/chat, or when changing how the chat manages its context window.
|
||||
---
|
||||
|
||||
## Always benchmark before and after
|
||||
|
||||
No context or behavior change ships without an `ai_evals` A/B on the affected mode.
|
||||
Add or adjust cases for exactly what you changed — see the `ai-evals` skill for
|
||||
authoring and the full run reference.
|
||||
|
||||
Run the affected mode **before** your change and **after**, same model(s), same cases.
|
||||
|
||||
## Measure the window first, and cumulative second
|
||||
|
||||
Optimize **`finalContextTokens`** (window occupancy — what drives overflow and
|
||||
compaction), then cumulative prompt tokens.
|
||||
|
||||
## Context discipline
|
||||
|
||||
The dominant fixed cost is per-iteration overhead: the system prompt **plus every
|
||||
tool schema** is re-sent on every loop iteration. So:
|
||||
|
||||
- **Every tool and every parameter is a permanent tax.** Justify each one and measure
|
||||
it; an extra "locate" round-trip can cost more than the reads it saves. Strip dead
|
||||
params rather than leaving them in the schema.
|
||||
- **Tool results return the minimum.** Never echo content the model already has. The
|
||||
canonical mistake: a write tool that returns the whole edited artifact right after
|
||||
the model authored it — return `{ success, message }` instead. When you touch a
|
||||
*shared* write helper (e.g. `finishAppDraftWrite` in `global/core.ts`), re-check
|
||||
this invariant for **all** the write tools routing through it — the echo has
|
||||
regressed before via a shared refactor.
|
||||
|
||||
## Prompts and tool descriptions are part of the surface
|
||||
|
||||
The system prompt and tool descriptions steer behavior as much as the tools
|
||||
themselves, and are benchmarkable the same way. A description that advertises
|
||||
truncation makes the model self-limit; the path-conventions block changes where
|
||||
drafts land. Treat prompt/description edits as real changes and A/B them — a
|
||||
pure-prompt change is a legitimate, measurable improvement.
|
||||
@@ -1,87 +0,0 @@
|
||||
---
|
||||
name: ai-evals
|
||||
description: Author and run black-box benchmark cases for the Windmill AI generation modes (flow/app/script/cli/global) in ai_evals/. Use when adding or changing eval cases, or when running before/after benchmarks for AI chat / copilot changes.
|
||||
---
|
||||
|
||||
# AI evals — authoring and running benchmark cases
|
||||
|
||||
`ai_evals/` is a black-box benchmark runner for the Windmill AI generation modes:
|
||||
`flow`, `app`, `script`, `cli`, `global`. It always tests the **current** production
|
||||
prompts, tools, and guidance in this checkout. Each attempt runs the real production
|
||||
path, deterministic validation, then LLM judging.
|
||||
|
||||
The goal is to test current production guidance with realistic user requests — **not**
|
||||
to pin one exact implementation shape.
|
||||
|
||||
## Running benchmarks
|
||||
|
||||
```bash
|
||||
cd ai_evals
|
||||
bun install # first time; frontend modes also need `cd frontend && bun install`
|
||||
bun run cli -- models # list model aliases
|
||||
bun run cli -- cases global # list cases for a mode
|
||||
bun run cli -- run global global-test1-script-create --model sonnet
|
||||
```
|
||||
|
||||
Frontend modes (`flow`/`script`/`app`/`global`) route model calls through a Windmill
|
||||
backend's `/api/w/<ws>/ai/proxy`, so you need **any** reachable backend:
|
||||
|
||||
```bash
|
||||
WMILL_AI_EVAL_BACKEND_URL=http://127.0.0.1:<port> WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests \
|
||||
bun run cli -- run global <caseIds...> --models sonnet,gpt-5.5,gemini-3.1-pro-preview
|
||||
```
|
||||
|
||||
- **Reuse an existing workspace.** CE builds cap workspaces, so temp-workspace
|
||||
creation 400s ("reached workspace limit"). Always set
|
||||
`WMILL_AI_EVAL_BACKEND_WORKSPACE=integration-tests` (or any existing workspace) to
|
||||
reuse one. The only side effect of a run is upserting an `f/evals/ai/<provider>`
|
||||
resource there.
|
||||
- Provider keys live in `ai_evals/.env` and are auto-loaded by bun. The judge is a
|
||||
separate Anthropic call (default `claude-sonnet-4-6`) regardless of the model under
|
||||
test.
|
||||
|
||||
## Authoring core rules
|
||||
|
||||
1. Write prompts like a real user request.
|
||||
2. Prefer behavior, inputs, constraints, and outcomes over internal implementation.
|
||||
3. Keep deterministic validation narrow and hard.
|
||||
4. Put semantic expectations in `judgeChecklist`.
|
||||
5. Use `expected` fixtures only when exact structure really matters.
|
||||
|
||||
### Prompt writing
|
||||
|
||||
Prompts should sound like something a user would naturally ask. Do not write prompts
|
||||
as if the user knows Windmill internals unless the case explicitly tests a power-user
|
||||
workflow.
|
||||
|
||||
Good:
|
||||
- "Create a flow that routes support requests based on customer tier."
|
||||
- "Add a reset button that sets the counter back to 0."
|
||||
- "Create a flow that reuses the existing greeting script instead of duplicating the logic."
|
||||
|
||||
Bad:
|
||||
- "Use `branchone` with 3 branches and a default branch."
|
||||
- "Create a `rawscript` step with this exact topology."
|
||||
- "This is a benchmark harness."
|
||||
|
||||
### Deterministic validation
|
||||
|
||||
Use deterministic checks only for hard failures: missing required files; unexpected
|
||||
extra files when the prompt says not to create them; syntax errors; unresolved flow
|
||||
refs; missing required special modules or suspend config; obvious corruption.
|
||||
|
||||
Do **not** encode one preferred implementation. Bad hard checks: exact step topology
|
||||
for a creation flow; exact branch structure when the prompt only asked for routing;
|
||||
exact input shape when multiple reasonable shapes are acceptable.
|
||||
|
||||
### Judge checklist
|
||||
|
||||
Every non-trivial case should have a `judgeChecklist` capturing user-visible behavior
|
||||
that must be present, important constraints, and key completion criteria — not
|
||||
low-level implementation details unless truly required.
|
||||
|
||||
Good: "the flow calculates the order total with 8% tax"; "the flow reuses the existing
|
||||
workspace script instead of rewriting the logic". Bad: "uses `branchone`"; "contains a
|
||||
`rawscript` node".
|
||||
|
||||
See `ai_evals/README.md` for the full case format, fields, and fixture details.
|
||||
@@ -1,58 +0,0 @@
|
||||
---
|
||||
name: commit
|
||||
user_invocable: true
|
||||
description: Create a git commit with conventional commit format. MUST use anytime you want to commit changes.
|
||||
---
|
||||
|
||||
# Git Commit Skill
|
||||
|
||||
Create a focused, single-line commit following conventional commit conventions.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Analyze changes**: Run `git status` and `git diff` to understand what was modified
|
||||
2. **Stage only modified files**: Add files individually by name. NEVER use `git add -A` or `git add .`
|
||||
3. **Write commit message**: Follow the conventional commit format as a single line
|
||||
|
||||
## Conventional Commit Format
|
||||
|
||||
```
|
||||
<type>: <description>
|
||||
```
|
||||
|
||||
### Types
|
||||
- `feat`: New feature or capability
|
||||
- `fix`: Bug fix
|
||||
- `refactor`: Code change that neither fixes a bug nor adds a feature
|
||||
- `docs`: Documentation only changes
|
||||
- `style`: Formatting, missing semicolons, etc (no code change)
|
||||
- `test`: Adding or correcting tests
|
||||
- `chore`: Maintenance tasks, dependency updates, etc
|
||||
- `perf`: Performance improvement
|
||||
|
||||
### Rules
|
||||
- Message MUST be a single line (no multi-line messages)
|
||||
- Description should be lowercase, imperative mood ("add" not "added")
|
||||
- No period at the end
|
||||
- Keep under 72 characters total
|
||||
|
||||
### Examples
|
||||
```
|
||||
feat: add token usage tracking for AI providers
|
||||
fix: resolve null pointer in job executor
|
||||
refactor: extract common validation logic
|
||||
docs: update API endpoint documentation
|
||||
chore: upgrade sqlx to 0.7
|
||||
```
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Run `git status` to see all changes
|
||||
2. Run `git diff` to understand the changes in detail
|
||||
3. Run `git log --oneline -5` to see recent commit style
|
||||
4. Stage ONLY the modified/relevant files: `git add <file1> <file2> ...`
|
||||
5. Create the commit with conventional format:
|
||||
```bash
|
||||
git commit -m "<type>: <description>"
|
||||
```
|
||||
6. Run `git status` to verify the commit succeeded
|
||||
@@ -1,50 +0,0 @@
|
||||
---
|
||||
name: local-review-codex
|
||||
description: Run the CI Codex PR review locally against this branch's unpushed work (committed + uncommitted) before pushing. Same policy, model, and reasoning effort as the codex-pr-review GitHub action.
|
||||
---
|
||||
|
||||
# Local Codex Review (pre-push)
|
||||
|
||||
Runs the exact same review Codex performs in CI (`.github/workflows/codex-pr-review.yml`),
|
||||
but locally and scoped to work you have not pushed yet — so you catch what CI would flag
|
||||
before the PR exists. Use this before `git push` on a non-trivial change.
|
||||
|
||||
**Correspondence with CI** — identical:
|
||||
- Policy: `REVIEW.md` (severity triage, public-surface checklist, AGENTS.md compliance, test coverage).
|
||||
- Model: `gpt-5.6-sol`, `model_reasoning_effort="xhigh"`.
|
||||
- Output: markdown starting with `## Codex Review`, findings tagged P0 / P1 / P2 with file:line.
|
||||
|
||||
**Differences from CI** — local-only:
|
||||
- Scope is the current branch vs `main` at the merge-base, **including uncommitted changes** (CI reviews a pushed PR diff).
|
||||
- Sandbox is `read-only` (CI uses `danger-full-access` on an ephemeral runner). Codex reads the diff and files but cannot modify your working tree.
|
||||
- Fresh context is inherent: `codex exec` is a separate cold process, so it does not anchor on the current chat session — the same reason `local-review` insists on a subagent.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `codex` CLI **>= 0.144.1** installed and authed (`codex login` or `OPENAI_API_KEY`). Older CLIs reject `gpt-5.6-sol` with "requires a newer version of Codex". Upgrade with `npm install --global @openai/codex@0.144.1` (may need `sudo` for a global install). Keep this in sync with the pin in `.github/workflows/codex-pr-review.yml`.
|
||||
- `git fetch` the base ref if it's stale, so the merge-base is accurate.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
bash .agents/skills/local-review-codex/run.sh # review vs main (default)
|
||||
bash .agents/skills/local-review-codex/run.sh <base> # review vs a different base ref
|
||||
```
|
||||
|
||||
Invoke with `bash` (or run the executable directly) — the script needs Bash for
|
||||
`set -o pipefail`; `sh` is Dash on Debian/Ubuntu and would fail. If `main` isn't a
|
||||
local branch (e.g. a fresh single-branch checkout), the runner falls back to
|
||||
`origin/main` automatically.
|
||||
|
||||
The script computes `BASE_SHA = git merge-base HEAD <base>`, feeds Codex `REVIEW.md` plus a
|
||||
diff context pointing at `git diff <BASE_SHA>` (which folds in uncommitted edits), and prints
|
||||
the review. It writes only temp files — nothing lands in the working tree.
|
||||
|
||||
## Relaying the result
|
||||
|
||||
Print the Codex output verbatim. Do not re-summarize or filter it — the value of a cold Codex
|
||||
pass is surfacing what the current session would rationalize away. Then decide with the user
|
||||
whether to address findings before pushing.
|
||||
|
||||
For a Claude-native review instead, use `local-review` (branch-diff-reviewer subagent). This
|
||||
skill is the Codex counterpart; run both for independent perspectives.
|
||||
@@ -1,91 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Local Codex review — mirrors the .github/workflows/codex-pr-review.yml CI job,
|
||||
# but scoped to this branch's unpushed work (committed + uncommitted) so you can
|
||||
# review before pushing. Same policy (REVIEW.md), same model (gpt-5.6-sol) and
|
||||
# reasoning effort (xhigh) as CI. Runs read-only: Codex cannot modify your tree.
|
||||
#
|
||||
# Usage: run.sh [BASE_REF] (BASE_REF defaults to "main")
|
||||
set -euo pipefail
|
||||
|
||||
BASE_REF="${1:-main}"
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
if ! command -v codex >/dev/null 2>&1; then
|
||||
echo "codex CLI not found. Install with: npm install --global @openai/codex@0.144.1" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Resolve the base to a concrete commit, preferring a local ref but falling back to
|
||||
# the remote-tracking ref — checkouts (CI, single-branch clones) often have only
|
||||
# origin/main, not a local main.
|
||||
if git rev-parse --verify --quiet "${BASE_REF}^{commit}" >/dev/null; then
|
||||
BASE_COMMITISH="$BASE_REF"
|
||||
elif git rev-parse --verify --quiet "origin/${BASE_REF}^{commit}" >/dev/null; then
|
||||
BASE_COMMITISH="origin/${BASE_REF}"
|
||||
else
|
||||
echo "Base ref '$BASE_REF' not found as '$BASE_REF' or 'origin/$BASE_REF'. Try: git fetch origin $BASE_REF" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Diff from the merge-base so only this branch's changes are reviewed. Using the
|
||||
# base SHA with a single-ref `git diff` also folds in uncommitted working-tree edits,
|
||||
# but `git diff` never sees untracked files — those are gathered separately below so
|
||||
# brand-new files (a whole new module, a new skill dir) are not silently skipped.
|
||||
BASE_SHA="$(git merge-base HEAD "$BASE_COMMITISH")"
|
||||
HEAD_SHA="$(git rev-parse HEAD)"
|
||||
UNTRACKED="$(git ls-files --others --exclude-standard)"
|
||||
|
||||
if [ "$BASE_SHA" = "$HEAD_SHA" ] && git diff --quiet "$BASE_SHA" && [ -z "$UNTRACKED" ]; then
|
||||
echo "No changes vs $BASE_REF — nothing to review." >&2
|
||||
exit 0
|
||||
fi
|
||||
|
||||
PROMPT="$(mktemp)"
|
||||
OUT="$(mktemp)"
|
||||
trap 'rm -f "$PROMPT" "$OUT"' EXIT
|
||||
|
||||
# REVIEW.md is the shared policy CI feeds Codex. Append the local output-format
|
||||
# and diff context inline (CI reads these from a generated context file; inlining
|
||||
# keeps the working tree clean — no scratch files land in the repo).
|
||||
cat REVIEW.md > "$PROMPT"
|
||||
cat >> "$PROMPT" <<EOF
|
||||
|
||||
# Codex output format
|
||||
|
||||
- This is a pre-push LOCAL review of unpushed work; there is no PR yet.
|
||||
- Inspect the changes by running the diff commands in the review context below.
|
||||
- Untracked files do NOT appear in \`git diff\`. Review every untracked path listed below by reading it directly (\`cat\`) — treat its entire contents as newly added.
|
||||
- Return markdown starting with \`## Codex Review\`.
|
||||
- Tag each finding with a severity (P0 / P1 / P2), file path, and line number when known confidently.
|
||||
|
||||
# Review context
|
||||
|
||||
Local review (pre-push): current branch vs $BASE_REF
|
||||
Base SHA: $BASE_SHA
|
||||
Head SHA: $HEAD_SHA (plus any uncommitted working-tree changes)
|
||||
|
||||
Changed commits command:
|
||||
git log --oneline $BASE_SHA..HEAD
|
||||
|
||||
Changed files command:
|
||||
git diff --stat $BASE_SHA
|
||||
|
||||
Full review diff command (tracked changes, includes uncommitted edits):
|
||||
git diff --unified=0 $BASE_SHA
|
||||
|
||||
Untracked files (NOT in the diff above — read each one directly, it is entirely new):
|
||||
$(if [ -n "$UNTRACKED" ]; then printf '%s\n' "$UNTRACKED"; else echo "(none)"; fi)
|
||||
EOF
|
||||
|
||||
codex exec \
|
||||
-C "$REPO_ROOT" \
|
||||
-m gpt-5.6-sol \
|
||||
-c 'model_reasoning_effort="xhigh"' \
|
||||
-s read-only \
|
||||
-o "$OUT" \
|
||||
- < "$PROMPT"
|
||||
|
||||
echo
|
||||
echo "===== Codex review ====="
|
||||
cat "$OUT"
|
||||
@@ -1,98 +0,0 @@
|
||||
---
|
||||
name: local-review
|
||||
description: Code review the current PR (or branch diff against main) for bugs, security, and AGENTS.md compliance. MUST use when asked to review code.
|
||||
---
|
||||
|
||||
# Local Code Review
|
||||
|
||||
Run the same review locally that the GitHub auto-review actions run on PRs (Claude / Codex / Pi). The review policy lives in `REVIEW.md`.
|
||||
|
||||
**Why a subagent**: the review MUST run in a fresh context — not inline in the current session. If the user has been iterating on the diff, the main session has absorbed their reasoning and rationalizations, so it anchors and misses things CI catches. A subagent starts cold, like CI does.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Determine the PR scope** (cheap, do this in the main session):
|
||||
- If an argument is provided, treat it as a PR number or branch.
|
||||
- Otherwise, detect from the current branch vs `main`.
|
||||
- Confirm the PR/branch exists (`gh pr view <n>` or `git rev-parse <branch>`).
|
||||
|
||||
2. **Delegate the review to a fresh-context subagent** with a self-contained prompt. The prompt MUST include:
|
||||
- The PR number or branch name to review.
|
||||
- The instruction to read `REVIEW.md` first for the policy, then `AGENTS.md` files in directories touched by the diff.
|
||||
- The exact output format (see below).
|
||||
- Whether `--comment` was requested (so the subagent emits inline-comment payloads if needed).
|
||||
- Any "Additional reviewer instructions" the user provided.
|
||||
|
||||
- **Claude Code**: use the `Agent` tool with `subagent_type: branch-diff-reviewer` (read-only tools, purpose-built for this). If unavailable, fall back to `general-purpose`.
|
||||
- **Codex / Pi**: if the CLI exposes a fresh-session subagent mechanism, use it. Otherwise tell the user to run the skill in a fresh CLI session and stop — running inline in the current session defeats the purpose.
|
||||
|
||||
3. **Receive the findings** from the subagent and relay them to the user verbatim. Do not re-summarize, re-judge, or filter — the whole point of fresh context is to surface what the main session would dismiss.
|
||||
|
||||
4. **Post comments if `--comment` was requested**: use the `gh` commands below with the subagent's output as the body. The main session does the posting because the subagent is read-only.
|
||||
|
||||
## Subagent prompt template
|
||||
|
||||
```
|
||||
Review <PR #N | branch X> against main per the policy in REVIEW.md.
|
||||
|
||||
Steps:
|
||||
1. Read REVIEW.md (repo root) for the full policy: severity triage, public-surface
|
||||
checklist, AGENTS.md compliance, test coverage assessment.
|
||||
2. Read AGENTS.md (repo root) and any AGENTS.md in directories touched by the diff.
|
||||
3. Get the diff: `gh pr diff <N>` (if PR) or `git diff main...<branch>`.
|
||||
4. Get context: `gh pr view <N>` (if PR) or `git log main..<branch> --oneline`.
|
||||
5. Read changed files only when the diff alone is insufficient to validate a finding.
|
||||
6. Self-validate each finding: "is this definitely a real issue a senior engineer
|
||||
would flag?" Discard if uncertain.
|
||||
7. Output findings in the exact format below. Do not modify any files.
|
||||
|
||||
<paste output format from below>
|
||||
|
||||
<if --comment requested:>
|
||||
Additionally emit a JSON array of inline comments suitable for the GitHub reviews
|
||||
API, one per finding that maps to a specific line:
|
||||
[{"path": "...", "line": N, "side": "RIGHT", "body": "[P1] ..."}, ...]
|
||||
```
|
||||
|
||||
## Output format
|
||||
|
||||
```
|
||||
## Code review
|
||||
|
||||
<verdict line per REVIEW.md>
|
||||
|
||||
Found N issues:
|
||||
|
||||
1. [P0|P1|P2] <description>
|
||||
<file_path:line_number>
|
||||
|
||||
2. [P0|P1|P2] <description>
|
||||
<file_path:line_number>
|
||||
```
|
||||
|
||||
End with a `Test coverage` section per the shared policy.
|
||||
|
||||
If no issues are found:
|
||||
|
||||
```
|
||||
## Code review
|
||||
|
||||
Good to merge.
|
||||
|
||||
No issues found. Checked for bugs, security, and AGENTS.md compliance.
|
||||
```
|
||||
|
||||
## Posting comments (`--comment`)
|
||||
|
||||
For a top-level PR comment:
|
||||
|
||||
```bash
|
||||
gh pr review --comment --body "<summary from subagent>"
|
||||
```
|
||||
|
||||
For inline comments on specific lines (using the JSON the subagent emitted):
|
||||
|
||||
```bash
|
||||
gh api repos/{owner}/{repo}/pulls/{pr}/reviews \
|
||||
-f body="<summary>" -f event="COMMENT" -f comments="<json from subagent>"
|
||||
```
|
||||
@@ -1,793 +0,0 @@
|
||||
---
|
||||
name: native-trigger
|
||||
description: Guidance for adding native trigger services to Windmill. Use when implementing or modifying native trigger integrations across the backend and frontend.
|
||||
---
|
||||
|
||||
# Skill: Adding Native Trigger Services
|
||||
|
||||
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The native trigger system consists of:
|
||||
|
||||
1. **Database Layer** - PostgreSQL tables and enum types
|
||||
2. **Backend Rust Implementation** - Core trait, handlers, and service modules in the `windmill-native-triggers` crate
|
||||
3. **Frontend Svelte Components** - Configuration forms and UI components
|
||||
|
||||
### Key Files
|
||||
|
||||
| Component | Path |
|
||||
|-----------|------|
|
||||
| Core module with `External` trait | `backend/windmill-native-triggers/src/lib.rs` |
|
||||
| Generic CRUD handlers | `backend/windmill-native-triggers/src/handler.rs` |
|
||||
| Background sync logic | `backend/windmill-native-triggers/src/sync.rs` |
|
||||
| OAuth/workspace integration | `backend/windmill-native-triggers/src/workspace_integrations.rs` |
|
||||
| Re-export shim (windmill-api) | `backend/windmill-api/src/native_triggers/mod.rs` |
|
||||
| TriggerKind enum | `backend/windmill-common/src/triggers.rs` |
|
||||
| JobTriggerKind enum | `backend/windmill-common/src/jobs.rs` |
|
||||
| Frontend service registry | `frontend/src/lib/components/triggers/native/utils.ts` |
|
||||
| Frontend trigger utilities | `frontend/src/lib/components/triggers/utils.ts` |
|
||||
| Trigger badges (icons + counts) | `frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte` |
|
||||
| Workspace integrations UI | `frontend/src/lib/components/workspaceSettings/WorkspaceIntegrations.svelte` |
|
||||
| OAuth config form component | `frontend/src/lib/components/workspaceSettings/OAuthClientConfig.svelte` |
|
||||
| OpenAPI spec | `backend/windmill-api/openapi.yaml` |
|
||||
| Reference: Nextcloud module | `backend/windmill-native-triggers/src/nextcloud/` |
|
||||
| Reference: Google module | `backend/windmill-native-triggers/src/google/` |
|
||||
|
||||
### Crate Structure
|
||||
|
||||
The native trigger code lives in the `windmill-native-triggers` crate (`backend/windmill-native-triggers/`). The `windmill-api` crate re-exports everything via a shim:
|
||||
|
||||
```rust
|
||||
// backend/windmill-api/src/native_triggers/mod.rs
|
||||
pub use windmill_native_triggers::*;
|
||||
```
|
||||
|
||||
All new service modules go in `backend/windmill-native-triggers/src/`.
|
||||
|
||||
---
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### The `External` Trait
|
||||
|
||||
Every native trigger service implements the `External` trait defined in `lib.rs`:
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
pub trait External: Send + Sync + 'static {
|
||||
// Associated types:
|
||||
type ServiceConfig: Debug + DeserializeOwned + Serialize + Send + Sync;
|
||||
type TriggerData: Debug + Serialize + Send + Sync;
|
||||
type OAuthData: DeserializeOwned + Serialize + Clone + Send + Sync;
|
||||
type CreateResponse: DeserializeOwned + Send + Sync;
|
||||
|
||||
// Constants:
|
||||
const SUPPORT_WEBHOOK: bool;
|
||||
const SERVICE_NAME: ServiceName;
|
||||
const DISPLAY_NAME: &'static str;
|
||||
const TOKEN_ENDPOINT: &'static str;
|
||||
const REFRESH_ENDPOINT: &'static str;
|
||||
const AUTH_ENDPOINT: &'static str;
|
||||
|
||||
// Required methods:
|
||||
async fn create(&self, w_id, oauth_data, webhook_token, data, db, tx) -> Result<Self::CreateResponse>;
|
||||
async fn update(&self, w_id, oauth_data, external_id, webhook_token, data, db, tx) -> Result<serde_json::Value>;
|
||||
async fn get(&self, w_id, oauth_data, external_id, db, tx) -> Result<Self::TriggerData>;
|
||||
async fn delete(&self, w_id, oauth_data, external_id, db, tx) -> Result<()>;
|
||||
async fn exists(&self, w_id, oauth_data, external_id, db, tx) -> Result<bool>;
|
||||
async fn maintain_triggers(&self, db, workspace_id, triggers, oauth_data, synced, errors);
|
||||
fn external_id_and_metadata_from_response(&self, resp) -> (String, Option<serde_json::Value>);
|
||||
|
||||
// Methods with defaults:
|
||||
async fn prepare_webhook(&self, db, w_id, headers, body, script_path, is_flow) -> Result<PushArgsOwned>;
|
||||
fn service_config_from_create_response(&self, data, resp) -> Option<serde_json::Value>;
|
||||
fn additional_routes(&self) -> axum::Router;
|
||||
async fn http_client_request<T, B>(&self, url, method, workspace_id, tx, db, headers, body) -> Result<T>;
|
||||
}
|
||||
```
|
||||
|
||||
Key design points:
|
||||
- **`update()` returns `serde_json::Value`** - the resolved service_config to store. Each service is responsible for building the final config.
|
||||
- **`maintain_triggers()`** - periodic background maintenance. Each service implements its own strategy (Nextcloud: reconcile with external state; Google: renew expiring channels).
|
||||
- **No `list_all()` in the trait** - services that need it (Nextcloud) implement it privately; services that don't (Google) use different maintenance strategies.
|
||||
- **No `get_external_id_from_trigger_data()` or `extract_service_config_from_trigger_data()`** - removed in favor of the `maintain_triggers` pattern.
|
||||
|
||||
### Create Lifecycle: Two Paths
|
||||
|
||||
The `create_native_trigger` handler in `handler.rs` supports two creation flows, controlled by `service_config_from_create_response()`:
|
||||
|
||||
**Path A: Short (Google pattern)** - `service_config_from_create_response()` returns `Some(config)`:
|
||||
1. `create()` registers on external service
|
||||
2. `external_id_and_metadata_from_response()` extracts the ID
|
||||
3. `service_config_from_create_response()` builds the config directly from input data + response metadata
|
||||
4. Stores trigger in DB -- done, no extra round-trip
|
||||
|
||||
Use this when the external_id is known before the create call (e.g., Google generates the channel_id as a UUID upfront and includes it in the webhook URL).
|
||||
|
||||
**Path B: Long (Nextcloud pattern)** - `service_config_from_create_response()` returns `None` (default):
|
||||
1. `create()` registers on external service (webhook URL has no external_id yet)
|
||||
2. `external_id_and_metadata_from_response()` extracts the ID
|
||||
3. `update()` is called to fix the webhook URL with the now-known external_id
|
||||
4. `update()` returns the resolved service_config
|
||||
5. Stores trigger in DB
|
||||
|
||||
Use this when the external_id is assigned by the remote service and the webhook URL needs to be corrected after creation.
|
||||
|
||||
### OAuth Token Storage (Three-Table Pattern)
|
||||
|
||||
OAuth tokens are stored across three tables, NOT in `workspace_integrations.oauth_data` directly:
|
||||
|
||||
| Table | What's Stored |
|
||||
|-------|---------------|
|
||||
| `workspace_integrations` | `oauth_data` JSON with `base_url`, `client_id`, `client_secret`, `instance_shared` flag; `resource_path` pointing to the variable |
|
||||
| `variable` | Encrypted `access_token` (at the path stored in `resource_path`), linked to `account` via `account` column |
|
||||
| `account` | `refresh_token`, keyed by `workspace_id` + `client` (service name) + `is_workspace_integration = true` |
|
||||
|
||||
The `decrypt_oauth_data()` function in `lib.rs` assembles these into a unified struct:
|
||||
```rust
|
||||
pub struct OAuthConfig {
|
||||
pub base_url: String,
|
||||
pub access_token: String, // decrypted from variable
|
||||
pub refresh_token: Option<String>, // from account table
|
||||
pub client_id: String, // from oauth_data or instance settings
|
||||
pub client_secret: String, // from oauth_data or instance settings
|
||||
}
|
||||
```
|
||||
|
||||
Instance-level sharing: when `oauth_data.instance_shared == true`, `client_id` and `client_secret` are read from global settings instead of workspace_integrations.
|
||||
|
||||
### URL Resolution
|
||||
|
||||
The `resolve_endpoint()` helper handles both absolute and relative OAuth URLs:
|
||||
|
||||
```rust
|
||||
pub fn resolve_endpoint(base_url: &str, endpoint: &str) -> String {
|
||||
if endpoint.starts_with("http://") || endpoint.starts_with("https://") {
|
||||
endpoint.to_string() // Google: absolute URLs
|
||||
} else {
|
||||
format!("{}{}", base_url, endpoint) // Nextcloud: relative paths
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### ServiceName Methods
|
||||
|
||||
`ServiceName` is the central registry enum. Each variant must implement these match arms:
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `as_str()` | Lowercase identifier (e.g., `"google"`) |
|
||||
| `as_trigger_kind()` | Maps to `TriggerKind` enum |
|
||||
| `as_job_trigger_kind()` | Maps to `JobTriggerKind` enum |
|
||||
| `token_endpoint()` | OAuth token endpoint (relative or absolute) |
|
||||
| `auth_endpoint()` | OAuth authorization endpoint |
|
||||
| `oauth_scopes()` | Space-separated OAuth scopes |
|
||||
| `resource_type()` | Resource type for token storage (e.g., `"gworkspace"`) |
|
||||
| `extra_auth_params()` | Extra OAuth params (e.g., Google needs `access_type=offline`, `prompt=consent`) |
|
||||
| `integration_service()` | Maps to the workspace integration service (usually `*self`) |
|
||||
| `TryFrom<String>` | Parse from string |
|
||||
| `Display` | Delegates to `as_str()` |
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Implementation Guide
|
||||
|
||||
### Step 1: Database Migration
|
||||
|
||||
Create a new migration file: `backend/migrations/YYYYMMDDHHMMSS_newservice_trigger.up.sql`
|
||||
|
||||
```sql
|
||||
-- Add the service to the native_trigger_service enum
|
||||
ALTER TYPE native_trigger_service ADD VALUE IF NOT EXISTS 'newservice';
|
||||
|
||||
-- Add to TRIGGER_KIND enum (used for trigger tracking)
|
||||
ALTER TYPE TRIGGER_KIND ADD VALUE IF NOT EXISTS 'newservice';
|
||||
|
||||
-- Add to job_trigger_kind enum (used for job tracking)
|
||||
ALTER TYPE job_trigger_kind ADD VALUE IF NOT EXISTS 'newservice';
|
||||
```
|
||||
|
||||
Also create the corresponding down migration.
|
||||
|
||||
### Step 2: Update windmill-common Enums
|
||||
|
||||
#### `backend/windmill-common/src/triggers.rs`
|
||||
|
||||
Add variant to `TriggerKind` enum, and update `to_key()` and `fmt()` implementations.
|
||||
|
||||
#### `backend/windmill-common/src/jobs.rs`
|
||||
|
||||
Add variant to `JobTriggerKind` enum and update the `Display` implementation.
|
||||
|
||||
### Step 3: Backend Service Module
|
||||
|
||||
Create a new directory: `backend/windmill-native-triggers/src/newservice/`
|
||||
|
||||
#### `mod.rs` - Type Definitions
|
||||
|
||||
```rust
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub mod external;
|
||||
// pub mod routes; // Only if you need additional service-specific routes
|
||||
|
||||
/// OAuth data deserialized from the three-table pattern.
|
||||
/// The actual structure is built by decrypt_oauth_data() from variable + account + workspace_integrations.
|
||||
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||
pub struct NewServiceOAuthData {
|
||||
pub base_url: String, // from workspace_integrations.oauth_data
|
||||
pub access_token: String, // decrypted from variable table
|
||||
pub refresh_token: Option<String>, // from account table
|
||||
// Note: client_id and client_secret are in OAuthConfig, not here
|
||||
// unless the service needs them at runtime for API calls
|
||||
}
|
||||
|
||||
/// Configuration provided by user when creating/updating a trigger.
|
||||
/// Stored as JSON in native_trigger.service_config.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct NewServiceConfig {
|
||||
// Service-specific configuration fields
|
||||
pub folder_path: String,
|
||||
pub file_filter: Option<String>,
|
||||
}
|
||||
|
||||
/// Data retrieved from the external service about a trigger.
|
||||
/// Returned by the get() method and shown in the UI.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct NewServiceTriggerData {
|
||||
pub folder_path: String,
|
||||
pub file_filter: Option<String>,
|
||||
// Fields that shouldn't affect service_config comparison should use #[serde(skip_serializing)]
|
||||
}
|
||||
|
||||
/// Response from external service when creating a trigger/webhook.
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreateTriggerResponse {
|
||||
pub id: String,
|
||||
}
|
||||
|
||||
/// Handler struct (stateless, used for routing)
|
||||
#[derive(Copy, Clone)]
|
||||
pub struct NewService;
|
||||
```
|
||||
|
||||
#### `external.rs` - External Trait Implementation
|
||||
|
||||
```rust
|
||||
use async_trait::async_trait;
|
||||
use reqwest::Method;
|
||||
use sqlx::PgConnection;
|
||||
use std::collections::HashMap;
|
||||
use windmill_common::{
|
||||
error::{Error, Result},
|
||||
BASE_URL, DB,
|
||||
};
|
||||
|
||||
use crate::{
|
||||
generate_webhook_service_url, External, NativeTrigger, NativeTriggerData, ServiceName,
|
||||
sync::{SyncError, TriggerSyncInfo},
|
||||
};
|
||||
use super::{NewService, NewServiceConfig, NewServiceOAuthData, NewServiceTriggerData, CreateTriggerResponse};
|
||||
|
||||
#[async_trait]
|
||||
impl External for NewService {
|
||||
type ServiceConfig = NewServiceConfig;
|
||||
type TriggerData = NewServiceTriggerData;
|
||||
type OAuthData = NewServiceOAuthData;
|
||||
type CreateResponse = CreateTriggerResponse;
|
||||
|
||||
const SERVICE_NAME: ServiceName = ServiceName::NewService;
|
||||
const DISPLAY_NAME: &'static str = "New Service";
|
||||
const SUPPORT_WEBHOOK: bool = true;
|
||||
const TOKEN_ENDPOINT: &'static str = "/oauth/token";
|
||||
const REFRESH_ENDPOINT: &'static str = "/oauth/token";
|
||||
const AUTH_ENDPOINT: &'static str = "/oauth/authorize";
|
||||
|
||||
async fn create(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
webhook_token: &str,
|
||||
data: &NativeTriggerData<Self::ServiceConfig>,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<Self::CreateResponse> {
|
||||
let base_url = &*BASE_URL.read().await;
|
||||
|
||||
// external_id is None during create (we get it from the response)
|
||||
let webhook_url = generate_webhook_service_url(
|
||||
base_url, w_id, &data.script_path, data.is_flow,
|
||||
None, Self::SERVICE_NAME, webhook_token,
|
||||
);
|
||||
|
||||
let url = format!("{}/api/webhooks/create", oauth_data.base_url);
|
||||
let payload = serde_json::json!({
|
||||
"callback_url": webhook_url,
|
||||
"folder_path": data.service_config.folder_path,
|
||||
});
|
||||
|
||||
let response: CreateTriggerResponse = self
|
||||
.http_client_request(&url, Method::POST, w_id, tx, db, None, Some(&payload))
|
||||
.await?;
|
||||
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
/// Update returns the resolved service_config as JSON.
|
||||
/// For services using the update+get pattern, call self.get() and serialize.
|
||||
async fn update(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
webhook_token: &str,
|
||||
data: &NativeTriggerData<Self::ServiceConfig>,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<serde_json::Value> {
|
||||
let base_url = &*BASE_URL.read().await;
|
||||
|
||||
let webhook_url = generate_webhook_service_url(
|
||||
base_url, w_id, &data.script_path, data.is_flow,
|
||||
Some(external_id), Self::SERVICE_NAME, webhook_token,
|
||||
);
|
||||
|
||||
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
|
||||
let payload = serde_json::json!({
|
||||
"callback_url": webhook_url,
|
||||
"folder_path": data.service_config.folder_path,
|
||||
});
|
||||
|
||||
let _: serde_json::Value = self
|
||||
.http_client_request(&url, Method::PUT, w_id, tx, db, None, Some(&payload))
|
||||
.await?;
|
||||
|
||||
// Fetch back the updated state to get the resolved config
|
||||
let trigger_data = self.get(w_id, oauth_data, external_id, db, tx).await?;
|
||||
serde_json::to_value(&trigger_data)
|
||||
.map_err(|e| Error::InternalErr(format!("Failed to serialize trigger data: {}", e)))
|
||||
}
|
||||
|
||||
async fn get(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<Self::TriggerData> {
|
||||
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
|
||||
self.http_client_request::<_, ()>(&url, Method::GET, w_id, tx, db, None, None).await
|
||||
}
|
||||
|
||||
async fn delete(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<()> {
|
||||
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
|
||||
let _: serde_json::Value = self
|
||||
.http_client_request::<_, ()>(&url, Method::DELETE, w_id, tx, db, None, None)
|
||||
.await
|
||||
.or_else(|e| match &e {
|
||||
Error::InternalErr(msg) if msg.contains("404") => Ok(serde_json::Value::Null),
|
||||
_ => Err(e),
|
||||
})?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn exists(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<bool> {
|
||||
match self.get(w_id, oauth_data, external_id, db, tx).await {
|
||||
Ok(_) => Ok(true),
|
||||
Err(Error::NotFound(_)) => Ok(false),
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
/// Background maintenance. Choose the right pattern for your service:
|
||||
/// - For services with queryable external state: use reconcile_with_external_state()
|
||||
/// - For channel-based services with expiration: implement renewal logic
|
||||
async fn maintain_triggers(
|
||||
&self,
|
||||
db: &DB,
|
||||
workspace_id: &str,
|
||||
triggers: &[NativeTrigger],
|
||||
oauth_data: &Self::OAuthData,
|
||||
synced: &mut Vec<TriggerSyncInfo>,
|
||||
errors: &mut Vec<SyncError>,
|
||||
) {
|
||||
// Option A: Reconcile with external state (Nextcloud pattern)
|
||||
// Fetch all triggers from external service and compare with DB
|
||||
let external_triggers = match self.list_all(workspace_id, oauth_data, db).await {
|
||||
Ok(triggers) => triggers,
|
||||
Err(e) => {
|
||||
errors.push(SyncError {
|
||||
resource_path: format!("workspace:{}", workspace_id),
|
||||
error_message: format!("Failed to list triggers: {}", e),
|
||||
error_type: "api_error".to_string(),
|
||||
});
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Convert to (external_id, config_json) pairs
|
||||
let external_pairs: Vec<(String, serde_json::Value)> = external_triggers
|
||||
.into_iter()
|
||||
.map(|t| (t.id.clone(), serde_json::to_value(&t).unwrap_or_default()))
|
||||
.collect();
|
||||
|
||||
crate::sync::reconcile_with_external_state(
|
||||
db, workspace_id, Self::SERVICE_NAME, triggers, &external_pairs, synced, errors,
|
||||
).await;
|
||||
}
|
||||
|
||||
fn external_id_and_metadata_from_response(
|
||||
&self,
|
||||
resp: &Self::CreateResponse,
|
||||
) -> (String, Option<serde_json::Value>) {
|
||||
(resp.id.clone(), None)
|
||||
}
|
||||
|
||||
// service_config_from_create_response: NOT overridden (returns None).
|
||||
// This means the handler uses the update+get pattern after create.
|
||||
// Override and return Some(...) to skip the update+get cycle (Google pattern).
|
||||
}
|
||||
|
||||
impl NewService {
|
||||
/// Private helper to list all triggers from the external service.
|
||||
async fn list_all(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &<Self as External>::OAuthData,
|
||||
db: &DB,
|
||||
) -> Result<Vec<<Self as External>::TriggerData>> {
|
||||
// Implementation depends on the external service's API
|
||||
todo!()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Update lib.rs Registry
|
||||
|
||||
In `backend/windmill-native-triggers/src/lib.rs`:
|
||||
|
||||
```rust
|
||||
// Service modules - add new services here:
|
||||
#[cfg(feature = "native_trigger")]
|
||||
pub mod newservice; // <-- Add this
|
||||
|
||||
// ServiceName enum - add variant:
|
||||
pub enum ServiceName {
|
||||
Nextcloud,
|
||||
Google,
|
||||
NewService, // <-- Add this
|
||||
}
|
||||
|
||||
// Then add match arms in ALL ServiceName methods:
|
||||
// as_str(), as_trigger_kind(), as_job_trigger_kind(), token_endpoint(),
|
||||
// auth_endpoint(), oauth_scopes(), resource_type(), extra_auth_params(),
|
||||
// integration_service(), TryFrom<String>, Display
|
||||
```
|
||||
|
||||
### Step 5: Update handler.rs Routes
|
||||
|
||||
In `backend/windmill-native-triggers/src/handler.rs`:
|
||||
|
||||
```rust
|
||||
pub fn generate_native_trigger_routers() -> Router {
|
||||
// ...
|
||||
#[cfg(feature = "native_trigger")]
|
||||
{
|
||||
use crate::newservice::NewService;
|
||||
return router
|
||||
.nest("/nextcloud", service_routes(NextCloud))
|
||||
.nest("/google", service_routes(Google))
|
||||
.nest("/newservice", service_routes(NewService)); // <-- Add this
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 6: Update sync.rs
|
||||
|
||||
In `backend/windmill-native-triggers/src/sync.rs`:
|
||||
|
||||
```rust
|
||||
pub async fn sync_all_triggers(db: &DB) -> Result<BackgroundSyncResult> {
|
||||
// ...
|
||||
#[cfg(feature = "native_trigger")]
|
||||
{
|
||||
use crate::newservice::NewService;
|
||||
|
||||
// ... existing service syncs ...
|
||||
|
||||
// New service sync
|
||||
let (service_name, result) = sync_service_triggers(db, NewService).await;
|
||||
total_synced += result.synced_triggers.len();
|
||||
total_errors += result.errors.len();
|
||||
service_results.insert(service_name, result);
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 7: Frontend Service Registry
|
||||
|
||||
In `frontend/src/lib/components/triggers/native/utils.ts`:
|
||||
|
||||
Add to `NATIVE_TRIGGER_SERVICES`, `getTriggerIconName()`, and `getServiceIcon()`.
|
||||
|
||||
### Step 8: Frontend Trigger Form Component
|
||||
|
||||
Create: `frontend/src/lib/components/triggers/native/services/newservice/NewServiceTriggerForm.svelte`
|
||||
|
||||
### Step 9: Frontend Icon Component
|
||||
|
||||
Create: `frontend/src/lib/components/icons/NewServiceIcon.svelte`
|
||||
|
||||
### Step 10: Update NativeTriggerEditor
|
||||
|
||||
Check `frontend/src/lib/components/triggers/native/NativeTriggerEditor.svelte` to ensure it dynamically loads form components based on service name.
|
||||
|
||||
### Step 11: Workspace Integration UI
|
||||
|
||||
Add your service to the `supportedServices` map in `frontend/src/lib/components/workspaceSettings/WorkspaceIntegrations.svelte`:
|
||||
|
||||
```typescript
|
||||
const supportedServices: Record<string, ServiceConfig> = {
|
||||
// ... existing services ...
|
||||
newservice: {
|
||||
name: 'newservice',
|
||||
displayName: 'New Service',
|
||||
description: 'Connect to New Service for triggers',
|
||||
icon: NewServiceIcon,
|
||||
docsUrl: 'https://www.windmill.dev/docs/integrations/newservice',
|
||||
requiresBaseUrl: false, // false for cloud services, true for self-hosted
|
||||
setupInstructions: [
|
||||
'Step 1: Create an OAuth app on the service',
|
||||
'Step 2: Configure the redirect URI shown below',
|
||||
'Step 3: Enter the client credentials below'
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 12: Update `frontend/src/lib/components/triggers/utils.ts`
|
||||
|
||||
Update ALL of these maps/functions:
|
||||
1. `triggerIconMap` - import and add icon
|
||||
2. `triggerDisplayNamesMap` - add display name
|
||||
3. `triggerTypeOrder` in `sortTriggers()` - add type
|
||||
4. `getLightConfig()` - add case for your service
|
||||
5. `getTriggerLabel()` - add case for your service
|
||||
6. `jobTriggerKinds` - add to array
|
||||
7. `countPropertyMap` - add count property
|
||||
8. `triggerSaveFunctions` - add save function
|
||||
|
||||
### Step 13: Update TriggersBadge Component
|
||||
|
||||
In `frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte`:
|
||||
|
||||
1. Import the icon
|
||||
2. Add to `baseConfig` with `countKey` (the dynamic `availableNativeServices` loop does NOT set `countKey`)
|
||||
3. Add to the `allTypes` array
|
||||
|
||||
### Step 14: Update TriggersWrapper.svelte
|
||||
|
||||
In `frontend/src/lib/components/triggers/TriggersWrapper.svelte`:
|
||||
|
||||
Add a `{:else if selectedTrigger.type === 'yourservice'}` case that renders `<NativeTriggersPanel service="yourservice" ...>` with the same props pattern as the existing native trigger cases (e.g., `nextcloud`).
|
||||
|
||||
### Step 15: Update AddTriggersButton.svelte
|
||||
|
||||
In `frontend/src/lib/components/triggers/AddTriggersButton.svelte`:
|
||||
|
||||
1. Add `yourserviceAvailable` state variable
|
||||
2. Add `setYourserviceState()` async function using `isServiceAvailable('yourservice', $workspaceStore!)`
|
||||
3. Call it at module level
|
||||
4. Add a dropdown entry to `addTriggerItems` with `hidden: !yourserviceAvailable`
|
||||
|
||||
### Step 16: Update TriggersEditor.svelte Delete Handling
|
||||
|
||||
In `frontend/src/lib/components/triggers/TriggersEditor.svelte`:
|
||||
|
||||
Add your service to the `nativeTriggerServices` map in `deleteDeployedTrigger()`. Native triggers use `NativeTriggerService.deleteNativeTrigger({ workspace, serviceName, externalId })` instead of the standard `path`-based delete.
|
||||
|
||||
### Step 17: Update `getUsedTriggers` for Sidebar Visibility
|
||||
|
||||
The sidebar (`frontend/src/lib/components/sidebar/SidebarContent.svelte`) shows native-trigger links only if `$usedTriggerKinds` includes the service — without this, your trigger page will never appear in the nav bar even when triggers exist.
|
||||
|
||||
1. **Backend** — add `{service}_used: bool` to the `UsedTriggers` struct and SELECT in `backend/windmill-api-workspaces/src/workspaces.rs::get_used_triggers()`:
|
||||
```rust
|
||||
EXISTS(SELECT 1 FROM native_trigger WHERE workspace_id = $1 AND service_name = '{service}'::native_trigger_service) AS "{service}_used!"
|
||||
```
|
||||
2. **OpenAPI** — add `{service}_used: boolean` to the response schema for `GET /w/{workspace}/workspaces/used_triggers` (under both `properties` and `required`).
|
||||
3. **Layout** — in `frontend/src/routes/(root)/(logged)/+layout.svelte::loadUsedTriggerKinds()`, destructure `{service}_used` and push `'{service}'` to `usedKinds`.
|
||||
|
||||
### Step 18: Update OpenAPI Spec and Regenerate Types
|
||||
|
||||
Add to `JobTriggerKind` enum in `backend/windmill-api/openapi.yaml`, then:
|
||||
|
||||
```bash
|
||||
cd frontend && npm run generate-backend-client
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Special Patterns
|
||||
|
||||
### Unified Service with `trigger_type` (Google Pattern)
|
||||
|
||||
When a single service handles multiple trigger types (e.g., Google Drive + Calendar share OAuth and API patterns), use a single `ServiceName` variant with a discriminator field:
|
||||
|
||||
```rust
|
||||
pub enum GoogleTriggerType { Drive, Calendar }
|
||||
|
||||
pub struct GoogleServiceConfig {
|
||||
pub trigger_type: GoogleTriggerType,
|
||||
// Drive-specific fields (only used when trigger_type = Drive)
|
||||
pub resource_id: Option<String>,
|
||||
pub resource_name: Option<String>,
|
||||
// Calendar-specific fields (only used when trigger_type = Calendar)
|
||||
pub calendar_id: Option<String>,
|
||||
pub calendar_name: Option<String>,
|
||||
// Metadata set after creation
|
||||
pub google_resource_id: Option<String>,
|
||||
pub expiration: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Branch in trait methods based on `trigger_type`. Frontend uses a `ToggleButtonGroup` to switch between types. This keeps the codebase simpler (one service, one OAuth flow, one set of routes).
|
||||
|
||||
See `backend/windmill-native-triggers/src/google/` for the reference implementation.
|
||||
|
||||
### Skipping update+get After Create (Google Pattern)
|
||||
|
||||
Override `service_config_from_create_response()` to return `Some(config)` when the external_id is known before the create call:
|
||||
|
||||
```rust
|
||||
fn service_config_from_create_response(
|
||||
&self,
|
||||
data: &NativeTriggerData<Self::ServiceConfig>,
|
||||
resp: &Self::CreateResponse,
|
||||
) -> Option<serde_json::Value> {
|
||||
// Clone input config, add metadata from response
|
||||
let mut config = data.service_config.clone();
|
||||
config.google_resource_id = Some(resp.resource_id.clone());
|
||||
config.expiration = Some(resp.expiration.clone());
|
||||
Some(serde_json::to_value(&config).unwrap())
|
||||
}
|
||||
```
|
||||
|
||||
### Services with Absolute OAuth Endpoints (Google)
|
||||
|
||||
Unlike self-hosted services where OAuth endpoints are relative paths appended to `base_url`, services like Google have absolute URLs:
|
||||
|
||||
```rust
|
||||
// Nextcloud: relative paths
|
||||
ServiceName::Nextcloud => "/apps/oauth2/api/v1/token",
|
||||
// Google: absolute URLs
|
||||
ServiceName::Google => "https://oauth2.googleapis.com/token",
|
||||
```
|
||||
|
||||
The `resolve_endpoint()` function handles both. For services with absolute endpoints:
|
||||
- `base_url` can be empty
|
||||
- `requiresBaseUrl: false` in the frontend workspace integration config
|
||||
- Add `extra_auth_params()` if needed (Google requires `access_type=offline` and `prompt=consent`)
|
||||
|
||||
### Channel-Based Push Notifications with Renewal (Google Pattern)
|
||||
|
||||
For services using expiring watch channels instead of persistent webhooks:
|
||||
|
||||
1. Store expiration in `service_config` (as part of `ServiceConfig`)
|
||||
2. In `maintain_triggers()`, implement renewal logic instead of using `reconcile_with_external_state()`:
|
||||
```rust
|
||||
async fn maintain_triggers(&self, db, workspace_id, triggers, oauth_data, synced, errors) {
|
||||
for trigger in triggers {
|
||||
if should_renew_channel(trigger) {
|
||||
self.renew_channel(db, trigger, oauth_data).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
3. Renewal: best-effort stop old channel, create new one with same external_id, update service_config with new expiration
|
||||
4. Google example: Drive channels expire in 24h (renew when <1h left), Calendar channels expire in 7 days (renew when <1 day left)
|
||||
|
||||
### reconcile_with_external_state (Nextcloud Pattern)
|
||||
|
||||
The reusable function in `sync.rs` compares external triggers with DB state:
|
||||
- Triggers missing externally: sets error "Trigger no longer exists on external service"
|
||||
- Triggers present externally: clears errors, updates service_config if it differs
|
||||
|
||||
Usage in `maintain_triggers()`:
|
||||
```rust
|
||||
let external_pairs: Vec<(String, serde_json::Value)> = /* fetch from external */;
|
||||
crate::sync::reconcile_with_external_state(
|
||||
db, workspace_id, Self::SERVICE_NAME, triggers, &external_pairs, synced, errors,
|
||||
).await;
|
||||
```
|
||||
|
||||
### Webhook Payload Processing
|
||||
|
||||
Override `prepare_webhook()` to parse service-specific payloads into script/flow args:
|
||||
|
||||
```rust
|
||||
async fn prepare_webhook(&self, db, w_id, headers, body, script_path, is_flow) -> Result<PushArgsOwned> {
|
||||
let mut args = HashMap::new();
|
||||
args.insert("event_type".to_string(), Box::new(headers.get("x-event-type").cloned()) as _);
|
||||
args.insert("payload".to_string(), Box::new(serde_json::from_str::<serde_json::Value>(&body)?) as _);
|
||||
Ok(PushArgsOwned { extra: None, args })
|
||||
}
|
||||
```
|
||||
|
||||
Then register in `prepare_native_trigger_args()` in `lib.rs`:
|
||||
```rust
|
||||
pub async fn prepare_native_trigger_args(service_name, db, w_id, headers, body) -> Result<Option<PushArgsOwned>> {
|
||||
match service_name {
|
||||
ServiceName::Google => { /* ... */ Ok(Some(args)) }
|
||||
ServiceName::NewService => { /* ... */ Ok(Some(args)) }
|
||||
ServiceName::Nextcloud => Ok(None), // Uses default body parsing
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Instance-Level OAuth Credentials
|
||||
|
||||
When `workspace_integrations.oauth_data.instance_shared == true`, `decrypt_oauth_data()` reads `client_id` and `client_secret` from instance-level global settings instead of workspace-level. This allows admins to share OAuth app credentials across workspaces.
|
||||
|
||||
The frontend handles this via the `generate_instance_connect_url` endpoint in `workspace_integrations.rs`.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Database migration runs successfully
|
||||
- [ ] `cargo check -p windmill-native-triggers --features native_trigger` passes
|
||||
- [ ] `npx svelte-check --threshold error` passes (in frontend/)
|
||||
- [ ] Service appears in workspace integrations list
|
||||
- [ ] OAuth flow completes successfully
|
||||
- [ ] Can create a new trigger
|
||||
- [ ] Can view trigger details
|
||||
- [ ] Can update trigger configuration
|
||||
- [ ] Can delete trigger
|
||||
- [ ] Webhook receives and processes payloads
|
||||
- [ ] Background sync works correctly (reconciliation or channel renewal)
|
||||
- [ ] Error handling works (expired tokens, service unavailable)
|
||||
|
||||
---
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
### Nextcloud (Self-Hosted, Update+Get Pattern)
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `nextcloud/mod.rs` | Types: NextCloudOAuthData, NextcloudServiceConfig, NextCloudTriggerData |
|
||||
| `nextcloud/external.rs` | External trait: uses update+get pattern, reconcile_with_external_state for sync |
|
||||
| `nextcloud/routes.rs` | Additional route: `GET /events` |
|
||||
|
||||
Key patterns: relative OAuth endpoints, base_url required, list_all + reconcile for sync, update returns JSON from get().
|
||||
|
||||
### Google (Cloud, Unified Service, Short Create)
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `google/mod.rs` | Types: GoogleServiceConfig with trigger_type discriminator, GoogleTriggerType enum |
|
||||
| `google/external.rs` | External trait: overrides service_config_from_create_response, channel renewal for sync |
|
||||
| `google/routes.rs` | Additional routes: `GET /calendars`, `GET /drive/files`, `GET /drive/shared_drives` |
|
||||
|
||||
Key patterns: absolute OAuth endpoints, empty base_url, trigger_type for Drive/Calendar, expiring watch channels with renewal, service_config_from_create_response skips update+get, get() reconstructs data from stored service_config (no external "get channel" API).
|
||||
@@ -1,182 +0,0 @@
|
||||
---
|
||||
name: pr
|
||||
user_invocable: true
|
||||
description: Open a draft pull request on GitHub and drive CI review rounds until it is ready. MUST use when you want to create/open a PR.
|
||||
---
|
||||
|
||||
# Pull Request Skill
|
||||
|
||||
Create a draft pull request with a clear title and explicit description of changes, then drive it through CI review rounds to ready.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Analyze branch changes**: Understand all commits since diverging from main
|
||||
2. **Push to remote**: Ensure all commits are pushed
|
||||
3. **Create draft PR**: Always open as draft for review before merging
|
||||
4. **Drive review rounds**: trigger CI reviews on the draft and only flip to ready once every verdict is a go (see "Review rounds" below)
|
||||
|
||||
## PR Title Format
|
||||
|
||||
Follow conventional commit format for the PR title:
|
||||
```
|
||||
<type>: <description>
|
||||
```
|
||||
|
||||
### Types
|
||||
- `feat`: New feature or capability
|
||||
- `fix`: Bug fix
|
||||
- `refactor`: Code restructuring
|
||||
- `docs`: Documentation changes
|
||||
- `chore`: Maintenance tasks
|
||||
- `perf`: Performance improvements
|
||||
|
||||
### Title Rules
|
||||
- Keep under 70 characters
|
||||
- Use lowercase, imperative mood
|
||||
- No period at the end
|
||||
- If `*_ee.rs` files were modified, prefix with `[ee]`: `[ee] <type>: <description>`
|
||||
|
||||
## PR Body Format
|
||||
|
||||
The body MUST be explicit about what changed. Structure:
|
||||
|
||||
```markdown
|
||||
## Summary
|
||||
<Clear description of what this PR does and why>
|
||||
|
||||
## Changes
|
||||
- <Specific change 1>
|
||||
- <Specific change 2>
|
||||
- <Specific change 3>
|
||||
|
||||
## Test plan
|
||||
- [ ] <How to verify change 1>
|
||||
- [ ] <How to verify change 2>
|
||||
```
|
||||
|
||||
The harness/tooling that invoked the skill may add its own attribution trailer; the skill itself does not prescribe one.
|
||||
|
||||
## Screenshots (required for frontend changes)
|
||||
|
||||
If `git diff main...HEAD --name-only` matches `^frontend/`, the PR body **must** include
|
||||
screenshots of the affected UI. Skip only when there is no visible UI effect (types,
|
||||
tests, build config) — and say so in the body.
|
||||
|
||||
1. Verify the change in the browser (AGENTS.md → "Verifying Frontend Changes").
|
||||
2. Screenshot each affected page with `mcp__playwright__browser_take_screenshot` (save to a file).
|
||||
3. Host each image and get its Markdown embed by pushing to the public
|
||||
`windmill-labs/agent-screenshots-internal` repo. **Pipe base64 through stdin** —
|
||||
passing it as `-f content=…` fails with `argument list too long` on real images:
|
||||
|
||||
```bash
|
||||
REPO=windmill-labs/agent-screenshots-internal
|
||||
IMG=screenshot.png # repeat per page
|
||||
DEST="shots/$(git branch --show-current)/$(date +%s)-$(basename "$IMG")"
|
||||
base64 -w0 "$IMG" | jq -Rs --arg m "add $DEST" '{message:$m, content:.}' \
|
||||
| gh api -X PUT "repos/$REPO/contents/$DEST" --input - >/dev/null
|
||||
echo ""
|
||||
```
|
||||
Derive `$DEST` from the file name (as above) so distinct pages never collide — a
|
||||
fixed name would make same-second uploads reuse one path, and the second `PUT`
|
||||
then 422s (the Contents API needs the existing file's `sha` to overwrite).
|
||||
4. Put the printed `` lines under a `## Screenshots` heading in the PR body.
|
||||
|
||||
Requires `gh` (`repo` scope), `jq`, `base64` — all in the devShell. The host repo is
|
||||
public (so the raw URLs render for reviewers without a token) and its history is
|
||||
permanent — **never screenshot pages that show secrets or sensitive values** (workspace
|
||||
variables, resource values, instance settings, OAuth/SMTP config); deleting the file
|
||||
can't undo an accidental capture. (GitHub's drag-and-drop uploader needs a browser
|
||||
session and can't be driven from a token.)
|
||||
|
||||
If `gh` can't push to the host repo (e.g. a CI token scoped only to `windmill`), do
|
||||
**not** fail the PR or skip silently — hand the upload to the user, who has push access,
|
||||
and continue once they confirm it's done.
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Run `git status` to check for uncommitted changes
|
||||
2. Run `git log main..HEAD --oneline` to see all commits in this branch
|
||||
3. Run `git diff main...HEAD` to see the full diff against main
|
||||
4. **Review the diff before creating the PR — run both reviews, do not skip:**
|
||||
- **`local-review`** — Claude-native branch-diff-reviewer (`/local-review` in Claude Code, `$local-review` in Codex, `pi --skill local-review` / `/skill:local-review` in Pi).
|
||||
- **`local-review-codex`** — cold Codex pass, the same review CI runs, for an independent perspective the Claude pass misses (`/local-review-codex` in Claude Code, or `bash .agents/skills/local-review-codex/run.sh`). If the `codex` CLI is missing or older than the version pinned in that skill, note it in your summary and continue — never block the PR on codex being unavailable.
|
||||
|
||||
Run both — they catch different things. If either surfaces issues, fix them and commit before proceeding.
|
||||
5. **Screenshots for frontend changes**: if `git diff main...HEAD --name-only` matches `^frontend/`, capture and embed screenshots of the affected UI per "Screenshots" above before writing the PR body (skip only if there is no visible UI effect).
|
||||
6. Check if remote branch exists and is up to date:
|
||||
```bash
|
||||
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
|
||||
```
|
||||
7. Push to remote if needed: `git push -u origin HEAD`
|
||||
8. Create draft PR using gh CLI:
|
||||
```bash
|
||||
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
|
||||
## Summary
|
||||
<description>
|
||||
|
||||
## Changes
|
||||
- <change 1>
|
||||
- <change 2>
|
||||
|
||||
## Test plan
|
||||
- [ ] <test 1>
|
||||
- [ ] <test 2>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
9. Return the PR URL to the user
|
||||
10. Drive the PR through CI review rounds to ready (see "Review rounds" below)
|
||||
|
||||
## Review rounds (draft → ready)
|
||||
|
||||
A PR leaves draft **only after a clean CI review round**. Never run `gh pr ready` before that.
|
||||
|
||||
1. **Trigger a round and wait for it**: launch the waiter as a background Bash task (a round takes 10–30 min; you are woken when it exits — do not stop the session or poll in the foreground while it runs):
|
||||
|
||||
```bash
|
||||
bash .agents/skills/pr/review-round.sh <PR_NUMBER>
|
||||
```
|
||||
|
||||
It comments `/review` on the PR — which runs the Codex, Claude and Pi CI reviewers even on a draft — waits for the spawned `PR Review Commands` workflow run(s) to complete, then prints one verdict line per reviewer and saves the full review comments to files.
|
||||
|
||||
`/review` (and `/codex`) are **idempotent per head SHA**: if a running or successful review already covers the current head, they skip that agent and post nothing new — the waiter reads the existing verdict for that head, so a skipped agent is *not* a missing one. A cancelled/failed head run is re-run in place; a fresh run is launched only when nothing covers the head. So an unchanged-head re-review is a near no-op, not a new round — push a commit to get genuinely fresh reviews.
|
||||
|
||||
2. **Judge the round.** Codex is mandatory; Claude, Pi and cubic count whenever they posted. Every review starts with one of the three `REVIEW.md` verdicts:
|
||||
- Codex verdict missing → the round is void: the waiter warns only when the head has no green Codex run (cancelled/failed/absent — not merely skipped-because-already-reviewed). Comment `/codex` on the PR, which re-runs the interrupted run in place (or launches one if none exists), wait the same way, and judge again.
|
||||
- Any **"Should address issues before merging"** → fix the P0/P1 findings (and the nits while you're there), commit, push, and start a new round (step 1).
|
||||
- Only **"Mergeable, but should ideally address nits"** and/or **"Good to merge"** → fix the nits too; a nit that is wrong or genuinely not worth fixing may instead be dismissed by replying to the review comment with your reasoning. Push nit-only fixes without starting another full round.
|
||||
|
||||
3. **Flip to ready with the marker comment.** The review workflows skip the redundant `ready_for_review`-triggered round when the PR author has posted a marker naming the current head SHA **and** the PR's latest Codex review *posted before the marker* has a non-blocking verdict (reviewer evidence — a bare marker with no round behind it, or one whose last pre-marker Codex verdict is "Should address issues", skips nothing). Keep the prefix exact and use the full 40-char SHA of the head you are flipping:
|
||||
- every verdict was "Good to merge" (head unchanged since the round):
|
||||
|
||||
`✅ Review round clean @ <head-sha>`
|
||||
|
||||
- nit-only round, nits fixed or dismissed afterwards (head may have moved past the reviewed SHA — say so):
|
||||
|
||||
`✅ Review round clean @ <head-sha> — nit-only verdicts at <round-sha>; nits addressed in <commit sha(s)> / dismissed in review replies`
|
||||
|
||||
```bash
|
||||
gh pr comment <PR_NUMBER> --body "✅ Review round clean @ $(git rev-parse HEAD)"
|
||||
gh pr ready <PR_NUMBER>
|
||||
```
|
||||
|
||||
If any P0/P1 finding is unaddressed or the head moved for reasons other than nit fixes, do **not** post the marker or flip — run another round instead.
|
||||
|
||||
## EE Companion PR (when `*_ee.rs` files were modified)
|
||||
|
||||
The `*_ee.rs` files in the windmill repo are **symlinks** to `windmill-ee-private` — changes won't appear in `git diff` of the windmill repo. Instead, check the EE repo for uncommitted or unpushed changes.
|
||||
|
||||
Follow the full EE PR workflow in `docs/enterprise.md`. The key PR-specific details:
|
||||
|
||||
1. Find the EE repo/worktree: see "Finding the EE Repo" in `docs/enterprise.md`
|
||||
2. Check for changes: `git -C <ee-path> status --short`
|
||||
- If there are no changes in the EE repo, skip this entire section
|
||||
3. Follow steps 1–5 from the "EE PR Workflow" in `docs/enterprise.md`
|
||||
4. Create the companion PR (title does NOT get the `[ee]` prefix):
|
||||
```bash
|
||||
gh pr create --draft --repo windmill-labs/windmill-ee-private --title "<type>: <description>" --body "$(cat <<'EOF'
|
||||
Companion PR for windmill-labs/windmill#<PR_NUMBER>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
5. Commit `ee-repo-ref.txt` and push the updated windmill branch
|
||||
@@ -1,172 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Trigger a CI review round on a PR and wait for it to finish.
|
||||
#
|
||||
# Usage: bash .agents/skills/pr/review-round.sh [PR_NUMBER]
|
||||
# PR_NUMBER defaults to the current branch's PR.
|
||||
#
|
||||
# Comments `/review` on the PR (works on drafts), waits for the spawned
|
||||
# "PR Review Commands" workflow run(s) to complete, then prints one verdict
|
||||
# line per reviewer and saves each full review comment to a file. A round
|
||||
# takes 10-30 minutes: run this in the background and act on its output when
|
||||
# it exits, per the pr skill ("Review rounds").
|
||||
set -euo pipefail
|
||||
|
||||
REPO=${REPO:-$(gh repo view --json nameWithOwner --jq .nameWithOwner)}
|
||||
PR=${1:-$(gh pr view --json number --jq .number)}
|
||||
|
||||
# Timestamp of the trigger comment, straight from GitHub, so local clock skew
|
||||
# can't make the run/comment filters below miss part of the round.
|
||||
TRIGGER_TIME=$(gh api "repos/$REPO/issues/$PR/comments" -f body='/review' --jq .created_at)
|
||||
echo "Review round triggered on $REPO#$PR at $TRIGGER_TIME"
|
||||
|
||||
# Retry wrapper for one-off gh/API hiccups: a 45-minute wait must not die on
|
||||
# a single transient failure.
|
||||
retry() {
|
||||
local attempt
|
||||
for attempt in 1 2 3; do
|
||||
if "$@"; then return 0; fi
|
||||
sleep 10
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# The /review comment spawns one "PR Review Commands" run holding the
|
||||
# claude/codex/pi jobs. Runs aren't linked to a PR, so wait on every run of
|
||||
# that workflow created after the trigger: a concurrent round on another PR
|
||||
# can only delay the answer, never truncate it. Every issue comment on any PR
|
||||
# spawns a fast-completing parse run of the same workflow, so the round's own
|
||||
# run may briefly lag the listing while unrelated runs already show completed:
|
||||
# require the all-completed state to hold past a floor and across two
|
||||
# consecutive polls before trusting it.
|
||||
DEADLINE=$(( $(date +%s) + 45 * 60 ))
|
||||
NO_RUN_DEADLINE=$(( $(date +%s) + 5 * 60 ))
|
||||
MIN_WAIT_UNTIL=$(( $(date +%s) + 3 * 60 ))
|
||||
STABLE=0
|
||||
FAILURES=0
|
||||
while :; do
|
||||
if RUNS=$(gh run list --repo "$REPO" --workflow=pr-review-commands.yml \
|
||||
--created ">=$TRIGGER_TIME" --limit 100 --json status); then
|
||||
FAILURES=0
|
||||
else
|
||||
FAILURES=$(( FAILURES + 1 ))
|
||||
if [ "$FAILURES" -ge 5 ]; then
|
||||
echo "ERROR: listing workflow runs failed $FAILURES times in a row; aborting the wait." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "WARNING: listing workflow runs failed (attempt $FAILURES/5); retrying in 60s." >&2
|
||||
sleep 60
|
||||
continue
|
||||
fi
|
||||
TOTAL=$(jq length <<<"$RUNS")
|
||||
PENDING=$(jq '[.[] | select(.status != "completed")] | length' <<<"$RUNS")
|
||||
NOW=$(date +%s)
|
||||
if [ "$TOTAL" -gt 0 ] && [ "$PENDING" -eq 0 ] && [ "$NOW" -gt "$MIN_WAIT_UNTIL" ]; then
|
||||
STABLE=$(( STABLE + 1 ))
|
||||
if [ "$STABLE" -ge 2 ]; then
|
||||
break
|
||||
fi
|
||||
else
|
||||
STABLE=0
|
||||
fi
|
||||
if [ "$TOTAL" -eq 0 ] && [ "$NOW" -gt "$NO_RUN_DEADLINE" ]; then
|
||||
echo "ERROR: no 'PR Review Commands' run appeared within 5 minutes of the /review comment; check that the comment author has write access and the workflow is enabled." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$NOW" -gt "$DEADLINE" ]; then
|
||||
echo "WARNING: review round still pending after 45 minutes; reporting whatever has been posted so far." >&2
|
||||
break
|
||||
fi
|
||||
sleep 60
|
||||
done
|
||||
|
||||
# Head SHA at trigger time. `/review` is idempotent per head: it skips an agent a
|
||||
# running/successful review already covers, re-runs a cancelled/failed one in place on a
|
||||
# separate head-tied run, and launches fresh only when nothing covers the head. Verdict
|
||||
# reading below therefore keys off the head, not just the trigger timestamp.
|
||||
HEAD_SHA=$(retry gh api "repos/$REPO/pulls/$PR" --jq .head.sha)
|
||||
echo "Reviewing head $HEAD_SHA"
|
||||
|
||||
# Newest non-skipped run of <workflow> tied to the head ("status conclusion"), or empty
|
||||
# when none exists. A re-run-in-place or an already-covering review resolves on such a
|
||||
# head-tied run — separate from the pr-review-commands run waited on above (a fresh
|
||||
# launch instead runs inside it, and posts after the trigger). A `skipped` run is the
|
||||
# draft/fork gate and produced no review, so it is ignored.
|
||||
head_run_state() {
|
||||
gh run list --repo "$REPO" --workflow "$1" --commit "$HEAD_SHA" --limit 20 \
|
||||
--json databaseId,status,conclusion \
|
||||
--jq '[.[] | select(.conclusion != "skipped")] | sort_by(.databaseId) | last | if . then "\(.status) \(.conclusion // "-")" else empty end' 2>/dev/null || true
|
||||
}
|
||||
|
||||
# A re-run-in-place review lands on a head-tied run that finishes after the fast
|
||||
# pr-review-commands run, so let those settle before reading verdicts.
|
||||
for wf in codex-pr-review.yml pi-pr-review.yml pr-ready-review.yml; do
|
||||
while :; do
|
||||
case "$(head_run_state "$wf")" in
|
||||
""|"completed "*) break ;;
|
||||
*) if [ "$(date +%s)" -gt "$DEADLINE" ]; then break; fi; sleep 30 ;;
|
||||
esac
|
||||
done
|
||||
done
|
||||
|
||||
OUT_DIR=$(mktemp -d -t review-round-XXXXXX)
|
||||
COMMENTS_RAW=$(retry gh api "repos/$REPO/issues/$PR/comments?per_page=100" --paginate)
|
||||
# Two views: comments from THIS round (after the trigger) and the full history. A fresh
|
||||
# launch posts after the trigger; an idempotent skip leaves the covering verdict in the
|
||||
# earlier run's comment, so fall back to history when that agent's head run is green.
|
||||
jq -s --arg t "$TRIGGER_TIME" '[.[][] | select(.created_at > $t)]' \
|
||||
<<<"$COMMENTS_RAW" > "$OUT_DIR/comments.json"
|
||||
jq -s '[.[][]]' <<<"$COMMENTS_RAW" > "$OUT_DIR/comments-all.json"
|
||||
# cubic posts through the PR reviews API, not issue comments.
|
||||
REVIEWS_RAW=$(retry gh api "repos/$REPO/pulls/$PR/reviews?per_page=100" --paginate)
|
||||
jq -s --arg t "$TRIGGER_TIME" '[.[][] | select((.submitted_at // "") > $t)]' \
|
||||
<<<"$REVIEWS_RAW" > "$OUT_DIR/pr-reviews.json"
|
||||
|
||||
VERDICT_RE='(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)'
|
||||
|
||||
body_by_header() { # <file> <header-substring>
|
||||
jq -r --arg h "$2" '[.[] | select(.body // "" | contains($h))] | last | .body // empty' "$1"
|
||||
}
|
||||
body_by_login() { # <file> <login>
|
||||
jq -r --arg l "$2" '[.[] | select(.user.login == $l)] | last | .body // empty' "$1"
|
||||
}
|
||||
head_ok() { [ "$(head_run_state "$1")" = "completed success" ]; }
|
||||
# Latest verdict for a reviewer: prefer this round's comment; if none and the reviewer's
|
||||
# head run succeeded (an idempotent /review skipped re-reviewing an already-green head),
|
||||
# fall back to the covering comment from the full history.
|
||||
verdict_body() { # <header|login> <value> <workflow>
|
||||
local body
|
||||
body=$("body_by_$1" "$OUT_DIR/comments.json" "$2")
|
||||
if [ -z "$body" ] && head_ok "$3"; then
|
||||
body=$("body_by_$1" "$OUT_DIR/comments-all.json" "$2")
|
||||
fi
|
||||
printf '%s' "$body"
|
||||
}
|
||||
report() { # <reviewer-name> <comment-body>
|
||||
local name=$1 body=$2 verdict
|
||||
if [ -z "$body" ]; then
|
||||
echo "$name: (no review posted for this head)"
|
||||
return
|
||||
fi
|
||||
printf '%s\n' "$body" > "$OUT_DIR/$name.md"
|
||||
verdict=$(printf '%s\n' "$body" | grep -m1 -oE "${VERDICT_RE}.*" | sed 's/\*\*//g' || true)
|
||||
echo "$name: ${verdict:-(review posted but no verdict line; read $OUT_DIR/$name.md)}"
|
||||
}
|
||||
|
||||
echo
|
||||
echo "=== Review round verdicts for $REPO#$PR (head $HEAD_SHA) ==="
|
||||
CODEX_BODY=$(verdict_body header '## Codex Review' codex-pr-review.yml)
|
||||
report codex "$CODEX_BODY"
|
||||
report claude "$(verdict_body login 'claude[bot]' pr-ready-review.yml)"
|
||||
report pi "$(verdict_body header '## Pi Review' pi-pr-review.yml)"
|
||||
CUBIC_BODY=$(jq -r '[.[] | select(.user.login | test("^cubic(-dev-ai)?(\\[bot\\])?$"; "i"))] | last | .body // empty' \
|
||||
"$OUT_DIR/pr-reviews.json")
|
||||
if [ -z "$CUBIC_BODY" ]; then
|
||||
CUBIC_BODY=$(jq -r '[.[] | select(.user.login | test("^cubic(-dev-ai)?(\\[bot\\])?$"; "i"))] | last | .body // empty' \
|
||||
"$OUT_DIR/comments.json")
|
||||
fi
|
||||
report cubic "$CUBIC_BODY"
|
||||
echo
|
||||
echo "Full round output: $OUT_DIR (comments.json, pr-reviews.json, one .md per reviewer)"
|
||||
if [ -z "$CODEX_BODY" ]; then
|
||||
echo "WARNING: no Codex verdict for $HEAD_SHA - its head run is not green (cancelled/failed/absent, not merely skipped-because-already-reviewed). Re-trigger with a '/codex' PR comment (re-runs the interrupted run in place, or launches one) and wait again." >&2
|
||||
fi
|
||||
@@ -1,39 +0,0 @@
|
||||
---
|
||||
name: refine
|
||||
user_invocable: true
|
||||
description: End-of-session reflection. Reviews friction encountered during the session and proposes updates to docs/ to capture lessons learned.
|
||||
---
|
||||
|
||||
# Refine Skill
|
||||
|
||||
Reflect on the current session and update documentation with lessons learned.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Identify friction**: Review what happened in this session:
|
||||
- Run `git diff main...HEAD --stat` to see what files were touched
|
||||
- Think about: what was slow, what failed, what required multiple attempts, what information was missing or hard to find
|
||||
|
||||
2. **Read current docs**: Read the docs that were relevant to this session:
|
||||
- `docs/validation.md`
|
||||
- `docs/enterprise.md`
|
||||
- `docs/autonomous-mode.md`
|
||||
- Any skills that were invoked
|
||||
|
||||
3. **Propose updates**: For each piece of friction, decide if it warrants a doc update:
|
||||
- **Missing knowledge**: Information you had to discover that should be documented
|
||||
- **Wrong guidance**: Instructions that led you astray
|
||||
- **Missing validation rule**: A check that should be in the validation matrix
|
||||
- **New pattern**: A codebase pattern worth capturing for next time
|
||||
|
||||
4. **Apply updates**: Edit the relevant `docs/` files. Keep changes minimal and specific — add only what would have saved time this session.
|
||||
|
||||
5. **Report**: Summarize what was added/changed and why.
|
||||
|
||||
## Rules
|
||||
|
||||
- Only add knowledge confirmed by this session — no speculative additions
|
||||
- Keep docs concise — add a line or two, not a paragraph
|
||||
- If a whole new doc is needed, create it in `docs/` and add a pointer in `CLAUDE.md`
|
||||
- Don't update skills unless a coding pattern was genuinely wrong
|
||||
- Don't add things Claude already knows — only Windmill-specific knowledge
|
||||
@@ -1,107 +0,0 @@
|
||||
---
|
||||
name: rust-backend
|
||||
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
|
||||
---
|
||||
|
||||
# Windmill Rust Patterns
|
||||
|
||||
Apply these Windmill-specific patterns when writing Rust code in `backend/`.
|
||||
|
||||
## Error Handling
|
||||
|
||||
Use `Error` from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>`:
|
||||
|
||||
```rust
|
||||
use windmill_common::error::{Error, Result};
|
||||
|
||||
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
|
||||
sqlx::query_as!(Job, "SELECT id, workspace_id FROM v2_job WHERE id = $1", id)
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
|
||||
}
|
||||
```
|
||||
|
||||
Never panic in library code. Reserve `.unwrap()` for compile-time guarantees.
|
||||
|
||||
## SQLx Patterns
|
||||
|
||||
**Never use `SELECT *`** — always list columns explicitly. Critical for backwards compatibility when workers lag behind API version:
|
||||
|
||||
```rust
|
||||
// Correct
|
||||
sqlx::query_as!(Job, "SELECT id, workspace_id, path FROM v2_job WHERE id = $1", id)
|
||||
|
||||
// Wrong — breaks when columns are added
|
||||
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", id)
|
||||
```
|
||||
|
||||
Use batch operations to avoid N+1:
|
||||
|
||||
```rust
|
||||
// Preferred — single query with IN clause
|
||||
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
|
||||
```
|
||||
|
||||
Use transactions for multi-step operations. Parameterize all queries.
|
||||
|
||||
## JSON Handling
|
||||
|
||||
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when storing/passing JSON without inspection:
|
||||
|
||||
```rust
|
||||
pub struct Job {
|
||||
pub args: Option<Box<serde_json::value::RawValue>>,
|
||||
}
|
||||
```
|
||||
|
||||
Only use `serde_json::Value` when you need to inspect or modify the JSON.
|
||||
|
||||
## Serde Optimizations
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub struct Job {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub parent_job: Option<Uuid>,
|
||||
#[serde(skip_serializing_if = "Vec::is_empty")]
|
||||
pub tags: Vec<String>,
|
||||
#[serde(default)]
|
||||
pub priority: i32,
|
||||
}
|
||||
```
|
||||
|
||||
## Async & Concurrency
|
||||
|
||||
Never block the async runtime. Use `spawn_blocking` for CPU-intensive work:
|
||||
|
||||
```rust
|
||||
let result = tokio::task::spawn_blocking(move || expensive_computation(&data)).await?;
|
||||
```
|
||||
|
||||
**Mutex selection**: Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) for data protection. Only use `tokio::sync::Mutex` when holding locks across `.await` points.
|
||||
|
||||
Use `tokio::sync::mpsc` (bounded) for channels. Avoid `std::thread::sleep` in async contexts.
|
||||
|
||||
## Module Structure & Visibility
|
||||
|
||||
- Use `pub(crate)` instead of `pub` when possible
|
||||
- Place new code in the appropriate crate based on functionality
|
||||
- API endpoints go in `windmill-api/src/` organized by domain
|
||||
- Shared functionality goes in `windmill-common/src/`
|
||||
|
||||
## Code Navigation
|
||||
|
||||
Always use rust-analyzer LSP for go-to-definition, find-references, and type info. Do not guess at module paths.
|
||||
|
||||
## Axum Handlers
|
||||
|
||||
Destructure extractors directly in function signatures:
|
||||
|
||||
```rust
|
||||
async fn process_job(
|
||||
Extension(db): Extension<DB>,
|
||||
Path((workspace, job_id)): Path<(String, Uuid)>,
|
||||
Query(pagination): Query<Pagination>,
|
||||
) -> Result<Json<Job>> { ... }
|
||||
```
|
||||
@@ -1,84 +0,0 @@
|
||||
---
|
||||
name: svelte-frontend
|
||||
description: Svelte coding guidelines for the Windmill frontend. MUST use when writing or modifying code in the frontend directory.
|
||||
---
|
||||
|
||||
# Windmill Svelte Patterns
|
||||
|
||||
Apply these Windmill-specific patterns when writing Svelte code in `frontend/`. For general Svelte 5 syntax (runes, snippets, event handling), use the Svelte MCP server.
|
||||
|
||||
## Windmill UI Components (MUST use)
|
||||
|
||||
Always use Windmill's design-system components. Never use raw HTML elements.
|
||||
|
||||
### Buttons — `<Button>`
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { Button } from '$lib/components/common'
|
||||
import { ChevronLeft } from 'lucide-svelte'
|
||||
</script>
|
||||
|
||||
<Button variant="default" onclick={handleClick}>Label</Button>
|
||||
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prev} />
|
||||
```
|
||||
|
||||
Props: `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`, `unifiedSize?: 'sm' | 'md' | 'lg'`, `startIcon?: { icon: SvelteComponent }`, `iconOnly?: boolean`, `disabled?: boolean`
|
||||
|
||||
### Text inputs — `<TextInput>`
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { TextInput } from '$lib/components/common'
|
||||
</script>
|
||||
|
||||
<TextInput bind:value={val} placeholder="Enter value" />
|
||||
```
|
||||
|
||||
Props: `value?: string | number` (bindable), `placeholder?: string`, `disabled?: boolean`, `error?: string | boolean`, `size?: 'sm' | 'md' | 'lg'`
|
||||
|
||||
### Selects — `<Select>`
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import Select from '$lib/components/select/Select.svelte'
|
||||
</script>
|
||||
|
||||
<Select items={[{ label: 'Jan', value: 1 }]} bind:value={selected} />
|
||||
```
|
||||
|
||||
Props: `items?: Array<{ label?: string; value: any }>`, `value` (bindable), `placeholder?: string`, `clearable?: boolean`, `size?: 'sm' | 'md' | 'lg'`
|
||||
|
||||
### Icons — `lucide-svelte`
|
||||
|
||||
Never write inline SVGs. Import from `lucide-svelte`:
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { ChevronLeft, X } from 'lucide-svelte'
|
||||
</script>
|
||||
<ChevronLeft size={16} />
|
||||
```
|
||||
|
||||
## Form Components
|
||||
|
||||
Form components (TextInput, Toggle, Select, etc.) should use the unified size system when placed together.
|
||||
|
||||
## Styling
|
||||
|
||||
- Use Tailwind CSS for all styling — no custom CSS
|
||||
- Use Windmill's theming classes for colors/surfaces (see `frontend/brand-guidelines.md`)
|
||||
- Read component props JSDoc before using them
|
||||
|
||||
## Svelte MCP Server
|
||||
|
||||
Use the Svelte MCP tools when working on Svelte code:
|
||||
|
||||
1. **list-sections**: Call first to discover available docs
|
||||
2. **get-documentation**: Fetch relevant sections based on use_cases
|
||||
3. **svelte-autofixer**: MUST use on all Svelte code before finalizing — keep calling until no issues
|
||||
4. **playground-link**: Only after user confirms and code was NOT written to project files
|
||||
|
||||
## Verifying in the Browser
|
||||
|
||||
After changing Svelte code, use the **Playwright MCP** (`mcp__playwright__*`) to drive the running frontend and confirm the change works. See AGENTS.md → "Verifying Frontend Changes" for the full flow. Use `playwright` (headless) on devboxes; `playwright-headed` when a display is available.
|
||||
@@ -1,82 +0,0 @@
|
||||
---
|
||||
name: update-sqlx
|
||||
description: How to safely update SQLx offline query cache. MUST use when SQL queries change.
|
||||
---
|
||||
|
||||
# SQLx Offline Query Cache
|
||||
|
||||
Windmill uses `SQLX_OFFLINE=true` in CI, which requires all `sqlx::query!` / `sqlx::query_as!` macros to have matching cached query data in `backend/.sqlx/`.
|
||||
|
||||
## When to Run
|
||||
|
||||
Run after any change to SQL queries in Rust source files. Without it, CI will fail with:
|
||||
```
|
||||
error: `SQLX_OFFLINE=true` but there is no cached data for this query
|
||||
```
|
||||
|
||||
## The Problem
|
||||
|
||||
`cargo sqlx prepare --workspace` **deletes all existing cache files** and regenerates only the ones found in the current compilation. If you don't compile with every feature flag (especially `private` for EE files), you will **silently delete EE query caches**, breaking CI for enterprise tests.
|
||||
|
||||
The standard `./update_sqlx.sh` script tries to compile with all features, but it often fails locally because the EE symlinks can be out of sync with `main`.
|
||||
|
||||
## Safe Procedure
|
||||
|
||||
Always preserve the existing EE caches from `origin/main`. Use this workflow:
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
|
||||
# 1. Restore the full cache from main (includes EE caches)
|
||||
git checkout origin/main -- .sqlx/
|
||||
|
||||
# 2. Run prepare with OSS features (what compiles locally)
|
||||
# This regenerates OSS caches to match your code changes.
|
||||
cargo sqlx prepare --workspace -- --workspace --features all_sqlx_features
|
||||
|
||||
# 3. Restore any EE caches that were deleted in step 2.
|
||||
# These are files present in origin/main but missing after prepare.
|
||||
git ls-tree origin/main backend/.sqlx/ \
|
||||
| awk '{print $4}' | sed 's|backend/\.sqlx/||' | sort > /tmp/main_files.txt
|
||||
|
||||
find backend/.sqlx -name "*.json" -printf '%P\n' | sort > /tmp/current_files.txt
|
||||
|
||||
comm -23 /tmp/main_files.txt /tmp/current_files.txt > /tmp/missing_files.txt
|
||||
|
||||
while read f; do
|
||||
git show "origin/main:backend/.sqlx/$f" > "backend/.sqlx/$f"
|
||||
done < /tmp/missing_files.txt
|
||||
|
||||
# 4. Verify nothing was lost from main
|
||||
find backend/.sqlx -name "*.json" -printf '%P\n' | sort > /tmp/current_files.txt
|
||||
comm -23 /tmp/main_files.txt /tmp/current_files.txt | wc -l
|
||||
# Should output: 0
|
||||
```
|
||||
|
||||
## If EE Compiles Locally
|
||||
|
||||
If your EE repo happens to be in sync, you can use the full script (faster):
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
./update_sqlx.sh
|
||||
```
|
||||
|
||||
But if it fails with EE compilation errors, use the safe procedure above.
|
||||
|
||||
## What NOT to Do
|
||||
|
||||
- **Never** run `cargo sqlx prepare --workspace` with only OSS features and commit the result — it will delete EE caches.
|
||||
- **Never** set `SQLX_OFFLINE=true` for local `cargo sqlx prepare` — use a live database per CLAUDE.md. (CI runs with `SQLX_OFFLINE=true`, which is why the cache must be complete.)
|
||||
- **Never** skip the verification step (step 4 above).
|
||||
|
||||
## Verification
|
||||
|
||||
After committing, the diff against `origin/main` should show:
|
||||
- A few **new** cache files (for your changed queries)
|
||||
- A few **deleted** cache files (for old queries that no longer exist)
|
||||
- **Zero** net deletions from the EE cache set
|
||||
|
||||
```bash
|
||||
git diff origin/main --stat backend/.sqlx/
|
||||
```
|
||||
@@ -1,3 +0,0 @@
|
||||
/*
|
||||
!/backend/
|
||||
!/frontend/
|
||||
@@ -1,127 +0,0 @@
|
||||
---
|
||||
name: branch-diff-reviewer
|
||||
description: Use this agent when you want a comprehensive code review of changes in the current branch compared to main. This includes reviewing for bugs, optimization opportunities, code style issues, potential mistakes, and adherence to project conventions. The agent should be invoked after completing a feature branch or before creating a pull request.\n\nExamples:\n\n<example>\nContext: User has finished implementing a new feature and wants feedback before merging.\nuser: "I've finished the new kafka trigger implementation, can you review my changes?"\nassistant: "I'll use the branch-diff-reviewer agent to analyze your changes against the main branch and provide comprehensive feedback."\n<commentary>\nSince the user wants a review of their branch changes, use the Task tool to launch the branch-diff-reviewer agent to compare the current branch against main and provide detailed feedback.\n</commentary>\n</example>\n\n<example>\nContext: User wants to check their code quality before submitting a PR.\nuser: "Review my branch before I create a PR"\nassistant: "Let me launch the branch-diff-reviewer agent to examine all your changes and identify any issues or improvements."\n<commentary>\nThe user is preparing for a PR, so use the branch-diff-reviewer agent to provide a thorough review of all branch differences.\n</commentary>\n</example>\n\n<example>\nContext: User is unsure if their implementation follows project patterns.\nuser: "Does my implementation look correct? I'm not sure if I followed the existing patterns"\nassistant: "I'll use the branch-diff-reviewer agent to compare your changes against main and check for pattern consistency, potential issues, and optimization opportunities."\n<commentary>\nThe user needs validation of their implementation against project standards. Launch the branch-diff-reviewer agent to analyze the diff and provide feedback on patterns, correctness, and improvements.\n</commentary>\n</example>
|
||||
tools: Glob, Grep, Read, WebFetch, TodoWrite, WebSearch, ListMcpResourcesTool, ReadMcpResourceTool, mcp__svelte__get-documentation, mcp__svelte__list-sections, mcp__svelte__playground-link, mcp__svelte__svelte-autofixer, mcp__ide__getDiagnostics, mcp__ide__executeCode, Bash, Skill
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are an elite code reviewer with deep expertise in software engineering best practices, performance optimization, and security. Your role is to provide thorough, actionable feedback on code changes between the current branch and main.
|
||||
|
||||
## Your Review Process
|
||||
|
||||
1. **First, gather the diff**: Use git commands to obtain the complete diff between the current branch and main:
|
||||
- Run `git diff main...HEAD` to see all changes
|
||||
- Run `git log main..HEAD --oneline` to understand the commit history
|
||||
- Identify all modified, added, and deleted files
|
||||
|
||||
2. **Analyze each changed file** in the context of:
|
||||
- The project's established patterns (check CLAUDE.md and related documentation)
|
||||
- The file's purpose and its role in the broader codebase
|
||||
- Dependencies and how changes might affect other parts of the system
|
||||
|
||||
## Review Categories
|
||||
|
||||
For each significant change, evaluate and report on:
|
||||
|
||||
### 🐛 Bugs & Correctness
|
||||
- Logic errors or edge cases not handled
|
||||
- Null/undefined handling issues
|
||||
- Race conditions in async code
|
||||
- Incorrect error handling
|
||||
- Type mismatches or unsafe casts
|
||||
|
||||
### ⚡ Performance
|
||||
- Inefficient algorithms or data structures
|
||||
- N+1 query problems in database code
|
||||
- Unnecessary re-renders in frontend code
|
||||
- Missing indexes for database queries
|
||||
- Blocking operations in async contexts
|
||||
- Memory leaks or excessive allocations
|
||||
- For Rust: Check for unnecessary clones, inefficient serde usage, blocking in async
|
||||
- For Svelte: Check for inefficient reactivity, missing keys in loops, excessive effects
|
||||
|
||||
### 🔒 Security
|
||||
- SQL injection vulnerabilities
|
||||
- Missing input validation
|
||||
- Exposed sensitive data
|
||||
- Authentication/authorization gaps
|
||||
- Unsafe deserialization
|
||||
|
||||
### 📐 Code Quality & Style
|
||||
- Adherence to project conventions (CLAUDE.md guidelines)
|
||||
- Code duplication that should be refactored
|
||||
- Unclear or misleading naming
|
||||
- Missing or inadequate documentation
|
||||
- Overly complex logic that could be simplified
|
||||
- Dead code or unused imports
|
||||
|
||||
### 🏗️ Architecture & Design
|
||||
- Proper separation of concerns
|
||||
- Appropriate use of existing utilities vs. new code
|
||||
- Consistency with established patterns
|
||||
- Proper error propagation
|
||||
- API design issues
|
||||
|
||||
### 🧪 Testing Considerations
|
||||
- Suggest test cases for new functionality
|
||||
- Identify untested edge cases
|
||||
- Note if changes break existing test assumptions
|
||||
|
||||
## Project-Specific Rules
|
||||
|
||||
### For Rust (Backend)
|
||||
- Verify `SELECT` statements list explicit columns (never `SELECT *` in worker code)
|
||||
- Check for proper use of `sqlx` with parameterized queries
|
||||
- Ensure errors use the custom `Error` enum from `windmill-common::error`
|
||||
- Verify async code doesn't block the tokio runtime
|
||||
- Check serde attributes for optimal serialization
|
||||
- Ensure openapi.yaml is updated for API changes
|
||||
|
||||
### For Svelte (Frontend)
|
||||
- For Svelte 5 files: Verify proper use of Runes (`$state`, `$derived`, `$effect`)
|
||||
- Check for `key` attributes in `{#each}` blocks
|
||||
- Ensure event handlers use the new syntax (`onclick` not `on:click`) in Svelte 5
|
||||
- Verify snippets are used instead of slots in Svelte 5
|
||||
- Check for proper props declaration with `$props()`
|
||||
|
||||
## Output Format
|
||||
|
||||
Structure your review as follows:
|
||||
|
||||
```
|
||||
## Summary
|
||||
[Brief overview of the changes and overall assessment]
|
||||
|
||||
## Critical Issues 🚨
|
||||
[Issues that must be fixed before merging]
|
||||
|
||||
## Recommendations 💡
|
||||
[Improvements that would significantly enhance the code]
|
||||
|
||||
## Minor Suggestions 📝
|
||||
[Nice-to-haves and style improvements]
|
||||
|
||||
## Positive Observations ✅
|
||||
[Well-done aspects worth acknowledging]
|
||||
|
||||
## File-by-File Details
|
||||
[Detailed feedback organized by file]
|
||||
```
|
||||
|
||||
For each issue, provide:
|
||||
1. **Location**: File path and line number(s)
|
||||
2. **Issue**: Clear description of the problem
|
||||
3. **Impact**: Why this matters
|
||||
4. **Suggestion**: Concrete fix or improvement with code example when helpful
|
||||
|
||||
## Behavioral Guidelines
|
||||
|
||||
- Be thorough but prioritize: focus most on critical issues
|
||||
- Be constructive: every criticism should come with a suggestion
|
||||
- Be specific: vague feedback is not actionable
|
||||
- Acknowledge good work: positive reinforcement matters
|
||||
- Consider context: understand why decisions might have been made
|
||||
- Ask clarifying questions if the intent of changes is unclear
|
||||
- Reference project documentation when pointing out convention violations
|
||||
|
||||
Begin by fetching the diff and then proceed with your comprehensive review.
|
||||
@@ -1,76 +0,0 @@
|
||||
---
|
||||
name: openapi-sync
|
||||
description: Use this agent when backend API endpoints are added, modified, or removed, or when the Flow structure changes and the OpenAPI specification files need to be updated. This includes changes to route handlers in windmill-api, modifications to request/response schemas, changes to authentication requirements, or updates to the Flow data structures that affect the API.\n\nExamples:\n\n<example>\nContext: User has just added a new API endpoint for managing workspace templates.\nuser: "Add a new endpoint POST /api/w/{workspace}/templates to create workspace templates"\nassistant: "I've created the new endpoint handler in windmill-api. Now let me use the openapi-sync agent to update the OpenAPI specification files."\n<commentary>\nSince a new API endpoint was added, use the openapi-sync agent to ensure the openapi.yaml and openflow.openapi.yaml files are updated with the new endpoint definition.\n</commentary>\n</example>\n\n<example>\nContext: User has modified the response schema for an existing endpoint.\nuser: "Update the GET /api/w/{workspace}/flows endpoint to also return a 'versions' array in the response"\nassistant: "I've updated the flow listing endpoint to include the versions array. Now let me use the openapi-sync agent to update the OpenAPI specification."\n<commentary>\nSince the response schema of an existing endpoint was modified, use the openapi-sync agent to update the corresponding schema in the OpenAPI files.\n</commentary>\n</example>\n\n<example>\nContext: User has made changes to the Flow structure in the codebase.\nuser: "Add a new 'retry_policy' field to the Flow value structure"\nassistant: "I've added the retry_policy field to the Flow struct. Now let me use the openapi-sync agent to update the OpenAPI specification to reflect this schema change."\n<commentary>\nSince the Flow structure was modified, use the openapi-sync agent to ensure the flow-related schemas in openapi.yaml and openflow.openapi.yaml are updated.\n</commentary>\n</example>
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are an expert API documentation engineer specializing in OpenAPI specifications for the Windmill platform. Your primary responsibility is to maintain synchronization between the Rust backend API implementation and the OpenAPI specification files.
|
||||
|
||||
## Your Core Responsibilities
|
||||
|
||||
1. **Update OpenAPI Specifications**: When API endpoints are added, modified, or removed in the windmill-api crate, you must update:
|
||||
- `backend/windmill-api/openapi.yaml` - The main OpenAPI specification
|
||||
- `backend/windmill-api/openflow.openapi.yaml` - Flow-specific OpenAPI definitions (if flow-related changes)
|
||||
|
||||
2. **Maintain Schema Accuracy**: Ensure all request/response schemas accurately reflect the Rust structs used in the API handlers.
|
||||
|
||||
3. **Document Comprehensively**: Include proper descriptions, examples, and parameter documentation.
|
||||
|
||||
## Key Files to Reference
|
||||
|
||||
- **API Route Definitions**: Look in `backend/windmill-api/src/` for route handlers organized by domain
|
||||
- **Data Structures**: Check `backend/windmill-common/src/` for shared structs and types
|
||||
- **Database Schema**: Reference `backend/summarized_schema.txt` for understanding data models
|
||||
- **Existing OpenAPI Files**: Always review the current state of `openapi.yaml` and `openflow.openapi.yaml` before making changes
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Identify Changes**: Determine what API changes were made by examining:
|
||||
- New or modified route handlers in windmill-api
|
||||
- Changes to request/response structs
|
||||
- Modifications to the Flow structure or related types
|
||||
|
||||
2. **Analyze the Implementation**: For each endpoint, identify:
|
||||
- HTTP method and path
|
||||
- Path parameters, query parameters, and request body schema
|
||||
- Response schema(s) and status codes
|
||||
- Authentication requirements
|
||||
- Any tags or groupings
|
||||
|
||||
3. **Update OpenAPI Files**:
|
||||
- Add or modify path definitions with accurate operation IDs
|
||||
- Update or create schema definitions in the components section
|
||||
- Ensure $ref references are correct
|
||||
- Maintain consistent naming conventions with existing patterns
|
||||
|
||||
4. **Validate Changes**: Ensure the YAML syntax is valid and follows OpenAPI 3.0 specification.
|
||||
|
||||
## OpenAPI Conventions for Windmill
|
||||
|
||||
- **Operation IDs**: Use camelCase, descriptive names (e.g., `createScript`, `listFlows`, `updateWorkspaceSettings`)
|
||||
- **Tags**: Group endpoints by domain (e.g., `scripts`, `flows`, `workspaces`, `users`)
|
||||
- **Schema Naming**: Use PascalCase for schema names matching Rust struct names
|
||||
- **Path Parameters**: Use `{workspace}` for workspace_id, maintain consistency with existing patterns
|
||||
- **Security**: Most endpoints require Bearer token authentication - include appropriate security requirements
|
||||
|
||||
## Schema Mapping from Rust to OpenAPI
|
||||
|
||||
- `String` / `&str` → `type: string`
|
||||
- `i32`, `i64` → `type: integer` (with appropriate format)
|
||||
- `f32`, `f64` → `type: number`
|
||||
- `bool` → `type: boolean`
|
||||
- `Vec<T>` → `type: array` with `items`
|
||||
- `Option<T>` → property is not in `required` array
|
||||
- `HashMap<K, V>` → `type: object` with `additionalProperties`
|
||||
- Enums → `type: string` with `enum` array
|
||||
- Custom structs → `$ref` to schema definition
|
||||
|
||||
## Important Notes
|
||||
|
||||
- Always preserve existing documentation and descriptions when updating
|
||||
- Maintain backward compatibility warnings in descriptions when applicable
|
||||
- Include example values where they aid understanding
|
||||
- For Flow-related changes, update BOTH openapi.yaml AND openflow.openapi.yaml as needed
|
||||
- Follow the existing indentation and formatting style in the YAML files
|
||||
|
||||
When you complete updates, summarize what changes were made to which files and highlight any schema additions or modifications that downstream consumers should be aware of.
|
||||
@@ -1,22 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Format backend Rust files with rustfmt after Claude edits them
|
||||
|
||||
# Get the file path from the tool result (passed via stdin as JSON)
|
||||
INPUT=$(cat)
|
||||
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
|
||||
|
||||
# Exit if no file path
|
||||
if [ -z "$FILE_PATH" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if the file is in the backend directory and is a Rust file
|
||||
if [[ "$FILE_PATH" == *"/backend/"* ]] && [[ "$FILE_PATH" =~ \.rs$ ]]; then
|
||||
cd "$CLAUDE_PROJECT_DIR/backend" || exit 0
|
||||
# Run rustfmt, surface errors as context but don't block Claude
|
||||
if rustfmt --config-path rustfmt.toml "$FILE_PATH" 2>&1; then
|
||||
echo "Formatted $(basename "$FILE_PATH")"
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,25 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Format frontend files with prettier after Claude edits them
|
||||
|
||||
# Get the file path from the tool result (passed via stdin as JSON)
|
||||
INPUT=$(cat)
|
||||
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
|
||||
|
||||
# Exit if no file path
|
||||
if [ -z "$FILE_PATH" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if the file is in the frontend directory
|
||||
if [[ "$FILE_PATH" == *"/frontend/"* ]]; then
|
||||
# Check if it's a formattable file type
|
||||
if [[ "$FILE_PATH" =~ \.(ts|js|svelte|json|css|html|md)$ ]]; then
|
||||
cd "$CLAUDE_PROJECT_DIR/frontend" || exit 0
|
||||
# Run prettier, surface errors as context but don't block Claude
|
||||
if ./node_modules/.bin/prettier --plugin prettier-plugin-svelte --write "$FILE_PATH" 2>&1; then
|
||||
echo "Formatted $(basename "$FILE_PATH")"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,38 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# PreToolUse hook: block destructive git operations when on the main branch.
|
||||
# Non-git tool calls and read-only git commands pass through silently.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
input="$(cat)"
|
||||
tool_name="$(echo "$input" | jq -r '.tool_name // empty')"
|
||||
|
||||
# Only care about Bash tool calls
|
||||
[[ "$tool_name" == "Bash" ]] || exit 0
|
||||
|
||||
command="$(echo "$input" | jq -r '.tool_input.command // empty')"
|
||||
|
||||
# Only care about git write commands
|
||||
if [[ "$command" =~ ^git\ (push|reset|revert|checkout|merge|rebase|commit|add) ]]; then
|
||||
branch="$(git rev-parse --abbrev-ref HEAD 2>/dev/null || true)"
|
||||
if [[ "$branch" == "main" ]]; then
|
||||
echo "BLOCK: You are on the main branch. Create or switch to a feature branch first." >&2
|
||||
exit 2
|
||||
fi
|
||||
fi
|
||||
|
||||
# Block force-push targeting main from any branch.
|
||||
if [[ "$command" =~ ^git[[:space:]]+push([[:space:]]|$) ]]; then
|
||||
has_force=false
|
||||
if [[ "$command" =~ (--force([[:space:]]|=|$)|--force-with-lease|[[:space:]]-f([[:space:]]|$)) ]]; then
|
||||
has_force=true
|
||||
fi
|
||||
# `+ref` refspec syntax is also a force push.
|
||||
if [[ "$command" =~ [[:space:]]\+[A-Za-z] ]]; then
|
||||
has_force=true
|
||||
fi
|
||||
if $has_force && [[ "$command" =~ (^|[[:space:]:])\+?main([[:space:]]|$) ]]; then
|
||||
echo "BLOCK: Force-push to main is not allowed via Claude. Run it yourself if you really mean to." >&2
|
||||
exit 2
|
||||
fi
|
||||
fi
|
||||
@@ -1,25 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Notify user when Claude requires input (works on macOS and Linux)
|
||||
|
||||
# Check if we're in an SSH session
|
||||
if [[ -n "$SSH_CLIENT" || -n "$SSH_TTY" || -n "$SSH_CONNECTION" ]]; then
|
||||
# SSH session - use terminal bell
|
||||
# If using VSCode, enable audible terminal bell for SSH sessions:
|
||||
# Add the following to .vscode/settings.json:
|
||||
# "accessibility.signals.terminalBell": {
|
||||
# "sound": "on"
|
||||
# },
|
||||
# "terminal.integrated.enableVisualBell": true
|
||||
printf '\a'
|
||||
else
|
||||
# Local session - use native notifications
|
||||
if [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
osascript -e 'display notification "Claude is waiting for your input" with title "Claude Code" sound name "Glass"' 2>/dev/null || printf '\a'
|
||||
elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
|
||||
notify-send "Claude Code" "Claude is waiting for your input" 2>/dev/null || printf '\a'
|
||||
else
|
||||
printf '\a'
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,4 +0,0 @@
|
||||
# Claude output format
|
||||
|
||||
- Use inline comments at the relevant lines for specific issues.
|
||||
- Use a top-level comment for the summary, severity-tagged finding list, AGENTS.md compliance check, and the test-coverage assessment.
|
||||
@@ -1,149 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"additionalDirectories": [
|
||||
"../windmill-ee-private"
|
||||
],
|
||||
"allow": [
|
||||
"Bash(ls:*)",
|
||||
"Bash(grep:*)",
|
||||
"Bash(cat:*)",
|
||||
"Bash(head:*)",
|
||||
"Bash(tail:*)",
|
||||
"Bash(less:*)",
|
||||
"Bash(more:*)",
|
||||
"Bash(find:*)",
|
||||
"Bash(wc:*)",
|
||||
"Bash(diff:*)",
|
||||
"Bash(file:*)",
|
||||
"Bash(stat:*)",
|
||||
"Bash(tree:*)",
|
||||
"Bash(pwd)",
|
||||
"Bash(which:*)",
|
||||
"Bash(whereis:*)",
|
||||
"Bash(echo:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git log:*)",
|
||||
"Bash(git branch:*)",
|
||||
"Bash(git show:*)",
|
||||
"Bash(git blame:*)",
|
||||
"Bash(cargo check:*)",
|
||||
"Bash(cargo build --release:*)",
|
||||
"Bash(sh wm-ts-nav/nav:*)",
|
||||
"Bash(wm-ts-nav/nav:*)",
|
||||
"Bash(./wm-ts-nav/nav:*)",
|
||||
"Bash(wm-ts-nav/target/release/wm-ts-nav:*)",
|
||||
"Bash(./wm-ts-nav/target/release/wm-ts-nav:*)",
|
||||
"mcp__ide__getDiagnostics",
|
||||
"Bash(npm run generate-backend-client:*)",
|
||||
"Bash(npm run check:*)",
|
||||
"Bash(git push:*)",
|
||||
"Bash(git reset:*)",
|
||||
"Bash(git revert:*)",
|
||||
"Bash(git checkout:*)",
|
||||
"Bash(git merge:*)",
|
||||
"Bash(git rebase:*)",
|
||||
"Bash(git add:*)",
|
||||
"Bash(git commit:*)",
|
||||
"Read(/tmp/**)",
|
||||
"Write(/tmp/**)",
|
||||
"Edit(/tmp/**)",
|
||||
"Bash(rm:/tmp/*)",
|
||||
"Bash(rm:/tmp/**)",
|
||||
"Bash(rmdir:/tmp/*)",
|
||||
"Bash(mkdir:/tmp/*)",
|
||||
"Bash(mkdir:/tmp/**)",
|
||||
"Bash(cp:/tmp/*)",
|
||||
"Bash(cp:/tmp/**)",
|
||||
"Bash(mv:/tmp/*)",
|
||||
"Bash(mv:/tmp/**)",
|
||||
"Bash(touch:/tmp/*)",
|
||||
"Bash(touch:/tmp/**)",
|
||||
"Bash(chmod:/tmp/*)",
|
||||
"Bash(chmod:/tmp/**)",
|
||||
"Bash(tar * /tmp/*)",
|
||||
"Bash(unzip * /tmp/*)"
|
||||
],
|
||||
"deny": [
|
||||
"Read(.env)",
|
||||
"Read(.env.*)",
|
||||
"Read(**/.env)",
|
||||
"Read(**/.env.*)",
|
||||
"Read(**/secrets/**)",
|
||||
"Read(**/*.pem)",
|
||||
"Read(**/*.key)",
|
||||
"Read(**/credentials.json)",
|
||||
"Read(**/.secret*)",
|
||||
"Read(**/.secrets*)",
|
||||
"Read(**/*.secret)",
|
||||
"Read(**/*.secrets)",
|
||||
"Edit(.env)",
|
||||
"Edit(.env.*)",
|
||||
"Edit(**/.env)",
|
||||
"Edit(**/.env.*)"
|
||||
],
|
||||
"ask": [
|
||||
"Bash(rm:*)",
|
||||
"Bash(rmdir:*)",
|
||||
"Bash(mv:*)",
|
||||
"Bash(chmod:*)",
|
||||
"Bash(chown:*)",
|
||||
"Bash(truncate:*)",
|
||||
"Bash(shred:*)",
|
||||
"Bash(unlink:*)",
|
||||
"mcp__claude_ai_Stripe",
|
||||
"mcp__claude_ai_Gmail",
|
||||
"mcp__claude_ai_Google_Calendar",
|
||||
"mcp__claude_ai_Google_Drive",
|
||||
"mcp__claude_ai_Slack",
|
||||
"mcp__claude_ai_Linear"
|
||||
]
|
||||
},
|
||||
"enableAllProjectMcpServers": true,
|
||||
"hooks": {
|
||||
"PreToolUse": [
|
||||
{
|
||||
"matcher": "Bash",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/guard-main-branch.sh",
|
||||
"timeout": 5
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Edit|Write",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-frontend.sh",
|
||||
"timeout": 30
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-backend.sh",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"Notification": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/notify-user.sh",
|
||||
"timeout": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"enabledPlugins": {
|
||||
"typescript-lsp@claude-plugins-official": true,
|
||||
"code-review@claude-plugins-official": true
|
||||
}
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/adding-a-trigger/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/ai-chat/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/ai-evals/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/commit/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/local-review-codex/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/local-review/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/native-trigger/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/pr/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/refine/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/rust-backend/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/svelte-frontend/SKILL.md
|
||||
@@ -1 +0,0 @@
|
||||
../../../.agents/skills/update-sqlx/SKILL.md
|
||||
@@ -1,4 +0,0 @@
|
||||
#:schema https://developers.openai.com/codex/config-schema.json
|
||||
|
||||
[mcp_servers.svelte]
|
||||
url = "https://mcp.svelte.dev/mcp"
|
||||
@@ -7,6 +7,7 @@ services:
|
||||
# image: mcr.microsoft.com/vscode/devcontainers/rust:bullseye
|
||||
environment:
|
||||
- DENO_PATH=/usr/local/cargo/bin/deno
|
||||
- PYTHON_PATH=/usr/bin/python3
|
||||
- NSJAIL_PATH=/bin/nsjail
|
||||
volumes:
|
||||
- .:/workspace:cached
|
||||
|
||||
@@ -3,4 +3,3 @@ frontend/build/
|
||||
frontend/.svelte-kit/
|
||||
|
||||
backend/target/
|
||||
backend/windmill-duckdb-ffi-internal/target/
|
||||
@@ -7,7 +7,3 @@ WM_IMAGE=ghcr.io/windmill-labs/windmill:main
|
||||
|
||||
# To use another port than :80, setup the Caddyfile and the caddy section of the docker-compose to your needs: https://caddyserver.com/docs/getting-started
|
||||
# To have caddy take care of automatic TLS
|
||||
|
||||
# To rotate logs, set the following variables:
|
||||
#LOG_MAX_SIZE=10m
|
||||
#LOG_MAX_FILE=3
|
||||
@@ -1,7 +1 @@
|
||||
use flake
|
||||
|
||||
# Per-worktree overrides (ports, DATABASE_URL, etc.) written by webmux/workmux
|
||||
# post-create hooks. Must come after `use flake` so they take precedence over
|
||||
# the flake's defaults.
|
||||
# shellcheck source=/dev/null
|
||||
[ -f .env.local ] && source .env.local
|
||||
use nix
|
||||
|
||||
+3
-3
@@ -1,4 +1,4 @@
|
||||
* @rubenfiszel @hugocasa @alpetric
|
||||
* @rubenfiszel
|
||||
|
||||
/community/ @rubenfiszel @hugocasa @alpetric
|
||||
/frontend/ @rubenfiszel @hugocasa @alpetric
|
||||
/community/ @fatonramadani @rubenfiszel
|
||||
/frontend/ @fatonramadani @rubenfiszel
|
||||
|
||||
@@ -27,46 +27,29 @@ RUN wget https://golang.org/dl/go1.21.5.linux-amd64.tar.gz && tar -C /usr/local
|
||||
ENV PATH="${PATH}:/usr/local/go/bin"
|
||||
ENV GO_PATH=/usr/local/go/bin/go
|
||||
|
||||
# UV
|
||||
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.11.24/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
|
||||
# Install UV
|
||||
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.4.18/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
|
||||
|
||||
ENV TZ=Etc/UTC
|
||||
|
||||
ENV PYTHON_VERSION 3.11.4
|
||||
|
||||
# Python
|
||||
RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz \
|
||||
&& tar -xf Python-${PYTHON_VERSION}.tgz && cd Python-${PYTHON_VERSION}/ && ./configure --enable-optimizations \
|
||||
&& make -j 4 && make install
|
||||
|
||||
RUN /usr/local/bin/python3 -m pip install pip-tools
|
||||
|
||||
# Bun
|
||||
COPY --from=oven/bun:1.3.10 /usr/local/bin/bun /usr/bin/bun
|
||||
|
||||
# Install windmill CLI
|
||||
RUN bun install -g windmill-cli \
|
||||
&& ln -s $(bun pm bin -g)/wmill /usr/bin/wmill
|
||||
COPY --from=oven/bun:1.1.31 /usr/local/bin/bun /usr/bin/bun
|
||||
|
||||
ARG TARGETPLATFORM
|
||||
|
||||
# Deno
|
||||
RUN curl -Lsf https://github.com/denoland/deno/releases/download/v2.0.2/deno-x86_64-unknown-linux-gnu.zip -o deno.zip
|
||||
# RUN [ "$TARGETPLATFORM" == "linux/arm64" ] && curl -Lsf https://github.com/denoland/deno/releases/download/v2.0.0/deno-aarch64-unknown-linux-gnu.zip -o deno.zip || true
|
||||
|
||||
RUN unzip deno.zip && rm deno.zip && mv deno /usr/bin/deno
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y postgresql-client --allow-unauthenticated
|
||||
|
||||
RUN rustup component add rustfmt
|
||||
|
||||
# C#
|
||||
RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh \
|
||||
&& chmod +x dotnet-install.sh \
|
||||
&& ./dotnet-install.sh --channel 9.0 --install-dir /usr/share/dotnet \
|
||||
&& ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet \
|
||||
&& rm dotnet-install.sh
|
||||
|
||||
|
||||
# Nushell
|
||||
COPY --from=ghcr.io/nushell/nushell:0.101.0-bookworm /usr/bin/nu /usr/bin/nu
|
||||
RUN rustup component add rustfmt
|
||||
@@ -7,7 +7,7 @@ VERSION=$1
|
||||
echo "Updating versions to: $VERSION"
|
||||
|
||||
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
|
||||
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/src/core/constants.ts
|
||||
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/main.ts
|
||||
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
|
||||
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
|
||||
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
|
||||
@@ -15,15 +15,12 @@ sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescrip
|
||||
sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
|
||||
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
|
||||
sed -i '' -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
|
||||
sed -i '' -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
|
||||
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
|
||||
sed -i '' -e "/^ModuleVersion =/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
|
||||
# sed -i '' -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill_pg/pyproject.toml
|
||||
sed -i '' -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
|
||||
sed -i '' -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
|
||||
|
||||
sed -i '' -E "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
|
||||
|
||||
# windmill-parser-wasm is its own workspace (excluded from the backend workspace
|
||||
# because of nightly-only cargo-features), so its version lives in
|
||||
# [workspace.package] and its Cargo.lock is not regenerated by the backend step.
|
||||
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.toml
|
||||
sed -i '' -E "s/(name = \"windmill[^\"]*\"\nversion = )\"[^\"]*\"/\\1\"$VERSION\"/g" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.lock
|
||||
|
||||
cd ${root_dirpath}/frontend && npm i --package-lock-only
|
||||
|
||||
@@ -7,7 +7,7 @@ VERSION=$1
|
||||
echo "Updating versions to: $VERSION"
|
||||
|
||||
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
|
||||
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/src/core/constants.ts
|
||||
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/main.ts
|
||||
sed -i -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
|
||||
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
|
||||
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
|
||||
@@ -16,15 +16,12 @@ sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-c
|
||||
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
|
||||
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
|
||||
sed -i -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
|
||||
sed -i -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
|
||||
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
|
||||
sed -i -e "/^ModuleVersion =/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
|
||||
# sed -i -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
|
||||
sed -i -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
|
||||
sed -i -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
|
||||
|
||||
sed -i -zE "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
|
||||
|
||||
# windmill-parser-wasm is its own workspace (excluded from the backend workspace
|
||||
# because of nightly-only cargo-features), so its version lives in
|
||||
# [workspace.package] and its Cargo.lock is not regenerated by the backend step.
|
||||
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.toml
|
||||
sed -i -zE "s/(name = \"windmill[^\"]*\"\nversion = )\"[^\"]*\"/\\1\"$VERSION\"/g" ${root_dirpath}/backend/parsers/windmill-parser-wasm/Cargo.lock
|
||||
|
||||
cd ${root_dirpath}/frontend && npm i --package-lock-only --ignore-scripts
|
||||
cd ${root_dirpath}/frontend && npm i --package-lock-only
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
# Codex output format
|
||||
|
||||
- Read the review context file whose absolute path is given at the end of these instructions; it holds the PR metadata and the diff commands.
|
||||
- Return a markdown PR comment starting with `## Codex Review`.
|
||||
- Tag each finding with a severity (P0 / P1 / P2), file path, and line number when known confidently.
|
||||
@@ -31,3 +31,9 @@ updates:
|
||||
directory: "/python-client/wmill"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
|
||||
# Maintain dependencies for wmill_pg python client
|
||||
- package-ecosystem: "pip"
|
||||
directory: "/python-client/wmill_pg"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
# Pi output format
|
||||
|
||||
- Read the review context file whose absolute path is given at the end of these instructions; it holds the PR metadata and the diff (or the git commands to produce it).
|
||||
- Return a markdown PR comment starting with `## Pi Review`.
|
||||
- Tag each finding with a severity (P0 / P1 / P2), file path, and line number when known confidently.
|
||||
- Output ONLY the final review markdown — no preamble, no thinking, no tool transcripts.
|
||||
@@ -1,126 +0,0 @@
|
||||
// Extracts every windmill.dev/docs link referenced in the frontend source and
|
||||
// verifies none of them 404. Run: `node .github/scripts/check-docs-links.mjs`.
|
||||
// Used by the check-docs-links GitHub workflow (release / manual trigger only).
|
||||
|
||||
import { readdir, readFile } from 'node:fs/promises'
|
||||
import { join, extname } from 'node:path'
|
||||
|
||||
const ROOT = 'frontend/src'
|
||||
const EXTS = new Set(['.ts', '.js', '.svelte', '.mjs', '.cjs'])
|
||||
const DOCS_RE = /https?:\/\/(?:www\.)?windmill\.dev\/docs\/[^\s"'`)>\]}]*/g
|
||||
// `const someBaseUrl = 'https://www.windmill.dev/docs/...'` used later as `${someBaseUrl}/foo`
|
||||
const BASE_RE = /(?:const|let|var)\s+([A-Za-z_$][\w$]*)\s*=\s*['"`](https?:\/\/(?:www\.)?windmill\.dev\/docs\/[^'"`]+)['"`]/g
|
||||
|
||||
const CONCURRENCY = 24
|
||||
const TIMEOUT_MS = 20000
|
||||
const RETRIES = 2
|
||||
|
||||
async function walk(dir) {
|
||||
const out = []
|
||||
for (const entry of await readdir(dir, { withFileTypes: true })) {
|
||||
const p = join(dir, entry.name)
|
||||
if (entry.isDirectory()) {
|
||||
if (entry.name === 'node_modules' || entry.name === '.svelte-kit') continue
|
||||
out.push(...(await walk(p)))
|
||||
} else if (EXTS.has(extname(entry.name))) {
|
||||
out.push(p)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// url (no fragment) -> Set of source files it appears in
|
||||
const urls = new Map()
|
||||
const unresolved = []
|
||||
|
||||
function record(url, file) {
|
||||
const clean = url
|
||||
.replace(/\\.*$/, '') // cut at an escape sequence embedded in a string literal (e.g. \n)
|
||||
.replace(/#.*$/, '') // drop anchor fragment — irrelevant to a 404 check
|
||||
.replace(/[.,;:'")\]]+$/, '')
|
||||
if (!clean) return
|
||||
// A `{`/`${` means the URL is built from an unresolved template/interpolation var.
|
||||
if (clean.includes('{')) {
|
||||
unresolved.push(`${clean} (${file})`)
|
||||
return
|
||||
}
|
||||
if (!urls.has(clean)) urls.set(clean, new Set())
|
||||
urls.get(clean).add(file)
|
||||
}
|
||||
|
||||
for (const file of await walk(ROOT)) {
|
||||
let content = await readFile(file, 'utf8')
|
||||
// Inline file-local base-url constants so `${base}/page` template literals resolve.
|
||||
const bases = []
|
||||
for (const m of content.matchAll(BASE_RE)) bases.push({ name: m[1], value: m[2], decl: m[0] })
|
||||
for (const { name, value } of bases) {
|
||||
content = content.replaceAll('${' + name + '}', value)
|
||||
}
|
||||
// Blank each base declaration so a prefix-only base (no index page of its own,
|
||||
// e.g. .../app_configuration_settings) isn't checked as a standalone link.
|
||||
// A genuinely bare `${base}` usage was already inlined above, so it's still covered.
|
||||
for (const { decl } of bases) content = content.replace(decl, '')
|
||||
for (const m of content.matchAll(DOCS_RE)) record(m[0], file)
|
||||
}
|
||||
|
||||
const allUrls = [...urls.keys()].sort()
|
||||
console.log(`Found ${allUrls.length} distinct docs links across ${ROOT}`)
|
||||
if (unresolved.length) {
|
||||
console.log(`\n⚠️ ${unresolved.length} link(s) built from an unrecognized base URL — skipped (register the base const so they can be checked):`)
|
||||
for (const u of [...new Set(unresolved)].sort()) console.log(` ${u}`)
|
||||
}
|
||||
|
||||
async function check(url) {
|
||||
for (let attempt = 0; attempt <= RETRIES; attempt++) {
|
||||
const ctrl = new AbortController()
|
||||
const timer = setTimeout(() => ctrl.abort(), TIMEOUT_MS)
|
||||
try {
|
||||
let res = await fetch(url, {
|
||||
method: 'HEAD',
|
||||
redirect: 'follow',
|
||||
signal: ctrl.signal,
|
||||
headers: { 'user-agent': 'windmill-docs-link-check' }
|
||||
})
|
||||
// Some hosts reject HEAD — fall back to GET.
|
||||
if (res.status === 405 || res.status === 501) {
|
||||
res = await fetch(url, {
|
||||
method: 'GET',
|
||||
redirect: 'follow',
|
||||
signal: ctrl.signal,
|
||||
headers: { 'user-agent': 'windmill-docs-link-check' }
|
||||
})
|
||||
}
|
||||
clearTimeout(timer)
|
||||
return { url, status: res.status, ok: res.status < 400 }
|
||||
} catch (err) {
|
||||
clearTimeout(timer)
|
||||
if (attempt === RETRIES) return { url, status: 0, ok: false, error: String(err?.message || err) }
|
||||
await new Promise((r) => setTimeout(r, 500 * (attempt + 1)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Simple concurrency pool.
|
||||
const results = []
|
||||
let idx = 0
|
||||
async function worker() {
|
||||
while (idx < allUrls.length) {
|
||||
const url = allUrls[idx++]
|
||||
results.push(await check(url))
|
||||
}
|
||||
}
|
||||
await Promise.all(Array.from({ length: CONCURRENCY }, worker))
|
||||
|
||||
const failures = results.filter((r) => !r.ok)
|
||||
if (failures.length === 0) {
|
||||
console.log(`\n✅ All ${allUrls.length} docs links are reachable.`)
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
console.log(`\n❌ ${failures.length} broken docs link(s):`)
|
||||
for (const f of failures.sort((a, b) => a.url.localeCompare(b.url))) {
|
||||
console.log(`\n ${f.url}`)
|
||||
console.log(` status: ${f.error ? `error (${f.error})` : f.status}`)
|
||||
for (const file of urls.get(f.url)) console.log(` ↳ ${file}`)
|
||||
}
|
||||
process.exit(1)
|
||||
@@ -1,132 +0,0 @@
|
||||
name: AI Agent Integration Tests
|
||||
|
||||
# Exercises the AI agent flow path (preview_flow with `aiagent` modules) against
|
||||
# real LLM providers. Runs only when AI-agent backend code or the tests change,
|
||||
# because each run makes real (paid) LLM calls. To avoid spending on every commit,
|
||||
# the PR side triggers only when a PR is marked ready for review (out of draft) —
|
||||
# not on `synchronize` — plus push to main and manual dispatch.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "integration_tests/ai_agent_tests/**"
|
||||
- "backend/windmill-ai/**"
|
||||
- "backend/windmill-api/src/ai.rs"
|
||||
- "backend/windmill-worker/src/ai_executor.rs"
|
||||
- "backend/windmill-worker/src/ai/**"
|
||||
- "backend/windmill-worker/src/memory_common.rs"
|
||||
- "backend/windmill-common/src/flow_conversations.rs"
|
||||
- ".github/workflows/ai-agent-tests.yml"
|
||||
pull_request:
|
||||
types: [opened, reopened, ready_for_review]
|
||||
paths:
|
||||
- "integration_tests/ai_agent_tests/**"
|
||||
- "backend/windmill-ai/**"
|
||||
- "backend/windmill-api/src/ai.rs"
|
||||
- "backend/windmill-worker/src/ai_executor.rs"
|
||||
- "backend/windmill-worker/src/ai/**"
|
||||
- "backend/windmill-worker/src/memory_common.rs"
|
||||
- "backend/windmill-common/src/flow_conversations.rs"
|
||||
- ".github/workflows/ai-agent-tests.yml"
|
||||
|
||||
concurrency:
|
||||
group: ai-agent-tests-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
ai_agent_e2e:
|
||||
# Skip draft PRs; the `opened`/`reopened` types would otherwise fire while
|
||||
# still a draft. `ready_for_review` always arrives non-draft.
|
||||
if: github.event_name != 'pull_request' || github.event.pull_request.draft == false
|
||||
runs-on: ubicloud-standard-16
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
ports:
|
||||
- 5432:5432
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
# CE build (no enterprise/license needed for AI agents). `quickjs` powers
|
||||
# flow input-transform JS eval; `mcp` is required by the deepwiki MCP tool
|
||||
# test. Bun tool scripts run via the always-on worker (BUN_PATH).
|
||||
- name: Build Windmill
|
||||
working-directory: ./backend
|
||||
env:
|
||||
SQLX_OFFLINE: true
|
||||
CARGO_BUILD_JOBS: 12
|
||||
RUSTFLAGS: ""
|
||||
run: cargo build --features quickjs,mcp
|
||||
|
||||
- name: Start Windmill
|
||||
working-directory: ./backend
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
|
||||
BUN_PATH: bun
|
||||
NODE_BIN_PATH: node
|
||||
RUST_LOG: info
|
||||
run: |
|
||||
mkdir -p ../integration_tests/logs
|
||||
./target/debug/windmill > ../integration_tests/logs/windmill.log 2>&1 &
|
||||
echo "Waiting for Windmill to be ready..."
|
||||
for i in $(seq 1 60); do
|
||||
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
|
||||
echo "Windmill is ready"
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; tail -50 ../integration_tests/logs/windmill.log; exit 1; }
|
||||
|
||||
- name: Run AI agent integration tests
|
||||
timeout-minutes: 20
|
||||
working-directory: ./integration_tests/ai_agent_tests
|
||||
env:
|
||||
WINDMILL_URL: http://localhost:8000
|
||||
# Only the providers we have org secrets for. Other providers
|
||||
# (Azure, Bedrock, OpenRouter) are skipped by conftest when their
|
||||
# keys are absent — see skip_provider_without_credentials.
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
GOOGLE_AI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
run: |
|
||||
python -m venv .venv
|
||||
.venv/bin/pip install -r requirements.txt
|
||||
# The S3/vision-attachment tests need MinIO large-file storage and
|
||||
# image-capable provider setup; out of scope for this cost-controlled
|
||||
# smoke. Add MinIO secrets + a storage service to enable them.
|
||||
.venv/bin/python -m pytest -v \
|
||||
--ignore=test_user_attachments.py \
|
||||
--ignore=test_user_images.py \
|
||||
--ignore=test_image_output.py
|
||||
|
||||
- name: Archive Windmill logs
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: ai-agent-tests-windmill-logs
|
||||
path: integration_tests/logs
|
||||
@@ -1,166 +0,0 @@
|
||||
name: AI Evals (global mode)
|
||||
|
||||
# Smoke-tests the production global AI chat proxy/frontend execution path via
|
||||
# the ai_evals harness, one case across one cheap model per provider. Runs only
|
||||
# when the eval harness or the global chat code change, since each run makes real
|
||||
# (paid) LLM calls. The backend is built from source purely as the AI proxy the
|
||||
# harness routes model calls through; the global tools/drafts run in-process in
|
||||
# the Vitest bridge against production frontend code. To avoid spending on every
|
||||
# commit, the PR side triggers only when a PR is marked ready for review (out of
|
||||
# draft) — not on `synchronize` — plus push to main and manual dispatch.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "ai_evals/**"
|
||||
- "backend/windmill-api/src/ai.rs"
|
||||
- "backend/windmill-ai/**"
|
||||
- "frontend/src/lib/components/copilot/**"
|
||||
# The eval harness runs production frontend code in-process; these are the
|
||||
# AI/draft-specific deps outside copilot/ that the global smoke exercises.
|
||||
- "frontend/src/lib/userDraft.svelte.ts"
|
||||
- "frontend/src/lib/userDraftDbSyncer.svelte.ts"
|
||||
- "frontend/src/lib/infer.ts"
|
||||
- ".github/workflows/ai-evals-test.yml"
|
||||
pull_request:
|
||||
types: [opened, reopened, ready_for_review]
|
||||
paths:
|
||||
- "ai_evals/**"
|
||||
- "backend/windmill-api/src/ai.rs"
|
||||
- "backend/windmill-ai/**"
|
||||
- "frontend/src/lib/components/copilot/**"
|
||||
# The eval harness runs production frontend code in-process; these are the
|
||||
# AI/draft-specific deps outside copilot/ that the global smoke exercises.
|
||||
- "frontend/src/lib/userDraft.svelte.ts"
|
||||
- "frontend/src/lib/userDraftDbSyncer.svelte.ts"
|
||||
- "frontend/src/lib/infer.ts"
|
||||
- ".github/workflows/ai-evals-test.yml"
|
||||
|
||||
concurrency:
|
||||
group: ai-evals-test-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
ai_evals_global:
|
||||
# Provider secrets are unavailable to forked and Dependabot PRs.
|
||||
if: >-
|
||||
github.event_name != 'pull_request' ||
|
||||
(
|
||||
github.event.pull_request.draft == false &&
|
||||
github.event.pull_request.head.repo.full_name == github.repository &&
|
||||
github.event.pull_request.user.login != 'dependabot[bot]'
|
||||
)
|
||||
runs-on: ubicloud-standard-16
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
ports:
|
||||
- 5432:5432
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
# Node 22.19+ is required by the frontend's undici 8.x, which the
|
||||
# Vitest bridge loads; Node 20 fails with markAsUncloneable.
|
||||
node-version: "22"
|
||||
|
||||
# CE build used only as the AI proxy (login, workspace, provider resource,
|
||||
# /ai/proxy). No worker execution or MCP needed — global tools/drafts run
|
||||
# in the Vitest bridge. quickjs matches the standard CE feature set.
|
||||
- name: Build Windmill (AI proxy)
|
||||
working-directory: ./backend
|
||||
env:
|
||||
SQLX_OFFLINE: true
|
||||
CARGO_BUILD_JOBS: 12
|
||||
RUSTFLAGS: ""
|
||||
run: cargo build --features quickjs
|
||||
|
||||
- name: Start Windmill
|
||||
working-directory: ./backend
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
|
||||
RUST_LOG: info
|
||||
run: |
|
||||
mkdir -p ../ai_evals/logs
|
||||
./target/debug/windmill > ../ai_evals/logs/windmill.log 2>&1 &
|
||||
echo "Waiting for Windmill to be ready..."
|
||||
for i in $(seq 1 60); do
|
||||
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
|
||||
echo "Windmill is ready"
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; tail -50 ../ai_evals/logs/windmill.log; exit 1; }
|
||||
|
||||
- name: Install frontend deps + generate client
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
npm ci
|
||||
npm run generate-backend-client
|
||||
|
||||
- name: Run global AI evals
|
||||
timeout-minutes: 20
|
||||
working-directory: ./ai_evals
|
||||
env:
|
||||
WMILL_AI_EVAL_BACKEND_URL: http://localhost:8000
|
||||
WMILL_AI_EVAL_BACKEND_WORKSPACE: integration-tests
|
||||
# Anthropic backs the haiku model. Google AI uses GEMINI_API_KEY.
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
run: |
|
||||
bun install
|
||||
mkdir -p results
|
||||
# One cheap model per provider (anthropic/openai/googleai/deepseek).
|
||||
fail=0
|
||||
for m in haiku 4o gemini-3-flash-preview deepseek-v4-flash; do
|
||||
echo "::group::global-test1-script-create ($m)"
|
||||
if ! bun run cli -- run global global-test1-script-create \
|
||||
--model "$m" --execution-only --output "$PWD/results/ci-$m.json"; then
|
||||
echo "$m: harness/proxy errored"
|
||||
fail=1
|
||||
echo "::endgroup::"
|
||||
continue
|
||||
fi
|
||||
# The CLI exits 0 when the harness records failed attempts, so gate
|
||||
# on execution-only pass counts while ignoring model output quality.
|
||||
if jq -e \
|
||||
'.attemptCount > 0 and .passedAttempts == .attemptCount' \
|
||||
"results/ci-$m.json" > /dev/null; then
|
||||
echo "$m: OK — proxy/frontend execution completed"
|
||||
else
|
||||
echo "$m: FAILED proxy/frontend execution"
|
||||
jq -c '.cases[0].attempts[0].checks' "results/ci-$m.json" || true
|
||||
fail=1
|
||||
fi
|
||||
echo "::endgroup::"
|
||||
done
|
||||
[ "$fail" = 0 ] || { echo "ai_evals global smoke failed"; exit 1; }
|
||||
|
||||
- name: Archive logs and results
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: ai-evals-global-logs
|
||||
path: |
|
||||
ai_evals/logs
|
||||
ai_evals/results
|
||||
@@ -1,94 +0,0 @@
|
||||
name: Aider Auto-fix PR Review Change Requests
|
||||
|
||||
on:
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
check-membership:
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]')
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
is_member: ${{ steps.check-membership.outputs.is_member }}
|
||||
steps:
|
||||
- name: Check organization membership
|
||||
id: check-membership
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
REVIEWER: ${{ github.event.review.user.login }}
|
||||
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
run: |
|
||||
ORG="windmill-labs"
|
||||
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "Authorization: token $ORG_ACCESS_TOKEN" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG/members/$REVIEWER")
|
||||
|
||||
if [ "$STATUS" -eq 204 ]; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
check-and-prepare:
|
||||
needs: check-membership
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
prompt_content: ${{ steps.prepare_prompt.outputs.prompt_content }}
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Acknowledge Request
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
echo "Commenting on PR #${{ github.event.pull_request.number }} to acknowledge the /aider command."
|
||||
gh pr comment ${{ github.event.pull_request.number }} --body "🤖 Aider is starting to work on your request. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
|
||||
|
||||
- name: Prepare prompt for Aider
|
||||
id: prepare_prompt
|
||||
shell: bash
|
||||
env:
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
REVIEW_BODY: ${{ github.event.review.body }}
|
||||
run: |
|
||||
REVIEW_BODY_ESCAPED="${REVIEW_BODY//\\/\\\\}"
|
||||
REVIEW_BODY_ESCAPED="${REVIEW_BODY_ESCAPED//\"/\\\"}"
|
||||
|
||||
ALL_REVIEW_COMMENTS=$(gh api \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
/repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments)
|
||||
|
||||
FORMATTED_COMMENTS=$(jq -r '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]' <<< "$ALL_REVIEW_COMMENTS")
|
||||
|
||||
BASE_PROMPT="Fix the following issues in the PR based on the review feedback. The review body is prepended with REVIEW. The review comments are prepended with REVIEW_COMMENTS. The review body and comments are separated by a blank line."
|
||||
|
||||
COMPLETE_PROMPT="${BASE_PROMPT}"$'\n'"REVIEW:"$'\n'"${REVIEW_BODY_ESCAPED}"$'\n'"REVIEW_COMMENTS:"$'\n'"${FORMATTED_COMMENTS}"
|
||||
|
||||
echo "prompt_content<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$COMPLETE_PROMPT" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
run-aider:
|
||||
needs: [check-membership, check-and-prepare]
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
|
||||
uses: ./.github/workflows/aider-common.yml
|
||||
with:
|
||||
needs_processing: false
|
||||
base_prompt: ${{ needs.check-and-prepare.outputs.prompt_content }}
|
||||
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
|
||||
secrets: inherit
|
||||
@@ -1,522 +0,0 @@
|
||||
name: Aider Common Steps
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
issue_title:
|
||||
description: "Title of the issue or PR"
|
||||
required: false
|
||||
type: string
|
||||
issue_body:
|
||||
description: "Body of the issue or PR"
|
||||
required: false
|
||||
type: string
|
||||
instruction:
|
||||
description: "Instruction for Aider"
|
||||
required: false
|
||||
type: string
|
||||
issue_id:
|
||||
description: "ID of the issue or PR"
|
||||
required: false
|
||||
type: string
|
||||
needs_processing:
|
||||
description: "Whether the issue needs to be processed by the external API"
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
base_prompt:
|
||||
description: "Base prompt for Aider"
|
||||
required: false
|
||||
type: string
|
||||
default: "Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
|
||||
probe_prompt:
|
||||
description: "Prompt for probe-chat"
|
||||
required: false
|
||||
type: string
|
||||
default: 'I''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST. Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: ["file1.py", "file2.py"]'
|
||||
rules_files:
|
||||
description: "Rules files for Aider"
|
||||
required: false
|
||||
type: string
|
||||
outputs:
|
||||
files_to_edit:
|
||||
description: "Files identified by probe-chat for editing"
|
||||
value: ${{ jobs.common-steps.outputs.files_to_edit }}
|
||||
final_prompt:
|
||||
description: "Final prompt for Aider"
|
||||
value: ${{ jobs.common-steps.outputs.final_prompt }}
|
||||
pr_branch_name:
|
||||
description: "Name of the branch used for PR"
|
||||
value: ${{ jobs.common-steps.outputs.pr_branch_name }}
|
||||
changes_applied_message:
|
||||
description: "Message indicating changes were applied"
|
||||
value: ${{ jobs.common-steps.outputs.changes_applied_message }}
|
||||
changes_applied:
|
||||
description: "Boolean indicating if changes were successfully applied"
|
||||
value: ${{ jobs.common-steps.outputs.changes_applied }}
|
||||
|
||||
jobs:
|
||||
common-steps:
|
||||
runs-on: ubicloud-standard-8
|
||||
outputs:
|
||||
files_to_edit: ${{ steps.probe_files.outputs.files_to_edit }}
|
||||
final_prompt: ${{ steps.create_prompt.outputs.final_prompt }}
|
||||
pr_branch_name: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
|
||||
changes_applied_message: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED_MESSAGE }}
|
||||
changes_applied: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@v2
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Check out code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Checkout PR Branch
|
||||
id: checkout_pr
|
||||
if: (github.event_name == 'issue_comment' && github.event.issue.pull_request) || (github.event_name == 'pull_request_review')
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
echo "Issue comment trigger: Checking out PR branch..."
|
||||
PR_NUMBER=""
|
||||
if [ -n "${{ github.event.issue.number }}" ]; then
|
||||
PR_NUMBER="${{ github.event.issue.number }}"
|
||||
elif [ -n "${{ github.event.pull_request.number }}" ]; then
|
||||
PR_NUMBER="${{ github.event.pull_request.number }}"
|
||||
else
|
||||
echo "::error::Could not determine PR number."
|
||||
exit 1
|
||||
fi
|
||||
PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
|
||||
if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
|
||||
echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
|
||||
exit 1
|
||||
fi
|
||||
echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
|
||||
git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
|
||||
git checkout "$PR_HEAD_REF"
|
||||
echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
|
||||
echo "PR_BRANCH=$PR_HEAD_REF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Configure Git User
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Cache Python dependencies
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/setup.py') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-
|
||||
|
||||
- name: Install Aider and Dependencies
|
||||
run: |
|
||||
echo "Installing Aider..."
|
||||
python -m pip install uv
|
||||
python -m venv ~/uv-env
|
||||
source ~/uv-env/bin/activate
|
||||
uv pip install configargparse==1.7
|
||||
uv pip install aider-chat==0.83.1
|
||||
uv pip install -U google-generativeai
|
||||
sudo apt-get update && sudo apt-get install -y jq
|
||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||
echo "VIRTUAL_ENV_PATH=$HOME/uv-env" >> $GITHUB_ENV
|
||||
|
||||
- name: Create Prompt for Aider
|
||||
id: create_prompt
|
||||
shell: bash
|
||||
env:
|
||||
BASE_PROMPT_ENV: ${{ inputs.base_prompt }}
|
||||
ISSUE_TITLE_ENV: ${{ inputs.issue_title }}
|
||||
ISSUE_BODY_ENV: ${{ inputs.issue_body }}
|
||||
INSTRUCTION_ENV: ${{ inputs.instruction }}
|
||||
NEEDS_PROCESSING_ENV: ${{ inputs.needs_processing }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
run: |
|
||||
set -e
|
||||
FINAL_PROMPT_CONTENT=""
|
||||
|
||||
if [[ "$ISSUE_TITLE_ENV" != "" && "$ISSUE_BODY_ENV" != "" ]]; then
|
||||
echo "Processing issue with title: $ISSUE_TITLE_ENV"
|
||||
if [[ "$NEEDS_PROCESSING_ENV" == "true" ]]; then
|
||||
echo "Needs processing is true. Calling Windmill API..."
|
||||
JSON_PAYLOAD=$(jq -n \
|
||||
--arg title "$ISSUE_TITLE_ENV" \
|
||||
--arg body "$ISSUE_BODY_ENV" \
|
||||
'{"body":{"issue_title":$title,"issue_body":$body}}')
|
||||
|
||||
echo "Windmill JSON Payload: $JSON_PAYLOAD"
|
||||
|
||||
API_RESULT_FILE=$(mktemp)
|
||||
HTTP_CODE=$(curl -s -o "$API_RESULT_FILE" -w "%{http_code}" \
|
||||
-X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $WINDMILL_TOKEN" \
|
||||
--data-binary "$JSON_PAYLOAD" \
|
||||
--max-time 90)
|
||||
|
||||
BODY_CONTENT=$(cat "$API_RESULT_FILE")
|
||||
rm -f "$API_RESULT_FILE" # Clean up temp file
|
||||
|
||||
echo "Windmill API HTTP Code: $HTTP_CODE"
|
||||
if [[ "$HTTP_CODE" -eq 200 ]]; then
|
||||
PROCESSED_ISSUE_PROMPT=$(echo "$BODY_CONTENT" | jq -r '.effective_body // empty')
|
||||
if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
|
||||
echo "::warning::Windmill API returned 200 but effective_body was empty or null."
|
||||
EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$ISSUE_BODY_ENV"
|
||||
else
|
||||
echo "Successfully processed issue via Windmill API."
|
||||
EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$PROCESSED_ISSUE_PROMPT"
|
||||
fi
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT_ENV" "$EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT" "$INSTRUCTION_ENV")
|
||||
else
|
||||
echo "::error::Windmill API call failed (HTTP $HTTP_CODE). Using raw issue content for prompt."
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
|
||||
fi
|
||||
else
|
||||
echo "Needs processing is false. Using raw issue content for prompt."
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
|
||||
fi
|
||||
else
|
||||
echo "No issue title or body given. Using base prompt."
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nINSTRUCTION:\n%s" "$BASE_PROMPT_ENV" "$INSTRUCTION_ENV")
|
||||
fi
|
||||
|
||||
echo "Final prompt: $FINAL_PROMPT_CONTENT"
|
||||
echo "final_prompt<<EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
|
||||
echo "$FINAL_PROMPT_CONTENT" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Probe Chat for Relevant Files
|
||||
id: probe_files
|
||||
shell: bash
|
||||
env:
|
||||
FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
|
||||
PROBE_PROMPT: ${{ inputs.probe_prompt }}
|
||||
run: |
|
||||
echo "Running probe-chat to find relevant files..."
|
||||
|
||||
MESSAGE_FOR_PROBE=$(printf "%s\nREQUEST:\n%s" "$PROBE_PROMPT" "$FINAL_PROMPT")
|
||||
|
||||
set -o pipefail
|
||||
PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
|
||||
echo "::error::probe-chat command failed. Output:"
|
||||
echo "$PROBE_OUTPUT"
|
||||
exit 1
|
||||
}
|
||||
set +o pipefail
|
||||
echo "Probe-chat raw output:"
|
||||
echo "$PROBE_OUTPUT"
|
||||
|
||||
JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
|
||||
echo "Extracted JSON block:"
|
||||
echo "$JSON_FILES"
|
||||
|
||||
FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | join(" ")' || echo "")
|
||||
|
||||
if [[ -z "$FILES_LIST" ]]; then
|
||||
echo "::warning::probe-chat did not identify any relevant files."
|
||||
fi
|
||||
|
||||
echo "Formatted files list for aider: $FILES_LIST"
|
||||
echo "files_to_edit=$FILES_LIST" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Cache Aider tags
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: .aider.tags.cache.v4
|
||||
key: ${{ runner.os }}-aider-${{ github.sha }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-aider-
|
||||
|
||||
- name: Prepare branch for Aider
|
||||
id: prepare_branch
|
||||
env:
|
||||
ISSUE_ID: ${{ inputs.issue_id }}
|
||||
run: |
|
||||
if [[ "$ISSUE_ID" != "" ]]; then
|
||||
BRANCH_NAME="aider-fix-issue-${ISSUE_ID}"
|
||||
|
||||
# Check if branch exists remotely
|
||||
if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
|
||||
echo "Branch $BRANCH_NAME already exists remotely, fetching it"
|
||||
git fetch origin $BRANCH_NAME
|
||||
git checkout $BRANCH_NAME
|
||||
git pull origin $BRANCH_NAME
|
||||
else
|
||||
echo "Creating new branch $BRANCH_NAME"
|
||||
git checkout -b $BRANCH_NAME
|
||||
fi
|
||||
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
else
|
||||
# We're in a pull_request_review event
|
||||
PR_NUMBER="${{ github.event.pull_request.number }}"
|
||||
PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
|
||||
|
||||
echo "Handling pull_request_review for PR #$PR_NUMBER on branch $PR_HEAD_REF"
|
||||
|
||||
# Ensure we're on the correct branch
|
||||
git config pull.rebase true
|
||||
git fetch origin $PR_HEAD_REF
|
||||
git checkout $PR_HEAD_REF
|
||||
git pull origin $PR_HEAD_REF
|
||||
|
||||
echo "Using PR branch $PR_HEAD_REF for PR #$PR_NUMBER"
|
||||
echo "BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Run Aider
|
||||
id: run_aider
|
||||
shell: bash
|
||||
env:
|
||||
FILES_TO_EDIT: ${{ steps.probe_files.outputs.files_to_edit }}
|
||||
FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
|
||||
RULES_FILES: ${{ inputs.rules_files }}
|
||||
run: |
|
||||
source $VIRTUAL_ENV_PATH/bin/activate
|
||||
echo "$FINAL_PROMPT" > .aider_final_prompt.txt
|
||||
echo "FILES_TO_EDIT: $FILES_TO_EDIT"
|
||||
|
||||
RULES=""
|
||||
if [ -n "$RULES_FILES" ]; then
|
||||
for rule in $RULES_FILES; do
|
||||
RULES="$RULES --read $rule"
|
||||
done
|
||||
fi
|
||||
|
||||
aider \
|
||||
$RULES \
|
||||
$FILES_TO_EDIT \
|
||||
--model gemini/gemini-2.5-pro-preview-05-06 \
|
||||
--message-file .aider_final_prompt.txt \
|
||||
--yes \
|
||||
--no-check-update \
|
||||
--auto-commits \
|
||||
--no-analytics \
|
||||
--no-gitignore \
|
||||
| tee .aider_output.txt || true
|
||||
|
||||
echo "Aider command completed. Output saved to .aider_output.txt"
|
||||
|
||||
- name: Cache Node.js dependencies
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ~/.npm
|
||||
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json', '**/yarn.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-
|
||||
|
||||
- name: Commit and Push Changes
|
||||
id: commit_and_push
|
||||
env:
|
||||
ISSUE_ID: ${{ inputs.issue_id }}
|
||||
BRANCH_NAME: ${{ steps.prepare_branch.outputs.BRANCH_NAME }}
|
||||
run: |
|
||||
if [[ "$ISSUE_ID" != "" ]]; then
|
||||
# Check if there are any uncommitted changes
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "Found uncommitted changes, committing them"
|
||||
git add .
|
||||
git commit -m "Aider changes"
|
||||
fi
|
||||
|
||||
# Push changes to the branch
|
||||
if git push origin $BRANCH_NAME; then
|
||||
echo "Pushed to branch $BRANCH_NAME"
|
||||
echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "::warning::Push to PR branch $BRANCH_NAME failed."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
else
|
||||
# We're in a pull_request_review event
|
||||
PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
|
||||
echo "Attempting to push changes to PR branch $PR_HEAD_REF"
|
||||
if git push origin $PR_HEAD_REF; then
|
||||
echo "Push to $PR_HEAD_REF successful (or no new changes to push)."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
|
||||
echo "PR_BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "::warning::Push to PR branch $PR_HEAD_REF failed."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Create Pull Request
|
||||
if: always() && (github.event_name == 'issue_comment' || github.event_name == 'repository_dispatch') && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
|
||||
id: create_pr
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
|
||||
ISSUE_NUM: ${{ inputs.issue_id }}
|
||||
ISSUE_TITLE: ${{ inputs.issue_title }}
|
||||
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
||||
run: |
|
||||
# Create PR description in a temporary file to avoid command line length limits and ensure it stays under 40k chars
|
||||
HEADER="This PR was created automatically by Aider to fix issue #${ISSUE_NUM}."
|
||||
# if event is repository_dispatch, add the issue title to the header
|
||||
if [ "$GITHUB_EVENT_NAME" == "repository_dispatch" ]; then
|
||||
if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
|
||||
HEADER="This PR was created automatically by Aider to fix issue #linear:${ISSUE_NUM}."
|
||||
elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
|
||||
HEADER="This PR was created automatically by Aider to fix issue #discord:${ISSUE_NUM}."
|
||||
fi
|
||||
fi
|
||||
cat > /tmp/pr-description.md << EOL | head -c 40000
|
||||
$HEADER
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .aider_output.txt || echo "No output available")
|
||||
\`\`\`
|
||||
EOL
|
||||
|
||||
# Create PR using the file for the body content, handle errors gracefully
|
||||
set +e # Don't exit on error
|
||||
PR_TITLE="[Aider PR] Fix: ${ISSUE_TITLE}"
|
||||
if [ -z "$ISSUE_TITLE" ]; then
|
||||
PR_TITLE="[Aider PR] AI changes after request"
|
||||
fi
|
||||
gh pr create \
|
||||
--title "$PR_TITLE" \
|
||||
--body-file /tmp/pr-description.md \
|
||||
--head "$PR_BRANCH" \
|
||||
--base main \
|
||||
--draft
|
||||
PR_CREATE_EXIT_CODE=$?
|
||||
set -e # Re-enable exit on error
|
||||
|
||||
if [ $PR_CREATE_EXIT_CODE -eq 0 ]; then
|
||||
echo "PR created successfully"
|
||||
PR_URL=$(gh pr view $PR_BRANCH --json url --jq .url)
|
||||
echo "PR_URL=$PR_URL" >> $GITHUB_OUTPUT
|
||||
echo "PR_CREATED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "Warning: Failed to create PR. Exit code: $PR_CREATE_EXIT_CODE"
|
||||
echo "PR_CREATED=false" >> $GITHUB_OUTPUT
|
||||
# Continue workflow despite PR creation failure
|
||||
fi
|
||||
|
||||
- name: Comment on PR with Aider Output
|
||||
if: always() && github.event_name == 'pull_request_review' && steps.commit_and_push.outputs.CHANGES_APPLIED != ''
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_NUM: ${{ github.event.pull_request.number }}
|
||||
JOB_STATUS: ${{ job.status }}
|
||||
run: |
|
||||
# Create comment body in a temporary file to avoid command line length limits
|
||||
if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
STATUS_PREFIX="🤖 I've automatically addressed the feedback based on the review."
|
||||
else
|
||||
STATUS_PREFIX="⚠️ I attempted to address the feedback, but encountered some issues."
|
||||
fi
|
||||
else
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
STATUS_PREFIX="🤖 I attempted to address the review feedback, but no modifications were made."
|
||||
else
|
||||
STATUS_PREFIX="⚠️ I encountered issues while attempting to address the feedback, and no modifications were made."
|
||||
fi
|
||||
fi
|
||||
|
||||
cat > /tmp/pr-comment.md << EOL
|
||||
${STATUS_PREFIX}
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .aider_output.txt || echo 'No output available')
|
||||
\`\`\`
|
||||
|
||||
Please review the output and provide additional guidance if needed.
|
||||
EOL
|
||||
|
||||
# Use the file for comment body
|
||||
gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
|
||||
|
||||
- name: Comment on issue/PR to let the user know Aider has finished working on the request
|
||||
if: always() && github.event_name == 'issue_comment'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
JOB_STATUS: ${{ job.status }}
|
||||
PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
|
||||
PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
|
||||
run: |
|
||||
echo "Commenting on issue/PR #${{ github.event.issue.number }} to let the user know Aider has finished working on the request."
|
||||
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
if [[ "$PR_CREATED" == "true" ]]; then
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
|
||||
else
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
|
||||
fi
|
||||
else
|
||||
COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
|
||||
fi
|
||||
|
||||
gh issue comment ${{ github.event.issue.number }} --body "$COMMENT_BODY" --repo $GITHUB_REPOSITORY
|
||||
|
||||
- name: Comment on linear issue to let the user know Aider has finished working on the request
|
||||
if: always() && github.event_name == 'repository_dispatch'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
JOB_STATUS: ${{ job.status }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
|
||||
PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
SOURCE: ${{ github.event.client_payload.source }}
|
||||
run: |
|
||||
echo "Notifying user about Aider completion status for $SOURCE request #${{ github.event.client_payload.issue_id }}"
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
if [[ "$PR_CREATED" == "true" ]]; then
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
|
||||
else
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
|
||||
fi
|
||||
else
|
||||
COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
|
||||
fi
|
||||
|
||||
if [[ "$SOURCE" == "discord" ]]; then
|
||||
curl -X POST \
|
||||
-H "Authorization: Bot $DISCORD_BOT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
|
||||
-d "{\"content\":\"${COMMENT_BODY}\"}"
|
||||
else
|
||||
curl -X POST \
|
||||
-H "Authorization: $LINEAR_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://api.linear.app/graphql" \
|
||||
-d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"${COMMENT_BODY}\\\" }) { success } }\"}"
|
||||
fi
|
||||
@@ -1,80 +0,0 @@
|
||||
name: External Aider Issue Fix
|
||||
|
||||
on:
|
||||
repository_dispatch:
|
||||
types: [external_issue_fix]
|
||||
|
||||
jobs:
|
||||
check-and-prepare:
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
|
||||
issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
|
||||
instruction: ${{ steps.determine_inputs.outputs.INSTRUCTION }}
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Acknowledge Request
|
||||
env:
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
run: |
|
||||
if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
|
||||
echo "Commenting on Linear issue #${{ github.event.client_payload.issue_id }} to acknowledge the request."
|
||||
curl -X POST \
|
||||
-H "Authorization: $LINEAR_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://api.linear.app/graphql" \
|
||||
-d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\\\" }) { success } }\"}"
|
||||
elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
|
||||
echo "Commenting on Discord thread #${{ github.event.client_payload.channel_id }} to acknowledge the request."
|
||||
curl -X POST \
|
||||
-H "Authorization: Bot $DISCORD_BOT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
|
||||
-d "{\"content\":\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\"}"
|
||||
fi
|
||||
|
||||
- name: Determine inputs for Aider
|
||||
id: determine_inputs
|
||||
shell: bash
|
||||
env:
|
||||
ISSUE_TITLE: ${{ github.event.client_payload.issue_title }}
|
||||
ISSUE_BODY: ${{ github.event.client_payload.issue_body }}
|
||||
INSTRUCTION: ${{ github.event.client_payload.instruction }}
|
||||
run: |
|
||||
echo "Determining inputs for Aider..."
|
||||
|
||||
echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_TITLE" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_BODY" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "INSTRUCTION<<EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
|
||||
echo "$INSTRUCTION" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
|
||||
echo "Finished determining inputs."
|
||||
|
||||
run-aider:
|
||||
needs: check-and-prepare
|
||||
uses: ./.github/workflows/aider-common.yml
|
||||
with:
|
||||
issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
|
||||
issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
|
||||
instruction: ${{ needs.check-and-prepare.outputs.instruction }}
|
||||
issue_id: ${{ github.event.client_payload.issue_id }}
|
||||
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
|
||||
secrets: inherit
|
||||
@@ -1,165 +0,0 @@
|
||||
name: Aider Auto-fix issues and PR comments via external prompt
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
check-membership:
|
||||
runs-on: ubicloud-standard-2
|
||||
if: |
|
||||
github.event_name == 'issue_comment' &&
|
||||
contains(github.event.comment.body, '/aider') &&
|
||||
!contains(github.event.comment.user.login, '[bot]')
|
||||
outputs:
|
||||
is_member: ${{ steps.check-membership.outputs.is_member }}
|
||||
steps:
|
||||
- name: Check organization membership
|
||||
id: check-membership
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
COMMENTER: ${{ github.event.comment.user.login }}
|
||||
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
run: |
|
||||
ORG="windmill-labs"
|
||||
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "Authorization: token $ORG_ACCESS_TOKEN" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
|
||||
|
||||
if [ "$STATUS" -eq 204 ]; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
check-and-prepare:
|
||||
needs: check-membership
|
||||
runs-on: ubicloud-standard-2
|
||||
if: needs.check-membership.outputs.is_member == 'true'
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
outputs:
|
||||
issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
|
||||
issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
|
||||
comment_content: ${{ steps.determine_inputs.outputs.COMMENT_CONTENT }}
|
||||
pr_branch: ${{ steps.checkout_pr.outputs.PR_BRANCH }}
|
||||
|
||||
steps:
|
||||
- name: Acknowledge Request
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
echo "Commenting on issue/PR #${{ github.event.issue.number }} to acknowledge the /aider command."
|
||||
gh issue comment ${{ github.event.issue.number }} --body "🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
|
||||
|
||||
- name: Determine inputs for Aider
|
||||
id: determine_inputs
|
||||
shell: bash
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
COMMENT_BODY: ${{ github.event.comment.body }}
|
||||
ISSUE_NUMBER: ${{ github.event.issue.number }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
run: |
|
||||
echo "Determining inputs for Aider..."
|
||||
ISSUE_TITLE_VAL=""
|
||||
ISSUE_BODY_VAL=""
|
||||
|
||||
if [[ ! -z "${{ github.event.issue.pull_request }}" ]]; then
|
||||
echo "This is a comment on a Pull Request"
|
||||
PR_NUMBER="$ISSUE_NUMBER"
|
||||
|
||||
PR_BODY_JSON=$(gh pr view "$PR_NUMBER" --json body --repo "$GITHUB_REPOSITORY")
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Error fetching PR body for PR #$PR_NUMBER"
|
||||
PR_BODY_VAL=""
|
||||
else
|
||||
PR_BODY_VAL=$(jq -r '.body // ""' <<< "$PR_BODY_JSON")
|
||||
fi
|
||||
|
||||
if [[ ! -z "$PR_BODY_VAL" ]]; then
|
||||
REFERENCED_ISSUE=""
|
||||
if [[ "$PR_BODY_VAL" =~ \#linear:([a-f0-9-]+) ]]; then
|
||||
REFERENCED_ISSUE="${BASH_REMATCH[1]}"
|
||||
echo "Found referenced Linear issue #$REFERENCED_ISSUE in PR description"
|
||||
LINEAR_ISSUE_JSON=$(curl -s -H "Authorization: $LINEAR_API_KEY" \
|
||||
"https://api.linear.app/graphql" \
|
||||
-X POST \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"query\":\"query { issue(id: \\\"$REFERENCED_ISSUE\\\") { title description } }\"}")
|
||||
|
||||
if [[ $? -eq 0 && ! "$LINEAR_ISSUE_JSON" =~ "error" ]]; then
|
||||
ISSUE_TITLE_VAL=$(jq -r '.data.issue.title // ""' <<< "$LINEAR_ISSUE_JSON")
|
||||
ISSUE_BODY_VAL=$(jq -r '.data.issue.description // ""' <<< "$LINEAR_ISSUE_JSON")
|
||||
echo "Successfully fetched Linear issue details"
|
||||
else
|
||||
echo "Error fetching Linear issue details for #$REFERENCED_ISSUE"
|
||||
fi
|
||||
elif [[ "$PR_BODY_VAL" =~ \#([0-9]+) ]]; then
|
||||
REFERENCED_ISSUE="${BASH_REMATCH[1]}"
|
||||
echo "Found referenced GitHub issue #$REFERENCED_ISSUE in PR description"
|
||||
|
||||
ISSUE_DETAILS_JSON=$(gh issue view "$REFERENCED_ISSUE" --json title,body --repo "$GITHUB_REPOSITORY")
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Error fetching issue details for #$REFERENCED_ISSUE"
|
||||
else
|
||||
ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "PR body is empty or could not be fetched."
|
||||
fi
|
||||
else
|
||||
echo "This is a comment on a regular issue"
|
||||
|
||||
ISSUE_DETAILS_JSON=$(gh issue view "$ISSUE_NUMBER" --json title,body --repo "$GITHUB_REPOSITORY")
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Error fetching issue details for #$ISSUE_NUMBER"
|
||||
else
|
||||
ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_TITLE_VAL" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_BODY_VAL" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
|
||||
CLEAN_COMMENT="${COMMENT_BODY/\/aider/}"
|
||||
CLEAN_COMMENT="${CLEAN_COMMENT#"${CLEAN_COMMENT%%[![:space:]]*}"}"
|
||||
CLEAN_COMMENT="${CLEAN_COMMENT%"${CLEAN_COMMENT##*[![:space:]]}"}"
|
||||
|
||||
echo "COMMENT_CONTENT<<EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
|
||||
echo "$CLEAN_COMMENT" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
|
||||
echo "Finished determining inputs."
|
||||
|
||||
run-aider:
|
||||
needs: [check-membership, check-and-prepare]
|
||||
if: needs.check-membership.outputs.is_member == 'true'
|
||||
uses: ./.github/workflows/aider-common.yml
|
||||
with:
|
||||
issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
|
||||
issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
|
||||
instruction: ${{ needs.check-and-prepare.outputs.comment_content }}
|
||||
issue_id: ${{ github.event.issue.number }}
|
||||
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
|
||||
secrets: inherit
|
||||
@@ -1,57 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
ECR_REGISTRY: 976079455550.dkr.ecr.us-east-1.amazonaws.com
|
||||
IMAGE_NAME: ${{ github.repository }}-multiplayer
|
||||
|
||||
name: Publish websocket multiplayer server
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
packages: write
|
||||
|
||||
jobs:
|
||||
publish_multiplayer:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
# - uses: depot/setup-action@v1
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v2
|
||||
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push publicly
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/DockerfileMultiplayer
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta.outputs.labels }}
|
||||
org.opencontainers.image.licenses=AGPLv3
|
||||
@@ -1,139 +0,0 @@
|
||||
name: Backend check
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Change versions"]
|
||||
types:
|
||||
- completed
|
||||
push:
|
||||
paths:
|
||||
- "backend/**"
|
||||
- ".github/workflows/backend-check.yml"
|
||||
|
||||
jobs:
|
||||
check_oss:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Install mold and clang
|
||||
run: sudo apt-get update && sudo apt-get install -y mold clang
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: false
|
||||
toolchain: 1.97.0
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
run: SQLX_OFFLINE=true cargo check
|
||||
|
||||
check_oss_full:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: install xmlsec1 and gssapi
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: false
|
||||
toolchain: 1.97.0
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
run: |
|
||||
mkdir -p fake_frontend_build
|
||||
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features all_sqlx_features
|
||||
|
||||
check_ee:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Substitute EE code (EE logic is behind feature flag)
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Install mold and clang
|
||||
run: sudo apt-get update && sudo apt-get install -y mold clang
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: false
|
||||
toolchain: 1.97.0
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
run: SQLX_OFFLINE=true cargo check
|
||||
|
||||
check_ee_full:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: install xmlsec1 and gssapi
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
|
||||
|
||||
- name: Substitute EE code (EE logic is behind feature flag)
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
- name: Fix stale v8 build cache
|
||||
working-directory: ./backend
|
||||
run: |
|
||||
# Cargo cache may preserve v8 build fingerprints without the actual
|
||||
# librusty_v8.a library. Since fingerprints look valid, cargo skips
|
||||
# build.rs re-run, causing "could not find native static library rusty_v8".
|
||||
for profile in debug release; do
|
||||
if [ -d "target/$profile/.fingerprint" ] && [ ! -f "target/$profile/gn_out/obj/librusty_v8.a" ]; then
|
||||
echo "Cleaning stale v8 build artifacts in target/$profile"
|
||||
rm -rf "target/$profile/build/v8-"* "target/$profile/.fingerprint/v8-"*
|
||||
fi
|
||||
done
|
||||
- name: cargo check
|
||||
timeout-minutes: 16
|
||||
working-directory: ./backend
|
||||
run: |
|
||||
mkdir -p fake_frontend_build
|
||||
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features all_sqlx_features,private
|
||||
@@ -1,222 +0,0 @@
|
||||
name: Backend integration tests (Windows)
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- "ci-windows-tests"
|
||||
tags:
|
||||
- "v*"
|
||||
|
||||
env:
|
||||
CARGO_INCREMENTAL: 0
|
||||
SQLX_OFFLINE: true
|
||||
DISABLE_EMBEDDING: true
|
||||
|
||||
jobs:
|
||||
cargo_test_windows:
|
||||
runs-on: blacksmith-16vcpu-windows-2025
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
shell: pwsh
|
||||
run: |
|
||||
$ee_repo_ref = Get-Content .\backend\ee-repo-ref.txt
|
||||
echo "ee_repo_ref=$ee_repo_ref" | Out-File -FilePath $env:GITHUB_ENV -Append
|
||||
|
||||
- name: Checkout windmill-ee-private repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Substitute EE code
|
||||
shell: bash
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Setup PostgreSQL
|
||||
uses: ikalnytskyi/action-setup-postgres@v6
|
||||
with:
|
||||
username: postgres
|
||||
password: changeme
|
||||
database: windmill
|
||||
port: 5432
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
# This action defaults RUSTFLAGS to "-D warnings"; unset it so the test
|
||||
# run is not failed by cross-platform dead-code (cfg(unix)-only helpers
|
||||
# are unused on Windows). Warning hygiene is enforced on the Linux CI
|
||||
# and the build_windows_worker_ release build, not this test job.
|
||||
rustflags: ""
|
||||
|
||||
- uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: "9.0.x"
|
||||
|
||||
- uses: denoland/setup-deno@v2
|
||||
with:
|
||||
# Pin to the Deno version shipped in the runtime image (Dockerfile) so CI
|
||||
# tests what production runs, instead of floating on the latest v2.x.
|
||||
deno-version: 2.2.1
|
||||
|
||||
- uses: actions/setup-go@v2
|
||||
with:
|
||||
go-version: 1.21.5
|
||||
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- uses: astral-sh/setup-uv@v6.2.1
|
||||
with:
|
||||
version: "0.11.24"
|
||||
|
||||
- uses: shivammathur/setup-php@v2
|
||||
with:
|
||||
php-version: "8.3"
|
||||
tools: composer
|
||||
|
||||
- name: Install windmill CLI
|
||||
shell: bash
|
||||
run: |
|
||||
cd cli
|
||||
bash gen_wm_client.sh
|
||||
bun install
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
|
||||
chmod +x "$HOME/.local/bin/wmill"
|
||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||
|
||||
- name: Install OpenSSL via vcpkg
|
||||
run: |
|
||||
vcpkg.exe install openssl-windows:x64-windows
|
||||
vcpkg.exe install openssl:x64-windows-static
|
||||
vcpkg.exe integrate install
|
||||
|
||||
- name: Free disk space (post-vcpkg)
|
||||
shell: pwsh
|
||||
run: |
|
||||
# vcpkg leaves multi-GB of buildtrees/downloads after installing openssl;
|
||||
# we only need the installed/ dir for linking.
|
||||
$vcpkgRoot = $env:VCPKG_INSTALLATION_ROOT
|
||||
foreach ($sub in @("buildtrees", "downloads", "packages")) {
|
||||
$path = Join-Path $vcpkgRoot $sub
|
||||
if (Test-Path $path) {
|
||||
Write-Host "Removing $path"
|
||||
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue $path
|
||||
}
|
||||
}
|
||||
Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
|
||||
|
||||
- name: Get runtime paths
|
||||
id: runtime-paths
|
||||
shell: pwsh
|
||||
run: |
|
||||
echo "DENO_PATH=$($(Get-Command deno).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "BUN_PATH=$($(Get-Command bun).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "NODE_BIN_PATH=$($(Get-Command node).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "GO_PATH=$($(Get-Command go).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "UV_PATH=$($(Get-Command uv).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "PHP_PATH=$($(Get-Command php).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "COMPOSER_PATH=$($(Get-Command composer).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "POWERSHELL_PATH=$($(Get-Command pwsh).Source)" >> $env:GITHUB_OUTPUT
|
||||
echo "DOTNET_PATH=$($(Get-Command dotnet).Source)" >> $env:GITHUB_OUTPUT
|
||||
|
||||
- name: Build DuckDB FFI module
|
||||
working-directory: backend/windmill-duckdb-ffi-internal
|
||||
timeout-minutes: 30
|
||||
run: |
|
||||
cargo build --release -p windmill_duckdb_ffi_internal
|
||||
New-Item -ItemType Directory -Path ..\target\debug -Force
|
||||
Copy-Item target\release\windmill_duckdb_ffi_internal.dll ..\target\debug\
|
||||
# duckdb is bundled (~2GB of build artifacts); the DLL is the only
|
||||
# thing we need from this excluded-crate target dir.
|
||||
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue target
|
||||
Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
|
||||
|
||||
- name: Print runtime versions and env
|
||||
shell: pwsh
|
||||
run: |
|
||||
deno --version
|
||||
bun -v
|
||||
node --version
|
||||
go version
|
||||
python3 --version
|
||||
php --version
|
||||
pwsh --version
|
||||
dotnet --version
|
||||
echo "TEMP=$env:TEMP"
|
||||
echo "TMP=$env:TMP"
|
||||
echo "USERPROFILE=$env:USERPROFILE"
|
||||
echo "HOME=$env:HOME"
|
||||
|
||||
- name: Disk space before cargo test
|
||||
shell: pwsh
|
||||
run: Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
|
||||
|
||||
- name: cargo test
|
||||
working-directory: backend
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
|
||||
RUST_LOG: "off"
|
||||
RUST_LOG_STYLE: never
|
||||
CARGO_NET_GIT_FETCH_WITH_CLI: true
|
||||
# 16-vcpu runners with disabled PDB still hit LNK1180 ("insufficient
|
||||
# disk space") at link time with 12 parallel link jobs: each test
|
||||
# binary link spikes several hundred MB of transient I/O. Capping at
|
||||
# 8 trades ~25% wall time for headroom on the ~75GB runner disk.
|
||||
CARGO_BUILD_JOBS: 8
|
||||
# backend/Cargo.toml leaves profile.dev at the default debug = 2 for
|
||||
# the (large) windmill workspace crates; that debuginfo is emitted
|
||||
# into every object file and embedded in each test binary, and on
|
||||
# windows-msvc also spawns the mspdbsrv.exe PDB type server. Across a
|
||||
# full --all --features build it is the dominant consumer of the
|
||||
# ~63GB free on the runner disk (LNK1180 / disk-full during linking).
|
||||
# CI needs no debug info, so drop it entirely for the dev/test
|
||||
# profiles here. debug = 0 supersedes the previous split-debuginfo=off
|
||||
# knob (no debuginfo => no .pdb and no LNK1318 type-server limit).
|
||||
CARGO_PROFILE_DEV_DEBUG: "0"
|
||||
CARGO_PROFILE_TEST_DEBUG: "0"
|
||||
# Tests' poll-time stack frames (deep nested async fn chains in
|
||||
# debug builds) reach ~1.8MB. 4MB gives ~2x headroom against flaky
|
||||
# overflows under parallel-test contention.
|
||||
RUST_MIN_STACK: 4194304
|
||||
VCPKGRS_DYNAMIC: 1
|
||||
OPENSSL_DIR: ${{ env.VCPKG_INSTALLATION_ROOT }}\installed\x64-windows-static
|
||||
DENO_PATH: ${{ steps.runtime-paths.outputs.DENO_PATH }}
|
||||
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
|
||||
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
|
||||
GO_PATH: ${{ steps.runtime-paths.outputs.GO_PATH }}
|
||||
UV_PATH: ${{ steps.runtime-paths.outputs.UV_PATH }}
|
||||
PHP_PATH: ${{ steps.runtime-paths.outputs.PHP_PATH }}
|
||||
COMPOSER_PATH: ${{ steps.runtime-paths.outputs.COMPOSER_PATH }}
|
||||
POWERSHELL_PATH: ${{ steps.runtime-paths.outputs.POWERSHELL_PATH }}
|
||||
DOTNET_PATH: ${{ steps.runtime-paths.outputs.DOTNET_PATH }}
|
||||
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
|
||||
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
|
||||
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
|
||||
# Windows ships a worker-only binary, so test the crates a worker runs
|
||||
# (windmill-worker/-common/-queue) via -p, not `--all`: this skips the
|
||||
# disk-heavy windmill-api test binaries (LNK1180) and the server-only
|
||||
# windmill-trigger-* crates (amqp does not build on Windows). Linux CI runs the rest.
|
||||
run: >
|
||||
cargo test
|
||||
--no-fail-fast
|
||||
-p windmill-worker
|
||||
-p windmill-common
|
||||
-p windmill-queue
|
||||
--features private,enterprise,deno_core,duckdb,python,rust,csharp,php,quickjs,parquet,mcp,scoped_cache,windmill-git-sync/private,windmill-object-store/private,windmill-object-store/enterprise
|
||||
-- --nocapture --test-threads=10
|
||||
@@ -1,7 +1,6 @@
|
||||
name: Backend only integration tests
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- "main"
|
||||
@@ -14,282 +13,40 @@ on:
|
||||
- "backend/**"
|
||||
- ".github/workflows/backend-test.yml"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./backend
|
||||
|
||||
jobs:
|
||||
cargo_test:
|
||||
runs-on: ubicloud-standard-16
|
||||
runs-on: ubicloud-standard-8
|
||||
container:
|
||||
image: ghcr.io/windmill-labs/backend-tests
|
||||
services:
|
||||
postgres:
|
||||
image: postgres
|
||||
ports:
|
||||
- 5432:5432
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c max_connections=500"
|
||||
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5 --shm-size=256mb
|
||||
mysql:
|
||||
image: mysql:8.0
|
||||
ports:
|
||||
- 3306:3306
|
||||
env:
|
||||
MYSQL_ROOT_PASSWORD: changeme
|
||||
MYSQL_DATABASE: windmill_test
|
||||
options: >-
|
||||
--health-cmd "mysqladmin ping -h localhost" --health-interval 10s
|
||||
--health-timeout 5s --health-retries 5
|
||||
--health-retries 5
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: "9.0.x"
|
||||
- uses: denoland/setup-deno@v2
|
||||
with:
|
||||
# Pin to the Deno version shipped in the runtime image (Dockerfile) so CI
|
||||
# tests what production runs, instead of floating on the latest v2.x.
|
||||
deno-version: 2.2.1
|
||||
- uses: actions/setup-go@v2
|
||||
with:
|
||||
go-version: 1.21.5
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
- uses: astral-sh/setup-uv@v6.2.1
|
||||
with:
|
||||
version: "0.11.24"
|
||||
- uses: shivammathur/setup-php@v2
|
||||
with:
|
||||
php-version: "8.3"
|
||||
tools: composer
|
||||
- uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: "3.3"
|
||||
bundler-cache: false
|
||||
- name: Install windmill CLI from source
|
||||
run: |
|
||||
cd $GITHUB_WORKSPACE/cli
|
||||
bash gen_wm_client.sh
|
||||
bun install
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
|
||||
chmod +x "$HOME/.local/bin/wmill"
|
||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||
working-directory: /
|
||||
- name: Install PowerShell, mold and clang
|
||||
run: |
|
||||
sudo apt-get update && sudo apt-get install -y powershell mold clang libcurl4-openssl-dev
|
||||
working-directory: /
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
- name: Fix stale v8 build cache
|
||||
working-directory: ./backend
|
||||
run: |
|
||||
# Cargo cache may preserve v8 build fingerprints without the actual
|
||||
# librusty_v8.a library. Since fingerprints look valid, cargo skips
|
||||
# build.rs re-run, causing "could not find native static library rusty_v8".
|
||||
for profile in debug release; do
|
||||
if [ -d "target/$profile/.fingerprint" ] && [ ! -f "target/$profile/gn_out/obj/librusty_v8.a" ]; then
|
||||
echo "Cleaning stale v8 build artifacts in target/$profile"
|
||||
rm -rf "target/$profile/build/v8-"* "target/$profile/.fingerprint/v8-"*
|
||||
fi
|
||||
done
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Substitute EE code (EE logic is behind feature flag)
|
||||
run: |
|
||||
./substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
- name: Setup private npm registry with test package
|
||||
working-directory: /tmp
|
||||
run: |
|
||||
set -e
|
||||
|
||||
# Install Verdaccio globally
|
||||
npm install -g verdaccio
|
||||
|
||||
# Create Verdaccio config that requires authentication for @windmill-test packages
|
||||
mkdir -p /tmp/verdaccio/storage
|
||||
cat > /tmp/verdaccio/config.yaml << 'VERDACCIO_CONFIG'
|
||||
storage: /tmp/verdaccio/storage
|
||||
auth:
|
||||
htpasswd:
|
||||
file: /tmp/verdaccio/htpasswd
|
||||
max_users: 100
|
||||
uplinks:
|
||||
npmjs:
|
||||
url: https://registry.npmjs.org/
|
||||
packages:
|
||||
'@windmill-test/*':
|
||||
access: $authenticated
|
||||
publish: $authenticated
|
||||
'@*/*':
|
||||
access: $all
|
||||
publish: $authenticated
|
||||
proxy: npmjs
|
||||
'**':
|
||||
access: $all
|
||||
publish: $authenticated
|
||||
proxy: npmjs
|
||||
server:
|
||||
keepAliveTimeout: 60
|
||||
middlewares:
|
||||
audit:
|
||||
enabled: true
|
||||
log: { type: stdout, format: pretty, level: warn }
|
||||
VERDACCIO_CONFIG
|
||||
|
||||
# Create empty htpasswd file (users will be created via API)
|
||||
touch /tmp/verdaccio/htpasswd
|
||||
|
||||
# Start Verdaccio in background
|
||||
verdaccio --config /tmp/verdaccio/config.yaml &
|
||||
VERDACCIO_PID=$!
|
||||
|
||||
# Wait for Verdaccio to be ready
|
||||
echo "Waiting for Verdaccio to start..."
|
||||
for i in {1..30}; do
|
||||
if curl -s http://localhost:4873/-/ping > /dev/null 2>&1; then
|
||||
echo "Verdaccio is ready"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# Login to get a token
|
||||
echo "Getting auth token..."
|
||||
RESPONSE=$(curl -s -X PUT \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"testuser","password":"testpass123"}' \
|
||||
http://localhost:4873/-/user/org.couchdb.user:testuser)
|
||||
|
||||
echo "Auth response: $RESPONSE"
|
||||
NPM_TOKEN=$(echo "$RESPONSE" | jq -r '.token')
|
||||
|
||||
if [ -z "$NPM_TOKEN" ] || [ "$NPM_TOKEN" = "null" ]; then
|
||||
echo "Failed to get NPM token from response"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "NPM_TOKEN=${NPM_TOKEN}" >> $GITHUB_ENV
|
||||
{
|
||||
echo "TEST_NPMRC<<NPMRC_EOF"
|
||||
echo "@windmill-test:registry=http://localhost:4873/"
|
||||
echo "//localhost:4873/:_authToken=${NPM_TOKEN}"
|
||||
echo "NPMRC_EOF"
|
||||
} >> $GITHUB_ENV
|
||||
echo "Got NPM token successfully: ${NPM_TOKEN:0:10}..."
|
||||
|
||||
# Configure npm globally with the auth token
|
||||
echo "//localhost:4873/:_authToken=${NPM_TOKEN}" > ~/.npmrc
|
||||
echo "Configured ~/.npmrc with auth token"
|
||||
|
||||
# Create a simple test package
|
||||
mkdir -p /tmp/windmill-test-private-pkg
|
||||
cat > /tmp/windmill-test-private-pkg/package.json << 'PKG_JSON'
|
||||
{
|
||||
"name": "@windmill-test/private-pkg",
|
||||
"version": "1.0.0",
|
||||
"main": "index.js"
|
||||
}
|
||||
PKG_JSON
|
||||
cat > /tmp/windmill-test-private-pkg/index.js << 'PKG_JS'
|
||||
module.exports.greet = (name) => `Hello from private package, ${name}!`;
|
||||
PKG_JS
|
||||
|
||||
# Publish to Verdaccio with auth
|
||||
cd /tmp/windmill-test-private-pkg
|
||||
echo "Publishing package..."
|
||||
npm publish --registry http://localhost:4873
|
||||
echo "Package published successfully"
|
||||
|
||||
# Verify the package requires auth by trying anonymous access (should fail)
|
||||
rm -f ~/.npmrc
|
||||
echo "Testing anonymous access (should fail)..."
|
||||
if npm view @windmill-test/private-pkg --registry http://localhost:4873 2>/dev/null; then
|
||||
echo "ERROR: Package should require authentication but anonymous access worked"
|
||||
exit 1
|
||||
fi
|
||||
echo "Verified: Package requires authentication for @windmill-test/private-pkg"
|
||||
- name: Cache DuckDB FFI module build
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ./backend/windmill-duckdb-ffi-internal/target
|
||||
key: ${{ runner.os }}-duckdb-ffi-${{ hashFiles('./backend/windmill-duckdb-ffi-internal/src/**/*.rs', './backend/windmill-duckdb-ffi-internal/Cargo.toml', './backend/windmill-duckdb-ffi-internal/Cargo.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-duckdb-ffi-
|
||||
toolchain: 1.80.0
|
||||
# - uses: Swatinem/rust-cache@v2
|
||||
# with:
|
||||
# workspaces: |
|
||||
# backend
|
||||
# backend -> target
|
||||
- name: cargo test
|
||||
timeout-minutes: 30
|
||||
env:
|
||||
# setup-rust-toolchain exports RUSTFLAGS=-D warnings, and the RUSTFLAGS env
|
||||
# var fully REPLACES (never merges with) target.*.rustflags in
|
||||
# backend/.cargo/config.toml. That silently drops the config's
|
||||
# `-C link-arg=-fuse-ld=mold`, so CI links the many large integration-test
|
||||
# binaries (v8 + duckdb + every language runtime, statically linked) with the
|
||||
# default bfd linker. Its peak memory across ~12 parallel links OOM-kills the
|
||||
# runner mid-link (SIGTERM => exit 143, before any test runs). Re-add the mold
|
||||
# link arg here so CI links with mold like local dev, keeping -D warnings.
|
||||
# (config.toml's `linker = "clang"` still applies; env only overrides rustflags.)
|
||||
RUSTFLAGS: "-D warnings -C link-arg=-fuse-ld=mold"
|
||||
SQLX_OFFLINE: true
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
|
||||
DISABLE_EMBEDDING: true
|
||||
RUST_LOG: "off"
|
||||
RUST_LOG_STYLE: never
|
||||
CARGO_NET_GIT_FETCH_WITH_CLI: true
|
||||
# Cap parallel rustc/link jobs below the 16 available cores. The tail of
|
||||
# the build links ~128 full-graph test binaries (one per tests/*.rs file
|
||||
# across the workspace); at high parallelism enough heavy codegen+link
|
||||
# units (rustc ~2.6GB, mold ~1GB each) overlap to exhaust the 64GB
|
||||
# runner. Matches backend-test-windows.yml, which already uses 8.
|
||||
CARGO_BUILD_JOBS: 8
|
||||
# Incremental compilation is per-run dead weight in CI: rust-cache
|
||||
# (cache-workspaces above) restores compiled dependency artifacts but
|
||||
# never persists target/**/incremental, so there is no prior state to
|
||||
# reuse in a one-shot `cargo test`. It only adds per-crate memory
|
||||
# overhead and extra disk. Off here (kept on for local dev via
|
||||
# .cargo/config.toml). Matches backend-test-windows.yml.
|
||||
CARGO_INCREMENTAL: "0"
|
||||
# backend/Cargo.toml leaves profile.dev at the default debug = 2 for
|
||||
# the (large) windmill workspace crates; that debug info is emitted
|
||||
# into every object file and embedded in each test binary. Across the
|
||||
# full --all --features build it is the dominant memory/disk consumer
|
||||
# when mold links the windmill-api-integration-tests binary, tipping
|
||||
# the runner over (lost runner reported as a canceled step). CI needs
|
||||
# no debug info, so drop it entirely for the dev/test profiles here.
|
||||
# (test profile inherits dev, but the workspace crates link in as
|
||||
# dev-profile deps, so both must be set.) CI-only; local dev builds
|
||||
# are unaffected.
|
||||
CARGO_PROFILE_DEV_DEBUG: "0"
|
||||
CARGO_PROFILE_TEST_DEBUG: "0"
|
||||
# Tests' poll-time stack frames (deep nested async fn chains in
|
||||
# debug builds) reach ~1.8MB, leaving very thin headroom on the
|
||||
# default 2MB thread stack. 4MB gives ~2x buffer against flaky
|
||||
# overflows under parallel-test contention.
|
||||
RUST_MIN_STACK: 4194304
|
||||
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
|
||||
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
|
||||
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
|
||||
TEST_NPM_REGISTRY: "http://localhost:4873/:_authToken=${{ env.NPM_TOKEN }}"
|
||||
run: |
|
||||
deno --version && bun -v && node --version && go version && python3 --version && php --version && ruby --version && pwsh --version && dotnet --version
|
||||
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..
|
||||
DENO_PATH=$(which deno) BUN_PATH=$(which bun) NODE_BIN_PATH=$(which node) GO_PATH=$(which go) UV_PATH=$(which uv) PHP_PATH=$(which php) COMPOSER_PATH=$(which composer) RUBY_PATH=$(which ruby) RUBY_BUNDLE_PATH=$(which bundle) RUBY_GEM_PATH=$(which gem) POWERSHELL_PATH=$(which pwsh) DOTNET_PATH=$(which dotnet) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp,run_inline --all -- --nocapture --test-threads=10
|
||||
timeout-minutes: 15
|
||||
run:
|
||||
/usr/bin/deno --version &&
|
||||
/usr/bin/bun -v &&
|
||||
go version &&
|
||||
/usr/local/bin/python3 --version &&
|
||||
mkdir frontend/build && cd backend && touch
|
||||
windmill-api/openapi-deref.yaml &&
|
||||
DATABASE_URL=postgres://postgres:changeme@postgres:5432/windmill
|
||||
DISABLE_EMBEDDING=true RUST_LOG=info cargo test --features
|
||||
enterprise,deno_core --all -- --nocapture
|
||||
|
||||
@@ -14,13 +14,9 @@ jobs:
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
--shm-size=2g
|
||||
|
||||
|
||||
windmill:
|
||||
image: ghcr.io/windmill-labs/windmill-ee:main
|
||||
env:
|
||||
@@ -35,12 +31,12 @@ jobs:
|
||||
ports:
|
||||
- 8000:8000
|
||||
steps:
|
||||
- uses: denoland/setup-deno@v2
|
||||
- uses: denoland/setup-deno@v1
|
||||
with:
|
||||
deno-version: v2.x
|
||||
deno-version: v1.x
|
||||
- name: benchmark
|
||||
timeout-minutes: 30
|
||||
run: deno run -A -r
|
||||
timeout-minutes: 20
|
||||
run: deno run --unstable -A -r
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
|
||||
-c
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_config.json
|
||||
@@ -59,7 +55,6 @@ jobs:
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
@@ -77,12 +72,12 @@ jobs:
|
||||
ports:
|
||||
- 8000:8000
|
||||
steps:
|
||||
- uses: denoland/setup-deno@v2
|
||||
- uses: denoland/setup-deno@v1
|
||||
with:
|
||||
deno-version: v2.x
|
||||
deno-version: v1.x
|
||||
- name: benchmark
|
||||
timeout-minutes: 20
|
||||
run: deno run -A -r
|
||||
run: deno run --unstable -A -r
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
|
||||
--no-warm-up -c
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_dedicated.json
|
||||
@@ -101,7 +96,6 @@ jobs:
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
@@ -152,17 +146,16 @@ jobs:
|
||||
--pull always
|
||||
|
||||
steps:
|
||||
- uses: denoland/setup-deno@v2
|
||||
- uses: denoland/setup-deno@v1
|
||||
with:
|
||||
deno-version: v2.x
|
||||
deno-version: v1.x
|
||||
- name: benchmark
|
||||
timeout-minutes: 20
|
||||
run: deno run -A -r
|
||||
run: deno run --unstable -A -r
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
|
||||
-c
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_config.json
|
||||
--workers 4
|
||||
--factor 3
|
||||
- name: Save benchmark results
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
@@ -178,7 +171,6 @@ jobs:
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
@@ -272,17 +264,16 @@ jobs:
|
||||
options: >-
|
||||
--pull always
|
||||
steps:
|
||||
- uses: denoland/setup-deno@v2
|
||||
- uses: denoland/setup-deno@v1
|
||||
with:
|
||||
deno-version: v2.x
|
||||
deno-version: v1.x
|
||||
- name: benchmark
|
||||
timeout-minutes: 20
|
||||
run: deno run -A -r
|
||||
run: deno run --unstable -A -r
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
|
||||
-c
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_config.json
|
||||
--workers 8
|
||||
--factor 3
|
||||
- name: Save benchmark results
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
@@ -290,49 +281,6 @@ jobs:
|
||||
path: |
|
||||
*.json
|
||||
|
||||
benchmark_wac:
|
||||
runs-on: ubicloud-standard-8
|
||||
services:
|
||||
postgres:
|
||||
image: postgres
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
--shm-size=2g
|
||||
windmill:
|
||||
image: ghcr.io/windmill-labs/windmill-ee:main
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
|
||||
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
|
||||
WORKER_GROUP: main
|
||||
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
|
||||
options: >-
|
||||
--pull always --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5 --health-cmd "curl
|
||||
http://localhost:8000/api/version"
|
||||
ports:
|
||||
- 8000:8000
|
||||
steps:
|
||||
- uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
- name: benchmark
|
||||
timeout-minutes: 30
|
||||
run: deno run -A -r
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
|
||||
-c
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_wac.json
|
||||
- name: Save benchmark results
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: benchmark_wac
|
||||
path: |
|
||||
*.json
|
||||
|
||||
benchmark_graphs:
|
||||
runs-on: ubicloud
|
||||
needs:
|
||||
@@ -340,11 +288,10 @@ jobs:
|
||||
- benchmark_dedicated
|
||||
- benchmark_4workers
|
||||
- benchmark_8workers
|
||||
- benchmark_wac
|
||||
steps:
|
||||
- uses: denoland/setup-deno@v2
|
||||
- uses: denoland/setup-deno@v1
|
||||
with:
|
||||
deno-version: v2.x
|
||||
deno-version: v1.x
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: benchmarks
|
||||
@@ -353,7 +300,7 @@ jobs:
|
||||
with:
|
||||
merge-multiple: true
|
||||
- name: graphs
|
||||
run: deno run -A -r
|
||||
run: deno run --unstable -A -r
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_graphs.ts
|
||||
-c
|
||||
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/graphs_config.json
|
||||
|
||||
@@ -5,20 +5,6 @@ env:
|
||||
name: Build caddy-l4
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- docker/DockerfileCaddyL4
|
||||
- docker/entrypoint-caddy.sh
|
||||
- docker/caddy-compat-normalize.awk
|
||||
- docker/caddy-l4.version
|
||||
- docker/test-caddy-compat.sh
|
||||
- Caddyfile
|
||||
# The version check below reads the pin out of docker-compose.yml, so a
|
||||
# compose-only bump has to trigger this workflow or the check never runs.
|
||||
- docker-compose.yml
|
||||
- .github/workflows/build-caddy-l4-image.yml
|
||||
|
||||
permissions: write-all
|
||||
|
||||
@@ -28,35 +14,6 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
# docker-compose.yml pins an exact tag, and the Caddyfile it must agree
|
||||
# with lives in the same checkout. Fail the build rather than publish a
|
||||
# version nothing references, which is how :latest drifted from the
|
||||
# Caddyfile in the first place.
|
||||
- name: Resolve and check image version
|
||||
id: version
|
||||
run: |
|
||||
set -euo pipefail
|
||||
version="$(tr -d '[:space:]' < docker/caddy-l4.version)"
|
||||
pinned="$(grep -oE 'caddy-l4:[^[:space:]"]+' docker-compose.yml | head -1 | cut -d: -f2-)"
|
||||
caddy="$(grep -m1 -oE '^FROM caddy:[0-9]+\.[0-9]+\.[0-9]+' docker/DockerfileCaddyL4 | cut -d: -f2)"
|
||||
if [ "$version" != "$pinned" ]; then
|
||||
echo "docker/caddy-l4.version is '$version' but docker-compose.yml pins '$pinned'" >&2
|
||||
echo "Bump both together." >&2
|
||||
exit 1
|
||||
fi
|
||||
# Otherwise a caddy bump that forgets the version file publishes a tag
|
||||
# that names the wrong caddy.
|
||||
case "$version" in
|
||||
"$caddy"-*) ;;
|
||||
*)
|
||||
echo "docker/caddy-l4.version is '$version' but the Dockerfile pins caddy '$caddy'" >&2
|
||||
echo "The version must be <caddy-version>-<revision>." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
@@ -66,22 +23,8 @@ jobs:
|
||||
tags: |
|
||||
type=sha
|
||||
type=ref,event=branch
|
||||
# Not gated on the default branch: docker-compose.yml pins this tag,
|
||||
# so it has to be publishable from a branch (workflow_dispatch)
|
||||
# before the pin merges, or main would reference a tag that does not
|
||||
# exist yet. The version is immutable, so republishing from main is
|
||||
# a no-op. Only branch pushes to main and manual dispatch run this
|
||||
# workflow, so a branch cannot claim the tag by accident.
|
||||
type=raw,value=${{ steps.version.outputs.version }}
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
# The shim rewrites config a self-hoster never sees, so a silent
|
||||
# regression here strands them on a restart loop or a dead :80.
|
||||
- name: Test the legacy-Caddyfile compatibility shim
|
||||
run: |
|
||||
docker build -f docker/DockerfileCaddyL4 -t caddy-l4:ci ./docker
|
||||
docker/test-caddy-compat.sh caddy-l4:ci
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
name: Build windmill-extra
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: "Tag for the image"
|
||||
required: false
|
||||
default: "dev"
|
||||
type: string
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
sleep:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- name: Sleep for 900 seconds waiting for pypi to update index
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
run: sleep 900
|
||||
shell: bash
|
||||
build_extra:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-extra
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=raw,value=${{ github.event.inputs.tag }}
|
||||
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
file: "./docker/DockerfileExtra"
|
||||
tags: |
|
||||
${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta.outputs.labels }}
|
||||
@@ -3,13 +3,14 @@ env:
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
name: Build and publish windmill for RHEL9
|
||||
on: workflow_dispatch
|
||||
on:
|
||||
workflow_dispatch
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
build_ee:
|
||||
runs-on: ubicloud-standard-4
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -27,6 +28,8 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
@@ -55,62 +58,71 @@ jobs:
|
||||
run: |
|
||||
cp ./docker/RHEL9/Dockerfile ./Dockerfile
|
||||
|
||||
- name: Build and push EE (multi-arch)
|
||||
- name: Build and push publicly ee amd64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee_rhel
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,deno_core
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
${{ steps.meta-ee-public.outputs.tags }}-amd64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
${{ steps.meta-ee-public.outputs.labels }}-amd64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- name: Build and push publicly ee arm64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,deno_core
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}-arm64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}-arm64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- name: Install crane
|
||||
uses: imjasonh/setup-crane@v0.4
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee-amd64
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
|
||||
path: "/windmill/target/release/windmill"
|
||||
|
||||
- name: Extract binaries with crane
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee-arm64
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
|
||||
path: "/windmill/target/release/windmill"
|
||||
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
mkdir -p extracted
|
||||
|
||||
# Extract arm64 binary (include deps/ for hard link resolution)
|
||||
mkdir -p /tmp/arm64
|
||||
crane export --platform linux/arm64 ${{ steps.meta-ee-public.outputs.tags }} - \
|
||||
| tar -xf - -C /tmp/arm64 windmill/target/release/ usr/src/app/libwindmill_duckdb_ffi_internal.so
|
||||
cp /tmp/arm64/windmill/target/release/windmill extracted/windmill-ee-arm64-rhel9
|
||||
cp /tmp/arm64/usr/src/app/libwindmill_duckdb_ffi_internal.so extracted/libwindmill_duckdb_ffi_internal-arm64.so
|
||||
rm -rf /tmp/arm64
|
||||
|
||||
# Extract amd64 binary
|
||||
mkdir -p /tmp/amd64
|
||||
crane export --platform linux/amd64 ${{ steps.meta-ee-public.outputs.tags }} - \
|
||||
| tar -xf - -C /tmp/amd64 windmill/target/release/ usr/src/app/libwindmill_duckdb_ffi_internal.so
|
||||
cp /tmp/amd64/windmill/target/release/windmill extracted/windmill-ee-amd64-rhel9
|
||||
cp /tmp/amd64/usr/src/app/libwindmill_duckdb_ffi_internal.so extracted/libwindmill_duckdb_ffi_internal-amd64.so
|
||||
rm -rf /tmp/amd64
|
||||
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9"
|
||||
mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9"
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-amd64 build
|
||||
path: ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-arm64 build
|
||||
path: extracted/windmill-ee-arm64-rhel9
|
||||
path: ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-amd64 build
|
||||
path: extracted/windmill-ee-amd64-rhel9
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-arm64 dynamic libraries build
|
||||
path: extracted/libwindmill_duckdb_ffi_internal-arm64.so
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-amd64 dynamic libraries build
|
||||
path: extracted/libwindmill_duckdb_ffi_internal-amd64.so
|
||||
# - name: Attach binary to release
|
||||
# uses: softprops/action-gh-release@v2
|
||||
# if: startsWith(github.ref, 'refs/tags/')
|
||||
# with:
|
||||
# files: |
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9
|
||||
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9
|
||||
|
||||
@@ -1,140 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
name: Build and publish windmill for RHEL8
|
||||
on: workflow_dispatch
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
build_ee:
|
||||
runs-on: ubicloud-standard-4
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-rhel8
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=sha
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Substitute EE code
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Copy RHEL8 Dockerfile
|
||||
run: |
|
||||
cp ./docker/RHEL8/Dockerfile ./Dockerfile
|
||||
|
||||
- name: Build and push publicly ee amd64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee_rhel
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}-amd64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}-amd64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- name: Build and push publicly ee arm64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee_rhel
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}-arm64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}-arm64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee-amd64
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
|
||||
path: "/windmill/target/release/windmill"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-duckdb-ffi-internal
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
|
||||
path: "/usr/src/app/libwindmill_duckdb_ffi_internal.so"
|
||||
|
||||
# - uses: shrink/actions-docker-extract@v3
|
||||
# id: extract-ee-arm64
|
||||
# with:
|
||||
# image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
|
||||
# path: "/windmill/target/release/windmill"
|
||||
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8"
|
||||
# mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel8"
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL8-amd64 build
|
||||
path: ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL8-amd64 dynamic libraries build
|
||||
path: ${{ steps.extract-duckdb-ffi-internal.outputs.destination }}/libwindmill_duckdb_ffi_internal.so
|
||||
|
||||
# - uses: actions/upload-artifact@v4
|
||||
# with:
|
||||
# name: RHEL8-arm64 build
|
||||
# path:
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination
|
||||
# }}/windmill-ee-arm64-rhel8
|
||||
|
||||
# - name: Attach binary to release
|
||||
# uses: softprops/action-gh-release@v2
|
||||
# if: startsWith(github.ref, 'refs/tags/')
|
||||
# with:
|
||||
# files: |
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel8
|
||||
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8
|
||||
@@ -0,0 +1,70 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
name: Build windmill-staging
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
build_ee:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-staging-ee
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=sha
|
||||
type=ref,event=branch
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Substitute EE code
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Build and push publicly ee
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,deno_core
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
@@ -11,7 +11,7 @@ env:
|
||||
|
||||
jobs:
|
||||
cargo_build_windows:
|
||||
runs-on: blacksmith-16vcpu-windows-2025
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -30,57 +30,29 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
|
||||
- name: Substitute EE code
|
||||
shell: bash
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Cargo check (fail fast on warnings)
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
RUSTFLAGS: "-D warnings"
|
||||
run: |
|
||||
cd backend
|
||||
# Stub the openapi specs to empty: they are compiled in via an ungated
|
||||
# include_str! but a worker binary never serves them, so this avoids
|
||||
# embedding ~2.5MB of spec.
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.json" -ItemType "File" -Force
|
||||
cargo check --features=ee_windows
|
||||
|
||||
- name: Cargo build dynamic libraries windows
|
||||
timeout-minutes: 180
|
||||
run: |
|
||||
cd backend/windmill-duckdb-ffi-internal
|
||||
cargo build --release -p windmill_duckdb_ffi_internal
|
||||
|
||||
- name: Cargo build binary windows
|
||||
timeout-minutes: 180
|
||||
- name: Cargo build windows
|
||||
timeout-minutes: 90
|
||||
run: |
|
||||
vcpkg.exe install openssl-windows:x64-windows
|
||||
vcpkg.exe install openssl:x64-windows-static
|
||||
vcpkg.exe integrate install
|
||||
$env:VCPKGRS_DYNAMIC=1
|
||||
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
|
||||
cd backend
|
||||
cargo build --release --features=ee_windows
|
||||
mkdir frontend/build && cd backend
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
|
||||
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,deno_core
|
||||
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
|
||||
|
||||
- name: Upload binary artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: windmill-ee-binary
|
||||
path: ./backend/target/release/windmill-ee.exe
|
||||
|
||||
- name: Upload dynamic libraries artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: windmill_duckdb_ffi_internal.dll
|
||||
path: ./backend/windmill-duckdb-ffi-internal/target/release/windmill_duckdb_ffi_internal.dll
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
ECR_REGISTRY: 976079455550.dkr.ecr.us-east-1.amazonaws.com
|
||||
IMAGE_NAME: ${{ github.repository }}-multiplayer
|
||||
|
||||
name: Publish websocket multiplayer server
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
packages: write
|
||||
|
||||
jobs:
|
||||
publish_multiplayer:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: depot/setup-action@v1
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push publicly
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/DockerfileMultiplayer
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta.outputs.labels }}
|
||||
org.opencontainers.image.licenses=AGPLv3
|
||||
@@ -9,14 +9,7 @@ jobs:
|
||||
runs-on: ubicloud
|
||||
container: node:18
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
- run: git config --system --add safe.directory /__w/windmill/windmill
|
||||
- name: Change versions
|
||||
run: ./.github/change-versions.sh "$(cat version.txt)"
|
||||
@@ -27,9 +20,4 @@ jobs:
|
||||
run: |
|
||||
cd backend
|
||||
cargo generate-lockfile
|
||||
- uses: stefanzweifel/git-auto-commit-action@v5
|
||||
with:
|
||||
commit_user_name: windmill-internal-app[bot]
|
||||
commit_user_email: windmill-internal-app[bot]@users.noreply.github.com
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ steps.app.outputs.token }}
|
||||
- uses: stefanzweifel/git-auto-commit-action@v4
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
name: Check frontend docs links
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
check-docs-links:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
sparse-checkout: |
|
||||
frontend/src
|
||||
.github/scripts
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22.x"
|
||||
- name: Verify docs links are not 404
|
||||
run: node .github/scripts/check-docs-links.mjs
|
||||
@@ -1,19 +0,0 @@
|
||||
name: Check fixture is empty
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "fixtures/**"
|
||||
pull_request:
|
||||
paths:
|
||||
- "fixtures/**"
|
||||
|
||||
jobs:
|
||||
check-empty-fixture:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Ensure fixtures/cli-sync/ has no committed snapshot
|
||||
run: bash fixtures/check-empty.sh
|
||||
@@ -1,39 +0,0 @@
|
||||
name: Check system prompts freshness
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- "system_prompts/**"
|
||||
- "typescript-client/**"
|
||||
- "python-client/wmill/wmill/client.py"
|
||||
- "openflow.openapi.yaml"
|
||||
- "backend/windmill-api/openapi.yaml"
|
||||
- "cli/src/main.ts"
|
||||
- "cli/src/commands/**"
|
||||
- "frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts"
|
||||
pull_request:
|
||||
paths:
|
||||
- "system_prompts/**"
|
||||
- "typescript-client/**"
|
||||
- "python-client/wmill/wmill/client.py"
|
||||
- "openflow.openapi.yaml"
|
||||
- "backend/windmill-api/openapi.yaml"
|
||||
- "cli/src/main.ts"
|
||||
- "cli/src/commands/**"
|
||||
- "frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts"
|
||||
|
||||
jobs:
|
||||
check-freshness:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install dependencies
|
||||
run: pip install pyyaml
|
||||
|
||||
- name: Check auto-generated files are up-to-date
|
||||
run: bash system_prompts/check-freshness.sh
|
||||
@@ -1,66 +0,0 @@
|
||||
name: Check Write Access
|
||||
|
||||
# Authorizes a user to trigger privileged command workflows (/review, /ai, /plan,
|
||||
# /updatesqlx, ...). The webhook author_association reports PRIVATE org members as
|
||||
# CONTRIBUTOR/NONE (only public members show as MEMBER), so command jobs can't gate on
|
||||
# it alone. This mints the internal GitHub App token — which can see private members —
|
||||
# and confirms the user is a member or has write access to the repo. The app token is
|
||||
# minted fresh per run, so unlike the old ORG_ACCESS_TOKEN PAT it never expires.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
username:
|
||||
required: true
|
||||
type: string
|
||||
description: 'The user whose access to verify'
|
||||
trusted_bot:
|
||||
required: false
|
||||
type: string
|
||||
default: 'windmill-internal-app[bot]'
|
||||
description: 'A bot login that is always authorized'
|
||||
outputs:
|
||||
authorized:
|
||||
description: 'true if the user is the trusted bot, an org member, or has repo write access'
|
||||
value: ${{ jobs.check.outputs.authorized }}
|
||||
|
||||
jobs:
|
||||
check:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
authorized: ${{ steps.check.outputs.authorized }}
|
||||
steps:
|
||||
- name: Mint internal app token
|
||||
id: app
|
||||
uses: actions/create-github-app-token@v2
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
owner: ${{ github.repository_owner }}
|
||||
|
||||
- name: Resolve authorization
|
||||
id: check
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
USERNAME: ${{ inputs.username }}
|
||||
TRUSTED_BOT: ${{ inputs.trusted_bot }}
|
||||
REPO: ${{ github.repository }}
|
||||
run: |
|
||||
if [ "$USERNAME" = "$TRUSTED_BOT" ]; then
|
||||
echo "authorized=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
ORG="${REPO%%/*}"
|
||||
# Org membership resolves private members too (204 = member, 404 = not).
|
||||
if gh api "orgs/$ORG/members/$USERNAME" --silent 2>/dev/null; then
|
||||
echo "authorized=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
# Fallback: effective repo permission (also covers outside collaborators).
|
||||
PERM=$(gh api "repos/$REPO/collaborators/$USERNAME/permission" --jq '.permission' 2>/dev/null || echo none)
|
||||
if [ "$PERM" = "admin" ] || [ "$PERM" = "write" ]; then
|
||||
echo "authorized=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "authorized=false" >> "$GITHUB_OUTPUT"
|
||||
echo "$USERNAME is neither the trusted bot, an org member, nor a repo writer."
|
||||
fi
|
||||
@@ -1,73 +0,0 @@
|
||||
name: Claude Plan Assistant
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
# author_association misses private org members; check-access resolves them via the
|
||||
# internal app token. Both are OR'd below so public members still pass instantly.
|
||||
check-access:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/plan')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/plan')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/plan')) ||
|
||||
(github.event_name == 'issues' && contains(github.event.issue.body, '/plan'))
|
||||
uses: ./.github/workflows/check-write-access.yml
|
||||
with:
|
||||
username: ${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}
|
||||
secrets: inherit
|
||||
|
||||
claude-plan-action:
|
||||
needs: [check-access]
|
||||
if: |
|
||||
needs.check-access.outputs.authorized == 'true' ||
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association)
|
||||
runs-on: ubicloud-standard-4
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude Plan Action
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
allowed_bots: 'windmill-internal-app[bot]'
|
||||
trigger_phrase: '/plan'
|
||||
claude_args: |
|
||||
--model claude-opus-4-8
|
||||
--system-prompt "# Claude Planning Mode
|
||||
|
||||
You are operating in PLANNING MODE ONLY. Your role is to create detailed, structured plans without making any code changes.
|
||||
|
||||
## Your Responsibilities:
|
||||
|
||||
1. **Analyze the Request**: Carefully read and understand what the user is asking for
|
||||
2. **Explore the Codebase**: Understand the relevant code structure
|
||||
3. **Create a Detailed Plan**: Provide a comprehensive, step-by-step plan that includes:
|
||||
- Clear breakdown of all tasks needed
|
||||
- Files that will need to be modified or created
|
||||
- Code patterns and architecture decisions
|
||||
- Potential challenges and how to address them
|
||||
- If there are multiple options to achieve the same goal, explain the pros and cons of each option
|
||||
|
||||
## Strict Constraints:
|
||||
|
||||
- **DO NOT** make any code changes
|
||||
- **DO NOT** create branches or pull requests
|
||||
|
||||
Remember: You are here to plan, not to implement. Provide thorough analysis and clear guidance for implementation."
|
||||
@@ -1,96 +0,0 @@
|
||||
name: Fast Claude
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
# author_association misses private org members; check-access resolves them via the
|
||||
# internal app token. Both are OR'd below so public members still pass instantly.
|
||||
check-access:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
|
||||
(github.event_name == 'pull_request_review' && startsWith(github.event.review.body, '/ai') && !startsWith(github.event.review.body, '/ai-fast')) ||
|
||||
(github.event_name == 'issues' && startsWith(github.event.issue.body, '/ai') && !startsWith(github.event.issue.body, '/ai-fast'))
|
||||
uses: ./.github/workflows/check-write-access.yml
|
||||
with:
|
||||
username: ${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}
|
||||
secrets: inherit
|
||||
|
||||
claude-code-action:
|
||||
needs: [check-access]
|
||||
if: |
|
||||
needs.check-access.outputs.authorized == 'true' ||
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association || github.event.review.author_association || github.event.issue.author_association)
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
# Make the EE source (the *_ee.rs files in the companion repo) available so the
|
||||
# reviewer can see EE-only code (e.g. windmill-queue/src/jobs_ee.rs), not just the
|
||||
# CE surface. The EE ref is read from the PR head's backend/ee-repo-ref.txt (via the
|
||||
# API, so it reflects the PR's EE pin regardless of which ref is checked out here).
|
||||
- name: Check EE access
|
||||
id: ee
|
||||
env:
|
||||
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
|
||||
run: |
|
||||
if [ -z "$EE_TOKEN" ] || [ -z "$PR_NUMBER" ]; then
|
||||
echo "available=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
HEAD_SHA=$(gh api "repos/${{ github.repository }}/pulls/$PR_NUMBER" --jq .head.sha)
|
||||
REF=$(gh api "repos/${{ github.repository }}/contents/backend/ee-repo-ref.txt?ref=$HEAD_SHA" --jq .content | base64 -d | tr -d '[:space:]')
|
||||
if [ -z "$REF" ]; then
|
||||
echo "available=false" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "available=true" >> "$GITHUB_OUTPUT"
|
||||
echo "ee_repo_ref=$REF" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Checkout EE repository
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ steps.ee.outputs.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Substitute EE code
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Run Claude PR Action
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
allowed_bots: "windmill-internal-app[bot]"
|
||||
trigger_phrase: "/ai"
|
||||
settings: |
|
||||
{
|
||||
"env": {
|
||||
"SQLX_OFFLINE": "true"
|
||||
}
|
||||
}
|
||||
claude_args: |
|
||||
--allowedTools "Bash,WebFetch,WebSearch"
|
||||
--model claude-opus-4-8
|
||||
@@ -1,185 +0,0 @@
|
||||
name: CLI Tests
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "cli/**"
|
||||
- "backend/migrations/**"
|
||||
- ".github/workflows/cli-tests.yml"
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "cli/**"
|
||||
- "backend/migrations/**"
|
||||
- ".github/workflows/cli-tests.yml"
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
SQLX_OFFLINE: true
|
||||
|
||||
jobs:
|
||||
build-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Generate Windmill client
|
||||
working-directory: cli
|
||||
run: ./gen_wm_client.sh
|
||||
|
||||
- name: Run CLI build
|
||||
working-directory: cli
|
||||
run: ./build.sh
|
||||
|
||||
test-linux:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_DB: windmill
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 5432:5432
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Rust toolchain
|
||||
uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: true
|
||||
cache-workspaces: backend
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Symlink Bun to /usr/bin/bun
|
||||
run: sudo ln -sf $(which bun) /usr/bin/bun
|
||||
|
||||
- name: Symlink Node to /usr/bin/node
|
||||
run: sudo ln -sf $(which node) /usr/bin/node
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: cli
|
||||
run: bun install
|
||||
|
||||
- name: Generate Windmill clients
|
||||
working-directory: cli
|
||||
run: |
|
||||
./gen_wm_client.sh
|
||||
./windmill-utils-internal/gen_wm_client.sh
|
||||
|
||||
- name: Run CLI tests
|
||||
working-directory: cli
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432
|
||||
CI_MINIMAL_FEATURES: "true"
|
||||
run: bun test --timeout 120000 test/
|
||||
|
||||
test-windows:
|
||||
runs-on: blacksmith-16vcpu-windows-2025
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup PostgreSQL
|
||||
uses: ikalnytskyi/action-setup-postgres@v6
|
||||
with:
|
||||
username: postgres
|
||||
password: changeme
|
||||
database: windmill
|
||||
port: 5432
|
||||
|
||||
- name: Setup Rust toolchain
|
||||
uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: true
|
||||
cache-workspaces: backend
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Get Bun and Node paths
|
||||
id: runtime-paths
|
||||
shell: pwsh
|
||||
run: |
|
||||
$bunPath = (Get-Command bun).Source
|
||||
$nodePath = (Get-Command node).Source
|
||||
echo "BUN_PATH=$bunPath" >> $env:GITHUB_OUTPUT
|
||||
echo "NODE_BIN_PATH=$nodePath" >> $env:GITHUB_OUTPUT
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: cli
|
||||
run: bun install
|
||||
|
||||
- name: Generate Windmill clients
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
run: |
|
||||
./gen_wm_client.sh
|
||||
./windmill-utils-internal/gen_wm_client.sh
|
||||
|
||||
- name: Run CLI tests
|
||||
working-directory: cli
|
||||
shell: pwsh
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432
|
||||
CI_MINIMAL_FEATURES: "true"
|
||||
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
|
||||
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
|
||||
run: bun test --timeout 120000 test/
|
||||
|
||||
# Combined summary job for branch protection
|
||||
test-summary:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [build-check, test-linux, test-windows]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check test results
|
||||
run: |
|
||||
if [ "${{ needs.build-check.result }}" != "success" ]; then
|
||||
echo "Build check failed"
|
||||
exit 1
|
||||
fi
|
||||
if [ "${{ needs.test-linux.result }}" != "success" ] || [ "${{ needs.test-windows.result }}" != "success" ]; then
|
||||
echo "Some tests failed"
|
||||
exit 1
|
||||
fi
|
||||
echo "All checks passed"
|
||||
@@ -1,421 +0,0 @@
|
||||
name: Codex Auto Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [ready_for_review, opened, synchronize]
|
||||
workflow_call:
|
||||
inputs:
|
||||
pr_number:
|
||||
description: 'PR number to review'
|
||||
required: true
|
||||
type: number
|
||||
extra_prompt:
|
||||
description: 'Additional reviewer instructions appended to the standard review prompt'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
triggered_by:
|
||||
description: 'GitHub username that triggered this review (for audit only)'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
secrets:
|
||||
OPENAI_API_KEY:
|
||||
required: false
|
||||
CODEX_AUTH_JSON:
|
||||
required: false
|
||||
WINDMILL_EE_PRIVATE_ACCESS:
|
||||
required: false
|
||||
|
||||
concurrency:
|
||||
group: codex-review-${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
codex-review:
|
||||
runs-on: ubicloud-standard-2
|
||||
timeout-minutes: 30
|
||||
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
|
||||
# access to this repo, so fork==false already enforces write access. Do NOT re-add
|
||||
# an author_association gate: the pull_request webhook payload reports private org
|
||||
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
|
||||
# skips auto-review for every private member.
|
||||
if: |
|
||||
github.event_name == 'workflow_call' ||
|
||||
(
|
||||
github.event.pull_request.draft == false &&
|
||||
github.event.pull_request.head.repo.fork == false
|
||||
)
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Check Codex configuration
|
||||
id: codex_config
|
||||
env:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
|
||||
run: |
|
||||
if [ -n "$OPENAI_API_KEY" ]; then
|
||||
echo "enabled=true" >> "$GITHUB_OUTPUT"
|
||||
echo "auth_mode=api_key" >> "$GITHUB_OUTPUT"
|
||||
elif [ -n "$CODEX_AUTH_JSON" ]; then
|
||||
echo "enabled=true" >> "$GITHUB_OUTPUT"
|
||||
echo "auth_mode=oauth_json" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "enabled=false" >> "$GITHUB_OUTPUT"
|
||||
echo "Codex auth is not configured; set OPENAI_API_KEY or CODEX_AUTH_JSON to enable Codex review."
|
||||
fi
|
||||
|
||||
- name: Resolve PR metadata
|
||||
if: steps.codex_config.outputs.enabled == 'true'
|
||||
id: pr
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
INPUT_PR_NUMBER: ${{ inputs.pr_number }}
|
||||
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
|
||||
EVENT_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
EVENT_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
EVENT_TITLE: ${{ github.event.pull_request.title }}
|
||||
EVENT_BODY: ${{ github.event.pull_request.body }}
|
||||
EVENT_FORK: ${{ github.event.pull_request.head.repo.fork }}
|
||||
EVENT_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
EVENT_ACTION: ${{ github.event.action }}
|
||||
run: |
|
||||
if [ -n "$INPUT_PR_NUMBER" ]; then
|
||||
PR_JSON=$(gh pr view "$INPUT_PR_NUMBER" --repo "${{ github.repository }}" \
|
||||
--json number,baseRefName,baseRefOid,headRefOid,title,body,isCrossRepository,author)
|
||||
PR_NUMBER=$(echo "$PR_JSON" | jq -r '.number')
|
||||
BASE_REF=$(echo "$PR_JSON" | jq -r '.baseRefName')
|
||||
BASE_SHA=$(echo "$PR_JSON" | jq -r '.baseRefOid')
|
||||
HEAD_SHA=$(echo "$PR_JSON" | jq -r '.headRefOid')
|
||||
PR_TITLE=$(echo "$PR_JSON" | jq -r '.title')
|
||||
PR_BODY=$(echo "$PR_JSON" | jq -r '.body // ""')
|
||||
IS_FORK=$(echo "$PR_JSON" | jq -r '.isCrossRepository')
|
||||
PR_AUTHOR=$(echo "$PR_JSON" | jq -r '.author.login // ""')
|
||||
else
|
||||
PR_NUMBER="$EVENT_PR_NUMBER"
|
||||
BASE_REF="$EVENT_BASE_REF"
|
||||
BASE_SHA="$EVENT_BASE_SHA"
|
||||
HEAD_SHA="$EVENT_HEAD_SHA"
|
||||
PR_TITLE="$EVENT_TITLE"
|
||||
PR_BODY="$EVENT_BODY"
|
||||
IS_FORK="$EVENT_FORK"
|
||||
PR_AUTHOR="$EVENT_AUTHOR"
|
||||
fi
|
||||
# Fork PRs run untrusted code with secrets present, so the automatic
|
||||
# pull_request trigger never reviews them. A non-empty INPUT_PR_NUMBER
|
||||
# means we arrived via workflow_call (a maintainer /codex comment gated
|
||||
# by check-write-access), so allow forks only on that path.
|
||||
if [ "$IS_FORK" = "true" ] && [ -z "$INPUT_PR_NUMBER" ]; then
|
||||
echo "Skipping Codex review for fork PR (automatic trigger)."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
# An agent-driven PR flips to ready only after a clean /review round on
|
||||
# a draft, marked by an author comment naming the head SHA (pr skill,
|
||||
# "Review rounds"). Re-reviewing that same head on ready_for_review is
|
||||
# redundant. The marker alone is author attestation, so also require
|
||||
# reviewer evidence: a Codex review (posted by github-actions[bot], not
|
||||
# forgeable by the author) that predates the marker and carries a
|
||||
# non-blocking verdict. Comment-triggered and synchronize runs never
|
||||
# skip. Keep the three copies of this check in sync (pr-ready-review /
|
||||
# codex-pr-review / pi-pr-review); a shared local action would need the
|
||||
# repo checked out before the check, which the fork paths here
|
||||
# deliberately avoid.
|
||||
if [ "$EVENT_ACTION" = "ready_for_review" ] && [ -z "$INPUT_PR_NUMBER" ]; then
|
||||
# Fetch failures fail open (no skip): an API hiccup must run the
|
||||
# review, never skip it or fail the job.
|
||||
COMMENTS=$(gh api "repos/${{ github.repository }}/issues/$PR_NUMBER/comments?per_page=100" --paginate | jq -s '[.[][]]') || COMMENTS='[]'
|
||||
MARKER_TIME=$(jq -r --arg author "$PR_AUTHOR" --arg marker "✅ Review round clean @ $HEAD_SHA" \
|
||||
'[.[] | select(.user.login == $author) | select(.body | contains($marker)) | .created_at] | min // empty' <<<"$COMMENTS")
|
||||
CODEX_VERDICT=''
|
||||
if [ -n "$MARKER_TIME" ]; then
|
||||
# Only Codex evidence that predates the marker counts: the ready-
|
||||
# triggered Codex run itself posts after the flip and must not
|
||||
# vouch for a sibling reviewer's skip.
|
||||
CODEX_VERDICT=$(jq -r --arg mt "$MARKER_TIME" \
|
||||
'[.[] | select(.user.login == "github-actions[bot]") | select(.body | contains("## Codex Review")) | select(.created_at < $mt)] | last | .body // ""' <<<"$COMMENTS" \
|
||||
| grep -m1 -oE '(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)' || true)
|
||||
fi
|
||||
if [ -n "$MARKER_TIME" ] && [ -n "$CODEX_VERDICT" ] && [ "$CODEX_VERDICT" != "Should address issues before merging" ]; then
|
||||
echo "Clean review round marker found for $HEAD_SHA with pre-marker non-blocking Codex verdict; skipping redundant review."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
# PR title/body are attacker-controlled free text. Use an unguessable
|
||||
# per-run delimiter so a fork can't embed a fixed heredoc terminator to
|
||||
# inject extra outputs — e.g. is_fork=false (last-write-wins), which
|
||||
# would re-enable the EE checkout and trusted-path settings for forks.
|
||||
RAND=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')
|
||||
TITLE_EOF="TITLE_EOF_${RAND}"
|
||||
BODY_EOF="BODY_EOF_${RAND}"
|
||||
{
|
||||
echo "skip=false"
|
||||
echo "is_fork=$IS_FORK"
|
||||
echo "pr_number=$PR_NUMBER"
|
||||
echo "base_ref=$BASE_REF"
|
||||
echo "base_sha=$BASE_SHA"
|
||||
echo "head_sha=$HEAD_SHA"
|
||||
echo "pr_author=$PR_AUTHOR"
|
||||
echo "title<<$TITLE_EOF"
|
||||
printf '%s\n' "$PR_TITLE"
|
||||
echo "$TITLE_EOF"
|
||||
echo "body<<$BODY_EOF"
|
||||
printf '%s\n' "$PR_BODY"
|
||||
echo "$BODY_EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Checkout repository
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
ref: refs/pull/${{ steps.pr.outputs.pr_number }}/merge
|
||||
fetch-depth: 1
|
||||
# Don't persist github.token in .git/config: the review agent can read
|
||||
# the checkout, and on the fork path that token (issue/PR write) would
|
||||
# otherwise be exfiltratable. All later git ops target the public origin
|
||||
# and need no auth; EE checkout and gh use their own explicit tokens.
|
||||
persist-credentials: false
|
||||
|
||||
# Never expose the EE private-repo token to untrusted fork code. Skipping
|
||||
# this step leaves steps.ee.outputs.available empty, so the EE checkout and
|
||||
# substitution steps below are skipped too.
|
||||
- name: Check EE access
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true' && steps.pr.outputs.is_fork != 'true'
|
||||
id: ee
|
||||
env:
|
||||
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
run: |
|
||||
if [ -n "$EE_TOKEN" ]; then
|
||||
echo "available=true" >> "$GITHUB_OUTPUT"
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "available=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Checkout EE repository
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ steps.ee.outputs.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Substitute EE code
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Set up Node.js
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Install Codex CLI
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
run: npm install --global @openai/codex@0.144.1
|
||||
|
||||
- name: Configure Codex auth
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
|
||||
run: |
|
||||
CODEX_HOME="$HOME/.codex"
|
||||
echo "CODEX_HOME=$CODEX_HOME" >> "$GITHUB_ENV"
|
||||
mkdir -p "$CODEX_HOME"
|
||||
chmod 700 "$CODEX_HOME"
|
||||
cat > "$CODEX_HOME/config.toml" <<'EOF'
|
||||
cli_auth_credentials_store = "file"
|
||||
EOF
|
||||
if [ -n "$OPENAI_API_KEY" ]; then
|
||||
printf '%s' "$OPENAI_API_KEY" | codex login --with-api-key
|
||||
else
|
||||
printf '%s' "$CODEX_AUTH_JSON" > "$CODEX_HOME/auth.json"
|
||||
chmod 600 "$CODEX_HOME/auth.json"
|
||||
node -e 'JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"))' "$CODEX_HOME/auth.json"
|
||||
fi
|
||||
|
||||
- name: Pre-fetch base and head refs for the PR
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
run: |
|
||||
git fetch --no-tags origin \
|
||||
"$PR_BASE_REF" \
|
||||
"+refs/pull/$PR_NUMBER/head"
|
||||
|
||||
- name: Fetch prior PR discussion
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
run: |
|
||||
# Write outside the checkout: on the fork path the merge tree is
|
||||
# attacker-controlled, and a committed symlink at this path would
|
||||
# redirect the write.
|
||||
gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" \
|
||||
--jq '[.[] | {user: .user.login, created_at: .created_at, body: (.body | .[:4000])}] | sort_by(.created_at) | .[-20:]' \
|
||||
> "$RUNNER_TEMP/prior-comments.json" || echo "[]" > "$RUNNER_TEMP/prior-comments.json"
|
||||
|
||||
- name: Write Codex review context
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
PR_REPOSITORY: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
|
||||
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
|
||||
PR_TITLE: ${{ steps.pr.outputs.title }}
|
||||
PR_BODY: ${{ steps.pr.outputs.body }}
|
||||
PR_AUTHOR: ${{ steps.pr.outputs.pr_author }}
|
||||
EXTRA_PROMPT: ${{ inputs.extra_prompt }}
|
||||
run: |
|
||||
node <<'NODE'
|
||||
const fs = require('fs');
|
||||
const tmp = process.env.RUNNER_TEMP;
|
||||
const lines = [
|
||||
`Repository: ${process.env.PR_REPOSITORY}`,
|
||||
`PR number: ${process.env.PR_NUMBER}`,
|
||||
];
|
||||
if (process.env.PR_AUTHOR) {
|
||||
lines.push(`PR AUTHOR: ${process.env.PR_AUTHOR}`);
|
||||
}
|
||||
lines.push(
|
||||
`Base SHA: ${process.env.PR_BASE_SHA}`,
|
||||
`Head SHA: ${process.env.PR_HEAD_SHA}`,
|
||||
'',
|
||||
'PR title:',
|
||||
process.env.PR_TITLE || '(empty)',
|
||||
'',
|
||||
'PR body:',
|
||||
process.env.PR_BODY || '(empty)',
|
||||
'',
|
||||
'Changed commits command:',
|
||||
`git log --oneline ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
|
||||
'',
|
||||
'Changed files command:',
|
||||
`git diff --stat ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
|
||||
'',
|
||||
'Full review diff command:',
|
||||
`git diff --unified=0 ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`
|
||||
);
|
||||
if (process.env.EXTRA_PROMPT && process.env.EXTRA_PROMPT.trim()) {
|
||||
lines.push('', 'Additional reviewer instructions:', process.env.EXTRA_PROMPT.trim());
|
||||
}
|
||||
if (fs.existsSync(`${tmp}/prior-comments.json`)) {
|
||||
try {
|
||||
const comments = JSON.parse(fs.readFileSync(`${tmp}/prior-comments.json`, 'utf8'));
|
||||
if (Array.isArray(comments) && comments.length > 0) {
|
||||
lines.push(
|
||||
'',
|
||||
'Prior PR discussion (most recent up to 20 comments):',
|
||||
'',
|
||||
'If you have already reviewed this PR (look for your own earlier "## Codex Review" comment), focus on what changed since then per the diff and respect any decisions the human made in replies. Do not re-flag findings the human already pushed back on.',
|
||||
''
|
||||
);
|
||||
for (const c of comments) {
|
||||
lines.push(`### @${c.user} (${c.created_at})`, '', c.body, '', '---', '');
|
||||
}
|
||||
}
|
||||
} catch (_) {}
|
||||
}
|
||||
fs.writeFileSync(`${tmp}/pr-review-context.md`, `${lines.join('\n')}\n`);
|
||||
NODE
|
||||
|
||||
- name: Run Codex review
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
PR_IS_FORK: ${{ steps.pr.outputs.is_fork }}
|
||||
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
|
||||
run: |
|
||||
if [ "$PR_IS_FORK" = "true" ]; then
|
||||
# Fork code is untrusted. Read the review policy/prompt from the base
|
||||
# ref (git show) rather than the attacker-controlled merge checkout,
|
||||
# so a malicious fork can't rewrite the reviewer's own instructions,
|
||||
# and run in a network-disabled sandbox to block secret exfiltration.
|
||||
git show "origin/$PR_BASE_REF:REVIEW.md" > /tmp/codex-prompt.md
|
||||
git show "origin/$PR_BASE_REF:.github/codex/pr-review.prompt.md" >> /tmp/codex-prompt.md
|
||||
SANDBOX_MODE=workspace-write
|
||||
else
|
||||
cat REVIEW.md .github/codex/pr-review.prompt.md > /tmp/codex-prompt.md
|
||||
SANDBOX_MODE=danger-full-access
|
||||
fi
|
||||
# The context file lives in RUNNER_TEMP (outside the attacker-controlled
|
||||
# checkout); tell the agent its absolute path.
|
||||
printf '\nReview context file (absolute path): %s\n' "$RUNNER_TEMP/pr-review-context.md" >> /tmp/codex-prompt.md
|
||||
# Write the final message outside the checkout too: a fork could commit
|
||||
# codex-final-message.md as a symlink and redirect this write to overwrite
|
||||
# e.g. a GitHub Action's index.js, which then runs with our credentials.
|
||||
codex exec \
|
||||
-C "$GITHUB_WORKSPACE" \
|
||||
-m gpt-5.6-sol \
|
||||
-c 'model_reasoning_effort="xhigh"' \
|
||||
-s "$SANDBOX_MODE" \
|
||||
-o "$RUNNER_TEMP/codex-final-message.md" \
|
||||
- < /tmp/codex-prompt.md
|
||||
|
||||
- name: Post Codex review comment
|
||||
if: steps.codex_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
uses: actions/github-script@v7
|
||||
env:
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
|
||||
GH_JOB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
github-token: ${{ github.token }}
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
const path = `${process.env.RUNNER_TEMP}/codex-final-message.md`;
|
||||
if (!fs.existsSync(path)) {
|
||||
core.info('Codex did not produce a final message; skipping PR comment.');
|
||||
return;
|
||||
}
|
||||
let body = fs.readFileSync(path, 'utf8').trim();
|
||||
if (!body) {
|
||||
core.info('Codex final message was empty; skipping PR comment.');
|
||||
return;
|
||||
}
|
||||
// Defense-in-depth for fork reviews: the model call needs the provider
|
||||
// credential in the env, and the posted comment bypasses Actions log
|
||||
// masking. Strip any credential (API key, raw auth JSON, nested
|
||||
// tokens) that leaked into the review text before posting.
|
||||
const secrets = [];
|
||||
const addSecret = (v, min) => {
|
||||
if (typeof v === 'string' && v.length >= min) secrets.push(v);
|
||||
};
|
||||
addSecret(process.env.OPENAI_API_KEY, 8);
|
||||
addSecret(process.env.CODEX_AUTH_JSON, 8);
|
||||
addSecret(process.env.GH_JOB_TOKEN, 8);
|
||||
if (process.env.CODEX_AUTH_JSON) {
|
||||
try {
|
||||
const collect = (o) => {
|
||||
if (typeof o === 'string') addSecret(o, 20);
|
||||
else if (Array.isArray(o)) o.forEach(collect);
|
||||
else if (o && typeof o === 'object') Object.values(o).forEach(collect);
|
||||
};
|
||||
collect(JSON.parse(process.env.CODEX_AUTH_JSON));
|
||||
} catch (_) {}
|
||||
}
|
||||
for (const s of [...new Set(secrets)].sort((a, b) => b.length - a.length)) {
|
||||
body = body.split(s).join('[REDACTED]');
|
||||
}
|
||||
body = body.trim();
|
||||
if (!body) {
|
||||
core.info('Codex final message was empty after redaction; skipping PR comment.');
|
||||
return;
|
||||
}
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: Number(process.env.PR_NUMBER),
|
||||
body,
|
||||
});
|
||||
@@ -0,0 +1,48 @@
|
||||
name: Publish deno-client
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
env:
|
||||
repo: windmill-deno-client
|
||||
|
||||
jobs:
|
||||
build_deno_and_push_to_repo:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: generate_deno
|
||||
run: |
|
||||
cd deno-client
|
||||
rm .gitignore
|
||||
./build.sh
|
||||
- name: Pushes to another repository
|
||||
id: push_directory
|
||||
uses: cpina/github-action-push-to-another-repository@devel
|
||||
env:
|
||||
API_TOKEN_GITHUB: ${{ secrets.DENO_PAT }}
|
||||
with:
|
||||
source-directory: deno-client/
|
||||
destination-github-username: ${{ github.repository_owner }}
|
||||
destination-repository-name: ${{ env.repo }}
|
||||
user-email: ruben@windmill.dev
|
||||
commit-message: See ORIGIN_COMMIT from $GITHUB_REF
|
||||
target-branch: main
|
||||
|
||||
tag_repo:
|
||||
needs: [build_deno_and_push_to_repo]
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: ${{ github.repository_owner }}/${{ env.repo }}
|
||||
token: ${{ secrets.DENO_PAT }}
|
||||
path: ./client
|
||||
|
||||
- name: Push client
|
||||
run: |
|
||||
cd ./client
|
||||
git config --global user.email "ruben@windmill.dev"
|
||||
git config --global user.name "rubenfiszel[bot]"
|
||||
git tag -a ${{ github.ref_name }} -m "${{ github.ref_name }}"
|
||||
git push --tags
|
||||
@@ -1,58 +0,0 @@
|
||||
name: Create discord thread when a PR is opened, react with green checkmark when PR is merged
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types:
|
||||
- opened
|
||||
- ready_for_review
|
||||
- closed
|
||||
issue_comment:
|
||||
types:
|
||||
- created
|
||||
- edited
|
||||
|
||||
jobs:
|
||||
notify_discord_when_pr_opened:
|
||||
if: (github.event.pull_request.draft == false) && (github.event.action == 'opened' || github.event.action == 'ready_for_review')
|
||||
uses: ./.github/workflows/shareable-discord-notification.yml
|
||||
with:
|
||||
PR_TITLE: ${{ github.event.pull_request.title }}
|
||||
PR_URL: ${{ github.event.pull_request.html_url }}
|
||||
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
PR_STATUS: "opened"
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
DISCORD_CHANNEL_ID: "1372204995868491786"
|
||||
DISCORD_GUILD_ID: "930051556043276338"
|
||||
secrets:
|
||||
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_PR_REVIEWS_WEBHOOK }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
merge_success_emoji:
|
||||
if: github.event.action == 'closed'
|
||||
uses: ./.github/workflows/shareable-discord-notification.yml
|
||||
with:
|
||||
PR_STATUS: "merged"
|
||||
DISCORD_CHANNEL_ID: "1372204995868491786"
|
||||
DISCORD_GUILD_ID: "930051556043276338"
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
secrets:
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
notify_discord_on_comment:
|
||||
if: >
|
||||
github.event_name == 'issue_comment'
|
||||
&& github.event.issue.pull_request
|
||||
&& github.event.comment.user.login != 'cloudflare-workers-and-pages[bot]'
|
||||
&& github.event.comment.user.login != 'ellipsis-dev[bot]'
|
||||
uses: ./.github/workflows/shareable-discord-notification.yml
|
||||
with:
|
||||
PR_STATUS: "comment"
|
||||
PR_NUMBER: ${{ github.event.issue.number }}
|
||||
COMMENT_BODY: ${{ github.event.comment.body }}
|
||||
COMMENT_AUTHOR: ${{ github.event.comment.user.login }}
|
||||
COMMENT_URL: ${{ github.event.comment.html_url }}
|
||||
COMMENT_IS_EDIT: ${{ github.event.action == 'edited' }}
|
||||
DISCORD_CHANNEL_ID: "1372204995868491786"
|
||||
DISCORD_GUILD_ID: "930051556043276338"
|
||||
secrets:
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
@@ -67,7 +67,7 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ce_rpi
|
||||
features=embedding,parquet,openidconnect,deno_core
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:dev
|
||||
${{ steps.meta-public.outputs.tags }}
|
||||
|
||||
+220
-108
@@ -1,9 +1,12 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.event_name != 'pull_request' && github.event_name !=
|
||||
'workflow_dispatch' && github.repository || 'windmill-labs/windmill-test' }}
|
||||
DEV_SHA: ${{ github.event_name != 'pull_request' && github.event_name !=
|
||||
'workflow_dispatch' && 'dev' || github.event.inputs.tag || github.sha }}
|
||||
IMAGE_NAME:
|
||||
${{ github.event_name != 'pull_request' && github.repository ||
|
||||
'windmill-labs/windmill-test' }}
|
||||
DEV_SHA:
|
||||
${{ github.event_name != 'pull_request' && 'dev' || format('pr-{0}',
|
||||
github.event.number) }}
|
||||
|
||||
name: Build windmill:main
|
||||
on:
|
||||
push:
|
||||
@@ -13,37 +16,20 @@ on:
|
||||
types: [opened, synchronize, reopened]
|
||||
paths:
|
||||
- "Dockerfile"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
ee:
|
||||
description: "Build EE image (true, false)"
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
tag:
|
||||
description: "Tag the image"
|
||||
required: true
|
||||
default: "test"
|
||||
slim:
|
||||
description: "Build slim image (true, false)"
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'workflow_dispatch') || (github.event.inputs &&
|
||||
!github.event.inputs.ee)
|
||||
if: (github.event_name != 'issue_comment') || (contains(github.event.comment.body, '/buildimage_all') || contains(github.event.comment.body, '/buildimage_base'))
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
@@ -92,20 +78,20 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ce
|
||||
features=embedding,parquet,openidconnect,jemalloc,deno_core
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
${{ steps.meta-public.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=AGPLv3
|
||||
|
||||
build_ee:
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'workflow_dispatch') || github.event.inputs.ee
|
||||
if: (github.event_name != 'issue_comment') || (contains(github.event.comment.body, '/buildimage_ee') || contains(github.event.comment.body, '/buildimage_nsjail')) || contains(github.event.comment.body, '/buildimage_all')
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
@@ -154,7 +140,7 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,deno_core
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
@@ -162,6 +148,101 @@ jobs:
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
build_ee_312:
|
||||
runs-on: ubicloud
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public-py312
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-py312
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Substitute EE code
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Build and push publicly ee
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,deno_core
|
||||
PYTHON_IMAGE=python:3.12.2-slim-bookworm
|
||||
tags: |
|
||||
${{ steps.meta-ee-public-py312.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta-ee-public-py312.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
# disabled until we make it 100% reliable and add more meaningful tests
|
||||
# playwright:
|
||||
# runs-on: [self-hosted, new]
|
||||
# needs: [build]
|
||||
# services:
|
||||
# postgres:
|
||||
# image: postgres
|
||||
# env:
|
||||
# POSTGRES_DB: windmill
|
||||
# POSTGRES_USER: admin
|
||||
# POSTGRES_PASSWORD: changeme
|
||||
# ports:
|
||||
# - 5432:5432
|
||||
# options: >-
|
||||
# --health-cmd pg_isready
|
||||
# --health-interval 10s
|
||||
# --health-timeout 5s
|
||||
# --health-retries 5
|
||||
# steps:
|
||||
# - uses: actions/checkout@v4
|
||||
# - name: "Docker"
|
||||
# run: echo "::set-output name=id::$(docker run --network=host --rm -d -p 8000:8000 --privileged -it -e DATABASE_URL=postgres://admin:changeme@localhost:5432/windmill -e BASE_INTERNAL_URL=http://localhost:8000 ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest)"
|
||||
# id: docker-container
|
||||
# - uses: actions/setup-node@v3
|
||||
# with:
|
||||
# node-version: 16
|
||||
# - name: "Playwright run"
|
||||
# timeout-minutes: 2
|
||||
# run: cd frontend && npm ci @playwright/test && npx playwright install && export BASE_URL=http://localhost:8000 && npm run test
|
||||
# - name: "Clean up"
|
||||
# run: docker kill ${{ steps.docker-container.outputs.id }}
|
||||
# if: always()
|
||||
|
||||
attach_amd64_binary_to_release:
|
||||
needs: [build, build_ee]
|
||||
runs-on: ubicloud
|
||||
@@ -170,6 +251,7 @@ jobs:
|
||||
ARCH: amd64
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- run: |
|
||||
# pulling docker image with desired arch so that actions-docker-extract doesn't do it
|
||||
docker pull --platform "linux/$ARCH" ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
@@ -187,12 +269,6 @@ jobs:
|
||||
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
path: "/usr/src/app/windmill"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-duckdb-ffi-internal
|
||||
with:
|
||||
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
path: "/usr/src/app/libwindmill_duckdb_ffi_internal.so"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee
|
||||
with:
|
||||
@@ -210,60 +286,6 @@ jobs:
|
||||
files: |
|
||||
${{ steps.extract.outputs.destination }}/*
|
||||
${{ steps.extract-ee.outputs.destination }}/*
|
||||
${{ steps.extract-duckdb-ffi-internal.outputs.destination }}/*
|
||||
|
||||
attach_ee_debug_to_release:
|
||||
needs: [build_ee]
|
||||
runs-on: ubicloud
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
strategy:
|
||||
matrix:
|
||||
platform: [linux/amd64, linux/arm64]
|
||||
include:
|
||||
- platform: linux/amd64
|
||||
arch: amd64
|
||||
- platform: linux/arm64
|
||||
arch: arm64
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
- name: Substitute EE code
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Extract EE debug info from builder stage (depot cache hit)
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: ${{ matrix.platform }}
|
||||
target: debuginfo
|
||||
build-args: |
|
||||
features=ee
|
||||
outputs: type=local,dest=./debuginfo
|
||||
|
||||
- name: Rename debug file with corresponding architecture
|
||||
run: |
|
||||
mv ./debuginfo/windmill.debug ./debuginfo/windmill-ee-${{ matrix.arch }}.debug
|
||||
|
||||
- name: Attach debug file to release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: ./debuginfo/windmill-ee-${{ matrix.arch }}.debug
|
||||
|
||||
# attach_arm64_binary_to_release:
|
||||
# needs: [build, build_ee]
|
||||
@@ -315,7 +337,6 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
fetch-depth: 0
|
||||
- name: Prepare test run
|
||||
if: ${{ ! startsWith(github.ref, 'refs/tags/v') }}
|
||||
@@ -327,7 +348,7 @@ jobs:
|
||||
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
|
||||
run: cd integration_tests && ./run.sh
|
||||
- name: Archive logs
|
||||
uses: actions/upload-artifact@v4
|
||||
uses: actions/upload-artifact@v3
|
||||
if: always()
|
||||
with:
|
||||
name: Windmill Integration Tests Logs
|
||||
@@ -337,9 +358,7 @@ jobs:
|
||||
tag_latest:
|
||||
runs-on: ubicloud
|
||||
needs: [run_integration_test, build]
|
||||
if:
|
||||
github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' ||
|
||||
startsWith(github.ref, 'refs/tags/v')) && (github.event_name != 'workflow_dispatch')
|
||||
if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -358,9 +377,7 @@ jobs:
|
||||
tag_latest_ee:
|
||||
runs-on: ubicloud
|
||||
needs: [run_integration_test, build_ee]
|
||||
if:
|
||||
github.event_name != 'pull_request' && (github.event_name != 'workflow_dispatch') && (github.ref == 'refs/heads/main' ||
|
||||
startsWith(github.ref, 'refs/tags/v'))
|
||||
if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -379,7 +396,7 @@ jobs:
|
||||
verify_ee_image_vulnerabilities:
|
||||
runs-on: ubicloud
|
||||
needs: [tag_latest_ee]
|
||||
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name != 'workflow_dispatch')
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -418,10 +435,103 @@ jobs:
|
||||
# ignore-unchanged: true
|
||||
# only-fixed: true
|
||||
|
||||
build_ee_nsjail:
|
||||
needs: [build_ee]
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'issue_comment') || (github.event_name != 'pull_request') || (contains(github.event.comment.body, '/buildimage_nsjail') || contains(github.event.comment.body, '/buildimage_all'))
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push publicly ee
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
file: "./docker/DockerfileNsjail"
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
build_ee_reports_privately:
|
||||
needs: [build_ee_nsjail]
|
||||
runs-on: ubicloud
|
||||
if: github.event_name != 'pull_request'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-reports
|
||||
tags: |
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
|
||||
|
||||
- name: Build and push publicly ee reports
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
file: "./docker/DockerfileReports"
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
publish_ecr_s3:
|
||||
needs: [build_ee_full]
|
||||
needs: [build_ee_nsjail]
|
||||
runs-on: ubicloud-standard-2-arm
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
if: github.event_name != 'pull_request'
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
@@ -440,18 +550,23 @@ jobs:
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Get version from tag
|
||||
id: version
|
||||
run: echo "VERSION=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"
|
||||
- name: get git hash
|
||||
if: github.event_name != 'pull_request'
|
||||
id: git_hash
|
||||
run: |
|
||||
git_hash=$(git rev-parse --short "$GITHUB_SHA")
|
||||
echo "GIT_HASH=${git_hash:0:7}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
if: github.event_name != 'pull_request'
|
||||
id: extract
|
||||
with:
|
||||
image: |-
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-full:${{ steps.version.outputs.VERSION }}
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail:${{ steps.git_hash.outputs.GIT_HASH }}
|
||||
path: "/static_frontend/."
|
||||
|
||||
- uses: reggionick/s3-deploy@v4
|
||||
if: github.event_name != 'pull_request'
|
||||
with:
|
||||
folder: ${{ steps.extract.outputs.destination }}
|
||||
bucket: windmill-frontend
|
||||
@@ -465,7 +580,6 @@ jobs:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
ref: ${{ github.ref }}
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
@@ -546,10 +660,9 @@ jobs:
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
|
||||
build_ee_slim:
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
needs: [build_ee]
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'pull_request') && ((github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.slim))
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -567,7 +680,6 @@ jobs:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
@@ -582,7 +694,7 @@ jobs:
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
file: "./docker/DockerfileSlimEe"
|
||||
tags: |
|
||||
|
||||
@@ -1,27 +1,21 @@
|
||||
name: check frontend build
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Change versions"]
|
||||
types:
|
||||
- completed
|
||||
|
||||
merge_group:
|
||||
push:
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, closed]
|
||||
paths:
|
||||
- "frontend/**"
|
||||
- ".github/workflows/frontend-check.yml"
|
||||
merge_group:
|
||||
|
||||
jobs:
|
||||
npm_check:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v5
|
||||
- uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: 24
|
||||
cache: "npm"
|
||||
cache-dependency-path: "frontend/package-lock.json"
|
||||
node-version: 18
|
||||
- name: "npm check"
|
||||
timeout-minutes: 5
|
||||
run: cd frontend && npm ci && npm run generate-backend-client && npm run
|
||||
timeout-minutes: 2
|
||||
run:
|
||||
cd frontend && npm ci && npm run generate-backend-client && npm run
|
||||
check
|
||||
|
||||
@@ -1,355 +0,0 @@
|
||||
name: Git commands
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
# /command comments can come from anyone; author_association misses private org
|
||||
# members, so check-access resolves them via the internal app token. Runs once and is
|
||||
# OR'd into each job's guard (public members still pass on author_association alone).
|
||||
check-access:
|
||||
if: github.event.issue.pull_request != null && startsWith(github.event.comment.body, '/')
|
||||
uses: ./.github/workflows/check-write-access.yml
|
||||
with:
|
||||
username: ${{ github.event.comment.user.login }}
|
||||
secrets: inherit
|
||||
|
||||
update-sqlx:
|
||||
needs: [check-access]
|
||||
if: >-
|
||||
github.event.issue.pull_request &&
|
||||
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
|
||||
startsWith(github.event.comment.body, '/updatesqlx')
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_DB: windmill
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
|
||||
- name: Comment on PR - Starting
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: `Starting sqlx update...\n\n[View workflow run](${runUrl})`
|
||||
})
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
ref: ${{ github.event.issue.pull_request.head.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Checkout windmill-ee-private
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: windmill-ee-private
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
# Setup Rust toolchain
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
|
||||
- name: Install xmlsec and gssapi build-time deps
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y --no-install-recommends \
|
||||
pkg-config libxml2-dev libssl-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang \
|
||||
xmlsec1 libxmlsec1-dev libxmlsec1-openssl
|
||||
|
||||
- name: Run update-sqlx script
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:postgres@localhost:5432/windmill
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
set -e # Exit on any command failure
|
||||
PR_NUMBER=${{ github.event.issue.number }}
|
||||
|
||||
# Set up error trap to comment on PR for any failure
|
||||
trap 'gh pr comment $PR_NUMBER --body "❌ SQLx update failed. Please check the workflow logs for details."' ERR
|
||||
|
||||
BRANCH_NAME=$(gh pr view $PR_NUMBER --json headRefName --jq .headRefName)
|
||||
echo "Checking out PR branch: $BRANCH_NAME"
|
||||
git checkout $BRANCH_NAME
|
||||
git config --local user.email "windmill-internal-app[bot]@users.noreply.github.com"
|
||||
git config --local user.name "windmill-internal-app[bot]"
|
||||
git config pull.rebase true
|
||||
git pull origin $BRANCH_NAME
|
||||
|
||||
# Checkout the correct windmill-ee-private commit from ee-repo-ref.txt
|
||||
if [ -f backend/ee-repo-ref.txt ]; then
|
||||
EE_REF=$(cat backend/ee-repo-ref.txt | tr -d '[:space:]')
|
||||
echo "Checking out windmill-ee-private at commit: $EE_REF"
|
||||
cd windmill-ee-private
|
||||
git fetch origin $EE_REF
|
||||
git checkout $EE_REF
|
||||
cd ..
|
||||
else
|
||||
echo "Warning: ee-repo-ref.txt not found, using default branch"
|
||||
fi
|
||||
|
||||
mkdir -p frontend/build
|
||||
cd backend
|
||||
cargo install sqlx-cli --version 0.8.5
|
||||
sqlx migrate run
|
||||
./substitute_ee_code.sh --dir ./windmill-ee-private
|
||||
./update_sqlx.sh
|
||||
# Pass the branch name to the next step
|
||||
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
|
||||
|
||||
- name: Commit changes if any
|
||||
run: |
|
||||
git add backend/.sqlx
|
||||
git commit -m "Update SQLx metadata"
|
||||
git push origin ${{ env.BRANCH_NAME }}
|
||||
|
||||
- name: Comment on PR - Completed
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: 'Successfully ran sqlx update'
|
||||
})
|
||||
|
||||
demo:
|
||||
needs: [check-access]
|
||||
if: >-
|
||||
github.event.issue.pull_request &&
|
||||
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
|
||||
startsWith(github.event.comment.body, '/demo')
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Run Claude Code for Demo Generation
|
||||
uses: anthropics/claude-code-action@beta
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
timeout_minutes: "10"
|
||||
allowed_tools: "Bash"
|
||||
direct_prompt: |
|
||||
You need to:
|
||||
|
||||
1. Extract the Cloudflare preview URL from the cloudflare-workers-and-pages bot comment in this PR
|
||||
2. Analyze the PR changes to understand what feature was added/modified
|
||||
3. Create detailed instructions to give to an AI agent that will click and interact with buttons and inputs to showcase the new feature. Only include the instructions, nothing else.
|
||||
4. Create a demo.json file with a valid JSON object containing:
|
||||
- instructions: the demo instructions
|
||||
- url: the preview URL
|
||||
5. VALIDATE the JSON file using `jq` before finishing
|
||||
DO NOT COMMIT THIS FILE TO THE PR.
|
||||
|
||||
Example demo.json:
|
||||
{
|
||||
"instructions": "Click on settings, then account settings, then 'generate new token'",
|
||||
"url": "https://example.pages.dev"
|
||||
}
|
||||
|
||||
CRITICAL: After creating demo.json, you MUST:
|
||||
1. Run `jq empty demo.json` to validate the JSON is properly formatted
|
||||
2. If validation fails, fix the JSON and validate again
|
||||
3. Only proceed once the JSON passes validation
|
||||
4. Use proper JSON escaping for newlines, quotes, and special characters
|
||||
|
||||
Make sure to:
|
||||
- Create a valid JSON object that passes `jq empty demo.json`
|
||||
- Extract the correct preview URL (should be a .pages.dev domain)
|
||||
- Create specific, actionable demo steps based on the actual changes in the PR
|
||||
- Properly escape all strings in the JSON (use jq to create the file if needed)
|
||||
- NOT COMMIT THE DEMO.JSON FILE TO THE PR
|
||||
|
||||
- name: Send instructions to Windmill
|
||||
env:
|
||||
DEMO_WEBHOOK_TOKEN: ${{ secrets.DEMO_WEBHOOK_TOKEN }}
|
||||
run: |
|
||||
if [[ -f "demo.json" ]]; then
|
||||
echo "Found demo.json, sending to Windmill..."
|
||||
cat demo.json
|
||||
|
||||
# Validate JSON one more time (Claude should have already done this)
|
||||
if ! jq empty demo.json; then
|
||||
echo "Error: demo.json is not valid JSON"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RESULT=$(curl -s \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H "Authorization: Bearer $DEMO_WEBHOOK_TOKEN" \
|
||||
-X POST \
|
||||
-d @demo.json \
|
||||
'https://app.windmill.dev/api/w/windmill-labs/jobs/run/f/f/ai/browserbase_demo')
|
||||
|
||||
echo "Windmill response:"
|
||||
echo -E "$RESULT"
|
||||
else
|
||||
echo "Error: demo.json file not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
update-ee-ref:
|
||||
needs: [check-access]
|
||||
if: >-
|
||||
github.event.issue.pull_request &&
|
||||
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
|
||||
startsWith(github.event.comment.body, '/eeref')
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
|
||||
- name: Comment on PR - Starting
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: `Starting ee ref update...\n\n[View workflow run](${runUrl})`
|
||||
})
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
ref: ${{ github.event.issue.pull_request.head.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Checkout windmill-ee-private
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: windmill-ee-private
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
- name: Get last commit hash of private-repo
|
||||
id: get-commit-hash
|
||||
run: |
|
||||
cd windmill-ee-private
|
||||
COMMIT_HASH=$(git rev-parse HEAD)
|
||||
echo "commit_hash=$COMMIT_HASH" >> $GITHUB_OUTPUT
|
||||
echo "Latest commit hash: $COMMIT_HASH"
|
||||
|
||||
- name: Update ee-repo-ref.txt
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
set -e # Exit on any command failure
|
||||
PR_NUMBER=${{ github.event.issue.number }}
|
||||
|
||||
# Set up error trap to comment on PR for any failure
|
||||
trap 'gh pr comment $PR_NUMBER --body "❌ EE ref update failed. Please check the workflow logs for details."' ERR
|
||||
|
||||
BRANCH_NAME=$(gh pr view $PR_NUMBER --json headRefName --jq .headRefName)
|
||||
echo "Checking out PR branch: $BRANCH_NAME"
|
||||
git checkout $BRANCH_NAME
|
||||
git config --local user.email "windmill-internal-app[bot]@users.noreply.github.com"
|
||||
git config --local user.name "windmill-internal-app[bot]"
|
||||
git config pull.rebase true
|
||||
git pull origin $BRANCH_NAME
|
||||
echo "${{ steps.get-commit-hash.outputs.commit_hash }}" > backend/ee-repo-ref.txt
|
||||
echo "Updated backend/ee-repo-ref.txt with commit hash: ${{ steps.get-commit-hash.outputs.commit_hash }}"
|
||||
# commit and push the changes
|
||||
git add backend/ee-repo-ref.txt
|
||||
git commit -m "Update ee-repo-ref.txt" || echo "No changes to commit"
|
||||
git push origin $BRANCH_NAME
|
||||
|
||||
- name: Comment on PR - Completed
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: 'Successfully updated ee-repo-ref.txt'
|
||||
})
|
||||
|
||||
update-docs:
|
||||
needs: [check-access]
|
||||
if: >-
|
||||
github.event.issue.pull_request &&
|
||||
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) || needs.check-access.outputs.authorized == 'true') &&
|
||||
startsWith(github.event.comment.body, '/docs')
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
owner: ${{ github.repository_owner }}
|
||||
repositories: |
|
||||
windmilldocs
|
||||
|
||||
- name: Trigger docs update
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
COMMENT_TEXT: ${{ github.event.comment.body }}
|
||||
run: |
|
||||
jq -n \
|
||||
--argjson pr_number ${{ github.event.issue.number }} \
|
||||
--arg repo "${{ github.event.repository.name }}" \
|
||||
--arg comment "$COMMENT_TEXT" \
|
||||
'{event_type: "create-docs", client_payload: {pr_number: $pr_number, repo: $repo, comment_text: $comment}}' | \
|
||||
gh api repos/windmill-labs/windmilldocs/dispatches \
|
||||
--method POST \
|
||||
--input -
|
||||
@@ -1,222 +0,0 @@
|
||||
name: Git Sync Integration Tests
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- "backend/windmill-git-sync/**"
|
||||
- "backend/windmill-api-integration-tests/tests/git_sync*"
|
||||
- "backend/ee-repo-ref.txt"
|
||||
- "backend/windmill-common/src/workspaces.rs"
|
||||
- "backend/windmill-worker/src/result_processor.rs"
|
||||
- "backend/windmill-api-workspaces/**"
|
||||
- "cli/src/commands/sync/**"
|
||||
- "cli/src/utils/git.ts"
|
||||
- "integration_tests/test/git_sync_test.py"
|
||||
- ".github/workflows/git-sync-test.yml"
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened]
|
||||
paths:
|
||||
- "backend/windmill-git-sync/**"
|
||||
- "backend/windmill-api-integration-tests/tests/git_sync*"
|
||||
- "backend/ee-repo-ref.txt"
|
||||
- "backend/windmill-common/src/workspaces.rs"
|
||||
- "backend/windmill-worker/src/result_processor.rs"
|
||||
- "backend/windmill-api-workspaces/**"
|
||||
- "cli/src/commands/sync/**"
|
||||
- "cli/src/utils/git.ts"
|
||||
- "integration_tests/test/git_sync_test.py"
|
||||
- ".github/workflows/git-sync-test.yml"
|
||||
|
||||
concurrency:
|
||||
group: git-sync-test-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
check-relevance:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
should_run: ${{ steps.check.outputs.should_run }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Check if git sync related files changed
|
||||
id: check
|
||||
env:
|
||||
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" = "pull_request" ]; then
|
||||
BASE=${{ github.event.pull_request.base.sha }}
|
||||
else
|
||||
BASE=${{ github.event.before }}
|
||||
fi
|
||||
|
||||
CHANGED_FILES=$(git diff --name-only "$BASE"..HEAD 2>/dev/null || echo "")
|
||||
echo "Changed files:"
|
||||
echo "$CHANGED_FILES"
|
||||
|
||||
# Direct git sync file changes — always relevant.
|
||||
if echo "$CHANGED_FILES" | grep -qE '^(backend/windmill-git-sync/|backend/windmill-worker/src/result_processor\.rs|backend/windmill-api-workspaces/|backend/windmill-api-integration-tests/tests/git_sync|backend/windmill-common/src/workspaces\.rs|cli/src/commands/sync/|cli/src/utils/git\.ts|integration_tests/test/git_sync|\.github/workflows/git-sync-test\.yml)'; then
|
||||
echo "should_run=true" >> "$GITHUB_OUTPUT"
|
||||
echo "Relevant: direct git sync file changes"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# If ee-repo-ref.txt changed, check if the EE diff touches windmill-git-sync/
|
||||
if echo "$CHANGED_FILES" | grep -q '^backend/ee-repo-ref.txt$'; then
|
||||
NEW_REF=$(cat backend/ee-repo-ref.txt)
|
||||
OLD_REF=$(git show "$BASE:backend/ee-repo-ref.txt" 2>/dev/null || echo "")
|
||||
|
||||
if [ -n "$OLD_REF" ] && [ "$OLD_REF" != "$NEW_REF" ]; then
|
||||
# Clone EE repo and check diff
|
||||
git clone --bare "https://x-access-token:${WINDMILL_EE_PRIVATE_ACCESS}@github.com/windmill-labs/windmill-ee-private.git" /tmp/ee-repo 2>/dev/null
|
||||
EE_CHANGED=$(git -C /tmp/ee-repo diff --name-only "$OLD_REF".."$NEW_REF" 2>/dev/null || echo "")
|
||||
echo "EE changed files:"
|
||||
echo "$EE_CHANGED"
|
||||
|
||||
if echo "$EE_CHANGED" | grep -q '^windmill-git-sync/'; then
|
||||
echo "should_run=true" >> "$GITHUB_OUTPUT"
|
||||
echo "Relevant: EE git sync files changed"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "should_run=false" >> "$GITHUB_OUTPUT"
|
||||
echo "No git sync relevant changes detected, skipping tests"
|
||||
|
||||
git_sync_e2e:
|
||||
needs: [check-relevance]
|
||||
if: needs.check-relevance.outputs.should_run == 'true'
|
||||
runs-on: ubicloud-standard-16
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:14
|
||||
ports:
|
||||
- 5432:5432
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Substitute EE code
|
||||
run: |
|
||||
cd backend && ./substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
|
||||
- uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- name: Install wmill CLI
|
||||
run: |
|
||||
cd cli && bash gen_wm_client.sh && bun install
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
|
||||
chmod +x "$HOME/.local/bin/wmill"
|
||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||
|
||||
- name: Build Windmill
|
||||
working-directory: ./backend
|
||||
env:
|
||||
SQLX_OFFLINE: true
|
||||
CARGO_BUILD_JOBS: 12
|
||||
RUSTFLAGS: ""
|
||||
run: |
|
||||
cargo build --features enterprise,private,license,zip
|
||||
|
||||
- name: Start Gitea
|
||||
run: |
|
||||
docker run -d --name gitea \
|
||||
-e GITEA__database__DB_TYPE=sqlite3 \
|
||||
-e GITEA__security__INSTALL_LOCK=true \
|
||||
-e GITEA__server__HTTP_PORT=3000 \
|
||||
-e GITEA__server__ROOT_URL=http://localhost:3000 \
|
||||
-e GITEA__service__DISABLE_REGISTRATION=false \
|
||||
-p 3000:3000 \
|
||||
gitea/gitea:1.22-rootless
|
||||
echo "Waiting for Gitea to be ready..."
|
||||
for i in $(seq 1 30); do
|
||||
if curl -sf http://localhost:3000/api/v1/version > /dev/null 2>&1; then
|
||||
echo "Gitea is ready"
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
curl -sf http://localhost:3000/api/v1/version > /dev/null || { echo "Gitea failed to start"; exit 1; }
|
||||
|
||||
- name: Start Windmill
|
||||
working-directory: ./backend
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
|
||||
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
|
||||
DENO_PATH: deno
|
||||
BUN_PATH: bun
|
||||
NODE_BIN_PATH: node
|
||||
# The auto-pull poller's SSRF guard rejects localhost git remotes;
|
||||
# the tests' Gitea runs on localhost.
|
||||
ALLOW_LOCAL_GIT_REMOTES: "true"
|
||||
run: |
|
||||
./target/debug/windmill &
|
||||
echo "Waiting for Windmill to be ready..."
|
||||
for i in $(seq 1 60); do
|
||||
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
|
||||
echo "Windmill is ready"
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; exit 1; }
|
||||
|
||||
- name: Run git sync E2E tests
|
||||
timeout-minutes: 10
|
||||
env:
|
||||
GITEA_DOCKER_URL: http://localhost:3000
|
||||
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
|
||||
run: |
|
||||
python3 -m venv .venv
|
||||
.venv/bin/pip install -r integration_tests/requirements.txt
|
||||
cd integration_tests && ../.venv/bin/python -m unittest -v test.git_sync_test
|
||||
|
||||
- name: Archive logs
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: Git Sync Integration Tests Logs
|
||||
path: |
|
||||
integration_tests/logs
|
||||
@@ -27,7 +27,7 @@ jobs:
|
||||
go build
|
||||
- name: Pushes to another repository
|
||||
id: push_directory
|
||||
uses: cpina/github-action-push-to-another-repository@55306faa4ed53b815ae49e564af8cfb359d32ae2 # v1.7.3
|
||||
uses: cpina/github-action-push-to-another-repository@devel
|
||||
env:
|
||||
API_TOKEN_GITHUB: ${{ secrets.DENO_PAT }}
|
||||
with:
|
||||
|
||||
@@ -1,91 +0,0 @@
|
||||
name: Publish Helm Chart on Release
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
jobs:
|
||||
bump-helm-version:
|
||||
runs-on: ubicloud-standard-2
|
||||
|
||||
steps:
|
||||
- name: Generate an installation token
|
||||
id: app
|
||||
uses: actions/create-github-app-token@v2
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
owner: windmill-labs
|
||||
|
||||
- name: Checkout on helm repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: windmill-labs/windmill-helm-charts
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
|
||||
- name: Get version
|
||||
id: get_version
|
||||
run: |
|
||||
echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
|
||||
|
||||
- name: Create new branch
|
||||
run: |
|
||||
# Check if branch already exists remotely
|
||||
if git ls-remote --heads origin bump-helm-version-${{ env.VERSION }} | grep -q bump-helm-version-${{ env.VERSION }}; then
|
||||
# Branch exists, check it out
|
||||
git fetch origin bump-helm-version-${{ env.VERSION }}
|
||||
git checkout bump-helm-version-${{ env.VERSION }}
|
||||
else
|
||||
# Create new branch
|
||||
git checkout -b bump-helm-version-${{ env.VERSION }}
|
||||
fi
|
||||
|
||||
git config --local user.email "action@github.com"
|
||||
git config --local user.name "GitHub Action"
|
||||
|
||||
- name: Bump helm version
|
||||
run: |
|
||||
# Get current version and increment it by 1
|
||||
CURRENT_VERSION=$(grep "version:" ./charts/windmill/Chart.yaml | awk '{print $2}' | head -n 1)
|
||||
NEW_VERSION=$(echo "$CURRENT_VERSION" | awk -F. '{$NF = $NF + 1;} 1' | sed 's/ /./g')
|
||||
sed -i "s/^version: .*/version: $NEW_VERSION/" ./charts/windmill/Chart.yaml
|
||||
|
||||
# Get the app version from the version
|
||||
VERSION=${{ env.VERSION }}
|
||||
APP_VERSION=${VERSION#refs/tag/}
|
||||
APP_VERSION=${APP_VERSION#v}
|
||||
APP_VERSION=${APP_VERSION%/}
|
||||
sed -i "s/appVersion: .*/appVersion: $APP_VERSION/" ./charts/windmill/Chart.yaml
|
||||
|
||||
- name: Close existing bump-helm PRs
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
# List open PR numbers whose title starts with the prefix
|
||||
prs=$(gh pr list \
|
||||
--state open \
|
||||
--search '"helm: bump version to" in:title' \
|
||||
--json number \
|
||||
-q '.[].number')
|
||||
|
||||
for pr in $prs; do
|
||||
echo "Closing outdated bump PR #$pr"
|
||||
gh pr close "$pr" \
|
||||
--comment "Closed automatically – superseded by a newer Helm-chart bump PR."
|
||||
done
|
||||
|
||||
- name: Commit and push
|
||||
run: |
|
||||
git add .
|
||||
git commit -m "Bump helm version to ${{ env.VERSION }}"
|
||||
git push origin bump-helm-version-${{ env.VERSION }}
|
||||
|
||||
- name: Create PR
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
gh pr create \
|
||||
--title "helm: bump version to ${{ env.VERSION }}" \
|
||||
--body "This PR was auto-generated to bring the helm chart up to date for [release ${{ env.VERSION }}](https://github.com/windmill-labs/windmill/releases/tag/v${{ env.VERSION }}) in the main repo." \
|
||||
--head bump-helm-version-${{ env.VERSION }} \
|
||||
--base main
|
||||
@@ -1,38 +0,0 @@
|
||||
name: Claude PR Assistant
|
||||
|
||||
on:
|
||||
repository_dispatch:
|
||||
types: [external_claude_issue_fix]
|
||||
|
||||
jobs:
|
||||
claude-code-action:
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Process inputs
|
||||
id: process_inputs
|
||||
shell: bash
|
||||
run: |
|
||||
ISSUE_TITLE="${{ github.event.client_payload.issue_title }}"
|
||||
INSTRUCTION="${{ github.event.client_payload.instruction }}"
|
||||
ISSUE_BODY=$(printf '%q' "${{ github.event.client_payload.issue_body }}")
|
||||
BASE_PROMPT="Try to fix the following issue based on the instruction given. You are provided with the issue title, issue body, and instruction. You are to fix the issue based on the instruction. You are to create a pull request to fix the issue."
|
||||
CUSTOM_PROMPT=$(printf -v PROMPT "%s\n\nISSUE_TITLE: %s\n\nISSUE_BODY: %s\n\nINSTRUCTION: %s" "$BASE_PROMPT" "$ISSUE_TITLE" "$ISSUE_BODY" "$INSTRUCTION")
|
||||
echo "CUSTOM_PROMPT=$CUSTOM_PROMPT" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Run Claude PR Action
|
||||
uses: anthropics/claude-code-action@beta
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
timeout_minutes: "60"
|
||||
allowed_tools: "mcp__github__create_pull_request"
|
||||
direct_prompt: ${{ steps.process_inputs.outputs.CUSTOM_PROMPT }}
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
with:
|
||||
node-version: "20.x"
|
||||
registry-url: "https://registry.npmjs.org"
|
||||
- run: cd typescript-client && ./publish.sh --access public && cd ..
|
||||
- run: cd typescript-client && ./publish.sh && cd ..
|
||||
env:
|
||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
||||
publish_cli:
|
||||
@@ -25,9 +25,9 @@ jobs:
|
||||
with:
|
||||
node-version: "20.x"
|
||||
registry-url: "https://registry.npmjs.org"
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
- uses: denoland/setup-deno@v1
|
||||
with:
|
||||
bun-version: latest
|
||||
- run: cd cli && ./build.sh && cd npm && npm publish --access public
|
||||
deno-version: v1.x
|
||||
- run: cd cli && ./build.sh && cd npm && npm publish
|
||||
env:
|
||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
||||
|
||||
@@ -1,442 +0,0 @@
|
||||
name: Pi Auto Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [ready_for_review, opened, synchronize]
|
||||
workflow_call:
|
||||
inputs:
|
||||
pr_number:
|
||||
description: 'PR number to review'
|
||||
required: true
|
||||
type: number
|
||||
extra_prompt:
|
||||
description: 'Additional reviewer instructions appended to the standard review prompt'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
triggered_by:
|
||||
description: 'GitHub username that triggered this review (for audit only)'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
secrets:
|
||||
DEEPSEEK_API_KEY:
|
||||
required: false
|
||||
WINDMILL_EE_PRIVATE_ACCESS:
|
||||
required: false
|
||||
|
||||
concurrency:
|
||||
group: pi-review-${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
pi-review:
|
||||
runs-on: ubicloud-standard-2
|
||||
timeout-minutes: 30
|
||||
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
|
||||
# access to this repo, so fork==false already enforces write access. Do NOT re-add
|
||||
# an author_association gate: the pull_request webhook payload reports private org
|
||||
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
|
||||
# skips auto-review for every private member.
|
||||
if: |
|
||||
github.event_name == 'workflow_call' ||
|
||||
(
|
||||
github.event.pull_request.draft == false &&
|
||||
github.event.pull_request.head.repo.fork == false
|
||||
)
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Check Pi configuration
|
||||
id: pi_config
|
||||
env:
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
run: |
|
||||
if [ -n "$DEEPSEEK_API_KEY" ]; then
|
||||
echo "enabled=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "enabled=false" >> "$GITHUB_OUTPUT"
|
||||
echo "DEEPSEEK_API_KEY is not configured; skipping Pi review."
|
||||
fi
|
||||
|
||||
- name: Resolve PR metadata
|
||||
if: steps.pi_config.outputs.enabled == 'true'
|
||||
id: pr
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
INPUT_PR_NUMBER: ${{ inputs.pr_number }}
|
||||
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
|
||||
EVENT_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
EVENT_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
EVENT_TITLE: ${{ github.event.pull_request.title }}
|
||||
EVENT_BODY: ${{ github.event.pull_request.body }}
|
||||
EVENT_FORK: ${{ github.event.pull_request.head.repo.fork }}
|
||||
EVENT_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
EVENT_ACTION: ${{ github.event.action }}
|
||||
run: |
|
||||
if [ -n "$INPUT_PR_NUMBER" ]; then
|
||||
PR_JSON=$(gh pr view "$INPUT_PR_NUMBER" --repo "${{ github.repository }}" \
|
||||
--json number,baseRefName,baseRefOid,headRefOid,title,body,isCrossRepository,author)
|
||||
PR_NUMBER=$(echo "$PR_JSON" | jq -r '.number')
|
||||
BASE_REF=$(echo "$PR_JSON" | jq -r '.baseRefName')
|
||||
BASE_SHA=$(echo "$PR_JSON" | jq -r '.baseRefOid')
|
||||
HEAD_SHA=$(echo "$PR_JSON" | jq -r '.headRefOid')
|
||||
PR_TITLE=$(echo "$PR_JSON" | jq -r '.title')
|
||||
PR_BODY=$(echo "$PR_JSON" | jq -r '.body // ""')
|
||||
IS_FORK=$(echo "$PR_JSON" | jq -r '.isCrossRepository')
|
||||
PR_AUTHOR=$(echo "$PR_JSON" | jq -r '.author.login // ""')
|
||||
else
|
||||
PR_NUMBER="$EVENT_PR_NUMBER"
|
||||
BASE_REF="$EVENT_BASE_REF"
|
||||
BASE_SHA="$EVENT_BASE_SHA"
|
||||
HEAD_SHA="$EVENT_HEAD_SHA"
|
||||
PR_TITLE="$EVENT_TITLE"
|
||||
PR_BODY="$EVENT_BODY"
|
||||
IS_FORK="$EVENT_FORK"
|
||||
PR_AUTHOR="$EVENT_AUTHOR"
|
||||
fi
|
||||
# Fork PRs run untrusted code with secrets present, so the automatic
|
||||
# pull_request trigger never reviews them. A non-empty INPUT_PR_NUMBER
|
||||
# means we arrived via workflow_call (a maintainer /pi comment gated by
|
||||
# check-write-access), so allow forks only on that path.
|
||||
if [ "$IS_FORK" = "true" ] && [ -z "$INPUT_PR_NUMBER" ]; then
|
||||
echo "Skipping Pi review for fork PR (automatic trigger)."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
# An agent-driven PR flips to ready only after a clean /review round on
|
||||
# a draft, marked by an author comment naming the head SHA (pr skill,
|
||||
# "Review rounds"). Re-reviewing that same head on ready_for_review is
|
||||
# redundant. The marker alone is author attestation, so also require
|
||||
# reviewer evidence: a Codex review (posted by github-actions[bot], not
|
||||
# forgeable by the author) that predates the marker and carries a
|
||||
# non-blocking verdict. Comment-triggered and synchronize runs never
|
||||
# skip. Keep the three copies of this check in sync (pr-ready-review /
|
||||
# codex-pr-review / pi-pr-review); a shared local action would need the
|
||||
# repo checked out before the check, which the fork paths here
|
||||
# deliberately avoid.
|
||||
if [ "$EVENT_ACTION" = "ready_for_review" ] && [ -z "$INPUT_PR_NUMBER" ]; then
|
||||
# Fetch failures fail open (no skip): an API hiccup must run the
|
||||
# review, never skip it or fail the job.
|
||||
COMMENTS=$(gh api "repos/${{ github.repository }}/issues/$PR_NUMBER/comments?per_page=100" --paginate | jq -s '[.[][]]') || COMMENTS='[]'
|
||||
MARKER_TIME=$(jq -r --arg author "$PR_AUTHOR" --arg marker "✅ Review round clean @ $HEAD_SHA" \
|
||||
'[.[] | select(.user.login == $author) | select(.body | contains($marker)) | .created_at] | min // empty' <<<"$COMMENTS")
|
||||
CODEX_VERDICT=''
|
||||
if [ -n "$MARKER_TIME" ]; then
|
||||
# Only Codex evidence that predates the marker counts: the ready-
|
||||
# triggered Codex run itself posts after the flip and must not
|
||||
# vouch for a sibling reviewer's skip.
|
||||
CODEX_VERDICT=$(jq -r --arg mt "$MARKER_TIME" \
|
||||
'[.[] | select(.user.login == "github-actions[bot]") | select(.body | contains("## Codex Review")) | select(.created_at < $mt)] | last | .body // ""' <<<"$COMMENTS" \
|
||||
| grep -m1 -oE '(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)' || true)
|
||||
fi
|
||||
if [ -n "$MARKER_TIME" ] && [ -n "$CODEX_VERDICT" ] && [ "$CODEX_VERDICT" != "Should address issues before merging" ]; then
|
||||
echo "Clean review round marker found for $HEAD_SHA with pre-marker non-blocking Codex verdict; skipping redundant review."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
# PR title/body are attacker-controlled free text. Use an unguessable
|
||||
# per-run delimiter so a fork can't embed a fixed heredoc terminator to
|
||||
# inject extra outputs — e.g. is_fork=false (last-write-wins), which
|
||||
# would re-enable the EE checkout and trusted-path settings for forks.
|
||||
RAND=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')
|
||||
TITLE_EOF="TITLE_EOF_${RAND}"
|
||||
BODY_EOF="BODY_EOF_${RAND}"
|
||||
{
|
||||
echo "skip=false"
|
||||
echo "is_fork=$IS_FORK"
|
||||
echo "pr_number=$PR_NUMBER"
|
||||
echo "base_ref=$BASE_REF"
|
||||
echo "base_sha=$BASE_SHA"
|
||||
echo "head_sha=$HEAD_SHA"
|
||||
echo "pr_author=$PR_AUTHOR"
|
||||
echo "title<<$TITLE_EOF"
|
||||
printf '%s\n' "$PR_TITLE"
|
||||
echo "$TITLE_EOF"
|
||||
echo "body<<$BODY_EOF"
|
||||
printf '%s\n' "$PR_BODY"
|
||||
echo "$BODY_EOF"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Checkout repository
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
ref: refs/pull/${{ steps.pr.outputs.pr_number }}/merge
|
||||
fetch-depth: 1
|
||||
# Don't persist github.token in .git/config: the review agent can read
|
||||
# the checkout, and on the fork path that token (issue/PR write) would
|
||||
# otherwise be exfiltratable. All later git ops target the public origin
|
||||
# and need no auth; EE checkout and gh use their own explicit tokens.
|
||||
persist-credentials: false
|
||||
|
||||
# Never expose the EE private-repo token to untrusted fork code. Skipping
|
||||
# this step leaves steps.ee.outputs.available empty, so the EE checkout and
|
||||
# substitution steps below are skipped too.
|
||||
- name: Check EE access
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true' && steps.pr.outputs.is_fork != 'true'
|
||||
id: ee
|
||||
env:
|
||||
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
run: |
|
||||
if [ -n "$EE_TOKEN" ]; then
|
||||
echo "available=true" >> "$GITHUB_OUTPUT"
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "available=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Checkout EE repository
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ steps.ee.outputs.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Substitute EE code
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Set up Node.js
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Install Pi CLI
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
run: npm install --global @mariozechner/pi-coding-agent
|
||||
|
||||
- name: Pre-fetch base and head refs for the PR
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
run: |
|
||||
git fetch --no-tags origin \
|
||||
"$PR_BASE_REF" \
|
||||
"+refs/pull/$PR_NUMBER/head"
|
||||
|
||||
- name: Fetch prior PR discussion
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
run: |
|
||||
# Write outside the checkout: on the fork path the merge tree is
|
||||
# attacker-controlled, and a committed symlink at this path would
|
||||
# redirect the write.
|
||||
gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" \
|
||||
--jq '[.[] | {user: .user.login, created_at: .created_at, body: (.body | .[:4000])}] | sort_by(.created_at) | .[-20:]' \
|
||||
> "$RUNNER_TEMP/prior-comments.json" || echo "[]" > "$RUNNER_TEMP/prior-comments.json"
|
||||
|
||||
- name: Write Pi review context
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
PR_REPOSITORY: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
|
||||
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
|
||||
PR_TITLE: ${{ steps.pr.outputs.title }}
|
||||
PR_BODY: ${{ steps.pr.outputs.body }}
|
||||
PR_AUTHOR: ${{ steps.pr.outputs.pr_author }}
|
||||
EXTRA_PROMPT: ${{ inputs.extra_prompt }}
|
||||
run: |
|
||||
node <<'NODE'
|
||||
const fs = require('fs');
|
||||
const tmp = process.env.RUNNER_TEMP;
|
||||
const lines = [
|
||||
`Repository: ${process.env.PR_REPOSITORY}`,
|
||||
`PR number: ${process.env.PR_NUMBER}`,
|
||||
];
|
||||
if (process.env.PR_AUTHOR) {
|
||||
lines.push(`PR AUTHOR: ${process.env.PR_AUTHOR}`);
|
||||
}
|
||||
lines.push(
|
||||
`Base SHA: ${process.env.PR_BASE_SHA}`,
|
||||
`Head SHA: ${process.env.PR_HEAD_SHA}`,
|
||||
'',
|
||||
'PR title:',
|
||||
process.env.PR_TITLE || '(empty)',
|
||||
'',
|
||||
'PR body:',
|
||||
process.env.PR_BODY || '(empty)',
|
||||
'',
|
||||
'Changed commits command:',
|
||||
`git log --oneline ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
|
||||
'',
|
||||
'Changed files command:',
|
||||
`git diff --stat ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
|
||||
'',
|
||||
'Full review diff command:',
|
||||
`git diff --unified=0 ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`
|
||||
);
|
||||
if (process.env.EXTRA_PROMPT && process.env.EXTRA_PROMPT.trim()) {
|
||||
lines.push('', 'Additional reviewer instructions:', process.env.EXTRA_PROMPT.trim());
|
||||
}
|
||||
if (fs.existsSync(`${tmp}/prior-comments.json`)) {
|
||||
try {
|
||||
const comments = JSON.parse(fs.readFileSync(`${tmp}/prior-comments.json`, 'utf8'));
|
||||
if (Array.isArray(comments) && comments.length > 0) {
|
||||
lines.push(
|
||||
'',
|
||||
'Prior PR discussion (most recent up to 20 comments):',
|
||||
'',
|
||||
'If you have already reviewed this PR (look for your own earlier "## Pi Review (DeepSeek V4)" comment), focus on what changed since then per the diff and respect any decisions the human made in replies. Do not re-flag findings the human already pushed back on.',
|
||||
''
|
||||
);
|
||||
for (const c of comments) {
|
||||
lines.push(`### @${c.user} (${c.created_at})`, '', c.body, '', '---', '');
|
||||
}
|
||||
}
|
||||
} catch (_) {}
|
||||
}
|
||||
fs.writeFileSync(`${tmp}/pr-review-context.md`, `${lines.join('\n')}\n`);
|
||||
NODE
|
||||
|
||||
- name: Run Pi review
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
env:
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
PI_SKIP_VERSION_CHECK: '1'
|
||||
PR_IS_FORK: ${{ steps.pr.outputs.is_fork }}
|
||||
PR_BASE_REF: ${{ steps.pr.outputs.base_ref }}
|
||||
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
|
||||
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
|
||||
run: |
|
||||
set -o pipefail
|
||||
PI_HARDEN_FLAGS=()
|
||||
# Keep generated files (final message, events, context) outside the
|
||||
# checkout: on the fork path a committed symlink at any of these paths
|
||||
# would redirect our write and could overwrite an action's code that
|
||||
# then runs with our credentials. RUNNER_TEMP is outside the checkout.
|
||||
OUT_DIR="$RUNNER_TEMP"
|
||||
CTX="$RUNNER_TEMP/pr-review-context.md"
|
||||
if [ "$PR_IS_FORK" = "true" ]; then
|
||||
# Fork code is untrusted. Read the review policy/prompt from the base
|
||||
# ref (git show) rather than the attacker-controlled merge checkout,
|
||||
# so a malicious fork can't rewrite the reviewer's own instructions,
|
||||
# and drop the bash tool so the agent has no shell to exfiltrate with.
|
||||
git show "origin/$PR_BASE_REF:REVIEW.md" > /tmp/pi-prompt.md
|
||||
git show "origin/$PR_BASE_REF:.github/pi/pr-review.prompt.md" >> /tmp/pi-prompt.md
|
||||
PI_TOOLS=read,grep,find,ls
|
||||
|
||||
# The agent has no shell, so pre-compute the diff (base...head SHAs are
|
||||
# trusted) into the context file it reads. It may still read fork files
|
||||
# by absolute path for extra context — reads are safe.
|
||||
{
|
||||
echo ""
|
||||
echo "## Pre-computed review diff (base...head)"
|
||||
echo "You have no shell. The full diff is below. The repository checkout"
|
||||
echo "is at $GITHUB_WORKSPACE — you may read files there by absolute path."
|
||||
echo '```diff'
|
||||
git -C "$GITHUB_WORKSPACE" diff --unified=0 "$PR_BASE_SHA...$PR_HEAD_SHA"
|
||||
echo '```'
|
||||
} >> "$CTX"
|
||||
|
||||
# Pi resolves ALL project config from <cwd>/.pi (settings/packages,
|
||||
# extensions, skills, themes, prompts, SYSTEM.md); inside the fork
|
||||
# checkout a fork could inject any to run code or rewrite our system
|
||||
# prompt. Discovery is cwd-based, so run from a fresh empty dir.
|
||||
PI_WORKDIR=$(mktemp -d)
|
||||
cd "$PI_WORKDIR"
|
||||
|
||||
# Belt-and-suspenders on top of the isolated cwd: refuse discovery of
|
||||
# extensions/skills/templates/themes/context-files, and PI_OFFLINE=1 to
|
||||
# block any startup network op or package install. PI_OFFLINE gates only
|
||||
# startup network ops, not the provider inference call.
|
||||
PI_HARDEN_FLAGS=(--no-extensions --no-skills --no-prompt-templates --no-themes --no-context-files)
|
||||
export PI_OFFLINE=1
|
||||
else
|
||||
cat REVIEW.md .github/pi/pr-review.prompt.md > /tmp/pi-prompt.md
|
||||
PI_TOOLS=read,grep,find,ls,bash
|
||||
fi
|
||||
# The context file lives in RUNNER_TEMP (outside the checkout); tell the
|
||||
# agent its absolute path.
|
||||
printf '\nReview context file (absolute path): %s\n' "$CTX" >> /tmp/pi-prompt.md
|
||||
pi -p \
|
||||
--provider deepseek \
|
||||
--model deepseek-v4-pro \
|
||||
--tools "$PI_TOOLS" \
|
||||
"${PI_HARDEN_FLAGS[@]}" \
|
||||
--mode json \
|
||||
< /tmp/pi-prompt.md \
|
||||
| tee "$OUT_DIR/pi-events.jsonl" \
|
||||
| jq -rc --unbuffered '
|
||||
if .type == "agent_start" then "🤖 pi agent started"
|
||||
elif .type == "turn_start" then "── turn ──"
|
||||
elif .type == "message_end" then
|
||||
"[\(.message.role)] " + (
|
||||
(.message.content // [])
|
||||
| map(
|
||||
if .type == "text" then "text(\(.text | length)c)"
|
||||
elif .type == "tool_use" then "🔧 \(.name) \(.input | @json | .[:160])"
|
||||
elif .type == "tool_result" then "✅ result"
|
||||
else .type
|
||||
end
|
||||
)
|
||||
| join(" | ")
|
||||
)
|
||||
elif .type == "turn_end" then "── turn done (\((.toolResults // []) | length) tool result(s)) ──"
|
||||
elif .type == "agent_end" then "🏁 pi agent done"
|
||||
else empty
|
||||
end
|
||||
'
|
||||
|
||||
jq -r '
|
||||
select(.type == "agent_end")
|
||||
| .messages
|
||||
| map(select(.role == "assistant"))
|
||||
| last
|
||||
| (.content[]? | select(.type == "text") | .text)
|
||||
' "$OUT_DIR/pi-events.jsonl" > "$OUT_DIR/pi-final-message.md"
|
||||
|
||||
- name: Post Pi review comment
|
||||
if: steps.pi_config.outputs.enabled == 'true' && steps.pr.outputs.skip != 'true'
|
||||
uses: actions/github-script@v7
|
||||
env:
|
||||
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
GH_JOB_TOKEN: ${{ github.token }}
|
||||
with:
|
||||
github-token: ${{ github.token }}
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
const path = `${process.env.RUNNER_TEMP}/pi-final-message.md`;
|
||||
if (!fs.existsSync(path)) {
|
||||
core.info('Pi did not produce a final message; skipping PR comment.');
|
||||
return;
|
||||
}
|
||||
let body = fs.readFileSync(path, 'utf8').trim();
|
||||
if (!body) {
|
||||
core.info('Pi final message was empty; skipping PR comment.');
|
||||
return;
|
||||
}
|
||||
// Defense-in-depth for fork reviews: the model call needs the provider
|
||||
// credential in the environment (readable via /proc/self/environ), and
|
||||
// the posted comment is an exfiltration channel that bypasses GitHub
|
||||
// Actions log masking. Strip the credential if it leaked into the text.
|
||||
for (const s of [process.env.DEEPSEEK_API_KEY, process.env.GH_JOB_TOKEN]) {
|
||||
if (typeof s === 'string' && s.length >= 8) {
|
||||
body = body.split(s).join('[REDACTED]');
|
||||
}
|
||||
}
|
||||
body = body.trim();
|
||||
if (!body) {
|
||||
core.info('Pi final message was empty after redaction; skipping PR comment.');
|
||||
return;
|
||||
}
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: Number(process.env.PR_NUMBER),
|
||||
body,
|
||||
});
|
||||
@@ -1,202 +0,0 @@
|
||||
name: Claude Auto Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [ready_for_review, opened]
|
||||
workflow_call:
|
||||
inputs:
|
||||
pr_number:
|
||||
description: 'PR number to review'
|
||||
required: true
|
||||
type: number
|
||||
extra_prompt:
|
||||
description: 'Additional reviewer instructions appended to the standard review prompt'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
triggered_by:
|
||||
description: 'GitHub username that triggered this review (for audit only)'
|
||||
required: false
|
||||
type: string
|
||||
default: ''
|
||||
secrets:
|
||||
CLAUDE_CODE_OAUTH_TOKEN:
|
||||
required: true
|
||||
WINDMILL_EE_PRIVATE_ACCESS:
|
||||
required: false
|
||||
|
||||
concurrency:
|
||||
group: claude-review-${{ inputs.pr_number || github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
auto-review:
|
||||
runs-on: ubuntu-latest
|
||||
# A non-fork PR (head.repo.fork == false) can only be opened by someone with push
|
||||
# access to this repo, so fork==false already enforces write access. Do NOT re-add
|
||||
# an author_association gate: the pull_request webhook payload reports private org
|
||||
# members as CONTRIBUTOR/NONE (only public members show as MEMBER), which silently
|
||||
# skips auto-review for every private member.
|
||||
if: |
|
||||
github.event_name == 'workflow_call' ||
|
||||
(
|
||||
(github.event.pull_request.draft == false || github.event.pull_request.ready_for_review == true) &&
|
||||
github.event.pull_request.head.repo.fork == false
|
||||
)
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
id-token: write
|
||||
steps:
|
||||
# An agent-driven PR flips to ready only after a clean /review round on a
|
||||
# draft, marked by an author comment naming the head SHA (pr skill, "Review
|
||||
# rounds"). Re-reviewing that same head on ready_for_review is redundant.
|
||||
# The marker alone is author attestation, so also require reviewer evidence:
|
||||
# a Codex review (posted by github-actions[bot], not forgeable by the author)
|
||||
# that predates the marker and carries a non-blocking verdict. Comment-
|
||||
# triggered (workflow_call) and opened runs never skip. Keep the three
|
||||
# copies of this check in sync (pr-ready-review / codex-pr-review /
|
||||
# pi-pr-review); a shared local action would need the repo checked out
|
||||
# before the check, which the codex/pi fork paths deliberately avoid.
|
||||
- name: Check clean-round marker
|
||||
id: marker
|
||||
if: github.event_name == 'pull_request' && github.event.action == 'ready_for_review'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
run: |
|
||||
# Fetch failures fail open (skip=false): an API hiccup must run the
|
||||
# review, never skip it or fail the job.
|
||||
COMMENTS=$(gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" --paginate | jq -s '[.[][]]') || COMMENTS='[]'
|
||||
MARKER_TIME=$(jq -r --arg author "$PR_AUTHOR" --arg marker "✅ Review round clean @ $HEAD_SHA" \
|
||||
'[.[] | select(.user.login == $author) | select(.body | contains($marker)) | .created_at] | min // empty' <<<"$COMMENTS")
|
||||
CODEX_VERDICT=''
|
||||
if [ -n "$MARKER_TIME" ]; then
|
||||
# Only Codex evidence that predates the marker counts: the ready-
|
||||
# triggered Codex run itself posts after the flip and must not vouch
|
||||
# for a sibling reviewer's skip.
|
||||
CODEX_VERDICT=$(jq -r --arg mt "$MARKER_TIME" \
|
||||
'[.[] | select(.user.login == "github-actions[bot]") | select(.body | contains("## Codex Review")) | select(.created_at < $mt)] | last | .body // ""' <<<"$COMMENTS" \
|
||||
| grep -m1 -oE '(Good to merge|Mergeable, but should ideally address nits|Should address issues before merging)' || true)
|
||||
fi
|
||||
if [ -n "$MARKER_TIME" ] && [ -n "$CODEX_VERDICT" ] && [ "$CODEX_VERDICT" != "Should address issues before merging" ]; then
|
||||
echo "Clean review round marker found for $HEAD_SHA with pre-marker non-blocking Codex verdict; skipping redundant review."
|
||||
echo "skip=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "skip=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Checkout repository
|
||||
if: steps.marker.outputs.skip != 'true'
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Check EE access
|
||||
if: steps.marker.outputs.skip != 'true'
|
||||
id: ee
|
||||
env:
|
||||
EE_TOKEN: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
run: |
|
||||
if [ -n "$EE_TOKEN" ]; then
|
||||
echo "available=true" >> "$GITHUB_OUTPUT"
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "available=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Checkout EE repository
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ steps.ee.outputs.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Substitute EE code
|
||||
if: steps.ee.outputs.available == 'true'
|
||||
run: ./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Resolve PR number
|
||||
if: steps.marker.outputs.skip != 'true'
|
||||
id: resolve
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
INPUT_PR_NUMBER: ${{ inputs.pr_number }}
|
||||
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
EVENT_PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
run: |
|
||||
if [ -n "$INPUT_PR_NUMBER" ]; then
|
||||
PR_NUMBER="$INPUT_PR_NUMBER"
|
||||
PR_AUTHOR=$(gh api "repos/$REPO/pulls/$PR_NUMBER" --jq '.user.login')
|
||||
else
|
||||
PR_NUMBER="$EVENT_PR_NUMBER"
|
||||
PR_AUTHOR="$EVENT_PR_AUTHOR"
|
||||
fi
|
||||
echo "pr_number=$PR_NUMBER" >> "$GITHUB_OUTPUT"
|
||||
echo "pr_author=$PR_AUTHOR" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Fetch prior PR discussion
|
||||
if: steps.marker.outputs.skip != 'true'
|
||||
id: prior
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ steps.resolve.outputs.pr_number }}
|
||||
run: |
|
||||
gh api "repos/$REPO/issues/$PR_NUMBER/comments?per_page=100" \
|
||||
--jq '[.[] | {user: .user.login, created_at: .created_at, body: (.body | .[:4000])}] | sort_by(.created_at) | .[-20:]' \
|
||||
> prior-comments.json || echo "[]" > prior-comments.json
|
||||
jq -r '
|
||||
if length == 0 then ""
|
||||
else
|
||||
"## Prior PR discussion (most recent up to 20 comments)\n\nIf you have already reviewed this PR (look for your own earlier comment), focus on what changed since then per the diff and respect any decisions the human made in replies. Do not re-flag findings the human already pushed back on.\n\n" +
|
||||
(map("### @\(.user) (\(.created_at))\n\n\(.body)") | join("\n\n---\n\n"))
|
||||
end
|
||||
' prior-comments.json > prior-comments.md
|
||||
|
||||
- name: Read review prompt
|
||||
if: steps.marker.outputs.skip != 'true'
|
||||
id: review-prompt
|
||||
env:
|
||||
EXTRA_PROMPT: ${{ inputs.extra_prompt }}
|
||||
run: |
|
||||
{
|
||||
echo 'REVIEW_PROMPT<<EOF'
|
||||
cat REVIEW.md
|
||||
echo ''
|
||||
cat .claude/review-prompt.md
|
||||
if [ -n "$EXTRA_PROMPT" ]; then
|
||||
echo ''
|
||||
echo '## Additional reviewer instructions'
|
||||
echo ''
|
||||
printf '%s\n' "$EXTRA_PROMPT"
|
||||
fi
|
||||
if [ -s prior-comments.md ]; then
|
||||
echo ''
|
||||
cat prior-comments.md
|
||||
fi
|
||||
echo 'EOF'
|
||||
} >> "$GITHUB_ENV"
|
||||
|
||||
- name: Automatic PR Review
|
||||
if: steps.marker.outputs.skip != 'true'
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
track_progress: true
|
||||
prompt: |
|
||||
REPO: ${{ github.repository }}
|
||||
PR NUMBER: ${{ steps.resolve.outputs.pr_number }}
|
||||
PR AUTHOR: ${{ steps.resolve.outputs.pr_author }}
|
||||
|
||||
${{ env.REVIEW_PROMPT }}
|
||||
claude_args: |
|
||||
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
|
||||
--model claude-opus-4-8
|
||||
@@ -1,310 +0,0 @@
|
||||
name: PR Review Commands
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
parse:
|
||||
if: github.event.issue.pull_request != null
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
command: ${{ steps.parse.outputs.command }}
|
||||
extra_prompt: ${{ steps.parse.outputs.extra_prompt }}
|
||||
steps:
|
||||
- name: Parse command from comment
|
||||
id: parse
|
||||
env:
|
||||
BODY: ${{ github.event.comment.body }}
|
||||
run: |
|
||||
FIRST_LINE=$(printf '%s' "$BODY" | head -n 1 | sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//')
|
||||
FIRST_WORD=${FIRST_LINE%% *}
|
||||
case "$FIRST_WORD" in
|
||||
/review|/codex|/pi|/claude)
|
||||
COMMAND="${FIRST_WORD#/}"
|
||||
REMAINDER_FIRST_LINE=${FIRST_LINE#"$FIRST_WORD"}
|
||||
REMAINDER_FIRST_LINE=${REMAINDER_FIRST_LINE# }
|
||||
REST=$(printf '%s' "$BODY" | tail -n +2)
|
||||
{
|
||||
echo "command=$COMMAND"
|
||||
echo 'extra_prompt<<EXTRA_EOF'
|
||||
if [ -n "$REMAINDER_FIRST_LINE" ]; then
|
||||
printf '%s\n' "$REMAINDER_FIRST_LINE"
|
||||
fi
|
||||
if [ -n "$REST" ]; then
|
||||
printf '%s\n' "$REST"
|
||||
fi
|
||||
echo 'EXTRA_EOF'
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
;;
|
||||
*)
|
||||
echo "command=" >> "$GITHUB_OUTPUT"
|
||||
;;
|
||||
esac
|
||||
|
||||
# author_association misses private org members; check-access resolves them via the
|
||||
# internal app token. Both are OR'd so public members still pass instantly.
|
||||
check-access:
|
||||
needs: [parse]
|
||||
if: needs.parse.outputs.command != ''
|
||||
uses: ./.github/workflows/check-write-access.yml
|
||||
with:
|
||||
username: ${{ github.event.comment.user.login }}
|
||||
secrets: inherit
|
||||
|
||||
acknowledge:
|
||||
needs: [parse, check-access]
|
||||
if: |
|
||||
needs.parse.outputs.command != '' &&
|
||||
(
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
|
||||
needs.check-access.outputs.authorized == 'true'
|
||||
)
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: React to comment with eyes
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
COMMENT_ID: ${{ github.event.comment.id }}
|
||||
run: |
|
||||
gh api -X POST \
|
||||
"/repos/$REPO/issues/comments/$COMMENT_ID/reactions" \
|
||||
-f content=eyes >/dev/null
|
||||
|
||||
# Decide, per agent, whether to launch a fresh run, re-run in place, or skip. A push
|
||||
# already auto-triggers codex/pi (and claude on open) against the PR head. Relaunching
|
||||
# via this issue_comment path both cancels those in-flight auto runs (shared concurrency
|
||||
# group) AND lands the new run's status on main — issue_comment runs never attach a
|
||||
# check to the PR head — leaving the PR showing only a cancelled review. So for every
|
||||
# command, launch an agent only when nothing covers the head commit; if the head's run
|
||||
# was cancelled/failed, re-run it in place (a re-run keeps the original pull_request
|
||||
# event, so its checks re-attach to the PR head); skip when a running or successful run
|
||||
# already covers it. `/review` applies this to all three agents; `/codex`, `/pi`,
|
||||
# `/claude` apply the same decision to just their own agent.
|
||||
plan:
|
||||
needs: [parse, check-access]
|
||||
if: |
|
||||
needs.parse.outputs.command != '' &&
|
||||
(
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
|
||||
needs.check-access.outputs.authorized == 'true'
|
||||
)
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
actions: write
|
||||
pull-requests: read
|
||||
statuses: write
|
||||
outputs:
|
||||
head_sha: ${{ steps.plan.outputs.head_sha }}
|
||||
launch_codex: ${{ steps.plan.outputs.launch_codex }}
|
||||
launch_pi: ${{ steps.plan.outputs.launch_pi }}
|
||||
launch_claude: ${{ steps.plan.outputs.launch_claude }}
|
||||
steps:
|
||||
- name: Decide per-agent launch vs re-run for the head commit
|
||||
id: plan
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ github.event.issue.number }}
|
||||
COMMAND: ${{ needs.parse.outputs.command }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
HEAD_SHA=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json headRefOid --jq '.headRefOid')
|
||||
echo "PR #$PR_NUMBER head: $HEAD_SHA"
|
||||
echo "head_sha=$HEAD_SHA" >> "$GITHUB_OUTPUT"
|
||||
|
||||
RUN_URL="$GITHUB_SERVER_URL/$REPO/actions/runs/$GITHUB_RUN_ID"
|
||||
|
||||
# A fresh launch runs from this issue_comment workflow (associated with main),
|
||||
# so it never appears in the PR-head run query below and its own check lands on
|
||||
# main, not the head. To keep fresh launches idempotent per head, mark the head
|
||||
# SHA with a `review-launch/<agent>` commit status at launch; the `finalize` job
|
||||
# resolves it to success/failure. A prior launch's status covering the head lets
|
||||
# a second comment skip instead of relaunching (which would cancel the first via
|
||||
# the reviewer's shared concurrency group). All status calls are best-effort — a
|
||||
# GitHub API hiccup must degrade to a relaunch, never abort the decision.
|
||||
mark_launch() {
|
||||
agent="$1"
|
||||
gh api -X POST "repos/$REPO/statuses/$HEAD_SHA" \
|
||||
-f state=pending -f "context=review-launch/$agent" -f "target_url=$RUN_URL" \
|
||||
-f "description=Review launched via /$COMMAND" >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
# Returns "covered" if a prior fresh launch (this or an earlier comment run)
|
||||
# already covers the head: a success status, or a pending status whose launching
|
||||
# run is still alive. A pending whose run has completed is stale (that run
|
||||
# crashed before finalize) and does not count.
|
||||
launch_coverage() {
|
||||
agent="$1"
|
||||
st_json=$(gh api "repos/$REPO/commits/$HEAD_SHA/statuses" \
|
||||
--jq "[.[] | select(.context == \"review-launch/$agent\")] | first // empty" 2>/dev/null || true)
|
||||
[ -n "$st_json" ] || return 0
|
||||
state=$(jq -r '.state // empty' <<<"$st_json" 2>/dev/null || true)
|
||||
[ "$state" = success ] && { echo covered; return 0; }
|
||||
[ "$state" = pending ] || return 0
|
||||
target=$(jq -r '.target_url // empty' <<<"$st_json" 2>/dev/null || true)
|
||||
run_id=$(printf '%s' "$target" | grep -oE '[0-9]+$' || true)
|
||||
if [ -n "$run_id" ]; then
|
||||
run_state=$(gh run view "$run_id" --repo "$REPO" --json status --jq '.status' 2>/dev/null || true)
|
||||
[ "$run_state" = completed ] && return 0 # stale pending -> not covered
|
||||
fi
|
||||
echo covered
|
||||
}
|
||||
|
||||
decide() {
|
||||
wf="$1"; key="$2"; agent="$3"
|
||||
if [ "$(launch_coverage "$agent")" = covered ]; then
|
||||
echo "$key: a prior launch already covers $HEAD_SHA (review-launch/$agent) -> skip"
|
||||
echo "$key=false" >> "$GITHUB_OUTPUT"
|
||||
return
|
||||
fi
|
||||
# `--commit` matches runs whose head SHA is the PR head. Auto reviews run on
|
||||
# `pull_request` against that SHA; `/review` (issue_comment) runs execute on
|
||||
# main, so they never match and are not counted as covering the head commit.
|
||||
runs=$(gh run list --repo "$REPO" --workflow "$wf" --commit "$HEAD_SHA" --limit 40 \
|
||||
--json databaseId,status,conclusion)
|
||||
# Healthy = still running, or completed successfully: a review already
|
||||
# covers this commit, so skip.
|
||||
healthy=$(jq -r '[.[] | select(.status != "completed" or .conclusion == "success")] | length' <<<"$runs")
|
||||
if [ "$healthy" -gt 0 ]; then
|
||||
echo "$key: a running or successful review already covers $HEAD_SHA -> skip"
|
||||
echo "$key=false" >> "$GITHUB_OUTPUT"
|
||||
return
|
||||
fi
|
||||
# Re-run only genuinely interrupted runs (cancelled/failed/timed out) in
|
||||
# place, so their checks re-attach to the PR head instead of posting on
|
||||
# main. A `skipped` run produced no review and would just skip again (it is
|
||||
# the draft/fork gate), so it does not count — fall through to a fresh launch.
|
||||
retry_id=$(jq -r '[.[] | select(.status == "completed" and (.conclusion == "cancelled" or .conclusion == "failure" or .conclusion == "timed_out"))] | sort_by(.databaseId) | last | .databaseId // empty' <<<"$runs")
|
||||
if [ -n "$retry_id" ]; then
|
||||
if gh run rerun "$retry_id" --repo "$REPO" >/dev/null 2>&1; then
|
||||
echo "$key: re-ran interrupted run $retry_id (re-attaches to PR head)"
|
||||
echo "$key=false" >> "$GITHUB_OUTPUT"
|
||||
return
|
||||
fi
|
||||
echo "$key: re-run of $retry_id failed -> fresh launch"
|
||||
mark_launch "$agent"
|
||||
echo "$key=true" >> "$GITHUB_OUTPUT"
|
||||
return
|
||||
fi
|
||||
echo "$key: no usable review for $HEAD_SHA -> launch"
|
||||
mark_launch "$agent"
|
||||
echo "$key=true" >> "$GITHUB_OUTPUT"
|
||||
}
|
||||
|
||||
# `/review` targets all three agents; `/codex`, `/pi`, `/claude` target only
|
||||
# their own. A non-targeted agent is left untouched (no launch, no re-run).
|
||||
decide_if_targeted() {
|
||||
wf="$1"; key="$2"; agent="$3"
|
||||
if [ "$COMMAND" = review ] || [ "$COMMAND" = "$agent" ]; then
|
||||
decide "$wf" "$key" "$agent"
|
||||
else
|
||||
echo "$key: /$COMMAND does not target $agent -> skip"
|
||||
echo "$key=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
}
|
||||
|
||||
decide_if_targeted codex-pr-review.yml launch_codex codex
|
||||
decide_if_targeted pi-pr-review.yml launch_pi pi
|
||||
decide_if_targeted pr-ready-review.yml launch_claude claude
|
||||
|
||||
claude:
|
||||
needs: [parse, check-access, plan]
|
||||
if: |
|
||||
(
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
|
||||
needs.check-access.outputs.authorized == 'true'
|
||||
) &&
|
||||
needs.plan.outputs.launch_claude == 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
id-token: write
|
||||
uses: ./.github/workflows/pr-ready-review.yml
|
||||
with:
|
||||
pr_number: ${{ github.event.issue.number }}
|
||||
extra_prompt: ${{ needs.parse.outputs.extra_prompt }}
|
||||
triggered_by: ${{ github.event.comment.user.login }}
|
||||
secrets:
|
||||
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
codex:
|
||||
needs: [parse, check-access, plan]
|
||||
if: |
|
||||
(
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
|
||||
needs.check-access.outputs.authorized == 'true'
|
||||
) &&
|
||||
needs.plan.outputs.launch_codex == 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
uses: ./.github/workflows/codex-pr-review.yml
|
||||
with:
|
||||
pr_number: ${{ github.event.issue.number }}
|
||||
extra_prompt: ${{ needs.parse.outputs.extra_prompt }}
|
||||
triggered_by: ${{ github.event.comment.user.login }}
|
||||
secrets:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
|
||||
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
pi:
|
||||
needs: [parse, check-access, plan]
|
||||
if: |
|
||||
(
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) ||
|
||||
needs.check-access.outputs.authorized == 'true'
|
||||
) &&
|
||||
needs.plan.outputs.launch_pi == 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
uses: ./.github/workflows/pi-pr-review.yml
|
||||
with:
|
||||
pr_number: ${{ github.event.issue.number }}
|
||||
extra_prompt: ${{ needs.parse.outputs.extra_prompt }}
|
||||
triggered_by: ${{ github.event.comment.user.login }}
|
||||
secrets:
|
||||
DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }}
|
||||
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
# Resolve the `review-launch/<agent>` head statuses that `plan` set to pending, so a
|
||||
# fresh launch's outcome is visible on the PR head (not just on main) and never lingers
|
||||
# as a stale pending check. Targets the exact SHA `plan` launched against, so a push
|
||||
# that moved the head mid-review does not stamp a status on the new head.
|
||||
finalize:
|
||||
needs: [plan, claude, codex, pi]
|
||||
if: always() && needs.plan.result == 'success' && needs.plan.outputs.head_sha != ''
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
statuses: write
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
HEAD_SHA: ${{ needs.plan.outputs.head_sha }}
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
steps:
|
||||
- name: Finalize launch statuses on the PR head
|
||||
run: |
|
||||
set -uo pipefail
|
||||
finalize() {
|
||||
agent="$1"; launched="$2"; result="$3"
|
||||
[ "$launched" = true ] || return 0
|
||||
state=$([ "$result" = success ] && echo success || echo failure)
|
||||
gh api -X POST "repos/$REPO/statuses/$HEAD_SHA" \
|
||||
-f "state=$state" -f "context=review-launch/$agent" -f "target_url=$RUN_URL" \
|
||||
-f "description=Review $result" >/dev/null 2>&1 || true
|
||||
}
|
||||
finalize codex "${{ needs.plan.outputs.launch_codex }}" "${{ needs.codex.result }}"
|
||||
finalize pi "${{ needs.plan.outputs.launch_pi }}" "${{ needs.pi.result }}"
|
||||
finalize claude "${{ needs.plan.outputs.launch_claude }}" "${{ needs.claude.result }}"
|
||||
@@ -1,84 +0,0 @@
|
||||
name: Publish CLI docs repo
|
||||
|
||||
# Regenerates the windmill-cli-docs repo (consumed by context7) from the
|
||||
# canonical sources in this repo on every Windmill release.
|
||||
#
|
||||
# Required secret:
|
||||
# CLI_DOCS_DEPLOY_KEY — ed25519 private key whose public half is registered
|
||||
# as a write-access deploy key on
|
||||
# windmill-labs/windmill-cli-docs.
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
|
||||
# Serialize pushes to windmill-cli-docs so two release tags landing close
|
||||
# together (e.g. a release-please bump + a hotfix) can't race to force-push
|
||||
# the docs repo.
|
||||
concurrency:
|
||||
group: publish-cli-docs
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout windmill (source of truth)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
path: windmill
|
||||
|
||||
- name: Checkout windmill-cli-docs (publish target)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-cli-docs
|
||||
path: windmill-cli-docs
|
||||
ssh-key: ${{ secrets.CLI_DOCS_DEPLOY_KEY }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install dependencies
|
||||
run: pip install pyyaml
|
||||
|
||||
- name: Regenerate docs
|
||||
run: |
|
||||
python3 windmill/system_prompts/generate.py \
|
||||
--context7-dir "$GITHUB_WORKSPACE/windmill-cli-docs"
|
||||
|
||||
- name: Commit and push if changed
|
||||
working-directory: windmill-cli-docs
|
||||
env:
|
||||
REF_NAME: ${{ github.ref_name }}
|
||||
REF_TYPE: ${{ github.ref_type }}
|
||||
run: |
|
||||
git config user.name "windmill-bot"
|
||||
git config user.email "bot@windmill.dev"
|
||||
git add -A
|
||||
if git diff --cached --quiet; then
|
||||
echo "No doc changes for ${REF_NAME}."
|
||||
committed=false
|
||||
else
|
||||
committed=true
|
||||
if [ "${REF_TYPE}" = "tag" ]; then
|
||||
git commit -m "chore: sync from windmill ${REF_NAME}"
|
||||
else
|
||||
git commit -m "chore: sync from windmill (manual dispatch from ${REF_NAME})"
|
||||
fi
|
||||
git push origin HEAD
|
||||
fi
|
||||
# Always mirror the version tag on tag pushes, even when content
|
||||
# didn't change — downstream consumers tie snapshots to releases by
|
||||
# tag, and skipping it would leave the docs repo without a tag for
|
||||
# the new Windmill release.
|
||||
# workflow_dispatch from a non-tag ref skips this so we don't
|
||||
# create a junk tag named after a branch.
|
||||
if [ "${REF_TYPE}" = "tag" ]; then
|
||||
git tag -f "${REF_NAME}"
|
||||
git push origin "${REF_NAME}" --force
|
||||
echo "Mirrored tag ${REF_NAME} to windmill-cli-docs (content changed: ${committed})."
|
||||
fi
|
||||
@@ -1,126 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
ECR_REGISTRY: 976079455550.dkr.ecr.us-east-1.amazonaws.com
|
||||
IMAGE_NAME: ${{ github.repository }}-extra
|
||||
|
||||
name: Publish windmill-extra
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
sleep:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- name: Sleep for 900 seconds waiting for pypi to update index
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
run: sleep 900
|
||||
shell: bash
|
||||
|
||||
# Build and test the image before publishing
|
||||
test_extra:
|
||||
needs: [sleep]
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Build test image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/DockerfileExtra
|
||||
load: true
|
||||
tags: windmill-extra:test
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Start container
|
||||
run: |
|
||||
docker run -d --name windmill-extra-test \
|
||||
-p 3001:3001 -p 3002:3002 -p 3003:3003 \
|
||||
-e ENABLE_LSP=true \
|
||||
-e ENABLE_MULTIPLAYER=true \
|
||||
-e ENABLE_DEBUGGER=true \
|
||||
-e DEBUGGER_PORT=3003 \
|
||||
-e REQUIRE_SIGNED_DEBUG_REQUESTS=false \
|
||||
windmill-extra:test
|
||||
|
||||
# Wait for container to start
|
||||
echo "Waiting for container to initialize..."
|
||||
sleep 10
|
||||
|
||||
# Show container logs for debugging
|
||||
docker logs windmill-extra-test
|
||||
|
||||
- name: Run integration tests
|
||||
run: |
|
||||
bun run docker/test_windmill_extra.ts
|
||||
|
||||
- name: Show container logs on failure
|
||||
if: failure()
|
||||
run: |
|
||||
echo "=== Container logs ==="
|
||||
docker logs windmill-extra-test
|
||||
|
||||
- name: Cleanup
|
||||
if: always()
|
||||
run: |
|
||||
docker stop windmill-extra-test || true
|
||||
docker rm windmill-extra-test || true
|
||||
|
||||
publish_extra:
|
||||
needs: [sleep, test_extra]
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push publicly
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/DockerfileExtra
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta.outputs.labels }}
|
||||
org.opencontainers.image.licenses=AGPLv3
|
||||
@@ -13,7 +13,7 @@ env:
|
||||
|
||||
jobs:
|
||||
cargo_build_windows:
|
||||
runs-on: blacksmith-16vcpu-windows-2025
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -32,37 +32,23 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.97.0
|
||||
|
||||
- name: Substitute EE code
|
||||
shell: bash
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Cargo build dynamic libraries windows
|
||||
timeout-minutes: 180
|
||||
run: |
|
||||
cd backend/windmill-duckdb-ffi-internal
|
||||
cargo build --release -p windmill_duckdb_ffi_internal
|
||||
|
||||
- name: Cargo build windows
|
||||
timeout-minutes: 180
|
||||
timeout-minutes: 90
|
||||
run: |
|
||||
vcpkg.exe install openssl-windows:x64-windows
|
||||
vcpkg.exe install openssl:x64-windows-static
|
||||
vcpkg.exe integrate install
|
||||
$env:VCPKGRS_DYNAMIC=1
|
||||
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
|
||||
cd backend
|
||||
# Stub the openapi specs to empty: they are compiled in via an ungated
|
||||
# include_str! but a worker binary never serves them, so this avoids
|
||||
# embedding ~2.5MB of spec.
|
||||
mkdir frontend/build && cd backend
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.json" -ItemType "File" -Force
|
||||
cargo build --release --features=ee_windows
|
||||
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,deno_core
|
||||
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
|
||||
@@ -72,9 +58,3 @@ jobs:
|
||||
with:
|
||||
files: |
|
||||
./backend/target/release/windmill-ee.exe
|
||||
|
||||
- name: Attach dynamic libraries to release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: |
|
||||
./backend/windmill-duckdb-ffi-internal/target/release/windmill_duckdb_ffi_internal.dll
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
name: Refresh docs snapshot
|
||||
|
||||
# The backend embeds a vendored docs snapshot (backend/windmill-api/docs_snapshot/*.gz)
|
||||
# so in-product docs search works with no runtime egress. This job re-fetches it from
|
||||
# windmill.dev on a schedule and opens a PR when it changed, keeping the embedded docs
|
||||
# fresh independently of the release cadence (the binary embeds whatever is on the
|
||||
# source tree at build time, so a merged refresh rides into the next release build).
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 6 * * 1" # Mondays 06:00 UTC
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
refresh:
|
||||
runs-on: ubicloud
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
- name: Fetch + re-gzip docs snapshot
|
||||
run: cd backend/windmill-api/docs_snapshot && ./fetch.sh
|
||||
- name: Sanity-check the fetched corpus
|
||||
# curl -f in fetch.sh rejects HTTP errors, but not a valid-but-garbage 200
|
||||
# (truncated file, error page). Guard against embedding a broken snapshot.
|
||||
run: |
|
||||
cd backend/windmill-api/docs_snapshot
|
||||
test "$(wc -c < llms-full.txt.gz)" -gt 100000
|
||||
test "$(wc -c < llms.txt.gz)" -gt 1000
|
||||
pages=$(gzip -dc llms-full.txt.gz | grep -c '^Source:' || true)
|
||||
echo "pages in snapshot: $pages"
|
||||
test "${pages:-0}" -ge 200
|
||||
- uses: peter-evans/create-pull-request@v6
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
branch: chore/refresh-docs-snapshot
|
||||
add-paths: backend/windmill-api/docs_snapshot/*.gz
|
||||
commit-message: "chore: refresh vendored docs snapshot"
|
||||
title: "chore: refresh vendored docs snapshot"
|
||||
body: |
|
||||
Automated refresh of the embedded docs snapshot
|
||||
(`backend/windmill-api/docs_snapshot/*.gz`) from windmill.dev.
|
||||
|
||||
Review the diff for unexpected churn (a bad upstream docs deploy would
|
||||
show up as a large drop in pages or content) before merging.
|
||||
@@ -1,24 +0,0 @@
|
||||
name: Rust Client Check
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Change versions"]
|
||||
types:
|
||||
- completed
|
||||
push:
|
||||
paths:
|
||||
- "rust-client/**"
|
||||
- "backend/**/*.rs"
|
||||
- "backend/windmill-api/openapi.yaml"
|
||||
- "version.txt"
|
||||
- "flake.nix"
|
||||
- ".github/workflows/rust-client-check.yml"
|
||||
|
||||
jobs:
|
||||
check_rust_client:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: cachix/install-nix-action@v31
|
||||
- name: Check rust client builds
|
||||
run: cd rust-client && nix develop ../ --command ./dev.nu --check
|
||||
timeout-minutes: 16
|
||||
@@ -1,16 +0,0 @@
|
||||
name: Publish rust-client to crates.io on release
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
build_rust_and_publish_to_crates_io:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: cachix/install-nix-action@v31
|
||||
- run: cd rust-client && nix develop ../ --command ./dev.nu --check --publish
|
||||
env:
|
||||
CRATES_IO_TOKEN: ${{ secrets.CRATES_IO_TOKEN }}
|
||||
@@ -1,214 +0,0 @@
|
||||
name: "Notify Discord when a PR is opened or merged"
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
PR_TITLE:
|
||||
description: "The title of the PR"
|
||||
type: string
|
||||
PR_URL:
|
||||
description: "The URL of the PR"
|
||||
type: string
|
||||
PR_AUTHOR:
|
||||
description: "The author of the PR"
|
||||
type: string
|
||||
PR_STATUS:
|
||||
description: "The status of the PR"
|
||||
type: string
|
||||
DISCORD_CHANNEL_ID:
|
||||
description: "The Discord channel ID"
|
||||
type: string
|
||||
PR_NUMBER:
|
||||
description: "The number of the PR"
|
||||
type: string
|
||||
DISCORD_GUILD_ID:
|
||||
description: "The Discord guild ID"
|
||||
type: string
|
||||
COMMENT_BODY:
|
||||
description: "The comment body"
|
||||
type: string
|
||||
default: ""
|
||||
COMMENT_AUTHOR:
|
||||
description: "The comment author"
|
||||
type: string
|
||||
default: ""
|
||||
COMMENT_URL:
|
||||
description: "The comment URL"
|
||||
type: string
|
||||
default: ""
|
||||
COMMENT_IS_EDIT:
|
||||
description: "Whether this is an edit of an existing comment"
|
||||
type: string
|
||||
default: "false"
|
||||
secrets:
|
||||
DISCORD_WEBHOOK_URL:
|
||||
description: "Discord Webhook URL"
|
||||
required: false
|
||||
DISCORD_BOT_TOKEN:
|
||||
description: "Discord Bot Token"
|
||||
|
||||
jobs:
|
||||
open_thread:
|
||||
runs-on: ubicloud-standard-2
|
||||
if: ${{ inputs.PR_STATUS == 'opened' }}
|
||||
steps:
|
||||
- name: Send Discord notification and start a thread
|
||||
env:
|
||||
WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
|
||||
BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
|
||||
CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
|
||||
GUILD_ID: ${{ inputs.DISCORD_GUILD_ID }}
|
||||
PR_TITLE: ${{ inputs.PR_TITLE }}
|
||||
PR_NUMBER: ${{ inputs.PR_NUMBER }}
|
||||
PR_URL: ${{ inputs.PR_URL }}
|
||||
PR_AUTHOR: ${{ inputs.PR_AUTHOR }}
|
||||
run: |
|
||||
# Check if thread already exists
|
||||
thread_exists=false
|
||||
if threads=$(curl -s -H "Authorization: Bot $BOT_TOKEN" "https://discord.com/api/v10/guilds/${GUILD_ID}/threads/active"); then
|
||||
if thread_id=$(echo "$threads" | jq -r --arg cid "$CHANNEL_ID" --arg pref "#${PR_NUMBER}:" '.threads[] | select(.parent_id == $cid and (.name | startswith($pref))) | .id' 2>/dev/null); then
|
||||
if [ -n "$thread_id" ]; then
|
||||
thread_exists=true
|
||||
echo "Thread already exists, skipping creation"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "Failed to check for existing threads, will create new thread"
|
||||
fi
|
||||
|
||||
# Create thread if it doesn't exist or if check failed
|
||||
if [ "$thread_exists" = false ]; then
|
||||
echo "Creating new thread"
|
||||
THREAD_TITLE="#${PR_NUMBER}: ${PR_TITLE} by \`${PR_AUTHOR}\`"
|
||||
payload=$(jq -n \
|
||||
--arg content "${PR_URL}" \
|
||||
--arg thread "${THREAD_TITLE:0:99}" \
|
||||
'{
|
||||
content: $content,
|
||||
thread_name: $thread,
|
||||
auto_archive_duration: 10080
|
||||
}'
|
||||
)
|
||||
curl -H "Content-Type: application/json" \
|
||||
-X POST \
|
||||
-d "$payload" \
|
||||
"$WEBHOOK_URL"
|
||||
fi
|
||||
|
||||
merge_success_emoji:
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ inputs.PR_STATUS == 'merged' }}
|
||||
steps:
|
||||
- name: React
|
||||
env:
|
||||
BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
|
||||
CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
|
||||
GUILD_ID: ${{ inputs.DISCORD_GUILD_ID }}
|
||||
PR_NUMBER: ${{ inputs.PR_NUMBER }}
|
||||
run: |
|
||||
# 1) get PR thread
|
||||
threads=$(curl -H "Authorization: Bot $BOT_TOKEN" "https://discord.com/api/v10/guilds/${GUILD_ID}/threads/active")
|
||||
thread_id=$(
|
||||
echo "$threads" \
|
||||
| jq -r --arg cid "$CHANNEL_ID" \
|
||||
--arg pref "#${PR_NUMBER}:" \
|
||||
'.threads[]
|
||||
| select(.parent_id == $cid and (.name | startswith($pref)))
|
||||
| .id'
|
||||
)
|
||||
if [ -z "$thread_id" ]; then
|
||||
echo "Thread not found"
|
||||
exit 1
|
||||
fi
|
||||
# 2) get the first message in that thread
|
||||
messages=$(curl -H "Authorization: Bot $BOT_TOKEN" \
|
||||
"https://discord.com/api/v10/channels/$thread_id/messages")
|
||||
message_id=$(echo "$messages" | jq -r '.[-1].id')
|
||||
|
||||
if [ -z "$message_id" ]; then
|
||||
echo "Message not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 3) add the ✅ reaction
|
||||
curl -X PUT \
|
||||
-H "Authorization: Bot $BOT_TOKEN" \
|
||||
"https://discord.com/api/v10/channels/$thread_id/messages/$message_id/reactions/%E2%9C%85/@me"
|
||||
|
||||
post_comment:
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ inputs.PR_STATUS == 'comment' }}
|
||||
steps:
|
||||
- name: Post or update comment in Discord thread
|
||||
env:
|
||||
BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
|
||||
CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
|
||||
GUILD_ID: ${{ inputs.DISCORD_GUILD_ID }}
|
||||
PR_NUMBER: ${{ inputs.PR_NUMBER }}
|
||||
COMMENT_BODY: ${{ inputs.COMMENT_BODY }}
|
||||
COMMENT_AUTHOR: ${{ inputs.COMMENT_AUTHOR }}
|
||||
COMMENT_URL: ${{ inputs.COMMENT_URL }}
|
||||
COMMENT_IS_EDIT: ${{ inputs.COMMENT_IS_EDIT }}
|
||||
run: |
|
||||
# 1) Find the thread by PR number
|
||||
threads=$(curl -s -H "Authorization: Bot $BOT_TOKEN" \
|
||||
"https://discord.com/api/v10/guilds/${GUILD_ID}/threads/active")
|
||||
thread_id=$(echo "$threads" | jq -r \
|
||||
--arg cid "$CHANNEL_ID" \
|
||||
--arg pref "#${PR_NUMBER}:" \
|
||||
'.threads[] | select(.parent_id == $cid and (.name | startswith($pref))) | .id')
|
||||
|
||||
if [ -z "$thread_id" ]; then
|
||||
echo "Thread not found for PR #${PR_NUMBER}, skipping"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 2) Truncate comment body to fit Discord's 2000 char limit
|
||||
# Reserve space for the author line + link (~100 chars)
|
||||
max_body=1800
|
||||
if [ ${#COMMENT_BODY} -gt $max_body ]; then
|
||||
# For bot comments, show the tail (conclusions/code tend to be at the end)
|
||||
if [[ "$COMMENT_AUTHOR" == *"[bot]"* ]] || [[ "$COMMENT_AUTHOR" == *"-bot"* ]]; then
|
||||
truncated_body="...${COMMENT_BODY: -$max_body}"
|
||||
else
|
||||
truncated_body="${COMMENT_BODY:0:$max_body}..."
|
||||
fi
|
||||
else
|
||||
truncated_body="$COMMENT_BODY"
|
||||
fi
|
||||
|
||||
# 3) Build the message content
|
||||
if [ "$COMMENT_IS_EDIT" = "true" ]; then
|
||||
message=$(printf '**%s** [edited comment](%s):\n%s' "$COMMENT_AUTHOR" "$COMMENT_URL" "$truncated_body")
|
||||
else
|
||||
message=$(printf '**%s** [commented](%s):\n%s' "$COMMENT_AUTHOR" "$COMMENT_URL" "$truncated_body")
|
||||
fi
|
||||
payload=$(jq -n --arg content "$message" '{content: $content, flags: 4, allowed_mentions: {parse: []}}')
|
||||
|
||||
# 4) If this is an edit, try to find and update the existing Discord message
|
||||
if [ "$COMMENT_IS_EDIT" = "true" ]; then
|
||||
# Search recent messages in the thread for one containing the comment URL
|
||||
messages=$(curl -s -H "Authorization: Bot $BOT_TOKEN" \
|
||||
"https://discord.com/api/v10/channels/${thread_id}/messages?limit=100")
|
||||
existing_msg_id=$(echo "$messages" | jq -r \
|
||||
--arg url "$COMMENT_URL" \
|
||||
'[.[] | select(.content | contains($url))] | first | .id // empty')
|
||||
|
||||
if [ -n "$existing_msg_id" ]; then
|
||||
echo "Updating existing Discord message $existing_msg_id"
|
||||
curl -s -X PATCH \
|
||||
-H "Authorization: Bot $BOT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$payload" \
|
||||
"https://discord.com/api/v10/channels/${thread_id}/messages/${existing_msg_id}"
|
||||
exit 0
|
||||
fi
|
||||
echo "Original Discord message not found, posting as new message"
|
||||
fi
|
||||
|
||||
# 5) Post a new message to the thread
|
||||
curl -s -X POST \
|
||||
-H "Authorization: Bot $BOT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$payload" \
|
||||
"https://discord.com/api/v10/channels/${thread_id}/messages"
|
||||
@@ -15,7 +15,7 @@ jobs:
|
||||
== 'I have read the CLA Document and I hereby sign the CLA') ||
|
||||
github.event_name == 'pull_request_target'
|
||||
# Beta Release
|
||||
uses: cla-assistant/github-action@v2.6.1
|
||||
uses: cla-assistant/github-action@v2.3.1
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PERSONAL_ACCESS_TOKEN: ${{ secrets.CLA_PAT }}
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
name: Validate OpenAPI Spec
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- 'backend/windmill-api/openapi*'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'backend/windmill-api/openapi*'
|
||||
jobs:
|
||||
validate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Install openapi-generator-cli
|
||||
run: npm install @openapitools/openapi-generator-cli -g
|
||||
|
||||
- name: Validate openapi.yaml
|
||||
run: npx @openapitools/openapi-generator-cli validate -i backend/windmill-api/openapi.yaml
|
||||
|
||||
- name: Validate openapi-deref.json
|
||||
run: npx @openapitools/openapi-generator-cli validate -i backend/windmill-api/openapi-deref.json
|
||||
|
||||
# Does not work well with dereferenced yaml
|
||||
# - name: Validate openapi-deref.yaml
|
||||
# run: npx @openapitools/openapi-generator-cli validate -i backend/windmill-api/openapi-deref.yaml
|
||||
|
||||
@@ -1,145 +0,0 @@
|
||||
name: Weekly PR Summary
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Every Friday at 8:00 AM UTC
|
||||
- cron: '0 8 * * 5'
|
||||
workflow_dispatch:
|
||||
# Allow manual triggering for testing
|
||||
|
||||
jobs:
|
||||
weekly-pr-summary:
|
||||
runs-on: ubicloud-standard-4
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Generate Weekly PR Summary
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
prompt: |
|
||||
REPO: ${{ github.repository }}
|
||||
|
||||
Generate a categorized weekly summary of ONLY MERGED Pull Requests from the past 7 days.
|
||||
|
||||
## Your Task:
|
||||
|
||||
1. **Calculate Date Range**:
|
||||
- Run: `CUTOFF_DATE=$(date --date='7 days ago' --iso-8601)`
|
||||
- Run: `TODAY=$(date --iso-8601)`
|
||||
- This gives you the exact 7-day window (store these in variables for use in commands)
|
||||
|
||||
2. **Fetch ONLY Merged PRs from Past Week**:
|
||||
- Command: `gh pr list --repo ${{ github.repository }} --state merged --search "merged:>=$CUTOFF_DATE" --limit 100 --json number,title,author,mergedAt,url`
|
||||
- This returns ONLY PRs that were merged in the last 7 days
|
||||
- The --search flag filters by merge date using GitHub's search syntax
|
||||
- **FILTER OUT** any PRs with titles starting with "chore: release" or "chore(release)"
|
||||
|
||||
3. **Categorize PRs**: Group PRs into three categories by analyzing titles and labels:
|
||||
- **Features**: PRs with titles starting with "feat:", "feature:", or containing "add", "implement", "new"
|
||||
- **Bug Fixes**: PRs with titles starting with "fix:", "bug:", or containing "fix", "resolve", "patch"
|
||||
- **Other**: All remaining PRs (improvements, refactors, docs, chores, etc.)
|
||||
|
||||
4. **Gather Details**: For each feature and bug fix merged PR, include:
|
||||
- Full PR title (NO truncation, NO links)
|
||||
- Author (extract login from author.login in JSON)
|
||||
- Brief summary: Use `gh pr view <number> --json body` to get PR description, then extract first paragraph or key points (1-2 sentences max)
|
||||
|
||||
5. **Character Limit Enforcement**:
|
||||
- The final summary MUST be under 5000 characters
|
||||
- If the summary exceeds 5000 characters, truncate PR descriptions (NOT titles) and add at the end: "_and X more PRs_" where X is the count of omitted PRs
|
||||
|
||||
6. **Save Summary to Markdown File**: Write the summary to a file for webhook delivery:
|
||||
- Save the complete formatted markdown to: `summary.md`
|
||||
- Do not commit the file to the repository
|
||||
|
||||
## Output Format:
|
||||
|
||||
```markdown
|
||||
### 📈 Weekly overview
|
||||
- **Total merged**: X
|
||||
- **Features**: Y
|
||||
- **Bug Fixes**: Z
|
||||
- **Other**: W
|
||||
|
||||
### ✨ Features (Y)
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
|
||||
### 🐛 Bug Fixes (Z)
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
|
||||
_and X more PRs_
|
||||
```
|
||||
|
||||
## Important Notes:
|
||||
- **CRITICAL**: ONLY include PRs with state "merged" from the last 7 days
|
||||
- **CRITICAL**: EXCLUDE all PRs with titles starting with "chore: release" or "chore(release)"
|
||||
- **CRITICAL**: Total character count MUST be under 5000 characters
|
||||
- Count the number of "Other" PRs but do not include a section for them in the output
|
||||
- Only use ### markdown headers for major sections and emoji indicators
|
||||
- NO links to PRs
|
||||
- NO merged date in output
|
||||
- NEVER truncate PR titles - show full titles
|
||||
- Use GitHub CLI (`gh`) for all operations
|
||||
- Sort PRs within each category by merge date (most recent first)
|
||||
- If a PR has no description, write "(No description provided)"
|
||||
- Extract meaningful summary from PR body - look for the first paragraph or key bullet points
|
||||
- Parse JSON responses carefully using `jq` or similar tools
|
||||
- If summary exceeds 5000 chars, shorten PR descriptions and add "_and X more PRs_" at the end
|
||||
- Count PRs in each category and display in both overview and section headers
|
||||
|
||||
## Saving the Markdown Output:
|
||||
After generating the markdown summary, save it to a file, BUT DO NOT COMMIT IT TO THE REPOSITORY.
|
||||
|
||||
## Write Tool Fallback:
|
||||
- First, attempt to use the Write tool to create `summary.md` with the markdown content
|
||||
- If the Write tool returns ANY error or fails:
|
||||
1. Use the Bash tool with the `echo` command instead
|
||||
2. Use a heredoc to write the content: `cat > summary.md << 'EOF'` followed by your markdown content and `EOF` on a new line
|
||||
3. Example: `cat > summary.md << 'EOF'\n[your markdown content here]\nEOF`
|
||||
4. This ensures the file is always created regardless of Write tool issues
|
||||
- Verify the file was created by running: `ls -lh summary.md`
|
||||
claude_args: |
|
||||
--allowedTools "Edit,MultiEdit,Write,Read,Glob,Grep,LS,Bash"
|
||||
--model haiku
|
||||
|
||||
- name: Send Summary to Windmill
|
||||
if: hashFiles('summary.md') != ''
|
||||
env:
|
||||
WEEKLY_SUMMARY_TOKEN: ${{ secrets.WEEKLY_SUMMARY_TOKEN }}
|
||||
run: |
|
||||
if [[ -f "summary.md" ]]; then
|
||||
echo "Found summary.md, sending to Windmill..."
|
||||
|
||||
# Read the markdown content
|
||||
MARKDOWN_CONTENT=$(cat summary.md)
|
||||
|
||||
# Create JSON payload
|
||||
PAYLOAD=$(jq -n --arg markdown "$MARKDOWN_CONTENT" '{markdown: $markdown}')
|
||||
|
||||
# Send to Windmill webhook
|
||||
RESULT=$(curl -s \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H "Authorization: Bearer $WEEKLY_SUMMARY_TOKEN" \
|
||||
-X POST \
|
||||
-d "$PAYLOAD" \
|
||||
'https://app.windmill.dev/api/w/windmill-labs/jobs/run/f/f/ai/send_past_week_pr_summaries_to_discord')
|
||||
|
||||
echo "Windmill response:"
|
||||
echo -E "$RESULT"
|
||||
echo "✅ Summary sent successfully to Windmill!"
|
||||
else
|
||||
echo "⚠️ Warning: summary.md not found, skipping delivery"
|
||||
exit 1
|
||||
fi
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user