diff --git a/.agents/skills/pr/SKILL.md b/.agents/skills/pr/SKILL.md index 6a72949c96..f2a9ec8dad 100644 --- a/.agents/skills/pr/SKILL.md +++ b/.agents/skills/pr/SKILL.md @@ -210,7 +210,7 @@ the diff first — `git diff --name-only main...HEAD` answers most of these. **Flip without asking** when it is self-contained: a single-file fix, test-only, docs-only, one call site, no new public surface. -Unattended (webmux oneshot) there is nobody to ask, so the judgement holds and the action +Unattended (a one-shot run) there is nobody to ask, so the judgement holds and the action degrades: flip the self-contained ones, and leave the rest at a clean draft with a line in the PR description saying why — `left in draft: adds a migration, wants a human look before ready`. Don't flip a wide-blast-radius change just because the round came back clean, and don't ask a diff --git a/AGENTS.md b/AGENTS.md index a315f1a67c..77b38c3267 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,12 +55,23 @@ Open-source platform for internal tools, workflows, API integrations, background > defaults in this section apply only to a plain single checkout. **Discover the real > values before running anything** — see "Per-worktree ports and database" below. -**Check whether they are already running before starting anything.** In a webmux worktree -(`$WEBMUX_WORKTREE_PATH` is set) the backend and frontend are already up in sibling tmux panes — -use those, don't spawn your own. `tmux list-panes -t "$(tmux display-message -p -t "$TMUX_PANE" -'#{window_id}')" -F '#{pane_index} #{pane_current_command}'` shows what is running; read its log -with `tmux capture-pane`, and see `backend/AGENTS.md` to restart it with different cargo features. -A second server started in your own shell fights the first one for the port. The commands below +**Check whether they are already running before starting anything.** In a managed worktree the +backend and frontend are already up in sibling panes — use those, don't spawn your own. A second +server started in your own shell fights the first one for the port. + +**herdr** is the worktree manager; `$HERDR_ENV=1` marks one of its panes. `herdr pane list +--workspace "$HERDR_WORKSPACE_ID"` lists this worktree's panes — match on `cwd`, since the servers +run from `backend/` and `frontend/` — and `herdr pane read --source recent-unwrapped +--lines 50` reads one's log; `herdr --skill` prints the full reference for +inspecting and driving panes and agents. The plugins that provision these worktrees live in the +private `windmill-labs/windmill-herdr` — clone it and run `./setup.sh` to install them and the +keybindings they need. + +A worktree from the older **webmux** setup sets `$WEBMUX_WORKTREE_PATH` instead and is driven +through tmux: `tmux list-panes -t "$(tmux display-message -p -t "$TMUX_PANE" '#{window_id}')" -F +'#{pane_index} #{pane_current_command}'` for what is running, `tmux capture-pane` for a pane's log. + +See `backend/AGENTS.md` to restart the backend with different cargo features. The commands below are for a plain checkout with nothing running. - **Backend**: `cargo run` from `backend/` (API at http://localhost:8000) @@ -72,10 +83,16 @@ are for a plain checkout with nothing running. ### Per-worktree ports and database -In a webmux worktree the authoritative values live in -`$(git rev-parse --git-dir)/webmux/runtime.env` — `BACKEND_PORT`, `FRONTEND_PORT`, -`DATABASE_URL`, `CARGO_FEATURES`, `WM_DB_NAME`. Every pane sources it at startup. Read that -first: it is not a `.env*` file, so the repo's secret-file read rules don't stand in the way. +**`.env.local` in the worktree root holds the real values** — `BACKEND_PORT`, `FRONTEND_PORT`, +`REMOTE`, `DATABASE_URL`, `CARGO_FEATURES`, `WM_DB_NAME`. Every manager writes those fields, so it +is correct whichever one you are in. Read it with `cat .env.local` from a shell: the file-read +tool denies every `.env.*` path, and `DATABASE_URL` is written with an `export` prefix that a +`^DATABASE_URL=` grep misses. + +herdr keeps nothing besides that file; its hooks write it and read it back. A webmux worktree +additionally has `$(git rev-parse --git-dir)/webmux/runtime.env`, which every pane sources at +startup and which carries the extras `.env.local` lacks: `WEBMUX_*`, `WM_CLONE_DB`, +`USE_RUST_PLUGIN`. In a plain checkout, fall back to `.env` / `.env.local` (repo root) and `backend/.env`. @@ -84,19 +101,25 @@ hook. It is not a copy of the main dev instance: you get the `admins` workspace, `admin@windmill.dev` superadmin, the license key copied from the base database, and whatever the migrations seed — and none of your own workspaces, scripts, flows or apps. Create whatever a test needs. Cloning the base `windmill` database instead is -opt-in per project via `WM_CLONE_DB` in `.webmux.yaml`; read the note there before turning it on. +opt-in via `WM_CLONE_DB` (the `windmill.worktree` plugin's `config.env` under herdr, or +`.webmux.yaml` under webmux); read the note in `.webmux.yaml` before turning it on. The database is named after the **worktree directory, not the branch** (`scripts/worktree-common.sh`): `windmill_` + the directory basename with `-` → `_`, which Postgres then truncates at 63 -characters. Branch `hugo/win-2340-ai-agent-evals-standalone-agent-runs-and-eval-datasets` sits in -a worktree directory named `win-2340-…`, so its database is -`windmill_win_2340_ai_agent_evals_standalone_agent_runs_and_eval` — no `hugo_`, and the tail -chopped. Take `WM_DB_NAME` from `runtime.env` instead of reconstructing the name. Read those, or -discover from what is already running: +characters. herdr creates the directory under `~/.herdr/worktrees//` from the branch name +with each `/` turned into `-`, so branch +`hugo/win-2544-add-options-field-to-the-postgresql-resource-type` gets the database +`windmill_hugo_win_2544_add_options_field_to_the_postgresql_reso` — the whole branch name, cut +mid-word at the limit. The two drift apart as soon as the branch is renamed, and webmux named its +directories after the ticket alone, so reconstructing the name from the branch you are on is +wrong in both. Take `WM_DB_NAME` from `.env.local` instead. Read that, or discover from what is +already running: ```bash +# match on the worktree directory, truncated the way Postgres truncates it (63 - len('windmill_')): psql postgres://postgres:changeme@localhost:5432/postgres -tAc \ - "select datname from pg_database where datname like 'windmill%'" | grep "$(git branch --show-current | tr - _)" + "select datname from pg_database where datname like 'windmill%'" \ + | grep -F "$(basename "$(git rev-parse --show-toplevel)" | tr '/-' '_' | cut -c1-54)" # the port the frontend actually proxies to (REMOTE of this worktree's vite): for p in $(pgrep -f vite); do case "$(readlink /proc/$p/cwd)" in *"$(basename "$(git rev-parse --show-toplevel)")"*) tr '\0' '\n' < /proc/$p/environ | grep -E '^REMOTE=|^PORT=';; esac; done diff --git a/backend/AGENTS.md b/backend/AGENTS.md index 7bd84dfca3..750e0594e1 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -23,7 +23,7 @@ ## Cargo features & running the dev backend The dev backend runs under `cargo watch` and is launched by default with **only -`--features quickjs`** (see the tmux backend pane). That baseline compiles fast but +`--features quickjs`** (see the backend pane). That baseline compiles fast but **deliberately omits most functionality** — notably S3/object storage, the S3 proxy, all EE code, MCP, and every non-JS language runtime. A running server never gains a feature you didn't compile in: feature-gated routes 404 or return a `"requires "` stub. So if @@ -33,22 +33,30 @@ MUST **restart the backend with the appropriate features** for what you're worki ### Restarting the dev backend with the right features Restart in the **same pane**, so the relaunch inherits that pane's `DATABASE_URL`, `BACKEND_PORT` -and the rest of `runtime.env`. Scope every kill to this worktree — **never** +and the rest of the worktree environment. Scope every kill to this worktree — **never** `pkill -f target/debug/windmill`, which kills every sibling worktree's backend. -1. Find the backend pane by what it is running, not by index. The index depends on the webmux - profile: pane 1 is the backend under `full`, but the *frontend* under `frontendOnly`. +1. Find the backend pane by where it is running, not by index. Under herdr the two server panes + sit in the worktree's `backend/` and `frontend/`, so match on `cwd`: + + ```bash + herdr pane list --workspace "$HERDR_WORKSPACE_ID" + ``` + + Under webmux they are tmux panes, and the index depends on the profile: pane 1 is the backend + under `full`, but the *frontend* under `frontendOnly`. ```bash WIN=$(tmux display-message -p -t "$TMUX_PANE" '#{window_id}') tmux list-panes -t "$WIN" -F '#{pane_index} #{pane_current_command} #{pane_pid}' ``` -2. Read the feature set it is **actually** running. `CARGO_FEATURES` in `runtime.env` is only what - the pane started with, and goes stale the first time anyone restarts by hand: +2. Read the feature set it is **actually** running. The `CARGO_FEATURES` the worktree was created + with is only what the pane started with, and goes stale the first time anyone restarts by hand: ```bash - ps --ppid -o args= + herdr pane process-info --pane # foreground_processes[].argv + ps --ppid -o args= # under webmux # /home/hugo/.cargo/bin/cargo-watch watch -x run --features quickjs ``` @@ -56,22 +64,25 @@ and the rest of `runtime.env`. Scope every kill to this worktree — **never** explicitly: ```bash - tmux send-keys -t "$WIN." C-c - tmux send-keys -t "$WIN." 'PORT=$BACKEND_PORT cargo watch -x "run --features quickjs,private,parquet"' Enter + herdr pane send-keys C-c + herdr pane run 'PORT=$BACKEND_PORT cargo watch -x "run --features quickjs,private,parquet"' + # under webmux: tmux send-keys -t "$WIN." C-c, then the same command line with Enter ``` Carry over every feature the old command had unless you mean to drop one — rebuilding the list from memory is how a backend silently loses `quickjs`. -4. Persist the new set so a recreated pane starts with it: set `CARGO_FEATURES` in the - worktree's `.env.local`, which `scripts/post-create.sh` writes and webmux reads. Do **not** - edit `runtime.env` for this — webmux regenerates it from metadata and `.env.local` every time - the worktree is opened, so an edit there is lost on the next reopen. Either way the change - only affects a future pane; step 3 is what takes effect now. +4. Persist the new set so a recreated pane starts with it: set `CARGO_FEATURES` in the worktree's + `.env.local`, which the worktree hooks write. A herdr pane sources that file directly, so the + next pane picks the value up. A webmux pane sources `webmux/runtime.env` instead, which webmux + regenerates from metadata and `.env.local` when the worktree is *opened* — so the value lands + on the next reopen, not the next pane, and editing `runtime.env` by hand is lost at that same + reopen. Either way the change only affects a future pane; step 3 is what takes effect now. -5. Re-capture the pane until `health check completed` appears before hitting the API. A cold - rebuild takes ~60s, and the previous run's success line is still in the scrollback, so a - capture taken too early reads as ready when it isn't. +5. Re-read the pane (`herdr pane read --source recent-unwrapped`, or `tmux + capture-pane`) until `health check completed` appears before hitting the API. A cold rebuild + takes ~60s, and the previous run's success line is still in the scrollback, so a capture taken + too early reads as ready when it isn't. cargo-watch only re-runs on a file change, so after an idle/failed run `touch README.md` (from `backend/`, where the watch runs) is a cheap retrigger (touching a `.rs` forces a full rebuild).