docs: make herdr the documented worktree manager (#10727)

* chore: share worktree port assignment across worktree tooling

The slot scan and the .env.local writer existed twice, once inline in
scripts/worktree-env and once in scripts/post-create.sh. Both are now
wm_assign_ports and wm_write_env_local in scripts/worktree-common.sh, so a
third caller cannot drift from the other two.

wm_assign_ports takes the worktree root instead of reading the process cwd,
which lets it run from outside the worktree, and prints its slot line to stderr
so stdout carries only the port pair.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: describe both worktree managers in the dev environment section

The section assumed webmux and tmux throughout. Worktree tooling is being
trialled on herdr alongside it, so the detection markers and the pane
inspection commands now appear side by side.

Corrects the pointer for per-worktree values: .env.local is written by both
managers through wm_write_env_local and is readable, so it is the portable
source. webmux's runtime.env stays documented as the place carrying the extras
.env.local lacks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: point at the herdr plugin setup

Naming the repo left the reader to work out what to do with it. The plugins do
nothing until they are installed and their actions are bound, and Herdr has no
action palette to discover them through.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: narrow the shared helper to .env.local writing

Slot assignment goes back into worktree-env: it had one caller, and the herdr plugin it was extracted for provisions worktrees before the checkout is usable, so it carries its own copy rather than sourcing one whose contents depend on whichever branch the main checkout sits on.

Writing .env.local stays shared - worktree-env and post-create.sh each had the same eleven lines.

Also corrects AGENTS.md: .env.local is a shared format rather than a shared function, and WM_DB_NAME comes from it rather than webmux's runtime.env.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: correct the .env.local readability claim

The file-read tool denies every .env.* path via .claude/settings.json, .env.local included, so the section points at cat instead of implying the file is readable.

DATABASE_URL carries an export prefix, which the bare 'source .env.local' in .envrc needs to reach cargo and sqlx, and which a ^DATABASE_URL= grep would miss. Documented rather than removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: target the backend pane in the webmux capture-pane cell

Without -t the command dumps the agent's own pane. Matches the form .webmux.yaml already uses in its startup prompt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: make herdr the documented worktree manager

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: split how the two managers pick up a CARGO_FEATURES change

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
hugocasa
2026-09-21 12:44:17 +02:00
committed by GitHub
co-authored by Claude Opus 5
parent 56a6ad322e
commit 4f7f705f8c
3 changed files with 69 additions and 35 deletions
+1 -1
View File
@@ -210,7 +210,7 @@ the diff first — `git diff --name-only main...HEAD` answers most of these.
**Flip without asking** when it is self-contained: a single-file fix, test-only, docs-only, one
call site, no new public surface.
Unattended (webmux oneshot) there is nobody to ask, so the judgement holds and the action
Unattended (a one-shot run) there is nobody to ask, so the judgement holds and the action
degrades: flip the self-contained ones, and leave the rest at a clean draft with a line in the PR
description saying why — `left in draft: adds a migration, wants a human look before ready`.
Don't flip a wide-blast-radius change just because the round came back clean, and don't ask a
+40 -17
View File
@@ -55,12 +55,23 @@ Open-source platform for internal tools, workflows, API integrations, background
> defaults in this section apply only to a plain single checkout. **Discover the real
> values before running anything** — see "Per-worktree ports and database" below.
**Check whether they are already running before starting anything.** In a webmux worktree
(`$WEBMUX_WORKTREE_PATH` is set) the backend and frontend are already up in sibling tmux panes —
use those, don't spawn your own. `tmux list-panes -t "$(tmux display-message -p -t "$TMUX_PANE"
'#{window_id}')" -F '#{pane_index} #{pane_current_command}'` shows what is running; read its log
with `tmux capture-pane`, and see `backend/AGENTS.md` to restart it with different cargo features.
A second server started in your own shell fights the first one for the port. The commands below
**Check whether they are already running before starting anything.** In a managed worktree the
backend and frontend are already up in sibling panes — use those, don't spawn your own. A second
server started in your own shell fights the first one for the port.
**herdr** is the worktree manager; `$HERDR_ENV=1` marks one of its panes. `herdr pane list
--workspace "$HERDR_WORKSPACE_ID"` lists this worktree's panes — match on `cwd`, since the servers
run from `backend/` and `frontend/` — and `herdr pane read <pane_id> --source recent-unwrapped
--lines 50` reads one's log; `herdr --skill` prints the full reference for
inspecting and driving panes and agents. The plugins that provision these worktrees live in the
private `windmill-labs/windmill-herdr` — clone it and run `./setup.sh` to install them and the
keybindings they need.
A worktree from the older **webmux** setup sets `$WEBMUX_WORKTREE_PATH` instead and is driven
through tmux: `tmux list-panes -t "$(tmux display-message -p -t "$TMUX_PANE" '#{window_id}')" -F
'#{pane_index} #{pane_current_command}'` for what is running, `tmux capture-pane` for a pane's log.
See `backend/AGENTS.md` to restart the backend with different cargo features. The commands below
are for a plain checkout with nothing running.
- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
@@ -72,10 +83,16 @@ are for a plain checkout with nothing running.
### Per-worktree ports and database
In a webmux worktree the authoritative values live in
`$(git rev-parse --git-dir)/webmux/runtime.env` — `BACKEND_PORT`, `FRONTEND_PORT`,
`DATABASE_URL`, `CARGO_FEATURES`, `WM_DB_NAME`. Every pane sources it at startup. Read that
first: it is not a `.env*` file, so the repo's secret-file read rules don't stand in the way.
**`.env.local` in the worktree root holds the real values** — `BACKEND_PORT`, `FRONTEND_PORT`,
`REMOTE`, `DATABASE_URL`, `CARGO_FEATURES`, `WM_DB_NAME`. Every manager writes those fields, so it
is correct whichever one you are in. Read it with `cat .env.local` from a shell: the file-read
tool denies every `.env.*` path, and `DATABASE_URL` is written with an `export` prefix that a
`^DATABASE_URL=` grep misses.
herdr keeps nothing besides that file; its hooks write it and read it back. A webmux worktree
additionally has `$(git rev-parse --git-dir)/webmux/runtime.env`, which every pane sources at
startup and which carries the extras `.env.local` lacks: `WEBMUX_*`, `WM_CLONE_DB`,
`USE_RUST_PLUGIN`.
In a plain checkout, fall back to `.env` / `.env.local` (repo root) and `backend/.env`.
@@ -84,19 +101,25 @@ hook. It is not a copy of the main dev instance: you get the `admins` workspace,
`admin@windmill.dev` superadmin, the license key copied from the base database, and whatever the
migrations seed — and none of your own workspaces, scripts, flows or apps. Create whatever a test
needs. Cloning the base `windmill` database instead is
opt-in per project via `WM_CLONE_DB` in `.webmux.yaml`; read the note there before turning it on.
opt-in via `WM_CLONE_DB` (the `windmill.worktree` plugin's `config.env` under herdr, or
`.webmux.yaml` under webmux); read the note in `.webmux.yaml` before turning it on.
The database is named after the **worktree directory, not the branch** (`scripts/worktree-common.sh`):
`windmill_` + the directory basename with `-` → `_`, which Postgres then truncates at 63
characters. Branch `hugo/win-2340-ai-agent-evals-standalone-agent-runs-and-eval-datasets` sits in
a worktree directory named `win-2340-…`, so its database is
`windmill_win_2340_ai_agent_evals_standalone_agent_runs_and_eval` — no `hugo_`, and the tail
chopped. Take `WM_DB_NAME` from `runtime.env` instead of reconstructing the name. Read those, or
discover from what is already running:
characters. herdr creates the directory under `~/.herdr/worktrees/<repo>/` from the branch name
with each `/` turned into `-`, so branch
`hugo/win-2544-add-options-field-to-the-postgresql-resource-type` gets the database
`windmill_hugo_win_2544_add_options_field_to_the_postgresql_reso` — the whole branch name, cut
mid-word at the limit. The two drift apart as soon as the branch is renamed, and webmux named its
directories after the ticket alone, so reconstructing the name from the branch you are on is
wrong in both. Take `WM_DB_NAME` from `.env.local` instead. Read that, or discover from what is
already running:
```bash
# match on the worktree directory, truncated the way Postgres truncates it (63 - len('windmill_')):
psql postgres://postgres:changeme@localhost:5432/postgres -tAc \
"select datname from pg_database where datname like 'windmill%'" | grep "$(git branch --show-current | tr - _)"
"select datname from pg_database where datname like 'windmill%'" \
| grep -F "$(basename "$(git rev-parse --show-toplevel)" | tr '/-' '_' | cut -c1-54)"
# the port the frontend actually proxies to (REMOTE of this worktree's vite):
for p in $(pgrep -f vite); do case "$(readlink /proc/$p/cwd)" in *"$(basename "$(git rev-parse --show-toplevel)")"*)
tr '\0' '\n' < /proc/$p/environ | grep -E '^REMOTE=|^PORT=';; esac; done
+28 -17
View File
@@ -23,7 +23,7 @@
## Cargo features & running the dev backend
The dev backend runs under `cargo watch` and is launched by default with **only
`--features quickjs`** (see the tmux backend pane). That baseline compiles fast but
`--features quickjs`** (see the backend pane). That baseline compiles fast but
**deliberately omits most functionality** — notably S3/object storage, the S3 proxy, all
EE code, MCP, and every non-JS language runtime. A running server never gains a feature you
didn't compile in: feature-gated routes 404 or return a `"requires <feature>"` stub. So if
@@ -33,22 +33,30 @@ MUST **restart the backend with the appropriate features** for what you're worki
### Restarting the dev backend with the right features
Restart in the **same pane**, so the relaunch inherits that pane's `DATABASE_URL`, `BACKEND_PORT`
and the rest of `runtime.env`. Scope every kill to this worktree — **never**
and the rest of the worktree environment. Scope every kill to this worktree — **never**
`pkill -f target/debug/windmill`, which kills every sibling worktree's backend.
1. Find the backend pane by what it is running, not by index. The index depends on the webmux
profile: pane 1 is the backend under `full`, but the *frontend* under `frontendOnly`.
1. Find the backend pane by where it is running, not by index. Under herdr the two server panes
sit in the worktree's `backend/` and `frontend/`, so match on `cwd`:
```bash
herdr pane list --workspace "$HERDR_WORKSPACE_ID"
```
Under webmux they are tmux panes, and the index depends on the profile: pane 1 is the backend
under `full`, but the *frontend* under `frontendOnly`.
```bash
WIN=$(tmux display-message -p -t "$TMUX_PANE" '#{window_id}')
tmux list-panes -t "$WIN" -F '#{pane_index} #{pane_current_command} #{pane_pid}'
```
2. Read the feature set it is **actually** running. `CARGO_FEATURES` in `runtime.env` is only what
the pane started with, and goes stale the first time anyone restarts by hand:
2. Read the feature set it is **actually** running. The `CARGO_FEATURES` the worktree was created
with is only what the pane started with, and goes stale the first time anyone restarts by hand:
```bash
ps --ppid <pane_pid> -o args=
herdr pane process-info --pane <pane_id> # foreground_processes[].argv
ps --ppid <pane_pid> -o args= # under webmux
# /home/hugo/.cargo/bin/cargo-watch watch -x run --features quickjs
```
@@ -56,22 +64,25 @@ and the rest of `runtime.env`. Scope every kill to this worktree — **never**
explicitly:
```bash
tmux send-keys -t "$WIN.<idx>" C-c
tmux send-keys -t "$WIN.<idx>" 'PORT=$BACKEND_PORT cargo watch -x "run --features quickjs,private,parquet"' Enter
herdr pane send-keys <pane_id> C-c
herdr pane run <pane_id> 'PORT=$BACKEND_PORT cargo watch -x "run --features quickjs,private,parquet"'
# under webmux: tmux send-keys -t "$WIN.<idx>" C-c, then the same command line with Enter
```
Carry over every feature the old command had unless you mean to drop one — rebuilding the list
from memory is how a backend silently loses `quickjs`.
4. Persist the new set so a recreated pane starts with it: set `CARGO_FEATURES` in the
worktree's `.env.local`, which `scripts/post-create.sh` writes and webmux reads. Do **not**
edit `runtime.env` for this — webmux regenerates it from metadata and `.env.local` every time
the worktree is opened, so an edit there is lost on the next reopen. Either way the change
only affects a future pane; step 3 is what takes effect now.
4. Persist the new set so a recreated pane starts with it: set `CARGO_FEATURES` in the worktree's
`.env.local`, which the worktree hooks write. A herdr pane sources that file directly, so the
next pane picks the value up. A webmux pane sources `webmux/runtime.env` instead, which webmux
regenerates from metadata and `.env.local` when the worktree is *opened* — so the value lands
on the next reopen, not the next pane, and editing `runtime.env` by hand is lost at that same
reopen. Either way the change only affects a future pane; step 3 is what takes effect now.
5. Re-capture the pane until `health check completed` appears before hitting the API. A cold
rebuild takes ~60s, and the previous run's success line is still in the scrollback, so a
capture taken too early reads as ready when it isn't.
5. Re-read the pane (`herdr pane read <pane_id> --source recent-unwrapped`, or `tmux
capture-pane`) until `health check completed` appears before hitting the API. A cold rebuild
takes ~60s, and the previous run's success line is still in the scrollback, so a capture taken
too early reads as ready when it isn't.
cargo-watch only re-runs on a file change, so after an idle/failed run `touch README.md` (from
`backend/`, where the watch runs) is a cheap retrigger (touching a `.rs` forces a full rebuild).