starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-25 08:00:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(dev): correct the worktree dev-environment guidance

Browse Source 
Several things agents were told to do did not match what the machine does.

- Env discovery pointed at .env / .env.local / backend/.env. In a webmux
  worktree the real values are in $(git rev-parse --git-dir)/webmux/runtime.env
  (BACKEND_PORT, FRONTEND_PORT, DATABASE_URL, CARGO_FEATURES, WM_DB_NAME),
  sourced by every pane and undocumented. Reading it is also not blocked by the
  Read(**/.env) deny rules, which the old instruction walked straight into.
- The database name rule said branch-with-underscores. worktree-common.sh uses
  the worktree directory basename, and Postgres truncates at 63 characters, so
  branch hugo/win-2340-… resolves to windmill_win_2340_…_and_eval with no hugo_
  prefix and the tail chopped. A wrong DATABASE_URL guts the sqlx cache.
- The restart procedure said "tmux pane 1" and sent keys to an undefined
  <pane1>. Pane 1 is the backend under the full profile and the frontend under
  frontendOnly. Replaced with finding the pane by pane_current_command,
  recovering the live feature set from the running process (CARGO_FEATURES in
  runtime.env only records what the pane started with), and restarting in place.
- Added recovery for an orphaned backend holding the port: it reparents to
  systemd when its shell dies, so it survives anything that looks like cleanup.
  Three checks before killing a single pid, because pkill -f windmill takes out
  every sibling worktree.
- Agents spawned their own servers because AGENTS.md opened by telling them to.
  Now it checks for the existing panes first; the spawn commands are scoped to
  a plain checkout.
- New EE worktrees branched from the EE repo's local main, which nothing
  fast-forwards, so they started behind the commit pinned in
  backend/ee-repo-ref.txt — the one CI builds against. They now base on the pin,
  falling back to main only when it is unreadable.
- Enabled webmux autoPull so local main stays current; new worktrees are
  branched from it. Documented what WM_CLONE_DB does, including that it
  terminates every connection to the base windmill database.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
hugocasa
2026-08-12 18:23:58 +02:00
parent cbcdcbd28b
commit 2d977f8ffe
4 changed files with 103 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.webmux.yaml
+10
View File
@@ -5,9 +5,19 @@ workspace:
mainBranch: main
worktreeRoot: ../windmill__worktrees
defaultAgent: claude
# A new worktree is branched from the *local* `main` ref, so a stale local main means every
# new worktree starts behind. This keeps it current: fetch origin/main + fast-forward merge.
# Fast-forward only — it no-ops rather than forcing if local main has diverged.
autoPull:
enabled: true
intervalSeconds: 300
startupEnvs:
CARGO_FEATURES: "quickjs"
# true clones the base `windmill` DB via CREATE DATABASE ... TEMPLATE, which first
# terminates every open connection to `windmill` — expect the main dev instance to drop.
# false creates an empty DB and runs migrations. Either way the license key is copied over
# and pre-remove drops the DB. See scripts/worktree-common.sh.
WM_CLONE_DB: false
USE_RUST_PLUGIN: false
AGENTS.md
+22 -6
View File
@@ -50,9 +50,15 @@ Open-source platform for internal tools, workflows, API integrations, background
> defaults in this section apply only to a plain single checkout. **Discover the real
> values before running anything** — see "Per-worktree ports and database" below.
**Check whether they are already running before starting anything.** In a webmux worktree
(`$WEBMUX_WORKTREE_PATH` is set) the backend and frontend are already up in sibling tmux panes —
use those, don't spawn your own. `tmux list-panes -t "$(tmux display-message -p -t "$TMUX_PANE"
'#{window_id}')" -F '#{pane_index} #{pane_current_command}'` shows what is running; read its log
with `tmux capture-pane`, and see `backend/CLAUDE.md` to restart it with different cargo features.
A second server started in your own shell fights the first one for the port. The commands below
are for a plain checkout with nothing running.
- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
- **DuckDB local jobs**: before running DuckDB scripts locally, build the FFI shared library with `cd backend/windmill-duckdb-ffi-internal && ./build_dev.sh`. Re-run it after clean builds or when `backend/target/debug/libwindmill_duckdb_ffi_internal.*` is missing. The bundled DuckDB compile (~2min) is cached in a per-user dir shared across worktrees, so a fresh worktree reuses it and the build is near-instant.
- **Data pipelines (DuckLake) from source**: a plain `cargo run` (even `--features quickjs`) advertises a `duckdb` worker tag but **cannot** execute DuckDB scripts and has **no** working S3 proxy (DuckLake writes 404). Build CE DuckLake with `cargo run --features quickjs,duckdb,parquet,private` (add `,python` for Python scripts, `,enterprise,license` for EE) **and** build the FFI (bullet above). See `backend/CLAUDE.md` → "Running data pipelines (DuckLake) from source" for the exact feature sets and the two feature-gate gotchas.
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/` (port 3000+)
- **DB**: `psql postgres://postgres:changeme@localhost:5432/windmill`
- **Login**: `admin@windmill.dev` / `changeme`
@@ -61,10 +67,20 @@ Open-source platform for internal tools, workflows, API integrations, background
### Per-worktree ports and database
A worktree's `.env` / `.env.local` (repo root) and `backend/.env` hold its own
`DATABASE_URL` and `PORT`; the database is typically `windmill_<branch_with_underscores>`
(branch `dbt-runtime` → `windmill_dbt_runtime`). Read them, or discover from what is
already running:
In a webmux worktree the authoritative values live in
`$(git rev-parse --git-dir)/webmux/runtime.env` — `BACKEND_PORT`, `FRONTEND_PORT`,
`DATABASE_URL`, `CARGO_FEATURES`, `WM_DB_NAME`. Every pane sources it at startup. Read that
first: it is not a `.env*` file, so the repo's secret-file read rules don't stand in the way.
In a plain checkout, fall back to `.env` / `.env.local` (repo root) and `backend/.env`.
The database is named after the **worktree directory, not the branch** (`scripts/worktree-common.sh`):
`windmill_` + the directory basename with `-` → `_`, which Postgres then truncates at 63
characters. Branch `hugo/win-2340-ai-agent-evals-standalone-agent-runs-and-eval-datasets` sits in
a worktree directory named `win-2340-…`, so its database is
`windmill_win_2340_ai_agent_evals_standalone_agent_runs_and_eval` — no `hugo_`, and the tail
chopped. Take `WM_DB_NAME` from `runtime.env` instead of reconstructing the name. Read those, or
discover from what is already running:
```bash
psql postgres://postgres:changeme@localhost:5432/postgres -tAc \
backend/CLAUDE.md
+56 -10
View File
@@ -32,23 +32,69 @@ MUST **restart the backend with the appropriate features** for what you're worki
### Restarting the dev backend with the right features
The backend runs in tmux pane 1 as `cargo watch -x "run --features <…>"`. To restart it with a
different feature set — scope kills by pid/cwd, **never** `pkill -f target/debug/windmill` (it
kills every sibling worktree's backend):
Restart in the **same pane**, so the relaunch inherits that pane's `DATABASE_URL`, `BACKEND_PORT`
and the rest of `runtime.env`. Scope every kill to this worktree — **never**
`pkill -f target/debug/windmill`, which kills every sibling worktree's backend.
1. Find the backend pane by what it is running, not by index. The index depends on the webmux
profile: pane 1 is the backend under `full`, but the *frontend* under `frontendOnly`.
1. Stop the current run: `tmux send-keys -t <pane1> C-c`, then kill *this worktree's*
`cargo-watch` pid (find it via `/proc/<pid>/cwd`).
2. Relaunch in the same pane so it inherits the shell's `DATABASE_URL` etc.; the pane env's
`PORT` may be stale, so set it explicitly:
```bash
export PORT=$BACKEND_PORT
cargo watch -x "run --features enterprise,private,parquet,quickjs"
WIN=$(tmux display-message -p -t "$TMUX_PANE" '#{window_id}')
tmux list-panes -t "$WIN" -F '#{pane_index} #{pane_current_command} #{pane_pid}'
```
3. Wait for `health check completed` in the pane before hitting the API.
2. Read the feature set it is **actually** running. `CARGO_FEATURES` in `runtime.env` is only what
the pane started with, and goes stale the first time anyone restarts by hand:
```bash
ps --ppid <pane_pid> -o args=
# /home/hugo/.cargo/bin/cargo-watch watch -x run --features quickjs
```
3. Stop it and relaunch with the extended set. `PORT` in the pane shell can be stale, so pass it
explicitly:
```bash
tmux send-keys -t "$WIN.<idx>" C-c
tmux send-keys -t "$WIN.<idx>" 'PORT=$BACKEND_PORT cargo watch -x "run --features quickjs,private,parquet"' Enter
```
Carry over every feature the old command had unless you mean to drop one — rebuilding the list
from memory is how a backend silently loses `quickjs`.
4. Persist the new set so a recreated pane starts with it: set `CARGO_FEATURES` in
`$(git rev-parse --git-dir)/webmux/runtime.env`. That file is read at pane startup only, so it
changes nothing about the process you just relaunched — step 3 is what takes effect now.
5. Re-capture the pane until `health check completed` appears before hitting the API. A cold
rebuild takes ~60s, and the previous run's success line is still in the scrollback, so a
capture taken too early reads as ready when it isn't.
cargo-watch only re-runs on a file change, so after an idle/failed run `touch README.md` (from
`backend/`, where the watch runs) is a cheap retrigger (touching a `.rs` forces a full rebuild).
### An orphaned backend is holding the port
If the pane's `cargo watch` looks alive but the API never answers, or the build ends in an
address-already-in-use error, a backend from an earlier run is probably still bound to the port.
It gets reparented to `systemd --user` when its shell dies, so it survives everything that looks
like a cleanup.
Confirm all three before killing anything — a dozen sibling worktrees run their own backend, and
`pkill -f windmill` (or `-f target/debug/windmill`) kills every one of them:
```bash
ss -ltnp | grep ":$BACKEND_PORT" # 1. which pid holds the port
readlink /proc/<pid>/cwd # 2. must be THIS worktree's backend/
ps -o ppid= -p <pid> # 3. parent is systemd/pid 1, not your pane's cargo-watch
```
Only when the port owner is this worktree's backend **and** it is orphaned, kill that single pid
(`kill <pid>`, then `kill -9` if it does not exit). Ask first when there is a human in the loop;
unattended, the three checks are what make it safe. Then `touch README.md` to retrigger the
watch.
### What each feature gate does (the ones you'll actually toggle)
`backend/Cargo.toml` `[features]` is the source of truth; this is the practical dev map. Combine
scripts/worktree-common.sh
+15 -2
View File
@@ -164,10 +164,20 @@ wm_find_ee_repo() {
return 1
}
# The EE commit this CE checkout builds against. CI reads the same file, so basing a new EE
# worktree on it keeps a local `cargo check --features private` on the tree CI compiles. Local
# `main` in the EE repo is not a substitute: nothing fast-forwards it, so it drifts behind the pin.
wm_ee_pinned_ref() {
local repo_root=$1 ref
ref="$(tr -d '[:space:]' < "${repo_root}/backend/ee-repo-ref.txt" 2>/dev/null)" || return 1
[[ -n "$ref" ]] || return 1
printf '%s' "$ref"
}
wm_setup_ee_worktree() {
local repo_root=$1
local main_repo_root=$2
local ee_repo branch wt_basename ee_worktree_dir ee_rel rust_plugin
local ee_repo branch wt_basename ee_worktree_dir ee_rel rust_plugin ee_ref
if ! ee_repo="$(wm_find_ee_repo "$repo_root" "$main_repo_root")"; then
return
@@ -189,8 +199,11 @@ wm_setup_ee_worktree() {
elif git -C "$ee_repo" show-ref --verify --quiet "refs/remotes/origin/$branch" \
&& git -C "$ee_repo" worktree add --track -b "$branch" "$ee_worktree_dir" "origin/$branch" 2>/dev/null; then
echo "Created EE worktree at $ee_worktree_dir (tracking origin/$branch)"
elif ee_ref="$(wm_ee_pinned_ref "$repo_root")" \
&& git -C "$ee_repo" worktree add -b "$branch" "$ee_worktree_dir" "$ee_ref" 2>/dev/null; then
echo "Created EE worktree at $ee_worktree_dir (new branch: $branch from pinned ${ee_ref:0:12})"
elif git -C "$ee_repo" worktree add -b "$branch" "$ee_worktree_dir" main 2>/dev/null; then
echo "Created EE worktree at $ee_worktree_dir (new branch: $branch from main)"
echo "Created EE worktree at $ee_worktree_dir (new branch: $branch from main — pin unavailable)"
else
echo "Warning: Could not create EE worktree for branch $branch"
fi