* fix(self-host): accept pre-2.11 Caddyfiles in the caddy-l4 image
The Caddyfile is a bind-mounted file the user owns, so `docker compose pull`
updates the image but never their config. #10106 and #10113 changed the syntax
the image requires (native caddy-l4 `route { proxy { upstream } }`, and a
non-empty `bind`), which strands every existing self-host on their next pull:
Error: adapting config using caddyfile: parsing caddyfile tokens for 'layer4':
wrong argument count or unexpected line ending after 'proxy', at line 4
Normalize legacy Caddyfiles in the entrypoint instead. Only rewrite when the
config cannot be used as-is, and on any failure exec caddy against the user's
original file so it reports a real error against what they wrote.
The bind rewrite is not cosmetic: an empty `bind {$ADDRESS}` adapts and
validates cleanly on caddy >= 2.9 but drops the whole HTTP site, so a
syntax-only shim would trade a restart loop for a container that boots clean
and serves nothing on :80.
The reference for correctness is the image published before #10106
(sha-989c9e6): whatever it adapts today is what self-hosters run, so the shim
must reproduce it byte for byte. docker/test-caddy-compat.sh asserts that over
five legacy variants, plus the :80 listener under an unset ADDRESS, every
--config spelling, relative and glob imports, and the no-op on the current
Caddyfile.
Details worth knowing:
- `to a b` becomes one `upstream` per address; `upstream a b` would be a single
upstream with two dials, which is a different load-balancing topology.
- The rewrite lands next to the original, because caddy resolves `import`
relative to the importing file and a glob import would otherwise silently
expand to nothing.
- The image has no ENTRYPOINT and CMD ["caddy", ...], so an existing
`command:` override starts with a `caddy` token the entrypoint absorbs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(self-host): route ws_mp and ws_debug to the extra gateway
reverse_proxy only reads its first argument as a matcher, so
reverse_proxy /ws/* /ws_mp/* /ws_debug/* http://windmill_extra:3000
adapts to a single /ws/* route whose upstreams are `ws_mp/*:80`,
`ws_debug/*:80` and `windmill_extra:3000`. LSP therefore round-robins across
two garbage hostnames and connects only one time in three, while /ws_mp/* and
/ws_debug/* match no route at all and fall through to windmill_server:8000.
Use a named matcher so all three paths reach the gateway. Verified with traffic
against separate windmill_server and windmill_extra backends: before, /ws/lsp
fails and /ws_mp/room reaches windmill_server; after, all three reach the
gateway with the path preserved and /user/login still reaches windmill_server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(self-host): pin the caddy-l4 image to an explicit version
:latest and the bind-mounted ./Caddyfile it has to agree with are updated by
different mechanisms, so they drift. Publish an explicit version alongside
:latest and pin docker-compose.yml to it, so a checkout is self-consistent:
compose, Caddyfile and image version now move together in one commit.
CI fails the build when docker/caddy-l4.version and the docker-compose.yml pin
disagree, and runs the compatibility-shim tests before publishing. The path
filter now covers the entrypoint, the normalizer, the Caddyfile and
docker-compose.yml, so a change to any guarded input actually triggers the
workflow rather than leaving the check unrun.
:latest keeps being published, since existing deployments reference it and that
is how they pick up the compatibility shim.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(self-host): make the caddy-l4 version tag publishable before the pin merges
docker-compose.yml pins an exact tag, but the version tag was gated on the
default branch, so the tag only appeared after the pin had already merged.
Between the merge and the build finishing, a fresh `docker compose up -d` off
main fails with "manifest unknown", and a failed build leaves main permanently
referencing an image that does not exist.
Drop the gate so the tag can be published from the branch via
workflow_dispatch before merging the pin. The version is immutable, so
republishing it from main is a no-op, and only pushes to main and manual
dispatch run this workflow, so a branch cannot claim the tag by accident.
:latest stays gated on main.
Also check the version file against the caddy version the Dockerfile pins.
Without it, a caddy bump that forgets the version file publishes a tag naming
the wrong caddy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(self-host): do not log Caddyfile contents from the compat shim
The shim logged a unified diff of the rewrite, which carries three lines of
context around each change. A Caddyfile is user-owned and can hold basic_auth
hashes, proxy Authorization headers or TLS provider tokens, and container logs
are routinely shipped off the host, so normalizing a customized config could
copy secrets into them. Reproduced with a basic_auth bcrypt hash landing in the
log as context around the bind rewrite.
Log the number of rewritten lines and the path to the rewritten file instead.
It sits next to the original, so an operator can diff it themselves.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to GIT-903 / PR #10106. That PR migrated the caddy-l4 image to
mholt's native `layer4` Caddyfile support, which required bumping the
Caddy base image to 2.11.4.
End-to-end testing (running the image and proxying real traffic, not
just `caddy adapt`) revealed that caddy >= 2.9 changed how `bind` treats
an empty argument. The shipped Caddyfile has `bind {$ADDRESS}` inside the
`{$BASE_URL}` site, and docker-compose leaves ADDRESS unset -- the
default self-host case. On 2.11.4 the empty `bind` makes Caddy drop the
entire `{$BASE_URL}` site, so the container listens only on :25 (layer4)
and the :80 HTTP reverse proxy to windmill_server silently disappears.
config-only checks (adapt/validate/boot) pass, so only real traffic
surfaces it.
Fix in the Caddyfile rather than downgrading Caddy (which would
reintroduce known CVEs on an internet-facing proxy): default the bind to
all interfaces when ADDRESS is unset via `bind {$ADDRESS:0.0.0.0 ::}`.
When ADDRESS is set it is honored unchanged; when unset the site binds
IPv4 + IPv6, matching the pre-2.9 behavior.
Verified on the caddy:2.11.4 image with a mock windmill_server backend
(HTTP :8000 + layer4 echo :2525):
- ADDRESS unset -> :80 and :25 both bind; HTTP and layer4 both proxy
- ADDRESS=0.0.0.0 -> same
- ADDRESS=127.0.0.1 -> HTTP site binds 127.0.0.1:80 (knob preserved)
Fixes GIT-903
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The self-hosted Caddy image relied on the abandoned
RussellLuo/caddy-ext/layer4 shim to provide the `layer4` Caddyfile
global option, alongside an old (May 2024) pin of mholt/caddy-l4 that
predated native Caddyfile support. This combination is fragile:
- If the image is ever built without the RussellLuo shim, the `layer4`
global option disappears and Caddy fails with
"unrecognized global option: layer4" — the reported bug.
- Bumping mholt/caddy-l4 to any version with native Caddyfile support
makes both modules register `layer4`, panicking at startup with
"global option 'layer4' already registered".
mholt/caddy-l4 now natively registers the `layer4` global option, so
drop the RussellLuo dependency entirely and switch the Caddyfile to the
native `route { proxy { upstream ... } }` syntax. The adapted layer4
JSON is byte-identical to the previous output, so runtime behavior is
unchanged.
Also bump the Caddy base image to 2.11.4 (required by current
caddy-l4) and add a path-filtered push trigger so the published
`:latest` image is rebuilt whenever the Caddy Dockerfile changes,
instead of only on manual dispatch (which is how `:latest` drifted out
of sync with the checked-in Caddyfile in the first place).
Fixes GIT-903
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add gateway reverse proxy for extra services
Add a lightweight Node.js gateway on port 3000 that routes requests
by URL prefix (/ws/*, /ws_mp/*, /ws_debug/*) to the correct backend
service, stripping the prefix before forwarding. This allows all
extra services to be accessed through a single port.
Also makes the multiplayer server more tolerant by generically
stripping /ws_mp/ prefix on HTTP requests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: enable gateway by default for extra services
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: add REMOTE_EXTRA env var for unified extra services proxy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: make gateway port configurable via PORT env var
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: simplify Caddyfile extra services routing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Updates the debugger service to use port 3003 instead of 5679 across
all configuration files, documentation, and code references. This aligns
the debugger with the other windmill-extra services which use ports
3001 (LSP) and 3002 (Multiplayer).
Changes:
- docker-compose.yml: Update port exposure and add DEBUGGER_PORT env
- docker/entrypoint-extra.sh: Change default port from 5679 to 3003
- debugger/dap_debug_service.ts: Update default port in code and docs
- debugger/README.md: Update port documentation
- debugger/test_debug_service.ts: Update test URLs
- docker/test_windmill_extra.ts: Update test configuration
- .github/workflows/publish_extra.yml: Update test container ports
- frontend/src/lib/components/debug/*: Update frontend examples and defaults
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add indexer crate and files
* POC searcher
incomplete schema
only indexes at startup
* POC search component frontend
* Demo of the frontend element
* add Results and Args as text
* minimal functionality
* Make jump to scripts by name
also flows and apps
* Add button on sidebar to open search
* Update lock on indexer after merge
* Make arrow key navigation compatible with scrol
* Show empty result screen and log as a coming feat
* Add summary to script searchable items
* Catch `parts is undefined` error (uFuzzy)
* Index refreshing using tokio interval
* Fix JobLoader workspace being wrongly defined
* Fix click outside
* Add debouncing for completed run search
* Binary mode working + job index tracker
* Warning for no license + fix height scrollbars on content search
* Make it compile without EE files
* remove panic to use errors
* Move global search
* Cleanup UI, no more tab switcher but clear placeholders and actions
* Add tantivy feature flag for windmill-api
* Rework indexer mode
* Mac compatibility for shortcut
* Update test for new run_server
* Prepare sqlx
* Mac compatibility
* Fix openapi yaml
* Fix frontend
* Frontend api fix
* Update docker-compose.yml and caddyfile
With the (by default deactivated) container and reverse proxy to use the
windmill indexer
* fix feature flag for tests
* fix feature falg for running tests
* fix feature flag for running tests
* Make content search use search modal instead
* Add tantivy feature to ee build steps
* Remove old Content search
* change volume location for indexer
* Update dependencies
* Prepare sqlx
* Uncomment line on docker compose
* Add line between input and results
* Update ee repo ref
Ruben Fiszel
wendrul
HugoCasa
Guillaume Bouvignies
Ruben Fiszel
Mike