Files
windmill/docker/entrypoint-caddy.sh
Ruben Fiszel 2f6c35b15b fix(self-host): unbreak self-hosted Caddy after the caddy-l4 syntax change (#10156)
* fix(self-host): accept pre-2.11 Caddyfiles in the caddy-l4 image

The Caddyfile is a bind-mounted file the user owns, so `docker compose pull`
updates the image but never their config. #10106 and #10113 changed the syntax
the image requires (native caddy-l4 `route { proxy { upstream } }`, and a
non-empty `bind`), which strands every existing self-host on their next pull:

  Error: adapting config using caddyfile: parsing caddyfile tokens for 'layer4':
  wrong argument count or unexpected line ending after 'proxy', at line 4

Normalize legacy Caddyfiles in the entrypoint instead. Only rewrite when the
config cannot be used as-is, and on any failure exec caddy against the user's
original file so it reports a real error against what they wrote.

The bind rewrite is not cosmetic: an empty `bind {$ADDRESS}` adapts and
validates cleanly on caddy >= 2.9 but drops the whole HTTP site, so a
syntax-only shim would trade a restart loop for a container that boots clean
and serves nothing on :80.

The reference for correctness is the image published before #10106
(sha-989c9e6): whatever it adapts today is what self-hosters run, so the shim
must reproduce it byte for byte. docker/test-caddy-compat.sh asserts that over
five legacy variants, plus the :80 listener under an unset ADDRESS, every
--config spelling, relative and glob imports, and the no-op on the current
Caddyfile.

Details worth knowing:
- `to a b` becomes one `upstream` per address; `upstream a b` would be a single
  upstream with two dials, which is a different load-balancing topology.
- The rewrite lands next to the original, because caddy resolves `import`
  relative to the importing file and a glob import would otherwise silently
  expand to nothing.
- The image has no ENTRYPOINT and CMD ["caddy", ...], so an existing
  `command:` override starts with a `caddy` token the entrypoint absorbs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(self-host): route ws_mp and ws_debug to the extra gateway

reverse_proxy only reads its first argument as a matcher, so

  reverse_proxy /ws/* /ws_mp/* /ws_debug/* http://windmill_extra:3000

adapts to a single /ws/* route whose upstreams are `ws_mp/*:80`,
`ws_debug/*:80` and `windmill_extra:3000`. LSP therefore round-robins across
two garbage hostnames and connects only one time in three, while /ws_mp/* and
/ws_debug/* match no route at all and fall through to windmill_server:8000.

Use a named matcher so all three paths reach the gateway. Verified with traffic
against separate windmill_server and windmill_extra backends: before, /ws/lsp
fails and /ws_mp/room reaches windmill_server; after, all three reach the
gateway with the path preserved and /user/login still reaches windmill_server.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(self-host): pin the caddy-l4 image to an explicit version

:latest and the bind-mounted ./Caddyfile it has to agree with are updated by
different mechanisms, so they drift. Publish an explicit version alongside
:latest and pin docker-compose.yml to it, so a checkout is self-consistent:
compose, Caddyfile and image version now move together in one commit.

CI fails the build when docker/caddy-l4.version and the docker-compose.yml pin
disagree, and runs the compatibility-shim tests before publishing. The path
filter now covers the entrypoint, the normalizer, the Caddyfile and
docker-compose.yml, so a change to any guarded input actually triggers the
workflow rather than leaving the check unrun.

:latest keeps being published, since existing deployments reference it and that
is how they pick up the compatibility shim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(self-host): make the caddy-l4 version tag publishable before the pin merges

docker-compose.yml pins an exact tag, but the version tag was gated on the
default branch, so the tag only appeared after the pin had already merged.
Between the merge and the build finishing, a fresh `docker compose up -d` off
main fails with "manifest unknown", and a failed build leaves main permanently
referencing an image that does not exist.

Drop the gate so the tag can be published from the branch via
workflow_dispatch before merging the pin. The version is immutable, so
republishing it from main is a no-op, and only pushes to main and manual
dispatch run this workflow, so a branch cannot claim the tag by accident.
:latest stays gated on main.

Also check the version file against the caddy version the Dockerfile pins.
Without it, a caddy bump that forgets the version file publishes a tag naming
the wrong caddy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(self-host): do not log Caddyfile contents from the compat shim

The shim logged a unified diff of the rewrite, which carries three lines of
context around each change. A Caddyfile is user-owned and can hold basic_auth
hashes, proxy Authorization headers or TLS provider tokens, and container logs
are routinely shipped off the host, so normalizing a customized config could
copy secrets into them. Reproduced with a basic_auth bcrypt hash landing in the
log as context around the bind rewrite.

Log the number of rewritten lines and the path to the rewritten file instead.
It sits next to the original, so an operator can diff it themselves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 16:39:33 +02:00

182 lines
6.6 KiB
Bash
Executable File

#!/bin/sh
# The Caddyfile is a bind-mounted file the user owns, so `docker compose pull`
# updates this image but never their config. Caddy >= 2.9 with native caddy-l4
# rejects the syntax we shipped before 2026-07, which would strand every
# existing self-host on a pull. Normalize legacy Caddyfiles here instead.
#
# Rewrite only when the config cannot be used as-is, and never fail closed: on
# any failure, exec caddy against the user's original file so it reports a real
# error against what they wrote. A shim that exits, or that claims a rewrite it
# did not apply, is worse than no shim at all.
set -eu
NORMALIZE_AWK=/usr/local/lib/caddy-compat/normalize.awk
NORMALIZED_DIR=/tmp/caddy-compat
DOCS_HINT="https://github.com/windmill-labs/windmill/blob/main/Caddyfile"
log() { echo "caddy-compat: $*" >&2; }
# The image had no ENTRYPOINT and CMD ["caddy", "run", ...], so an existing
# `command:` override in a user's compose file starts with `caddy`. Accept it
# with or without that leading token.
if [ "${1:-}" = "caddy" ]; then
shift
fi
# caddy accepts --config X, --config=X, -c X, -c=X and -cX. Record which form
# was used and at which position, so the rewritten path can be substituted back
# in the same form rather than by matching on the string.
config=""
config_idx=0
config_form=""
envfile=""
prev=""
i=0
for arg in "$@"; do
i=$((i + 1))
case "$arg" in
--config=*) config="${arg#--config=}"; config_idx=$i; config_form="eq-long" ;;
-c=*) config="${arg#-c=}"; config_idx=$i; config_form="eq-short" ;;
-c?*) config="${arg#-c}"; config_idx=$i; config_form="attached" ;;
esac
case "$prev" in
--config | -c) config="$arg"; config_idx=$i; config_form="space" ;;
--envfile) envfile="$arg" ;;
esac
case "$arg" in
--envfile=*) envfile="${arg#--envfile=}" ;;
esac
prev="$arg"
done
# With no explicit --config, caddy resolves its own default relative to the
# working directory. Guessing at that path risks inspecting a different file
# than the one caddy will load, so leave it alone.
if [ "$config_idx" -eq 0 ] || [ ! -r "$config" ]; then
exec caddy "$@"
fi
# The shim's own validate has to see the same env expansion caddy will.
validate_config() {
if [ -n "$envfile" ]; then
caddy validate --config "$1" --adapter caddyfile --envfile "$envfile" 2>&1
else
caddy validate --config "$1" --adapter caddyfile 2>&1
fi
}
original_err=""
normalized=""
dump_failure() {
log "---------------- caddy-l4 compatibility shim ----------------"
log "Your Caddyfile could not be used as-is, and the compatibility"
log "rewrite did not produce a valid config either. Caddy will now"
log "run against your original file and report its own error below."
log ""
log "Config: $config"
log "Normalized: ${normalized:-<not produced>}"
log "Reference: $DOCS_HINT"
if [ -n "$original_err" ]; then
log ""
log "Error validating your Caddyfile as written:"
echo "$original_err" | sed 's/^/caddy-compat: /' >&2
fi
if [ -n "${1:-}" ]; then
log ""
log "Error from the compatibility rewrite:"
echo "$1" | sed 's/^/caddy-compat: /' >&2
fi
log ""
log "Most likely fix: replace your Caddyfile with the current one from"
log "the Windmill repo, then re-apply your customizations on top."
log "-------------------------------------------------------------"
}
# A bare `bind {$ADDRESS}` adapts and validates cleanly, it just silently drops
# the HTTP site at runtime, so validate alone cannot decide this.
needs_rewrite=0
if ! original_err=$(validate_config "$config"); then
needs_rewrite=1
log "'$config' is not valid for this caddy version, attempting compatibility rewrite"
else
original_err=""
fi
if grep -Eq '^[[:space:]]*bind[[:space:]]+\{\$ADDRESS\}[[:space:]]*$' "$config"; then
needs_rewrite=1
log "'$config' has a bare 'bind {\$ADDRESS}', which caddy >= 2.9 treats as an"
log "empty bind and drops the entire HTTP site; attempting compatibility rewrite"
fi
if [ "$needs_rewrite" -eq 0 ]; then
exec caddy "$@"
fi
# caddy resolves `import` relative to the directory of the file containing it,
# so the rewritten file has to stay next to the original. Falling back to a
# different directory would make an exact import fail and, worse, make a glob
# import silently match nothing and drop whatever it pulled in.
config_dir=$(dirname "$config")
sibling="$config_dir/.caddy-compat.Caddyfile"
# `touch`, not `: >`: a redirection failure on a special builtin is fatal to the
# shell even inside an `if`, which would kill the shim instead of falling
# through to caddy.
if touch "$sibling" 2>/dev/null; then
normalized="$sibling"
elif grep -Eq '^[[:space:]]*import[[:space:]]' "$config"; then
log "'$config_dir' is not writable and '$config' uses 'import', whose relative"
log "paths would not resolve from anywhere else. Refusing to rewrite rather than"
log "silently drop imported config."
dump_failure ""
exec caddy "$@"
elif mkdir -p "$NORMALIZED_DIR" 2>/dev/null; then
normalized="$NORMALIZED_DIR/Caddyfile"
else
log "nowhere writable to put the rewritten config, skipping the rewrite"
dump_failure ""
exec caddy "$@"
fi
if ! awk_err=$(awk -f "$NORMALIZE_AWK" "$config" 2>&1 >"$normalized"); then
dump_failure "$awk_err"
exec caddy "$@"
fi
if ! normalized_err=$(validate_config "$normalized"); then
dump_failure "$normalized_err"
exec caddy "$@"
fi
# Report a count and a path, never the config itself: a Caddyfile can hold
# basic_auth hashes, proxy Authorization headers or TLS provider tokens, and
# container logs are routinely shipped off the host.
# busybox diff defaults to unified output while GNU diff defaults to normal, so
# match changed lines in either format without matching a +++/--- header.
changed="$(diff "$config" "$normalized" 2>/dev/null | grep -cE '^([<>]|[+-][^+-])' || true)"
log "'$config' uses a deprecated Windmill Caddyfile syntax and was adapted at"
log "startup: $changed line(s) rewritten. Diff it against '$normalized' to see"
log "exactly what changed. Copy the current Caddyfile from $DOCS_HINT"
log "to silence this; the shim will be removed in a later release."
# caddy now reads the rewritten file, so `--watch` follows it rather than the
# bind-mounted original, and `caddy reload --config /etc/caddy/Caddyfile` would
# reload the un-normalized file. Both are non-default; a legacy config that
# needs this shim should be replaced rather than live-edited.
i=0
for arg in "$@"; do
i=$((i + 1))
if [ "$i" -eq "$config_idx" ]; then
case "$config_form" in
space) set -- "$@" "$normalized" ;;
eq-long) set -- "$@" "--config=$normalized" ;;
eq-short) set -- "$@" "-c=$normalized" ;;
attached) set -- "$@" "-c$normalized" ;;
esac
else
set -- "$@" "$arg"
fi
shift
done
exec caddy "$@"