mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 08:01:26 +00:00
fix(self-host): unbreak self-hosted Caddy after the caddy-l4 syntax change (#10156)
* fix(self-host): accept pre-2.11 Caddyfiles in the caddy-l4 image The Caddyfile is a bind-mounted file the user owns, so `docker compose pull` updates the image but never their config. #10106 and #10113 changed the syntax the image requires (native caddy-l4 `route { proxy { upstream } }`, and a non-empty `bind`), which strands every existing self-host on their next pull: Error: adapting config using caddyfile: parsing caddyfile tokens for 'layer4': wrong argument count or unexpected line ending after 'proxy', at line 4 Normalize legacy Caddyfiles in the entrypoint instead. Only rewrite when the config cannot be used as-is, and on any failure exec caddy against the user's original file so it reports a real error against what they wrote. The bind rewrite is not cosmetic: an empty `bind {$ADDRESS}` adapts and validates cleanly on caddy >= 2.9 but drops the whole HTTP site, so a syntax-only shim would trade a restart loop for a container that boots clean and serves nothing on :80. The reference for correctness is the image published before #10106 (sha-989c9e6): whatever it adapts today is what self-hosters run, so the shim must reproduce it byte for byte. docker/test-caddy-compat.sh asserts that over five legacy variants, plus the :80 listener under an unset ADDRESS, every --config spelling, relative and glob imports, and the no-op on the current Caddyfile. Details worth knowing: - `to a b` becomes one `upstream` per address; `upstream a b` would be a single upstream with two dials, which is a different load-balancing topology. - The rewrite lands next to the original, because caddy resolves `import` relative to the importing file and a glob import would otherwise silently expand to nothing. - The image has no ENTRYPOINT and CMD ["caddy", ...], so an existing `command:` override starts with a `caddy` token the entrypoint absorbs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(self-host): route ws_mp and ws_debug to the extra gateway reverse_proxy only reads its first argument as a matcher, so reverse_proxy /ws/* /ws_mp/* /ws_debug/* http://windmill_extra:3000 adapts to a single /ws/* route whose upstreams are `ws_mp/*:80`, `ws_debug/*:80` and `windmill_extra:3000`. LSP therefore round-robins across two garbage hostnames and connects only one time in three, while /ws_mp/* and /ws_debug/* match no route at all and fall through to windmill_server:8000. Use a named matcher so all three paths reach the gateway. Verified with traffic against separate windmill_server and windmill_extra backends: before, /ws/lsp fails and /ws_mp/room reaches windmill_server; after, all three reach the gateway with the path preserved and /user/login still reaches windmill_server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(self-host): pin the caddy-l4 image to an explicit version :latest and the bind-mounted ./Caddyfile it has to agree with are updated by different mechanisms, so they drift. Publish an explicit version alongside :latest and pin docker-compose.yml to it, so a checkout is self-consistent: compose, Caddyfile and image version now move together in one commit. CI fails the build when docker/caddy-l4.version and the docker-compose.yml pin disagree, and runs the compatibility-shim tests before publishing. The path filter now covers the entrypoint, the normalizer, the Caddyfile and docker-compose.yml, so a change to any guarded input actually triggers the workflow rather than leaving the check unrun. :latest keeps being published, since existing deployments reference it and that is how they pick up the compatibility shim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(self-host): make the caddy-l4 version tag publishable before the pin merges docker-compose.yml pins an exact tag, but the version tag was gated on the default branch, so the tag only appeared after the pin had already merged. Between the merge and the build finishing, a fresh `docker compose up -d` off main fails with "manifest unknown", and a failed build leaves main permanently referencing an image that does not exist. Drop the gate so the tag can be published from the branch via workflow_dispatch before merging the pin. The version is immutable, so republishing it from main is a no-op, and only pushes to main and manual dispatch run this workflow, so a branch cannot claim the tag by accident. :latest stays gated on main. Also check the version file against the caddy version the Dockerfile pins. Without it, a caddy bump that forgets the version file publishes a tag naming the wrong caddy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(self-host): do not log Caddyfile contents from the compat shim The shim logged a unified diff of the rewrite, which carries three lines of context around each change. A Caddyfile is user-owned and can hold basic_auth hashes, proxy Authorization headers or TLS provider tokens, and container logs are routinely shipped off the host, so normalizing a customized config could copy secrets into them. Reproduced with a basic_auth bcrypt hash landing in the log as context around the bind rewrite. Log the number of rewritten lines and the path to the rewritten file instead. It sits next to the original, so an operator can diff it themselves. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ruben Fiszel
GitHub
@@ -10,6 +10,14 @@ on:
|
||||
- main
|
||||
paths:
|
||||
- docker/DockerfileCaddyL4
|
||||
- docker/entrypoint-caddy.sh
|
||||
- docker/caddy-compat-normalize.awk
|
||||
- docker/caddy-l4.version
|
||||
- docker/test-caddy-compat.sh
|
||||
- Caddyfile
|
||||
# The version check below reads the pin out of docker-compose.yml, so a
|
||||
# compose-only bump has to trigger this workflow or the check never runs.
|
||||
- docker-compose.yml
|
||||
- .github/workflows/build-caddy-l4-image.yml
|
||||
|
||||
permissions: write-all
|
||||
@@ -20,6 +28,35 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
# docker-compose.yml pins an exact tag, and the Caddyfile it must agree
|
||||
# with lives in the same checkout. Fail the build rather than publish a
|
||||
# version nothing references, which is how :latest drifted from the
|
||||
# Caddyfile in the first place.
|
||||
- name: Resolve and check image version
|
||||
id: version
|
||||
run: |
|
||||
set -euo pipefail
|
||||
version="$(tr -d '[:space:]' < docker/caddy-l4.version)"
|
||||
pinned="$(grep -oE 'caddy-l4:[^[:space:]"]+' docker-compose.yml | head -1 | cut -d: -f2-)"
|
||||
caddy="$(grep -m1 -oE '^FROM caddy:[0-9]+\.[0-9]+\.[0-9]+' docker/DockerfileCaddyL4 | cut -d: -f2)"
|
||||
if [ "$version" != "$pinned" ]; then
|
||||
echo "docker/caddy-l4.version is '$version' but docker-compose.yml pins '$pinned'" >&2
|
||||
echo "Bump both together." >&2
|
||||
exit 1
|
||||
fi
|
||||
# Otherwise a caddy bump that forgets the version file publishes a tag
|
||||
# that names the wrong caddy.
|
||||
case "$version" in
|
||||
"$caddy"-*) ;;
|
||||
*)
|
||||
echo "docker/caddy-l4.version is '$version' but the Dockerfile pins caddy '$caddy'" >&2
|
||||
echo "The version must be <caddy-version>-<revision>." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
@@ -29,8 +66,22 @@ jobs:
|
||||
tags: |
|
||||
type=sha
|
||||
type=ref,event=branch
|
||||
# Not gated on the default branch: docker-compose.yml pins this tag,
|
||||
# so it has to be publishable from a branch (workflow_dispatch)
|
||||
# before the pin merges, or main would reference a tag that does not
|
||||
# exist yet. The version is immutable, so republishing from main is
|
||||
# a no-op. Only branch pushes to main and manual dispatch run this
|
||||
# workflow, so a branch cannot claim the tag by accident.
|
||||
type=raw,value=${{ steps.version.outputs.version }}
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
# The shim rewrites config a self-hoster never sees, so a silent
|
||||
# regression here strands them on a restart loop or a dead :80.
|
||||
- name: Test the legacy-Caddyfile compatibility shim
|
||||
run: |
|
||||
docker build -f docker/DockerfileCaddyL4 -t caddy-l4:ci ./docker
|
||||
docker/test-caddy-compat.sh caddy-l4:ci
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
|
||||
@@ -16,8 +16,12 @@
|
||||
# site, silently disabling the HTTP proxy while the :25 layer4 listener stays up.
|
||||
bind {$ADDRESS:0.0.0.0 ::}
|
||||
|
||||
# Extra services: LSP, Multiplayer, Debugger (windmill_extra gateway)
|
||||
reverse_proxy /ws/* /ws_mp/* /ws_debug/* http://windmill_extra:3000
|
||||
# Extra services: LSP, Multiplayer, Debugger (windmill_extra gateway).
|
||||
# reverse_proxy only reads its first argument as a matcher, so listing
|
||||
# several paths inline turns the rest into upstream addresses. The paths
|
||||
# have to go through a named matcher.
|
||||
@extra path /ws/* /ws_mp/* /ws_debug/*
|
||||
reverse_proxy @extra http://windmill_extra:3000
|
||||
|
||||
# Search indexer, Enterprise Edition (windmill_indexer:8002)
|
||||
# reverse_proxy /api/srch/* http://windmill_indexer:8002
|
||||
|
||||
+4
-1
@@ -190,7 +190,10 @@ services:
|
||||
logging: *default-logging
|
||||
|
||||
caddy:
|
||||
image: ghcr.io/windmill-labs/caddy-l4:latest
|
||||
# Pinned: this image and the ./Caddyfile next to it are version-coupled, so
|
||||
# they have to move together. Bump docker/caddy-l4.version in the same
|
||||
# commit as any Caddyfile change.
|
||||
image: ghcr.io/windmill-labs/caddy-l4:2.11.4-1
|
||||
restart: unless-stopped
|
||||
# Configure the mounted Caddyfile and the exposed ports or use another reverse proxy if needed
|
||||
volumes:
|
||||
|
||||
@@ -10,3 +10,14 @@ RUN xcaddy build \
|
||||
FROM caddy:2.11.4-alpine
|
||||
|
||||
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
|
||||
|
||||
COPY caddy-compat-normalize.awk /usr/local/lib/caddy-compat/normalize.awk
|
||||
COPY entrypoint-caddy.sh /usr/local/bin/entrypoint-caddy.sh
|
||||
RUN chmod +x /usr/local/bin/entrypoint-caddy.sh
|
||||
|
||||
# The base image has no ENTRYPOINT and puts `caddy` in CMD, so setting an
|
||||
# ENTRYPOINT here both resets CMD (restated below) and changes what a user's
|
||||
# existing `command:` override means. The entrypoint absorbs a leading `caddy`
|
||||
# token so both forms keep working.
|
||||
ENTRYPOINT ["/usr/local/bin/entrypoint-caddy.sh"]
|
||||
CMD ["run", "--config", "/etc/caddy/Caddyfile", "--adapter", "caddyfile"]
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# Rewrites a pre-2.11 Windmill Caddyfile to the form caddy >= 2.9 with native
|
||||
# caddy-l4 requires. Two rewrites, both no-ops on a current Caddyfile so this is
|
||||
# idempotent:
|
||||
#
|
||||
# layer4 { :25 { proxy { to X } } } -> layer4 { :25 { route { proxy { upstream X } } } }
|
||||
# bind {$ADDRESS} -> bind {$ADDRESS:0.0.0.0 ::}
|
||||
#
|
||||
# The second is not cosmetic: an empty bind makes caddy >= 2.9 drop the whole
|
||||
# HTTP site, so without it a legacy Caddyfile boots clean and serves nothing on
|
||||
# :80 while :25 keeps working.
|
||||
#
|
||||
# Only the layer4 block and that one literal bind token are touched; a proxy
|
||||
# already nested in a route block is left alone.
|
||||
|
||||
BEGIN { depth = 0; inL4 = 0; l4depth = 0; inRoute = 0; rdepth = 0; inProxy = 0; pdepth = 0 }
|
||||
{
|
||||
line = $0
|
||||
|
||||
if (line ~ /^[[:space:]]*bind[[:space:]]+\{\$ADDRESS\}[[:space:]]*$/) {
|
||||
sub(/\{\$ADDRESS\}/, "{$ADDRESS:0.0.0.0 ::}", line)
|
||||
print line
|
||||
next
|
||||
}
|
||||
|
||||
if (!inL4 && line ~ /^[[:space:]]*layer4[[:space:]]*\{/) { inL4 = 1; l4depth = depth; print; depth++; next }
|
||||
if (inL4 && line ~ /^[[:space:]]*route[[:space:]]*\{/) { inRoute = 1; rdepth = depth; print; depth++; next }
|
||||
|
||||
if (inL4 && !inRoute && !inProxy && line ~ /^[[:space:]]*proxy[[:space:]]*\{[[:space:]]*$/) {
|
||||
match(line, /^[[:space:]]*/); ind = substr(line, 1, RLENGTH)
|
||||
print ind "route {"
|
||||
print ind "\tproxy {"
|
||||
inProxy = 1; pdepth = depth; depth++
|
||||
next
|
||||
}
|
||||
|
||||
# `to a b` is two load-balanced upstreams, whereas `upstream a b` is a single
|
||||
# upstream with two dial addresses. Emit one `upstream` per address to keep
|
||||
# the adapted JSON identical.
|
||||
if (inProxy && line ~ /^[[:space:]]*to[[:space:]]+/) {
|
||||
match(line, /^[[:space:]]*/); ind = substr(line, 1, RLENGTH)
|
||||
nf = split(line, parts, /[[:space:]]+/)
|
||||
for (i = 1; i <= nf; i++) if (parts[i] == "to") break
|
||||
for (j = i + 1; j <= nf; j++) if (parts[j] != "") print "\t" ind "upstream " parts[j]
|
||||
next
|
||||
}
|
||||
|
||||
n = gsub(/\{/, "{", line); m = gsub(/\}/, "}", line); depth += n - m
|
||||
|
||||
if (inProxy && depth == pdepth) {
|
||||
match(line, /^[[:space:]]*/); ind = substr(line, 1, RLENGTH)
|
||||
print "\t" line
|
||||
print ind "}"
|
||||
inProxy = 0
|
||||
next
|
||||
}
|
||||
|
||||
if (inRoute && depth <= rdepth) inRoute = 0
|
||||
if (inL4 && depth <= l4depth) inL4 = 0
|
||||
|
||||
if (inProxy) print "\t" line; else print line
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
2.11.4-1
|
||||
Executable
+181
@@ -0,0 +1,181 @@
|
||||
#!/bin/sh
|
||||
# The Caddyfile is a bind-mounted file the user owns, so `docker compose pull`
|
||||
# updates this image but never their config. Caddy >= 2.9 with native caddy-l4
|
||||
# rejects the syntax we shipped before 2026-07, which would strand every
|
||||
# existing self-host on a pull. Normalize legacy Caddyfiles here instead.
|
||||
#
|
||||
# Rewrite only when the config cannot be used as-is, and never fail closed: on
|
||||
# any failure, exec caddy against the user's original file so it reports a real
|
||||
# error against what they wrote. A shim that exits, or that claims a rewrite it
|
||||
# did not apply, is worse than no shim at all.
|
||||
set -eu
|
||||
|
||||
NORMALIZE_AWK=/usr/local/lib/caddy-compat/normalize.awk
|
||||
NORMALIZED_DIR=/tmp/caddy-compat
|
||||
DOCS_HINT="https://github.com/windmill-labs/windmill/blob/main/Caddyfile"
|
||||
|
||||
log() { echo "caddy-compat: $*" >&2; }
|
||||
|
||||
# The image had no ENTRYPOINT and CMD ["caddy", "run", ...], so an existing
|
||||
# `command:` override in a user's compose file starts with `caddy`. Accept it
|
||||
# with or without that leading token.
|
||||
if [ "${1:-}" = "caddy" ]; then
|
||||
shift
|
||||
fi
|
||||
|
||||
# caddy accepts --config X, --config=X, -c X, -c=X and -cX. Record which form
|
||||
# was used and at which position, so the rewritten path can be substituted back
|
||||
# in the same form rather than by matching on the string.
|
||||
config=""
|
||||
config_idx=0
|
||||
config_form=""
|
||||
envfile=""
|
||||
prev=""
|
||||
i=0
|
||||
for arg in "$@"; do
|
||||
i=$((i + 1))
|
||||
case "$arg" in
|
||||
--config=*) config="${arg#--config=}"; config_idx=$i; config_form="eq-long" ;;
|
||||
-c=*) config="${arg#-c=}"; config_idx=$i; config_form="eq-short" ;;
|
||||
-c?*) config="${arg#-c}"; config_idx=$i; config_form="attached" ;;
|
||||
esac
|
||||
case "$prev" in
|
||||
--config | -c) config="$arg"; config_idx=$i; config_form="space" ;;
|
||||
--envfile) envfile="$arg" ;;
|
||||
esac
|
||||
case "$arg" in
|
||||
--envfile=*) envfile="${arg#--envfile=}" ;;
|
||||
esac
|
||||
prev="$arg"
|
||||
done
|
||||
|
||||
# With no explicit --config, caddy resolves its own default relative to the
|
||||
# working directory. Guessing at that path risks inspecting a different file
|
||||
# than the one caddy will load, so leave it alone.
|
||||
if [ "$config_idx" -eq 0 ] || [ ! -r "$config" ]; then
|
||||
exec caddy "$@"
|
||||
fi
|
||||
|
||||
# The shim's own validate has to see the same env expansion caddy will.
|
||||
validate_config() {
|
||||
if [ -n "$envfile" ]; then
|
||||
caddy validate --config "$1" --adapter caddyfile --envfile "$envfile" 2>&1
|
||||
else
|
||||
caddy validate --config "$1" --adapter caddyfile 2>&1
|
||||
fi
|
||||
}
|
||||
|
||||
original_err=""
|
||||
normalized=""
|
||||
|
||||
dump_failure() {
|
||||
log "---------------- caddy-l4 compatibility shim ----------------"
|
||||
log "Your Caddyfile could not be used as-is, and the compatibility"
|
||||
log "rewrite did not produce a valid config either. Caddy will now"
|
||||
log "run against your original file and report its own error below."
|
||||
log ""
|
||||
log "Config: $config"
|
||||
log "Normalized: ${normalized:-<not produced>}"
|
||||
log "Reference: $DOCS_HINT"
|
||||
if [ -n "$original_err" ]; then
|
||||
log ""
|
||||
log "Error validating your Caddyfile as written:"
|
||||
echo "$original_err" | sed 's/^/caddy-compat: /' >&2
|
||||
fi
|
||||
if [ -n "${1:-}" ]; then
|
||||
log ""
|
||||
log "Error from the compatibility rewrite:"
|
||||
echo "$1" | sed 's/^/caddy-compat: /' >&2
|
||||
fi
|
||||
log ""
|
||||
log "Most likely fix: replace your Caddyfile with the current one from"
|
||||
log "the Windmill repo, then re-apply your customizations on top."
|
||||
log "-------------------------------------------------------------"
|
||||
}
|
||||
|
||||
# A bare `bind {$ADDRESS}` adapts and validates cleanly, it just silently drops
|
||||
# the HTTP site at runtime, so validate alone cannot decide this.
|
||||
needs_rewrite=0
|
||||
if ! original_err=$(validate_config "$config"); then
|
||||
needs_rewrite=1
|
||||
log "'$config' is not valid for this caddy version, attempting compatibility rewrite"
|
||||
else
|
||||
original_err=""
|
||||
fi
|
||||
if grep -Eq '^[[:space:]]*bind[[:space:]]+\{\$ADDRESS\}[[:space:]]*$' "$config"; then
|
||||
needs_rewrite=1
|
||||
log "'$config' has a bare 'bind {\$ADDRESS}', which caddy >= 2.9 treats as an"
|
||||
log "empty bind and drops the entire HTTP site; attempting compatibility rewrite"
|
||||
fi
|
||||
|
||||
if [ "$needs_rewrite" -eq 0 ]; then
|
||||
exec caddy "$@"
|
||||
fi
|
||||
|
||||
# caddy resolves `import` relative to the directory of the file containing it,
|
||||
# so the rewritten file has to stay next to the original. Falling back to a
|
||||
# different directory would make an exact import fail and, worse, make a glob
|
||||
# import silently match nothing and drop whatever it pulled in.
|
||||
config_dir=$(dirname "$config")
|
||||
sibling="$config_dir/.caddy-compat.Caddyfile"
|
||||
# `touch`, not `: >`: a redirection failure on a special builtin is fatal to the
|
||||
# shell even inside an `if`, which would kill the shim instead of falling
|
||||
# through to caddy.
|
||||
if touch "$sibling" 2>/dev/null; then
|
||||
normalized="$sibling"
|
||||
elif grep -Eq '^[[:space:]]*import[[:space:]]' "$config"; then
|
||||
log "'$config_dir' is not writable and '$config' uses 'import', whose relative"
|
||||
log "paths would not resolve from anywhere else. Refusing to rewrite rather than"
|
||||
log "silently drop imported config."
|
||||
dump_failure ""
|
||||
exec caddy "$@"
|
||||
elif mkdir -p "$NORMALIZED_DIR" 2>/dev/null; then
|
||||
normalized="$NORMALIZED_DIR/Caddyfile"
|
||||
else
|
||||
log "nowhere writable to put the rewritten config, skipping the rewrite"
|
||||
dump_failure ""
|
||||
exec caddy "$@"
|
||||
fi
|
||||
|
||||
if ! awk_err=$(awk -f "$NORMALIZE_AWK" "$config" 2>&1 >"$normalized"); then
|
||||
dump_failure "$awk_err"
|
||||
exec caddy "$@"
|
||||
fi
|
||||
|
||||
if ! normalized_err=$(validate_config "$normalized"); then
|
||||
dump_failure "$normalized_err"
|
||||
exec caddy "$@"
|
||||
fi
|
||||
|
||||
# Report a count and a path, never the config itself: a Caddyfile can hold
|
||||
# basic_auth hashes, proxy Authorization headers or TLS provider tokens, and
|
||||
# container logs are routinely shipped off the host.
|
||||
# busybox diff defaults to unified output while GNU diff defaults to normal, so
|
||||
# match changed lines in either format without matching a +++/--- header.
|
||||
changed="$(diff "$config" "$normalized" 2>/dev/null | grep -cE '^([<>]|[+-][^+-])' || true)"
|
||||
log "'$config' uses a deprecated Windmill Caddyfile syntax and was adapted at"
|
||||
log "startup: $changed line(s) rewritten. Diff it against '$normalized' to see"
|
||||
log "exactly what changed. Copy the current Caddyfile from $DOCS_HINT"
|
||||
log "to silence this; the shim will be removed in a later release."
|
||||
|
||||
# caddy now reads the rewritten file, so `--watch` follows it rather than the
|
||||
# bind-mounted original, and `caddy reload --config /etc/caddy/Caddyfile` would
|
||||
# reload the un-normalized file. Both are non-default; a legacy config that
|
||||
# needs this shim should be replaced rather than live-edited.
|
||||
i=0
|
||||
for arg in "$@"; do
|
||||
i=$((i + 1))
|
||||
if [ "$i" -eq "$config_idx" ]; then
|
||||
case "$config_form" in
|
||||
space) set -- "$@" "$normalized" ;;
|
||||
eq-long) set -- "$@" "--config=$normalized" ;;
|
||||
eq-short) set -- "$@" "-c=$normalized" ;;
|
||||
attached) set -- "$@" "-c$normalized" ;;
|
||||
esac
|
||||
else
|
||||
set -- "$@" "$arg"
|
||||
fi
|
||||
shift
|
||||
done
|
||||
|
||||
exec caddy "$@"
|
||||
Executable
+146
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regression tests for the caddy-l4 legacy-Caddyfile compatibility shim.
|
||||
#
|
||||
# The reference for "correct" is the image published before #10106
|
||||
# (REFERENCE_IMAGE): whatever config it adapts today is what existing
|
||||
# self-hosters are running, so the shim must reproduce it byte for byte.
|
||||
#
|
||||
# Usage: docker/test-caddy-compat.sh [image-tag]
|
||||
set -euo pipefail
|
||||
|
||||
IMAGE="${1:-caddy-l4:test}"
|
||||
REFERENCE_IMAGE="${REFERENCE_IMAGE:-ghcr.io/windmill-labs/caddy-l4:sha-989c9e6}"
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
ok() { printf ' \033[32mok\033[0m %s\n' "$1"; pass=$((pass + 1)); }
|
||||
no() { printf ' \033[31mFAIL\033[0m %s\n' "$1"; fail=$((fail + 1)); }
|
||||
|
||||
# SELinux-labelled hosts (Fedora) reject plain bind mounts into containers.
|
||||
MOUNT_OPTS=()
|
||||
if command -v getenforce >/dev/null 2>&1 && [ "$(getenforce 2>/dev/null)" != "Disabled" ]; then
|
||||
MOUNT_OPTS=(--security-opt label=disable)
|
||||
fi
|
||||
|
||||
adapt() { # adapt <image> <caddyfile> [env...]
|
||||
local image="$1" file="$2"
|
||||
shift 2
|
||||
local env_args=()
|
||||
local e
|
||||
for e in "$@"; do env_args+=(-e "$e"); done
|
||||
# The `caddy` prefix is how the pre-#10106 image is invoked (no ENTRYPOINT,
|
||||
# CMD ["caddy", ...]); the shim accepts it too, which this exercises.
|
||||
docker run --rm "${MOUNT_OPTS[@]}" "${env_args[@]}" \
|
||||
-v "$file:/etc/caddy/Caddyfile:ro" "$image" \
|
||||
caddy adapt --config /etc/caddy/Caddyfile 2>/dev/null
|
||||
}
|
||||
|
||||
# The legacy Caddyfile as shipped before #10106, which is what a self-hoster who
|
||||
# has never touched their config still has on disk.
|
||||
cat >"$WORK/legacy.Caddyfile" <<'EOF'
|
||||
{
|
||||
layer4 {
|
||||
:25 {
|
||||
proxy {
|
||||
to windmill_server:2525
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{$BASE_URL} {
|
||||
bind {$ADDRESS}
|
||||
reverse_proxy /ws/* /ws_mp/* /ws_debug/* http://windmill_extra:3000
|
||||
reverse_proxy /* http://windmill_server:8000
|
||||
}
|
||||
EOF
|
||||
|
||||
echo "==> equivalence with the pre-#10106 image (ADDRESS set, so bind is neutral)"
|
||||
docker pull -q "$REFERENCE_IMAGE" >/dev/null 2>&1 || true
|
||||
if ! docker image inspect "$REFERENCE_IMAGE" >/dev/null 2>&1; then
|
||||
echo " skipped: $REFERENCE_IMAGE unavailable"
|
||||
else
|
||||
# Each variant is a plausible user modification of the legacy Caddyfile.
|
||||
make_variant() {
|
||||
case "$1" in
|
||||
stock) cat "$WORK/legacy.Caddyfile" ;;
|
||||
custom_smtp) sed 's|to windmill_server:2525|to mailhog:2626|' "$WORK/legacy.Caddyfile" ;;
|
||||
multi_upstream) sed 's|to windmill_server:2525|to a:2525 b:2525|' "$WORK/legacy.Caddyfile" ;;
|
||||
extra_listener) sed 's|\t\t:25 {|\t\t:587 {\n\t\t\tproxy {\n\t\t\t\tto windmill_server:2525\n\t\t\t}\n\t\t}\n\t\t:25 {|' "$WORK/legacy.Caddyfile" ;;
|
||||
extra_directive) sed 's|reverse_proxy /\*|encode gzip\n\treverse_proxy /*|' "$WORK/legacy.Caddyfile" ;;
|
||||
esac
|
||||
}
|
||||
for name in stock custom_smtp multi_upstream extra_listener extra_directive; do
|
||||
make_variant "$name" >"$WORK/v.Caddyfile"
|
||||
a="$(adapt "$REFERENCE_IMAGE" "$WORK/v.Caddyfile" 'BASE_URL=:80' 'ADDRESS=0.0.0.0')"
|
||||
b="$(adapt "$IMAGE" "$WORK/v.Caddyfile" 'BASE_URL=:80' 'ADDRESS=0.0.0.0')"
|
||||
if [ -n "$a" ] && [ "$a" = "$b" ]; then ok "$name: adapted config identical"; else no "$name: adapted config differs"; fi
|
||||
done
|
||||
fi
|
||||
|
||||
echo "==> the HTTP site must survive an unset ADDRESS (#10113: validate cannot catch this)"
|
||||
listen="$(adapt "$IMAGE" "$WORK/legacy.Caddyfile" 'BASE_URL=:80' | grep -o '"listen":\[[^]]*\]' | head -1)"
|
||||
case "$listen" in
|
||||
*:80*) ok "legacy Caddyfile still binds :80 (${listen})" ;;
|
||||
*) no "legacy Caddyfile dropped the HTTP site (listen=${listen:-none})" ;;
|
||||
esac
|
||||
|
||||
echo "==> the shipped Caddyfile must be untouched by the shim"
|
||||
noise="$(docker run --rm "${MOUNT_OPTS[@]}" -e BASE_URL=':80' \
|
||||
-v "$REPO_ROOT/Caddyfile:/etc/caddy/Caddyfile:ro" "$IMAGE" \
|
||||
adapt --config /etc/caddy/Caddyfile 2>&1 >/dev/null | grep -c 'caddy-compat' || true)"
|
||||
[ "$noise" -eq 0 ] && ok "no rewrite on the current Caddyfile" || no "shim fired on the current Caddyfile"
|
||||
|
||||
echo "==> every --config form must load the rewritten file"
|
||||
check_form() {
|
||||
local label="$1"
|
||||
shift
|
||||
local out
|
||||
out="$(docker run --rm "${MOUNT_OPTS[@]}" -e BASE_URL=':80' \
|
||||
-v "$WORK/legacy.Caddyfile:/etc/caddy/Caddyfile:ro" "$IMAGE" "$@" 2>&1 >/dev/null |
|
||||
grep -E '"file":' | head -1 || true)"
|
||||
case "$out" in
|
||||
*caddy-compat*) ok "$label" ;;
|
||||
*) no "$label (loaded: ${out:-nothing})" ;;
|
||||
esac
|
||||
}
|
||||
check_form '--config PATH' adapt --config /etc/caddy/Caddyfile
|
||||
check_form '--config=PATH' adapt --config=/etc/caddy/Caddyfile
|
||||
check_form '-c PATH' adapt -c /etc/caddy/Caddyfile
|
||||
check_form '-c=PATH' adapt -c=/etc/caddy/Caddyfile
|
||||
check_form '-cPATH' adapt -c/etc/caddy/Caddyfile
|
||||
|
||||
echo "==> relative imports must survive the rewrite"
|
||||
# caddy resolves `import` against the directory of the file holding it, so a
|
||||
# rewrite that relocates the config would break an exact import and make a glob
|
||||
# import silently expand to nothing.
|
||||
mkdir -p "$WORK/withimport"
|
||||
sed 's|reverse_proxy /\* http://windmill_server:8000|import extra.caddy\n\treverse_proxy /* http://windmill_server:8000|' \
|
||||
"$WORK/legacy.Caddyfile" >"$WORK/withimport/Caddyfile"
|
||||
echo 'reverse_proxy /imported/* http://imported_backend:9999' >"$WORK/withimport/extra.caddy"
|
||||
printf 'reverse_proxy /globbed/* http://globbed_backend:9998\n' >"$WORK/withimport/glob-one.caddy"
|
||||
sed -i 's|import extra.caddy|import extra.caddy\n\timport glob-*.caddy|' "$WORK/withimport/Caddyfile"
|
||||
imported="$(docker run --rm "${MOUNT_OPTS[@]}" -e BASE_URL=':80' \
|
||||
-v "$WORK/withimport:/etc/caddy" "$IMAGE" \
|
||||
adapt --config /etc/caddy/Caddyfile 2>/dev/null)"
|
||||
case "$imported" in
|
||||
*imported_backend:9999*) ok "exact import survives" ;;
|
||||
*) no "exact import lost after rewrite" ;;
|
||||
esac
|
||||
case "$imported" in
|
||||
*globbed_backend:9998*) ok "glob import survives" ;;
|
||||
*) no "glob import silently dropped after rewrite" ;;
|
||||
esac
|
||||
|
||||
echo "==> a config the shim cannot fix must still reach caddy's own error"
|
||||
broken="$(docker run --rm "${MOUNT_OPTS[@]}" --read-only -e BASE_URL=':80' \
|
||||
-v "$WORK/legacy.Caddyfile:/etc/caddy/Caddyfile:ro" "$IMAGE" \
|
||||
run --config /etc/caddy/Caddyfile --adapter caddyfile 2>&1 | grep -c '^Error:' || true)"
|
||||
[ "$broken" -ge 1 ] && ok "read-only rootfs still execs caddy and reports its error" || no "read-only rootfs swallowed caddy's error"
|
||||
|
||||
echo
|
||||
echo "passed: $pass failed: $fail"
|
||||
[ "$fail" -eq 0 ]
|
||||
Reference in New Issue
Block a user