* fix(multiplayer): don't drop client messages during cold-start token verification
`wss.on('connection')` awaits `verifyToken()` before `setupWSConnection()`
attaches the 'message' listener. On a cold process that await includes the
first `/api/debug/jwks` fetch (~30ms on ECS). A y-websocket client sends sync
step 1 the instant the socket opens, and `ws` drops messages emitted with no
listener attached, so that step 1 was lost and never answered with step 2 —
the client's provider never became `synced`.
Buffer messages from the moment the connection is accepted and replay them, in
order, once `setupWSConnection()` has installed its handlers. Rejected
connections drop the buffer and close with the same 4401/4403 codes as before.
Also prefetch the public key at startup when WINDMILL_BASE_URL is set. That is
insurance, not the fix: a connection arriving before the prefetch resolves
still relies on the buffer.
Adds `npm test` in multiplayer/ (node:test, no docker or backend needed) with a
fake JWKS endpoint that answers with a delay, which holds the cold window open
and makes the race deterministic; wired into the existing test_extra CI job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(multiplayer): cap what an unauthenticated peer can buffer pre-auth
Review follow-up.
The pre-auth buffer was unbounded: `ws` sets no `maxPayload` here and the JWKS
fetch has no timeout, so a peer that never authenticates could stream frames
into memory for as long as `verifyToken` was stalled. Cap it at 32 frames /
1 MiB — a real client only has sync step 1 and its first awareness update in
flight there — and close 1009 past that, dropping what was buffered.
A socket closed during verification (by the peer, or by that cap) is no longer
handed to setupWSConnection: it would be added to `doc.conns` with a 'close'
listener that can never fire.
The startup prefetch's .catch was dead code — getPublicKey() logs its own
failures and resolves to null rather than rejecting.
Test helper: pin REQUIRE_SIGNED_MULTIPLAYER_REQUESTS and BASE_INTERNAL_URL so an
ambient value cannot turn the rejection tests into false passes; bind the JWKS
server on port 0 instead of a released probe port, and retry the spawned server
on EADDRINUSE; destroy still-delayed JWKS responses on teardown, since
server.close() waits for in-flight requests.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(multiplayer): gate the JWKS response instead of delaying it
Review follow-up.
The cold window was held open by a 1500 ms delay on the fake JWKS response, but
that timer started when the startup prefetch reached the fake server, not when
the client sent its first frame. A slow enough machine could load the key before
the client connected, and the race test would then pass without ever exercising
the buffer — a false pass.
The fake JWKS server now parks every response until the test calls release(), so
the server provably holds no key while the client is sending. The race test
releases only after both frames are written to the socket, and asserts the
server has not logged the key as loaded at that point; the flood test never
releases until after the cap has closed the connection.
What is left to wall-clock time is 250 ms for bytes already written to the socket
to cross loopback into an otherwise idle server, rather than a window that had to
cover process startup, connect and handshake.
Also drops the prefetch precondition from the forged-token and flood tests so
each test still maps to one behaviour. Suite runs in ~1.1s instead of ~5.3s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DockerfileExtra copied multiplayer/package.json alone and ran `npm install`,
so the image shipped whatever npm resolved at build time and the committed
multiplayer/package-lock.json described nothing that ran in production:
Dependabot alerts on it were meaningless either way. Copy the lockfile and
install with `npm ci --omit=dev` so the image is reproducible and the lock is
the source of truth for what ships.
The lock is refreshed with `npm update <dep>` per direct dependency, which
moves each to the newest version inside its existing caret range without
touching package.json. That is exactly what a lockfile-less `npm install`
resolves today (verified: a fresh `npm install --package-lock-only` from the
same package.json produces identical versions), so the image does not regress:
ws 8.19.0 -> 8.21.3 (covers the open alerts on ws < 8.21.0)
y-websocket 3.0.0 -> 3.1.0
yjs 13.6.29 -> 13.6.32
lib0 0.2.117, y-protocols 1.0.7, isomorphic.js 0.2.5 unchanged
package.json has no devDependencies and the lock has no `dev: true` entries,
so `--omit=dev` changes nothing today and only guards against future ones.
Validation:
- `npm ci` on the pre-update lock and `npm ci --omit=dev` on the updated lock
both succeed; `node --check server.mjs gateway.mjs` passes.
- Smoke start: server.mjs listens on PORT=3999 and logs "Multiplayer server
running"; gateway.mjs listens on PORT=3998 and logs its route table.
- The same COPY/RUN lines built on node:22-slim with the repo root as build
context (the context build-extra-image.yml uses) install the six locked
packages and pass `node --check`. A full DockerfileExtra build was skipped:
it is a single stage on an uncached multi-GB base image.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat: add gateway reverse proxy for extra services
Add a lightweight Node.js gateway on port 3000 that routes requests
by URL prefix (/ws/*, /ws_mp/*, /ws_debug/*) to the correct backend
service, stripping the prefix before forwarding. This allows all
extra services to be accessed through a single port.
Also makes the multiplayer server more tolerant by generically
stripping /ws_mp/ prefix on HTTP requests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: enable gateway by default for extra services
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: add REMOTE_EXTRA env var for unified extra services proxy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: make gateway port configurable via PORT env var
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: simplify Caddyfile extra services routing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix debugger HTTP health endpoint to also match /ws_debug/health
(ingress forwards the full path, not just /health)
- Add request logging to all three extra services (LSP, multiplayer,
debugger) for HTTP and WebSocket ping/upgrade events
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add ws_base_url instance setting to override WebSocket base URL
Allow deployments behind reverse proxies to route WebSocket traffic
(LSP, debugger, multiplayer) to a different host/port than the main
frontend via a new instance setting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: move ws_base_url to Advanced section with toggle and connectivity test
- Move setting from Core to Advanced > WebSocket section
- Render as toggle "Custom websocket base url from frontend to
multiplayer/lsp/debugger" with conditional URL text field
- Add Test connectivity button (always visible) that checks HTTP health
and WebSocket ping for all three services (LSP, Multiplayer, Debugger)
- Add /ws/ping and /ws/health endpoints to LSP service
- Add /ws_mp/health HTTP and __ping__ WS handlers to multiplayer service
- Add /ping WS handler to debugger service
- Add CORS headers to health endpoints for cross-origin testing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: toggle enabled check and testWs promise resolution
- Fix enabled derived to check only for null (not empty string),
otherwise the toggle never turns on since toggleEnabled sets ''
- Fix testWs onclose handler to resolve(false) so the promise
doesn't hang if the server closes without sending a message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make connectivity test work with existing services
- HTTP test: accept plain text "ok"/"okay" (old services) in addition
to JSON {"status": "ok"} (new services), reject HTML (SPA fallback)
- WS test: resolve on onopen (connection established) instead of
waiting for a specific pong message, so the test works even with
services that don't have the new /ping handler yet
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Alexander PetricandClaude Opus 5.5
Ruben FiszelandClaude Opus 4.6