* fix(multiplayer): don't drop client messages during cold-start token verification
`wss.on('connection')` awaits `verifyToken()` before `setupWSConnection()`
attaches the 'message' listener. On a cold process that await includes the
first `/api/debug/jwks` fetch (~30ms on ECS). A y-websocket client sends sync
step 1 the instant the socket opens, and `ws` drops messages emitted with no
listener attached, so that step 1 was lost and never answered with step 2 —
the client's provider never became `synced`.
Buffer messages from the moment the connection is accepted and replay them, in
order, once `setupWSConnection()` has installed its handlers. Rejected
connections drop the buffer and close with the same 4401/4403 codes as before.
Also prefetch the public key at startup when WINDMILL_BASE_URL is set. That is
insurance, not the fix: a connection arriving before the prefetch resolves
still relies on the buffer.
Adds `npm test` in multiplayer/ (node:test, no docker or backend needed) with a
fake JWKS endpoint that answers with a delay, which holds the cold window open
and makes the race deterministic; wired into the existing test_extra CI job.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(multiplayer): cap what an unauthenticated peer can buffer pre-auth
Review follow-up.
The pre-auth buffer was unbounded: `ws` sets no `maxPayload` here and the JWKS
fetch has no timeout, so a peer that never authenticates could stream frames
into memory for as long as `verifyToken` was stalled. Cap it at 32 frames /
1 MiB — a real client only has sync step 1 and its first awareness update in
flight there — and close 1009 past that, dropping what was buffered.
A socket closed during verification (by the peer, or by that cap) is no longer
handed to setupWSConnection: it would be added to `doc.conns` with a 'close'
listener that can never fire.
The startup prefetch's .catch was dead code — getPublicKey() logs its own
failures and resolves to null rather than rejecting.
Test helper: pin REQUIRE_SIGNED_MULTIPLAYER_REQUESTS and BASE_INTERNAL_URL so an
ambient value cannot turn the rejection tests into false passes; bind the JWKS
server on port 0 instead of a released probe port, and retry the spawned server
on EADDRINUSE; destroy still-delayed JWKS responses on teardown, since
server.close() waits for in-flight requests.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(multiplayer): gate the JWKS response instead of delaying it
Review follow-up.
The cold window was held open by a 1500 ms delay on the fake JWKS response, but
that timer started when the startup prefetch reached the fake server, not when
the client sent its first frame. A slow enough machine could load the key before
the client connected, and the race test would then pass without ever exercising
the buffer — a false pass.
The fake JWKS server now parks every response until the test calls release(), so
the server provably holds no key while the client is sending. The race test
releases only after both frames are written to the socket, and asserts the
server has not logged the key as loaded at that point; the flood test never
releases until after the cap has closed the connection.
What is left to wall-clock time is 250 ms for bytes already written to the socket
to cross loopback into an otherwise idle server, rather than a window that had to
cover process startup, connect and handshake.
Also drops the prefetch precondition from the forged-token and flood tests so
each test still maps to one behaviour. Suite runs in ~1.1s instead of ~5.3s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: add gateway reverse proxy for extra services
Add a lightweight Node.js gateway on port 3000 that routes requests
by URL prefix (/ws/*, /ws_mp/*, /ws_debug/*) to the correct backend
service, stripping the prefix before forwarding. This allows all
extra services to be accessed through a single port.
Also makes the multiplayer server more tolerant by generically
stripping /ws_mp/ prefix on HTTP requests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: enable gateway by default for extra services
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: add REMOTE_EXTRA env var for unified extra services proxy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: make gateway port configurable via PORT env var
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: simplify Caddyfile extra services routing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix debugger HTTP health endpoint to also match /ws_debug/health
(ingress forwards the full path, not just /health)
- Add request logging to all three extra services (LSP, multiplayer,
debugger) for HTTP and WebSocket ping/upgrade events
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add ws_base_url instance setting to override WebSocket base URL
Allow deployments behind reverse proxies to route WebSocket traffic
(LSP, debugger, multiplayer) to a different host/port than the main
frontend via a new instance setting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: move ws_base_url to Advanced section with toggle and connectivity test
- Move setting from Core to Advanced > WebSocket section
- Render as toggle "Custom websocket base url from frontend to
multiplayer/lsp/debugger" with conditional URL text field
- Add Test connectivity button (always visible) that checks HTTP health
and WebSocket ping for all three services (LSP, Multiplayer, Debugger)
- Add /ws/ping and /ws/health endpoints to LSP service
- Add /ws_mp/health HTTP and __ping__ WS handlers to multiplayer service
- Add /ping WS handler to debugger service
- Add CORS headers to health endpoints for cross-origin testing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: toggle enabled check and testWs promise resolution
- Fix enabled derived to check only for null (not empty string),
otherwise the toggle never turns on since toggleEnabled sets ''
- Fix testWs onclose handler to resolve(false) so the promise
doesn't hang if the server closes without sending a message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make connectivity test work with existing services
- HTTP test: accept plain text "ok"/"okay" (old services) in addition
to JSON {"status": "ok"} (new services), reject HTML (SPA fallback)
- WS test: resolve on onopen (connection established) instead of
waiting for a specific pong message, so the test works even with
services that don't have the new /ping handler yet
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Alexander PetricandClaude Opus 5.5
Ruben FiszelandClaude Opus 4.6