* feat: register mounted CA certificates in windmill_extra at startup Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: only claim a CA update when update-ca-certificates can read the mount Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: detect mounted CA certificates the way update-ca-certificates finds them Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
9.5 KiB
Windmill Debug Module
A DAP (Debug Adapter Protocol) implementation for debugging Python and TypeScript/Bun scripts in Windmill's Monaco editor.
Overview
This module provides step-through debugging capabilities with breakpoints, variable inspection, and stack traces. It uses WebSocket communication between the Monaco editor frontend and language-specific debug backends.
Supported Languages
- Python - Uses a bdb-based debugger via
dap_websocket_server.py - TypeScript/Bun - Uses V8 Inspector Protocol via
dap_websocket_server_bun.ts
Architecture
┌─────────────────────┐ WebSocket ┌──────────────────────────┐
│ Monaco Editor │◄──────────────────►│ DAP Debug Service │
│ (dapClient.ts) │ DAP Protocol │ (dap_debug_service.ts) │
└─────────────────────┘ └──────────┬───────────────┘
│
┌──────────┴───────────┐
│ │
┌──────▼──────┐ ┌───────▼───────┐
│ Python │ │ Bun/TS │
│ Debugger │ │ Debugger │
└─────────────┘ └───────────────┘
Files
| File | Description |
|---|---|
dap_debug_service.ts |
Unified WebSocket server that routes to Python or Bun debuggers |
dap_websocket_server.py |
Python debugger backend (bdb-based) |
dap_websocket_server_bun.ts |
Bun/TypeScript debugger backend (V8 Inspector) |
dapClient.ts |
Client-side DAP WebSocket client with Svelte store |
MonacoDebugger.svelte |
Monaco editor integration component |
DebugToolbar.svelte |
Debug control buttons (step, continue, etc.) |
DebugPanel.svelte |
Variables and stack trace display panel |
index.ts |
Module exports |
Usage
Starting the Debug Service
bun run debug/dap_debug_service.ts
Options:
--port PORT- Server port (default: 3003)--host HOST- Server host (default: 0.0.0.0)--python-path PATH- Python binary path (default: python3)--bun-path PATH- Bun binary path (default: bun)--nsjail- Enable nsjail sandboxing for debugger processes--nsjail-config PATH- Path to nsjail config file--nsjail-path PATH- Path to nsjail binary (default: nsjail)
Endpoints
/python- Python debugging/typescript- TypeScript/Bun debugging/bun- Alias for/typescript
Environment Variables
| Variable | Description | Default |
|---|---|---|
DAP_PORT |
Server port | 3003 |
DAP_HOST |
Server host | 0.0.0.0 |
DAP_PYTHON_PATH |
Python binary path | python3 |
DAP_BUN_PATH |
Bun binary path | bun |
DAP_NSJAIL_ENABLED |
Enable nsjail sandboxing | false |
DAP_NSJAIL_PATH |
nsjail binary path | nsjail |
DAP_NSJAIL_CONFIG |
nsjail config file path | - |
Python dependency preparation
Before debugging a Python script, its imports are installed through windmill prepare-deps, which
runs uv without a database connection. It cannot read the instance settings, so it takes its
registry configuration from the environment of the debug service instead, and the Python server is
handed the resulting venv with --venv-path. The install runs in the service rather than in the
session because a private index URL usually embeds credentials and the Python server executes the
debugged script inside its own interpreter, where anything it holds is readable by that script.
Set these on the debug service. Where two names are listed the first wins; a worker reads the
PIP_* / PY_* names in the same way, except for the index URLs, whose worker env fallbacks are
only PIP_INDEX_URL / PIP_EXTRA_INDEX_URL (the PY_* spellings are accepted here for symmetry
with the other settings):
| Variable | Description | Default |
|---|---|---|
PY_INDEX_URL / PIP_INDEX_URL |
Package index (--index-url) |
PyPI |
PY_EXTRA_INDEX_URL / PIP_EXTRA_INDEX_URL |
Extra indexes, comma-separated (--extra-index-url) |
- |
PY_TRUSTED_HOST / PIP_TRUSTED_HOST |
Hosts to trust, whitespace-separated (--trusted-host) |
- |
PY_INDEX_CERT / PIP_INDEX_CERT |
CA bundle for the index, passed to uv as SSL_CERT_FILE. Falls back to SSL_CERT_FILE, then REQUESTS_CA_BUNDLE, then CURL_CA_BUNDLE, so a host that configures its CA under any of those names is picked up. Whichever is used replaces uv's own roots rather than adding to them, so it has to be a complete bundle: one holding only a private CA leaves every public index untrusted. bun install gets the same bundle as NODE_EXTRA_CA_CERTS, the only spelling Bun reads |
- |
SSL_CERT_DIR |
Directory of certificates, forwarded to uv as-is. Replaces uv's roots the same way the bundle does, so a directory holding only a private CA leaves public indexes untrusted | - |
PY_NATIVE_CERT / UV_NATIVE_TLS |
true to also trust the platform certificate store (--native-tls) |
false |
UV_INDEX_STRATEGY |
uv index strategy | unsafe-best-match |
UV_HTTP_TIMEOUT |
uv HTTP request timeout, in seconds | uv's own default |
DAP_PREPARE_DEPS_TIMEOUT_MS |
How long to wait for the install before starting the session without it | 120000 |
When the install fails, the CLI answers success: false and carries the installer's stderr in both
error and install_stderr; the service reports it to the client as an output event, so the
reason (unreachable mirror, untrusted certificate, unknown package) reaches the user instead of a
bare ModuleNotFoundError at the first import.
Proxy variables (HTTP_PROXY / HTTPS_PROXY / NO_PROXY, in either case) are forwarded from the
service into each session, since the debugged script needs them for its own outbound calls, exactly
as a job's script does on a worker. When a proxy is set without a bypass list, NO_PROXY defaults
to localhost,127.0.0.1 so calls to BASE_INTERNAL_URL are not proxied.
Trust roots are forwarded alongside them: SSL_CERT_FILE, SSL_CERT_DIR, REQUESTS_CA_BUNDLE,
CURL_CA_BUNDLE and NODE_EXTRA_CA_CERTS. Behind a TLS-intercepting proxy these are what let the
debugged script's own HTTPS calls verify, and installing the CA in the container's system store is
not enough on its own, since requests carries its own bundle and Node reads only
NODE_EXTRA_CA_CERTS. Registry settings are deliberately not forwarded: they carry credentials and
only the service needs them.
Registering that CA in the container's system store happens on its own: mount it into
/usr/local/share/ca-certificates/ named *.crt, the only extension update-ca-certificates
reads, and windmill_extra runs it before starting any service. RUN_UPDATE_CA_CERTIFICATE_AT_START=true forces the same thing whether or not
certificates are mounted there, and RUN_UPDATE_CA_CERTIFICATE_PATH overrides the tool, matching
the server and worker. Both are best-effort: a UID that cannot write /etc/ssl/certs logs a warning
and the container still boots. INIT_SCRIPT remains the hook for anything more involved, and unlike
the CA update it aborts startup when it fails.
Note what the system store does not cover, which is most of what a debug session installs with:
uv trusts its own bundled roots unless PY_NATIVE_CERT/UV_NATIVE_TLS is true, Bun and Node read
only NODE_EXTRA_CA_CERTS, and requests carries certifi. Registering the CA fixes Python's stdlib
ssl, curl and git; the rest still needs the variables above.
Keeping the settings out of the session's environment only bounds what the debugged script can read
from itself. An unsandboxed session runs under the same user as the service and can still read the
service's environment through /proc, the same way a job can read a worker's when the worker runs
unsandboxed. Isolating sessions from the service takes --nsjail --nsjail-config nsjail.debug.config.proto: it is that config's PID namespace and mount_proc that put the service
out of reach, not the flag on its own.
The installer is jailed on the same terms, in both languages: uv pip install builds source
distributions and bun install runs postinstall scripts, so a package's own code executes there
too. It keeps the service's environment across that boundary — the config sets keep_env, which is
how the settings above reach it — so replacing that with an allowlist would have to carry the
registry and CA variables in explicitly. It also runs in its own process group, because uv and
bun are grandchildren: signalling only the installer reparents them to init and they keep
downloading, which would make the timeout and the cancel-on-disconnect half-measures.
Frontend Integration
<script>
import { MonacoDebugger } from './debug'
let editor // Monaco editor instance
let code = 'print("Hello")'
</script>
<MonacoDebugger {editor} {code} language="python3" />
Testing
# Test Python debugger
bun run debug/test_dap_server.py
# Test Bun debugger
bun run debug/test_dap_server_bun.ts
# Test unified service
bun run debug/test_debug_service.ts