mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-21 00:02:30 +00:00
hugocasaandClaude Opus 5
53afecd458
* fix(worker): register the job token with the log masking system The masking system covered secrets fetched through `get_value_internal` and `$encrypted:` args, but not the job's own token, so a script that echoed `$WM_TOKEN` wrote it verbatim into logs that are persisted to the database and, when configured, to object storage. `run_worker` now registers the token for the job it just pulled, alongside the existing `register_running_job` call, so it is redacted like any other registered secret. That makes every job carry at least one registered value, where before the per-batch mask snapshot was skipped entirely for the majority of jobs that touched no secret. Cache the compiled Aho-Corasick automaton per job and invalidate it when a new secret is registered, so a chatty job no longer rebuilds it once per log batch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 * fix(nativets): mask secrets in the in-process log path NativeTS hands `console.log` output to a task that drains a channel into `append_logs`, so it never reaches the masking in `handle_child::write_lines` and a script logging `$WM_TOKEN` persisted the raw JWT. That drain can still be flushing after the job is unregistered, so a plain per-line `snapshot` would leave the tail unmasked. `JobMasker` keeps the last masks it saw for exactly that window, and refreshes while the job is alive so secrets fetched mid-run are covered too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 * fix(nativets): seed the job masker at construction A `JobMasker` that only looked up its masks on the first `mask` call had the same hole at the head of the log that its retention closes at the tail: if the drain task's first productive poll landed after the job was unregistered, the registry was already gone and every line was written raw. `new` now takes the snapshot, and its callers construct it from the job's own execution while the job is still registered. Also cover the nativets sink with an integration test, gated on `deno_core` the way the CI test build is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 * fix(nativets): mask on the producing side of the log channel Masking as the drain task wrote to `append_logs` left two holes, because that task is detached and outlives the job: a secret registered mid-run could still be queued when the job was unregistered and would then be written raw, and the `windmill:job_log` tracing emission that EE forwards job logs on never went through the mask at all. Mask where the line is produced instead. That loop is joined before the job completes, so the job's secrets are always still registered, and one call now covers both the tracing mirror and the channel. The result stream keeps reading the raw text, the way `handle_child` keeps its raw `line` for results. `JobMasker` is no longer load-bearing for the post-unregistration window, so it is documented for what it now does: keep the security notice to once per set of secrets for a sink that masks line by line. Also drop the nativets test's tag override — `DEFAULT_TAGS` does advertise `nativets`, so the comment justifying it was wrong — and pin the automaton cache invalidation, whose failure mode is an unmasked secret. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 * fix(worker): hold the log-masking lifecycle at the job boundary Registering the job around the poller's call left every other way of running a job uncovered: the interactive worker shell and inline AI agent tools both call `handle_queued_job` directly, and a script logging `$WM_TOKEN` from either persisted the live credential. Register from inside `handle_queued_job` instead, under a drop guard, so each path is covered by construction rather than by remembering to add a call. Nothing is lost by unregistering earlier: the writes that follow go through `append_logs`, which never consulted the registry. In nativets, decide the stream/log routing before masking. `MaskSnapshot`'s notice is one-shot, so a secret-bearing `WM_STREAM:` chunk used to spend it on text that is then discarded, leaving later redactions in `job_logs` unexplained. Restore the masker's post-unregistration test: the memory-limit path never joins the producing loop, so that fallback is still load-bearing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 * docs: correct the nativets masking comments The producer loop is not joined on the memory-limit path, so it does not "always" run while the job is registered — say normally, which is what `JobMasker`'s fallback is there for. Name the reason a stream chunk stays raw everywhere it goes, including the tracing mirror: it is result data that no log sink persists, so masking it would be masking a result. State the masker test's invariant without asserting a mechanism behind it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 * fix(masking): keep the security notice on a line of its own `mask` appended the notice as a newline plus the notice text, which assumes the caller hands it a bare log line. nativets hands it a chunk that already ends in a newline, and its sink concatenates chunks verbatim, so the notice arrived after a blank line and the next log line was welded onto the end of it. Emit the notice as its own line for either shape. `handle_child` is unaffected: its input never ends in a newline, so it keeps the original path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Windmill Worker
The worker. Used to process and execute flows & jobs.
This crate exposes both a library as well as a binary target.