Files
windmill/backend/windmill-worker
Ruben Fiszel cec84849b9 feat: OTEL span status on failed jobs + Python stderr severity classification (#8918)
* feat: set OTEL span status on failed jobs and add stderr severity toggle

Record otel.status_code=ERROR and otel.status_description on the job /
job_postprocessing tracing spans when a job fails, so OTel exporters
(Sentry, Honeycomb, Datadog) see the standard span-level failure signal
instead of just the success=false attribute. Description is truncated to
512 chars to keep span payloads bounded.

Add OtelSettings.stderr_default_severity instance setting (error | warn |
info | debug, default error) to let operators downgrade the OTEL severity
used for job stderr output. Python logging routes every record >= WARNING
to stderr, so the historical blind stderr->error mapping produces false
positives for scripts like dlt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref for stderr severity toggle

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: clarify OTEL span description for already-completed jobs

When handle_queued_job returns Ok(false) on Error::AlreadyCompleted
(another worker already finished the job during a race), the span
was labeled "job returned false" which is opaque to operators.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: parse Python logging severity from job stderr

Replace the global stderr_default_severity instance toggle with a
worker-side classifier that recognizes Python's canonical
basicConfig() format (LEVELNAME:logger.name:message) and emits the
corresponding tracing level. Lines that don't match keep the
historical tracing::error! fallback, so genuine failures still
surface and non-Python output is unaffected.

Removes StderrLogSeverity, STDERR_LOG_SEVERITY, and the
otel.stderr_default_severity field; adds
classify_python_logging_line in windmill-common.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: cover classify_python_logging_line + fix truncate_description doc

Add unit tests for the Python stderr-severity classifier and correct
the truncate_description docstring to say "bytes" (the cap is byte-
based, with UTF-8 boundary rounding for safety). Addresses Claude
review feedback on PR #8918.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
2026-04-29 14:14:47 +00:00
..

Windmill Worker

The worker. Used to process and execute flows & jobs.

This crate exposes both a library as well as a binary target.