Add five new attributes to the `job` and `job_postprocessing` tracing spans
so that OTEL-consuming backends (Sentry, Honeycomb, Datadog, etc.) can
filter and group telemetry by how a job was triggered and what type it is.
New span attributes:
- `job_kind` — Script, Flow, AppScript, AIAgent, Preview, etc.
- `created_by` — the user or system identity that queued the job
- `trigger_kind` — schedule, webhook, kafka, http, sqs, etc.
- `trigger` — the schedule/trigger path (when applicable)
- `runnable_id` — the id of the runnable that ran
Also adds `JobKind::as_str()` for a consistent lowercase string
representation, following the same pattern as `ScriptLang::as_str()`.
Existing attributes (job_id, workspace_id, script_path, language, tag,
flow_step_id, parent_job, root_job) are unchanged.
Note: the EE `full_job` span in `otel_ee.rs` and the log records emitted
by `job_logger_ee.rs` would also benefit from these attributes. This PR
covers only the public-repo spans; a follow-up EE change would propagate
the same fields to logs and the full_job span.
* test: add OTEL coverage tests
Add 38 unit tests covering OpenTelemetry infrastructure:
- OtelSettings serde (empty, partial, full, roundtrip, skip_serializing)
- OtelTracingProxySettings serde (defaults, languages, dedup, rejection)
- ScriptLang rename cases
- LogCounter initialization and CountingLayer event counting
- Targets filter suppression of windmill:job_log
- get_otel_context_envs traceparent format verification
- Worker OtelTracingProxySettings (HashSet variant)
Companion EE PR adds tests for span_cx_from_job_id, metric functions,
proto conversion, SpanBuilder, and tracing proxy handler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add E2E OTEL tests with in-memory exporters
Add integration tests that verify metrics and spans flow correctly
through the OpenTelemetry pipeline using in-memory exporters:
Metrics (1 comprehensive test):
- All 20 metric names registered correctly
- Counter values (push/delete/pull/zombie/execution/failed/started)
- Gauge values with attributes (queue count by tag, worker busy, db pool, health)
- Histogram values (execution duration, pull duration)
- Health status phase encoding (healthy=1, degraded=0, unhealthy=0)
Spans (6 tests):
- Root job span created with "full_job" name and Ok status
- Error status with "Job failed" description on failure
- trace_id derived from job UUID
- span_id derived from job UUID low bits
- Child jobs (with parent_job) produce no span
- Attribute values (job_id, workspace_id, script_path) match job data
Also:
- Add testing feature to opentelemetry_sdk for InMemoryMetricExporter
- Update otel_oss.rs for SdkTracer type rename in 0.30
- Add opentelemetry/opentelemetry_sdk to dev-dependencies
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: remove unit tests in favor of E2E OTEL tests
The E2E integration tests in backend/tests/otel.rs cover the same
ground more thoroughly with in-memory exporters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Yoaquim Cintrón
Ruben Fiszel