mirror of https://github.com/lancedb/lancedb.git synced 2026-09-04 12:38:38 +00:00

Files

T

Will Jones 285add40dd feat: expose Lance metrics via OpenTelemetry in Python and Node (#3609 )

Bridges Lance's internal `metrics`-crate instrumentation (object store
request counts, bytes, latency, errors, and throttles) into
OpenTelemetry, in both the Python and Node bindings, with a shared
adapter in the Rust core. This is the LanceDB counterpart to
lance-format/lance#7537.

## Rust core (`rust/lancedb`)
Two new, **off-by-default** features:
- `metrics` — re-exports the [`metrics`](https://docs.rs/metrics) crate
as `lancedb::metrics` and turns on Lance's object-store instrumentation.
Install any `metrics`-compatible recorder to collect them.
- `metrics-otel` — adds `lancedb::metrics_otel`, a pull-based adapter
that installs a process-global recorder aggregating into lock-free
cumulative storage and exposes a snapshot/catalog API
(`register_metrics_recorder`, `metrics_catalog`, `snapshot_metrics`,
`MetricPoint`/`MetricValue`/`MetricKind`/`MetricDescription`). Both
bindings build on this.

## Python
`lancedb.otel.instrument_lancedb_metrics()` registers each metric as an
OpenTelemetry observable instrument on the given (or global)
`MeterProvider`. Available via the `otel` extra (`pip install
lancedb[otel]`), which pulls in only `opentelemetry-api` — the
application supplies and configures the SDK.

## Node
`instrumentLanceDbMetrics()` provides the equivalent wiring against
`@opentelemetry/api`. This is the only public entry point; the
underlying recorder/catalog/snapshot functions stay internal.

Because OpenTelemetry has no asynchronous histogram instrument,
histograms are exported Prometheus-style as `<name>_bucket` (with an
`le` attribute), `<name>_count`, and `<name>_sum`. Only `_sum` carries
the histogram's unit; `_bucket` and `_count` observe cumulative counts
and are unitless. The adapter is enabled by default in the Python and
Node builds, and off by default in the Rust crate.

## Notes
- Requires Lance ≥ `v9.0.0-beta.19`, which ships the object-store
metrics APIs (upstream lance-format/lance#7537, now merged). `main` is
already on beta.19, so this is a single feature commit with no
dependency bump.
- Tests: 8 Rust unit tests, 3 Python tests, 2 Node tests, all covering
the end-to-end object-store-metrics → OpenTelemetry path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-09 15:36:03 -07:00

5.6 KiB

Raw Blame History

@lancedb/lancedb • Docs

5.6 KiB

Raw Blame History

@lancedb/lancedb

Namespaces

Enumerations

Classes

Interfaces

Type Aliases

Functions

5.6 KiB Raw Blame History

@lancedb/lancedb

Namespaces

Enumerations

Classes

Interfaces

Type Aliases

Functions

5.6 KiB

Raw Blame History