Will JonesandClaude Opus 4.8
285add40dd
Bridges Lance's internal `metrics`-crate instrumentation (object store request counts, bytes, latency, errors, and throttles) into OpenTelemetry, in both the Python and Node bindings, with a shared adapter in the Rust core. This is the LanceDB counterpart to lance-format/lance#7537. ## Rust core (`rust/lancedb`) Two new, **off-by-default** features: - `metrics` — re-exports the [`metrics`](https://docs.rs/metrics) crate as `lancedb::metrics` and turns on Lance's object-store instrumentation. Install any `metrics`-compatible recorder to collect them. - `metrics-otel` — adds `lancedb::metrics_otel`, a pull-based adapter that installs a process-global recorder aggregating into lock-free cumulative storage and exposes a snapshot/catalog API (`register_metrics_recorder`, `metrics_catalog`, `snapshot_metrics`, `MetricPoint`/`MetricValue`/`MetricKind`/`MetricDescription`). Both bindings build on this. ## Python `lancedb.otel.instrument_lancedb_metrics()` registers each metric as an OpenTelemetry observable instrument on the given (or global) `MeterProvider`. Available via the `otel` extra (`pip install lancedb[otel]`), which pulls in only `opentelemetry-api` — the application supplies and configures the SDK. ## Node `instrumentLanceDbMetrics()` provides the equivalent wiring against `@opentelemetry/api`. This is the only public entry point; the underlying recorder/catalog/snapshot functions stay internal. Because OpenTelemetry has no asynchronous histogram instrument, histograms are exported Prometheus-style as `<name>_bucket` (with an `le` attribute), `<name>_count`, and `<name>_sum`. Only `_sum` carries the histogram's unit; `_bucket` and `_count` observe cumulative counts and are unitless. The adapter is enabled by default in the Python and Node builds, and off by default in the Rust crate. ## Notes - Requires Lance ≥ `v9.0.0-beta.19`, which ships the object-store metrics APIs (upstream lance-format/lance#7537, now merged). `main` is already on beta.19, so this is a single feature commit with no dependency bump. - Tests: 8 Rust unit tests, 3 Python tests, 2 Node tests, all covering the end-to-end object-store-metrics → OpenTelemetry path. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1.5 KiB
@lancedb/lancedb • Docs
@lancedb/lancedb / instrumentLanceDbMetrics
Function: instrumentLanceDbMetrics()
function instrumentLanceDbMetrics(meterProvider?): boolean
Register LanceDB metrics as OpenTelemetry observable instruments.
Installs a process-global metrics recorder and creates one observable
instrument per LanceDB metric (currently object store request counts, bytes,
latency, errors, and throttles) on the given (or global) MeterProvider. The
configured MetricReader then collects them on its own schedule.
Counters and gauges map directly to observable counters/gauges. Because
OpenTelemetry has no asynchronous histogram instrument, each histogram is
exported Prometheus-style as cumulative le bucket counts (<name>_bucket,
with an le attribute) plus <name>_count and <name>_sum.
Requires @opentelemetry/api (a dependency) and, to actually export, an
OpenTelemetry SDK such as @opentelemetry/sdk-metrics.
Parameters
- meterProvider?:
MeterProviderThe provider to register instruments on. Defaults to the global provider from@opentelemetry/api.
Returns
boolean
true if the recorder is installed and instruments are registered.
false if a different metrics recorder is already installed in this
process (only one global recorder is permitted), in which case a warning is
emitted and no instruments are created. Calling this more than once is safe;
instruments are created only on the first successful call.