feat: expose queued Function refresh status (#4959)

## Problem

The existing Function `column_status` surface cannot decode the Agent
scheduler's `queued` state or the `failure_threshold_reached` and
`binding_revision_missing` blocked reasons. It also preserves only the
latest refresh job, so a later cancellation can hide an earlier actual
failure.

## Change

- Adds the `queued`, `failure_threshold_reached`, and
`binding_revision_missing` wire states in Rust, Python, and Node. The
wire change is additive, while Rust callers must update exhaustive enum
matches and public struct literals.
- Adds optional `last_failure_job` alongside `last_refresh_job`. The
latest cancellation and the most recent actual failure retain separate
meanings.
- Reuses the canonical Rust `JobFailure` and existing Python/Node
`JobFailureInfo` models for nested job failure details instead of
introducing a parallel status error model.
- Maps the existing public `sql_job_id` field through Node's
`JobFailureInfo`.
- Keeps `column_status`, `column_statuses`, and `wait_fresh` as the
existing API surfaces. There is no new endpoint, row-error list, durable
ledger, or unknown-enum fallback.
- Documents `queued` versus `running` and the separate latest-attempt
and latest-failure jobs in the existing hidden freshness release draft.

The new status values are additive but cross-service: `queued`, the two
new blocked reasons, and `last_failure_job` require matching Agent/QN
and SDK deployments before clients can rely on them. This SDK follow-up
does not change the normal `7cc3904a` dependency pin or claim whole
ENT-3487 completion.

## Validation

The tested private source is `b19c8adbdc936c5976def6c17be47f6d5e3faf3e`,
based on product commit `c9bac25ecb8b73b1bd0256bbea65c2621bcf6a94`. The
final product docs commit changes generated TypeDoc only.

- Rust: 2 selected tests passed, 0 failed, exit 0. The same two Rust
blobs are unchanged from their tested `788ef85b` source through
`c9bac25e`.
- Python: the PyO3 extension built successfully, and the selected native
PyO3 SDK test against a mock HTTP REST transport passed 1/0, exit 0. The
test requests pinned version 42.
- Node: the addon and TypeScript builds passed, Biome passed, and the
selected Jest transport test passed 1/0 with 57 filtered tests. Addon
SHA-256:
`c888e260b61d41e79689ed2f20dd0dc98586a2377ce34f21610b06111734d426`.
- Python extension SHA-256:
`36c961d8a0cbd2e102a7765f2e47efb4c5af6310baa6262c8e3120407463da2b`.
- TypeDoc was generated from the native `native.d.ts` produced by that
Node build. A second generation produced no remaining diff.

The public diff contains only product, test, and generated API
documentation changes. Native runner scripts and preserved
compile-failure logs remain private validation evidence.

Agent/QN production mapping, mixed-version rollout, and the remaining
ENT-3487 server-side work are tracked separately.
This commit is contained in:
Xuanwo
2026-11-11 21:14:30 +08:00
committed by GitHub
parent 7cc3904a7e
commit a356950bce
16 changed files with 394 additions and 55 deletions
+16 -6
View File
@@ -138,12 +138,7 @@ impl Job {
/// Why the job failed, when it failed and the server reports a reason.
#[napi(getter)]
pub fn failure(&self) -> Option<JobFailureInfo> {
self.inner.failure().map(|failure| JobFailureInfo {
phase: failure.phase,
message: failure.message,
traceback: failure.traceback,
retryable: failure.retryable,
})
self.inner.failure().map(Into::into)
}
/// This job's recorded lifecycle events, as an Arrow IPC stream buffer.
@@ -216,10 +211,25 @@ impl From<lancedb::database::JobInfo> for JobInfo {
/// The server's account of why a job failed.
#[napi(object)]
#[derive(Clone, Debug)]
pub struct JobFailureInfo {
pub phase: Option<String>,
pub message: Option<String>,
/// User-code traceback, possibly truncated by the server.
pub traceback: Option<String>,
pub retryable: Option<bool>,
/// Isolated SQL job that raised this failure, when supplied by the server.
pub sql_job_id: Option<String>,
}
impl From<lancedb::error::JobFailure> for JobFailureInfo {
fn from(failure: lancedb::error::JobFailure) -> Self {
Self {
phase: failure.phase,
message: failure.message,
traceback: failure.traceback,
retryable: failure.retryable,
sql_job_id: failure.sql_job_id,
}
}
}
+17 -8
View File
@@ -20,6 +20,7 @@ use napi_derive::napi;
use crate::blob::{BlobFile, copy_blob_buffers, parse_row_ids, spawn_abortable};
use crate::error::NapiErrorExt;
use crate::index::Index;
use crate::job::JobFailureInfo;
use crate::merge::NativeMergeInsertBuilder;
use crate::query::{Query, TakeQuery, VectorQuery};
use crate::util::schema_to_buffer;
@@ -1868,6 +1869,19 @@ pub struct FunctionColumnLastRefreshJob {
pub state: String,
pub automatic: Option<bool>,
pub completed_at_millis: Option<BigInt>,
pub failure: Option<JobFailureInfo>,
}
impl From<lancedb::table::FunctionColumnLastRefreshJob> for FunctionColumnLastRefreshJob {
fn from(job: lancedb::table::FunctionColumnLastRefreshJob) -> Self {
Self {
job_id: job.job_id,
state: job.state,
automatic: job.automatic,
completed_at_millis: job.completed_at_millis.map(BigInt::from),
failure: job.failure.map(Into::into),
}
}
}
/// Automatic-refresh projection for a Function output.
@@ -1899,6 +1913,7 @@ pub struct FunctionColumnStatus {
pub never_computed: BigInt,
pub failed: BigInt,
pub last_refresh_job: Option<FunctionColumnLastRefreshJob>,
pub last_failure_job: Option<FunctionColumnLastRefreshJob>,
pub auto_refresh: FunctionColumnAutoRefreshStatus,
}
@@ -1921,14 +1936,8 @@ impl From<lancedb::table::FunctionColumnStatus> for FunctionColumnStatus {
stale: BigInt::from(status.stale),
never_computed: BigInt::from(status.never_computed),
failed: BigInt::from(status.failed),
last_refresh_job: status
.last_refresh_job
.map(|job| FunctionColumnLastRefreshJob {
job_id: job.job_id,
state: job.state,
automatic: job.automatic,
completed_at_millis: job.completed_at_millis.map(BigInt::from),
}),
last_refresh_job: status.last_refresh_job.map(Into::into),
last_failure_job: status.last_failure_job.map(Into::into),
auto_refresh: FunctionColumnAutoRefreshStatus {
enabled: status.auto_refresh.enabled,
interval_seconds: status.auto_refresh.interval_seconds.map(BigInt::from),