feat: expose exact Function column freshness (#4958)

## Problem

Function-backed columns need an exact `fresh`, `stale`,
`never_computed`, and
`failed` partition for the table version a caller is reading. The
existing
error listing is capped, and composing it with job and schema listings
cannot
prove which rows currently failed.

## Behavior

- Adds a strict freshness reader beside the existing permissive planner.
It
  distinguishes an absent certificate from legacy evidence that omits
  failures, and from a current certificate carrying failure coverage.
- Extends the immutable freshness sidecar with per-fragment known and
failed
  row bitmaps. Partial or `WHERE` publications cover only observed rows;
  untouched rows stay unknown. Deleted rows do not block classification.
- Adds failure-aware whole-fragment and row-repair writers. A rejected
older
attempt cannot overwrite failure evidence already recorded for a newer
input
  signature.
- Adds `column_status`, `column_statuses`, and `wait_fresh` to Python
and the
  corresponding Node methods. The client calls the server's exact status
endpoint, validates the four-way partition and one-version guarantee,
and
preserves the complete Function version, last-job, and automatic-refresh
  metadata.
- Adds an MDX-commented draft usage section to the Enterprise
computed-column
refresh guide, covering Python, Rust, and TypeScript and remaining
hidden
  until the server integration ships.
- Sends `column`, `branch`, and exact `version` selectors to the status
route.
A pinned handle rejects a nonempty response from any other table
version;
  latest handles keep the existing read-freshness headers.
- Retries `wait_fresh` only for HTTP 503 with
`function_column_status_finalization_pending`. Existing failed rows
remain
  visible and do not keep the wait pending; other errors propagate.
- Models the shared scheduler projection, including `up_to_date`, real
`blocked` states, their reason, failure streak, and effective cooldown.

## Validation

This is the SDK side of ENT-3563. Sophon's failure-coverage publication
and
status endpoint are a separate integration; deployed end-to-end and
whole-ticket
acceptance remain pending.

- Rust: 9 exact freshness/status tests passed, plus 1 exact
latest-handle
  freshness-header test. The pinned transport test was extended with a
wrong-version response and rerun at `ebd6649a` (1 passed). Coverage
includes untouched partial rows,
  selected-row failure clearing, cross-generation evidence, typed and
non-retried errors, `up_to_date`, blocked scheduler fields, and
exact-version
  requests.
- Python: 2 native PyO3 remote-transport tests passed together,
including
typed retry; one local facade test reaches the existing unsupported
boundary.
- Node: the NAPI addon built successfully, TypeScript and Biome passed,
and the
selected native remote-transport test passed 1/0, including
exact-version
  selection and blocked scheduler metadata.
- `cargo fmt`, `git diff --check`, Ruff format/lint, Python compile,
Biome, and
  TypeScript checks passed.

Validation is mapped by the relevant unchanged source rather than
treated as one
final-head run: the nine Rust tests ran at `e4d51fc8`, the latest-header
and Python
remote tests at `31e68b7d`, the local Python test and Node build at
`6f213584`, and
the final TypeScript/Jest delta at `4f2d5c27`. The final product tree
excludes those five private EC2 runner files and adds
six generated TypeDoc files plus one hidden draft usage section in the
computed-column refresh guide. The Node delta corrected a private
TypeScript
command and reused the unchanged native addon.

nodejs/__test__/remote.test.ts

@@ -647,6 +647,69 @@ describe("remote connection", () => {
     ]);
   });
 
+  it("passes a pinned version and decodes Function scheduler status", async () => {
+    const queries: Record<string, string>[] = [];
+    await withMockDatabase(
+      (req, res) => {
+        const url = new URL(req.url ?? "/", "http://localhost");
+        if (url.pathname.endsWith("/describe/")) {
+          res.writeHead(200, { "Content-Type": "application/json" }).end(
+            JSON.stringify({
+              name: "docs",
+              version: 42,
+              schema: { fields: [] },
+            }),
+          );
+          return;
+        }
+        if (url.pathname === "/v1/table/docs/function_columns/status") {
+          queries.push(Object.fromEntries(url.searchParams.entries()));
+          res.writeHead(200, { "Content-Type": "application/json" }).end(
+            `[{
+              "column":"embedding",
+              "binding_id":"fb_01",
+              "function":{
+                "name":"embed",
+                "object_id":"embed",
+                "location":"objects/embed",
+                "version":"7",
+                "manifest_digest":"sha256:abc"
+              },
+              "table_version":42,
+              "rows":4,
+              "fresh":2,
+              "stale":2,
+              "never_computed":0,
+              "failed":0,
+              "auto_refresh":{
+                "enabled":true,
+                "interval_seconds":60,
+                "state":"blocked",
+                "failure_streak":3,
+                "effective_cooldown_millis":8000,
+                "blocked_reason":"preparation_backoff",
+                "next_run_at_millis":null
+              }
+            }]`,
+          );
+          return;
+        }
+        res.writeHead(404).end();
+      },
+      async (db) => {
+        const table = await db.openTable("docs");
+        await table.checkout(42);
+        const status = await table.functionColumnStatus("embedding");
+        expect(status.tableVersion).toBe(42n);
+        expect(status.autoRefresh.state).toBe("blocked");
+        expect(status.autoRefresh.failureStreak).toBe(3);
+        expect(status.autoRefresh.effectiveCooldownMillis).toBe(8000n);
+        expect(status.autoRefresh.blockedReason).toBe("preparation_backoff");
+      },
+    );
+    expect(queries).toEqual([{ column: "embedding", version: "42" }]);
+  });
+
   it("forwards refresh options to the backfill request", async () => {
     const bodies: unknown[] = [];
     await withMockDatabase(

nodejs/lancedb/index.ts

@@ -154,6 +154,10 @@ export {
 
 export {
   Table,
+  FunctionColumnAutoRefreshStatus,
+  FunctionColumnLastRefreshJob,
+  FunctionColumnStatus,
+  FunctionColumnVersionStatus,
   Branches,
   BranchColumnSummary,
   BranchColumnChange,

nodejs/lancedb/table.ts

@@ -258,6 +258,66 @@ export interface RTEWriteSpec {
   writerConfigDefaults?: Record<string, string>;
 }
 
+/** Exact immutable Function revision attached to a table output. */
+export interface FunctionColumnVersionStatus {
+  name: string;
+  namespacePath: string[];
+  objectId: string;
+  location: string;
+  version: string;
+  manifestDigest: string;
+}
+
+/** Latest matching refresh Job projected for a Function output. */
+export interface FunctionColumnLastRefreshJob {
+  jobId: string;
+  state: string;
+  automatic?: boolean;
+  completedAtMillis?: bigint;
+}
+
+/** Automatic-refresh projection for a Function output. */
+export interface FunctionColumnAutoRefreshStatus {
+  enabled: boolean;
+  intervalSeconds?: bigint;
+  activeJobId?: string;
+  state:
+    | "disabled"
+    | "running"
+    | "ready"
+    | "up_to_date"
+    | "waiting_interval"
+    | "waiting_backoff"
+    | "blocked"
+    | "unobservable";
+  failureStreak: number;
+  effectiveCooldownMillis?: bigint;
+  blockedReason?:
+    | "table_has_mem_wal"
+    | "inputs_not_settled"
+    | "dependency_in_flight"
+    | "table_capacity"
+    | "cancelled"
+    | "preparation_backoff";
+  nextRunAtMillis: bigint | null;
+  nextRunAtReason?: string;
+}
+
+/** Exact four-way status of one Function output at one table version. */
+export interface FunctionColumnStatus {
+  column: string;
+  bindingId: string;
+  function: FunctionColumnVersionStatus;
+  tableVersion: bigint;
+  rows: bigint;
+  fresh: bigint;
+  stale: bigint;
+  neverComputed: bigint;
+  failed: bigint;
+  lastRefreshJob?: FunctionColumnLastRefreshJob;
+  autoRefresh: FunctionColumnAutoRefreshStatus;
+}
+
 /**
  * A Table is a collection of Records in a LanceDB Database.
  *
@@ -780,6 +840,32 @@ export abstract class Table {
     options?: FunctionErrorsOptions,
   ): Promise<FunctionErrors>;
 
+  /**
+   * Return exact freshness status for Function-backed output columns.
+   *
+   * All results describe one server table version. Omit `column` for every
+   * Function output. Rows already failed remain visible and do not prevent
+   * `waitFresh` from completing. A handle pinned to an exact version requests
+   * that version; a latest handle retains normal read-freshness behavior.
+   */
+  abstract functionColumnStatuses(
+    column?: string,
+  ): Promise<FunctionColumnStatus[]>;
+
+  /** Return exact freshness status for one Function-backed output column. */
+  abstract functionColumnStatus(column: string): Promise<FunctionColumnStatus>;
+
+  /**
+   * Poll until no stale or never-computed rows remain.
+   *
+   * Only HTTP 503 with the exact finalization-pending error code is retried;
+   * every other error propagates. Failed rows do not block completion.
+   */
+  abstract waitFresh(
+    column: string,
+    timeoutSeconds?: number,
+  ): Promise<FunctionColumnStatus>;
+
   /**
    * Recompute this table's contents from its materialized-view definition.
    *
@@ -1557,6 +1643,31 @@ export class LocalTable extends Table {
     return await this.inner.functionErrors(options);
   }
 
+  async functionColumnStatuses(
+    column?: string,
+  ): Promise<FunctionColumnStatus[]> {
+    return (await this.inner.functionColumnStatuses(
+      column,
+    )) as FunctionColumnStatus[];
+  }
+
+  async functionColumnStatus(column: string): Promise<FunctionColumnStatus> {
+    return (await this.inner.functionColumnStatus(
+      column,
+    )) as FunctionColumnStatus;
+  }
+
+  async waitFresh(
+    column: string,
+    timeoutSeconds?: number,
+  ): Promise<FunctionColumnStatus> {
+    validateNonNegativeInteger(timeoutSeconds, "timeoutSeconds");
+    return (await this.inner.waitForFunctionColumn(
+      column,
+      timeoutSeconds,
+    )) as FunctionColumnStatus;
+  }
+
   async refreshMaterializedView(
     sourceVersion?: number,
     full?: boolean,

nodejs/src/table.rs

@@ -534,6 +534,55 @@ impl Table {
         Ok(errors.into())
     }
 
+    #[napi(catch_unwind)]
+    pub async fn function_column_statuses(
+        &self,
+        column: Option<String>,
+    ) -> napi::Result<Vec<FunctionColumnStatus>> {
+        let statuses = self
+            .inner_ref()?
+            .function_column_statuses(column.as_deref())
+            .await
+            .default_error()?;
+        Ok(statuses.into_iter().map(Into::into).collect())
+    }
+
+    #[napi(catch_unwind)]
+    pub async fn function_column_status(
+        &self,
+        column: String,
+    ) -> napi::Result<FunctionColumnStatus> {
+        let status = self
+            .inner_ref()?
+            .function_column_status(&column)
+            .await
+            .default_error()?;
+        Ok(status.into())
+    }
+
+    #[napi(catch_unwind)]
+    pub async fn wait_for_function_column(
+        &self,
+        column: String,
+        timeout_seconds: Option<i64>,
+    ) -> napi::Result<FunctionColumnStatus> {
+        let timeout = timeout_seconds
+            .map(|seconds| {
+                u64::try_from(seconds)
+                    .map(std::time::Duration::from_secs)
+                    .map_err(|_| {
+                        napi::Error::from_reason("timeoutSeconds must be a non-negative integer")
+                    })
+            })
+            .transpose()?;
+        let status = self
+            .inner_ref()?
+            .wait_for_function_column(&column, timeout)
+            .await
+            .default_error()?;
+        Ok(status.into())
+    }
+
     #[napi(catch_unwind)]
     pub async fn refresh_materialized_view(
         &self,
@@ -1799,6 +1848,108 @@ pub struct FunctionErrors {
     pub truncated: bool,
 }
 
+/// One exact immutable Function revision attached to a table output.
+#[napi(object)]
+#[derive(Clone, Debug)]
+pub struct FunctionColumnVersionStatus {
+    pub name: String,
+    pub namespace_path: Vec<String>,
+    pub object_id: String,
+    pub location: String,
+    pub version: String,
+    pub manifest_digest: String,
+}
+
+/// Latest matching refresh Job projected for a Function output.
+#[napi(object)]
+#[derive(Clone, Debug)]
+pub struct FunctionColumnLastRefreshJob {
+    pub job_id: String,
+    pub state: String,
+    pub automatic: Option<bool>,
+    pub completed_at_millis: Option<BigInt>,
+}
+
+/// Automatic-refresh projection for a Function output.
+#[napi(object)]
+#[derive(Clone, Debug)]
+pub struct FunctionColumnAutoRefreshStatus {
+    pub enabled: bool,
+    pub interval_seconds: Option<BigInt>,
+    pub active_job_id: Option<String>,
+    pub state: String,
+    pub failure_streak: u32,
+    pub effective_cooldown_millis: Option<BigInt>,
+    pub blocked_reason: Option<String>,
+    pub next_run_at_millis: Option<BigInt>,
+    pub next_run_at_reason: Option<String>,
+}
+
+/// Exact four-way status of one Function output at one table version.
+#[napi(object)]
+#[derive(Clone, Debug)]
+pub struct FunctionColumnStatus {
+    pub column: String,
+    pub binding_id: String,
+    pub function: FunctionColumnVersionStatus,
+    pub table_version: BigInt,
+    pub rows: BigInt,
+    pub fresh: BigInt,
+    pub stale: BigInt,
+    pub never_computed: BigInt,
+    pub failed: BigInt,
+    pub last_refresh_job: Option<FunctionColumnLastRefreshJob>,
+    pub auto_refresh: FunctionColumnAutoRefreshStatus,
+}
+
+impl From<lancedb::table::FunctionColumnStatus> for FunctionColumnStatus {
+    fn from(status: lancedb::table::FunctionColumnStatus) -> Self {
+        Self {
+            column: status.column,
+            binding_id: status.binding_id,
+            function: FunctionColumnVersionStatus {
+                name: status.function.name,
+                namespace_path: status.function.namespace_path,
+                object_id: status.function.object_id,
+                location: status.function.location,
+                version: status.function.version,
+                manifest_digest: status.function.manifest_digest,
+            },
+            table_version: BigInt::from(status.table_version),
+            rows: BigInt::from(status.rows),
+            fresh: BigInt::from(status.fresh),
+            stale: BigInt::from(status.stale),
+            never_computed: BigInt::from(status.never_computed),
+            failed: BigInt::from(status.failed),
+            last_refresh_job: status
+                .last_refresh_job
+                .map(|job| FunctionColumnLastRefreshJob {
+                    job_id: job.job_id,
+                    state: job.state,
+                    automatic: job.automatic,
+                    completed_at_millis: job.completed_at_millis.map(BigInt::from),
+                }),
+            auto_refresh: FunctionColumnAutoRefreshStatus {
+                enabled: status.auto_refresh.enabled,
+                interval_seconds: status.auto_refresh.interval_seconds.map(BigInt::from),
+                active_job_id: status.auto_refresh.active_job_id,
+                state: status.auto_refresh.state.as_str().to_string(),
+                failure_streak: status.auto_refresh.failure_streak,
+                effective_cooldown_millis: status
+                    .auto_refresh
+                    .effective_cooldown_millis
+                    .map(BigInt::from),
+                blocked_reason: status
+                    .auto_refresh
+                    .blocked_reason
+                    .map(|reason| reason.as_str().to_string()),
+                next_run_at_millis: status.auto_refresh.next_run_at_millis.map(BigInt::from),
+                next_run_at_reason: status.auto_refresh.next_run_at_reason,
+            },
+        }
+    }
+}
+
 impl From<lancedb::function::FunctionErrors> for FunctionErrors {
     fn from(errors: lancedb::function::FunctionErrors) -> Self {
         Self {