feat!: replace get_job/job_history with describe_job/query_job_events (#4130)

A 1M-row column refresh over 200 fragments produced no visible result,
and the client could only ever say `"running"`. Everything needed to
diagnose it already existed server-side — the job registry records a
`claim`/`claim_complete` pair per fragment carrying `rows_processed` —
but none of it was reachable.

## Before

Four ways to ask about a job, none of which told you much.

```python
job = table.refresh_column_async("embedding")
job.status()               # "running". That was the entire debug surface.
db.get_job(job_id)         # state, and a spec. No result, no progress.
db.job_history(job_id)     # raw record batches, no limit, no filter
db.job(job_id)             # a handle that knew nothing
```

## After

Open a job the way you open a table; the handle answers everything.

```python
job = db.open_job(job_id)      # raises JobNotFoundError if there is no such job
```

```python
>>> print(job)
Job(
    id='job-1',
    state='failed',
    job_type='refresh_column',
    creation_ms=1757000000000,
    spec={
        "column": "embedding",
        "num_workers": 4
    },
    failure=JobFailureInfo(phase='execute', message='worker died', retryable=True),
)
```

Individual fields are there too — `job.state`, `job.job_type`,
`job.creation_ms`, `job.spec`, `job.result`, `job.failure` — and
`job.result` carries `rows_assigned` / `rows_failed` as soon as the job
succeeds, with no `wait()` required.

Per-fragment progress *while it is still running*:

```python
done = job.events(filter="state = 'claim_complete'", limit=10_000)
done.column("rows_processed").to_pylist()      # [5000, 5000, ...]
```

The handle an async action returns is the same object, one `refresh()`
away:

```python
job = table.refresh_column_async("embedding")
job.refresh()
job.state, job.result
```

TypeScript is the same experience, down to `console.log`:

```ts
const job = await db.openJob(jobId);   // rejects if there is no such job
console.log(job);                      // same multi-line layout
job.state; job.jobType; job.spec; job.result; job.failure;
const done = await job.events({ filter: "state = 'claim_complete'", limit: 10_000 });
```

## Why each piece matters

- **A result without waiting.** `rows_assigned` / `rows_failed` used to
live only on the terminal result, so a job that never terminated
reported nothing at all.
- **`limit`.** The server caps event rows at 1000 and truncates without
saying so, which silently hid most of a 200-fragment job's history.
- **`filter`.** `claim_complete` rows carry per-claim `rows_processed` —
the only progress signal that exists mid-flight.
- **Events outlive the worker.** They live in the job registry, not in
pod logs that vanish with the pod.
- **One place to ask.** `open_job` replaces `describe_job`,
`query_job_events` and `job`, so a question about a job has one answer
instead of one per calling location.
- **A missing job is an error, not a `None`.** The common case is a job
id copied out of a log, where absence is the surprise worth raising —
and it matches `open_table`.
- **Printing is the debug surface.** Every field on its own line, JSON
payloads keeping their structure. An unrefreshed handle stays on one
line, because there is nothing to lay out.
- **In-process jobs say so.** A local refresh reports `state` and leaves
the rest null rather than inventing fields it has no record for.

`list_jobs` and `cancel_job` stay as they were: one lists, the other is
a one-shot action that should not need a describe first.

## Breaking

All shipped in 0.38.0. No deprecated aliases.

| Was | Now |
| --- | --- |
| `Connection.get_job` → `describe_job` | `Connection.open_job` returns
a populated `Job`, or raises |
| `Connection.job_history` → `query_job_events` | `job.events(...)` |
| `Connection.job` | `Connection.open_job` |
| Python events → `List[pa.RecordBatch]` | `pa.Table` |
| `JobDescription.spec_json` / `.result_json` | internal; use `job.spec`
/ `job.result` |

Node's `Job` is now a TypeScript class wrapping the native handle, so it
returns an Arrow table and parsed values like Python does. New
`Error::JobNotFound` / `JobNotFoundError`; the three job exceptions are
now in the Python API reference.

nodejs/src/connection.rs

@@ -442,13 +442,15 @@ impl Connection {
         self.get_inner()?.drop_all_tables(&ns).await.default_error()
     }
 
-    /// A `Job` handle for a server-side job by id.
+    /// Open a server-side job by id, returning a handle with its record
+    /// already populated. Rejects when the server has no such job.
     ///
-    /// The handle is constructed without a server round trip; an unknown id
-    /// surfaces when the handle is used.
-    #[napi]
-    pub fn job(&self, job_id: String) -> napi::Result<crate::job::Job> {
-        let job = self.get_inner()?.job(job_id).default_error()?;
+    /// The returned handle answers for its own state, specification, result,
+    /// failure and event history, so there is no separate connection-level
+    /// call for any of them.
+    #[napi(catch_unwind)]
+    pub async fn open_job(&self, job_id: String) -> napi::Result<crate::job::Job> {
+        let job = self.get_inner()?.open_job(&job_id).await.default_error()?;
         Ok(crate::job::Job::new(job))
     }
 
@@ -459,17 +461,6 @@ impl Connection {
         Ok(jobs.into_iter().map(Into::into).collect())
     }
 
-    /// Describe a single server-side job by id. `null` when the server has
-    /// no such job.
-    #[napi(catch_unwind)]
-    pub async fn get_job(
-        &self,
-        job_id: String,
-    ) -> napi::Result<Option<crate::job::JobDescription>> {
-        let description = self.get_inner()?.get_job(&job_id).await.default_error()?;
-        Ok(description.map(Into::into))
-    }
-
     /// Request cancellation of a server-side job by id. Returns true if the
     /// server accepted the cancellation, false if no such job exists.
     #[napi(catch_unwind)]
@@ -477,34 +468,6 @@ impl Connection {
         self.get_inner()?.cancel_job(&job_id).await.default_error()
     }
 
-    /// The lifecycle event history of a server-side job (all jobs when
-    /// `job_id` is null), as an Arrow IPC stream buffer. Empty when there is
-    /// no history.
-    #[napi(catch_unwind)]
-    pub async fn job_history(&self, job_id: Option<String>) -> napi::Result<Buffer> {
-        let batches = self
-            .get_inner()?
-            .job_history(job_id.as_deref())
-            .await
-            .default_error()?;
-        let Some(first) = batches.first() else {
-            return Ok(Buffer::from(Vec::<u8>::new()));
-        };
-        let mut out = Vec::new();
-        let mut writer = arrow_ipc::writer::StreamWriter::try_new(&mut out, &first.schema())
-            .map_err(|e| napi::Error::from_reason(e.to_string()))?;
-        for batch in &batches {
-            writer
-                .write(batch)
-                .map_err(|e| napi::Error::from_reason(e.to_string()))?;
-        }
-        writer
-            .finish()
-            .map_err(|e| napi::Error::from_reason(e.to_string()))?;
-        drop(writer);
-        Ok(Buffer::from(out))
-    }
-
     #[napi(catch_unwind)]
     /// Describe a namespace and return its properties.
     pub async fn describe_namespace(

nodejs/src/job.rs

@@ -3,6 +3,9 @@
 
 use std::sync::Arc;
 
+use arrow_array::RecordBatch;
+use lancedb::job::JobEventsRequest;
+use napi::bindgen_prelude::Buffer;
 use napi_derive::napi;
 
 use crate::error::NapiErrorExt;
@@ -55,12 +58,98 @@ impl Job {
     pub async fn cancel(&self) -> napi::Result<()> {
         self.inner.cancel().await.default_error()
     }
+
+    /// Ask the backend for this job's current state, and for a server-side job
+    /// its full record, then cache it for the getters below.
+    ///
+    /// They are all null until this runs, because submitting an operation
+    /// returns only a job id. {@link Job.status} fetches the whole record too;
+    /// {@link Job.wait} records only the terminal state it establishes.
+    #[napi(catch_unwind)]
+    pub async fn refresh(&self) -> napi::Result<()> {
+        self.inner.refresh().await.default_error()
+    }
+
+    /// The last observed lifecycle state, without contacting the backend.
+    #[napi(getter)]
+    pub fn state(&self) -> Option<String> {
+        self.inner.state()
+    }
+
+    /// The job's type, as the server names it. Null for an in-process job,
+    /// which has no server-side record.
+    #[napi(getter)]
+    pub fn job_type(&self) -> Option<String> {
+        self.inner.job_type()
+    }
+
+    /// When the job was created, in milliseconds since the epoch.
+    #[napi(getter)]
+    pub fn creation_ms(&self) -> Option<i64> {
+        self.inner.creation_ms()
+    }
+
+    /// The job-type-specific specification as a JSON string, when present.
+    #[napi(getter)]
+    pub fn spec_json(&self) -> Option<String> {
+        self.inner.spec().map(|spec| spec.to_string())
+    }
+
+    /// The job-type-specific terminal result as a JSON string. Null until the
+    /// job succeeds, so a job that never terminates reports its progress
+    /// through {@link Job.events} instead.
+    #[napi(getter)]
+    pub fn result_json(&self) -> Option<String> {
+        self.inner.result().map(|result| result.to_string())
+    }
+
+    /// Why the job failed, when it failed and the server reports a reason.
+    #[napi(getter)]
+    pub fn failure(&self) -> Option<JobFailureInfo> {
+        self.inner.failure().map(|failure| JobFailureInfo {
+            phase: failure.phase,
+            message: failure.message,
+            retryable: failure.retryable,
+        })
+    }
+
+    /// This job's recorded lifecycle events, as an Arrow IPC stream buffer.
+    /// The TypeScript wrapper turns it into an Arrow table.
+    #[napi(catch_unwind)]
+    pub async fn events(&self, limit: Option<u32>, filter: Option<String>) -> napi::Result<Buffer> {
+        let batches = self
+            .inner
+            .events(JobEventsRequest { limit, filter })
+            .await
+            .default_error()?;
+        batches_to_ipc_buffer(&batches)
+    }
+}
+
+/// Serialise Arrow batches as a single IPC stream for the TypeScript layer.
+pub(crate) fn batches_to_ipc_buffer(batches: &[RecordBatch]) -> napi::Result<Buffer> {
+    let Some(first) = batches.first() else {
+        return Ok(Buffer::from(Vec::<u8>::new()));
+    };
+    let mut out = Vec::new();
+    let mut writer = arrow_ipc::writer::StreamWriter::try_new(&mut out, &first.schema())
+        .map_err(|e| napi::Error::from_reason(e.to_string()))?;
+    for batch in batches {
+        writer
+            .write(batch)
+            .map_err(|e| napi::Error::from_reason(e.to_string()))?;
+    }
+    writer
+        .finish()
+        .map_err(|e| napi::Error::from_reason(e.to_string()))?;
+    drop(writer);
+    Ok(Buffer::from(out))
 }
 
 /// A row from `Connection.listJobs`: one server-side job.
 #[napi(object)]
 pub struct JobInfo {
-    /// The job id -- what `Connection.getJob` and `Connection.cancelJob`
+    /// The job id -- what `Connection.openJob` and `Connection.cancelJob`
     /// accept.
     pub job_id: String,
     /// The table the job runs against, without URI or namespace.
@@ -91,36 +180,3 @@ pub struct JobFailureInfo {
     pub message: Option<String>,
     pub retryable: Option<bool>,
 }
-
-/// A described job from `Connection.getJob`.
-#[napi(object)]
-pub struct JobDescription {
-    pub job_id: String,
-    pub job_type: String,
-    /// Lifecycle state: "running", "finished", "failed", or "cancelled".
-    pub state: String,
-    /// When the job was created, in milliseconds since the epoch.
-    pub creation_ms: i64,
-    /// The job-type-specific specification as a JSON string, when present.
-    pub spec_json: Option<String>,
-    /// Why the job failed, when the job is failed and the server reports a
-    /// reason.
-    pub failure: Option<JobFailureInfo>,
-}
-
-impl From<lancedb::database::JobDescription> for JobDescription {
-    fn from(description: lancedb::database::JobDescription) -> Self {
-        Self {
-            job_id: description.job_id,
-            job_type: description.job_type,
-            state: description.state,
-            creation_ms: description.creation_ms,
-            spec_json: (!description.spec.is_null()).then(|| description.spec.to_string()),
-            failure: description.failure.map(|failure| JobFailureInfo {
-                phase: failure.phase,
-                message: failure.message,
-                retryable: failure.retryable,
-            }),
-        }
-    }
-}