mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-14 06:36:54 +00:00
9e65bd2c272aac7016cdadbf0b5f6c224ba61c02
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  TaylorandBrendan Clement
|
9e65bd2c27 |
docs: use Enterprise terminology across website and SDK documentation (#4787)
Use “LanceDB Enterprise” throughout the staged website guides, SDK reference sources, generated API pages, and user-facing SDK messages. A separate branded “LanceDB Distributed” name adds terminology without clarifying behavior; describe where operations run directly. - Update `docs/web`, including the architecture diagram title and alt text. Use Enterprise in headings, table labels, and prose; simplify redundant product/execution-model wording and name the product only once in the diagram alt text. - Update Python docstrings and the Python/Java reference pages, TypeScript comments and native declaration comments, and Rust API documentation. Regenerate the TypeScript reference, including 24 replacements across six pages. - Update SDK warnings/errors and existing test expectations to match, along with contributor guidance and Vale rules. Remove the obsolete rule distinguishing Enterprise from Distributed. - Keep lowercase “distributed” for technical descriptions. API identifiers, anchors, asset paths, and execution behavior are unchanged. No branded-name exceptions are proposed. Validation: - Python and Node native builds succeeded; focused Python tests passed (36 passed, 4 skipped), and focused Node tests passed (14). - TypeDoc generation and the complete MkDocs reference build passed. MkDocs retains an existing unresolved `lancedb.functions.SecretReference` cross-reference. - Rust formatting, Node lint/format, changed-file Python lint/format, 32 Vale tests, and whitespace checks passed. Repository-wide Python lint has 48 existing findings in six untouched files. - Combined documentation assembly passed. Mintlify validation and link/anchor/redirect checks previously passed for both site builds; no code snippets changed. Companion PRs: [saved editions, terminology guide, and shared rules](https://github.com/lancedb/docs/pull/413), [Enterprise overview and writing rules](https://github.com/lancedb/sophon/pull/9511). --------- Co-authored-by: Brendan Clement <brendan.clement@lancedb.com> |
||
  Will JonesandClaude Opus 5.5
|
4d7a64712f |
docs: adopt Embedded/Distributed terminology across docs and SDKs (#4713)
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
  lancedb-gatefixer[bot]andXuanwo
|
2cd635d41c |
fix: correct remote materialized view refresh results and options (#4672)
Remote materialized-view refreshes drained the SQL result and then fabricated `mode=rebuild`, `rows_written=<total view rows>`, and `source_version=0`. This made incremental refreshes report incorrect statistics. Make these three result fields optional across Rust, Python, and TypeScript, and leave them unset for remote SQL refreshes. Remove the post-refresh row count request. Local refreshes continue to populate all metrics, and `version` remains the view version observed after completion. Remote per-operation statistics remain unavailable until the SQL job adapter can retrieve them. Expose the forced-rebuild option through the Rust builder, synchronous/asynchronous Python refresh and job methods, and TypeScript refresh. Remote `full=true` sends `WITH (full = true)`; the default lets the server choose its refresh mode. Local source-version pinning and remote job cancellation are preserved. Local refresh regression tests respect the `experimental-functions` build gate, while remote refresh coverage also runs in default builds. Validation: - Rust: 20 materialized-view tests and 21 remote SQL tests passed with `remote,experimental-functions`; `cargo fmt --all` passed. - Python: rebuilt the extension in the uv-managed environment. The default build passed 23 tests with 27 feature-dependent skips. The opt-in build passed 61 tests with 3 expected default-build-only skips, covering the materialized-view suite and 14 async query-serialization cases from current main. Python Ruff lint and formatting checks passed. - TypeScript: default and opt-in native builds and TypeScript compilation passed. The default build passed 5 tests with 10 expected feature-dependent skips; the opt-in build passed all 14 materialized-view tests with the default-build-only test skipped. Lint/format checks and documentation generation passed. - Repository-wide `ruff check .` reports 48 pre-existing violations in unchanged `docs/web-tests` files; `ruff check python` passes. Fixes #4670 <!-- lance-gatekeeper-fix:v1 agent=ece43ec4b88a48d1789f60005fa3ecfc generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
 Jack Ye
|
7e8f0d4982 |
refactor: always fully refresh materialized views (#4373)
Simplify experimental materialized views to one output-only storage
layout and one full-rebuild refresh implementation.
## Current layout
Local materialized views currently maintain incremental-refresh state
alongside the materialized values:
- an internal `__source_row_id` column for source provenance;
- `mv.definition` plus `mv.incarnation`;
- source version/timestamp, view version, and last-refresh watermarks;
- optional `mv.staging` state for table-function sources; and
- stable-row-ID requirements on the source table.
Refresh classifies source changes, appends or evicts rows when possible,
and falls back to a rebuild for unsupported deltas. Remote
create/refresh also use separate REST mutation paths.
## New layout
An MV is an ordinary Lance backing table containing only:
- the defining query's output columns;
- its current data fragments; and
- `mv.definition`, encoded as a versioned SQL envelope such as
`{"kind":"query","format":1,"query":"SELECT ..."}`.
There is no provenance column, incarnation, source/view watermark,
staging metadata, or stable-row-ID requirement. Local refresh may still
pin a source version for reproducibility, but it always recomputes the
complete result.
Full refresh streams the new result into uncommitted fragments and then
publishes a single transaction that removes every old fragment and
installs the new set. A shared conflict token prevents concurrent
replacements from both committing. This fragment-replacement primitive
is also used by Sophon's SQL worker.
## New remote create workflow
```text
Connection::create_materialized_view(...)
-> builder renders the defining SELECT
-> remote Database receives a CreateMaterializedViewRequest
-> submit CREATE MATERIALIZED VIEW ... AS ... through Flight SQL
-> retain the SQL Query handle while the Job is running
-> consume the durable SQL-job result
-> open the created table and read mv.definition
-> return MaterializedView
```
Remote refresh similarly submits `REFRESH MATERIALIZED VIEW` through
Flight SQL. Cancelling either wrapper forwards cancellation to the
retained SQL query before aborting the local task, so the server
operation is not silently left running.
## API changes
- remove incremental/no-op modes and the `full` switch;
- remove distributed grouped/partitioned refresh APIs and native
vector-source MV APIs;
- keep full-refresh results across Rust, Python, and TypeScript;
- preserve list/describe/drop as object-management operations while
routing create/refresh mutations through SQL.
The storage contract is experimental and has no compatibility shim;
views using the removed layout must be recreated.
|
||
 Wyatt Alt
|
68749ecfa3 |
feat(nodejs): materialized view bindings (#3935)
Exposes materialized views to TypeScript: createMaterializedView,
openMaterializedView and listMaterializedViews on Connection, and a
MaterializedView handle carrying the parsed definition and
refresh({full, sourceVersion}), which returns the typed refresh result.
select accepts column names, [alias, expression] pairs, or a record of
the
same; the definition reads back off the stored schema, so a reopened
handle
needs no side channel. Remote connections surface the core's
not-supported
error up front.
The napi crate needed the same recursion-limit raise as the core crate:
the
refresh future's type graph overflows the default trait-recursion depth.
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
|