lancedb

mirror of https://github.com/lancedb/lancedb.git synced 2026-05-19 04:50:40 +00:00

Files

Heng Ge 0d30b31998 feat: support setting LSM write spec for a table (#3396 )

## Summary

Split out from #3354

Adds `LsmWriteSpec` and `Table::set_lsm_write_spec` /
`unset_lsm_write_spec` to
install and clear the spec that selects Lance's MemWAL LSM-style write
path for
`merge_insert`.

`LsmWriteSpec` offers three sharding strategies, all built on Lance's
`InitializeMemWalBuilder`:

- `LsmWriteSpec::bucket(column, num_buckets)` — hash-bucket sharding by
the
  single-column unenforced primary key.
- `LsmWriteSpec::identity(column)` — identity sharding by the raw value
of a
  scalar column.
- `LsmWriteSpec::unsharded()` — a single MemWAL shard.

Each can be refined with `with_maintained_indexes(...)` (indexes the
MemWAL
keeps up to date as rows are appended) and
`with_writer_config_defaults(...)`
(default `ShardWriter` configuration recorded in the MemWAL index, so
every
writer starts from the same defaults). All variants require the table to
have
an unenforced primary key.

- `set_lsm_write_spec` installs the spec by initializing the MemWAL
index;
`unset_lsm_write_spec` removes it (dropping the MemWAL index), reverting
to
  the standard `merge_insert` path. `unset` is idempotent.
- Bindings: Python (`LsmWriteSpec.bucket` / `.identity` / `.unsharded`,
  `set_lsm_write_spec` / `unset_lsm_write_spec`) and TypeScript
  (`setLsmWriteSpec` with `specType` `"bucket"` / `"identity"` /
  `"unsharded"`). `RemoteTable` returns `NotSupported`.

The actual `merge_insert` LSM dispatch and `ShardWriter` write path are
a
follow-up — this PR only installs and clears the spec.

2026-05-18 00:11:33 -07:00

arrow.test.ts

feat: infer js native arrays (#3119 )

2026-03-09 10:13:59 -07:00

connection.test.ts

feat(nodejs): add Connection.renameTable with namespace support (#3365 )

2026-05-14 15:30:31 -07:00

embedding.test.ts

fix(node): handle undefined vector fields with embedding functions (#2655 )

2025-09-19 09:17:28 -07:00

permutation.test.ts

feat: add python Permutation class to mimic hugging face dataset and provide pytorch dataloader (#2725 )