Files
lancedb/nodejs/__test__
lancedb-gatefixer[bot] 3348087d09 fix(node): accept code as a base tokenizer (#4368)
TypeScript rejected `Index.fts({ baseTokenizer: "code" })` because the
public `BaseTokenizer` union omitted `"code"`, although the native
tokenizer already supports it.

Add `"code"` to the shared union, document its identifier behavior, and
regenerate the API reference. The regression test creates and searches a
code-tokenized FTS index with a typed configuration across Arrow
versions 15–18.

Validation:

- The regression failed before the fix with TS2322: `"code"` was not
assignable to `BaseTokenizer`.
- After the fix, all four cases pass: `pnpm test __test__/table.test.ts
--runInBand -t 'full text search code tokenizer' --verbose=false`.
- `pnpm build`, `pnpm lint`, `pnpm run docs`, and formatting of the
changed TypeScript files pass.

Fixes #4366

<!-- lance-gatekeeper-fix:v1 agent=8c37ac529cec8862bd5dbc218c5aaa8d
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
2026-10-08 19:46:02 -07:00
..