Empty local TypeScript queries returned a zero-row Arrow table with no
columns because `toArrow()` built the table solely from record batches.
Local streams can yield no batches, while remote streams often supply an
empty batch carrying the schema.
Expose the executed stream's schema through the native iterator and
decode it into an empty Arrow table when no batches arrive. This
preserves projected and search columns, types, nullability, and metadata
without executing another query. Document the empty-result behavior.
Validation:
- Eleven regression cases failed with the original `toArrow()`
implementation; all twelve now pass, including remote schema-only and
empty-batch responses.
- All 1,075 tests in the query, remote, Arrow, and table suites pass.
- Native build, TypeScript compilation, Rust formatting and Clippy (`-D
warnings`), TypeScript lint/formatting, and documentation generation
pass.
Fixes#4386
<!-- lance-gatekeeper-fix:v1 agent=7d589da1f7db5c7383f9600490e111a9
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Thanks @vincentkoc for starting this! I was just looking into this
myself. I started with your PR and added:
- make arrow 21 the default
- widen the peer range to <22 so if they release 21.3 that'll be
included
- have to do skipLibCheck in tsconfig.json, because of flatbuffers 25
which needs TypeScript 5.7
- fix override in Symbol.hasInstance
- support new types in Arrow 20-21 (LargeList, Utf8View, BinaryView,
IntervalMonthDayNano)
- arrow-compat job in nodejs.yml that runs tests with arrow 19 and 20
(adds 2 sec)
- put the list of arrow types in one place
- update NODEJS_THIRD_PARTY_LICENSES.md
- regenerated typescript docs (sorry about reorderings, one time cost)
- reformat arrow.test.ts (sorry, almost all whitespace, thanks to biome.
git diff -w shows 110 lines edited)
- fixed Sliced Utf8View / BinaryView serialization error, by just
converting them to Utf8 and Binary (as our rust code would anyway)
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Adds client-side support for analyze_plan distributed metrics modes
across Rust, Python, and TypeScript clients. Defaults to aggregate for
backward compatibility and sends the remote distributed_metrics
parameter only when a non-default mode is requested.
JS native Async Generator, more efficient asynchronous iteration, fewer
synthetic promises, and the ability to handle `catch` or `break` of
parent loop in `finally` block
These operations have existed in lance for a long while and many users
need to drop down to lance for this capability. This PR adds the API and
implements it using filters (e.g. `_rowid IN (...)`) so that in doesn't
currently add any load to `BaseTable`. I'm not sure that is sustainable
as base table implementations may want to specialize how they handle
this method. However, I figure it is a good starting point.
In addition, unlike Lance, this API does not currently guarantee
anything about the order of the take results. This is necessary for the
fallback filter approach to work (SQL filters cannot guarantee result
order)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated dependency versions for improved performance and
compatibility.
- **New Features**
- Added support for structured full-text search with expanded query
types (e.g., match, phrase, boost, multi-match) and flexible input
formats.
- Introduced a new method to check server support for structural
full-text search features.
- Enhanced the query system with new classes and interfaces for handling
various full-text queries.
- Expanded the functionality of existing methods to accept more complex
query structures, including updates to method signatures.
- **Bug Fixes**
- Improved error handling and reporting for full-text search queries.
- **Refactor**
- Enhanced query processing with streamlined input handling and improved
error reporting, ensuring more robust and consistent search results
across platforms.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
Co-authored-by: BubbleCal <bubble-cal@outlook.com>
add analyze plan api to allow executing the queries and see runtime
metrics.
Which help identify the query IO overhead and help identify query
slowness
* Make `npm run docs` fail if there are any warnings. This will catch
items missing from the API reference.
* Add a check in our CI to make sure `npm run dos` runs without warnings
and doesn't generate any new files (indicating it might be out-of-date.
* Hide constructors that aren't user facing.
* Remove unused enum `WriteMode`.
Closes#2068
* Sets `"useCodeBlocks": true`
* Adds a post-processing script `nodejs/typedoc_post_process.js` that
puts the parameter description on the same line as the parameter name,
like it is in our Python docs. This makes the text hierarchy clearer in
those sections and also makes the sections shorter.
We aren't yet ready to switch over the examples since almost all JS
examples rely on embeddings and we haven't yet ported those over.
However, this makes it possible for those that are interested to start
using `@lancedb/lancedb`
lancedb-gatefixer[bot]
Jack Ye
S.A.N
Weston Pace
LuQQiu
Will Jones
Cory GrinsteadandWill Jones