mirror of https://github.com/lancedb/lancedb.git synced 2026-01-03 18:32:55 +00:00

Files

Ryan Green af54e0ce06 feat: add table stats API (#2363 )

* Add a new "table stats" API to expose basic table and fragment
statistics with local and remote table implementations

### Questions
* This is using `calculate_data_stats` to determine total bytes in the
table. This seems like a potentially expensive operation - are there any
concerns about performance for large datasets?

### Notes
* bytes_on_disk seems to be stored at the column level but there does
not seem to be a way to easily calculate total bytes per fragment. This
may need to be added in lance before we can support fragment size
(bytes) statistics.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Added a method to retrieve comprehensive table statistics, including
total rows, index counts, storage size, and detailed fragment size
metrics such as minimum, maximum, mean, and percentiles.
- Enabled fetching of table statistics from remote sources through
asynchronous requests.
- Extended table interfaces across Python, Rust, and Node.js to support
synchronous and asynchronous retrieval of table statistics.
- **Tests**
- Introduced tests to verify the accuracy of the new table statistics
feature for both populated and empty tables.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

2025-04-29 15:19:08 -02:30

481 B

Raw Blame History

@lancedb/lancedb • Docs

@lancedb/lancedb / FragmentStatistics

Interface: FragmentStatistics

Properties

lengths

lengths: FragmentSummaryStats;

Statistics on the number of rows in the table fragments

numFragments

numFragments: number;

The number of fragments in the table

numSmallFragments

numSmallFragments: number;

The number of uncompacted fragments in the table

481 B Raw Blame History