1172 Commits

Author	SHA1	Message	Date
fys	c4a7cc0adb	chore: improve create trigger display (#7027) ... * chore: improve create_trigger_statement display * improve display of create trigger * add components for frontend * Revert "add components for frontend" This reverts commit 8d71540a72.	2025-09-29 02:22:43 +00:00
discord9	aca8b690d1	fix: step aggr merge phase not order nor filter (#6998) ... * fix: not order Signed-off-by: discord9 <discord9@163.com> * test: redacted Signed-off-by: discord9 <discord9@163.com> * feat: fix up state wrapper Signed-off-by: discord9 <discord9@163.com> * df last_value state not as promised! Signed-off-by: discord9 <discord9@163.com> * fix?: could fix better Signed-off-by: discord9 <discord9@163.com> * test: unstable result Signed-off-by: discord9 <discord9@163.com> * fix: work around by fixing state Signed-off-by: discord9 <discord9@163.com> * chore: after rebase fix Signed-off-by: discord9 <discord9@163.com> * chore: finish some todo Signed-off-by: discord9 <discord9@163.com> * chore: per copilot Signed-off-by: discord9 <discord9@163.com> * refactor: not fix but just notify mismatch Signed-off-by: discord9 <discord9@163.com> * chore: warn -> debug state mismatch Signed-off-by: discord9 <discord9@163.com> * chore: refine error msg Signed-off-by: discord9 <discord9@163.com> * test: sqlness add last_value date_bin test Signed-off-by: discord9 <discord9@163.com> * ?: substrait order by decode failure Signed-off-by: discord9 <discord9@163.com> * unit test reproduce that Signed-off-by: discord9 <discord9@163.com> * feat: support state wrapper's order serde in substrait Signed-off-by: discord9 <discord9@163.com> * refactor: stuff Signed-off-by: discord9 <discord9@163.com> * test: standalone/distributed different exec Signed-off-by: discord9 <discord9@163.com> * fmt Signed-off-by: discord9 <discord9@163.com> * chore: per review Signed-off-by: discord9 <discord9@163.com> * refactor: closure Signed-off-by: discord9 <discord9@163.com> * test: first value order by Signed-off-by: discord9 <discord9@163.com> * refactor: per cr Signed-off-by: discord9 <discord9@163.com> * feat: ScanHint last_value last row selector Signed-off-by: discord9 <discord9@163.com> * docs: per cr Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-09-26 09:12:45 +00:00
Ning Sun	c1acce9943	refactor: cleanup datafusion-pg-catalog dependencies (#7025) ... * refactor: cleanup datafusion-pg-catalog dependencies Signed-off-by: Ning Sun <sunning@greptime.com> * chore: toml format * feat: update upstream --------- Signed-off-by: Ning Sun <sunning@greptime.com>	2025-09-26 03:07:00 +00:00
Ning Sun	964dc254aa	feat: upgraded pg_catalog support (#6918) ... * refactor: add datafusion-postgres dependency * refactor: move and include pg_catalog udfs * chore: update upstream * feat: register table function pg_get_keywords * feat: bridge CatalogInfo for our CatalogManager Signed-off-by: Ning Sun <sunning@greptime.com> * feat: convert pg_catalog table to our system table * feat: bridge system catalog with datafusion-postgres Signed-off-by: Ning Sun <sunning@greptime.com> * feat: add more udfs * feat: add compatibility rewriter to postgres handler * fix: various fix * fmt: fix * fix: use functions from pg_catalog library * fmt * fix: sqlness runner Signed-off-by: Ning Sun <sunning@greptime.com> * test: adopt arrow 56.0 to 56.1 memory size change * fix: add additional udfs * chore: format * refactor: return None when creating system table failed Signed-off-by: Ning Sun <sunning@greptime.com> * chore: provide safety comments about expect usage --------- Signed-off-by: Ning Sun <sunning@greptime.com>	2025-09-25 04:05:34 +00:00
Weny Xu	0c038f755f	refactor(cli): refactor object storage config (#7009) ... * refactor: refactor object storage config Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: public common config Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-09-24 06:50:47 +00:00
Weny Xu	b3d413258d	feat: extract standalone functionality and introduce plugin-based router configuration (#7002) ... * feat: extract standalone functionality and introduce plugin-based router configuration Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: ensure dump file does not exist Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: introduce `External` error Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-09-22 11:21:04 +00:00
discord9	03954e8b3b	chore: update proto (#6992) ... * chore: update proto Signed-off-by: discord9 <discord9@163.com> * update lockfile Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-09-19 09:01:46 +00:00
discord9	e26b98f452	refactor: put FileId to store-api (#6988) ... * refactor: put FileId to store-api Signed-off-by: discord9 <discord9@163.com> * per review Signed-off-by: discord9 <discord9@163.com> * chore: lock file Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-09-18 03:20:42 +00:00
Lei, HUANG	c92ab4217f	fix: avoid truncating SST statistics during flush (#6977) ... fix/disable-parquet-stats-truncate: - **Update `memcomparable` Dependency**: Switched from crates.io to a Git repository for `memcomparable` in `Cargo.lock`, `mito-codec/Cargo.toml`, and removed it from `mito2/Cargo.toml`. - **Enhance Parquet Writer Properties**: Added `set_statistics_truncate_length` and `set_column_index_truncate_length` to `WriterProperties` in `parquet.rs`, `bulk/part.rs`, `partition_tree/data.rs`, and `writer.rs`. - **Add Test for Corrupt Scan**: Introduced a new test module `scan_corrupt.rs` in `mito2/src/engine` to verify handling of corrupt data. - **Update Test Data**: Modified test data in `flush.rs` to reflect changes in file sizes and sequences. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>	2025-09-17 03:02:52 +00:00
Ruihang Xia	83290be3ba	feat: store partition expr per file in region manifest (#6849) ... * feat: store partition expr per file in region manifest Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * use deserialized form in memory Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * test non-existing partition expr Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * try remove partition expr dup Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * Revert "try remove partition expr dup" This reverts commit a0f2c8ca6a127ef356a5510466e84513cf4b13d8. * fix merge test Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * format doc Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2025-09-09 23:37:29 +00:00
shuiyisong	ac051af201	feat(pipeline): generate create table sql from pipeline config (#6930) ... * chore: add expr_to_create function using cc and polish Signed-off-by: shuiyisong <xixing.sys@gmail.com> * feat: add get pipeline create table Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: update test_expr_to_create Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: column extension Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: add message hint for variant table name Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: add message hint for existing table Signed-off-by: shuiyisong <xixing.sys@gmail.com> * test: create table for pipeline Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: update comment Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: rename to query pipeline ddl Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com>	2025-09-09 02:47:51 +00:00
Weny Xu	16febbd4c2	feat: add CPU, memory and node status info to cluster_info (#6897) ... * feat: add CPU and memory info to `cluster_info` Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: add `node_status` to `cluster_info` table Signed-off-by: WenyXu <wenymedia@gmail.com> * test: update sqlness Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: update proto Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-09-08 08:59:34 +00:00
LFC	48f5db3f5f	refactor: use DataFusion's Signature directly in UDF (#6908) ... * refactor: use DataFusion's Signature directly in UDF Signed-off-by: luofucong <luofc@foxmail.com> * fix sqlness Signed-off-by: luofucong <luofc@foxmail.com> * resolve PR comments Signed-off-by: luofucong <luofc@foxmail.com> * fix ci Signed-off-by: luofucong <luofc@foxmail.com> --------- Signed-off-by: luofucong <luofc@foxmail.com>	2025-09-05 05:04:56 +00:00
discord9	f7c8d86ebb	feat: file ref mgr (#6844) ... * feat: file ref manager Signed-off-by: discord9 <discord9@163.com> * refactor: put file ref mgr to sep file Signed-off-by: discord9 <discord9@163.com> * chore: rename path Signed-off-by: discord9 <discord9@163.com> * test: ref path Signed-off-by: discord9 <discord9@163.com> * fix: pass node id Signed-off-by: discord9 <discord9@163.com> * chore: after rebase fix Signed-off-by: discord9 <discord9@163.com> * feat: exlucde already in manifest files Signed-off-by: discord9 <discord9@163.com> * docs: explain why it can work Signed-off-by: discord9 <discord9@163.com> * feat: also include manifest versions Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * rename func to imply what's it actually doing Signed-off-by: discord9 <discord9@163.com> * more docs Signed-off-by: discord9 <discord9@163.com> * docs: expect gc worker to do the job right Signed-off-by: discord9 <discord9@163.com> * refactor: partially per review Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * chore: unused Signed-off-by: discord9 <discord9@163.com> * metrics: change to per datanode instead Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-09-05 03:45:01 +00:00
Ruihang Xia	faae1db066	feat: humanize analyze numbers (#6889) ... * feat: humanize analyze numbers Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * update cargo lock Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * tailing space Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * use readable size Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2025-09-05 03:42:38 +00:00
Weny Xu	2512f9456d	chore: bump main branch version to 0.18 (#6890) ... Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-09-03 07:50:24 +00:00
Zhenchi	9175fa643d	feat: add schema and recordbatch builder for sst entry (#6841) ... * feat: add schema and recordbatch builder for sst entry Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * add build plan Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * address comments Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * address comments Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> --------- Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>	2025-09-01 12:46:49 +00:00
dennis zhuang	1234911ed3	refactor: query config options (#6781) ... * feat: refactor columnar and vector conversion Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat: initialize config options from query context Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: failure tests Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: revert ColumnarValue::try_from_vector Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com>	2025-09-01 07:00:26 +00:00
discord9	bacd9c7d15	feat: add event ts to region manifest (#6751) ... * feat: add event ts to region manifest Signed-off-by: discord9 <discord9@163.com> delete files Signed-off-by: discord9 <discord9@163.com> chore: clippy Signed-off-by: discord9 <discord9@163.com> lower concurrency Signed-off-by: discord9 <discord9@163.com> feat: gc use delta manifest to get expel time Signed-off-by: discord9 <discord9@163.com> docs: terminalogy Signed-off-by: discord9 <discord9@163.com> refactor: some advices from review Signed-off-by: discord9 <discord9@163.com> feat: manifest add removed files field Signed-off-by: discord9 <discord9@163.com> feat(WIP): add remove time in manifest Signed-off-by: discord9 <discord9@163.com> wip: more config Signed-off-by: discord9 <discord9@163.com> feat: manifest uodate removed files Signed-off-by: discord9 <discord9@163.com> test: add remove file opts field Signed-off-by: discord9 <discord9@163.com> test: fix test Signed-off-by: discord9 <discord9@163.com> chore: delete gc.rs Signed-off-by: discord9 <discord9@163.com> * feat: proper option name Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * test: update manifest size Signed-off-by: discord9 <discord9@163.com> * test: fix eq Signed-off-by: discord9 <discord9@163.com> * refactor: some per review Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * refactor: more per review Signed-off-by: discord9 <discord9@163.com> * test: update manifest size Signed-off-by: discord9 <discord9@163.com> * test: update config Signed-off-by: discord9 <discord9@163.com> * feat: keep count 0 means only use ttl Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-08-28 09:08:18 +00:00
discord9	8452a9d579	feat(flow): add eval interval option (#6623) ... * feat: add flow eval interval Signed-off-by: discord9 <discord9@163.com> * feat: tql flow must have eval interval Signed-off-by: discord9 <discord9@163.com> * chore: clippy Signed-off-by: discord9 <discord9@163.com> * test: update sqlness Signed-off-by: discord9 <discord9@163.com> * wip Signed-off-by: discord9 <discord9@163.com> * wip Signed-off-by: discord9 <discord9@163.com> * feat: check for now func Signed-off-by: discord9 <discord9@163.com> * refactor: use ms instead Signed-off-by: discord9 <discord9@163.com> * fix: not panic&proper simplifier Signed-off-by: discord9 <discord9@163.com> * test: update to fix Signed-off-by: discord9 <discord9@163.com> * feat: not allow month in interval Signed-off-by: discord9 <discord9@163.com> * test: update remov months Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * chore: after rebase fix Signed-off-by: discord9 <discord9@163.com> * feat: use seconds and add to field instead Signed-off-by: discord9 <discord9@163.com> * chore: aft rebase fix Signed-off-by: discord9 <discord9@163.com> * fix: add check for month Signed-off-by: discord9 <discord9@163.com> * chore: fmt Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * refactor: rm clone per review Signed-off-by: discord9 <discord9@163.com> * chore: update proto Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-08-27 09:44:32 +00:00
Yan Tingwang	32a3ef36f9	feat(metasrv): support tls for etcd client (#6818) ... * add TLS support for etcd client connections~ Signed-off-by: codephage2020 <tingwangyan2020@163.com> * locate correct certs Signed-off-by: codephage2020 <tingwangyan2020@163.com> * Updated certs Signed-off-by: codephage2020 <tingwangyan2020@163.com> * Updated CI Signed-off-by: codephage2020 <tingwangyan2020@163.com> * Updated CI Signed-off-by: codephage2020 <tingwangyan2020@163.com> * Update docker-compose.yml * tests for TLS client creation Signed-off-by: codephage2020 <tingwangyan2020@163.com> * modify tests Signed-off-by: codephage2020 <tingwangyan2020@163.com> --------- Signed-off-by: codephage2020 <tingwangyan2020@163.com>	2025-08-27 07:41:05 +00:00
Weny Xu	566a647ec7	feat: add replay checkpoint to reduce overhead for remote WAL (#6816) ... * feat: introduce `TopicRegionValue` Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: persist region replay checkpoint Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: introduce checkpoint Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: udpate config.md Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: minor refactor Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: send open region instructions with reply checkpoint Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: use usize Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix unit tests Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix unit tests Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: add topic name pattern Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: enable wal prune by default Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-08-27 07:24:33 +00:00
Weny Xu	b921e41abf	chore: remove unused deps (#6828) ... Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-08-26 12:29:17 +00:00
Arshdeep	8894cb5406	feat: resolve unused dependencies with cargo-udeps (#6578) (#6619) ... * feat:resolve unused dependencies with cargo-udeps (#6578) Signed-off-by: Arshdeep54 <balarsh535@gmail.com> * Apply suggestion from @zyy17 Co-authored-by: zyy17 <zyylsxm@gmail.com> * Apply suggestion from @zyy17 Co-authored-by: zyy17 <zyylsxm@gmail.com> --------- Signed-off-by: Arshdeep54 <balarsh535@gmail.com> Co-authored-by: Ning Sun <classicning@gmail.com> Co-authored-by: zyy17 <zyylsxm@gmail.com>	2025-08-26 10:22:53 +00:00
Lei, HUANG	d99734b97b	perf: sparse encoder (#6809) ... * perf/sparse-encoder: - **Update Dependencies**: Updated `criterion-plot` to version `0.5.0` and added `criterion` version `0.7.0` in `Cargo.lock`. Added `bytes` to `Cargo.toml` in `src/metric-engine`. - **Benchmarking**: Added a new benchmark for sparse encoding in `bench_sparse_encoding.rs` and updated `Cargo.toml` in `src/mito-codec` to include `criterion` as a dev-dependency. - **Sparse Encoding Enhancements**: Modified `SparsePrimaryKeyCodec` in `sparse.rs` to include new methods `encode_raw_tag_value` and `encode_internal`. Added public constants `RESERVED_COLUMN_ID_TSID` and `RESERVED_COLUMN_ID_TABLE_ID`. - **HTTP Server**: Made `try_decompress` function public in `prom_store.rs`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * perf/sparse-encoder: Improve buffer handling in `sparse.rs` - Refactored buffer reservation logic to use `value_len` for clarity. - Optimized chunk processing by calculating `num_chunks` and `remainder` for efficient data handling. - Enhanced manual serialization of bytes to avoid byte-by-byte operations, improving performance. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * Update src/mito-codec/src/row_converter/sparse.rs Co-authored-by: Yingwen <realevenyag@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Co-authored-by: Yingwen <realevenyag@gmail.com>	2025-08-26 04:10:11 +00:00
dennis zhuang	c6c33d14aa	feat: bump opendal to v0.54 (#6792) ... * feat: bump opendal to v0.54.0 Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: update cargo Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com>	2025-08-22 10:51:06 +00:00
dennis zhuang	3014972202	chore: improve error message when there are more than one time index (#6796) ... * chore: improve error message when there are more than one time index Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: style Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com>	2025-08-22 09:22:55 +00:00
Weny Xu	896d72191e	feat: introduce PersistStatsHandler (#6777) ... * feat: add `Inserter` trait and impl Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: import items Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: introduce `PersistStatsHandler` Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: disable persisting stats in sqlness Signed-off-by: WenyXu <wenymedia@gmail.com> * reset channel manager Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: avoid to collect Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: remove insert options Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: use `write_bytes` instead of `write_bytes_per_sec` Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: compute write bytes delta Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * test: add unit tests Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * Update src/meta-srv/src/handler/persist_stats_handler.rs Co-authored-by: Yingwen <realevenyag@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> Co-authored-by: Yingwen <realevenyag@gmail.com>	2025-08-21 09:34:58 +00:00
Weny Xu	9cdd0d8251	feat: derive macro ToRow (#6768) ... * feat: introduce `ToRow` derive marco Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions from CR Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: use map Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: simplify Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-08-19 14:06:03 +00:00
Ning Sun	5a4036cc66	feat: update opentelemetry family (#6762) ... * feat: update opentelemetry family Signed-off-by: Ning Sun <sunning@greptime.com> * doc: update doc samples Signed-off-by: Ning Sun <sunning@greptime.com> * chore: toml format Signed-off-by: Ning Sun <sunning@greptime.com> * chore: update default otel enpoint Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com>	2025-08-19 09:23:50 +00:00
fys	326198162e	refactor: enhanced trigger interval (#6740) ... * refactor: enhance trigger interval * update greptime-proto * fix: build	2025-08-18 04:03:26 +00:00
LFC	f9d2a89a0c	chore: update datafusion family (#6675) ... * chore: update datafusion family Signed-off-by: luofucong <luofc@foxmail.com> * fix ci Signed-off-by: luofucong <luofc@foxmail.com> * use official otel-arrow-rust Signed-off-by: luofucong <luofc@foxmail.com> * rebase Signed-off-by: luofucong <luofc@foxmail.com> * use the official orc-rust Signed-off-by: luofucong <luofc@foxmail.com> * resolve PR comments Signed-off-by: luofucong <luofc@foxmail.com> * remove the empty lines Signed-off-by: luofucong <luofc@foxmail.com> * try following PR comments Signed-off-by: luofucong <luofc@foxmail.com> --------- Signed-off-by: luofucong <luofc@foxmail.com>	2025-08-15 12:41:49 +00:00
Ning Sun	351826cd32	fix: refine shadowrs dependency, remove libgit2, libz and potentally libssl (#6748) ... * feat: update shadown-rs config and remove libgit/libz/libssl deps * chore: remove native inputs from flake	2025-08-15 03:01:14 +00:00
Weny Xu	021ad09c21	refactor: simplify WAL pruning procedure and introduce region flush trigger (#6741) ... * chore: add logs Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: update wal config for metasrv Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: introduce region flush trigger Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: debug assert Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: log level Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: simplify wal prune procedure Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: upgrade rskafka Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: always flush inactive regions Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: refactor flush trigger Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: remove unused code Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: typo Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: update unit tests Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: add metrics Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: rename Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions from CR Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-08-14 14:15:30 +00:00
dennis zhuang	c0fe800e79	feat: improve slow queries options deserialization (#6734) ... * feat: improve slow queries options deserialization Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor: use serde default for struct Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com>	2025-08-14 03:43:13 +00:00
Zhenchi	fb3b1d4866	feat: Store partition expr in RegionMetadata (#6699) ... * wire partition.expr_json option constant and parsing test(mito2): manifest roundtrip persists partition_expr JSON test(mito2): create/open with partition.expr_json persists in manifest docs: add comments for partition.expr_json option and RegionOptions.partition_expr serde: include RegionOptions.partition_expr (skip if None) test(mito2): doc intent and verify runtime backfill + persistence-after-alter for partition expr add partition expr to create request Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * add create_with_partition_expr_persists_manifest Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * pass partition expr to create request Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * polish Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * fix sqlness Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * fix test Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * remove unused dep Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> --------- Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> Co-authored-by: Ruihang Xia <waynestxia@gmail.com>	2025-08-13 22:05:37 +00:00
Weny Xu	8659412cac	feat: introduce PeriodicTopicStatsReporter (#6730) ... * refactor: introduce `PeriodicTopicStatsReporter` Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions from CR Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix typo Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: remote wal tests styling Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix unit test Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: handling region wal options not found Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions from CR Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: minor Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: upgrade greptime-proto Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-08-13 11:46:50 +00:00
Zhenchi	dea87b7e57	refactor: use DummyCatalog to construct query engine for datanode (#6723) ... * refactor: use DummyCatalog to construct query engine for datanode Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * fix clippy Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> * move to query/dummy_catalog Signed-off-by: Zhenchi <zhongzc_arch@outlook.com> --------- Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>	2025-08-13 09:49:51 +00:00
zyy17	777da35b0d	refactor: unify the event recorder (#6689) ... * refactor: unify the event recorder Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: add `table_name()` in `Event` trait Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: add `slow_query_options` in `Instance` Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: add `EventHandlerOptions` and `options()` in `EventHandler` trait Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: add `aggregate_events_by_type()` and support log mode of slow query Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: polish the code Signed-off-by: zyy17 <zyylsxm@gmail.com> * fix: clippy errors Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: support to set ttl by using extension of query context Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: refine the configs fields Signed-off-by: zyy17 <zyylsxm@gmail.com> * fix: sqlness test errors Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: use `Duration` type instead of `String` for ttl fields Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: use pre-allocation for building RowInsertRequests Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: fix clippy errors Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: code review Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: fix integration errors Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: polish code for `group_events_by_type()` and `build_row_inserts_request()`, also add the unit tests Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: refine comments --------- Signed-off-by: zyy17 <zyylsxm@gmail.com>	2025-08-12 18:26:12 +00:00
Lei, HUANG	ff5d672583	chore: impl cast from primitives to PathType (#6724) ... chore/impl-cast-to-primitives-for-path-type: ### Add `num_enum` for Enum Conversion and Update `PathType` - **Added `num_enum` Dependency**: Updated `Cargo.lock` and `Cargo.toml` to include `num_enum` for enum conversion functionality. - Files: `Cargo.lock`, `src/store-api/Cargo.toml` - **Enhanced `PathType` Enum**: Implemented `TryFromPrimitive` for `PathType` to enable conversion from primitive types. - Files: `src/store-api/src/region_request.rs` - **Added Unit Tests**: Introduced tests to verify the conversion of `PathType` enum to and from primitive types. - Files: `src/store-api/src/region_request.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>	2025-08-12 13:07:01 +00:00
Ruihang Xia	e495c614f7	perf: improve bloom filter reader's byte reading logic (#6658) ... * perf: improve bloom filter reader's byte reading logic Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * revert toml change Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * clearify comment Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * benchmark Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * update lock file Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * pub util fn Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * note endian Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2025-08-12 11:37:25 +00:00
Ning Sun	e80e4a9ed7	fix: update pgwire to fix windows timeout issue (#6710) ... * test: reproduce windows ci issue * chore: update sqlx * chore: update pgwire * chore: update to a debug version of pgwire * fix: update pgwire to resolve peek after read on windows * ci: remove windows task from regular ci	2025-08-12 08:24:58 +00:00
LFC	253d89b5cc	feat: able to set read preference to flownode (#6696) ... fix: correctly compare the opened follower regions in startup Signed-off-by: luofucong <luofc@foxmail.com>	2025-08-08 09:08:09 +00:00
Ning Sun	bbe48e9e8b	feat: update pgwire to 0.32 (#6674) ... * feat: update pgwire api * feat: update pgwire and override on_query/on_execute * feat: update pgwire to 0.32 * chore: remove code example Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com>	2025-08-07 06:17:52 +00:00
Weny Xu	e2015ce1af	feat(metric-engine): add metadata region cache (#6657) ... * feat(metric-engine): add metadata region cache Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: use lru Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: rename Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: rename Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: add comments Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: default ttl Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: longer ttl Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>	2025-08-07 06:16:23 +00:00
Zhenchi	6c6487ab30	chore: bump version to 0.17.0 (#6663) ... Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>	2025-08-06 07:45:39 +00:00
Ruihang Xia	757694ae38	feat: count underscore in English tokenizer and improve performance (#6660) ... * feat: count underscore in English tokenizer and improve performance Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * update lock file Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * update test results Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * assert lookup table Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * handle utf8 alphanumeric Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * finalize Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com>	2025-08-06 07:23:18 +00:00
zyy17	3a9f0220b5	fix: unable to record slow query (#6590) ... * refactor: add process manager for prometheus query Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: modify `register_query()` API to accept parsed statement(`catalog::process_manager::QueryStatement`) Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: add the slow query timer in the `Tikcet` of ProcessManager Signed-off-by: zyy17 <zyylsxm@gmail.com> * test: add integration tests Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: add process manager in `do_exec_plan()` Signed-off-by: zyy17 <zyylsxm@gmail.com> * tests: add `test_postgres_slow_query` integration test Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: polish the code Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: create a query ticket and slow query timer if the statement is a query in `query_statement()` Signed-off-by: zyy17 <zyylsxm@gmail.com> * fix: sqlness errors Signed-off-by: zyy17 <zyylsxm@gmail.com> --------- Signed-off-by: zyy17 <zyylsxm@gmail.com>	2025-08-06 03:35:12 +00:00
zyy17	cc35bab5e4	feat: record the migration events in metasrv (#6579) ... * feat: collect procedure event Signed-off-by: zyy17 <zyylsxm@gmail.com> * feat: collect region migration events Signed-off-by: zyy17 <zyylsxm@gmail.com> * test: add integration test Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: fix docs error Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: fix integration test error Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: change status code for errors Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: add `event()` in Procedure Signed-off-by: zyy17 <zyylsxm@gmail.com> * refactor: improve trait design 1. Add `user_metadata()` in `Procedure` trait; 2. Add `Eventable` trait; Signed-off-by: zyy17 <zyylsxm@gmail.com> * chore: polish the code Signed-off-by: zyy17 <zyylsxm@gmail.com> --------- Signed-off-by: zyy17 <zyylsxm@gmail.com>	2025-08-06 03:30:33 +00:00
discord9	1afa0afc67	feat: add partial truncate (#6602) ... * feat: add partial truncate Signed-off-by: discord9 <discord9@163.com> * fix: per review Signed-off-by: discord9 <discord9@163.com> * feat: add proto partial truncate kind Signed-off-by: discord9 <discord9@163.com> * chore: clippy Signed-off-by: discord9 <discord9@163.com> * chore: update branched proto Signed-off-by: discord9 <discord9@163.com> * feat: grpc support truncate WIP sql support Signed-off-by: discord9 <discord9@163.com> * wip: parse truncate range Signed-off-by: discord9 <discord9@163.com> * feat: truncate by range Signed-off-by: discord9 <discord9@163.com> * fix: truncate range display Signed-off-by: discord9 <discord9@163.com> * chore: resolve todo Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * test: more invalid parse Signed-off-by: discord9 <discord9@163.com> * chore: per review Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * chore: unused Signed-off-by: discord9 <discord9@163.com> * chore: per review Signed-off-by: discord9 <discord9@163.com> * chore: update branch Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>	2025-08-04 10:50:27 +00:00