Commit Graph

5286 Commits

Author SHA1 Message Date
ZonaHe
a04ec07b61 feat: update dashboard to v0.11.0 (#6794)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2025-08-22 09:08:48 +00:00
discord9
d774996e89 chore: make internal grpc optional (#6789)
* chore: make internal grpc optional

Signed-off-by: discord9 <discord9@163.com>

* revert sqlness runner too

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-08-22 04:22:49 +00:00
discord9
2b43ff30b6 feat: provide plan info when flow exec (#6783)
* feat: provide  plan info when flow exec

Signed-off-by: discord9 <discord9@163.com>

* backoff?

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-08-22 03:47:26 +00:00
discord9
eaceae4c91 feat: frontend internal grpc port (#6784)
* feat: frontend internal grpc port

Signed-off-by: discord9 <discord9@163.com>

* fix: grpc server naming

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness test fix

Signed-off-by: discord9 <discord9@163.com>

* fix: internal not use process manager

Signed-off-by: discord9 <discord9@163.com>

* test: test integration port alloc

Signed-off-by: discord9 <discord9@163.com>

* feat: skip auth for internal grpc

Signed-off-by: discord9 <discord9@163.com>

* test: is distributed

Signed-off-by: discord9 <discord9@163.com>

* what:

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
2025-08-22 02:46:35 +00:00
Ning Sun
cdc168e753 feat: support for custom headers in otel exporter (#6773)
* feat: support for custom headers in otel exporter

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: remove wrapping option

Signed-off-by: Ning Sun <sunning@greptime.com>

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
2025-08-21 13:10:13 +00:00
discord9
5d5817b851 chore: no logging when init table_flow cache if empty (#6785)
chore: no logging

Signed-off-by: discord9 <discord9@163.com>
2025-08-21 12:38:11 +00:00
Alex Araujo
a98c48a9b2 feat: optimize CreateFlowData with lightweight FlowQueryContext (#6780)
* feat: optimize CreateFlowData with lightweight FlowQueryContext

Replace full QueryContext with FlowQueryContext containing only essential fields (catalog, schema, timezone) in CreateFlowData struct. This improves serialization performance by eliminating unused extensions HashMap and channel fields.

Key changes:
- Add FlowQueryContext struct with conversion implementations
- Update CreateFlowData to use FlowQueryContext with backward compatibility
- Add tests for serialization and conversions

Signed-off-by: Alex Araujo <alexaraujo@gmail.com>

* chore: run make fmt

Signed-off-by: Alex Araujo <alexaraujo@gmail.com>

---------

Signed-off-by: Alex Araujo <alexaraujo@gmail.com>
2025-08-21 12:02:57 +00:00
Weny Xu
6692957e08 refactor: simplify WAL Pruning procedure part2 (#6782)
refactor: simplify prune wal procedure

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-21 11:09:31 +00:00
Weny Xu
896d72191e feat: introduce PersistStatsHandler (#6777)
* feat: add `Inserter` trait and impl

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: import items

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: introduce `PersistStatsHandler`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: disable persisting stats in sqlness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* reset channel manager

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: avoid to collect

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: remove insert options

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: use `write_bytes` instead of `write_bytes_per_sec`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: compute write bytes delta

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* Update src/meta-srv/src/handler/persist_stats_handler.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
2025-08-21 09:34:58 +00:00
Weny Xu
5eec3485fe feat: add IntoRow and Schema derive macros (#6778)
* chore: import items

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add `IntoRow` and `Schema` derive macros

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: styling

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-21 06:29:59 +00:00
Ruihang Xia
7e573e497c feat: simplify more regex patterns in promql (#6747)
* feat: simplify more regex patterns in promql

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-20 18:51:10 +00:00
Ruihang Xia
474a689309 feat: region prune part 2 (#6752)
* skeleton

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* get rule set

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adjust style

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adjust params

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reuse collider

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* canonize

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* more robust predicate extractor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify predicate extractor's test and impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* unify import

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplification, remove unnecessary interfaces

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle partial referenced exprs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finalize predicate extractor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* document region pruner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: reduce diff

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify checker

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine overlapping check method

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reduce diff

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* coerce types

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unused errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* apply review comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor use Bound

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify hashmap

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

* sqlness tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* redact region id

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test: update sqlness result after udpate datafusion

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
2025-08-20 18:47:38 +00:00
Yingwen
2995eddca5 feat: Implements FlatCompatBatch to adapt schema in flat format (#6771)
* feat: compat flat wip

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: rewrite key

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: support append key

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: Split rewrite logic

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: rename CompatFlatBatch to FlatCompatBatch

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: compat primary key

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: address fixme

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: new CompatBatch if need convert

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: use different compat

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: plain_projection -> flat_projection

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: test compat

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: test FlatCompatBatch

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: fix warnings and remove unused code

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: support compat tags with dictionary types

Signed-off-by: evenyag <realevenyag@gmail.com>

* style: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: reuse column_id_values

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: avoid zero pk values len

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2025-08-20 12:36:14 +00:00
LFC
05529387d9 refactor: use DataFusion's UDAF implementation directly (#6776)
* refactor: use DataFusion's UDAF implementation directly

Signed-off-by: luofucong <luofc@foxmail.com>

* remove: delete how-to guide for writing aggregate functions

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* refactor: port json_encode_path to datafusion udaf

Signed-off-by: Ning Sun <sunning@greptime.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
2025-08-20 08:25:00 +00:00
fys
819531393f feat: disable month in trigger interval expr (#6774)
* feat: disable month in trigger interval expr

* fix: cargo clippy

* fix: cargo clippy

* add unit test

* remove unused comment
2025-08-20 07:21:39 +00:00
dennis zhuang
d6bc117408 refactor: refactor admin functions with async udf (#6770)
* refactor: use async udf for admin functions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: sqlness test

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: code style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: clippy

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: remove unused error

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: code style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: apply suggestions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: logical_metric_table test

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
2025-08-20 03:35:38 +00:00
yihong
7402320abc fix: time() function should the same as behavior prometheus (#6704)
* fix: close issue_6701 phase 1 make it return now

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: make tests stable

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: drop useless tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: close issue_6701 phase 1 make it return now

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: make tests stable

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: drop useless tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: make time() real right

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: fix tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* add two sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-20 03:05:21 +00:00
zyy17
5420d6f7fb refactor: change plugin option type from &[PluginOptions] to Option<&PluginOptions> for understandability (#6763)
refactor: change plugin option type from `[PluginOptions]` to `Option<&PluginOptions>` for understandability

Signed-off-by: zyy17 <zyylsxm@gmail.com>
2025-08-20 02:11:00 +00:00
liyang
7af471c5aa ci: add is-current-version-latest check to helm-charts/homebrew-greptime bump jobs (#6772)
ci: add  check to helm/homebrew bump jobs

Signed-off-by: liyang <daviderli614@gmail.com>
2025-08-19 17:49:07 +00:00
Weny Xu
9cdd0d8251 feat: derive macro ToRow (#6768)
* feat: introduce `ToRow` derive marco

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: use map

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: simplify

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-19 14:06:03 +00:00
Ning Sun
5a4036cc66 feat: update opentelemetry family (#6762)
* feat: update opentelemetry family

Signed-off-by: Ning Sun <sunning@greptime.com>

* doc: update doc samples

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: toml format

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: update default otel enpoint

Signed-off-by: Ning Sun <sunning@greptime.com>

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
2025-08-19 09:23:50 +00:00
Yingwen
8fc3a9a9d7 feat: Implements async FlatMergeReader and FlatDedupReader (#6761)
* refactor: Add Flat prefix to MergeIterator and DedupIterator

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: implement MergeReader for RecordBatch

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: use GenericNode for IterNode

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: flat merge reader to stream

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: implement FlatDedupReader

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: add a benchmark for FlatMergeIterator

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: rename plain_projection to flat_projection

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2025-08-19 06:30:59 +00:00
liyang
0b29b41c17 ci: add Signed-off-by in update-dev-builder-version script (#6765)
Signed-off-by: liyang <daviderli614@gmail.com>
2025-08-19 04:15:13 +00:00
github-actions[bot]
78dca8a0d7 ci: update dev-builder image tag (#6759)
* ci: update dev-builder image tag

Signed-off-by: evenyag <realevenyag@gmail.com>

* ci: ignore makefile

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Co-authored-by: greptimedb-ci <greptimedb-ci@greptime.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
2025-08-19 02:51:34 +00:00
Ning Sun
bf191c5763 test: fix sqlness hash character count (#6758) 2025-08-18 11:06:05 +00:00
Ruihang Xia
b652ea52ee fix: partition tree's dict size metrics mismatch (#6746)
fix: partition tree metrics mismatch

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-18 08:36:35 +00:00
Weny Xu
f64fc3a57a feat: add RateMeter for tracking memtable write throughput (#6744)
* feat: introduce `RateMeter`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-18 06:45:31 +00:00
fys
326198162e refactor: enhanced trigger interval (#6740)
* refactor: enhance trigger interval

* update greptime-proto

* fix: build
2025-08-18 04:03:26 +00:00
LFC
f9d2a89a0c chore: update datafusion family (#6675)
* chore: update datafusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* use official otel-arrow-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* rebase

Signed-off-by: luofucong <luofc@foxmail.com>

* use the official orc-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* remove the empty lines

Signed-off-by: luofucong <luofc@foxmail.com>

* try following PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
2025-08-15 12:41:49 +00:00
discord9
dfc29eb3b3 feat: flownode grpc client to frontend tls option (#6750)
* feat: flownode grpc client to frontend tls option

Signed-off-by: discord9 <discord9@163.com>

* refactor: client tls option

Signed-off-by: discord9 <discord9@163.com>

* refactor: client_tls to frontend_tls

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-08-15 10:44:27 +00:00
Ning Sun
351826cd32 fix: refine shadowrs dependency, remove libgit2, libz and potentally libssl (#6748)
* feat: update shadown-rs config and remove libgit/libz/libssl deps

* chore: remove native inputs from flake
2025-08-15 03:01:14 +00:00
Yingwen
60e01c7c3d feat: Implements last-non-null dedup strategy for flat format (#6709)
* feat: Implements FlatLastNonNull strategy

Dedup rows and keep last non null fields

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add basic test for FlatLastNonNull

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: port more last non null test

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: do merge rows after delete op

Signed-off-by: evenyag <realevenyag@gmail.com>

* style: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: rename num_pk_columns to field_column_start

So we can support different format later

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: address comment

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2025-08-14 14:38:47 +00:00
Weny Xu
021ad09c21 refactor: simplify WAL pruning procedure and introduce region flush trigger (#6741)
* chore: add logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: update wal config for metasrv

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: introduce region flush trigger

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: debug assert

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: log level

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: simplify wal prune procedure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: upgrade rskafka

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: always flush inactive regions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: refactor flush trigger

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: remove unused code

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: typo

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add metrics

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: rename

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-14 14:15:30 +00:00
discord9
2a3e4c7a82 fix: truncate manifest action compat (#6742)
* fix: truncate manifest action compat

Signed-off-by: discord9 <discord9@163.com>

* refactor: use simpler compat

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2025-08-14 14:05:17 +00:00
Zhenchi
92fd34ba22 refactor: split node manager trait (#6743)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2025-08-14 08:33:57 +00:00
sunheyi
d03f85287e feat: mysql add prepared_stmt_cache_capacity (#6639)
* feat: your clear and concise commit message

Signed-off-by: sunheyi <1061867552@qq.com>

* fix error

Signed-off-by: sunheyi <1061867552@qq.com>

* add param

Signed-off-by: sunheyi <1061867552@qq.com>

* fix

Signed-off-by: sunheyi <1061867552@qq.com>

* fix doc error

Signed-off-by: sunheyi <1061867552@qq.com>

---------

Signed-off-by: sunheyi <1061867552@qq.com>
2025-08-14 08:19:10 +00:00
Ruihang Xia
4d97754cb4 feat: persist manifest, SST and index files to staging dir (#6726)
* make flush and access layer aware of staging mode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* staging flag in flush notify

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* staging manifest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* index methods

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* only stage manifest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* improve comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-14 06:02:31 +00:00
Ruihang Xia
1b6d924169 feat: predicate extractor (region prune part 1) (#6729)
* feat: predicate extractor (region prune part 1)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* stricter check

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-14 04:42:50 +00:00
dennis zhuang
80f3ae650c docs: improve CONTRIBUTING.md (#6698)
* docs: improve CONTRIBUTING.md

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: features

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* chore: adds features to Makefile

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: adds make commands

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-14 03:48:05 +00:00
dennis zhuang
c0fe800e79 feat: improve slow queries options deserialization (#6734)
* feat: improve slow queries options deserialization

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: use serde default for struct

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
2025-08-14 03:43:13 +00:00
yihong
56f5ccf823 fix: support unknown for timestamp function (#6708)
* fix: support unknown for timestamp function

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: some sqlness now no error

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: make clippy happy

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-08-14 02:56:33 +00:00
Zhenchi
fb3b1d4866 feat: Store partition expr in RegionMetadata (#6699)
* wire partition.expr_json option constant and parsing

test(mito2): manifest roundtrip persists partition_expr JSON

test(mito2): create/open with partition.expr_json persists in manifest

docs: add comments for partition.expr_json option and RegionOptions.partition_expr

serde: include RegionOptions.partition_expr (skip if None)

test(mito2): doc intent and verify runtime backfill + persistence-after-alter for partition expr

add partition expr to create request

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* add create_with_partition_expr_persists_manifest

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* pass partition expr to create request

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* polish

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* remove unused dep

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2025-08-13 22:05:37 +00:00
Weny Xu
4fb7d92f7c feat(metasrv): implement topic statistics collection (#6732)
* feat(metasrv): collect topic stats

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: fix tests and apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
v0.17.0-nightly-20250813
2025-08-13 12:52:28 +00:00
Weny Xu
8659412cac feat: introduce PeriodicTopicStatsReporter (#6730)
* refactor: introduce `PeriodicTopicStatsReporter`

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix typo

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: remote wal tests styling

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit test

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: handling region wal options not found

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions from CR

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: minor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: upgrade greptime-proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-13 11:46:50 +00:00
Zhenchi
dea87b7e57 refactor: use DummyCatalog to construct query engine for datanode (#6723)
* refactor: use DummyCatalog to construct query engine for datanode

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* move to query/dummy_catalog

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2025-08-13 09:49:51 +00:00
localhost
a678b4dfd6 chore: add u64 for EqualValue and set expr is true when filter is empty (#6731)
* chore: add u64 for EqualValue and set expr is true when filter is empty

* Update src/log-query/src/log_query.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: update EqualValue Uinit to UInt

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2025-08-13 08:25:33 +00:00
Ruihang Xia
ccccaf7734 feat(log-query): try infer and cast type for literal value (#6712)
* initial impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* one more test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicated test cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicated methods

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* initial impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* one more test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicated test cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicated methods

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: add eq for log query

* skip for both literals

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: paomian <xpaomian@gmail.com>
2025-08-13 06:28:37 +00:00
yihong
f0bec4940f fix: two label_replace different from promql (#6720)
* fix: two label_replace different from promql

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Jiachun Feng <jiachun_feng@proton.me>

* fix: another address

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Jiachun Feng <jiachun_feng@proton.me>
2025-08-13 06:27:49 +00:00
yihong
5eb491df12 fix: label_join should work with unknown (#6714)
* fix: label_join should work with unknown

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Jiachun Feng <jiachun_feng@proton.me>

* fix: address forget comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: Jiachun Feng <jiachun_feng@proton.me>
2025-08-13 03:45:40 +00:00
Weny Xu
1d84e802d8 feat: add integration tests for table reconciliation procedures part1 (#6705)
* feat: add integration tests for table reconciliation procedures

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-08-13 03:29:39 +00:00