rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-07-08 15:00:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,573 Commits 147 Branches 261 Tags
release/v1.1
Commit Graph

19 Commits

Author SHA1 Message Date
Weny Xu
8ad2d2414c chore: pick fixes and bump version to v1.1.2 (#8404) 
* fix: improve Grafana metrics dashboards (#8298)

* chore: initial changes

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: improve troubleshooting dashboard

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: rm troubleshooting-dashboard.md

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: optimize metrics dashboard

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: move troubleshooting-dashboard.md

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: move mito gc duration panel

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: cleanup the dashboard

- Overview trend panels are now aggregate-only:
    - Total Ingestion Rate Trend
    - Total Query Rate Trend

- Protocol breakdowns remain in Ingestion and Queries.
- Mito Backpressure and Failures no longer duplicates scan/GC signals.
- Removed Write Stall per Instance.
- Split Object Store and WAL into collapsed Object Store and collapsed
  WAL.
- Moved WAL/logstore panels out of Storage into WAL.
- Normalized OpenDAL “other request” matchers.
- Normalized trigger elapsed p99/p75/avg aggregation.
- Regenerated standalone JSON and dashboard YAML/Markdown.
- Updated docs/troubleshooting-dashboard.md.

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: rearrange metasrv dashboard panels

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: improve troubleshooting dashboard layout

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: remove obsolete troubleshooting dashboard doc

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: correct cluster dashboard panel queries (missing _bucket, raw counters, rate normalization)

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: correct trigger panel datasource, collapse flush/compaction, split request latency panels

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: update grafana metrics dashboard panels

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: correct Grafana dashboard units

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: regenerate Grafana dashboards

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: use throughput unit for index IO bytes

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: redact Kafka SASL password in debug output (#8337)

## Summary
- Mask `KafkaClientSaslConfig` password fields in debug output while keeping usernames visible.
- Cover metasrv WAL debug output with a regression test.

## Files
- `src/common/wal/src/config/kafka/common.rs`
- `src/common/wal/src/config.rs`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): run optimizer rules before MergeScan (#8339)

* fix(query): push down join filters before MergeScan

Signed-off-by: discord9 <discord9@163.com>

* fix(query): run optimizer before MergeScan pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): narrow pre-MergeScan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): refine pre-MergeScan optimizer prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): satisfy predicate extractor clippy

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover pre-MergeScan optimizer edges

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover set comparison prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): guard remote scan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve subquery planning errors

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve usable scan predicates

Signed-off-by: discord9 <discord9@163.com>

* fix(query): simplify scan predicate extraction

Signed-off-by: discord9 <discord9@163.com>

* fix(query): keep scan filter extraction scoped

Signed-off-by: discord9 <discord9@163.com>

* docs(query): explain pre-MergeScan optimizer

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: preserve bulk write grpc error details (#8349)

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: include index files in GC listing (#8327)

* fix: include index files in GC listing

Signed-off-by: discord9 <discord9@163.com>

* chore: filter GC index listing to puffins

Signed-off-by: discord9 <discord9@163.com>

* chore: simplify GC index listing stream

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: stream tables for prometheus label discovery (#8341)

Signed-off-by: Ritwij Aryan Parmar <ritwij.aryan.parmar@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: account parquet metadata cache size (#8368)

* fix: account parquet metadata cache size

Use Parquet metadata memory sizing for SST metadata cache weight and add regression coverage for byte-array page-index buffers.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: saturate sst meta cache weight

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

---------

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: respect gc mailbox timeout for admin gc (#8363)

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: record catalog and schema in slow queries (#8387)

* fix: record catalog and schema in slow queries

Add catalog and schema context to slow query records while appending the new columns after existing fields to preserve column order.

- `src/common/frontend/src/slow_query_event.rs`: extend `SlowQueryEvent` schema and rows with `catalog_name` and `schema_name`, and cover append-only ordering.
- `src/catalog/src/process_manager.rs`: carry catalog and schema through `SlowQueryTimer`.
- `src/frontend/src/instance.rs`: capture context for SQL, plan, and PromQL slow query timers.
- `tests-integration/tests/sql.rs`: assert MySQL and PostgreSQL slow query records include catalog and schema.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: address slow query review comment

Use `String::clone` when writing slow query catalog and schema values.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: keep slow query schema only

Remove the slow query `catalog_name` column and keep `schema_name` as a non-null tag dimension.

- `src/common/frontend/src/slow_query_event.rs`: expose only `schema_name` in `SlowQueryEvent` rows and mark it as a tag.

- `src/catalog/src/process_manager.rs`: stop carrying catalog context in `SlowQueryTimer`.

- `src/frontend/src/instance.rs`: pass only schema context to slow query timers.

- `tests-integration/tests/sql.rs`: assert slow query records include `schema_name` without `catalog_name`.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: schema name semantic should be field

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: typo

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: invalidate comment DDL cache and lock by object ID (#8390)

* fix: invalidate comment ddl cache locally

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix typos

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: client_ip error logs skip internal API (#8362)

* chore: client_ip error logs skip internal API

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: fmt

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: use const

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: use const

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: update dashboard to v0.13.6 (#8369)

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: use ENV for building dashboard (#8384)

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: handle PromQL time binary aggregation (#8398)

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(mito): prune files by manifest time range (#8352)

* perf(mito): prune files by manifest time range

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): address file pruning review

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): remove verbose file pruning log

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): expose file pruning metric

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): shorten file pruning metric

Signed-off-by: discord9 <discord9@163.com>

* test(mito): cover file pruning edge cases

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(mito): skip manifest-pruned file ranges (#8366)

* perf(mito): skip manifest-pruned file ranges

Signed-off-by: discord9 <discord9@163.com>

* test(mito): allow empty prune benchmark output

Signed-off-by: discord9 <discord9@163.com>

* fix(mito): avoid caching stale pruned builders

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): address pruner clippy

Signed-off-by: discord9 <discord9@163.com>

* fix(mito): account worker pruner builder metrics

Signed-off-by: discord9 <discord9@163.com>

* test(mito): keep empty prune benchmark local

Signed-off-by: discord9 <discord9@163.com>

* refactor(mito): share manifest-pruned range skip

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): shorten prune cache comment

Signed-off-by: discord9 <discord9@163.com>

* fix(mito): keep manifest prune state in pruner

Signed-off-by: discord9 <discord9@163.com>

* test(mito): cover manifest prune fast skip edge cases

Signed-off-by: discord9 <discord9@163.com>

* chore: fix typo in logical table alter

Signed-off-by: discord9 <discord9@163.com>

* chore(mito): address pruner review comments

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump version to v1.1.2

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: Ritwij Aryan Parmar <ritwij.aryan.parmar@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: Ritwij Aryan Parmar <88580521+RitwijParmar@users.noreply.github.com>
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
Co-authored-by: sun <sunchang_long@163.com>
 2026-07-02 21:21:19 +08:00
discord9
b623cb1aa2 perf: no longer window sort when limit (#7912) 
* perf: no longer window sort when limit

Signed-off-by: discord9 <discord9@163.com>

* test: confusing vector sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: redact sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: redact every thing

Signed-off-by: discord9 <discord9@163.com>

* REDACTED

Signed-off-by: discord9 <discord9@163.com>

* what

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-04-08 02:54:22 +00:00
discord9
922f9cb3d6 feat: use dyn filter (#7545) 
* parent b2074e3863
author discord9 <discord9@163.com> 1767869295 +0800
committer discord9 <discord9@163.com> 1772529023 +0800

feat: use dyn filter

Signed-off-by: discord9 <discord9@163.com>

not supported

Signed-off-by: discord9 <discord9@163.com>

refactor: use make_mut instead

Signed-off-by: discord9 <discord9@163.com>

refactor: rm need to clone stream ctx

Signed-off-by: discord9 <discord9@163.com>

r

Signed-off-by: discord9 <discord9@163.com>

pcr

Signed-off-by: discord9 <discord9@163.com>

test: wait for datafusion update

Signed-off-by: discord9 <discord9@163.com>

refactor: use arc swap for dyn filters

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* chore: comment out sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness fix

Signed-off-by: discord9 <discord9@163.com>

* refactor: predicate without option

Signed-off-by: discord9 <discord9@163.com>

* feat: print dyn filters& more tests

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness vector result update

Signed-off-by: discord9 <discord9@163.com>

* chore: log

Signed-off-by: discord9 <discord9@163.com>

* test: properly redact

Signed-off-by: discord9 <discord9@163.com>

* test: better data dist for non empty dyn filter

Signed-off-by: discord9 <discord9@163.com>

* test: properly redacted

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* properly redact

Signed-off-by: discord9 <discord9@163.com>

* docs:  explain why not  do it

Signed-off-by: discord9 <discord9@163.com>

* chore: rename update to add as its more proper

Signed-off-by: discord9 <discord9@163.com>

* chore: rm no need clone

Signed-off-by: discord9 <discord9@163.com>

* docs: per review

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-03-11 03:06:24 +00:00
LFC
b2074e3863 chore: upgrade DataFusion family, again (#7578) 
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch to released version of datafusion-pg-catalog

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
 2026-03-03 07:36:39 +00:00
dennis zhuang
8883022742 refactor(vector-index): use protobuf for metadata and align code (#7648) 
* refactor(vector-index): use protobuf for metadata and introduce lifecycle traits

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: minor change

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: by suggestions

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: format

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: remove usearch from mito2

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: tweak errors

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: update index size in result

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: clippy

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: update proto deps

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-02-05 02:41:48 +00:00
dennis zhuang
c08f3a4472 test: adds sqlness test for vector index (#7634) 
* test: adds sqlness test for vector index

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: CI

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: redacted flat map and size

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: simplify the replace rules

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: update comments and tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-02-04 03:54:47 +00:00
Alan Tang
910a383420 feat(expr): support avg functions on vector (#7146) 
* feat(expr): support vec_elem_avg function

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: support vec_avg function

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* test: add more query test for avg aggregator

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* fix: fix the merge batch mode

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* refactor: use sum and count as state for avg function

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* refactor: refactor merge batch mode for avg function

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: add additional vector restrictions for validation

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

---------

Signed-off-by: Alan Tang <jmtangcs@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2025-11-07 13:42:14 +00:00
dennis zhuang
d53afa849d fix: interval cast expression can't work in range query, #5805 (#5813) 
* fix: interval cast expression can't work in range query, #5805

* fix: nested cast

* test: make vector test stable
 2025-04-02 08:46:17 +00:00
pikady
eea8b1c730 feat: add vec_kth_elem function (#5674) 
* feat: add vec_kth_elem function

Signed-off-by: pikady <2652917633@qq.com>

* code format

Signed-off-by: pikady <2652917633@qq.com>

* add test sql

Signed-off-by: pikady <2652917633@qq.com>

* change indexing from 1-based to 0-based

Signed-off-by: pikady <2652917633@qq.com>

* improve code formatting and correct spelling errors

Signed-off-by: pikady <2652917633@qq.com>

* Update tests/cases/standalone/common/function/vector/vector.sql

I noticed the two lines are identical. Could you clarify the reason for the change? Thanks!

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: pikady <2652917633@qq.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
 2025-03-18 07:25:53 +00:00
Sicong Hu
09dacc8e9b feat: add vec_subvector function (#5683) 
* feat: add vec_subvector function

* change datatype of arg1 and arg2 from u64 to i64

* add sqlness test

* improve description comments
 2025-03-16 10:43:53 +00:00
xiaoniaoyouhuajiang
50b521c526 feat: add vec_dim function (#5587) 
* feat:add `vec_dim` function

* delete unused imports

* Modified to be implemented correctly

* fix comment

* add order for sqlness test
 2025-02-27 15:54:48 +00:00
Zhenchi
ab4663ec2b feat: add vec_add function (#5471) 
* feat: add vec_add function

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix unexpected utf8

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2025-02-06 06:48:50 +00:00
Lin Yihai
fdbfebf4be feat: Add VEC_PRODUCT, VEC_ELEM_PRODUCT, VEC_NORM. (#5303) 
* feat: Add `vec_product(col)` function.

* feat: Add `vec_elem_product` function

* feat: Add `vec_norm` function.
 2025-01-09 06:26:51 +00:00
Lin Yihai
d1108ab581 feat: add vec_div function (#5245) 2024-12-30 07:08:26 +00:00
Kould
dad8ac6f71 feat(vector): add vector functions vec_sub & vec_sum & vec_elem_sum (#5230) 
* feat(vector): add sub function

* chore: added check for vector length misalignment

* feat(vector): add `vec_sum` & `vec_elem_sum`

* chore: codefmt

* update lock file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-12-26 15:07:13 +00:00
Lin Yihai
039989f77b feat: Add vec_mul function. (#5205) 2024-12-25 14:17:22 +00:00
Lin Yihai
f82af15eba feat: Add vector_scalar_mul function. (#5166) 2024-12-16 06:46:38 +00:00
Zhenchi
e1e39993f7 feat(vector): add scalar add function (#5119) 
* refactor: extract implicit conversion helper functions of vector

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(vector): add scalar add function

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix fmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-12-11 09:25:56 +00:00
Zhenchi
0aab68c23b feat(vector): add conversion between vector and string (#5029) 
* feat(vector): add conversion between vector and string

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-11-20 08:42:00 +00:00