Lei, HUANG
931556dbd3
perf(metric-engine)!: Replace mur3 with fxhash for faster TSID generation ( #7316 )
...
* feat/change-tsid-gen:
perf(metric-engine): replace mur3 with fxhash for faster TSID generation
- Switches from mur3::Hasher128 to fxhash::FxHasher for TSID hashing
- Pre-computes label-name hash when no nulls are present, avoiding redundant work
- Adds fast-path for rows without nulls; falls back to slow path otherwise
- Updates Cargo.toml and lockfile to reflect dependency change
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* feat/change-tsid-gen:
fix: only check primary-key labels for null when re-using cached hash
- Rename has_null() → has_null_labels() and restrict the check to the
primary-key columns so that non-label NULLs do not force a full
TSID re-computation.
- Update expected hashes in tests to match the new logic.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* feat/change-tsid-gen:
test: add comprehensive TSID generation tests for label ordering and null handling
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* feat/change-tsid-gen:
bench: add criterion benchmark for TSID generator
- Compare original mur3 vs current fxhash fast/slow paths
- Test 2, 5, 10 label sets plus null-value slow path
- Add mur3 & criterion dev-deps; register bench target
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* feat/change-tsid-gen:
test: stabilize metric-engine tests by fixing non-deterministic row order
- Add ORDER BY to SELECTs in TTL tests to ensure consistent output
- Update expected __tsid values after hash function change
- Swap expected OTLP metric rows to match new ordering
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* feat/change-tsid-gen:
refactor: simplify Default impls and remove redundant code
- Replace manual Default for TsidGenerator with derive
- Remove unnecessary into_iter() call
- Simplify Option::unwrap_or_else to unwrap_or
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
2025-12-02 08:38:29 +00:00
LFC
fdab75ce27
feat: simple read write new json type values ( #7175 )
...
feat: basic json read and write
Signed-off-by: luofucong <luofc@foxmail.com >
2025-11-27 12:40:35 +00:00
shuiyisong
5472bdfc0f
chore: return 404 if trace not found ( #7304 )
...
* chore: return 404 if trace not found
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add test and fix
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-11-26 09:39:28 +00:00
shuiyisong
713525797a
chore: optimize search traces from Grafana ( #7298 )
...
* chore: minor update
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update ua setup
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-11-26 08:06:15 +00:00
shuiyisong
9f4902b10a
feat: reloadable tls client config ( #7230 )
...
* feat: add ReloadableClientTlsConfig
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* refactor: merge tls option with the reloadable
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: rename function
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update comment
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: extract tls loader
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: minor comment update
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add serde default to watch field
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: minor update
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add log
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: add error log
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-11-24 11:52:11 +00:00
LFC
4a7c16586b
refactor: remove Vectors from RecordBatch completely ( #7184 )
...
* refactor: remove `Vector`s from `RecordBatch` completely
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2025-11-21 08:53:35 +00:00
discord9
29bbff3c90
feat: gc worker only local regions&test ( #7203 )
...
* feat: gc worker only on local region
Signed-off-by: discord9 <discord9@163.com >
* more check
Signed-off-by: discord9 <discord9@163.com >
* chore: stuff
Signed-off-by: discord9 <discord9@163.com >
* fix: ignore async index file for now
Signed-off-by: discord9 <discord9@163.com >
* fix: file removal rate calc
Signed-off-by: discord9 <discord9@163.com >
* chore: per review
Signed-off-by: discord9 <discord9@163.com >
* chore: per review
Signed-off-by: discord9 <discord9@163.com >
* clippy
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2025-11-18 02:45:09 +00:00
Ruihang Xia
cc61af7c65
feat: dynamic enable or disable trace ( #6609 )
...
* wip
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* set `TRACE_RELOAD_HANDLE`
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* wrap http api
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update dependencies
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* type alias and unwrap_or_else
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* better error handling
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lazy initialize tracer
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
2025-11-17 12:16:46 +00:00
WaterWhisperer
de9ae6066f
refactor: remove export_metrics and related configuration ( #7236 )
...
Signed-off-by: WaterWhisperer <waterwhisperer24@qq.com >
2025-11-17 02:32:22 +00:00
shuiyisong
e7928aaeee
chore: add tls-watch option in cmd ( #7226 )
...
* chore: add tls-watch cmd option
* chore: add watch tls option to standalone and fe cmd
* chore: fix clippy
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: address CR comment
Co-authored-by: Yingwen <realevenyag@gmail.com >
* chore: address CR issue
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2025-11-14 09:58:52 +00:00
Yingwen
24671b60b4
feat: tracks index files in another cache and preloads them ( #7181 )
...
* feat: divide parquet and puffin index
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: download index files when we open the region
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: use different label for parquet/puffin
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: control parallelism and cache size by env
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: change gauge to counter
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: correct file type labels in file cache
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: move env to config and change cache ratio to percent
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: checks capacity before download and refine metrics
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: change open to return MitoRegionRef
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: extract download to FileCache
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: run load cache task in write cache
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: check region state before downloading files
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: update config docs and test
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: use file id from index_file_id to compute puffin key
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: skip loading cache in some states
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2025-11-11 08:37:32 +00:00
jeremyhi
c7fded29ee
feat: query mem limiter ( #7078 )
...
* feat: query mem limiter
* feat: config docs
* feat: frontend query limit config
* fix: unused imports
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: add metrics for query memory tracker
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: right postion for tracker
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: avoid race condition
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: soft and hard limit
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: docs
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: when soft_limit == 0
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: upgrade limit algorithm
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: remove batch window
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: batch mem size
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: refine limit algorithm
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: get sys mem
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: minor change
* feat: up tracker to the top stream
* feat: estimated_size for batch
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: minor refactor
* feat: scan_memory_limit connect to max_concurrent_queries
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: make callback clearly
* feat: add unlimted enum
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: by review comment
* chore: comment on recursion_limit
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: refactor and put permit into RegionScanExec
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: multiple lazy static blocks
* chore: minor change
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
2025-11-11 07:47:55 +00:00
shuiyisong
a3dbd029c5
chore: remove ttl option if presents in trace meta table ( #7197 )
...
* chore: remove ttl option if presents in trace meta table
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-11-06 11:51:45 +00:00
shuiyisong
50c9600ef8
fix: stabilize test results ( #7182 )
...
* fix: stablize test results
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-11-05 09:19:23 +00:00
shuiyisong
6caff50d01
chore: improve search traces and jaeger resp ( #7166 )
...
* chore: add jaeger field in trace query
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update search v1 with tags
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update col matching using col names
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: minify code with macro
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: fix test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: change macro to inline function
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: fix filter with tags & add test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-11-04 05:49:08 +00:00
LFC
109b70750a
refactor: convert to prometheus values directly from arrow ( #7153 )
...
* refactor: convert to prometheus values directly from arrow
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2025-10-30 10:24:12 +00:00
discord9
37bc2e6b07
feat: gc worker heartbeat instruction ( #7118 )
...
again
false by default
test: config api
refactor: per code review
less info!
even less info!!
docs: gc regions instr
refactor: grp by region id
per code review
per review
error handling?
test: fix
todos
aft rebase fix
after refactor
Signed-off-by: discord9 <discord9@163.com >
2025-10-29 02:59:36 +00:00
zyy17
0a3961927d
refactor!: add a opentelemetry_traces_operations table to aggregate (service_name, span_name, span_kind) to improve query performance ( #7144 )
...
refactor: add a `*_operations` table to aggregate `(service_name, span_name, span_kind)` to improve query performance
Signed-off-by: zyy17 <zyylsxm@gmail.com >
2025-10-27 03:36:22 +00:00
LFC
2f637a262e
chore: update datafusion to 50 ( #7076 )
...
* chore: update datafusion to 50
Signed-off-by: luofucong <luofc@foxmail.com >
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com >
* fix: update datafusion_pg_catalog import
* chore: fix toml format
* chore: fix toml format again
* fix nextest
Signed-off-by: luofucong <luofc@foxmail.com >
* fix sqlness
Signed-off-by: luofucong <luofc@foxmail.com >
* chore: switch datafusion-orc to upstream tag
* fix sqlness
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
Co-authored-by: Ning Sun <sunning@greptime.com >
2025-10-23 07:18:36 +00:00
jeremyhi
62b51c6736
feat: writer mem limiter for http and grpc service ( #7092 )
...
* feat: writer mem limiter for http and grpc service
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: docs
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: add metrics for limiter
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* Apply suggestion from @MichaelScofield
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
* chore: refactor try_acquire
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: make size human readable
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
2025-10-22 09:30:36 +00:00
Alan Tang
8073e552df
feat: add updated_on to tablemeta with a default of created_on ( #7072 )
...
* feat: add updated_on to tablemeta with a default of created_on
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
* feat: support the update_on on alter procedure
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
* feat: add updated_on into information_schema.tables
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
* fix: make sqlness happy
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
* test: add test case for tablemeta update
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
* fix: fix failing test for ALTER TABLE
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
* feat: use created_on as default for updated_on when missing
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
---------
Signed-off-by: Alan Tang <jmtangcs@gmail.com >
2025-10-15 11:12:27 +00:00
Yingwen
a9c342b0f7
feat: support setting sst_format in table options ( #7068 )
...
* feat: add FormatType to support multi format in the future
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: add sst_format to RegionOptions
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: sets the sst_format based on RegionOptions
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: add sst_format to mito table options
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: fix RegionManifest deserialization without sst_format
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: remove Parquet suffix from FormatType
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: prefer RegionOptions::sst_format in compactor/memtable builder
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: rename enable_experimental_flat_format to
default_experimental_flat_format
Signed-off-by: evenyag <realevenyag@gmail.com >
* docs: update config.md
Signed-off-by: evenyag <realevenyag@gmail.com >
* style: fmt
Signed-off-by: evenyag <realevenyag@gmail.com >
* test: update manifest test
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: fix compiler errors, handle sst_format in remap_manifest
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2025-10-13 08:38:37 +00:00
LFC
8fe17d43d5
chore: update rust to nightly 2025-10-01 ( #7069 )
...
* chore: update rust to nightly 2025-10-01
Signed-off-by: luofucong <luofc@foxmail.com >
* chore: nix update
---------
Signed-off-by: luofucong <luofc@foxmail.com >
Co-authored-by: Ning Sun <sunning@greptime.com >
2025-10-11 07:30:52 +00:00
Ning Sun
aa84642afc
refactor!: remove pb_value to json conversion, keep json output consistent ( #7063 )
...
* refactor: remove pb_value to json
* chore: remove unused module
2025-10-10 07:09:20 +00:00
Sicong Hu
779865d389
feat: introduce IndexBuildTask for async index build ( #6927 )
...
* feat: add framework for asynchronous index building
Signed-off-by: SNC123 <sinhco@outlook.com >
* test: add unit tests for IndexBuildTask
Signed-off-by: SNC123 <sinhco@outlook.com >
* chore: clippy,format,fix-udeps
Signed-off-by: SNC123 <sinhco@outlook.com >
* fix: correct write cache logic in IndexBuildTask
Signed-off-by: SNC123 <sinhco@outlook.com >
* chore: clippy, resolve conflicts
Signed-off-by: SNC123 <sinhco@outlook.com >
* chore: resolve conflicts
Signed-off-by: SNC123 <sinhco@outlook.com >
* fix: apply review suggestions
Signed-off-by: SNC123 <sinhco@outlook.com >
* chore: resolve conflicts
Signed-off-by: SNC123 <sinhco@outlook.com >
* fix: clean up index files in aborted case
Signed-off-by: SNC123 <sinhco@outlook.com >
* refactor: move manifest update logic into IndexBuildTask
Signed-off-by: SNC123 <sinhco@outlook.com >
* fix: enhance check file logic and error handling
Signed-off-by: SNC123 <sinhco@outlook.com >
---------
Signed-off-by: SNC123 <sinhco@outlook.com >
2025-10-10 03:29:32 +00:00
shuiyisong
b24a55cea4
chore: rename the default ts column name to greptime_timestamp for influxdb line protocol ( #7046 )
...
* chore: rename influxdb ts column name to greptime_timestamp
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: tests
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-10-09 02:14:11 +00:00
Ning Sun
660790148d
fix: various typos reported by CI ( #7047 )
...
* fix: various typos reported by CI
* fix: additional typo
2025-10-02 15:11:09 +00:00
LFC
aa05b3b993
feat: add max_connection_age config to grpc server ( #7031 )
...
* feat: add `max_connection_age` config to grpc server
Signed-off-by: luofucong <luofc@foxmail.com >
* Apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2025-09-29 07:32:43 +00:00
zyy17
0717773f62
refactor!: add enable_read_cache config to support disable read cache explicitly ( #6834 )
...
* refactor: add `enable_read_cache` config to support disable read cache explicitly
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: if `cache_path` is empty and `enable_read_cache` is true, set the default cache dir
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: remove the unessary Option type for `ObjectStorageCacheConfig`
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: sanitize cache config in `DatanodeOptions` and `StandaloneOptions`
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: code review comment
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: apply code review comments
Signed-off-by: zyy17 <zyylsxm@gmail.com >
---------
Signed-off-by: zyy17 <zyylsxm@gmail.com >
2025-09-26 09:44:12 +00:00
Ruihang Xia
0790835c77
feat!: improve greptime_identity pipeline behavior ( #6932 )
...
* flat by default, store array in string
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* expose max_nested_levels param, store string instead of error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove flatten option
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2025-09-25 15:28:28 +00:00
dennis zhuang
91a727790d
feat: supports permission mode for static user provider ( #7017 )
...
* feat: supports permission mode for static user provider
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* chore: style
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: comment
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-09-25 03:45:31 +00:00
Yingwen
b8e0c49cb4
feat: add an flag to enable the experimental flat format ( #6976 )
...
* feat: add enable_experimental_flat_format flag to enable flat format
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: extract build_scan_input for CompactionSstReaderBuilder
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: add compact memtable cost to flush metrics
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: Sets compact dispatcher for bulk memtable
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: Cast dictionary to target type in FlatProjectionMapper
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: add time index to FlatProjectionMapper::batch_schema
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: update config toml
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: pass flat_format to ProjectionMapper in CompactionSstReaderBuilder
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2025-09-16 09:33:12 +00:00
shuiyisong
ac051af201
feat(pipeline): generate create table sql from pipeline config ( #6930 )
...
* chore: add expr_to_create function using cc and polish
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* feat: add get pipeline create table
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update test_expr_to_create
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: column extension
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add message hint for variant table name
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add message hint for existing table
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* test: create table for pipeline
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update comment
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: rename to query pipeline ddl
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-09-09 02:47:51 +00:00
Ruihang Xia
c9377e7c5a
build: bump rust edition to 2024 ( #6920 )
...
* bump edition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* gen keyword
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lifetime and env var
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* one more gen fix
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lifetime of temporaries in tail expressions
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clippy nested if
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clippy let and return
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2025-09-08 02:37:18 +00:00
localhost
756856799f
refactor(pipeline)!: change dispatch table name format ( #6901 )
...
* chore(breaking): change dispatch table name format
* chore: fix test
2025-09-03 13:47:40 +00:00
Yingwen
ab96703d8f
chore: enlarge max file limit to 384 ( #6868 )
...
chore: enlarge max concurrent files to 384
Signed-off-by: evenyag <realevenyag@gmail.com >
2025-09-01 09:26:20 +00:00
discord9
bacd9c7d15
feat: add event ts to region manifest ( #6751 )
...
* feat: add event ts to region manifest
Signed-off-by: discord9 <discord9@163.com >
delete files
Signed-off-by: discord9 <discord9@163.com >
chore: clippy
Signed-off-by: discord9 <discord9@163.com >
lower concurrency
Signed-off-by: discord9 <discord9@163.com >
feat: gc use delta manifest to get expel time
Signed-off-by: discord9 <discord9@163.com >
docs: terminalogy
Signed-off-by: discord9 <discord9@163.com >
refactor: some advices from review
Signed-off-by: discord9 <discord9@163.com >
feat: manifest add removed files field
Signed-off-by: discord9 <discord9@163.com >
feat(WIP): add remove time in manifest
Signed-off-by: discord9 <discord9@163.com >
wip: more config
Signed-off-by: discord9 <discord9@163.com >
feat: manifest uodate removed files
Signed-off-by: discord9 <discord9@163.com >
test: add remove file opts field
Signed-off-by: discord9 <discord9@163.com >
test: fix test
Signed-off-by: discord9 <discord9@163.com >
chore: delete gc.rs
Signed-off-by: discord9 <discord9@163.com >
* feat: proper option name
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* test: update manifest size
Signed-off-by: discord9 <discord9@163.com >
* test: fix eq
Signed-off-by: discord9 <discord9@163.com >
* refactor: some per review
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* refactor: more per review
Signed-off-by: discord9 <discord9@163.com >
* test: update manifest size
Signed-off-by: discord9 <discord9@163.com >
* test: update config
Signed-off-by: discord9 <discord9@163.com >
* feat: keep count 0 means only use ttl
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2025-08-28 09:08:18 +00:00
shuiyisong
ec817f6877
fix: gRPC auth ( #6827 )
...
* fix: internal service
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* refactor: gRPC auth
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add permission check for bulk ingest
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: remove unused grpc auth middleware
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: extract header function
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* refactor: extract common code and add auth to otel arrow api
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: rename utils to context_auth
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* test: otel arrow auth
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add support for old auth value
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-08-28 04:00:45 +00:00
Ning Sun
69eed2c3fa
feat: only show prometheus logical tables for __name__ values query ( #6814 )
...
feat: only show prometheus logical tables for __name__ query
Signed-off-by: Ning Sun <sunning@greptime.com >
2025-08-25 15:04:42 +00:00
Ning Sun
48572d18a8
feat: name label regex matcher in label values api ( #6799 )
...
* test: add failing test for #6791
* test: add support for = and =~
* fix: lint
* fix: code merge issue
Signed-off-by: Ning Sun <sunning@greptime.com >
---------
Signed-off-by: Ning Sun <sunning@greptime.com >
2025-08-25 08:48:53 +00:00
Ning Sun
68ac37461b
feat: add limit to label value api ( #6795 )
...
* feat: add limit to label value api
Signed-off-by: Ning Sun <sunning@greptime.com >
* feat: limit for special labels
Signed-off-by: Ning Sun <sunning@greptime.com >
---------
Signed-off-by: Ning Sun <sunning@greptime.com >
2025-08-23 06:23:51 +00:00
LFC
f9d2a89a0c
chore: update datafusion family ( #6675 )
...
* chore: update datafusion family
Signed-off-by: luofucong <luofc@foxmail.com >
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com >
* use official otel-arrow-rust
Signed-off-by: luofucong <luofc@foxmail.com >
* rebase
Signed-off-by: luofucong <luofc@foxmail.com >
* use the official orc-rust
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
* remove the empty lines
Signed-off-by: luofucong <luofc@foxmail.com >
* try following PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2025-08-15 12:41:49 +00:00
discord9
dfc29eb3b3
feat: flownode grpc client to frontend tls option ( #6750 )
...
* feat: flownode grpc client to frontend tls option
Signed-off-by: discord9 <discord9@163.com >
* refactor: client tls option
Signed-off-by: discord9 <discord9@163.com >
* refactor: client_tls to frontend_tls
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2025-08-15 10:44:27 +00:00
sunheyi
d03f85287e
feat: mysql add prepared_stmt_cache_capacity ( #6639 )
...
* feat: your clear and concise commit message
Signed-off-by: sunheyi <1061867552@qq.com >
* fix error
Signed-off-by: sunheyi <1061867552@qq.com >
* add param
Signed-off-by: sunheyi <1061867552@qq.com >
* fix
Signed-off-by: sunheyi <1061867552@qq.com >
* fix doc error
Signed-off-by: sunheyi <1061867552@qq.com >
---------
Signed-off-by: sunheyi <1061867552@qq.com >
2025-08-14 08:19:10 +00:00
Weny Xu
8659412cac
feat: introduce PeriodicTopicStatsReporter ( #6730 )
...
* refactor: introduce `PeriodicTopicStatsReporter`
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions from CR
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: fix typo
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: remote wal tests styling
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: fix unit test
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: handling region wal options not found
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions from CR
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: minor
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: upgrade greptime-proto
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
2025-08-13 11:46:50 +00:00
zyy17
777da35b0d
refactor: unify the event recorder ( #6689 )
...
* refactor: unify the event recorder
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: add `table_name()` in `Event` trait
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: add `slow_query_options` in `Instance`
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: add `EventHandlerOptions` and `options()` in `EventHandler` trait
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: add `aggregate_events_by_type()` and support log mode of slow query
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: polish the code
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* fix: clippy errors
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: support to set ttl by using extension of query context
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: refine the configs fields
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* fix: sqlness test errors
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: use `Duration` type instead of `String` for ttl fields
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: use pre-allocation for building RowInsertRequests
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: fix clippy errors
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: code review
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: fix integration errors
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: polish code for `group_events_by_type()` and `build_row_inserts_request()`, also add the unit tests
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: refine comments
---------
Signed-off-by: zyy17 <zyylsxm@gmail.com >
2025-08-12 18:26:12 +00:00
Ruihang Xia
0781adaa3d
feat: new HTTP API for formatting SQL ( #6691 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2025-08-09 07:39:41 +00:00
LFC
253d89b5cc
feat: able to set read preference to flownode ( #6696 )
...
fix: correctly compare the opened follower regions in startup
Signed-off-by: luofucong <luofc@foxmail.com >
2025-08-08 09:08:09 +00:00
localhost
3a2f5413e0
chore: add and/or for log query ( #6681 )
...
* chore: add and/or for log query
* chore: remove impl From<Vec<ColumnFilters>> for Filters
2025-08-08 08:48:03 +00:00
zyy17
3a9f0220b5
fix: unable to record slow query ( #6590 )
...
* refactor: add process manager for prometheus query
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: modify `register_query()` API to accept parsed statement(`catalog::process_manager::QueryStatement`)
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: add the slow query timer in the `Tikcet` of ProcessManager
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* test: add integration tests
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: add process manager in `do_exec_plan()`
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* tests: add `test_postgres_slow_query` integration test
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: polish the code
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: create a query ticket and slow query timer if the statement is a query in `query_statement()`
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* fix: sqlness errors
Signed-off-by: zyy17 <zyylsxm@gmail.com >
---------
Signed-off-by: zyy17 <zyylsxm@gmail.com >
2025-08-06 03:35:12 +00:00