shuiyisong
ba32c5fe9e
chore: remove unused deps using udeps ( #7906 )
...
* chore: remove unused deps using udeps
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: fmt toml
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2026-04-02 06:49:27 +00:00
dennis zhuang
0572a680af
fix: allow empty string for env values ( #7803 )
...
* fix: allow empty string for env values
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* chore: strip suffix
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
2026-03-13 03:57:08 +00:00
shuiyisong
a85864067e
chore: remove canonicalize ( #7430 )
...
* chore: remove canonicalize
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add match file name option
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update field name
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: modify tls option
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update config file
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update config md
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: update option to `enable_filename_match`
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: address CR issues
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: remove option
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: remove unused test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-12-18 09:39:10 +00:00
shuiyisong
fa2b4e5e63
refactor: extract file watcher to common-config ( #7357 )
...
* refactor: extract file watcher to common-config
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: add file check
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: watch dir instead of file
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: address CR issues
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2025-12-09 11:23:26 +00:00
zyy17
a0e6bcbeb3
feat: add cpu_usage_millicores and memory_usage_bytes in information_schema.cluster_info table. ( #7051 )
...
* refactor: add `hostname` in cluster_info table
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: update information schema result
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* feat: enable zstd for bulk memtable encoded parts (#7045 )
feat: enable zstd in bulk memtable
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: add `get_total_cpu_millicores()` / `get_total_cpu_cores()` / `get_total_memory_bytes()` / `get_total_memory_readable()` in common-stat
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* feat: add `cpu_usage_millicores` and `memory_usage_bytes` in `information_schema.cluster_info` table
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* fix: compile warning and integration test failed
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* fix: integration test failed
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: add `ResourceStat`
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* refactor: apply code review comments
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* chore: update greptime-proto
Signed-off-by: zyy17 <zyylsxm@gmail.com >
---------
Signed-off-by: zyy17 <zyylsxm@gmail.com >
Signed-off-by: evenyag <realevenyag@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2025-10-24 03:12:45 +00:00
zyy17
cf1b8392af
refactor!: unify the API of getting total cpu and memory ( #7049 )
...
* refactor: add `get_total_cpu_millicores()` / `get_total_cpu_cores()` / `get_total_memory_bytes()` / `get_total_memory_readable()` in common-stat
Signed-off-by: zyy17 <zyylsxm@gmail.com >
* tests: update sqlness test cases
Signed-off-by: zyy17 <zyylsxm@gmail.com >
---------
Signed-off-by: zyy17 <zyylsxm@gmail.com >
2025-10-16 12:41:34 +00:00
Weny Xu
0c038f755f
refactor(cli): refactor object storage config ( #7009 )
...
* refactor: refactor object storage config
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: public common config
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
2025-09-24 06:50:47 +00:00
Weny Xu
16febbd4c2
feat: add CPU, memory and node status info to cluster_info ( #6897 )
...
* feat: add CPU and memory info to `cluster_info`
Signed-off-by: WenyXu <wenymedia@gmail.com >
* feat: add `node_status` to `cluster_info` table
Signed-off-by: WenyXu <wenymedia@gmail.com >
* test: update sqlness
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: update proto
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
2025-09-08 08:59:34 +00:00
Ruihang Xia
c9377e7c5a
build: bump rust edition to 2024 ( #6920 )
...
* bump edition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* gen keyword
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lifetime and env var
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* one more gen fix
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lifetime of temporaries in tail expressions
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clippy nested if
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clippy let and return
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2025-09-08 02:37:18 +00:00
Lei, HUANG
1d07864b29
refactor(object-store): move backends building functions back to object-store ( #6400 )
...
refactor/building-backend-in-object-store:
### Refactor Object Store Configuration
- **Centralize Object Store Configurations**: Moved object store configurations (`FileConfig`, `S3Config`, `OssConfig`, `AzblobConfig`, `GcsConfig`) to `object-store/src/config.rs`.
- **Error Handling Enhancements**: Introduced `object-store/src/error.rs` for improved error handling related to object store operations.
- **Factory Pattern for Object Store**: Implemented `object-store/src/factory.rs` to create object store instances, consolidating logic from `datanode/src/store.rs`.
- **Remove Redundant Store Implementations**: Deleted individual store files (`azblob.rs`, `fs.rs`, `gcs.rs`, `oss.rs`, `s3.rs`) from `datanode/src/store/`.
- **Update Usage of Object Store Config**: Updated references to `ObjectStoreConfig` in `datanode.rs`, `standalone.rs`, `config.rs`, and `error.rs` to use the new centralized configuration.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
2025-06-25 13:49:55 +00:00
zyy17
fdf32a8f46
refactor: respect data_home as root data home directory ( #6050 )
...
* refactor: initialize logging dir by using data_home
* chore: remove tail '/' for dir name
2025-06-09 02:31:21 +00:00
LFC
49a936e2e1
refactor: remove the "mode" configuration item completely ( #6042 )
2025-05-06 03:27:15 +00:00
Ning Sun
dc76571166
feat: move default data path from /tmp to current directory ( #5719 )
2025-03-16 09:57:46 +00:00
Weny Xu
c80d2a3222
fix: introduce gc task for metadata store ( #5461 )
...
* fix: introduce gc task for metadata kvbackend
* refactor: refine KvbackendConfig
* chore: apply suggestions from CR
2025-02-06 12:12:43 +00:00
discord9
66f63ae981
feat: more than one flow workers ( #5315 )
...
* feat: more workers
* feat: use round robin
* refactor: per review
* refactor: per bot review
* chore: per review
* docs: example
* docs: update config.md
* docs: update
* chore: per review
* refactor: set workers to cpu/2.max(1)
* fix: flow config in standalone mode
* test: fix config test
* docs: update docs&opt name
* chore: update config.md
* refactor: per review, sanitize at top
* chore: per review
* chore: config.md
2025-01-14 12:28:18 +00:00
LFC
c0aed1d267
feat: set global runtime size by config file ( #4063 )
...
* set global runtime size
* fix: resolve PR comments
* fix: log the whole option
* fix ci
* debug ci
* debug ci
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-06-04 10:03:33 +00:00
zyy17
63a8d293a1
refactor: add Configurable trait ( #3917 )
...
* refactor: add Configurable trait
* refactor: add merge_with_cli_options() to simplify load_options()
* docs: add comments
* fix: clippy errors
* fix: toml format
* fix: build error
* fix: clippy errors
* build: downgrade config-rs
* refactor: use '#[snafu(source(from()))'
* refactor: minor modification for load_layered_options() to make it clean
2024-05-15 12:56:40 +00:00
dennis zhuang
65d47bab56
feat: adds information_schema cluster_info table ( #3832 )
...
* feat: adds server running mode to KvBackendCatalogManager
* feat: adds MetaClient to KvBackendCatalogManager
* feat: impl information_schema.cluster_info table
* fix: forgot files
* test: update information_schema result
* feat: adds start_time and uptime to cluster_info
* chore: tweak cargo and comment
* feat: rename greptime_region_peers to region_peers
* fix: cluster_info result
* chore: simplify sqlness commands
* chore: set peer_id to -1 for frontends
* fix: move cluster_info to greptime catalog
* chore: use official proto
* feat: adds active_time
* chore: apply suggestion
Co-authored-by: Jeremyhi <jiachun_feng@proton.me >
* chore: STANDALONE for runtime_metrics
---------
Co-authored-by: Jeremyhi <jiachun_feng@proton.me >
Co-authored-by: tison <wander4096@gmail.com >
2024-05-02 02:49:46 +00:00
dimbtp
dd18d8c97b
build(deps): remove some unused dependencies ( #3582 )
...
* build(deps): remove some unused dependencies
* add `arc-swap` dependency back
2024-03-26 12:48:28 +00:00
SteveLauC
e9a2b0a9ee
chore: use workspace-wide lints ( #3352 )
...
* chore: use workspace-wide lints
* respond to review
2024-02-22 01:01:10 +00:00
niebayes
63205907fb
refactor: introduce common-wal to aggregate wal stuff ( #3171 )
...
* refactor: aggregate wal configs
* refactor: move wal options to common-wal
* chore: slim Cargo.toml
* fix: add missing crates
* fix: format
* chore: update comments
* chore: add testing feature gate for test_util
* fix: apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix: apply suggestions from code review
* fix: compiling
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-18 03:49:37 +00:00
Wei
3d7d2fdb4a
feat: auto config cache size according to memory size ( #3165 )
...
* feat: auto config cache and buffer size according to mem size
* feat: utils
* refactor: add util function to common config
* refactor: check cgroups
* refactor: code
* fix: test
* fix: test
* chore: cr comment
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
* chore: remove default comment
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
2024-01-17 14:35:35 +00:00
niebayes
a0a31c8acc
chore(remote_wal): remove topic alias ( #3120 )
...
chore: remove topic alias
2024-01-09 07:35:02 +00:00
Lei, HUANG
62db28b465
feat: add options to enable log recycle and periodical fsync ( #3114 )
...
* feat: add options to enable log recycle and periodical fsync
* fix: resolve review comments
* fix: conflicts
2024-01-09 06:41:23 +00:00
niebayes
8c58d3f85b
test(remote_wal): add unit tests for kafka remote wal ( #2993 )
...
* test: add unit tests
* feat: introduce kafka runtime backed by testcontainers
* test: add test for kafka runtime
* fix: format
* chore: make kafka image ready to be used
* feat: add entry builder
* tmp
* test: add unit tests for client manager
* test: add some unit tests for kafka log store
* chore: resolve some todos
* chore: resolve some todos
* test: add unit tests for kafka log store
* chore: add deprecate develop branch warning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tmp: ready to move unit tests to an indie dir
* test: update unit tests for client manager
* test: add unit tests for meta srv remote wal
* fix: license
* fix: test
* refactor: kafka image
* doc: add doc example for kafka image
* chore: migrate kafka image to an indie PR
* fix: CR
* fix: CR
* fix: test
* fix: CR
* fix: update Cargo.toml
* fix: CR
* feat: skip test if no endpoints env
* fix: format
* test: rewrite parallel test with barrier
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-08 10:48:11 +00:00
niebayes
78303639db
feat(remote_wal): split an entry if it's too large ( #3092 )
...
* feat: split an entry if it's too large
* chore: rewrite check records
* test: add some unit tests for record
* chore: rewrite entry splitting
* chore: add unit tests for build records
* chore: add more unit tests for record
* chore: rewrite encdec of record
* revert: ignored test
* fix: set limit for max_batch_size
* fix: clippy
* chore: remove heavy logging
* fix: CR
* fix: properly terminate
* fix: CR
* fix: compiling
* fix: sqlness
* fix: CR
* fix: license
* fix: license
2024-01-05 12:41:43 +00:00
Weny Xu
5653389063
feat!: correct the kafka config option ( #3065 )
...
* feat: correct the kafka config option
* refactor: rewrite the verbose comments
2024-01-02 07:31:37 +00:00
Zhenchi
69a53130c2
feat(inverted_index): Add applier builder to convert Expr to Predicates (Part 1) ( #3034 )
...
* feat(inverted_index.integration): Add applier builder to convert Expr to Predicates (Part 1)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: add docs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: typos
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/mito2/src/sst/index/applier/builder.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: remove unwrap
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: error source
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-12-30 07:32:32 +00:00
niebayes
d061bf3d07
feat(remote_wal): introduce kafka remote wal ( #3001 )
...
* feat: integrate remote wal to standalone
* fix: test
* chore: ready to debug kafka remote wal
* fix: test
* chore: add some logs for remote wal
* chore: add logs for topic manager
* fix: properly terminate stream consumer
* fix: properly handle TopicAlreadyExists error
* fix: parse config file error
* fix: properly handle last entry id
* chore: prepare for merge
* fix: test
* fix: typo
* fix: set replication_factor properly
* fix: CR
* test: tmp for test
* Revert "test: tmp for test"
This reverts commit 093a3e0038
2023-12-26 12:35:24 +00:00
Weny Xu
c902d43380
refactor(remote_wal): add StandaloneWalConfig ( #3002 )
...
* feat: integrate remote wal to standalone
* fix: test
* refactor: refactor standalone wal config
* chore: change default kafka port to 9092
* chore: apply suggestions from CR
---------
Co-authored-by: niebayes <niebayes@gmail.com >
2023-12-26 04:31:41 +00:00
niebayes
bab198ae68
feat(remote_wal): impl kafka log store ( #2971 )
...
* feat: introduce client manager
* chore: add errors for client manager
* chore: add record utils
* chore: impl kafka log store
* chore: build kafka log store upon starting datanode
* chore: update comments for kafka log store
* chore: add a todo for getting entry offset
* fix: typo
* chore: remove unused
* chore: update comments
* fix: typo
* fix: resolve some review conversations
* chore: move commonly referenced crates to workspace Cargo.toml
* fix: style
* fix: style
* chore: unify topic name prefix
* chore: make backoff config configurable by users
* chore: properly use backoff config in wal config
* refactor: read/write of kafka log store
* fix: typo
* fix: typo
* fix: resolve review conversations
2023-12-25 09:21:52 +00:00
niebayes
7d509e97f6
chore: move some commonly referenced crates to workspace Cargo.toml ( #2981 )
...
fix: resolve conflicts
2023-12-22 09:13:18 +00:00
niebayes
9da1f236d9
feat(remote_wal): add skeleton for remote wal related to datanode ( #2941 )
...
* refactor: refactor wal config
* test: update tests related to wal
* feat: introduce kafka wal config
* chore: augment proto with wal options
* feat: augment region open request with wal options
* feat: augment mito region with wal options
* feat: augment region create request with wal options
* refactor: refactor log store trait
* feat: add skeleton for kafka log store
* feat: generalize building log store when starting datanode
* feat: integrate wal options to region write
* chore: minor update
* refactor: remove wal options from region create/open requests
* fix: compliation issues
* chore: insert wal options into region options upon initializing region server
* chore: integrate wal options into region options
* chore: fill in kafka wal config
* chore: reuse namespaces while writing to wal
* chore: minor update
* chore: fetch wal options from region while handling truncate/flush
* fix: region options test
* fix: resolve some review conversations
* refactor: serde with wal options
* fix: resolve some review conversations
2023-12-20 09:01:17 +00:00
Weny Xu
fb8d0c6ce5
refactor: unify the KvBackend ( #2684 )
...
* refactor: unify the KvBackend
* refactor: rename kv_store to kv_backend
* chore: apply suggestions from CR
* refactor: rename kv_store to kv_backend
* refactor: rename KvStoreConfig to KvBackendConfig
2023-11-03 02:46:43 +00:00
ZhangJian He
d9e7b898a3
feat: add walconfig dir back ( #2606 )
...
Signed-off-by: ZhangJian He <shoothzj@gmail.com >
2023-10-16 11:26:06 +00:00
Baasit
19f300fc5a
feat: renaming kv directory to metadata ( #2549 )
...
* fix: renamed kv directory to metadata directory
* fix: changed function name
* fix: changed function name
2023-10-09 11:43:17 +00:00
Zhenchi
201acd152d
fix: missing file engine with default options ( #2519 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-09-28 10:25:12 +00:00
Weny Xu
f386329e29
refactor: introduce DdlTaskExecutor and refactor statement executor ( #2341 )
...
* feat: add kv store option
* refactor: refactor statement executor
* refactor: refactor standalone table creator
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* refactor: move ShowCreateTable and CreateDatabase to StatementExecutor
* fix: fix RegionDistribution
* feat: build standalone
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2023-09-12 07:57:15 -05:00
Ruihang Xia
f71aa373c1
feat: start datanode with config ( #2312 )
...
* remove memory-catalog and procedure
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* derive serde for MitoConfig
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* start datanode with configs
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove dir in WalConfig
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add rename field attr
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add stupid duplicated mito config
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove wrong import
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* wired compile error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-09-12 07:57:15 -05:00
