Yingwen
06dcd0f6ed
fix: freeze data buffer in shard ( #3468 )
...
* feat: call freeze if the active data buffer in a shard is full
* chore: more metrics
* chore: print metrics
* chore: enlarge freeze threshold
* test: test freeze
* test: fix config test
2024-03-11 14:51:06 +00:00
Ruihang Xia
b7ac8d6aa8
ci: use another mirror for etcd image ( #3486 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-03-11 10:40:19 +00:00
crwen
aa125a50f9
refactor: make http api returns non-200 status code ( #3473 )
...
* refactor: make http api returns non-200 status code
* recover some code
2024-03-11 03:38:36 +00:00
shuiyisong
0bb949787c
refactor: introduce new Output with OutputMeta ( #3466 )
...
* refactor: introduce new output struct
* chore: add helper function
* chore: update comment
* chore: update commit
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: rename according to cr
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-03-11 02:24:09 +00:00
Weny Xu
a309cd018a
fix: fix incorrect COM_STMT_PREPARE reply ( #3463 )
...
* fix: fix incorrect `COM_STMT_PREPARE` reply
* chore: use column name instead of index
2024-03-08 07:31:20 +00:00
JeremyHi
352bd7b6fd
feat: max-txn-ops option ( #3458 )
...
* feat: max-txn-ops limit
* chore: by comment
2024-03-08 02:34:40 +00:00
Ruihang Xia
020635063c
feat: implement multi-dim partition rule ( #3409 )
...
* generate expr rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* implement show create for new partition rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* implement row spliter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: fix failed tests
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: fix lint issues
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: ignore tests for deprecated partition rule
* chore: remove unused partition rule tests setup
* test(sqlness): add basic partition tests
* test(multi_dim): add basic find region test
* address CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
Co-authored-by: WenyXu <wenymedia@gmail.com >
2024-03-05 11:39:15 +00:00
Lei, HUANG
7183fa198c
refactor: make MergeTreeMemtable the default choice ( #3430 )
...
* refactor: make MergeTreeMemtable the default choice
* refactor: reformat
* chore: add doc to config
2024-03-05 10:00:08 +00:00
Yingwen
49157868f9
feat: Correct server metrics and add more metrics for scan ( #3426 )
...
* feat: drop timer on stream terminated
* refactor: combine metrics into a histogram vec
* refactor: frontend grpc metrics
* feat: add metrics middleware layer to grpc server
* refactor: move http metrics layer to metrics mod
* feat: bucket for grpc/http elapsed
* feat: remove duplicate metrics
* style: fix cilppy
* fix: incorrect bucket of promql series
* feat: more metrics for mito
* feat: convert cost
* test: fix metrics test
2024-03-04 10:15:10 +00:00
dennis zhuang
e6819412c5
refactor: show tables and show databases ( #3423 )
...
* refactor: show tables and show databases
* chore: clean code
2024-03-04 06:15:17 +00:00
Ning Sun
d4a54a085b
feat: add configuration for tls watch option ( #3395 )
...
* feat: add configuration for tls watch option
* test: sleep longer to ensure async task run
* test: update config api integration test
* refactor: rename function
2024-03-01 03:49:54 +00:00
Weny Xu
a8cbec824c
refactor: refactor TableRouteManager ( #3392 )
...
* feat: introduce TableRouteStorage
* refactor: remove get & batch_get in TableRouteManager
* refactor: move txn related fn to TableRouteStorage
* chore: apply suggestions from CR
* chore(codecov): ingore tests-integration dir
2024-02-28 06:18:09 +00:00
dennis zhuang
4b36c285f1
feat: flush or compact table and region functions ( #3363 )
...
* feat: adds Requester to process table flush and compaction request
* feat: admin_fn macros for administration functions
* test: add query result
* feat: impl flush_region, flush_table, compact_region, and flush_region functions
* docs: add Arguments to admin_fn macro
* chore: apply suggestion
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: apply suggestion
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: group_requests_by_peer and adds log
* Update src/common/macro/src/admin_fn.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* feat: adds todo for spawan thread
* feat: rebase with main
---------
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-27 08:57:38 +00:00
Ruihang Xia
3544c9334c
feat!: new partition grammar - parser part ( #3347 )
...
* parser part
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test in sql
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* comment out and ignore some logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update region migration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* temporary disable region migration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* allow dead code
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-27 07:20:16 +00:00
Ning Sun
3887d207b6
feat: make tls certificates/keys reloadable (part 1) ( #3335 )
...
* feat: make tls certificates/keys reloadable (part 1)
* feat: add notify watcher for cert/key files
* test: add unit test for watcher
* fix: correct usage of watcher
* fix: skip watch when tls disabled
2024-02-26 09:37:54 +00:00
dennis zhuang
564fe3beca
feat: impl migrate_region and procedure_state SQL function ( #3325 )
...
* fix: logical region can't find region routes
* feat: fetch partitions info in batch
* refactor: rename batch functions
* refactor: rename DdlTaskExecutor to ProcedureExecutor
* feat: impl migrate_region and query_procedure_state for ProcedureExecutor
* feat: adds SQL function procedure_state and finish migrate_region impl
* fix: constant vector
* feat: unit tests for migrate_region and procedure_state
* test: test region migration by SQL
* fix: compile error after rebeasing
* fix: clippy warnings
* feat: ensure procedure_state and migrate_region can be only called under greptime catalog
* fix: license header
2024-02-22 02:37:11 +00:00
SteveLauC
e9a2b0a9ee
chore: use workspace-wide lints ( #3352 )
...
* chore: use workspace-wide lints
* respond to review
2024-02-22 01:01:10 +00:00
tison
085a380019
build(deps): axum-tets-helper has included patch-1 ( #3333 )
...
Signed-off-by: tison <wander4096@gmail.com >
2024-02-21 07:49:42 +00:00
Weny Xu
41656c8635
refactor: allocate table id in the procedure ( #3271 )
...
* refactor: replace TableMetadataManager with TableNameManager
* refactor: allocate table id in the procedure
* refactor: refactor client logical of handling retries
* feat(test_util): add TestCreateTableExprBuilder
* feat(test_util): add MockDatanodeManager
* feat(test_util): add new_ddl_context
* feat(test_util): add build_raw_table_info_from_expr
* feat(test_util): add MockDatanodeManager::new
* feat(procedure): add downcast_output_ref to Status
* test(create_table): add tests for CreateTableProcedure on_prepare
* refactor(ddl): rename handle_operate_region_error to add_peer_context_if_need
* test(create_table): add tests for CreateTableProcedure on_datanode_create_regions
* test(create_table): add tests for CreateTableProcedure on_create_metadata
* refactor(meta): use CreateTableExprBuilder
* feat(create_table): ensure number of partitions is greater than 0
* refactor: rename to add_peer_context_if_needed
* feat: add context for panic
* refactor: simplify the should_retry
* refactor: use Option<&T> instead of &Option<T>
* refactor: move downcast_output_ref under cfg(test)
* chore: fmt toml
2024-02-21 04:38:46 +00:00
shuiyisong
bf5e1905cd
refactor: bring metrics to http output ( #3247 )
...
* refactor: bring metrics to http output
* chore: remove unwrap
* chore: make walk plan accumulate
* chore: change field name and comment
* chore: add metrics to http resp header
* chore: move PrometheusJsonResponse to a separate file and impl IntoResponse
* chore: put metrics in prometheus resp header too
2024-02-20 03:25:18 +00:00
tison
4e04a4e48f
build: support build without git ( #3309 )
...
* build: support build without git
Signed-off-by: tison <wander4096@gmail.com >
* chore
Signed-off-by: tison <wander4096@gmail.com >
* address comment
Signed-off-by: tison <wander4096@gmail.com >
* fix syntax
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-02-18 10:30:01 +00:00
fys
74bfb09195
feat: support cache for batch_get in CachedMetaKvBackend ( #3277 )
...
* feat: support cache for batch_get in CachedMetaKvBackend.
* add doc of CachedMetaKvBackend
* fix: cr
* fix: correct some words
* fix cr
2024-02-06 03:15:03 +00:00
Ning Sun
1bc4f25de2
feat: http sql api return schema on empty resultset ( #3237 )
...
* feat: return schema on empty resultset
* refactor: make schema a required field in http output
* test: update integration test and add schema output
2024-01-25 06:44:28 +00:00
Weny Xu
f82ddc9491
fix: fix MockInstance rebuild issue ( #3218 )
...
* fix: fix MockInstance rebuild issue
* chore: apply suggestions from CR
2024-01-24 07:52:47 +00:00
LFC
f81e37f508
refactor: make http server built flexibly ( #3225 )
...
* refactor: make http server built flexibly
* Apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix: resolve PR comments
* Fix CI.
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-24 03:45:08 +00:00
Weny Xu
d75cf86467
fix: only register region keeper while creating physical table ( #3223 )
...
fix: only register region keeper during create physical table
2024-01-23 09:42:32 +00:00
Weny Xu
19413eb345
refactor!: rename initialize_region_in_background to init_regions_in_background ( #3216 )
...
refactor!: change initialize_region_in_background to init_regions_in_background
2024-01-23 04:33:48 +00:00
Ruihang Xia
31787f4bfd
feat!: switch prom remote write to metric engine ( #3198 )
...
* feat: switch prom remote write to metric engine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* read physical table name from url
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove physical table from header
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix merge error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add with_metric_engine option to config remote write behavior
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* check parameter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add specific config param
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* default with_metric_engine to true
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update UT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-01-22 14:48:28 +00:00
dennis zhuang
5e89472b2e
feat: adds parse options for SQL parser ( #3193 )
...
* feat: adds parse options for parser and timezone to scan request
* chore: remove timezone in ScanRequest
* feat: remove timezone in parse options and adds type checking to parititon columns
* fix: comment
* chore: apply suggestions
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: format
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-19 09:16:36 +00:00
Zhenchi
2e4c48ae7a
fix(index): S3 EntityTooSmall error ( #3192 )
...
* fix(index): S3 `EntityTooSmall` error
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: config api
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-01-19 02:57:07 +00:00
niebayes
63205907fb
refactor: introduce common-wal to aggregate wal stuff ( #3171 )
...
* refactor: aggregate wal configs
* refactor: move wal options to common-wal
* chore: slim Cargo.toml
* fix: add missing crates
* fix: format
* chore: update comments
* chore: add testing feature gate for test_util
* fix: apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix: apply suggestions from code review
* fix: compiling
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-18 03:49:37 +00:00
Wei
3d7d2fdb4a
feat: auto config cache size according to memory size ( #3165 )
...
* feat: auto config cache and buffer size according to mem size
* feat: utils
* refactor: add util function to common config
* refactor: check cgroups
* refactor: code
* fix: test
* fix: test
* chore: cr comment
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
* chore: remove default comment
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
2024-01-17 14:35:35 +00:00
niebayes
d020a3db23
chore: expose promql test to distributed instance ( #3176 )
2024-01-17 02:44:55 +00:00
Zhenchi
6f07d69155
feat(mito): enable inverted index ( #3158 )
...
* feat(mito): enable inverted index
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix typos
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix typos
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* accidentally resolved the incorrect filtering issue within the Metric Engine
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix test
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/mito2/src/access_layer.rs
* Update src/mito2/src/test_util/scheduler_util.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: format -> join_dir
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: move intermediate_manager from arg of write_and_upload_sst to field of WriteCache
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: add IndexerBuidler
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix clippy
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-15 09:08:07 +00:00
WU Jingdi
816d94892c
feat: support HTTP&gRPC&pg set timezone ( #3125 )
...
* feat: support HTTP&gRPC&pg set timezone
* chore: fix code advice
* chore: fix code advice
2024-01-15 06:29:31 +00:00
LFC
93f28c2a37
refactor: make grpc service able to be added dynamically ( #3160 )
2024-01-15 04:33:27 +00:00
Ruihang Xia
0882da4d01
feat: support PromQL operations over the same metric ( #3124 )
...
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update ut cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deadcode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-11 23:07:17 +00:00
LFC
51a3fbc7bf
refactor: change how frontend grpc services are orchestrated ( #3134 )
2024-01-11 02:26:44 +00:00
Weny Xu
3ab370265a
feat: expose the region migration replay_timeout argument ( #3129 )
...
* feat: expose region migration args
* fix: fix ci
2024-01-10 09:47:59 +00:00
Lei, HUANG
2c1b1cecc8
chore: add bound check for raft-engine logstore ( #3073 )
...
* chore: add bound check for raft-engine logstore
* feat: add bound check to append_batch API
* chore: check entry id during replay
* chore: resolve conflicts
* feat: add allow_stale_entries options to force obsolete wal entries
* chore: resolve some comments
2024-01-09 06:42:46 +00:00
Lei, HUANG
62db28b465
feat: add options to enable log recycle and periodical fsync ( #3114 )
...
* feat: add options to enable log recycle and periodical fsync
* fix: resolve review comments
* fix: conflicts
2024-01-09 06:41:23 +00:00
fys
6e860bc0fd
feat: support grpc for otlp trace and metrics ( #3105 )
...
* feat: add grpc support for otlp trace and metrics
* cr: add some comment
* fix: ut
* fix: cr
2024-01-09 05:01:48 +00:00
Yingwen
8bd4a36136
feat(mito): Init the write cache in datanode ( #3100 )
...
* feat: add builder to build cache manager
* refactor: make MitoEngine::new async
* refactor: refactor object store creation
* refactor: add helper fn to attaches layers
* feat: fn to build fs store
* feat: add write cache to engine
* feat: config write cache
* style: fix clippy
* test: fix test
* feat: add warning
* chore: add experimental prefix to configs
* test: fix config test
* test: test weighted size
* feat: add switch to enable write cache
* fix: update cache stats by using get
* style: use then
2024-01-09 04:40:22 +00:00
tison
316d843482
feat: support CSV format in sql HTTP API ( #3062 )
...
* chore: fix typo
Signed-off-by: tison <wander4096@gmail.com >
* add csv format
Signed-off-by: tison <wander4096@gmail.com >
* flatten response
Signed-off-by: tison <wander4096@gmail.com >
* more flatten response
Signed-off-by: tison <wander4096@gmail.com >
* add CSV format
Signed-off-by: tison <wander4096@gmail.com >
* format InfluxdbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format ErrorResponse
Signed-off-by: tison <wander4096@gmail.com >
* propagate ErrorResponse to InfluxdbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format GreptimedbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format CsvResponse
Signed-off-by: tison <wander4096@gmail.com >
* impl IntoResponse for QueryResponse
Signed-off-by: tison <wander4096@gmail.com >
* promql
Signed-off-by: tison <wander4096@gmail.com >
* sql
Signed-off-by: tison <wander4096@gmail.com >
* compile
Signed-off-by: tison <wander4096@gmail.com >
* fixup aide
Signed-off-by: tison <wander4096@gmail.com >
* clear debt
Signed-off-by: tison <wander4096@gmail.com >
* fixup UT test_recordbatches_conversion
Signed-off-by: tison <wander4096@gmail.com >
* fixup IT cases
Signed-off-by: tison <wander4096@gmail.com >
* fixup more IT cases
Signed-off-by: tison <wander4096@gmail.com >
* fixup test-integration cases
Signed-off-by: tison <wander4096@gmail.com >
* update comment
Signed-off-by: tison <wander4096@gmail.com >
* fixup deserialize and most query < 1ms
Signed-off-by: tison <wander4096@gmail.com >
* fixup auth tests
Signed-off-by: tison <wander4096@gmail.com >
* fixup tests
Signed-off-by: tison <wander4096@gmail.com >
* fixup and align X-GreptimeDB headers
Signed-off-by: tison <wander4096@gmail.com >
* fixup compile
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-01-08 10:54:27 +00:00
Weny Xu
f1a4750576
feat(tests-integration): add more region migration integration tests ( #3094 )
2024-01-04 08:18:46 +00:00
WU Jingdi
4d250ed054
fix: Optimize export metric behavior ( #3047 )
...
* fix: optimze export metric bahavior
* chor: fix ci
* chore: update config format
* chore: fix format
2024-01-04 06:40:50 +00:00
LFC
ec43b9183d
feat: table route for metric engine ( #3053 )
...
* feat: table route for metric engine
* feat: register logical regions
* fix: open logical region (#96 )
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-04 06:30:17 +00:00
ClSlaid
5c66ce6e88
chore: remove unnecessary result wrappings ( #3084 )
...
patch: remove unnecessary result wrappings
Signed-off-by: 蔡略 <cailue@bupt.edu.cn >
2024-01-03 10:20:33 +00:00
Weny Xu
611a8aa2fe
feat(tests-integration): add a naive region migration integration test ( #3078 )
...
* fix: fix heartbeat handler ignore upgrade candidate instruction
* fix: fix handler did not inject wal options
* feat: expose `RegionMigrationProcedureTask`
* feat(tests-integration): add a naive region migration test
* chore: apply suggestions from CR
* feat: add test if the target region has migrated
* chore: apply suggestions from CR
2024-01-03 07:12:59 +00:00
Weny Xu
128d3717fa
test(tests-integration): add a naive test with kafka wal ( #3071 )
...
* chore(tests-integration): add setup tests with kafka wal to README.md
* feat(tests-integration): add meta wal config
* fix(tests-integration): fix sign of both_instances_cases_with_kafka_wal
* chore(tests-integration): set num_topic to 3 for tests
* test(tests-integration): add a naive test with kafka wal
* chore: apply suggestions from CR
2024-01-02 09:05:20 +00:00
