WU Jingdi
816d94892c
feat: support HTTP&gRPC&pg set timezone ( #3125 )
...
* feat: support HTTP&gRPC&pg set timezone
* chore: fix code advice
* chore: fix code advice
2024-01-15 06:29:31 +00:00
LFC
93f28c2a37
refactor: make grpc service able to be added dynamically ( #3160 )
2024-01-15 04:33:27 +00:00
Eugene Tolbakov
ca4d690424
feat: add modulo function ( #3147 )
...
* feat: add modulo function
* fix: address CR feedback
2024-01-13 00:24:25 +00:00
Weny Xu
75975adcb6
fix: fix tests failed on windows ( #3155 )
...
* fix: fix tests failed on windows
* feat: add comments
* Update src/object-store/src/util.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-12 09:12:11 +00:00
Ruihang Xia
527e523a38
fix: handle non-identical time index and field column in PromQL set operation ( #3145 )
...
* handle different field columns
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix and/unless on different time index
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-12 06:27:03 +00:00
Weny Xu
aad2afd3f2
chore: bump version to 0.6.0 ( #3154 )
2024-01-12 06:25:14 +00:00
Weny Xu
bf88b3b4a0
fix: fix store all wal options ( #3149 )
...
* fix: fix store all wal options
* fix: incorrect updating DatanodeTable value
2024-01-12 04:48:14 +00:00
Weny Xu
bf96ce3049
fix: print detailed error ( #3146 )
2024-01-12 04:02:32 +00:00
Weny Xu
430ffe0e28
fix(kafka): overwrite the EntryId with Offset while consuming records ( #3148 )
...
* fix(kafka): overwrite the EntryId with Offset while consuming the KafkaRecords
* fix: temporarily workaround of incorrect entry Id
2024-01-12 03:46:17 +00:00
Zhenchi
c1190bae7b
feat(mito): support write cache for index file ( #3144 )
...
* feat(mito): support write cache for index file
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: merge main
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-01-12 02:40:56 +00:00
Ruihang Xia
0882da4d01
feat: support PromQL operations over the same metric ( #3124 )
...
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update ut cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deadcode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-11 23:07:17 +00:00
Wei
8ec1e42754
feat: read data from write cache ( #3128 )
...
* feat: read from write cache
* chore: add read ranges test
* fix: use get instead of contains_key
* chore: clippy
* chore: cr comment
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: with_label_values
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-11 12:06:28 +00:00
Ruihang Xia
b00b49284e
feat: manager kafka cluster in sqlness runner ( #3143 )
...
* feat: manager kafka cluster in sqlness runner
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* pull up clippy config
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: niebayes <niebayes@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: niebayes <niebayes@gmail.com >
2024-01-11 09:47:19 +00:00
Ruihang Xia
09b3c7029b
feat: handle drop request for metric table ( #3136 )
...
* handle drop request
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* adjust procedure manager
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add create table sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* insert/query metric table
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* address CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/common/meta/src/kv_backend.rs
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* reuse region option for metadata region
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tweak variable name
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-11 09:38:43 +00:00
niebayes
f5798e2833
fix: remove incorrect wal comments in config file ( #3142 )
...
fix: kafka config comments
2024-01-11 09:34:24 +00:00
Zhenchi
fd8fb641fd
feat(parquet): introduce inverted index applier to reader ( #3130 )
...
* feat(parquet): introduce inverted index applier to reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: purger removes index file
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix test
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: add TODO for escape route
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: add TODO for escape route
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/mito2/src/access_layer.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* Update src/mito2/src/sst/parquet/reader.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* feat: min-max index to prune row groups filtered by inverted index
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: file_meta.inverted_index_available -> file_meta.available_indexes
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: add TODO for leveraging WriteCache
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix fmt
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: misset available indexes
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add index file size
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: use smallvec to reduce heap allocation
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: add index size to disk usage
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-01-11 08:04:59 +00:00
Weny Xu
312e8e824e
fix: save code in debug_assert! ( #3137 )
...
fix: save code in debug_assert!
2024-01-11 06:07:08 +00:00
Wei
29a7f301df
feat: write and upload sst ( #3106 )
...
* feat: write and upload sst file
* refactor: unit test
* cr comment
* chore: typos
* chore: cr comment
* chore: conflict
* Apply suggestions from code review
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* chore: fmt
* chore: style
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-11 02:34:16 +00:00
LFC
51a3fbc7bf
refactor: change how frontend grpc services are orchestrated ( #3134 )
2024-01-11 02:26:44 +00:00
Lanqing Yang
d521bc9dc5
chore: impl KvBackend for MetaPeerClient ( #3076 )
2024-01-10 14:16:03 +00:00
Weny Xu
7fad4e8356
fix: incorrect parsing broker_endpoints env variable ( #3135 )
2024-01-10 13:59:49 +00:00
Ning Sun
b6033f62cd
refactor: implement version as built-in function and use fixed mysql version ( #3133 )
...
* refactor: implement version as built-in function
* test: add sqlness test for version()
2024-01-10 11:04:18 +00:00
dennis zhuang
fd3f23ea15
feat: adds runtime_metrics ( #3127 )
...
* feat: adds runtime_metrics
* fix: comment
* feat: refactor metrics table
* chore: ensure build_info and runtime_metrics only avaiable in greptime catalog
* feat: adds timestamp column
2024-01-10 10:51:30 +00:00
niebayes
1b0e39a7f2
chore: stop exposing num_partitions ( #3132 )
2024-01-10 10:45:18 +00:00
Weny Xu
3ab370265a
feat: expose the region migration replay_timeout argument ( #3129 )
...
* feat: expose region migration args
* fix: fix ci
2024-01-10 09:47:59 +00:00
Weny Xu
ec8266b969
refactor: refactor the locks in the procedure ( #3126 )
...
* feat: add lock key
* refactor: procedure lock keys
* chore: apply suggestions from CR
2024-01-10 09:46:39 +00:00
Zhenchi
490312bf57
fix: unstable time record test ( #3131 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-01-10 09:41:52 +00:00
Ning Sun
1fc168bf6a
feat: update our cross schema check to cross catalog ( #3123 )
2024-01-09 09:38:48 +00:00
Zhenchi
db98484796
feat(inverted_index): introduce SstIndexCreator ( #3107 )
...
* feat(inverted_index): introduce SstIndexCreator
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: tiny polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: distinguish intermediate store and index store
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: move comment as doc comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: column id as index name
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-01-09 09:24:16 +00:00
Ruihang Xia
7d0d2163d2
fix: expose unsupported datatype error on mysql protocol ( #3121 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-09 09:13:53 +00:00
Ruihang Xia
c4582c05cc
chore: change the default doc checkbox to no need ( #3122 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-09 17:12:54 +08:00
niebayes
a0a31c8acc
chore(remote_wal): remove topic alias ( #3120 )
...
chore: remove topic alias
2024-01-09 07:35:02 +00:00
tison
0db1861452
chore(python): Print Python interpreter version ( #3118 )
...
* chore(pyo3_backend): Print bundle Python interpreter version
Signed-off-by: tison <wander4096@gmail.com >
* print RustPython interpreter version on init
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-01-09 07:04:23 +00:00
Wei
225ae953d1
feat: add parquet metadata to cache ( #3097 )
...
* feat: parquet metadata to sst meta cache
* chore: clippy
* refactor: move code to access_layer
* chore: clone()
2024-01-09 07:00:42 +00:00
Lei, HUANG
2c1b1cecc8
chore: add bound check for raft-engine logstore ( #3073 )
...
* chore: add bound check for raft-engine logstore
* feat: add bound check to append_batch API
* chore: check entry id during replay
* chore: resolve conflicts
* feat: add allow_stale_entries options to force obsolete wal entries
* chore: resolve some comments
2024-01-09 06:42:46 +00:00
Lei, HUANG
62db28b465
feat: add options to enable log recycle and periodical fsync ( #3114 )
...
* feat: add options to enable log recycle and periodical fsync
* fix: resolve review comments
* fix: conflicts
2024-01-09 06:41:23 +00:00
fys
6e860bc0fd
feat: support grpc for otlp trace and metrics ( #3105 )
...
* feat: add grpc support for otlp trace and metrics
* cr: add some comment
* fix: ut
* fix: cr
2024-01-09 05:01:48 +00:00
Yingwen
8bd4a36136
feat(mito): Init the write cache in datanode ( #3100 )
...
* feat: add builder to build cache manager
* refactor: make MitoEngine::new async
* refactor: refactor object store creation
* refactor: add helper fn to attaches layers
* feat: fn to build fs store
* feat: add write cache to engine
* feat: config write cache
* style: fix clippy
* test: fix test
* feat: add warning
* chore: add experimental prefix to configs
* test: fix config test
* test: test weighted size
* feat: add switch to enable write cache
* fix: update cache stats by using get
* style: use then
2024-01-09 04:40:22 +00:00
Ruihang Xia
af0c4c068a
feat: support PromQL function vector ( #3036 )
...
* produce vector plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* work with OR
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* apply review sugg
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move common const strings to common_query
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add comment for GREPTIME_COUNT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-09 03:44:00 +00:00
dennis zhuang
26cbcb8b3a
docs: update issue template ( #3119 )
2024-01-09 02:45:55 +00:00
niebayes
122b47210e
chore: bump version to 0.5.1 ( #3116 )
2024-01-08 11:32:56 +00:00
tison
316d843482
feat: support CSV format in sql HTTP API ( #3062 )
...
* chore: fix typo
Signed-off-by: tison <wander4096@gmail.com >
* add csv format
Signed-off-by: tison <wander4096@gmail.com >
* flatten response
Signed-off-by: tison <wander4096@gmail.com >
* more flatten response
Signed-off-by: tison <wander4096@gmail.com >
* add CSV format
Signed-off-by: tison <wander4096@gmail.com >
* format InfluxdbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format ErrorResponse
Signed-off-by: tison <wander4096@gmail.com >
* propagate ErrorResponse to InfluxdbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format GreptimedbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format CsvResponse
Signed-off-by: tison <wander4096@gmail.com >
* impl IntoResponse for QueryResponse
Signed-off-by: tison <wander4096@gmail.com >
* promql
Signed-off-by: tison <wander4096@gmail.com >
* sql
Signed-off-by: tison <wander4096@gmail.com >
* compile
Signed-off-by: tison <wander4096@gmail.com >
* fixup aide
Signed-off-by: tison <wander4096@gmail.com >
* clear debt
Signed-off-by: tison <wander4096@gmail.com >
* fixup UT test_recordbatches_conversion
Signed-off-by: tison <wander4096@gmail.com >
* fixup IT cases
Signed-off-by: tison <wander4096@gmail.com >
* fixup more IT cases
Signed-off-by: tison <wander4096@gmail.com >
* fixup test-integration cases
Signed-off-by: tison <wander4096@gmail.com >
* update comment
Signed-off-by: tison <wander4096@gmail.com >
* fixup deserialize and most query < 1ms
Signed-off-by: tison <wander4096@gmail.com >
* fixup auth tests
Signed-off-by: tison <wander4096@gmail.com >
* fixup tests
Signed-off-by: tison <wander4096@gmail.com >
* fixup and align X-GreptimeDB headers
Signed-off-by: tison <wander4096@gmail.com >
* fixup compile
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-01-08 10:54:27 +00:00
niebayes
8c58d3f85b
test(remote_wal): add unit tests for kafka remote wal ( #2993 )
...
* test: add unit tests
* feat: introduce kafka runtime backed by testcontainers
* test: add test for kafka runtime
* fix: format
* chore: make kafka image ready to be used
* feat: add entry builder
* tmp
* test: add unit tests for client manager
* test: add some unit tests for kafka log store
* chore: resolve some todos
* chore: resolve some todos
* test: add unit tests for kafka log store
* chore: add deprecate develop branch warning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tmp: ready to move unit tests to an indie dir
* test: update unit tests for client manager
* test: add unit tests for meta srv remote wal
* fix: license
* fix: test
* refactor: kafka image
* doc: add doc example for kafka image
* chore: migrate kafka image to an indie PR
* fix: CR
* fix: CR
* fix: test
* fix: CR
* fix: update Cargo.toml
* fix: CR
* feat: skip test if no endpoints env
* fix: format
* test: rewrite parallel test with barrier
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-08 10:48:11 +00:00
LFC
fcacb100a2
chore: expose some codes to let other projects use them ( #3115 )
2024-01-08 06:32:01 +00:00
Weny Xu
58ada1dfef
fix: check env before running kafka test ( #3110 )
...
* fix: check env before running kafka test
* Apply suggestions from code review
Co-authored-by: niebayes <niebayes@gmail.com >
---------
Co-authored-by: niebayes <niebayes@gmail.com >
2024-01-08 06:30:43 +00:00
Weny Xu
f78c467a86
chore: bump opendal to 0.44.1 ( #3111 )
2024-01-08 03:55:58 +00:00
niebayes
78303639db
feat(remote_wal): split an entry if it's too large ( #3092 )
...
* feat: split an entry if it's too large
* chore: rewrite check records
* test: add some unit tests for record
* chore: rewrite entry splitting
* chore: add unit tests for build records
* chore: add more unit tests for record
* chore: rewrite encdec of record
* revert: ignored test
* fix: set limit for max_batch_size
* fix: clippy
* chore: remove heavy logging
* fix: CR
* fix: properly terminate
* fix: CR
* fix: compiling
* fix: sqlness
* fix: CR
* fix: license
* fix: license
2024-01-05 12:41:43 +00:00
JeremyHi
bd1a5dc265
feat: metric engine support alter ( #3098 )
...
* feat: metric engine support alter
* chore: by comment
* feat: get physical table route for frontend
2024-01-05 09:46:39 +00:00
Weny Xu
e0a43f37d7
chore: bump opendal to 0.44 ( #3058 )
...
* chore: bump opendal to 0.44
* fix: fix test_object_store_cache_policy
* Revert "fix: fix test_object_store_cache_policy"
This reverts commit 46c37c343f66114e0f6ee7a0a3b9ee2b79c810af.
* fix: fix test_object_store_cache_policy
* fix: fix test_file_backend_with_lru_cache
* chore: apply suggestions from CR
* fix(mito): fix mito2 cache
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-01-05 09:05:41 +00:00
zyy17
a89840f5f9
refactor(metrics): add 'greptime_' prefix for every metrics ( #3093 )
...
* refactor(metrics): add 'greptimedb_' prefix for every metrics
* chore: use 'greptime_' as prefix
* chore: add some prefix for new metrics
* chore: fix format error
2024-01-05 08:12:23 +00:00
