Weny Xu
97c3755ab6
feat(mito): add skip_wal_replay option to OpenRegionRequest ( #2955 )
...
* feat(mito): add skip_replay_wal option to OpenRegionRequest
* test: add tests for skip replay wal
* chore: rename `skip_replay_wal` to `skip_wal_replay`
2023-12-20 03:31:48 +00:00
JeremyHi
5f8c17514f
chore: SelectorType use snake_case ( #2962 )
...
chore: let SelectorType use snake_case
2023-12-20 02:28:29 +00:00
niebayes
839e653e0d
feat(remote_wal): add skeleton for remote wal related to meta srv ( #2933 )
...
* feat: introduce wal config and kafka config
* feat: introduce kafka topic manager and selector
* feat: introduce region wal options
* chore: build region wal options upon starting meta srv
* feat: integrate region wal options allocator into table meta allocator
* chore: add wal config to metasrv.example.toml
* chore: add region wal options map to create table procedure
* feat: augment region create request with wal options
* feat: augment DatanodeTableValue with region wal options map
* chore: encode region wal options upon constructing table creator
* feat: persist region wal options when creating table meta
* fix: sqlness test
* chore: set default wal provider to raft-engine
* refactor: refactor wal options
* chore: update wal options allocator
* refactor: rename region wal options to wal options
* chore: update usages of region wal options
* chore: add some comments to kafka
* chore: fill in kafka config
* test: add tests for serde wal config
* test: add tests for wal options
* refactor: refactor wal options allocator to enum
* refactor: store wal options into the request options instead
* fix: typo
* fix: typo
* refactor: move wal options map to region info
* refactor: refacto serialization and deserialization of wal options
* refactor: use serde_json to encode wal options
* chore: rename wal_options_map to region_wal_options
* chore: resolve some review comments
* fix: typo
* refactor: replace kecab-case with snake_case
* fix: sqlness and converage tests
* fix: typo
* fix: coverage test
* fix: coverage test
* chore: resolve some review conversations
* fix: resolve some review conversations
* chore: format comments in metasrv.example.toml
* chore: update import style
* feat: integrate wal options allocator to standalone mode
* test: add compatible test for OpenRegion
* test: add compatible test for UpdateRegionMetadata
* chore: remove based suffix from topic selector type
2023-12-19 12:43:47 +00:00
LFC
c7b36779c1
fix: correctly decode mysql timestamp binary value ( #2946 )
...
* fix: correctly decode mysql timestamp binary value
* fix: ut
* fix: resolve PR comments
2023-12-19 11:05:28 +00:00
Wei
bbcac3a541
fix: auto create datatype_extension missing ( #2953 )
2023-12-19 10:53:31 +00:00
dennis zhuang
600cde1ff2
fix: wrong link for selector ( #2958 )
2023-12-19 10:50:48 +00:00
Zhenchi
83de399bef
feat(inverted_index.create): add external sorter ( #2950 )
...
* feat(inverted_index.create): add read/write for external intermediate files
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: MAGIC_CODEC_V1 -> CODEC_V1_MAGIC
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: polish comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: fix typos intermedia -> intermediate
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: typos
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat(inverted_index.create): add external sorter
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: fix typos intermedia -> intermediate
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: polish comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: polish comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: drop the stream as early as possible to avoid recursive calls to poll
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: project merge sorted stream
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add total_row_count to SortOutput
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: remove change of format
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: rename segment null bitmap
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: test type alias
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: allow `memory_usage_threshold` to be None to turn off dumping
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: change segment_row_count type to NonZeroUsize
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: accept BytesRef instead
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add `push_n` to adapt mito2
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: add k-way merge TODO
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: more sorter cases
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: make the merge tree balance
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/index/src/inverted_index/create/sort/external_sort.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* chore: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: stable feature
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-12-19 08:14:37 +00:00
Ruihang Xia
6b8dbcfb54
chore: update toolchain to 20231219 ( #2932 )
...
* update toolchain file, remove unused feature gates
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update action file
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update to 12-19
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-19 07:24:08 +00:00
Wei
3e6a564f8e
fix: blocking read timer lack of parameter ( #2954 )
2023-12-19 06:16:55 +00:00
Zhenchi
ccbd49777d
fix: correct the error message for snappy compress ( #2956 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-12-19 06:14:40 +00:00
Weny Xu
29fc2ea9d8
feat: fetch manifests in concurrent ( #2951 )
...
* feat: fetch manifests in concurrent
* chore: set fetching manifest concurrency limit to 16
2023-12-19 06:12:30 +00:00
WU Jingdi
d180e41230
fix: change range query time slot to [align_ts, align_ts + range) ( #2938 )
2023-12-19 02:35:14 +00:00
liyang
62d5fcbd76
refactor: greptimedb cluster sqlness test scripts ( #2947 )
2023-12-18 10:59:14 +00:00
Zhenchi
d339191e29
refactor: remove reduntant .compat() ( #2949 )
...
* refactor: remove reduntant `.compat()`
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* remove dep
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-12-18 10:36:45 +00:00
Zhenchi
029ff2f1e3
feat(inverted_index.create): add read/write for external intermediate files ( #2942 )
...
* feat(inverted_index.create): add read/write for external intermediate files
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: MAGIC_CODEC_V1 -> CODEC_V1_MAGIC
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: polish comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: fix typos intermedia -> intermediate
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: typos
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: futures_code -> asynchronous_codec
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: bump bytes to 1.5
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-12-18 09:44:48 +00:00
Ruihang Xia
9af9c0229a
feat: create table procedure for metric engine, part 1 ( #2943 )
...
* implementation
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* initialize
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove empty file
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* apply review sugg
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-18 08:03:28 +00:00
WU Jingdi
4383a69876
fix!: remove range calendar as To option ( #2940 )
...
fix: remove range calendar as `To` option
2023-12-18 07:48:49 +00:00
LFC
033a065359
refactor: make sequence bounded with max value ( #2937 )
...
* refactor: make sequence bounded with max value
(cherry picked from commit 3a8eba6f863327a96b617cd86ee2d39fac30abb2)
* fix: resolve PR comments
2023-12-18 07:05:28 +00:00
dennis zhuang
262a79a170
feat: adds some tables to information_schema ( #2935 )
...
* feat: adds engines table to information_schema
* feat: adds COLUMN_PRIVILEGES and COLUMN_STATISTICS
* feat: refactor memory tables
* chore: rename memory_tables
* test: adds unit tests
* chore: format
* chore: style
* fix: by cr comments
* refactor: tables
2023-12-18 06:10:22 +00:00
Wei
5dc7ce1791
fix: typos and bit operations ( #2944 )
...
* fix: typos and bit operation
* fix: helper
* chore: add tests in decimal128.rs and interval.rs
* chore: test
* chore: change proto version
* chore: clippy
2023-12-18 03:06:11 +00:00
discord9
e35a494a3f
test: fix a wronged test script ( #2934 )
2023-12-14 14:27:36 +00:00
shuiyisong
5dba373ede
chore: return json body under http status 401 ( #2924 )
...
* chore: change auth_fn to function and return response with json body
* chore: move unsupported to debug level
* chore: add docs and tests
* chore: rebase and update test
2023-12-14 10:01:12 +00:00
Weny Xu
518bac35bc
feat: add log and tracing layers for Copy From statement ( #2929 )
...
feat: add log and tracing layers
2023-12-14 06:15:30 +00:00
Ning Sun
39f80876cd
feat: upgrade rustls library family, opensrv-mysql and pgwire ( #2927 )
...
* feat: deps up
* fmt: toml format
2023-12-14 05:56:39 +00:00
LFC
181e16a11a
refactor: make instance started separately ( #2911 )
...
* refactor: make instance started separately, to support further integrated into other binaries
* fix: resolve PR comments
* fix: resolve PR comments
2023-12-14 03:44:29 +00:00
JeremyHi
99dda93f0e
feat: sql with influxdb v1 result format ( #2917 )
...
* feat: sql with influxdb v1 result format
* chore: add unit tests
* feat: minor refactor
* chore: by comment
* chore; u128 to u64 since serde can't deser u128 in enum
* chore: by comment
* chore: apply suggestion
* chore: revert suggestion
* chore: try again
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-12-13 16:15:37 +00:00
ZonaHe
d3da128d66
feat: update dashboard to v0.4.3 ( #2926 )
...
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com >
2023-12-13 11:12:17 +00:00
WU Jingdi
370ec04a9d
fix: use linear interpolation to implement range LINEAR fill strategy ( #2903 )
...
* fix: use linear interpolation to implement range LINEAR fill strategy
* chore: update test case
* chore: optimize linear interpolation implementation
* chore: update test and add comment
2023-12-13 09:53:35 +00:00
LFC
c13d2fd11d
ci: correctly find the binary in dev-build when using "dev" profile ( #2925 )
2023-12-13 09:04:44 +00:00
Yue Deng
3d651522c2
feat: add build() function to return the database build info ( #2919 )
...
* feat: add build function and register it
build() function to return the database build info #2909
* refactor: fix typos and change code structure
* test: add test for build()
* refactor: cargo fmt and eliminate warnings
* Apply suggestions from code review
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* refactor: move system.sql to a new directory
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2023-12-13 09:02:00 +00:00
Wei
fec3fcf4ef
feat: builder to vector without resetting ( #2915 )
...
* feat: finish_cloned() without resetting
* test: add unit cases
* chore: port comment
* chore: apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-12-13 06:48:09 +00:00
LFC
3555e1644c
ci: able to choose cargo profile in dev-build action ( #2923 )
2023-12-13 04:14:16 +00:00
niebayes
c42168d7c2
chore: remove useless storage apis ( #2904 )
...
* chore: remove metadata.rs
* chore: remove snapshot.rs
* chore: remove chunk.rs
* chore: remove engine.rs
* chore: remove MIN_OP_TYPE from types.rs
* chore: remove region.rs
* chore: remove almost all codes in requests.rs
* chore: remove WriteRequest from requests.rs
* chore: remove responses.rs
* chore: remove unused descriptors from descriptors.rs
* chore: remove unused consts from consts.rs
* chore: remove useless comments
2023-12-13 03:36:14 +00:00
Weny Xu
3c24ca1a7a
test: add more tests for region migration procedure ( #2895 )
...
* test: add flow test for open candidate region with retryable error
* test: add flow test for upgrade candidate retry failed
* test: add flow test for upgrade candidate with retry
2023-12-13 03:00:58 +00:00
Wei
9531469660
fix: date - interval sqlness ( #2912 )
...
fix: date - interval can work
2023-12-12 12:45:38 +00:00
LFC
880ca2e786
fix: return error to client if prepare stmt param not match ( #2918 )
...
* fix: return error to client if prepare stmt param not match
* Update src/servers/src/mysql/handler.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-12 12:41:09 +00:00
Ruihang Xia
0ce2b50676
feat!: do not get TZ info from server local env ( #2905 )
...
* feat: do not get TZ info from server local env
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add sqlness case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add empty line
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-12 12:41:05 +00:00
WU Jingdi
34635558d2
fix: support tailing commas in SQL ( #2913 )
...
* fix: support tailing commas in SQL
* fix: broken ci
2023-12-12 11:56:23 +00:00
Ruihang Xia
8a74bd36f5
style: rename *Adaptor to *Adapter ( #2914 )
...
* rename RecordBatchStreamAdaptor to RecordBatchStreamWrapper
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename ServerSqlQueryHandlerAdaptor to ServerSqlQueryHandlerAdapter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-12 09:45:09 +00:00
Weny Xu
cf6bba09fd
refactor: use downgrading the region instead of closing region ( #2863 )
...
* refactor: use downgrading the region instead of closing region
* feat: enhance the tests for alive keeper
* feat: add a metric to track region lease expired
* chore: apply suggestions from CR
* chore: enable logging for test_distributed_handle_ddl_request
* refactor: simplify lease keeper
* feat: add metrics for lease keeper
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* refactor: move OpeningRegionKeeper to common_meta
* feat: register operating regions to MemoryRegionKeeper
2023-12-12 09:24:17 +00:00
Ruihang Xia
89a0d3af1e
feat: set default debug level of release and dev profiles to 1 ( #2916 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-12 08:01:18 +00:00
tison
47e51545dd
ci: prevent running nightly FT in forks ( #2906 )
2023-12-12 02:49:52 +00:00
Zhenchi
1e22f1cb4f
feat(inverted_index.format): add writer ( #2900 )
...
* feat(inverted_index.format): add writer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: remove clippy allow
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/index/src/inverted_index/error.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-11 09:55:25 +00:00
dennis zhuang
cf8b6c77dc
chore: clean up unused errors ( #2901 )
2023-12-11 09:08:50 +00:00
Yingwen
6a57f4975e
perf(mito): scan SSTs and memtables in parallel ( #2852 )
...
* feat: seq scan support parallelism
* feat: scan region by parallelism in config
* feat: enlarge channel size
* chore: parallel builder logs
* feat: use parallel reader accroding to source num
* chore: 128 channel size
* feat: add fetch cost metrics
* feat: add channel size to config
* feat: builder cost
* feat: logs
* feat: compiler error
* feat: fetch cost
* feat: convert cost
* chore: Revert "feat: logs"
This reverts commit 01e0df2c3a
2023-12-11 06:43:17 +00:00
tison
178018143d
ci: prevent running nightly CI in forks ( #2898 )
...
Signed-off-by: tison <wander4096@gmail.com >
2023-12-11 02:34:59 +00:00
tison
73227bbafd
chore: ignore MySQL client sent SELECT $$ ( #2896 )
...
Signed-off-by: tison <wander4096@gmail.com >
2023-12-11 02:27:22 +00:00
Weny Xu
5a99f098c5
test: add tests for region migration procedure ( #2857 )
...
* feat: add backward compatibility test for persistent ctx
* refactor: refactor State of region migration
* feat: add test utils for region migration tests
* test: add simple region migration tests
* chore: apply suggestions from CR
2023-12-08 08:47:09 +00:00
Ruihang Xia
7cf9945161
fix: re-enable ignored case test_query_prepared ( #2892 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-08 08:35:56 +00:00
tison
bfb4794cfa
fix: handle heartbeat shutdown gracefully ( #2886 )
...
* fix: handle heartbeat shutdown gracefully
Signed-off-by: tison <wander4096@gmail.com >
* improve logging
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2023-12-08 03:59:05 +00:00
