rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-03 20:02:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,581 Commits 103 Branches 225 Tags
a84a8ad04fff72e6f05bfc5b98b8c35a764c0275
Commit Graph

1581 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yingwen
a84a8ad04f fix: alter table procedure panics while renaming table (#2397) 
* fix: procedure panic on renaming table

* test: fix test_insert_and_select invalid arguments

* test: fix test_standalone_insert_and_query using wrong semantic type

* test: fix test_distributed_insert_delete_and_query semantic type
 2023-09-14 11:50:00 +00:00
JeremyHi
7bb8a5999c feat!: add engine name to DatanodeTableValue (#2395) 
* feat: add engine name to DatanodeTableValue

* fix: by cr
 2023-09-14 09:50:35 +00:00
Yingwen
26992d58cd chore: decrease mutable write buffer limit (#2390) 
* chore: set mutable limit to half of the global write buffer size

* refactor: put handle_flush_finished after handle_flush_request

* refactor: rename tests.rs to basic_test.rs

* style: fmt code
 2023-09-14 08:24:14 +00:00
Ruihang Xia
47bf300869 fix: update sqlness result for order_by (#2389) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-14 07:28:40 +00:00
Yingwen
a7df5a7c9a fix(mito): incorrect field index in ProjectionMapper (#2388) 
* chore: update todo comments

* test: add test for projection

* fix: panics when projecting fields

* chore: remove todos
 2023-09-14 04:15:15 +00:00
Yingwen
d4ae8a6fed feat(mito): Add writable flag to region (#2349) 
* feat: add writable flag to region.

* refactor: rename MitoEngine to MitoEngine::scanner

* feat: add set_writable() to RegionEngine

* feat: check whether region is writable

* feat: make set_writable sync

* test: test set_writable

* docs: update comments

* feat: send result on compaction failure

* refactor: wrap output sender in new type

* feat: on failure

* refactor: use get_region_or/writable_region_or

* refactor: remove send_result

* feat: notify waiters on flush scheduler drop

* test: fix tests

* fix: only alter writable region
 2023-09-14 02:45:30 +00:00
Yingwen
da54a0c139 fix: alter table procedure forgets to update next column id (#2385) 
* feat: add more info to error messages

* feat: store next column id in procedure

* fix: update next column id for table info

* test: fix add col test

* chore: remove location from invalid request error

* test: update test

* test: fix test
 2023-09-14 02:06:57 +00:00
Ruihang Xia
cc7eb3d317 fix: querying temporary table (#2387) 
* fix information schema

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove log

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-13 14:30:56 +00:00
Weny Xu
93f3048f4f refactor: migrate OpenDal to 0.39 (#2383) 
* chore: bump opendal to 7d552

* refactor: migrate OpenDal to 0.39

* chore: apply suggestions from CR
 2023-09-13 12:43:53 +00:00
LFC
d08b05c963 fix: make test-integration able to compile (#2384) 
* fix: make test-integration able to compile

* chore: fmt toml

---------

Co-authored-by: WenyXu <wenymedia@gmail.com>
 2023-09-13 12:42:46 +00:00
JeremyHi
f76aa278fd feat: atomic metadata (#2366) 
* feat: atomic creating metadata

* chore: exist exists

* chore: license header

* chore: weny never say that

* feat: add put_conditionally to kv_backend
 2023-09-13 10:51:05 +00:00
JeremyHi
6f4779b474 feat: engine name in heartbeat (#2377) 2023-09-13 09:10:10 +00:00
Ruihang Xia
de723d9c1c fix: update sqlness result in distributed mode (#2381) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-13 09:07:55 +00:00
dennis zhuang
7448e975c2 chore: change error messages (#2379) 
* chore: change error messages

* chore: remove locaton in table not found error msg
 2023-09-13 08:21:03 +00:00
dennis zhuang
3f97a0d285 fix: gRPC max mesage size limitation (#2375) 
* fix: gRPC max mesage size limitation

* chore: don't set max_encoding_message_size
 2023-09-13 08:13:49 +00:00
Ruihang Xia
60bdf9685f feat: use the latest command line options for sqlness runner (#2371) 
feat: use the latest command line options

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-13 03:38:43 +00:00
Ruihang Xia
9c76d2cf54 feat: convert sql number to values with target type (#2370) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-13 11:14:42 +08:00
Weny Xu
1a7268186b chore: bump raft-engine to 22dfb4 (#2360) 2023-09-12 07:57:15 -05:00
Ruihang Xia
eeecce4623 refactor: remove table procedure (#2359) 
remove table procedure

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
1ad5f6e5d5 refactor: system tables in FrontendCatalogManager (#2358) 
* rename method names

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove system table, table engine, register/deregister

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add system catalog

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* run nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* some documents

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix clippy

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2023-09-12 07:57:15 -05:00
Yingwen
46eca5026e fix(mito): Stores and recovers flushed sequence (#2355) 
* test: add test for reopen

* feat: last entry id starts from flushed entry id

* fix: store flushed sequence and recover it from manifest

* test: check sequence in alter test

* test: more tests for alter
 2023-09-12 07:57:15 -05:00
Weny Xu
912341e4fa fix: fix start issues under standalone mode (#2352) 
* fix: fix standalone starts

* chore: bump raft-engine to 571462e

* refactor: remove MetadataService
 2023-09-12 07:57:15 -05:00
JeremyHi
80c5d52015 feat: stop region server (#2356) 
* feat: stop region server

* fix: close region first
 2023-09-12 07:57:15 -05:00
Zhenchi
4af126eb1b feat: consolidate Insert request related partitioning and distributed processing operations into Inserter (#2346) 
* refactor: RegionRequest as param of RegionRequestHandler.handle

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: partition insert & delete reqs for both standalone and distributed mode

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: nit change

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: wrong function nameg

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: do request in inserter & deleter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: remove RegionRequestHandler.handle

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: rename table_creator

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: nit change

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: nit change

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
LFC
fe954b78a2 refactor: system tables in new region server (#2344) 
refactor: inverse the dependency between system tables and catalog manager
 2023-09-12 07:57:15 -05:00
JeremyHi
3cab6de391 feat: filter out empty heartbeat req (#2345) 
* feat: filter out empty heartbeat request

* fix: big mistake
 2023-09-12 07:57:15 -05:00
Yingwen
606ee43f1d feat(mito): Implement skeleton for alteration (#2343) 
* feat: impl handle_alter wip

* refactor: move send_result to worker.rs

* feat: skeleton for handle_alter_request

* feat: write requests should wait for alteration

* feat: define alter request

* chore: no warnings

* fix: remove memtables after flush

* chore: update comments and impl add_write_request_to_pending

* feat: add schema version to RegionMetadata

* feat: impl alter_schema/can_alter_directly

* chore: use send_result

* test: pull next_batch again

* feat: convert pb AlterRequest to RegionAlterRequest

* feat: validate alter request

* feat: validate request and alter metadata

* feat: allow none location

* test: test alter

* fix: recover files and flushed entry id from manifest

* test: test alter

* chore: change comments and variables

* chore: fix compiler errors

* feat: add is_empty() to MemtableVersion

* test: fix metadata alter test

* fix: Compaction picker doesn't notify waiters if it returns None

* chore: address CR comments

* test: add tests for alter request

* refactor: use send_result
 2023-09-12 07:57:15 -05:00
Lei, HUANG
3331e3158c feat(mito2): compaction (#2317) 
* feat: compaction component

* feat: mito2 compaction

* Avoid building time range predicates when merge SST files since in TWCS we don't enforce strict time window.

* fix: some CR comments

* minor: change CompactionRequest::senders to an option

* chore: handle compaction finish error

* feat: integrate compaction into region worker

* chore: rebase upstream

* fix: Some CR comments

* chore: Apply suggestions from code review

* style: fix clippy

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-09-12 07:57:15 -05:00
Weny Xu
a4604afde5 refactor: rename NEXT_TABLE_ROUTE_PREFIX to TABLE_ROUTE_PREFIX (#2348) 
* refactor: rename NEXT_TABLE_ROUTE_PREFIX to TABLE_ROUTE_PREFIX

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
Weny Xu
f386329e29 refactor: introduce DdlTaskExecutor and refactor statement executor (#2341) 
* feat: add kv store option

* refactor: refactor statement executor

* refactor: refactor standalone table creator

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor: move ShowCreateTable and CreateDatabase to StatementExecutor

* fix: fix RegionDistribution

* feat: build standalone

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
Yingwen
3f6d557b8d feat: Implements a reader to make schema compatible (#2326) 
* docs: update comment

* feat: Add compat reader to SeqScan

* feat: add struct to compat pk and fields

* refactor: remove unused fields from ParquetReader

* feat: compat framework

* feat: Implement CompatPrimaryKey and CompatFields

* feat: implement compat reader

* feat: Test compat reader

* test: test compat reader

* feat: add more checks to concat

* style: fix clippy

* test: more tests for compat reader

* test: test reader with projection
 2023-09-12 07:57:15 -05:00
Ruihang Xia
6215f124f7 refactor: remove datanode instance (#2342) 
* pass nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename region_alive_keepers

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
LFC
1d83c942a9 refactor: script table creation (#2340) 
* refactor:
1. remove method `register_system_table` from CatalogManager
2. the creation of ScriptTable (as a system table) is removed from CatalogManager. Instead, the ScriptTable is created when Frontend instance is starting; and is created by calling Frontend instance's grpc handler.

* rebase
 2023-09-12 07:57:15 -05:00
Ruihang Xia
f287a5db9f feat: adapt region keep aliver for region server (#2333) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor, collapse one layer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove old heartbeat handler impls

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove old region alive keeper

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove remote catalog manager

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* global replace

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test countdown task

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Zhenchi
dac6b2e80a feat(frontend): migrate delete to region server (#2329) 
* feat(frontend): migrate delete to region server

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add more check and do trim columns

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: RegionRequestHandler.handle retrun AffectedRows

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
Yingwen
1e44e86d81 feat(mito): Stall write requests and add more flush tests (#2322) 
* feat: impl reject write

* feat: sanitize reject size

* feat: add should_stall to WriteBufferManager

* feat: stall requests

* test: mock WriteBufferManager

* feat: add new_with_manager for test and remove object_store from inner

* feat: add an event listener for tests

* feat: Use listener to test flush

* refactor: add flush_test.rs

* style: fix clippy

* feat: test write stall

* test: test flush empty
 2023-09-12 07:57:15 -05:00
JeremyHi
56691ff03b refactor: mailbox timeout (#2330) 
refactor: Optimize the timeout mechanism of the mailbox
 2023-09-12 07:57:15 -05:00
Weny Xu
e4de63625f refactor: refactor raft engine backend and state store (#2336) 
* refactor: remove redundant code

* refactor: refactor RaftEngineBackend Error to common_meta::error::Error

* refactor: refactor state store

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
Ruihang Xia
4b2b59c31b refactor: clean unnecessary disabled lints (#2338) 
* clean manifest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean region

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean asscess_layer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean manifest manager

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean row_converter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean scheduler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean worker

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Weny Xu
2ee2d29085 refactor: move Sequence to common meta (#2337) 2023-09-12 07:57:15 -05:00
Yingwen
c3f6529178 fix: improve error message in validate_proto_value (#2328) 
* fix: correct error message in validate_proto_value()

* fix: print location in InvalidRequest error

* style: format
 2023-09-12 07:57:15 -05:00
Ruihang Xia
eb7116ab56 feat: read/write works in distributed mode 🎉 (#2327) 
* add do_get method to RegionRequestHandler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move RegionRequestHandler to client crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use RegionRequestHandler in MergeScan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* minor fix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Zhenchi
5f7d48f107 feat(frontend): reorg insert converters and introduce stmt_to_region (#2324) 
* feat(frontend): reorg insert converters and introduce stmt_to_region

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: shorten import path

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: add check for column count

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
LFC
711e27d9fa feat: distributed alter table in region server (#2311) 
* feat: distributed alter table in region server

* rebase
 2023-09-12 07:57:15 -05:00
Weny Xu
922e342b63 refactor: refactor ddl manager (#2306) 
* refactor: refactor ddl manager

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
Zhenchi
7dde9ce3ce feat(frontend): migrate insert to region server (#2318) 
* feat(frontend): migrate insert to region server

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: move converter to Inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: rename convert function

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: add span id

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: compilation

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* retrigger action

* retrigger action

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
Yingwen
3eccb36047 feat: avoid using vector to get default value (#2323) 2023-09-12 07:57:15 -05:00
Ruihang Xia
f71aa373c1 feat: start datanode with config (#2312) 
* remove memory-catalog and procedure

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* derive serde for MitoConfig

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start datanode with configs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove dir in WalConfig

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add rename field attr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add stupid duplicated mito config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove wrong import

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wired compile error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
50fca2400e feat: adapt methods from RegionEngine for MitoEngine (#2315) 
* feat: adapt methods from RegionEngine for MitoEngine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* minor fixes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
JeremyHi
920763d7dd feat: add metric and manage tool for InactiveRegionKey (#2313) 
* feat: add metric and manage tool for InactiveRegionKey

* chore: by review comment
 2023-09-12 07:57:15 -05:00