Lei, HUANG
d1b5ce0d35
chore: check catalog deregister result ( #1810 )
...
* chore: check deregister result and return error on failure
* refactor: SystemCatalog::deregister_table returns Result<()>
2023-06-21 08:09:11 +00:00
Ruihang Xia
8c78368374
refactor: replace #[snafu(backtrace)] with Location ( #1753 )
...
* remove snafu backtrace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-06-12 11:55:33 +08:00
Yingwen
70e17ead68
fix: Print source error in subprocedure failure message ( #1683 )
...
* fix: print source error in subprocedure failed error
* feat: print source error in subprocedure failure message
2023-06-01 09:51:31 +08:00
Yingwen
32ad358323
fix(table-procedure): Open table in RegisterCatalog state ( #1617 )
...
* fix(table-procedure): on_register_catalog should use open_table
* test: Test recover RegisterCatalog state
* test: Fix subprocedure does not execute in test
* feat(mito): adjust procedure log level
* refactor: rename execute_parent_procedure
execute_parent_procedure -> execute_until_suspended_or_done
2023-05-22 17:54:02 +08:00
Niwaka
856ab5bea7
feat: make RepeatedTask invoke remove_outdated_meta method ( #1578 )
...
* feat: make RepeatedTask invoke remove_outdated_meta method
* fix: typo
* chore: improve error message
2023-05-16 10:21:35 +08:00
Yingwen
8d54d40b21
feat: Add FlushPicker to flush regions periodically ( #1559 )
...
* feat: Add FlushPicker
* feat(storage): Add close to StorageEngine
* style(storage): fix clippy
* feat(storage): Close regions in StorageEngine::close
* chore(storage): Clear requests on scheduler stop
* test(storage): Test flush picker
* feat(storage): Add metrics for auto flush
* feat(storage): Add flush reason and record it in metrics
* feat: Expose flush config
docs(config): Update config example
* refactor(storage): Run auto flush task in FlushScheduler
* refactor(storage): Add FlushItem trait to make FlushPicker easy to test
2023-05-15 17:29:28 +08:00
Ning Sun
a1587595d9
feat: add information_schema as exception of cross schema check ( #1551 )
...
* feat: add information_schema as a cross-schema query exception
* fix: resolve lint issue
2023-05-10 10:55:00 +08:00
Ruihang Xia
d4f3f617e4
chore(toolchain): update rust-toolchain to 2023-05-03 ( #1524 )
...
* chore(toolchain): update rust-toolchain to 2023-05-03
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update workflow yaml
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-05-06 11:34:09 +08:00
Lei, HUANG
fb9978e95d
refactor: catalog ( #1454 )
...
* wip
* add schema_async
* remove CatalogList
* remove catalog provider and schema provider
* fix
* fix: rename table
* fix: sqlness
* fix: ignore tonic error metadata
* fix: table engine name
* feat: rename catalog_async to catalog
* respect engine name in table regional value when deregistering tables
* fix: CR
2023-04-26 08:36:40 +00:00
Yingwen
eb50cee601
feat: Switch to the procedure framework ( #1448 )
...
* feat: Remove create_mock_sql_handler()
create_to_request() and alter_to_request() don't need `&self`, so
we don't need to mock the sql handler to test them
* feat: Enable procedure manager by default
* docs: Update config example
* test: Enable procedure framework in all tests
* refactor(datanode): rename methods using procedure
* test(catalog): Fix temp dir drops before test finishes
* tests: Enable procedure framework in sqlness
* test: Fix sqlness standalone rename test
* fix: Drop procedure allows table not in engine
* test: Change rename table test
* fix: add options to table meta when creating table by procedure
* test: adjust error message in schema test case
* test: Fix test_sql_api error message
2023-04-25 12:04:02 +08:00
Yingwen
1ee9ad4ca1
feat: manage multiple engine procedure in the engine manager ( #1434 )
...
* feat(table): Add engine procedure to engine manager
* feat(datanode): Get engine procedure from engine manager
* feat(table-procedure): Add source error to SubprocedureFailed
* test: Enable procedure in tests and pass all tests
* style(table-procedure): Fix clippy
2023-04-23 10:04:09 +08:00
Yingwen
830367b8f4
feat: Drop table by procedure ( #1401 )
...
* feat: Add drop table procedure
* feat: support dropping table by procedure on datanode
* test: Add test for DropTableProcedure
* test: Test drop table by procedure
* chore: update comments
* fix: Make on_remove_from_catalog idempotent
2023-04-20 15:57:56 +08:00
Yingwen
0819582a26
feat: Add alter table procedure ( #1354 )
...
* feat: Implement AlterTableProcedure
* test: Test alter table procedure
* feat: support alter table by procedure in datanode
* chore: update comment
2023-04-13 14:05:53 +08:00
Ruihang Xia
da66138e80
refactor(error): remove backtrace, and introduce call-site location for debugging ( #1329 )
...
* wip: global replace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unneeded tests of errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix ErrorExt trait implementator
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyo3 tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-06 04:06:00 +00:00
Weny Xu
451f9d2d4e
feat: support multi table engines ( #1277 )
...
* feat: support multi table engines
* refactor: adapt SqlHandler to support multiple table engines
* refactor: refactor TableEngineManager
* chore: apply review suggestions
* chore: apply review suggestions
* chore: apply review suggestions
* chore: snafu context styling
2023-04-03 14:49:12 +00:00
LFC
eb77f9aafd
feat: start LocalManager in Metasrv ( #1279 )
...
* feat: procedure store in Metasrv, backed by Etcd; start `LocalManager` in Metasrv leader
* fix: resolve PR comments
* fix: resolve PR comments
2023-03-31 15:32:59 +08:00
Lei, HUANG
5edd2a3dbe
feat: upgrade opendal ( #1245 )
...
* chore: upgrade opendal
* chore: finish upgrading opendal
* fix: clippy complaints
* fix some tests
* fix: all unit tests
* chore: rebase develop
* fix: sqlness tests
* optimize imports
* chore: rebase develop
* doc: add todo
2023-03-28 09:47:33 +08:00
Xieqijun
cd8be77968
feat(procedure): Max retry time ( #1095 )
...
* feat: procedure config
* fix: modify config
* feat: add retry logic
* feat: add error
* feat: add it
* feat: add it
* feat: add it
* feat: rm retry from runner
* feat: use backon
* feat: add retry_interval
* feat: add retry_interval
* fix: conflict
* fix: cr
* feat: add retry error and id
* feat: rename
* refactor: execute
* feat: use config dir
* fix: cr
* fix: cr
* fix: fmt
* fix: fmt
* fix: pr
* fix: it
* fix: rm unless cmd params
* feat: add toml
* fix: ut
* feat: add rolling back
* fix: cr
* fix: cr
* fix: cr
* fix: ci
* fix: ci
* fix: ci
* chore: Apply suggestions from code review
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-03-15 08:28:08 +00:00
Eugene Tolbakov
b31a6cb506
refactor: replace tempdir with tempfile ( #1123 )
...
* refactor: replace tempdir with tempfile
* refactor(query): move tempfile dependency under the workspace's Cargo.toml
* refactor(tempfile): create common-test-util
* refactor(tempfile): fix toml format
* refactor(tempfile): remove tempfile out of dependencies
* refactor(tempfile): fix incorrect toml
2023-03-08 11:15:56 +08:00
Yingwen
3fd9c2f144
feat: Store error in procedure state ( #1062 )
...
* docs: Change comment position
* refactor(procedure): Store error in ProcedureState
* test: Mock instance with procedure enabled
* feat: Add wait method to wait for procedure
* test(datanode): Test create table by procedure
* chore: Fix clippy
2023-03-01 14:37:50 +08:00
LFC
11d45e2918
refactor: upgrade DataFusion, Arrow and Sqlparser ( #1074 )
...
* refactor: upgrade DataFusion, Arrow and Sqlparser
* fix: resolve PR comments
2023-02-27 22:20:08 +08:00
Xieqijun
0b3f955ca7
feat: Add an error variant RetryLater ( #1058 )
...
* feat: support retry error
* fix: ci
* fix: ci
* fix: fmt
* feat: add convert procedure error
* Docs : add rustdoc
* fix: cr
* fix: cr
* fix: rm unless code
2023-02-27 17:19:37 +08:00
Yingwen
bd377ef329
feat: Procedure to create table and register table to catalog ( #1040 )
...
* feat: Add table-procedures crate
* feat: Implement procedure to create table
* feat: Integrate procedure manager to datanode
* test: Test CreateTableProcedure
* refactor: Rename table-procedures to table-procedure
* feat: Implement create_table_by_procedure
* chore: Remove comment
* chore: Add todo
* feat: Add procedure config to standalone mode
* feat: Register table-procedure loaders
* feat: Address review comments
CreateTableProcedure just return error if the subprocedure is failed
* chore: Address CR comments
2023-02-27 11:49:23 +08:00
