* feat: use fixed error message for unknown error
* feat: return fixed message for internal error as well
* chore: include status code in error message
* test: update tests for asserts of error message
* feat: change status code of some datafusion error
* fix: make CollectRecordbatch an query error
* test: update sqlness results
* chore: call handle_flush_request
* feat: alias SchedulerRef and clean scheduler on drop
* feat: add scheduler to workers
* feat: remove RegionMemtableStats
* feat: pick regions to flush
* feat: add more fields to region flush task
* feat: smallvec workspace dep
* feat: Use list to hold immutable memtables
* feat: flush job wip
* feat: use access layer to read write sst
* feat: flush memtables to l0
* feat: write manifest
* feat: schedule next flush on success
* feat: schedule flush on success and failure
* feat: add purger to region
* feat: apply edit after flush
* feat: collect stats for SSTs
* feat: manual flush
* test: test flush and fix manifest test
* feat: remove flush scheduler job limit
* fix: typo
* style: clippy
* feat: clean flushed files on failure
* chore: address CR comment
* refactor: Use put_rows
* feat: Clean flush scheduler on drop
* feat: remove region flush status on drop and close
* chore: address CR comment
* chore: update proto
* chore: add try from for schema name value
* chore: merge schema opts to table opts while creating table
* chore: use table ttl opts first
* chore: add unit test
* chore: update proto version
* feat: meta procedure options
* chore: tune meta procedure options in tests
* Update src/common/procedure/Cargo.toml
Co-authored-by: dennis zhuang <killme2008@gmail.com>
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com>
* fix(table-procedure): on_register_catalog should use open_table
* test: Test recover RegisterCatalog state
* test: Fix subprocedure does not execute in test
* feat(mito): adjust procedure log level
* refactor: rename execute_parent_procedure
execute_parent_procedure -> execute_until_suspended_or_done
* feat(common-procedure): pub(crate) use proc_path
* feat(common-procedure): Implement delete_procedure
* feat(common-procedure): Clean procedure after it is finished
* chore(common-procedure): put path_string in front of try_stream
* test(common-procedure): Test cleaning up procedures
* feat(common-procedure): Clean procedure states in recover()
* feat(common-procedure): Use VecDeque for finished procedures
* docs: Change comment position
* refactor(procedure): Store error in ProcedureState
* test: Mock instance with procedure enabled
* feat: Add wait method to wait for procedure
* test(datanode): Test create table by procedure
* chore: Fix clippy
* refactor: Use watch channel to store ProcedureState
* feat: Add a watcher to wait for state change
* test: test watcher on procedure failure
* feat: Only clear message cache on success
* feat: submit returns Watcher
* feat: wip
* feat: Implement procedure to create mito table
* feat: Add create_table_procedure to TableEngine
* feat: Impl dump and lock for CreateMitoTable
* feat: Impl CreateMitoTable::execute and register it to manager
* feat(common-procedure): pub local mod
* feat: Add simple test for MitoCreateTable
* style: Fix clippy
* refactor: Move create_table_procedure to a new trait TableEngineProcedure
* feat: Add ContextProvider to Context
So procedures can query states of other procedures via the
ContextProvider and they don't need to hold a ProcedureManagerRef
* feat: Procedure supports acquring multiple lock keys
* test: Use multi-locks in test
* feat: Add keys_to_lock/unlock
* feat: Runner executes procedure
* feat: Add rollback key type to ParsedKey
* feat: Write rollback key when procedure is unable to execute
* feat: Use loaded step to re-submit subprocedure
* feat: Track subprocedures in ProcedureMeta
* feat: Clean message cache after the root procedure is done
* feat: Runner returns execution result
* fix: Fix tests
* test: Test Runner
* test: Test procedures_in_tree
* chore: Refine test and comments
* feat: Remove support of lock inheritance
A deadlock happens if a subprocedure acquires the same lock key as
its parent.
The main concern is if the subprocedure directly inherits its parent's
lock, then how should we behave when multiple subprocedures acquire
this same lock? Each procedure may assume it has unique access to the
same object but it actually shares the resource with others.
Now subprocedures need to use different keys to lock objects, which is
reasonable. For example:
- A parent procedure wants to create a table so it locks the table with
a key like `catalog.schema.table`
- Subprocedures create regions for the table so they lock the regions
with keys `catalog.schema.table.region-0 ~ catalog.schema.table.region-n`
* style: Fix clippy
* feat: insert_procedure returns false on duplicate procedure
Also rename this method to try_insert_procedure
* chore: Address CR comments
