Commit Graph

9 Commits

Author SHA1 Message Date
Ruihang Xia
8bea853954 refactor(mito2): implement RegionManifestManager (#1984)
* finilise manager and related API

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl manifest initialize and update

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* more test and utils

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-18 14:03:35 +00:00
Yingwen
ef7c5dd311 feat(mito): Implement WorkerGroup to handle requests (#1950)
* feat: engine worker framework

* feat: worder comments

* feat: divide worker requests by type

* feat: handlers for worker thread

* refactor: rename requests to ddl and dml requests

* feat: methods to stop and submit requests

* refactor: rename request queue to request buffer

* refactor: remove ddl and dml request

* feat: send request to worker

* test: test stop

* docs(mito): worker group docs

* style: fix clippy

* docs: update WorkerGroup comment

* chore: address CR comments

* chore: fix comment issues

* feat: use mpsc::channel

* feat: check is_running flag

* chore: Add stop request to notify a worker

* refactor: add join_dir to join paths

* feat: redefine region requests

* docs: more comments

* refactor: rename worker thread to worker loop

* chore: address CR comments
2023-07-14 08:06:44 +00:00
Ruihang Xia
ce43896a0b refactor(mito2): implement serialize/deserialize for RegionMetadata (#1964)
* feat: implement serialize/deserialize for RegionMetadata

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove Raw*

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* render mermaid

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* derive Serialize

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename symbols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-14 14:44:12 +08:00
Ruihang Xia
5bfd0d9857 refactor(mito): define region metadata (#1960)
* refactor(mito): define region metadata

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unneeded message

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/mito2/src/metadata.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* add primary keys vector

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update mermaid

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-07-13 13:40:41 +00:00
Ben Baoyi
9ff7670adf refactor:remove common_error::prelude (#1946)
* feat:Remove common_error::prelude

* fix merge error

* cr comment

* fix error
2023-07-13 10:36:36 +08:00
Ruihang Xia
16be56a743 refactor(mito): port manifest storage to mito2 (#1948)
* refactor(mito): port manifest storage to mito2

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-12 11:21:11 +00:00
Ruihang Xia
077785cf1e refactor(mito): define manifest related API (#1942)
* refactor: port some manifest struct to mito2

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy and nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert lock file and resolve clippy warnings

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-12 03:42:55 +00:00
Yingwen
65f5349767 feat(mito2): Define basic structs for MitoEngine (#1928)
* chore: metadata wip

* docs(mito2): Add struct relationships

* feat(mito2): define basic structs

* feat: add version and refactor other metadata

* docs: remove generics param from MitoEngine

* chore: Update src/mito2/Cargo.toml

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* chore: Apply suggestions from code review

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-10 12:25:33 +00:00
Yingwen
884731a2c8 chore: initialize mito2 crate (#1875) 2023-07-04 17:55:00 +08:00