Zhenchi
|
04ac0c8da0
|
feat(fulltext_index): integrate full-text indexer with parquet reader (#4348)
* feat(fulltext_index): integrate full-text indexer with parquet reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* disable reload
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: range allow exceeding total row
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* test: unit tests in index
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* test: prune row groups
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: rename creator
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* test: sst fulltext index
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: address comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-15 08:14:44 +00:00 |
|
Zhenchi
|
da1ea253ba
|
perf(puffin): not to stage uncompressed blob (#4333)
* feat(puffin): not to stage blob
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat: back with compressed blob
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-11 12:32:07 +00:00 |
|
Zhenchi
|
6e2c21dd3f
|
refactor(puffin): adjust generic parameters (#4279)
* refactor(puffin): adjust generic parameters
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: remove Box impl
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-04 12:22:04 +00:00 |
|
Zhenchi
|
65c9fbbd2f
|
feat(fulltext_index): integrate puffin manager with inverted index applier (#4266)
* feat(fulltext_index): integrate puffin manager with inverted index applier
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: get rid of unexpected not found from write cache
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: move create_dir_all to BoundedStager::new
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: update config.md
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* config: unify directories
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: silent remove
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: config docs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: auxiliary -> aux
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-04 06:18:58 +00:00 |
|
Lei, HUANG
|
705b22411b
|
fix(puffin): fix dependency (#4267)
fix/puffin-dependency: Update async-compression to 0.4 with features and add features to moka in Cargo.toml
|
2024-07-03 12:13:26 +00:00 |
|
Zhenchi
|
e64379d4f7
|
feat(fulltext_index): introduce creator (#4249)
* feat(fulltext_index): introduce creator
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: typo
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: typo
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: return error if writer not found
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* refactor: helper function for tests
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-02 09:06:14 +00:00 |
|
Zhenchi
|
db5d1162f0
|
feat(puffin): complete dir support (#4240)
* feat(puffin): implement CachedPuffinReader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: next PR to introduce CachedPuffinManager
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: rename
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat(puffin): implement MokaCacheManager
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: clippy
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: +1s
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat(puffin): implement CachedPuffinManager and add tests
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: corner case to get a blob
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: keep dir in used
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: add more tests
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: add doc comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: toml format
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: rename unreleased_dirs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: refine some comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: handle more cornor cases
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: refine
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* refactor: simplify
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: more explanation
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: comment compressed
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: fmt
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: address comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* refactor: Cached* -> Fs*
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* refactor: CacheManager -> Stager
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* refactor: rename Puffin(A)sync* -> (A)sync*
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: fmt
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-02 05:43:06 +00:00 |
|
Zhenchi
|
b69b24a237
|
feat(puffin): implement MokaCacheManager (#4211)
* feat(puffin): implement MokaCacheManager
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: clippy
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: +1s
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: corner case to get a blob
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: keep dir in used
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: add more tests
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: add doc comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: toml format
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: rename unreleased_dirs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: refine some comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: handle more cornor cases
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: refine
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* refactor: simplify
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: more explanation
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: use recycle bin
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: remove instead
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: address comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: remove unnecessary removing
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-07-01 13:10:13 +00:00 |
|
Zhenchi
|
5dde148b3d
|
feat(puffin): implement CachedPuffinReader (#4209)
* feat(puffin): implement CachedPuffinReader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: next PR to introduce CachedPuffinManager
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: rename
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-06-25 12:27:06 +00:00 |
|
Zhenchi
|
1e815dddf1
|
feat(puffin): implement CachedPuffinWriter (#4203)
* feat(puffin): support lz4 compression for footer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat(puffin): introduce puffin manager trait
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat(puffin): implement CachedPuffinWriter
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-06-25 08:00:48 +00:00 |
|
Zhenchi
|
4b42c7b840
|
feat(puffin): introduce puffin manager trait (#4195)
* feat(puffin): support lz4 compression for footer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat(puffin): introduce puffin manager trait
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-06-24 16:02:52 +00:00 |
|
Zhenchi
|
07cbabab7b
|
feat(puffin): support lz4 compression for footer (#4194)
* feat(puffin): support lz4 compression for footer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-06-24 11:39:03 +00:00 |
|
Ruihang Xia
|
115c74791d
|
build(deps): bump snafu to 0.8 (#3911)
* change Cargo.toml
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* global replace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* handle alias in script engine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
|
2024-05-10 13:36:25 +00:00 |
|
SteveLauC
|
e9a2b0a9ee
|
chore: use workspace-wide lints (#3352)
* chore: use workspace-wide lints
* respond to review
|
2024-02-22 01:01:10 +00:00 |
|
tison
|
4e04a4e48f
|
build: support build without git (#3309)
* build: support build without git
Signed-off-by: tison <wander4096@gmail.com>
* chore
Signed-off-by: tison <wander4096@gmail.com>
* address comment
Signed-off-by: tison <wander4096@gmail.com>
* fix syntax
Signed-off-by: tison <wander4096@gmail.com>
---------
Signed-off-by: tison <wander4096@gmail.com>
|
2024-02-18 10:30:01 +00:00 |
|
Zhenchi
|
e1ad7af10c
|
feat(puffin): finish return written bytes (#3082)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2024-01-03 06:55:09 +00:00 |
|
Zhenchi
|
b3edbef1f3
|
feat(inverted_index): add index reader (#2803)
* feat(inverted_index): add reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: toml format
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: add prefix relative_ to the offset parameter
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* docs: add doc comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: update proto
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: outdated docs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2023-11-27 03:31:44 +00:00 |
|
Zhenchi
|
9e5cdf47d9
|
chore(puffin): re-add tests file (#2790)
* chore(puffin): re-add tests file
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: fmt
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2023-11-21 11:02:18 +00:00 |
|
Zhenchi
|
bdb677dc52
|
chore(puffin): remove tests (#2789)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2023-11-21 10:02:34 +00:00 |
|
Zhenchi
|
a7bbd61f28
|
feat(puffin): add file writer (#2776)
* feat(puffin): add file writer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* Update src/puffin/src/file_format/writer/file.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com>
* Update src/puffin/src/file_format/writer/file.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com>
* feat: footer bytes with capacity
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat: footer bytes with capacity
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* Update src/puffin/src/file_format/writer.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
* feat: add flush
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: specify default flags
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* feat: close async writer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
|
2023-11-21 06:35:48 +00:00 |
|
Zhenchi
|
d9eeeee06e
|
feat(puffin): add file reader (#2751)
* feat(puffin): add file reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: toml format
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: rename PuffinParser to PuffinFileReader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* chore: polish comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* Update src/puffin/src/file_format/reader/footer.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
* Update src/puffin/src/file_format/reader/file.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
* Update src/puffin/src/file_format/reader/footer.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
* Update src/puffin/src/file_format/reader/footer.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
* fix: check file size
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: redundant type cast
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: reuse read buffer
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: check payload size
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: check payload size
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: validate blob offset
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: validate blob offset
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
|
2023-11-20 04:29:41 +00:00 |
|
Zhenchi
|
3329da5b72
|
feat(puffin): add partial reader (#2741)
* feat(puffin): add partial reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* address comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2023-11-15 06:28:20 +00:00 |
|
Zhenchi
|
1757061272
|
feat: introduce metadata struct of puffin (#2725)
* feat: introduce metadata struct of puffin
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: fmt
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
* fix: address suggestions
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
|
2023-11-13 10:02:40 +00:00 |
|