PSeitz
8024ecf013
Merge pull request #1389 from quickwit-oss/doc_writer_thread
...
use separate thread to compress block store
2022-06-23 16:17:41 +08:00
PSeitz
9baefbe2ab
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
PSeitz
ad76d11008
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
PSeitz
c3220bece0
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
PSeitz
2b713f0977
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
Pascal Seitz
0bc6b4a117
renames and refactoring
2022-06-23 15:34:21 +08:00
PSeitz
79e42d4a6d
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
PSeitz
0135fbc4c8
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
PSeitz
449594f67a
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
Pascal Seitz
8b6647e908
move writer to compressor thread
2022-06-23 15:34:21 +08:00
PSeitz
efabcbcdf5
Update src/store/writer.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-06-23 15:34:21 +08:00
Pascal Seitz
7bf5962554
merge match, explicit type
2022-06-23 15:34:21 +08:00
Pascal Seitz
4c7dedef29
use seperate thread to compress block store
...
Use seperate thread to compress block store for increased indexing performance. This allows to use slower compressors with higher compression ratio, with less or no perfomance impact (with enough cores).
A seperate thread is spawned to compress the docstore, which handles single blocks and stacking from other docstores.
The spawned compressor thread does not write, instead it sends back the compressed data. This is done in order to avoid writing multithreaded on the same file.
2022-06-23 15:34:21 +08:00
PSeitz
93f356a7a7
Extend FAQ ( #1388 )
...
* Extend FAQ
Co-authored-by: Maxim Kraynyuchenko <100854040+maximkpa@users.noreply.github.com >
2022-06-23 11:53:20 +09:00
PSeitz
6ca5f77466
Merge pull request #1363 from quickwit-oss/refactor_aggregation
...
Add aggregation bucket limit
2022-06-23 10:27:57 +08:00
Paul Masurel
2e2822f89d
Apply suggestions from code review
2022-06-23 09:48:28 +09:00
PSeitz
de178a1901
Merge pull request #1395 from PSeitz/fix_clippy
...
fix clippy
2022-06-21 16:30:59 +08:00
Antoine G
11e4225f23
doc fix ( #1391 )
...
Documentation fix.
2022-06-21 15:53:33 +09:00
Paul Masurel
f21b73d1f6
Apply suggestions from code review
2022-06-21 15:52:43 +09:00
Pascal Seitz
1440f3243b
fix clippy
2022-06-21 14:47:01 +08:00
Kanji Yomoda
83d0c13fb0
Fix outdated variable naming and comments to alive bitset ( #1387 )
...
* Fix outdated variables and comments for alive bitset
* Fix expired link to delete bitset
2022-06-14 15:59:15 +09:00
PSeitz
88054aa333
Merge pull request #1382 from boraarslan/bool-fields
...
Add boolean fields
2022-06-13 13:20:05 +08:00
boraarslan
635c39ba48
cargo fmt
2022-06-10 19:54:44 +03:00
boraarslan
eab2257637
Change var name
2022-06-10 19:36:25 +03:00
PSeitz
328bd96c24
Merge pull request #1378 from quickwit-oss/test_compression
...
enable setting compression level
2022-06-10 11:10:07 +08:00
dependabot[bot]
fc24842a43
Update more-asserts requirement from 0.2.1 to 0.3.0 ( #1384 )
...
Updates the requirements on [more-asserts](https://github.com/thomcc/rust-more-asserts ) to permit the latest version.
- [Release notes](https://github.com/thomcc/rust-more-asserts/releases )
- [Commits](https://github.com/thomcc/rust-more-asserts/compare/v0.2.2...v0.3.0 )
---
updated-dependencies:
- dependency-name: more-asserts
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-10 10:38:28 +09:00
boraarslan
2d6f1d43ff
Add bool and explicit types for merger
2022-06-07 10:10:33 +03:00
boraarslan
ca0973ec78
Fix tests
2022-06-07 10:10:33 +03:00
boraarslan
38ee60d792
Edit Test
2022-06-07 10:10:33 +03:00
boraarslan
f68be28284
Add bool
2022-06-07 10:09:37 +03:00
boraarslan
fc43ab9280
Add tests
2022-06-07 10:09:37 +03:00
boraarslan
38c2ea6a5d
Remove unnecessary line
2022-06-07 10:09:37 +03:00
boraarslan
26a0fd1fbe
cargo fmt
2022-06-07 10:09:37 +03:00
boraarslan
811b91ecb3
Edit and add tests
2022-06-07 10:09:37 +03:00
boraarslan
25c00ce856
Fix indexing for bool
2022-06-07 10:09:37 +03:00
boraarslan
e5debb97a7
Edit test
2022-06-07 10:09:37 +03:00
boraarslan
bc4cd9ffaa
typo fix
2022-06-07 10:09:37 +03:00
boraarslan
9a13d8709b
Explicitly write types
2022-06-07 10:09:37 +03:00
boraarslan
e6eadf1a2f
Add tests
2022-06-07 10:09:37 +03:00
boraarslan
7cca7e6a47
Fix of last commit
2022-06-07 10:09:37 +03:00
boraarslan
ef2492dba6
Broken commit
2022-06-07 10:09:37 +03:00
boraarslan
2981e6c1df
First commit
2022-06-07 10:09:37 +03:00
Ryan Russell
b33b4c0092
Fix various occurrence var names and references ( #1385 )
...
Thank you Ryan!
Signed-off-by: Ryan Russell <git@ryanrussell.org >
2022-06-07 11:08:19 +09:00
Pascal Seitz
4d9d2b6db0
split into compressor/decompressor
...
use custom de/serializer for compressor
accept parameters like zstd(compression_level=5) as compressor
2022-06-02 23:29:24 +08:00
Pascal Seitz
ed868f93a3
enable setting compression level
2022-06-02 16:47:29 +08:00
PSeitz
5e599d96d7
Merge pull request #1372 from quickwit-oss/doc_store_api
...
refactor doc store
2022-06-02 15:19:57 +08:00
Pascal Seitz
314ae43a45
fix fmt
2022-06-02 14:54:23 +08:00
Pascal Seitz
fce91b2f3a
vec without capacity
2022-06-02 13:50:18 +08:00
Pascal Seitz
9bcd2b8104
fix read_block_async
2022-06-02 13:37:52 +08:00
Pascal Seitz
0c9c257150
move cache handling into single function
2022-06-02 13:25:29 +08:00
