rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-05-19 09:40:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
382 Commits 243 Branches 81 Tags
2895a1feb121d8b98e48dca48339eb5772dec983
Commit Graph

357 Commits

Author SHA1 Message Date
Paul Masurel
2895a1feb1 Merge remote-tracking branch 'origin/master' into bug8/experimental 2016-09-15 00:01:53 +09:00
Paul Masurel
7969fb3a71 Use logging. 2016-09-15 00:00:14 +09:00
Paul Masurel
346fc31ac2 Chaining heaps. 
We commit close segments when the indexer heap is close to its capacity.
(currently we use a limit of 10_000_000).

Because we do this check before indexing a document, and before
also because serialization starts by closing the postingswriter, and
therefore all of the recorders open for the last document, we may still
overflow the heap.

We don't want to resize the heap because we may have references to objects
in the current heap.

Because of that, heap are actually chained list.
In an ideal settings, the limit should work fine and this overflow behavior should
never be activated.
 2016-09-14 10:27:55 +09:00
Paul Masurel
b911c4dc98 Indexing works. 3'22 2016-09-13 00:36:42 +09:00
Paul Masurel
50687a1c7c Renaming + new unit test 2016-09-08 09:26:14 +09:00
Paul Masurel
385794419e Bigger heap in unit testwq 2016-09-07 10:04:02 +09:00
Paul Masurel
a612504e26 #8 Hashmap size as a function of the heap size 2016-09-06 22:13:55 +09:00
Paul Masurel
33aa151916 BUG-21 Remove useless 64 bits padding in fastfield. 
Closes #21
 2016-09-06 09:57:17 +09:00
Paul Masurel
24d2e3f6c1 switching for the stacker datastructure 2016-09-05 10:27:14 +09:00
Paul Masurel
45bb1c5ab8 Working... Test passing. Indexing wikipediua in 4min37 2016-08-30 00:52:52 +09:00
Paul Masurel
0ee2d4c612 Using the entry API to get the block list. 2016-08-29 10:18:20 +09:00
Paul Masurel
93c984c8f2 Working but dirty... 2016-08-29 09:40:35 +09:00
Paul Masurel
7f52da2c87 Unit tests passing. 2016-08-29 08:55:40 +09:00
Paul Masurel
b10227ea44 (test not passing) 2016-08-29 00:51:59 +09:00
Paul Masurel
619b65b0f3 ISSUE-8 Replacing Vec by an Unrolled Linked List. 2016-08-28 15:38:56 +09:00
Paul Masurel
a599614a94 Code clean up. 2016-08-27 17:00:14 +09:00
Paul Masurel
5e806c88ef Issue 20 Searcher pool implemented using a channel. 
Operational but not really ready for merge.
 2016-08-27 16:15:02 +09:00
Paul Masurel
46e2b73ea5 small refactoring 2016-08-26 10:32:49 +09:00
Paul Masurel
b2afe8592c Erased text in README 2016-08-26 10:17:09 +09:00
Paul Masurel
59150ad802 superficial refactoring 2016-08-26 09:30:09 +09:00
Paul Masurel
ce95bfd9a5 Closes #19. Move example to examples directory. 2016-08-24 10:58:08 +09:00
Paul Masurel
e6200e85af Schema now: 
- internally wrapped by an Arc, cloning aggressively is ok.
- a field in schema, redundant with metas
- read-only. Needs to be built via a SchemaBuilder.
 2016-08-23 22:23:47 +09:00
Paul Masurel
054405adf1 Removing the locks from the index to keep only those in Directory 2016-08-23 11:18:19 +09:00
Paul Masurel
182d2549b9 Added commit / rollback / docstamp 2016-08-21 14:38:46 +09:00
Paul Masurel
07997e67c9 Store throws an io::Error 2016-08-20 16:38:53 +09:00
Paul Masurel
2e1ad4bbce Moving code the thread code to an index_worker object 2016-08-20 16:07:32 +09:00
Paul Masurel
0639911cc3 Moved index code to another module. 2016-08-20 15:29:27 +09:00
Paul Masurel
13248a08fa Removed bug in store bench. 2016-08-20 12:04:11 +09:00
Paul Masurel
d244a198e2 Added unit tests and related bugfix. 
Closes #17
 2016-08-20 00:11:09 +09:00
Paul Masurel
f1d2f6b34a Moved directory errors to a submodule 2016-08-19 17:07:32 +09:00
Paul Masurel
7b5e457e1e Added delete. 2016-08-19 10:35:19 +09:00
Paul Masurel
08612899eb Change error used in directory. 2016-08-19 00:56:13 +09:00
Paul Masurel
04238d9e65 Documented the Directory behavior. 
Removed sync calls, and moved then on file creation
and flush instead.
 2016-08-18 23:41:58 +09:00
Paul Masurel
0972a1c6a0 Removing data copy in the RAMDirectory 
The fst crate recently added support for sliced `Arc<Vec<u8>>`.
This called for a rewrite of the RAMDirectory for tantivy's RAMDirectory.
Previously every single read was copying data.

In addition:
- RAMDirectory's Write object panic if someone does not flush
right before the destruction of the object.
- In the same spirit, the postings serializer panics if someone
opens a term without closing the previous one.

Closes #16
 2016-08-18 10:45:34 +09:00
Paul Masurel
043b1945af Closes #12 - parse json array as multiple values for the field 
... Also adds some unit tests.
 2016-08-16 19:02:23 +09:00
Paul Masurel
ed38007e9f NOBUG made building index from directory public 2016-08-16 11:55:10 +09:00
Paul Masurel
1f4fafd964 Added usage snippet for tantivy 2016-08-16 00:58:11 +09:00
Paul Masurel
d526928e38 Bumped version 2016-08-15 00:19:41 +09:00
Paul Masurel
cf50b97f8e FieldType::Text -> Str 2016-08-14 23:29:00 +09:00
Paul Masurel
4773c8581c Moved cli to another project 2016-08-14 23:08:17 +09:00
Paul Masurel
e847de1ebb Refactoring, and making server working with a dynamic schema 2016-08-14 14:28:36 +09:00
Paul Masurel
1807e0710c Bugfix in schema json decoder 2016-08-14 10:58:04 +09:00
Paul Masurel
29abea07d4 Refactoring 2016-08-14 10:26:56 +09:00
Paul Masurel
32aad98c15 Added test cases for parse_document. 2016-08-14 01:48:34 +09:00
Paul Masurel
9a65b0a8e9 Changed the serialization format. 2016-08-14 00:30:35 +09:00
Paul Masurel
4c42c314e3 Changed serialization of field_entry 2016-08-13 23:45:12 +09:00
Paul Masurel
f66a48af42 Bugfix in merger 2016-08-13 20:14:00 +09:00
Paul Masurel
a3ab3940c3 Bugfix positions. 2016-08-13 16:50:48 +09:00
Paul Masurel
654769bb60 merging positions. 2016-08-13 15:59:12 +09:00
Paul Masurel
2a8262e15e Added positions in postings. 2016-08-13 15:36:39 +09:00