rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-05-21 18:50:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,942 Commits 242 Branches 81 Tags
fbda511a1a0e7ce494ab70ce91c16ad718bce049
Commit Graph

324 Commits

Author SHA1 Message Date
Paul Masurel
fbda511a1a Making more things public for quickwit. (#2005) 2023-04-20 11:37:45 +09:00
PSeitz
5c4ea6a708 tokenizer option on text fastfield (#1945) 
* tokenizer option on text fastfield

allow to set tokenizer option on text fastfield (fixes #1901)
handle PreTokenized strings in fast field

* change visibility

* remove custom de/serialization
 2023-03-31 10:03:38 +02:00
PSeitz
faa706d804 add coerce option for text and numbers types (#1904) 
* add coerce option for text and numbers types

allow to coerce the field type when indexing if the type does not match

* Apply suggestions from code review

Co-authored-by: Paul Masurel <paul@quickwit.io>

* add tests,add COERCE flag, include bool in coercion

---------

Co-authored-by: Paul Masurel <paul@quickwit.io>
 2023-03-01 11:36:59 +01:00
Paul Masurel
f537334e4f Adding a write schema to columnar's merge operations. (#1884) 
* Adding a write schema to columnar's merge operations.

* Added unit test checking min/max when columns are empty.

* CR comment

* Rename to value_type_to_column_type
 2023-02-21 18:25:16 +09:00
PSeitz
111f25a8f7 clippy (#1879) 
* fix clippy

* fix clippy

* fmt
 2023-02-17 11:34:21 +01:00
Paul Masurel
7423f99719 Issue/columnar for json (#1876) 
Adding support for JSON fast field.
 2023-02-16 20:38:32 +09:00
trinity-1686a
539ff08a79 move DateTime to tantivy_common (#1861) 
* move DateTime to tantivy_common

* resolve imports of columnar::DateTime as import of common::DateTime
 2023-02-11 17:03:06 +01:00
PSeitz
36c6138e7f fix: auto downgrade index record option, instead of vint error (#1857) 
Prev: thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: IoError(Custom { kind: InvalidData, error: "Reach end of buffer while reading VInt" })', src/main.rs:46:14
Now: Automatic downgrade to next available level
 2023-02-10 13:45:23 +01:00
trinity-1686a
1390834ae8 make Term::as_slice public (#1846) 2023-02-09 15:37:07 +01:00
trinity-1686a
3ac973bea4 fix invalid endianness in documentation (#1845) 
* fix doc about term endianness

* rustfmt
 2023-02-09 15:36:38 +01:00
Paul Masurel
bd5eea9852 Integrated columnar work. 2023-02-09 13:14:31 +01:00
Lonre Wang
8ba333f1b4 Typo fix (#1803) 
* Update text_options.rs

* Update src/schema/text_options.rs

Co-authored-by: Paul Masurel <paul@quickwit.io>
 2023-01-19 17:56:05 +09:00
PSeitz
f687b3a5aa start migrate Field to &str (#1772) 
start migrate Field to &str in preparation of columnar
return Result for get_field
 2023-01-18 16:12:07 +09:00
Paul Masurel
7a8fce0ae7 Minor mini fixes 2023-01-10 14:15:30 +09:00
Adam Reichold
2080c370c2 Enable usage of FuzzyTermQuery for specific fields via QueryParser (#1750) 
* Make nightly Clippy mostly happy.

* Document how to produce TermSetQuery queries using QueryParser.

* Enable construction of queries using FuzzyTermQuery via the QueryParser

* Use FxHashMap instead of HashMap in the QueryParser as these hash tables are not exposed to DoS attacks.

* Use a struct instead of a tuple to improve readability.
 2023-01-04 18:11:27 +09:00
PSeitz
f9171a3981 fix clippy (#1725) 
* fix clippy

* fix clippy fastfield codecs

* fix clippy bitpacker

* fix clippy common

* fix clippy stacker

* fix clippy sstable

* fmt
 2022-12-20 07:30:06 +01:00
boraarslan
495824361a Move split_full_path to Schema (#1692) 2022-11-29 20:56:13 +09:00
PSeitz
ee1f2c1f28 add aggregation support for date type (#1693) 
* add aggregation support for date type
fixes #1332

* serialize key_as_string as rfc3339 in date histogram
* update docs
* enable date for range aggregation
 2022-11-28 09:12:08 +09:00
Paul Masurel
0b40a7fe43 Added a expand_dots JsonObjectOptions. (#1687) 
Related with quickwit#2345.
 2022-11-21 23:03:00 +09:00
PSeitz
0c2bd36fe3 Panic on duplicate field names (#1647) 
fixes #1601
 2022-10-26 16:17:33 +09:00
Pascal Seitz
6bb73a527f add range query via ip fast field 2022-10-24 16:00:38 +08:00
Pascal Seitz
6800fdec9d add indexing for ip field 
Closes #1595
 2022-10-18 10:07:48 +08:00
Pascal Seitz
024e53a99c remove truncate 2022-10-17 12:14:35 +08:00
Pascal Seitz
8d75e451bd fix truncate, remove mutable access from term 2022-10-17 12:14:35 +08:00
Pascal Seitz
fcfd76ec55 refactor Term 
fixes some issues with Term
Remove duplicate calls to truncate or resize
Replace Magic Number 5 with constant
Enforce minimum size of 5 for metadata
Fix broken truncate docs
use constructor instead new + set calls
normalize constructor stack
replace assert on internal behavior fixes #1585
 2022-10-17 12:14:34 +08:00
Pascal Seitz
952b048341 add term aggregation clarification 2022-10-14 16:12:19 +08:00
PSeitz
8b69aab0fc avoid prepare_doc allocation (#1610) 
avoid prepare_doc allocation, ~10% more thoughput best case
 2022-10-11 14:15:55 +09:00
PSeitz
3650d1f36a Merge pull request #1553 from quickwit-oss/ip_field 
ip field
 2022-10-11 13:09:47 +08:00
François Massot
e443ca63aa Merge pull request #1608 from quickwit-oss/nigel/serialise-bytes-as-b64-#2042 
Serialise bytes as base64 strings instead of arrays.
 2022-10-10 11:51:23 +02:00
Pascal Seitz
5c9cbee29d handle IpV4 serialization case 2022-10-07 19:52:00 +08:00
Pascal Seitz
b2ca83a93c switch to ipv6, add monotonic_mapping tests 2022-10-07 18:47:55 +08:00
Nigel Andrews
3b189080d4 Use raw string literals in tests 2022-10-07 12:28:25 +02:00
Nigel Andrews
00a6586efe Replaced String::serialize for serializer.serialize_str 2022-10-07 11:55:05 +02:00
PSeitz
534b1d33c3 use ipv6 
Co-authored-by: Paul Masurel <paul@quickwit.io>
 2022-10-07 16:56:00 +08:00
Pascal Seitz
5171ff611b serialize ip as u128, add test for positions_to_docid 2022-10-07 16:25:01 +08:00
Pascal Seitz
e50e74acf8 remove u128 type 2022-10-07 16:25:01 +08:00
Pascal Seitz
0b86658389 rename ip addr, use buffer 2022-10-07 16:25:01 +08:00
Pascal Seitz
4d29ff4d01 finalize ip addr rename 2022-10-07 16:25:01 +08:00
Pascal Seitz
787a37bacf expect instead of unwrap 2022-10-07 16:25:01 +08:00
Pascal Seitz
eeb1f19093 rename to iter_gen 2022-10-07 16:25:01 +08:00
Pascal Seitz
309449dba3 rename to IpAddr 2022-10-07 16:25:01 +08:00
Pascal Seitz
c8713a01ed use iter api 2022-10-07 16:25:01 +08:00
Pascal Seitz
400a20b7af add ip field 
add u128 multivalue reader and writer
add ip to schema
add ip writers, handle merge
 2022-10-07 16:25:01 +08:00
Bruce Mitchener
b3bf9a5716 Documentation improvements. 2022-10-05 14:18:10 +07:00
Nigel Andrews
e5043d78d2 added a couple of tests + make fmt 2022-10-04 12:52:44 +02:00
Nigel Andrews
6d0bb82bd2 Fix issue 1576: serialize bytes as base64 strings 2022-10-04 12:18:13 +02:00
Pascal Seitz
0e94213af0 validate index settings on create 2022-09-29 18:58:09 +08:00
Bruce Mitchener
97ccd6d712 Avoid slicing a string in DocParsingError. (#1559) 
Fixes #1339.
 2022-09-26 20:27:15 +09:00
Bruce Mitchener
cb252a42af docs: "associated to" -> "associated with" (#1557) 
This reads better this way.
 2022-09-26 20:23:37 +09:00
Pascal Seitz
f757471077 prepare for ip field 2022-09-26 16:27:35 +08:00