mirror of
https://github.com/quickwit-oss/tantivy.git
synced 2026-01-04 16:22:55 +00:00
21 lines
1.4 KiB
Plaintext
21 lines
1.4 KiB
Plaintext
Project Overview:
|
|
The solution requires two distinct components:
|
|
1. Query Parser: Converts input queries into an AST
|
|
2. Log Matcher: Evaluates log lines against the parsed AST
|
|
|
|
These components can be implemented independently and in different languages.
|
|
|
|
Status Update (2025-04-01):
|
|
|
|
Current Options:
|
|
|
|
|Option|Parser|Matcher|Status|
|
|
|---|---|---|---|
|
|
|1|Node QW Parser (Marcelo)|Custom matcher|ON HOLD - Parser handles 90% of syntax but misses a few other cases (question marks, AND/OR precedence). AST structure is unsuitable for evaluation.|
|
|
|2|Rust QW Parser (Node)|Custom matcher|ON HOLD - Working but has memory leaks (it can be fixed, in theory). Python version is more promising.|
|
|
|3.A|QW Python Parser|QW Python Matcher|Functional but requires full in-memory index/search cycle. Scalability concerns for 3-5TB/day.|
|
|
|3.B|QW Python Parser|Custom matcher|Waiting for other options outcome|
|
|
|3.C|QW Python Parser|Separate QW instance|Proposed by Remi from QW team. Potential overkill - high maintenance, latency, complexity. Awaiting final feedback from him.|
|
|
|
|
Note: Custom matcher implementations (Options 1, 2, 3.B) will likely support basic search syntax only (exact match, contains match, AND, OR, parenthesis). Customer and Research team use cases align with this scope. Future improvements may be possible but it depends on the ROI analysis compared to running a full interim instance.
|