Zhenchi
d51b65a8bf
feat(index-cache): abstract IndexCache to be shared by multi types of indexes ( #5219 )
...
* feat(index-cache): abstract `IndexCache` to be shared by multi types of indexes
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix typo
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: remove added label
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: simplify cached reader impl
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* rename func
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-24 05:10:30 +00:00
zyy17
2082c4b6e4
docs: add greptimedb-operator project link in 'Tools & Extensions' and other small improvements ( #5216 )
2024-12-24 03:09:41 +00:00
Ning Sun
c623404fff
ci: fix nightly ci task on nix build ( #5198 )
2024-12-21 10:09:32 +00:00
Yingwen
fa3b7ed5ea
build: use 8xlarge as arm default ( #5214 )
2024-12-21 08:39:24 +00:00
Yiran
8ece853076
fix: dead links ( #5212 )
2024-12-20 12:01:57 +00:00
Zhenchi
4245bff8f2
feat(bloom-filter): add bloom filter reader ( #5204 )
...
* feat(bloom-filter): add bloom filter reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: remove unused dep
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix conflict
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-20 08:29:18 +00:00
Zhenchi
3d4121aefb
feat(bloom-filter): add memory control for creator ( #5185 )
...
* feat(bloom-filter): add memory control for creator
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: remove meaningless buf
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add codec for intermediate
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-20 06:59:44 +00:00
Weny Xu
1910d71cb3
chore: adjust fuzz tests cfg ( #5207 )
2024-12-20 06:58:51 +00:00
LFC
a578eea801
ci: install latest protobuf in dev-builder image ( #5196 )
2024-12-20 02:45:53 +00:00
discord9
6bf574f098
fix: auto created table ttl check ( #5203 )
...
* fix: auto created table ttl check
* tests: with hint
2024-12-19 11:23:01 +00:00
discord9
a4d61bcaf1
fix(flow): batch builder with type ( #5195 )
...
* fix: typed builder
* chore: clippy
* chore: rename
* fix: unit tests
* refactor: per review
2024-12-19 09:16:56 +00:00
dennis zhuang
7ea8a44d3a
chore: update PR template ( #5199 )
2024-12-19 08:28:20 +00:00
discord9
2d6f63a504
feat: show flow's mem usage in INFORMATION_SCHEMA.FLOWS ( #4890 )
...
* feat: add flow mem size to sys table
* chore: rm dup def
* chore: remove unused variant
* chore: minor refactor
* refactor: per review
2024-12-19 08:24:04 +00:00
Ruihang Xia
422d18da8b
feat: bump opendal and switch prometheus layer to the upstream impl ( #5179 )
...
* feat: bump opendal and switch prometheus layer to the upstream impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused files
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused things
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove root dir on recovering cache
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* filter out non-files entry in test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-19 03:42:05 +00:00
Weny Xu
66f0581f5b
fix: ensure table route metadata is eventually rolled back on failure ( #5174 )
...
* fix: ensure table route metadata is eventually rolled back on procedure failure
* fix(fuzz): enhance procedure condition checking
* chore: add logs
* feat: close downgraded leader region actively
* chore: apply suggestions from CR
2024-12-19 03:29:34 +00:00
Ning Sun
c9ad8c7101
feat: show create postgresql foreign table ( #5143 )
...
* feat: add show create table for pg in parser
* feat: implement show create table operation
* fix: adopt upstream changes
2024-12-18 15:15:55 +00:00
Ning Sun
2107737db1
chore: make nix compilation environment config more robust ( #5183 )
...
* chore: improve nix-shell support
* fix: add pkg-config
* ci: add a github action to ensure build on clean system
* ci: optimise dependencies of task
* ci: move clean build to nightly
2024-12-18 12:41:24 +00:00
Ruihang Xia
548e1988ab
refactor: remove unused symbols ( #5193 )
...
chore: remove unused symbols
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-18 11:24:43 +00:00
Yingwen
218236cc5b
docs: fix grafana dashboard row ( #5192 )
2024-12-18 09:10:56 +00:00
Ruihang Xia
f04d380259
fix: validate matcher op for __name__ in promql ( #5191 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-18 08:51:46 +00:00
Lei, HUANG
fa773cf480
fix(sqlness): enforce order in union tests ( #5190 )
...
Add ORDER BY clause to subquery union tests
Updated the SQL and result files for subquery union tests to include an ORDER BY clause, ensuring consistent result ordering. This change aligns with the test case from the DuckDB repository.
2024-12-18 08:24:15 +00:00
jeremyhi
9b4e8555e2
feat: extract hints from http header ( #5128 )
...
* feat: extract hints from http header
* Update src/servers/src/http/hints.rs
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com >
* chore: by comment
* refactor: get instead of loop
---------
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com >
2024-12-18 08:17:34 +00:00
Yingwen
c6b7caa2ec
feat: do not remove time filters in ScanRegion ( #5180 )
...
* feat: do not remove time filters
* chore: remove `time_range` from parquet reader
* chore: print more message in the check script
* chore: fix unused error
2024-12-18 06:39:49 +00:00
Yingwen
58d6982c93
feat: do not keep MemtableRefs in ScanInput ( #5184 )
2024-12-18 06:37:22 +00:00
dennis zhuang
e662c241e6
feat: impl label_join and label_replace for promql ( #5153 )
...
* feat: impl label_join and label_replace for promql
* chore: style
* fix: dst_label is eqauls to src_label
* fix: forgot to sort the results
* fix: processing empty source label
2024-12-18 06:35:45 +00:00
Lanqing Yang
266919c226
fix: display inverted and fulltext index in show index ( #5169 )
2024-12-18 06:10:59 +00:00
Yohan Wal
7d1bcc9d49
feat: introduce Buffer for non-continuous bytes ( #5164 )
...
* feat: introduce Buffer for non-continuous bytes
* Update src/mito2/src/cache/index.rs
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* chore: apply review comments
* refactor: use opendal::Buffer
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-12-18 03:45:38 +00:00
LFC
18e8c45384
refactor: produce BatchBuilder from a Batch to modify it again ( #5186 )
...
chore: pub some mods
2024-12-18 02:42:33 +00:00
Lei, HUANG
c33cf59398
perf: avoid holding memtable during compaction ( #5157 )
...
* perf/avoid-holding-memtable-during-compaction: Refactor Compaction Version Handling
• Introduced CompactionVersion struct to encapsulate region version details for compaction, removing dependency on VersionRef.
• Updated CompactionRequest and CompactionRegion to use CompactionVersion.
• Modified open_compaction_region to construct CompactionVersion without memtables.
• Adjusted WindowedCompactionPicker to work with CompactionVersion.
• Enhanced flush logic in WriteBufferManager to improve memory usage checks and logging.
* reformat code
* chore: change log level
* reformat code
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-12-17 07:06:07 +00:00
discord9
421088a868
test: sqlness upgrade compatibility tests ( #5126 )
...
* feat: simple version switch
* chore: remove debug print
* chore: add common folder
* tests: add drop table
* feat: pull versioned binary
* chore: don't use native-tls
* chore: rm outdated docs
* chore: new line
* fix: save old bin dir
* fix: switch version restart all node
* feat: use etcd
* fix: wait for election
* fix: normal sqlness
* refactor: hashmap for bin dir
* test: past 3 major version compat crate table
* refactor: allow using without setup etcd
2024-12-17 07:00:02 +00:00
Zhenchi
d821dc5a3e
feat(bloom-filter): add basic bloom filter creator (Part 1) ( #5177 )
...
* feat(bloom-filter): add a simple bloom filter creator (Part 1)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: clippy
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: header
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* docs: add format comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-17 06:55:42 +00:00
Yingwen
bfc777e6ac
fix: deletion between two put may not work in last_non_null mode ( #5168 )
...
* fix: deletion between rows with the same key may not work
* test: add sqlness test case
* chore: comments
2024-12-17 04:01:32 +00:00
Yingwen
8a5384697b
chore: add aquamarine to dep lists ( #5181 )
2024-12-17 01:45:50 +00:00
Weny Xu
d0245473a9
fix: correct set_region_role_state_gracefully behaviors ( #5171 )
...
* fix: reduce default max rows for fuzz testing
* chore: remove Postgres setup from fuzz test workflow
* chore(fuzz): increase resource limits for GreptimeDB cluster
* chore(fuzz): increase resource limits for kafka
* fix: correct `set_region_role_state_gracefully` behaviors
* chore: remove Postgres setup from fuzz test workflow
* chore(fuzz): redue resource limits for GreptimeDB & kafka
2024-12-16 14:01:40 +00:00
discord9
043d0bd7c2
test: flow rebuild ( #5162 )
...
* tests: rebuild flow
* tests: more rebuild
* tests: restart
* chore: drop clean
2024-12-16 12:25:23 +00:00
Ning Sun
acedff030b
chore: add nix-shell configure for a minimal environment for development ( #5175 )
...
* chore: add nix-shell development environment
* chore: add rust-analyzer
* chore: use .envrc as a private file
2024-12-16 11:47:18 +00:00
ZonaHe
88f7075a2a
feat: update dashboard to v0.7.3 ( #5172 )
...
Co-authored-by: sunchanglong <sunchanglong@users.noreply.github.com >
2024-12-16 10:56:41 +00:00
Ruihang Xia
54698325b6
feat: introduce SKIPPING index (part 1) ( #5155 )
...
* skip index parser
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* wip: sqlness
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* impl show create part
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add empty line
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change keyword to SKIPPING INDEX
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename local variables
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-16 09:21:00 +00:00
Lei, HUANG
5ffda7e971
chore: gauge for flush compaction ( #5156 )
...
* add metrics
* chore/bench-metrics: Add INFLIGHT_FLUSH_COUNT Metric to Flush Process
• Introduced INFLIGHT_FLUSH_COUNT metric to track the number of ongoing flush operations.
• Incremented INFLIGHT_FLUSH_COUNT in FlushScheduler to monitor active flushes.
• Removed redundant increment of INFLIGHT_FLUSH_COUNT in RegionWorkerLoop to prevent double counting.
* chore/bench-metrics: Add Metrics for Compaction and Flush Operations
• Introduced INFLIGHT_COMPACTION_COUNT and INFLIGHT_FLUSH_COUNT metrics to track the number of ongoing compaction and flush operations.
• Incremented INFLIGHT_COMPACTION_COUNT when scheduling remote and local compaction jobs, and decremented it upon completion.
• Added INFLIGHT_FLUSH_COUNT increment and decrement logic around flush tasks to monitor active flush operations.
• Removed redundant metric updates in worker.rs and handle_compaction.rs to streamline metric handling.
* chore: add metrics for remote compaction jobs
* chore: format
* chore: also add dashbaord
2024-12-16 07:08:07 +00:00
Lin Yihai
f82af15eba
feat: Add vector_scalar_mul function. ( #5166 )
2024-12-16 06:46:38 +00:00
shuiyisong
9d7fea902e
chore: remove unused dep ( #5163 )
...
* chore: remove unused dep
* chore: remove more unused dep
2024-12-16 06:17:27 +00:00
Niwaka
358d5e1d63
fix: support alter table ~ add ~ custom_type ( #5165 )
2024-12-15 09:05:29 +00:00
Yingwen
579059d99f
ci: use 4xlarge for nightly build ( #5158 )
2024-12-13 12:53:11 +00:00
localhost
53d55c0b6b
fix: loki write row len error ( #5161 )
2024-12-13 10:10:59 +00:00
Yingwen
bef6896280
docs: Add index panels to standalone grafana dashboard ( #5140 )
...
* docs: Add index panels to standalnoe grafana dashboard
* docs: fix flush/compaction op
2024-12-13 08:17:49 +00:00
Yohan Wal
4b4c6dbb66
refactor: cache inverted index with fixed-size page ( #5114 )
...
* feat: cache inverted index by page instead of file
* fix: add unit test and fix bugs
* chore: typo
* chore: ci
* fix: math
* chore: apply review comments
* chore: renames
* test: add unit test for index key calculation
* refactor: use ReadableSize
* feat: add config for inverted index page size
* chore: update config file
* refactor: handle multiple range read and fix some related bugs
* fix: add config
* test: turn to a fs reader to match behaviors of object store
2024-12-13 07:34:24 +00:00
localhost
e8e9526738
chore: pipeline dryrun api can currently receives pipeline raw content ( #5142 )
...
* chore: pipeline dryrun api can currently receives pipeline raw content
* chore: remove dryrun v1 and add test
* chore: change dryrun pipeline api body schema
* chore: remove useless struct PipelineInfo
* chore: update PipelineDryrunParams doc
* chore: increase code readability
* chore: add some comment for pipeline dryrun test
* Apply suggestions from code review
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com >
* chore: format code
---------
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com >
2024-12-12 11:47:21 +00:00
Yingwen
fee75a1fad
feat: collect reader metrics from prune reader ( #5152 )
2024-12-12 11:27:22 +00:00
localhost
b8a78b7838
chore: decide tag column in log api follow table schema if table exists ( #5138 )
...
* chore: decide tag column in log api follow table schema if table exists
* chore: add more test for greptime_identity pipeline
* chore: change pipeline get_table function signature
* chore: change identity_pipeline_inner tag_column_names type
2024-12-12 09:01:21 +00:00
Weny Xu
2137c53274
feat(index): add file_size_hint for remote blob reader ( #5147 )
...
feat(index): add file_size_hint for remote blob reader
2024-12-12 04:45:40 +00:00
