rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-20 23:10:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,286 Commits 133 Branches 253 Tags
feat/expr-split
Commit Graph

4372 Commits

Author SHA1 Message Date
fys
3517c13192 fix: incremental compilation always compile the common-version crate (#4605) 
fix: wrong cargo:rerun
 2024-08-22 11:00:33 +00:00
Ruihang Xia
b9cedf2c1a perf: optimize series divide algo (#4603) 
* perf: optimize series divide algo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove dead code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-22 09:16:36 +00:00
LFC
883c5bc5b0 refactor: skip checking the existence of the SST files (#4602) 
refactor: skip checking the existence of the SST files when region is directly edited
 2024-08-22 08:32:27 +00:00
Yingwen
d628079f4c feat: collect filters metrics for scanners (#4591) 
* feat: collect filter metrics

* refactor: reuse ReaderFilterMetrics

* feat: record read rows from parquet by type

* feat: unordered scan observe rows

also fix read type

* chore: rename label
 2024-08-22 03:22:05 +00:00
Weny Xu
0025fa6ec7 chore: bump opendal version to 0.49 (#4587) 
* chore: bump opendal version to 0.49

* chore: apply suggestions from CR

* Update src/object-store/src/util.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-08-22 03:05:36 +00:00
Yingwen
a12a905578 chore: disable ttl for write cache by default (#4595) 
* chore: remove default write cache ttl

* docs: update example config

* chore: fix ci
 2024-08-21 08:38:38 +00:00
localhost
bf16422cee fix: pipeline prepare loop break detects a conditional error (#4593) 2024-08-21 06:20:09 +00:00
Ran Joe
9db08dbbe0 refactor(mito2): reduce duplicate IndexOutput struct (#4592) 
* refactor(mito2): reduce duplicate IndexOutput struct

* docs(mito2): add index output note
 2024-08-20 12:30:17 +00:00
fys
9d885fa0c2 chore: bump tikv-jemalloc* to "0.6" (#4590) 
chore: bump tikv-jemalloc* ti "0.6"
 2024-08-20 09:08:21 +00:00
Ruihang Xia
b25a2b117e feat: remove sql in error desc (#4589) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-20 06:37:30 +00:00
ozewr
30af78700f feat: Implement the Buf to avoid extra memory allocation (#4585) 
* feat: Implement the Buf to avoid extra memory allocation

* fmt toml

* fmt code

* mv entry.into_buffer to raw_entry_buffer

* less reuse opendal

* remove todo #4065

* Update src/mito2/src/wal/entry_reader.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fmt code

---------

Co-authored-by: ozewr <l19ht@google.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-08-19 12:11:08 +00:00
Ruihang Xia
8de11a0e34 perf: set simple filter on primary key columns to exact filter (#4564) 
* perf: set simple filter on primary key columns to exact filter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-19 09:07:35 +00:00
Weny Xu
76dc906574 feat(log_store): introduce the CollectionTask (#4530) 
* feat: introduce the `CollectionTask`

* feat: add config of index collector

* chore: remove unused code

* feat: truncate indexes

* chore: apply suggestions from CR

* chore: update config examples

* refactor: retrieve latest offset while dumping indexes

* chore: print warn
 2024-08-19 03:48:35 +00:00
Ran Joe
2a73e0937f fix(common_version): short_version with empty branch (#4572) 2024-08-19 03:14:49 +00:00
Zhenchi
c8de8b80f4 fix(fulltext-index): single segment is not sufficient for >50M rows SST (#4552) 
* fix(fulltext-index): single segment is not sufficient for a >50M rows SST

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: update doc comment

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-08-16 09:14:33 +00:00
LFC
ec59ce5c9a feat: able to handle concurrent region edit requests (#4569) 
* feat: able to handle concurrent region edit requests

* resolve PR comments
 2024-08-16 03:29:03 +00:00
liyang
f578155602 feat: add GcsConfig credential field (#4568) 2024-08-16 03:11:20 +00:00
Weny Xu
d1472782d0 chore(log_store): remove redundant metrics (#4570) 
chore(log_store): remove unused metrics
 2024-08-16 02:23:21 +00:00
Lanqing Yang
93be81c041 feat: implement postgres kvbackend (#4421) 2024-08-14 22:49:32 +00:00
discord9
2c3fccb516 feat(flow): add eval_batch for ScalarExpr (#4551) 
* refactor: better perf flow

* feat(WIP): batching proc

* feat: UnaryFunc::eval_batch untested

* feat: BinaryFunc::eval_batch untested

* feat: VariadicFunc::eval_batch un tested

* feat: literal eval_batch

* refactor: move DfScalarFunc to separate file

* chore: remove unused imports

* feat: eval_batch df func&ifthen

* chore: remove unused file

* refactor: use Batch type

* chore: remove unused

* chore: remove a done TODO

* refactor: per review

* chore: import

* refactor: eval_batch if then

* chore: typo
 2024-08-14 11:29:30 +00:00
Lei, HUANG
c1b1be47ba fix: append table stats (#4561) 
* fix: append table stats

* fix: clippy
 2024-08-14 09:01:42 +00:00
Weny Xu
0f85037024 chore: remove unused code (#4559) 2024-08-14 06:55:54 +00:00
discord9
cbb06cd0c6 feat(flow): add some metrics (#4539) 
* feat: add some metrics

* fix: tmp rate limiter

* feat: add task count metrics

* refactor: use bounded channel anyway

* refactor: better metrics
 2024-08-14 03:23:49 +00:00
discord9
b59a93dfbc chore: Helper function to convert Vec<Value> to VectorRef (#4546) 
* chore: `try_from_row_into_vector` helper

* test: try_from_row

* refactor: simplify with builder

* fix: deicmal set prec&scale

* refactor: more simplify

* refactor: use ref
 2024-08-14 03:11:44 +00:00
localhost
202c730363 perf: Optimizing pipeline performance (#4390) 
* chore: improve pipeline performance

* chore: use arc to improve time type

* chore: improve pipeline coerce

* chore: add vec refactor

* chore: add vec pp

* chore: improve pipeline

* inprocess

* chore: set log ingester use new pipeline

* chore: fix some error by pr comment

* chore: fix typo

* chore: use enum_dispatch to simplify code

* chore: some minor fix

* chore: format code

* chore: update by pr comment

* chore: fix typo

* chore: make clippy happy

* chore: fix by pr comment

* chore: remove epoch and date process add new timestamp process

* chore: add more test for pipeline

* chore: restore epoch and date processor

* chore: compatibility issue

* chore: fix by pr comment

* chore: move the evaluation out of the loop

* chore: fix by pr comment

* chore: fix dissect output key filter

* chore: fix transform output greptime value has order error

* chore: keep pipeline transform output order

* chore: revert tests

* chore: simplify pipeline prepare implementation

* chore: add test for timestamp pipelin processor

* chore: make clippy happy

* chore: replace is_some check to match

---------

Co-authored-by: shuiyisong <xixing.sys@gmail.com>
 2024-08-13 11:32:04 +00:00
zyy17
63e1892dc1 refactor(plugin): add SetupPlugin and StartPlugin error (#4554) 2024-08-13 11:22:48 +00:00
Lei, HUANG
216bce6973 perf: count(*) for append-only tables (#4545) 
* feat: support fast count(*) for append-only tables

* fix: total_rows stats in time series memtable

* fix: sqlness result changes for SinglePartitionScanner -> StreamScanAdapter

* fix: some cr comments
 2024-08-13 09:27:50 +00:00
shuiyisong
5aa4c70057 chore: update validator signature (#4548) 2024-08-13 08:06:12 +00:00
Weny Xu
c821d21111 feat(log_store): introduce the IndexCollector (#4461) 
* feat: introduce the IndexCollector

* refactor: separate BackgroundProducerWorker code into files

* feat: introduce index related operations

* feat: introduce the `GlobalIndexCollector`

* refactor: move collector to index mod

* refactor: refactor `GlobalIndexCollector`

* chore: remove unused collector.rs

* chore: add comments

* chore: add comments

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-08-13 06:15:24 +00:00
Weny Xu
2e2eacf3b2 feat: add SASL and TLS config for Kafka client (#4536) 
* feat: add SASL and TLS config

* feat: add SASL/PLAIN and TLS config for Kafka client

* chore: use `ring`

* feat: support SASL SCRAM-SHA-256 and SCRAM-SHA-512

* fix: correct unit test

* test: add integration test

* chore: apply suggestions from CR

* refactor: introduce `KafkaConnectionConfig`

* chore: refine toml examples

* docs: add missing fields

* chore: refine examples

* feat: allow no server ca cert

* chore: refine examples

* chore: fix clippy

* feat: load system ca certs

* chore: fmt toml

* chore: unpin version

* Update src/common/wal/src/error.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-08-12 12:27:11 +00:00
Ruihang Xia
9bcaeaaa0e refactor: reuse aligned ts array in range manipulate exec (#4535) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-12 06:26:11 +00:00
JohnsonLee
6694d2a930 fix: change the type of oid in pg_namespace to u32 (#4541) 
* fix:  change the type of oid in pg_namespace to u32

* fix: header and correct logic of update oid
 2024-08-10 15:06:14 +00:00
Ning Sun
9532ffb954 fix: configuration example for selector (#4532) 
* fix: configuration example for selector

* docs: update config docs

* test: update unit tests for configuration in meta
 2024-08-09 09:51:05 +00:00
Weny Xu
665b7e5c6e perf: merge small byte ranges for optimized fetching (#4520) 2024-08-09 08:17:54 +00:00
Weny Xu
27d9aa0f3b fix: rollback only if dropping the metric physical table fails (#4525) 
* fix: rollback only if dropping the metric physical table fails

* chore: apply suggestions from CR
 2024-08-09 08:01:11 +00:00
discord9
8f3293d4fb fix: larger stack size in debug mode (#4521) 
* fix: larger stack size in debug mode

* chore: typo

* chore: clippy

* chore: per review

* chore: rename thread

* chore: per review

* refactor: better looking cfg

* chore: async main entry
 2024-08-09 07:01:20 +00:00
LFC
7dd20b0348 chore: make mysql server version changable (#4531) 2024-08-09 03:43:43 +00:00
Jeremyhi
0d70961448 feat: change the default selector to RoundRobin (#4528) 
* feat: change the default selector to rr

* Update src/meta-srv/src/selector.rs

* fix: unit test
 2024-08-08 04:58:20 +00:00
LFC
a75cfaa516 chore: update snafu to make clippy happy (#4507) 
* chore: update snafu to make clippy happy

* fix ci
 2024-08-07 16:12:00 +00:00
Ruihang Xia
8f0959fa9f fix: fix incorrect result of topk with cte (#4523) 
* fix: fix incorrect result of topk with cte

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up cargo toml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-07 09:13:38 +00:00
Lei, HUANG
be5364a056 chore: support swcs as the short name for strict window compaction (#4517) 2024-08-06 07:38:07 +00:00
Weny Xu
a25d9f736f chore: set default otlp_endpoint (#4508) 
* chore: set default `otlp_endpoint`

* fix: fix ci
 2024-08-06 06:48:14 +00:00
discord9
e6cc4df8c8 feat: flow recreate on reboot (#4509) 
* feat: flow reboot clean

* refactor: per review

* refactor: per review

* test: sqlness flow reboot
 2024-08-05 13:57:48 +00:00
LFC
803780030d fix: too large shadow-rs consts (#4506) 2024-08-05 07:05:14 +00:00
Weny Xu
3937e67694 feat: introduce new kafka topic consumer respecting WAL index (#4424) 
* feat: introduce new kafka topic consumer respecting WAL index

* chore: fmt

* chore: fmt toml

* chore: add comments

* feat: merge close ranges

* fix: fix unit tests

* chore: fix typos

* chore: use loop

* chore: use unstable sort

* chore: use gt instead of gte

* chore: add comments

* chore: rename to `current_entry_id`

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor: minor refactor

* chore: apply suggestions from CR
 2024-08-05 06:56:25 +00:00
LFC
c4717abb68 chore: bump shadow-rs version to set the path to find the correct git repo (#4494) 2024-08-05 02:24:12 +00:00
shuiyisong
3b701d8f5e test: more on processors (#4493) 
* test: add date test

* test: add epoch test

* test: add letter test and complete some others

* test: add urlencoding test

* chore: typo
 2024-08-04 08:29:31 +00:00
Weny Xu
cb4cffe636 chore: bump opendal version to 0.48 (#4499) 2024-08-04 00:46:04 +00:00
Ruihang Xia
cc7f33c90c fix(tql): avoid unwrap on parsing tql query (#4502) 
* fix(tql): avoid unwrap on parsing tql query

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-03 20:58:53 +00:00
Ruihang Xia
fe1cfbf2b3 fix: partition column with mixed quoted and unquoted idents (#4491) 
* fix: partition column with mixed quoted and unquoted idents

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update error message

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-02 09:06:31 +00:00