rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-13 16:52:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,065 Commits 105 Branches 227 Tags
cbb06cd0c635a63aed4265d0cfc100bac7f48eef
Commit Graph

2500 Commits

Author SHA1 Message Date
discord9
cbb06cd0c6 feat(flow): add some metrics (#4539) 
* feat: add some metrics

* fix: tmp rate limiter

* feat: add task count metrics

* refactor: use bounded channel anyway

* refactor: better metrics
 2024-08-14 03:23:49 +00:00
discord9
b59a93dfbc chore: Helper function to convert Vec<Value> to VectorRef (#4546) 
* chore: `try_from_row_into_vector` helper

* test: try_from_row

* refactor: simplify with builder

* fix: deicmal set prec&scale

* refactor: more simplify

* refactor: use ref
 2024-08-14 03:11:44 +00:00
localhost
202c730363 perf: Optimizing pipeline performance (#4390) 
* chore: improve pipeline performance

* chore: use arc to improve time type

* chore: improve pipeline coerce

* chore: add vec refactor

* chore: add vec pp

* chore: improve pipeline

* inprocess

* chore: set log ingester use new pipeline

* chore: fix some error by pr comment

* chore: fix typo

* chore: use enum_dispatch to simplify code

* chore: some minor fix

* chore: format code

* chore: update by pr comment

* chore: fix typo

* chore: make clippy happy

* chore: fix by pr comment

* chore: remove epoch and date process add new timestamp process

* chore: add more test for pipeline

* chore: restore epoch and date processor

* chore: compatibility issue

* chore: fix by pr comment

* chore: move the evaluation out of the loop

* chore: fix by pr comment

* chore: fix dissect output key filter

* chore: fix transform output greptime value has order error

* chore: keep pipeline transform output order

* chore: revert tests

* chore: simplify pipeline prepare implementation

* chore: add test for timestamp pipelin processor

* chore: make clippy happy

* chore: replace is_some check to match

---------

Co-authored-by: shuiyisong <xixing.sys@gmail.com>
 2024-08-13 11:32:04 +00:00
zyy17
63e1892dc1 refactor(plugin): add SetupPlugin and StartPlugin error (#4554) 2024-08-13 11:22:48 +00:00
Lei, HUANG
216bce6973 perf: count(*) for append-only tables (#4545) 
* feat: support fast count(*) for append-only tables

* fix: total_rows stats in time series memtable

* fix: sqlness result changes for SinglePartitionScanner -> StreamScanAdapter

* fix: some cr comments
 2024-08-13 09:27:50 +00:00
shuiyisong
5aa4c70057 chore: update validator signature (#4548) 2024-08-13 08:06:12 +00:00
Weny Xu
c821d21111 feat(log_store): introduce the IndexCollector (#4461) 
* feat: introduce the IndexCollector

* refactor: separate BackgroundProducerWorker code into files

* feat: introduce index related operations

* feat: introduce the `GlobalIndexCollector`

* refactor: move collector to index mod

* refactor: refactor `GlobalIndexCollector`

* chore: remove unused collector.rs

* chore: add comments

* chore: add comments

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-08-13 06:15:24 +00:00
Weny Xu
2e2eacf3b2 feat: add SASL and TLS config for Kafka client (#4536) 
* feat: add SASL and TLS config

* feat: add SASL/PLAIN and TLS config for Kafka client

* chore: use `ring`

* feat: support SASL SCRAM-SHA-256 and SCRAM-SHA-512

* fix: correct unit test

* test: add integration test

* chore: apply suggestions from CR

* refactor: introduce `KafkaConnectionConfig`

* chore: refine toml examples

* docs: add missing fields

* chore: refine examples

* feat: allow no server ca cert

* chore: refine examples

* chore: fix clippy

* feat: load system ca certs

* chore: fmt toml

* chore: unpin version

* Update src/common/wal/src/error.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-08-12 12:27:11 +00:00
Ruihang Xia
9bcaeaaa0e refactor: reuse aligned ts array in range manipulate exec (#4535) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-12 06:26:11 +00:00
JohnsonLee
6694d2a930 fix: change the type of oid in pg_namespace to u32 (#4541) 
* fix:  change the type of oid in pg_namespace to u32

* fix: header and correct logic of update oid
 2024-08-10 15:06:14 +00:00
Ning Sun
9532ffb954 fix: configuration example for selector (#4532) 
* fix: configuration example for selector

* docs: update config docs

* test: update unit tests for configuration in meta
 2024-08-09 09:51:05 +00:00
Weny Xu
665b7e5c6e perf: merge small byte ranges for optimized fetching (#4520) 2024-08-09 08:17:54 +00:00
Weny Xu
27d9aa0f3b fix: rollback only if dropping the metric physical table fails (#4525) 
* fix: rollback only if dropping the metric physical table fails

* chore: apply suggestions from CR
 2024-08-09 08:01:11 +00:00
discord9
8f3293d4fb fix: larger stack size in debug mode (#4521) 
* fix: larger stack size in debug mode

* chore: typo

* chore: clippy

* chore: per review

* chore: rename thread

* chore: per review

* refactor: better looking cfg

* chore: async main entry
 2024-08-09 07:01:20 +00:00
LFC
7dd20b0348 chore: make mysql server version changable (#4531) 2024-08-09 03:43:43 +00:00
Jeremyhi
0d70961448 feat: change the default selector to RoundRobin (#4528) 
* feat: change the default selector to rr

* Update src/meta-srv/src/selector.rs

* fix: unit test
 2024-08-08 04:58:20 +00:00
LFC
a75cfaa516 chore: update snafu to make clippy happy (#4507) 
* chore: update snafu to make clippy happy

* fix ci
 2024-08-07 16:12:00 +00:00
Ruihang Xia
8f0959fa9f fix: fix incorrect result of topk with cte (#4523) 
* fix: fix incorrect result of topk with cte

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up cargo toml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-07 09:13:38 +00:00
Lei, HUANG
be5364a056 chore: support swcs as the short name for strict window compaction (#4517) 2024-08-06 07:38:07 +00:00
Weny Xu
a25d9f736f chore: set default otlp_endpoint (#4508) 
* chore: set default `otlp_endpoint`

* fix: fix ci
 2024-08-06 06:48:14 +00:00
discord9
e6cc4df8c8 feat: flow recreate on reboot (#4509) 
* feat: flow reboot clean

* refactor: per review

* refactor: per review

* test: sqlness flow reboot
 2024-08-05 13:57:48 +00:00
LFC
803780030d fix: too large shadow-rs consts (#4506) 2024-08-05 07:05:14 +00:00
Weny Xu
3937e67694 feat: introduce new kafka topic consumer respecting WAL index (#4424) 
* feat: introduce new kafka topic consumer respecting WAL index

* chore: fmt

* chore: fmt toml

* chore: add comments

* feat: merge close ranges

* fix: fix unit tests

* chore: fix typos

* chore: use loop

* chore: use unstable sort

* chore: use gt instead of gte

* chore: add comments

* chore: rename to `current_entry_id`

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor: minor refactor

* chore: apply suggestions from CR
 2024-08-05 06:56:25 +00:00
LFC
c4717abb68 chore: bump shadow-rs version to set the path to find the correct git repo (#4494) 2024-08-05 02:24:12 +00:00
shuiyisong
3b701d8f5e test: more on processors (#4493) 
* test: add date test

* test: add epoch test

* test: add letter test and complete some others

* test: add urlencoding test

* chore: typo
 2024-08-04 08:29:31 +00:00
Weny Xu
cb4cffe636 chore: bump opendal version to 0.48 (#4499) 2024-08-04 00:46:04 +00:00
Ruihang Xia
cc7f33c90c fix(tql): avoid unwrap on parsing tql query (#4502) 
* fix(tql): avoid unwrap on parsing tql query

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-03 20:58:53 +00:00
Ruihang Xia
fe1cfbf2b3 fix: partition column with mixed quoted and unquoted idents (#4491) 
* fix: partition column with mixed quoted and unquoted idents

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update error message

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-02 09:06:31 +00:00
Yingwen
ded874da04 feat: enlarge default page cache size (#4490) 2024-08-02 07:24:20 +00:00
Yingwen
b388829a96 fix: avoid total size overflow (#4487) 
feat: avoid total size overflow
 2024-08-02 06:16:37 +00:00
Lei, HUANG
9d5d7c1f9a feat(compaction): add file number limits to TWCS compaction (#4481) 
* Add file number limits to TWCS compaction

 - Introduce `max_active_window_files` and `max_inactive_window_files` to `TwcsOptions`.

* feat/limit-files-in-windows: Add max active/inactive window files options to mito engine config

* feat/limit-files-in-windows: Add Debug derive to TwcsPicker and implement max file enforcement logging in TWCS compaction

* fix: clippy
 2024-08-01 12:42:09 +00:00
ZonaHe
efe5eeef14 feat: update dashboard to v0.5.4 (#4483) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2024-08-01 12:19:38 +00:00
Ruihang Xia
ca54b05be3 feat: time poll elapsed for RegionScan plan (#4482) 
* feat: time poll elapsed for RegionScan plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* also record await time

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-01 12:19:15 +00:00
Jeremyhi
d67314789c feat: export all schemas and data at once in export tool (#4478) 
* feat: export all schemas and data at onece

* feat: introduce export all to export schemas and data at once

* feat: default value for target

* feat: refactor export target

* chore: fix unit test
 2024-08-01 09:14:44 +00:00
Yingwen
6c4b8b63a5 fix: notify flush receiver after write buffer is released (#4476) 
* fix: notify the worker after write buffer is released

* feat: worker level region count
 2024-08-01 07:15:36 +00:00
Jeremyhi
62a0defd63 feat: improve extract hints (#4479) 2024-08-01 07:06:13 +00:00
Jeremyhi
291d9d55a4 feat: hint options for gRPC insert (#4454) 
* feat: hint options for gRPC isnert

* chore: unit test for extract_hints

* feat: add integration test for grpc hint

* test: add integration test for hints
 2024-08-01 02:59:38 +00:00
shuiyisong
c66d3090b6 fix: prometheus api only returns 200 (#4471) 
fix: prometheus api returns http status other than 200
 2024-07-31 07:42:50 +00:00
dennis zhuang
656050722c fix: overflow when parsing default value with negative numbers (#4459) 
* fix: overflow when parsing default value with negative numbers

* test: adds sqlness test
 2024-07-31 07:41:49 +00:00
Ning Sun
b741a7181b feat: track channels with query context and w/rcu (#4448) 
* feat: add source channel to meter recorders

* feat: provide channel for query context

* fix: testing and extension get for query context

* chore: revert cargo toml structure changes

* fix: querycontext modification for prometheus and pipeline

* chore: switch git dependency to main branches

* chore: remove TODO

* refactor: rename other to unknown

---------

Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
 2024-07-31 07:30:50 +00:00
Ran Miller
c24de8b908 refactor(servers): improve postgres error message (#4463) 
* refactor(servers): improve postgres error message

* refactor(servers): remove numerical representation of ErrorSeverity
 2024-07-31 06:06:15 +00:00
Yingwen
f382a7695f perf: reduce lock scope and improve log (#4453) 
* feat: refine logs for scan

* feat: improve build parts and  unordered scan metrics

* feat: change to debug log

* fix: release lock before reading part

* test: replace region id

* test: fix sqlness

* chore: add todo

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-07-31 04:07:34 +00:00
Jeremyhi
1ea43da9ea feat: default export catalog name (#4464) 
* feat: default export catalog name

* chore: default catalog name
 2024-07-31 03:39:39 +00:00
LFC
6d8a502430 chore: add more metrics about parquet and cache (#4410) 
* chore: add more metrics about parquet and cache

* resolve PR comments

* resolve PR comments

* resolve PR comments

* resolve PR comments
 2024-07-30 12:01:49 +00:00
Ruihang Xia
2d992f4f12 fix: check_partition uses unqualified name (#4452) 
* fix: check_partition uses unqualified name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-07-30 11:28:28 +00:00
Ruihang Xia
7daf24c47f feat: remove dedicated runtime for grpc, mysql and pg protocols (#4436) 
* feat: remove dedicated runtime for grpc, mysql and pg protocols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other runtimes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* spawn compact task into compact_runtime

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/tests/mysql/mysql_server_test.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* turnoff fuzz test matrix fail fast option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update rt config for ci tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-07-30 06:17:58 +00:00
shuiyisong
567f5105bf fix: missing pre_write check on prometheus remote write (#4460) 
fix: missing pre_write check on prometheus remote write
 2024-07-30 04:55:19 +00:00
taobo
1138f32af9 feat: support setting time range in Copy From statement (#4405) 
* feat: support setting time range in Copy From statement

* test: add batch_filter_test

* fix: ts data type inconsistent error

* test: add sqlness test for copy from with statement

* fix: sqlness result error

* fix: cr comments
 2024-07-29 16:55:19 +00:00
shuiyisong
53fc14a50b fix: use status code to http status mapping in error IntoResponse (#4455) 2024-07-29 16:37:04 +00:00
Ruihang Xia
1895a5478b feat: track prometheus HTTP API's query latency (#4458) 
* feat: track prometheus HTTP API's query latency

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update grafana config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: Update src/servers/src/metrics.rs

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-07-29 15:00:09 +00:00