Commit Graph

1941 Commits

Author SHA1 Message Date
dennis zhuang
cf8b6c77dc chore: clean up unused errors (#2901) 2023-12-11 09:08:50 +00:00
Yingwen
6a57f4975e perf(mito): scan SSTs and memtables in parallel (#2852)
* feat: seq scan support parallelism

* feat: scan region by parallelism in config

* feat: enlarge channel size

* chore: parallel builder logs

* feat: use parallel reader accroding to source num

* chore: 128 channel size

* feat: add fetch cost metrics

* feat: add channel size to config

* feat: builder cost

* feat: logs

* feat: compiler error

* feat: fetch cost

* feat: convert cost

* chore: Revert "feat: logs"

This reverts commit 01e0df2c3a.

* chore: fix compiler errors

* feat: reduce channel size to 32

* chore: use workspace tokio-stream

* test: test scan in parallel

* chore: comment typos

* refactor: build all sources first

* test: test 0 parallelism

* feat: use parallel scan by default

* docs: update config example

* feat: log parallelism

* refactor: keep config in engine inner

* refactor: rename parallelism method

* docs: update docs

* test: fix config api test

* docs: update parallel scan comment

* feat: 0 for default parallelism
2023-12-11 06:43:17 +00:00
tison
178018143d ci: prevent running nightly CI in forks (#2898)
Signed-off-by: tison <wander4096@gmail.com>
2023-12-11 02:34:59 +00:00
tison
73227bbafd chore: ignore MySQL client sent SELECT $$ (#2896)
Signed-off-by: tison <wander4096@gmail.com>
2023-12-11 02:27:22 +00:00
Weny Xu
5a99f098c5 test: add tests for region migration procedure (#2857)
* feat: add backward compatibility test for persistent ctx

* refactor: refactor State of region migration

* feat: add test utils for region migration tests

* test: add simple region migration tests

* chore: apply suggestions from CR
v0.5.0-nightly-20231211
2023-12-08 08:47:09 +00:00
Ruihang Xia
7cf9945161 fix: re-enable ignored case test_query_prepared (#2892)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-08 08:35:56 +00:00
tison
bfb4794cfa fix: handle heartbeat shutdown gracefully (#2886)
* fix: handle heartbeat shutdown gracefully

Signed-off-by: tison <wander4096@gmail.com>

* improve logging

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2023-12-08 03:59:05 +00:00
Ruihang Xia
58183fe72f fix: align linear_regression to PromQL's behavior (#2879)
* fix: accept f64 and i64 as predict_linear's param

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use second instead of millisecond

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test to linear_regression

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-08 02:41:10 +00:00
Niwaka
09aa4b72a5 chore: update storage config example (#2887)
chore: update config example
2023-12-07 03:18:56 +00:00
dennis zhuang
43f32f4499 feat: impl date_add/date_sub functions (#2881)
* feat: adds date_add and date_sub function

* test: add date function

* fix: adds interval to date returns wrong result

* fix: header

* fix: typo

* fix: timestamp resolution

* fix: capacity

* chore: apply suggestion

* fix: wrong behavior when adding intervals to timestamp, date and datetime

* chore: remove unused error

* test: refactor and add some tests
2023-12-07 03:02:15 +00:00
tison
ea80570cb1 fix: mysql version function result (#2884)
Signed-off-by: tison <wander4096@gmail.com>
2023-12-06 16:14:09 +00:00
Niwaka
cfe3a2c55e feat!: support table ddl for custom storage (#2733)
* feat: support table ddl for custom_storage

* refactor: rename extract_variant_name to name

* chore: add blank

* chore: keep compatible

* feat: rename custom_stores to providers

* chore: rename

* chore: config

* refactor: add should_retry in client Error

* fix: test fail

* chore: remove unused options

* chore: remove unused import

* chore: remove the blanks.

* chore: revert

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
2023-12-06 15:59:01 +00:00
Ruihang Xia
2cca267a32 chore: tweak status code of promql errors (#2883)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-06 13:50:53 +00:00
tison
f74715ce52 refactor: RegionEngine::handle_request always returns affected rows (#2874)
* refactor: RegionEngine::handle_request -> handle_execution

Signed-off-by: tison <wander4096@gmail.com>

* propagate refactor

Signed-off-by: tison <wander4096@gmail.com>

* revert spell change

Signed-off-by: tison <wander4096@gmail.com>

* propagate refactor

Signed-off-by: tison <wander4096@gmail.com>

* cargo clippy

Signed-off-by: tison <wander4096@gmail.com>

* propagate refactor

Signed-off-by: tison <wander4096@gmail.com>

* cargo fmt

Signed-off-by: tison <wander4096@gmail.com>

* more name clarification

Signed-off-by: tison <wander4096@gmail.com>

* revert rename

Signed-off-by: tison <wander4096@gmail.com>

* wrap affected rows into RegionResponse

Signed-off-by: tison <wander4096@gmail.com>

* flatten return AffectedRows

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2023-12-06 13:27:19 +00:00
Weny Xu
1141dbe946 chore: unify the meta metrics styling (#2875)
* chore: unify the meta metrics styling

* chore: apply suggestions from CR
2023-12-06 09:20:41 +00:00
ZonaHe
a415685bf1 feat: update dashboard to v0.4.2 (#2882)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-12-06 09:13:02 +00:00
dennis zhuang
f9e7762c5b fix: add new column as primary key can't work (#2876) 2023-12-05 11:07:53 +00:00
Zhenchi
0b421b5177 feat(inverted_index.search): add index applier (#2868)
* feat(inverted_index.search): add fst applier

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(inverted_index.search): add fst values mapper

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: remove meta check

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: fmt & clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: one expect for test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(inverted_index.search): add index applier

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: bitmap_full -> bitmap_full_range

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add check for segment_row_count

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: remove redundant code

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: reader test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: match error in test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: fmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: add helper function to construct fst value

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: polish unit tests

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: bytemuck to extract offset and size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: toml format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: use bytemuck

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: reorg value in unit tests

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: update proto

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: add a TODO reminder to consider optimizing the order of apply

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: InList predicates are applied first to benefit from higher selectivity

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: update proto

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add read options to control the behavior of index not found

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: polish

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: move read options to implementation instead of trait

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add SearchContext, refine doc comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: move index_not_found_strategy as a field of SearchContext

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: rename varient

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-05 08:24:24 +00:00
WU Jingdi
aa89d9deef fix: replace opendal PrometheusLayer (#2861)
* fix: replace opendal `PrometheusLayer`

* chore: add docs on `PrometheusMetricsLayer`

* chore: fix code advice

* chore: fix bug on `PrometheusMetricsLayer`
2023-12-05 07:15:45 +00:00
Weny Xu
b3ffe5cd1e feat: handle the downgrade region instruction (#2855)
* feat: handle the downgrade region instruction

* test: add tests for RegionHeartbeatResponseHandler

* refactor: remove unused code
2023-12-05 03:30:55 +00:00
Ruihang Xia
d6ef7a75de fix: type conversion rule reverses operands (#2871)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-05 03:25:29 +00:00
LFC
6344b1e0db fix: fragile integration tests (#2870) 2023-12-05 02:35:23 +00:00
tison
7d506b3c5f feat: drop if exists (#2859)
* feat: drop if exists

Signed-off-by: tison <wander4096@gmail.com>

* sqlness cases

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2023-12-05 02:18:33 +00:00
Zhenchi
96e12e9ee5 fix: correct the previously unsuccessful decimal_ops sort result fix (#2869)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-04 15:29:02 +00:00
Zhenchi
a9db80ab1a feat(inverted_index.search): add fst values mapper (#2862)
* feat(inverted_index.search): add fst applier

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(inverted_index.search): add fst values mapper

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: remove meta check

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: fmt & clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: one expect for test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: match error in test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: fmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: add helper function to construct fst value

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: bytemuck to extract offset and size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: toml format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-04 13:29:02 +00:00
Ruihang Xia
5f5dbe0172 fix: sort result of sqlness case decimal_ops (#2867)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-04 12:34:27 +00:00
Wei
dac7a41cbd feat: sqlness for decimal128 (#2822) 2023-12-04 11:22:38 +00:00
Ruihang Xia
de416465a6 feat: support time() and related functions in PromQL (#2854)
* enhance empty_metric

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implementation

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix lhs & rhs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo, update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add cast to bool modifier

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-04 11:21:54 +00:00
Zhenchi
58c13739f0 feat(inverted_index.search): add fst applier (#2851)
* feat(inverted_index.search): add fst applier

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-04 09:21:09 +00:00
WU Jingdi
806400caff feat: add align to / interval support in range query (#2842)
* feat: add align to / interval support in range query

* chore: fix ci

* chore: simplify `parse_duration_expr`

* chore: change s to ms
2023-12-04 08:00:41 +00:00
Weny Xu
f78dab078c chore: correct closeable typos (#2860) 2023-12-04 06:25:48 +00:00
Weny Xu
7a14db68a6 feat: add upgrade candidate region step (#2829)
* feat: add upgrade candidate region step

* chore: apply suggestions from CR

* chore: apply suggestions from CR
2023-12-04 05:09:27 +00:00
Weny Xu
c26f2f94c0 chore: add logs and metrics (#2858)
* chore: add logs and metrics

* feat: add the timer to track heartbeat intervel

* feat: add the gauge to track region leases

* refactor: use gauge instead of the timer

* chore: apply suggestions from CR

* feat: add hit rate and etcd txn metrics
v0.5.0-nightly-20231204
2023-12-04 02:51:30 +00:00
Weny Xu
781f2422b3 feat: add update metadata step for rollbacking downgraded region (#2812)
* feat: add update metadata step for rollbacking downgraded region

* feat: invalidate table cache after updating metadata

* feat: add migration abort step
2023-12-01 11:36:05 +00:00
Yingwen
7e68ecc498 feat: do not concat batches in MergeReader (#2833) 2023-12-01 06:52:43 +00:00
LFC
9ce9421850 refactor: add builder for Frontend (#2849) 2023-12-01 04:39:47 +00:00
zyy17
c0df2b9086 ci: set 'omitBody' true when releasing (#2845)
ci: set 'omitBody'
2023-11-30 10:53:07 +00:00
Yiran
29d344ccd2 docs: update getting-started document link (#2843) 2023-11-30 10:03:09 +00:00
Wei
fe2fc723bc refactor: DataType name function (#2836)
* refactor: DataType name function

* chore: test case
2023-11-30 03:49:09 +00:00
Wei
2332305b90 refactor: replace usage of ArrayData by clone (#2827)
* refactor: use array clone()

* refactor: slice

* chore: clippy
2023-11-30 03:27:29 +00:00
Ruihang Xia
9ccd182109 feat: implement PromQL set op AND/UNLESS (#2839)
* initial impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable OR for now

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-30 03:17:57 +00:00
Weny Xu
ae8153515b feat: add update metadata step for upgrading candidate region (#2811) 2023-11-29 11:10:38 +00:00
Weny Xu
cce5edc88e feat: add downgrade leader region step (#2792)
* feat: add downgrade leader region step

* chore: apply suggestions from CR

* chore: rename exist to exists

* chore: apply suggestions from CR
2023-11-29 09:17:28 +00:00
Weny Xu
616eb04914 chore: bump version to 0.4.4 (#2840)
chore: bump to v0.4.4
v0.4.4
2023-11-29 08:59:42 +00:00
ZonaHe
7c53f92e4b feat: update dashboard to v0.4.1 (#2841)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-11-29 08:50:25 +00:00
Ruihang Xia
445bd92c7a feat: add arg to standalone start command (#2837)
* feat: add  arg to standalone start command

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add this arg to metasrv

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove short arg name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-29 07:44:43 +00:00
Ruihang Xia
92a9802343 feat: canonicalize all unquoted identifier to lowercase (#2828)
* feat: canonicalize all unquoted identifier to lowercase

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add more tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test altering table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* primary key declare

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix primary key declare

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* partition by and time index

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove redundent call to canonicalize

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-29 06:40:10 +00:00
Yingwen
abbac46c05 fix: do not expose manifest compression algorithm (#2835)
* fix: don't allow to set manifest compression algorithm

* docs: update config examples
2023-11-29 05:49:40 +00:00
ZonaHe
d0d0f091f0 feat: update dashboard to v0.4.0 (#2832)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-11-28 10:38:37 +00:00
fys
707a0d5626 fix: urldecode when influxdb auth (#2831)
* fix: add url decode when influxdb auth

* chore: fmt toml
2023-11-28 09:35:03 +00:00