Commit Graph

391 Commits

Author SHA1 Message Date
Ruihang Xia
6b8dbcfb54 chore: update toolchain to 20231219 (#2932)
* update toolchain file, remove unused feature gates

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update action file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update to 12-19

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-19 07:24:08 +00:00
Zhenchi
ccbd49777d fix: correct the error message for snappy compress (#2956)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-19 06:14:40 +00:00
Zhenchi
029ff2f1e3 feat(inverted_index.create): add read/write for external intermediate files (#2942)
* feat(inverted_index.create): add read/write for external intermediate files

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: MAGIC_CODEC_V1 -> CODEC_V1_MAGIC

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: fix typos intermedia -> intermediate

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: futures_code -> asynchronous_codec

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: bump bytes to 1.5

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-18 09:44:48 +00:00
shuiyisong
5dba373ede chore: return json body under http status 401 (#2924)
* chore: change auth_fn to function and return response with json body

* chore: move unsupported to debug level

* chore: add docs and tests

* chore: rebase and update test
2023-12-14 10:01:12 +00:00
Ning Sun
39f80876cd feat: upgrade rustls library family, opensrv-mysql and pgwire (#2927)
* feat: deps up

* fmt: toml format
2023-12-14 05:56:39 +00:00
JeremyHi
99dda93f0e feat: sql with influxdb v1 result format (#2917)
* feat: sql with influxdb v1 result format

* chore: add unit tests

* feat: minor refactor

* chore: by comment

* chore; u128 to u64 since serde can't deser u128 in enum

* chore: by comment

* chore: apply suggestion

* chore: revert suggestion

* chore: try again

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
2023-12-13 16:15:37 +00:00
ZonaHe
d3da128d66 feat: update dashboard to v0.4.3 (#2926)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-12-13 11:12:17 +00:00
LFC
880ca2e786 fix: return error to client if prepare stmt param not match (#2918)
* fix: return error to client if prepare stmt param not match

* Update src/servers/src/mysql/handler.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-12 12:41:09 +00:00
Ruihang Xia
8a74bd36f5 style: rename *Adaptor to *Adapter (#2914)
* rename RecordBatchStreamAdaptor to RecordBatchStreamWrapper

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename ServerSqlQueryHandlerAdaptor to ServerSqlQueryHandlerAdapter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-12 09:45:09 +00:00
Yingwen
6a57f4975e perf(mito): scan SSTs and memtables in parallel (#2852)
* feat: seq scan support parallelism

* feat: scan region by parallelism in config

* feat: enlarge channel size

* chore: parallel builder logs

* feat: use parallel reader accroding to source num

* chore: 128 channel size

* feat: add fetch cost metrics

* feat: add channel size to config

* feat: builder cost

* feat: logs

* feat: compiler error

* feat: fetch cost

* feat: convert cost

* chore: Revert "feat: logs"

This reverts commit 01e0df2c3a.

* chore: fix compiler errors

* feat: reduce channel size to 32

* chore: use workspace tokio-stream

* test: test scan in parallel

* chore: comment typos

* refactor: build all sources first

* test: test 0 parallelism

* feat: use parallel scan by default

* docs: update config example

* feat: log parallelism

* refactor: keep config in engine inner

* refactor: rename parallelism method

* docs: update docs

* test: fix config api test

* docs: update parallel scan comment

* feat: 0 for default parallelism
2023-12-11 06:43:17 +00:00
tison
73227bbafd chore: ignore MySQL client sent SELECT $$ (#2896)
Signed-off-by: tison <wander4096@gmail.com>
2023-12-11 02:27:22 +00:00
Ruihang Xia
7cf9945161 fix: re-enable ignored case test_query_prepared (#2892)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-08 08:35:56 +00:00
tison
ea80570cb1 fix: mysql version function result (#2884)
Signed-off-by: tison <wander4096@gmail.com>
2023-12-06 16:14:09 +00:00
ZonaHe
a415685bf1 feat: update dashboard to v0.4.2 (#2882)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-12-06 09:13:02 +00:00
Zhenchi
58c13739f0 feat(inverted_index.search): add fst applier (#2851)
* feat(inverted_index.search): add fst applier

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
2023-12-04 09:21:09 +00:00
Wei
fe2fc723bc refactor: DataType name function (#2836)
* refactor: DataType name function

* chore: test case
2023-11-30 03:49:09 +00:00
ZonaHe
7c53f92e4b feat: update dashboard to v0.4.1 (#2841)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-11-29 08:50:25 +00:00
ZonaHe
d0d0f091f0 feat: update dashboard to v0.4.0 (#2832)
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
2023-11-28 10:38:37 +00:00
fys
707a0d5626 fix: urldecode when influxdb auth (#2831)
* fix: add url decode when influxdb auth

* chore: fmt toml
2023-11-28 09:35:03 +00:00
Ning Sun
bd9c2f2666 fix: windows build and check ci check for windows (#2819) 2023-11-27 03:42:44 +00:00
Wei
3a4c9f2b45 feat: supports decimal type in RPC (#2788)
* refactor: ColumnDataTypeWrapper

* feat: decimal128 grpc

* feat: add test case

* chore: add TODO

* chore: empty line

* chore: remove precision and scale

* refactor: remove precision and scale

* chore: remove sqlness test

* chore: rename

* chore: proto version

* chore: cr comment.

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/mito2/src/memtable/time_series.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-11-24 09:19:33 +00:00
LFC
64a36e9b36 refactor: start datanode more flexibly (#2800)
* refactor: start datanode more flexibly

* Update src/datanode/src/datanode.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fix: resolve PR comments

* Apply suggestions from code review

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
2023-11-24 08:16:21 +00:00
Ning Sun
85eebcb16f fix: correct mysql timestamp encoding for binary protocol (#2797)
* fix: correct mysql timestamp encoding for binary protocol

* test: add tests for mysql timestamp encoding
2023-11-23 18:54:59 +00:00
Ruihang Xia
56fc77e573 fix: add missing error display message (#2791)
* fix: add missing error display message

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-23 02:59:49 +00:00
WU Jingdi
500e299e40 feat: Enable distributed tracing in greptimedb (#2755)
* feat: implement distributed tracing

* fix: change usage of span

* fix: use otlp as exporter

* chore: update dependence

* chore: add span info

* chore: add alias

* chore: use instrument instead of trace
2023-11-17 08:51:57 +00:00
Yingwen
06327fba1e feat: improve http error message (#2767)
* feat: improve http error message

* feat: log http error

* style: fix clippy

* test: fix test
2023-11-17 04:03:30 +00:00
Yingwen
06da33b1ed fix: mysql writer does not print error message in some cases (#2763)
* fix: mysql writer does not print error in some cases

* feat: http error msg
2023-11-17 03:07:54 +00:00
Yingwen
142035340d fix: compiler errors under non-default features (#2761)
* feat: bump prost and fix pprof feature compiler errors

* feat: fix compiler errors on tokio-console

* chore: fix compiler errors

* ci: add all features check to ci
2023-11-16 19:43:44 +00:00
Wei
ae27fbc7f2 feat: impl Vector, DataType traits for decimal128 (#2727)
* feat: decimal128 vector

* feat: Decimal128Type in datatypes

* refactor: cast function

* feat: decimal128 type in ConcreteDataType

* chore: decimal

* chore: sql_data_type_to_concrete_data_type

* chore: cr comment

Co-authored-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: fix finish()

* chore: add debug_assert to new decimal128

---------

Co-authored-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-11-16 06:44:26 +00:00
Ruihang Xia
506e6887f3 chore: add rerun-if-changed to dashboard feature (#2728)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-13 18:20:59 +00:00
Ruihang Xia
6599bb5a46 feat: do not require worker's number to be power of 2 (#2732)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-13 09:04:44 +00:00
tison
f5eede4ce1 feat: support prometheus format_query endpoint (#2731)
* feat: support prometheus format_query endpoint

Signed-off-by: tison <wander4096@gmail.com>

* add test

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2023-11-10 10:33:28 +00:00
tison
0cd6dacb45 chore: multiple licenses fixes (#2714)
* chore: avoid license risk from akka

Signed-off-by: tison <wander4096@gmail.com>

* chore: LICENSE test is AS IS

Signed-off-by: tison <wander4096@gmail.com>

* ci: use hawkeye to pin in inceptionYear

Signed-off-by: tison <wander4096@gmail.com>

* add instruction for running license header apply

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2023-11-09 10:38:12 +00:00
Ruihang Xia
93f21b188d chore: unify workspace dependencies and remove some unused (#2713)
* log-store

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* base64

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* udeps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* snafu

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* snafu style

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change the workspace deps' style

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-11-09 07:34:22 +00:00
yuanbohan
5b70881098 feat: specialize Display for String OtlpAnyValue (#2699) 2023-11-08 03:24:13 +00:00
yuanbohan
68f92ecf08 feat(otlp): json attributes via impl Serialeze trait (#2685)
* feat: json attributes via impl Serialeze trait

* chore: rename TraceLink to SpanLink

* feat: support serialize null value
2023-11-03 07:27:04 +00:00
yuanbohan
4b48c716b2 fix(OTLP/Trace): use Timestamp instead of Time for json (#2677)
fix: use Timestamp instead of Time for trace time
2023-11-01 09:41:51 +00:00
WU Jingdi
180bc64cb0 feat: change metric crate from metrics to prometheus (#2655)
* feat: change metrics to prometheus

* chore: fix code advise

* chore: resolve merge conflict

* chore: fix code advise
2023-10-31 15:46:57 +00:00
yuanbohan
84bcca9117 fix: events or links to string (#2667) 2023-10-31 02:41:47 +00:00
Ruihang Xia
5d8b0e8154 chore: bump dependencies (#2659)
* fix all hard error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* trivial changes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix order by

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix sql keyword and data type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix range exec's input partitioning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix cover input type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix explain analyze

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update distributed mode sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix lints

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update locks

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* downgrade dlv-list

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-10-26 12:24:14 +00:00
yuanbohan
44280f7c9d feat(otlp): initial OTLP trace support (#2627)
* feat: otlp tracing framework via http

* feat: otlp trace transformer plugin

* feat: successfully write traces into db

* chore: plugin to parse request

* test: helper functions

* feat: parse_request_to_spans function

* chore: remove implicite calling parse in PraceParser

* chore: fix clippy

* chore: add TODO marker for span fields

* refactor TraceParser trait

* refactor TraceParser trait

* table_name method in OTLP TraceParser trait

* fix: approximate row, column count

* chore: function signature without row

* chore: do not clone by moving span.kind upper

* docs for parse and to_grpc_insert_requests

---------

Co-authored-by: fys <fengys1996@gmail.com>
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com>
2023-10-23 06:37:43 +00:00
Ning Sun
0fbde48655 feat: hide internal error and unknown error message from end user (#2544)
* feat: use fixed error message for unknown error

* feat: return fixed message for internal error as well

* chore: include status code in error message

* test: update tests for asserts of error message

* feat: change status code of some datafusion error

* fix: make CollectRecordbatch an query error

* test: update sqlness results
2023-10-23 03:07:35 +00:00
Baasit
346b57cf10 feat: row protocol support for opentsdb (#2623)
* feat: opentsdb row protocol

* fix: added commnets for num of rows and failure if output is not of affecetd rows

* fix: added extra 1 to number of columns

* fix: avoided cloning datapoints, took ownership instead

* fix: avoided cloning datapoints, took ownership instead

* fix: changed vecotr slice to vector

* fix: remove clone

* fix: combined datapoints and requests with zip instead of enumerating

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-43-183.us-east-2.compute.internal>
2023-10-20 06:25:59 +00:00
WU Jingdi
46e106bcc3 feat: allow nest range expr in Range Query (#2557)
* feat: eable range expr nest

* fix: change range expr rewrite format

* chore: organize range query tests

* chore: change range expr name(e.g. MAX(v) RANGE 5s FILL 6)

* chore: add range query test

* chore: fix code advice

* chore: fix ca
2023-10-18 07:03:26 +00:00
Yun Chen
f859932745 fix: convert to ReadableSize & Durations (#2594)
* fix: convert to ReadableSize & Durations

* fix: change more grpc sender/recv message size to ReadableSize

fix: format

fix: cargo fmt

fix: change cmd test to use durations

fix: revert metaclient change

fix: convert default fields in meta client options

fix: human serde meta client durations

* fix: remove milisecond postfix in heartbeat option

* fix: humantime serde on heartbeat

* fix: update config example

* fix: update integration test config

* fix: address pr comments

* fix: fix pr comment on default annotation
2023-10-13 03:28:29 +00:00
shuiyisong
007f7ba03c refactor: extract plugins crate (#2487)
* chore: move frontend plugins fn

* chore: move datanode plugins to fn

* chore: add opt plugins

* chore: add plugins to meta-srv

* chore: setup meta plugins, wait for router extension

* chore: try use configurator for grpc too

* chore: minor fix fmt

* chore: minor fix fmt

* chore: add start meta_srv for hook

* chore: merge develop

* chore: minor fix

* chore: replace Arc<Plugins> with PluginsRef

* chore: fix header

* chore: remove empty file

* chore: modify comments

* chore: remove PluginsRef type alias

* chore: remove `OptPlugins`
2023-10-09 04:54:27 +00:00
Yingwen
17b385a985 fix: compiler errors under pprof and mem-prof features (#2537)
* fix: compiler errors under pprof feature

* fix: compiler errors under mem-prof feature
2023-10-08 08:28:45 +00:00
Weny Xu
a680133acc feat: enable no delay for mysql, opentsdb, http (#2530)
* refactor: enable no delay for mysql, opentsdb, http

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
2023-10-08 06:19:52 +00:00
Ning Sun
0ad3fb6040 fix: mysql timezone settings (#2534)
* fix: restore time zone settings for mysql

* test: add integration test for time zone

* test: fix unit test for check
2023-10-07 10:21:32 +00:00
Weny Xu
00fe7d104e feat: enable tcp no_delay by default for internal services (#2527) 2023-10-07 02:35:28 +00:00