rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-14 01:02:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,726 Commits 105 Branches 227 Tags
20ce7d428d23f573c3364ebd4596b3e3794de549
Commit Graph

2236 Commits

Author SHA1 Message Date
Zhenchi
dbf62f3273 chore(index): add BiError to fulfil the requirement of returning two errors (#3291) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-02-06 16:03:03 +00:00
Ruihang Xia
e4cd294ac0 feat: put all filter exprs in a filter plan separately (#3288) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-02-06 09:32:26 +00:00
fys
74bfb09195 feat: support cache for batch_get in CachedMetaKvBackend (#3277) 
* feat: support cache for batch_get in CachedMetaKvBackend.

* add doc of CachedMetaKvBackend

* fix: cr

* fix: correct some words

* fix cr
 2024-02-06 03:15:03 +00:00
shuiyisong
4cbdf64d52 chore: start plugins during standalone startup & comply with current catalog while changing database (#3282) 
* chore: start plugins in standalone

* chore: respect current catalog in use statement for mysql

* chore: reduce unnecessory convert to string

* chore: reduce duplicate code
 2024-02-06 02:41:37 +00:00
Weny Xu
770da02810 fix: fix incorrect StatusCode parsing (#3281) 
* fix: fix incorrect StatusCode parsing

* chore: apply suggestions from CR
 2024-02-05 08:06:43 +00:00
discord9
c62c67cf18 feat: Basic Definitions for Expression&Functions for Dataflow (#3267) 
* added expression&func

* fix: EvalError derive&imports

* chore: add header

* feat: variadic func

* chore: minor adjust

* feat: accum

* feat: use accum for eval func

* feat: montonic min/max as accumulative

* feat: support min/max Date&DateTime

* chore: fix compile error&add test(WIP)

* test: sum, count, min, max

* feat: remove trait impl for EvalError

* chore: remove all impl retain only type definitions

* refactor: nest datatypes errors

* fix: remove `.build()`

* fix: not derive Clone

* docs: add comment for types

* feat: more func&remove `CurrentDatabase`
 2024-02-05 07:49:34 +00:00
Ruihang Xia
51feec2579 feat: use simple filter to prune memtable (#3269) 
* switch on clippy warnings

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: use simple filter to prune memtable

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine util function

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-02-04 11:35:55 +00:00
shuiyisong
6ab3a88042 fix: use fe_opts after setup_frontend_plugins in standalone (#3275) 
* chore: modify standalone startup opts

* chore: move frontend and datanode options
 2024-02-02 10:36:08 +00:00
discord9
e89f5dc908 feat: support fraction part in timestamp (#3272) 
* feat: support fraction part in timestamp

* test: with timezone
 2024-02-01 08:51:26 +00:00
LFC
e375060b73 refactor: add same SST files (#3270) 
* Make adding same SST file multiple times possible, instead of panic there.

* Update src/mito2/src/sst/version.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-01-31 07:21:30 +00:00
tison
60e760b168 fix: cli export database default value (#3259) 
Signed-off-by: tison <wander4096@gmail.com>
 2024-01-30 09:56:05 +00:00
dennis zhuang
43ef0820c0 fix: SQL insertion and column default constraint aware of timezone (#3266) 
* fix: missing timezone when parsing sql value to greptimedb value

* test: sql_value_to_value

* fix: column default constraint missing timezone

* test: column def default constraint  with timezone

* test: adds sqlness test for default constraint aware of timezone

* fix: typo

* chore: comment
 2024-01-30 09:15:38 +00:00
discord9
0a9361a63c feat: basic types for Dataflow Framework (#3186) 
* feat: basic types for Dataflow Framework

* docs: license header

* chore: add name in todo, remove deprecated code

* chore: typo

* test: linear&repr unit test

* refactor: avoid mod.rs

* feat: Relation Type

* feat: unmat funcs

* feat: simple temporal filter(sliding window)

* refactor: impl Snafu for EvalError

* feat: cast as type

* feat: temporal filter

* feat: time index in RelationType

* refactor: move EvalError to expr

* refactor: error handling for func

* chore: fmt&comment

* make EvalError pub again

* refactor: move ScalarExpr to scalar.rs

* refactor: remove mod.rs for relation

* chore: slim down PR size

* chore: license header

* chore: per review

* chore: more fix per review

* chore: even more fix per review

* chore: fmt

* chore: fmt

* feat: impl From/Into ProtoRow instead

* chore: use cast not cast_with_opt&`Key` struct

* chore: new_unchecked

* feat: `Key::subset_of` method

* chore: toml in order
 2024-01-30 07:48:22 +00:00
Weny Xu
ddbd0abe3b fix(Copy From): fix incorrect type casts (#3264) 
* refactor: refactor RecordBatchStreamTypeAdapter

* fix(Copy From): fix incorrect type casts

* fix: unit tests

* chore: add comment
 2024-01-30 07:16:36 +00:00
Ruihang Xia
a079955d38 chore: adjust storage engine related metrics (#3261) 
* chore: adjust metrics to metric engine and mito engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adjust more mito bucket

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-01-30 06:43:03 +00:00
JeremyHi
e5a2b0463a feat: Only allow inserts and deletes operations to be executed in parallel (#3257) 
* feat: Only allow inserts and deletes operations to be executed in parallel.

* feat: add comment
 2024-01-29 11:27:06 +00:00
Ruihang Xia
9a28a1eb5e fix: decouple columns in projection and prune (#3253) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-01-29 08:29:21 +00:00
shuiyisong
91c8c62d6f chore: adjust MySQL connection log (#3251) 
* chore: adjust mysql connection log level

* chore: adjust import
 2024-01-29 06:53:50 +00:00
JeremyHi
3201aea360 feat: create tables in batch on prom write (#3246) 
* feat: create tables in batch on prom write

* feat: add logic table ids to log

* fix: miss tabble ids in response
 2024-01-26 12:47:24 +00:00
Ruihang Xia
7da8f22cda fix: IntermediateWriter closes underlying writer twice (#3248) 
* fix: IntermediateWriter closes underlying writer twice

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* close writer manually on error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-01-26 10:03:50 +00:00
Ning Sun
14b233c486 test: align chrono and some forked test deps to upstream (#3245) 
* test: update chrono and its tests

* chore: switch some deps to upstream version

* test: update timestamp range in sqlness tests
 2024-01-26 07:39:51 +00:00
Ruihang Xia
b0c3be35fb feat: don't map semantic type in metric engine (#3243) 
* feat: don't map semantic type in metric engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicate set_null

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-01-26 03:50:05 +00:00
Ruihang Xia
5617b284c5 feat: return request outdated error on handling alter (#3239) 
* feat: return request outdated error on handling alter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix tonic code mapping

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy, add comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix deadloop

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update UT

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* address CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: Update log message

* Update src/common/meta/src/ddl/alter_table.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fix compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-01-26 03:37:46 +00:00
JeremyHi
1fab7ab75a feat: batch create ddl (#3194) 
* feat: batch ddl to region request

* feat: return table ids

chore: by comment

chore: remove wal_options

chore: create logical tables lock key

feat: get metadata in procedure

* chore: by comment
 2024-01-26 02:43:57 +00:00
Yingwen
3fa070a0cc fix: init parquet reader metrics twice (#3242) 2024-01-26 01:54:51 +00:00
Weny Xu
8bade8f8e4 fix: fix create table ddl return incorrect table id (#3232) 
* fix: fix create table ddl return incorrect table id

* refactor: refactor param of Status::done_with_output
 2024-01-25 13:58:43 +00:00
Wei
6c2f0c9f53 feat: read metadata from write cache (#3224) 
* feat: read meta from write cache

* test: add case

* chore: cr comment

* chore: clippy

* chore: code style

* feat: put metadata to sst cache
 2024-01-25 11:39:41 +00:00
Ning Sun
1bc4f25de2 feat: http sql api return schema on empty resultset (#3237) 
* feat: return schema on empty resultset

* refactor: make schema a required field in http output

* test: update integration test and add schema output
 2024-01-25 06:44:28 +00:00
ZonaHe
b0a8046179 feat: update dashboard to v0.4.7 (#3229) 2024-01-24 08:50:21 +00:00
dennis zhuang
7323e9b36f feat: change Range Query’s default align behavior aware of timezone (#3219) 
* feat: change Range Query’s default align behavior to 1970-01-01 00:00:00 aware of timezone

* test: test with +23:00 timezone
 2024-01-24 08:17:57 +00:00
Ning Sun
1711ad4631 feat: add Arrow IPC output format for http rest api (#3177) 
* feat: add arrow format output for sql api

* refactor: remove unwraps

* test: add test for arrow format

* chore: update cargo toml format

* fix: resolve lint warrnings

* fix: ensure outputs size is one
 2024-01-24 06:10:05 +00:00
LFC
f81e37f508 refactor: make http server built flexibly (#3225) 
* refactor: make http server built flexibly

* Apply suggestions from code review

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* fix: resolve PR comments

* Fix CI.

---------

Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2024-01-24 03:45:08 +00:00
Weny Xu
d75cf86467 fix: only register region keeper while creating physical table (#3223) 
fix: only register region keeper during create physical table
 2024-01-23 09:42:32 +00:00
Weny Xu
26535f577d feat: enable concurrent write (#3214) 
* feat: enable concurrent write

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-01-23 09:20:12 +00:00
Ruihang Xia
8485c9af33 feat: read column and region info from state cache (#3222) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-01-23 09:10:27 +00:00
Weny Xu
19413eb345 refactor!: rename initialize_region_in_background to init_regions_in_background (#3216) 
refactor!: change initialize_region_in_background to init_regions_in_background
 2024-01-23 04:33:48 +00:00
Weny Xu
007b63dd9d fix: fix default value cannot accept negative number (#3217) 
* fix: fix default value cannot accept negative number

* chore: apply suggestions from CR
 2024-01-23 03:33:13 +00:00
Ruihang Xia
31787f4bfd feat!: switch prom remote write to metric engine (#3198) 
* feat: switch prom remote write to metric engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* fix compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* read physical table name from url

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove physical table from header

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix merge error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add with_metric_engine option to config remote write behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check parameter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add specific config param

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* default with_metric_engine to true

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update UT

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-01-22 14:48:28 +00:00
dennis zhuang
6a12c27e78 feat: make query be aware of timezone setting (#3175) 
* feat: let TypeConversionRule aware query context timezone setting

* chore: don't optimize explain command

* feat: parse string into timestamp with timezone

* fix: compile error

* chore: check the scalar value type in predicate

* chore: remove mut for engine context

* chore: return none if the scalar value is utf8 in time range predicate

* fix: some fixme

* feat: let Date and DateTime parsing from string value be aware of timezone

* chore: tweak

* test: add datetime from_str test with timezone

* feat: construct function context from query context

* test: add timezone test for to_unixtime and date_format function

* fix: typo

* chore: apply suggestion

* test: adds string with timezone

* chore: apply CR suggestion

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* chore: apply suggestion

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-01-22 14:14:03 +00:00
shuiyisong
2bf4b08a6b chore: change default factor to compute memory size (#3211) 
* chore: change default factor to compute memory size

* chore: update doc

* chore: update comment in example config

* chore: extract factor to const and update comments

* chore: update comment by cr suggestion

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-01-22 09:03:29 +00:00
Ruihang Xia
8cc7129397 fix: remove __name__ matcher from processed matcher list (#3213) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-01-22 08:50:28 +00:00
Ruihang Xia
278e4c8c30 feat: lazy initialize vector builder on write (#3210) 
* feat: lazy initialize vector builder on write

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* avoid using ConstantVector

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify expression

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/metric-engine/src/engine/create.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-01-22 07:00:04 +00:00
Weny Xu
de13de1454 feat: introduce information schema provider cache (#3208) 
fix: introduce information schema provider cache
 2024-01-22 06:41:39 +00:00
Lei, HUANG
3834ea7422 feat: copy database from (#3164) 
* wip: impl COPY DATABASE FROM parser

* wip: impl copy database from

* wip: add some ut

* wip: add continue_on_error option

* test: add sqlness cases for copy database

* fix: trailing newline

* fix: typo

* fix: some cr comments

* chore: resolve confilicts

* fix: some cr comments
 2024-01-22 06:33:54 +00:00
Wei
e5a8831fa0 refactor: read parquet metadata (#3199) 
* feat: MetadataLoader

* refactor code

* chore: clippy

* chore: cr comment

* chore: add TODO

* chore: cr comment

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: clippy

---------

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-01-21 07:21:29 +00:00
Weny Xu
4278c858f3 feat: make procedure able to return output (#3201) 
* feat: make procedure able to return output

* refactor: change Output to Any
 2024-01-21 06:56:45 +00:00
Weny Xu
986f3bb07d refactor: reduce number of parquet metadata reads and enable reader buffer (#3197) 
refactor: reduce reading parquet metadata times and enable read buffer
 2024-01-19 12:26:38 +00:00
Weny Xu
440cd00ad0 feat(tests-fuzz): add CreateTableExprGenerator & AlterTableExprGenerator (#3182) 
* feat(tests-fuzz): add CreateTableExprGenerator

* refactor: move Column to root of ir mod

* feat: add AlterTableExprGenerator

* feat: add Serialize and Deserialize derive

* chore: refactor the AlterExprGenerator
 2024-01-19 09:39:28 +00:00
dennis zhuang
5e89472b2e feat: adds parse options for SQL parser (#3193) 
* feat: adds parse options for parser and timezone to scan request

* chore: remove timezone in ScanRequest

* feat: remove timezone in parse options and adds type checking to parititon columns

* fix: comment

* chore: apply suggestions

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix: format

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-01-19 09:16:36 +00:00
Zhenchi
2e4c48ae7a fix(index): S3 EntityTooSmall error (#3192) 
* fix(index): S3 `EntityTooSmall` error

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: config api

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-01-19 02:57:07 +00:00