JeremyHi
3201aea360
feat: create tables in batch on prom write ( #3246 )
...
* feat: create tables in batch on prom write
* feat: add logic table ids to log
* fix: miss tabble ids in response
2024-01-26 12:47:24 +00:00
Ruihang Xia
7da8f22cda
fix: IntermediateWriter closes underlying writer twice ( #3248 )
...
* fix: IntermediateWriter closes underlying writer twice
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* close writer manually on error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-26 10:03:50 +00:00
Ning Sun
14b233c486
test: align chrono and some forked test deps to upstream ( #3245 )
...
* test: update chrono and its tests
* chore: switch some deps to upstream version
* test: update timestamp range in sqlness tests
2024-01-26 07:39:51 +00:00
discord9
d5648c18c1
docs: RFC of Dataflow Framework ( #3185 )
...
* docs: RFC of Dataflow Framework
* docs: middle layer&metadata store
* chore: fix typo
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* docs: add figure
* chore: use mermaid instead
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-26 07:13:28 +00:00
Ruihang Xia
b0c3be35fb
feat: don't map semantic type in metric engine ( #3243 )
...
* feat: don't map semantic type in metric engine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove duplicate set_null
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-26 03:50:05 +00:00
Ruihang Xia
5617b284c5
feat: return request outdated error on handling alter ( #3239 )
...
* feat: return request outdated error on handling alter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tonic code mapping
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy, add comment
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix deadloop
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update UT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* address CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: Update log message
* Update src/common/meta/src/ddl/alter_table.rs
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-01-26 03:37:46 +00:00
Weny Xu
f99b08796d
feat: add insert/select generator & translator ( #3240 )
...
* feat: add insert into expr generator & translator
* feat: add select expr generator & translator
* chore: apply suggestions from CR
* fix: fix unit tests
2024-01-26 02:59:17 +00:00
JeremyHi
1fab7ab75a
feat: batch create ddl ( #3194 )
...
* feat: batch ddl to region request
* feat: return table ids
chore: by comment
chore: remove wal_options
chore: create logical tables lock key
feat: get metadata in procedure
* chore: by comment
2024-01-26 02:43:57 +00:00
Yingwen
3fa070a0cc
fix: init parquet reader metrics twice ( #3242 )
2024-01-26 01:54:51 +00:00
Weny Xu
8bade8f8e4
fix: fix create table ddl return incorrect table id ( #3232 )
...
* fix: fix create table ddl return incorrect table id
* refactor: refactor param of Status::done_with_output
2024-01-25 13:58:43 +00:00
Wei
6c2f0c9f53
feat: read metadata from write cache ( #3224 )
...
* feat: read meta from write cache
* test: add case
* chore: cr comment
* chore: clippy
* chore: code style
* feat: put metadata to sst cache
2024-01-25 11:39:41 +00:00
LFC
2d57bf0d2a
ci: adding DOCKER_BUILD_ROOT docker arg for dev build ( #3241 )
...
Update Dockerfile
2024-01-25 09:05:27 +00:00
Weny Xu
673a4bd4ef
feat: add pg create alter table expr translator ( #3206 )
...
* feat: add pg create table expr translator
* feat: add pg alter table expr translator
* refactor: refactor MappedGenerator
* chore: apply suggestions from CR
2024-01-25 08:00:42 +00:00
LFC
fca44098dc
ci: make git's "safe.directory" accept all ( #3234 )
...
* Update Dockerfile
* Update Dockerfile
* Update Dockerfile
2024-01-25 07:32:03 +00:00
Ning Sun
1bc4f25de2
feat: http sql api return schema on empty resultset ( #3237 )
...
* feat: return schema on empty resultset
* refactor: make schema a required field in http output
* test: update integration test and add schema output
2024-01-25 06:44:28 +00:00
Ning Sun
814924f0b6
fix: security update for shlex and h2 ( #3227 )
2024-01-24 13:31:34 +00:00
ZonaHe
b0a8046179
feat: update dashboard to v0.4.7 ( #3229 )
2024-01-24 08:50:21 +00:00
dennis zhuang
7323e9b36f
feat: change Range Query’s default align behavior aware of timezone ( #3219 )
...
* feat: change Range Query’s default align behavior to 1970-01-01 00:00:00 aware of timezone
* test: test with +23:00 timezone
2024-01-24 08:17:57 +00:00
Weny Xu
f82ddc9491
fix: fix MockInstance rebuild issue ( #3218 )
...
* fix: fix MockInstance rebuild issue
* chore: apply suggestions from CR
2024-01-24 07:52:47 +00:00
Ning Sun
1711ad4631
feat: add Arrow IPC output format for http rest api ( #3177 )
...
* feat: add arrow format output for sql api
* refactor: remove unwraps
* test: add test for arrow format
* chore: update cargo toml format
* fix: resolve lint warrnings
* fix: ensure outputs size is one
2024-01-24 06:10:05 +00:00
LFC
f81e37f508
refactor: make http server built flexibly ( #3225 )
...
* refactor: make http server built flexibly
* Apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix: resolve PR comments
* Fix CI.
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-24 03:45:08 +00:00
Weny Xu
d75cf86467
fix: only register region keeper while creating physical table ( #3223 )
...
fix: only register region keeper during create physical table
2024-01-23 09:42:32 +00:00
Weny Xu
26535f577d
feat: enable concurrent write ( #3214 )
...
* feat: enable concurrent write
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-01-23 09:20:12 +00:00
Ruihang Xia
8485c9af33
feat: read column and region info from state cache ( #3222 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-23 09:10:27 +00:00
Weny Xu
19413eb345
refactor!: rename initialize_region_in_background to init_regions_in_background ( #3216 )
...
refactor!: change initialize_region_in_background to init_regions_in_background
2024-01-23 04:33:48 +00:00
Weny Xu
007b63dd9d
fix: fix default value cannot accept negative number ( #3217 )
...
* fix: fix default value cannot accept negative number
* chore: apply suggestions from CR
2024-01-23 03:33:13 +00:00
Weny Xu
364754afa2
feat: add create alter table expr translator ( #3203 )
...
* feat: add create table expr translator
* feat: add alter table expr translator
* refactor: expose mod
* refactor: expr generator
* chore: ignore typos check for lorem_words
* feat: add string map helper functions
* chore: remove unit tests
2024-01-23 02:53:42 +00:00
Ruihang Xia
31787f4bfd
feat!: switch prom remote write to metric engine ( #3198 )
...
* feat: switch prom remote write to metric engine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* read physical table name from url
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove physical table from header
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix merge error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add with_metric_engine option to config remote write behavior
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* check parameter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add specific config param
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* default with_metric_engine to true
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update UT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-01-22 14:48:28 +00:00
dennis zhuang
6a12c27e78
feat: make query be aware of timezone setting ( #3175 )
...
* feat: let TypeConversionRule aware query context timezone setting
* chore: don't optimize explain command
* feat: parse string into timestamp with timezone
* fix: compile error
* chore: check the scalar value type in predicate
* chore: remove mut for engine context
* chore: return none if the scalar value is utf8 in time range predicate
* fix: some fixme
* feat: let Date and DateTime parsing from string value be aware of timezone
* chore: tweak
* test: add datetime from_str test with timezone
* feat: construct function context from query context
* test: add timezone test for to_unixtime and date_format function
* fix: typo
* chore: apply suggestion
* test: adds string with timezone
* chore: apply CR suggestion
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
* chore: apply suggestion
---------
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2024-01-22 14:14:03 +00:00
shuiyisong
2bf4b08a6b
chore: change default factor to compute memory size ( #3211 )
...
* chore: change default factor to compute memory size
* chore: update doc
* chore: update comment in example config
* chore: extract factor to const and update comments
* chore: update comment by cr suggestion
Co-authored-by: dennis zhuang <killme2008@gmail.com >
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-01-22 09:03:29 +00:00
Ruihang Xia
8cc7129397
fix: remove __name__ matcher from processed matcher list ( #3213 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-22 08:50:28 +00:00
Ruihang Xia
278e4c8c30
feat: lazy initialize vector builder on write ( #3210 )
...
* feat: lazy initialize vector builder on write
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* avoid using ConstantVector
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify expression
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/metric-engine/src/engine/create.rs
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2024-01-22 07:00:04 +00:00
Weny Xu
de13de1454
feat: introduce information schema provider cache ( #3208 )
...
fix: introduce information schema provider cache
2024-01-22 06:41:39 +00:00
Lei, HUANG
3834ea7422
feat: copy database from ( #3164 )
...
* wip: impl COPY DATABASE FROM parser
* wip: impl copy database from
* wip: add some ut
* wip: add continue_on_error option
* test: add sqlness cases for copy database
* fix: trailing newline
* fix: typo
* fix: some cr comments
* chore: resolve confilicts
* fix: some cr comments
2024-01-22 06:33:54 +00:00
Weny Xu
966875ee11
chore: bump opendal to v0.44.2 ( #3209 )
2024-01-21 11:47:18 +00:00
Wei
e5a8831fa0
refactor: read parquet metadata ( #3199 )
...
* feat: MetadataLoader
* refactor code
* chore: clippy
* chore: cr comment
* chore: add TODO
* chore: cr comment
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: clippy
---------
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
2024-01-21 07:21:29 +00:00
Weny Xu
4278c858f3
feat: make procedure able to return output ( #3201 )
...
* feat: make procedure able to return output
* refactor: change Output to Any
2024-01-21 06:56:45 +00:00
Weny Xu
986f3bb07d
refactor: reduce number of parquet metadata reads and enable reader buffer ( #3197 )
...
refactor: reduce reading parquet metadata times and enable read buffer
2024-01-19 12:26:38 +00:00
Weny Xu
440cd00ad0
feat(tests-fuzz): add CreateTableExprGenerator & AlterTableExprGenerator ( #3182 )
...
* feat(tests-fuzz): add CreateTableExprGenerator
* refactor: move Column to root of ir mod
* feat: add AlterTableExprGenerator
* feat: add Serialize and Deserialize derive
* chore: refactor the AlterExprGenerator
2024-01-19 09:39:28 +00:00
dennis zhuang
5e89472b2e
feat: adds parse options for SQL parser ( #3193 )
...
* feat: adds parse options for parser and timezone to scan request
* chore: remove timezone in ScanRequest
* feat: remove timezone in parse options and adds type checking to parititon columns
* fix: comment
* chore: apply suggestions
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: format
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-19 09:16:36 +00:00
Yiran
632edd05e5
docs: update SDK links in README.md ( #3156 )
...
* docs: update SDK links in README.md
* chore: typo
2024-01-19 08:54:43 +00:00
Zhenchi
2e4c48ae7a
fix(index): S3 EntityTooSmall error ( #3192 )
...
* fix(index): S3 `EntityTooSmall` error
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: config api
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-01-19 02:57:07 +00:00
Ruihang Xia
cde5a36f5e
feat: precise filter for mito parquet reader ( #3178 )
...
* impl SimpleFilterEvaluator
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* time index and field filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* finish parquet filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove empty Batch
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix fmt
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update metric
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use projected schema from batch
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* correct naming
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unnecessary error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-18 06:59:48 +00:00
niebayes
63205907fb
refactor: introduce common-wal to aggregate wal stuff ( #3171 )
...
* refactor: aggregate wal configs
* refactor: move wal options to common-wal
* chore: slim Cargo.toml
* fix: add missing crates
* fix: format
* chore: update comments
* chore: add testing feature gate for test_util
* fix: apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix: apply suggestions from code review
* fix: compiling
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-18 03:49:37 +00:00
Wei
3d7d2fdb4a
feat: auto config cache size according to memory size ( #3165 )
...
* feat: auto config cache and buffer size according to mem size
* feat: utils
* refactor: add util function to common config
* refactor: check cgroups
* refactor: code
* fix: test
* fix: test
* chore: cr comment
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
* chore: remove default comment
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
2024-01-17 14:35:35 +00:00
LFC
3cfd60e139
refactor: expose region edit in mito engine ( #3179 )
...
* refactor: expose region edit in mito engine
* feat: add a method for editing region directly
* fix: resolve PR comments
* Apply suggestions from code review
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: resolve PR comments
* fix: resolve PR comments
* fix: resolve PR comments
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-17 14:25:08 +00:00
shuiyisong
a29b9f71be
chore: carry metrics in flight metadata from datanode to frontend ( #3113 )
...
* chore: carry metrics in flight metadata from datanode to frontend
* chore: fix typo
* fix: ignore metric flight message on client
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: add cr comment
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: add cr comment
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: update proto
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-17 11:38:03 +00:00
shuiyisong
ae160c2def
fix: change back GREPTIME_DB_HEADER_NAME header key name ( #3184 )
...
fix: change back dbname header key
2024-01-17 09:30:32 +00:00
Ruihang Xia
fbd0197794
refactor: remove TableEngine ( #3181 )
...
* refactor: remove TableEngine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/table/src/table_reference.rs
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
2024-01-17 04:18:37 +00:00
dennis zhuang
204b9433b8
feat: adds date_format function ( #3167 )
...
* feat: adds date_format function
* fix: compile error
* chore: use system timezone for FunctionContext and EvalContext
* test: as_formatted_string
* test: sqlness test
* chore: rename function
2024-01-17 03:24:40 +00:00
