rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-05 21:02:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,469 Commits 105 Branches 225 Tags
20b7f907b2ea28668e422ae1e5744510c7bdfd56
Commit Graph

1469 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruihang Xia
20b7f907b2 fix: promql planner should clear its states on each selector (#2247) 
* reset planner status on selector

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add empty line

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* mask fields to keep ordering

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-25 03:07:44 +00:00
Weny Xu
b13d932e4e fix: fix RegionAliveKeeper does not find the table after restarting (#2249) 2023-08-25 03:05:17 +00:00
Bamboo1
48348aa364 fix: fix test_scheduler_continuous_stop in scheduler (#2252) 
* fix: fix test_scheduler_continuous_stop in scheduler

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: add document annotation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

---------

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>
 2023-08-25 02:59:48 +00:00
Zhenchi
9ce73e7ca1 refactor(frontend): TableScan instead of scan_to_stream for COPY TO (#2244) 
* refactor(frontend): TableScan instead of `scan_to_stream` for `COPY TO`

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-24 12:46:54 +00:00
Ruihang Xia
b633a16667 feat: apply rewriter to subquery exprs (#2245) 
* apply rewriter to subquery exprs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* workaround for datafusion's check

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change time index type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-24 11:48:04 +00:00
Zhenchi
0a6ab2a287 refactor(script): not to call scan_to_stream on table (#2241) 
* refactor(script): not to call `scan_to_stream` on table

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: build plan via LogicalPlanBuilder

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-24 08:10:07 +00:00
JeremyHi
7746e5b172 feat: dist row inserter (#2231) 
* feat: fronend row inserter

* feat: row splitter

chore: row splitter's unit test

* feat: RowDistInserter

* feat: make influxdb line protocol using row-based protocol

* Update src/partition/src/row_splitter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by review comment

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* chore: by comment

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-08-24 06:58:05 +00:00
Weny Xu
a7e0e2330e fix: invalidate cache after altering (#2239) 2023-08-24 03:56:17 +00:00
Lei, HUANG
19d2d77b41 fix: parse large timestamp (#2185) 
* feat: support parsing large timestamp values

* chore: update sqlness tests

* fix: tests

* fix: allow larger window
 2023-08-24 03:52:15 +00:00
Yingwen
4ee1034012 feat(mito): merge reader for mito2 (#2210) 
* feat: Implement slice and first/last timestamp for Batch

* feat(mito): implements sort/concat for Batch

* chore: fix typo

* chore: remove comments

* feat: sort and dedup

* test: test batch operations

* chore: cast enum to test op type

* test: test filter related api

* sytle: fix clippy

* feat: implement Node and CompareFirst

* feat: merge reader wip

* feat: merge wip

* feat: use batch's operation to sort and dedup

* feat: implement BatchReader for MergeReader

* feat: simplify codes

* test: test merge reader

* refactor: use test util to create batch

* refactor: remove unused imports

* feat: update comment

* chore: remove metadata() from Source

* chroe: update comment

* feat: source supports batch iterator

* chore: update comment
 2023-08-24 03:37:51 +00:00
Ruihang Xia
e5ba3d1708 feat: rewrite the dist analyzer (#2238) 
* it works!

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add documents

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unstable timestamp from sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename rewriter struct

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-24 03:29:08 +00:00
dennis zhuang
8b1f4eb958 feat: types sqlness tests (#2073) 
* feat: timestamp types sqlness tests

* feat: adds timestamp tests

* test: add string tests

* test: comment a case in timestamp

* test: add float type tests

* chore: adds TODO

* feat: set TZ=UTC for sqlness test
 2023-08-24 03:26:19 +00:00
discord9
eca7e87129 chore: try from value (#2236) 
* chore: try from value

* chore: add TryFromValueError variant
 2023-08-24 02:44:13 +00:00
Weny Xu
beb92ba1d2 refactor: use table id instead of table ident (#2233) 2023-08-23 13:28:08 +00:00
Lei, HUANG
fdb5ad23bf refactor: use Batch::sort_and_dedup instead of Values::sort_in_place (#2235) 2023-08-23 08:56:49 +00:00
Ruihang Xia
d581688fd2 fix: dist planner has wrong behavior in table with multiple partitions (#2237) 
* fix: dist planner has wrong behavior in table with multiple partitions

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update tests/cases/distributed/explain/multi_partitions.sql

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-23 08:32:20 +00:00
Bamboo1
4dbc32f532 refactor: remove associate type in scheduler to simplify it #2153 (#2194) 
* feature: add a simple scheduler using flume

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: only use a sender rather clone many senders

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: use select to avoid loop

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* feat: add parameters in new function to build the flume capacity and number of receivers

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* test: add countdownlatch test concurrency

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* test: add barrier replacing countdownlatch to test concurrency and add wait all tasks finished in stop

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: add some document annotation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: add license header

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: add Cargo.lock

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: Cargo.toml format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: delete println in test

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* feat: add error handle

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: fix error handle and add test scheduler stop

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: spelling mistake

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: wait all tasks finished

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: add todo which need wrap Future returned by send_async

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* test: remove unnessary sleep in test

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: resolve some conflicts

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: resolve conversation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* feat: modify the function of schedule to synchronize and drop sender after stopping scheduler

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

---------

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>
 2023-08-23 06:28:00 +00:00
Zhenchi
af95e46512 refactor(table): eliminate calls to DistTable.delete (#2225) 
* refactor(table): eliminate calls to DistTable.delete

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-23 02:33:48 +00:00
Weny Xu
d81ddd8879 chore: fix clippy (#2232) 2023-08-23 02:24:29 +00:00
Ning Sun
88247e4284 fix!: resolve residual issues with removing prometheus port (#2227) 
* fix: resolve residual issues when removing prometheus port

* fix: remove prometheus from sample config as well
 2023-08-23 01:49:11 +00:00
Ruihang Xia
18250c4803 feat: implement Flight and gRPC services for RegionServer (#2226) 
* extract FlightCraft trait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split service handler in GrpcServer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* left grpc server implement

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start region server if configured

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-22 13:30:09 +00:00
dennis zhuang
18fa0e01ed feat: remove checkpoint_on_startup (#2228) 
feat: update flushed manifest version when it is larger
 2023-08-22 13:09:34 +00:00
Yingwen
cc3e198975 feat(mito): Implement operations like concat and sort for Batch (#2203) 
* feat: Implement slice and first/last timestamp for Batch

* feat(mito): implements sort/concat for Batch

* chore: fix typo

* chore: remove comments

* feat: sort and dedup

* test: test batch operations

* chore: cast enum to test op type

* test: test filter related api

* sytle: fix clippy

* docs: comment for slice

* chore: address CR comment

Don't return Option in get_timestamp()/get_sequence()
 2023-08-22 12:03:02 +00:00
Yingwen
cd3755c615 feat(mito): Support handling RegionWriteRequest (#2218) 
* feat: convert region request to worker write request

* chore: remove unused codes

* test: fix tests compiler errors

* chore: remove create/close/open request from worker requests

* chore: add comment

* chore: fix typo
 2023-08-22 11:16:00 +00:00
Lei, HUANG
be1e13c713 feat(mito2): time series memtable (#2208) 
* feat: time series memtable

* feat: add some test

* fix: some clippy warnings

* chore: some rustdoc

* refactor: test

* fix: remove useless functions

* feat: add config for TimeSeriesMemtable

* chore: some optimize

* refactor: remove bucketing

* refactor: avoid cloing RegionMetadataRef across all Series; make initial_builder_capacity a const; sort batch only by timestamp and sequence
 2023-08-22 08:40:46 +00:00
Zhenchi
cb3561f3b3 refactor(table): eliminate calls to DistTable.insert (#2219) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-22 06:15:02 +00:00
Niwaka
b3b43fe1c3 fix: table options can't be found in distributed mode (#2209) 
* fix: table options can't be found in distributed mode

* refactor: use iterator for regions_numbers

* chore: remove TODO
 2023-08-22 03:53:56 +00:00
WU Jingdi
b411769de6 feat: Implement a basical range select query (#2138) 
* feat: Implement a basical range select query

* chore: support any timestamp type & CR fix
 2023-08-22 03:07:14 +00:00
niebayes
e5f4ca2dab feat: streaming do_get (#2171) 
* feat: rewrite do_get for streaming get flight data

* feat: rewrite do_get call stack but leave the async stream adapter not modified yet

* feat: rewrite the async stream adapter to accept greptime record batch stream

* fix: resolve some PR comments

* feat: rewrite tests to adapt to the streaming do_get

* feat: add unit tests for streaming do_get

* feat: rewrite timer metric of merge scan

* remove unhelpful unit tests for streaming do_get

* add a new metric timer for merge scan and fix some test errors

* rewrite mysql writer to write query results in a streaming manner

* fix: fix fmt errors

* fix: rewrite sqlness runner to take into account the streaming do_get

* fix: fix toml format errors

* fix: resolve some PR comments

* fix: resolve some PR comments

* fix: refactor do_get to increase readability

* fix: refactor mysql try_write_one to increase readability
 2023-08-22 02:54:05 +00:00
Weny Xu
5b7b2cf77d fix: fix ddl client can not update leader addr (#2205) 
* fix: fix ddl client can not update leader addr

* chore: apply suggestions from CR

* feat: add message to context

* fix: only retry if unavailable or deadline exceeded

* chore: apply suggestions from CR
 2023-08-21 13:57:29 +00:00
shuiyisong
9352649f22 chore: add table region key to delete in upgrade tool (#2214) 2023-08-21 08:16:10 +00:00
shuiyisong
c5f507c20e fix: add user_info extension to prom_store handler (#2212) 
chore: add user_info extention to prom_store auth
 2023-08-21 04:55:34 +00:00
JeremyHi
033b650d0d feat: row write protocol (#2189) 
* feat: datanode's row insrter

* refactor: ExprFactory

* feat: row inserter in standalon mode

* chore: minor refactor

* feat: influxdb line protocol's row protocol

* chore: minor refactor

* improve: avoid to use too many string

* no longer async

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: do not check empty data

* chore: by review comment

* chore: by comment

* chore: by review comment

* chore: by review comment

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
v0.4.0-nightly-20230821 		2023-08-19 13:08:44 +00:00
dennis zhuang
272f649b22 fix: some TODO in sqlness cases and refactor meta-client error (#2207) 
* fix: some TODO in sqlness cases and refactor meta-client error

* fix: delete tests/cases/standalone/alter/drop_col_not_null_next.output
 2023-08-18 10:09:11 +00:00
Ruihang Xia
3150f4b22e fix: specify input ordering and distribution for prom plan (#2204) 
* fix: specify input ordering and distribution for prom plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-18 09:45:46 +00:00
Weny Xu
e1ce1d86a1 refactor: unite key serialization method (#2195) 2023-08-18 09:42:19 +00:00
ZonaHe
b8595e1960 feat: update dashboard to v0.3.1 (#2192) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-08-18 09:42:18 +00:00
shuiyisong
61e6656fea fix: auth in prometheus gateway service (#2206) 
* fix: auth in prometheus gateway service

* chore: remove unused code

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-18 09:41:38 +00:00
Ruihang Xia
1bbec75f5b fix: skip partition clause in show create table (#2200) 
* fix: skip partition clause in show create table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-18 09:10:31 +00:00
Zhenchi
8d6a2d0b59 refactor: apply numbers to ThinTable (#2202) 
* refactor: apply numbers to `ThinTable`

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: tiny polish

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: unused import

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-18 03:11:37 +00:00
Weny Xu
177036475a fix: support to copy from parquet with typecast (#2201) 2023-08-18 03:09:54 +00:00
Zhenchi
87a730658a refactor: add ThinTable to proxy tables from infoschema (#2193) 
* refactor: add thin table to proxy tables in info_schema

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix(catalog): fix typo in DataSourceAdapter struct name

* fix: remove redundant Send + Sync

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor(catalog): rename DataSourceAdapter to InformationTableDataSource

* feat(catalog): add ThinTableAdapter for adapting ThinTable to Table interface

* rebase develop

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: default impl for table_type of InformationTable

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: filter_pushdown as table field

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: remove explicit type declaration

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-17 15:19:14 +00:00
JeremyHi
b67e5bbf70 fix: invalid err msg (#2196) 2023-08-17 11:12:35 +00:00
Ruihang Xia
4aaf6aa51b feat: implement query API for RegionServer (#2197) 
* some initial change

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl dummy structs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* decode and send logical plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement table scan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add some comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-17 11:02:31 +00:00
Weny Xu
6e6ff5a606 refactor: update table metadata in single txn (#2172) 
* refactor: table-metadata-manager

* feat: remove comparing when deleting metadata

* fix: fix comment typos

* chore: apply suggestions from CR

* test: add tests for updating DatanodeTable

* fix: fix clippy

* chore: apply suggestions from CR

* refactor: improve update table route tests

* refactor: return Txn instead of TxnRequest

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor: update table metadata in single txn

* feat: check table exists before drop table executing

* test: add tests for table metadata manager

* refactor: remove table region manager

* chore: apply suggestions from CR

* feat: add bench program

* chore: apply suggestions from CR
 2023-08-17 06:29:19 +00:00
Yingwen
4ba12155fe feat(mito): Implement SST format for mito2 (#2178) 
* chore: update comment

* feat: stream writer takes arrow's types

* feat: Define Batch struct

* feat: arrow_schema_to_store

* refactor: rename

* feat: write parquet in new format with tsids

* feat: reader support projection

* feat: Impl read compat

* refactor: rename SchemaCompat to CompatRecordBatch

* feat: changing sst format

* feat: make it compile

* feat: remove tsid and some structs

* feat: from_sst_record_batch wip

* chore: push array

* chore: wip

* feat: decode batches from RecordBatch

* feat: reader converts record batches

* feat: remove compat mod

* chore: remove some codes

* feat: sort fields by column id

* test: test to_sst_arrow_schema

* feat: do not sort fields

* test: more test helpers

* feat: simplify projection

* fix: projection indices is incorrect

* refactor: define write/read format

* test: test write format

* test: test projection

* test: test convert record batch

* feat: remove unused errors

* refactor: wrap get_field_batch_columns

* chore: clippy

* chore: fix clippy

* feat: build arrow schema from region meta in ReadFormat

* feat: initialize the parquet reader at `build()`

* chore: fix typo
 2023-08-17 06:25:50 +00:00
Weny Xu
832e5dcfd7 chore: remove allow-unused (#2184) 2023-08-17 03:15:12 +00:00
shuiyisong
d45ee8b42a chore: fix collect region stat on non-base table (#2190) 2023-08-17 02:13:49 +00:00
JeremyHi
6cd7319d67 refactor: grpc insert (#2188) 
* feat: interval type for row protocol

* feat: minor refactor grpc insert

* Update src/common/grpc-expr/src/util.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: by comment

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-16 11:25:25 +00:00
Yingwen
bb062003ef ci: fallback to run_id to avoid cancelling other jobs (#2186) 
ci: fallback to run id to avoid cancelling other jobs
 2023-08-16 09:24:17 +00:00