rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-06 21:32:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,512 Commits 105 Branches 226 Tags
fa542f6e935d2c5850f9b653e5cde94283af1425
Commit Graph

1512 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
JeremyHi
fa542f6e93 feat: new heartbeat (#2299) 2023-09-12 07:57:15 -05:00
Zhenchi
d6c82867d5 refactor: remove the most Table impls (#2274) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Lei, HUANG
86d56f71ef fix: flume bug (#2298) 
fix: flume
 2023-09-12 07:57:15 -05:00
Zhenchi
b42d343ae6 feat(frontend): unify column inserter and row inserter (#2293) 
* refactor: unify inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(frontend): unify column inserter and row inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: remove redundant clone

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: move empty check ahead

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add more logs

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: leading license

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* adjust indent

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
Yingwen
365e557e7a feat(mito): Integrate access layer and file purger to region (#2296) 
* feat: alias SchedulerRef and clean scheduler on drop

* feat: add scheduler to workers

* feat: use access layer to read write sst

* feat: add purger to region

* refactor: allow getting region_dir from AccessLayer

* feat: add scheduler to FlushScheduler

* feat: getter for object store

* chore: fix typo

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
46d171d341 chore: bump greptime-proto to replace region_dir (#2290) 
chore: bump greptime-proto to replace region_dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
718246ea1a feat: implement heartbeat for region server (#2279) 
* retrieve region stats from region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat handler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start datanode with region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable non-unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat task

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
JeremyHi
58d07e0e62 feat: v04 rm unused exprs (#2285) 
* feat: rm compact and flush exprs

* refactor: continue to rm compact and flush
 2023-09-12 07:57:15 -05:00
dennis zhuang
db89235474 feat: only allow timestamp type as time index (#2281) 
* feat: only allow timestamp data type as time index

* test: update sqltest cases, todo: need some fixes

* fix: sqlness tests

* fix: forgot adding back cte test

* chore: style
 2023-09-12 07:57:15 -05:00
shuiyisong
6e593401f7 refactor: collecting memory usage during scan (#2353) 
* chore: try custom metrics

* chore: fix header

* chore: minor change
 2023-09-12 15:52:57 +08:00
Yingwen
466fbaca5d fix: panic in try_into_vector() (#2351) 2023-09-11 19:06:31 +08:00
ZonaHe
de966af83b feat: update dashboard to v0.3.3 (#2339) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-09-06 19:19:56 +08:00
Zou Wei
b8c50d00aa feat: sqlness test for interval type (#2265) 
* feat: add integration-test for interval type.

* chore: add two cases.

* chore: cr

* chore: Field to Column
 2023-09-04 14:30:48 +08:00
Ruihang Xia
a12ee5cab8 fix: qualify inputs on handling join in promql (#2297) 
* add qualifier to join inputs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add one more case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-01 11:51:34 +08:00
ZonaHe
a0d15b489a feat: update dashboard to v0.3.2 (#2295) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-08-31 22:05:00 +08:00
shuiyisong
baa372520d fix: json compatibility to null (#2287) 
* fix: existing null value for schema name value

* chore: fix null check

* fix: change catalognamevalue and schemanamevalue to option

* fix: fix null case
 2023-08-31 14:21:10 +08:00
shuiyisong
5df4d44761 feat: schema level opts (#2283) 
* chore: update proto

* chore: add try from for schema name value

* chore: merge schema opts to table opts while creating table

* chore: use table ttl opts first

* chore: add unit test

* chore: update proto version
 2023-08-30 08:11:08 +00:00
Weny Xu
8e9f2ffce4 fix: skip procedure if target route is not found (#2277) 
* fix: skip procedure if target route is not found

* chore: apply suggestions from CR
 2023-08-30 06:59:50 +00:00
Weny Xu
1101e7bb18 fix: deregister table after keeper closes table (#2278) 
* fix: deregister table after keeper closes table

* chore: apply suggestions from CR
 2023-08-30 03:43:04 +00:00
zyy17
5fbc941023 ci: upload the latest artifacts to 'latest/' directory of S3 bucket in scheduled and formal release (#2276) 
Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2023-08-29 09:00:45 +00:00
Bamboo1
68600a2cf9 feat(mito2): add file purger and cooperate with scheduler to purge sst files (#2251) 
* feat: add file purger and use scheduler

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* feat: print some information about handling error message

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: resolve conversion

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: resolve conversation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: resolve conflicting files

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

---------

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>
 2023-08-29 07:55:03 +00:00
Yingwen
805f254d15 feat(mito): Flush framework for mito2 (#2262) 
* feat: write buffer manager

* feat: skeleton

* feat: add flush logic to write path

* feat: add methods to memtable trait

* feat: freeze memtable

* feat: define flush task

* feat: schedule_flush wip

* feat: adding pending requests/tasks

* feat: separate ddl request and background request

* feat: Remove RegionTask and RequestBody

* feat: handle flush related requests

* feat: make tests pass

* style: fix clippy

* docs: update comment

* refactor: rename background requests

* feat: replace Option<RegionWriteCtx> with an enum MaybeStalling
 2023-08-29 07:13:15 +00:00
Zhenchi
2a6c830ca7 refactor(table): remove Table impl for system (#2270) 
* refactor(table): remove Table impl for system

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format & import

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-29 03:43:43 +00:00
Weny Xu
22dea02485 fix: use RegionId region number instead (#2273) 2023-08-29 02:52:24 +00:00
LFC
ef75e8f7c3 feat: create distributed Mito2 table (#2246) 
* feat: create distributed Mito2 table

* rebase develop
 2023-08-28 12:07:52 +00:00
Weny Xu
71fc3c42d9 fix: open region does not register catalog/schema (#2271) 
* fix: open region does not register catalog/schema

* fix: fix ci
 2023-08-28 12:06:10 +00:00
JeremyHi
c02ac36ce8 feat: avoid confusion in desc table (#2272) 
feat: Field to Column to aviod confusion in DESC TABLE
 2023-08-28 11:50:33 +00:00
Lei, HUANG
c112b9a763 feat(mito2): WAL replay (#2264) 
* feat: replay memtable when opening table

* test: region replay

* refactor: save logstore in TestEnv

* fix: some cr comments

* chore: rebase develop

* chore: update last entry id during replay
 2023-08-28 11:45:23 +00:00
Weny Xu
96fd17aa0a fix: fix typoes (#2268) 2023-08-28 09:26:00 +00:00
Ruihang Xia
6b8cf0bbf0 feat: impl region engine for mito (#2269) 
* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* convert request

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* import result convertor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename symbols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-28 09:24:12 +00:00
Yingwen
e2522dff21 feat(mito): Skeleton for scanning a region (#2230) 
* feat: define stream builder

* feat: scan region wip

* feat: create SeqScan in ScanRegion

* feat: scanner

* feat: engine handles scan request

* feat: map projection index to column id

* feat: Impl record batch stream

* refactor: change BatchConverter to ProjectionMapper

* feat: add column_ids to mapper

* feat: implement SeqScan::build()

* chore: fix typo

* docs: add mermaid for ScanRegion

* style: fix clippy

* test: fix record batch test

* fix: update sequence and entry id

* test: test query

* feat: address CR comment

* chore: address CR comments

* chore: Update src/mito2/src/read/scan_region.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-08-28 06:59:31 +00:00
LFC
d8f851bef2 fix: keep region failover state not changed upon failure (#2261) 2023-08-28 04:40:47 +00:00
JeremyHi
63b22b2403 feat: prometheus row inserter (#2263) 
* feat: prometheus row inserter

* chore: add unit test

* refactor: to row_insert_requests

* chore: typo

* chore: alloc row by TableData

* chore: by review comment
 2023-08-28 03:22:23 +00:00
Weny Xu
c56f5e39cd refactor: set default metasrv procedure retry times to 12 (#2242) v0.4.0-nightly-20230828 2023-08-26 07:41:15 +00:00
Weny Xu
7ff200c0fa fix: align region numbers to real regions (#2257) 2023-08-25 11:48:58 +00:00
dennis zhuang
5160838d04 chore: change version to 0.4.0-nightly (#2258) 
* chore: change version to 0.4.0-nightly

* fix: test
 2023-08-25 09:44:39 +00:00
shuiyisong
f16f58266e refactor: query_ctx from http middleware (#2253) 
* chore: change userinfo to query_ctx in http handler

* chore: minor change

* chore: move prometheus http to http mod

* chore: fix uni test:

* chore: add back schema check

* chore: minor change

* chore: remove clone
 2023-08-25 09:36:33 +00:00
Ruihang Xia
8d446ed741 fix: quote ident on rendered SQL (#2248) 
* fix: quote ident on rendered SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* read quote style from query context

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-25 07:25:21 +00:00
JeremyHi
de1daec680 feat: upgrade desc table output (#2256) 2023-08-25 06:52:22 +00:00
Zhenchi
9d87c8b6de refactor(table): cleanup dist table (#2255) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-25 06:37:39 +00:00
Lei, HUANG
6bf260a05c chore: write to mito2 (#2250) 
* chore: write to mito2

* fix: clippy

* feat: brdige memtable

* chore: rebase develop
 2023-08-25 06:18:42 +00:00
WU Jingdi
15912afd96 fix: the inconsistent order of input/output in range select (#2229) 
* fix: the inconsistent order of input/output in range select

* chore: apply CR
 2023-08-25 04:12:59 +00:00
Lei, HUANG
dbe0e95f2f feat(mito2): concat and projection (#2243) 
* refactor: use arrow::compute::concat instead of push values to vector builders

* feat: support projection

* refactor: remove sequence

* refactor: concatenate

* fix: series must not be empty

* refactor: projection
 2023-08-25 03:25:27 +00:00
Ruihang Xia
20b7f907b2 fix: promql planner should clear its states on each selector (#2247) 
* reset planner status on selector

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add empty line

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* mask fields to keep ordering

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-25 03:07:44 +00:00
Weny Xu
b13d932e4e fix: fix RegionAliveKeeper does not find the table after restarting (#2249) 2023-08-25 03:05:17 +00:00
Bamboo1
48348aa364 fix: fix test_scheduler_continuous_stop in scheduler (#2252) 
* fix: fix test_scheduler_continuous_stop in scheduler

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: add document annotation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

---------

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>
 2023-08-25 02:59:48 +00:00
Zhenchi
9ce73e7ca1 refactor(frontend): TableScan instead of scan_to_stream for COPY TO (#2244) 
* refactor(frontend): TableScan instead of `scan_to_stream` for `COPY TO`

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-24 12:46:54 +00:00
Ruihang Xia
b633a16667 feat: apply rewriter to subquery exprs (#2245) 
* apply rewriter to subquery exprs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* workaround for datafusion's check

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change time index type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-24 11:48:04 +00:00
Zhenchi
0a6ab2a287 refactor(script): not to call scan_to_stream on table (#2241) 
* refactor(script): not to call `scan_to_stream` on table

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: build plan via LogicalPlanBuilder

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-24 08:10:07 +00:00
JeremyHi
7746e5b172 feat: dist row inserter (#2231) 
* feat: fronend row inserter

* feat: row splitter

chore: row splitter's unit test

* feat: RowDistInserter

* feat: make influxdb line protocol using row-based protocol

* Update src/partition/src/row_splitter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by review comment

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* chore: by comment

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-08-24 06:58:05 +00:00