rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-03 20:02:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,524 Commits 103 Branches 225 Tags
271f80daad54aea53aee81afbeff577ecfd5cd04
Commit Graph

1524 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weny Xu
271f80daad fix: LoadBase Selector cannot follow the region distribution rules (#2259) 
* fix: LoadBase Selector cannot follow the region distribution rules

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
Lei, HUANG
36231a5d50 feat(mito2): add alloc_tracker for memtable (#2266) 
* feat: add alloc_tracker for memtable

* chore: integrate WriteBufferManager
 2023-09-12 07:57:15 -05:00
JeremyHi
a7fa40e16d fix: filter out outdated heartbeat (#2303) 
* fix: filter out outdated heartbeat, #1707

* feat: reorder handlers

* refactor: disableXXX to enableXXX

* feat: make full use of region leases to facilitate failover

* chore: minor refactor

* chore: by comment

* feat: logging on inactive/active
 2023-09-12 07:57:15 -05:00
Yingwen
648b2ae293 feat(mito): Flush region (#2291) 
* chore: call handle_flush_request

* feat: alias SchedulerRef and clean scheduler on drop

* feat: add scheduler to workers

* feat: remove RegionMemtableStats

* feat: pick regions to flush

* feat: add more fields to region flush task

* feat: smallvec workspace dep

* feat: Use list to hold immutable memtables

* feat: flush job wip

* feat: use access layer to read write sst

* feat: flush memtables to l0

* feat: write manifest

* feat: schedule next flush on success

* feat: schedule flush on success and failure

* feat: add purger to region

* feat: apply edit after flush

* feat: collect stats for SSTs

* feat: manual flush

* test: test flush and fix manifest test

* feat: remove flush scheduler job limit

* fix: typo

* style: clippy

* feat: clean flushed files on failure

* chore: address CR comment

* refactor: Use put_rows

* feat: Clean flush scheduler on drop

* feat: remove region flush status on drop and close

* chore: address CR comment
 2023-09-12 07:57:15 -05:00
Weny Xu
fa5e3b94d3 refactor: refactor ddl procedure (#2304) 2023-09-12 07:57:15 -05:00
Weny Xu
4818887e38 refactor: refactor DistInstance (#2305) 2023-09-12 07:57:15 -05:00
Ruihang Xia
eddff17523 feat: drop region in mito2 (#2286) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check in opening region

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/mito2/src/worker/handle_drop.rs

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* remove file in order

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix remove logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use scan to list files

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-09-12 07:57:15 -05:00
Weny Xu
c839ed271c refactor: refactor: ddl context (#2301) 
* refactor: refactor: ddl context

* refactor: remove unused code

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
JeremyHi
4d2cae4174 refactor: inactive node manager (#2300) 
refactor: use region_id instead of table&region_num in InactiveNodeManager
 2023-09-12 07:57:15 -05:00
Yingwen
b234733c61 feat(mito): Support deleting rows in mito2 (#2275) 
* feat: check delete request

* test: test delete and overwrite
 2023-09-12 07:57:15 -05:00
Lei, HUANG
9691d19601 feat: impl kv backend for raft engine (#2280) 
* feat: kv backend on raft-engine

* feat: raft-engine kvbackend

* fix: toml

* fix: some review comments

* chore: optimize delete

* fix: lift lock in batch_delete
 2023-09-12 07:57:15 -05:00
LFC
ff3881f0e1 feat: drop distributed Mito2 table (#2260) 
* feat: drop distributed Mito2 table

* rebase develop

* fix: resolve PR comments

* fix: resolve PR comments
 2023-09-12 07:57:15 -05:00
JeremyHi
fa542f6e93 feat: new heartbeat (#2299) 2023-09-12 07:57:15 -05:00
Zhenchi
d6c82867d5 refactor: remove the most Table impls (#2274) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Lei, HUANG
86d56f71ef fix: flume bug (#2298) 
fix: flume
 2023-09-12 07:57:15 -05:00
Zhenchi
b42d343ae6 feat(frontend): unify column inserter and row inserter (#2293) 
* refactor: unify inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(frontend): unify column inserter and row inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: remove redundant clone

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: move empty check ahead

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add more logs

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: leading license

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* adjust indent

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
Yingwen
365e557e7a feat(mito): Integrate access layer and file purger to region (#2296) 
* feat: alias SchedulerRef and clean scheduler on drop

* feat: add scheduler to workers

* feat: use access layer to read write sst

* feat: add purger to region

* refactor: allow getting region_dir from AccessLayer

* feat: add scheduler to FlushScheduler

* feat: getter for object store

* chore: fix typo

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
46d171d341 chore: bump greptime-proto to replace region_dir (#2290) 
chore: bump greptime-proto to replace region_dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
718246ea1a feat: implement heartbeat for region server (#2279) 
* retrieve region stats from region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat handler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start datanode with region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable non-unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat task

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
JeremyHi
58d07e0e62 feat: v04 rm unused exprs (#2285) 
* feat: rm compact and flush exprs

* refactor: continue to rm compact and flush
 2023-09-12 07:57:15 -05:00
dennis zhuang
db89235474 feat: only allow timestamp type as time index (#2281) 
* feat: only allow timestamp data type as time index

* test: update sqltest cases, todo: need some fixes

* fix: sqlness tests

* fix: forgot adding back cte test

* chore: style
 2023-09-12 07:57:15 -05:00
shuiyisong
6e593401f7 refactor: collecting memory usage during scan (#2353) 
* chore: try custom metrics

* chore: fix header

* chore: minor change
 2023-09-12 15:52:57 +08:00
Yingwen
466fbaca5d fix: panic in try_into_vector() (#2351) 2023-09-11 19:06:31 +08:00
ZonaHe
de966af83b feat: update dashboard to v0.3.3 (#2339) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-09-06 19:19:56 +08:00
Zou Wei
b8c50d00aa feat: sqlness test for interval type (#2265) 
* feat: add integration-test for interval type.

* chore: add two cases.

* chore: cr

* chore: Field to Column
 2023-09-04 14:30:48 +08:00
Ruihang Xia
a12ee5cab8 fix: qualify inputs on handling join in promql (#2297) 
* add qualifier to join inputs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add one more case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-01 11:51:34 +08:00
ZonaHe
a0d15b489a feat: update dashboard to v0.3.2 (#2295) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-08-31 22:05:00 +08:00
shuiyisong
baa372520d fix: json compatibility to null (#2287) 
* fix: existing null value for schema name value

* chore: fix null check

* fix: change catalognamevalue and schemanamevalue to option

* fix: fix null case
 2023-08-31 14:21:10 +08:00
shuiyisong
5df4d44761 feat: schema level opts (#2283) 
* chore: update proto

* chore: add try from for schema name value

* chore: merge schema opts to table opts while creating table

* chore: use table ttl opts first

* chore: add unit test

* chore: update proto version
 2023-08-30 08:11:08 +00:00
Weny Xu
8e9f2ffce4 fix: skip procedure if target route is not found (#2277) 
* fix: skip procedure if target route is not found

* chore: apply suggestions from CR
 2023-08-30 06:59:50 +00:00
Weny Xu
1101e7bb18 fix: deregister table after keeper closes table (#2278) 
* fix: deregister table after keeper closes table

* chore: apply suggestions from CR
 2023-08-30 03:43:04 +00:00
zyy17
5fbc941023 ci: upload the latest artifacts to 'latest/' directory of S3 bucket in scheduled and formal release (#2276) 
Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2023-08-29 09:00:45 +00:00
Bamboo1
68600a2cf9 feat(mito2): add file purger and cooperate with scheduler to purge sst files (#2251) 
* feat: add file purger and use scheduler

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* feat: print some information about handling error message

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: resolve conversion

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: resolve conversation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: resolve conflicting files

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: code format

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

---------

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>
 2023-08-29 07:55:03 +00:00
Yingwen
805f254d15 feat(mito): Flush framework for mito2 (#2262) 
* feat: write buffer manager

* feat: skeleton

* feat: add flush logic to write path

* feat: add methods to memtable trait

* feat: freeze memtable

* feat: define flush task

* feat: schedule_flush wip

* feat: adding pending requests/tasks

* feat: separate ddl request and background request

* feat: Remove RegionTask and RequestBody

* feat: handle flush related requests

* feat: make tests pass

* style: fix clippy

* docs: update comment

* refactor: rename background requests

* feat: replace Option<RegionWriteCtx> with an enum MaybeStalling
 2023-08-29 07:13:15 +00:00
Zhenchi
2a6c830ca7 refactor(table): remove Table impl for system (#2270) 
* refactor(table): remove Table impl for system

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format & import

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-29 03:43:43 +00:00
Weny Xu
22dea02485 fix: use RegionId region number instead (#2273) 2023-08-29 02:52:24 +00:00
LFC
ef75e8f7c3 feat: create distributed Mito2 table (#2246) 
* feat: create distributed Mito2 table

* rebase develop
 2023-08-28 12:07:52 +00:00
Weny Xu
71fc3c42d9 fix: open region does not register catalog/schema (#2271) 
* fix: open region does not register catalog/schema

* fix: fix ci
 2023-08-28 12:06:10 +00:00
JeremyHi
c02ac36ce8 feat: avoid confusion in desc table (#2272) 
feat: Field to Column to aviod confusion in DESC TABLE
 2023-08-28 11:50:33 +00:00
Lei, HUANG
c112b9a763 feat(mito2): WAL replay (#2264) 
* feat: replay memtable when opening table

* test: region replay

* refactor: save logstore in TestEnv

* fix: some cr comments

* chore: rebase develop

* chore: update last entry id during replay
 2023-08-28 11:45:23 +00:00
Weny Xu
96fd17aa0a fix: fix typoes (#2268) 2023-08-28 09:26:00 +00:00
Ruihang Xia
6b8cf0bbf0 feat: impl region engine for mito (#2269) 
* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* convert request

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* import result convertor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename symbols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-28 09:24:12 +00:00
Yingwen
e2522dff21 feat(mito): Skeleton for scanning a region (#2230) 
* feat: define stream builder

* feat: scan region wip

* feat: create SeqScan in ScanRegion

* feat: scanner

* feat: engine handles scan request

* feat: map projection index to column id

* feat: Impl record batch stream

* refactor: change BatchConverter to ProjectionMapper

* feat: add column_ids to mapper

* feat: implement SeqScan::build()

* chore: fix typo

* docs: add mermaid for ScanRegion

* style: fix clippy

* test: fix record batch test

* fix: update sequence and entry id

* test: test query

* feat: address CR comment

* chore: address CR comments

* chore: Update src/mito2/src/read/scan_region.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-08-28 06:59:31 +00:00
LFC
d8f851bef2 fix: keep region failover state not changed upon failure (#2261) 2023-08-28 04:40:47 +00:00
JeremyHi
63b22b2403 feat: prometheus row inserter (#2263) 
* feat: prometheus row inserter

* chore: add unit test

* refactor: to row_insert_requests

* chore: typo

* chore: alloc row by TableData

* chore: by review comment
 2023-08-28 03:22:23 +00:00
Weny Xu
c56f5e39cd refactor: set default metasrv procedure retry times to 12 (#2242) v0.4.0-nightly-20230828 2023-08-26 07:41:15 +00:00
Weny Xu
7ff200c0fa fix: align region numbers to real regions (#2257) 2023-08-25 11:48:58 +00:00
dennis zhuang
5160838d04 chore: change version to 0.4.0-nightly (#2258) 
* chore: change version to 0.4.0-nightly

* fix: test
 2023-08-25 09:44:39 +00:00
shuiyisong
f16f58266e refactor: query_ctx from http middleware (#2253) 
* chore: change userinfo to query_ctx in http handler

* chore: minor change

* chore: move prometheus http to http mod

* chore: fix uni test:

* chore: add back schema check

* chore: minor change

* chore: remove clone
 2023-08-25 09:36:33 +00:00
Ruihang Xia
8d446ed741 fix: quote ident on rendered SQL (#2248) 
* fix: quote ident on rendered SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* read quote style from query context

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-25 07:25:21 +00:00