rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-14 09:12:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,459 Commits 104 Branches 227 Tags
28fd0dc276cfd3e6c94de73492dfee4a5ae70ec3
Commit Graph

2459 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
fys
7fe417e740 fix: an error occurred when requesting the http doc api (#984) 2023-02-13 11:17:27 +00:00
fys
c1a9f84c7f feat: meta provides the ability to distribute lock (#961) 
* add DistLock trait and a implement based etcd

wip

impl lock grpc service for meta-srv

reuse the etcd client instead of repeatedly creating etcd client

add some docs and comments

add some comment

meta client support distribute lock

fix: dead lock

self-cr

* cr

* rename "expire" -> "expire_secs"
v0.1.0-alpha-20230213-weekly 		2023-02-13 15:58:30 +08:00
Yingwen
be897efd01 feat: Execute procedure in LocalManager (#953) 
* feat: Runner executes procedure

* feat: Add rollback key type to ParsedKey

* feat: Write rollback key when procedure is unable to execute

* feat: Use loaded step to re-submit subprocedure

* feat: Track subprocedures in ProcedureMeta

* feat: Clean message cache after the root procedure is done

* feat: Runner returns execution result

* fix: Fix tests

* test: Test Runner

* test: Test procedures_in_tree

* chore: Refine test and comments

* feat: Remove support of lock inheritance

A deadlock happens if a subprocedure acquires the same lock key as
its parent.

The main concern is if the subprocedure directly inherits its parent's
lock, then how should we behave when multiple subprocedures acquire
this same lock? Each procedure may assume it has unique access to the
same object but it actually shares the resource with others.

Now subprocedures need to use different keys to lock objects, which is
reasonable. For example:
- A parent procedure wants to create a table so it locks the table with
a key like `catalog.schema.table`
- Subprocedures create regions for the table so they lock the regions
with keys `catalog.schema.table.region-0 ~ catalog.schema.table.region-n`

* style: Fix clippy

* feat: insert_procedure returns false on duplicate procedure

Also rename this method to try_insert_procedure

* chore: Address CR comments
 2023-02-13 10:38:56 +08:00
Eugene Tolbakov
c06e04afbb refactor(query): tests from query/tests to query/src (#973) 
* refactor(query): tests from query/tests to query/src

* chore(query): address rust fmt issues

* chore(query): add licence header
 2023-02-12 20:55:17 +08:00
Lei, HUANG
e77a7f253c feat: L0 to L1 compaction strategy (#964) 
* feat: impl simple compaction strategy

* chore: rebase to develop and fix clippy warnings

* chore: simplify time bucket strcut

* chore: some typos
 2023-02-11 21:10:24 +08:00
Eugene Tolbakov
7d6f4cd88b feat: remove backtrace from sql::error::Error (#966) 
* feat: remove backtrace from sql::error::Error

* fix: address formatting issues

---------

Co-authored-by: Evgeny Tolbakov <evgeny.tolbakov@jpmorgan.com>
 2023-02-11 14:52:29 +08:00
Ruihang Xia
83ac6598b6 feat: add start, end and step to promql http api (#969) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-10 14:16:16 +08:00
Ruihang Xia
4c925e0079 chore(deps): bump promql-parser (#968) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-10 14:12:55 +08:00
LFC
c6128ec0a4 refactor: use remote proto (#963) 
* refactor: use remote proto (see our new repo "GreptimeTeam/greptime-proto")

* fix: resolve PR comments
 2023-02-10 13:35:18 +08:00
discord9
7c34b009ec feat: bind DataFrame API into python script (#945) 
* chore: remove unused magic fn

* feat: dataframe

* feat: add data_frame crate

* feat: more api binded

* fix: `Comparable` for overload op

* fix: license&more test

* chore: PR advices

* chore: more PR advices
 2023-02-10 11:21:57 +08:00
shuiyisong
70edd4d55b fix: remove incorrect table_idents_to_full_name (#967) 2023-02-10 03:15:48 +00:00
Ning Sun
6beea73590 fix: use query_ctx in distributed inserts (#965) 2023-02-10 10:09:13 +08:00
Yun Chen
c0d3533d10 fix: Sql Inline Primary Key definition (#957) 
* fix: invalid inline primary key syntax

* fix: format

* fix: clippy fix

* fix: added sqlness tests

* fix: throw exception when multiple inline pk defined

* fix: pr comments

* fix: add ending blank line for create.sql
 2023-02-09 18:57:19 +08:00
shuiyisong
9989a8c192 fix: check full table name during logical plan creation (#948) 2023-02-09 17:23:28 +08:00
Ruihang Xia
19dd8b1246 feat: SeriesDivide plan for PromQL (#960) 
* implement SeriesDivide plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* planner part

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy and typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-09 11:50:29 +08:00
Lei, HUANG
1e9918ddf9 feat: compaction scheduler and rate limiter (#947) 
* wip: compaction schdduler

* feat: imple simple compaction scheduler

* fix: typo

* feat: add generic parameter to make scheduler friendly to tests

* chore: add more tests

* fix: CR comments

* fix: CR comments

* fix: ensure idempotency for rate limit token

* fix: Cr ct omments
 2023-02-09 11:43:20 +08:00
fys
4ce62f850b chore: add an opaque error type in meta (#959) 
add boxed err in meta
 2023-02-08 09:47:33 +00:00
Ning Sun
83d57f9111 fix: setting postgres query context (#958) 2023-02-08 16:34:10 +08:00
LFC
803b7f0633 feat: implement "drop table" in distributed mode (both in SQL and gRPC) (#944) 
* feat: implement "drop table" in distributed mode (both in SQL and gRPC)

refactor: create distributed table
some details:
- set table global value in Meta, as well as table routes value. Datanode only set table regional value
- complete instance SQL tests both in standalone and distributed mode

* fix: rebase develop

* fix: resolve PR comments
 2023-02-08 07:36:38 +00:00
Ruihang Xia
37ca5ba380 chore: alias sqlness subcommand (#956) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-07 18:53:46 +08:00
Ning Sun
c1d32bdf2b fix: add form data support for http sql api (#955) 
fix: add form data support for http apis
 2023-02-07 10:15:39 +00:00
fys
83509f31f4 feat: datanode stats is stored in the mem_kv of meta leader (#943) 
* store heartbeat data in memory, instead of etcd

* fix: typo

* fix: license header

* cr
 2023-02-07 17:09:28 +08:00
elijah
926022e14c feat: enable caching when using object store (#928) 
* feat: enable caching when using object store

* feat: support file cache for object store

* feat: maintaining the cached files with lru

* fix: improve the code

* empty commit

* improve the code
 2023-02-07 15:46:37 +08:00
Ruihang Xia
2f2609d8c6 build(ci): disable release workflow for forked repo (#954) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-07 15:22:32 +08:00
Yingwen
ecadbc1435 feat: Add procedure manager LocalManager (#946) 
* feat: Add ManagerContext and LocalManager

* test: Add register_loader test

* feat: Remove some unused methods

* fix: Fix submit_procedure ensure condition
 2023-02-07 11:33:13 +08:00
ShenJunkun
afac885c10 refactor: add schema column to the scripts table (#868) v0.1.0-alpha-20230207-weekly 2023-02-07 11:07:32 +08:00
Lei, HUANG
5d62e193bd feat: support multi regions on datanode (#653) 
* wip: fix compile errors

* chore: move splitter to partition crate

* fix: remove useless variants in frontend errors

* chore: move more partition related code to partition manager

* fix: license header

* wip: move WriteSplitter to PartitionRuleManager

* fix: clippy warnings

* chore: remove useless error variant and format toml

* fix: cr comments

* chore: resolve conflicts

* chore: rebase develop

* fix: cr comments

* feat: support multi regions on datanode

* chore: rebase onto develop

* chore: rebase develop

* chore: rebase develop

* wip

* fix: compile errors

* feat: multi region

* fix: CR comments

* feat: allow stat existing regions without actually open it

* fix: use table meta in manifest to recover region info
 2023-02-07 10:46:18 +08:00
elijah
7d77913e88 chore: fix rfc typo (#952) 2023-02-07 08:47:06 +08:00
Lei, HUANG
3f45a0d337 docs: rfc for table compaction (#939) 
* doc: rfc for table compaction

* docs: update compaction rfc
 2023-02-06 22:15:53 +08:00
Zhizhen He
a1e97c990f chore: fix typo (#949) 2023-02-06 22:13:56 +08:00
Ning Sun
4ae63b7089 feat: Initial prepare statement support for Postgres protocol (#925) 
* feat: add describe statement to query_engine

* feat: add ability to describe statement for sql handler

* refactor: return schema instead of wrapped ref

* test: resolve tests

* feat: add initial support for prepared statements

* feat: add parameter types to query statement

* test: fix parser test

* chore: add todo task

* fix: turn on integer_datetime for binary timestamp

* fix: format string using single quote

* test: add tests for prepared statement

* Apply suggestions from code review

Co-authored-by: LFC <bayinamine@gmail.com>

* refactor: use stream api from recordbatches

---------

Co-authored-by: LFC <bayinamine@gmail.com>
 2023-02-06 22:06:00 +08:00
Yingwen
b0925d94ed feat: Implement lock component for ProcedureManager (#937) 
* feat: Add procedure meta

* feat: Implement lock for procedures

* chore: Allow dead code

* docs: Fix comment

* docs: Update docs of acquire_lock
 2023-02-03 18:42:03 +08:00
Ruihang Xia
fc9276c79d feat: export promql service in server (#924) 
* chore: some tiny typo/style fix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: add promql server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* works for mocked query

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* integration test case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* expose promql api to our http server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adjust router structure

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-03 08:28:56 +00:00
LFC
184ca78a4d revert: removed all "USE"s in sqlness tests introduced in #922 (#938) 2023-02-03 15:44:58 +08:00
discord9
ebbf1e43b5 feat: Query using sql inside python script (#884) 
* feat: add weakref to QueryEngine in copr

* feat: sql query in python

* fix: make_class for Query Engine

* fix: use `Handle::try_current` instead

* fix: cache `Runtime`

* fix: lock file conflict

* fix: dedicated thread for blocking&fix test

* test: remove unnecessary print
 2023-02-03 15:05:27 +08:00
dennis zhuang
54fe81dad9 docs: add dashboard to resources in README (#934) 2023-02-03 13:47:19 +08:00
LFC
af935671b2 feat: support "use" in GRPC requests (#922) 
* feat: support "use catalog and schema"(behave like the "use" in MySQL) in GRPC requests

* fix: rebase develop
 2023-02-02 20:02:56 +08:00
Yingwen
74adb077bc feat: Implement ProcedureStore (#927) 
* test: Add more tests for ProcedureId

* feat: Add ObjectStore based state store

* feat: Implement ProcedureStore

* test: Add tests for ParsedKey

* refactor: Rename list to walk_top_down

* fix: Test ProcedureStore and handles unordered key values.

* style: Fix clippy

* docs: Update comment

* chore: Adjust log level for printing invalid key
 2023-02-02 17:49:31 +08:00
Ruihang Xia
54c7a8be02 docs: document sqlness-runner usage (#931) 
docs: paste doc from greptime-doc

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-02 15:56:51 +08:00
Ruihang Xia
ea5146762a chore(deps): bump promql-parser (#929) 
* fix promql crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* migrate to new api

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix aggregator test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix styles

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-02 07:31:41 +00:00
Yingwen
788b5362a1 docs: Add procedure framework RFC (#836) 
* docs: Add procedure framework RFC

* docs: Add dump, rollback and locking to procedure framework

* docs: Change ProcedureBuilder to ProcedureLoader

* docs: Add sub-procedures section

* docs: Add a link to explain idempotent

* docs: Add link to the tracking issue

* docs: Fix ProcedureLoader type alias

* docs: Update procedure API

* docs: Address CR comments

* docs: Update path and make the docs more clear
 2023-02-02 11:28:56 +08:00
Lei, HUANG
028a69e349 refactor: move partition related code to partition manager (#906) 
* wip: fix compile errors

* chore: move splitter to partition crate

* fix: remove useless variants in frontend errors

* chore: move more partition related code to partition manager

* fix: license header

* wip: move WriteSplitter to PartitionRuleManager

* fix: clippy warnings

* chore: remove useless error variant and format toml

* fix: cr comments

* chore: resolve conflicts

* chore: rebase develop

* fix: cr comments
 2023-02-01 19:24:49 +08:00
elijah
9a30ba00c4 test: run sqlness test in distributed mode (#916) 
* test: run sqlness test in distributed mode

* chore: fix ci test

* chore: improve the ci yaml

* chore: improve the code

* chore: fix conflicts
 2023-01-31 15:00:11 +08:00
LFC
8149932bad feat: local catalog drop table (#913) 
* feat: local catalog drop table

* Update src/catalog/src/local/manager.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* Update src/catalog/src/local/manager.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* fix: resolve PR comments

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-01-31 14:44:03 +08:00
Ruihang Xia
89e4084af4 build(ci): upload sqlness log files (#920) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-01-31 14:31:27 +08:00
Ning Sun
39df25a8f6 refactor: make postgres handler stateful (#914) 
* feat: update pgwire to 0.8 and unify postgres handler

* fix: correct password message matching
 2023-01-31 14:19:18 +08:00
Yingwen
b2ad0e972b feat: Define procedure related traits (#904) 
* chore: Move uuid to workspace.dependencies

* feat: Define procedure related traits

* test: Add tests

* chore: Update imports

* feat: Submit ProcedureWithId to manager

* chore: pub ProcedureId::parse_str

* refactor: ProcedureId::parse_str returns Result

* chore: Address CR comments

Also implements FromStr for ProcedureId
 2023-01-31 14:17:28 +08:00
shuiyisong
18e6740ac9 chore: add interceptor err in frontend::error::Error (#917) 
* chore: add interceptor boxed err

* chore: rename

* chore: update err msg

Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com>

---------

Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com>
 2023-01-30 03:12:03 +00:00
Yun Chen
a7dc86ffe5 feat: oss storage support (#911) 
* feat: add oss storage support

* fix: ci build format check

* fix: align OSS to Oss

* fix: cr comments

* fix: rename OSS to Oss in integration tests

* fix: clippy fix
v0.1.0-alpha-20230130-weekly 		2023-01-29 20:09:38 +08:00
Ruihang Xia
71482b38d7 feat: PromQL binary expr planner (#889) 
* feat: PromQL binary expr planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* column & column test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* column & literal test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* mark literal-literal unsupported

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-01-29 17:02:11 +08:00