rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-20 15:00:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,286 Commits 133 Branches 253 Tags
feat/expr-split
Commit Graph

4372 Commits

Author SHA1 Message Date
Ning Sun
c39de9072f refactor: use workspace dependencies for internal modules (#2119) 
* refactor: use workspace dependencies for internal modules

* fix: resolve issue with mock module in datanode

* refactor: update test modules
 2023-08-08 11:02:34 +00:00
Lei, HUANG
2d9ea595cb chore!: change logstore namespace prefix (#1998) 
* chore: change logstore namespace prefix

* chore: change delimiter
 2023-08-08 07:36:46 +00:00
LFC
46fa3eb629 chore: upgrade rust toolchain to latest nightly (#2049) 
* chore: upgrade rust toolchain to latest nightly

* rebase develop

* update rust toolchain in ci
 2023-08-08 07:17:51 +00:00
Weny Xu
7d0d8dc6e3 feat: return metasrv leader addr (#2110) 2023-08-07 10:01:42 +00:00
Zhenchi
f8d152231d feat(information_schema): implement table_factory method (#2108) 
* feat(information_schema): implement table_factory method

* refactor(catalog): simplify table_factory method

* Update src/table/src/data_source.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-07 08:07:25 +00:00
Weny Xu
c8cb1ef5bc feat: add schema and catalog key migration tool (#2048) 
* feat: add schema and catalog key migration tool

* chore: apply suggestions from CR
 2023-08-07 06:22:05 +00:00
Zou Wei
d5cadeeec3 feat: conversion between interval and gRPC (#2064) 
* feat: support grpc for interval type

* chore: add unit test cases.

* chore: cargo clippy

* chore: modify greptime-proto version

* chore: cr comment.

* chore: cargo fmt

* refactor: convert function.
 2023-08-07 06:22:04 +00:00
Vanish
cf7e8c9142 feat: truncate region (#2097) 
* feat: impl truncate region

* test: test truncate region

* chore: typo

* refactor: table truncate

* chore: remove useless changes

* chore: reset version

* fix: wait for flush task to complete

* fix: clippy

* chore: remove useless changes

* CR

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/engine.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/engine.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/region.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/region/tests/truncate.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/region/tests/truncate.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/region/writer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* CR

* Update src/storage/src/engine.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/storage/src/manifest/region.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-08-04 12:26:25 +00:00
Yingwen
cb4dd89754 feat(mito): Implement mito2 Wal (#2103) 
* feat: define wal struct

* feat: Implement Wal read/write

* feat: obsolete wal

* test: test wal

* refactor: use try_stream and remove async from scan
 2023-08-04 11:04:25 +00:00
Ruihang Xia
9718aa17c9 feat: define region group and sequence (#2100) 
* define region group

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* define region sequence

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check partition number

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test region seq and group

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-04 09:08:07 +00:00
shuiyisong
7efff2d704 fix: introduce taplo.toml and sort Cargo.toml (#2096) 
* fix: add taplo.toml

* fix: introduce taplo.toml & sort cargo.toml

* chore: remove option in ci too
 2023-08-04 06:44:45 +00:00
Yingwen
e5663a075f feat(mito): preparation to implementing write (#2085) 
* refactor: move request mod

* feat: add mutation

* feat: add handle_write mod

* feat: one mutation at a time

* feat: handle write requests

* feat: validate schema

* refactor: move schema check to write request

* feat: add convert value

* feat: fill default values

* chore: remove comments

* feat: remove code

* feat: remove code

* feat: buf requests

* style: fix clippy

* refactor: rename check functions

* chore: fix compile error

* chore: Revert "feat: remove code"

This reverts commit 6516597540.

* chore: Revert "feat: remove code"

This reverts commit 5f2b790a01.

* chore: upgrade greptime-proto

* chore: Update comment

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-08-04 02:53:02 +00:00
JeremyHi
7987e08ca2 chore: typo (#2092) 2023-08-04 01:38:17 +00:00
Eugene Tolbakov
1492700acc fix(timestamp): add trim for the input date string (#2078) 
* fix(timestamp): add trim for the input date string

* fix(timestamp): add analyzer rule to trim strings before conversion

* fix: adjust according to CR
 2023-08-03 23:33:47 +00:00
shuiyisong
6f1094db0a fix: arc() usage in non-test code (#2091) 
* chore: try fix arc issue

* chore: move `parse_catalog_and_schema_from_client_database_name` to catalog crate

* fix: arc issue

* fix: arc issue

* fix: arc issue

* fix: arc issue

* fix: minor change
 2023-08-03 10:16:02 +00:00
Ruihang Xia
5f0403c245 feat: improve /label and /labels APIs in prometheus server (#2087) 
* support __name__ for /label

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* make match[] in labels optional

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-08-03 07:51:08 +00:00
fys
d7002caca7 chore: add meter-core dependency (#2088) 2023-08-03 07:24:34 +00:00
fys
dda922507f feat: impl pubsub in metasrv (#2045) 
* feat: impl pubsub

* add test_subscriber_disconnect unit test

* chore: cr

* cr

* cr
 2023-08-03 03:56:43 +00:00
Yingwen
fdd4929c8f refactor(mito): mv mito2 request (#2086) 
* refactor: mv request mod to crate level

* refactor: mv SkippedFields
 2023-08-03 03:38:46 +00:00
Vanish
e3a079a142 fix: session features (#2084) 2023-08-02 13:39:17 +00:00
discord9
c55841988e feat: necessary Hash derive for types (#2075) 
* feat: necessary derive for types

* impl (Partial)Ord for ConcreteDataType
 2023-08-02 13:08:43 +00:00
Ning Sun
7a27ef8d11 fix: remove openssl from reqwest and use rustls instead (#2081) 
* fix: remove openssl from reqwest and use rustls instead

* fix: correct server url

* style: fix toml format
 2023-08-02 10:23:21 +00:00
Weny Xu
4626c2efe5 feat: add Catalog and Schema Manager (#2037) 
* feat: add Range Stream

* feat: add catalog and schema manager

* feat: enhance KeyValueDecoderFn

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2023-08-02 03:56:29 +00:00
shuiyisong
5eb2c609a3 fix: auth in grpc (#2056) 
* fix: auth in grpc

* fix: change to return err

* fix: add grpc test

* fix: add http test

* fix: add mysql and pg test
 2023-08-01 15:18:31 +00:00
JeremyHi
fd7eb87a52 refactor: common semantic-type (#2057) 2023-08-01 11:18:05 +00:00
Sunray Ley
090b7e61ca feat: make the gRPC channel between Frontend and Datanode configurable (#2044) 
* feat: expose frontend datanode_client_options

* chore: add configuration options to the configuration file

* refactor(frontend): extract DatanodeOptions to service_config

* refactor(frontend): extract DatanodeOptions to service_config

* style: remove unnecessary suffix in variable name

Co-authored-by: Yingwen <realevenyag@gmail.com>

* feat: use humantime_serde for readable duration

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-08-01 10:49:41 +00:00
Yingwen
c529c8a41b feat(mito): Implement open and close for mito2 regions (#2052) 
* feat: add close request

* feat: handle close and open request

* feat: Implement open

* test: add TestEnv::new

* feat: close region/engine and test

* style: fix clippy

* style: import log macros

* docs: update docs

* docs: add mermaid for manifest manager
 2023-08-01 10:49:07 +00:00
gongzhengyang
0eac56a442 chore: remove unused dependencies (#2055) 
Co-authored-by: gongzhengyang <gongzhengyang@bolean.com.cn>
 2023-08-01 07:43:03 +00:00
Ruihang Xia
44f3ed2f74 chore(deps): bump datafusion to the latest commit (#1967) 
* bump deps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile except pyo3 backend

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix promql-parser metric name matcher

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyarrow convert

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyo3 compiling

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update stream adapter display format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix physical optimizer rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-01 02:10:49 +00:00
Ruihang Xia
5bd80a74ab feat: prepare for implementing considering partition key in the distributed planner (#2000) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix frontend logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check substrait compatibility before pushdown

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* going to revert some rules

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test and clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove println

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-07-31 12:36:23 +00:00
Yingwen
4d5ecb54c5 feat(mito): Implement open for RegionManifestManager (#2036) 
* feat: file purger trait

* feat: Implement open for RegionManifestManager

* feat: remove RegionVersion

* feat: Use RwLock

* chore: remove AtomicManifestVersion

* feat: Remove unused error

* feat: store meta action

* chore: update comment
 2023-07-31 10:04:22 +00:00
shuiyisong
922d826347 chore: make tables() return kv instead of key only (#2047) 
* chore: make tables return kv

* chore: remove comment code
 2023-07-31 07:30:47 +00:00
localhost
7681864eb4 chore: add version reporter (#2007) 
* chore: add version reporter

* chore: add uuid for version report

* chore: add file license

* chore: format code

* chore: fix by pr comment

* chore: change version report api url

* chore: change greptimedb opentelemetry crate name

* chore: minor code beautification

* chore: add keys only option when range etcd

* chore: fix by pr comment

* chore: fix by pr comment

* chore: change uuid file location

* chore: only run telemetry in meta leader

* chore: add more test and some minor fix

* chore: make clippy happy

* chore: fix by pr comment

* chore: fix by pr comment

* chore: add debug log for greptimedb telemetry
 2023-07-31 06:58:00 +00:00
Zou Wei
7727508485 feat: impl interval type (#1952) 
* feat: impl interval type in common time

* feat: impl datatype, vectors, value for interval

    pick 0c1d9f297 feat: impl interval type in common time
    pick d528c647f feat: impl datatype, vectors, value for interval
    pick 1e12dd5c7 comments update
    pick 74103e36c add license header

* comments update

* add license header

* cargo clippy

* refactor interval type

* add unit test and case to dummy.sql

* cargo clippy

* chore: add doc comments

* chore: cargo fmt

* feat: add formats, refactor comparison

* add docs comments

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: cr comment

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-07-31 03:54:39 +00:00
Niwaka
695398652c feat: accept influxdb request without timestamp even if table doesn't exist (#2041) 
* feat: accept influxdb request without timestamp even if table doesn't exist

* refactor: InsertRequests::try_from

* feat: check row number
 2023-07-31 02:55:09 +00:00
parkma99
fc6ebf58b4 refactor: create_current_timestamp_vector by using VectorOp::cast (#2042) 
* refactor using VectorOp cast

* add test case
 2023-07-31 02:51:06 +00:00
Zou Wei
f22b787fd9 chore: return error in arrow array convert function (#2043) 
fix: return error instead of unreachable!()
 2023-07-31 02:47:40 +00:00
Lei, HUANG
81ea61ba43 fix: window inferer (#2033) 
* fix: window inferer

* chore: rename
 2023-07-26 02:18:19 +00:00
LFC
48996b0646 fix: etcd range pagenation in table metadata migration tool (#2035) 2023-07-25 10:02:26 +00:00
fys
0b4ac987cd refactor: arrange lease kvs randomly in lease_based selector (#2028) 
* refactor: arrange lease kvs randomly in lease_based selector

* fix: cr
 2023-07-25 07:32:10 +00:00
shuiyisong
9c1f0234de refactor: query context (#2022) 
* chore: refactor query_context

* chore: remove use statement

* chore: make query_context build return arc

* fix: sqlness test

* fix: cr issue

* fix: use unwrap or else
 2023-07-25 06:11:34 +00:00
Ruihang Xia
f55bff51ac feat: set and retrieve trace id in log macro (#2016) 
* trace id passed by task local store

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* modify log macro

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove tokio::spawn

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use real trace id

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-07-25 03:50:27 +00:00
Weny Xu
0fc0f74cd7 fix: fix parking_lot unresolved (#2025) 2023-07-25 03:20:12 +00:00
Yingwen
5f65e3ff44 feat(mito): Port parquet writer and reader to mito2 (#2018) 
* feat(mito): Port Batch and BufferedWriter

* feat: encode metadata to parquet

* feat: define BatchReader trait

* chore: ParquetWriter write_all takes `&mut self`

* feat(mito): port ParquetReader

* chore: fix typo

* chore: address CR comment
 2023-07-24 09:35:21 +00:00
dennis zhuang
1f371f5e6e fix: checkpoint metadata file dirty caching (#2020) 
fix: dirty last checkpoint metadata file when enable object store caching, #2013
 2023-07-24 08:18:19 +00:00
shuiyisong
632cb26430 feat: trace_id in query context (#2014) 
* chore: unify once_cell version

* chore: update cargo lock

* chore: add gen_trace_id

* chore: add trace_id to query_ctx

* chore: add debug log

* Revert "chore: add debug log"

This reverts commit f52ab3bb300f1d73117cd6ebbd8e0162829b1aba.

* chore: add frontend node id option

* chore: add query ctx to query engine ctx

* chore: set trace_id to logical_plan api

* chore: add trace_id in grpc entrance

* chore: generate trace_id while creating query_ctx

* chore: fix typo

* chore: extract trace_id from grpc header

* chore: extract trace_id from grpc header

* chore: fix clippy

* chore: add `QueryContextBuilder`

* chore: change node_id in fe to string
 2023-07-24 07:35:06 +00:00
JeremyHi
41139ec11d feat: region lease improve (#2004) 
* feat: add exists api into KvBackend

* refactor: region lease

* feat: fiter out inactive node in keep-lease

* feat: register&deregister inactive node

* chore: doc

* chore: ut

* chore: minor refactor

* feat: use memory_kv to store inactive node

* fix: use real error in

* chore: make inactive_node_manager's func compact

* chore: more efficiently

* feat: clear inactive status on cadidate node
 2023-07-24 03:49:14 +00:00
Yingwen
792d8dfe33 feat(mito): create region in mito2 engine (#1999) 
* chore: check table existence

* refactor: rename LevelMetaVec

* feat: create request to metadata

* refactor: Share MitoConfig between workers

* feat: impl handle_create_request

* refactor: move tests mod

* feat: validate time index nullable

* feat: test create region

* feat: test create if not exists

* feat: remove option

* style: fix clippy

* chore: address CR comments
 2023-07-21 06:41:34 +00:00
gobraves
e3ac3298b1 feat: add orc stream (#1981) 
* add orc stream #1820

* update orc stream

* fix: create orcstreamadapter with opt projection

* fix: license header

* docs: delete comment
 2023-07-21 05:54:02 +00:00
LFC
953b8a0132 feat: benchmark table metadata managers (#2008) 
* feat: benchmark table metadata managers

* feat: benchmark table metadata managers
 2023-07-21 05:41:06 +00:00