Commit Graph

28 Commits

Author SHA1 Message Date
dennis zhuang
df577ad7f8 feat: impl create table sql parser (#115)
* feat: impl create table sql parser

* feat: checking time index columns size

* refactor: move TS_INDEX to statement

* fix: address CR problems

* refactor: remove parser_error! macro
2022-07-29 11:03:36 +08:00
LFC
2b064265bf feat: UDAF made generically (#91)
* feat: UDAF implementation backed by DataFusion.

Directly Transplant DataFusion's UDAF related structs, traits and functions, like `AggregateUDF`, `Accumulator` or `create_udaf` etc.

Implement median UDAF on top of it and used in unit testing.

Refs: #61

* feat: UDAF made generically

Refs: #61

* fix: cargo fmt

* fix: use prelude

* fix: uniform the name

* fix: move maybe commonly used functions together

* fix: make comments more clear

* fix: resolve conversations in CR

* fix: store input types in AccumulatorCreator, and use ScalarVector's iterator

* feat: introducing List value and List datatype

* refactor: use ArcSwap instead of Mutext

* refactor: shorten some namings

* refactor: move median UDAF out of tests

* refactor: rename

* feat: aggregate function registry

* fix: make `Value` satisfy ordering again

* fix: clippy warnings

* doc: add "how to write aggregate function"

* fix: address PR comments

* fix: trying to get rid of unwraps

Co-authored-by: luofucong <luofucong@greptime.com>
2022-07-25 10:35:36 +08:00
天空好像下雨~
267a47e9dd move interp from test to numpy (#88)
* move interp from test to numpy

* move interp from test to numpy

* move interp from test to numpy

* move interp from test to numpy

* move interp from test to numpy
2022-07-18 15:38:30 +08:00
天空好像下雨~
403b94c948 feat: add operator interp (#66)
* benchmark

* bench:add read/write for memtable

* numpy-interp

* fix cast

* implement tests

* implement tests

Co-authored-by: 张心怡 <zhangxinyi@zhangxinyideMacBook-Pro.local>
2022-07-15 10:49:44 +08:00
dennis zhuang
e78c015fc0 TableEngine and SqlHandler impl (#45)
* Impl TableEngine, bridge to storage

* Impl sql handler to process insert sql

* fix: minor changes and typo

* test: add datanode test

* test: add table-engine test

* fix: code style

* refactor: split out insert mod from sql and minor changes by CR

* refactor: replace with_context with context
2022-06-17 11:36:49 +08:00
evenyag
4171173b76 feat: Support creating in memory region and writing to memtable (#40)
* chore(store-api): Fix typo in region comments

* feat(storage): Init storage crate

* feat(store-api): Make some method async

* feat(storage): Blank StorageEngine implementation

* feat(storage): StorageEngine returns owned SchemaRef

* feat: pub use arrow in datatypes

* feat(store-api): Implement RegionMetadata

* feat(storage): Impl create region in memory.

* chore(object-store): Format cargo toml

* chore(storage): Log on region created

* feat: Impl CowCell

* feat: Store id to cf meta mapping

* refactor: Refactor version and rename it to VersionControl

* feat: Impl write batch for put, refactor column family

* feat(storage): Skeleton of writing to memtable

* refactor(storage): MemTable returns MemTableSchema

* feat: Add ColumnSchema and conversion between schema and arrow's schema

* feat: Validate put data

* feat: Valid schema of write batch

* feat: insert memtable WIP

* feat: Impl Inserter for memtable

* feat(datatypes): Implement Eq/Ord for Value

feat: Implement Ord/Eq for Bytes/StringBytes and Deref for Bytes

test: Test Value::from()

* feat: Define BTreeMemTable

* Fix: Rename get/get_unchecked to try_get/get and fix get not consider null.

* feat: Impl BTreeMemTable::write()

* refactor: Remove useless ColumnFamilyHandle now

* chore: Clean comment

* feat(common): Add from `String/&str/Vec<u8>/&[u8]` for Value

* test(storage): Add tests for WriteBatch

* chore: Fix clippy

* feat: Add builder for RowKey/ColumnFamilyDescriptor

* test: Add test for metadata

* chore: Fix clippy

* test: Add test for region and engine

* chore: Fix clippy

* chore: Address CR comment
2022-06-09 16:50:02 +08:00
dennis zhuang
f7136819fc function crate and scalars function (#39)
* feat: adds scalars mod and enhance vectors

* temp commit

* fix compile error

* Impl pow function with new udf framework

* Adds common-function crate and impl scalars function

* fix: remove used code

* test: adds test for function crate and refactor vectors

* fix: fmt style

* feat: impl numpy.clip function

* feat: improve clip function returning int64 type when arguments do not have float type

* feat: adds more test for vectors

* feat: adds replicate method test for primitive vector

* fix: by code review

* feat: clip function returns uint64 when all arguments type are unsigned

* refactor: improve vectors#only_null

* fix: clippy warning

* fix: clippy warning

* fix: clip should return float64 when arguments have both signed and unsigned types
2022-06-08 13:15:22 +08:00
dennis zhuang
a2331366f6 feat: adds adds register_udf api to query engine and refactor datatypes (#34)
* feat: adds ColumnarValue and refactor vectors

* fix: ConcreteDataType compile error

* feat:adds udf/function mods

* feat: adds test for common_query crate

* feat: adds register_udf api to query engine

* feat: adds common_query::error test

* refactor: by CR comments

* refactor: adds impl_new_concret_type_functions! macro to reduce boilerplate codes

* fix: typo
2022-05-24 16:50:56 +08:00
dennis zhuang
93cbdbee9a feat: adds common-telemetry crate and adds logs/metrics to query engine (#25)
* feat: adds common-telemetry crate and logging mod

* refactor: common telemetry

* feat: adds metric mod and exports metrics by http

* feat: adds metrics to query engine and datanodes

* refactor: rename metrics

* refactor: rename ElapsedTimer struct and elapsed_timer macro

* refactor: log error when fail to new data node.

* fix: use backtrace crate instead of unstable feature

* feat: add must_use attr to Timer

* fix: only reserve Debug attribute for Timer
2022-05-12 17:42:48 +08:00
evenyag
d5de030600 feat: Add as_any() to opaque error 2022-05-09 12:35:55 +08:00
evenyag
5f48b4996b chore: Address CR comments 2022-05-09 11:52:01 +08:00
evenyag
d2d4d88c89 test: Add MockError and add more tests for error 2022-05-07 17:29:56 +08:00
evenyag
e0c1bbfd21 refactor: Rename errors::ParserError to error::Error 2022-05-07 16:15:05 +08:00
evenyag
ec63353364 refactor: Move planner and adapters to datafusion mod 2022-05-07 15:27:00 +08:00
evenyag
6a20657591 refactor: query::query_engine::datafusion -> query::datafusion 2022-05-07 14:38:57 +08:00
evenyag
99c7ffb456 refactor: Define error for memory catalog 2022-05-07 14:13:31 +08:00
evenyag
83262acc0e refactor: Add more imports to common_error::prelude 2022-05-07 11:42:33 +08:00
evenyag
63d9aa1bff refactor: Refactor datanode error and impl ErrorExt for it 2022-05-07 11:34:23 +08:00
evenyag
7e2e3e3429 feat: Impl ErrorExt for opaque error and ParseError 2022-05-06 20:04:16 +08:00
evenyag
08ccb466cb fix(query): Remove unnecessary unsafe impl Send/Sync for ExecutionPlanAdapter 2022-05-06 20:04:16 +08:00
evenyag
aa1a6b2205 feat: Add common-error crate and implement opaque error type. 2022-05-06 20:04:14 +08:00
dennis zhuang
f0db878bf5 feat: http server and cmd crate etc. (#15)
* feat: adds cmd crate and http server

* feat: impl sql http handler

* feat: convert all arrow array types

* feat: adds query test

* feat: adds test for datanode

* fix: format

* feat: refactor state.rs

* feat: adds collect test

* fix: by code review

* fix: style
2022-05-06 17:34:29 +08:00
Lei, Huang
f889ed5488 feat: INSERT statement and planner implementation (#16)
* feat: Add SQL parser definition and SHOW DATABASE implementation

* chores: Eliminate clippy warnings and errors.

* chores: remove Gt prefix in some structs; rename some mod; remove print(s) in unit tests; refactor crate layout;
feat: wrap sqlparser error;

* chores: reorder cargo dependency

* chores: fix code style

* chores: add #[cfg(test)] to unit tests

* style: fix test mod style

* feat: implement select query parser

* chores: remove unused dependency

* feat: implement TryInto<sqlparser::ast::Statement> for Statement

* chore: fix style issues

* refactor: wrap sqlparser Query inside Query statement variant to reduce complexity

* refactor: replace TryInto to TryFrom

* refactor: use [Rust 2018 mod convention](https://doc.rust-lang.org/edition-guide/rust-2018/path-changes.html#no-more-modrs)

* refactor: remove unnecessary file prefix in statement mod

* feat: implement INSERT parser (currently without INSERT validation)

* feat: wrap DataFusion planner and add simple query planner implementation

* refactor: move planner mod to query crate

* fix: styles and conventions
2022-05-05 16:28:38 +08:00
dennis zhuang
bf331ec4ac feat: impl QueryEngine#execute, adds numbers table and query engine test (#13)
* feat: impl QueryEngine#execute, adds numbers table and query engine test

* fix: clippy warning

* fix: reuse runtime in context in table adapter

* fix: by CR comments
2022-04-27 15:15:26 +08:00
dennis zhuang
3a2f794f6c feat: query engine impl on datafusion (#10)
* feat: query engine impl on datafusion

* feat: adds physical_optimizer, physical_planner and executor

* feat: impl adpaters between datafuion and greptime query engine core APIs.

* feat: impl PhysicalPlanAdapter and ExecutionPlanAdapter

* feat: rename table datafusion mod to adapter

* fix: clippy warning

* fix: conflicts with develop branch

* feat: add database mod

* fix: CR comment

* fix: by CR comments

* fix: conflicts with develop branch

* fix: by CR comments
2022-04-26 15:17:32 +08:00
Lei, Huang
cb74f1ac34 feat: Add sql parser definition and trivial SHOW DATABASE implementation (#8)
* feat: Add SQL parser definition and SHOW DATABASE implementation

* chores: Eliminate clippy warnings and errors.

* chores: remove Gt prefix in some structs; rename some mod; remove print(s) in unit tests; refactor crate layout;
feat: wrap sqlparser error;

* chores: reorder cargo dependency

* chores: fix code style

* chores: add #[cfg(test)] to unit tests

* style: fix test mod style
2022-04-25 17:01:55 +08:00
evenyag
1fa0b4e3f9 chore: Setup code skeleton of datanode 2022-04-21 11:40:46 +08:00
evenyag
ced9a7c97f chore: Create datanode, datatypes, log-store, object-store, query, table crates 2022-04-20 16:34:22 +08:00