Commit Graph

28 Commits

Author SHA1 Message Date
dennis zhuang
e78c015fc0 TableEngine and SqlHandler impl (#45)
* Impl TableEngine, bridge to storage

* Impl sql handler to process insert sql

* fix: minor changes and typo

* test: add datanode test

* test: add table-engine test

* fix: code style

* refactor: split out insert mod from sql and minor changes by CR

* refactor: replace with_context with context
2022-06-17 11:36:49 +08:00
evenyag
7700a167f2 chores: Address CR comment 2022-06-14 16:53:12 +08:00
evenyag
268598eb57 test: Fix VectorBuilder test and add Value data type test 2022-06-10 17:31:59 +08:00
evenyag
69b39e7846 feat: Impl BatchIterator for btree memtable
feat: Impl MapIterWrapper

refactor: Rename RowKey to InnerKey
2022-06-09 17:13:02 +08:00
evenyag
4171173b76 feat: Support creating in memory region and writing to memtable (#40)
* chore(store-api): Fix typo in region comments

* feat(storage): Init storage crate

* feat(store-api): Make some method async

* feat(storage): Blank StorageEngine implementation

* feat(storage): StorageEngine returns owned SchemaRef

* feat: pub use arrow in datatypes

* feat(store-api): Implement RegionMetadata

* feat(storage): Impl create region in memory.

* chore(object-store): Format cargo toml

* chore(storage): Log on region created

* feat: Impl CowCell

* feat: Store id to cf meta mapping

* refactor: Refactor version and rename it to VersionControl

* feat: Impl write batch for put, refactor column family

* feat(storage): Skeleton of writing to memtable

* refactor(storage): MemTable returns MemTableSchema

* feat: Add ColumnSchema and conversion between schema and arrow's schema

* feat: Validate put data

* feat: Valid schema of write batch

* feat: insert memtable WIP

* feat: Impl Inserter for memtable

* feat(datatypes): Implement Eq/Ord for Value

feat: Implement Ord/Eq for Bytes/StringBytes and Deref for Bytes

test: Test Value::from()

* feat: Define BTreeMemTable

* Fix: Rename get/get_unchecked to try_get/get and fix get not consider null.

* feat: Impl BTreeMemTable::write()

* refactor: Remove useless ColumnFamilyHandle now

* chore: Clean comment

* feat(common): Add from `String/&str/Vec<u8>/&[u8]` for Value

* test(storage): Add tests for WriteBatch

* chore: Fix clippy

* feat: Add builder for RowKey/ColumnFamilyDescriptor

* test: Add test for metadata

* chore: Fix clippy

* test: Add test for region and engine

* chore: Fix clippy

* chore: Address CR comment
2022-06-09 16:50:02 +08:00
evenyag
8fe577649f feat: Constructing Bytes/StringBytes from Vec<u8>/&[u8]/String/&str (#41) 2022-06-08 14:25:25 +08:00
dennis zhuang
f7136819fc function crate and scalars function (#39)
* feat: adds scalars mod and enhance vectors

* temp commit

* fix compile error

* Impl pow function with new udf framework

* Adds common-function crate and impl scalars function

* fix: remove used code

* test: adds test for function crate and refactor vectors

* fix: fmt style

* feat: impl numpy.clip function

* feat: improve clip function returning int64 type when arguments do not have float type

* feat: adds more test for vectors

* feat: adds replicate method test for primitive vector

* fix: by code review

* feat: clip function returns uint64 when all arguments type are unsigned

* refactor: improve vectors#only_null

* fix: clippy warning

* fix: clippy warning

* fix: clip should return float64 when arguments have both signed and unsigned types
2022-06-08 13:15:22 +08:00
evenyag
23f235524d feat: Implements validity() and null_count() for Vector (#38)
* feat: Add validity() to Vector

* test(datatypes): Add more tests and fix get_data() not returns None for null
2022-06-01 20:55:58 +08:00
dennis zhuang
a2331366f6 feat: adds adds register_udf api to query engine and refactor datatypes (#34)
* feat: adds ColumnarValue and refactor vectors

* fix: ConcreteDataType compile error

* feat:adds udf/function mods

* feat: adds test for common_query crate

* feat: adds register_udf api to query engine

* feat: adds common_query::error test

* refactor: by CR comments

* refactor: adds impl_new_concret_type_functions! macro to reduce boilerplate codes

* fix: typo
2022-05-24 16:50:56 +08:00
evenyag
1594da337f feat(store-api): Prototype of storage engine api (#33) 2022-05-20 18:51:51 +08:00
Lei, Huang
e75a54b766 feat: impl From arrow array for exsisting vectors (#32)
* feat: impl From arrow array for exsisting vectors

* fix: review comments

* feat: clippy forbid prints
2022-05-19 16:10:00 +08:00
dennis zhuang
b0d2e2e91b feat: adds ConcretDataType and more datatypes impl (#31)
* feat: adds ConcretDataType and impl binary/boolean/null types and vectors

* feat: adds String to ConcretDataType

* docs:  ConcretDataType::from_arrow_type may panic
2022-05-19 11:21:11 +08:00
Lei, Huang
519cbc832a feat: add StringVector datatype (#28) 2022-05-18 14:49:36 +08:00
Lei, Huang
bd4fe1f5bc feat: RecordBatch serialization (#26) 2022-05-17 17:01:00 +08:00
Lei, Huang
641f4b39bd feat: add serialization support to existing data types (#24)
* feat: add serialization support to existing data types

* fix: refactor errors mod to error. fix other review comments
2022-05-11 17:17:27 +08:00
dennis zhuang
f0db878bf5 feat: http server and cmd crate etc. (#15)
* feat: adds cmd crate and http server

* feat: impl sql http handler

* feat: convert all arrow array types

* feat: adds query test

* feat: adds test for datanode

* fix: format

* feat: refactor state.rs

* feat: adds collect test

* fix: by code review

* fix: style
2022-05-06 17:34:29 +08:00
dennis zhuang
3a2f794f6c feat: query engine impl on datafusion (#10)
* feat: query engine impl on datafusion

* feat: adds physical_optimizer, physical_planner and executor

* feat: impl adpaters between datafuion and greptime query engine core APIs.

* feat: impl PhysicalPlanAdapter and ExecutionPlanAdapter

* feat: rename table datafusion mod to adapter

* fix: clippy warning

* fix: conflicts with develop branch

* feat: add database mod

* fix: CR comment

* fix: by CR comments

* fix: conflicts with develop branch

* fix: by CR comments
2022-04-26 15:17:32 +08:00
evenyag
b505c87e8d refactor: Rename base to common-base 2022-04-26 11:48:22 +08:00
evenyag
58f19b2a33 refactor(datatypes): pub use some types 2022-04-25 17:03:05 +08:00
evenyag
49081d818c refactor: Divide common crate into small crates 2022-04-25 17:03:05 +08:00
evenyag
7f075d2c0f refactor: Rename scalar mod to scalars 2022-04-25 17:03:05 +08:00
evenyag
6535f5c4cd refactor: Rename CreateDataType to DataTypeBuilder 2022-04-25 17:03:05 +08:00
evenyag
7a015d4f2a feat: Allow DataType/Vector converting into arrow's type 2022-04-25 17:03:05 +08:00
evenyag
445fd75712 feat: Implement ScalarVector for primitive/binary vector 2022-04-25 17:03:05 +08:00
evenyag
c9c5e69adf feat: Implement BinaryType and BinaryVector 2022-04-25 17:03:05 +08:00
evenyag
d20191572e feat: Implement PrimitiveType and PrimitiveVector 2022-04-25 17:03:05 +08:00
evenyag
1fa0b4e3f9 chore: Setup code skeleton of datanode 2022-04-21 11:40:46 +08:00
evenyag
ced9a7c97f chore: Create datanode, datatypes, log-store, object-store, query, table crates 2022-04-20 16:34:22 +08:00