* feat: Change signature of the Region::alter method
* refactor: Add builders for ColumnsMetadata and ColumnFamiliesMetadata
* feat: Support altering the region metadata
Altering the region metadata is done in a copy-write fashion:
1. Convert the `RegionMetadata` into `RegionDescriptor` which is more
convenient to mutate
2. Apply the `AlterOperation` to the `RegionDescriptor`. This would
mutate the descriptor in-place
3. Create a `RegionMetadataBuilder` from the descriptor, bump the
version and then build the new metadata
* feat: Implement altering table using the new Region::alter api
* refactor: Replaced wal name by region id
Region id is cheaper to clone than name
* chore: Remove pub(crate) of build_xxxx in engine mod
* style: fix clippy
* test: Add tests for AlterOperation and RegionMetadata::alter
* chore: ColumnsMetadataBuilder methods return &mut Self
* feat: add type conversion optimizer
* feat: add expr rewrite logical plan optimizer
* chore: add some doc
* fix: unit test
* fix: time zone issue in unit tests
* chore: add more tests
* fix: some CR comments
* chore: rebase develop
* chore: fix unit tests
* fix: unit test use timestamp with time zone
* chore: add more tests
* feat: ensure time index column can't be included in primary key
* feat: sql parser supports show tables statement
* feat: impl show databases and show tables, #183
* feat: impl like expression for show databases/tables and add tests
* fix: typo
* fix: address CR problems
* feat: Handle empty NullVector in replicate_null
* chore: Rename ChunkReaderImpl::sst_reader to batch_reader
* feat: dedup reader wip
* feat: Add BatchOp
Add BatchOp to support dedup/filter Batch and implement BatchOp for
ProjectedSchema.
Moves compare_row_of_batch to BatchOp::compare_row.
* feat: Allow Batch has empty columns
* feat: Implement DedupReader
Also add From<MutableBitmap> for BooleanVector
* test: Test dedup reader
Fix issue that compare_row compare by full key not row key
* chore: Add comments to BatchOp
* feat: Dedup results from merge reader
* test: Test merge read after flush
* test: Test merge read after flush and reopen
* test: Test replicate empty NullVector
* test: Add tests for `ProjectedSchema::dedup/filter`
* feat: Filter empty batch in DedepReader
Also fix clippy warnings and refactor some codes
The log store use start sequence instead of file start id to filter
log stream. Add more tests about flush, including flush empty memtable
and reopen after flush
* wip add predicate definition
* fix value move
* implement predicate and prune
* impl filter push down in chunk reader
* add more expr tests
* chore: rebase develop
* fix: unit test
* fix: field name/index lookup when building pruning stats
* chore: add some meaningless test
* fix: remove unnecessary extern crate
* fix: use datatypes::schema::SchemaRef
* feat: Dedup vector
* refactor: Re-export Date/DateTime/Timestamp
* refactor: Named field for ListValueRef::Ref
Use field val instead of tuple for variant ListValueRef::Ref to keep
consistence with ListValueRef::Indexed
* feat: Implement ScalarVector for ListVector
Also implements ScalarVectorBuilder for ListVectorBuilder, Scalar for
ListValue and ScalarRef for ListValueRef
* test: Add tests for ScalarVector implementation of ListVector
* feat: Implement dedup using match_scalar_vector
* refactor: Move dedup func to individual mod
* chore: Update ListValueRef comments
* refactor: Move replicate to VectorOp
Move compute operations to VectorOp trait and acts as an super trait of
Vector. So we could later put dedup/filter methods to VectorOp trait,
avoid to define too many methods in Vector trait.
* refactor: Move scalar bounds to PrimitiveElement
Move Scalar and ScalarRef trait bounds to PrimitiveElement, so for each
native type which implements PrimitiveElement, its PrimitiveVector
always implements ScalarVector, so we could use it as ScalarVector
without adding additional trait bounds
* refactor: Move dedup to VectorOp
Remove compute mod and move dedup logic to operations::dedup
* feat: Implement VectorOp::filter
* test: Move replicate test of primitive to replicate.rs
* test: Add more replicate tests
* test: Add tests for dedup and filter
Also fix NullVector::dedup and ConstantVector::dedup
* style: fix clippy
* chore: Remove unused scalar.rs
* test: Add more tests for VectorOp and fix failed tests
Also fix TimestampVector eq not implemented.
* chore: Address CR comments
* chore: mention vector should be sorted in comment
* refactor: slice the vector directly in replicate_primitive_with_type
* feat: initial commit of postgres protocol adapter
* initial commit of postgres server
* feat: use common_io runtime and correct testcase
* fix previous tests
* feat: adopt pgwire api changes and add support for text encoded data
* feat: initial integration with datanode
* test: add feature flag to test
* fix: resolve lint warnings
* feat: add postgres feature flags for datanode
* feat: add support for newly introduced timestamp type
* feat: adopt latest datanode changes
* fix: address clippy warning for flattern scenario
* fix: make clippy great again
* fix: address issues found in review
* chore: sort dependencies by name
* feat: adopt new Output api
* fix: return error on unsupported data types
* refactor: extract common code dealing with record batches
* fix: resolve clippy warnings
* test: adds some unit tests postgres handler
* test: correct test for cargo update
* fix: update query module name
* test: add assertion for error content
* feat: supports DateTime, Date and Timestamp column type to be returned by py scripts
* feat: improve coprocessor compiler, make it work better
* fix: comments
* fix: typo
* Update src/script/src/python/vector.rs
Co-authored-by: LFC <bayinamine@gmail.com>
* Update src/script/src/python/coprocessor.rs
Co-authored-by: LFC <bayinamine@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
* fix: forbid use int64 as timestamp column data type
* fix unit test
* fix unit tests
* change gmt_created and gmt_modified data type in system tables to timestamp
* also change data type in readme
* feat: frontend instance
* no need to carry column length in `Column` proto
* add more tests
* rebase develop
* create a new variant with already provisioned RecordBatches in Output
* resolve code review comments
* new frontend instance does not connect datanode grpc
* add more tests
* add more tests
* rebase develop
Co-authored-by: luofucong <luofucong@greptime.com>
* feat: impl scripts table and /execute restful api
* fix: test failures
* fix: test failures
* feat: impl /run_script API
* refactor: rename run_script api to run-script and test script manager
* fix: remove println
* refactor: error mod
* refactor: by CR comments
* feat: rebase develop and change timestamp/gmt_crated/gmt_modified type to timestamp
* refactor: use assert_eq instread of assert
* doc: fix comment in Script#execute function
LogicalTypeId to ConcreteDataType is only allowed in tests, since some
additional info is not stored in LogicalTypeId now. It is just an id, or
kind, not contains full type info.
* test: Fix merge tests
The merge tests still use Int64Vector for timestamp, which should
use TimestampVector instead.
* test: Test Debug format for Source::Reader
Mainly for improve code coverage
* wip: impl timestamp data type
* add timestamp vectors
* adapt to recent changes to vector module
* fix all unit test
* rebase develop
* fix slice
* change default time unit to millisecond
* add more tests
* fix some CR comments
* fix some CR comments
* fix clippy
* fix some cr comments
* fix some CR comments
* fix some CR comments
* remove time unit in LogicalTypeId::Timestamp
* feat: Check columns when constructing Batch
* feat: Merge reader skeleton
* test: Add tests for MergeReader
* feat: Use get_ref to compare row
* feat: Implement MergeReader
* test: Add more tests
* feat: Use MergeReader to implement ChunkReader
Now the ChunkReaderImpl use MergeReader as default reader. Also add more tests to MergeReader.
* docs: Describe the merge algo in merge.rs
Ports the doc comments from kudu to merge.rs to describe the idea of the
merge algorithm we used.
* test: Fix unit tests
* chore: Address CR comments
Panics if number of columns in batch is not equal to `BatchBuilder`'s
* chore: Address CR comments
* chore: Implement Debug and add test for Node
* feat: Add tests for different batch
Add region scan test with different batch size
* fix: Fix table scan only returns one batch
* style: Fix clippy
* test: Add tests to scan table with rows more than batch size
* fix: Fix MockChunkReader never stop
* feat: Impl cmp_element() for Vector
* chore: Add doc comments to MutableVector
* feat: Add create_mutable() to DataType
Add `create_mutable()` to create a MutableVector for each DataType.
Implement ListVectorBuilder and NullVectorBuilder for ListType and
NullType.
* feat: Add ValueRef
ValueRef is a reference to value, could be used to avoid some allocation
when getting data from Vector. To support ValueRef, also implement a
ListValueRef for ListValue, but comparision of ListValueRef still
requires some allocation, due to the complexity of ListValue and
ListVector.
Impl some From trait for ValueRef
* feat: Implement get_ref for Vector
* feat: Remove cmp_element from Vector
`cmp_element` could be replaced by `get_ref` and then compare
* feat: Implement push/extend for PrimitiveVectorBuilder
Implement push_value_ref() and extend_slice_of() for
PrimitiveVectorBuilder.
Also refactor the DataTypeBuilder trait for
primitive types to PrimitiveElement trait, adds necessary cast helper
methods to it.
- Cast a reference to Vector to reference arrow's primitive array
- Cast a ValueRef to primitive type
- Also make PrimitiveElement super trait of Primitive
* feat: Implement push/extend for all vector builders
Implement push_value_ref() and extend_slice_of() for remaining vector
builders. Add some helpful cast method to ValueRef and a method to
cast Value to ValueRef.
Change the behavior of PrimitiveElement::cast_xxx to panic when unable
to cast, since push_value_ref() and extend_slice_of() always panic
when given invalid input data type.
* feat: MutableVector returns error if data type unmatch
* test: Add tests for ValueRef
* feat: Add tests for Vector::get_ref
* feat: NullVector returns error if data type unmatch
* test: Add tests for vector builders
* fix: Fix compile error in python coprocessor
* refactor: Add lifetime param to IntoValueRef
The Primitive trait just use the `IntoValueRef<'static>` bound. Also
rename create_mutable to create_mutable_vector.
* chore: Address CR comments
* feat: Customize PartialOrd/Ord for Value/ValueRef
Panics if values/refs have different data type
* style: Fix clippy
* refactor: Use macro to generate body of ValueRef::as_xxx
* feat: implement alter table
* Currently we have no plans to support altering the primary keys (maybe never), so removed the related codes.
* make `alter` a trait function in table
* address other CR comments
* cleanup
* rebase develop
* resolve code review comments
Co-authored-by: luofucong <luofucong@greptime.com>
