* refactor: dependency, from frontend depends on datanode to datanode depends on frontend
* wip: start frontend in datanode
* wip: migrate create database to frontend
* wip: impl alter table
* fix: CR comments
* feat: add table id and region ids field to CreateExpr
* chore: rebase develop
* refactor: frontend catalog should set from datanode
* feat: gRPC AddColumn request support add multi columns
* wip: move create table and create-on-insertion to frontend
* wip: error handling
* fix: some unit tests
* fix: all unit tests
* chore: merge develop
* feat: add create/alter-on-insertion to dist_insert/sql_dist_insert
* fix: add region number/catalog/schema to InsertExpr
* feat: add handle_create_table/handle_create_database...
* fix: remove catalog from insert expr
* fix: CR comments
* fix: when running in standalone mode, mysql opts and postgres opts should pass to frontend so that auctually running service can change the port to listen on
* refactor: add a standalone subcommand, move frontend start stuff to cmd package
* chore: optimize create table failure logs
* docs: change readme
* docs: update readme
* refactor: dependency, from frontend depends on datanode to datanode depends on frontend
* wip: start frontend in datanode
* wip: migrate create database to frontend
* wip: impl alter table
* fix: CR comments
* feat: move time index metadata from schema into field
* chore: remove useless code
* test: test select with column alias
* fix: conflicts with develop branch
* test: add test
* test: order by timestamp to ensure query results order
* fix: comment
* chore: refactor dir for local catalog manager
* refactor: CatalogProvider returns Result
* refactor: SchemaProvider returns Result
* feat: add kv operations to remote catalog
* chore: refactor some code
* feat: impl catalog initialization
* feat: add register table and register system table function
* refactor: add table_info method for Table trait
* chore: add some tests
* chore: add register schema test
* chore: fix build issue after rebase onto develop
* refactor: mock to separate file
* build: failed to compile
* fix: use a container struct to bridge KvBackend and Accessor trait
* feat: upgrade opendal to 0.17
* test: add more tests
* chore: add catalog name and schema name to table info
* chore: add catalog name and schema name to table info
* chore: rebase onto develop
* refactor: common-catalog crate
* refactor: remove remote catalog related files
* fix: compilation
* feat: add table version to TableKey
* feat: add node id to TableValue
* fix: some CR comments
* chore: change async fn create_expr_to_request to sync
* fix: add backtrace to errors
* fix: code style
* fix: CatalogManager::table also requires both catalog_name and schema_name
* chore: merge develop
* refactor: add table_info method for Table trait
* feat: add table_info method to Table trait
* test: add more unit test
* fix: impl table_info for SystemTable
* test: fix failing test
* feat: Adds ColumnDefaultConstraint::create_default_vector
ColumnDefaultConstraint::create_default_vector is ported from
MitoTable::try_get_column_default_constraint_vector.
* refactor: Replace try_get_column_default_constraint_vector by create_default_vector
* style: Remove unnecessary map_err in MitoTable::insert
* feat: Adds compat_write
For column in `dest_schema` but not in `write_batch`, this method would insert a
vector with default value to the `write_batch`. If there are columns not in
`dest_schema`, an error would be returned.
* chore: Add info log to RegionInner::alter
* feat(storage): RegionImpl::write support request with old version
* feat: Add nullable check when creating default value
* feat: Validate nullable and default value
* chore: Modify PutOperation comments
* chore: Make ColumnDescriptor::is_nullable readonly and validate name
* feat: Use CompatWrite trait to replace campat::compat_write method
Adds a CompactWrite trait to support padding columns to WriteBatch:
- The WriteBatch and PutData implements this trait
- Fix the issue that WriteBatch::schema is not updated to the
schema after compat
- Also validate the created column when adding to PutData
The WriteBatch would also pad default value to missing columns in
PutData, so the memtable inserter don't need to manually check whether
the column is nullable and then insert a NullVector. All WriteBatch is
ensured to have all columns defined by the schema in its PutData.
* feat: Validate constraint by ColumnDefaultConstraint::validate()
The ColumnDefaultConstraint::validate() would also ensure the default
value has the same data type as the column's.
* feat: Use NullVector for null columns
* fix: Fix BinaryType returns wrong logical_type_id
* fix: Fix tests and revert NullVector for null columns
NullVector doesn't support custom logical type make it hard to
encode/decode, which also cause the arrow/protobuf codec of write batch
fail.
* fix: create_default_vector use replicate to create vector with default value
This would fix the test_codec_with_none_column_protobuf test, as we need
to downcast the vector to construct the protobuf values.
* test: add tests for column default constraints
* test: Add tests for CompatWrite trait impl
* test: Test write region with old schema
* fix(storage): Fix replay() applies metadata too early
The committed sequence of the RegionChange action is the sequence of the
last entry that use the old metadata (schema). During replay, we should
apply the new metadata after we see an entry that has sequence greater
than (not equals to) the `RegionChange::committed_sequence`
Also remove duplicate `set_committed_sequence()` call in
persist_manifest_version()
* chore: Removes some unreachable codes
Also add more comments to document codes in these files
* refactor: Refactor MitoTable::insert
Return error if we could not create a default vector for given column,
instead of ignoring the error
* chore: Fix incorrect comments
* chore: Fix typo in error message
* feat: add type conversion optimizer
* feat: add expr rewrite logical plan optimizer
* chore: add some doc
* fix: unit test
* fix: time zone issue in unit tests
* chore: add more tests
* fix: some CR comments
* chore: rebase develop
* chore: fix unit tests
* fix: unit test use timestamp with time zone
* chore: add more tests
* feat: ensure time index column can't be included in primary key
* feat: sql parser supports show tables statement
* feat: impl show databases and show tables, #183
* feat: impl like expression for show databases/tables and add tests
* fix: typo
* fix: address CR problems
* feat: frontend instance
* no need to carry column length in `Column` proto
* add more tests
* rebase develop
* create a new variant with already provisioned RecordBatches in Output
* resolve code review comments
* new frontend instance does not connect datanode grpc
* add more tests
* add more tests
* rebase develop
Co-authored-by: luofucong <luofucong@greptime.com>
* feat: implement alter table
* Currently we have no plans to support altering the primary keys (maybe never), so removed the related codes.
* make `alter` a trait function in table
* address other CR comments
* cleanup
* rebase develop
* resolve code review comments
Co-authored-by: luofucong <luofucong@greptime.com>
* feat: upgrade rust to nightly-2022-07-14
* style: Fix some clippy warnings
* style: clippy fix
* style: fix clippy
* style: Fix clippy
Some PartialEq warnings have been work around using cfg_attr test
* feat: Implement Eq and PartialEq for PrimitiveType
* chore: Remove unnecessary allow
* chore: Remove usage of cfg_attr for PartialEq
* catalog manager allocates table id
* rebase develop
* add some tests
* add some more test
* fix some cr comments
* insert into system catalog
* use slice pattern to simplify code
* add optional dependencies
* add sql-to-request test
* successfully recover
* fix unit tests
* rebase develop
* add some tests
* fix some cr comments
* fix some cr comments
* add a lock to CatalogManager
* feat: add gmt_created and gmt_modified columns to system catalog table
Implement catalog manager that provides a vision of all existing tables while instance start. Current implementation is based on local table engine, all catalog info is stored in an system catalog table.
* feat: add buffer abstraction and rewrite entry encode/decode process
* add some tests
* remove pad.rst
* fix some comments
* fix comments
* remove mmap mod
* feat: Bytes type implementation switch to bytes::Bytes
* fix: use Bytes::from(String) and Bytes::from(Vec<u8>)
* feat: add new method to Entry trait
* Impl TableEngine, bridge to storage
* Impl sql handler to process insert sql
* fix: minor changes and typo
* test: add datanode test
* test: add table-engine test
* fix: code style
* refactor: split out insert mod from sql and minor changes by CR
* refactor: replace with_context with context
