* feat: impl recovering version from manifest for region
* refactor: rename try_apply_edit to replay_edit
* fix: remove println
* fix: address CR problems
* feat: remove Metadata in manifest trait and update region manifest state after recovering
* refactor: Move fields in SharedData to EngineInner
Since `SharedData` isn't shared now, we move all its fields to
EngineInner, and remove the `SharedData` struct, also remove the
unused config field.
* feat: Store RegionSlot in engine's region map
A `RegionSlot` has three possible state:
- Opening
- Creating
- Ready (Holds the `RegionImpl`)
Also use the `RegionSlot` as a placeholder in the region map to indicate
the region is opening/creating, so another open/create request will
fail immediately. The `SlotGuard` is used to clean the slot if we failed
to create/open the region.
* feat: Add a blank method `RegionImpl::open`
* feat: Remove MetadataId from Manifest
Now metadata id of manifest is unused, also unnecessary as we have
manifest dir to build the manifest, but constructing the manifest
still needs a passing region id as argument, which is unavailable
during opening region. So we remove the metadata id from manifest so
`region_store_config()` don't need region id as input anymore
* feat: Remove region id from logstore::Namespace and Wal
This is necessary for implementing open, since we don't have region
id this time, but we need to build Wal and its logstore namespace. Now
this is ok as id is not actually used by logstore.
* feat: Setup `open_region` code skeleton
* fix: Fix write stall blocks flush applying version
refactor: Use store config to help constructing Region
chore: Address CR comments
* feat: adds manifest protocol supporting and refactor region metadata protocol
feat: ignore sqlparser log
refactor: PREV_VERSION_KEY constant
refactor: minor change for checking readable/writable
fix: address CR problems
refactor: use binary literal
Co-authored-by: Dennis Zhuang <killme2008@gmail.com>
* feat: UDAF implementation backed by DataFusion.
Directly Transplant DataFusion's UDAF related structs, traits and functions, like `AggregateUDF`, `Accumulator` or `create_udaf` etc.
Implement median UDAF on top of it and used in unit testing.
Refs: #61
* feat: UDAF made generically
Refs: #61
* fix: cargo fmt
* fix: use prelude
* fix: uniform the name
* fix: move maybe commonly used functions together
* fix: make comments more clear
* fix: resolve conversations in CR
* fix: store input types in AccumulatorCreator, and use ScalarVector's iterator
* feat: introducing List value and List datatype
* refactor: use ArcSwap instead of Mutext
* refactor: shorten some namings
* refactor: move median UDAF out of tests
* refactor: rename
* feat: aggregate function registry
* fix: make `Value` satisfy ordering again
* fix: clippy warnings
* doc: add "how to write aggregate function"
* fix: address PR comments
* fix: trying to get rid of unwraps
Co-authored-by: luofucong <luofucong@greptime.com>
* move interp from test to numpy
* move interp from test to numpy
* move interp from test to numpy
* move interp from test to numpy
* move interp from test to numpy
* feat: implement InsertExpr
* 1.InsertExpr reverted to previous version 2.add InsertBatch message
* add two SemanticTypes: TAG, TIMESTAMP
* chore: format proto files
* chore: add some comments about "Column"
* fix: rename "semanticType" -> "semantic_type"
* fix: unique number in InsertBatch
* fix: type of f64_values
* chore: move insertbatch and column to insert.proto
* chore: rename "ExprHeader" to "Header"
* fix: ExprHeader not found in this scope
* feat: add grpc impl
* feat: add grpc server
* some ut
* verson format: a.b
* code style
* admin request/response
* by cr
* admin api
* by cr
* chore: by cr
* chore: by cr
* feat: add buffer abstraction and rewrite entry encode/decode process
* add some tests
* remove pad.rst
* fix some comments
* fix comments
* remove mmap mod
* feat: Bytes type implementation switch to bytes::Bytes
* fix: use Bytes::from(String) and Bytes::from(Vec<u8>)
* feat: add new method to Entry trait
* feat: improve try_into_vector function
* feat: impl memory_size function for vectors
* fix: forgot memory_size assertion in null vector test
* feat: use LargeUtf8 instead of utf8 for string, and rename LargeBianryArray to BinaryArray
* feat: memory_size only calculates heap size
* feat: impl scanning data from storage for MitoTable
* adds test mod to setup table engine test
* fix: comment error
* fix: boyan -> dennis in todo comments
* fix: remove necessary send in BatchIteratorPtr
* feat: Maintain last sequence in VersionControl
* refactor(recordbatch): Replace `Arc<Schema>` by SchemaRef
* feat: Memtable support filter rows with invisible sequence
* feat: snapshot wip
* feat: Implement scan for SnapshotImpl
* test: Add a test that simply puts and scans a region
* chore: Fix clippy
* fix(memtable): Fix memtable returning duplicate keys
* test(memtable): Add sequence visibility test
* test: Add ValueType test
* chore: Address cr comments
* fix: Fix value is not storing but adding to committed sequence
* Impl TableEngine, bridge to storage
* Impl sql handler to process insert sql
* fix: minor changes and typo
* test: add datanode test
* test: add table-engine test
* fix: code style
* refactor: split out insert mod from sql and minor changes by CR
* refactor: replace with_context with context
* add log store impl
* add some test
* delete failing test
* fix: concurrent close issue
* feat: use arcswap to replace unsafe AtomicPtr
* fix: use lock to protect rolling procedure.
fix: use try_recv to replace poll_recv on appender task.
* chores: 1. use direct tmp dir instead of creating TempDir instance; 2. inline some short function; 3. rename some structs; 4. optimize namespace to arc wrapper inner struct.
* chore(store-api): Fix typo in region comments
* feat(storage): Init storage crate
* feat(store-api): Make some method async
* feat(storage): Blank StorageEngine implementation
* feat(storage): StorageEngine returns owned SchemaRef
* feat: pub use arrow in datatypes
* feat(store-api): Implement RegionMetadata
* feat(storage): Impl create region in memory.
* chore(object-store): Format cargo toml
* chore(storage): Log on region created
* feat: Impl CowCell
* feat: Store id to cf meta mapping
* refactor: Refactor version and rename it to VersionControl
* feat: Impl write batch for put, refactor column family
* feat(storage): Skeleton of writing to memtable
* refactor(storage): MemTable returns MemTableSchema
* feat: Add ColumnSchema and conversion between schema and arrow's schema
* feat: Validate put data
* feat: Valid schema of write batch
* feat: insert memtable WIP
* feat: Impl Inserter for memtable
* feat(datatypes): Implement Eq/Ord for Value
feat: Implement Ord/Eq for Bytes/StringBytes and Deref for Bytes
test: Test Value::from()
* feat: Define BTreeMemTable
* Fix: Rename get/get_unchecked to try_get/get and fix get not consider null.
* feat: Impl BTreeMemTable::write()
* refactor: Remove useless ColumnFamilyHandle now
* chore: Clean comment
* feat(common): Add from `String/&str/Vec<u8>/&[u8]` for Value
* test(storage): Add tests for WriteBatch
* chore: Fix clippy
* feat: Add builder for RowKey/ColumnFamilyDescriptor
* test: Add test for metadata
* chore: Fix clippy
* test: Add test for region and engine
* chore: Fix clippy
* chore: Address CR comment
* feat: adds scalars mod and enhance vectors
* temp commit
* fix compile error
* Impl pow function with new udf framework
* Adds common-function crate and impl scalars function
* fix: remove used code
* test: adds test for function crate and refactor vectors
* fix: fmt style
* feat: impl numpy.clip function
* feat: improve clip function returning int64 type when arguments do not have float type
* feat: adds more test for vectors
* feat: adds replicate method test for primitive vector
* fix: by code review
* feat: clip function returns uint64 when all arguments type are unsigned
* refactor: improve vectors#only_null
* fix: clippy warning
* fix: clippy warning
* fix: clip should return float64 when arguments have both signed and unsigned types
