* feat: memtable support filter pushdown to prune primary keys
* fix: switch to next time series when pk not selected
* fix: allow predicate evaluation failure
* fix: some clippy warnings
* fix: panic when no primary key in schema
* feat: cache decoded record batch for primary key
* refactor: use arcswap instead of rwlock
* fix: format toml
* test: test different order
* test: add tests for missing and invalid columns
* fix: do not skip schema validation while missing columns
* chore: use field_columns()
* test: add tests for different column order
* feat: add vector_cache to CacheManager
* feat: cache repeated vectors
* feat: skip decoding pk if output doesn't contain tags
* test: add TestRegionMetadataBuilder
* test: test ProjectionMapper
* test: test vector cache
* test: test projection mapper convert
* style: fix clippy
* feat: do not cache vector if it is too large
* docs: update comment
* feat: merge by heap
* fix: fix heap order
* feat: avoid pop/push next and refactor some functions
* feat: replace merge_batches and fixe tests
* test: add test that a key is deleted
* fix: skip empty batch
* style: clippy
* chore: fix typos
* test: test on_compaction_finished
* fix: avoid submit same region to compact
* feat: persist and recover compaction time window
* test: fix test
* test: sort like result
* feat: RegionMetadataBuilder allow adding/dropping columns multiple times
* test: test add if not exists/drop if exists
* feat: change validator and add need_alter
* test: fix tests and test need_alter
* test: test alter retry
* feat: open before create
* style: fix clippy
* refactor: move RegionOptions to options mod
* refactor: define compaction strategy in region/options.rs
* feat: use duration for time window
* refactor: rename CompactionStrategy to CompactionOptions
* feat: use serde to parse options
* feat: parse options
* feat: set options on creation/opening
* test: test create/open with options
* chore: remove todo
* feat: get compaction ttl and options from RegionOptions
* style: fix clippy
* chore: Remove unused engine_options
* style: fix clippy
* chore: remove todo
* fix: check version before alter region
* chore: apply suggestions from CR
* Update src/mito2/src/worker/handle_alter.rs
Co-authored-by: dennis zhuang <killme2008@gmail.com>
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com>
* feat: allow multiple waiters in compaction request
* feat: compaction status wip
* feat: track region status in compaction scheduler
* feat: impl compaction scheduler
* feat: call compaction scheduler
* feat: remove status if nothing to compact
* feat: schedule compaction after flush
* feat: set compacting to false after compaction finished
* refactor: flush status only needs region id and version control
* refactor: schedule_compaction don't need region as argument
* test: test flush/scheduler for empty requests
* test: trigger compaction in test
* feat: notify scheduler on truncated
* chore: Apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
* refactor:
1. remove TableIdent, use TableId directly
2. use the latest greptime-proto
3. independently invalidate table id cache and table name cache
* rebase
* fix: resolve PR comments
* fix: resolve PR comments
* chore: set mutable limit to half of the global write buffer size
* refactor: put handle_flush_finished after handle_flush_request
* refactor: rename tests.rs to basic_test.rs
* style: fmt code
* feat: add writable flag to region.
* refactor: rename MitoEngine to MitoEngine::scanner
* feat: add set_writable() to RegionEngine
* feat: check whether region is writable
* feat: make set_writable sync
* test: test set_writable
* docs: update comments
* feat: send result on compaction failure
* refactor: wrap output sender in new type
* feat: on failure
* refactor: use get_region_or/writable_region_or
* refactor: remove send_result
* feat: notify waiters on flush scheduler drop
* test: fix tests
* fix: only alter writable region
* feat: add more info to error messages
* feat: store next column id in procedure
* fix: update next column id for table info
* test: fix add col test
* chore: remove location from invalid request error
* test: update test
* test: fix test
* test: add test for reopen
* feat: last entry id starts from flushed entry id
* fix: store flushed sequence and recover it from manifest
* test: check sequence in alter test
* test: more tests for alter
* feat: impl handle_alter wip
* refactor: move send_result to worker.rs
* feat: skeleton for handle_alter_request
* feat: write requests should wait for alteration
* feat: define alter request
* chore: no warnings
* fix: remove memtables after flush
* chore: update comments and impl add_write_request_to_pending
* feat: add schema version to RegionMetadata
* feat: impl alter_schema/can_alter_directly
* chore: use send_result
* test: pull next_batch again
* feat: convert pb AlterRequest to RegionAlterRequest
* feat: validate alter request
* feat: validate request and alter metadata
* feat: allow none location
* test: test alter
* fix: recover files and flushed entry id from manifest
* test: test alter
* chore: change comments and variables
* chore: fix compiler errors
* feat: add is_empty() to MemtableVersion
* test: fix metadata alter test
* fix: Compaction picker doesn't notify waiters if it returns None
* chore: address CR comments
* test: add tests for alter request
* refactor: use send_result
* feat: compaction component
* feat: mito2 compaction
* Avoid building time range predicates when merge SST files since in TWCS we don't enforce strict time window.
* fix: some CR comments
* minor: change CompactionRequest::senders to an option
* chore: handle compaction finish error
* feat: integrate compaction into region worker
* chore: rebase upstream
* fix: Some CR comments
* chore: Apply suggestions from code review
* style: fix clippy
---------
Co-authored-by: Yingwen <realevenyag@gmail.com>
