mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-03 18:45:35 +00:00
75bd8e9ce6fa554c2da3fa7d72dc63c8b50e99ce
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 dennis zhuang
|
d59725b04a |
test(sqlness): run environments concurrently and drop redundant restarts and sleeps (#9333)
* test(sqlness): run environments concurrently with external backends The runner forced both environment and instance parallelism to 1 whenever etcd/PG/MySQL or an external Kafka was set up, so the standalone and distributed environments ran one after the other. Only distributed uses the kv backend, and the two environments use different Kafka topic prefixes, so they can run at the same time. Keep one instance per environment, since instances would share the metadata table and Kafka topics. Runs with a runner-managed Kafka, an external server, a test filter, or `-j 1` stay fully serial. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(sqlness): drop redundant restarts and sleeps - flow_rebuild: drop the R2 block, which repeats R1 from the same state, and the restart in R5, whose result R1 already asserts. - flow_advance_ttl, flow_view, ttl_instant: drop sleeps that no assertion depends on. - flow_basic, flow_null, flow_call_df_func: drop the bytes_log section (covered by flow_insert), a duplicate state_size query, an unchecked insert, and flushes that run with no new data. - alter_table_options, skip_wal: share restarts between independent tables. - session_skip_wal, copy_skip_wal: check every case after a single restart. Tables copied or inserted with skip_wal = true first get a WAL row that is truncated, so the check that truncated WAL entries are not replayed stays. - region_statistics: wait once for the statistics of all three tables. - build_index_table: fold its index_size checks into build_index_table_restart, which runs the same fixture and waits. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(sqlness): restore session_skip_wal and copy_skip_wal Checking every case after a single restart dropped two state transitions the original cases cover: recovering a region whose memtable only held skipped rows, and writing to the recovered region before restarting again. Restore the original cases. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(sqlness): restore the one-restart check in flow_auto_sink_table #5112 checked the auto-created sink before a restart and the flow and sink after it. #5987 moved the restart in front of the first SHOW and added a second one. Flow recovery only creates the sink when it is missing, so the second restart recovers from the same persisted state as the first. Restore the original before/after check with one restart. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(sqlness): keep one instance per environment with external store addresses Instances of the distributed environment share the etcd behind --store-addrs, so run one instance per environment when it is set, same as --setup-etcd. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(sqlness): restore the false-branch batch in flow_basic and assert it The kept batch (20, 22) is all above the threshold. The second batch (10, 23) is the only input that hits the false branch of the CASE and makes the flow compute again. Restore it and check the result, which the original case never did. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
 Dhruv Vaishnav
|
7e2a75f771 |
feat: allow re-enabling WAL after disabling (#9130)
* feat: allow re-enabling WAL after disabling Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * test: regenerate skip_wal sqlness result The previous commit changed tests/cases/standalone/common/skip_wal.sql without regenerating the matching .result, so every Sqlness suite failed on the mismatch. tests/cases/distributed/common is a symlink to standalone/common, so the single stale file accounted for all five failing variants. Regenerated from a real run. The recorded output now covers the cases the .sql added: * A table created with skip_wal = 'true' has no real WAL provider, so enabling WAL is refused with Unsupported rather than the previous InvalidArguments. * A RaftEngine-backed table accepts the true -> false transition, and repeating it is a no-op. * SHOW CREATE TABLE reports skip_wal = 'false' after the transition, including after a restart. * A row written while WAL was skipped is absent after restart, while a row written after WAL was restored survives. Verified locally with nothing else running on the machine: sqlness skip_wal passes in both the standalone and distributed environments, and the regenerated file is byte-identical to the output of an independent earlier run. Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * test: strengthen skip_wal provider coverage Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * Fix WAL provider handling during re-enable Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * Refactor WAL provider check Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> --------- Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> |
||
 jeremyhi
|
c7c1a61d56 |
feat: support enabling skip_wal with ALTER TABLE (#8730)
* feat: support enabling skip_wal with alter table Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: cover skip wal on metric physical region Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: retry skip wal alter on route changes Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: address skip wal review feedback Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: preserve skip wal create options Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: prefer typed skip wal option Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: address skip wal review comments Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: canonicalize legacy skip wal option Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: keep typed skip wal canonical Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: simplify skip wal option tracking Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> |
||
 Lei, HUANG
|
40b52f3b13 |
feat(mito): allow skipping wal while creating tables (#5740)
* chore: add Noop Wal option * remove: WalOptionsAllocator::alloc method * feat/no-op-wal: ### Add Noop WAL Option - **`engine.rs`, `opener.rs`, `wal.rs`, `entry_reader.rs`, `handle_write.rs`, `provider.rs`**: - Introduced a new `WalOptions::Noop` variant to handle scenarios where no write-ahead logging is required. - Implemented `NoopEntryReader` to provide a no-operation entry reader. - Updated logic to skip WAL operations for regions with `Noop` option. - Added `Provider::Noop` to handle `Noop` operations in the provider logic. * feat/no-op-wal: ### Add `skip_wal` Option to Table Metadata - **Enhancements in `table_meta.rs`**: - Added a `skip_wal` parameter to the `create_wal_options` function to allow skipping WAL writes. - Updated the `create_table_route` function to utilize the `skip_wal` option from `table_info.meta.options`. - **Updates in `wal_options_allocator.rs`**: - Modified `alloc_batch` to handle the `skip_wal` flag, setting WAL options to `Noop` when true. - Added a test case `test_allocator_with_skip_wal` to verify the `skip_wal` functionality. - **Changes in `requests.rs`**: - Introduced `skip_wal` in `TableOptions` and added parsing logic. - Updated `TableOptions` display to include `skip_wal`. These changes introduce the ability to skip WAL writes for tables, enhancing flexibility in table metadata management. * feat/no-op-wal: **Add WAL Option Handling and Table Option Validation** - **`handle_write.rs`**: Introduced a check for `WalOptions::Noop` in the `RegionWorkerLoop` to skip WAL writing for regions with this option. - **`requests.rs`**: Added `SKIP_WAL_KEY` to the list of valid table options for enhanced table configuration validation. * feat/no-op-wal: ### Update WAL Options Allocation - **`key.rs`**: Modified the `allocate_region_wal_options` function to include an additional boolean parameter, enhancing the allocation logic. - **`wal_options_allocator.rs`**: Simplified the `test_allocator_with_skip_wal` test by removing unnecessary variable declarations and directly using `WalOptionsAllocator::RaftEngine`. These changes improve the flexibility and efficiency of WAL options allocation in the system. * chore: reformat code * feat/no-op-wal: **Enhancement:** Conditional Addition of `SKIP_WAL_KEY` in `requests.rs` - Updated `TableOptions` implementation in `requests.rs` to conditionally add `SKIP_WAL_KEY` to `key_vals` only when `self.skip_wal` is true, optimizing the key-value pair generation. * feat/no-op-wal: Update `requests.rs` tests to reflect changes in `skip_wal` option - Modified test assertions in `requests.rs` to remove `skip_wal=false` from expected strings. - Added a new test case to verify `skip_wal=true` is correctly represented in `TableOptions`. * feat/no-op-wal: Add Debug Logging and Improve Error Handling for WAL and Table Options • Introduced debug logging in wal.rs to skip obsolete regions, enhancing traceability. • Improved error handling in requests.rs by replacing warn with error propagation for invalid skip_wal values. • Added new test cases for skip_wal functionality, including SQL scripts and expected results, to ensure correct behavior and validation of the changes. |