40b52f3b13
* chore: add Noop Wal option * remove: WalOptionsAllocator::alloc method * feat/no-op-wal: ### Add Noop WAL Option - **`engine.rs`, `opener.rs`, `wal.rs`, `entry_reader.rs`, `handle_write.rs`, `provider.rs`**: - Introduced a new `WalOptions::Noop` variant to handle scenarios where no write-ahead logging is required. - Implemented `NoopEntryReader` to provide a no-operation entry reader. - Updated logic to skip WAL operations for regions with `Noop` option. - Added `Provider::Noop` to handle `Noop` operations in the provider logic. * feat/no-op-wal: ### Add `skip_wal` Option to Table Metadata - **Enhancements in `table_meta.rs`**: - Added a `skip_wal` parameter to the `create_wal_options` function to allow skipping WAL writes. - Updated the `create_table_route` function to utilize the `skip_wal` option from `table_info.meta.options`. - **Updates in `wal_options_allocator.rs`**: - Modified `alloc_batch` to handle the `skip_wal` flag, setting WAL options to `Noop` when true. - Added a test case `test_allocator_with_skip_wal` to verify the `skip_wal` functionality. - **Changes in `requests.rs`**: - Introduced `skip_wal` in `TableOptions` and added parsing logic. - Updated `TableOptions` display to include `skip_wal`. These changes introduce the ability to skip WAL writes for tables, enhancing flexibility in table metadata management. * feat/no-op-wal: **Add WAL Option Handling and Table Option Validation** - **`handle_write.rs`**: Introduced a check for `WalOptions::Noop` in the `RegionWorkerLoop` to skip WAL writing for regions with this option. - **`requests.rs`**: Added `SKIP_WAL_KEY` to the list of valid table options for enhanced table configuration validation. * feat/no-op-wal: ### Update WAL Options Allocation - **`key.rs`**: Modified the `allocate_region_wal_options` function to include an additional boolean parameter, enhancing the allocation logic. - **`wal_options_allocator.rs`**: Simplified the `test_allocator_with_skip_wal` test by removing unnecessary variable declarations and directly using `WalOptionsAllocator::RaftEngine`. These changes improve the flexibility and efficiency of WAL options allocation in the system. * chore: reformat code * feat/no-op-wal: **Enhancement:** Conditional Addition of `SKIP_WAL_KEY` in `requests.rs` - Updated `TableOptions` implementation in `requests.rs` to conditionally add `SKIP_WAL_KEY` to `key_vals` only when `self.skip_wal` is true, optimizing the key-value pair generation. * feat/no-op-wal: Update `requests.rs` tests to reflect changes in `skip_wal` option - Modified test assertions in `requests.rs` to remove `skip_wal=false` from expected strings. - Added a new test case to verify `skip_wal=true` is correctly represented in `TableOptions`. * feat/no-op-wal: Add Debug Logging and Improve Error Handling for WAL and Table Options • Introduced debug logging in wal.rs to skip obsolete regions, enhancing traceability. • Improved error handling in requests.rs by replacing warn with error propagation for invalid skip_wal values. • Added new test cases for skip_wal functionality, including SQL scripts and expected results, to ensure correct behavior and validation of the changes.
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!