* feat: allow re-enabling WAL after disabling
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* test: regenerate skip_wal sqlness result
The previous commit changed tests/cases/standalone/common/skip_wal.sql without
regenerating the matching .result, so every Sqlness suite failed on the
mismatch. tests/cases/distributed/common is a symlink to standalone/common, so
the single stale file accounted for all five failing variants.
Regenerated from a real run. The recorded output now covers the cases the .sql
added:
* A table created with skip_wal = 'true' has no real WAL provider, so enabling
WAL is refused with Unsupported rather than the previous InvalidArguments.
* A RaftEngine-backed table accepts the true -> false transition, and repeating
it is a no-op.
* SHOW CREATE TABLE reports skip_wal = 'false' after the transition, including
after a restart.
* A row written while WAL was skipped is absent after restart, while a row
written after WAL was restored survives.
Verified locally with nothing else running on the machine: sqlness skip_wal
passes in both the standalone and distributed environments, and the regenerated
file is byte-identical to the output of an independent earlier run.
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* test: strengthen skip_wal provider coverage
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* Fix WAL provider handling during re-enable
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* Refactor WAL provider check
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
---------
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
verify_rows deep-cloned the whole HashMap<String, ColumnMetadata> of the
physical region on every put batch. On a datanode serving wide physical
tables this showed up as ~18% of total CPU in a CPU flame graph (HashMap
clone + RawTable/ColumnMetadata drop).
Wrap physical_columns in an Arc inside PhysicalRegionState and take a
cheap Arc snapshot instead; add_physical_columns now goes through
Arc::make_mut.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix(mysql): strip leading comments before the federated statement filter
JDBC clients prefix every statement with a comment. DataGrip sends
`/* ApplicationName=DataGrip <version> */` in front of each one, and every
pattern in the federated filter is anchored with `^`, so the prefix makes all
of them miss.
Two failures follow. `SET TRANSACTION READ WRITE` reaches the SQL parser and is
rejected. Worse, the DBeaver-specific entries such as
`^(/\* ApplicationName=(.*)SELECT @@(.*))` do match DataGrip's prefix, so
`SELECT @@GLOBAL.event_scheduler` and `SHOW VARIABLES LIKE ...` are absorbed
into a zero-column output, which the MySQL writer sends as an OK packet. The
JDBC driver reports that the statement returned no cursor.
Strip leading whitespace and comments before matching, and drop the
ApplicationName-specific entries the stripping makes redundant. The three that
had no unprefixed counterpart (`SHOW PLUGINS`, `SHOW ENGINES`, `SHOW @@...`)
keep their behaviour as plain patterns.
Also covers gaps found while probing the same path:
- `BEGIN` is absorbed like `START TRANSACTION`/`COMMIT`/`ROLLBACK`.
- `SHOW [GLOBAL|SESSION|LOCAL] VARIABLES|STATUS` parses; the scope is ignored
because GreptimeDB keeps no global/session split.
- `USER()`, `CURRENT_USER()`, `SYSTEM_USER()` and `SCHEMA()` are registered.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(mysql): keep executable comments, reject multi-statement absorption
Review follow-up on the comment stripping, plus the remaining MySQL
compatibility gaps from #9155.
`/*!...*/` is an executable comment: mysqldump emits its initialization as
`/*!40101 SET NAMES ... */`, and the patterns match those verbatim. Stripping
it left an empty statement, so nothing matched and the original SQL reached the
parser, which rejects `SetNames` and `MultipleAssignments`. Leave executable
comments in place.
Absorbing a request also has to stop at a statement boundary, because every
pattern ends in `(.*)`. `BEGIN; INSERT INTO t VALUES (1)` used to fail on the
unsupported `BEGIN`; once `BEGIN` became absorbable the whole request would
report success and write nothing. A request is now scanned for a second
statement and handed to the query engine if it has one. The scan skips plain
comments and string literals so a `;` inside either is not a boundary, and
treats a `/*!...*/` that is not the request itself as a statement, since it
carries SQL.
`check()` now dispatches on the leading statement keyword, so INSERT, UPDATE,
CREATE and ordinary SELECTs run no regex at all — this replaces the narrower
hand-rolled INSERT shortcut. The statement scan runs only for a request the
patterns already matched, which is always a short one.
New compatibility surface:
- `SELECT CURRENT_USER|SESSION_USER|SYSTEM_USER|USER` without parentheses, and
`SELECT @var`, are answered here rather than in the parser. Both are anchored
to the whole statement, so `SELECT user FROM t` still reads the column.
- `information_schema.plugins`, `user_privileges` and `processlist` are
registered as empty tables, like the other MySQL-shape tables around them.
Sessions are reported through `information_schema.process_list` and
`SHOW PROCESSLIST`; `processlist` carries the column shape only.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(mito2): reconcile series indexes in background
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(mito2): clean up series indexes published during region drop
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(config): align series index examples with upstream enable flag
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(mito2): run series index tasks on compaction runtime
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(mito2): deliver complete WAL entries without waiting for next input
Entry completeness is self-contained via Entry::is_complete(); no
lookahead to the next entry is needed. Decode and yield complete entries
immediately in both the log-store reader and the entry distributor.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(mito2): simplify live WAL entry fixtures
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix: use join_dir for series index config path
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: configure series indexes with an enable flag
Signed-off-by: evenyag <realevenyag@gmail.com>
* docs: omit experimental series index from example configs
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: preserve legacy cache cleanup path behavior
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: remove trivial path joining tests
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: isolate worker group WAL directories on Windows
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
* perf(servers): group Prometheus response rows by label runs
Query output tends to be clustered by series, but a matrix response read
the same label values out of the tag columns once per row, and allocated
a key vector per row to look the series up.
Use `arrow::compute::partition` to find the runs of rows that share their
labels and build the series key once per run. The key buffer is hoisted
out of the row loop and handed to the map only when the series is new,
through the raw entry API so the key is hashed once either way.
Partitioning does not pay off when rows are not clustered, so a few
adjacent row pairs are probed first to pick between the run path and the
row-by-row path. Both paths produce the same series.
Drop the per-row "same labels as the previous row" check from #8815. Runs
cover the clustered case it was written for, and it now costs more than
it saves: 10% on a result with one row per series, 1-3% on clustered ones.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* refactor(servers): find label runs with cmp::distinct
`arrow::compute::partition` computes the same ranges on the same kernel,
but its contract takes lexicographically sorted columns, and query output
is not sorted: range queries run without the plan's output sort since
#9090, `sort`/`topk` order by value, and the tag column order in the
schema does not have to match any sort key. An implementation that
exploited the precondition would merge `a, b, a` into one run and
attribute one series' samples to another, without failing.
`cmp::distinct` is element-wise, so it holds for any row order, and its
null handling is the one a series key needs: a null label and an empty
one are distinct, two nulls are not. Building the ranges from the
boundary bitmask also folds away the `tag_columns.is_empty()` case, since
no columns means no boundaries means a single run.
Same kernel, so the benchmark does not move: -1.4% to +1.4% across shapes
with no consistent sign, against +-3% run-to-run drift.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
---------
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
discord9
Weny Xu
discord9
Dhruv Vaishnav
Lei, HUANG
fys
Lanqing Yang
jeremyhi
LFC
dennis zhuang
Yingwen
Immanuel Tikhonov