* refactor: port query regression runner to Rust
Signed-off-by: discord9 <discord9@163.com>
* ci: remove optional OTLP report plotter
Signed-off-by: discord9 <discord9@163.com>
* refactor: split query regression runner into modules
Signed-off-by: discord9 <discord9@163.com>
* style: use crate-qualified imports in query regression runner
Signed-off-by: discord9 <discord9@163.com>
* refactor: simplify query regression runner internals
Signed-off-by: discord9 <discord9@163.com>
* feat: abstract inspect-footer storage access behind object store destination
Add an optional --destination <TOML> to inspect-footer (and
--base-destination/--candidate-destination to finalize-remote) so the
storage inspection reads DB data files through the opendal-backed
object_store abstraction instead of bare std::fs. Local paths keep
working unchanged via the --root shortcut (File backend); remote
backends (S3/GCS/...) are described by a DestinationConfig TOML
reusing the object-store crate's ObjectStoreConfig serde shape.
- inspect_footer: list via ObjectStore::list + ObjectMeta filtering
(parquet keys, non-zero size, metadata/ segment), read footers
async via ParquetObjectReader + ParquetMetaDataReader with known
file size (no extra HEAD); output JSON schema unchanged
- finalize-remote: --base-data-home/--candidate-data-home become
optional, mutually exclusive with the new --*-destination args
- cmd deps: add object_store_opendal + datafusion_object_store
- tests: fs-backend list+footer integration tests (metadata filtering,
destination TOML mode, root/destination exclusivity)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* style: drop needless borrow in inspect footer test
Fix clippy::needless_borrows_for_generic_args in the inspect-footer test
(fs::create_dir_all(table.join("metadata"))). Missed by the earlier
focused clippy run because it only covered --bin targets.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* ci: automate compatibility version window
Signed-off-by: discord9 <discord9@163.com>
* ci: address compat version window review feedback
Address all four review comments on the compat version window
automation:
- Keep the PR window to the sliding window only (latest patch of the two
newest stable minor lines). Exact =vX.Y.Z anchors from case.toml are no
longer unioned into from_versions; they are validated by the new
--check-anchors mode and exercised by nightly runs via --nightly-window.
- Add --published-only: the window is computed over stable git tags that
have a published, non-draft GitHub release carrying the sqlness compat
artifacts (greptime-linux-amd64 tar.gz and sha256sum), so failed releases
cannot land in the window.
- Run the updater Python tests plus the window/anchor consistency check in
PR and merge-group CI (new compat-updater-check job in integration.yml).
- Regenerate tests/compatibility/ci.toml to the current sliding window
[v1.0.2, v1.1.4].
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* chore: gate soft-drop table behind the enterprise feature
Soft-drop table becomes an enterprise-only feature:
- metasrv rejects gc.experimental_soft_drop.enable=true at startup in
non-enterprise builds, and ddl_soft_drop_enabled is hard-disabled
without the enterprise feature as a second line of defense
- the UNDROP TABLE parser/AST/statement variant, ADMIN purge_table()
registration, and information_schema.recycle_bin registration are
compiled out unless the enterprise feature is enabled
- common-meta procedures, tombstone keys, and DdlTask serde stay
unconditional for persisted-procedure recovery and wire compatibility
- the [gc.experimental_soft_drop] section is removed from the OSS
example config and generated docs (moving to the enterprise repo)
- the soft-drop sqlness cases and their CI job are removed from OSS
(moving to the enterprise repo); affected information_schema .result
files are regenerated
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: limit unused_variables allow to non-enterprise builds
Addresses review comment: apply the allow via cfg_attr so enterprise
builds still catch accidental unused variables in register_admin_only.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: include the config key in the soft-drop enterprise gate error
Addresses review comment: name gc.experimental_soft_drop.enable in the
startup validation error so users can locate the setting quickly when
it is set via env vars or layered config.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: limit unused_mut allow to non-enterprise builds
Addresses review comment: apply the allow via cfg_attr so enterprise
builds still catch unused mut in the table_ddl_event test setup.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: reject soft-drop DDL submissions in non-enterprise builds
Addresses review comment: clients could bypass the SQL-level gates by
submitting DdlTask::UndropTable or DdlTask::PurgeDroppedTable directly
to the procedure service. Reject fresh submissions at the DdlManager
boundary in non-enterprise builds while keeping the procedure loaders
registered for crash recovery and wire compatibility.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: stop --enable-gc from enabling soft drop in the sqlness template
Addresses review comment: the metasrv test template rendered
[gc.experimental_soft_drop] enable = true under the generic --enable-gc
flag, which non-enterprise metasrv now rejects at startup, making the
documented --enable-gc mode unusable in OSS. Keep the flag scoped to
plain GC; enterprise soft-drop coverage moves to the enterprise repo.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: gate fresh soft-drop procedures
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: gate soft-drop fallback coverage
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: gate soft-drop procedure implementation
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: gate drop table soft-drop behavior
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: gate expired soft-drop gc behavior
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* ci: test enterprise table ddl lifecycle
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: mark purge_table as enterprise licensed
The purge_table module is compiled only with the enterprise feature, so
apply the Enterprise License header and register it with both license
header configurations.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: mark recycle_bin as enterprise licensed
The recycle_bin module is compiled only with the enterprise feature, so
apply the Enterprise License header and register it with both license
header configurations.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: mark soft-drop procedure sources as enterprise licensed
The purge and undrop procedure implementations plus the recycle-bin test
module compile only with the enterprise feature. Apply the Enterprise
License header and register them with both license configurations.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: check enterprise-gated files are listed in both license configs
A file reachable only through `#[cfg(feature = "enterprise")] mod ...;` is
governed by the GreptimeDB Enterprise License, so it must appear in the
`includes` of licenserc-enterprise.toml and the `excludes` of licenserc.toml.
hawkeye stays silent when it does not: the file keeps its Apache-2.0 header and
passes the default check precisely because it was never excluded from it.
scripts/check-enterprise-license.py walks enterprise-gated `mod` declarations,
resolves them to files (submodules included) and diffs that set against both
configs, also reporting stale entries. It runs in the license job in CI and as
`make check-enterprise-license`.
Documents the split it cannot decide for you — whole enterprise features get
their own file, a gated match arm stays inline — in
.agents/architecture-invariants.md.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: tighten enterprise license checks
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: declare GreptimeDB Enterprise License for enterprise-gated sources
The `enterprise`-feature-gated sources (triggers, mito2 extension) were
excluded from the Apache-2.0 header check but carried no license of their
own. Declare a separate GreptimeDB Enterprise License and enforce it.
- Add LICENSE-ENTERPRISE (open-core split; core stays Apache-2.0).
- Add an Enterprise License header to each enterprise source file.
- Add licenserc-enterprise.toml and a second hawkeye step in CI to enforce
the Enterprise header on exactly those files.
- Cross-reference the two complementary file lists; document the layout in
licenses/README.md and the README License section.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: reference per-customer Enterprise Agreement instead of a terms URL
There is no public enterprise-terms page; each customer signs an individually
negotiated agreement. Point the license at a "separate written commercial
agreement with GrepTime Inc." and direct readers to the existing contact page.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: add fuzz_repartition_metric_table target scaffold
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: add metric logical lifecycle in repartition fuzz target
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: support partitioned metric tables in repartition fuzz
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: add repartition loop and partition assertions for metric target
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: use shared timestamp clock in metric repartition writes
Signed-off-by: WenyXu <wenymedia@gmail.com>
* refactor: unify string value and bound generation for fuzzing
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: use fixed physical table name in metric repartition fuzz
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: fmt
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: update ci config
Signed-off-by: WenyXu <wenymedia@gmail.com>
* refactor: use btreemap
Signed-off-by: WenyXu <wenymedia@gmail.com>
* print count result
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: add csv translator for insert expr
Introduce a dedicated top-level csv translator so fuzz insert expressions can be converted into writer-ready records through a structured path instead of ad-hoc formatting in targets.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: add csv dump session utilities
Introduce CSV dump env helpers and a session writer that creates run directories, emits seed metadata, and flushes staged CSV records for fuzz workflows.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: bound csv dump buffer with auto flush
Parse readable buffer sizes from env and flush staged CSV records automatically when the in-memory threshold is reached to prevent unbounded growth during long fuzz runs.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: flush csv dump before repartition validation
Wire csv dump session into the metric repartition fuzz flow so successful inserts are translated from insert expressions into CSV records during write loops and flushed to disk right before row validation.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: keep csv dumps on failure and cleanup on pass
Capture run outcomes in metric repartition fuzz, remove dump directories only after successful validation, and retain dump paths on failures so CI and local investigations can use the same artifacts.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: align partial csv records with table headers
Keep append payload compact by storing partial insert-expression columns, then expand to full table-context headers at flush time and fill missing values with empty strings.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: add logs
Signed-off-by: WenyXu <wenymedia@gmail.com>
* dump csv
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: dump csv
Signed-off-by: WenyXu <wenymedia@gmail.com>
* refactor
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: add table-scoped sql dump writer primitives
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: capture table-scoped sql traces after execution
Record insert and repartition SQL only after successful execution, include started_at_ms and elapsed_ms in trace comments, and broadcast repartition events into every logical-table trace file for consistent debugging context.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: harden sql trace comments and include create sql
Normalize multiline trace comments into valid SQL comment lines and append logical-table CREATE SQL to per-table traces for better timeline reconstruction during repartition debugging.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: dump physical create and repartition SQL traces
Signed-off-by: WenyXu <wenymedia@gmail.com>
* dump repartition sql
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: scaffold writer control channel for barrier flow
Add Barrier/Resume/Stop control skeleton and channel wiring in write_loop to prepare per-repartition validation barriers. Also align SQL dump tests with broadcast SQL payload behavior.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: implement writer barrier pause and resume control
Make writer control messages effective by pausing writes on barrier, resuming on resume, and stopping via channel signaling so the next commit can enforce deterministic per-repartition validation boundaries.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: validate rows after each repartition barrier
Add per-action barrier/ack synchronization with timeout, run immediate logical-table row validation after each repartition, and resume writer only after validation completes to improve minimal failure localization.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test: flush dump sessions before per-epoch validation
Extract a shared flush-and-snapshot helper and call it before each immediate row validation so CSV/SQL artifacts are persisted at the same epoch boundary being validated.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: fix unit tests
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: add retry
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: apply suggestions from CR
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: impl vector index query
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: remove VectorSearchRule and merge it into scan hint rule
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: vector search hint
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: join and subquery
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: clippy when feature disabled
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: push hint only when column is non-nullable or an explicit IS NOT NULL filter exists
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: transformed = true
Co-authored-by: Yingwen <realevenyag@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: remove adpater vector hint
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: revert transformed
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
discord9
Lei, HUANG
dennis zhuang
Weny Xu
jeremyhi
shuiyisong
LFC
Ning Sun
Yingwen
liyang
Copilot
QuakeWang