* fix: accept a value for the CLI --auto-create-schema flag
clap derives ArgAction::SetTrue for bool fields, so --auto-create-schema
took no value and could only set true, which is already the default.
Users could not turn automatic schema creation off from the CLI, unlike
metasrv's auto_create_schema = false.
Set action = clap::ArgAction::Set so the flag accepts an explicit
true/false value while keeping true as the default.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* chore: enhance CLI argument for auto_create_schema option
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Signed-off-by: fengyue-xve <1759519137@qq.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: use relative object keys for Windows filesystem access
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: cover Windows path and time limits in full test CI
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: use relative keys in metadata snapshot tests
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore(deps): replace cargo-udeps with cargo-shear for unused dependency checks
cargo-udeps requires a nightly toolchain and its pinned version (0.1.61)
no longer detects unused dependencies against current cargo internals —
unused deps have landed on main undetected (e.g. humantime in
common-frontend since #6689). cargo-shear is a standalone static analyzer
that runs on any toolchain.
- Swap 'make check-udeps' / 'make fix-udeps' recipes to 'cargo shear' /
'cargo shear --fix' and retire scripts/fix-udeps.py
- CI: install cargo-shear in the check-udeps job; drop the build cache
and protoc steps (cargo-shear never compiles)
- Remove ~150 unused dependency declarations found by cargo-shear, move
misplaced deps to the correct sections, drop orphaned
[workspace.dependencies] entries (arrow-cast, rustc-hash)
- Add [package.metadata.cargo-shear] ignored entries with explanations
for dependencies that are structurally required despite no textual
reference: sqlparser (required by sqlparser_derive expansions in
datatypes, common-query), common-error (required by common-macro's
stack_trace_debug expansions in session, tests-fuzz), k8s-openapi
(feature-pinning for the transitive kube dependency in tests-fuzz),
tikv-jemalloc-sys (link-only, enables jemalloc profiling features in
common-mem-prof), protobuf (required by build.rs-generated bindings in
log-store)
- Drop the obsolete [package.metadata.cargo-udeps.ignore] sections
Part of #9289
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix(meta): populate physical metric table column ids (#9286)
* fix(meta): populate physical metric table column ids
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* test(meta): verify physical metric column ids
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
---------
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* fix(postgres): return empty responses for comment-only SQL (#9295)
fix(postgres): handle parsed empty queries in both protocols
Signed-off-by: houyuwushang <180804215+houyuwushang@users.noreply.github.com>
* ci: create docs follow-up issue on PR merge instead of on label (#9237)
* ci: create docs follow-up issue on PR merge instead of on label
The docbot workflow previously created a docs-repo issue as soon as the
'docs-required' condition was detected (PR opened/edited with the docs
checkbox ticked), even if the PR was never merged.
Now the workflow also triggers on PR 'closed':
- opened/edited: only manage the docs-required/docs-not-required labels
- closed: create the docs issue only when the PR was actually merged and
carries the docs-required label
This also lets maintainers control issue creation by manually adding or
removing the docs-required label before merging.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: address review comments on docs issue creation timing
- Only touch docs labels when the docs checkbox state actually changed
in an edit. Previously, editing any other part of the PR body while
the checkbox stayed checked removed the docs-required label, silently
dropping the docs follow-up now that issue creation happens at merge.
Unchanged checkbox now leaves labels untouched, which also preserves
manual label overrides.
- Do not trust the closed event's stale label snapshot at merge time:
re-read the live PR via the API and create the docs issue if the
docs-required label is present OR the checkbox is ticked in the
current body.
- Make the workflow concurrency group action-aware so a merge run does
not cancel an in-flight label update from an edit run.
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: make docs-required label the single source of truth at merge
The label-OR-checkbox merge condition could not distinguish an
intentional opt-out from an unfinished label update: removing
docs-required while the checkbox stayed checked still produced an
issue, and unchecking the box could still produce one if the merge
read the stale label before the edit run removed it.
At merge time, wait for any pending docbot runs on the PR head SHA to
finish their label updates (bounded to 5 minutes), then decide solely
by the live docs-required label. Adds actions: read permission for
listing workflow runs.
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* perf(promql): push label filters into grouped join inputs (#9280)
* perf(promql): propagate matching filters through grouped joins
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(promql): check matcher safety on the receiving operand
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): spell out the shapes a filter may cross
`preserves_filter` ended in `_ => true`, which was only sound because
`selector_matchers` independently rejects label rewriting, `count_values`,
subqueries and non-rollup calls on the same operand. Loosening the latter
alone would have silently pushed a matcher below a label rewrite. List the
shapes that carry a scan filter instead and default to `false`.
Cite #9207 for the result labels the grouped cases record: the join
projects the right operand's tag set, so `zone` is missing wherever the
right side aggregates it away.
No behavior change.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): assert the new pushdowns reach the scan
The grouped-join unit tests feed tag columns by hand and the SQLness case
only checks results, which are identical whether or not the rewrite fires.
Nothing would have failed if scalar arithmetic, ranking or grouped
matching stopped propagating. Assert through the planner that the matcher
reaches both scans, with a global topk one-side as the counter-example.
Also state that the duplicate-one-side cases record a cross product
Prometheus rejects (#9209), so the baseline is not read as intended
semantics.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(ci): build tests-integration lib with meta-srv/mock (#9299)
* fix(ci): build tests-integration lib with meta-srv/mock
tests-integration's lib code (src/cluster.rs) uses meta_srv::mocks, but
the dependency carrying the mock feature sits in [dev-dependencies].
Builds that only touch the lib, such as the apidoc job's cargo doc
--workspace, resolve meta-srv without mock and fail with E0432.
--all-targets builds unify dev-dependency features, which is why check,
clippy and nextest stayed green.
Move the mock-enabled meta-srv entry back to [dependencies]. The other
testing features moved out in #9072 are not needed by the lib and stay
in [dev-dependencies].
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(repartition): split per-case repartition tests
test_repartition_metric ran four format/primary-key-encoding cases in a
single test function, and test_repartition_mito ran two format cases.
Each case builds its own 3-datanode cluster and runs a full repartition
plus GC cycle, so on S3 the metric test took 165-178s against the 180s
nextest terminate-after. Merge queue runs failed on it at random.
Split each case into its own test. Cases were already independent, so
they now run in parallel and each stays far inside the timeout, and a
failure points at one encoding instead of four.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(json2): support altering JSON2 settings (#9029)
* feat(sql): support alter syntax for JSON2 columns
Signed-off-by: fys <fengys1996@gmail.com>
* fix(json2): preserve rows on type hint mismatch during compaction
* refactor(json2): simplify alter settings handling
* fix(json2): preserve coerced values during compaction
* chore: remove unnecessary clone
* chor: reduce memory allocations
* fix: cargo clippy
* chore: update greptime-proto to main branch
* refactor(datatypes): unify string handling with other JSON type hints
* fix: cr
---------
Signed-off-by: fys <fengys1996@gmail.com>
* fix: keep compaction pruning, metadata, and index work on compact runtime (#9304)
* fix: run compaction pruner tasks on compact runtime
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: keep compaction metadata and index work on compact runtime
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: add AI matching, classification, and scoring functions (#9300)
* feat: return matching scores from jev
Replace the experimental three-argument Boolean function with jev(text, prompt) returning a Float64 probability in [0, 1]. Move threshold comparisons into SQL and update tests and migration examples.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: add Jev choice and score functions
Share asynchronous execution across Noul, Choice, and Score. Validate JSON criteria before requests and return typed scalar answers. Add SQL and HTTP mock coverage with usage examples.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: use generic AI SQL function names
Expose ai_match, ai_choose, and ai_score and move their implementation, tests, and usage guide under generic AI names. Document the current unreleased interface without migration history.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: share constant AI criteria within each batch
Borrow scalar string arguments and lazily parse constant criteria once per batch. Share the parsed allocation across requests while preserving NULL propagation and batch validation before HTTP calls.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: preserve AI score uncertainty in JSONB results
Return score, confidence, and probabilities in criteria-level order from one evaluation. Validate the distribution and preserve provider precision. Add JSON extraction, uncertainty, and single-request regressions, and document confidence-aware ranking.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs: explain reuse of volatile AI evaluations
Document repeated SELECT and WHERE evaluation costs as N + M requests, and show subquery aliases for reusing scalar or structured AI results without additional model calls.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: share logical table batching with OTLP metrics (#9288)
* feat: share logical table batching with OTLP metrics
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: unify pending rows batch acknowledgement policy
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: align logical batcher example configuration expectations
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: align batcher worker channel defaults to 65536
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* perf(mito2): lazily decode dense primary key columns (#9226)
* perf(mito2): lazily decode dense primary key columns
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* perf(mito2): bypass lazy decoding for full primary keys
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito-codec): preserve prefix decoding errors
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito-codec): align encoded length helper naming
Rename encoded_length to encoded_len and update all callers to match the other length helpers in the module.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): clarify conditional dense key decoding
Rename decode_dense_pk to ensure_dense_pk_decoded so callers can see that existing decoded values are preserved and only missing caches are populated.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito-codec): share string framing in row converter
Move encoded_string_len to the parent module so Dense and Sparse use the same framing helper without depending on each other. Preserve its implementation and visibility.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: sync lock
* fix: shear and check issues
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
Signed-off-by: houyuwushang <180804215+houyuwushang@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: fys <fengys1996@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: Dhruv Vaishnav <dhruvvaishnav687@gmail.com>
Co-authored-by: houyuwushang <180804215+houyuwushang@users.noreply.github.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
* fix(cli): sanitize store_addrs in kvbackend build log
Close#7525. The CLI's kvbackend construction log was printing raw
store_addrs which could contain sensitive connection strings (e.g.
PostgreSQL DSNs with passwords).
Changes:
- Add sanitize_store_addrs() helper that reuses
common_meta::kv_backend::util::sanitize_connection_string(),
consistent with MetasrvOptions and StartCommand patterns.
- Replace raw store_addrs in the info! log with sanitized version.
- Add unit tests covering MySQL URLs, PostgreSQL DSNs, etcd addresses,
and empty store_addrs cases.
Signed-off-by: qiang_liu
Signed-off-by: qiang_liu <qiang_liu@trendmicro.com>
Signed-off-by: LiuQhahah <liuqiang9596@gmail.com>
* fix(cli): drop redundant sanitize tests per review
sanitize_connection_string in common_meta already covers MySQL URLs,
PostgreSQL DSNs and credential-free etcd addresses with its own tests.
The added tests only exercised a trivial map+collect wrapper, so remove
them per review nit.
Signed-off-by: LiuQhahah <liuqiang9596@gmail.com>
---------
Signed-off-by: qiang_liu
Signed-off-by: qiang_liu <qiang_liu@trendmicro.com>
Signed-off-by: LiuQhahah <liuqiang9596@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
* feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (fixes#7987)
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (fixes#7987)
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* test(flow): add sqlness golden result for flow_status
Signed-off-by: Palak Jha <palakjha916@gmail.com>
* fix(catalog): remove unused OptionExt import in flow_statistics
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* docs(flow): fix stale 'recent errors' comment on QueryFlowExecStats
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* test: regenerate golden results for flow_statistics table
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* style: apply rustfmt to flow_statistics changes
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* refactor(catalog): hoist current_time_millis out of flow loop and clamp uptime
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* chore: remove accidentally committed fmt_check.log
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* Update flow_status.result
del eof trailing blank line as per review
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* test(flow): restore runner-generated trailing blank line for sqlness
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* postgres: include SHOW FLOW STATUS in extended-query describe (return flow_statistics fields)
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix: address reviewer feedback on flow_statistics PR
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(flow): resolve merge conflicts with main
Signed-off-by: polar <palakjha916@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* sqlness check post gen (information_schema.result)
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(flow): record start_time after req/snapshot_seqs built, before dispatch
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(sql): handle ShowFlowStatus in match statement at util.rs
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat: review patch implementation
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* chore: remove accidentally committed local tool output files and fix fmt
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix worker.rs return type formatting
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(flow): apply rustfmt to get_full_flow_stat return type
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix worker.rs return type formatting
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(flow): re-apply rustfmt to get_full_flow_stat return type after merge
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix: merge conflicts
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(auth): warn when credential load disables Postgres SCRAM or drops a line (#8652)
* fix(auth): warn when credential load disables Postgres SCRAM or drops a line
Static and watch user providers degraded silently in two ways:
- A single non-SCRAM verifier (mysql_native_password, or a legacy
pbkdf2_sha256 hash that predates SCRAM) disables Postgres SCRAM for
every user and falls back to cleartext, with no signal to the operator.
- A malformed credential line (commonly a plaintext password containing
'=', which splits into more than two parts) was dropped without a trace.
Emit a warning at each credential load for both cases so operators don't
unknowingly serve cleartext passwords over Postgres or lose a user. This
is logging only; authentication behavior is unchanged. The SCRAM check
never logs secrets, and the malformed-line warning logs the line number
and file, never the line content.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(auth): warn on credential file read error before truncating
A read error from lines() (I/O failure or invalid UTF-8) ends the
iterator via map_while, silently dropping every remaining credential.
Warn with the line number and file before truncating, matching the
malformed-line handling, so the drop is observable.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* test(object-store): fix racy SecureFs abort test (#8720)
test_writer_abort_is_unsupported_without_atomic_write asserted the file
content immediately after abort() returned Unsupported. SecureFsWriter
writes through tokio::fs::File, whose write_all() only enqueues a blocking
write task (tokio's poll_write returns Ready before the write completes),
so the data may not be visible yet when the test reads the file. Drop the
race-prone content assertion and only verify the Unsupported contract.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat(query): plan native histogram functions (#8705)
* feat(query): plan native histogram functions
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
* fix: cr issue & add tests
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
* fix: cr issue
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
* fix: cr issue
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* perf(promql): avoid repeated scans in sliding range evaluation (#8646)
* perf(promql): use two pointers for sliding range boundaries
Replace the stale cursor heuristic in RangeManipulateStream::calculate_range
with monotonic left/right cursors. The old path rescanned each evaluation
window (O(E x samples-per-window)) and could lose valid samples after sparse
gaps or trailing empty windows. The two pointers keep strict monotonic
progress, reducing boundary generation to O(N + E) while preserving
(curr-range, curr] semantics, start/end shortening, and empty-window output.
Controlled release benchmarks (fixed CPU, ABBA):
- Public RangeManipulate wall time: ~28% faster at 1m/15s, ~66% at 5m/15s,
~96% at 1h/15s.
- Warmed distributed TQL ANALYZE 1h queries: ~17-21% faster end to end;
shorter windows stayed within run-order noise.
Signed-off-by: discord9 <discord9@163.com>
* perf(promql): specialize changes/resets with adaptive edge counting
The generic range_fn macro slices, downcasts, and rescans every overlapping
window for changes() and resets(). Replace the macro path for these two
functions with hand-written UDF wrappers backed by a shared private
edge-count kernel: direct raw-offset scans when requested edges are few,
otherwise one global u64 edge prefix so each window is answered by a prefix
difference.
Behavior is preserved bit-for-bit, including raw null-buffer values, NaN
semantics, signed zero, infinities, empty/singleton windows, independent
timestamp/value offsets, arbitrary window layouts, and exact DataFusion
error messages. The shared proc macro, planner, serializer, and other range
functions are untouched.
Controlled release benchmarks (fixed CPU, ABBA):
- Dense sliding windows (k=4/20/240): 91.7-95.6% less public UDF wall time.
- Low-coverage fallback (N=4096, 8 windows): 73.9-74.4% faster.
- Warmed distributed TQL ANALYZE 5m/1h changes/resets: 12.1-19.7% client
and 12.0-20.9% server latency improvement; controls stayed within drift.
Signed-off-by: discord9 <discord9@163.com>
* ci(query-regression): include PromQL range boundary case in defaults
An audit of historical query-regression runs found zero range-query
coverage: all 208 PromQL ANALYZE samples were bare selectors, so range
evaluation could regress without CI noticing. Wire the
promql_range_boundary case (introduced in #8646) into DEFAULT_CASES so
label-triggered runs measure the range path. The case is cheap: a ~0.3s
synthetic fixture and about a minute of query execution per base/candidate
pass.
Signed-off-by: discord9 <discord9@163.com>
* chore(promql): address sliding range review nits
Move test-only imports into their test modules and remove the unused
pre-specialization changes and resets helpers.
Signed-off-by: discord9 <discord9@163.com>
* style(promql): apply pinned rustfmt
Signed-off-by: discord9 <discord9@163.com>
* test(promql): cover sparse range results
Share the changes and resets test scaffolding while keeping their behavior
oracles independent. Add an end-to-end sqlness regression for sparse samples,
empty intermediate windows, and a valid trailing sample.
Signed-off-by: discord9 <discord9@163.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* ci: optimize fuzz and split workflows (#8710)
* ci: batch fuzz targets in GitHub Actions
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: improve fuzz test observability
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(ci): preserve fuzz setup failure artifacts
Signed-off-by: WenyXu <wenymedia@gmail.com>
* test(ci): keep fuzz mock output in logs
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: optimize fuzz worker cache
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: warm fuzz target binaries
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: isolate fuzz workflow
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: centralize fuzz target preparation
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: split general workflows
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: streamline docs required checks
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: transfer fuzz targets as artifacts
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: preserve fuzz binary permissions
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: streamline fuzz workers
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: cache PR build dependencies
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: retain main build cache policy
Signed-off-by: WenyXu <wenymedia@gmail.com>
* ci: address fuzz review feedback
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix: add public constructor for compactor (#8724)
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat(logging): add enable_file_logging option to disable file logging (#8721)
Signed-off-by: xhwhis <hi@whis.me>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* avoid cloning final Prometheus remote write row (#8733)
perf: avoid cloning final Prometheus remote write row
Signed-off-by: lyang24 <lanqingy93@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat(function): add json_object_keys scalar function (#8722)
Expose JSON object key listing for outermost objects, with sqlness coverage.
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* refactor(mito2): revise compaction trigger behavior (#8706)
* refactor(mito2): revise compaction trigger behavior
Distinguish automatic and manual triggers, coalesce explicit automatic follow-ups, and reject concurrent manual compactions.
Remove implicit post-execution continuation and transient idle statuses so scheduler entries always represent an active lifecycle.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): track automatic compaction follow-ups
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito2): fix compaction transition rustdoc
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): mark manual compaction conflict retryable
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor(mito2): drop unused RequestCancelResult::NotRunning variant
request_cancel is only called in tests where the region is guaranteed to be
running, so the NotRunning case was dead code. Simplify to unwrap() and
remove the variant.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): gate test-only cancellation import
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): prioritize DDL after compaction planning
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat: update dashboard to v0.13.11 (#8737)
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(object-store): skip removed-entry lister test on Windows (#8735)
DirEntry on Windows is a snapshot from FindFirstFileW: file_type() and
metadata() keep returning cached data after the file is removed, so
read_list_entry() cannot observe the deletion. The test asserts the
Unix behavior (lstat returns ENOENT) and fails deterministically on
Windows nightly CI (4/4 tries). Gate it with #[cfg(not(windows))].
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(query): preserve remote dynamic filter target (#8615)
* fix(query): preserve remote dynamic filter target
Signed-off-by: discord9 <discord9@163.com>
* fix(query): check RDF subscriber registration
Signed-off-by: discord9 <discord9@163.com>
* fix(query): refresh initial dyn filter snapshot before dispatch and handle RDF unregister
The remote dynamic filter dispatch ordering regression: freeze the target,
pre-register subscribers, refresh the initial snapshot, then dispatch.
Also implement handle_remote_dyn_filter_unregister to keep unregister
targets consistent with do_get/update.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): update test-only RegionQueryHandler impl to new trait signatures
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* docs: rework README release badges, drop star history, fix grpc flag (#8743)
* docs: show stable, latest and nightly version badges in README
The single release badge rendered whatever GitHub considered newest, so a
pre-release such as v1.2.0-beta.1 looked like the recommended version.
Split it into three self-updating badges using the shields.io `filter`
parameter, keyed off the existing tag naming:
- stable: `!*-*` matches tags without a hyphen (v1.1.4)
- latest: `!*-*-*` excludes nightly and dev builds (v1.2.0-beta.1)
- nightly: `*-nightly-*` matches the weekly build (v1.2.0-nightly-20260706)
No workflow changes are needed; the badges track new releases on their own.
A one-line caption below them says which channel to pick. The release-date
badge is dropped as the three version badges already carry that signal.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: remove star history chart from README
The chart carried a sealed_token in three URLs and added a large
third-party image to the Project Status section without saying anything
the badges and case studies do not already cover.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: use --grpc-bind-addr in README quickstart
--rpc-bind-addr is now only a hidden alias of --grpc-bind-addr and no
longer shows up in --help.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* Update README.md
Co-authored-by: Ning Sun <classicning@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Ning Sun <classicning@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* chore: check enterprise-gated files are listed in both license configs (#8750)
* chore: check enterprise-gated files are listed in both license configs
A file reachable only through `#[cfg(feature = "enterprise")] mod ...;` is
governed by the GreptimeDB Enterprise License, so it must appear in the
`includes` of licenserc-enterprise.toml and the `excludes` of licenserc.toml.
hawkeye stays silent when it does not: the file keeps its Apache-2.0 header and
passes the default check precisely because it was never excluded from it.
scripts/check-enterprise-license.py walks enterprise-gated `mod` declarations,
resolves them to files (submodules included) and diffs that set against both
configs, also reporting stale entries. It runs in the license job in CI and as
`make check-enterprise-license`.
Documents the split it cannot decide for you — whole enterprise features get
their own file, a gated match arm stays inline — in
.agents/architecture-invariants.md.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: tighten enterprise license checks
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(operator): invalidate local cache after dropping view (#8748)
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* chore!: gate soft-drop table behind the enterprise feature (#8747)
* chore: gate soft-drop table behind the enterprise feature
Soft-drop table becomes an enterprise-only feature:
- metasrv rejects gc.experimental_soft_drop.enable=true at startup in
non-enterprise builds, and ddl_soft_drop_enabled is hard-disabled
without the enterprise feature as a second line of defense
- the UNDROP TABLE parser/AST/statement variant, ADMIN purge_table()
registration, and information_schema.recycle_bin registration are
compiled out unless the enterprise feature is enabled
- common-meta procedures, tombstone keys, and DdlTask serde stay
unconditional for persisted-procedure recovery and wire compatibility
- the [gc.experimental_soft_drop] section is removed from the OSS
example config and generated docs (moving to the enterprise repo)
- the soft-drop sqlness cases and their CI job are removed from OSS
(moving to the enterprise repo); affected information_schema .result
files are regenerated
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: limit unused_variables allow to non-enterprise builds
Addresses review comment: apply the allow via cfg_attr so enterprise
builds still catch accidental unused variables in register_admin_only.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: include the config key in the soft-drop enterprise gate error
Addresses review comment: name gc.experimental_soft_drop.enable in the
startup validation error so users can locate the setting quickly when
it is set via env vars or layered config.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: limit unused_mut allow to non-enterprise builds
Addresses review comment: apply the allow via cfg_attr so enterprise
builds still catch unused mut in the table_ddl_event test setup.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: reject soft-drop DDL submissions in non-enterprise builds
Addresses review comment: clients could bypass the SQL-level gates by
submitting DdlTask::UndropTable or DdlTask::PurgeDroppedTable directly
to the procedure service. Reject fresh submissions at the DdlManager
boundary in non-enterprise builds while keeping the procedure loaders
registered for crash recovery and wire compatibility.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: stop --enable-gc from enabling soft drop in the sqlness template
Addresses review comment: the metasrv test template rendered
[gc.experimental_soft_drop] enable = true under the generic --enable-gc
flag, which non-enterprise metasrv now rejects at startup, making the
documented --enable-gc mode unusable in OSS. Keep the flag scoped to
plain GC; enterprise soft-drop coverage moves to the enterprise repo.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: gate fresh soft-drop procedures
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: gate soft-drop fallback coverage
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: gate soft-drop procedure implementation
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: gate drop table soft-drop behavior
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: gate expired soft-drop gc behavior
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* ci: test enterprise table ddl lifecycle
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: mark purge_table as enterprise licensed
The purge_table module is compiled only with the enterprise feature, so
apply the Enterprise License header and register it with both license
header configurations.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: mark recycle_bin as enterprise licensed
The recycle_bin module is compiled only with the enterprise feature, so
apply the Enterprise License header and register it with both license
header configurations.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* chore: mark soft-drop procedure sources as enterprise licensed
The purge and undrop procedure implementations plus the recycle-bin test
module compile only with the enterprise feature. Apply the Enterprise
License header and register them with both license configurations.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat: make frontend heartbeat extensible and lifecycle-safe (#8726)
* feat: make frontend heartbeat extensible and lifecycle-safe
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix: isolate heartbeat extension response handlers
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix: cancel in-flight heartbeat response handling
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test: cover heartbeat wire compatibility
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix: clean up failed heartbeat startup
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix: address frontend heartbeat review feedback
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(meta): release region guards after drop rollback (#8751)
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* refactor!: move native histogram config and `prom_validation_mode` to prom_store (#8744)
* chore: adjust the position of experimental_enable_prometheus_native_histogram
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
* chore: move prom_validation_mode as well
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat: add health-aware gRPC client routing (#8684)
* feat: add gRPC client health routing
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: harden gRPC client health routing
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix: defer gRPC client health checks until first use
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat(mito2): discard unflushed region data safely (#8600)
* feat: support discarding unflushed region data
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(mito2): wake stalled writers after discard
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor(mito2): drop redundant manifest check for discarding unflushed data
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* docs: align wal.sync_period documented default with actual fallback (5s) (#8753)
The example TOMLs and generated config.md documented the default of
wal.sync_period as "10s", but since #5677 moved the WAL sync task to a
background RepeatedTask, an unset sync_period falls back to 5s in
RaftEngineLogStore. The two paths therefore had different fsync
periods: deployments based on the example configs used 10s while bare
configs used 5s.
Align the documentation with the actual code behavior (5s) instead of
changing the code fallback to 10s, so that no existing deployment
silently gets a larger data-loss window on host power loss.
- config/datanode.example.toml, config/standalone.example.toml: 10s -> 5s
- config/config.md: regenerated via make config-docs
- src/cmd/tests/load_config_test.rs: update assertions accordingly
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix: support Utf8View labels in Prometheus response (#8754)
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix(query): validate merge scan remote schema (#8579)
* fix(query): validate merge scan remote schema
Signed-off-by: discord9 <discord9@163.com>
* fix(query): treat JSON columns as schema-compatible across wire/decode forms
CI (Sqlness json2_limit standalone + distributed) failed on the new
remote-schema validation: a JSON column is Binary + extension metadata
(ARROW:extension:name=greptime.json, greptime:type=Json) on the wire but
decodes to Struct(...) with the extension metadata — validate_remote_schema
compared raw arrow data_type and rejected it as a mismatch.
Adds json_fields_compatible(): JSON fields are equal when name and
nullability match, greptime:type matches, and the JSON2 settings
(ARROW:extension:metadata type hints) match, ignoring the physical arrow
type. Only JSON fields may bypass the raw-type comparison; non-JSON
validation stays strict.
Adds 4 regression tests mirroring the CI failure (wire-binary vs
decoded-struct accepted both directions; different JSON2 settings
rejected; JSON vs plain Binary rejected).
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat(grafana): add events dashboard (#8725)
* feat(grafana): add events dashboard
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): tolerate evolving event schemas
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): address events dashboard review
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): restore events dashboard panels
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): bound events dashboard queries
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): include historical event catalogs
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): preserve events drill-down context
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): correct events lifecycle outcomes
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): handle empty event type ranges
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): scope event catalogs to submissions
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): handle empty events dashboard
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(grafana): refresh event schema variables
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* refactor: separate a json2 extension type (#8745)
Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* feat: add admin function registrar (#8762)
* feat: add admin function registrar
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix: reject admin function name collisions
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* chore: fix typo in admin function test
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* fix: information_schema.rs table initialization issue
Signed-off-by: onepizzateam <palakjha916@gmail.com>
* rustfmt fix
Signed-off-by: onepizzateam <palakjha916@gmail.com>
---------
Signed-off-by: onepizzateam <palakjha916@gmail.com>
Signed-off-by: Palak Jha <palakjha916@gmail.com>
Signed-off-by: polar <palakjha916@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: xhwhis <hi@whis.me>
Signed-off-by: lyang24 <lanqingy93@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: Whis Liao <xhwhis@gmail.com>
Co-authored-by: Lanqing Yang <lanqingy93@gmail.com>
Co-authored-by: sun <sunchang_long@163.com>
Co-authored-by: Ning Sun <classicning@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
* refactor: remove the `RawTableMeta` and `RawTableInfo` to make codes more concise
Signed-off-by: luofucong <luofc@foxmail.com>
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com>
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com>
---------
Signed-off-by: luofucong <luofc@foxmail.com>
jeremyhi
Weny Xu
Qiang-Liuanddennis zhuang
XuanwoandWenyXu
dependabot[bot]andNing Sun
LFC
