mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-09-06 05:28:57 +00:00
bd7d2c1dfa166f579bc33c496d50d41eb29b775f
245
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Weny Xu
|
a7ba8b01e1 |
fix(flow): restore FrontendClient::sql API (#8963)
fix(flow): restore batching frontend SQL Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
Weny Xu
|
1409e66837 |
refactor(flight): add request builder and defer DoGet execution (#8953)
* refactor: add flight request builder Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: use flight request builder in flow Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: defer frontend flight query execution Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grpc): avoid cloning requests during auth Signed-off-by: WenyXu <wenymedia@gmail.com> * test(grpc): cover flight request timeout Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(client): share Flight message reader Signed-off-by: WenyXu <wenymedia@gmail.com> * feat(flow): add Flight DoGet timeout Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(client): gate Flight DDL helpers for testing Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(client): restore Flight stream error semantics Signed-off-by: WenyXu <wenymedia@gmail.com> * docs(grpc): document Flight stream input constructors Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(flight): preserve deferred stream context Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(client): use Flight stream SNAFU context Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
 dennis zhuang
|
5d4699db1c |
docs: refine coding agent maps (#8790)
* docs: refine coding agent maps Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs: trim license header guidance Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs: update README links and project status Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
Weny Xu
|
1f1c9270a8 |
feat(event): add event context to procedure events (#8734)
* feat(event): add trigger context to procedure events Signed-off-by: WenyXu <wenymedia@gmail.com> * test(event): fix trigger context event contracts Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): preserve trigger context origins Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): preserve lifecycle trigger contexts Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(meta): gate enterprise trigger imports Signed-off-by: WenyXu <wenymedia@gmail.com> * test(event): assert trigger contexts exactly Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(test): restore migration test literals Signed-off-by: WenyXu <wenymedia@gmail.com> * feat(event): add trigger context to non-table ddl Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): simplify trigger context encoding Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): preserve auto alter trigger reason Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): require explicit trigger context Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): derive trigger context at ddl boundary Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): propagate DDL trigger reasons Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(meta): resolve alter table rebase conflict Signed-off-by: WenyXu <wenymedia@gmail.com> * test(meta): fix alter table trigger context setup Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(meta): pass trigger context to region migration Signed-off-by: WenyXu <wenymedia@gmail.com> * test(event): fix unknown trigger context assertions Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(meta): centralize trigger context protocol mapping Signed-off-by: WenyXu <wenymedia@gmail.com> * test(event): fix migration trigger context assertion Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(procedure): rename event runtime context Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): rename trigger context Signed-off-by: WenyXu <wenymedia@gmail.com> * test(event): fix migration event context assertion Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
 discord9
|
70470bafbe |
fix(flow): fix flow stats aggregation and df_plan_to_sql quoting (#8729)
* fix(flow): fix flow stats aggregation and df_plan_to_sql quoting
1. Distributed-mode flow stats last-writer-wins overwrite:
Each flownode heartbeat put its local flow state map into the single
global __flow/state key, so reports from different nodes overwrote
each other. Store per-flownode reports under
__flow/state/node/{node_id} in the in-memory KV and aggregate on each
heartbeat (last_exec_time_map/state_size/start_time_map take the max
across nodes) into the global key. FlowStateHandler derives node
identity from header.member_id (fallback peer.id) and ignores
identity-less reports. Per-node keys clear automatically on leader
change KV reset. Adapts to FlowStateValue.start_time_map added in
#8392.
2. df_plan_to_sql unquoted special characters break flush/scheduled
execution: ForceQuoteIdentifiers only quoted uppercase identifiers,
so Prometheus-style table names with ':' (e.g. cpu_cores:sum) were
left unquoted, producing invalid SQL ('keyword: :'). Quote any
identifier with non-[a-z0-9_] chars using double quotes
(dialect-neutral).
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(flow): address review comments on quoting and logging
- df_plan_to_sql: also quote digit-leading identifiers (e.g. 123metrics)
which would produce invalid SQL when re-parsed. SQL keywords are
intentionally not checked (ALL_KEYWORDS would over-quote common column
names like number; the unparse failure path has an InsertIntoPlan
fallback).
- flow_state_handler: downgrade identity-less report log from warn! to
debug! to avoid an anomalous sender spamming warn every heartbeat.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
|
||
+4      
|
cabc2f6cc6 |
feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (#7987) (#8392)
* feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (fixes #7987) Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat(flow): add information_schema.flow_statistics and SHOW FLOW STATUS (fixes #7987) Signed-off-by: onepizzateam <palakjha916@gmail.com> * test(flow): add sqlness golden result for flow_status Signed-off-by: Palak Jha <palakjha916@gmail.com> * fix(catalog): remove unused OptionExt import in flow_statistics Signed-off-by: onepizzateam <palakjha916@gmail.com> * docs(flow): fix stale 'recent errors' comment on QueryFlowExecStats Signed-off-by: onepizzateam <palakjha916@gmail.com> * test: regenerate golden results for flow_statistics table Signed-off-by: onepizzateam <palakjha916@gmail.com> * style: apply rustfmt to flow_statistics changes Signed-off-by: onepizzateam <palakjha916@gmail.com> * refactor(catalog): hoist current_time_millis out of flow loop and clamp uptime Signed-off-by: onepizzateam <palakjha916@gmail.com> * chore: remove accidentally committed fmt_check.log Signed-off-by: onepizzateam <palakjha916@gmail.com> * Update flow_status.result del eof trailing blank line as per review Signed-off-by: onepizzateam <palakjha916@gmail.com> * test(flow): restore runner-generated trailing blank line for sqlness Signed-off-by: onepizzateam <palakjha916@gmail.com> * postgres: include SHOW FLOW STATUS in extended-query describe (return flow_statistics fields) Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix: address reviewer feedback on flow_statistics PR Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(flow): resolve merge conflicts with main Signed-off-by: polar <palakjha916@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * sqlness check post gen (information_schema.result) Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(flow): record start_time after req/snapshot_seqs built, before dispatch Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(sql): handle ShowFlowStatus in match statement at util.rs Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat: review patch implementation Signed-off-by: onepizzateam <palakjha916@gmail.com> * chore: remove accidentally committed local tool output files and fix fmt Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix worker.rs return type formatting Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(flow): apply rustfmt to get_full_flow_stat return type Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix worker.rs return type formatting Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(flow): re-apply rustfmt to get_full_flow_stat return type after merge Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix: merge conflicts Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(auth): warn when credential load disables Postgres SCRAM or drops a line (#8652) * fix(auth): warn when credential load disables Postgres SCRAM or drops a line Static and watch user providers degraded silently in two ways: - A single non-SCRAM verifier (mysql_native_password, or a legacy pbkdf2_sha256 hash that predates SCRAM) disables Postgres SCRAM for every user and falls back to cleartext, with no signal to the operator. - A malformed credential line (commonly a plaintext password containing '=', which splits into more than two parts) was dropped without a trace. Emit a warning at each credential load for both cases so operators don't unknowingly serve cleartext passwords over Postgres or lose a user. This is logging only; authentication behavior is unchanged. The SCRAM check never logs secrets, and the malformed-line warning logs the line number and file, never the line content. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(auth): warn on credential file read error before truncating A read error from lines() (I/O failure or invalid UTF-8) ends the iterator via map_while, silently dropping every remaining credential. Warn with the line number and file before truncating, matching the malformed-line handling, so the drop is observable. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * test(object-store): fix racy SecureFs abort test (#8720) test_writer_abort_is_unsupported_without_atomic_write asserted the file content immediately after abort() returned Unsupported. SecureFsWriter writes through tokio::fs::File, whose write_all() only enqueues a blocking write task (tokio's poll_write returns Ready before the write completes), so the data may not be visible yet when the test reads the file. Drop the race-prone content assertion and only verify the Unsupported contract. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat(query): plan native histogram functions (#8705) * feat(query): plan native histogram functions Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: cr issue & add tests Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: cr issue Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: cr issue Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * perf(promql): avoid repeated scans in sliding range evaluation (#8646) * perf(promql): use two pointers for sliding range boundaries Replace the stale cursor heuristic in RangeManipulateStream::calculate_range with monotonic left/right cursors. The old path rescanned each evaluation window (O(E x samples-per-window)) and could lose valid samples after sparse gaps or trailing empty windows. The two pointers keep strict monotonic progress, reducing boundary generation to O(N + E) while preserving (curr-range, curr] semantics, start/end shortening, and empty-window output. Controlled release benchmarks (fixed CPU, ABBA): - Public RangeManipulate wall time: ~28% faster at 1m/15s, ~66% at 5m/15s, ~96% at 1h/15s. - Warmed distributed TQL ANALYZE 1h queries: ~17-21% faster end to end; shorter windows stayed within run-order noise. Signed-off-by: discord9 <discord9@163.com> * perf(promql): specialize changes/resets with adaptive edge counting The generic range_fn macro slices, downcasts, and rescans every overlapping window for changes() and resets(). Replace the macro path for these two functions with hand-written UDF wrappers backed by a shared private edge-count kernel: direct raw-offset scans when requested edges are few, otherwise one global u64 edge prefix so each window is answered by a prefix difference. Behavior is preserved bit-for-bit, including raw null-buffer values, NaN semantics, signed zero, infinities, empty/singleton windows, independent timestamp/value offsets, arbitrary window layouts, and exact DataFusion error messages. The shared proc macro, planner, serializer, and other range functions are untouched. Controlled release benchmarks (fixed CPU, ABBA): - Dense sliding windows (k=4/20/240): 91.7-95.6% less public UDF wall time. - Low-coverage fallback (N=4096, 8 windows): 73.9-74.4% faster. - Warmed distributed TQL ANALYZE 5m/1h changes/resets: 12.1-19.7% client and 12.0-20.9% server latency improvement; controls stayed within drift. Signed-off-by: discord9 <discord9@163.com> * ci(query-regression): include PromQL range boundary case in defaults An audit of historical query-regression runs found zero range-query coverage: all 208 PromQL ANALYZE samples were bare selectors, so range evaluation could regress without CI noticing. Wire the promql_range_boundary case (introduced in #8646) into DEFAULT_CASES so label-triggered runs measure the range path. The case is cheap: a ~0.3s synthetic fixture and about a minute of query execution per base/candidate pass. Signed-off-by: discord9 <discord9@163.com> * chore(promql): address sliding range review nits Move test-only imports into their test modules and remove the unused pre-specialization changes and resets helpers. Signed-off-by: discord9 <discord9@163.com> * style(promql): apply pinned rustfmt Signed-off-by: discord9 <discord9@163.com> * test(promql): cover sparse range results Share the changes and resets test scaffolding while keeping their behavior oracles independent. Add an end-to-end sqlness regression for sparse samples, empty intermediate windows, and a valid trailing sample. Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * ci: optimize fuzz and split workflows (#8710) * ci: batch fuzz targets in GitHub Actions Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: improve fuzz test observability Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(ci): preserve fuzz setup failure artifacts Signed-off-by: WenyXu <wenymedia@gmail.com> * test(ci): keep fuzz mock output in logs Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: optimize fuzz worker cache Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: warm fuzz target binaries Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: isolate fuzz workflow Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: centralize fuzz target preparation Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: split general workflows Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: streamline docs required checks Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: transfer fuzz targets as artifacts Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: preserve fuzz binary permissions Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: streamline fuzz workers Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: cache PR build dependencies Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: retain main build cache policy Signed-off-by: WenyXu <wenymedia@gmail.com> * ci: address fuzz review feedback Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix: add public constructor for compactor (#8724) Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat(logging): add enable_file_logging option to disable file logging (#8721) Signed-off-by: xhwhis <hi@whis.me> Signed-off-by: onepizzateam <palakjha916@gmail.com> * avoid cloning final Prometheus remote write row (#8733) perf: avoid cloning final Prometheus remote write row Signed-off-by: lyang24 <lanqingy93@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat(function): add json_object_keys scalar function (#8722) Expose JSON object key listing for outermost objects, with sqlness coverage. Signed-off-by: onepizzateam <palakjha916@gmail.com> * refactor(mito2): revise compaction trigger behavior (#8706) * refactor(mito2): revise compaction trigger behavior Distinguish automatic and manual triggers, coalesce explicit automatic follow-ups, and reject concurrent manual compactions. Remove implicit post-execution continuation and transient idle statuses so scheduler entries always represent an active lifecycle. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): track automatic compaction follow-ups Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * docs(mito2): fix compaction transition rustdoc Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): mark manual compaction conflict retryable Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor(mito2): drop unused RequestCancelResult::NotRunning variant request_cancel is only called in tests where the region is guaranteed to be running, so the NotRunning case was dead code. Simplify to unwrap() and remove the variant. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): gate test-only cancellation import Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): prioritize DDL after compaction planning Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat: update dashboard to v0.13.11 (#8737) Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(object-store): skip removed-entry lister test on Windows (#8735) DirEntry on Windows is a snapshot from FindFirstFileW: file_type() and metadata() keep returning cached data after the file is removed, so read_list_entry() cannot observe the deletion. The test asserts the Unix behavior (lstat returns ENOENT) and fails deterministically on Windows nightly CI (4/4 tries). Gate it with #[cfg(not(windows))]. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(query): preserve remote dynamic filter target (#8615) * fix(query): preserve remote dynamic filter target Signed-off-by: discord9 <discord9@163.com> * fix(query): check RDF subscriber registration Signed-off-by: discord9 <discord9@163.com> * fix(query): refresh initial dyn filter snapshot before dispatch and handle RDF unregister The remote dynamic filter dispatch ordering regression: freeze the target, pre-register subscribers, refresh the initial snapshot, then dispatch. Also implement handle_remote_dyn_filter_unregister to keep unregister targets consistent with do_get/update. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(query): update test-only RegionQueryHandler impl to new trait signatures Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * docs: rework README release badges, drop star history, fix grpc flag (#8743) * docs: show stable, latest and nightly version badges in README The single release badge rendered whatever GitHub considered newest, so a pre-release such as v1.2.0-beta.1 looked like the recommended version. Split it into three self-updating badges using the shields.io `filter` parameter, keyed off the existing tag naming: - stable: `!*-*` matches tags without a hyphen (v1.1.4) - latest: `!*-*-*` excludes nightly and dev builds (v1.2.0-beta.1) - nightly: `*-nightly-*` matches the weekly build (v1.2.0-nightly-20260706) No workflow changes are needed; the badges track new releases on their own. A one-line caption below them says which channel to pick. The release-date badge is dropped as the three version badges already carry that signal. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs: remove star history chart from README The chart carried a sealed_token in three URLs and added a large third-party image to the Project Status section without saying anything the badges and case studies do not already cover. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs: use --grpc-bind-addr in README quickstart --rpc-bind-addr is now only a hidden alias of --grpc-bind-addr and no longer shows up in --help. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * Update README.md Co-authored-by: Ning Sun <classicning@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> Co-authored-by: Ning Sun <classicning@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * chore: check enterprise-gated files are listed in both license configs (#8750) * chore: check enterprise-gated files are listed in both license configs A file reachable only through `#[cfg(feature = "enterprise")] mod ...;` is governed by the GreptimeDB Enterprise License, so it must appear in the `includes` of licenserc-enterprise.toml and the `excludes` of licenserc.toml. hawkeye stays silent when it does not: the file keeps its Apache-2.0 header and passes the default check precisely because it was never excluded from it. scripts/check-enterprise-license.py walks enterprise-gated `mod` declarations, resolves them to files (submodules included) and diffs that set against both configs, also reporting stale entries. It runs in the license job in CI and as `make check-enterprise-license`. Documents the split it cannot decide for you — whole enterprise features get their own file, a gated match arm stays inline — in .agents/architecture-invariants.md. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: tighten enterprise license checks Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(operator): invalidate local cache after dropping view (#8748) Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * chore!: gate soft-drop table behind the enterprise feature (#8747) * chore: gate soft-drop table behind the enterprise feature Soft-drop table becomes an enterprise-only feature: - metasrv rejects gc.experimental_soft_drop.enable=true at startup in non-enterprise builds, and ddl_soft_drop_enabled is hard-disabled without the enterprise feature as a second line of defense - the UNDROP TABLE parser/AST/statement variant, ADMIN purge_table() registration, and information_schema.recycle_bin registration are compiled out unless the enterprise feature is enabled - common-meta procedures, tombstone keys, and DdlTask serde stay unconditional for persisted-procedure recovery and wire compatibility - the [gc.experimental_soft_drop] section is removed from the OSS example config and generated docs (moving to the enterprise repo) - the soft-drop sqlness cases and their CI job are removed from OSS (moving to the enterprise repo); affected information_schema .result files are regenerated Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: limit unused_variables allow to non-enterprise builds Addresses review comment: apply the allow via cfg_attr so enterprise builds still catch accidental unused variables in register_admin_only. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: include the config key in the soft-drop enterprise gate error Addresses review comment: name gc.experimental_soft_drop.enable in the startup validation error so users can locate the setting quickly when it is set via env vars or layered config. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: limit unused_mut allow to non-enterprise builds Addresses review comment: apply the allow via cfg_attr so enterprise builds still catch unused mut in the table_ddl_event test setup. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: reject soft-drop DDL submissions in non-enterprise builds Addresses review comment: clients could bypass the SQL-level gates by submitting DdlTask::UndropTable or DdlTask::PurgeDroppedTable directly to the procedure service. Reject fresh submissions at the DdlManager boundary in non-enterprise builds while keeping the procedure loaders registered for crash recovery and wire compatibility. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: stop --enable-gc from enabling soft drop in the sqlness template Addresses review comment: the metasrv test template rendered [gc.experimental_soft_drop] enable = true under the generic --enable-gc flag, which non-enterprise metasrv now rejects at startup, making the documented --enable-gc mode unusable in OSS. Keep the flag scoped to plain GC; enterprise soft-drop coverage moves to the enterprise repo. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: gate fresh soft-drop procedures Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: gate soft-drop fallback coverage Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: gate soft-drop procedure implementation Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: gate drop table soft-drop behavior Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: gate expired soft-drop gc behavior Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * ci: test enterprise table ddl lifecycle Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: mark purge_table as enterprise licensed The purge_table module is compiled only with the enterprise feature, so apply the Enterprise License header and register it with both license header configurations. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: mark recycle_bin as enterprise licensed The recycle_bin module is compiled only with the enterprise feature, so apply the Enterprise License header and register it with both license header configurations. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: mark soft-drop procedure sources as enterprise licensed The purge and undrop procedure implementations plus the recycle-bin test module compile only with the enterprise feature. Apply the Enterprise License header and register them with both license configurations. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat: make frontend heartbeat extensible and lifecycle-safe (#8726) * feat: make frontend heartbeat extensible and lifecycle-safe Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: isolate heartbeat extension response handlers Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: cancel in-flight heartbeat response handling Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: cover heartbeat wire compatibility Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: clean up failed heartbeat startup Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: address frontend heartbeat review feedback Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(meta): release region guards after drop rollback (#8751) Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * refactor!: move native histogram config and `prom_validation_mode` to prom_store (#8744) * chore: adjust the position of experimental_enable_prometheus_native_histogram Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: move prom_validation_mode as well Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat: add health-aware gRPC client routing (#8684) * feat: add gRPC client health routing Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: harden gRPC client health routing Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: defer gRPC client health checks until first use Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat(mito2): discard unflushed region data safely (#8600) * feat: support discarding unflushed region data Signed-off-by: evenyag <realevenyag@gmail.com> * fix(mito2): wake stalled writers after discard Signed-off-by: evenyag <realevenyag@gmail.com> * refactor(mito2): drop redundant manifest check for discarding unflushed data Signed-off-by: evenyag <realevenyag@gmail.com> --------- Signed-off-by: evenyag <realevenyag@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * docs: align wal.sync_period documented default with actual fallback (5s) (#8753) The example TOMLs and generated config.md documented the default of wal.sync_period as "10s", but since #5677 moved the WAL sync task to a background RepeatedTask, an unset sync_period falls back to 5s in RaftEngineLogStore. The two paths therefore had different fsync periods: deployments based on the example configs used 10s while bare configs used 5s. Align the documentation with the actual code behavior (5s) instead of changing the code fallback to 10s, so that no existing deployment silently gets a larger data-loss window on host power loss. - config/datanode.example.toml, config/standalone.example.toml: 10s -> 5s - config/config.md: regenerated via make config-docs - src/cmd/tests/load_config_test.rs: update assertions accordingly Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix: support Utf8View labels in Prometheus response (#8754) Signed-off-by: evenyag <realevenyag@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix(query): validate merge scan remote schema (#8579) * fix(query): validate merge scan remote schema Signed-off-by: discord9 <discord9@163.com> * fix(query): treat JSON columns as schema-compatible across wire/decode forms CI (Sqlness json2_limit standalone + distributed) failed on the new remote-schema validation: a JSON column is Binary + extension metadata (ARROW:extension:name=greptime.json, greptime:type=Json) on the wire but decodes to Struct(...) with the extension metadata — validate_remote_schema compared raw arrow data_type and rejected it as a mismatch. Adds json_fields_compatible(): JSON fields are equal when name and nullability match, greptime:type matches, and the JSON2 settings (ARROW:extension:metadata type hints) match, ignoring the physical arrow type. Only JSON fields may bypass the raw-type comparison; non-JSON validation stays strict. Adds 4 regression tests mirroring the CI failure (wire-binary vs decoded-struct accepted both directions; different JSON2 settings rejected; JSON vs plain Binary rejected). Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat(grafana): add events dashboard (#8725) * feat(grafana): add events dashboard Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): tolerate evolving event schemas Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): address events dashboard review Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): restore events dashboard panels Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): bound events dashboard queries Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): include historical event catalogs Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): preserve events drill-down context Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): correct events lifecycle outcomes Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): handle empty event type ranges Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): scope event catalogs to submissions Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): handle empty events dashboard Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(grafana): refresh event schema variables Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * refactor: separate a json2 extension type (#8745) Signed-off-by: luofucong <luofc@foxmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * feat: add admin function registrar (#8762) * feat: add admin function registrar Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: reject admin function name collisions Signed-off-by: jeremyhi <fengjiachun@gmail.com> * chore: fix typo in admin function test Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: onepizzateam <palakjha916@gmail.com> * fix: information_schema.rs table initialization issue Signed-off-by: onepizzateam <palakjha916@gmail.com> * rustfmt fix Signed-off-by: onepizzateam <palakjha916@gmail.com> --------- Signed-off-by: onepizzateam <palakjha916@gmail.com> Signed-off-by: Palak Jha <palakjha916@gmail.com> Signed-off-by: polar <palakjha916@gmail.com> Signed-off-by: Dennis Zhuang <killme2008@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: shuiyisong <xixing.sys@gmail.com> Signed-off-by: discord9 <discord9@163.com> Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: xhwhis <hi@whis.me> Signed-off-by: lyang24 <lanqingy93@gmail.com> Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: evenyag <realevenyag@gmail.com> Signed-off-by: luofucong <luofc@foxmail.com> Co-authored-by: dennis zhuang <killme2008@gmail.com> Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com> Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com> Co-authored-by: discord9 <discord9@163.com> Co-authored-by: Weny Xu <wenymedia@gmail.com> Co-authored-by: Ning Sun <sunng@protonmail.com> Co-authored-by: Whis Liao <xhwhis@gmail.com> Co-authored-by: Lanqing Yang <lanqingy93@gmail.com> Co-authored-by: sun <sunchang_long@163.com> Co-authored-by: Ning Sun <classicning@gmail.com> Co-authored-by: jeremyhi <jiachun_feng@proton.me> Co-authored-by: Yingwen <realevenyag@gmail.com> Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com> |
||
|
Weny Xu
|
8026064659 |
feat: add health-aware gRPC client routing (#8684)
* feat: add gRPC client health routing Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: harden gRPC client health routing Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: defer gRPC client health checks until first use Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
 jeremyhi
|
448f973593 |
fix: sandbox SQL local filesystem access (#8708)
* fix: sandbox SQL local filesystem access Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: address local file sandbox review findings Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: support Windows local copy paths Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: improve sandbox path errors Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: simplify local path error context Signed-off-by: jeremyhi <fengjiachun@gmail.com> * perf: stream secure filesystem listings Signed-off-by: jeremyhi <fengjiachun@gmail.com> * style: derive local file access default Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: improve local file access errors Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: address local file access review findings Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: simplify local file access coverage Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: harden sandboxed local file backends Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: reject directory copy targets before creation Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: avoid implicit string clone in file table listing Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> |
||
|
discord9
|
a0e6f0b4ca |
fix(flow): avoid duplicate incremental planning warnings (#8611)
fix(flow): reduce duplicate incremental fallback warning Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
4c5be47400 |
fix(flow): downgrade disabled incremental checkpoint log (#8572)
Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
9f5a11e82a |
fix(flow): lower routine batching messages to debug (#8592)
fix(flow): reduce expected batching log noise Signed-off-by: discord9 <discord9@163.com> |
||
|
Lei, HUANG
|
ed8a4990fe |
feat(flow): handle time_ranges in DirtyWindowRequest (#8582)
* feat(flow): handle time_ranges in DirtyWindowRequest Bump greptime-proto to include the new `time_ranges` field on DirtyWindowRequest (GreptimeTeam/greptime-proto#330) and mark the corresponding aligned time windows as dirty in the batching engine, in addition to the existing per-timestamp dirty marking. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * style(flow): fix doc comment spacing in align_time_window Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test(flow): cover time_ranges in handle_mark_dirty_time_window Verify a valid [start_inclusive, end_exclusive) range is aligned to time window boundaries and stored with an explicit end, and that empty or reversed ranges are skipped. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(flow): union merged dirty windows with the larger end Merging a bounded dirty range with a window contained in it (e.g. [0s, 15s) with nested [5s, 10s), or an unbounded dirty window inside a bounded range) previously assigned the contained window's upper bound, shrinking the merged window and permanently dropping the tail range from re-computation. Keep max(prev_upper, cur_upper) instead. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(flow): clip bounded dirty ranges at the expire bound merge_dirty_time_windows dropped every window whose start is before expire_lower_bound, so a bounded dirty range crossing the bound (e.g. [0s, 15s) with expire 10s) lost its still-live suffix [10s, 15s). Now bounded ranges are dropped only when their end is at/before the expire bound, and crossing ranges are clipped to the bound (which the caller aligns to the time window boundary). Unbounded windows keep the existing start-based behavior. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(flow): fall back to full dirty on dirty-window alignment failure An eval/alignment error previously aborted the per-task dirty-marking closure, losing every dirty timestamp and range accumulated for that task, while the RPC still returned Ok so the producer would not retry. On alignment failure now log a warning and mark the whole task dirty (set_dirty) instead, so the affected data is conservatively recomputed. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * style(flow): apply rustfmt to new dirty-window merge tests Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test(flow): cover time index units in dirty window marking Document that DirtyWindowRequest timestamps/time_ranges are bare i64s interpreted in the source table's time index native unit, and add a test expressing the same [3s, 11s) range in second/millisecond/ microsecond/nanosecond units across four tables, asserting all align to the same dirty window [0s, 15s). Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: bump greptime-proto to 8127f179 Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(flow): return dirty-window alignment errors to callers Do not acknowledge a DirtyWindowRequest when a time-windowed task cannot align a timestamp or range. The previous conservative fallback used set_dirty(), but that marker only represents a single epoch-start window for time-windowed flows, so it could still lose the affected dirty range. Propagate task errors through the join loop instead so producers can retry. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: update proto to commits on main Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> |
||
 Ruihang Xia
|
0479c05092 |
perf: preserve dictionary-encoded query labels (#8541)
* perf: preserve dictionary-encoded query labels Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(client): skip dictionary Flight batches Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(servers): decode dictionary labels in HTTP output Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * test(mito2): support dictionary tags in series scans Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(servers): preserve dictionary child nulls in SQL Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(promql): compare dictionary tags by logical nulls Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): keep dictionary tags within query paths Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * refactor(query): scope PK dictionary encoding to reads Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): preserve pushdown for dictionary labels Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): handle dictionary labels in query operators Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): complete label type matching Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): align dictionary query schemas Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): decode incompatible OR labels Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix(query): preserve dictionary partition pruning Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix: handle dictionary query edge cases Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * fix: import dictionary downcast macro Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com> |
||
 QuakeWang
|
5d9dddffa7 |
fix(flow): convert streaming expiration to milliseconds (#8481)
* fix(flow): convert streaming expiration to milliseconds Flow EXPIRE AFTER values are stored in seconds, while streaming flow timestamps and durations use milliseconds. Passing the value through unchanged expires state and limits refill scans 1000 times too early. Convert the value at streaming create and refill boundaries, reject overflow, and document the CreateFlowArgs unit. Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com> * fix(flow): reject negative streaming expiration Reject negative EXPIRE AFTER values at the streaming adapter boundary so refill ranges cannot move into the future. Keep zero valid and retain checked seconds-to-milliseconds conversion. Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com> --------- Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com> |
||
|
discord9
|
b6438d3ea2 |
fix(flow): rebind stale snapshot fence (#8409)
* fix(flow): rebind stale snapshot fence Signed-off-by: discord9 <discord9@163.com> * refactor(flow): avoid repeated stale fence error formatting Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
9936a83384 |
feat(query): add runtime provider interface (#8386)
* feat(query): add runtime provider interface Signed-off-by: discord9 <discord9@163.com> * docs(query): document runtime provider interface Signed-off-by: discord9 <discord9@163.com> * feat(query): pass runtime builder to provider Signed-off-by: discord9 <discord9@163.com> * fix(query): make runtime provider fallible Signed-off-by: discord9 <discord9@163.com> * fix(query): propagate runtime provider errors Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
d3cc1b1888 |
feat(flow): stabilize eval interval scheduling (#8360)
* feat(flow): stabilize eval interval scheduling Signed-off-by: discord9 <discord9@163.com> * fix(flow): satisfy eval schedule clippy Signed-off-by: discord9 <discord9@163.com> * test(flow): trim eval schedule coverage Signed-off-by: discord9 <discord9@163.com> * test(flow): cover stable eval scheduling Signed-off-by: discord9 <discord9@163.com> * fix(flow): reserve scheduled runtime hint Signed-off-by: discord9 <discord9@163.com> * test(flow): trim sqlness result eof Signed-off-by: discord9 <discord9@163.com> * fix(flow): harden eval schedule edges Signed-off-by: discord9 <discord9@163.com> * fix(flow): address scheduled flow review Signed-off-by: discord9 <discord9@163.com> * fix(flow): clean scheduled config handling Signed-off-by: discord9 <discord9@163.com> * test(flow): add eval interval compat case Signed-off-by: discord9 <discord9@163.com> * test(flow): cover show create flow in compat Signed-off-by: discord9 <discord9@163.com> * fix(flow): drop scheduled time from flow context Signed-off-by: discord9 <discord9@163.com> * test(flow): assert scheduled now binding Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
dennis zhuang
|
3b8f55e490 |
docs(agents): add per-crate guides, architecture invariants, and generated-files list (#8346)
* docs(agents): add per-crate guides, architecture invariants, and generated-files list Add agent/contributor navigation docs modeled on the AGENTS.md convention: - Per-crate AGENTS.md for hot crates (mito2, metric-engine, flow, frontend, meta-srv): module map, read/write paths, change-coupling points, test commands, and gotchas. - .agents/architecture-invariants.md: repo-wide rules that clippy and the style guide do not cover (format compatibility, crate layering, async runtimes, error handling, experimental gating, the DataFusion fork). - .agents/generated-files.md: tool-generated artifacts that must not be hand-edited (sqlness .result, config.md, dashboards, build.rs output, proto). - Anchor the .gitignore CLAUDE.md/AGENTS.md rules to the repo root so per-crate AGENTS.md files are tracked while root-level personal config stays ignored. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: update crate AGENTS.md and fix config.md path Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs(agents): fix DataFusion patch layout and SQL query lifecycle order Address review feedback on #8346: - architecture-invariants: the DataFusion sub-crates pin an exact crates.io version in [workspace.dependencies] and are redirected to the fork rev in [patch.crates-io]; the two sections hold different forms, not the same rev. - frontend: the SQL query lifecycle runs the pre_parsing/post_parsing interceptors around parsing, before the per-statement permission check. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
Lei, HUANG
|
a3461caf9d |
feat: expose region read load metrics (#8316)
* feat: expose region read load through Prometheus metrics and heartbeat Introduce region-level query load tracking (CPU time and scanned bytes) collected by `RegionScanExec`, exposed via Prometheus metrics and optionally reported through heartbeat region stats. - **Region metrics** (`src/mito2/src/metrics.rs`, `src/store-api/src/metrics.rs`): Add `greptime_mito_region_query_cpu_time`, `greptime_mito_region_query_scanned_bytes`, and `greptime_mito_region_written_bytes_since_open` gauge metrics. - **MitoRegion** (`src/mito2/src/region.rs`, `src/mito2/src/region/opener.rs`, `src/mito2/src/region_write_ctx.rs`): Replace `AtomicU64` `written_bytes` with `IntGauge`; add `query_cpu_time`/`query_scanned_bytes` fields with lifecycle management (init, reset, remove-on-drop). - **RegionStatistic** (`src/store-api/src/region_engine.rs`, `src/store-api/src/storage/requests.rs`): Add `query_cpu_time` and `query_scanned_bytes` fields. - **Metric-engine** (`src/metric-engine/src/utils.rs`): Aggregate query load from metadata and data regions. - **Heartbeat** (`src/datanode/src/heartbeat.rs`, `src/common/meta/src/datanode.rs`): Relay region query load via heartbeat `RegionStat`; add test. - **Query engine** (`src/query/src/options.rs`, `src/query/src/query_engine/state.rs`, `src/query/src/datafusion.rs`, `src/query/src/dist_plan/merge_scan.rs`, `src/query/src/dist_plan/analyzer.rs`, `src/query/src/dummy_catalog.rs`): Add `enable_region_query_load_report` config; wire `RegionScanExec` to accumulate CPU time and scanned bytes. - **Table scan** (`src/table/src/table/scan.rs`, `src/table/src/table/metrics.rs`): Wire table scan metrics. - **Config** (`config/standalone.example.toml`, `config/datanode.example.toml`, `config/frontend.example.toml`, `config/config.md`): Add example config and documentation for `enable_region_query_load_report`. - **Tests** (`src/mito2/src/engine/basic_test.rs`, `src/mito2/src/engine/close_test.rs`, `src/cmd/tests/load_config_test.rs`, `src/flow/src/adapter.rs`): Add unit tests for region query load reporting and metric cleanup on region close; set default config values. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: move region read load report config from query layer to mito engine Move the `enable_region_query_load_report` setting from query-level config (`QueryOptions`/`DistPlannerOptions`) into the mito2 storage engine config (`MitoConfig`), and expose it through the `RegionScanner` trait instead of `ScanRequest`/`PrepareRequest`. - Mito config: `src/mito2/src/config.rs`, `src/mito2/src/engine.rs` - Scan region plumbing: `src/mito2/src/read/scan_region.rs` - RegionScanner trait: `src/store-api/src/region_engine.rs` - Scanner impls: `src/mito2/src/read/seq_scan.rs`, `src/mito2/src/read/series_scan.rs`, `src/mito2/src/read/unordered_scan.rs` - RegionScanExec: `src/table/src/table/scan.rs` - Removed from query layer: `src/query/src/options.rs`, `src/query/src/dist_plan/analyzer.rs`, `src/query/src/query_engine/state.rs`, `src/query/src/datafusion.rs`, `src/query/src/dummy_catalog.rs` - Removed from test/config: `src/query/src/dist_plan/analyzer/test.rs`, `src/flow/src/adapter.rs`, `src/cmd/tests/load_config_test.rs`, `src/store-api/src/storage/requests.rs` - Config docs: `config/config.md`, `config/datanode.example.toml`, `config/frontend.example.toml`, `config/standalone.example.toml` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: move region query load report config from MitoConfig to LoggingOptions Relocate the `enable_region_query_load_report` setting from `MitoConfig` to `LoggingOptions` (as `enable_per_region_metrics`), and thread it into `MitoEngineBuilder` instead of reading from the engine config directly. This makes the region read-load reporting a per-node logging/observability concern rather than a per-engine storage setting. - `config/config.md` - `config/datanode.example.toml` - `config/standalone.example.toml` - `src/common/telemetry/src/logging.rs` - `src/datanode/src/datanode.rs` - `src/mito2/src/config.rs` - `src/mito2/src/engine.rs` - `src/mito2/src/region.rs` Signed-off-by: Lei Huang <lei@huang.to> Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: report region query load on stream drop instead of stream end Move `report_region_query_load()` from `StreamWithMetricWrapper::poll_next()` to `Drop::drop()` so that region query load is reported even when the stream is dropped prematurely (not just when fully consumed). Affected files: - `src/table/src/table/scan.rs` Signed-off-by: Lei, Huang <huanglei@qiyi.com> Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: make region query load reporting configurable Introduce `enable_region_query_load_report` flag to optionally report per-region `query_cpu_time` and `query_scanned_bytes` metrics instead of always creating them. When disabled, the Prometheus gauges are not created (`None`), avoiding metric churn for workloads that do not need query-level load tracking. - `src/common/meta/src/datanode.rs` — Placeholder fields for query load - `src/mito2/src/region.rs` — Make query metrics `Option<IntGauge>`, conditional create/remove/reset - `src/mito2/src/region/opener.rs` — Thread flag through `RegionOpener` - `src/mito2/src/worker.rs` — Thread flag through `WorkerGroup`/`WorkerStarter`/`RegionWorkerLoop` - `src/mito2/src/worker/handle_catchup.rs` — Pass flag on region open - `src/mito2/src/worker/handle_create.rs` — Pass flag on region create - `src/mito2/src/worker/handle_open.rs` — Pass flag on region open - `src/mito2/src/engine.rs` — Pass flag from `MitoEngineBuilder` - `src/mito2/src/test_util.rs` — Test helpers for both modes - `src/mito2/src/engine/basic_test.rs` — Cover disabled and preserve cases - `src/mito2/src/engine/close_test.rs` — Adapt to optional metrics Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor: remove elapsed_compute metric from scan stream The elapsed_compute metric conflated poll-wait time with actual CPU computation, making it misleading. Removed the metric and its recording path from StreamMetrics and StreamWithMetricWrapper. Added a test asserting that poll duration is not reported as elapsed_compute. - `src/table/src/table/metrics.rs` — removed elapsed_compute field, builder, and record_elapsed_compute method - `src/table/src/table/scan.rs` — removed record_elapsed_compute call; added SlowRecordBatchStream test helper and wrapper_poll_time_is_not_elapsed_compute test Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: disable region query load report for compaction scans Compaction scans are internal operations initiated by the engine, not user queries. Disable region query load reporting when the scan input is marked as compaction to avoid misleading load metrics. - `src/mito2/src/read/scan_region.rs` — set `enable_region_query_load_report` to `false` when compaction is enabled; add unit test Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * test: add `enable_per_region_metrics` config to HTTP integration test - Enable per-region metrics config in HTTP test setup \`tests-integration/tests/http.rs\` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor: remove region query load reporting tests and helpers Remove the region query load reporting feature from the codebase, including tests, test utilities, and helper infrastructure that were part of this now-deprecated functionality. Specifically: - Remove region query load reporting tests from `src/mito2/src/engine/basic_test.rs` and `src/table/src/table/scan.rs`, and the region close metrics test from `src/mito2/src/engine/close_test.rs` - Remove region query load report test utilities and simplify engine construction helpers in `src/mito2/src/test_util.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * perf: avoid disabled region query load timing Summary: - Avoid per-poll `Instant::now` and elapsed-time accumulation when `enable_region_query_load_report` is disabled. - Keep region query-load CPU accounting active only when reporting is enabled. Files: - `src/table/src/table/scan.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: move per-region query load reporting from storage to query engine Move `enable_per_region_metrics` from datanode to frontend config and migrate query load tracking (CPU time, scanned bytes) from mito2 storage engine to the query engine's distributed scan planner. The storage-level metrics plumbing and `enable_region_query_load_report` flag are removed from mito2, `ScanInput`, `ScanRegion`, and `RegionScanner`. Query-level metrics are now collected in `merge_scan.rs` via `scan_region_load`. - `src/mito2/` -- Remove `query_cpu_time`, `query_scanned_bytes` metrics, `enable_region_query_load_report` plumbing from engine, region, opener, scanner types, workers - `src/store-api/` -- Remove `query_cpu_time`, `query_scanned_bytes` from `RegionStatistic` - `src/metric-engine/` -- Remove query load fields from `get_region_statistic` - `src/query/` -- Add `enable_per_region_metrics` to `QueryOptions`; wire through planner, optimizer, merge scan with `scan_region_load` metrics - `src/frontend/` -- Pass `enable_per_region_metrics` into `QueryOptions` - `src/common/meta/` -- Remove TODO for query load fields - `config/` -- Move `enable_per_region_metrics` from datanode to frontend and standalone example configs - `src/cmd/tests/` -- Add `enable_per_region_metrics` to flownode config test - `src/flow/` -- Add `enable_per_region_metrics` default to flownode options - `src/table/` -- Remove unused query load fields from scan - `src/datanode/` -- Remove `with_enable_region_query_load_report` calls Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor: remove obsolete mito write load metric Remove obsolete mito-side region written-bytes metric plumbing that is not needed by the frontend read-load reporting path. Related files: - \`src/mito2/src/metrics.rs\` - \`src/mito2/src/region.rs\` - \`src/mito2/src/region/opener.rs\` - \`src/mito2/src/region_write_ctx.rs\` - \`src/mito2/src/engine/basic_test.rs\` - \`src/mito2/src/worker.rs\` - \`src/mito2/src/config.rs\` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: change region query load metrics from gauge to counter Change `REGION_QUERY_CPU_TIME` and `REGION_QUERY_SCANNED_BYTES` from `IntGaugeVec` to `IntCounterVec` since these values are monotonically increasing and do not need gauge semantics. Update corresponding `add` calls to `inc_by` in merge scan reporting. Files: - `src/store-api/src/metrics.rs` — metric type and label changes - `src/query/src/dist_plan/merge_scan.rs` — caller adaptation Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor: pass ReadItem directly to report_region_query_load Move `region_scan_load` call to the caller, so `report_region_query_load` accepts the already-computed `ReadItem` instead of `RecordBatchMetrics`. - `src/query/src/dist_plan/merge_scan.rs` — update signature, inline call, remove stale test Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat: ensure region query load is reported on MergeScanExec drop Remove the `enable_per_region_metrics` parameter from `report_region_query_load` so region load metrics are always emitted. Add a `Drop` impl for `MergeScanExec` that reports sub-stage metrics when the executor is dropped, covering edge cases where per-region metric emission was missed. Add a unit test verifying CPU time and scanned bytes are recorded on drop. Affected file: `src/query/src/dist_plan/merge_scan.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: gate region query load reporting Guard drop-time region query load reporting with the configured per-region metrics flag. Related files: - \`src/query/src/dist_plan/merge_scan.rs\` Symbols: - \`MergeScanExec::drop\` - \`enable_per_region_metrics\` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: clean region query load metrics on drop Remove per-region query load metric labels when a region is dropped so stale label series do not remain in the registry. Related files: - \`src/mito2/src/region.rs\` Symbols: - \`MitoRegion::drop\` - \`REGION_QUERY_CPU_TIME\` - \`REGION_QUERY_SCANNED_BYTES\` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: Lei Huang <lei@huang.to> Signed-off-by: Lei, Huang <huanglei@qiyi.com> |
||
|
Weny Xu
|
e520ff8300 |
feat: decouple error retryability from status codes (#8301)
* feat: add retry hint to common error Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: propagate retry hints in core errors Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: propagate retry hint through RPC metadata Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fallback retry hints to status codes Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: add explicit retry hints for retryable errors Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: remove status code retry fallback Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: classify io retry hints Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: preserve retry hints across error wrappers Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: minior Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: preserve datasource retry hints in query Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: preserve metric engine retry hints Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: preserve flow and frontend retry hints Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: share rskafka retry hint mapping Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
discord9
|
f6f56e6bc5 |
fix!: fence scoped flow repair snapshots (#8277)
* fix: fence scoped flow repair snapshots Signed-off-by: discord9 <discord9@163.com> * test: trim duplicate flow task tests Signed-off-by: discord9 <discord9@163.com> * test: moreless Signed-off-by: discord9 <discord9@163.com> * refactor: simplify flow query failure fallback Signed-off-by: discord9 <discord9@163.com> * test: update need eval interval Signed-off-by: discord9 <discord9@163.com> * docs: for helper fn Signed-off-by: discord9 <discord9@163.com> * chore: less test Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * test: update for eval interval Signed-off-by: discord9 <discord9@163.com> * fix: consume dirty windows after successful query Signed-off-by: discord9 <discord9@163.com> * test: rm useless tests Signed-off-by: discord9 <discord9@163.com> * test: standalone seq&rm dead if Signed-off-by: discord9 <discord9@163.com> * chore: move to pending window instead Signed-off-by: discord9 <discord9@163.com> * chore: mark full also call abandon_fenced_repair Signed-off-by: discord9 <discord9@163.com> * chore: instant for pending fenced repair Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
3608c90b2e |
feat: pass snapshot read bounds over flight (#8279)
Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
1c7b4c11fc |
fix(flow): support complex SQL full-query flows (#8242)
* fix(flow): skip TWE detection for complex plans Signed-off-by: discord9 <discord9@163.com> * chore: helper Signed-off-by: discord9 <discord9@163.com> * fix(flow): allow SQL CTE flow parsing Signed-off-by: discord9 <discord9@163.com> * fix(flow): execute unscoped SQL snapshots as SQL Signed-off-by: discord9 <discord9@163.com> * fix: column list Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
shuiyisong
|
5fd5b91b29 |
chore: remove auth in flownode (#8244)
* chore: remove auth in flownode Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: update docs Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: add flow startup check Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: clippy Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> |
||
|
Ning Sun
|
fd64ced4da |
feat: introduce plugin setup functions with richer context (#8256)
feat: enrich plugin setup context |
||
|
discord9
|
d304df6e75 |
fix: run eval-interval flow without time window (#8231)
* fix: run eval-interval flow without time window Signed-off-by: discord9 <discord9@163.com> * test: cover eval-interval flow join query Signed-off-by: discord9 <discord9@163.com> * fix: address eval interval flow review comments Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
7f76ad5439 |
feat: validate batching flow sink schema on create (#8176)
* feat: check schema on create Signed-off-by: discord9 <discord9@163.com> * chore: update sqlness Signed-off-by: discord9 <discord9@163.com> * fix(flow): avoid duplicate fields when matching sink schema Signed-off-by: discord9 <discord9@163.com> * fix: null handling Signed-off-by: discord9 <discord9@163.com> * chore: per review Signed-off-by: discord9 <discord9@163.com> * chore: debug log Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
28fd796f4e |
fix(flow): harden incremental read correctness (#8196)
* fix(flow): harden incremental read correctness Signed-off-by: discord9 <discord9@163.com> * fix(flow): propagate dirty window options Signed-off-by: discord9 <discord9@163.com> * test: more Signed-off-by: discord9 <discord9@163.com> * chore: test config api Signed-off-by: discord9 <discord9@163.com> * refactor: split gen Signed-off-by: discord9 <discord9@163.com> * chore: per review Signed-off-by: discord9 <discord9@163.com> * fix: allowlist key Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
dennis zhuang
|
ed9312f8e3 |
feat: global switch for creating tables automatically (#8203)
* feat: global switch for creating table automatically Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: make auto_create_table as comment by default Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat: respect gloabl switch for metric engine Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
discord9
|
eccd97b5c7 |
feat(flow): support incremental read checkpoints (#8179)
* feat: flownode inc mode Signed-off-by: discord9 <discord9@163.com> * chore: rename fallback reason Signed-off-by: discord9 <discord9@163.com> * fix: harden flow incremental checkpoints Signed-off-by: discord9 <discord9@163.com> * fix: address flow watermark lint Signed-off-by: discord9 <discord9@163.com> * fix: address flow clippy Signed-off-by: discord9 <discord9@163.com> * refactor: clarify incremental plan preparation Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * test: more sqlness test Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
shuiyisong
|
17815830ed |
chore: add LeaderServicesContext control to standalone (#8164)
* chore: add refresh hook Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: merge start_with_context and start Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: place reset in recover Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: revert stop changes Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: CR issue Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> |
||
|
discord9
|
44f1804b5e |
feat: add flow query-context plumbing for terminal watermarks (#8154)
* feat: add flow checkpoint plumbing Signed-off-by: discord9 <discord9@163.com> * fix: restore when fail Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * chore: clean up some test Signed-off-by: discord9 <discord9@163.com> * clippy Signed-off-by: discord9 <discord9@163.com> * refactor: move more to pr3b Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
454344ab4d |
feat: inc query join rewrite helper (#8108)
* feat: inc query join rewrite helper Signed-off-by: discord9 <discord9@163.com> * chore: rm unwrap Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * chore Signed-off-by: discord9 <discord9@163.com> * per review Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * refactor: pre review Signed-off-by: discord9 <discord9@163.com> * more per review Signed-off-by: discord9 <discord9@163.com> * test: add expected plan test Signed-off-by: discord9 <discord9@163.com> * fix: disallow having Signed-off-by: discord9 <discord9@163.com> * refactor: allow list Signed-off-by: discord9 <discord9@163.com> * fix: also reject groupings Signed-off-by: discord9 <discord9@163.com> * fix: rm global aggr Signed-off-by: discord9 <discord9@163.com> * refactor: split test Signed-off-by: discord9 <discord9@163.com> * fix: computed shadow expr Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
Lei, HUANG
|
2fdbe6c8c3 |
feat: expose node info for placement selectors (#8095)
* feat: expose node info for placement selectors Return `NodeInfo` from `PeerDiscovery` methods and keep OSS selectors mapping back to `Peer`. Carry `__greptime_origin_frontend.addr` from frontend create-table DDLs into selector `extensions`, and thread `PeerAllocContext` through table-route allocation. Persist datanode `NodeInfo` when heartbeat stats are absent so collected env vars remain available after restart. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: skip datanode node info without stats Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: avoid unnecessary workload clones Skip workload cloning for inactive nodes and for active node-info lookups without workload filters. Files: `src/meta-srv/src/discovery/utils.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: require frontend origin address Require `StatementExecutor` to carry a concrete frontend origin address and always attach it to meta DDL query contexts. Files: `src/operator/src/statement.rs`, `src/operator/src/statement/ddl.rs`, `src/operator/src/utils.rs`, `src/frontend/src/instance/builder.rs`, `src/frontend/src/heartbeat.rs`, `src/flow/src/server.rs`, `src/cmd/src/standalone.rs`, `src/cmd/src/flownode.rs`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor: reuse resolved frontend address Resolve the frontend peer address once in the frontend builder, store it on the instance, and reuse it for heartbeat and flow invoker origins. Files: `src/frontend/src/instance/builder.rs`, `src/frontend/src/instance.rs`, `src/frontend/src/heartbeat.rs`, `src/cmd/src/frontend.rs`, `src/cmd/src/standalone.rs`, `src/frontend/src/frontend.rs`, `src/frontend/src/heartbeat/tests.rs`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: preserve datanode lease liveness Filter active datanode node infos through lease timestamps and workloads while preserving node info fields such as reported env vars. Files: `src/meta-srv/src/discovery/utils.rs`, `src/meta-srv/src/discovery/lease.rs`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * Remove stale datanode lease helper - `discovery`: remove the obsolete `alive_datanodes` helper and related tests in `src/meta-srv/src/discovery/utils.rs` and `src/meta-srv/src/discovery/lease.rs` - `integration`: update cluster and standalone setup paths in `tests-integration/src/cluster.rs` and `tests-integration/src/standalone.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/env-based-region-selector-oss: simplify lease discovery - `lease-discovery`: simplify logic and remove unused utilities in `src/meta-srv/src/discovery/lease.rs` and `src/meta-srv/src/discovery/utils.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> |
||
|
discord9anddennis zhuang
|
a282b207a6 |
feat: flow inc query terminal metrics transport (#8045)
* feat: OutputMetrics for inc query Signed-off-by: discord9 <discord9@163.com> * chore: rm unused Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * docs: explain should_collect_region_watermark_from_extensions Signed-off-by: discord9 <discord9@163.com> * fix: transport flow extensions via metadata Avoid routing Flow-specific query extensions through comma-separated hints so checkpoint JSON values remain intact over Flight. Signed-off-by: discord9 <discord9@163.com> * refactor: per review Signed-off-by: discord9 <discord9@163.com> * more per review Signed-off-by: discord9 <discord9@163.com> * chore: more review Signed-off-by: discord9 <discord9@163.com> * test: compat test Signed-off-by: discord9 <discord9@163.com> * fix: standalone also monkey patch Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Co-authored-by: dennis zhuang <killme2008@gmail.com> |
||
|
Weny Xu
|
0c942fc23a |
refactor: unify frontend discovery with active peer discovery (#8031)
* feat: add MetaClient peer discovery support Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: reuse peer discovery for flow frontend discovery Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: use active frontend peers in process manager Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions from CR Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: add comments Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions from CR Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
Ning Sun
|
e14404c677 |
chore: update rust toolchain to 2026-03-21 (#7849)
* chore: update rust toolchain to 2026-03-21 * chore: new format * fix: lint * chore: resolve lint issues * chore: remove as_millis_f64 * chore: deps up |
||
 discord9
|
3beb538aa8 |
fix: rm useless analyzer (#7797)
* fix: rm useless analyzer Signed-off-by: discord9 <discord9@163.com> * test: rm related test Signed-off-by: discord9 <discord9@163.com> * test: flow tql avg Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
discord9
|
d1151b665b |
feat: flow tql cte (#7702)
* feat: flow tql cte Signed-off-by: discord9 <discord9@163.com> * fix: creating flow TQL CTE source tables lose cte part in query Signed-off-by: discord9 <discord9@163.com> * test: update sqlness result Signed-off-by: discord9 <discord9@163.com> * chore Signed-off-by: discord9 <discord9@163.com> * fix: properly canonicalize ident Signed-off-by: discord9 <discord9@163.com> * feat: even stricter check Signed-off-by: discord9 <discord9@163.com> * chore: sqlness update Signed-off-by: discord9 <discord9@163.com> * chore: after rebase fix Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
Ruihang Xia
|
77013d9085 |
feat: report flow stats from streaming and batching engines (#7701)
* fix: report flow stats from streaming and batching engines * handle restart report handler Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * rename fn name Signed-off-by: Ruihang Xia <waynestxia@gmail.com> --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com> |
||
|
Ning Sun
|
43afb7962a |
refactor: remove session from common meta (#7698)
* refactor: remove session dependency from common-meta * chore: add udeps * chore: format * fix: lint issues * chore: update oneshot * chore: update unused deps |
||
|
Weny Xu
|
0ed3b83099 |
refactor: rename partition rule version to partition expr version (#7696)
* refactor: rename partition rule version to partition expr version Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: update proto Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: clippy Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
 LFC
|
8c23b29725 |
refactor: remove the RawTableMeta and RawTableInfo to make codes more concise (#7626)
* refactor: remove the `RawTableMeta` and `RawTableInfo` to make codes more concise Signed-off-by: luofucong <luofc@foxmail.com> * fix ci Signed-off-by: luofucong <luofc@foxmail.com> * fix ci Signed-off-by: luofucong <luofc@foxmail.com> --------- Signed-off-by: luofucong <luofc@foxmail.com> |
||
|
Weny Xu
|
8026b23834 |
feat: partition rule version validation for writes and staging (#7628)
* feat: verify partition rule Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: add partition version cache Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: header check Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: fmt toml Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: minor refactor Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: header Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: fix clippy Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix unit tests Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: minor refactor Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: nit Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: nit Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
discord9
|
a6e69bc180 |
feat: flow last non null (#7646)
* feat: flow last non null support Signed-off-by: discord9 <discord9@163.com> * clippy Signed-off-by: discord9 <discord9@163.com> * test: report error when sink is not last non null Signed-off-by: discord9 <discord9@163.com> * fix: error if query column matches nothing if partial Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
Ruihang Xia
|
38b5df574c |
refactor: SQL and gRPC server handlers (#7637)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com> |
||
|
discord9andRuihang Xia
|
085e9dfc7b |
fix: use phy table when need fitler by tsid/table id (#7609)
* fix: use phy table when need fitler by tsid/table id Signed-off-by: discord9 <discord9@163.com> * feat: add subquery alias Signed-off-by: discord9 <discord9@163.com> * test: update after alias Signed-off-by: discord9 <discord9@163.com> * test: sort&redact Signed-off-by: discord9 <discord9@163.com> * test: per review Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Co-authored-by: Ruihang Xia <waynestxia@gmail.com> |
||
|
discord9
|
38e4a94956 |
feat: ser/de for HistogramFold&UnionDistinctOn (#7601)
* feat: ser/de for histogramFold&UnionDistinctOn Signed-off-by: discord9 <discord9@163.com> * chore: update proto Signed-off-by: discord9 <discord9@163.com> * fix: skip find time window when in tql Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> |
||
|
Weny Xu
|
25687bb282 |
feat: add ddl timeout/wait options, repartition WITH parsing, meta-client startup refactor (#7589)
* feat: add ddl request timeouts and unify meta client startup Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: omplement ALTER TABLE repartition DDL options parsing Signed-off-by: WenyXu <wenymedia@gmail.com> * test: add sqlness tests Signed-off-by: WenyXu <wenymedia@gmail.com> * test: add unit tests Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: pass timeout argument to procedure Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: refine comments Signed-off-by: WenyXu <wenymedia@gmail.com> * test: assert timeout Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: update proto Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
Weny Xu
|
ada4666e10 |
refactor: remove region_numbers from TableMeta and TableInfo (#7519)
* refactor: remove `region_numbers` from `TableMeta` and `TableInfo` Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: create partitions from region route Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix build Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
