mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-14 07:52:34 +00:00
5fe2a57adbe2f022009d8564ad2e459d7b90439e
820
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 dennis zhuang
|
67db5419ce |
fix(otlp): coerce empty and "-" trace attribute strings to null (#9461)
* fix(otlp): coerce empty and "-" trace attribute strings to null
Instrumentations fill numeric attributes from absent headers with an empty
string (PSR-7 getHeaderLine returns ""). Coercing that into an existing
numeric column failed and rejected the whole span.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: trim redundant row from empty-string rewrite test
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit
|
||
  LFCandLei, HUANG
|
325415912a |
feat: support automatic column addition for Flight bulk inserts [Backport release/v1.3] (#9399)
feat: support automatic column addition for Flight bulk inserts (#9285)
* fix: auto-add columns when initializing bulk insert streams
* fix: reject nested columns in bulk schema auto-add
* fix: reject unknown columns in non-empty bulk inserts
---------
(cherry picked from commit
|
||
 LFCandshuiyisong
|
bb6e1591d5 |
refactor: expose OTLP write and batching helpers for downstream routers [Backport release/v1.3] (#9462)
refactor: expose OTLP write and batching helpers for downstream routers (#9401)
(cherry picked from commit
|
||
 Ning Sun
|
fedce5c5ec |
feat: support non-millisecond time index units in the logical batcher (#9346)
* feat: allow customized time index unit for metric engine table * test: provide query tests * refactor: revert unnecessary change * refactor: share timestamp unit conversions in api helper Address review feedback on the time index unit changeset: - Add shared timestamp_unit/timestamp_datatype helpers to api::helper (the only crate that sees both proto ColumnDataType and TimeUnit due to layering; common-time and datatypes have no greptime-proto dep). This removes the ColumnDataType -> TimeUnit match duplicated between operator's insert path and the OTLP logs path. - Collapse the two TimeUnit <-> ValueData matches in convert_timestamp_value_data by reusing api::helper::to_grpc_value for the construction side. - Note that convert_rows_time_unit rewrites the schema before the values, so an overflow mid-batch leaves the request half-converted; harmless because the error aborts the whole insert request. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: align time units per destination table and floor remote-read timestamps Address review feedback on PR #9236: - Align each metric insert request to the unit of the table it actually targets: an existing logical table keeps its own unit (it may be bound to a different physical table than the one selected by the request), and only new tables use the selected physical table's unit. The previous blanket conversion rewrote valid millisecond samples to the selected physical table's unit and the engine rejected them. Regression test: writing an existing millisecond logical table and a new table in one request that selects a microsecond physical table. - Remote read now floors narrowing timestamp conversions towards negative infinity (div_euclid), consistent with Timestamp::convert_to on the ingestion path; arrow's cast truncates towards zero and returned -1ms for a stored -1001us. Widening (second -> millisecond) keeps the exact arrow cast. Regression test: a negative, non-aligned timestamp round-trips as -2ms. Signed-off-by: Ning Sun <sunning@greptime.com> * perf: fold time unit alignment into existing table lookups Address review feedback on PR #9236: - The per-destination unit alignment no longer runs its own pass of table lookups: create_or_alter_tables_on_demand gains an align_time_index_unit parameter (metric engine path only) and converts each request inside the lookups it already performs — existing tables to their own unit, new tables to the selected physical table's. Default ingest paths now issue zero additional catalog lookups compared to main; the separate alignment pass remains only in the opt-in logical batcher pre-gate, next to the eligibility check that already looks up the same tables. - convert_rows_time_unit indexes the time index position directly (validate_column_count_match guarantees row widths) instead of Optional get_mut; the gate-side alignment validates widths itself. Signed-off-by: Ning Sun <sunning@greptime.com> * perf: resolve the batcher time index guard once per write target All batches of one remote write request share the same write target (catalog, schema, physical table), so the batcher time index guard now resolves each distinct target once instead of once per batch. Signed-off-by: Ning Sun <sunning@greptime.com> * feat: support non-millisecond time index units in the logical batcher Make the logical table batcher's bulk encode path unit-aware so physical metric tables with a non-millisecond time index (e.g. TIMESTAMP(6)) can use logical batching instead of falling back to the ordinary insert path. - rows_to_aligned_record_batch builds the time index column in the TARGET schema's unit, converting any timestamp encoding via Timestamp::convert_to (flooring on narrowing, consistent with the ordinary insert path). - New tables created by the batcher use the selected physical table's time index unit (resolved once per submit; a missing physical table keeps the millisecond auto-create default). - columns_taxonomy and the can_batch_metric_rows schema whitelist accept any timestamp unit; the prometheus remote write v1/v2 batcher gates and the OTLP pre-gate alignment are removed together with Inserter::align_metric_row_inserts_time_unit, as the batcher now converts internally. Closes #9342 Signed-off-by: Ning Sun <sunning@greptime.com> * fix: address review comments * fix: address review issue * refactor: drop the OTLP pre-gate unit alignment made redundant by the bulk path The main merge of #9236 (squash) resurrected the OTLP pre-gate alignment and Inserter::align_metric_row_inserts_time_unit, which this branch had removed. Drop them again: - The pre-gate existed because the #9236-era bulk eligibility gate only accepted millisecond schemas, so nanosecond-encoded OTLP requests had to be converted before the check. This branch makes the bulk path unit-aware (the gate accepts all time index units and batch alignment converts each request to its destination's unit), so the pre-gate is redundant and only added N+1 catalog lookups per batched request — the very lookup-count overhead raised in the #9236 review. - The per-destination unit semantics it implemented remain enforced in the two paths that need them: the ordinary insert path (create_or_alter_tables_on_demand converts inside its existing table lookups) and the batched path (batch alignment resolves each destination schema and converts to it). test_otlp_logical_batcher_alignment (the test the pre-gate originally fixed) and the mixed-physical-table regression both pass without it. Signed-off-by: Ning Sun <sunning@greptime.com> * test: cover OTLP batcher cross-physical fallback and nanosecond physical Extend the logical batcher integration coverage for the cases previously guarded by the removed OTLP pre-gate alignment: - test_otlp_logical_batcher_fallback_for_cross_physical_destination: with the batcher enabled, an OTLP request targeting an existing logical table bound to another physical table must NOT enter the batcher (the bulk eligibility check rejects the destination binding) and the ordinary insert path must convert it to the destination's unit (60s -> 60_000_000us). - test_otlp_logical_batcher_non_millisecond_physical_table now covers both microsecond and nanosecond physical tables (parameterized), asserting batcher submissions and unit-precise stored values. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: validate per-batch physical bindings and avoid intermediate timestamp buffers Address review feedback on PR #9346: - accepts_bulk_destinations dedupes on (schema, table, selected physical) instead of (schema, table): one request can select different physical tables per batch (per-series x_greptime_physical_table labels), and the old key let a second selection skip validation and flush rows through the wrong physical's regions. Missing tables additionally reject conflicting physical selections within the same request. Regression test covers an existing destination, a missing destination, and a consistent selection (which must still batch). - The timestamp column builder appends each value directly into the target-unit Arrow builder; values already in the target unit (the unchanged millisecond fast path) are appended without conversion, so the default millisecond physical pays no Timestamp construction or intermediate Vec allocation. - The non-millisecond batching tests assert the submit_build_and_align counter, which increments on every batcher submission in both acknowledgement modes, so a silent fallback to ordinary insertion fails the tests instead of passing on stored values alone. Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com> |
||
|
Ning Sun
|
55b7e08a1a |
feat: allow customized time index unit for metric engine table (#9236)
* feat: allow customized time index unit for metric engine table * test: provide query tests * refactor: revert unnecessary change * refactor: share timestamp unit conversions in api helper Address review feedback on the time index unit changeset: - Add shared timestamp_unit/timestamp_datatype helpers to api::helper (the only crate that sees both proto ColumnDataType and TimeUnit due to layering; common-time and datatypes have no greptime-proto dep). This removes the ColumnDataType -> TimeUnit match duplicated between operator's insert path and the OTLP logs path. - Collapse the two TimeUnit <-> ValueData matches in convert_timestamp_value_data by reusing api::helper::to_grpc_value for the construction side. - Note that convert_rows_time_unit rewrites the schema before the values, so an overflow mid-batch leaves the request half-converted; harmless because the error aborts the whole insert request. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: align time units per destination table and floor remote-read timestamps Address review feedback on PR #9236: - Align each metric insert request to the unit of the table it actually targets: an existing logical table keeps its own unit (it may be bound to a different physical table than the one selected by the request), and only new tables use the selected physical table's unit. The previous blanket conversion rewrote valid millisecond samples to the selected physical table's unit and the engine rejected them. Regression test: writing an existing millisecond logical table and a new table in one request that selects a microsecond physical table. - Remote read now floors narrowing timestamp conversions towards negative infinity (div_euclid), consistent with Timestamp::convert_to on the ingestion path; arrow's cast truncates towards zero and returned -1ms for a stored -1001us. Widening (second -> millisecond) keeps the exact arrow cast. Regression test: a negative, non-aligned timestamp round-trips as -2ms. Signed-off-by: Ning Sun <sunning@greptime.com> * perf: fold time unit alignment into existing table lookups Address review feedback on PR #9236: - The per-destination unit alignment no longer runs its own pass of table lookups: create_or_alter_tables_on_demand gains an align_time_index_unit parameter (metric engine path only) and converts each request inside the lookups it already performs — existing tables to their own unit, new tables to the selected physical table's. Default ingest paths now issue zero additional catalog lookups compared to main; the separate alignment pass remains only in the opt-in logical batcher pre-gate, next to the eligibility check that already looks up the same tables. - convert_rows_time_unit indexes the time index position directly (validate_column_count_match guarantees row widths) instead of Optional get_mut; the gate-side alignment validates widths itself. Signed-off-by: Ning Sun <sunning@greptime.com> * perf: resolve the batcher time index guard once per write target All batches of one remote write request share the same write target (catalog, schema, physical table), so the batcher time index guard now resolves each distinct target once instead of once per batch. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: address review comments * fix: address review issue --------- Signed-off-by: Ning Sun <sunning@greptime.com> |
||
 discord9
|
678aa81cae |
fix(client): complete transport lane isolation (#9030)
* fix(client): complete transport lane isolation Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * refactor(client): deprecate legacy single-manager constructors Per review: mark the legacy single-manager constructors and helper as deprecated so callers move to the isolated query/control manager pair. Tests intentionally exercising the legacy path are annotated with `#[allow(deprecated)]`. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(client): update deprecated constructor callers for CI Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> |
||
 jeremyhi
|
bf67246645 |
perf: bound concurrent Metric export writers (#9296)
* fix: preserve objects after conditional Metric export collisions Signed-off-by: jeremyhi <fengjiachun@gmail.com> * perf: share Metric export writer and payload budgets Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: cover bounded Metric export conversion and mixed roundtrips Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: preserve export output after ambiguous close Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: align HTTP export timeout with managed cancellation Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: trim redundant Metric export roundtrips Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: bound pending Metric export writer handles Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: protect export files on failed writes Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: preserve ordinary export cleanup and large rows Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: report Metric export collisions as invalid arguments Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: clean up failed conditional export closes Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: support view arrays in managed ordinary export Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: expect successful cleanup after failed close Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: clean up unsynced overwrite after close failure Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: scope export JSON expansion estimates to JSON columns Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: rechunk ordinary export batches within the write budget Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> |
||
|
shuiyisong
|
20dde2601f |
feat: enable native histogram ingestion by default (#9301)
* feat: enable native histogram ingestion by default Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: add comments Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: test Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> |
||
   
|
1d8d95c12a |
chore(deps): replace cargo-udeps with cargo-shear for unused dependency checks (#9294)
* chore(deps): replace cargo-udeps with cargo-shear for unused dependency checks cargo-udeps requires a nightly toolchain and its pinned version (0.1.61) no longer detects unused dependencies against current cargo internals — unused deps have landed on main undetected (e.g. humantime in common-frontend since #6689). cargo-shear is a standalone static analyzer that runs on any toolchain. - Swap 'make check-udeps' / 'make fix-udeps' recipes to 'cargo shear' / 'cargo shear --fix' and retire scripts/fix-udeps.py - CI: install cargo-shear in the check-udeps job; drop the build cache and protoc steps (cargo-shear never compiles) - Remove ~150 unused dependency declarations found by cargo-shear, move misplaced deps to the correct sections, drop orphaned [workspace.dependencies] entries (arrow-cast, rustc-hash) - Add [package.metadata.cargo-shear] ignored entries with explanations for dependencies that are structurally required despite no textual reference: sqlparser (required by sqlparser_derive expansions in datatypes, common-query), common-error (required by common-macro's stack_trace_debug expansions in session, tests-fuzz), k8s-openapi (feature-pinning for the transitive kube dependency in tests-fuzz), tikv-jemalloc-sys (link-only, enables jemalloc profiling features in common-mem-prof), protobuf (required by build.rs-generated bindings in log-store) - Drop the obsolete [package.metadata.cargo-udeps.ignore] sections Part of #9289 Signed-off-by: Ning Sun <sunning@greptime.com> * fix(meta): populate physical metric table column ids (#9286) * fix(meta): populate physical metric table column ids Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * test(meta): verify physical metric column ids Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> --------- Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * fix(postgres): return empty responses for comment-only SQL (#9295) fix(postgres): handle parsed empty queries in both protocols Signed-off-by: houyuwushang <180804215+houyuwushang@users.noreply.github.com> * ci: create docs follow-up issue on PR merge instead of on label (#9237) * ci: create docs follow-up issue on PR merge instead of on label The docbot workflow previously created a docs-repo issue as soon as the 'docs-required' condition was detected (PR opened/edited with the docs checkbox ticked), even if the PR was never merged. Now the workflow also triggers on PR 'closed': - opened/edited: only manage the docs-required/docs-not-required labels - closed: create the docs issue only when the PR was actually merged and carries the docs-required label This also lets maintainers control issue creation by manually adding or removing the docs-required label before merging. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: address review comments on docs issue creation timing - Only touch docs labels when the docs checkbox state actually changed in an edit. Previously, editing any other part of the PR body while the checkbox stayed checked removed the docs-required label, silently dropping the docs follow-up now that issue creation happens at merge. Unchanged checkbox now leaves labels untouched, which also preserves manual label overrides. - Do not trust the closed event's stale label snapshot at merge time: re-read the live PR via the API and create the docs issue if the docs-required label is present OR the checkbox is ticked in the current body. - Make the workflow concurrency group action-aware so a merge run does not cancel an in-flight label update from an edit run. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: make docs-required label the single source of truth at merge The label-OR-checkbox merge condition could not distinguish an intentional opt-out from an unfinished label update: removing docs-required while the checkbox stayed checked still produced an issue, and unchecking the box could still produce one if the merge read the stale label before the edit run removed it. At merge time, wait for any pending docbot runs on the PR head SHA to finish their label updates (bounded to 5 minutes), then decide solely by the live docs-required label. Adds actions: read permission for listing workflow runs. Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com> * perf(promql): push label filters into grouped join inputs (#9280) * perf(promql): propagate matching filters through grouped joins Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * perf(promql): check matcher safety on the receiving operand Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor(promql): spell out the shapes a filter may cross `preserves_filter` ended in `_ => true`, which was only sound because `selector_matchers` independently rejects label rewriting, `count_values`, subqueries and non-rollup calls on the same operand. Loosening the latter alone would have silently pushed a matcher below a label rewrite. List the shapes that carry a scan filter instead and default to `false`. Cite #9207 for the result labels the grouped cases record: the join projects the right operand's tag set, so `zone` is missing wherever the right side aggregates it away. No behavior change. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(promql): assert the new pushdowns reach the scan The grouped-join unit tests feed tag columns by hand and the SQLness case only checks results, which are identical whether or not the rewrite fires. Nothing would have failed if scalar arithmetic, ranking or grouped matching stopped propagating. Assert through the planner that the matcher reaches both scans, with a global topk one-side as the counter-example. Also state that the duplicate-one-side cases record a cross product Prometheus rejects (#9209), so the baseline is not read as intended semantics. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(ci): build tests-integration lib with meta-srv/mock (#9299) * fix(ci): build tests-integration lib with meta-srv/mock tests-integration's lib code (src/cluster.rs) uses meta_srv::mocks, but the dependency carrying the mock feature sits in [dev-dependencies]. Builds that only touch the lib, such as the apidoc job's cargo doc --workspace, resolve meta-srv without mock and fail with E0432. --all-targets builds unify dev-dependency features, which is why check, clippy and nextest stayed green. Move the mock-enabled meta-srv entry back to [dependencies]. The other testing features moved out in #9072 are not needed by the lib and stay in [dev-dependencies]. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(repartition): split per-case repartition tests test_repartition_metric ran four format/primary-key-encoding cases in a single test function, and test_repartition_mito ran two format cases. Each case builds its own 3-datanode cluster and runs a full repartition plus GC cycle, so on S3 the metric test took 165-178s against the 180s nextest terminate-after. Merge queue runs failed on it at random. Split each case into its own test. Cases were already independent, so they now run in parallel and each stays far inside the timeout, and a failure points at one encoding instead of four. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(json2): support altering JSON2 settings (#9029) * feat(sql): support alter syntax for JSON2 columns Signed-off-by: fys <fengys1996@gmail.com> * fix(json2): preserve rows on type hint mismatch during compaction * refactor(json2): simplify alter settings handling * fix(json2): preserve coerced values during compaction * chore: remove unnecessary clone * chor: reduce memory allocations * fix: cargo clippy * chore: update greptime-proto to main branch * refactor(datatypes): unify string handling with other JSON type hints * fix: cr --------- Signed-off-by: fys <fengys1996@gmail.com> * fix: keep compaction pruning, metadata, and index work on compact runtime (#9304) * fix: run compaction pruner tasks on compact runtime Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: keep compaction metadata and index work on compact runtime Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: add AI matching, classification, and scoring functions (#9300) * feat: return matching scores from jev Replace the experimental three-argument Boolean function with jev(text, prompt) returning a Float64 probability in [0, 1]. Move threshold comparisons into SQL and update tests and migration examples. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: add Jev choice and score functions Share asynchronous execution across Noul, Choice, and Score. Validate JSON criteria before requests and return typed scalar answers. Add SQL and HTTP mock coverage with usage examples. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: use generic AI SQL function names Expose ai_match, ai_choose, and ai_score and move their implementation, tests, and usage guide under generic AI names. Document the current unreleased interface without migration history. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: share constant AI criteria within each batch Borrow scalar string arguments and lazily parse constant criteria once per batch. Share the parsed allocation across requests while preserving NULL propagation and batch validation before HTTP calls. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: preserve AI score uncertainty in JSONB results Return score, confidence, and probabilities in criteria-level order from one evaluation. Validate the distribution and preserve provider precision. Add JSON extraction, uncertainty, and single-request regressions, and document confidence-aware ranking. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * docs: explain reuse of volatile AI evaluations Document repeated SELECT and WHERE evaluation costs as N + M requests, and show subquery aliases for reusing scalar or structured AI results without additional model calls. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: share logical table batching with OTLP metrics (#9288) * feat: share logical table batching with OTLP metrics Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: unify pending rows batch acknowledgement policy Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: align logical batcher example configuration expectations Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: align batcher worker channel defaults to 65536 Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> * perf(mito2): lazily decode dense primary key columns (#9226) * perf(mito2): lazily decode dense primary key columns Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * perf(mito2): bypass lazy decoding for full primary keys Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito-codec): preserve prefix decoding errors Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito-codec): align encoded length helper naming Rename encoded_length to encoded_len and update all callers to match the other length helpers in the module. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): clarify conditional dense key decoding Rename decode_dense_pk to ensure_dense_pk_decoded so callers can see that existing decoded values are preserved and only missing caches are populated. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito-codec): share string framing in row converter Move encoded_string_len to the parent module so Dense and Sparse use the same framing helper without depending on each other. Preserve its implementation and visibility. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * chore: sync lock * fix: shear and check issues --------- Signed-off-by: Ning Sun <sunning@greptime.com> Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> Signed-off-by: houyuwushang <180804215+houyuwushang@users.noreply.github.com> Signed-off-by: Dennis Zhuang <killme2008@gmail.com> Signed-off-by: fys <fengys1996@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> Co-authored-by: Dhruv Vaishnav <dhruvvaishnav687@gmail.com> Co-authored-by: houyuwushang <180804215+houyuwushang@users.noreply.github.com> Co-authored-by: dennis zhuang <killme2008@gmail.com> Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com> Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com> Co-authored-by: Weny Xu <wenymedia@gmail.com> |
||
|
Weny Xu
|
953d01ac54 |
feat: support pending rows batching for MySQL and PostgreSQL (#9302)
* feat: support pending rows batching for MySQL and PostgreSQL Signed-off-by: WenyXu <wenymedia@gmail.com> * style: group batcher imports before item definitions Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: use 65536 as the default batcher worker channel capacity Signed-off-by: WenyXu <wenymedia@gmail.com> * test: use a distinct custom worker channel capacity Signed-off-by: WenyXu <wenymedia@gmail.com> * test: complete Prom config in worker capacity override case Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
dennis zhuang
|
f3eb8e6c72 |
test: cut integration test time and make the storage matrix meaningful (#9308)
* test: cut integration test time and make the storage matrix meaningful
tests-integration is ~85% of workspace test CPU, and 81% of that is the
S3/S3WithCache variants of the HTTP and gRPC suites. Those suites do not
touch the object store: of the 70 matrix HTTP tests only one flushed and
read back an SST, so the matrix was paying real AWS round trips to
re-prove protocol parsing.
- Point the PR CI object-store matrix at the MinIO already started by
tests-integration/fixtures. Three GT_S3_* consumers did not read
GT_S3_ENDPOINT_URL and would have hit real AWS with MinIO credentials;
they now do.
- Add a nightly Linux job against real AWS S3, and pass GT_S3_* into the
release integration-test container. The release previously ran every
remote-backend case as a skip and only exercised the file backend.
- Give each S3WithCache test its own read cache directory. They shared
/tmp/greptimedb_cache, which the datanode wipes on startup, so a
starting test deleted the read cache of a running one.
- Add flush -> read-back assertions to the tests whose columns have a
non-trivial SST representation: JSON/JSON2 columns, native histograms,
metric-engine logical tables, and tables carrying fulltext or skipping
indexes whose puffin files only exist after a flush.
- Move eight tests that create no table out of the storage matrix.
- Make the event recorder flush interval a constructor parameter and
shorten it in the event tests, which otherwise wait a 5s window per DDL
they assert on. It is skipped by serde and never reaches config files.
- Drop duplicates: test_grpc_zstd_compression was a verbatim copy of
test_grpc_message_size_ok and is now rewritten to assert the negotiated
grpc-encoding; test_execute_copy_to_{s3,oss,gcs,azblob} were strict
prefixes of their copy_from siblings; two standalone/distributed event
test pairs shared one assertion body.
- Fix and un-ignore stddev_by_label. stddev_pop merges partial aggregates
in a parallelism-dependent order, so its last digits are unstable; the
test now compares values with a tolerance.
- Rebase the jaeger v1 fixture on the current instant. It carries
ttl=7d with 2025 timestamps, so its rows were only readable as long as
they stayed in the memtable.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: address review — wire nightly real-S3 job into check-status, keep the short event interval
The nightly `check-status` job did not depend on the new real-S3 job, so a
failure there would not have reached the status or Slack notification.
In database_ddl_event the short interval was set by a first
`with_event_recorder_options` call and then overwritten by the pre-existing
one, which carries `..Default::default()`. Merged into a single call.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
|
||
|
Weny Xu
|
723da69b21 |
feat: share logical table batching with OTLP metrics (#9288)
* feat: share logical table batching with OTLP metrics Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: unify pending rows batch acknowledgement policy Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: align logical batcher example configuration expectations Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: align batcher worker channel defaults to 65536 Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
LFC
|
16978cf6c2 |
feat(trace): support Semantic Graph for Trace V2 (follow-up to #9192) (#9278)
* feat(trace): support Semantic Graph for Trace V2 Signed-off-by: luofucong <luofc@foxmail.com> * fix: remove unused annotation context import Signed-off-by: luofucong <luofc@foxmail.com> --------- Signed-off-by: luofucong <luofc@foxmail.com> |
||
|
jeremyhi
|
19c127d9c3 |
feat: restore packed metric snapshots (#9250)
* test: cover snapshot parquet restore compatibility Signed-off-by: jeremyhi <fengjiachun@gmail.com> * feat: restore packed metric snapshots Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: stream large packed parquet entries Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: allow default S3 endpoint in packed restore test Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: skip unconfigured S3 in packed restore test Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: use portable file URLs in packed restore fixtures Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: validate packed snapshot structure before restore Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: preserve strict manifest decoding and verify fixture Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: reject packed layout on both database export paths Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: implement file size in coordinator test storage Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: use expect_err for rejected export layouts Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> |
||
|
jeremyhi
|
263e229103 |
feat: add experimental Metric export to V2 snapshots (#9233)
* feat: add experimental Metric export to V2 snapshots Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: validate the complete Metric export capability response Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: construct portable file URLs for Metric export fixtures Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: address Metric export review nits Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> |
||
 shuiyisongandLei, HUANG
|
66d38e8e1c |
feat: add database ingestion admission through metering (#9239)
* feat: add `ingest_rows_rate_limit` database option Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * feat: add `InsertLimitInterceptor` hook to `Inserter` Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: attribute insert limit checks to the target table's database Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor: unify write admission through metering Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: enforce write admission for pending row batches Signed-off-by: shuiyisong <xixing.sys@gmail.com> * feat: distinguish internal requests for ingestion metering Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: admit split ingestion requests once per database Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: OpenTSDB throws error reason Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: use meter crate main rev Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: exclude database ingest rate limit from table options Signed-off-by: shuiyisong <xixing.sys@gmail.com> * refactor: reserve channel 255 for internal requests Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: shuiyisong <xixing.sys@gmail.com> Co-authored-by: Lei, HUANG <ratuthomm@gmail.com> |
||
|
Weny Xu
|
941193e9c1 |
refactor: reorganize logical table batching and isolate encoding (#9210)
* refactor: relocate the logical table batcher Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: isolate logical batch conversion and region writes Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
LFC
|
fcb165cf77 |
feat(trace): support Jaeger queries for Trace V2 and optimize writes (follow-up to #9192) (#9257)
feat(trace): support Jaeger queries for v2 and optimize fixed-column writes Signed-off-by: luofucong <luofc@foxmail.com> |
||
|
dennis zhuang
|
bbc4c4bf1d |
perf(otlp): share trace resource and scope attributes (#9253)
* perf(otlp): share trace resource and scope attributes Parsing an OTLP trace request copied the resource attributes and the scope attributes once per span. A request with wide resource attributes kept one full copy per span alive until the rows were built. Spans and their group now share one `Arc` per resource and per scope. Empty attributes stay `None`, so a resource or scope without attributes costs no allocation and no refcounting. v0 takes ownership back when it encodes, v1 clones attributes item by item instead of rebuilding a Vec, and v2 borrows the span instead of cloning it for every row. Column values, attribute order, the `service.name` lookup and the auxiliary table writes are unchanged. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * perf(otlp): stop copying v1 attribute keys The v1 row writer rebuilds every column name as `{prefix}.{key}`, so the key string it cloned from the shared resource and scope attributes was dropped unused. `resource_attributes.service.name` was also cloned in full before being skipped for the top level service name column. Shared attributes are now read in place and only values that reach a row are copied. Span attributes keep moving their values as before. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
dennis zhuang
|
007836ba7a |
fix(prometheus): honor label matchers in __name__ values query (#9134)
* fix(prometheus): honor label matchers in __name__ values query
`/api/v1/label/__name__/values?match[]={pod="abc"}` dropped every matcher
other than `__name__` and returned all metrics in the schema. No error,
just the wrong list. Grafana's metrics browser sends this request, so
picking a label value there did nothing.
Selectors that only constrain `__name__` keep answering from table
metadata. A selector constraining an ordinary label now goes to the data:
scan each metric engine physical table for distinct `__table_id` in the
time range, map the ids back to metric names, then apply the selector's
own `__name__` matchers.
Only metric engine tables are covered; other engines share no column space
to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): batch-resolve metric names by table id
Building a full table-id-to-name map meant walking every table in the
schema and holding all of them in memory, just to name the handful the
scan returned. Use `tables_by_ids` instead — one batch KV read over the
ids the scan actually produced.
The catalog walk stays, but only to find the physical tables to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): read an absent label as the empty string
A matcher on a label the series does not carry only worked when the table
had no column for it at all. Where the column exists but is NULL on that
row -- the norm for logical metrics sharing a metric engine physical
table, which holds the union of their label columns -- three-valued logic
dropped the row, so `host!="host1"` and `host=""` missed every metric
without a host label.
Coalesce nullable string label columns to "" for matchers that accept the
empty string, rather than only for the OTLP temporality marker. Equality
matchers are untouched; they cannot match NULL either way.
This is the Prometheus compatibility fix #8970 deliberately kept out of
its own scope. The cost is visible in the regex sqlness plan: the
predicate becomes a CASE, so the scan loses its LastRow selector and
grows a FilterExec. Only negative and empty-accepting matchers pay it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): don't panic on pre-epoch label value bounds
`rewrite_label_values_query` unwrapped `duration_since(UNIX_EPOCH)`, which
returns an error for an instant before the epoch. `start=1969-12-31T23:59:59Z`
parses as valid RFC3339, so the request panicked instead of answering.
Recover the sign from the error branch, and report a value beyond i64
milliseconds as an error rather than wrapping the cast.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): drop applicable_matchers, share the distinct scan
With the planner reading an absent label as empty, the frontend no longer
needs to pre-filter matchers per physical table. Removing that exposed a
second problem: a physical table that never took a column from a logical
table exposes no `__table_id`, and projecting it failed the whole request.
Skip those tables; the only thing that can miss is a metric with no labels.
Also pulls out the plan-build-execute-collect sequence the two label value
scans had in common.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
|
||
|
jeremyhi
|
dca01654bc |
feat: prepare and execute database Metric exports (#9180)
* feat: prepare and authorize captured database exports Signed-off-by: jeremyhi <fengjiachun@gmail.com> * feat: bound database export jobs and drain failures Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: validate database export identity and restore equivalence Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: validate database export directory URLs Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: preserve Windows export directory paths Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: reject local database export filename aliases Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: consolidate database export planning policies Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix: restore escaped database export filenames Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test: align database restore assertions with shared policies Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor: clarify database export boundaries and names Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com> |
||
|
LFC
|
aacf04cf6e |
feat(otlp): add trace v2 ingestion with JSON2 attributes (#9192)
Signed-off-by: luofucong <luofc@foxmail.com> |
||
|
Weny Xu
|
be15c88e92 |
feat: batch ordinary table writes across HTTP protocols (#9115)
* feat: integrate table batching across HTTP protocols Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: skip empty prepared writes before batch admission Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: load batching protocols from environment and document frontend wiring Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: remove experimental prefix from pending rows batcher config Signed-off-by: WenyXu <wenymedia@gmail.com> * style: sort frontend test dependencies Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: count batched ingestion once and update config snapshot Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
Weny Xu
|
e0915a8380 |
refactor: reuse common batching components in Prom ingestion (#9114)
* refactor: compose Prom ingestion with common batcher components Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: share bulk insert IPC encoding Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: centralize batcher worker channel creation Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: share timestamp extraction for batched writes Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: preserve sharded batcher worker lookup Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: align batcher worker lookup and configuration validation Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: validate batcher limits inside fallible constructors Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
discord9andNing Sun
|
94d7e2c7fc |
feat!: upgrade DataFusion to 55 (#8555)
* feat!: upgrade DataFusion dependencies to 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * refactor: migrate DataFusion 55 APIs Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve table function planning behavior Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve PostgreSQL query compatibility Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve distributed execution plan behavior Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: cover DataFusion 55 behavior regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: update DataFusion 55 SQLness expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: complete DataFusion 55 test API migration Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: address DataFusion 55 CI regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: address remaining DataFusion 55 regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: adapt latest base code to DataFusion 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: normalize environment-specific DataFusion 55 plans Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: align final DataFusion 55 expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: isolate DataFusion 55 regression cases Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: preserve empty result schema in timestamp widening Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: preserve JSON source column order Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * chore: use released DataFusion 55 integrations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: adapt latest execution plan mock to DataFusion 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: pin DataFusion recursive schema and date repairs Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(promql): align dictionary temporality match keys Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: retain Greptime DataFusion fork behaviors on version 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore ordinary function error expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh distributed count compatibility plan Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(query): adapt last-row cast hint to DataFusion 55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh instant last-row empty results for Arrow 59 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * style: simplify DataFusion expression visitor imports Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore sorting and PostgreSQL column-order assertions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(function): restore primitive numeric coercion signatures Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * refactor(function): share geo integer signature types Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: cover timestamp widening overflow boundaries Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: fix decimal coercion regression imports Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(function): preserve scalar count_hash NULL state semantics Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: simplify decimal clamp case type inference Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: retain historical count_hash wrapper result Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: restore timestamp widening equality and IN pruning Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: carry upstream aggregate dynamic filter correctness fix Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: carry upstream null and predicate simplification fixes Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore baseline JSON ordering expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: restore histogram JSON ordering expectations Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh empty PromQL range result schemas Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: align native timestamp plan with DF55 decimal display Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: refresh native timestamp SQLness results for DF55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: regenerate NULL sample empty result headers for DF55 Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: use DF55 child replacement API in timestamp regressions Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: expose pushed scan dynamic filters to DF55 producers Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: encode string-backed PostgreSQL OID aliases in binary results Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: verify REGPROC binary and text over PostgreSQL protocol Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: register real PostgreSQL catalogs in server fixtures Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: complete DF55 expression inventories for custom query plans Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: correct RangeSelect expression fixture and column identities Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * ci: wait for Kafka WAL helper deployment rollout Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: update custom storage empty result headers Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: require exact row counts in scan statistics Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: suppress deprecated partition_statistics warning in test Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Co-authored-by: Ning Sun <sunng@protonmail.com> |
||
|
Weny Xu
|
737025760e |
feat: support request-level WAL skipping for bulk inserts (#9110)
* feat: support request-level WAL skipping for bulk inserts Signed-off-by: WenyXu <wenymedia@gmail.com> * test: cover bulk insert WAL skipping across protocols Signed-off-by: WenyXu <wenymedia@gmail.com> * test: align WAL snapshot naming with sequence watermarks Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: bump proto Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
dennis zhuang
|
2933f95672 |
perf(servers): drop the output sort for Prometheus range queries (#9090)
* perf(servers): drop the output sort for Prometheus range queries Matrix responses sort samples by timestamp and series by labels while serializing, so the sort the PromQL planner puts at the root of a range query plan never reaches the client. Mark the query as not requiring output ordering in `do_range_query`, and let the frontend strip the root sort (through projections, and only when it carries no fetch) before execution. Instant queries are unchanged, so `sort()` and `sort_desc()` keep their observable order. Signed-off-by: Dennis Zhuang <xzhuang@greptime.com> * test(servers): pin the matrix response against input row order Signed-off-by: Dennis Zhuang <xzhuang@greptime.com> * test(servers): drop redundant response assertions in the ordering test Signed-off-by: Dennis Zhuang <xzhuang@greptime.com> * perf(servers): drop the output sort for gRPC gateway range queries Signed-off-by: Dennis Zhuang <xzhuang@greptime.com> --------- Signed-off-by: Dennis Zhuang <xzhuang@greptime.com> |
||
|
discord9
|
54fcf36452 |
fix(frontend): isolate internal Flight authentication (#9045)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> |
||
|
dennis zhuang
|
ed1f2d9f4e |
fix(pipeline): coalesce concurrent pipeline cache misses (#9022)
* fix(pipeline): coalesce concurrent pipeline cache misses The pipeline cache reads with a plain `moka::sync::Cache::get` and falls through to a distributed query on a miss, so when the 10s TTL expires every in-flight write request on a frontend issues its own scan of the single-region `greptime_private.pipelines` table. Concurrent scans per expiry scale with write QPS, and every frontend's burst lands on the same datanode. A user running high-throughput ingestion through a pipeline saw that datanode overloaded. Switch to `moka::future::Cache::try_get_with` so concurrent misses on the same key share one loader. This requires a single-key lookup, so cache entries are now keyed by the requested schema rather than the schema the pipeline is stored under; resolving a request to a stored schema stays in the loader, which is the authoritative path and already handles the empty-schema and multi-schema cases. A lookup for a schema not yet cached costs one extra read, now protected from amplification by the coalescing it enables. `remove_cache` previously only walked the compiled-pipeline cache, so an entry populated by `get_pipeline_str` alone (the pipeline read API) survived deletion until it expired. It now walks all three caches. Also make the TTL configurable as `pipeline.cache_ttl`, default unchanged at 10s. The TTL is what propagates a pipeline change to other frontends, so raising it trades staleness for fewer reads. Refs #9021 Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(pipeline): restore cross-schema semantics broken by the new cache key Keying cache entries by the requested schema dropped two behaviours that the previous stored-schema key provided for free. Creating a new version only wrote the creating request's schema, so another schema on the same frontend kept serving its cached `latest` — an older version — until the entry expired. Since the whole point of making the TTL configurable is to let operators raise it, that window is not bounded by anything useful. Creation now invalidates every schema's `latest` alias for that name before priming the cache, leaving the version-pinned keys alone. The failover cache lost its reach across schemas the same way: a global pipeline (stored under the empty schema) loaded by schema A was cached under `A`, so schema B using it for the first time while the pipeline table was down missed and failed ingestion. The failover cache has no loader and so is not subject to the single-key model of `try_get_with`; it keeps the stored-schema key and the empty-schema-first resolution. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor(pipeline): drop cache priming on create and fold the sweep helpers Priming the cache on create saved one read on a low-frequency operation and cost a concept: entries were written under the creating request's schema while `PipelineContent.schema` said empty, so the two schemas in play disagreed. Invalidating the `latest` aliases is required regardless — that is what makes a new version visible to other schemas — so dropping the priming loses only the saved read, which coalescing now protects anyway. `insert_and_compile` no longer needs the caller's schema. `remove_cache` and the create-time invalidation collapse into one `invalidate(name, version)`; `None` sweeps only the `latest` aliases, which is exactly what creation wants. That leaves `invalidate_by_suffixes` and `cache_keys` with a single caller each, so both are inlined. Drop the `PipelineOptions` humantime test: `load_config_test` loads both example TOMLs, which now carry `cache_ttl = "10s"`, and would fail the same way if the serde attribute were lost. The `toml` dev-dependency goes with it. The two invalidation tests are now checked to be orthogonal: removing the version suffix fails only the delete test, and sweeping just the compiled cache fails both. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(pipeline): keep failover populated across a create The `latest` sweep on create clears the failover cache along with the loaded ones, and after dropping the priming there was nothing writing it back. An outage between the create and the first read-back left neither `latest` nor the explicit version with anything to fall back on, failing ingestion — worse than before, since the previous version's failover entry was swept too. Creation now goes through `PipelineCache::on_pipeline_created`, which pairs the sweep with a failover write of the new empty-schema definition. The two must happen together, so they live behind one method rather than at the call site. Also commit the Cargo.lock entry for the dropped `toml` dev-dependency, and trim the comments added over the last few commits down to what the code does not already say. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
shuiyisong
|
b86da3d35f |
feat: support raw OTLP delta metrics (#8970)
* feat: support raw OTLP delta metrics Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: fmt Signed-off-by: shuiyisong <xixing.sys@gmail.com> * test(promql): update sqlness results for normalized label matching Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: derive temporality label from default column prefix Signed-off-by: shuiyisong <xixing.sys@gmail.com> * test(promql): add analyze coverage for delta temporality Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix(promql): scope label alignment to temporality marker Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: handle count-only histograms and vector broadcasts Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: exclude temporality marker from entity descriptions Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: use a fixed label for OTLP aggregation temporality Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix(promql): preserve mixed-range semantics for raw delta Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> |
||
|
discord9
|
9c135ebcb3 |
feat!: stabilize streaming analyze metrics (#8966)
* feat: stabilize streaming analyze metrics Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * feat: expose analyze memory usage Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * refactor: simplify analyze stream handling Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix: preserve analyze stream sequence on panic Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * chore: log analyze stream worker panic Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> |
||
|
dennis zhuang
|
02278d301b |
feat: derive k8s.node from OTLP resource attributes (#9002)
The k8s.pod runs_on k8s.node rule can only fire from kube_pod_info: it is the only table declaring both endpoints, and co-declared edges require both on the same row. A deployment sending only OTLP has no kube-state-metrics tables, so its node layer is invisible and that rule has no source at all, even though the resource attributes carry k8s.node.name. Declare k8s.node in otlp_trace_entities and in the synthesized resource descriptor, and project k8s.node.name in the descriptor writer so the column the declaration needs exists. Identity is the name rather than k8s.node.uid: kube-state-metrics carries no node UID, so the name is the only identity both sources can agree on. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
dennis zhuang
|
109257505e |
feat: report what the graph derives and fix two duplicate-node bugs (#8936)
* feat(semantic-graph): let the generic container yield to k8s.container A container inside a Kubernetes pod reaches the graph twice: as the k8s.container entity kube-state-metrics describes, identified by [pod uid, container name], and as the generic container the OTel resource attributes describe, identified by container.id. One physical container, two nodes. Kubernetes is the primary scenario, so k8s.container keeps its identity and the generic type stands down where it applies. Conventions gain a row-level condition for that: `suppressed_by` withdraws a declaration on rows where any of the named columns has a value. The test has to be per row, not per table — one descriptor table holds both pod rows and bare-runtime rows. Every branch that turns a declaration into rows now shares one guard (`declaration_predicate`), so the condition cannot apply to entities but not to the edges they carry. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(semantic-graph): report derived entity declarations in table_semantics information_schema.table_semantics only read table options, so the declarations the built-in conventions derive — for trace tables, for whitelisted Prometheus and OTel descriptor metrics — were invisible. "Why is my table not in the graph?" was answerable only from debug logs, which is not a self-service path. A new `entity_declarations` column reports the entities a table actually contributes: each one's identity, whether it came from an option or from the conventions, and any row-level condition attached to it. An expected entity missing from the list is the answer — the table name is not whitelisted, the source stamp is wrong, an id column is absent. The row filter widens to match: a table that declares nothing by option but derives entities by convention now appears, since it is in the graph and the view has to say so. The provider reaches the derivation through a new metadata-only trait method, keeping catalog below operator. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(semantic-graph): never yield a container to an entity nothing derives The generic container withdrew on any row carrying a pod UID, on the assumption that k8s.container would cover it. Nothing guaranteed that: k8s.container came only from kube-state-metrics, so an OTLP-only deployment — or one whose KSM data had expired or fell outside the query window — lost the container node and its edges entirely instead of gaining a more specific one. The rule now names the superseding type rather than a trigger column, and withdraws only where that type's full identity is on the row. Both OTel sources declare k8s.container themselves, under the identity kube-state-metrics gives it ([pod uid, container name]), so the two sources name one node; `k8s.container.name` joins the descriptor's projected attributes to carry it. Resolution runs once every declaration for the table is known, so a superseding type that ends up undeclared — its columns are gone, or an explicit declaration of it was skipped — leaves the guard empty and the generic container standing. A container can change type; it cannot disappear. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor(semantic-graph): build the declaration JSON by serializing a type The column was assembled entry by entry into a serde_json::Map, cloning every value and allocating a String per key. A Serialize struct that consumes the declaration moves the same data instead, and the field order now reads type, origin, identity, then description. The scan path around it was doing the same kind of avoidable work: declarations were derived before the predicate could discard the table, the supersession pass cloned every declaration's identity to look up one, and the option parse claimed the time index for tables that turn out to declare nothing. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(semantic-graph): keep the structured identity for single-column ids `entity_id_attrs` was NULL whenever the identity came from one column, so a consumer holding `host` = `a3f2...` had no way to tell which column produced it, and no way back to the source table. Most identities are single-column — host, k8s.pod, k8s.node, service — so the common case was the opaque one. Build the JSON object unconditionally. Entity equality still reads `entity_id` alone, so this changes no merging: it records which attributes the id was assembled from, beside an id that deliberately omits them. Also corrects the `entity_id` column doc, which still described the `k=v,k=v` rendering replaced in #8904 by values joined in declared order. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix(semantic-graph): keep what a container carried when it yields Yielding to k8s.container cost the row two things it had as a generic container. The runtime container id, which kube_pod_container_info keeps descriptive precisely because it is the handle back to runtime logs and metrics, went missing entirely: neither an id nor an attribute of any node. And the edge vocabulary knew only the generic type, so a row with the more complete labels ended up with fewer connections than one without — the container layer no longer reached its host. Both OTel sources now keep the runtime id and name descriptive on k8s.container, and the vocabulary gains the two edges that mirror the generic type's. Nothing checks a supersession against the edge vocabulary, so that requirement is written down where the rule is. Also: the conventions-failure path now reports the explicit half as its comment always claimed, entity_declarations reports scope columns, and identifies() is private again now that only declaration_predicate calls it. The two RFCs catch up with entity_id_attrs being unconditional, the view listing convention-derived tables, and supersession existing. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(semantic-graph): report unmatched clients and the longest request `real_wins` pins a `calls` edge's RED metrics to the observed span pairs: when a window's edge key holds any pair, the unmatched client spans are suppressed so one (window, edge) yields one row. That leaves no way to tell a callee that stopped responding from traffic that stopped arriving — both show up as a lower request_count. Add two columns to `semantic_relationships`: - `unmatched_count` — client spans with no server span, counted outside `real_wins` on the same row, so the suppressed population stays visible without splitting the edge into two rows. NULL for agent calls, whose inner join leaves nothing unmatched, and for declared edges. - `duration_max` — the longest single request. It goes through `real_wins`: a pair is timed by the server span while an unmatched client is timed by its own (network wait included), so mixing them would make the max describe a different population than duration_sum and duration_count. Agent calls compute it from the child spans they already aggregate. The projection contract goes from 16 to 18 columns; every branch projects both. Explicit column queries are unaffected, `SELECT *` and ordinal-based readers see the new shape. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test(semantic-graph): drop the duplicated source-gate case The whitelist gate on `source=opentelemetry` is already covered by `otel_implicit_declarations_are_gated` and by the wrong-source case in `table_semantics`; here it only paid for another table create, insert and full graph derivation. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * style(semantic-graph): trim the comments back to what the code cannot say Several comments restated the code, repeated a rationale already stated at the type or in the RFC, or explained a test in more words than the test body. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
Ning Sun
|
d32cd77505 |
fix: postgres describe for more statements (#8974)
* fix: postgres describe for more statements Signed-off-by: Ning Sun <sunning@greptime.com> * fix: cover more show statements Signed-off-by: Ning Sun <sunning@greptime.com> * fix: address review comments - add missing `clippy::too_many_arguments` allow on `query_from_information_schema_dataframe` (CI clippy failure) - take `&ShowKind` in the information-schema dataframe helper so `kind` is no longer cloned at every call site; only the WHERE arm (which needs an owned expression for `sql_to_expr`) clones internally - document why re-applying TQL explain formats never overwrites an existing value (per-query context state) Signed-off-by: Ning Sun <sunning@greptime.com> * chore: trim comments to essentials Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com> |
||
|
Lei, HUANG
|
a2f39ecf7b |
feat: harden frontend heartbeat extensions (#8803)
Add a typed `Command::build_with_heartbeat_extensions` seam in `src/cmd/src/frontend.rs`. Freeze and harden `FrontendHeartbeatExtensions` in `src/frontend/src/heartbeat.rs`, with lifecycle and race coverage in `src/frontend/src/heartbeat/tests.rs`. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> |
||
|
dennis zhuang
|
6d86e6ff06 |
feat: synthesize OTLP resource descriptor for the semantic entity graph (#8904)
* fix(servers): compose OTLP metrics job from service.namespace/service.name
The OTel Prometheus compatibility spec defines job as
"<service.namespace>/<service.name>" when the namespace is present.
The OTLP metrics path only used the bare service.name, so the job tag
diverged from target_info produced by Prometheus-side exporters for the
same resource. Compose the namespace form, and keep not fabricating a
job when service.name is absent.
Behavior change: resources carrying service.namespace now get
"namespace/name" as their job tag value.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(otlp): synthesize otel_resource_info at OTLP metrics ingestion
Ordinary OTLP metrics scatter filtered resource attributes as tags over
every logical metric table, so metrics-only services contribute nothing
to the semantic entity graph. Each request now also projects its
distinct resources into one info-metric-shaped mito table,
otel_resource_info: a fixed allowlist of identity-relevant attributes
under their raw OTel keys (independent of the label translation
strategy and the promote/ignore headers) plus derived job/instance
compatibility columns, value 1.0, and the newest data-point timestamp.
The descriptor is written after the main insert is committed; a failure
there (conflicting pre-existing table, auto-create disabled) degrades
to an OTLP partial_success warning with rejected_data_points = 0
instead of failing the request and triggering client retries of
already-accepted data. A request writing a metric named
otel_resource_info suppresses synthesis. Legacy mode is unchanged.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(operator): otel info-metric conventions with host/container entities
Whitelist the ingestion-synthesized otel_resource_info descriptor via a
new otel_info_metrics conventions map, gated on source=opentelemetry
(the existing gate hardcoded source=prometheus). Its declarations use
explicit descriptive lists instead of descriptive_rest so identifying
attributes of other entities do not leak into service.instance.
Conventions tightened per the Astronomy Shop findings: host identity is
host.id with host.name descriptive only (host.name is not stable across
SDKs and resource detectors), a generic container entity (new entity
type) is declared only when container.id is present, and trace-v1
tables now synthesize host/container from their flattened resource
attributes too. New co-declared edges: service.instance runs_on
container, container runs_on host.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(otlp): cover the resource descriptor in integration tests
Covers the descriptor's raw-key columns and info-metric options through
the HTTP path, the namespace/name job composition end-to-end, column
names being independent of the translation strategy, the allowlist
excluding unlisted resource attributes, auto-create after a drop, the
metric-name collision suppressing synthesis, and the partial-success
warning (rejected_data_points = 0) when a pre-existing incompatible
table fails the descriptor write while metric data is accepted.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: cargo fmt
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(frontend): degrade descriptor permission denial to a warning
A table-level permission policy denying otel_resource_info would have
failed the whole OTLP metrics request because the descriptor's
permission check ran before the main insert. The descriptor is derived
enrichment: check its permission in the degrade path so a denial skips
the write and surfaces as the partial-success warning, like any other
descriptor write failure.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(otlp): guard descriptor writes with semantic ownership markers
A pre-existing schema-compatible table named otel_resource_info would
silently receive descriptor rows while its missing semantic stamps kept
it out of the entity graph. The descriptor write now requires the
auto-created table's ownership markers (mito engine + signal_type +
source + metric.type=info + metadata_quality=declared) and otherwise
degrades to the partial-success warning; the entity-graph gate for the
otel whitelist likewise requires metric.type=info, so a user table
stamped with only signal/source no longer picks up implicit
declarations.
Also fold the descriptor write cost into the response and surface the
degrade warning through the otel-arrow BatchStatus status_message.
Integration tests pin the full marker set on auto-create and that an
existing owned descriptor keeps accepting writes without degrading —
a missing marker would otherwise silently stop every descriptor write
after the first request.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(otlp): build descriptor rows without the per-resource BTreeMap
Projecting a resource allocated a BTreeMap and then collected it into the
row key, and every attribute was matched against the allowlist by linear
scan. Collect the tags into a Vec and sort once, and match the allowlist
instead of scanning it. Measured on the conversion path: descriptor work
drops 16-18%, from 10.6% to 8.9% of conversion CPU on the worst shape
(1000 resources with 4 data points each), where the cost tracks resource
count rather than data-point count.
Also trims the comments and tests added with the descriptor to what
carries information.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(otlp): pin the descriptor permission-denial degrade path
A policy denying the descriptor table must not fail the metrics request,
which the fix in
|
||
|
shuiyisong
|
1c5eabcbbf |
feat(otlp): support cumulative exponential histograms (#8900)
* feat: implement exponential histogram Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: remove duplicate tests Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix(otlp): enforce exponential histogram ingestion safety Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: update rfc Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: test Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix(otlp): remove protocol-coupled histogram checks Signed-off-by: shuiyisong <xixing.sys@gmail.com> * perf(otlp): reuse native histogram schema across data points Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: merge repeated OTLP histogram fragments Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix(otlp): build rejection messages lazily Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: add doc Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> |
||
|
Ning Sun
|
174577164a |
feat: otlp duration_nano and trace_flag signed integer coercion (#8816)
* feat(servers): add signed→unsigned int coercion to OTLP ingest path Phase 0 of transitioning built-in data models from unsigned to signed integers (#8793): add lossless Int64→UInt64 and Int32→UInt32 coercion arms so existing UInt64/UInt32 columns (e.g. trace `duration_nano`, log `trace_flags`) keep accepting new signed ingest without an ALTER TABLE. The OTLP ingest path already reconciles every incoming column against the existing table schema and treats it as authoritative. With these arms, `choose_trace_reconcile_decision` returns `UseExisting(UInt64/Uint32)` for an existing unsigned column receiving signed data: the table keeps its type byte-for-byte and the request value is coerced. No persisted format is mutated; existing data stays readable as-is. This is the safety net that makes the actual schema flip (Phase 1) safe. Only the signed→unsigned direction is supported — the reverse would be lossy for values above the signed range and is intentionally rejected. Tests cover both new arms plus an end-to-end log test proving an existing UInt64 column coerces an incoming Int64 value while keeping its type. Signed-off-by: Ning Sun <sunning@greptime.com> * feat: add compatibility layer for uint trace/log fields Signed-off-by: Ning Sun <sunning@greptime.com> * fix: jaeger test Signed-off-by: Ning Sun <sunning@greptime.com> * fix: keep trace v0 unsigned, reject negative span durations - Keep the frozen v0 data model on UInt64 duration_nano: the signed ingest compatibility layer only runs on the v1 path, so flipping v0 would break writes into every pre-existing v0 table at mito's schema check. Pin the schema with unit and integration tests. - Reject spans whose end precedes their start (or whose duration does not fit i64) on the v1 path instead of wrapping: new Int64 tables and existing UInt64 tables now fail identically, rather than storing negative durations that break the Jaeger query API. - Extract is_supported_signed_to_unsigned_coercion so the trace and log ingest paths share one supported-pair predicate and cannot drift. Signed-off-by: Ning Sun <sunning@greptime.com> * fix: clamp negative span durations to zero, revert semantic_graph comment - Record duration 0 for spans whose end precedes their start instead of erroring: a malformed span no longer fails the request, and the value written is always a non-negative, in-range i64 so new Int64 tables and existing UInt64 tables (via the checked coercion) behave identically. Durations above i64::MAX saturate rather than wrap. - Revert the doc-comment tweak on the semantic_graph test fixture; the file is untouched by this PR again. Signed-off-by: Ning Sun <sunning@greptime.com> --------- Signed-off-by: Ning Sun <sunning@greptime.com> |
||
|
dennis zhuang
|
3de4feddd5 |
feat(otlp): report the cause of rejected trace spans (#8897)
* feat(otlp): report the cause of rejected trace spans
When trace-v1 ingestion cannot coerce an attribute value, it falls back to
single-span writes and rejects the bad span. That behavior is correct, but the
OTLP partial-success message only carried `Rejected span <trace_id>:<span_id>
(InvalidArguments)`: the column, the source value, the source type and the
target type were all dropped, so locating the bad attribute required adding a
detailed exporter on the collector side and replaying traffic.
Two places lost the information. `prepare_trace_column_rewrites` built a message
without the failing value, and the span rejection path kept only the status code
from the error.
Coercion errors now name the failing value, e.g.
failed to coerce trace column 'span_attributes.http.response.body.size'
in table 'opentelemetry_traces' from String("") to Int64
and the rejection detail carries that cause. Values are user data, so a string
keeps at most 16 characters, is escaped, and binary payloads report only their
length; the cause itself is bounded at 256 characters. Both truncations cut on a
char boundary.
Failure details now deduplicate: repeats of the same (site, cause) collapse into
one entry with an occurrence count, keyed on the untruncated cause so two
failures that differ past the display limit stay separate. Only four distinct
entries are retained and the rest are counted, which keeps the state bounded no
matter how many distinct bad values a request carries. A fully rejected request
is logged at warn level and a partial success at debug level, since the latter
repeats every export interval; the detail goes out as a Debug field so a newline
in an attribute key cannot forge log lines.
Rejection semantics are unchanged: partial success, same accepted and rejected
counts, same HTTP status mapping.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(otlp): compare failure keys directly instead of hashing
The dedup identity was a DefaultHasher fingerprint of `(label, key)`, which
bought a fixed 8 bytes per entry at the cost of an import, four lines, and a
collision argument the reader has to make. Entries are capped at four and a
cause runs a couple of hundred characters, so the saving is about a kilobyte
per in-flight request while the column name it avoids retaining is already held
several times over by the request itself.
Compare the strings instead, keeping the untruncated cause as the key so
failures differing past the display limit still stay apart. Labels are metric
label values and always static, so the entry borrows them.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
|
||
|
Ning Sun
|
8dbfbee611 | refactor: remove open metrics parser (#8905) | ||
|
Lei, HUANG
|
7539e60139 |
refactor: remove trivial tests (#8877)
* refactor: remove trivial tests Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: remove unused trace test import Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> |
||
|
Weny Xu
|
1af4c33524 |
refactor(event): separate procedure submission context (#8856)
* refactor(event): separate procedure submission context Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): map extensions and forward GC context Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(gc): initialize integration test context Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(test): pass procedure context to DDL helpers Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor: simplify procedure submission contexts Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): separate procedure and query contexts Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): clarify procedure context propagation Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): preserve procedure submission context Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): move DDL context by value Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(test): retain manual GC event context Signed-off-by: WenyXu <wenymedia@gmail.com> * refactor(event): tighten procedure context API Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: update greptime-proto Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
dennis zhuang
|
546625c45a |
feat: embedded convention pack for the entity graph (prom/k8s, gen_ai naming) (#8854)
* feat: embed the derivation conventions as data and adopt gen_ai entity naming Move the co-declared edge vocabulary, the agent-edge vocabulary and the virtual-destination candidates from Rust consts into an embedded conventions.yaml (include_str!), parsed once behind a LazyLock and validated against the entity-type grammar and the closed rel_type set; a broken file propagates as a plan error instead of panicking. The agent vocabulary entity types follow the GenAI semantic-convention namespace as written: gen_ai.agent / gen_ai.model / gen_ai.tool. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat: drop the tag requirement for entity identity columns Entity declarations no longer require id columns to be tag/primary-key columns; only column existence is validated. Trace pipelines flatten the identifying attributes (span_attributes.gen_ai.agent.id, ...) into field columns, so the tag rule locked real trace tables out of declaring entities while buying no correctness — the read-time derivation works on any column. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat: implicit declarations for well-known prometheus info metrics Tables stamped signal_type=metric + source=prometheus whose name matches the conventions.yaml whitelist (kube_pod_info, kube_node_info, kube_pod_owner, target_info) get implicit entity declarations: k8s.pod / k8s.node / k8s.workload with name-based identity and target_info's service / service.instance with the remaining tags as the descriptive snapshot. The existing co-declared vocabulary then derives runs_on and part_of from the same rows, so no new edge branch is needed. Explicit declarations of a type always suppress the implicit one, and the metric engine's physical table is excluded (it aggregates every logical table's columns). Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test: cover the prometheus conventions in sqlness and compact the graph cases Add the whitelisted-info-metric scenario (kube_pod_info, kube_pod_owner, target_info deriving runs_on / part_of, a non-whitelisted metric contributing nothing), fold the single-table calls, cross-table pairing and virtual-node cases into one trace scenario (they exercise the same union-before-join path), merge the two declaring-metric-table cases, and reuse one rename probe for both reserved names. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: reject entity id columns without a stable string form Review follow-ups: the DDL check now validates against the schema and rejects binary-backed and nested types for identity columns (the derivation renders ids via CAST to Utf8, so the failure used to surface only when the graph was scanned); the agent sqlness case keeps its identity columns as fields to cover the relaxed tag rule end to end; stale tag-rule comments and a dangling const reference are cleaned up. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: type-check every entity column role, not only ids The registry renders scope and descriptive values through the same CAST-to-string path as ids, so a binary-backed column in any role fails at scan time; the DDL check is now role-independent (and simpler). Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor: name the code-anchored vocabulary constants Entity types and edge attributes the derivation code itself anchors on (service, gen_ai.agent, calls, trace/attribute provenance) become constants in the conventions module; the rest of the vocabulary stays YAML-only data. ImplicitEntity is renamed PromImplicitEntity, and the implicit-declaration path logs each skip of a whitelisted info metric (wrong stamps, suppressed by an explicit declaration, missing id column) so a missing graph entity is diagnosable. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor: single-source the graph constants The graph tables' column names move to common-catalog (the schemas catalog exposes and the plans operator builds must match column by column), and the conventions module now carries the complete built-in vocabulary — entity types, rel_types, provenances and connection types — with the embedded YAML validated by membership against it, so an edit drifting outside the vocabulary fails the conventions test instead of deriving nothing. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: treat empty identity components as absent kube-state-metrics emits empty-string labels an entity id must not be built from: an unscheduled pod's node and an owner-less pod's owner_kind / owner_name. Standard Prometheus drops empty labels (they arrive as NULL and the existing predicate handles them), but other remote-write agents may keep them, which produced ghost entities with empty ids and false runs_on / part_of edges. Every identity predicate (registry, co-declared edges, span endpoints) now requires non-NULL and non-empty components through one shared helper. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * refactor: tighten the conventions DSL semantics Rename the co-declaration rule lists to what they are (co_declared_edges / trace_co_declared_edges — derivation rules, not a relation vocabulary), stop overstating the GenAI entity types (Greptime types derived from GenAI attributes; OTel defines no model/tool entities), move target_info's descriptive snapshot to service.instance (the remaining labels are the target's resource attributes, and instances would write conflicting snapshots onto the logical service), and extend the descriptor whitelist with the stable KSM sources: container info metrics (closing the k8s.pod contains k8s.container rule), kube_service_info (new k8s.service entity type) and the fuller descriptive label sets. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: guard entity column types on ALTER as well ALTER MODIFY COLUMN could change a declared entity column to a type without a stable string form, deferring the failure to graph scan time; verify_alter now checks the post-alter schema. Dropping a declared column stays allowed — the read-time derivation skips the stale declaration, and semantic options cannot be altered off yet. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat: bridge traces and kube-state-metrics on the pod UID Trace-v1 tables now get implicit declarations from their flattened resource attributes (otlp_trace_entities in conventions.yaml): the service identity — replacing the hardcoded fallback — plus service.instance and k8s.pod, each applied only when its columns exist. A new co-declared rule derives service.instance runs_on k8s.pod, and the whitelisted kube-state-metrics pod identity switches from namespace+pod names to the UID, so the trace-side pod and every KSM descriptor land on one entity while names stay descriptive. This also removes pod identity from the multi-cluster same-name collision. The conventions rejection tests were passing for the wrong reason (a half-renamed fixture key failed deserialization before reaching any validation rule); they now assert the specific error each case targets. Sqlness covers the UID merge across descriptor tables, pod-contains- container, the k8s.service node, and the empty-uid/empty-node rows deriving nothing. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test: cover the OTLP-to-graph chain end to end One real OTLP export must come out of semantic_relationships as the zero-configuration chain: service calls service, instance part_of service, instance runs_on pod (bridged by k8s.pod.uid). Resources without service.instance.id or k8s.pod.uid derive nothing extra. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: identify k8s.service by UID Same reasoning as pods: a recreated same-name service must not merge into the old entity and same-named services across clusters must not collide; kube_service_info carries a stable uid and nothing joins on the service's name. Also drop a stale tag-rule mention from the option validation docs. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: cut duplicated test coverage and redundant comments The trace service-fallback test collapsed into the resource-entities test (same synthesis path since the fallback moved to YAML; only the invalid-explicit-no-fallback case was distinct), role-duplicate and subsumed DDL cases are gone, the embedded-conventions test is just the parse (its assertions were decorative), and the YAML section comments no longer restate the struct docs. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
dennis zhuang
|
e778a72829 |
feat: complete the derived-edge vocabulary of the entity graph (#8836)
* feat(operator): pair calls edges across trace tables and derive virtual-node edges Union the normalized client and server spans of all trace tables before the join, so a client span pairs with a server span stored in a different table. A client span with no matching server span becomes an edge to a virtual node named by span attributes (peer.service / db.name / server.address), with confidence < 1.0 and attributes.connection_type; a window's real pairs win over virtual candidates for the same edge key. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(operator): derive same-row co-declared edges from the built-in vocabulary A table declaring both entity types of a vocabulary pair witnesses the edge on every row carrying both identities: runs_on / contains / part_of for any declaring table (provenance 'attribute'), agent uses model / agent invoked tool only for trace sources (span-structure observations, provenance 'trace'). Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(operator): derive parent_agent-calls-agent edges from span structure Trace tables declaring an agent entity pair each span with its child span across tables (no span-kind filter), keep pairs whose agent identities differ, and aggregate RED metrics per window, anchored on the parent span like the service derivation is anchored on the client. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(frontend): feed co-declared and agent sources into the relationships scan scan_relationships now passes every declaring table (with its trace-ness) to the co-declared branch and the trace tables' agent declarations to the agent-calls derivation. enumerate validates the fixed trace-v1 columns and derives around a malformed trace table instead of failing the whole scan. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test: cover cross-table pairing, virtual nodes, co-declared and agent edges sqlness exercises the new derivations end to end (including a malformed trace-model table being skipped); the integration authorization test now also pins that a pair split across tables derives no edge when the caller cannot read one side. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: update the relationships module doc for the new branches Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: import shared derivation helpers via crate paths The fmt CI gate rejects module-level 'use super::' imports. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: fold co-declared duplicates, decouple agent calls, verify the trace time index Review findings: the co-declared branch lacked a cross-source DISTINCT, so two tables witnessing the same edge in one window emitted duplicate rows; the agent-calls derivation was gated on a usable service declaration; the trace schema guard accepted a table whose time index is not the column the derivations bucket by. The empty-trace-table test asserted a union invariant with no information and is dropped. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: rename the agent-tool edge to invokes and track current OTel peer attributes The vocabulary's other relation names are present tense; semconv 1.39/1.26 replaced peer.service and db.name with service.peer.name and db.namespace, so the virtual-node candidates now check the current names first and keep the deprecated ones for existing telemetry. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: trust the trace-v1 table option instead of matching the fixed schema The option is only ever stamped by the ingest path, which guarantees the fixed span columns; matching column types here couples the graph to every trace schema evolution (e.g. #8816) for a case that cannot occur. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
shuiyisong
|
154f90b365 |
fix: harden permission checks and process visibility (#8852)
Signed-off-by: shuiyisong <xixing.sys@gmail.com> |
||
|
Weny Xu
|
943eee852f |
feat(event): record admin function executions (#8835)
* feat(event): record admin function executions Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): handle admin function recording edge cases Signed-off-by: WenyXu <wenymedia@gmail.com> * feat(event): record actor for admin functions Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): preserve admin function event values Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(event): preserve non-finite admin results Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> |
||
|
dennis zhuang
|
4eeb4052d6 |
feat(servers): stamp prometheus remote write v2 metadata as semantic table options (#8797)
* feat(servers): stamp prometheus remote write v2 metadata as semantic table options
Remote write 2.0 carries per-series metadata (type, unit, help) that the
v2 ingest decoded and dropped; tables kept the name-based 'inferred'
quality. Wire it into the semantic layer:
- generalize the OTLP per-table semantic index into a shared, schema-
aware servers::semantic module: v2 lets each series override its
target schema, so the index is keyed {schema -> table -> options} and
the same metric name in two schemas no longer collapses;
- into_write_requests records metric type and unit per written table;
an explicit type upgrades the table's metadata quality to declared,
UNSPECIFIED series keep the request-level inferred stamp, and units
are canonicalised from OpenMetrics words to the UCUM codes the
vocabulary is defined in (unknown units are dropped, help text is not
persisted);
- the consumer folds the index in on both auto-create paths: the
operator row-insert path and the pending-rows batched create, which
bypasses the former.
Table options are stamped at auto-create only; updating existing tables
from later metadata is future work (a metadata registry, see the native
histograms RFC).
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: trim over-commenting in the remote write metadata path
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: parse the per-table semantic index once per create round
The index was re-parsed from JSON for every table being created — a
first write creating N tables (a fleet's first scrape) paid
O(N x index size). Parsing now happens lazily once per create-planning
round, on both consumers: the operator row-insert auto-create (also
serving OTLP metrics) and the pending-rows batched create.
Also validate every non-zero metadata symbol reference up front, as the
remote write 2.0 spec requires: help_ref was never checked, and
unit_ref escaped checking when the metric type was UNSPECIFIED.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(servers): stamp remote-write v2 unit independently of metric type
OpenMetrics models TYPE and UNIT as independent MetricFamily metadata, and
the Prometheus v2 sender emits UNSPECIFIED-type series that still carry a
unit. The early return on UNSPECIFIED dropped that unit, which is
unrecoverable after table auto-create (units are not stored in rows).
Stamp the mapped unit whenever present; the type and the
metadata_quality=declared upgrade still require an explicit type.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: trim restating comments in the v2 metadata path
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
|
||
|
dennis zhuang
|
335a95a369 |
feat: declared edges and the derivation contract for the entity graph (#8794)
* feat(frontend): run entity-graph derivation as the caller The derivation contract requires the computed graph tables to run under the outer query's identity. Capture the caller's QueryContext when the computed table is resolved, thread it through EntityGraphProvider, and: - authorize every contributing source table against the caller via the new semantic_graph.query permission action, silently excluding denied sources (entities, edges and source_tables never appear); - execute the derivation plan under the caller's context so it inherits permissions, cancellation and deadline instead of a fresh default. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(operator): derive the entity-graph window from the scan's time predicate Implements the RFC window contract for the computed graph tables: - table: add extract_time_range_strict, a strict variant of the lenient time-range extraction that distinguishes an absent observed_at filter from one that cannot be safely turned into a range; - operator: replace GraphWindow with GraphQueryWindow, splitting the queried observed_at range from the source-scan range widened to whole 60s buckets, so boundary buckets aggregate over their full extent; - frontend: resolve the window from ScanRequest filters — no predicate keeps the last-hour default, a missing upper bound means now, and a missing lower bound or unextractable shape is an explicit error, never a silent fallback. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(operator): system-defined declared-edge table for the entity graph Reintroduces greptime_private.semantic_relationships_declared with a canonical, system-owned definition: - the CREATE TABLE expr (8-tag primary key, business validity columns, RED fields, 30d TTL); attributes is now a json column so the future union branch matches the computed table without a per-scan parse; - created on first use on every write path: SQL INSERT creates it before executing, and the gRPC row-insert auto-create substitutes the canonical expr instead of deriving a schema from the request; - user DDL (CREATE/ALTER/DROP/RENAME/TRUNCATE) and write-path auto-ALTER are rejected via the new is_ddl_reserved_table guard, while INSERT/DELETE stay allowed. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat(operator): union declared edges into semantic_relationships Adds the declared-edge branch to the relationship derivation (build_relationships_plan replaces build_calls_plan): - latest revision per edge key first (mito dedups on primary key plus observed_at, so a re-asserted edge stores a new revision), then the business-validity overlap against the queried window; valid_from defaults to the declaration time and a NULL valid_until means the edge holds while its row exists; - the projected observed_at is synthesized inside the queried range (Inexact pushdown re-applies the scan's filters above the computed table, which would drop rows keyed by the physical revision time); window_end/fresh_until of open-ended edges take the window's upper bound so 'fresh_until >= now() - ...' queries see them; - tag columns are cast out of dictionary encoding, and the union is re-projected to the 16-column contract; - the frontend feeds the branch only when the physical table exists, the caller may read it, and its schema still matches the canonical definition (mismatch is an explicit error, not a silent drop). Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test: cover declared edges, window contract and caller authorization - sqlness: system auto-create on first INSERT, latest-revision reads, open-ended vs retired validity, explicit/lower-only/upper-only window behavior, user-DDL rejection, rename-into rejection, DELETE cleanup; - integration: a permission checker denying one trace table excludes it from both semantic_relationships and semantic_entities. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: allow DROP/TRUNCATE on the declared-edge table and fix CI lints The definition guard rejected every DDL, which left sqlness (and any shared deployment) no way to remove the table the semantic_graph case creates — its extra region then broke unrelated region/partition case expectations. Narrow the guard to what actually protects the canonical definition: user CREATE, ALTER, RENAME-into and repartition stay rejected, while DROP and TRUNCATE are allowed — dropping loses nothing structural, the next INSERT recreates the table canonically, and DROP doubles as the recovery path if the canonical definition ever changes. The sqlness case now verifies drop-then-recreate and cleans up after itself. Also: rustfmt for the catalog crate and two typo fixes. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: adapt canonical declared-table create to TriggerReason Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: address review on the declared-edge table lifecycle and revision reads - gRPC first writes actually work now: the reserved table's creation went through the generic create_table_inner, which the definition guard itself rejects; both branches of create_or_alter_tables_on_demand route it to create_declared_relationships_table instead, and being a system action it also bypasses the auto_create_table config/hint; - revision selection is as-of the queried window: revisions recorded after the window's end, or whose validity starts after it, no longer outrank (and hide) the revision that was in effect inside it; - the canonical-schema check validates the whole definition the union semantics lean on — time index, primary key, engine, append/merge mode — not just column names and types; - UNDROP TABLE of the reserved name is rejected like CREATE: it could resurrect a pre-canonical shape, and the next INSERT recreates the table anyway. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * chore: trim over-commenting in the entity-graph code Comments that restated adjacent code or narrated justification are cut; the ones stating non-obvious contracts and gotchas stay. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: reject CREATE VIEW against DDL-reserved table names A view named greptime_private.semantic_relationships_declared would squat the reserved name: the first INSERT then skips the canonical create (an object already exists) and graph reads fail on the schema mismatch. CREATE VIEW now passes the same definition guard as CREATE TABLE. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * feat: debug-log authorization exclusions; declared-edge TTL to 90d Sources the derivation contract silently excludes (per-table denial, whole-scan denial, the declared-edge table) are invisible from outside; a debug log at each names what was excluded and why. The declared-edge table's default TTL becomes 90d, overridable at creation time via GREPTIMEDB_DECLARED_RELATIONSHIPS_TTL (a proper configuration option is a TODO). Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: rank declared-edge revisions by the visible edge identity Ranking partitioned by the full primary key, but the projection drops scope and generation_id: two assertions of the same visible edge under different generations both ranked first and came out as duplicate, indistinguishable rows. Rank by the exposed identity (endpoints, rel_type, provenance) instead, with generation_id/scope as deterministic tie-breakers for same-timestamp assertions. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * test: drop redundant declared-edge tests The generations regression is already asserted by the revision and as-of tests; the DDL shape test restated the declarative builder against itself. Its one non-tautological check (attributes maps to the json type) moves into the schema-matcher test. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * fix: reject disjunctive graph windows and unmatchable future windows - OR/IN over observed_at collapse disjoint ranges into their convex hull; a declared edge's synthesized timestamp can land in a gap and be dropped by the re-applied filter even though the edge is valid at a requested instant. The strict extractor now rejects those shapes. - A lower bound in the future inverts against the implicit up-to-now upper bound; the declared branch then fabricated an edge observed at the future bound. Such windows now derive nothing. - The reserved-table gRPC create path classifies an instant-TTL table like every sibling path. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> |
||
|
shuiyisongandfys
|
d4af650ec0 |
perf: reduce cold workspace compile time (#8801)
* refactor: remove datanode and meta-srv dep from frontend Signed-off-by: shuiyisong <xixing.sys@gmail.com> * refactor: use on-device protoc if possible Signed-off-by: shuiyisong <xixing.sys@gmail.com> * refactor: remove unused dep Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: CR issue Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: version and docs Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: update logs Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com> |
||
|
Ning Sun
|
606e5fd6a0 |
feat: update opentelemetry family to 0.32 series (#8776)
* feat: update openetelemtry family to 0.32 series * chore: resolve warning * fix: update tests |