* docs: add entity relationships and graph query RFC
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: revise entity-graph RFC after review
- Zero-configuration declarations: the Prometheus-on-Kubernetes convention
pack (job/instance per the OTel-Prometheus compatibility spec, k8s SD
labels, *_info descriptors, target_info enrichment) alongside the OTLP
trace auto-stamp, plus Remote Write 2.0 inline metadata.
- Calls endpoints follow the service entity declaration; self-calls
compare full endpoint ids; no silent identity fallback.
- Strict time-window contract: the source window is never narrower than
the query's observed_at range; unsafe-to-extract predicates error
instead of silently defaulting.
- scope removed from the relationship schema (kept on entities as a
display property); entity row contract restated per projected
observation; endpoint encoding documented as the v1 storage-level key
with its known collision limitation.
- Sampling caveats corrected (ratios are representative only under
unbiased sampling); snapshot relation synthesizes endpoint-only
vertices; shared attributes provide join keys while co-declaration
provides relationship semantics.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: align entity-graph RFC contracts and tighten prose
- Metric/trace service unification is promised only when service.namespace
is empty and job is not relabeled (the compatibility spec renders job as
<namespace>/<name>); otherwise alignment needs pipeline normalization or
explicit declarations.
- Entity row contract stated once (per projected observation); the calls
defining SQL is marked as the single-column simplification of the
declaration-derived endpoint ids; Remote Write 2.0 metadata intake and
the Prometheus implicit declarations are listed as M1 work.
- Compress survey/example/reference prose.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: pin down the k8s convention pack rules and edge directions
- kube_pod_owner implicitly declares k8s.workload with id
(namespace, owner_kind, owner_name) and derives pod part_of workload;
target_info's non-job/instance labels are implicit service descriptive
columns — fixed rules, no new declaration syntax.
- One direction for pod placement: k8s.pod runs_on k8s.node (pod added to
runs_on sources; node->pod removed from contains).
- Drop the remaining 'canonical' wording for the v1 storage-level id.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: make contains/part_of a true inverse pair
part_of covers service.instance->service and k8s.pod->k8s.workload with
contains as its inverse; has_instance is dropped.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: rewrite entity-graph RFC as a design narrative
Restructure for human review: mainline first, cases illustrate the
design instead of specifying it.
- New Architecture and Benefits and Drawbacks sections; the calls
derivation stays expanded as the flagship example while schema
enumerations, window-rule listings, and executor edge-case handling
move out of the document.
- The cross-signal promise is stated honestly: neighbours and their
source tables are discovered first, their telemetry is the next
query — one engine, two statements; the worked example shows the
full declaration -> entity -> edge -> telemetry flow.
- Default materialisation added as the most direct alternative to
read-time derivation, with its costs.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: final wording pass on the entity-graph RFC
Mark the property-graph DDL as illustrative rather than settled M2
syntax, credit standards as foundations rather than claiming wholesale
alignment, and clean up punctuation-heavy prose.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: keep the RFC at design altitude
Demote the convention-pack rule details and snapshot property-merge
semantics to the implementing changes; correct the single-trace-table
assumption (traces can be routed to multiple tables); record
attribute-key participation in entity equality as an open question.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: correct service graph terminology
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: table semantic layer information_schema view (Phase 3)
Add `information_schema.table_semantics`, a queryable view over the table
semantic layer. One row per table that carries at least one
`greptime.semantic.*` option: the signal-agnostic keys
(signal_type/source/pipeline/metadata_quality) are promoted to columns and
the remaining signal-specific keys are folded into a `semantic_options`
JSON string. Tables with no semantic key are excluded.
Stacked on Phase 2.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: address PR review on table_semantics
- fold JSON serialization failure into None instead of unwrap/panic
- drop per-row Vec allocation in predicate eval; use a fixed array
- align RFC view name with the shipped `table_semantics`
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: update results
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: table semantic layer per-table enrichment (Phase 2)
Phase 2 of the table semantic layer, plus a vocabulary trim so the layer only
records what a machine consumer cannot cheaply recover on its own.
Per-table metric enrichment (OTLP), via an internal per-table channel:
- A `SemanticIndex` accumulator records, per emitted table, the declared metric
keys: type / unit / temporality / metadata_quality=declared / original_name.
Conflicting single-valued keys collapse to `mixed`/`unknown`.
- Recording happens at the `encode_metrics` level where the base name, metric
type, and proto fields are all in scope, so histogram/summary fan-out gets the
correct per-subtable type (`_bucket`=histogram, `_sum`/`_count`=counter)
without threading state through every encoder.
- The index is serialized onto the `greptime.internal.semantic.per_table_index`
context extension; `apply_per_table_semantic_options` folds each table's keys
into its options at auto-create time.
- `trace.conventions` is refined from the request's resource/scope `schema_url`s
(concrete when uniform, else `mixed`/`unknown`).
Vocabulary trimmed to only meaningful keys. Kept: signal_type, source, pipeline,
trace.conventions, metric.{type,unit,temporality,metadata_quality,original_name}.
Dropped: metric.monotonic (a function of type), trace.has_events/has_links
(constant + derivable from columns), log.severity_scheme/body_format (constant /
derivable, and body_format cost an O(rows) scan), resource/scope lineage
(restates columns / collector-config concern), source_version (no cheap
non-constant value today). Prometheus carries type/unit in the metric name by
convention, so it gets identity only — no inferred enrichment.
Identity (signal_type + source) extended to the remaining ingest protocols so
the discovery view is complete: InfluxDB and OpenTSDB (metric), Loki and
Elasticsearch (log). These protocols carry no type/unit metadata, so identity is
all that applies.
Tests: unit coverage for the accumulator, per-metric-type fan-out, and trace
conventions; integration goldens updated for the OTLP metric/trace SHOW CREATE
output and the new Loki identity.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: validate the option value
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: Runner executes procedure
* feat: Add rollback key type to ParsedKey
* feat: Write rollback key when procedure is unable to execute
* feat: Use loaded step to re-submit subprocedure
* feat: Track subprocedures in ProcedureMeta
* feat: Clean message cache after the root procedure is done
* feat: Runner returns execution result
* fix: Fix tests
* test: Test Runner
* test: Test procedures_in_tree
* chore: Refine test and comments
* feat: Remove support of lock inheritance
A deadlock happens if a subprocedure acquires the same lock key as
its parent.
The main concern is if the subprocedure directly inherits its parent's
lock, then how should we behave when multiple subprocedures acquire
this same lock? Each procedure may assume it has unique access to the
same object but it actually shares the resource with others.
Now subprocedures need to use different keys to lock objects, which is
reasonable. For example:
- A parent procedure wants to create a table so it locks the table with
a key like `catalog.schema.table`
- Subprocedures create regions for the table so they lock the regions
with keys `catalog.schema.table.region-0 ~ catalog.schema.table.region-n`
* style: Fix clippy
* feat: insert_procedure returns false on duplicate procedure
Also rename this method to try_insert_procedure
* chore: Address CR comments
* docs: Add procedure framework RFC
* docs: Add dump, rollback and locking to procedure framework
* docs: Change ProcedureBuilder to ProcedureLoader
* docs: Add sub-procedures section
* docs: Add a link to explain idempotent
* docs: Add link to the tracking issue
* docs: Fix ProcedureLoader type alias
* docs: Update procedure API
* docs: Address CR comments
* docs: Update path and make the docs more clear
shuiyisong
dennis zhuang
discord9
jeremyhi
Sicong Hu
Ruihang Xia
yihong
Yuhan Wang
Yohan Wal
gcmutator
tison
Wei
Weny Xu
Zhenchi
zyy17
Lei, HUANG
Yingwen
LFC
Zhizhen He
elijah