mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-14 23:06:59 +00:00
discord9
577508b66b
* refactor(promql): report every duplicate point from overlap `overlap` returned only the first point where two members of a group are present and a caller-supplied predicate held, so the caller could not tell which other points had a duplicate. Turn the predicate into a callback and invoke it for every such point in ascending order; the caller decides what a duplicate point means. The one caller reproduces its previous behaviour exactly: it ignores points its gate excludes and keeps the smallest point per group, which is what the group fold already took. Signed-off-by: discord9 <discord9@163.com> * refactor(promql): number the binary runtimes of a tree `RuntimeNode::build` walks the compiled tree in pre-order, so a counter threaded alongside the input ids numbers each binary node deterministically: the same tree yields the same index on every partition, because every partition builds from the same compiled tree. That gives per-node execution state a key without depending on pointer identity. Signed-off-by: discord9 <discord9@163.com> * feat(promql): reduce duplicate findings across partitions A partitioned match cannot tell whether the other operand has a sample at a point, because each partition holds only its own slice. This adds the reducer that makes that decision possible without deciding anything yet: per binary node and field, it merges the operands' presence as partitions publish it and keeps the duplicate findings they report. Findings are retained regardless of what the gate says at the moment they arrive. A partition that publishes its presence late must not cause an earlier partition's finding to be discarded, so the gate is applied only when the merged presence is asked for a decision, with the same formula the kernel uses locally: `lhs && rhs`, or `lhs || rhs` where a fill value makes a point live with only one side. Presence and the retained findings are charged to the pool as they are added, by their real owned bytes, and a refusal is reported rather than swallowed. Storage is bounded by the grid length: one presence flag per point per operand and at most one finding per point, replaced only by one with a lower rank. Nothing constructs this yet. Signed-off-by: discord9 <discord9@163.com> * refactor(promql): carry duplicate evidence down to the kernel Plumbing only: `FusedSeriesExec` holds a plan-instance slot for the execution's duplicate evidence, `NativeExecutionState` and every `BinaryRuntime` carry a handle to it, and the three vector-match layers that reach `binary_field` accept it. Nothing reads or writes the evidence yet — every call site passes `None` or forwards what it was given — so the crate behaves exactly as before. The slot starts empty in `build` rather than in `execute`: the partitions of one execution are separate `execute` calls on the same plan instance, so a slot created per call would give each partition its own state, while a rebuilt plan is a different execution and gets a fresh empty slot from its own `build`. `BinaryRuntime` gains a `slot(field)` accessor for the index the compiler already assigns it; the next change is its first caller. Signed-off-by: discord9 <discord9@163.com> * feat(promql): publish what a field's match saw, without deciding yet `binary_field` now publishes each side's per-point presence into the execution's evidence and records every duplicate point `overlap` reports, including the ones the local gate excludes. Recording them is the point: a partition whose finding arrives before the other side's presence is known must still be able to fire once that presence does arrive, so the gate is applied when a decision is asked for, not when the finding is made. Each published field carries its node and field index, which is what keeps two binary nodes or two fields from sharing presence and findings. The message a recorded candidate carries comes from the same helper the immediate error uses, so a deferred report and a direct one say the same thing. Nothing decides anything yet: `evaluated` is still computed from the local presence vectors exactly as before, and `settled` has no caller outside the tests. `FieldEvidence` groups node, field and handle so the kernel signatures gained one parameter rather than three. Signed-off-by: discord9 <discord9@163.com> * feat(promql): report a duplicate match group found on another partition A partitioned match decides whether to check a duplicate group from the presence of the other operand, and a partition only holds its own slice of that operand. So the partition holding `job="other"` had no left-hand data, skipped every point, and answered where Prometheus refuses the query. The findings a partition makes are now decided once the whole execution has reported. Each partition publishes its operand presence and records every duplicate point it sees, including points its own gate excludes; the partition that finishes last holds complete evidence and asks for the earliest point that is eligible once all the presence is merged. Earlier partitions return their rows and their stream ends normally, so nothing waits on a partition that is never polled — an abandoned execution simply never reaches a verdict, and a failing one aborts its claim rather than being counted as having contributed. Presence is merged per binary node and field, so two nodes or two fields cannot share a verdict, and `filling` is recorded once per slot: a second publisher that disagrees is an error rather than a silent pick. The gate itself is unchanged, which is why a point whose other operand is genuinely absent still answers instead of erroring: the merged presence still says that point is not live. Signed-off-by: discord9 <discord9@163.com> * fix(promql): size duplicate evidence by the grid, not by partition count `DuplicateEvidence::new` stored the output partition count as its grid length. Those are unrelated numbers, and the caller passed `partition_count`: `publish` kept the operand presence for at most that many points and `record_candidate` returned success for anything past it. With two partitions and a five-point grid, a duplicate group at point four was discarded by the very check that exists to find it, and the query answered. Take the point count from the tree's evaluation grid and the partition count separately. Every claim now leaves the counter exactly once, whether it completes or aborts, and the exit that closes the last claim releases what the evidence holds: the verdict is kept only when the run was not aborted, then the slots are cleared and the reservation released. Previously the last completion shrank the reservation while the vectors and strings stayed allocated, and an aborted claim could leave another claim's exit unaccounted, so the charge outlived the run. Candidate storage becomes a sparse map keyed by point instead of a grid-length vector, charged before it grows: the group's strings, its vector capacity, the candidate itself and a per-entry estimate. A grid thousand points long with one duplicate no longer pays for a thousand empty slots, and the charge now covers the containers it claims to cover. The execution test evaluates a five-point range at parallelism two — more points than partitions — with the duplicate at the last point, which is exactly the boundary the partition-sized array dropped. Its control runs the same duplicate against a left side that is absent at every point and still answers. Signed-off-by: discord9 <discord9@163.com> * fix(promql): decide a partitioned duplicate from the whole execution's presence A partitioned match decided whether to check a duplicate match group from the presence of the other operand, and a partition only ever holds its own slice of that operand. `left{job="api"} / on(job) right` with two `job="other"` series on the right returned `5.0`, because the partition holding `other` had no left-hand data and its gate was false at every point, while Prometheus checks every group of the one side whenever the other side is non-empty at that point. Each binary evaluation now publishes both operands' presence and records every duplicate point it sees, including the points its own gate excludes, because a finding that arrives before the other side's presence is known has to be able to fire once that presence does arrive. The partition that finishes last reduces the evidence and asks for the earliest eligible point. Presence and findings are keyed by binary node and field: a tree can hold several binary nodes, and the kernel evaluates fields separately with presence read from that field's own operand. No partition waits on another. A slot takes its length from the operands it is published with rather than from the statement's grid, because a binary inside a subquery runs on that subquery's own grid, which can be longer. A partition that publishes a different length for the same slot is an error instead of a truncation. The evidence exists only when the node has more than one output partition and the tree holds a binary with two vector operands; otherwise the local gate already sees every operand and there is no verdict to reach. A claim for a partition this generation already issued, or one arriving after it finished, starts a new generation, so a second execution of the same plan does not join an accumulator whose claims were spent. A retained finding holds its match group behind an `Arc` with the side and timestamp, and the error text is built only when a verdict is reported, so recording a duplicate point costs no formatting. Signed-off-by: discord9 <discord9@163.com> * perf(promql): record a duplicate only where this partition cannot decide it A point whose merged-in-partition presence makes it evaluated already aborts the execution from the partition that sees it, so storing it in the shared evidence took the lock to record a verdict nobody would ask for. Only the points whose local gate is false are recorded now, which is exactly the set whose eligibility depends on presence this partition does not hold. Both readings are in the comment at the call site, because the previous round's test depended on recording every point and the next reader should see why that changed. Signed-off-by: discord9 <discord9@163.com> * fix(promql): decide an evidence generation from its state, not from call order `claim_evidence` inferred an execution boundary from "this partition has not claimed yet". Cancellation breaks that inference: an execution that claimed partition 0 and was dropped leaves an unfinished generation behind, and the next execution claiming partition 1 joins it; only when that same execution reaches partition 0 does a new generation start. The two partitions of one execution then sit in different accumulators, and a duplicate that only the merged presence can decide is missed. No two executions have to be alive at once for this. A generation is now dead when it has finished or when any of its claims was aborted, and a claim for a partition that a live generation already issued is an error that names the cause instead of silently starting a new one. The previous review allowed refusing re-execution outright, and this is that: any inference from call order is a guess that cancellation can invalidate. The consequence is stated where it is caused: a plan instance whose first execution claimed some partitions and never claimed the rest has an unfinished generation that no later claim can join. That is the price of refusing re-execution; a legitimate second execution would need an explicit execution lifetime rather than a heuristic over partition indices. Signed-off-by: discord9 <discord9@163.com> * fix(promql): keep duplicate evidence single execution Signed-off-by: discord9 <discord9@163.com> * test(promql): pin disjoint duplicate groups and trim unused ranking Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>