Return InvalidParameter for an invalid applied generic read_preference while preserving dedicated-header precedence. Format the query.parallelism diagnostic from the supplied maximum so wire errors do not incorrectly report the public cap.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Preserve per-query optimizer configuration while retaining expression-depth validation before and after extension analyzer rules.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* feat(mito): decouple compaction unit execution and publication
Execute shared-input dependency groups as independent local compaction units. Publish and acknowledge each unit separately, retain successful sibling results, and release each unit's input leases after worker apply.
Preserve picker priority, panic notifications, snapshot-free execution context, and indexed input leases. Include regression coverage for independent visibility, purge, cancellation, partial failure, and restart recovery.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): use crate visibility for compaction unit items
Replace the newly introduced pub(super) declarations with pub(crate), and make CompactingFiles::try_reserve private since it is only used within its module, per docs/style-guide.md.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): simplify compaction unit state and test fixtures
Keep local input leases in the task and derive running units from tracked and pending work. Replace the single-implementation task trait with an inherent async method and align exposed state type visibility.
Use snapshot-free region fixtures and named test scenarios, and cover independent input purging across three units with bounded parallelism.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): align compaction task and test module names
Rename CompactionTaskImpl to CompactionTask now that the CompactionTask
trait has been removed and the Impl suffix no longer marks a trait
implementation. Rename the state test module from lease_tests to tests
to follow the test module naming convention.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): align compaction task debug label
The Debug impl reported "TwcsCompactionTask" while the type is now
CompactionTask. Report the type name for consistency.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): label compaction elapsed metric per published unit
Record greptime_mito_compaction_total_elapsed once per published unit and
label it by type: a local dependency unit or a remote job. Update the
metric docs and group the Grafana compaction OPS/latency panels by type.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito): clarify compacting file lease sharing
`CompactingFiles::for_inputs` clones the shared per-file leases instead of
transferring them. Document that the original collection keeps its leases,
so an input is released only after every owner drops its lease.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito): document local compaction unit map key
Document that `LocalCompaction::units` is keyed by each unit's execution
plan ID, which matches `CompactionExecution::plan_id()` and the `plan_id`
carried by unit notifications.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): rename submit_prepared_compaction
The function submits remote jobs but only prepares local unit tasks; the
caller dispatches them after installing the returned phase. Rename it to
`prepare_or_submit_compaction` and document that ordering.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(hdfs): clean up temporary file when an atomic write fails
Signed-off-by: fengyue-xve <1759519137@qq.com>
* style: apply rustfmt to hdfs layer test
Signed-off-by: fengyue-xve <1759519137@qq.com>
* fix(hdfs): report earlier write failures and log cleanup failures
- Later `write`/`close` calls on a writer whose failed write was
cleaned up now mention the original failure instead of the bare
"unavailable after close or abort" message.
- `abort_and_delete` logs abort and deletion failures with the
temporary path instead of discarding them; the original write error
is still returned to the caller.
- The temporary-file cleanup test now lists the directory holding the
temporary file, so it fails if the cleanup is reverted.
Signed-off-by: 风月 <1759519137@qq.com>
Signed-off-by: fengyue-xve <1759519137@qq.com>
---------
Signed-off-by: fengyue-xve <1759519137@qq.com>
Signed-off-by: 风月 <1759519137@qq.com>
* fix: align fulltext bloom tokens with matches_term and use them for LIKE
The bloom fulltext index stored tokens that did not follow
matches_term's word boundaries (`_`, ASCII next to Han, Han substrings),
so pruning dropped rows matches_term accepts.
Tokens and query probes now share the character classes of
matches_term, Han runs are indexed as bigrams, and LIKE patterns use the
same probes. The bloom blob is written under a new key; blobs from older
versions are no longer probed. Tantivy no longer serves matches_term.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: drop unused lazy_static from index and avoid typo-flagged test tokens
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: link like_probes to tokenize instead of the downstream ScriptTokenizer
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: name probe extraction with verbs and document CharClass
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: rebuild legacy fulltext bloom indexes and probe lower(col)
The manifest records the fulltext bloom version per column, so ADMIN
BUILD_INDEX now rebuilds SSTs whose bloom index predates the current
tokens, and ssts_index_meta shows the version.
Case-insensitive bloom indexes also store the tokens of the whole
lowercased text for rows where it can differ (U+0130, U+212A, sigma),
so matches_term(lower(col), ..) and lower(col) LIKE .. use the index.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: cover legacy bloom index rewrite by compaction in the compat case
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: log outdated fulltext bloom indexes in the consistency check
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs: explain the unrecorded fulltext bloom version
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): centralize PromQL function registration in promql::functions
Move the PromQL scalar UDF and UDAF registrations out of the query plan
decoder into a single `promql::functions::register_promql_functions`
entry point, so plan decoding no longer has to enumerate every PromQL
function implementation. The registered function set, aliases (e.g.
`prom_holt_winters`) and registration order are unchanged.
Add a registration test pinning the full set of `prom_*` scalar
functions and the `prom_holt_winters` alias, plus a decode regression
test covering the centralized registration path.
Fixes#9493
Signed-off-by: fengyue-xve <1759519137@qq.com>
* refactor(promql): register Ieee UDFs centrally and address review
Rebase onto main picked up #9479, which registered the Ieee comparison
and sqrt UDFs from the decoder directly. Register them from
`register_promql_functions` instead so the single entry point stays
complete, and add the seven `prom_eq`..`prom_sqrt` names to the
registration test.
Also per review: drop the doc paragraph that explains why the function
was moved, drop the redundant `test_serializer_decode_centralized_promql_registration`
test and the registration comment in the decoder, move `use std::sync::Arc`
up with the imports, and simplify the registration test assertions.
Signed-off-by: 风月 <1759519137@qq.com>
Signed-off-by: fengyue-xve <1759519137@qq.com>
---------
Signed-off-by: fengyue-xve <1759519137@qq.com>
Signed-off-by: 风月 <1759519137@qq.com>
SET "query.parallelism" = 4 previously succeeded without taking effect:
ObjectName::to_string() keeps the quotes, so the name never matched the
known query options and fell through to the unknown-setting compatibility
branch, bypassing validation of invalid values such as 0.
Resolve the option name from the parsed identifier values
(object_name_to_string_unquoted) for both SET and SHOW so quoted and
unquoted spellings behave identically.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(log-store): keep the enqueued backlog on permanent object store errors
On object storage the WAL's object store sits behind OpenDAL's retry
layer, which marks every error it returns as persistent. The enqueued
mode classified create failures by that status, so a 403 or a missing
bucket was repeated every 100 ms on every create in flight, logged
nothing, and made appends hang once the sealed batches were full.
Object store errors are now classified by kind: denied access, an
invalid configuration (how OpenDAL reports a missing S3 bucket) and a
write to a missing location are non-retriable. In the enqueued mode such
an error makes the store failing instead of poisoning it: the
acknowledged backlog is kept under its sequences and bytes, its lowest
pending batch is attempted alone with a delay that doubles from 100 ms
to 30 s, appends are rejected at once with a retryable error that
carries the last object store error, and the first create that succeeds
ends the state. Stop uploads the backlog at once and reports a failure
as before. The state is logged on entry, at most once a minute while it
lasts and on exit, and exported as the `failing` gauge and the
`rejected_appends_total` counter. Conflicts, encoding and catalog errors
still poison the store; the durable mode is unchanged.
An append held back because the sealed batches are full is now counted
and timed by the stall metrics in both modes.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* docs(rfc): describe how the enqueued object store WAL handles refused creates
Describe the enqueued failure contract as the store now implements it:
non-retriable object store errors are told apart by kind because
OpenDAL's retry layer under the store marks every error persistent, a
failing store keeps its backlog and attempts it with backoff while it
rejects appends, and stop uploads the backlog at once. Add the failure
matrix row, the failing gauge and the rejected appends counter, the
sealed-batch admission limit and the stall metrics that now count it,
and the fault matrix case counts.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): time the failing attempts where they start and fix two statements
The test of the attempts of a failing store now records when each
refused write starts and checks the gaps between them, instead of
sampling a counter after a sleep of the test task. The delay comment and
the RFC describe 30 seconds as the longest delay between two attempts,
not as a bound on when appends resume, and the RFC states that the next
append is held once seven sealed batches wait.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): drop the wait_until helper that main now defines
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): name errors that need an operator and trim the RFC contract
Rename is_non_retriable to requires_intervention: such a create is still
repeated, only slowly, until someone fixes the credentials, configuration
or bucket. Name the backoff test after what it asserts, cut the test of
the lowest pending batch to two failed rounds since the cadence is
covered by the backoff test, and shorten the RFC failure contract to
the contract and its rationale, leaving error mappings, request counts
and logging cadence to the code comments.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* docs(log-store): tie the slow repeat to errors requiring intervention
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
The manage-region-follower chain submitted its procedures with an empty
executor context: `ProcedureServiceHandler::manage_region_follower`
accepted no query context, the operator built an
`ExecutorContext::default()`, and `MetaClient` dropped the context before
delegating to `RegionFollowerClient`. The submitting user (actor) and the
procedure event metadata therefore never reached the submitted
procedures.
- `ProcedureServiceHandler::manage_region_follower` now takes a
`QueryContextRef`, consistent with `migrate_region`, `gc_regions` and
`gc_table`.
- `ProcedureServiceOperator` derives the `ExecutorContext` (actor and
`TriggerReason::Manual` event input) from the query context.
- The `RegionFollowerClient` trait methods now accept `&ExecutorContext`
and `MetaClient` forwards it, so client implementations can transport
the actor and event context on the wire.
- `procedure_event_context`/`procedure_actor` are now public so client
implementations can reuse the trusted protocol derivation.
The previous review fix absorbed any SHOW VARIABLES whose remainder was not a
dotted identifier, which also swallowed `SHOW VARIABLES MAX_EXECUTION_TIME`
and broke the basic sqlness case on both standalone and distributed. The
filter keyword (WHERE/LIKE) boundary check already covers the old Connector/J
`loadServerVariables` empty-table behavior; a named variable must fall back to
the real handler. Revert to that and add tests for both.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(operator): route shared requests through NodeManager
Remove the concrete NodeClients requester API and use the existing node manager for owned and shared requests. Preserve the original write hint while the legacy error adapter remains in place.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(meta): preserve retry policy through error hints
Propagate client errors through External and let DDL read retry hints while preserving cleanup and terminal-child policies. Region transport failures retain the legacy Retryable classification, which now also participates in bounded write retries.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(operator): restore sparse partition tags when rebuilding writes
Decode retained primary keys when refreshed partition expressions need different tags. Keep the partitioner and multi-region sparse write regression tests with the fix.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(operator): stop detached write retries when callers cancel
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat(frontend): configure retries across distributed write paths
Wire the shared dispatcher through frontend and logical batching, preserving partition columns and request headers. Keep standalone retries disabled and include configuration, batching, and storage regression coverage.
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: support compaction options in HTTP hints
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor: share TWCS compaction option prefix
Use the store-api constant for both HTTP hint inference and Mito region option validation so the two paths cannot drift.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* perf(log-store): fetch object store WAL replay segments concurrently
A region read fetched each segment only after the previous object's
entries were yielded, so a replay paid the full request latency once per
object. Fetch up to READ_PREFETCH_DEPTH (8) segments ahead, each on its
own task, and handle their outcomes in object sequence order: entries,
collected-object skips, corrupted-segment holes and errors surface
exactly as before, and dropping the stream aborts the fetches in flight.
A read holds at most eight decoded segments besides the one it yields.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* perf(log-store): raise object store WAL recovery concurrency to 64
On S3, recovery reads the header and tail of every object, about 57 to
91 ms per object per lane on 4, 8 and 16 vCPUs alike, and its time
scales with objects divided by concurrency up to 128 lanes without
throttling: about 9,500 objects recovered in 99 s at 8 and 10.5 s at
64. At 64 recovery issues about 1,800 reads per second, a third of
S3's request rate per prefix, and holds one 64 KiB tail window per
lane, 4 MiB in all.
The recovery concurrency tests now scale with the constant.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* perf(log-store): share one prefetch byte budget among object store WAL reads
Each read fetched up to eight segments ahead on its own, so regions
opened in parallel (16 by default) could together hold 128 segments of
up to max_batch_bytes each. A read now always fetches the segment it
yields next, as before, and fetches a later one only while its length
fits in a 64 MiB budget shared by every read of the store. Regions
replayed in parallel thus hold one segment each plus at most 64 MiB,
however many open at once. Nothing waits on the budget: a read that
cannot reserve stops fetching ahead until its next poll.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): fail an object store WAL read whose fetch task panics
A fetch task that panicked was re-raised in the reader with
resume_unwind, which unwinds through the caller, such as Mito's region
opener task, past its cleanup. Return it as a WalSegmentFetch error
instead, so the read ends at that object like any other failed fetch.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): bound replay prefetch by decoded size held by fetch tasks
The prefetch reservation sat in the read's queue, so dropping a read
released it at once while an aborted fetch could still hold its
buffers until it next yielded. The fetch task now holds the reservation
and returns it with its outcome, so it is released with the buffers.
A reservation also counted only the encoded segment length, while a
fetched segment is held decoded. It now counts the length plus one
Record per entry, and the comment states the whole bound.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* docs(log-store): round the measured figures in the recovery concurrency comment
Keep the magnitudes in the comment; the exact S3 figures belong to the
measurement run rather than the code.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): abort replay fetch tasks with tokio-util's AbortOnDropHandle
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): rename the replay prefetch reservation to try_reserve_prefetch_bytes
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): share the bounded polling helper across the object store WAL tests
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
- federated: absorb SHOW VARIABLES with a WHERE/LIKE filter or a
non-identifier remainder as an empty table; only a bare or
dotted-identifier remainder falls back to the parser so named variables
like query.parallelism reach the real handler (restores Connector/J 5.1
loadServerVariables behavior).
- datanode: report engine_context query-option failures as
InvalidQueryContext instead of NewPlanDecoder.
- http: drop the redundant Option around the always-present timeout layer.
- session: split the query.parallelism 1024 cap to public entry points
(SQL SET, HTTP hints) and keep wire decoding at u16::MAX so a legacy
frontend's nonzero value such as 2048 still restores.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* feat: add per-column COMPRESSION option for SST columns
CREATE TABLE accepts COMPRESSION WITH (type = 'zstd', level = N,
page_size = '<size>') on a column. The options are stored in the column
metadata, carried through gRPC column options, shown by SHOW CREATE
TABLE, and applied by the mito SST writer and bulk part encoder as
parquet column properties. Unset fields keep the SST defaults. Nested
types are rejected.
A page size below the dictionary page limit also lowers the column's
dictionary page limit: parquet sizes write batches by the dictionary
budget and keeps that size after falling back to plain encoding, which
would otherwise ignore the smaller data page limit.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: validate column compression options before persistence
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: reject compression overrides on metric logical tables
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(mito): retain region compaction and DDL lifecycle state
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito): finalize opening batch in the worker on registration
Clean up the `opening_regions` entry and reply to its waiters in the same
serialized worker step that registers an opened region, instead of waiting
for the background opener to resume after the registration acknowledgement.
Previously the entry was only removed by the background opener, so a close
followed by a reopen could join the stale opening batch and receive a false
success after the region no longer existed. Replying from the worker closes
that window because registration and waiter cleanup are now atomic with
respect to the worker loop.
Add a test-only `EventListener::on_region_open_registered` hook and a
regression test that holds the first background opener to verify a reopen
does not wait for the previous open task. On failure the region is still
stopped and waiters receive `RegionClosed`.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito): unregister drop runtime state before WAL cleanup
The region leaves the region map before its WAL entries are obsoleted. A
failure in `wal.obsolete(...).await?` returned early and skipped
`cleanup_dropped_region_runtime_state`, so the resident compaction, flush,
and index scheduler entries were never unregistered. Since the region is no
longer in the map, a later close cannot clean them up either, leaking the
entries for the rest of the worker's lifetime.
Move the runtime-state cleanup to immediately after the region leaves the
map, before the fallible WAL cleanup. The schedulers only touch their own
state and do not depend on the region map or WAL, so the reordering is safe
and leaves the success path unchanged.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): tighten visibility of new compaction state items
Keep `ActiveCompaction::idle` private (it is only used inside `state.rs`)
and expose the other items added by the resident-region refactor as
`pub(crate)` instead of `pub(super)`, per the style guide. Also drop a
stale comment in `on_compaction_finished`: DDL handoff now happens in
`take_ready_ddls` before this block, which only runs when no DDL is
queued.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito): skip compaction scheduling while a region is truncating
`handle_flush_finished` dispatches queued DDLs before scheduling
compaction. A truncate queued behind a committing flush marks the region
as `Truncating` synchronously, but `schedule_compaction` only skipped
staging states, so it could schedule a new automatic compaction while the
truncate was still running.
Add `MitoRegion::is_truncating` and skip compaction scheduling for a
truncating region. Add a regression test that queues a full truncate
behind a committing flush and asserts no compaction is scheduled.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito): log DDL-fenced compaction rejection
The refactor dropped the log emitted when a region was skipped because of
pending DDL requests. Automatic triggers carry no waiter, so the rejection
became silent and could not be told apart from a scheduling interval that
had not elapsed. Log the queued and executing DDL state before cancelling.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito): document DDL reply fence invariants
Explain why fence_ddl_replies installs executing_ddl before handing off
the batch and why CompactionDdlComplete carries both a scheduler-wide
generation and a per-slot request_id. The generation fences replies from
superseded batches or region incarnations, while request_id makes
completion idempotent so only the last distinct reply releases the fence.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito): rename DDL completion request_id to request_index
The value is the DDL's index within the handed-off batch, not a globally
stable request identifier; request_id suggested otherwise. Rename it to
request_index across the scheduler, worker dispatch, and tests.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
- Move the historical filter_null_join_keys=true default from the dist
plan analyzer into session state initialization so an explicit
datafusion.optimizer.filter_null_join_keys=false override now reaches
both the pre-MergeScan and main optimizer passes.
- Keep non-option SHOW VARIABLES filter forms (WHERE, LIKE split across
lines, unquoted LIKE) on the federated compatibility result instead of
rejecting them, while named query options still reach the query handler.
- Accept the legacy query_parallelism=0 hint from older Flownode clients
as an unset request instead of failing the whole Flight/gRPC/HTTP query.
- Mirror the canonical query.parallelism override as the legacy
query_parallelism key in per-query snapshots so older Datanodes keep
honoring the requested parallelism during rolling upgrades.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(mito2): cover regex inverted index pruning
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* build: keep build inputs checkout-relative for cross-worktree cache reuse
Two small changes so compilations are byte-identical across worktrees
of the same commit and can share a build cache (mbx, sccache):
- src/common/version/build.rs: derive the workspace root relative to
CARGO_MANIFEST_DIR instead of the CARGO_WORKSPACE_DIR injected by
.cargo/config.toml (an absolute path). The env var remains as an
explicit override for the submodule case; the [env] injection in
.cargo/config.toml is removed so the relative derivation is used by
default.
- Replace three env!("CARGO_MANIFEST_DIR") uses in test/bench code
(servers prom_decode bench + decode test, common-meta test_util)
with a runtime std::env::var lookup, so the compiled artifact no
longer embeds the checkout path.
No behavior change: same files are read, same version info emitted.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* build: address cross-worktree cache review feedback
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(log-store): drive recovery, read, delete and stall failures through the fault matrix
Add the second half of the object store WAL fault matrix: the failures of
the object store access and of the store state that the conditional-create
rows did not drive. Each case asserts the failed waiters and their error,
whether the store is poisoned, the sequence, both indexed gauges at the
failure, the exact movement of every counter and what a read sees after.
Recovery: a listing, a trailer fetch and a footer fetch that fail, the
catalog rejecting an object after every footer was read, and an object
gone by the time it is fetched, below the boundary of the tip (ignored)
and at it (construction fails); each leaves the gauges and the recovery
metrics untouched. Reads: a segment fetch that fails, without a hole or a
read timer sample, and a later read of the region that succeeds.
Collection: a delete that fails and is attempted again at the next
collection, and a stop that waits for a delete in flight, with the catalog
agreeing with the listing in both outcomes. Store state: per-region entry
id exhaustion refusing the append without touching the batch it sealed,
and the durable stop paths with several creates in flight that succeed,
fail transiently or conflict after stop began. Stalled appends released by
admission, by a stop, by a terminal error and by the teardown of the actor,
the first two timed as a pair. The recovery and read timers observed around
parked fetches, twice each, with holds that cannot overlap.
The cases use the existing doubles, extended with a failing listing and a
failing range read at an offset on RecordingIo and parked range reads on
ParkedIo. Case::row is now optional, and the assertion that every row of
the failure matrix has a case is unchanged.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): lose a wide object at its header and footer reads in the fault matrix
The vanished-object case now runs three ways: a short object read whole,
and a wide object deleted before its header and tail window reads or
before its separate footer read, each below and at the boundary of the
tip. The entry id exhaustion case reads the other region of the sealed
batch back, the teardown case asserts the indexed gauges before and after
the actor is gone, and two comments that restated their names are gone.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): prove the stop-path ordering and share the wide-footer fixture in the fault matrix
The case of creates that succeed after stop began keeps object 1 parked
while objects 4 to 2 complete, round-trips a command through the actor and
asserts that no append is acknowledged and nothing is indexed before
object 1 is released. The stop-with-delete case replaces its yields with
the same command barrier, queued behind the stop command. The wide-footer
fixture moves to the tests module and is shared with the recovery test
that fetches a footer longer than the tail window. The gauge helper is
named for its side effect and the vanished-object stage enum for what it
names.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): keep the four-create scenario doc on its fault matrix case
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat(jaeger): support v3 HTTP query APIs
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(jaeger): preserve full traces with v3 time hints
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs(jaeger): clarify validation of v3 lookup hints
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(jaeger): bound v3 searches and preserve v1 binary attributes
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(jaeger): match resource attributes in v0 tag filters
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): follow IEEE 754 float semantics in operators and aggregations
- Comparisons: Arrow orders floats totally (NaN == NaN, NaN > +Inf); PromQL
comparisons with NaN are false except !=.
- min/max: skip NaN unless the whole group is NaN. DataFusion's grouped
min/max let NaN replace and be replaced by neighbours, so the result
depended on input order.
- topk/bottomk rank NaN last.
- round() rounds ties up and returns NaN for a zero step, like Prometheus.
- sqrt() of a negative number is NaN instead of an error.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): round() without a step rounds to integers
The planner passed a zero step for single-argument round(), which now
yields NaN like Prometheus; pass 1 instead. Found by the compatibility harness.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): apply the sqrt guard to builtin functions and register min/max for plan decoding
sqrt resolves through the session's builtin functions, not the UDF branch.
Distributed plans are decoded with the serializer's session, which did not
know prom_min/prom_max.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover NaN and rounding semantics
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): keep the nested count rewrite for PromQL min/max
count(min(x) by (l)) was rewritten by count_nest_aggr only for the builtin
min/max names. prom_min/prom_max return NULL for the same groups, so the
rewrite stays valid. Update plan snapshots for the IEEE comparison, sqrt,
topk and min/max changes.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): register prom_min/prom_max in the function registry
Distributed plans split an aggregation into state and merge steps only for
functions whose state function exists in FUNCTION_REGISTRY; prom_min and
prom_max lived in the promql crate, so min/max stopped being pushed down to
datanodes. Move them to common-function and register them before the state
functions are derived.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: update sqlness results for PromQL float semantics
Plans and column names now show prom_min/prom_max, the NaN guards of
comparisons and sqrt, and the NaN sort key of topk. round() results follow
Prometheus' formula. Distributed min/max keep the state/merge split.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): keep column names and NULLs in IEEE comparisons and sqrt
Rewriting comparisons as `a op b AND NOT (isnan(a) OR isnan(b))` changed
TQL output column names and turned NULL operands into true for `!=` and
false for the bool forms. Implement comparisons and sqrt as PromQL UDFs
that propagate NULL and report the column name of the expression they
replace; prom_min/prom_max report min/max column names the same way.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: restore column names in sqlness results for PromQL float semantics
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): share the prom_min/prom_max groups accumulator with tests
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): keep distinct names for prom_min/prom_max
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): default a stepless subquery to a one-minute step
`x[5m:]` evaluated the inner expression at the query step, which is 1s
for instant queries. Prometheus uses the evaluation interval, one minute
by default.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover the default subquery step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover the default subquery step in a range query
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): return range-vector results for instant queries
An instant query of a range selector or subquery (e.g. `up[5m]`) returned
the internal range arrays, which panicked when the frontend decoded them as
dictionaries. Plan such queries as the selected samples with their own
timestamps, and reject range-vector expressions in range queries like
Prometheus does.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): keep encode_substrait on a valid range query
A range query of a range-vector expression now fails like in Prometheus.
Wrap the subquery in max_over_time so the case still encodes both
RangeManipulate nodes through substrait.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): plan wrapped range-vector roots and reject a zero subquery step
TQL EXPLAIN, ANALYZE and AS alias wrap the root in an extension, which the
range-vector path did not unwrap. A zero subquery step divided by zero.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover wrapped range-vector roots and a zero subquery step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): default subquery step and reject range vectors in range queries
A root subquery without a step uses the one-minute default instead of
the query step. The HTTP range query endpoint rejects range-vector
expressions even when start equals end.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: cover root subquery default step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): align root subqueries with subquery_child_window
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): return millisecond timestamps for range-vector results
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): keep the time index timezone and name its table in errors
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
discord9
Lei, HUANG
fengyue-xve
Weny Xu
dennis zhuang
Yingwen
sun
jeremyhi
discord9
Ning Sun
liyang
shuiyisong
fys