rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-10-28 22:56:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,296 Commits 380 Branches 282 Tags
badceec12a0e98fd69e83e9b9b9b06d9358b78dd
Commit Graph
51 Commits
Author SHA1 Message Date
+1
Weny Xufengyue-xvedennis zhuangLei, HUANGfyslocalhostjeremyhiDeviousCardiAumYingwendiscord9
 		badceec12a fix: backport selected fixes to v1.3 (#10006) 
* fix(hdfs): clean up temporary file when an atomic write fails (#9483)

* fix(hdfs): clean up temporary file when an atomic write fails

Signed-off-by: fengyue-xve <1759519137@qq.com>

* style: apply rustfmt to hdfs layer test

Signed-off-by: fengyue-xve <1759519137@qq.com>

* fix(hdfs): report earlier write failures and log cleanup failures

- Later `write`/`close` calls on a writer whose failed write was
  cleaned up now mention the original failure instead of the bare
  "unavailable after close or abort" message.
- `abort_and_delete` logs abort and deletion failures with the
  temporary path instead of discarding them; the original write error
  is still returned to the caller.
- The temporary-file cleanup test now lists the directory holding the
  temporary file, so it fails if the cleanup is reverted.

Signed-off-by: 风月 <1759519137@qq.com>
Signed-off-by: fengyue-xve <1759519137@qq.com>

---------

Signed-off-by: fengyue-xve <1759519137@qq.com>
Signed-off-by: 风月 <1759519137@qq.com>
(cherry picked from commit a35661d69c)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(jaeger): return the most recent traces when a search hits its limit (#9595)

* fix(jaeger): return the most recent traces when a search hits its limit

Trace search applied LIMIT to an unordered DISTINCT trace_id, so a search
matching more traces than the limit returned an arbitrary subset that
changed between identical requests. Order traces by their latest matching
span before applying the limit.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor(jaeger): name the trace dataframe builder with a verb and document execution

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit cf16c7313b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(pprof): initialize framehop before signal sampling (#9612)

fix(pprof): pin upstream framehop initialization fix

Pin pprof workspace-wide to 01cff82d containing tikv/pprof-rs#282. Initialize framehop before sampling instead of allocating and discovering shared libraries in the first SIGPROF handler. Regenerate the lockfile without unrelated dependency changes.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit 0dc81777e3)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump rustls to 0.23.46 (#9614)

rustls 0.23.46 fixes rustls/rustls#3301: on TLS 1.2 with AES-GCM, reaching the 2^24 record limit dropped a fragment while reporting the write as complete, then kept sending records after close_notify. This stalled Kafka WAL produce requests until the request timeout.

(cherry picked from commit 33879b07c8)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(prom): return empty remote read results for missing metrics (#9613)

* fix: return empty remote read results for missing metrics

Signed-off-by: fys <fengys1996@gmail.com>

* test(prom): cover empty reads for missing databases

Signed-off-by: fys <fengys1996@gmail.com>

* test(prom): simplify remote read regression cases

Signed-off-by: fys <fengys1996@gmail.com>

* fix(prom): log missing tables during remote read

Signed-off-by: fys <fengys1996@gmail.com>

---------

Signed-off-by: fys <fengys1996@gmail.com>
(cherry picked from commit cf0f6c308a)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): compare release validation against the previous stable version (#9636)

Ancestry-based tag selection skips stable tags that live only on older release branches.

Signed-off-by: paomian <xpaomian@gmail.com>
(cherry picked from commit 3de8610e2e)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): keep processing bulk requests after one fails to encode (#9638)

* fix(mito2): keep processing bulk requests after one fails to encode

When `push_bulk` fails to build the WAL entry of a bulk request, it answers
that request with the error. `process_bulk_requests` then returned from the
loop, so the remaining requests of the drained batch were dropped without an
answer and their callers only saw a receive error. Skip the failed request
and continue with the rest of the batch instead.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(mito2): keep bulk install barriers independent per target

The bulk install test barrier was a single process-wide slot, so tests
that arm it while running in the same process replaced each other's
barrier. Keep one barrier per armed target instead.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 88bdab0f66)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): honor @ modifier on subqueries (#9446)

* fix(promql): honor @ modifier on subqueries

`prom_subquery_expr_to_plan` never read the `at` field of a subquery, so
`sum_over_time(m[20s:10s] @ 60)` was evaluated like `m[20s:10s]` and
followed the outer evaluation grid.

Handle `@` the way Prometheus does: fold the anchor into the subquery
offset (`offset + (eval_start - anchor)`, as `setOffsetForAtModifier`
does), evaluate the subquery once at the start of the evaluation, and
replay the folded window over the outer grid, reusing the anchored
range selector machinery. A call whose range argument is an anchored
subquery is promoted as step-invariant, like one over an anchored range
selector. `@ start()`/`@ end()` and `offset` combine as for selectors,
and an unrepresentable anchor is rejected with the existing error.

Closes #9411

Signed-off-by: Aarav <aaravsjadav@gmail.com>

* test(promql): add sqlness cases for @ on subqueries

Cover the #9411 reproducer end to end: `@ <ts>`, `@ start()`/`@ end()`,
`@` combined with `offset` in either order, an instant query away from
the anchor, `*_over_time` functions and `rate` over an anchored
subquery, a computed inner expression, and an aggregation above the
anchored call. Two series per table so a dropped or merged series
shows up as a wrong row.

Refs #9411

Signed-off-by: Aarav <aaravsjadav@gmail.com>

---------

Signed-off-by: Aarav <aaravsjadav@gmail.com>
(cherry picked from commit 095c288f16)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): validate bulk insert batches against the region schema (#9698)

* fix(mito2): validate bulk insert batches against the region schema

Bulk inserts were only checked for the time index column, unlike row
writes, which WriteRequest::check_schema checks against the region
schema. A batch with columns of another type, nulls in the time index or
in non-nullable columns, duplicate or unknown columns reached the region
worker: a time index that is not a timestamp or holds only nulls made
the worker fail, and other mismatches failed after the WAL append or
were stored as written, depending on the SST format.

Move the column checks of check_schema into a core shared by row writes
and bulk inserts, and check bulk insert batches at the same points as
rows: when the engine builds the worker request, and again in the worker
if the region schema changed since. check_schema keeps its behavior.
Bulk insert columns must have the column's Arrow type, except that tags
may use any string encoding, as the memtables already convert them.
Sparse batches must carry one encoded primary key column of binary
without nulls. A batch that misses columns gets them filled by the
worker even if it claims to match the current schema.

The metric engine checks logical bulk inserts against the logical region
before rewriting them into the physical region, as the rewrite copies
columns without checking them and would encode a duplicated tag into the
primary key twice.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(mito2): report an error when a write is interrupted by a panic

WriteNotify answers its waiter when it is dropped and reported success
unless an error had been set, also when it was dropped while unwinding
from a panic in the middle of the write. Report an error in that case.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(mito2): tolerate a poisoned series lock when dropping a series map

SeriesMap::drop reads every series to update the memtable metrics and
unwrapped the lock, so dropping the map while unwinding from a panic
that poisoned a series lock panicked again. Read through the poisoned
lock instead.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 791f429246)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): fence writes before a graceful downgrade (#9716)

* test(mito2): cover graceful role changes racing with writes

A graceful role change flips the region state in a background task while
the worker keeps handling writes. Add bulk and row write tests that check a
write handled after the role change request is either rejected or covered
by the returned last_entry_id.

Add tests for DowngradingLeader while an ALTER keeps the region in
Altering, alone and in the downgrade sequence the datanode runs
(DowngradingLeader, Flush, Follower) with a migration candidate on its own
local WAL. Add a test that the downgrade sequence covers a write in flight
when a lease renewal sets Downgrading.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(mito2): fence writes before a graceful downgrade

set_region_role_state_gracefully hands the role change to a background task
that waits for the manifest lock. The worker keeps handling writes until the
task flips the state, so a write could be acknowledged with an entry id above
the returned last_entry_id. Set Downgrading in the worker before spawning the
task for Follower and DowngradingLeader requests, as lease renewal already
does, so every write the worker handles afterwards is rejected.

DowngradingLeader also did nothing and returned success when the region was
in another leader state such as Altering. The region became writable again
when the operation finished, and the datanode then flushed and turned it
into a follower in one step. Return a retryable error instead, so metasrv
retries the downgrade.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(mito2): assert the retryable downgrade error during ALTER

Check that DowngradingLeader during an ALTER returns a retryable error, and
release the paused ALTER on that path. The local WAL test now retries the
downgrade after the ALTER, as metasrv does.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(mito2): write before the retried downgrade in the local WAL test

Rows written before an ALTER were already flushed, so the candidate check
did not depend on the downgrade's final flush. Write rows after the ALTER
and before the downgrade is retried, which only that flush can carry to the
candidate.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 660c4efe1f)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): validate persisted partition bounds before altering regions (#9487)

* fix(meta): validate persisted partition bounds before altering regions

A partition bound is persisted with the type of the column at the time
it was written. When an ALTER TABLE changes the type of a column
referenced by a partition expression (widening the time index unit is
the common case), each region rewrites the bound literal to the new
type while building its scan predicate. A millisecond bound like year
3000 fits milliseconds but overflows nanoseconds, so every scan of the
region fails once the table schema is updated -- and some regions may
already be altered when the failure surfaces.

Validate every region's persisted expression on prepare, before any
region is modified: parse the persisted JSON (mirrored types, since
common-meta cannot depend on the partition crate which depends on it),
replay the same value -> ScalarValue conversion and checked cast the
region performs, and reject the ALTER naming the offending region when
a bound cannot be represented with the new column type. Incompatible
changes are rejected atomically; compatible widening proceeds unchanged.

Adds meta-level tests (multi-region rejection naming the second region,
compatible widening completing) and an end-to-end sqlness case.

Fixes #9454

Signed-off-by: fengyue-xve <1759519137@qq.com>

* fix: access ColumnSchema fields directly and use plain format args

ColumnSchema exposes name and data_type as public fields, not methods,
and the telemetry macros only accept a format string after the error.

Signed-off-by: fengyue-xve <1759519137@qq.com>

* test: access ColumnSchema data_type field in alter table tests

Same fix as the previous commit: data_type is a public field on
ColumnSchema, not a method.

Signed-off-by: fengyue-xve <1759519137@qq.com>

* test: drop borrows now that data_type is read as a field

Signed-off-by: fengyue-xve <1759519137@qq.com>

* test: drop useless format! in nested expression test

Signed-off-by: fengyue-xve <1759519137@qq.com>

* test: cover the partition tail in change_col_type sqlness case

Signed-off-by: fengyue-xve <1759519137@qq.com>

* test: drop spurious blank line in change_col_type result

Signed-off-by: fengyue-xve <1759519137@qq.com>

* fix(meta): compare partition bounds with the scan comparison coercion

The preflight validator cast the persisted bound directly to the altered
column type, which rejected ALTERs the scan itself accepts: the region
coerces both the column and the bound to the common comparison type
(ExprSimplifier::coerce in PredicateGroup::new), so narrowing a field
column with an out-of-range bound still scans. Compare through
comparison_coercion(target_type, bound_type) instead and only require the
bound to be castable to the common type.

Also add a round-trip test in the partition crate that serializes real
PartitionExprs and runs them through the validator, so drift between the
Persisted* mirror and the partition types fails CI, and fix the error
message to match the comparison semantics.

Signed-off-by: fengyue-xve <1759519137@qq.com>

* test: update the sqlness fixture for the new incompatibility message

The error message now reflects the comparison semantics: the persisted
bound can no longer be compared with the new column type.

Signed-off-by: fengyue-xve <1759519137@qq.com>

* build(deps): refresh Cargo.lock for the common-meta datafusion-expr dependency

common-meta gained a datafusion-expr dependency in ff9c3f4 but the lock
file was not refreshed, so cargo check --locked refused to run on CI.

Signed-off-by: fengyue-xve <1759519137@qq.com>

* fix(meta): reject every non-widening type change of a partitioned column

The region routes rows on write through `MultiDimPartitionRule::find_region`,
which compares the persisted bound against row values directly: `Value::cmp`
panics on mismatched variants and equality misroutes silently. After a column
partitioned at `v < 3000000000` is altered from BIGINT to INT, every INSERT
compares `Value::Int32` rows against the persisted `Value::Int64` bound and
panics in the frontend, even though the scan-side coercion this check mirrors
still works.

Reject any type change of a column referenced by a persisted partition
expression unless it is a timestamp unit widening, which keeps the bound and
the rows comparable because `Timestamp` comparisons convert units first; the
widening keeps the scan cast check unchanged.

Signed-off-by: fengyue-xve <1759519137@qq.com>

---------

Signed-off-by: fengyue-xve <1759519137@qq.com>
(cherry picked from commit 1e29e8d4b6)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(partition): route NULLs like row inserts and accept signed-minimum bounds (#9697)

* fix(partition): route NULLs like row inserts and accept signed-minimum bounds

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(partition): keep equality two-valued for NULL rows and accept unsigned -0

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(partition): treat NULL masks as false in batch routing and keep comparisons bare

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(compat): update partition filter in analyze verbose expected output

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit da88ff8653)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): re-check a filled write as sent after a schema change (#9701)

* fix(mito2): re-check a filled write as sent after a schema change

The engine fills the columns a write leaves out with their defaults when it
accepts the write, appending them to the rows without adding them to
`name_to_index` and `has_null`. If the region schema changes before the
worker handles the write, the worker checks the request again and
`check_schema` looks up a filled column in `name_to_index`, which panics and
stops the region worker. If the filled column was dropped, the write fails
with an unknown column that the client never sent.

Remove the filled columns before checking, so the request is checked as sent
and filled again for the current schema. Nothing is filled yet when the
engine checks the request, so that check does no extra work.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(mito2): rename put_request helper to build_put_request

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 93c3b116da)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(mito2): gather last rows with one take per input batch (#9737)

* perf(mito2): gather last rows with one take per input batch

FlatLastTimestampSelector buffered a single-row slice per series and
concatenated up to 8192 of them. Concatenating slices with dictionary columns
merges the dictionaries once per slice, which made high-cardinality instant
PromQL queries using the LastRow scan path (#9034) slower. Select the rows of
series that end inside a batch with one take instead.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(mito2): cover duplicate last timestamps in the take path

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(mito2): group last rows by primary key bytes

Flat batches may assign different dictionary keys to the same primary key, for
example when binary keys are wrapped with identity keys. Merge adjacent ranges
with equal key bytes before selecting last rows.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 4645ab9ed7)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): apply SST internal column encodings to bulk-encoded parts (#9735)

* fix(mito2): apply SST internal column encodings to bulk-encoded parts

Bulk memtable parts are encoded to parquet and flushed as SSTs as-is, but the
bulk encoder did not apply the time index / __sequence DELTA_BINARY_PACKED
encoding (with dictionary disabled) and the uncompressed op_type setting that
the regular SST writer uses. Share these settings through one helper.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore(mito2): drop the answered bulk encoder compression TODO

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 977e76d0a4)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: avoid exact scan row counts for SSTs with unknown counts (#9759)

fix: avoid exact scan row counts for legacy SSTs

Treat selected SSTs with unknown row counts as non-exact before checking region partition visibility. Preserve exact statistics for known counts and cover legacy metadata mixed with known-count files.

Fixes GreptimeTeam/greptimedb#9718

Signed-off-by: Aumnertic <hznuzc@gmail.com>
(cherry picked from commit 6f53d57d15)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: avoid duplicate keys in update flow metadata txn (#9816)

CREATE OR REPLACE FLOW deleted and then put the same flow route,
flownode flow and table flow keys in a single txn when the replacement
kept the partitions, flownodes or source tables, which etcd rejects
with "duplicate key given in txn request".

Only delete the old keys that are not put again. Closes #9810.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit a1b2362e58)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): increase release integration test stack size (#9817)

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit e6fa1a0b21)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(datanode): release region map guard before awaiting role changes (#9824)

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 0dc649e028)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): align nightly Windows and macOS artifact test stack sizes (#9860)

* fix(ci): increase Windows nightly test stack size

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(ci): align optional macOS artifact test stack size

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 8b1d68369b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): keep SSTs when a manifest save ends with RateLimited or PermissionDenied (#9898)

* fix(mito2): keep SSTs when a manifest save ends with RateLimited

The object store's retry layer retries temporary errors and returns the
last attempt's error. A manifest delta PUT can be stored by the server
while the client sees a timeout, reset or 5xx, and the retries that
follow can be throttled, so the save ends with RateLimited even though
the delta exists. may_have_persisted_manifest_update treated RateLimited
as proof that nothing was written, so flush and compaction deleted the
SSTs the stored delta references. A later open or follower sync replays
the delta and reads fail on the missing files.

Treat RateLimited like other ambiguous errors: the SSTs are kept and
become orphans when the save really failed.

Fixes #9897

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(mito2): keep SSTs when a manifest save ends with PermissionDenied

The retry layer returns the last attempt's error, so a 403 from a revoked
key, a changed bucket policy or clock skew can follow a delta PUT that
the server already stored. Like RateLimited, PermissionDenied does not
prove that nothing was written, so it no longer lets flush or compaction
delete the SSTs; they become orphans when the save really failed.

The compaction test that rejects a manifest write before persistence now
injects Unsupported, which is raised before any request is sent.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
(cherry picked from commit 1cb01d0d44)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: upload staging manifests in chunks (#9919)

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit f2be11a69e)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf: stream and prune SST manifest scans (#9931)

* refactor: add reusable metadata pruning and projected SST batches

Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(mito2): stream and prune manifest SST metadata

Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(datanode): stream inspection queries through reusable providers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(catalog): push manifest filters through inspection queries

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: align manifest column constants with module conventions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf: reduce manifest inspection allocations

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: narrow manifest inspection API visibility

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: retain residual filters for manifest inspection

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit c96cdd0026)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: improve SQL analyze scan diagnostics (#9565)

* feat: improve SQL analyze scan diagnostics

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: redact dynamic filters in sqlness results

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: stabilize analyze snapshots for merge queue

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: remove obsolete scan_details assertion

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: refresh explain snapshots after partition filter changes

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 3e2751d71d)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): avoid lossy casts in partition pruning (#9600)

* fix(query): keep lossy casts out of partition pruning

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): align cast pruning expectations with conservative fallback

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): record lossy cast partition pruning regression

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): retain safe pruning conjuncts beside unsupported casts

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): refresh cast pruning plans after rebase

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit df8f6848cc)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): preserve IGNORE NULLS semantics in distributed queries (#9601)

* fix(query): skip LastRow pushdown for IGNORE NULLS

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): keep IGNORE NULLS aggregates above remote scans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): record IGNORE NULLS results before and after flush

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): share IGNORE NULLS coverage across deployments

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): refresh last-value plans after rebase

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 0dab3aef21)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): share peer discovery with plugin hooks (#9977)

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 9f0e58e2af)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: adapt release backport test dependencies

Copy the writer observation test utility from #9505 (18726c7602) for the #9919 staging regression, initialize the release-only catchup location hint, and remove main-only unused imports.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump version to 1.3.0

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): pin DataFusion cast constraint propagation fix

Update the pinned GreptimeTeam/datafusion revision from e3221b8 to
cc55344 (the 55.1 fork carrying apache/datafusion#25531 plus a stronger
regression test).

CastExpr::propagate_constraints used to cast a filter's output interval
straight back to the input type. For CAST(k AS BIGINT) = 1 the singleton
output [1, 1] was inverted to [1.0, 1.0] although k = 1.5 matches too, so
FilterExec advertised k as constant, the equivalence classes dropped the
ordering required by ORDER BY k, and rows returned in scan order.

The sqlness case keeps every #9600 assertion and the explicit ORDER BY,
and adds ascending and descending checks for the partitioned table, the
control table and a table whose matching rows are stored in neither
ascending nor descending k order.

Cargo.lock only rewrites the 35 datafusion source lines.

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit fe637b04ce)
Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: fengyue-xve <1759519137@qq.com>
Signed-off-by: 风月 <1759519137@qq.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: fys <fengys1996@gmail.com>
Signed-off-by: paomian <xpaomian@gmail.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: Aarav <aaravsjadav@gmail.com>
Signed-off-by: Aumnertic <hznuzc@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: fengyue-xve <1759519137@qq.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com>
Co-authored-by: localhost <xpaomian@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: DeviousCardi <115358213+DeviousCardi@users.noreply.github.com>
Co-authored-by: Aum <hznuzc@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: discord9 <discord9@163.com>
 2026-10-28 19:04:51 +08:00
LFCanddiscord9 56c5978647 fix(query): stop windowed sort on precision-lowering time casts [Backport release/v1.3] (#10013) 
fix(query): stop windowed sort on precision-lowering time casts (#9524)

* test: cover windowed sort cast unit mismatch



* fix(query): stop windowed sort on unit- or timezone-changing time casts

A cast on the time index column keeps the windowed-sort optimization
eligible only when it preserves the underlying timestamp values. Casts
that change the time unit (e.g. ms -> s) or that add a timezone to a
naive timestamp re-interpret the values, so the working ranges derived
from the time index no longer describe the sort column and the scan
fails with a working-range error or silently wrong order. Reject both
cast shapes so such queries fall back to the normal SortExec path.

The integration fixture uses two rows 1000 seconds apart so the cast
output stays distinct and the ordering assertions are not vacuous; the
descending arm gets its own expectation.



* fix(query): stop windowed sort on unit-changing timestamp wrappers

to_timestamp_seconds/millis/... cast their argument to their own unit
inside the function, so the hidden cast bypassed the explicit-cast guard
and kept the windowed-sort optimization eligible on a sort column whose
values no longer match the working ranges. The wrapper branch now applies
the same unit- and value-preserving predicate as the cast branch, with
the predicate extracted into preserves_timestamp_values and shared by
both.

The integration regression gains to_timestamp_seconds ASC/DESC arms and
a to_timestamp_millis value-preserving control; unit tests cover
unit-changing, tz-adding, and non-timestamp wrapper inputs.



* test: regen windowed sort sqlness after wrapper guard

The windowed_sort_nanos and windowed_sort_advance cases sort by
to_timestamp_nanos over a millisecond index and to_timestamp_millis over
a nanosecond index. Both wrappers change the time unit inside the
function, so the windowed-sort rule no longer applies and the plans
render a plain SortExec. Query results are unchanged; only the EXPLAIN
plan line differs. Regenerated by the sqlness runner.



* test(query): simplify windowed sort cast regressions



---------


(cherry picked from commit 984f80e277)

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
 2026-10-27 22:12:04 +08:00
LFCanddennis zhuang 7a20d38702 fix(query): forward scalar subquery filters to region scans [Backport release/v1.3] (#9781) 
fix(query): forward scalar subquery filters to region scans (#9738)

* fix(query): forward scalar subquery filters to region scans

Since DataFusion 55, uncorrelated scalar subqueries run in ScalarSubqueryExec
instead of being rewritten into joins, so filters such as
ts = (SELECT ts ... ORDER BY ts DESC LIMIT 1) no longer reach region scans
through a join dynamic filter. MergeScanExec now captures such filters as
remote dynamic filters, keeps them in the parent FilterExec, and replaces the
subquery with its result before querying regions.



* fix(query): check the remote dyn filter budget across pushdown phases

Filters captured in the Pre and Post phases are sent in one request, but each
phase validated the initial registration budget alone. When their sum exceeded
the budget, serialization dropped every registration, including join and TopK
filters. Validate the combined set and keep filters accepted by the other phase.



* fix(query): keep IN-bearing scalar subquery filters at the parent

Oversized remote dyn filter payloads replace IN lists with true. That only
widens a conjunction of bounds, but inside an arbitrary predicate such as
(host IN (...)) = (SELECT flag ...) it can drop rows the parent filter would
keep. Do not forward scalar subquery filters that contain IN lists.



* chore(query): note the IN fallback restriction for scalar subquery filters



* chore(query): document the scalar subquery dyn filter bridge removal condition



* test(query): keep only dyn_filters of region scans in the scalar subquery plan case

#9565 added scan diagnostics to EXPLAIN ANALYZE VERBOSE and partition filter
rendering changed; the case only checks the forwarded dyn filter.



---------


(cherry picked from commit b9aa627180)

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2026-10-23 09:39:06 +08:00
discord9 8af3a04ed7 fix: preserve structured query errors through distributed execution (#9161) 
* fix: preserve structured query errors through distributed execution

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update SQL expectations for preserved query error codes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
 2026-09-16 06:28:15 +00:00
discord9andNing Sun 94d7e2c7fc feat!: upgrade DataFusion to 55 (#8555) 
* feat!: upgrade DataFusion dependencies to 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor: migrate DataFusion 55 APIs

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve table function planning behavior

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve PostgreSQL query compatibility

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve distributed execution plan behavior

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: cover DataFusion 55 behavior regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update DataFusion 55 SQLness expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: complete DataFusion 55 test API migration

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: address DataFusion 55 CI regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: address remaining DataFusion 55 regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: adapt latest base code to DataFusion 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: normalize environment-specific DataFusion 55 plans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: align final DataFusion 55 expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: isolate DataFusion 55 regression cases

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: preserve empty result schema in timestamp widening

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: preserve JSON source column order

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* chore: use released DataFusion 55 integrations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: adapt latest execution plan mock to DataFusion 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: pin DataFusion recursive schema and date repairs

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): align dictionary temporality match keys

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: retain Greptime DataFusion fork behaviors on version 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore ordinary function error expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh distributed count compatibility plan

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): adapt last-row cast hint to DataFusion 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh instant last-row empty results for Arrow 59

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* style: simplify DataFusion expression visitor imports

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore sorting and PostgreSQL column-order assertions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(function): restore primitive numeric coercion signatures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(function): share geo integer signature types

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: cover timestamp widening overflow boundaries

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: fix decimal coercion regression imports

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(function): preserve scalar count_hash NULL state semantics

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: simplify decimal clamp case type inference

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: retain historical count_hash wrapper result

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: restore timestamp widening equality and IN pruning

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: carry upstream aggregate dynamic filter correctness fix

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: carry upstream null and predicate simplification fixes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore baseline JSON ordering expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore histogram JSON ordering expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh empty PromQL range result schemas

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: align native timestamp plan with DF55 decimal display

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh native timestamp SQLness results for DF55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: regenerate NULL sample empty result headers for DF55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: use DF55 child replacement API in timestamp regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: expose pushed scan dynamic filters to DF55 producers

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: encode string-backed PostgreSQL OID aliases in binary results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: verify REGPROC binary and text over PostgreSQL protocol

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: register real PostgreSQL catalogs in server fixtures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: complete DF55 expression inventories for custom query plans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: correct RangeSelect expression fixture and column identities

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* ci: wait for Kafka WAL helper deployment rollout

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update custom storage empty result headers

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: require exact row counts in scan statistics

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: suppress deprecated partition_statistics warning in test

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
 2026-09-15 11:42:38 +00:00
discord9 da5cb1a190 perf(promql): push down last row for instant queries (#9034) 
* perf(promql): push down last row for instant queries

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: guard instant last row correctness

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update instant query explain results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: apply last row after source deduplication

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: scope post-merge last row selection

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): cover instant PromQL last row

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(perf): sort generated SST rows before writing

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): cover instant last row selection in sqlness

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): avoid last row hints for lossy timestamp casts

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): preserve stale marker semantics across flushes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): decode dictionary labels in stale regression

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): avoid reserved column name in stale fixture

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): exercise LastRow hints and filtered results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor: keep after-merge mode in LastRow selector

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: reject instant LastRow across residual filters

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: expect after-merge selector in instant vector guards

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs: explain instant LastRow filter eligibility

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: restrict instant LastRow to safe selector nodes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor: show LastRow merge mode directly in diagnostics

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh LastRow display in explain expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
 2026-09-09 12:38:52 +00:00
jeremyhi 764c93bf43 perf(query): choose bounded CTE as hash join build side (#8807) 
* fix(query): choose bounded CTE as hash join build side

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf(query): remove join estimate cap

* test(compat): accept repartition in analyze plan

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-08-13 10:51:53 +00:00
discord9 5ed7128f06 perf(query): prune RangeSelect input projections (#8570) 
* perf(query): prune RangeSelect input projections

Signed-off-by: discord9 <discord9@163.com>

* test(query): stabilize RangeSelect projection snapshot

Signed-off-by: discord9 <discord9@163.com>

* test(query): update RangeSelect nested plan snapshot

Signed-off-by: discord9 <discord9@163.com>

* fix(query): handle nested RangeSelect aliases

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-30 07:09:49 +00:00
discord9 9776d88dc8 fix: preserve dictionary regex filter semantics (#8688) 
* fix: preserve dictionary regex filter semantics

Signed-off-by: discord9 <discord9@163.com>

* test(query): update dictionary regex plan results

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-30 06:20:57 +00:00
discord9 b30765e5e9 fix: preserve distributed topk merge ordering (#8432) 
* fix: preserve distributed topk for scalar latest

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve scalar latest ordering across merge scan

Signed-off-by: discord9 <discord9@163.com>

* fix: carry merge scan ordering from planner rewrite

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan partition ordering gate

Signed-off-by: discord9 <discord9@163.com>

* fix: require adjacent merge sort for ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: update ordering sqlness plans

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve distributed topk merge ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover latest per series queries

Signed-off-by: discord9 <discord9@163.com>

* test: update distributed merge sort sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering partition gate

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan over-partition ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover distributed latest with low parallelism

Signed-off-by: discord9 <discord9@163.com>

* fix: keep distributed merge sort opaque

Signed-off-by: discord9 <discord9@163.com>

* test: assert distributed scalar latest merge sort

Signed-off-by: discord9 <discord9@163.com>

* test: clarify merge scan ordering helper

Signed-off-by: discord9 <discord9@163.com>

* fix: complete merge sort exec delegation

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort limit pushdown

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer opacity

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve merge sort child topk

Signed-off-by: discord9 <discord9@163.com>

* test: update order by topk plan

Signed-off-by: discord9 <discord9@163.com>

* fix: recognize merge sort global fetch

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-14 12:53:07 +00:00
discord9 ff2fa71d50 fix: repartition subset partition key joins (#8460) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-10 06:10:28 +00:00
discord9 9fafd879ed fix: window sort off by one precision TimeRange&better alias track (#8019) 
* fix: window sort track alias&off by one precision TimeRange

Signed-off-by: discord9 <discord9@163.com>

* chore: more test

Signed-off-by: discord9 <discord9@163.com>

* refactor: clear helper

Signed-off-by: discord9 <discord9@163.com>

* dedup a bit

Signed-off-by: discord9 <discord9@163.com>

* feat: even more guard

Signed-off-by: discord9 <discord9@163.com>

* fix: case insensitive

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-04-24 13:56:51 +00:00
LFCNing SunNing Sun
 		b2074e3863 chore: upgrade DataFusion family, again (#7578) 
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch to released version of datafusion-pg-catalog

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
 2026-03-03 07:36:39 +00:00
LFC e64c31e59a chore: upgrade DataFusion family (#7558) 
* chore: upgrade DataFusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* use main proto

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-01-14 14:02:31 +00:00
Ruihang Xia fce1687fa7 fix: incorrect timestamp index inference (#7530) 
* add sqlness case, but can't reproduce

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reproduction

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix wildcard rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-01-07 11:18:25 +00:00
discord9 ac6d68aa2d fix: simp expr recursively (#7523) 
* fix: simp expr recursively

Signed-off-by: discord9 <discord9@163.com>

* test: some simple constant folding case

Signed-off-by: discord9 <discord9@163.com>

* fix: literal ts cast to UTC

Signed-off-by: discord9 <discord9@163.com>

* fix: patch merge scan batch col tz instead

Signed-off-by: discord9 <discord9@163.com>

* test: fix

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-01-07 09:22:26 +00:00
discord9 68247fc9b1 fix: count_state use stat to eval&predicate w/out region (#7116) 
* fix: count_state use stat to eval

Signed-off-by: discord9 <discord9@163.com>

* cleanup

Signed-off-by: discord9 <discord9@163.com>

* fix: use predicate without region

Signed-off-by: discord9 <discord9@163.com>

* test: diverge standalone/dist impl

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2025-10-27 02:14:45 +00:00
LFCandNing Sun 2f637a262e chore: update datafusion to 50 (#7076) 
* chore: update datafusion to 50

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* fix: update datafusion_pg_catalog import

* chore: fix toml format

* chore: fix toml format again

* fix nextest

Signed-off-by: luofucong <luofc@foxmail.com>

* fix sqlness

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: switch datafusion-orc to upstream tag

* fix sqlness

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
 2025-10-23 07:18:36 +00:00
discord9 aca8b690d1 fix: step aggr merge phase not order nor filter (#6998) 
* fix: not order

Signed-off-by: discord9 <discord9@163.com>

* test: redacted

Signed-off-by: discord9 <discord9@163.com>

* feat: fix up state wrapper

Signed-off-by: discord9 <discord9@163.com>

* df last_value state not as promised!

Signed-off-by: discord9 <discord9@163.com>

* fix?: could fix better

Signed-off-by: discord9 <discord9@163.com>

* test: unstable result

Signed-off-by: discord9 <discord9@163.com>

* fix: work around by fixing state

Signed-off-by: discord9 <discord9@163.com>

* chore: after rebase fix

Signed-off-by: discord9 <discord9@163.com>

* chore: finish some todo

Signed-off-by: discord9 <discord9@163.com>

* chore: per copilot

Signed-off-by: discord9 <discord9@163.com>

* refactor: not fix but just notify mismatch

Signed-off-by: discord9 <discord9@163.com>

* chore: warn -> debug state mismatch

Signed-off-by: discord9 <discord9@163.com>

* chore: refine error msg

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness add last_value date_bin test

Signed-off-by: discord9 <discord9@163.com>

* ?: substrait order by decode failure

Signed-off-by: discord9 <discord9@163.com>

* unit test reproduce that

Signed-off-by: discord9 <discord9@163.com>

* feat: support state wrapper's order serde in substrait

Signed-off-by: discord9 <discord9@163.com>

* refactor: stuff

Signed-off-by: discord9 <discord9@163.com>

* test: standalone/distributed different exec

Signed-off-by: discord9 <discord9@163.com>

* fmt

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* refactor: closure

Signed-off-by: discord9 <discord9@163.com>

* test: first value order by

Signed-off-by: discord9 <discord9@163.com>

* refactor: per cr

Signed-off-by: discord9 <discord9@163.com>

* feat: ScanHint last_value last row selector

Signed-off-by: discord9 <discord9@163.com>

* docs: per cr

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2025-09-26 09:12:45 +00:00
Ruihang Xia 288f69a30f fix: plan disorder from upgrading datafusion (#6787) 
* fix: plan disorder from upgrading datafusion

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-08-23 12:29:47 +00:00
LFC f9d2a89a0c chore: update datafusion family (#6675) 
* chore: update datafusion family

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

* use official otel-arrow-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* rebase

Signed-off-by: luofucong <luofc@foxmail.com>

* use the official orc-rust

Signed-off-by: luofucong <luofc@foxmail.com>

* resolve PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

* remove the empty lines

Signed-off-by: luofucong <luofc@foxmail.com>

* try following PR comments

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-08-15 12:41:49 +00:00
discord9 6501b0b13c feat: MergeScan print input (#6563) 
* feat: MergeScan print input

Signed-off-by: discord9 <discord9@163.com>

* test: fix ut

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2025-07-21 06:40:15 +00:00
Ruihang Xia 7468a8ab2a feat: organize EXPLAIN ANALYZE VERBOSE's output in JSON format (#6308) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-06-12 09:55:53 +00:00
Ruihang Xia bbb6f8685e feat: implement commutativity rule for prom-related plans (#5875) 
* feat: implement commutativity rule for prom-related plans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix range manipulate deserializer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* blocklist in commutativity rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change dictionary type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle partition and ordering

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add rate, increase and delta

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* regexp_replace uses empty string instead of null value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-05-13 09:06:25 +00:00
discord9 0124a0d156 fix: window sort not apply when other column alias to time index name (#5634) 
* fix: other col alias to time index column handle

* test: update sqlness

* chore: per review

* test: more sqlness

* test: mv some to optimizer folder

* fix: resolve alias properly

* fix: also retain old name

* chore: remove wrong comment

* chore: fix sqlness

* test: standalone/dist more projection diff
 2025-03-06 08:05:57 +00:00
YingwenjeremyhiRuihang Xialuofucongdiscord9shuiyisong
 		35b635f639 feat!: Bump datafusion, prost, hyper, tonic, tower, axum (#5417) 
* change dep

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: adapt to arrow's interval array

* chore: fix compile errors in datatypes crate

* chore: fix api crate compiler errors

* chore: fix compiler errors in common-grpc

* chore: fix common-datasource errors

* chore: fix deprecated code in common-datasource

* fix promql and physical plan related

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wip: upgrading network deps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* block on updating `sqlparser`

* upgrade sqlparser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adapt new df's trait requirements

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix compiler errors in mito2

* chore: fix common-function crate errors

* chore: fix catalog errors

* change import path

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix some errors in query crate

* chore: fix some errors in query crate

* aggr expr and some other tiny fixes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix expr related errors in query crate

* chore: fix query serializer and admin command

* chore: fix grpc services

* feat: axum serve

* chore: fix http server

* remove handle_error handler
* refactor timeout layer
* serve axum

* chore: fix flow aggr functions

* chore: fix flow

* feat: fix errors in meta-srv

* boxed()
* use TokioIo

* feat!: Remove script crate and python feature (#5321)

* feat: exclude script crate

* chore: simplify feature

* feat: remove the script crate

* chore: remove python feature and some comments

* chore: fix warning

* chore: fix servers tests compiler errors

* feat: fix tests-integration errors

* chore: fix unused

* test: fix catalog test

* chore: fix compiler errors for crates using common-meta

testing feature is enabled when check with --workspace

* test: use display for logical plan test

* test: implement rewrite for ScanHintRule

* fix: http server build panic

* test: fix mito test

* fix: sql parser type alias error

* test: fix TestClient not listen

* test: some flow tests

* test(flow): more fix

* fix: test_otlp_logs

* test: fix promql test that using deprecated method fun()

* fix: sql type replace supports Int8 ~ Int64, UInt8 ~ UInt64

* test: fix infer schema test case

* test: fix tests related to plan display

* chore: fix last flow test

* test: fix function format related assertion

* test: use larger port range for tests

* fix: test_otlp_traces

* fix: test_otlp_metrics

* fix range query and dist plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: flow handle distinct use deprecated field

* fix: can't pass Join plan expressions to LogicalPlan::with_new_exprs

* test: fix deserialize test

* test: reduce split key case num

* tests: lower case aggr func name

* test: fix some sqlness tests

* tests: more sqlness fix

* tests: fixed sqlness test

* commit non-bug changes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: make our udf correct

* fix: implement empty methods of ContextProvider for DfContextProviderAdapter

* test: update sqlness test result

* chore: remove unused

* fix: provide alias name for AggregateExprBuilder in range plan

* test: update range query result

* fix: implement missing ContextProvider methods for DfContextProviderAdapter

* test: update timestamps, cte result

* fix: supports empty projection in mito

* test: update comment for cte test

* fix: support projection for numbers

* test: update test cases after projection fix

* fix: fix range select first_value/last_value

* fix: handle CAST and time index conflict

* fix: handle order by correctly in range first_value/last_value

* test: update sqlness result

* test: update view test result

* test: update decimal test

wait for https://github.com/apache/datafusion/pull/14126 to fix this

* feat: remove redundant physical optimization

todo(ruihang): Check if we can remove this.

* test: update sqlness test result

* chore: range select default sort use nulls_first = false

* test: update filter push down test result

* test: comment deciaml test to avoid different panic message

* test: update some distributed test result

* test: update test for distributed count and filter push down

* test: update subqueries test

* fix: SessionState may overwrite our UDFs

* chore: fix compiler errors after merging main

* fix: fix elasticsearch and dashboard router panic

* chore: fix common-functions tests

* chore: update sqlness result

* test: fix id keyword and update sqlness result

* test: fix flow_null test

* fix: enlarge thread size in debug mode to avoid overflow

* chore: fix warnings in common-function

* chore: fix warning in flow

* chore: fix warnings in query crate

* chore: remove unused warnings

* chore: fix deprecated warnings for parquet

* chore: fix deprecated warning in servers crate

* style: fix clippy

* test: enlarge mito cache tttl test ttl time

* chore: fix typo

* style: fmt toml

* refactor: reimplement PartialOrd for RangeSelect

* chore: remove script crate files introduced by merge

* fix: return error if sql option is not kv

* chore: do not use ..default::default()

* chore: per review

* chore: update error message in BuildAdminFunctionArgsSnafu

Co-authored-by: jeremyhi <jiachun_feng@proton.me>

* refactor: typed precision

* update sqlness view case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: flow per review

* chore: add example in comment

* chore: warn if parquet stats of timestamp is not INT64

* style: add a newline before derive to make the comment more clear

* test: update sqlness result

* fix: flow from substrait

* chore: change update_range_context log to debug level

* chore: move axum-extra axum-macros to workspace

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: luofucong <luofc@foxmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: shuiyisong <xixing.sys@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
 2025-01-23 06:15:40 +00:00
Lei, HUANG 216bce6973 perf: count(*) for append-only tables (#4545) 
* feat: support fast count(*) for append-only tables

* fix: total_rows stats in time series memtable

* fix: sqlness result changes for SinglePartitionScanner -> StreamScanAdapter

* fix: some cr comments
 2024-08-13 09:27:50 +00:00
Ruihang Xia a7aa556763 feat: output multiple partition in MergeScanExec (#4227) 
* feat: output multiple partition in MergeScanExec

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix range manipulate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-06-28 13:45:22 +00:00
Ruihang XiaandJeremyhi c2218f8be8 build(deps): bump datafusion 20240528 (#4061) 
* build(deps): bump datafusion 20240528

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* another update

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update expected sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix first/last value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reformat comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix remaining errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert toml format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyo3 feature

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove dead code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* format file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-06-01 14:03:00 +00:00
Yingwen 179c8c716c feat: Adds RegionScanner trait (#3948) 
* feat: define region scanner

* feat: single partition scanner

* feat: use single partition scanner

* feat: implement ExecutionPlan wip

* feat: mito engine returns single partition scanner

* feat: implement DisplayAs for region server

* feat: dummy table provider use handle_partitioned_query()

* test: update sqlness test

* feat: table provider use ReadFromRegion

* refactor: remove StreamScanAdapter

* chore: update lock

* style: fix clippy

* refactor: remove handle_query from the RegionEngine trait

* chore: address CR comments

* refactor: rename methods

* refactor: rename ReadFromRegion to RegionScanExec
 2024-05-20 11:52:00 +00:00
Ruihang XiaandJeremyhi aec5cca2c7 feat: support distributed EXPLAIN ANALYZE (#3908) 
* feat: fetch and pass per-plan metrics

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl DistAnalyzeExec

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/analyze.rs

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-05-10 12:31:29 +00:00
LFCYingwenRuihang XiaWUJingdidiscord9tison
 		314f2704d4 build(deps): update datafusion to latest and arrow to 51.0 (#3661) 
* chore: update datafusion

* update sqlness case of time.sql

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: adjust range query partition

* fix: hisogram incorrect result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: ignore filter pushdown temporarily

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: update limit sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: histogram with wrong distribution

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: update negative ordinal sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: bump df to cd7a00b

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve conflicts

* ignore test_range_filter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix promql exec panic

* fix "select count(*)" exec error

* re-enable the "test_range_filter" test since the filter push down seems not necessary to be removed

* fix: range query schema error

* update sqlness results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve conflicts

* update datafusion, again

* fix pyo3 compile error, and update some sqlness results

* update decimal sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: promql literal

* fix udaf tests

* fix filter pushdown sqlness tests

* fix?: test_cast

* fix: rspy test fail due to datafusion `sin` signature change

* rebase main to see if there are any failed tests

* debug ci

* debug ci

* debug ci

* enforce input partition

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* debug ci

* fix ci

* fix ci

* debug ci

* debug ci

* debug ci

* fix sqlness

* feat: do not return error while creating a filter

* chore: remove array from error

* chore: replace todo with unimplemented

* Update src/flow/clippy.toml

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WUJingdi <taylor-lagrange@qq.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-04-18 12:07:18 +00:00
Ruihang Xia 56fc77e573 fix: add missing error display message (#2791) 
* fix: add missing error display message

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-23 02:59:49 +00:00
Ruihang Xia 5d8b0e8154 chore: bump dependencies (#2659) 
* fix all hard error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* trivial changes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix order by

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix sql keyword and data type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix range exec's input partitioning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix cover input type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix explain analyze

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update distributed mode sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix lints

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update locks

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* downgrade dlv-list

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-10-26 12:24:14 +00:00
Ning Sun 0fbde48655 feat: hide internal error and unknown error message from end user (#2544) 
* feat: use fixed error message for unknown error

* feat: return fixed message for internal error as well

* chore: include status code in error message

* test: update tests for asserts of error message

* feat: change status code of some datafusion error

* fix: make CollectRecordbatch an query error

* test: update sqlness results
 2023-10-23 03:07:35 +00:00
Ruihang XiaandYingwen 10ecc30817 feat: pushdown aggr, limit and sort plan (#2495) 
* check partition for aggr plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle empty partition rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove CheckPartition option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update some valid sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* opt-out promql plan and update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix limit

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix insert select subquery

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update unit test result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/dist_plan/analyzer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-09-28 06:35:45 +00:00
JeremyHi 52ac093110 fix: drop table 0 rows affected (#2515) 2023-09-28 06:21:18 +00:00
Ruihang Xia 92824d1c66 fix: update several sqlness results (#2422) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-17 11:33:40 +00:00
Ruihang Xia de723d9c1c fix: update sqlness result in distributed mode (#2381) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-13 09:07:55 +00:00
dennis zhuang db89235474 feat: only allow timestamp type as time index (#2281) 
* feat: only allow timestamp data type as time index

* test: update sqltest cases, todo: need some fixes

* fix: sqlness tests

* fix: forgot adding back cte test

* chore: style
 2023-09-12 07:57:15 -05:00
Ruihang Xia e5ba3d1708 feat: rewrite the dist analyzer (#2238) 
* it works!

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add documents

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unstable timestamp from sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename rewriter struct

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-24 03:29:08 +00:00
Ruihang XiaandZhenchi d581688fd2 fix: dist planner has wrong behavior in table with multiple partitions (#2237) 
* fix: dist planner has wrong behavior in table with multiple partitions

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update tests/cases/distributed/explain/multi_partitions.sql

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-23 08:32:20 +00:00
Zhenchiandzhongzc 7abe71f399 fix(table): return correct table types (#2131) 
* fix(table): return correct table types

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix: NumbersTable to be Temporary table

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix(test): fix affected cases

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix(test): fix affected cases

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix: fmt

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix(tests): fix instance_test expected result

* retrigger action

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>
 2023-08-09 11:07:00 +00:00
Ruihang XiaandLei, HUANG 4c693799d8 fix: bugs related to merge scan (#2118) 
* fix: prevent optimize merge scan, mark distinct as unsupported

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix some other problems

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix unit tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add some comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/optimizer/type_conversion.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-08-08 11:42:57 +00:00
Ruihang Xia 44f3ed2f74 chore(deps): bump datafusion to the latest commit (#1967) 
* bump deps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile except pyo3 backend

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix promql-parser metric name matcher

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyarrow convert

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyo3 compiling

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update stream adapter display format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix physical optimizer rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-01 02:10:49 +00:00
Ruihang XiaandYingwen 5bd80a74ab feat: prepare for implementing considering partition key in the distributed planner (#2000) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix frontend logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check substrait compatibility before pushdown

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* going to revert some rules

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test and clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove println

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-07-31 12:36:23 +00:00
Ruihang Xia bee8323bae chore: bump sqlness to 0.5.0 (#1877) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-07-04 19:49:12 +08:00
Ruihang Xia b1ccc7ef5d fix: prevent filter pushdown in distributed planner (#1806) 
* fix: prevent filter pushdown in distributed planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix metadata

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-21 16:25:50 +08:00
Ruihang XiaYingwenJeremyHi
 		2615718999 feat: merge scan for distributed execution (#1660) 
* generate exec plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move DatanodeClients to client crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wip MergeScanExec::to_stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix default catalog

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix expand order of new stage

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move sqlness cases contains plan out of common dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor information schema to allow duplicated scan call

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: ignore two cases due to substrait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reorganise sqlness common cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* redact round robin partition number

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

* skip tranforming projection

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert common/order

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/dist_plan/merge_scan.rs

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore region failover IT

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result again and again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* unignore some tests about projection

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enable failover tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-06-02 06:42:54 +00:00
LFC 8d3999df5f fix: failed to run subquery wrapped in two parentheses (#1157) 2023-03-14 10:59:43 +08:00