16e9abe4a9
chore: pick fixes and bump version to v1.1.4 ( #8630 )
...
* fix(query): validate DistAnalyzeExec child count (#8510 )
* fix(query): validate DistAnalyzeExec child count
Signed-off-by: discord9 <discord9@163.com >
* test(query): avoid implicit clone lint
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 623145e635#8484 )
Value::try_negative and the temporal negative() helpers negated with raw
unary minus, which panics (debug) or wraps (release) on MIN values such as
-i64::MIN. try_negative already returns None for the unsigned arms; make
the signed and temporal arms honor that contract via checked_neg /
checked_negative so a MIN literal produces a clean error instead.
Signed-off-by: raphaelroshan <raphaelroshan@gmail.com >
(cherry picked from commit 1a476f76e2#8508 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit c7bcddacc5#8227 ) (#8238 )
* fix: timestamp display precision should respect column schema (#8227 )
Previously, the MySQL writer converted timestamps to NaiveDateTime via
to_chrono_datetime_with_timezone(), then passed the NaiveDateTime to
write_col(). NaiveDateTime::Display uses a fixed 6-digit fractional-
second format, causing:
- TIMESTAMP(3) to show '.195000' instead of '.195'
- TIMESTAMP(9) to show '.195123' instead of '.195123456'
Fix: use Timestamp::to_timezone_aware_string() directly, which formats
with chrono's '%.f' specifier — it strips trailing zeros and preserves
full nanosecond fidelity based on the actual stored unit.
Also add comprehensive unit tests in mysql_writer_test.rs covering:
- All concrete data type → MySQL column type mappings
- UNSIGNED_FLAG propagation for unsigned integer types
- Timestamp precision for all four units (0/3/6/9 decimal places)
- Edge cases: zero subseconds, trailing-zero stripping, Unix epoch,
negative timestamps, and timezone offset shifts
- Column-def ordering, empty schema, and decimal variants
Signed-off-by: Divyansh <anshmcs@gmail.com >
* perf: reuse format buffer for timestamp serialization to avoid per-row heap allocation
Signed-off-by: Divyansh <anshmcs@gmail.com >
* refactor: rename test variable and update visibility of create_mysql_column function
Signed-off-by: Divyansh <anshmcs@gmail.com >
* test: fix timestamp test constants and add sqlness case for display precision
- Correct the base epoch constant: 1_748_836_200 is 2025-06-02, not
2026-06-02; use 1_780_372_200 to match the documented instant.
- Fix subsecond expectations: chrono's %.f renders fractional digits in
groups of 3 (.100 / .010), it does not strip to .1 / .01.
- Add sqlness case timestamp_precision_display reproducing issue #8227
over the MySQL protocol (TIMESTAMP(0/3/6/9) rendering).
- cargo fmt reflow of create_mysql_column signature.
Signed-off-by: Divyansh <anshmcs@gmail.com >
* fix: support binary protocol for timestamp column
Signed-off-by: Divyansh <anshmcs@gmail.com >
* test: update sqlness results for new timestamp display precision
Signed-off-by: Divyansh <anshmcs@gmail.com >
---------
Signed-off-by: Divyansh <anshmcs@gmail.com >
(cherry picked from commit 1acdec9334#8567 )
Signed-off-by: WenyXu <wenymedia@gmail.com >
(cherry picked from commit d1e61a4ad2#8558 )
* fix: display codes in metasrv client errors
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: normalize meta client errors in sqlness
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
(cherry picked from commit b1580c98f6#8563 )
* fix(mito): notify bulk writes on WAL error
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* fix(mito): move write ctx tests after items
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* docs: mention PR template in agent guide
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
(cherry picked from commit 25909bd1aa#8588 )
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
(cherry picked from commit 036db3d1ec#8519 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 4a0dd8f9bc#8592 )
fix(flow): reduce expected batching log noise
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 9f5a11e82a#8572 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 4c5be47400#8574 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 45fc51ebd2#8539 )
Parse PostgreSQL DSNs (URL or libpq keyword) with tokio_postgres::Config —
the backend's own parser — and log its Debug, which redacts the password.
This matches the backend grammar exactly (multi-host URIs, backslash
escapes, any Unicode whitespace, percent-encoded query keys, and '&'/';'/
'://' inside values) rather than approximating it by hand. Other URLs are
redacted via the url crate; a best-effort keyword fallback covers inputs
neither parser accepts.
Signed-off-by: raphaelroshan <raphaelroshan@gmail.com >
(cherry picked from commit e19df8960e#8618 )
When a compaction finishes with no pending request/DDL, on_compaction_finished
leaves the region's CompactionStatus in the map with no active compaction. If
the worker then skips schedule_next_compaction because min_compaction_interval
has not passed, the stale status is never cleaned up: all subsequent compaction
requests are swallowed by the "status exists" branch (regular waiters hang,
manual StrictWindow requests pend forever) and the region never compacts again
until restart.
Add CompactionScheduler::remove_inactive_status() and call it from
handle_compaction_finished when the interval gate skips scheduling the next
compaction, restoring the invariant that a status in the map always has an
active compaction.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
(cherry picked from commit d1ac9e2da7#8616 )
* fix(meta): configure gRPC message limits
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(meta): preserve follower range limits
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
(cherry picked from commit 82de463940#8481 )
* fix(flow): convert streaming expiration to milliseconds
Flow EXPIRE AFTER values are stored in seconds, while streaming flow timestamps and durations use milliseconds. Passing the value through unchanged expires state and limits refill scans 1000 times too early.
Convert the value at streaming create and refill boundaries, reject overflow, and document the CreateFlowArgs unit.
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
* fix(flow): reject negative streaming expiration
Reject negative EXPIRE AFTER values at the streaming adapter boundary so refill ranges cannot move into the future. Keep zero valid and retain checked seconds-to-milliseconds conversion.
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
---------
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
* fix: qualify scalar-subquery tables in persisted views (#8581 )
fix(query): traverse scalar subqueries in views
Signed-off-by: discord9 <discord9@163.com >
* chore: bump version
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: raphaelroshan <raphaelroshan@gmail.com >
Signed-off-by: Divyansh <anshmcs@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
Signed-off-by: evenyag <realevenyag@gmail.com >
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: raphaelroshan <49832307+raphaelroshan@users.noreply.github.com >
Co-authored-by: Div <anshmcs@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
Co-authored-by: QuakeWang <45645138+QuakeWang@users.noreply.github.com >
2026-07-24 15:28:47 +08:00
63ef18a74a
chore: pick fixes and bump version to v1.1.3 ( #8545 )
...
* fix(query): push down PromQL cast filters (#8407 )
* fix(query): push down PromQL cast filters
Signed-off-by: discord9 <discord9@163.com >
* test(query): cover cast preimage pushdown
Signed-off-by: discord9 <discord9@163.com >
* test(query): expand cast preimage coverage
Signed-off-by: discord9 <discord9@163.com >
* test(query): compare cast normalization rules
Signed-off-by: discord9 <discord9@163.com >
* test(query): update cast pushdown sqlness plans
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 4f5dccf6bb#8417 )
(cherry picked from commit 434e0c7682#8412 )
* fix: global limit for distributed inspect streams
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: honor tighter filter fetch under global limit
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: restore distribution after global limit
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: preserve inherited distribution for global limit
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: propagate inherited hash partitioning
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
(cherry picked from commit 5bcc2e9d6b#8431 )
(cherry picked from commit ebfe4f6eb2#8445 )
* fix: preserve repartitioned file refs during GC
Signed-off-by: discord9 <discord9@163.com >
* refactor: clarify file refs target region
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 149b687a92#8460 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit ff2fa71d50#8495 )
* fix: exclude logs end boundary
Signed-off-by: discord9 <discord9@163.com >
* fix: normalize log query date bounds
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 99235e4b2d#8466 )
fix: skip oversized compaction tasks
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
(cherry picked from commit 0fcf61c3fb#8276 )
* feat: track scan output bytes and use them for read costing
Track the actual byte count of each record batch produced by
`RegionScanExec` and use it in place of the aggregated per-plan
`memory_usage` as the `table_scan` cost input. This avoids double
counting bytes that flow through multiple operators.
A named constant `REGION_SCAN_EXEC_NAME` exposes the plan node name
so downstream metric parsers remain correct if the struct is renamed.
Affected files:
- `src/table/src/table/metrics.rs` -- add `output_bytes` counter
- `src/table/src/table/scan.rs` -- record bytes, export name constant
- `src/query/src/dist_plan/merge_scan.rs` -- consume scan bytes
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* refactor: remove misleading `mem_used` gauge from scan metrics
The `mem_used` gauge was used with `add()` on every output batch, but a
RegionScan does not hold memory across polls — each batch is yielded
immediately to the upstream operator. The gauge semantics were
incorrect (cumulative `add()` on a gauge) and the value duplicated
`output_bytes` anyway.
Affected files:
- `src/table/src/table/metrics.rs` — drop `mem_used` field and methods
- `src/table/src/table/scan.rs` — remove `record_mem_usage` call
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* fix: use stable plan names for scan byte metrics
Expose `plan_name` alongside rendered plan text so scan byte extraction no longer depends on EXPLAIN formatting. Keep old serialized metrics compatible by defaulting missing `plan_name`.
Affected files:
- `src/common/recordbatch/src/adapter.rs` -- add `plan_name` to `PlanMetrics` and populate it from `ExecutionPlan::name`
- `src/query/src/dist_plan/merge_scan.rs` -- match `REGION_SCAN_EXEC_NAME` through `plan_name`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* fix: preserve merge scan CPU read cost
Record read cost whenever stream metrics are available, default scan bytes to zero, and aggregate scan output bytes across region scan nodes.
Files: `src/query/src/dist_plan/merge_scan.rs`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* fix: correct doc comment in `StreamMetrics`
Fix doc comment on `StreamMetrics::new()` to reference the correct
struct name (`StreamMetrics`) instead of the old `MemoryUsageMetrics`.
- `src/table/src/table/metrics.rs` — fix struct name in doc comment
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
(cherry picked from commit 4373c77d35#8480 )
* fix: account logical record batch slice memory
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: account nested view payload memory
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* test: align scan memory expectation with logical size
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* refactor: name Arrow inline view limit
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: account visible list child memory
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: exclude null list child memory
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: exclude null struct child memory
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: account fixed-size list slices
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: limit scan accounting to flat slices
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* docs: clarify nested view accounting scope
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* perf: reduce view memory accounting overhead
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: fix benchmark license header
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* perf: optimize view slice accounting
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* test: cover sliced mixed-null view accounting
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* refactor: narrow slice accounting to arrow buffers
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* docs: note view accounting assumption
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
(cherry picked from commit 9cde31a647#8432 )
* fix: preserve distributed topk for scalar latest
Signed-off-by: discord9 <discord9@163.com >
* fix: preserve scalar latest ordering across merge scan
Signed-off-by: discord9 <discord9@163.com >
* fix: carry merge scan ordering from planner rewrite
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan ordering metadata
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan partition ordering gate
Signed-off-by: discord9 <discord9@163.com >
* fix: require adjacent merge sort for ordering metadata
Signed-off-by: discord9 <discord9@163.com >
* test: update ordering sqlness plans
Signed-off-by: discord9 <discord9@163.com >
* fix: preserve distributed topk merge ordering
Signed-off-by: discord9 <discord9@163.com >
* test: cover latest per series queries
Signed-off-by: discord9 <discord9@163.com >
* test: update distributed merge sort sqlness
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan ordering partition gate
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan over-partition ordering
Signed-off-by: discord9 <discord9@163.com >
* test: cover distributed latest with low parallelism
Signed-off-by: discord9 <discord9@163.com >
* fix: keep distributed merge sort opaque
Signed-off-by: discord9 <discord9@163.com >
* test: assert distributed scalar latest merge sort
Signed-off-by: discord9 <discord9@163.com >
* test: clarify merge scan ordering helper
Signed-off-by: discord9 <discord9@163.com >
* fix: complete merge sort exec delegation
Signed-off-by: discord9 <discord9@163.com >
* docs: explain merge sort limit pushdown
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge sort optimizer opacity
Signed-off-by: discord9 <discord9@163.com >
* docs: explain merge sort optimizer hooks
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge sort optimizer hooks
Signed-off-by: discord9 <discord9@163.com >
* fix: preserve merge sort child topk
Signed-off-by: discord9 <discord9@163.com >
* test: update order by topk plan
Signed-off-by: discord9 <discord9@163.com >
* fix: recognize merge sort global fetch
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit b30765e5e9#8517 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit d48f296144#8514 )
* fix(log-store): deduplicate Kafka WAL multipart records
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(log-store): preserve delayed Kafka WAL entry offsets
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(log-store): reject conflicting Kafka WAL last records
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(log-store): emit Kafka WAL entries on last record
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(log-store): discard duplicate Kafka WAL first records
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
(cherry picked from commit 912db22417#8496 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 1122d1e7d7#8532 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 887fdf7513#8438 )
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
(cherry picked from commit f12a1da3dewenymedia@gmail.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: Ning Sun <sunng@protonmail.com >
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
Co-authored-by: sun <sunchang_long@163.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2026-07-17 11:30:08 +08:00
8ad2d2414c
chore: pick fixes and bump version to v1.1.2 ( #8404 )
...
* fix: improve Grafana metrics dashboards (#8298 )
* chore: initial changes
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: improve troubleshooting dashboard
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: rm troubleshooting-dashboard.md
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: optimize metrics dashboard
Signed-off-by: evenyag <realevenyag@gmail.com >
* docs: move troubleshooting-dashboard.md
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: move mito gc duration panel
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: cleanup the dashboard
- Overview trend panels are now aggregate-only:
- Total Ingestion Rate Trend
- Total Query Rate Trend
- Protocol breakdowns remain in Ingestion and Queries.
- Mito Backpressure and Failures no longer duplicates scan/GC signals.
- Removed Write Stall per Instance.
- Split Object Store and WAL into collapsed Object Store and collapsed
WAL.
- Moved WAL/logstore panels out of Storage into WAL.
- Normalized OpenDAL “other request” matchers.
- Normalized trigger elapsed p99/p75/avg aggregation.
- Regenerated standalone JSON and dashboard YAML/Markdown.
- Updated docs/troubleshooting-dashboard.md.
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: rearrange metasrv dashboard panels
Signed-off-by: evenyag <realevenyag@gmail.com >
* feat: improve troubleshooting dashboard layout
Signed-off-by: evenyag <realevenyag@gmail.com >
* docs: remove obsolete troubleshooting dashboard doc
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: correct cluster dashboard panel queries (missing _bucket, raw counters, rate normalization)
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: correct trigger panel datasource, collapse flush/compaction, split request latency panels
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: update grafana metrics dashboard panels
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: correct Grafana dashboard units
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: regenerate Grafana dashboards
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: use throughput unit for index IO bytes
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: redact Kafka SASL password in debug output (#8337 )
## Summary
- Mask `KafkaClientSaslConfig` password fields in debug output while keeping usernames visible.
- Cover metasrv WAL debug output with a regression test.
## Files
- `src/common/wal/src/config/kafka/common.rs`
- `src/common/wal/src/config.rs`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(query): run optimizer rules before MergeScan (#8339 )
* fix(query): push down join filters before MergeScan
Signed-off-by: discord9 <discord9@163.com >
* fix(query): run optimizer before MergeScan pushdown
Signed-off-by: discord9 <discord9@163.com >
* fix(query): narrow pre-MergeScan filter pushdown
Signed-off-by: discord9 <discord9@163.com >
* fix(query): refine pre-MergeScan optimizer prepass
Signed-off-by: discord9 <discord9@163.com >
* fix(query): satisfy predicate extractor clippy
Signed-off-by: discord9 <discord9@163.com >
* test(query): cover pre-MergeScan optimizer edges
Signed-off-by: discord9 <discord9@163.com >
* test(query): cover set comparison prepass
Signed-off-by: discord9 <discord9@163.com >
* fix(query): guard remote scan filter pushdown
Signed-off-by: discord9 <discord9@163.com >
* fix(query): preserve subquery planning errors
Signed-off-by: discord9 <discord9@163.com >
* fix(query): preserve usable scan predicates
Signed-off-by: discord9 <discord9@163.com >
* fix(query): simplify scan predicate extraction
Signed-off-by: discord9 <discord9@163.com >
* fix(query): keep scan filter extraction scoped
Signed-off-by: discord9 <discord9@163.com >
* docs(query): explain pre-MergeScan optimizer
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: preserve bulk write grpc error details (#8349 )
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: include index files in GC listing (#8327 )
* fix: include index files in GC listing
Signed-off-by: discord9 <discord9@163.com >
* chore: filter GC index listing to puffins
Signed-off-by: discord9 <discord9@163.com >
* chore: simplify GC index listing stream
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: stream tables for prometheus label discovery (#8341 )
Signed-off-by: Ritwij Aryan Parmar <ritwij.aryan.parmar@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: account parquet metadata cache size (#8368 )
* fix: account parquet metadata cache size
Use Parquet metadata memory sizing for SST metadata cache weight and add regression coverage for byte-array page-index buffers.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
* fix: saturate sst meta cache weight
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: respect gc mailbox timeout for admin gc (#8363 )
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: record catalog and schema in slow queries (#8387 )
* fix: record catalog and schema in slow queries
Add catalog and schema context to slow query records while appending the new columns after existing fields to preserve column order.
- `src/common/frontend/src/slow_query_event.rs`: extend `SlowQueryEvent` schema and rows with `catalog_name` and `schema_name`, and cover append-only ordering.
- `src/catalog/src/process_manager.rs`: carry catalog and schema through `SlowQueryTimer`.
- `src/frontend/src/instance.rs`: capture context for SQL, plan, and PromQL slow query timers.
- `tests-integration/tests/sql.rs`: assert MySQL and PostgreSQL slow query records include catalog and schema.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* fix: address slow query review comment
Use `String::clone` when writing slow query catalog and schema values.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* fix: keep slow query schema only
Remove the slow query `catalog_name` column and keep `schema_name` as a non-null tag dimension.
- `src/common/frontend/src/slow_query_event.rs`: expose only `schema_name` in `SlowQueryEvent` rows and mark it as a tag.
- `src/catalog/src/process_manager.rs`: stop carrying catalog context in `SlowQueryTimer`.
- `src/frontend/src/instance.rs`: pass only schema context to slow query timers.
- `tests-integration/tests/sql.rs`: assert slow query records include `schema_name` without `catalog_name`.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* fix: schema name semantic should be field
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* fix: typo
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: invalidate comment DDL cache and lock by object ID (#8390 )
* fix: invalidate comment ddl cache locally
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: fix typos
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: client_ip error logs skip internal API (#8362 )
* chore: client_ip error logs skip internal API
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: fmt
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: use const
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: use const
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* feat: update dashboard to v0.13.6 (#8369 )
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: use ENV for building dashboard (#8384 )
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: handle PromQL time binary aggregation (#8398 )
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* perf(mito): prune files by manifest time range (#8352 )
* perf(mito): prune files by manifest time range
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): address file pruning review
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): remove verbose file pruning log
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): expose file pruning metric
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): shorten file pruning metric
Signed-off-by: discord9 <discord9@163.com >
* test(mito): cover file pruning edge cases
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* perf(mito): skip manifest-pruned file ranges (#8366 )
* perf(mito): skip manifest-pruned file ranges
Signed-off-by: discord9 <discord9@163.com >
* test(mito): allow empty prune benchmark output
Signed-off-by: discord9 <discord9@163.com >
* fix(mito): avoid caching stale pruned builders
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): address pruner clippy
Signed-off-by: discord9 <discord9@163.com >
* fix(mito): account worker pruner builder metrics
Signed-off-by: discord9 <discord9@163.com >
* test(mito): keep empty prune benchmark local
Signed-off-by: discord9 <discord9@163.com >
* refactor(mito): share manifest-pruned range skip
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): shorten prune cache comment
Signed-off-by: discord9 <discord9@163.com >
* fix(mito): keep manifest prune state in pruner
Signed-off-by: discord9 <discord9@163.com >
* test(mito): cover manifest prune fast skip edge cases
Signed-off-by: discord9 <discord9@163.com >
* chore: fix typo in logical table alter
Signed-off-by: discord9 <discord9@163.com >
* chore(mito): address pruner review comments
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: bump version to v1.1.2
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com >
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
Signed-off-by: Ritwij Aryan Parmar <ritwij.aryan.parmar@gmail.com >
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
Co-authored-by: Ritwij Aryan Parmar <88580521+RitwijParmar@users.noreply.github.com >
Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com >
Co-authored-by: sun <sunchang_long@163.com >
2026-07-02 21:21:19 +08:00
discord9
88fa784810
fix(flow): bind scheduled now in dist plan ( #8396 )
...
* fix(flow): bind scheduled now in dist plan (#8389 )
Signed-off-by: discord9 <discord9@163.com >
(cherry picked from commit 5f6b88b782discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-07-02 10:55:49 +08:00
Yingwen and discord9
4071cb56ae
feat(flow): stabilize eval interval scheduling ( #8360 ) ( #8378 )
...
* feat(flow): stabilize eval interval scheduling
* fix(flow): satisfy eval schedule clippy
* test(flow): trim eval schedule coverage
* test(flow): cover stable eval scheduling
* fix(flow): reserve scheduled runtime hint
* test(flow): trim sqlness result eof
* fix(flow): harden eval schedule edges
* fix(flow): address scheduled flow review
* fix(flow): clean scheduled config handling
* test(flow): add eval interval compat case
* test(flow): cover show create flow in compat
* fix(flow): drop scheduled time from flow context
* test(flow): assert scheduled now binding
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: evenyag <realevenyag@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
2026-06-29 21:27:22 +08:00
126c212546
chore: pick fixes and bump version to v1.1.1 ( #8332 )
...
* fix: remove flow sink table lock (#8317 )
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: guard structured JSON alignment paths against legacy JSONB columns (#8323 )
* Initial plan
* fix: guard structured json alignment to fix Clippy CI failure
- Add `is_structured_json_field` function that only returns true for
fields with both JSON extension type AND Struct Arrow data type
- Replace all usages of `is_json_extension_type` / `has_json_extension_field`
with `is_structured_json_field` to prevent legacy JSONB binary columns
from entering structured JSON alignment paths
- Fix logic in `FlatProjectionMapper::new_with_read_columns` to guard
JSON type hint concretization for JSON2 columns only
- Fix `create_column` in show_create_table.rs to only emit JSON structure
settings for JSON2 columns
- Move `mod tests` to end of flat_projection.rs to fix clippy::items_after_test_module
- Add tests for legacy JSON behavior
* fix: guard narrow_read_columns_by_json_type_hint with is_structured_json_field check
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: bump version to v1.1.1
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Signed-off-by: evenyag <realevenyag@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
2026-06-18 16:20:04 +08:00
Yingwen
4a4fe749d8
feat: add per-partition timings to merge scan partition metrics ( #8293 )
...
* feat: add per-partition timings to merge scan partition metrics
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: stop global timers for partitions with no region
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2026-06-12 13:22:49 +00:00
discord9
3608c90b2e
feat: pass snapshot read bounds over flight ( #8279 )
...
Signed-off-by: discord9 <discord9@163.com >
2026-06-11 12:05:02 +00:00
dennis zhuang
ca21c9f048
fix!: correct information_schema index metadata ( #8275 )
...
fix: correct information_schema index metadata
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
2026-06-11 01:36:23 +00:00
discord9
8fc5a3b1c7
feat: apply remote dynamic filters on datanode scans ( #8262 )
...
* feat: apply rdf
Signed-off-by: discord9 <discord9@163.com >
* chore: clippy
Signed-off-by: discord9 <discord9@163.com >
* fix: drop remote dyn filter fallback exec
Signed-off-by: discord9 <discord9@163.com >
* Revert "fix: drop remote dyn filter fallback exec"
This reverts commit bb757a596c3c2c078a836aa623226b7934103634.
Signed-off-by: discord9 <discord9@163.com >
* refactor: use rdf receiver logical plan instead
Signed-off-by: discord9 <discord9@163.com >
* test: update sqlness
Signed-off-by: discord9 <discord9@163.com >
* feat: rdf disable option
Signed-off-by: discord9 <discord9@163.com >
* tests: large int tests
Signed-off-by: discord9 <discord9@163.com >
* chore: clippy
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* test: update prec fix
Signed-off-by: discord9 <discord9@163.com >
* fix: make receiver node works
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* fix: tql disable rdf
Signed-off-by: discord9 <discord9@163.com >
* chore: rm useless joins
Signed-off-by: discord9 <discord9@163.com >
* fix: also disable in flow tql
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review rm to promql
Signed-off-by: discord9 <discord9@163.com >
* chore: promql ut
Signed-off-by: discord9 <discord9@163.com >
* per review
Signed-off-by: discord9 <discord9@163.com >
* test: rm misleading&add some nested/cleanup
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-06-10 12:57:23 +00:00
dennis zhuang
e403133eb2
feat: add information_schema statistics table ( #8253 )
...
* feat: add information_schema statistics table
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: use index-local sequence in statistics
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: ordinal_position for pk
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: statistics.nullable uses empty string for non-nullable columns
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
2026-06-08 12:52:02 +00:00
Weny Xu
9a814ca123
chore: add repartition debug logs ( #8245 )
...
Signed-off-by: WenyXu <wenymedia@gmail.com >
2026-06-08 08:48:23 +00:00
discord9
651cf04525
feat: fan out remote dynamic filter updates from FE ( #8241 )
...
* feat: fe send update
Signed-off-by: discord9 <discord9@163.com >
* refactor: after rebase
Signed-off-by: discord9 <discord9@163.com >
* fix: encode at least send bounds
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-06-08 08:24:57 +00:00
Ruihang Xia
428cd8137d
feat: identify noneffective binary modifiers ( #8230 )
...
* perf(promql): use tsid for full-label modifier joins
* fix(promql): collect narrow range joins by tsid
* verify on for both sides
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* check if tsid exists
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* correct behavior
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-06-05 08:44:50 +00:00
discord9
6b7772e457
feat: add remote dynamic filter frontend registration ( #8148 )
...
* feat: filter id
Signed-off-by: discord9 <discord9@163.com >
* feat: dyn filter registry
Signed-off-by: discord9 <discord9@163.com >
* feat: filter id&refactor to type
Signed-off-by: discord9 <discord9@163.com >
* feat: merge scan register dyn filter(not send yet)
Signed-off-by: discord9 <discord9@163.com >
* feat: init reg dyn filter
Signed-off-by: discord9 <discord9@163.com >
* wip: remote dyn filter task 03
Signed-off-by: discord9 <discord9@163.com >
* fix: resolve remote dyn filter rebase fallout
Signed-off-by: discord9 <discord9@163.com >
* chore: keep remote dyn filter docs local
Signed-off-by: discord9 <discord9@163.com >
* chore: remove stale filter id allow
Signed-off-by: discord9 <discord9@163.com >
* chore: clippy
Signed-off-by: discord9 <discord9@163.com >
* chore: fix remote dyn filter import style
Signed-off-by: discord9 <discord9@163.com >
* chore: fix query metrics test fallout
Signed-off-by: discord9 <discord9@163.com >
* fix: exclude region from remote dyn filter id
Signed-off-by: discord9 <discord9@163.com >
* chore: import
Signed-off-by: discord9 <discord9@163.com >
* refactor: rm some to latter
Signed-off-by: discord9 <discord9@163.com >
* feat: add initial dyn filter snapshot
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* docs: better comment, rm some slop
Signed-off-by: discord9 <discord9@163.com >
* chore: per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-06-04 12:31:42 +00:00
Ruihang Xia
edd563d2e1
feat: join simplifier for promql binary op ( #8211 )
...
* basic impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor, remove shallow methods
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* more sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clear ctx
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* prefix leaf aliase
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-06-04 08:13:24 +00:00
Ruihang Xia
85ae29cb0c
perf: collect narrow binary join ( #8193 )
...
* perf(promql): collect narrow binary join build side
* fix projection
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* finalize
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename mod
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-05-28 11:30:30 +00:00
discord9
eccd97b5c7
feat(flow): support incremental read checkpoints ( #8179 )
...
* feat: flownode inc mode
Signed-off-by: discord9 <discord9@163.com >
* chore: rename fallback reason
Signed-off-by: discord9 <discord9@163.com >
* fix: harden flow incremental checkpoints
Signed-off-by: discord9 <discord9@163.com >
* fix: address flow watermark lint
Signed-off-by: discord9 <discord9@163.com >
* fix: address flow clippy
Signed-off-by: discord9 <discord9@163.com >
* refactor: clarify incremental plan preparation
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* test: more sqlness test
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-05-28 09:31:46 +00:00
LFC
bf7e3551fe
test: add jsonbench tests ( #8165 )
...
Signed-off-by: luofucong <luofc@foxmail.com >
2026-05-27 08:34:06 +00:00
Yingwen
9487e2c3ca
fix: divide series for subquery output ( #8173 )
...
* fix: divide series for subquery output
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: propagate time index lookup error in prom_call_manipulate
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2026-05-27 07:10:24 +00:00
discord9
44f1804b5e
feat: add flow query-context plumbing for terminal watermarks ( #8154 )
...
* feat: add flow checkpoint plumbing
Signed-off-by: discord9 <discord9@163.com >
* fix: restore when fail
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* chore: clean up some test
Signed-off-by: discord9 <discord9@163.com >
* clippy
Signed-off-by: discord9 <discord9@163.com >
* refactor: move more to pr3b
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-05-26 07:24:18 +00:00
Yingwen
a25152664b
fix: qualify HistogramFold schema ( #8157 )
...
* test: add regression test for binary op on histogram_quantile (#8144 )
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: preserve column qualifiers in HistogramFold output schema (#8144 )
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2026-05-25 07:40:48 +00:00
discord9
f1ad472075
fix(mysql): infer LIMIT placeholders in prepare ( #8149 )
...
Signed-off-by: discord9 <discord9@163.com >
2026-05-21 23:40:10 +00:00
Ruihang Xia and discord9
3b3f5d628d
feat: bump datafusion to 53 ( #8107 )
...
* feat: bump datafusion to 53
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use updated datafusion-orc
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* maintain topk in part sort
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* docs: clarify part_sort range detection and Null threshold semantics
* fix limit 0
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: discord9 <discord9@163.com >
2026-05-14 08:33:31 +00:00
Ruihang Xia
aa0ff3cce7
feat: some optimistic paths for instant manipulate ( #7812 )
...
* feat: some optimistic paths for instant manipulate
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use tsid in manipulate plan, resolve_tag_columns walks whole plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tsid reuse
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test assertions
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* cap max points
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-05-14 02:36:25 +00:00
Ruihang Xia
9133d0464f
feat: pre-cast constants ( #7926 )
...
* init impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* handle no cast
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor using common-expr
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* extend matching pattern
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* more tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplification
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix zero timestamp
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: normalize sqlness partition count output
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: normalize remaining sqlness plan output
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: normalize sqlness repartition details in tql explain
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: tighten const normalization casts
* test: normalize standalone tql explain repartition output
* resolve cr comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-05-12 09:23:43 +00:00
Ruihang Xia
0d90f7407c
fix: infer time index from column meta on derived table ( #8013 )
...
* rough fix
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* reorganize
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplification
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add comment
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* enhance default by infer
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* supply comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-05-12 08:18:46 +00:00
discord9 and dennis zhuang
a282b207a6
feat: flow inc query terminal metrics transport ( #8045 )
...
* feat: OutputMetrics for inc query
Signed-off-by: discord9 <discord9@163.com >
* chore: rm unused
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* docs: explain should_collect_region_watermark_from_extensions
Signed-off-by: discord9 <discord9@163.com >
* fix: transport flow extensions via metadata
Avoid routing Flow-specific query extensions through comma-separated hints so checkpoint JSON values remain intact over Flight.
Signed-off-by: discord9 <discord9@163.com >
* refactor: per review
Signed-off-by: discord9 <discord9@163.com >
* more per review
Signed-off-by: discord9 <discord9@163.com >
* chore: more review
Signed-off-by: discord9 <discord9@163.com >
* test: compat test
Signed-off-by: discord9 <discord9@163.com >
* fix: standalone also monkey patch
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2026-05-11 11:58:51 +00:00
Ning Sun
5b47ec24ec
fix: type inference for sql rewrite ( #8052 )
...
fix: type inference for rewrited sql
2026-05-11 08:20:46 +00:00
LFC
7a285c2890
feat: concretize json type from query ( #8081 )
...
* feat: concretize json type from query
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
* add more tests
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2026-05-09 07:27:40 +00:00
LFC
160b7e720b
feat: json expr planner ( #8066 )
...
Signed-off-by: luofucong <luofc@foxmail.com >
2026-05-07 06:44:46 +00:00
discord9
760581b2a0
fix: windows windowed sort ci ( #8039 )
...
* fix: windows windowed sort ci
Signed-off-by: discord9 <discord9@163.com >
* chore
Signed-off-by: discord9 <discord9@163.com >
* c
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-04-28 02:35:44 +00:00
discord9
d2d256909f
feat(flow): parse defer on miss src table ( #7980 )
...
* feat: parse create flow with
Signed-off-by: discord9 <discord9@163.com >
* feat: validate after parse
Signed-off-by: discord9 <discord9@163.com >
* pcr
Signed-off-by: discord9 <discord9@163.com >
* chore: sqlness
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-04-27 03:02:13 +00:00
discord9
9fafd879ed
fix: window sort off by one precision TimeRange&better alias track ( #8019 )
...
* fix: window sort track alias&off by one precision TimeRange
Signed-off-by: discord9 <discord9@163.com >
* chore: more test
Signed-off-by: discord9 <discord9@163.com >
* refactor: clear helper
Signed-off-by: discord9 <discord9@163.com >
* dedup a bit
Signed-off-by: discord9 <discord9@163.com >
* feat: even more guard
Signed-off-by: discord9 <discord9@163.com >
* fix: case insensitive
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-04-24 13:56:51 +00:00
discord9
1167c36c4e
feat: add seq watermark in record batch metrics ( #8015 )
...
* feat: add seq watermark in record batch metrics
Signed-off-by: discord9 <discord9@163.com >
* per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-04-24 07:54:10 +00:00
Ning Sun
80c395ee23
refactor: update SqlPlan for more cleaner variants ( #7966 )
...
* refactor: update SqlPlan for more cleaner variants
* refactor: change how we check readonly plan
* fix: don't return schema for non-query statement
* chore: reflect review comments
* fix: federated statements
2026-04-21 11:50:47 +00:00
discord9
8123406fae
feat(flow): inc query scan bind seq ( #7879 )
...
* feat: add sequence scan boundaries and stale detection
Signed-off-by: discord9 <discord9@163.com >
* fix: only scan bind once
Signed-off-by: discord9 <discord9@163.com >
* refactor: flow snapshot decision
Signed-off-by: discord9 <discord9@163.com >
* refactor: one place modify
Signed-off-by: discord9 <discord9@163.com >
* chore: per review
Signed-off-by: discord9 <discord9@163.com >
* refactor: pre review
Signed-off-by: discord9 <discord9@163.com >
* per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-04-20 03:32:31 +00:00
Ruihang Xia
3fe8a61fad
perf: join metrics tables on the tsid key whenever possible ( #7927 )
...
* feat: prefilter flat parquet scans by primary key
* perf: skip redundant series divide repartitions
* perf: optimize tsid promql join planning
* perf: preserve tsid distribution through merge scans
* perf: remove redundant tsid join repartitions
* fix multi-field join case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Revert "feat: prefilter flat parquet scans by primary key"
This reverts commit 767c3b44c8waynestxia@gmail.com >
* isolate rule into a dedicated mod
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* more sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix filter join case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: normalize sqlness repartition input count
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: normalize sqlness partition count in promql regression
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: normalize sqlness hash partition fanout
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplification
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-04-14 19:43:58 +00:00
fys
c90e4147de
refactor: introduce the ProjectInput structure ( #7908 )
...
* refactor: introduce the ProjectInput structure
* remove unused import
* fix: cr
* fix: cr
* fix: code review
* add more unit test
* avoid clone of input.projection
2026-04-14 09:29:33 +00:00
Ruihang Xia
09b368c00a
feat: tune constants ( #7851 )
...
* feat: tune constants
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* cap output batch size
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* handle empty input
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* one more ut for cr
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-04-08 23:34:13 +00:00
discord9
b623cb1aa2
perf: no longer window sort when limit ( #7912 )
...
* perf: no longer window sort when limit
Signed-off-by: discord9 <discord9@163.com >
* test: confusing vector sqlness
Signed-off-by: discord9 <discord9@163.com >
* chore: redact sqlness
Signed-off-by: discord9 <discord9@163.com >
* chore: redact every thing
Signed-off-by: discord9 <discord9@163.com >
* REDACTED
Signed-off-by: discord9 <discord9@163.com >
* what
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-04-08 02:54:22 +00:00
Ning Sun
e14404c677
chore: update rust toolchain to 2026-03-21 ( #7849 )
...
* chore: update rust toolchain to 2026-03-21
* chore: new format
* fix: lint
* chore: resolve lint issues
* chore: remove as_millis_f64
* chore: deps up
2026-03-30 12:13:14 +00:00
discord9
d7bc5ad16b
feat: add incremental read context and scan boundaries ( #7848 )
...
* feat: add incremental read context and scan boundaries
Signed-off-by: discord9 <discord9@163.com >
* chore: per review
Signed-off-by: discord9 <discord9@163.com >
* docs: explain field
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-03-26 07:00:22 +00:00
jeremyhi
8058ce7cf2
refactor: simplify scan memory tracking ( #7827 )
...
* refactor: simplify scan memory tracking
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: make confg-docs
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: by codex review comment
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* feat: track_with_policy
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: minor change
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: mem granularity mb to kb
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: by review comment
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: by scan_memory_on_exhausted comment
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: by review comment
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* chore: typo
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
2026-03-26 03:25:50 +00:00
Ning Sun
73b48b14c1
feat: update postgres ParameterDescription size limit ( #7861 )
...
* feat: update postgres ParameterDescription size limit
* chore: don't log io error
2026-03-26 03:10:39 +00:00
Ruihang Xia
59dd418629
feat: simplify nested aggr inside count query ( #7859 )
...
* as optimizer rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* dump changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* perf: tighten count-count optimizer rewrite
* extend inner op set
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* simplify and more coverage
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove prom-non-null
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* preserve value column through pruning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* more sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* rename
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* enforce is not null before inner aggr
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* finalize
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-03-26 00:08:38 +00:00
discord9
30e895abbe
fix: prom cast to f64 ( #7840 )
...
* fix: cast to f64
Signed-off-by: discord9 <discord9@163.com >
* test: div case
Signed-off-by: discord9 <discord9@163.com >
* test: int test
Signed-off-by: discord9 <discord9@163.com >
* chore: sqlness update
Signed-off-by: discord9 <discord9@163.com >
* chore: test
Signed-off-by: discord9 <discord9@163.com >
* chore: update test
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-03-24 06:24:52 +00:00
Ning Sun
d14817bfa6
fix: resolve optimization issue for extended query ( #7824 )
...
* fix: resolve optimization issue for extended query
* fix: type cast from subquery
* chore: update error information in sqlness
* chore: switch to released pgwire
* refactor: remove optimize function completely
* chore: add more tests
* test: attempt to fix the fuzz issue
* fix: try to resolve the test issue
2026-03-20 03:58:39 +00:00
discord9
922f9cb3d6
feat: use dyn filter ( #7545 )
...
* parent b2074e3863discord9@163.com > 1767869295 +0800
committer discord9 <discord9@163.com > 1772529023 +0800
feat: use dyn filter
Signed-off-by: discord9 <discord9@163.com >
not supported
Signed-off-by: discord9 <discord9@163.com >
refactor: use make_mut instead
Signed-off-by: discord9 <discord9@163.com >
refactor: rm need to clone stream ctx
Signed-off-by: discord9 <discord9@163.com >
r
Signed-off-by: discord9 <discord9@163.com >
pcr
Signed-off-by: discord9 <discord9@163.com >
test: wait for datafusion update
Signed-off-by: discord9 <discord9@163.com >
refactor: use arc swap for dyn filters
Signed-off-by: discord9 <discord9@163.com >
* test: update sqlness
Signed-off-by: discord9 <discord9@163.com >
* chore: comment out sqlness
Signed-off-by: discord9 <discord9@163.com >
* test: update sqlness
Signed-off-by: discord9 <discord9@163.com >
* test: sqlness fix
Signed-off-by: discord9 <discord9@163.com >
* refactor: predicate without option
Signed-off-by: discord9 <discord9@163.com >
* feat: print dyn filters& more tests
Signed-off-by: discord9 <discord9@163.com >
* test: sqlness vector result update
Signed-off-by: discord9 <discord9@163.com >
* chore: log
Signed-off-by: discord9 <discord9@163.com >
* test: properly redact
Signed-off-by: discord9 <discord9@163.com >
* test: better data dist for non empty dyn filter
Signed-off-by: discord9 <discord9@163.com >
* test: properly redacted
Signed-off-by: discord9 <discord9@163.com >
* chore: per review
Signed-off-by: discord9 <discord9@163.com >
* properly redact
Signed-off-by: discord9 <discord9@163.com >
* docs: explain why not do it
Signed-off-by: discord9 <discord9@163.com >
* chore: rename update to add as its more proper
Signed-off-by: discord9 <discord9@163.com >
* chore: rm no need clone
Signed-off-by: discord9 <discord9@163.com >
* docs: per review
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-03-11 03:06:24 +00:00
Ning Sun and Ruihang Xia
9b1288f66a
feat: add function rewrite rule for json_get with cast ( #7631 )
...
* feat: initial function rewriter for json_get
* feat: make sure rewrite rule is applied
* feat: keep analyzer's default rules
* feat: implement rewriter for arrow_cast
* test: add unit test for tht rewriter
* chore: format
* refactor: extract some more functions
* Apply suggestion from @waynexia
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2026-03-09 02:39:48 +00:00
dennis zhuang
Ruihang Xia
LFC
discord9
fys