shuiyisong
20dde2601f
feat: enable native histogram ingestion by default ( #9301 )
...
* feat: enable native histogram ingestion by default
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: add comments
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: test
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
2026-09-23 13:03:51 +00:00
dennis zhuang
f3eb8e6c72
test: cut integration test time and make the storage matrix meaningful ( #9308 )
...
* test: cut integration test time and make the storage matrix meaningful
tests-integration is ~85% of workspace test CPU, and 81% of that is the
S3/S3WithCache variants of the HTTP and gRPC suites. Those suites do not
touch the object store: of the 70 matrix HTTP tests only one flushed and
read back an SST, so the matrix was paying real AWS round trips to
re-prove protocol parsing.
- Point the PR CI object-store matrix at the MinIO already started by
tests-integration/fixtures. Three GT_S3_* consumers did not read
GT_S3_ENDPOINT_URL and would have hit real AWS with MinIO credentials;
they now do.
- Add a nightly Linux job against real AWS S3, and pass GT_S3_* into the
release integration-test container. The release previously ran every
remote-backend case as a skip and only exercised the file backend.
- Give each S3WithCache test its own read cache directory. They shared
/tmp/greptimedb_cache, which the datanode wipes on startup, so a
starting test deleted the read cache of a running one.
- Add flush -> read-back assertions to the tests whose columns have a
non-trivial SST representation: JSON/JSON2 columns, native histograms,
metric-engine logical tables, and tables carrying fulltext or skipping
indexes whose puffin files only exist after a flush.
- Move eight tests that create no table out of the storage matrix.
- Make the event recorder flush interval a constructor parameter and
shorten it in the event tests, which otherwise wait a 5s window per DDL
they assert on. It is skipped by serde and never reaches config files.
- Drop duplicates: test_grpc_zstd_compression was a verbatim copy of
test_grpc_message_size_ok and is now rewritten to assert the negotiated
grpc-encoding; test_execute_copy_to_{s3,oss,gcs,azblob} were strict
prefixes of their copy_from siblings; two standalone/distributed event
test pairs shared one assertion body.
- Fix and un-ignore stddev_by_label. stddev_pop merges partial aggregates
in a parallelism-dependent order, so its last digits are unstable; the
test now compares values with a tolerance.
- Rebase the jaeger v1 fixture on the current instant. It carries
ttl=7d with 2025 timestamps, so its rows were only readable as long as
they stayed in the memtable.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* test: address review — wire nightly real-S3 job into check-status, keep the short event interval
The nightly `check-status` job did not depend on the new real-S3 job, so a
failure there would not have reached the status or Slack notification.
In database_ddl_event the short interval was set by a first
`with_event_recorder_options` call and then overwritten by the pre-existing
one, which carries `..Default::default()`. Merged into a single call.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
2026-09-23 01:52:52 +00:00
shuiyisong and Lei, HUANG
66d38e8e1c
feat: add database ingestion admission through metering ( #9239 )
...
* feat: add `ingest_rows_rate_limit` database option
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* feat: add `InsertLimitInterceptor` hook to `Inserter`
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* fix: attribute insert limit checks to the target table's database
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
* refactor: unify write admission through metering
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: enforce write admission for pending row batches
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* feat: distinguish internal requests for ingestion metering
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: admit split ingestion requests once per database
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: OpenTSDB throws error reason
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* chore: use meter crate main rev
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* fix: exclude database ingest rate limit from table options
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
* refactor: reserve channel 255 for internal requests
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com >
Signed-off-by: shuiyisong <xixing.sys@gmail.com >
Co-authored-by: Lei, HUANG <ratuthomm@gmail.com >
2026-09-21 07:27:22 +00:00
discord9 and Ning Sun
94d7e2c7fc
feat!: upgrade DataFusion to 55 ( #8555 )
...
* feat!: upgrade DataFusion dependencies to 55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* refactor: migrate DataFusion 55 APIs
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: preserve table function planning behavior
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: preserve PostgreSQL query compatibility
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: preserve distributed execution plan behavior
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: cover DataFusion 55 behavior regressions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: update DataFusion 55 SQLness expectations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: complete DataFusion 55 test API migration
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: address DataFusion 55 CI regressions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: address remaining DataFusion 55 regressions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: adapt latest base code to DataFusion 55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: normalize environment-specific DataFusion 55 plans
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: align final DataFusion 55 expectations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: isolate DataFusion 55 regression cases
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: preserve empty result schema in timestamp widening
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: preserve JSON source column order
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* chore: use released DataFusion 55 integrations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: adapt latest execution plan mock to DataFusion 55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: pin DataFusion recursive schema and date repairs
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix(promql): align dictionary temporality match keys
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: retain Greptime DataFusion fork behaviors on version 55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: restore ordinary function error expectations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: refresh distributed count compatibility plan
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix(query): adapt last-row cast hint to DataFusion 55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: refresh instant last-row empty results for Arrow 59
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* style: simplify DataFusion expression visitor imports
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: restore sorting and PostgreSQL column-order assertions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix(function): restore primitive numeric coercion signatures
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* refactor(function): share geo integer signature types
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: cover timestamp widening overflow boundaries
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: fix decimal coercion regression imports
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix(function): preserve scalar count_hash NULL state semantics
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: simplify decimal clamp case type inference
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: retain historical count_hash wrapper result
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: restore timestamp widening equality and IN pruning
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: carry upstream aggregate dynamic filter correctness fix
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: carry upstream null and predicate simplification fixes
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: restore baseline JSON ordering expectations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: restore histogram JSON ordering expectations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: refresh empty PromQL range result schemas
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: align native timestamp plan with DF55 decimal display
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: refresh native timestamp SQLness results for DF55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: regenerate NULL sample empty result headers for DF55
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: use DF55 child replacement API in timestamp regressions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: expose pushed scan dynamic filters to DF55 producers
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: encode string-backed PostgreSQL OID aliases in binary results
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: verify REGPROC binary and text over PostgreSQL protocol
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: register real PostgreSQL catalogs in server fixtures
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: complete DF55 expression inventories for custom query plans
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: correct RangeSelect expression fixture and column identities
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* ci: wait for Kafka WAL helper deployment rollout
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* test: update custom storage empty result headers
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: require exact row counts in scan statistics
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
* fix: suppress deprecated partition_statistics warning in test
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com >
Co-authored-by: Ning Sun <sunng@protonmail.com >
2026-09-15 11:42:38 +00:00
Weny Xu
a673e084b2
test: cover request-level insert WAL skipping end to end ( #9093 )
...
* test: cover request-level WAL skipping end to end
Signed-off-by: WenyXu <wenymedia@gmail.com >
* test: cover session WAL policy and COPY recovery in sqlness
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix(test): isolate Mito test feature in dev dependencies
Signed-off-by: WenyXu <wenymedia@gmail.com >
* test: parameterize WAL protocol cases and make setup explicit
Signed-off-by: WenyXu <wenymedia@gmail.com >
* test: cover skip-WAL hints across streaming messages
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
2026-09-11 07:48:57 +00:00
LFC
529f046110
refactor: json2 v2 storage layout ( #8979 )
...
* refactor: json2 v2 storage layout
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com >
* rethinking when "needs_remainder"
Signed-off-by: luofucong <luofc@foxmail.com >
* restore "ReadColumns"
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2026-09-01 13:39:56 +00:00
Yingwen
78084a9d44
feat: add admin function to discard unflushed data ( #8768 )
...
* feat: add admin function to discard unflushed data
Signed-off-by: evenyag <realevenyag@gmail.com >
* test: cover discarding unflushed data by table
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore: fix license header
Signed-off-by: evenyag <realevenyag@gmail.com >
* fix: reject discarding logical metric table data
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: defer table name formatting in error paths
Signed-off-by: evenyag <realevenyag@gmail.com >
* chore(deps): update greptime-proto revision
Signed-off-by: evenyag <realevenyag@gmail.com >
* refactor: rename discard unflushed admin function
Signed-off-by: evenyag <realevenyag@gmail.com >
---------
Signed-off-by: evenyag <realevenyag@gmail.com >
2026-08-10 12:19:27 +00:00
jeremyhi
448f973593
fix: sandbox SQL local filesystem access ( #8708 )
...
* fix: sandbox SQL local filesystem access
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: address local file sandbox review findings
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: support Windows local copy paths
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: improve sandbox path errors
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* refactor: simplify local path error context
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* perf: stream secure filesystem listings
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* style: derive local file access default
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: improve local file access errors
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: address local file access review findings
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* test: simplify local file access coverage
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: harden sandboxed local file backends
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: reject directory copy targets before creation
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
* fix: avoid implicit string clone in file table listing
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com >
2026-07-31 13:23:15 +00:00
discord9
b30765e5e9
fix: preserve distributed topk merge ordering ( #8432 )
...
* fix: preserve distributed topk for scalar latest
Signed-off-by: discord9 <discord9@163.com >
* fix: preserve scalar latest ordering across merge scan
Signed-off-by: discord9 <discord9@163.com >
* fix: carry merge scan ordering from planner rewrite
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan ordering metadata
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan partition ordering gate
Signed-off-by: discord9 <discord9@163.com >
* fix: require adjacent merge sort for ordering metadata
Signed-off-by: discord9 <discord9@163.com >
* test: update ordering sqlness plans
Signed-off-by: discord9 <discord9@163.com >
* fix: preserve distributed topk merge ordering
Signed-off-by: discord9 <discord9@163.com >
* test: cover latest per series queries
Signed-off-by: discord9 <discord9@163.com >
* test: update distributed merge sort sqlness
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan ordering partition gate
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge scan over-partition ordering
Signed-off-by: discord9 <discord9@163.com >
* test: cover distributed latest with low parallelism
Signed-off-by: discord9 <discord9@163.com >
* fix: keep distributed merge sort opaque
Signed-off-by: discord9 <discord9@163.com >
* test: assert distributed scalar latest merge sort
Signed-off-by: discord9 <discord9@163.com >
* test: clarify merge scan ordering helper
Signed-off-by: discord9 <discord9@163.com >
* fix: complete merge sort exec delegation
Signed-off-by: discord9 <discord9@163.com >
* docs: explain merge sort limit pushdown
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge sort optimizer opacity
Signed-off-by: discord9 <discord9@163.com >
* docs: explain merge sort optimizer hooks
Signed-off-by: discord9 <discord9@163.com >
* test: cover merge sort optimizer hooks
Signed-off-by: discord9 <discord9@163.com >
* fix: preserve merge sort child topk
Signed-off-by: discord9 <discord9@163.com >
* test: update order by topk plan
Signed-off-by: discord9 <discord9@163.com >
* fix: recognize merge sort global fetch
Signed-off-by: discord9 <discord9@163.com >
---------
Signed-off-by: discord9 <discord9@163.com >
2026-07-14 12:53:07 +00:00
Yingwen
758664eefa
fix: Use prepared file locations for CSV strict headers integration test ( #8493 )
...
fix: normalize Windows COPY test paths
Signed-off-by: Codex <codex@openai.com >
2026-07-13 10:19:57 +00:00
QuakeWang
4e14ac8676
feat: add strict CSV header validation ( #8426 )
...
* feat: add strict CSV header validation
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
* style: simplify strict header parse error
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
---------
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
2026-07-10 02:15:08 +00:00
fys
3bf8b9722d
feat(json2): validate append mode for tables with JSON2 columns ( #8434 )
...
* feat: validate append mode for JSON2 tables
* fix: cr by codex
* fix: unit test
* fix: validate JSON2 append_mode for all ALTER TABLE operations
2026-07-08 03:33:36 +00:00
fys
aa563628fb
feat(json2): type hint ( #8247 )
...
* feat(json2): type hint
* test(datatypes): add JsonSettings serde round-trip test
* minor refactor
This reverts commit 7ff5a5249a09be5396536284fe822b5761ef4e6a.
* fix: code review
2026-06-30 07:37:31 +00:00
QuakeWang
6ed67167bb
feat: support headerless CSV copy from ( #8233 )
...
* feat: support headerless CSV copy from
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
* fix: update csv copy sqlness result
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
* test: cover headerless CSV copy from
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
* test: cover headerless CSV column count mismatch
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
---------
Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com >
2026-06-04 12:04:15 +00:00
Weny Xu
bb1de2a744
test: move CSV skip bad records coverage to integration ( #8237 )
...
Signed-off-by: WenyXu <wenymedia@gmail.com >
2026-06-04 07:58:58 +00:00
LFC
209880b991
feat: json2 flush ( #8011 )
...
Signed-off-by: luofucong <luofc@foxmail.com >
2026-04-23 03:03:37 +00:00
Weny Xu
b17947b5d8
feat: add COPY support for GCS/Azblob with OSS/GCS/Azblob integration coverage ( #7743 )
...
* feat: add gcs and azblob backends to copy datasource
Signed-off-by: WenyXu <wenymedia@gmail.com >
* test: add copy to/from oss integration coverage
Signed-off-by: WenyXu <wenymedia@gmail.com >
* test: add copy to/from gcs/azure integration coverage
Signed-off-by: WenyXu <wenymedia@gmail.com >
* refactor: use existing layers
Signed-off-by: WenyXu <wenymedia@gmail.com >
* feat: hide `credential_path` in cli
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: remove logs
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: apply suggestions
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
2026-03-04 04:21:21 +00:00
Ruihang Xia
38b5df574c
refactor: SQL and gRPC server handlers ( #7637 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2026-01-30 07:19:45 +00:00
dennis zhuang and Copilot
1f91422bae
feat!: improve mysql/pg compatibility ( #7315 )
...
* feat(mysql): add SHOW WARNINGS support and return warnings for unsupported SET variables
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* feat(function): add MySQL IF() function and PostgreSQL description functions for connector compatibility
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: show tables for mysql
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: partitions table in information_schema and add starrocks external catalog compatibility
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* refactor: async udf
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: set warnings
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* feat: impl pg_my_temp_schema and make description functions simple
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* test: add test for issue 7313
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* feat: apply suggestions
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: partition_expression and partition_description
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: test
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: unit tests
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: saerch_path only works for pg
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* feat: improve warnings processing
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* fix: warnings while writing affected rows and refactor
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* chore: improve ShobjDescriptionFunction signature
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
* refactor: array_to_boolean
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-12-01 20:41:14 +00:00
LFC
4a7c16586b
refactor: remove Vectors from RecordBatch completely ( #7184 )
...
* refactor: remove `Vector`s from `RecordBatch` completely
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2025-11-21 08:53:35 +00:00
LFC
b53a0b86fb
feat: create table with new json datatype ( #7128 )
...
* feat: create table with new json datatype
Signed-off-by: luofucong <luofc@foxmail.com >
* resolve PR comments
Signed-off-by: luofucong <luofc@foxmail.com >
---------
Signed-off-by: luofucong <luofc@foxmail.com >
2025-10-24 02:16:49 +00:00
Ruihang Xia
c9377e7c5a
build: bump rust edition to 2024 ( #6920 )
...
* bump edition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* gen keyword
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lifetime and env var
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* one more gen fix
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* lifetime of temporaries in tail expressions
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* format again
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clippy nested if
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clippy let and return
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2025-09-08 02:37:18 +00:00
Weny Xu
0e962844ac
fix: fix incorrect timestamp precision in information_schema.tables ( #6872 )
...
* fix: fix incorrect timestamp precision in information_schema.tables
Signed-off-by: WenyXu <wenymedia@gmail.com >
* fix: fix unit test
Signed-off-by: WenyXu <wenymedia@gmail.com >
---------
Signed-off-by: WenyXu <wenymedia@gmail.com >
2025-09-01 12:20:23 +00:00
Arshdeep
2e571e351f
fix: add map datatype conversion in copy_table_from ( #6185 ) ( #6422 )
...
Signed-off-by: Arshdeep54 <balarsh535@gmail.com >
2025-07-28 03:53:10 +00:00
Zhenchi
2298227e0c
refactor: refactor partition mod to use PartitionExpr instead of PartitionDef ( #6554 )
...
* refactor: refactor partition mod to use PartitionExpr instead of PartitionDef
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix snafu
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Puts expression into PbPartition
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix compile
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* update proto
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* add serde test
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* add serde test
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2025-07-23 03:51:28 +00:00
Lei, HUANG
69870e2762
fix(mito): use 1day as default time partition duration ( #6202 )
...
* fix unit tests
* fix: sqlness
* fix/default-time-window:
## Add Helper Functions and Enhance Compaction Tests
- **Refactor Compaction Logic**: Introduced helper functions `flush` and `compact` in `compaction_test.rs` to streamline compaction operations.
- **Enhance Compaction Tests**: Added a new test `test_infer_compaction_time_window` in `compaction_test.rs` to verify compaction time window inference.
- **Testing Improvements**: Added `#[cfg(test)]` attribute to `new_multi_partitions` in `time_partition.rs` to ensure it's only included in test builds.
* fix/default-time-window:
- **Refactor `TimePartition` Struct**: Removed unnecessary comments regarding `time_range` in `time_partition.rs`.
- **Enhance `TimePartitions` Functionality**: Added a method `part_duration_or_default` to provide a default partition duration in `time_partition.rs`.
- **Update SQL Test Cases**: Modified SQL operations and expected results in `scan_big_varchar.result` and `scan_big_varchar.sql` to reflect changes in data manipulation logic.
* fix/default-time-window:
### Update Time Partition Default Duration
- **Refactor Default Duration**: Introduced `INITIAL_TIME_WINDOW` constant to define the default time window duration as `Duration::from_days(1)`. This change replaces multiple instances of the hardcoded default duration across the `time_partition.rs` file.
- **Files Affected**: `time_partition.rs`
* fix/default-time-window:
## Update Partition Duration Handling
- **`time_partition.rs`**: Refactored `part_duration` to be non-optional, removing `Option` wrapper. Updated logic to use `unwrap_or` with `INITIAL_TIME_WINDOW` where necessary. Adjusted related methods and tests to accommodate this change.
- **`version.rs` (memtable and region)**: Updated handling of `part_duration` to align with changes in `time_partition.rs`, ensuring consistent use of non-optional `Duration`.
* fix/default-time-window:
### Improve Error Context in `time_partition.rs`
- Enhanced error context message in `time_partition.rs` to provide clearer information on partition time range issues, including bucket size details.
Signed-off-by: Lei, HUANG <lhuang@greptime.com >
---------
Signed-off-by: Lei, HUANG <lhuang@greptime.com >
2025-06-08 16:20:26 +00:00
Lin Yihai
2cbf51d0be
refactor!: Remove Value::DateTime and ValueRef::DateTime. ( #5616 )
...
* refactor: Remove Value::DateTime and ValueRef::DateTime
* fix: don't panic if arrow cast field.
* fix: map `ColumnDataType::Datetime` to `ConcreteDataType::timestamp_microsecond_datatype`
* fix: Map `ValueData::DatetimeValue` correctly.
* refactor: Replace `datetime` with `timestamp_micro_second`
2025-03-11 07:03:27 +00:00
Niwaka
50583815de
feat: support alter add multiple columns ( #5262 )
...
* feat: support alter add multiple columns
* fix: address review
* chore: add column format
2025-01-10 06:14:17 +00:00
JohnsonLee
a8477e4142
fix: table resolving logic related to pg_catalog ( #4580 )
...
* fix: table resolving logic related to pg_catalog
refer to
https://github.com/GreptimeTeam/greptimedb/issues/3560#issuecomment-2287794348
and #4543
* refactor: remove CatalogProtocol type
* fix: sqlness
* fix: forbid create database pg_catalog with mysql client
* refactor: use QueryContext as arguments rather than Channel
* refactor: pass None as default behaviour in information_schema
* test: fix test
2024-09-09 00:47:59 +00:00
JohnsonLee and johnsonlee
072d7c2022
feat: introduce 'pg_catalog.pg_type' ( #4332 )
...
* WIP: pg_catalog
* refactor: move memory_table to crate public level to reuse it in pgcatalog
* refactor: new system_schema mod to manage implementation of information_schema and pg_catalog
* feat: pg_catalog.pg_type
* fix: remove unused code to avoid warning
* test: add pg_catalog sqlness test
* feat: pg_catalog_cache in system_catalog
* fix: integration test
* test: rollback unit test
* refactor: mix pg_catalog table_id with old ones
* fix: add todo information
* tests: rerun sqlness
---------
Co-authored-by: johnsonlee <johnsonlee@localhost.localdomain >
2024-07-15 17:41:08 +00:00
Ning Sun
81308b9063
fix: error on show databases in non-default catalog ( #4316 )
2024-07-08 16:28:00 +00:00
Ning Sun and dennis zhuang
11cf9c827e
feat: dbeaver mysql compatibility, use statement and information_schema.tables ( #4218 )
...
* feat: add more placeholder field in information_schema.tables
* feat: make schema modifiable for use statement
* chore: add todo items
* fix: resolve lint issues after data type changes
* chore: update sqlness results
* refactor: patch for select database is no longer needed
* test: align tests and data types
* Apply suggestions from code review
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* fix: use canonicalize_identifier for database name
* feat: add all columns for information_schema.tables
* test: remove vairables from sqlness results
* feat: add to_string impl for table options
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-07-03 01:31:13 +00:00
Kelvin Wu and dennis zhuang
154f561da1
refactor: passing QueryContext to RegionServer ( #3829 )
...
* refactor: passing QueryContext to RegionServer
* refactor: change the return type of build() in QueryContextBuilder
* fix: update greptime-proto reference
* chore: apply suggestion
* chore: revert the last commit
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-05-07 15:10:59 +08:00
Ruihang Xia
530353785c
refactor: remove re-export from logging ( #3865 )
...
* refactor: remove re-export from logging
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix merge problem
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* run formatter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-05-06 13:26:01 +00:00
tison
bba3108e0d
refactor!: unify sql options into OptionMap ( #3792 )
...
* unify sql options into OptionMap
Signed-off-by: tison <wander4096@gmail.com >
* fixup
Signed-off-by: tison <wander4096@gmail.com >
* Update src/sql/src/util.rs
* drop legacy regions option
Signed-off-by: tison <wander4096@gmail.com >
* fixup
Signed-off-by: tison <wander4096@gmail.com >
* fixup
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-04-25 04:06:05 +00:00
314f2704d4
build(deps): update datafusion to latest and arrow to 51.0 ( #3661 )
...
* chore: update datafusion
* update sqlness case of time.sql
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: adjust range query partition
* fix: hisogram incorrect result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: ignore filter pushdown temporarily
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update limit sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: histogram with wrong distribution
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: update negative ordinal sqlness case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* feat: bump df to cd7a00b
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve conflicts
* ignore test_range_filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix promql exec panic
* fix "select count(*)" exec error
* re-enable the "test_range_filter" test since the filter push down seems not necessary to be removed
* fix: range query schema error
* update sqlness results
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve conflicts
* update datafusion, again
* fix pyo3 compile error, and update some sqlness results
* update decimal sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: promql literal
* fix udaf tests
* fix filter pushdown sqlness tests
* fix?: test_cast
* fix: rspy test fail due to datafusion `sin` signature change
* rebase main to see if there are any failed tests
* debug ci
* debug ci
* debug ci
* enforce input partition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* debug ci
* fix ci
* fix ci
* debug ci
* debug ci
* debug ci
* fix sqlness
* feat: do not return error while creating a filter
* chore: remove array from error
* chore: replace todo with unimplemented
* Update src/flow/clippy.toml
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: WUJingdi <taylor-lagrange@qq.com >
Co-authored-by: discord9 <discord9@163.com >
Co-authored-by: evenyag <realevenyag@gmail.com >
Co-authored-by: tison <wander4096@gmail.com >
2024-04-18 12:07:18 +00:00
87e0189e58
fix!: columns table in information_schema misses some columns ( #3639 )
...
* fix: columns table in information_schema misses some columns
* fix: test_information_schema_dot_columns
* fix: fuzz test
* feat: adds srs_id and refactor some columns with constant vector
* fix: test_information_schema_dot_columns
* chore: update comment
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* build(deps): bump h2 from 0.3.24 to 0.3.26 (#3642 )
Bumps [h2](https://github.com/hyperium/h2 ) from 0.3.24 to 0.3.26.
- [Release notes](https://github.com/hyperium/h2/releases )
- [Changelog](https://github.com/hyperium/h2/blob/v0.3.26/CHANGELOG.md )
- [Commits](https://github.com/hyperium/h2/compare/v0.3.24...v0.3.26 )
---
updated-dependencies:
- dependency-name: h2
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump whoami from 1.4.1 to 1.5.1 (#3643 )
Bumps [whoami](https://github.com/ardaku/whoami ) from 1.4.1 to 1.5.1.
- [Changelog](https://github.com/ardaku/whoami/blob/v1/CHANGELOG.md )
- [Commits](https://github.com/ardaku/whoami/compare/v1.4.1...v1.5.1 )
---
updated-dependencies:
- dependency-name: whoami
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat: adding victoriametrics remote write (#3641 )
* feat: adding victoria metrics remote write
* test: add e2e tests for prom and vm remote writes
* fix: construct correct pk list with pre-existing pk (#3614 )
* fix: construct correct pk list with pre-existing pk
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update UT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* test(sqlness): release databases after tests (#3648 )
* refactor: rename Greptime_Type to Greptime_type
---------
Signed-off-by: dependabot[bot] <support@github.com >
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ning Sun <sunng@protonmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-04-08 03:20:49 +00:00
tison
cafb4708ce
refactor: validate constraints eagerly ( #3472 )
...
* chore: validate constraints eagerly
Signed-off-by: tison <wander4096@gmail.com >
* use timestamp column
Signed-off-by: tison <wander4096@gmail.com >
* fixup
Signed-off-by: tison <wander4096@gmail.com >
* lint
Signed-off-by: tison <wander4096@gmail.com >
* compile
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-03-12 13:09:34 +00:00
Yingwen
9aa8f756ab
fix: allow passing extra table options ( #3484 )
...
* fix: do not check options in parser
* test: fix tests
* test: fix sqlness
* test: add sqlness test
* chore: log options
* chore: must specify compaction type
* feat: validate option key
* feat: add option key validation back
2024-03-12 07:03:52 +00:00
shuiyisong and Ruihang Xia
0bb949787c
refactor: introduce new Output with OutputMeta ( #3466 )
...
* refactor: introduce new output struct
* chore: add helper function
* chore: update comment
* chore: update commit
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: rename according to cr
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-03-11 02:24:09 +00:00
Ruihang Xia and WenyXu
020635063c
feat: implement multi-dim partition rule ( #3409 )
...
* generate expr rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* implement show create for new partition rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* implement row spliter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: fix failed tests
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: fix lint issues
Signed-off-by: WenyXu <wenymedia@gmail.com >
* chore: ignore tests for deprecated partition rule
* chore: remove unused partition rule tests setup
* test(sqlness): add basic partition tests
* test(multi_dim): add basic find region test
* address CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Signed-off-by: WenyXu <wenymedia@gmail.com >
Co-authored-by: WenyXu <wenymedia@gmail.com >
2024-03-05 11:39:15 +00:00
dennis zhuang
e6819412c5
refactor: show tables and show databases ( #3423 )
...
* refactor: show tables and show databases
* chore: clean code
2024-03-04 06:15:17 +00:00
Ruihang Xia
3544c9334c
feat!: new partition grammar - parser part ( #3347 )
...
* parser part
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test in sql
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* comment out and ignore some logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update region migration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* temporary disable region migration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* allow dead code
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-27 07:20:16 +00:00
shuiyisong
bf5e1905cd
refactor: bring metrics to http output ( #3247 )
...
* refactor: bring metrics to http output
* chore: remove unwrap
* chore: make walk plan accumulate
* chore: change field name and comment
* chore: add metrics to http resp header
* chore: move PrometheusJsonResponse to a separate file and impl IntoResponse
* chore: put metrics in prometheus resp header too
2024-02-20 03:25:18 +00:00
LFC
51a3fbc7bf
refactor: change how frontend grpc services are orchestrated ( #3134 )
2024-01-11 02:26:44 +00:00
dennis zhuang
11ae85b1cd
feat: adds information_schema.schemata ( #3051 )
...
* feat: improve information_schema.columns
* feat: adds information_schema.schemata
* fix: instance test
* fix: comment
2023-12-29 09:22:31 +00:00
Ning Sun
43f01cc594
feat: add a default internal schema ( #2974 )
2023-12-22 06:25:19 +00:00
dennis zhuang
262a79a170
feat: adds some tables to information_schema ( #2935 )
...
* feat: adds engines table to information_schema
* feat: adds COLUMN_PRIVILEGES and COLUMN_STATISTICS
* feat: refactor memory tables
* chore: rename memory_tables
* test: adds unit tests
* chore: format
* chore: style
* fix: by cr comments
* refactor: tables
2023-12-18 06:10:22 +00:00
Niwaka and dennis zhuang
cfe3a2c55e
feat!: support table ddl for custom storage ( #2733 )
...
* feat: support table ddl for custom_storage
* refactor: rename extract_variant_name to name
* chore: add blank
* chore: keep compatible
* feat: rename custom_stores to providers
* chore: rename
* chore: config
* refactor: add should_retry in client Error
* fix: test fail
* chore: remove unused options
* chore: remove unused import
* chore: remove the blanks.
* chore: revert
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-12-06 15:59:01 +00:00
Ruihang Xia
5d8b0e8154
chore: bump dependencies ( #2659 )
...
* fix all hard error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix nextest
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* trivial changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix order by
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix sql keyword and data type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix range exec's input partitioning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix cover input type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix explain analyze
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update distributed mode sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix lints
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update locks
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* downgrade dlv-list
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-10-26 12:24:14 +00:00
shuiyisong
dennis zhuang
discord9andNing Sun
Weny Xu
LFC
Yingwen
jeremyhi
QuakeWang
fys
Ruihang Xia
Arshdeep
Zhenchi
Lei, HUANG
Lin Yihai
Niwaka
JohnsonLee
Kelvin Wuanddennis zhuang
tison
