rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-10-15 16:32:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,367 Commits 274 Branches 279 Tags
stack/pr9412-capability
Commit Graph
39 Commits
Author SHA1 Message Date
jeremyhi f9ab3c296b feat(log-store): skip corrupted object store WAL segments per region (#9425) 
* feat(log-store): skip corrupted object store WAL segments per region

A segment that does not decode, because its checksum does not match or
its content disagrees with its footer entry, is fetched once more. If it
still does not decode, the read follows `on_corrupted_segment`: `skip`
records a WAL hole for the region with the object key, sequence and the
footer's entry id range, increments
`greptime_logstore_object_store_wal_skipped_segments_total`, logs a
warning and continues with the next object; `fail` fails the read. A
failed fetch fails the read in both modes. Holes are kept in memory per
region and can be listed from the store.

The standalone test builder allocates WAL options through
`build_standalone_wal_provider`, and integration tests run a standalone
instance on the object store WAL across restarts, on S3 when the
`GT_S3_*` variables are set. The standalone shutdown runs every step and
returns the first error.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test(cmd): cover the best-effort standalone stop

Build a standalone instance whose leader services fail to stop and whose
datanode servers fail afterwards, and check that `Instance::stop` still
stops the object store WAL and returns the leader services error.

Rename `skip_segment` to `record_skipped_segment` and the file store
integration test to `test_standalone_object_store_wal_survives_restarts_on_file`.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-10-08 06:33:40 +00:00
Yingwen 3c2aac0a55 feat: add manual series index reconciliation (#9323) 
* feat(mito): support manual series index reconciliation

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat(storage): route series index build requests

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat(admin): add BUILD_SERIES_INDEX

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: specify compaction type in series index fixtures

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: correct series index SQL fixtures and error assertions

Signed-off-by: evenyag <realevenyag@gmail.com>

* test(compat): preserve legacy index rebuild across upgrades

Signed-off-by: evenyag <realevenyag@gmail.com>

* test(sql): cover series index admin validation

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor(mito): bound series index maintenance queue

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: remove series index how-to guide

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: remove index build upgrade compatibility case

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: address series index reconciliation review feedback

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: bound manual series index reconciliation admission

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: update greptime-proto to merged index build options

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-09-24 09:39:21 +00:00
dennis zhuang f3eb8e6c72 test: cut integration test time and make the storage matrix meaningful (#9308) 
* test: cut integration test time and make the storage matrix meaningful

tests-integration is ~85% of workspace test CPU, and 81% of that is the
S3/S3WithCache variants of the HTTP and gRPC suites. Those suites do not
touch the object store: of the 70 matrix HTTP tests only one flushed and
read back an SST, so the matrix was paying real AWS round trips to
re-prove protocol parsing.

- Point the PR CI object-store matrix at the MinIO already started by
  tests-integration/fixtures. Three GT_S3_* consumers did not read
  GT_S3_ENDPOINT_URL and would have hit real AWS with MinIO credentials;
  they now do.
- Add a nightly Linux job against real AWS S3, and pass GT_S3_* into the
  release integration-test container. The release previously ran every
  remote-backend case as a skip and only exercised the file backend.
- Give each S3WithCache test its own read cache directory. They shared
  /tmp/greptimedb_cache, which the datanode wipes on startup, so a
  starting test deleted the read cache of a running one.
- Add flush -> read-back assertions to the tests whose columns have a
  non-trivial SST representation: JSON/JSON2 columns, native histograms,
  metric-engine logical tables, and tables carrying fulltext or skipping
  indexes whose puffin files only exist after a flush.
- Move eight tests that create no table out of the storage matrix.
- Make the event recorder flush interval a constructor parameter and
  shorten it in the event tests, which otherwise wait a 5s window per DDL
  they assert on. It is skipped by serde and never reaches config files.
- Drop duplicates: test_grpc_zstd_compression was a verbatim copy of
  test_grpc_message_size_ok and is now rewritten to assert the negotiated
  grpc-encoding; test_execute_copy_to_{s3,oss,gcs,azblob} were strict
  prefixes of their copy_from siblings; two standalone/distributed event
  test pairs shared one assertion body.
- Fix and un-ignore stddev_by_label. stddev_pop merges partial aggregates
  in a parallelism-dependent order, so its last digits are unstable; the
  test now compares values with a tolerance.
- Rebase the jaeger v1 fixture on the current instant. It carries
  ttl=7d with 2025 timestamps, so its rows were only readable as long as
  they stayed in the memtable.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: address review — wire nightly real-S3 job into check-status, keep the short event interval

The nightly `check-status` job did not depend on the new real-S3 job, so a
failure there would not have reached the status or Slack notification.

In database_ddl_event the short interval was set by a first
`with_event_recorder_options` call and then overwritten by the pre-existing
one, which carries `..Default::default()`. Merged into a single call.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-09-23 01:52:52 +00:00
jeremyhi d7ada1761d feat: export logical tables from Metric physical scans (#9159) 
* feat: add physical Metric table exporter

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: drain Metric export writes before cancellation cleanup

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf: construct Metric export error context lazily

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: name the logical table export entry point

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: share Parquet writer for logical table exports

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve Parquet destinations and cancellation boundaries

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: clarify logical table export field names

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: clarify logical table export helper responsibilities

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: validate logical export membership by table ID

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test: simplify logical table export coverage and strengthen assertions

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-09-16 02:48:47 +00:00
Weny Xu a673e084b2 test: cover request-level insert WAL skipping end to end (#9093) 
* test: cover request-level WAL skipping end to end

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: cover session WAL policy and COPY recovery in sqlness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(test): isolate Mito test feature in dev dependencies

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: parameterize WAL protocol cases and make setup explicit

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: cover skip-WAL hints across streaming messages

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-09-11 07:48:57 +00:00
Dhruv Vaishnav 9198462869 feat(meta): record physical table reconciliation events (#8935) 
* feat(meta): record physical table reconciliation events

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* docs(config): add reconciliation table event

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* fix(meta): address reconciliation event review feedback

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* fix(meta): keep reconciliation event summary volatile

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* refactor(meta): remove unused table state downcasting

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

---------

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
 2026-08-28 06:17:55 +00:00
discord9anddiscord9 8d887ddd00 fix(query): restore columnar group-by for dictionary-encoded tags (#8902) 
* fix(query): restore columnar group-by for dictionary-encoded tags

Bump the DataFusion fork to be93ffd85 (feat/dict-group-column-53), which
backports apache/datafusion #23187: DictionaryGroupValuesColumn lets
dictionary-encoded group keys use the columnar GroupValuesColumn fast
path (hash distinct dictionary values once per batch, resolve rows by
key index) instead of falling back to row-based GroupValuesRows.

This fixes the TSBS double-groupby regression introduced by #8541
(preserve dictionary-encoded query labels): v1.2.0-beta.1 scan output
changed tag columns to Dictionary(UInt32, Utf8), which DataFusion 53.1.0
did not support in GroupValuesColumn's supported_type allow-list, so
GROUP BY queries silently dropped to the ~60% slower row path
(time_calculating_group_ids +57%, peak_mem +50%, end-to-end +38%).

Adds an end-to-end integration test (dict_groupby_sst) that flushes a
flat-format SST with dictionary-encoded hostname, runs the tsbs-style
double-groupby query, and asserts correct results with no CastExec
inserted before the aggregate.

Signed-off-by: discord9 <discord9@greptime.dev>

* chore(deps): pin merged dictionary group-by support

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <discord9@greptime.dev>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@greptime.dev>
 2026-08-21 03:50:30 +00:00
Weny Xu 943eee852f feat(event): record admin function executions (#8835) 
* feat(event): record admin function executions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(event): handle admin function recording edge cases

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(event): record actor for admin functions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(event): preserve admin function event values

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(event): preserve non-finite admin results

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-08-11 12:50:50 +00:00
Weny Xu 8f11629e34 feat(metasrv): add batch GC lifecycle events (#8673) 
* feat(metasrv): add batch GC lifecycle events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): reduce batch GC event fanout

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): fix batch GC event import

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): refine batch GC events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): preserve batch GC reports on failure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): retain batch GC reports on retry

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): harden batch GC report merging

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: scope repartition SST assertions to target table

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-31 07:13:56 +00:00
Weny Xu 979a22b38a feat(metasrv): record WAL prune procedure events (#8677) 
* feat(metasrv): record WAL prune procedure events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(metasrv): expand WAL prune procedure events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix toml fmt

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): clarify WAL prune event semantics

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-30 09:09:27 +00:00
Weny Xu 47ca5c362e feat: add table DDL procedure events (#8627) 
* feat(meta): emit table DDL procedure events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): honor table DDL event filters

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(meta): cover table DDL event filters

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): align table DDL event conventions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): bound table DDL event payloads

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(meta): consolidate table DDL event tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(meta): use crate visibility in event tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: stabilize table DDL event assertions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): exclude repartition from alter table events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): resolve table event rebase conflicts

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-30 03:24:47 +00:00
Weny Xu 31f9a9a6fd feat(metasrv): add repartition lifecycle events (#8665) 
* feat(metasrv): add repartition lifecycle events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(metasrv): simplify event module names

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(procedure): emit submitted events for child procedures

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(metasrv): flatten repartition event payload

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(metasrv): defer repartition topology rows

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(procedure): avoid events on failed child spawn

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-29 13:15:43 +00:00
Weny Xu 8ca6132b84 feat: add events for create and drop view (#8626) 
* feat(procedure): add view ddl events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(procedure): satisfy view event clippy

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(meta): add view DDL procedure events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(meta): group view event tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): align view DDL events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): centralize view event schema

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(integration): use singular view event module

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(integration): align view event assertions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(integration): share DDL event assertions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* docs: document view event recorder types

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(meta): align view DDL event conventions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(meta): simplify view event tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-29 03:13:44 +00:00
Weny Xu 775a9af3b8 feat: add procedure events for Flow DDL (#8632) 
* feat(meta): record Flow DDL procedure events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): remove query schema from Flow events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(tests): fix Flow DDL event test

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-28 09:02:11 +00:00
Weny Xu 09e1d24365 feat: add database DDL procedure events (#8623) 
* feat(meta): add database DDL procedure events

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(sqlness): disable event recording

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: poll database DDL event assertions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* docs(config): list database DDL event types

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(test): satisfy clippy

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-28 05:03:06 +00:00
fys cce70b9427 fix(mito): failed to compact memtable with json2 (#8297) 
* fix(json2): failed to compact memtable

* fix: cargo clippy

* refactor: align schema with json2 filed in flush

* chore: add unit test for json aligner

* chore: add json2 integration test

* fix: cr by codex

* fix: use parquet schema for encoded JSON2 memtable parts

* Use is_structured_json_field to determine whether the field is of JSON2 type.

* fix: cargo clippy

* fix: only align structured json fields

* chore: assert bulk JSON2 aligner input schemas in debug
 2026-06-26 08:52:06 +00:00
Han b5997c6797 test: cover standalone user provider config (#8067) 
* test: cover standalone user provider config

Signed-off-by: Detachm <42765252+Detachm@users.noreply.github.com>

* test: cover config-driven http auth

Signed-off-by: Detachm <42765252+Detachm@users.noreply.github.com>

---------

Signed-off-by: Detachm <42765252+Detachm@users.noreply.github.com>
 2026-05-08 08:56:22 +00:00
LFC a870b53f68 fix: mysql prepare correctly returns error instead of panic (#7963) 
feat: mysql writer support multiple statement execution

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-04-15 01:59:16 +00:00
Ning Sun e14404c677 chore: update rust toolchain to 2026-03-21 (#7849) 
* chore: update rust toolchain to 2026-03-21

* chore: new format

* fix: lint

* chore: resolve lint issues

* chore: remove as_millis_f64

* chore: deps up
 2026-03-30 12:13:14 +00:00
Weny Xu 0ed3b83099 refactor: rename partition rule version to partition expr version (#7696) 
* refactor: rename partition rule version to partition expr version

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: clippy

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-02-10 10:12:47 +00:00
Weny Xu 8026b23834 feat: partition rule version validation for writes and staging (#7628) 
* feat: verify partition rule

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add partition version cache

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: header check

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: fmt toml

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: minor refactor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: header

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: fix clippy

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: minor refactor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: nit

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: nit

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-02-06 12:16:34 +00:00
Weny Xuanddiscord9 4fb61047cb test: add integration tests for repartition (#7560) 
* test: add integration tests for mito repartition

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update test result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add integration tests for metric repartition

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: correct results

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: enable tests for object store

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add compaction and gc

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: fix unit test

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* more cases

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: file ref also in repart mapping

Signed-off-by: discord9 <discord9@163.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: set a longer timeout for mock metasrv channel manager

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: discord9 <discord9@163.com>
 2026-01-22 10:14:40 +00:00
LFC dc9f3a702e refactor: explicitly define json struct to ingest jsonbench data (#7462) 
ingest jsonbench data

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-12-24 07:30:22 +00:00
dennis zhuangandYingwen 03a28320d6 feat!: enable read cache and write cache when using remote object stores (#5093) 
* feat: enable read cache and write cache when using remote object stores

* feat: make read cache be aware of remote store names

* chore: docs

* chore: apply review suggestions

* chore: trim write cache path

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-12-10 04:03:44 +00:00
Weny Xu 4f0984c1d7 chore: remove original region failover implementation (#4237) 
chore: remove original region failure implementation
 2024-07-05 08:03:46 +00:00
Ruihang XiaandWenyXu 020635063c feat: implement multi-dim partition rule (#3409) 
* generate expr rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement show create for new partition rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement row spliter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix failed tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: fix lint issues

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: ignore tests for deprecated partition rule

* chore: remove unused partition rule tests setup

* test(sqlness): add basic partition tests

* test(multi_dim): add basic find region test

* address CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2024-03-05 11:39:15 +00:00
Ruihang Xia 3544c9334c feat!: new partition grammar - parser part (#3347) 
* parser part

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test in sql

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* comment out and ignore some logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update region migration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* temporary disable region migration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* allow dead code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-02-27 07:20:16 +00:00
Weny Xu f1a4750576 feat(tests-integration): add more region migration integration tests (#3094) 2024-01-04 08:18:46 +00:00
niebayes 830a91c548 feat(remote_wal): implement topic allocation (#2970) 
* chore: implement wal options allocator

* chore: implement round-robin topic selector

* feat: add shuffle to round-robin topic selector

* chore: implement kafka topic manager

* test: add tests for wal options allocator

* test: add wal provider to test config files

* test: leave todos for adding tests for remote wal

* fix: resolve review conversations

* fix: typo
 2023-12-22 08:26:48 +00:00
Ruihang XiaandWenyXu 34d6288945 feat: bring back sqlness and integration tests (#2448) 
* enable integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable sqlness region failover

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enable sqlness in CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort unstable result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* set require_lease_before_startup to true

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix inconsistent cache

* replace windows path chars

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore some integration cases in windows

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Revert "ignore some integration cases in windows"

This reverts commit 122478b7c1.

* disable windows for now

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix close region bug in RegionHeartbeatResponseHandler

* disable failover tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2023-09-20 09:17:30 +00:00
Niwaka 66e5ed5483 feat: support gcs storage (#1781) 2023-07-05 23:03:51 +08:00
Weny Xu 803940cfa4 feat: enable azblob tests (#1765) 
* feat: enable azblob tests

* fix: add missing arg
 2023-06-13 07:44:57 +00:00
dennis zhuang ae8203fafa fix: prepare statement doesn't support insert clause (#1680) 
* fix: insert clause doesn't support prepare statement

* fix: manifeste dir

* fix: format

* fix: temp path
 2023-05-31 20:14:58 +08:00
Weny Xu 0460f3ae30 test: add write test for region failover (#1673) 
* test: add write test for region failover

* test: add written data assertion after failover

* test: support more storage types
 2023-05-31 15:42:00 +08:00
Zou Wei ae81c7329d feat: support azblob storage. (#1659) 
* feat:support azblob storage.

* test:add some tests.

* refactor:use if-let.
 2023-05-30 19:59:38 +08:00
dennis zhuang 8d8a480dc1 fix: object store caching bug, #1466 (#1467) 
* fix: object store caching bug, #1466

* fix: forgot to add S3WithCache tests
 2023-04-25 21:48:51 +08:00
Yun Chen a7dc86ffe5 feat: oss storage support (#911) 
* feat: add oss storage support

* fix: ci build format check

* fix: align OSS to Oss

* fix: cr comments

* fix: rename OSS to Oss in integration tests

* fix: clippy fix
 2023-01-29 20:09:38 +08:00
Lei, HUANG 8ffc078f88 fix: license header (#815) 2023-01-03 15:09:49 +08:00
dennis zhuang 2e17e9c4b5 feat: supports s3 storage (#656) 
* feat: adds s3 object storage configuration

* feat: adds s3 integration test

* chore: use map

* fix: forgot license header

* fix: checking if bucket is empty in test_on

* chore: address CR issues

* refactor: run s3 test with dotenv

* chore: randomize grpc port for test

* fix: README in tests-integration

* chore: remove redundant comments
 2022-12-01 10:59:14 +08:00