Commit Graph

80 Commits

Author SHA1 Message Date
Ruihang Xia
0c5f4801b7 build: update toolchain to nightly-2024-04-18 (#3740)
* chore: update toolchain to nightly-2024-04-17

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix ut

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update fuzz test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update to nightly-2024-04-18

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add document

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* avoid unnecessary allow clippy attrs

Signed-off-by: tison <wander4096@gmail.com>

* help the compiler find the clone is unnecessary and make clippy happy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
2024-04-19 05:42:34 +00:00
zyy17
3e9bda3267 ci: use greptimedb-ci-tester account (#3719) 2024-04-16 14:43:17 +00:00
zyy17
8060c81e1d refactor: use toml2docs to generate config docs (#3704)
* refactor: use toml2docs to generate config docs

* ci: add docs check in 'check-typos-and-docs'
2024-04-15 09:08:32 +00:00
Yohan Wal
682b04cbe4 feat(fuzz): add create database target (#3675)
* feat(fuzz): add create database target

* chore(ci): add fuzz_create_database ci cfg
2024-04-09 01:33:29 +00:00
tison
2fca45b048 ci: setup-protoc always with token (#3674)
Signed-off-by: tison <wander4096@gmail.com>
2024-04-08 18:13:24 +00:00
Ning Sun
5ca8521e87 ci: attempt to setup docker cache for etcd (#3488)
* ci: attempt to setup docker cache for etcd

* ci: do not use file hash for cache key
2024-03-14 00:48:02 +00:00
Weny Xu
e4333969b4 feat(fuzz): add alter table target (#3503)
* feat(fuzz): validate semantic type of column

* feat(fuzz): add fuzz_alter_table target

* feat(fuzz): validate columns

* chore(ci): add fuzz_alter_table ci cfg
2024-03-13 14:11:47 +00:00
Weny Xu
a218f12bd9 test: add fuzz test for create table (#3441)
* feat: add create table fuzz test

* chore: add ci cfg for fuzz tests

* refactor: remove redundant nightly config

* chore: run fuzz test in debug mode

* chore: use ubuntu-latest

* fix: close connection

* chore: add cache in fuzz test ci

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: refactor the fuzz test action
2024-03-07 06:51:19 +00:00
Weny Xu
9ec288cab9 chore: specify binary name (#3449) 2024-03-07 03:56:24 +00:00
tison
fa08085119 ci: upgrade actions to node20-based version (#3345)
* ci: upgrade actions to node20-based version

Signed-off-by: tison <wander4096@gmail.com>

* distinguish artifact name

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-02-21 08:09:09 +00:00
tison
d9a96344ee ci: try fix log location (#3342)
Signed-off-by: tison <wander4096@gmail.com>
2024-02-21 07:01:51 +00:00
LFC
eded08897d test: add data compatibility test (#3109)
* test: data files compatibility test

* rework compatibility test

* revert unneeded changes

* revert unneeded changes

* debug CI

* Update .github/workflows/develop.yml

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2024-02-20 07:44:04 +00:00
tison
29f11d7b7e ci: upgrade actions to avoid node16 deprecation warning (#3317)
* ci: upgrade actions to avoid node16 deprecantion warning

Signed-off-by: tison <wander4096@gmail.com>

* try delete size label action

Signed-off-by: tison <wander4096@gmail.com>

* one more action

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-02-18 15:20:26 +00:00
Ruihang Xia
94fd51c263 ci: run CI jobs in draft PR (#3314)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
2024-02-18 13:14:57 +00:00
tison
2a26c01412 fix: commit_short sqlness test case (#3313)
* ci: debug sqlness jobs

Signed-off-by: tison <wander4096@gmail.com>

* fix: commit_short test case

Signed-off-by: tison <wander4096@gmail.com>

* fixup! fix: commit_short test case

Signed-off-by: tison <wander4096@gmail.com>

* fixup! fixup! fix: commit_short test case

Signed-off-by: tison <wander4096@gmail.com>

* revert uploading

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-02-18 11:51:24 +00:00
tison
4e04a4e48f build: support build without git (#3309)
* build: support build without git

Signed-off-by: tison <wander4096@gmail.com>

* chore

Signed-off-by: tison <wander4096@gmail.com>

* address comment

Signed-off-by: tison <wander4096@gmail.com>

* fix syntax

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-02-18 10:30:01 +00:00
Weny Xu
96f32a166a chore: share cache corss jobs (#3284) 2024-02-05 09:30:22 +00:00
LFC
50d16d6330 ci: build GreptimeDB binary for later use (#3244)
* ci: build GreptimeDB binaries for later use

* debug CI

* try larger runner host

* Revert "try larger runner host"

This reverts commit 03c18c0f51.

* fix: resolve PR comments

* revert some unrelated action yamls

* fix CI

* use artifact upload v4 for faster upload and download speed
2024-01-31 03:02:27 +00:00
Ruihang Xia
e0e635105e feat: initial configuration for grafana dashboard (#3263)
* feat: initial configuration for grafana dashboard

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* lift up dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update readme

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add paths to CI config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tweak config details

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add OpenDAL traffic panel

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove sync count

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update grafana/README.md

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2024-01-30 09:11:50 +00:00
Weny Xu
691b649f67 chore: switch to free machine (#3256)
* chore: switch to free machine

* chore: switch sqlness runner to 4core
2024-01-29 08:35:12 +00:00
Weny Xu
e16f093282 test(remote_wal): add sqlness with kafka wal (#3027)
* feat(sqlness): add kafka wal config

* chore: add sqlness with kafka wal ci config

* fix: fix config

* chore: apply suggestions from CR

* fix: add metasrv config to sqlness with kafka

* fix: replay memtable should from flushed_entry_id + 1

* fix: should set append flag to fopen

* feat: start wal allocator in standalone meta mode

* feat: append a noop record after kafka topic initialization

* test: ignore tests temporally

* test: change sqlness kafka wal config
2023-12-29 08:17:22 +00:00
Ruihang Xia
286b9af661 chore: change all reference from develop to main (#3026)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-28 04:11:00 +00:00
Weny Xu
89129c99c8 chore: setup kafka standalone in coverage test (#2984)
* chore: setup kafka standalone in coverage test

* test: add a naive test for topic manager
2023-12-25 11:18:54 +00:00
niebayes
830a91c548 feat(remote_wal): implement topic allocation (#2970)
* chore: implement wal options allocator

* chore: implement round-robin topic selector

* feat: add shuffle to round-robin topic selector

* chore: implement kafka topic manager

* test: add tests for wal options allocator

* test: add wal provider to test config files

* test: leave todos for adding tests for remote wal

* fix: resolve review conversations

* fix: typo
2023-12-22 08:26:48 +00:00
Ruihang Xia
6b8dbcfb54 chore: update toolchain to 20231219 (#2932)
* update toolchain file, remove unused feature gates

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update action file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update to 12-19

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-12-19 07:24:08 +00:00
Ning Sun
bd9c2f2666 fix: windows build and check ci check for windows (#2819) 2023-11-27 03:42:44 +00:00
tison
102e43aace test: use EtcdStore in IT cases (#2734)
* test: use EtcdStore in IT cases

Signed-off-by: tison <wander4096@gmail.com>

* retrigger CI

Signed-off-by: tison <wander4096@gmail.com>

* refactor: KvPair can take etcd KeyValue

Signed-off-by: tison <wander4096@gmail.com>

* temporary use fork

Signed-off-by: tison <wander4096@gmail.com>

* drop cloned

Signed-off-by: tison <wander4096@gmail.com>

* chroot_key_value

Signed-off-by: tison <wander4096@gmail.com>

* chroot and prepend in each point

Signed-off-by: tison <wander4096@gmail.com>

* adjust call points

Signed-off-by: tison <wander4096@gmail.com>

* cargo clippy

Signed-off-by: tison <wander4096@gmail.com>

* point to upstream etcd-client

Signed-off-by: tison <wander4096@gmail.com>

* test etcd chroot

Signed-off-by: tison <wander4096@gmail.com>

* add NO_CHROOT constant

Signed-off-by: tison <wander4096@gmail.com>

* check

Signed-off-by: tison <wander4096@gmail.com>

* handle range end

Signed-off-by: tison <wander4096@gmail.com>

* handle special encoded key or range_end

Signed-off-by: tison <wander4096@gmail.com>

* fixup implementation

Signed-off-by: tison <wander4096@gmail.com>

* clippy

Signed-off-by: tison <wander4096@gmail.com>

* avoid test name conflict

Signed-off-by: tison <wander4096@gmail.com>

* chroot to kvbackend level

Signed-off-by: tison <wander4096@gmail.com>

* fixup all occurances

Signed-off-by: tison <wander4096@gmail.com>

* fix type

Signed-off-by: tison <wander4096@gmail.com>

* Update src/common/meta/src/kv_backend/txn.rs

* make github happy

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
2023-11-23 05:47:00 +00:00
tison
efc5abfc02 build: upgrade etcd-client to 0.12.2 (#2781)
* build: upgrade etcd-client to 0.12.2

Signed-off-by: tison <wander4096@gmail.com>

* upgrade nightly toolchain

Signed-off-by: tison <wander4096@gmail.com>

* chore: run clippy

Signed-off-by: tison <wander4096@gmail.com>

* Update Cargo.toml

Co-authored-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: Ning Sun <classicning@gmail.com>
2023-11-21 06:33:41 +00:00
Ruihang Xia
825e4beead build(ci): pin linux runner to ubuntu-20.04 (#2586)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-10-12 18:08:05 +08:00
Ning Sun
9e33ddceea ci: run windows tests every night instead of every commit (#2577)
ci: move windows ci to nightly-ci
2023-10-12 02:53:42 +00:00
JeremyHi
40781ec754 fix: test on windows (#2462)
* fix: test on windows

* fix: fix windows root

* fix: use relative path instead of root

* fix: remove incorrect replace

* fix: fix tests

---------

Co-authored-by: WenyXu <wenymedia@gmail.com>
2023-09-21 10:57:56 +00:00
Ruihang Xia
34d6288945 feat: bring back sqlness and integration tests (#2448)
* enable integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable sqlness region failover

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enable sqlness in CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* sort unstable result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* set require_lease_before_startup to true

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix inconsistent cache

* replace windows path chars

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore some integration cases in windows

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Revert "ignore some integration cases in windows"

This reverts commit 122478b7c1.

* disable windows for now

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: fix close region bug in RegionHeartbeatResponseHandler

* disable failover tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
2023-09-20 09:17:30 +00:00
Ruihang Xia
718246ea1a feat: implement heartbeat for region server (#2279)
* retrieve region stats from region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat handler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start datanode with region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable non-unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat task

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-09-12 07:57:15 -05:00
Yingwen
bb062003ef ci: fallback to run_id to avoid cancelling other jobs (#2186)
ci: fallback to run id to avoid cancelling other jobs
2023-08-16 09:24:17 +00:00
Ruihang Xia
814c599029 ci: cancel in-progress actions on new commit (#2182)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-08-16 04:21:14 +00:00
LFC
2dcc67769e fix: runs sqlness test on windows-latest-8-cores (#2158) 2023-08-11 17:34:58 +08:00
LFC
5c19913a91 build: on windows (#2054)
* build on windows

* rebase develop

* fix: resolve PR comments
2023-08-10 08:08:37 +00:00
LFC
46fa3eb629 chore: upgrade rust toolchain to latest nightly (#2049)
* chore: upgrade rust toolchain to latest nightly

* rebase develop

* update rust toolchain in ci
2023-08-08 07:17:51 +00:00
shuiyisong
7efff2d704 fix: introduce taplo.toml and sort Cargo.toml (#2096)
* fix: add taplo.toml

* fix: introduce taplo.toml & sort cargo.toml

* chore: remove option in ci too
2023-08-04 06:44:45 +00:00
Ning Sun
93cd4ab89d ci: require cargo.lock up to date (#2094) 2023-08-04 02:59:01 +00:00
Ruihang Xia
bddaf265a9 chore(ci): run clippy, coverage and sqlness in parallel (#2050)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-31 10:37:30 +00:00
Ruihang Xia
f152568701 fix(ci): add merge queue trigger (#1980)
* fix(ci): add merge queue trigger

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update docs.yml also

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-07-17 18:01:05 +08:00
Ruihang Xia
ef15de5f17 ci: always upload sqlness log (#1692)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-06-01 20:01:26 +08:00
Ruihang Xia
d4f3f617e4 chore(toolchain): update rust-toolchain to 2023-05-03 (#1524)
* chore(toolchain): update rust-toolchain to 2023-05-03

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update workflow yaml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-05-06 11:34:09 +08:00
Weny Xu
47f1cbaaed fix: add s3 region info (#1486) 2023-04-27 17:35:34 +08:00
Ruihang Xia
b6647af2e3 test: add integration case to check dashboard path (#1422)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-20 11:17:01 +08:00
Ruihang Xia
6b6617f9cb build: specify clippy denies in cargo config (#1351)
* build: specify clippy denies in cargo config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* deny implicit clone

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-11 09:48:52 +00:00
discord9
379f581780 test: add Integrated Test for Coprocessor& fix minor bugs (#1122)
* feat: cache `Runtime`

* fix: coprstream schema not set

* test: integrated tests for Coprocessor

* fix: UDF fixed

* style: remove unused import

* chore: remove more unused import

* feat: `filter`, (r)floordiv for Vector

* chore: CR advices

* feat: auto convert to `lit`

* chore: fix typo

* feat: from&to `pyarrow.array`

* feat: allow `pyarrow.array` as args to builtins

* chore: cargo fmt

* test: CI add `pyarrow`

* test: install Python&PyArrow in CI

* test: not cache depend for now

* chore: CR advices

* test: fix name

* style: rename
2023-03-06 19:20:59 +08:00
Ning Sun
d402f83442 ci: generate apidocs when pushing to default branch (#1093)
* ci: generate apidocs when pushing to default branch

* ci: require clippy before running tests

* fix: resolve new clippy warnings on primitive slice

* fix: resolve more clippy warnings

* Update .github/workflows/apidoc.yml

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* ci: add an index html to redirect

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-03-01 13:18:26 +08:00
discord9
c5c6494e0b feat: add PyO3(Hence CPython as a Optional Backend (#976)
* refactor: ffi_types

* style: fmt

* refactor: use `String` for return when possible

* todo: vector_impl

* feat: pyobj_try_typed_val

* refactor: more backend indep function

* feat: +-*/ magic methods

* refactor: copr

* style: fmt

* feat: add paired tests

* refactor: more

* refactor: move inside `python` folder

* refactor: all but test code

* feat: builtins for PyO3

* chore: add licenses

* chore: remove unused&add todos

* refactor: remove old files

* chore: mark unused

* chore: fmt

* chore: license

* feat: query in PyO3

* test: paired testcases for rspy&pyo3

* feat: PyDataFrame(Untested)

* feat: some allow_threads

* style: fmt

* style: add license

* feat: rebase manually of #962

* feat: more `allow_threads`

* chore: typo

* chore: remove some `TODO`

* test: allow margin of epsilon

* chore: code review advices

* chore: more CR adjust

* chore: more adjust

* feat: kwargs&its test

* chore: remove some `dbg!`

* chore: allow params

* fix: put `dataframe` into scope

* chore: newline

* fix: adjust after rebase

* fix: test serde skip attr

* style: taplo

* feat: add `pyo3_backend` feature

* doc: update CI&readme
2023-03-01 10:45:55 +08:00