rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-20 15:00:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,286 Commits 133 Branches 253 Tags
feat/expr-split
Commit Graph

4372 Commits

Author SHA1 Message Date
Weny Xu
57f8afcb70 chore: avoid sending create table requests for already existing tables (#5347) 
* chore: avoid sending create table requests for already existing tables

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2025-01-15 02:50:25 +00:00
Ruihang Xia
bd37e086c2 fix: improve error handling in pipeline and log in log store (#5357) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-01-14 13:58:30 +00:00
discord9
66f63ae981 feat: more than one flow workers (#5315) 
* feat: more workers

* feat: use round robin

* refactor: per review

* refactor: per bot review

* chore: per review

* docs: example

* docs: update config.md

* docs: update

* chore: per review

* refactor: set workers to cpu/2.max(1)

* fix: flow config in standalone mode

* test: fix config test

* docs: update docs&opt name

* chore: update config.md

* refactor: per review, sanitize at top

* chore: per review

* chore: config.md
 2025-01-14 12:28:18 +00:00
discord9
95b20592ac fix: handle insert default value (#5307) 
* fix: handle flow inserts with default values

* test: sqlness

* chore: typo

* chore: newline

* feat(WIP): impure default filler

* feat: fill impure default values

* test: add test for default fill impure

* feat: check for impure

* fix: also handle stmt to region

* refactor: per review

* refactor: per review

* chore: rebase fix

* chore: clippy

* chore: per review
 2025-01-14 09:06:53 +00:00
ZonaHe
1855dccdf1 feat: update dashboard to v0.7.8 (#5355) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2025-01-14 09:00:08 +00:00
Ning Sun
f5829364a2 fix: security fix, sqlx, hashbrown, idna and CI updates (#5330) 
* fix: security fix, sqlx, hashbrown, idna

* ci: optimize ci cache generation

* feat: update pprof
 2025-01-14 08:19:33 +00:00
Xuanwo
87bd12d6df refactor: Bump opendal to 0.51.1 (#5354) 
* refactor: Bump opendal to 0.51.1

Signed-off-by: Xuanwo <github@xuanwo.io>

* Ignore dirs from cache

Signed-off-by: Xuanwo <github@xuanwo.io>

* Reduce extra alloc

Signed-off-by: Xuanwo <github@xuanwo.io>

---------

Signed-off-by: Xuanwo <github@xuanwo.io>
 2025-01-14 07:28:09 +00:00
ZonaHe
c370b4b40d feat: update dashboard to v0.7.7 (#5350) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2025-01-13 07:35:12 +00:00
zyy17
3f01f67f94 refactor(elasticsearch): use _index as greptimedb table in log ingestion and add /${index}/_bulk API (#5335) 
* refactor(elasticsearch): use `_index` as greptimedb table in log ingestion and add `/${index}/_bulk` API

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: code review

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-01-13 06:58:04 +00:00
Weny Xu
6eb746d994 fix: skip building indexer when indexed_column_ids are empty (#5348) 2025-01-13 05:15:20 +00:00
yihong
03a144fa56 chore: drop useless import raw_normalize_path in object-store lib (#5349) 
chore: drop use less import raw_normalize_path in store lib

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-13 04:08:36 +00:00
Yohan Wal
f069ea082f fix(test): use different table name (#5334) 2025-01-10 17:09:23 +00:00
ZonaHe
9ae48010f0 feat: update dashboard to v0.7.6 (#5340) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2025-01-10 17:06:09 +00:00
yihong
3a996c2f00 feat: add set search_path to 'xxx' for pg (#5342) 
* feat: add set search_path to 'xxx' for pg

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-10 17:05:10 +00:00
ZonaHe
45d4065fd6 feat: update dashboard to v0.7.4 (#5336) 
Co-authored-by: sunchanglong <sunchanglong@users.noreply.github.com>
 2025-01-10 08:37:02 +00:00
discord9
9e09be7ba6 chore: update rustls (#5337) 
* deps: update rustls

* chore: update lock

* chore: server use workspace rustls

* chore: comment on version
 2025-01-10 08:17:40 +00:00
Niwaka
50583815de feat: support alter add multiple columns (#5262) 
* feat: support alter add multiple columns

* fix: address review

* chore: add column format
 2025-01-10 06:14:17 +00:00
yihong
24ea9cf215 feat: add show search_path for pg (#5328) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-10 04:04:05 +00:00
yihong
78d0fa75c9 fix: android build (#5329) 
* fix: android build

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: better fix address comments

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-09 14:47:29 +00:00
Weny Xu
be22da775a build: disable local IP detection feature in Android binary (#5327) 
build: disable local ip detection feature in android binary
 2025-01-09 11:13:48 +00:00
Lin Yihai
fdbfebf4be feat: Add VEC_PRODUCT, VEC_ELEM_PRODUCT, VEC_NORM. (#5303) 
* feat: Add `vec_product(col)` function.

* feat: Add `vec_elem_product` function

* feat: Add `vec_norm` function.
 2025-01-09 06:26:51 +00:00
Yohan Wal
751fa4ede9 feat(config): make table name of pg backend configurable (#5244) 
* feat: configurable table name

* fix: election sql

* feat: configurable lock_id

* chore: update config file

* perf: useless allocation

* perf: useless allocation

* chore: remove unused type hint

* Apply suggestions from code review

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* chore: update config file

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2025-01-08 14:55:35 +00:00
zyy17
03a2e6d0c1 feat: support elasticsearch _bulk API to ingest logs (#5261) 
* feat: support elasticsearch '_bulk' API to ingest logs

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: code review

* refactor: add metrics

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-01-08 14:54:53 +00:00
Yingwen
c19a56c79f feat!: Remove script crate and python feature (#5321) 
* feat: exclude script crate

* chore: simplify feature

* feat: remove the script crate

* chore: remove python feature and some comments

* chore: fix warning
 2025-01-08 12:11:53 +00:00
Weny Xu
7f307a4cac feat: auto detecting local IP to use as hostname (#5314) 
* feat: auto detecting hostname

* chore: update config.md

* chore: bring back hostname

* chore: apply suggestions from CR

* chore: use const

* chore: apply suggesions from CR

* fix: fix sqlness
 2025-01-08 12:06:31 +00:00
yihong
52eebfce77 feat: support select session_user; (#5313) 
* feat: support `select session_user;`

This commit is part of support DBeaver that support function
select session_user like postgres did.

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: lint problem

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: address comments add tests

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-08 09:44:54 +00:00
shuiyisong
d1f8ea7880 perf: parse Loki labels in protobuf write path (#5305) 
* chore: parse loki labels

* chore: add bench

* chore: add comment

* chore: add test

* chore: remove unnecessory default value and update test

* fix: typo and test

* chore: cr issue

* chore: cr issue
 2025-01-08 07:55:06 +00:00
LFC
2cd1b08ff7 refactor: optimize out partition split insert requests (#5298) 
* test: optimize out partition split insert requests if there is only one region

* Now that the optimization for single region insert has been lifted up, the original "fast path" can be obsoleted.

* resolve PR comments
 2025-01-08 07:26:29 +00:00
discord9
0ee41339aa feat(flow): flow refill state (Part 1) (#5295) 
* feat(flow): (Part 1) refill utils

* chore: after rebase fix

* chore: more rebase

* rm refill.rs to reduce pr size

* chore: simpler args

* refactor: per review

* docs: more explain for instant requests

* refactor: per review
 2025-01-08 06:45:56 +00:00
yihong
369b59c84a fix: drop unused dep using udeps to minial the size (#5301) 
* fix: drop unused dep using udeps to minial the size

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: adress comments fix the problem

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-08 06:36:22 +00:00
Zhenchi
c305b2b406 fix: show index to display skipping index (#5297) 
* fix: show index to display skipping index

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2025-01-08 04:59:40 +00:00
Weny Xu
3d9df822ad refactor: refactor PgStore (#5309) 
* refactor: refactor PgStore

* fix: election use bytea and txn use Serializable to avoid read unrepeatable (#4)

* fix: election use bytea as well

* fix: use Serializable to avoid read unrepeatable

* chore: remove unused error

* ci: enable pg kvbackend and sqlness

* ci: switch on pg_kvbackend feature

* fix: fix sqlness runner

* chore: add pg_kvbackend feature gate

* build(ci): add feature gate

* fix: add retry for `PgStore` txn

* fix: correct `SET_IDLE_SESSION_TIMEOUT`

---------

Co-authored-by: Yohan Wal <1035325592@qq.com>
Co-authored-by: CookiePieWw <profsyb@gmail.com>
 2025-01-07 07:27:58 +00:00
Weny Xu
05f115e047 fix(pg_backend): correct set idle_in_transaction_session_timeout statement (#5304) 
* fix(metasrv): correct `backend` field configuration

* refactor!: added `#[serde(rename_all = "snake_case")]` macro to the `BackendImpl` enum

* fix(metasrv): correct `set idle_in_transaction_session_timeout` statement

* build: enable `pg_backend` by default
 2025-01-06 12:07:35 +00:00
Zhenchi
5cf9d7b6ca fix(bloom-filter): filter rows with segment precision (#5286) 
* fix(bloom-filter): filter rows with segment precision

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* add case

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* address TODO

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2025-01-06 11:45:15 +00:00
shuiyisong
a56c430db0 fix: pass unknown encoding in decompression layer (#5300) 
fix: allow pass unknown in decompression layer
 2025-01-06 09:10:56 +00:00
Ning Sun
04708f10aa feat(pipeline): allow coerce timestamp from integer values (#5270) 
* feat: allow coerce timestamp from integer values

* test: add tests for parsing integer values
 2025-01-06 08:53:09 +00:00
Weny Xu
96b2a5fb28 feat: introduce ParallelFstValuesMapper (#5276) 
* refactor: `RangeReader` to use `&self`

* refactor: `InvertedIndexReader` to use `&self`

* refactor: refactor: `BloomFilterReader` to use `&self`

* feat: introduce `ParallelFstValuesMapper`

* chore: change prefetch size to 8KiB

* chore: add `file_size_hint` for cached blob reader

* chore: fix clippy

* refactor: remove `FstValuesMapper`

* chore: apply suggestions from CR
 2025-01-06 07:33:35 +00:00
shuiyisong
bbbba29afc feat: support Loki JSON write (#5288) 
* perf: small updates

* refactor: move loki to a separate file

* chore: extract content_type pattern matching

* chore: minor update

* feat: loki json write

* chore: add decompression http layer

* fix: label string value instead of to_string

* chore: add test

* fix: typo

* fix: license header

* chore: rename
 2025-01-06 07:09:48 +00:00
Kould
2ad50332cb feat: impl COPY a query resultset to external file (#5250) 
* feat: impl COPY a query resultset to external file

* chore: add more tests for parse `copy_table_to`

* chore: add more tests for parse `copy_table_to`
 2025-01-06 04:32:51 +00:00
Yohan Wal
513569ed5d feat: add Txn for pg kv backend (#5266) 
* feat: txn for pg kv backend

* chore: clippy

* fix: txn uses one client

* test: clean up and txn test

* test: clean up

* test: change lock_id to avoid conflict in test

* test: use different prefix in pg election test

* fix(test): just a fix

* test: aggregate multiple test to avoid concurrency problem

* test: use uuid instead of rng

* perf: batch cmp in txn

* perf: batch same op in txn
 2025-01-06 03:29:09 +00:00
Kould
1067357b72 chore(config)!: refactor configs of write cache (#5259) 
* chore: refactor configs of write cache

* chore: write_cache_size `10GiB` -> `5GiB`
 2025-01-04 07:14:38 +00:00
Ning Sun
2caf003db0 fix: brings back functions required by downstream projects (#5283) 2025-01-04 03:49:01 +00:00
discord9
353c8230db refactor: flow replace check&better error msg (#5277) 
* chore: better error msg

* chore eof newline

* refactor: move replace check to flow worker

* chore: add ctx to insert flow failure

* chore: Update src/flow/src/adapter/flownode_impl.rs

* test: add order by for deterministic

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2025-01-03 10:59:58 +00:00
Lei, HUANG
577d81f14c chore: suppress list warning (#5280) 
chore/suppress-list-warning:
 ### Update logging level in `intermediate.rs`

 - Changed logging level from `warn` to `debug` for unexpected directory entries in index creation.
 - Added `debug` to the `common_telemetry` import to support the logging level change.
 2025-01-03 09:05:03 +00:00
yihong
856bba5d95 fix: better fmt check from 40s to 4s (#5279) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-03 08:12:49 +00:00
Yingwen
89399131dd feat: support add if not exists in the gRPC alter kind (#5273) 
* test: test adding existing columns

* chore: add more checks to AlterKind

* chore: update logs

* fix: check and build table info first

* feat: Add add_if_not_exists flag to alter expr

* feat: skip existing columns when building alter kind

* checks in make_region_alter_kind()
* reuse the alter kind

* test: fix tests in common-meta

* chore: fix typos

* chore: update comments
 2025-01-03 07:23:17 +00:00
discord9
d20b592fe8 fix: flow handle reordered inserts (#5275) 
* fix: reorder correct schema

* tests: reorder insert handled correctly

* chore: rm unused

* refactor: per review

* chore: more comment

* chore: per review
 2025-01-03 06:25:39 +00:00
Yohan Wal
bcb0f14227 refactor: adjust index cache page size (#5267) 
* refactor: adjust index cache page size

* fix: wrong docs

* Update config/datanode.example.toml

* Update config/config.md

* Update config/config.md

* chore: adjust to 64KiB

* Apply suggestions from code review
 2025-01-03 03:26:17 +00:00
discord9
4d6fe31fff fix(flow): flow's table schema cache (#5251) 
* fix: flow schema cache

* refactor: location for `to_meta_err`

* chore: endfile emptyline

* chore: review(partially)

* chore: per review

* refactor: per review

* refactor: per review
 2025-01-02 10:33:23 +00:00
Yohan Wal
1b0b9add90 feat: use connection pool for pg kv backend in preparation for txn (#5260) 
* feat: use connection pool

* chore: follow review comments

* fix: create table before test
 2025-01-02 06:33:21 +00:00