rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-06 21:32:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,983 Commits 105 Branches 226 Tags
89b86c87a2fee894f57b9e1ab5cad5f1a610bf45
Commit Graph

2983 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weny Xu
89b86c87a2 chore: add docs for config file (#4432) 2024-07-25 08:11:10 +00:00
Lei, HUANG
0b0ed03ee6 fix(metrics): RowGroupLastRowCachedReader metrics (#4418) 
fix/reader-metrics:
 Refactor cache hit/miss logic and update metrics in mito2

 - Simplify cache retrieval logic in CacheManager by removing inline update_hit_miss function call.
 - Add separate functions for incrementing cache hit and miss metrics.
 - Update RowGroupLastRowCachedReader to use new cache hit/miss functions and refactor to new helper methods for creating Hit and Miss variants.
 2024-07-25 06:45:43 +00:00
dennis zhuang
ea4a71b387 docs: update readme (#4431) 2024-07-25 06:17:45 +00:00
dennis zhuang
4cd5ec7769 docs: update readme (#4430) 2024-07-25 02:42:18 +00:00
Ruihang Xia
c8f4a85720 chore: update grafana dashboard to reflect recent metric changes (#4417) 
* chore: update grafana dashboard to reflect recent metric changes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: add a blank line at the end

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-07-24 20:05:44 +00:00
discord9
024dac8171 chore: add a compile cfg for python in cmd package (#4406) 
* chore: add a compile cfg for python

* fix: feature gate additive turn off default features in workspace&add cfg in place

* chore: remove unused in different cfg
 2024-07-24 20:03:53 +00:00
Ran Miller
918be099cd docs(common_error): format enum StatusCode docs (#4427) 
* fix: format comments end with . symbol
* docs: add commnet for RegionReadonly
* fix: comment error for DatabaseAlreadyExists
 2024-07-24 15:54:35 +00:00
Zhenchi
91dbac4141 fix(fulltext-index): clean up 0-value timer (#4423) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-07-24 15:03:36 +00:00
Ran Miller
e935bf7574 refactor: Remove PhysicalOptimizer and LogicalOptimizer trait (#4426) 
* refactor(query): Remove LogicalOptimizer trait

* refactor(query): Remove PhysicalOptimizer trait
 2024-07-24 13:01:44 +00:00
Ran Miller
f7872654cc refactor(query): Remove PhysicalPlanner trait (#4412) 2024-07-24 03:06:46 +00:00
shuiyisong
547730a467 chore: add metrics for log ingestion (#4411) 
* chore: add metrics for log ingestion

* chore: record result as well
 2024-07-23 08:05:11 +00:00
Ning Sun
49f22f0fc5 fix: add back AuthBackend which is required by custom auth backend (#4409) 2024-07-23 05:35:29 +00:00
zyy17
2ae2a6674e refactor: add get_storage_path() and get_catalog_and_schema() (#4397) 
refactor: add get_storage_path() and get_catalog_and_schema()
v0.10.0-nightly-20240722 		2024-07-20 01:55:48 +00:00
Lei, HUANG
c8cf3b1677 fix(wal): handle WAL deletion on region drop (#4400) 
Add LogStore trait bound to RegionWorkerLoop and handle WAL deletion on region drop.
 2024-07-19 13:24:10 +00:00
Yingwen
7aae19aa8b fix: dictionary key type use u32 (#4396) 
* fix: dictionary key type use u32

* fix: fix error whle reading content

* fix: bulk memtable dictionary type
 2024-07-19 09:51:29 +00:00
Jeremyhi
b90267dd80 feat: export database data (#4382) 
* feat: export database data

* feat: export data with time range

* feat: refactor the data dir

* feat: by comment
 2024-07-19 09:29:45 +00:00
discord9
9fa9156bde feat: FLOWS table in information_schema&SHOW FLOWS (#4386) 
* feat(WIP): flow info table

refactor: better err handling&log

feat: add flow metadata to info schema provider

feat(WIP): info_schema.flows

feat: info_schema.flows table

* fix: err after rebase

* fix: wrong comparsion op

* feat: SHOW FLOWS&tests

* refactor: per review

* chore: unused

* refactor: json error

* chore: per review

* test: sqlness

* chore: rm inline error

* refactor: per review
 2024-07-19 09:29:36 +00:00
zyy17
ce900e850a fix: user provider can't be configured by config file or environment variables (#4398) 2024-07-19 08:41:29 +00:00
zyy17
5274c5a407 refactor: add &mut Plugins argument in plugins setup api and remove unnecessary mut (#4389) 
refactor: add '&mut Plugins' argument in plugins setup api and remove unnecessary mut

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2024-07-19 08:12:06 +00:00
Yingwen
0b13ac6e16 ci: disable auto review (#4387) 2024-07-18 08:03:37 +00:00
shuiyisong
8ab6136d1c chore: support pattern as pipeline key name (#4368) 
* chore: add pattern to processor key name

* fix: typo

* refactor: test
 2024-07-18 03:32:26 +00:00
Weny Xu
e39f49fe56 fix: ensure keep alive is completed in time (#4349) 
* fix: ensure keep alive is completed in time

* chore: apply suggestions from CR

* chore: use write runtime

* refactor: set META_LEASE_SECS to 5

* chore: set etcd replicas to 1

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* fix: set `MissedTickBehavior::Delay`

* chore: apply suggestions from CR
 2024-07-17 06:14:45 +00:00
discord9
c595a56ac8 test(flow): ignore flow tests for now (#4377) 
* tests: ignore flow tests for now

* chore: typo

* test: remove file altogether

* fix: last_value sqlness test regex

* fix: last_value part REDACTED
v0.9.0 		2024-07-16 10:07:21 +00:00
Yingwen
d6c7b848da chore: bump version to v0.9.0 (#4376) 
* chore: bump version to v0.9.0

* chore: format
 2024-07-16 07:50:45 +00:00
Lei, HUANG
2010a2a33d feat: Add caching for last row reader and expose cache manager (#4375) 
* Add caching for last row reader and expose cache manager
 - Implement `RowGroupLastRowCachedReader` to handle cache hits and misses for last row reads.

* Add projection field to SelectorResultValue and refactor RowGroupLastRowReader

 - Introduced `projection` field in `SelectorResultValue` to store projection indices.
 2024-07-16 07:13:39 +00:00
dennis zhuang
be3ea0fae7 feat: improve datafusion external error and mysql error (#4362) 
* feat: improve datafusion external error and mysql error

* chore: address CR comments and fix tests

---------

Co-authored-by: evenyag <realevenyag@gmail.com>
 2024-07-16 07:01:09 +00:00
Lei, HUANG
7b28da277d refactor: LastRowReader to use LastRowSelector (#4374) 
Refactor LastRowReader to use LastRowSelector
 - Replaced `last_batch` in `LastRowReader` with `LastRowSelector`.
v0.9.0-b2c5f8eef-20240716-1721102282 		2024-07-16 03:47:41 +00:00
discord9
b2c5f8eefa test: more sleep when flow insert makes it serial (#4373) 
* tests: more sleep

* fix: wait more time

* fix: update result file
 2024-07-16 03:36:31 +00:00
JohnsonLee
072d7c2022 feat: introduce 'pg_catalog.pg_type' (#4332) 
* WIP: pg_catalog

* refactor: move memory_table to crate public level to reuse it in pgcatalog

* refactor: new system_schema mod to manage implementation of information_schema and pg_catalog

* feat: pg_catalog.pg_type

* fix: remove unused code to avoid warning

* test: add pg_catalog sqlness test

* feat: pg_catalog_cache in system_catalog

* fix: integration test

* test: rollback unit test

* refactor: mix pg_catalog table_id with old ones

* fix: add todo information

* tests: rerun sqlness

---------

Co-authored-by: johnsonlee <johnsonlee@localhost.localdomain>
 2024-07-15 17:41:08 +00:00
Yingwen
7900367433 test: replace unstable output of last value test (#4371) 
test: replace unstable output
 2024-07-15 16:51:16 +00:00
Lei, HUANG
9fbc4ba649 feat: add PruneReader for optimized row filtering (#4370) 
* Add PruneReader for optimized row filtering and error handling

 - Introduced `PruneReader` to replace `RowGroupReader` for optimized row filtering.

* Commit Message:

 Make ReaderMetrics fields public for external access

* Add row selection support to SeqScan and FileRange readers

 - Updated `SeqScan::build_part_sources` to accept an optional `TimeSeriesRowSelector`.

* Refactor `scan_region.rs` to remove unnecessary cloning of `series_row_selector`. Enhance `file_range.rs` by adding `select_all` method to check if all rows in a row group are selected, and update the logic in `reader` method to use `LastRowReader` only when all rows are
 selected and no DELETE operations are present.

* Commit Message:

Enhance PruneReader and ParquetReader with reset functionality and metrics handling

Summary:

 • Made Source enum public in prune.rs.

* chore: Update src/mito2/src/sst/parquet/reader.rs

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-07-15 14:23:34 +00:00
Yingwen
2e7b12c344 feat: add a cache for last value result in row group (#4369) 
* feat: add selector result cache to cache manager

* feat: expose config
 2024-07-15 12:33:36 +00:00
discord9
2b912d93fb feat: flow perf&fix df func call (#4347) 
* feat: flow perf&fix df func call

feat: make source sender `send` non-blocking

feat: better control of flow worker freq

feat: support transform interval

fix: const folding df func args&tests

tests: update cast const fold

chore: adjust flow work's freq

refactor: batch split

feat: adaptive run freq flow worker&check for errors

chore: better debug log

* refactor: per review

* chore: per zc's review

* chore: per bot review

* chore: remove some `TODO` completed

* docs: add comments for a test
 2024-07-15 09:20:04 +00:00
Zhenchi
04ac0c8da0 feat(fulltext_index): integrate full-text indexer with parquet reader (#4348) 
* feat(fulltext_index): integrate full-text indexer with parquet reader

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* disable reload

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: range allow exceeding total row

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: unit tests in index

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: prune row groups

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: rename creator

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: sst fulltext index

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: address comment

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-07-15 08:14:44 +00:00
dennis zhuang
64cad4e891 feat: tweak error and status codes (#4359) 
* feat: tweak status codes

* fix: typo

* fix: by cr comments
 2024-07-15 07:50:16 +00:00
Yingwen
20d9c0a345 fix: scan hint checks order asc (#4365) 
* fix: order by asc check

* feat: print selector in explain

* test: move last_value opt test to standalone

* test: sqlness remove space

* test: update regex for datetime

* test: fix partitioning

* chore: update comment

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-07-15 07:49:14 +00:00
Lanqing Yang
9501318ce5 feat: support show views statement (#4360) 
This commit enables show view statement which will display a list of views
names.
 2024-07-15 07:24:27 +00:00
Ning Sun
b8bd8456f0 fix: remove path label for cache store (#4336) 
* fix: remove path label for cache store

* fix: ignore path label for intermediatemanager

* refactor: remove unused object store
 2024-07-15 03:34:19 +00:00
tison
4b8b04ffa2 chore: update project slogan (#4361) 
* chore: update project slogan

Signed-off-by: tison <wander4096@gmail.com>

* keep in oneline

Signed-off-by: tison <wander4096@gmail.com>

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* Update README.md

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-07-15 03:02:44 +00:00
Lanqing Yang
15ac8116ea feat: adding information_schema.views table (#4342) 
This commit introduces information_schema.views table. The VIEWS table provides
information about views in databases.
 2024-07-14 09:50:19 +00:00
Zhenchi
377a513690 feat(index): distinguish different types of index metrics (#4337) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-07-14 08:03:09 +00:00
Yingwen
5a1732279b feat: Implement reader that returns the last row of each series (#4354) 
* feat: last row reader

* feat: scan use last row reader

* test: test last row selector

* chore: update comment
 2024-07-12 14:40:06 +00:00
Ruihang Xia
16075ada67 feat: impl optimizer rule to handle last_value case (#4357) 
* feat: impl optimizer rule to handle last_value case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/optimizer/scan_hint.rs

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* split methods

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-07-12 14:36:23 +00:00
shuiyisong
67dfdd6c61 feat: support text/plain format for log ingestion (#4300) 
* feat: support text/plain format of log input

* refactor: pipeline query and delete using dataframe api

* chore: minor refactor

* refactor: skip jsonify when processing plan/text

* refactor: support array(string) as pipeline engine input
 2024-07-12 09:17:15 +00:00
irenjj
9f2d53c3df refactor: Remove the StandaloneKafkaConfig struct (#4253) 
* refactor: Remove the StandaloneKafkaConfig struct

* remove the redundant assignment

* remove rudundant struct

* simplify replication_factor

* add KafkaTopicConfig

* fix check

* fix check

* fix check

* add flatten with

* revert config.md

* fix test params

* fix test param

* fix missing params when provider is kafka

* remove unsed files

* remove with prefix

* fix doc

* fix test

* fix clippy
 2024-07-12 08:17:18 +00:00
Weny Xu
05c7d3eb42 docs(config): add enable_region_failover option to configuration (#4355) 
docs(config): Add enable_region_failover option to configuration
 2024-07-12 08:09:25 +00:00
Ruihang Xia
63acc30ce7 perf: fine–tuned plan steps (#4258) 
* perf: fine–tuned plan steps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle explain plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle explain plan again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-07-12 06:56:13 +00:00
tison
285ffc5850 fix: build info should use build time env var (#4343) 
* fix: build info should use build time env var

Signed-off-by: tison <wander4096@gmail.com>

* catch up

Signed-off-by: tison <wander4096@gmail.com>

* fixup lifetime

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* fix more

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-07-12 03:08:30 +00:00
dennis zhuang
ab22bbac84 feat: impl drop view (#4231) 
* feat: impl drop view

* fix: metric name

* fix: comments

* test: add DropViewProcedure test

* test: drop view meets a table

* test: update sqlness tests by drop view

* feat: apply suggestion from AI

* chore: apply suggestion

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* chore: apply suggestion

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* chore: apply suggestion

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* fix: TYPE_NAME for DropFlowProcedure

---------

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-07-11 19:53:54 +00:00
Weny Xu
7ad248d6f6 fix(config): enable file engine by default (#4345) 
* fix: enable file engine by default

* fix: fix tests
 2024-07-11 17:35:18 +00:00