rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-07 13:52:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,347 Commits 104 Branches 226 Tags
d7c3c8e124489fef3e04152cc6e63dd8926bb50f
Commit Graph

3347 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dennis zhuang
d7c3c8e124 fix: physical table statistics info (#4975) 
* fix: physical table statistics info

* refactor: is_physical_table

* fix: remove file
 2024-11-13 14:29:44 +00:00
Yohan Wal
f4b9eac465 build(deps): switch to upstream jsonb (#4986) 
chore: jsonb
 2024-11-13 11:15:37 +00:00
Yohan Wal
aa6c2de42a refactor: use UNSET instead of enable (#4983) 
* refactor: use UNSERT instead of enable

* fix: test

* chore: comment
 2024-11-13 08:28:20 +00:00
discord9
175fddb3b5 fix: alter table add column id alloc mismatch (#4972) 
* fix: alter table add column id alloc mismatch

* chore: remove debug code

* chore: typos

* chore: error variant

* chore: more checks for invariant

* refactor: better check&explain

* fix: exist column metadata correct

* chore: remove unused error variant
 2024-11-13 07:02:35 +00:00
Lei, HUANG
6afc4e778a refactor(mito): tidy memtable stats (#4982) 
* wip: share same WriteMetrics struct between different memtable implementations

* refactor: extract function to update memtable timestamp range
 2024-11-13 06:57:27 +00:00
Lanqing Yang
3bbcde8e58 feat: support alter twcs compaction options (#4965) 
support alter twcs compression options
 2024-11-13 06:27:32 +00:00
Weny Xu
3bf9981aab refactor: support distinct JSON format and improve type conversions (#4979) 
refactor: support distinct JSON format
 2024-11-13 03:03:51 +00:00
Weny Xu
c47ad548a4 feat: refine region state checks and handle stalled requests (#4971) 
* feat: refine region state checks and handle stalled requests

* test: add tests

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: add comments
 2024-11-13 02:51:27 +00:00
Lin Yihai
0b6d78a527 refactor: consolidate DatanodeClientOptions (#4966) 
refactor!: consolidate `DatanodeClientOptions`
 2024-11-12 09:57:41 +00:00
Zhenchi
d616bd92ef feat: introduce vector type (#4964) 
* feat: introduce vector type

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: fix prepared stmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: add grpc test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: parse vector value

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: column to row

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: merge issue

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: add check for bytes size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update tests/cases/standalone/common/types/vector/vector.sql

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update proto

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: simplify cargo

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: address comment

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-11-12 08:28:44 +00:00
Yohan Wal
84aa5b7b22 feat: alter fulltext options (#4952) 
* feat(WIP): alter fulltext index

Co-Authored-By: irenjj <renj.jiang@gmail.com>

* feat: alter column fulltext option

Co-Authored-By: irenjj <renj.jiang@gmail.com>

* chore: fmt

* test: add unit and integration tests

Co-Authored-By: irenjj <renj.jiang@gmail.com>

* test: update sqlness test

* chore: new line

* chore: lock file update

* chore: apply review comments

* test: update sqlness test

* test: update sqlness test

* fix: convert

* chore: apply review comments

* fix: toml fmt

* fix: tests

* test: add test for mito

* chore: error message

* fix: test

* fix: test

* fix: wrong comment

* chore: change proto rev

* chore: apply review comments

* chore: apply review comments

* chore: fmt

---------

Co-authored-by: irenjj <renj.jiang@gmail.com>
 2024-11-12 03:04:04 +00:00
Zhenchi
cbf21e53a9 feat(puffin): apply range reader (#4928) 
* wip

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(puffin): apply range reader

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: read_vec reduce iteration

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: simplify rangereader for vec<u8>

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: add unit test

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: toml format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-11-12 02:36:38 +00:00
Zhenchi
6248a6ccf5 feat(index): support SQL to specify inverted index columns (#4929) 
* feat(index): support building inverted index for the field column

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(index): support SQL to specify inverted index columns

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: fix sqlness

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: consider compatibility

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* polish

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* compatibility

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: ignore case

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: reduce dup

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-11-11 08:06:23 +00:00
localhost
0e0c4faf0d fix(otlp): replace otlp trace attr type from string to jsonb (#4918) 
* chore: minor update

* chore: replace otlp trace attr type from string to jsonb

* chore: add new util file and remove useless code

* chore: add license header

* chore: remove unused error

* chore: adjust otlp traces column order

* chore: update test

* chore: minor fix

---------

Co-authored-by: shuiyisong <xixing.sys@gmail.com>
v0.10.0-nightly-20241111 		2024-11-08 06:34:49 +00:00
Kaifeng Zheng
1a02fc31c2 fix: json_path_exists null results (#4881) 
* fix: result of nulls

* update test result

* fix null behaviors, add null tests

* update NULL tests

* error handler when parsing json_path

* change the logic to: items' datatype in the input arrays are all the same.

* remove a comment

* refactor: better logic

* drop unnecessary err check

* added an error test case
 2024-11-08 03:01:45 +00:00
Ruihang Xia
8efbafa538 feat: support filter with windowed sort (#4960) 
* feat: support windowed sort with where condition

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix split logic

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* modify fuzz test to reflect logic change

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: handle sort that wont preserving partition

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test case and add more cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-11-08 02:49:36 +00:00
Ruihang Xia
fcd0ceea94 fix: column already exists (#4961) 
* fix: merge fetched logical metadata with existing cache

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix log acquire

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/metric-engine/src/engine/region_metadata.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-11-07 13:25:05 +00:00
jeremyhi
22f31f5929 chore: paginated query region stats (#4942) 2024-11-07 03:01:12 +00:00
Lei, HUANG
5d20acca44 fix: round euclidean result in sqlness (#4956) 2024-11-07 02:29:49 +00:00
Yingwen
e3733344fe fix: do not pick compacting/expired files (#4955) 2024-11-06 21:38:26 +00:00
dennis zhuang
305767e226 fix: bugs introduced by alter table options (#4953) 
* fix: ChangeTableOptions display

* fix: partition number disappear after altering table options

* Update src/table/src/metadata.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-11-06 19:56:49 +00:00
dennis zhuang
22a662f6bc docs: add TOC to readme (#4949) 
* docs: add TOC to readme

* chore: remove some links

* chore: tweak TOC

* chore: tweak TOC
 2024-11-06 13:27:00 +00:00
Lin Yihai
1431393fc8 fix: the region_stats API will return an error in instance test (#4951) 2024-11-06 13:15:48 +00:00
localhost
dfe8cf25f9 chore: add json path for pipeline (#4925) 
* chore: add json path for pipeline

* chore: change jsonpath lib verion

* chore: remove useless doc

* chore: fix json path test

* chore: fix pipeline json path test
 2024-11-06 11:44:59 +00:00
Weny Xu
cccd25ddbb chore: fix typos in change log level doc (#4948) 
chore: fix typos in change log level
 2024-11-06 07:49:00 +00:00
discord9
ac387bd2af fix: pprof (#4938) 2024-11-05 09:22:29 +00:00
LFC
2e9737c01d refactor: pass LogicalPlan to promql execution interceptor (#4937) 2024-11-05 08:49:05 +00:00
Ning Sun
a8b426aebe feat: add more geo functions (#4888) 
* chore: add type conversion for array types

* feat: add h3_cells_contains

* refactor: resolve lint issues

* feat: add sphere distance function

* feat: euclidean distance between h3 centroids

* test: round float number

* feat: add more geospatial functions

* test: add tests for geometry functions

* refactor: move wkt function to dedicated module

* feat: add st_area

* refactor: only allow sphere distance between points
 2024-11-05 03:44:25 +00:00
jeremyhi
f3509fa312 chore: minor refactor for weighted choose (#4917) 
* chore: minor refactor for weighted choose

* chore: by comment, remove the fast path of choose_multiple
 2024-11-05 02:55:11 +00:00
Lei, HUANG
3dcd6b8e51 fix: database base ttl (#4926) 
* main:
 Add common-meta dependency and implement SchemaMetadataManager

 - Introduce `common-meta` as a new dependency in `mito2`.
 - Implement `SchemaMetadataManager` for managing schema-level metadata.
 - Update `DatanodeBuilder` and `MitoEngine` to pass `KvBackendRef` for schema metadata management.
 - Add `SchemaMetadataManager` to `RegionWorkerLoop` for compaction handling.
 - Include `SchemaNameKey` usage in compaction-related code.
 - Add `database_metadata_manager` module with `SchemaMetadataManager` struct and associated logic.

* fix/database-base-ttl:
 Refactor metadata management and update compaction logic

 - Remove `database_metadata_manager` and introduce `schema_metadata_manager`
 - Update compaction logic to handle TTL based on schema metadata
 - Adjust tests to use `schema_metadata_manager` for setting up schema options
 - Fix engine creation in tests to pass `kv_backend` explicitly
 - Remove unused imports and apply minor code cleanups

* fix/database-base-ttl:
 Extend CREATE TABLE LIKE to inherit schema options

 - Implement inheritance of database level options for CREATE TABLE LIKE
 - Add schema options to SHOW CREATE TABLE output
 - Refactor create_table_stmt to include schema_options in SQL generation
 - Update error handling to include TableMetadataManagerSnafu

* fix/database-base-ttl:
 Refactor error handling and remove schema dependency in table creation

 - Replace expect with the ? operator for error handling in open_compaction_region
 - Simplify create_logical_tables by removing catalog and schema name parameters
 - Remove unnecessary schema retrieval and merging of schema options in create_table_info
 - Clean up unused imports and redundant code

* fix/database-base-ttl:
 Refactor error handling and update documentation comments

 - Update comment to reflect retrieval of schema options instead of metadata
 - Introduce new error type `GetSchemaMetadataSnafu` for schema metadata retrieval failures
 - Implement error handling for schema metadata retrieval in `find_ttl` function

* fix: toml

* fix/database-base-ttl:
 Refactor SchemaMetadataManager and adjust Cargo.toml dependencies

 - Remove unused imports in schema_metadata_manager.rs
 - Add conditional compilation for SchemaMetadataManager::new
 - Update Cargo.toml to remove "testing" feature from common-meta dependency in main section and add it to dev-dependencies

* fix/database-base-ttl:
 Fix typos in comments and function names across multiple modules

 - Correct spelling of 'parallelism' in region_server, engine, and scan_region modules
 - Amend typo in TODO comment from 'persisent' to 'persistent' in server module
 - Update incorrect test query from 'versiona' to 'version' in federated module tests

* fix/database-base-ttl: Add schema existence check in StatementExecutor for CREATE TABLE operation

* fix/database-base-ttl: Add warning log for failed TTL retrieval in compaction region open function

* fix/database-base-ttl:
 Refactor to use SchemaMetadataManagerRef in Datanode and MitoEngine

 - Replace KvBackendRef with SchemaMetadataManagerRef across various components.
 - Update DatanodeBuilder and MitoEngine to pass SchemaMetadataManagerRef instead of KvBackendRef.
 - Adjust test cases to use get_schema_metadata_manager method for consistency.
 2024-11-05 02:51:32 +00:00
Weny Xu
f221ee30fd fix: violations of elided_named_lifetimes (#4936) 2024-11-04 10:52:39 +00:00
Yohan Wal
fb822987a9 refactor: refactor alter parser (#4933) 
refactor: alter parser
 2024-11-04 09:00:30 +00:00
Weny Xu
4ab6dc2825 feat: support to insert json data via grpc protocol (#4908) 
* feat: support to insert json data via grpc protocol

* chore: handle error

* feat: introduce `prepare_rows`

* chore: fmt toml

* test: add row deletion test

* test: fix unit test

* chore: remove log

* chore: apply suggestions from CR
 2024-11-04 08:55:47 +00:00
dennis zhuang
191755fc42 fix: data_length, index_length, table_rows in tables (#4927) 
* fix: data_length, index_length, table_rows in tables

* feat: table stats only works for mito engine currently

* fix: tests

* fix: typo

* chore: log error when region_stats fails
 2024-11-04 07:44:13 +00:00
Yohan Wal
1676d02149 fix: panic when jsonb corrupted (#4919) 
* refactor: json type update

* test: update test

* fix: convert when needed

* revert: leave sqlness tests unchanged

* fix: fmt

* refactor: just refactor

* Apply suggestions from code review

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* refactor: parse jsonb first

* test: add bad cases

* Update src/datatypes/src/vectors/binary.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fix: fmt

* fix: fix clippy/check

* fix: corrupted jsonb panic

* chore(deps): change to rev

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-11-04 06:55:14 +00:00
dennis zhuang
edc49623de chore: update default cache size to 1Gib (#4923) 
* chore: update default cache size to 1Gib for object storage read/write cache

* feat: update docs

* fix: test
 2024-11-04 03:53:17 +00:00
jeremyhi
9405d1c578 feat: heartbeat_flush_threshold option (#4924) 
* feat: heartbeat_flush_threshold

* chore: rename to flush_stats_factor

* Update src/meta-srv/src/handler/collect_stats_handler.rs
 2024-11-04 03:34:50 +00:00
dennis zhuang
7a4276c24a fix: typo (#4931) 
fix/database-base-ttl:
 Fix typos in comments and function names across multiple modules

 - Correct spelling of 'parallelism' in region_server, engine, and scan_region modules
 - Amend typo in TODO comment from 'persisent' to 'persistent' in server module
 - Update incorrect test query from 'versiona' to 'version' in federated module tests

Co-authored-by: Lei, HUANG <mrsatangel@gmail.com>
 2024-11-04 01:56:11 +00:00
Ruihang Xia
be72d3bedb feat: simple limit impl in PartSort (#4922) 
* feat: simple limit impl in PartSort

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: update time_index method to return a non-optional String

Co-authored-by: Yingwen <realevenyag@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use builtin limit

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add more info to analyze display

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
v0.10.0-nightly-20241104 		2024-11-01 09:25:03 +00:00
discord9
1ff29d8fde chore: short desc markdown about change log level (#4921) 
* chore: tiny doc about change log level

* chore: per review

* chore
 2024-11-01 07:10:57 +00:00
Yingwen
39ab1a6415 feat: get row group time range from cached metadata (#4869) 
* feat: get part range min-max from cache for unordered scan

* feat: seq scan push row groups if num_row_groups > 0

* test: test split

* feat: update comment

* test: fix split test

* refactor: rename get meta data method
 2024-11-01 06:35:03 +00:00
Weny Xu
758ad0a8c5 refactor: simplify WeightedChoose (#4916) 
* refactor: simplify WeightedChoose

* chore: remove unused errors
 2024-10-31 06:22:30 +00:00
Ruihang Xia
8b60c27c2e feat: enhance windowed-sort optimizer rule (#4910) 
* add RegionScanner::metadata

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* skip PartSort when there is no tag column

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add more sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle desc

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: should keep part sort on DESC

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-10-31 06:15:45 +00:00
Yingwen
ea6df9ba49 fix: prune batches from memtable by time range (#4913) 
* feat: add an iter to prune by time range

* feat: filter rows from mem range
 2024-10-31 05:13:35 +00:00
Ning Sun
69420793e2 feat: implement parse_query api (#4860) 
* feat: implement parse_query api

* chore: switch to upstream

* fix: add post method for parse_query

* chore: bump promql-parser

* test: use latest promql ast serialization
 2024-10-30 12:16:22 +00:00
Yingwen
0da112b335 chore: provide more info in check batch message (#4906) 
* chore: provide more info in check message

* chore: set timeout to 240s

---------

Co-authored-by: WenyXu <wenymedia@gmail.com>
 2024-10-30 11:56:10 +00:00
dennis zhuang
dcc08f6b3e feat: adds the number of rows and index files size to region_statistics table (#4909) 
* feat: adds index size to region statistics

* feat: adds the number of rows for region statistics

* test: adds sqlness test for region_statistics

* fix: test
 2024-10-30 11:12:58 +00:00
dennis zhuang
a34035a1f2 fix: set transaction variables not working in mysql protocol (#4912) 2024-10-30 10:59:13 +00:00
LFC
fd8eba36a8 refactor: make use of the "pre_execute" in sql execution interceptor (#4875) 
* feat: dynamic definition of plugin options

* rebase

* revert

* fix ci
 2024-10-30 09:16:46 +00:00
Ruihang Xia
9712295177 fix(config): update tracing section headers in example TOML files (#4898) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-10-30 08:31:31 +00:00