rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-13 08:43:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,195 Commits 104 Branches 227 Tags
v0.3.2
Commit Graph

143 Commits

Author SHA1 Message Date
Weny Xu
ee16262b45 feat: add create table procedure (#1845) 
* feat: add create table procedure

* feat: change table_info type from vec u8 to RawTableInfo

* feat: return create table status

* fix: fix uncaught error

* refactor: use a notifier to respond to callers

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: add comment

* chore: apply suggestions from CR

* refacotr: make CreateMetadata step after DatanodeCreateTable step
 2023-07-04 22:24:43 +08:00
Ruihang Xia
c77b94650c refactor: remove Table::scan method (#1855) 
* remove scan method

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-30 12:13:14 +08:00
Lei, HUANG
559d1f73a2 feat: push all possible filters down to parquet exec (#1839) 
* feat: push all possible filters down to parquet exec

* fix: project

* test: add ut for DatafusionArrowPredicate

* fix: according to CR comments
 2023-06-28 20:14:37 +08:00
Lei, HUANG
f287d3115b chore: replace result assertions (#1840) 
* s/assert!\((.*)\.is_ok\(\)\);/\1.unwrap\(\);/g

* s/assert!\((.*)\.is_some\(\)\);/\1.unwrap\(\);/g
 2023-06-27 19:14:48 +08:00
LFC
fcff66e039 chore: deny unused results (#1825) 
* chore: deny unused results

* rebase
 2023-06-27 15:33:53 +08:00
Ruihang Xia
a95f8767a8 refactor: merge catalog provider & schema provider into catalog manager (#1803) 
* move  to expr_factory

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move configs into service_config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move GrpcQueryHandler into distributed.rs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile and test in catalog sub-crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix table-procedure compile and test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix query compile and tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix datanode compile and tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix catalog/query/script/servers compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix frontend compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix nextest except information_schema

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* support information_schema

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix merge errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other structs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change deregister_table's return type to empty tuple

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-26 15:08:59 +08:00
Yingwen
fd412b7b07 refactor!: Uses table id to locate tables in table engines (#1817) 
* refactor: add table_id to get_table()/table_exists()

* refactor: Add table_id to alter table request

* refactor: Add table id to DropTableRequest

* refactor: add table id to DropTableRequest

* refactor: Use table id as key for the tables map

* refactor: use table id as file engine's map key

* refactor: Remove table reference from engine's get_table/table_exists

* style: remove unused imports

* feat!: Add table id to TableRegionalValue

* style: fix cilppy

* chore: add comments and logs
 2023-06-25 15:05:20 +08:00
Ruihang Xia
62f660e439 feat: implement metrics for Scan plan (#1812) 
* add metrics in some interfaces

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* calc elapsed time and rows

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-25 14:06:50 +08:00
Ruihang Xia
23bf55a265 fix: __field__ matcher on single value column (#1805) 
* fix error text and field_column_names

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add empty line

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* improve style

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-21 10:59:58 +08:00
Lei, HUANG
16c1ee2618 feat: incremental database backup (#1240) 
* feat: incremental database backup

* chore: rebase develop

* chore: move backup to StatementExecutor

* feat: copy database parser

* chore: remove some todos

* chore: use timestamp string instead of i64 string

* fix: typo
 2023-06-20 18:26:55 +08:00
LFC
cbc2620a59 feat: start region alive keepers (#1796) 
* feat: start region alive keepers
 2023-06-20 15:45:29 +08:00
LFC
2dd86b686f feat: extend region leases in Metasrv (#1784) 
* feat: extend region leases in Metasrv

* fix: resolve PR comments
 2023-06-19 19:55:59 +08:00
LFC
128c6ec98c feat: region alive keeper in Datanode (#1780) 2023-06-19 14:50:33 +08:00
Lei, HUANG
7961de25ad feat: persist compaction time window (#1757) 
* feat: persist compaction time window

* refactor: remove useless compaction window fields

* chore: revert some useless change

* fix: some CR comments

* fix: comment out unstable sqlness test

* revert commented sqlness
 2023-06-13 10:15:42 +08:00
Ruihang Xia
8c78368374 refactor: replace #[snafu(backtrace)] with Location (#1753) 
* remove snafu backtrace

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-12 11:55:33 +08:00
Ruihang Xia
e5b6f8654a feat: optimizer rule to pass expected output ordering hint (#1675) 
* move type convertsion rule into optimizer dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement order_hint rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* it works!

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use column name instead

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* accomplish test case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update lock file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-02 03:43:51 +00:00
Zheming Li
5467ea496f feat: Add column supports at first or after the existing columns (#1621) 
* feat: Add column supports at first or after the existing columns

* Update src/common/query/Cargo.toml

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-06-01 02:13:00 +00:00
Ruihang Xia
ac3666b841 chore(deps): bump arrow/parquet to 40.0, datafuson to the latest HEAD (#1677) 
* fix compile error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deprecated substrait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update deps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* downgrade opendal to 0.33.1

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change finish's impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore failing cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-31 18:55:02 +08:00
Weny Xu
0460f3ae30 test: add write test for region failover (#1673) 
* test: add write test for region failover

* test: add written data assertion after failover

* test: support more storage types
 2023-05-31 15:42:00 +08:00
Yingwen
c5f6d7c99a refactor: update proto and rename incorrect region_id fields (#1670) 2023-05-30 15:19:04 +09:00
Weny Xu
bb1b71bcf0 feat: acquire table_id from region_id (#1656) 
feat: acquire table_id from region_id
 2023-05-30 03:36:47 +00:00
Ruihang Xia
b27c569ae0 refactor: add scan_to_stream() to Table trait to postpone the stream generation (#1639) 
* add scan_to_stream to Table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl parquet stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reorganise adapters

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement scan_to_stream for mito table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add location info

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: table scan

* UT pass

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl project record batch

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix information schema

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove one todo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix errors generated by merge commit

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add output_ordering method to record batch stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix rustfmt

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enhance error types

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Lei, HUANG <mrsatangel@gmail.com>
 2023-05-29 20:03:47 +08:00
dennis zhuang
7c55783e53 feat!: reorganize the storage layout (#1609) 
* feat: adds data_home to DataOptions

* refactor: split out object store stuffs from datanode instance

* feat: move data_home into FileConfig

* refactor: object storage layers

* feat: adds datanode path to procedure paths

* feat: temp commit

* refactor: clean code

* fix: forgot files

* fix: forgot files

* Update src/common/test-util/src/ports.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update tests/runner/src/env.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix: compile error

* chore: cr comments

* fix: dependencies order in cargo

* fix: data path in test

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-05-23 13:58:26 +08:00
Weny Xu
9f67ad8bce fix: fix doesn't release closed regions issue (#1596) 
* fix: fix close region issue

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* refactor: remove close method from Region trait

* chore: remove PartialEq from CloseTableResult
 2023-05-23 11:40:12 +08:00
Lei, HUANG
8e7ec4626b refactor: remove useless error (#1624) 
* refactor: remove useless

* fix: remove useless error variant
 2023-05-22 22:55:27 +08:00
Vanish
8764ce7845 feat: add delete WAL in drop_region (#1577) 
* feat: add delete WAL in drop_region

* chore: fix typo err.

* feat: mark all SSTs deleted and remove the region from StorageEngine's region map.

* test: add test_drop_region for StorageEngine.

* chore: make clippy happy

* fix: fix conflict

* chore: CR.

* chore: CR

* chore: fix clippy

* fix: temp file life time
 2023-05-18 18:02:34 +08:00
Weny Xu
66903d42e1 feat: implement OpenTableHandler (#1567) 
* feat: implement OpenTableHandler

* chore: apply suggestion from CR

* chore: apply suggestion from CR
 2023-05-15 10:47:28 +08:00
Weny Xu
fbb7db42aa chore: unify code styling (#1523) 2023-05-10 11:10:39 +08:00
discord9
2c82ded975 feat: table metrics (#1469) 
* feat: Statistic

* add todo

* fmt: cargo fmt

* feat: some simple impl for MemTable

* chore: a try on adding statistics

* Update src/table/src/stats.rs

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* docs: fix typo

* newlines unnecessary

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-05-06 14:59:49 +08:00
Lei, HUANG
fb9978e95d refactor: catalog (#1454) 
* wip

* add schema_async

* remove CatalogList

* remove catalog provider and schema provider

* fix

* fix: rename table

* fix: sqlness

* fix: ignore tonic error metadata

* fix: table engine name

* feat: rename catalog_async to catalog

* respect engine name in table regional value when deregistering tables

* fix: CR
 2023-04-26 08:36:40 +00:00
Weny Xu
f2167663b2 feat: support to create external table (#1372) 
* feat: support to create external table

* chore: apply suggestions from CR

* test: add create external table without ts type

* chore: apply suggestions from CR

* fix: fix import typo

* refactor: move consts to table crate

* chore: apply suggestions from CR

* refactor: rename create_table_schema
 2023-04-24 14:43:12 +08:00
LFC
17daf4cdff feat: support "delete" in distributed mode (#1441) 
* feat: support "delete" in distributed mode

* fix: resolve PR comments
 2023-04-24 12:07:50 +08:00
shuiyisong
7c6754d03e feat: meter write request (#1447) 
* chore: add write meter

* chore: update meter macro

* chore: update meter framework url to https
 2023-04-24 11:42:06 +08:00
Weny Xu
d374859e24 refactor: replace Copy Format with datasource Format (#1435) 
* refactor: replace Copy Format with datasource Format

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2023-04-23 08:31:54 +00:00
Yingwen
1ee9ad4ca1 feat: manage multiple engine procedure in the engine manager (#1434) 
* feat(table): Add engine procedure to engine manager

* feat(datanode): Get engine procedure from engine manager

* feat(table-procedure): Add source error to SubprocedureFailed

* test: Enable procedure in tests and pass all tests

* style(table-procedure): Fix clippy
 2023-04-23 10:04:09 +08:00
dennis zhuang
2a9f482bc7 feat: show create table (#1336) 
* temp commit

* feat: impl Display for CreateTable statement

* feat: impl show create table for standalone

* fix: forgot show.rs

* feat: clean code

* fix: typo

* feat: impl show create table for distributed

* test: add show create table sqlness test

* fix: typo

* fix: sqlness tests

* feat: render partition rules for distributed table

* Update src/sql/src/statements.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/sql/src/statements.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/sql/src/statements.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/sql/src/statements/create.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by CR comments

* fix: compile error

* fix: missing column comments and extra table options

* test: add show create table test

* test: add show create table test

* chore: timestamp precision

* fix: test

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-04-21 11:37:16 +08:00
Yingwen
9cd2cf630d feat: procedures for file table engine (#1417) 
* refactor: Add table_ref() to requests as their methods

* feat: Add CreateImmutableFileTable

* feat: Add DropImmutableFileTable

* feat: Implement TableEngineProcedure for ImmutableFileTableEngine

* feat: Add common-procedure-test crate

* refactor: mito engine use common-procedure-test to test procedures

* test: Add test for create and drop table

* chore: Address review comments
 2023-04-20 18:52:44 +08:00
Ruihang Xia
7152a1b79e feat: expose output_ordering on scan plan (#1425) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-20 17:58:48 +08:00
Yingwen
d988b43996 feat: Add drop table procedure to mito (#1377) 
* feat: Add drop table procedure to mito

* feat: remove table from engine and then close it
 2023-04-14 13:09:38 +08:00
Lei, HUANG
3e8ec8b73a fix: avoid panic when no region found in table (#1359) 2023-04-11 16:58:18 +08:00
Weny Xu
29c6155ae3 feat: introduce file table engine (#1323) 
* feat: introduce file table engine

* chore: apply cr suggestions

* refactor: refactor immutable manifest

* chore: apply cr suggestions

* refactor: refactor immutable manifest

* chore: apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* chore: apply suggestions from CR

* chore: apply suggestions from code review

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-04-10 12:03:36 +08:00
Ruihang Xia
da66138e80 refactor(error): remove backtrace, and introduce call-site location for debugging (#1329) 
* wip: global replace

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix warnings

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unneeded tests of errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix ErrorExt trait implementator

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix warnings

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyo3 tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-06 04:06:00 +00:00
Eugene Tolbakov
59f7630000 feat: initial changes for compaction_time_window field support (#1083) 
* feat(compaction_time_window): initial changes for compaction_time_window field support

* feat(compaction_time_window): move PickerContext creation

* feat(compaction_time_window): update region descriptor, fix formatting

* feat(compaction_time_window): add minor enhancements

* feat(compaction_time_window): fix failing test

* feat(compaction_time_window):  return an error instead silently skip for the user provided compaction_time_window

* feat(compaction_time_window): add TODO reminder
 2023-04-06 10:32:41 +08:00
Ruihang Xia
99353c6ce7 refactor: rename "value" semantic type to "field" (#1326) 
* global replace

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change desc table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-04 11:14:28 +00:00
Weny Xu
451f9d2d4e feat: support multi table engines (#1277) 
* feat: support multi table engines

* refactor: adapt SqlHandler to support multiple table engines

* refactor: refactor TableEngineManager

* chore: apply review suggestions

* chore: apply review suggestions

* chore: apply review suggestions

* chore: snafu context styling
 2023-04-03 14:49:12 +00:00
Yingwen
dee20144d7 feat: Implement procedure to alter a table for mito engine (#1259) 
* feat: wip

* fix: Fix CreateMitoTable::table_schema not initialized from json

* feat: Implement AlterMitoTable procedure

* test: Add test for alter procedure

* feat: Register alter procedure

* fix: Recover procedures after catalog manager is started

* feat: Simplify usage of table schema in create table procedure

* test: Add rename test

* test: Add drop columns test
 2023-03-31 14:40:54 +08:00
Lei, HUANG
15ee4ac729 fix: noop flush impl for numbers table (#1247) 
* fix: noop flush impl for numbers table

* fix: clippy
 2023-03-27 10:54:07 +08:00
LFC
92963b9614 feat: execute "delete" in query engine (in the form of "LogicalPlan") (#1222) 
fix: execute "delete" in query engine (in the form of "LogicalPlan")
 2023-03-24 12:11:58 +08:00
LFC
b77b561bc8 refactor: execute insert with select in query engine (#1181) 
* refactor: execute insert with select in query engine

* fix: resolve PR comments
 2023-03-23 10:38:26 +08:00
dennis zhuang
8fb97ea1d8 fix: losing region numbers after altering table (#1209) 2023-03-21 11:19:43 +08:00