Commit Graph

1936 Commits

Author SHA1 Message Date
Lei, Huang
deb7d5fc2c fix: opentsdb/influxdb tags are not put to primary key indices (#506) 2022-11-15 11:06:51 +08:00
Jiachun Feng
3f12f5443d feat: meta election (#492)
* feat: meta election

* feat: election by etcd

* chore: redirect on re-election

* chore: by cr

* chore: by cr

* chore: by cr

* chore: rename CI
2022-11-15 11:04:15 +08:00
Ruihang Xia
a7d311e480 chore: enlarge CI and disable test job (#503)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2022-11-15 10:53:07 +08:00
Ning Sun
57304ec091 docs: remove database creation (#500)
* docs: remove database creation

* docs: add project status
2022-11-15 08:22:29 +08:00
dennis zhuang
448e8f139e fix: table and database conflicts (#491)
* fix: table conflicts in different database, #483

* feat: support db query param in prometheus remoting read/write

* feat: support db query param in influxdb line protocol

* fix: make schema_name work in gRPC

* fix: table data path

* fix: table manifest dir

* feat: adds opendal logging layer to object store

* Update src/frontend/src/instance.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* Update src/frontend/src/instance.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* Update src/servers/src/line_writer.rs

Co-authored-by: Lei, Huang <6406592+v0y4g3r@users.noreply.github.com>

* Update src/servers/src/line_writer.rs

Co-authored-by: Lei, Huang <6406592+v0y4g3r@users.noreply.github.com>

* fix: compile error

* ci: use larger runner for running coverage

* fix: address already in use in test

Co-authored-by: LFC <bayinamine@gmail.com>
Co-authored-by: Lei, Huang <6406592+v0y4g3r@users.noreply.github.com>
2022-11-14 23:16:52 +08:00
Ning Sun
76732d6506 fix: add more parameters to postgresql for python client (#493) 2022-11-14 21:55:26 +08:00
Ning Sun
74c236a308 feat: stream write for postgresql query results (#472) 2022-11-14 21:50:11 +08:00
Ning Sun
c673debc89 feat: Update Http SQL api for dashboard requirements (#474)
* feat: make sql api output a vector to support multi-statement

* feat: add execution_time_ms to http sql and script api

* fix: use u128 for execution time

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix: lint error

Co-authored-by: Yingwen <realevenyag@gmail.com>
2022-11-14 21:40:31 +08:00
Yingwen
281eae9f44 fix: Fix filtering out rows incorrectly during dedup phase (#484)
* fix: dedup should not mark element as unneeded

It should only mark element as selected, because some column of
different rows may have same value.

* refactor: Rename dedup to find_unique

As the original `dedup` method only mark bitmap to true when it finds
the element is unique, so `find_unique` is more appropriate for its
name.

* test: Renew bitmap in test_batch_find_unique

* chore: Update comments
2022-11-14 21:40:17 +08:00
Ning Sun
fdae67b43e docs: Simplify code in readme (#488)
* docs: simplify readme

* docs: update content

* docs: add start docker section

* docs: add c/c++ toolchain description

* docs: minor tweak

* docs: minor tweak again

* docs: address review comments
2022-11-14 21:18:23 +08:00
Ruihang Xia
ab9b1a91d4 chore: turn-off codecov's patch comment (#498)
* chore: turn-off codecov's patch comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: fix style

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2022-11-14 21:18:14 +08:00
Lei, Huang
4e7efbbe7e fix: insert batch missing semantic type (#499) 2022-11-14 21:18:01 +08:00
Yingwen
508f4cdfd0 fix: Fix test_insert_and_select hangs occasionally (#496)
* fix: Also handles admin request in another runtime

* chore: Describe why executes admin request in another runtime

* test: Enable test_insert_and_select
2022-11-14 21:11:25 +08:00
dennis zhuang
68b299e04a fix: apply recovered metadata after last WAL entry (#461)
* fix: apply recovered metadata after last WAL entry

* fix: condition error
2022-11-14 20:43:47 +08:00
Lei, Huang
c90832ea6c feat: distribute mode support auto create table (#489) 2022-11-14 19:53:35 +08:00
LFC
d10e45f4aa feat: distributed query in Frontend (#486)
* feat: distributed query in Frontend

Co-authored-by: luofucong <luofucong@greptime.com>
2022-11-14 18:15:49 +08:00
shuiyisong
dcd5e34dbd feat: generating context in http middleware & mysql auth method (#453) 2022-11-14 17:24:11 +08:00
xiaomin tang
7e49493e34 docs: add more sections to readme (#478)
* docs: add badges & logo

* docs: add What_is_GreptimeDB section

* docs: add Community&Documentation&License section

* docs: simplify name of CI badge
2022-11-14 16:46:09 +08:00
LFC
e7b4a00ef0 feat: create distributed table in Frontend (#475)
* feat: create distributed table in Frontend

* fix: some table creation issues (#482)

Co-authored-by: luofucong <luofucong@greptime.com>
Co-authored-by: Lei, Huang <6406592+v0y4g3r@users.noreply.github.com>
2022-11-14 15:49:25 +08:00
Yingwen
ef12bb7f24 ci: Fix codecov.yml syntax (#464) 2022-11-14 14:21:09 +08:00
Lei, Huang
70442f6810 feat: add mysql protocol handler back to datanode for debugging (#479) 2022-11-14 13:15:44 +08:00
Lei, Huang
fae331d2ba feat: Move create table logic to frontend (#455)
* refactor: dependency, from frontend depends on datanode to datanode depends on frontend

* wip: start frontend in datanode

* wip: migrate create database to frontend

* wip: impl alter table

* fix: CR comments

* feat: add table id and region ids field to CreateExpr

* chore: rebase develop

* refactor: frontend catalog should set from datanode

* feat: gRPC AddColumn request support add multi columns

* wip: move create table and create-on-insertion to frontend

* wip: error handling

* fix: some unit tests

* fix: all unit tests

* chore: merge develop

* feat: add create/alter-on-insertion to dist_insert/sql_dist_insert

* fix: add region number/catalog/schema to InsertExpr

* feat: add handle_create_table/handle_create_database...

* fix: remove catalog from insert expr

* fix: CR comments

* fix: when running in standalone mode, mysql opts and postgres opts should pass to frontend so that auctually running service can change the port to listen on

* refactor: add a standalone subcommand, move frontend start stuff to cmd package

* chore: optimize create table failure logs

* docs: change readme

* docs: update readme
2022-11-14 10:54:35 +08:00
fys
488eabce4a feat: support standalone and distributed insert in frontend (#473)
* feat: support standalone and distributed insert in frontend

* cr
2022-11-13 11:57:23 +08:00
Lei, Huang
2d869e1e43 refactor: datanode starts frontend (#471)
* refactor: dependency, from frontend depends on datanode to datanode depends on frontend

* wip: start frontend in datanode

* wip: migrate create database to frontend

* wip: impl alter table

* fix: CR comments
2022-11-12 21:07:18 +08:00
Ning Sun
0d4c191a06 fix: improve postgresql protocol implementation and fix time/date format (#452)
* feat: add server_version as postgresql jdbc connector requires

* refactor: do not require password at the moment

* fix: correct datetime output as required by postgresql

* docs: corrected timestamp on our readme

* refactor: simplify import

* fix: address review issues
2022-11-11 21:28:28 +08:00
zyy17
1d78f8db1f ci: use larger runner in release building (#467) 2022-11-11 19:04:04 +08:00
LFC
f375e18a76 feat: table route cache (#462)
* feat: table route cache

Co-authored-by: luofucong <luofucong@greptime.com>
2022-11-11 18:54:56 +08:00
Ruihang Xia
e30879f638 feat: Remove memtable's time bucket (#442)
* refactor: partially replace MemtableSet with Memtable

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove MemtableWithMeta and MemtableSet in non-test mod

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove dead code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* make test compile 🤣

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix broken tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* make all tests pass

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippys

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove redundant clone

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update comment

Co-authored-by: Yingwen <realevenyag@gmail.com>

* resolve review comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
2022-11-11 18:02:34 +08:00
dennis zhuang
74ea529d1a feat: move time index metadata from schema into field (#444)
* feat: move time index metadata from schema into field

* chore: remove useless code

* test: test select with column alias

* fix: conflicts with develop branch

* test: add test

* test: order by timestamp to ensure query results order

* fix: comment
2022-11-11 15:36:27 +08:00
dennis zhuang
e7b4d24df5 feat: create database (#451)
* feat: parsing create database statement

* feat: impl create database in datanode

* feat: supports insert into catalog.schema.table

* fix: conflicts with develop branch

* test: create database then insert and query

* fix: grpc schema provider

* feat: use CatalogManager::register_schema instead of CatalogProvide::register_schema

* refactor: revert InsertExpr catalog_name and schema_name

* fix: revert database.proto

* fix: revert client cargo

* feat: accepts schema.table as table name in sql

Co-authored-by: Lei, HUANG <mrsatangel@gmail.com>
2022-11-11 14:15:38 +08:00
Yingwen
d5ae5e6afa fix: Ignore test_insert_and_select (#459)
It sometimes hangs in CI
2022-11-11 12:02:29 +08:00
Ruihang Xia
a179481966 docs: refine contributing.md (#450)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
v0.1.0-alpha.3
2022-11-10 19:29:38 +08:00
Yingwen
3ae7362f58 ci: Upgrade ci dependencies and switch to nextest (#446)
* ci: Upgrade rust-cache to v2.2.0

v2.0.0 uses API that is deprecated

* ci: Use --workspace in cargo llvm-cov

* ci: Replace actions-rs/toolchain by dtolnay/rust-toolchain

actions-rs/toolchain is under inactive maintenance, it uses node12 that
would soon becomes deprecated

* ci: Replace actions-rs/cargo by run

* ci: rust-cache and cleanup-disk-action try not to specific full version

* ci: Use nextest

Also sets timeout for nextest to avoid a test hanging too long

* ci: Upgrade actions/checkout to v3

To upgrade node from 12 to 16

* ci: Specific cleanup-disk-action version
2022-11-10 19:21:53 +08:00
Lei, Huang
2e9c9f2176 fix: sort system catalog entries to ensure catalog entry are firstlt processed (#449) 2022-11-10 19:21:27 +08:00
zyy17
89b942798c feat: add 'scripts/install.sh' to make the installation more easy (#443) 2022-11-10 19:10:49 +08:00
Lei, Huang
952e646e1d refactor: move insert-to-create logic to a separate crate (#447) 2022-11-10 17:16:40 +08:00
Jiachun Feng
23f0320ffb feat: route for insert&select (#425)
* feat: route for insert/select

* chore: remove redundant tests

* chore: add fouce quit loop count limit to sequence

* chore: by code review

* chore: use ref with TableRouteKey

* chore: minor refactor
2022-11-10 16:13:15 +08:00
xiaomin tang
49403012b5 docs: Add Apache 2.0 license (#434) 2022-11-10 14:21:24 +08:00
zyy17
b87d5334d1 ci: modify docker image tag and push the latest tag image (#419) 2022-11-10 13:25:45 +08:00
Yingwen
fa4a74a408 ci: Use cargo-llvm-cov to generate coverage data (#438)
* ci: Use cargo-llvm-cov to generate coverage data

* ci: Remove usage of lld in coverage.yml

* ci: Move codecov.yml to project root
2022-11-10 13:25:18 +08:00
dennis zhuang
e62b302fb2 feat: some improvements on python coprocessor (#423)
* feat: supports list array in arrow_array_get

* feat: supports string and list type conversions in python coprocessor

* test: add test cases for returning list in coprocessor
2022-11-10 11:53:27 +08:00
Lei, Huang
6288fdb6bc feat: frontend catalog (#437)
* feat: add frontend catalog
2022-11-10 11:52:57 +08:00
Yingwen
cefdffff09 fix: CURRENT_TIMESTAMP supports int64 type (#436)
* fix: Fix int64 type not considered in DEFAULT CURRENT_TIMESTAMP() constraint

Also avoid using `ConstantVector` in default constraint, as other user
may try to downcast it to a concrete type, and sometimes may forget to
check whether it is a constant vector.

* test: Add test for writing default value
2022-11-10 11:35:16 +08:00
Lei, Huang
c3776ddd18 fix: stop background task when LogFile is dropped (#440) 2022-11-10 11:33:33 +08:00
fys
056d7cb911 feat: support convert prometheus write request to InsertRequest (#433)
* support convert prometheus to InsertRequest.
2022-11-10 10:48:41 +08:00
Ruihang Xia
16d1132733 chore: github action manually trigger (#439)
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2022-11-09 20:35:01 +08:00
LFC
37dc85a29e feat: add gRPC reflection service (#431)
Co-authored-by: luofucong <luofucong@greptime.com>
2022-11-09 19:13:01 +08:00
fys
d08f8b87a6 feat: convert different protocol to InsertRequest (#426)
* add line_writer and convert insert_stmt to InsertRequest

* support convert influxdb line protocol to InsertRequest

* support convert opentsdb to InsertRequest

* cr
2022-11-09 16:18:54 +08:00
Ning Sun
64a706d6f0 ci: disable lld in release workflow for now (#427) 2022-11-09 14:37:25 +08:00
Yingwen
cf4e876e51 feat: mito table supports RemoveColumns alter kind (#395)
* feat: Support removing columns from mito table

Implements drop column for mito table engine, and adjusts the execution
order of altering table, persists the table manifest first, then alter
the schema of the region.

* feat(storage): Remove duplicate table_info() impl

Table already provides a table_info() now, some downcast in tests are
also no longer needed.

* test: Add tests for add/remove columns

* style(table): Fix clippy

* fix: Find timestamp index by its column name

Previous implementation updates the timestamp index too early, which
would cause the index check that compare the index to remove with
timestamp index failed.

* chore: Remove generated comment in Cargo.toml

* chore: Rename alter to builder_with_alter_kind

* refactor: Alloc new column from TableMeta

* style: Fix clippy
2022-11-09 11:50:02 +08:00