JeremyHi
99dda93f0e
feat: sql with influxdb v1 result format ( #2917 )
...
* feat: sql with influxdb v1 result format
* chore: add unit tests
* feat: minor refactor
* chore: by comment
* chore; u128 to u64 since serde can't deser u128 in enum
* chore: by comment
* chore: apply suggestion
* chore: revert suggestion
* chore: try again
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-12-13 16:15:37 +00:00
Yingwen
6a57f4975e
perf(mito): scan SSTs and memtables in parallel ( #2852 )
...
* feat: seq scan support parallelism
* feat: scan region by parallelism in config
* feat: enlarge channel size
* chore: parallel builder logs
* feat: use parallel reader accroding to source num
* chore: 128 channel size
* feat: add fetch cost metrics
* feat: add channel size to config
* feat: builder cost
* feat: logs
* feat: compiler error
* feat: fetch cost
* feat: convert cost
* chore: Revert "feat: logs"
This reverts commit 01e0df2c3a
2023-12-11 06:43:17 +00:00
Niwaka
cfe3a2c55e
feat!: support table ddl for custom storage ( #2733 )
...
* feat: support table ddl for custom_storage
* refactor: rename extract_variant_name to name
* chore: add blank
* chore: keep compatible
* feat: rename custom_stores to providers
* chore: rename
* chore: config
* refactor: add should_retry in client Error
* fix: test fail
* chore: remove unused options
* chore: remove unused import
* chore: remove the blanks.
* chore: revert
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-12-06 15:59:01 +00:00
Yingwen
abbac46c05
fix: do not expose manifest compression algorithm ( #2835 )
...
* fix: don't allow to set manifest compression algorithm
* docs: update config examples
2023-11-29 05:49:40 +00:00
Yingwen
b9146c88ff
refactor: Remove usages of the old storage crate ( #2777 )
...
* chore: remove storage from some crate
* feat: remove storage config
* feat: remove storage from cmd
* feat: impl stream_to_parquet
* feat: remove storage from operator
* feat: remove stream writer from mito2
* feat: remove storage from project toml
* test: fix config api test
* docs: remove outdated configs
* refactor: remove storage directory
2023-11-20 20:29:41 +08:00
Yingwen
ce959ddd3f
feat(mito): implements row group level page cache ( #2688 )
...
* feat: add page cache
* docs: update mito config toml
* feat: impl CachedPageReader
* feat: use cache reader to read row group
* feat: do not fetch data if we have pages in cache
* chore: return if nothing to fetch
* feat: enlarge page cache size
* test: test write read parquet
* test: test cache
* docs: update comments
* test: fix config api test
* feat: cache metrics
* feat: change default page cache size
* test: fix config api test
2023-11-20 02:55:50 +00:00
WU Jingdi
500e299e40
feat: Enable distributed tracing in greptimedb ( #2755 )
...
* feat: implement distributed tracing
* fix: change usage of span
* fix: use otlp as exporter
* chore: update dependence
* chore: add span info
* chore: add alias
* chore: use instrument instead of trace
2023-11-17 08:51:57 +00:00
Lanqing Yang
5608035074
fix!: improve user experience on setting compression type ( #2765 )
...
* fixes: https://github.com/GreptimeTeam/greptimedb/issues/2758
Chore: improve user experience on setting compression type
* Update src/common/datasource/src/compression.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-11-17 06:27:13 +00:00
Ruihang Xia
6599bb5a46
feat: do not require worker's number to be power of 2 ( #2732 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-11-13 09:04:44 +00:00
tison
f5eede4ce1
feat: support prometheus format_query endpoint ( #2731 )
...
* feat: support prometheus format_query endpoint
Signed-off-by: tison <wander4096@gmail.com >
* add test
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2023-11-10 10:33:28 +00:00
Lei, HUANG
b53537e69b
fix: unify all sst_write_buffer_size usage ( #2712 )
...
* fix: unify all sst_write_buffer_size usage
* fix: some CR comments
* fix: logs
2023-11-09 11:22:47 +00:00
Niwaka
06d273b75a
fix: GET: /config return all configurations when running standalone ( #2630 )
...
* fix: config api return all configurations when running standalone
* chore: follow same style
* fix: avoid panic
2023-11-08 03:22:19 +00:00
WU Jingdi
180bc64cb0
feat: change metric crate from metrics to prometheus ( #2655 )
...
* feat: change metrics to prometheus
* chore: fix code advise
* chore: resolve merge conflict
* chore: fix code advise
2023-10-31 15:46:57 +00:00
Yun Chen
f859932745
fix: convert to ReadableSize & Durations ( #2594 )
...
* fix: convert to ReadableSize & Durations
* fix: change more grpc sender/recv message size to ReadableSize
fix: format
fix: cargo fmt
fix: change cmd test to use durations
fix: revert metaclient change
fix: convert default fields in meta client options
fix: human serde meta client durations
* fix: remove milisecond postfix in heartbeat option
* fix: humantime serde on heartbeat
* fix: update config example
* fix: update integration test config
* fix: address pr comments
* fix: fix pr comment on default annotation
2023-10-13 03:28:29 +00:00
Yingwen
657542c0b8
feat(mito): Cache repeated vector for tags ( #2523 )
...
* feat: add vector_cache to CacheManager
* feat: cache repeated vectors
* feat: skip decoding pk if output doesn't contain tags
* test: add TestRegionMetadataBuilder
* test: test ProjectionMapper
* test: test vector cache
* test: test projection mapper convert
* style: fix clippy
* feat: do not cache vector if it is too large
* docs: update comment
2023-10-07 11:36:00 +00:00
Yingwen
a6116bb866
feat(mito): Add cache manager ( #2488 )
...
* feat: add cache manager
* feat: add cache to reader builder
* feat: add AsyncFileReaderCache
* feat: Impl AsyncFileReaderCache
* chore: move moka dep to workspace
* feat: add moka cache to the manager
* feat: implement parquet meta cache
* test: test cache manager
* feat: consider vec size
* style: fix clippy
* test: fix config api test
* feat: divide cache
* test: test disabling meta cache
* test: fix config api test
* feat: remove meta cache if file is purged
2023-09-26 11:46:19 +00:00
LinFeng
621c6f371b
feat: limit grpc message size ( #2459 )
...
* feat: add two grpc config options
Those options are for:
* Limit receiving(decoding) message size
* Limit sending(enoding) message size
* test: add integration tests for message size limit
2023-09-22 02:07:46 +00:00
shuiyisong
20f4f7971a
refactor: remove source and location in snafu display ( #2428 )
...
* refactor: remove source pt 1
* refactor: remove source pt 2
* refactor: remove source pt 3
* refactor: remove location pt 1
* refactor: remove location pt 2
* chore: remove rustc files
* chore: fix error case
* chore: fix test
* chore: fix test
* chore: fix cr issue
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com >
---------
Co-authored-by: fys <40801205+fengys1996@users.noreply.github.com >
2023-09-21 02:55:24 +00:00
Ruihang Xia
34d6288945
feat: bring back sqlness and integration tests ( #2448 )
...
* enable integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* disable sqlness region failover
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* enable sqlness in CI
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* sort unstable result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* set require_lease_before_startup to true
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix: fix inconsistent cache
* replace windows path chars
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore some integration cases in windows
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Revert "ignore some integration cases in windows"
This reverts commit 122478b7c1waynestxia@gmail.com >
* fix: fix close region bug in RegionHeartbeatResponseHandler
* disable failover tests
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: WenyXu <wenymedia@gmail.com >
2023-09-20 09:17:30 +00:00
Zhenchi
d6d46378a1
test: fix some integration tests ( #2432 )
...
* test: fix some integration tests
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* test: add timezone setting
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2023-09-18 10:52:14 +00:00
LFC
4b13c88752
fix: resolve more integration tests ( #2406 )
...
* fix: resolve more integration tests
* Update tests-integration/tests/http.rs
Co-authored-by: Weny Xu <wenymedia@gmail.com >
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2023-09-15 09:43:16 +00:00
shuiyisong
f16f58266e
refactor: query_ctx from http middleware ( #2253 )
...
* chore: change userinfo to query_ctx in http handler
* chore: minor change
* chore: move prometheus http to http mod
* chore: fix uni test:
* chore: add back schema check
* chore: minor change
* chore: remove clone
2023-08-25 09:36:33 +00:00
Ning Sun
88247e4284
fix!: resolve residual issues with removing prometheus port ( #2227 )
...
* fix: resolve residual issues when removing prometheus port
* fix: remove prometheus from sample config as well
2023-08-23 01:49:11 +00:00
dennis zhuang
18fa0e01ed
feat: remove checkpoint_on_startup ( #2228 )
...
feat: update flushed manifest version when it is larger
2023-08-22 13:09:34 +00:00
sh2
202540823f
refactor!: move prometheus routes to default http server ( #2005 )
...
* move prometheus routes to default http server
Signed-off-by: sh2 <shawnhxh@outlook.com >
* fix ci test and remove the server logic of prometheus
* remove unused import and prometheus relevant code
* fix ci: rustfmt and test
* fix ci: silly fmt
* fix ci: silly silly fmt
* change `/prom_store` back to `/prometheus`
* remove unused variable
---------
Signed-off-by: sh2 <shawnhxh@outlook.com >
2023-08-16 03:21:14 +00:00
shuiyisong
7f51141ed0
refactor: auth crate ( #2148 )
...
* chore: move user_info to auth crate
* chore: temp commit before resolving tests compile error
* chore: fix compile issue
* chore: minor fix
* chore: tmp save
* chore: change user_info to trait
* chore: minor change & use auth result user info in pg session setup
* chore: add as_any to user_info
* chore: rename user_info
* chore: remove ice file
* chore: add permission checker
* chore: add grpc permission check
* chore: add session spawn user_info to query_ctx
* chore: minor update
* chore: add permission checker to sql handler & temp save
* chore: add permission checker to prometheus handler
* chore: add permission checker to opentsdb handler
* chore: add permission checker to other handlers
* chore: add test
* chore: add user_info setting on http entrance
* chore: fix toml
* chore: remove box in permission req
* chore: cr issue
* chore: cr issue
2023-08-14 02:51:26 +00:00
dennis zhuang
6f40128058
feat!: enable telemetry by default ( #2137 )
...
* feat: remove greptimedb-telemetry feature
* feat: adds enable_telemetry option to metasrv and datanode
* refactor: move data_home from file config to storage config
* feat: store the installation uuid into datanode and metasrv working home
* fix: cargo toml fmt
* test: ignore region failver test when using local fle storage
* test: ignore telemetry reporter in test mode
* feat: print warning log when enabling telemetry
* chore: the telemetry doc link
* chore: remove enable_telemetry from datanode example config file
* refactor: rename GREPTIMEDB_TELEMETRY_CLIENT_REQUEST_TIMEOUT
* chore: rename print_warn_log to print_anonymous_usage_data_disclaimer
2023-08-11 14:50:40 +00:00
LFC
5c19913a91
build: on windows ( #2054 )
...
* build on windows
* rebase develop
* fix: resolve PR comments
2023-08-10 08:08:37 +00:00
Ruihang Xia
5f0403c245
feat: improve /label and /labels APIs in prometheus server ( #2087 )
...
* support __name__ for /label
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* make match[] in labels optional
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-08-03 07:51:08 +00:00
shuiyisong
5eb2c609a3
fix: auth in grpc ( #2056 )
...
* fix: auth in grpc
* fix: change to return err
* fix: add grpc test
* fix: add http test
* fix: add mysql and pg test
2023-08-01 15:18:31 +00:00
Eugene Tolbakov
6811acb314
fix(status_endpoint): add default value for hostname ( #1972 )
...
* fix(status_endpoint): add default value for hostname
* fix: adjust according to clippy suggestions
* fix: adjust according to CR suggestions
2023-07-17 11:55:11 +00:00
Kree0
8f71ac2172
refactor: move heartbeat configuration into an independent section ( #1976 )
...
refactor: move heartbeat configuration into an independent section in config file
* refactor: move heartbeat configuration into an independent section in config file
* feat: add HeartbeatOptions struct
* test: modify corresponding test case
* chore: modify corresponding example file
2023-07-17 11:29:02 +08:00
Eugene Tolbakov
674bfd85c7
chore(prom)!: rename prometheus(remote storage) to prom-store and promql(HTTP server) to prometheus ( #1931 )
...
* chore(prom): rename prometheus(remote storage) to prom-store and promql(HTTP server) to prometheus
* chore: apply clippy suggestions
* chore: adjust format according to rustfmt
2023-07-12 14:47:09 +08:00
Weny Xu
2d30f4c373
fix: fix broken CI ( #1933 )
2023-07-11 14:48:41 +08:00
Eugene Tolbakov
fc850c9988
feat(config-endpoint): add initial implementation ( #1896 )
...
* feat(config-endpoint): add initial implementation
* feat: add initial handler implementation
* fix: apply clippy suggestions, use axum response instead of string
* feat: address CR suggestions
* fix: minor adjustments in formatting
* fix: add a test
* feat: add to_toml_string method to options
* fix: adjust the assertion for the integration test
* fix: adjust expected indents
* fix: adjust assertion for the integration test
* fix: improve according to clippy
2023-07-11 11:08:32 +08:00
Lei, HUANG
f287d3115b
chore: replace result assertions ( #1840 )
...
* s/assert!\((.*)\.is_ok\(\)\);/\1.unwrap\(\);/g
* s/assert!\((.*)\.is_some\(\)\);/\1.unwrap\(\);/g
2023-06-27 19:14:48 +08:00
Niwaka
a103614fd2
feat: support /api/v1/series for Prometheus ( #1620 )
...
* feat: support /api/v1/series for Prometheus
* chore: error handling
* feat: update tests
2023-06-06 10:29:16 +08:00
Chuanle Chen
77497ca46a
feat: support /api/v1/label/<label_name>/values from Prometheus ( #1604 )
...
* feat: support `/api/v1/label/<label_name>/values` from Prometheus
* chore: apply CR
* chore: apply CR
2023-05-22 07:24:12 +00:00
Huaijin
715e1a321f
feat: implement /api/v1/labels for prometheus ( #1580 )
...
* feat: implement /api/v1/labels for prometheus
* fix: only gather match[]
* chore: fix typo
* chore: fix typo
* chore: change style
* fix: suggestion
* fix: suggestion
* chore: typo
* fix: fmt
* fix: add more test
2023-05-17 03:56:22 +00:00
Ning Sun
497b1f9dc9
feat: metrics for storage engine ( #1574 )
...
* feat: add storage engine region count gauge
* test: remove catalog metrics because we can't get a correct number
* feat: add metrics for log store write and compaction
* fix: address review issues
2023-05-15 15:22:00 +08:00
Ning Sun
b0ab641602
feat: add catalog/schema/table count as catalog metrics ( #1499 )
...
* feat: add catalog/schema/table count as catalog metrics
* test: add integration tests for catalog metrics
2023-05-05 05:54:12 +00:00
Yingwen
eb50cee601
feat: Switch to the procedure framework ( #1448 )
...
* feat: Remove create_mock_sql_handler()
create_to_request() and alter_to_request() don't need `&self`, so
we don't need to mock the sql handler to test them
* feat: Enable procedure manager by default
* docs: Update config example
* test: Enable procedure framework in all tests
* refactor(datanode): rename methods using procedure
* test(catalog): Fix temp dir drops before test finishes
* tests: Enable procedure framework in sqlness
* test: Fix sqlness standalone rename test
* fix: Drop procedure allows table not in engine
* test: Change rename table test
* fix: add options to table meta when creating table by procedure
* test: adjust error message in schema test case
* test: Fix test_sql_api error message
2023-04-25 12:04:02 +08:00
Ruihang Xia
b6647af2e3
test: add integration case to check dashboard path ( #1422 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-20 11:17:01 +08:00
Ruihang Xia
e8bb00f0be
feat: impl instant query interface ( #1410 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-18 23:25:14 +08:00
LFC
48c2841e4d
feat: execute python script in distributed mode ( #1264 )
...
* feat: execute python script in distributed mode
* fix: rebase develop
2023-04-02 20:36:48 +08:00
discord9
379f581780
test: add Integrated Test for Coprocessor& fix minor bugs ( #1122 )
...
* feat: cache `Runtime`
* fix: coprstream schema not set
* test: integrated tests for Coprocessor
* fix: UDF fixed
* style: remove unused import
* chore: remove more unused import
* feat: `filter`, (r)floordiv for Vector
* chore: CR advices
* feat: auto convert to `lit`
* chore: fix typo
* feat: from&to `pyarrow.array`
* feat: allow `pyarrow.array` as args to builtins
* chore: cargo fmt
* test: CI add `pyarrow`
* test: install Python&PyArrow in CI
* test: not cache depend for now
* chore: CR advices
* test: fix name
* style: rename
2023-03-06 19:20:59 +08:00
yuanbohan
d4e0dc3685
feat: specify prom server start addr ( #1111 )
...
* feat: specify promql server start addr
* refactor: rename promql to prom in Prometheus API server scenario
2023-03-06 11:07:21 +08:00
LFC
11d45e2918
refactor: upgrade DataFusion, Arrow and Sqlparser ( #1074 )
...
* refactor: upgrade DataFusion, Arrow and Sqlparser
* fix: resolve PR comments
2023-02-27 22:20:08 +08:00
Ruihang Xia
68b231987c
feat: improve Prometheus compliance ( #1022 )
...
* initial impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* minor (useless) refactor
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* retrieve metric name
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add time index column to group by columns
filter out NaN in normalize
remove NULL in instant manipulator
accept form data as HTTP params
correct API URL
accept second literal as step param
* happy clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-02-20 07:29:43 +00:00
Ruihang Xia
dfe7bfb07f
feat: handle PromQL HTTP API parameters ( #985 )
...
* feat: impl EvalStmt parser
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* resolve CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* impl From<PromqlQuery> for PromQuery
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move format into with_context
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* shorthand compound error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use rfc3339 error to report float parsing error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove CompoundError
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-02-15 17:15:44 +08:00
