dennis zhuang
7323e9b36f
feat: change Range Query’s default align behavior aware of timezone ( #3219 )
...
* feat: change Range Query’s default align behavior to 1970-01-01 00:00:00 aware of timezone
* test: test with +23:00 timezone
2024-01-24 08:17:57 +00:00
Weny Xu
d75cf86467
fix: only register region keeper while creating physical table ( #3223 )
...
fix: only register region keeper during create physical table
2024-01-23 09:42:32 +00:00
Weny Xu
26535f577d
feat: enable concurrent write ( #3214 )
...
* feat: enable concurrent write
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-01-23 09:20:12 +00:00
Ruihang Xia
31787f4bfd
feat!: switch prom remote write to metric engine ( #3198 )
...
* feat: switch prom remote write to metric engine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: dennis zhuang <killme2008@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* read physical table name from url
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove physical table from header
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix merge error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add with_metric_engine option to config remote write behavior
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* check parameter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add specific config param
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* default with_metric_engine to true
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update UT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-01-22 14:48:28 +00:00
dennis zhuang
6a12c27e78
feat: make query be aware of timezone setting ( #3175 )
...
* feat: let TypeConversionRule aware query context timezone setting
* chore: don't optimize explain command
* feat: parse string into timestamp with timezone
* fix: compile error
* chore: check the scalar value type in predicate
* chore: remove mut for engine context
* chore: return none if the scalar value is utf8 in time range predicate
* fix: some fixme
* feat: let Date and DateTime parsing from string value be aware of timezone
* chore: tweak
* test: add datetime from_str test with timezone
* feat: construct function context from query context
* test: add timezone test for to_unixtime and date_format function
* fix: typo
* chore: apply suggestion
* test: adds string with timezone
* chore: apply CR suggestion
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
* chore: apply suggestion
---------
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2024-01-22 14:14:03 +00:00
Weny Xu
4278c858f3
feat: make procedure able to return output ( #3201 )
...
* feat: make procedure able to return output
* refactor: change Output to Any
2024-01-21 06:56:45 +00:00
Ruihang Xia
cde5a36f5e
feat: precise filter for mito parquet reader ( #3178 )
...
* impl SimpleFilterEvaluator
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* time index and field filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* finish parquet filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove empty Batch
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix fmt
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update metric
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use projected schema from batch
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* correct naming
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unnecessary error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-18 06:59:48 +00:00
niebayes
63205907fb
refactor: introduce common-wal to aggregate wal stuff ( #3171 )
...
* refactor: aggregate wal configs
* refactor: move wal options to common-wal
* chore: slim Cargo.toml
* fix: add missing crates
* fix: format
* chore: update comments
* chore: add testing feature gate for test_util
* fix: apply suggestions from code review
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix: apply suggestions from code review
* fix: compiling
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-18 03:49:37 +00:00
Wei
3d7d2fdb4a
feat: auto config cache size according to memory size ( #3165 )
...
* feat: auto config cache and buffer size according to mem size
* feat: utils
* refactor: add util function to common config
* refactor: check cgroups
* refactor: code
* fix: test
* fix: test
* chore: cr comment
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
* chore: remove default comment
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
Co-authored-by: Dennis Zhuang <killme2008@gmail.com >
2024-01-17 14:35:35 +00:00
shuiyisong
a29b9f71be
chore: carry metrics in flight metadata from datanode to frontend ( #3113 )
...
* chore: carry metrics in flight metadata from datanode to frontend
* chore: fix typo
* fix: ignore metric flight message on client
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: add cr comment
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: add cr comment
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* chore: update proto
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-17 11:38:03 +00:00
Ruihang Xia
fbd0197794
refactor: remove TableEngine ( #3181 )
...
* refactor: remove TableEngine
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/table/src/table_reference.rs
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
2024-01-17 04:18:37 +00:00
dennis zhuang
204b9433b8
feat: adds date_format function ( #3167 )
...
* feat: adds date_format function
* fix: compile error
* chore: use system timezone for FunctionContext and EvalContext
* test: as_formatted_string
* test: sqlness test
* chore: rename function
2024-01-17 03:24:40 +00:00
JeremyHi
c2edaffa5c
feat: let tables API return a stream ( #3170 )
2024-01-15 12:36:39 +00:00
WU Jingdi
816d94892c
feat: support HTTP&gRPC&pg set timezone ( #3125 )
...
* feat: support HTTP&gRPC&pg set timezone
* chore: fix code advice
* chore: fix code advice
2024-01-15 06:29:31 +00:00
Eugene Tolbakov
ca4d690424
feat: add modulo function ( #3147 )
...
* feat: add modulo function
* fix: address CR feedback
2024-01-13 00:24:25 +00:00
Weny Xu
bf88b3b4a0
fix: fix store all wal options ( #3149 )
...
* fix: fix store all wal options
* fix: incorrect updating DatanodeTable value
2024-01-12 04:48:14 +00:00
Weny Xu
bf96ce3049
fix: print detailed error ( #3146 )
2024-01-12 04:02:32 +00:00
Ruihang Xia
09b3c7029b
feat: handle drop request for metric table ( #3136 )
...
* handle drop request
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* adjust procedure manager
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add create table sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* insert/query metric table
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* address CR comments
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/common/meta/src/kv_backend.rs
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* reuse region option for metadata region
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tweak variable name
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-11 09:38:43 +00:00
LFC
51a3fbc7bf
refactor: change how frontend grpc services are orchestrated ( #3134 )
2024-01-11 02:26:44 +00:00
Ning Sun
b6033f62cd
refactor: implement version as built-in function and use fixed mysql version ( #3133 )
...
* refactor: implement version as built-in function
* test: add sqlness test for version()
2024-01-10 11:04:18 +00:00
dennis zhuang
fd3f23ea15
feat: adds runtime_metrics ( #3127 )
...
* feat: adds runtime_metrics
* fix: comment
* feat: refactor metrics table
* chore: ensure build_info and runtime_metrics only avaiable in greptime catalog
* feat: adds timestamp column
2024-01-10 10:51:30 +00:00
niebayes
1b0e39a7f2
chore: stop exposing num_partitions ( #3132 )
2024-01-10 10:45:18 +00:00
Weny Xu
ec8266b969
refactor: refactor the locks in the procedure ( #3126 )
...
* feat: add lock key
* refactor: procedure lock keys
* chore: apply suggestions from CR
2024-01-10 09:46:39 +00:00
niebayes
a0a31c8acc
chore(remote_wal): remove topic alias ( #3120 )
...
chore: remove topic alias
2024-01-09 07:35:02 +00:00
Lei, HUANG
62db28b465
feat: add options to enable log recycle and periodical fsync ( #3114 )
...
* feat: add options to enable log recycle and periodical fsync
* fix: resolve review comments
* fix: conflicts
2024-01-09 06:41:23 +00:00
Ruihang Xia
af0c4c068a
feat: support PromQL function vector ( #3036 )
...
* produce vector plan
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* work with OR
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* apply review sugg
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* move common const strings to common_query
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add comment for GREPTIME_COUNT
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-09 03:44:00 +00:00
tison
316d843482
feat: support CSV format in sql HTTP API ( #3062 )
...
* chore: fix typo
Signed-off-by: tison <wander4096@gmail.com >
* add csv format
Signed-off-by: tison <wander4096@gmail.com >
* flatten response
Signed-off-by: tison <wander4096@gmail.com >
* more flatten response
Signed-off-by: tison <wander4096@gmail.com >
* add CSV format
Signed-off-by: tison <wander4096@gmail.com >
* format InfluxdbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format ErrorResponse
Signed-off-by: tison <wander4096@gmail.com >
* propagate ErrorResponse to InfluxdbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format GreptimedbV1Response
Signed-off-by: tison <wander4096@gmail.com >
* format CsvResponse
Signed-off-by: tison <wander4096@gmail.com >
* impl IntoResponse for QueryResponse
Signed-off-by: tison <wander4096@gmail.com >
* promql
Signed-off-by: tison <wander4096@gmail.com >
* sql
Signed-off-by: tison <wander4096@gmail.com >
* compile
Signed-off-by: tison <wander4096@gmail.com >
* fixup aide
Signed-off-by: tison <wander4096@gmail.com >
* clear debt
Signed-off-by: tison <wander4096@gmail.com >
* fixup UT test_recordbatches_conversion
Signed-off-by: tison <wander4096@gmail.com >
* fixup IT cases
Signed-off-by: tison <wander4096@gmail.com >
* fixup more IT cases
Signed-off-by: tison <wander4096@gmail.com >
* fixup test-integration cases
Signed-off-by: tison <wander4096@gmail.com >
* update comment
Signed-off-by: tison <wander4096@gmail.com >
* fixup deserialize and most query < 1ms
Signed-off-by: tison <wander4096@gmail.com >
* fixup auth tests
Signed-off-by: tison <wander4096@gmail.com >
* fixup tests
Signed-off-by: tison <wander4096@gmail.com >
* fixup and align X-GreptimeDB headers
Signed-off-by: tison <wander4096@gmail.com >
* fixup compile
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-01-08 10:54:27 +00:00
niebayes
8c58d3f85b
test(remote_wal): add unit tests for kafka remote wal ( #2993 )
...
* test: add unit tests
* feat: introduce kafka runtime backed by testcontainers
* test: add test for kafka runtime
* fix: format
* chore: make kafka image ready to be used
* feat: add entry builder
* tmp
* test: add unit tests for client manager
* test: add some unit tests for kafka log store
* chore: resolve some todos
* chore: resolve some todos
* test: add unit tests for kafka log store
* chore: add deprecate develop branch warning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tmp: ready to move unit tests to an indie dir
* test: update unit tests for client manager
* test: add unit tests for meta srv remote wal
* fix: license
* fix: test
* refactor: kafka image
* doc: add doc example for kafka image
* chore: migrate kafka image to an indie PR
* fix: CR
* fix: CR
* fix: test
* fix: CR
* fix: update Cargo.toml
* fix: CR
* feat: skip test if no endpoints env
* fix: format
* test: rewrite parallel test with barrier
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-01-08 10:48:11 +00:00
LFC
fcacb100a2
chore: expose some codes to let other projects use them ( #3115 )
2024-01-08 06:32:01 +00:00
niebayes
78303639db
feat(remote_wal): split an entry if it's too large ( #3092 )
...
* feat: split an entry if it's too large
* chore: rewrite check records
* test: add some unit tests for record
* chore: rewrite entry splitting
* chore: add unit tests for build records
* chore: add more unit tests for record
* chore: rewrite encdec of record
* revert: ignored test
* fix: set limit for max_batch_size
* fix: clippy
* chore: remove heavy logging
* fix: CR
* fix: properly terminate
* fix: CR
* fix: compiling
* fix: sqlness
* fix: CR
* fix: license
* fix: license
2024-01-05 12:41:43 +00:00
JeremyHi
bd1a5dc265
feat: metric engine support alter ( #3098 )
...
* feat: metric engine support alter
* chore: by comment
* feat: get physical table route for frontend
2024-01-05 09:46:39 +00:00
Weny Xu
e0a43f37d7
chore: bump opendal to 0.44 ( #3058 )
...
* chore: bump opendal to 0.44
* fix: fix test_object_store_cache_policy
* Revert "fix: fix test_object_store_cache_policy"
This reverts commit 46c37c343f66114e0f6ee7a0a3b9ee2b79c810af.
* fix: fix test_object_store_cache_policy
* fix: fix test_file_backend_with_lru_cache
* chore: apply suggestions from CR
* fix(mito): fix mito2 cache
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-01-05 09:05:41 +00:00
zyy17
a89840f5f9
refactor(metrics): add 'greptime_' prefix for every metrics ( #3093 )
...
* refactor(metrics): add 'greptimedb_' prefix for every metrics
* chore: use 'greptime_' as prefix
* chore: add some prefix for new metrics
* chore: fix format error
2024-01-05 08:12:23 +00:00
Weny Xu
342faa4e07
test: add tests for lease keeper with logical table ( #3096 )
2024-01-05 02:29:48 +00:00
Weny Xu
f1a4750576
feat(tests-integration): add more region migration integration tests ( #3094 )
2024-01-04 08:18:46 +00:00
LFC
ec43b9183d
feat: table route for metric engine ( #3053 )
...
* feat: table route for metric engine
* feat: register logical regions
* fix: open logical region (#96 )
---------
Co-authored-by: JeremyHi <jiachun_feng@proton.me >
2024-01-04 06:30:17 +00:00
ClSlaid
5c66ce6e88
chore: remove unnecessary result wrappings ( #3084 )
...
patch: remove unnecessary result wrappings
Signed-off-by: 蔡略 <cailue@bupt.edu.cn >
2024-01-03 10:20:33 +00:00
Weny Xu
b2b752337b
fix: fix non-physical error msg ( #3087 )
2024-01-03 09:40:03 +00:00
Weny Xu
aa22f9c94a
refactor: allow procedure to acquire share lock ( #3061 )
...
* feat: implement `KeyRwLock`
* refactor: use KeyRwLock instead of LockMap
* refactor: use StringKey instead of String
* chore: remove redundant code
* refactor: cleanup KeyRwLock staled locks before granting new lock
* feat: clean staled locks manually
* feat: sort lock key in lexicographically order
* feat: ensure the ref count before dropping the rwlock
* feat: add more tests for rwlock
* feat: drop the key guards first
* feat: drops the key guards in the reverse order
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2024-01-03 08:05:45 +00:00
Weny Xu
2b181e91e0
refactor: unify the injection of WAL option ( #3066 )
...
* feat: add prepare_wal_option
* refactor: use integer hashmap
* feat: unify the injection of WAL option
* fix: fix procedure_flow_upgrade_candidate_with_retry
* chore: apply suggestions from CR
2024-01-02 07:40:02 +00:00
Weny Xu
5653389063
feat!: correct the kafka config option ( #3065 )
...
* feat: correct the kafka config option
* refactor: rewrite the verbose comments
2024-01-02 07:31:37 +00:00
dimbtp
c4d7b0d91d
feat: add some tables for information_schema ( #3060 )
...
* feat: add information_schema.optimizer_trace
* feat: add information_schema.parameters
* feat: add information_schema.profiling
* feat: add information_schema.referential_constraints
* feat: add information_schema.routines
* feat: add information_schema.schema_privileges
* feat: add information_schema.table_privileges
* feat: add information_schema.triggers
* fix: update sql test result
* feat: add information_schema.global_status
* feat: add information_schema.session_status
* fix: update sql test result
* fix: add TODO for some tables
* Update src/catalog/src/information_schema/memory_table/tables.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-01-02 04:10:59 +00:00
dimbtp
f735f739e5
feat: add information_schema.key_column_usage ( #3057 )
...
* feat: add information_schema.key_column_usage
* fix: follow #3057 review comments
* fix: add sql test for `key_column_usage` table
* fix: fix spell typo
* fix: resolve conflict in sql test result
2023-12-31 12:29:06 +00:00
dimbtp
6070e88077
feat: add information_schema.files ( #3054 )
...
* feat: add information_schema.files
* fix: update information_schema.result
* fix: change `EXTRA` field type to string
2023-12-31 02:08:16 +00:00
AntiTopQuark
4460af800f
feat(TableRouteValue): add panic notes and type checks ( #3031 )
...
* refactor(TableRouteValue): add panic notes and type checks
* chore: add deprecate develop branch warning
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add error defines and checks
* Update README.md
* update code format and fix tests
* update name of error
* delete unused note
* fix unsafe .expect() for region_route()
* update error name
* update unwrap
* update code format
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2023-12-30 13:02:26 +00:00
Zhenchi
69a53130c2
feat(inverted_index): Add applier builder to convert Expr to Predicates (Part 1) ( #3034 )
...
* feat(inverted_index.integration): Add applier builder to convert Expr to Predicates (Part 1)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: add docs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: typos
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* Update src/mito2/src/sst/index/applier/builder.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* fix: remove unwrap
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: error source
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-12-30 07:32:32 +00:00
dennis zhuang
11ae85b1cd
feat: adds information_schema.schemata ( #3051 )
...
* feat: improve information_schema.columns
* feat: adds information_schema.schemata
* fix: instance test
* fix: comment
2023-12-29 09:22:31 +00:00
LFC
7551432cff
refactor: merge standalone and metasrv table metadata allocators ( #3035 )
...
* refactor: merge standalone and metasrv table metadata allocators
* Update src/common/meta/src/ddl/table_meta.rs
Co-authored-by: niebayes <niebayes@gmail.com >
* Update src/common/meta/src/ddl/table_meta.rs
Co-authored-by: Weny Xu <wenymedia@gmail.com >
---------
Co-authored-by: niebayes <niebayes@gmail.com >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2023-12-29 08:50:59 +00:00
Weny Xu
e16f093282
test(remote_wal): add sqlness with kafka wal ( #3027 )
...
* feat(sqlness): add kafka wal config
* chore: add sqlness with kafka wal ci config
* fix: fix config
* chore: apply suggestions from CR
* fix: add metasrv config to sqlness with kafka
* fix: replay memtable should from flushed_entry_id + 1
* fix: should set append flag to fopen
* feat: start wal allocator in standalone meta mode
* feat: append a noop record after kafka topic initialization
* test: ignore tests temporally
* test: change sqlness kafka wal config
2023-12-29 08:17:22 +00:00
Weny Xu
301ffc1d91
feat(remote_wal): append a noop record after kafka topic initialization ( #3040 )
...
* feat: append a noop record after kafka topic initialization
* chore: apply suggestions from CR
* feat: ignore the noop record during the read
2023-12-29 07:46:48 +00:00
