localhost
627a326273
refactor: Change the error type in the pipeline crate from String to Error ( #4763 )
...
* chore: in process
* chore: change pipeline crate error type
* chore: improve event error
* chore: fix by pr comment
* chore: use snafu context replace ok_or_else
* refactor: update snafu usage
---------
Co-authored-by: Ning Sun <sunng@protonmail.com >
2024-09-25 19:32:34 +00:00
discord9
0274e752ae
chore: make sure aws-lc-sys wouldn't be built ( #4767 )
...
* chore: make sure aws-lc-sys wouldn't be built
* fix: bash
* refactor: multiple line bash
2024-09-25 17:11:37 +00:00
Lei, HUANG
cd4bf239d0
chore: relax table name constraint ( #4766 )
...
chore/relax-table-name-constraint: Updated NAME_PATTERN to allow '@' and '#' characters and adjusted tests for new table name validation rules.
2024-09-25 02:45:18 +00:00
Ning Sun
e3c0b5482f
feat: returning warning instead of error on unsupported SET statement ( #4761 )
...
* feat: add capability to send warning to pgclient
* fix: refactor query context to carry query scope data
* feat: return a warning for unsupported postgres statement
2024-09-24 08:45:55 +00:00
Weny Xu
d1b252736d
refactor: unify the styling in create_or_alter_tables_on_demand ( #4756 )
...
* refactor: refactor `create_or_alter_tables_on_demand`
* chore: apply suggestions from CR
2024-09-24 03:48:16 +00:00
discord9
54f6e13d13
fix: Release CI & make rustls use ring ( #4750 )
...
* feat: new dev-build
* fix: copy
* chore: rm dbg run
* fix: binstall install script
* chore: typos
* fix: update useable image
* fix: properly uninstall gcc-9
* chore: another try
* chore: print current cc version
* chore: another try to release
* fix: use gcc-10 by `update-alternatives`
* chore: update dev-build image
* remove gcc-9 again and install make/cmake
* use new image
* alias gcc-10/g++-10 to every variant
* again.....
* fix....
* again release ....
* chore: remove auto remove
* feat: rustls default to ring
* chore: update Cargo.lock
* chore: update Cargo.lock
* Apply suggestions from code review
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-24 03:11:12 +00:00
Ruihang Xia
5c64f0ce09
refactor!: simplify NativeType trait and remove percentile UDAF ( #4758 )
...
* refactor!: simplify NativeType trait and remove percentile UDAF
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove NativeType
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* recover a mis-deleted case
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-23 10:55:20 +00:00
Ruihang Xia
2feddca1cb
feat: include order by to commutativity rule set ( #4753 )
...
* feat: include order by to commutativity rule set
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* tune sqlness replace interceptor
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-23 08:35:06 +00:00
Ning Sun
0f99218386
feat: list/array/timezone support for postgres output ( #4727 )
...
* feat: list/array support for postgres output
* fix: implement time zone support for postgrsql
* feat: add a geohash function that returns array
* fix: typo
* fix: lint warnings
* test: add sqlness test
* refactor: check resolution range before convert value
* fix: test result for sqlness
* feat: upgrade pgwire apis
2024-09-22 02:39:38 +00:00
Weny Xu
163cea81c2
feat: migrate local WAL regions ( #4715 )
...
* feat: allow to flush region before migrating
* fix: fix unit tests
* feat: allow to set `flush_timeout`
* feat: skip to replay memtable
* fix: fix unit tests
* test: add more tests
* refactor: simplify timeout logical
* test: add unit tests
* test: add unit tests
* chore: update comments
* fix: fix unit tests
* fix: fmt and clippy
* feat: change default timeout to 30s
* fix: throw `ExceededDeadline` error
* test: add tests for `downgrade_region_with_retry`
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: update proto to `3633474`
* refactor: refactor `upgrade_region_with_retry`
* chore: apply suggestions from CR
2024-09-20 08:27:20 +00:00
taobo
0c9b8eb0d2
feat: improve observability for procedure ( #4675 )
...
* feat: improve observability for procedure
* fix: test error
* test: add sqlness test for information_schema.procedure_info
* fix: sqlness test error
* fix: cr comment
* chore: update proto version
* fix: apply cr comment
* update version
* fix: cr comment
* optimize procedure type output format
* upgrade dep version
* fix: clippy error
* fix: `procedure` borrowed error
* fix: optimize code
2024-09-20 06:07:53 +00:00
Ning Sun
75c6fad1a3
feat: add more h3 scalar functions ( #4707 )
...
* feat: add more h3 scalar functions
* chore: comment up
2024-09-20 04:19:50 +00:00
Yingwen
e12ffbeb2f
feat: flush other workers if still need flush ( #4746 )
2024-09-20 02:55:31 +00:00
discord9
c4e52ebf91
feat: use new image for gcc-10 ( #4748 )
...
feat: use new image
2024-09-20 02:34:45 +00:00
Yingwen
f02410c39b
fix: disable field pruning in last non null mode ( #4740 )
...
* fix: don't prune fields in last non null mode
* test: add sqlness test for field pruning
* test: add flush
* refine implementation
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-20 00:35:37 +00:00
Ruihang Xia
f5cf25b0db
refactor: remove DfPlan wrapper ( #4733 )
...
* refactor: remove DfPlan wrapper
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* clean up
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test assertion
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-19 12:29:33 +00:00
liyang
1acda74c26
fix: cannot input tag for the dev-builder image ( #4743 )
2024-09-19 11:14:34 +00:00
Yohan Wal
95787825f1
build(deps): use original jsonb repo ( #4742 )
2024-09-19 09:44:44 +00:00
Weny Xu
49004391d3
chore(fuzz): print table name for debugging ( #4738 )
...
* chore(fuzz): print table name for debugging
* chore: apply suggestions
2024-09-19 09:40:10 +00:00
discord9
d0f5b2ad7d
fix: use gcc-10 in release dev build ( #4741 )
2024-09-19 09:34:06 +00:00
Yohan Wal
0295f8dbea
docs: json datatype rfc ( #4515 )
...
* docs: json datatype rfc
* docs: turn to a jsonb proposal
* chore: fix typo
* feat: add store and query process
* fix: typo
* fix: use query nodes instead of query plans
* feat: a detailed overview of query
* fix: grammar
* fix: use independent cast function
* fix: unify cast function
* fix: refine, make statements clear
* docs: update rfc according to impl
* docs: refine
* docs: fix wrong arrows
* docs: refine
* docs: fix some errors qaq
2024-09-19 05:49:10 +00:00
Ning Sun
8786624515
feat: improve support for postgres extended protocol ( #4721 )
...
* feat: improve support for postgres extended protocol
* fix: lint fix
* fix: test code
* fix: adopt upstream
* refactor: remove dup code
* refactor: avoid copy on error message
2024-09-19 05:30:56 +00:00
shuiyisong
52d627e37d
chore: add log ingest interceptor ( #4734 )
...
* chore: add log ingest interceptor
* chore: rename
* chore: update interceptor signature
2024-09-19 05:14:47 +00:00
Lei, HUANG
b5f7138d33
refactor(tables): improve tables performance ( #4737 )
...
* chore: cherrypick 52e8eebb2dbbbe81179583c05094004a5eedd7fd
* refactor/tables: Change variable from immutable to mutable in KvBackendCatalogManager's method
* refactor/tables: Replace unbounded channel with bounded and use semaphore for concurrency control in KvBackendCatalogManager
* refactor/tables: Add common-runtime dependency and update KvBackendCatalogManager to use common_runtime::spawn_global
* refactor/tables: Await on sending error through channel in KvBackendCatalogManager
2024-09-19 04:44:02 +00:00
Ning Sun
08bd40333c
feat: add an option to turn on compression for arrow output ( #4730 )
...
* feat: add an option to turn on compression for arrow output
* fix: typo
2024-09-19 04:38:41 +00:00
discord9
d1e0602c76
fix: opensrv Use After Free update ( #4732 )
...
* chore: version skew
* fix: even more version skew
* feat: use `ring` instead of `aws-lc` for remove nasm assembler on windows
* feat: use `ring` for pgwire
* feat: change to use `aws-lc-sys` on windows instead
* feat: change back to use `ring`
* chore: provide CryptoProvider
* feat: use upstream repo
* feat: install ring crypto lib in main
* chore: use same fn to install in tests
* feat: make pgwire use `ring`
2024-09-19 04:12:13 +00:00
Weny Xu
befb6d85f0
fix: determine region role by using is_readonly ( #4725 )
...
fix: correct `is_writable` behavior
2024-09-18 22:17:39 +00:00
Yohan Wal
f73fb82133
feat: add respective json_is UDFs for JSON type ( #4726 )
...
* feat: add respective json_is UDFs
* refactor: rename to_json to parse_json
* chore: happy clippy
* chore: some rename
* fix: small fixes
2024-09-18 11:07:30 +00:00
shuiyisong
50b3bb4c0d
fix: sort cargo toml ( #4735 )
2024-09-18 09:19:05 +00:00
zyy17
0847ff36ce
fix: config test failed and use similar_asserts::assert_eq to replace assert_eq for long string compare ( #4731 )
...
* fix: config test failed and use 'similar_asserts::assert_eq' to replace 'assert_eq' for long string compare
* Update Cargo.toml
Co-authored-by: Yingwen <realevenyag@gmail.com >
* Update src/cmd/tests/load_config_test.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2024-09-18 07:41:25 +00:00
shuiyisong
c014e875f3
chore: add auto-decompression layer for otlp http request ( #4723 )
...
* chore: add auto-decompression for http request
* test: otlp
2024-09-18 04:32:00 +00:00
Zhenchi
3b5b906543
feat(index): add explicit adapter between RangeReader and AsyncRead ( #4724 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-09-18 03:33:55 +00:00
Weny Xu
d1dfffcdaf
chore: enable fuzz test for append table ( #4702 )
...
* chore: enable fuzz test for append table
* fix: fix mysql translator
2024-09-18 03:01:30 +00:00
localhost
36b1bafbf0
fix: pipeline dissert error is returned directly to the user, instead of printing a warn log ( #4709 )
...
* fix: pipeline dissert error is returned directly to the user, instead of printing a warn log
* chore: add more test for pipeline
2024-09-12 18:21:05 +00:00
Yohan Wal
67fb3d003e
feat: add respective get_by_path UDFs for JSON type ( #4720 )
...
* feat: add respectiv get_by_path udf for json type
* Apply review comments
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* fix: fix compile error
* refactor: change name of UDFs, add some tests
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-09-11 08:17:57 +00:00
zyy17
aa03d3b11c
docs: use docs comment prefix and bump toml2docs version ( #4711 )
2024-09-11 07:49:23 +00:00
discord9
a3d567f0c9
perf(flow): use batch mode for flow ( #4599 )
...
* generic bundle trait
* feat: impl get/let
* fix: drop batch
* test: tumble batch
* feat: use batch eval flow
* fix: div use arrow::div not mul
* perf: not append batch
* perf: use bool mask for reduce
* perf: tiny opt
* perf: refactor slow path
* feat: opt if then
* fix: WIP
* perf: if then
* chore: use trace instead
* fix: reduce missing non-first batch
* perf: flow if then using interleave
* docs: add TODO
* perf: remove unnecessary eq
* chore: remove unused import
* fix: run_available no longer loop forever
* feat: blocking on high input buf
* chore: increase threhold
* chore: after rebase
* chore: per review
* chore: per review
* fix: allow empty values in reduce&test
* tests: more flow doc example tests
* chore: per review
* chore: per review
2024-09-11 03:31:52 +00:00
Zhenchi
f252599ac6
feat(index): add RangeReader trait ( #4718 )
...
* feat(index): add `RangeReader` trait`
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: return content_length as read bytes
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: remove buffer & use `BufMut`
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-09-10 15:24:06 +00:00
Ruihang Xia
ff40d512bd
fix: support append-only physical table ( #4716 )
...
* fix: support append-only physical table
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/metric-engine/src/engine/create.rs
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
2024-09-10 12:23:23 +00:00
jeremyhi
dcae21208b
chore: refresh route table ( #4673 )
...
* chore: remove error::
* chore: avoid to use get_raw if unnecessary
* chore: clearer method name
* feat: remap node addresses in table route
* chore: add unit test for remap address
* feat: refresh node address mapping via heartbeat
* feat: broadcast table cache invalidate on new epoch
* chore: clarify heartbeat log
* chore: remove InvalidHeartbeatRequest
* chore: add log
* feat: add role into NodeAddressKey
* chore: fix test
* Update src/common/meta/src/key/table_route.rs
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
* chore: simplify code
---------
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com >
2024-09-10 12:08:59 +00:00
Weny Xu
d0fd79ac7f
chore: remove validate_request_with_table ( #4710 )
...
perf: remove `validate_request_with_table`
2024-09-10 11:56:18 +00:00
Yingwen
3e17c09e45
feat: skip caching uncompressed pages if they are large ( #4705 )
...
* feat: cache each uncompressed page
* chore: remove unused function
* chore: log
* chore: log
* chore: row group pages cache kv
* feat: also support row group level cache
* chore: fix range count
* feat: don't cache compressed page for row group cache
* feat: use function to get part
* chore: log whether scan is from compaction
* chore: avoid get column
* feat: add timer metrics
* chore: Revert "feat: add timer metrics"
This reverts commit 4618f57fa2ba13b1e1a8dec83afd01c00ae4c867.
* feat: don't cache individual uncompressed page
* feat: append in row group level under append mode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: fetch pages cost
* perf: yield
* Update src/mito2/src/sst/parquet/row_group.rs
* refactor: cache key
* feat: print file num and row groups num in explain
* test: update sqlness test
* chore: Update src/mito2/src/sst/parquet/page_reader.rs
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-10 11:52:16 +00:00
jeremyhi
04de3ed929
chore: avoid schema check when auto_create_table_hint is disabled ( #4712 )
...
chore: avoid schema check when auto-create-table-hint is disabled
2024-09-10 07:13:28 +00:00
Ruihang Xia
29f215531a
feat: parallel in row group level under append mode ( #4704 )
...
feat: append in row group level under append mode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-09-10 07:12:23 +00:00
jeremyhi
545a80c6e0
chore: remove unused method ( #4703 )
2024-09-09 12:14:17 +00:00
Yohan Wal
04e7dd6fd5
feat: add json data type ( #4619 )
...
* feat: add json type and vector
* fix: allow to create and insert json data
* feat: udf to query json as string
* refactor: remove JsonbValue and JsonVector
* feat: show json value as strings
* chore: make ci happy
* test: adunit test and sqlness test
* refactor: use binary as grpc value of json
* fix: use non-preserve-order jsonb
* test: revert changed test
* refactor: change udf get_by_path to jq
* chore: make ci happy
* fix: distinguish binary and json in proto
* chore: delete udf for future pr
* refactor: remove Value(Json)
* chore: follow review comments
* test: some tests and checks
* test: fix unit tests
* chore: follow review comments
* chore: corresponding changes to proto
* fix: change grpc and pgsql server behavior alongside with sqlness/crud tests
* chore: follow review comments
* feat: udf of conversions between json and strings, used for grpc server
* refactor: rename to_string to json_to_string
* test: add more sqlness test for json
* chore: thanks for review :)
* Apply suggestions from code review
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-09-09 11:41:36 +00:00
jeremyhi
dc89944570
feat: gRPC auto create table hint ( #4700 )
...
* feat: gRPC auto create table hint
* chore: remove the checking of auto_create_table_hint
2024-09-09 09:07:07 +00:00
Weny Xu
8bf549c2fa
chore: print downgraded region last_entry_id ( #4701 )
2024-09-09 08:14:55 +00:00
Lei, HUANG
208afe402b
feat(wal): increase recovery parallelism ( #4689 )
...
* Refactor RaftEngineLogStore to use references for config
- Updated `RaftEngineLogStore::try_new` to accept a reference to `RaftEngineConfig` instead of taking ownership.
- Replaced direct usage of `config` with individual fields (`sync_write`, `sync_period`, `read_batch_size`).
- Adjusted test cases to pass references to `RaftEngineConfig`.
* Add parallelism configuration for WAL recovery
- Introduced `recovery_parallelism` setting in `datanode.example.toml` and `standalone.example.toml` for configuring parallelism during WAL recovery.
- Updated `Cargo.lock` and `Cargo.toml` to include `num_cpus` dependency.
- Modified `RaftEngineConfig` to include `recovery_parallelism` with a default value set to the number of CP
* feat/wal-recovery-parallelism:
Add `wal.recovery_parallelism` configuration option
- Introduced `wal.recovery_parallelism` to config.md for specifying parallelism during WAL recovery.
- Updated `RaftEngineLogStore` to include `recovery_threads` from the new configuration.
* fix: ut
2024-09-09 04:25:24 +00:00
Ning Sun
c22a398f59
fix: return version string based on request protocol ( #4680 )
...
* fix: return version string based on request protocol
* fix: resolve lint issue
2024-09-09 03:36:54 +00:00
