yihong
369b59c84a
fix: drop unused dep using udeps to minial the size ( #5301 )
...
* fix: drop unused dep using udeps to minial the size
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
* fix: adress comments fix the problem
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
---------
Signed-off-by: yihong0618 <zouzou0208@gmail.com >
2025-01-08 06:36:22 +00:00
Weny Xu
3d9df822ad
refactor: refactor PgStore ( #5309 )
...
* refactor: refactor PgStore
* fix: election use bytea and txn use Serializable to avoid read unrepeatable (#4 )
* fix: election use bytea as well
* fix: use Serializable to avoid read unrepeatable
* chore: remove unused error
* ci: enable pg kvbackend and sqlness
* ci: switch on pg_kvbackend feature
* fix: fix sqlness runner
* chore: add pg_kvbackend feature gate
* build(ci): add feature gate
* fix: add retry for `PgStore` txn
* fix: correct `SET_IDLE_SESSION_TIMEOUT`
---------
Co-authored-by: Yohan Wal <1035325592@qq.com >
Co-authored-by: CookiePieWw <profsyb@gmail.com >
2025-01-07 07:27:58 +00:00
Zhenchi
5cf9d7b6ca
fix(bloom-filter): filter rows with segment precision ( #5286 )
...
* fix(bloom-filter): filter rows with segment precision
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* add case
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* address TODO
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2025-01-06 11:45:15 +00:00
shuiyisong
bbbba29afc
feat: support Loki JSON write ( #5288 )
...
* perf: small updates
* refactor: move loki to a separate file
* chore: extract content_type pattern matching
* chore: minor update
* feat: loki json write
* chore: add decompression http layer
* fix: label string value instead of to_string
* chore: add test
* fix: typo
* fix: license header
* chore: rename
2025-01-06 07:09:48 +00:00
Weny Xu
b229c94fba
fix(fuzz): ensure all regions leases are renewed ( #5294 )
...
* fix(fuzz): ensure all regions leases are renewed
* fix: fix clippy
2025-01-06 06:21:41 +00:00
Yohan Wal
513569ed5d
feat: add Txn for pg kv backend ( #5266 )
...
* feat: txn for pg kv backend
* chore: clippy
* fix: txn uses one client
* test: clean up and txn test
* test: clean up
* test: change lock_id to avoid conflict in test
* test: use different prefix in pg election test
* fix(test): just a fix
* test: aggregate multiple test to avoid concurrency problem
* test: use uuid instead of rng
* perf: batch cmp in txn
* perf: batch same op in txn
2025-01-06 03:29:09 +00:00
Yingwen
9bf9aa1082
chore: update greptime-proto to include add_if_not_exists ( #5289 )
2025-01-03 15:41:00 +00:00
Yingwen
89399131dd
feat: support add if not exists in the gRPC alter kind ( #5273 )
...
* test: test adding existing columns
* chore: add more checks to AlterKind
* chore: update logs
* fix: check and build table info first
* feat: Add add_if_not_exists flag to alter expr
* feat: skip existing columns when building alter kind
* checks in make_region_alter_kind()
* reuse the alter kind
* test: fix tests in common-meta
* chore: fix typos
* chore: update comments
2025-01-03 07:23:17 +00:00
Yohan Wal
1b0b9add90
feat: use connection pool for pg kv backend in preparation for txn ( #5260 )
...
* feat: use connection pool
* chore: follow review comments
* fix: create table before test
2025-01-02 06:33:21 +00:00
chenmortal
2b89970d45
fix: import tokio-metrics and tokio-metrics-collector ( #5264 )
2025-01-02 05:58:31 +00:00
Kould
dad8ac6f71
feat(vector): add vector functions vec_sub & vec_sum & vec_elem_sum ( #5230 )
...
* feat(vector): add sub function
* chore: added check for vector length misalignment
* feat(vector): add `vec_sum` & `vec_elem_sum`
* chore: codefmt
* update lock file
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-26 15:07:13 +00:00
Ruihang Xia
00ad27dd2e
feat(bloom-filter): bloom filter applier ( #5220 )
...
* wip
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* draft search logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use defined BloomFilterReader
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* round the range end
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* finish index applier
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* integrate applier into mito2 with cache layer
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix cache key and add unit test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* provide bloom filter index size hint
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert BloomFilterReaderImpl::read_vec
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove dead code
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore null on eq
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add more tests and fix bloom filter logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-26 02:51:18 +00:00
discord9
abf34b845c
feat(flow): check sink table mismatch on flow creation ( #5112 )
...
* tests: more mismatch errors
* feat: check sink table schema if exists&prompt nice err msg
* chore: rm unused variant
* chore: fmt
* chore: cargo clippy
* feat: check schema on create
* feat: better err msg when mismatch
* tests: fix a schema mismatch
* todo: create sink table
* feat: create sink table
* fix: find time index
* tests: auto created sink table
* fix: remove empty keys
* refactor: per review
* chore: fmt
* test: sqlness
* chore: after rebase
2024-12-25 13:42:37 +00:00
Ruihang Xia
88d46a38ae
chore: bump opendal to fork version to fix prometheus layer ( #5223 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-24 08:54:59 +00:00
Ruihang Xia
68dd2916fb
feat: logs query endpoint ( #5202 )
...
* define endpoint
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* planner
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update lock file
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add unit test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix toml format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert metric change
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Update src/query/src/log_query/planner.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refactor and tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2024-12-24 06:21:19 +00:00
Zhenchi
4245bff8f2
feat(bloom-filter): add bloom filter reader ( #5204 )
...
* feat(bloom-filter): add bloom filter reader
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: remove unused dep
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix conflict
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* address comments
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-20 08:29:18 +00:00
Zhenchi
3d4121aefb
feat(bloom-filter): add memory control for creator ( #5185 )
...
* feat(bloom-filter): add memory control for creator
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* refactor: remove meaningless buf
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* feat: add codec for intermediate
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-20 06:59:44 +00:00
discord9
2d6f63a504
feat: show flow's mem usage in INFORMATION_SCHEMA.FLOWS ( #4890 )
...
* feat: add flow mem size to sys table
* chore: rm dup def
* chore: remove unused variant
* chore: minor refactor
* refactor: per review
2024-12-19 08:24:04 +00:00
Ruihang Xia
422d18da8b
feat: bump opendal and switch prometheus layer to the upstream impl ( #5179 )
...
* feat: bump opendal and switch prometheus layer to the upstream impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused files
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unused things
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove root dir on recovering cache
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* filter out non-files entry in test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-19 03:42:05 +00:00
Yohan Wal
7d1bcc9d49
feat: introduce Buffer for non-continuous bytes ( #5164 )
...
* feat: introduce Buffer for non-continuous bytes
* Update src/mito2/src/cache/index.rs
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* chore: apply review comments
* refactor: use opendal::Buffer
---------
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-12-18 03:45:38 +00:00
discord9
421088a868
test: sqlness upgrade compatibility tests ( #5126 )
...
* feat: simple version switch
* chore: remove debug print
* chore: add common folder
* tests: add drop table
* feat: pull versioned binary
* chore: don't use native-tls
* chore: rm outdated docs
* chore: new line
* fix: save old bin dir
* fix: switch version restart all node
* feat: use etcd
* fix: wait for election
* fix: normal sqlness
* refactor: hashmap for bin dir
* test: past 3 major version compat crate table
* refactor: allow using without setup etcd
2024-12-17 07:00:02 +00:00
Zhenchi
d821dc5a3e
feat(bloom-filter): add basic bloom filter creator (Part 1) ( #5177 )
...
* feat(bloom-filter): add a simple bloom filter creator (Part 1)
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: clippy
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: header
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* docs: add format comment
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-12-17 06:55:42 +00:00
Yingwen
8a5384697b
chore: add aquamarine to dep lists ( #5181 )
2024-12-17 01:45:50 +00:00
shuiyisong
9d7fea902e
chore: remove unused dep ( #5163 )
...
* chore: remove unused dep
* chore: remove more unused dep
2024-12-16 06:17:27 +00:00
Yohan Wal
4b4c6dbb66
refactor: cache inverted index with fixed-size page ( #5114 )
...
* feat: cache inverted index by page instead of file
* fix: add unit test and fix bugs
* chore: typo
* chore: ci
* fix: math
* chore: apply review comments
* chore: renames
* test: add unit test for index key calculation
* refactor: use ReadableSize
* feat: add config for inverted index page size
* chore: update config file
* refactor: handle multiple range read and fix some related bugs
* fix: add config
* test: turn to a fs reader to match behaviors of object store
2024-12-13 07:34:24 +00:00
Yohan Wal
03ad6e2a8d
feat(fuzz): add alter table options for alter fuzzer ( #5074 )
...
* feat(fuzz): add set table options to alter fuzzer
* chore: clippy is happy, I'm sad
* chore: happy ci happy
* fix: unit test
* feat(fuzz): add unset table options to alter fuzzer
* fix: unit test
* feat(fuzz): add table option validator
* fix: make clippy happy
* chore: add comments
* chore: apply review comments
* fix: unit test
* feat(fuzz): add more ttl options
* fix : #5108
* chore: add comments
* chore: add comments
2024-12-12 04:21:38 +00:00
Weny Xu
d53fbcb936
feat: introduce PuffinMetadataCache ( #5148 )
...
* feat: introduce `PuffinMetadataCache`
* refactor: remove too_many_arguments
* chore: fmt toml
2024-12-12 04:09:36 +00:00
Ruihang Xia
60f8dbf7f0
feat: implement v1/sql/parse endpoint to parse GreptimeDB's SQL dialect ( #5144 )
...
* derive ser/de
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* impl method
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deserialize
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-11 13:33:54 +00:00
discord9
d91517688a
chore: fix aws_lc not in depend tree check in CI ( #5121 )
...
* chore: fix aws_lc check in CI
* chore: update lock file
2024-12-11 07:02:03 +00:00
Yingwen
7c69ca0502
chore: bump main branch version to 0.12 ( #5133 )
...
chore: bump version to v0.12.0
2024-12-10 13:10:37 +00:00
Ning Sun
c0f498b00c
feat: update pgwire to 0.28 ( #5113 )
...
* feat: update pgwire to 0.28
* test: update tests
2024-12-09 03:12:11 +00:00
Ning Sun
3133f3fb4e
feat: add cursor statements ( #5094 )
...
* feat: add sql parsers for cursor operations
* feat: cursor operator
* feat: implement RecordBatchStreamCursor
* feat: implement cursor storage and execution
* test: add tests
* chore: update docstring
* feat: add a temporary sql rewrite for cast in limit
this issue is described in #5097
* test: add more sql for cursor integration test
* feat: reject non-select query for cursor statement
* refactor: address review issues
* test: add empty result case
* feat: address review comments
2024-12-06 09:32:22 +00:00
discord9
8b944268da
feat: ttl=0/instant/forever/humantime&ttl refactor ( #5089 )
...
* feat: ttl zero filter
* refactor: use TimeToLive enum
* fix: unit test
* tests: sqlness
* refactor: Option<TTL> None means UNSET
* tests: sqlness
* fix: 10000 years --> forever
* chore: minor refactor from reviews
* chore: rename back TimeToLive
* refactor: split imme request from normal requests
* fix: use correct lifetime
* refactor: rename immediate to instant
* tests: flow sink table default ttl
* refactor: per review
* tests: sqlness
* fix: ttl alter to instant
* tests: sqlness
* refactor: per review
* chore: per review
* feat: add db ttl type&forbid instant for db
* tests: more unit test
2024-12-06 09:20:42 +00:00
Weny Xu
01a308fe6b
refactor: relocate CLI to a dedicated directory ( #5101 )
...
* refactor: relocate CLI to a dedicated directory
* chore: expose method and const
* refactor: use BoxedError
* chore: expose DatabaseClient
* chore: add clone derive
2024-12-05 12:12:24 +00:00
Ruihang Xia
5092f5f451
feat: define basic structures and implement TimeFilter ( #5086 )
...
* feat: define basic structures and implement TimeFilter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* document column filter
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* define context
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change variable name to avoid typo checker
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change error referring style
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* refine context definition
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-12-04 07:39:33 +00:00
Lei, HUANG
a51853846a
fix: schema cache invalidation ( #5067 )
...
* fix: use SchemaCache to locate database metadata
* main:
Refactor SchemaMetadataManager to use TableInfoCacheRef
- Replace TableInfoManagerRef with TableInfoCacheRef in SchemaMetadataManager
- Update DatanodeBuilder to pass TableInfoCacheRef to SchemaMetadataManager
- Rename error MissingCacheRegistrySnafu to MissingCacheSnafu in datanode module
- Adjust tests to use new mock_schema_metadata_manager with TableInfoCacheRef
* fix/schema-cache-invalidation: Add cache module and integrate cache registry into datanode
• Implement build_datanode_cache_registry function to create cache registry for datanode
• Integrate cache registry into datanode by modifying DatanodeBuilder and HeartbeatTask
• Refactor InvalidateTableCacheHandler to InvalidateCacheHandler and move to common-meta crate
• Update Cargo.toml to include cache as a dev-dependency for datanode
• Adjust related modules (flownode, frontend, tests-integration, standalone) to use new cache handler and registry
• Remove obsolete handler module from frontend crate
* fix: fuzz imports
* chore: add some doc for cahce builder functions
* refactor: change table info cache to table schema cache
* fix: remove unused variants
* fix fuzz
* chore: apply suggestion
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* chore: apply suggestion
Co-authored-by: Weny Xu <wenymedia@gmail.com >
* fix: compile
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-12-03 10:44:29 +00:00
Ning Sun
bcadce3988
chore: remove openssl deps ( #5079 )
...
* chore: remove openssl deps
* ci: add ci task to check blacklisted dependencies
2024-12-02 07:12:15 +00:00
Ning Sun
36263830bb
refactor: remove built-in apidocs and schemars ( #5068 )
...
* feat: feature gate apidocs
* refactor: remove built-in apidocs and schemars
* remove redoc html
2024-11-29 03:31:02 +00:00
Zhenchi
91e933517a
chore: bump version of main branch to v0.11.0 ( #5057 )
...
chore: bump nightly version to v0.11
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-11-27 02:19:24 +00:00
Zhenchi
1cd6abb61f
chore: bump version to v0.10.1 ( #5048 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-11-25 03:20:54 +00:00
Zhenchi
a6571d3392
chore: bump version to 0.10.0 ( #5040 )
...
* chore: bump version to 0.10.0
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix sqlness version regex
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-11-22 03:56:25 +00:00
Yohan Wal
5f8d849981
feat: alter database ttl ( #5035 )
...
* feat: alter databaset ttl
* fix: make clippy happy
* feat: add unset database option
* fix: happy ci
* fix: happy clippy
* chore: fmt toml
* fix: fix header
* refactor: introduce `AlterDatabaseKind`
* chore: apply suggestions from CR
* refactor: add unset database option support
* test: add unit tests
* test: add sqlness tests
* feat: invalidate schema name value cache
* Apply suggestions from code review
* chore: fmt
* chore: update error messages
* test: add more test cases
* test: add more test cases
* Apply suggestions from code review
* chore: apply suggestions from CR
---------
Co-authored-by: WenyXu <wenymedia@gmail.com >
2024-11-21 12:41:41 +00:00
Weny Xu
14d997e2d1
feat: add unset table options support ( #5034 )
...
* feat: add unset table options support
* test: add tests
* chore: update greptime-proto
* chore: add comments
2024-11-21 03:52:56 +00:00
Zhenchi
db345c92df
feat(vector): remove simsimd and use nalgebra instead ( #5027 )
...
* feat(vector): remove `simsimd` and use `nalgebra` instead
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* keep thing simple
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-11-20 06:57:10 +00:00
Yohan Wal
55ced9aa71
refactor: split up different stmts ( #4997 )
...
* refactor: set and unset
* chore: error message
* fix: reset Cargo.lock
* Apply suggestions from code review
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
* Apply suggestions from code review
Co-authored-by: Weny Xu <wenymedia@gmail.com >
---------
Co-authored-by: jeremyhi <jiachun_feng@proton.me >
Co-authored-by: Weny Xu <wenymedia@gmail.com >
2024-11-20 06:02:51 +00:00
Zhenchi
2f260d8b27
fix: android build failed due to simsimd ( #5019 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-11-19 11:21:58 +00:00
Zhenchi
73e6bf399d
test: reduce round precision to avoid platform diff ( #5013 )
...
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-11-18 16:37:15 +00:00
shuiyisong
c199604ece
feat: Loki remote write ( #4941 )
...
* chore: add debug loki remote write url
* chore: add decode snappy
* chore: format output
* feature: impl loki remote write
* fix: special labels deserialize
* chore: move result to folder
* chore: finish todo in loki write
* test: loki write
* chore: fix cr issue
* chore: fix cr issue
* chore: fix cr issue
* chore: update pre-commit config
* chore: fix cr issue
Co-authored-by: dennis zhuang <killme2008@gmail.com >
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2024-11-18 08:39:17 +00:00
Weny Xu
7c135c0ef9
feat: introduce DynamicTimeoutLayer ( #5006 )
...
* feat: introduce `DynamicTimeoutLayer`
* test: add unit test
* chore: apply suggestions from CR
* feat: add timeout option for cli
2024-11-18 07:10:40 +00:00
Yohan Wal
a2852affeb
chore: rename change to modify ( #5000 )
...
* chore: rename change to modify
* chore: proto rev
2024-11-15 06:58:26 +00:00
