rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-03 11:52:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,733 Commits 103 Branches 225 Tags
a9c342b0f7c34f547fecd3dc131f55a89e8a1321
Commit Graph

4733 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yingwen
a9c342b0f7 feat: support setting sst_format in table options (#7068) 
* feat: add FormatType to support multi format in the future

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: add sst_format to RegionOptions

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: sets the sst_format based on RegionOptions

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: add sst_format to mito table options

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: fix RegionManifest deserialization without sst_format

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: remove Parquet suffix from FormatType

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: prefer RegionOptions::sst_format in compactor/memtable builder

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: rename enable_experimental_flat_format to
default_experimental_flat_format

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: update config.md

Signed-off-by: evenyag <realevenyag@gmail.com>

* style: fmt

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: update manifest test

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fix compiler errors, handle sst_format in remap_manifest

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-10-13 08:38:37 +00:00
Ruihang Xia
1a73b485fe feat: apply region partition expr to region scan (#7067) 
* handle null in partition expr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* apply region partition expr on scanning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tidy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix gt/gteq

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-10-13 07:38:19 +00:00
Ruihang Xia
ab46127414 feat: remap SST files for partition change (#7071) 
* initial impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update expr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* immutable file meta

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tidy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reduce state

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* inherit manifest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify test cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix rebase error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* log new exprs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-10-13 03:13:14 +00:00
LFC
8fe17d43d5 chore: update rust to nightly 2025-10-01 (#7069) 
* chore: update rust to nightly 2025-10-01

Signed-off-by: luofucong <luofc@foxmail.com>

* chore: nix update

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
 2025-10-11 07:30:52 +00:00
Weny Xu
40e9ce90a7 refactor: restructure sqlness to support multiple envs and extract common utils (#7066) 
* refactor: restructure sqlness to support multiple envs and extract common utils

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore(ci): update sqlness cmd

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: error fmt

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: only reconnect mysql and pg client

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-10-11 06:34:17 +00:00
discord9
ba034c5a9e feat: explain custom statement (#7058) 
* feat: explain tql cte

Signed-off-by: discord9 <discord9@163.com>

* chore: unused

Signed-off-by: discord9 <discord9@163.com>

* fix: analyze format

Signed-off-by: discord9 <discord9@163.com>

* Update src/sql/src/statements/statement.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>
Signed-off-by: discord9 <discord9@163.com>

* test: sqlness

Signed-off-by: discord9 <discord9@163.com>

* pcr

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2025-10-11 06:27:51 +00:00
Ruihang Xia
e46ce7c6da feat: divide subtasks from old/new partition rules (#7003) 
* feat: divide subtasks from old/new partition rules

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change copyright year

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify filter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/partition/src/subtask.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
 2025-10-11 06:17:25 +00:00
dennis zhuang
57d84b9de5 feat: supports value aliasing in TQL (#7041) 
* feat: supports value aliasing in TQL

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: invalid checking

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: remove invalid checking

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: add explain test

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: improve parser

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: add explain TQL-CTE

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2025-10-11 02:49:09 +00:00
Ning Sun
749a5ab165 feat: struct value and vector (#7033) 
* feat: struct value

Signed-off-by: Ning Sun <sunning@greptime.com>

* feat: update for proto module

* feat: wip struct type

* feat: implement more vector operations

* feat: make datatype and api

* feat: reoslve some compilation issues

* feat: resolve all compilation issues

* chore: format update

* test: resolve tests

* test: test and refactor value-to-pb

* feat: add more tests and fix for value types

* chore: remove dbg

* feat: test and fix iterator

* fix: resolve struct_type issue

* refactor: use vec for struct items

* chore: update proto to main branch

* refactor: address some of review issues

* refactor: update for further review

* Add validation on new methods

* feat: update struct/list json serialization

* refactor: reimplement get in struct_vector

* refactor: struct vector functions

* refactor: fix lint issue

* refactor: address review comments

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
 2025-10-10 21:49:51 +00:00
LFC
3738440753 feat: align influxdb line timestamp with table time index (#7057) 
* feat: align influxdb line timestamp with table time index

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-10-10 07:37:52 +00:00
Ning Sun
aa84642afc refactor!: remove pb_value to json conversion, keep json output consistent (#7063) 
* refactor: remove pb_value to json

* chore: remove unused module
 2025-10-10 07:09:20 +00:00
Ning Sun
af213be403 refactor: remove duplicated valueref to json (#7062) 2025-10-10 07:08:26 +00:00
Sicong Hu
779865d389 feat: introduce IndexBuildTask for async index build (#6927) 
* feat: add framework for asynchronous index building

Signed-off-by: SNC123 <sinhco@outlook.com>

* test: add unit tests for IndexBuildTask

Signed-off-by: SNC123 <sinhco@outlook.com>

* chore: clippy,format,fix-udeps

Signed-off-by: SNC123 <sinhco@outlook.com>

* fix: correct write cache logic in IndexBuildTask

Signed-off-by: SNC123 <sinhco@outlook.com>

* chore: clippy, resolve conflicts

Signed-off-by: SNC123 <sinhco@outlook.com>

* chore: resolve conflicts

Signed-off-by: SNC123 <sinhco@outlook.com>

* fix: apply review suggestions

Signed-off-by: SNC123 <sinhco@outlook.com>

* chore: resolve conflicts

Signed-off-by: SNC123 <sinhco@outlook.com>

* fix: clean up index files in aborted case

Signed-off-by: SNC123 <sinhco@outlook.com>

* refactor: move manifest update logic into IndexBuildTask

Signed-off-by: SNC123 <sinhco@outlook.com>

* fix: enhance check file logic and error handling

Signed-off-by: SNC123 <sinhco@outlook.com>

---------

Signed-off-by: SNC123 <sinhco@outlook.com>
 2025-10-10 03:29:32 +00:00
Yingwen
47c1ef672a fix: support dictionary in regex match (#7055) 
* fix: support dictionary in regex match

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: get key from keys buffer directly

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-10-10 03:03:34 +00:00
shyam
591b9f3e81 fix: show proper error msg, when executing non-admin functions as admin functions (#7061) 
Signed-off-by: Shyamnatesan <shyamnatesan21@gmail.com>
 2025-10-10 01:25:49 +00:00
LFC
979c8be51b feat: able to pass external service for sqlness test (#7032) 
feat: able to pass external service instead of creating inside for sqlness test

Signed-off-by: luofucong <luofc@foxmail.com>
 2025-10-09 07:02:19 +00:00
Yingwen
45b1458254 fix: only skips auto convert when encoding is sparse (#7056) 
* fix: only skips auto convert when encoding is sparse

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: address comment and add tests

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-10-09 06:50:48 +00:00
fys
4cdcf2ef39 chore: add trigger querier factory trait (#7053) 
feat: add trigger-querier-factory-ent
 2025-10-09 02:16:50 +00:00
shuiyisong
b24a55cea4 chore: rename the default ts column name to greptime_timestamp for influxdb line protocol (#7046) 
* chore: rename influxdb ts column name to greptime_timestamp

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: tests

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2025-10-09 02:14:11 +00:00
Ning Sun
1aa4f346a0 fix: build_grpc_server visibility (#7054) 2025-10-06 03:16:48 +00:00
Ning Sun
f7202bc176 feat: pgwire 0.33 update (#7048) v0.18.0-nightly-20251006 2025-10-03 08:06:05 +00:00
Yingwen
b7045e57a5 feat: enable zstd for bulk memtable encoded parts (#7045) 
feat: enable zstd in bulk memtable

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-10-02 16:05:33 +00:00
Ning Sun
660790148d fix: various typos reported by CI (#7047) 
* fix: various typos reported by CI

* fix: additional typo
 2025-10-02 15:11:09 +00:00
zyy17
d777e8c52f refactor: add cgroup metrics collector (#7038) 
Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-09-30 02:26:02 +00:00
zyy17
efa616ce44 fix: use instance lables to fetch greptime_memory_limit_in_bytes and greptime_cpu_limit_in_millicores metrics (#7043) 
fix: remove unnecessary labels of standalone dashboard.json

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-09-29 11:43:35 +00:00
LFC
5b13fba65b refactor: make Function trait a simple shim of DataFusion UDF (#7036) 
Signed-off-by: luofucong <luofc@foxmail.com>
 2025-09-29 09:07:39 +00:00
LFC
aa05b3b993 feat: add max_connection_age config to grpc server (#7031) 
* feat: add `max_connection_age` config to grpc server

Signed-off-by: luofucong <luofc@foxmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2025-09-29 07:32:43 +00:00
fys
c4a7cc0adb chore: improve create trigger display (#7027) 
* chore: improve create_trigger_statement display

* improve display of create trigger

* add components for frontend

* Revert "add components for frontend"

This reverts commit 8d71540a72.
 2025-09-29 02:22:43 +00:00
Yingwen
90d37cb10e fix: fix panic and limit concurrency in flat format (#7035) 
* feat: add a semaphore to control flush concurrency

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: build FlatSchemaOptions from encoding in FlatWriteFormat

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: remove allow dead_code

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: handle sparse encoding in FlatCompatBatch

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: add time index column in try_new_compact_sparse

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add test for compaction and sparse encoding

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: remove comment

Signed-off-by: evenyag <realevenyag@gmail.com>

* style: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-09-29 02:20:06 +00:00
localhost
4a3c5f85e5 fix: fix test_resolve_relative_path_relative on windows (#7039) v0.18.0-nightly-20250929 2025-09-28 13:03:57 +00:00
discord9
3ca5c77d91 chore: not warning (#7037) 
Signed-off-by: discord9 <discord9@163.com>
 2025-09-28 08:11:27 +00:00
discord9
8bcf4a8ab5 test: update unit test by passing extra sort columns (#7030) 
* tests: fix unit test by passing one sort columns

Signed-off-by: discord9 <discord9@163.com>

* chore: per copilot

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2025-09-28 03:22:43 +00:00
zyy17
0717773f62 refactor!: add enable_read_cache config to support disable read cache explicitly (#6834) 
* refactor: add `enable_read_cache` config to support disable read cache explicitly

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: if `cache_path` is empty and `enable_read_cache` is true, set the default cache dir

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: remove the unessary Option type for `ObjectStorageCacheConfig`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: sanitize cache config in `DatanodeOptions` and `StandaloneOptions`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: code review comment

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: apply code review comments

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-09-26 09:44:12 +00:00
shuiyisong
195ed73448 chore: disable file not exist on watch_file_user_provider (#7028) 
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2025-09-26 09:29:59 +00:00
LFC
243dbde3d5 refactor: rewrite some UDFs to DataFusion style (final part) (#7023) 
Signed-off-by: luofucong <luofc@foxmail.com>
 2025-09-26 09:24:29 +00:00
discord9
aca8b690d1 fix: step aggr merge phase not order nor filter (#6998) 
* fix: not order

Signed-off-by: discord9 <discord9@163.com>

* test: redacted

Signed-off-by: discord9 <discord9@163.com>

* feat: fix up state wrapper

Signed-off-by: discord9 <discord9@163.com>

* df last_value state not as promised!

Signed-off-by: discord9 <discord9@163.com>

* fix?: could fix better

Signed-off-by: discord9 <discord9@163.com>

* test: unstable result

Signed-off-by: discord9 <discord9@163.com>

* fix: work around by fixing state

Signed-off-by: discord9 <discord9@163.com>

* chore: after rebase fix

Signed-off-by: discord9 <discord9@163.com>

* chore: finish some todo

Signed-off-by: discord9 <discord9@163.com>

* chore: per copilot

Signed-off-by: discord9 <discord9@163.com>

* refactor: not fix but just notify mismatch

Signed-off-by: discord9 <discord9@163.com>

* chore: warn -> debug state mismatch

Signed-off-by: discord9 <discord9@163.com>

* chore: refine error msg

Signed-off-by: discord9 <discord9@163.com>

* test: sqlness add last_value date_bin test

Signed-off-by: discord9 <discord9@163.com>

* ?: substrait order by decode failure

Signed-off-by: discord9 <discord9@163.com>

* unit test reproduce that

Signed-off-by: discord9 <discord9@163.com>

* feat: support state wrapper's order serde in substrait

Signed-off-by: discord9 <discord9@163.com>

* refactor: stuff

Signed-off-by: discord9 <discord9@163.com>

* test: standalone/distributed different exec

Signed-off-by: discord9 <discord9@163.com>

* fmt

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* refactor: closure

Signed-off-by: discord9 <discord9@163.com>

* test: first value order by

Signed-off-by: discord9 <discord9@163.com>

* refactor: per cr

Signed-off-by: discord9 <discord9@163.com>

* feat: ScanHint last_value last row selector

Signed-off-by: discord9 <discord9@163.com>

* docs: per cr

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2025-09-26 09:12:45 +00:00
ZonaHe
9564180a6a feat: update dashboard to v0.11.6 (#7026) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2025-09-26 06:35:37 +00:00
dennis zhuang
17d16da483 feat: supports expression in TQL params (#7014) 
* feat: supports expression in TQL params

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: by cr comments

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: comment

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: by cr comments

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2025-09-26 03:43:50 +00:00
Ning Sun
c1acce9943 refactor: cleanup datafusion-pg-catalog dependencies (#7025) 
* refactor: cleanup datafusion-pg-catalog dependencies

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: toml format

* feat: update upstream

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
 2025-09-26 03:07:00 +00:00
Ruihang Xia
0790835c77 feat!: improve greptime_identity pipeline behavior (#6932) 
* flat by default, store array in string

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* expose max_nested_levels param, store string instead of error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove flatten option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unused errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-09-25 15:28:28 +00:00
zyy17
280df064c7 chore: add some trace logs in fetching data from cache and object store (#6877) 
* chore: add some important debug logs

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* chore: add traces logs in `fetch_byte_ranges()`

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2025-09-25 11:54:22 +00:00
discord9
11a08d1381 fix: not step when aggr have order by/filter (#7015) 
* fix: not applied

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* test: confirm order by not push down

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2025-09-25 08:43:18 +00:00
shuiyisong
06a4f0abea chore: add function for getting started on metasrv (#7022) 
Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2025-09-25 08:24:23 +00:00
dennis zhuang
c6e5552f05 test: migrate aggregation tests from duckdb, part4 (#6965) 
* test: migrate aggregation tests from duckdb, part4

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: rename tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: comments

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: ignore zero weights test

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: remove duplicated sql

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2025-09-25 08:00:17 +00:00
discord9
9c8ff1d8a0 fix: skip placeholder when partition columns (#7020) 
Signed-off-by: discord9 <discord9@163.com>
 2025-09-25 07:01:49 +00:00
Yingwen
cff9cb6327 feat: converts batches in old format to the flat format in query time (#6987) 
* feat: use correct projection index for old format

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: remove allow dead_code from format

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: check and convert old format to flat format

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: sub primary key num from projection

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: always convert the batch in FlatRowGroupReader

Signed-off-by: evenyag <realevenyag@gmail.com>

* style: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: Change &Option<&[]> to Option<&[]>

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: only build arrow schema once

adds a method flat_sst_arrow_schema_column_num() to get the field num

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: Handle flat format and old format separately

Adds two structs ParquetFlat and ParquetPrimaryKeyToFlat.
ParquetPrimaryKeyToFlat delegates stats and projection to the
PrimaryKeyReadFormat.

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: handle non string tag correctly

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: do not register file cache twice

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: clean temp files

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: add rows and bytes to flush success log

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: convert format in memtable

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: add compaction flag to ScanInput

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: compaction should use old format for sparse encoding

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: merge schema use old format in sparse encoding

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: reads legacy format but not convert if skip_auto_convert

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: suppport sparse encoding in bulk parts

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2025-09-25 06:42:22 +00:00
Ning Sun
964dc254aa feat: upgraded pg_catalog support (#6918) 
* refactor: add datafusion-postgres dependency

* refactor: move and include pg_catalog udfs

* chore: update upstream

* feat: register table function pg_get_keywords

* feat: bridge CatalogInfo for our CatalogManager

Signed-off-by: Ning Sun <sunning@greptime.com>

* feat: convert pg_catalog table to our system table

* feat: bridge system catalog with datafusion-postgres

Signed-off-by: Ning Sun <sunning@greptime.com>

* feat: add more udfs

* feat: add compatibility rewriter to postgres handler

* fix: various fix

* fmt: fix

* fix: use functions from pg_catalog library

* fmt

* fix: sqlness runner

Signed-off-by: Ning Sun <sunning@greptime.com>

* test: adopt arrow 56.0 to 56.1 memory size change

* fix: add additional udfs

* chore: format

* refactor: return None when creating system table failed

Signed-off-by: Ning Sun <sunning@greptime.com>

* chore: provide safety comments about expect usage

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
 2025-09-25 04:05:34 +00:00
dennis zhuang
91a727790d feat: supports permission mode for static user provider (#7017) 
* feat: supports permission mode for static user provider

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: style

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: comment

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-09-25 03:45:31 +00:00
Weny Xu
07b9de620e fix(cli): fix FS object store handling of absolute paths (#7018) 
* fix(cli): fix FS object store handling of absolute paths

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add unit tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* Update src/cli/src/utils.rs

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
 2025-09-25 03:38:33 +00:00
LFC
6d0dd2540e refactor: rewrite some UDFs to DataFusion style (part 4) (#7011) 
Signed-off-by: luofucong <luofc@foxmail.com>
 2025-09-24 19:50:58 +00:00