Weny Xu
acfa229641
chore: bump orc-rust to 0319acd ( #1847 )
2023-06-29 10:45:05 +08:00
LFC
fcff66e039
chore: deny unused results ( #1825 )
...
* chore: deny unused results
* rebase
2023-06-27 15:33:53 +08:00
Weny Xu
223cf31409
feat: support to copy from orc format ( #1814 )
...
* feat: support to copy from orc format
* test: add copy from orc test
* chore: add license header
* refactor: remove unimplemented macro
* chore: apply suggestions from CR
* chore: bump orc-rust to 0.2.3
2023-06-25 14:07:16 +08:00
Lei, HUANG
16c1ee2618
feat: incremental database backup ( #1240 )
...
* feat: incremental database backup
* chore: rebase develop
* chore: move backup to StatementExecutor
* feat: copy database parser
* chore: remove some todos
* chore: use timestamp string instead of i64 string
* fix: typo
2023-06-20 18:26:55 +08:00
Lei, HUANG
960b84262b
fix: abort parquet writer ( #1785 )
...
* fix: sst file size
* fix: avoid creating file when no row's been written
* chore: rename tests
* fix: some clippy issues
* fix: some cr comments
2023-06-19 03:19:31 +00:00
Ruihang Xia
ac3666b841
chore(deps): bump arrow/parquet to 40.0, datafuson to the latest HEAD ( #1677 )
...
* fix compile error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deprecated substrait
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update deps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* downgrade opendal to 0.33.1
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change finish's impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update test results
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* ignore failing cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-05-31 18:55:02 +08:00
Weny Xu
4094907c09
fix: fix type casting issue ( #1652 )
...
* fix: fix type casting issue
* chore: apply suggestion from CR
2023-05-27 00:17:56 +08:00
Lei, HUANG
8e7ec4626b
refactor: remove useless error ( #1624 )
...
* refactor: remove useless
* fix: remove useless error variant
2023-05-22 22:55:27 +08:00
WU Jingdi
486bb2ee8e
feat: Compress manifest and checkpoint ( #1497 )
...
* feat: Compress manifest and checkpoint
* refactor: use file extention infer compression type
* chore: apply suggestions from CR
* Update src/storage/src/manifest/storage.rs
Co-authored-by: Yingwen <realevenyag@gmail.com >
* chore: CR advices
* chore: Fix bugs, strengthen test
* chore: Fix CR, strengthen test
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-05-10 07:53:06 +00:00
Weny Xu
fbb7db42aa
chore: unify code styling ( #1523 )
2023-05-10 11:10:39 +08:00
Weny Xu
abd5a8ecbb
chore(datasource): make CompressionType follow the style of the guide ( #1522 )
2023-05-10 10:50:24 +08:00
Weny Xu
c471007edd
feat: support to copy table from/to CSV and JSON format file ( #1475 )
...
* refactor: refactor copy from executor
* feat: support to copy from CSV and JSON format files
* feat: support to copy table to the CSV and JSON format file
* test: add tests copy from/to
* chore: apply suggestions from CR
2023-05-04 17:20:28 +08:00
Weny Xu
bf35620904
refactor: refactor BufferedWriter ( #1439 )
...
* feat: implement ApproximateBufWriter
* refactor: refactor BufferedWriter
* refactor: remove ApproximateBufWriter
* fix: fix losing pending writes issue
* chore: fmt
* chore: remove unused import
* chore: rename method name
* feat: return written row count
* chore: apply suggestions from CR
* fix: fix counting the bytes_written twice issue
2023-04-27 14:45:33 +08:00
Weny Xu
f2167663b2
feat: support to create external table ( #1372 )
...
* feat: support to create external table
* chore: apply suggestions from CR
* test: add create external table without ts type
* chore: apply suggestions from CR
* fix: fix import typo
* refactor: move consts to table crate
* chore: apply suggestions from CR
* refactor: rename create_table_schema
2023-04-24 14:43:12 +08:00
Weny Xu
d374859e24
refactor: replace Copy Format with datasource Format ( #1435 )
...
* refactor: replace Copy Format with datasource Format
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2023-04-23 08:31:54 +00:00
Weny Xu
f2cc912c87
feat: implement ParquetFileReaderFactory ( #1423 )
...
* feat: implement ParquetFileReaderFactory
* refactor: use LazyParquetFileReader instead
* chore: apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-04-21 13:40:58 +08:00
Weny Xu
5a05e3107c
feat: implement parsing format from hashmap ( #1420 )
...
* feat: implement parsing format from hashmap
* chore: apply suggestions from CR
2023-04-19 16:29:31 +08:00
Weny Xu
408de51be8
feat: implement JsonOpener and CsvOpener ( #1367 )
...
* feat: introduce JsonOpener and CsvOpener
* refactor: refactor Opener
* docs: add doc
2023-04-17 11:42:16 +08:00
Weny Xu
fac9c17a9b
feat: implement infer schema from single file ( #1348 )
...
* feat: implement infer schema from file
* feat: implement compression type
* refactor: remove unnecessary BufReader
* refactor: remove SyncIoBridge and using tokio_util::io::SyncIoBridge instead
* chore: apply suggestions from CR
2023-04-11 16:59:30 +08:00
Ruihang Xia
da66138e80
refactor(error): remove backtrace, and introduce call-site location for debugging ( #1329 )
...
* wip: global replace
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove unneeded tests of errors
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix ErrorExt trait implementator
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyo3 tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-04-06 04:06:00 +00:00
Lei, HUANG
5edd2a3dbe
feat: upgrade opendal ( #1245 )
...
* chore: upgrade opendal
* chore: finish upgrading opendal
* fix: clippy complaints
* fix some tests
* fix: all unit tests
* chore: rebase develop
* fix: sqlness tests
* optimize imports
* chore: rebase develop
* doc: add todo
2023-03-28 09:47:33 +08:00
Weny Xu
e19c8fa2b6
refactor: combine Copy To and Copy From ( #1197 )
...
* refactor: combine Copy To and Copy From
* Apply suggestions from code review
Co-authored-by: LFC <bayinamine@gmail.com >
* Apply suggestions from code review
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
---------
Co-authored-by: LFC <bayinamine@gmail.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2023-03-20 19:23:25 +08:00
