Commit Graph
30 Commits
Author SHA1 Message Date
Subhra264 51e32f4b35 chore: don't ignore cargo test failure in unit tests ci (#4951)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Improved error handling during coverage testing to ensure failures are
no longer ignored.

- **New Features**
- Streamlined command execution flow with a default command set to
`cmd_check` when no arguments are provided.

- **Documentation**
	- Enhanced user feedback for unrecognized commands.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-30 16:11:27 +05:30
Taiming Liu 7f75a4d6db feat: use FST for inverted indexing (#4270) 2024-09-02 22:39:44 +08:00
Hengfei Yang 8a0dbe2cf4 feat: remove sql_mode, query_context and aggs from search API (#4089) 2024-07-31 11:23:11 +08:00
Hengfei Yang e33ae00ea1 feat: merge dev into main (#3016) 2024-03-17 22:38:07 +08:00
Ashish Kolhe 67aeec6d5b fix: unit test issue (#2761) 2024-02-23 12:30:33 +05:30
Hengfei Yang 79682120f0 feat: search queue management (#2621) 2024-02-01 11:54:01 +08:00
Hengfei Yang 325aa4b3f2 fix: unit tests (#2542)
- [x] change sqlite backend to a Mutex
- [x] fix unit tests
2024-01-24 22:42:20 +08:00
Ankur Srivastava 9519e2a34c feat: run unit tests on github actions (#2394) 2024-01-16 15:23:18 +01:00
Hengfei Yang c6a46ac315 feat: upgrade datafusion to 31 (#1506) 2023-09-19 17:43:17 +08:00
Hengfei Yang 019c4ccc53 fix: swagger under sub directory (#1476) 2023-09-09 12:36:23 +08:00
Hengfei Yang 5a3173c689 feat: implement sqlite for metadata (#1449) 2023-09-04 11:00:28 +08:00
Hengfei Yang a506782fa5 fix: query fast mode with order by and distinct (#1396) 2023-08-18 16:21:57 +08:00
Hengfei Yang 2abbcc8af2 fix: unit test (#1160) 2023-07-12 17:19:01 +08:00
Hengfei Yang 4fb16a863f ci: add debug image for production version (#1036) 2023-06-26 18:23:59 +08:00
Hengfei Yang 23f4dd48c7 perf: improve query when no enough memory cache (#847) 2023-05-30 16:11:07 +08:00
Hengfei Yang ca2f55a57e feat: support skip tls verify when create alert destination (#819) 2023-05-26 17:58:38 +08:00
Valeriy V. Vorotyntsev 26448a9820 chore: don't let unmet coverage targets fail the CI
```
❌ Functions coverage is below threshold: 62.95% < 63.0%
✅ Lines coverage: 55.87%
✅ Regions coverage: 43.13%

[Container] 2023/05/09 09:07:40 Command did not exit successfully ./coverage.sh exit status 1
[Container] 2023/05/09 09:07:40 Phase complete: BUILD State: FAILED
[Container] 2023/05/09 09:07:40 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: ./coverage.sh. Reason: exit status 1
```

As I wrote on Slack:

> ❓ How useful is coverage.sh, particularly ‘Functions coverage’?
>
> I don't know how `cargo llvm-cov` calculates this ratio. Probably it divides the number of tested functions by the total number of functions.
>
> Splitting big functions into smaller pieces improves code readability. It also increases the total number of functions. If my assumption about `cargo llvm-cov` algorithm is correct, splitting code into smaller functions decreases the ‘Functions coverage’ ratio.
>
> If you look at [the patch] that fails the CI due to insufficient ‘Functions coverage’, you'll see that the new code there is accompanied with just-about-right amount of unit tests. I mean, apart from `MetricType`-to-string conversion, what else is there to be tested? How useful would those tests be?
>
> IMO, ‘Functions coverage’ target of `coverage.sh` adds more friction than benefit.
>
> I propose that we don't let `coverage.sh` fail the CI.

[the patch]: https://github.com/zinclabs/zincobserve/pull/695/files
2023-05-09 16:49:13 +03:00
Ashish Kolheandhengfeiyang 4ee2776719 feat: mlua with luau as sandbox (#597)
* feat: lua with sandbox

* add lua to query

* feat: enable feature zo_functions for CI

* update coverage

---------

Co-authored-by: hengfeiyang <hengfei.yang@gmail.com>
2023-04-19 16:15:15 +05:30
Valeriy V. Vorotyntsev 409e37fd53 test: reduce function coverage by 1% :-/ 2023-04-15 16:41:12 +03:00
Hengfei Yang 81375d3e82 feat: rewrite tmpfs (#544) 2023-04-12 12:41:13 +08:00
Valeriy V. Vorotyntsev 0722d456fa chore: revamp coverage.sh
- Generate coverage report in JSON format --- it used to be a raw text
  file (non-JSON) with `.json` extension.
- Parse JSON data with Python instead of counting and `cut`ting fields.
  The resulting code is more readable.
- Allow specifying of coverage thresholds via environment variables.
- Generate HTML report and open it in the default browser with
  `./coverage.sh html`.
- Implement CLI.
2023-04-07 21:26:05 +03:00
Ashish Kolhe 22fa7e51af feat: added support for vpc flow logs (#516)
* Added support for vpc flowlogs

* fixed UT

* updated cov.sh
2023-04-07 13:36:29 +05:30
d57c57c432 feat: Accept cloud watch logs through amazon kinesis firehose and VRL functions (#512)
* added vrl functions

* vrl functions

* incorporated comments

* Added kinesis changes

* fixes

* Apply suggestions from code review

Co-authored-by: Valeriy V. Vorotyntsev <valery.vv@gmail.com>

* Revert "Apply suggestions from code review"

This reverts commit b25690802d.

* Latest Changes

* access key changes

* clippy fixes

* fix: router support aws API

* fix: format code

* fix: test coverage

---------

Co-authored-by: Valeriy V. Vorotyntsev <valery.vv@gmail.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2023-04-07 08:24:14 +05:30
e289762604 fix: typed dashboard structs (#466)
* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* Added all aggregate functions

* FIXED TESTS

* fixed integration tests

* Updated coverage threshold

---------

Co-authored-by: ktx-keshavi <keshavi.bhalani@kiara.tech>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
2023-03-31 17:03:34 +05:30
Hengfei Yang cab41742a2 fix: unit test (#272) 2023-03-13 15:51:20 +08:00
Hengfei Yang 7b2ad6a2db feat: support encode SQL (#223)
* feat: support encode SQL

* fix: add more unit tests

* perf: improve inline
2023-03-09 11:49:23 +08:00
Hengfei Yang 3cca9ad12b feat: stream delete (#193)
* feat: update datafusion to 19

* feat: add stream deleting

* feat: data delete

* feat: stream delete done for local mode

* feat: data delete work on s3

* feat: data delete works with gcs

* feat: improve data delete

* feat: improve data delete

* feat: data delete done with merge file list

* fix: comment some unused code

* test: change coverage

* docs: update readme
2023-03-06 18:44:42 +08:00
Ashish Kolhe fb49c29415 added limit on number of fields in record (#166) 2023-02-27 13:58:31 +05:30
Ashish KolheandHengfei Yang 4108197943 fix: added UT (#158)
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2023-02-27 10:51:04 +08:00
oasisk 147362cb40 zinc-observe initial commit 2023-02-02 09:03:11 +05:30