Commit Graph
67 Commits
Author SHA1 Message Date
Subhra264 b88f21aad3 test: add alert integration tests regarding scheduler (#4929)
New integration tests for cargo to test alert scheduling

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Tests**
- Introduced new asynchronous integration tests for multi-range alert
handling.
- Enhanced existing alert tests with improved assertions for creation
and deletion processes.
- Updated alert trigger conditions to ensure comprehensive validation of
alert functionalities.
- Added tests for alert behavior during retries after destination
failures and evaluation errors.
- Updated tests for the `SearchHistoryQueryBuilder` to reflect new SQL
query structure tailored to search events.
- Modified tests for `RedirectResponse` to accommodate multiple query
parameter orders.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-30 10:27:23 +05:30
Huaijin 56dcacb513 feat: use seaorm to refactor short_urls (#4862) 2024-10-24 19:59:06 +08:00
Hengfei Yang b72ac5b8d7 feat: Alerts support SNS destination (#4824) 2024-10-18 20:27:43 +08:00
Prabhat Sharma f39a3de5a1 fix: rename zinc labs to OpenObserve (#4822)
Rename Zinc Labs to OpenObserve
2024-10-18 13:11:52 +08:00
John Vandenberg d0079bb9bf docs: Fix typos (#4492) 2024-09-22 17:57:47 +08:00
Hengfei Yang d1ed457369 feat: new search engine (#4182) 2024-09-20 15:10:54 +08:00
Huaijin a7990a4d1d feat: impl union plan for single node (#4352) 2024-08-28 23:31:00 +08:00
BhargavandSubhradeep Chakraborty a5af316129 fix: issue #4326 User defined schema settings (#4350)
issue #4326 
  
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced new structures for updating stream settings, allowing for
more granular management of stream configurations.
- Added an asynchronous function for updating stream settings via HTTP
PUT requests.
- Implemented a new service route for handling updates to stream
settings.
  - Enhanced the `AddStream` component to retrieve updated settings.

- **Improvements**
- Improved schema management by tracking previous settings during
updates.
- Enhanced utility functions for comparing and managing stream settings.

- **Documentation**
- Updated documentation to reflect changes in API functionality for
creating and updating stream settings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
2024-08-27 20:52:26 +05:30
Ankur SrivastavaandHengfei Yang 8f3df9e7a1 feat: support reloading config vars using an API (#3623)
The new endpoint `/config/reload` will refresh the
environment variables.

There are a bunch of env-vars which will not be reloaded
for e.g. HTTP_PORT etc. these will need a restart.

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2024-05-30 08:23:27 +08:00
Hengfei Yang 6d0fbdd91d refactor: separate proto to a new crate (#3146) 2024-04-02 23:15:14 +08:00
Hengfei Yang e33ae00ea1 feat: merge dev into main (#3016) 2024-03-17 22:38:07 +08:00
Hengfei Yang d887b6eed2 feat: impl hash partition (#2647) 2024-02-04 19:07:03 +08:00
Hengfei Yang 79682120f0 feat: search queue management (#2621) 2024-02-01 11:54:01 +08:00
Hengfei Yang 325aa4b3f2 fix: unit tests (#2542)
- [x] change sqlite backend to a Mutex
- [x] fix unit tests
2024-01-24 22:42:20 +08:00
Hengfei Yang a4bae1a584 feat: Implement memtable + wal (#2261) 2023-12-28 07:40:22 +08:00
Hengfei Yang 1b0b8c2bf5 chore: update format (#2144)
fixed #2102
2023-12-07 10:10:45 +05:30
Prabhat Sharma 5db4ca191a change license to AGPL 2023-11-26 18:34:59 -08:00
Hengfei Yang 681f6e3af9 feat: support mysql for metadata (#1968) 2023-11-20 07:42:12 +08:00
Hengfei Yang 76fb283191 ci: upgrade rust to nightly-2023-10-24 (#1804) 2023-10-30 02:04:43 -05:00
Nrishinghananda Roy 414b4e0d63 fix: Clippy errors in tests (#1566)
`tests` module had a few clippy errors/warnings. Applied the fixes.
2023-09-26 03:29:51 -05:00
Hengfei Yang 019c4ccc53 fix: swagger under sub directory (#1476) 2023-09-09 12:36:23 +08:00
Ashish Kolhe c0114e36f6 feat: dashboard versioning (#1464) 2023-09-06 15:45:44 +05:30
Hengfei Yang 5a3173c689 feat: implement sqlite for metadata (#1449) 2023-09-04 11:00:28 +08:00
Hengfei Yang af521fde54 fix: update license (#1404) 2023-08-21 15:37:50 +08:00
Ankur Srivastava e2dd047bf5 refactor: move infra and meta inside common (#1105)
This is the first step towards splitting the repo into smaller sub
projects.
2023-07-06 12:32:07 +05:30
Ashish Kolhe f58ad652d1 fix: functions disabled due to feature flag (#1047) 2023-06-27 18:43:40 +08:00
Hengfei Yang 8c74eaac84 feat: impl metrics http json API (#999)
## Design Spec:

- POST `/api/:org/ingest/metrics/_json`

```json
[
    {
        "__name__": "stream name",
        "__type__": "counter / gauge / histogram / summary",
        "label_name1": "label_value1",
        "label_name2": "label_value2",
        "_timestamp": 0, // unix timestamp in millisecond
        "value": 0.0 // float64 value
    }
]
```

## How to use this API:
- must use `__name__` set stream name.
- must use `__type__` set metrics type. and the type can't be updated
after first record
- `_timestamp` needs UNIX timestamp or empty will fill by current
timestamp
- `value` should be float64 or int64

## Example:

POST `/api/default/ingest/metrics/_json`
```json
[
    {
        "__name__": "cpu",
        "__type__": "gauge",
        "namespace": "ziox",
        "container": "zo1",
        "_timestamp": 1687326643584062,
        "value": 10.1
    },
    {
        "__name__": "cpu",
        "__type__": "gauge",
        "namespace": "ziox",
        "container": "zo2",
        "value": 0.1
    }
]
```
2023-06-21 23:02:41 +08:00
BhargavandPrabhat Sharma 10dd9bab13 fix: multitenancy changes (#864)
Co-authored-by: Prabhat Sharma <hi.prabhat@gmail.com>
2023-06-02 16:29:11 +05:30
Hengfei Yang 7518811da0 feat: return global data retention in config api (#859) 2023-05-31 20:58:34 +08:00
Hengfei Yang 53e1a23921 feat: rename zincobserve to openobserve (#848) 2023-05-31 11:29:10 +08:00
Hengfei Yang ca2f55a57e feat: support skip tls verify when create alert destination (#819) 2023-05-26 17:58:38 +08:00
Ashish Kolhe b94e7e94d8 feat: enrichment functions (#772) 2023-05-20 14:24:46 +05:30
Prabhat Sharma 3843631ade fix: fix for grafana querying prometheus endpoint (#713)
Make change to prometheus endpoints so grafana queries can succeed.
2023-05-11 06:52:00 +05:30
Ashish Kolhe f20c9805bb feat: syslog ingestion (#711) 2023-05-11 06:18:24 +05:30
Hengfei Yang e5a7803d98 feat: metrics API (#686)
* feat: add prometheus API

* fix: unit test
2023-05-04 18:10:10 +08:00
Valeriy V. Vorotyntsev 89380dd8f5 feat: delete local search (#570) 2023-04-15 09:08:31 +08:00
Ashish Kolhe a54bf2f242 fix: refactor functions (#545)
* functions refactoring - removed lua

* update cargo

* fixed error when feature disabled
2023-04-12 12:00:20 +05:30
Hengfei Yang 81375d3e82 feat: rewrite tmpfs (#544) 2023-04-12 12:41:13 +08:00
Valeriy V. Vorotyntsev 2efc41b26c fix: fix e2e tests 2023-04-08 00:00:43 +03:00
d57c57c432 feat: Accept cloud watch logs through amazon kinesis firehose and VRL functions (#512)
* added vrl functions

* vrl functions

* incorporated comments

* Added kinesis changes

* fixes

* Apply suggestions from code review

Co-authored-by: Valeriy V. Vorotyntsev <valery.vv@gmail.com>

* Revert "Apply suggestions from code review"

This reverts commit b25690802d.

* Latest Changes

* access key changes

* clippy fixes

* fix: router support aws API

* fix: format code

* fix: test coverage

---------

Co-authored-by: Valeriy V. Vorotyntsev <valery.vv@gmail.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2023-04-07 08:24:14 +05:30
Sebastian YEPESandHengfei Yang d72976c3d8 feat: add storage operation times (#493)
* add storage operation times
* fix typo
* move to counter
* Final fix
* format code

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2023-04-04 11:08:41 +08:00
e289762604 fix: typed dashboard structs (#466)
* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* Added all aggregate functions

* FIXED TESTS

* fixed integration tests

* Updated coverage threshold

---------

Co-authored-by: ktx-keshavi <keshavi.bhalani@kiara.tech>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
2023-03-31 17:03:34 +05:30
Hengfei Yang e4359ccf11 fix: change error logs (#314) 2023-03-18 16:21:25 +05:30
Hengfei Yang b2f9e1a041 fix: add organizations and auth to openAPI (#303)
* fix: response organizations for login

* feat: add organization to openAPI

* fix: error this is no field _timestamp
2023-03-17 14:54:08 +08:00
Hengfei Yang f98275bdc8 feat: format ingest field name #294 (#301)
* feat: format ingest field name #294
* fix: delete add quote for sql
* fix: remove unused code
* fix: format code
* ci: test sccache
2023-03-17 11:29:23 +08:00
Ashish Kolheandhengfeiyang d51a6cf171 feat: remove org from user login screen (#298)
* remove org from user login screen

* fix: delete no need code

* fix: format code

---------

Co-authored-by: hengfeiyang <hengfei.yang@gmail.com>
2023-03-16 17:22:00 +05:30
Ashish KolheandHengfei Yang c453f38d58 feat: alert templates & destinations (#280)
* v1

* alerts refactoring

* added unit tests

* swagger changes

* fixed openAPI specs

* Changes to alerts

* v1

* alerts refactoring

* added unit tests

* swagger changes

* fixed openAPI specs

* Changes to alerts

* Fixed docs

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2023-03-16 14:23:12 +05:30
Hengfei Yang 11e70a0691 feat: add more metrics (#276)
* feat: add more metrics
* fix: query case count(Athena)
2023-03-14 20:35:54 +08:00
Hengfei Yang efea62fff2 fix: query error with != (#267)
* fix: query error with !=

* fix: add more unit tests
2023-03-13 00:26:55 +08:00
Hengfei Yang 0dbe81dab8 fix: add more unit tests (#241)
* fix: add more unit tests

* fix: slack icon
2023-03-10 10:46:26 +08:00