starred/openobserve
1
0
Fork 0
mirror of https://github.com/openobserve/openobserve.git synced 2026-11-06 08:09:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,153 Commits 631 Branches 244 Tags
b9fcbb2a63da609c722ea56f8fd249dcd51dcf87
Commit Graph
54 Commits
Author SHA1 Message Date
Prabhat Sharma 5db4ca191a change license to AGPL 2023-11-26 18:34:59 -08:00
Ankur Srivastava c9008588f3 feat: implement ability to save search views (#1756) 
Created the standard GET,PUT,DELETE verbs for retrieving and
saving views which can be reused later.

Further details in:
Fixes https://github.com/openobserve/openobserve/issues/970
 2023-11-23 12:31:33 +01:00
Hengfei Yang 681f6e3af9 feat: support mysql for metadata (#1968) 2023-11-20 07:42:12 +08:00
Hengfei Yang 76fb283191 ci: upgrade rust to nightly-2023-10-24 (#1804) 2023-10-30 02:04:43 -05:00
Ankur Srivastava 5b01947491 feat: rum/maxmind integration (#1520) 
This PR implements the first draft for the Real User Monitoring.

The endpoints at the moment are:

- `/rum/v1/{org_id}/rum`
- `/rum/v1/{org_id}/replay`
- `/rum/v1/{org_id}/logs`

For now the authentication is done using `oo-api-key` which is
being passed in the query-param using `browser-sdk`.

As of now, the data is being collected in the following three streams:

- `_rumdata` for rum data
- `_rumlog` for log data
- `_sessionreplay` for session-replay data

The data is being ingested using the multi-json implementation, with a
small change where some extra data is also ingested in the json from
the headers, query-params etc. using a middleware extractor and stores
the data in `HashMap<String, String>` and gets ingested in this
multi-json.

A file containing regexes for common user-agents is also committed which
is read during the initialization time which is used to parse incoming
user-agents and browser, os etc are inserted into the incoming row of
data.

Added the endpoints for `get`, `put`, `post` for rum-tokens.

- `/api/{org_id}/organizations/rumtoken`

#### maxmind integration 
Integrated the maxmind-db data in the source. Currently the data
is injected via the middleware and the file gets uploaded every
24 hours.
The file pointer gets mutated and updated when there is a change
in the mmdb file. To efficiently do this, we also query the sha256
of our file and compare it using the .sha256 file available on the
public bucket.
 2023-10-18 15:09:24 +02:00
Nrishinghananda Roy 414b4e0d63 fix: Clippy errors in tests (#1566) 
`tests` module had a few clippy errors/warnings. Applied the fixes.
 2023-09-26 03:29:51 -05:00
Hengfei Yang 019c4ccc53 fix: swagger under sub directory (#1476) 2023-09-09 12:36:23 +08:00
Ashish Kolhe c0114e36f6 feat: dashboard versioning (#1464) 2023-09-06 15:45:44 +05:30
Hengfei Yang 5a3173c689 feat: implement sqlite for metadata (#1449) 2023-09-04 11:00:28 +08:00
Hengfei Yang af521fde54 fix: update license (#1404) 2023-08-21 15:37:50 +08:00
Ankur Srivastava e2dd047bf5 refactor: move infra and meta inside common (#1105) 
This is the first step towards splitting the repo into smaller sub
projects.
 2023-07-06 12:32:07 +05:30
Ashish Kolhe f58ad652d1 fix: functions disabled due to feature flag (#1047) 2023-06-27 18:43:40 +08:00
Hengfei Yang 8c74eaac84 feat: impl metrics http json API (#999) 
## Design Spec:

- POST `/api/:org/ingest/metrics/_json`

```json
[
    {
        "__name__": "stream name",
        "__type__": "counter / gauge / histogram / summary",
        "label_name1": "label_value1",
        "label_name2": "label_value2",
        "_timestamp": 0, // unix timestamp in millisecond
        "value": 0.0 // float64 value
    }
]
```

## How to use this API:
- must use `__name__` set stream name.
- must use `__type__` set metrics type. and the type can't be updated
after first record
- `_timestamp` needs UNIX timestamp or empty will fill by current
timestamp
- `value` should be float64 or int64

## Example:

POST `/api/default/ingest/metrics/_json`
```json
[
    {
        "__name__": "cpu",
        "__type__": "gauge",
        "namespace": "ziox",
        "container": "zo1",
        "_timestamp": 1687326643584062,
        "value": 10.1
    },
    {
        "__name__": "cpu",
        "__type__": "gauge",
        "namespace": "ziox",
        "container": "zo2",
        "value": 0.1
    }
]
```
 2023-06-21 23:02:41 +08:00
BhargavandPrabhat Sharma 10dd9bab13 fix: multitenancy changes (#864) 
Co-authored-by: Prabhat Sharma <hi.prabhat@gmail.com>
 2023-06-02 16:29:11 +05:30
Hengfei Yang 7518811da0 feat: return global data retention in config api (#859) 2023-05-31 20:58:34 +08:00
Hengfei Yang 53e1a23921 feat: rename zincobserve to openobserve (#848) 2023-05-31 11:29:10 +08:00
Hengfei Yang ca2f55a57e feat: support skip tls verify when create alert destination (#819) 2023-05-26 17:58:38 +08:00
Ashish Kolhe b94e7e94d8 feat: enrichment functions (#772) 2023-05-20 14:24:46 +05:30
Prabhat Sharma 3843631ade fix: fix for grafana querying prometheus endpoint (#713) 
Make change to prometheus endpoints so grafana queries can succeed.
 2023-05-11 06:52:00 +05:30
Ashish Kolhe f20c9805bb feat: syslog ingestion (#711) 2023-05-11 06:18:24 +05:30
Hengfei Yang e5a7803d98 feat: metrics API (#686) 
* feat: add prometheus API

* fix: unit test
 2023-05-04 18:10:10 +08:00
Valeriy V. Vorotyntsev 89380dd8f5 feat: delete local search (#570) 2023-04-15 09:08:31 +08:00
Ashish Kolhe a54bf2f242 fix: refactor functions (#545) 
* functions refactoring - removed lua

* update cargo

* fixed error when feature disabled
 2023-04-12 12:00:20 +05:30
Hengfei Yang 81375d3e82 feat: rewrite tmpfs (#544) 2023-04-12 12:41:13 +08:00
Valeriy V. Vorotyntsev 2efc41b26c fix: fix e2e tests 2023-04-08 00:00:43 +03:00
Ashish KolheValeriy V. VorotyntsevHengfei Yang
 		d57c57c432 feat: Accept cloud watch logs through amazon kinesis firehose and VRL functions (#512) 
* added vrl functions

* vrl functions

* incorporated comments

* Added kinesis changes

* fixes

* Apply suggestions from code review

Co-authored-by: Valeriy V. Vorotyntsev <valery.vv@gmail.com>

* Revert "Apply suggestions from code review"

This reverts commit b25690802d.

* Latest Changes

* access key changes

* clippy fixes

* fix: router support aws API

* fix: format code

* fix: test coverage

---------

Co-authored-by: Valeriy V. Vorotyntsev <valery.vv@gmail.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-04-07 08:24:14 +05:30
Sebastian YEPESandHengfei Yang d72976c3d8 feat: add storage operation times (#493) 
* add storage operation times
* fix typo
* move to counter
* Final fix
* format code

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-04-04 11:08:41 +08:00
Valeriy V. Vorotyntsevktx-keshavioasisk
 		e289762604 fix: typed dashboard structs (#466) 
* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* WIP: introduce `DashboardXxx`; use it for `save_dashboard`

* service::db::dashboard::list: handle errors, don't unwrap

* [WIP] Update `service::db::dashboard::list`

* XXX The code compiles, but the Dashboards page shows nothing

* web: repair 'dashboards' page

Don't `JSON.parse` backend's response, it's JSON already.

* [WIP] Repair the "view dashboard" page

Saving panel(s) is still broken.

* meta::dashboards.rs: define `Panel` struct

* Fix `Panel` struct definition

* Use `crate::common::json` instead of `serde_json`

* remove json stringify for the dashboard

* Fixed dashboards

* Added all aggregate functions

* FIXED TESTS

* fixed integration tests

* Updated coverage threshold

---------

Co-authored-by: ktx-keshavi <keshavi.bhalani@kiara.tech>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
 2023-03-31 17:03:34 +05:30
Hengfei Yang e4359ccf11 fix: change error logs (#314) 2023-03-18 16:21:25 +05:30
Hengfei Yang b2f9e1a041 fix: add organizations and auth to openAPI (#303) 
* fix: response organizations for login

* feat: add organization to openAPI

* fix: error this is no field _timestamp
 2023-03-17 14:54:08 +08:00
Hengfei Yang f98275bdc8 feat: format ingest field name #294 (#301) 
* feat: format ingest field name #294
* fix: delete add quote for sql
* fix: remove unused code
* fix: format code
* ci: test sccache
 2023-03-17 11:29:23 +08:00
Ashish Kolheandhengfeiyang d51a6cf171 feat: remove org from user login screen (#298) 
* remove org from user login screen

* fix: delete no need code

* fix: format code

---------

Co-authored-by: hengfeiyang <hengfei.yang@gmail.com>
 2023-03-16 17:22:00 +05:30
Ashish KolheandHengfei Yang c453f38d58 feat: alert templates & destinations (#280) 
* v1

* alerts refactoring

* added unit tests

* swagger changes

* fixed openAPI specs

* Changes to alerts

* v1

* alerts refactoring

* added unit tests

* swagger changes

* fixed openAPI specs

* Changes to alerts

* Fixed docs

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-03-16 14:23:12 +05:30
Hengfei Yang caf547af15 feat: add more metrics (#281) 2023-03-15 20:22:34 +08:00
Hengfei Yang 11e70a0691 feat: add more metrics (#276) 
* feat: add more metrics
* fix: query case count(Athena)
 2023-03-14 20:35:54 +08:00
Hengfei Yang efea62fff2 fix: query error with != (#267) 
* fix: query error with !=

* fix: add more unit tests
 2023-03-13 00:26:55 +08:00
Hengfei Yang 0dbe81dab8 fix: add more unit tests (#241) 
* fix: add more unit tests

* fix: slack icon
 2023-03-10 10:46:26 +08:00
Ashish Kolhe 373d33f831 Alert Manager & integration test fixes (#239) 2023-03-09 22:58:31 +08:00
Hengfei Yang 7b2ad6a2db feat: support encode SQL (#223) 
* feat: support encode SQL

* fix: add more unit tests

* perf: improve inline
 2023-03-09 11:49:23 +08:00
Hengfei Yang 3cca9ad12b feat: stream delete (#193) 
* feat: update datafusion to 19

* feat: add stream deleting

* feat: data delete

* feat: stream delete done for local mode

* feat: data delete work on s3

* feat: data delete works with gcs

* feat: improve data delete

* feat: improve data delete

* feat: data delete done with merge file list

* fix: comment some unused code

* test: change coverage

* docs: update readme
 2023-03-06 18:44:42 +08:00
Ashish KolheandHengfei Yang 4108197943 fix: added UT (#158) 
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-02-27 10:51:04 +08:00
Ashish KolheandHengfei Yang d82ac3b74e added more unit tests (#145) 
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-02-22 22:10:30 +08:00
Hengfei Yang 66457a26c4 fix: ingest for vector (#144) 
* fix: ingest for vector

* fix: user email for example
 2023-02-22 21:52:32 +08:00
Ashish Kolhe cbf2493f02 fix: add unit tests (#137) 
* added UT

* added UT's

* added UT

* added UT's

* added line break
 2023-02-22 11:17:22 +05:30
Hengfei Yang 46cb48b3ea fix: s3 support aliyun oss (#135) 2023-02-21 19:51:02 +08:00
Ashish Kolhe 4c2982aa64 test: fixed integration tests & added UTs (#126) 
* fixed integration tests & added UTs

* deleted ds_store
 2023-02-20 19:52:53 +05:30
Prabhat Sharma ed4b26586f refactor: add logo, replace zinc-observe with zincobserve (#78) 
* add logo, replace zinc-observe with zincobserve

* refactor: add logo, replace zinc_observe

* renamed missing ones
 2023-02-15 07:51:37 +05:30
Hengfei Yang 7041d78b1c fix: compression for UI assets (#59) 2023-02-13 21:09:00 +08:00
Hengfei Yang 4f9b682d33 style: add License Header (#48) 2023-02-13 11:15:23 +08:00
Ashish Kolhe b972894035 feat: Functions as feature (#35) 2023-02-09 21:56:12 +08:00