starred/openobserve
1
0
Fork 0
mirror of https://github.com/openobserve/openobserve.git synced 2026-11-05 00:09:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,228 Commits 630 Branches 244 Tags
27c8d2420fbf074eec322f1075e90ee50323793e
Commit Graph
212 Commits
Author SHA1 Message Date
Hengfei Yang 27c8d2420f chore: update datafusion to 34 (#2222) 2023-12-18 10:49:08 +08:00
baerwangandHengfei Yang 7b32681617 feat: cli import or export data (#2071) 
feat #2056

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-12-16 12:45:09 +08:00
Hengfei Yang b210fb0146 fix: add keyword search for values API (#2191) 2023-12-14 17:03:38 +08:00
Hengfei Yang 8d9d38f60d fix: add empty name check (#2184) 
fixed #2171
 2023-12-14 11:23:58 +08:00
Hengfei Yang 48c555a076 fix: #2147 (#2163) 2023-12-12 15:35:13 +08:00
Ashish Kolhe 6d9afc71de feat: ldap integration - enterprise (#2145) 2023-12-08 14:34:05 +05:30
Hengfei Yang 60a0511d2b feat: add aggregation support for alert (#2148) 
impl #2142 backend
 2023-12-08 12:58:24 +08:00
Hengfei Yang 1b0b8c2bf5 chore: update format (#2144) 
fixed #2102
 2023-12-07 10:10:45 +05:30
Hengfei Yang d3b6fd0084 fix: optimise alert message (#2139) 2023-12-06 12:29:39 +08:00
Ashish Kolheandhengfeiyang f58bd81249 feat:use arrow ipc streams for data in wal (#2114) 
- [x] Use arrow IPC stream to write WAL data
- [x]  Enable WAL search on arrow IPC stream data
- [x]  Convert arrow data to parquet & move files to storage
- [ ]  Stats calculation ??
- [ ]  Optimisation

---------

Co-authored-by: hengfeiyang <hengfei.yang@gmail.com>
 2023-12-02 16:23:18 +05:30
Hengfei Yang 63d2f1522c feat: add more variables for alerts (#1973) 2023-12-01 23:27:43 +08:00
Hengfei Yang 9dfd50e1c3 fix: savedview openapi (#2098) 2023-11-30 17:04:31 +08:00
Ankur Srivastava 429ebcee36 fix: rum browser sdk sends keys with updated o2 prefix (#2082) 
We have updated the prefix of the keys from `oo` -> `o2` in rum-browser
sdk. This change is to support both the versions of keys in the payload
 2023-11-29 07:47:26 +01:00
Prabhat Sharma 5db4ca191a change license to AGPL 2023-11-26 18:34:59 -08:00
Hengfei Yang c305aaf90a feat: add table format for search response (#2012) 2023-11-23 23:11:26 +08:00
Ankur Srivastava c9008588f3 feat: implement ability to save search views (#1756) 
Created the standard GET,PUT,DELETE verbs for retrieving and
saving views which can be reused later.

Further details in:
Fixes https://github.com/openobserve/openobserve/issues/970
 2023-11-23 12:31:33 +01:00
Ashish KolheandBhargav 27a5252b46 feat:append data to enrichment tables (#2011) 
Added append toggle to allow appending data:

![image](https://github.com/openobserve/openobserve/assets/35160958/2778a19c-c1da-41bb-aaa1-a1fbdc46bad4)

---------

Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
 2023-11-23 16:18:20 +05:30
Hengfei Yang 3ce58a4d66 fix: add _timestamp for the traces latest data query (#1999) 2023-11-22 21:09:23 +08:00
Hengfei Yang 8097f47d8a fix: change latest traces data api to get (#1990) 2023-11-22 17:14:58 +08:00
Hengfei Yang de026b1a31 fix: don't need to send broadcast with mysql (#1985) 
fixed #1984
 2023-11-21 20:47:39 +08:00
Hengfei Yang f4a0a75921 feat: implement bloom filter (#1625) 2023-11-17 12:21:33 +08:00
Hengfei Yang ca52eca9d8 feat: delete API should return 200, 404 or 500 (#1924) 2023-11-10 21:38:34 +08:00
Hengfei Yang 395374834b fix: data type conflict for traces (#1855) 2023-11-03 08:34:33 -05:00
Ankur Srivastava 237edccad4 feat: Overcome the cors issue using a proxy (#1820) 
Can be queried using http://localhost:5080/proxy/{target_url}
 2023-11-03 00:47:24 +01:00
Hengfei Yang a77adc8848 fix: Kinesis firehose API (#1838) 2023-11-02 04:59:22 -05:00
Ankur Srivastava ba250f7287 fix: Swagger was missing a rum-model (#1810) 
Added the `RumIngestionResponse` and `RumIngestionToken` models to the
swagger-ui
 2023-10-30 17:14:50 +01:00
Hengfei Yang 150316e7d6 feat: implement distinct_values (#1773) 2023-10-27 04:20:50 -05:00
Ankur Srivastava 3da01ec408 fix: refactor the setting of default scrape-interval (#1731) 2023-10-24 08:20:31 -05:00
Hengfei Yang 0a35aad529 feat: add new configure for control query queue (#1738) 2023-10-24 02:06:55 -05:00
Hengfei Yang 8e21f62487 feat: improve files deletion (#1711) 2023-10-23 19:32:10 -05:00
Ankur SrivastavaandHengfei Yang b7a52b0d72 fix: use a const to represent the org-setting-key (#1718) 
A small fix to ensure that the prefix for organization setting is
consistent everywhere, we use it.

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
 2023-10-23 13:18:22 +02:00
Ankur Srivastava cba8e9bb78 feat: refactor actix-http responses (#1713) 
Introduced two new methods in meta::http::HttpResponse struct
- json()
- bad_request()

Main motivation behind this is to unify the actix-response as we had no
consistent way for sending out them, especially `bad_request()`
 2023-10-23 11:28:28 +02:00
Ankur Srivastava 13de85c410 feat: integrate organization level settings (#1608) 
Motivation behind this implementation is to be able to store some
dynamic organization level settings which can be used to build further
features like `__rate_interval` etc.

In this PR, we are saving the scrape_interval for the given
organization, if unset, this is going to be 15s by default.
 2023-10-19 14:02:13 +02:00
Ashish Kolhe e842a2e6e1 fix: improve data ingestion (#1641) 
- [x] Move files - changed to spawn blocking
- [x] Ingestion - remove clones , pass by reference
- [x] Invoke ingester service using url
 2023-10-19 17:06:23 +05:30
Hengfei Yang 098f7df841 feat: support delete fields (#1674) 2023-10-18 09:55:55 -05:00
Ankur Srivastava 5b01947491 feat: rum/maxmind integration (#1520) 
This PR implements the first draft for the Real User Monitoring.

The endpoints at the moment are:

- `/rum/v1/{org_id}/rum`
- `/rum/v1/{org_id}/replay`
- `/rum/v1/{org_id}/logs`

For now the authentication is done using `oo-api-key` which is
being passed in the query-param using `browser-sdk`.

As of now, the data is being collected in the following three streams:

- `_rumdata` for rum data
- `_rumlog` for log data
- `_sessionreplay` for session-replay data

The data is being ingested using the multi-json implementation, with a
small change where some extra data is also ingested in the json from
the headers, query-params etc. using a middleware extractor and stores
the data in `HashMap<String, String>` and gets ingested in this
multi-json.

A file containing regexes for common user-agents is also committed which
is read during the initialization time which is used to parse incoming
user-agents and browser, os etc are inserted into the incoming row of
data.

Added the endpoints for `get`, `put`, `post` for rum-tokens.

- `/api/{org_id}/organizations/rumtoken`

#### maxmind integration 
Integrated the maxmind-db data in the source. Currently the data
is injected via the middleware and the file gets uploaded every
24 hours.
The file pointer gets mutated and updated when there is a change
in the mmdb file. To efficiently do this, we also query the sha256
of our file and compare it using the .sha256 file available on the
public bucket.
 2023-10-18 15:09:24 +02:00
Ankur Srivastava 76d7678acd fix: fixes the create-folder docs in #1649 (#1654) 2023-10-15 21:20:35 -05:00
Ashish Kolhe 8cd8e0b550 fix: logs ingestion refactor (#1527) 2023-10-02 11:20:30 +05:30
Hengfei Yang 031d5dc856 fix: optimize file list broadcast (#1589) 2023-09-30 22:25:35 -05:00
Ashish Kolhe 84fc09ef0b fix: default folder rename issue (#1585) 
We create default folder , when we first receive call to create a
dashboard in default folder. If an API call tries to create a default
folder , it should not be allowed
 2023-09-29 12:32:31 +05:30
Ashish Kolhe 9063a8c7b3 fix: Chunking of Data Prevents Uploads of Large CSV Files to Enrichment Tables (#1560) 2023-09-25 11:31:32 +05:30
Ashish Kolhe 47bea0abb9 feat: folders for dashboards (#1559) 
Adds support for folder for dashboards, which helps user organise
dashboards in folders.

We should support folders for dashboards like grafana
:https://grafana.com/docs/grafana-cloud/infrastructure-as-code/grizzly/dashboards-folders-datasources/#add-a-folder

Provides API's for :
https://github.com/openobserve/openobserve/issues/1053
 2023-09-22 15:54:10 +05:30
Ashish Kolhe af2d805be2 fix: OTLP ingestion fixes (#1547) 
Fixes https://github.com/openobserve/openobserve/issues/1544

- [x] Accept ingestion token for v1/logs v1/metrics 
- [x] Introduce optional header Stream_Name to accept stream name
- [x] change zinc_org_id to Organisation
 2023-09-20 13:05:26 +05:30
Hengfei Yang c6a46ac315 feat: upgrade datafusion to 31 (#1506) 2023-09-19 17:43:17 +08:00
Hengfei Yang e885950b70 feat: add more debug logs (#1515) 2023-09-15 12:30:57 +08:00
Ashish Kolhe e2c0c8c91b fix: vrl , values api filtering (#1490) 
- [x] Add . if not already added to vrl functions
- [x] Server side fix for :
https://github.com/openobserve/openobserve/issues/699
- [x] Add error logs
- [x] Fixes : https://github.com/openobserve/openobserve/issues/1381
 2023-09-12 14:44:33 +05:30
Hengfei Yang 9d1f82d0fd fix: cancel datafusion execution when it timeout (#1481) 2023-09-11 18:46:59 +08:00
Ashish Kolhe 56655cdbef feat: otlp http for logs & metrics (#1482) 2023-09-11 15:25:31 +05:30
Hengfei Yang 019c4ccc53 fix: swagger under sub directory (#1476) 2023-09-09 12:36:23 +08:00
Hengfei Yang c690bca900 fix: top values api error when the filter is empty (#1475) 2023-09-09 11:01:31 +08:00