A small fix to ensure that the prefix for organization setting is
consistent everywhere, we use it.
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Introduced two new methods in meta::http::HttpResponse struct
- json()
- bad_request()
Main motivation behind this is to unify the actix-response as we had no
consistent way for sending out them, especially `bad_request()`
Motivation behind this implementation is to be able to store some
dynamic organization level settings which can be used to build further
features like `__rate_interval` etc.
In this PR, we are saving the scrape_interval for the given
organization, if unset, this is going to be 15s by default.
This PR implements the first draft for the Real User Monitoring.
The endpoints at the moment are:
- `/rum/v1/{org_id}/rum`
- `/rum/v1/{org_id}/replay`
- `/rum/v1/{org_id}/logs`
For now the authentication is done using `oo-api-key` which is
being passed in the query-param using `browser-sdk`.
As of now, the data is being collected in the following three streams:
- `_rumdata` for rum data
- `_rumlog` for log data
- `_sessionreplay` for session-replay data
The data is being ingested using the multi-json implementation, with a
small change where some extra data is also ingested in the json from
the headers, query-params etc. using a middleware extractor and stores
the data in `HashMap<String, String>` and gets ingested in this
multi-json.
A file containing regexes for common user-agents is also committed which
is read during the initialization time which is used to parse incoming
user-agents and browser, os etc are inserted into the incoming row of
data.
Added the endpoints for `get`, `put`, `post` for rum-tokens.
- `/api/{org_id}/organizations/rumtoken`
#### maxmind integration
Integrated the maxmind-db data in the source. Currently the data
is injected via the middleware and the file gets uploaded every
24 hours.
The file pointer gets mutated and updated when there is a change
in the mmdb file. To efficiently do this, we also query the sha256
of our file and compare it using the .sha256 file available on the
public bucket.
## Implement
- [x] Implement file_list API for gRPC
- [x] Debuging deadlock issue
## Todo
- [ ] Querier should use the file_list API for search
---------
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
We create default folder , when we first receive call to create a
dashboard in default folder. If an API call tries to create a default
folder , it should not be allowed
Before the fix supported length in bytes by both TCP & UDP was limited
to 1024 bytes, which resulted in values being cut off.
We had two options as fix :
- Support max size in line with standard [IEEE
802.3](https://en.wikipedia.org/wiki/IEEE_802.3) , which limits 1500
bytes of payload , with this we can support 1472 bytes (1500 - 20 (IP
header ) -8 (UDP header) ) for UDP & 1460 bytes(1500 - 20 (IP header )
-20 (TCP header) ) for TCP
- JUMBO frames - Ruled out because : for jumbo frames , frame size used
by an end-to-end connection is typically limited by the lowest frame
size in intermediate links ,not only sender & receiver has to enable
jumbo frames but all intermediate links will have to support it.
Fixes#1514
**Summary : we will support max 1472 bytes for UDP & 1460 bytes for
TCP**
Hengfei Yang
Ankur Srivastava
baerwang
Prabhat Sharma
Ashish KolheandBhargav
