1. This PR adds
**previous approach**
1. we were showing both All Fields and UDS tabs even user doesnot have
UDS enabled
2. UDS is shown as second tab
**current approach**
1. only show UDS if enabled otherwise show only All fields
3. if UDS enabled show it as first tab and instead of All Fields(when
UDS enabled) renamed it to Other Fields and also added tooltip with some
explanatory text after Other Fields with a info icon as
**_
> Other fields show only the schema fields that existed before the
stream was configured to use a user-defined schema.
_**
1. This PR handles the issue of alerts of being overridden when user
tries to shift the folders rapidly
example:
folder1 -> 100alerts
folder2 -> 2alerts
folder3 -> 0 alerts
so first users goes to folder2 and fetches the alerts meanwhile user
goes to folder1 and immediately switches to folder3 , lets assume the
folder3 alerts are already fetched and they are in store then it will
show folder1 alerts because api takes time to resolve, overrides the
current alerts after resolving so tries to show the wrong folder alerts
this PR fixes
1. histogram loader should while both search and histogram request as
well as no data found should only show when there is no actual data not
at the time of loading
2. in the detail table when we add a field to the table it again shows
add a field to the table instead should show remove from the table
3. when user try to add fields using quick mode with sql mode off there
is one console error
1. this PR fixes the issue that is related to when user tries to run
distinct query when histogram is on
eg:
`SELECT DISTINCT body FROM "default"`
currently it will show total events as if 50 records per page it will
show
1 to 50 out of 50 events
- [x] ws v2 to use tokio `RwHashMap` instead of `Dashmap` to avoid dead
locks
- [x] `trace_id` when retry to be different after the first retry to get
a different querier from consistent hash
- [x] `user_id` for enterprise search event when run as single node
---------
Co-authored-by: loaki07 <rustyloaki@gmail.com>
- [x] `Router-Client` if sending `pong` from `router` fails force close
the ws connection
- [ ] `Router-Querier` conn if `pong` is not received from querier for
certain time (ping_interval + 10 secs) then force close the connection
- [x] ws behavior should be such that if one querier disconnects from
the connection pool the entire `Router-Client` connection should not be
disconnected, searches/events should continue with other queriers in the
pool. Also, if one of the querier disconnects send `should_client_retry`
as `true`
---------
Co-authored-by: omkark06 <omkar@zinclabs.io>
- [x] add `jitter` to `ping_interval` for heartbeats
- [x] deprecate `is_connected` in `Connection` trait
- [x] remove `maintain_connection`, instead the `health_check` tokio
tasks for each querier will do the similar functionality by manual heart
beats
- [x] add `clean_up` for `QuerrierConnectionPool` & `QuerierConnection`
with graceful shutdown
- [x] use `ZO_INSTANCE_NAME` as the `req_id` for the connection between
the `Rotuer-Querier`
- [x] remove ws v1
When introducing service accounts rbac, we didn't do a rbac openfga
migration to save openfga tuples (`_all_{org_id}`) for existing
organizations. Hence as a result, viewer and editor are not able to
view/edit service accounts by default. Currently, they can do it only
after given service accounts explicitly through the roles edit UI page.
This pr fixes this by adding a migration which saves `_all_{org_id}`
tuples for every organizations.
## What is changed
- [x] Try to improve task starving issue
- [x] Update mmdb frequency
- [x] Improve slow log detection
## MMDB
- Removed ENV `ZO_MMDB_UPDATE_DURATION` because is in seconds
- Add new ENV `ZO_MMDB_UPDATE_DURATION_DAYS` is in days
## Tokio task
We try to use `task::coop` to improve task starving issue in Tokio.
<img width="1496" alt="image"
src="https://github.com/user-attachments/assets/1666488a-146b-49d7-b460-c629528e06f9"
/>
The second chart is for this PR, it is a big slow but more stable.
- In case of errors related to pipeline not found/not enabled/not
derived stream, we no longer delete the scheduled_job when handling the
derived stream trigger in the alert manager. We push the errors to the
`triggers` stream and try to run it again after 5 mins.
- Add field `key` in the `db_query_nums` metric that includes the module
key that was deleted in the scheduled_jobs table.
This PR has some improves:
- Improve tantivy generate, from 7.7s -> 5.9s
- Improve STREAM_SETTINGS cache with atomic
- Add process metrics
- Remove zo_node_open_fd, node_tcp_conn_resets( use
`node_tcp_connections{state="resets"}`)
- Add debug logs for bulk api
- Add debug logs for tantivy panic
Sai Nikhil
Shyam Panjiyar
Zhao.wei
Hengfei Yang
Loakesh Indiran
dependabot[bot]
Ashish Kolheandloaki07
Taiming Liu
Subhra264
Prabhat Sharma
Abhay Padamani
ktx-vaidehi