Currently, individual level RBAC permission does not work for
dashboards. This pr fixes that. So a role permission like below
<img width="1399" alt="Screenshot 2025-03-04 at 5 27 11 PM"
src="https://github.com/user-attachments/assets/06c2030d-4227-4109-b73f-dc2337151362"
/>
will allow the user to access only the dashboards with `Get` and `All`
permission even if the `Get` permission for the folder is not specified.
1. reduced query time for sql validation from 15mins to 5 seconds
2. silence column is removed from the pipeline list as we are not
supporting in the UI
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced search job scheduling functionality.
- Added ability to create, list, cancel, retry, and delete scheduled
searches.
- Enhanced user interface for managing scheduled search jobs, including
a new component for displaying scheduled jobs.
- Implemented conditional rendering for search history and search
scheduler components.
- Added new methods for handling scheduled searches within the service
layer.
- **Improvements**
- Refined search bar interface with new scheduling options.
- Updated pagination handling for job-related data.
- Enhanced overall functionality related to job management and
pagination.
- Improved loading indicators and user experience during data retrieval.
- **Bug Fixes**
- Corrected conditional rendering of search history components.
- Optimized job data retrieval and pagination logic.
- Simplified sorting logic within the search history component.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
1. no data error before rendering
2. chart rerenders even after loading
3. all charts rererenders if we click on refresh of one chart
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
1. this PR introduces new design for query node
2. dynamic stream name for destination node instead of create new stream
it allows user to input any non existing stream name as destination and
at the time of ingestion it will be created dynamically
3. Fixes issues of #5886
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
1. Fixed cancel query flow
2. Reset request trace_id's and cancel query operation before initiating
new search operation
3. Run query disabling histogram, later enable histogram. This was
throwing error
New Env:
```sh
# default 0ms
# preferred to be set at 10ms to 100ms
# milliseconds
ZO_WEBSOCKET_CLOSE_FRAME_DELAY=10
```
Changes:
- [x] add support to dashboard for streaming aggregate queries from in
memory cache
- [x] new sort logic for web socket search results
> - [x] introduced a new `optional` parameter `fallback_order_by_col`
for ws search req `SearchEventReq`
> - For queries where there is no `_timestamp` field and no `order_by`
specified:
```sql
SELECT level, COUNT(*) as count
FROM logs
GROUP BY level
```
> - The default sql behavior is to return the results in a non-ordered
manner.
> - However, with web sockets this behavior impacts the UX in both
`logs` and `dashboards` when `streaming aggregates` is `enabled` as the
results in page/dashboards will keep fluctuating.
> - To fix this there a new sort behavior introduced in web sockets
```rust
/// Represents different sorting strategies
enum SortStrategy {
SqlOrderBy,
FallbackColumn(String, OrderBy),
AutoDetermine(String, bool), // (column, is_string)
NoSort,
}
```
> 1. If `order_by` is already specified in the sql query and the search
service has populated the `order_by` field in its response for the
search, then there is no sort action required on the search results.
> 2. If `fallback_order_by_col` is specified that col is selected for
sort. This will mainly be used in `Dashboards`.
> 3. If none of the above is specified, there is `auto-determine`
strategy which would check for the `first non-numeric col` else the
`first numeric col`
> 4. `non-numeric` col i.e. `text/string` is always sorted in `asc`.
While, `numeric` col is sorted in `desc`. This applies to both
`FallbackColumn` & `AutoDetermine` strategies.
- [x] redesign web socket search event handling
> - Previously the `cleanup_and_close_session` used to happen inside the
search task thread. Which is suspected :suspect: to be one of the causes
for the `CloseFrame` description and the `DataFrame` data to get merged
and malform the message in the socket.
> - New behavior `cleanup_and_close_session` will happen outside of the
search task spawned once search is complete.
> - Since its suspected :suspect: that there is some race condition 🏃🏾
when the web socket frames are being sent the env variable
`ZO_WEBSOCKET_CLOSE_FRAME_DELAY` default `0` is created to introduce a
delay preferred `10 ms` to ` 100ms` before sending the `CloseFrame`
> - `SEARCH_REGISTRY` now will hold the different states of search which
are `Running`, `Cancelled` & `Completed`
> - The search task is modified to now listen for a `cancel` from the
`mpsc` channel and does `cleanup_search_resources`
Related Issues:
- #5967
- #5974
---------
Co-authored-by: loaki07 <rustyloaki@gmail.com>
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Abhay Padamani
ktx-vaidehiandHengfei Yang
Subhra264
Sai Nikhil
Bhargav
Omkar Kesarkhane
dependabot[bot]
