1. Fixed cancel query flow
2. Reset request trace_id's and cancel query operation before initiating
new search operation
3. Run query disabling histogram, later enable histogram. This was
throwing error
New Env:
```sh
# default 0ms
# preferred to be set at 10ms to 100ms
# milliseconds
ZO_WEBSOCKET_CLOSE_FRAME_DELAY=10
```
Changes:
- [x] add support to dashboard for streaming aggregate queries from in
memory cache
- [x] new sort logic for web socket search results
> - [x] introduced a new `optional` parameter `fallback_order_by_col`
for ws search req `SearchEventReq`
> - For queries where there is no `_timestamp` field and no `order_by`
specified:
```sql
SELECT level, COUNT(*) as count
FROM logs
GROUP BY level
```
> - The default sql behavior is to return the results in a non-ordered
manner.
> - However, with web sockets this behavior impacts the UX in both
`logs` and `dashboards` when `streaming aggregates` is `enabled` as the
results in page/dashboards will keep fluctuating.
> - To fix this there a new sort behavior introduced in web sockets
```rust
/// Represents different sorting strategies
enum SortStrategy {
SqlOrderBy,
FallbackColumn(String, OrderBy),
AutoDetermine(String, bool), // (column, is_string)
NoSort,
}
```
> 1. If `order_by` is already specified in the sql query and the search
service has populated the `order_by` field in its response for the
search, then there is no sort action required on the search results.
> 2. If `fallback_order_by_col` is specified that col is selected for
sort. This will mainly be used in `Dashboards`.
> 3. If none of the above is specified, there is `auto-determine`
strategy which would check for the `first non-numeric col` else the
`first numeric col`
> 4. `non-numeric` col i.e. `text/string` is always sorted in `asc`.
While, `numeric` col is sorted in `desc`. This applies to both
`FallbackColumn` & `AutoDetermine` strategies.
- [x] redesign web socket search event handling
> - Previously the `cleanup_and_close_session` used to happen inside the
search task thread. Which is suspected :suspect: to be one of the causes
for the `CloseFrame` description and the `DataFrame` data to get merged
and malform the message in the socket.
> - New behavior `cleanup_and_close_session` will happen outside of the
search task spawned once search is complete.
> - Since its suspected :suspect: that there is some race condition 🏃🏾
when the web socket frames are being sent the env variable
`ZO_WEBSOCKET_CLOSE_FRAME_DELAY` default `0` is created to introduce a
delay preferred `10 ms` to ` 100ms` before sending the `CloseFrame`
> - `SEARCH_REGISTRY` now will hold the different states of search which
are `Running`, `Cancelled` & `Completed`
> - The search task is modified to now listen for a `cancel` from the
`mpsc` channel and does `cleanup_search_resources`
Related Issues:
- #5967
- #5974
---------
Co-authored-by: loaki07 <rustyloaki@gmail.com>
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
- #5384
- #5292
- #4303
- #5795
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced drilldown functionality in dashboard panels.
- Improved series data handling in chart generation.
- **Refactor**
- Updated series name processing in dashboard components.
- Simplified series object creation logic.
- Streamlined chart type handling for bar and scatter charts.
- **Bug Fixes**
- Improved accuracy of series name retrieval during drilldown
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-kirtan <124270523+ktx-kirtan@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
- #5569
**Summary:** Removes validation constraints for complex queries and
ensures execution on apply, while extracting fields for panel
construction.
**Key Changes:**
- Removed validation to support complex queries.
- Guaranteed query execution on "Apply" click regardless of validation
status
- Fields will be extracted, which can be used for further panel
building.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced panel validation logic with more granular control over field
validation.
- Introduced new utility functions for validating panel data and SQL
panel fields.
- **Bug Fixes**
- Improved error handling for dashboard panel creation and query
execution.
- Refined validation checks for different chart types and data
configurations.
- **Refactor**
- Streamlined validation processes across multiple dashboard and query
components.
- Updated validation methods to support more flexible error checking.
These changes improve the robustness of panel and query validation in
the dashboard creation workflow.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- #5844
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
Based on the comprehensive summary of changes, here are the updated
release notes:
- **New Features**
- Added timed annotations functionality for dashboards.
- Introduced ability to create, edit, and delete annotations on panels.
- Implemented annotation management across frontend and backend systems.
- **Improvements**
- Enhanced chart rendering to support annotation visualization.
- Added new API endpoints for annotation operations.
- Improved dashboard interaction with annotation mode.
- **Backend Changes**
- Created new database migrations for timed annotations and timed
annotation panels.
- Added database entities and relationships for annotations.
- Implemented service-level logic for annotation management.
- **Frontend Changes**
- Added new components for annotation management, including
`AddAnnotation`.
- Integrated annotation services and composables.
- Updated chart rendering to display annotations and improved panel
interactivity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech>
Co-authored-by: Shyam Panjiyar <shyam@openobserve.ai>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
Co-authored-by: Neha P <144308844+neha00290@users.noreply.github.com>
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ktx-vaidehi
Omkar Kesarkhane
Bhargav
Abhay Padamani
Sai Nikhil
dependabot[bot]
Taiming Liu
