1. validate and close text instead of save button
2. when user selects stream name query wipes out if any previous query
is there
3.add the click event on whole expand bar
4. after opening the sidebar clicking on the SQL editor moves the focus
away from the SQL editor
5. remove the pipeline view on the row and introduce a eye icon in
actions section
6. when user clicks on edit the schema should auto populate respective
to the query
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced search job scheduling functionality.
- Added ability to create, list, cancel, retry, and delete scheduled
searches.
- Enhanced user interface for managing scheduled search jobs, including
a new component for displaying scheduled jobs.
- Implemented conditional rendering for search history and search
scheduler components.
- Added new methods for handling scheduled searches within the service
layer.
- **Improvements**
- Refined search bar interface with new scheduling options.
- Updated pagination handling for job-related data.
- Enhanced overall functionality related to job management and
pagination.
- Improved loading indicators and user experience during data retrieval.
- **Bug Fixes**
- Corrected conditional rendering of search history components.
- Optimized job data retrieval and pagination logic.
- Simplified sorting logic within the search history component.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
1. this PR introduces new design for query node
2. dynamic stream name for destination node instead of create new stream
it allows user to input any non existing stream name as destination and
at the time of ingestion it will be created dynamically
3. Fixes issues of #5886
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
1. Fixed cancel query flow
2. Reset request trace_id's and cancel query operation before initiating
new search operation
3. Run query disabling histogram, later enable histogram. This was
throwing error
- #5569
**Summary:** Removes validation constraints for complex queries and
ensures execution on apply, while extracting fields for panel
construction.
**Key Changes:**
- Removed validation to support complex queries.
- Guaranteed query execution on "Apply" click regardless of validation
status
- Fields will be extracted, which can be used for further panel
building.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced panel validation logic with more granular control over field
validation.
- Introduced new utility functions for validating panel data and SQL
panel fields.
- **Bug Fixes**
- Improved error handling for dashboard panel creation and query
execution.
- Refined validation checks for different chart types and data
configurations.
- **Refactor**
- Streamlined validation processes across multiple dashboard and query
components.
- Updated validation methods to support more flexible error checking.
These changes improve the robustness of panel and query validation in
the dashboard creation workflow.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR fixes the omitting the first row in firefox browser
#5279
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved browser compatibility for Firefox in the virtualized table
rendering.
- Adjusted table row positioning and measurement to ensure consistent
display across different browsers.
- Enhanced vertical translation of rows based on browser type.
- Minor formatting adjustments for consistent spacing in the rendering
functions.
- **New Features**
- Added a computed property to detect Firefox browser support.
- Introduced a variable to adjust row positioning based on browser type.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added search capability for saved functions.
- Introduced search input in function dropdown to filter options.
- **Documentation**
- Updated English language localization with new search-related text.
- **Style**
- Minor styling adjustments to dropdown padding.
- **Bug Fixes**
- Improved date selection logic in the logs page for better accuracy.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added advanced search functionality for traces, including a search
input and navigation buttons.
- Implemented automatic scrolling to specific spans based on route
parameters.
- **Improvements**
- Added dynamic table cell width calculation for better layout.
- Improved visual feedback for search results with highlighting.
- Enhanced navigation through search results with "Previous" and "Next"
buttons.
- Updated duration field display to "duration (µs)" for clarity.
- Introduced a resize feature for the header component.
- **User Experience**
- Search input for trace details.
- Smooth scrolling to relevant spans.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Updated English language localization strings
- Changed "Saved Views" label to "Save View" in user interface
- **Style**
- Modified button label in save view dialog from "OK" to "Save"
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#5463
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced auto-refresh interval configuration across multiple
components.
- Added minimum refresh interval support with configurable restrictions.
- Introduced new error handling and validation mechanisms for report
creation.
- Added new utility functions for validating frequency intervals.
- **Improvements**
- Introduced dynamic minimum refresh interval settings.
- Added user feedback mechanisms for refresh interval constraints.
- Improved error handling for alert creation and updates.
- Enhanced clarity and maintainability of code through formatting
adjustments.
- Updated frequency input logic to dynamically calculate minimum values.
- Added validation for refresh intervals during data refresh and URL
query restoration.
- **Configuration**
- Added new props for minimum refresh interval and related messages.
- Implemented dynamic minimum value calculations for frequency inputs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added ability to update saved views with a new confirmation dialog.
- Introduced edit button for saved views.
- Enhanced user notifications for the view update process.
- **Bug Fixes**
- Implemented stream selection validation before updating saved views.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR solves #5551
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced theme-based icon management in the search bar
- Dynamically updated icons based on application theme
- **Improvements**
- Improved maintainability of icon path handling
- Centralized icon selection logic for better code readability
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
# Objectives:
1. Stream results to the client using web-socket.
2. Cache results for a query as a single result cache.
## New Endpoint:
```sh
/api/{org_id}/ws/{req_id}
```
## WS Design:
1. The request_id is used to handle the sessions `pub static
WS_SESSIONS: Lazy<RwHashMap<String, WsSession>> =
Lazy::new(DashMap::default);`.
2. The request_id is removed from the session cache and the web-socket
is disconnected once single search is complete. (e.g. for 5 dashboards
in a single page, 5 individual requests are made). The design choice is
discussed [here](https://github.com/openobserve/walmart1/issues/207).
3. The supported events and responses the web-socket are described
below.
4. Auditing for web-socket linked
[here](https://github.com/openobserve/o2-enterprise/pull/248)
5. Org level setting to enable and disable web-socket communication.
(Default: disabled)
## New Env:
```sh
ZO_WEBSOCKET_ENABLED=true
```
## Env to be enabled:
```sh
ZO_USE_MULTIPLE_RESULT_CACHE=true
ZO_RESULT_CACHE_ENABLED=true
```
- If `ZO_RESULT_CACHE_ENABLED` is set to `false` which is default.
Results cache will not be written.
## Nginx config:
```yaml
nginx.ingress.kubernetes.io/proxy-http-version: "1.1" # Enable HTTP/1.1 for WebSockets
nginx.ingress.kubernetes.io/enable-websocket: "true"
nginx.ingress.kubernetes.io/proxy-set-headers: |
Upgrade $http_upgrade;
Connection "Upgrade";
```
## Note: Disable `keep-alive` settings if present
```yaml
# nginx.ingress.kubernetes.io/connection-proxy-header: keep-alive
```
## Client Behavior before this pr:
1. For a single query, client calls the `_search_partition` api which
returns the list of partitions `Vec<[start_time, end_time]>` -> then
makes a `_search` api call for each of the partition.
2. The client also handles the logic for `max_query_range` for the
stream.
3. Handle the logic for dashboards loading from right to left
irrespective of `order_by` but honors the result by `order_by`
## New Behavior:
1. Client makes a web socket request with a request_id (uuid). This
req_id is stored in-memory cache along with its session object.
2. Server Search
- STEP 1: Search result cache
- STEP 1(a): Handle the cache responses & query the deltas
- STEP 2: Search without cache if no caches found
- STEP 3: Write to result cache
6. Server will now handle querying the partitions and then calling
search for each.
7. The results are streamed as in when the responses are received.
8. Server will handle the logic for dashboards loading from right to
left irrespective of `order_by` but honors the result by `order_by`
## Modified Code:
1. The `search` service will now use `write_results_v2` when writing to
cache.
2. The main difference between `write_results` and `write_results_v2` is
the strategy for deduplication. In `write_results_v2` we remove all the
hits within the same Date, Hour, Minute and Second; before writing to
cache.
```rust
/// # Caching Strategy
/// 1. **Select Hit for Deduplication**:
/// - Selects either the first or last record based on the `is_descending` flag.
/// - `is_descending = true`: Selects the last record.
/// - `is_descending = false`: Selects the first record.
///
/// 2. **Remove All Hits Within the Same Date, Hour, Minute, and Second for Deduplication**:
/// - Identifies the timestamp of the hit to remove (based on the `is_descending` flag, either
/// the first or the last record is chosen).
/// - Removes all hits that match the exact same date, hour, minute, and second
/// (`YYYY-MM-DDTHH:MM:SS`) as the identified hit.
/// - Example: If the timestamp to remove is `2024-12-06T04:15:30`, only hits with the exact
/// timestamp `2024-12-06T04:15:30` are removed, while others with different seconds (e.g.,
/// `2024-12-06T04:15:29` or `2024-12-06T04:15:31`) are retained.
```
## Web-socket Events:
1. Search
```sh
{
"type": "search",
"content": {
"trace_id": "d56755eb02cf47ecb3064ee096735c4a",
"payload": {
"query": {
"sql": "select histogram(_timestamp, '1 hour') AS zo_sql_key, count(*) AS zo_sql_num from \"olympics\" GROUP BY zo_sql_key ORDER BY zo_sql_key",
"start_time": 1733499294217000,
"end_time": 1734708894217000,
"size": -1,
"query_fn": "LmFiYywgZXJyID0gZ2V0X2VucmljaG1lbnRfdGFibGVfcmVjb3JkKCJvbHltcGljcyIsCnsKICAicmFuayI6IHRvX3N0cmluZyEoLnJhbmspCn0pCi5yYW5rX2tleXdvcmQgPSAuYWJjLmtleXdvcmQKLg..",
"sql_mode": "full"
}
},
"stream_type": "enrichment_tables",
"search_type": "ui",
"use_cache": true
}
}
```
2. Cancel
```sh
{
"type": "cancel",
"content": {
"trace_id": "b221bb019f3742e8bd93f817d530ef70"
}
}
```
Related Issues: #5173, #5188, #5266, #5320
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced WebSocket search functionality with a toggle in settings.
- Added support for real-time data retrieval via WebSocket in various
components.
- Enhanced search operations with new methods for handling WebSocket
queries.
- Added new properties and methods to improve handling of WebSocket
sessions and requests.
- New interface for search request payloads and improvements in query
handling.
- Implemented WebSocket proxy handling for improved connection
management.
- Enhanced caching functionality with new methods for managing cached
responses.
- Introduced optional field for enabling WebSocket search in
organization settings.
- Added a new constant for handling partial error responses in search
operations.
- Added a new configuration option for enabling WebSocket support.
- Updated package version and various dependency versions to enhance
stability.
- **Bug Fixes**
- Improved error handling and logging for search requests and caching
mechanisms.
- Enhanced validation and URL construction for alerts.
- **Documentation**
- Updated comments and documentation for clarity on new features and
changes.
- **Style**
- Minor formatting adjustments for consistency across components and
services.
- **Chores**
- Updated dependency versions to enhance stability and performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
- #5482
- for metrics page, default chart type as `line` chart.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Default chart type for the metrics dashboard is now set to "line".
- Enhanced font size calculations for chart labels and tooltips.
- **Bug Fixes**
- Tooltip configuration updated to reflect the first value of the metric
for improved accuracy.
- **Refactor**
- Consolidation of font size calculation functions for better
performance and functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced trace stream handling with a new "View Trace" button that
appears based on available data.
- Added functionality to redirect to trace details from search results.
- **Bug Fixes**
- Improved loading state management for trace streams.
- **Documentation**
- Updated event emissions and method declarations for clarity and
maintainability.
- **Style**
- Cosmetic changes to improve code readability in various components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- #5193
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced PromQL mode handling in the dashboard panel.
- Enhanced metrics page layout with new components:
DateTimePickerDashboard, AutoRefreshInterval, ChartSelection,
CustomHTMLEditor, CustomMarkdownEditor, DashboardQueryBuilder,
DashboardErrorsComponent, PanelSchemaRenderer, PanelSidebar,
RelativeTime, and QueryInspector.
- Added localization support for metrics in the search context.
- **Bug Fixes**
- Improved error handling for loading stream fields and validating panel
configurations.
- Stream type selection visibility now adapts based on dashboard panel
data type.
- Enhanced user interaction by simulating keyboard input for
organization selection.
- Reduced validation requirements for the metrics page.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#4871#5127
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced a new "sankey" chart type in the chart selection.
- Added a configuration section for trellis layouts in dashboard panel
settings.
- Enhanced error handling and data validation for various panel types.
- New utility function `deepCopy` for creating deep copies of objects.
- Enhanced grid management capabilities for trellis layouts.
- Improved rendering logic for charts and dynamic height adjustments.
- Introduced new computed properties for dynamic chart panel height and
trellis configuration.
- New parameter `chartPanelStyle` added for enhanced customization in
data conversion.
- Improved handling of no data scenarios for metric and heatmap panel
types.
- Added new localization keys for layout and dashboard settings.
- **Bug Fixes**
- Improved error messages for dashboard retrieval and data validation
logic.
- **Documentation**
- Expanded localization options for layout and dashboard settings.
- **Chores**
- Minor formatting and consistency improvements across multiple files.
- Added trailing commas for improved code consistency in various files.
- Updated import paths to reflect codebase restructuring.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced log display functionality with improved query context
handling for multiple streams.
- Improved error handling for filter validity when selecting multiple
streams.
- Adjusted pagination logic for accurate total fields display.
- Added new buttons and toggles in the search bar for better control
over query results.
- Introduced confirmation dialog for switching between visualization and
log modes.
- **Bug Fixes**
- Refined logic for updating field values to ensure correct aggregation
and display.
- Enhanced error handling for download attempts in the search bar.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#5330#5343
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new method for handling stream exploration based on the
stream type.
- Added props for error messaging, timestamp highlighting, and default
column settings in the table component.
- **Improvements**
- Enhanced responsiveness and usability of the table component with
better column resizing and row management.
- Improved readability and maintainability of the `schema` component
through formatting adjustments.
- Updated error handling for data retention days in the schema
component.
- Added a new asynchronous method to calculate the time range for stream
exploration.
- **Bug Fixes**
- Refined error handling in stream deletion methods for consistency.
- **Documentation**
- Updated method signatures for clarity and improved functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced SQL autocomplete with fuzzy matching suggestions.
- Updated syntax guide to include fuzzy matching instructions for SQL
queries.
- **Bug Fixes**
- Maintained existing logic for other completion items, ensuring
seamless integration of new features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced error handling and logic improvements in SQL query parsing
for the `useLogs` composable.
- Improved functionality of the search bar component, including better
handling of SQL queries and user notifications.
- **Bug Fixes**
- Refined logic for managing selected streams and improved feedback for
download attempts in the search bar.
- **Style**
- Adjusted button classes and added scoped styles to improve the visual
layout of the search bar component.
- **Chores**
- Updated GitHub Actions workflow configuration for Playwright UI tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced new stream types: "metrics" and "traces" in the Query
component.
- Added a warning message for unsaved changes in the Pipeline Editor.
- Enhanced user experience with a new informational card in the
Condition component.
- **Improvements**
- Simplified components by removing previous node selection
functionality across multiple components.
- Streamlined drag-and-drop functionality for better user interactions.
- Improved error handling and feedback for SQL query validation.
- **Bug Fixes**
- Reactivated tests for error handling in pipeline functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced layout structure and visual presentation in the MainLayout
component.
- Updated navigation item display properties for improved user
experience.
- Added a tooltip to the Theme Switcher for contextual information on
theme modes.
- Introduced a new language selection entry in the user interface.
- Improved error handling and user feedback in the log search interface.
- Adjusted layout heights in various components for better visual
presentation.
- Added a new method for refreshing timezone in the traces component.
- Improved loading spinner visibility in RUM components.
- **Bug Fixes**
- Corrected localization for "Real User Monitoring" to "RUM" in the
English language file.
- **Refactor**
- Simplified the template structure in the MenuLink component for better
semantic clarity.
- Improved drawer visibility control and responsiveness in the
MainLayout component.
- Enhanced error handling and loading state management in the traces and
logs components.
- Updated height calculations for various components to optimize layout.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
issue #5251
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced search functionality in the Search Bar, including improved
handling of saved views and function management.
- Streamlined methods for saving and deleting views with clearer
notifications.
- **Bug Fixes**
- Improved robustness in applying saved views, ensuring checks for
stream existence.
- **Style**
- Adjusted layout and appearance of buttons and dropdowns for a more
consistent user interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Improved handling of logs tab lifecycle and error messages for better
user experience.
- Enhanced responsiveness of table cell styling based on state changes.
- Introduced utility function for validating date and time strings to
enhance error handling.
- Added new enumeration and interface to track UI component states
related to logs and stream exploration.
- **Bug Fixes**
- Adjusted error message display logic for clarity.
- Refined timezone selection handling to ensure correct initial
settings.
- **Refactor**
- Reorganized component methods for better maintainability and
readability.
- Transitioned table cell class management to a reactive reference for
dynamic updates.
- Minor adjustments to the DateTime component for improved code clarity
and functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced organization selection and user authentication processes in
the login component.
- Improved session management with a new method for token refresh
timing.
- Conditional rendering for error messages and search results in the
logs component.
- **Bug Fixes**
- Robust error handling for user login, ensuring better feedback for
invalid credentials.
- Improved error handling and user feedback in the logs component.
- **Refactor**
- Updated organization handling logic to support more complex data
structures and improve user context management during login.
- Enhanced logic flow and state management in the logs component for a
better user experience.
- **Documentation**
- Minor formatting changes for improved readability in the login
component template.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#5073
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced SQL query handling for interesting fields in the logs
component.
- Improved search bar functionality to generate tailored SQL queries for
multiple streams.
- **Bug Fixes**
- Adjusted logic for handling filters to ensure accuracy in generated
SQL queries.
- **Documentation**
- Updated method signatures to reflect changes in parameters and
functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
impl #4968
## How to use
When using pipeline to configure/modify data ingestion, users can set
the destination `Stream` dynamically based on the records that are being
ingested now.
When configuring the destination stream in pipeline editor, instead of
selecting any existing streams from the dropdown or creating a new
stream, one can use `{}` to wrap _**flattened**_ `field_name`, the value
of such field will be the final destination stream.
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced error handling for search queries, providing clearer feedback
to users.
- Improved logic for resetting SQL query parameters in the search bar.
- Added new functions for filtering fields and constructing queries
based on selected streams.
- Introduced a standardized SQL query format with `QUERY_TEMPLATE`.
- **Bug Fixes**
- Adjusted conditions for displaying error messages, ensuring
appropriate prompts for users.
- **Refactor**
- Streamlined control flow in data fetching and error handling methods
for better predictability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced user feedback during function deletions by displaying
associated pipelines or a message when none are linked.
- Improved pipeline management functionality, including state toggling
notifications and refined deletion processes.
- **Bug Fixes**
- Streamlined the fitting view logic for the VueFlow component, ensuring
proper rendering of nodes and edges.
- **Improvements**
- Simplified data filtering and expanded row logic for better user
experience in the pipeline list.
- Adjusted the deletion dialog to accurately reflect the current state
of associated pipelines.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sai Nikhil
Hengfei Yang
Omkar Kesarkhane
Abhay Padamani
harshiv-7
Zhao.wei
