Addresses #4854
Currently, when updating organization settings, if relevant fields (like
`scrape_interval`, `trace_id_field_name` etc.) are not found, their
default values are used and hence all the fields that are not in the
request payload, are set to default.
This PR only updates the fields that are present in the request payload,
if none of the relevant fields are present, it returns 400 bad request
error.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `OrganizationSettingPayload` structure for optional
organization settings, enhancing flexibility in configuration.
- Updated the settings creation process to handle optional fields and
improve validation.
- **Bug Fixes**
- Enhanced error handling for organization settings, ensuring proper
responses for invalid or missing data.
- **Documentation**
- Updated method signatures to reflect changes in the expected input
structure for organization settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Closes: #4749
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced logic for updating stream settings, including synchronization
with index streams for data retention.
- Improved error handling for stream cache operations, returning
immediate responses for disabled caches.
- **Bug Fixes**
- Added checks to ensure proper handling of cache deletion based on
cache status.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `RedirectResponse` struct for improved HTTP
redirection handling.
- Enhanced authentication validation with a new redirect mechanism for
`/short/` requests.
- Updated the `retrieve` function to improve logging and response
handling.
- **Bug Fixes**
- Improved error management for authentication failures.
- **Refactor**
- Restructured route configurations to integrate new authentication
middleware and enhance service management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
Addresses #4619
- [x] Multi timerange alerts
- [x] Add `is_partial` field to the `triggers` usage stream.
### Note: Only available in SQL alerts
<img width="1248" alt="Screenshot 2024-10-07 at 12 39 36 PM"
src="https://github.com/user-attachments/assets/57c99b29-6d59-496d-bb91-f958b6f2e0bd">
- Historical timeranges can be selected through the `Multi Window
Selection` field.
- When multi-window timeranges are selected, the vrl of the alert is
applied only after fetching all the timerange data and is applied to the
combined multi-timerange records.
- If `ResultArray` is specified in the vrl function, the input to the
vrl function will be multi-dimensional i.e. if two timeranges are
selected, then the input to the vrl function will be a two-dimensional
array containing a total of three arrays. Each array represents the data
of each timerange.
- If `ResultArray` is not specified, the input of vrl function is an
one-dimensional array containing records of all the timeranges in a
sorted order (from most recent to oldest).
- After applying the vrl function, the output array (irrespective of
multi/single dimension array) is first converted into a one-dimensional
array and then only the `threshold` rule is applied to the output of vrl
function. If the `threshold` satisfies then the alert triggers otherwise
not.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced support for multiple time ranges in alert configurations.
- Added a new `search_multi` function for handling multiple search
queries in a single request.
- Enhanced alert creation with new properties and methods for managing
multiple time ranges.
- New `is_partial` field added to `TriggerData` for tracking partial
evaluations.
- **Bug Fixes**
- Improved error handling for partial alert evaluations.
- **Documentation**
- Updated comments and documentation for better clarity on new features
and changes.
- **Refactor**
- Streamlined the logic for SQL query handling and alert evaluation
processes.
- **Style**
- Minor formatting adjustments across various components for improved
readability.
- **Tests**
- Enhanced validation checks and filtering logic in alert management
components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
#4698
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new "Seconds" option in the date-time picker for more
precise time selection.
- Updated time formats to include seconds in the start and end time
fields.
- Enhanced the component's ability to manage relative time selections
with the addition of seconds.
- Improved handling of duration strings to recognize seconds.
- Adjusted dashboard interactions to handle date-time values more
effectively during zoom events.
- **Bug Fixes**
- Improved validation and logic for handling new time formats and
relative periods.
- Enhanced clarity and efficiency in query duration handling for search
functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
- Add `function`, `event`, `_timestamp` and `unit` to
`SearchHistoryHitResponse`.
- Only include `Search` events for search history
- Revert the change made in #4694 i.e. reuse `SearchHIstoryHitResponse`
for search history.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced search history functionality with additional metadata fields
for more detailed responses.
- Improved error handling and logging for search history operations.
- Updated data processing logic in the Search History component for
accurate representation of timestamps and durations.
- **Bug Fixes**
- Streamlined data processing and presentation in the Search History
component, ensuring accurate display of search history data.
- **Documentation**
- Updated component and method signatures to reflect changes in data
handling and structure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
1.New Fields --> executed at introduced --> this helps in user to know
when this search got executed.
2. when expands Function defination got introduced if user searched with
function then it will be displayed.
3. Pagination got introduced to show all the results by default per page
user can able to see 100 rows.
4.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced search history component with pagination and improved data
handling.
- Added new fields in usage tracking to capture function-specific data.
- New method for dynamic pagination control in the search history
component.
- **Improvements**
- Streamlined search result processing by removing unnecessary
filtering.
- Improved timestamp handling in usage reporting.
- Flexible handling of the size parameter in search history retrieval.
- Refined partitioning logic and filtering capabilities in search
functionality.
- Enhanced user interface responsiveness in the search history
component.
- **Bug Fixes**
- Adjusted sorting functionality and pagination logic in the search
history component.
These updates enhance the user experience and improve data accuracy
throughout the application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced search history functionality with organization-specific
filtering.
- Added a warning message for potential delays in search history
retrieval.
- Introduced new configuration fields for usage tracking in the health
status response.
- **Bug Fixes**
- Improved error handling for search functions, enhancing traceability.
- **Chores**
- Updated the configuration management and health monitoring
capabilities for better system oversight.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Improved handling of query time ranges by setting default start and
end times when not defined.
- Enhanced robustness in response handling by ensuring histogram
intervals are assigned when not previously set.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced query handling with restrictions based on stream settings.
- Improved response messaging to indicate partial data availability.
- **Bug Fixes**
- Adjusted query start time when exceeding maximum duration limits,
ensuring compliance with stream settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced processing of query functions with base64 decoding for
improved compatibility.
- Introduced advanced handling of VRL functions for more dynamic search
results.
- **Bug Fixes**
- Improved logic for applying VRL functions to search results, ensuring
accurate response processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `SqlQuery` structure for improved SQL query handling.
- Added a `per_query_response` feature to enhance query response
flexibility.
- **Bug Fixes**
- Improved error handling and response clarity in search functions.
- Enhanced request formatting for better readability and consistency.
- **Refactor**
- Restructured request and response handling in `search_multi` and
`_search_partition_multi` functions for improved code clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fixed#4568
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced a method to clear and manage internal storage for file
locking, enhancing resource management.
- Added functionality to clean lock files under specific conditions when
nodes are enabled or flushed.
- Enhanced response handling for partial data in search results,
providing clearer error messages to users.
- **Bug Fixes**
- Improved error handling in the search functionality to ensure
resources are released properly during errors, preventing potential
leaks.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced time value determination for enrichment tables by sourcing
from statistics instead of user input.
- Expanded the `AddSettingVariable` component to support an additional
stream type: "enrichment_tables."
- **Bug Fixes**
- Improved control flow to ensure correct application behavior based on
stream type.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Destination less report for caching purpose.
- [x] Empty destination array is supported for caching purpose. If
destinations array is empty it only spawns a chromium and loads the
dashboard, does not capture pdf.
- [x] For Cache type reports, it uses ui search type so that the
dashboard queries land on only the interactive node and not in the
background node.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new filtering system for reports, allowing users to
specify criteria such as dashboard and folder.
- Added support for two report types: PDF and Cache, enhancing report
generation options.
- Enhanced report generation functionality to include dynamic handling
based on report type and recipient count.
- **Bug Fixes**
- Improved error handling and logging for report generation processes.
- **Documentation**
- Updated documentation to reflect new parameters and functionalities in
report handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- [x] Enrichment tables are not listed correctly with RBAC.
- [x] Create ofga ownership for metadata streams
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Enhanced handling of stream types for improved flexibility and
functionality.
- Introduced enterprise-level ownership management features for metadata
handling in the context of the enterprise feature.
- **Bug Fixes**
- Streamlined logic for retrieving model keys associated with stream
types, ensuring more accurate results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Added `owner` and `enabled` URI query parameter for the
`{org_id}/alerts` and `{org}/{stream_name}/alerts` `GET` APIs. E.g. -
`{org}/alerts?owner=user@user.com&enabled=true`.
Fixes#4273
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new filtering capability for alerts, allowing users to
filter by owner and enabled status.
- Added a new structure, `AlertListFilter`, to manage alert filtering
criteria.
- **Improvements**
- Enhanced alert retrieval functions to support user-defined query
parameters for more granular control over alert listings, improving the
overall usability of the alert system.
- Expanded filtering logic in the AlertList component for a more
comprehensive search capability within the alert list.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Fixes#4455
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced authorization checks for `query`, `query_range`, and `series`
functions to dynamically reflect stream types.
- Improved permission validation logic in the `search` and
`search_multi` functions, allowing for more accurate access control
based on user roles and stream types.
- Updated logic in the `get_latest_traces` function to refine the
construction of the `o2_type` field for better permission checks.
- **Bug Fixes**
- Corrected the mapping of stream types to model keys, enhancing
security and accuracy in permission checks.
- **Chores**
- Added necessary imports for `OFGA_MODELS` to support new permission
logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Fixes#4394
- When cloning alert, don't copy `last_satisfied_at`,
`last_triggered_at`
When the alert query is invalid, the scheduler stops processing the
alert and returns, but the error message is not reported in the triggers
usage stream. This PR fixes that.
When the alert query is invalid, it will be retried again and again
(default maximum number of retries is `3`). After that, the associated
`scheduled_jobs` record will be automatically removed from the table and
hence the alert will not be triggered anymore.
In order to schedule the alert again, the user needs to update the query
and again save it, it will start scheduling again.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Enhanced alert handling logic for improved robustness and clarity.
- Introduced structured handling of alert data with better error
management.
- Added initialization of `last_triggered_at` and `last_satisfied_at`
fields to ensure clear state management for alerts.
- **Bug Fixes**
- Improved error handling for alert evaluation, ensuring accurate status
updates for alert triggers.
- Enhanced state preservation for alerts by updating the
`last_satisfied_at` timestamp during saves.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
issue #4326
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced new structures for updating stream settings, allowing for
more granular management of stream configurations.
- Added an asynchronous function for updating stream settings via HTTP
PUT requests.
- Implemented a new service route for handling updates to stream
settings.
- Enhanced the `AddStream` component to retrieve updated settings.
- **Improvements**
- Improved schema management by tracking previous settings during
updates.
- Enhanced utility functions for comparing and managing stream settings.
- **Documentation**
- Updated documentation to reflect changes in API functionality for
creating and updating stream settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
Fixes#4291
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new method for handling 409 Conflict responses in the
HTTP responses, improving error messaging clarity.
- **Bug Fixes**
- Updated error handling to return conflict responses instead of
forbidden responses when certain delete operations fail due to resource
conflicts.
- Enhanced clarity in user membership responses by changing the error
message when a user is already part of an organization.
These changes improve user experience by providing clearer and more
accurate feedback on error conditions related to resource management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Addresses #4273
- Adds `last_triggered_at`, `owner`, `updated_at` and `last_edited_by`
fields (similar to Reports) to Alerts.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced alert and report management with additional metadata fields
for tracking last triggered times, owners, and edit history.
- Introduced user identification in report creation and updates for
better ownership tracking.
- **Bug Fixes**
- Improved logic for saving alerts and reports to accurately reflect
their statuses and metadata during creation and updates.
- **Documentation**
- Restructured import organization for improved clarity and
maintainability in the codebase.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
`DerivedStream`, as part of `Pipeline`, enables the use case of querying
an existing stream, `source`, with optional conditions and aggregations.
The resulted data will then be ingested into a new stream,
`destination`.
This PR provides the feature's initial offering. Please follow below
issue to track & suggest future improvements
#4170
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved caching behavior and response processing in search requests.
- Enhanced error handling and response generation for better user
experience.
- **Tests**
- Disabled specific UI interaction in Playwright tests to prevent test
failures.
- **Chores**
- Updated configuration defaults to improve clarity and efficiency.
- Made internal constants publicly accessible for broader usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hengfei Yang
Subhra264
Loakesh Indiranandhengfei yang
Prabhat Sharma
Huaijin
Ashish Kolhe
John Vandenberg
Yashodhan Joshi
