Addresses #4273
- Adds `last_triggered_at`, `owner`, `updated_at` and `last_edited_by`
fields (similar to Reports) to Alerts.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced alert and report management with additional metadata fields
for tracking last triggered times, owners, and edit history.
- Introduced user identification in report creation and updates for
better ownership tracking.
- **Bug Fixes**
- Improved logic for saving alerts and reports to accurately reflect
their statuses and metadata during creation and updates.
- **Documentation**
- Restructured import organization for improved clarity and
maintainability in the codebase.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new structure for enhanced management of cached query
responses, allowing for detailed handling of multiple cached results.
- Added functionality to retrieve multiple cached results in a single
request, optimizing performance and reducing service calls.
- Implemented a mechanism for efficient caching and retrieval of query
results in a distributed cluster environment.
- Enhanced configuration options for managing result caching behavior.
- **Bug Fixes**
- Improved handling of cache misses and response merging to ensure
robustness against empty responses.
- **Refactor**
- Updated internal logic to support new cached query response structure,
streamlining caching operations and enhancing clarity.
- Introduced new modules to improve code organization related to
multi-cluster operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Improved Period of derived stream to be same as frequency
Complete the `Frequency` task in the tracking issue #4170
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced a new component for managing scheduled alerts with a tabbed
interface for custom alerts, SQL queries, and PromQL queries.
- Enhanced user experience with dynamic input fields, real-time
validation, and aggregation settings for alert conditions.
- Added cron job configuration options for scheduling alerts.
- **Changes**
- Renamed the scheduled alert component to reflect the new
functionality, impacting how the application handles alert triggering
and scheduling.
- Adjusted timing and frequency parameters for triggering conditions in
the alert management system.
- **Improvements**
- Enhanced logging functionality for monitoring enrichment data saving,
providing insights into operations and parameters involved.
- Improved error handling and response validation processes for
ingestion tasks, ensuring more informative logging and error reporting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
`DerivedStream`, as part of `Pipeline`, enables the use case of querying
an existing stream, `source`, with optional conditions and aggregations.
The resulted data will then be ingested into a new stream,
`destination`.
This PR provides the feature's initial offering. Please follow below
issue to track & suggest future improvements
#4170
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved caching behavior and response processing in search requests.
- Enhanced error handling and response generation for better user
experience.
- **Tests**
- Disabled specific UI interaction in Playwright tests to prevent test
failures.
- **Chores**
- Updated configuration defaults to improve clarity and efficiency.
- Made internal constants publicly accessible for broader usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- [x] Add two consistent hashing table for `Interactive` and
`Background` querier group
- [x] Only cache latest parquet files for `Interactive` queriers.
Categorizes nodes into role groups selected to serve different query
requests based on query search event type.
---
Each node can be configured into different role groups by using the new
env: `ZO_NODE_ROLE_GROUP`
- default: `` (empty: none)
- node for high-priority tasks: `interactive`
- node for low-priority tasks: `background`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced `node_role_group` configuration to specify role groups for
nodes.
- Added `SearchEventType` to `SearchRequest` to enhance search
capabilities.
- Implemented `RoleGroup` enum to categorize nodes based on priority.
- **Improvements**
- Updated node registration and status setting to include `role_group`.
- Enhanced logging to include node's role group in cluster operations.
- Streamlined search logic by assigning `search_type` earlier in the
request handling process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added sorting functionality for search results based on an
`is_descending` flag.
- Enhanced logging to provide information about cache and result
lengths.
- **Improvements**
- Optimized internal calculations for search results cache.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Improved tracing instrumentation across various search-related
functions by removing the `trace_id` field and refining how `trace_id`
is handled in logs.
- **Bug Fixes**
- Enhanced logging consistency and accuracy by standardizing trace ID
processing in `get_file_list`, `search`, `cluster_search`,
`search_parquet`, and `search_memtable` functions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- User roles are now dynamically assigned based on the `role` parameter
during user addition to an organization.
- **Enhancements**
- Improved role management by incorporating a feature flag for
enterprise-level role determination.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added new fields `is_descending` and `limit` to query responses for
enhanced sorting and pagination.
- **Improvements**
- Enhanced logging and tracing for better query time tracking and
debugging.
- Streamlined span handling for more efficient tracing and
instrumentation.
- **Bug Fixes**
- Fixed validation for `result_cache_enabled` configuration to prevent
cache-related errors.
- Improved accuracy in cache limit handling in query responses.
- **Chores**
- Updated Rust toolchain version for improved compatibility and
performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved the request handling for job cancellation, ensuring a
smoother and more reliable user experience when canceling multiple jobs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for canceling queries within a specific organization.
- Introduced a new RPC method, `ClusterCancelQuery`, for enhanced query
management.
- **Enhancements**
- Improved the `cancel_query` functionality to include organization ID
parameters for more targeted query management.
- Updated error handling and logging for better debugging and user
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced authentication and authorization handling for user requests
with more efficient token validation.
- **Bug Fixes**
- Improved error handling and response based on user permissions and
authorization status.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added `result_cache_ratio` to the search feature to improve response
management.
- Introduced `discard_interval` parameter to enhance cache handling.
- **Improvements**
- Enhanced cache retention and discarding logic with `discard_interval`
for better performance.
- Improved timestamp parsing and rounding functions for more accurate
data handling.
- **Refactor**
- Reorganized logging and utility functions within the search and cache
modules for better code maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced trace ID management with functions for adding, removing,
and generating trace contexts.
- Added the ability to cancel running queries.
- **Enhancements**
- Integrated tracing functionality into search operations for improved
observability.
- Updated CORS configuration to allow `traceparent` headers.
- **Debugging**
- Added console logs for traceparent parameters in search functions.
- **Localization**
- Added new localization key for "cancel" message in English.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved search function performance and reliability by managing
resources more effectively.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced traceability by adding `trace_id` support in various search
and cache request functions.
- **Improvements**
- Updated logging messages to include `trace_id` for better traceability
across services.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support basic-auth header parsing and login for enterprise-GET requests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved logging for user validation to aid in debugging.
- **New Features**
- Enhanced authentication process with better handling of headers and
tokens.
- Introduced new cookie preparation functionality to streamline cookie
management during authentication.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced logging for query start and end times in the search
functionality for better tracking and debugging.
- **Improvements**
- Enhanced cache management by adding logic to discard cached results
based on their duration, ensuring more efficient use of cache storage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Added licensing information to various files to comply with GNU Affero
General Public License.
- Updated copyright information from Zinc Labs Inc. to OpenObserve Inc.
These changes do not affect the application's functionality but ensure
proper licensing documentation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added the ability to delete cached search results for specific
streams.
- Introduced query caching functionalities, including disk-based caching
and cluster-wide query result retrieval.
- **Enhancements**
- Implemented new methods for removing keys from caches based on
different strategies (e.g., LRU, FIFO).
- Added new utility functions for handling and optimizing SQL queries.
- **Improvements**
- Enhanced caching mechanism to support better performance and
efficiency through new data structures and asynchronous operations.
- Expanded public APIs to include additional regex patterns and utility
functions for SQL processing.
- **Documentation**
- Updated public API documentation to reflect new functionalities and
changes in existing functions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a button with tooltip in the dashboard panel to display query
range values.
- **Improvements**
- Updated error messages in search functionality to mention query range
restrictions in hours.
- Enhanced log and dashboard pages with a function to generate formatted
error messages for query range limits.
- **Bug Fixes**
- Fixed alignment issues in the `usePanelDataLoader` function call
within the dashboard components.
- **Event Handling**
- Introduced a new event `"result-metadata-update"` in dashboard panel
renderer for better metadata updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added timestamp tracking to various functions to capture the start
time of operations. This enhancement improves monitoring and debugging
by logging when operations begin.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Subhra264
Ashish Kolhe
Hengfei Yang
Huaijin
胡同里有猫
baerwang
Ankur Srivastava
ktx-vaidehiandoasisk