<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved caching behavior and response processing in search requests.
- Enhanced error handling and response generation for better user
experience.
- **Tests**
- Disabled specific UI interaction in Playwright tests to prevent test
failures.
- **Chores**
- Updated configuration defaults to improve clarity and efficiency.
- Made internal constants publicly accessible for broader usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- [x] Add two consistent hashing table for `Interactive` and
`Background` querier group
- [x] Only cache latest parquet files for `Interactive` queriers.
Categorizes nodes into role groups selected to serve different query
requests based on query search event type.
---
Each node can be configured into different role groups by using the new
env: `ZO_NODE_ROLE_GROUP`
- default: `` (empty: none)
- node for high-priority tasks: `interactive`
- node for low-priority tasks: `background`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced `node_role_group` configuration to specify role groups for
nodes.
- Added `SearchEventType` to `SearchRequest` to enhance search
capabilities.
- Implemented `RoleGroup` enum to categorize nodes based on priority.
- **Improvements**
- Updated node registration and status setting to include `role_group`.
- Enhanced logging to include node's role group in cluster operations.
- Streamlined search logic by assigning `search_type` earlier in the
request handling process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added sorting functionality for search results based on an
`is_descending` flag.
- Enhanced logging to provide information about cache and result
lengths.
- **Improvements**
- Optimized internal calculations for search results cache.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Improved tracing instrumentation across various search-related
functions by removing the `trace_id` field and refining how `trace_id`
is handled in logs.
- **Bug Fixes**
- Enhanced logging consistency and accuracy by standardizing trace ID
processing in `get_file_list`, `search`, `cluster_search`,
`search_parquet`, and `search_memtable` functions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- User roles are now dynamically assigned based on the `role` parameter
during user addition to an organization.
- **Enhancements**
- Improved role management by incorporating a feature flag for
enterprise-level role determination.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added new fields `is_descending` and `limit` to query responses for
enhanced sorting and pagination.
- **Improvements**
- Enhanced logging and tracing for better query time tracking and
debugging.
- Streamlined span handling for more efficient tracing and
instrumentation.
- **Bug Fixes**
- Fixed validation for `result_cache_enabled` configuration to prevent
cache-related errors.
- Improved accuracy in cache limit handling in query responses.
- **Chores**
- Updated Rust toolchain version for improved compatibility and
performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved the request handling for job cancellation, ensuring a
smoother and more reliable user experience when canceling multiple jobs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for canceling queries within a specific organization.
- Introduced a new RPC method, `ClusterCancelQuery`, for enhanced query
management.
- **Enhancements**
- Improved the `cancel_query` functionality to include organization ID
parameters for more targeted query management.
- Updated error handling and logging for better debugging and user
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced authentication and authorization handling for user requests
with more efficient token validation.
- **Bug Fixes**
- Improved error handling and response based on user permissions and
authorization status.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added `result_cache_ratio` to the search feature to improve response
management.
- Introduced `discard_interval` parameter to enhance cache handling.
- **Improvements**
- Enhanced cache retention and discarding logic with `discard_interval`
for better performance.
- Improved timestamp parsing and rounding functions for more accurate
data handling.
- **Refactor**
- Reorganized logging and utility functions within the search and cache
modules for better code maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced trace ID management with functions for adding, removing,
and generating trace contexts.
- Added the ability to cancel running queries.
- **Enhancements**
- Integrated tracing functionality into search operations for improved
observability.
- Updated CORS configuration to allow `traceparent` headers.
- **Debugging**
- Added console logs for traceparent parameters in search functions.
- **Localization**
- Added new localization key for "cancel" message in English.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved search function performance and reliability by managing
resources more effectively.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced traceability by adding `trace_id` support in various search
and cache request functions.
- **Improvements**
- Updated logging messages to include `trace_id` for better traceability
across services.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support basic-auth header parsing and login for enterprise-GET requests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved logging for user validation to aid in debugging.
- **New Features**
- Enhanced authentication process with better handling of headers and
tokens.
- Introduced new cookie preparation functionality to streamline cookie
management during authentication.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced logging for query start and end times in the search
functionality for better tracking and debugging.
- **Improvements**
- Enhanced cache management by adding logic to discard cached results
based on their duration, ensuring more efficient use of cache storage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Documentation**
- Added licensing information to various files to comply with GNU Affero
General Public License.
- Updated copyright information from Zinc Labs Inc. to OpenObserve Inc.
These changes do not affect the application's functionality but ensure
proper licensing documentation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added the ability to delete cached search results for specific
streams.
- Introduced query caching functionalities, including disk-based caching
and cluster-wide query result retrieval.
- **Enhancements**
- Implemented new methods for removing keys from caches based on
different strategies (e.g., LRU, FIFO).
- Added new utility functions for handling and optimizing SQL queries.
- **Improvements**
- Enhanced caching mechanism to support better performance and
efficiency through new data structures and asynchronous operations.
- Expanded public APIs to include additional regex patterns and utility
functions for SQL processing.
- **Documentation**
- Updated public API documentation to reflect new functionalities and
changes in existing functions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a button with tooltip in the dashboard panel to display query
range values.
- **Improvements**
- Updated error messages in search functionality to mention query range
restrictions in hours.
- Enhanced log and dashboard pages with a function to generate formatted
error messages for query range limits.
- **Bug Fixes**
- Fixed alignment issues in the `usePanelDataLoader` function call
within the dashboard components.
- **Event Handling**
- Introduced a new event `"result-metadata-update"` in dashboard panel
renderer for better metadata updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added timestamp tracking to various functions to capture the start
time of operations. This enhancement improves monitoring and debugging
by logging when operations begin.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for query duration adjustments based on a maximum query
range in search functionality.
- Introduced new fields for time range validation in search responses.
- Included maximum query range settings in stream configurations.
- **Improvements**
- Enhanced log stream components to handle maximum query range
restrictions effectively.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
- [x] Includes the following APIs endpoints in auditing -
- `/auth/login` - If the login is successful, the user email along with
the event data (such as response code, path, method, query params and
the timestamp) and if it is unsuccessful, it still audits the event, but
username will be present only if available.
- `/auth/presigned-url`
- `/auth/login` - `GET` request. Similar to the first `login` endpoint.
- `/config/reload` - Audits this event, but as this is not a protected
route, the username will not be present in the audit record.
- `/config/redirect` - Similar to the first `login` endpoint.
- `/config/logout`
- [x] Audit Cron job is run on all the nodes including the router. The
interval can be set through `O2_AUDIT_PUBLISH_INTERVAL` env (in
seconds).
- [x] In case of remote usage reporing, print the error if response from
remote usage url is unsuccessful
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced enhanced auditing functionalities for user authentication
and authorization in the enterprise version.
- Added asynchronous task for auditing and publishing tasks within job
module.
- **Improvements**
- Refined error handling in usage data ingestion to better manage
success and failure responses.
- **Conditional Compilation**
- Multiple new functionalities and auditing enhancements are now
conditionally compiled based on the "enterprise" feature.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced multi-select stream functionality in UI, allowing users to
select and query multiple data streams simultaneously.
- Added new search functions to handle multiple streams of data and
provide aggregated search results.
- **Bug Fixes**
- Corrected UI test cases to improve selection and verification of
stream names.
- **Enhancements**
- Improved search functionalities and error handling within the logs
plugin interface.
- Enhanced stream handling and validation in the search feature.
- **Tests**
- Added new Playwright test cases for stream multi-select functionality.
- Updated existing test cases to adapt to new multi-select stream logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Neha P <neha@openobserve.ai>
Co-authored-by: Neha P <144308844+neha00290@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
In case of generating a presigned-url the base url should be a
combination of
```
let base_url = format!("{}{}", cfg.common.web_url, cfg.common.base_uri);
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Improved URL construction by introducing a `base_url` variable for
better maintainability and flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hengfei Yang
Huaijin
胡同里有猫
Taiming Liu
Ashish Kolhe
baerwang
Subhra264
Ankur Srivastava
Omkar KesarkhaneandTaiming Liu
ktx-vaidehiandoasisk
