<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced layout structure and visual presentation in the MainLayout
component.
- Updated navigation item display properties for improved user
experience.
- Added a tooltip to the Theme Switcher for contextual information on
theme modes.
- Introduced a new language selection entry in the user interface.
- Improved error handling and user feedback in the log search interface.
- Adjusted layout heights in various components for better visual
presentation.
- Added a new method for refreshing timezone in the traces component.
- Improved loading spinner visibility in RUM components.
- **Bug Fixes**
- Corrected localization for "Real User Monitoring" to "RUM" in the
English language file.
- **Refactor**
- Simplified the template structure in the MenuLink component for better
semantic clarity.
- Improved drawer visibility control and responsiveness in the
MainLayout component.
- Enhanced error handling and loading state management in the traces and
logs components.
- Updated height calculations for various components to optimize layout.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
issue #5251
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced search functionality in the Search Bar, including improved
handling of saved views and function management.
- Streamlined methods for saving and deleting views with clearer
notifications.
- **Bug Fixes**
- Improved robustness in applying saved views, ensuring checks for
stream existence.
- **Style**
- Adjusted layout and appearance of buttons and dropdowns for a more
consistent user interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Improved handling of logs tab lifecycle and error messages for better
user experience.
- Enhanced responsiveness of table cell styling based on state changes.
- Introduced utility function for validating date and time strings to
enhance error handling.
- Added new enumeration and interface to track UI component states
related to logs and stream exploration.
- **Bug Fixes**
- Adjusted error message display logic for clarity.
- Refined timezone selection handling to ensure correct initial
settings.
- **Refactor**
- Reorganized component methods for better maintainability and
readability.
- Transitioned table cell class management to a reactive reference for
dynamic updates.
- Minor adjustments to the DateTime component for improved code clarity
and functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced comprehensive UI testing for the Home, Ingestion, Login,
Logs, Metrics, Traces, Pipelines, Rum, Dashboard, Alerts, Reports, Data,
IAM, and About pages.
- Added functionality to handle organization changes and validations
across multiple pages, including default organization selections.
- New test suite for validating organization change scenarios across
various pages.
- Enhanced login functionality for internal users.
- Introduced new classes and methods for the Metrics, Streams, Rum, and
Alerts pages to support UI testing.
- Added new classes for the Data and IAM pages, expanding testing
capabilities.
- Enhanced Management page with navigation and validation capabilities.
- New methods for validating page URLs and content across various pages.
- **Bug Fixes**
- Improved validation methods to ensure proper navigation and content
checks on various pages.
- **Documentation**
- Updated testing framework to include new test cases and improved
structure for organization change tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
This PR only refactors code and should not result in any behavioral
changes.
This PR follows #5206 and attempts to refactor dashboards in a similar
way to how that PR refactors folders. This PR aims to divide our
dashboard code into distinct "HTTP Layer" and "Service Layer"s - where
the "HTTP Layer" contains only logic and models related to
serializing/deserializing for the REST HTTP API, and the "Service Layer"
contains only domain models/logic and is agnostic to the REST HTTP API.
This PR only partially achieves the above goals. Specifically, it makes
following changes...
1. Move all HTTP responses out of the `service` module and and into the
`http/request` module.
2. Introduce a new `DashboardError` in the `service` module to model all
types of "domain logic" errors that occur when interacting with folders.
3. Remove the `service/db/folders` module since, after the migration of
folders from the `meta` table to the `folders` table, this module mostly
just acted as a wrapper around the `infra/table/folders` module.
This PR is still missing the following changes. These changes will be
saved for a subsequent PR in order to keep the scope of this PR limited:
- Introduce new `...Response` and `...Request` types in the
`http/request/models` module whose role is specifically to define
serialization/deserialization definitions for the REST HTTP API.
⚠️ Notably, at this point the "HTTP Layer", "Service Layer", and "DB
Layer" models for dashboards are still tightly coupled together since
all still use the shared JSON serialization/deserialization logic
defined on the shared `Dashboard` struct in `config::meta`. Decoupling
the models and de/serialization logic at each layer will take more
effort than folders since the `Dashboard` struct is large and
complicated. I plan to start on this decoupling in the follow-up PR,
though that PR likely won't complete the process.
## Overall design goals
The design goals here are the same as in #5206 . Please see that PR's
description for a detailed explanation.
## Immediate motivation and next steps
Motivations and next steps are generally the same as in #5206 . Please
see that PR's description for a detailed explanation.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced dashboard creation and update processes with improved error
handling and response management.
- Added support for parsing dashboard data across multiple versions.
- **Bug Fixes**
- Streamlined error handling for dashboard operations, ensuring more
consistent and informative responses.
- **Chores**
- Refactored dashboard management logic to remove unnecessary
complexity, improving maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
This PR only refactors code and should not result in any behavioral
changes.
This PR divides our folder code into distinct "HTTP Layer" and "Service
Layer"s - where the "HTTP Layer" contains only logic and models related
to serializing/deserializing for the REST HTTP API, and the "Service
Layer" contains only domain models/logic and is agnostic to the REST
HTTP API.
Specifically, the following changes are made to the code...
1. Move all HTTP responses out of the `service` module and and into the
`http/request` module.
2. Introduce a new `FolderError` in the `service` module to model all
types of "domain logic" errors that occur when interacting with folders.
3. Introduce new `...Response` and `...Request` types in the
`http/request/models` module whose role is specifically to define
serialization/deserialization definitions for the REST HTTP API.
4. Remove the `service/db/folders` module since, after the migration of
folders from the `meta` table to the `folders` table, this module mostly
just acted as a wrapper around the `infra/table/folders` module.
## Overall design goals
The design goal here is to create a clearer divide between the "HTTP
Layer" and "Service Layer" of the application and to decouple the models
in these layers so that models in one layer can change without
introducing breaking changes to models in the other layer. Changes to a
model in one layer only need to be accompanied with appropriate changes
to the translation functions between that model and others.
## Immediate motivation and next steps
I would like to replicate this pattern with Dashboards as well as Alerts
and Reports. Implementing this pattern in Folders seems like a
reasonable place to start since Folders are much less complicated that
Dashboards or Alerts, and Dashboards (and soon-to-be Alerts) are
dependent on Folders since each Dashboard belongs to a parent Folder.
With Dashboards, I have an immediate need to divide "HTTP Layer" and
"Service Layer" models because as parts of #5198 and #5196 I need to
change the data structure of dashboards that are returned by
`ListDashboards` so that they are different from the `Dashboard` data
structure defined in the "Service Layer" - specifically I need each
"dashboard" returned by `ListDashboards` to contain the parent folder's
name and ID, and I need each "dashboard" returned by `ListDashboards` to
exclude the details inside the `v1`,...,`v5` fields.
Additionally, when implementing the migration of dashboards from the
`meta` table to the `dashboards` table, I ran into a lot of avoidable
bugs because our "HTTP Layer", "Service Layer", and "DB Layer" models
for dashboards are all coupled together - reusing the same structure and
serialization/deserialization logic. Therefore, when I changed the model
in which we stored dashboards in the DB, this also accidentally
introduced a breaking change (with no compiler error) to the way
dashboards are serialized/deserialized in the HTTP API. Maintaining
separate models and translation functions for these 3 layers would have
prevented these issues. Therefore, when I start migrating alerts to
there own table, I would like to start by dividing our models for alerts
into "HTTP Layer", "Service Layer", and "DB Layer" models to avoid the
same problems I faced while migrating dashboards.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced new data models for folder management in the API, enhancing
request and response structures.
- Added an asynchronous function to check for folder existence by ID.
- **Improvements**
- Enhanced error handling with a new `FolderError` enum for better
clarity in folder management operations.
- Improved type safety by specifying request body types for folder
creation and updates.
- Refactored folder management API endpoints for clearer response
structures.
- **Bug Fixes**
- Updated folder management API endpoints to ensure more accurate
responses and error handling.
- **Documentation**
- Updated OpenAPI documentation to reflect new folder management schemas
and remove outdated components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
In my previous PR for distinct values,
https://github.com/openobserve/openobserve/pull/5087 the schema defined
for `distinct_values_fields` table did not work with mysql, and OO
failed to start. In this PR I fix that by updating the schema.
The core issue was that mysql does not allow unbounded (text) data type
cols in its index/primary keys whereas sqlite and pgsql do. To fix the
issue, I have updated the table schema to instead used bounded length
columns. The following considerations has been done for size
determination :
- origin, stream_type and origin_id are given size based on sensible
estimates of what the length of possible values could be
- org and stream_name is matched with org in file_llist table
- field_name is given remaining size around power of 2.
The limit is `sum of sizes *4 <=3072 `.
I have tested this on
- new sqlite, mysql, pgsql to make sure this scheme works with all three
dbs
- existing sqlite and pgsql db, where code form current main was run,
and the table was already created. This works with them too, so it is
backward compatible.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Increased maximum length for several database fields, enhancing data
storage capabilities.
- Updated field types in the data model for better structure and
clarity.
These changes improve the application's ability to handle larger data
inputs while maintaining data integrity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Addresses the first point of #4996
Alert emails currently sends the alert template body as a html content.
This might create problems for email clients that does not
prefer/support html content. This PR addresses this issue by offering
both text and html version of the same message. Email clients can now
use whatever version it prefers.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced email notifications to support both plain text and HTML
formats, improving compatibility with various email clients.
- **Bug Fixes**
- Adjusted email message construction for better rendering in email
clients without altering the overall functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
As mentioned in the comments of #4996
- [x] When using row templates with emails, O2 automatically adds `\n`
at the last of the row in email body. Since, email body is currently
treated as html body, the below row template -
```
<p> This is an event {alert_start_time} </p>
```
generates this email body -
```
This is an event 2024-12-02T11:25:40
\n
This is an event 2024-12-02T11:26:40
\n
This is an event 2024-12-02T11:27:40
\n
...
```
- [x] Adds new variables to represent when the alert got evaluated -
`alert_trigger_time` (unix timestamp in microseconds) and
`alert_trigger_time_str` (formatted, e.g. - `2024-12-02T11:27:40`)
- [x] Update document add `alert_trigger_time` and
`alert_trigger_time_str`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Enhanced alert notifications with improved timestamp handling.
- Updated alert template documentation to include new variables for
better clarity.
- **Bug Fixes**
- Refined error handling in the notification sending process for better
logging and context.
- **Documentation**
- Added documentation for new variables in the alert template section to
assist users in utilizing timestamps effectively.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix#5218
Backend:
- [x] allow condition node to evaluate `null` and `""`, empty, for both
`=` and `!=` conditions
- [x] stop ingestion when resolved dynamic stream name is empty
**How to Use(In ConditionNode):**
To evaluate field value is not empty:
`app_name != ""`
To evaluate field value is not null:
`app_name != null`
Fixes#5009
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced email validation for alert destinations to ensure consistent
formatting (trimmed and lowercased).
- **Bug Fixes**
- Improved error handling during email validation for a more consistent
user experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
https://github.com/openobserve/openobserve/issues/5198
This PR adds a new optional `title` query parameter to the
`ListDashboards` endpoint.
The endpoint now has the following behavior when these parameters are
provided:
- `folder` is provided and `title` is provided: The endpoint returns all
dashboards in the folder whose titles contain the value of `title` as
case-insensitive substring. This is new non-breaking behavior.
- `folder` is provided and `title` is not provided: The endpoint returns
all dashboards in the folder. This is the same behavior as before.
- `folder` is not provided and `title` is provided: The endpoint returns
all dashboards (regardless of folder) whose titles contain the value of
`title` as a case-insensitive substring. This is new non-breaking
behavior and enables the feature of searching for dashboards across
folders.
- `folder` is not provided and `title` is not provided: The endpoint
returns all dashboards inside the "default" folder. More obvious
behavior would be to return all dashboards, however the endpoint returns
only dashboards inside the "default" folder in order to maintain
backwards comparability and to avoid making a breaking change.
The PR should not result an any behavior changes on the UI.
Related Issues: #5191
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced search event tracking with additional context fields: report
key, dashboard folder ID, and dashboard folder name.
- **Bug Fixes**
- Improved serialization of search event context for better data
handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
- #5165
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced drag-and-drop functionality with improved error handling and
user notifications.
- Updated computed properties to provide context-specific hints and
accurate value labels for map data.
- Integrated dependency injection for better access to dashboard panel
data.
- **Bug Fixes**
- Implemented checks to ensure items are added or removed from the
correct target axis during drag-and-drop operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
When use TantivyCountExec for count(*) optimizer, we also can collect
stats
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced search functionality with improved handling of search
statistics.
- Introduced a new feature for collecting and aggregating file
processing statistics.
- **Bug Fixes**
- Improved error handling during file retrieval and processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced `LogsPage` with improved date and time selection, timezone
management, and clipboard operations.
- Introduced `ManagementPage` for UI testing, allowing logo updates and
management functionalities.
- Added new tests for time zone changes and streamlined test scenarios
on the logs page.
- Introduced a suite of tests for joining logs, validating various join
scenarios.
- **Bug Fixes**
- Updated confirm button interaction in `ReportsPage` to ensure
visibility before clicking.
- **Documentation**
- Improved clarity and purpose of test scenarios through restructuring
and simplification.
- **Chores**
- Added `logoManagement.spec.js` and `join.spec.js` to the Playwright UI
integration tests for expanded coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced organization selection dropdown for improved user experience.
- Conditional rendering for custom logos and menu items based on user
configuration.
- Improved handling of dynamic stream names in the input fields based on
selected node type.
- **Bug Fixes**
- Improved logic for fetching summary data, ensuring valid organization
identifiers are used.
- **Style**
- Adjusted styling for consistency across layout components.
- **Documentation**
- Updated error handling and logging for better troubleshooting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced organization selection and management for improved user
experience.
- Streamlined logic for retrieving organization identifiers and options.
- Conditional navigation to subscription plans based on user actions.
- **Bug Fixes**
- Improved error handling during organization selection to prevent
exceptions.
- **Documentation**
- Updated method signatures to reflect new logic and error handling
capabilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- #5163
- #2236
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced logic to prevent navigation to empty dashboards.
- Improved routing when returning to the dashboard list.
- Added organization identifier to routing for adding panels.
- **Bug Fixes**
- Added checks to handle undefined or empty dashboard data in the
`AddPanel` component.
- Refined error handling for panel operations, ensuring better user
feedback.
- **Documentation**
- Updated method signatures to reflect changes in navigation and data
handling logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced a configuration option to enable or disable the inverted
index count optimizer.
- Added a new module for handling `TantivyCountExec` execution plans,
optimizing count queries.
- Enhanced the search functionality to support concurrent processing
with DataFusion and Tantivy.
- Added a new `QueryParams` struct for improved query management and
organization.
- Enhanced caching and filtering functions to provide total hits
feedback during operations.
- Introduced a new function to evaluate simple count queries for better
optimization.
- **Bug Fixes**
- Improved error handling and logging across search and caching
functions.
- **Documentation**
- Enhanced comments for clarity in SQL query handling and execution
plans.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
Fixes#5161.
Currently, for dashboard queries (`search_type = dashboard`), the same
metrics query is reported twice in the usage stream - one as `logs`,
other as `metrics`. Hence, the dashboard metrics search is shown in the
search history page as well. This pr fixes this by reporting the search
query only once for dashboards search type.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced reporting of usage statistics with the addition of a cache
ratio for results.
- Improved formatting and error handling in the search functionality for
better query management.
- **Bug Fixes**
- Refined logic for accumulating results and managing errors in
multi-query execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for line, area charts for single x, y axis, label not working
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced chart rendering options based on SQL data input.
- Improved tooltip formatting for better user experience with hovered
series.
- Updated series configuration for various chart types to include label
options.
- Added validation for search query data to ensure robust data
processing.
- **Bug Fixes**
- Improved handling of missing values in chart data.
- **Refactor**
- Streamlined data processing logic for better performance and
reliability.
- Encapsulated data processing logic in a dedicated function for
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced dashboard management with new methods for retrieving
properties like ID, owner, title, and description.
- Added functionality for creating, updating, listing, and deleting
dashboards through a new service module.
- Introduced a new `dashboards` table in the database, with migrations
for creation and population.
- **Bug Fixes**
- Improved error handling for dashboard retrieval and management
operations, providing clearer responses for various conditions.
- **Documentation**
- Updated documentation to reflect new methods and error handling
improvements.
- **Refactor**
- Streamlined the dashboard handling logic in service and database
modules, reducing complexity and enhancing modularity.
- Enhanced error handling logic in dashboard management functions for
better clarity and control flow.
- Refined the dashboard retrieval process to improve clarity in control
flow and error handling.
- Updated the logic for dashboard operations to simplify version
handling and reduce redundancy.
- Improved the overall structure and organization of the dashboard
management code.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced organization management with improved responsiveness to user
interactions.
- Added a watcher to track changes in the selected organization,
ensuring prompt updates to the layout.
- **Bug Fixes**
- Improved error handling and logging during organization selection.
- **Refactor**
- Restructured organization change handling for better performance and
clarity.
- Updated logic for setting the default organization when none is
selected.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced query execution flexibility with the addition of an
optimization mode for search operations.
- Improved handling of query parameters to accommodate new optimization
strategies.
- **Bug Fixes**
- Updated error handling to ensure compatibility with the new
optimization mode parameter.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced organization selection and user authentication processes in
the login component.
- Improved session management with a new method for token refresh
timing.
- Conditional rendering for error messages and search results in the
logs component.
- **Bug Fixes**
- Robust error handling for user login, ensuring better feedback for
invalid credentials.
- Improved error handling and user feedback in the logs component.
- **Refactor**
- Updated organization handling logic to support more complex data
structures and improve user context management during login.
- Enhanced logic flow and state management in the logs component for a
better user experience.
- **Documentation**
- Minor formatting changes for improved readability in the login
component template.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hengfei Yang
Bhargav
Omkar Kesarkhane
Huaijin
ShyamOOAI
Patrick Sullivan
Yashodhan JoshiandYashodhan Joshi
Subhra264
dependabot[bot]
Taiming Liu
胡同里有猫
Loakesh IndiranandTaiming Liu
ktx-vaidehi
Florent Clairambault
