#5253
The overall goal of this PR is to migrate alerts data out of `meta`
table into `alerts` table. To accomplish this three main changes are
made. Unfortunately, all three of these changes need to be synchronized
and submitted in the same PR.
1. A new migration script is added to
- Parse alerts JSON currently stored in the `meta` table into Rust
models for alerts. These Rust models are isolated to the migration
script since they represent the schema of alerts as they are stored in
the `meta` table before this migration executes.
- Convert those Rust models into Sea ORM entities for alerts.
- Insert all the Sea ORM entities for alerts into the new `alerts`
table.
2. Create get/put/delete/list methods for accessing the data in the
`alerts` table
- These functions use SeaORM to query and update the database.
- These functions must also convert between SeaORM alert entities and
the application's current data model for alerts (as defined in
`config::meta::alerts`.
- It's entirely expected that the models inside `config::meta::alerts`
will change over time. When this happens in the future, developers will
need to update the transformation functions between SeaORM alert
entities and the models in `config::meta::alerts`. Luckily, the compiler
should help to direct developers to where these transformation functions
need to be updated.
3. Update the old logic in `/src/service/db/alerts.rs` that previously
read and wrote alerts to the `meta` table so that it now reads and
writes alerts to the `alerts` table.
- The methods for accessing the `meta` table included logic for
triggering "cache update events" to the cluster coordinator and for
sending events to the super cluster queue. Since the new logic accesses
the `alerts` table rather than the `meta` table, this logic for emitting
events had to be reproduced.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `ListAlertsParams` struct for flexible querying of
alerts.
- Enhanced `TriggerCondition` struct with new timing parameters.
- Added new error variants for improved error handling in alert
management.
- Created a comprehensive alert management system with CRUD operations.
- Added integration tests for various functionalities related to alerts
and dashboards.
- Introduced a new `Alerts` variant in the `FolderType` enum for better
folder management.
- **Bug Fixes**
- Improved error handling for folder deletion when alerts are present.
- **Documentation**
- Updated documentation for several functions to clarify their scope and
functionality.
- **Chores**
- Added new dependencies to support additional functionality.
- Introduced migration scripts for database schema changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- #4885
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced refresh button functionality in dashboard panels, providing
visual feedback when variable data changes.
- Introduced a new utility function to check if variables are fully
loaded before updates.
- Added computed properties and event emissions to improve user
interaction and responsiveness to variable updates.
- Improved error handling and logging in dashboard components for better
user visibility.
- Introduced new methods for handling user interactions in variable
selection components.
- **Bug Fixes**
- Improved test case flows for dashboard settings and variable settings,
ensuring smoother interactions in automated tests.
- **Documentation**
- Updated tooltips and button states for better user guidance regarding
variable changes.
- **Chores**
- Streamlined test cases by removing redundant steps, enhancing clarity
and efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Kirtan Thakkar <kirtan@kiara.tech>
- #5193
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
- **New Features**
- Introduced PromQL mode handling in the dashboard panel.
- Enhanced metrics page layout with new components:
DateTimePickerDashboard, AutoRefreshInterval, ChartSelection,
CustomHTMLEditor, CustomMarkdownEditor, DashboardQueryBuilder,
DashboardErrorsComponent, PanelSchemaRenderer, PanelSidebar,
RelativeTime, and QueryInspector.
- Added localization support for metrics in the search context.
- **Bug Fixes**
- Improved error handling for loading stream fields and validating panel
configurations.
- Stream type selection visibility now adapts based on dashboard panel
data type.
- Enhanced user interaction by simulating keyboard input for
organization selection.
- Reduced validation requirements for the metrics page.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new suite of Playwright tests for validating enrichment
table functionality.
- Added new functions for enhanced navigation and interaction within the
pipeline UI.
- Added tests to assert the data in every derived stream with condition,
function and only source + derived stream
- Added tests to assert if user clicks on any menu option and cancel on
dialog box, user stays on pipeline page
- Added tests to assert if user clicks on dashboard menu option and
accept on dialog box, user navigates to dashboard page
- **Bug Fixes**
- Improved error handling in tests to ensure accurate feedback during
interactions.
- **Chores**
- Updated the GitHub Actions workflow to include additional integration
tests.
- Removed outdated test files related to function management, Kinesis
ingestion, and reporting functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Removed automated UI tests for managing service accounts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#5378
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Enhanced error handling for invalid stream names during ingestion,
ensuring appropriate logging and response for empty or invalid entries.
- Updated assertions in tests to reflect the correct handling of invalid
bulk payloads by the API, ensuring errors are present for invalid
submissions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Introduced a new end-to-end test for pipeline creation, enabling, and
disabling.
- Utilized parameterization to test various scenarios with different
pipeline names, actions, and expected HTTP status codes.
- Removed the previous end-to-end tests related to user token
management, which included user token retrieval and email verification
processes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Updated the locator for the management menu item to enhance navigation
reliability.
- Streamlined the control flow by removing unnecessary wait times in the
management navigation process.
- **New Features**
- Added methods for managing service accounts, including adding,
validating, and deleting service accounts.
- Introduced a new test suite for validating service account
functionality, covering error handling and successful operations during
creation.
- Added a new method for automatic login based on visibility of the
login option.
- **Improvements**
- Updated the login method to ensure it waits for the correct login page
URL before proceeding.
- Simplified the login process in UI tests by removing unnecessary
parameters and wait times.
- Enhanced test scripts with improved structure and clarity, including
comments for better understanding.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced end-to-end testing for bulk ingestion with comprehensive
coverage.
- Enhanced functionality for managing enrichment tables, including
adding, uploading, and deleting entries.
- **Bug Fixes**
- Improved error handling and user interaction checks in the enrichment
table upload process.
- **Tests**
- Added multiple new test cases for enrichment table functionality,
including upload and error handling scenarios.
- Updated existing test cases for consistency and improved logic.
- **Chores**
- Standardized import statements and improved overall test structure for
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new test case for uploading enrichment tables, enhancing
pipeline functionality validation.
- **Bug Fixes**
- Improved error handling for required fields in existing tests.
- **Tests**
- Organized and structured existing test cases for better execution flow
and added wait times for asynchronous operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR solves the issue #5216#5224
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced user interface for the log stream input fields with improved
layout and visibility, including dynamic width adjustments and clearer
labeling.
- Added a new column to display index size in the log streams table.
- Introduced pagination management for schema fields, allowing for
dynamic updates to the number of rows displayed.
- Added a new prop, `isInSchema`, to enhance component layout based on
schema inclusion.
- **Bug Fixes**
- Improved error messaging for invalid data retention periods.
- **Documentation**
- Updated language localization for clarity in schema field
descriptions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced new stream types: "metrics" and "traces" in the Query
component.
- Added a warning message for unsaved changes in the Pipeline Editor.
- Enhanced user experience with a new informational card in the
Condition component.
- **Improvements**
- Simplified components by removing previous node selection
functionality across multiple components.
- Streamlined drag-and-drop functionality for better user interactions.
- Improved error handling and feedback for SQL query validation.
- **Bug Fixes**
- Reactivated tests for error handling in pipeline functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced new routing for "destinations" and "templates" under the
settings section.
- Added a new `alertList` route, replacing the previous alerts route.
- **User Interface Changes**
- Removed tabbed navigation for alerts, destinations, and templates,
simplifying the layout to direct routing.
- Enhanced styling for icons in the header section for a more polished
appearance.
- Updated the icon for the "organization" tab.
- **Bug Fixes**
- Standardized notification messages during save operations for better
user feedback.
- Updated functionality of back navigation buttons to emit a cancel
event instead of navigating back.
These updates enhance navigation and improve user experience within the
application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Updated locator strings for the management menu item to improve
element selection accuracy.
- Ensured correct targeting of the file upload input for logo uploads.
These changes enhance the reliability of the management page
interactions without altering existing functionalities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
This PR only refactors code and should not result in any behavioral
changes.
This PR follows #5220 and creates separate "HTTP Layer" models for each
our dashboard HTTP API endpoints.
By separating the "HTTP Layer" models from the "Service Layer" dashboard
models we will now (in future PRs) be able to...
- add folder name and folder ID fields to each item returned by the
`ListDashboards` endpoint - Step 3 of #5198
- remove dashboard details from the each item returned by the
`ListDashboards` endpoint - Step 1 of #5196
Unfortunately, this PR does not yet accomplish fully decoupling the
"HTTP Layer" dashboard model from the "DB Layer" dashboard model since
both layers still use the same same JSON de/serialization logic defined
on the structs inside `config/meta/dashboards`. So changes to that model
and the corresponding JSON de/serialization logic could still introduce
silent breaking changes into both the interface with the DB and the HTTP
API. This PR moves us closer to decoupling those models, though future
work will be necessary to fully decouple them.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced new data structures for handling dashboard-related HTTP
requests and responses, enhancing clarity and type safety.
- Added support for creating, updating, and listing dashboards with
structured request and response bodies.
- **Documentation**
- Updated OpenAPI documentation to reflect new schemas for dashboard
operations, improving API clarity and specificity.
- **Refactor**
- Removed outdated dashboard structures and streamlined request handling
logic for improved maintainability.
- Updated integration tests to align with the new dashboard data
structures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced layout structure and visual presentation in the MainLayout
component.
- Updated navigation item display properties for improved user
experience.
- Added a tooltip to the Theme Switcher for contextual information on
theme modes.
- Introduced a new language selection entry in the user interface.
- Improved error handling and user feedback in the log search interface.
- Adjusted layout heights in various components for better visual
presentation.
- Added a new method for refreshing timezone in the traces component.
- Improved loading spinner visibility in RUM components.
- **Bug Fixes**
- Corrected localization for "Real User Monitoring" to "RUM" in the
English language file.
- **Refactor**
- Simplified the template structure in the MenuLink component for better
semantic clarity.
- Improved drawer visibility control and responsiveness in the
MainLayout component.
- Enhanced error handling and loading state management in the traces and
logs components.
- Updated height calculations for various components to optimize layout.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced comprehensive UI testing for the Home, Ingestion, Login,
Logs, Metrics, Traces, Pipelines, Rum, Dashboard, Alerts, Reports, Data,
IAM, and About pages.
- Added functionality to handle organization changes and validations
across multiple pages, including default organization selections.
- New test suite for validating organization change scenarios across
various pages.
- Enhanced login functionality for internal users.
- Introduced new classes and methods for the Metrics, Streams, Rum, and
Alerts pages to support UI testing.
- Added new classes for the Data and IAM pages, expanding testing
capabilities.
- Enhanced Management page with navigation and validation capabilities.
- New methods for validating page URLs and content across various pages.
- **Bug Fixes**
- Improved validation methods to ensure proper navigation and content
checks on various pages.
- **Documentation**
- Updated testing framework to include new test cases and improved
structure for organization change tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced `LogsPage` with improved date and time selection, timezone
management, and clipboard operations.
- Introduced `ManagementPage` for UI testing, allowing logo updates and
management functionalities.
- Added new tests for time zone changes and streamlined test scenarios
on the logs page.
- Introduced a suite of tests for joining logs, validating various join
scenarios.
- **Bug Fixes**
- Updated confirm button interaction in `ReportsPage` to ensure
visibility before clicking.
- **Documentation**
- Improved clarity and purpose of test scenarios through restructuring
and simplification.
- **Chores**
- Added `logoManagement.spec.js` and `join.spec.js` to the Playwright UI
integration tests for expanded coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced dashboard management with new methods for retrieving
properties like ID, owner, title, and description.
- Added functionality for creating, updating, listing, and deleting
dashboards through a new service module.
- Introduced a new `dashboards` table in the database, with migrations
for creation and population.
- **Bug Fixes**
- Improved error handling for dashboard retrieval and management
operations, providing clearer responses for various conditions.
- **Documentation**
- Updated documentation to reflect new methods and error handling
improvements.
- **Refactor**
- Streamlined the dashboard handling logic in service and database
modules, reducing complexity and enhancing modularity.
- Enhanced error handling logic in dashboard management functions for
better clarity and control flow.
- Refined the dashboard retrieval process to improve clarity in control
flow and error handling.
- Updated the logic for dashboard operations to simplify version
handling and reduce redundancy.
- Improved the overall structure and organization of the dashboard
management code.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The previous mechanism for generating hashes for dashboard models
depended on hashing the JSON representation of the dashboard that was
stored in the `meta` table.
As part of the migration of dashboards out of the `meta` table and into
a dedicated `dashboards` table, this ☝️ presented some problems:
Specifically when we want to translate from the Dashboard ORM model to a
`core::meta::dashboards::Dashboard` model, we need to generate the hash
at the time of translation since `hash` is a field on
`core::meta::dashboards::Dashboard`. We can't simply generate a hash
from the JSON stored in the `dashboards.data` column because this JSON
no longer contains fields for `title`, `owner`, `description`, and
`role`. Those fields are now stored in DB columns, and changes to those
columns in the DB columns would *not* result in changes to the hash on
the JSON blob.
Therefore, rather than hashing the JSON blob, this PR implements hash
functions on the Rust types for the dashboards. Therefore changes to
fields in those Rust structs will result in changes to the hash values,
but changes to the way in which the data is stored in the DB will not
affect the hash values.
⚠️ Note that this PR is changing the hash function on the dashboard
models, so hashes generated for the same model before and after this
deployment will not be the same. My understanding is that this shouldn't
be an issue since hashes are just used as a way of checking on the UI if
the dashboard model has been updated to avoid overwriting the changes of
another user. I believe that this change to the dashboard function might
just have the effect (around the time of deployment) where it would
appear to the UI that the dashboard has been updated when, in fact, it
has not.
https://github.com/openobserve/o2-enterprise/actions/runs/11979723563
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new type alias `OrdF64` for enhanced precision in
floating-point operations.
- Added the `Hash` trait to multiple dashboard-related structs and
enums, enabling their use in hash-based collections.
- **Bug Fixes**
- Improved error handling and assertion clarity in API testing for
folder and dashboard operations.
- **Chores**
- Standardized hashing mechanism for dashboard objects in database
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This is the first PR for #5068 .
We need to move `dashboards` into the `config` crate so that DB access
functions in the `infra` crate can return types defined in the
`dashboards` module.
When the `dashboards` module is in the `openobserve` crate we cannot use
them in the `infra` crate since that would introduce a circular
dependency.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced a new function to move dashboards, enhancing dashboard
management capabilities.
- Added a new `SendReport` trait for improved report handling and
modularity.
- Enhanced error handling across various dashboard and report management
functions.
- **Improvements**
- Updated import paths for better clarity and organization, particularly
for dashboard-related modules.
- Improved timestamp handling for alerts to ensure accurate tracking of
updates.
- Enhanced OpenAPI documentation to reflect updated schema references
for dashboards.
- Expanded development dependencies to support improved functionality.
- Enhanced the logic for handling report triggers based on frequency
types.
- **Bug Fixes**
- Refined error handling in dashboard creation and deletion functions to
provide clearer responses.
- Standardized error handling across report management functions for
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new JSON file for SQL join query test cases, enhancing
the variety and coverage of tests.
- Added multiple new test cases for various SQL join operations,
improving testing capabilities.
- **Bug Fixes**
- Improved error handling in data ingestion steps to ensure successful
test execution.
- **Refactor**
- Transitioned from hardcoded test data to a parameterized approach for
better maintainability and flexibility in testing.
- Updated method signature to streamline test parameters.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a reactive variable for organization data, improving state
management.
- Updated functionality to accept an object for organization data,
enhancing input validation and handling.
- **Bug Fixes**
- Enhanced error handling for better debugging without altering existing
functionality.
- **Refactor**
- Improved readability and maintainability of the layout component with
formatting adjustments and refined logic.
- Updated test cases to reflect changes in URL structure for
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced Playwright test suite for dashboard functionalities,
including comprehensive tests for VRL functions, dashboard settings, and
data validation.
- Added new test cases for creating, deleting, and updating dashboards,
as well as applying dynamic filters.
- Introduced automation for the login process and improved the
validation of user inputs in dashboard tests.
- **Bug Fixes**
- Improved error handling and visibility checks in dashboard tests to
ensure user inputs are validated correctly.
- **Chores**
- Updated import statements across multiple test files to utilize a
local module for better fixture management.
- Added a new test file for VRL visualization tests to expand
integration coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `PipelinePage` class for enhanced UI testing of the
pipeline management interface.
- Added a comprehensive suite of Playwright tests to validate pipeline
creation, editing, and deletion functionalities.
- **Bug Fixes**
- Corrected the test filename in the workflow configuration to align
with naming conventions.
- **Chores**
- Removed the outdated `pipeline.spec.js` test file to streamline the
testing suite.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added support for code coverage reporting with the integration of
`vite-plugin-istanbul`.
- Introduced a new configuration file for code coverage settings.
- **Improvements**
- Updated project to use ECMAScript modules for better compatibility.
- Enhanced UI test coverage with additional test cases and improved
error handling.
- Refactored test suites to utilize a custom module for better
organization and maintainability.
- Improved the login process and dashboard loading in test cases for
better reliability.
- **Bug Fixes**
- Resolved issues related to import statements affecting test execution.
- **Documentation**
- Updated `.gitignore` to exclude coverage directories from version
control.
These updates enhance the functionality, maintainability, and
reliability of the application and its testing framework.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced multiple new test files for dashboard functionalities,
including settings for variables, tabs, and general settings.
- Enhanced testing capabilities with new test cases covering various
scenarios for managing dashboard elements.
- **Improvements**
- Added `data-test` attributes to various UI components across multiple
files, improving testability for automated testing.
- **Bug Fixes**
- Updated import paths for JSON data files to ensure proper referencing
in tests, enhancing overall test reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR solves the below fixes
SQL error to be displayed while creating query type source node.
Function is overlapping when we get. error while creating function.
All validations need to be fixed
Confirmation message to not display, if user edits but does not change
anything and clicks on back icon
Pipeline Preview for every node we are getting extra white background
Function added in a pipeline should not be allowed to delete until user
deletes the pipeline
when user cancels any node creation any selection will be reset
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced user interaction and responsiveness in the pipeline editor
components, including improved handling of node selections and
validations.
- Introduced new tab for "streamPipelines" in the Functions view,
replacing the previous "enrichmentTables" tab.
- Added notifications for missing required fields during function
creation and condition saving.
- Implemented dynamic styling for function nodes based on user
interactions.
- Improved validation logic for cron expressions and SQL queries,
providing immediate feedback on input errors.
- Introduced user selection tracking and tooltips for various node types
to enhance clarity and usability.
- Updated data-test attributes across components for improved testing
capabilities.
- **Bug Fixes**
- Improved error handling and notifications for user inputs,
particularly regarding cron expressions and SQL query validations.
- Enhanced state management and error feedback for user selections
across various components.
- **Documentation**
- Updated internal logic descriptions for better clarity on component
behaviors and user interactions.
- **Style**
- Added tooltips for node types to enhance user experience and
visibility of node details.
- Introduced new styling for node handles to improve maintainability and
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
test case report e2e
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added a new test suite for report functionalities, including creating
and deleting dashboards and reports.
- Introduced a new `IngestionPage` class for handling data ingestion
through an API.
- Enhanced `DashboardPage` with improved dashboard creation and deletion
processes.
- Updated `ReportsPage` to support report creation and verification with
new methods.
- **Bug Fixes**
- Improved element selection methods for better reliability in the login
process.
- **Documentation**
- Streamlined import statements and improved code organization for
better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced test coverage for dashboard UI with new Playwright tests
focusing on the transpose feature and dynamic columns.
- Added `data-test` attributes to various components in the General
Settings for improved testability.
- **Bug Fixes**
- Improved consistency in the GitHub Actions workflow configuration for
UI tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
This PR implements the new design of **Pipeline** introduced by #4432
---
**2 types of pipelines differentiates by the source**
Type I: **Realtime**
- source is a stream (Logs, Metrics, & Traces)
- takes effect when the source stream is being ingested
Type II: **Scheduled** (aka DerivedSteram)
- source is a SQL query
- takes effect based on the given schedule
The new pipeline is represented as a graph consists of different types
of Nodes that are connected by edges.
**Pipeline Data Model**
```rs
pub struct Pipeline {
pub id: String,
pub version: i32,
pub enabled: bool,
pub org: String,
pub name: String,
pub description: String,
pub source: PipelineSource,
pub nodes: Vec<Node>,
pub edges: Vec<Edge>,
}
```
**4 types of node**
I. StreamNode: either used for source node of realtime pipeline or for
destination nodes
II. QueryNode: used for source node of scheduled pipeline
III. FunctionNode: used for executing vrl functions
IV. ConditionNode: used for checking routing conditions
Rules applied to validating a pipeline when it's created:
1. non-empty nodes list
2. non-empty edges list
3. 1st node in nodes list is either StreamNode or QueryNode
4. non-empty `conditions` in all ConditionNode nodes in nodes list
5. all leaf nodes are of type StreamNode
6. In the same branch, unchecked `after_flattened` FunctionNode can't
follow checked
`after_flattened` checked FunctionNode
Pipeline execution is implemented with the struct `ExecutablePipeline`,
a ready-execute pipeline object cached in memory based on the Pipeline
objects modified on the UI side.
`ExecutablePipeline` object
- builds the relationships among all the nodes based on the edges
- topologically sorts the nodes based on the level to determine which
node to process first
- registers all the vrl function node once
`ExecutablePipeline` object processes ingested records in batch.
Starting from the source node, each connecting edge has a channel
connecting the parent and child node, used for passing the records. This
approach enables batch processing in parallel.
---
**Deprecated Features**
1. previous implementation of pipeline
2. Function x Stream associations
- Functions can still be added/edited/removed as before
- Needs to create a pipeline in order to apply chosen functions to the
desired stream
The new release when this pr is merged will automatically migrate **old
pipelines** and **Function x Stream associations** to new pipeline
format and can be found in `Stream Pipelines` tab with `Migrated-`
prefix names.
**_Note: Auto generated pipelines are paused by default. Please verify
the pipelines before enabling them_**
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced alert creation and cloning functionality with improved
validation checks.
- Added support for comprehensive search capabilities in the alert list.
- Introduced functionality to verify user interactions with search
history.
- **Bug Fixes**
- Improved error handling and user feedback during alert cloning.
- Enhanced validation for required fields in alert creation.
- **Tests**
- New test cases added for alert management and search history
functionalities.
- Updated existing tests to ensure better coverage and error handling,
including the addition of new scenarios for invalid inputs.
- **Chores**
- Cleaned up the test suite by removing unused code and improving
structure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
New integration tests for cargo to test alert scheduling
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Introduced new asynchronous integration tests for multi-range alert
handling.
- Enhanced existing alert tests with improved assertions for creation
and deletion processes.
- Updated alert trigger conditions to ensure comprehensive validation of
alert functionalities.
- Added tests for alert behavior during retries after destination
failures and evaluation errors.
- Updated tests for the `SearchHistoryQueryBuilder` to reflect new SQL
query structure tailored to search events.
- Modified tests for `RedirectResponse` to accommodate multiple query
parameter orders.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Introduced a new parameterized test suite for validating SQL queries
involving `cast` and `try_cast` operations.
- Enhanced testing framework to ensure API reliability when handling
various SQL casting scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Introduced a series of end-to-end tests for SQL queries with various
join types to ensure functionality against the API.
- Added tests for `LIKE`, inner join, `IN`, left join, right join, and
full join SQL queries.
- Implemented parameterized testing for improved coverage and
maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added end-to-end tests for creating alerts based on SQL and float
values.
- Introduced a page object model for unflattened logs to streamline UI
interactions.
- Implemented a function to toggle quick mode in the test suite.
- **Bug Fixes**
- Enhanced error handling and validation in alert tests.
- Improved reliability of tests by ensuring UI elements are visible
before interactions.
- **Refactor**
- Refactored Playwright tests to utilize the `UnflattenedPage` object
for better maintainability.
- Consolidated parameters in fetch requests for clarity.
- **Chores**
- Removed redundant tests from the alert test suite to improve clarity
and maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `UnflattenedPage` class for structured UI
interactions in tests.
- Added functionality to toggle quick mode in the testing framework.
- **Bug Fixes**
- Improved error handling and visibility assertions in various test
cases.
- **Documentation**
- Enhanced comments and code formatting for better readability and
maintainability.
- **Refactor**
- Restructured test cases to utilize the `UnflattenedPage` object,
promoting a page object model for improved code organization.
- Updated test configurations for better clarity and consistency.
- **Chores**
- Removed several outdated test files related to dashboard and logging
functionalities, streamlining the test suite.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced UI testing for time precision across multiple pages,
including Logs, Traces, Reports, Dashboards, Alerts, and Metrics.
- Added tests for both relative and absolute seconds to ensure accurate
time settings.
- **Bug Fixes**
- Updated existing tests to improve verification of time settings on the
Metrics and RUM pages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced new UI testing classes for various application pages,
including `AlertsPage`, `DashboardPage`, `LogsPage`, `ReportsPage`,
`TracesPage`, and `LoginPage`.
- Added a new test suite focusing on time-related features across
multiple pages.
- **Bug Fixes**
- Improved consistency in the GitHub Actions workflow for UI integration
tests.
- **Documentation**
- Enhanced organization of locators and methods for better usability in
UI testing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Neha P <144308844+neha00290@users.noreply.github.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a method to restore URL query parameters on component
mount.
- Enhanced data loading responsiveness based on route changes.
- Improved handling of search history visibility based on query
parameters.
- **Bug Fixes**
- Added validation checks for executing queries to ensure only valid
queries are processed.
- **Style**
- Updated layout and scrollbar styles for a better user interface
experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#4256
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced new functions for enhanced chart rendering:
`getLargestLabel` for improved label sizing, `largestStackLabel` for
stacked charts, and `getPieChartRadius` for pie charts.
- **Improvements**
- Enhanced axis label handling for better visual representation in
horizontal bar and stacked charts.
- Updated text width calculation to be more flexible with an optional
font size parameter.
- Improved clarity in variable naming for better understanding.
- Enhanced readability of the ViewPanel component template and improved
management of the loading state.
- Refined UI test interactions with more descriptive data-test
selectors, enhancing maintainability.
- Improved handling of large datasets and complex SQL queries in tests.
- **Bug Fixes**
- Enhanced error handling in tests for invalid queries, improving
overall robustness of the application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Patrick Sullivan
Hengfei Yang
ktx-vaidehiandKirtan Thakkar
Abhay Padamani
Neha P
Shyam Panjiyar
Loakesh Indiranandhengfei yang
Taiming Liu
Sai Nikhil
Bhargav
ktx-akshayandktx-vaidehi
Omkar Kesarkhane
Subhra264
Ashish Kolhe
Huaijin
Prabhat Sharma
