Commit Graph
682 Commits
Author SHA1 Message Date
Sai Nikhil ea2fff7f43 fix: Stream settings page redesign (#5265)
This PR solves the issue #5216 #5224 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced user interface for the log stream input fields with improved
layout and visibility, including dynamic width adjustments and clearer
labeling.
	- Added a new column to display index size in the log streams table.
- Introduced pagination management for schema fields, allowing for
dynamic updates to the number of rows displayed.
- Added a new prop, `isInSchema`, to enhance component layout based on
schema inclusion.

- **Bug Fixes**
	- Improved error messaging for invalid data retention periods.

- **Documentation**
- Updated language localization for clarity in schema field
descriptions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-09 19:52:27 +05:30
Sai Nikhil f895dc32e2 fix: removal of previous node option in pipelines node (#5142)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced new stream types: "metrics" and "traces" in the Query
component.
	- Added a warning message for unsaved changes in the Pipeline Editor.
- Enhanced user experience with a new informational card in the
Condition component.

- **Improvements**
- Simplified components by removing previous node selection
functionality across multiple components.
	- Streamlined drag-and-drop functionality for better user interactions.
	- Improved error handling and feedback for SQL query validation.

- **Bug Fixes**
	- Reactivated tests for error handling in pipeline functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-09 16:57:42 +05:30
Bhargav 8a439782b0 feat: destination and template movement under management (#5263)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced new routing for "destinations" and "templates" under the
settings section.
	- Added a new `alertList` route, replacing the previous alerts route.

- **User Interface Changes**
- Removed tabbed navigation for alerts, destinations, and templates,
simplifying the layout to direct routing.
- Enhanced styling for icons in the header section for a more polished
appearance.
	- Updated the icon for the "organization" tab.

- **Bug Fixes**
- Standardized notification messages during save operations for better
user feedback.
- Updated functionality of back navigation buttons to emit a cancel
event instead of navigating back.

These updates enhance navigation and improve user experience within the
application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-06 11:12:11 +05:30
Bhargav c686380d97 feat: Initial left navigation changes (#5241)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced layout structure and visual presentation in the MainLayout
component.
- Updated navigation item display properties for improved user
experience.
- Added a tooltip to the Theme Switcher for contextual information on
theme modes.
	- Introduced a new language selection entry in the user interface.
- Improved error handling and user feedback in the log search interface.
- Adjusted layout heights in various components for better visual
presentation.
	- Added a new method for refreshing timezone in the traces component.
	- Improved loading spinner visibility in RUM components.

- **Bug Fixes**
- Corrected localization for "Real User Monitoring" to "RUM" in the
English language file.

- **Refactor**
- Simplified the template structure in the MenuLink component for better
semantic clarity.
- Improved drawer visibility control and responsiveness in the
MainLayout component.
- Enhanced error handling and loading state management in the traces and
logs components.
- Updated height calculations for various components to optimize layout.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-04 18:48:03 +05:30
Omkar Kesarkhane 60104aeb07 fix: Date Time shows incorrect date on UI (#5121)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Improved handling of logs tab lifecycle and error messages for better
user experience.
	- Enhanced responsiveness of table cell styling based on state changes.
- Introduced utility function for validating date and time strings to
enhance error handling.
- Added new enumeration and interface to track UI component states
related to logs and stream exploration.

- **Bug Fixes**
	- Adjusted error message display logic for clarity.
- Refined timezone selection handling to ensure correct initial
settings.

- **Refactor**
- Reorganized component methods for better maintainability and
readability.
- Transitioned table cell class management to a reactive reference for
dynamic updates.
- Minor adjustments to the DateTime component for improved code clarity
and functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-04 16:02:59 +05:30
Subhra264 683182f7b8 fix: don't add \n in email row templates, add trigger time variables in alert template (#5226)
As mentioned in the comments of #4996 

- [x] When using row templates with emails, O2 automatically adds `\n`
at the last of the row in email body. Since, email body is currently
treated as html body, the below row template -
```
<p> This is an event {alert_start_time} </p>
```
generates this email body -
```
This is an event 2024-12-02T11:25:40
\n
This is an event 2024-12-02T11:26:40
\n
This is an event 2024-12-02T11:27:40
\n
...
```

- [x] Adds new variables to represent when the alert got evaluated -
`alert_trigger_time` (unix timestamp in microseconds) and
`alert_trigger_time_str` (formatted, e.g. - `2024-12-02T11:27:40`)
- [x] Update document add `alert_trigger_time` and
`alert_trigger_time_str`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
	- Enhanced alert notifications with improved timestamp handling.
- Updated alert template documentation to include new variables for
better clarity.

- **Bug Fixes**
- Refined error handling in the notification sending process for better
logging and context.

- **Documentation**
- Added documentation for new variables in the alert template section to
assist users in utilizing timestamps effectively.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-12-03 13:14:57 +05:30
ktx-vaidehi 834f6c651b feat: maps-support-for-visualization (#5182)
- #5165 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced drag-and-drop functionality with improved error handling and
user notifications.
- Updated computed properties to provide context-specific hints and
accurate value labels for map data.
- Integrated dependency injection for better access to dashboard panel
data.

- **Bug Fixes**
- Implemented checks to ensure items are added or removed from the
correct target axis during drag-and-drop operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-29 17:11:01 +05:30
BhargavandSai Nikhil e81e1a4e56 fix: undefined call fix (#5176)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced organization selection dropdown for improved user experience.
- Conditional rendering for custom logos and menu items based on user
configuration.
- Improved handling of dynamic stream names in the input fields based on
selected node type.

- **Bug Fixes**
- Improved logic for fetching summary data, ensuring valid organization
identifiers are used.

- **Style**
	- Adjusted styling for consistency across layout components.

- **Documentation**
	- Updated error handling and logging for better troubleshooting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
2024-11-28 15:38:10 +05:30
Bhargav fc9565bf10 fix: org selection and change organization flow (#5140)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced organization selection and user authentication processes in
the login component.
- Improved session management with a new method for token refresh
timing.
- Conditional rendering for error messages and search results in the
logs component.

- **Bug Fixes**
- Robust error handling for user login, ensuring better feedback for
invalid credentials.
	- Improved error handling and user feedback in the logs component.

- **Refactor**
- Updated organization handling logic to support more complex data
structures and improve user context management during login.
- Enhanced logic flow and state management in the logs component for a
better user experience.

- **Documentation**
- Minor formatting changes for improved readability in the login
component template.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-26 10:22:09 +05:30
Taiming LiuandSai Nikhil b011d8a93d feat: allow dynamic stream name in pipeline destinations (#5007)
impl #4968 

## How to use
When using pipeline to configure/modify data ingestion, users can set
the destination `Stream` dynamically based on the records that are being
ingested now.

When configuring the destination stream in pipeline editor, instead of
selecting any existing streams from the dropdown or creating a new
stream, one can use `{}` to wrap _**flattened**_ `field_name`, the value
of such field will be the final destination stream.


Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
2024-11-22 06:19:00 -08:00
Sai Nikhil 860e705504 fix: Pipeline minor improvements (#5120)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced user feedback during function deletions by displaying
associated pipelines or a message when none are linked.
- Improved pipeline management functionality, including state toggling
notifications and refined deletion processes.

- **Bug Fixes**
- Streamlined the fitting view logic for the VueFlow component, ensuring
proper rendering of nodes and edges.

- **Improvements**
- Simplified data filtering and expanded row logic for better user
experience in the pipeline list.
- Adjusted the deletion dialog to accurately reflect the current state
of associated pipelines.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-22 12:28:19 +05:30
3c37c272b5 fix: join & subquery related UI changes on logs page (#5055)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced SQL query handling for multiple streams in logging
functionality.
- Added visual feedback for the "Visualize" button based on selected
streams.
	- Introduced organization management improvements in the login process.
	- New utility function for array comparison added for improved logic.
	- Added a new action to the Vuex store for setting user information.

- **Bug Fixes**
- Improved error handling and display for SQL query construction and
organization data fetching.

- **Documentation**
- Updated comments and structure in various components for better
readability and maintainability.

- **Chores**
- Minor formatting and style adjustments across multiple files for
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: hengfei yang <hengfei.yang@gmail.com>
Co-authored-by: Huaijin <haohuaijin@gmail.com>
2024-11-22 10:12:16 +05:30
Sai Nikhil b370371408 fix: pipeline custom edge fix (#5102)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
  - Introduced a new "promql" tab for metrics in scheduled pipelines.
  - Added a toggle for a VRL function editor.
  - Enhanced edge rendering with a new `CustomEdge` component.
- Compact table view for the pipeline list with a new `dense` attribute.

- **Improvements**
  - Improved validation and error handling for SQL and PromQL queries.
  - Enhanced logic for stream selection and filtering.
- Refined edge type handling and visual representation in the pipeline
editor.
  - Updated initial viewport settings for better user experience.
- Enhanced aggregation settings and dynamic updates to group-by columns.

- **Bug Fixes**
- Fixed row expansion behavior for real-time pipelines in the pipeline
list.

- **Style**
- Adjusted CSS styles for new layouts and functionalities across
components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-21 18:01:46 +05:30
ktx-vaidehiandktx-abhay 806444f2c7 fix: dashboard css & stream select in add panel issue (#5091)
- #5089 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced a new property, `streamResultsType`, to enhance data
handling in the dashboard panel.
- Dynamic button styling based on the application's theme in the Query
Type Selector component.

- **Bug Fixes**
- Improved error handling for field extraction to ensure correct fields
are loaded based on stream type.

- **Chores**
	- Updated CSS for button styling to support dynamic theming.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
2024-11-20 18:34:04 +05:30
ktx-vaidehi 1295daec69 feat: dashboard geojson map (#3023)
- #4825 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced new `MapsRenderer` component for enhanced map
visualizations.
  - Added support for selecting map types in the configuration panel.
  - New dashboard components for managing map queries and data.
  - Added new fields for map configurations, including names and values.

- **Improvements**
  - Enhanced chart selection options, including new map-related types.
  - Updated localization for better support of map functionalities.
  - Improved handling of field aliases in the panel schema.
- Enhanced functionality for adding map names and values within the
dashboard.
  - Refined drag-and-drop functionality for map fields.
- Enhanced data validation and error handling in map-related components.

- **Bug Fixes**
- Improved error handling and data validation in map-related components.

- **Chores**
  - Updated component names and structure for consistency.
  - Enhanced internal methods for better data handling and processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-19 16:22:18 +05:30
Juan Calderon-Perez fcb864356a fix: Simplify recommended setup for Elastic Beats (#5083) 2024-11-18 11:12:36 +08:00
Prabhat Sharmaandomkark06 1ca280e648 feat: Add new CRDs (#5071)
Add scrapeconfigs nad probes CRDs

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced the Kubernetes configuration with dynamic command properties
for improved maintainability.
- Introduced a search input field in the Recommended component for
filtering displayed tabs in real-time.
- Implemented dynamic rendering of tabs based on user input, improving
usability and accessibility.

- **Bug Fixes**
	- Corrected command syntax for installing existing CRDs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: omkark06 <omkar@zinclabs.io>
2024-11-15 12:05:36 +05:30
ktx-vaidehi 8456ae5ecf test: dashboard settings (#4990)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced multiple new test files for dashboard functionalities,
including settings for variables, tabs, and general settings.
- Enhanced testing capabilities with new test cases covering various
scenarios for managing dashboard elements.

- **Improvements**
- Added `data-test` attributes to various UI components across multiple
files, improving testability for automated testing.

- **Bug Fixes**
- Updated import paths for JSON data files to ensure proper referencing
in tests, enhancing overall test reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-13 10:44:13 +05:30
ktx-vaidehi 6bf4f943bd feat: skip auto-refresh if api is pending (#5044)
- #5050 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced refresh functionality in dashboard components to prevent
actions during loading states.
	- Improved control flow for date selection and refresh operations.

- **Bug Fixes**
- Addressed issues with unnecessary refresh calls when panels are
loading.

- **Documentation**
	- Updated method signatures for clarity on new conditional checks.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-12 19:24:33 +05:30
Sai Nikhil d8a62bd908 fix: pipeline issues (#4993)
This PR solves the below fixes
SQL error to be displayed while creating query type source node.
 Function is overlapping when we get. error while creating function.
 All validations need to be fixed
Confirmation message to not display, if user edits but does not change
anything and clicks on back icon
 Pipeline Preview for every node we are getting extra white background
Function added in a pipeline should not be allowed to delete until user
deletes the pipeline
 when user cancels any node creation any selection will be reset


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced user interaction and responsiveness in the pipeline editor
components, including improved handling of node selections and
validations.
- Introduced new tab for "streamPipelines" in the Functions view,
replacing the previous "enrichmentTables" tab.
- Added notifications for missing required fields during function
creation and condition saving.
- Implemented dynamic styling for function nodes based on user
interactions.
- Improved validation logic for cron expressions and SQL queries,
providing immediate feedback on input errors.
- Introduced user selection tracking and tooltips for various node types
to enhance clarity and usability.
- Updated data-test attributes across components for improved testing
capabilities.

- **Bug Fixes**
- Improved error handling and notifications for user inputs,
particularly regarding cron expressions and SQL query validations.
- Enhanced state management and error feedback for user selections
across various components.

- **Documentation**
- Updated internal logic descriptions for better clarity on component
behaviors and user interactions.

- **Style**
- Added tooltips for node types to enhance user experience and
visibility of node details.
- Introduced new styling for node handles to improve maintainability and
consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-12 16:05:58 +05:30
Yashodhan Joshi 736fc1c768 fix: cheery-pick ingestion error logging (#4991) 2024-11-12 05:58:53 +08:00
Taiming LiuandSai Nikhil 7b6dfd5885 fix: delete related pipelines when deleting functions and streams (#4995)
Adds a new endpoint for `/functions` to find a list of associate
pipelines. This list is also used to warn user to remove associated pipelines
before deleting a function.
---------

Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
2024-11-11 09:34:21 -08:00
ktx-vaidehiandktx-abhay e133108831 feat: dashboard charts colors options (#4994)
- #5001 
- #5013 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
	- Enhanced color palette management for dynamic user selection.
	- Improved handling of missing values in chart data processing.

- **Bug Fixes**
- Refined logic for calculating minimum and maximum values in charts to
ensure accuracy.

- **Documentation**
- Updated method signatures and variable types for clarity and better
understanding of functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
2024-11-11 17:25:03 +05:30
Omkar Kesarkhane 6b2864c9aa fix: Added AWS, GCP and Azure Data Sources to recommended tab (#5037)
#5038 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced new tabs for AWS, GCP, and Azure configurations in the
ingestion section.
- Added a new component to display AWS configuration details with
service documentation links.
- Enhanced GCP configuration component with additional documentation
links.
  - Updated Azure configuration component layout and links.

- **Bug Fixes**
- Removed outdated ingestion options for Kinesis Firehose and GCP Logs
from the interface.

- **Refactor**
- Simplified route structure by removing obsolete components and adding
new ones for AWS, GCP, and Azure.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-11 13:29:37 +05:30
Abhay Padamani b067c7d681 fix: dashboard markdown renderer issue (#5035)
- #5036 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced styling for rendered HTML and Markdown content using Tailwind
CSS typography utilities.
- Integrated the `@tailwindcss/typography` plugin for improved text
styling capabilities.

- **Chores**
- Updated project dependencies to include `@tailwindcss/typography` in
development dependencies.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-11 12:06:21 +05:30
ktx-vaidehi 3e87c0fc26 fix: table chart override popup, if override has duplicate name then show error (#5026)
- #5025
2024-11-11 11:43:15 +05:30
Omkar Kesarkhane 0bdbced143 fix: fixed issue on aggregating field with non-existent alias and expand log detail (#5011)
#5010 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced new methods for enhanced log data handling, including
`quoteTableNameDirectly`, `getRegionInfo`, and `addTraceId`.
  
- **Improvements**
- Enhanced error handling and state management for better user
experience.
	- Refined query construction logic to optimize performance.
- Updated list presentations in configuration components for improved
styling.

- **Style**
- Improved formatting and readability in the `JsonPreview`,
`KubernetesConfig`, `LinuxConfig`, and `WindowsConfig` components,
enhancing code maintainability.
	- Added SCSS support for styling in `KubernetesConfig`. 

- **Localization**
	- Added a new label for the search functionality: "Original".
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-07 18:27:43 +05:30
Omkar Kesarkhane 3775997ea8 feat: UI improvements in Query Management UI (#4930)
#4936 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced a new component, **RunningQueries**, for managing and
displaying running queries with enhanced filtering options.
- Added a **RunningQueriesList** component for displaying query data
with pagination and action capabilities.

- **Improvements**
- Enhanced layout and styling of the **QueryList** component for better
user experience.
- Streamlined data handling in **RunningQueriesList** by utilizing props
and emitted events, improving flexibility.

- **Localization**
- Added new localization keys for query metrics, including "Number of
Queries," "Total Exec. Duration," "Total Query Range," and "Search
Type," to improve user interface clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-11-05 13:02:52 +05:30
Subhra264andTaiming Liu 287185ac46 fix: pipeline rbac fixes (#4976)
Fixes #4982 

- Adds migration for pipelines for ofga tuple database.
- Fixes some rbac permission logic for pipelines.

Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
2024-11-04 10:50:20 -08:00
ktx-akshayandktx-vaidehi 0ca6524994 test: dashboard table transpose (#4925)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced test coverage for dashboard UI with new Playwright tests
focusing on the transpose feature and dynamic columns.
- Added `data-test` attributes to various components in the General
Settings for improved testability.

- **Bug Fixes**
- Improved consistency in the GitHub Actions workflow configuration for
UI tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
2024-11-04 16:58:13 +05:30
Chaitanya Sistla 4603b26059 feat: fixing logstash url (#4970) 2024-11-01 23:22:48 +08:00
499f2bfbc7 feat: SearchEventContext for search usage reports (#4934)
impl #4914 

new parameters acceptable for search queries to add additional context
in usage report:
**Dashboard**
`dashboard_id` 
`dashboard_name` 
`dashboard_folder`
`dashboard_folder_id`
`dashboard_folder_name`

**Report**
`report_key`

**Alert**
`alert_key`

Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2024-10-31 05:16:00 -07:00
0ebaf21f33 feat: Pipeline integration v2 (#4732)
This PR implements the new design of **Pipeline** introduced by #4432 

---

**2 types of pipelines differentiates by the source** 

Type I: **Realtime**
- source is a stream (Logs, Metrics, & Traces)
- takes effect when the source stream is being ingested

Type II: **Scheduled** (aka DerivedSteram)
- source is a SQL query
- takes effect based on the given schedule

The new pipeline is represented as a graph consists of different types
of Nodes that are connected by edges.
**Pipeline Data Model**
```rs
pub struct Pipeline {
    pub id: String,
    pub version: i32,
    pub enabled: bool,
    pub org: String,
    pub name: String,
    pub description: String,
    pub source: PipelineSource,
    pub nodes: Vec<Node>,
    pub edges: Vec<Edge>,
}
```
**4 types of node**
I. StreamNode: either used for source node of realtime pipeline or for
destination nodes
II. QueryNode: used for source node of scheduled pipeline
III. FunctionNode: used for executing vrl functions
IV. ConditionNode: used for checking routing conditions

Rules applied to validating a pipeline when it's created:
1. non-empty nodes list
2. non-empty edges list
3. 1st node in nodes list is either StreamNode or QueryNode
4. non-empty `conditions` in all ConditionNode nodes in nodes list
5. all leaf nodes are of type StreamNode
6. In the same branch, unchecked `after_flattened` FunctionNode can't
follow checked
   `after_flattened` checked FunctionNode

Pipeline execution is implemented with the struct `ExecutablePipeline`,
a ready-execute pipeline object cached in memory based on the Pipeline
objects modified on the UI side.
`ExecutablePipeline` object
- builds the relationships among all the nodes based on the edges
- topologically sorts the nodes based on the level to determine which
node to process first
- registers all the vrl function node once

`ExecutablePipeline` object processes ingested records in batch.
Starting from the source node, each connecting edge has a channel
connecting the parent and child node, used for passing the records. This
approach enables batch processing in parallel.

---

**Deprecated Features**
1. previous implementation of pipeline
2. Function x Stream associations
 - Functions can still be added/edited/removed as before
- Needs to create a pipeline in order to apply chosen functions to the
desired stream

The new release when this pr is merged will automatically migrate **old
pipelines** and **Function x Stream associations** to new pipeline
format and can be found in `Stream Pipelines` tab with `Migrated-`
prefix names.
**_Note: Auto generated pipelines are paused by default. Please verify
the pipelines before enabling them_**


Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
2024-10-30 21:53:28 -07:00
Omkar Kesarkhane c4d164c230 feat: added Logstash data source in logs (#4957) 2024-10-30 22:13:36 +08:00
Neha PandSai Nikhil 5306ad899f test: clone alert (#4931)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced alert creation and cloning functionality with improved
validation checks.
- Added support for comprehensive search capabilities in the alert list.
- Introduced functionality to verify user interactions with search
history.

- **Bug Fixes**
	- Improved error handling and user feedback during alert cloning.
	- Enhanced validation for required fields in alert creation.

- **Tests**
- New test cases added for alert management and search history
functionalities.
- Updated existing tests to ensure better coverage and error handling,
including the addition of new scenarios for invalid inputs.

- **Chores**
- Cleaned up the test suite by removing unused code and improving
structure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
2024-10-30 17:53:27 +05:30
Abhay Padamani 273330405c feat: color palette for dashboard chart (#4774)
- #4837 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced a new color configuration option for dashboard panels,
allowing users to customize colors via a new dropdown component.
- Added a utility for managing color palettes and calculating color
values based on data metrics.

- **Enhancements**
- Improved SQL and PromQL data conversion processes to dynamically apply
color schemes and enhance tooltip formatting.
- Enhanced the configuration options for dashboard panels with
additional color settings.

- **Dependencies**
	- Added new dependencies for improved data visualization capabilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-30 15:25:00 +05:30
ktx-vaidehiandcoderabbitai[bot] 618a6f7690 feat: dashboard table override (#4798)
- #4829 
- #4215 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Introduced the `OverrideConfigPopup` component for managing dashboard
override configurations.
- Added the `OverrideConfig` component for enhanced configuration
options in table-type dashboards.
	- Enhanced `PanelConfig` to support override configurations.

- **Improvements**
- Expanded dashboard panel data structure to include `override_config`
for flexible configuration management.
	- Improved unit handling and date formatting in table data conversion.

These updates enhance customization and usability for dashboard
configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-10-29 12:37:08 +05:30
ktx-vaidehi f6c0fc8940 feat: Dashboard Config Option (#4617)
- #4589 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced dashboard panel configuration with new options for line
thickness, label positioning, symbol visibility, and line interpolation.
- Introduced new icons for visual representation of chart options,
including Linear, NoSymbol, Smooth, StepBefore, StepAfter, and
StepMiddle.

- **Bug Fixes**
- Improved error handling and validation for dashboard panel
configurations.

- **Documentation**
- Added new localization entries for enhanced user interface labels
related to dashboard features.

These updates significantly improve the customization and usability of
dashboard panels, providing users with greater flexibility in
visualizing their data.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-29 10:54:33 +05:30
ktx-vaidehi eeecb3f130 feat: if print mode hide unnecessary empty space (#4869)
- #4870

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced table rendering for print mode, ensuring proper styling and
positioning.
- Introduced conditional rendering for dashboard panels specifically in
print scenarios, improving layout adaptability.

- **Bug Fixes**
- Improved responsiveness of the dashboard display based on the
application's print state.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-28 18:10:44 +05:30
ktx-vaidehi cb6ac0ab90 feat: added new options for histogram intervals (#4830)
- #4832
2024-10-28 12:27:10 +05:30
Ashish Kolhe 59e9f73e16 feat: use stream stats for partitioning api (#4901) 2024-10-26 10:43:32 +08:00
Bhargavandktx-vaidehi bf236e34a1 fix: trim before and after space in query editor and vrl function (#4886)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced SQL query handling across multiple components by trimming
whitespace from user inputs, ensuring cleaner and more reliable query
execution.
- Added a `refreshTimezone` method to update grid columns and regenerate
histogram data in the search functionality.

- **Bug Fixes**
- Improved error handling and user feedback for session data retrieval
and search requests.

These updates enhance the overall robustness and user experience within
the application, particularly in handling search queries and alerts.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
2024-10-25 14:18:05 +05:30
Yashodhan JoshiandSai Nikhil 99dfffd71b fix: add prefix partition in add stream section (#4874)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced a new "Prefix partition" option for stream index types,
enhancing user selection.
- Improved validation logic to prevent incompatible index type
selections.

- **Bug Fixes**
- Refined handling of stream input fields and validation to ensure
consistent functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
2024-10-22 11:53:28 +05:30
Sai Nikhil 6d78ff6e98 fix: Prefix partition fix (#4861)
1.This PR fixes #4850 
2.additionally in the custom logo issues I have removed .tif format in
supported formats along with it throws an error if the custom logo text
is more than 100 chars

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced a "Prefix partition" option for enhanced schema
configuration.
- Added character limit enforcement for custom logo text input (max 100
characters).
- Expanded file upload support to include additional image formats (.gif
and .bmp).

- **Bug Fixes**
- Improved error handling for logo updates and deletions with user
notifications.

- **Documentation**
- Streamlined file type descriptions for uploads in the settings
component.

These updates enhance user experience by improving schema management,
input validation, and file upload functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-21 15:45:37 +05:30
Prabhat Sharma 6866a2b753 fix: update cert-manager (#4849)
update cert-manager
2024-10-19 14:52:16 -07:00
Sai Nikhil 3c9cea7302 fix: custom logo issues (#4843)
1. the limit of text is fixed to 100 char

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
	- Introduced input sanitization to enhance text handling and security.
- Expanded file format validation for uploads to include additional
formats (.gif, .bmp, .tif).

- **Improvements**
- Enhanced input handling with a character limit of 100 for custom text.
	- Updated validation messages for file uploads.

- **Type Safety**
- Added TypeScript type annotations for improved code safety and
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-19 14:46:56 +05:30
Bhargav 6714d26d30 fix: moved restore URL params before api calls (#4804)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced a method to restore URL query parameters on component
mount.
	- Enhanced data loading responsiveness based on route changes.
- Improved handling of search history visibility based on query
parameters.

- **Bug Fixes**
- Added validation checks for executing queries to ensure only valid
queries are processed.

- **Style**
- Updated layout and scrollbar styles for a better user interface
experience.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-18 22:00:45 +05:30
Omkar Kesarkhane 9b4668db66 fix: UI freezes on enabling histogram (#4813)
#4806 

Issue replication steps:
Go to logs, turn off histogram, select stream and click run-query after
getting results, turn on histogram.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
	- Introduced functionality for refreshing data based on user settings.
	- Added dynamic SQL query management capabilities.
	- Added a method for generating histogram data skeletons.
	- Updated the region data fetching process for improved structure.

- **Improvements**
- Enhanced error handling with detailed messages and better trace ID
management.
	- Improved pagination management and histogram data handling.
- Updated component structure for better search query handling and
visualizations.
- Enhanced clarity and maintainability of data filtering logic in the
permission table.

- **Minor Adjustments**
	- Various code adjustments for clarity and efficiency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-10-18 14:32:05 +05:30
Prabhat SharmaandTaiming Liu 9ace66ac75 chore: rename zinc to OpenObserve (#4807)
rename zinc to OpenObserve

---------

Co-authored-by: Taiming Liu <90490946+taimingl@users.noreply.github.com>
2024-10-16 10:12:34 -07:00
b53766d76a fix: Dashboard chart UI issues (#4745)
#4256 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced new functions for enhanced chart rendering:
`getLargestLabel` for improved label sizing, `largestStackLabel` for
stacked charts, and `getPieChartRadius` for pie charts.

- **Improvements**
- Enhanced axis label handling for better visual representation in
horizontal bar and stacked charts.
- Updated text width calculation to be more flexible with an optional
font size parameter.
	- Improved clarity in variable naming for better understanding.
- Enhanced readability of the ViewPanel component template and improved
management of the loading state.
- Refined UI test interactions with more descriptive data-test
selectors, enhancing maintainability.
	- Improved handling of large datasets and complex SQL queries in tests.

- **Bug Fixes**
- Enhanced error handling in tests for invalid queries, improving
overall robustness of the application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
2024-10-16 16:59:00 +05:30