Commit Graph
1230 Commits
Author SHA1 Message Date
Subhra264 c68b80bdce fix: use alert data period for trigger report start time and end time, also report partial send notification error (#4164) 2024-08-09 19:05:59 +08:00
Abhay Padamani c697b5a103 feat: threshold line for dashboard chart (#4087)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced the ability to configure mark lines within dashboard
panels, allowing for more detailed visual representations.
- Added a new structure, `MarkLine`, to support configuration options
for mark lines, enhancing dashboard customization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-08-09 12:40:12 +05:30
Hengfei Yang 9a3b86da1e fix: request timeout after an error happened (#4148) 2024-08-09 12:06:15 +08:00
Hengfei Yang 3e6390ef81 fix: plan error in some case (#4156) 2024-08-08 20:27:17 +08:00
Subhra264 1681c90948 fix: use next cron timestamp after silence period (#4154)
Fixes #4155 


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Improved handling of alert triggers for `Cron` frequency types,
allowing for more accurate scheduling after silence periods.
- **Bug Fixes**
- Enhanced functionality for alert rescheduling, ensuring alerts are
appropriately silenced and triggered based on their defined schedules.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-08-08 17:26:38 +05:30
Hengfei Yang 631bfbc588 feat: return multiple partition when the query have _timestamp (#4152) 2024-08-08 16:08:50 +08:00
cfdc09fcba feat: Logs Traces correlation (#3705)
fix: #2369 
      #3379 
      #4090 
      
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
- Introduced new UI components for managing organization settings with
validation.
  - Added "View Logs" button functionality in the trace sidebar.
- Enhanced span logs management with new selection and viewing features.
  - Added search capabilities for log stream selection in trace details.
- Added new routes for Trace Details and Organization Settings with
access control.

- **Improvements**
- Updated layout and styling for better user experience, including
full-width trace details.
  - Integrated services for fetching trace metadata and log streams.
- Enhanced user feedback mechanisms for actions like copying trace URLs.
- Improved interactivity and responsiveness in span-related components.
- Refined navigation and routing for trace details and organization
settings.
- Improved error handling and state management in log fetching
functionalities.

- **Bug Fixes**
- Adjusted CSS to correct layout issues in trace details and sidebar
views.
- Improved test reliability in UI interaction testing by extending wait
times.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
2024-08-07 20:07:40 +05:30
Hengfei Yang 1afd08c55e chore: add debug logs for router (#4143) 2024-08-07 21:00:03 +08:00
Subhra264andomkark06 89a912e7dd fix: do not allow * in first column of alert and report cron expression and introduce vrl in alerts (#4126)
- Replace first column `*` with the current second in cron expression
while saving report and alerts
- Introduce vrl functions in alerts. The new field `vrl_function` in
alert payload stores the vrl function in the alert in base64 encoded
format.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
- Introduced an optional field for alerts to allow the association of
VRL functions.
- Enhanced scheduling for alerts and reports by dynamically modifying
cron expressions to include the current second.
- Added a VRL function editor in the Scheduled Alert component for
improved query editing, with toggles for enhanced user interactivity.
- Improved query construction with optional parameters and enhanced
error handling for timestamp handling.

- **Bug Fixes**
- Improved robustness of the alert and report scheduling systems with
better handling of cron expressions.
- Enhanced error handling for decoding failures related to VRL functions
in alerts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: omkark06 <omkar@zinclabs.io>
2024-08-07 17:40:45 +05:30
Hengfei Yang 5eb1ab91fe fix: some small issues of alert (#4140) 2024-08-07 18:11:00 +08:00
Shashi Kant a7e29e5991 fix: top-level parenthesized sub-expressions (#4093) 2024-08-07 15:06:52 +08:00
Hengfei Yang 564ac641a8 fix: should exit when backend init failed (#4138) 2024-08-07 14:47:50 +08:00
Hengfei Yang b9dac884b0 fix: search pagination error (#4135) 2024-08-06 23:42:57 +08:00
Hengfei Yang 0e9df3dcbc fix: value api for number values (#4134) 2024-08-06 21:37:21 +08:00
Sai Nikhil 3a2655bc0d fix: #4114 error message when entering incorrect old password (#4132) 2024-08-06 20:57:28 +08:00
Hengfei Yang 78e3389781 fix: rum token deadlock (#4131) 2024-08-06 20:38:44 +08:00
Hengfei Yang 4181ca3b0b fix: rum token invalid when user update (#4122)
When user update we `clean` the `USER_RUM_TOKEN` and in the next we
never set `{DEFAULT_ORG}/{token}` back to `USER_RUM_TOKEN`, so the RUM
ingestion report token not found for root user.

The solution is we don't `clean` the tokens, just `remove` the token
which already invalid. also will add `{DEFAULT_ORG}/{token}` if it is
root user when we fetch data from db.
2024-08-05 22:23:17 -07:00
udevandHengfei Yang f3a6fd4faa fix: add check to avoid root user role change (#4120)
fixes #4115 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced security for user role management by preventing changes to
the role of users with the 'Root' designation during updates.
  
- **Bug Fixes**
- Resolved potential privilege escalation by ensuring only non-'Root'
user roles can be modified.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2024-08-06 10:00:15 +05:30
Hengfei Yang 46c7e067f8 fix: parse index error when using field is null (#4118)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Enhanced data handling to prevent processing of empty values, ensuring
more robust performance.
- Improved control flow to skip unnecessary actions for iterations with
empty results.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-08-05 18:54:49 +05:30
df2488fa44 feat: optimize sql to plan execution (#4100)
Now we use sql to plan to execute the query. 

we split the PhysicalPlan to two stage:
1. Partial plan, it can be distributed will run on all the nodes.
2. Final plan, it can't be distributed will run on the leader querier
and merge the result.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
- Updated to version 0.10.9, introducing enhancements and new
dependencies for improved data processing functionalities.
- Added new Protocol Buffers schemas to define structures for cluster
management and data processing frameworks, enhancing integration
capabilities.

- **Bug Fixes**
  - Improved error handling in configuration to prevent runtime panics.
- Enhanced logic in caching mechanisms for more reliable query
responses.

- **Documentation**
- Added module declarations to improve organization and clarity within
the codebase.

- **Refactor**
- Streamlined execution plans and SQL processing for better performance
and maintainability.

- **Tests**
- Introduced new functions for log ingestion in test suites to enhance
reliability and coverage.

- **Style**
- Made formatting adjustments across various files to improve
readability and consistency.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Neha P <neha@openobserve.ai>
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
2024-08-05 18:38:35 +08:00
Ashish Kolhe 8ec1be9eaa chore: upgrade vrl version (#4105)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
	- Updated critical dependencies to enhance software capabilities.
- Improved data structure management in StreamTable and Geoip
implementations for better performance and clarity.

- **Bug Fixes**
	- Enhanced efficiency of entry removal in the data persistence process.

- **Chores**
- Routine updates to external package versions for improved
functionality and security.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-08-05 09:29:01 +05:30
udevandHengfei Yang 4e8bace735 fix: allow IPv6 addresses in RUM headers (#4111)
fixes #4080

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Improved IP address parsing logic to handle both `IpAddr` and
`SocketAddr`, enhancing versatility with input formats.
- Added a fallback mechanism to default to the loopback address in case
of parsing failure.

- **Bug Fixes**
- Enhanced error handling in IP address parsing to ensure functionality
with invalid input.

- **Tests**
- Introduced a new test function to validate the updated IP
parsing logic against various formats.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2024-08-04 11:15:48 +05:30
Taiming Liu 0a6a7914a5 fix: add req_node_group back to node selection in search (#4113)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced the search functionality to target specific node groups,
potentially improving retrieval speed and relevance of results.
  
- **Bug Fixes**
- Maintained existing error handling for online querier nodes, ensuring
consistent performance during node retrieval.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-08-02 21:44:56 +05:30
Hengfei Yang 233b875ddb feat: add some logs for alert (#4109) 2024-08-02 16:10:10 +08:00
Huaijin caed2c8c75 fix: use config timestamp and error message improve (#4104) 2024-08-01 20:44:35 +08:00
Hengfei Yang d2bfb960dd fix: multi stream search panic (#4101) 2024-08-01 18:14:16 +08:00
Hengfei Yang 8a0dbe2cf4 feat: remove sql_mode, query_context and aggs from search API (#4089) 2024-07-31 11:23:11 +08:00
Hengfei Yang 3802004889 feat: add median support for aggregation (#4088) 2024-07-31 09:48:50 +08:00
Hengfei Yang 03567a33a2 fix: return single partition if this is an aggregation query (#4081)
fixed #4077 
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced the search functionality to support aggregate queries,
allowing for more efficient data handling in search results.
  
- **Bug Fixes**
- Improved response structure for aggregate queries to simplify user
experience.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-29 09:15:40 +05:30
Hengfei Yang 742c259bf7 fix: remove ZO_BLOOM_FILTER_ON_ALL_FIELDS (#4078)
`ZO_BLOOM_FILTER_ON_ALL_FIELDS` will huge increase parquet file size and
memory usage when the fields num over 1000 in a stream.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Simplified configuration for bloom filters by removing the
`bloom_filter_on_all_fields` option.
  
- **Bug Fixes**
- Streamlined logic in the bloom filter configuration handling, ensuring
consistent field processing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-27 12:28:14 -07:00
Hengfei Yang 4eb9fcfa26 fix: added wrong _timestamp with distinct (#4076)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced query handling to recognize aggregate queries that utilize
the `DISTINCT` keyword, improving correctness and efficiency in query
processing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-26 23:57:09 +05:30
Hengfei Yang 2dcd4844f2 fix: panic when the ip_address is invalid (#4068) 2024-07-27 01:21:47 +08:00
Ashish Kolhe 21f3166d12 fix:clean in memory cache references (#4074)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
- Improved disk cache management with targeted removal of specific
result files during the garbage collection process, enhancing memory
usage and overall application performance.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-26 19:00:16 +05:30
Ashish Kolhe 4e78db3538 fix:result caching issue (#4065)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Improved sorting logic for search responses, enhancing clarity and
maintainability.
  
- **Bug Fixes**
- Simplified conditional logic for cache checks, improving efficiency
and reducing complexity.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-25 19:47:24 +05:30
Hengfei Yang 32442a1756 fix: panic when the parentSpanId is null (#4063) 2024-07-25 21:05:02 +08:00
Ashish Kolhe dbdcc92967 fix: caching to consider query fn when caching (#4062)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced query handling with improved hashing logic that incorporates
an optional VRL function for better query processing and caching.
  
- **Bug Fixes**
- Improved control flow and functionality of the query handling process
to ensure more accurate results based on the presence of additional
input.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-25 17:27:26 +05:30
Ashish Kolhe b5b791a355 fix: apply vrl functions over hits as array (#4060) 2024-07-25 15:25:21 +05:30
Ashish Kolhe 9fc95bb6df feat: apply vrl functions over hits as array (#4058)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced search functionality to accommodate dynamic query processing.
	- Improved handling of results based on specific query formats.

- **Bug Fixes**
- Corrected response handling for queries with the `#ResultArray#`
prefix, ensuring accurate results.

- **Refactor**
- Streamlined logic in the search function for better performance and
adaptability to varying input queries.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-25 14:38:46 +05:30
Taiming Liu c8afd43830 fix: alerts attrs created but not saved (#4054) 2024-07-24 21:58:38 -07:00
Ashish Kolhe 5ab6feb251 fix: get querier node panic (#4052)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced error handling in various asynchronous functions to prevent
runtime errors and improve reliability when no online nodes are
available.
  
- **Bug Fixes**
- Improved logging and robustness in handling scenarios where no query
nodes are found, preventing potential panics.

- **Chores**
- Reformatted dependency configuration for better readability without
impacting functionality.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-25 09:10:03 +05:30
Hengfei Yang 8e51607dde fix: count error when limit 0 (#4050) 2024-07-25 00:25:09 +08:00
ktx-abhay 5b0bc20311 feat: vrl function support for dashboard (#4022) 2024-07-24 23:05:48 +08:00
Ashish Kolhe 2e6fc12c98 fix: cacher panic for json deserialisation (#4049)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit


- **New Features**
- Enhanced error handling for cached response data parsing, improving
application stability.
- **Bug Fixes**
- Resolved potential panic situations by implementing a more robust
error management strategy during data parsing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-24 19:56:53 +05:30
Subhra264andomkark06 ba36ed505b feat: add apply_before_flattening option to function StreamOrder (#4001)
Fixes #3994 #4012 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
- Introduced a new `applyBeforeFlattening` toggle in the
`AssociatedStreamFunction.vue` component for stream functions.
- Enhanced ingestion functionality to support transformations before and
after the flattening of data.

- **Bug Fixes**
- Improved JSON flattening and transformation logic in multiple
ingestion functions.

- **Refactor**
- Updated function signatures and variable assignments to incorporate
new transformation logic across various modules and handlers.
- Refactored the `functionsColumns` definition to dynamically include
the `applyBeforeFlattening` column.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: omkark06 <omkar@zinclabs.io>
2024-07-24 14:41:08 +05:30
Hengfei Yang 6ca73b465c feat: add multiple tokio runtime (#4035) 2024-07-24 15:25:46 +08:00
Taiming Liu 4aa09c8bc6 feat: clean nats locks (#4032) 2024-07-24 11:14:24 +08:00
Ashish Kolhe 3d21d8d672 fix: cache historical log records (#4019)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Improved caching logic to enhance the accuracy and relevance of search
results.
- Enhanced conditions for evaluating cached results, reducing the chance
of returning outdated data.

- **Bug Fixes**
- Refined cache expiration criteria to ensure timely and relevant data
delivery.

- **Style**
- Updated variable names for improved readability and clarity in the
caching logic.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-22 11:52:42 +05:30
Ashish Kolhe 866a655aba fix: caching issue (#4017)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Adjusted the default behavior for cache checks to prioritize
descending order results.
- Enhanced the caching mechanism by broadening the time range considered
for cache results.

- **Bug Fixes**
- Improved logic in the caching process to ensure more accurate timing
for cached data.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-07-22 06:20:45 +05:30
Taiming Liu 6e374f1c4c fix: default web url and correct report server endpoint (#4015) 2024-07-20 16:18:01 -07:00
Hengfei Yang 82fc1dfb58 feat: value API without count (#4013) 2024-07-19 20:08:28 +08:00