Commit Graph
2024 Commits
Author SHA1 Message Date
Hengfei Yang 5e2ea1aaed perf: improve search limit logic (#3638)
- [x] If the search request size is 0 and in full sql mod and track
total hits, will not apply default limit `1000`.
- [x] change consistent hashing vnode default value to `16`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Improvements**
- Enhanced performance and accuracy of data aggregation by updating
internal logic and default values.
- Improved configuration flexibility by allowing custom `history`
values.

These changes aim to provide a more robust and configurable experience
for end-users.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-31 10:56:50 +05:30
Bhargav 05d0d23fbe fix: histogram for aggregation query (#3639)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Refactor**
- Updated internal logic for handling `aggs` in search results to
improve performance and reliability.


<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-31 10:15:02 +05:30
ktx-vaidehiandktx-abhay 60a64fcc07 feat: Multi Select Variable (#3372)
- #3460 
- #2015 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
  - Added multi-select functionality for variables in dashboards.
  - Introduced a custom value selector component for dashboard settings.
  - Updated the app to use Quasar's fullscreen module.

- **Enhancements**
  - Improved query verification process in DashboardQueryEditor.
- Enhanced filtering logic for variable placeholders and values in
queries.
  - Added "IN" operator support for dashboard queries.

- **Bug Fixes**
- Fixed issues with handling and displaying selected values in various
components.

- **Performance Improvements**
- Increased size parameter for dashboard panel data loading from 10 to
100.

- **Localization**
- Added new localization key: "multiSelect" for multiple selection
support.

- **UI/UX**
  - Updated styles and key bindings in DrilldownPopUp.
- Enhanced display logic for selected values in
VariableQueryValueSelector.

- **Dependencies**
  - Updated Quasar package from version ^2.7.5 to ^2.15.4.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
2024-05-30 21:15:05 +05:30
ktx-vaidehi a4a48d6d53 fix: Auto Complete promql legend on select not getting proper value with { (#3523)
When a user directly selects anything from the dropdown, it incorrectly
puts the value in the Legend config in dashboard's add panel config for
PromQL.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved input handling in the configuration panel to ensure proper
formatting of values.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-30 20:47:57 +05:30
ktx-vaidehi 523ca46ae6 feat: dashboard variables adding options (#3543)
- Multiple filters for the same field.
- Added p99, p95, p90 and p50 as an option on y-axis
- In Drilldown popup will now get variables value of current dashboard

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added new percentile aggregation options (`p50`, `p90`, `p95`, `p99`)
across various dashboard components.
- Introduced `variablesData` prop to enhance data handling in several
components.

- **Improvements**
- Enhanced error handling and asynchronous operations in dashboard panel
functionalities.
- Improved readability and maintainability of aggregation functions in
query components.

- **Localization**
- Added new keys for percentiles (`p50`, `p90`, `p95`, `p99`) in the
English language file.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-30 20:26:36 +05:30
ktx-abhayandktx-vaidehi 3528c26823 fix: dashboard stacked chart sorting issue (#3517)
- #3499 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Improved handling and sorting for "h-stacked" and "h-bar" chart types.
- Added sorting functionality for stacked charts based on the y-axis
field when not a time series chart.

- **Improvements**
- Enhanced data processing performance with asynchronous operations for
data conversion.

- **Bug Fixes**
- Corrected data inversion issues for "h-stacked" and "h-bar" chart
types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
2024-05-30 20:07:07 +05:30
Hengfei Yang 5cabf55ad2 fix: change the log level for move file (#3636) 2024-05-30 22:12:27 +08:00
Hengfei Yang 950e14c54d perf: improve file move on ingester (#3622) 2024-05-30 20:55:16 +08:00
Bhargav b6899a7bdb fix: openapi menu changes (#3634) 2024-05-30 17:31:21 +05:30
Bhargav 755c0eb225 fix: NaN issue on pagination message (#3632)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Fixed an issue where `totalCount` and `endCount` could be `NaN` by
setting them to 0 if they are `NaN`.

- **Improvements**
- Enhanced the `DateTime` component to initialize `startTime` and
`endTime` based on provided values or default to the current time with
adjustments.

- **Refactor**
- Simplified the `logs` plugin UI by removing the `<Suspense>` wrapper
around the `<search-bar>` component.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-30 16:37:15 +05:30
Hengfei Yang 569bb121ce feat: add new node role flatten_compactor (#3589) 2024-05-30 18:34:41 +08:00
Hengfei Yang 5a8c30e8a1 fix: select case when report field not found (#3633) 2024-05-30 17:49:55 +08:00
veth 1f3036201a chore: fix some comments (#3629) 2024-05-30 17:41:08 +08:00
Bhargav 98540b718d fix: pagination and show message for aggregation query (#3628)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Introduced a new tracking mechanism for total counts and aggregation
status in logs.
  - Enhanced pagination logic to account for aggregation presence.

- **Improvements**
  - Refined the handling of start and end times in logs.
  - Improved the logic for displaying histograms and aggregations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-30 13:02:24 +05:30
Hengfei Yang 7d6b983955 fix: reload .env (#3625) 2024-05-30 09:48:27 +08:00
Ankur SrivastavaandHengfei Yang 8f3df9e7a1 feat: support reloading config vars using an API (#3623)
The new endpoint `/config/reload` will refresh the
environment variables.

There are a bunch of env-vars which will not be reloaded
for e.g. HTTP_PORT etc. these will need a restart.

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2024-05-30 08:23:27 +08:00
Bhargav d502edce96 fix: histogram error sql mode (#3608)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Enhanced SQL query manipulation and result processing in the logs
functionality.
  - Improved handling of aggregation flags in the logs.

- **Bug Fixes**
- Adjusted logic for calculating total query results to ensure accurate
data representation.

- **Tests**
- Added UI tests for resetting filters, refreshing the page, and
interacting with elements in SQL mode.

- **Chores**
- Commented out `loadLogsData()` function call in the logs module to
optimize performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-29 16:23:27 +05:30
Omkar Kesarkhane ba10524050 fix: prefetching monaco editor (#3617)
Prefetched Monao Editor when user hover's over the left side menubar for
first time.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
  - Added functionality to expand the menu on mouseover.
  - Introduced script prefetching for improved performance.

- **Enhancements**
- Wrapped the `<search-bar>` component with `<Suspense>` for better
loading experience.

- **Configuration**
- Updated asset naming strategy to include versioning for specific
assets.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-29 14:29:06 +05:30
Neha P 2067e561d5 test: e2e-sanity -testcases (#3616)
- Verify pagination displayed for schema 
- Verify pagination displayed when histogram is off and user clicks on
log result and closes the result
- Verify if pagination is displayed when SQL is on and user clicks on
log result and closes the result
- Verify if results displayed when SQL+histogram is on and then stream
is selected

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
  - Enhanced login process to improve user authentication flow.
- Added pagination functionality for better navigation through results
in various scenarios.
- Improved display of results when SQL and histogram features are used
together.
- Enhanced user interface for selecting streams and displaying
corresponding results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-29 12:28:53 +05:30
Hengfei Yang 26c81f4af1 fix: allow add older data with schema change (#3610) 2024-05-29 12:40:41 +08:00
Taiming Liu fc05d06c9a refactor: rename match_all & match_all_indexed (#3613)
match_all                  renamed to match_all_raw
match_all_indexed   renamed to match_all

same pattern applied to *_ignore_case functions

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Improved clarity in function names for better understanding and
usability.

- **Tests**
- Updated SQL queries in test cases to reflect changes in function
names, ensuring accurate test results.

- **Chores**
- Adjusted logging statements to provide better insights during
debugging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-28 14:04:36 -07:00
Hengfei Yang 83305a2d44 feat: add histogram interval for search api (#3607) 2024-05-28 17:16:20 +08:00
ktx-abhay 1b88e3cf8a fix: dashboard view panel refresh issue (#3606)
- #3605 

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved data conversion in dashboard panels for more consistent
behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
v0.10.7-rc2
2024-05-28 10:57:15 +05:30
Hengfei Yang c830fa6532 perf: use parquet file static cache (#3604) 2024-05-28 10:43:24 +08:00
Ashish Kolhe b3a3edcd34 fix: ofga check versions before acquiring lock (#3603)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Improvements**
- Enhanced initialization process to prevent redundant cluster lock
creation when the existing OFGA model version matches the fetched
version. This improvement reduces unnecessary operations and improves
system efficiency.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
v0.10.7-rc1
2024-05-27 21:17:10 +05:30
Ankur Srivastava 28cf5b0776 feat: integrate embedded reportserver (#3583)
There are three cases to consider:

if ZO_ENABLE_EMBEDDED_REPORT_SERVER=true

If running a binary, alertmanager node will spawn an embedded server
running at localhost:5082.
All the requests from alert-manager will go to localhost:5082 for pdf
generation related tasks.

For single node k8s deployment, as this is running in a container there
is no chromium installed, so even if you set the variable to true, the
report server will panic due to missing dependencies. For this situation
helm chart will spawn a report-server instance which can be used.

Same for cluster deployment.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
  - Introduced a new report server for generating and sending reports.
  - Added health check and report sending endpoints.

- **Improvements**
  - Enhanced logging for report server operations.
- Configurable option to skip TLS verification for report server
connections.

- **Bug Fixes**
  - Fixed formatting issue in HTTP status request handler.

- **Chores**
- Updated dependencies and configurations for the new report server
module.

- **Documentation**
- Added new structures and functions for email configuration, report
generation, and server operations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
v0.10.6
2024-05-27 11:19:50 +02:00
Omkar Kesarkhane fa0233a61c fix: evoked updateQueryValue on loading of parser (#3600)
fix #3598 
SQL query was getting parsed before loading of node-sql-parser

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
  - Improved initialization process for dashboard query editor.
- Enhanced query update mechanism with conditional logic based on
context.

- **Performance Improvements**
  - Optimized the loading process for dashboard queries.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2024-05-27 13:40:35 +05:30
Hengfei Yang 470714616b fix: search partition with base64 (#3597)
fix #3557
2024-05-26 17:00:44 +08:00
abe6206361 fix: logs page histogram optimisation (#3586)
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: omkark06 <omkar@zinclabs.io>
2024-05-25 18:57:48 +05:30
Ashish Kolhe a000dad31b fix: get auth headers (#3578) 2024-05-25 03:17:57 -07:00
Omkar Kesarkhane aa0e69f072 fix: fixed rrweb player import (#3594)
Fixed rrweb player import in RUM session VideoPlayer
2024-05-25 10:30:12 +05:30
Taiming LiuandNeha P 37ebd33637 fix: _around api for query with limit (#3590)
fix #3587 

<img width="1286" alt="image"
src="https://github.com/openobserve/openobserve/assets/90490946/38ed9e95-6569-4015-a156-6040b0d3fc7c">

---------

Co-authored-by: Neha P <neha@openobserve.ai>
2024-05-24 16:50:35 -07:00
Omkar Kesarkhane fd9cafde0d fix: FE optimization issues (#3593)
#3591
2024-05-25 00:08:54 +05:30
Omkar Kesarkhane 402d450321 chore: Frontend Optimization (#3532)
Lazy loaded packages
Lazy loaded Components
2024-05-24 21:02:06 +05:30
Subhra264 bd07961a1f fix: sso member roles for an org should not be changeable via UI (#3536) 2024-05-24 10:45:28 +05:30
Hengfei Yangandoasisk 2fbec75612 perf: testing optimise aggeration query (#3582)
add a new test environment:
```
ZO_DATAFUSION_PARQUET_STAT_DISABLE_FOR_AGGS=false
```

Set to `true` will skip collect stat for parquet files, then it will
faster.

And, i am thinking the final solution should be cache parquet
statistics, or we can optimize datafusion let it only collect stat for
registered schema fields.

---------

Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
2024-05-24 10:21:27 +05:30
Taiming LiuandBhargav 70fde3ac67 fix: _value api missing "clusters" (#3580)
add the missing clusters parameters to _value api calls

---------

Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
2024-05-23 12:14:29 -07:00
Omkar Kesarkhane ba42221c9b fix: fields getting duplicated in user defined schema (#3554)
fix #3522 
#3568
2024-05-23 22:26:52 +05:30
Ankur Srivastava e4f4389974 feat: support build-type in configuration (#3581)
![image](https://github.com/openobserve/openobserve/assets/1695056/35b2adcf-b461-4471-8962-0bd0410dca31)


![image](https://github.com/openobserve/openobserve/assets/1695056/118726c1-b276-4cae-8689-cb21ef3ae53c)
2024-05-23 18:20:18 +02:00
Subhra264 6936b32bb3 fix: map_group_to_roles custome roles should be scoped under org in rbac (#3546) 2024-05-23 20:44:36 +05:30
Hengfei Yang 9918e24623 feat: let consistent hash vnode num become an evironment (#3577) 2024-05-23 22:31:50 +08:00
Neha P 4ccb02c081 test: e2e-reports-testcases (#3571)
- Verify if user is able to create and delete Once - type report
- Verify if user is able to create and delete Hourly - type report
- Verify if user is able to create and delete weekly- type report
- Verify if user is able to create and delete monthly - type report
- Verify if user is able to create a new dashboards for all the reports
and delete it
- Verify if user is able to delete all the reports after creating
2024-05-23 19:12:57 +05:30
Bhargav ab3891d144 fix: count mismatch (#3576) 2024-05-23 18:29:50 +05:30
Hengfei Yang 0f5929093a perf: optimize scan_files (#3575) 2024-05-23 19:00:27 +08:00
Subhra264 fca47ef225 fix: add dynamic variables to report and allow v1, v2 dashboards in reports (#3574)
- [x] Add dynamic variables
- [x] Allow v1, v2 dashboards in reports
2024-05-23 15:48:04 +05:30
Omkar Kesarkhane 00cb1a168d fix: report UI issues (#3572)
fix #3558 

1. Extra star appearing in cron field on edit
2. Editing an existing report gives error but works after reload
3. Change error message when saving report with same name
2024-05-23 13:49:55 +05:30
Subhra264 41d9eefc1f feat: use a separate chrome server for report (#3563)
Fixes #3355 

User must use a separate chrome server if `ZO_CHROME_PATH` is not
specified. `ZO_REPORT_SERVER_URL` is introduced to specify the report
server url.
2024-05-23 13:14:35 +05:30
Taiming LiuandHengfei Yang a83c46c656 feat: cluster selection in logs (#3559)
- Add cluster selection in logs search queries for additional filtering
based on region/clusters
- add org/stream info in ingestion http handler error logging

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2024-05-22 07:33:34 -07:00
Bhargav 3cf429a85b fix: su total took for two api endpoints and toLocaleString fix (#3570) 2024-05-22 19:44:04 +05:30
dependabot[bot] af00810f00 chore(deps): bump eifinger/setup-rye from 2 to 3 (#3503) 2024-05-22 21:21:41 +08:00