This uses background jobs in order to download files asynchronously, and
a mpsc queue in order to send file names to the jobs threads. This
limits the number of threads we spawn to a fixed number configurable via
config var.
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated GitHub Actions workflow to dynamically use branch names for
repository dispatch events
- Improved workflow flexibility by removing hardcoded repository branch
references
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Tests covered**
- Ingest 79k fields with 100 user defined schema
- Reset stream selection while reloading home page and navigating to
logs
- Selecting stream and checking schema and values
- Go to streams page and enable store original data
- Go to stresstest stream via stream explorer
- Check if results are visible
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Tests**
- Added new Playwright test suite for pipeline management.
- Implemented comprehensive UI interaction tests.
- Included tests for pipeline creation, deletion, and log management.
- Enhanced test coverage with parallel test execution.
- Expanded integration tests with additional scenarios for pipeline
interactions.
- Introduced a new test file for improved CI integration.
- **New Features**
- Added functionality for managing remote destinations within the
pipeline.
- Introduced new methods for creating and deleting remote destinations.
- **Bug Fixes**
- Improved the logic for selecting the current date in the logs page to
ensure accurate date selection.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#5463
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced auto-refresh interval configuration across multiple
components.
- Added minimum refresh interval support with configurable restrictions.
- Introduced new error handling and validation mechanisms for report
creation.
- Added new utility functions for validating frequency intervals.
- **Improvements**
- Introduced dynamic minimum refresh interval settings.
- Added user feedback mechanisms for refresh interval constraints.
- Improved error handling for alert creation and updates.
- Enhanced clarity and maintainability of code through formatting
adjustments.
- Updated frequency input logic to dynamically calculate minimum values.
- Added validation for refresh intervals during data refresh and URL
query restoration.
- **Configuration**
- Added new props for minimum refresh interval and related messages.
- Implemented dynamic minimum value calculations for frequency inputs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
impl #5656
## Todo
- [x] Router consistent hashing for PromQL
- [x] Remove the key from index if the key get nothing from cache
- [x] Some cache item is empty, it should not
- [x] Add cache control ENV: `ZO_METRICS_CACHE_ENABLED=true`
- [ ] Optimize cache gc logic
- [ ] New encoding of RangeValue for cache
- [x] New cache key generate
- [x] Load cache from disk
- [ ] Histogram query OOM without cache for 2 days
- [x] Fix response series order
- [x] Need to check stream is exists
- [x] Add trace_id to the response
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new suite of Playwright tests for validating enrichment
table functionality.
- Added new functions for enhanced navigation and interaction within the
pipeline UI.
- Added tests to assert the data in every derived stream with condition,
function and only source + derived stream
- Added tests to assert if user clicks on any menu option and cancel on
dialog box, user stays on pipeline page
- Added tests to assert if user clicks on dashboard menu option and
accept on dialog box, user navigates to dashboard page
- **Bug Fixes**
- Improved error handling in tests to ensure accurate feedback during
interactions.
- **Chores**
- Updated the GitHub Actions workflow to include additional integration
tests.
- Removed outdated test files related to function management, Kinesis
ingestion, and reporting functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Updated the locator for the management menu item to enhance navigation
reliability.
- Streamlined the control flow by removing unnecessary wait times in the
management navigation process.
- **New Features**
- Added methods for managing service accounts, including adding,
validating, and deleting service accounts.
- Introduced a new test suite for validating service account
functionality, covering error handling and successful operations during
creation.
- Added a new method for automatic login based on visibility of the
login option.
- **Improvements**
- Updated the login method to ensure it waits for the correct login page
URL before proceeding.
- Simplified the login process in UI tests by removing unnecessary
parameters and wait times.
- Enhanced test scripts with improved structure and clarity, including
comments for better understanding.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced comprehensive UI testing for the Home, Ingestion, Login,
Logs, Metrics, Traces, Pipelines, Rum, Dashboard, Alerts, Reports, Data,
IAM, and About pages.
- Added functionality to handle organization changes and validations
across multiple pages, including default organization selections.
- New test suite for validating organization change scenarios across
various pages.
- Enhanced login functionality for internal users.
- Introduced new classes and methods for the Metrics, Streams, Rum, and
Alerts pages to support UI testing.
- Added new classes for the Data and IAM pages, expanding testing
capabilities.
- Enhanced Management page with navigation and validation capabilities.
- New methods for validating page URLs and content across various pages.
- **Bug Fixes**
- Improved validation methods to ensure proper navigation and content
checks on various pages.
- **Documentation**
- Updated testing framework to include new test cases and improved
structure for organization change tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix#5218
Backend:
- [x] allow condition node to evaluate `null` and `""`, empty, for both
`=` and `!=` conditions
- [x] stop ingestion when resolved dynamic stream name is empty
**How to Use(In ConditionNode):**
To evaluate field value is not empty:
`app_name != ""`
To evaluate field value is not null:
`app_name != null`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced `LogsPage` with improved date and time selection, timezone
management, and clipboard operations.
- Introduced `ManagementPage` for UI testing, allowing logo updates and
management functionalities.
- Added new tests for time zone changes and streamlined test scenarios
on the logs page.
- Introduced a suite of tests for joining logs, validating various join
scenarios.
- **Bug Fixes**
- Updated confirm button interaction in `ReportsPage` to ensure
visibility before clicking.
- **Documentation**
- Improved clarity and purpose of test scenarios through restructuring
and simplification.
- **Chores**
- Added `logoManagement.spec.js` and `join.spec.js` to the Playwright UI
integration tests for expanded coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Enhanced Playwright test suite for dashboard functionalities,
including comprehensive tests for VRL functions, dashboard settings, and
data validation.
- Added new test cases for creating, deleting, and updating dashboards,
as well as applying dynamic filters.
- Introduced automation for the login process and improved the
validation of user inputs in dashboard tests.
- **Bug Fixes**
- Improved error handling and visibility checks in dashboard tests to
ensure user inputs are validated correctly.
- **Chores**
- Updated import statements across multiple test files to utilize a
local module for better fixture management.
- Added a new test file for VRL visualization tests to expand
integration coverage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new `PipelinePage` class for enhanced UI testing of the
pipeline management interface.
- Added a comprehensive suite of Playwright tests to validate pipeline
creation, editing, and deletion functionalities.
- **Bug Fixes**
- Corrected the test filename in the workflow configuration to align
with naming conventions.
- **Chores**
- Removed the outdated `pipeline.spec.js` test file to streamline the
testing suite.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added support for code coverage reporting with the integration of
`vite-plugin-istanbul`.
- Introduced a new configuration file for code coverage settings.
- **Improvements**
- Updated project to use ECMAScript modules for better compatibility.
- Enhanced UI test coverage with additional test cases and improved
error handling.
- Refactored test suites to utilize a custom module for better
organization and maintainability.
- Improved the login process and dashboard loading in test cases for
better reliability.
- **Bug Fixes**
- Resolved issues related to import statements affecting test execution.
- **Documentation**
- Updated `.gitignore` to exclude coverage directories from version
control.
These updates enhance the functionality, maintainability, and
reliability of the application and its testing framework.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Introduced multiple new test files for dashboard functionalities,
including settings for variables, tabs, and general settings.
- Enhanced testing capabilities with new test cases covering various
scenarios for managing dashboard elements.
- **Improvements**
- Added `data-test` attributes to various UI components across multiple
files, improving testability for automated testing.
- **Bug Fixes**
- Updated import paths for JSON data files to ensure proper referencing
in tests, enhancing overall test reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
test case report e2e
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
- **New Features**
- Added a new test suite for report functionalities, including creating
and deleting dashboards and reports.
- Introduced a new `IngestionPage` class for handling data ingestion
through an API.
- Enhanced `DashboardPage` with improved dashboard creation and deletion
processes.
- Updated `ReportsPage` to support report creation and verification with
new methods.
- **Bug Fixes**
- Improved element selection methods for better reliability in the login
process.
- **Documentation**
- Streamlined import statements and improved code organization for
better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced test coverage for dashboard UI with new Playwright tests
focusing on the transpose feature and dynamic columns.
- Added `data-test` attributes to various components in the General
Settings for improved testability.
- **Bug Fixes**
- Improved consistency in the GitHub Actions workflow configuration for
UI tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced new UI testing classes for various application pages,
including `AlertsPage`, `DashboardPage`, `LogsPage`, `ReportsPage`,
`TracesPage`, and `LoginPage`.
- Added a new test suite focusing on time-related features across
multiple pages.
- **Bug Fixes**
- Improved consistency in the GitHub Actions workflow for UI integration
tests.
- **Documentation**
- Enhanced organization of locators and methods for better usability in
UI testing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Neha P <144308844+neha00290@users.noreply.github.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced new test cases for the search history feature, enhancing
verification of navigation between the search history and logs pages.
- **Chores**
- Added new environment variables `ZO_FILE_PUSH_INTERVAL`,
`ZO_USAGE_BATCH_SIZE`, `ZO_USAGE_PUBLISH_INTERVAL`, and
`ZO_USAGE_REPORTING_ENABLED` to the Playwright workflow configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new suite of UI tests for a logging application,
enhancing testing coverage.
- Added functionality to automate user login and data ingestion
processes within the tests.
- Implemented a test case to toggle settings and verify UI interactions.
- **Bug Fixes**
- Expanded the test file list to include additional scenarios for more
comprehensive testing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This fixes the `deny.toml` config file to work with the v2 of
cargo-deny. This does not change any deps, and adds any exceptions or
permissions in `deny.toml` to make the existing deps pass the action. We
can take a look and clean dependencies if needed in a separate PR.
This also updates the CI file to run on any changes in `deny.toml` and
CI yaml file itself as well.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced GitHub Actions workflow to trigger on changes to relevant
configuration files, improving responsiveness to updates.
- Restructured license management strategy, allowing more flexibility
with specific licenses and exceptions for various dependencies.
- **Bug Fixes**
- Removed outdated settings related to vulnerabilities and unmaintained
packages, simplifying the advisories section.
- **Chores**
- Cleaned up the configuration file by removing unnecessary entries and
commenting out the disallowed licenses.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Shyam Panjiyar
Yashodhan JoshiandYashodhan Joshi
Hengfei Yang
Chaitanya Sistla
dependabot[bot]
Neha Pandsai nikhil kethe
Prabhat SharmaandHengfei Yang
udev
Bhargav
Omkar Kesarkhane
ktx-vaidehi