Fixes#4455
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Enhanced authorization checks for `query`, `query_range`, and `series`
functions to dynamically reflect stream types.
- Improved permission validation logic in the `search` and
`search_multi` functions, allowing for more accurate access control
based on user roles and stream types.
- Updated logic in the `get_latest_traces` function to refine the
construction of the `o2_type` field for better permission checks.
- **Bug Fixes**
- Corrected the mapping of stream types to model keys, enhancing
security and accuracy in permission checks.
- **Chores**
- Added necessary imports for `OFGA_MODELS` to support new permission
logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added new fields `is_descending` and `limit` to query responses for
enhanced sorting and pagination.
- **Improvements**
- Enhanced logging and tracing for better query time tracking and
debugging.
- Streamlined span handling for more efficient tracing and
instrumentation.
- **Bug Fixes**
- Fixed validation for `result_cache_enabled` configuration to prevent
cache-related errors.
- Improved accuracy in cache limit handling in query responses.
- **Chores**
- Updated Rust toolchain version for improved compatibility and
performance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced trace ID management with functions for adding, removing,
and generating trace contexts.
- Added the ability to cancel running queries.
- **Enhancements**
- Integrated tracing functionality into search operations for improved
observability.
- Updated CORS configuration to allow `traceparent` headers.
- **Debugging**
- Added console logs for traceparent parameters in search functions.
- **Localization**
- Added new localization key for "cancel" message in English.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced `searchType` attribute across various components to enhance
search functionality.
- Added support for different search event types like `UI`,
`Dashboards`, `Reports`, `Alerts`, `Values`, and `RUM`.
- **Bug Fixes**
- Improved search parameter handling in multiple components and API
calls.
- **Enhancements**
- Updated search-related functions to include `searchType` for more
precise search results.
- Enhanced dashboard and RUM views with new `searchType` attributes for
better data filtering.
- **Refactor**
- Standardized search functionality across different modules and
components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
The query `size` works like this, it should more clear now:
1. size > 0, it will return hits equal to `size`
2. size = 0, it will return empty hits, this used to other purposes,
like: get total, get aggs.
3. size = -1, it will use a backend default limit to return hits,
setting by `ZO_QUERY_DEFAULT_LIMIT=1000`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Corrected handling of query parameters by changing types from `usize`
to `i64` to support larger values and prevent potential overflow issues.
- **Improvements**
- Enhanced the logic for setting query limits to handle negative values
more gracefully, ensuring better default behavior.
- **Refactor**
- Updated various internal structures and function signatures to use
`i64` instead of `usize` for consistency and improved range handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The new endpoint `/config/reload` will refresh the
environment variables.
There are a bunch of env-vars which will not be reloaded
for e.g. HTTP_PORT etc. these will need a restart.
---------
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
close#3215
use follow to only enable tracing in search
```
ZO_TRACING_ENABLED=false
ZO_TRACING_SEARCH_ENABLED=true
```
---------
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
Use **Fast Mode** when dealing with thousands of fields. It retrieves
the first N fields for a quicker response.
Enable it through this configuration:
```
ZO_FAST_MODE_NUM_FIELDS=50
ZO_FAST_MODE_STRATEGY=first/last/both
```
And use the search API:
```json
{
"query": {
"sql": "select * from default ",
"start_time": 1709711116396000,
"end_time": 1709712016396000,
"from": 0,
"size": 250,
"sql_mode": "full",
"fast_mode": true
}
}
```
When the backend identifies a request:
- Using `fast_mode`
- With an `SQL` prefix of `select *`
- Where schema fields exceed `ZO_FAST_MODE_NUM_FIELDS`
It will then rewrite the `SQL` to `select f1,f2,f3`, where 'fn'
represents the first 'n' fields from schema. The value of 'n' is
determined by `ZO_FAST_MODE_NUM_FIELDS`.
In the end you can set `ZO_FAST_MODE_STRATEGY=first/last/both` to
control the order for pickup fields from schema.
---------
Co-authored-by: oasisk <ashish.j.kolhe@gmail.com>
## Design Spec:
- POST `/api/:org/ingest/metrics/_json`
```json
[
{
"__name__": "stream name",
"__type__": "counter / gauge / histogram / summary",
"label_name1": "label_value1",
"label_name2": "label_value2",
"_timestamp": 0, // unix timestamp in millisecond
"value": 0.0 // float64 value
}
]
```
## How to use this API:
- must use `__name__` set stream name.
- must use `__type__` set metrics type. and the type can't be updated
after first record
- `_timestamp` needs UNIX timestamp or empty will fill by current
timestamp
- `value` should be float64 or int64
## Example:
POST `/api/default/ingest/metrics/_json`
```json
[
{
"__name__": "cpu",
"__type__": "gauge",
"namespace": "ziox",
"container": "zo1",
"_timestamp": 1687326643584062,
"value": 10.1
},
{
"__name__": "cpu",
"__type__": "gauge",
"namespace": "ziox",
"container": "zo2",
"value": 0.1
}
]
```
Prabhat Sharma
Hengfei Yang
Yashodhan Joshi
Subhra264
Taiming Liu
Ashish Kolhe
Huaijin
Omkar KesarkhaneandTaiming Liu
Ankur SrivastavaandHengfei Yang
