mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 16:10:43 +00:00
c868605736dc1a7a92abea925cb9cace57c07ca2
877
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ByteBaker
|
c868605736 | chore: add tests, fmt, clippy | ||
|
ByteBaker
|
da96d301ab |
refactor: extract reusable Headers extractor for actix-web handlers
This commit: - Creates a new extractors.rs module with a generic Headers<T> extractor - Implements FromRequest trait for type-safe header deserialization - Centralizes header parsing logic to reduce code duplication across HTTP handlers |
||
|
ByteBaker
|
a288a0604a |
chore: update default dashboard version from 1 to 5
In `CreateDashboard` API, the default dashboard version when `version` key was absent in the payload used to be 1, which was wrong. It is now 5 (for v5). Going forward, this value should default to the latest version supported by OpenObserve. This is tracked by `const LATEST_DASHBOARD_VERSION`, which is automatically populated at compile time by counting the number of variants of `enum DashboardRequestBody`, requiring no human intervention beyond adding the new variant itself to the enum. |
||
 Ashish Kolhe
|
d6339499f8 |
feat: pipeline errors on ui (#8695)
### **PR Type** Enhancement, Bug fix ___ ### **Description** - Surface last pipeline error in API - Persist pipeline errors in new table - Batch upsert pipeline errors from reporting - Safer error messages in VRL failures ___ ### Diagram Walkthrough ```mermaid flowchart LR Reporting["Self-reporting errors"] -- "batch_upsert" --> DB["Table: pipeline_last_errors"] API["GET /pipelines"] -- "join with errors" --> UIData["PipelineList.last_error"] UIData -- "render" --> ListUI["PipelinesList error icon"] UIData -- "per-node errors" --> NodeUI["CustomNode error badge"] Ingestion["VRL apply_vrl_fn"] -- "clean error message" --> Reporting ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Configuration changes</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Bump DB schema version to 10</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire migration for last errors table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>11 files</summary><table> <tr> <td><strong>pipelines.rs</strong><dd><code>Add last_error to pipeline model</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-61bb28d7091130297c917a0530d96202fcb4b073b514410531f9fe71acfc814f">+22/-3</a> </td> </tr> <tr> <td><strong>pipeline.rs</strong><dd><code>Join pipelines with last error data</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-c69f4f94eee97ce226caddc497b2adf89e26869f2d72b0d0206dc31ab2714daf">+26/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register pipeline_last_errors entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-c539d7557d1425986d5c09849d8dbca42d1b309c2bc33d49db641d6a783098f6">+1/-0</a> </td> </tr> <tr> <td><strong>pipeline_last_errors.rs</strong><dd><code>Define ORM entity for last errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-e4ff5af0cb9c6e6343ecaad06fbb03515ace8868be6ee6d192ff0f01ee4d3e17">+22/-0</a> </td> </tr> <tr> <td><strong>prelude.rs</strong><dd><code>Export PipelineLastErrors entity</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-a818656b9fb01945155f28450270dd2ae20b67c8248c851ffa0f25e859d263ed">+1/-0</a> </td> </tr> <tr> <td><strong>m20250930_000001_create_pipeline_last_errors_table.rs</strong><dd><code>Create pipeline_last_errors table and index</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-173fe20fccfdcfa68563a13f9d69c2e6cb0901ac257d43386962bbac90f296de">+169/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Expose pipeline_errors DB module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+1/-0</a> </td> </tr> <tr> <td><strong>pipeline_errors.rs</strong><dd><code>CRUD and batch upsert for pipeline errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-627b6bb9ca334984b8f08f1405a67e5110c238788f165e30abda39b03f7ab68f">+331/-0</a> </td> </tr> <tr> <td><strong>queues.rs</strong><dd><code>Batch upsert pipeline errors during ingest</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-ed15a56d4d3becc2dfb3d05af2dac7f92962ec3091199728fa953e6aed7ed767">+45/-5</a> </td> </tr> <tr> <td><strong>PipelinesList.vue</strong><dd><code>Show pipeline-level last error indicator</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-85e3ef322f3eea0fc6d74d89431ba338a1153cb82ee06bef0f7c5f439c04267d">+27/-0</a> </td> </tr> <tr> <td><strong>CustomNode.vue</strong><dd><code>Show node-level error badge and navigation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-b9d83cd503b6d3470b238accaaea0b4decf567add8ad4727c63c9e583725f408">+98/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Return sanitized VRL error messages</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+12/-8</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Clarify queuing of error data</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8695/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+2/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
sktrpathi
|
b220f55435 |
fix: handle LLM-specific errors (#8690)
### **User description**
Enterprise code now returns error when raised by LLM API. These errors
are handled in this commit.
___
### **PR Type**
Bug fix
___
### **Description**
- Map rate limit errors to 429 responses
- Normalize other AI errors to 500
- Improve error logging messages consistency
- Audit reporting updated for error paths
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["AI service call"] --> B{"Error?"}
B -- "No" --> C["200 OK with response"]
B -- "Yes" --> D{"Rate limit (429)?"}
D -- "Yes" --> E["Return 429 Too Many Requests"]
D -- "No" --> F["Return 500 Internal Error"]
E -- "Audit report" --> G["Audit trail updated"]
F -- "Audit report" --> G
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>chat.rs</strong><dd><code>Standardize AI chat error
handling and 429 pass-through</code> </dd></summary>
<hr>
src/handler/http/request/ai/chat.rs
<ul><li>Log errors using extracted message string<br> <li> Return 429
JSON on rate limit errors<br> <li> Convert other AI errors to 500
internal errors<br> <li> Align stream path with audit reporting and
responses</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8690/files#diff-12a78cbb31b313cb3a0ca50f25c177470cf49b02708f179fe138ed3d66a2926b">+40/-6</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
ByteBaker
|
a5507e9dc0 |
feat (AI): add context support to chat requests and improve type safety
- Add context parameter to PromptRequest for passing additional data - Update AiServerRequest initialization to include context across chat endpoints - Change PromptResponse role field from String to Role enum for type safety - Add Serialize trait to DashboardRequestBody for serialization support |
||
 Huaijin
|
55004257f3 | feat: impl broadcast join for enrichment table (#8672) | ||
  Yashodhan JoshiandYashodhan Joshi
|
97dda39d97 |
feat: cloud email domains (#8584)
### **User description**
This adds a validation check for emails in cloud.
___
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Enforce cloud email domain validation
- Propagate errors via Result returns
- Update redirect to handle unauthorized
- Improve logging for blocked domains
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["JWT process_token"] -- "cloud feature" --> B["check_and_add_to_org"]
B -- "invalid domain -> Err" --> C["Result error"]
C -- "map to 401" --> D["redirect handler returns Unauthorized"]
B -- "ok -> (new_user,pending)" --> E["redirect builds URL params"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>jwt.rs</strong><dd><code>Result-based flow and cloud
email validation</code>
</dd></summary>
<hr>
src/handler/http/auth/jwt.rs
<ul><li>Change return type to Result for error propagation<br> <li>
Validate cloud email domain on new user creation<br> <li> Convert
multiple None returns to Ok(None)<br> <li> Wrap org/user operations to
return Result from cloud path</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8584/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+18/-11</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Unauthorized handling for
invalid email domains</code>
</dd></summary>
<hr>
src/handler/http/request/status/mod.rs
<ul><li>Handle process_token errors explicitly<br> <li> Return 401
Unauthorized on blocked email domain<br> <li> Preserve URL params on
successful processing</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8584/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+9/-3</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
|
||
|
sktrpathi
|
d0ffe710ab |
fix: improve dashboard API models & add field annotations (#8546)
### **User description** - Replace separate request/response models with unified `DashboardRequestBody/DashboardResponseBody` - Add `#[schema(required)]` to `title` and `description` fields across all dashboard versions (v1-v5) - Eliminate redundant `CreateDashboard/UpdateDashboard` `GetDashboard` model types - Use tagged enum for version-specific dashboard deserialization instead of manual JSON parsing. This helps generate `OneOf` schema type in OpenAPI spec. - Simplify owner validation in dashboard creation/update - Update OpenAPI schema to use consolidated model types ___ ### **PR Type** Enhancement, Documentation ___ ### **Description** - Add required schema for titles - Unify dashboard request/response models - Use tagged enum for versions - Simplify owner assignment logic |
||
|
51d9782e5d |
feat: add separate telemetry url (#8564)
### **PR Type**
Enhancement
___
### **Description**
- Add keyevent telemetry support and API
- Introduce `ZO_KEYEVENT_TELEMETRY_URL` config
- Send both standard and keyevent telemetry
- Update handlers to use new API
___
### Diagram Walkthrough
```mermaid
flowchart LR
Cfg["Config: add ZO_KEYEVENT_TELEMETRY_URL"]
Tel["Telemetry: send_keyevent_track_event()"]
JWT["JWT auth: new user registered"]
Bill["Billing: new subscription started"]
Mkt["Marketing: new user attribution"]
Job["Job: no ingestion after creation"]
Cfg -- "provides URL" --> Tel
JWT -- "send standard + keyevent" --> Tel
Bill -- "send standard + keyevent" --> Tel
Mkt -- "send standard + keyevent" --> Tel
Job -- "send standard + keyevent" --> Tel
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>telemetry.rs</strong><dd><code>Add keyevent telemetry
send method</code>
</dd></summary>
<hr>
src/common/meta/telemetry.rs
<ul><li>Add <code>send_keyevent_track_event</code> method.<br> <li> Use
separate keyevent telemetry URL.<br> <li> Respect telemetry enabled flag
(non-cloud).<br> <li> Robust error/status logging for keyevent
sends.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8564/files#diff-3ff026b46d9431e0657828a3bde4ab118fec006b353c2abc80be4b74633c98b4">+60/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>jwt.rs</strong><dd><code>Emit keyevent telemetry on
user registration</code>
</dd></summary>
<hr>
src/handler/http/auth/jwt.rs
<ul><li>Instantiate telemetry once per flow.<br> <li> Send both standard
and keyevent events for registration.<br> <li> Duplicate payload reuse
for both sends.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8564/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+11/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>billings.rs</strong><dd><code>Add keyevent telemetry
for subscription start</code>
</dd></summary>
<hr>
src/handler/http/request/cloud/billings.rs
<ul><li>Send standard and keyevent for new subscription.<br> <li> Reuse
event data across sends.<br> <li> Single telemetry instance per
request.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8564/files#diff-725936e3d6e955175e6100e8ca6f4437f334a2b554d6f11fa349ee9cbac70425">+11/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>marketing.rs</strong><dd><code>Add keyevent telemetry
for attribution events</code>
</dd></summary>
<hr>
src/handler/http/request/cloud/marketing.rs
<ul><li>Send standard and keyevent for attribution.<br> <li> Share
segment event data between sends.<br> <li> Use single telemetry
instance.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8564/files#diff-424975d70871fd725536b17e4aa2131af8fd29e3c9d30ded551526e8ae71cfcc">+11/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>cloud.rs</strong><dd><code>Add keyevent telemetry to
no-ingestion job</code>
</dd></summary>
<hr>
src/job/cloud.rs
<ul><li>Send standard and keyevent for no-ingestion report.<br> <li>
Reuse event data; single telemetry instance.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8564/files#diff-7fc16d008216c106d63f78f75c867fbefe7102433fcaa8893c314f0b7ea8d0e6">+11/-1</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Introduce keyevent
telemetry configuration</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Add <code>keyevent_telemetry_url</code> to
<code>Common</code>.<br> <li> Map to
<code>ZO_KEYEVENT_TELEMETRY_URL</code> env var.<br> <li> Default empty
string disables keyevent sending.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8564/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+2/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Prabhat Sharma <hi.prabhat@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
|
||
 Hengfei Yang
|
24bc37a80e | feat: remove grpc call for get result cache also improve RW logic (#8246) | ||
 udev
|
c09f2822e7 |
feat: custom config file path (#8464)
### **PR Type** Enhancement This feature adds a new cli argument to openobserve binary, i.e. -c which allows user to pass in path to a config file, adding flexibility to load env variables from different files. Earlier versions strictly looked for .env file in the current directory. If this flag is not specified when running o2, the default file path (.env) from current directory will be used to load the config files. This feature also adds a job to watch the config file. By default this job will run every 30 seconds, and can be configured via ZO_CONFIG_WATCHER_INTERVAL env variable. The watcher will watch the config file and reload the config if any changes are detected. By default the watcher watches .env file if no other file is configured using -c flag. ___ ### **Description** - Add CLI config file path support - Introduce config file path manager - Implement auto config reload watcher - Expose watcher interval configuration ___ ### Diagram Walkthrough ```mermaid flowchart LR CLI["CLI adds --config option"] -- "set path + reload" --> Manager["ConfigPathManager stores path/hash"] Manager -- "path available" --> Watcher["ConfigWatcher job (interval)"] Watcher -- "hash change" --> Reload["Reload dotenv + refresh configs"] Reload -- "update hash" --> Manager ConfigInit["Config init"] -- "load from path or .env discovery" --> Manager ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>7 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>Add --config option and initial reload handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+16/-2</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Config loading via path, hashing, and watcher interval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+44/-3</a> </td> </tr> <tr> <td><strong>config_path_manager.rs</strong><dd><code>New config path and hash manager</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-20874a97fbe99ee6fd692896482ad236916c5f813c3857f824fe0ff6e8afd72a">+64/-0</a> </td> </tr> <tr> <td><strong>lib.rs</strong><dd><code>Export config path manager module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-c1f72c148113548ea854a05d09e3576632fcd7367279563bf3f42f0aef5454e2">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Refactor enterprise config reload helper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+8/-3</a> </td> </tr> <tr> <td><strong>config_watcher.rs</strong><dd><code>Add periodic config file change watcher and reloader</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-598ec79e0686bf314514227cf1f0b1dc560770b0e056761d6a214289d2eeb7f2">+94/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire up config watcher on job init</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>pausable_job.rs</strong><dd><code>Improve macro documentation punctuation</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8464/files#diff-7e2dce7a3942789804bae659939c214c4eeb68d89315e23a08139214c339ea24">+3/-3</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
sktrpathi
|
d40b6afc2e |
refactor: fix OpenAPI schemas and improvements (#8505)
### **User description**
- **OpenAPI schema fixes**: Replace generic `Object` content types with
proper typed schemas
- `ai/prompt.rs`: Use `UpdatePromptRequest` instead of `Object`
- `authz/fga.rs`: Use `RoleRequest` for role operations
- `keys/mod.rs`: Use `KeyAddRequest` for cipher key operations
- `dashboards/mod.rs`: Use versioned dashboard schemas for create/update
endpoints
- `actions/action.rs`: Remove incorrect request_body specifications
- **Deprecated endpoint disambiguation**: Add "Deprecated" suffix to
alert operation IDs to prevent OpenAPI conflicts
- **Dashboard macro refactoring**: Replace repetitive getter/setter
methods with declarative macros
- `make_getter!` macro generates type-safe field accessors across all
dashboard versions (v1-v5)
- `make_setter!` macro generates unified field mutators with
version-specific matching
- Add comprehensive unit tests covering edge cases (empty values,
unsupported versions, missing inner objects)
- **Pipeline schema**: Add `ToSchema` derive to Pipeline struct for API
documentation
- **Enterprise feature gating**: Properly gate cipher key endpoints with
conditional compilation
___
### **PR Type**
Enhancement, Tests
___
### **Description**
- Replace OpenAPI Object with typed schemas
- Introduce dashboard getter/setter macros
- Gate cipher key endpoints by feature
- Add ToSchema for Pipeline and multipart
|
||
|
6e75dc0e62 |
feat: add bulk enable/disable for alert and pipelines (#8438)
### **User description**
This adds two apis, one for bulk enable/disable of pipeline and one for
bulk enable/disable of alerts.
___
### **PR Type**
Enhancement
___
### **Description**
- Add bulk enable/disable for alerts
- Add bulk enable/disable for pipelines
- Introduce enterprise permission checks helper
- Update auth bypass for bulk endpoints
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["New models: Bulk request/response (alerts, pipelines)"]
B["Alerts: POST /v2/{org_id}/alerts/bulk/enable"]
C["Pipelines: POST /{org_id}/pipelines/bulk/enable"]
D["Enterprise permission check: check_resource_permissions"]
E["Router: register bulk endpoints"]
F["Auth: bypass for /bulk/enable paths"]
A -- "used by" --> B
A -- "used by" --> C
D -- "enforced in" --> B
D -- "enforced in" --> C
E -- "exposes" --> B
E -- "exposes" --> C
F -- "allows unauth extraction for" --> B
F -- "allows unauth extraction for" --> C
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>6
files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Allow auth extractor bypass for
bulk enable endpoints</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>responses.rs</strong><dd><code>Add alerts bulk enable
request/response models</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-224027f90893122eea28e35778467acf1a4b5572ce612fc9bc9ec44ca8ebd9fb">+12/-0</a>
</td>
</tr>
<tr>
<td><strong>pipelines.rs</strong><dd><code>Add pipelines bulk enable
request/response models</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-61bb28d7091130297c917a0530d96202fcb4b073b514410531f9fe71acfc814f">+12/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Implement alerts bulk enable
endpoint with permissions</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-83431801d27626ef27482e67b62661f900043544a45ddf958bd027cf2fa3bd8d">+91/-2</a>
</td>
</tr>
<tr>
<td><strong>pipeline.rs</strong><dd><code>Implement pipelines bulk
enable endpoint with permissions</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-c69f4f94eee97ce226caddc497b2adf89e26869f2d72b0d0206dc31ab2714daf">+89/-1</a>
</td>
</tr>
<tr>
<td><strong>utils.rs</strong><dd><code>Add generic enterprise resource
permission checker</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-96bb9f4a8f547c821f26c957e8505d0437095b08d2b36687531f71565d287bd6">+38/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Register bulk enable endpoints in
service routes</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>openapi.rs</strong><dd><code>Expose bulk enable endpoints in
OpenAPI</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8438/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+2/-0</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
|
||
|
Huaijin
|
c2e8d73e65 | feat: implement broadcast join (#8506) | ||
|
1645873275 |
fix: interesting field config (#8480)
### **User description**
This provides the default tab for logs ui sidebar via config api and env
config
___
### **PR Type**
Enhancement
___
### **Description**
- Add env to set default fields tab
- Validate and expose config via status API
- Set frontend default tab based on config
- Add watcher to apply tab on toggle
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Env var ZO_LOG_PAGE_DEFAULT_FIELD_LIST"] --> B["Backend Config (Common.log_page_default_field_list)"]
B --> C["Config validation (uds|all|interesting)"]
C --> D["/api/config response (zo_config)"]
D --> E["Frontend store.state.zoConfig.log_page_default_field_list"]
E --> F["Logs UI sets default tab via setDefaultFieldTab()"]
F --> G["Watch toggle to re-apply default"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Introduce and validate
default field list config</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Add <code>log_page_default_field_list</code> env-config
option.<br> <li> Default to <code>uds</code>; document valid values.<br>
<li> Normalize to lowercase and validate allowed values.<br> <li>
Fallback to <code>uds</code> on invalid input.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8480/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+14/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Expose default field list in
status/config API</code>
</dd></summary>
<hr>
src/handler/http/request/status/mod.rs
<ul><li>Extend <code>ConfigResponse</code> with
<code>log_page_default_field_list</code>.<br> <li> Populate response
from common config.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8480/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+2/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>IndexList.vue</strong><dd><code>Use config to set Logs
UI default fields tab</code>
</dd></summary>
<hr>
web/src/plugins/logs/IndexList.vue
<ul><li>Add <code>setDefaultFieldTab()</code> to pick initial tab.<br>
<li> Use backend config to select UDS vs Interesting.<br> <li> Watch
user-defined schema toggle to re-apply default.<br> <li> Remove
hardcoded default to Interesting fields.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8480/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+24/-2</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
|
||
|
Hengfei Yang
|
97d09eff1e | fix: support prefix star match_all (#8451) | ||
|
Hengfei Yang
|
237c757c7c | fix: improve nats queue with limit policy (#8436) | ||
|
Ashish Kolhe
|
34a7341e87 |
chore: node list & user session refresh apis (#8198) (#8200)
### **PR Type** Enhancement ___ ### **Description** - Add CLI commands to refresh caches - Expose HTTP endpoints for refresh actions - Cache node list building in cluster module - Replace retention flag with mode enum ___ ### Diagram Walkthrough ```mermaid flowchart LR CLI["CLI subcommands"] -- triggers --> HTTP["HTTP endpoints"] HTTP -- calls --> Cluster["cluster::cache_node_list"] HTTP -- calls --> Sessions["db::session::cache()"] Config["Config: file_list_deleted_mode"] -- controls --> Retention["Compactor retention logic"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>New CLI subcommands for cache refresh</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+6/-0</a> </td> </tr> <tr> <td><strong>http.rs</strong><dd><code>HTTP client helpers for refresh endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-790b7780a51e8eecd4754b623af8d15ad87baaaf29455ba646e2e410f83f5498">+22/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Cache node list and update consistent hash</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+33/-0</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Replace retention flag with mode string config</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+3/-2</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Add FileListBookKeepMode enum and conversions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+29/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add refresh nodes and user sessions endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+20/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire refresh endpoints into basic routes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+3/-1</a> </td> </tr> <tr> <td><strong>retention.rs</strong><dd><code>Use retention mode for history/deleted behavior</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8200/files#diff-6abd048f5ef4e7163cffbb51cb4bee10c07fb88b871469d5b3724f7cd04743dc">+16/-6</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Ashish Kolhe
|
49a6744266 |
fix: vrl panic issues (#8355) (#8366)
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Guard flattening for non-object JSON values
- Prevent panics by checking null/object types
- Improve timestamp handling on null values
- Tighten pipeline flattening conditions
___
### Diagram Walkthrough
```mermaid
flowchart LR
VRL["VRL result"] -- "may be null/non-object" --> Guard["Check is_object && not null"]
Guard -- "true" --> Flatten["Flatten JSON"]
Guard -- "false" --> Bypass["Use value as-is"]
Timestamp["Ingest timestamp"] -- "null" --> Now["Default to current time"]
Pipeline["Pipeline nodes"] -- "flatten only objects" --> Flatten
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>multi_streams.rs</strong><dd><code>Safe flattening in
multi-stream search</code>
</dd></summary>
<hr>
src/handler/http/request/search/multi_streams.rs
<ul><li>Flatten only when value is non-null object<br> <li> Preserve
non-object items in per-query responses<br> <li> Avoid unwrap panics by
using ok()/None</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8366/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+13/-5</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>functions.rs</strong><dd><code>Guard VRL transform
flattening</code>
</dd></summary>
<hr>
src/service/functions.rs
<ul><li>Only flatten VRL return if object<br> <li> Prevent empty-string
on non-object by-pass</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8366/files#diff-ed5611a9aa6b5c033e70d7631ba72fafad3cc842672077a7fcf965d11a4fef23">+1/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>ingest.rs</strong><dd><code>Robust null timestamp
handling</code>
</dd></summary>
<hr>
src/service/logs/ingest.rs
<ul><li>Default null timestamp to now<br> <li> Keep error on unparsable
non-null timestamps</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8366/files#diff-e62118e45be183266d39ffa08491f255017954235ff0f3d6937683e314da406f">+10/-4</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>http.rs</strong><dd><code>Safe flattening in clustered
search</code>
</dd></summary>
<hr>
src/service/search/cluster/http.rs
<ul><li>Flatten array items only if objects<br> <li> Guard VRL result
flattening<br> <li> Add conditional hit flattening in query_fn path</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8366/files#diff-fe0a1bbec7ef6418120544b589a531e5144b28cd2650f99135ed8a70ac17ac03">+17/-12</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>batch_execution.rs</strong><dd><code>Pipeline: guarded
flattening and timestamp checks</code>
</dd></summary>
<hr>
src/service/pipeline/batch_execution.rs
<ul><li>Flatten only non-null objects across nodes<br> <li> Apply guards
before condition/func/destination steps<br> <li> Conditional timestamp
handling only for objects<br> <li> Improve error logging and flow
control</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8366/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+26/-18</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
0730d5f7a4 |
fix: landing page / Home page revamp (#6729)
### **User description**
1. This PR implements the latest UI of landing page / Home page with 2
charts for showing alerts and pipelines
2.
___
### **PR Type**
Enhancement
___
### **Description**
- Redesigned Home page layout and tiles
- Added alerts and pipelines charts
- Implemented theme-aware icons and styles
- Added loading state for summary fetch
___
### Diagram Walkthrough
```mermaid
flowchart LR
HomeView["HomeView.vue"]
SummaryAPI["Summary fetch & parsing"]
Charts["CustomChartRenderer charts"]
Tiles["Responsive metric tiles"]
Theming["Dark/Light computed icons & styles"]
Loading["Loading spinner state"]
HomeView -- "fetches org summary" --> SummaryAPI
SummaryAPI -- "sets metrics & statuses" --> Tiles
SummaryAPI -- "feeds alert/pipeline data" --> Charts
HomeView -- "applies theme classes" --> Theming
HomeView -- "shows while fetching" --> Loading
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>HomeView.vue</strong><dd><code>Home page redesign with
charts and theme-aware UI</code>
</dd></summary>
<hr>
web/src/views/HomeView.vue
<ul><li>Rebuilt template with responsive sections and tiles.<br> <li>
Added alerts pie and pipelines bar charts via CustomChartRenderer.<br>
<li> Introduced loading state, improved summary parsing, and theme-aware
<br>icons.<br> <li> Added extensive SCSS for dark/light theming and
layout.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/6729/files#diff-9af441af191d3254ef5f9db603dca75486de7638cda41a3c050e7b3c33d68280">+568/-229</a></td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Taiming Liu <liutaiming3@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: ByteBaker <42913098+ByteBaker@users.noreply.github.com>
|
||
|
Hengfei Yang
|
5d6700fa58 | feat: add per-stream flatten level configuration (#8353) | ||
|
Hengfei Yang
|
6e614f23e3 | fix: user update mode for enterprise (#8363) | ||
|
Hengfei Yang
|
55a0c62d3f | fix: root user password reset (#8348) | ||
  Simha Gandktx-vaidehi
|
aff48e3a56 |
fix: Dashboards Names needed on the QueryManager (#8335)
### **User description** # PR Type Enhancement # Description On the Query Management view, the user must be able to see the Query generating sources like Dashboards and Alerts by their names. This PR addresses this by introducing an enhancement at the backend which responds with QueryStatus carrying search_event_context which has almost all the necessary details for this change. # !WIP + Need to validate the same for Alerts + QueryList.vue needs to be updated for the same changes. + A simple test on the backend for the responses guaranteeing that SearchEventContext is populated appropriately. ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Add search event context to statuses - Enrich dashboard names in task status - Propagate dashboard/folder names to frontend - Extend UI with Query Source column ___ ### Diagram Walkthrough ```mermaid flowchart LR BESEC["Backend SearchEventContext"] -- "added to proto + structs" --> QS["QueryStatus"] QS -- "get_task_status enriches names" --> Handler["gRPC handler"] Handler -- "QueryStatusResponse includes context" --> FEAPI["Frontend APIs"] FEAPI -- "pass dashboard/folder names" --> Panels["Panel Data Loader"] Panels -- "meta and URLs include names" --> Streams["Streaming/Search services"] Streams -- "display source" --> RunningQueries["Running Queries UI"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13 files</summary><table> <tr> <td><strong>search.rs</strong><dd><code>Add SearchEventContext to QueryStatus and conversions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+31/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Enrich dashboard context in task status retrieval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+19/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Thread search_event_context through search and responses</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+2/-0</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Pass dashboard/folder names in requests and meta</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+14/-0</a> </td> </tr> <tr> <td><strong>useStreamingSearch.ts</strong><dd><code>Include dashboard/folder names in stream URLs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-8fce3e25a161cd458e734bd9b1f98aadefe3482da7b6529a5b1f4fa20ec115e2">+4/-0</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>Add dashboard/folder names to search and metrics URLs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+17/-0</a> </td> </tr> <tr> <td><strong>PanelContainer.vue</strong><dd><code>Forward dashboard/folder names to renderer</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-414737cba6064461f7026e0b016afcb9620bd5bb29c4708fc170dee789dbcd4d">+4/-0</a> </td> </tr> <tr> <td><strong>PanelSchemaRenderer.vue</strong><dd><code>Accept and propagate dashboard/folder name props</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-e5376e7e77bf3aaf6590c2522d8c2dbffcc95a8006ae0d6dc0040a37825367d6">+12/-0</a> </td> </tr> <tr> <td><strong>RunningQueries.vue</strong><dd><code>Show query source from search_event_context</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-6f0e18626a986d934c08c368ac383179c56bfcd914a27619ba748f43629c926b">+1/-0</a> </td> </tr> <tr> <td><strong>RunningQueriesList.vue</strong><dd><code>Add Query Source column to table</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-eb568491630bd1b80935cc0b02498d40274ede0e5be78593a0b49ca17fbbe627">+7/-0</a> </td> </tr> <tr> <td><strong>RenderDashboardCharts.vue</strong><dd><code>Provide dashboard/folder names to panel containers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-313f7c69102161242c893c863c50f1f1cd1fc19b68b5c99bf840b4b5c72f0f2e">+12/-0</a> </td> </tr> <tr> <td><strong>ViewDashboard.vue</strong><dd><code>Supply dashboard and folder names to charts view</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-052badb6758aff09ceaf1645869b9ba96125e6a0ea21abb01a8ee0a40d6467f3">+2/-0</a> </td> </tr> <tr> <td><strong>search.proto</strong><dd><code>Define SearchEventContext and add to QueryStatus</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-789f17326b818d5fbd1b5b752bed17e5341fde3d3261951634edcf78cacf3dbe">+11/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Dependencies</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>cluster.rs</strong><dd><code>Generate proto for SearchEventContext and QueryStatus field</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-542d8f27a174c845740ca1c3fe0b697db89c3db666437c1c4dd8c9745510080c">+20/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>RunningQueriesList.spec.ts</strong><dd><code>Update tests for new Query Source column</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-b1229a7a60180bf83bd9c81e1306941b8414522339566763c92b5d06c979ff38">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>launch.json</strong><dd><code>Add LLDB config for server debugging</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-bd5430ee7c51dc892a67b3f2829d1f5b6d223f0fd48b82322cfd45baf9f5e945">+19/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>en.json</strong><dd><code>Add i18n key for Query Source column</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8335/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+2/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech> |
||
|
Huaijin
|
8307dfb237 | perf: imporve string view gc (#8290) | ||
|
Ashish Kolhe
|
aaf06d2118 |
feat: use uuid v7 for internal trace ids (#8261)
### **PR Type**
Enhancement, Tests
___
### **Description**
- Switch trace IDs to UUID v7
- Add timestamp extraction from trace ID
- Improve HTTP header parsing robustness
- Add frontend UUID v7 utils and tests
___
### Diagram Walkthrough
```mermaid
flowchart LR
RustGen["Rust: generate_trace_id uses UUID v7"] -- "u128 -> OTel TraceId" --> OTel["OpenTelemetry TraceId"]
RustExtract["Rust: get_start_time_from_trace_id"] -- "parse v7 timestamp" --> QueryTime["Adjust query time window"]
HTTP["HTTP traces handler"] -- "robust headers + trace_id param" --> RustExtract
WebUtil["Web utils: getUUIDv7 + generateTraceContext"] -- "v7 traceId, v4 spanId" --> BrowserTraces["Browser trace context"]
TestsRust["Rust tests"] -- "validate v7, ordering, parsing" --> RustGen
TestsWeb["Web tests"] -- "validate v7, ordering, context" --> WebUtil
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>ider.rs</strong><dd><code>Switch to UUID v7 and add
timestamp extraction</code>
</dd></summary>
<hr>
src/config/src/ider.rs
<ul><li>Generate trace IDs using <code>Uuid::now_v7()</code>.<br> <li>
Add <code>get_start_time_from_trace_id</code> to extract microsecond
timestamp.<br> <li> Expand tests for v7 validity, ordering, and
parsing.<br> <li> Strengthen span ID and snowflake generator tests.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8261/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+279/-17</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Robust headers and time-window
from trace_id</code>
</dd></summary>
<hr>
src/handler/http/request/traces/mod.rs
<ul><li>Make Content-Type/header parsing tolerant to errors.<br> <li>
Allow <code>trace_id</code> param or filter to set time window.<br> <li>
Derive query window from UUID v7 timestamp.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8261/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+37/-2</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>zincutils.ts</strong><dd><code>Frontend UUID v7 support
and trace context</code>
</dd></summary>
<hr>
web/src/utils/zincutils.ts
<ul><li>Import and expose <code>getUUIDv7</code>.<br> <li> Use UUID v7
for traceId in trace context.<br> <li> Keep spanId generation via UUID
v4.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8261/files#diff-d3d59688cccdcc6c84ebe88b698c6910030abecc3125f4c0ef81b0df023699ba">+8/-2</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>zincutils.spec.ts</strong><dd><code>Tests for UUID v7
and trace context</code>
</dd></summary>
<hr>
web/src/utils/zincutils.spec.ts
<ul><li>Add tests for <code>getUUIDv7</code> validity, uniqueness,
ordering.<br> <li> Validate trace context uses v7 traceId.<br> <li>
Ensure uniqueness of trace and span IDs.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8261/files#diff-225d7ad1afe0a7e6f522c267bfd9ac4928cb5c23047fc6c02a287519a34aacb7">+45/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Dependencies</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>Cargo.toml</strong><dd><code>Add uuid v7
dependency</code>
</dd></summary>
<hr>
Cargo.toml
- Add `uuid` crate with v7 feature.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8261/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+1/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>Cargo.toml</strong><dd><code>Add uuid to config crate
dependencies</code>
</dd></summary>
<hr>
src/config/Cargo.toml
- Wire `uuid` crate via workspace dependency.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8261/files#diff-a70836e42ee2ff2af5068df047dd443758a0edc910d8ec5a01780396a69c9a9e">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
sktrpathi
|
f8cc585408 |
feat: store ratelimit info in vendor extension (#8259)
### **PR Type** Enhancement, Documentation ___ ### **Description** - Replace ratelimit comments with OpenAPI extensions - Add `x-o2-ratelimit` metadata to endpoints - Import `serde_json::json` where needed - Clean up obsolete inline ratelimit markers |
||
 
|
8af3495611 |
fix: cloud sync (#8222)
### **User description**
This is for syncing the changes done in cloud to main branch. No changes
for non-cloud code.
___
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Add invite listing and decline APIs
- Bypass auth for user-scoped invite routes
- Improve trial-expiry handling and logging
- Add org cache and invalidation hooks
___
### Diagram Walkthrough
```mermaid
flowchart LR
FE["Login & Invitations UI"] -- "list/accept/decline" --> API["/api/invites*, /{org}/member_subscription"]
API -- "auth bypass for user-scoped invites" --> Auth["AuthExtractor/TokenValidator"]
Auth -- "pending invites flag" --> LoginRedir["/config/redirect builds URL params"]
OrgSvc["Organization Service"] -- "generate/list/decline/delete" --> InvitesDB["Org Invites"]
OrgSvc -- "invalidate cache" --> OrgCache["Org table in-memory cache"]
Ingest["Ingestion services"] -- "trial expired -> 429" --> Clients["Ingestion Clients"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>25
files</summary><table>
<tr>
<td><strong>user.rs</strong><dd><code>Expand invite payload with org and
inviter</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>auth.rs</strong><dd><code>Auth bypass and key handling for
invite routes</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+36/-0</a>
</td>
</tr>
<tr>
<td><strong>jwt.rs</strong><dd><code>Return new user and pending invite
flags; defer org creation</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+22/-4</a>
</td>
</tr>
<tr>
<td><strong>token.rs</strong><dd><code>Allow invite/member routes
without org user; header injection</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-3339b4a964c121766ef87fcec0070f27a2b7085cf81281b53154e53d33cb28cf">+34/-0</a>
</td>
</tr>
<tr>
<td><strong>ingest.rs</strong><dd><code>Suppress logging for trial
expiry errors</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-24f250254ecfdbe004af9dc0ea23f8a2c0105ae0898690ed3e565089e5d8eef9">+30/-9</a>
</td>
</tr>
<tr>
<td><strong>org.rs</strong><dd><code>Pending-only invites; delete invite
API; trial extend cache poke</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-b47c0c1a8a78524196176430268eecdbfaf6f46d6b95873d3cbe70cfd67182b0">+40/-4</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Include pending_invites flag in
login redirect</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+4/-4</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>List pending invites and decline
invite endpoint</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-56897e42f9436a640bc70fddc25686485ba8ae1223635b32d252a0333309f748">+36/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Wire new invite endpoints into
router</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add TrialPeriodExpired error
variant</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>organizations.rs</strong><dd><code>Add in-memory org cache
with invalidation</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-4d87f9fe379a5d161a3e7c3c77f7f320be738e6a70c785aa5ce93cb0830c0147">+77/-7</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Start cloud billing watcher on
cloud builds</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+3/-0</a>
</td>
</tr>
<tr>
<td><strong>organization.rs</strong><dd><code>Invalidate org cache on
watch and mutations</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ce65cea926a09721a8fea14d07fe188bc2b887218e95d74a8bffefb54001da50">+4/-0</a>
</td>
</tr>
<tr>
<td><strong>user.rs</strong><dd><code>Delete invites for user
helper</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-6ee6b0fc9ed03224d4ddf0b7b631c6f81ff7d1d579bdb98d2c5d4453852eb5d6">+4/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Return trial expiry as dedicated
error</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+1/-3</a>
</td>
</tr>
<tr>
<td><strong>loki.rs</strong><dd><code>Silence trial expiry logs for Loki
ingestion</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a970c8c4785d4e4d878218615b17eec0cf4b4976121766b002c2b2f76371bfe6">+4/-1</a>
</td>
</tr>
<tr>
<td><strong>json.rs</strong><dd><code>Map trial expiry to HTTP 429 in
JSON ingest</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-506987e61095a54e47f7f208dad89f9f1bce595fa6bf1ba8a7d746f2b2ff6c4a">+15/-6</a>
</td>
</tr>
<tr>
<td><strong>otlp.rs</strong><dd><code>Return 429 for trial expiry;
reduce noise</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a24391bef75a4678c50724e8b5f305ec0ee8c2c74c68be9d677765de68b5d8dd">+17/-7</a>
</td>
</tr>
<tr>
<td><strong>organization.rs</strong><dd><code>Invite generation
validation; delete/decline invite flows</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-6b3c3c2b027c1158c0d41302cc69dfcd20ab68a6a4fa365cda403f838940bbb4">+48/-1</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Handle trial expiry with 429;
logging tweaks</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+34/-14</a>
</td>
</tr>
<tr>
<td><strong>users.rs</strong><dd><code>Enrich invite list; optional
pending filter; cleanup on removal</code></dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+50/-6</a>
</td>
</tr>
<tr>
<td><strong>organizations.ts</strong><dd><code>Add decline subscription
API client</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-2c3bad4c7a9fcedd7a653599381149b82f9c350b177de9d72431013807d4c36f">+10/-7</a>
</td>
</tr>
<tr>
<td><strong>users.ts</strong><dd><code>Add pending/accept/reject invite
APIs</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a334d8d570d9bdcaa3d4c8c2a993eff009a1ec8eb499c8a3d1a292d9224159d0">+12/-4</a>
</td>
</tr>
<tr>
<td><strong>InvitationList.vue</strong><dd><code>New UI to accept or
reject invites</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-ee97546ff1aba36cc0b24cb20782502c7c88279850cc3d540ea5685209f10702">+409/-0</a>
</td>
</tr>
<tr>
<td><strong>Login.vue</strong><dd><code>First-login invite handling and
logo tweaks</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-a135c379ef6ac8f92199ea5f7ff19a227971a9b0a87ccccf87c0268740522c2c">+178/-58</a></td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>en.json</strong><dd><code>Add invitation-related
translations</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8222/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+18/-0</a>
</td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Subhra264 <chakrabortysubhradeep556@gmail.com>
Co-authored-by: udev <uddhav@openobserve.ai>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: udev <uddhavdave1@gmail.com>
|
||
|
sktrpathi
|
d87d346bba |
fix: OpenAPI descriptions for endpoints (#8231)
### **User description** So far the OpenAPI descriptions for endpoint handlers was directly derived from the doc comments, leaving little control. This also leaked any dev-specific information into API spec. Now the summary and description are explicitly populated for all the endpoints, detaching them from doc comments. Also update response body annotations wherever necessary Also bump - utoipa: 4.0.0 -> 5.0.0 - utoipa-swagger-ui: 4.0.0 -> 9.0.0 ___ ### **PR Type** Documentation, Enhancement ___ ### **Description** - Add explicit OpenAPI summaries/descriptions - Correct response body schemas to Object/() - Update params/types for accuracy - Improve endpoint examples and metadata |
||
|
Hengfei Yang
|
c0897ae259 | refactor: stream stats (#8163) | ||
|
Subhra264
|
c49df079c7 |
test: add unit tests for alerts and self reporting (#8228)
### **PR Type**
Tests
___
### **Description**
- Add comprehensive unit tests for alerts
- Add extensive self-reporting queue tests
- Add serialization and model tests for usage
- Validate batching and timing behaviors
___
### Diagram Walkthrough
```mermaid
flowchart LR
Alerts["Alert model"] -- "unit tests" --> AlertsTests["alerts::tests"]
Reporting["Self-reporting queue/runner"] -- "unit tests" --> ReportingTests["self_reporting::tests"]
Usage["Usage/Trigger types"] -- "serde, traits tests" --> UsageTests["usage::tests"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>alert.rs</strong><dd><code>Add comprehensive tests for
Alert and params</code>
</dd></summary>
<hr>
src/config/src/meta/alerts/alert.rs
<ul><li>Add tests for <code>Alert</code> defaults and equality<br> <li>
Validate unique key logic and timestamps getters/setters<br> <li> Test
<code>ListAlertsParams</code> builder methods and chaining<br> <li>
Cover ancillary fields: destinations, context, owner/editor, flags</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-45c7eaf1a764ab52f3e014fe0555c03dc59d77c5ce974684cdee4ed90463a234">+415/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add tests for self-reporting
queue and runner</code>
</dd></summary>
<hr>
src/config/src/meta/self_reporting/mod.rs
<ul><li>Add async tests for <code>ReportingQueue</code>
enqueue/start/shutdown<br> <li> Test batching logic in
<code>ReportingRunner</code><br> <li> Validate debug formatting and
mixed data handling<br> <li> Add ignored test for channel-full
behavior</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-f4101fa8423f7a25f311dd82dab6796ed4ac8fef68449b5f018157dc33ca7de3">+674/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>usage.rs</strong><dd><code>Add serialization and model
tests for usage types</code>
</dd></summary>
<hr>
src/config/src/meta/self_reporting/usage.rs
<ul><li>Add serde tests for <code>UsageData</code>,
<code>TriggerData</code>, enums<br> <li> Test display helpers and
classification methods<br> <li> Validate <code>RequestStats</code> and
<code>Stats</code> defaults/from/serde<br> <li> Test hashing/equality
for grouping keys and enums</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8228/files#diff-b15a6adcb163dc27993c83528105a6726c44c86d8a84c5cf5d1d382c6ec8cc29">+684/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Hengfei Yang
|
1989432ba7 |
feat: remove syslog (#8227)
### **User description** close #8186 ___ ### **PR Type** Enhancement, Documentation ___ ### **Description** - Remove syslog feature across codebase - Clean config and routing references - Update metadata prefixes and SBOM files - Refresh i18n and OpenAPI assets ___ ### Diagram Walkthrough ```mermaid flowchart LR SYS["Syslog feature"] -- "deprecated & removed" --> CODE["Code paths"] CODE -- "delete flags/routes" --> CFG["Configuration"] CODE -- "remove storage keys" --> META["Metadata prefixes"] CODE -- "update specs" --> SBOM["SBOM/XML assets"] CODE -- "update docs/i18n" --> WEB["Web locales & OpenAPI"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Remove syslog config import from job module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+3/-26</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Stop glob importing config; drop syslog usage</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+1/-3</a> </td> </tr> <tr> <td><strong>meta.rs</strong><dd><code>Drop `/syslog` from metadata item prefixes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-1797798cd07803457a6273ef793d56568d8c91e743f6c4649c6db733325fb0af">+1/-2</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Remove syslog-related configuration options</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+0/-34</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Purge syslog feature flag and defaults</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+0/-25</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Clean references tied to syslog ingestion</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-5061e662a8899b7e00e5744bd72f808b28ac479a8921ac519f7409bffdc4438a">+0/-60</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Delete syslog router endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+0/-5</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Detach syslog handler wiring</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-99620cff6123acf0598ed2ec9d1c06f02cb0ddece1bef898047eb303d0b71fc2">+0/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Remove syslog-specific log ingestion paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Clean DB interactions for syslog keys</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Adjust request handling without syslog</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Unexport syslog services</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Remove syslog metadata definitions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-fbf797574839e69491ea0429141c038c0ff96297bdf3d8c60d60cdc4acec74b1">+0/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>integration_test.rs</strong><dd><code>Update tests to exclude syslog paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-7ff815b68943f08bef79f083c09e55ca8c18db8e25033034b12fa31f1d43dab4">+0/-57</a> </td> </tr> <tr> <td><strong>useIngestionRoutes.spec.ts</strong><dd><code>Update UI tests to remove syslog routes</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-3b7ebacf9481f4643a2b56ffe53ab161fc89d416e280d45b48568866a4dee40c">+0/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>20 files</summary><table> <tr> <td><strong>openapi.rs</strong><dd><code>Remove syslog routes from OpenAPI</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+0/-5</a> </td> </tr> <tr> <td><strong>config.cdx.xml</strong><dd><code>Update config component index (remove syslog)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-41ee11320f37e20fc96f754d35a890df5068af5072fd5388169d1cf305a1ed3b">+15011/-4611</a></td> </tr> <tr> <td><strong>wal.cdx.xml</strong><dd><code>Update WAL component index references</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-3d72e8b949a91b0bd78e97aa3ebf69e2887e2851223ea3d10fedeb0125c35a19">+19476/-358</a></td> </tr> <tr> <td><strong>openobserve.cdx.xml</strong><dd><code>Refresh SBOM root; remove syslog entries</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-454f38f03cf69f6b5358f5a528be4214ec2d33081b9e98ebe5d92c8bef47dc48">+14015/-7932</a></td> </tr> <tr> <td><strong>infra.cdx.xml</strong><dd><code>Update infra component index, no syslog</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-e51bf18198bc65a604b53ee18152ad9f2453ab7870d5be949022058e054ada73">+14127/-6086</a></td> </tr> <tr> <td><strong>proto.cdx.xml</strong><dd><code>Update proto index; purge syslog mentions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-a15fa99b53917ea85b4e268bb7612d43ee881dbe4ecf70a6682bb919b1f2f219">+8000/-1513</a></td> </tr> <tr> <td><strong>ingester.cdx.xml</strong><dd><code>Adjust ingester index excluding syslog</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-e72a9be4ba2aaff1a6b837a7f7149489603f8fe720766757c4291082e98b5a65">+15845/-4615</a></td> </tr> <tr> <td><strong>flight.cdx.xml</strong><dd><code>Add/refresh flight component index</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-f41393a0191b0560da893a5e24b2e4529f7c9bb34fc71231a532b8790630cba2">+19900/-0</a></td> </tr> <tr> <td><strong>hi.json</strong><dd><code>Remove syslog strings from Hindi locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-1e7b52b5c4e762db2c923d0d3efb6d693702825ed3a1a112d3069f26b0d5f605">+0/-13</a> </td> </tr> <tr> <td><strong>fr.json</strong><dd><code>Remove syslog strings from French locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-db80e762b88ddf183820c050cdc6b215e6b02b365b5e73f5152af46b0d26f1a2">+0/-13</a> </td> </tr> <tr> <td><strong>tr.json</strong><dd><code>Remove syslog strings from Turkish locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-6f0f131e9e2591480c4c5d1eaffc72d02dc31cbd8d3f90d8f012f08cabac8818">+0/-13</a> </td> </tr> <tr> <td><strong>de.json</strong><dd><code>Remove syslog strings from German locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-78f24ecb2dbcfc38fdf3ec22b2559a5b40d236ea689d1c55f5d3e87168310264">+0/-13</a> </td> </tr> <tr> <td><strong>ko.json</strong><dd><code>Remove syslog strings from Korean locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-50e27ca1b4825a5026f3a251bfcf79c97cab07cfade2ebd512d332453d707c48">+0/-13</a> </td> </tr> <tr> <td><strong>it.json</strong><dd><code>Remove syslog strings from Italian locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-2de78074a5f887afddce60d694ead9322e6ab223eb70251c13819f3028bdc70d">+0/-13</a> </td> </tr> <tr> <td><strong>nl.json</strong><dd><code>Remove syslog strings from Dutch locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-a1706021687ae5d8a1c006c49eab3946174fecd895f9b70a89c2e76d97250906">+0/-13</a> </td> </tr> <tr> <td><strong>es.json</strong><dd><code>Remove syslog strings from Spanish locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-ac2a2167b8b973e3f776f9fe3b43bef4576470d8ad33f3d7cfaff6edff0e5778">+0/-13</a> </td> </tr> <tr> <td><strong>pt.json</strong><dd><code>Remove syslog strings from Portuguese locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-38e804f64cb5e0e451849fc84e9b46e53c954a4a4263d51c18665ad45f4b4815">+0/-13</a> </td> </tr> <tr> <td><strong>zh.json</strong><dd><code>Remove syslog strings from Chinese locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-8818055db58f83230aeea4ebfd4cec31d924e8c47b6f91298e356959ab25339e">+0/-13</a> </td> </tr> <tr> <td><strong>ja.json</strong><dd><code>Remove syslog strings from Japanese locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-092e49c5f9fbc1b539ab9c2866d108d0ce1d314022b4c294eb4611b285e7bbd3">+0/-13</a> </td> </tr> <tr> <td><strong>en.json</strong><dd><code>Remove syslog strings from English locale</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+0/-13</a> </td> </tr> </table></details></td></tr><tr><td><strong>Dependencies</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>Cargo.toml</strong><dd><code>Update dependencies/features after syslog removal</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+0/-3</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>syslog.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-dc6813e31e32c23aae5760091460d2ec4023d37ae73e49d29052d9f4e9f2e1fa">+0/-43</a> </td> </tr> <tr> <td><strong>websocket.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-48f05dcd4b6f1d8345cb5707e1a3c34eedd76d94a3c3904eeabd799c7820789d">+0/-90</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-58010c6e78f69dc57c353890a807371cd43c2b4cceeeb1e9378130f26d898397">+0/-143</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+0/-135</a> </td> </tr> <tr> <td><strong>syslog_server.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+0/-137</a> </td> </tr> <tr> <td><strong>report_server.cdx.xml</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-d7a2fd1b635919119af9187ef44188c384bd01e444e74a9d15eaa21b0840b7ed">+20472/-0</a></td> </tr> <tr> <td><strong>syslog.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-4c0e9282e3d2352f9a382099901fe034bae2961eed01d764f16ad52f27f55db4">+0/-180</a> </td> </tr> <tr> <td><strong>syslog.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+0/-559</a> </td> </tr> <tr> <td><strong>syslogs_route.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-325df35568c993fc0727bfb0499b682e6bd30af781687706306b3bdb429c49e2">+0/-191</a> </td> </tr> <tr> <td><strong>syslog.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8227/files#diff-69ec0272ddac65f2cc96c79d84ca3933fc335a8d21ab5dd6909d71ea02a46486">+0/-37</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
 Hengfei YangandBhargav
|
2c0395124f |
feat: deprecated syslog (#8187)
### **User description** impl #8186 ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Mark syslog modules as deprecated - Add allow(deprecated) to callers - Block enabling syslog setting - Remove syslog-related tests and OpenAPI types ___ ### Diagram Walkthrough ```mermaid flowchart LR meta["meta::syslog module"] -- "mark deprecated" --> callers["call sites"] callers -- "add allow(deprecated)" --> compile_ok["compiles with warnings suppressed"] db_set["db::syslog::toggle_syslog_setting"] -- "reject enable=true" --> error["returns deprecation error"] openapi["OpenAPI schema"] -- "remove syslog types" --> spec["updated API spec"] tests["syslog tests"] -- "removed" --> leaner["reduced test surface"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Suppress deprecation warnings for syslog globals</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog handlers; annotate endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-58010c6e78f69dc57c353890a807371cd43c2b4cceeeb1e9378130f26d898397">+6/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Suppress deprecation in API scope setup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog server wiring</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog cache/watch tasks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+4/-0</a> </td> </tr> <tr> <td><strong>syslog_server.rs</strong><dd><code>Allow deprecated syslog server run linkage</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+2/-0</a> </td> </tr> <tr> <td><strong>syslog.rs</strong><dd><code>Allow deprecated syslog ingest; remove tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+3/-14</a> </td> </tr> <tr> <td><strong>syslogs_route.rs</strong><dd><code>Allow deprecated route endpoints; remove tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-325df35568c993fc0727bfb0499b682e6bd30af781687706306b3bdb429c49e2">+7/-52</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>6 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Mark syslog meta module as deprecated</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-fbf797574839e69491ea0429141c038c0ff96297bdf3d8c60d60cdc4acec74b1">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Mark syslog HTTP request module deprecated</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+1/-0</a> </td> </tr> <tr> <td><strong>openapi.rs</strong><dd><code>Remove syslog types from OpenAPI imports</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslog DB module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslog logs module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslogs_route service module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>syslog.rs</strong><dd><code>Remove unit tests for syslog meta types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-dc6813e31e32c23aae5760091460d2ec4023d37ae73e49d29052d9f4e9f2e1fa">+0/-127</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>syslog.rs</strong><dd><code>Block enabling syslog; allow deprecated routes/cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-4c0e9282e3d2352f9a382099901fe034bae2961eed01d764f16ad52f27f55db4">+11/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> |
||
|
sktrpathi
|
75fc81cecf |
chore: remove bogus unit tests (#8172)
### **User description** A significant chunk of API unit tests were not testing any actual business logic, but performing trivial actions such as struct-field-access, derive trait validation such as `Eq`, `PartialEq`, `Clone`, `Serialize` etc, and many of the tests were also just comparing constant values. They added no value to the test suite and have now been removed from the codebase, making the tests more focused. Additionally, - A lot of flaky tests have been corrected. These used to fail due to occasional race conditions when ran locally. - Some mock setup and teardown functions have also been cleaned up, which makes tests a lot more consistent. ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Remove trivial, non-value unit tests - Relax flaky assertions and panics handling - Improve test isolation via DB/cache resets - Minor test output formatting improvements ___ ### Diagram Walkthrough ```mermaid flowchart LR tests["Remove trivial/boilerplate tests"] -- cleaner suite --> stability["Stabilize assertions and panics"] stability -- reduce flakiness --> isolation["DB/Cache resets in tests"] isolation -- consistent runs --> outputs["Improve test log formatting"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><details><summary>11 files</summary><table> <tr> <td><strong>organization.rs</strong><dd><code>Remove struct-field and default-value trivial tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-4f089715a7ac4f78b45039d773f2fccbb4fbc6a409e0ab9ceee1fcf7911bc44b">+4/-325</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Drop JSON roundtrip serialization tests; tidy imports</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-c4bd542e4e07ed7706b0643086f5a7d4edbb3647c7ceedbe4bea516c5274d8bd">+1/-183</a> </td> </tr> <tr> <td><strong>auth.rs</strong><dd><code>Remove Debug/PartialEq trivial tests; format strings</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+2/-104</a> </td> </tr> <tr> <td><strong>main.rs</strong><dd><code>Make tracing test tolerant; ignore enterprise tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+23/-60</a> </td> </tr> <tr> <td><strong>event.rs</strong><dd><code>Remove constants and simple struct creation tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+1/-95</a> </td> </tr> <tr> <td><strong>validator.rs</strong><dd><code>Ensure test isolation with ORM init and cache clears</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+28/-22</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Make base URI tests robust to env errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+25/-4</a> </td> </tr> <tr> <td><strong>users.rs</strong><dd><code>Use best-effort cleanup; clear caches; improve logs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+12/-9</a> </td> </tr> <tr> <td><strong>integration_test.rs</strong><dd><code>Compact println formatting and panic messages</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7ff815b68943f08bef79f083c09e55ca8c18db8e25033034b12fa31f1d43dab4">+3/-3</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Remove strict zero-length assertions on globals</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+1/-13</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Remove unused constant import in tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-a5ea87a7766cb24619f64a6100f385f4b42c47af7d33b8e2f060229c536d7418">+1/-8</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>26 files</summary><table> <tr> <td><strong>authz.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-28a815d1c4728aff854a9996de0ba3d3dafe1bea6bfb0cf2a9f4186c40f985b6">+0/-52</a> </td> </tr> <tr> <td><strong>loki.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-121c166af5eab8b9cb83716afa7b65fa7856c006c809b3d910ced184c52ec587">+0/-7</a> </td> </tr> <tr> <td><strong>proxy.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7f12a4f2483e737671ac41f9c35b411e7323e0ff2731ad92a3b309e144d1316a">+0/-46</a> </td> </tr> <tr> <td><strong>service.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-61d456e7217d19bcec3920a4d1c17fd396a257c81e5394655db3632a057bdd9e">+0/-7</a> </td> </tr> <tr> <td><strong>service_account.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-878ad520e1c84500cc21ff6fc4088c8da64ff2febd79e60c0d4cc8c695c49798">+0/-80</a> </td> </tr> <tr> <td><strong>traces.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-bda8182c1f17a8360337c769edade0cb9478ff1443dd6aedce7f526afbe97afe">+0/-106</a> </td> </tr> <tr> <td><strong>user.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+0/-101</a> </td> </tr> <tr> <td><strong>auth_tests.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-2961</a></td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+0/-1</a> </td> </tr> <tr> <td><strong>action.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fc87c75a7df9539d7e7ee6184dfab02d4a8932323e81f95d57c05363aa44752d">+0/-18</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-5bf716db7e1d91dfbd427a2e320ce407c4f0c92d55d68f8e636daed1b02abd56">+0/-13</a> </td> </tr> <tr> <td><strong>reports.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d2117a3d67d4ec47b0017861845c92143baf917e10e6746df5dd97f925f81eeb">+0/-10</a> </td> </tr> <tr> <td><strong>destinations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e0ba33d95ef7f0eac13bdac7e750462907a0308f01c027ea206d058e282c977b">+0/-103</a> </td> </tr> <tr> <td><strong>inverted_index.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8bf9d9403f05f53f0c6fd5e6a6dbf0204ea80465b233567552cb7395d08d1c3c">+0/-22</a> </td> </tr> <tr> <td><strong>meta_store.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-45018c838da377fa9ded9305a1ee752adde06acdb607addf26c95f5747537615">+0/-6</a> </td> </tr> <tr> <td><strong>ratelimit.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fa7dc03fb526c09d5eb58021872d821fcd72eeb4d6a670224f5286012485bc3d">+0/-10</a> </td> </tr> <tr> <td><strong>timed_annotations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f94b4d6d12d68e7054ad33601e666b08e4594586013dc34384fa21b1df5fe6e8">+0/-12</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+0/-34</a> </td> </tr> <tr> <td><strong>m20250611_000002_populate_reports_table.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8247af6d6f8ccb6a5a51daff4cb1e352585be2d1c785d86ae5ebfea1dbc1099c">+0/-1</a> </td> </tr> <tr> <td><strong>folders.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-402482c22fda5dff8f33382ac11d0908f693975dec7c08b37b1c5fe24a20486a">+0/-1</a> </td> </tr> <tr> <td><strong>bulk.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48">+0/-9</a> </td> </tr> <tr> <td><strong>ingestion.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-3edd5f4ed1eed0f765b7b450ef0fbf587632e6b22302006a5e397049e297057f">+0/-7</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2f7f0bac01afc6e37908521d1d33d05fcd650b7a08792062976e1e7465b4a1f9">+0/-42</a> </td> </tr> <tr> <td><strong>footer_cache.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f9278e53ed7131c953fc30f8b15cda801b74b81fb2d570fe7f80a3b7fc1de872">+0/-7</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-1602f85399080c1e2838d1ef1d62af23d96f353379e49a7770026e892c954018">+0/-20</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+0/-31</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
sktrpathi
|
459d901e9b |
chore: add UTs, run fmt and clipyy (#8122)
### **PR Type** Tests, Enhancement |
||
|
sktrpathi
|
9a11a680ae |
chore: add test cases (#8095)
### **User description** - also clear over 400 clippy issues ___ ### **PR Type** Tests, Enhancement ___ ### **Description** Add extensive unit tests for enrichment storage Expose helpers for testability in storage module Modernize tests: clippy-friendly string formatting Normalize constants in time-based tests |
||
|
Huaijin
|
20a5fce67c |
feat: improve tracing for flight (#8097)
- [x] add span for tantivy_search - [x] improve tracing for flight after this pr the tracing for arrow flight will like below <img width="1770" height="506" alt="image" src="https://github.com/user-attachments/assets/55e30700-122c-4dec-bff7-8eed0f823bce" /> |
||
|
Huaijin
|
0581f626e5 | feat: impl distribute query analyze (#8063) | ||
|
|
75f2235d04 |
feat: create stream with field and data type (#7851)
### **User description** closes #6660 ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add field data types and UDS support - Validate field names with strict regex - Store defined schema as objects, not strings - Backward-compatible parsing and UI updates ___ ### Diagram Walkthrough ```mermaid flowchart LR API["API: Stream settings/schema"] --> Parse["Parse defined_schema_fields (strings|objects)"] Parse -- "normalize to DataField{name,type}" --> Settings["StreamSettings.defined_schema_fields: DataField[]"] Settings -- "new stream" --> SchemaAdd["Add fields to Arrow schema with types"] Settings -- "existing + UDS" --> UDSUpdate["Add new fields to UDS + persist"] Types["DataType enum (Utf8, Int64, Uint64, Float64, Boolean)"] --> Arrow["Arrow DataType mapping"] Validate["Regex field name validation"] --> Save["save/update settings endpoints"] UI["UI: Add/Edit stream fields"] --> ObjectFmt["Send/handle {name,type} objects"] ObjectFmt -- "compat" --> Compat["Store original + derive string list for display"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>stream.rs</strong><dd><code>Rename stream property field to raw type key</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+5/-6</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Introduce DataType and DataField; settings parsing</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+99/-13</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Optimize settings fetch; adapt defined fields helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+13/-6</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Use DataField names when building UDS sets</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+6/-1</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Add DataField flows; new/UDS logic and conversions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+270/-18</a></td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Validate field names; resolve defined field types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+126/-7</a> </td> </tr> <tr> <td><strong>AddStream.vue</strong><dd><code>Send defined fields as objects with type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-78b8c84a558e60d942bacffee365633d074042e464be465eb6984b0ee5df82b9">+6/-2</a> </td> </tr> <tr> <td><strong>StreamFieldInputs.vue</strong><dd><code>UI for field data type and UX tweaks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-c8173413939f320f197b0f8790e619da18666f79009345f72b3325e89bbb7566">+90/-27</a> </td> </tr> <tr> <td><strong>schema.vue</strong><dd><code>Support object-based defined fields; sync logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-9196be2ccae4054a00b0b1aa9d1e87b366c10920d6eaf9ef8c6e6293d50bcf46">+119/-25</a></td> </tr> <tr> <td><strong>useStreams.ts</strong><dd><code>Compare/sync object-based defined fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-a083369344f022cdf8d4f536719179b1d920369e33e2e26234e70dd36d151a7c">+50/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Map UDS fields by object name not string</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-619eafba0f0509d905b021f4a16659010aca23c62cffd9cd3e6c2bbe204ce200">+1/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Hengfei Yang
|
b2767cdebc | feat: create data retention by day (#8058) | ||
|
Subhra264andudev
|
c46fe2bf8c |
fix: new sso admin/viewer/editor should be able to enjoy given permissions (#8002)
Currently, openfga returns error when new sso user (admin/editor/viewer) tries to login into openobserve. This pr fixes that. --------- Co-authored-by: udev <uddhav@openobserve.ai> |
||
|
Huaijin
|
c51142de03 | fix: streaming aggregate config (#8029) | ||
|
Huaijin
|
3eb271620a | refactor: try move remotescan, stream aggregate, topk from rewrite to optimize rule (#7998) | ||
   
|
1d0490875b |
fix: is_multi_stream_search and enable_align_histogram (#8003)
### **User description** Fixes: - https://github.com/openobserve/openobserve/issues/7986 ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream search histogram support - Gate histogram alignment via query flag - Remove unused search_type plumbing - Propagate multi-stream flag through services ___ ### Diagram Walkthrough ```mermaid flowchart LR HTTP["HTTP handlers"] GRPC["gRPC handlers"] Utils["HTTP query utils"] SQL["SQL utils: histogram eligibility"] Hist["SQL: histogram query builder"] Svc["Search service"] Cache["Search cache"] Stream["Streaming pipeline"] Utils -- "parse is_multi_stream_search / enable_align_histogram" --> HTTP Utils -- "parse flags" --> GRPC HTTP -- "pass flags" --> Svc GRPC -- "pass flags" --> Svc Svc -- "eligibility check (multi/unified)" --> SQL Svc -- "partitioning uses align flag" --> Svc Svc -- "search() with multi-stream flag" --> Cache Cache -- "eligibility(query, multi)" --> SQL HTTP -- "convert_to_histogram_query(..., multi)" --> Hist Stream -- "do_search(..., multi)" --> Cache Hist -- "build single/multi-stream histogram query" --> Svc ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>15 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove SearchPartitionRequest.search_type field</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-3</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Add multi-stream histogram eligibility (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass multi-stream flag into search service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire flags; histogram conversion uses multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi: add align flag; remove search_type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Pass multi-stream flag in search profile</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Stream API supports multi-stream histogram and flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Propagate multi-stream flag in traces search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Cache search considers multi-stream eligibility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align partitions via flag; multi partition API updates</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build histogram for multi-stream queries (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Streaming cache handles multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Execution path passes multi-stream flag to searches</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Stream request processing threads multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream histogram query support - Align histogram via query flag - Remove search_type plumbing in partitions - Propagate multi-stream flag end-to-end ___ ### Diagram Walkthrough ```mermaid flowchart LR Http["HTTP handlers"] Grpc["gRPC handlers"] Utils["HTTP query utils"] Front["Web client"] Service["Search service"] Cache["Search cache"] SQLU["SQL utils: histogram eligibility"] Hist["Histogram query builder"] Front -- "is_multi_stream, align flag" --> Http Utils -- "parse is_multi_stream/align" --> Http Http -- "enable_align_histogram" --> Service Http -- "is_multi_stream_search" --> Cache Grpc -- "flags default false" --> Service Cache -- "eligibility(sql, multi)" --> SQLU Http -- "convert_to_histogram_query(..., multi)" --> Hist Service -- "partition(gen interval if enabled)" --> Service ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>19 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align-histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove search_type from partition request</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-4</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Histogram eligibility with multi-stream UNION ALL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass default multi-stream flag to service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add flags to HTTP search; align histogram handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi endpoint supports align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Forward multi-stream flag in inspector search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Streaming search supports multi-stream histogram</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Forward multi-stream flag in traces helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Eligibility check aware of multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align histogram in partition; multi aggregate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build single/multi-stream histogram SQL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Propagate multi-stream flag in streaming cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Pass multi flag through streaming execution</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire multi-stream flag across streaming path</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> <tr> <td><strong>IndexList.vue</strong><dd><code>Use align flag in partition requests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-1</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Toggle align histogram for dashboards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+3/-1</a> </td> </tr> <tr> <td><strong>useLogs.ts</strong><dd><code>Multi-stream histogram flow and params</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+3/-2</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>API params for is_multi_stream and align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-5</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> Co-authored-by: omkark06 <omkar@zinclabs.io> Co-authored-by: ktx-akshay <akshay.patil@kiara.tech> |
||
|
Hengfei Yang
|
9f5aa745a7 |
feat: remove tantivy footer cache (#8004)
### **User description**
We can't control the memory with the cache, it doesn't only cache footer
it will cache all the data, it caused memory issue. so i removed this
feature.
___
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Remove Tantivy reader cache feature
- Drop related config flag and module
- Rename result cache static to `GLOBAL_CACHE`
- Expose result cache size and memory stats
___
### Diagram Walkthrough
```mermaid
flowchart LR
cfg["Config: remove inverted index reader cache flag"]
http["HTTP status: report Tantivy result cache stats"]
grpc_use["gRPC search: remove reader cache usage"]
resultCache["Result cache: rename to GLOBAL_CACHE + stats"]
module["Puffin directory: remove reader_cache module"]
cfg -- "delete ZO_INVERTED_INDEX_CACHE_ENABLED" --> grpc_use
grpc_use -- "open index per request (no reader cache)" --> module
grpc_use -- "use GLOBAL_CACHE get/put" --> resultCache
http -- "display len + memory_size" --> resultCache
module -- "cleanup module export" --> ( )
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Drop inverted index reader
cache config flag</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Remove <code>inverted_index_cache_enabled</code> env config.<br>
<li> Delete flag <code>ZO_INVERTED_INDEX_CACHE_ENABLED</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+0/-6</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Expose Tantivy result cache
metrics in status</code>
</dd></summary>
<hr>
src/handler/http/request/status/mod.rs
<ul><li>Replace reader cache stats with result cache stats.<br> <li>
Import <code>grpc::tantivy_result_cache</code>.<br> <li> Report
<code>len</code> and <code>memory_size</code> for result cache.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+8/-3</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>storage.rs</strong><dd><code>Remove reader cache; use
GLOBAL_CACHE for results</code>
</dd></summary>
<hr>
src/service/search/grpc/storage.rs
<ul><li>Remove reader cache usage and logic paths.<br> <li> Always open
index and reader directly.<br> <li> Switch to
<code>tantivy_result_cache::GLOBAL_CACHE</code>.<br> <li> Clean up
imports for removed reader_cache.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-ab2d867e1e2e3dd8307505c2e130039628a84f8131a9b38cd2e7b4e1c141e094">+29/-45</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>tantivy_result_cache.rs</strong><dd><code>Result cache
rename and stats helpers</code>
</dd></summary>
<hr>
src/service/search/grpc/tantivy_result_cache.rs
<ul><li>Rename global static to <code>GLOBAL_CACHE</code>.<br> <li> Add
<code>len()</code> and <code>memory_size()</code> helpers.<br> <li>
Update test to use <code>GLOBAL_CACHE</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-f4bcf95cba21da14d907701d1440b1be9c9140c23de8681639770364e78af01e">+13/-2</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>reader_cache.rs</strong><dd><code>Delete Tantivy reader
cache module</code>
</dd></summary>
<hr>
src/service/tantivy/puffin_directory/reader_cache.rs
- Delete reader cache implementation and tests.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-baabb8633f8966f2407c724632e85a34f469a1591d52ded16a6d66c40950c190">+0/-423</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Miscellaneous</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Remove reader_cache module
reference</code>
</dd></summary>
<hr>
src/service/tantivy/puffin_directory/mod.rs
- Remove `reader_cache` module export.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-1602f85399080c1e2838d1ef1d62af23d96f353379e49a7770026e892c954018">+0/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Hengfei Yang
|
1f15b7ed58 | ci: bump version (#7992) | ||
  
|
748cca3ecb |
feat: visualization enhancement (#7456)
### **User description**
### Which query will be used?
- If query is empty will use histogram query
- Else query will be available:
- If query has group by clause :- will use same query
- Else will use histogram query
___
### **PR Type**
Enhancement, Tests, Bug fix
___
### **Description**
- Add server API to extract result schema
- Implement DataFusion plan visitor for projections
- Integrate visualization flow with schema and histogram
- Numerous UI fixes and UX improvements
___
### Diagram Walkthrough
```mermaid
flowchart LR
DF["DataFusion Plan Visitor (Rust)"]
API["HTTP API /result_schema (Rust)"]
FE["Logs Visualize UI (Vue)"]
Loader["Panel Data Loader (TS)"]
Cache["Histogram Handler & Cache (Rust)"]
DF -- "extract projections/group_by/timeseries" --> API
API -- "JSON ResultSchemaResponse" --> FE
FE -- "build config, set fields, decide histogram" --> Loader
Loader -- "inject searchResponse / is_ui_histogram" --> FE
Cache -- "normalize histogram()" --> DF
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>projections.rs</strong><dd><code>Add DataFusion logical plan
visitor to extract schema</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7a0420d3649da44b80be36d7f1043d6ff81d88395c961266882c3c700f7edff7">+651/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add /result_schema endpoint and
wire parsing/security</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+224/-2</a>
</td>
</tr>
<tr>
<td><strong>cacher.rs</strong><dd><code>Improve histogram() handling and
make public</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+22/-11</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Export projections
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-70bb941c2eef1365e5ead1a59b65c5397457b9197a4e22596e18d17db7eaf922">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>Index.vue</strong><dd><code>Integrate visualization with
schema API and histogram</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+564/-154</a></td>
</tr>
<tr>
<td><strong>VisualizeLogsQuery.vue</strong><dd><code>Field list UX,
props for searchResponse/histogram</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-979b01d943d58c5d70faeeac5948b7f02f0b2392a12b84ef07ba7f7393ab0261">+236/-174</a></td>
</tr>
<tr>
<td><strong>SearchBar.vue</strong><dd><code>Visualize toggle gating,
SQL/VRL disabled states</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+102/-52</a></td>
</tr>
<tr>
<td><strong>FieldList.vue</strong><dd><code>Readonly/visibility options
for logs visualize</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-a071c06e3e26629cb4ae4751450acc39c61fa49b07a1cc4be3711697829668d4">+13/-1</a>
</td>
</tr>
<tr>
<td><strong>TransformSelector.vue</strong><dd><code>Disable transform
controls in visualize mode</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0037d8dd9cfbacc26309b26e98ae953e7ccbd0891ac2fd01c6314ebd7eed9d2c">+11/-4</a>
</td>
</tr>
<tr>
<td><strong>ChartSelection.vue</strong><dd><code>Allow restricting
selectable chart types</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-384ce178717731e2c85f1312f12a54b42ceabd0b86f7e2befb0e421ef568a6a3">+14/-11</a>
</td>
</tr>
<tr>
<td><strong>PanelSchemaRenderer.vue</strong><dd><code>Accept
searchResponse and is_ui_histogram props</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-e5376e7e77bf3aaf6590c2522d8c2dbffcc95a8006ae0d6dc0040a37825367d6">+13/-0</a>
</td>
</tr>
<tr>
<td><strong>usePanelDataLoader.ts</strong><dd><code>Support injected
searchResponse and ui histogram</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+30/-5</a>
</td>
</tr>
<tr>
<td><strong>visualizationUtils.ts</strong><dd><code>Add SQL alias
validator for visualization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c2c391afcaa0b44c33d3f4fa531a4f31f044ae036f0baa182c8c88e83de066c5">+80/-0</a>
</td>
</tr>
<tr>
<td><strong>query.ts</strong><dd><code>Extend response metadata for
visualization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add test for compact delete value
check</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+17/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Allow auth bypass for
result_schema route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+3/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Register result_schema HTTP
route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>search.rs</strong><dd><code>Add ResultSchemaResponse
contract</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+7/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>TableRenderer.vue</strong><dd><code>Render null/undefined as
empty string</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-794706ea9da3a70012a0327c0464f9fd16802f78ece26055fe951716c2d73d59">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>sqlUtils.ts</strong><dd><code>Minor formatting
fix</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-dde40e8fcb5c37b06c011ede08922c44a3c025682fbc341aee97ba046b73a0e3">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>11 files</summary><table>
<tr>
<td><strong>playwright.yml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7afcd2d8f7b49bda74843f209eefb7b2da45f7e7803bf2e4bd636699b76aa2d3">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>visualise.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c14ac1db3e4aa86190d7525e8e38c410bcbbae7f46b01890e74937cc11be67e6">+1/-5</a>
</td>
</tr>
<tr>
<td><strong>logsPage.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8045b343491d8943869c1af74cdafab204dc49c02a70294417e73f47d35e0b59">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>visualize.spec.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-d144f23552ec150db375c89e52c3a53ede9ff3fa5dcd6092694fbe0a308d22ec">+225/-183</a></td>
</tr>
<tr>
<td><strong>useDashboardPanel.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-fbba1700750053b9eaf38ddd30dab84531bc44131c795ebb713811e4930cd7cb">+420/-6</a>
</td>
</tr>
<tr>
<td><strong>useLogs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+151/-19</a></td>
</tr>
<tr>
<td><strong>useNotifications.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-32ff5072df9badfebc0a72c401fe337de3324f76b186a52d202eb6cc2fa71e3a">+25/-0</a>
</td>
</tr>
<tr>
<td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>search.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+43/-0</a>
</td>
</tr>
<tr>
<td><strong>logs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-637857fa918f47ffb02ebfc8a35b124001ef4f65fd180c467f0f153382d2dc76">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>convertDataIntoUnitValue.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-14c6fe442ef231566ec411f9e0262cdd24ec912fc67b9154eec743282ab7211a">+120/-29</a></td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: loaki07 <rustyloaki@gmail.com>
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai>
|
||
|
Huaijin
|
e74ad0c932 |
feat: use custom FlightData encoder/decoder send ScanStats (#7979)
|
||
|
Loakesh Indiranandsai nikhil kethe
|
e46f6d5b82 |
feat: custom histogram interval & aligned partitions (#7909)
- `is_ui_histogram` query param for search api, the backend will convert the sql to histogram query and respond accordingly - `search_type` for search partition query, and if search type is `ui` only then we generate partitions with mini partition that is aligned with histogram interval ### **PR Type** Enhancement, Bug fix, Tests, Documentation ___ ### **Description** - Add UI histogram query conversion and flag - Align partitions to histogram with mini - Stream empty metadata+hints consistently - Expose histogram eligibility and interval ___ ### Diagram Walkthrough ```mermaid flowchart LR FE["Web UI logs & services"] -- "is_ui_histogram, searchType" --> HTTP["HTTP search endpoints"] HTTP -- "convert to histogram SQL\n(resolve streams, eligibility)" --> SQL["Histogram SQL converter"] HTTP -- "pass search_type, flags" --> SVC["Search service"] SVC -- "is_eligible, gen interval" --> PART["PartitionGenerator"] PART -- "aligned partitions\n(+mini optional)" --> STREAM["Streaming execution"] STREAM -- "SearchResponse + metadata\n(empty hits emitted)" --> FE SVC -- "Response: histogram_interval, is_histogram_eligible, converted_query" --> FE ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>16 files</summary><table> <tr> <td><strong>partition.rs</strong><dd><code>Histogram-aligned partitions with optional mini support</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-348fa23375d0cea3e5fb039cc6b381d79aa1d5121e6b9792acc048d3fd02e37c">+798/-269</a></td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Always emit responses; accumulate and metadata tweaks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+148/-152</a></td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>HTTP2 stream: UI histogram conversion and interval handling</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+149/-69</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Partition API: histogram eligibility, interval, UI paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+71/-5</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Convert original SQL to histogram-safe query</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+140/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>HTTP search: UI histogram conversion and response enrichment</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+38/-3</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Response/request fields for histogram meta and streaming chunks</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+45/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>New error: SearchHistogramNotAvailable</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-90c725d0e77a6e7d4bcc130cf5783848a5cf0efdb304b351cd63cacf8775fc36">+7/-0</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Multi-stream partition search accepts search_type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+16/-4</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Detect histogram-eligible queries and subqueries</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+27/-0</a> </td> </tr> <tr> <td><strong>http.rs</strong><dd><code>Add query flag parser for is_ui_histogram</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+8/-0</a> </td> </tr> <tr> <td><strong>Index.vue</strong><dd><code>UI: histogram streaming flow, timeouts cleanup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+45/-12</a> </td> </tr> <tr> <td><strong>IndexList.vue</strong><dd><code>Pass searchType=ui to partition API</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-0</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>API: support is_ui_histogram, searchType plumbed</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-2</a> </td> </tr> <tr> <td><strong>useStreamingSearch.ts</strong><dd><code>Include is_ui_histogram in streaming URL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8fce3e25a161cd458e734bd9b1f98aadefe3482da7b6529a5b1f4fa20ec115e2">+1/-0</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Dashboard partition calls pass searchType</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>3 files</summary><table> <tr> <td><strong>cacher.rs</strong><dd><code>Adjust tests for new response fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+6/-0</a> </td> </tr> <tr> <td><strong>SearchResult.spec.ts</strong><dd><code>Skip flaky resetPlotChart unit test</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-8ff7d14fc3cdac80eeec5550ecd89a77f015b4ebe928da9d4a76a9585d64d788">+1/-1</a> </td> </tr> <tr> <td><strong>useLogs.spec.ts</strong><dd><code>Skip failing query partitions test</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-842307ae8db03aa51891ddde98cfdbbabc712db1e1614c9eee99a59add395f70">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>error_utils.rs</strong><dd><code>Map histogram unavailability to 400 with code</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-d4de63bc9275a19c868f6967de0f7835bd2d7bda74ff55ce97b8fb3f9af54da9">+3/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Miscellaneous</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Export histogram SQL module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-dc0ab53d0ff551cf3cd12027c73d0da5e66680c0bf2d1830778639a33bac3464">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>SearchResult.vue</strong><dd><code>UI: robust empty-histogram handling and reset watcher</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-5e65533aa4754bb6d0074fda18ffc1ecacded3b9ca6bc28a161c3f6b12f20195">+17/-15</a> </td> </tr> <tr> <td><strong>SearchBar.vue</strong><dd><code>Cancel running query before applying saved view</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+2/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>query.ts</strong><dd><code>Types: histogram interval and eligibility fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>fga.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-e7c99f37f3abbb5e0507d2760bbf67df010d9218704b38dce84c5477acecd22e">+0/-2</a> </td> </tr> <tr> <td><strong>useLogs.ts</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/7909/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+154/-168</a></td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> |