## Summary
- add shared lifetime AI credit limits and persisted per-organization
usage tracking
- expose used and total AI credits in organization management with an
editable limit dialog
- keep router quota caches synchronized and delegate billing and
contract exhaustion policy to enterprise
- clean up the redundant quota table only in non-cloud builds while
preserving cloud credit data
## Companion PR
- https://github.com/openobserve/o2-enterprise/pull/2208
## Testing
- cargo check --features cloud
- cargo test -p openobserve-core --features cloud trial_quota::tests
--lib
- cargo test -p infra --features cloud
m20260717_000001_drop_trial_quota_usage_for_non_cloud::tests --lib
- cargo test -p infra
m20260717_000001_drop_trial_quota_usage_for_non_cloud::tests --lib
- cargo fmt checks for changed Rust packages
- npm run type-check
- 200 focused frontend unit tests covering organization management,
billing usage, organization services, and required numeric input
- live organization-management verification with per-org 0 used / 1,000
total values
## What
Redesign of the log patterns UI.
- **`PatternCard`** reworked: template tokens, a badges row (trend /
rarity / anomaly), and inline frequency + percentage replace the old
fixed count and action columns.
- **`PatternVolumeCell`** (new) + **`usePatternVolume`** (new) render
per-pattern volume sparklines.
- **`PatternDetailsDialog`** now owns the include / exclude /
create-alert actions that previously lived on each card.
- **`patternUtils`** / **`usePatternActions`** extended to back the
above.
## Test plan
- `web` unit tests for the touched areas: 356 passing, 0 failing
(`plugins/logs/patterns`, `plugins/logs/data-table`, `SearchResult`,
`FieldValuesPanel`).
- New spec coverage for `PatternVolumeCell`, `usePatternVolume`, and
`patternUtils`.
- `npm run type-check` clean.
- `lint:design`, `lint:tokens`, `lint:token-purity` all clean.
## Notes
Rebased onto latest `main`. Five conflicts came from main's repo-wide
type-check pass (#13174), which made `OIcon`'s `name` prop required.
Main's fix (`name=""` + slot) was kept in `CodeQueryEditor.vue`,
`FieldValuesPanel.vue`, and `CellActions.vue`; in `PatternCard.vue` the
surrounding block was removed by this redesign, so the deletion stands.
## What
Adds **Agent Behavior signals** — a PB-safe rollup that surfaces two
behavioural questions the platform couldn't answer before: *is an agent
stuck in a loop?* and *why is it failing?* Plus a batch of Agent Graph /
AI Observability UX and correctness fixes found while building it.
Companion changes land in **o2-enterprise**
(`feat/agent-signals-enterprise-split`): the SQL builders, mappers,
config-driven failure taxonomy, the service-graph agent-classification
fix, and the RBAC route entry.
## Backend (agent-signals)
- Bounded-key rollup co-located in the service-graph job
(alert-manager-only, reuses its watermark + self-ingest). Groups only on
agent/tool/class — never trace_id — so it stays PB-safe.
- Config-driven **failure taxonomy** (rules as data, evolvable without
code) + config-driven error-detail columns; ancestor-agent resolution so
error spans attribute to the owning agent.
- Read API `GET /{org}/traces/agent_signals` (dual `#[cfg]`, reads the
derived `_agent_signals` stream) + RBAC route entry (enterprise
`route_permissions`) mapping it to `traces` read.
- OSS/enterprise split respected: pure SQL/mapping in enterprise, thin
gated I/O in OSS.
- Fix: `traces/latest` only narrows to a UUIDv7 trace_id's embedded time
when that time is inside the caller's window (fixes "trace not found"
for time-shifted/re-ingested traces).
## Frontend (AI Observability)
- New **Agent Behavior** page under Monitor: looping-agents +
failure-taxonomy tables, each row opening an actionable detail drawer
(real grouped error messages / worst-repeat traces), click-through to
Trace Details in a new tab.
- Agent-node **Behavior tab** on the Agent Graph node drawer (reuses the
same signals).
- Nav rename: LLM Insights → **Insights**, LLM Sessions → **Sessions**.
- **Agent** is the default scope across the AI pages, with the
Agent|Stream toggle placed consistently (left scope bar); explicit
URL/localStorage choices still win.
- Removed the duplicate in-table refresh on Sessions.
## Agent Graph fixes
- **Agent classification**: agents that also call tools now correctly
classify as `service_type: "agent"` (they glow), fixing a latent
precedence bug where the tool/model client-role overrode the agent role
and `is_real_service` forced agents to plain services. Rust
`classify_entity` brought in line with the (already-correct) TS twin.
- **Drag-to-pan restored**: the shared ChartRenderer armed
`dataZoomSelect` cursor unconditionally, hijacking drag away from roam
pan. Disabled for this graph (no dataZoom).
- **Nodes scale on zoom** (`nodeScaleRatio: 1`) so zooming enlarges
nodes/labels, not just spacing.
## Testing
- Enterprise unit tests: agent-signals SQL/mappers/config,
classification precedence, RBAC route resolution — all pass.
- FE: type-check + lint clean.
- Verified end-to-end on real re-ingested introspection data: taxonomy
unclassified rate ~100% → ~1–3%, agents resolved, both agents
classify/glow, trace-open works.
## Notes
- `Cargo.toml`/`Cargo.lock` and unrelated Slack-bot files were
intentionally excluded from the commits (local build-config drift).
---------
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Fixes several MCP-related API issues:
- Enrich `x-o2-mcp` metadata with `summary_fields` on kv list, backfill
list, alert templates, dashboard annotations, and search jobs so MCP
tools return well-shaped summaries.
- Add missing `org_id`/`dashboard_id` OpenAPI path params to the timed
annotations list endpoint.
- Return an empty result (instead of a 500) from pipeline history when
the triggers stream doesn't exist yet.
- Return a properly-shaped `PaginatedResponse` (preserving the
query-range warning) from the traces `get_latest_users` endpoint when
there are no matching hits.
- Sanitize error-message headers so error responses propagate the real
status/body instead of failing with a 500.
## Summary
- add `ZO_FILE_DOWNLOAD_MIN_RECORDS` with a default of 100 records
- skip undersized files before latest-file batch downloads and
background queue insertion
- apply the threshold consistently to Parquet and index files
- preserve caching when `records == 0`, which can represent missing
metadata from older gRPC senders
## Why
Downloading and caching very small files consumes queue capacity and
cache space without providing enough reuse benefit. Filtering them
before enqueueing avoids unnecessary download work.
## Impact
Files with a known positive record count below the configured threshold
are read from object storage without being added to the download cache.
A zero count is treated as unknown and remains cache-eligible; files at
or above the threshold keep the existing behavior.
## Validation
- `cargo fmt --all -- --check`
- focused minimum-record boundary test
- `cargo check -p search -p openobserve-core -p openobserve-api`
- `git diff --check`
Fixes#13288
Design at: #13285Fixes#13285
## Problem
Internal columns (`_timestamp`, `_all`, `_o2_id`, `_original`,
`_all_values`) were handled inconsistently by the UDS logic: the
effective-schema path merged them implicitly, but the persisted
`defined_schema_fields` treated them as ordinary fields competing for
`ZO_USER_DEFINED_SCHEMA_MAX_FIELDS` slots.
Since `_all` is created by the UDS packing itself (`refactor_map`), it
never exists at auto-enable time, so every auto-enabled stream persisted
a full field list without `_all` and could never add it back (the
settings API rejects any list above the cap) — hiding `_all` from the
stream field list even though `SELECT _all` still worked.
## Fix
Internal columns are now implicit everywhere:
- **Auto-enable** (`src/core/src/schema.rs`): internal columns are no
longer persisted into `defined_schema_fields` and no longer occupy slots
in the `schema_max_fields_to_enable_uds` cap — all slots go to real user
fields.
- **Settings API** (`src/core/src/stream.rs`): the
`user_defined_schema_max_fields` check counts only non-internal fields.
Existing persisted lists containing internal columns are left as-is for
backward compatibility — they are simply exempt from the count.
- **Schema endpoint** (`GET /streams/{stream}/schema`): `uds_schema` now
merges internal columns in, so the field list always shows them — this
fixes already-affected streams retroactively with no settings change
needed.
- New helpers: `config::is_uds_internal_column()` and
`StreamSettings::uds_internal_columns()` (gated variants included per
`store_original_data` / `index_original_data` / `index_all_values`).
## Tests
- New: `test_uds_internal_columns` (config),
`test_generate_schema_for_defined_schema_fields_includes_internal_columns`
(core)
- Existing `schema::tests`, `stream::tests`, and api stream tests all
pass; `cargo clippy` and `cargo fmt --check` clean.
## Summary
- move the MCP protocol, tool discovery, schema simplification,
execution, and response filtering implementation into a new OSS
`openobserve-mcp` workspace crate
- enable the real MCP POST, GET, and DELETE handlers in community builds
- initialize the MCP tool catalog from the OpenAPI document for every
edition
- keep OAuth authorization-server discovery enterprise-only because it
depends on Dex
- add OSS handler and OpenAPI initialization coverage
Closes#13266
Design at: #13266
## Impact
Community users can connect MCP clients directly to OpenObserve and use
the OpenAPI-backed tool catalog. Enterprise builds continue to use the
same MCP implementation from the OSS workspace crate.
Companion PR openobserve/o2-enterprise#2206 removes the old private copy
and updates the enterprise workspace overlay.
Design at: #13235 (design described in this PR description)
## Summary
MCP tools are auto-generated from the OpenAPI spec, filtered/described
via the `x-o2-mcp` extension. Two gaps kept trace/session analysis from
working well over MCP:
1. **`GetSessionDetails` was not an MCP tool at all.** The endpoint
(`/{org_id}/{stream_name}/traces/session/details`) was registered in the
axum router but missing from the `paths()` list in `openapi.rs`, so it
never entered the OpenAPI spec and was invisible to MCP. Without it an
LLM can list sessions but cannot drill into a session's per-turn traces.
2. **The traces endpoints had no `x-o2-mcp` metadata.**
`GetLatestTraces`, `GetLatestSessions`, `GetLatestUsers`, `GetTraceDAG`
were exposed only with their long OpenAPI descriptions and no
`category`, hurting `tool_search` (BM25) discoverability and wasting
tokens.
## Changes
- Register `get_session_details` in `openapi.rs` → becomes MCP tool
`GetSessionDetails`
- Add `x-o2-mcp` with `category: "traces"` and short descriptions to
`GetLatestTraces`, `GetLatestSessions`, `GetSessionDetails`,
`GetLatestUsers`, `GetTraceDAG`
- Mark `GetLatestTraces` as `pinned: true` so it appears directly in
`tools/list` (alongside
SearchSQL/StreamList/GetSchema/QueryRangeMetrics/GetIncident), letting
LLMs start trace analysis without a `tool_search` round-trip
No behavior change for regular HTTP API consumers; `GetSessionDetails`
additionally becomes visible in Swagger.
## Testing
- `cargo check -p openobserve-api-query -p openobserve-api` passes
## Problem
Every rebuild of `openobserve-api` re-runs the `RustEmbed` macro
expansion, which embeds `web/dist` (53MB, 1035 files) and compresses
every file with gzip + brotli(q11) — single-threaded, non-incremental,
and it peaks at 8GB RSS. `-Ztime-passes` attributes ~540s of the 769s
api compile to `expand_crate`. Net effect: touching one file in
`openobserve-core` costs a **16m45s** rebuild on an M-series laptop, and
neither incremental compilation nor codegen-unit tuning can help,
because macro expansion happens before both.
## Fix
Move the `#[derive(RustEmbed)]` into a new `src/web` leaf crate with no
workspace dependencies. The embed + compression now only recompiles when
`web/dist` changes; edits anywhere in the backend no longer pay for it.
**Measured (same machine, default profile settings, `--profile
release-profiling`):**
| | before | after |
|---|---|---|
| `openobserve-api` compile | 769s | 231s |
| touch `src/core` → binary | 16m45s | **6m44s** |
## Also in this PR
- **`src/web/build.rs`**: the derive has no file tracking, so a rebuilt
`web/dist` silently shipped stale embedded assets until something else
dirtied the api crate. `rerun-if-changed` on the embed folder fixes
that. The folder is parsed from the `#[folder]` attribute rather than
hardcoded, so it stays correct when the e2e workflows rewrite it. A
missing `index.html` now prints a build warning instead of silently
embedding nothing (the #13197 failure mode).
- **Playwright workflows**: the embed-folder `sed` now targets
`src/web/src/lib.rs` and is guarded with `grep -q`, so a future path
drift fails at the sed step instead of silently shipping an empty UI and
failing at the login page (see openobserve/o2-enterprise#2175 for that
incident).
- **`deny.toml`**: allow the new crate's AGPL-3.0 license (CI runs
`cargo deny check licenses`, which requires a per-crate exception).
## Validation
- `cargo deny check licenses` passes
- Smoke-tested the binary: `/healthz` ok, `/web/` serves `index.html`,
JS assets return 200 from the embedded bundle
- Enterprise overlay resolves: `cargo metadata` passes with the
companion `Cargo.toml.openobserve` change
## Companion
openobserve/o2-enterprise#2184 carries the overlay + e2e workflow sync
on the **same branch name** (`perf/web-embed-crate`); the two should
merge together.
## Summary
- add the `openobserve-api-management` crate under `src/api_management`
- physically move management handlers, models, and authentication
helpers out of `openobserve-api`
- preserve the existing API module paths through re-exports
- move shared stream permission checking into `openobserve-core` to
avoid a management-to-query dependency
- allow the new crate's `AGPL-3.0` license in `deny.toml` (fixes the
cargo-deny `check` job)
- keep router, OpenAPI generation, and rate-limit path mapping in
`openobserve-api`
## Dependency boundary
`openobserve-api-management` depends on `openobserve-api-common` and
`openobserve-core`; it does not depend on `openobserve-api-query`.
## Note on `split-debuginfo`
Removing `split-debuginfo = "packed"` from `[profile.release-profiling]`
is intentional (not a formatting artifact): profiling builds now keep
DWARF debuginfo embedded in the binary, so profiling artifacts can be
symbolized at runtime without shipping a separate debuginfo file. The
o2-enterprise overlay `Cargo.toml.openobserve` is kept in sync.
## Companion PR
- openobserve/o2-enterprise#2175
## Validation
- OSS check with `mimalloc`
- OSS strict clippy with all targets
- enterprise check and strict clippy
- cloud check and strict clippy
- `cargo test -p openobserve-api-management --lib`: 300 passed
- `cargo deny check licenses`
- `cargo fmt --all`
- `git diff --check`
Build-time benchmarking is intentionally deferred for this stage.
## Summary
- add `openobserve-api-common` for HTTP extractors and request DTOs
shared across API domain crates
- add `openobserve-api-query` and physically move the search, PromQL,
and traces handlers (about 16.4K lines)
- preserve the existing `handler::http::request::{search,promql,traces}`
paths through re-exports so routing, Swagger generation, and rate-limit
metadata remain compatible
- forward enterprise, cloud, vectorscan, and vortex features through the
new crate boundary
- avoid `#[path]` source inclusion
## Enterprise overlay
Requires openobserve/o2-enterprise#2174, which registers the two new
workspace crates in `Cargo.toml.openobserve`.
## Validation
- `cargo check -p openobserve --features mimalloc`
- enterprise check with `--no-default-features --features
enterprise,mimalloc`
- cloud check with `--no-default-features --features cloud,mimalloc`
- OSS and cloud/enterprise all-targets Clippy with `-D warnings`
- `openobserve-api-common`: 9 tests passed
- `openobserve-api-query`: 136 independent tests passed single-threaded;
the unchanged DB-dependent saved-view test was excluded because
standalone crate tests do not initialize the infra database
- `cargo fmt --all -- --check`
Build-time benchmarking is intentionally deferred until the crate-split
series is ready for a controlled `release-profiling` comparison.
Design at: #13200
## What changed
Adds a max-age admission policy for the file download cache:
- New configs `ZO_DISK_CACHE_MAX_AGE_DAYS` and
`ZO_MEMORY_CACHE_MAX_AGE_DAYS` (default `0` = disabled, current behavior
unchanged).
- `queue_download()` now skips files whose `max_ts` is older than the
configured max age — they never enter the download queue; queries read
them directly from object storage (the existing cache-miss fallback).
- The `cache_latest_files` gRPC event handler applies the same filter
when collecting files, covering the batch `download_from_node` path that
writes to the cache via `file_data::set` without going through the queue
(e.g. merged files produced by compaction of old partitions).
- The check is exposed as `file_downloader::exceeds_cache_max_age(ts,
cache_type)`; `ts <= 0` (unknown timestamp) is never skipped.
## Why
`time_lru` has an eviction policy but no admission policy. With a cache
sized for the recent hot window (e.g. 3 days), a query on old data (e.g.
30 days ago):
1. Triggers gc **before** insert: the first old-file insert evicts
`max(release_size, data_size * 100)` — by default ~10% of the cache —
from the oldest hours of the *hot* window.
2. Once inserted, the old files themselves become the oldest `time_lru`
bucket and are evicted first, often within the same query — S3 download
→ disk write → immediate delete, pure wasted work with zero retention
for repeat queries.
With a max age matching the cache window, deep-history queries no longer
disturb the hot cache or waste S3 bandwidth and disk IO.
## Coverage
All query-side download paths funnel through `queue_download()` with the
file's real `max_ts` and are covered: search (parquet), tantivy index
(`.ttv`), promql/metrics, and file_list dump. Deliberately **not**
gated: compactor working downloads (`cache_remote_files` and
merge-output writes — required reads/writes, not speculative caching)
and the results/aggregations/metrics_results caches (separate concern).
## Tests
Unit tests for the age check (disabled, unknown ts, recent file, old
file); `cargo test -p openobserve-core --lib file_downloader` passes.
## Problem
The web UI returns **HTTP 404 for all `/web/*` paths** on builds from
current `main`. The router process itself is healthy (`/config`,
`/healthz` return 200; `/` redirects to `/web/`) — only the UI is
missing.
## Root cause
The static UI assets are embedded at compile time in
`src/api/src/handler/http/router/ui.rs`:
```rust
#[derive(RustEmbed)]
#[folder = "web/dist/"]
struct WebAssets;
```
`rust-embed-for-web` resolves a relative `folder` against the crate's
`CARGO_MANIFEST_DIR`. When the HTTP handlers were moved into the new
`src/api/` sub-crate, this path was left as `web/dist/`, so it now
resolves to `src/api/web/dist/` — which does not exist.
`rust-embed-for-web` does not fail the build on a missing folder; it
simply embeds **zero** assets. The binary compiles cleanly and ships
with an empty UI, so `WebAssets::get()` returns `None` and the handler
returns `404 Not Found` for every asset (including `index.html`).
Previously (before the `src/api/` move) this file lived in the root
crate, where `web/dist/` correctly resolved to the workspace-root
`web/dist/`.
## Fix
Point the embed folder back at the workspace-root `web/dist` relative to
`src/api/`:
```rust
#[folder = "../../web/dist/"]
```
## Notes
- The Playwright workflows already `sed` this path to an absolute
`/tmp/dist/`, which is why CI didn't catch the regression — the
production build relies on the relative path.
## Summary
Second step of the crate split (after #13168 which extracted
`openobserve-core`): move `src/handler` and `src/router` into a new
`openobserve-api` crate at `src/api`, so the HTTP/gRPC transport
frontend (routing, auth, extractors, OpenAPI generation) builds as its
own compilation unit.
All 155 moved source files are byte-identical moves (`git mv` only, zero
content changes). The real changes are limited to manifests and
scaffolding:
- **`src/api/src/lib.rs`** re-exports `openobserve_core::{common, job,
service}` (plus `cipher` under `enterprise`) at the crate root, so the
existing `crate::common::...` / `crate::service::...` paths inside the
moved files keep resolving without touching them.
- **`src/lib.rs`** re-exports `openobserve_api::{handler, router}`, so
external paths like `openobserve::handler::...` used by `main.rs` and
`tests/integration_test.rs` are unchanged.
- **Feature forwarding**: `enterprise` / `cloud` / `vectorscan` /
`vortex` / `profiling` on the root crate now forward to
`openobserve-api` as well.
- **`src/enterprise-stubs` → `src/enterprise`**: directory rename, plus
a new `o2_ratelimit` dependency-resolution stub (the api crate depends
on it under the `enterprise` feature; there was no stub for it before).
- `#![feature(variant_count)]` moves with its only use site
(`models/dashboards.rs`) into the api crate.
- **CI**: added `deny.toml` license exceptions for the two new AGPL-3.0
workspace crates (`openobserve-api`, `o2_ratelimit`); updated the
`ui.rs` path patched by the playwright workflows.
`src/api/Cargo.toml` is shared verbatim between OSS and enterprise
builds — only the root manifest is replaced by the enterprise overlay;
the `o2_*` deps resolve via `workspace = true` to either the stubs (OSS)
or the real crates (enterprise).
Companion PR: openobserve/o2-enterprise#2168
## Testing
- `cargo check --workspace` passes (OSS).
- Verified all moved files are 0-diff renames (`git diff -M90% --stat`).
- Verified every `crate::` reference in the moved code resolves through
the re-exports (`service`/`common`/`handler`/`job`/`router`/`cipher`),
with the single `crate::cipher` use correctly gated behind `enterprise`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
udev
Prabhat Sharma
Harsh Mahajan
Hengfei Yang
Huaijin
