Design at: #12620Closes#12620Fixes#12564
## What
Rewrites both tantivy histogram optimize-rule collectors following the
same approach as the recent `TopNCollector` /
`SingleSegmentDocIdCollector` rewrites, and adds timezone support to
both rules.
- `MultiHistogramCollector` (replaces the aggregation framework for
`SimpleMultiHistogram`)
- `SimpleHistogramCollector` (replaces
`tantivy::collector::HistogramCollector` for `SimpleHistogram`)
### Timezone support (#12564)
`histogram(ts, interval, tz)` rewrites to `date_bin(interval,
to_timestamp_micros(_timestamp + offset), origin)`; the optimizer's
`get_column_name` hit the `BinaryExpr` and silently skipped both rules.
Now:
- the visitors extract the fixed offset from that shape
(`get_timestamp_offset`), so the rules fire with a timezone
- `IndexOptimizeMode::SimpleHistogram` / `SimpleMultiHistogram` carry a
`ts_offset` field (no proto change — histogram modes never cross gRPC,
they're re-derived per node)
- bucket edges are computed in shifted wall-clock space and the
collectors bucket on `_timestamp + ts_offset`, so emitted keys match
`date_bin` exactly when merging with non-tantivy partials
## Testing
- New collector unit tests: dense vs sparse equality, ts_offset
shifting, bounds filtering, missing column / missing breakdown docs,
per-bucket truncation with tie-breaks
- Visitor tests now include real-DataFusion-plan timezone cases
(`'+08:00'` simple, `'-05:30'` multi) asserting the extracted shifted
bounds and offset
- `cargo test -p config` (1936 passed), plus `index_optimizer` /
`search::tantivy` / `tantivy_optimize_exec` suites in the main crate;
clippy clean
- Low cardinality: fast `20%` than main
```
histogram(_timestamp, '1h') AS ts, count(*) FROM default GROUP BY ts ORDER BY ts ASC
```
- Low cardinality: fast `30%` than main
```
histgoram(_timestamp, '1h') AS ts, breakdown, count(*) FROM default GROUP BY ts, breakdown ORDER BY ts ASC
```
- High cardinality: faster `5x` than main
```
histgoram(_timestamp, '1h') AS ts, breakdown, count(*) FROM default GROUP BY ts, breakdown ORDER BY ts ASC
```
## What changed
Adds stream-change confirmation dialogs across Traces tabs. When a user
changes the stream in **Service Graph** or **Services Catalog**, the
change is no longer applied silently — instead an `ODialog` prompts the
user to confirm, since changing the stream clears the existing query in
the Traces/Spans tabs.
## Why
Previously, stream selection was only synced one-way: changing the
stream in Traces/Spans propagated to all other tabs. Changing the stream
in Service Graph or Services Catalog did not sync back because those
tabs don't show a query — so doing so could erase an active query in
Traces/Spans without user awareness. This change ensures the user is
explicitly asked before clearing their query.
## Why
The custom spinner-based loading states were inconsistent across traces
views anddid not align with the logs TenstackTable skeleton pattern
already established inthe codebase. The new skeleton provides a unified,
low-friction loading experience — shimmer pills at column-aligned
positions that mirror real cell widths — and removes the need for each
consumer to write its own loading template.
## Data Sources - Linux & Windows Agent Installer Improvements
### What changed
**LinuxConfig.vue** and **WindowsConfig.vue** have been updated to
support environment-specific installation options. Both now show a
two-card selector so users can pick their deployment target before
copying the install command.
---
### Linux
Added an **AWS EC2** option alongside Generic Linux.
- Two clickable selection cards: Generic Linux and AWS EC2
- Switching cards updates the install command automatically
- EC2 card uses the new `linux/ec2/install.sh` script which includes:
- `resourcedetection/ec2` processor to read EC2 instance metadata
- `resource/hostname` processor to set `host.name` from the EC2 Name tag
- Amber prerequisite callout shown when EC2 is selected, explaining that
an IAM role with `ec2:DescribeTags` and `ec2:DescribeInstances` is
required
---
### Windows
Added an **AWS EC2** option alongside Generic Windows.
- Same two-card selection pattern as Linux
- EC2 card uses the new `windows/ec2/install.ps1` script
- Amber IAM prerequisite callout shown when EC2 is selected
- Updated Generic Windows to use the improved `install.ps1` which:
- Bumps OTel version from 0.90.1 to 0.111.0
- Switches download source back to official GitHub releases
- Removes deprecated `memory_ballast` extension
- Fixes processor perf counter instances from hardcoded `[1, 2]` to
`"*"` for all cores
---
### Agent scripts (openobserve/agents repo)
- `linux/ec2/install.sh` - new EC2-specific Linux installer
- `windows/install.ps1` - updated with OTel 0.111.0 and config fixes
- `windows/ec2/install.ps1` - new EC2-specific Windows installer
---
### Test plan
- [ ] Open Data Sources > Recommended > Linux, verify two cards render
correctly in light and dark mode
- [ ] Select EC2 card, verify IAM callout appears and command URL
changes to `linux/ec2/install.sh`
- [ ] Open Data Sources > Recommended > Windows, verify same behaviour
- [ ] Select EC2 Windows card, verify command URL changes to
`windows/ec2/install.ps1`
- [ ] Verify Generic selection on both shows the original install script
URLs
---------
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
Close#12345
## Problem
OTLP histogram metrics sent over HTTP/JSON (e.g. otel-collector with
`encoding: json`) created streams but stored 0 events, with no
`_count`/`_sum`/`_bucket` streams. Int-valued counters/gauges were
silently recorded as 0. The protobuf path was unaffected, which is why
this only surfaced for collectors explicitly configured with JSON
encoding.
## Root cause
`otlp_json` deserializes the body directly into opentelemetry-proto's
serde-derived types, which only partially implement the proto3 JSON
mapping (64-bit integers encoded as strings, enums by name, zero/empty
fields omitted). `Metric.data` is a flattened `Option` oneof, and serde
converts any inner deserialization error of a flattened `Option` into
`None` instead of failing — so the whole metric silently became `data:
None`, while the stream itself had already been created by the schema
setup. Same mechanism on `NumberDataPoint.value` turned string `asInt`
values into 0.
## Changes
- Upgrade opentelemetry stack 0.31 → 0.32, which fixes string-encoded
u64 handling for plain histograms and summaries upstream.
- Add `otlp_json_compat` normalizer for the gaps still present in 0.32
(each verified empirically, with unit tests):
- string `asInt` values on data points and exemplars
- `aggregationTemporality` as enum name
- `Exemplar`'s value oneof is not flattened in 0.32
- missing `serde(default)` on
`ExponentialHistogramDataPoint`/`Buckets`/`ValueAtQuantile`/`Exemplar`,
which rejected omitted zero/empty fields
- Count and log metrics whose data oneof failed to decode
(`partial_success.rejected_data_points`) instead of silently skipping
them, and stop overwriting specific partial-success error messages with
the generic retention message.
- Adapt to OTLP 0.32 API additions (`KeyValue.key_strindex`,
`AnyValue::StringValueStrindex`).
- Upgrade workspace reqwest 0.12 → 0.13, required because
opentelemetry-http 0.32 only implements `HttpClient` for reqwest 0.13
(`rustls` feature replaces `rustls-tls-native-roots` via the platform
verifier; `json`/`query`/`form` are now explicit features). The
`segment` crate is pinned to reqwest 0.12, so `TELEMETRY_CLIENT` moved
into `telemetry.rs` as a plain `reqwest::Client` that posts the Track
message itself, replicating `segment::HttpClient::send`.
- Remove the unused pyroscope profiling integration (optional feature,
dependencies, agent setup, `ZO_PROF_PYROSCOPE_*` config).
## Verification
- 10 unit tests in `otlp_json_compat` covering the full payload matrix
(collector-style string integers, enum names, omitted zero fields,
exemplars with and without trace context, already-normalized payloads,
malformed input).
- End-to-end: posted the issue's collector-style JSON payload to
`/api/default/v1/metrics` — `_bucket` (3 events, correct cumulative
counts with exemplars attached), `_count`, `_sum` populated for
histogram and exponential histogram, summary quantiles stored, `asInt`
counter stored as 42 instead of 0.
- `cargo clippy -- -D warnings` clean; lib test failures are
pre-existing local-environment issues that reproduce identically on
main.
## What changed
Fixed three interrelated issues in the Traces:
(1) duplicated search calls caused by redundant watchers and
programmatic triggers, and (3) `isDatetimeChanged` returning `false`
when switching from absolute to relative mode.
## Why
Separately, loading Traces in any mode triggered 2 redundant search API
calls — watchers on `searchObj.data.query` duplicated explicit trigger
calls at every query writer, mount-time filter restoration also called
`searchData()` despite `loadPageData()` already owning the initial
search, and the DateTime picker's mount replay fired a false positive
"change" that triggered an auto-run.
## Approach
- Added a `markProgrammaticDateChange()` / `userChangedValue` flag to
`DateTime.vue` so the SearchBar can distinguish user-driven date changes
from programmatic ones (redirect, metrics brush, mount replay), using
`nextTick` to reset the flag after the flush.
- Removed the `watch` on `searchObj.data.query` and the debounced
auto-run watchers on query/datetime — every writer of `data.query` in
the codebase already calls a search explicitly, so a watcher doubled
every trigger.
- Removed the `searchData()` call inside the `restoreFilters` block on
mount; the single mount search is owned by `loadPageData()`.
- In `SearchBar.vue`, gated the live-mode auto-run on `userChangedValue
=== true` and `datetimeChanged`, and removed the `searchObj.data.query =
newValue` writes in `applyFilters`/`removeFilter` (filter operations
already trigger search via emit).
- Fixed `isDatetimeChanged` to return `true` when the previous type was
`absolute` and the value is now `relative`, so switching mode always
registers as a change.
## Problem
`get_sre_agent_credentials` early-returned on the `Ok(record)` branch
when the SRE-agent SA DB row already existed, so `ensure_sys_rca_agent`
only ran for orgs whose SA was *missing*. If the startup backfill ever
failed for an org (FGA transient, OpenFGA not yet ready, etc.), there
was no runtime path to retry → every subsequent RCA trigger
short-circuited past the self-heal, the SA's OpenFGA tuples stayed
broken, MCP returned 0 tools, and the agent failed with `Tool
'GetIncident' not found`.
Also bumps the `sre_readonly_afolder_incidents` migration flag (`v1` →
`v2`) so the OpenFGA top-up patch re-runs once on clusters seeded by
older code — required to add the new `mcp:_all_{org}` grants from the
companion PR. Also bumps the reqwest timeout to o2-ai (300s → 600s) —
multi-turn RCA legitimately needs the headroom.
## Fix
Always run `ensure_sys_rca_agent` before the DB lookup. Idempotent, safe
to call every request, self-heals the SA + role state on the next
trigger after a transient backfill failure.
## Pairs with
openobserve/o2-enterprise#1885 — drops the equivalent early-return in
`create_and_assign_sre_readonly_role` and adds the missing `mcp:`
resource grants.
Resolves https://github.com/openobserve/o2-enterprise/issues/1901
Design at: #12596closes#12596
## Problem
`index_updated_at` is a single stream-level timestamp, bumped whenever
any field is added to `full_text_search_keys` or `index_fields`. The
tantivy optimize path (`SimpleCount` / `SimpleHistogram` /
`SimpleMultiHistogram` / `SimpleTopN` / `SimpleTopNMulti` /
`SimpleDistinct`) excludes every file with `min_ts < index_updated_at`,
because optimize results are used directly as final answers with no
DataFusion re-check.
Consequence: adding one index field to a long-retention stream knocks
every histogram/count/top-N over historical data off the fast path until
the cutoff ages out of retention — even when the queried fields have
been indexed since day one.
## Design
### Storage
`StreamSettings` gains `index_fields_updated_at: HashMap<String, i64>`,
recording when each field was added to `full_text_search_keys` or
`index_fields`:
- A field add stamps only genuinely-new fields. A no-op re-add of an
existing field no longer bumps anything (previously it pointlessly
bumped the stream-level value and disabled the index for all history).
- The stream-level `index_updated_at` is still bumped on every real add,
so older queriers in a mixed-version cluster keep the old conservative
behavior during rolling upgrades.
- A field removal prunes its entry (in `normalize_stream_settings`), so
a later re-add stamps a fresh time — files written in the gap lack that
field's index, and a single timestamp can't represent a gap.
- Bloom-only fields auto-folded into `index_fields` by normalization are
stamped like a normal add.
- The `reset-index-updated-at` CLI clears the map, since per-field
entries override the scalar.
- The empty map is skipped during serialization; legacy settings without
the key deserialize to an empty map. Fully backward/forward compatible,
no migration.
A field present in both `full_text_search_keys` and `index_fields`
shares one entry (last add wins). That is over-conservative for at most
one of the two uses and only in the rare case where the same field is
added to both lists at different times.
### Query
The cutoff in `flight.rs` is now computed from the fields the query
actually reads from the index — the optimize-rule target fields
(`SimpleTopN` / `SimpleTopNMulti` / `SimpleMultiHistogram` /
`SimpleDistinct`):
```
cutoff = max over referenced fields of:
BASE_TIME if the field is a default-enabled index field
else the per-field entry, else stream-level index_updated_at, else stream created_at
```
- Default-enabled index fields (`SQL_SECONDARY_INDEX_SEARCH_FIELDS`:
`trace_id`, `service_name`, `operation_name`, plus
`feature_secondary_index_extra_fields`) are indexed since the stream's
first file, so they always resolve to `BASE_TIME` and never constrain
the cutoff — even if they also carry a per-field entry.
- Index-condition fields (equality and FTS/`match_all`) are deliberately
**not** part of the cutoff: each file's tantivy schema is the per-file
source of truth there, so the timestamps add nothing.
- A query that references no per-stream fields — e.g. a count or
histogram without filters — gets cutoff `0`: its tantivy result only
depends on data every `.ttv` file has. The existing global
`ttv_timestamp_updated_at` / `ttv_secondary_index_updated_at` checks are
unchanged and still apply on top. This also covers the comment on #12596
that pure histograms don't need the `index_updated_at` check — it falls
out of the rule instead of being special-cased.
The regular filter-pushdown path is untouched: it never used
`index_updated_at` (it checks the actual per-file tantivy schema and
adds the filter back to DataFusion when a field is missing).
### Example
Stream created 2024-01 (so `trace_id` etc. indexed since then),
`user_id` added to `index_fields` on 2026-06-01:
| Query | Before | After |
|---|---|---|
| top-N on `service_name` (default field) | index only for files after
2026-06-01 | all indexed files (`BASE_TIME`) |
| top-N / distinct on `user_id` | files after 2026-06-01 | files after
2026-06-01 (correct: older files lack the `user_id` index) |
| `histogram(_timestamp)` / `count(*)` | files after 2026-06-01 | all
indexed files |
## Intentionally out of scope
The cutoff still conflates data time with index build time: a file the
compactor merges *after* a settings change carries a fresh, complete
index but keeps its old `min_ts`, so it stays off the optimize path.
Fixing that requires fetching `created_at` per file from file_list — one
more column over millions of rows on every query — to rescue roughly one
compaction cycle of data around the settings change, which self-heals
within the next hour. Not worth it.
## Summary
- Super cluster leader was hardcoding `0` for `took_wait_in_queue` in
all search responses
- Added `wait_in_queue` field to `ScanStats` proto and struct
- Follower now propagates its queue wait time via `ScanStats`
- Leader reads aggregated value (`max` across followers) instead of `0`
Single-cluster deployments were unaffected.
## Summary
Rewrites the tantivy `GROUP BY ... count(*) ... ORDER BY cnt LIMIT n`
optimization around a custom flat collector over columnar term ordinals,
replacing the nested `TermsAggregation` approach from #12348 that was
disabled in #12528 for performance. One unified `SimpleTopN` path now
serves **1..=4 group by fields** at every layer (SQL matcher, internal
enum, wire protocol, collector, result types).
Closes the performance issue documented in
https://github.com/openobserve/openobserve/pull/12348#issuecomment-4659231020
and re-enables multi-field group by.
## Design
**`TopNCollector`** (`config::tantivy::query::topn_collector`), per
file:
1. read each group field's term ordinal per doc, fetched in blocks via
`Column::first_vals` (vectorized for full columns)
2. count into a **dense per-ordinal array** indexed by the mixed-radix
ordinal when the fields' dictionary-size product fits 4M cells (no
hashing — ordinals are dense), falling back to a hash map keyed by
bit-packed ordinals (`u64` for 1-2 fields, `u128` for 3-4) beyond that
3. keep groups per the exactness rule below, selecting top-k with a
bounded heap directly off the counting array (no full materialization)
4. resolve only the surviving ordinals to strings, in one sorted forward
pass per field via `Dictionary::sorted_ords_to_term_cb` (point lookups
with `ord_to_term` re-scan a dictionary block per call and were the
dominant cost)
DataFusion re-aggregates the per-file partials and applies the final
ORDER BY + LIMIT.
## Exactness contract
Per-file top-K partials are only exact when nothing is truncated; once
truncation kicks in, a group's total can be spread thin across files and
the merged top-N is approximate. Measured on near-uniform data, no
practical keep size restores accuracy (keeping 100k of 1.4M groups per
file still matched 0/5 of the true top-5), so the rule is bimodal,
decided per file at harvest where the distinct count is known for free:
- **distinct groups <= max(limit-derived top-k,
`ZO_INVERTED_INDEX_TOPN_MAX_GROUP_NUM`)** (env default 1000): the file
returns all groups — its contribution is **exact**
- **beyond**: keep only the limit-derived top-k — **approximate but
fast**; correct for the skewed distributions top-n queries target (true
heavy hitters survive), and the env lets deployments trade speed for
accuracy (raising it above the per-file cardinality reproduces exact
counts; verified)
## Wire format
`SimpleTopN` now carries `repeated string fields` (wire-compatible with
the former singular `field`: same tag, length-delimited) and
`SimpleTopNMulti` is removed with its oneof tag reserved — it only
existed unreleased since #12348.
## Benchmarks
112.7M rows, 72 files, single node, mimalloc build,
`ZO_QUERY_THREAD_NUM=12`:
Group By | Cardinality | Limit | Parquet | this PR | Winner
-- | -- | -- | -- | -- | --
path | low (16) | 1000 | 259ms | 48ms | 🟢 Tantivy 5.4×
trace_id | high (112M) | 1000 | 2459ms | 245ms | 🟢 Tantivy 10.0×
trace_id | high (112M) | 10000 | 2450ms | 466ms | 🟢 Tantivy 5.3×
trace_id, path | high × low | 1000 | 2385ms | 610ms | 🟢 Tantivy 3.9×
trace_id, path | high × low | 10000 | 2400ms | 899ms | 🟢 Tantivy 2.7×
trace_id, server_name | high × high | 1000 | 2380ms | 723ms | 🟢 Tantivy
3.3×
trace_id, server_name | high × high | 10000 | 2403ms | 1624ms | 🟢
Tantivy 1.5×
Correctness verified against DataFusion (`sum(1)` baseline): exact count
match for every query within the exact regime, including cross-segment
merging and the legacy-file (missing column) guard.
## Notes
- files whose tantivy index predates a field being added to
`index_fields` log a warning and contribute nothing, same convention as
the histogram handlers
- a debug log marks per-file truncation so operators can see when a
result enters the approximate regime
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Summary
Fixes duplicate error notifications when NL-to-SQL generation fails due
to 403. The `throw` in `generateAndUpdateQuery` was caught by the catch
block which showed a second generic "Failed to generate SQL query"
notification on top of the correct Unauthorized message. Now returns
early for auth errors, matching the pattern used in alerts.
## Before
User sees two notifications:
1. "Unauthorized Access: You are not authorized..."
2. "Failed to generate SQL query. Please try again."
## After
User sees only:
1. "Unauthorized Access: You are not authorized..."
## Test plan
- [ ] User without AI access sees only one error notification (the
unauthorized message)
- [ ] Non-auth errors still show the generic "Failed to generate SQL
query" message
- [ ] Auth errors don't trigger SearchBar error handling via re-throw
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## What changed
Fixed three issues in the Trace Graph:
- **Multi-root traces no longer crash** — traces where root spans
belonged to distinct services, the same service, or had cyclic service
relationships now render correctly instead of showing an empty graph.
- **Overlapping node labels are truncated** — long service names in
chart nodes are capped at 24 characters with an ellipsis, preventing
overlap with the next tree level.
- **Tooltip now shows the service name** — the pattern node tooltip
header was missing the service name, making it impossible to seethe full
name of a truncated node on hover.
Also added `truncateText` utility to `zincutils.ts` and comprehensive
test fixtures covering all multi-root scenarios.
## Summary
- Adds a single GeneralTests spec (`edition-features.spec.js`) that
validates the header edition button, the `EnterpriseUpgradeDialog`,
every enterprise feature card, and the `o2.ws/*` documentation links
each card opens.
- Two tests live in the spec; exactly one runs per environment. The
active edition is detected from the rendered header button label (most
direct signal of what the user sees, tolerates frontend/backend
build-flag mismatches like an OSS frontend served by an ENT backend).
- Pairs with the same-named branch on `openobserve/o2-enterprise` so ENT
CI tree-merges this spec in and runs the ENT variant.
## What the spec validates
- Header button label per edition (`Get OpenObserve Enterprise (Free)`
vs `Edition: Enterprise`)
- Hero panel (title, badge, primary button, features title, Learn More
button)
- 20 enterprise feature cards (presence + external-link icon + BETA/HA
badges)
- Cloud-only `Bring Your Own Bucket` card stays hidden outside Cloud
- Every feature card click opens a popup that lands on the expected
`openobserve.ai` doc page
- Hero external link buttons (`Download Now`/`Learn More` on OSS, `Learn
More` on ENT) — `Get Free License` intentionally not clicked because it
switches the active org to the meta org
## Files
-
`tests/ui-testing/playwright-tests/GeneralTests/edition-features.spec.js`
(new)
- `tests/ui-testing/pages/generalPages/editionFeaturesPage.js` (new —
page object + truth table of slugs / expected doc paths)
- `tests/ui-testing/pages/page-manager.js` (register
`pm.editionFeaturesPage`)
## Test plan
- [x] OSS local run: 1 passed, 1 skipped (~2.0m)
- [x] ENT local run: 1 passed, 1 skipped (~2.0m)
- [x] Sentinel audit passed (POM compliance, no raw selectors in spec,
no hardcoded secrets, no console.log)
- [ ] OSS CI green on this PR
- [ ] ENT CI green on companion PR
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Adds support to show tentative cost in usage page in cloud
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
## Summary
When the AI chat endpoint returns HTTP 403, `generateSQL` returned
`null` without setting `streamingResponse`, so `CodeQueryEditor` fell
through to the generic "Failed to generate SQL query" message. Now sets
`UNAUTHORIZED_MESSAGE` so the UI displays the proper error.
## Why
The earlier auth fix (#12558) only handled SSE error events inside the
stream. But when the middleware blocks the request with HTTP 403,
`!response.ok` triggers before the SSE reader starts — leaving
`streamingResponse.value` empty. `isAuthError('')` returns false, so the
code falls to the generic message.
## Test plan
- [ ] User without AI access sees "Unauthorized Access" when using
NL-to-SQL generation
- [ ] Non-auth HTTP errors continue to show generic "Failed to generate
SQL query"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## Summary
Fixes - https://github.com/openobserve/o2-enterprise/issues/1892
- Added explicit permission check in `chat` and `chat_stream` handlers
that returns `MetaHttpResponse::forbidden("Unauthorized Access")` (HTTP
403) when the user lacks AI access, ensuring consistent 403 behavior
regardless of OpenFGA state
- Updated all 4 SSE error event handlers in `O2AIChat.vue` to recognize
authorization-related error messages and display the standard
"Unauthorized Access" message
## Why
When a user without AI permissions used the query editor, the auth
middleware could pass the request through (e.g., when OpenFGA was
disabled), and the agent service error would arrive as an SSE event
within a 200 response — bypassing the standard 403 flow. This fix
ensures the backend returns proper HTTP 403 and the frontend displays
the standard unauthorized message consistently.
## Test plan
- [ ] Verify non-root user without AI permission gets HTTP 403 from
`/api/{org}/ai/chat_stream`
- [ ] Verify non-root user without AI permission gets HTTP 403 from
`/api/{org}/ai/chat`
- [ ] Verify root users are unaffected (bypass permission check)
- [ ] Verify the frontend displays "Unauthorized Access: You are not
authorized to perform this operation..." for auth-related SSE errors
- [ ] Verify non-auth SSE errors continue to display normally
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
## What changed
Added a stream-selected guard before setting `searchObj.loading = true`
in the auto-query trigger in `SearchBar.vue`, and added an explicit
`searchObj.loading = false` reset in `Index.vue`'s `runQueryFn` when no
stream is selected.
## Why
When no stream is selected, auto-query is enabled, and the user switches
the date-time mode (relative → absolute → relative), `searchObj.loading`
was set to `true` by SearchBar but never reset to `false`. The
`runQueryFn` in Index.vue would early-return due to an empty
`selectedStream`, but did not clear the loading state, leaving the UI
stuck in an infinite loading spinner.
## Approach
- **SearchBar.vue** (line 2719): Added
`searchObj.data.stream.selectedStream.length > 0` to the condition that
triggers an auto-query on date-time change, preventing
`searchObj.loading = true` when there is no stream to query.
- **Index.vue** (lines 978–981): Inside `runQueryFn`, when
`selectedStream.length` is 0, explicitly set `searchObj.loading = false`
and `searchObj.runQuery = false` before returning, as a defense-in-depth
reset.
This PR add bloom filter for parquet again under a env
`ZO_BLOOM_FILTER_PARQUET_ENABLED=false`, if you want to generate bloom
filter for parquet set it to `true`.
> We no longer use it even you generate it.
## What changed
Enriches the ServiceGraph Node Detail Panel for **inferred services**
(uninstrumented dependencies discovered from span attributes —
databases, message queues, RPC backends, external APIs). Instead of
showing only a single operation for these services, the panel now
displays type-specific dynamic tabs (Hosts, Databases, Queries,
Destinations, URLs, RPC Services) using field fallback chains with
COALESCE expressions, filtered against the actual stream schema.
## Why
Inferred services (detected via `infer_service_name`) previously showed
only one operation in the panel because they're derived from
CLIENT/PRODUCER spans. There was no way to see the different servers,
databases, or URLs they connect to. This PR adds structured context by
inspecting peer attributes (`server_address`, `net_peer_name`,
`db_namespace`, `messaging_destination_name`, etc.) and grouping data
into meaningful tabs per service type.
## Approach
- **INFERRED_SERVICE_TABS registry** — maps each `service_type`
(database/queue/rpc/external) to its relevant tabs with fallback field
chains (e.g., `["server_address", "net_peer_name", "net_peer_ip",
"network_peer_address"]`)
- **SQL helper functions** — `buildCoalesceExpr` and `buildNullCheck`
generate COALESCE expressions for GROUP BY/SELECT and OR clauses for
WHERE filters
- **Schema-aware field filtering** — `resolveStreamSchema` fetches the
trace stream schema; each tab's fallback fields are filtered against it,
dropping tabs with zero surviving fields
- **Caller-aware operations** — Operations table gains a "Caller" column
for inferred services, showing which upstream service invokes each
operation
- **Filter construction** — Row clicks on inferred service tabs generate
`(field1 = 'val' OR field2 = 'val')` filters instead of single-field
equality
- **Metrics tab hidden** for inferred services — no meaningful metric
correlation for uninstrumented backends
- Non-breaking — existing single-field `resourceFilter` paths left
unchanged; `handleServicesCatalogViewTraces` normalizes legacy string
payloads
## What changed
- Extracted parser-limitation guards into `isParserLimitation()` and
refactored `useSearchQuery.ts` to **not block the search** when the PEG
parser flags a false-positive error — the squiggle still shows but the
query proceeds to the backend
- Added a 4th parser-limitation guard for `found="("` inside window
function clauses (covers `SUM(COUNT(*)) OVER (...)`, `NTILE(N) OVER
(PARTITION BY COALESCE(...)...)`)
- Unified all four `fetchRumEventsForTrace` queries to a consistent
±60,000,000 µs buffer around the trace window
- Added `allowedTracingUrls` config + `propagatorTypes` to the RUM SDK
config snippet in `FrontendRumConfig.vue`
- Added an "Explain" query dropdown menu item in the logs SearchBar
- Fixed various traces/RUM UI padding and border issues
(PlayerEventsSidebar, TraceDetails, TracesMetricsDashboard,
SessionViewer)
- Updated i18n keys across all 12 language files
## Why
Two valid SQL queries produced false-positive validation errors
(`SUM(COUNT(*)) OVER (...)`, `NTILE(...) OVER (PARTITION BY
COALESCE(...)...)`), and even after suppressing those, the query was
still **blocked from executing** by `useSearchQuery.ts`. The PEG parser
has known gaps that the DataFusion backend handles correctly — there was
no reason to block valid queries. The RUM event queries also had
inconsistent time buffers across different calls (some used ±10s, others
±60s), causing missed RUM events in trace correlation.
## Approach
- Extracted the 4 parser-limitation guards from `validateSql()` into a
pure sync `isParserLimitation()` function exported from
`sqlDiagnostics.ts`, then reused it in `useSearchQuery.ts` to let query
execution continue when only parser limitations are detected
- Replaced inline constants with a shared `RUM_TIME_BUFFER_US =
60_000_000` in `useRumSpanBuilder.ts` so all four
`fetchRumEventsForTrace` queries use the same ±60s window
- Added `{ timeout: 30000 }` to the aggregate mutation test that was
timing out on ~5,500 broken queries
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
## Summary
- **Root cause**: `alertManagement.js` used
`alert-list-search-across-folders-toggle-btn` but `AlertList.vue`
renders a plain `<button>` with
`data-test="alert-list-search-across-folders-toggle"` — no `-btn`
suffix.
- Fixed the same stale `-btn` selector in `dashboardPage.js` (dashboard
toggle is also a plain `<button>`).
## Failing tests fixed
- `alerts-regression.spec.js:128` — Bug #9967: PromQL alert creation
- `alerts-stream-switching-regression.spec.js:338` — Bug #11577: Alert
preview chart y-axis
Both were timing out at 45s trying to click the nonexistent
`*-toggle-btn` selector.
## Files changed
- `tests/ui-testing/pages/alertsPages/alertManagement.js` — 2 selector
fixes (lines 214, 481)
- `tests/ui-testing/pages/dashboardPages/dashboardPage.js` — 1 selector
fix + removed stale OSwitch comment
## Test plan
- [ ] Run `npx playwright test
playwright-tests/RegressionSet/alerts-regression.spec.js` locally
- [ ] Run `npx playwright test
playwright-tests/RegressionSet/alerts-stream-switching-regression.spec.js`
locally
- [ ] Verify CI Alerts-Regression job passes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
Extends the `auto-assign-metadata` workflow with three improvements:
- **Creator classification** — a new first step buckets every issue/PR
creator into one of three categories:
- `team` — 23-person engineering allowlist → full workflow runs (assign
self, milestone, add to project, set start date)
- `internal` — other org members not on the allowlist → silently skipped
- `stranger` — not an org member at all → assigned to designated
triagers
- **Start date** — on new issues, captures the project item ID and sets
the `Start Date` field in GitHub Project #19 to today's UTC date via
`updateProjectV2ItemFieldValue`
- **Stranger triage** (OSS only) — issues from external contributors are
assigned to `Shrinath-O2` + `Manazsharma` + `neha00290`; PRs are
assigned to `oasisk` + `hengfeiyang`
### Who gets what
| Category | Members | Issues | PRs |
|---|---|---|---|
| **Engineering team** | `bjp232004` `hengfeiyang` `prabhatsharma`
`uddhavdave` `oasisk` `haohuaijin` `Subhra264` `Loaki07`
`007harshmahajan` `chaitanya-sistla` `ktx-kirtan` `omkarK06`
`ktx-vaidehi` `ktx-abhay` `neha00290` `ktx-akshay` `ktx-sadhna`
`nikhilsaikethe` `YashodhanJoshi1` `ktx-riya` `mmosarafO2` `Shrinath-O2`
`ktx-mihir` | Assign self · milestone · project · start date | Assign
self · milestone · project |
| **Internal — skipped** | `sowjanyapuvvula` `ravenwh` `abvarun226`
`ysantosh` `luckykurhe` `achatterjee4` `akshaye-blip` `Udhankaran`
`jakeopenobserve` `Manazsharma` `manoj0881-git` `gorakhnathy7`
`simranquirky` `roniandgit` `sandydgenius` `javasks` `Yadumathur` | Skip
| Skip |
| **Stranger** | Anyone not in org | Assign `Shrinath-O2` +
`Manazsharma` + `neha00290` | Assign `oasisk` + `hengfeiyang` |
## Test plan
- [ ] Open a new issue as an engineering team member → verify
self-assigned, milestone set, added to project, Start Date populated
- [ ] Open a new issue as a stranger → verify assigned to Shrinath-O2 +
Manazsharma + neha00290 only
- [ ] Open a PR as an engineering team member → verify self-assigned and
milestone set
- [ ] Open a PR as a stranger → verify assigned to oasisk + hengfeiyang
only
- [ ] Open an issue as an internal (skipped) member → verify no changes
made
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
Makes the compactor's bloom-filter **build side** enterprise-only by
moving it into the `o2-enterprise` crate (see
openobserve/o2-enterprise#1889). The search-side pruner and shared
`infra::bloom` SBBF support are unchanged.
- Delete `src/service/compact/bloom_build.rs` and
`src/service/tantivy/bloom_builder.rs` — they now live in
`o2_enterprise::enterprise::bloom`.
- Gate the `merge.rs` build hook behind `#[cfg(feature =
"enterprise")]`, calling
`o2_enterprise::enterprise::bloom::compact::build_for_stream`. In
non-enterprise builds the hook is a no-op.
- Drop the now-unused `get_tantivy_directory` re-export from
`grpc::storage`.
## Dependency
Requires openobserve/o2-enterprise#1889 (the enterprise build is broken
until that lands).
## Test plan
- [x] `cargo check --bin openobserve` (OSS, default features) —
warning-clean
- [x] `cargo check --bin openobserve --features enterprise` — gated call
site resolves
- [x] `cargo check -p o2_enterprise`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- close https://github.com/openobserve/openobserve/issues/12533
it will improve all case when we use tantivy to get row_id, then give
parquet to execute.
# Performance
400M records, original size 750GB
## Q1
```sql
SELECT
kubernetes_namespace_name,
avg(code) AS cnt
FROM
default
WHERE
match_all('zinc')
GROUP BY
kubernetes_namespace_name ORDER BY cnt DESC LIMIT 10
```
the `zinc` apprear in 140M records, so after match_all('zinc'), tantivy
will return 140M row_ids.
### main
tantivy time: 3s
totol time: 4.5s
### this pr
tantivy time: 95ms
total time: 1.5s
## Summary
- TestDino quota (~75k/month) was being consumed at ~4x rate because
each failing test uploads one result entry per retry attempt (original +
3 retries = 4 entries per test)
- This PR adds deduplication: if all retry results share the same
`status` and error first-line, only the final attempt is uploaded
- Retries with differing status or error message are preserved in full —
they carry real diagnostic signal (e.g. a test that times out then fails
with a different error)
## How it works
```
getErrorFingerprint(result) → first line of error.message (stable across retries)
allIdentical = every retry has same status AND same error fingerprint
→ true: upload only results[last] (strips ~3 entries per test)
→ false: upload all results as before (keeps diagnostic data)
```
## Expected quota impact
Based on last month's data: ~65k attempts → ~16k (estimated ~75%
reduction for OSS Portal alone)
## Test plan
- [ ] Trigger a CI run that has failures with retries — confirm the
workflow log shows `stripped N identical retry results`
- [ ] Trigger a run with a genuinely flaky/environment-variable failure
— confirm all retries are preserved in TestDino
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- **13 Playwright E2E journey tests** for the Model Pricing feature
(\`/settings/model_pricing\`), consolidating all P0–P2 scenarios into
journey-based flows
- New page object \`ModelPricingPage\` under \`pages/generalPages/\`
- API cleanup integrated into the centralized \`APICleanup\` class and
called from \`cleanup.spec.js\`
- CI matrix updated: \`ZO_MODEL_PRICING_ENABLED=true\` env var added;
\`model-pricing.spec.js\` added to the \`GeneralTests\` job
- **Bug fix**: \`waitForBuildTabLoaded\` now polls the Monaco editor for
non-empty content (Phase 3), fixing a timing race where Build tab SQL
tests read an empty query string on slower CI machines
## Test coverage
| Journey | Scenarios covered |
|---------|------------------|
| List, tabs & search | Controls, tab switching, search, built-in tab |
| Create & validation | All form validation cases + successful save |
| Edit, cancel & delete | Full lifecycle |
| Duplicate & clone | Org model duplicate + built-in clone |
| Bulk actions | Select, export JSON, bulk delete |
| Toggle | Disable/re-enable with correct visual state |
| Test match dialog | Empty state, no-result, clear, match result |
| Import | Valid JSON, correction flow, cancel |
| Editor features | Multi-tier, pricing template, pattern examples |
| Pricing drawer | Overflow link → drawer with all price keys |
| Empty state | CTA when org has no custom models |
| Cost applied | LLM span matched → cost stored correctly |
| No matching model | Unknown model name → zero cost stored |
## Bug fix: Build tab query editor timing
\`waitForBuildTabLoaded\`'s Phase 2 (chart-type-selected signal) fires
as soon as \`initializeFromQuery\` sets the chart type — before
\`makeAutoSQLQuery\` has run. On slower CI machines this left
\`queries[0].query\` empty when tests read the editor, causing the "SQL
ON + Auto mode" and "Auto → Custom" tests to receive \`""\` instead of
generated SQL.
Phase 3 polls the Monaco editor
(\`[data-test="logs-search-bar-query-editor"]\`) for non-empty content,
confirming \`onBuildQueryGenerated\` has been called and the query is
ready.
## Test plan
- [x] All 13 model pricing tests pass locally
- [x] \`logsQueryBuilder-editor.spec.js\` (23 tests) — all pass with
Phase 3 fix applied
- [x] Sentinel code audit passed (no blocking issues)
- [x] API cleanup added to \`cleanup.spec.js\` via
\`apiCleanup.cleanupModelPricingModels()\` — 404-safe for instances
where the feature is disabled
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Abhay Padamani
Hengfei Yang
Omkar Kesarkhane
udevandsai nikhil kethe
ktx-vaidehi
dependabot[bot]
Chaitanya Sistlaandsai nikhil kethe
Harsh Mahajan
Manan Shukla
Loakesh Indiran
Neha PandClaude Opus 4.7
Yashodhan JoshiandYashodhan Joshi
Subhra264
Huaijin