Commit Graph
891 Commits
Author SHA1 Message Date
Abhay Padamani 03dfc88bfb feat: default fields for add Panel and metrics page (#12509)
Design at: dashboards/default-panel-fields
2026-06-09 08:39:09 +00:00
Sai Nikhilandudev 57e57a56c9 fix: import and export functionality for score and score configs in llm evals (#12517)
Co-authored-by: udev <uddhav@openobserve.ai>
2026-06-09 05:38:24 +00:00
Neha PandClaude Sonnet 4.6 146d5e7612 feat(tests): vortex file format test suite (scenarios 1–83) (#12472)
## Summary

- Adds **239 new tests** covering the vortex columnar file format across
78 scenarios from the test plan
- **`tests/api-testing/tests/vortex/`** — 9 test modules (83 tests):
format conversion, schema evolution, data integrity, FTS, downsampling,
API format awareness, server config, and performance/stress
- **`tests/api-testing/tests/query_agent/test_3_vortex.py`** — 156
tests: the full 155-query SQL correctness suite + scenario 39
(self-join) run post-flush against vortex files
- **`api-testing.yml`** — adds `--ignore=tests/vortex` to the OSS
default-set run; vortex tests require the enterprise binary with
`ZO_FILE_FORMAT=vortex` and are gated to enterprise CI only

## Test plan coverage

| Module | Scenarios |
|--------|-----------|
| `test_format_conversion.py` | 1–13, 7, 8, 9, 10 |
| `test_schema_evolution.py` | 14–25 |
| `test_data_integrity.py` | 55–62 |
| `test_fts.py` | 51–54 |
| `test_downsampling.py` | 47–50 |
| `test_api_format.py` | 71–73 |
| `test_config.py` | 66–67 |
| `test_performance.py` | 74–78 |
| `test_3_vortex.py` | 26–46 (155 queries + JOIN) |

## Enterprise CI

Two dedicated jobs added in the companion `openobserve/o2-enterprise` PR
on the same branch:
- **`api_query_agent_vortex`** — runs `test_3_vortex.py` with
`ZO_FILE_FORMAT=vortex`
- **`api_vortex_format_tests`** — runs the full `tests/vortex/` suite
with 4-phase server lifecycle (parquet→vortex→parquet→vortex+compaction)

OSS CI is unchanged except for the ignore rule — no vortex tests run
against the OSS binary.

## Test checklist

- [ ] `tests/vortex/` collects cleanly: `83 tests collected`
- [ ] `tests/query_agent/test_3_vortex.py` collects cleanly: `156 tests
collected`
- [ ] Enterprise CI `api_query_agent_vortex` job passes
- [ ] Enterprise CI `api_vortex_format_tests` job passes

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:57:17 +00:00
Abhay Padamani 8c23dd000e feat: multi-query support (#12374)
Design At:
https://github.com/openobserve/designs/tree/main/dashboards/multi-sql-query-support

Issue list : https://github.com/openobserve/openobserve/issues/11278
2026-06-08 10:51:37 +00:00
Shrinath RaoandClaude Sonnet 4.6 a6e833bb99 test: fix isQueryEditorExpanded selector and enableHistogram state detection (#12502)
## Summary

- **Streams-Regression** — `isQueryEditorExpanded()` used
`container.locator('.editor-fullscreen')` which searches *descendants*,
but Vue adds `editor-fullscreen` directly to the container element
itself. Fixed by using combined CSS selector
`.query-editor-container.editor-fullscreen`.
- **Logs-Regression** — `enableHistogram()` checked canvas visibility to
detect histogram state. Canvas is absent before the first query runs, so
it always assumed histogram was OFF and toggled it OFF when it was
already ON. This removed `[data-test="logs-search-result-bar-chart"]`
from the DOM (it is `v-if`-gated), breaking
`expectBarChartHasContent()`. Fixed by using
`logsQueryPage.isHistogramOn()` (reads button `data-state` attribute),
consistent with how `ensureHistogramState()` already works.

## Failing tests fixed
- `should allow query expand icon to work after navigating from stream
explorer (#9337)` — Streams-Regression
- `should not show blank histogram after cancelling query @bug-4315` —
Logs-Regression
- `collapsing/expanding index list should redraw histogram chart` —
Logs-Regression

## Test plan
- [ ] Ran `streams-regression.spec.js --grep "9337"` locally — passes
- [ ] Ran `logs-regression-bugs.spec.js --grep "blank histogram after
cancelling"` locally — passes
- [ ] Ran `logs-regression-bugs-2.spec.js --grep "collapsing/expanding
index list"` locally — passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 08:22:07 +00:00
Neha PandClaude Opus 4.7 008398163b test: add more queries to query regression (#12487)
## Summary
- Adds **100 new structural-pattern queries** (Q411-Q510) across all 13
categories, bringing the suite from 425 to **525 queries**
- Adds **cross-stream JOIN/UNION support**: `stream_offset` in
data_gen.py, `STREAM2` in conftest, dual-table DuckDB oracle
- 15 cross-stream queries (Q401-Q415) covering INNER/LEFT/RIGHT JOIN,
EXISTS/NOT EXISTS, UNION ALL across streams
- Bumps data_gen.py from 510 to 520 query capacity

## Changes
- **Infrastructure**:
[conftest.py](tests/api-testing/tests/query_agent/conftest.py),
[compute_counts.py](tests/test-data/query-agent/compute_counts.py),
[data_gen.py](tests/test-data/query-agent/data_gen.py)
- **New file**:
[cross_stream.json](tests/test-data/query-agent/queries/cross_stream.json)
— 15 queries
- **12 categories**: 85 queries across aggregation, basic_select,
combined, cte_subquery, date_time, full_text_search, histogram,
math_functions, pagination, string_functions, union, window

## Test plan
- [x] `compute_counts.py` generates expected results for all 525 queries
- [ ] Run `pytest tests/query_agent/ -v` against target environment (565
tests expected)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-08 04:38:40 +00:00
Shrinath RaoandClaude Sonnet 4.6 af7d0e6696 test: harden rerun check step against GitHub API failures (#12486)
## Summary

- Adds `|| echo '{"jobs":[]}'` fallback on the `curl` call so a network
error or GitHub API rate-limit response never produces an empty/invalid
variable
- Adds `2>/dev/null || echo "0"` on both `jq` calls so a malformed API
response (non-JSON error body) falls back to `"0"` instead of
propagating a non-zero exit under `-eo pipefail`
- Changes the arithmetic comparison to `[ "${CURRENT_ATTEMPT_JOBS:-0}"
-eq "${TOTAL_MATRIX_JOBS:-0}" ]` with `2>/dev/null` so a `null` jq
output can't trigger "integer expression expected" and kill the step

**Failure mode fixed:** On re-runs, multiple matrix jobs query the same
GitHub API endpoint concurrently with the shared `GITHUB_TOKEN`. A
rate-limit `403` or transient network hiccup returns non-JSON, `jq`
exits non-zero, and with `set -eo pipefail` the whole "Check if this is
a rerun" step fails — blocking the job before any tests run.

**Degraded behaviour:** When the API is unavailable, both counts fall to
`0`, `0 -eq 0` → `rerun_type=all` → step succeeds → all tests run (same
as a normal rerun-all). The TestDino optimisation is skipped but nothing
is blocked.

Companion PR: openobserve/o2-enterprise#<ENT_PR>

## Test plan
- [ ] Trigger a re-run of a workflow run — "Check if this is a rerun"
step should pass even if GitHub API is slow/rate-limited
- [ ] Verify `rerun_type` is still correctly set to `failed` when the
API responds normally

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 12:25:29 +00:00
Neha PandClaude Opus 4.7 0334dda9a3 test: add data variety and NULL stress to query agent data generation (#12485)
## Summary

Two high-value improvements to the query agent regression test data:

1. **Data variety** — Per-field rotation offsets (`qi*13 + i*7`) replace
the old uniform `idx % len(pool)` approach. Previously all fields picked
from the same pool position, creating only 5 fixed row shapes per query.
Now field-value combinations vary independently per field.

2. **NULL stress** — 8 fields (`bot_flag`, `info_tag`,
`upstream_error_code`, `page_url`, `endpoint_path`, `resource_path`,
`variant_tag`, `threat_flag`) get `None` for 40% of records per query.
This exercises `IS NULL` / `IS NOT NULL` / `COALESCE` paths that were
previously untested.

## Changes
- `data_gen.py`: Rewrote `make_record()` with per-field rotation and
NULL injection
- 24 queries: Adjusted WHERE/HAVING filters to match new data
distribution
- All 12 query JSONs: Regenerated expected results via
`compute_counts.py`

## Impact
- 400 queries all have results (18 pre-existing legacy/skip queries
unchanged)
- No change to record count (still 5/query, 2000 total) — test time
unaffected (~7 min)

## Test plan
- [x] `compute_counts.py` regenerates all expected results
- [ ] Run `pytest tests/query_agent/ -v --tb=short` against dev
environment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-07 08:11:47 +00:00
61aebff552 fix: refactor!: remove SQL mode toggle in favor of auto-detection (#12483)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-07 04:43:00 +00:00
Neha PandClaude Opus 4.7 5eb63395f1 test: broaden coverage with structural query templates (#12482)
## Summary
- Adds 200 new SQL queries (Q201-Q400) across all 12 categories,
extracted as structural templates from production `june6.sql` query
patterns
- All queries use the existing synthetic `FIELD_POOL` schema — zero real
stream/field names
- Bumps `data_gen.py` from 200 → 400 queries (4000 records total, 10 per
query)
- Introduces new structural patterns previously missing from the suite

## New patterns
- **CONCAT + COALESCE chains**: Multi-field string construction for
display messages
- **CASE WHEN inside histogram COUNT**: Platform mapping + conditional
aggregation
- **Multi-field LIKE / NOT LIKE OR chains**: Text search across 4-8
fields
- **NOT IN + IS NOT NULL**: Exclusion filters with null safety
- **Long IN lists**: 5+ values in WHERE clauses
- **Percentage calculations**: `100.0 * COUNT(CASE WHEN ...) / COUNT(*)`
patterns
- **approx_percentile_cont in subqueries**: P95/P99 in nested SELECTs
- **Histogram with breakdown**: `GROUP BY _time, breakdown` pattern
- **char_length CASE fallbacks**: `CASE WHEN char_length(f) > 0 THEN f
ELSE fallback END`
- **CTE with multiple subquery references**: Self-join and multi-CTE
patterns

## Distribution
| Category | New | Range |
|----------|-----|-------|
| aggregation | 25 | Q201-Q225 |
| basic_select | 25 | Q226-Q250 |
| combined | 20 | Q251-Q270 |
| cte_subquery | 25 | Q271-Q295 |
| date_time | 10 | Q296-Q305 |
| full_text_search | 20 | Q306-Q325 |
| histogram | 25 | Q326-Q350 |
| math_functions | 8 | Q351-Q358 |
| pagination | 7 | Q359-Q365 |
| string_functions | 15 | Q366-Q380 |
| union | 12 | Q381-Q392 |
| window | 8 | Q393-Q400 |

## Test plan
- [x] `compute_counts.py` generates DuckDB expected results for all 400
queries
- [ ] CI: 800 tests (400 × 2 phases) pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-07 02:31:06 +00:00
Neha PandClaude Opus 4.7 4d48bbb21b test: expand query agent regression suite (#12479)
## Summary
- Expands the SQL regression catalog from **155 to 200 queries** across
all 12 categories
- Adds **7 queries** extracted from abc.sql (Q156-Q162) — synthesized
from real-world OpenObserve SQL patterns with all data/fields/streams
generated from scratch
- Adds **230 auto-generated queries** (Q163-Q392) from templates to
cover query format diversity
- Both test phases pass: **200 memtable + 200 parquet = 400 total**

## Key changes
- **data_gen.py**: 43 new fields (record_type, site_name, latency_ms,
payload_size, alert_tags, phase_data, db_info, etc.), generates 3920
deterministic records
- **compute_counts.py**: new translations for regexp_like,
str_match_ignore_case, approx_distinct, json_get_str/json_get_json
chaining; empty-aggregate result detection (OO returns 0 rows vs
DuckDB's 1 NULL row)
- **conftest.py**: PID-based stream name suffix for cross-invocation
isolation (prevents data accumulation)

## Test plan
- [x] test_1_memtable.py — 392 passed
- [x] test_2_parquet.py — 392 passed
- [x] All 7 abc.sql queries pass (Q156-Q162)
- [x] No real stream names, field names, or data copied from external
sources

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 04:59:20 +00:00
0aab1ae261 feat: online evaluations (#12352)
- **feat(online-eval): add PipelineKind::Evaluation and dual realtime
pipeline support**
- **feat(online-eval): add provider, score_config, scorer, and
online_eval_jobs DB entities**
- **feat(online-eval): add config CRUD services and API handlers**
- **feat(online-eval): reconcile jobs into hidden evaluation pipelines**
- **feat(online-eval): add scorer preparation and execution backends**
- **feat(online-eval): write scores and evaluator traces to system
streams**
- **refactor(online-eval): remove legacy eval templates backend**
- **test(online-eval): add API integration coverage for eval resources**

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
2026-06-04 16:03:14 +00:00
Neha PandClaude Sonnet 4.6 73ce75c883 fix(tests): bump query_agent ingest timeout 120s → 300s (#12444)
## Summary

- ENT CI runners are slower than local/OSS runners — the
`ingest_query_agent_data` fixture was hitting the 120s timeout before
1550 records became searchable, causing all 310 query_agent tests to
fail at setup
- Bumped `wait_until` timeout to 300s

## Test plan
- [ ] OSS `api_query_agent_tests` passes as before
- [ ] ENT `api_query_agent_tests` no longer hits WaitTimeout on ingest

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-04 14:56:50 +00:00
ktx-mihir 004e320b6d fix: ux revamp add condition layout and crosslink combobox (#12443)
- Add condition layout fix
- drilldown close button alignment and size sm
- OCombobox item align center
- Crosslink combobox and input based on condition
- Actions timezone alignment
- Ai toogle expand shortcut enhance
2026-06-04 11:55:35 +00:00
aa81dc3f13 fix: OTable resize and add/remove column feature (#12433)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
2026-06-04 09:52:51 +00:00
Neha PandClaude Opus 4.7 1196045636 test: add org-level ingestion token API and Playwright tests (#12439)
PR #11691 introduces org-level ingestion tokens with o2oi_ prefix (37
chars). This commit adds comprehensive automated test coverage:

## API Tests (35 tests in test_ingestion_tokens.py)

### Token CRUD (9 tests)
- Create token returns o2oi_ prefix and 37-char length
- Create token persists name, created_by, enabled=true, is_default=false
- Create token with name only (no description) — description is empty
- Create token with hyphens and underscores allowed
- Duplicate name returns 400 with "already exists" message
- Empty name returns 400 with "cannot be empty" message
- Whitespace-only name returns 400 with "cannot be empty" message
- Name too long (>256 chars) returns 400 with non-empty error message
- Parametrized: special chars (spaces, @, ., /) rejected with
"alphanumeric" message

### List Tokens (3 tests)
- GET returns data array
- Created token appears in list
- Token values are unmasked in list response

### Enable/Disable (5 tests)
- Disable token via PATCH {enabled: false}
- Enable disabled token via PATCH {enabled: true}
- Double disable is a no-op (idempotent)
- Toggle nonexistent token returns 404
- DELETE returns 405 (not supported)

### Ingestion Auth — Org Token (6 tests)
- Ingest with org token (org_id:o2oi_token) succeeds
- Ingest with disabled org token fails (401)
- Ingest with nonexistent o2oi_ token fails (401)
- Ingest to wrong org with valid token fails (401)
- Org token blocked on search endpoint (401)
- Org token blocked on dashboards endpoint (401)

### Ingestion Auth — Backward Compat (6 tests)
- User password ingestion still works (pre-PR behavior)
- Non-o2oi_ service account token ingestion works (falls through to user
lookup)
- User password on search succeeds (200) — mirrors org token blocking
tests
- User password on dashboards succeeds (200) — mirrors org token
blocking tests
- Non-o2oi_ SA token NOT blocked by o2oi_ prefix check on search (≠401)
- Non-o2oi_ SA token NOT blocked by o2oi_ prefix check on dashboards
(≠401)

### Multi-token and Security (6 tests)
- Create 5 tokens, all visible in list
- Disable one token, others remain enabled
- SQL injection in token name rejected (special chars → 400)
- Empty name, whitespace name, too-long name (with message assertions)
- Special chars parametrized: spaces, @, ., /

## Playwright UI Tests (7 tests in ingestionTokens.spec.js)

- Ingestion Tokens tab renders with table and columns (P0)
- Create token via UI dialog with name and description (P0)
- Create token with empty name — button disabled, enabled on fill,
disabled on clear
- Revealed token dialog — copy token with clipboard verification
- Toggle token enable/disable from table actions (P0)
- Manage Tokens button on ingestion page navigates to IAM
- Ingestion page token selector dropdown shows available tokens

## Page Object (ingestionTokensPage.js)

- data-test selectors for IAM nav, token list, create dialog, toggle,
toast, revealed dialog
- Registered in centralized PageManager

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 08:03:53 +00:00
ktx-vaidehiandktx-mihir 2fcd670aac fix: ux revamp drawer to dialog for less fields (#12424)
Co-authored-by: ktx-mihir <mihir.patel@kiara.tech>
2026-06-04 07:17:43 +00:00
Shrinath Rao a647e6aee9 test: add retry_on_db_lock helper to support/wait.py (#12434)
## Summary

- Adds `retry_on_db_lock()` to `tests/api-testing/support/wait.py`
- Companion to the enterprise-side fix (openobserve/o2-enterprise#TBD)
that applies the helper across all RBAC entity base tests

## Root cause

The `ent_rbac` CI shard runs with `-n 2` (two pytest-xdist workers).
SQLite's single-writer model means parallel workers racing on write
operations produce transient `500 {"message": "database is locked"}`
responses (SQLite error codes 5 / 517). These surface as flaky assertion
failures across many PRs.

## Fix

`retry_on_db_lock(fn)` wraps an HTTP call and retries up to 3 times with
exponential backoff (0.5 → 1 → 2 s) only when the response is `500` with
`"database is locked"` in the body. Parallelism is preserved; the brief
retry window absorbs the time the other writer needs to commit and
release the lock.

## Test plan

- [ ] OSS CI passes with this branch
- [ ] Enterprise PR with the same branch name picks up the new utility
and applies it in all RBAC entity base tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-03 12:23:48 +00:00
Neha PandClaude Opus 4.7 eb7cce9b96 test: query agent SQL regression suite — 155 queries, sqllogictest-style, two-phase (#12384)
## Summary

155-query SQL regression test suite covering 12 categories. Adding a new
query requires zero test code changes — drop a JSON entry into the right
category file and run `compute_counts.py`.

**Sqllogictest-style comparison**: DuckDB pre-computes full expected
result sets (not just row counts). The test runner extracts actual
values from OO hits, sorts both sides, and compares cell-by-cell with 5%
relative float tolerance. This catches value-level regressions that
row-count-only checks miss.

Two-phase execution: **memtable** (pre-flush) then **parquet**
(post-flush + Tantivy). **310/310 pass**.

## Architecture

```
tests/
├── api-testing/tests/query_agent/
│   ├── conftest.py            # Shared fixtures, run_query(), tolerance comparison
│   ├── test_1_memtable.py     # Phase 1: all queries pre-flush (FTS skipped)
│   └── test_2_parquet.py      # Phase 2: post-flush (module-scoped fixture triggers flush)
└── test-data/query-agent/
    ├── data_gen.py             # Deterministic data generator (single source of truth)
    ├── compute_counts.py       # DuckDB oracle: computes expected result sets
    ├── base_ts_override.json   # Locks BASE_TS so compute and tests share same timestamps
    └── queries/                # 12 JSON files, 155 queries with expected results
```

### How it works

1. **Deterministic data** — `data_gen.py` generates 10 records per query
(1550 total) with a warehouse/supply-chain IoT schema. Each query Q{N}
gets isolated timestamps. `BASE_TS` is locked via
`base_ts_override.json` so compute and tests share identical timestamps
(prevents histogram bucket drift on minute boundaries).

2. **Unique stream per session** — `STREAM =
f"query_agent_test_{int(time.time())}"` prevents accumulated-data bleed
from prior runs.

3. **DuckDB oracle** — `compute_counts.py` loads the same dataset into
DuckDB, creates a pre-filtered VIEW per query (so time filters work for
CTEs, subqueries, UNIONs — no fragile SQL injection), translates
OO-specific functions, and writes `expected.results` blocks into the
query JSONs.

4. **Histogram translation** — `histogram(_timestamp, 'N unit')` is
translated to integer arithmetic matching OO's `date_bin` with origin at
2001-01-01T00:00:00 UTC. Verified against DataFusion CLI — 100/100
records match across all interval types (15s, 30s, 1m, 2m, 3m, 4m, 5m).
ISO timestamp post-processing converts epoch-µs bucket values to
`YYYY-MM-DDTHH:MM:SS` strings matching OO's serialization format.

5. **Sqllogictest comparison** — `run_query()` checks for
`expected.results`. When present, it extracts column values from hits as
strings, sorts both actual and expected (`results_mode: "rowsort"`), and
compares cell-by-cell with 5% relative float tolerance (handles
cross-engine floating-point drift in AVG/STDDEV/etc.).

6. **Two-phase testing** — `test_1_memtable.py` runs all queries
pre-flush (FTS sqllogictest skipped — Tantivy indexes don't exist yet).
`test_2_parquet.py` has a module-scoped `post_flush` fixture that
triggers `/node/flush`, re-polls until data is searchable, then runs all
queries with full assertions.

### Comparison modes

| Mode | Count | Mechanism |
|------|-------|-----------|
| **Sqllogictest** | 140 | Cell-by-cell comparison with 5% float
tolerance |
| **Skip sqllogictest** | 9 | DuckDB results for documentation, fall
back to legacy assertions |
| **Legacy** | 6 | 5 auto-interval histograms + Q122 (STRING_AGG
DISTINCT ORDER BY) |

**Skip sqllogictest queries** — known OO-DuckDB divergence:

| Query | Reason |
|-------|--------|
| Q037 | ROW_NUMBER tie-breaking differs between engines |
| Q058 | Self-join pairings non-deterministic across engines |
| Q072 | FULL OUTER JOIN histogram — NULL handling differs |
| Q080 | FTS `match_all` + histogram — Tantivy vs SQL LIKE |
| Q111 | ROW_NUMBER ties in pagination |
| Q117 | STRING_AGG ordering differs between engines |
| Q121 | STRING_AGG concatenation order |
| Q127 | UNION across 5 overlapping groups — boundary records compete
for LIMIT spots |
| Q152 | PERCENT_RANK tie-breaking differs between engines |

### Key design decisions

| Decision | Why |
|----------|-----|
| Full result-set comparison | Catches value-level regressions (e.g.,
ROUND producing wrong decimal) |
| `rowsort` mode | DuckDB and OO order ties differently — sorting both
sides eliminates spurious failures |
| 5% float tolerance | Cross-engine AVG/STDDEV/PERCENTILE differ at
least-significant digits |
| `''` = `'None'` normalization | OO serializes NULL as empty string;
DuckDB uses `"None"` |
| DuckDB VIEW for time filtering | WHERE injection into CTEs/subqueries
is fragile — VIEW filters at scan level |
| Dynamic columns excluded | `_timestamp` and `_time` differ between
compute time and test time |
| OO-dropped LAG columns excluded | `prev_tput`, `hours_2_back`,
`delta_from_prev`, `temp_diff` |
| Histogram → DuckDB integer math | DuckDB lacks `date_bin`; integer
formula verified via DataFusion CLI |
| BASE_TS override file | Prevents histogram timestamp drift when
compute and test run at different minutes |
| 1s time window margins | Minimizes cross-query data bleed while
capturing all 10 records |
| `pytest.fail()` on fixture failure | Stops all tests instead of
running 310 against an empty stream |

### Categories covered

| Category | Queries | Description |
|---|---|---|
| aggregation | Q001-Q020 | GROUP BY, COUNT, AVG, SUM, MIN, MAX, FILTER,
HAVING |
| basic_select | Q021-Q035 | WHERE, IN, BETWEEN, LIKE, ORDER BY,
DISTINCT, CASE |
| combined | Q036-Q045 | JOINs, multi-condition filters, combined
aggregations |
| cte_subquery | Q046-Q062 | CTEs, subqueries, derived tables, EXISTS |
| date_time | Q063-Q073 | Date parsing, formatting, extraction, interval
arithmetic, histogram |
| full_text_search | Q074-Q082 | match_all, match against fields |
| histogram | Q083-Q093 | histogram() bucketing, aggregation over
buckets |
| math_functions | Q094-Q102 | ROUND, SQRT, ABS, POWER, arithmetic
expressions |
| pagination | Q103-Q112 | LIMIT, OFFSET, combination with filters and
sorts |
| string_functions | Q113-Q122 | SUBSTR, CONCAT, UPPER, LOWER, TRIM,
LIKE patterns |
| union | Q123-Q134 | UNION ALL, UNION (dedup), CTE + UNION combinations
|
| window | Q135-Q155 | ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD, SUM/AVG
OVER |

## Test plan

- [x] All 155 queries pass in both phases (310/310)
- [x] 140 queries with sqllogictest cell-by-cell comparison
- [x] 17 histogram queries converted from legacy to sqllogictest
- [x] 9 queries with skip_sqllogictest for known OO-DuckDB divergence
- [x] 6 queries in legacy mode (auto-interval histograms + Q122)
- [x] BASE_TS locked via override file — no timestamp drift
- [x] Dedicated CI job (`api_query_agent_tests`) runs in parallel

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 09:30:02 +00:00
Hengfei YangandClaude Opus 4.8 ad9fef2242 fix: fuzzy tests (#12405)
- [x] Bulk ingestion panic
- [x] Create function panic
- [x] Dashboards/Alerts/Incidents panic when list data
- [x] Match http response error code
- [x] Skip some endpoint that fuzz can't generate data

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 05:12:57 +00:00
Shrinath RaoandClaude Sonnet 4.6 b6093cb996 test: repair two failing dashboard E2E tests (#12395)
## Summary

- **camelCase zero-value line chart** (`dashboard.spec.js:826`): SQL
referenced `kubernetes_namespace_name` / `kubernetes_container_name` /
`kubernetes_pod_id` with underscores, but `e2e_automate` stores those
fields with dot notation (`kubernetes.namespace_name`). DataFusion
returned "column not found", `errorDetail.message` was set,
`ChartRenderer` was unmounted (behind a `v-if`), and the test timed out
waiting for `[data-test="chart-renderer"]`. Fixed by switching to
`level` / `method` / `stream` — fields guaranteed to exist — with
impossible conditions so all series values are zero.

- **Sankey no-data test** (`dashboard-sankey.spec.js:400`): After
`waitForChartToRender()` there is an async gap between the apply button
re-enabling and Vue committing the "No Data" text to the DOM. The old
`waitForFunction` used `offsetParent !== null`, which fires immediately
for the empty zero-height no-data div (`position: absolute`, no content
→ 0px height). `isVisible()` then returned `false`, producing `Expected
true, Received false`. Fixed by replacing the custom logic with
`expect.poll()` that reads `textContent` — retrying until the text is
actually committed to the DOM.

## Test plan

- [x] Both tests verified passing locally against a hosted enterprise OO
instance (localhost:5080)
- [ ] CI run on `o2-enterprise` PR to confirm `Dashboards-Core` and
`Dashboards-Charts` jobs go green

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 04:49:40 +00:00
Shrinath RaoandClaude Opus 4.7 b0118090e1 test: api-testing follow-ups — pytest-xdist dep + standardize test user display name (#12393)
## Summary

Companion PR to o2-enterprise PR 1841 (api-testing follow-ups). All
ENT-side test/CI work lands there; this OSS PR carries the two changes
that must live on the OSS side because they touch shared code.

Both PRs live on the same-named branch (`test/api-test-followups`) so
ENT CI's cp-merge picks up this branch and sees both changes together.

## What's in this PR

### 1 — Add `pytest-xdist` dependency

Adds `pytest-xdist>=3.0` to `tests/api-testing/pyproject.toml`. Needed
by the ENT api-testing CI's `api_integration / ent_rbac` shard to run
RBAC tests in parallel (`-n 4`).

CI-verified speedup: **~70 s → 21.7 s (~69% reduction)** on the ENT RBAC
shard's 280+ tests, with zero failures.

Foundation only — no test code in this repo uses xdist yet. Available
for any future shard that wants it.

### 2 — Standardize automated-test user display name to "Automation
TestUser"

Two helper files in `tests/api-testing/helpers/workflow/pages/` had a
personal name (`"Shyam P"` / `"Shyam Panjiyar"`) hardcoded in their user
/ service-account create payloads. Those helpers are consumed by the
workflow chain (`helpers/workflow/create_objects.py` +
`edit_objects.py`), which doesn't reliably clean up its created users —
so the names persisted in the IAM users page and looked confusingly like
real teammates' accounts.

Files changed:
- `tests/api-testing/helpers/workflow/pages/user_page.py`
- `tests/api-testing/helpers/workflow/pages/serviceaccount_page.py`

Both now use:

```
first_name: "Automation"
last_name:  "TestUser"
```

Convention going forward: any account named `Automation TestUser` in the
IAM users page is automatically identifiable as test-created and safe to
clean up. Email addresses still use random suffixes
(`d4m21_{random_int}`) so uniqueness isn't affected — only the display
name standardizes.

## Test plan

- [ ] CI green on this branch
- [ ] Companion ENT PR 1841 (which depends on the `pytest-xdist` dep
added here) stays green

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 01:55:01 +00:00
ktx-akshay 47b3944774 test: restore missing visualize e2e test cases (#12347)
## Test Inspector Review

### Summary
- 0 Critical issues
- 0 Selector mismatches (14/14 validated against Vue source)
- 0 CI/Workflow issues
- 11/11 tests passed (localhost)

---

### ✅ Framework Compliance — PASS
| Check | Status |
|-------|--------|
| Raw selectors in spec | 0 critical (1 acceptable: dynamic panel name)
|
| console.log | None |
| Vacuous pass / soft assertions | None |
| testLogger | ✅ 14 calls |
| Tags | ✅ All 11 tests |
| waitForTimeout | 0 |

### ✅ Selector Cross-Validation — PASS
14/14 page object selectors verified against Vue source.
0 mismatches.

### ✅ CI/Workflow — PASS
All spec files present in OSS `.github/workflows/playwright.yml`.

### ✅ Test Execution — 11/11 PASSED (localhost, 1.2m)

*Reviewed by Test Inspector*
2026-06-01 11:28:08 +00:00
ktx-akshay e890b83d2c test: fix flaky dashboard end-to-end tests (#12375) 2026-06-01 07:57:19 +00:00
Shrinath RaoandClaude Opus 4.7 61aa772d57 test: revamp api-testing suite (framework + 16 file rewrites + CI matrix) (#12372)
## Summary

Comprehensive rework of the `tests/api-testing/` suite. 30+ commits
implementing all 6 phases of the [revamp
plan](../../tests/ui-testing/MD_Files/api%20tests%20revamp/API_TESTS_REVAMP.md).

**Before → After:**

| | Baseline | Final | Δ |
|---|---|---|---|
| passed | 421 | **443** | **+22 ✓** |
| failed | 3 | **2** | -1 (only same pre-existing) |
| skipped | 53 | 39 | -14 (unskipped dead tests now run) |
| xfailed | 1 | 2 | +1 (new visible backend-bug signal) |
| runtime | 588s | **387s** | **-201s (34% faster)** |

**Zero tests that passed in baseline are now failing.** The 2 remaining
failures are the exact pre-existing ones
(`test_enrichment_table_join.py` backend panic + `test_workflow.py` SMTP
env-miss).

## What's in the diff

- **New `support/` framework** — `OpenObserveClient`, endpoint wrappers
(`alerts`, `dashboards`, `streams`, `search`, etc.), `wait_until()`,
payload factories, pytest fixtures, SSE parser. 14 framework selfcheck
tests.
- **16 of 16 OLD-tier test files rewritten** — using the new framework,
unique-suffix names, body validation, `wait_until` instead of
`time.sleep`, parametrize where appropriate.
- **29 files moved into 12 feature folders** mirroring the UI suite
layout
(`tests/{alerts,dashboards,ingest,orgs,pipelines,regression,rum,search,streams,users,workflow}/`).
- **−6,500 LOC net** (copy-paste + dead code removal; new framework adds
back ~1,500).
- **Ruff config** in pyproject.toml: T201 ban on `print()`, PT pytest
rules, B bugbear, UP pyupgrade.

## CI workflow changes

- New job `api_regression_tests`: `strategy.matrix` over the 4 env-flag
combinations (UTF8_VIEW × JOIN_MATCH_ONE) — now runs in parallel instead
of as 4 sequential server reboots in the main job.
- New job `api_fuzz_tests`: schemathesis OpenAPI-driven fuzz against
`/api-doc/openapi.json` (214 endpoints tested). Informational
(`continue-on-error: true`). Each run surfaces **7–10 distinct 5xx
panics** (e.g., empty `sql` body returning 500 instead of 400, traces
endpoints crashing on unescaped path params that get interpolated into
SQL). Filed to backend team for separate triage.
- Server `o2_*.log` files uploaded as artifacts on failure (was:
silently lost).
- `api_tests_summary` now requires BOTH integration + regression to
pass.

### CI infrastructure fixes

A short hardening pass landed after the initial CI run surfaced several
issues:

- **pytest-sugar count inflation** — dropped `--force-sugar` from both
pytest invocations. The flag double-registered sugar on top of the
auto-loaded entry point, inflating the CI test counter ~3x (display
showed 1364 when collection was 486). Counter is now honest.
- **schemathesis 4.x flag migration** — `--base-url` → `-u`,
`--hypothesis-max-examples` → `-n`, `--hypothesis-deadline` →
`--request-timeout`, `--data-generation-method` → `-m`, dropped
`--auth-type basic`. The fuzz job was failing immediately on flag-parse
before this.
- **Regression matrix binary path** — used `${{ github.workspace
}}/target/debug/openobserve` so the absolute path resolves correctly
under `working-directory: tests/api-testing/` (was: every matrix job
failed with "No such file or directory").
- **`permissions: contents: read`** added to all 4 workflow jobs —
closes CodeQL alert #369.
- **Removed redundant server boot** in `api_integration_tests` (leftover
from main's structure after the regression matrix split; was silently
failing to bind to :5080 and getting masked by the first server).

## Backend bugs surfaced (for backend team to track)

The rework uncovered and visibly flagged these as `xfail`/documented
test comments:

1. `DELETE /api/organizations/{id}` returns 404 unconditionally even for
just-created orgs — endpoint appears unimplemented.
2. `test_dashboard_missing_required_fields` triggers backend panic
(returns 500 instead of 400/422).
3. JS `getEnrichmentTableData()` restricted to `_meta` org AND not
exposed in the `/functions/test` endpoint runtime — `ReferenceError`
even with explicit `enrichment_tables: [...]`.
4. Bulk API silently sanitizes invalid `_index` values (`****` → `_`)
without flagging — user data lands in unexpected index.
5. `AVG` of a single value drifts by ~1e-14 (likely float reduction
order; documented tolerance in `test_advanced_sql.py`).
6. Org name validation rejects hyphens (only
`A-Z/a-z/0-9/spaces/underscores`) — not in spec.

Plus 7–10 additional 5xx panics surfaced per fuzz run (input validation
gaps — separate triage list in fuzz job logs).

## Test bugs uncovered + fixed

- `test_pipeline.py`: had two function definitions named
`test_pipeline_creation_and_action_with_schedule` — Python's last-wins
meant the first test's body never executed for years.
- `test_alertsV2.py`: `assert alert_disabled == "False"` was indented
inside an `else:` branch that never executed because the response is
always a dict; AND `enabled` is a bool, not the string `"False"`. Two
layers of broken.
- `test_bulk.py`: variable typo (`valid_bulk_payload` reused on the POST
call) made the invalid-index test silently duplicate the valid case.
- `test_organisations.py`: `test_create_organization` used pre-pytest-4
`yield`-style fixtures — silently ignored by pytest 8.
- `test_js_result_array.py`: 8 tests skipped with `"Temporarily disabled
- failing test"` — the actual cause was that JS functions are restricted
to `_meta` org. One-line fix unskipped all 8.
- `test_fuzzy.py`: 4 tests `@pytest.mark.skip`-ed because they used a
1-minute time window against older static test data → 0 hits → flake →
skip. Wider window unskips them.
- 547 LOC of zombie commented-out test blocks deleted (15 commented `#
def test_` definitions, including one orphan case where the `def` was
commented but the indented body wasn't).

## Folder layout

```
tests/api-testing/
├── support/                    # NEW framework
│   ├── client.py               # OpenObserveClient
│   ├── endpoints/              # 7 per-resource wrappers
│   ├── factories.py            # unique_name, time_window, *_payload
│   ├── wait.py                 # wait_until + WaitTimeout
│   ├── sse.py                  # read_sse_response
│   └── fixtures.py             # client, temp_user_email, temp_stream_name, ...
├── helpers/                    # Non-test shared code (was scattered in tests/)
│   ├── sourcemap_helpers.py
│   └── workflow/               # workflow chain + pages/ live here now
├── fixtures/
│   └── rum/                    # Standalone RUM ingest scripts (was in tests/)
└── tests/
    ├── alerts/                 # 3 files
    ├── dashboards/             # 4 files
    ├── ingest/                 # 4 files
    ├── orgs/                   # 2 files
    ├── pipelines/              # 10 files
    ├── regression/             # 2 files (env-flag matrix targets)
    ├── rum/                    # 3 files
    ├── search/                 # 8 files
    ├── streams/                # 1 file
    ├── users/                  # 1 file
    ├── workflow/               # 3 files
    ├── conftest.py
    └── test_framework_selfcheck.py
```

## Test plan

- [x] `rye run pytest --collect-only` — 486 tests discovered
- [x] Full default-set run (`--ignore=tests/regression`) — 443 pass / 2
fail / 39 skip / 2 xfail in 387s
- [x] `rye run pytest -m framework` — 14/14 selfcheck tests pass
- [x] `rye run ruff check support/ tests/test_framework_selfcheck.py` —
clean
- [x] CI run on this PR — `api_integration_tests` + all 4
`api_regression_tests` matrix jobs green
- [x] CI run — `api_fuzz_tests` runs cleanly (informational; surfaces
7–10 unique 5xx panics per run)

## Note on test count

The `test_streaming.py` rewrite folded 7 "tests" (`test_ingest_data`,
`test_ingest_join`, `test_disable_streaming`,
`test_update_max_query_range`, `test_enable_streaming`,
`test_delete_stream`, `test_delete_stream_join`) into pytest fixtures.
Each only checked `status_code == 200` of a setup/teardown operation —
they were imperative steps masquerading as tests. The fixtures still
perform the same operations and assert their success; they're just no
longer counted as discrete pytest items. **Functional coverage is
identical.**

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 07:19:48 +00:00
Subhra264 9a1ed606aa feat: org level ingestion token (#11691)
Design at: observability/org-level-ingestion-token
2026-06-01 06:12:06 +00:00
Shrinath RaoandClaude Sonnet 4.6 859f5be016 test: fix clickScheduledSearchRow selector for logs-regression-ent tests (#12371)
## Summary
- `clickScheduledSearchRow` in `logsPage.js` was waiting for
`[data-test="search-scheduler-table-{traceId}-row"]` which never exists
in the DOM
- `OTableBodyRow` renders rows as `data-test="o2-table-row-{index}"`
(zero-based integer), not by trace_id
- Fixed to wait for `o2-table-row-0` (first row) and click it — the
table has `expand-on-row-click="true"` so no separate expand button
click is needed
- Timeout bumped from 10 s → 30 s to allow the API data fetch to
complete

## Failing tests fixed
- `logs-regression-ent.spec.js` › No Undefined Length error after
navigating Logs -> Scheduled Search -> Streams -> Logs (`@P0`)
- `logs-regression-ent.spec.js` › No console errors after navigating the
reproduction path (`@P1`)

## Test plan
- [x] Both tests run and pass locally against the enterprise binary (`2
passed (1.0m)`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 16:07:30 +00:00
Shrinath RaoandClaude Sonnet 4.6 3e172523d5 test: sync regression workflow runners and TestDino with playwright.yml (#12370)
## Summary

- **Runners**: `merge_and_upload_reports` and `playwright_summary` now
use `ubicloud-standard-4` (was `-8`), matching `playwright.yml`
- **TestDino rerun optimization**: added `check_rerun` step + full
`last-failed` logic (same as `playwright.yml`) — on re-run-failed, only
the previously failed test in each shard is re-executed
- **Upload step**: added failures-only filter (`node -e` filterSuite),
`startTime` ISO 8601 conversion, and `--verbose`; now uploads
`report-failures-only.json` instead of the full report

## Test plan
- [ ] Trigger a regression run and confirm it completes with the new
runner sizes
- [ ] Intentionally fail a test, re-run failed jobs, and verify TestDino
optimization runs only the failed test
- [ ] Confirm Upload to TestDino step produces a filtered report

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 12:54:33 +00:00
yujinqiu 833b2bdc8e feat(security): enforce strong password policy (#12327) 2026-05-31 08:53:14 +08:00
Shrinath RaoandClaude Sonnet 4.6 3f3248fe64 fix(metrics): poll for invalid PromQL error indicator (enterprise CI fix) (#12368)
## Summary

- Fixes the `Invalid PromQL syntax error handling` test failing in
enterprise CI
- Enterprise CI renders the no-data/error state slower than the 3s fixed
timeout used previously
- Replaced `isVisible({ timeout: 3000})` with `expect.poll` (10s window,
500ms–2s intervals) so the assertion is real but tolerates CI rendering
lag

## Root cause

When `sum(rate(` is entered and applied, the UI renders a no-data
indicator asynchronously. Locally (enterprise) this appears in ~2s. In
enterprise CI (slower hardware, more features enabled) it takes longer
than 3s, causing all three indicator checks to return `false` before
they appear.

## Test plan

- [x] Passes locally: `hasNoData=true` confirmed
- [ ] Enterprise CI Metrics shard goes green

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 07:03:57 +00:00
Shrinath RaoandClaude Sonnet 4.6 0929fa5678 test: Sentinel audit fixes + parallelize Metrics E2E shard (~22 min → ~5-7 min) (#12367)
## Summary

Two independent improvements to the Playwright E2E test suite.

---

### Part 1 — Sentinel audit fixes (original PR scope)

Fixes from the Sentinel audit review of PR #12358 (selector drift
stabilization).

- **Dead code removal**: delete `clickJobID()` from `logsPageEP.js`
(unreachable code containing `console.log`); remove unused `chromium`
import
- **Critical fix**: replace `console.log`/`console.error` in
`decryptLogSQL()` with `testLogger` equivalents
- **POM compliance**: replace raw `page.locator()` call in
`alerts-ui-operations.spec.js` `beforeEach` with
`pm.alertsPage.isIncidentsFeatureEnabled()`; add that method to
`alertsPage.js`
- **POM compliance**: move per-test `const pm = new PageManager(page)`
into `beforeEach` in `service-graph.spec.js`; replace raw
`page.waitForLoadState()` with
`pm.serviceGraphPage.waitForGraphReload()`; add `waitForGraphReload()`
to `serviceGraphPage.js`
- **Inline selector cleanup**: move metrics tab inline string literals
in `serviceGraphPage.js` into constructor properties
- **Tagging**: add
`@cipherKeys/@enterprise/@P1/@P2/@negative/@validation` to all 6 tests
in `cipherKeys.spec.js` (were untagged); add `@P1/@P2` to all 5 tests in
`regexPatternManagement.spec.js`
- **ESM fix**: correct `export const path = require('path')` → `import
path from 'path'` in `logoManagementPage.js`
- **Deduplication**: consolidate
`createFirstFunction`/`createSecondFunction` into shared
`createFunction(name, vrlCode)` in `pipelinesEP.js`

---

### Part 2 — Parallelize Metrics E2E shard

**Problem**: The `e2e / Metrics` CI shard was consistently the slowest
job at ~22 minutes per PR run, while all other shards complete in ~5-7
minutes.

**Root cause**: 9 of 11 Metrics spec files used
`test.describe.configure({ mode: 'serial' })` because they shared a
describe-scope `let pm` variable reassigned in `beforeEach`. This forced
every test in those files onto a single Playwright worker, eliminating
all parallelism despite `fullyParallel: true` and 5 workers being
configured.

**Fix applied to all 9 affected files** (`metrics.spec.js`,
`metrics-config.spec.js`, `metrics-config-tabs.spec.js`,
`metrics-visualizations.spec.js`,
`metrics-promql-query-persistence.spec.js`,
`metrics-table-column-order.spec.js`, `metrics-queries.spec.js`,
`metrics-aggregations.spec.js`, `metrics-advanced.spec.js`):
- Remove `test.describe.configure({ mode: 'serial' })`
- Replace shared `let pm` + `beforeEach` pattern with a `setupTest(page,
testInfo)` helper that creates and returns a fresh `PageManager` per
test — the same pattern already used in `metrics-promql-builder.spec.js`
(the one file that was already parallel)
- `metrics-table-column-order.spec.js`: pass `pm` explicitly into
`setupTableChart(pm, page, mode)` since it can no longer close over a
describe-scope variable
- Split consolidated mega-tests (for-loops over multiple independent
scenarios) in `metrics-queries`, `metrics-aggregations`, and
`metrics-advanced` into individual `test()` blocks so Playwright can
distribute them across workers

**No test assertions, selectors, query logic, or validation changed.**

| File | Tests before | Tests after |
|---|---|---|
| `metrics.spec.js` | 10 (serial) | 10 (parallel) |
| `metrics-config.spec.js` | 2 (serial) | 2 (parallel) |
| `metrics-config-tabs.spec.js` | 3 (serial) | 3 (parallel) |
| `metrics-visualizations.spec.js` | 4 (serial) | 4 (parallel) |
| `metrics-promql-query-persistence.spec.js` | 6 (serial) | 6 (parallel)
|
| `metrics-table-column-order.spec.js` | 20 (serial) | 20 (parallel) |
| `metrics-queries.spec.js` | 4 mega-tests (serial) | 12 individual
(parallel) |
| `metrics-aggregations.spec.js` | 3 mega-tests (serial) | 10 individual
(parallel) |
| `metrics-advanced.spec.js` | 4 mega-tests (serial) | 12 individual
(parallel) |

**Expected impact**: ~1300s (~22 min) → ~300-400s (~5-7 min) wall-clock
time.

---

## Test plan
- [ ] All 40 tests in the 4 Sentinel-audit spec files enumerate cleanly
via `npx playwright test --list`
- [ ] Tag filters `@cipherKeys`, `@P2`, `@sdr` match expected tests
- [ ] All Metrics spec files enumerate cleanly with no serial mode
warnings
- [ ] Metrics shard CI time drops to ~5-7 min on next PR run

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 02:31:44 +00:00
dependabot[bot]andHengfei Yang 48ff43f776 chore(deps): bump qs from 6.15.1 to 6.15.2 in /tests/ui-testing (#12168)
Bumps [qs](https://github.com/ljharb/qs) from 6.15.1 to 6.15.2.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/ljharb/qs/blob/main/CHANGELOG.md">qs's
changelog</a>.</em></p>
<blockquote>
<h2><strong>6.15.2</strong></h2>
<ul>
<li>[Fix] <code>stringify</code>: skip null/undefined entries in
<code>arrayFormat: 'comma'</code> + <code>encodeValuesOnly</code>
instead of crashing in <code>encoder</code></li>
<li>[Fix] <code>stringify</code>: use configured <code>delimiter</code>
after <code>charsetSentinel</code> (<a
href="https://redirect.github.com/ljharb/qs/issues/555">#555</a>)</li>
<li>[Fix] <code>stringify</code>: apply <code>formatter</code> to
encoded key under <code>strictNullHandling</code> (<a
href="https://redirect.github.com/ljharb/qs/issues/554">#554</a>)</li>
<li>[Fix] <code>stringify</code>: skip null/undefined filter-array
entries instead of crashing in <code>encoder</code> (<a
href="https://redirect.github.com/ljharb/qs/issues/551">#551</a>)</li>
<li>[Fix] <code>parse</code>: handle nested bracket groups and add
regression tests (<a
href="https://redirect.github.com/ljharb/qs/issues/530">#530</a>)</li>
<li>[readme] fix grammar (<a
href="https://redirect.github.com/ljharb/qs/issues/550">#550</a>)</li>
<li>[Dev Deps] update <code>@ljharb/eslint-config</code></li>
<li>[Tests] add regression tests for keys containing percent-encoded
bracket text</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/ljharb/qs/commit/9aca4076fe788338c67cf7e115f0be6bc58d85a8"><code>9aca407</code></a>
v6.15.2</li>
<li><a
href="https://github.com/ljharb/qs/commit/5e33d33447ed0bf1ddab9abc41d27dea4687d992"><code>5e33d33</code></a>
[Dev Deps] update <code>@ljharb/eslint-config</code></li>
<li><a
href="https://github.com/ljharb/qs/commit/21f80b33e5c8b3f7eba1034fff0da4a4a37a1d41"><code>21f80b3</code></a>
[Fix] <code>stringify</code>: skip null/undefined entries in
<code>arrayFormat: 'comma'</code> + `e...</li>
<li><a
href="https://github.com/ljharb/qs/commit/a0a81ea2071acce3eff41a040f719ac8f5c4f64c"><code>a0a81ea</code></a>
[Fix] <code>stringify</code>: use configured <code>delimiter</code>
after <code>charsetSentinel</code></li>
<li><a
href="https://github.com/ljharb/qs/commit/e3062f78f5233b338ceeb8e8dfa5a07dea4b32a8"><code>e3062f7</code></a>
[Fix] <code>stringify</code>: apply <code>formatter</code> to encoded
key under <code>strictNullHandling</code></li>
<li><a
href="https://github.com/ljharb/qs/commit/0c180a40adb8c6703fffc85b2ff06ca209f5c1e0"><code>0c180a4</code></a>
[Fix] <code>stringify</code>: skip null/undefined filter-array entries
instead of crashi...</li>
<li><a
href="https://github.com/ljharb/qs/commit/3a8b94aec19bd664720f6f6b1e66c4a0dfe4b656"><code>3a8b94a</code></a>
[Tests] add regression tests for keys containing percent-encoded bracket
text</li>
<li><a
href="https://github.com/ljharb/qs/commit/96755abd357c0e534dd3442a84a04d08864bfe0d"><code>96755ab</code></a>
[readme] fix grammar</li>
<li><a
href="https://github.com/ljharb/qs/commit/a419ce5bbfcdb98a299f1a0bb47ea055baef20e6"><code>a419ce5</code></a>
[Fix] <code>parse</code>: handle nested bracket groups and add
regression tests</li>
<li>See full diff in <a
href="https://github.com/ljharb/qs/compare/v6.15.1...v6.15.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=qs&package-manager=npm_and_yarn&previous-version=6.15.1&new-version=6.15.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/openobserve/openobserve/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2026-05-29 17:11:21 +00:00
a749599898 fix: update e2e selectors for O2 component migration (#12346)
Replace Quasar-based selectors with O2 component equivalents across
alerts, dashboard, logs, and streams page objects. Key changes:
- OSwitch: target -btn suffix on inner button
- OSelect: use -trigger/-popover/-option suffixes, fallback to
[role="option"][data-test-value] when parent has no data-test
- OCombobox: use placeholder-based locators where no data-test exists
- OInput: descendant input selectors, pagination text format handling
- OTable: custom bottom slot pagination format ("N Stream(s)")

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
2026-05-29 17:07:42 +00:00
ktx-vaidehiandktx-riya b7ffc10a5d fix: ux revamp logs stream single or multi select (#12350)
Co-authored-by: ktx-riya <riya.kukadiya@kiara.tech>
2026-05-29 17:07:08 +00:00
Shrinath RaoandClaude Sonnet 4.6 cc1f09c5c1 test: fix selector drift across ENT specs, re-enable enterprise tests, and improve CI (#12358)
## Summary

Broad test stabilization pass covering Quasar component selector drift
in ENT UI specs, re-enabling previously disabled enterprise tests, and
CI infrastructure fixes.

### UI test fixes (selector drift)

Quasar's O2 component library wraps native elements — `OInput`,
`OTextarea`, `ODropdown`, `OToast`, `OToggleGroup`, `OToggleGroupItem`,
`ODrawer`, `ODialog`, `OFile` — in ways that break naive attribute/role
selectors. This PR updates affected specs to use the correct locators.

| Spec | Fixes |
|------|-------|
| `service-graph.spec.js` | `OToggleGroupItem` selector, `ODrawer` close
btn, `beforeAll` timeout |
| `alerts-ui-operations.spec.js` | Remove `waitForResponse` overhead
from enterprise block |
| `logoManagement.spec.js` | `OInput` / `OFile` / `ODialog` / `OToast`
locators |
| `jobSearch.spec.js` | `OInput` fill targets wrapper div, `OTable`
index-based rows |
| `searchPatterns.spec.js` | `OToggleGroup` same-element selector,
pattern details dialog |
| `pipelineImport.spec.js` | `OInput` / `ODropdown` / Monaco / `OToast`
selector drift |
| `regexPatternManagement.spec.js` | Correct test-data import file path
|
| `cipherKeys.spec.js` | `OInput`/`OTextarea` `-field` suffix, `OToast`
selector, 403 skip guard, all-toasts polling |

### Enterprise tests re-enabled

- Removed `/* ... */` block comment disabling `"Alerts & Incidents Page
Navigation"` describe block (`@enterprise`) in
`alerts-ui-operations.spec.js`
- 3 tests restored: page navigation, element visibility, round-trip nav
back to Alerts
- Requires `O2_INCIDENTS_ENABLED=true` (set in ENT CI)

### CI improvements

- **playwright.yml**: Install `ffmpeg` via local Playwright binary
(required for `video: retain-on-failure` recording)
- **api-testing.yml**: Reduce runner from `ubicloud-standard-8` →
`ubicloud-standard-4`
- **test_stream_settings_dedup.py**: Fix field-filtering logic and
assertion count in `test_multiple_distinct_fields_no_duplicates`

## Paired ENT PR
Full ENT-only test restoration (SDR, RUM, cipherKeys, etc.):
https://github.com/openobserve/o2-enterprise/pull/1835

## Test plan
- [ ] ENT CI: `"Alerts & Incidents Page Navigation"` enterprise block
passes
- [ ] Playwright CI: video artifacts generated on failure (ffmpeg
install works)
- [ ] API tests: `test_multiple_distinct_fields_no_duplicates` passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 16:55:05 +00:00
ktx-riyaandktx-vaidehi 2af558441a fix: theme switching in revamped (#12340)
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
2026-05-29 10:17:10 +00:00
1afabd35ce feat: bloom filter pruning layer above tantivy (#12121)
## Summary

Design at:
https://github.com/openobserve/designs/blob/main/infrastructure/bloomfilter/DESIGN.md

A bloom-filter pruning layer that sits **between `file_list` and
tantivy** in the search path. It drops candidate files before any `.ttv`
is opened, for **positive equality / IN** lookups on high-cardinality
fields (`trace_id`, `request_id`, …).

It is built **on top of** tantivy: the `.bf` is constructed by iterating
the tantivy term dictionary, so a field must be a tantivy secondary
index (`index_fields`) **and** opted into `bloom_filter_fields`. Only
the intersection is built — a bloom-only field produces nothing. To make
that intersection non-empty by construction, enabling a field for bloom
now folds it into `index_fields` automatically (see Config). Values are
not inspected; the operator chooses which fields are worth it (see
guidance below).

Design notes live in the module-level docs under `src/infra/src/bloom/`
(`mod.rs`, `writer.rs`, `reader.rs`, `sbbf.rs`).

## Design

- **Transposed (block-major) `.bf` layout.** All files in a `.bf` share
`num_blocks = B`, so `block_index = fastmap(hash(value), B)` is
identical across files and the blocks a query needs form **one
contiguous row**. The search side reads **one `M × 32`-byte row per
`.bf`** to answer membership for all `M` files — prune cost is
`O(groups)`, not `O(files)`. (`infra::bloom`, own ~120-LOC SBBF over
gxhash64.)
- **Sub-grouping.** A hour's new files are sorted by record count and
split into chunks of ≤ `ZO_BLOOM_FILTER_MAX_FILES_PER_BF` (default 256),
one `.bf` each. Bounds compactor build memory (`M × B × 32`) and `.bf`
object size; query reads become `ceil(files / 256)` rows.
- **Footer-only reader + batched fetch.** Footer (≈ `M × 12` B) comes
from `BLOOM_FOOTER_CACHE` or one 16 KB suffix GET (with a precise
re-read when a large footer overflows the probe); rows are pulled with
one `cache_storage::get_ranges` call. `cache_storage` was reworked to do
real batched cache reads (memory: 1 lookup + N slices; disk: 1
`File::open` + N `pread`).
- **B sizing.** `num_blocks_for(max_records_in_chunk,
ZO_BLOOM_FILTER_FPP)` (default fpp `0.01`) — record count is a safe NDV
upper bound, so the SBBF never saturates for high-cardinality fields.
- **Versioned, with orphan cleanup.** A file's `bloom_ver` is write-once
— a row is never re-stamped, so dumped rows keep pointing at the `.bf`
they were built into. But `bloom_ver` is **not** append-only at the
bucket level: a merge produces new output files with a fresh `bloom_ver`
and deletes the small inputs, so the old inputs' `.bf` can become
orphaned. After each merge round the compactor retires any `.bf` whose
`bloom_ver` is no longer referenced by a live `file_list` row in the
bucket (enqueued through the existing `file_list_deleted` path).
`bloom_ver` is preserved through every file_list INSERT/SELECT path
(incl. the SQLite read-cache) and the dump parquet.
- **Observability.** Prometheus metrics for prune
(`bloom_prune_keep_ratio`, `bloom_prune_duration_seconds`);
`bloom-inspect` CLI dumps a `.bf` footer (fields, B, row bytes, file
names); footer-cache hit/miss via the shared `BytesCache` tagged
metrics; `Local::get_ranges` now reports `STORAGE_READ_*` (previously
bypassed).

## When it helps (measured, S3, no disk cache)

Same stream `benchtest`, 170 files / 14.64 GB index, single matching
row. The only variable is whether the field's values are
time-correlated:

| Field | Type | tantivy only | bloom + tantivy | Outcome |
|---|---|---|---|---|
| `trace_id` (16-byte random) | random | **2584 ms** (opens all 170
`.ttv`) | 24 ms prune + 65 ms = **89 ms** | **~29× faster** |
| `request_id` (`…-7e74-…`, UUIDv7) | time-ordered | **154 ms** (sparse
index range-prunes) | 42 ms + 154 ms = **196 ms** | ~1.3× **slower** |

The whole 170-file group was answered by **one row read of 5440 bytes**
(`170 × 32`) + a cached footer.

**Guidance — enable only when all three hold (advisory; not enforced):**
1. **High cardinality** field.
2. **Random values** (not time-ordered) — so tantivy can't range-prune
and must scan every file. Time-ordered ids (UUIDv7, snowflake) are
already pruned by tantivy's sparse index in memory → bloom is pure
overhead there.
3. **Many files per hour** — hourly volume ≳ 10 ×
`ZO_COMPACT_MAX_FILE_SIZE` (≳10 `.ttv` files), so there's something to
prune. Bloom turns N file opens into one row read; at 1–2 files/hour
it's net overhead.

The 29× result above had 170 files/hour all answered by one `.bf`. ❌
Low-cardinality / non-indexed / range / `LIKE` / regex / `!=` / OR: no
effect.

## Config

- `ZO_BLOOM_FILTER_ENABLED` (default true) — global gate.
- Per stream: a field must be in **both** `index_fields` and
`bloom_filter_fields`. Adding a field to `bloom_filter_fields` auto-adds
it to `index_fields` (bloom is built on the secondary index), and the
default index fields include the default bloom fields.
- `ZO_BLOOM_FILTER_FPP` (default 0.01) — target false-positive
probability; sizes the SBBF block count (smaller fpp = fewer false
survivors, larger `.bf`). Out-of-range falls back to 0.01.
- `ZO_BLOOM_FILTER_MAX_FILES_PER_BF` (default 256) — files per `.bf`;
trades query reads vs build memory.
- `ZO_BLOOM_FILTER_DEFAULT_FIELDS` (default empty) — fields bloomed for
every stream.
- `ZO_BLOOM_FOOTER_CACHE_MAX_SIZE` (default 1% mem, clamped 32–256 MB).

Removed in this PR: `ZO_BLOOM_FILTER_DISABLED_ON_SEARCH`,
`ZO_BLOOM_FILTER_NDV_RATIO` (B is now sized from record count + fpp, not
an ndv ratio).

## Test plan

- [x] `cargo build`, `cargo clippy -p infra -p openobserve --lib -- -D
warnings`, `cargo fmt --check` clean
- [x] Unit tests: `cargo test -p infra --lib bloom`, `cargo test -p
openobserve --lib bloom`
- [x] E2E on S3 (results above): random field prunes 170→1 and is ~29×
faster; transposed read = 1 request / 5440 B per group
- [x] Reviewer: confirm gating expectations (operator opts fields in; no
auto value detection)
- [x] Test: orphan bloom: 2 * 256MB -> merge to 512MB -> merge to
1024MB, the first .bf should be deleted
- [x] Test: orphan bloom: 6 * 256MB -> merge to 1GB + 512MB -> 512 MB
merge to 1024MB, the first .bf include 2 files(1GB + 512MB), after merge
again, the first still have 2 files but one of them already deleted. we
should keep the first .bf, because still have files in use.

## Notes

- `.bf` format is `VERSION 0x01` (transposed). Not compatible with any
pre-merge experimental `.bf`; rebuild by wiping the `bloom/` dir and
resetting `bloom_ver=0`.
- Hash is gxhash64 (project-wide `config::utils::hash`), not the Parquet
spec's xxhash64 — `.bf` is internal, never read by external tools.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-05-29 07:33:11 +00:00
ktx-mihir cf9c0ea179 fix: dark mode text area and pagination (#12339)
- SDR Save Settings trigger point discrepency 
- Edit the existing alert - Ai icon is overlapping
- On stream field Scroller is not working
- With other languages, the Side Navigation UI is broken
- Search box dont having any border (List search view )
- text area color not looks proper
- Pipeline history - pagination is not working properly
- iam page default pagination value change
2026-05-28 16:07:21 +00:00
ktx-vaidehiandClaude Sonnet 4.6 1d60431204 fix: ux revamp toast design (#12336)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 15:49:12 +00:00
98d1d1c378 test: fix Chromium CI stability and disable ENT-only tests (runner resize, timeouts, cache, no-sandbox) (#12330)
## Summary

- **`--no-sandbox` / `--disable-setuid-sandbox`** added to
`chromium.launch()` in `globalSetup.js` and `launchOptions` in
`playwright.config.js` (CI-only) — prevents silent Chromium hangs on
restricted runner environments
- **`globalTimeout: 110 min`** added to `playwright.config.js` — caps
the entire suite at 110 min (10 min under the job timeout) so hung runs
fail with a clear Playwright error instead of a silent CI kill
- **`restore-keys` fallback** added to the browser cache step — a
`package-lock.json` change no longer forces a full 163 MB re-download on
every shard simultaneously
- **Runner right-sizing**: `ui_integration_tests` →
`repo-openobserve-standard-16`; `merge_and_upload_reports` +
`playwright_summary` → `repo-openobserve-standard-4`

## Test plan

- [ ] Confirm `ui_integration_tests` shards start without Chromium
launch hangs
- [ ] Confirm cache hit on second run (restore-keys fallback working)
- [ ] Confirm `merge_and_upload_reports` and `playwright_summary`
succeed on 4 GB runners

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-05-28 19:23:41 +05:30
+5 ae22a74e62 feat: ux improvement main (#11838)
#11720

Design at:

https://github.com/openobserve/designs/blob/main/dashboards/ux-improvement-dialog/design.md

---------

---------

Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Abhay Padamani <141122205+ktx-abhay@users.noreply.github.com>
Co-authored-by: Sai Nikhil <nikhil@openobserve.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: ktx-mihir <mihir.patel@kiara.tech>
Co-authored-by: ktx-riya <riya.kukadiya@kiara.tech>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Omkar Kesarkhane <127078473+omkarK06@users.noreply.github.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech>
Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
2026-05-28 11:00:49 +00:00
Shrinath RaoandClaude Sonnet 4.6 e79cb30b30 test: move enterprise-only regression tests to *-ent spec files (#12298)
## Summary

- Moves 3 enterprise-only tests out of OSS regression spec files into
dedicated `*-ent` spec files
- These spec files live in the OSS repo but are **not referenced in the
OSS CI workflow** — they are picked up only by the enterprise CI
- Fixes recurring failures in the OSS regression run caused by features
that don't exist in OSS builds

## Why the tests were failing in OSS

| Shard | Test | Root cause |
|---|---|---|
| Traces-Regression | Service graph side panel (Bug #11687) |
`[data-test="traces-service-graph-toggle"]` is wrapped in
`v-if="config.isEnterprise == 'true'"` in `SearchBar.vue` — element
never renders in OSS → 45s timeout |
| Logs-Regression | Scheduled Search navigation (Bug #7354) | `POST
/api/{org}/search_jobs` returns **404** in OSS — endpoint is
enterprise-only |

## Changes

**New files:**
-
`tests/ui-testing/playwright-tests/RegressionSet/logs-regression-ent.spec.js`
— Scheduled Search / `search_jobs` describe block (2 tests)
-
`tests/ui-testing/playwright-tests/RegressionSet/traces-regression-ent.spec.js`
— service graph side panel test (Bug #11687)

**Modified files:**
- `logs-regression-9044-7354.spec.js` — removed the `#7354` Scheduled
Search describe block; `#9044` scroll retention tests remain
- `traces-regression.spec.js` — removed Bug #11687 service graph test
block

The companion PR in `o2-enterprise` adds these two files to the
Logs-Regression and Traces-Regression shards of the enterprise workflow.

## Test plan
- [ ] OSS regression CI (`playwright_regression.yml`) should pass — no
more enterprise-only test failures
- [ ] Enterprise regression CI picks up `*-ent.spec.js` files via the
companion workflow PR

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 11:22:44 +00:00
Neha PandClaude Opus 4.7 b369746c30 fix: use raw table_name for URL job lookup in cross-type check (#12277)
## Summary

Follow-up to #12249. The cross-type collision check in the file upload
handler passes the formatted `stream_name` to `get_url_jobs_for_table`,
but URL jobs are stored with the raw `table_name` from the HTTP path.
When names contain uppercase or special characters,
`format_stream_name("My_Table")` produces `"my_table"`, which won't
match the stored URL job name `"My_Table"` — causing the cross-type
protection to be silently bypassed.

## Fix

Changed `get_url_jobs_for_table(&org_id, &stream_name)` to
`get_url_jobs_for_table(&org_id, table_name.trim())` — matching how the
URL handler itself queries URL jobs at line 299.

## Before/After

```rust
// Before: formatted name passed to URL jobs lookup (WRONG)
get_url_jobs_for_table(&org_id, &stream_name)

// After: raw table_name passed (CORRECT, matches how URL handler queries)
get_url_jobs_for_table(&org_id, table_name.trim())
```

## Notes

- `ENRICHMENT_TABLES` cache key continues to use `stream_name`
(formatted) — that's correct
- `get_url_jobs_for_table` now uses `table_name.trim()` — matching the
URL handler at line 299

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 05:59:56 +00:00
e9266c4510 fix: restore retry guard in enrichment table URL validation (#12218)
Fixes https://github.com/openobserve/o2-enterprise/issues/1796 (item 1:
"Reload Existing URLs" returns error).

## Summary

Commit
[`85d261b5d0`](https://github.com/openobserve/openobserve/commit/85d261b5d0e4b68dbc79ca18b844fc53e2368ee9)
(vulnerability scan fixes, Apr 14 2026) removed the `!retry` guard from
enrichment table URL validation. This broke the **reload** update mode
because the frontend sends an empty URL when retrying/reprocessing
existing URLs — the backend already has them stored and just needs to
re-trigger processing.

## Root Cause

Before the breaking commit, URL validation was guarded by `!retry`:
```rust
if !retry && let Err(err_msg) = validate_enrichment_url(&request_body.url) {
    return MetaHttpResponse::bad_request(err_msg);
}
```

The security fix removed the guard under the assumption that skipping
validation on retry was an SSRF bypass vector. However, the retry flow
sends `url=""` intentionally — the existing URLs are in the database.

## Fix

Re-adds the `!retry` guard around URL validation. In retry mode, URL
validation is skipped entirely (the URLs come from DB). All other modes
(create, append, replace, replace_failed) still require a non-empty,
valid URL with full SSRF protection.

## Test Plan

- [ ] Create enrichment table from URL
- [ ] Edit table → Reload existing URLs → Should succeed (no "URL cannot
be empty")
- [ ] Edit table → Add new URL → Should still validate URL format
- [ ] Edit table → Replace all URLs → Should still validate URL format
- [ ] Edit table → Replace failed URL → Should still validate URL format

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Subhradeep Chakraborty <chakrabortysubhradeep556@gmail.com>
2026-05-25 17:00:13 +00:00
Huaijin 2cee3bf12d refactor: unify search partition handle (4) (#12193) 2026-05-25 14:49:26 +00:00
Neha PandClaude Opus 4.7 2125f6e6da test: add 15 E2E regression tests for logs and traces bugs (#12084)
## Summary

Adds Playwright E2E regression tests for 15 verified bug fixes. Tests
are added to the existing CI matrix files:

- 6 logs tests in `logs-regression-bugs-2.spec.js`
- 9 traces tests in `traces-regression-bugs.spec.js`

Also fixes framework compliance: all raw `page.locator()` calls moved to
page object getter methods, missing assertions added, and flaky test
logic fixed in #10595.

## Logs bugs covered

| Bug | Description |
|-----|-------------|
| #11606 | include/exclude search term gets added at incorrect position
|
| #9339 | collapse or expand the indexlist wont redraw the histogram
chart |
| #7310 | search term appended instead of replaced in streams dropdown |
| #5277 | column positions change back after re-running query |
| #4426 | text wrap toggle causes column misalignment on saved view
switch |
| #4091 | cancel query button on visualize page |

## Traces bugs covered

| Bug | Description |
|-----|-------------|
| #11816 | trace API called multiple times with auto search enabled |
| #11815 | traces not sorted correctly when sorted by spans column |
| #11531 | spans tab method column references non-existent field |
| #11446 | incorrect duration value in logs detail traces tab |
| #11384 | switching organization does not clear trace explorer results
|
| #11244 | timestamp values inconsistent between traces and logs |
| #8978 | no cancel option while running a query in traces |
| #4151 | trace values not visible on hover in dark mode |
| #2887 | traces should be orderable by duration column |

## Files changed

| File | Change |
|------|--------|
| `tests/ui-testing/pages/logsPages/logsPage.js` | 9 new getter methods
for selectors |
| `tests/ui-testing/pages/tracesPages/tracesPage.js` | 14 new getter
methods for selectors |
|
`tests/ui-testing/playwright-tests/RegressionSet/logs-regression-bugs-2.spec.js`
| 6 new tests, 0 raw locators, fixed #10595 |
|
`tests/ui-testing/playwright-tests/RegressionSet/traces-regression-bugs.spec.js`
| 9 new tests, 0 raw locators |

## Test plan

All 21 tests (15 new + 6 existing) pass against the dev environment. No
new CI config needed — both spec files are already listed in
`playwright_regression.yml`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 13:02:02 +00:00
Neha PandClaude Opus 4.7 7157b8feb3 test: add E2E regression tests for scroll retention (#9044) and undefined length (#7354) (#12061)
## Summary
- Adds 2 new E2E regression test specs to `RegressionSet/` for recently
fixed bugs
- Adds 13 new page object methods to `logsPage.js` for scroll,
pagination, scheduled search, and sidebar navigation

https://github.com/openobserve/openobserve/issues/7354
https://github.com/openobserve/openobserve/issues/9044
## Tests Added

### scroll-retention-9044.spec.js (3 tests) 
- Scroll resets to top when navigating page 1 → page 2 via page number
button
- Scroll resets to top when navigating page 2 → page 1
- Scroll resets to top via Next Page (>) button

### undefined-length-7354.spec.js (2 tests)
- No "Undefined Length" error after navigating Logs → Scheduled Search →
Streams → Logs
- No console errors containing "undefined" or "length" after the
reproduction path

## CI
- Updated `playwright_regression.yml` to include both new spec files in
the `Logs-Regression` matrix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 09:09:20 +00:00
Subhra264 6ec195ac81 fix: revamp rbac (#11610)
Revamps the complex rbac logic to make it simpler.
Also, 
- Adds RBAC for anomaly apis
- Fixes https://github.com/openobserve/openobserve/issues/11710
2026-05-20 15:03:23 +00:00
Neha PandClaude Opus 4.7 ab86391649 test: add E2E tests for report folders functionality (#11985)
## Summary
- **Add `reportFoldersPage.js`** page object with 30+ methods for folder
CRUD, report move, search, and cross-folder toggle
- **Add `reportFolders.spec.js`** with 12 tests covering:
  - **P0 Smoke**: Sidebar renders with default folder, create new folder
- **P1 Functional**: Create report via API (creates temp dashboard from
scratch), move report between folders, cross-folder search, folder
sidebar search filter
- **P2 Edge Cases**: Move button disabled when source=destination, save
disabled for empty name, cancel closes dialog, default folder has no
edit/delete options, special characters allowed in folder names
- **Cleanup**: Deletes all test reports, folders, and leaves no
resources behind
- **Add `createDashboardViaApi`** to apiCleanup.js — creates a minimal
v8 dashboard from scratch, removing dependency on pre-existing
dashboards
- **Add `createReportViaApi`**, `fetchReportFolders`,
`deleteReportFolder`, `cleanupReportFolders` methods to apiCleanup.js
- Register ReportFoldersPage in PageManager

## Approach
- Everything created from scratch — unique report via API, unique
folders via UI, all cleaned up afterward
- No dependency on pre-existing dashboards or reports
- Zero `waitForTimeout` calls — all waits use `waitForLoadState`,
`waitFor({ state: 'visible/hidden' })`, or role-based waits
- Serial execution (tests build on each other: create folder → create
report → move report → cleanup)

## Test Results
```
12 passed (2.1m)
```

## CI Integration
- Added to `.github/workflows/playwright.yml` under `Reports` testfolder
alongside existing report tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-20 07:53:49 +00:00
Shrinath RaoandClaude Opus 4.7 916cc79c5c fix: restore search priming in QB lite init, fix SQL OFF filter parsing (#11823)
## Summary
- Restores `applyQueryButton` call in `initQueryBuilderTestLite` — the
initial match_all search serves as a UI priming step that ensures the
auto-search from stream selection has completed before the test runs its
own query
- Adds `runQueryAndWaitForResults` before `clickBuildToggle` in SQL OFF
mode tests — matching the flow in `setupQueryAndSwitchToBuild` so the
builder has query results to parse into filters

## Root Cause
PR #11797 moved ingestion from `beforeEach` to `beforeAll` and
introduced `initQueryBuilderTestLite` which skipped the initial
match_all search. Two issues emerged:

1. **No UI priming**: After `selectStream`, an auto-search fires
automatically. Without waiting for it to complete, the test's own query
could run while the UI was in an intermediate state, causing the builder
to parse zero filter conditions.

2. **SQL OFF tests skipped query execution**: These tests set search bar
content and clicked build toggle without running the query first. The
builder had no executed query results to parse into filters.

## Changes
| File | Change |
|------|--------|
| `queryBuilder-helpers.js` | Add `applyQueryButton(pm)` to
`initQueryBuilderTestLite` (same as full init) |
| `logsQueryBuilder-filters-advanced.spec.js` | Add
`runQueryAndWaitForResults()` before `clickBuildToggle()` in 2 SQL OFF
tests |

## Test Results
- Local (5 workers, 77 tests): **74 passed**, 3 pre-existing
intermittent failures (CTE chart type, auto-mode chart type, NOT LIKE
filter parsing)
- The 8 CI flaky tests from #11797 are all in the passing set

## Test plan
- [ ] CI: Logs-Builder-Basic and Logs-Builder-Advanced shards pass with
0 flaky tests
- [ ] All 77 QB tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:38:31 +00:00