mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 16:10:43 +00:00
Shrinath RaoandClaude Opus 4.8
c178de4013
## Problem
The 7 `@combined` SDR specs (`queryTime{Redaction,Drop,Hash}`,
`ingestionTime{Redaction,Hash,Drop}`, `multipleSDRPatterns`) fail
deterministically on `e2e / SDR` (ENT [run
28216495512](https://github.com/openobserve/o2-enterprise/actions/runs/28216495512))
with `Field <name> should be visible but was not found`, even after the
content-gate retry in #12893.
## Root cause (a logic bug, not timing)
Verification scraped the virtualized logs results table. Two structural
defects:
1. **Row-indexed selector misused as a count.**
`[data-test="log-table-column-0-source"]` — `TenstackTable.vue` renders
`log-table-column-{rowIndex}-source`, so it only ever matches **row 0**
(1 element). The combined path's `count >= 4` gate can never be met by
the reliable source locator.
2. **Dead fallback selector.** `.logs-result-table tbody tr[role="row"]`
references a class that **does not exist** in `web/src` → always 0.
So the fetch collapsed to whole-page `tbody tr` scraping whose text
never matched the field keys → the flat *"4 entries found but fields not
yet indexed"* across every retry. #12893's retry couldn't help because
the scraped data was structurally wrong, not late. Single-field tests
passed only because they read row 0 directly.
## Fix
Verify against the **Search API** — the actual source of truth for SDR.
Query-time redaction is applied server-side in `apply_regex_to_response`
(`ApplyPolicy::AtSearch`); the web only displays API output, so
`[REDACTED]` / `[REDACTED:<hash>]` / dropped-field semantics are all
present in the API hits.
- Each ingest tags records with a unique `sdr_test_id` marker and
returns it.
- Verification queries `_search` with `WHERE sdr_test_id = '<marker>'`,
polls only until the **known** record count is searchable, then asserts
on the JSON hits.
- Markers threaded through all 7 specs: query-time reuses one marker
(visible → transformed); ingestion-time uses distinct per-batch markers;
`multipleSDRPatterns` threads one per sequential ingest.
Net **−80 lines** — removes all brittle DOM scraping from SDR
verification.
## Validation
`node --check` passes on all changed files; every verify call confirmed
to thread a marker; root-cause selectors + server-side redaction path
confirmed against source. Live `e2e / SDR` run validated via a
matching-name ENT branch (CI tree-merge picks up this OSS branch).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>