openobserve

mirror of https://github.com/openobserve/openobserve.git synced 2026-11-04 16:09:05 +00:00

Files

T

Shrinath Rao d1d643814e test: enhance cleanup patterns for orphaned test data (#10232 )

### **User description**
## Summary

- Add URL enrichment tables cleanup support via new
`/api/{org}/enrichment_tables/status` endpoint
- Add cleanup patterns for orphaned streams, functions, pipelines, and
enrichment tables
- Skip data ingestion when running cleanup.spec.js alone (optimization
for cleanup-only runs)

## Changes

### URL Enrichment Tables (apiCleanup.js)
- `fetchUrlEnrichmentTables()` - Fetches URL-based tables from status
API
- `cleanupUrlEnrichmentTables(patterns)` - Cleans up tables showing "NaN
MB" in UI

### New Cleanup Patterns (cleanup.spec.js)

**Streams:**
- `e2e_apostrophe_*` - Bug #9475 apostrophe test streams
- `e2e_traces_pipeline_test`
- `e2e_traces_<timestamp>_<suffix>`

**Functions:**
- `vrl_func_*` - Pipeline VRL function tests
- `vrl_traces_func_*` - Traces VRL function tests
- `vrl_metrics_func_*` - Metrics VRL function tests

**Pipelines:**
- `metrics-pipeline-*`
- `traces-pipeline-*`
- `condition-pipeline-*`
- `metrics-condition-pipeline-*`

**URL Enrichment Tables:**
- `schema_mismatch_*`, `schema_view_*`, `url_404_*`, `url_lifecycle_*`
- `invalid_url_*`, `cancel_test_*`, `toggle_test_*`, `edit_form_*`

### Global Setup Optimization (global-setup.js)
- Auto-detect cleanup-only runs via `process.argv`
- Skip logs, metrics, and traces ingestion for cleanup runs

## Test plan
- [ ] Run cleanup.spec.js standalone and verify no data ingestion occurs
- [ ] Verify orphaned test data is cleaned up correctly


___

### **PR Type**
Enhancement, Tests


___

### **Description**
- Add URL enrichment tables cleanup methods

- Extend cleanup.spec.js with new regex patterns

- Invoke cleanupUrlEnrichmentTables in tests

- Skip data ingestion on cleanup-only runs


___



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>apiCleanup.js</strong><dd><code>Add URL enrichment
tables cleanup</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

tests/ui-testing/pages/apiCleanup.js

<ul><li>Added fetchUrlEnrichmentTables method<br> <li> Added
cleanupUrlEnrichmentTables method<br> <li> Handled fetch and delete
errors</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10232/files#diff-537fe76c0d8e26e81c5f77c83bd0ee35daaa973822105c809bf83171a26b33c5">+79/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>global-setup.js</strong><dd><code>Skip ingestion in
cleanup-only runs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

tests/ui-testing/playwright-tests/utils/global-setup.js

<ul><li>Introduced isCleanupOnly flag<br> <li> Skipped data ingestion
for cleanup-only runs<br> <li> Retained conditional ingestion logic</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10232/files#diff-06323dad7214f86b41b62e121e716eeea0054bc4d950c6bcf4a0efe53894acc8">+20/-14</a>&nbsp;
</td>

</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>cleanup.spec.js</strong><dd><code>Extend cleanup test
patterns and URL tables</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

tests/ui-testing/playwright-tests/cleanup.spec.js

<ul><li>Updated pipeline cleanup regex patterns<br> <li> Extended
function cleanup patterns<br> <li> Modified enrichment tables
patterns<br> <li> Added cleanupUrlEnrichmentTables invocation</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10232/files#diff-c214275a512fa285a7e4f2ab61dfcb42d71a0436b0e4241f94d7b45693587983">+27/-4</a>&nbsp;
&nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___

2026-01-29 12:21:35 +05:30

data-ingestion.js

test: add bug regression tests to RegressionSet suite (#9899 )

2026-01-09 12:09:54 +05:30

enhanced-baseFixtures.js

test: refactor and optimize enrichment table tests (#9629 )

2025-12-13 07:59:08 +05:30

global-setup.js

test: enhance cleanup patterns for orphaned test data (#10232 )

2026-01-29 12:21:35 +05:30

global-teardown.js

test: playwright upload artifact error fixes (#8190 )