starred/openobserve
1
0
Fork 0
mirror of https://github.com/openobserve/openobserve.git synced 2026-11-06 00:09:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(sdr): add traces SDR e2e suite (redact/drop/hash × ingestion/query) (#13390)

Browse Source 
## What

Adds an end-to-end **traces SDR (Sensitive Data Redaction)** test suite,
extending SDR coverage from logs to traces. This validates the fix in
this PR branch (`feat/fix-traces-sdr-bug`) — SDR regex patterns were
previously applied to logs only; these tests assert the same policies
now apply to **trace span attributes** at both ingestion and query time.

Targets `feat/fix-traces-sdr-bug` so the tests land together with the
fix they cover.

## Tests added

**6 new spec files → 12 tests** (each spec = 1 `setup: cleanup patterns`
+ 1 combined functional test), covering **24 field-level policy
assertions** (4 sensitive fields × 6 specs). All tagged `@enterprise
@sdr @traces`.

| # | Spec | Policy | Timing | What it asserts |
|---|------|--------|--------|-----------------|
| 1 | `tracesIngestionTimeRedaction.spec.js` | Redact | Ingestion |
Span-attribute values matching the pattern are stored as `[REDACTED]` |
| 2 | `tracesQueryTimeRedaction.spec.js` | Redact | Query | Stored
values are untouched; the search response returns `[REDACTED]` |
| 3 | `tracesIngestionTimeDrop.spec.js` | Drop | Ingestion | Matching
fields are removed from the record at write time |
| 4 | `tracesQueryTimeDrop.spec.js` | Drop | Query | Fields present in
storage are dropped from the search response |
| 5 | `tracesIngestionTimeHash.spec.js` | Hash | Ingestion | Matching
values are stored as `[REDACTED:<hash>]` |
| 6 | `tracesQueryTimeHash.spec.js` | Hash | Query | Search response
returns `[REDACTED:<hash>]`; storage untouched |

Each combined test drives the full flow for **4 fields / patterns**:

- **email** (`user_email` = `john.doe@example.com`) —
`^[\w\.-]+@[\w\.-]+\.\w{2,}$`
- **US phone** (`phone` = `5551234567`) — `^\d{10}$`
- **credit card** (`cc_number` = `1234 5678 9012 3456`) —
`^\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}$`
- **SSN** (`ssn` = `123-45-6789`) — `^\d{3}-\d{2}-\d{4}$`

(drop/hash specs use `website_url`, `user_password`, `server_ip`,
`theme_color` with URL / password / IPv4 / hex-color patterns.)

Each functional test follows: ingest → assert field visible (no pattern)
→ create 4 patterns → associate to the traces stream's fields →
ingest/re-query → assert the field is redacted / dropped / hashed.

## How it works

Traces reuse the **shared** pattern create/associate/verify flow with
logs — only the ingest and read seams differ:

- **`sdrTracesIngestionPage.js`** (new): ingests via OTLP
`/api/{org}/v1/traces` (`stream-name` header). Each test field is a span
attribute that flattens to an identical top-level field; every span also
carries an `sdr_test_id` marker for batch isolation.
- **`sdrVerificationPage` / `streamAssociationPage`**: parameterized by
`streamType` — `?type=traces` search, Traces stream-type tab selection,
and schema-field filtering so a field row in the (virtualized,
~20-field) traces schema is reliably found.
- **`apiCleanup`**: `fetchStreams` / `deleteStream` / `cleanupStreams` /
`isStreamStillDeleting` / `waitForStreamDeletion` take an optional
`streamType` (default `'logs'` — no change for existing callers).

## Artifact cleanup

- `cleanup.spec.js` sweeps `sdr_traces_*` streams via `streamType:
'traces'` (the logs sweep is `type=logs`-only and never listed them).
- `cleanupRegexPatterns` adds the `traces_` prefix.
- **Verified live**: the sweep deleted 8 leftover traces streams + 32
traces patterns; 0 remained.

## Verification

- All **6 specs / 12 tests pass** against a local enterprise build
carrying the fix.
- The existing **logs** SDR suite still passes unchanged (no regression
from the shared page-object changes).

## Companion

CI wiring is a paired o2-enterprise PR (branch `test/traces-sdr-e2e`)
that splits the SDR matrix into `SDR-Logs` and `SDR-Traces` shards so
the new specs run as their own shard.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Chaitanya Sistla <sistlachaitanya2@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Shrinath Rao
2026-07-23 06:46:10 +00:00
committed by GitHub
co-authored by Chaitanya Sistla Claude Opus 4.8
parent 89b3e8e14a
commit 4f458a0fab
12 changed files with 1249 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/ui-testing/pages/apiCleanup.js
+24 -17
View File
@@ -1034,12 +1034,13 @@ class APICleanup {
}
/**
* Fetch all log streams
* Fetch all streams of the given type
* @param {string} [streamType='logs'] - Stream type to list (e.g. 'logs', 'traces', 'metrics')
* @returns {Promise<Array>} Array of stream objects
*/
async fetchStreams() {
async fetchStreams(streamType = 'logs') {
try {
const response = await this._fetch(`${this.baseUrl}/api/${this.org}/streams?type=logs`, {
const response = await this._fetch(`${this.baseUrl}/api/${this.org}/streams?type=${streamType}`, {
method: 'GET',
headers: {
'Authorization': this.authHeader,
@@ -1063,11 +1064,12 @@ class APICleanup {
/**
* Delete a single stream
* @param {string} streamName - The stream name
* @param {string} [streamType='logs'] - Stream type to delete (e.g. 'logs', 'traces', 'metrics')
* @returns {Promise<Object>} Deletion result
*/
async deleteStream(streamName) {
async deleteStream(streamName, streamType = 'logs') {
try {
const response = await this._fetch(`${this.baseUrl}/api/${this.org}/streams/${streamName}?type=logs&delete_all=true`, {
const response = await this._fetch(`${this.baseUrl}/api/${this.org}/streams/${streamName}?type=${streamType}&delete_all=true`, {
method: 'DELETE',
headers: {
'Authorization': this.authHeader,
@@ -1094,13 +1096,14 @@ class APICleanup {
* 1. GET stream settings - checks if schema exists
* 2. If schema gone, PUT settings to check if deletion marker is still active
* @param {string} streamName - The stream name to check
* @param {string} [streamType='logs'] - Stream type to check (e.g. 'logs', 'traces', 'metrics')
* @returns {Promise<boolean>} True if stream still exists or is being deleted
*/
async isStreamStillDeleting(streamName) {
async isStreamStillDeleting(streamName, streamType = 'logs') {
try {
// Phase 1: Check if stream schema exists via GET
const getResponse = await this._fetch(
`${this.baseUrl}/api/${this.org}/streams/${streamName}/settings?type=logs`,
`${this.baseUrl}/api/${this.org}/streams/${streamName}/settings?type=${streamType}`,
{
method: 'GET',
headers: {
@@ -1126,7 +1129,7 @@ class APICleanup {
// Phase 2: Schema is gone (404), but deletion marker might still be active
// Try PUT settings - this triggers is_deleting_stream check without creating data
const putResponse = await this._fetch(
`${this.baseUrl}/api/${this.org}/streams/${streamName}/settings?type=logs`,
`${this.baseUrl}/api/${this.org}/streams/${streamName}/settings?type=${streamType}`,
{
method: 'PUT',
headers: {
@@ -1160,15 +1163,16 @@ class APICleanup {
* @param {string} streamName - The stream name to wait for
* @param {number} maxWaitMs - Maximum time to wait in milliseconds (default: 120000 = 2 minutes)
* @param {number} pollIntervalMs - Polling interval in milliseconds (default: 3000 = 3 seconds)
* @param {string} [streamType='logs'] - Stream type to poll (e.g. 'logs', 'traces', 'metrics')
* @returns {Promise<boolean>} True if deletion completed, false if timed out
*/
async waitForStreamDeletion(streamName, maxWaitMs = 120000, pollIntervalMs = 3000) {
async waitForStreamDeletion(streamName, maxWaitMs = 120000, pollIntervalMs = 3000, streamType = 'logs') {
const startTime = Date.now();
let attempts = 0;
while (Date.now() - startTime < maxWaitMs) {
attempts++;
const stillDeleting = await this.isStreamStillDeleting(streamName);
const stillDeleting = await this.isStreamStillDeleting(streamName, streamType);
if (!stillDeleting) {
testLogger.debug('Stream deletion confirmed complete', {
@@ -1371,19 +1375,21 @@ class APICleanup {
* @param {Object} options - Optional configuration
* @param {boolean} options.waitForDeletion - Whether to wait for deletions to complete (default: true)
* @param {number} options.maxWaitPerStreamMs - Max wait time per stream in ms (default: 120000)
* @param {string} [options.streamType='logs'] - Stream type to sweep (e.g. 'logs', 'traces', 'metrics')
*/
async cleanupStreams(patterns = [], protectedStreams = [], options = {}) {
const { waitForDeletion = true, maxWaitPerStreamMs = 120000 } = options;
const { waitForDeletion = true, maxWaitPerStreamMs = 120000, streamType = 'logs' } = options;
testLogger.info('Starting streams cleanup', {
patterns: patterns.map(p => p.source),
protectedStreams,
waitForDeletion
waitForDeletion,
streamType
});
try {
// Fetch all log streams
const streams = await this.fetchStreams();
// Fetch all streams of the requested type (logs by default; e.g. traces for SDR trace streams)
const streams = await this.fetchStreams(streamType);
testLogger.info('Fetched streams', { total: streams.length });
// Filter streams matching patterns but excluding protected streams
@@ -1404,7 +1410,7 @@ class APICleanup {
const streamsToWaitFor = [];
for (const stream of matchingStreams) {
const result = await this.deleteStream(stream.name);
const result = await this.deleteStream(stream.name, streamType);
if (result.code === 200) {
deletedCount++;
@@ -1437,7 +1443,7 @@ class APICleanup {
for (let i = 0; i < streamsToWaitFor.length; i += batchSize) {
const batch = streamsToWaitFor.slice(i, i + batchSize);
const results = await Promise.all(
batch.map(streamName => this.waitForStreamDeletion(streamName, maxWaitPerStreamMs))
batch.map(streamName => this.waitForStreamDeletion(streamName, maxWaitPerStreamMs, 3000, streamType))
);
results.forEach((completed, index) => {
@@ -1855,7 +1861,8 @@ class APICleanup {
'email_format_', // SDR tests
'us_phone_', // SDR tests
'credit_card_', // SDR tests
'ssn_' // SDR tests
'ssn_', // SDR tests
'traces_' // Traces SDR tests (traces_<field>_<action>_<runId> patterns)
];
// Filter patterns that match test data names (using prefix matching to catch patterns with unique suffixes)
tests/ui-testing/pages/page-manager.js
+2
View File
@@ -94,6 +94,7 @@ import UnflattenedPage from "./logsPages/unflattened.js";
// ===== SDR (SENSITIVE DATA REDACTION) PAGE OBJECTS =====
import { SDRPatternsPage } from "./sdrPages/sdrPatternsPage.js";
import { SDRVerificationPage } from "./sdrPages/sdrVerificationPage.js";
import { SDRTracesIngestionPage } from "./sdrPages/sdrTracesIngestionPage.js";
import { StreamAssociationPage } from "./streamsPages/streamAssociationPage.js";
import { StreamsFormValidationPage } from "./streamsPages/streamsFormValidationPage.js";
@@ -205,6 +206,7 @@ class PageManager {
// ===== SDR (SENSITIVE DATA REDACTION) PAGE OBJECTS =====
this.sdrPatternsPage = new SDRPatternsPage(page);
this.sdrVerificationPage = new SDRVerificationPage(page);
this.sdrTracesPage = new SDRTracesIngestionPage(page);
this.streamAssociationPage = new StreamAssociationPage(page);
this.streamsFormValidation = new StreamsFormValidationPage(page);
tests/ui-testing/pages/sdrPages/sdrTracesIngestionPage.js
+143
View File
@@ -0,0 +1,143 @@
const crypto = require('crypto');
const testLogger = require('../../playwright-tests/utils/test-logger.js');
const { getAuthHeaders, getOrgIdentifier } = require('../../playwright-tests/utils/cloud-auth.js');
/**
* Traces-flavoured ingestion for SDR (Sensitive Data Redaction) tests.
*
* This is the traces analogue of `logsPage.ingestMultipleFields`. The SDR specs
* are stream-type agnostic above the ingest/verify seam — they create regex
* patterns, associate them with a stream's field, ingest a batch, then assert
* the field is redacted/hashed/dropped. Only two things differ per stream type:
* 1) how the batch is written (this file, for traces), and
* 2) how it is read back (`sdrVerificationPage`, via `?type=traces`).
*
* Each descriptor `{ fieldName, fieldValue }` is written as its OWN span, so the
* traces stream ends up with one record per field — exactly mirroring the logs
* helper's one-record-per-field shape, which is what `verifyMultipleFields`
* (minCount = number of fields) expects.
*
* The test field is carried as a span ATTRIBUTE. OpenObserve flattens span
* attributes to top-level string fields at ingest (`span_attribute_key` +
* `flatten::format_key`), so a plain snake_case attribute key with no dots and
* no reserved-field collision (e.g. `user_email`) lands as the identical
* top-level field name — the same field the pattern engine matches on and the
* same field the specs assert on. Every span also carries the per-batch
* `sdr_test_id` marker attribute so verification can query ONLY this batch.
*/
export class SDRTracesIngestionPage {
constructor(page) {
this.page = page;
}
static _hexId(bytes) {
return crypto.randomBytes(bytes).toString('hex');
}
/**
* Ingest one span per `{ fieldName, fieldValue }` into a TRACES stream via the
* OTLP JSON endpoint (`/api/{org}/v1/traces`, `stream-name` header). Retries on
* the transient "stream is being deleted" race, mirroring the logs helper.
*
* @param {string} streamName - target traces stream (set via `stream-name` header)
* @param {Array<{fieldName: string, fieldValue: string}>} dataObjects
* @param {number} maxRetries
* @returns {Promise<string>} the per-batch `sdr_test_id` marker
*/
async ingestMultipleFields(streamName, dataObjects, maxRetries = 5) {
const orgId = getOrgIdentifier();
// OTLP traces are ingested via /v1/traces with the target stream in the
// `stream-name` header (config: ZO_GRPC_STREAM_HEADER_KEY, default stream-name).
const headers = { ...getAuthHeaders(), 'stream-name': streamName };
const baseUrl = (process.env.INGESTION_URL || process.env.ZO_BASE_URL).replace(/\/$/, '');
// Unique per-batch marker. Verification queries the traces search API by this
// exact value (WHERE sdr_test_id = '<marker>'), so it inspects ONLY the records
// this call ingested — never stale rows from an earlier batch on the same stream.
const marker = `sdr-${crypto.randomUUID()}`;
// OTLP timestamps are nanoseconds. Space spans 1ms apart so they are distinct
// records with a stable order, matching the logs helper's per-entry timestamps.
const baseTimeNs = Date.now() * 1000000;
const spans = dataObjects.map(({ fieldName, fieldValue }, index) => {
const startNs = baseTimeNs + index * 1000000;
return {
traceId: SDRTracesIngestionPage._hexId(16),
spanId: SDRTracesIngestionPage._hexId(8),
name: `sdr-test-span-${index}`,
kind: 2, // SERVER
startTimeUnixNano: String(startNs),
endTimeUnixNano: String(startNs + 1000000),
attributes: [
// The field under test — becomes the top-level field `fieldName`.
{ key: fieldName, value: { stringValue: String(fieldValue) } },
// Per-batch isolation marker — becomes the top-level field `sdr_test_id`.
{ key: 'sdr_test_id', value: { stringValue: marker } },
],
status: { code: 1 }, // OK
};
});
const traceData = {
resourceSpans: [
{
resource: {
attributes: [
{ key: 'service.name', value: { stringValue: 'sdr-test-service' } },
],
},
scopeSpans: [
{
scope: { name: 'sdr-e2e', version: '1.0.0' },
spans,
},
],
},
],
};
testLogger.info(`Preparing to ingest ${spans.length} trace span(s) to stream ${streamName} (marker: ${marker})`);
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
const response = await this.page.request.post(`${baseUrl}/api/${orgId}/v1/traces`, {
headers,
data: traceData,
});
const status = response.status();
const responseBody = await response.json().catch(() => ({ error: 'Failed to parse JSON' }));
testLogger.info(`Traces ingestion response (attempt ${attempt}/${maxRetries}) - Status: ${status}, Body:`, responseBody);
// OTLP returns 200 on full success and 206 on partial success.
if (status === 200 || status === 206) {
testLogger.info('Traces ingestion successful, waiting for indexing...');
// Small head start to reduce poll iterations — verification polls the
// traces search API for this marker, so this is not the readiness gate.
await this.page.waitForTimeout(5000);
return marker;
}
const errorMessage = responseBody?.message || JSON.stringify(responseBody);
if (errorMessage.includes('being deleted') && attempt < maxRetries) {
const waitTime = attempt * 5000;
testLogger.info(`Stream is being deleted, waiting ${waitTime / 1000}s before retry...`);
await this.page.waitForTimeout(waitTime);
continue;
}
testLogger.error(`Traces ingestion failed! Status: ${status}, Response:`, responseBody);
throw new Error(`Traces ingestion failed with status ${status}: ${JSON.stringify(responseBody)}`);
} catch (e) {
if (attempt === maxRetries) {
testLogger.error(`Traces ingestion failed after ${maxRetries} attempts:`, e.message);
throw e;
}
testLogger.info(`Traces ingestion attempt ${attempt} failed, retrying...`);
await this.page.waitForTimeout(attempt * 5000);
}
}
}
}
tests/ui-testing/pages/sdrPages/sdrVerificationPage.js
+14 -8
View File
@@ -54,9 +54,12 @@ export class SDRVerificationPage {
* ingestion indexing for THIS batch — once `minCount` records appear, any SDR
* transform on them is already final, so the result is deterministic.
*
* @param {string} streamType - search stream type ('logs' | 'traces'). SDR is
* identical across stream types except which index the search reads: logs SDR
* reads `?type=logs`, traces SDR reads `?type=traces`. Same marker-scoped SQL.
* @returns {Promise<Object[]>} hit objects (may be fewer than minCount on timeout)
*/
async fetchRecordsByMarker(streamName, marker, minCount = 1) {
async fetchRecordsByMarker(streamName, marker, minCount = 1, streamType = 'logs') {
expect(marker, 'SDR verification requires an ingest marker — pass the value returned by ingestMultipleFields/ingestSingleLog').toBeTruthy();
const headers = getAuthHeaders();
@@ -77,7 +80,7 @@ export class SDRVerificationPage {
try {
const response = await this.page.request.post(
`${baseUrl}/api/${orgName}/_search?type=logs`,
`${baseUrl}/api/${orgName}/_search?type=${streamType}`,
{ headers, data: searchPayload, timeout: 15000 }
);
if (!response.ok()) {
@@ -118,11 +121,12 @@ export class SDRVerificationPage {
*
* @param {Object} logsPage - kept for call-site compatibility (unused; API-based)
* @param {string} marker - marker returned by the ingest that wrote this record
* @param {string} streamType - 'logs' | 'traces' (which search index to read)
*/
async verifySingleFieldInLatestLog(logsPage, streamName, fieldName, shouldBeDropped, shouldBeRedacted, marker) {
async verifySingleFieldInLatestLog(logsPage, streamName, fieldName, shouldBeDropped, shouldBeRedacted, marker, streamType = 'logs') {
testLogger.info(`Verifying field: ${fieldName}, shouldBeDropped: ${shouldBeDropped}, shouldBeRedacted: ${shouldBeRedacted}, marker: ${marker}`);
const hits = await this.fetchRecordsByMarker(streamName, marker, 1);
const hits = await this.fetchRecordsByMarker(streamName, marker, 1, streamType);
expect(hits.length, `No record found for marker ${marker} in stream ${streamName}`).toBeGreaterThan(0);
const hit = hits[0];
@@ -149,11 +153,12 @@ export class SDRVerificationPage {
*
* @param {Object} logsPage - kept for call-site compatibility (unused; API-based)
* @param {string} marker - marker returned by the ingest under test
* @param {string} streamType - 'logs' | 'traces' (which search index to read)
*/
async verifyMultipleFields(logsPage, streamName, fieldsToVerify, marker) {
async verifyMultipleFields(logsPage, streamName, fieldsToVerify, marker, streamType = 'logs') {
testLogger.info(`Verifying ${fieldsToVerify.length} fields in stream: ${streamName} (marker: ${marker})`);
const hits = await this.fetchRecordsByMarker(streamName, marker, fieldsToVerify.length);
const hits = await this.fetchRecordsByMarker(streamName, marker, fieldsToVerify.length, streamType);
expect(hits.length, `Expected ${fieldsToVerify.length} records for marker ${marker} but found ${hits.length}`).toBeGreaterThanOrEqual(fieldsToVerify.length);
for (const { fieldName, shouldBeDropped, shouldBeRedacted, shouldBeHashed } of fieldsToVerify) {
@@ -190,11 +195,12 @@ export class SDRVerificationPage {
*
* @param {Object} logsPage - kept for call-site compatibility (unused; API-based)
* @param {string} marker - marker of the post-drop ingest batch
* @param {string} streamType - 'logs' | 'traces' (which search index to read)
*/
async verifyFieldsAreAbsent(logsPage, streamName, fieldNames, marker) {
async verifyFieldsAreAbsent(logsPage, streamName, fieldNames, marker, streamType = 'logs') {
testLogger.info(`Verifying ${fieldNames.length} fields are ABSENT in stream ${streamName} (marker: ${marker})`);
const hits = await this.fetchRecordsByMarker(streamName, marker, fieldNames.length);
const hits = await this.fetchRecordsByMarker(streamName, marker, fieldNames.length, streamType);
expect(hits.length, `Expected ${fieldNames.length} records for marker ${marker} but found ${hits.length}`).toBeGreaterThanOrEqual(fieldNames.length);
for (const fieldName of fieldNames) {
tests/ui-testing/pages/streamsPages/streamAssociationPage.js
+77 -14
View File
@@ -39,14 +39,38 @@ export class StreamAssociationPage {
this.dropDescription = page.getByText('Drop This will drop the field');
}
async navigateToStreams() {
async navigateToStreams(streamType = 'logs') {
const orgName = process.env.ORGNAME || 'default';
const baseUrl = process.env.ZO_BASE_URL;
const targetUrl = `${baseUrl}/web/streams?org_identifier=${orgName}`;
testLogger.info(`Navigating to Streams page with org: ${orgName}`);
testLogger.info(`Navigating to Streams page with org: ${orgName} (stream type: ${streamType})`);
await this.page.goto(targetUrl);
await this.page.waitForLoadState('domcontentloaded');
await this.page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
await this.selectStreamTypeTab(streamType);
}
/**
* Select the stream-type tab in the Streams list toolbar. The list defaults to
* the "logs" tab, so a traces/metrics stream is NOT visible until its tab is
* active — the search box only filters within the active tab's list. The tab is
* an OToggleGroupItem rendered by reka-ui with `data-otoggle-value="<type>"` and
* `data-state=on|off`; we click until it reports `on`.
*/
async selectStreamTypeTab(streamType = 'logs') {
if (!streamType || streamType === 'logs') return; // logs is the default active tab
testLogger.info(`Selecting "${streamType}" stream-type tab`);
const tab = this.page.locator(`[data-otoggle-value="${streamType}"]`).first();
await tab.waitFor({ state: 'visible', timeout: 10000 });
for (let attempt = 1; attempt <= 3; attempt++) {
const state = await tab.getAttribute('data-state').catch(() => null);
if (state === 'on') break;
await tab.click({ force: true });
await this.page.waitForTimeout(500);
}
// Let the list reload for the newly-selected type before callers search/open rows.
await this.page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
await this.page.waitForTimeout(500);
}
async searchForStream(streamName) {
@@ -56,8 +80,8 @@ export class StreamAssociationPage {
await this.page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
}
async openStreamDetail(streamName) {
testLogger.info(`Opening stream detail for: ${streamName}`);
async openStreamDetail(streamName, streamType = 'logs') {
testLogger.info(`Opening stream detail for: ${streamName} (stream type: ${streamType})`);
await this.page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
// Find the cell with stream name. A freshly-ingested stream can lag the
@@ -71,7 +95,7 @@ export class StreamAssociationPage {
let cellVisible = await streamCell.waitFor({ state: 'visible', timeout: 8000 }).then(() => true).catch(() => false);
for (let attempt = 1; attempt <= 6 && !cellVisible; attempt++) {
testLogger.info(`Stream "${streamName}" row not yet in /streams (attempt ${attempt}) — reloading + re-searching`);
await this.navigateToStreams();
await this.navigateToStreams(streamType);
await this.searchForStream(streamName);
cellVisible = await streamCell.waitFor({ state: 'visible', timeout: 12000 }).then(() => true).catch(() => false);
}
@@ -105,7 +129,7 @@ export class StreamAssociationPage {
})().catch(() => false);
if (!opened && attempt < 4) {
testLogger.info(`Stream Detail did not open for "${streamName}" (attempt ${attempt}) — reloading + re-searching streams list`);
await this.navigateToStreams();
await this.navigateToStreams(streamType);
await this.searchForStream(streamName);
}
}
@@ -119,6 +143,30 @@ export class StreamAssociationPage {
testLogger.info('Stream detail sidebar opened successfully');
}
/**
* Filter the stream-detail schema field table to a single field via the field
* search input. The schema table virtualizes its rows, so a field that sorts
* outside the rendered window (common on traces streams, which carry ~20 auto
* fields plus the span-attribute fields) is not in the DOM until filtered. This
* makes the target row reliably present for both logs and traces streams.
* No-op-safe if the search input is absent (older schema drawer variants).
*/
async filterSchemaField(fieldName) {
// OSearchInput forwards the consumer data-test to a wrapper; the real <input>
// is the inner element — resolve it directly so fill() targets the field.
const searchInput = this.page.locator('[data-test="schema-field-search-input"]').locator('input').first();
const present = await searchInput.waitFor({ state: 'visible', timeout: 5000 }).then(() => true).catch(() => false);
if (!present) {
testLogger.info('Schema field search input not present — skipping field filter');
return;
}
await searchInput.fill('');
await searchInput.fill(fieldName);
// Let the client-side filter settle so only the matching field row remains.
await this.page.waitForTimeout(1000);
testLogger.info(`Filtered schema field table to: ${fieldName}`);
}
async clickAddPatternCell(fieldName) {
testLogger.info(`Clicking Add Pattern cell for field: ${fieldName}`);
@@ -129,6 +177,9 @@ export class StreamAssociationPage {
// Wait for table content to stabilize (reduced from 3000ms)
await this.page.waitForTimeout(1000);
// Filter to the target field so its (possibly virtualized) row is in the DOM.
await this.filterSchemaField(fieldName);
// Find row with the field name WITHIN the schema table only
const fieldRow = schemaTable.locator(`tr:has-text("${fieldName}")`).first();
await fieldRow.waitFor({ state: 'visible', timeout: 10000 });
@@ -359,8 +410,8 @@ export class StreamAssociationPage {
await this.cancelButton.click();
}
async associatePatternWithStream(streamName, patternName, action, timeType, fieldName) {
testLogger.info(`Associating pattern ${patternName} with stream ${streamName} for field ${fieldName} (Action: ${action}, Time: ${timeType})`);
async associatePatternWithStream(streamName, patternName, action, timeType, fieldName, streamType = 'logs') {
testLogger.info(`Associating pattern ${patternName} with stream ${streamName} for field ${fieldName} (Action: ${action}, Time: ${timeType}, Type: ${streamType})`);
// Close any open sidebar first - use try-catch to handle unstable elements
try {
@@ -375,9 +426,9 @@ export class StreamAssociationPage {
testLogger.info('No sidebar to close or sidebar already closed');
}
await this.navigateToStreams();
await this.navigateToStreams(streamType);
await this.searchForStream(streamName);
await this.openStreamDetail(streamName);
await this.openStreamDetail(streamName, streamType);
await this.clickAddPatternCell(fieldName);
await this.searchAndSelectPattern(patternName);
@@ -435,12 +486,12 @@ export class StreamAssociationPage {
}
}
async unlinkPatternFromField(streamName, fieldName, patternName) {
testLogger.info(`Unlinking pattern ${patternName} from stream ${streamName}, field ${fieldName}`);
async unlinkPatternFromField(streamName, fieldName, patternName, streamType = 'logs') {
testLogger.info(`Unlinking pattern ${patternName} from stream ${streamName}, field ${fieldName} (Type: ${streamType})`);
await this.navigateToStreams();
await this.navigateToStreams(streamType);
await this.searchForStream(streamName);
await this.openStreamDetail(streamName);
await this.openStreamDetail(streamName, streamType);
await this.page.waitForTimeout(1500);
// Wait for schema table to be visible
@@ -448,6 +499,9 @@ export class StreamAssociationPage {
await schemaTable.waitFor({ state: 'visible', timeout: 10000 });
await this.page.waitForTimeout(2000);
// Filter to the target field so its (possibly virtualized) row is in the DOM.
await this.filterSchemaField(fieldName);
// Find the field row WITHIN the schema table
const fieldRow = schemaTable.locator(`tr:has-text("${fieldName}")`).first();
await fieldRow.waitFor({ state: 'visible', timeout: 5000 });
@@ -575,6 +629,9 @@ export class StreamAssociationPage {
await schemaTable.waitFor({ state: 'visible', timeout: 10000 });
await this.page.waitForTimeout(2000);
// Filter to the target field so its (possibly virtualized) row is in the DOM.
await this.filterSchemaField(fieldName);
// Find the field row WITHIN the schema table
const fieldRow = schemaTable.locator(`tr:has-text("${fieldName}")`).first();
await fieldRow.waitFor({ state: 'visible', timeout: 5000 });
@@ -709,6 +766,9 @@ export class StreamAssociationPage {
await schemaTable.waitFor({ state: 'visible', timeout: 10000 });
await this.page.waitForTimeout(1000);
// Filter to the target field so its (possibly virtualized) row is in the DOM.
await this.filterSchemaField(fieldName);
// Find row with the field name WITHIN the schema table only
const fieldRow = schemaTable.locator(`tr:has-text("${fieldName}")`).first();
await fieldRow.waitFor({ state: 'visible', timeout: 10000 });
@@ -817,6 +877,9 @@ export class StreamAssociationPage {
await schemaTable.waitFor({ state: 'visible', timeout: 10000 });
await this.page.waitForTimeout(2000);
// Filter to the target field so its (possibly virtualized) row is in the DOM.
await this.filterSchemaField(fieldName);
// Find the field row WITHIN the schema table
const fieldRow = schemaTable.locator(`tr:has-text("${fieldName}")`).first();
await fieldRow.waitFor({ state: 'visible', timeout: 5000 });
tests/ui-testing/playwright-tests/SDR/tracesIngestionTimeDrop.spec.js
+157
View File
@@ -0,0 +1,157 @@
const { test, expect, navigateToBase } = require('../utils/enhanced-baseFixtures.js');
const testLogger = require('../utils/test-logger.js');
const PageManager = require('../../pages/page-manager.js');
// Traces variant of the SDR ingestion-time drop suite. Dropped fields are removed
// from the record at write time; the span record itself persists (marker + trace
// fields), so the batch is still queryable while the dropped field is absent.
test.describe("Traces Ingestion Time Drop - Combined Test", { tag: '@enterprise' }, () => {
test.describe.configure({ mode: 'serial' });
let pm;
// Generate unique test run ID for isolation
const testRunId = Date.now().toString(36);
// All patterns used in this spec file - with full pattern definitions for uniqueness
const patternsToTest = [
{
name: `traces_url_drop_${testRunId}`,
description: 'URL validation pattern (for traces drop tests)',
pattern: '^https?://[\\w.-]+\\.[\\w.-]+.*$',
field: 'website_url',
value: 'https://example.com'
},
{
name: `traces_password_drop_${testRunId}`,
description: 'Strong password pattern (for traces drop tests)',
pattern: '^[A-Za-z0-9@$!%*?&]{8,}$',
field: 'user_password',
value: 'P@ssw0rd!123'
},
{
name: `traces_ipv4_drop_${testRunId}`,
description: 'IPv4 address format (for traces drop tests)',
pattern: '^(\\d{1,3}\\.){3}\\d{1,3}$',
field: 'server_ip',
value: '192.168.1.1'
},
{
name: `traces_hex_color_drop_${testRunId}`,
description: 'Hexadecimal color code (for traces drop tests)',
pattern: '^#[0-9a-fA-F]{6}$',
field: 'theme_color',
value: '#FF5733'
}
];
// Use unique stream name to avoid "stream being deleted" conflicts
const testStreamName = `sdr_traces_drop_${testRunId}`;
const streamType = 'traces';
test.beforeEach(async ({ page }, testInfo) => {
testLogger.testStart(testInfo.title, testInfo.file);
await navigateToBase(page);
pm = new PageManager(page);
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
testLogger.info('Combined traces ingestion time drop test setup completed');
});
// Cleanup test - runs first
test('setup: cleanup patterns', {
tag: ['@sdr', '@traces', '@cleanup', '@ingestionDrop']
}, async ({ page }) => {
testLogger.info('=== SPEC-LEVEL CLEANUP: Cleaning up patterns for traces drop test ===');
for (const pattern of patternsToTest) {
testLogger.info(`Checking and cleaning up pattern: ${pattern.name}`);
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.checkPatternExists(pattern.name);
if (exists) {
testLogger.info(`Pattern ${pattern.name} exists. Deleting (and unlinking if needed).`);
const deleteResult = await pm.sdrPatternsPage.deletePatternByName(pattern.name);
if (!deleteResult.success && deleteResult.reason === 'in_use') {
testLogger.info(`Pattern ${pattern.name} in use, unlinking from streams first`);
for (const association of deleteResult.associations) {
await pm.streamAssociationPage.unlinkPatternFromField(
association.streamName,
association.fieldName,
pattern.name,
streamType
);
}
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.deletePatternByName(pattern.name);
}
testLogger.info(`✓ Cleaned up pattern: ${pattern.name}`);
} else {
testLogger.info(`✓ Pattern ${pattern.name} does not exist, no cleanup needed`);
}
}
testLogger.info('=== SPEC-LEVEL CLEANUP COMPLETE ===');
});
// Combined test for all 4 patterns
test("should drop multiple trace fields at ingestion time when patterns are applied", {
tag: ['@sdr', '@traces', '@ingestionDrop', '@all', '@combined']
}, async ({ page }, testInfo) => {
testInfo.annotations.push({ type: 'dependency', description: 'setup: cleanup patterns' });
testLogger.info('=== Testing combined traces DROP for 4 patterns at ingestion time ===');
// STEP 1: Ingest trace data WITHOUT any SDR patterns - all fields should be visible
testLogger.info('STEP 1: Ingest 4 trace spans WITHOUT any SDR patterns linked');
const dataToIngest = patternsToTest.map(p => ({ fieldName: p.field, fieldValue: p.value }));
const markerStep1 = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
const fieldsToVerifyStep1 = patternsToTest.map(p => ({ fieldName: p.field, shouldBeDropped: false }));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsToVerifyStep1, markerStep1, streamType);
testLogger.info('✓ STEP 1 PASSED: All trace fields visible without SDR');
// STEP 2: Create all 4 SDR patterns
testLogger.info('STEP 2: Create all 4 SDR patterns');
for (const patternConfig of patternsToTest) {
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.createPattern(patternConfig.name, patternConfig.description, patternConfig.pattern);
await pm.sdrPatternsPage.verifyPatternCreatedSuccess();
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.waitForPatternCreated(patternConfig.name); // backend-consistent create gate (API source of truth)
if (!exists) {
testLogger.error(`Pattern ${patternConfig.name} was not created successfully!`);
throw new Error(`Pattern ${patternConfig.name} creation failed - not found in list`);
}
testLogger.info(`✓ Created and verified pattern: ${patternConfig.name}`);
}
testLogger.info('✓ STEP 2 PASSED: All 4 patterns created and verified');
// STEP 3: Link all 4 patterns to their respective trace fields with DROP action
testLogger.info('STEP 3: Link all 4 patterns to their respective trace fields with DROP action');
for (const patternConfig of patternsToTest) {
await pm.streamAssociationPage.associatePatternWithStream(
testStreamName,
patternConfig.name,
'drop',
'ingestion',
patternConfig.field,
streamType
);
testLogger.info(`✓ Linked pattern ${patternConfig.name} to field ${patternConfig.field} with DROP action`);
}
testLogger.info('✓ STEP 3 PASSED: All 4 patterns linked to trace fields with DROP action');
// STEP 4: Ingest trace data WITH SDR patterns linked - all fields should be DROPPED
// Ingestion-time drop removes the fields at write time, so this is a NEW batch
// (new marker) — distinct from the still-visible STEP 1 batch on the same stream.
testLogger.info('STEP 4: Ingest 4 trace spans WITH SDR patterns linked');
const markerStep4 = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
const fieldsToCheck = patternsToTest.map(p => p.field);
await pm.sdrVerificationPage.verifyFieldsAreAbsent(pm.logsPage, testStreamName, fieldsToCheck, markerStep4, streamType);
testLogger.info('✓ STEP 4 PASSED: All trace fields are DROPPED (not present)');
testLogger.info('=== ✓ COMBINED TRACES DROP TEST COMPLETED SUCCESSFULLY ===');
});
});
tests/ui-testing/playwright-tests/SDR/tracesIngestionTimeHash.spec.js
+157
View File
@@ -0,0 +1,157 @@
const { test, expect, navigateToBase } = require('../utils/enhanced-baseFixtures.js');
const testLogger = require('../utils/test-logger.js');
const PageManager = require('../../pages/page-manager.js');
// Traces variant of the SDR ingestion-time hash suite. Matching values are
// replaced with a [REDACTED:<hash>] marker at write time. Traces read back via
// ?type=traces.
test.describe("Traces Ingestion Time Hash - Combined Test", { tag: '@enterprise' }, () => {
test.describe.configure({ mode: 'serial' });
let pm;
// Generate unique test run ID for isolation
const testRunId = Date.now().toString(36);
// All patterns used in this spec file - with full pattern definitions for uniqueness
const patternsToTest = [
{
name: `traces_url_hash_${testRunId}`,
description: 'URL validation pattern (for traces hash tests)',
pattern: '^https?://[\\w.-]+\\.[\\w.-]+.*$',
field: 'website_url',
value: 'https://example.com'
},
{
name: `traces_password_hash_${testRunId}`,
description: 'Strong password pattern (for traces hash tests)',
pattern: '^[A-Za-z0-9@$!%*?&]{8,}$',
field: 'user_password',
value: 'P@ssw0rd!123'
},
{
name: `traces_ipv4_hash_${testRunId}`,
description: 'IPv4 address format (for traces hash tests)',
pattern: '^(\\d{1,3}\\.){3}\\d{1,3}$',
field: 'server_ip',
value: '192.168.1.1'
},
{
name: `traces_hex_color_hash_${testRunId}`,
description: 'Hexadecimal color code (for traces hash tests)',
pattern: '^#[0-9a-fA-F]{6}$',
field: 'theme_color',
value: '#FF5733'
}
];
// Use unique stream name to avoid "stream being deleted" conflicts
const testStreamName = `sdr_traces_hash_${testRunId}`;
const streamType = 'traces';
test.beforeEach(async ({ page }, testInfo) => {
testLogger.testStart(testInfo.title, testInfo.file);
await navigateToBase(page);
pm = new PageManager(page);
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
testLogger.info('Combined traces ingestion time hash test setup completed');
});
// Cleanup test - runs first
test('setup: cleanup patterns', {
tag: ['@sdr', '@traces', '@cleanup', '@ingestionHash']
}, async ({ page }) => {
testLogger.info('=== SPEC-LEVEL CLEANUP: Cleaning up patterns for traces hash test ===');
for (const pattern of patternsToTest) {
testLogger.info(`Checking and cleaning up pattern: ${pattern.name}`);
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.checkPatternExists(pattern.name);
if (exists) {
testLogger.info(`Pattern ${pattern.name} exists. Deleting (and unlinking if needed).`);
const deleteResult = await pm.sdrPatternsPage.deletePatternByName(pattern.name);
if (!deleteResult.success && deleteResult.reason === 'in_use') {
testLogger.info(`Pattern ${pattern.name} in use, unlinking from streams first`);
for (const association of deleteResult.associations) {
await pm.streamAssociationPage.unlinkPatternFromField(
association.streamName,
association.fieldName,
pattern.name,
streamType
);
}
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.deletePatternByName(pattern.name);
}
testLogger.info(`✓ Cleaned up pattern: ${pattern.name}`);
} else {
testLogger.info(`✓ Pattern ${pattern.name} does not exist, no cleanup needed`);
}
}
testLogger.info('=== SPEC-LEVEL CLEANUP COMPLETE ===');
});
// Combined test for all 4 patterns
test("should hash multiple trace fields at ingestion time when patterns are applied", {
tag: ['@sdr', '@traces', '@ingestionHash', '@all', '@combined']
}, async ({ page }, testInfo) => {
testInfo.annotations.push({ type: 'dependency', description: 'setup: cleanup patterns' });
testLogger.info('=== Testing combined traces HASH for 4 patterns at ingestion time ===');
// STEP 1: Ingest trace data WITHOUT any SDR patterns - all fields should be visible
testLogger.info('STEP 1: Ingest 4 trace spans WITHOUT any SDR patterns linked');
const dataToIngest = patternsToTest.map(p => ({ fieldName: p.field, fieldValue: p.value }));
const markerStep1 = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
const fieldsToVerifyStep1 = patternsToTest.map(p => ({ fieldName: p.field, shouldBeHashed: false }));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsToVerifyStep1, markerStep1, streamType);
testLogger.info('✓ STEP 1 PASSED: All trace fields visible without SDR');
// STEP 2: Create all 4 SDR patterns
testLogger.info('STEP 2: Create all 4 SDR patterns');
for (const patternConfig of patternsToTest) {
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.createPattern(patternConfig.name, patternConfig.description, patternConfig.pattern);
await pm.sdrPatternsPage.verifyPatternCreatedSuccess();
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.waitForPatternCreated(patternConfig.name); // backend-consistent create gate (API source of truth)
if (!exists) {
testLogger.error(`Pattern ${patternConfig.name} was not created successfully!`);
throw new Error(`Pattern ${patternConfig.name} creation failed - not found in list`);
}
testLogger.info(`✓ Created and verified pattern: ${patternConfig.name}`);
}
testLogger.info('✓ STEP 2 PASSED: All 4 patterns created and verified');
// STEP 3: Link all 4 patterns to their respective trace fields with HASH action
testLogger.info('STEP 3: Link all 4 patterns to their respective trace fields with HASH action');
for (const patternConfig of patternsToTest) {
await pm.streamAssociationPage.associatePatternWithStream(
testStreamName,
patternConfig.name,
'hash',
'ingestion',
patternConfig.field,
streamType
);
testLogger.info(`✓ Linked pattern ${patternConfig.name} to field ${patternConfig.field} with HASH action`);
}
testLogger.info('✓ STEP 3 PASSED: All 4 patterns linked to trace fields with HASH action');
// STEP 4: Ingest trace data WITH SDR patterns linked - all fields should be HASHED
// Ingestion-time hash is baked in at write time, so this is a NEW batch
// (new marker) — distinct from the still-visible STEP 1 batch on the same stream.
testLogger.info('STEP 4: Ingest 4 trace spans WITH SDR patterns linked');
const markerStep4 = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
const fieldsToVerifyStep4 = patternsToTest.map(p => ({ fieldName: p.field, shouldBeHashed: true }));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsToVerifyStep4, markerStep4, streamType);
testLogger.info('✓ STEP 4 PASSED: All trace fields are HASHED');
testLogger.info('=== ✓ COMBINED TRACES HASH TEST COMPLETED SUCCESSFULLY ===');
});
});
tests/ui-testing/playwright-tests/SDR/tracesIngestionTimeRedaction.spec.js
+158
View File
@@ -0,0 +1,158 @@
const { test, expect, navigateToBase } = require('../utils/enhanced-baseFixtures.js');
const testLogger = require('../utils/test-logger.js');
const PageManager = require('../../pages/page-manager.js');
// Traces variant of the SDR ingestion-time redaction suite. SDR was originally
// applied to logs only; this suite covers the traces pipeline (PR: apply SDR
// patterns to traces). Only the ingest (OTLP /v1/traces) and read (?type=traces)
// seams differ — the pattern create/associate/verify flow is shared with logs.
test.describe("Traces Ingestion Time Redaction - Combined Test", { tag: '@enterprise' }, () => {
test.describe.configure({ mode: 'serial' });
let pm;
// Generate unique test run ID for isolation
const testRunId = Date.now().toString(36);
// All patterns used in this spec file - with full pattern definitions for uniqueness
const patternsToTest = [
{
name: `traces_email_redact_${testRunId}`,
description: 'Email address validation pattern (for traces redaction tests)',
pattern: '^[\\w\\.-]+@[\\w\\.-]+\\.\\w{2,}$',
field: 'user_email',
value: 'john.doe@example.com'
},
{
name: `traces_phone_redact_${testRunId}`,
description: 'US phone number (10 digits) (for traces redaction tests)',
pattern: '^\\d{10}$',
field: 'phone',
value: '5551234567'
},
{
name: `traces_cc_redact_${testRunId}`,
description: 'Credit card number pattern (for traces redaction tests)',
pattern: '^\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}$',
field: 'cc_number',
value: '1234 5678 9012 3456'
},
{
name: `traces_ssn_redact_${testRunId}`,
description: 'Social Security Number pattern (for traces redaction tests)',
pattern: '^\\d{3}-\\d{2}-\\d{4}$',
field: 'ssn',
value: '123-45-6789'
}
];
// Use unique stream name to avoid "stream being deleted" conflicts
const testStreamName = `sdr_traces_redact_${testRunId}`;
const streamType = 'traces';
test.beforeEach(async ({ page }, testInfo) => {
testLogger.testStart(testInfo.title, testInfo.file);
await navigateToBase(page);
pm = new PageManager(page);
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
testLogger.info('Combined traces ingestion time redaction test setup completed');
});
// Cleanup test - runs first
test('setup: cleanup patterns', {
tag: ['@sdr', '@traces', '@cleanup', '@ingestionRedact']
}, async ({ page }) => {
testLogger.info('=== SPEC-LEVEL CLEANUP: Cleaning up patterns for traces redaction test ===');
for (const pattern of patternsToTest) {
testLogger.info(`Checking and cleaning up pattern: ${pattern.name}`);
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.checkPatternExists(pattern.name);
if (exists) {
testLogger.info(`Pattern ${pattern.name} exists. Deleting (and unlinking if needed).`);
const deleteResult = await pm.sdrPatternsPage.deletePatternByName(pattern.name);
if (!deleteResult.success && deleteResult.reason === 'in_use') {
testLogger.info(`Pattern ${pattern.name} in use, unlinking from streams first`);
for (const association of deleteResult.associations) {
await pm.streamAssociationPage.unlinkPatternFromField(
association.streamName,
association.fieldName,
pattern.name,
streamType
);
}
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.deletePatternByName(pattern.name);
}
testLogger.info(`✓ Cleaned up pattern: ${pattern.name}`);
} else {
testLogger.info(`✓ Pattern ${pattern.name} does not exist, no cleanup needed`);
}
}
testLogger.info('=== SPEC-LEVEL CLEANUP COMPLETE ===');
});
// Combined test for all 4 patterns
test("should redact multiple trace fields at ingestion time when patterns are applied", {
tag: ['@sdr', '@traces', '@ingestionRedact', '@all', '@combined']
}, async ({ page }, testInfo) => {
testInfo.annotations.push({ type: 'dependency', description: 'setup: cleanup patterns' });
testLogger.info('=== Testing combined traces redaction for 4 patterns at ingestion time ===');
// STEP 1: Ingest trace data WITHOUT any SDR patterns - all fields should be visible
testLogger.info('STEP 1: Ingest 4 trace spans WITHOUT any SDR patterns linked');
const dataToIngest = patternsToTest.map(p => ({ fieldName: p.field, fieldValue: p.value }));
const markerStep1 = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
const fieldsToVerifyStep1 = patternsToTest.map(p => ({ fieldName: p.field, shouldBeRedacted: false }));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsToVerifyStep1, markerStep1, streamType);
testLogger.info('✓ STEP 1 PASSED: All trace fields visible without SDR');
// STEP 2: Create all 4 SDR patterns
testLogger.info('STEP 2: Create all 4 SDR patterns');
for (const patternConfig of patternsToTest) {
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.createPattern(patternConfig.name, patternConfig.description, patternConfig.pattern);
await pm.sdrPatternsPage.verifyPatternCreatedSuccess();
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.waitForPatternCreated(patternConfig.name); // backend-consistent create gate (API source of truth)
if (!exists) {
testLogger.error(`Pattern ${patternConfig.name} was not created successfully!`);
throw new Error(`Pattern ${patternConfig.name} creation failed - not found in list`);
}
testLogger.info(`✓ Created and verified pattern: ${patternConfig.name}`);
}
testLogger.info('✓ STEP 2 PASSED: All 4 patterns created and verified');
// STEP 3: Link all 4 patterns to their respective trace fields
testLogger.info('STEP 3: Link all 4 patterns to their respective trace fields');
for (const patternConfig of patternsToTest) {
await pm.streamAssociationPage.associatePatternWithStream(
testStreamName,
patternConfig.name,
'redact',
'ingestion',
patternConfig.field,
streamType
);
testLogger.info(`✓ Linked pattern ${patternConfig.name} to field ${patternConfig.field}`);
}
testLogger.info('✓ STEP 3 PASSED: All 4 patterns linked to trace fields');
// STEP 4: Ingest trace data WITH SDR patterns linked - all fields should be REDACTED
// Ingestion-time redaction is baked in at write time, so this is a NEW batch
// (new marker) — distinct from the still-visible STEP 1 batch on the same stream.
testLogger.info('STEP 4: Ingest 4 trace spans WITH SDR patterns linked');
const markerStep4 = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
const fieldsToVerifyStep4 = patternsToTest.map(p => ({ fieldName: p.field, shouldBeRedacted: true }));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsToVerifyStep4, markerStep4, streamType);
testLogger.info('✓ STEP 4 PASSED: All trace fields are REDACTED');
testLogger.info('=== ✓ COMBINED TRACES REDACTION TEST COMPLETED SUCCESSFULLY ===');
});
});
tests/ui-testing/playwright-tests/SDR/tracesQueryTimeDrop.spec.js
+168
View File
@@ -0,0 +1,168 @@
const { test, expect, navigateToBase } = require('../utils/enhanced-baseFixtures.js');
const testLogger = require('../utils/test-logger.js');
const PageManager = require('../../pages/page-manager.js');
// Traces variant of the SDR query-time drop suite. Data stays intact in storage;
// the field is dropped from the search response at query time. One ingested batch
// is inspected both before (visible) and after (dropped) the pattern is applied.
test.describe("Traces Query Time Drop - Combined Test", { tag: '@enterprise' }, () => {
test.describe.configure({ mode: 'serial' });
let pm;
// Generate unique test run ID for isolation
const testRunId = Date.now().toString(36);
// All patterns used in this spec file - with full pattern definitions for uniqueness
const patternsToTest = [
{
name: `traces_url_query_drop_${testRunId}`,
description: 'URL validation pattern (for traces query drop tests)',
pattern: '^https?://[\\w.-]+\\.[\\w.-]+.*$',
field: 'website_url',
value: 'https://example.com'
},
{
name: `traces_password_query_drop_${testRunId}`,
description: 'Strong password pattern (for traces query drop tests)',
pattern: '^[A-Za-z0-9@$!%*?&]{8,}$',
field: 'user_password',
value: 'P@ssw0rd!123'
},
{
name: `traces_ipv4_query_drop_${testRunId}`,
description: 'IPv4 address format (for traces query drop tests)',
pattern: '^(\\d{1,3}\\.){3}\\d{1,3}$',
field: 'server_ip',
value: '192.168.1.1'
},
{
name: `traces_hex_color_query_drop_${testRunId}`,
description: 'Hexadecimal color code (for traces query drop tests)',
pattern: '^#[0-9a-fA-F]{6}$',
field: 'theme_color',
value: '#FF5733'
}
];
// Use unique stream name to avoid "stream being deleted" conflicts
const testStreamName = `sdr_traces_query_drop_${testRunId}`;
const streamType = 'traces';
test.beforeEach(async ({ page }, testInfo) => {
testLogger.testStart(testInfo.title, testInfo.file);
await navigateToBase(page);
pm = new PageManager(page);
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
testLogger.info('Combined traces query time drop test setup completed');
});
test("setup: cleanup patterns", {
tag: ['@sdr', '@traces', '@cleanup', '@queryDrop']
}, async ({ page }) => {
testLogger.info('=== SPEC-LEVEL CLEANUP: Cleaning up patterns for traces query drop test ===');
for (const pattern of patternsToTest) {
testLogger.info(`Checking and cleaning up pattern: ${pattern.name}`);
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.checkPatternExists(pattern.name);
if (exists) {
testLogger.info(`Pattern ${pattern.name} exists. Deleting (and unlinking if needed).`);
const deleteResult = await pm.sdrPatternsPage.deletePatternByName(pattern.name);
if (!deleteResult.success && deleteResult.reason === 'in_use') {
testLogger.info('Pattern in use, unlinking from streams first');
for (const association of deleteResult.associations) {
await pm.streamAssociationPage.unlinkPatternFromField(
association.streamName,
association.fieldName,
pattern.name,
streamType
);
}
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.deletePatternByName(pattern.name);
}
testLogger.info(`✓ Pattern ${pattern.name} deleted successfully`);
} else {
testLogger.info(`✓ Pattern ${pattern.name} does not exist, no cleanup needed`);
}
}
testLogger.info('=== SPEC-LEVEL CLEANUP COMPLETE ===');
});
test("should drop multiple trace fields at query time when patterns are applied", {
tag: ['@sdr', '@traces', '@queryDrop', '@all', '@combined']
}, async ({ page }, testInfo) => {
testInfo.annotations.push({ type: 'dependency', description: 'setup: cleanup patterns' });
testLogger.info('=== Testing combined traces drop for 4 patterns at query time ===');
// STEP 1: Ingest trace data once (data will remain unchanged in storage)
testLogger.info('STEP 1: Ingest 4 trace spans (before any patterns are applied)');
const dataToIngest = patternsToTest.map(p => ({
fieldName: p.field,
fieldValue: p.value
}));
// Query-time SDR transforms data at search time, so STEP 1 (visible) and STEP 4
// (dropped) inspect the SAME ingested batch — reuse one marker for both.
const ingestMarker = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
// Verify all fields are visible before drop
const fieldsBeforeDrop = patternsToTest.map(p => ({
fieldName: p.field,
shouldBeDropped: false
}));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsBeforeDrop, ingestMarker, streamType);
testLogger.info('✓ STEP 1 PASSED: All trace fields visible without SDR');
// STEP 2: Create all 4 SDR patterns
testLogger.info('STEP 2: Create all 4 SDR patterns');
for (const patternConfig of patternsToTest) {
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.createPattern(patternConfig.name, patternConfig.description, patternConfig.pattern);
await pm.sdrPatternsPage.verifyPatternCreatedSuccess();
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.waitForPatternCreated(patternConfig.name); // backend-consistent create gate (API source of truth)
expect(exists).toBeTruthy();
testLogger.info(`✓ Created and verified pattern: ${patternConfig.name}`);
}
testLogger.info('✓ STEP 2 PASSED: All 4 patterns created and verified');
// STEP 3: Link all 4 patterns to their respective trace fields with QUERY TIME drop
testLogger.info('STEP 3: Link all 4 patterns to their respective trace fields with DROP action');
for (const patternConfig of patternsToTest) {
await pm.streamAssociationPage.associatePatternWithStream(
testStreamName,
patternConfig.name,
'drop',
'query', // QUERY TIME - key difference from ingestion time
patternConfig.field,
streamType
);
testLogger.info(`✓ Linked pattern ${patternConfig.name} to field ${patternConfig.field} with DROP action`);
}
testLogger.info('✓ STEP 3 PASSED: All 4 patterns linked to trace fields with DROP action');
// STEP 4: Verify all fields are DROPPED at query time (NO RE-INGESTION)
testLogger.info('STEP 4: Verify trace fields are DROPPED at query time (data unchanged in storage)');
const fieldsAfterDrop = patternsToTest.map(p => ({
fieldName: p.field,
shouldBeDropped: true
}));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsAfterDrop, ingestMarker, streamType);
testLogger.info('✓ STEP 4 PASSED: All trace fields are DROPPED at query time');
testLogger.info('=== ✓ COMBINED TRACES QUERY TIME DROP TEST COMPLETED SUCCESSFULLY ===');
});
});
tests/ui-testing/playwright-tests/SDR/tracesQueryTimeHash.spec.js
+169
View File
@@ -0,0 +1,169 @@
const { test, expect, navigateToBase } = require('../utils/enhanced-baseFixtures.js');
const testLogger = require('../utils/test-logger.js');
const PageManager = require('../../pages/page-manager.js');
// Traces variant of the SDR query-time hash suite. Data stays intact in storage;
// matching values are replaced with [REDACTED:<hash>] in the search response at
// query time. One ingested batch is inspected both before (visible) and after
// (hashed) the pattern is applied.
test.describe("Traces Query Time Hash - Combined Test", { tag: '@enterprise' }, () => {
test.describe.configure({ mode: 'serial' });
let pm;
// Generate unique test run ID for isolation
const testRunId = Date.now().toString(36);
// All patterns used in this spec file - with full pattern definitions for uniqueness
const patternsToTest = [
{
name: `traces_url_query_hash_${testRunId}`,
description: 'URL validation pattern (for traces query hash tests)',
pattern: '^https?://[\\w.-]+\\.[\\w.-]+.*$',
field: 'website_url',
value: 'https://example.com'
},
{
name: `traces_password_query_hash_${testRunId}`,
description: 'Strong password pattern (for traces query hash tests)',
pattern: '^[A-Za-z0-9@$!%*?&]{8,}$',
field: 'user_password',
value: 'P@ssw0rd!123'
},
{
name: `traces_ipv4_query_hash_${testRunId}`,
description: 'IPv4 address format (for traces query hash tests)',
pattern: '^(\\d{1,3}\\.){3}\\d{1,3}$',
field: 'server_ip',
value: '192.168.1.1'
},
{
name: `traces_hex_color_query_hash_${testRunId}`,
description: 'Hexadecimal color code (for traces query hash tests)',
pattern: '^#[0-9a-fA-F]{6}$',
field: 'theme_color',
value: '#FF5733'
}
];
// Use unique stream name to avoid "stream being deleted" conflicts
const testStreamName = `sdr_traces_query_hash_${testRunId}`;
const streamType = 'traces';
test.beforeEach(async ({ page }, testInfo) => {
testLogger.testStart(testInfo.title, testInfo.file);
await navigateToBase(page);
pm = new PageManager(page);
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
testLogger.info('Combined traces query time hash test setup completed');
});
test("setup: cleanup patterns", {
tag: ['@sdr', '@traces', '@cleanup', '@queryHash']
}, async ({ page }) => {
testLogger.info('=== SPEC-LEVEL CLEANUP: Cleaning up patterns for traces query hash test ===');
for (const pattern of patternsToTest) {
testLogger.info(`Checking and cleaning up pattern: ${pattern.name}`);
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.checkPatternExists(pattern.name);
if (exists) {
testLogger.info(`Pattern ${pattern.name} exists. Deleting (and unlinking if needed).`);
const deleteResult = await pm.sdrPatternsPage.deletePatternByName(pattern.name);
if (!deleteResult.success && deleteResult.reason === 'in_use') {
testLogger.info('Pattern in use, unlinking from streams first');
for (const association of deleteResult.associations) {
await pm.streamAssociationPage.unlinkPatternFromField(
association.streamName,
association.fieldName,
pattern.name,
streamType
);
}
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.deletePatternByName(pattern.name);
}
testLogger.info(`✓ Pattern ${pattern.name} deleted successfully`);
} else {
testLogger.info(`✓ Pattern ${pattern.name} does not exist, no cleanup needed`);
}
}
testLogger.info('=== SPEC-LEVEL CLEANUP COMPLETE ===');
});
test("should hash multiple trace fields at query time when patterns are applied", {
tag: ['@sdr', '@traces', '@queryHash', '@all', '@combined']
}, async ({ page }, testInfo) => {
testInfo.annotations.push({ type: 'dependency', description: 'setup: cleanup patterns' });
testLogger.info('=== Testing combined traces hash for 4 patterns at query time ===');
// STEP 1: Ingest trace data once (data will remain unchanged in storage)
testLogger.info('STEP 1: Ingest 4 trace spans (before any patterns are applied)');
const dataToIngest = patternsToTest.map(p => ({
fieldName: p.field,
fieldValue: p.value
}));
// Query-time SDR transforms data at search time, so STEP 1 (visible) and STEP 4
// (hashed) inspect the SAME ingested batch — reuse one marker for both.
const ingestMarker = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
// Verify all fields are visible without hashing
const fieldsBeforeHash = patternsToTest.map(p => ({
fieldName: p.field,
shouldBeHashed: false
}));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsBeforeHash, ingestMarker, streamType);
testLogger.info('✓ STEP 1 PASSED: All trace fields visible without SDR');
// STEP 2: Create all 4 SDR patterns
testLogger.info('STEP 2: Create all 4 SDR patterns');
for (const patternConfig of patternsToTest) {
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.createPattern(patternConfig.name, patternConfig.description, patternConfig.pattern);
await pm.sdrPatternsPage.verifyPatternCreatedSuccess();
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.waitForPatternCreated(patternConfig.name); // backend-consistent create gate (API source of truth)
expect(exists).toBeTruthy();
testLogger.info(`✓ Created and verified pattern: ${patternConfig.name}`);
}
testLogger.info('✓ STEP 2 PASSED: All 4 patterns created and verified');
// STEP 3: Link all 4 patterns to their respective trace fields with QUERY TIME hash
testLogger.info('STEP 3: Link all 4 patterns to their respective trace fields with HASH action');
for (const patternConfig of patternsToTest) {
await pm.streamAssociationPage.associatePatternWithStream(
testStreamName,
patternConfig.name,
'hash',
'query', // QUERY TIME - key difference from ingestion time
patternConfig.field,
streamType
);
testLogger.info(`✓ Linked pattern ${patternConfig.name} to field ${patternConfig.field} with HASH action`);
}
testLogger.info('✓ STEP 3 PASSED: All 4 patterns linked to trace fields with HASH action');
// STEP 4: Verify all fields are HASHED at query time (NO RE-INGESTION)
testLogger.info('STEP 4: Verify trace fields are HASHED at query time (data unchanged in storage)');
const fieldsAfterHash = patternsToTest.map(p => ({
fieldName: p.field,
shouldBeHashed: true
}));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsAfterHash, ingestMarker, streamType);
testLogger.info('✓ STEP 4 PASSED: All trace fields are HASHED at query time');
testLogger.info('=== ✓ COMBINED TRACES QUERY TIME HASH TEST COMPLETED SUCCESSFULLY ===');
});
});
tests/ui-testing/playwright-tests/SDR/tracesQueryTimeRedaction.spec.js
+168
View File
@@ -0,0 +1,168 @@
const { test, expect, navigateToBase } = require('../utils/enhanced-baseFixtures.js');
const testLogger = require('../utils/test-logger.js');
const PageManager = require('../../pages/page-manager.js');
// Traces variant of the SDR query-time redaction suite. Query-time SDR transforms
// data at search time, so one ingested batch is inspected both before (visible)
// and after (redacted) the pattern is applied. Traces read back via ?type=traces.
test.describe("Traces Query Time Redaction - Combined Test", { tag: '@enterprise' }, () => {
test.describe.configure({ mode: 'serial' });
let pm;
// Generate unique test run ID for isolation
const testRunId = Date.now().toString(36);
// All patterns used in this spec file - with full pattern definitions for uniqueness
const patternsToTest = [
{
name: `traces_email_query_redact_${testRunId}`,
description: 'Email address validation pattern (for traces query time redaction tests)',
pattern: '^[\\w\\.-]+@[\\w\\.-]+\\.\\w{2,}$',
field: 'user_email',
value: 'john.doe@example.com'
},
{
name: `traces_phone_query_redact_${testRunId}`,
description: 'US phone number (10 digits) (for traces query time redaction tests)',
pattern: '^\\d{10}$',
field: 'phone',
value: '5551234567'
},
{
name: `traces_cc_query_redact_${testRunId}`,
description: 'Credit card number pattern (for traces query time redaction tests)',
pattern: '^\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}$',
field: 'cc_number',
value: '1234 5678 9012 3456'
},
{
name: `traces_ssn_query_redact_${testRunId}`,
description: 'Social Security Number pattern (for traces query time redaction tests)',
pattern: '^\\d{3}-\\d{2}-\\d{4}$',
field: 'ssn',
value: '123-45-6789'
}
];
// Use unique stream name to avoid "stream being deleted" conflicts
const testStreamName = `sdr_traces_query_redact_${testRunId}`;
const streamType = 'traces';
test.beforeEach(async ({ page }, testInfo) => {
testLogger.testStart(testInfo.title, testInfo.file);
await navigateToBase(page);
pm = new PageManager(page);
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
testLogger.info('Combined traces query time redaction test setup completed');
});
test("setup: cleanup patterns", {
tag: ['@sdr', '@traces', '@cleanup', '@queryRedact']
}, async ({ page }) => {
testLogger.info('=== SPEC-LEVEL CLEANUP: Cleaning up patterns for traces query redaction test ===');
for (const pattern of patternsToTest) {
testLogger.info(`Checking and cleaning up pattern: ${pattern.name}`);
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.checkPatternExists(pattern.name);
if (exists) {
testLogger.info(`Pattern ${pattern.name} exists. Deleting (and unlinking if needed).`);
const deleteResult = await pm.sdrPatternsPage.deletePatternByName(pattern.name);
if (!deleteResult.success && deleteResult.reason === 'in_use') {
testLogger.info('Pattern in use, unlinking from streams first');
for (const association of deleteResult.associations) {
await pm.streamAssociationPage.unlinkPatternFromField(
association.streamName,
association.fieldName,
pattern.name,
streamType
);
}
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.deletePatternByName(pattern.name);
}
testLogger.info(`✓ Pattern ${pattern.name} deleted successfully`);
} else {
testLogger.info(`✓ Pattern ${pattern.name} does not exist, no cleanup needed`);
}
}
testLogger.info('=== SPEC-LEVEL CLEANUP COMPLETE ===');
});
test("should redact multiple trace fields at query time when patterns are applied", {
tag: ['@sdr', '@traces', '@queryRedact', '@all', '@combined']
}, async ({ page }, testInfo) => {
testInfo.annotations.push({ type: 'dependency', description: 'setup: cleanup patterns' });
testLogger.info('=== Testing combined traces redaction for 4 patterns at query time ===');
// STEP 1: Ingest trace data once (data will remain unchanged in storage)
testLogger.info('STEP 1: Ingest 4 trace spans (before any patterns are applied)');
const dataToIngest = patternsToTest.map(p => ({
fieldName: p.field,
fieldValue: p.value
}));
// Query-time SDR transforms data at search time, so STEP 1 (visible) and STEP 4
// (redacted) inspect the SAME ingested batch — reuse one marker for both.
const ingestMarker = await pm.sdrTracesPage.ingestMultipleFields(testStreamName, dataToIngest);
// Verify all fields are visible without redaction
const fieldsBeforeRedaction = patternsToTest.map(p => ({
fieldName: p.field,
shouldBeRedacted: false
}));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsBeforeRedaction, ingestMarker, streamType);
testLogger.info('✓ STEP 1 PASSED: All trace fields visible without SDR');
// STEP 2: Create all 4 SDR patterns
testLogger.info('STEP 2: Create all 4 SDR patterns');
for (const patternConfig of patternsToTest) {
await pm.sdrPatternsPage.navigateToRegexPatterns();
await pm.sdrPatternsPage.createPattern(patternConfig.name, patternConfig.description, patternConfig.pattern);
await pm.sdrPatternsPage.verifyPatternCreatedSuccess();
await pm.sdrPatternsPage.navigateToRegexPatterns();
const exists = await pm.sdrPatternsPage.waitForPatternCreated(patternConfig.name); // backend-consistent create gate (API source of truth)
expect(exists).toBeTruthy();
testLogger.info(`✓ Created and verified pattern: ${patternConfig.name}`);
}
testLogger.info('✓ STEP 2 PASSED: All 4 patterns created and verified');
// STEP 3: Link all 4 patterns to their respective trace fields with QUERY TIME redaction
testLogger.info('STEP 3: Link all 4 patterns to their respective trace fields');
for (const patternConfig of patternsToTest) {
await pm.streamAssociationPage.associatePatternWithStream(
testStreamName,
patternConfig.name,
'redact',
'query', // QUERY TIME - key difference from ingestion time
patternConfig.field,
streamType
);
testLogger.info(`✓ Linked pattern ${patternConfig.name} to field ${patternConfig.field}`);
}
testLogger.info('✓ STEP 3 PASSED: All 4 patterns linked to trace fields');
// STEP 4: Verify all fields are REDACTED at query time (NO RE-INGESTION)
testLogger.info('STEP 4: Verify trace fields are REDACTED at query time (data unchanged in storage)');
const fieldsAfterRedaction = patternsToTest.map(p => ({
fieldName: p.field,
shouldBeRedacted: true
}));
await pm.sdrVerificationPage.verifyMultipleFields(pm.logsPage, testStreamName, fieldsAfterRedaction, ingestMarker, streamType);
testLogger.info('✓ STEP 4 PASSED: All trace fields are REDACTED at query time');
testLogger.info('=== ✓ COMBINED TRACES QUERY TIME REDACTION TEST COMPLETED SUCCESSFULLY ===');
});
});
tests/ui-testing/playwright-tests/cleanup.spec.js
+12 -1
View File
@@ -285,7 +285,7 @@ test.describe("Pre-Test Cleanup", () => {
/^sanitylogstream_/, // sanitylogstream_61hj, etc.
/^test\d+$/, // test1, test2, test3, etc.
/^stress_test/, // stress_test*, stress_test_<runId>_w0, stress_test1, etc.
/^sdr_/, // sdr_* (SDR test streams)
/^sdr_/, // sdr_* (SDR logs test streams; traces SDR streams are swept separately by type below)
/^e2e_join_/, // e2e_join_* (UNION test streams)
/^e2e_conditions_/, // e2e_conditions_* (Pipeline conditions UI test streams)
/^e2e_apostrophe_/, // e2e_apostrophe_* (Bug #9475 apostrophe test streams from logs-regression.spec.js)
@@ -338,6 +338,17 @@ test.describe("Pre-Test Cleanup", () => {
['default', 'sensitive', 'important', 'critical', 'production', 'staging', 'automation', 'e2e_automate', 'k8s_json']
);
// Clean up TRACES streams created by the traces SDR suite (sdr_traces_* — one
// per run, unique testRunId suffix). These are type=traces, so the logs sweep
// above (type=logs) never lists them; sweep them with streamType: 'traces'.
await pm.apiCleanup.cleanupStreams(
[
/^sdr_traces_/, // sdr_traces_* (traces SDR test streams: redact/drop/hash × ingestion/query)
],
['default'], // protect the default traces stream
{ streamType: 'traces' }
);
// Note: Pipeline regression test streams (e2e_test_cpu_usage for metrics, e2e_test_traces for traces)
// are created by pipeline-regression.spec.js for Issue #9901 regression tests.
// Custom traces stream uses "stream-name" header (ZO_GRPC_STREAM_HEADER_KEY config).