mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 00:08:55 +00:00
test(functions): un-skip JS-in-all-orgs tests + add per-event runtime-error coverage (#13341)
Replaces the `skip` band-aids that PR #13156 ("workflows v1") left on the **Functions** tests with real, running coverage of the current behaviour — on the correct edition — plus new tests for behaviour #13156 introduced but never covered. Test files only, no feature-code churn. Clean branch off `main`. ## Background — what #13156 actually changed #13156 bundled a Functions backend change unrelated to Workflows, and the old tests still asserted the *pre*-#13156 rules, so they were silenced with `pytest.skip(...)` / `test.skip(...)`. Two rules changed: 1. **JS is no longer `_meta`-only at the backend.** `save_function` / `test_run_function` have **no org gate on any edition** — JS (`transType:1`) is accepted in every org, including `default`, on **both OSS and Enterprise**. The `_meta`-only rule now lives **only in the frontend** (`AddFunction.vue` gates the JS radio on `isEnterprise || isCloud || org === "_meta"`, baked at build time by `VITE_OPENOBSERVE_ENTERPRISE`). 2. **The compile-check reports only *syntax* errors, not *runtime* errors.** A syntax error still returns **400**; code that compiles but throws at runtime now returns **200** with the error surfaced **per-event** in `results[].message` — for **both VRL and JS**. Because the API tests hit the backend directly, there is **no edition difference to assert for them** — the earlier "OSS asserts 400 / ENT asserts 200" framing was incorrect. Only the **e2e** (frontend JS-radio) is genuinely edition-dependent. ## API tests (pytest · `tests/api-testing`) **`test_functions.py`** - Keep the compile-error → 400 test (`test_test_endpoint_with_invalid_vrl_returns_400`) — it was never actually failing; `test`/`=====` are genuine VRL *compile* errors (`undefined variable` / syntax) and still 400. - Add `test_test_endpoint_with_js_runtime_error_returns_200` and `test_test_endpoint_with_vrl_runtime_error_returns_200` — runtime errors now return 200 with a per-event `results[].message`. **`test_functions_meta_org_restriction.py`** - Flip the 3 previously-skipped "blocked in default" tests → assert **200** (JS allowed in every org): `..._blocked` → `..._allowed`. - Repurpose `TestErrorMessages` (its old restriction-message assertions no longer exist) into per-event runtime-error coverage: JS `ReferenceError` content, VRL + JS surfacing consistency, partial-failure (only the failing event carries a message), and a syntax-error → 400 regression guard. - Update the class/module docstrings to the real contract. **`test_js_enrichment.py`** - Flip `test_js_function_in_default_org_returns_400` → `test_js_function_in_default_org_allowed` (200, transform runs). ## e2e (Playwright · `tests/ui-testing/Functions/js-transform-type.spec.js`) The JS-radio rule *is* edition-dependent. Instead of a skip (or an env-var hack), the two tests now detect edition at runtime via `EditionFeaturesPage.detectEdition()` — which reads the header edition button, driven by the **same** build flag that gates the JS radio, so it's the exact source of truth and needs **no CI change**: - **OSS build** → assert JS radio **hidden** in a non-meta org. - **Enterprise / cloud build** → assert JS radio **visible** in every org. The edition-agnostic steps that were trapped inside the skipped tests now **always run**: VRL-function create in a non-meta org (a P0 smoke path), and JS-visible in `_meta`. Added `P1: VRL runtime error surfaces in error-details` (mirrors the existing JS `ReferenceError` case; covers the VRL half of the per-event-error change). ## Skips removed (before → after) | Old (skipped) | New (runs & passes) | |---|---| | `test_create_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_test_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_update_vrl_to_js_in_default_org_blocked` | `..._default_org_allowed` | | `test_error_message_content` / `_consistency` | per-event runtime-error tests | | `test_js_function_in_default_org_returns_400` (enrichment) | `..._default_org_allowed` | | e2e `P0: Non-meta org - JS hidden, VRL works` | `P0: Non-meta org - JS radio edition-gated, VRL works` | | e2e `P1: Radio visibility across orgs` (skipped) | same name, edition-aware, no `test.skip` | `grep -rn "JS fns are now allowed" tests/` and `grep -rn "JS is allowed in all orgs on enterprise" tests/` → **both empty**. (The 7 remaining pytest skips are the unrelated `JS_HELPER_UNAVAILABLE` enrichment-helper cases — a separate pre-existing limitation, not part of this change.) ## Verification (local, against current `main` source) Built the current-`main` OSS binary and ran everything live: - **API:** `32 passed`, `0` target skips (7 unrelated `JS_HELPER_UNAVAILABLE` skips remain). - **e2e OSS build:** `7/7` — `detectEdition=opensource`, JS hidden in non-meta. - **e2e Enterprise-frontend build:** `3/3` edition tests — `detectEdition=enterprise`, JS visible everywhere, VRL runtime error renders in error-details. All backend behaviour was confirmed by probing the live `/functions` + `/functions/test` endpoints, not assumed. ## What CI proves, and where - **This PR (openobserve):** `api-testing.yml` runs the API tests; `playwright.yml` (`Functions` shard) runs the e2e on the **OSS** edition (JS-hidden branch). - **Enterprise edition:** dispatch `o2-enterprise` → `playwright.yml` (workflow_dispatch) with `o2_opensource_repo = test/functions-skipped-tests-13318` to run the same specs on the **enterprise** build (JS-visible branch). Pairs with the ENT Functions work in #13318 / o2-enterprise#2217. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
This commit is contained in:
Vamsi Karthik R
co-authored by
Claude Opus 4.8
Hengfei Yang
Claude Opus 4.8
Hengfei Yang
parent
30c80cd2ce
commit
442ca09688
@@ -157,7 +157,7 @@ def test_test_endpoint_with_valid_vrl_returns_transformed_events(client: OpenObs
|
||||
("vrl_function", "description"),
|
||||
[
|
||||
(".test=2jehwkhhe\n.", "VRL with syntax error"),
|
||||
("test", "plain text (not a VRL function)"),
|
||||
("test", "plain text (undefined VRL variable)"),
|
||||
("=====", "VRL with only special chars"),
|
||||
],
|
||||
ids=["syntax-error", "plain-text", "special-chars-only"],
|
||||
@@ -165,13 +165,67 @@ def test_test_endpoint_with_valid_vrl_returns_transformed_events(client: OpenObs
|
||||
def test_test_endpoint_with_invalid_vrl_returns_400(
|
||||
client: OpenObserveClient, vrl_function: str, description: str
|
||||
):
|
||||
"""Various invalid VRL function bodies passed to /functions/test all return 400.
|
||||
"""COMPILE-invalid VRL bodies passed to /functions/test return 400.
|
||||
|
||||
Consolidates 3 near-identical legacy tests (testinvalidfunction,
|
||||
onlytextfunction, testonlyspecialcharfunction) — each was 25-30 LOC
|
||||
of copy-paste differing only in one field.
|
||||
|
||||
Note (PR #13156): the compile-check now reports only COMPILE errors, not
|
||||
runtime errors. All three bodies below are genuine VRL *compile* errors
|
||||
when trans_type=0 (`test` and `=====` are "call to undefined variable" /
|
||||
syntax errors, not merely runtime failures), so they still 400. Runtime
|
||||
failures that compile cleanly now return 200 with a per-event message —
|
||||
see test_test_endpoint_with_*_runtime_error_returns_200 below.
|
||||
"""
|
||||
payload = {"function": vrl_function, "events": SAMPLE_EVENTS, "trans_type":0 }
|
||||
payload = {"function": vrl_function, "events": SAMPLE_EVENTS, "trans_type": 0}
|
||||
resp = client.post("functions/test", json=payload)
|
||||
assert resp.status_code == 400, \
|
||||
f"{description}: expected 400, got {resp.status_code}: {resp.text}"
|
||||
|
||||
|
||||
def test_test_endpoint_with_js_runtime_error_returns_200(client: OpenObserveClient):
|
||||
"""JS that compiles but throws at RUNTIME returns 200, with the error
|
||||
surfaced per-event in results[].message (PR #13156 behaviour change).
|
||||
|
||||
`undefinedVar` is a valid JS identifier (syntactically fine), so the
|
||||
compile-check passes; the ReferenceError only happens when the code runs
|
||||
against each event. Previously this 200-path dropped results[].message;
|
||||
#13156 wires the runtime error through per-event.
|
||||
"""
|
||||
payload = {
|
||||
"function": "row.field = undefinedVar;",
|
||||
"events": SAMPLE_EVENTS,
|
||||
"trans_type": 1,
|
||||
}
|
||||
resp = client.post("functions/test", json=payload)
|
||||
assert resp.status_code == 200, \
|
||||
f"JS runtime error should be 200, got {resp.status_code}: {resp.text}"
|
||||
body = resp.json()
|
||||
assert "results" in body, f"response missing 'results': {body}"
|
||||
assert len(body["results"]) == len(SAMPLE_EVENTS), \
|
||||
f"expected one result per event, got: {body['results']}"
|
||||
assert all("ReferenceError" in r.get("message", "") for r in body["results"]), \
|
||||
f"expected a per-event ReferenceError message, got: {body['results']}"
|
||||
|
||||
|
||||
def test_test_endpoint_with_vrl_runtime_error_returns_200(client: OpenObserveClient):
|
||||
"""VRL that compiles but aborts at RUNTIME returns 200, with the error
|
||||
surfaced per-event in results[].message (PR #13156 behaviour change).
|
||||
|
||||
`to_int!(.s)` compiles fine; it only aborts when `.s` can't be coerced to
|
||||
an int at runtime. Before #13156 VRL runtime errors were tooltip-only —
|
||||
now they render per-event, mirroring the JS path.
|
||||
"""
|
||||
payload = {
|
||||
"function": ".x = to_int!(.s)",
|
||||
"events": [{"s": "notanumber"}, {"s": "alsobad"}],
|
||||
"trans_type": 0,
|
||||
}
|
||||
resp = client.post("functions/test", json=payload)
|
||||
assert resp.status_code == 200, \
|
||||
f"VRL runtime error should be 200, got {resp.status_code}: {resp.text}"
|
||||
body = resp.json()
|
||||
assert "results" in body, f"response missing 'results': {body}"
|
||||
assert all("vrl runtime error" in r.get("message", "") for r in body["results"]), \
|
||||
f"expected a per-event VRL runtime error message, got: {body['results']}"
|
||||
|
||||
@@ -1,9 +1,19 @@
|
||||
import pytest
|
||||
"""
|
||||
API tests for JavaScript function _meta organization restriction.
|
||||
API tests for JavaScript function organization behaviour.
|
||||
|
||||
This module tests that JavaScript functions (transType=1) are only allowed
|
||||
in the _meta organization, while VRL functions (transType=0) work in all orgs.
|
||||
Backend contract (validated on OSS + ENT builds, current main):
|
||||
The functions *backend* imposes NO org restriction on JavaScript functions —
|
||||
JS (transType=1) is accepted in every org, including `default`, on BOTH OSS
|
||||
and Enterprise builds. The `_meta`-only rule that used to exist now lives ONLY
|
||||
in the frontend (AddFunction.vue gates the JS radio on
|
||||
`isEnterprise || isCloud || org === "_meta"`, baked at build time). Since API
|
||||
tests hit the backend directly, there is no edition difference to assert here.
|
||||
|
||||
Historical note (PR #13156): earlier the backend returned 400 with
|
||||
"JavaScript functions are only allowed in the '_meta' organization" for JS in
|
||||
non-_meta orgs. That restriction was removed. These tests assert the current
|
||||
"JS allowed everywhere" contract; the class name is kept for git continuity.
|
||||
|
||||
Note: The functions API uses camelCase for field names (transType) due to
|
||||
serde(rename_all = "camelCase") on the Transform struct.
|
||||
@@ -12,7 +22,7 @@ The test endpoint uses snake_case (trans_type) for TestVRLRequest struct.
|
||||
|
||||
|
||||
class TestMetaOrgJavaScriptRestriction:
|
||||
"""Test JavaScript functions restricted to _meta org only.
|
||||
"""JavaScript functions are accepted in all orgs at the backend/API layer.
|
||||
|
||||
IMPORTANT - API Field Name Format:
|
||||
- Create/Update endpoints (POST/PUT /api/{org}/functions): use `transType` (camelCase)
|
||||
@@ -42,34 +52,36 @@ class TestMetaOrgJavaScriptRestriction:
|
||||
# Cleanup
|
||||
session.delete(f"{base_url}api/{org_id}/functions/test_js_meta")
|
||||
|
||||
def test_create_js_function_in_default_org_blocked(self, create_session, base_url):
|
||||
"""JavaScript function creation should fail in default org."""
|
||||
def test_create_js_function_in_default_org_allowed(self, create_session, base_url):
|
||||
"""JavaScript function creation succeeds in default org (JS no longer _meta-only).
|
||||
|
||||
pytest.skip("JS fns are now allowed in non _meta orgs")
|
||||
return
|
||||
|
||||
Previously this returned 400 with a restriction message. The backend
|
||||
restriction was removed (PR #13156), so JS create in a non-_meta org
|
||||
now returns 200 on every edition. (Positive side of "JS allowed
|
||||
everywhere" — the negative case no longer exists at the API layer.)
|
||||
"""
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
payload = {
|
||||
"name": "test_js_default",
|
||||
"name": "test_js_default_allowed",
|
||||
"function": "row.processed = true;",
|
||||
"params": "row",
|
||||
"transType": 1, # JavaScript - should be blocked (camelCase for Transform struct)
|
||||
"transType": 1, # JavaScript (camelCase for Transform struct)
|
||||
}
|
||||
|
||||
# Cleanup any leftover from previous runs
|
||||
session.delete(f"{base_url}api/{org_id}/functions/test_js_default_allowed")
|
||||
|
||||
resp = session.post(f"{base_url}api/{org_id}/functions", json=payload)
|
||||
|
||||
assert resp.status_code == 400, (
|
||||
f"Expected 400 for JS function in default org, "
|
||||
f"but got {resp.status_code}: {resp.content}"
|
||||
assert resp.status_code == 200, (
|
||||
f"Expected 200 for JS function in default org (JS is no longer "
|
||||
f"_meta-only), but got {resp.status_code}: {resp.content}"
|
||||
)
|
||||
|
||||
# Verify error message
|
||||
response_text = resp.text
|
||||
assert "JavaScript functions are only allowed in the '_meta' organization" in response_text, (
|
||||
f"Expected restriction error message, but got: {response_text}"
|
||||
)
|
||||
# Cleanup
|
||||
session.delete(f"{base_url}api/{org_id}/functions/test_js_default_allowed")
|
||||
|
||||
def test_test_js_function_in_meta_org_success(self, create_session, base_url):
|
||||
"""Testing JavaScript function should succeed in _meta org."""
|
||||
@@ -89,46 +101,44 @@ class TestMetaOrgJavaScriptRestriction:
|
||||
f"but got {resp.status_code}: {resp.content}"
|
||||
)
|
||||
|
||||
def test_test_js_function_in_default_org_blocked(self, create_session, base_url):
|
||||
"""Testing JavaScript function should fail in default org."""
|
||||
|
||||
pytest.skip("JS fns are now allowed in non _meta orgs")
|
||||
return
|
||||
def test_test_js_function_in_default_org_allowed(self, create_session, base_url):
|
||||
"""Testing a JavaScript function succeeds in default org (JS no longer _meta-only).
|
||||
|
||||
/functions/test with a valid JS transform in a non-_meta org now
|
||||
returns 200 and the transformed event, on every edition (PR #13156).
|
||||
"""
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
payload = {
|
||||
"function": "row.count = (row.count || 0) + 1;",
|
||||
"events": [{"name": "test", "count": 5}],
|
||||
"trans_type": 1, # JavaScript - should be blocked
|
||||
"trans_type": 1, # JavaScript
|
||||
}
|
||||
|
||||
resp = session.post(f"{base_url}api/{org_id}/functions/test", json=payload)
|
||||
|
||||
assert resp.status_code == 400, (
|
||||
f"Expected 400 for JS function test in default org, "
|
||||
f"but got {resp.status_code}: {resp.content}"
|
||||
assert resp.status_code == 200, (
|
||||
f"Expected 200 for JS function test in default org (JS is no longer "
|
||||
f"_meta-only), but got {resp.status_code}: {resp.content}"
|
||||
)
|
||||
|
||||
# Verify error message
|
||||
response_text = resp.text
|
||||
assert "JavaScript functions are only allowed in the '_meta' organization" in response_text
|
||||
# The transform runs: count 5 -> 6
|
||||
body = resp.json()
|
||||
assert body["results"][0]["event"].get("count") == 6, (
|
||||
f"expected JS transform to run (count 5 -> 6), got: {body['results']}"
|
||||
)
|
||||
|
||||
def test_update_vrl_to_js_in_default_org_blocked(self, create_session, base_url):
|
||||
def test_update_vrl_to_js_in_default_org_allowed(self, create_session, base_url):
|
||||
"""
|
||||
Test that updating a VRL function to JavaScript in default org is BLOCKED.
|
||||
Updating a VRL function to JavaScript in default org succeeds (JS no longer _meta-only).
|
||||
|
||||
This ensures users cannot bypass the JS restriction by:
|
||||
1. Creating a VRL function in default org
|
||||
2. Updating it to JavaScript
|
||||
|
||||
The backend enforces JS restriction on both create AND update operations.
|
||||
Previously the backend rejected VRL->JS updates outside _meta with a
|
||||
400 restriction message. That restriction was removed (PR #13156), so:
|
||||
1. Create a VRL function in default org
|
||||
2. Update it to JavaScript
|
||||
both now return 200 on every edition.
|
||||
"""
|
||||
|
||||
pytest.skip("JS fns are now allowed in non _meta orgs")
|
||||
return
|
||||
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
@@ -146,7 +156,7 @@ class TestMetaOrgJavaScriptRestriction:
|
||||
create_resp = session.post(f"{base_url}api/{org_id}/functions", json=vrl_payload)
|
||||
assert create_resp.status_code == 200, f"Failed to create VRL function: {create_resp.content}"
|
||||
|
||||
# Try to update to JavaScript - should be BLOCKED
|
||||
# Update to JavaScript - now allowed
|
||||
js_payload = {
|
||||
"name": "test_update_to_js",
|
||||
"function": "row.processed = true;",
|
||||
@@ -159,16 +169,9 @@ class TestMetaOrgJavaScriptRestriction:
|
||||
json=js_payload
|
||||
)
|
||||
|
||||
# Should return 400 - JS restriction enforced on updates
|
||||
assert update_resp.status_code == 400, (
|
||||
f"Expected 400 for VRL->JS update in default org, "
|
||||
f"but got {update_resp.status_code}: {update_resp.content}"
|
||||
)
|
||||
|
||||
# Verify error message
|
||||
response_text = update_resp.text
|
||||
assert "JavaScript functions are only allowed in the '_meta' organization" in response_text, (
|
||||
f"Expected restriction error message, but got: {response_text}"
|
||||
assert update_resp.status_code == 200, (
|
||||
f"Expected 200 for VRL->JS update in default org (JS is no longer "
|
||||
f"_meta-only), but got {update_resp.status_code}: {update_resp.content}"
|
||||
)
|
||||
|
||||
# Cleanup
|
||||
@@ -320,67 +323,133 @@ class TestTransTypeAutoDetection:
|
||||
|
||||
|
||||
class TestErrorMessages:
|
||||
"""Test error message quality and consistency."""
|
||||
"""Runtime-error surfacing on /functions/test (PR #13156 behaviour change).
|
||||
|
||||
def test_error_message_content(self, create_session, base_url):
|
||||
"""Error message should be clear and actionable."""
|
||||
|
||||
pytest.skip("JS fns are now allowed in non _meta orgs")
|
||||
return
|
||||
The old "JS is only allowed in _meta" 400 restriction is gone (see
|
||||
TestMetaOrgJavaScriptRestriction). What matters now for error messaging is
|
||||
the per-event surfacing of RUNTIME errors: code that compiles cleanly but
|
||||
throws when run against an event returns 200, with the error attached to
|
||||
that event in results[].message (for both VRL and JS). These tests pin that
|
||||
contract. (Absorbs the former restriction-message tests, which asserted
|
||||
behaviour that no longer exists.)
|
||||
"""
|
||||
|
||||
def test_js_runtime_error_message_content(self, create_session, base_url):
|
||||
"""A JS runtime error is surfaced per-event with a clear ReferenceError."""
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
payload = {
|
||||
"name": "test_error_msg",
|
||||
"function": "row.test = true;",
|
||||
"params": "row",
|
||||
"transType": 1, # JavaScript - blocked (camelCase for Transform struct)
|
||||
}
|
||||
|
||||
resp = session.post(f"{base_url}api/{org_id}/functions", json=payload)
|
||||
|
||||
assert resp.status_code == 400
|
||||
response_text = resp.text
|
||||
|
||||
# Check error message contains key information
|
||||
assert "JavaScript" in response_text, "Error should mention JavaScript"
|
||||
assert "_meta" in response_text, "Error should mention _meta org"
|
||||
assert "organization" in response_text, "Error should mention organization"
|
||||
assert "VRL" in response_text, "Error should suggest VRL alternative"
|
||||
|
||||
def test_error_message_consistency(self, create_session, base_url):
|
||||
"""Error message should be consistent across save and test endpoints."""
|
||||
|
||||
pytest.skip("JS fns are now allowed in non _meta orgs")
|
||||
return
|
||||
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
# Test save endpoint (camelCase for Transform struct)
|
||||
save_payload = {
|
||||
"name": "test_consistency",
|
||||
"function": "row.test = true;",
|
||||
"params": "row",
|
||||
"transType": 1,
|
||||
}
|
||||
save_resp = session.post(f"{base_url}api/{org_id}/functions", json=save_payload)
|
||||
save_error = save_resp.text
|
||||
|
||||
# Test test endpoint (snake_case for TestVRLRequest struct)
|
||||
test_payload = {
|
||||
"function": "row.test = true;",
|
||||
"function": "row.field = undefinedVar;", # valid syntax, throws at runtime
|
||||
"events": [{"name": "test"}],
|
||||
"trans_type": 1,
|
||||
}
|
||||
test_resp = session.post(f"{base_url}api/{org_id}/functions/test", json=test_payload)
|
||||
test_error = test_resp.text
|
||||
resp = session.post(f"{base_url}api/{org_id}/functions/test", json=payload)
|
||||
|
||||
# Both should have same core error message
|
||||
expected_message = "JavaScript functions are only allowed in the '_meta' organization"
|
||||
assert expected_message in save_error, "Save endpoint should have restriction message"
|
||||
assert expected_message in test_error, "Test endpoint should have restriction message"
|
||||
assert resp.status_code == 200, (
|
||||
f"JS runtime error should be 200 (per-event surfacing), "
|
||||
f"but got {resp.status_code}: {resp.content}"
|
||||
)
|
||||
results = resp.json()["results"]
|
||||
assert results, f"expected per-event results, got: {results}"
|
||||
msg = results[0].get("message", "")
|
||||
assert "ReferenceError" in msg, f"expected ReferenceError in message, got: {msg!r}"
|
||||
assert "undefinedVar" in msg, f"expected the offending name in message, got: {msg!r}"
|
||||
|
||||
def test_runtime_error_surfaced_for_both_vrl_and_js(self, create_session, base_url):
|
||||
"""Runtime errors surface per-event for BOTH languages (200 + results[].message).
|
||||
|
||||
Consistency check: whether the transform is VRL or JS, a runtime
|
||||
failure that compiles cleanly returns 200 and carries the error in
|
||||
results[].message rather than a top-level 400.
|
||||
"""
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
# JS runtime error
|
||||
js_resp = session.post(
|
||||
f"{base_url}api/{org_id}/functions/test",
|
||||
json={
|
||||
"function": "row.field = undefinedVar;",
|
||||
"events": [{"name": "test"}],
|
||||
"trans_type": 1,
|
||||
},
|
||||
)
|
||||
assert js_resp.status_code == 200, f"JS: {js_resp.status_code} {js_resp.content}"
|
||||
js_msg = js_resp.json()["results"][0].get("message", "")
|
||||
assert js_msg, f"JS runtime error should carry a per-event message, got: {js_msg!r}"
|
||||
|
||||
# VRL runtime error (to_int! aborts when the value can't be coerced)
|
||||
vrl_resp = session.post(
|
||||
f"{base_url}api/{org_id}/functions/test",
|
||||
json={
|
||||
"function": ".x = to_int!(.s)",
|
||||
"events": [{"s": "notanumber"}],
|
||||
"trans_type": 0,
|
||||
},
|
||||
)
|
||||
assert vrl_resp.status_code == 200, f"VRL: {vrl_resp.status_code} {vrl_resp.content}"
|
||||
vrl_msg = vrl_resp.json()["results"][0].get("message", "")
|
||||
assert "vrl runtime error" in vrl_msg, (
|
||||
f"VRL runtime error should carry a per-event message, got: {vrl_msg!r}"
|
||||
)
|
||||
|
||||
def test_partial_failure_flags_only_failing_events(self, create_session, base_url):
|
||||
"""Per-event semantics: in a mixed batch, only the failing event carries a message.
|
||||
|
||||
The JS function throws only when row.bad is truthy, so event 1 fails
|
||||
(ReferenceError) and event 2 transforms cleanly. Response is 200 with
|
||||
one message-bearing result and one clean result.
|
||||
"""
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
payload = {
|
||||
"function": "if (row.bad) { row.x = undefinedVar; } else { row.x = 1; }",
|
||||
"events": [{"bad": True}, {"bad": False}],
|
||||
"trans_type": 1,
|
||||
}
|
||||
resp = session.post(f"{base_url}api/{org_id}/functions/test", json=payload)
|
||||
|
||||
assert resp.status_code == 200, f"{resp.status_code}: {resp.content}"
|
||||
results = resp.json()["results"]
|
||||
assert len(results) == 2, f"expected one result per event, got: {results}"
|
||||
assert "ReferenceError" in results[0].get("message", ""), (
|
||||
f"failing event should carry a runtime error, got: {results[0]}"
|
||||
)
|
||||
assert not results[1].get("message"), (
|
||||
f"clean event should have no error message, got: {results[1]}"
|
||||
)
|
||||
assert results[1]["event"].get("x") == 1, (
|
||||
f"clean event should be transformed (x=1), got: {results[1]}"
|
||||
)
|
||||
|
||||
def test_syntax_error_still_returns_400(self, create_session, base_url):
|
||||
"""Regression guard: a genuine SYNTAX error still 400s (not the 200 runtime path).
|
||||
|
||||
Pins the line between "syntax error -> 400" and "runtime error -> 200"
|
||||
so a future change can't blur it. Covers both languages.
|
||||
"""
|
||||
session = create_session
|
||||
org_id = "default"
|
||||
|
||||
# JS syntax error
|
||||
js_resp = session.post(
|
||||
f"{base_url}api/{org_id}/functions/test",
|
||||
json={"function": "function broken {", "events": [{"a": 1}], "trans_type": 1},
|
||||
)
|
||||
assert js_resp.status_code == 400, (
|
||||
f"JS syntax error should be 400, got {js_resp.status_code}: {js_resp.content}"
|
||||
)
|
||||
|
||||
# VRL syntax error
|
||||
vrl_resp = session.post(
|
||||
f"{base_url}api/{org_id}/functions/test",
|
||||
json={"function": "=====", "events": [{"a": 1}], "trans_type": 0},
|
||||
)
|
||||
assert vrl_resp.status_code == 400, (
|
||||
f"VRL syntax error should be 400, got {vrl_resp.status_code}: {vrl_resp.content}"
|
||||
)
|
||||
|
||||
|
||||
class TestEdgeCases:
|
||||
|
||||
@@ -234,27 +234,24 @@ def test_js_enrichment_multiple_lookups(client: OpenObserveClient):
|
||||
# ----- documented org-restriction (always runs) -----
|
||||
|
||||
|
||||
def test_js_function_in_default_org_returns_400(client: OpenObserveClient):
|
||||
"""OO restricts JS functions to the `_meta` org. Documents this constraint.
|
||||
def test_js_function_in_default_org_allowed(client: OpenObserveClient):
|
||||
"""JS functions run in the `default` (non-_meta) org. Documents the current contract.
|
||||
|
||||
The original suite missed this — it POSTed JS to `default` and silently
|
||||
failed. This test makes the constraint a first-class contract: if/when
|
||||
the backend opens JS to other orgs, this test starts failing and you
|
||||
know to update.
|
||||
The backend used to restrict JS to `_meta` (400 elsewhere). That
|
||||
restriction was removed (PR #13156): JS is now accepted in every org on
|
||||
every edition. This test pins the positive contract — a valid JS transform
|
||||
in `default` returns 200 and transforms the event. If the backend ever
|
||||
re-introduces an org restriction, this starts failing and you know to update.
|
||||
"""
|
||||
|
||||
pytest.skip("JS fns are now allowed in non _meta orgs")
|
||||
return
|
||||
|
||||
payload = {
|
||||
"function": "row.x = 'js works here';",
|
||||
"events": [{"id": 1}],
|
||||
"trans_type": 1,
|
||||
}
|
||||
resp = client.post("functions/test", json=payload, org=DEFAULT_ORG)
|
||||
assert resp.status_code == 400, \
|
||||
f"JS in non-_meta org should be 400, got {resp.status_code}: {resp.text}"
|
||||
assert resp.status_code == 200, \
|
||||
f"JS in non-_meta org should now be 200, got {resp.status_code}: {resp.text}"
|
||||
|
||||
body = resp.json()
|
||||
assert "JavaScript" in body.get("message", "") or "_meta" in body.get("message", ""), \
|
||||
f"error message should mention JS restriction, got: {body}"
|
||||
assert body["results"][0]["event"].get("x") == "js works here", \
|
||||
f"JS transform should have run, got: {body['results']}"
|
||||
|
||||
@@ -218,7 +218,7 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio
|
||||
// - test_create_js_function_in_meta_org_success
|
||||
});
|
||||
|
||||
test.describe('Non-Meta Organization - JavaScript Blocked', () => {
|
||||
test.describe('Non-Meta Organization - JavaScript edition-gated', () => {
|
||||
test.beforeEach(async ({ page }) => {
|
||||
// Switch to a non-meta organization (self-hosted: 'default', cloud: user's own org)
|
||||
const nonMetaOrg = getNonMetaOrg();
|
||||
@@ -235,33 +235,36 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio
|
||||
|
||||
// NOTE: Consolidated from meta-org "JS radio NOT visible" + "VRL functions work" tests
|
||||
// Both validations preserved: JS hidden check + VRL creation
|
||||
test('P0: Non-meta org - JS hidden, VRL works', { tag: ['@smoke', '@P0'] }, async ({ page }) => {
|
||||
// SKIPPED — asserts the OSS-only rule, but runs on the enterprise build.
|
||||
// JS is no longer _meta-only: AddFunction.vue offers it whenever
|
||||
// `isEnterprise || isCloud || org === "_meta"`, so on ENT/cloud the JS radio
|
||||
// is visible in EVERY org and step 1 below fails. The rule still holds on OSS.
|
||||
// Re-enabling needs an edition signal, which is awkward: Vite bakes
|
||||
// VITE_OPENOBSERVE_ENTERPRISE into the bundle at build time and /config
|
||||
// exposes no edition key, so it can't be detected at runtime.
|
||||
// TODO(QA): move to the OSS suite, or branch on an edition env var.
|
||||
// NOTE: step 2 (VRL function creation in a non-meta org) is edition-agnostic
|
||||
// and is lost while this is skipped.
|
||||
test.skip(true, 'JS is allowed in all orgs on enterprise/cloud builds; OSS-only assertion');
|
||||
|
||||
test('P0: Non-meta org - JS radio edition-gated, VRL works', { tag: ['@smoke', '@P0'] }, async ({ page }) => {
|
||||
// JS radio visibility in a non-meta org is EDITION-dependent (see file header):
|
||||
// OSS -> JS is _meta-only, so the radio is HIDDEN here.
|
||||
// ENT/cloud-> JS is offered in every org, so the radio is VISIBLE here.
|
||||
// We detect the edition at runtime from the header edition button label
|
||||
// (data-test="upgrade-to-enterprise-btn"), which is driven by the SAME
|
||||
// build-time flags (config.isEnterprise/isCloud via @/aws-exports) that gate
|
||||
// the JS radio — so it's the exact source of truth, no CI env var needed.
|
||||
// Step 2 (VRL create in a non-meta org) is edition-agnostic and always runs.
|
||||
const nonMetaOrg = getNonMetaOrg();
|
||||
testLogger.info(`Test: Non-meta org (${nonMetaOrg}) - JS hidden, VRL works`);
|
||||
const edition = await pm.editionFeaturesPage.detectEdition();
|
||||
testLogger.info(`Test: Non-meta org (${nonMetaOrg}) - edition=${edition}`);
|
||||
|
||||
// Step 1: Verify JS radio is hidden, VRL is visible
|
||||
// Step 1: VRL is always visible; JS radio visibility depends on edition.
|
||||
await pm.functionsPage.clickAddFunctionButton(nonMetaOrg);
|
||||
await pm.functionsPage.expectVrlRadioVisible();
|
||||
testLogger.info('VRL radio button visible');
|
||||
|
||||
const isJsRadioVisible = await pm.functionsPage.isJsRadioVisible();
|
||||
expect(isJsRadioVisible).toBe(false);
|
||||
testLogger.info(`JavaScript radio button correctly hidden in ${nonMetaOrg} org`);
|
||||
if (edition === 'opensource') {
|
||||
const isJsRadioVisible = await pm.functionsPage.isJsRadioVisible();
|
||||
expect(isJsRadioVisible).toBe(false);
|
||||
testLogger.info(`OSS: JavaScript radio correctly hidden in ${nonMetaOrg} org`);
|
||||
} else {
|
||||
// enterprise / cloud
|
||||
await pm.functionsPage.expectJsRadioVisible();
|
||||
testLogger.info(`${edition}: JavaScript radio correctly visible in ${nonMetaOrg} org`);
|
||||
}
|
||||
await pm.functionsPage.clickCancelButton();
|
||||
|
||||
// Step 2: Create VRL function to verify it works
|
||||
// Step 2 (edition-agnostic): Create a VRL function to verify it works.
|
||||
const functionName = `nonmeta_vrl_fn_${Date.now()}`;
|
||||
const vrlCode = '.processed = true\n.org = "non_meta"';
|
||||
|
||||
@@ -283,26 +286,62 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio
|
||||
await pm.functionsPage.deleteFunctionByName(functionName, nonMetaOrg);
|
||||
});
|
||||
|
||||
// NOTE: API tests for JavaScript function blocking in default org have been migrated to:
|
||||
// tests/api-testing/tests/test_functions_meta_org_restriction.py
|
||||
// - test_create_js_function_in_default_org_blocked
|
||||
// - test_test_js_function_in_default_org_blocked
|
||||
// Edition-agnostic: a VRL runtime error surfaces in the error-details section
|
||||
// (mirrors the JS ReferenceError case in the _meta describe). VRL works in every
|
||||
// org on every edition, so this runs unconditionally. Covers the VRL half of the
|
||||
// "runtime errors now render per-event" behaviour change (PR #13156).
|
||||
test('P1: VRL runtime error surfaces in error-details', { tag: ['@P1', '@errorHandling'] }, async ({ page }) => {
|
||||
const nonMetaOrg = getNonMetaOrg();
|
||||
testLogger.info(`Test: VRL runtime error in error-details (${nonMetaOrg} org)`);
|
||||
|
||||
// to_int!(.s) compiles fine but aborts at runtime when .s can't be coerced.
|
||||
const functionName = `nonmeta_vrl_err_${Date.now()}`;
|
||||
const vrlCode = '.x = to_int!(.s)';
|
||||
|
||||
await pm.functionsPage.clickAddFunctionButton(nonMetaOrg);
|
||||
await pm.functionsPage.fillFunctionName(functionName);
|
||||
// VRL is the default type
|
||||
await pm.functionsPage.enterFunctionCode(vrlCode);
|
||||
|
||||
// Run against an event that triggers the runtime abort.
|
||||
await pm.functionsPage.testFunctionExecution('[{"s":"notanumber"}]');
|
||||
|
||||
// The per-event VRL runtime error renders in the error-details section
|
||||
// (200 response with results[].message, not a 400).
|
||||
await pm.functionsPage.expectFunctionErrorContains('runtime error');
|
||||
testLogger.info('VRL runtime error rendered in error-details section');
|
||||
await pm.functionsPage.clickCancelButton();
|
||||
});
|
||||
|
||||
// NOTE: API tests for JavaScript function behaviour in default org live in:
|
||||
// tests/api-testing/tests/pipelines/test_functions_meta_org_restriction.py
|
||||
// - test_create_js_function_in_default_org_allowed
|
||||
// - test_test_js_function_in_default_org_allowed
|
||||
});
|
||||
|
||||
test.describe('Organization Switching & VRL Control', () => {
|
||||
// NOTE: Consolidated from meta-org "Org switching" + "VRL always visible" tests
|
||||
// Both validations preserved: JS visibility changes with org + VRL always visible
|
||||
test('P1: Radio visibility across orgs', { tag: ['@P1', '@orgSwitching', '@control'] }, async ({ page }) => {
|
||||
// SKIPPED — same reason as 'P0: Non-meta org - JS hidden, VRL works' above:
|
||||
// steps 1 and 3 expect the JS radio hidden in a non-meta org, which is only
|
||||
// true on OSS. TODO(QA): move to the OSS suite, or branch on an edition env var.
|
||||
// NOTE: step 2 (JS visible in _meta) is edition-agnostic and is lost meanwhile.
|
||||
test.skip(true, 'JS is allowed in all orgs on enterprise/cloud builds; OSS-only assertion');
|
||||
|
||||
// JS radio visibility in a NON-meta org is edition-gated (OSS hides it,
|
||||
// ENT/cloud shows it); in the _meta org JS is ALWAYS visible on every
|
||||
// edition. VRL is always visible everywhere. We detect the edition from
|
||||
// the header edition button (same build-flag source as the JS gate).
|
||||
const nonMetaOrg = getNonMetaOrg();
|
||||
testLogger.info(`Test: Radio visibility across orgs (non-meta: ${nonMetaOrg})`);
|
||||
const edition = await pm.editionFeaturesPage.detectEdition();
|
||||
testLogger.info(`Test: Radio visibility across orgs (non-meta: ${nonMetaOrg}, edition=${edition})`);
|
||||
|
||||
// Step 1: Non-meta org - JS hidden, VRL visible
|
||||
const expectJsRadioForNonMeta = async (label) => {
|
||||
if (edition === 'opensource') {
|
||||
await pm.functionsPage.expectJsRadioHidden();
|
||||
testLogger.info(`OSS: JS hidden in ${nonMetaOrg} org (${label})`);
|
||||
} else {
|
||||
await pm.functionsPage.expectJsRadioVisible();
|
||||
testLogger.info(`${edition}: JS visible in ${nonMetaOrg} org (${label})`);
|
||||
}
|
||||
};
|
||||
|
||||
// Step 1: Non-meta org - VRL visible, JS edition-gated
|
||||
await page.goto(`${process.env.ZO_BASE_URL}/web/pipeline/functions?org_identifier=${nonMetaOrg}`);
|
||||
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
|
||||
await page.waitForTimeout(2000);
|
||||
@@ -310,11 +349,10 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio
|
||||
await pm.functionsPage.clickAddFunctionButton(nonMetaOrg);
|
||||
await pm.functionsPage.expectVrlRadioVisible();
|
||||
testLogger.info(`VRL visible in ${nonMetaOrg} org`);
|
||||
await pm.functionsPage.expectJsRadioHidden();
|
||||
testLogger.info(`JS hidden in ${nonMetaOrg} org`);
|
||||
await expectJsRadioForNonMeta('step 1');
|
||||
await pm.functionsPage.clickCancelButton();
|
||||
|
||||
// Step 2: Switch to _meta org - JS visible, VRL visible
|
||||
// Step 2 (edition-agnostic): Switch to _meta org - JS visible, VRL visible
|
||||
await page.goto(`${process.env.ZO_BASE_URL}/web/pipeline/functions?org_identifier=_meta`);
|
||||
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
|
||||
await page.waitForTimeout(2000);
|
||||
@@ -323,10 +361,10 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio
|
||||
await pm.functionsPage.expectVrlRadioVisible();
|
||||
testLogger.info('VRL visible in _meta org');
|
||||
await pm.functionsPage.expectJsRadioVisible();
|
||||
testLogger.info('JS visible in _meta org');
|
||||
testLogger.info('JS visible in _meta org (always, every edition)');
|
||||
await pm.functionsPage.clickCancelButton();
|
||||
|
||||
// Step 3: Switch back to non-meta - JS hidden again, VRL still visible
|
||||
// Step 3: Switch back to non-meta - JS gated again, VRL still visible
|
||||
await page.goto(`${process.env.ZO_BASE_URL}/web/pipeline/functions?org_identifier=${nonMetaOrg}`);
|
||||
await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {});
|
||||
await page.waitForTimeout(2000);
|
||||
@@ -334,8 +372,7 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio
|
||||
await pm.functionsPage.clickAddFunctionButton(nonMetaOrg);
|
||||
await pm.functionsPage.expectVrlRadioVisible();
|
||||
testLogger.info('VRL still visible after switching back');
|
||||
await pm.functionsPage.expectJsRadioHidden();
|
||||
testLogger.info(`JS hidden after switching back to ${nonMetaOrg}`);
|
||||
await expectJsRadioForNonMeta('step 3');
|
||||
await pm.functionsPage.clickCancelButton();
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user