From 442ca096887a3eadcc718baba41c1b550875fa67 Mon Sep 17 00:00:00 2001 From: Vamsi Karthik R Date: Mon, 27 Jul 2026 07:15:28 +0530 Subject: [PATCH] test(functions): un-skip JS-in-all-orgs tests + add per-event runtime-error coverage (#13341) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the `skip` band-aids that PR #13156 ("workflows v1") left on the **Functions** tests with real, running coverage of the current behaviour — on the correct edition — plus new tests for behaviour #13156 introduced but never covered. Test files only, no feature-code churn. Clean branch off `main`. ## Background — what #13156 actually changed #13156 bundled a Functions backend change unrelated to Workflows, and the old tests still asserted the *pre*-#13156 rules, so they were silenced with `pytest.skip(...)` / `test.skip(...)`. Two rules changed: 1. **JS is no longer `_meta`-only at the backend.** `save_function` / `test_run_function` have **no org gate on any edition** — JS (`transType:1`) is accepted in every org, including `default`, on **both OSS and Enterprise**. The `_meta`-only rule now lives **only in the frontend** (`AddFunction.vue` gates the JS radio on `isEnterprise || isCloud || org === "_meta"`, baked at build time by `VITE_OPENOBSERVE_ENTERPRISE`). 2. **The compile-check reports only *syntax* errors, not *runtime* errors.** A syntax error still returns **400**; code that compiles but throws at runtime now returns **200** with the error surfaced **per-event** in `results[].message` — for **both VRL and JS**. Because the API tests hit the backend directly, there is **no edition difference to assert for them** — the earlier "OSS asserts 400 / ENT asserts 200" framing was incorrect. Only the **e2e** (frontend JS-radio) is genuinely edition-dependent. ## API tests (pytest · `tests/api-testing`) **`test_functions.py`** - Keep the compile-error → 400 test (`test_test_endpoint_with_invalid_vrl_returns_400`) — it was never actually failing; `test`/`=====` are genuine VRL *compile* errors (`undefined variable` / syntax) and still 400. - Add `test_test_endpoint_with_js_runtime_error_returns_200` and `test_test_endpoint_with_vrl_runtime_error_returns_200` — runtime errors now return 200 with a per-event `results[].message`. **`test_functions_meta_org_restriction.py`** - Flip the 3 previously-skipped "blocked in default" tests → assert **200** (JS allowed in every org): `..._blocked` → `..._allowed`. - Repurpose `TestErrorMessages` (its old restriction-message assertions no longer exist) into per-event runtime-error coverage: JS `ReferenceError` content, VRL + JS surfacing consistency, partial-failure (only the failing event carries a message), and a syntax-error → 400 regression guard. - Update the class/module docstrings to the real contract. **`test_js_enrichment.py`** - Flip `test_js_function_in_default_org_returns_400` → `test_js_function_in_default_org_allowed` (200, transform runs). ## e2e (Playwright · `tests/ui-testing/Functions/js-transform-type.spec.js`) The JS-radio rule *is* edition-dependent. Instead of a skip (or an env-var hack), the two tests now detect edition at runtime via `EditionFeaturesPage.detectEdition()` — which reads the header edition button, driven by the **same** build flag that gates the JS radio, so it's the exact source of truth and needs **no CI change**: - **OSS build** → assert JS radio **hidden** in a non-meta org. - **Enterprise / cloud build** → assert JS radio **visible** in every org. The edition-agnostic steps that were trapped inside the skipped tests now **always run**: VRL-function create in a non-meta org (a P0 smoke path), and JS-visible in `_meta`. Added `P1: VRL runtime error surfaces in error-details` (mirrors the existing JS `ReferenceError` case; covers the VRL half of the per-event-error change). ## Skips removed (before → after) | Old (skipped) | New (runs & passes) | |---|---| | `test_create_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_test_js_function_in_default_org_blocked` | `..._default_org_allowed` | | `test_update_vrl_to_js_in_default_org_blocked` | `..._default_org_allowed` | | `test_error_message_content` / `_consistency` | per-event runtime-error tests | | `test_js_function_in_default_org_returns_400` (enrichment) | `..._default_org_allowed` | | e2e `P0: Non-meta org - JS hidden, VRL works` | `P0: Non-meta org - JS radio edition-gated, VRL works` | | e2e `P1: Radio visibility across orgs` (skipped) | same name, edition-aware, no `test.skip` | `grep -rn "JS fns are now allowed" tests/` and `grep -rn "JS is allowed in all orgs on enterprise" tests/` → **both empty**. (The 7 remaining pytest skips are the unrelated `JS_HELPER_UNAVAILABLE` enrichment-helper cases — a separate pre-existing limitation, not part of this change.) ## Verification (local, against current `main` source) Built the current-`main` OSS binary and ran everything live: - **API:** `32 passed`, `0` target skips (7 unrelated `JS_HELPER_UNAVAILABLE` skips remain). - **e2e OSS build:** `7/7` — `detectEdition=opensource`, JS hidden in non-meta. - **e2e Enterprise-frontend build:** `3/3` edition tests — `detectEdition=enterprise`, JS visible everywhere, VRL runtime error renders in error-details. All backend behaviour was confirmed by probing the live `/functions` + `/functions/test` endpoints, not assumed. ## What CI proves, and where - **This PR (openobserve):** `api-testing.yml` runs the API tests; `playwright.yml` (`Functions` shard) runs the e2e on the **OSS** edition (JS-hidden branch). - **Enterprise edition:** dispatch `o2-enterprise` → `playwright.yml` (workflow_dispatch) with `o2_opensource_repo = test/functions-skipped-tests-13318` to run the same specs on the **enterprise** build (JS-visible branch). Pairs with the ENT Functions work in #13318 / o2-enterprise#2217. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: Hengfei Yang --- .../tests/pipelines/test_functions.py | 60 +++- .../test_functions_meta_org_restriction.py | 277 +++++++++++------- .../tests/pipelines/test_js_enrichment.py | 25 +- .../Functions/js-transform-type.spec.js | 115 +++++--- 4 files changed, 317 insertions(+), 160 deletions(-) diff --git a/tests/api-testing/tests/pipelines/test_functions.py b/tests/api-testing/tests/pipelines/test_functions.py index 1b50b5ba2d..219f4b6c0f 100644 --- a/tests/api-testing/tests/pipelines/test_functions.py +++ b/tests/api-testing/tests/pipelines/test_functions.py @@ -157,7 +157,7 @@ def test_test_endpoint_with_valid_vrl_returns_transformed_events(client: OpenObs ("vrl_function", "description"), [ (".test=2jehwkhhe\n.", "VRL with syntax error"), - ("test", "plain text (not a VRL function)"), + ("test", "plain text (undefined VRL variable)"), ("=====", "VRL with only special chars"), ], ids=["syntax-error", "plain-text", "special-chars-only"], @@ -165,13 +165,67 @@ def test_test_endpoint_with_valid_vrl_returns_transformed_events(client: OpenObs def test_test_endpoint_with_invalid_vrl_returns_400( client: OpenObserveClient, vrl_function: str, description: str ): - """Various invalid VRL function bodies passed to /functions/test all return 400. + """COMPILE-invalid VRL bodies passed to /functions/test return 400. Consolidates 3 near-identical legacy tests (testinvalidfunction, onlytextfunction, testonlyspecialcharfunction) — each was 25-30 LOC of copy-paste differing only in one field. + + Note (PR #13156): the compile-check now reports only COMPILE errors, not + runtime errors. All three bodies below are genuine VRL *compile* errors + when trans_type=0 (`test` and `=====` are "call to undefined variable" / + syntax errors, not merely runtime failures), so they still 400. Runtime + failures that compile cleanly now return 200 with a per-event message — + see test_test_endpoint_with_*_runtime_error_returns_200 below. """ - payload = {"function": vrl_function, "events": SAMPLE_EVENTS, "trans_type":0 } + payload = {"function": vrl_function, "events": SAMPLE_EVENTS, "trans_type": 0} resp = client.post("functions/test", json=payload) assert resp.status_code == 400, \ f"{description}: expected 400, got {resp.status_code}: {resp.text}" + + +def test_test_endpoint_with_js_runtime_error_returns_200(client: OpenObserveClient): + """JS that compiles but throws at RUNTIME returns 200, with the error + surfaced per-event in results[].message (PR #13156 behaviour change). + + `undefinedVar` is a valid JS identifier (syntactically fine), so the + compile-check passes; the ReferenceError only happens when the code runs + against each event. Previously this 200-path dropped results[].message; + #13156 wires the runtime error through per-event. + """ + payload = { + "function": "row.field = undefinedVar;", + "events": SAMPLE_EVENTS, + "trans_type": 1, + } + resp = client.post("functions/test", json=payload) + assert resp.status_code == 200, \ + f"JS runtime error should be 200, got {resp.status_code}: {resp.text}" + body = resp.json() + assert "results" in body, f"response missing 'results': {body}" + assert len(body["results"]) == len(SAMPLE_EVENTS), \ + f"expected one result per event, got: {body['results']}" + assert all("ReferenceError" in r.get("message", "") for r in body["results"]), \ + f"expected a per-event ReferenceError message, got: {body['results']}" + + +def test_test_endpoint_with_vrl_runtime_error_returns_200(client: OpenObserveClient): + """VRL that compiles but aborts at RUNTIME returns 200, with the error + surfaced per-event in results[].message (PR #13156 behaviour change). + + `to_int!(.s)` compiles fine; it only aborts when `.s` can't be coerced to + an int at runtime. Before #13156 VRL runtime errors were tooltip-only — + now they render per-event, mirroring the JS path. + """ + payload = { + "function": ".x = to_int!(.s)", + "events": [{"s": "notanumber"}, {"s": "alsobad"}], + "trans_type": 0, + } + resp = client.post("functions/test", json=payload) + assert resp.status_code == 200, \ + f"VRL runtime error should be 200, got {resp.status_code}: {resp.text}" + body = resp.json() + assert "results" in body, f"response missing 'results': {body}" + assert all("vrl runtime error" in r.get("message", "") for r in body["results"]), \ + f"expected a per-event VRL runtime error message, got: {body['results']}" diff --git a/tests/api-testing/tests/pipelines/test_functions_meta_org_restriction.py b/tests/api-testing/tests/pipelines/test_functions_meta_org_restriction.py index 11f530eb16..5a4b09af52 100644 --- a/tests/api-testing/tests/pipelines/test_functions_meta_org_restriction.py +++ b/tests/api-testing/tests/pipelines/test_functions_meta_org_restriction.py @@ -1,9 +1,19 @@ import pytest """ -API tests for JavaScript function _meta organization restriction. +API tests for JavaScript function organization behaviour. -This module tests that JavaScript functions (transType=1) are only allowed -in the _meta organization, while VRL functions (transType=0) work in all orgs. +Backend contract (validated on OSS + ENT builds, current main): +The functions *backend* imposes NO org restriction on JavaScript functions — +JS (transType=1) is accepted in every org, including `default`, on BOTH OSS +and Enterprise builds. The `_meta`-only rule that used to exist now lives ONLY +in the frontend (AddFunction.vue gates the JS radio on +`isEnterprise || isCloud || org === "_meta"`, baked at build time). Since API +tests hit the backend directly, there is no edition difference to assert here. + +Historical note (PR #13156): earlier the backend returned 400 with +"JavaScript functions are only allowed in the '_meta' organization" for JS in +non-_meta orgs. That restriction was removed. These tests assert the current +"JS allowed everywhere" contract; the class name is kept for git continuity. Note: The functions API uses camelCase for field names (transType) due to serde(rename_all = "camelCase") on the Transform struct. @@ -12,7 +22,7 @@ The test endpoint uses snake_case (trans_type) for TestVRLRequest struct. class TestMetaOrgJavaScriptRestriction: - """Test JavaScript functions restricted to _meta org only. + """JavaScript functions are accepted in all orgs at the backend/API layer. IMPORTANT - API Field Name Format: - Create/Update endpoints (POST/PUT /api/{org}/functions): use `transType` (camelCase) @@ -42,34 +52,36 @@ class TestMetaOrgJavaScriptRestriction: # Cleanup session.delete(f"{base_url}api/{org_id}/functions/test_js_meta") - def test_create_js_function_in_default_org_blocked(self, create_session, base_url): - """JavaScript function creation should fail in default org.""" + def test_create_js_function_in_default_org_allowed(self, create_session, base_url): + """JavaScript function creation succeeds in default org (JS no longer _meta-only). - pytest.skip("JS fns are now allowed in non _meta orgs") - return - + Previously this returned 400 with a restriction message. The backend + restriction was removed (PR #13156), so JS create in a non-_meta org + now returns 200 on every edition. (Positive side of "JS allowed + everywhere" — the negative case no longer exists at the API layer.) + """ session = create_session org_id = "default" payload = { - "name": "test_js_default", + "name": "test_js_default_allowed", "function": "row.processed = true;", "params": "row", - "transType": 1, # JavaScript - should be blocked (camelCase for Transform struct) + "transType": 1, # JavaScript (camelCase for Transform struct) } + # Cleanup any leftover from previous runs + session.delete(f"{base_url}api/{org_id}/functions/test_js_default_allowed") + resp = session.post(f"{base_url}api/{org_id}/functions", json=payload) - assert resp.status_code == 400, ( - f"Expected 400 for JS function in default org, " - f"but got {resp.status_code}: {resp.content}" + assert resp.status_code == 200, ( + f"Expected 200 for JS function in default org (JS is no longer " + f"_meta-only), but got {resp.status_code}: {resp.content}" ) - # Verify error message - response_text = resp.text - assert "JavaScript functions are only allowed in the '_meta' organization" in response_text, ( - f"Expected restriction error message, but got: {response_text}" - ) + # Cleanup + session.delete(f"{base_url}api/{org_id}/functions/test_js_default_allowed") def test_test_js_function_in_meta_org_success(self, create_session, base_url): """Testing JavaScript function should succeed in _meta org.""" @@ -89,46 +101,44 @@ class TestMetaOrgJavaScriptRestriction: f"but got {resp.status_code}: {resp.content}" ) - def test_test_js_function_in_default_org_blocked(self, create_session, base_url): - """Testing JavaScript function should fail in default org.""" - - pytest.skip("JS fns are now allowed in non _meta orgs") - return + def test_test_js_function_in_default_org_allowed(self, create_session, base_url): + """Testing a JavaScript function succeeds in default org (JS no longer _meta-only). + /functions/test with a valid JS transform in a non-_meta org now + returns 200 and the transformed event, on every edition (PR #13156). + """ session = create_session org_id = "default" payload = { "function": "row.count = (row.count || 0) + 1;", "events": [{"name": "test", "count": 5}], - "trans_type": 1, # JavaScript - should be blocked + "trans_type": 1, # JavaScript } resp = session.post(f"{base_url}api/{org_id}/functions/test", json=payload) - assert resp.status_code == 400, ( - f"Expected 400 for JS function test in default org, " - f"but got {resp.status_code}: {resp.content}" + assert resp.status_code == 200, ( + f"Expected 200 for JS function test in default org (JS is no longer " + f"_meta-only), but got {resp.status_code}: {resp.content}" ) - # Verify error message - response_text = resp.text - assert "JavaScript functions are only allowed in the '_meta' organization" in response_text + # The transform runs: count 5 -> 6 + body = resp.json() + assert body["results"][0]["event"].get("count") == 6, ( + f"expected JS transform to run (count 5 -> 6), got: {body['results']}" + ) - def test_update_vrl_to_js_in_default_org_blocked(self, create_session, base_url): + def test_update_vrl_to_js_in_default_org_allowed(self, create_session, base_url): """ - Test that updating a VRL function to JavaScript in default org is BLOCKED. + Updating a VRL function to JavaScript in default org succeeds (JS no longer _meta-only). - This ensures users cannot bypass the JS restriction by: - 1. Creating a VRL function in default org - 2. Updating it to JavaScript - - The backend enforces JS restriction on both create AND update operations. + Previously the backend rejected VRL->JS updates outside _meta with a + 400 restriction message. That restriction was removed (PR #13156), so: + 1. Create a VRL function in default org + 2. Update it to JavaScript + both now return 200 on every edition. """ - - pytest.skip("JS fns are now allowed in non _meta orgs") - return - session = create_session org_id = "default" @@ -146,7 +156,7 @@ class TestMetaOrgJavaScriptRestriction: create_resp = session.post(f"{base_url}api/{org_id}/functions", json=vrl_payload) assert create_resp.status_code == 200, f"Failed to create VRL function: {create_resp.content}" - # Try to update to JavaScript - should be BLOCKED + # Update to JavaScript - now allowed js_payload = { "name": "test_update_to_js", "function": "row.processed = true;", @@ -159,16 +169,9 @@ class TestMetaOrgJavaScriptRestriction: json=js_payload ) - # Should return 400 - JS restriction enforced on updates - assert update_resp.status_code == 400, ( - f"Expected 400 for VRL->JS update in default org, " - f"but got {update_resp.status_code}: {update_resp.content}" - ) - - # Verify error message - response_text = update_resp.text - assert "JavaScript functions are only allowed in the '_meta' organization" in response_text, ( - f"Expected restriction error message, but got: {response_text}" + assert update_resp.status_code == 200, ( + f"Expected 200 for VRL->JS update in default org (JS is no longer " + f"_meta-only), but got {update_resp.status_code}: {update_resp.content}" ) # Cleanup @@ -320,67 +323,133 @@ class TestTransTypeAutoDetection: class TestErrorMessages: - """Test error message quality and consistency.""" + """Runtime-error surfacing on /functions/test (PR #13156 behaviour change). - def test_error_message_content(self, create_session, base_url): - """Error message should be clear and actionable.""" - - pytest.skip("JS fns are now allowed in non _meta orgs") - return + The old "JS is only allowed in _meta" 400 restriction is gone (see + TestMetaOrgJavaScriptRestriction). What matters now for error messaging is + the per-event surfacing of RUNTIME errors: code that compiles cleanly but + throws when run against an event returns 200, with the error attached to + that event in results[].message (for both VRL and JS). These tests pin that + contract. (Absorbs the former restriction-message tests, which asserted + behaviour that no longer exists.) + """ + def test_js_runtime_error_message_content(self, create_session, base_url): + """A JS runtime error is surfaced per-event with a clear ReferenceError.""" session = create_session org_id = "default" payload = { - "name": "test_error_msg", - "function": "row.test = true;", - "params": "row", - "transType": 1, # JavaScript - blocked (camelCase for Transform struct) - } - - resp = session.post(f"{base_url}api/{org_id}/functions", json=payload) - - assert resp.status_code == 400 - response_text = resp.text - - # Check error message contains key information - assert "JavaScript" in response_text, "Error should mention JavaScript" - assert "_meta" in response_text, "Error should mention _meta org" - assert "organization" in response_text, "Error should mention organization" - assert "VRL" in response_text, "Error should suggest VRL alternative" - - def test_error_message_consistency(self, create_session, base_url): - """Error message should be consistent across save and test endpoints.""" - - pytest.skip("JS fns are now allowed in non _meta orgs") - return - - session = create_session - org_id = "default" - - # Test save endpoint (camelCase for Transform struct) - save_payload = { - "name": "test_consistency", - "function": "row.test = true;", - "params": "row", - "transType": 1, - } - save_resp = session.post(f"{base_url}api/{org_id}/functions", json=save_payload) - save_error = save_resp.text - - # Test test endpoint (snake_case for TestVRLRequest struct) - test_payload = { - "function": "row.test = true;", + "function": "row.field = undefinedVar;", # valid syntax, throws at runtime "events": [{"name": "test"}], "trans_type": 1, } - test_resp = session.post(f"{base_url}api/{org_id}/functions/test", json=test_payload) - test_error = test_resp.text + resp = session.post(f"{base_url}api/{org_id}/functions/test", json=payload) - # Both should have same core error message - expected_message = "JavaScript functions are only allowed in the '_meta' organization" - assert expected_message in save_error, "Save endpoint should have restriction message" - assert expected_message in test_error, "Test endpoint should have restriction message" + assert resp.status_code == 200, ( + f"JS runtime error should be 200 (per-event surfacing), " + f"but got {resp.status_code}: {resp.content}" + ) + results = resp.json()["results"] + assert results, f"expected per-event results, got: {results}" + msg = results[0].get("message", "") + assert "ReferenceError" in msg, f"expected ReferenceError in message, got: {msg!r}" + assert "undefinedVar" in msg, f"expected the offending name in message, got: {msg!r}" + + def test_runtime_error_surfaced_for_both_vrl_and_js(self, create_session, base_url): + """Runtime errors surface per-event for BOTH languages (200 + results[].message). + + Consistency check: whether the transform is VRL or JS, a runtime + failure that compiles cleanly returns 200 and carries the error in + results[].message rather than a top-level 400. + """ + session = create_session + org_id = "default" + + # JS runtime error + js_resp = session.post( + f"{base_url}api/{org_id}/functions/test", + json={ + "function": "row.field = undefinedVar;", + "events": [{"name": "test"}], + "trans_type": 1, + }, + ) + assert js_resp.status_code == 200, f"JS: {js_resp.status_code} {js_resp.content}" + js_msg = js_resp.json()["results"][0].get("message", "") + assert js_msg, f"JS runtime error should carry a per-event message, got: {js_msg!r}" + + # VRL runtime error (to_int! aborts when the value can't be coerced) + vrl_resp = session.post( + f"{base_url}api/{org_id}/functions/test", + json={ + "function": ".x = to_int!(.s)", + "events": [{"s": "notanumber"}], + "trans_type": 0, + }, + ) + assert vrl_resp.status_code == 200, f"VRL: {vrl_resp.status_code} {vrl_resp.content}" + vrl_msg = vrl_resp.json()["results"][0].get("message", "") + assert "vrl runtime error" in vrl_msg, ( + f"VRL runtime error should carry a per-event message, got: {vrl_msg!r}" + ) + + def test_partial_failure_flags_only_failing_events(self, create_session, base_url): + """Per-event semantics: in a mixed batch, only the failing event carries a message. + + The JS function throws only when row.bad is truthy, so event 1 fails + (ReferenceError) and event 2 transforms cleanly. Response is 200 with + one message-bearing result and one clean result. + """ + session = create_session + org_id = "default" + + payload = { + "function": "if (row.bad) { row.x = undefinedVar; } else { row.x = 1; }", + "events": [{"bad": True}, {"bad": False}], + "trans_type": 1, + } + resp = session.post(f"{base_url}api/{org_id}/functions/test", json=payload) + + assert resp.status_code == 200, f"{resp.status_code}: {resp.content}" + results = resp.json()["results"] + assert len(results) == 2, f"expected one result per event, got: {results}" + assert "ReferenceError" in results[0].get("message", ""), ( + f"failing event should carry a runtime error, got: {results[0]}" + ) + assert not results[1].get("message"), ( + f"clean event should have no error message, got: {results[1]}" + ) + assert results[1]["event"].get("x") == 1, ( + f"clean event should be transformed (x=1), got: {results[1]}" + ) + + def test_syntax_error_still_returns_400(self, create_session, base_url): + """Regression guard: a genuine SYNTAX error still 400s (not the 200 runtime path). + + Pins the line between "syntax error -> 400" and "runtime error -> 200" + so a future change can't blur it. Covers both languages. + """ + session = create_session + org_id = "default" + + # JS syntax error + js_resp = session.post( + f"{base_url}api/{org_id}/functions/test", + json={"function": "function broken {", "events": [{"a": 1}], "trans_type": 1}, + ) + assert js_resp.status_code == 400, ( + f"JS syntax error should be 400, got {js_resp.status_code}: {js_resp.content}" + ) + + # VRL syntax error + vrl_resp = session.post( + f"{base_url}api/{org_id}/functions/test", + json={"function": "=====", "events": [{"a": 1}], "trans_type": 0}, + ) + assert vrl_resp.status_code == 400, ( + f"VRL syntax error should be 400, got {vrl_resp.status_code}: {vrl_resp.content}" + ) class TestEdgeCases: diff --git a/tests/api-testing/tests/pipelines/test_js_enrichment.py b/tests/api-testing/tests/pipelines/test_js_enrichment.py index 9c3d730fec..baa80c1a47 100644 --- a/tests/api-testing/tests/pipelines/test_js_enrichment.py +++ b/tests/api-testing/tests/pipelines/test_js_enrichment.py @@ -234,27 +234,24 @@ def test_js_enrichment_multiple_lookups(client: OpenObserveClient): # ----- documented org-restriction (always runs) ----- -def test_js_function_in_default_org_returns_400(client: OpenObserveClient): - """OO restricts JS functions to the `_meta` org. Documents this constraint. +def test_js_function_in_default_org_allowed(client: OpenObserveClient): + """JS functions run in the `default` (non-_meta) org. Documents the current contract. - The original suite missed this — it POSTed JS to `default` and silently - failed. This test makes the constraint a first-class contract: if/when - the backend opens JS to other orgs, this test starts failing and you - know to update. + The backend used to restrict JS to `_meta` (400 elsewhere). That + restriction was removed (PR #13156): JS is now accepted in every org on + every edition. This test pins the positive contract — a valid JS transform + in `default` returns 200 and transforms the event. If the backend ever + re-introduces an org restriction, this starts failing and you know to update. """ - - pytest.skip("JS fns are now allowed in non _meta orgs") - return - payload = { "function": "row.x = 'js works here';", "events": [{"id": 1}], "trans_type": 1, } resp = client.post("functions/test", json=payload, org=DEFAULT_ORG) - assert resp.status_code == 400, \ - f"JS in non-_meta org should be 400, got {resp.status_code}: {resp.text}" + assert resp.status_code == 200, \ + f"JS in non-_meta org should now be 200, got {resp.status_code}: {resp.text}" body = resp.json() - assert "JavaScript" in body.get("message", "") or "_meta" in body.get("message", ""), \ - f"error message should mention JS restriction, got: {body}" + assert body["results"][0]["event"].get("x") == "js works here", \ + f"JS transform should have run, got: {body['results']}" diff --git a/tests/ui-testing/playwright-tests/Functions/js-transform-type.spec.js b/tests/ui-testing/playwright-tests/Functions/js-transform-type.spec.js index f40f2304cc..8b1f0e198a 100644 --- a/tests/ui-testing/playwright-tests/Functions/js-transform-type.spec.js +++ b/tests/ui-testing/playwright-tests/Functions/js-transform-type.spec.js @@ -218,7 +218,7 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio // - test_create_js_function_in_meta_org_success }); - test.describe('Non-Meta Organization - JavaScript Blocked', () => { + test.describe('Non-Meta Organization - JavaScript edition-gated', () => { test.beforeEach(async ({ page }) => { // Switch to a non-meta organization (self-hosted: 'default', cloud: user's own org) const nonMetaOrg = getNonMetaOrg(); @@ -235,33 +235,36 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio // NOTE: Consolidated from meta-org "JS radio NOT visible" + "VRL functions work" tests // Both validations preserved: JS hidden check + VRL creation - test('P0: Non-meta org - JS hidden, VRL works', { tag: ['@smoke', '@P0'] }, async ({ page }) => { - // SKIPPED — asserts the OSS-only rule, but runs on the enterprise build. - // JS is no longer _meta-only: AddFunction.vue offers it whenever - // `isEnterprise || isCloud || org === "_meta"`, so on ENT/cloud the JS radio - // is visible in EVERY org and step 1 below fails. The rule still holds on OSS. - // Re-enabling needs an edition signal, which is awkward: Vite bakes - // VITE_OPENOBSERVE_ENTERPRISE into the bundle at build time and /config - // exposes no edition key, so it can't be detected at runtime. - // TODO(QA): move to the OSS suite, or branch on an edition env var. - // NOTE: step 2 (VRL function creation in a non-meta org) is edition-agnostic - // and is lost while this is skipped. - test.skip(true, 'JS is allowed in all orgs on enterprise/cloud builds; OSS-only assertion'); - + test('P0: Non-meta org - JS radio edition-gated, VRL works', { tag: ['@smoke', '@P0'] }, async ({ page }) => { + // JS radio visibility in a non-meta org is EDITION-dependent (see file header): + // OSS -> JS is _meta-only, so the radio is HIDDEN here. + // ENT/cloud-> JS is offered in every org, so the radio is VISIBLE here. + // We detect the edition at runtime from the header edition button label + // (data-test="upgrade-to-enterprise-btn"), which is driven by the SAME + // build-time flags (config.isEnterprise/isCloud via @/aws-exports) that gate + // the JS radio — so it's the exact source of truth, no CI env var needed. + // Step 2 (VRL create in a non-meta org) is edition-agnostic and always runs. const nonMetaOrg = getNonMetaOrg(); - testLogger.info(`Test: Non-meta org (${nonMetaOrg}) - JS hidden, VRL works`); + const edition = await pm.editionFeaturesPage.detectEdition(); + testLogger.info(`Test: Non-meta org (${nonMetaOrg}) - edition=${edition}`); - // Step 1: Verify JS radio is hidden, VRL is visible + // Step 1: VRL is always visible; JS radio visibility depends on edition. await pm.functionsPage.clickAddFunctionButton(nonMetaOrg); await pm.functionsPage.expectVrlRadioVisible(); testLogger.info('VRL radio button visible'); - const isJsRadioVisible = await pm.functionsPage.isJsRadioVisible(); - expect(isJsRadioVisible).toBe(false); - testLogger.info(`JavaScript radio button correctly hidden in ${nonMetaOrg} org`); + if (edition === 'opensource') { + const isJsRadioVisible = await pm.functionsPage.isJsRadioVisible(); + expect(isJsRadioVisible).toBe(false); + testLogger.info(`OSS: JavaScript radio correctly hidden in ${nonMetaOrg} org`); + } else { + // enterprise / cloud + await pm.functionsPage.expectJsRadioVisible(); + testLogger.info(`${edition}: JavaScript radio correctly visible in ${nonMetaOrg} org`); + } await pm.functionsPage.clickCancelButton(); - // Step 2: Create VRL function to verify it works + // Step 2 (edition-agnostic): Create a VRL function to verify it works. const functionName = `nonmeta_vrl_fn_${Date.now()}`; const vrlCode = '.processed = true\n.org = "non_meta"'; @@ -283,26 +286,62 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio await pm.functionsPage.deleteFunctionByName(functionName, nonMetaOrg); }); - // NOTE: API tests for JavaScript function blocking in default org have been migrated to: - // tests/api-testing/tests/test_functions_meta_org_restriction.py - // - test_create_js_function_in_default_org_blocked - // - test_test_js_function_in_default_org_blocked + // Edition-agnostic: a VRL runtime error surfaces in the error-details section + // (mirrors the JS ReferenceError case in the _meta describe). VRL works in every + // org on every edition, so this runs unconditionally. Covers the VRL half of the + // "runtime errors now render per-event" behaviour change (PR #13156). + test('P1: VRL runtime error surfaces in error-details', { tag: ['@P1', '@errorHandling'] }, async ({ page }) => { + const nonMetaOrg = getNonMetaOrg(); + testLogger.info(`Test: VRL runtime error in error-details (${nonMetaOrg} org)`); + + // to_int!(.s) compiles fine but aborts at runtime when .s can't be coerced. + const functionName = `nonmeta_vrl_err_${Date.now()}`; + const vrlCode = '.x = to_int!(.s)'; + + await pm.functionsPage.clickAddFunctionButton(nonMetaOrg); + await pm.functionsPage.fillFunctionName(functionName); + // VRL is the default type + await pm.functionsPage.enterFunctionCode(vrlCode); + + // Run against an event that triggers the runtime abort. + await pm.functionsPage.testFunctionExecution('[{"s":"notanumber"}]'); + + // The per-event VRL runtime error renders in the error-details section + // (200 response with results[].message, not a 400). + await pm.functionsPage.expectFunctionErrorContains('runtime error'); + testLogger.info('VRL runtime error rendered in error-details section'); + await pm.functionsPage.clickCancelButton(); + }); + + // NOTE: API tests for JavaScript function behaviour in default org live in: + // tests/api-testing/tests/pipelines/test_functions_meta_org_restriction.py + // - test_create_js_function_in_default_org_allowed + // - test_test_js_function_in_default_org_allowed }); test.describe('Organization Switching & VRL Control', () => { // NOTE: Consolidated from meta-org "Org switching" + "VRL always visible" tests // Both validations preserved: JS visibility changes with org + VRL always visible test('P1: Radio visibility across orgs', { tag: ['@P1', '@orgSwitching', '@control'] }, async ({ page }) => { - // SKIPPED — same reason as 'P0: Non-meta org - JS hidden, VRL works' above: - // steps 1 and 3 expect the JS radio hidden in a non-meta org, which is only - // true on OSS. TODO(QA): move to the OSS suite, or branch on an edition env var. - // NOTE: step 2 (JS visible in _meta) is edition-agnostic and is lost meanwhile. - test.skip(true, 'JS is allowed in all orgs on enterprise/cloud builds; OSS-only assertion'); - + // JS radio visibility in a NON-meta org is edition-gated (OSS hides it, + // ENT/cloud shows it); in the _meta org JS is ALWAYS visible on every + // edition. VRL is always visible everywhere. We detect the edition from + // the header edition button (same build-flag source as the JS gate). const nonMetaOrg = getNonMetaOrg(); - testLogger.info(`Test: Radio visibility across orgs (non-meta: ${nonMetaOrg})`); + const edition = await pm.editionFeaturesPage.detectEdition(); + testLogger.info(`Test: Radio visibility across orgs (non-meta: ${nonMetaOrg}, edition=${edition})`); - // Step 1: Non-meta org - JS hidden, VRL visible + const expectJsRadioForNonMeta = async (label) => { + if (edition === 'opensource') { + await pm.functionsPage.expectJsRadioHidden(); + testLogger.info(`OSS: JS hidden in ${nonMetaOrg} org (${label})`); + } else { + await pm.functionsPage.expectJsRadioVisible(); + testLogger.info(`${edition}: JS visible in ${nonMetaOrg} org (${label})`); + } + }; + + // Step 1: Non-meta org - VRL visible, JS edition-gated await page.goto(`${process.env.ZO_BASE_URL}/web/pipeline/functions?org_identifier=${nonMetaOrg}`); await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {}); await page.waitForTimeout(2000); @@ -310,11 +349,10 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio await pm.functionsPage.clickAddFunctionButton(nonMetaOrg); await pm.functionsPage.expectVrlRadioVisible(); testLogger.info(`VRL visible in ${nonMetaOrg} org`); - await pm.functionsPage.expectJsRadioHidden(); - testLogger.info(`JS hidden in ${nonMetaOrg} org`); + await expectJsRadioForNonMeta('step 1'); await pm.functionsPage.clickCancelButton(); - // Step 2: Switch to _meta org - JS visible, VRL visible + // Step 2 (edition-agnostic): Switch to _meta org - JS visible, VRL visible await page.goto(`${process.env.ZO_BASE_URL}/web/pipeline/functions?org_identifier=_meta`); await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {}); await page.waitForTimeout(2000); @@ -323,10 +361,10 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio await pm.functionsPage.expectVrlRadioVisible(); testLogger.info('VRL visible in _meta org'); await pm.functionsPage.expectJsRadioVisible(); - testLogger.info('JS visible in _meta org'); + testLogger.info('JS visible in _meta org (always, every edition)'); await pm.functionsPage.clickCancelButton(); - // Step 3: Switch back to non-meta - JS hidden again, VRL still visible + // Step 3: Switch back to non-meta - JS gated again, VRL still visible await page.goto(`${process.env.ZO_BASE_URL}/web/pipeline/functions?org_identifier=${nonMetaOrg}`); await page.waitForLoadState('networkidle', { timeout: 10000 }).catch(() => {}); await page.waitForTimeout(2000); @@ -334,8 +372,7 @@ test.describe('JavaScript Transform Type', { tag: ['@jsTransformType', '@functio await pm.functionsPage.clickAddFunctionButton(nonMetaOrg); await pm.functionsPage.expectVrlRadioVisible(); testLogger.info('VRL still visible after switching back'); - await pm.functionsPage.expectJsRadioHidden(); - testLogger.info(`JS hidden after switching back to ${nonMetaOrg}`); + await expectJsRadioForNonMeta('step 3'); await pm.functionsPage.clickCancelButton(); }); });