From 4427a3d37f6e1edef93082322ee346077e0c1ff6 Mon Sep 17 00:00:00 2001 From: Diego Imbert <70353967+diegoimbert@users.noreply.github.com> Date: Thu, 7 May 2026 15:35:54 +0200 Subject: [PATCH] feat: add workspace-specific flag for resources and variables (#8836) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: add workspace-specific flag for resources and variables Co-Authored-By: Claude Opus 4.6 (1M context) * fix: remove set_ws_specific endpoint and fix rust-client compilation Co-Authored-By: Claude Opus 4.6 (1M context) * fix: fall back to workspace name for ws_specific file naming When wsNameForFiles is not set (no wmill.yaml workspace config), ws_specific items would not get workspace-suffixed filenames during pull. Now falls back to workspace.name/workspaceId. Co-Authored-By: Claude Opus 4.6 (1M context) * fix: use workspace ID instead of CLI name for ws_specific file naming Co-Authored-By: Claude Opus 4.6 (1M context) * fix: pass workspace ID fallback to elementsToMap for ws_specific push Without this, workspace-specific files (e.g., a.admins.resource.yaml) were not recognized during push when no wmill.yaml or git branch was available, causing spurious deletions. Co-Authored-By: Claude Opus 4.6 (1M context) * ui nits * nit * Fix variable edit when only editing ws_specific * mark_linked_variables_ws_specific * Helper label * Support json format alongside yaml * Fix file naming push/pull asymetry & ws_specific orphans * Revert all CLI diffs * CLI now appends the remote ws_specific list to the local specificItems * UI for Env switcher * Refactor Resource/Variable editors to use dumb component * Refactor side effects * Editor works with multi workspaces * Fix can_save * Fix As JSON * nit * UI nits * list_ws_specific_versions as pl sql function to avoid round trips * UI Nits * Per-workspace version read-only check * fix: reset session context in list_ws_specific_versions to prevent RLS leakage The function calls set_session_context() in a loop. Although SET LOCAL is transaction-scoped (so settings revert at autocommit), defending against the function being invoked inside a longer outer transaction: - wrap the loop in a sub-block with EXCEPTION WHEN OTHERS that resets the session to a deny-default (windmill_user, empty session.* GUCs) before re-raising, - on the happy path, reset to the same deny-default at the end of the function. * feat: audit auto-marked ws_specific variables When a resource is saved as ws_specific, every variable referenced via $var: inside its value is auto-INSERTed into ws_specific. Previously this happened silently. Now: - mark_linked_variables_ws_specific takes the authed user, - the INSERT uses RETURNING path so we know exactly which variables were freshly flipped (not the ones already ws_specific), - each newly flipped variable gets a 'variables.set_ws_specific' audit entry pointing at the resource that triggered it. * perf: skip mark_linked_variables_ws_specific when nothing relevant changed update_resource was calling mark_linked_variables_ws_specific on every save when the resource was ws_specific, even on a description-only or label-only edit. Gate the call on `ns.value.is_some() || ns.ws_specific == Some(true)` so we only re-mark when the $var: refs could actually have changed or ws_specific was freshly enabled. * docs: explain asymmetric ws_specific toggle in resource tooltip Enabling the resource's 'Workspace specific' toggle silently marks every variable referenced via $var: inside the value as ws_specific, but disabling it does not un-mark those variables (they may be referenced by other resources). Surface this in the tooltip so users know what to expect. * fix: surface non-404 errors when fetching ws_specific items in CLI sync mergeWsSpecificFromServer was catching every error from listWsSpecific and logging it at debug. That's correct for old servers without the endpoint (404), but a 401/403/network failure would silently produce an incomplete sync. Now distinguish 404 (debug, expected) from everything else (warn with status + message) so users notice when the merge fails for real reasons. * perf: collapse compare_two_variables presence checks into one round-trip The early-return path was issuing four sequential EXISTS queries (ws_specific × {source, fork}, variable × {source, fork}). Combine them into a single SELECT so the per-variable diff cost drops ~4x. * sqlx prepare * docs: clarify has_sql_updates invariant in update_variable The else branch of the npath resolution is only reachable for non-rename edits (labels-only, ws_specific-only) because ns.path being Some always forces has_sql_updates=true at the top of the function. Add a debug_assert and a comment explaining the invariant so a future change that decouples ns.path from has_sql_updates trips immediately. Also use `path` directly instead of unwrap_or_default-ing ns.path, since we know it's None here. * chore: drop redundant ws_specific type augmentations ListableResource and ListableVariable from $lib/gen now include `ws_specific?: boolean` after the openapi.yaml additions in this branch were regenerated. The intersection types in resources/+page and variables/+page were duplicating the field — drop them. * Put WsSpecificVersions toggle in top drawer bar * nit size * feat: detect local-only ws_specific items on sync push When wmill.yaml lists a resource/variable in specificItems but the remote isn't yet marked ws_specific for that item, sync push silently dropped the flag because: 1. file-content diff alone never noticed (ws_specific is metadata, not YAML body) — push{Resource,Variable} were never called for those items; 2. even when called, isSuperset(local, remote) returned true and the early-return skipped the API call. Now: - mergeWsSpecificFromServer returns the raw server list alongside the merged config so push can compare 'in local' vs 'in server'; - a new computeWsSpecificFlagOnlyPushes helper walks the local file map, finds ws_specific-flagged paths absent from the server list, and the push function injects them as synthetic 'edited' changes (same before and after content) so the standard display + apply pipeline picks them up; - push{Resource,Variable} no longer early-return when content matches but the ws_specific flag differs. Pull is unaffected — only the push-side caller of mergeWsSpecific takes the new (merged, serverItems) tuple. * getDeployTo for selected ws * refactor: ws_specific kind handling, support .json files The ws_specific helpers had two warts: 1. computeWsSpecificFlagOnlyPushes hardcoded `.resource.{yaml,json}` / `.variable.{yaml,json}` magic strings, even though the existing getTypeStrFromPath / removeType helpers already do that work and already cover both extensions. 2. isSpecificItem / isItemTypeConfigured only matched `.yaml` paths, so users with opts.json local files got no specificItems coverage at all — patterns from wmill.yaml (and from mergeWsSpecificFromServer) are expressed with `.yaml`, and a `.json` file never matched. Changes: - Replace WS_SPECIFIC_KIND_MAP (a closed enum of resource+variable) with configKeyForItemKind, a generic kind→SpecificItemsConfig key mapping. Triggers fold into 'triggers' via the `_trigger` suffix, so adding a kind to the backend's list_ws_specific_versions doesn't require a CLI change. - mergeWsSpecificFromServer now appends `${item.path}.${item.item_kind}.yaml` through the same helper. - computeWsSpecificFlagOnlyPushes uses getTypeStrFromPath + removeType, gated by configKeyForItemKind. No more magic strings. - isSpecificItem and isItemTypeConfigured normalize trailing `.json` to `.yaml` once at the entry, so a single set of patterns covers both extensions for the same logical item. * refactor: dedicated change type for ws_specific flag-only pushes Previously the sync push code injected a synthetic 'edited' Change with before === after to nudge the apply loop into calling pushResource / pushVariable for ws_specific-flag-only diffs, and a guard inside those two functions skipped the early-return when the flag differed. The contract was implicit and easy to break — any future 'skip identical edits' optimization in the change pipeline would silently drop these pushes. Replace with an explicit Change variant: type WsSpecificFlag = { name: 'ws_specific_flag'; path: string; kind: string; wsSpecific: boolean; }; The push apply loop now has a dedicated branch for it that calls wmill.updateResource / updateVariable with just the ws_specific flag. prettyChanges renders it on its own line. The dry-run JSON output picks it up via the existing change.name / change.path passthrough. The defensive wsSpecificMatches check inside push{Resource,Variable} is no longer needed (sync push doesn't go through them for flag-only diffs) and is reverted. * drop folders * feat(cli): warn on remote ws_specific items missing from local config When 'wmill sync pull' fetches the server's ws_specific list, items the server marks as ws_specific but that aren't matched by the local wmill.yaml's specificItems patterns now produce a warning. The merge already preserves correctness (those items are still treated as ws_specific during this pull), but the user's config drifts from the remote — and a later push from another machine without that config would push the item as non-ws_specific. Surface the drift so the user can update wmill.yaml. Also filter ws_specific_flag changes out before preCheckPermissionedAs (it expects added/edited/deleted only and they have no content payload so on_behalf_of resolution doesn't apply). * fix(cli): scope ws_specific drift warning to items in this pull's changes Previously the warning iterated every ws_specific item the server returned, producing log spam for items unrelated to the current pull (items that exist locally with no change, or items the user has nothing to do with this round). Move the loop after compareDynFSElement and only warn for items whose path appears in the changes list — i.e., items the user is actually pulling right now. * fix: clean up linked-side ws_specific rows on resource/variable delete Three places left orphaned ws_specific rows behind: 1. delete_resource deleted the resource's own ws_specific row and the linked variables, but never the ws_specific 'variable' rows that mark_linked_variables_ws_specific had auto-inserted for those variable paths. 2. delete_variable deleted its own ws_specific row and the linked resource at the same path, but never a ws_specific 'resource' row at that path. 3. delete_resources_bulk didn't even cascade to linked variables, let alone clean up their ws_specific rows. A new resource or variable later created at one of those paths would silently inherit a stale ws_specific flag — list_ws_specific would report it as workspace-specific, workspace diffs would treat it as 'no changes', and CLI sync would skip it. Fix: - delete_resource: DELETE FROM ws_specific WHERE item_kind = 'variable' AND path = ANY(linked_var_paths) before the linked-variable delete. - delete_variable: DELETE FROM ws_specific WHERE item_kind = 'resource' AND path = path before the linked-resource delete. - delete_resources_bulk: collect $var: refs from each bulk-deleted resource (mirror of single delete), then delete ws_specific 'variable' rows AND the variable rows themselves. Brings bulk delete in line with single delete semantics, including the orphan cleanup. * fix: gate list_ws_specific by resource/variable RLS The endpoint queried ws_specific directly under user_db, but ws_specific itself has no per-item RLS — only a workspace-level column. Any workspace member could enumerate every ws_specific path including those in folders they lack read access to (e.g. f/finance/prod_db_creds), revealing path existence that list_resources / list_variables would have hidden. Add EXISTS clauses against resource and variable so the same path-based RLS policies that govern those tables (see_own / see_member / see_extra_perms_user / see_extra_perms_groups / see_folder_extra_perms_user) also gate visibility here. The user transaction already establishes the session context; the joins make the policies apply. * only resources and variables * fix(cli): make workspace-specific path mapping handle .json files isSpecificItem() was extended to normalize .json -> .yaml so .json files could be matched against patterns, but the surrounding helpers remained yaml-only: - toWorkspaceSpecificPath only mapped folder.meta.yaml / settings.yaml / .X.yaml — a foo.resource.json went through unchanged, so the workspace-specific filename was never produced. - fromWorkspaceSpecificPath only matched .yaml extensions — pushing foo.dev.resource.json could not map back to foo.resource.json. - isCurrentWorkspaceFile / isWorkspaceSpecificFile regexes ended in \.yaml$, missing every branch-specific .json file. Replace the literal '.yaml' anchors with '(yaml|json)' alternations, preserve the actual extension on round-trips, and rename the helper buildYamlTypePattern -> buildItemTypePattern (it never had anything extension-specific in it). getFileTypeSuffix now returns the matching suffix for either extension. Changed: - getFileTypeSuffix - toWorkspaceSpecificPath / fromWorkspaceSpecificPath - isCurrentWorkspaceFile / isWorkspaceSpecificFile - isTriggerFile / isScheduleFile isItemTypeConfigured / isSpecificItem don't need touching — their checks run after normalizeJsonToYaml(), which already collapses both extensions to .yaml at the entry. * fix: create_resource?update_if_exists=true honors ws_specific=false The upsert path matched on `unwrap_or(false)`, so an explicit `ws_specific: false` and an absent flag were indistinguishable — both fell through with no DELETE on the existing ws_specific row. Callers trying to clear the flag via PUT-with-update_if_exists silently saw their request ignored. Mirror update_resource's three-way handling: Some(true) -> INSERT (+ mark linked variables) Some(false) -> DELETE (only when update_if_exists, since a pure create has no existing row anyway) None -> leave the existing flag alone create_variable doesn't have an upsert path (no ON CONFLICT), so the same bug doesn't apply there. * sqlx prepare * test: cover ws_specific cleanup, RLS filtering, upsert clearing, and CLI .json paths Backend (backend/tests/ws_specific.rs + fixture): - test_linked_delete_cleanup: creates a ws_specific resource that references a variable via $var:, deletes the resource, asserts the cross-kind ws_specific row for the auto-marked variable is also removed. Then does the inverse for delete_variable, verifying the ws_specific 'resource' row at the same path is cleaned by variable delete. - test_list_ws_specific_filters_by_rls: admin creates ws_specific items in u/test-user/ and u/test-user-2/; verifies admin sees both via list_ws_specific while a non-admin (test-user-2) only sees their own path — the RLS see_own policy on the joined resource/variable tables hides the other. - test_create_resource_upsert_clears_ws_specific: covers the three-way Option handling on the upsert path: Some(true) inserts, Some(false) clears the existing row, None leaves it alone. CLI: - specific_items_unit.test.ts: add 14 tests covering toWorkspaceSpecificPath / fromWorkspaceSpecificPath / isWorkspaceSpecificFile / isCurrentWorkspaceFile / isSpecificItem / isItemTypeConfigured for .json files (variable, resource, trigger, schedule, folder.meta, settings). - ws_specific_flag_only_unit.test.ts (new): covers computeWsSpecificFlagOnlyPushes — emits flag-only changes only for resource and variable kinds (the backend's list_ws_specific_versions scope), does not emit for schedules or triggers, returns empty when serverItems is null (older server), respects existing server entries, preserves .json extension on filePath. - Export computeWsSpecificFlagOnlyPushes so it can be unit-tested. * perf: index workspace_settings.deploy_to for the recursive CTE list_ws_specific_versions's recursive CTE probes WHERE ws.deploy_to = r.ws_id every iteration; without an index on workspace_settings.deploy_to each iteration seq-scans the table — at 10M workspaces with the depth cap of 32 that's up to 320M row reads per call. deploy_to is sparse (most workspaces don't deploy anywhere), so a partial index WHERE deploy_to IS NOT NULL stays small while still covering every probe. Tucked into the existing migration since the function and the index ship together. --------- Co-authored-by: Claude Opus 4.6 (1M context) --- .gitignore | 1 + ...c1ed508d83695ef6d62ca06cfb612fd332b87.json | 28 + ...42d730a45b431b057cee66688256e455b62e3.json | 15 + ...b91e3896009a047286747a9c9f4e8e0192e4a.json | 15 + ...cb2cb3345869afea8a82b8416f762aacf5dd9.json | 15 + ...ba6adcb5170d2825644fb02f6a1a57447f4fb.json | 15 + ...320bb3eff65255a44bd66799ae14288312ba4.json | 23 + ...5b52aaf1f79874b3369e7a01171491f6d9083.json | 15 + ...c31e10f89481908c479b4039af5e94fa0f8ac.json | 28 + ...81a20b92779d6c56b116c7189f3055ad15d35.json | 25 + ...659e2eb3f3d912e1c062bb8000a2e0a653e8.json} | 20 +- ...69657fd73742b6ee8289b1e9736e381314cfb.json | 23 + ...dc52cc9f471ce8e61a00132aed2dcce8653af.json | 16 + ...1864734ae5329e31a39d7af7bad4e3e2fe5b2.json | 42 ++ ...813046e86b8768a276f1b724d32721424fa08.json | 15 + ...ist_ws_specific_versions_function.down.sql | 2 + ..._list_ws_specific_versions_function.up.sql | 166 +++++ backend/tests/fixtures/ws_specific.sql | 45 ++ backend/tests/ws_specific.rs | 357 +++++++++++ .../windmill-api-workspaces/src/workspaces.rs | 106 ++++ backend/windmill-api/openapi.yaml | 71 +++ backend/windmill-common/src/variables.rs | 4 + backend/windmill-store/src/resources.rs | 208 ++++++- backend/windmill-store/src/variables.rs | 124 +++- cli/src/commands/resource/resource.ts | 6 +- cli/src/commands/sync/sync.ts | 262 +++++++- cli/src/commands/variable/variable.ts | 5 +- cli/src/core/specific_items.ts | 115 ++-- cli/src/types.ts | 7 +- cli/test/specific_items_unit.test.ts | 130 ++++ cli/test/ws_specific_flag_only_unit.test.ts | 117 ++++ .../src/lib/components/AppConnectInner.svelte | 44 +- .../src/lib/components/ResourceEditor.svelte | 569 ++++++++---------- .../components/ResourceEditorDrawer.svelte | 27 +- .../src/lib/components/ResourceForm.svelte | 283 +++++++++ .../src/lib/components/VariableEditor.svelte | 413 ++++++------- .../src/lib/components/VariableForm.svelte | 168 ++++++ .../lib/components/WsSpecificVersions.svelte | 54 ++ .../(root)/(logged)/resources/+page.svelte | 4 +- .../(root)/(logged)/variables/+page.svelte | 9 +- rust-client/src/client.rs | 4 + 41 files changed, 2954 insertions(+), 642 deletions(-) create mode 100644 backend/.sqlx/query-0c6e8f03a4e9f543cb85582e0aec1ed508d83695ef6d62ca06cfb612fd332b87.json create mode 100644 backend/.sqlx/query-44dc4a9fe7b51a184ed4599be8342d730a45b431b057cee66688256e455b62e3.json create mode 100644 backend/.sqlx/query-5c705ea49ceda8b281155e38ebdb91e3896009a047286747a9c9f4e8e0192e4a.json create mode 100644 backend/.sqlx/query-69378789e2d14778812c8d6f1a6cb2cb3345869afea8a82b8416f762aacf5dd9.json create mode 100644 backend/.sqlx/query-6a7e0a5ef270d7effeccbb8f9edba6adcb5170d2825644fb02f6a1a57447f4fb.json create mode 100644 backend/.sqlx/query-6be4bf59c404d2f557d1106c48c320bb3eff65255a44bd66799ae14288312ba4.json create mode 100644 backend/.sqlx/query-7e11b36055ed15a8f3a68511bc15b52aaf1f79874b3369e7a01171491f6d9083.json create mode 100644 backend/.sqlx/query-8b92a7d04fcdd8e61178d7dab97c31e10f89481908c479b4039af5e94fa0f8ac.json create mode 100644 backend/.sqlx/query-8dbe7d38df3493958456bb9454681a20b92779d6c56b116c7189f3055ad15d35.json rename backend/.sqlx/{query-41f2c271514ee254739c3a097871526adcecdd8729f28c15e0db8cd28eaa8cf0.json => query-9254e2a0e1be830fa4cab660c69a659e2eb3f3d912e1c062bb8000a2e0a653e8.json} (61%) create mode 100644 backend/.sqlx/query-b4162468afae99cf31c4668ca6769657fd73742b6ee8289b1e9736e381314cfb.json create mode 100644 backend/.sqlx/query-d1dabd750d2304107a981c992b3dc52cc9f471ce8e61a00132aed2dcce8653af.json create mode 100644 backend/.sqlx/query-e4d2a19d56e561e903c883d503f1864734ae5329e31a39d7af7bad4e3e2fe5b2.json create mode 100644 backend/.sqlx/query-f7e273ab42c1632ffe4d127b447813046e86b8768a276f1b724d32721424fa08.json create mode 100644 backend/migrations/20260505125724_list_ws_specific_versions_function.down.sql create mode 100644 backend/migrations/20260505125724_list_ws_specific_versions_function.up.sql create mode 100644 backend/tests/fixtures/ws_specific.sql create mode 100644 backend/tests/ws_specific.rs create mode 100644 cli/test/ws_specific_flag_only_unit.test.ts create mode 100644 frontend/src/lib/components/ResourceForm.svelte create mode 100644 frontend/src/lib/components/VariableForm.svelte create mode 100644 frontend/src/lib/components/WsSpecificVersions.svelte diff --git a/.gitignore b/.gitignore index b2741131a5..10889080ad 100644 --- a/.gitignore +++ b/.gitignore @@ -32,3 +32,4 @@ backend/chrome_profiler.json .fast-check/ __pycache__/ .playwright-mcp/ +.codex \ No newline at end of file diff --git a/backend/.sqlx/query-0c6e8f03a4e9f543cb85582e0aec1ed508d83695ef6d62ca06cfb612fd332b87.json b/backend/.sqlx/query-0c6e8f03a4e9f543cb85582e0aec1ed508d83695ef6d62ca06cfb612fd332b87.json new file mode 100644 index 0000000000..7de0416a12 --- /dev/null +++ b/backend/.sqlx/query-0c6e8f03a4e9f543cb85582e0aec1ed508d83695ef6d62ca06cfb612fd332b87.json @@ -0,0 +1,28 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT item_kind, path FROM ws_specific WHERE workspace_id = $1", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "item_kind", + "type_info": "Varchar" + }, + { + "ordinal": 1, + "name": "path", + "type_info": "Varchar" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false + ] + }, + "hash": "0c6e8f03a4e9f543cb85582e0aec1ed508d83695ef6d62ca06cfb612fd332b87" +} diff --git a/backend/.sqlx/query-44dc4a9fe7b51a184ed4599be8342d730a45b431b057cee66688256e455b62e3.json b/backend/.sqlx/query-44dc4a9fe7b51a184ed4599be8342d730a45b431b057cee66688256e455b62e3.json new file mode 100644 index 0000000000..0aa56e6a3a --- /dev/null +++ b/backend/.sqlx/query-44dc4a9fe7b51a184ed4599be8342d730a45b431b057cee66688256e455b62e3.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "DELETE FROM variable WHERE workspace_id = $1 AND path = ANY($2)", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Text", + "TextArray" + ] + }, + "nullable": [] + }, + "hash": "44dc4a9fe7b51a184ed4599be8342d730a45b431b057cee66688256e455b62e3" +} diff --git a/backend/.sqlx/query-5c705ea49ceda8b281155e38ebdb91e3896009a047286747a9c9f4e8e0192e4a.json b/backend/.sqlx/query-5c705ea49ceda8b281155e38ebdb91e3896009a047286747a9c9f4e8e0192e4a.json new file mode 100644 index 0000000000..85077b883b --- /dev/null +++ b/backend/.sqlx/query-5c705ea49ceda8b281155e38ebdb91e3896009a047286747a9c9f4e8e0192e4a.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "INSERT INTO ws_specific (workspace_id, item_kind, path) VALUES ($1, 'variable', $2) ON CONFLICT DO NOTHING", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Varchar" + ] + }, + "nullable": [] + }, + "hash": "5c705ea49ceda8b281155e38ebdb91e3896009a047286747a9c9f4e8e0192e4a" +} diff --git a/backend/.sqlx/query-69378789e2d14778812c8d6f1a6cb2cb3345869afea8a82b8416f762aacf5dd9.json b/backend/.sqlx/query-69378789e2d14778812c8d6f1a6cb2cb3345869afea8a82b8416f762aacf5dd9.json new file mode 100644 index 0000000000..dcb44dbb90 --- /dev/null +++ b/backend/.sqlx/query-69378789e2d14778812c8d6f1a6cb2cb3345869afea8a82b8416f762aacf5dd9.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'variable' AND path = ANY($2)", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Text", + "TextArray" + ] + }, + "nullable": [] + }, + "hash": "69378789e2d14778812c8d6f1a6cb2cb3345869afea8a82b8416f762aacf5dd9" +} diff --git a/backend/.sqlx/query-6a7e0a5ef270d7effeccbb8f9edba6adcb5170d2825644fb02f6a1a57447f4fb.json b/backend/.sqlx/query-6a7e0a5ef270d7effeccbb8f9edba6adcb5170d2825644fb02f6a1a57447f4fb.json new file mode 100644 index 0000000000..9c35d02b73 --- /dev/null +++ b/backend/.sqlx/query-6a7e0a5ef270d7effeccbb8f9edba6adcb5170d2825644fb02f6a1a57447f4fb.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'variable' AND path = $2", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [] + }, + "hash": "6a7e0a5ef270d7effeccbb8f9edba6adcb5170d2825644fb02f6a1a57447f4fb" +} diff --git a/backend/.sqlx/query-6be4bf59c404d2f557d1106c48c320bb3eff65255a44bd66799ae14288312ba4.json b/backend/.sqlx/query-6be4bf59c404d2f557d1106c48c320bb3eff65255a44bd66799ae14288312ba4.json new file mode 100644 index 0000000000..0a39db6822 --- /dev/null +++ b/backend/.sqlx/query-6be4bf59c404d2f557d1106c48c320bb3eff65255a44bd66799ae14288312ba4.json @@ -0,0 +1,23 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT EXISTS(SELECT 1 FROM variable WHERE workspace_id = $1 AND path = $2)", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "exists", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [ + null + ] + }, + "hash": "6be4bf59c404d2f557d1106c48c320bb3eff65255a44bd66799ae14288312ba4" +} diff --git a/backend/.sqlx/query-7e11b36055ed15a8f3a68511bc15b52aaf1f79874b3369e7a01171491f6d9083.json b/backend/.sqlx/query-7e11b36055ed15a8f3a68511bc15b52aaf1f79874b3369e7a01171491f6d9083.json new file mode 100644 index 0000000000..253dde7a8e --- /dev/null +++ b/backend/.sqlx/query-7e11b36055ed15a8f3a68511bc15b52aaf1f79874b3369e7a01171491f6d9083.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "DELETE FROM ws_specific\n WHERE workspace_id = $1 AND item_kind = 'variable' AND path = ANY($2)", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Text", + "TextArray" + ] + }, + "nullable": [] + }, + "hash": "7e11b36055ed15a8f3a68511bc15b52aaf1f79874b3369e7a01171491f6d9083" +} diff --git a/backend/.sqlx/query-8b92a7d04fcdd8e61178d7dab97c31e10f89481908c479b4039af5e94fa0f8ac.json b/backend/.sqlx/query-8b92a7d04fcdd8e61178d7dab97c31e10f89481908c479b4039af5e94fa0f8ac.json new file mode 100644 index 0000000000..415544ece9 --- /dev/null +++ b/backend/.sqlx/query-8b92a7d04fcdd8e61178d7dab97c31e10f89481908c479b4039af5e94fa0f8ac.json @@ -0,0 +1,28 @@ +{ + "db_name": "PostgreSQL", + "query": "\n SELECT s.item_kind, s.path\n FROM ws_specific s\n WHERE s.workspace_id = $1\n AND (\n (s.item_kind = 'resource' AND EXISTS (\n SELECT 1 FROM resource r\n WHERE r.workspace_id = s.workspace_id AND r.path = s.path\n ))\n OR (s.item_kind = 'variable' AND EXISTS (\n SELECT 1 FROM variable v\n WHERE v.workspace_id = s.workspace_id AND v.path = s.path\n ))\n )\n ", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "item_kind", + "type_info": "Varchar" + }, + { + "ordinal": 1, + "name": "path", + "type_info": "Varchar" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + false, + false + ] + }, + "hash": "8b92a7d04fcdd8e61178d7dab97c31e10f89481908c479b4039af5e94fa0f8ac" +} diff --git a/backend/.sqlx/query-8dbe7d38df3493958456bb9454681a20b92779d6c56b116c7189f3055ad15d35.json b/backend/.sqlx/query-8dbe7d38df3493958456bb9454681a20b92779d6c56b116c7189f3055ad15d35.json new file mode 100644 index 0000000000..4ca7a134e3 --- /dev/null +++ b/backend/.sqlx/query-8dbe7d38df3493958456bb9454681a20b92779d6c56b116c7189f3055ad15d35.json @@ -0,0 +1,25 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT ws AS \"ws!\" FROM list_ws_specific_versions($1, $2, $3, $4)", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "ws!", + "type_info": "Varchar" + } + ], + "parameters": { + "Left": [ + "Text", + "Text", + "Text", + "Text" + ] + }, + "nullable": [ + null + ] + }, + "hash": "8dbe7d38df3493958456bb9454681a20b92779d6c56b116c7189f3055ad15d35" +} diff --git a/backend/.sqlx/query-41f2c271514ee254739c3a097871526adcecdd8729f28c15e0db8cd28eaa8cf0.json b/backend/.sqlx/query-9254e2a0e1be830fa4cab660c69a659e2eb3f3d912e1c062bb8000a2e0a653e8.json similarity index 61% rename from backend/.sqlx/query-41f2c271514ee254739c3a097871526adcecdd8729f28c15e0db8cd28eaa8cf0.json rename to backend/.sqlx/query-9254e2a0e1be830fa4cab660c69a659e2eb3f3d912e1c062bb8000a2e0a653e8.json index 78efa6ccbc..7959ccdbcb 100644 --- a/backend/.sqlx/query-41f2c271514ee254739c3a097871526adcecdd8729f28c15e0db8cd28eaa8cf0.json +++ b/backend/.sqlx/query-9254e2a0e1be830fa4cab660c69a659e2eb3f3d912e1c062bb8000a2e0a653e8.json @@ -1,6 +1,6 @@ { "db_name": "PostgreSQL", - "query": "SELECT resource.*, (now() > account.expires_at) as is_expired, account.refresh_token != '' as is_refreshed,\n account.refresh_error,\n variable.path IS NOT NULL as is_linked,\n variable.is_oauth as \"is_oauth?\",\n variable.account\n FROM resource\n LEFT JOIN variable ON variable.path = resource.path AND variable.workspace_id = $2\n LEFT JOIN account ON variable.account = account.id AND account.workspace_id = $2\n WHERE resource.path = $1 AND resource.workspace_id = $2", + "query": "SELECT resource.workspace_id, resource.path, resource.value, resource.description,\n resource.resource_type, resource.extra_perms, resource.created_by, resource.edited_at,\n resource.labels,\n (now() > account.expires_at) as is_expired, account.refresh_token != '' as is_refreshed,\n account.refresh_error,\n variable.path IS NOT NULL as is_linked,\n variable.is_oauth as \"is_oauth?\",\n variable.account,\n ws_specific.path IS NOT NULL as ws_specific\n FROM resource\n LEFT JOIN variable ON variable.path = resource.path AND variable.workspace_id = $2\n LEFT JOIN account ON variable.account = account.id AND account.workspace_id = $2\n LEFT JOIN ws_specific ON ws_specific.path = resource.path AND ws_specific.workspace_id = $2 AND ws_specific.item_kind = 'resource'\n WHERE resource.path = $1 AND resource.workspace_id = $2", "describe": { "columns": [ { @@ -35,13 +35,13 @@ }, { "ordinal": 6, - "name": "edited_at", - "type_info": "Timestamptz" + "name": "created_by", + "type_info": "Varchar" }, { "ordinal": 7, - "name": "created_by", - "type_info": "Varchar" + "name": "edited_at", + "type_info": "Timestamptz" }, { "ordinal": 8, @@ -77,6 +77,11 @@ "ordinal": 14, "name": "account", "type_info": "Int4" + }, + { + "ordinal": 15, + "name": "ws_specific", + "type_info": "Bool" } ], "parameters": { @@ -100,8 +105,9 @@ true, null, false, - true + true, + null ] }, - "hash": "41f2c271514ee254739c3a097871526adcecdd8729f28c15e0db8cd28eaa8cf0" + "hash": "9254e2a0e1be830fa4cab660c69a659e2eb3f3d912e1c062bb8000a2e0a653e8" } diff --git a/backend/.sqlx/query-b4162468afae99cf31c4668ca6769657fd73742b6ee8289b1e9736e381314cfb.json b/backend/.sqlx/query-b4162468afae99cf31c4668ca6769657fd73742b6ee8289b1e9736e381314cfb.json new file mode 100644 index 0000000000..327032afb5 --- /dev/null +++ b/backend/.sqlx/query-b4162468afae99cf31c4668ca6769657fd73742b6ee8289b1e9736e381314cfb.json @@ -0,0 +1,23 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT EXISTS(SELECT 1 FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'variable' AND path = $2)", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "exists", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [ + null + ] + }, + "hash": "b4162468afae99cf31c4668ca6769657fd73742b6ee8289b1e9736e381314cfb" +} diff --git a/backend/.sqlx/query-d1dabd750d2304107a981c992b3dc52cc9f471ce8e61a00132aed2dcce8653af.json b/backend/.sqlx/query-d1dabd750d2304107a981c992b3dc52cc9f471ce8e61a00132aed2dcce8653af.json new file mode 100644 index 0000000000..1cc8d7fa64 --- /dev/null +++ b/backend/.sqlx/query-d1dabd750d2304107a981c992b3dc52cc9f471ce8e61a00132aed2dcce8653af.json @@ -0,0 +1,16 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE ws_specific SET path = $1 WHERE workspace_id = $2 AND item_kind = 'variable' AND path = $3", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Text", + "Text" + ] + }, + "nullable": [] + }, + "hash": "d1dabd750d2304107a981c992b3dc52cc9f471ce8e61a00132aed2dcce8653af" +} diff --git a/backend/.sqlx/query-e4d2a19d56e561e903c883d503f1864734ae5329e31a39d7af7bad4e3e2fe5b2.json b/backend/.sqlx/query-e4d2a19d56e561e903c883d503f1864734ae5329e31a39d7af7bad4e3e2fe5b2.json new file mode 100644 index 0000000000..39e187e227 --- /dev/null +++ b/backend/.sqlx/query-e4d2a19d56e561e903c883d503f1864734ae5329e31a39d7af7bad4e3e2fe5b2.json @@ -0,0 +1,42 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT\n EXISTS(SELECT 1 FROM ws_specific\n WHERE workspace_id = $1 AND item_kind = 'variable' AND path = $3) AS \"src_ws!\",\n EXISTS(SELECT 1 FROM ws_specific\n WHERE workspace_id = $2 AND item_kind = 'variable' AND path = $3) AS \"tgt_ws!\",\n EXISTS(SELECT 1 FROM variable\n WHERE workspace_id = $1 AND path = $3) AS \"src_var!\",\n EXISTS(SELECT 1 FROM variable\n WHERE workspace_id = $2 AND path = $3) AS \"tgt_var!\"", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "src_ws!", + "type_info": "Bool" + }, + { + "ordinal": 1, + "name": "tgt_ws!", + "type_info": "Bool" + }, + { + "ordinal": 2, + "name": "src_var!", + "type_info": "Bool" + }, + { + "ordinal": 3, + "name": "tgt_var!", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text", + "Text", + "Text" + ] + }, + "nullable": [ + null, + null, + null, + null + ] + }, + "hash": "e4d2a19d56e561e903c883d503f1864734ae5329e31a39d7af7bad4e3e2fe5b2" +} diff --git a/backend/.sqlx/query-f7e273ab42c1632ffe4d127b447813046e86b8768a276f1b724d32721424fa08.json b/backend/.sqlx/query-f7e273ab42c1632ffe4d127b447813046e86b8768a276f1b724d32721424fa08.json new file mode 100644 index 0000000000..dee0944da5 --- /dev/null +++ b/backend/.sqlx/query-f7e273ab42c1632ffe4d127b447813046e86b8768a276f1b724d32721424fa08.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "DELETE FROM ws_specific\n WHERE workspace_id = $1 AND item_kind = 'resource' AND path = $2", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [] + }, + "hash": "f7e273ab42c1632ffe4d127b447813046e86b8768a276f1b724d32721424fa08" +} diff --git a/backend/migrations/20260505125724_list_ws_specific_versions_function.down.sql b/backend/migrations/20260505125724_list_ws_specific_versions_function.down.sql new file mode 100644 index 0000000000..894ad8e409 --- /dev/null +++ b/backend/migrations/20260505125724_list_ws_specific_versions_function.down.sql @@ -0,0 +1,2 @@ +DROP FUNCTION IF EXISTS list_ws_specific_versions(TEXT, TEXT, TEXT, TEXT); +DROP INDEX IF EXISTS workspace_settings_deploy_to_idx; diff --git a/backend/migrations/20260505125724_list_ws_specific_versions_function.up.sql b/backend/migrations/20260505125724_list_ws_specific_versions_function.up.sql new file mode 100644 index 0000000000..84e7d5e345 --- /dev/null +++ b/backend/migrations/20260505125724_list_ws_specific_versions_function.up.sql @@ -0,0 +1,166 @@ +-- Index supporting the recursive CTE in list_ws_specific_versions: each +-- iteration probes `WHERE ws.deploy_to = r.ws_id`, which without an index +-- on workspace_settings.deploy_to seq-scans the whole table per iteration +-- (up to depth-cap × |workspace_settings| row reads per call). deploy_to +-- is sparse — most workspaces don't deploy anywhere — so a partial index +-- keeps the index small while still covering every probe. +CREATE INDEX IF NOT EXISTS workspace_settings_deploy_to_idx + ON workspace_settings (deploy_to) + WHERE deploy_to IS NOT NULL; + +-- Returns the list of workspace ids related to `seed_workspace` via the +-- `workspace_settings.deploy_to` graph (in either direction) for which a +-- `resource` or `variable` row at `item_path` exists AND is visible to +-- `user_email` under that workspace's RLS context. +-- +-- For each related workspace this function: +-- 1. resolves the user's per-workspace identity (`usr` row by email, +-- groups via `usr_to_group` + `email_to_igroup`, folders via +-- `folder.extra_perms`), +-- 2. switches the role + session.* settings via `set_session_context` +-- so the next query is evaluated under that workspace's RLS, +-- 3. runs an `EXISTS` against the requested item table. +-- +-- This consolidates the "fan out per-workspace, set RLS, EXISTS" loop +-- that previously lived in the Rust handler into a single round trip. +CREATE OR REPLACE FUNCTION list_ws_specific_versions( + seed_workspace TEXT, + user_email TEXT, + item_kind TEXT, + item_path TEXT +) RETURNS TABLE(ws VARCHAR) AS $$ +DECLARE + rel RECORD; + usr_row RECORD; + user_perms TEXT[]; + groups_csv TEXT; + pgroups_csv TEXT; + folders_read_csv TEXT; + folders_write_csv TEXT; + item_exists BOOLEAN; + is_super BOOLEAN; +BEGIN + IF item_kind NOT IN ('resource', 'variable') THEN + RAISE EXCEPTION 'Invalid kind: %', item_kind; + END IF; + + SELECT COALESCE(super_admin, false) INTO is_super + FROM password WHERE email = user_email; + is_super := COALESCE(is_super, false); + + BEGIN + FOR rel IN + WITH RECURSIVE related_workspaces(ws_id, depth) AS ( + SELECT seed_workspace::VARCHAR, 0 + UNION + SELECT CASE + WHEN ws.workspace_id = r.ws_id THEN ws.deploy_to + ELSE ws.workspace_id + END, r.depth + 1 + FROM workspace_settings ws, related_workspaces r + WHERE r.depth < 32 + AND ((ws.workspace_id = r.ws_id AND ws.deploy_to IS NOT NULL) + OR ws.deploy_to = r.ws_id) + ) + SELECT DISTINCT r.ws_id + FROM related_workspaces r + INNER JOIN workspace w ON w.id = r.ws_id AND w.deleted = false + LOOP + SELECT u.username, u.is_admin + INTO usr_row + FROM usr u + WHERE u.email = user_email + AND u.workspace_id = rel.ws_id + AND u.disabled = false; + + IF NOT FOUND AND NOT is_super THEN + CONTINUE; + END IF; + + IF NOT FOUND THEN + -- super admin without a usr row in this workspace: synthesize an + -- admin identity so RLS is bypassed (windmill_admin role). + usr_row.username := user_email; + usr_row.is_admin := true; + groups_csv := ''; + pgroups_csv := ''; + folders_read_csv := ''; + folders_write_csv := ''; + ELSE + SELECT + COALESCE(string_agg(g, ','), ''), + COALESCE(string_agg('g/' || g, ','), '') + INTO groups_csv, pgroups_csv + FROM ( + SELECT group_ AS g FROM usr_to_group + WHERE usr_to_group.usr = usr_row.username + AND usr_to_group.workspace_id = rel.ws_id + UNION ALL + SELECT igroup FROM email_to_igroup WHERE email = user_email + ) gs; + + user_perms := ARRAY['u/' || usr_row.username] || ARRAY( + SELECT 'g/' || g FROM ( + SELECT group_ AS g FROM usr_to_group + WHERE usr = usr_row.username AND workspace_id = rel.ws_id + UNION ALL + SELECT igroup FROM email_to_igroup WHERE email = user_email + ) gs2 + ); + + -- folders_read: every folder the user can see (write implies read); + -- folders_write: only those granting write access. + WITH user_folders AS ( + SELECT name, EXISTS ( + SELECT 1 FROM jsonb_each_text(extra_perms) t + WHERE t.key = ANY(user_perms) AND t.value::boolean IS true + ) AS is_write + FROM folder + WHERE extra_perms ?| user_perms AND folder.workspace_id = rel.ws_id + ) + SELECT + COALESCE(string_agg(name, ','), ''), + COALESCE(string_agg(name, ',') FILTER (WHERE is_write), '') + INTO folders_read_csv, folders_write_csv + FROM user_folders; + + IF is_super THEN + usr_row.is_admin := true; + END IF; + END IF; + + PERFORM set_session_context( + usr_row.is_admin, + usr_row.username, + groups_csv, + pgroups_csv, + folders_read_csv, + folders_write_csv + ); + + EXECUTE format( + 'SELECT EXISTS(SELECT 1 FROM %I WHERE workspace_id = $1 AND path = $2)', + item_kind + ) + INTO item_exists + USING rel.ws_id, item_path; + + IF item_exists THEN + ws := rel.ws_id; + RETURN NEXT; + END IF; + END LOOP; + EXCEPTION WHEN OTHERS THEN + -- Reset to a deny-default state before re-raising so a half-set + -- session context can't leak past the failed call. + PERFORM set_session_context(false, '', '', '', '', ''); + RAISE; + END; + + -- Reset to a deny-default state on the happy path too. SET LOCAL is + -- transaction-scoped so this also unwinds at transaction end, but + -- being explicit defends against the function being called inside a + -- longer outer transaction. + PERFORM set_session_context(false, '', '', '', '', ''); +END; +$$ LANGUAGE plpgsql; diff --git a/backend/tests/fixtures/ws_specific.sql b/backend/tests/fixtures/ws_specific.sql new file mode 100644 index 0000000000..fcad6ce55c --- /dev/null +++ b/backend/tests/fixtures/ws_specific.sql @@ -0,0 +1,45 @@ +-- Fixture for ws_specific integration tests. +-- Adds a non-admin user (test-user-2) as a regular workspace member with +-- access to user-owned items at u/test-user-2/* but not u/test-user/*. + +INSERT INTO workspace + (id, name, owner) + VALUES ('test-workspace', 'test-workspace', 'test-user') +ON CONFLICT DO NOTHING; + +INSERT INTO usr(workspace_id, email, username, is_admin, role) VALUES + ('test-workspace', 'test@windmill.dev', 'test-user', true, 'Admin') +ON CONFLICT DO NOTHING; + +INSERT INTO usr(workspace_id, email, username, is_admin, role) VALUES + ('test-workspace', 'test2@windmill.dev', 'test-user-2', false, 'User') +ON CONFLICT DO NOTHING; + +INSERT INTO workspace_key(workspace_id, kind, key) VALUES + ('test-workspace', 'cloud', 'test-key') +ON CONFLICT DO NOTHING; + +INSERT INTO workspace_settings (workspace_id) VALUES + ('test-workspace') +ON CONFLICT DO NOTHING; + +INSERT INTO group_ (workspace_id, name, summary, extra_perms) VALUES + ('test-workspace', 'all', 'All users', '{}') +ON CONFLICT DO NOTHING; + +INSERT INTO password(email, password_hash, login_type, super_admin, verified, name, username) + VALUES ('test@windmill.dev', 'not-a-real-hash', 'password', true, true, 'Test User', 'test-user') +ON CONFLICT DO NOTHING; + +INSERT INTO password(email, password_hash, login_type, super_admin, verified, name) + VALUES ('test2@windmill.dev', 'not-a-real-hash', 'password', false, true, 'Test User 2') +ON CONFLICT DO NOTHING; + +-- Tokens +insert INTO token(token_hash, token_prefix, token, email, label, super_admin) VALUES + (encode(sha256('SECRET_TOKEN'::bytea), 'hex'), 'SECRET_TOK', 'SECRET_TOKEN', 'test@windmill.dev', 'admin token', true) +ON CONFLICT DO NOTHING; + +insert INTO token(token_hash, token_prefix, token, email, label, super_admin) VALUES + (encode(sha256('SECRET_TOKEN_2'::bytea), 'hex'), 'SECRET_TOK', 'SECRET_TOKEN_2', 'test2@windmill.dev', 'user2 token', false) +ON CONFLICT DO NOTHING; diff --git a/backend/tests/ws_specific.rs b/backend/tests/ws_specific.rs new file mode 100644 index 0000000000..51b3590678 --- /dev/null +++ b/backend/tests/ws_specific.rs @@ -0,0 +1,357 @@ +//! Integration tests for the workspace-specific (ws_specific) feature. +//! +//! Covers three regression-prone areas: +//! +//! 1. **Linked-delete cleanup** — deleting a resource (or variable) must also +//! drop the cross-kind ws_specific row that was auto-inserted by +//! `mark_linked_variables_ws_specific` so a later item recreated at the +//! same path doesn't inherit a stale flag. +//! 2. **list_ws_specific authorization filtering** — the endpoint must hide +//! paths the caller cannot see via the underlying resource/variable RLS. +//! 3. **Resource upsert with `ws_specific: false`** — `create_resource` with +//! `update_if_exists=true` and `ws_specific: false` must clear an +//! existing flag (was previously a silent no-op). + +use serde_json::json; +use sqlx::{Pool, Postgres}; +use windmill_test_utils::*; + +fn client() -> reqwest::Client { + reqwest::Client::new() +} + +fn authed(builder: reqwest::RequestBuilder, token: &str) -> reqwest::RequestBuilder { + builder.header("Authorization", format!("Bearer {}", token)) +} + +/// Helper: count rows in ws_specific for (workspace, kind, path). +async fn ws_specific_row_count( + db: &Pool, + workspace: &str, + kind: &str, + path: &str, +) -> anyhow::Result { + let n: Option = sqlx::query_scalar( + "SELECT COUNT(*) FROM ws_specific + WHERE workspace_id = $1 AND item_kind = $2 AND path = $3", + ) + .bind(workspace) + .bind(kind) + .bind(path) + .fetch_one(db) + .await?; + Ok(n.unwrap_or(0)) +} + +#[sqlx::test(fixtures("ws_specific"))] +async fn test_linked_delete_cleanup(db: Pool) -> anyhow::Result<()> { + initialize_tracing().await; + let server = ApiServer::start(db.clone()).await?; + let port = server.addr.port(); + let base = format!("http://localhost:{port}/api/w/test-workspace"); + + // Create a referenced variable. + let resp = authed( + client().post(format!("{base}/variables/create")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/db_pwd", + "value": "hunter2", + "is_secret": false, + "description": "" + })) + .send() + .await?; + assert_eq!(resp.status(), 201, "create var: {}", resp.text().await?); + + // Create a ws_specific resource that references the variable via $var:. + let resp = authed( + client().post(format!("{base}/resources/create")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/db", + "value": { "user": "admin", "password": "$var:u/test-user/db_pwd" }, + "description": "", + "resource_type": "object", + "ws_specific": true + })) + .send() + .await?; + assert_eq!(resp.status(), 201, "create res: {}", resp.text().await?); + + // The auto-mark on save inserts a ws_specific 'variable' row for the + // linked variable. + assert_eq!( + ws_specific_row_count(&db, "test-workspace", "variable", "u/test-user/db_pwd").await?, + 1, + "linked variable should be auto-marked ws_specific" + ); + assert_eq!( + ws_specific_row_count(&db, "test-workspace", "resource", "u/test-user/db").await?, + 1 + ); + + // Delete the resource — it should cascade to the linked variable AND + // the ws_specific row for that variable. + let resp = authed( + client().delete(format!("{base}/resources/delete/u/test-user/db")), + "SECRET_TOKEN", + ) + .send() + .await?; + assert_eq!(resp.status(), 200, "delete res: {}", resp.text().await?); + + assert_eq!( + ws_specific_row_count(&db, "test-workspace", "resource", "u/test-user/db").await?, + 0, + "resource ws_specific row should be gone" + ); + assert_eq!( + ws_specific_row_count(&db, "test-workspace", "variable", "u/test-user/db_pwd").await?, + 0, + "orphaned linked-variable ws_specific row should also be gone" + ); + + // The same fix applies in reverse: delete_variable must clean the + // ws_specific 'resource' row at the same path. + let resp = authed( + client().post(format!("{base}/variables/create")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/twin", + "value": "v", + "is_secret": false, + "description": "" + })) + .send() + .await?; + assert_eq!(resp.status(), 201); + + let resp = authed( + client().post(format!("{base}/resources/create")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/twin", + "value": { "x": 1 }, + "resource_type": "object", + "ws_specific": true + })) + .send() + .await?; + assert_eq!(resp.status(), 201); + + // ws_specific row exists for resource at u/test-user/twin + assert_eq!( + ws_specific_row_count(&db, "test-workspace", "resource", "u/test-user/twin").await?, + 1 + ); + + // Delete the variable at the same path: cascades to the resource AND + // its ws_specific row. + let resp = authed( + client().delete(format!("{base}/variables/delete/u/test-user/twin")), + "SECRET_TOKEN", + ) + .send() + .await?; + assert_eq!(resp.status(), 200, "delete var: {}", resp.text().await?); + + assert_eq!( + ws_specific_row_count(&db, "test-workspace", "resource", "u/test-user/twin").await?, + 0, + "ws_specific resource row should be cleaned by variable delete" + ); + + Ok(()) +} + +#[sqlx::test(fixtures("ws_specific"))] +async fn test_list_ws_specific_filters_by_rls(db: Pool) -> anyhow::Result<()> { + initialize_tracing().await; + let server = ApiServer::start(db.clone()).await?; + let port = server.addr.port(); + let base = format!("http://localhost:{port}/api/w/test-workspace"); + + // Admin creates two ws_specific items: one in u/test-user/* (private to + // test-user) and one in u/test-user-2/* (private to test-user-2). + for path in ["u/test-user/admin_only", "u/test-user-2/user2_only"] { + let resp = authed( + client().post(format!("{base}/variables/create")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": path, + "value": "v", + "is_secret": false, + "description": "", + "ws_specific": true + })) + .send() + .await?; + assert_eq!( + resp.status(), + 201, + "create var {path}: {}", + resp.text().await? + ); + } + + // Admin sees both via list_ws_specific. + let resp = authed( + client().get(format!("{base}/workspaces/list_ws_specific")), + "SECRET_TOKEN", + ) + .send() + .await?; + assert_eq!(resp.status(), 200); + let admin_items: Vec = resp.json().await?; + let admin_paths: Vec<&str> = admin_items + .iter() + .filter_map(|i| i.get("path").and_then(|p| p.as_str())) + .collect(); + assert!(admin_paths.contains(&"u/test-user/admin_only")); + assert!(admin_paths.contains(&"u/test-user-2/user2_only")); + + // Non-admin (test-user-2) only sees their own u/test-user-2/* path — + // u/test-user/admin_only is filtered by RLS see_own (path requires + // SPLIT_PART(path,'/',2) = session.user). + let resp = authed( + client().get(format!("{base}/workspaces/list_ws_specific")), + "SECRET_TOKEN_2", + ) + .send() + .await?; + assert_eq!(resp.status(), 200); + let user2_items: Vec = resp.json().await?; + let user2_paths: Vec<&str> = user2_items + .iter() + .filter_map(|i| i.get("path").and_then(|p| p.as_str())) + .collect(); + assert!( + user2_paths.contains(&"u/test-user-2/user2_only"), + "user2 should see their own ws_specific item, got: {user2_paths:?}" + ); + assert!( + !user2_paths.contains(&"u/test-user/admin_only"), + "user2 should NOT see admin's ws_specific item, got: {user2_paths:?}" + ); + + Ok(()) +} + +#[sqlx::test(fixtures("ws_specific"))] +async fn test_create_resource_upsert_clears_ws_specific(db: Pool) -> anyhow::Result<()> { + initialize_tracing().await; + let server = ApiServer::start(db.clone()).await?; + let port = server.addr.port(); + let base = format!("http://localhost:{port}/api/w/test-workspace"); + + // Step 1: create resource with ws_specific=true. + let resp = authed( + client().post(format!("{base}/resources/create")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/upsert_target", + "value": { "host": "h" }, + "resource_type": "object", + "ws_specific": true + })) + .send() + .await?; + assert_eq!(resp.status(), 201, "{}", resp.text().await?); + assert_eq!( + ws_specific_row_count( + &db, + "test-workspace", + "resource", + "u/test-user/upsert_target" + ) + .await?, + 1 + ); + + // Step 2: upsert (update_if_exists=true) with ws_specific=false — must + // CLEAR the existing row. + let resp = authed( + client().post(format!("{base}/resources/create?update_if_exists=true")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/upsert_target", + "value": { "host": "h2" }, + "resource_type": "object", + "ws_specific": false + })) + .send() + .await?; + assert_eq!(resp.status(), 201, "{}", resp.text().await?); + assert_eq!( + ws_specific_row_count( + &db, + "test-workspace", + "resource", + "u/test-user/upsert_target" + ) + .await?, + 0, + "ws_specific=false on upsert must clear the existing row" + ); + + // Step 3: upsert without ws_specific (None) leaves whatever's there + // alone — re-flag it true, then upsert with no field, expect row stays. + let resp = authed( + client().post(format!("{base}/resources/create?update_if_exists=true")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/upsert_target", + "value": { "host": "h3" }, + "resource_type": "object", + "ws_specific": true + })) + .send() + .await?; + assert_eq!(resp.status(), 201); + assert_eq!( + ws_specific_row_count( + &db, + "test-workspace", + "resource", + "u/test-user/upsert_target" + ) + .await?, + 1 + ); + + let resp = authed( + client().post(format!("{base}/resources/create?update_if_exists=true")), + "SECRET_TOKEN", + ) + .json(&json!({ + "path": "u/test-user/upsert_target", + "value": { "host": "h4" }, + "resource_type": "object" + // no ws_specific field + })) + .send() + .await?; + assert_eq!(resp.status(), 201); + assert_eq!( + ws_specific_row_count( + &db, + "test-workspace", + "resource", + "u/test-user/upsert_target" + ) + .await?, + 1, + "absent ws_specific field must leave the existing flag alone" + ); + + Ok(()) +} diff --git a/backend/windmill-api-workspaces/src/workspaces.rs b/backend/windmill-api-workspaces/src/workspaces.rs index 1a557f7877..f479b8c719 100644 --- a/backend/windmill-api-workspaces/src/workspaces.rs +++ b/backend/windmill-api-workspaces/src/workspaces.rs @@ -184,6 +184,8 @@ pub fn workspaced_service() -> Router { .route("/log_chat", post(log_ai_chat)) .route("/cloud_quotas", get(get_cloud_quotas)) .route("/prune_versions", post(prune_versions)) + .route("/list_ws_specific", get(list_ws_specific)) + .route("/list_ws_specific_versions", get(list_ws_specific_versions)) } pub fn global_service() -> Router { Router::new() @@ -6822,6 +6824,35 @@ async fn compare_two_variables( fork_workspace_id: &str, path: &str, ) -> Result { + // Combine the four EXISTS checks (ws_specific × {source, fork}, variable + // × {source, fork}) into a single round-trip; this runs per variable + // during a workspace diff so the savings add up. + let presence = sqlx::query!( + r#"SELECT + EXISTS(SELECT 1 FROM ws_specific + WHERE workspace_id = $1 AND item_kind = 'variable' AND path = $3) AS "src_ws!", + EXISTS(SELECT 1 FROM ws_specific + WHERE workspace_id = $2 AND item_kind = 'variable' AND path = $3) AS "tgt_ws!", + EXISTS(SELECT 1 FROM variable + WHERE workspace_id = $1 AND path = $3) AS "src_var!", + EXISTS(SELECT 1 FROM variable + WHERE workspace_id = $2 AND path = $3) AS "tgt_var!""#, + source_workspace_id, + fork_workspace_id, + path, + ) + .fetch_one(db) + .await?; + + // If either side is ws_specific, consider unchanged + if presence.src_ws || presence.tgt_ws { + return Ok(ItemComparison { + has_changes: false, + exists_in_source: presence.src_var, + exists_in_fork: presence.tgt_var, + }); + } + // Get variable from each workspace let source_variable = sqlx::query!( "SELECT value, is_secret, description @@ -7285,3 +7316,78 @@ async fn prune_versions( Ok(Json(PruneVersionsResponse { pruned })) } + +#[derive(Serialize)] +struct WsSpecificItem { + item_kind: String, + path: String, +} + +async fn list_ws_specific( + authed: ApiAuthed, + Extension(user_db): Extension, + Path(w_id): Path, +) -> JsonResult> { + // ws_specific itself has no per-item RLS — only the workspace_id column. + // Joining against resource/variable under user_db forces the same + // path-based RLS policies that govern those tables (see_own / see_member / + // see_extra_perms_* / see_folder_extra_perms_user) to also gate visibility + // here. Without these joins, any workspace member could enumerate paths + // in folders they lack read access to (e.g. f/finance/prod_db_creds). + let mut tx = user_db.begin(&authed).await?; + let items = sqlx::query_as!( + WsSpecificItem, + r#" + SELECT s.item_kind, s.path + FROM ws_specific s + WHERE s.workspace_id = $1 + AND ( + (s.item_kind = 'resource' AND EXISTS ( + SELECT 1 FROM resource r + WHERE r.workspace_id = s.workspace_id AND r.path = s.path + )) + OR (s.item_kind = 'variable' AND EXISTS ( + SELECT 1 FROM variable v + WHERE v.workspace_id = s.workspace_id AND v.path = s.path + )) + ) + "#, + &w_id + ) + .fetch_all(&mut *tx) + .await?; + tx.commit().await?; + Ok(Json(items)) +} + +#[derive(Deserialize)] +struct ListWsSpecificVersionsQuery { + kind: String, + path: String, +} + +async fn list_ws_specific_versions( + authed: ApiAuthed, + Extension(db): Extension, + Path(w_id): Path, + Query(q): Query, +) -> JsonResult> { + if q.kind != "resource" && q.kind != "variable" { + return Err(Error::BadRequest(format!( + "Invalid kind '{}'. Must be 'resource' or 'variable'", + q.kind + ))); + } + + let versions: Vec = sqlx::query_scalar!( + r#"SELECT ws AS "ws!" FROM list_ws_specific_versions($1, $2, $3, $4)"#, + &w_id, + &authed.email, + &q.kind, + &q.path, + ) + .fetch_all(&db) + .await?; + + Ok(Json(versions)) +} diff --git a/backend/windmill-api/openapi.yaml b/backend/windmill-api/openapi.yaml index 4a213e20a5..ee5c7f3b8f 100644 --- a/backend/windmill-api/openapi.yaml +++ b/backend/windmill-api/openapi.yaml @@ -5756,6 +5756,63 @@ paths: required: - pruned + /w/{workspace}/workspaces/list_ws_specific: + get: + summary: list all workspace-specific items + operationId: listWsSpecific + tags: + - workspace + parameters: + - $ref: "#/components/parameters/WorkspaceId" + responses: + "200": + description: list of workspace-specific items + content: + application/json: + schema: + type: array + items: + type: object + properties: + item_kind: + type: string + path: + type: string + required: + - item_kind + - path + + /w/{workspace}/workspaces/list_ws_specific_versions: + get: + summary: list workspace ids that have a version of the given item + operationId: listWsSpecificVersions + tags: + - workspace + parameters: + - $ref: "#/components/parameters/WorkspaceId" + - name: kind + in: query + required: true + schema: + type: string + enum: + - resource + - variable + - name: path + in: query + required: true + schema: + type: string + responses: + "200": + description: list of workspace ids that have a version of the item + content: + application/json: + schema: + type: array + items: + type: string + /w/{workspace}/workspaces/public_app_rate_limit: post: summary: Set public app rate limit for this workspace @@ -22613,6 +22670,8 @@ components: type: array items: type: string + ws_specific: + type: boolean required: - workspace_id - path @@ -22665,6 +22724,8 @@ components: type: array items: type: string + ws_specific: + type: boolean required: - path - value @@ -22690,6 +22751,8 @@ components: type: array items: type: string + ws_specific: + type: boolean AuditLog: type: object @@ -23086,6 +23149,8 @@ components: type: array items: type: string + ws_specific: + type: boolean required: - path - value @@ -23108,6 +23173,8 @@ components: type: array items: type: string + ws_specific: + type: boolean Resource: type: object @@ -23136,6 +23203,8 @@ components: type: array items: type: string + ws_specific: + type: boolean required: - path - resource_type @@ -23178,6 +23247,8 @@ components: type: array items: type: string + ws_specific: + type: boolean required: - path - resource_type diff --git a/backend/windmill-common/src/variables.rs b/backend/windmill-common/src/variables.rs index 3dbdadd924..57e41bb487 100644 --- a/backend/windmill-common/src/variables.rs +++ b/backend/windmill-common/src/variables.rs @@ -48,6 +48,8 @@ pub struct ListableVariable { pub expires_at: Option>, #[serde(skip_serializing_if = "Option::is_none")] pub labels: Option>, + #[serde(skip_serializing_if = "Option::is_none")] + pub ws_specific: Option, } #[derive(Serialize, Deserialize, sqlx::FromRow)] @@ -83,6 +85,8 @@ pub struct CreateVariable { pub expires_at: Option>, #[serde(default)] pub labels: Option>, + #[serde(default)] + pub ws_specific: Option, } pub async fn build_crypt(db: &DB, w_id: &str) -> crate::error::Result { diff --git a/backend/windmill-store/src/resources.rs b/backend/windmill-store/src/resources.rs index ed471bfaa7..6b169a0c0d 100644 --- a/backend/windmill-store/src/resources.rs +++ b/backend/windmill-store/src/resources.rs @@ -147,6 +147,8 @@ pub struct ListableResource { pub account: Option, #[serde(skip_serializing_if = "Option::is_none")] pub labels: Option>, + #[serde(skip_serializing_if = "Option::is_none")] + pub ws_specific: Option, } #[derive(Deserialize)] @@ -156,6 +158,8 @@ pub struct CreateResource { pub description: Option, pub resource_type: String, pub labels: Option>, + #[serde(default)] + pub ws_specific: Option, } #[derive(Deserialize)] struct EditResource { @@ -163,6 +167,7 @@ struct EditResource { description: Option, value: Option>, labels: Option>, + ws_specific: Option, } #[derive(Deserialize)] @@ -260,11 +265,15 @@ async fn list_resources( "resource.created_by", "resource.edited_at", "resource.labels", + "ws_specific.path IS NOT NULL as ws_specific", ]) .left() .join("variable") .on("variable.path = resource.path AND variable.workspace_id = resource.workspace_id") .left() + .join("ws_specific") + .on("ws_specific.path = resource.path AND ws_specific.workspace_id = resource.workspace_id AND ws_specific.item_kind = 'resource'") + .left() .join("account") .on("variable.account = account.id AND account.workspace_id = variable.workspace_id") .order_by("path", true) @@ -350,14 +359,19 @@ async fn get_resource( let resource_o = sqlx::query_as!( ListableResource, - "SELECT resource.*, (now() > account.expires_at) as is_expired, account.refresh_token != '' as is_refreshed, + "SELECT resource.workspace_id, resource.path, resource.value, resource.description, + resource.resource_type, resource.extra_perms, resource.created_by, resource.edited_at, + resource.labels, + (now() > account.expires_at) as is_expired, account.refresh_token != '' as is_refreshed, account.refresh_error, variable.path IS NOT NULL as is_linked, variable.is_oauth as \"is_oauth?\", - variable.account + variable.account, + ws_specific.path IS NOT NULL as ws_specific FROM resource LEFT JOIN variable ON variable.path = resource.path AND variable.workspace_id = $2 LEFT JOIN account ON variable.account = account.id AND account.workspace_id = $2 + LEFT JOIN ws_specific ON ws_specific.path = resource.path AND ws_specific.workspace_id = $2 AND ws_specific.item_kind = 'resource' WHERE resource.path = $1 AND resource.workspace_id = $2", path.to_owned(), &w_id @@ -849,6 +863,34 @@ async fn create_resource( ) .execute(&mut *tx) .await?; + + // Mirror update_resource: Some(true) inserts, Some(false) clears (only + // meaningful on the upsert path, since a pure create has no existing row), + // None leaves the existing flag alone. + match resource.ws_specific { + Some(true) => { + sqlx::query!( + "INSERT INTO ws_specific (workspace_id, item_kind, path) VALUES ($1, 'resource', $2) ON CONFLICT DO NOTHING", + w_id, + resource.path, + ) + .execute(&mut *tx) + .await?; + + mark_linked_variables_ws_specific(&mut tx, &authed, &w_id, &resource.path).await?; + } + Some(false) if update_if_exists => { + sqlx::query!( + "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'resource' AND path = $2", + w_id, + resource.path, + ) + .execute(&mut *tx) + .await?; + } + _ => {} + } + audit_log( &mut *tx, &authed, @@ -971,6 +1013,20 @@ async fn delete_resource( let deleted_linked_variables: Vec = if linked_var_paths.is_empty() { Vec::new() } else { + // Clean up any ws_specific rows for these variables first + // (mark_linked_variables_ws_specific may have auto-inserted them) so + // they don't survive the variable deletion as orphans — a variable + // later recreated at the same path would otherwise inherit the stale + // ws_specific flag. + sqlx::query!( + "DELETE FROM ws_specific + WHERE workspace_id = $1 AND item_kind = 'variable' AND path = ANY($2)", + w_id, + &linked_var_paths + ) + .execute(&mut *tx) + .await?; + let placeholders: Vec = linked_var_paths .iter() .enumerate() @@ -1082,6 +1138,66 @@ fn collect_var_refs(value: &serde_json::Value, out: &mut Vec) { } } +async fn mark_linked_variables_ws_specific( + tx: &mut Transaction<'_, Postgres>, + authed: &ApiAuthed, + w_id: &str, + resource_path: &str, +) -> Result<()> { + let resource_value: Option> = + sqlx::query_scalar("SELECT value FROM resource WHERE path = $1 AND workspace_id = $2") + .bind(resource_path) + .bind(w_id) + .fetch_optional(&mut **tx) + .await?; + + let mut linked_var_paths: Vec = Vec::new(); + if let Some(Some(ref value)) = resource_value { + collect_var_refs(value, &mut linked_var_paths); + } + + linked_var_paths.sort(); + linked_var_paths.dedup(); + + if linked_var_paths.is_empty() { + return Ok(()); + } + + // RETURNING gives us only the rows actually inserted (RFC: ON CONFLICT + // DO NOTHING + RETURNING returns the affected rows, i.e. the new ones). + let newly_marked: Vec = sqlx::query_scalar( + "INSERT INTO ws_specific (workspace_id, item_kind, path) + SELECT workspace_id, 'variable', path + FROM variable + WHERE workspace_id = $1 AND path = ANY($2::text[]) + ON CONFLICT DO NOTHING + RETURNING path", + ) + .bind(w_id) + .bind(&linked_var_paths) + .fetch_all(&mut **tx) + .await?; + + // Audit each variable that was actually flipped to ws_specific so the + // change is traceable to the resource save that caused it. + for var_path in &newly_marked { + let mut params = HashMap::new(); + params.insert("via_resource", resource_path); + audit_log( + &mut **tx, + authed, + "variables.set_ws_specific", + ActionKind::Update, + w_id, + Some(var_path), + Some(params), + ) + .await?; + } + + Ok(()) +} + async fn delete_resources_bulk( authed: ApiAuthed, Extension(db): Extension, @@ -1108,7 +1224,10 @@ async fn delete_resources_bulk( let mut tx = user_db.begin(&authed).await?; - // Capture resources for trashbin per path before bulk delete + // Capture resources for trashbin per path before bulk delete, and + // collect $var: references so we can cascade-delete the linked variables + // (matching single-resource delete semantics). + let mut linked_var_paths: Vec = Vec::new(); for path in &request.paths { let trash_resource: Option = sqlx::query_scalar( "SELECT to_jsonb(t) FROM resource t WHERE path = $1 AND workspace_id = $2", @@ -1119,6 +1238,9 @@ async fn delete_resources_bulk( .await?; if let Some(res_data) = trash_resource { + if let Some(value) = res_data.get("value") { + collect_var_refs(value, &mut linked_var_paths); + } let trash_data = serde_json::json!({"row": res_data}); windmill_common::trashbin::move_to_trash( &mut *tx, @@ -1131,6 +1253,8 @@ async fn delete_resources_bulk( .await?; } } + linked_var_paths.sort(); + linked_var_paths.dedup(); sqlx::query!( "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'resource' AND path = ANY($2)", @@ -1148,6 +1272,30 @@ async fn delete_resources_bulk( .fetch_all(&mut *tx) .await?; + // Cascade-clean linked variables: delete any ws_specific 'variable' rows + // (typically auto-inserted by mark_linked_variables_ws_specific when the + // resource was ws_specific) BEFORE deleting the variable rows themselves + // — otherwise those ws_specific rows survive as orphans and a later + // variable created at the same path would inherit a stale flag. + if !linked_var_paths.is_empty() { + sqlx::query!( + "DELETE FROM ws_specific + WHERE workspace_id = $1 AND item_kind = 'variable' AND path = ANY($2)", + w_id, + &linked_var_paths + ) + .execute(&mut *tx) + .await?; + + sqlx::query!( + "DELETE FROM variable WHERE workspace_id = $1 AND path = ANY($2)", + w_id, + &linked_var_paths + ) + .execute(&mut *tx) + .await?; + } + audit_log( &mut *tx, &authed, @@ -1219,7 +1367,7 @@ async fn update_resource( if let Some(npath) = &ns.path { sqlb.set_str("path", npath); } - if let Some(nvalue) = ns.value { + if let Some(nvalue) = &ns.value { sqlb.set_str("value", nvalue.to_string()); } if let Some(ndesc) = ns.description { @@ -1302,6 +1450,15 @@ async fn update_resource( ) .execute(&mut *tx) .await?; + + sqlx::query!( + "UPDATE ws_specific SET path = $1 WHERE workspace_id = $2 AND item_kind = 'variable' AND path = $3", + npath, + w_id, + path + ) + .execute(&mut *tx) + .await?; } } @@ -1321,6 +1478,49 @@ async fn update_resource( .await?; } + if let Some(ws_specific) = ns.ws_specific { + if ws_specific { + sqlx::query!( + "INSERT INTO ws_specific (workspace_id, item_kind, path) VALUES ($1, 'resource', $2) ON CONFLICT DO NOTHING", + w_id, + &npath, + ) + .execute(&mut *tx) + .await?; + } else { + sqlx::query!( + "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'resource' AND path = $2", + w_id, + &npath, + ) + .execute(&mut *tx) + .await?; + } + } + + // Only re-mark linked variables when something that could change them + // actually changed: a new value (different $var: refs) or ws_specific + // freshly enabled. Skipping when neither changed avoids re-running an + // INSERT (and audit logs) on every save. + let needs_remark = ns.value.is_some() || ns.ws_specific == Some(true); + if needs_remark { + let effective_ws_specific = if let Some(ws_specific) = ns.ws_specific { + ws_specific + } else { + sqlx::query_scalar::<_, bool>( + "SELECT EXISTS(SELECT 1 FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'resource' AND path = $2)", + ) + .bind(&w_id) + .bind(&npath) + .fetch_one(&mut *tx) + .await? + }; + + if effective_ws_specific { + mark_linked_variables_ws_specific(&mut tx, &authed, &w_id, &npath).await?; + } + } + audit_log( &mut *tx, &authed, diff --git a/backend/windmill-store/src/variables.rs b/backend/windmill-store/src/variables.rs index 6b9cd0581c..a9f8734647 100644 --- a/backend/windmill-store/src/variables.rs +++ b/backend/windmill-store/src/variables.rs @@ -133,6 +133,7 @@ async fn list_variables( "account.refresh_token != '' as is_refreshed", "variable.expires_at", "variable.labels", + "ws_specific.path IS NOT NULL as ws_specific", ]) .left() .join("account") @@ -140,6 +141,9 @@ async fn list_variables( .left() .join("resource") .on("resource.path = variable.path AND resource.workspace_id = ?".bind(&w_id)) + .left() + .join("ws_specific") + .on("ws_specific.path = variable.path AND ws_specific.workspace_id = variable.workspace_id AND ws_specific.item_kind = 'variable'") .and_where("variable.workspace_id = ?".bind(&w_id)) .and_where("variable.path NOT LIKE 'u/' || ? || '/secret_arg/%'".bind(&authed.username)) .order_by("path", false) @@ -209,12 +213,17 @@ async fn get_variable( let mut tx = user_db.begin(&authed).await?; let variable_o = sqlx::query_as::<_, ListableVariable>( - "SELECT variable.*, (now() > account.expires_at) as is_expired, account.refresh_error, + "SELECT variable.workspace_id, variable.path, variable.value, variable.is_secret, + variable.description, variable.extra_perms, variable.account, variable.is_oauth, + variable.expires_at, variable.labels, + (now() > account.expires_at) as is_expired, account.refresh_error, resource.path IS NOT NULL as is_linked, - account.refresh_token != '' as is_refreshed - from variable + account.refresh_token != '' as is_refreshed, + ws_specific.path IS NOT NULL as ws_specific + FROM variable LEFT JOIN account ON variable.account = account.id LEFT JOIN resource ON resource.path = variable.path AND resource.workspace_id = $2 + LEFT JOIN ws_specific ON ws_specific.path = variable.path AND ws_specific.workspace_id = $2 AND ws_specific.item_kind = 'variable' WHERE variable.path = $1 AND variable.workspace_id = $2 LIMIT 1", ) @@ -447,6 +456,16 @@ async fn create_variable( .execute(&mut *tx) .await?; + if variable.ws_specific.unwrap_or(false) { + sqlx::query!( + "INSERT INTO ws_specific (workspace_id, item_kind, path) VALUES ($1, 'variable', $2) ON CONFLICT DO NOTHING", + w_id, + variable.path, + ) + .execute(&mut *tx) + .await?; + } + audit_log( &mut *tx, &authed, @@ -552,6 +571,18 @@ async fn delete_variable( ) .execute(&mut *tx) .await?; + // Clean up any ws_specific row for the linked resource at the same path + // before deleting the resource itself — symmetric with the cleanup + // delete_resource does for linked variables. Without this, a resource + // later recreated at the same path would inherit a stale ws_specific flag. + sqlx::query!( + "DELETE FROM ws_specific + WHERE workspace_id = $1 AND item_kind = 'resource' AND path = $2", + w_id, + path + ) + .execute(&mut *tx) + .await?; let deleted_linked_resource = sqlx::query_scalar!( "DELETE FROM resource WHERE path = $1 AND workspace_id = $2 RETURNING path", path, @@ -576,6 +607,14 @@ async fn delete_variable( .await?; } + sqlx::query!( + "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'variable' AND path = $2", + w_id, + path + ) + .execute(&mut *tx) + .await?; + audit_log( &mut *tx, &authed, @@ -724,6 +763,14 @@ async fn delete_variables_bulk( .execute(&mut *tx) .await?; + sqlx::query!( + "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'variable' AND path = ANY($2)", + w_id, + &deleted_paths + ) + .execute(&mut *tx) + .await?; + audit_log( &mut *tx, &authed, @@ -779,6 +826,7 @@ struct EditVariable { description: Option, account: Option, labels: Option>, + ws_specific: Option, } #[derive(Deserialize)] @@ -814,11 +862,13 @@ async fn update_variable( let authed = maybe_refresh_folders(&path, &w_id, authed, &db).await; let mut sqlb = SqlBuilder::update_table("variable"); + let mut has_sql_updates = false; sqlb.and_where_eq("path", "?".bind(&path)); sqlb.and_where_eq("workspace_id", "?".bind(&w_id)); if let Some(npath) = &ns.path { sqlb.set_str("path", npath); + has_sql_updates = true; } let ns_value_is_none = ns.value.is_none(); // Determine the target path for storing secrets (use new path if provided) @@ -845,14 +895,17 @@ async fn update_variable( nvalue }; sqlb.set_str("value", &value); + has_sql_updates = true; } if let Some(desc) = ns.description { sqlb.set_str("description", &desc); + has_sql_updates = true; } if let Some(account_id) = ns.account { sqlb.set_str("account", account_id); + has_sql_updates = true; } if let Some(nbool) = ns.is_secret { @@ -870,8 +923,8 @@ async fn update_variable( )); } sqlb.set_str("is_secret", nbool); + has_sql_updates = true; } - sqlb.returning("path"); // Get old account_id if we're updating the account field let old_account_id = if ns.account.is_some() { @@ -916,6 +969,7 @@ async fn update_variable( { // Update the variable's value to point to the new Vault path sqlb.set_str("value", &new_value); + has_sql_updates = true; } } } @@ -956,14 +1010,46 @@ async fn update_variable( ) .execute(&mut *tx) .await?; + + sqlx::query!( + "UPDATE ws_specific SET path = $1 WHERE workspace_id = $2 AND item_kind = 'variable' AND path = $3", + npath, + w_id, + path + ) + .execute(&mut *tx) + .await?; } } - let sql = sqlb.sql().map_err(|e| Error::internal_err(e.to_string()))?; - - let npath_o: Option = sqlx::query_scalar(&sql).fetch_optional(&mut *tx).await?; - - let npath = not_found_if_none(npath_o, "Variable", path)?; + let npath = if has_sql_updates { + sqlb.returning("path"); + let sql = sqlb.sql().map_err(|e| Error::internal_err(e.to_string()))?; + let npath_o: Option = sqlx::query_scalar(&sql).fetch_optional(&mut *tx).await?; + not_found_if_none(npath_o, "Variable", path)? + } else { + // `has_sql_updates` is guaranteed true whenever `ns.path` is provided + // (set unconditionally above when ns.path is Some). So this branch + // only runs for non-rename edits (e.g. labels-only or ws_specific-only) + // and the rename-side queries (secret table, variable_history, + // ws_specific path UPDATE) in the rename block above are skipped. + debug_assert!( + ns.path.is_none(), + "has_sql_updates should be true when ns.path is Some" + ); + let exists = sqlx::query_scalar::<_, bool>( + "SELECT EXISTS(SELECT 1 FROM variable WHERE path = $1 AND workspace_id = $2)", + ) + .bind(path) + .bind(&w_id) + .fetch_one(&mut *tx) + .await?; + not_found_if_none( + if exists { Some(path.to_string()) } else { None }, + "Variable", + path, + )? + }; if let Some(nlabels) = &ns.labels { sqlx::query!( @@ -976,6 +1062,26 @@ async fn update_variable( .await?; } + if let Some(ws_specific) = ns.ws_specific { + if ws_specific { + sqlx::query!( + "INSERT INTO ws_specific (workspace_id, item_kind, path) VALUES ($1, 'variable', $2) ON CONFLICT DO NOTHING", + w_id, + &npath, + ) + .execute(&mut *tx) + .await?; + } else { + sqlx::query!( + "DELETE FROM ws_specific WHERE workspace_id = $1 AND item_kind = 'variable' AND path = $2", + w_id, + &npath, + ) + .execute(&mut *tx) + .await?; + } + } + audit_log( &mut *tx, &authed, diff --git a/cli/src/commands/resource/resource.ts b/cli/src/commands/resource/resource.ts index f8a8713928..f954b35adb 100644 --- a/cli/src/commands/resource/resource.ts +++ b/cli/src/commands/resource/resource.ts @@ -51,7 +51,8 @@ export async function pushResource( remotePath: string, resource: ResourceFile | Resource | undefined, localResource: ResourceFile, - originalLocalPath?: string + originalLocalPath?: string, + wsSpecific?: boolean, ): Promise { remotePath = removeType(remotePath, "resource"); try { @@ -103,7 +104,7 @@ export async function pushResource( await wmill.updateResource({ workspace: workspace, path: remotePath.replaceAll(SEP, "/"), - requestBody: { ...localResource }, + requestBody: { ...localResource, ...(wsSpecific !== undefined ? { ws_specific: wsSpecific } : {}) }, }); } else { // New resource - resolve inline content @@ -123,6 +124,7 @@ export async function pushResource( requestBody: { path: remotePath.replaceAll(SEP, "/"), ...localResource, + ...(wsSpecific !== undefined ? { ws_specific: wsSpecific } : {}), }, }); } diff --git a/cli/src/commands/sync/sync.ts b/cli/src/commands/sync/sync.ts index eb45abf1a9..6cd3545c5d 100644 --- a/cli/src/commands/sync/sync.ts +++ b/cli/src/commands/sync/sync.ts @@ -18,6 +18,7 @@ import { GlobalOptions, parseFromPath, pushObj, + removeType, showConflict, showDiff, extractNativeTriggerInfo, @@ -120,6 +121,125 @@ import { let branchDeprecationWarned = false; +// Map a ws_specific `item_kind` returned by the backend to its corresponding +// SpecificItemsConfig array. The backend currently only emits `resource` and +// `variable`, but the mapping rule (kind → its plural — `_trigger` kinds all +// fold into `triggers`) is generic so a future kind doesn't require a change +// here. Returns null for kinds we have no place to store (e.g. `settings`, +// which is a single boolean, not a list). +function configKeyForItemKind( + kind: string, +): keyof Omit | null { + switch (kind) { + case "resource": + return "resources"; + case "variable": + return "variables"; + // case "schedule": + // return "schedules"; + // default: + // return kind.endsWith("_trigger") ? "triggers" : null; + } + return null +} + +// Fetch ws_specific items from the server and merge their paths into specificItems. +// Each ws_specific entry (item_kind + path) is appended as an exact file-path pattern +// to the corresponding array in the config, so the existing pattern-matching logic picks them up. +// Returns the merged config plus the raw server list (or null if the server list could +// not be fetched) — callers performing push-side reconciliation need to know which +// (kind, path) pairs are *not yet* ws_specific on the server. +async function mergeWsSpecificFromServer( + workspaceId: string, + specificItems: SpecificItemsConfig | undefined, +): Promise<{ + merged: SpecificItemsConfig | undefined; + serverItems: Array<{ item_kind: string; path: string }> | null; +}> { + let wsSpecificItems: Array<{ item_kind: string; path: string }>; + try { + wsSpecificItems = await wmill.listWsSpecific({ workspace: workspaceId }); + } catch (err) { + // 404 = endpoint not present on an older server: expected, log at debug. + // Anything else (401/403/network) is a real failure that produces an + // incomplete sync — surface it so the user notices. + const isApiError = err && typeof err === "object" && + "name" in err && (err as { name: unknown }).name === "ApiError"; + const status = isApiError ? (err as { status?: number }).status : undefined; + if (status === 404) { + log.debug("listWsSpecific endpoint not available on server, skipping"); + } else { + const msg = err instanceof Error ? err.message : String(err); + log.warn( + `Could not fetch ws_specific items from server (${status ?? "no status"}): ${msg}. ` + + `Sync will proceed without server-side ws_specific items.`, + ); + } + return { merged: specificItems, serverItems: null }; + } + + if (wsSpecificItems.length === 0) { + return { merged: specificItems, serverItems: wsSpecificItems }; + } + + const merged: SpecificItemsConfig = specificItems ? { ...specificItems } : {}; + + for (const item of wsSpecificItems) { + const configKey = configKeyForItemKind(item.item_kind); + if (!configKey) continue; + if (!merged[configKey]) { + merged[configKey] = []; + } + // Patterns are .yaml regardless of opts.json — isSpecificItem normalizes + // .json file paths to .yaml before matching, so a single .yaml pattern + // covers both extensions for the same item. + merged[configKey]!.push(`${item.path}.${item.item_kind}.yaml`); + } + + return { merged, serverItems: wsSpecificItems }; +} + +// Compute (kind, serverPath) pairs that are flagged ws_specific by the local +// config (specificItems) but are *not* marked ws_specific on the server. These +// represent flag-only changes that the file-content diff misses (because the +// flag isn't part of the YAML body), so push needs to handle them explicitly. +// +// Generic over item kind (uses getTypeStrFromPath + removeType so .yaml/.json +// work the same), gated by configKeyForItemKind so only kinds the backend can +// mark ws_specific make it through. As of writing the backend only emits +// `resource` and `variable`, but the gating handles future kinds without +// touching this function. +export function computeWsSpecificFlagOnlyPushes( + localMap: Record, + localSpecificItems: SpecificItemsConfig | undefined, + serverItems: Array<{ item_kind: string; path: string }> | null, +): Array<{ kind: string; serverPath: string; filePath: string }> { + if (!localSpecificItems || serverItems === null) return []; + + const serverSet = new Set( + serverItems.map((i) => `${i.item_kind}:${i.path}`), + ); + + const out: Array<{ kind: string; serverPath: string; filePath: string }> = []; + for (const filePath of Object.keys(localMap)) { + let kind: string; + try { + kind = getTypeStrFromPath(filePath); + } catch { + continue; + } + if (configKeyForItemKind(kind) === null) continue; + + if (!isSpecificItem(filePath, localSpecificItems)) continue; + + const serverPath = removeType(filePath, kind); + if (serverSet.has(`${kind}:${serverPath}`)) continue; + + out.push({ kind, serverPath, filePath }); + } + return out; +} + // Resolve workspace name from a --branch override (git branch → workspace name). // Falls back to using the branch value as-is (backward compat: old key = branch name). function resolveWsNameFromBranch(opts: SyncOptions, branchName: string): string { @@ -673,6 +793,7 @@ async function pushFilesetParentResource( workspaceId: string, alreadySynced: string[], cachedWsName: string | null, + specificItems?: SpecificItemsConfig, ): Promise { let resourceFilePath: string; try { @@ -691,8 +812,12 @@ async function pushFilesetParentResource( ); let serverPath = resourceFilePath; + let wsSpecific = false; if (cachedWsName && isWorkspaceSpecificFile(resourceFilePath)) { serverPath = fromWorkspaceSpecificPath(resourceFilePath, cachedWsName); + wsSpecific = true; + } else if (specificItems && isSpecificItem(childPath, specificItems)) { + wsSpecific = true; } await pushResource( @@ -701,6 +826,7 @@ async function pushFilesetParentResource( undefined, newObj, resourceFilePath, + wsSpecific ? true : undefined, ); return { status: "pushed", resourceFilePath }; } @@ -1447,8 +1573,20 @@ type Edit = { after: string; codebase?: string; }; +// Flag-only change: the file content matches the server but the local +// specificItems config flags the item as ws_specific while the server +// does not (or vice-versa). Emitted only by computeWsSpecificFlagOnlyPushes +// and routed through its own apply branch — do NOT model this as +// Edit{before === after}, because any future "skip identical edits" +// optimization would silently drop these. +type WsSpecificFlag = { + name: "ws_specific_flag"; + path: string; + kind: string; + wsSpecific: boolean; +}; -type Change = Added | Deleted | Edit; +type Change = Added | Deleted | Edit | WsSpecificFlag; export async function elementsToMap( els: DynFSElement, @@ -2308,10 +2446,15 @@ export async function pull( ); // Extract specific items configuration - const specificItems = getSpecificItemsForCurrentBranch(opts, wsNameForConfig); + let specificItems = getSpecificItemsForCurrentBranch(opts, wsNameForConfig); - // Compute the workspace name for file naming - const wsNameForFiles = wsNameForConfig ? resolveWsNameForFiles(opts, wsNameForConfig) : undefined; + // Compute the workspace name for file naming (default to workspaceId) + let wsNameForFiles = wsNameForConfig ? resolveWsNameForFiles(opts, wsNameForConfig) : workspace.workspaceId; + + // Augment specificItems with server-side ws_specific entries + const localSpecificItems = specificItems; + const wsSpecificMerge = await mergeWsSpecificFromServer(workspace.workspaceId, specificItems); + specificItems = wsSpecificMerge.merged; // Merge CLI flags with resolved settings (CLI flags take precedence only for explicit overrides) opts = mergeCliWithEffectiveOptions(originalCliOpts, effectiveOpts); @@ -2386,6 +2529,27 @@ export async function pull( `remote (${workspace.name}) -> local: ${changes.length} changes to apply`, ); + // Warn about items the server marks ws_specific that aren't covered by + // wmill.yaml's specificItems patterns — but only for items affected by + // this pull (changes list), so unrelated server-flagged items don't + // spam the log. The merge above already ensures correctness for the + // pull itself; this is a heads-up that the user's config drifts from + // the remote and a future push from another machine without that config + // would push the item as non-ws_specific. + if (wsSpecificMerge.serverItems && wsSpecificMerge.serverItems.length > 0) { + const changedPaths = new Set(changes.map((c) => c.path)); + for (const item of wsSpecificMerge.serverItems) { + const filePath = `${item.path}.${item.item_kind}.yaml`; + if (!changedPaths.has(filePath)) continue; + if (!isSpecificItem(filePath, localSpecificItems)) { + log.warn( + `${item.item_kind} ${item.path} is workspace-specific on the remote ` + + `but not flagged in wmill.yaml's specificItems — consider adding it.`, + ); + } + } + } + // Handle JSON output for dry-run if (opts.dryRun && opts.jsonOutput) { const result = { @@ -2809,6 +2973,15 @@ function prettyChanges( showDiff(change.before, change.after); } } + } else if (change.name === "ws_specific_flag") { + log.info( + colors.cyan( + `~ ${change.kind} ${displayPath} ` + + (change.wsSpecific + ? "(mark as workspace-specific)" + : "(unmark as workspace-specific)"), + ), + ); } } } @@ -2900,10 +3073,20 @@ export async function push( ); // Extract specific items configuration - const specificItems = getSpecificItemsForCurrentBranch(opts, wsNameForConfig); + let specificItems = getSpecificItemsForCurrentBranch(opts, wsNameForConfig); - // Compute the workspace name for file naming - const wsNameForFiles = wsNameForConfig ? resolveWsNameForFiles(opts, wsNameForConfig) : undefined; + // Compute the workspace name for file naming (default to workspaceId) + let wsNameForFiles = wsNameForConfig ? resolveWsNameForFiles(opts, wsNameForConfig) : workspace.workspaceId; + + // Keep the pre-merge specificItems so we can detect entries that are + // flagged locally but not yet ws_specific on the server (post-merge would + // include both, making the comparison impossible). + const localSpecificItems = specificItems; + + // Augment specificItems with server-side ws_specific entries + const wsSpecificMerge = await mergeWsSpecificFromServer(workspace.workspaceId, specificItems); + specificItems = wsSpecificMerge.merged; + const serverWsSpecificItems = wsSpecificMerge.serverItems; // Merge CLI flags with resolved settings (CLI flags take precedence only for explicit overrides) opts = mergeCliWithEffectiveOptions(originalCliOpts, effectiveOpts); @@ -2996,7 +3179,7 @@ export async function push( ); const local = await FSFSElement(path.join(process.cwd(), ""), codebases, false); - const { changes } = await compareDynFSElement( + const { changes, localMap } = await compareDynFSElement( local, remote, await ignoreF(opts), @@ -3010,6 +3193,29 @@ export async function push( false, // els1 (local) is not the remote source ); + // Detect resources/variables that the local config flags as ws_specific + // but that aren't ws_specific on the server. The file-content diff misses + // these because ws_specific isn't part of the YAML body — emit a dedicated + // "ws_specific_flag" change so the apply loop can call updateResource / + // updateVariable with just the metadata flag (no content payload). When the + // file *also* has a content change, the existing "edited" change handles + // both (push{Resource,Variable} forwards the wsSpecific arg), so we skip + // injection in that case to avoid pushing twice. + const wsSpecificFlagOnly = computeWsSpecificFlagOnlyPushes( + localMap, + localSpecificItems, + serverWsSpecificItems, + ); + for (const item of wsSpecificFlagOnly) { + if (changes.some((c) => c.path === item.filePath)) continue; + changes.push({ + name: "ws_specific_flag", + path: item.filePath, + kind: item.kind, + wsSpecific: true, + }); + } + const rawWorkspaceDependencies = await getRawWorkspaceDependencies(true); const tracker: ChangeTracker = await buildTracker(changes); @@ -3399,8 +3605,11 @@ export async function push( userEmail: user.email, }; + // ws_specific_flag changes have no content payload, so they don't + // affect permissioned_as resolution — filter them out before the + // pre-check (which expects only added/edited/deleted). await preCheckPermissionedAs( - changes, + changes.filter((c) => c.name !== "ws_specific_flag"), user.email, userIsAdminOrDeployer, opts.acceptOverridingPermissionedAsWithSelf ?? false, @@ -3591,12 +3800,16 @@ export async function push( // This ensures workspace-specific files are stored with their base names in the workspace let serverPath = resourceFilePath; const currentBranch = cachedWsNameForPush; + let isFileResWsSpecific = false; if (currentBranch && isWorkspaceSpecificFile(resourceFilePath)) { serverPath = fromWorkspaceSpecificPath( resourceFilePath, currentBranch, ); + isFileResWsSpecific = true; + } else if (specificItems && isSpecificItem(change.path, specificItems)) { + isFileResWsSpecific = true; } await pushResource( @@ -3605,6 +3818,7 @@ export async function push( undefined, newObj, resourceFilePath, + isFileResWsSpecific ? true : undefined, ); if (stateTarget) { await writeFile(stateTarget, change.after, "utf-8"); @@ -3618,6 +3832,7 @@ export async function push( workspace.workspaceId, alreadySynced, cachedWsNameForPush, + specificItems, ); if (result.status === "parent-missing") { throw new Error( @@ -3637,7 +3852,8 @@ export async function push( // Check if this is a branch-specific item and get the original workspace-specific path let originalWorkspaceSpecificPath: string | undefined; - if (specificItems && isSpecificItem(change.path, specificItems)) { + const isWsSpecific = specificItems && isSpecificItem(change.path, specificItems); + if (isWsSpecific) { originalWorkspaceSpecificPath = getWorkspaceSpecificPath( change.path, specificItems, @@ -3655,6 +3871,7 @@ export async function push( opts.message, originalWorkspaceSpecificPath, permissionedAsContext, + isWsSpecific ? true : undefined, ); if (stateTarget) { @@ -3670,6 +3887,7 @@ export async function push( workspace.workspaceId, alreadySynced, cachedWsNameForPush, + specificItems, ); continue; } @@ -3716,7 +3934,8 @@ export async function push( // Determine the actual local file path for this change // For branch-specific items, we read from workspace-specific files but push to base server paths let localFilePath = change.path; - if (specificItems && isSpecificItem(change.path, specificItems)) { + const isAddedWsSpecific = specificItems && isSpecificItem(change.path, specificItems); + if (isAddedWsSpecific) { const workspaceSpecificPath = getWorkspaceSpecificPath( change.path, specificItems, @@ -3737,6 +3956,7 @@ export async function push( opts.message, localFilePath, // Pass the actual local file path permissionedAsContext, + isAddedWsSpecific ? true : undefined, ); if (stateTarget) { @@ -3768,6 +3988,7 @@ export async function push( workspace.workspaceId, alreadySynced, cachedWsNameForPush, + specificItems, ); continue; } @@ -4103,6 +4324,25 @@ export async function push( // state target may not exist already } } + } else if (change.name === "ws_specific_flag") { + const target = change.path.replaceAll(SEP, "/"); + if (change.kind === "resource") { + await wmill.updateResource({ + workspace: workspace.workspaceId, + path: removeType(target, "resource"), + requestBody: { ws_specific: change.wsSpecific }, + }); + } else if (change.kind === "variable") { + await wmill.updateVariable({ + workspace: workspace.workspaceId, + path: removeType(target, "variable"), + requestBody: { ws_specific: change.wsSpecific }, + }); + } else { + log.warn( + `ws_specific_flag change for unsupported kind '${change.kind}' at ${change.path} — skipping`, + ); + } } } })(); diff --git a/cli/src/commands/variable/variable.ts b/cli/src/commands/variable/variable.ts index d902b8f79f..8d95db5b2b 100644 --- a/cli/src/commands/variable/variable.ts +++ b/cli/src/commands/variable/variable.ts @@ -104,7 +104,8 @@ export async function pushVariable( remotePath: string, variable: VariableFile | ListableVariable | undefined, localVariable: VariableFile, - plainSecrets: boolean + plainSecrets: boolean, + wsSpecific?: boolean, ): Promise { remotePath = removeType(remotePath, "variable"); log.debug(`Processing local variable ${remotePath}`); @@ -137,6 +138,7 @@ export async function pushVariable( ...localVariable, is_secret: localVariable.is_secret && !variable.is_secret ? true : undefined, + ...(wsSpecific !== undefined ? { ws_specific: wsSpecific } : {}), }, }); } else { @@ -147,6 +149,7 @@ export async function pushVariable( requestBody: { path: remotePath.replaceAll(SEP, "/"), ...localVariable, + ...(wsSpecific !== undefined ? { ws_specific: wsSpecific } : {}), }, }); } diff --git a/cli/src/core/specific_items.ts b/cli/src/core/specific_items.ts index fbdffad153..6e64381876 100644 --- a/cli/src/core/specific_items.ts +++ b/cli/src/core/specific_items.ts @@ -29,44 +29,46 @@ export interface SpecificItemsConfig { settings?: boolean; } -// Define all workspace-specific file types (computed lazily) -function getWorkspaceSpecificTypes() { - return { - variable: '.variable.yaml', - resource: '.resource.yaml', - schedule: '.schedule.yaml', - // Generate trigger patterns from the list - ...Object.fromEntries( - TRIGGER_TYPES.map(t => [`${t}_trigger`, `.${t}_trigger.yaml`]) - ) - } as const; +// Define all workspace-specific file type *kinds* (extension-less). The +// actual on-disk file is `.yaml` or `.json` depending on +// opts.json — every helper below treats both as equivalent. +function getWorkspaceSpecificTypeKinds(): string[] { + return [ + 'variable', + 'resource', + 'schedule', + ...TRIGGER_TYPES.map(t => `${t}_trigger`), + ]; } /** * Check if a path ends with any trigger type */ function isTriggerFile(path: string): boolean { - return TRIGGER_TYPES.some(type => path.endsWith(`.${type}_trigger.yaml`)); + return TRIGGER_TYPES.some(type => + path.endsWith(`.${type}_trigger.yaml`) || path.endsWith(`.${type}_trigger.json`) + ); } /** * Check if a path is a schedule file */ function isScheduleFile(path: string): boolean { - return path.endsWith('.schedule.yaml'); + return path.endsWith('.schedule.yaml') || path.endsWith('.schedule.json'); } /** - * Extract the file type suffix from a path + * Extract the file type suffix from a path. Preserves the original `.yaml` + * or `.json` extension so callers like toWorkspaceSpecificPath round-trip + * correctly. */ function getFileTypeSuffix(path: string): string | null { - for (const [_, suffix] of Object.entries(getWorkspaceSpecificTypes())) { - if (path.endsWith(suffix)) { - return suffix; - } + for (const kind of getWorkspaceSpecificTypeKinds()) { + if (path.endsWith(`.${kind}.yaml`)) return `.${kind}.yaml`; + if (path.endsWith(`.${kind}.json`)) return `.${kind}.json`; } - const resourceFileMatch = path.match(/(\\.resource\\.file\\..+)$/); + const resourceFileMatch = path.match(/(\.resource\.file\..+)$/); if (resourceFileMatch) { return resourceFileMatch[1]; } @@ -75,9 +77,9 @@ function getFileTypeSuffix(path: string): string | null { } /** - * Build regex pattern for all supported yaml file types + * Build regex pattern for all supported item-kind file types */ -function buildYamlTypePattern(): string { +function buildItemTypePattern(): string { const basicTypes = ['variable', 'resource', 'schedule']; const triggerTypes = TRIGGER_TYPES.map(t => `${t}_trigger`); return `((${basicTypes.join('|')})|(${triggerTypes.join('|')}))`; @@ -170,6 +172,18 @@ function matchesPatterns(path: string, patterns: string[]): boolean { return patterns.some(pattern => minimatch(path, pattern)); } +/** + * Normalize a `.json` extension to `.yaml` for pattern matching. Patterns in + * wmill.yaml (and those appended by mergeWsSpecificFromServer) are expressed + * with `.yaml` regardless of opts.json, so a `.json` local file would never + * match without this. The rest of `isSpecificItem`/`isItemTypeConfigured` + * checks `endsWith('.X.yaml')`, so normalizing once at the entry point keeps + * both the helpers and the patterns single-extension. + */ +function normalizeJsonToYaml(path: string): string { + return path.endsWith('.json') ? path.slice(0, -'.json'.length) + '.yaml' : path; +} + /** * Check if the item type for a given path is configured in specificItems. * This checks if the TYPE is configured, not whether it matches the pattern. @@ -180,6 +194,8 @@ export function isItemTypeConfigured(path: string, specificItems: SpecificItemsC return false; } + path = normalizeJsonToYaml(path); + if (path.endsWith('.variable.yaml')) { return specificItems.variables !== undefined; } @@ -219,6 +235,8 @@ export function isSpecificItem(path: string, specificItems: SpecificItemsConfig return false; } + path = normalizeJsonToYaml(path); + // Determine the item type from the file path if (path.endsWith('.variable.yaml')) { return specificItems.variables ? matchesPatterns(path, specificItems.variables) : false; @@ -286,15 +304,16 @@ export function toWorkspaceSpecificPath(basePath: string, workspaceName: string) console.warn(`Warning: Workspace name "${workspaceName}" contains filesystem-unsafe characters (/ \\ : * ? " < > | .) and was sanitized to "${sanitizedName}". This may cause collisions with other similarly named branches.`); } - // Check for folder meta file pattern: folder.meta.yaml -> folder.workspaceName.meta.yaml - if (basePath.endsWith('/folder.meta.yaml')) { - const pathWithoutMeta = basePath.substring(0, basePath.length - '/folder.meta.yaml'.length); - return `${pathWithoutMeta}/folder.${sanitizedName}.meta.yaml`; + // Check for folder meta file pattern: folder.meta.{yaml,json} -> folder.workspaceName.meta.{yaml,json} + const folderMetaMatch = basePath.match(/^(.*)\/folder\.meta\.(yaml|json)$/); + if (folderMetaMatch) { + return `${folderMetaMatch[1]}/folder.${sanitizedName}.meta.${folderMetaMatch[2]}`; } - // Check for settings.yaml: settings.yaml -> settings.workspaceName.yaml - if (basePath === 'settings.yaml') { - return `settings.${sanitizedName}.yaml`; + // Check for settings.{yaml,json}: settings.{ext} -> settings.workspaceName.{ext} + const settingsMatch = basePath.match(/^settings\.(yaml|json)$/); + if (settingsMatch) { + return `settings.${sanitizedName}.${settingsMatch[1]}`; } // Check for resource file pattern (e.g., .resource.file.ini) @@ -327,16 +346,18 @@ export function fromWorkspaceSpecificPath(workspaceSpecificPath: string, workspa const sanitizedName = workspaceName.replace(/[\/\\:*?"<>|.]/g, '_'); const escapedName = sanitizedName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - // Check for folder meta file pattern: /folder.workspaceName.meta.yaml -> /folder.meta.yaml - const folderPattern = new RegExp(`/folder\\.${escapedName}\\.meta\\.yaml$`); - if (folderPattern.test(workspaceSpecificPath)) { - return workspaceSpecificPath.replace(folderPattern, '/folder.meta.yaml'); + // Check for folder meta file pattern: /folder.workspaceName.meta.{yaml,json} -> /folder.meta.{ext} + const folderPattern = new RegExp(`/folder\\.${escapedName}\\.meta\\.(yaml|json)$`); + const folderMatch = workspaceSpecificPath.match(folderPattern); + if (folderMatch) { + return workspaceSpecificPath.replace(folderPattern, `/folder.meta.${folderMatch[1]}`); } - // Check for settings file pattern: settings.workspaceName.yaml -> settings.yaml - const settingsPattern = new RegExp(`^settings\\.${escapedName}\\.yaml$`); - if (settingsPattern.test(workspaceSpecificPath)) { - return 'settings.yaml'; + // Check for settings file pattern: settings.workspaceName.{yaml,json} -> settings.{ext} + const settingsPattern = new RegExp(`^settings\\.${escapedName}\\.(yaml|json)$`); + const settingsMatch = workspaceSpecificPath.match(settingsPattern); + if (settingsMatch) { + return `settings.${settingsMatch[1]}`; } // Check for resource file pattern @@ -352,14 +373,14 @@ export function fromWorkspaceSpecificPath(workspaceSpecificPath: string, workspa return `${pathWithoutBranchAndExtension}${extension}`; } - const yamlPattern = new RegExp(`\\.${escapedName}(\\.${buildYamlTypePattern()}\\.yaml)$`); - const yamlMatch = workspaceSpecificPath.match(yamlPattern); + const typedPattern = new RegExp(`\\.${escapedName}(\\.${buildItemTypePattern()}\\.(yaml|json))$`); + const typedMatch = workspaceSpecificPath.match(typedPattern); - if (!yamlMatch) { + if (!typedMatch) { return workspaceSpecificPath; // Return unchanged if not a workspace-specific path } - const extension = yamlMatch[1]; + const extension = typedMatch[1]; const pathWithoutBranchAndExtension = workspaceSpecificPath.substring( 0, workspaceSpecificPath.length - `.${sanitizedName}${extension}`.length @@ -427,10 +448,10 @@ export function isCurrentWorkspaceFile(path: string, workspaceNameOverride?: str let pattern = workspacePatternCache.get(currentWorkspace); if (!pattern) { pattern = new RegExp( - `\\.${escapedName}\\.${buildYamlTypePattern()}\\.yaml$|` + + `\\.${escapedName}\\.${buildItemTypePattern()}\\.(yaml|json)$|` + `\\.${escapedName}\\.resource\\.file\\..+$|` + - `/folder\\.${escapedName}\\.meta\\.yaml$|` + - `^settings\\.${escapedName}\\.yaml$` + `/folder\\.${escapedName}\\.meta\\.(yaml|json)$|` + + `^settings\\.${escapedName}\\.(yaml|json)$` ); workspacePatternCache.set(currentWorkspace, pattern); } @@ -443,11 +464,11 @@ export function isCurrentWorkspaceFile(path: string, workspaceNameOverride?: str * Used to identify and skip files from other branches during sync operations */ export function isWorkspaceSpecificFile(path: string): boolean { - const yamlTypePattern = buildYamlTypePattern(); + const typePattern = buildItemTypePattern(); return new RegExp( - `\\.[^.]+\\.${yamlTypePattern}\\.yaml$|` + + `\\.[^.]+\\.${typePattern}\\.(yaml|json)$|` + `\\.[^.]+\\.resource\\.file\\..+$|` + - `/folder\\.[^.]+\\.meta\\.yaml$|` + - `^settings\\.[^.]+\\.yaml$` + `/folder\\.[^.]+\\.meta\\.(yaml|json)$|` + + `^settings\\.[^.]+\\.(yaml|json)$` ).test(path); } diff --git a/cli/src/types.ts b/cli/src/types.ts index a854b16419..e0d45adb66 100644 --- a/cli/src/types.ts +++ b/cli/src/types.ts @@ -154,7 +154,8 @@ export async function pushObj( alreadySynced: string[], message?: string, originalLocalPath?: string, - permissionedAsContext?: PermissionedAsContext + permissionedAsContext?: PermissionedAsContext, + wsSpecific?: boolean, ) { const typeEnding = getTypeStrFromPath(p); @@ -173,7 +174,7 @@ export async function pushObj( } else if (typeEnding === "folder") { await pushFolder(workspace, p, befObj, newObj); } else if (typeEnding === "variable") { - await pushVariable(workspace, p, befObj, newObj, plainSecrets); + await pushVariable(workspace, p, befObj, newObj, plainSecrets, wsSpecific); } else if (typeEnding === "flow") { const flowName = extractResourceName(p, "flow"); if (!flowName) { @@ -183,7 +184,7 @@ export async function pushObj( } else if (typeEnding === "resource") { if (!alreadySynced.includes(p)) { alreadySynced.push(p); - await pushResource(workspace, p, befObj, newObj, originalLocalPath || p); + await pushResource(workspace, p, befObj, newObj, originalLocalPath || p, wsSpecific); } } else if (typeEnding === "resource-type") { await pushResourceType(workspace, p, befObj, newObj); diff --git a/cli/test/specific_items_unit.test.ts b/cli/test/specific_items_unit.test.ts index ae94603dbc..38437fe3ce 100644 --- a/cli/test/specific_items_unit.test.ts +++ b/cli/test/specific_items_unit.test.ts @@ -855,3 +855,133 @@ test("mixed config: variables and folders configured - resources and triggers us expect(isItemTypeConfigured("f/hook.http_trigger.yaml", config)).toEqual(false); expect(isItemTypeConfigured("settings.yaml", config)).toEqual(false); }); + +// ============================================================================= +// JSON workspace-specific path conversion +// (every helper that previously hard-coded .yaml must also handle .json) +// ============================================================================= + +test("toWorkspaceSpecificPath: converts .json variable path", () => { + expect(toWorkspaceSpecificPath("f/test.variable.json", "main")).toEqual( + "f/test.main.variable.json", + ); +}); + +test("toWorkspaceSpecificPath: converts .json resource path", () => { + expect(toWorkspaceSpecificPath("u/admin/db.resource.json", "develop")).toEqual( + "u/admin/db.develop.resource.json", + ); +}); + +test("toWorkspaceSpecificPath: converts .json trigger path", () => { + expect( + toWorkspaceSpecificPath("f/my_trigger.http_trigger.json", "feature-x"), + ).toEqual("f/my_trigger.feature-x.http_trigger.json"); +}); + +test("toWorkspaceSpecificPath: converts .json schedule path", () => { + expect(toWorkspaceSpecificPath("f/cron.schedule.json", "main")).toEqual( + "f/cron.main.schedule.json", + ); +}); + +test("toWorkspaceSpecificPath: converts folder.meta.json", () => { + expect(toWorkspaceSpecificPath("f/myfolder/folder.meta.json", "dev")).toEqual( + "f/myfolder/folder.dev.meta.json", + ); +}); + +test("toWorkspaceSpecificPath: converts settings.json", () => { + expect(toWorkspaceSpecificPath("settings.json", "prod")).toEqual( + "settings.prod.json", + ); +}); + +test("fromWorkspaceSpecificPath: round-trips .json variable", () => { + expect(fromWorkspaceSpecificPath("f/test.main.variable.json", "main")).toEqual( + "f/test.variable.json", + ); +}); + +test("fromWorkspaceSpecificPath: round-trips .json resource", () => { + expect( + fromWorkspaceSpecificPath("u/admin/db.develop.resource.json", "develop"), + ).toEqual("u/admin/db.resource.json"); +}); + +test("fromWorkspaceSpecificPath: round-trips .json trigger", () => { + expect( + fromWorkspaceSpecificPath( + "f/my_trigger.feature-x.http_trigger.json", + "feature-x", + ), + ).toEqual("f/my_trigger.http_trigger.json"); +}); + +test("fromWorkspaceSpecificPath: round-trips folder..meta.json", () => { + expect( + fromWorkspaceSpecificPath("f/myfolder/folder.dev.meta.json", "dev"), + ).toEqual("f/myfolder/folder.meta.json"); +}); + +test("fromWorkspaceSpecificPath: round-trips settings..json", () => { + expect(fromWorkspaceSpecificPath("settings.prod.json", "prod")).toEqual( + "settings.json", + ); +}); + +test("isWorkspaceSpecificFile: detects .json branch-specific files", () => { + expect(isWorkspaceSpecificFile("f/test.main.variable.json")).toEqual(true); + expect(isWorkspaceSpecificFile("u/admin/db.develop.resource.json")).toEqual( + true, + ); + expect( + isWorkspaceSpecificFile("f/my_trigger.feature.http_trigger.json"), + ).toEqual(true); + expect(isWorkspaceSpecificFile("f/myfolder/folder.dev.meta.json")).toEqual( + true, + ); + expect(isWorkspaceSpecificFile("settings.prod.json")).toEqual(true); +}); + +test("isWorkspaceSpecificFile: rejects base .json files", () => { + expect(isWorkspaceSpecificFile("f/test.variable.json")).toEqual(false); + expect(isWorkspaceSpecificFile("settings.json")).toEqual(false); + expect(isWorkspaceSpecificFile("f/myfolder/folder.meta.json")).toEqual(false); +}); + +test("isCurrentWorkspaceFile: detects .json branch-specific files for given workspace", () => { + expect( + isCurrentWorkspaceFile("f/test.main.variable.json", "main"), + ).toEqual(true); + expect( + isCurrentWorkspaceFile("f/test.dev.variable.json", "main"), + ).toEqual(false); + expect( + isCurrentWorkspaceFile("settings.prod.json", "prod"), + ).toEqual(true); +}); + +test("isSpecificItem: matches .json paths against .yaml-only patterns", () => { + // Patterns from wmill.yaml are conventionally written with .yaml; the + // helper should normalize a local .json file before matching so users + // configured for opts.json still get specificItems coverage. + const config: SpecificItemsConfig = { + variables: ["f/*.variable.yaml"], + resources: ["u/admin/db.resource.yaml"], + }; + expect(isSpecificItem("f/test.variable.json", config)).toEqual(true); + expect(isSpecificItem("u/admin/db.resource.json", config)).toEqual(true); + expect(isSpecificItem("u/admin/other.resource.json", config)).toEqual(false); +}); + +test("isItemTypeConfigured: works for .json folder.meta and settings", () => { + const config: SpecificItemsConfig = { + folders: ["f/env_*"], + settings: true, + }; + expect( + isItemTypeConfigured("f/env_staging/folder.meta.json", config), + ).toEqual(true); + expect(isItemTypeConfigured("settings.json", config)).toEqual(true); +}); diff --git a/cli/test/ws_specific_flag_only_unit.test.ts b/cli/test/ws_specific_flag_only_unit.test.ts new file mode 100644 index 0000000000..8affcb7cfe --- /dev/null +++ b/cli/test/ws_specific_flag_only_unit.test.ts @@ -0,0 +1,117 @@ +import { expect, test } from "bun:test"; + +import { computeWsSpecificFlagOnlyPushes } from "../src/commands/sync/sync.ts"; +import type { SpecificItemsConfig } from "../src/core/specific_items.ts"; + +// ============================================================================= +// computeWsSpecificFlagOnlyPushes +// Verifies which (kind, path) pairs are emitted as ws_specific_flag changes +// during sync push when local config flags an item ws_specific but the server +// does not yet. +// ============================================================================= + +test("emits a flag-only push for a resource missing from the server list", () => { + const localMap: Record = { + "u/admin/db.resource.yaml": "value: foo", + }; + const local: SpecificItemsConfig = { resources: ["u/admin/db.resource.yaml"] }; + const server: Array<{ item_kind: string; path: string }> = []; + + const out = computeWsSpecificFlagOnlyPushes(localMap, local, server); + expect(out).toEqual([ + { + kind: "resource", + serverPath: "u/admin/db", + filePath: "u/admin/db.resource.yaml", + }, + ]); +}); + +test("emits a flag-only push for a variable missing from the server list", () => { + const localMap: Record = { + "u/admin/v.variable.yaml": "value: bar", + }; + const local: SpecificItemsConfig = { variables: ["u/admin/v.variable.yaml"] }; + + const out = computeWsSpecificFlagOnlyPushes(localMap, local, []); + expect(out).toEqual([ + { + kind: "variable", + serverPath: "u/admin/v", + filePath: "u/admin/v.variable.yaml", + }, + ]); +}); + +test("does not emit a flag-only push when the server already has the row", () => { + const localMap: Record = { + "u/admin/db.resource.yaml": "value: foo", + }; + const local: SpecificItemsConfig = { resources: ["u/admin/db.resource.yaml"] }; + const server = [{ item_kind: "resource", path: "u/admin/db" }]; + + expect(computeWsSpecificFlagOnlyPushes(localMap, local, server)).toEqual([]); +}); + +test("does not emit anything for schedules in specificItems", () => { + // The backend's list_ws_specific_versions endpoint only handles + // resource + variable today; schedules have their ws_specific concept + // expressed through workspace-specific files but the server has no + // ws_specific row for them, so flag-only pushes don't apply. + const localMap: Record = { + "u/admin/cron.schedule.yaml": "schedule: '* * * * *'", + }; + const local: SpecificItemsConfig = { schedules: ["u/admin/cron.schedule.yaml"] }; + + expect(computeWsSpecificFlagOnlyPushes(localMap, local, [])).toEqual([]); +}); + +test("does not emit anything for triggers in specificItems", () => { + // Same reason as schedules — the backend doesn't track ws_specific for + // any *_trigger kind. + const localMap: Record = { + "u/admin/hook.http_trigger.yaml": "method: GET", + }; + const local: SpecificItemsConfig = { triggers: ["u/admin/hook.http_trigger.yaml"] }; + + expect(computeWsSpecificFlagOnlyPushes(localMap, local, [])).toEqual([]); +}); + +test("returns empty when serverItems is null (server doesn't expose the endpoint)", () => { + const localMap: Record = { + "u/admin/db.resource.yaml": "value: foo", + }; + const local: SpecificItemsConfig = { resources: ["u/admin/db.resource.yaml"] }; + + expect(computeWsSpecificFlagOnlyPushes(localMap, local, null)).toEqual([]); +}); + +test("handles .json resource files (extension is preserved on filePath)", () => { + const localMap: Record = { + "u/admin/db.resource.json": "{\"value\":\"foo\"}", + }; + const local: SpecificItemsConfig = { + // Patterns are conventionally .yaml; isSpecificItem normalizes .json -> .yaml + resources: ["u/admin/db.resource.yaml"], + }; + + const out = computeWsSpecificFlagOnlyPushes(localMap, local, []); + expect(out).toEqual([ + { + kind: "resource", + serverPath: "u/admin/db", + filePath: "u/admin/db.resource.json", + }, + ]); +}); + +test("ignores files not in specificItems", () => { + const localMap: Record = { + "u/admin/db.resource.yaml": "value: foo", + "u/admin/other.resource.yaml": "value: bar", + }; + const local: SpecificItemsConfig = { resources: ["u/admin/db.resource.yaml"] }; + + const out = computeWsSpecificFlagOnlyPushes(localMap, local, []); + expect(out.map((p) => p.filePath)).toEqual(["u/admin/db.resource.yaml"]); +}); diff --git a/frontend/src/lib/components/AppConnectInner.svelte b/frontend/src/lib/components/AppConnectInner.svelte index dd9f767c8e..4f0a5a76b2 100644 --- a/frontend/src/lib/components/AppConnectInner.svelte +++ b/frontend/src/lib/components/AppConnectInner.svelte @@ -7,6 +7,7 @@ import { OauthService, ResourceService, + WorkspaceService, VariableService, type TokenResponse, type ResourceType @@ -33,6 +34,7 @@ import TextInput from './text_input/TextInput.svelte' import { sameTopDomainOrigin } from '$lib/cookies' import SyncResourceTypes from './SyncResourceTypes.svelte' + import Label from './Label.svelte' interface Props { step?: number @@ -111,6 +113,8 @@ let path: string = $state('') let description = $state('') let labels: string[] | undefined = $state(undefined) + let wsSpecific = $state(false) + let deployTo: string | undefined = $state(undefined) /** * Client credentials OAuth flow support @@ -147,6 +151,7 @@ value = '' description = '' labels = undefined + wsSpecific = false resourceType = rt ?? '' valueToken = undefined @@ -290,6 +295,17 @@ window.removeEventListener('storage', handleStorageEvent) }) + $effect(() => { + if (!effectiveWorkspace) { + deployTo = undefined + return + } + + WorkspaceService.getDeployTo({ workspace: effectiveWorkspace }).then((x) => { + deployTo = x.deploy_to + }) + }) + function processPopupData(data) { console.log('Processing oauth popup data') if (data.type === 'error') { @@ -518,7 +534,8 @@ ? `OAuth token for ${resourceType}` : description, is_oauth: true, - account: account + account: account, + ws_specific: wsSpecific } }) resourceValue['token'] = `$var:${path}` @@ -536,7 +553,8 @@ value: v, is_secret: true, description: emptyString(description) ? `Token for ${resourceType}` : description, - is_oauth: false + is_oauth: false, + ws_specific: wsSpecific } }) resourceValue[secretField] = `$var:${path}` @@ -557,7 +575,8 @@ description: emptyString(description) ? `${secretField} for ${resourceType}` : description, - is_oauth: false + is_oauth: false, + ws_specific: wsSpecific } }) resourceValue[secretField] = `$var:${varPath}` @@ -572,7 +591,8 @@ path, value: resourceValue, description, - labels + labels, + ws_specific: wsSpecific } }) dispatch('refresh', path) @@ -739,6 +759,14 @@ kind="resource" /> + {#if deployTo} + + {/if} {#if apiTokenApps[resourceType]}

Instructions

@@ -946,6 +974,14 @@ kind="resource" /> + {#if deployTo} + + {/if} {#if apiTokenApps[resourceType] || !manual}
  • diff --git a/frontend/src/lib/components/ResourceEditor.svelte b/frontend/src/lib/components/ResourceEditor.svelte index 85c88ee6ff..b023097bfe 100644 --- a/frontend/src/lib/components/ResourceEditor.svelte +++ b/frontend/src/lib/components/ResourceEditor.svelte @@ -1,32 +1,17 @@
    - {#if !hidePath} -
    - {#if !can_write} -
    - - You only have read access to this resource and cannot edit it - -
    - {/if} - 0} + + You are going to edit the value in: {otherDirty.join(', ')} + + {/if} + + {#if current} + {#key current} + -
    - {/if} - - - {#if !emptyString(resourceTypeInfo?.description)} -
    -

    {resourceTypeInfo?.name} description

    -
    - -
    -
    - {/if} - -
    -

    Resource description - {#if can_write} -

    - {#if can_write && editDescription} -
    -
    GH Markdown
    - -
    - {:else if description == undefined || description == ''} -
    No description provided
    - {:else} -
    - -
    - {/if} -
    - -
    -
    - switchTab(e.detail)} - options={{ - right: 'As JSON' - }} - /> - resourceTypeResource.refetch()} /> - {#if resourceToEdit?.resource_type === 'nats' || resourceToEdit?.resource_type === 'kafka'} - - {:else} - - {/if} - {#if resource_type === 'git_repository' && $workspaceStore && ($userStore?.is_admin || $userStore?.is_super_admin)} - { - args = newArgs - // Update rawCode if in JSON view mode - if (viewJsonSchema) { - rawCode = JSON.stringify(args, null, 2) - } - }} - onDescriptionUpdate={(newDescription) => (description = newDescription)} - /> - {/if} -
    - -
    - {#if loadingSchema} - - {:else if !viewJsonSchema && resourceTypeInfo?.is_fileset} -
    -
    Fileset
    - -
    - - {:else if !viewJsonSchema && resourceSchema && resourceSchema?.properties} - {#if resourceTypeInfo?.format_extension} -
    - File content ({resourceTypeInfo.format_extension}) -
    -
    - -
    - {:else} - - {/if} - {:else if !can_write} - - {:else} - {#if !viewJsonSchema} -
    -

    - Resource type '{resource_type}' not found in your workspace -

    - -

    Define the value in JSON directly

    -
    - {/if} - - {#if !emptyString(jsonError)}{jsonError}{:else}
    {/if} -
    - -
    - {/if} -
    -
    + {/key} + {/if}
    diff --git a/frontend/src/lib/components/ResourceEditorDrawer.svelte b/frontend/src/lib/components/ResourceEditorDrawer.svelte index 54348e81c1..ed80717a32 100644 --- a/frontend/src/lib/components/ResourceEditorDrawer.svelte +++ b/frontend/src/lib/components/ResourceEditorDrawer.svelte @@ -4,6 +4,8 @@ import DrawerContent from './common/drawer/DrawerContent.svelte' import { Loader2, Save } from 'lucide-svelte' + import WsSpecificVersions from './WsSpecificVersions.svelte' + import { workspaceStore } from '$lib/stores' let { workspace = undefined, @@ -15,14 +17,17 @@ let resource_type: string | undefined = $state(undefined) let defaultValues: Record | undefined = $state(undefined) - let resourceEditor: { editResource: () => void; createResource: () => void } | undefined = - $state(undefined) + let resourceEditor: { save: () => void } | undefined = $state(undefined) let path: string | undefined = $state(undefined) + let selected: string | undefined = $state(undefined) + + let effectiveWorkspace = $derived(workspace ?? $workspaceStore!) export async function initEdit(p: string): Promise { resource_type = undefined path = p + selected = effectiveWorkspace drawer?.openDrawer?.() } @@ -33,13 +38,14 @@ path = undefined resource_type = resourceType defaultValues = nDefaultValues + selected = effectiveWorkspace drawer?.openDrawer?.() } let mode: 'edit' | 'new' = $derived(!path ? 'new' : 'edit') - + {/await} {#snippet actions()} + {#if mode == 'edit' && path && effectiveWorkspace} + + {/if} - {/if} - {/if} - -
    -
    - - {#snippet children({ item })} - - - - {/snippet} - - {#if editorKind == 'plain'} - - {:else if editorKind == 'json'} -
    - {#await import('$lib/components/SimpleEditor.svelte')} - - {:then Module} - - {/await} -
    - {:else if editorKind == 'yaml'} -
    - {#await import('$lib/components/SimpleEditor.svelte')} - - {:then Module} - - {/await} -
    - {/if} -
    -
    - - + {#if current} + {#key current} + + {/key} + {/if} {#snippet actions()} + {#if edit && $workspaceStore} + + {/if} + {/if} + {/if} + +
    +
    + + {#snippet children({ item })} + + + + {/snippet} + + {#if editorKind == 'plain'} + + {:else if editorKind == 'json'} +
    + {#await import('$lib/components/SimpleEditor.svelte')} + + {:then Module} + + {/await} +
    + {:else if editorKind == 'yaml'} +
    + {#await import('$lib/components/SimpleEditor.svelte')} + + {:then Module} + + {/await} +
    + {/if} +
    +
    + + diff --git a/frontend/src/lib/components/WsSpecificVersions.svelte b/frontend/src/lib/components/WsSpecificVersions.svelte new file mode 100644 index 0000000000..ec1e35b853 --- /dev/null +++ b/frontend/src/lib/components/WsSpecificVersions.svelte @@ -0,0 +1,54 @@ + + +{#if versions.length > 1} + + {#snippet children({ item })} + {#each regular as v (v)} + + {/each} + {#if more.length > 0} + ({ label: v, value: v }))} + {item} + bind:selected + /> + {/if} + {/snippet} + +{/if} diff --git a/frontend/src/routes/(root)/(logged)/resources/+page.svelte b/frontend/src/routes/(root)/(logged)/resources/+page.svelte index 285be0acbb..11f357a26f 100644 --- a/frontend/src/routes/(root)/(logged)/resources/+page.svelte +++ b/frontend/src/routes/(root)/(logged)/resources/+page.svelte @@ -989,7 +989,7 @@ {#if filteredItems} - {#each filteredItems as { path, description, resource_type, extra_perms, canWrite, is_oauth, is_linked, account, refresh_error, is_expired, marked, is_refreshed, labels }} + {#each filteredItems as { path, description, resource_type, extra_perms, canWrite, is_oauth, is_linked, account, refresh_error, is_expired, marked, is_refreshed, labels, ws_specific }} @@ -1165,7 +1165,7 @@ resourceEditor?.initEdit?.(path) } }, - ...(isDeployable('resource', path, deployUiSettings) + ...(!ws_specific && isDeployable('resource', path, deployUiSettings) ? [ { displayName: 'Deploy to prod/staging', diff --git a/frontend/src/routes/(root)/(logged)/variables/+page.svelte b/frontend/src/routes/(root)/(logged)/variables/+page.svelte index 8092358509..670a7e7653 100644 --- a/frontend/src/routes/(root)/(logged)/variables/+page.svelte +++ b/frontend/src/routes/(root)/(logged)/variables/+page.svelte @@ -346,7 +346,7 @@ - {#each filteredItems as { path, value, is_secret, description, extra_perms, canWrite, account, is_refreshed, is_expired, refresh_error, is_linked, labels }} + {#each filteredItems as { path, value, is_secret, description, extra_perms, canWrite, account, is_refreshed, is_expired, refresh_error, is_linked, labels, ws_specific }} @@ -512,11 +512,8 @@ }, disabled: !owner || !showCreateButtons }, - ...(isDeployable( - is_secret ? 'secret' : 'variable', - path, - deployUiSettings - ) + ...(!ws_specific && + isDeployable(is_secret ? 'secret' : 'variable', path, deployUiSettings) ? [ { displayName: 'Deploy to prod/staging', diff --git a/rust-client/src/client.rs b/rust-client/src/client.rs index 5cf152f520..fa53774a97 100644 --- a/rust-client/src/client.rs +++ b/rust-client/src/client.rs @@ -242,6 +242,7 @@ impl Windmill { is_oauth: None, expires_at: None, labels: None, + ws_specific: None, }, None, ) @@ -257,6 +258,7 @@ impl Windmill { is_secret: None, description: None, labels: None, + ws_specific: None, }, None, ) @@ -403,6 +405,7 @@ impl Windmill { description: None, resource_type: resource_type.to_owned(), labels: None, + ws_specific: None, }, // Some(true), None, @@ -419,6 +422,7 @@ impl Windmill { value: Some(value), resource_type: Some(resource_type.to_owned()), labels: None, + ws_specific: None, }, ) .await?;