From ca8f0a16c18de3307e1b545afe72140ff1bc604d Mon Sep 17 00:00:00 2001 From: Ruben Fiszel Date: Wed, 28 Oct 2026 00:43:33 +0100 Subject: [PATCH] feat: per-role fork capability and fork role mapping on rulesets (#12054) * feat: fork capability per role and fork role mapping on rulesets Co-Authored-By: Claude Opus 5.5 * fix: keep unset fork for unrestricted roles and apply fork role mapping to fork-owner adds Co-Authored-By: Claude Opus 5.5 * feat: fork_roles feature usage telemetry Co-Authored-By: Claude Opus 5.5 * fix: derive an unset fork in the role editor as the server does Co-Authored-By: Claude Opus 5.5 * chore: update ee-repo-ref to d535972f0b35c6a7f257720816bdaec18465adef This commit updates the EE repository reference after PR #929 was merged in windmill-ee-private. Previous ee-repo-ref: 4a8d59cd1563382fcec85b1367f7e15e09bc13c2 New ee-repo-ref: d535972f0b35c6a7f257720816bdaec18465adef Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: windmill-internal-app[bot] --- ...e5326c5d344a0106e219867acc2447f417160.json | 29 ++ ...3ea7c4da6b1368408c541cc2f9c390a5e840.json} | 7 +- ...13902ed94816142faf15446b4b9f19c3097d7.json | 16 - ...64c4b6a41d96df223b6d3d1d1bc972ab10d42.json | 16 - ...ef2875ef34dd304ff21979095bb7b9304bccd.json | 35 ++ ...5702eba0b0d674b99f952af94487b3e54211c.json | 23 -- ...7182490293e210602f5cc7b42978e3b89109.json} | 7 +- ...df755ed21a35339cae18728d9cb0b6b7ea530.json | 20 ++ ...eda071ba59b874fd56ec3eeb9b054df7eb64.json} | 10 +- ...87f1d9ff80b7298846d45adcf854c9052424d.json | 23 ++ backend/ee-repo-ref.txt | 2 +- ...protection_rule_fork_role_mapping.down.sql | 1 + ...5_protection_rule_fork_role_mapping.up.sql | 2 + backend/summarized_schema.txt | 2 +- .../tests/fork_members.rs | 105 ++++++ .../windmill-api-workspaces/src/workspaces.rs | 316 +++++++++++++----- backend/windmill-api/openapi.yaml | 20 +- .../src/mcp/session/gen/instructions.json | 2 +- .../src/workspace_settings_document.rs | 41 ++- backend/windmill-common/src/roles.rs | 103 +++++- backend/windmill-common/src/workspaces.rs | 23 +- .../commands/protection-rules/converter.ts | 18 +- cli/src/commands/protection-rules/push.ts | 2 + cli/src/commands/protection-rules/types.ts | 2 + cli/src/commands/protection-rules/utils.ts | 4 + cli/src/guidance/skills.gen.ts | 2 +- .../protection_rules_converter_unit.test.ts | 11 + docs/feature-telemetry.md | 4 +- docs/workspace-roles.md | 15 + .../WorkspaceRolesSetting.svelte | 4 +- .../settings/ForkMemberSettings.svelte | 6 +- .../settings/RoleCapabilitiesEditor.svelte | 15 +- .../settings/WorkspaceRoleOverrides.svelte | 22 +- .../CreateWorkspaceInner.svelte | 9 +- .../workspaceSettings/RulesetEditor.svelte | 92 ++++- frontend/src/lib/roleCapabilities.ts | 22 +- frontend/src/lib/utils/editInFork.ts | 11 +- frontend/src/lib/workspaceRoles.test.ts | 17 + frontend/src/lib/workspaceRoles.ts | 35 ++ .../user/(user)/fork_workspace/+page.svelte | 9 +- system_prompts/auto-generated/prompts.ts | 2 +- .../skills/workspace-settings/SKILL.md | 2 +- system_prompts/base/workspace-settings.md | 2 +- 43 files changed, 877 insertions(+), 232 deletions(-) create mode 100644 backend/.sqlx/query-0be5632227efca9d58cfa67bf3ce5326c5d344a0106e219867acc2447f417160.json rename backend/.sqlx/{query-adbe6ce27d46ba0d2d3ce87d875f60c179e5bccc54a36c8d5dcf04fd84445309.json => query-11a315b1b019c4bbd6b6676029943ea7c4da6b1368408c541cc2f9c390a5e840.json} (59%) delete mode 100644 backend/.sqlx/query-42a0ba479ff164cc190c350927e13902ed94816142faf15446b4b9f19c3097d7.json delete mode 100644 backend/.sqlx/query-464286216add454cb64dba810d964c4b6a41d96df223b6d3d1d1bc972ab10d42.json create mode 100644 backend/.sqlx/query-4e8c9610c697c2fb1b08ce78773ef2875ef34dd304ff21979095bb7b9304bccd.json delete mode 100644 backend/.sqlx/query-67d86d334029276d78bf8fa14b25702eba0b0d674b99f952af94487b3e54211c.json rename backend/.sqlx/{query-dbf2982f43577999dec0c488dfb67f56e5bd0e5fbf8da5132f6fc5a282d2b0e1.json => query-8c3b028e01aaf693a69ef0f9644a7182490293e210602f5cc7b42978e3b89109.json} (53%) create mode 100644 backend/.sqlx/query-b80d2fd5368eb0bad94bf529b61df755ed21a35339cae18728d9cb0b6b7ea530.json rename backend/.sqlx/{query-4cfb35e423a75ca2701f03d5a30a7c0778af5e548254f3e0f29004d7f2058eef.json => query-c0b4a4914f738a6a9975bb75d8ffeda071ba59b874fd56ec3eeb9b054df7eb64.json} (68%) create mode 100644 backend/.sqlx/query-ee2a8ae43f2d0a748f1e55185df87f1d9ff80b7298846d45adcf854c9052424d.json create mode 100644 backend/migrations/20261027164425_protection_rule_fork_role_mapping.down.sql create mode 100644 backend/migrations/20261027164425_protection_rule_fork_role_mapping.up.sql diff --git a/backend/.sqlx/query-0be5632227efca9d58cfa67bf3ce5326c5d344a0106e219867acc2447f417160.json b/backend/.sqlx/query-0be5632227efca9d58cfa67bf3ce5326c5d344a0106e219867acc2447f417160.json new file mode 100644 index 0000000000..d98d5ea0cf --- /dev/null +++ b/backend/.sqlx/query-0be5632227efca9d58cfa67bf3ce5326c5d344a0106e219867acc2447f417160.json @@ -0,0 +1,29 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT workspace_role, operator FROM usr WHERE workspace_id = $1 AND email = $2", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "workspace_role", + "type_info": "Varchar" + }, + { + "ordinal": 1, + "name": "operator", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [ + true, + false + ] + }, + "hash": "0be5632227efca9d58cfa67bf3ce5326c5d344a0106e219867acc2447f417160" +} diff --git a/backend/.sqlx/query-adbe6ce27d46ba0d2d3ce87d875f60c179e5bccc54a36c8d5dcf04fd84445309.json b/backend/.sqlx/query-11a315b1b019c4bbd6b6676029943ea7c4da6b1368408c541cc2f9c390a5e840.json similarity index 59% rename from backend/.sqlx/query-adbe6ce27d46ba0d2d3ce87d875f60c179e5bccc54a36c8d5dcf04fd84445309.json rename to backend/.sqlx/query-11a315b1b019c4bbd6b6676029943ea7c4da6b1368408c541cc2f9c390a5e840.json index 04ef8a7d24..e2eda4cd56 100644 --- a/backend/.sqlx/query-adbe6ce27d46ba0d2d3ce87d875f60c179e5bccc54a36c8d5dcf04fd84445309.json +++ b/backend/.sqlx/query-11a315b1b019c4bbd6b6676029943ea7c4da6b1368408c541cc2f9c390a5e840.json @@ -1,6 +1,6 @@ { "db_name": "PostgreSQL", - "query": "\n UPDATE workspace_protection_rule\n SET name = $1, rules = $2, bypass_groups = $3, bypass_users = $4\n WHERE workspace_id = $5 AND name = $6\n ", + "query": "\n UPDATE workspace_protection_rule\n SET name = $1, rules = $2, bypass_groups = $3, bypass_users = $4,\n fork_role_mapping = COALESCE($7, fork_role_mapping)\n WHERE workspace_id = $5 AND name = $6\n ", "describe": { "columns": [], "parameters": { @@ -10,10 +10,11 @@ "TextArray", "TextArray", "Text", - "Text" + "Text", + "Jsonb" ] }, "nullable": [] }, - "hash": "adbe6ce27d46ba0d2d3ce87d875f60c179e5bccc54a36c8d5dcf04fd84445309" + "hash": "11a315b1b019c4bbd6b6676029943ea7c4da6b1368408c541cc2f9c390a5e840" } diff --git a/backend/.sqlx/query-42a0ba479ff164cc190c350927e13902ed94816142faf15446b4b9f19c3097d7.json b/backend/.sqlx/query-42a0ba479ff164cc190c350927e13902ed94816142faf15446b4b9f19c3097d7.json deleted file mode 100644 index 76eab1d117..0000000000 --- a/backend/.sqlx/query-42a0ba479ff164cc190c350927e13902ed94816142faf15446b4b9f19c3097d7.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "db_name": "PostgreSQL", - "query": "INSERT INTO usr\n (workspace_id, email, username, is_admin)\n SELECT $1, email, username, is_admin FROM usr\n WHERE workspace_id = $3 AND email = $2\n ON CONFLICT DO NOTHING\n ", - "describe": { - "columns": [], - "parameters": { - "Left": [ - "Varchar", - "Text", - "Text" - ] - }, - "nullable": [] - }, - "hash": "42a0ba479ff164cc190c350927e13902ed94816142faf15446b4b9f19c3097d7" -} diff --git a/backend/.sqlx/query-464286216add454cb64dba810d964c4b6a41d96df223b6d3d1d1bc972ab10d42.json b/backend/.sqlx/query-464286216add454cb64dba810d964c4b6a41d96df223b6d3d1d1bc972ab10d42.json deleted file mode 100644 index 43bb3da025..0000000000 --- a/backend/.sqlx/query-464286216add454cb64dba810d964c4b6a41d96df223b6d3d1d1bc972ab10d42.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "db_name": "PostgreSQL", - "query": "INSERT INTO usr (workspace_id, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via, workspace_role)\n SELECT $1, username, email, is_admin, created_at, operator, disabled, role, is_service_account,\n CASE WHEN $3 THEN NULL ELSE added_via END, workspace_role\n FROM usr WHERE workspace_id = $2\n AND (NOT $3 OR (NOT operator AND NOT disabled AND NOT is_service_account))\n ON CONFLICT DO NOTHING", - "describe": { - "columns": [], - "parameters": { - "Left": [ - "Varchar", - "Text", - "Bool" - ] - }, - "nullable": [] - }, - "hash": "464286216add454cb64dba810d964c4b6a41d96df223b6d3d1d1bc972ab10d42" -} diff --git a/backend/.sqlx/query-4e8c9610c697c2fb1b08ce78773ef2875ef34dd304ff21979095bb7b9304bccd.json b/backend/.sqlx/query-4e8c9610c697c2fb1b08ce78773ef2875ef34dd304ff21979095bb7b9304bccd.json new file mode 100644 index 0000000000..54c5e257dd --- /dev/null +++ b/backend/.sqlx/query-4e8c9610c697c2fb1b08ce78773ef2875ef34dd304ff21979095bb7b9304bccd.json @@ -0,0 +1,35 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT username, workspace_role, operator FROM usr\n WHERE workspace_id = $1 AND email = $2 AND NOT disabled", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "username", + "type_info": "Varchar" + }, + { + "ordinal": 1, + "name": "workspace_role", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "operator", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [ + false, + true, + false + ] + }, + "hash": "4e8c9610c697c2fb1b08ce78773ef2875ef34dd304ff21979095bb7b9304bccd" +} diff --git a/backend/.sqlx/query-67d86d334029276d78bf8fa14b25702eba0b0d674b99f952af94487b3e54211c.json b/backend/.sqlx/query-67d86d334029276d78bf8fa14b25702eba0b0d674b99f952af94487b3e54211c.json deleted file mode 100644 index 9a3cbd6703..0000000000 --- a/backend/.sqlx/query-67d86d334029276d78bf8fa14b25702eba0b0d674b99f952af94487b3e54211c.json +++ /dev/null @@ -1,23 +0,0 @@ -{ - "db_name": "PostgreSQL", - "query": "SELECT username FROM usr\n WHERE workspace_id = $1 AND email = $2 AND NOT operator AND NOT disabled", - "describe": { - "columns": [ - { - "ordinal": 0, - "name": "username", - "type_info": "Varchar" - } - ], - "parameters": { - "Left": [ - "Text", - "Text" - ] - }, - "nullable": [ - false - ] - }, - "hash": "67d86d334029276d78bf8fa14b25702eba0b0d674b99f952af94487b3e54211c" -} diff --git a/backend/.sqlx/query-dbf2982f43577999dec0c488dfb67f56e5bd0e5fbf8da5132f6fc5a282d2b0e1.json b/backend/.sqlx/query-8c3b028e01aaf693a69ef0f9644a7182490293e210602f5cc7b42978e3b89109.json similarity index 53% rename from backend/.sqlx/query-dbf2982f43577999dec0c488dfb67f56e5bd0e5fbf8da5132f6fc5a282d2b0e1.json rename to backend/.sqlx/query-8c3b028e01aaf693a69ef0f9644a7182490293e210602f5cc7b42978e3b89109.json index ae5792117d..3ee168958f 100644 --- a/backend/.sqlx/query-dbf2982f43577999dec0c488dfb67f56e5bd0e5fbf8da5132f6fc5a282d2b0e1.json +++ b/backend/.sqlx/query-8c3b028e01aaf693a69ef0f9644a7182490293e210602f5cc7b42978e3b89109.json @@ -1,6 +1,6 @@ { "db_name": "PostgreSQL", - "query": "\n INSERT INTO workspace_protection_rule (workspace_id, name, rules, bypass_groups, bypass_users)\n VALUES ($1, $2, $3, $4, $5)\n ", + "query": "\n INSERT INTO workspace_protection_rule (workspace_id, name, rules, bypass_groups, bypass_users, fork_role_mapping)\n VALUES ($1, $2, $3, $4, $5, $6)\n ", "describe": { "columns": [], "parameters": { @@ -9,10 +9,11 @@ "Varchar", "Int4", "TextArray", - "TextArray" + "TextArray", + "Jsonb" ] }, "nullable": [] }, - "hash": "dbf2982f43577999dec0c488dfb67f56e5bd0e5fbf8da5132f6fc5a282d2b0e1" + "hash": "8c3b028e01aaf693a69ef0f9644a7182490293e210602f5cc7b42978e3b89109" } diff --git a/backend/.sqlx/query-b80d2fd5368eb0bad94bf529b61df755ed21a35339cae18728d9cb0b6b7ea530.json b/backend/.sqlx/query-b80d2fd5368eb0bad94bf529b61df755ed21a35339cae18728d9cb0b6b7ea530.json new file mode 100644 index 0000000000..66d8d1b38f --- /dev/null +++ b/backend/.sqlx/query-b80d2fd5368eb0bad94bf529b61df755ed21a35339cae18728d9cb0b6b7ea530.json @@ -0,0 +1,20 @@ +{ + "db_name": "PostgreSQL", + "query": "WITH m AS (SELECT * FROM unnest($4::text[], $5::text[], $6::bool[]) AS m(from_role, to_role, to_operator))\n INSERT INTO usr (workspace_id, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via, workspace_role)\n SELECT $1, u.username, u.email, u.is_admin, u.created_at, COALESCE(m.to_operator, u.operator), u.disabled, u.role,\n u.is_service_account, CASE WHEN $3 OR $7::text IS NOT NULL THEN NULL ELSE u.added_via END,\n COALESCE(m.to_role, u.workspace_role)\n FROM usr u\n LEFT JOIN m ON m.from_role = COALESCE(u.workspace_role, CASE WHEN u.operator THEN 'operator' ELSE 'developer' END)\n WHERE u.workspace_id = $2\n AND ($7::text IS NULL OR u.email = $7)\n AND (NOT $3 OR (NOT u.operator AND NOT u.disabled AND NOT u.is_service_account))\n ON CONFLICT DO NOTHING", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Text", + "Bool", + "TextArray", + "TextArray", + "BoolArray", + "Text" + ] + }, + "nullable": [] + }, + "hash": "b80d2fd5368eb0bad94bf529b61df755ed21a35339cae18728d9cb0b6b7ea530" +} diff --git a/backend/.sqlx/query-4cfb35e423a75ca2701f03d5a30a7c0778af5e548254f3e0f29004d7f2058eef.json b/backend/.sqlx/query-c0b4a4914f738a6a9975bb75d8ffeda071ba59b874fd56ec3eeb9b054df7eb64.json similarity index 68% rename from backend/.sqlx/query-4cfb35e423a75ca2701f03d5a30a7c0778af5e548254f3e0f29004d7f2058eef.json rename to backend/.sqlx/query-c0b4a4914f738a6a9975bb75d8ffeda071ba59b874fd56ec3eeb9b054df7eb64.json index abe21b0930..e612e41481 100644 --- a/backend/.sqlx/query-4cfb35e423a75ca2701f03d5a30a7c0778af5e548254f3e0f29004d7f2058eef.json +++ b/backend/.sqlx/query-c0b4a4914f738a6a9975bb75d8ffeda071ba59b874fd56ec3eeb9b054df7eb64.json @@ -1,6 +1,6 @@ { "db_name": "PostgreSQL", - "query": "\n SELECT\n workspace_id,\n name,\n rules as \"rules: ProtectionRules\",\n bypass_groups,\n bypass_users\n FROM workspace_protection_rule\n WHERE workspace_id = $1\n ORDER BY name\n ", + "query": "\n SELECT\n workspace_id,\n name,\n rules as \"rules: ProtectionRules\",\n bypass_groups,\n bypass_users,\n fork_role_mapping as \"fork_role_mapping: ForkRoleMapping\"\n FROM workspace_protection_rule\n WHERE workspace_id = $1\n ORDER BY name\n ", "describe": { "columns": [ { @@ -27,6 +27,11 @@ "ordinal": 4, "name": "bypass_users", "type_info": "TextArray" + }, + { + "ordinal": 5, + "name": "fork_role_mapping: ForkRoleMapping", + "type_info": "Jsonb" } ], "parameters": { @@ -39,8 +44,9 @@ false, false, false, + false, false ] }, - "hash": "4cfb35e423a75ca2701f03d5a30a7c0778af5e548254f3e0f29004d7f2058eef" + "hash": "c0b4a4914f738a6a9975bb75d8ffeda071ba59b874fd56ec3eeb9b054df7eb64" } diff --git a/backend/.sqlx/query-ee2a8ae43f2d0a748f1e55185df87f1d9ff80b7298846d45adcf854c9052424d.json b/backend/.sqlx/query-ee2a8ae43f2d0a748f1e55185df87f1d9ff80b7298846d45adcf854c9052424d.json new file mode 100644 index 0000000000..69e19f1ecd --- /dev/null +++ b/backend/.sqlx/query-ee2a8ae43f2d0a748f1e55185df87f1d9ff80b7298846d45adcf854c9052424d.json @@ -0,0 +1,23 @@ +{ + "db_name": "PostgreSQL", + "query": "SELECT COUNT(*) FROM workspace_protection_rule WHERE ($1::text IS NULL OR workspace_id = $1)\n AND (fork_role_mapping ? $2\n OR EXISTS (SELECT 1 FROM jsonb_each_text(fork_role_mapping) m WHERE m.value = $2))", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "count", + "type_info": "Int8" + } + ], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [ + null + ] + }, + "hash": "ee2a8ae43f2d0a748f1e55185df87f1d9ff80b7298846d45adcf854c9052424d" +} diff --git a/backend/ee-repo-ref.txt b/backend/ee-repo-ref.txt index 172a67556a..f5d5dd8275 100644 --- a/backend/ee-repo-ref.txt +++ b/backend/ee-repo-ref.txt @@ -1 +1 @@ -5e7b1de9a07114e277a22e9e3c68b0809c06f18d +d535972f0b35c6a7f257720816bdaec18465adef diff --git a/backend/migrations/20261027164425_protection_rule_fork_role_mapping.down.sql b/backend/migrations/20261027164425_protection_rule_fork_role_mapping.down.sql new file mode 100644 index 0000000000..a5b27f1681 --- /dev/null +++ b/backend/migrations/20261027164425_protection_rule_fork_role_mapping.down.sql @@ -0,0 +1 @@ +ALTER TABLE workspace_protection_rule DROP COLUMN IF EXISTS fork_role_mapping; diff --git a/backend/migrations/20261027164425_protection_rule_fork_role_mapping.up.sql b/backend/migrations/20261027164425_protection_rule_fork_role_mapping.up.sql new file mode 100644 index 0000000000..7d49d9b2f4 --- /dev/null +++ b/backend/migrations/20261027164425_protection_rule_fork_role_mapping.up.sql @@ -0,0 +1,2 @@ +ALTER TABLE workspace_protection_rule + ADD COLUMN IF NOT EXISTS fork_role_mapping JSONB NOT NULL DEFAULT '{}'::jsonb; diff --git a/backend/summarized_schema.txt b/backend/summarized_schema.txt index 9b89ad2561..95fcdc8d0a 100644 --- a/backend/summarized_schema.txt +++ b/backend/summarized_schema.txt @@ -241,7 +241,7 @@ workspace_invite: workspace_id(char), email(char), is_admin(bool), operator(bool FK: (workspace_id) -> workspace(id) workspace_key: workspace_id(char), kind(workspace_key_kind), key(char) FK: (workspace_id) -> workspace(id) -workspace_protection_rule: workspace_id(char), name(char), rules(int), bypass_groups(text[]), bypass_users(text[]), created_at(ts) +workspace_protection_rule: workspace_id(char), name(char), rules(int), bypass_groups(text[]), bypass_users(text[]), created_at(ts), fork_role_mapping(jsonb) FK: (workspace_id) -> workspace(id) workspace_runnable_dependencies: flow_path(char), runnable_path(char), script_hash(bigint), runnable_is_flow(bool), workspace_id(char), app_path(char), id(bigint), runnable_is_agent(bool) FK: (app_path, workspace_id) -> app(path, workspace_id) | (flow_path, workspace_id) -> flow(path, workspace_id) diff --git a/backend/windmill-api-integration-tests/tests/fork_members.rs b/backend/windmill-api-integration-tests/tests/fork_members.rs index 5f4baced1f..c3e069862c 100644 --- a/backend/windmill-api-integration-tests/tests/fork_members.rs +++ b/backend/windmill-api-integration-tests/tests/fork_members.rs @@ -73,3 +73,108 @@ async fn test_fork_adds_parent_admins_and_developers(db: Pool) -> anyh Ok(()) } + +/// A forker joins the fork with the role they hold in the parent, unless a ruleset of the parent +/// maps it; a role without `fork` cannot fork at all, and the fork's owner cannot seat a member +/// as a developer the mapping makes something else. EE: CE caps the workspace count below the +/// forks this creates. +#[cfg(feature = "enterprise")] +#[sqlx::test(migrations = "../migrations", fixtures("base"))] +async fn test_fork_carries_role_and_mapping(db: Pool) -> anyhow::Result<()> { + initialize_tracing().await; + // Not `test-workspace`: the role and ruleset caches are process-wide and keyed by workspace + // id, so the other test here, running concurrently on its own database, would fill them. + sqlx::raw_sql( + "INSERT INTO workspace (id, name, owner) VALUES ('rf-ws', 'rf-ws', 'test-user'); + INSERT INTO workspace_settings (workspace_id) VALUES ('rf-ws'); + INSERT INTO workspace_key (workspace_id, kind, key) VALUES ('rf-ws', 'cloud', 'rf-key'); + INSERT INTO group_ (workspace_id, name, summary, extra_perms) VALUES + ('rf-ws', 'all', 'All users', '{}'); + INSERT INTO usr (workspace_id, email, username, is_admin, operator, role) VALUES + ('rf-ws', 'test@windmill.dev', 'test-user', true, false, 'Admin'), + ('rf-ws', 'test2@windmill.dev', 'test-user-2', false, true, 'User'), + ('rf-ws', 'test3@windmill.dev', 'test-user-3', false, false, 'User')", + ) + .execute(&db) + .await?; + windmill_common::roles::invalidate_all_roles_cache(); + windmill_common::workspaces::invalidate_protection_rules_cache("rf-ws"); + + let server = ApiServer::start(db.clone()).await?; + let base_url = format!( + "http://localhost:{}/api/w/rf-ws/workspaces", + server.addr.port() + ); + let client = reqwest::Client::new(); + let fork = |id: &str| { + client + .post(format!("{base_url}/create_fork")) + .header("Authorization", "Bearer SECRET_TOKEN_2") + .json(&json!({ "id": id, "name": id })) + .send() + }; + let fork_role = |id: &'static str| { + sqlx::query_as::<_, (Option, bool)>( + "SELECT workspace_role, operator FROM usr WHERE workspace_id = $1 AND username = 'test-user-2'", + ) + .bind(id) + .fetch_one(&db) + }; + + let resp = fork("wm-fork-refused").await?; + assert_eq!(resp.status(), 403, "operator lacks fork: {}", resp.text().await?); + + // `code` makes the copy write `operator = false`: the legacy row's NULL role must not then + // read as developer. + sqlx::query( + r#"UPDATE workspace_settings SET role_overrides = '{"operator": {"capabilities": {"fork": true, "code": true}}}' + WHERE workspace_id = 'rf-ws'"#, + ) + .execute(&db) + .await?; + windmill_common::roles::invalidate_all_roles_cache(); + + let resp = fork("wm-fork-kept").await?; + assert!(resp.status().is_success(), "{}", resp.text().await?); + assert_eq!( + fork_role("wm-fork-kept").await?, + (Some("operator".to_string()), false), + "stays an operator" + ); + + let resp = client + .post(format!("{base_url}/protection_rules")) + .header("Authorization", "Bearer SECRET_TOKEN") + .json(&json!({ + "name": "forks", "rules": [], "bypass_groups": [], "bypass_users": [], + "fork_role_mapping": { "operator": "developer", "developer": "operator" } + })) + .send() + .await?; + assert!(resp.status().is_success(), "{}", resp.text().await?); + + let resp = fork("wm-fork-mapped").await?; + assert!(resp.status().is_success(), "{}", resp.text().await?); + assert_eq!( + fork_role("wm-fork-mapped").await?, + (Some("developer".to_string()), false), + "mapped to developer" + ); + + let resp = reqwest::Client::new() + .post(format!( + "http://localhost:{}/api/w/wm-fork-mapped/workspaces/add_user", + server.addr.port() + )) + .header("Authorization", "Bearer SECRET_TOKEN_2") + .json(&json!({ "email": "test3@windmill.dev", "is_admin": false, "operator": false })) + .send() + .await?; + let (status, text) = (resp.status(), resp.text().await?); + assert!( + status == 403 && text.contains("role in its forks"), + "a developer mapped to operator: {status} {text}" + ); + + Ok(()) +} diff --git a/backend/windmill-api-workspaces/src/workspaces.rs b/backend/windmill-api-workspaces/src/workspaces.rs index 9a6efec148..7c4c649ad2 100644 --- a/backend/windmill-api-workspaces/src/workspaces.rs +++ b/backend/windmill-api-workspaces/src/workspaces.rs @@ -29,7 +29,7 @@ use regex::Regex; use hex; use sha2::{Digest, Sha256}; -use std::collections::{HashMap, HashSet}; +use std::collections::{BTreeMap, HashMap, HashSet}; use strum::IntoEnumIterator; use uuid::Uuid; use windmill_audit::audit_oss::{audit_log, AuditAuthorable}; @@ -8084,34 +8084,48 @@ async fn clone_groups( Ok(()) } -/// Copy the source workspace's members (the `usr` rows, carrying each member's role) into the -/// target so a fork/dev can be a shared environment. Idempotent — skips members the target already -/// has. Group memberships are not handled here: the sole caller is the create-fork path, where -/// `clone_groups` already copies the source's full group structure (including `all` membership). -async fn copy_workspace_members( +/// The role each of the parent's roles becomes in a fork of it, after the parent's ruleset +/// mappings, with the `usr.operator` value that goes with it. +struct ForkRoles { + from: Vec, + to: Vec, + to_operator: Vec, +} + +impl ForkRoles { + fn maps(&self, role: &str) -> bool { + self.from + .iter() + .zip(&self.to) + .any(|(from, to)| from == role && from != to) + } +} + +async fn fork_roles(db: &DB, parent_workspace_id: &str) -> Result { + let mapping = windmill_common::workspaces::fork_role_mapping(parent_workspace_id, db).await?; + let roles = windmill_common::roles::workspace_roles(db, parent_workspace_id).await?; + let mut out = ForkRoles { from: vec![], to: vec![], to_operator: vec![] }; + for role in roles.iter() { + let to = mapping.get(&role.name).unwrap_or(&role.name); + let caps = roles + .iter() + .find(|r| &r.name == to) + .map(|r| r.capabilities) + .unwrap_or(windmill_common::roles::RoleCapabilities::NONE); + out.from.push(role.name.clone()); + out.to.push(to.clone()); + out.to_operator.push(!caps.code); + } + Ok(out) +} + +/// Give the fork the parent's workspace-only roles and overrides, so a role carried into the fork +/// means there what it means in the parent. +async fn copy_workspace_roles( tx: &mut Transaction<'_, Postgres>, source_workspace_id: &str, target_workspace_id: &str, - admins_and_developers_only: bool, ) -> Result<()> { - // Admins and developers join as manual members: the fork does not inherit the source's - // instance-group config, so a copied `instance_group` provenance would let the fork's - // reconciliation delete them and their data. - sqlx::query!( - "INSERT INTO usr (workspace_id, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via, workspace_role) - SELECT $1, username, email, is_admin, created_at, operator, disabled, role, is_service_account, - CASE WHEN $3 THEN NULL ELSE added_via END, workspace_role - FROM usr WHERE workspace_id = $2 - AND (NOT $3 OR (NOT operator AND NOT disabled AND NOT is_service_account)) - ON CONFLICT DO NOTHING", - target_workspace_id, - source_workspace_id, - admins_and_developers_only, - ) - .execute(&mut **tx) - .await?; - // The members keep their roles, so the fork needs the source's workspace-only roles and - // overrides to give them the same meaning. sqlx::query!( "UPDATE workspace_settings SET role_overrides = (SELECT role_overrides FROM workspace_settings WHERE workspace_id = $2) @@ -8134,6 +8148,50 @@ async fn copy_workspace_members( Ok(()) } +/// Copy the source workspace's members (the `usr` rows) into the target so a fork/dev can be a +/// shared environment, each with its role after `roles`. Idempotent — skips members the target +/// already has. `only_email` restricts the copy to one member. Group memberships are not handled +/// here: the sole caller is the create-fork path, where `clone_groups` already copies the source's +/// full group structure (including `all` membership). +async fn copy_workspace_members( + tx: &mut Transaction<'_, Postgres>, + source_workspace_id: &str, + target_workspace_id: &str, + roles: &ForkRoles, + admins_and_developers_only: bool, + only_email: Option<&str>, +) -> Result<()> { + // Admins and developers, and a lone member, join as manual members: the fork does not inherit + // the source's instance-group config, so a copied `instance_group` provenance would let the fork's + // reconciliation delete them and their data. The role is written by name even when unmapped: + // `operator` is recomputed from it, and a NULL role would then read as whatever that flag + // names. A role the parent no longer defines is carried as is, and resolves to no + // capabilities in the fork as it does in the parent. + sqlx::query!( + "WITH m AS (SELECT * FROM unnest($4::text[], $5::text[], $6::bool[]) AS m(from_role, to_role, to_operator)) + INSERT INTO usr (workspace_id, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via, workspace_role) + SELECT $1, u.username, u.email, u.is_admin, u.created_at, COALESCE(m.to_operator, u.operator), u.disabled, u.role, + u.is_service_account, CASE WHEN $3 OR $7::text IS NOT NULL THEN NULL ELSE u.added_via END, + COALESCE(m.to_role, u.workspace_role) + FROM usr u + LEFT JOIN m ON m.from_role = COALESCE(u.workspace_role, CASE WHEN u.operator THEN 'operator' ELSE 'developer' END) + WHERE u.workspace_id = $2 + AND ($7::text IS NULL OR u.email = $7) + AND (NOT $3 OR (NOT u.operator AND NOT u.disabled AND NOT u.is_service_account)) + ON CONFLICT DO NOTHING", + target_workspace_id, + source_workspace_id, + admins_and_developers_only, + &roles.from, + &roles.to, + &roles.to_operator, + only_email, + ) + .execute(&mut **tx) + .await?; + Ok(()) +} + async fn clone_resource_types( tx: &mut Transaction<'_, Postgres>, source_workspace_id: &str, @@ -9303,6 +9361,32 @@ async fn clone_workspace_dependencies( Ok(()) } +/// Who may fork `w_id` at all: the instance switch, the caller's role, then the parent's rulesets. +async fn enforce_fork_allowed(db: &DB, authed: &ApiAuthed, w_id: &str) -> Result<()> { + if *DISABLE_WORKSPACE_FORK { + require_super_admin(db, authed).await?; + } + if !windmill_common::roles::caller_can_fork(db, w_id, authed).await? { + windmill_common::feature_usage::log_feature_usage("fork_roles", "fork_refused", ""); + return Err(Error::PermissionDenied(format!( + "Your role in {w_id} cannot fork it" + ))); + } + if let RuleCheckResult::Blocked(msg) = check_user_against_rule( + w_id, + &ProtectionRuleKind::DisableWorkspaceForking, + AuditAuthorable::username(authed), + &authed.groups, + authed.is_admin, + db, + ) + .await? + { + return Err(Error::PermissionDenied(msg)); + } + Ok(()) +} + async fn deprecated_create_workspace_fork(_authed: ApiAuthed) -> Result { return Err(Error::BadRequest("This API endpoint has been relocated. Your Windmill CLI version is outdated and needs to be updated.".to_string())); } @@ -9322,21 +9406,7 @@ async fn create_workspace_fork_branch( enforce_cloud_fork_cap(&db, &w_id).await?; } - if *DISABLE_WORKSPACE_FORK { - require_super_admin(&db, &authed).await?; - } - if let RuleCheckResult::Blocked(msg) = check_user_against_rule( - &w_id, - &ProtectionRuleKind::DisableWorkspaceForking, - AuditAuthorable::username(&authed), - &authed.groups, - authed.is_admin, - &db, - ) - .await? - { - return Err(Error::PermissionDenied(msg)); - } + enforce_fork_allowed(&db, &authed, &w_id).await?; enforce_isolated_forks(&db, &authed, &w_id, &nw).await?; // Two-phase create for git-synced workspaces: this endpoint only creates the git branch(es) and @@ -10073,21 +10143,7 @@ async fn create_workspace_fork( #[cfg(not(feature = "enterprise"))] _check_nb_of_workspaces(&db).await?; - if *DISABLE_WORKSPACE_FORK { - require_super_admin(&db, &authed).await?; - } - if let RuleCheckResult::Blocked(msg) = check_user_against_rule( - &parent_workspace_id, - &ProtectionRuleKind::DisableWorkspaceForking, - AuditAuthorable::username(&authed), - &authed.groups, - authed.is_admin, - &db, - ) - .await? - { - return Err(Error::PermissionDenied(msg)); - } + enforce_fork_allowed(&db, &authed, &parent_workspace_id).await?; enforce_isolated_forks(&db, &authed, &parent_workspace_id, &nw).await?; if nw.is_dev_workspace { @@ -10652,6 +10708,8 @@ async fn write_workspace_fork( copies: &[crate::datatable_clone::MadeCopy], progress: &ForkProgress, ) -> Result { + // Read on the pool before the transaction holds a connection of its own. + let roles = fork_roles(&db, &parent_workspace_id).await?; let mut tx: Transaction<'_, Postgres> = db.begin().await?; // Held until this transaction ends: after an error, the copies' cleanup waits on it, so it reads // whether the fork exists only once a commit still being resolved has settled. @@ -10721,13 +10779,20 @@ async fn write_workspace_fork( // Optionally bring the parent's members into the fork (a shared dev env). Dev-only: it's part of the // dev-workspace feature (and the frontend only offers it there), so the backend enforces it rather // than trusting the client — copying the parent's whole team into an ordinary throwaway fork isn't - // intended. Dev creation is already admin-gated, so this is transitively admin-only too. Done before - // the explicit creator insert below so the creator (a parent member) is copied with full metadata - // (operator/role/is_service_account/added_via), not the bare row the insert alone would leave. + // intended. Dev creation is already admin-gated, so this is transitively admin-only too. // Independently, the parent's admins can have every fork of it start with its admins and // developers; the forker cannot opt out, since the point is that those admins can review it. + copy_workspace_roles(&mut tx, &parent_workspace_id, &forked_id).await?; if nw.copy_members && nw.is_dev_workspace { - copy_workspace_members(&mut tx, &parent_workspace_id, &forked_id, false).await?; + copy_workspace_members( + &mut tx, + &parent_workspace_id, + &forked_id, + &roles, + false, + None, + ) + .await?; } else if sqlx::query_scalar!( "SELECT add_admins_and_developers_to_forks FROM workspace_settings WHERE workspace_id = $1", parent_workspace_id @@ -10736,24 +10801,44 @@ async fn write_workspace_fork( .await? .unwrap_or(false) { - copy_workspace_members(&mut tx, &parent_workspace_id, &forked_id, true).await?; + copy_workspace_members( + &mut tx, + &parent_workspace_id, + &forked_id, + &roles, + true, + None, + ) + .await?; } - // Ensure the creator is a member of the fork even without copy_members (or if they aren't a parent - // member). No-op when copy_members already brought their full row. - sqlx::query!( - "INSERT INTO usr - (workspace_id, email, username, is_admin) - SELECT $1, email, username, is_admin FROM usr - WHERE workspace_id = $3 AND email = $2 - ON CONFLICT DO NOTHING - ", - forked_id, - authed.email, - parent_workspace_id, + // The creator joins with the role they hold in the parent, after its ruleset mappings, even + // without copy_members. No-op when copy_members already brought their row. + copy_workspace_members( + &mut tx, + &parent_workspace_id, + &forked_id, + &roles, + false, + Some(&authed.email), ) - .execute(&mut *tx) .await?; + if let Some(forker) = sqlx::query!( + "SELECT workspace_role, operator FROM usr WHERE workspace_id = $1 AND email = $2", + parent_workspace_id, + authed.email, + ) + .fetch_optional(&mut *tx) + .await? + { + let role = + windmill_common::roles::resolve_role_name(forker.workspace_role, forker.operator); + windmill_common::feature_usage::log_feature_usage( + "fork_roles", + "fork_created", + if roles.maps(&role) { "mapped" } else { "kept" }, + ); + } // The pointers this fork writes to the parent's data tables stay invisible until it commits, so // a rename of one of them cannot carry them. Holding the parent's settings row makes such a @@ -11929,8 +12014,10 @@ If you do not have an account on {}, login with SSO or ask an admin to create an /// - only on a fork they created, never on a root workspace; /// - only as a developer, so it can never mint an admin (nor an operator, which would need the /// workspace's operator settings to be meaningful); -/// - only for someone who is already a developer or admin of the parent, so pulling them into the -/// fork cannot widen who can read the parent's data. +/// - only for a member of the parent whose role in its forks, after the parent's ruleset +/// mappings, is `developer`, so pulling them into the fork cannot widen who can read the +/// parent's data nor undo a mapping. A custom role is refused even when unrestricted: the grant +/// seats a developer, not that role. /// /// Anything outside those bounds stays an admin's call. Returns the username the new member must be /// given in the fork. @@ -11957,19 +12044,29 @@ async fn authorize_fork_owner_add_user( ))); } - let parent_username = sqlx::query_scalar!( - "SELECT username FROM usr - WHERE workspace_id = $1 AND email = $2 AND NOT operator AND NOT disabled", + let member = sqlx::query!( + "SELECT username, workspace_role, operator FROM usr + WHERE workspace_id = $1 AND email = $2 AND NOT disabled", parent, nu.email, ) .fetch_optional(db) .await?; + let mut fork_role = None; + if let Some(m) = &member { + let role = windmill_common::roles::resolve_role_name(m.workspace_role.clone(), m.operator); + let mapping = windmill_common::workspaces::fork_role_mapping(&parent, db).await?; + fork_role = Some(mapping.get(&role).cloned().unwrap_or(role)); + } + let parent_username = member + .filter(|_| fork_role.as_deref() == Some(windmill_common::roles::DEVELOPER_ROLE)) + .map(|m| m.username); let Some(parent_username) = parent_username else { return Err(Error::PermissionDenied(format!( - "as the creator of fork {w_id} you can only add developers or admins of its parent \ - workspace {parent}; {} is not one, so only an admin of {w_id} can add them", + "as the creator of fork {w_id} you can only add members of its parent workspace \ + {parent} whose role in its forks is developer; {} is not one, so only an admin of \ + {w_id} can add them", nu.email ))); }; @@ -12581,6 +12678,8 @@ struct CreateProtectionRuleRequest { rules: Vec, bypass_groups: Vec, bypass_users: Vec, + #[serde(default)] + fork_role_mapping: BTreeMap, } #[derive(Deserialize)] @@ -12591,6 +12690,9 @@ struct UpdateProtectionRuleRequest { rules: Vec, bypass_groups: Vec, bypass_users: Vec, + /// Absent leaves the stored mapping alone, which is what a client that predates it gets. + #[serde(default)] + fork_role_mapping: Option>, } #[derive(Serialize)] @@ -12600,6 +12702,7 @@ struct ProtectionRulesetResponse { pub rules: Vec, pub bypass_groups: Vec, pub bypass_users: Vec, + pub fork_role_mapping: BTreeMap, } impl From for ProtectionRulesetResponse { @@ -12618,10 +12721,38 @@ impl From for ProtectionRulesetResponse { name: value.name, bypass_groups: value.bypass_groups, bypass_users: value.bypass_users, + fork_role_mapping: value.fork_role_mapping.0, } } } +/// A fork role mapping may only name roles `w_id` defines, and maps each to a different one. +async fn validate_fork_role_mapping( + db: &DB, + w_id: &str, + mapping: &BTreeMap, +) -> Result<()> { + if mapping.is_empty() { + return Ok(()); + } + let roles = windmill_common::roles::workspace_roles(db, w_id).await?; + for (from, to) in mapping { + for name in [from, to] { + if !roles.iter().any(|r| &r.name == name) { + return Err(Error::BadRequest(format!( + "fork role mapping: role `{name}` is not defined in workspace {w_id}" + ))); + } + } + if from == to { + return Err(Error::BadRequest(format!( + "fork role mapping: `{from}` is mapped to itself" + ))); + } + } + Ok(()) +} + /// List all protection rules for a workspace async fn list_protection_rules( Extension(db): Extension, @@ -13003,6 +13134,10 @@ async fn create_protection_rule( ) -> Result { require_admin(authed.is_admin, &authed.username)?; reject_reserved_rule_name(&req.name)?; + validate_fork_role_mapping(&db, &w_id, &req.fork_role_mapping).await?; + if !req.fork_role_mapping.is_empty() { + windmill_common::feature_usage::log_feature_usage("fork_roles", "mapping_saved", "create"); + } let mut tx = db.begin().await?; @@ -13026,14 +13161,15 @@ async fn create_protection_rule( // Insert the new rule sqlx::query!( r#" - INSERT INTO workspace_protection_rule (workspace_id, name, rules, bypass_groups, bypass_users) - VALUES ($1, $2, $3, $4, $5) + INSERT INTO workspace_protection_rule (workspace_id, name, rules, bypass_groups, bypass_users, fork_role_mapping) + VALUES ($1, $2, $3, $4, $5, $6) "#, &w_id, &req.name, ProtectionRules::from(&req.rules).bits(), &req.bypass_groups, &req.bypass_users, + sqlx::types::Json(&req.fork_role_mapping) as _, ) .execute(&mut *tx) .await?; @@ -13099,6 +13235,16 @@ async fn update_protection_rule( } reject_reserved_rule_name(new_name)?; } + if let Some(mapping) = &req.fork_role_mapping { + validate_fork_role_mapping(&db, &w_id, mapping).await?; + if !mapping.is_empty() { + windmill_common::feature_usage::log_feature_usage( + "fork_roles", + "mapping_saved", + "update", + ); + } + } let mut tx = db.begin().await?; @@ -13142,7 +13288,8 @@ async fn update_protection_rule( sqlx::query!( r#" UPDATE workspace_protection_rule - SET name = $1, rules = $2, bypass_groups = $3, bypass_users = $4 + SET name = $1, rules = $2, bypass_groups = $3, bypass_users = $4, + fork_role_mapping = COALESCE($7, fork_role_mapping) WHERE workspace_id = $5 AND name = $6 "#, final_name, @@ -13150,7 +13297,8 @@ async fn update_protection_rule( &req.bypass_groups, &req.bypass_users, &w_id, - &rule_name + &rule_name, + req.fork_role_mapping.as_ref().map(sqlx::types::Json) as _, ) .execute(&mut *tx) .await?; diff --git a/backend/windmill-api/openapi.yaml b/backend/windmill-api/openapi.yaml index b27bdca80a..422070c196 100644 --- a/backend/windmill-api/openapi.yaml +++ b/backend/windmill-api/openapi.yaml @@ -8239,6 +8239,8 @@ paths: $ref: "#/components/schemas/RuleBypasserGroups" bypass_users: $ref: "#/components/schemas/RuleBypasserUsers" + fork_role_mapping: + $ref: "#/components/schemas/ForkRoleMapping" responses: "200": description: protection rule created successfully @@ -8287,6 +8289,8 @@ paths: $ref: "#/components/schemas/RuleBypasserGroups" bypass_users: $ref: "#/components/schemas/RuleBypasserUsers" + fork_role_mapping: + $ref: "#/components/schemas/ForkRoleMapping" responses: "200": description: protection rule updated successfully @@ -39157,9 +39161,12 @@ components: type: boolean manage_triggers: type: boolean + fork: + type: boolean + description: Fork the workspace. visibility: $ref: "#/components/schemas/RoleVisibility" - required: [code, flows, apps, manage_schedules, manage_triggers, visibility] + required: [code, flows, apps, manage_schedules, manage_triggers, fork, visibility] RoleCapabilitiesPatch: type: object @@ -39172,6 +39179,7 @@ components: $ref: "#/components/schemas/AuthorLevel" manage_schedules: { type: boolean } manage_triggers: { type: boolean } + fork: { type: boolean } visibility: type: object additionalProperties: @@ -39836,6 +39844,16 @@ components: $ref: "#/components/schemas/RuleBypasserGroups" bypass_users: $ref: "#/components/schemas/RuleBypasserUsers" + fork_role_mapping: + $ref: "#/components/schemas/ForkRoleMapping" + ForkRoleMapping: + type: object + description: >- + Role a member of this workspace gets in a fork of it, keyed by the role it holds here. + Unmapped roles carry over unchanged; where two rulesets map the same role, the first by + name wins. On update, omitting it leaves the stored mapping unchanged. + additionalProperties: + type: string ProtectionRules: type: array description: Configuration of protection restrictions diff --git a/backend/windmill-api/src/mcp/session/gen/instructions.json b/backend/windmill-api/src/mcp/session/gen/instructions.json index 2ed15e3eaf..33ba824a4d 100644 --- a/backend/windmill-api/src/mcp/session/gen/instructions.json +++ b/backend/windmill-api/src/mcp/session/gen/instructions.json @@ -261,6 +261,6 @@ "datatable.python3": "# Data tables: schema changes, migrations and permissions\n\nA data table is a PostgreSQL database managed by Windmill and shared by the whole workspace: scripts, flows and apps reach it by name, `main` being the default. It is configured in the workspace settings (Data Tables), never created by SQL.\n\n## Changing the schema\n\nA data table with migrations enabled keeps its schema as an ordered list of versioned SQL migrations, each an `up` and an optional `down`. That list is what git sync and a workspace merge carry, and what rebuilds the schema in another workspace. So first find out whether migrations are enabled on the data table:\n\n- **Enabled**: make every schema change (CREATE, ALTER, DROP, anything but row data) as a new migration. A change made with plain SQL exists in that one database only, and the history no longer describes it.\n- **Not enabled**: schema changes are plain SQL. Enabling migrations is a workspace admin's decision, made in the Data Tables settings, not a step to take on the way to something else.\n\nWriting a migration:\n\n- Wrap the statements in `BEGIN; ... END;` so it applies atomically. Leave the frame out only for a statement that cannot run in a transaction, such as `CREATE INDEX CONCURRENTLY`: such a migration can fail half applied.\n- Give it a `down` that undoes the `up` whenever the change is reversible. Without one it cannot be rolled back.\n- Declare the schemas it changes on a leading line of its `up`, `-- schemas sales, billing`, and qualify every name with its schema (`sales.orders`). A migration waits only on earlier migrations whose schemas overlap its own. One that declares none applies to the whole database and every later migration waits for it, so leave the line out only for a change that really spans the database, such as an extension.\n- Migrations apply oldest first. One that is applied is not edited: correct it with a new migration.\n- Row data is not schema. Insert, update and delete rows with plain SQL, migrations or not.\n\n## A migration is not a draft\n\nA migration is saved to the workspace and changes the database as soon as it is applied, for everything that reads those tables. To try a schema out without touching tables already in use:\n\n- put the new tables in a schema of their own, created by the same migration (`CREATE SCHEMA IF NOT EXISTS`), and point what you are building at it. A raw app names its schema in its `data` config, which is why a new app gets a new schema by default;\n- or work in a workspace fork whose data table was cloned when the fork was created (\"Clone schema only\" or \"Clone schema and data\"): migrations applied there change that copy alone, and reach the parent workspace when the fork is merged. A fork that kept the original data table points at the parent's database, so a migration applied in it changes the parent's tables at once.\n\nAltering or dropping a table other scripts or apps use is neither of those: say what depends on it before doing it.\n\n## Roles and permissions\n\nData table roles are a self-hosted Enterprise Edition feature, for data tables on managed Postgres (the instance database or the external managed instance). Without them, and always on a data table backed by a Postgres resource, everyone who can use the data table reaches all of it: there is no per-user, per-schema or per-row access to promise.\n\nA data table under roles is reached as a Postgres role, and Postgres enforces what that role was granted:\n\n- roles are assigned, per data table, to users, groups or folders. `admin` is the connection the data table had before roles: it owns the existing objects and ignores grants;\n- a connection either names a role (`wmill.datatable('main', { role: 'analyst' })`, `role='analyst'` in Python, `datatable://main?role=analyst`) and holds that role alone, or names none and is resolved from the data table's default role;\n- `permission denied` means the role in use lacks a grant. Say which role and which object: the grant is a data table admin's to give, in the roles drawer of the Database Manager. Do not switch to `admin` to get past it;\n- a migration runs as the role its SQL declares on its first line, `-- role `, the same one in its `down`. One that declares none runs as `admin`, which only those who may use `admin` can do.\n\nRead the Permissions section of https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables before designing who reaches what: it covers tenants, grants, the default role and the roles a connection that names none ends up holding, rows that differ per viewer through row-level security, and how forks and raw apps behave under roles.\n\n\n# Datatable SQL SDK reference\n\nDatatables are workspace-scoped managed PostgreSQL databases. In chat, explore and shape them with the datatable tools you were given. The reference below is for code you author inside runnables (inline app runnables, scripts, or flow rawscript modules) that reads or writes datatable data at runtime.\n\n- A runnable accesses a datatable via `wmill.datatable()` (the default \"main\") or `wmill.datatable('')`, referencing tables as `schema.table`.\n- Use parameterized queries (the tagged template in TypeScript, `$1`/`$2` placeholders in Python) — never interpolate untrusted values into SQL strings.\n\n## Python Datatable API (wmill)\n\nImport: `import wmill`\n\n# Get a DataTable client for SQL queries.\n# \n# Args:\n# name: Database name (default: \"main\")\n# role: Connect as this data table role instead of the data table's default one.\n# \n# Returns:\n# DataTableClient instance\ndef datatable(name: str = 'main', *, role: Optional[str] = None) -> DataTableClient\n\n# Client for executing SQL queries against Windmill DataTables.\nclass DataTableClient:\n # Initialize DataTableClient.\n # \n # Args:\n # client: Windmill client instance\n # name: DataTable name\n # role: Data table role to connect as, or None for the data table's default\n def __init__(client: Windmill, name: str, role: Optional[str] = None)\n\n # Execute a SQL query against the DataTable.\n # \n # Args:\n # sql: SQL query string with $1, $2, etc. placeholders\n # *args: Positional arguments to bind to query placeholders\n # \n # Returns:\n # SqlQuery instance for fetching results\n def query(sql: str, *args) -> SqlQuery\n\n\n# Query result handler for DataTable and DuckLake queries.\nclass SqlQuery:\n # Initialize SqlQuery.\n # \n # Args:\n # sql: SQL query string\n # fetch_fn: Function to execute the query\n def __init__(sql: str, fetch_fn)\n\n # Execute query and fetch results.\n # \n # Args:\n # result_collection: Optional result collection mode\n # \n # Returns:\n # Query results\n def fetch(result_collection: str | None = None)\n\n # Execute query and fetch first row of results.\n # \n # Returns:\n # First row of query results\n def fetch_one()\n\n # Execute query and fetch first row of results. Return result as a scalar value.\n # \n # Returns:\n # First row of query result as a scalar value\n def fetch_one_scalar()\n\n # Execute query and don't return any results.\n # \n def execute()\n\n\n", "datatable.go": "# Data tables: schema changes, migrations and permissions\n\nA data table is a PostgreSQL database managed by Windmill and shared by the whole workspace: scripts, flows and apps reach it by name, `main` being the default. It is configured in the workspace settings (Data Tables), never created by SQL.\n\n## Changing the schema\n\nA data table with migrations enabled keeps its schema as an ordered list of versioned SQL migrations, each an `up` and an optional `down`. That list is what git sync and a workspace merge carry, and what rebuilds the schema in another workspace. So first find out whether migrations are enabled on the data table:\n\n- **Enabled**: make every schema change (CREATE, ALTER, DROP, anything but row data) as a new migration. A change made with plain SQL exists in that one database only, and the history no longer describes it.\n- **Not enabled**: schema changes are plain SQL. Enabling migrations is a workspace admin's decision, made in the Data Tables settings, not a step to take on the way to something else.\n\nWriting a migration:\n\n- Wrap the statements in `BEGIN; ... END;` so it applies atomically. Leave the frame out only for a statement that cannot run in a transaction, such as `CREATE INDEX CONCURRENTLY`: such a migration can fail half applied.\n- Give it a `down` that undoes the `up` whenever the change is reversible. Without one it cannot be rolled back.\n- Declare the schemas it changes on a leading line of its `up`, `-- schemas sales, billing`, and qualify every name with its schema (`sales.orders`). A migration waits only on earlier migrations whose schemas overlap its own. One that declares none applies to the whole database and every later migration waits for it, so leave the line out only for a change that really spans the database, such as an extension.\n- Migrations apply oldest first. One that is applied is not edited: correct it with a new migration.\n- Row data is not schema. Insert, update and delete rows with plain SQL, migrations or not.\n\n## A migration is not a draft\n\nA migration is saved to the workspace and changes the database as soon as it is applied, for everything that reads those tables. To try a schema out without touching tables already in use:\n\n- put the new tables in a schema of their own, created by the same migration (`CREATE SCHEMA IF NOT EXISTS`), and point what you are building at it. A raw app names its schema in its `data` config, which is why a new app gets a new schema by default;\n- or work in a workspace fork whose data table was cloned when the fork was created (\"Clone schema only\" or \"Clone schema and data\"): migrations applied there change that copy alone, and reach the parent workspace when the fork is merged. A fork that kept the original data table points at the parent's database, so a migration applied in it changes the parent's tables at once.\n\nAltering or dropping a table other scripts or apps use is neither of those: say what depends on it before doing it.\n\n## Roles and permissions\n\nData table roles are a self-hosted Enterprise Edition feature, for data tables on managed Postgres (the instance database or the external managed instance). Without them, and always on a data table backed by a Postgres resource, everyone who can use the data table reaches all of it: there is no per-user, per-schema or per-row access to promise.\n\nA data table under roles is reached as a Postgres role, and Postgres enforces what that role was granted:\n\n- roles are assigned, per data table, to users, groups or folders. `admin` is the connection the data table had before roles: it owns the existing objects and ignores grants;\n- a connection either names a role (`wmill.datatable('main', { role: 'analyst' })`, `role='analyst'` in Python, `datatable://main?role=analyst`) and holds that role alone, or names none and is resolved from the data table's default role;\n- `permission denied` means the role in use lacks a grant. Say which role and which object: the grant is a data table admin's to give, in the roles drawer of the Database Manager. Do not switch to `admin` to get past it;\n- a migration runs as the role its SQL declares on its first line, `-- role `, the same one in its `down`. One that declares none runs as `admin`, which only those who may use `admin` can do.\n\nRead the Permissions section of https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables before designing who reaches what: it covers tenants, grants, the default role and the roles a connection that names none ends up holding, rows that differ per viewer through row-level security, and how forks and raw apps behave under roles.\n\n\n# Datatable SQL SDK reference\n\nDatatables are workspace-scoped managed PostgreSQL databases. In chat, explore and shape them with the datatable tools you were given. The reference below is for code you author inside runnables (inline app runnables, scripts, or flow rawscript modules) that reads or writes datatable data at runtime.\n\n- A runnable accesses a datatable via `wmill.datatable()` (the default \"main\") or `wmill.datatable('')`, referencing tables as `schema.table`.\n- Use parameterized queries (the tagged template in TypeScript, `$1`/`$2` placeholders in Python) — never interpolate untrusted values into SQL strings.\n\n## TypeScript Datatable API (windmill-client)\n\nImport: `import * as wmill from 'windmill-client'`\n\nSQL statement object with query content, arguments, and execution methods\n```typescript\ntype SqlStatement = {\n /** Raw SQL content with formatted arguments */\n content: string;\n\n /** Argument values keyed by parameter name */\n args: Record;\n\n /**\n * Execute the SQL query and return results\n * @param params - Optional parameters including result collection mode\n * @returns Query results based on the result collection mode\n */\n fetch(\n params?: FetchParams // The union is for auto-completion\n ): Promise>;\n\n /**\n * Execute the SQL query and return only the first row\n * @param params - Optional parameters\n * @returns First row of the query result\n */\n fetchOne(\n params?: Omit, \"resultCollection\">\n ): Promise>;\n\n /**\n * Execute the SQL query and return only the first row as a scalar value\n * @param params - Optional parameters\n * @returns First row of the query result\n */\n fetchOneScalar(\n params?: Omit<\n FetchParams<\"last_statement_first_row_scalar\">,\n \"resultCollection\"\n >\n ): Promise>;\n\n /**\n * Execute the SQL query without fetching rows\n * @param params - Optional parameters\n */\n execute(\n params?: Omit, \"resultCollection\">\n ): Promise;\n};\n```\n\n```typescript\n// Template tag function: sql`SELECT * FROM table WHERE id = ${id}`.fetch()\ninterface DatatableSqlTemplateFunction {\n // Tagged template usage:\n (strings: TemplateStringsArray, ...values: any[]): SqlStatement;\n query(sql: string, ...params: any[]): SqlStatement;\n};\n```\n\nCreate a SQL template function for PostgreSQL/datatable queries\n@param name - Database/datatable name (default: \"main\")\n@param opts.role - Connect as this data table role instead of the data table's default one.\n Only meaningful on a data table under roles, and only for a role you are a tenant of.\n@returns SQL template function for building parameterized queries\n@example\nlet sql = wmill.datatable()\nlet name = 'Robin'\nlet age = 21\nawait sql`\n SELECT * FROM friends\n WHERE name = ${name} AND age = ${age}::int\n`.fetch()\n@example\n// Read through a restricted role\nlet sql = wmill.datatable(\"main\", { role: \"analytics\" })\n```typescript\nfunction datatable(name: string = \"main\",\n opts?: DatatableOptions): DatatableSqlTemplateFunction\n```\n\n\n## Python Datatable API (wmill)\n\nImport: `import wmill`\n\n# Get a DataTable client for SQL queries.\n# \n# Args:\n# name: Database name (default: \"main\")\n# role: Connect as this data table role instead of the data table's default one.\n# \n# Returns:\n# DataTableClient instance\ndef datatable(name: str = 'main', *, role: Optional[str] = None) -> DataTableClient\n\n# Client for executing SQL queries against Windmill DataTables.\nclass DataTableClient:\n # Initialize DataTableClient.\n # \n # Args:\n # client: Windmill client instance\n # name: DataTable name\n # role: Data table role to connect as, or None for the data table's default\n def __init__(client: Windmill, name: str, role: Optional[str] = None)\n\n # Execute a SQL query against the DataTable.\n # \n # Args:\n # sql: SQL query string with $1, $2, etc. placeholders\n # *args: Positional arguments to bind to query placeholders\n # \n # Returns:\n # SqlQuery instance for fetching results\n def query(sql: str, *args) -> SqlQuery\n\n\n# Query result handler for DataTable and DuckLake queries.\nclass SqlQuery:\n # Initialize SqlQuery.\n # \n # Args:\n # sql: SQL query string\n # fetch_fn: Function to execute the query\n def __init__(sql: str, fetch_fn)\n\n # Execute query and fetch results.\n # \n # Args:\n # result_collection: Optional result collection mode\n # \n # Returns:\n # Query results\n def fetch(result_collection: str | None = None)\n\n # Execute query and fetch first row of results.\n # \n # Returns:\n # First row of query results\n def fetch_one()\n\n # Execute query and fetch first row of results. Return result as a scalar value.\n # \n # Returns:\n # First row of query result as a scalar value\n def fetch_one_scalar()\n\n # Execute query and don't return any results.\n # \n def execute()\n\n\n", "pipeline": "# Data pipeline authoring\n\nA **data pipeline** is NOT a flow. A flow is one runnable that orchestrates steps internally. A data pipeline is a set of **independent scripts**, each deployed on its own, that form a DAG by reading and writing shared **storage assets** (DuckLake tables, data tables, S3 objects, volumes, resources) and by declaring execution **triggers**. The pipeline is visualized and edited at `/pipeline/`; every node is a normal workspace script that happens to carry pipeline annotations. When the user asks for a \"data pipeline\" (or to \"ingest / transform / materialize\" data across steps), build pipeline-annotated scripts — do NOT build a flow.\n\n## Default to DuckDB + DuckLake\n\nA pipeline node that produces a table should almost always be a **`duckdb`** node that materializes its output into a **DuckLake** table with `-- materialize ducklake:///` (in a DuckDB node the annotation uses SQL `--` comment syntax; write the body as a bare `SELECT` and let the runtime do the write). DuckLake is the default lakehouse store for pipelines and is the shape the pipeline editor is built around, so prefer it unless the work specifically calls for something else:\n\n- `postgresql` / data tables — only for row-level, OLTP-style mutations against an existing Postgres data table (frequent single-row upserts/updates, transactional reads that an app queries live).\n- `bun` / `python3` — only for work that doesn't map to SQL: calling an external API, wrangling files, arbitrary glue. When such a node produces tabular data for downstream steps, write it **straight into DuckLake** (or a data table) from the same script — see \"Ingesting from Python / TypeScript\" below — rather than inventing a parallel store.\n\nDo not spread a pipeline across postgres, S3, and DuckLake when one DuckLake lake would do; a consistent DuckLake lakehouse is the goal.\n\n## Ingesting from Python / TypeScript\n\nA node that fetches data in code (an API, a SaaS export, a scraper) writes the rows **directly** into their DuckLake table with `wmill.ducklake()`, or into a data table with `wmill.datatable()`. DuckLake already stores every table as parquet files in the workspace object storage, so **never stage the raw payload first** — no intermediate JSON / CSV / parquet file in S3 and no second \"load the raw file\" node. One ingestion node, one table.\n\nPass the rows as a query argument (a list of dicts / array of objects is sent as JSON) and let SQL unnest them; declare the column types so the table gets a real schema:\n\n```python\n# pipeline\n# on schedule\nimport wmill\n\ndef main():\n rows = fetch_charges() # list[dict]\n lake = wmill.ducklake()\n lake.query(\"\"\"\n CREATE OR REPLACE TABLE stripe_charges AS\n SELECT unnest(from_json($rows, '[{\"id\":\"VARCHAR\",\"amount\":\"BIGINT\",\"created\":\"BIGINT\"}]'), recursive := true)\n \"\"\", rows=rows).fetch()\n```\n\n```ts\n// pipeline\n// on schedule\nimport * as wmill from \"windmill-client\"\n\nexport async function main() {\n const rows = await fetchCharges()\n const lake = wmill.ducklake()\n await lake`CREATE OR REPLACE TABLE stripe_charges AS\n SELECT unnest(from_json(${rows}, '[{\"id\":\"VARCHAR\",\"amount\":\"BIGINT\",\"created\":\"BIGINT\"}]'), recursive := true)`.fetch()\n}\n```\n\n- Use `INSERT INTO
SELECT ...` instead of `CREATE OR REPLACE` to append (an incremental pull), with a one-time `CREATE TABLE IF NOT EXISTS` first.\n- Assign the client to a variable (`lake = wmill.ducklake()`) and keep the SQL a **string literal** (Python) or a template without `sql.raw` (TS): that is what lets the graph record the node's write to `ducklake://main/
`. A chained `wmill.ducklake().query(...)` or SQL built in a variable records no output, so the downstream edge never forms.\n- `wmill.ducklake(\"\")` targets a non-default catalog; `wmill.datatable(\"\")` writes a Postgres data table the same way (Python `db.query(sql, *args)` with `$1` placeholders, TS the same template form) — use it only when the data belongs in Postgres (see above).\n- The downstream node triggers on the table itself: `-- on ducklake://main/stripe_charges`.\n\n## Storage prerequisites\n\nA DuckLake pipeline only runs once the workspace has **object storage** (S3 / Azure Blob / GCS) **and a DuckLake catalog** configured — DuckLake tables and `s3://` assets can't be materialized or read without it. Check which DuckLake catalogs the workspace has before you build.\nDrafting the annotated scripts does not require storage, but the pipeline can't ingest, materialize, or read its assets until it exists. So if there is none (or the user hits \"storage not configured\" errors), say so and give the right next step **by role**:\n\n- a workspace **admin** sets it up in Workspace settings → Object Storage (add an S3/Azure/GCS storage), then adds a DuckLake catalog on top of it;\n- anyone **without admin rights** should ask a workspace admin to configure object storage + a DuckLake catalog.\n\nNever hand back a DuckLake pipeline that cannot run without flagging the missing storage and pointing to who sets it up.\n\n## What makes a script a pipeline node\n\nA script joins the pipeline when its source begins with the `pipeline` annotation as a top-of-file comment, **written in the script's own comment syntax** — `//` for TS/JS (bun), `--` for SQL (DuckDB/Postgres), `#` for Python/Bash. So it's `-- pipeline` in a DuckDB node, `# pipeline` in a Python node, `// pipeline` in a bun node. Every annotation below uses that same prefix (the `//` shown is the TS form). All other wiring is expressed as annotation comments near the top of the file:\n\n- `// on ` — declares an execution-DAG **input** (what triggers/feeds this node). `` is either:\n - an **asset URI** (the node runs when that asset is produced upstream): `ducklake://main/orders`, `datatable://main/users`, `$res:f/folder/my_resource`, `volume://name/path`, or an S3 object (see the S3 storage-form rule below).\n - a **native trigger kind**: `schedule`, `webhook`, `email`, `kafka`, `mqtt`, `amqp`, `nats`, `postgres`, `sqs`, `gcp`, or `data_upload` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. **`data_upload` is special**: there is no trigger row — the node instead declares an **`S3Object` input parameter** fed by the auto-generated upload picker; it never hard-codes a key. Any language can be the `data_upload` node:\n - Python (has `import wmill`): `def main(file: wmill.S3Object):` then `wmill.load_s3_file(file)`; TS (has `import * as wmill from \"windmill-client\"`): `export async function main(file: wmill.S3Object)`. Qualify the type as `wmill.S3Object` (or add `from wmill import S3Object` / `import { S3Object } from \"windmill-client\"`) — a bare `S3Object` is undefined.\n - **DuckDB** takes the s3object arg via a `-- $ (s3object)` declaration and reads it directly, so a single DuckDB node can ingest **and** materialize:\n ```\n -- pipeline\n -- on data_upload\n -- materialize ducklake://main/raw_uploads\n -- $file (s3object)\n SELECT * FROM read_csv($file)\n ```\n- **Outputs** are inferred from what the body writes — a `CREATE TABLE`, a `wmill.writeS3File(...)`, a DuckLake/datatable write. To declare a managed output explicitly, use `// materialize `.\n- Optional badges: `// partitioned `, `// freshness ` (e.g. `1h`), `// tag `, `// retry [delay]`, `// data_test ...` (managed DuckLake targets only — deploy rejects it beside a `dbt://` one), `// measure = [where ]` and `// dimension = ` (see \"Declared metrics\" below).\n\n## S3 object wiring (storage form matters)\n\nAn `s3://` URI's first slashes select the **storage**, not part of the key — get this wrong and the producer/consumer edge silently won't connect:\n\n- `s3:///` (**triple** slash, empty first segment) = the **default** workspace storage. A downstream node reading or triggering on that object uses `s3:///` — e.g. DuckDB `-- on s3:///orders/2024.parquet` and `read_parquet('s3:///orders/2024.parquet')`.\n- `s3:///` (**double** slash, non-empty first segment) = a **named secondary** storage called `` — so `s3://ingest/x` means storage `ingest`, key `x`, NOT key `ingest/x`. Only use this when the object genuinely lives in a configured secondary storage; never invent a bucket/storage name for a default-storage object (it breaks the edge).\n\nTo make the producer side visible to lineage, a Python/TS node MUST pass the **`S3Object` form**, not a bare key string: Python `wmill.write_s3_file(wmill.S3Object(s3=\"\"), data)` (or the import-free dict `{\"s3\": \"\"}`), TS `wmill.writeS3File({ s3: \"\" }, data)`. That records the default-storage asset `/`, which a downstream `s3:///` reader connects to (same key both sides). A bare `write_s3_file(\"\", ...)` records **no** asset and produces **no** edge. Add `storage=\"\"` only for a named secondary storage.\n\nThe key must be a **string literal** — the graph parser is static and cannot follow a variable, f-string, or computed path, so `write_s3_file(wmill.S3Object(s3=key_var), ...)` records no edge. Inline the literal (`s3=\"events/user_events.parquet\"`) on both the writing and reading node. The same rule applies to every asset URI in an annotation or SDK call (`ducklake://`, `datatable://`, `s3://`): write them literally, not via a variable.\n\n## Materialize (the managed output)\n\n> **A MANAGED `// materialize` is DuckDB-only**, and its target must be a DuckLake table (`ducklake:///
`). Deploy **rejects** a `ducklake://` `// materialize` on any other language (`python3`, `bun`, `postgresql`). For a non-DuckDB node writing the lake, do **not** use `// materialize` — write the table itself with `wmill.ducklake()` (see \"Ingesting from Python / TypeScript\") and let the output be inferred. Use `duckdb` when a node should materialize a DuckLake table.\n>\n> The one target ANY language may declare (except a dbt script, whose writes come from its manifest) is a **warehouse relation**: `// materialize manual dbt:////`, where `` is a warehouse the workspace configures under Settings → dbt. `manual` is the only mode it has — nothing generates warehouse DDL, so the node issues its own write (a postgresql `CREATE TABLE` / `INSERT`, an SDK load, …) and the annotation records the outcome. Use it on an ingestion node whose output a dbt project reads as a `source`: the declared relation and the dbt model land on ONE graph node, and a downstream `// on dbt:////` fires when the ingestion node completes.\n\nA managed `// materialize ducklake:///
` tells the runtime to write the node's output table **for you**: write the body as a single `SELECT` and the runtime wraps it in the create/replace — do **not** also write your own `CREATE TABLE` / `INSERT`. (The opposite holds for the `dbt://` target above: there the node writes its own DDL and the strategies below do not apply.) Write strategy:\n\n- no option → **replace** the whole table each run (full refresh; the only mode whose output columns may change);\n- `// materialize append` → INSERT-append rows (incremental);\n- `// materialize key=` → merge/upsert on ``.\n\n`// materialize manual ` opts **out** of managed writes — the script writes its own DDL and the annotation only records the output asset for lineage.\n\n`materialize` pairs with partitioning for incremental pipelines: a `// partitioned ` node runs **once per partition** (append/merge into a fixed-schema table). The `{partition}` token, usable in any asset URI **and** in the body SQL, is replaced at run time by the current partition's **identity string**:\n\n- To filter the source to the active slice on a time grain, use the runtime-injected macro: `WHERE wm_partition() = {partition}`. `wm_partition(ts)` buckets a timestamp in exactly the identity format the runtime uses for daily/hourly/weekly/monthly, so it always matches; never hand-write a `strftime` format.\n- Do NOT write `= TIMESTAMP {partition}`: the identity string is not a valid timestamp literal for hourly/weekly/monthly and errors at run time.\n- For `dynamic` partitioning the identity is the caller-supplied key (not a timestamp, no macro), so filter on it directly: `WHERE = {partition}`.\n\n`materialize` is an output **declaration** on a node — not a command. There is no \"materialize run\".\n\n## Declared metrics (`measure` / `dimension`)\n\nOn a node that materializes a DuckLake table, `// measure = [where ]` names the canonical way to aggregate that table (e.g. `// measure revenue = sum(amount) where not is_refund`), and `// dimension = ` names a way to slice it (e.g. `// dimension region = region`, `// dimension month = date_trunc('month', ordered_at)`). They execute nothing: they are catalogued at deploy so the editor and other agents reuse the definition instead of re-deriving it and silently disagreeing.\n\n- Keep the predicate in the `where` clause rather than folding it into the aggregate: it is rendered as ` FILTER (WHERE )`, which is what lets two measures with different predicates share one GROUP BY.\n- DuckLake-only, and only meaningful next to `// materialize`.\n- Declare one when a number carries a judgement call someone else would get wrong (refunds excluded, test rows dropped, which column is the amount); do NOT blanket every table with measures — an obvious `count(*)` earns nothing.\n- To use a metric another node declares, read that node and reuse its exact expression rather than guessing it.\n\n## How to build one\n\n1. Put every node in the **same folder**: `f//`. The folder is the pipeline.\n2. Write each node as its own script. Default to `duckdb` materializing into DuckLake (see \"Default to DuckDB + DuckLake\" above); pick `postgresql`, `bun`, or `python3` only when that section says the work calls for it.\n3. Start each body with `// pipeline`, then the `// on` input declarations, then the transform that writes the output.\n4. **Chain nodes by asset URI**: read an upstream node's output asset, then `// on ` in the downstream node so the edge forms. Reuse exact asset paths from existing nodes rather than inventing parallel ones.\n5. Don't deploy nodes unless the user asks to. A pipeline only \"runs\" once its scripts are deployed and their triggers exist.\n\n## Example (DuckDB → DuckLake, scheduled ingest + downstream transform)\n\nNode `f/sales/orders_ingest` (runs on a schedule, materializes a DuckLake table):\n\n```sql\n-- pipeline\n-- on schedule\n-- materialize ducklake://main/orders\nSELECT * FROM read_csv('s3:///raw/orders/*.csv')\n```\n\nNode `f/sales/orders_daily` (runs when `orders` is produced, writes a rollup):\n\n```sql\n-- pipeline\n-- on ducklake://main/orders\n-- materialize ducklake://main/orders_daily\nSELECT date_trunc('day', ts) AS day, count(*) AS n\nFROM ducklake.main.orders GROUP BY 1\n```\n", - "workspace_settings": "# Workspace settings\n\nEverything an admin sets on the Workspace settings page is one document: a mapping of top-level\nkeys, the same layout as the `settings.yaml` file `wmill sync` keeps. Only a workspace admin can\nread or change it.\n\n## Applying a document\n\n- A partial document changes only the keys it states; every key it omits is left as it is.\n- Each key is replaced whole, so read its current value and send it back edited: a nested object\n given with one field replaces the stored object with that one field.\n- A key set to `null` clears that setting; an on/off switch cleared this way reads as `false`.\n- Each setting is written through the same path the settings page uses, with the same validation\n and audit log. The writes are not one transaction: when one fails the apply stops, and the\n error says what was already written.\n- A dry run reports the changes, as `before` / `after` per key, without writing anything.\n- `name` is never applied: rename a workspace on the settings page. `color` is applied only when\n set to a value, so `null` does not clear it.\n- A key the server does not know is ignored and named in the warnings: check them, since a typo\n otherwise reads as \"nothing to change\".\n- A setting this edition does not offer (Enterprise only) is refused, by the dry run too.\n\n## Secrets\n\n`slack_oauth_client_secret` is the one secret. A read shows `__SENSITIVE_AND_UNCHANGED__` in place\nof a set secret; sending that placeholder back keeps the stored value. A value starting with\n`$encrypted:` is decrypted with the workspace key, and one encrypted with another workspace's key\nis left unchanged with a warning; any other value is taken as plaintext. Never\nwrite a secret you were not given: keep the placeholder.\n\n## Keys\n\n| Key | Value |\n|---|---|\n| `auto_invite` | `{ enabled, mode: invite \\| add, operator, instance_groups?, instance_groups_roles? }`. The instance groups are managed only when `instance_groups` is stated; clearing a group removes the members it added. |\n| `error_handler` / `success_handler` | `{ path, extra_args?, muted_on_cancel?, muted_on_user_path? }`, `path` being `script/` or `flow/`. |\n| `error_handler_fallback_to_instance_alerts` | Report failures to the instance critical alert channels when no error handler is set. |\n| `webhook` | URL POSTed a JSON event on every change to the workspace's items. |\n| `ai_config` | Providers, default and completion models, custom prompts, `copilot_disabled`. |\n| `large_file_storage` | Workspace object storage: `{ type, s3_resource_path \\| azure_blob_resource_path \\| gcs_resource_path, public_resource?, secondary_storage?, advanced_permissions? }`. |\n| `git_sync` | `{ repositories: [...] }`, each with `git_repo_resource_path`, `script_path` and `settings` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. |\n| `datatable` | `{ datatables: { : { database: { resource_type, resource_path } } } }`, both required: `resource_type: postgresql` with `resource_path` a Postgres resource's path, or `instance`, or `external_instance` (Enterprise only), with `resource_path` the name of a database a superadmin set up. A fork's entry can instead be `{ reference: { workspace_id, datatable } }`, pointing at its parent's data table; keep it as is. `main` is the one scripts use by default. Role permissions are `datatable_permissions`. |\n| `datatable_permissions` | `{ : { default_role, roles: { : [tenants] } } }`: who may connect to each data table as which role. Tenants are `u/`, `g/`, `f/` or `*`; `admin` is always a role, and `default_role: null` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. |\n| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...] } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. |\n| `ducklake` | `{ ducklakes: { : { catalog, storage, ... } } }`. |\n| `dbt_warehouses` | dbt warehouse connections. |\n| `default_app` | Path of the app shown as the workspace home. |\n| `default_scripts` | `{ order?, hidden?, default_script_content? }` for the script language picker. |\n| `deploy_ui` | `{ include_path?, include_type? }`: what the deploy-to-workspace UI offers. |\n| `operator_settings` | What operators see and do: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). A write right the value omits keeps its stored value. |\n| `mute_critical_alerts` | Mute the workspace's critical alerts. |\n| `slack_team_id`, `slack_name` | Read only: set by connecting Slack on the settings page. |\n| `slack_command_script` / `teams_command_script` | Script or flow handling the Slack / Teams command. |\n| `slack_oauth_client_id` / `slack_oauth_client_secret` | The workspace's own Slack OAuth app. Clearing either removes both. |\n| `public_app_execution_limit_per_minute` | Rate limit for public apps, `null` for none. |\n| `guest_access_enabled`, `guest_jwt_public_key`, `guest_jwt_jwks_url` | Guest access to public apps and the key that signs guest JWTs. |\n| `add_admins_and_developers_to_forks` | Add the workspace's admins and developers to each fork. |\n| `color` | Workspace color, `#rrggbb`. |\n" + "workspace_settings": "# Workspace settings\n\nEverything an admin sets on the Workspace settings page is one document: a mapping of top-level\nkeys, the same layout as the `settings.yaml` file `wmill sync` keeps. Only a workspace admin can\nread or change it.\n\n## Applying a document\n\n- A partial document changes only the keys it states; every key it omits is left as it is.\n- Each key is replaced whole, so read its current value and send it back edited: a nested object\n given with one field replaces the stored object with that one field.\n- A key set to `null` clears that setting; an on/off switch cleared this way reads as `false`.\n- Each setting is written through the same path the settings page uses, with the same validation\n and audit log. The writes are not one transaction: when one fails the apply stops, and the\n error says what was already written.\n- A dry run reports the changes, as `before` / `after` per key, without writing anything.\n- `name` is never applied: rename a workspace on the settings page. `color` is applied only when\n set to a value, so `null` does not clear it.\n- A key the server does not know is ignored and named in the warnings: check them, since a typo\n otherwise reads as \"nothing to change\".\n- A setting this edition does not offer (Enterprise only) is refused, by the dry run too.\n\n## Secrets\n\n`slack_oauth_client_secret` is the one secret. A read shows `__SENSITIVE_AND_UNCHANGED__` in place\nof a set secret; sending that placeholder back keeps the stored value. A value starting with\n`$encrypted:` is decrypted with the workspace key, and one encrypted with another workspace's key\nis left unchanged with a warning; any other value is taken as plaintext. Never\nwrite a secret you were not given: keep the placeholder.\n\n## Keys\n\n| Key | Value |\n|---|---|\n| `auto_invite` | `{ enabled, mode: invite \\| add, operator, instance_groups?, instance_groups_roles? }`. The instance groups are managed only when `instance_groups` is stated; clearing a group removes the members it added. |\n| `error_handler` / `success_handler` | `{ path, extra_args?, muted_on_cancel?, muted_on_user_path? }`, `path` being `script/` or `flow/`. |\n| `error_handler_fallback_to_instance_alerts` | Report failures to the instance critical alert channels when no error handler is set. |\n| `webhook` | URL POSTed a JSON event on every change to the workspace's items. |\n| `ai_config` | Providers, default and completion models, custom prompts, `copilot_disabled`. |\n| `large_file_storage` | Workspace object storage: `{ type, s3_resource_path \\| azure_blob_resource_path \\| gcs_resource_path, public_resource?, secondary_storage?, advanced_permissions? }`. |\n| `git_sync` | `{ repositories: [...] }`, each with `git_repo_resource_path`, `script_path` and `settings` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. |\n| `datatable` | `{ datatables: { : { database: { resource_type, resource_path } } } }`, both required: `resource_type: postgresql` with `resource_path` a Postgres resource's path, or `instance`, or `external_instance` (Enterprise only), with `resource_path` the name of a database a superadmin set up. A fork's entry can instead be `{ reference: { workspace_id, datatable } }`, pointing at its parent's data table; keep it as is. `main` is the one scripts use by default. Role permissions are `datatable_permissions`. |\n| `datatable_permissions` | `{ : { default_role, roles: { : [tenants] } } }`: who may connect to each data table as which role. Tenants are `u/`, `g/`, `f/` or `*`; `admin` is always a role, and `default_role: null` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. |\n| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. |\n| `ducklake` | `{ ducklakes: { : { catalog, storage, ... } } }`. |\n| `dbt_warehouses` | dbt warehouse connections. |\n| `default_app` | Path of the app shown as the workspace home. |\n| `default_scripts` | `{ order?, hidden?, default_script_content? }` for the script language picker. |\n| `deploy_ui` | `{ include_path?, include_type? }`: what the deploy-to-workspace UI offers. |\n| `operator_settings` | What operators see and do: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). A write right the value omits keeps its stored value. |\n| `mute_critical_alerts` | Mute the workspace's critical alerts. |\n| `slack_team_id`, `slack_name` | Read only: set by connecting Slack on the settings page. |\n| `slack_command_script` / `teams_command_script` | Script or flow handling the Slack / Teams command. |\n| `slack_oauth_client_id` / `slack_oauth_client_secret` | The workspace's own Slack OAuth app. Clearing either removes both. |\n| `public_app_execution_limit_per_minute` | Rate limit for public apps, `null` for none. |\n| `guest_access_enabled`, `guest_jwt_public_key`, `guest_jwt_jwks_url` | Guest access to public apps and the key that signs guest JWTs. |\n| `add_admins_and_developers_to_forks` | Add the workspace's admins and developers to each fork. |\n| `color` | Workspace color, `#rrggbb`. |\n" } } diff --git a/backend/windmill-api/src/workspace_settings_document.rs b/backend/windmill-api/src/workspace_settings_document.rs index d222188319..cb0c2e67aa 100644 --- a/backend/windmill-api/src/workspace_settings_document.rs +++ b/backend/windmill-api/src/workspace_settings_document.rs @@ -262,8 +262,8 @@ fn datatable_permissions_document(datatable: Option<&Value>, roles: &DatatableNa /// Protection rules by name, their rules in a fixed order so a reordered file is no change. The /// dev workspace lock is left out: the dev workspace setting creates and removes it. async fn protection_rules_document(conn: &mut PgConnection, w_id: &str) -> Result { - let rows = sqlx::query_as::<_, (String, i32, Vec, Vec)>( - "SELECT name, rules, bypass_groups, bypass_users FROM workspace_protection_rule + let rows = sqlx::query_as::<_, (String, i32, Vec, Vec, Value)>( + "SELECT name, rules, bypass_groups, bypass_users, fork_role_mapping FROM workspace_protection_rule WHERE workspace_id = $1 AND name <> $2 ORDER BY name", ) .bind(w_id) @@ -272,19 +272,27 @@ async fn protection_rules_document(conn: &mut PgConnection, w_id: &str) -> Resul .await?; Ok(Value::Object( rows.into_iter() - .map(|(name, bits, bypass_groups, bypass_users)| { + .map(|(name, bits, bypass_groups, bypass_users, fork_role_mapping)| { let rules = ProtectionRules::from_bits_truncate(bits); - let rule = json!({ + let mut rule = json!({ "rules": rule_kinds(rules), "bypass_groups": bypass_groups, "bypass_users": bypass_users, }); + with_fork_role_mapping(&mut rule, fork_role_mapping); (name, rule) }) .collect(), )) } +/// Written only when set, so documents of rulesets that map no role read as they always did. +fn with_fork_role_mapping(rule: &mut Value, mapping: Value) { + if mapping.as_object().is_some_and(|m| !m.is_empty()) { + rule["fork_role_mapping"] = mapping; + } +} + fn rule_kinds(rules: ProtectionRules) -> Value { json!(rules.kinds()) } @@ -860,14 +868,16 @@ fn normalize_protection_rules(v: &Value, warnings: &mut Vec) -> Result = serde_json::from_value(rule.get("rules").cloned().unwrap_or(json!([]))) .map_err(|e| Error::BadRequest(format!("Invalid {what}.rules: {e}")))?; - out.insert( - name.clone(), - json!({ - "rules": rule_kinds(ProtectionRules::from(&kinds)), - "bypass_groups": string_list(rule.get("bypass_groups"), &format!("{what}.bypass_groups"))?, - "bypass_users": string_list(rule.get("bypass_users"), &format!("{what}.bypass_users"))?, - }), - ); + let fork_role_mapping: std::collections::BTreeMap = + serde_json::from_value(rule.get("fork_role_mapping").cloned().unwrap_or(json!({}))) + .map_err(|e| Error::BadRequest(format!("Invalid {what}.fork_role_mapping: {e}")))?; + let mut normalized = json!({ + "rules": rule_kinds(ProtectionRules::from(&kinds)), + "bypass_groups": string_list(rule.get("bypass_groups"), &format!("{what}.bypass_groups"))?, + "bypass_users": string_list(rule.get("bypass_users"), &format!("{what}.bypass_users"))?, + }); + with_fork_role_mapping(&mut normalized, json!(fork_role_mapping)); + out.insert(name.clone(), normalized); } Ok(Value::Object(out)) } @@ -1761,6 +1771,7 @@ fn plan_protection_rules(p: &mut Plan, desired: Value) { "rules": r.get("rules"), "bypass_groups": r.get("bypass_groups"), "bypass_users": r.get("bypass_users"), + "fork_role_mapping": r.get("fork_role_mapping").cloned().unwrap_or(json!({})), }) }; match (before, after) { @@ -2178,7 +2189,8 @@ mod tests { json!({ "old": { "rules": ["DisableWorkspaceForking"], "bypass_groups": [], "bypass_users": [] } }), ); let d = doc(json!({ "protection_rules": { - "new": { "rules": ["RestrictPublicRunSharing", "DisableDirectDeployment"] }, + "new": { "rules": ["RestrictPublicRunSharing", "DisableDirectDeployment"], + "fork_role_mapping": { "operator": "developer" } }, "dev_workspace_lock": { "rules": [] }, } })); let p = plan_document(cur, d, false, &mc, &DatatableNames::default()).unwrap(); @@ -2186,7 +2198,8 @@ mod tests { assert_eq!( p.writes[0].body, json!({ "name": "new", "rules": ["DisableDirectDeployment", "RestrictPublicRunSharing"], - "bypass_groups": [], "bypass_users": [] }) + "bypass_groups": [], "bypass_users": [], + "fork_role_mapping": { "operator": "developer" } }) ); assert!(matches!( p.writes[1].target, diff --git a/backend/windmill-common/src/roles.rs b/backend/windmill-common/src/roles.rs index b3fd9937f4..d22ffd33fc 100644 --- a/backend/windmill-common/src/roles.rs +++ b/backend/windmill-common/src/roles.rs @@ -92,6 +92,10 @@ pub struct RoleCapabilities { pub apps: AuthorLevel, pub manage_schedules: bool, pub manage_triggers: bool, + /// Fork the workspace. Like visibility, it does not make a role restricted: it is read from + /// the role at the fork routes, never through [`caller_capabilities`]'s fast path. Left unset + /// by every layer, it is granted exactly to an unrestricted role (see [`Self::resolve`]). + pub fork: bool, pub visibility: RoleVisibility, } @@ -102,6 +106,7 @@ impl RoleCapabilities { apps: AuthorLevel::Full, manage_schedules: true, manage_triggers: true, + fork: true, visibility: RoleVisibility::ALL, }; @@ -113,6 +118,7 @@ impl RoleCapabilities { apps: AuthorLevel::None, manage_schedules: true, manage_triggers: true, + fork: false, visibility: RoleVisibility::ALL, }; @@ -124,6 +130,7 @@ impl RoleCapabilities { apps: AuthorLevel::None, manage_schedules: false, manage_triggers: false, + fork: false, visibility: RoleVisibility::NONE, }; @@ -141,7 +148,7 @@ impl RoleCapabilities { /// left out: it only hides pages, and must not turn a full author into an operator for the /// raw `is_operator` checks, job permissions and credential caches keyed on it. pub fn is_restricted(&self) -> bool { - Self { visibility: RoleVisibility::ALL, ..*self } != Self::FULL + Self { visibility: RoleVisibility::ALL, fork: true, ..*self } != Self::FULL } /// Whether the role authors deployable artifacts and so costs a full seat rather than half. @@ -164,6 +171,20 @@ impl RoleCapabilities { } } + /// Role `name` with `patches` applied over [`Self::base_for`], bottom layer first. A `fork` no + /// layer sets follows [`Self::is_restricted`]: roles that predate the capability keep the + /// forking they had, which every unrestricted role held and no restricted one. + pub fn resolve(name: &str, patches: &[&RoleCapabilitiesPatch]) -> Self { + let mut caps = Self::base_for(name); + for p in patches { + caps.apply(p); + } + if patches.iter().all(|p| p.fork.is_none()) { + caps.fork = !caps.is_restricted(); + } + caps + } + fn apply(&mut self, patch: &RoleCapabilitiesPatch) { if let Some(v) = patch.code { self.code = v; @@ -180,6 +201,9 @@ impl RoleCapabilities { if let Some(v) = patch.manage_triggers { self.manage_triggers = v; } + if let Some(v) = patch.fork { + self.fork = v; + } if let Some(v) = &patch.visibility { v.apply(&mut self.visibility); } @@ -213,6 +237,8 @@ pub struct RoleCapabilitiesPatch { #[serde(default, skip_serializing_if = "Option::is_none")] pub manage_triggers: Option, #[serde(default, skip_serializing_if = "Option::is_none")] + pub fork: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] pub visibility: Option, } @@ -279,6 +305,7 @@ pub fn diff_capabilities( apps: d(base.apps, target.apps), manage_schedules: d(base.manage_schedules, target.manage_schedules), manage_triggers: d(base.manage_triggers, target.manage_triggers), + fork: d(base.fork, target.fork), visibility: (visibility != RoleVisibilityPatch::default()).then_some(visibility), } } @@ -393,9 +420,7 @@ pub fn parse_instance_roles(value: &serde_json::Value) -> Result Result> { let mut out = Vec::with_capacity(instance.len() + overrides.len()); for def in instance { - let mut caps = RoleCapabilities::base_for(&def.name); - caps.apply(&def.capabilities); let ovr = overrides.get(&def.name); - if let Some(o) = ovr { - caps.apply(&o.capabilities); - } + let mut patches = vec![&def.capabilities]; + patches.extend(ovr.map(|o| &o.capabilities)); + let caps = RoleCapabilities::resolve(&def.name, &patches); out.push(EffectiveRole { label: ovr .and_then(|o| o.label.clone()) @@ -432,8 +455,7 @@ pub fn effective_roles( .collect(); extra.sort_by(|a, b| a.0.cmp(b.0)); for (name, o) in extra { - let mut caps = RoleCapabilities::base_for(name); - caps.apply(&o.capabilities); + let caps = RoleCapabilities::resolve(name, &[&o.capabilities]); out.push(EffectiveRole { name: name.clone(), label: o.label.clone().unwrap_or_else(|| name.clone()), @@ -721,6 +743,36 @@ pub async fn caller_capabilities( } } +/// Whether the caller's role in `workspace_id` holds [`RoleCapabilities::fork`]. Read from the role +/// even for an unrestricted caller, since withdrawing `fork` alone leaves a role unrestricted. A +/// caller with no membership is left to the fork routes' other gates. +pub async fn caller_can_fork( + db: &DB, + workspace_id: &str, + authed: &A, +) -> Result { + if authed.is_admin() { + return Ok(true); + } + // Not through `member_role`: its cache would then hold every forker, restricted or not. + let row = sqlx::query!( + "SELECT workspace_role, operator FROM usr WHERE workspace_id = $1 AND username = $2", + workspace_id, + authed.username() + ) + .fetch_optional(db) + .await?; + match row { + Some(r) => { + let role = resolve_role_name(r.workspace_role, r.operator); + Ok(role_capabilities_by_name(db, workspace_id, &role) + .await? + .fork) + } + None => Ok(true), + } +} + pub fn invalidate_workspace_roles_cache(workspace_id: &str) { WORKSPACE_ROLES.remove(workspace_id); } @@ -793,6 +845,21 @@ pub async fn ensure_role_unused(db: &DB, w_id: Option<&str>, name: &str) -> Resu "role `{name}` is still held by {users} member(s) and {invites} pending invite(s), or named by {settings} workspace auto-invite or instance group mapping(s): reassign them first" ))); } + let mapping_rulesets = sqlx::query_scalar!( + "SELECT COUNT(*) FROM workspace_protection_rule WHERE ($1::text IS NULL OR workspace_id = $1) + AND (fork_role_mapping ? $2 + OR EXISTS (SELECT 1 FROM jsonb_each_text(fork_role_mapping) m WHERE m.value = $2))", + w_id, + name + ) + .fetch_one(db) + .await? + .unwrap_or(0); + if mapping_rulesets > 0 { + return Err(Error::BadRequest(format!( + "role `{name}` is named by the fork role mapping of {mapping_rulesets} protection ruleset(s): edit them first" + ))); + } // An override left behind would silently re-base onto `base_for` instead of the instance // definition it was written against. if w_id.is_none() { @@ -986,6 +1053,20 @@ mod tests { assert!(op.capabilities.is_author()); } + #[test] + fn unset_fork_follows_restriction() { + let instance: Vec = serde_json::from_value(serde_json::json!([ + {"name": "developer"}, + {"name": "operator"}, + {"name": "maintainer", "capabilities": {"code": true, "flows": "full", "apps": "full"}}, + {"name": "auditor", "capabilities": {"fork": true}} + ])) + .unwrap(); + let roles = effective_roles(&instance, &RoleOverrides::new()).unwrap(); + let fork: Vec = roles.iter().map(|r| r.capabilities.fork).collect(); + assert_eq!(fork, [true, false, true, true]); + } + #[test] fn full_authoring_requires_code() { let err = parse_instance_roles(&serde_json::json!([ diff --git a/backend/windmill-common/src/workspaces.rs b/backend/windmill-common/src/workspaces.rs index f905657849..898446f0c3 100644 --- a/backend/windmill-common/src/workspaces.rs +++ b/backend/windmill-common/src/workspaces.rs @@ -63,8 +63,13 @@ pub struct ProtectionRuleset { pub rules: ProtectionRules, pub bypass_groups: Vec, pub bypass_users: Vec, + /// Role a member of the workspace gets in a fork of it, keyed by the role it holds here. + /// Unmapped roles carry over unchanged. + pub fork_role_mapping: ForkRoleMapping, } +pub type ForkRoleMapping = sqlx::types::Json>; + bitflags::bitflags! { #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] // #[sqlx(transparent)] @@ -1185,7 +1190,8 @@ pub async fn get_protection_rules( name, rules as "rules: ProtectionRules", bypass_groups, - bypass_users + bypass_users, + fork_role_mapping as "fork_role_mapping: ForkRoleMapping" FROM workspace_protection_rule WHERE workspace_id = $1 ORDER BY name @@ -1475,6 +1481,21 @@ pub async fn check_user_against_rule( Ok(RuleCheckResult::Allowed) } +/// The roles members of `workspace_id` get in a fork of it, merged across its rulesets: where two +/// rulesets map the same role, the first by name wins. Unmapped roles carry over unchanged. +pub async fn fork_role_mapping( + workspace_id: &str, + db: &DB, +) -> Result> { + let mut mapping = std::collections::BTreeMap::new(); + for ruleset in get_protection_rules(workspace_id, db).await?.iter() { + for (from, to) in ruleset.fork_role_mapping.iter() { + mapping.entry(from.clone()).or_insert_with(|| to.clone()); + } + } + Ok(mapping) +} + /// Check all deploy-gating protection rules at once. /// /// Evaluates `DisableDirectDeployment` first (so its message wins when both diff --git a/cli/src/commands/protection-rules/converter.ts b/cli/src/commands/protection-rules/converter.ts index 0d3ece5e36..f5ba781a74 100644 --- a/cli/src/commands/protection-rules/converter.ts +++ b/cli/src/commands/protection-rules/converter.ts @@ -15,6 +15,16 @@ function sortedUnique(arr: readonly string[]): string[] { return [...new Set(arr)].sort(); } +// Absent when empty, so a file written before the field existed stays in sync. +function sortedMapping( + m: Record | undefined, +): Record | undefined { + const keys = Object.keys(m ?? {}).sort(); + return keys.length > 0 + ? Object.fromEntries(keys.map((k) => [k, m![k]])) + : undefined; +} + export class ProtectionRulesConverter { // Canonicalize a single rule so comparisons are insensitive to array order // and duplicates. @@ -24,6 +34,9 @@ export class ProtectionRulesConverter { rules: sortedUnique(entry.rules ?? []) as ProtectionRuleEntry["rules"], bypass_groups: sortedUnique(entry.bypass_groups ?? []), bypass_users: sortedUnique(entry.bypass_users ?? []), + ...(sortedMapping(entry.fork_role_mapping) + ? { fork_role_mapping: sortedMapping(entry.fork_role_mapping) } + : {}), }; } @@ -45,6 +58,7 @@ export class ProtectionRulesConverter { rules: [...(r.rules ?? [])], bypass_groups: [...(r.bypass_groups ?? [])], bypass_users: [...(r.bypass_users ?? [])], + fork_role_mapping: { ...(r.fork_role_mapping ?? {}) }, })), ); } @@ -62,7 +76,9 @@ export class ProtectionRulesConverter { na.bypass_groups.length === nb.bypass_groups.length && na.bypass_groups.every((v, i) => v === nb.bypass_groups[i]) && na.bypass_users.length === nb.bypass_users.length && - na.bypass_users.every((v, i) => v === nb.bypass_users[i]) + na.bypass_users.every((v, i) => v === nb.bypass_users[i]) && + JSON.stringify(na.fork_role_mapping ?? {}) === + JSON.stringify(nb.fork_role_mapping ?? {}) ); } diff --git a/cli/src/commands/protection-rules/push.ts b/cli/src/commands/protection-rules/push.ts index 2ee253eaab..5aae9e0365 100644 --- a/cli/src/commands/protection-rules/push.ts +++ b/cli/src/commands/protection-rules/push.ts @@ -208,6 +208,7 @@ export async function pushProtectionRules( rules: n.rules, bypass_groups: n.bypass_groups, bypass_users: n.bypass_users, + fork_role_mapping: n.fork_role_mapping ?? {}, }, }); applied.created++; @@ -221,6 +222,7 @@ export async function pushProtectionRules( rules: n.rules, bypass_groups: n.bypass_groups, bypass_users: n.bypass_users, + fork_role_mapping: n.fork_role_mapping ?? {}, }, }); applied.updated++; diff --git a/cli/src/commands/protection-rules/types.ts b/cli/src/commands/protection-rules/types.ts index 0256f24e8f..8f476f2b96 100644 --- a/cli/src/commands/protection-rules/types.ts +++ b/cli/src/commands/protection-rules/types.ts @@ -10,6 +10,8 @@ export interface ProtectionRuleEntry { rules: ProtectionRuleKind[]; bypass_groups: string[]; bypass_users: string[]; + // Role a member gets in a fork of the workspace, keyed by the role it holds there. + fork_role_mapping?: Record; } // protection-rules.yaml is a flat map: workspace name -> its protection rules. diff --git a/cli/src/commands/protection-rules/utils.ts b/cli/src/commands/protection-rules/utils.ts index 9809e5808c..0fd35a9e34 100644 --- a/cli/src/commands/protection-rules/utils.ts +++ b/cli/src/commands/protection-rules/utils.ts @@ -43,6 +43,10 @@ function describeEntry(entry: ProtectionRuleEntry): string { if (n.bypass_users.length > 0) { parts.push(`bypass_users=[${n.bypass_users.join(", ")}]`); } + if (n.fork_role_mapping) { + const pairs = Object.entries(n.fork_role_mapping).map(([f, t]) => `${f}->${t}`); + parts.push(`fork_role_mapping=[${pairs.join(", ")}]`); + } return parts.join(" "); } diff --git a/cli/src/guidance/skills.gen.ts b/cli/src/guidance/skills.gen.ts index 3cace1d0b2..3986678a91 100644 --- a/cli/src/guidance/skills.gen.ts +++ b/cli/src/guidance/skills.gen.ts @@ -8497,7 +8497,7 @@ write a secret you were not given: keep the placeholder. | \`git_sync\` | \`{ repositories: [...] }\`, each with \`git_repo_resource_path\`, \`script_path\` and \`settings\` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. | | \`datatable\` | \`{ datatables: { : { database: { resource_type, resource_path } } } }\`, both required: \`resource_type: postgresql\` with \`resource_path\` a Postgres resource's path, or \`instance\`, or \`external_instance\` (Enterprise only), with \`resource_path\` the name of a database a superadmin set up. A fork's entry can instead be \`{ reference: { workspace_id, datatable } }\`, pointing at its parent's data table; keep it as is. \`main\` is the one scripts use by default. Role permissions are \`datatable_permissions\`. | | \`datatable_permissions\` | \`{ : { default_role, roles: { : [tenants] } } }\`: who may connect to each data table as which role. Tenants are \`u/\`, \`g/\`, \`f/\` or \`*\`; \`admin\` is always a role, and \`default_role: null\` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. | -| \`protection_rules\` | \`{ : { rules: [...], bypass_groups: [...], bypass_users: [...] } }\`, rules among \`DisableDirectDeployment\`, \`DisableWorkspaceForking\`, \`RestrictDeployToDeployers\`, \`RestrictAnonymousAppDeployment\`, \`RestrictPublicRunSharing\`, \`RestrictGuestAppDeployment\`, \`RequireIsolatedForks\`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | +| \`protection_rules\` | \`{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }\`, rules among \`DisableDirectDeployment\`, \`DisableWorkspaceForking\`, \`RestrictDeployToDeployers\`, \`RestrictAnonymousAppDeployment\`, \`RestrictPublicRunSharing\`, \`RestrictGuestAppDeployment\`, \`RequireIsolatedForks\`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | | \`ducklake\` | \`{ ducklakes: { : { catalog, storage, ... } } }\`. | | \`dbt_warehouses\` | dbt warehouse connections. | | \`default_app\` | Path of the app shown as the workspace home. | diff --git a/cli/test/protection_rules_converter_unit.test.ts b/cli/test/protection_rules_converter_unit.test.ts index c024cdf502..504140f0a9 100644 --- a/cli/test/protection_rules_converter_unit.test.ts +++ b/cli/test/protection_rules_converter_unit.test.ts @@ -238,3 +238,14 @@ describe("protection-rules.yaml read/write", () => { } }); }); + +describe("fork_role_mapping", () => { + test("a changed mapping is an update, an empty one is no change", () => { + const local = { ...rule("forks", []), fork_role_mapping: { operator: "developer" } }; + const plan = ProtectionRulesConverter.computePlan([local], [rule("forks", [])]); + expect(plan.toUpdate.map((e) => e.name)).toEqual(["forks"]); + + const empty = { ...rule("forks", []), fork_role_mapping: {} }; + expect(ProtectionRulesConverter.entriesEqual(empty, rule("forks", []))).toBe(true); + }); +}); diff --git a/docs/feature-telemetry.md b/docs/feature-telemetry.md index af0dd55942..6ed1c40004 100644 --- a/docs/feature-telemetry.md +++ b/docs/feature-telemetry.md @@ -4,13 +4,13 @@ anonymous usage-stats payload. It answers "does anyone use this, and which variant do they pick" without any identifying data leaving the instance. -It currently carries 86 registered actions across thirty-three features (`ai_session`, `ai_chat`, +It currently carries 89 registered actions across thirty-four features (`ai_session`, `ai_chat`, `ai_fix`, `ai_agent`, `ai_decision`, `ai_agent_eval`, `app_sandbox`, `datatable`, `db_manager`, `flow_editor`, `flow_run`, `flow_step`, `home`, `header_nav`, `run_form`, `debugger`, `trigger`, `command_script`, `mcp`, `mcp_workspace`, `hub_script`, `usage_meter`, `sso_groups_claim`, `cloud_trial_offer`, `job_token_scopes`, `worker_draining`, `worker_group`, `worker_resource_stats`, `dev_workspace_isolation`, `slack_agent`, `variable_expiration`, `anthropic_federation`, -`deployment_request`). Nearly all of the +`deployment_request`, `fork_roles`). Nearly all of the product is uninstrumented, so new user-facing work is the opportunity to change that. ## When to instrument diff --git a/docs/workspace-roles.md b/docs/workspace-roles.md index 9ce3294c54..96c6d3c796 100644 --- a/docs/workspace-roles.md +++ b/docs/workspace-roles.md @@ -19,6 +19,7 @@ code is `windmill_common::roles`. | `flows` | `none` / `compose` / `full` | yes; also AI agents, whose tools are checked as composed steps | | `apps` | `none` / `compose` / `full` | yes; low-code apps and worker-built apps need `full` | | `manage_schedules`, `manage_triggers` | bool | yes, see `docs/operator-write-rights.md` | +| `fork` | bool | yes, at both fork routes; unset by every layer, it is granted exactly to an unrestricted role | | `visibility.*` | bool per page | no, UI only | `compose` is the former operator builder right: the composition-only checks of @@ -33,6 +34,20 @@ deleting one that a member, invite, auto-invite or instance group mapping still and so is deleting an instance role that a workspace still overrides (the override would fall onto `base_for` instead of the definition it was written against). +## Forks + +A member's role carries into a fork: the forker, and every member `copy_members` or +`add_admins_and_developers_to_forks` brings, joins the fork with the role they hold in the parent, +and the fork gets the parent's `role_overrides` so the name means the same there. A protection +ruleset's `fork_role_mapping` (`{ role here: role in the fork }`) replaces that role; where several +rulesets map one role, the first by name wins. A role named by a mapping cannot be deleted. A +fork's non-admin creator may add a parent member only when the mapping makes them a developer, so +re-adding cannot undo a mapping that restricts. + +`fork`, like `visibility`, does not make a role restricted: withdrawing it from `developer` must not +turn every developer into an operator for the raw `is_operator` checks. So the fork routes read it +from the member's role (`caller_can_fork`), never through `caller_capabilities`' fast path. + ## Who is restricted `ApiAuthed::is_operator` means "this member's role is restricted", computed at authentication from diff --git a/frontend/src/lib/components/instanceSettings/WorkspaceRolesSetting.svelte b/frontend/src/lib/components/instanceSettings/WorkspaceRolesSetting.svelte index 404f27e638..446c6f33aa 100644 --- a/frontend/src/lib/components/instanceSettings/WorkspaceRolesSetting.svelte +++ b/frontend/src/lib/components/instanceSettings/WorkspaceRolesSetting.svelte @@ -6,7 +6,7 @@ import RoleCapabilitiesEditor from '../settings/RoleCapabilitiesEditor.svelte' import type { RoleCapabilities, RoleCapabilitiesPatch, RoleDefinition } from '$lib/gen' import { - applyCapabilitiesPatch, + resolveCapabilities, authorLevelLabel, baseCapabilitiesFor, DEFAULT_INSTANCE_ROLES, @@ -154,7 +154,7 @@ > {#each rows as row, i (row.id)} {@const base = baseCapabilitiesFor(row.name)} - {@const caps = applyCapabilitiesPatch(base, row.capabilities)} + {@const caps = resolveCapabilities(base, row.capabilities, false)}
- You are not an admin of this workspace, so you can only add developers or admins of - {parentWorkspaceId} as developers of this fork, and remove the ones who are not admins here. - Ask an admin of this workspace for any other membership change. + You are not an admin of this workspace, so you can only add members of + {parentWorkspaceId} whose role in its forks is developer, as developers of this fork, and remove + the ones who are not admins here. Ask an admin of this workspace for any other membership change.
diff --git a/frontend/src/lib/components/settings/RoleCapabilitiesEditor.svelte b/frontend/src/lib/components/settings/RoleCapabilitiesEditor.svelte index a9fc0e9911..c6e93eceea 100644 --- a/frontend/src/lib/components/settings/RoleCapabilitiesEditor.svelte +++ b/frontend/src/lib/components/settings/RoleCapabilitiesEditor.svelte @@ -10,9 +10,9 @@ RoleVisibility } from '$lib/gen' import { - applyCapabilitiesPatch, capabilitiesError, isEmptyPatch, + resolveCapabilities, VISIBILITY_KEYS } from '$lib/workspaceRoles' import { RotateCcw } from 'lucide-svelte' @@ -22,6 +22,8 @@ base: RoleCapabilities /** Fields whose `base` value is not known here, as `code` or `visibility.runs`. */ unknownBase?: string[] + /** Whether a layer `base` includes sets `fork`; otherwise an unset one is derived. */ + baseSetsFork?: boolean patch: RoleCapabilitiesPatch /** What the layer below is called: "default" on the instance, "instance" in a workspace. */ baseName: string @@ -35,6 +37,7 @@ let { base, unknownBase = [], + baseSetsFork = false, patch, baseName, setText, @@ -43,10 +46,10 @@ onchange }: Props = $props() - const effective = $derived(applyCapabilitiesPatch(base, patch)) + const effective = $derived(resolveCapabilities(base, patch, baseSetsFork)) const error = $derived(capabilitiesError(effective)) - type BoolKey = 'code' | 'manage_schedules' | 'manage_triggers' + type BoolKey = 'code' | 'manage_schedules' | 'manage_triggers' | 'fork' type LevelKey = 'flows' | 'apps' function boolValue(path: string): boolean { @@ -115,6 +118,12 @@ key: 'manage_triggers', label: 'Manage triggers', description: 'Create, edit and delete triggers where folder permissions allow.' + }, + { + key: 'fork', + label: 'Fork the workspace', + description: + 'Create forks of this workspace. Their role in the fork is this one, unless a ruleset maps it.' } ] diff --git a/frontend/src/lib/components/settings/WorkspaceRoleOverrides.svelte b/frontend/src/lib/components/settings/WorkspaceRoleOverrides.svelte index 12e1b3e68e..9260820d09 100644 --- a/frontend/src/lib/components/settings/WorkspaceRoleOverrides.svelte +++ b/frontend/src/lib/components/settings/WorkspaceRoleOverrides.svelte @@ -13,10 +13,11 @@ } from '$lib/gen' import { sendUserToast } from '$lib/toast' import { - applyCapabilitiesPatch, authorLevelLabel, baseCapabilitiesFor, capabilitiesError, + forkLooksSet, + resolveCapabilities, isEmptyPatch, roleNameError, VISIBILITY_KEYS @@ -58,7 +59,7 @@ function overriddenKeys(o: RoleOverride | undefined): string[] { const c = o?.capabilities ?? {} - const keys = ['code', 'flows', 'apps', 'manage_schedules', 'manage_triggers'].filter( + const keys = ['code', 'flows', 'apps', 'manage_schedules', 'manage_triggers', 'fork'].filter( (k) => c[k] != null ) for (const { key } of VISIBILITY_KEYS) { @@ -78,8 +79,20 @@ return { base: role.capabilities, unknown: overriddenKeys(stored) } } + /** Whether the layers under the draft set `fork`, so an unset one is not derived again. */ + function forkSetBelow(name: string): boolean { + const { base, unknown } = baseFor(name) + return ( + instanceRoles.some((r) => r.name === name) && (unknown.includes('fork') || forkLooksSet(base)) + ) + } + function effectiveFor(name: string): RoleCapabilities { - return applyCapabilitiesPatch(baseFor(name).base, draft.get(name)?.capabilities) + return resolveCapabilities( + baseFor(name).base, + draft.get(name)?.capabilities, + forkSetBelow(name) + ) } const AUTHORING_KEYS = ['code', 'flows', 'apps'] @@ -204,7 +217,7 @@ {#snippet roleCard(name: string, role: EffectiveRole | undefined)} {@const { base, unknown } = baseFor(name)} - {@const caps = applyCapabilitiesPatch(base, draft.get(name)?.capabilities)} + {@const caps = effectiveFor(name)} {@const overridden = Object.hasOwn(cleanedDraft, name)} {@const frozen = !role && !data?.custom_roles_enabled}
@@ -283,6 +296,7 @@ !isRestricted(capabilitiesIn(undefined, w))) + ).filter((w) => canFork(capabilitiesIn(undefined, w))) ) let baseItems = $derived( baseCandidates.map((w) => ({ diff --git a/frontend/src/lib/components/workspaceSettings/RulesetEditor.svelte b/frontend/src/lib/components/workspaceSettings/RulesetEditor.svelte index 43f5434c4d..3454ac2572 100644 --- a/frontend/src/lib/components/workspaceSettings/RulesetEditor.svelte +++ b/frontend/src/lib/components/workspaceSettings/RulesetEditor.svelte @@ -16,8 +16,9 @@ import { clone } from '$lib/utils' import { DEV_WORKSPACE_LOCK_RULE_NAME } from '$lib/workspaceProtectionRules.svelte' import { untrack } from 'svelte' - import { Save } from 'lucide-svelte' + import { ArrowRight, Plus, Save, Trash2 } from 'lucide-svelte' import { safeSelectItems } from '$lib/components/select/utils.svelte' + import Select from '$lib/components/select/Select.svelte' interface Props { rule?: ProtectionRuleset @@ -52,6 +53,10 @@ let selectedUsers = $state( untrack(() => rule)?.bypass_users?.map((u) => u.replace('u/', '')) ?? [] ) + type RoleMapping = { from: string | undefined; to: string | undefined }[] + const mappingRows = (m: Record | undefined): RoleMapping => + Object.entries(m ?? {}).map(([from, to]) => ({ from, to })) + let forkRoleMapping = $state(mappingRows(untrack(() => rule)?.fork_role_mapping)) // Initial state for unsaved changes tracking let initialName = $state(untrack(() => rule)?.name ?? '') @@ -67,6 +72,9 @@ ? untrack(() => rule)!.bypass_groups.map((g) => g.replace('g/', '')) : [] ) + let initialForkRoleMapping = $state( + mappingRows(untrack(() => rule)?.fork_role_mapping) + ) let initialSelectedUsers = $state( untrack(() => rule)?.bypass_users ? untrack(() => rule)!.bypass_users.map((u) => u.replace('u/', '')) @@ -76,6 +84,7 @@ // Available options let availableGroups = $state([]) let availableUsers = $state([]) + let availableRoles = $state<{ value: string; label: string }[]>([]) let saving = $state(false) @@ -90,11 +99,17 @@ availableUsers = users } + async function loadAvailableRoles() { + const { roles } = await WorkspaceService.listWorkspaceRoles({ workspace: $workspaceStore! }) + availableRoles = roles.map((r) => ({ value: r.name, label: r.label })) + } + $effect(() => { if ($workspaceStore) { untrack(() => { loadAvailableGroups() loadAvailableUsers() + loadAvailableRoles() }) } }) @@ -111,7 +126,8 @@ restrictGuestAppDeployment || requireIsolatedForks || selectedGroups.length > 0 || - selectedUsers.length > 0 + selectedUsers.length > 0 || + forkRoleMapping.length > 0 : name !== initialName || disableDirectDeployment !== initialDisableDirectDeployment || disableFork !== initialDisableFork || @@ -123,7 +139,23 @@ JSON.stringify([...selectedGroups].sort()) !== JSON.stringify([...initialSelectedGroups].sort()) || JSON.stringify([...selectedUsers].sort()) !== - JSON.stringify([...initialSelectedUsers].sort()) + JSON.stringify([...initialSelectedUsers].sort()) || + JSON.stringify(forkRoleMapping) !== JSON.stringify(initialForkRoleMapping) + ) + + const mappingError = $derived.by(() => { + const seen = new Set() + for (const { from, to } of forkRoleMapping) { + if (!from || !to) return 'Pick both roles of every mapping' + if (from === to) return `${from} is mapped to itself` + if (seen.has(from)) return `${from} is mapped twice` + seen.add(from) + } + return undefined + }) + + const forkRoleMappingBody = $derived( + Object.fromEntries(forkRoleMapping.map(({ from, to }) => [from!, to!])) ) const nameError = $derived.by(() => { @@ -136,7 +168,7 @@ return undefined }) - const canSave = $derived(!nameError && hasUnsavedChanges) + const canSave = $derived(!nameError && !mappingError && hasUnsavedChanges) async function create() { if (!canSave || !$workspaceStore) return @@ -163,7 +195,8 @@ ...(requireIsolatedForks ? ['RequireIsolatedForks' as ProtectionRuleKind] : []) ], bypass_groups: selectedGroups, - bypass_users: selectedUsers + bypass_users: selectedUsers, + fork_role_mapping: forkRoleMappingBody } }) @@ -203,7 +236,8 @@ ...(requireIsolatedForks ? ['RequireIsolatedForks' as ProtectionRuleKind] : []) ], bypass_groups: selectedGroups, - bypass_users: selectedUsers + bypass_users: selectedUsers, + fork_role_mapping: forkRoleMappingBody } }) @@ -220,6 +254,7 @@ initialRequireIsolatedForks = requireIsolatedForks initialSelectedGroups = clone(selectedGroups) initialSelectedUsers = clone(selectedUsers) + initialForkRoleMapping = clone(forkRoleMapping) onUpdate?.() } catch (error) { @@ -345,6 +380,51 @@ )}
+ +
+ {#each forkRoleMapping as mapping, i (i)} +
+ +
+ {/each} + {#if mappingError} +

{mappingError}

+ {/if} +
+ +
+
{#snippet actions()} diff --git a/frontend/src/lib/roleCapabilities.ts b/frontend/src/lib/roleCapabilities.ts index 1a9fc70a61..1827998bae 100644 --- a/frontend/src/lib/roleCapabilities.ts +++ b/frontend/src/lib/roleCapabilities.ts @@ -1,6 +1,6 @@ import type { OperatorSettings, RoleCapabilities, RoleVisibility } from '$lib/gen' import type { UserExt, UserWorkspace } from '$lib/stores' -import { FULL_CAPABILITIES, VISIBILITY_KEYS } from '$lib/workspaceRoles' +import { capabilitiesRestricted, FULL_CAPABILITIES, VISIBILITY_KEYS } from '$lib/workspaceRoles' /** * What a member's workspace role lets them author, manage and see: the frontend's copy of @@ -32,6 +32,7 @@ export function legacyOperatorCapabilities( apps: settings?.builder_apps === true ? 'compose' : 'none', manage_schedules: settings?.manage_schedules !== false, manage_triggers: settings?.manage_triggers !== false, + fork: false, visibility } } @@ -55,17 +56,16 @@ export function capabilitiesIn( return operator ? legacyOperatorCapabilities(workspace?.operator_settings) : FULL_CAPABILITIES } -/** Whether the role is gated at all, as `RoleCapabilities::is_restricted`: visibility is left out, - * since hiding a page must not cost a full author what restricted roles are refused, such as - * forking. */ +/** Whether the role is gated at all, as `RoleCapabilities::is_restricted`: visibility and `fork` + * are left out, since neither may cost a full author what restricted roles are refused. */ export function isRestricted(caps: RoleCapabilities): boolean { - return ( - !caps.code || - caps.flows !== 'full' || - caps.apps !== 'full' || - !caps.manage_schedules || - !caps.manage_triggers - ) + return capabilitiesRestricted(caps) +} + +/** Whether the role may fork the workspace. A server that predates the capability sends none, and + * then refused every restricted role in the UI. */ +export function canFork(caps: RoleCapabilities): boolean { + return caps.fork ?? !isRestricted(caps) } /** The role with nothing left to build and the same pages in view: what a session in Use mode diff --git a/frontend/src/lib/utils/editInFork.ts b/frontend/src/lib/utils/editInFork.ts index 9e74878f39..34497bb3f5 100644 --- a/frontend/src/lib/utils/editInFork.ts +++ b/frontend/src/lib/utils/editInFork.ts @@ -1,5 +1,5 @@ import { get } from 'svelte/store' -import { isRestricted } from '$lib/roleCapabilities' +import { canFork } from '$lib/roleCapabilities' import type { RoleCapabilities } from '$lib/gen' import { roleCapabilities, @@ -58,17 +58,12 @@ export function editInForkDescription( } /** - * Whether the user may CREATE a new fork of the current workspace: their role is unrestricted, and + * Whether the user may CREATE a new fork of the current workspace: their role holds `fork`, and * either forking isn't disabled or they can bypass the rule (workspace admins). Keeps the "Fork * workspace" entry available to admins as the last-resort escape hatch on a locked prod. - * - * Any restricted role is refused because a fork does not carry the role: the server writes the - * forker's membership row from the parent's `is_admin` alone, so the forker gets the default role - * and forking would hand a restricted member a full-rights copy of the workspace. The server does - * not refuse it: this hides the affordance, it does not enforce the rule. */ export function canCreateFork(user: UserExt | undefined, caps: RoleCapabilities): boolean { - if (isRestricted(caps)) return false + if (!canFork(caps)) return false return ( !isRuleActive('DisableWorkspaceForking') || canUserBypassRuleKind('DisableWorkspaceForking', user) diff --git a/frontend/src/lib/workspaceRoles.test.ts b/frontend/src/lib/workspaceRoles.test.ts index 84f3ff8a71..09cc03633d 100644 --- a/frontend/src/lib/workspaceRoles.test.ts +++ b/frontend/src/lib/workspaceRoles.test.ts @@ -4,6 +4,7 @@ import { applyCapabilitiesPatch, baseCapabilitiesFor, FULL_CAPABILITIES, + resolveCapabilities, validRoleOr, workspaceRolesErrors } from './workspaceRoles' @@ -62,3 +63,19 @@ describe('validRoleOr', () => { expect(validRoleOr([role('builder')], 'admin')).toBe('admin') }) }) + +describe('resolveCapabilities', () => { + it('derives an unset fork from the restriction, as the server does', () => { + const maintainer = resolveCapabilities( + baseCapabilitiesFor('maintainer'), + { code: true, flows: 'full', apps: 'full' }, + false + ) + expect(maintainer.fork).toBe(true) + const developer = resolveCapabilities(baseCapabilitiesFor('developer'), { code: false }, false) + expect(developer.fork).toBe(false) + expect(resolveCapabilities(baseCapabilitiesFor('operator'), { fork: true }, false).fork).toBe( + true + ) + }) +}) diff --git a/frontend/src/lib/workspaceRoles.ts b/frontend/src/lib/workspaceRoles.ts index 4cfb3a6089..b44e800190 100644 --- a/frontend/src/lib/workspaceRoles.ts +++ b/frontend/src/lib/workspaceRoles.ts @@ -48,6 +48,7 @@ export const FULL_CAPABILITIES: RoleCapabilities = { apps: 'full', manage_schedules: true, manage_triggers: true, + fork: true, visibility: ALL_VISIBLE } @@ -57,6 +58,7 @@ const OPERATOR_CAPABILITIES: RoleCapabilities = { apps: 'none', manage_schedules: true, manage_triggers: true, + fork: false, visibility: ALL_VISIBLE } @@ -86,6 +88,7 @@ export function applyCapabilitiesPatch( if (patch.apps != null) out.apps = patch.apps if (patch.manage_schedules != null) out.manage_schedules = patch.manage_schedules if (patch.manage_triggers != null) out.manage_triggers = patch.manage_triggers + if (patch.fork != null) out.fork = patch.fork for (const { key } of VISIBILITY_KEYS) { const v = patch.visibility?.[key] if (v != null) out.visibility[key] = v @@ -93,6 +96,38 @@ export function applyCapabilitiesPatch( return out } +/** `RoleCapabilities::is_restricted`: visibility and `fork` are left out. */ +export function capabilitiesRestricted(caps: RoleCapabilities): boolean { + return ( + !caps.code || + caps.flows !== 'full' || + caps.apps !== 'full' || + !caps.manage_schedules || + !caps.manage_triggers + ) +} + +/** + * `patch` over `base` as `RoleCapabilities::resolve` does it: a `fork` that neither `patch` nor a + * layer under it sets is granted exactly to an unrestricted role. `forkSetBelow` says whether a + * layer `base` already includes sets it. + */ +export function resolveCapabilities( + base: RoleCapabilities, + patch: RoleCapabilitiesPatch | undefined, + forkSetBelow: boolean +): RoleCapabilities { + const out = applyCapabilitiesPatch(base, patch) + if (patch?.fork == null && !forkSetBelow) out.fork = !capabilitiesRestricted(out) + return out +} + +/** Whether a server-resolved role's `fork` was set by a layer rather than derived. The listing + * does not say, so a value the fallback would have given anyway reads as derived. */ +export function forkLooksSet(caps: RoleCapabilities): boolean { + return caps.fork !== !capabilitiesRestricted(caps) +} + /** The error the server raises for these capabilities, if any (`RoleCapabilities::validate`). */ export function capabilitiesError(caps: RoleCapabilities): string | undefined { if (!caps.code && (caps.flows === 'full' || caps.apps === 'full')) { diff --git a/frontend/src/routes/(root)/(logged)/user/(user)/fork_workspace/+page.svelte b/frontend/src/routes/(root)/(logged)/user/(user)/fork_workspace/+page.svelte index bb72bc844e..9520236526 100644 --- a/frontend/src/routes/(root)/(logged)/user/(user)/fork_workspace/+page.svelte +++ b/frontend/src/routes/(root)/(logged)/user/(user)/fork_workspace/+page.svelte @@ -2,14 +2,11 @@ import CreateWorkspace from '$lib/components/workspaceSettings/CreateWorkspace.svelte' import { Alert } from '$lib/components/common' import { roleCapabilities } from '$lib/stores' - import { isRestricted } from '$lib/roleCapabilities' + import { canFork } from '$lib/roleCapabilities' - -{#if isRestricted($roleCapabilities)} + +{#if !canFork($roleCapabilities)}
Page not available for your role
diff --git a/system_prompts/auto-generated/prompts.ts b/system_prompts/auto-generated/prompts.ts index 3dc567d157..178e98a419 100644 --- a/system_prompts/auto-generated/prompts.ts +++ b/system_prompts/auto-generated/prompts.ts @@ -1605,7 +1605,7 @@ write a secret you were not given: keep the placeholder. | \`git_sync\` | \`{ repositories: [...] }\`, each with \`git_repo_resource_path\`, \`script_path\` and \`settings\` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. | | \`datatable\` | \`{ datatables: { : { database: { resource_type, resource_path } } } }\`, both required: \`resource_type: postgresql\` with \`resource_path\` a Postgres resource's path, or \`instance\`, or \`external_instance\` (Enterprise only), with \`resource_path\` the name of a database a superadmin set up. A fork's entry can instead be \`{ reference: { workspace_id, datatable } }\`, pointing at its parent's data table; keep it as is. \`main\` is the one scripts use by default. Role permissions are \`datatable_permissions\`. | | \`datatable_permissions\` | \`{ : { default_role, roles: { : [tenants] } } }\`: who may connect to each data table as which role. Tenants are \`u/\`, \`g/\`, \`f/\` or \`*\`; \`admin\` is always a role, and \`default_role: null\` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. | -| \`protection_rules\` | \`{ : { rules: [...], bypass_groups: [...], bypass_users: [...] } }\`, rules among \`DisableDirectDeployment\`, \`DisableWorkspaceForking\`, \`RestrictDeployToDeployers\`, \`RestrictAnonymousAppDeployment\`, \`RestrictPublicRunSharing\`, \`RestrictGuestAppDeployment\`, \`RequireIsolatedForks\`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | +| \`protection_rules\` | \`{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }\`, rules among \`DisableDirectDeployment\`, \`DisableWorkspaceForking\`, \`RestrictDeployToDeployers\`, \`RestrictAnonymousAppDeployment\`, \`RestrictPublicRunSharing\`, \`RestrictGuestAppDeployment\`, \`RequireIsolatedForks\`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | | \`ducklake\` | \`{ ducklakes: { : { catalog, storage, ... } } }\`. | | \`dbt_warehouses\` | dbt warehouse connections. | | \`default_app\` | Path of the app shown as the workspace home. | diff --git a/system_prompts/auto-generated/skills/workspace-settings/SKILL.md b/system_prompts/auto-generated/skills/workspace-settings/SKILL.md index 21f02750db..73f276c255 100644 --- a/system_prompts/auto-generated/skills/workspace-settings/SKILL.md +++ b/system_prompts/auto-generated/skills/workspace-settings/SKILL.md @@ -46,7 +46,7 @@ write a secret you were not given: keep the placeholder. | `git_sync` | `{ repositories: [...] }`, each with `git_repo_resource_path`, `script_path` and `settings` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. | | `datatable` | `{ datatables: { : { database: { resource_type, resource_path } } } }`, both required: `resource_type: postgresql` with `resource_path` a Postgres resource's path, or `instance`, or `external_instance` (Enterprise only), with `resource_path` the name of a database a superadmin set up. A fork's entry can instead be `{ reference: { workspace_id, datatable } }`, pointing at its parent's data table; keep it as is. `main` is the one scripts use by default. Role permissions are `datatable_permissions`. | | `datatable_permissions` | `{ : { default_role, roles: { : [tenants] } } }`: who may connect to each data table as which role. Tenants are `u/`, `g/`, `f/` or `*`; `admin` is always a role, and `default_role: null` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. | -| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...] } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | +| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | | `ducklake` | `{ ducklakes: { : { catalog, storage, ... } } }`. | | `dbt_warehouses` | dbt warehouse connections. | | `default_app` | Path of the app shown as the workspace home. | diff --git a/system_prompts/base/workspace-settings.md b/system_prompts/base/workspace-settings.md index 0a2ae51866..145cd89c66 100644 --- a/system_prompts/base/workspace-settings.md +++ b/system_prompts/base/workspace-settings.md @@ -41,7 +41,7 @@ write a secret you were not given: keep the placeholder. | `git_sync` | `{ repositories: [...] }`, each with `git_repo_resource_path`, `script_path` and `settings` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. | | `datatable` | `{ datatables: { : { database: { resource_type, resource_path } } } }`, both required: `resource_type: postgresql` with `resource_path` a Postgres resource's path, or `instance`, or `external_instance` (Enterprise only), with `resource_path` the name of a database a superadmin set up. A fork's entry can instead be `{ reference: { workspace_id, datatable } }`, pointing at its parent's data table; keep it as is. `main` is the one scripts use by default. Role permissions are `datatable_permissions`. | | `datatable_permissions` | `{ : { default_role, roles: { : [tenants] } } }`: who may connect to each data table as which role. Tenants are `u/`, `g/`, `f/` or `*`; `admin` is always a role, and `default_role: null` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. | -| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...] } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | +| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. | | `ducklake` | `{ ducklakes: { : { catalog, storage, ... } } }`. | | `dbt_warehouses` | dbt warehouse connections. | | `default_app` | Path of the app shown as the workspace home. |