From 84a7672b4ed76ac3aa7843dc45c2bf4873f9dce2 Mon Sep 17 00:00:00 2001 From: Ruben Fiszel Date: Thu, 5 Nov 2026 19:03:53 +0000 Subject: [PATCH] fix: harden role_overrides apply from the settings document (#12255) * fix: refuse unknown roles and stale operator_settings in settings apply Co-Authored-By: Claude Opus 5.5 * fix: keep omitted-key clears when a push only removes a workspace role Co-Authored-By: Claude Opus 5.5 * fix: load instance roles on the held connection in the settings document Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- .../windmill-api-workspaces/src/workspaces.rs | 14 +- .../ai_session_tools/gen/instructions.json | 2 +- .../src/workspace_settings_document.rs | 368 +++++++++++++++--- backend/windmill-common/src/roles.rs | 18 +- cli/src/guidance/skills.gen.ts | 5 +- docs/workspace-roles.md | 22 ++ system_prompts/auto-generated/prompts.ts | 3 +- .../skills/workspace-settings/SKILL.md | 5 +- system_prompts/base/workspace-settings.md | 5 +- 9 files changed, 363 insertions(+), 79 deletions(-) diff --git a/backend/windmill-api-workspaces/src/workspaces.rs b/backend/windmill-api-workspaces/src/workspaces.rs index e1b013cd70..206e683547 100644 --- a/backend/windmill-api-workspaces/src/workspaces.rs +++ b/backend/windmill-api-workspaces/src/workspaces.rs @@ -7125,17 +7125,9 @@ async fn user_workspaces( }; // The pre-roles shape the frontend gates on: present exactly for a restricted member. let operator_settings = (capabilities.is_restricted()).then(|| { - let v = &capabilities.visibility; - Some(serde_json::json!({ - "runs": v.runs, "schedules": v.schedules, "resources": v.resources, - "variables": v.variables, "assets": v.assets, "triggers": v.triggers, - "audit_logs": v.audit_logs, "groups": v.groups, "folders": v.folders, - "workers": v.workers, - "builder_flows": capabilities.flows != windmill_common::roles::AuthorLevel::None, - "builder_apps": capabilities.apps != windmill_common::roles::AuthorLevel::None, - "manage_schedules": capabilities.manage_schedules, - "manage_triggers": capabilities.manage_triggers, - })) + Some(windmill_common::roles::legacy_operator_settings( + &capabilities, + )) }); workspaces.push(UserWorkspace { id: r.id, diff --git a/backend/windmill-api/src/ai_session_tools/gen/instructions.json b/backend/windmill-api/src/ai_session_tools/gen/instructions.json index ec4ccf8c42..4fdecf854d 100644 --- a/backend/windmill-api/src/ai_session_tools/gen/instructions.json +++ b/backend/windmill-api/src/ai_session_tools/gen/instructions.json @@ -261,6 +261,6 @@ "datatable.python3": "# Data tables: schema changes, migrations and permissions\n\nA data table is a PostgreSQL database managed by Windmill and shared by the whole workspace: scripts, flows and apps reach it by name, `main` being the default. It is configured in the workspace settings (Data Tables), never created by SQL.\n\n## Changing the schema\n\nA data table with migrations enabled keeps its schema as an ordered list of versioned SQL migrations, each an `up` and an optional `down`. That list is what git sync and a workspace merge carry, and what rebuilds the schema in another workspace. So first find out whether migrations are enabled on the data table:\n\n- **Enabled**: make every schema change (CREATE, ALTER, DROP, anything but row data) as a new migration. A change made with plain SQL exists in that one database only, and the history no longer describes it.\n- **Not enabled**: schema changes are plain SQL. Enabling migrations is a workspace admin's decision, made in the Data Tables settings, not a step to take on the way to something else.\n\nWriting a migration:\n\n- Wrap the statements in `BEGIN; ... END;` so it applies atomically. Leave the frame out only for a statement that cannot run in a transaction, such as `CREATE INDEX CONCURRENTLY`: such a migration can fail half applied.\n- Give it a `down` that undoes the `up` whenever the change is reversible. Without one it cannot be rolled back.\n- Declare the schemas it changes on a leading line of its `up`, `-- schemas sales, billing`, and qualify every name with its schema (`sales.orders`). A migration waits only on earlier migrations whose schemas overlap its own. One that declares none applies to the whole database and every later migration waits for it, so leave the line out only for a change that really spans the database, such as an extension.\n- Migrations apply oldest first. One that is applied is not edited: correct it with a new migration.\n- Row data is not schema. Insert, update and delete rows with plain SQL, migrations or not.\n\n## A migration is not a draft\n\nA migration is saved to the workspace and changes the database as soon as it is applied, for everything that reads those tables. To try a schema out without touching tables already in use:\n\n- put the new tables in a schema of their own, created by the same migration (`CREATE SCHEMA IF NOT EXISTS`), and point what you are building at it. A raw app names its schema in its `data` config, which is why a new app gets a new schema by default;\n- or work in a workspace fork whose data table was cloned when the fork was created (\"Clone schema only\" or \"Clone schema and data\"): migrations applied there change that copy alone, and reach the parent workspace when the fork is merged. A fork that kept the original data table points at the parent's database, so a migration applied in it changes the parent's tables at once.\n\nAltering or dropping a table other scripts or apps use is neither of those: say what depends on it before doing it.\n\n## Roles and permissions\n\nData table roles are a self-hosted Enterprise Edition feature, for data tables on managed Postgres (the instance database or the external managed instance). Without them, and always on a data table backed by a Postgres resource, everyone who can use the data table reaches all of it: there is no per-user, per-schema or per-row access to promise.\n\nA data table under roles is reached as a Postgres role, and Postgres enforces what that role was granted:\n\n- roles are assigned, per data table, to users, groups or folders. `admin` is the connection the data table had before roles: it owns the existing objects and ignores grants;\n- a connection either names a role (`wmill.datatable('main', { role: 'analyst' })`, `role='analyst'` in Python, `datatable://main?role=analyst`) and holds that role alone, or names none and is resolved from the data table's default role;\n- `permission denied` means the role in use lacks a grant. Say which role and which object: the grant is a data table admin's to give, in the roles drawer of the Database Manager. Do not switch to `admin` to get past it;\n- a migration runs as the role its SQL declares on its first line, `-- role `, the same one in its `down`. One that declares none runs as `admin`, which only those who may use `admin` can do.\n\nRead the Permissions section of https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables before designing who reaches what: it covers tenants, grants, the default role and the roles a connection that names none ends up holding, rows that differ per viewer through row-level security, and how forks and raw apps behave under roles.\n\n\n# Datatable SQL SDK reference\n\nDatatables are workspace-scoped managed PostgreSQL databases. In chat, explore and shape them with the datatable tools you were given. The reference below is for code you author inside runnables (inline app runnables, scripts, or flow rawscript modules) that reads or writes datatable data at runtime.\n\n- A runnable accesses a datatable via `wmill.datatable()` (the default \"main\") or `wmill.datatable('')`, referencing tables as `schema.table`.\n- Use parameterized queries (the tagged template in TypeScript, `$1`/`$2` placeholders in Python) — never interpolate untrusted values into SQL strings.\n\n## Python Datatable API (wmill)\n\nImport: `import wmill`\n\n# Get a DataTable client for SQL queries.\n# \n# Args:\n# name: Database name (default: \"main\")\n# role: Connect as this data table role instead of the data table's default one.\n# \n# Returns:\n# DataTableClient instance\ndef datatable(name: str = 'main', *, role: Optional[str] = None) -> DataTableClient\n\n# Client for executing SQL queries against Windmill DataTables.\nclass DataTableClient:\n # Initialize DataTableClient.\n # \n # Args:\n # client: Windmill client instance\n # name: DataTable name\n # role: Data table role to connect as, or None for the data table's default\n def __init__(client: Windmill, name: str, role: Optional[str] = None)\n\n # Execute a SQL query against the DataTable.\n # \n # Args:\n # sql: SQL query string with $1, $2, etc. placeholders\n # *args: Positional arguments to bind to query placeholders\n # \n # Returns:\n # SqlQuery instance for fetching results\n def query(sql: str, *args) -> SqlQuery\n\n\n# Query result handler for DataTable and DuckLake queries.\nclass SqlQuery:\n # Initialize SqlQuery.\n # \n # Args:\n # sql: SQL query string\n # fetch_fn: Function to execute the query\n def __init__(sql: str, fetch_fn)\n\n # Execute query and fetch results.\n # \n # Args:\n # result_collection: Optional result collection mode\n # \n # Returns:\n # Query results\n def fetch(result_collection: str | None = None)\n\n # Execute query and fetch first row of results.\n # \n # Returns:\n # First row of query results\n def fetch_one()\n\n # Execute query and fetch first row of results. Return result as a scalar value.\n # \n # Returns:\n # First row of query result as a scalar value\n def fetch_one_scalar()\n\n # Execute query and don't return any results.\n # \n def execute()\n\n\n", "datatable.go": "# Data tables: schema changes, migrations and permissions\n\nA data table is a PostgreSQL database managed by Windmill and shared by the whole workspace: scripts, flows and apps reach it by name, `main` being the default. It is configured in the workspace settings (Data Tables), never created by SQL.\n\n## Changing the schema\n\nA data table with migrations enabled keeps its schema as an ordered list of versioned SQL migrations, each an `up` and an optional `down`. That list is what git sync and a workspace merge carry, and what rebuilds the schema in another workspace. So first find out whether migrations are enabled on the data table:\n\n- **Enabled**: make every schema change (CREATE, ALTER, DROP, anything but row data) as a new migration. A change made with plain SQL exists in that one database only, and the history no longer describes it.\n- **Not enabled**: schema changes are plain SQL. Enabling migrations is a workspace admin's decision, made in the Data Tables settings, not a step to take on the way to something else.\n\nWriting a migration:\n\n- Wrap the statements in `BEGIN; ... END;` so it applies atomically. Leave the frame out only for a statement that cannot run in a transaction, such as `CREATE INDEX CONCURRENTLY`: such a migration can fail half applied.\n- Give it a `down` that undoes the `up` whenever the change is reversible. Without one it cannot be rolled back.\n- Declare the schemas it changes on a leading line of its `up`, `-- schemas sales, billing`, and qualify every name with its schema (`sales.orders`). A migration waits only on earlier migrations whose schemas overlap its own. One that declares none applies to the whole database and every later migration waits for it, so leave the line out only for a change that really spans the database, such as an extension.\n- Migrations apply oldest first. One that is applied is not edited: correct it with a new migration.\n- Row data is not schema. Insert, update and delete rows with plain SQL, migrations or not.\n\n## A migration is not a draft\n\nA migration is saved to the workspace and changes the database as soon as it is applied, for everything that reads those tables. To try a schema out without touching tables already in use:\n\n- put the new tables in a schema of their own, created by the same migration (`CREATE SCHEMA IF NOT EXISTS`), and point what you are building at it. A raw app names its schema in its `data` config, which is why a new app gets a new schema by default;\n- or work in a workspace fork whose data table was cloned when the fork was created (\"Clone schema only\" or \"Clone schema and data\"): migrations applied there change that copy alone, and reach the parent workspace when the fork is merged. A fork that kept the original data table points at the parent's database, so a migration applied in it changes the parent's tables at once.\n\nAltering or dropping a table other scripts or apps use is neither of those: say what depends on it before doing it.\n\n## Roles and permissions\n\nData table roles are a self-hosted Enterprise Edition feature, for data tables on managed Postgres (the instance database or the external managed instance). Without them, and always on a data table backed by a Postgres resource, everyone who can use the data table reaches all of it: there is no per-user, per-schema or per-row access to promise.\n\nA data table under roles is reached as a Postgres role, and Postgres enforces what that role was granted:\n\n- roles are assigned, per data table, to users, groups or folders. `admin` is the connection the data table had before roles: it owns the existing objects and ignores grants;\n- a connection either names a role (`wmill.datatable('main', { role: 'analyst' })`, `role='analyst'` in Python, `datatable://main?role=analyst`) and holds that role alone, or names none and is resolved from the data table's default role;\n- `permission denied` means the role in use lacks a grant. Say which role and which object: the grant is a data table admin's to give, in the roles drawer of the Database Manager. Do not switch to `admin` to get past it;\n- a migration runs as the role its SQL declares on its first line, `-- role `, the same one in its `down`. One that declares none runs as `admin`, which only those who may use `admin` can do.\n\nRead the Permissions section of https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables before designing who reaches what: it covers tenants, grants, the default role and the roles a connection that names none ends up holding, rows that differ per viewer through row-level security, and how forks and raw apps behave under roles.\n\n\n# Datatable SQL SDK reference\n\nDatatables are workspace-scoped managed PostgreSQL databases. In chat, explore and shape them with the datatable tools you were given. The reference below is for code you author inside runnables (inline app runnables, scripts, or flow rawscript modules) that reads or writes datatable data at runtime.\n\n- A runnable accesses a datatable via `wmill.datatable()` (the default \"main\") or `wmill.datatable('')`, referencing tables as `schema.table`.\n- Use parameterized queries (the tagged template in TypeScript, `$1`/`$2` placeholders in Python) — never interpolate untrusted values into SQL strings.\n\n## TypeScript Datatable API (windmill-client)\n\nImport: `import * as wmill from 'windmill-client'`\n\nSQL statement object with query content, arguments, and execution methods\n```typescript\ntype SqlStatement = {\n /** Raw SQL content with formatted arguments */\n content: string;\n\n /** Argument values keyed by parameter name */\n args: Record;\n\n /**\n * Execute the SQL query and return results\n * @param params - Optional parameters including result collection mode\n * @returns Query results based on the result collection mode\n */\n fetch(\n params?: FetchParams // The union is for auto-completion\n ): Promise>;\n\n /**\n * Execute the SQL query and return only the first row\n * @param params - Optional parameters\n * @returns First row of the query result\n */\n fetchOne(\n params?: Omit, \"resultCollection\">\n ): Promise>;\n\n /**\n * Execute the SQL query and return only the first row as a scalar value\n * @param params - Optional parameters\n * @returns First row of the query result\n */\n fetchOneScalar(\n params?: Omit<\n FetchParams<\"last_statement_first_row_scalar\">,\n \"resultCollection\"\n >\n ): Promise>;\n\n /**\n * Execute the SQL query without fetching rows\n * @param params - Optional parameters\n */\n execute(\n params?: Omit, \"resultCollection\">\n ): Promise;\n};\n```\n\n```typescript\n// Template tag function: sql`SELECT * FROM table WHERE id = ${id}`.fetch()\ninterface DatatableSqlTemplateFunction {\n // Tagged template usage:\n (strings: TemplateStringsArray, ...values: any[]): SqlStatement;\n query(sql: string, ...params: any[]): SqlStatement;\n};\n```\n\nCreate a SQL template function for PostgreSQL/datatable queries\n@param name - Database/datatable name (default: \"main\")\n@param opts.role - Connect as this data table role instead of the data table's default one.\n Only meaningful on a data table under roles, and only for a role you are a tenant of.\n@returns SQL template function for building parameterized queries\n@example\nlet sql = wmill.datatable()\nlet name = 'Robin'\nlet age = 21\nawait sql`\n SELECT * FROM friends\n WHERE name = ${name} AND age = ${age}::int\n`.fetch()\n@example\n// Read through a restricted role\nlet sql = wmill.datatable(\"main\", { role: \"analytics\" })\n```typescript\nfunction datatable(name: string = \"main\",\n opts?: DatatableOptions): DatatableSqlTemplateFunction\n```\n\n\n## Python Datatable API (wmill)\n\nImport: `import wmill`\n\n# Get a DataTable client for SQL queries.\n# \n# Args:\n# name: Database name (default: \"main\")\n# role: Connect as this data table role instead of the data table's default one.\n# \n# Returns:\n# DataTableClient instance\ndef datatable(name: str = 'main', *, role: Optional[str] = None) -> DataTableClient\n\n# Client for executing SQL queries against Windmill DataTables.\nclass DataTableClient:\n # Initialize DataTableClient.\n # \n # Args:\n # client: Windmill client instance\n # name: DataTable name\n # role: Data table role to connect as, or None for the data table's default\n def __init__(client: Windmill, name: str, role: Optional[str] = None)\n\n # Execute a SQL query against the DataTable.\n # \n # Args:\n # sql: SQL query string with $1, $2, etc. placeholders\n # *args: Positional arguments to bind to query placeholders\n # \n # Returns:\n # SqlQuery instance for fetching results\n def query(sql: str, *args) -> SqlQuery\n\n\n# Query result handler for DataTable and DuckLake queries.\nclass SqlQuery:\n # Initialize SqlQuery.\n # \n # Args:\n # sql: SQL query string\n # fetch_fn: Function to execute the query\n def __init__(sql: str, fetch_fn)\n\n # Execute query and fetch results.\n # \n # Args:\n # result_collection: Optional result collection mode\n # \n # Returns:\n # Query results\n def fetch(result_collection: str | None = None)\n\n # Execute query and fetch first row of results.\n # \n # Returns:\n # First row of query results\n def fetch_one()\n\n # Execute query and fetch first row of results. Return result as a scalar value.\n # \n # Returns:\n # First row of query result as a scalar value\n def fetch_one_scalar()\n\n # Execute query and don't return any results.\n # \n def execute()\n\n\n", "pipeline": "# Data pipeline authoring\n\nA **data pipeline** is NOT a flow. A flow is one runnable that orchestrates steps internally. A data pipeline is a set of **independent scripts**, each deployed on its own, that form a DAG by reading and writing shared **storage assets** (DuckLake tables, data tables, S3 objects, volumes, resources) and by declaring execution **triggers**. The pipeline is visualized and edited at `/pipeline/`; every node is a normal workspace script that happens to carry pipeline annotations. When the user asks for a \"data pipeline\" (or to \"ingest / transform / materialize\" data across steps), build pipeline-annotated scripts — do NOT build a flow.\n\n## Default to DuckDB + DuckLake\n\nA pipeline node that produces a table should almost always be a **`duckdb`** node that materializes its output into a **DuckLake** table with `-- materialize ducklake:///` (in a DuckDB node the annotation uses SQL `--` comment syntax; write the body as a bare `SELECT` and let the runtime do the write). DuckLake is the default lakehouse store for pipelines and is the shape the pipeline editor is built around, so prefer it unless the work specifically calls for something else:\n\n- `postgresql` / data tables — only for row-level, OLTP-style mutations against an existing Postgres data table (frequent single-row upserts/updates, transactional reads that an app queries live).\n- `bun` / `python3` — only for work that doesn't map to SQL: calling an external API, wrangling files, arbitrary glue. When such a node produces tabular data for downstream steps, write it **straight into DuckLake** (or a data table) from the same script — see \"Ingesting from Python / TypeScript\" below — rather than inventing a parallel store.\n\nDo not spread a pipeline across postgres, S3, and DuckLake when one DuckLake lake would do; a consistent DuckLake lakehouse is the goal.\n\n## Ingesting from Python / TypeScript\n\nA node that fetches data in code (an API, a SaaS export, a scraper) writes the rows **directly** into their DuckLake table with `wmill.ducklake()`, or into a data table with `wmill.datatable()`. DuckLake already stores every table as parquet files in the workspace object storage, so **never stage the raw payload first** — no intermediate JSON / CSV / parquet file in S3 and no second \"load the raw file\" node. One ingestion node, one table.\n\nPass the rows as a query argument (a list of dicts / array of objects is sent as JSON) and let SQL unnest them; declare the column types so the table gets a real schema:\n\n```python\n# pipeline\n# on schedule\nimport wmill\n\ndef main():\n rows = fetch_charges() # list[dict]\n lake = wmill.ducklake()\n lake.query(\"\"\"\n CREATE OR REPLACE TABLE stripe_charges AS\n SELECT unnest(from_json($rows, '[{\"id\":\"VARCHAR\",\"amount\":\"BIGINT\",\"created\":\"BIGINT\"}]'), recursive := true)\n \"\"\", rows=rows).fetch()\n```\n\n```ts\n// pipeline\n// on schedule\nimport * as wmill from \"windmill-client\"\n\nexport async function main() {\n const rows = await fetchCharges()\n const lake = wmill.ducklake()\n await lake`CREATE OR REPLACE TABLE stripe_charges AS\n SELECT unnest(from_json(${rows}, '[{\"id\":\"VARCHAR\",\"amount\":\"BIGINT\",\"created\":\"BIGINT\"}]'), recursive := true)`.fetch()\n}\n```\n\n- Use `INSERT INTO
SELECT ...` instead of `CREATE OR REPLACE` to append (an incremental pull), with a one-time `CREATE TABLE IF NOT EXISTS` first.\n- Assign the client to a variable (`lake = wmill.ducklake()`) and keep the SQL a **string literal** (Python) or a template without `sql.raw` (TS): that is what lets the graph record the node's write to `ducklake://main/
`. A chained `wmill.ducklake().query(...)` or SQL built in a variable records no output, so the downstream edge never forms.\n- `wmill.ducklake(\"\")` targets a non-default catalog; `wmill.datatable(\"\")` writes a Postgres data table the same way (Python `db.query(sql, *args)` with `$1` placeholders, TS the same template form) — use it only when the data belongs in Postgres (see above).\n- The downstream node triggers on the table itself: `-- on ducklake://main/stripe_charges`.\n\n## Storage prerequisites\n\nA DuckLake pipeline only runs once the workspace has **object storage** (S3 / Azure Blob / GCS) **and a DuckLake catalog** configured — DuckLake tables and `s3://` assets can't be materialized or read without it. Check which DuckLake catalogs the workspace has before you build.\nDrafting the annotated scripts does not require storage, but the pipeline can't ingest, materialize, or read its assets until it exists. So if there is none (or the user hits \"storage not configured\" errors), say so and give the right next step **by role**:\n\n- a workspace **admin** sets it up in Workspace settings → Object Storage (add an S3/Azure/GCS storage), then adds a DuckLake catalog on top of it;\n- anyone **without admin rights** should ask a workspace admin to configure object storage + a DuckLake catalog.\n\nNever hand back a DuckLake pipeline that cannot run without flagging the missing storage and pointing to who sets it up.\n\n## What makes a script a pipeline node\n\nA script joins the pipeline when its source begins with the `pipeline` annotation as a top-of-file comment, **written in the script's own comment syntax** — `//` for TS/JS (bun), `--` for SQL (DuckDB/Postgres), `#` for Python/Bash. So it's `-- pipeline` in a DuckDB node, `# pipeline` in a Python node, `// pipeline` in a bun node. Every annotation below uses that same prefix (the `//` shown is the TS form). All other wiring is expressed as annotation comments near the top of the file:\n\n- `// on ` — declares an execution-DAG **input** (what triggers/feeds this node). `` is either:\n - an **asset URI** (the node runs when that asset is produced upstream): `ducklake://main/orders`, `datatable://main/users`, `$res:f/folder/my_resource`, `volume://name/path`, or an S3 object (see the S3 storage-form rule below).\n - a **native trigger kind**: `schedule`, `webhook`, `email`, `kafka`, `mqtt`, `amqp`, `nats`, `postgres`, `sqs`, `gcp`, or `data_upload` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. **`data_upload` is special**: there is no trigger row — the node instead declares an **`S3Object` input parameter** fed by the auto-generated upload picker; it never hard-codes a key. Any language can be the `data_upload` node:\n - Python (has `import wmill`): `def main(file: wmill.S3Object):` then `wmill.load_s3_file(file)`; TS (has `import * as wmill from \"windmill-client\"`): `export async function main(file: wmill.S3Object)`. Qualify the type as `wmill.S3Object` (or add `from wmill import S3Object` / `import { S3Object } from \"windmill-client\"`) — a bare `S3Object` is undefined.\n - **DuckDB** takes the s3object arg via a `-- $ (s3object)` declaration and reads it directly, so a single DuckDB node can ingest **and** materialize:\n ```\n -- pipeline\n -- on data_upload\n -- materialize ducklake://main/raw_uploads\n -- $file (s3object)\n SELECT * FROM read_csv($file)\n ```\n- **Outputs** are inferred from what the body writes — a `CREATE TABLE`, a `wmill.writeS3File(...)`, a DuckLake/datatable write. To declare a managed output explicitly, use `// materialize `.\n- Optional badges: `// partitioned `, `// freshness ` (e.g. `1h`), `// tag `, `// retry [delay]`, `// data_test ...` (managed DuckLake targets only — deploy rejects it beside a `dbt://` one), `// measure = [where ]` and `// dimension = ` (see \"Declared metrics\" below).\n\n## S3 object wiring (storage form matters)\n\nAn `s3://` URI's first slashes select the **storage**, not part of the key — get this wrong and the producer/consumer edge silently won't connect:\n\n- `s3:///` (**triple** slash, empty first segment) = the **default** workspace storage. A downstream node reading or triggering on that object uses `s3:///` — e.g. DuckDB `-- on s3:///orders/2024.parquet` and `read_parquet('s3:///orders/2024.parquet')`.\n- `s3:///` (**double** slash, non-empty first segment) = a **named secondary** storage called `` — so `s3://ingest/x` means storage `ingest`, key `x`, NOT key `ingest/x`. Only use this when the object genuinely lives in a configured secondary storage; never invent a bucket/storage name for a default-storage object (it breaks the edge).\n\nTo make the producer side visible to lineage, a Python/TS node MUST pass the **`S3Object` form**, not a bare key string: Python `wmill.write_s3_file(wmill.S3Object(s3=\"\"), data)` (or the import-free dict `{\"s3\": \"\"}`), TS `wmill.writeS3File({ s3: \"\" }, data)`. That records the default-storage asset `/`, which a downstream `s3:///` reader connects to (same key both sides). A bare `write_s3_file(\"\", ...)` records **no** asset and produces **no** edge. Add `storage=\"\"` only for a named secondary storage.\n\nThe key must be a **string literal** — the graph parser is static and cannot follow a variable, f-string, or computed path, so `write_s3_file(wmill.S3Object(s3=key_var), ...)` records no edge. Inline the literal (`s3=\"events/user_events.parquet\"`) on both the writing and reading node. The same rule applies to every asset URI in an annotation or SDK call (`ducklake://`, `datatable://`, `s3://`): write them literally, not via a variable.\n\n## Materialize (the managed output)\n\n> **A MANAGED `// materialize` is DuckDB-only**, and its target must be a DuckLake table (`ducklake:///
`). Deploy **rejects** a `ducklake://` `// materialize` on any other language (`python3`, `bun`, `postgresql`). For a non-DuckDB node writing the lake, do **not** use `// materialize` — write the table itself with `wmill.ducklake()` (see \"Ingesting from Python / TypeScript\") and let the output be inferred. Use `duckdb` when a node should materialize a DuckLake table.\n>\n> The one target ANY language may declare (except a dbt script, whose writes come from its manifest) is a **warehouse relation**: `// materialize manual dbt:////`, where `` is a warehouse the workspace configures under Settings → dbt. `manual` is the only mode it has — nothing generates warehouse DDL, so the node issues its own write (a postgresql `CREATE TABLE` / `INSERT`, an SDK load, …) and the annotation records the outcome. Use it on an ingestion node whose output a dbt project reads as a `source`: the declared relation and the dbt model land on ONE graph node, and a downstream `// on dbt:////` fires when the ingestion node completes.\n\nA managed `// materialize ducklake:///
` tells the runtime to write the node's output table **for you**: write the body as a single `SELECT` and the runtime wraps it in the create/replace — do **not** also write your own `CREATE TABLE` / `INSERT`. (The opposite holds for the `dbt://` target above: there the node writes its own DDL and the strategies below do not apply.)\n\nA target that names no table — `// materialize`, `// materialize ducklake://`, or options alone such as `// materialize key=id` — writes the table named after the script: node `f/shop/orders_daily` materializes `ducklake://main/orders_daily` (characters other than letters, digits and `_` become `_`, and a name starting with a digit gets a leading `_`). Name the node after the table it produces and the bare form is enough; write the full target when the table's name differs from the node's, and use that same name in every downstream `// on` and read.\n\nWrite strategy:\n\n- no option → **replace** the whole table each run (full refresh; the only mode whose output columns may change);\n- `// materialize append` → INSERT-append rows (incremental);\n- `// materialize key=` → merge/upsert on ``.\n\n`// materialize manual ` opts **out** of managed writes — the script writes its own DDL and the annotation only records the output asset for lineage.\n\n`materialize` pairs with partitioning for incremental pipelines: a `// partitioned ` node runs **once per partition** (append/merge into a fixed-schema table). The `{partition}` token, usable in any asset URI **and** in the body SQL, is replaced at run time by the current partition's **identity string**:\n\n- To filter the source to the active slice on a time grain, use the runtime-injected macro: `WHERE wm_partition() = {partition}`. `wm_partition(ts)` buckets a timestamp in exactly the identity format the runtime uses for daily/hourly/weekly/monthly, so it always matches; never hand-write a `strftime` format.\n- Do NOT write `= TIMESTAMP {partition}`: the identity string is not a valid timestamp literal for hourly/weekly/monthly and errors at run time.\n- For `dynamic` partitioning the identity is the caller-supplied key (not a timestamp, no macro), so filter on it directly: `WHERE = {partition}`.\n\n`materialize` is an output **declaration** on a node — not a command. There is no \"materialize run\".\n\n## Declared metrics (`measure` / `dimension`)\n\nOn a node that materializes a DuckLake table, `// measure = [where ]` names the canonical way to aggregate that table (e.g. `// measure revenue = sum(amount) where not is_refund`), and `// dimension = ` names a way to slice it (e.g. `// dimension region = region`, `// dimension month = date_trunc('month', ordered_at)`). They execute nothing: they are catalogued at deploy so the editor and other agents reuse the definition instead of re-deriving it and silently disagreeing.\n\n- Keep the predicate in the `where` clause rather than folding it into the aggregate: it is rendered as ` FILTER (WHERE )`, which is what lets two measures with different predicates share one GROUP BY.\n- DuckLake-only, and only meaningful next to `// materialize`.\n- Declare one when a number carries a judgement call someone else would get wrong (refunds excluded, test rows dropped, which column is the amount); do NOT blanket every table with measures — an obvious `count(*)` earns nothing.\n- To use a metric another node declares, read that node and reuse its exact expression rather than guessing it.\n\n## How to build one\n\n1. Put every node in the **same folder**: `f//`. The folder is the pipeline.\n2. Write each node as its own script. Default to `duckdb` materializing into DuckLake (see \"Default to DuckDB + DuckLake\" above); pick `postgresql`, `bun`, or `python3` only when that section says the work calls for it.\n3. Start each body with `// pipeline`, then the `// on` input declarations, then the transform that writes the output.\n4. **Chain nodes by asset URI**: read an upstream node's output asset, then `// on ` in the downstream node so the edge forms. Reuse exact asset paths from existing nodes rather than inventing parallel ones.\n5. Never guess the columns of a file or table you have not seen: read them first with a query that writes nothing, such as `DESCRIBE SELECT * FROM read_csv('s3:///raw/orders/*.csv')` or `ATTACH 'ducklake://main' AS lake; DESCRIBE lake.orders`, and use those exact column names. A test run of a node that materializes a table writes that table into the workspace's lake: test-run such a node only while its table does not exist yet, and ask before running one that would replace a table already there.\n6. Don't deploy nodes unless the user asks to. A pipeline only \"runs\" once its scripts are deployed and their triggers exist.\n\n## Example (DuckDB → DuckLake, scheduled ingest + downstream transform)\n\nNode `f/sales/orders_ingest` (runs on a schedule, materializes a DuckLake table):\n\n```sql\n-- pipeline\n-- on schedule\n-- materialize ducklake://main/orders\nSELECT * FROM read_csv('s3:///raw/orders/*.csv')\n```\n\nNode `f/sales/orders_daily` (runs when `orders` is produced, writes a rollup):\n\n```sql\n-- pipeline\n-- on ducklake://main/orders\n-- materialize ducklake://main/orders_daily\nATTACH 'ducklake://main' AS lake;\nSELECT date_trunc('day', ts) AS day, count(*) AS n\nFROM lake.orders GROUP BY 1\n```\n\nA node reads a DuckLake table only through a lake it attaches itself (`ATTACH 'ducklake://' AS lake;`, then `lake.
`); the materialize target is attached by the runtime under a name of its own, so `ducklake.main.
` does not resolve.\n", - "workspace_settings": "# Workspace settings\n\nEverything an admin sets on the Workspace settings page is one document: a mapping of top-level\nkeys, the same layout as the `settings.yaml` file `wmill sync` keeps. Only a workspace admin can\nread or change it.\n\n## Applying a document\n\n- A partial document changes only the keys it states; every key it omits is left as it is.\n- Each key is replaced whole, so read its current value and send it back edited: a nested object\n given with one field replaces the stored object with that one field.\n- A key set to `null` clears that setting; an on/off switch cleared this way reads as `false`.\n- Each setting is written through the same path the settings page uses, with the same validation\n and audit log. The writes are not one transaction: when one fails the apply stops, and the\n error says what was already written.\n- A dry run reports the changes, as `before` / `after` per key, without writing anything.\n- `name` is never applied: rename a workspace on the settings page. `color` is applied only when\n set to a value, so `null` does not clear it.\n- A key the server does not know is ignored and named in the warnings: check them, since a typo\n otherwise reads as \"nothing to change\".\n- A setting this edition does not offer (Enterprise only) is refused, by the dry run too.\n\n## Secrets\n\n`slack_oauth_client_secret` is the one secret. A read shows `__SENSITIVE_AND_UNCHANGED__` in place\nof a set secret; sending that placeholder back keeps the stored value. A value starting with\n`$encrypted:` is decrypted with the workspace key, and one encrypted with another workspace's key\nis left unchanged with a warning; any other value is taken as plaintext. Never\nwrite a secret you were not given: keep the placeholder.\n\n## Keys\n\n| Key | Value |\n|---|---|\n| `auto_invite` | `{ enabled, mode: invite \\| add, operator, instance_groups?, instance_groups_roles? }`. The instance groups are managed only when `instance_groups` is stated; clearing a group removes the members it added. |\n| `error_handler` / `success_handler` | `{ path, extra_args?, muted_on_cancel?, muted_on_user_path? }`, `path` being `script/` or `flow/`. |\n| `error_handler_fallback_to_instance_alerts` | Report failures to the instance critical alert channels when no error handler is set. |\n| `webhook` | URL POSTed a JSON event on every change to the workspace's items. |\n| `ai_config` | Providers, default and completion models, custom prompts, `copilot_disabled`. |\n| `large_file_storage` | Workspace object storage: `{ type, s3_resource_path \\| azure_blob_resource_path \\| gcs_resource_path, public_resource?, secondary_storage?, advanced_permissions? }`. |\n| `git_sync` | `{ repositories: [...] }`, each with `git_repo_resource_path`, `script_path` and `settings` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. |\n| `datatable` | `{ datatables: { : { database: { resource_type, resource_path } } } }`, both required: `resource_type: postgresql` with `resource_path` a Postgres resource's path, or `instance`, or `external_instance` (Enterprise only), with `resource_path` the name of a database a superadmin set up. A fork's entry can instead be `{ reference: { workspace_id, datatable } }`, pointing at its parent's data table; keep it as is. `main` is the one scripts use by default. Role permissions are `datatable_permissions`. |\n| `datatable_permissions` | `{ : { default_role, roles: { : [tenants] } } }`: who may connect to each data table as which role. Tenants are `u/`, `g/`, `f/` or `*`; `admin` is always a role, and `default_role: null` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. |\n| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. |\n| `ducklake` | `{ ducklakes: { : { catalog, storage, ... } } }`. |\n| `dbt_warehouses` | dbt warehouse connections. |\n| `default_app` | Path of the app shown as the workspace home. |\n| `default_scripts` | `{ order?, hidden?, default_script_content? }` for the script language picker. |\n| `deploy_ui` | `{ include_path?, include_type? }`: what the deploy-to-workspace UI offers. |\n| `operator_settings` | What operators see and do: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). A write right the value omits keeps its stored value. |\n| `mute_critical_alerts` | Mute the workspace's critical alerts. |\n| `slack_team_id`, `slack_name` | Read only: set by connecting Slack on the settings page. |\n| `slack_command_script` / `teams_command_script` | Script or flow handling the Slack / Teams command. |\n| `slack_oauth_client_id` / `slack_oauth_client_secret` | The workspace's own Slack OAuth app. Clearing either removes both. |\n| `public_app_execution_limit_per_minute` | Rate limit for public apps, `null` for none. |\n| `guest_jwt_public_key`, `guest_jwt_jwks_url` | The key that signs guest JWTs for apps set to Guests. |\n| `add_admins_and_developers_to_forks` | Add the workspace's admins and developers to each fork. |\n| `color` | Workspace color, `#rrggbb`. |\n" + "workspace_settings": "# Workspace settings\n\nEverything an admin sets on the Workspace settings page is one document: a mapping of top-level\nkeys, the same layout as the `settings.yaml` file `wmill sync` keeps. Only a workspace admin can\nread or change it.\n\n## Applying a document\n\n- A partial document changes only the keys it states; every key it omits is left as it is.\n- Each key is replaced whole, so read its current value and send it back edited: a nested object\n given with one field replaces the stored object with that one field.\n- A key set to `null` clears that setting; an on/off switch cleared this way reads as `false`.\n- Each setting is written through the same path the settings page uses, with the same validation\n and audit log. The writes are not one transaction: when one fails the apply stops, and the\n error says what was already written.\n- A dry run reports the changes, as `before` / `after` per key, without writing anything.\n- `name` is never applied: rename a workspace on the settings page. `color` is applied only when\n set to a value, so `null` does not clear it.\n- A key the server does not know is ignored and named in the warnings: check them, since a typo\n otherwise reads as \"nothing to change\".\n- A setting this edition does not offer (Enterprise only) is refused, by the dry run too.\n\n## Secrets\n\n`slack_oauth_client_secret` is the one secret. A read shows `__SENSITIVE_AND_UNCHANGED__` in place\nof a set secret; sending that placeholder back keeps the stored value. A value starting with\n`$encrypted:` is decrypted with the workspace key, and one encrypted with another workspace's key\nis left unchanged with a warning; any other value is taken as plaintext. Never\nwrite a secret you were not given: keep the placeholder.\n\n## Keys\n\n| Key | Value |\n|---|---|\n| `auto_invite` | `{ enabled, mode: invite \\| add, operator, instance_groups?, instance_groups_roles? }`. The instance groups are managed only when `instance_groups` is stated; clearing a group removes the members it added. |\n| `error_handler` / `success_handler` | `{ path, extra_args?, muted_on_cancel?, muted_on_user_path? }`, `path` being `script/` or `flow/`. |\n| `error_handler_fallback_to_instance_alerts` | Report failures to the instance critical alert channels when no error handler is set. |\n| `webhook` | URL POSTed a JSON event on every change to the workspace's items. |\n| `ai_config` | Providers, default and completion models, custom prompts, `copilot_disabled`. |\n| `large_file_storage` | Workspace object storage: `{ type, s3_resource_path \\| azure_blob_resource_path \\| gcs_resource_path, public_resource?, secondary_storage?, advanced_permissions? }`. |\n| `git_sync` | `{ repositories: [...] }`, each with `git_repo_resource_path`, `script_path` and `settings` (include/exclude paths and types). Server-owned fields (webhook secrets, sync state, credentials) are never read nor written. |\n| `datatable` | `{ datatables: { : { database: { resource_type, resource_path } } } }`, both required: `resource_type: postgresql` with `resource_path` a Postgres resource's path, or `instance`, or `external_instance` (Enterprise only), with `resource_path` the name of a database a superadmin set up. A fork's entry can instead be `{ reference: { workspace_id, datatable } }`, pointing at its parent's data table; keep it as is. `main` is the one scripts use by default. Role permissions are `datatable_permissions`. |\n| `datatable_permissions` | `{ : { default_role, roles: { : [tenants] } } }`: who may connect to each data table as which role. Tenants are `u/`, `g/`, `f/` or `*`; `admin` is always a role, and `default_role: null` means no default. A data table the value omits is no longer under roles. Only a data table on a cluster Windmill manages can be. |\n| `protection_rules` | `{ : { rules: [...], bypass_groups: [...], bypass_users: [...], fork_role_mapping?: { : } } }`, rules among `DisableDirectDeployment`, `DisableWorkspaceForking`, `RestrictDeployToDeployers`, `RestrictAnonymousAppDeployment`, `RestrictPublicRunSharing`, `RestrictGuestAppDeployment`, `RequireIsolatedForks`. A rule the value omits is deleted. The dev workspace lock is not listed and cannot be set here. |\n| `ducklake` | `{ ducklakes: { : { catalog, storage, ... } } }`. |\n| `dbt_warehouses` | dbt warehouse connections. |\n| `default_app` | Path of the app shown as the workspace home. |\n| `default_scripts` | `{ order?, hidden?, default_script_content? }` for the script language picker. |\n| `deploy_ui` | `{ include_path?, include_type? }`: what the deploy-to-workspace UI offers. |\n| `role_overrides` | `{ : { label?, capabilities, workspace_only? } }`: the workspace's overrides of the instance's roles, and with `workspace_only: true` a role of this workspace only (Enterprise). A role the value omits loses its override. Overriding a role the instance does not define is refused. Edit operator rights in `role_overrides.operator`. |\n| `operator_settings` | Legacy view of the `operator` role, kept for older servers: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). Not applied when the document also states `role_overrides`. |\n| `mute_critical_alerts` | Mute the workspace's critical alerts. |\n| `slack_team_id`, `slack_name` | Read only: set by connecting Slack on the settings page. |\n| `slack_command_script` / `teams_command_script` | Script or flow handling the Slack / Teams command. |\n| `slack_oauth_client_id` / `slack_oauth_client_secret` | The workspace's own Slack OAuth app. Clearing either removes both. |\n| `public_app_execution_limit_per_minute` | Rate limit for public apps, `null` for none. |\n| `guest_jwt_public_key`, `guest_jwt_jwks_url` | The key that signs guest JWTs for apps set to Guests. |\n| `add_admins_and_developers_to_forks` | Add the workspace's admins and developers to each fork. |\n| `color` | Workspace color, `#rrggbb`. |\n" } } diff --git a/backend/windmill-api/src/workspace_settings_document.rs b/backend/windmill-api/src/workspace_settings_document.rs index 423bd07250..bcf7022184 100644 --- a/backend/windmill-api/src/workspace_settings_document.rs +++ b/backend/windmill-api/src/workspace_settings_document.rs @@ -25,7 +25,7 @@ use windmill_common::{ read_role_catalog, DatatableRoleCluster, ADMIN_DATATABLE_ROLE, NO_DEFAULT_DATATABLE_ROLE, }, error::{Error, JsonResult, Result}, - roles::{load_role_overrides, RoleOverrides}, + roles::{self, load_role_overrides, RoleDefinition, RoleOverride, RoleOverrides}, utils::require_admin, variables::build_crypt, workspaces::{ @@ -260,14 +260,48 @@ fn datatable_permissions_document(datatable: Option<&Value>, roles: &DatatableNa Value::Object(doc) } -/// Protection rules by name, their rules in a fixed order so a reordered file is no change. The -/// dev workspace lock is left out: the dev workspace setting creates and removes it. -/// The workspace's role overrides keyed by role name, sorted so a pulled file is stable. -fn role_overrides_document(overrides: &RoleOverrides) -> Value { - let sorted: BTreeMap<&String, _> = overrides.iter().collect(); - serde_json::to_value(sorted).unwrap_or_else(|_| json!({})) +/// The workspace's role overrides keyed by role name, sorted so a pulled file is stable. The stored +/// map does not tell an override of instance role X from adding role X, so the entry of a role +/// the instance does not define carries [`WORKSPACE_ONLY`]: without it, the target instance +/// lacking X would silently turn the override into a new role. +fn role_overrides_document(overrides: &RoleOverrides, instance: &[RoleDefinition]) -> Value { + let sorted: BTreeMap<&String, &RoleOverride> = overrides.iter().collect(); + let mut doc = serde_json::to_value(sorted).unwrap_or_else(|_| json!({})); + if let Some(doc) = doc.as_object_mut() { + for (name, entry) in doc.iter_mut() { + if !instance.iter().any(|d| &d.name == name) { + entry[WORKSPACE_ONLY] = Value::Bool(true); + } + } + } + doc } +const WORKSPACE_ONLY: &str = "workspace_only"; + +/// A `role_overrides` entry as the document states it. +#[derive(Deserialize)] +struct DocumentRoleOverride { + #[serde(flatten)] + role: RoleOverride, + #[serde(default)] + workspace_only: bool, +} + +/// What `operator_settings` holds for the effective `operator` role: the column is rewritten only +/// by the legacy endpoint, so after a role edit it holds stale flags. +fn implied_operator_settings( + instance: &[RoleDefinition], + overrides: &RoleOverrides, +) -> Result> { + Ok(roles::effective_roles(instance, overrides)? + .iter() + .find(|r| r.name == roles::OPERATOR_ROLE) + .map(|r| roles::legacy_operator_settings(&r.capabilities))) +} + +/// Protection rules by name, their rules in a fixed order so a reordered file is no change. The +/// dev workspace lock is left out: the dev workspace setting creates and removes it. async fn protection_rules_document(conn: &mut PgConnection, w_id: &str) -> Result { let rows = sqlx::query_as::<_, (String, i32, Vec, Vec, Value)>( "SELECT name, rules, bypass_groups, bypass_users, fork_role_mapping FROM workspace_protection_rule @@ -360,7 +394,14 @@ pub async fn load_settings_document( }; let datatable_permissions = datatable_permissions_document(row.datatable.as_ref(), &role_names); let protection_rules = protection_rules_document(&mut *conn, w_id).await?; - let role_overrides = role_overrides_document(&load_role_overrides(&mut *conn, w_id).await?); + let instance = roles::instance_roles_on(&mut *conn).await?; + let overrides = load_role_overrides(&mut *conn, w_id).await?; + let role_overrides = role_overrides_document(&overrides, &instance); + // Present exactly when the column is, as it always was. + let operator_settings = match row.operator_settings { + Some(stored) => Some(implied_operator_settings(&instance, &overrides)?.unwrap_or(stored)), + None => None, + }; let git_sync = redact_git_sync(row.git_sync); let git_sync = match fork_parent(&mut *conn, w_id).await? { Some(parent) => fork_git_sync_document(git_sync, parent.git_sync), @@ -393,7 +434,7 @@ pub async fn load_settings_document( false, ), ("color", row.color.map(Value::String), false), - ("operator_settings", row.operator_settings, false), + ("operator_settings", operator_settings, false), ("role_overrides", Some(role_overrides), true), ( "datatable", @@ -923,12 +964,29 @@ async fn plan(db: &DB, w_id: &str, mut desired: Map, replace: boo None => current.remove("git_sync"), }; } + if !desired.contains_key("role_overrides") { + // A file without the roles applies `operator_settings` as it always did: diffed against + // the column, since against the value the document derives, an unchanged file with a + // stale column would undo a role edit made since. + let column: Option = sqlx::query_scalar( + "SELECT operator_settings FROM workspace_settings WHERE workspace_id = $1", + ) + .bind(w_id) + .fetch_optional(db) + .await? + .flatten(); + match column { + Some(column) => current.insert("operator_settings".to_string(), column), + None => current.remove("operator_settings"), + }; + } let mc = build_crypt(db, w_id).await?; let names = match desired.get("datatable_permissions") { Some(v) if !v.is_null() => DatatableNames::load(db, w_id, v).await?, _ => DatatableNames::default(), }; - plan_document(current, desired, replace, &mc, &names) + let instance = roles::instance_roles(db).await?; + plan_document(current, desired, replace, &mc, &names, &instance) } /// A fork's parent, read for its `git_sync`. @@ -1078,6 +1136,7 @@ fn plan_document( replace: bool, mc: &windmill_common::variables::WorkspaceCrypt, names: &DatatableNames, + instance: &[RoleDefinition], ) -> Result { let mut warnings = vec![]; let unknown: Vec<&str> = desired @@ -1230,6 +1289,10 @@ fn plan_document( let stated: Vec = desired.keys().cloned().collect(); let mut p = Plan { current, desired, replace, changes: vec![], writes: vec![], warnings }; + // Before every other setting, so an instance group mapping or invite can name a role the + // file adds; a workspace-only role it removes is dropped last, after whatever named it. + let deferred_roles = plan_role_overrides(&mut p, instance)?; + let webhook = p.legacy("webhook"); // The endpoint clears the URL when the body omits it, so the events write sends it along. let url = webhook @@ -1443,7 +1506,13 @@ fn plan_document( |v| json!({ "color": v }), ); - if let Some(desired) = p.managed("operator_settings").filter(|o| !o.is_null()) { + // A file that states the roles applies the operator role through them alone: the legacy key + // beside them would otherwise re-grant or withdraw flags the file's roles do not hold. + let roles_stated = p.managed("role_overrides").is_some(); + if let Some(desired) = p + .managed("operator_settings") + .filter(|o| !o.is_null() && !roles_stated) + { // The endpoint keeps the stored value of a write right the document omits, so an // older document must not count as differing on those keys. let mut compared = desired.clone(); @@ -1472,23 +1541,6 @@ fn plan_document( } } - // Stated as a whole, after `operator_settings`: a role the map omits loses its override, and - // the `operator` override it states wins over the legacy key. - if let Some(desired) = p.managed("role_overrides") { - let desired = if desired.is_null() { - json!({}) - } else { - desired - }; - let overrides: RoleOverrides = serde_json::from_value(desired) - .map_err(|e| Error::BadRequest(format!("Invalid role_overrides: {e}")))?; - let desired = role_overrides_document(&overrides); - if p.differs("role_overrides", &desired) { - p.record("role_overrides", desired.clone()); - p.write(Target::Edit(SettingsEdit::RoleOverrides), desired); - } - } - let datatable = p.legacy("datatable"); p.simple( "datatable", @@ -1648,11 +1700,25 @@ fn plan_document( .iter() .any(|k| c.key == *k || c.key.starts_with(&format!("{k}."))) }); - if replace && !changes_stated { + if replace && !changes_stated && deferred_roles.is_none() { p.changes.clear(); p.writes.clear(); } + match deferred_roles { + Some((Some(after), body)) => { + p.record("role_overrides", after); + p.write(Target::Edit(SettingsEdit::RoleOverrides), body); + } + // Recorded with the staged write. + Some((None, body)) => p.writes.push(Write { + target: Target::Edit(SettingsEdit::RoleOverrides), + body, + keys: vec![], + }), + None => {} + } + // Refused here rather than by the endpoint, so a dry run already says so. let unavailable: Vec = p .writes @@ -1670,6 +1736,117 @@ fn plan_document( Ok(p) } +/// Plans `role_overrides`, stated as a whole: a role the map omits loses its override. Refuses, +/// before any setting is written, an override of a role the instance does not define. When the +/// file removes a workspace-only role, the role is kept until the final write it returns (with +/// the change still to record, unless a staged write recorded it), so the in-use check passes +/// once the settings naming the role have been rewritten. +fn plan_role_overrides( + p: &mut Plan, + instance: &[RoleDefinition], +) -> Result, Value)>> { + let Some(desired) = p.managed("role_overrides") else { + return Ok(None); + }; + let stated: BTreeMap = if desired.is_null() { + BTreeMap::new() + } else { + serde_json::from_value(desired) + .map_err(|e| Error::BadRequest(format!("Invalid role_overrides: {e}")))? + }; + let previous: RoleOverrides = object_of(p.current.get("role_overrides")) + .into_iter() + .filter_map(|(name, v)| { + Some(( + name, + serde_json::from_value::(v).ok()?.role, + )) + }) + .collect(); + let defined = |name: &str| instance.iter().any(|d| d.name == name); + let mut errors = vec![]; + for (name, o) in &stated { + match (defined(name), o.workspace_only) { + (true, true) => errors.push(format!( + "role_overrides marks `{name}` as workspace_only, but this instance defines it: \ + remove `workspace_only` to override the instance role" + )), + // An unmarked entry from a file pulled before the marker existed still round-trips + // a role the workspace already has. + (false, false) if !previous.contains_key(name) => errors.push(format!( + "role_overrides overrides role `{name}`, which this instance does not define: \ + define it in the instance settings (Workspace roles) first, or set \ + `workspace_only: true` on the entry to add it as a role of this workspace only" + )), + _ => {} + } + } + if !errors.is_empty() { + return Err(Error::BadRequest(errors.join("; "))); + } + let next: RoleOverrides = stated.into_iter().map(|(n, o)| (n, o.role)).collect(); + + // Silent only for a legacy key left as pulled, or agreeing with the role it is not applied to. + if let (Some(legacy), Some(implied)) = ( + p.managed("operator_settings").filter(Value::is_object), + implied_operator_settings(instance, &next)?, + ) { + let pulled = implied_operator_settings(instance, &previous)?; + if legacy_operator_settings_differ(&legacy, &implied) + && pulled.is_none_or(|b| legacy_operator_settings_differ(&legacy, &b)) + { + p.warnings.push( + "operator_settings was not applied: role_overrides.operator is what applies. \ + Edit the operator role there, or pull again to refresh operator_settings." + .to_string(), + ); + } + } + + let after = role_overrides_document(&next, instance); + if !p.differs("role_overrides", &after) { + return Ok(None); + } + roles::validate_overrides(instance, &next)?; + roles::check_frozen_overrides(instance, &previous, &next)?; + let body = |o: &RoleOverrides| { + serde_json::to_value(o) + .map_err(|e| Error::internal_err(format!("serializing role overrides: {e:#}"))) + }; + let mut staged = next.clone(); + for (name, o) in &previous { + if !next.contains_key(name) && !defined(name) { + staged.insert(name.clone(), o.clone()); + } + } + if staged == next { + p.record("role_overrides", after); + p.write(Target::Edit(SettingsEdit::RoleOverrides), body(&next)?); + return Ok(None); + } + if staged == previous { + return Ok(Some((Some(after), body(&next)?))); + } + p.record("role_overrides", after); + p.write(Target::Edit(SettingsEdit::RoleOverrides), body(&staged)?); + Ok(Some((None, body(&next)?))) +} + +/// Whether the endpoint would apply `stated` as something other than `reference`: it keeps the +/// stored value of an omitted write right, and reads an omitted visibility flag as `false`. +fn legacy_operator_settings_differ(stated: &Value, reference: &Value) -> bool { + object_of(Some(reference)).iter().any(|(key, value)| { + let kept = matches!( + key.as_str(), + "builder_flows" | "builder_apps" | "manage_schedules" | "manage_triggers" + ); + match stated.get(key) { + Some(v) => v != value, + None => !kept && value != &Value::Bool(false), + } + }) +} + fn object_of(v: Option<&Value>) -> Map { v.and_then(Value::as_object).cloned().unwrap_or_default() } @@ -1989,6 +2166,23 @@ mod tests { v.as_object().unwrap().clone() } + fn plan_document( + current: Map, + desired: Map, + replace: bool, + mc: &WorkspaceCrypt, + names: &DatatableNames, + ) -> Result { + super::plan_document( + current, + desired, + replace, + mc, + names, + &roles::default_instance_roles(), + ) + } + fn keys(p: &Plan) -> Vec<&str> { p.changes.iter().map(|c| c.key.as_str()).collect() } @@ -2266,48 +2460,104 @@ mod tests { } #[test] - fn role_overrides_are_applied_as_a_whole_after_operator_settings() { + fn role_overrides_name_only_roles_the_instance_defines_or_marks() { + let mc = WorkspaceCrypt::new("k"); + let names = DatatableNames::default(); + let mut cur = current(); + cur.insert( + "role_overrides".into(), + json!({ "auditor": { "workspace_only": true, "capabilities": { "code": false } } }), + ); + let plan = |roles: Value| { + plan_document( + cur.clone(), + doc(json!({ "webhook": "https://b", "role_overrides": roles })), + false, + &mc, + &names, + ) + }; + + // An override of a role the instance lacks refuses the whole document. + let err = plan(json!({ "reviewer": { "capabilities": { "code": false } } })) + .err() + .unwrap() + .to_string(); + assert!(err.contains("`reviewer`"), "{err}"); + assert!(plan(json!({ "operator": { "workspace_only": true } })).is_err()); + + // A file pulled before the marker still states the workspace's own role. + let p = plan(json!({ "auditor": { "capabilities": { "code": false } } })).unwrap(); + assert_eq!(keys(&p), ["webhook"]); + } + + #[test] + fn operator_settings_is_not_applied_beside_the_roles() { + let mc = WorkspaceCrypt::new("k"); + let names = DatatableNames::default(); + let mut cur = current(); + cur.insert("role_overrides".into(), json!({})); + let pulled = roles::legacy_operator_settings(&roles::RoleCapabilities::OPERATOR); + let mut edited = pulled.clone(); + edited["builder_flows"] = json!(true); + let plan = |legacy: &Value, roles: Value| { + let d = doc(json!({ "operator_settings": legacy, "role_overrides": roles })); + plan_document(cur.clone(), d, false, &mc, &names).unwrap() + }; + + let p = plan(&pulled, json!({})); + assert!(p.writes.is_empty() && p.warnings.is_empty()); + + let p = plan( + &pulled, + json!({ "operator": { "capabilities": { "flows": "compose" } } }), + ); + assert_eq!(keys(&p), ["role_overrides"]); + assert!(p.warnings.is_empty()); + + let p = plan(&edited, json!({})); + assert!(p.writes.is_empty()); + assert_eq!(p.warnings.len(), 1); + } + + #[cfg(feature = "enterprise")] + #[test] + fn workspace_only_roles_are_added_first_and_removed_last() { let mc = WorkspaceCrypt::new("k"); let mut cur = current(); cur.insert( "role_overrides".into(), - json!({ "operator": { "capabilities": { "manage_schedules": false } } }), + json!({ "auditor": { "workspace_only": true, "capabilities": {} } }), + ); + cur.insert( + "auto_invite".into(), + json!({ "instance_groups": ["g"], "instance_groups_roles": { "g": "auditor" } }), ); - cur.insert("operator_settings".into(), json!({ "runs": true })); - - // Unchanged: no write. - let d = doc(json!({ "role_overrides": { - "operator": { "capabilities": { "manage_schedules": false } }, - } })); - let p = plan_document(cur.clone(), d, false, &mc, &DatatableNames::default()).unwrap(); - assert!(keys(&p).is_empty()); - - // A custom role added next to the legacy key: role_overrides is written last. let d = doc(json!({ - "operator_settings": { "runs": false }, - "role_overrides": { - "operator": { "capabilities": { "manage_schedules": false } }, - "reviewer": { "label": "Reviewer", "capabilities": { "code": false } }, - }, + "role_overrides": { "reviewer": { "workspace_only": true, "capabilities": {} } }, + "auto_invite": { "instance_groups": ["g"], "instance_groups_roles": { "g": "reviewer" } }, })); let p = plan_document(cur.clone(), d, false, &mc, &DatatableNames::default()).unwrap(); - assert_eq!(keys(&p), ["operator_settings", "role_overrides"]); + assert_eq!(keys(&p), ["role_overrides", "auto_invite.instance_groups"]); + let bodies: Vec<_> = p.writes.iter().map(|w| (&w.target, &w.body)).collect(); + assert!(matches!( + bodies.as_slice(), + [ + (Target::Edit(SettingsEdit::RoleOverrides), staged), + (Target::Edit(SettingsEdit::InstanceGroups), _), + (Target::Edit(SettingsEdit::RoleOverrides), last), + ] if staged.get("auditor").is_some() && staged.get("reviewer").is_some() + && last.get("auditor").is_none() + )); + + // A push removing only the role still clears what the file omits. + let d = doc(json!({ "role_overrides": {} })); + let p = plan_document(cur, d, true, &mc, &DatatableNames::default()).unwrap(); + assert!(keys(&p).contains(&"webhook")); assert!(matches!( p.writes.last().unwrap().target, Target::Edit(SettingsEdit::RoleOverrides) )); - - // An empty map is stated, so it clears the workspace's overrides. - let p = plan_document( - cur, - doc(json!({ "role_overrides": {} })), - false, - &mc, - &DatatableNames::default(), - ) - .unwrap(); - assert_eq!(keys(&p), ["role_overrides"]); - assert_eq!(p.writes[0].body, json!({})); } #[test] diff --git a/backend/windmill-common/src/roles.rs b/backend/windmill-common/src/roles.rs index 6008253001..c2de5520c0 100644 --- a/backend/windmill-common/src/roles.rs +++ b/backend/windmill-common/src/roles.rs @@ -323,6 +323,21 @@ pub fn diff_capabilities( } } +/// A role's capabilities in the pre-roles `operator_settings` shape, which older clients gate on. +pub fn legacy_operator_settings(c: &RoleCapabilities) -> serde_json::Value { + let v = &c.visibility; + serde_json::json!({ + "runs": v.runs, "schedules": v.schedules, "resources": v.resources, + "variables": v.variables, "assets": v.assets, "triggers": v.triggers, + "audit_logs": v.audit_logs, "groups": v.groups, "folders": v.folders, + "workers": v.workers, + "builder_flows": c.flows != AuthorLevel::None, + "builder_apps": c.apps != AuthorLevel::None, + "manage_schedules": c.manage_schedules, + "manage_triggers": c.manage_triggers, + }) +} + /// One entry of the instance's [`WORKSPACE_ROLES_SETTING`]. #[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq)] pub struct RoleDefinition { @@ -573,7 +588,8 @@ fn cached_instance_roles() -> Option>> { .map(|(r, _)| r) } -async fn instance_roles_on(conn: &mut sqlx::PgConnection) -> Result>> { +/// [`instance_roles`] on a connection the caller already holds. +pub async fn instance_roles_on(conn: &mut sqlx::PgConnection) -> Result>> { if let Some(roles) = cached_instance_roles() { return Ok(roles); } diff --git a/cli/src/guidance/skills.gen.ts b/cli/src/guidance/skills.gen.ts index 13829cfb94..428dbad87b 100644 --- a/cli/src/guidance/skills.gen.ts +++ b/cli/src/guidance/skills.gen.ts @@ -8514,7 +8514,8 @@ write a secret you were not given: keep the placeholder. | \`default_app\` | Path of the app shown as the workspace home. | | \`default_scripts\` | \`{ order?, hidden?, default_script_content? }\` for the script language picker. | | \`deploy_ui\` | \`{ include_path?, include_type? }\`: what the deploy-to-workspace UI offers. | -| \`operator_settings\` | What operators see and do: page visibility flags (\`runs\`, \`schedules\`, \`resources\`, \`variables\`, \`assets\`, \`audit_logs\`, \`groups\`, \`folders\`, \`workers\`, \`triggers\`) and write rights (\`manage_schedules\`, \`manage_triggers\`, \`builder_flows\`, \`builder_apps\`). A write right the value omits keeps its stored value. | +| \`role_overrides\` | \`{ : { label?, capabilities, workspace_only? } }\`: the workspace's overrides of the instance's roles, and with \`workspace_only: true\` a role of this workspace only (Enterprise). A role the value omits loses its override. Overriding a role the instance does not define is refused. Edit operator rights in \`role_overrides.operator\`. | +| \`operator_settings\` | Legacy view of the \`operator\` role, kept for older servers: page visibility flags (\`runs\`, \`schedules\`, \`resources\`, \`variables\`, \`assets\`, \`audit_logs\`, \`groups\`, \`folders\`, \`workers\`, \`triggers\`) and write rights (\`manage_schedules\`, \`manage_triggers\`, \`builder_flows\`, \`builder_apps\`). Not applied when the document also states \`role_overrides\`. | | \`mute_critical_alerts\` | Mute the workspace's critical alerts. | | \`slack_team_id\`, \`slack_name\` | Read only: set by connecting Slack on the settings page. | | \`slack_command_script\` / \`teams_command_script\` | Script or flow handling the Slack / Teams command. | @@ -8536,7 +8537,7 @@ only show them, \`-y\` to skip the question, \`--replace\` to clear what the fil With \`includeSettings: true\` in \`wmill.yaml\`, \`wmill sync pull\` writes the document to \`settings.yaml\`, secrets encrypted with the workspace key (plaintext only with \`plainSecrets\`), and \`wmill sync push\` applies it. Unlike a partial apply, a push treats the file as the whole -document: a setting the file omits is cleared, except \`operator_settings\`, \`color\`, the auto-invite +document: a setting the file omits is cleared, except \`operator_settings\`, \`role_overrides\`, \`color\`, the auto-invite instance groups, and the keys \`ducklake\`, \`dbt_warehouses\`, \`deploy_ui\`, \`teams_command_script\`, \`public_app_execution_limit_per_minute\`, \`guest_jwt_public_key\`, \`guest_jwt_jwks_url\`, \`error_handler_fallback_to_instance_alerts\` and diff --git a/docs/workspace-roles.md b/docs/workspace-roles.md index 922710550d..8bae28309a 100644 --- a/docs/workspace-roles.md +++ b/docs/workspace-roles.md @@ -101,3 +101,25 @@ key equal to what the instance gives that role, and the payload is mirrored into workspace had changed from its default into `role_overrides.operator` the same way. The workspace list still returns an `operator_settings` object for a restricted member, synthesized from its role, alongside `role` and `capabilities`. + +## In the settings document + +`role_overrides` in the settings document (export `settings.json`, `settings.yaml`, git sync) is +the stored map, except that the entry of a role the instance does not define carries +`workspace_only: true`. The stored map cannot tell an override of instance role X from adding role +X, and the target of a promotion may lack X. So the apply refuses the whole document, dry run +included, before writing anything, if: + +- an unmarked entry names a role the target instance does not define, unless the target workspace + already has it as a workspace-only role (a file pulled before the marker existed), or +- a marked entry names a role the target instance defines. + +The marker is never stored. Roles are written before every other setting, so an instance group +mapping in the same file can name a role it adds. A workspace-only role the file removes is kept +until a final write, after whatever named it has been rewritten, so the in-use check passes. + +The document's `operator_settings` is derived from the effective `operator` role, not read from +the column: only the legacy endpoint rewrites the column, so after a role edit it holds stale +flags. A document that states `role_overrides` never applies `operator_settings`, and warns when +it was edited to disagree with the operator role. A document without `role_overrides` (pulled +before roles existed) applies it as before, diffed against the column. diff --git a/system_prompts/auto-generated/prompts.ts b/system_prompts/auto-generated/prompts.ts index 4f35733f6f..7141bd11b9 100644 --- a/system_prompts/auto-generated/prompts.ts +++ b/system_prompts/auto-generated/prompts.ts @@ -1622,7 +1622,8 @@ write a secret you were not given: keep the placeholder. | \`default_app\` | Path of the app shown as the workspace home. | | \`default_scripts\` | \`{ order?, hidden?, default_script_content? }\` for the script language picker. | | \`deploy_ui\` | \`{ include_path?, include_type? }\`: what the deploy-to-workspace UI offers. | -| \`operator_settings\` | What operators see and do: page visibility flags (\`runs\`, \`schedules\`, \`resources\`, \`variables\`, \`assets\`, \`audit_logs\`, \`groups\`, \`folders\`, \`workers\`, \`triggers\`) and write rights (\`manage_schedules\`, \`manage_triggers\`, \`builder_flows\`, \`builder_apps\`). A write right the value omits keeps its stored value. | +| \`role_overrides\` | \`{ : { label?, capabilities, workspace_only? } }\`: the workspace's overrides of the instance's roles, and with \`workspace_only: true\` a role of this workspace only (Enterprise). A role the value omits loses its override. Overriding a role the instance does not define is refused. Edit operator rights in \`role_overrides.operator\`. | +| \`operator_settings\` | Legacy view of the \`operator\` role, kept for older servers: page visibility flags (\`runs\`, \`schedules\`, \`resources\`, \`variables\`, \`assets\`, \`audit_logs\`, \`groups\`, \`folders\`, \`workers\`, \`triggers\`) and write rights (\`manage_schedules\`, \`manage_triggers\`, \`builder_flows\`, \`builder_apps\`). Not applied when the document also states \`role_overrides\`. | | \`mute_critical_alerts\` | Mute the workspace's critical alerts. | | \`slack_team_id\`, \`slack_name\` | Read only: set by connecting Slack on the settings page. | | \`slack_command_script\` / \`teams_command_script\` | Script or flow handling the Slack / Teams command. | diff --git a/system_prompts/auto-generated/skills/workspace-settings/SKILL.md b/system_prompts/auto-generated/skills/workspace-settings/SKILL.md index f739f95aa7..336910c27e 100644 --- a/system_prompts/auto-generated/skills/workspace-settings/SKILL.md +++ b/system_prompts/auto-generated/skills/workspace-settings/SKILL.md @@ -52,7 +52,8 @@ write a secret you were not given: keep the placeholder. | `default_app` | Path of the app shown as the workspace home. | | `default_scripts` | `{ order?, hidden?, default_script_content? }` for the script language picker. | | `deploy_ui` | `{ include_path?, include_type? }`: what the deploy-to-workspace UI offers. | -| `operator_settings` | What operators see and do: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). A write right the value omits keeps its stored value. | +| `role_overrides` | `{ : { label?, capabilities, workspace_only? } }`: the workspace's overrides of the instance's roles, and with `workspace_only: true` a role of this workspace only (Enterprise). A role the value omits loses its override. Overriding a role the instance does not define is refused. Edit operator rights in `role_overrides.operator`. | +| `operator_settings` | Legacy view of the `operator` role, kept for older servers: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). Not applied when the document also states `role_overrides`. | | `mute_critical_alerts` | Mute the workspace's critical alerts. | | `slack_team_id`, `slack_name` | Read only: set by connecting Slack on the settings page. | | `slack_command_script` / `teams_command_script` | Script or flow handling the Slack / Teams command. | @@ -74,7 +75,7 @@ only show them, `-y` to skip the question, `--replace` to clear what the file om With `includeSettings: true` in `wmill.yaml`, `wmill sync pull` writes the document to `settings.yaml`, secrets encrypted with the workspace key (plaintext only with `plainSecrets`), and `wmill sync push` applies it. Unlike a partial apply, a push treats the file as the whole -document: a setting the file omits is cleared, except `operator_settings`, `color`, the auto-invite +document: a setting the file omits is cleared, except `operator_settings`, `role_overrides`, `color`, the auto-invite instance groups, and the keys `ducklake`, `dbt_warehouses`, `deploy_ui`, `teams_command_script`, `public_app_execution_limit_per_minute`, `guest_jwt_public_key`, `guest_jwt_jwks_url`, `error_handler_fallback_to_instance_alerts` and diff --git a/system_prompts/base/workspace-settings.md b/system_prompts/base/workspace-settings.md index f5e4c7e7d1..29b2f20859 100644 --- a/system_prompts/base/workspace-settings.md +++ b/system_prompts/base/workspace-settings.md @@ -47,7 +47,8 @@ write a secret you were not given: keep the placeholder. | `default_app` | Path of the app shown as the workspace home. | | `default_scripts` | `{ order?, hidden?, default_script_content? }` for the script language picker. | | `deploy_ui` | `{ include_path?, include_type? }`: what the deploy-to-workspace UI offers. | -| `operator_settings` | What operators see and do: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). A write right the value omits keeps its stored value. | +| `role_overrides` | `{ : { label?, capabilities, workspace_only? } }`: the workspace's overrides of the instance's roles, and with `workspace_only: true` a role of this workspace only (Enterprise). A role the value omits loses its override. Overriding a role the instance does not define is refused. Edit operator rights in `role_overrides.operator`. | +| `operator_settings` | Legacy view of the `operator` role, kept for older servers: page visibility flags (`runs`, `schedules`, `resources`, `variables`, `assets`, `audit_logs`, `groups`, `folders`, `workers`, `triggers`) and write rights (`manage_schedules`, `manage_triggers`, `builder_flows`, `builder_apps`). Not applied when the document also states `role_overrides`. | | `mute_critical_alerts` | Mute the workspace's critical alerts. | | `slack_team_id`, `slack_name` | Read only: set by connecting Slack on the settings page. | | `slack_command_script` / `teams_command_script` | Script or flow handling the Slack / Teams command. | @@ -70,7 +71,7 @@ only show them, `-y` to skip the question, `--replace` to clear what the file om With `includeSettings: true` in `wmill.yaml`, `wmill sync pull` writes the document to `settings.yaml`, secrets encrypted with the workspace key (plaintext only with `plainSecrets`), and `wmill sync push` applies it. Unlike a partial apply, a push treats the file as the whole -document: a setting the file omits is cleared, except `operator_settings`, `color`, the auto-invite +document: a setting the file omits is cleared, except `operator_settings`, `role_overrides`, `color`, the auto-invite instance groups, and the keys `ducklake`, `dbt_warehouses`, `deploy_ui`, `teams_command_script`, `public_app_execution_limit_per_minute`, `guest_jwt_public_key`, `guest_jwt_jwks_url`, `error_handler_fallback_to_instance_alerts` and