feat: add Kubernetes operator and instance settings YAML editor (#7836)

* Add windmill-operator crate for Kubernetes CRD-based instance config Introduces a new `windmill-operator` crate that enables declarative management of Windmill instance configuration via a Kubernetes `WindmillInstance` CRD. The operator watches CRD resources and performs full declarative sync of global_settings and worker configs to the database, supporting GitOps workflows for instance-level configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add tests for windmill-operator CRD and db_sync - 9 unit tests for CRD serialization, deserialization, metadata, and status field behavior - 15 integration tests for db_sync using #[sqlx::test] with full declarative sync coverage: upsert, delete, protected keys, idempotency, worker config prefix handling, and end-to-end sync Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Replace untyped BTreeMap CRD fields with typed structs for schema validation GlobalSettings, SmtpSettings, IndexerSettings, and WorkerGroupConfig now have explicit typed fields with serde(flatten) catch-all for forward compatibility. The generated CRD YAML includes a full OpenAPI v3 schema that Kubernetes validates on kubectl apply. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Type opaque serde_json::Value CRD fields with real structs Replace most remaining serde_json::Value fields in WindmillInstance CRD with properly typed structs derived from the codebase: - oauths: BTreeMap<String, OAuthClient> - otel: OtelSettings - otel_tracing_proxy: OtelTracingProxySettings with ScriptLang enum - critical_error_channels: Vec<CriticalErrorChannel> (untagged enum) - critical_alerts_on_db_oversize: DbOversizeAlert - ducklake_settings: DucklakeSettings with nested catalog/storage types - custom_instance_pg_databases: CustomInstancePgDatabases - autoscaling (worker config): AutoscalingConfig with integration struct - custom_tags, default_tags_workspaces: Vec<String> - default_tags_per_workspace: bool Still opaque (serde_json::Value): object_store_cache_config (kube-core can't generate schemas for internally-tagged enums), secret_backend (EE-private), slack, teams (no clear struct definitions). Regenerated CRD YAML with full OpenAPI schema (352→703 lines). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Unify instance config types and add bulk GET/PUT API Move all typed settings (GlobalSettings, WorkerGroupConfig, etc.) from windmill-operator/crd.rs into windmill-common/instance_config.rs so both the API server and operator share a single source of truth. Add diff/apply logic (Merge mode for UI, Replace mode for operator) and InstanceConfig::from_db(). Add GET/PUT /settings/instance_config endpoints so the frontend loads all settings in 1 call instead of 42, and saves with a single bulk PUT. The backend handles the diff internally, running pre-write hooks for changed keys. Refactor windmill-operator/db_sync.rs to use the shared diff+apply functions and slim crd.rs down to the CRD wrapper with re-exports. Includes 32 unit tests and 30 integration tests covering serialization, diff logic, DB roundtrips, protected settings, and edge cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add Form/YAML toggle to instance settings UI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: show Form/YAML toggle regardless of hideTabs prop Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: replace toggle button group with simple YAML toggle Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: mask sensitive fields in YAML view with show/hide toggle Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: hide internal settings and mask sensitive fields in YAML view Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: hide jwt_secret and min_keep_alive_version from API and config exports Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * feat: add secretKeyRef support for sensitive fields in operator CRD Allow sensitive fields (license_key, hub_api_secret, scim_token, smtp_password, OAuthClient.secret, custom PG user_pwd) to reference Kubernetes Secrets via the standard secretKeyRef pattern instead of inlining values as plaintext YAML. The reconciler resolves all refs by reading K8s Secrets before syncing to the database. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * all * fix: merge main and update dev environment docs Resolve merge conflicts from origin/main, fix duplicate UV_INDEX_STRATEGY_SETTING import, and add Playwright MCP testing instructions to CLAUDE.md. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * fix: init tracing for CLI subcommands and deduplicate setting side-effects Initialize tracing subscriber before early-return CLI paths (sync-config, operator) so tracing calls are not silently dropped. Refactor set_global_setting_internal to call run_setting_pre_write_hook instead of duplicating the side-effect logic. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add `wmill instance get-config` CLI command Dumps the current instance config (global settings + worker configs) as YAML. Supports --output-file to write to a file instead of stdout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-08-24 16:01:42 +00:00 · 2026-02-14 23:42:01 +01:00
parent a9dbd1f73f
commit 82e5f6de48
79 changed files with 6340 additions and 393 deletions
@@ -3,6 +3,10 @@
    "svelte": {
      "type": "http",
      "url": "https://mcp.svelte.dev/mcp"
+    },
+    "playwright": {
+      "command": "npx",
+      "args": ["@playwright/mcp@latest"]
    }
  }
 }
@@ -20,6 +20,26 @@ When implementing new features in Windmill, follow these best practices:
 - Backend (Rust): see `backend/CLAUDE.md` and the `rust-backend` skill: `.claude/skills/rust-backend/SKILL.md`
 - Frontend (Svelte 5): see `frontend/CLAUDE.md` and the `svelte-frontend` skill: `.claude/skills/svelte-frontend/SKILL.md`

+## Dev Environment
+
+- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
+- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/`
+  - The `REMOTE` env var configures the Vite proxy target. Without it, API calls proxy to `https://app.windmill.dev` instead of the local backend.
+  - The dev server starts on port 3000 (or 3001+ if 3000 is in use).
+- **Default login**: `admin@windmill.dev` / `changeme`
+- **Instance settings**: navigate to `/#superadmin-settings` (opens the drawer overlay)
+
+## UI Testing with Playwright MCP
+
+When testing the frontend with the Playwright MCP tools:
+
+1. **Start servers**: Launch backend (`cargo run`) and frontend (`REMOTE=http://localhost:8000 npm run dev`) as background tasks
+2. **Wait for readiness**: Backend takes ~60s to compile; check output for `health check completed`. Frontend starts in ~5s.
+3. **Login flow**: Navigate to `/user/login`, click "Log in without third-party", fill email/password, submit
+4. **Instance settings drawer**: Navigate to `/#superadmin-settings` to open the drawer directly
+5. **Toggle components**: The YAML toggle uses a custom `<Toggle>` component where the checkbox is visually hidden (`sr-only`). Click the wrapper `<label>` element (the parent container with `cursor=pointer`), not the checkbox ref directly.
+6. **Console errors to ignore**: `critical_alerts` 404s are expected on CE builds (EE-only endpoint). VSCode worker 404s are dev-mode artifacts.
+
 ## Code Validation (MUST DO)

 After making code changes, you MUST run the appropriate checks and fix all errors before considering the work done:
@@ -46,11 +46,11 @@
      ]
    },
    "nullable": [
-      true,
-      true,
-      true,
-      true,
-      true,
+      false,
+      false,
+      false,
+      false,
+      false,
      true,
      true
    ]
@@ -30,8 +30,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -1,26 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "\n                        SELECT\n                            path,\n                            custom_path\n                        FROM \n                            app\n                        WHERE \n                            custom_path IN (\n                                SELECT \n                                    custom_path\n                                FROM \n                                    app\n                                GROUP \n                                    BY custom_path\n                                HAVING COUNT(*) > 1\n                            )\n                        ORDER BY custom_path\n                        ",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "path",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "custom_path",
-        "type_info": "Text"
-      }
-    ],
-    "parameters": {
-      "Left": []
-    },
-    "nullable": [
-      false,
-      true
-    ]
-  },
-  "hash": "11e24f758a70cd5f3a240bc81a05f40754826db0ee1194409227597a98603e92"
-}
@@ -122,8 +122,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -40,8 +40,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -34,8 +34,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -68,8 +67,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -40,8 +40,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -0,0 +1,26 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "\n                        SELECT\n                            path,\n                            custom_path\n                        FROM\n                            app\n                        WHERE\n                            custom_path IN (\n                                SELECT\n                                    custom_path\n                                FROM\n                                    app\n                                GROUP\n                                    BY custom_path\n                                HAVING COUNT(*) > 1\n                            )\n                        ORDER BY custom_path\n                        ",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 1,
+        "name": "custom_path",
+        "type_info": "Text"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      false,
+      true
+    ]
+  },
+  "hash": "26b35cf50959b1b1fd7e1cb33c65da40d29e20fd16b02355ba073f420c03a767"
+}
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -30,8 +30,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -37,8 +37,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -32,8 +32,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -71,8 +70,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -15,7 +15,7 @@
      ]
    },
    "nullable": [
-      true
+      null
    ]
  },
  "hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"
@@ -245,8 +245,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -35,8 +35,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -29,8 +29,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -40,8 +40,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -27,8 +27,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -35,8 +35,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -32,8 +32,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -30,8 +30,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -155,8 +155,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -1,20 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT EXISTS(SELECT 1 FROM app WHERE path = 'g/all/setup_app')",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "exists",
-        "type_info": "Bool"
-      }
-    ],
-    "parameters": {
-      "Left": []
-    },
-    "nullable": [
-      null
-    ]
-  },
-  "hash": "a264bbd8dbabb03854bd25350a7aeda0704770eb200bae635f1933eece90c9d6"
-}
@@ -185,8 +185,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -160,8 +160,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -105,8 +105,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -31,8 +31,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -105,8 +105,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -25,8 +25,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -185,8 +185,7 @@
                "sqs",
                "gcp",
                "mqtt",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -31,8 +31,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -24,8 +24,7 @@
                "mqtt",
                "gcp",
                "default_email",
-                "nextcloud",
-                "google"
+                "nextcloud"
              ]
            }
          }
@@ -15766,6 +15766,7 @@ dependencies = [
 "tokio",
 "tokio-stream",
 "tracing",
+ "tracing-subscriber",
 "url",
 "uuid",
 "windmill-api",
@@ -15777,6 +15778,7 @@ dependencies = [
 "windmill-dep-map",
 "windmill-git-sync",
 "windmill-indexer",
+ "windmill-operator",
 "windmill-queue",
 "windmill-runtime-nativets",
 "windmill-test-utils",
@@ -16476,6 +16478,7 @@ dependencies = [
 "reqwest-middleware",
 "reqwest-retry",
 "rsa",
+ "schemars 0.8.22",
 "semver 1.0.27",
 "serde",
 "serde_json",
@@ -16678,6 +16681,26 @@ dependencies = [
 "windmill-common",
 ]

+[[package]]
+name = "windmill-operator"
+version = "1.634.6"
+dependencies = [
+ "anyhow",
+ "chrono",
+ "futures",
+ "k8s-openapi",
+ "kube",
+ "schemars 0.8.22",
+ "serde",
+ "serde_json",
+ "serde_yml",
+ "sqlx",
+ "thiserror 2.0.18",
+ "tokio",
+ "tracing",
+ "windmill-common",
+]
+
 [[package]]
 name = "windmill-parser"
 version = "1.634.6"
@@ -52,6 +52,7 @@ members = [
    "./windmill-audit",
    "./windmill-git-sync",
    "./windmill-autoscaling",
+    "./windmill-operator",
    "./windmill-indexer",
    "./windmill-macros",
    "./windmill-oauth",
@@ -135,6 +136,8 @@ zip = ["windmill-api/zip"]
 static_frontend = ["windmill-api/static_frontend"]
 scoped_cache = ["windmill-common/scoped_cache"]
 no_auth = ["windmill-api/no_auth"]
+operator = ["dep:windmill-operator"]
+test_job_debouncing = []
 private_registry_test = []
 # Languages
 python = ["windmill-worker/python", "windmill-api/python", "windmill-test-utils/python"]
@@ -167,7 +170,7 @@ ce_core = ["oss_core", "private"]
 ee_core = [
    "enterprise", "stripe", "prometheus", "cloud",
    "kafka", "sqs_trigger", "nats", "gcp_trigger",
-    "jemalloc", "otel"
+    "jemalloc", "otel", "operator"
 ]
 ee_server = ["enterprise_saml", "tantivy", "agent_worker_server", "local_reports"]
 # Edition meta-features: CE variants
@@ -203,8 +206,10 @@ windmill-api-settings.workspace = true
 windmill-worker.workspace = true
 windmill-indexer = { workspace = true, optional = true }
 windmill-autoscaling = { workspace = true, optional = true }
+windmill-operator = { workspace = true, optional = true }
 futures.workspace = true
 tracing.workspace = true
+tracing-subscriber.workspace = true
 sqlx.workspace = true
 sql-builder.workspace = true
 rand.workspace = true
@@ -266,6 +271,7 @@ windmill-common = { path = "./windmill-common", default-features = false }
 windmill-audit = { path = "./windmill-audit" }
 windmill-git-sync = { path = "./windmill-git-sync" }
 windmill-autoscaling = { path = "./windmill-autoscaling" }
+windmill-operator = { path = "./windmill-operator" }
 windmill-indexer = {path = "./windmill-indexer"}
 windmill-mcp = {path = "./windmill-mcp"}
 windmill-oauth = {path = "./windmill-oauth"}
@@ -558,6 +564,7 @@ backon = "1.3.0"

 flume = { version = "0.11.1", features = ["async"] }
 kube = { version = "1.1.0", features = ["runtime", "derive"] }
+schemars = "0.8"
 k8s-openapi = { version = "0.25.0", features = ["latest"] }
 libloading = "0.8.8"

@@ -10,10 +10,11 @@ use monitor::{
    load_base_url, load_otel, reload_critical_alerts_on_db_oversize,
    reload_delete_logs_periodically_setting, reload_indexer_config,
    reload_instance_python_version_setting, reload_maven_repos_setting,
-    reload_maven_settings_xml_setting, reload_no_default_maven_setting, reload_nuget_config_setting,
-    reload_powershell_repo_pat_setting, reload_powershell_repo_url_setting,
-    reload_ruby_repos_setting, reload_timeout_wait_result_setting,
-    send_current_log_file_to_object_store, send_logs_to_object_store, WORKERS_NAMES,
+    reload_maven_settings_xml_setting, reload_no_default_maven_setting,
+    reload_nuget_config_setting, reload_powershell_repo_pat_setting,
+    reload_powershell_repo_url_setting, reload_ruby_repos_setting,
+    reload_timeout_wait_result_setting, send_current_log_file_to_object_store,
+    send_logs_to_object_store, WORKERS_NAMES,
 };
 use rand::Rng;
 use sqlx::{Pool, Postgres};
@@ -41,16 +42,16 @@ use windmill_common::{
        CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
        DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
        EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
-        JOB_ISOLATION_SETTING, HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
-        INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JWT_SECRET_SETTING,
-        KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING,
+        HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
+        INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING,
+        JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING,
        MAVEN_SETTINGS_XML_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
        NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OAUTH_SETTING, OTEL_SETTING,
-        OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, UV_INDEX_STRATEGY_SETTING,
+        OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING,
        POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
        REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
        RUBY_REPOS_SETTING, SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, SMTP_SETTING, TEAMS_SETTING,
-        TIMEOUT_WAIT_RESULT_SETTING,
+        TIMEOUT_WAIT_RESULT_SETTING, UV_INDEX_STRATEGY_SETTING,
    },
    scripts::ScriptLang,
    stats_oss::schedule_stats,
@@ -99,12 +100,11 @@ use crate::monitor::{
    reload_app_workspaced_route_setting, reload_base_url_setting,
    reload_bunfig_install_scopes_setting, reload_critical_alert_mute_ui_setting,
    reload_critical_error_channels_setting, reload_extra_pip_index_url_setting,
-    reload_job_isolation_setting, reload_hub_api_secret_setting, reload_hub_base_url_setting,
-    reload_job_default_timeout_setting, reload_jwt_secret_setting, reload_license_key,
-    reload_npm_config_registry_setting,
-    reload_otel_tracing_proxy_setting, reload_pip_index_url_setting,
-    reload_retention_period_setting, reload_scim_token_setting, reload_smtp_config,
-    reload_uv_index_strategy_setting, reload_worker_config, MonitorIteration,
+    reload_hub_api_secret_setting, reload_hub_base_url_setting, reload_job_default_timeout_setting,
+    reload_job_isolation_setting, reload_jwt_secret_setting, reload_license_key,
+    reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
+    reload_pip_index_url_setting, reload_retention_period_setting, reload_scim_token_setting,
+    reload_smtp_config, reload_uv_index_strategy_setting, reload_worker_config, MonitorIteration,
 };

 #[cfg(feature = "parquet")]
@@ -484,6 +484,9 @@ fn print_help() {
    println!("  version              Show Windmill version and exit");
    println!("  cache [hubPaths.json]  Pre-cache hub scripts (default: ./hubPaths.json)");
    println!("  cache-rt             Pre-cache hub resource types");
+    println!("  sync-config <file>   Sync instance config from a YAML file to the database");
+    println!("  operator             Run the Kubernetes operator (watches WindmillInstance CRDs)");
+    println!("  operator crd         Print the WindmillInstance CRD YAML to stdout");
    println!();
    println!("Environment variables (name = default):");
    println!("  DATABASE_URL = <required>              The Postgres database url.");
@@ -607,6 +610,43 @@ async fn windmill_main() -> anyhow::Result<()> {
            cache_hub_resource_types().await?;
            return Ok(());
        }
+        "sync-config" => {
+            tracing_subscriber::fmt::init();
+            let path = std::env::args().nth(2).unwrap_or_else(|| {
+                eprintln!("Usage: windmill sync-config <file>");
+                std::process::exit(1);
+            });
+            let contents = tokio::fs::read_to_string(&path)
+                .await
+                .with_context(|| format!("Could not read config file: {path}"))?;
+            let mut config: windmill_common::instance_config::InstanceConfig =
+                serde_yml::from_str(&contents)
+                    .with_context(|| format!("Could not parse YAML from: {path}"))?;
+            windmill_common::instance_config::resolve_env_refs(&mut config.global_settings)
+                .map_err(|var| anyhow::anyhow!("environment variable '{var}' not found"))?;
+
+            tracing::info!("Connecting to database...");
+            let db = crate::db_connect::initial_connection().await?;
+            config.sync_to_db(&db).await?;
+            tracing::info!("Synced instance config from {path}");
+            return Ok(());
+        }
+        #[cfg(feature = "operator")]
+        "operator" => {
+            let sub_arg = std::env::args().nth(2).unwrap_or_default();
+            if sub_arg == "crd" {
+                windmill_operator::print_crd_yaml();
+                return Ok(());
+            }
+
+            tracing_subscriber::fmt::init();
+            tracing::info!("Starting Windmill Kubernetes operator...");
+            tracing::info!("Connecting to database...");
+            let db = crate::db_connect::initial_connection().await?;
+            tracing::info!("Database connected. Starting controller...");
+            windmill_operator::run(db).await?;
+            return Ok(());
+        }
        _ => {}
    }

@@ -90,9 +90,8 @@ use windmill_worker::{
    OtelTracingProxySettings, SameWorkerSender, BUNFIG_INSTALL_SCOPES, CARGO_REGISTRIES,
    INSTANCE_PYTHON_VERSION, JAVA_HOME_DIR, JOB_DEFAULT_TIMEOUT, JOB_ISOLATION, KEEP_JOB_DIR,
    MAVEN_REPOS, MAVEN_SETTINGS_XML, NO_DEFAULT_MAVEN, NPM_CONFIG_REGISTRY, NSJAIL_AVAILABLE,
-    NUGET_CONFIG,
-    OTEL_TRACING_PROXY_SETTINGS, PIP_EXTRA_INDEX_URL, PIP_INDEX_URL, POWERSHELL_REPO_PAT,
-    POWERSHELL_REPO_URL, UV_INDEX_STRATEGY,
+    NUGET_CONFIG, OTEL_TRACING_PROXY_SETTINGS, PIP_EXTRA_INDEX_URL, PIP_INDEX_URL,
+    POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, UV_INDEX_STRATEGY,
 };

 #[cfg(feature = "parquet")]
@@ -0,0 +1,994 @@
+/*!
+ * Integration tests for windmill-common instance_config module.
+ *
+ * Tests verify the DB-level operations:
+ * - `InstanceConfig::from_db()` reads global_settings + worker configs
+ * - `apply_settings_diff()` applies upserts and deletes to global_settings
+ * - `apply_configs_diff()` applies upserts and deletes to config table
+ * - Full roundtrip: write → read → modify → diff → apply → read → verify
+ *
+ * Note: the test DB is created from migrations which seed default settings
+ * and worker configs. Tests either clean up first or assert on specific keys
+ * rather than exact counts.
+ */
+
+use std::collections::BTreeMap;
+
+use sqlx::{Pool, Postgres};
+use windmill_common::instance_config::{
+    apply_configs_diff, apply_settings_diff, diff_global_settings, diff_worker_configs, ApplyMode,
+    ConfigsDiff, InstanceConfig, SettingsDiff,
+};
+
+// ========================================================================
+// Helpers
+// ========================================================================
+
+async fn get_global_setting(db: &Pool<Postgres>, name: &str) -> Option<serde_json::Value> {
+    sqlx::query_as::<_, (serde_json::Value,)>("SELECT value FROM global_settings WHERE name = $1")
+        .bind(name)
+        .fetch_optional(db)
+        .await
+        .expect("query should succeed")
+        .map(|(v,)| v)
+}
+
+async fn get_config(db: &Pool<Postgres>, name: &str) -> Option<serde_json::Value> {
+    sqlx::query_as::<_, (serde_json::Value,)>("SELECT config FROM config WHERE name = $1")
+        .bind(name)
+        .fetch_optional(db)
+        .await
+        .expect("query should succeed")
+        .map(|(v,)| v)
+}
+
+async fn insert_global_setting(db: &Pool<Postgres>, name: &str, value: serde_json::Value) {
+    sqlx::query(
+        "INSERT INTO global_settings (name, value) VALUES ($1, $2) \
+         ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
+    )
+    .bind(name)
+    .bind(&value)
+    .execute(db)
+    .await
+    .expect("insert should succeed");
+}
+
+async fn insert_config(db: &Pool<Postgres>, name: &str, config: serde_json::Value) {
+    sqlx::query(
+        "INSERT INTO config (name, config) VALUES ($1, $2) \
+         ON CONFLICT (name) DO UPDATE SET config = EXCLUDED.config",
+    )
+    .bind(name)
+    .bind(&config)
+    .execute(db)
+    .await
+    .expect("insert should succeed");
+}
+
+async fn count_global_settings(db: &Pool<Postgres>) -> i64 {
+    sqlx::query_as::<_, (i64,)>("SELECT COUNT(*) FROM global_settings")
+        .fetch_one(db)
+        .await
+        .expect("count query should succeed")
+        .0
+}
+
+/// Clear all migration-seeded data so tests start from a clean slate.
+async fn clear_settings_and_configs(db: &Pool<Postgres>) {
+    sqlx::query("DELETE FROM global_settings")
+        .execute(db)
+        .await
+        .expect("clear global_settings should succeed");
+    sqlx::query("DELETE FROM config WHERE name LIKE 'worker__%'")
+        .execute(db)
+        .await
+        .expect("clear worker configs should succeed");
+}
+
+// ========================================================================
+// InstanceConfig::from_db() tests
+// ========================================================================
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_empty(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    let config = InstanceConfig::from_db(&db)
+        .await
+        .expect("from_db should succeed on empty DB");
+    assert!(config.global_settings.base_url.is_none());
+    assert!(config.global_settings.license_key.is_none());
+    assert!(config.worker_configs.is_empty());
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_with_typed_global_settings(db: Pool<Postgres>) {
+    insert_global_setting(&db, "base_url", serde_json::json!("https://windmill.test")).await;
+    insert_global_setting(&db, "retention_period_secs", serde_json::json!(86400)).await;
+    insert_global_setting(&db, "expose_metrics", serde_json::json!(true)).await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(
+        config.global_settings.base_url.as_deref(),
+        Some("https://windmill.test")
+    );
+    assert_eq!(config.global_settings.retention_period_secs, Some(86400));
+    assert_eq!(config.global_settings.expose_metrics, Some(true));
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_with_structured_settings(db: Pool<Postgres>) {
+    insert_global_setting(
+        &db,
+        "smtp_settings",
+        serde_json::json!({
+            "smtp_host": "mail.test.com",
+            "smtp_port": 587,
+            "smtp_tls_implicit": false
+        }),
+    )
+    .await;
+    insert_global_setting(
+        &db,
+        "otel",
+        serde_json::json!({
+            "metrics_enabled": true,
+            "logs_enabled": false,
+            "otel_exporter_otlp_endpoint": "http://otel:4317"
+        }),
+    )
+    .await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+
+    let smtp = config.global_settings.smtp_settings.as_ref().unwrap();
+    assert_eq!(smtp.smtp_host.as_deref(), Some("mail.test.com"));
+    assert_eq!(smtp.smtp_port, Some(587));
+    assert_eq!(smtp.smtp_tls_implicit, Some(false));
+
+    let otel = config.global_settings.otel.as_ref().unwrap();
+    assert_eq!(otel.metrics_enabled, Some(true));
+    assert_eq!(otel.logs_enabled, Some(false));
+    assert_eq!(
+        otel.otel_exporter_otlp_endpoint.as_deref(),
+        Some("http://otel:4317")
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_unknown_settings_go_to_extra(db: Pool<Postgres>) {
+    insert_global_setting(&db, "future_setting_xyz", serde_json::json!({"nested": 42})).await;
+    insert_global_setting(&db, "another_unknown", serde_json::json!("hello")).await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(
+        config.global_settings.extra["future_setting_xyz"],
+        serde_json::json!({"nested": 42})
+    );
+    assert_eq!(
+        config.global_settings.extra["another_unknown"],
+        serde_json::json!("hello")
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_with_worker_configs(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    insert_config(
+        &db,
+        "worker__default",
+        serde_json::json!({"init_bash": "echo default"}),
+    )
+    .await;
+    insert_config(
+        &db,
+        "worker__gpu",
+        serde_json::json!({
+            "dedicated_worker": "ws:f/gpu_script",
+            "worker_tags": ["gpu", "cuda"]
+        }),
+    )
+    .await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(config.worker_configs.len(), 2);
+    assert_eq!(
+        config.worker_configs["default"].init_bash.as_deref(),
+        Some("echo default")
+    );
+    assert_eq!(
+        config.worker_configs["gpu"].dedicated_worker.as_deref(),
+        Some("ws:f/gpu_script")
+    );
+    assert_eq!(
+        config.worker_configs["gpu"].worker_tags.as_ref().unwrap(),
+        &["gpu", "cuda"]
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_ignores_non_worker_configs(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Insert a config without worker__ prefix — should not appear
+    insert_config(&db, "server_config", serde_json::json!({"important": true})).await;
+    insert_config(
+        &db,
+        "worker__actual",
+        serde_json::json!({"init_bash": "echo hi"}),
+    )
+    .await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(config.worker_configs.len(), 1);
+    assert!(config.worker_configs.contains_key("actual"));
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_worker_config_prefix_stripping(db: Pool<Postgres>) {
+    insert_config(
+        &db,
+        "worker__my_group_name",
+        serde_json::json!({"cache_clear": 5}),
+    )
+    .await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert!(
+        config.worker_configs.contains_key("my_group_name"),
+        "worker__ prefix should be stripped"
+    );
+    assert_eq!(config.worker_configs["my_group_name"].cache_clear, Some(5));
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_worker_config_unknown_fields_in_extra(db: Pool<Postgres>) {
+    insert_config(
+        &db,
+        "worker__test_extra",
+        serde_json::json!({
+            "init_bash": "echo hello",
+            "future_field": 999,
+            "another_future": {"nested": true}
+        }),
+    )
+    .await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    let wc = &config.worker_configs["test_extra"];
+    assert_eq!(wc.init_bash.as_deref(), Some("echo hello"));
+    assert_eq!(wc.extra["future_field"], serde_json::json!(999));
+    assert_eq!(
+        wc.extra["another_future"],
+        serde_json::json!({"nested": true})
+    );
+}
+
+// ========================================================================
+// apply_settings_diff() tests
+// ========================================================================
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_upserts_only(db: Pool<Postgres>) {
+    let diff = SettingsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert("key_a".to_string(), serde_json::json!("val_a"));
+            m.insert("key_b".to_string(), serde_json::json!(123));
+            m
+        },
+        deletes: vec![],
+    };
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert_eq!(
+        get_global_setting(&db, "key_a").await,
+        Some(serde_json::json!("val_a"))
+    );
+    assert_eq!(
+        get_global_setting(&db, "key_b").await,
+        Some(serde_json::json!(123))
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_deletes_only(db: Pool<Postgres>) {
+    insert_global_setting(&db, "to_delete_1", serde_json::json!("bye")).await;
+    insert_global_setting(&db, "to_delete_2", serde_json::json!("gone")).await;
+    insert_global_setting(&db, "to_keep", serde_json::json!("stay")).await;
+
+    let diff = SettingsDiff {
+        upserts: BTreeMap::new(),
+        deletes: vec!["to_delete_1".to_string(), "to_delete_2".to_string()],
+    };
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert!(get_global_setting(&db, "to_delete_1").await.is_none());
+    assert!(get_global_setting(&db, "to_delete_2").await.is_none());
+    assert!(get_global_setting(&db, "to_keep").await.is_some());
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_upserts_and_deletes(db: Pool<Postgres>) {
+    insert_global_setting(&db, "old_key", serde_json::json!("old")).await;
+
+    let diff = SettingsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert("new_key".to_string(), serde_json::json!("new"));
+            m
+        },
+        deletes: vec!["old_key".to_string()],
+    };
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert!(get_global_setting(&db, "old_key").await.is_none());
+    assert_eq!(
+        get_global_setting(&db, "new_key").await,
+        Some(serde_json::json!("new"))
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_empty_noop(db: Pool<Postgres>) {
+    insert_global_setting(&db, "preexisting", serde_json::json!("value")).await;
+
+    let diff = SettingsDiff::default();
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert_eq!(
+        get_global_setting(&db, "preexisting").await,
+        Some(serde_json::json!("value"))
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_upsert_overwrites(db: Pool<Postgres>) {
+    insert_global_setting(&db, "overwrite_me", serde_json::json!("old_value")).await;
+
+    let diff = SettingsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert("overwrite_me".to_string(), serde_json::json!("new_value"));
+            m
+        },
+        deletes: vec![],
+    };
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert_eq!(
+        get_global_setting(&db, "overwrite_me").await,
+        Some(serde_json::json!("new_value"))
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_complex_json(db: Pool<Postgres>) {
+    let complex_value = serde_json::json!({
+        "host": "smtp.example.com",
+        "port": 587,
+        "auth": {"user": "admin", "pass": "secret"},
+        "tags": [1, 2, 3],
+        "nested": {"deep": {"value": null}}
+    });
+
+    let diff = SettingsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert("complex_setting".to_string(), complex_value.clone());
+            m
+        },
+        deletes: vec![],
+    };
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    let stored = get_global_setting(&db, "complex_setting").await.unwrap();
+    assert_eq!(stored, complex_value);
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_settings_diff_delete_nonexistent_is_noop(db: Pool<Postgres>) {
+    let diff =
+        SettingsDiff { upserts: BTreeMap::new(), deletes: vec!["does_not_exist".to_string()] };
+
+    // Should not error
+    apply_settings_diff(&db, &diff).await.unwrap();
+}
+
+// ========================================================================
+// apply_configs_diff() tests
+// ========================================================================
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_configs_diff_upserts_with_prefix(db: Pool<Postgres>) {
+    let diff = ConfigsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert(
+                "mygroup".to_string(),
+                serde_json::json!({"init_bash": "echo hi"}),
+            );
+            m
+        },
+        deletes: vec![],
+    };
+
+    apply_configs_diff(&db, &diff).await.unwrap();
+
+    let stored = get_config(&db, "worker__mygroup").await.unwrap();
+    assert_eq!(stored["init_bash"], "echo hi");
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_configs_diff_deletes_with_prefix(db: Pool<Postgres>) {
+    insert_config(&db, "worker__to_remove", serde_json::json!({"a": 1})).await;
+
+    let diff = ConfigsDiff { upserts: BTreeMap::new(), deletes: vec!["to_remove".to_string()] };
+
+    apply_configs_diff(&db, &diff).await.unwrap();
+
+    assert!(get_config(&db, "worker__to_remove").await.is_none());
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_configs_diff_upsert_overwrites(db: Pool<Postgres>) {
+    insert_config(&db, "worker__grp", serde_json::json!({"old": true})).await;
+
+    let diff = ConfigsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert("grp".to_string(), serde_json::json!({"new": true}));
+            m
+        },
+        deletes: vec![],
+    };
+
+    apply_configs_diff(&db, &diff).await.unwrap();
+
+    let stored = get_config(&db, "worker__grp").await.unwrap();
+    assert_eq!(stored, serde_json::json!({"new": true}));
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_configs_diff_empty_noop(db: Pool<Postgres>) {
+    insert_config(&db, "worker__keep", serde_json::json!({"keep": true})).await;
+
+    let diff = ConfigsDiff::default();
+    apply_configs_diff(&db, &diff).await.unwrap();
+
+    assert!(get_config(&db, "worker__keep").await.is_some());
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_apply_configs_diff_does_not_touch_non_worker_configs(db: Pool<Postgres>) {
+    insert_config(&db, "server_config", serde_json::json!({"x": 1})).await;
+
+    let diff = ConfigsDiff { upserts: BTreeMap::new(), deletes: vec!["server_config".to_string()] };
+
+    apply_configs_diff(&db, &diff).await.unwrap();
+
+    // The delete targets "worker__server_config", not "server_config"
+    assert!(
+        get_config(&db, "server_config").await.is_some(),
+        "Non-worker config should not be affected"
+    );
+}
+
+// ========================================================================
+// Full roundtrip tests
+// ========================================================================
+
+#[sqlx::test(fixtures("base"))]
+async fn test_roundtrip_write_read_modify_apply(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Step 1: Seed initial state
+    insert_global_setting(&db, "base_url", serde_json::json!("https://v1.test")).await;
+    insert_global_setting(&db, "retention_period_secs", serde_json::json!(3600)).await;
+    insert_config(
+        &db,
+        "worker__default",
+        serde_json::json!({"init_bash": "echo v1"}),
+    )
+    .await;
+
+    // Step 2: Read via from_db
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(
+        config.global_settings.base_url.as_deref(),
+        Some("https://v1.test")
+    );
+    assert_eq!(config.global_settings.retention_period_secs, Some(3600));
+    assert_eq!(config.worker_configs.len(), 1);
+
+    // Step 3: Modify — change base_url, add a new setting, remove retention
+    let mut desired_settings = config.global_settings.clone();
+    desired_settings.base_url = Some("https://v2.test".to_string());
+    desired_settings.expose_metrics = Some(true);
+    desired_settings.retention_period_secs = None;
+
+    let current_map = config.global_settings.to_settings_map();
+    let desired_map = desired_settings.to_settings_map();
+
+    // Step 4: Diff + apply (Merge mode — no deletes)
+    let diff = diff_global_settings(&current_map, &desired_map, ApplyMode::Merge);
+    assert!(diff.upserts.contains_key("base_url"));
+    assert!(diff.upserts.contains_key("expose_metrics"));
+    assert!(diff.deletes.is_empty());
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    // Step 5: Read back and verify
+    let config2 = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(
+        config2.global_settings.base_url.as_deref(),
+        Some("https://v2.test")
+    );
+    assert_eq!(config2.global_settings.expose_metrics, Some(true));
+    // retention_period_secs still in DB because Merge mode doesn't delete
+    assert_eq!(config2.global_settings.retention_period_secs, Some(3600));
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_roundtrip_replace_mode_deletes(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Seed
+    insert_global_setting(&db, "base_url", serde_json::json!("https://old.test")).await;
+    insert_global_setting(&db, "retention_period_secs", serde_json::json!(7200)).await;
+    insert_global_setting(&db, "expose_metrics", serde_json::json!(false)).await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    let current_map = config.global_settings.to_settings_map();
+
+    // Desired: only base_url — retention and expose_metrics should be deleted
+    let mut desired_map = BTreeMap::new();
+    desired_map.insert(
+        "base_url".to_string(),
+        serde_json::json!("https://old.test"),
+    );
+
+    let diff = diff_global_settings(&current_map, &desired_map, ApplyMode::Replace);
+    assert!(diff.upserts.is_empty());
+    assert!(diff.deletes.contains(&"retention_period_secs".to_string()));
+    assert!(diff.deletes.contains(&"expose_metrics".to_string()));
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert!(get_global_setting(&db, "retention_period_secs")
+        .await
+        .is_none());
+    assert!(get_global_setting(&db, "expose_metrics").await.is_none());
+    assert!(get_global_setting(&db, "base_url").await.is_some());
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_roundtrip_worker_configs(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Seed two worker configs
+    insert_config(
+        &db,
+        "worker__default",
+        serde_json::json!({"init_bash": "echo default"}),
+    )
+    .await;
+    insert_config(
+        &db,
+        "worker__legacy",
+        serde_json::json!({"init_bash": "echo legacy"}),
+    )
+    .await;
+
+    // Read
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(config.worker_configs.len(), 2);
+
+    // Desired: replace default, add gpu, remove legacy
+    let current_map: BTreeMap<String, serde_json::Value> = config
+        .worker_configs
+        .iter()
+        .map(|(k, v)| (k.clone(), serde_json::to_value(v).unwrap()))
+        .collect();
+
+    let mut desired_map = BTreeMap::new();
+    desired_map.insert(
+        "default".to_string(),
+        serde_json::json!({"init_bash": "echo default v2"}),
+    );
+    desired_map.insert(
+        "gpu".to_string(),
+        serde_json::json!({"dedicated_worker": "ws:f/gpu"}),
+    );
+
+    let diff = diff_worker_configs(&current_map, &desired_map, ApplyMode::Replace);
+    assert!(diff.upserts.contains_key("default")); // changed
+    assert!(diff.upserts.contains_key("gpu")); // new
+    assert_eq!(diff.deletes, vec!["legacy".to_string()]);
+
+    apply_configs_diff(&db, &diff).await.unwrap();
+
+    // Verify
+    let config2 = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(config2.worker_configs.len(), 2);
+    assert_eq!(
+        config2.worker_configs["default"].init_bash.as_deref(),
+        Some("echo default v2")
+    );
+    assert_eq!(
+        config2.worker_configs["gpu"].dedicated_worker.as_deref(),
+        Some("ws:f/gpu")
+    );
+    assert!(!config2.worker_configs.contains_key("legacy"));
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_roundtrip_to_settings_map_from_db_consistency(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Write a GlobalSettings via to_settings_map + apply, then read back via from_db
+    let original = windmill_common::instance_config::GlobalSettings {
+        base_url: Some("https://roundtrip.test".to_string()),
+        retention_period_secs: Some(43200),
+        expose_metrics: Some(true),
+        smtp_settings: Some(windmill_common::instance_config::SmtpSettings {
+            smtp_host: Some("smtp.roundtrip.test".to_string()),
+            smtp_port: Some(465),
+            ..Default::default()
+        }),
+        custom_tags: Some(vec!["tag1".to_string(), "tag2".to_string()]),
+        ..Default::default()
+    };
+
+    let map = original.to_settings_map();
+    let diff = SettingsDiff { upserts: map.into_iter().collect(), deletes: vec![] };
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    assert_eq!(config.global_settings.base_url, original.base_url);
+    assert_eq!(
+        config.global_settings.retention_period_secs,
+        original.retention_period_secs
+    );
+    assert_eq!(
+        config.global_settings.expose_metrics,
+        original.expose_metrics
+    );
+    assert_eq!(
+        config
+            .global_settings
+            .smtp_settings
+            .as_ref()
+            .unwrap()
+            .smtp_host,
+        original.smtp_settings.as_ref().unwrap().smtp_host
+    );
+    assert_eq!(
+        config
+            .global_settings
+            .smtp_settings
+            .as_ref()
+            .unwrap()
+            .smtp_port,
+        original.smtp_settings.as_ref().unwrap().smtp_port
+    );
+    assert_eq!(config.global_settings.custom_tags, original.custom_tags);
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_idempotent_apply(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    let diff = SettingsDiff {
+        upserts: {
+            let mut m = BTreeMap::new();
+            m.insert("idem_key".to_string(), serde_json::json!("idem_value"));
+            m
+        },
+        deletes: vec![],
+    };
+
+    // Apply twice
+    apply_settings_diff(&db, &diff).await.unwrap();
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    assert_eq!(
+        get_global_setting(&db, "idem_key").await,
+        Some(serde_json::json!("idem_value"))
+    );
+    assert_eq!(count_global_settings(&db).await, 1);
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_mixed_typed_and_extra(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Insert both typed and untyped settings
+    insert_global_setting(&db, "base_url", serde_json::json!("https://mixed.test")).await;
+    insert_global_setting(&db, "expose_metrics", serde_json::json!(true)).await;
+    insert_global_setting(
+        &db,
+        "unknown_future_setting",
+        serde_json::json!({"key": "val"}),
+    )
+    .await;
+    insert_global_setting(&db, "another_custom", serde_json::json!([1, 2, 3])).await;
+
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+
+    // Typed fields
+    assert_eq!(
+        config.global_settings.base_url.as_deref(),
+        Some("https://mixed.test")
+    );
+    assert_eq!(config.global_settings.expose_metrics, Some(true));
+
+    // Extra fields
+    assert_eq!(
+        config.global_settings.extra["unknown_future_setting"],
+        serde_json::json!({"key": "val"})
+    );
+    assert_eq!(
+        config.global_settings.extra["another_custom"],
+        serde_json::json!([1, 2, 3])
+    );
+
+    // Roundtrip: to_settings_map should include everything
+    let map = config.global_settings.to_settings_map();
+    assert!(map.contains_key("base_url"));
+    assert!(map.contains_key("expose_metrics"));
+    assert!(map.contains_key("unknown_future_setting"));
+    assert!(map.contains_key("another_custom"));
+    assert_eq!(map.len(), 4);
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_full_config_roundtrip(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Seed a realistic configuration
+    insert_global_setting(
+        &db,
+        "base_url",
+        serde_json::json!("https://prod.windmill.dev"),
+    )
+    .await;
+    insert_global_setting(&db, "license_key", serde_json::json!("prod-license-key")).await;
+    insert_global_setting(&db, "retention_period_secs", serde_json::json!(2592000)).await;
+    insert_global_setting(
+        &db,
+        "smtp_settings",
+        serde_json::json!({
+            "smtp_host": "smtp.prod.com",
+            "smtp_port": 587,
+            "smtp_from": "noreply@prod.com",
+            "smtp_tls_implicit": true
+        }),
+    )
+    .await;
+    insert_global_setting(
+        &db,
+        "critical_error_channels",
+        serde_json::json!([
+            {"email": "admin@prod.com"},
+            {"slack_channel": "#prod-alerts"}
+        ]),
+    )
+    .await;
+    insert_global_setting(
+        &db,
+        "otel",
+        serde_json::json!({
+            "metrics_enabled": true,
+            "tracing_enabled": true,
+            "otel_exporter_otlp_endpoint": "http://otel-collector:4317"
+        }),
+    )
+    .await;
+
+    insert_config(
+        &db,
+        "worker__default",
+        serde_json::json!({
+            "init_bash": "apt-get update",
+            "worker_tags": ["default", "deno", "bun"],
+            "cache_clear": 7
+        }),
+    )
+    .await;
+    insert_config(
+        &db,
+        "worker__gpu",
+        serde_json::json!({
+            "dedicated_worker": "ws:f/gpu_inference",
+            "autoscaling": {
+                "enabled": true,
+                "min_workers": 0,
+                "max_workers": 4,
+                "integration": {"type": "kubernetes"}
+            }
+        }),
+    )
+    .await;
+
+    // Read full config
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+
+    assert_eq!(
+        config.global_settings.base_url.as_deref(),
+        Some("https://prod.windmill.dev")
+    );
+    assert_eq!(
+        config
+            .global_settings
+            .license_key
+            .as_ref()
+            .and_then(|v| v.as_literal()),
+        Some("prod-license-key")
+    );
+    assert_eq!(config.global_settings.retention_period_secs, Some(2592000));
+
+    let smtp = config.global_settings.smtp_settings.as_ref().unwrap();
+    assert_eq!(smtp.smtp_host.as_deref(), Some("smtp.prod.com"));
+    assert_eq!(smtp.smtp_from.as_deref(), Some("noreply@prod.com"));
+
+    let channels = config
+        .global_settings
+        .critical_error_channels
+        .as_ref()
+        .unwrap();
+    assert_eq!(channels.len(), 2);
+
+    let otel = config.global_settings.otel.as_ref().unwrap();
+    assert_eq!(otel.metrics_enabled, Some(true));
+    assert_eq!(otel.tracing_enabled, Some(true));
+
+    assert_eq!(config.worker_configs.len(), 2);
+    assert_eq!(config.worker_configs["default"].cache_clear, Some(7));
+    let gpu_auto = config.worker_configs["gpu"].autoscaling.as_ref().unwrap();
+    assert!(gpu_auto.enabled);
+    assert_eq!(gpu_auto.min_workers, Some(0));
+    assert_eq!(gpu_auto.max_workers, Some(4));
+
+    // Verify settings count matches
+    let settings_map = config.global_settings.to_settings_map();
+    let db_count = count_global_settings(&db).await;
+    assert_eq!(
+        settings_map.len() as i64,
+        db_count,
+        "to_settings_map should produce same count as DB rows"
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_diff_apply_only_touches_changed_rows(db: Pool<Postgres>) {
+    clear_settings_and_configs(&db).await;
+
+    // Seed 3 settings
+    insert_global_setting(&db, "unchanged_1", serde_json::json!("val1")).await;
+    insert_global_setting(&db, "unchanged_2", serde_json::json!("val2")).await;
+    insert_global_setting(&db, "to_change", serde_json::json!("old")).await;
+
+    let mut current = BTreeMap::new();
+    current.insert("unchanged_1".to_string(), serde_json::json!("val1"));
+    current.insert("unchanged_2".to_string(), serde_json::json!("val2"));
+    current.insert("to_change".to_string(), serde_json::json!("old"));
+
+    let mut desired = current.clone();
+    desired.insert("to_change".to_string(), serde_json::json!("new"));
+    desired.insert("added".to_string(), serde_json::json!("fresh"));
+
+    let diff = diff_global_settings(&current, &desired, ApplyMode::Merge);
+
+    // Only "to_change" and "added" should be in upserts
+    assert_eq!(diff.upserts.len(), 2);
+    assert!(diff.upserts.contains_key("to_change"));
+    assert!(diff.upserts.contains_key("added"));
+    assert!(!diff.upserts.contains_key("unchanged_1"));
+    assert!(!diff.upserts.contains_key("unchanged_2"));
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    // Verify all 4 settings present
+    assert_eq!(count_global_settings(&db).await, 4);
+    assert_eq!(
+        get_global_setting(&db, "to_change").await,
+        Some(serde_json::json!("new"))
+    );
+    assert_eq!(
+        get_global_setting(&db, "added").await,
+        Some(serde_json::json!("fresh"))
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_from_db_reads_migration_defaults(db: Pool<Postgres>) {
+    // Verify that from_db correctly reads the migration-seeded state
+    // without clearing — tests that pre-existing data is properly deserialized
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+
+    // Migrations seed at least base_url, license_key, etc.
+    // Just verify from_db doesn't error and returns a populated struct
+    let map = config.global_settings.to_settings_map();
+    assert!(
+        !map.is_empty(),
+        "Migration-seeded DB should produce non-empty settings"
+    );
+    assert!(
+        !config.worker_configs.is_empty(),
+        "Migration-seeded DB should have worker configs"
+    );
+}
+
+#[sqlx::test(fixtures("base"))]
+async fn test_replace_mode_protects_settings_in_integration(db: Pool<Postgres>) {
+    // Seed a protected setting
+    insert_global_setting(
+        &db,
+        "ducklake_settings",
+        serde_json::json!({"ducklakes": {}}),
+    )
+    .await;
+    insert_global_setting(&db, "ducklake_user_pg_pwd", serde_json::json!("secret_pwd")).await;
+    insert_global_setting(
+        &db,
+        "custom_instance_pg_databases",
+        serde_json::json!({"databases": {}}),
+    )
+    .await;
+    insert_global_setting(&db, "normal_setting", serde_json::json!("will_be_deleted")).await;
+
+    // Read current state
+    let config = InstanceConfig::from_db(&db).await.unwrap();
+    let current_map = config.global_settings.to_settings_map();
+
+    // Desired: only keep_me — everything else should be deleted except protected
+    let mut desired_map = BTreeMap::new();
+    desired_map.insert("keep_me".to_string(), serde_json::json!("yes"));
+
+    let diff = diff_global_settings(&current_map, &desired_map, ApplyMode::Replace);
+
+    // Protected keys should NOT be in deletes
+    assert!(
+        !diff.deletes.contains(&"ducklake_settings".to_string()),
+        "ducklake_settings is protected"
+    );
+    assert!(
+        !diff.deletes.contains(&"ducklake_user_pg_pwd".to_string()),
+        "ducklake_user_pg_pwd is protected"
+    );
+    assert!(
+        !diff
+            .deletes
+            .contains(&"custom_instance_pg_databases".to_string()),
+        "custom_instance_pg_databases is protected"
+    );
+    // But normal_setting should be deleted
+    assert!(diff.deletes.contains(&"normal_setting".to_string()));
+
+    apply_settings_diff(&db, &diff).await.unwrap();
+
+    // Verify protected settings survived
+    assert!(get_global_setting(&db, "ducklake_settings").await.is_some());
+    assert!(get_global_setting(&db, "ducklake_user_pg_pwd")
+        .await
+        .is_some());
+    assert!(get_global_setting(&db, "custom_instance_pg_databases")
+        .await
+        .is_some());
+    // Normal setting is gone
+    assert!(get_global_setting(&db, "normal_setting").await.is_none());
+    // New setting is present
+    assert_eq!(
+        get_global_setting(&db, "keep_me").await,
+        Some(serde_json::json!("yes"))
+    );
+}
@@ -0,0 +1,448 @@
+/*!
+ * Integration tests for windmill-operator db_sync module.
+ *
+ * Tests verify full declarative sync of global_settings and worker configs:
+ * - Upsert desired settings into DB
+ * - Delete settings present in DB but absent from desired state
+ * - Protect certain internal settings from deletion
+ */
+
+#[cfg(feature = "operator")]
+mod tests {
+    use std::collections::BTreeMap;
+
+    use sqlx::{Pool, Postgres};
+
+    // ========================================================================
+    // Helpers
+    // ========================================================================
+
+    async fn get_global_setting(db: &Pool<Postgres>, name: &str) -> Option<serde_json::Value> {
+        sqlx::query_as::<_, (serde_json::Value,)>(
+            "SELECT value FROM global_settings WHERE name = $1",
+        )
+        .bind(name)
+        .fetch_optional(db)
+        .await
+        .expect("query should succeed")
+        .map(|(v,)| v)
+    }
+
+    async fn get_config(db: &Pool<Postgres>, name: &str) -> Option<serde_json::Value> {
+        sqlx::query_as::<_, (serde_json::Value,)>("SELECT config FROM config WHERE name = $1")
+            .bind(name)
+            .fetch_optional(db)
+            .await
+            .expect("query should succeed")
+            .map(|(v,)| v)
+    }
+
+    async fn insert_global_setting(db: &Pool<Postgres>, name: &str, value: serde_json::Value) {
+        sqlx::query(
+            "INSERT INTO global_settings (name, value) VALUES ($1, $2) \
+             ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
+        )
+        .bind(name)
+        .bind(&value)
+        .execute(db)
+        .await
+        .expect("insert should succeed");
+    }
+
+    async fn insert_config(db: &Pool<Postgres>, name: &str, config: serde_json::Value) {
+        sqlx::query(
+            "INSERT INTO config (name, config) VALUES ($1, $2) \
+             ON CONFLICT (name) DO UPDATE SET config = EXCLUDED.config",
+        )
+        .bind(name)
+        .bind(&config)
+        .execute(db)
+        .await
+        .expect("insert should succeed");
+    }
+
+    // ========================================================================
+    // sync_global_settings tests
+    // ========================================================================
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_upserts(db: Pool<Postgres>) {
+        let mut desired = BTreeMap::new();
+        desired.insert(
+            "test_op_setting_a".to_string(),
+            serde_json::json!("value_a"),
+        );
+        desired.insert("test_op_setting_b".to_string(), serde_json::json!(42));
+
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert_eq!(
+            get_global_setting(&db, "test_op_setting_a").await,
+            Some(serde_json::json!("value_a"))
+        );
+        assert_eq!(
+            get_global_setting(&db, "test_op_setting_b").await,
+            Some(serde_json::json!(42))
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_updates_existing(db: Pool<Postgres>) {
+        // Pre-populate a setting
+        insert_global_setting(&db, "test_op_existing", serde_json::json!("old")).await;
+
+        // Sync with new value
+        let mut desired = BTreeMap::new();
+        desired.insert("test_op_existing".to_string(), serde_json::json!("new"));
+
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert_eq!(
+            get_global_setting(&db, "test_op_existing").await,
+            Some(serde_json::json!("new"))
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_deletes_absent(db: Pool<Postgres>) {
+        // Pre-populate settings
+        insert_global_setting(&db, "test_op_keep", serde_json::json!("keep")).await;
+        insert_global_setting(&db, "test_op_remove", serde_json::json!("remove")).await;
+
+        // Sync with only one of them
+        let mut desired = BTreeMap::new();
+        desired.insert("test_op_keep".to_string(), serde_json::json!("keep"));
+
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(get_global_setting(&db, "test_op_keep").await.is_some());
+        assert!(
+            get_global_setting(&db, "test_op_remove").await.is_none(),
+            "Setting absent from desired should be deleted"
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_protects_ducklake(db: Pool<Postgres>) {
+        // Pre-populate a protected setting
+        insert_global_setting(
+            &db,
+            "ducklake_settings",
+            serde_json::json!({"protected": true}),
+        )
+        .await;
+
+        // Sync with empty desired — protected key should survive
+        let desired = BTreeMap::new();
+
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(
+            get_global_setting(&db, "ducklake_settings").await.is_some(),
+            "Protected setting ducklake_settings should not be deleted"
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_protects_all_protected_keys(db: Pool<Postgres>) {
+        let protected_keys = [
+            "ducklake_user_pg_pwd",
+            "ducklake_settings",
+            "custom_instance_pg_databases",
+        ];
+
+        for key in &protected_keys {
+            insert_global_setting(&db, key, serde_json::json!("protected_value")).await;
+        }
+
+        // Sync with empty desired
+        let desired = BTreeMap::new();
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        for key in &protected_keys {
+            assert!(
+                get_global_setting(&db, key).await.is_some(),
+                "Protected key {key} should not be deleted"
+            );
+        }
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_empty_desired(db: Pool<Postgres>) {
+        // Pre-populate a non-protected setting
+        insert_global_setting(&db, "test_op_ephemeral", serde_json::json!("gone")).await;
+
+        let desired = BTreeMap::new();
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(
+            get_global_setting(&db, "test_op_ephemeral").await.is_none(),
+            "Non-protected settings should be deleted when desired is empty"
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_global_settings_complex_json_values(db: Pool<Postgres>) {
+        let mut desired = BTreeMap::new();
+        desired.insert(
+            "test_op_complex".to_string(),
+            serde_json::json!({
+                "host": "smtp.example.com",
+                "port": 587,
+                "tls": true,
+                "nested": {"array": [1, 2, 3]}
+            }),
+        );
+
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        let stored = get_global_setting(&db, "test_op_complex")
+            .await
+            .expect("Setting should exist");
+        assert_eq!(stored["host"], "smtp.example.com");
+        assert_eq!(stored["port"], 587);
+        assert_eq!(stored["nested"]["array"][1], 2);
+    }
+
+    // ========================================================================
+    // sync_worker_configs tests
+    // ========================================================================
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_worker_configs_upserts_with_prefix(db: Pool<Postgres>) {
+        let mut desired = BTreeMap::new();
+        desired.insert(
+            "test_group".to_string(),
+            serde_json::json!({"dedicated_worker": false}),
+        );
+
+        windmill_operator::db_sync::sync_worker_configs(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        let config = get_config(&db, "worker__test_group")
+            .await
+            .expect("Config should exist with worker__ prefix");
+        assert_eq!(config["dedicated_worker"], false);
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_worker_configs_updates_existing(db: Pool<Postgres>) {
+        // Pre-populate
+        insert_config(
+            &db,
+            "worker__test_wc_existing",
+            serde_json::json!({"old": true}),
+        )
+        .await;
+
+        let mut desired = BTreeMap::new();
+        desired.insert(
+            "test_wc_existing".to_string(),
+            serde_json::json!({"new": true}),
+        );
+
+        windmill_operator::db_sync::sync_worker_configs(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        let config = get_config(&db, "worker__test_wc_existing")
+            .await
+            .expect("Config should exist");
+        assert_eq!(config["new"], true);
+        assert!(config.get("old").is_none());
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_worker_configs_deletes_absent(db: Pool<Postgres>) {
+        // Pre-populate two worker configs
+        insert_config(
+            &db,
+            "worker__test_wc_keep",
+            serde_json::json!({"keep": true}),
+        )
+        .await;
+        insert_config(
+            &db,
+            "worker__test_wc_remove",
+            serde_json::json!({"remove": true}),
+        )
+        .await;
+
+        // Sync with only one
+        let mut desired = BTreeMap::new();
+        desired.insert(
+            "test_wc_keep".to_string(),
+            serde_json::json!({"keep": true}),
+        );
+
+        windmill_operator::db_sync::sync_worker_configs(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(get_config(&db, "worker__test_wc_keep").await.is_some());
+        assert!(
+            get_config(&db, "worker__test_wc_remove").await.is_none(),
+            "Worker config absent from desired should be deleted"
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_worker_configs_does_not_touch_non_worker_configs(db: Pool<Postgres>) {
+        // Insert a non-worker config (no worker__ prefix)
+        insert_config(&db, "server_config", serde_json::json!({"important": true})).await;
+
+        // Sync with empty worker configs
+        let desired = BTreeMap::new();
+        windmill_operator::db_sync::sync_worker_configs(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(
+            get_config(&db, "server_config").await.is_some(),
+            "Non-worker configs should not be touched"
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_worker_configs_multiple_groups(db: Pool<Postgres>) {
+        let mut desired = BTreeMap::new();
+        desired.insert(
+            "default".to_string(),
+            serde_json::json!({"init_bash": "echo default"}),
+        );
+        desired.insert(
+            "gpu".to_string(),
+            serde_json::json!({"dedicated_worker": true}),
+        );
+        desired.insert(
+            "native".to_string(),
+            serde_json::json!({"init_bash": "echo native"}),
+        );
+
+        windmill_operator::db_sync::sync_worker_configs(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(get_config(&db, "worker__default").await.is_some());
+        assert!(get_config(&db, "worker__gpu").await.is_some());
+        assert!(get_config(&db, "worker__native").await.is_some());
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_sync_worker_configs_empty_desired(db: Pool<Postgres>) {
+        insert_config(
+            &db,
+            "worker__test_wc_gone",
+            serde_json::json!({"ephemeral": true}),
+        )
+        .await;
+
+        let desired = BTreeMap::new();
+        windmill_operator::db_sync::sync_worker_configs(&db, &desired)
+            .await
+            .expect("sync should succeed");
+
+        assert!(
+            get_config(&db, "worker__test_wc_gone").await.is_none(),
+            "Worker config should be deleted when desired is empty"
+        );
+    }
+
+    // ========================================================================
+    // End-to-end: both syncs together
+    // ========================================================================
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_full_declarative_sync(db: Pool<Postgres>) {
+        // Pre-populate some existing state
+        insert_global_setting(&db, "test_op_stale_setting", serde_json::json!("stale")).await;
+        insert_config(
+            &db,
+            "worker__test_stale_group",
+            serde_json::json!({"stale": true}),
+        )
+        .await;
+
+        // Define desired state
+        let mut global_settings = BTreeMap::new();
+        global_settings.insert(
+            "test_op_base_url".to_string(),
+            serde_json::json!("https://windmill.example.com"),
+        );
+        global_settings.insert(
+            "test_op_license_key".to_string(),
+            serde_json::json!("my-license"),
+        );
+
+        let mut worker_configs = BTreeMap::new();
+        worker_configs.insert(
+            "default".to_string(),
+            serde_json::json!({"init_bash": "echo hello"}),
+        );
+
+        // Sync both
+        windmill_operator::db_sync::sync_global_settings(&db, &global_settings)
+            .await
+            .expect("global sync should succeed");
+        windmill_operator::db_sync::sync_worker_configs(&db, &worker_configs)
+            .await
+            .expect("worker sync should succeed");
+
+        // Verify desired state is present
+        assert_eq!(
+            get_global_setting(&db, "test_op_base_url").await,
+            Some(serde_json::json!("https://windmill.example.com"))
+        );
+        assert_eq!(
+            get_global_setting(&db, "test_op_license_key").await,
+            Some(serde_json::json!("my-license"))
+        );
+        assert!(get_config(&db, "worker__default").await.is_some());
+
+        // Verify stale state is removed
+        assert!(
+            get_global_setting(&db, "test_op_stale_setting")
+                .await
+                .is_none(),
+            "Stale global setting should be removed"
+        );
+        assert!(
+            get_config(&db, "worker__test_stale_group").await.is_none(),
+            "Stale worker config should be removed"
+        );
+    }
+
+    #[sqlx::test(fixtures("base"))]
+    async fn test_idempotent_sync(db: Pool<Postgres>) {
+        let mut desired = BTreeMap::new();
+        desired.insert("test_op_idempotent".to_string(), serde_json::json!("value"));
+
+        // Run sync twice — should be idempotent
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("first sync should succeed");
+        windmill_operator::db_sync::sync_global_settings(&db, &desired)
+            .await
+            .expect("second sync should succeed");
+
+        assert_eq!(
+            get_global_setting(&db, "test_op_idempotent").await,
+            Some(serde_json::json!("value"))
+        );
+    }
+}
@@ -22,13 +22,10 @@ use ee_oss::validate_license_key;
 use windmill_common::usernames::generate_instance_username_for_all_users;

 use axum::{
-    extract::{Extension, Path},
+    extract::{Extension, Path, Query},
    routing::{get, post},
    Json, Router,
 };
-
-#[cfg(feature = "enterprise")]
-use axum::extract::Query;
 use serde_json::json;

 use serde::{Deserialize, Serialize};
@@ -45,6 +42,7 @@ use windmill_common::{
        CRITICAL_ALERT_MUTE_UI_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
        HUB_ACCESSIBLE_URL_SETTING, HUB_BASE_URL_SETTING,
    },
+    instance_config::{self, ApplyMode, InstanceConfig},
    server::Smtp,
 };
 use windmill_common::{error::to_anyhow, PgDatabase};
@@ -58,6 +56,10 @@ pub fn global_service() -> Router {
            post(set_global_setting).get(get_global_setting),
        )
        .route("/list_global", get(list_global_settings))
+        .route(
+            "/instance_config",
+            get(get_instance_config).put(set_instance_config),
+        )
        .route("/test_smtp", post(test_email))
        .route("/test_license_key", post(test_license_key))
        .route("/send_stats", post(send_stats))
@@ -271,9 +273,40 @@ pub async fn set_global_setting_internal(
    key: String,
    value: serde_json::Value,
 ) -> error::Result<()> {
-    match key.as_str() {
+    run_setting_pre_write_hook(db, &key, &value).await?;
+
+    match value {
+        serde_json::Value::Null => {
+            delete_global_setting(db, &key).await?;
+        }
+        serde_json::Value::String(x) if x.is_empty() => {
+            delete_global_setting(db, &key).await?;
+        }
+        v => {
+            sqlx::query!(
+                 "INSERT INTO global_settings (name, value) VALUES ($1, $2) ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value, updated_at = now()",
+                 key,
+                 v
+             )
+             .execute(db)
+             .await?;
+            tracing::info!("Set global setting {} to {}", key, v);
+        }
+    };
+
+    Ok(())
+}
+
+/// Run side-effect hooks for specific settings before writing to DB.
+/// Extracted from `set_global_setting_internal` for reuse by the bulk endpoint.
+async fn run_setting_pre_write_hook(
+    db: &DB,
+    key: &str,
+    value: &serde_json::Value,
+) -> error::Result<()> {
+    match key {
        AUTOMATE_USERNAME_CREATION_SETTING => {
-            if value.clone().as_bool().unwrap_or(false) {
+            if value.as_bool().unwrap_or(false) {
                generate_instance_username_for_all_users(db)
                    .await
                    .map_err(|err| {
@@ -285,14 +318,14 @@ pub async fn set_global_setting_internal(
            }
        }
        CRITICAL_ALERT_MUTE_UI_SETTING => {
-            if value.clone().as_bool().unwrap_or(false) {
+            if value.as_bool().unwrap_or(false) {
                sqlx::query!("UPDATE alerts SET acknowledged = true")
                    .execute(db)
                    .await?;
            }
        }
        APP_WORKSPACED_ROUTE_SETTING => {
-            let serde_json::Value::Bool(workspaced_route) = &value else {
+            let serde_json::Value::Bool(workspaced_route) = value else {
                return Err(error::Error::BadRequest(format!(
                    "{} setting Expected to be boolean",
                    APP_WORKSPACED_ROUTE_SETTING
@@ -312,15 +345,15 @@ pub async fn set_global_setting_internal(
                        SELECT
                            path,
                            custom_path
-                        FROM 
+                        FROM
                            app
-                        WHERE 
+                        WHERE
                            custom_path IN (
-                                SELECT 
+                                SELECT
                                    custom_path
-                                FROM 
+                                FROM
                                    app
-                                GROUP 
+                                GROUP
                                    BY custom_path
                                HAVING COUNT(*) > 1
                            )
@@ -356,25 +389,84 @@ pub async fn set_global_setting_internal(
        }
        _ => {}
    }
+    Ok(())
+}

-    match value {
-        serde_json::Value::Null => {
-            delete_global_setting(db, &key).await?;
-        }
-        serde_json::Value::String(x) if x.is_empty() => {
-            delete_global_setting(db, &key).await?;
-        }
-        v => {
-            sqlx::query!(
-                 "INSERT INTO global_settings (name, value) VALUES ($1, $2) ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value, updated_at = now()",
-                 key,
-                 v
-             )
-             .execute(db)
-             .await?;
-            tracing::info!("Set global setting {} to {}", key, v);
-        }
-    };
+// ---------------------------------------------------------------------------
+// Bulk instance config endpoints
+// ---------------------------------------------------------------------------
+
+async fn get_instance_config(
+    Extension(db): Extension<DB>,
+    authed: ApiAuthed,
+) -> JsonResult<InstanceConfig> {
+    require_super_admin(&db, &authed.email).await?;
+    let config = InstanceConfig::from_db(&db)
+        .await
+        .map_err(|e| error::Error::internal_err(e.to_string()))?;
+    Ok(Json(config))
+}
+
+#[derive(Deserialize)]
+struct SetInstanceConfigQuery {
+    #[serde(default)]
+    skip_worker_configs: Option<bool>,
+}
+
+async fn set_instance_config(
+    Extension(db): Extension<DB>,
+    authed: ApiAuthed,
+    Query(query): Query<SetInstanceConfigQuery>,
+    Json(desired): Json<InstanceConfig>,
+) -> error::Result<()> {
+    require_super_admin(&db, &authed.email).await?;
+
+    let current = InstanceConfig::from_db(&db)
+        .await
+        .map_err(|e| error::Error::internal_err(e.to_string()))?;
+
+    let current_map = current.global_settings.to_settings_map();
+    let desired_map = desired.global_settings.to_settings_map();
+    let settings_diff =
+        instance_config::diff_global_settings(&current_map, &desired_map, ApplyMode::Merge);
+
+    for (key, value) in &settings_diff.upserts {
+        run_setting_pre_write_hook(&db, key, value).await?;
+    }
+
+    instance_config::apply_settings_diff(&db, &settings_diff)
+        .await
+        .map_err(|e| error::Error::internal_err(e.to_string()))?;
+
+    if !query.skip_worker_configs.unwrap_or(false) {
+        let current_wc: std::collections::BTreeMap<String, serde_json::Value> = current
+            .worker_configs
+            .iter()
+            .map(|(k, v)| {
+                (
+                    k.clone(),
+                    serde_json::to_value(v)
+                        .expect("WorkerGroupConfig serialization cannot fail"),
+                )
+            })
+            .collect();
+        let desired_wc: std::collections::BTreeMap<String, serde_json::Value> = desired
+            .worker_configs
+            .iter()
+            .map(|(k, v)| {
+                (
+                    k.clone(),
+                    serde_json::to_value(v)
+                        .expect("WorkerGroupConfig serialization cannot fail"),
+                )
+            })
+            .collect();
+        let configs_diff =
+            instance_config::diff_worker_configs(&current_wc, &desired_wc, ApplyMode::Merge);
+        instance_config::apply_configs_diff(&db, &configs_diff)
+            .await
+            .map_err(|e| error::Error::internal_err(e.to_string()))?;
+    }

    Ok(())
 }
@@ -945,7 +945,7 @@ async fn join_workspace<'c>(
    .await?
    .map(|v| v.as_bool())
    .flatten()
-    .unwrap_or(false);
+    .unwrap_or(true);

    let username = if automate_username_creation {
        if username.is_some() && username.unwrap().len() > 0 {
@@ -2748,7 +2748,7 @@ async fn create_workspace(
    .await?
    .map(|v| v.as_bool())
    .flatten()
-    .unwrap_or(false);
+    .unwrap_or(true);

    let username = if automate_username_creation {
        if nw.username.is_some() && nw.username.unwrap().len() > 0 {
@@ -3807,7 +3807,7 @@ async fn add_user(
    .await?
    .map(|v| v.as_bool())
    .flatten()
-    .unwrap_or(false);
+    .unwrap_or(true);

    let username = if automate_username_creation {
        if nu.username.is_some() && nu.username.unwrap().len() > 0 {
@@ -1483,6 +1483,45 @@ paths:
                items:
                  $ref: "#/components/schemas/GlobalSetting"

+  /settings/instance_config:
+    get:
+      summary: get full instance config (global settings + worker configs)
+      operationId: getInstanceConfig
+      tags:
+        - setting
+      responses:
+        "200":
+          description: full instance configuration
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/InstanceConfig"
+    put:
+      summary: update instance config (bulk upsert, no deletes)
+      operationId: setInstanceConfig
+      tags:
+        - setting
+      parameters:
+        - name: skip_worker_configs
+          in: query
+          description: if true, ignore worker_configs in the request body
+          schema:
+            type: boolean
+      requestBody:
+        description: full instance configuration to apply
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/InstanceConfig"
+      responses:
+        "200":
+          description: instance config updated
+          content:
+            text/plain:
+              schema:
+                type: string
+
  /min_keep_alive_version:
    get:
      summary: get minimum worker versions required to stay alive
@@ -22966,6 +23005,26 @@ components:
        - name
        - value

+    InstanceConfig:
+      type: object
+      description: Unified instance configuration combining global settings and worker group configs
+      properties:
+        global_settings:
+          type: object
+          description: >
+            Global settings keyed by setting name. Known fields include base_url,
+            license_key, retention_period_secs, smtp_settings, otel, etc.
+            Unknown fields are preserved as-is.
+          additionalProperties: true
+        worker_configs:
+          type: object
+          description: >
+            Worker group configurations keyed by group name (e.g. "default", "gpu").
+            Each value contains worker_tags, init_bash, autoscaling, etc.
+          additionalProperties:
+            type: object
+            additionalProperties: true
+
    Config:
      type: object
      properties:
@@ -7,6 +7,7 @@ edition.workspace = true
 [features]
 default = []
 enterprise = ["dep:aws-config"]
+instance_config_schema = ["dep:schemars"]
 local_reports = ["dep:rsa", "dep:aes-gcm"]
 private = ["dep:aws-sdk-rds"]
 jemalloc = ["dep:tikv-jemalloc-ctl"]
@@ -96,6 +97,7 @@ windmill-parser.workspace = true
 jsonwebtoken.workspace = true
 backon.workspace = true
 openidconnect = { workspace = true, optional = true }
+schemars = { workspace = true, optional = true }
 strum.workspace = true
 strum_macros.workspace = true
 windmill-types.workspace = true
@@ -55,6 +55,7 @@ pub mod flow_status;
 pub mod flows;
 pub mod global_settings;
 pub mod indexer;
+pub mod instance_config;
 pub mod job_metrics;
 #[cfg(all(feature = "parquet", feature = "private"))]
 pub mod job_s3_helpers_ee;
@@ -0,0 +1,30 @@
+[package]
+name = "windmill-operator"
+version.workspace = true
+authors.workspace = true
+edition.workspace = true
+
+[lib]
+name = "windmill_operator"
+path = "./src/lib.rs"
+
+[features]
+enterprise = ["windmill-common/enterprise"]
+private = []
+default = []
+
+[dependencies]
+serde.workspace = true
+serde_json.workspace = true
+sqlx.workspace = true
+tracing.workspace = true
+windmill-common = { workspace = true, default-features = false, features = ["instance_config_schema"] }
+anyhow.workspace = true
+thiserror.workspace = true
+kube.workspace = true
+k8s-openapi.workspace = true
+tokio.workspace = true
+futures.workspace = true
+chrono.workspace = true
+schemars = "0.8"
+serde_yml.workspace = true
@@ -0,0 +1,703 @@
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  name: windmillinstances.windmill.dev
+spec:
+  group: windmill.dev
+  names:
+    categories: []
+    kind: WindmillInstance
+    plural: windmillinstances
+    shortNames:
+    - wmi
+    singular: windmillinstance
+  scope: Namespaced
+  versions:
+  - additionalPrinterColumns:
+    - jsonPath: '.status.synced'
+      name: Synced
+      type: string
+    - jsonPath: '.status.lastSyncedAt'
+      name: Last Synced
+      type: date
+    - jsonPath: '.metadata.creationTimestamp'
+      name: Age
+      type: date
+    name: v1alpha1
+    schema:
+      openAPIV3Schema:
+        description: Auto-generated derived type for WindmillInstanceSpec via `CustomResource`
+        properties:
+          spec:
+            description: |-
+              WindmillInstance CRD spec.
+
+              Declares the desired state for instance-level configuration: - `global_settings` maps directly to the `global_settings` table - `worker_configs` maps to the `config` table with a `worker__` prefix
+            properties:
+              global_settings:
+                default: {}
+                description: Global settings to sync to the `global_settings` table.
+                properties:
+                  app_workspaced_route:
+                    nullable: true
+                    type: boolean
+                  base_url:
+                    nullable: true
+                    type: string
+                  bunfig_install_scopes:
+                    nullable: true
+                    type: string
+                  critical_alert_mute_ui:
+                    nullable: true
+                    type: boolean
+                  critical_alerts_on_db_oversize:
+                    description: Configuration for critical alerts when the database exceeds a size threshold.
+                    nullable: true
+                    properties:
+                      enabled:
+                        default: false
+                        type: boolean
+                      value:
+                        default: 0.0
+                        format: float
+                        type: number
+                    type: object
+                  critical_error_channels:
+                    items:
+                      anyOf:
+                      - required:
+                        - email
+                      - required:
+                        - slack_channel
+                      - required:
+                        - teams_channel
+                      description: A channel for delivering critical error alerts.
+                      properties:
+                        email:
+                          type: string
+                        slack_channel:
+                          type: string
+                        teams_channel:
+                          description: Microsoft Teams channel reference.
+                          properties:
+                            channel_id:
+                              type: string
+                            channel_name:
+                              type: string
+                            team_id:
+                              type: string
+                            team_name:
+                              type: string
+                          required:
+                          - channel_id
+                          - channel_name
+                          - team_id
+                          - team_name
+                          type: object
+                      type: object
+                    nullable: true
+                    type: array
+                  custom_instance_pg_databases:
+                    description: Custom PostgreSQL databases managed by the instance.
+                    nullable: true
+                    properties:
+                      databases:
+                        additionalProperties:
+                          description: Status of a single custom instance database.
+                          properties:
+                            error:
+                              nullable: true
+                              type: string
+                            logs:
+                              default:
+                                super_admin: ''
+                              description: Setup log entries for a custom instance database.
+                              properties:
+                                created_database:
+                                  type: string
+                                database_credentials:
+                                  type: string
+                                db_connect:
+                                  type: string
+                                grant_permissions:
+                                  type: string
+                                super_admin:
+                                  default: ''
+                                  type: string
+                                valid_dbname:
+                                  type: string
+                              type: object
+                            success:
+                              default: false
+                              type: boolean
+                            tag:
+                              nullable: true
+                              type: string
+                          type: object
+                        type: object
+                      user_pwd:
+                        nullable: true
+                        type: string
+                    type: object
+                  custom_tags:
+                    items:
+                      type: string
+                    nullable: true
+                    type: array
+                  default_tags_per_workspace:
+                    nullable: true
+                    type: boolean
+                  default_tags_workspaces:
+                    items:
+                      type: string
+                    nullable: true
+                    type: array
+                  dev_instance:
+                    nullable: true
+                    type: boolean
+                  disable_stats:
+                    nullable: true
+                    type: boolean
+                  ducklake_settings:
+                    description: DuckLake catalog database settings.
+                    nullable: true
+                    properties:
+                      ducklakes:
+                        additionalProperties:
+                          description: A single DuckLake instance configuration.
+                          properties:
+                            catalog:
+                              description: DuckLake catalog backend reference.
+                              properties:
+                                resource_path:
+                                  type: string
+                                resource_type:
+                                  description: The type of database backing a DuckLake catalog.
+                                  enum:
+                                  - postgresql
+                                  - mysql
+                                  - instance
+                                  type: string
+                              required:
+                              - resource_path
+                              - resource_type
+                              type: object
+                            extra_args:
+                              nullable: true
+                              type: string
+                            storage:
+                              description: DuckLake storage location.
+                              properties:
+                                path:
+                                  type: string
+                                storage:
+                                  nullable: true
+                                  type: string
+                              required:
+                              - path
+                              type: object
+                          required:
+                          - catalog
+                          - storage
+                          type: object
+                        type: object
+                    required:
+                    - ducklakes
+                    type: object
+                  email_domain:
+                    nullable: true
+                    type: string
+                  expose_debug_metrics:
+                    nullable: true
+                    type: boolean
+                  expose_metrics:
+                    nullable: true
+                    type: boolean
+                  hub_accessible_url:
+                    nullable: true
+                    type: string
+                  hub_api_secret:
+                    nullable: true
+                    type: string
+                  hub_base_url:
+                    nullable: true
+                    type: string
+                  indexer_settings:
+                    description: Full-text search indexer configuration.
+                    nullable: true
+                    properties:
+                      commit_job_max_batch_size:
+                        format: uint64
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                      commit_log_max_batch_size:
+                        format: uint64
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                      max_indexed_job_log_size:
+                        format: uint64
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                      refresh_index_period:
+                        format: uint64
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                      refresh_log_index_period:
+                        format: uint64
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                      should_clear_job_index:
+                        nullable: true
+                        type: boolean
+                      should_clear_log_index:
+                        nullable: true
+                        type: boolean
+                      writer_memory_budget:
+                        format: uint64
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                    type: object
+                  instance_python_version:
+                    nullable: true
+                    type: string
+                  job_default_timeout:
+                    format: int64
+                    nullable: true
+                    type: integer
+                  jwt_secret:
+                    nullable: true
+                    type: string
+                  keep_job_dir:
+                    nullable: true
+                    type: boolean
+                  license_key:
+                    nullable: true
+                    type: string
+                  maven_repos:
+                    nullable: true
+                    type: string
+                  min_keep_alive_version:
+                    nullable: true
+                    type: string
+                  monitor_logs_on_s3:
+                    nullable: true
+                    type: boolean
+                  no_default_maven:
+                    nullable: true
+                    type: boolean
+                  npm_config_registry:
+                    nullable: true
+                    type: string
+                  nuget_config:
+                    nullable: true
+                    type: string
+                  oauths:
+                    additionalProperties:
+                      description: OAuth client configuration for a single provider.
+                      properties:
+                        allowed_domains:
+                          items:
+                            type: string
+                          nullable: true
+                          type: array
+                        connect_config:
+                          description: OAuth provider endpoint configuration.
+                          nullable: true
+                          properties:
+                            auth_url:
+                              type: string
+                            extra_params:
+                              additionalProperties:
+                                type: string
+                              nullable: true
+                              type: object
+                            extra_params_callback:
+                              additionalProperties:
+                                type: string
+                              nullable: true
+                              type: object
+                            req_body_auth:
+                              nullable: true
+                              type: boolean
+                            scopes:
+                              items:
+                                type: string
+                              nullable: true
+                              type: array
+                            token_url:
+                              type: string
+                            userinfo_url:
+                              nullable: true
+                              type: string
+                          required:
+                          - auth_url
+                          - token_url
+                          type: object
+                        id:
+                          type: string
+                        login_config:
+                          description: OAuth provider endpoint configuration.
+                          nullable: true
+                          properties:
+                            auth_url:
+                              type: string
+                            extra_params:
+                              additionalProperties:
+                                type: string
+                              nullable: true
+                              type: object
+                            extra_params_callback:
+                              additionalProperties:
+                                type: string
+                              nullable: true
+                              type: object
+                            req_body_auth:
+                              nullable: true
+                              type: boolean
+                            scopes:
+                              items:
+                                type: string
+                              nullable: true
+                              type: array
+                            token_url:
+                              type: string
+                            userinfo_url:
+                              nullable: true
+                              type: string
+                          required:
+                          - auth_url
+                          - token_url
+                          type: object
+                        secret:
+                          type: string
+                      required:
+                      - id
+                      - secret
+                      type: object
+                    nullable: true
+                    type: object
+                  object_store_cache_config:
+                    nullable: true
+                  openai_azure_base_path:
+                    nullable: true
+                    type: string
+                  otel:
+                    description: OpenTelemetry exporter configuration.
+                    nullable: true
+                    properties:
+                      logs_enabled:
+                        nullable: true
+                        type: boolean
+                      metrics_enabled:
+                        nullable: true
+                        type: boolean
+                      otel_exporter_otlp_compression:
+                        nullable: true
+                        type: string
+                      otel_exporter_otlp_endpoint:
+                        nullable: true
+                        type: string
+                      otel_exporter_otlp_headers:
+                        nullable: true
+                        type: string
+                      otel_exporter_otlp_protocol:
+                        nullable: true
+                        type: string
+                      tracing_enabled:
+                        nullable: true
+                        type: boolean
+                    type: object
+                  otel_tracing_proxy:
+                    description: Per-language HTTP request tracing proxy configuration.
+                    nullable: true
+                    properties:
+                      enabled:
+                        default: false
+                        type: boolean
+                      enabled_languages:
+                        items:
+                          description: Script language identifier.
+                          enum:
+                          - python3
+                          - deno
+                          - go
+                          - bash
+                          - powershell
+                          - postgresql
+                          - bun
+                          - bunnative
+                          - mysql
+                          - bigquery
+                          - snowflake
+                          - graphql
+                          - nativets
+                          - mssql
+                          - oracledb
+                          - duckdb
+                          - php
+                          - rust
+                          - ansible
+                          - csharp
+                          - nu
+                          - java
+                          - ruby
+                          type: string
+                        type: array
+                    type: object
+                  pip_extra_index_url:
+                    nullable: true
+                    type: string
+                  pip_index_url:
+                    nullable: true
+                    type: string
+                  powershell_repo_pat:
+                    nullable: true
+                    type: string
+                  powershell_repo_url:
+                    nullable: true
+                    type: string
+                  request_size_limit_mb:
+                    format: int64
+                    nullable: true
+                    type: integer
+                  require_preexisting_user_for_oauth:
+                    nullable: true
+                    type: boolean
+                  retention_period_secs:
+                    format: int64
+                    nullable: true
+                    type: integer
+                  ruby_repos:
+                    nullable: true
+                    type: string
+                  saml_metadata:
+                    nullable: true
+                    type: string
+                  scim_token:
+                    nullable: true
+                    type: string
+                  secret_backend:
+                    nullable: true
+                  slack:
+                    nullable: true
+                  smtp_settings:
+                    description: SMTP server configuration.
+                    nullable: true
+                    properties:
+                      smtp_disable_tls:
+                        nullable: true
+                        type: boolean
+                      smtp_from:
+                        nullable: true
+                        type: string
+                      smtp_host:
+                        nullable: true
+                        type: string
+                      smtp_password:
+                        nullable: true
+                        type: string
+                      smtp_port:
+                        format: uint16
+                        minimum: 0.0
+                        nullable: true
+                        type: integer
+                      smtp_tls_implicit:
+                        nullable: true
+                        type: boolean
+                      smtp_username:
+                        nullable: true
+                        type: string
+                    type: object
+                  teams:
+                    nullable: true
+                  timeout_wait_result:
+                    format: int64
+                    nullable: true
+                    type: integer
+                type: object
+                x-kubernetes-preserve-unknown-fields: true
+              worker_configs:
+                additionalProperties:
+                  description: Worker group configuration.
+                  properties:
+                    additional_python_paths:
+                      items:
+                        type: string
+                      nullable: true
+                      type: array
+                    autoscaling:
+                      description: Worker group autoscaling configuration.
+                      nullable: true
+                      properties:
+                        cooldown_seconds:
+                          format: uint64
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        custom_tags:
+                          items:
+                            type: string
+                          nullable: true
+                          type: array
+                        dec_scale_occupancy_rate:
+                          format: uint8
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        enabled:
+                          default: false
+                          type: boolean
+                        full_scale_cooldown_seconds:
+                          format: uint64
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        full_scale_jobs_waiting:
+                          format: uint64
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        inc_num_workers:
+                          format: uint32
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        inc_scale_num_jobs_waiting:
+                          format: uint64
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        inc_scale_occupancy_rate:
+                          format: uint8
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        integration:
+                          description: |-
+                            Autoscaling integration backend.
+
+                            The `type` field selects the backend: `"script"`, `"dryrun"`, or `"kubernetes"`. For `"script"`, `path` is required and `tag` is optional.
+                          nullable: true
+                          properties:
+                            path:
+                              nullable: true
+                              type: string
+                            tag:
+                              nullable: true
+                              type: string
+                            type:
+                              type: string
+                          required:
+                          - type
+                          type: object
+                        max_workers:
+                          format: uint32
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                        min_workers:
+                          format: uint32
+                          minimum: 0.0
+                          nullable: true
+                          type: integer
+                      type: object
+                    cache_clear:
+                      format: uint32
+                      minimum: 0.0
+                      nullable: true
+                      type: integer
+                    dedicated_worker:
+                      nullable: true
+                      type: string
+                    dedicated_workers:
+                      items:
+                        type: string
+                      nullable: true
+                      type: array
+                    env_vars_allowlist:
+                      items:
+                        type: string
+                      nullable: true
+                      type: array
+                    env_vars_static:
+                      additionalProperties:
+                        type: string
+                      nullable: true
+                      type: object
+                    init_bash:
+                      nullable: true
+                      type: string
+                    min_alive_workers_alert_threshold:
+                      format: uint32
+                      minimum: 0.0
+                      nullable: true
+                      type: integer
+                    periodic_script_bash:
+                      nullable: true
+                      type: string
+                    periodic_script_interval_seconds:
+                      format: uint64
+                      minimum: 0.0
+                      nullable: true
+                      type: integer
+                    pip_local_dependencies:
+                      items:
+                        type: string
+                      nullable: true
+                      type: array
+                    priority_tags:
+                      additionalProperties:
+                        format: uint8
+                        minimum: 0.0
+                        type: integer
+                      nullable: true
+                      type: object
+                    worker_tags:
+                      items:
+                        type: string
+                      nullable: true
+                      type: array
+                  type: object
+                  x-kubernetes-preserve-unknown-fields: true
+                default: {}
+                description: Worker group configs to sync to the `config` table. Keys are worker group names (e.g. "default", "gpu"). Each key is stored in the DB as `worker__<key>`.
+                type: object
+            type: object
+          status:
+            description: Status subresource for WindmillInstance.
+            nullable: true
+            properties:
+              lastSyncedAt:
+                description: Timestamp of the last successful sync.
+                nullable: true
+                type: string
+              message:
+                default: ''
+                description: Human-readable status message.
+                type: string
+              observedGeneration:
+                default: 0
+                description: The `.metadata.generation` that was last observed.
+                format: int64
+                type: integer
+              synced:
+                description: Whether the last reconciliation was successful.
+                type: boolean
+            required:
+            - synced
+            type: object
+        required:
+        - spec
+        title: WindmillInstance
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}
+
@@ -0,0 +1,218 @@
+use std::collections::BTreeMap;
+
+use kube::CustomResource;
+use schemars::JsonSchema;
+use serde::{Deserialize, Serialize};
+
+// Re-export all config types from windmill-common for downstream consumers.
+pub use windmill_common::instance_config::{
+    AutoscalingConfig, AutoscalingIntegration, CriticalErrorChannel, CustomInstanceDb,
+    CustomInstanceDbLogs, CustomInstancePgDatabases, DbOversizeAlert, Ducklake, DucklakeCatalog,
+    DucklakeCatalogResourceType, DucklakeSettings, DucklakeStorage, EnvRefWrapper, GlobalSettings,
+    IndexerSettings, OAuthClient, OAuthConfig, OtelSettings, OtelTracingProxySettings, ScriptLang,
+    SecretKeyRef, SecretKeyRefWrapper, SmtpSettings, StringOrSecretRef, TeamsChannel,
+    WorkerGroupConfig,
+};
+
+/// WindmillInstance CRD spec.
+///
+/// Declares the desired state for instance-level configuration:
+/// - `global_settings` maps directly to the `global_settings` table
+/// - `worker_configs` maps to the `config` table with a `worker__` prefix
+#[derive(CustomResource, Deserialize, Serialize, Clone, Debug, JsonSchema)]
+#[kube(
+    group = "windmill.dev",
+    version = "v1alpha1",
+    kind = "WindmillInstance",
+    namespaced,
+    shortname = "wmi",
+    status = "WindmillInstanceStatus",
+    printcolumn = r#"{"name":"Synced","type":"string","jsonPath":".status.synced"}"#,
+    printcolumn = r#"{"name":"Last Synced","type":"date","jsonPath":".status.lastSyncedAt"}"#,
+    printcolumn = r#"{"name":"Age","type":"date","jsonPath":".metadata.creationTimestamp"}"#
+)]
+pub struct WindmillInstanceSpec {
+    /// Global settings to sync to the `global_settings` table.
+    #[serde(default)]
+    pub global_settings: GlobalSettings,
+
+    /// Worker group configs to sync to the `config` table.
+    /// Keys are worker group names (e.g. "default", "gpu").
+    /// Each key is stored in the DB as `worker__<key>`.
+    #[serde(default)]
+    pub worker_configs: BTreeMap<String, WorkerGroupConfig>,
+}
+
+// ---------------------------------------------------------------------------
+// Status subresource
+// ---------------------------------------------------------------------------
+
+/// Status subresource for WindmillInstance.
+#[derive(Deserialize, Serialize, Clone, Debug, Default, JsonSchema)]
+#[serde(rename_all = "camelCase")]
+pub struct WindmillInstanceStatus {
+    /// Whether the last reconciliation was successful.
+    pub synced: bool,
+    /// Human-readable status message.
+    #[serde(default)]
+    pub message: String,
+    /// The `.metadata.generation` that was last observed.
+    #[serde(default)]
+    pub observed_generation: i64,
+    /// Timestamp of the last successful sync.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub last_synced_at: Option<String>,
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use kube::CustomResourceExt;
+
+    #[test]
+    fn crd_generation_produces_valid_yaml() {
+        let crd = WindmillInstance::crd();
+        let yaml = serde_yml::to_string(&crd).expect("CRD should serialize to YAML");
+        assert!(
+            yaml.contains("windmill.dev"),
+            "CRD should have group windmill.dev"
+        );
+        assert!(
+            yaml.contains("v1alpha1"),
+            "CRD should have version v1alpha1"
+        );
+        assert!(
+            yaml.contains("WindmillInstance"),
+            "CRD should have kind WindmillInstance"
+        );
+        assert!(yaml.contains("wmi"), "CRD should have shortname wmi");
+    }
+
+    #[test]
+    fn crd_metadata() {
+        let crd = WindmillInstance::crd();
+        assert_eq!(
+            crd.metadata.name.as_deref(),
+            Some("windmillinstances.windmill.dev")
+        );
+    }
+
+    #[test]
+    fn spec_deserializes_with_defaults() {
+        let json = r#"{"global_settings": {}, "worker_configs": {}}"#;
+        let spec: WindmillInstanceSpec =
+            serde_json::from_str(json).expect("Should deserialize empty spec");
+        assert!(spec.global_settings.to_settings_map().is_empty());
+        assert!(spec.worker_configs.is_empty());
+    }
+
+    #[test]
+    fn spec_deserializes_omitted_fields() {
+        let json = r#"{}"#;
+        let spec: WindmillInstanceSpec =
+            serde_json::from_str(json).expect("Should deserialize spec with missing fields");
+        assert!(spec.global_settings.to_settings_map().is_empty());
+        assert!(spec.worker_configs.is_empty());
+    }
+
+    #[test]
+    fn crd_schema_has_typed_properties() {
+        let crd = WindmillInstance::crd();
+        let yaml = serde_yml::to_string(&crd).expect("CRD should serialize to YAML");
+        assert!(
+            yaml.contains("base_url"),
+            "Schema should contain base_url property"
+        );
+        assert!(
+            yaml.contains("smtp_settings"),
+            "Schema should contain smtp_settings property"
+        );
+        assert!(
+            yaml.contains("worker_tags"),
+            "Schema should contain worker_tags property"
+        );
+        assert!(
+            yaml.contains("retention_period_secs"),
+            "Schema should contain retention_period_secs property"
+        );
+        assert!(
+            yaml.contains("otel_exporter_otlp_endpoint"),
+            "Schema should contain OTel endpoint property"
+        );
+        assert!(
+            yaml.contains("min_workers"),
+            "Schema should contain autoscaling min_workers field"
+        );
+    }
+
+    #[test]
+    fn status_serializes_with_camel_case() {
+        let status = WindmillInstanceStatus {
+            synced: true,
+            message: "OK".to_string(),
+            observed_generation: 3,
+            last_synced_at: Some("2025-01-01T00:00:00Z".to_string()),
+        };
+        let json = serde_json::to_value(&status).expect("Should serialize status");
+        assert!(json.get("lastSyncedAt").is_some(), "Should use camelCase");
+        assert!(
+            json.get("observedGeneration").is_some(),
+            "Should use camelCase"
+        );
+    }
+
+    #[test]
+    fn status_omits_null_last_synced_at() {
+        let status = WindmillInstanceStatus {
+            synced: false,
+            message: "Error".to_string(),
+            observed_generation: 1,
+            last_synced_at: None,
+        };
+        let json = serde_json::to_value(&status).expect("Should serialize status");
+        assert!(
+            json.get("lastSyncedAt").is_none(),
+            "Should omit null lastSyncedAt"
+        );
+    }
+
+    #[test]
+    fn status_default() {
+        let status = WindmillInstanceStatus::default();
+        assert!(!status.synced);
+        assert!(status.message.is_empty());
+        assert_eq!(status.observed_generation, 0);
+        assert!(status.last_synced_at.is_none());
+    }
+
+    #[test]
+    fn crd_schema_supports_secret_refs() {
+        let crd = WindmillInstance::crd();
+        let yaml = serde_yml::to_string(&crd).expect("CRD should serialize to YAML");
+        assert!(
+            yaml.contains("secretKeyRef"),
+            "CRD schema should contain secretKeyRef for secret reference support"
+        );
+    }
+
+    #[test]
+    fn spec_deserializes_secret_ref_fields() {
+        let json = r#"{
+            "global_settings": {
+                "license_key": {"secretKeyRef": {"name": "wm-secrets", "key": "license"}},
+                "base_url": "https://example.com"
+            }
+        }"#;
+        let spec: WindmillInstanceSpec = serde_json::from_str(json).unwrap();
+        assert!(spec
+            .global_settings
+            .license_key
+            .as_ref()
+            .unwrap()
+            .is_secret_ref());
+        assert_eq!(
+            spec.global_settings.base_url.as_deref(),
+            Some("https://example.com")
+        );
+    }
+}
@@ -0,0 +1,52 @@
+use std::collections::BTreeMap;
+
+use sqlx::{Pool, Postgres};
+use windmill_common::instance_config::{
+    apply_configs_diff, apply_settings_diff, diff_global_settings, diff_worker_configs, ApplyMode,
+};
+
+/// Perform a full declarative sync of global settings.
+///
+/// - Upserts every key in `desired` into the `global_settings` table.
+/// - Deletes keys that exist in DB but are absent from `desired`
+///   (except protected keys).
+pub async fn sync_global_settings(
+    db: &Pool<Postgres>,
+    desired: &BTreeMap<String, serde_json::Value>,
+) -> anyhow::Result<()> {
+    // Fetch current settings from DB
+    let current_rows: Vec<(String, serde_json::Value)> =
+        sqlx::query_as("SELECT name, value FROM global_settings")
+            .fetch_all(db)
+            .await?;
+    let current: BTreeMap<String, serde_json::Value> = current_rows.into_iter().collect();
+
+    let diff = diff_global_settings(&current, desired, ApplyMode::Replace);
+    apply_settings_diff(db, &diff).await
+}
+
+/// Perform a full declarative sync of worker configs.
+///
+/// - Upserts every key in `desired` into the `config` table with the
+///   `worker__` prefix.
+/// - Deletes `worker__*` rows that exist in DB but are absent from `desired`.
+pub async fn sync_worker_configs(
+    db: &Pool<Postgres>,
+    desired: &BTreeMap<String, serde_json::Value>,
+) -> anyhow::Result<()> {
+    // Fetch current worker configs from DB (strip prefix for comparison)
+    let current_rows: Vec<(String, serde_json::Value)> =
+        sqlx::query_as("SELECT name, config FROM config WHERE name LIKE 'worker__%'")
+            .fetch_all(db)
+            .await?;
+    let current: BTreeMap<String, serde_json::Value> = current_rows
+        .into_iter()
+        .map(|(name, config)| {
+            let group = name.strip_prefix("worker__").unwrap_or(&name).to_string();
+            (group, config)
+        })
+        .collect();
+
+    let diff = diff_worker_configs(&current, desired, ApplyMode::Replace);
+    apply_configs_diff(db, &diff).await
+}
@@ -0,0 +1,13 @@
+pub mod crd;
+pub mod db_sync;
+pub mod reconciler;
+pub mod resolve;
+
+pub use reconciler::run;
+
+/// Print the CRD YAML definition to stdout.
+pub fn print_crd_yaml() {
+    use kube::CustomResourceExt;
+    let crd = crd::WindmillInstance::crd();
+    println!("{}", serde_yml::to_string(&crd).unwrap());
+}
@@ -0,0 +1,179 @@
+use std::collections::BTreeMap;
+use std::sync::Arc;
+use std::time::Duration;
+
+use futures::StreamExt;
+use kube::api::{Api, Patch, PatchParams};
+use kube::runtime::controller::Action;
+use kube::runtime::Controller;
+use kube::{Client, ResourceExt};
+use sqlx::{Pool, Postgres};
+
+use crate::crd::{WindmillInstance, WindmillInstanceStatus};
+use crate::db_sync;
+use crate::resolve;
+
+/// Shared state available to the reconciler.
+struct Context {
+    db: Pool<Postgres>,
+    client: Client,
+}
+
+/// Run the operator controller loop. Blocks until shutdown.
+pub async fn run(db: Pool<Postgres>) -> anyhow::Result<()> {
+    let client = Client::try_default().await?;
+
+    // Verify the CRD is installed by attempting to list
+    let api: Api<WindmillInstance> = Api::all(client.clone());
+    api.list(&Default::default()).await.map_err(|e| {
+        anyhow::anyhow!("Failed to list WindmillInstance CRDs. Is the CRD installed? Error: {e}")
+    })?;
+    tracing::info!("WindmillInstance CRD verified, starting controller");
+
+    let ctx = Arc::new(Context { db, client: client.clone() });
+
+    Controller::new(api, Default::default())
+        .shutdown_on_signal()
+        .run(reconcile, error_policy, ctx)
+        .for_each(|res| async move {
+            match res {
+                Ok(o) => tracing::debug!("Reconciled: {:?}", o),
+                Err(e) => tracing::error!("Reconcile error: {:?}", e),
+            }
+        })
+        .await;
+
+    tracing::info!("Operator controller shut down");
+    Ok(())
+}
+
+/// Main reconciliation logic for a single WindmillInstance resource.
+async fn reconcile(
+    instance: Arc<WindmillInstance>,
+    ctx: Arc<Context>,
+) -> Result<Action, kube::Error> {
+    let name = instance.name_any();
+    let ns = instance.namespace().unwrap_or_default();
+    tracing::info!("Reconciling WindmillInstance {name} in namespace {ns}");
+
+    let generation = instance.metadata.generation.unwrap_or(0);
+
+    // Resolve any secretKeyRef fields by reading K8s Secrets
+    let resolved = match resolve::resolve_secret_refs(
+        &ctx.client,
+        &ns,
+        &instance.spec.global_settings,
+    )
+    .await
+    {
+        Ok(gs) => gs,
+        Err(e) => {
+            tracing::error!("Failed to resolve secret refs for {name}: {e:#}");
+            update_status(
+                &ctx.client,
+                &instance,
+                false,
+                format!("Error resolving secret references: {e}"),
+                generation,
+            )
+            .await?;
+            return Ok(Action::requeue(Duration::from_secs(30)));
+        }
+    };
+
+    // Convert typed structs to BTreeMaps for db_sync
+    let settings_map = resolved.to_settings_map();
+    let configs_map: BTreeMap<String, serde_json::Value> = instance
+        .spec
+        .worker_configs
+        .iter()
+        .map(|(k, v)| {
+            (
+                k.clone(),
+                serde_json::to_value(v).expect("WorkerGroupConfig serialization cannot fail"),
+            )
+        })
+        .collect();
+
+    // Sync global settings
+    if let Err(e) = db_sync::sync_global_settings(&ctx.db, &settings_map).await {
+        tracing::error!("Failed to sync global settings for {name}: {e:#}");
+        update_status(
+            &ctx.client,
+            &instance,
+            false,
+            format!("Error syncing global settings: {e}"),
+            generation,
+        )
+        .await?;
+        // Requeue after 30s on error
+        return Ok(Action::requeue(Duration::from_secs(30)));
+    }
+
+    // Sync worker configs
+    if let Err(e) = db_sync::sync_worker_configs(&ctx.db, &configs_map).await {
+        tracing::error!("Failed to sync worker configs for {name}: {e:#}");
+        update_status(
+            &ctx.client,
+            &instance,
+            false,
+            format!("Error syncing worker configs: {e}"),
+            generation,
+        )
+        .await?;
+        return Ok(Action::requeue(Duration::from_secs(30)));
+    }
+
+    tracing::info!("Successfully synced WindmillInstance {name}");
+    update_status(
+        &ctx.client,
+        &instance,
+        true,
+        "Synced successfully".to_string(),
+        generation,
+    )
+    .await?;
+
+    // Periodic re-sync every 5 minutes for drift detection
+    Ok(Action::requeue(Duration::from_secs(300)))
+}
+
+/// Error policy: requeue after 60 seconds on unhandled errors.
+fn error_policy(
+    _instance: Arc<WindmillInstance>,
+    _error: &kube::Error,
+    _ctx: Arc<Context>,
+) -> Action {
+    Action::requeue(Duration::from_secs(60))
+}
+
+/// Patch the status subresource of the WindmillInstance.
+async fn update_status(
+    client: &Client,
+    instance: &WindmillInstance,
+    synced: bool,
+    message: String,
+    observed_generation: i64,
+) -> Result<(), kube::Error> {
+    let name = instance.name_any();
+    let ns = instance.namespace().unwrap_or_default();
+    let api: Api<WindmillInstance> = Api::namespaced(client.clone(), &ns);
+
+    let now = chrono::Utc::now().to_rfc3339();
+    let status = WindmillInstanceStatus {
+        synced,
+        message,
+        observed_generation,
+        last_synced_at: if synced { Some(now) } else { None },
+    };
+
+    let patch = serde_json::json!({ "status": status });
+    api.patch_status(
+        &name,
+        &PatchParams::apply("windmill-operator"),
+        &Patch::Merge(&patch),
+    )
+    .await?;
+
+    Ok(())
+}
@@ -0,0 +1,126 @@
+use std::collections::BTreeMap;
+
+use k8s_openapi::api::core::v1::Secret;
+use kube::api::Api;
+use kube::Client;
+use windmill_common::instance_config::{GlobalSettings, SecretKeyRef, StringOrSecretRef};
+
+#[derive(Debug, thiserror::Error)]
+pub enum ResolveError {
+    #[error("failed to fetch Kubernetes Secret '{name}': {source}")]
+    FetchSecret { name: String, source: kube::Error },
+    #[error("key '{key}' not found in Secret '{secret}'")]
+    KeyNotFound { secret: String, key: String },
+    #[error("value for key '{key}' in Secret '{secret}' is not valid UTF-8")]
+    InvalidUtf8 { secret: String, key: String },
+    #[error("environment variable '{var}' not found")]
+    EnvVarNotFound { var: String },
+}
+
+/// Resolve all `StringOrSecretRef` fields in `GlobalSettings` by reading
+/// referenced Kubernetes Secrets. Returns a new `GlobalSettings` with every
+/// `SecretRef` replaced by its `Literal` value.
+pub async fn resolve_secret_refs(
+    client: &Client,
+    namespace: &str,
+    settings: &GlobalSettings,
+) -> Result<GlobalSettings, ResolveError> {
+    let mut settings = settings.clone();
+    let mut cache: BTreeMap<String, BTreeMap<String, String>> = BTreeMap::new();
+
+    resolve_option(client, namespace, &mut cache, &mut settings.license_key).await?;
+    resolve_option(client, namespace, &mut cache, &mut settings.hub_api_secret).await?;
+    resolve_option(client, namespace, &mut cache, &mut settings.scim_token).await?;
+
+    if let Some(smtp) = &mut settings.smtp_settings {
+        resolve_option(client, namespace, &mut cache, &mut smtp.smtp_password).await?;
+    }
+
+    if let Some(oauths) = &mut settings.oauths {
+        for oauth in oauths.values_mut() {
+            resolve_field(client, namespace, &mut cache, &mut oauth.secret).await?;
+        }
+    }
+
+    if let Some(pg) = &mut settings.custom_instance_pg_databases {
+        resolve_option(client, namespace, &mut cache, &mut pg.user_pwd).await?;
+    }
+
+    Ok(settings)
+}
+
+async fn resolve_option(
+    client: &Client,
+    namespace: &str,
+    cache: &mut BTreeMap<String, BTreeMap<String, String>>,
+    field: &mut Option<StringOrSecretRef>,
+) -> Result<(), ResolveError> {
+    if let Some(val) = field {
+        resolve_field(client, namespace, cache, val).await?;
+    }
+    Ok(())
+}
+
+async fn resolve_field(
+    client: &Client,
+    namespace: &str,
+    cache: &mut BTreeMap<String, BTreeMap<String, String>>,
+    field: &mut StringOrSecretRef,
+) -> Result<(), ResolveError> {
+    if let Some(var_name) = field.as_env_ref() {
+        let var_name = var_name.to_string();
+        let value =
+            std::env::var(&var_name).map_err(|_| ResolveError::EnvVarNotFound { var: var_name })?;
+        *field = StringOrSecretRef::Literal(value);
+        return Ok(());
+    }
+
+    let secret_ref = match field.as_secret_ref() {
+        Some(r) => r.clone(),
+        None => return Ok(()),
+    };
+
+    let value = fetch_secret_value(client, namespace, cache, &secret_ref).await?;
+    *field = StringOrSecretRef::Literal(value);
+    Ok(())
+}
+
+async fn fetch_secret_value(
+    client: &Client,
+    namespace: &str,
+    cache: &mut BTreeMap<String, BTreeMap<String, String>>,
+    secret_ref: &SecretKeyRef,
+) -> Result<String, ResolveError> {
+    if let Some(data) = cache.get(&secret_ref.name) {
+        return data
+            .get(&secret_ref.key)
+            .cloned()
+            .ok_or_else(|| ResolveError::KeyNotFound {
+                secret: secret_ref.name.clone(),
+                key: secret_ref.key.clone(),
+            });
+    }
+
+    let api: Api<Secret> = Api::namespaced(client.clone(), namespace);
+    let secret = api
+        .get(&secret_ref.name)
+        .await
+        .map_err(|e| ResolveError::FetchSecret { name: secret_ref.name.clone(), source: e })?;
+
+    let data = secret.data.unwrap_or_default();
+    let decoded: BTreeMap<String, String> = data
+        .into_iter()
+        .filter_map(|(k, v)| String::from_utf8(v.0).ok().map(|s| (k, s)))
+        .collect();
+
+    let value = decoded
+        .get(&secret_ref.key)
+        .cloned()
+        .ok_or_else(|| ResolveError::KeyNotFound {
+            secret: secret_ref.name.clone(),
+            key: secret_ref.key.clone(),
+        })?;
+
+    cache.insert(secret_ref.name.clone(), decoded);
+    Ok(value)
+}
@@ -652,6 +652,18 @@ export async function getActiveInstance(opts: {
  }
 }

+async function getConfig(opts: InstanceSyncOptions & { outputFile?: string }) {
+  await pickInstance(opts, false);
+  const config = await wmill.getInstanceConfig();
+  const yaml = yamlStringify(config as Record<string, unknown>);
+  if (opts.outputFile) {
+    await Deno.writeTextFile(opts.outputFile, yaml);
+    log.info(colors.green(`Instance config written to ${opts.outputFile}`));
+  } else {
+    console.log(yaml);
+  }
+}
+
 async function whoami(opts: {}) {
  await pickInstance({}, false);
  try {
@@ -774,6 +786,14 @@ const command = new Command()
  .action(instancePush as any)
  .command("whoami")
  .description("Display information about the currently logged-in user")
-  .action(whoami as any);
+  .action(whoami as any)
+  .command("get-config")
+  .description("Dump the current instance config (global settings + worker configs) as YAML")
+  .option("-o, --output-file <file:string>", "Write YAML to a file instead of stdout")
+  .option(
+    "--instance <instance:string>",
+    "Name of the instance, override the active instance",
+  )
+  .action(getConfig as any);

 export default command;
@@ -0,0 +1,6 @@
+DATABASE_URL=postgres://postgres:changeme@db/windmill
+WM_IMAGE=ghcr.io/windmill-labs/windmill-ee:main
+
+# Secrets referenced via envRef in windmill-config.yaml
+WM_LICENSE_KEY=your-license-key-here
+SMTP_PASSWORD=your-smtp-password-here
@@ -0,0 +1,348 @@
+# Windmill Instance Configuration as Code
+
+Windmill supports managing instance configuration (global settings + worker group configs) declaratively through YAML files. This enables Infrastructure-as-Code (IaC) workflows where your Windmill instance settings are version-controlled and applied automatically.
+
+Two deployment models are supported:
+
+| Approach | Best for | Requires |
+|---|---|---|
+| **`sync-config`** CLI | Docker Compose, VMs, CI/CD pipelines | Database access |
+| **Kubernetes Operator** | Kubernetes clusters | `operator` feature flag, RBAC |
+
+Both use the same YAML schema (`InstanceConfig`) and the same secret reference mechanisms.
+
+---
+
+## Config File Reference
+
+A Windmill instance config file has two top-level keys:
+
+```yaml
+global_settings:
+  # Instance-wide settings (stored in the global_settings table)
+  base_url: "https://windmill.example.com"
+  retention_period_secs: 2592000
+  # ...
+
+worker_configs:
+  # Worker group configurations (stored in the config table as worker__<name>)
+  default:
+    worker_tags: ["deno", "python3", "bun", "go", "bash"]
+  gpu:
+    dedicated_worker: "ws:f/gpu_inference"
+    # ...
+```
+
+All fields are optional. Only the fields you specify are synced to the database.
+
+### Sensitive Field References
+
+Fields that contain secrets (license keys, OAuth secrets, SMTP passwords, etc.) support three formats:
+
+```yaml
+# 1. Plain literal (not recommended for production)
+license_key: "my-license-key"
+
+# 2. Environment variable reference (works everywhere)
+license_key:
+  envRef: "WM_LICENSE_KEY"
+
+# 3. Kubernetes Secret reference (K8s only)
+license_key:
+  secretKeyRef:
+    name: windmill-secrets    # Secret resource name
+    key: license-key          # Key within the Secret
+```
+
+Fields that support `envRef` and `secretKeyRef`:
+
+- `license_key`
+- `hub_api_secret`
+- `scim_token`
+- `smtp_settings.smtp_password`
+- `oauths.<provider>.secret` (each OAuth client secret)
+- `custom_instance_pg_databases.user_pwd`
+
+---
+
+## Docker Compose (`sync-config`)
+
+The `sync-config` subcommand reads a YAML config file, resolves any `envRef` references from the process environment, and syncs the result to the database.
+
+### How it works
+
+1. Windmill reads and parses the YAML file
+2. Any `envRef` fields are resolved from the container's environment variables
+3. The current database state is read
+4. A diff is computed (using `Replace` mode: settings absent from the file are deleted, except protected ones like `ducklake_settings`)
+5. Changes are applied to the database
+
+### Setup
+
+See the included [`docker-compose.yml`](docker-compose.yml) for a complete working example. The key parts:
+
+**1. Create your config file** (`windmill-config.yaml`):
+
+```yaml
+global_settings:
+  base_url: "https://windmill.example.com"
+  license_key:
+    envRef: "WM_LICENSE_KEY"
+  retention_period_secs: 2592000
+  expose_metrics: true
+  smtp_settings:
+    smtp_host: "smtp.example.com"
+    smtp_port: 587
+    smtp_from: "windmill@example.com"
+    smtp_password:
+      envRef: "SMTP_PASSWORD"
+  oauths:
+    google:
+      id: "google-client-id"
+      secret:
+        envRef: "GOOGLE_OAUTH_SECRET"
+      login_config:
+        auth_url: "https://accounts.google.com/o/oauth2/v2/auth"
+        token_url: "https://oauth2.googleapis.com/token"
+        userinfo_url: "https://openidconnect.googleapis.com/v1/userinfo"
+        scopes: ["openid", "profile", "email"]
+  custom_tags:
+    - gpu
+    - high-mem
+
+worker_configs:
+  default:
+    worker_tags: ["deno", "python3", "bun", "go", "bash", "powershell"]
+    init_bash: "echo 'Worker starting'"
+  native:
+    worker_tags: ["nativets"]
+```
+
+**2. Add an init container to `docker-compose.yml`**:
+
+```yaml
+services:
+  windmill_config_sync:
+    image: ${WM_IMAGE}
+    # Run once at startup then exit
+    restart: "no"
+    command: ["windmill", "sync-config", "/config/windmill-config.yaml"]
+    environment:
+      - DATABASE_URL=${DATABASE_URL}
+      - WM_LICENSE_KEY=${WM_LICENSE_KEY}
+      - SMTP_PASSWORD=${SMTP_PASSWORD}
+      - GOOGLE_OAUTH_SECRET=${GOOGLE_OAUTH_SECRET}
+    volumes:
+      - ./windmill-config.yaml:/config/windmill-config.yaml:ro
+    depends_on:
+      db:
+        condition: service_healthy
+```
+
+**3. Set secrets in your `.env` file** (not committed to version control):
+
+```env
+DATABASE_URL=postgres://postgres:changeme@db/windmill
+WM_IMAGE=ghcr.io/windmill-labs/windmill-ee:main
+WM_LICENSE_KEY=your-license-key-here
+SMTP_PASSWORD=your-smtp-password
+GOOGLE_OAUTH_SECRET=your-google-oauth-secret
+```
+
+### Re-syncing after config changes
+
+The `sync-config` container runs once and exits. To re-apply after editing the YAML:
+
+```bash
+docker compose run --rm windmill_config_sync
+```
+
+Or, for CI/CD pipelines, run the binary directly:
+
+```bash
+windmill sync-config ./windmill-config.yaml
+```
+
+### Replace semantics
+
+`sync-config` uses **Replace** mode: any global setting present in the database but absent from your YAML file will be **deleted** (except protected settings like `ducklake_settings` and `custom_instance_pg_databases`). This ensures the database state matches the file exactly.
+
+If you only want to manage a subset of settings, include all settings you want to keep in the YAML file.
+
+---
+
+## Kubernetes (Operator)
+
+The Windmill Kubernetes operator watches `WindmillInstance` Custom Resources and continuously reconciles the database to match the declared state. It also supports `secretKeyRef` to pull values from Kubernetes Secrets natively.
+
+### Prerequisites
+
+- Windmill built with the `operator` feature flag
+- RBAC permissions for the operator pod (see below)
+- The CRD installed in the cluster
+
+### Setup
+
+**1. Install the CRD**:
+
+```bash
+windmill operator crd | kubectl apply -f -
+```
+
+**2. Create a Kubernetes Secret for sensitive values**:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: windmill-secrets
+  namespace: windmill
+type: Opaque
+stringData:
+  license-key: "your-license-key-here"
+  smtp-password: "your-smtp-password"
+  google-oauth-secret: "your-google-oauth-secret"
+```
+
+**3. Create the WindmillInstance resource** (`windmill-instance.yaml`):
+
+```yaml
+apiVersion: windmill.dev/v1alpha1
+kind: WindmillInstance
+metadata:
+  name: production
+  namespace: windmill
+spec:
+  global_settings:
+    base_url: "https://windmill.example.com"
+    license_key:
+      secretKeyRef:
+        name: windmill-secrets
+        key: license-key
+    retention_period_secs: 2592000
+    expose_metrics: true
+    smtp_settings:
+      smtp_host: "smtp.example.com"
+      smtp_port: 587
+      smtp_from: "windmill@example.com"
+      smtp_password:
+        secretKeyRef:
+          name: windmill-secrets
+          key: smtp-password
+    oauths:
+      google:
+        id: "google-client-id"
+        secret:
+          secretKeyRef:
+            name: windmill-secrets
+            key: google-oauth-secret
+        login_config:
+          auth_url: "https://accounts.google.com/o/oauth2/v2/auth"
+          token_url: "https://oauth2.googleapis.com/token"
+          userinfo_url: "https://openidconnect.googleapis.com/v1/userinfo"
+          scopes: ["openid", "profile", "email"]
+    custom_tags:
+      - gpu
+      - high-mem
+
+  worker_configs:
+    default:
+      worker_tags: ["deno", "python3", "bun", "go", "bash", "powershell"]
+      init_bash: "echo 'Worker starting'"
+    native:
+      worker_tags: ["nativets"]
+```
+
+**4. Apply**:
+
+```bash
+kubectl apply -f windmill-instance.yaml
+```
+
+**5. Check status**:
+
+```bash
+kubectl get wmi
+# NAME         SYNCED   LAST SYNCED              AGE
+# production   true     2025-01-15T10:30:00Z     2d
+```
+
+### Using `envRef` in Kubernetes
+
+`envRef` also works in the operator context. Values are resolved from the operator pod's environment. This is useful when secrets are injected via pod env vars (e.g., from a vault sidecar):
+
+```yaml
+spec:
+  global_settings:
+    license_key:
+      envRef: "WM_LICENSE_KEY"   # Read from operator pod env
+```
+
+The operator pod's Deployment would include:
+
+```yaml
+env:
+  - name: WM_LICENSE_KEY
+    valueFrom:
+      secretKeyRef:
+        name: windmill-secrets
+        key: license-key
+```
+
+This is functionally equivalent to using `secretKeyRef` directly in the CRD, but lets you use any secret injection mechanism your cluster supports (external-secrets, vault-agent, etc.).
+
+### RBAC
+
+The operator pod needs permissions to read Secrets and manage the CRD. Minimal ClusterRole:
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: windmill-operator
+rules:
+  - apiGroups: ["windmill.dev"]
+    resources: ["windmillinstances", "windmillinstances/status"]
+    verbs: ["get", "list", "watch", "patch", "update"]
+  - apiGroups: [""]
+    resources: ["secrets"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: [""]
+    resources: ["events"]
+    verbs: ["create", "patch"]
+```
+
+### Running the operator
+
+```bash
+# As a standalone process (for development)
+DATABASE_URL=postgres://... windmill operator
+
+# In production, deploy as a Kubernetes Deployment
+```
+
+---
+
+## Choosing Between `envRef` and `secretKeyRef`
+
+| Feature | `envRef` | `secretKeyRef` |
+|---|---|---|
+| Works in Docker Compose | Yes | No |
+| Works in Kubernetes | Yes | Yes |
+| Works with vault sidecars | Yes | No (use `envRef` instead) |
+| Reads from | Process environment | K8s Secrets API |
+| Requires RBAC for Secrets | No | Yes |
+
+**Recommendation**: Use `envRef` for portability across deployment targets. Use `secretKeyRef` when you want direct Kubernetes-native secret binding without intermediate env vars.
+
+---
+
+## Full Settings Reference
+
+For a complete list of available settings fields, generate the CRD schema:
+
+```bash
+windmill operator crd
+```
+
+The CRD's OpenAPI schema documents every field, its type, and whether it's optional. The same schema applies to `sync-config` YAML files.
@@ -0,0 +1,90 @@
+version: "3.7"
+
+# Example: Windmill with declarative instance configuration via sync-config.
+#
+# Usage:
+#   1. Copy .env.example to .env and fill in your secrets
+#   2. Edit windmill-config.yaml with your desired settings
+#   3. docker compose up -d
+#
+# The windmill_config_sync service runs once at startup, applies the YAML
+# config to the database, then exits. To re-apply after editing the config:
+#   docker compose run --rm windmill_config_sync
+
+services:
+  db:
+    image: postgres:16
+    shm_size: 1g
+    restart: unless-stopped
+    volumes:
+      - db_data:/var/lib/postgresql/data
+    expose:
+      - 5432
+    environment:
+      POSTGRES_PASSWORD: changeme
+      POSTGRES_DB: windmill
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U postgres"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+  # One-shot init container that syncs the YAML config to the database.
+  # Resolves envRef fields from container environment, then exits.
+  windmill_config_sync:
+    image: ${WM_IMAGE}
+    restart: "no"
+    command: ["windmill", "sync-config", "/config/windmill-config.yaml"]
+    environment:
+      - DATABASE_URL=${DATABASE_URL}
+      # Secrets are passed as env vars, referenced via envRef in the YAML
+      - WM_LICENSE_KEY=${WM_LICENSE_KEY}
+      - SMTP_PASSWORD=${SMTP_PASSWORD}
+    volumes:
+      - ./windmill-config.yaml:/config/windmill-config.yaml:ro
+    depends_on:
+      db:
+        condition: service_healthy
+
+  windmill_server:
+    image: ${WM_IMAGE}
+    restart: unless-stopped
+    expose:
+      - 8000
+    environment:
+      - DATABASE_URL=${DATABASE_URL}
+      - MODE=server
+    depends_on:
+      db:
+        condition: service_healthy
+      windmill_config_sync:
+        condition: service_completed_successfully
+
+  windmill_worker:
+    image: ${WM_IMAGE}
+    deploy:
+      replicas: 2
+    restart: unless-stopped
+    environment:
+      - DATABASE_URL=${DATABASE_URL}
+      - MODE=worker
+      - WORKER_GROUP=default
+    depends_on:
+      db:
+        condition: service_healthy
+
+  windmill_worker_native:
+    image: ${WM_IMAGE}
+    restart: unless-stopped
+    environment:
+      - DATABASE_URL=${DATABASE_URL}
+      - MODE=worker
+      - WORKER_GROUP=native
+      - NUM_WORKERS=8
+      - SLEEP_QUEUE=200
+    depends_on:
+      db:
+        condition: service_healthy
+
+volumes:
+  db_data: null
@@ -0,0 +1,15 @@
+# Example: Kubernetes Secret for WindmillInstance secret references.
+#
+# In production, manage this via sealed-secrets, external-secrets, or your
+# preferred secret management solution.
+
+apiVersion: v1
+kind: Secret
+metadata:
+  name: windmill-secrets
+  namespace: windmill
+type: Opaque
+stringData:
+  license-key: "your-license-key-here"
+  smtp-password: "your-smtp-password-here"
+  google-oauth-secret: "your-google-oauth-secret-here"
@@ -0,0 +1,67 @@
+# Example: WindmillInstance CRD for the Kubernetes operator.
+#
+# Prerequisites:
+#   1. Install the CRD:  windmill operator crd | kubectl apply -f -
+#   2. Create the Secret: kubectl apply -f k8s-secrets.yaml
+#   3. Apply this file:   kubectl apply -f k8s-windmill-instance.yaml
+#   4. Check status:      kubectl get wmi
+
+apiVersion: windmill.dev/v1alpha1
+kind: WindmillInstance
+metadata:
+  name: production
+  namespace: windmill
+spec:
+  global_settings:
+    base_url: "https://windmill.example.com"
+
+    # Secret reference: reads "license-key" from K8s Secret "windmill-secrets"
+    license_key:
+      secretKeyRef:
+        name: windmill-secrets
+        key: license-key
+
+    retention_period_secs: 2592000
+    job_default_timeout: 900
+    expose_metrics: true
+
+    smtp_settings:
+      smtp_host: "smtp.example.com"
+      smtp_port: 587
+      smtp_from: "windmill@example.com"
+      smtp_tls_implicit: false
+      smtp_password:
+        secretKeyRef:
+          name: windmill-secrets
+          key: smtp-password
+
+    oauths:
+      google:
+        id: "your-google-client-id"
+        secret:
+          secretKeyRef:
+            name: windmill-secrets
+            key: google-oauth-secret
+        login_config:
+          auth_url: "https://accounts.google.com/o/oauth2/v2/auth"
+          token_url: "https://oauth2.googleapis.com/token"
+          userinfo_url: "https://openidconnect.googleapis.com/v1/userinfo"
+          scopes: ["openid", "profile", "email"]
+
+    custom_tags:
+      - gpu
+      - high-mem
+
+  worker_configs:
+    default:
+      worker_tags:
+        - deno
+        - python3
+        - bun
+        - go
+        - bash
+        - powershell
+
+    native:
+      worker_tags:
+        - nativets
@@ -0,0 +1,42 @@
+# Windmill Instance Configuration
+#
+# This file is applied to the database via: windmill sync-config <file>
+# Sensitive fields use envRef to read values from environment variables.
+# See README.md for full documentation.
+
+global_settings:
+  base_url: "https://windmill.example.com"
+
+  # License key pulled from WM_LICENSE_KEY env var
+  license_key:
+    envRef: "WM_LICENSE_KEY"
+
+  retention_period_secs: 2592000    # 30 days
+  job_default_timeout: 900          # 15 minutes
+  expose_metrics: false
+
+  smtp_settings:
+    smtp_host: "smtp.example.com"
+    smtp_port: 587
+    smtp_from: "windmill@example.com"
+    smtp_tls_implicit: false
+    smtp_password:
+      envRef: "SMTP_PASSWORD"
+
+  custom_tags:
+    - gpu
+    - high-mem
+
+worker_configs:
+  default:
+    worker_tags:
+      - deno
+      - python3
+      - bun
+      - go
+      - bash
+      - powershell
+
+  native:
+    worker_tags:
+      - nativets
@@ -834,7 +834,6 @@
 			"version": "1.7.1",
 			"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.7.1.tgz",
 			"integrity": "sha512-o1uhUASyo921r2XtHYOHy7gdkGLge8ghBEQHMWmyJFoXlpU58kIrhhN3w26lpQb6dspetweapMn2CSNwQ8I4wg==",
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"dependencies": {
@@ -846,7 +845,6 @@
 			"version": "1.7.1",
 			"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.7.1.tgz",
 			"integrity": "sha512-PVtJr5CmLwYAU9PZDMITZoR5iAOShYREoR45EyyLrbntV50mdePTgUn4AmOw90Ifcj+x2kRjdzr1HP3RrNiHGA==",
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"dependencies": {
@@ -857,7 +855,6 @@
 			"version": "1.1.0",
 			"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.1.0.tgz",
 			"integrity": "sha512-WI0DdZ8xFSbgMjR1sFsKABJ/C5OnRrjT06JXbZKexJGrDuPTzZdDYfFlsgcCXCyf+suG5QU2e/y1Wo2V/OapLQ==",
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"dependencies": {
@@ -1347,7 +1344,6 @@
 			"version": "1.1.0",
 			"resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.0.tgz",
 			"integrity": "sha512-Fq6DJW+Bb5jaWE69/qOE0D1TUN9+6uWhCeZpdnSBk14pjLcCWR7Q8n49PTSPHazM37JqrsdpEthXy2xn6jWWiA==",
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"dependencies": {
@@ -1502,7 +1498,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1519,7 +1514,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1536,7 +1530,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1553,7 +1546,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1570,7 +1562,6 @@
 			"cpu": [
 				"arm"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1587,7 +1578,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1604,7 +1594,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1621,7 +1610,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1638,7 +1626,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1655,7 +1642,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1672,7 +1658,6 @@
 			"cpu": [
 				"wasm32"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"dependencies": {
@@ -1689,7 +1674,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -1706,7 +1690,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"os": [
@@ -2327,7 +2310,6 @@
 			"version": "0.10.1",
 			"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.1.tgz",
 			"integrity": "sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==",
-			"dev": true,
 			"license": "MIT",
 			"optional": true,
 			"dependencies": {
@@ -7076,7 +7058,7 @@
 			"version": "1.21.7",
 			"resolved": "https://registry.npmjs.org/jiti/-/jiti-1.21.7.tgz",
 			"integrity": "sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A==",
-			"dev": true,
+			"devOptional": true,
 			"license": "MIT",
 			"bin": {
 				"jiti": "bin/jiti.js"
@@ -7575,7 +7557,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7596,7 +7577,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7617,7 +7597,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7638,7 +7617,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7659,7 +7637,6 @@
 			"cpu": [
 				"arm"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7680,7 +7657,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7701,7 +7677,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7722,7 +7697,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7743,7 +7717,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7764,7 +7737,6 @@
 			"cpu": [
 				"arm64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -7785,7 +7757,6 @@
 			"cpu": [
 				"x64"
 			],
-			"dev": true,
 			"license": "MPL-2.0",
 			"optional": true,
 			"os": [
@@ -12412,21 +12383,6 @@
 				}
 			}
 		},
-		"node_modules/svelte-check/node_modules/picomatch": {
-			"version": "4.0.3",
-			"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
-			"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
-			"dev": true,
-			"license": "MIT",
-			"optional": true,
-			"peer": true,
-			"engines": {
-				"node": ">=12"
-			},
-			"funding": {
-				"url": "https://github.com/sponsors/jonschlinkert"
-			}
-		},
 		"node_modules/svelte-eslint-parser": {
 			"version": "0.43.0",
 			"resolved": "https://registry.npmjs.org/svelte-eslint-parser/-/svelte-eslint-parser-0.43.0.tgz",
@@ -33,6 +33,7 @@
 		readOnly?: boolean
 		buttons?: ButtonProp[]
 		modifiedModel?: meditor.ITextModel | meditor.IEditorModel
+		inlineDiff?: boolean
 	}

 	let {
@@ -46,7 +47,8 @@
 		defaultModified = undefined,
 		readOnly = false,
 		buttons = [],
-		modifiedModel
+		modifiedModel,
+		inlineDiff = false
 	}: Props = $props()

 	let diffEditor: meditor.IStandaloneDiffEditor | undefined = $state(undefined)
@@ -62,7 +64,7 @@

 		diffEditor = meditor.createDiffEditor(diffDivEl!, {
 			automaticLayout,
-			renderSideBySide: editorWidth >= SIDE_BY_SIDE_MIN_WIDTH,
+			renderSideBySide: inlineDiff ? false : editorWidth >= SIDE_BY_SIDE_MIN_WIDTH,
 			originalEditable: false,
 			readOnly,
 			minimap: {
@@ -168,7 +170,9 @@
 	}

 	function onWidthChange(editorWidth: number) {
-		diffEditor?.updateOptions({ renderSideBySide: editorWidth >= SIDE_BY_SIDE_MIN_WIDTH })
+		diffEditor?.updateOptions({
+			renderSideBySide: inlineDiff ? false : editorWidth >= SIDE_BY_SIDE_MIN_WIDTH
+		})
 	}

 	$effect(() => {
@@ -158,10 +158,13 @@
 	$effect(() => {
 		if (
 			(setting.fieldType == 'select' || setting.fieldType == 'select_python') &&
-			$values[setting.key] == undefined
+			$values[setting.key] == undefined &&
+			setting.defaultValue
 		) {
 			untrack(() => {
-				$values[setting.key] = setting.defaultValue ? setting.defaultValue() : 'default'
+				if (setting.defaultValue) {
+					$values[setting.key] = setting.defaultValue()
+				}
 			})
 		}
 	})
@@ -449,7 +452,8 @@
 							<div class="flex flex-row gap-1 items-center">
 								<Info size={12} class="text-blue-600" />
 								<span class="text-blue-600 dark:text-blue-400 text-xs">
-									License key is set but the current image is Community Edition ({version}). Switch to the EE image to finalize the upgrade.
+									License key is set but the current image is Community Edition ({version}). Switch
+									to the EE image to finalize the upgrade.
 								</span>
 							</div>
 						{/if}
@@ -1,5 +1,5 @@
 <script lang="ts">
-	import { scimSamlSetting, settings, settingsKeys, type SettingStorage } from './instanceSettings'
+	import { scimSamlSetting, settings, settingsKeys } from './instanceSettings'
 	import { Alert, Button, Tab, TabContent, Tabs } from '$lib/components/common'
 	import { SettingService, SettingsService } from '$lib/gen'
 	import type { TeamsChannel } from '$lib/gen/types.gen'
@@ -15,6 +15,10 @@
 	import AuthSettings from './AuthSettings.svelte'
 	import InstanceSetting from './InstanceSetting.svelte'
 	import { writable, type Writable } from 'svelte/store'
+	import { Loader2 } from 'lucide-svelte'
+	import YAML from 'yaml'
+	import Toggle from './Toggle.svelte'
+	import type SimpleEditor from './SimpleEditor.svelte'
 	import SettingsFooter from './workspaceSettings/SettingsFooter.svelte'
 	import SettingsPageHeader from './settings/SettingsPageHeader.svelte'

@@ -25,6 +29,9 @@
 		authSubTab?: 'sso' | 'oauth' | 'scim'
 		onNavigateToTab?: (category: string) => void
 		quickSetup?: boolean
+		yamlMode?: boolean
+		diffMode?: boolean
+		hasUnsavedChanges?: boolean
 	}

 	let {
@@ -33,15 +40,18 @@
 		closeDrawer = () => {},
 		authSubTab = $bindable('sso'),
 		onNavigateToTab,
-		quickSetup = false
+		quickSetup = false,
+		yamlMode = $bindable(false),
+		diffMode = $bindable(false),
+		hasUnsavedChanges = $bindable(false)
 	}: Props = $props()

 	let values: Writable<Record<string, any>> = writable({})
-	let initialOauths: Record<string, any> = {}
-	let initialRequirePreexistingUserForOauth: boolean = false
+	let initialOauths: Record<string, any> = $state({})
+	let initialRequirePreexistingUserForOauth: boolean = $state(false)
 	let requirePreexistingUserForOauth: boolean = $state(false)

-	let initialValues: Record<string, any> = {}
+	let initialValues: Record<string, any> = $state({})
 	let snowflakeAccountIdentifier = $state('')
 	let version: string = $state('')
 	let loading = $state(true)
@@ -56,72 +66,46 @@
 	}
 	let oauths: Record<string, any> = $state({})

+	/** Ensure object/array-typed settings have a non-null default for the form UI */
+	const formDefaults: Record<string, any> = {
+		smtp_settings: {},
+		otel: {},
+		indexer_settings: {},
+		critical_error_channels: []
+	}
+
+	function applyFormDefaults(vals: Record<string, any>): void {
+		for (const [key, defaultVal] of Object.entries(formDefaults)) {
+			if (vals[key] == undefined) {
+				vals[key] = typeof defaultVal === 'object' ? JSON.parse(JSON.stringify(defaultVal)) : defaultVal
+			}
+		}
+	}
+
 	async function loadSettings() {
 		loading = true

-		async function getValue(key: string, storage: SettingStorage) {
-			if (storage == 'setting') {
-				return SettingService.getGlobal({ key })
-			}
-		}
-		initialOauths = (await SettingService.getGlobal({ key: 'oauths' })) ?? {}
-		requirePreexistingUserForOauth =
-			((await SettingService.getGlobal({ key: 'require_preexisting_user_for_oauth' })) as any) ??
-			false
+		// Bulk-load all settings in a single API call
+		const config = await SettingService.getInstanceConfig()
+		const gs = (config.global_settings ?? {}) as Record<string, any>
+
+		initialOauths = gs['oauths'] ?? {}
+		requirePreexistingUserForOauth = gs['require_preexisting_user_for_oauth'] ?? false
 		initialRequirePreexistingUserForOauth = requirePreexistingUserForOauth
 		oauths = JSON.parse(JSON.stringify(initialOauths))
-		initialValues = Object.fromEntries(
-			(
-				await Promise.all(
-					[...Object.values(settings), scimSamlSetting].map(
-						async (y) =>
-							await Promise.all(y.map(async (x) => [x.key, await getValue(x.key, x.storage)]))
-					)
-				)
-			).flat()
-		)
-		// Normalize null to the field type's default so inputs don't appear dirty on load
-		const allSettings = [...Object.values(settings), scimSamlSetting].flat()
-		for (const s of allSettings) {
-			if (initialValues[s.key] == null) {
-				if (s.fieldType === 'boolean') {
-					initialValues[s.key] = false
-				} else if (
-					s.fieldType === 'text' ||
-					s.fieldType === 'textarea' ||
-					s.fieldType === 'codearea' ||
-					s.fieldType === 'password'
-				) {
-					initialValues[s.key] = ''
-				} else if (s.fieldType === 'secret_backend') {
-					initialValues[s.key] = { type: 'Database' }
-				} else if (s.fieldType === 'select' || s.fieldType === 'select_python') {
-					initialValues[s.key] = s.defaultValue ? s.defaultValue() : 'default'
-				}
-			}
-		}
-		let nvalues = JSON.parse(JSON.stringify(initialValues))
+
+		let nvalues: Record<string, any> = { ...gs }
+
 		if (!nvalues['base_url']) {
 			nvalues['base_url'] = window.location.origin
 		}
 		if (nvalues['retention_period_secs'] == undefined) {
 			nvalues['retention_period_secs'] = 60 * 60 * 24 * 30
 		}
-		if (nvalues['smtp_settings'] == undefined) {
-			nvalues['smtp_settings'] = {}
-		}
-		if (nvalues['otel'] == undefined) {
-			nvalues['otel'] = {}
-		}
-		if (nvalues['indexer_settings'] == undefined) {
-			nvalues['indexer_settings'] = {}
-		}
-
-		if (nvalues['critical_error_channels'] == undefined) {
-			nvalues['critical_error_channels'] = []
-		}
+		applyFormDefaults(nvalues)

 		$values = nvalues
+		initialValues = JSON.parse(JSON.stringify($values))
 		loading = false

 		// populate snowflake account identifier from db
@@ -132,6 +116,94 @@
 		}
 	}

+	export async function saveSettings() {
+		if (yamlMode) {
+			if (!syncYamlToForm()) {
+				return
+			}
+		}
+
+		if (
+			oauths?.snowflake_oauth &&
+			oauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier !==
+				snowflakeAccountIdentifier
+		) {
+			setupSnowflakeUrls()
+		}
+
+		// Remove empty or invalid entries for critical error channels
+		$values.critical_error_channels = $values.critical_error_channels.filter((entry: any) => {
+			if (!entry || typeof entry !== 'object') return false
+			if ('teams_channel' in entry) {
+				return isValidTeamsChannel(entry.teams_channel)
+			}
+			if ('slack_channel' in entry) {
+				return typeof entry.slack_channel === 'string' && entry.slack_channel.trim() !== ''
+			}
+			if ('email' in entry) {
+				return typeof entry.email === 'string' && entry.email.trim() !== ''
+			}
+			// Unknown shape
+			return false
+		})
+
+		let shouldReloadPage = false
+		if ($values) {
+			// Trim license key before saving
+			if ($values['license_key'] && typeof $values['license_key'] === 'string') {
+				$values['license_key'] = $values['license_key'].trim()
+			}
+
+			// Check which settings require a page reload
+			const allSettings = [...Object.values(settings), scimSamlSetting].flat()
+			let licenseKeySet = false
+			for (const s of allSettings) {
+				if (s.storage === 'setting' && !deepEqual(initialValues?.[s.key], $values?.[s.key])) {
+					if (s.key === 'license_key') {
+						licenseKeySet = true
+					}
+					if (s.requiresReloadOnChange) {
+						shouldReloadPage = true
+					}
+				}
+			}
+
+			// Build the full global_settings object for the bulk PUT
+			const globalSettings: Record<string, any> = { ...$values }
+
+			// Include oauths and require_preexisting_user_for_oauth
+			if (!deepEqual(initialOauths, oauths)) {
+				globalSettings['oauths'] = oauths
+			}
+			if (initialRequirePreexistingUserForOauth !== requirePreexistingUserForOauth) {
+				globalSettings['require_preexisting_user_for_oauth'] = requirePreexistingUserForOauth
+			}
+
+			await SettingService.setInstanceConfig({
+				requestBody: { global_settings: globalSettings },
+				skipWorkerConfigs: true
+			})
+
+			initialValues = JSON.parse(JSON.stringify($values))
+			initialOauths = JSON.parse(JSON.stringify(oauths))
+			initialRequirePreexistingUserForOauth = requirePreexistingUserForOauth
+
+			if (licenseKeySet) {
+				setLicense()
+			}
+		} else {
+			console.error('Values not loaded')
+		}
+		if (shouldReloadPage) {
+			sendUserToast('Settings updated, reloading page...')
+			await sleep(1000)
+			window.location.reload()
+		} else {
+			sendUserToast('Settings updated')
+			dispatch('saved')
+		}
+	}
+
 	function setupSnowflakeUrls() {
 		// strip all whitespaces from account identifier
 		snowflakeAccountIdentifier = snowflakeAccountIdentifier.replace(/\s/g, '')
@@ -205,19 +277,14 @@
 		}
 	}

-	// --- Per-category dirty state tracking ---
-
-	// Trigger to force re-derivation when initialValues changes (after save/load)
-	let dirtyCheckTrigger = $state(0)
+	// --- Dirty state tracking (YAML-based) ---

 	function stripEmpty(obj: Record<string, any>): Record<string, any> {
 		return Object.fromEntries(
 			Object.entries(obj)
 				.filter(([_, v]) => v !== undefined && v !== '')
 				.map(([k, v]) =>
-					v != null && typeof v === 'object' && !Array.isArray(v)
-						? [k, stripEmpty(v)]
-						: [k, v]
+					v != null && typeof v === 'object' && !Array.isArray(v) ? [k, stripEmpty(v)] : [k, v]
 				)
 		)
 	}
@@ -234,31 +301,85 @@
 			const jobSettings = settings['Jobs'] ?? []
 			const jobIsolation = jobSettings.find((s) => s.key === 'job_isolation')
 			const retentionPeriod = jobSettings.find((s) => s.key === 'retention_period_secs')
-			const objectStorage = settings['Object Storage']?.find((s) => s.key === 'object_store_cache_config')
-			return [...baseWithout, ...(jobIsolation ? [jobIsolation] : []), ...(licenseKey ? [licenseKey] : []), ...(retentionPeriod ? [retentionPeriod] : []), ...(objectStorage ? [objectStorage] : [])]
+			const objectStorage = settings['Object Storage']?.find(
+				(s) => s.key === 'object_store_cache_config'
+			)
+			return [
+				...baseWithout,
+				...(jobIsolation ? [jobIsolation] : []),
+				...(licenseKey ? [licenseKey] : []),
+				...(retentionPeriod ? [retentionPeriod] : []),
+				...(objectStorage ? [objectStorage] : [])
+			]
 		}
 		return base
 	}

+	function normalizeValue(value: any, key?: string): any {
+		if (value == null) return undefined
+		if (value === false) return undefined
+		if (typeof value === 'string' && value.trim() === '') return undefined
+		if (Array.isArray(value) && value.length === 0) return undefined
+		if (typeof value === 'object' && Object.keys(value).length === 0) return undefined
+
+		// Key-specific defaults: these values are equivalent to "not set"
+		if (key === 'secret_backend') {
+			if (typeof value === 'object' && value?.type === 'Database' && Object.keys(value).length === 1) {
+				return undefined
+			}
+		}
+		if (key === 'automate_username_creation' && value === true) {
+			return undefined
+		}
+		if (key === 'critical_alerts_on_db_oversize' && typeof value === 'object') {
+			if (!value.enabled && (!value.value || value.value === 0)) {
+				return undefined
+			}
+		}
+
+		return value
+	}
+	function buildCategoryYaml(
+		category: string,
+		vals: Record<string, any>,
+		oauthsObj: Record<string, any>,
+		reqPreexisting: boolean
+	): string {
+		const categorySettings = getSettingsForCategory(category)
+		const obj: Record<string, any> = {}
+		for (const s of categorySettings) {
+			const normalized = normalizeValue(vals[s.key], s.key)
+			if (normalized !== undefined) {
+				obj[s.key] = vals[s.key]
+			}
+		}
+		if (category === 'Auth/OAuth/SAML') {
+			if (Object.keys(stripEmpty(oauthsObj)).length > 0) {
+				obj['oauths'] = oauthsObj
+			}
+			if (reqPreexisting) {
+				obj['require_preexisting_user_for_oauth'] = reqPreexisting
+			}
+		}
+		return YAML.stringify(obj)
+	}
+
 	let dirtyCategories: Record<string, boolean> = $derived.by(() => {
-		void dirtyCheckTrigger
-		const currentValues = $values
 		const result: Record<string, boolean> = {}
 		for (const category of settingsKeys) {
-			if (category === 'Auth/OAuth/SAML') {
-				const scimDirty = scimSamlSetting.some(
-					(s) => !deepEqual(initialValues[s.key], currentValues?.[s.key])
-				)
-				const oauthsDirty = !deepEqual(stripEmpty(initialOauths), stripEmpty(oauths))
-				const requirePreexistingDirty =
-					initialRequirePreexistingUserForOauth !== requirePreexistingUserForOauth
-				result[category] = scimDirty || oauthsDirty || requirePreexistingDirty
-			} else {
-				const categorySettings = getSettingsForCategory(category)
-				result[category] = categorySettings.some(
-					(s) => !deepEqual(initialValues[s.key], currentValues?.[s.key])
-				)
-			}
+			const initialYaml = buildCategoryYaml(
+				category,
+				initialValues,
+				initialOauths,
+				initialRequirePreexistingUserForOauth
+			)
+			const currentYaml = buildCategoryYaml(
+				category,
+				$values,
+				oauths,
+				requirePreexistingUserForOauth
+			)
+			result[category] = initialYaml !== currentYaml
 		}
 		return result
 	})
@@ -304,6 +425,19 @@
 		}
 	}

+	export function discardAll() {
+		// Reset all values to initial state (deep copy to avoid reference sharing)
+		$values = JSON.parse(JSON.stringify(initialValues))
+		oauths = JSON.parse(JSON.stringify(initialOauths))
+		requirePreexistingUserForOauth = initialRequirePreexistingUserForOauth
+		const account_identifier =
+			initialOauths?.snowflake_oauth?.connect_config?.extra_params?.account_identifier
+		snowflakeAccountIdentifier = account_identifier ?? ''
+		if (yamlMode) {
+			syncFormToYaml()
+		}
+	}
+
 	export async function saveCategorySettings(category: string) {
 		// Category-specific pre-processing
 		if (category === 'Auth/OAuth/SAML') {
@@ -390,9 +524,6 @@

 		if (licenseKeySet) setLicense()

-		// Force dirty state re-check
-		dirtyCheckTrigger++
-
 		if (shouldReloadPage) {
 			sendUserToast('Settings updated, reloading page...')
 			await sleep(1000)
@@ -402,11 +533,285 @@
 			dispatch('saved')
 		}
 	}
+
+	let yamlCode = $state('')
+	let yamlCodeInitial = $state('')
+	let yamlEditor: SimpleEditor | undefined = $state(undefined)
+	let yamlError = $state('')
+	let showSensitive = $state(false)
+
+	const SENSITIVE_UNCHANGED = '__SENSITIVE_AND_UNCHANGED__'
+
+	const sensitiveKeys: Set<string> = new Set(
+		[...Object.values(settings), scimSamlSetting]
+			.flatMap((s) => Object.values(s))
+			.filter((s) => s.fieldType === 'password' || s.fieldType === 'license_key')
+			.map((s) => s.key)
+	)
+
+	// Settings that should never appear in YAML export/import
+	const excludedKeys: Set<string> = new Set([
+		'custom_instance_pg_databases',
+		'ducklake_settings',
+		'ducklake_user_pg_pwd'
+	])
+
+	// Nested fields inside object-valued settings that contain secrets.
+	// Each entry maps a top-level key to its sensitive sub-field names.
+	const nestedSensitiveFields: Record<string, string[]> = {
+		smtp_settings: ['smtp_password'],
+		secret_backend: ['token'],
+		object_store_cache_config: ['secret_key', 'serviceAccountKey']
+	}
+
+	/** Returns SENSITIVE_UNCHANGED if the value is non-empty and matches the initial */
+	function maskField(current: any, initial: any): string | undefined {
+		if (current != null && current !== '' && current === initial) return SENSITIVE_UNCHANGED
+		return undefined
+	}
+
+	function maskSensitive(obj: Record<string, any>): Record<string, any> {
+		const masked: Record<string, any> = {}
+		for (const [key, value] of Object.entries(obj)) {
+			if (key === 'oauths' && typeof value === 'object' && value !== null) {
+				const maskedOauths: Record<string, any> = {}
+				for (const [provider, config] of Object.entries(value as Record<string, any>)) {
+					if (typeof config === 'object' && config !== null && 'secret' in config) {
+						const m = maskField(config.secret, initialOauths?.[provider]?.secret)
+						maskedOauths[provider] = m ? { ...config, secret: m } : config
+					} else {
+						maskedOauths[provider] = config
+					}
+				}
+				masked[key] = maskedOauths
+			} else if (key in nestedSensitiveFields && typeof value === 'object' && value !== null) {
+				const cp = { ...value }
+				const init = initialValues?.[key]
+				for (const field of nestedSensitiveFields[key]) {
+					const m = maskField(
+						field === 'serviceAccountKey' ? JSON.stringify(cp[field]) : cp[field],
+						field === 'serviceAccountKey' ? JSON.stringify(init?.[field]) : init?.[field]
+					)
+					if (m) cp[field] = m
+				}
+				masked[key] = cp
+			} else if (sensitiveKeys.has(key) && value != null && value !== '') {
+				masked[key] = value === initialValues?.[key] ? SENSITIVE_UNCHANGED : value
+			} else {
+				masked[key] = value
+			}
+		}
+		return masked
+	}
+
+	/**
+	 * Builds a sorted YAML string of all instance settings.
+	 * - normalize: strip keys whose values match the default (empty/falsy or key-specific defaults)
+	 * - mask: replace sensitive values with placeholder (for display)
+	 */
+	function buildSettingsYaml(
+		vals: Record<string, any>,
+		oauthsObj: Record<string, any>,
+		reqPreexisting: boolean,
+		opts: { normalize?: boolean; mask?: boolean } = {}
+	): string {
+		const obj: Record<string, any> = {}
+		for (const key of Object.keys(vals).sort()) {
+			if (excludedKeys.has(key)) continue
+			if (opts.normalize && normalizeValue(vals[key], key) === undefined) continue
+			obj[key] = vals[key]
+		}
+		if (oauthsObj && Object.keys(stripEmpty(oauthsObj)).length > 0) {
+			obj['oauths'] = oauthsObj
+		}
+		if (reqPreexisting) {
+			obj['require_preexisting_user_for_oauth'] = reqPreexisting
+		}
+		return YAML.stringify(opts.mask ? maskSensitive(obj) : obj)
+	}
+
+	function syncFormToYaml() {
+		yamlCode = buildSettingsYaml($values, oauths, requirePreexistingUserForOauth, {
+			normalize: true,
+			mask: !showSensitive
+		})
+		yamlCodeInitial = yamlCode
+		yamlEditor?.setCode(yamlCode)
+		yamlError = ''
+	}
+
+	function syncYamlToForm(): boolean {
+		try {
+			// Flush the editor's current content (bypasses the 200ms debounce in SimpleEditor)
+			const currentCode = yamlEditor?.getCode() ?? yamlCode
+			if (currentCode !== yamlCode) {
+				yamlCode = currentCode
+			}
+			const parsed = YAML.parse(yamlCode)
+			if (typeof parsed !== 'object' || parsed === null) {
+				sendUserToast('YAML must be a mapping (key: value)', true)
+				return false
+			}
+
+			// Restore sensitive values that were not changed (placeholder → original value)
+			if ('oauths' in parsed && typeof parsed['oauths'] === 'object') {
+				for (const [provider, config] of Object.entries(parsed['oauths'] as Record<string, any>)) {
+					if (
+						typeof config === 'object' &&
+						config !== null &&
+						config.secret === SENSITIVE_UNCHANGED
+					) {
+						config.secret = initialOauths?.[provider]?.secret
+					}
+				}
+				oauths = parsed['oauths'] ?? {}
+				delete parsed['oauths']
+			}
+			if ('require_preexisting_user_for_oauth' in parsed) {
+				requirePreexistingUserForOauth = parsed['require_preexisting_user_for_oauth'] ?? false
+				delete parsed['require_preexisting_user_for_oauth']
+			}
+
+			// Restore unchanged sensitive settings (placeholder → original value)
+			for (const key of sensitiveKeys) {
+				if (key in parsed && parsed[key] === SENSITIVE_UNCHANGED) {
+					parsed[key] = initialValues?.[key]
+				}
+			}
+			// Restore nested sensitive fields
+			for (const [parentKey, fields] of Object.entries(nestedSensitiveFields)) {
+				if (parsed[parentKey] && typeof parsed[parentKey] === 'object') {
+					const init = initialValues?.[parentKey]
+					for (const field of fields) {
+						if (parsed[parentKey][field] === SENSITIVE_UNCHANGED) {
+							parsed[parentKey][field] = init?.[field]
+						}
+					}
+				}
+			}
+
+			// Preserve excluded keys from current form state
+			for (const key of excludedKeys) {
+				if (key in $values) {
+					parsed[key] = $values[key]
+				}
+			}
+
+			$values = parsed
+			applyFormDefaults($values)
+
+			yamlError = ''
+			return true
+		} catch (e) {
+			yamlError = String(e)
+			sendUserToast('Invalid YAML: ' + e, true)
+			return false
+		}
+	}
+
+	let prevYamlMode = false
+	let prevLoading = true
+	$effect(() => {
+		if (yamlMode && !prevYamlMode) {
+			syncFormToYaml()
+		} else if (!yamlMode && prevYamlMode) {
+			if (!syncYamlToForm()) {
+				// Reset toggle back to YAML on parse failure
+				yamlMode = true
+			}
+		} else if (yamlMode && prevLoading && !loading) {
+			// Settings just finished loading while in YAML mode
+			syncFormToYaml()
+		}
+		prevYamlMode = yamlMode
+		prevLoading = loading
+	})
+
+	function handleShowSensitiveToggle(checked: boolean) {
+		// Sync any in-progress edits back to form state before re-rendering
+		syncYamlToForm()
+		showSensitive = checked
+		syncFormToYaml()
+	}
+
+	/** Call before entering diff mode to sync YAML edits into form state */
+	export function syncBeforeDiff(): boolean {
+		if (yamlMode) {
+			return syncYamlToForm()
+		}
+		return true
+	}
+
+	export function buildFullDiff(): { original: string; modified: string } {
+		return {
+			original: buildSettingsYaml(
+				initialValues,
+				initialOauths,
+				initialRequirePreexistingUserForOauth,
+				{ normalize: true }
+			),
+			modified: buildSettingsYaml($values, oauths, requirePreexistingUserForOauth, {
+				normalize: true
+			})
+		}
+	}
+
+	$effect(() => {
+		if (yamlMode) {
+			// In YAML mode, compare editor content against snapshot taken on entry
+			hasUnsavedChanges = yamlCodeInitial !== '' && yamlCode !== yamlCodeInitial
+		} else {
+			// Reuse per-category dirty tracking instead of rebuilding full YAML
+			hasUnsavedChanges = Object.values(dirtyCategories).some(Boolean)
+		}
+	})
 </script>

 <div class="pb-12">
-	<!-- svelte-ignore a11y_label_has_associated_control -->
-	{#if hideTabs}
+	{#if diffMode}
+		<div class="w-full h-[calc(100vh-8rem)]">
+			{#await import('$lib/components/DiffEditor.svelte')}
+				<Loader2 class="animate-spin m-4" />
+			{:then Module}
+				{@const diff = buildFullDiff()}
+				<Module.default
+					open={true}
+					className="!h-full"
+					defaultLang="yaml"
+					defaultOriginal={diff.original}
+					defaultModified={diff.modified}
+					readOnly
+					inlineDiff={true}
+				/>
+			{/await}
+		</div>
+	{:else if yamlMode}
+		<!-- svelte-ignore a11y_label_has_associated_control -->
+		<div class="flex items-center justify-end gap-4 mb-2">
+			<Toggle
+				checked={showSensitive}
+				on:change={(e) => handleShowSensitiveToggle(e.detail)}
+				options={{ right: 'Show sensitive values' }}
+				size="xs"
+			/>
+		</div>
+		<div class="border rounded w-full h-[calc(100vh-12rem)]">
+			{#await import('$lib/components/SimpleEditor.svelte')}
+				<Loader2 class="animate-spin" />
+			{:then Module}
+				<Module.default
+					bind:this={yamlEditor}
+					class="h-full"
+					lang="yaml"
+					bind:code={yamlCode}
+					fixedOverflowWidgets={false}
+				/>
+			{/await}
+		</div>
+		{#if yamlError}
+			<div class="text-red-500 text-xs mt-1">{yamlError}</div>
+		{/if}
+	{:else if hideTabs}
 		{@render categoryContent(tab)}
 	{:else}
 		<Tabs bind:selected={tab}>
@@ -586,7 +991,8 @@
 					...settings['Jobs'].filter((s) => s.key === 'job_isolation'),
 					...(licenseKeySetting ? [licenseKeySetting] : []),
 					...settings['Jobs'].filter((s) => s.key === 'retention_period_secs'),
-					...(settings['Object Storage']?.filter((s) => s.key === 'object_store_cache_config') ?? [])
+					...(settings['Object Storage']?.filter((s) => s.key === 'object_store_cache_config') ??
+						[])
 				]}
 				{#each extraSettings as setting}
 					<InstanceSetting
@@ -602,7 +1008,7 @@
 			{/if}
 		</div>

-		{#if !loading && !quickSetup}
+		{#if !loading && !quickSetup && !hideTabs}
 			<SettingsFooter
 				hasUnsavedChanges={dirtyCategories[category] ?? false}
 				disabled={invalidCategories[category] ?? false}
@@ -3,9 +3,9 @@
 	import SuperadminSettingsInner from './SuperadminSettingsInner.svelte'
 	import Version from './Version.svelte'
 	import MeltTooltip from './meltComponents/Tooltip.svelte'
-	import { workspaceStore } from '$lib/stores'
-	import { base } from '$lib/base'
-	import { ExternalLink } from 'lucide-svelte'
+	import Toggle from './Toggle.svelte'
+	import ConfirmationModal from './common/confirmationModal/ConfirmationModal.svelte'
+	import { X, FileDiff, Save, Loader2 } from 'lucide-svelte'
 	import { SettingsService } from '$lib/gen'
 	import { isCloudHosted } from '$lib/cloud'

@@ -16,7 +16,14 @@
 	let { disableChatOffset = false }: Props = $props()

 	let drawer: Drawer | undefined = $state()
+	let innerComponent: SuperadminSettingsInner | undefined = $state()
 	let uptodateVersion: string | undefined = $state(undefined)
+	let yamlMode = $state(false)
+	let diffMode = $state(false)
+	let hasUnsavedChanges = $state(false)
+	let pendingSave = $state(false)
+	let isSaving = $state(false)
+	let showCloseConfirmModal = $state(false)

 	async function loadUptodate() {
 		try {
@@ -29,21 +36,77 @@
 	}
 	loadUptodate()

+	// When true, the next Drawer 'close' event is intentional and should not be intercepted
+	let bypassCloseCheck = false
+
 	export function openDrawer() {
 		drawer?.openDrawer()
 	}

 	export function closeDrawer() {
+		bypassCloseCheck = true
 		drawer?.closeDrawer()
 	}
+
+	function handleClose() {
+		if (hasUnsavedChanges) {
+			showCloseConfirmModal = true
+		} else {
+			closeDrawer()
+		}
+	}
+
+	/** Catches click-away and escape closes from the Drawer/Disposable layer */
+	function handleDrawerClose() {
+		if (!bypassCloseCheck && hasUnsavedChanges) {
+			drawer?.openDrawer()
+			showCloseConfirmModal = true
+		}
+		bypassCloseCheck = false
+	}
+
+	async function handleSave() {
+		if (!pendingSave) {
+			if (!innerComponent?.syncBeforeDiff()) return
+			diffMode = true
+			pendingSave = true
+			return
+		}
+		isSaving = true
+		try {
+			await innerComponent?.saveSettings()
+			diffMode = false
+			pendingSave = false
+		} catch (e) {
+			console.error('Save failed:', e)
+		} finally {
+			isSaving = false
+		}
+	}
+
+	function handleDiscard() {
+		innerComponent?.discardAll()
+		diffMode = false
+		pendingSave = false
+	}
+
+	function handleShowDiff() {
+		if (!diffMode) {
+			if (!innerComponent?.syncBeforeDiff()) return
+		}
+		diffMode = !diffMode
+		if (!diffMode) {
+			pendingSave = false
+		}
+	}
 </script>

-<Drawer bind:this={drawer} size="1200px" {disableChatOffset}>
-	<DrawerContent noPadding overflow_y={false} title="Instance settings" on:close={closeDrawer}>
-		{#snippet actions()}
+<Drawer bind:this={drawer} size="1200px" {disableChatOffset} on:close={handleDrawerClose}>
+	<DrawerContent noPadding overflow_y={false} title="Instance settings" on:close={handleClose}>
+		{#snippet titleExtra()}
 			<MeltTooltip disablePopup={!uptodateVersion}>
-				<div class="text-xs text-secondary flex items-center gap-1">
-					Windmill <Version />
+				<div class="text-xs text-secondary flex items-center gap-1 ml-6">
+					<Version />
 					{#if uptodateVersion}
 						<span class="text-accent">→ {uptodateVersion}</span>
 					{/if}
@@ -58,19 +121,86 @@
 					{/if}
 				</svelte:fragment>
 			</MeltTooltip>
-			{#if $workspaceStore !== 'admins'}
+		{/snippet}
+		{#snippet actions()}
+			<div class="flex items-center gap-2">
 				<Button
 					variant="default"
 					size="xs"
-					target="_blank"
-					href="{base}/?workspace=admins"
-					endIcon={{ icon: ExternalLink }}
-					wrapperClasses="ml-2"
+					startIcon={{ icon: X }}
+					onClick={handleDiscard}
+					disabled={!hasUnsavedChanges || isSaving}
 				>
-					Admins workspace
+					Discard
 				</Button>
-			{/if}
+				<Button
+					variant={diffMode ? 'accent' : 'default'}
+					size="xs"
+					startIcon={{ icon: FileDiff }}
+					onClick={handleShowDiff}
+					disabled={!hasUnsavedChanges}
+				>
+					{diffMode ? 'Hide diff' : 'Show diff'}
+				</Button>
+				<Toggle
+					bind:checked={yamlMode}
+					options={{ right: 'YAML' }}
+					size="sm"
+				/>
+				<Button
+					variant="accent"
+					size="xs"
+					startIcon={{
+						icon: isSaving ? Loader2 : Save,
+						classes: isSaving ? 'animate-spin' : ''
+					}}
+					disabled={!hasUnsavedChanges || isSaving}
+					onClick={handleSave}
+				>
+					{isSaving ? 'Saving...' : pendingSave ? 'Confirm & Save' : 'Save settings'}
+				</Button>
+			</div>
 		{/snippet}
-		<SuperadminSettingsInner {closeDrawer} showHeaderInfo={false} />
+		<SuperadminSettingsInner
+			bind:this={innerComponent}
+			closeDrawer={handleClose}
+			showHeaderInfo={false}
+			bind:yamlMode
+			bind:diffMode
+			bind:hasUnsavedChanges
+		/>
 	</DrawerContent>
 </Drawer>
+
+{#if showCloseConfirmModal}
+	<ConfirmationModal
+		open={showCloseConfirmModal}
+		title="Unsaved changes"
+		confirmationText="Discard & close"
+		on:canceled={() => {
+			showCloseConfirmModal = false
+		}}
+		on:confirmed={() => {
+			innerComponent?.discardAll()
+			showCloseConfirmModal = false
+			diffMode = false
+			pendingSave = false
+			closeDrawer()
+		}}
+	>
+		<div class="flex flex-col w-full space-y-4">
+			<span>You have unsaved changes. Are you sure you want to discard them and close?</span>
+			<Button
+				variant="default"
+				size="xs"
+				startIcon={{ icon: FileDiff }}
+				onClick={() => {
+					showCloseConfirmModal = false
+					diffMode = true
+				}}
+			>
+				Show diff
+			</Button>
+		</div>
+	</ConfirmationModal>
+{/if}
@@ -39,7 +39,13 @@

 	let filter = $state('')

-	let { closeDrawer, showHeaderInfo = true } = $props()
+	let {
+		closeDrawer,
+		showHeaderInfo = true,
+		yamlMode = $bindable(false),
+		diffMode = $bindable(false),
+		hasUnsavedChanges = $bindable(false)
+	} = $props()

 	function removeHash() {
 		const index = $page.url.href.lastIndexOf('#')
@@ -116,25 +122,21 @@
 	let instanceSettingsCategory = $derived(tabToCategoryMap[tab] ?? 'Core')
 	let authSubTab: 'sso' | 'oauth' | 'scim' = $derived(tabToAuthSubTab[tab] ?? 'sso')

-	function getCategoryForTab(tabId: string): string | undefined {
-		return tabToCategoryMap[tabId]
-	}
-
-	// --- Tab change interception for unsaved changes ---
-	let pendingTab: string | undefined = $state(undefined)
-	let showUnsavedChangesModal = $state(false)
-
 	function handleNavigate(newTab: string) {
 		if (newTab === tab) return
+		tab = newTab
+	}

-		// Check if current tab (if it's a settings tab) has unsaved changes
-		const currentCategory = getCategoryForTab(tab)
-		if (currentCategory && instanceSettings?.isDirty(currentCategory)) {
-			pendingTab = newTab
-			showUnsavedChangesModal = true
-		} else {
-			tab = newTab
-		}
+	export function saveSettings() {
+		return instanceSettings?.saveSettings()
+	}
+
+	export function discardAll() {
+		instanceSettings?.discardAll()
+	}
+
+	export function syncBeforeDiff(): boolean {
+		return instanceSettings?.syncBeforeDiff() ?? true
 	}
 </script>

@@ -169,20 +171,34 @@
 		{/if}
 	{/if}
 	<div class="{showHeaderInfo ? 'pt-4' : ''} flex grow min-h-0">
-		<!-- Sidebar Navigation -->
-		<div class="w-52 shrink-0 h-full overflow-auto p-4 bg-surface">
-			<SidebarNavigation
-				groups={instanceSettingsNavigationGroups}
-				selectedId={tab}
-				onNavigate={handleNavigate}
-			/>
-		</div>
+		{#if !yamlMode && !diffMode}
+			<!-- Sidebar Navigation -->
+			<div class="w-52 shrink-0 h-full overflow-auto p-4 bg-surface flex flex-col">
+				<SidebarNavigation
+					groups={instanceSettingsNavigationGroups}
+					selectedId={tab}
+					onNavigate={handleNavigate}
+				/>
+				{#if $workspaceStore !== 'admins'}
+					<div class="mt-auto pt-4 border-t border-surface-hover">
+						<a
+							href="{base}/?workspace=admins"
+							target="_blank"
+							class="flex items-center gap-2 px-2 py-1.5 text-xs text-secondary hover:text-primary transition-colors"
+						>
+							<ExternalLink size={14} />
+							Admins workspace
+						</a>
+					</div>
+				{/if}
+			</div>
+		{/if}

 		<!-- Main Content -->
 		<div class="flex-1 min-w-0 h-full">
 			<div class="h-full overflow-auto bg-surface">
 				<div class="h-fit px-8 py-4">
-					{#if tab === 'users'}
+					{#if tab === 'users' && !yamlMode && !diffMode}
 						<div class="h-full">
 							{#if !automateUsernameCreation && !isCloudHosted()}
 								<div class="mb-4">
@@ -457,6 +473,9 @@
 						<InstanceSettings
 							bind:this={instanceSettings}
 							hideTabs
+							bind:yamlMode
+							bind:diffMode
+							bind:hasUnsavedChanges
 							tab={instanceSettingsCategory}
 							{authSubTab}
 							{closeDrawer}
@@ -473,33 +492,6 @@
 		</div>
 	</div>
 </div>
-{#if showUnsavedChangesModal}
-	<ConfirmationModal
-		open={showUnsavedChangesModal}
-		title="Unsaved changes detected"
-		confirmationText="Discard changes"
-		on:canceled={() => {
-			showUnsavedChangesModal = false
-			pendingTab = undefined
-		}}
-		on:confirmed={() => {
-			if (pendingTab !== undefined) {
-				const currentCategory = getCategoryForTab(tab)
-				if (currentCategory) {
-					instanceSettings?.discardCategory(currentCategory)
-				}
-				tab = pendingTab
-			}
-			showUnsavedChangesModal = false
-			pendingTab = undefined
-		}}
-	>
-		<div class="flex flex-col w-full space-y-4">
-			<span>You have unsaved changes. Are you sure you want to discard them?</span>
-		</div>
-	</ConfirmationModal>
-{/if}
-
 <ConfirmationModal
 	open={Boolean(deleteConfirmedCallback)}
 	title="Remove user"
@@ -106,7 +106,7 @@ export const scimSamlSetting: Setting[] = [
 		label: 'SCIM token',
 		description: 'Token used to authenticate requests from the IdP',
 		key: 'scim_token',
-		fieldType: 'text',
+		fieldType: 'password',
 		placeholder: 'mytoken',
 		storage: 'setting',
 		ee_only: ''
@@ -101,6 +101,7 @@
 				classNames
 			)}
 			use:conditionalMelt={trigger}
+			aria-label={label}
 			title={isCollapsed ? undefined : label}
 			{...$trigger}
 		>
@@ -70,6 +70,7 @@
 			)}
 			data-light-mode={lightMode}
 			target={href.includes('http') ? '_blank' : null}
+			aria-label={label}
 			title={isCollapsed ? undefined : label}
 			use:conditionalMelt={item}
 			{...$item}