mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-15 00:02:56 +00:00
feat: workspace settings as one document, applied server-side (#11740)
* feat: add the workspace settings document, applied server-side * feat: add wmill workspace get-settings and apply-settings * fix: warn on unknown settings keys and refuse ee-only ones in the dry run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: stub the settings document endpoint in the push settings unit tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: refuse mistyped settings and keep 4xx status on a partial settings apply Reject non-boolean switches and non-string Slack credentials instead of coercing them into destructive writes, preserve the failing endpoint's status when wrapping a partial-apply error, keep the stored Slack secret when an older CLI pushes one encrypted under another workspace's key, print the plan's warnings in apply-settings, and state the admin check load_settings_document leaves to its caller. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: compare handler arguments verbatim and type-check scalar settings A handler's extra_args is stored as sent and its nulls reach the job as arguments, so an added or removed null argument is now a change. Scalar settings are type-checked in the plan so a dry run never reports a change the endpoint then refuses, and an error handler's omitted mute flags compare as the false the endpoint stores. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: require a 32-bit integer rate limit in the settings plan Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat: data table permissions and protection rules in the settings document The settings document gains datatable_permissions, each data table's roles named by their Postgres role name and written by catalog id, and protection_rules, keyed by rule name. Both are managed only when stated and stated as a whole. The dev workspace lock stays with the dev workspace setting. wmill sync carries datatable_permissions only with includeDatatablePermissions (or --include-datatable-permissions), since tenants and roles are specific to a workspace and an instance. protection_rules stay out of settings.yaml: protection-rules.yaml syncs them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: validate the default data table role and plan permissions against a replaced datatable Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
17fefcf777
commit
9a5ec223bc
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT slack_oauth_client_secret FROM workspace_settings WHERE workspace_id = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "slack_oauth_client_secret",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "0d04ec377bef28416a5721a43d2c687681b5d36ff1918ed9e495c5596f2ba9f2"
|
||||
}
|
||||
@@ -1765,6 +1765,43 @@ async fn set_slack_oauth_config(
|
||||
"Both client ID and client secret are required".to_string(),
|
||||
));
|
||||
}
|
||||
// A settings.yaml pulled by `wmill sync` holds the secret encrypted, and a CLI that
|
||||
// predates the settings document pushes it here as it reads it. One encrypted under another
|
||||
// workspace's key keeps the stored secret, as the settings document does, so that CLI's push
|
||||
// does not abort over it.
|
||||
let mut req = req;
|
||||
if let Some(ciphertext) = req
|
||||
.slack_oauth_client_secret
|
||||
.strip_prefix(windmill_common::variables::SETTINGS_ENCRYPTED_PREFIX)
|
||||
{
|
||||
match windmill_common::variables::build_crypt(&db, &w_id)
|
||||
.await?
|
||||
.decrypt_str(ciphertext)
|
||||
{
|
||||
Ok(plain) => req.slack_oauth_client_secret = plain,
|
||||
Err(_) => {
|
||||
let stored = sqlx::query_scalar!(
|
||||
"SELECT slack_oauth_client_secret FROM workspace_settings WHERE workspace_id = $1",
|
||||
&w_id
|
||||
)
|
||||
.fetch_optional(&db)
|
||||
.await?
|
||||
.flatten();
|
||||
tracing::warn!(
|
||||
workspace_id = %w_id,
|
||||
"Slack client secret left unchanged: it is encrypted with another workspace's key"
|
||||
);
|
||||
match stored {
|
||||
Some(stored) => req.slack_oauth_client_secret = stored,
|
||||
None => {
|
||||
return Ok("Slack OAuth config left unchanged: the client secret is \
|
||||
encrypted with another workspace's key"
|
||||
.to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
|
||||
@@ -14852,3 +14889,154 @@ async fn set_ws_specific(
|
||||
body.value, body.item_kind, body.path
|
||||
))
|
||||
}
|
||||
|
||||
/// A setting the workspace settings document writes, each through the endpoint that owns it,
|
||||
/// so applying a document validates, audits and has the side effects of the matching UI save.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum SettingsEdit {
|
||||
Webhook,
|
||||
AutoInvite,
|
||||
InstanceGroups,
|
||||
ErrorHandler,
|
||||
SuccessHandler,
|
||||
LargeFileStorage,
|
||||
GitSync,
|
||||
DefaultScripts,
|
||||
DefaultApp,
|
||||
Color,
|
||||
OperatorSettings,
|
||||
Datatable,
|
||||
SlackCommand,
|
||||
SetSlackOAuth,
|
||||
DeleteSlackOAuth,
|
||||
Ducklake,
|
||||
DbtWarehouses,
|
||||
DeployUi,
|
||||
GuestAccess,
|
||||
GuestJwtKey,
|
||||
PublicAppRateLimit,
|
||||
AddAdminsAndDevelopersToForks,
|
||||
/// Body: `{ datatable, permissioned, default_role, roles: [{ id, tenants }] }`, role ids.
|
||||
DatatablePermissions,
|
||||
CreateProtectionRule,
|
||||
/// Body: the update request plus `rule_name`, the rule it updates.
|
||||
UpdateProtectionRule,
|
||||
/// Body: `{ name }`.
|
||||
DeleteProtectionRule,
|
||||
}
|
||||
|
||||
fn settings_edit_body<T: serde::de::DeserializeOwned>(
|
||||
edit: SettingsEdit,
|
||||
body: serde_json::Value,
|
||||
) -> Result<Json<T>> {
|
||||
serde_json::from_value(body)
|
||||
.map(Json)
|
||||
.map_err(|e| Error::BadRequest(format!("Invalid value for {edit:?}: {e}")))
|
||||
}
|
||||
|
||||
/// Returns the endpoint's JSON response, or `null` for one that answers with prose.
|
||||
pub async fn apply_settings_edit(
|
||||
authed: ApiAuthed,
|
||||
db: DB,
|
||||
w_id: String,
|
||||
edit: SettingsEdit,
|
||||
body: serde_json::Value,
|
||||
) -> Result<serde_json::Value> {
|
||||
use SettingsEdit::*;
|
||||
let a = authed.clone();
|
||||
let (d, p) = (Extension(db), Path(w_id));
|
||||
match edit {
|
||||
Webhook => edit_webhook(a, d, p, authed, settings_edit_body(edit, body)?).await?,
|
||||
AutoInvite => edit_auto_invite(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
InstanceGroups => edit_instance_groups(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
ErrorHandler => {
|
||||
edit_error_handler(a, d, p, authed, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
SuccessHandler => edit_success_handler(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
LargeFileStorage => {
|
||||
edit_large_file_storage_config(a, d, p, authed, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
GitSync => edit_git_sync_config(a, d, p, authed, settings_edit_body(edit, body)?).await?,
|
||||
DefaultScripts => {
|
||||
edit_default_scripts(a, d, p, authed, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
#[cfg(feature = "enterprise")]
|
||||
DefaultApp => edit_default_app(a, d, p, authed, settings_edit_body(edit, body)?).await?,
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
DefaultApp => edit_default_app(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
Color => change_workspace_color(a, p, d, settings_edit_body(edit, body)?).await?,
|
||||
OperatorSettings => {
|
||||
update_operator_settings(a, p, d, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
Datatable => {
|
||||
let Json(result) =
|
||||
edit_datatable_config(a, d, p, authed, settings_edit_body(edit, body)?).await?;
|
||||
return serde_json::to_value(result).map_err(|e| Error::internal_err(e.to_string()));
|
||||
}
|
||||
SlackCommand => {
|
||||
edit_slack_command(a, d, p, authed, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
SetSlackOAuth => set_slack_oauth_config(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
DeleteSlackOAuth => delete_slack_oauth_config(a, d, p).await?,
|
||||
Ducklake => edit_ducklake_config(a, d, p, authed, settings_edit_body(edit, body)?).await?,
|
||||
DbtWarehouses => {
|
||||
edit_dbt_warehouses(a, d, p, authed, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
#[cfg(feature = "enterprise")]
|
||||
DeployUi => edit_deploy_ui_config(a, d, p, authed, settings_edit_body(edit, body)?).await?,
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
DeployUi => edit_deploy_ui_config(a, d, p).await?,
|
||||
GuestAccess => edit_guest_access(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
GuestJwtKey => edit_guest_jwt_key(a, d, p, settings_edit_body(edit, body)?).await?,
|
||||
PublicAppRateLimit => {
|
||||
edit_public_app_rate_limit(d, p, a, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
AddAdminsAndDevelopersToForks => {
|
||||
edit_add_admins_and_developers_to_forks(a, d, p, settings_edit_body(edit, body)?)
|
||||
.await?
|
||||
}
|
||||
#[cfg(all(feature = "private", feature = "enterprise"))]
|
||||
DatatablePermissions => {
|
||||
#[derive(Deserialize)]
|
||||
struct Body {
|
||||
datatable: String,
|
||||
#[serde(flatten)]
|
||||
req: crate::datatable_permissions_ee::SetDatatablePermissions,
|
||||
}
|
||||
let Json(Body { datatable, req }) = settings_edit_body(edit, body)?;
|
||||
crate::datatable_permissions_oss::set_datatable_permissions(
|
||||
a,
|
||||
d,
|
||||
Path((p.0, datatable)),
|
||||
Json(req),
|
||||
)
|
||||
.await?
|
||||
}
|
||||
#[cfg(not(all(feature = "private", feature = "enterprise")))]
|
||||
DatatablePermissions => {
|
||||
crate::datatable_permissions_oss::set_datatable_permissions(a).await?
|
||||
}
|
||||
CreateProtectionRule => {
|
||||
create_protection_rule(a, d, p, settings_edit_body(edit, body)?).await?
|
||||
}
|
||||
UpdateProtectionRule => {
|
||||
#[derive(Deserialize)]
|
||||
struct Body {
|
||||
rule_name: String,
|
||||
#[serde(flatten)]
|
||||
req: UpdateProtectionRuleRequest,
|
||||
}
|
||||
let Json(Body { rule_name, req }) = settings_edit_body(edit, body)?;
|
||||
update_protection_rule(a, d, Path((p.0, rule_name)), Json(req)).await?
|
||||
}
|
||||
DeleteProtectionRule => {
|
||||
#[derive(Deserialize)]
|
||||
struct Body {
|
||||
name: String,
|
||||
}
|
||||
let Json(Body { name }) = settings_edit_body(edit, body)?;
|
||||
delete_protection_rule(a, d, Path((p.0, name))).await?
|
||||
}
|
||||
};
|
||||
Ok(serde_json::Value::Null)
|
||||
}
|
||||
|
||||
@@ -5184,6 +5184,98 @@ paths:
|
||||
- has_instance_ai_config
|
||||
- uses_instance_ai_config
|
||||
|
||||
/w/{workspace}/workspaces/settings_document:
|
||||
get:
|
||||
summary: get the workspace settings document
|
||||
description: >-
|
||||
The workspace's settings in the layout of the settings.yaml that `wmill sync` writes.
|
||||
Admin only.
|
||||
operationId: getWorkspaceSettingsDocument
|
||||
tags:
|
||||
- workspace
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- name: secrets
|
||||
description: >-
|
||||
How secrets are written: `redacted` (the default) replaces each set secret with
|
||||
`__SENSITIVE_AND_UNCHANGED__`; `encrypted` writes it encrypted with the workspace key,
|
||||
prefixed with `$encrypted:`.
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
enum: [redacted, encrypted]
|
||||
responses:
|
||||
"200":
|
||||
description: the settings document
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
post:
|
||||
summary: apply a workspace settings document
|
||||
description: >-
|
||||
Applies a settings document, each setting through the endpoint that owns it. Admin only.
|
||||
operationId: applyWorkspaceSettingsDocument
|
||||
tags:
|
||||
- workspace
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
document:
|
||||
description: >-
|
||||
The settings to apply, in the layout of the document. A secret given as
|
||||
`__SENSITIVE_AND_UNCHANGED__` keeps its stored value; one prefixed with
|
||||
`$encrypted:` is decrypted with the workspace key.
|
||||
type: object
|
||||
additionalProperties: true
|
||||
replace:
|
||||
description: >-
|
||||
Clear the settings `wmill sync push` clears when settings.yaml omits them.
|
||||
Otherwise a setting the document omits is left as it is.
|
||||
type: boolean
|
||||
dry_run:
|
||||
description: Only compute the changes, without applying them.
|
||||
type: boolean
|
||||
required:
|
||||
- document
|
||||
responses:
|
||||
"200":
|
||||
description: the changes, applied unless dry_run
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
changes:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
key:
|
||||
type: string
|
||||
before: {}
|
||||
after: {}
|
||||
required:
|
||||
- key
|
||||
- before
|
||||
- after
|
||||
applied:
|
||||
type: boolean
|
||||
warnings:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
required:
|
||||
- changes
|
||||
- applied
|
||||
|
||||
/w/{workspace}/workspaces/get_copilot_settings_state:
|
||||
get:
|
||||
summary: get copilot settings state
|
||||
|
||||
@@ -209,6 +209,7 @@ mod workspaces;
|
||||
#[cfg(feature = "private")]
|
||||
pub mod workspaces_ee;
|
||||
mod workspaces_export;
|
||||
mod workspace_settings_document;
|
||||
|
||||
#[cfg(feature = "mcp")]
|
||||
mod mcp_tools;
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -61,6 +61,11 @@ pub fn workspaced_service() -> Router {
|
||||
)
|
||||
.route("/tarball", get(crate::workspaces_export::tarball_workspace))
|
||||
.route("/edit_copilot_config", post(edit_copilot_config))
|
||||
.route(
|
||||
"/settings_document",
|
||||
get(crate::workspace_settings_document::get_settings_document)
|
||||
.post(crate::workspace_settings_document::apply_settings_document),
|
||||
)
|
||||
.route("/get_copilot_info", get(get_copilot_info))
|
||||
.route("/critical_alerts", get(get_critical_alerts))
|
||||
.route(
|
||||
@@ -85,7 +90,7 @@ pub fn workspaced_service() -> Router {
|
||||
router
|
||||
}
|
||||
|
||||
async fn edit_copilot_config(
|
||||
pub(crate) async fn edit_copilot_config(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
@@ -190,7 +195,7 @@ async fn edit_copilot_config(
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct EditCopilotConfigResponse {
|
||||
pub(crate) struct EditCopilotConfigResponse {
|
||||
effective_ai_config: AIConfig,
|
||||
has_instance_ai_config: bool,
|
||||
uses_instance_ai_config: bool,
|
||||
@@ -321,7 +326,7 @@ pub struct MuteCriticalAlertRequest {
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
async fn mute_critical_alerts(
|
||||
pub(crate) async fn mute_critical_alerts(
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
authed: ApiAuthed,
|
||||
|
||||
@@ -359,6 +359,9 @@ pub(crate) struct ArchiveQueryParams {
|
||||
include_users: Option<bool>,
|
||||
include_groups: Option<bool>,
|
||||
include_settings: Option<bool>,
|
||||
/// Opt-in, with `settings_version=v2`: state the data tables' roles in settings.json. Their
|
||||
/// tenants name this workspace's users and groups, and their roles this instance's catalog.
|
||||
include_datatable_permissions: Option<bool>,
|
||||
include_key: Option<bool>,
|
||||
include_workspace_dependencies: Option<bool>,
|
||||
/// Default `false`: data table migrations ship in the tarball unless the repo
|
||||
@@ -513,49 +516,6 @@ struct SimplifiedGroup {
|
||||
admins: Vec<String>,
|
||||
}
|
||||
|
||||
// V2 format: New grouped format
|
||||
#[derive(Serialize)]
|
||||
struct SimplifiedSettings {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
auto_invite: Option<Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
webhook: Option<String>,
|
||||
// Always serialize (including as `null`) so that `wmill sync pull` emits
|
||||
// these fields in settings.yaml unconditionally. Makes round-trip
|
||||
// bijective: YAML is the source of truth, absence/null = "clear remote",
|
||||
// mirroring every other workspace setting.
|
||||
error_handler: Option<Value>,
|
||||
success_handler: Option<Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
ai_config: Option<serde_json::Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
large_file_storage: Option<Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
git_sync: Option<Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
default_app: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
default_scripts: Option<Value>,
|
||||
name: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
mute_critical_alerts: Option<bool>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
color: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
operator_settings: Option<serde_json::Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
datatable: Option<Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
slack_team_id: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
slack_name: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
slack_command_script: Option<String>,
|
||||
// Always serialize (see note above on error_handler / success_handler).
|
||||
slack_oauth_client_id: Option<String>,
|
||||
slack_oauth_client_secret: Option<String>,
|
||||
}
|
||||
|
||||
// V1 format: Legacy flat format for backward compatibility (matches main branch exactly)
|
||||
#[derive(Serialize)]
|
||||
struct SimplifiedSettingsLegacy {
|
||||
@@ -602,7 +562,6 @@ struct SettingsRow {
|
||||
auto_invite: Option<Value>,
|
||||
webhook: Option<String>,
|
||||
error_handler: Option<Value>,
|
||||
success_handler: Option<Value>,
|
||||
ai_config: Option<serde_json::Value>,
|
||||
large_file_storage: Option<Value>,
|
||||
git_sync: Option<Value>,
|
||||
@@ -616,8 +575,6 @@ struct SettingsRow {
|
||||
slack_team_id: Option<String>,
|
||||
slack_name: Option<String>,
|
||||
slack_command_script: Option<String>,
|
||||
slack_oauth_client_id: Option<String>,
|
||||
slack_oauth_client_secret: Option<String>,
|
||||
}
|
||||
|
||||
pub(crate) async fn tarball_workspace(
|
||||
@@ -638,6 +595,7 @@ pub(crate) async fn tarball_workspace(
|
||||
include_users,
|
||||
include_groups,
|
||||
include_settings,
|
||||
include_datatable_permissions,
|
||||
include_key,
|
||||
include_workspace_dependencies,
|
||||
skip_datatable_migrations,
|
||||
@@ -1593,7 +1551,6 @@ pub(crate) async fn tarball_workspace(
|
||||
auto_invite,
|
||||
webhook,
|
||||
error_handler,
|
||||
success_handler,
|
||||
ai_config,
|
||||
large_file_storage,
|
||||
git_sync,
|
||||
@@ -1606,9 +1563,7 @@ pub(crate) async fn tarball_workspace(
|
||||
datatable,
|
||||
slack_team_id,
|
||||
slack_name,
|
||||
slack_command_script,
|
||||
slack_oauth_client_id,
|
||||
slack_oauth_client_secret
|
||||
slack_command_script
|
||||
FROM workspace_settings
|
||||
LEFT JOIN workspace ON workspace.id = workspace_settings.workspace_id
|
||||
WHERE workspace_id = $1"#,
|
||||
@@ -1617,73 +1572,23 @@ pub(crate) async fn tarball_workspace(
|
||||
.fetch_one(&mut *tx)
|
||||
.await?;
|
||||
|
||||
// Use v2 format only if explicitly requested, otherwise use v1 (legacy) for backward compatibility
|
||||
// Server-owned state (the HMAC webhook secret + hook id/error, the
|
||||
// synced-sha / last-pull status, the admin automatic pulls run as, and what
|
||||
// the credential check observed) must never leave the server: keep it out of
|
||||
// export archives and synced repos, and don't let a re-imported workspace
|
||||
// inherit another install's hook/sync state or pull identity.
|
||||
fn redact_git_sync_for_export(git_sync: Option<Value>) -> Option<Value> {
|
||||
let mut git_sync = git_sync?;
|
||||
if let Some(repos) = git_sync
|
||||
.get_mut("repositories")
|
||||
.and_then(|r| r.as_array_mut())
|
||||
{
|
||||
for repo in repos {
|
||||
if let Some(auto_pull) =
|
||||
repo.get_mut("auto_pull").and_then(|a| a.as_object_mut())
|
||||
{
|
||||
for field in [
|
||||
"webhook_secret",
|
||||
"webhook_id",
|
||||
"webhook_error",
|
||||
"last_synced_sha",
|
||||
"last_pull_status",
|
||||
"enabled_by",
|
||||
] {
|
||||
auto_pull.remove(field);
|
||||
}
|
||||
}
|
||||
// What this install observed about its own credential: a token
|
||||
// id and expiry, and a `checked_at` that moves on its own.
|
||||
// None of it describes the workspace, and in a git-synced
|
||||
// `wmill.yaml` it would churn the file for no reason.
|
||||
if let Some(repo) = repo.as_object_mut() {
|
||||
repo.remove("credential");
|
||||
}
|
||||
}
|
||||
}
|
||||
Some(git_sync)
|
||||
}
|
||||
|
||||
let settings_str = if settings_version.as_deref() == Some("v2") {
|
||||
let settings = SimplifiedSettings {
|
||||
auto_invite: row.auto_invite,
|
||||
webhook: row.webhook,
|
||||
error_handler: row.error_handler,
|
||||
success_handler: row.success_handler,
|
||||
ai_config: row.ai_config,
|
||||
large_file_storage: row.large_file_storage,
|
||||
git_sync: redact_git_sync_for_export(row.git_sync),
|
||||
default_app: row.default_app,
|
||||
default_scripts: row.default_scripts,
|
||||
name: row.name.clone().unwrap_or_default(),
|
||||
mute_critical_alerts: row.mute_critical_alerts,
|
||||
color: row.color.clone(),
|
||||
operator_settings: row.operator_settings.clone(),
|
||||
datatable: windmill_common::workspaces::strip_datatable_permissions(
|
||||
row.datatable.clone(),
|
||||
),
|
||||
slack_team_id: row.slack_team_id.clone(),
|
||||
slack_name: row.slack_name.clone(),
|
||||
slack_command_script: row.slack_command_script.clone(),
|
||||
slack_oauth_client_id: row.slack_oauth_client_id.clone(),
|
||||
slack_oauth_client_secret: row.slack_oauth_client_secret.clone(),
|
||||
let secrets = if export_plain_secrets {
|
||||
crate::workspace_settings_document::SecretMode::Plain
|
||||
} else {
|
||||
crate::workspace_settings_document::SecretMode::Encrypted
|
||||
};
|
||||
serde_json::to_value(settings)
|
||||
.map(|v| serde_json::to_string_pretty(&v).ok())
|
||||
.ok()
|
||||
.flatten()
|
||||
let mut doc = crate::workspace_settings_document::load_settings_document(
|
||||
&mut *tx, &db, &w_id, secrets,
|
||||
)
|
||||
.await?;
|
||||
// A repo syncs its rules through protection-rules.yaml, so settings.yaml leaving
|
||||
// them out keeps one writer for them.
|
||||
doc.remove("protection_rules");
|
||||
if !include_datatable_permissions.unwrap_or(false) {
|
||||
doc.remove("datatable_permissions");
|
||||
}
|
||||
serde_json::to_string_pretty(&doc).ok()
|
||||
} else {
|
||||
// V1 (legacy) format: convert JSONB to flat fields (matches main branch exactly)
|
||||
let (auto_invite_enabled, auto_invite_as, auto_invite_mode) =
|
||||
@@ -1730,7 +1635,7 @@ pub(crate) async fn tarball_workspace(
|
||||
error_handler_muted_on_cancel,
|
||||
ai_config: row.ai_config,
|
||||
large_file_storage: row.large_file_storage,
|
||||
git_sync: redact_git_sync_for_export(row.git_sync),
|
||||
git_sync: crate::workspace_settings_document::redact_git_sync(row.git_sync),
|
||||
default_app: row.default_app,
|
||||
default_scripts: row.default_scripts,
|
||||
name: row.name.unwrap_or_default(),
|
||||
|
||||
@@ -289,9 +289,9 @@ pub fn error_source_chain(e: &dyn std::error::Error) -> String {
|
||||
msg
|
||||
}
|
||||
|
||||
impl IntoResponse for Error {
|
||||
fn into_response(self) -> axum::response::Response {
|
||||
let status = match self {
|
||||
impl Error {
|
||||
pub fn status_code(&self) -> axum::http::StatusCode {
|
||||
match self {
|
||||
Self::NotFound(_) => axum::http::StatusCode::NOT_FOUND,
|
||||
Self::NotAuthorized(_) => axum::http::StatusCode::UNAUTHORIZED,
|
||||
Self::RequireAdmin(_) | Self::PermissionDenied(_) => axum::http::StatusCode::FORBIDDEN,
|
||||
@@ -301,9 +301,15 @@ impl IntoResponse for Error {
|
||||
| Self::JsonErr(_)
|
||||
| Self::QuotaExceeded(_) => axum::http::StatusCode::BAD_REQUEST,
|
||||
Self::BadGateway(_) => axum::http::StatusCode::BAD_GATEWAY,
|
||||
Self::Generic(status_code, _) => status_code,
|
||||
Self::Generic(status_code, _) => *status_code,
|
||||
_ => axum::http::StatusCode::INTERNAL_SERVER_ERROR,
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl IntoResponse for Error {
|
||||
fn into_response(self) -> axum::response::Response {
|
||||
let status = self.status_code();
|
||||
|
||||
let e = &self;
|
||||
|
||||
|
||||
@@ -373,6 +373,10 @@ pub async fn get_secret_value_as_admin(
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
/// Marks a workspace setting's secret written into the settings document encrypted with the
|
||||
/// workspace key, as `wmill sync pull` writes it.
|
||||
pub const SETTINGS_ENCRYPTED_PREFIX: &str = "$encrypted:";
|
||||
|
||||
pub fn encrypt(mc: &WorkspaceCrypt, value: &str) -> String {
|
||||
mc.encrypt_str(value)
|
||||
}
|
||||
|
||||
@@ -21,6 +21,7 @@ use aes_gcm::aead::{Aead, AeadCore, KeyInit, OsRng};
|
||||
use aes_gcm::{Aes256Gcm, Key, Nonce};
|
||||
use base64::Engine;
|
||||
use hkdf::Hkdf;
|
||||
use hmac::{Hmac, Mac};
|
||||
use magic_crypt::{MagicCrypt256, MagicCryptTrait};
|
||||
use sha2::Sha256;
|
||||
|
||||
@@ -36,6 +37,7 @@ const NARROW_VARIABLE_VALUE_WIDTH: usize = 15000;
|
||||
pub const MAX_BYTES_OVERHEAD: usize = V2_MAGIC.len() + NONCE_LEN + BLOCK + TAG_LEN;
|
||||
|
||||
const AEAD_KEY_INFO: &[u8] = b"windmill workspace secret aes-256-gcm v1";
|
||||
const SYNTHETIC_NONCE_KEY_INFO: &[u8] = b"windmill workspace secret synthetic nonce v1";
|
||||
|
||||
const B64: base64::engine::GeneralPurpose = base64::engine::general_purpose::STANDARD;
|
||||
|
||||
@@ -79,6 +81,7 @@ impl CipherFormat {
|
||||
pub struct WorkspaceCrypt {
|
||||
legacy: MagicCrypt256,
|
||||
aead: Aes256Gcm,
|
||||
synthetic_nonce_key: [u8; 32],
|
||||
}
|
||||
|
||||
/// Code that drives `magic_crypt` directly on this cipher (app link secrets) keeps the legacy
|
||||
@@ -99,9 +102,14 @@ impl WorkspaceCrypt {
|
||||
Hkdf::<Sha256>::new(None, crypt_key.as_bytes())
|
||||
.expand(AEAD_KEY_INFO, &mut aead_key)
|
||||
.expect("32 bytes is a valid HKDF-SHA256 output length");
|
||||
let mut synthetic_nonce_key = [0u8; 32];
|
||||
Hkdf::<Sha256>::new(None, crypt_key.as_bytes())
|
||||
.expand(SYNTHETIC_NONCE_KEY_INFO, &mut synthetic_nonce_key)
|
||||
.expect("32 bytes is a valid HKDF-SHA256 output length");
|
||||
Self {
|
||||
legacy: magic_crypt::new_magic_crypt!(crypt_key, 256),
|
||||
aead: Aes256Gcm::new(Key::<Aes256Gcm>::from_slice(&aead_key)),
|
||||
synthetic_nonce_key,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -132,6 +140,18 @@ impl WorkspaceCrypt {
|
||||
}
|
||||
}
|
||||
|
||||
/// A v2 value whose nonce is an HMAC of the plaintext, so the same plaintext always gives
|
||||
/// the same ciphertext. For values written into synced files, where a fresh nonce on every
|
||||
/// export would show as a change. It reveals that two values are equal, nothing more; it
|
||||
/// decrypts with [`Self::decrypt_str`] like any v2 value.
|
||||
pub fn encrypt_str_deterministic(&self, plaintext: &str) -> String {
|
||||
let mut mac = <Hmac<Sha256> as Mac>::new_from_slice(&self.synthetic_nonce_key)
|
||||
.expect("HMAC accepts any key length");
|
||||
mac.update(plaintext.as_bytes());
|
||||
let tag = mac.finalize().into_bytes();
|
||||
B64.encode(self.seal_with_nonce(plaintext.as_bytes(), Nonce::from_slice(&tag[..NONCE_LEN])))
|
||||
}
|
||||
|
||||
pub fn decrypt_str(&self, ciphertext: &str) -> Result<String, DecryptFailure> {
|
||||
let bytes = B64
|
||||
.decode(ciphertext)
|
||||
@@ -175,18 +195,21 @@ impl WorkspaceCrypt {
|
||||
}
|
||||
|
||||
fn seal(&self, plaintext: &[u8]) -> Vec<u8> {
|
||||
let nonce = Aes256Gcm::generate_nonce(&mut OsRng);
|
||||
self.seal_with_nonce(plaintext, &Aes256Gcm::generate_nonce(&mut OsRng))
|
||||
}
|
||||
|
||||
fn seal_with_nonce(&self, plaintext: &[u8], nonce: &Nonce<<Aes256Gcm as AeadCore>::NonceSize>) -> Vec<u8> {
|
||||
let pad = BLOCK - plaintext.len() % BLOCK;
|
||||
let mut padded = Vec::with_capacity(plaintext.len() + pad);
|
||||
padded.extend_from_slice(plaintext);
|
||||
padded.resize(plaintext.len() + pad, pad as u8);
|
||||
let sealed = self
|
||||
.aead
|
||||
.encrypt(&nonce, padded.as_slice())
|
||||
.encrypt(nonce, padded.as_slice())
|
||||
.expect("AES-GCM encryption fails only past 64 GiB of plaintext");
|
||||
let mut out = Vec::with_capacity(V2_MAGIC.len() + NONCE_LEN + sealed.len());
|
||||
out.extend_from_slice(&V2_MAGIC);
|
||||
out.extend_from_slice(&nonce);
|
||||
out.extend_from_slice(nonce);
|
||||
out.extend_from_slice(&sealed);
|
||||
out
|
||||
}
|
||||
@@ -234,6 +257,17 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deterministic_is_stable_per_plaintext_and_key() {
|
||||
let wc = WorkspaceCrypt::new(KEY);
|
||||
let a = wc.encrypt_str_deterministic("secret");
|
||||
assert_eq!(a, wc.encrypt_str_deterministic("secret"));
|
||||
assert_ne!(a, wc.encrypt_str_deterministic("secret2"));
|
||||
assert_ne!(a, WorkspaceCrypt::new("other-key").encrypt_str_deterministic("secret"));
|
||||
assert!(wc.is_v2(&a));
|
||||
assert_eq!(wc.decrypt_str(&a).unwrap(), "secret");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn legacy_cbc_still_decrypts() {
|
||||
let wc = WorkspaceCrypt::new(KEY);
|
||||
|
||||
@@ -144,6 +144,15 @@ impl From<&Vec<ProtectionRuleKind>> for ProtectionRules {
|
||||
}
|
||||
}
|
||||
|
||||
impl ProtectionRules {
|
||||
/// The rules set, in declaration order.
|
||||
pub fn kinds(self) -> Vec<ProtectionRuleKind> {
|
||||
<ProtectionRuleKind as strum::IntoEnumIterator>::iter()
|
||||
.filter(|k| self.contains(k.flag()))
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug, Default)]
|
||||
pub struct WorkspaceGitSyncSettings {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
@@ -1547,9 +1556,10 @@ pub struct DataTable {
|
||||
/// which is how data tables behaved before roles existed. Only meaningful on a terminal entry;
|
||||
/// a pointer is governed by what it points at.
|
||||
///
|
||||
/// Never leaves the instance: stripped from the workspace export and ignored on import, since
|
||||
/// tenants are workspace-scoped names and syncing them would make repo write access a second
|
||||
/// door onto the access decision.
|
||||
/// Stripped from the `datatable` setting wherever it is read or written as a whole. It leaves
|
||||
/// the instance only as the settings document's `datatable_permissions`, which `wmill sync`
|
||||
/// carries only when the repo opts in: tenants are workspace-scoped names, and syncing them
|
||||
/// makes repo write access a second door onto the access decision.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub permissions: Option<DataTablePermissions>,
|
||||
}
|
||||
|
||||
@@ -109,6 +109,9 @@ export const CONFIG_REFERENCE: ConfigOption[] = [
|
||||
commented: true, templateValue: "true" },
|
||||
{ name: "includeSettings", type: "boolean", default: "false", description: "Include workspace settings in sync",
|
||||
commented: true, templateValue: "true" },
|
||||
{ name: "includeDatatablePermissions", type: "boolean", default: "false",
|
||||
description: "With includeSettings, also sync each data table's roles and who may connect as them",
|
||||
commented: true, templateValue: "true" },
|
||||
{ name: "includeKey", type: "boolean", default: "false", description: "Include encryption key in sync",
|
||||
commented: true, templateValue: "true" },
|
||||
|
||||
|
||||
@@ -86,7 +86,8 @@ export async function downloadZip(
|
||||
skipWorkspaceDependencies?: boolean,
|
||||
skipDatatableMigrations?: boolean,
|
||||
defaultTs?: "bun" | "deno",
|
||||
syncBehavior?: string
|
||||
syncBehavior?: string,
|
||||
includeDatatablePermissions?: boolean,
|
||||
): Promise<JSZip | TarAsZip | undefined> {
|
||||
const requestHeaders = new Headers();
|
||||
requestHeaders.set("Authorization", "Bearer " + workspace.token);
|
||||
@@ -112,6 +113,7 @@ export async function downloadZip(
|
||||
}&skip_secrets=${skipSecrets ?? false}&include_schedules=${includeSchedules ?? false
|
||||
}&include_triggers=${includeTriggers ?? false}&include_users=${includeUsers ?? false
|
||||
}&include_groups=${includeGroups ?? false}&include_settings=${includeSettings ?? false
|
||||
}&include_datatable_permissions=${includeDatatablePermissions ?? false
|
||||
}&include_key=${includeKey ?? false}&include_workspace_dependencies=${includeWorkspaceDependenciesValue}&skip_datatable_migrations=${skipDatatableMigrations ?? false}&default_ts=${defaultTs ?? "bun"}&skip_resource_types=${skipResourceTypes ?? false}&settings_version=v2&preserve_extra_perms=true&sync_behavior_version=${syncBehavior ?? "v0"}`;
|
||||
|
||||
const baseUrl = workspace.remote + "api/w/" + workspace.workspaceId + "/workspaces/tarball?";
|
||||
|
||||
@@ -14,6 +14,7 @@ import { colors } from "@cliffy/ansi/colors";
|
||||
import { Command } from "@cliffy/command";
|
||||
import { Confirm } from "@cliffy/prompt/confirm";
|
||||
import * as log from "../../core/log.ts";
|
||||
import { SETTINGS_MANAGED_ONLY_WHEN_PRESENT } from "../../core/settings.ts";
|
||||
import * as path from "node:path";
|
||||
import { sep as SEP } from "node:path";
|
||||
import {
|
||||
@@ -2576,6 +2577,14 @@ const isWorkspaceSettingsFile = (p: string) =>
|
||||
function stripUnappliedSettingsFields(local: any, remote: any) {
|
||||
delete local?.name;
|
||||
delete remote?.name;
|
||||
for (const key of ["operator_settings", ...SETTINGS_MANAGED_ONLY_WHEN_PRESENT]) {
|
||||
if (local?.[key] === undefined) delete remote?.[key];
|
||||
}
|
||||
// push clears an omitted mute_critical_alerts to false
|
||||
if (local?.mute_critical_alerts == null && remote?.mute_critical_alerts === false) {
|
||||
delete local?.mute_critical_alerts;
|
||||
delete remote.mute_critical_alerts;
|
||||
}
|
||||
if (local?.color == null) {
|
||||
delete local?.color;
|
||||
delete remote?.color;
|
||||
@@ -3843,6 +3852,7 @@ export async function pull(
|
||||
opts.skipDatatableMigrations,
|
||||
opts.defaultTs,
|
||||
opts.syncBehavior,
|
||||
opts.includeDatatablePermissions,
|
||||
);
|
||||
|
||||
const remote = ZipFSElement(
|
||||
@@ -4946,6 +4956,7 @@ export async function push(
|
||||
opts.skipDatatableMigrations,
|
||||
opts.defaultTs,
|
||||
opts.syncBehavior,
|
||||
opts.includeDatatablePermissions,
|
||||
))!,
|
||||
!opts.json,
|
||||
opts.defaultTs ?? "bun",
|
||||
@@ -6049,6 +6060,7 @@ export async function push(
|
||||
(opts.yes ?? false) || !process.stdin.isTTY,
|
||||
skipReencrypt: opts.skipReencryptOnKeyChange,
|
||||
},
|
||||
includeDatatablePermissions: opts.includeDatatablePermissions,
|
||||
defaultTs: opts.defaultTs,
|
||||
},
|
||||
);
|
||||
@@ -6187,6 +6199,7 @@ export async function push(
|
||||
(opts.yes ?? false) || !process.stdin.isTTY,
|
||||
skipReencrypt: opts.skipReencryptOnKeyChange,
|
||||
},
|
||||
includeDatatablePermissions: opts.includeDatatablePermissions,
|
||||
defaultTs: opts.defaultTs,
|
||||
},
|
||||
);
|
||||
@@ -6815,6 +6828,10 @@ const command = new Command()
|
||||
.option("--include-users", "Include syncing users")
|
||||
.option("--include-groups", "Include syncing groups")
|
||||
.option("--include-settings", "Include syncing workspace settings")
|
||||
.option(
|
||||
"--include-datatable-permissions",
|
||||
"With --include-settings, also sync who may connect to each data table as which role",
|
||||
)
|
||||
.option("--include-key", "Include workspace encryption key")
|
||||
.option(
|
||||
"--keep-deleted",
|
||||
@@ -6878,6 +6895,10 @@ const command = new Command()
|
||||
.option("--include-users", "Include syncing users")
|
||||
.option("--include-groups", "Include syncing groups")
|
||||
.option("--include-settings", "Include syncing workspace settings")
|
||||
.option(
|
||||
"--include-datatable-permissions",
|
||||
"With --include-settings, also sync who may connect to each data table as which role",
|
||||
)
|
||||
.option("--include-key", "Include workspace encryption key")
|
||||
.option(
|
||||
"--skip-reencrypt-on-key-change",
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
import { colors } from "@cliffy/ansi/colors";
|
||||
import { Confirm } from "@cliffy/prompt/confirm";
|
||||
import { stringify as yamlStringify } from "yaml";
|
||||
import { GlobalOptions } from "../../types.ts";
|
||||
import * as log from "../../core/log.ts";
|
||||
import { requireLogin } from "../../core/auth.ts";
|
||||
import { resolveWorkspace } from "../../core/context.ts";
|
||||
import { yamlParseFile } from "../../utils/yaml.ts";
|
||||
import * as wmill from "../../../gen/services.gen.ts";
|
||||
|
||||
export async function getSettings(
|
||||
opts: GlobalOptions & { encrypted?: boolean; json?: boolean },
|
||||
) {
|
||||
await requireLogin(opts);
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
const document = await wmill.getWorkspaceSettingsDocument({
|
||||
workspace: workspace.workspaceId,
|
||||
secrets: opts.encrypted ? "encrypted" : "redacted",
|
||||
});
|
||||
console.log(
|
||||
opts.json ? JSON.stringify(document, null, 2) : yamlStringify(document),
|
||||
);
|
||||
}
|
||||
|
||||
export async function applySettings(
|
||||
opts: GlobalOptions & { dryRun?: boolean; replace?: boolean; yes?: boolean },
|
||||
file: string,
|
||||
) {
|
||||
await requireLogin(opts);
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
const document = await yamlParseFile(file);
|
||||
if (!document || typeof document !== "object" || Array.isArray(document)) {
|
||||
throw new Error(`${file} must hold a settings document (a mapping of setting to value)`);
|
||||
}
|
||||
const apply = (dry_run: boolean) =>
|
||||
wmill.applyWorkspaceSettingsDocument({
|
||||
workspace: workspace.workspaceId,
|
||||
requestBody: { document, replace: opts.replace ?? false, dry_run },
|
||||
});
|
||||
|
||||
const plan = await apply(true);
|
||||
for (const warning of plan.warnings ?? []) {
|
||||
log.warn(warning);
|
||||
}
|
||||
if (plan.changes.length === 0) {
|
||||
log.info(colors.green("Workspace settings are already up to date"));
|
||||
return;
|
||||
}
|
||||
for (const change of plan.changes) {
|
||||
log.info(
|
||||
`${colors.bold(change.key)}: ${JSON.stringify(change.before)} -> ${JSON.stringify(change.after)}`,
|
||||
);
|
||||
}
|
||||
if (opts.dryRun) {
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!opts.yes &&
|
||||
!(await Confirm.prompt({
|
||||
message: `Apply ${plan.changes.length} change(s) to workspace ${workspace.workspaceId}?`,
|
||||
default: true,
|
||||
}))
|
||||
) {
|
||||
return;
|
||||
}
|
||||
const result = await apply(false);
|
||||
for (const warning of result.warnings ?? []) {
|
||||
log.warn(warning);
|
||||
}
|
||||
log.info(
|
||||
colors.bold.underline.green(
|
||||
`Applied ${result.changes.length} change(s) to workspace ${workspace.workspaceId}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -18,6 +18,7 @@ import { requireLogin } from "../../core/auth.ts";
|
||||
import { createWorkspaceFork, deleteWorkspaceFork } from "./fork.ts";
|
||||
import { mergeWorkspaces } from "./merge.ts";
|
||||
import { connectSlack, disconnectSlack } from "./slack.ts";
|
||||
import { applySettings, getSettings } from "./settings.ts";
|
||||
|
||||
import * as wmill from "../../../gen/services.gen.ts";
|
||||
|
||||
@@ -905,6 +906,22 @@ Arguments (omit both to be prompted interactively):
|
||||
.description(
|
||||
"Clear slack_team_id / slack_name on the active workspace (marks the workspace as disconnected). Does NOT remove the bot token variable/resource/folder/group — delete those from the local sync folder and run 'wmill sync push' to tear them down. Does NOT remove the workspace-level OAuth override — set slack_oauth_client_id/_secret to '' in settings.yaml and push."
|
||||
)
|
||||
.action(disconnectSlack as any);
|
||||
.action(disconnectSlack as any)
|
||||
.command("get-settings")
|
||||
.description(
|
||||
"Print the active workspace's settings in the layout of the settings.yaml that 'wmill sync' writes. Secrets show as __SENSITIVE_AND_UNCHANGED__ unless --encrypted. Admin only."
|
||||
)
|
||||
.option("--encrypted", "Write secrets encrypted with the workspace key, as 'wmill sync pull' does")
|
||||
.option("--json", "Output as JSON")
|
||||
.action(getSettings as any)
|
||||
.command("apply-settings")
|
||||
.description(
|
||||
"Apply a settings document (YAML or JSON, in the layout of settings.yaml) to the active workspace. Only the settings the file states are changed, unless --replace; a setting set to null is cleared. A secret left as __SENSITIVE_AND_UNCHANGED__ keeps its value. Shows the changes and asks before applying. Admin only."
|
||||
)
|
||||
.arguments("<file:string>")
|
||||
.option("--dry-run", "Only show the changes")
|
||||
.option("--replace", "Also clear the settings the file omits, as 'wmill sync push' does")
|
||||
.option("-y --yes", "Apply without asking")
|
||||
.action(applySettings as any);
|
||||
|
||||
export default command;
|
||||
|
||||
@@ -89,6 +89,9 @@ export interface SyncOptions {
|
||||
includeUsers?: boolean;
|
||||
includeGroups?: boolean;
|
||||
includeSettings?: boolean;
|
||||
/** With includeSettings: settings.yaml also states who may connect to each data table as which
|
||||
* role. Off by default, since tenants and roles are specific to a workspace and an instance. */
|
||||
includeDatatablePermissions?: boolean;
|
||||
includeKey?: boolean;
|
||||
skipReencryptOnKeyChange?: boolean;
|
||||
skipBranchValidation?: boolean;
|
||||
@@ -388,6 +391,7 @@ export const DEFAULT_SYNC_OPTIONS: Readonly<
|
||||
| "includeUsers"
|
||||
| "includeGroups"
|
||||
| "includeSettings"
|
||||
| "includeDatatablePermissions"
|
||||
| "includeKey"
|
||||
| "nonDottedPaths"
|
||||
| "syncBehavior"
|
||||
@@ -410,6 +414,7 @@ export const DEFAULT_SYNC_OPTIONS: Readonly<
|
||||
includeUsers: false,
|
||||
includeGroups: false,
|
||||
includeSettings: false,
|
||||
includeDatatablePermissions: false,
|
||||
includeKey: false,
|
||||
skipWorkspaceDependencies: false,
|
||||
skipDatatableMigrations: false,
|
||||
|
||||
@@ -58,8 +58,35 @@ export interface SimplifiedSettings {
|
||||
slack_command_script?: string;
|
||||
slack_oauth_client_id?: string;
|
||||
slack_oauth_client_secret?: string;
|
||||
ducklake?: any;
|
||||
dbt_warehouses?: any;
|
||||
deploy_ui?: any;
|
||||
teams_command_script?: string | null;
|
||||
public_app_execution_limit_per_minute?: number | null;
|
||||
guest_access_enabled?: boolean;
|
||||
guest_jwt_public_key?: string | null;
|
||||
guest_jwt_jwks_url?: string | null;
|
||||
error_handler_fallback_to_instance_alerts?: boolean;
|
||||
add_admins_and_developers_to_forks?: boolean;
|
||||
datatable_permissions?: Record<string, unknown> | null;
|
||||
}
|
||||
|
||||
/** Settings the server manages only when settings.yaml states them: a file written before
|
||||
* they existed must neither clear them nor show as differing from the workspace. */
|
||||
export const SETTINGS_MANAGED_ONLY_WHEN_PRESENT = [
|
||||
"datatable_permissions",
|
||||
"ducklake",
|
||||
"dbt_warehouses",
|
||||
"deploy_ui",
|
||||
"teams_command_script",
|
||||
"public_app_execution_limit_per_minute",
|
||||
"guest_access_enabled",
|
||||
"guest_jwt_public_key",
|
||||
"guest_jwt_jwks_url",
|
||||
"error_handler_fallback_to_instance_alerts",
|
||||
"add_admins_and_developers_to_forks",
|
||||
] as const;
|
||||
|
||||
// Legacy settings interface for reading old settings.yaml files
|
||||
interface LegacySimplifiedSettings {
|
||||
auto_invite_enabled?: boolean;
|
||||
@@ -108,6 +135,9 @@ export function migrateToGroupedFormat(settings: any): SimplifiedSettings {
|
||||
if (settings.slack_command_script !== undefined) result.slack_command_script = settings.slack_command_script;
|
||||
if (settings.slack_oauth_client_id !== undefined) result.slack_oauth_client_id = settings.slack_oauth_client_id;
|
||||
if (settings.slack_oauth_client_secret !== undefined) result.slack_oauth_client_secret = settings.slack_oauth_client_secret;
|
||||
for (const key of SETTINGS_MANAGED_ONLY_WHEN_PRESENT) {
|
||||
if (settings[key] !== undefined) (result as any)[key] = settings[key];
|
||||
}
|
||||
|
||||
// Handle auto_invite: check if already grouped or needs migration
|
||||
if (settings.auto_invite && typeof settings.auto_invite === "object") {
|
||||
@@ -173,10 +203,37 @@ export async function pushWorkspaceSettings(
|
||||
workspace: string,
|
||||
_path: string,
|
||||
settings: SimplifiedSettings | undefined,
|
||||
localSettings: SimplifiedSettings | any
|
||||
localSettings: SimplifiedSettings | any,
|
||||
opts: { includeDatatablePermissions?: boolean } = {},
|
||||
) {
|
||||
if (localSettings?.protection_rules !== undefined) {
|
||||
log.warn(
|
||||
"Ignoring protection_rules in settings.yaml: protection-rules.yaml syncs them (wmill protection-rules push)."
|
||||
);
|
||||
}
|
||||
// Migrate local settings from legacy format if needed
|
||||
localSettings = migrateToGroupedFormat(localSettings);
|
||||
if (!opts.includeDatatablePermissions) {
|
||||
delete localSettings.datatable_permissions;
|
||||
}
|
||||
|
||||
if (await applySettingsDocument(workspace, localSettings)) {
|
||||
return;
|
||||
}
|
||||
// The per-setting endpoints of a server without the settings document know nothing of
|
||||
// these keys, nor of a secret it would have to decrypt or keep.
|
||||
for (const key of SETTINGS_MANAGED_ONLY_WHEN_PRESENT) {
|
||||
delete (localSettings as any)[key];
|
||||
}
|
||||
const secret = localSettings.slack_oauth_client_secret;
|
||||
const slackSecretUnreadable =
|
||||
typeof secret === "string" &&
|
||||
(secret.startsWith("$encrypted:") || secret === "__SENSITIVE_AND_UNCHANGED__");
|
||||
if (slackSecretUnreadable) {
|
||||
log.warn(
|
||||
"Leaving the Slack OAuth client and secret unchanged: this server cannot read an encrypted or redacted secret. Upgrade it, or write the secret in plain text."
|
||||
);
|
||||
}
|
||||
|
||||
try {
|
||||
const remoteSettings = await wmill.getSettings({
|
||||
@@ -456,8 +513,9 @@ export async function pushWorkspaceSettings(
|
||||
// every other setting): both present → upsert; anything else → delete.
|
||||
// Pull always emits both fields as null when remote is NULL.
|
||||
if (
|
||||
localSettings.slack_oauth_client_id != settings.slack_oauth_client_id ||
|
||||
localSettings.slack_oauth_client_secret != settings.slack_oauth_client_secret
|
||||
!slackSecretUnreadable &&
|
||||
(localSettings.slack_oauth_client_id != settings.slack_oauth_client_id ||
|
||||
localSettings.slack_oauth_client_secret != settings.slack_oauth_client_secret)
|
||||
) {
|
||||
log.debug(`Updating slack oauth config...`);
|
||||
if (
|
||||
@@ -477,6 +535,38 @@ export async function pushWorkspaceSettings(
|
||||
}
|
||||
}
|
||||
|
||||
/** Applies settings.yaml server-side, which owns the rules for what an omitted or null setting
|
||||
* means. Returns false on a server that predates the settings document. */
|
||||
async function applySettingsDocument(
|
||||
workspace: string,
|
||||
document: SimplifiedSettings,
|
||||
): Promise<boolean> {
|
||||
let result;
|
||||
try {
|
||||
result = await wmill.applyWorkspaceSettingsDocument({
|
||||
workspace,
|
||||
requestBody: { document: document as unknown as Record<string, unknown>, replace: true },
|
||||
});
|
||||
} catch (err: any) {
|
||||
// A missing route answers 404 with no body; a handler's own 404 carries a message.
|
||||
if (err?.status === 404 && !err?.body) {
|
||||
return false;
|
||||
}
|
||||
throw new Error(`Failed to apply workspace settings: ${err?.body ?? err}`);
|
||||
}
|
||||
if (result.changes.length > 0) {
|
||||
log.debug(
|
||||
`Updated workspace settings: ${result.changes.map((c) => c.key).join(", ")}`,
|
||||
);
|
||||
} else {
|
||||
log.debug(`Workspace settings are up to date`);
|
||||
}
|
||||
for (const warning of result.warnings ?? []) {
|
||||
log.warn(warning);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
export interface PushWorkspaceKeyOptions {
|
||||
// True when no prompt may be shown (e.g. `--yes` was passed or stdin is not a
|
||||
// TTY). In that case the re-encryption decision is taken from `skipReencrypt`
|
||||
|
||||
Generated
+9
@@ -8889,6 +8889,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- \`--include-users\` - Include syncing users
|
||||
- \`--include-groups\` - Include syncing groups
|
||||
- \`--include-settings\` - Include syncing workspace settings
|
||||
- \`--include-datatable-permissions\` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- \`--include-key\` - Include workspace encryption key
|
||||
- \`--keep-deleted\` - Do not delete local files for items that no longer exist on the remote workspace. Only adds and updates.
|
||||
- \`--skip-branch-validation\` - Skip git branch validation and prompts
|
||||
@@ -8920,6 +8921,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- \`--include-users\` - Include syncing users
|
||||
- \`--include-groups\` - Include syncing groups
|
||||
- \`--include-settings\` - Include syncing workspace settings
|
||||
- \`--include-datatable-permissions\` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- \`--include-key\` - Include workspace encryption key
|
||||
- \`--skip-reencrypt-on-key-change\` - When the pushed encryption key differs from the remote, do NOT re-encrypt existing remote secrets. Only safe if they are already encrypted with the new key (e.g. workspace/instance migration). Default is to re-encrypt.
|
||||
- \`--keep-deleted\` - Do not delete remote items that no longer exist locally. Only adds and updates.
|
||||
@@ -9113,6 +9115,13 @@ Arguments (omit both to be prompted interactively):
|
||||
- \`--team-id <team_id:string>\` - Slack team id
|
||||
- \`--team-name <team_name:string>\` - Slack team name
|
||||
- \`workspace disconnect-slack\` - Clear slack_team_id / slack_name on the active workspace (marks the workspace as disconnected). Does NOT remove the bot token variable/resource/folder/group — delete those from the local sync folder and run 'wmill sync push' to tear them down. Does NOT remove the workspace-level OAuth override — set slack_oauth_client_id/_secret to '' in settings.yaml and push.
|
||||
- \`workspace get-settings\` - Print the active workspace's settings in the layout of the settings.yaml that 'wmill sync' writes. Secrets show as __SENSITIVE_AND_UNCHANGED__ unless --encrypted. Admin only.
|
||||
- \`--encrypted\` - Write secrets encrypted with the workspace key, as 'wmill sync pull' does
|
||||
- \`--json\` - Output as JSON
|
||||
- \`workspace apply-settings <file:string>\` - Apply a settings document (YAML or JSON, in the layout of settings.yaml) to the active workspace. Only the settings the file states are changed, unless --replace; a setting set to null is cleared. A secret left as __SENSITIVE_AND_UNCHANGED__ keeps its value. Shows the changes and asks before applying. Admin only.
|
||||
- \`--dry-run\` - Only show the changes
|
||||
- \`--replace\` - Also clear the settings the file omits, as 'wmill sync push' does
|
||||
- \`-y --yes\` - Apply without asking
|
||||
|
||||
|
||||
|
||||
|
||||
+6
-1
@@ -187,6 +187,8 @@ export interface PushObjOptions {
|
||||
wsSpecific?: boolean;
|
||||
/** encryption_key push: non-interactive flag and explicit re-encryption choice */
|
||||
keyPushOpts?: PushWorkspaceKeyOptions;
|
||||
/** settings push: whether settings.yaml manages the data tables' roles */
|
||||
includeDatatablePermissions?: boolean;
|
||||
/** TypeScript runtime a bare `.ts` denotes, for raw-app runnables */
|
||||
defaultTs?: "bun" | "deno";
|
||||
/** schedule push into a fork: the file's `enabled` is the parent's */
|
||||
@@ -218,6 +220,7 @@ export async function pushObj(
|
||||
permissionedAsContext,
|
||||
wsSpecific,
|
||||
keyPushOpts,
|
||||
includeDatatablePermissions,
|
||||
defaultTs,
|
||||
enabledOwnedByParent,
|
||||
} = opts;
|
||||
@@ -285,7 +288,9 @@ export async function pushObj(
|
||||
} else if (typeEnding === "workspace_dependencies") {
|
||||
await pushWorkspaceDependencies(workspace, p, befObj, newObj);
|
||||
} else if (typeEnding === "settings") {
|
||||
await pushWorkspaceSettings(workspace, p, befObj, newObj);
|
||||
await pushWorkspaceSettings(workspace, p, befObj, newObj, {
|
||||
includeDatatablePermissions,
|
||||
});
|
||||
} else if (typeEnding === "encryption_key") {
|
||||
await pushWorkspaceKey(workspace, p, befObj, newObj, keyPushOpts);
|
||||
} else {
|
||||
|
||||
@@ -21,6 +21,10 @@ const remoteAutoInvite = {
|
||||
// Every wmill.* call reachable from pushWorkspaceSettings is stubbed so the
|
||||
// function runs without a backend.
|
||||
mockServices({
|
||||
// A server without the settings document, so the per-setting endpoints run.
|
||||
applyWorkspaceSettingsDocument: async () => {
|
||||
throw { status: 404 };
|
||||
},
|
||||
getSettings: async () => ({ auto_invite: remoteAutoInvite }),
|
||||
getWorkspaceName: async () => "phoenix",
|
||||
changeWorkspaceName: async () => {},
|
||||
|
||||
@@ -17,6 +17,10 @@ let remoteWebhook: string | undefined = undefined;
|
||||
// Every wmill.* call reachable from pushWorkspaceSettings is stubbed so the
|
||||
// function runs without a backend; only the three we assert on record calls.
|
||||
mockServices({
|
||||
// A server without the settings document, so the per-setting endpoints run.
|
||||
applyWorkspaceSettingsDocument: async () => {
|
||||
throw { status: 404 };
|
||||
},
|
||||
getSettings: async (_a: { workspace: string }) => ({
|
||||
webhook: remoteWebhook,
|
||||
color: remoteColor,
|
||||
|
||||
@@ -618,6 +618,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- `--include-users` - Include syncing users
|
||||
- `--include-groups` - Include syncing groups
|
||||
- `--include-settings` - Include syncing workspace settings
|
||||
- `--include-datatable-permissions` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- `--include-key` - Include workspace encryption key
|
||||
- `--keep-deleted` - Do not delete local files for items that no longer exist on the remote workspace. Only adds and updates.
|
||||
- `--skip-branch-validation` - Skip git branch validation and prompts
|
||||
@@ -649,6 +650,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- `--include-users` - Include syncing users
|
||||
- `--include-groups` - Include syncing groups
|
||||
- `--include-settings` - Include syncing workspace settings
|
||||
- `--include-datatable-permissions` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- `--include-key` - Include workspace encryption key
|
||||
- `--skip-reencrypt-on-key-change` - When the pushed encryption key differs from the remote, do NOT re-encrypt existing remote secrets. Only safe if they are already encrypted with the new key (e.g. workspace/instance migration). Default is to re-encrypt.
|
||||
- `--keep-deleted` - Do not delete remote items that no longer exist locally. Only adds and updates.
|
||||
@@ -842,6 +844,13 @@ Arguments (omit both to be prompted interactively):
|
||||
- `--team-id <team_id:string>` - Slack team id
|
||||
- `--team-name <team_name:string>` - Slack team name
|
||||
- `workspace disconnect-slack` - Clear slack_team_id / slack_name on the active workspace (marks the workspace as disconnected). Does NOT remove the bot token variable/resource/folder/group — delete those from the local sync folder and run 'wmill sync push' to tear them down. Does NOT remove the workspace-level OAuth override — set slack_oauth_client_id/_secret to '' in settings.yaml and push.
|
||||
- `workspace get-settings` - Print the active workspace's settings in the layout of the settings.yaml that 'wmill sync' writes. Secrets show as __SENSITIVE_AND_UNCHANGED__ unless --encrypted. Admin only.
|
||||
- `--encrypted` - Write secrets encrypted with the workspace key, as 'wmill sync pull' does
|
||||
- `--json` - Output as JSON
|
||||
- `workspace apply-settings <file:string>` - Apply a settings document (YAML or JSON, in the layout of settings.yaml) to the active workspace. Only the settings the file states are changed, unless --replace; a setting set to null is cleared. A secret left as __SENSITIVE_AND_UNCHANGED__ keeps its value. Shows the changes and asks before applying. Admin only.
|
||||
- `--dry-run` - Only show the changes
|
||||
- `--replace` - Also clear the settings the file omits, as 'wmill sync push' does
|
||||
- `-y --yes` - Apply without asking
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -4262,6 +4262,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- \`--include-users\` - Include syncing users
|
||||
- \`--include-groups\` - Include syncing groups
|
||||
- \`--include-settings\` - Include syncing workspace settings
|
||||
- \`--include-datatable-permissions\` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- \`--include-key\` - Include workspace encryption key
|
||||
- \`--keep-deleted\` - Do not delete local files for items that no longer exist on the remote workspace. Only adds and updates.
|
||||
- \`--skip-branch-validation\` - Skip git branch validation and prompts
|
||||
@@ -4293,6 +4294,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- \`--include-users\` - Include syncing users
|
||||
- \`--include-groups\` - Include syncing groups
|
||||
- \`--include-settings\` - Include syncing workspace settings
|
||||
- \`--include-datatable-permissions\` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- \`--include-key\` - Include workspace encryption key
|
||||
- \`--skip-reencrypt-on-key-change\` - When the pushed encryption key differs from the remote, do NOT re-encrypt existing remote secrets. Only safe if they are already encrypted with the new key (e.g. workspace/instance migration). Default is to re-encrypt.
|
||||
- \`--keep-deleted\` - Do not delete remote items that no longer exist locally. Only adds and updates.
|
||||
@@ -4486,6 +4488,13 @@ Arguments (omit both to be prompted interactively):
|
||||
- \`--team-id <team_id:string>\` - Slack team id
|
||||
- \`--team-name <team_name:string>\` - Slack team name
|
||||
- \`workspace disconnect-slack\` - Clear slack_team_id / slack_name on the active workspace (marks the workspace as disconnected). Does NOT remove the bot token variable/resource/folder/group — delete those from the local sync folder and run 'wmill sync push' to tear them down. Does NOT remove the workspace-level OAuth override — set slack_oauth_client_id/_secret to '' in settings.yaml and push.
|
||||
- \`workspace get-settings\` - Print the active workspace's settings in the layout of the settings.yaml that 'wmill sync' writes. Secrets show as __SENSITIVE_AND_UNCHANGED__ unless --encrypted. Admin only.
|
||||
- \`--encrypted\` - Write secrets encrypted with the workspace key, as 'wmill sync pull' does
|
||||
- \`--json\` - Output as JSON
|
||||
- \`workspace apply-settings <file:string>\` - Apply a settings document (YAML or JSON, in the layout of settings.yaml) to the active workspace. Only the settings the file states are changed, unless --replace; a setting set to null is cleared. A secret left as __SENSITIVE_AND_UNCHANGED__ keeps its value. Shows the changes and asks before applying. Admin only.
|
||||
- \`--dry-run\` - Only show the changes
|
||||
- \`--replace\` - Also clear the settings the file omits, as 'wmill sync push' does
|
||||
- \`-y --yes\` - Apply without asking
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -623,6 +623,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- `--include-users` - Include syncing users
|
||||
- `--include-groups` - Include syncing groups
|
||||
- `--include-settings` - Include syncing workspace settings
|
||||
- `--include-datatable-permissions` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- `--include-key` - Include workspace encryption key
|
||||
- `--keep-deleted` - Do not delete local files for items that no longer exist on the remote workspace. Only adds and updates.
|
||||
- `--skip-branch-validation` - Skip git branch validation and prompts
|
||||
@@ -654,6 +655,7 @@ sync local with a remote workspaces or the opposite (push or pull)
|
||||
- `--include-users` - Include syncing users
|
||||
- `--include-groups` - Include syncing groups
|
||||
- `--include-settings` - Include syncing workspace settings
|
||||
- `--include-datatable-permissions` - With --include-settings, also sync who may connect to each data table as which role
|
||||
- `--include-key` - Include workspace encryption key
|
||||
- `--skip-reencrypt-on-key-change` - When the pushed encryption key differs from the remote, do NOT re-encrypt existing remote secrets. Only safe if they are already encrypted with the new key (e.g. workspace/instance migration). Default is to re-encrypt.
|
||||
- `--keep-deleted` - Do not delete remote items that no longer exist locally. Only adds and updates.
|
||||
@@ -847,6 +849,13 @@ Arguments (omit both to be prompted interactively):
|
||||
- `--team-id <team_id:string>` - Slack team id
|
||||
- `--team-name <team_name:string>` - Slack team name
|
||||
- `workspace disconnect-slack` - Clear slack_team_id / slack_name on the active workspace (marks the workspace as disconnected). Does NOT remove the bot token variable/resource/folder/group — delete those from the local sync folder and run 'wmill sync push' to tear them down. Does NOT remove the workspace-level OAuth override — set slack_oauth_client_id/_secret to '' in settings.yaml and push.
|
||||
- `workspace get-settings` - Print the active workspace's settings in the layout of the settings.yaml that 'wmill sync' writes. Secrets show as __SENSITIVE_AND_UNCHANGED__ unless --encrypted. Admin only.
|
||||
- `--encrypted` - Write secrets encrypted with the workspace key, as 'wmill sync pull' does
|
||||
- `--json` - Output as JSON
|
||||
- `workspace apply-settings <file:string>` - Apply a settings document (YAML or JSON, in the layout of settings.yaml) to the active workspace. Only the settings the file states are changed, unless --replace; a setting set to null is cleared. A secret left as __SENSITIVE_AND_UNCHANGED__ keeps its value. Shows the changes and asks before applying. Admin only.
|
||||
- `--dry-run` - Only show the changes
|
||||
- `--replace` - Also clear the settings the file omits, as 'wmill sync push' does
|
||||
- `-y --yes` - Apply without asking
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user