starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-10-27 00:02:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: run a workspace handler when a variable's value expires (#10837)

Browse Source 
* feat: run a workspace handler when a secret variable expires

Secret variables gain an optional `value_expires_at`, and a workspace can point
at a script or flow to run once that date passes, so a rotation is driven by the
platform instead of tracked out of band.

The secret never leaves the database. The sweep selects the path, description and
expiry and nothing else, so the handler is given a reference to rotate rather than
a value to handle. Job results are stored in cleartext and shown in Runs, which is
what makes that a boundary rather than a preference; a test asserts the job's args
never contain the value.

Each row is claimed by writing `expiration_dispatched_for` in the same transaction
as the push, so a failed push un-claims it and a lost race claims nothing. Moving
the date re-arms the row; re-saving the same date does not. A pass takes at most
100 rows under a transaction-scoped advisory lock, and orders so that one broken
workspace cannot starve the rest.

Dispatch runs as `g/variable_expiration_handler`, exempt from the CE group limit
and from non-premium exec metering the way the error and schedule handlers already
are. A handler that keeps failing backs off and raises a critical alert instead of
retrying every pass, and the back-off ends when the handler is repointed.

The expiry travels with the variable through workspace clone, tarball export and
workspace comparison. The dispatch marker deliberately does not, so a fork re-arms
on its own copy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(frontend): expose variable value expiration in the UI

The variable drawer takes an optional expiry on a secret value, and the variables
list badges it, turning red once the date has passed. A workspace settings tab
picks the handler, its extra args, and whether variables under `u/` are skipped.

That mute flag is stored inside the handler setting, so the toggle stays disabled
until a handler is picked: saving it on its own would clear the setting it lives
in and still report success.

The expiry is carried as a tri-state through the draft deploy and copilot paths —
absent leaves the remote value alone, an explicit null clears it — so deploying a
draft cannot silently drop a date someone set elsewhere.

The settings tab names the one case where the configured identity does not apply:
a handler script carrying `on_behalf_of` runs as that user rather than as
`g/variable_expiration_handler`, which matters because the same tab asks the admin
to grant that group write access to the variables it rotates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(cli): sync variable expiry and the workspace expiration handler

`wmill sync` carries `value_expires_at` on variables in both directions, and
pushes the workspace handler alongside the error and success handlers in the
grouped settings format. Dropping the key from a variable's yaml clears the
remote expiry, so the file stays canonical rather than accumulating fields that
can only ever be added.

The handler follows the same rule, with the caveat recorded where the type is
declared: an omitted handler only reaches the clear when some other setting
differs, because the push short-circuits on a comparison that walks the local
object's keys and so never sees a missing one. That gate is shared with the two
existing handlers and is left as it is here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(frontend): open the variable expiration handler template in a new tab

The "Create a handler" button hands its script template to /scripts/add through
the URL hash, which `load` reads from `window.location`. A client-side navigation
leaves the previous page's hash in place, so the payload was dropped and the
editor opened on the default template at a `u/` path. `target="_blank"` forces a
full page load, matching the success handler's "Create from template" button.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(frontend): make the variable expiry field state its own status

The expiry field rendered a DateTimeInput whose time defaults to 12:00 when it
holds no value, so a variable with no expiry looked like one already configured,
and "cleared" looked identical to "never set". Setting an expiry is optional, so
the field has to say which it is.

Rebuild it on the shape ScriptAdvancedSettings already uses for Timeout, Cache and
Concurrency limits: a toggle whose checked state is the stored value being present,
seeding a date on enable rather than revealing an empty picker, with the input
under its own label. Drop `clearable` with it, so the toggle is the single way to
mean "no expiry" instead of competing with an X.

Stop the surrounding copy promising a run that may not happen. A handler is only
dispatched if the workspace has one configured and does not skip u/ paths, so the
list tooltip now states the date alone, and the form hedges both conditions,
naming the u/ mute only on personal paths where it can apply.

Also anchor the mute toggle's tooltip: Popover's trigger is a span around a
block-level label, which stretched to the full width of its SettingCard and left
the popup floating beside the card rather than under the control.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(rust-client): initialize value_expires_at in the variable models

The generated `CreateVariable` / `EditVariable` structs gain the field from
openapi.yaml, and client.rs builds both with exhaustive literals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(frontend): drop extra_args when the variable expiration handler is cleared

Clearing the handler nulls the whole setting server-side, extra_args included,
but the tab kept its loaded copy. A handler picked afterwards without reloading
was persisted with the cleared one's arguments, which the tab does not display.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: back the variable expiration handler's built-in Slack and Teams destinations

Backfills the Slack bot token grant for g/variable_expiration_handler, mirroring
20231016142659 for g/error_handler: the slack_bot folder's extra_perms is written
once at Slack connect and never refreshed, and disconnecting leaves the folder
behind, so workspaces that already connected Slack cannot pick the grant up from
the creation-time list alone.

Adds run_variable_expiration_test_job so an admin can exercise the handler on
demand rather than waiting for a real expiry, which may be months away. It goes
through push_variable_expiration_handler so the test run proves what matters:
the handler resolves, and the group can read what it needs, under the same
identity and argument shape the sweep uses.

Also covers extra_args on the edit endpoint, which the settings tab now writes
for the built-ins' channel. The endpoint rebuilds the setting from the request,
so an omitted extra_args erases it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* feat(frontend): offer Slack and Teams as variable expiration handlers

Gives the variable expiration tab the same no-code destinations the workspace
error handler has, so "tell #ops before this credential expires" no longer
requires writing a script.

A sibling component rather than a fourth mode of ErrorOrRecoveryHandler: that
one is shared with the workspace error handler and the three schedule handlers,
and its stripped-args list, hub path suffixes and Email tab are all
error-shaped. This reuses the pieces that are already generic
(SlackConnectionStatus, TeamsConnectionStatus, ChannelSelector, ScriptPicker,
SchemaForm) and leaves those five call sites untouched.

extra_args stops being an API/CLI-only escape hatch now that the built-ins write
their channel into it, so it joins the unsaved-changes comparison and the
discard path, normalised through the existing helper that drops the two
auto-managed keys.

The two hub ids are placeholders until the scripts are pushed to the hub.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* fix(frontend): scope the handler identity notice to the chosen destination

The notice told every admin to grant g/variable_expiration_handler write access
to the variables it rotates. That is right for a custom handler and wrong for
the built-ins, which only post a message and are granted the Slack bot token by
migration. Following it for a Slack handler would hand a group write access to
the workspace's secrets for no reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* feat(frontend): flag the Slack and Teams handlers as EE

Connecting a workspace to Slack or Teams is EE-only (oauth2_oss.rs returns
"Slack only available on enterprise"), so on CE those two tabs can never be
completed. Says so up front, the way the error/success handler tab does,
instead of leaving the user at a "not connected" badge with no working path.

Only on those two tabs: a custom script or flow handler is CE.

Drops the "Connect Slack first" alert, now that the banner and the connection
badge above it already say it twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* chore(backend): regenerate the sqlx cache after the merge

The merged workspace_settings column list in change_workspace_id is a new
query text, so it needs a new cached entry. Drops the two pre-merge entries
it supersedes; both had been left with conflict markers by the merge, since
rename/rename markers are eight characters and escape the usual scan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* fix(cli): narrow the expiry-clear check to a variable that exists remotely

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* feat(frontend): point the built-in handlers at their published hub scripts

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(backend): regenerate the sqlx cache after the merge

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): disable the Slack and Teams handler controls without a license

Matches the workspace error handler, which greys out its built-in destinations
and hides the test button on an unlicensed instance. The custom destination stays
editable: unlike the error handler, the variable expiration handler itself is not
an EE feature.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: record which destination a variable expiration handler uses

handler_configured now counts slack, teams, custom or cleared instead of
set or cleared, so usage shows whether the built-in destinations are picked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(frontend): let the variable expiration handler template run on any worker

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: guard the variable expiration test endpoint and correct its help text

* fix: reuse the error handler's slack arg and built-in path helpers

* docs: trim variable expiration sweep and backfill comments

* feat: put the variable value expiration under an advanced section

* fix: restrict the expiration test job to built-ins and keep expiry on conflict reload

* fix: link the expiry hint to the variable's workspace and document built-in test paths

* fix: give the expiration sweep its own lock id and reuse double_option

* fix: pin the variable expiration presets so the hub-paths test accepts them

* feat: let the variable expiration handler set how far ahead it runs

* fix: give the notify before expiry setting its own card

* feat: remind the variable expiration handler at a set interval

* fix: tighten the gap between the Slack channel field and its test button

* feat: refuse a value expiry on variables linked to an OAuth account

* refactor: reuse the shared handler form for variable expiration

* refactor: sweep expiring variables round-robin and check the handler exists

* chore: trim comments to the constraints they record

* fix: compare expiry dates by moment when deploying a variable draft

* refactor: rename value_expires_at to value_valid_until

* fix: neutralize variable fields in expiration handler args

* fix: reserve the variable_expiration_handler group name for admins

* fix: send first expiry notices before reminders

* test: pin that a handler failing to start leaves its rows unclaimed

* chore: say any hub handler path skips the existence check

* fix: empty a pre-existing handler group before granting it the Slack token

* docs: correct the registered feature usage counts

* fix: let a workspace whose handler failed to start sit out for an hour

* fix: send first notices before reminders across workspaces

* fix: end a failed handler's wait as soon as it is redeployed

* fix: show an expiry-only variable change in the AI fork diff

* chore: regenerate workspace tool schemas

* test: keep one variable expiration guard per behavior

* fix: use the renamed hub expiration handlers and template

* feat: carry the variable expiry in windmill-utils-internal 1.8.4

* fix: keep the variable expiry through MCP drafts

* chore: bump windmill-utils-internal to 1.8.4

* fix: keep the variable expiry through mcp deploys

* fix: disable saving a slack or teams expiry handler without ee

* fix: skip the failure mark when the expiry handler was saved mid-sweep

* fix: count expiry telemetry only where the expiry changed

* test: assert the exact variable expiry in export and cli push tests

* fix: stamp an expiry handler failure with the pass's read time

* fix: wait for a pending handler lock instead of marking it failed

* test: isolate expiry handler tests from the deployed-script cache

* fix: retry a failing expiry handler every pass instead of waiting an hour

* chore: update ee-repo-ref to b6e23d229966cf0019c0a1926a254fda658d8def

This commit updates the EE repository reference after PR #744 was merged in windmill-ee-private.

Previous ee-repo-ref: c52c06d145b13a6fb04459e85ca5818bf3c1bff4

New ee-repo-ref: b6e23d229966cf0019c0a1926a254fda658d8def

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
This commit is contained in:
AlexRV12
2026-10-23 18:12:55 +02:00
committed by GitHub
co-authored by Claude Opus 5.5 windmill-internal-app[bot] hugocasa
parent 18c69360c9
commit 7a29abc7ee
83 changed files with 3230 additions and 162 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/.sqlx/query-4ff75cdf9ee681cccb4b793704c9d154f36986ee0da73725a40c860037b63310.json → backend/.sqlx/query-30f69fd4baef17716d26cc59bf6369d6768972478def50010f680103d7c4281b.json
Generated
+9 -3
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value::text AS \"value!\", is_secret, description\n FROM variable\n WHERE workspace_id = $1 AND path = $2",
"query": "SELECT value::text AS \"value!\", is_secret, description, value_valid_until\n FROM variable\n WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
@@ -17,6 +17,11 @@
"ordinal": 2,
"name": "description",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "value_valid_until",
"type_info": "Timestamptz"
}
],
"parameters": {
@@ -28,8 +33,9 @@
"nullable": [
null,
false,
false
false,
true
]
},
"hash": "4ff75cdf9ee681cccb4b793704c9d154f36986ee0da73725a40c860037b63310"
"hash": "30f69fd4baef17716d26cc59bf6369d6768972478def50010f680103d7c4281b"
}
backend/.sqlx/query-32c7dba63931c9bd35f82f145d095bcfb8f9400809ade1b173bac05d61bfe77a.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO variable\n (workspace_id, path, value, is_secret, description, account, is_oauth, expires_at, value_valid_until, labels, edited_by)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Int4",
"Bool",
"Timestamptz",
"Timestamptz",
"TextArray",
"Varchar"
]
},
"nullable": []
},
"hash": "32c7dba63931c9bd35f82f145d095bcfb8f9400809ade1b173bac05d61bfe77a"
}
backend/.sqlx/query-37275b26b092ed3ae1a11b78da5227a40a49d9435107e71033740f11cc8d2417.json
Generated
-15
View File
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "37275b26b092ed3ae1a11b78da5227a40a49d9435107e71033740f11cc8d2417"
}
backend/.sqlx/query-3c951e76bd0d86a488372e54fdfe4f5a2923cff642905c6410267be2eab24853.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM script WHERE workspace_id = $1 AND path = $2 AND archived = false AND deleted = false)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "3c951e76bd0d86a488372e54fdfe4f5a2923cff642905c6410267be2eab24853"
}
backend/.sqlx/query-4ba98c3b07a3bd2fa51e4ddf93c182b3ac165cb6a343ad616409fa80b4b93484.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET variable_expiration_failed_at = NULL\n WHERE workspace_id = $1 AND variable_expiration_failed_at IS NOT NULL\n RETURNING 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "4ba98c3b07a3bd2fa51e4ddf93c182b3ac165cb6a343ad616409fa80b4b93484"
}
backend/.sqlx/query-74742f785888f280bf1d7e28793150c3ddc2fd4be3b8c682f6faa7b917387593.json
Generated
+214
View File
@@ -0,0 +1,214 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n customer_id,\n plan,\n webhook,\n ai_config,\n dbt_warehouses,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_invite,\n error_handler,\n success_handler,\n variable_expiration_handler,\n public_app_execution_limit_per_minute,\n error_handler_fallback_to_instance_alerts\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "slack_team_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "teams_team_id",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "teams_team_name",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "teams_team_guid",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "slack_name",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "slack_command_script",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "teams_command_script",
"type_info": "Text"
},
{
"ordinal": 8,
"name": "slack_email",
"type_info": "Varchar"
},
{
"ordinal": 9,
"name": "slack_oauth_client_id",
"type_info": "Varchar"
},
{
"ordinal": 10,
"name": "slack_oauth_client_secret",
"type_info": "Varchar"
},
{
"ordinal": 11,
"name": "customer_id",
"type_info": "Varchar"
},
{
"ordinal": 12,
"name": "plan",
"type_info": "Varchar"
},
{
"ordinal": 13,
"name": "webhook",
"type_info": "Text"
},
{
"ordinal": 14,
"name": "ai_config",
"type_info": "Jsonb"
},
{
"ordinal": 15,
"name": "dbt_warehouses",
"type_info": "Jsonb"
},
{
"ordinal": 16,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 17,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 18,
"name": "ducklake",
"type_info": "Jsonb"
},
{
"ordinal": 19,
"name": "git_sync",
"type_info": "Jsonb"
},
{
"ordinal": 20,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 21,
"name": "default_app",
"type_info": "Varchar"
},
{
"ordinal": 22,
"name": "default_scripts",
"type_info": "Jsonb"
},
{
"ordinal": 23,
"name": "mute_critical_alerts",
"type_info": "Bool"
},
{
"ordinal": 24,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 25,
"name": "operator_settings",
"type_info": "Jsonb"
},
{
"ordinal": 26,
"name": "git_app_installations",
"type_info": "Jsonb"
},
{
"ordinal": 27,
"name": "auto_invite",
"type_info": "Jsonb"
},
{
"ordinal": 28,
"name": "error_handler",
"type_info": "Jsonb"
},
{
"ordinal": 29,
"name": "success_handler",
"type_info": "Jsonb"
},
{
"ordinal": 30,
"name": "variable_expiration_handler",
"type_info": "Jsonb"
},
{
"ordinal": 31,
"name": "public_app_execution_limit_per_minute",
"type_info": "Int4"
},
{
"ordinal": 32,
"name": "error_handler_fallback_to_instance_alerts",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
false
]
},
"hash": "74742f785888f280bf1d7e28793150c3ddc2fd4be3b8c682f6faa7b917387593"
}
backend/.sqlx/query-77d599e4f7c574dffac4824f37127c7ae2ef5f27d665ad9018f5cdea0f6f2cb1.json
Generated
-15
View File
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "77d599e4f7c574dffac4824f37127c7ae2ef5f27d665ad9018f5cdea0f6f2cb1"
}
backend/.sqlx/query-809a2dd647ddfc6fa79ca2c9039e4ebfc4b2880600a03b82e60b89c4a1fd5cc8.json
Generated
+15
View File
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at, value_valid_until)\n SELECT $2, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at, value_valid_until\n FROM variable\n WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Varchar"
]
},
"nullable": []
},
"hash": "809a2dd647ddfc6fa79ca2c9039e4ebfc4b2880600a03b82e60b89c4a1fd5cc8"
}
backend/.sqlx/query-9a3677f9bfc1b5b679578572302129371c7c62e30d77ba6c5e74b575de635b30.json
Generated
+26
View File
@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE variable SET expiration_dispatched_for = $3, expiration_notified_at = now()\n WHERE workspace_id = $1 AND path = $2 AND value_valid_until = $3\n AND expiration_dispatched_for IS NOT DISTINCT FROM $4\n AND expiration_notified_at IS NOT DISTINCT FROM $5\n RETURNING 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Timestamptz",
"Timestamptz",
"Timestamptz"
]
},
"nullable": [
null
]
},
"hash": "9a3677f9bfc1b5b679578572302129371c7c62e30d77ba6c5e74b575de635b30"
}
backend/.sqlx/query-9bbd373847d5cc8e0d84349dbea73af0d18e3b4ce2eb7a20929f8c4c101141b6.json
Generated
+12
View File
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO variable (workspace_id, path, value, is_secret, description, value_valid_until)\n VALUES ('test-workspace', 'u/admin/expiring', 'v', false, 'same', '2027-01-01T00:00:00Z')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9bbd373847d5cc8e0d84349dbea73af0d18e3b4ce2eb7a20929f8c4c101141b6"
}
backend/.sqlx/query-9c2b997126090a5bc191f8d7460e14e0f95b81c61241ffd0c9d633e562c4c4f6.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET variable_expiration_handler = NULL WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "9c2b997126090a5bc191f8d7460e14e0f95b81c61241ffd0c9d633e562c4c4f6"
}
backend/.sqlx/query-9dabc87e05f72fbd4b586c80d9c7f41c3a72838c5d4c5f795638506084561894.json
Generated
+15
View File
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, slack_signing_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at, variable_expiration_handler) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, slack_signing_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at, variable_expiration_handler FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "9dabc87e05f72fbd4b586c80d9c7f41c3a72838c5d4c5f795638506084561894"
}
backend/.sqlx/query-af37e5579b729a1aedcd63ab70d189c7e0376ca1f0071ae8360183ed1300d2bb.json
Generated
+238
View File
@@ -0,0 +1,238 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n customer_id,\n plan,\n webhook,\n ai_config,\n dbt_warehouses,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_invite,\n error_handler,\n success_handler,\n variable_expiration_handler,\n public_app_execution_limit_per_minute,\n error_handler_fallback_to_instance_alerts,\n guest_access_enabled,\n guest_jwt_public_key,\n guest_jwt_jwks_url,\n add_admins_and_developers_to_forks\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "slack_team_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "teams_team_id",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "teams_team_name",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "teams_team_guid",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "slack_name",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "slack_command_script",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "teams_command_script",
"type_info": "Text"
},
{
"ordinal": 8,
"name": "slack_email",
"type_info": "Varchar"
},
{
"ordinal": 9,
"name": "slack_oauth_client_id",
"type_info": "Varchar"
},
{
"ordinal": 10,
"name": "slack_oauth_client_secret",
"type_info": "Varchar"
},
{
"ordinal": 11,
"name": "customer_id",
"type_info": "Varchar"
},
{
"ordinal": 12,
"name": "plan",
"type_info": "Varchar"
},
{
"ordinal": 13,
"name": "webhook",
"type_info": "Text"
},
{
"ordinal": 14,
"name": "ai_config",
"type_info": "Jsonb"
},
{
"ordinal": 15,
"name": "dbt_warehouses",
"type_info": "Jsonb"
},
{
"ordinal": 16,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 17,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 18,
"name": "ducklake",
"type_info": "Jsonb"
},
{
"ordinal": 19,
"name": "git_sync",
"type_info": "Jsonb"
},
{
"ordinal": 20,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 21,
"name": "default_app",
"type_info": "Varchar"
},
{
"ordinal": 22,
"name": "default_scripts",
"type_info": "Jsonb"
},
{
"ordinal": 23,
"name": "mute_critical_alerts",
"type_info": "Bool"
},
{
"ordinal": 24,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 25,
"name": "operator_settings",
"type_info": "Jsonb"
},
{
"ordinal": 26,
"name": "git_app_installations",
"type_info": "Jsonb"
},
{
"ordinal": 27,
"name": "auto_invite",
"type_info": "Jsonb"
},
{
"ordinal": 28,
"name": "error_handler",
"type_info": "Jsonb"
},
{
"ordinal": 29,
"name": "success_handler",
"type_info": "Jsonb"
},
{
"ordinal": 30,
"name": "variable_expiration_handler",
"type_info": "Jsonb"
},
{
"ordinal": 31,
"name": "public_app_execution_limit_per_minute",
"type_info": "Int4"
},
{
"ordinal": 32,
"name": "error_handler_fallback_to_instance_alerts",
"type_info": "Bool"
},
{
"ordinal": 33,
"name": "guest_access_enabled",
"type_info": "Bool"
},
{
"ordinal": 34,
"name": "guest_jwt_public_key",
"type_info": "Text"
},
{
"ordinal": 35,
"name": "guest_jwt_jwks_url",
"type_info": "Text"
},
{
"ordinal": 36,
"name": "add_admins_and_developers_to_forks",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
false,
false,
true,
true,
false
]
},
"hash": "af37e5579b729a1aedcd63ab70d189c7e0376ca1f0071ae8360183ed1300d2bb"
}
backend/.sqlx/query-b4e9a0266f4112467f534826c664c928d1e608ddd063d9d5ce3cea8ea62faca4.json
Generated
+12
View File
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE variable SET value_valid_until = '2029-12-31T00:00:00Z'\n WHERE workspace_id = 'wm-fork-expiry' AND path = 'u/admin/expiring'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "b4e9a0266f4112467f534826c664c928d1e608ddd063d9d5ce3cea8ea62faca4"
}
backend/.sqlx/query-cc2dbdc132fe751c7e82445aae5031d05da988837ce41d63a43d13c2018a57d3.json
Generated
+71
View File
@@ -0,0 +1,71 @@
{
"db_name": "PostgreSQL",
"query": "SELECT v.workspace_id, v.path, v.description, v.value_valid_until AS \"value_valid_until!\",\n v.is_secret, v.expiration_dispatched_for, v.expiration_notified_at,\n ws.variable_expiration_handler->>'path' AS \"handler_path!\",\n (ws.variable_expiration_handler->'extra_args')::text::json\n AS \"extra_args: Json<Box<RawValue>>\"\n FROM variable v\n JOIN workspace_settings ws ON ws.workspace_id = v.workspace_id\n JOIN workspace w ON w.id = v.workspace_id\n WHERE NOT w.deleted\n AND v.value_valid_until IS NOT NULL\n AND (v.value_valid_until IS DISTINCT FROM v.expiration_dispatched_for\n OR (v.value_valid_until > now()\n AND v.expiration_notified_at <= now() - make_interval(secs =>\n NULLIF((ws.variable_expiration_handler->>'remind_every_secs')::bigint, 0) - $2)))\n AND v.value_valid_until <= now() + make_interval(secs =>\n COALESCE((ws.variable_expiration_handler->>'notify_before_secs')::bigint, $1))\n AND ws.variable_expiration_handler->>'path' IS NOT NULL\n AND (NOT COALESCE((ws.variable_expiration_handler->>'muted_on_user_path')::boolean, false)\n OR v.path NOT LIKE 'u/%')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "description",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "value_valid_until!",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"name": "is_secret",
"type_info": "Bool"
},
{
"ordinal": 5,
"name": "expiration_dispatched_for",
"type_info": "Timestamptz"
},
{
"ordinal": 6,
"name": "expiration_notified_at",
"type_info": "Timestamptz"
},
{
"ordinal": 7,
"name": "handler_path!",
"type_info": "Text"
},
{
"ordinal": 8,
"name": "extra_args: Json<Box<RawValue>>",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
true,
false,
true,
true,
null,
null
]
},
"hash": "cc2dbdc132fe751c7e82445aae5031d05da988837ce41d63a43d13c2018a57d3"
}
backend/.sqlx/query-bc6ebef9d41aba232f115f95404922c6054df01fa7f38f5d15d6d4af6c726a3c.json → backend/.sqlx/query-de0d78bf28a5b6457e5568dce9823a4c216b4be67b68241461b42326d9d65256.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM group_ WHERE name != 'all' AND name != 'error_handler' AND name != 'slack' AND name != 'wm_deployers'",
"query": "SELECT COUNT(*) FROM group_ WHERE name != 'all' AND name != 'error_handler' AND name != 'slack' AND name != 'wm_deployers' AND name != 'variable_expiration_handler'",
"describe": {
"columns": [
{
@@ -16,5 +16,5 @@
null
]
},
"hash": "bc6ebef9d41aba232f115f95404922c6054df01fa7f38f5d15d6d4af6c726a3c"
"hash": "de0d78bf28a5b6457e5568dce9823a4c216b4be67b68241461b42326d9d65256"
}
backend/.sqlx/query-e8eaa80631362dee4d696e41abf220cb6f8346d2bfcd901371b480a16a70109c.json
Generated
+15
View File
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET variable_expiration_handler = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Text"
]
},
"nullable": []
},
"hash": "e8eaa80631362dee4d696e41abf220cb6f8346d2bfcd901371b480a16a70109c"
}
backend/.sqlx/query-eac25b9ce5df40a896687d2ba3f9279396ed7213da4598034332bb0ba6ce271c.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET variable_expiration_failed_at = now()\n WHERE workspace_id = $1 AND variable_expiration_failed_at IS NULL\n RETURNING 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "eac25b9ce5df40a896687d2ba3f9279396ed7213da4598034332bb0ba6ce271c"
}
backend/.sqlx/query-eacce0ccde20eddc4aded69c35d62122478424a36193254d5e00b8723ca28079.json
Generated
+20
View File
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET variable_expiration_failed_at = NULL\n WHERE variable_expiration_failed_at IS NOT NULL AND variable_expiration_handler IS NULL\n RETURNING workspace_id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": []
},
"nullable": [
false
]
},
"hash": "eacce0ccde20eddc4aded69c35d62122478424a36193254d5e00b8723ca28079"
}
backend/.sqlx/query-ee8b92d51f220cac5a32f0edcb3c4effa302380d4b434d8c74677bfbf895333c.json
Generated
+12
View File
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-expiry', 'u/admin/expiring', 'variable', 0, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "ee8b92d51f220cac5a32f0edcb3c4effa302380d4b434d8c74677bfbf895333c"
}
backend/ee-repo-ref.txt
+1 -1
View File
@@ -1 +1 @@
5517ff637a675aee66d4e417d3f37266045f0e0c
b6e23d229966cf0019c0a1926a254fda658d8def
backend/migrations/20260824180615_variable_value_valid_until.down.sql
+9
View File
@@ -0,0 +1,9 @@
ALTER TABLE variable DROP CONSTRAINT IF EXISTS variable_oauth_no_value_expiry;
DROP INDEX IF EXISTS idx_variable_expiration_due;
COMMENT ON COLUMN variable.expires_at IS NULL;
ALTER TABLE variable
DROP COLUMN IF EXISTS value_valid_until,
DROP COLUMN IF EXISTS expiration_dispatched_for,
DROP COLUMN IF EXISTS expiration_notified_at;
backend/migrations/20260824180615_variable_value_valid_until.up.sql
+30
View File
@@ -0,0 +1,30 @@
-- `value_valid_until` is when the value stored in the variable stops working; the row itself
-- lives on. `expires_at` keeps its unrelated meaning — delete the row — so no existing writer
-- changes meaning under a rolling deploy and no data is rewritten here.
ALTER TABLE variable ADD COLUMN IF NOT EXISTS value_valid_until TIMESTAMP WITH TIME ZONE;
COMMENT ON COLUMN variable.expires_at IS
'Garbage collection deadline: the row is deleted once passed. Set by the ephemeral secret_arg mint. Not the value''s expiry — see value_valid_until.';
COMMENT ON COLUMN variable.value_valid_until IS
'When the value stops working. Drives the workspace variable expiration handler; never deletes the row.';
-- Holds the `value_valid_until` that was dispatched, rather than a plain "was dispatched" flag,
-- so that moving the date re-arms the handler by construction — no writer has to remember to
-- clear anything.
ALTER TABLE variable
ADD COLUMN IF NOT EXISTS expiration_dispatched_for TIMESTAMP WITH TIME ZONE;
-- When the handler last ran for this variable, which paces its reminders.
ALTER TABLE variable
ADD COLUMN IF NOT EXISTS expiration_notified_at TIMESTAMP WITH TIME ZONE;
-- Serves the sweep's due predicate. Partial so it stays the size of the opted-in set rather
-- than the whole variable table. Not partial on `expiration_dispatched_for`: reminders keep a
-- dispatched variable due until it expires.
CREATE INDEX IF NOT EXISTS idx_variable_expiration_due ON variable (value_valid_until)
WHERE value_valid_until IS NOT NULL;
-- An OAuth-linked value is refreshed by Windmill itself, so a rotation date on it would only
-- announce an expiry that never comes.
ALTER TABLE variable ADD CONSTRAINT variable_oauth_no_value_expiry
CHECK (account IS NULL OR value_valid_until IS NULL);
backend/migrations/20260824180616_workspace_variable_expiration_handler.down.sql
+3
View File
@@ -0,0 +1,3 @@
ALTER TABLE workspace_settings
DROP COLUMN IF EXISTS variable_expiration_handler,
DROP COLUMN IF EXISTS variable_expiration_failed_at;
backend/migrations/20260824180616_workspace_variable_expiration_handler.up.sql
+6
View File
@@ -0,0 +1,6 @@
-- Shape mirrors `error_handler` / `success_handler` as grouped by 20260124172000:
-- {"path": ..., "extra_args": ..., "muted_on_user_path": ...}
ALTER TABLE workspace_settings ADD COLUMN IF NOT EXISTS variable_expiration_handler JSONB;
-- When the handler last failed to start; the sweep skips the workspace for a while after it.
ALTER TABLE workspace_settings ADD COLUMN IF NOT EXISTS variable_expiration_failed_at TIMESTAMP WITH TIME ZONE;
backend/migrations/20260903161430_variable_expiration_handler_slack_perms.down.sql
+1
View File
@@ -0,0 +1 @@
UPDATE folder SET extra_perms = extra_perms - 'g/variable_expiration_handler' WHERE name = 'slack_bot';
backend/migrations/20260903161430_variable_expiration_handler_slack_perms.up.sql
+7
View File
@@ -0,0 +1,7 @@
-- Windmill never created this group before this release, so one present now was made by a user,
-- who would otherwise keep it as it gains read on the Slack bot token below.
DELETE FROM usr_to_group WHERE group_ = 'variable_expiration_handler';
UPDATE group_ SET extra_perms = '{}'::jsonb WHERE name = 'variable_expiration_handler';
-- Backfill for slack_bot folders created before this grant (see create_slack_workspace_artifacts).
UPDATE folder SET extra_perms = JSONB_SET(extra_perms, '{g/variable_expiration_handler}', 'false', true) WHERE name = 'slack_bot';
backend/src/monitor.rs
+11
View File
@@ -129,6 +129,7 @@ use windmill_object_store::reload_object_store_setting;
use windmill_queue::{
add_completed_job_error, cancel_job, canceled_result, get_queued_job_v2,
schedule::{find_unarmed_schedules, rearm_schedule, RearmOutcome},
variable_expiration::dispatch_expiring_variables,
CanceledBy, MiniCompletedJob, SameWorkerPayload,
};
use windmill_store::resources::MAX_RESOURCE_VERSIONS;
@@ -4411,6 +4412,15 @@ pub async fn monitor_db(
}
};
// Run every 5 minutes (10 iterations * 30s)
let dispatch_expiring_variables_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(10) {
if let Some(db) = conn.as_sql() {
dispatch_expiring_variables(&db).await;
}
}
};
// run every hour (120 iterations * 30s = 3600s)
let cleanup_stale_server_heartbeats_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(120) {
@@ -4617,6 +4627,7 @@ pub async fn monitor_db(
native_triggers_sync_f,
cleanup_notify_events_f,
check_expiring_tokens_f,
dispatch_expiring_variables_f,
cleanup_stale_server_heartbeats_f,
manage_audit_partitions_f,
export_audit_logs_to_object_store_f,
backend/summarized_schema.txt
+2 -2
View File
@@ -220,7 +220,7 @@ v2_job_runtime: id(uuid), ping(ts), memory_peak(int)
FK: (id) -> v2_job_queue(id)
v2_job_status: id(uuid), flow_status(jsonb), flow_leaf_jobs(jsonb), workflow_as_code_status(jsonb)
FK: (id) -> v2_job_queue(id)
variable: workspace_id(char), path(char), value(char), is_secret(bool), description(char), extra_perms(jsonb), account(int), is_oauth(bool), expires_at(ts), labels(text[])
variable: workspace_id(char), path(char), value(char), is_secret(bool), description(char), extra_perms(jsonb), account(int), is_oauth(bool), expires_at(ts), labels(text[]), value_valid_until(timestamptz), expiration_dispatched_for(timestamptz), expiration_notified_at(timestamptz)
FK: (workspace_id) -> workspace(id)
websocket_trigger: path(char), url(char), script_path(char), is_flow(bool), workspace_id(char), edited_by(char), email(char), edited_at(ts), extra_perms(jsonb), server_id(char), last_server_ping(ts), error(text), filters(jsonb[]), initial_messages(jsonb[]), url_runnable_args(jsonb), can_return_message(bool), error_handler_path(char), error_handler_args(jsonb), retry(jsonb), can_return_error_result(bool), mode(trigger_mode), labels(text[])
windmill_migrations: name(text), created_at(ts)
@@ -244,7 +244,7 @@ workspace_protection_rule: workspace_id(char), name(char), rules(int), bypass_gr
FK: (workspace_id) -> workspace(id)
workspace_runnable_dependencies: flow_path(char), runnable_path(char), script_hash(bigint), runnable_is_flow(bool), workspace_id(char), app_path(char), id(bigint), runnable_is_agent(bool)
FK: (app_path, workspace_id) -> app(path, workspace_id) | (flow_path, workspace_id) -> flow(path, workspace_id)
workspace_settings: workspace_id(char), slack_team_id(char), slack_name(char), slack_command_script(char), slack_email(char), customer_id(char), plan(char), webhook(text), ai_config(jsonb), large_file_storage(jsonb), git_sync(jsonb), default_app(char), default_scripts(jsonb), deploy_ui(jsonb), mute_critical_alerts(bool), color(char), operator_settings(jsonb), teams_command_script(text), teams_team_id(text), teams_team_name(text), git_app_installations(jsonb), ducklake(jsonb), slack_oauth_client_id(char), slack_oauth_client_secret(char), datatable(jsonb), teams_team_guid(text), auto_invite(jsonb), error_handler(jsonb), success_handler(jsonb), public_app_execution_limit_per_minute(int), dbt_warehouses(jsonb), guest_access_enabled(bool), guest_jwt_public_key(text), guest_jwt_jwks_url(text), ai_sessions_backup_generation(int), add_admins_and_developers_to_forks(bool), remote_deploy_target(jsonb), remote_deploy_target_changed_at(ts)
workspace_settings: workspace_id(char), slack_team_id(char), slack_name(char), slack_command_script(char), slack_email(char), customer_id(char), plan(char), webhook(text), ai_config(jsonb), large_file_storage(jsonb), git_sync(jsonb), default_app(char), default_scripts(jsonb), deploy_ui(jsonb), mute_critical_alerts(bool), color(char), operator_settings(jsonb), teams_command_script(text), teams_team_id(text), teams_team_name(text), git_app_installations(jsonb), ducklake(jsonb), slack_oauth_client_id(char), slack_oauth_client_secret(char), datatable(jsonb), teams_team_guid(text), auto_invite(jsonb), error_handler(jsonb), success_handler(jsonb), public_app_execution_limit_per_minute(int), dbt_warehouses(jsonb), guest_access_enabled(bool), guest_jwt_public_key(text), guest_jwt_jwks_url(text), ai_sessions_backup_generation(int), add_admins_and_developers_to_forks(bool), remote_deploy_target(jsonb), remote_deploy_target_changed_at(ts), variable_expiration_handler(jsonb), variable_expiration_failed_at(ts)
FK: (workspace_id) -> workspace(id)
zombie_job_counter: job_id(uuid), counter(int)
FK: (job_id) -> v2_job(id)
backend/tests/variable_expiration.rs
+534
View File
@@ -0,0 +1,534 @@
//! Tests for the variable expiration sweep. No worker: the sweep ends at the push, and the
//! queued job is what the tests assert on.
use sqlx::{Pool, Postgres};
use windmill_queue::variable_expiration::dispatch_expiring_variables;
use windmill_test_utils::{initialize_tracing, ApiServer};
const WS: &str = "test-workspace";
async fn configure_handler(db: &Pool<Postgres>, handler: &str) -> anyhow::Result<()> {
sqlx::query("UPDATE workspace_settings SET variable_expiration_handler = $2::jsonb WHERE workspace_id = $1")
.bind(WS)
.bind(handler)
.execute(db)
.await?;
Ok(())
}
/// Seed a deployed script the handler setting can point at.
async fn seed_handler_script(db: &Pool<Postgres>, path: &str) -> anyhow::Result<()> {
sqlx::query(
r#"INSERT INTO script (workspace_id, hash, path, summary, description, content, created_by, language, lock)
VALUES ($1, $2, $3, '', '', 'export function main() {}', 'test-user', 'bun', '')"#,
)
.bind(WS)
.bind(path.bytes().fold(0i64, |h, b| h.wrapping_mul(31).wrapping_add(b as i64)))
.bind(path)
.execute(db)
.await?;
Ok(())
}
async fn seed_variable(db: &Pool<Postgres>, path: &str, expires_in: &str) -> anyhow::Result<()> {
sqlx::query(&format!(
r#"INSERT INTO variable (workspace_id, path, value, is_secret, description, value_valid_until)
VALUES ($1, $2, 'MUST_NOT_LEAK', true, 'a credential', now() + interval '{expires_in}')"#
))
.bind(WS)
.bind(path)
.execute(db)
.await?;
Ok(())
}
async fn queued_handler_jobs(db: &Pool<Postgres>, path: &str) -> anyhow::Result<i64> {
Ok(
sqlx::query_scalar::<_, i64>("SELECT count(*) FROM v2_job WHERE runnable_path = $1")
.bind(path)
.fetch_one(db)
.await?,
)
}
/// The sweep dispatches inside the handler's `notify_before_secs`, leaves everything outside it
/// alone, and does not fire twice for the same expiry.
#[sqlx::test(fixtures("base"))]
async fn test_dispatch_within_lead_time_once(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
configure_handler(
&db,
r#"{"path": "script/u/test-user/expiry_handler", "notify_before_secs": 3600}"#,
)
.await?;
seed_variable(&db, "u/test-user/due", "30 minutes").await?;
seed_variable(&db, "u/test-user/far", "2 hours").await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
1,
"exactly the variable inside the lead window should dispatch"
);
// Admins grant this group access to the secrets it rotates, so the job must run as it.
let (path, is_secret, permissioned_as, permissioned_as_email): (String, bool, String, String) =
sqlx::query_as(
"SELECT args->>'variable_path', (args->>'is_secret')::boolean, permissioned_as,
permissioned_as_email FROM v2_job WHERE runnable_path = $1",
)
.bind("u/test-user/expiry_handler")
.fetch_one(&db)
.await?;
assert_eq!(path, "u/test-user/due");
assert!(is_secret);
assert_eq!(permissioned_as, "g/variable_expiration_handler");
assert_eq!(
permissioned_as_email,
"variable_expiration_handler@windmill.dev"
);
let args: String = sqlx::query_scalar("SELECT args::text FROM v2_job WHERE runnable_path = $1")
.bind("u/test-user/expiry_handler")
.fetch_one(&db)
.await?;
assert!(
!args.contains("MUST_NOT_LEAK"),
"handler args leaked the value: {args}"
);
// The claim is what makes concurrent replicas safe.
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
1,
"a dispatched variable must not fire again"
);
Ok(())
}
/// A variable's writer must not get the handler to resolve a reference as its own identity;
/// the admin's `extra_args` still resolve.
#[sqlx::test(fixtures("base"))]
async fn test_variable_fields_do_not_resolve_in_handler(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
configure_handler(
&db,
r#"{"path": "script/u/test-user/expiry_handler", "extra_args": {"slack": "$res:f/slack_bot/bot_token"}}"#,
)
.await?;
seed_variable(&db, "u/test-user/due", "30 minutes").await?;
sqlx::query(
"UPDATE variable SET description = '$var:u/test-user/other' WHERE path = 'u/test-user/due'",
)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
let (description, slack): (String, String) = sqlx::query_as(
"SELECT args->>'description', args->>'slack' FROM v2_job WHERE runnable_path = $1",
)
.bind("u/test-user/expiry_handler")
.fetch_one(&db)
.await?;
assert!(
!description.starts_with("$var:"),
"description still resolves: {description}"
);
assert_eq!(slack, "$res:f/slack_bot/bot_token");
Ok(())
}
/// `remind_every_secs` runs the handler again once the interval has elapsed, and not before.
#[sqlx::test(fixtures("base"))]
async fn test_remind_every_repeats_the_notice(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
configure_handler(
&db,
r#"{"path": "script/u/test-user/expiry_handler", "remind_every_secs": 600}"#,
)
.await?;
seed_variable(&db, "u/test-user/due", "30 minutes").await?;
dispatch_expiring_variables(&db).await;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
1,
"no reminder before the interval has elapsed"
);
// A sweep landing a few seconds short of the interval must still remind.
sqlx::query(
"UPDATE variable SET expiration_notified_at = now() - interval '9 minutes 50 seconds'",
)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
2
);
sqlx::query(
"UPDATE variable SET value_valid_until = now() - interval '1 minute',
expiration_dispatched_for = now() - interval '1 minute',
expiration_notified_at = now() - interval '11 minutes'",
)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
2
);
Ok(())
}
/// Moving `value_valid_until` re-arms the variable; re-writing the same date does not.
#[sqlx::test(fixtures("base"))]
async fn test_rearm_on_date_change(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let base_url = format!("http://localhost:{}", server.addr.port());
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
configure_handler(&db, r#"{"path": "script/u/test-user/expiry_handler"}"#).await?;
seed_variable(&db, "u/test-user/due", "30 minutes").await?;
// Through the real endpoint: writing the column directly would assert nothing.
let update = async |body: serde_json::Value| -> anyhow::Result<u16> {
Ok(reqwest::Client::new()
.post(format!(
"{base_url}/api/w/{WS}/variables/update/u/test-user/due"
))
.bearer_auth("SECRET_TOKEN")
.json(&body)
.send()
.await?
.status()
.as_u16())
};
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
1
);
let stored: chrono::DateTime<chrono::Utc> = sqlx::query_scalar(
"SELECT value_valid_until FROM variable WHERE workspace_id = $1 AND path = $2",
)
.bind(WS)
.bind("u/test-user/due")
.fetch_one(&db)
.await?;
// An unchanged `wmill sync push` rewrites the same date.
assert_eq!(
update(serde_json::json!({ "value_valid_until": stored })).await?,
200
);
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
1,
"an unchanged expiry must not re-arm"
);
assert_eq!(
update(serde_json::json!({ "description": "touched" })).await?,
200
);
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
1,
"an update that omits value_valid_until must not re-arm"
);
assert_eq!(
update(serde_json::json!({ "value_valid_until": stored + chrono::Duration::minutes(5) }))
.await?,
200
);
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
2,
"a moved expiry must re-arm the handler"
);
Ok(())
}
/// `muted_on_user_path` keeps personal-path variables out.
#[sqlx::test(fixtures("base"))]
async fn test_muted_user_paths(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
seed_variable(&db, "u/test-user/personal", "30 minutes").await?;
seed_variable(&db, "f/test_folder/shared", "30 minutes").await?;
configure_handler(
&db,
r#"{"path": "script/u/test-user/expiry_handler", "muted_on_user_path": true}"#,
)
.await?;
dispatch_expiring_variables(&db).await;
let dispatched: Vec<String> =
sqlx::query_scalar("SELECT args->>'variable_path' FROM v2_job WHERE runnable_path = $1")
.bind("u/test-user/expiry_handler")
.fetch_all(&db)
.await?;
assert_eq!(
dispatched,
vec!["f/test_folder/shared".to_string()],
"a muted personal path must not dispatch"
);
Ok(())
}
/// Broken workspaces' due rows must not hold back anyone else's first notice or reminder: every
/// due row is tried each pass, and a broken handler only once per pass. The broken rows go in
/// first, so a cap on the pass would cut off the working workspace.
#[sqlx::test(fixtures("base"))]
async fn test_broken_handlers_do_not_hold_back_others(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::raw_sql(
r#"INSERT INTO workspace (id, name, owner) SELECT 'broken-' || i, 'broken-' || i, 'test-user' FROM generate_series(1, 100) i;
INSERT INTO workspace_key (workspace_id, kind, key) SELECT 'broken-' || i, 'cloud', 'k' FROM generate_series(1, 100) i;
INSERT INTO workspace_settings (workspace_id, variable_expiration_handler)
SELECT 'broken-' || i, '{"path": "script/u/test-user/does_not_exist"}'::jsonb FROM generate_series(1, 100) i;
INSERT INTO variable (workspace_id, path, value, is_secret, description, value_valid_until)
SELECT 'broken-' || i, 'f/creds/v' || j, 'v', false, '', now() - interval '1 day'
FROM generate_series(1, 100) i, generate_series(1, 2) j;"#,
)
.execute(&db)
.await?;
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
configure_handler(
&db,
r#"{"path": "script/u/test-user/expiry_handler", "remind_every_secs": 60}"#,
)
.await?;
seed_variable(&db, "u/test-user/fresh", "30 minutes").await?;
sqlx::query(
"INSERT INTO variable (workspace_id, path, value, is_secret, description, value_valid_until,
expiration_dispatched_for, expiration_notified_at)
VALUES ($1, 'u/test-user/reminded', 'v', false, '', now() + interval '10 minutes',
now() + interval '10 minutes', now() - interval '2 minutes')",
)
.bind(WS)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/expiry_handler").await?,
2,
"the first notice and the reminder must both go out"
);
assert_eq!(
queued_handler_jobs(&db, "u/test-user/does_not_exist").await?,
0
);
Ok(())
}
/// A broken handler alerts once, however many passes it fails, runs on the first pass after
/// it is deployed, and that resolves its alert.
#[sqlx::test(fixtures("base"))]
async fn test_deploying_the_handler_ends_the_failure(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
configure_handler(&db, r#"{"path": "script/u/test-user/redeployed_handler"}"#).await?;
seed_variable(&db, "u/test-user/due", "30 minutes").await?;
let alerts = || async {
Ok::<_, anyhow::Error>(sqlx::query_as::<_, (i64, i64)>(
"SELECT count(*), count(*) FILTER (WHERE NOT acknowledged) FROM alerts
WHERE alert_type = 'critical_error' AND resource = 'variable_expiration_handler:test-workspace'",
)
.fetch_one(&db)
.await?)
};
dispatch_expiring_variables(&db).await;
dispatch_expiring_variables(&db).await;
assert_eq!(
alerts().await?,
(1, 1),
"one alert per failure, not per pass"
);
seed_handler_script(&db, "u/test-user/redeployed_handler").await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/redeployed_handler").await?,
1,
"the deployed handler did not run on the next pass"
);
assert_eq!(alerts().await?, (1, 0), "starting again resolves the alert");
Ok(())
}
/// A handler script still waiting on its dependency job runs on the first pass after its lock
/// lands, though nothing about the script changes but the lock.
#[sqlx::test(fixtures("base"))]
async fn test_handler_runs_once_its_lock_lands(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
configure_handler(
&db,
r#"{"path": "script/u/test-user/pending_lock_handler"}"#,
)
.await?;
seed_variable(&db, "u/test-user/due", "30 minutes").await?;
seed_handler_script(&db, "u/test-user/pending_lock_handler").await?;
sqlx::query("UPDATE script SET lock = NULL WHERE workspace_id = $1")
.bind(WS)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/pending_lock_handler").await?,
0
);
sqlx::query("UPDATE script SET lock = '' WHERE workspace_id = $1")
.bind(WS)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
assert_eq!(
queued_handler_jobs(&db, "u/test-user/pending_lock_handler").await?,
1,
"the handler did not run once its lock landed"
);
Ok(())
}
/// A removed handler has nothing left to retry, so the next pass ends its failure and resolves
/// its alert.
#[sqlx::test(fixtures("base"))]
async fn test_removing_a_failing_handler_resolves_its_alert(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query(
"UPDATE workspace_settings SET variable_expiration_handler = NULL,
variable_expiration_failed_at = now() WHERE workspace_id = $1",
)
.bind(WS)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO alerts (alert_type, message, acknowledged, acknowledged_workspace, workspace_id, resource)
VALUES ('critical_error', 'failed', false, false, $1, 'variable_expiration_handler:' || $1)",
)
.bind(WS)
.execute(&db)
.await?;
dispatch_expiring_variables(&db).await;
let (failing, open_alerts): (bool, i64) = sqlx::query_as(
"SELECT ws.variable_expiration_failed_at IS NOT NULL,
(SELECT count(*) FROM alerts WHERE alert_type = 'critical_error' AND NOT acknowledged
AND workspace_id = $1)
FROM workspace_settings ws WHERE ws.workspace_id = $1",
)
.bind(WS)
.fetch_one(&db)
.await?;
assert!(!failing, "the removed handler's failure must end");
assert_eq!(
open_alerts, 0,
"the removed handler's alert must be resolved"
);
Ok(())
}
/// The handler endpoint: admin-only, existence-checked, and clearing by omission, which
/// `wmill sync push` relies on to propagate an absent handler.
#[sqlx::test(fixtures("base"))]
async fn test_edit_handler_endpoint(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let base_url = format!("http://localhost:{}", server.addr.port());
let edit = async |token: &str, body: serde_json::Value| -> anyhow::Result<u16> {
Ok(reqwest::Client::new()
.post(format!(
"{base_url}/api/w/{WS}/workspaces/edit_variable_expiration_handler"
))
.bearer_auth(token)
.json(&body)
.send()
.await?
.status()
.as_u16())
};
let stored = async || -> anyhow::Result<Option<serde_json::Value>> {
Ok(sqlx::query_scalar(
"SELECT variable_expiration_handler FROM workspace_settings WHERE workspace_id = $1",
)
.bind(WS)
.fetch_one(&db)
.await?)
};
seed_handler_script(&db, "u/test-user/expiry_handler").await?;
let handler = serde_json::json!({ "path": "script/u/test-user/expiry_handler" });
assert_eq!(edit("SECRET_TOKEN_2", handler.clone()).await?, 403);
assert!(stored().await?.is_none());
for path in [
"u/test-user/expiry_handler",
"script/u/test-user/does_not_exist",
] {
assert_eq!(
edit("SECRET_TOKEN", serde_json::json!({ "path": path })).await?,
400,
"{path}"
);
}
// A recorded failure is the sweep's to end, so the endpoint leaves it alone either way.
sqlx::query("UPDATE workspace_settings SET variable_expiration_failed_at = now() WHERE workspace_id = $1")
.bind(WS)
.execute(&db)
.await?;
let failing = || async {
Ok::<_, anyhow::Error>(sqlx::query_scalar::<_, bool>(
"SELECT variable_expiration_failed_at IS NOT NULL FROM workspace_settings WHERE workspace_id = $1",
)
.bind(WS)
.fetch_one(&db)
.await?)
};
assert_eq!(edit("SECRET_TOKEN", handler).await?, 200);
assert_eq!(
stored()
.await?
.and_then(|h| h["path"].as_str().map(String::from)),
Some("script/u/test-user/expiry_handler".to_string())
);
assert!(failing().await?, "saving the handler must keep its failure");
assert_eq!(edit("SECRET_TOKEN", serde_json::json!({})).await?, 200);
assert!(
stored().await?.is_none(),
"an omitted path clears the handler"
);
Ok(())
}
backend/tests/workspace_export.rs
+42
View File
@@ -288,6 +288,48 @@ async fn test_tarball_export_gates_values_on_item_scopes(db: Pool<Postgres>) ->
Ok(())
}
/// The tarball drops variables by `expires_at`, never `value_valid_until`, and the exported
/// entry carries the date, or a pull/push round-trip clears it.
#[sqlx::test(fixtures("base"))]
async fn test_tarball_export_excludes_only_gc_variables(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let base_url = format!("http://localhost:{}", server.addr.port());
sqlx::query(
r#"INSERT INTO variable (workspace_id, path, value, is_secret, description, expires_at, value_valid_until)
VALUES ('test-workspace', 'u/test-user/expiring', 'v', false, '', NULL, '2030-01-15T08:00:00Z'),
('test-workspace', 'u/test-user/ephemeral', 'v', false, '', now() + interval '7 days', NULL)"#,
)
.execute(&db)
.await?;
let resp = reqwest::Client::new()
.get(format!(
"{base_url}/api/w/test-workspace/workspaces/tarball"
))
.bearer_auth("SECRET_TOKEN")
.send()
.await?;
assert_eq!(resp.status().as_u16(), 200);
let body = String::from_utf8_lossy(&resp.bytes().await?).into_owned();
assert!(
body.contains("u/test-user/expiring.variable.json"),
"a variable with an expiry must still reach git sync"
);
assert!(
body.contains("2030-01-15T08:00:00Z"),
"the exported entry must carry the expiry back"
);
assert!(
!body.contains("u/test-user/ephemeral.variable.json"),
"a variable awaiting garbage collection must stay out of the export"
);
Ok(())
}
/// `settings.json` carries the admin-managed integration config that `get_settings`
/// is admin-only for (the webhook URL, ai_config, git_sync, handler extra_args), so
/// `include_settings` takes the same admin check as `get_settings` rather than
backend/windmill-api-groups/src/groups.rs
+3 -2
View File
@@ -218,12 +218,13 @@ pub async fn require_is_owner(
}
async fn _check_nb_of_groups(db: &DB) -> Result<()> {
let nb_groups = sqlx::query_scalar!("SELECT COUNT(*) FROM group_ WHERE name != 'all' AND name != 'error_handler' AND name != 'slack' AND name != 'wm_deployers'",)
// Groups that back workspace settings do not count toward the limit.
let nb_groups = sqlx::query_scalar!("SELECT COUNT(*) FROM group_ WHERE name != 'all' AND name != 'error_handler' AND name != 'slack' AND name != 'wm_deployers' AND name != 'variable_expiration_handler'",)
.fetch_one(db)
.await?;
if nb_groups.unwrap_or(0) >= 3 {
return Err(Error::BadRequest(
"You have reached the maximum number of groups (3 outside of native groups 'all', 'slack', 'error_handler' and 'wm_deployers') without an enterprise license"
"You have reached the maximum number of groups (3 outside of native groups 'all', 'slack', 'error_handler', 'wm_deployers' and 'variable_expiration_handler') without an enterprise license"
.to_string(),
));
}
backend/windmill-api-integration-tests/tests/variables.rs
+68
View File
@@ -339,6 +339,74 @@ async fn test_variables_are_cached(db: Pool<Postgres>) -> anyhow::Result<()> {
Ok(())
}
/// A draft-only row is built field by field, so a forgotten field is dropped silently.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_draft_only_row_carries_value_valid_until(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
authed(client().post(format!(
"http://localhost:{port}/api/w/test-workspace/drafts/update/variable/u/test-user/draft_var"
)))
.json(&json!({
"value": {
"path": "u/test-user/draft_var",
"variable": { "value": "x", "is_secret": true, "description": "" },
"wsSpecific": false,
"value_valid_until": "2027-03-15T08:00:00Z"
}
}))
.send()
.await?;
let resp = authed(client().get(format!(
"http://localhost:{port}/api/w/test-workspace/variables/list?include_draft_only=true"
)))
.send()
.await?;
assert_eq!(resp.status(), 200);
let rows = resp.json::<Vec<serde_json::Value>>().await?;
let row = rows
.iter()
.find(|v| v["path"] == "u/test-user/draft_var")
.expect("draft-only row is listed");
assert_eq!(row["draft_only"], true);
assert_eq!(row["value_valid_until"], "2027-03-15T08:00:00Z");
Ok(())
}
/// An expiry on an OAuth-linked variable is refused with a readable error, not a raw
/// constraint violation.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_oauth_variable_refuses_value_expiry(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
sqlx::query(
"WITH a AS (INSERT INTO account (workspace_id, expires_at, refresh_token, client)
VALUES ('test-workspace', now() + interval '1 hour', 'r', 'github') RETURNING id)
INSERT INTO variable (workspace_id, path, value, is_secret, description, account, is_oauth)
SELECT 'test-workspace', 'u/test-user/oauth_var', 't', false, '', id, true FROM a",
)
.execute(&db)
.await?;
let resp = authed(client().post(format!(
"http://localhost:{port}/api/w/test-workspace/variables/update/u/test-user/oauth_var"
)))
.json(&json!({ "value_valid_until": "2027-03-15T08:00:00Z" }))
.send()
.await?;
assert_eq!(resp.status(), 400);
assert!(resp.text().await?.contains("OAuth"));
Ok(())
}
/// Reading a variable linked to an OAuth account refreshes through it and returns the fresh token,
/// so the variable endpoints must not let a caller choose the link.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
backend/windmill-api-integration-tests/tests/workspace_comparison.rs
+80
View File
@@ -2770,3 +2770,83 @@ async fn test_compare_secret_variables_by_value(db: Pool<Postgres>) -> anyhow::R
Ok(())
}
/// `compare_two_variables` hand-lists its columns, so a forgotten one hides that difference
/// from the fork diff, and it can never be promoted.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_compare_variables_detects_value_valid_until(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base_url = format!("http://localhost:{port}/api");
let admin = windmill_api_client::create_client(
&format!("http://localhost:{port}"),
"SECRET_TOKEN".to_string(),
);
sqlx::query!(
"INSERT INTO variable (workspace_id, path, value, is_secret, description, value_valid_until)
VALUES ('test-workspace', 'u/admin/expiring', 'v', false, 'same', '2027-01-01T00:00:00Z')"
)
.execute(&db)
.await?;
let resp = admin
.client()
.post(&format!(
"{base_url}/w/test-workspace/workspaces/create_fork"
))
.json(&json!({ "id": "wm-fork-expiry", "name": "Expiry Fork" }))
.send()
.await?;
assert!(
resp.status().is_success(),
"fork creation failed: {}",
resp.status()
);
// Before the UPDATE below overwrites it: the fork must inherit the date from the source.
let inherited: Option<String> = sqlx::query_scalar(
"SELECT to_char(value_valid_until, 'YYYY-MM-DD') FROM variable
WHERE workspace_id = 'wm-fork-expiry' AND path = 'u/admin/expiring'",
)
.fetch_one(&db)
.await?;
assert_eq!(inherited.as_deref(), Some("2027-01-01"));
// Only the expiry differs.
sqlx::query!(
"UPDATE variable SET value_valid_until = '2029-12-31T00:00:00Z'
WHERE workspace_id = 'wm-fork-expiry' AND path = 'u/admin/expiring'"
)
.execute(&db)
.await?;
// Forces the endpoint to re-evaluate the pair rather than trust a cached verdict.
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES ('test-workspace', 'wm-fork-expiry', 'u/admin/expiring', 'variable', 0, 1, NULL)"
)
.execute(&db)
.await?;
let comparison = admin
.client()
.get(&format!(
"{base_url}/w/test-workspace/workspaces/compare/wm-fork-expiry"
))
.send()
.await?
.json::<serde_json::Value>()
.await?;
assert_eq!(
comparison["summary"]["variables_changed"].as_u64(),
Some(1),
"an expiry-only change must show as deployable: {comparison}"
);
Ok(())
}
backend/windmill-api-workspaces/src/workspaces.rs
+171 -5
View File
@@ -127,6 +127,10 @@ pub fn workspaced_service() -> Router {
.route("/billable_seats", get(get_billable_seats))
.route("/edit_error_handler", post(edit_error_handler))
.route("/edit_success_handler", post(edit_success_handler))
.route(
"/edit_variable_expiration_handler",
post(edit_variable_expiration_handler),
)
.route(
"/edit_large_file_storage_config",
post(edit_large_file_storage_config),
@@ -342,6 +346,8 @@ pub struct WorkspaceSettings {
#[serde(skip_serializing_if = "Option::is_none")]
pub success_handler: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
pub variable_expiration_handler: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
pub public_app_execution_limit_per_minute: Option<i32>,
pub error_handler_fallback_to_instance_alerts: bool,
/// Whether this workspace admits guest sessions (`ExecutionMode::Guest`). An app's
@@ -1203,6 +1209,7 @@ async fn get_settings(
auto_invite,
error_handler,
success_handler,
variable_expiration_handler,
public_app_execution_limit_per_minute,
error_handler_fallback_to_instance_alerts,
guest_access_enabled,
@@ -6253,6 +6260,157 @@ async fn edit_error_handler(
Ok(format!("Edit error_handler for workspace {}", &w_id))
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub struct EditVariableExpirationHandler {
pub path: Option<String>,
pub extra_args: Option<serde_json::Value>,
#[serde(default)]
pub muted_on_user_path: bool,
pub notify_before_secs: Option<u32>,
pub remind_every_secs: Option<u32>,
}
async fn edit_variable_expiration_handler(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Path(w_id): Path<String>,
Json(es): Json<EditVariableExpirationHandler>,
) -> Result<String> {
require_admin(authed.is_admin, &authed.username)?;
let mut tx = db.begin().await?;
// Created with no access: an admin grants it write access to each variable it rotates.
sqlx::query_as!(
Group,
"INSERT INTO group_ (workspace_id, name, summary, extra_perms) VALUES ($1, $2, $3, $4) ON CONFLICT DO NOTHING",
w_id,
"variable_expiration_handler",
"The group the variable expiration handler acts on behalf of",
serde_json::json!({username_to_permissioned_as(&authed.username): true})
)
.execute(&mut *tx)
.await?;
if let Some(path) = &es.path {
if es
.extra_args
.as_ref()
.is_some_and(|extra_args| !extra_args.is_object())
{
return Err(Error::BadRequest(
"Field `extra_args` expected to be JSON object".to_string(),
));
}
// Any hub script is left unchecked, so saving never needs the hub.
let exists = if let Some(p) = path.strip_prefix("script/") {
p.starts_with("hub/")
|| sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM script WHERE workspace_id = $1 AND path = $2 AND archived = false AND deleted = false)",
w_id,
p
)
.fetch_one(&mut *tx)
.await?
.unwrap_or(false)
} else if let Some(p) = path.strip_prefix("flow/") {
sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM flow WHERE workspace_id = $1 AND path = $2 AND archived = false)",
w_id,
p
)
.fetch_one(&mut *tx)
.await?
.unwrap_or(false)
} else {
return Err(Error::BadRequest(format!(
"Field `path` must start with script/ or flow/ (got {path})"
)));
};
if !exists {
return Err(Error::BadRequest(format!(
"Variable expiration handler '{path}' not found. The handler must be an existing, non-archived script or flow."
)));
}
// Persisted even when false, like `error_handler`'s.
let mut variable_expiration_handler = serde_json::json!({
"path": path,
"muted_on_user_path": es.muted_on_user_path,
});
if let Some(extra_args) = &es.extra_args {
variable_expiration_handler["extra_args"] = extra_args.clone();
}
if let Some(notify_before_secs) = es.notify_before_secs {
variable_expiration_handler["notify_before_secs"] = notify_before_secs.into();
}
if let Some(remind_every_secs) = es.remind_every_secs {
variable_expiration_handler["remind_every_secs"] = remind_every_secs.into();
}
// A recorded failure is the sweep's: it ends at the handler's next start.
sqlx::query!(
"UPDATE workspace_settings SET variable_expiration_handler = $1 WHERE workspace_id = $2",
variable_expiration_handler,
&w_id
)
.execute(&mut *tx)
.await?;
} else {
sqlx::query!(
"UPDATE workspace_settings SET variable_expiration_handler = NULL WHERE workspace_id = $1",
&w_id
)
.execute(&mut *tx)
.await?;
}
audit_log(
&mut *tx,
&authed,
"workspaces.edit_variable_expiration_handler",
ActionKind::Update,
&w_id,
Some(&authed.email),
Some([("variable_expiration_handler", &format!("{:?}", es.path)[..])].into()),
)
.await?;
tx.commit().await?;
handle_deployment_metadata(
&authed.email,
&authed.username,
&db,
&w_id,
windmill_git_sync::DeployedObject::Settings {
setting_type: "variable_expiration_handler".to_string(),
},
Some("Variable expiration handler configuration updated".to_string()),
false,
None,
)
.await?;
windmill_common::feature_usage::log_feature_usage(
"variable_expiration",
"handler_configured",
// By slug, not hub id, so a bumped hub version still counts as the built-in.
match es.path.as_deref() {
None => "cleared",
Some(p) if p.ends_with("/variable-expiration-handler-slack") => "slack",
Some(p) if p.ends_with("/variable-expiration-handler-teams") => "teams",
Some(_) => "custom",
},
);
Ok(format!(
"Edit variable_expiration_handler for workspace {}",
&w_id
))
}
async fn edit_success_handler(
authed: ApiAuthed,
Extension(db): Extension<DB>,
@@ -7925,9 +8083,10 @@ async fn clone_variables(
source_workspace_id: &str,
target_workspace_id: &str,
) -> Result<()> {
// Dispatch markers are not copied, so the fork re-arms its own handler.
sqlx::query!(
"INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at)
SELECT $2, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at
"INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at, value_valid_until)
SELECT $2, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at, value_valid_until
FROM variable
WHERE workspace_id = $1",
source_workspace_id,
@@ -14503,7 +14662,7 @@ async fn compare_two_variables(
// Get variable from each workspace
let source_variable = sqlx::query!(
"SELECT value::text AS \"value!\", is_secret, description
"SELECT value::text AS \"value!\", is_secret, description, value_valid_until
FROM variable
WHERE workspace_id = $1 AND path = $2",
source_workspace_id,
@@ -14513,7 +14672,7 @@ async fn compare_two_variables(
.await?;
let target_variable = sqlx::query!(
"SELECT value::text AS \"value!\", is_secret, description
"SELECT value::text AS \"value!\", is_secret, description, value_valid_until
FROM variable
WHERE workspace_id = $1 AND path = $2",
fork_workspace_id,
@@ -14526,7 +14685,10 @@ async fn compare_two_variables(
// Check metadata differences
if let (Some(source), Some(target)) = (&source_variable, &target_variable) {
if source.is_secret != target.is_secret || source.description != target.description {
if source.is_secret != target.is_secret
|| source.description != target.description
|| source.value_valid_until != target.value_valid_until
{
has_changes = true;
} else if source.value != target.value
|| (source.is_secret && is_external_stored_value(&source.value))
@@ -15355,6 +15517,7 @@ pub enum SettingsEdit {
InstanceGroups,
ErrorHandler,
SuccessHandler,
VariableExpirationHandler,
LargeFileStorage,
GitSync,
DefaultScripts,
@@ -15407,6 +15570,9 @@ pub async fn apply_settings_edit(
InstanceGroups => edit_instance_groups(a, d, p, settings_edit_body(edit, body)?).await?,
ErrorHandler => edit_error_handler(a, d, p, settings_edit_body(edit, body)?).await?,
SuccessHandler => edit_success_handler(a, d, p, settings_edit_body(edit, body)?).await?,
VariableExpirationHandler => {
edit_variable_expiration_handler(a, d, p, settings_edit_body(edit, body)?).await?
}
LargeFileStorage => {
edit_large_file_storage_config(a, d, p, settings_edit_body(edit, body)?).await?
}
backend/windmill-api-workspaces/src/workspaces_extra.rs
+1 -1
View File
@@ -135,7 +135,7 @@ pub(crate) async fn change_workspace_id(
// Duplicate workspace settings (keep copy in old workspace for reference)
info!("Duplicating workspace_settings table");
sqlx::query!(
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, slack_signing_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, slack_signing_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at FROM workspace_settings WHERE workspace_id = $2",
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, slack_signing_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at, variable_expiration_handler) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, role_overrides, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, slack_signing_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks, remote_deploy_target, remote_deploy_target_changed_at, variable_expiration_handler FROM workspace_settings WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
backend/windmill-api/openapi.yaml
+89 -1
View File
@@ -4366,6 +4366,8 @@ paths:
$ref: "#/components/schemas/ErrorHandlerConfig"
success_handler:
$ref: "#/components/schemas/SuccessHandlerConfig"
variable_expiration_handler:
$ref: "#/components/schemas/VariableExpirationHandlerConfig"
large_file_storage:
$ref: "#/components/schemas/LargeFileStorage"
ducklake:
@@ -5432,6 +5434,29 @@ paths:
schema:
type: string
/w/{workspace}/workspaces/edit_variable_expiration_handler:
post:
summary: edit variable expiration handler
operationId: editVariableExpirationHandler
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
requestBody:
description: WorkspaceVariableExpirationHandler
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/EditVariableExpirationHandler"
responses:
"200":
description: status
content:
text/plain:
schema:
type: string
/w/{workspace}/workspaces/edit_success_handler:
post:
summary: edit success handler
@@ -29832,6 +29857,45 @@ components:
type: boolean
default: false
EditVariableExpirationHandler:
type: object
description: >
Request body for editing the workspace variable expiration handler. Omit
`path` (or send null) to unset the handler.
properties:
path:
type: string
nullable: true
description: Path to the variable expiration handler script or flow
extra_args:
$ref: "#/components/schemas/ScriptArgs"
muted_on_user_path:
type: boolean
description: Do not run the handler for variables on a personal (u/) path
notify_before_secs:
type: integer
minimum: 0
description: How long before a variable's value expires the handler runs. 1 day when omitted.
remind_every_secs:
type: integer
minimum: 0
description: Run the handler again at this interval until the value expires. No reminders when omitted or 0.
VariableExpirationHandlerConfig:
type: object
description: Workspace variable expiration handler configuration
properties:
path:
type: string
extra_args:
$ref: "#/components/schemas/ScriptArgs"
muted_on_user_path:
type: boolean
notify_before_secs:
type: integer
remind_every_secs:
type: integer
EditSuccessHandler:
description: Request body for editing the workspace success handler. Accepts both new grouped format and legacy flat format for backward compatibility.
oneOf:
@@ -32613,6 +32677,10 @@ components:
expires_at:
type: string
format: date-time
value_valid_until:
type: string
nullable: true
format: date-time
labels:
type: array
items:
@@ -32689,7 +32757,18 @@ components:
description: Whether the variable is an OAuth variable
expires_at:
type: string
description: The expiration date of the variable
description: >
When to garbage-collect the variable. Used for ephemeral secrets;
the variable is deleted once this is reached. Not the value's expiry
— see value_valid_until.
format: date-time
value_valid_until:
type: string
description: >
When the value stored in this variable stops working. Triggers the
workspace's variable expiration handler ahead of it, by the handler's
`notify_before_secs` (1 day by default); never causes the variable
to be deleted.
format: date-time
labels:
type: array
@@ -32718,6 +32797,14 @@ components:
description:
type: string
description: The new description of the variable
value_valid_until:
type: string
nullable: true
description: >
When the value stored in this variable stops working. Omit to leave
it unchanged; send null to clear it. Changing it re-arms the
workspace's variable expiration handler.
format: date-time
labels:
type: array
items:
@@ -32825,6 +32912,7 @@ components:
- "workspaces.edit_webhook"
- "workspaces.edit_copilot_config"
- "workspaces.edit_error_handler"
- "workspaces.edit_variable_expiration_handler"
- "workspaces.create"
- "workspaces.update"
- "workspaces.archive"
backend/windmill-api/src/mcp/auto_generated_endpoints.rs
+12 -1
View File
@@ -184,7 +184,12 @@ pub fn all_tools() -> Vec<EndpointTool> {
},
"expires_at": {
"type": "string",
"description": "The expiration date of the variable",
"description": "When to garbage-collect the variable. Used for ephemeral secrets; the variable is deleted once this is reached. Not the value's expiry — see value_valid_until.\n",
"format": "date-time"
},
"value_valid_until": {
"type": "string",
"description": "When the value stored in this variable stops working. Triggers the workspace's variable expiration handler ahead of it, by the handler's `notify_before_secs` (1 day by default); never causes the variable to be deleted.\n",
"format": "date-time"
},
"labels": {
@@ -272,6 +277,12 @@ pub fn all_tools() -> Vec<EndpointTool> {
"type": "string",
"description": "The new description of the variable"
},
"value_valid_until": {
"type": "string",
"nullable": true,
"description": "When the value stored in this variable stops working. Omit to leave it unchanged; send null to clear it. Changing it re-arms the workspace's variable expiration handler.\n",
"format": "date-time"
},
"labels": {
"type": "array",
"items": {
backend/windmill-api/src/mcp/session/deploy.rs
+6 -1
View File
@@ -268,11 +268,16 @@ async fn deploy_variable(
|| draft.get("is_oauth") == Some(&Value::Bool(true))
|| non_null(draft.get("account")).is_some();
let keep_stored = protected && stages_no_value;
let body = if keep_stored {
let mut body = if keep_stored {
without(draft, &["value"])
} else {
draft.clone()
};
// The draft holds the full variable, so an absent expiry was cleared, while the update
// endpoint reads an absent one as unchanged.
if let Some(body) = body.as_object_mut() {
body.entry("value_valid_until").or_insert(Value::Null);
}
ctx.post(
&ctx.ws(&format!("variables/update/{}", path_enc(path))),
body,
backend/windmill-api/src/mcp/session/gen/instructions.json
+1 -1
View File
File diff suppressed because one or more lines are too long
backend/windmill-api/src/mcp/session/gen/tools.json
+14 -1
View File
@@ -1680,11 +1680,24 @@
"type": "boolean"
},
"expires_at": {
"description": "The expiration date of the variable",
"description": "When to garbage-collect the variable. Used for ephemeral secrets; the variable is deleted once this is reached. Not the value's expiry — see value_valid_until.\n",
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$"
},
"value_valid_until": {
"description": "When the value stored in this variable stops working. Triggers the workspace's variable expiration handler ahead of it (1 day by default, set on the handler); never causes the variable to be deleted. Omit it to keep the current expiry, or pass null to clear one — for instance once the value has been rotated.",
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z|([+-](?:[01]\\d|2[0-3]):[0-5]\\d)))$"
},
{
"type": "null"
}
]
},
"labels": {
"type": "array",
"items": {
backend/windmill-api/src/mcp/session/items.rs
+20 -1
View File
@@ -363,7 +363,7 @@ fn variable_to_draft_state(variable: &Value) -> Value {
"wsSpecific".into(),
field("ws_specific").unwrap_or(json!(false)),
);
for key in ["account", "is_oauth", "expires_at"] {
for key in ["account", "is_oauth", "expires_at", "value_valid_until"] {
put(&mut state, key, field(key));
}
Value::Object(state)
@@ -429,6 +429,13 @@ fn variable_draft(args: &Value, base: Option<Value>) -> Result<Value, String> {
for (key, value) in extras {
put(&mut draft, key, value);
}
// Deploying a draft reads an absent expiry as cleared. The draft starts from the base, so a
// `null` has to remove its date rather than skip it.
if args.get("value_valid_until").is_some_and(Value::is_null) {
draft.remove("value_valid_until");
} else {
put(&mut draft, "value_valid_until", arg("value_valid_until"));
}
Ok(Value::Object(draft))
}
@@ -703,6 +710,18 @@ mod tests {
variable_draft(&json!({"path": "p", "is_secret": true}), Some(plain_empty)).is_err()
);
let dated = variable_to_draft_state(&json!({
"path": "p", "value": "v", "is_secret": false, "value_valid_until": "2027-01-15T00:00:00Z",
}));
let edit = |args: Value| variable_draft(&args, Some(dated.clone())).unwrap();
assert_eq!(
edit(json!({"path": "p", "description": "new"}))["value_valid_until"],
"2027-01-15T00:00:00Z"
);
assert!(edit(json!({"path": "p", "value_valid_until": null}))
.get("value_valid_until")
.is_none());
let oauth = variable_to_draft_state(&json!({
"path": "p", "value": "live-token", "is_secret": true, "account": 3,
"description": null, "expires_at": null,
backend/windmill-api/src/mcp/session/workspace.rs
+8
View File
@@ -499,6 +499,7 @@ pub(super) fn draft_item(kind: &str, path: &str, draft: &Value) -> Option<Value>
put(&mut value, "account", field("account"));
put(&mut value, "is_oauth", field("is_oauth"));
put(&mut value, "expires_at", field("expires_at"));
put(&mut value, "value_valid_until", field("value_valid_until"));
put(&mut value, "labels", field("labels"));
put(&mut value, "ws_specific", field("wsSpecific"));
item.insert("value".into(), Value::Object(value));
@@ -1972,4 +1973,11 @@ mod tests {
);
assert!(draft_item("app", "u/a/s", &draft).is_none());
}
#[test]
fn draft_item_carries_the_variable_expiry() {
let draft = json!({ "path": "u/a/v", "variable": { "value": "", "is_secret": true, "description": "" }, "value_valid_until": "2027-01-15T00:00:00Z" });
let item = draft_item("variable", "u/a/v", &draft).unwrap();
assert_eq!(item["value"]["value_valid_until"], "2027-01-15T00:00:00Z");
}
}
backend/windmill-api/src/workspace_settings_document.rs
+87 -11
View File
@@ -60,6 +60,7 @@ struct SettingsRow {
webhook: Option<String>,
error_handler: Option<Value>,
success_handler: Option<Value>,
variable_expiration_handler: Option<Value>,
ai_config: Option<Value>,
large_file_storage: Option<Value>,
git_sync: Option<Value>,
@@ -298,7 +299,8 @@ pub async fn load_settings_document(
) -> Result<Map<String, Value>> {
let row = sqlx::query_as::<_, SettingsRow>(
r#"SELECT
auto_invite, webhook, error_handler, success_handler, ai_config, large_file_storage,
auto_invite, webhook, error_handler, success_handler, variable_expiration_handler,
ai_config, large_file_storage,
git_sync, default_app, default_scripts, workspace.name AS name, mute_critical_alerts,
color, operator_settings, datatable, slack_team_id, slack_name, slack_command_script,
slack_oauth_client_id, slack_oauth_client_secret, ducklake, dbt_warehouses, deploy_ui,
@@ -350,11 +352,16 @@ pub async fn load_settings_document(
// In the order `wmill sync pull` has always written settings.yaml. A key that predates the
// document and is not marked `always` is written only when set, as it always was; a key
// added since is always written, so a pulled file states it and from then on manages it.
let entries: [(&str, Option<Value>, bool); 31] = [
let entries: [(&str, Option<Value>, bool); 32] = [
("auto_invite", row.auto_invite, false),
("webhook", row.webhook.map(Value::String), false),
("error_handler", row.error_handler, true),
("success_handler", row.success_handler, true),
(
"variable_expiration_handler",
row.variable_expiration_handler,
true,
),
("ai_config", row.ai_config, false),
("large_file_storage", row.large_file_storage, false),
("git_sync", git_sync, false),
@@ -680,6 +687,7 @@ const KNOWN_KEYS: &[&str] = &[
"webhook",
"error_handler",
"success_handler",
"variable_expiration_handler",
"ai_config",
"large_file_storage",
"git_sync",
@@ -1059,7 +1067,12 @@ fn plan_document(
if !unknown.is_empty() {
warnings.push(format!("Ignored unknown settings: {}", unknown.join(", ")));
}
for key in ["auto_invite", "error_handler", "success_handler"] {
for key in [
"auto_invite",
"error_handler",
"success_handler",
"variable_expiration_handler",
] {
if desired
.get(key)
.is_some_and(|v| !v.is_null() && !v.is_object())
@@ -1122,6 +1135,24 @@ fn plan_document(
Value::is_boolean,
"a boolean",
),
(
"variable_expiration_handler",
"muted_on_user_path",
Value::is_boolean,
"a boolean",
),
(
"variable_expiration_handler",
"notify_before_secs",
|v: &Value| v.as_u64().is_some_and(|n| n <= u32::MAX as u64),
"a non-negative number of seconds",
),
(
"variable_expiration_handler",
"remind_every_secs",
|v: &Value| v.as_u64().is_some_and(|n| n <= u32::MAX as u64),
"a non-negative number of seconds",
),
] {
let v = desired.get(parent).and_then(|p| p.get(key));
typed(&format!("{parent}.{key}"), v, ok, ty)?;
@@ -1135,14 +1166,19 @@ fn plan_document(
let v = normalize_protection_rules(v, &mut warnings)?;
desired.insert("protection_rules".to_string(), v);
}
// The endpoint stores an omitted mute flag as `false`.
if let Some(handler) = desired
.get_mut("error_handler")
.and_then(Value::as_object_mut)
{
for key in ["muted_on_cancel", "muted_on_user_path"] {
if is_null(handler.get(key)) {
handler.insert(key.to_string(), Value::Bool(false));
// The endpoints store an omitted mute flag as `false`.
for (handler, keys) in [
(
"error_handler",
&["muted_on_cancel", "muted_on_user_path"][..],
),
("variable_expiration_handler", &["muted_on_user_path"][..]),
] {
if let Some(handler) = desired.get_mut(handler).and_then(Value::as_object_mut) {
for key in keys {
if is_null(handler.get(*key)) {
handler.insert(key.to_string(), Value::Bool(false));
}
}
}
}
@@ -1289,6 +1325,23 @@ fn plan_document(
|v| json!({ "path": v.get("path"), "extra_args": v.get("extra_args") }),
);
// Managed only when stated, like every key added after the document.
let variable_expiration_handler = p.managed("variable_expiration_handler");
p.simple(
"variable_expiration_handler",
variable_expiration_handler,
Target::Edit(SettingsEdit::VariableExpirationHandler),
|v| {
json!({
"path": v.get("path"),
"extra_args": v.get("extra_args"),
"muted_on_user_path": v.get("muted_on_user_path").and_then(Value::as_bool).unwrap_or(false),
"notify_before_secs": v.get("notify_before_secs"),
"remind_every_secs": v.get("remind_every_secs"),
})
},
);
let lfs = p.legacy("large_file_storage");
p.simple(
"large_file_storage",
@@ -2010,6 +2063,29 @@ mod tests {
assert!(p.changes.is_empty(), "{:?}", keys(&p));
}
#[test]
fn variable_expiration_handler_is_managed_only_when_stated() {
let mc = WorkspaceCrypt::new("k");
let names = DatatableNames::default();
let mut cur = current();
cur.insert(
"variable_expiration_handler".into(),
json!({ "path": "script/f/x", "muted_on_user_path": false }),
);
let omitted = doc(json!({ "webhook": "https://b" }));
let p = plan_document(cur.clone(), omitted, true, &mc, &names).unwrap();
assert!(!keys(&p).contains(&"variable_expiration_handler"));
let same = doc(json!({ "variable_expiration_handler": { "path": "script/f/x" } }));
let p = plan_document(cur.clone(), same, false, &mc, &names).unwrap();
assert!(p.changes.is_empty(), "{:?}", keys(&p));
let cleared = doc(json!({ "variable_expiration_handler": null }));
let p = plan_document(cur, cleared, false, &mc, &names).unwrap();
assert_eq!(keys(&p), ["variable_expiration_handler"]);
assert_eq!(p.writes[0].body["path"], Value::Null);
}
#[cfg(all(feature = "private", feature = "enterprise"))]
#[test]
fn datatable_roles_are_named_in_the_document_and_written_by_id() {
backend/windmill-api/src/workspaces_export.rs
+2 -2
View File
@@ -972,9 +972,9 @@ pub(crate) async fn tarball_workspace(
// would never match the variable it created and every sync would report it changed.
let variables =
sqlx::query_as::<_, ExportableListableVariable>(if !skip_secrets.unwrap_or(false) {
"SELECT workspace_id, path, value::text AS value, is_secret, description, extra_perms, NULL::int AS account, is_oauth, expires_at, labels FROM variable WHERE workspace_id = $1 AND expires_at IS NULL"
"SELECT workspace_id, path, value::text AS value, is_secret, description, extra_perms, NULL::int AS account, is_oauth, expires_at, value_valid_until, labels FROM variable WHERE workspace_id = $1 AND expires_at IS NULL"
} else {
"SELECT workspace_id, path, value::text AS value, is_secret, description, extra_perms, NULL::int AS account, is_oauth, expires_at, labels FROM variable WHERE workspace_id = $1 AND is_secret = false AND expires_at IS NULL"
"SELECT workspace_id, path, value::text AS value, is_secret, description, extra_perms, NULL::int AS account, is_oauth, expires_at, value_valid_until, labels FROM variable WHERE workspace_id = $1 AND is_secret = false AND expires_at IS NULL"
})
.bind(&w_id)
.fetch_all(&mut *tx)
backend/windmill-common/src/lib.rs
+7 -1
View File
@@ -175,7 +175,13 @@ pub const DEFAULT_AI_SHARED_ARTIFACT_RETENTION_SECS: i64 = 60 * 60 * 24 * 30;
pub const WM_DEPLOYERS_GROUP: &str = "wm_deployers";
/// Groups Windmill creates lazily (`ON CONFLICT DO NOTHING`) and grants permissions to before
/// they exist; a member who created one first would inherit those grants.
pub const RESERVED_GROUP_NAMES: &[&str] = &["error_handler", "success_handler", "slack", "teams"];
pub const RESERVED_GROUP_NAMES: &[&str] = &[
"error_handler",
"success_handler",
"variable_expiration_handler",
"slack",
"teams",
];
/// A century. Every consumer has to survive `now - retention`, and the ceilings are much lower
/// than an `i64`: `DateTime` subtraction panics past year 262143, and the `(<n> s)::interval`
backend/windmill-common/src/utils.rs
+7
View File
@@ -286,6 +286,13 @@ pub fn sanitize_db_error(e: sqlx::Error) -> Error {
Some("23514") if db_err.constraint() == Some("variable_value_length") => {
Error::BadRequest("A field exceeds its maximum length".to_string())
}
Some("23514") if db_err.constraint() == Some("variable_oauth_no_value_expiry") => {
Error::BadRequest(
"A variable linked to an OAuth account cannot have a value expiry: Windmill \
refreshes its token"
.to_string(),
)
}
// check_violation — `proper_id` / `proper_name` and friends
Some("23514") => Error::BadRequest(
"Invalid path or name: it does not match the required format".to_string(),
backend/windmill-common/src/variables.rs
+7
View File
@@ -260,6 +260,7 @@ pub struct ListableVariable {
pub refresh_error: Option<String>,
pub is_linked: Option<bool>,
pub expires_at: Option<chrono::DateTime<Utc>>,
pub value_valid_until: Option<chrono::DateTime<Utc>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub labels: Option<Vec<String>>,
/// Labels inherited from the parent folder, computed at read time.
@@ -302,6 +303,8 @@ pub struct ExportableListableVariable {
#[serde(skip_serializing_if = "Option::is_none")]
pub expires_at: Option<chrono::DateTime<Utc>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub value_valid_until: Option<chrono::DateTime<Utc>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub labels: Option<Vec<String>>,
}
@@ -317,7 +320,11 @@ pub struct CreateVariable {
pub description: String,
pub account: Option<i32>,
pub is_oauth: Option<bool>,
/// When the row itself is garbage-collected.
pub expires_at: Option<chrono::DateTime<Utc>>,
/// When the value stops working. Drives the workspace's variable expiration handler;
/// never deletes the row.
pub value_valid_until: Option<chrono::DateTime<Utc>>,
#[serde(default)]
pub labels: Option<Vec<String>>,
#[serde(default)]
backend/windmill-queue/src/jobs.rs
+100 -1
View File
@@ -170,12 +170,15 @@ const ERROR_HANDLER_USERNAME: &str = "error_handler";
pub const SCHEDULE_ERROR_HANDLER_USERNAME: &str = "schedule_error_handler";
const GLOBAL_ERROR_HANDLER_USERNAME: &str = "global";
const SUCCESS_HANDLER_USERNAME: &str = "success_handler";
const VARIABLE_EXPIRATION_HANDLER_USERNAME: &str = "variable_expiration_handler";
pub const ERROR_HANDLER_USER_GROUP: &str = "g/error_handler";
pub const ERROR_HANDLER_USER_EMAIL: &str = "error_handler@windmill.dev";
pub const SCHEDULE_ERROR_HANDLER_USER_EMAIL: &str = "schedule_error_handler@windmill.dev";
pub const SUCCESS_HANDLER_USER_GROUP: &str = "g/success_handler";
pub const SUCCESS_HANDLER_USER_EMAIL: &str = "success_handler@windmill.dev";
pub const VARIABLE_EXPIRATION_HANDLER_USER_GROUP: &str = "g/variable_expiration_handler";
pub const VARIABLE_EXPIRATION_HANDLER_USER_EMAIL: &str = "variable_expiration_handler@windmill.dev";
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CanceledBy {
@@ -3599,6 +3602,100 @@ pub async fn push_error_handler<'a, 'c, T: Serialize + Send + Sync>(
return Ok(uuid);
}
/// Handler arguments, so the field names are wire format. Never add the value: job args and
/// results are stored in cleartext and shown in Runs.
#[derive(Serialize)]
pub struct ExpiringVariable {
pub workspace_id: String,
pub variable_path: String,
pub description: String,
pub value_valid_until: DateTime<Utc>,
pub is_secret: bool,
}
/// Takes and returns the caller's transaction so the claim and the push commit together.
pub(crate) async fn push_variable_expiration_handler<'c>(
db: &Pool<Postgres>,
tx: Transaction<'c, Postgres>,
w_id: &str,
(payload, tag, on_behalf_of): (
JobPayload,
Option<String>,
Option<windmill_common::jobs::OnBehalfOf>,
),
expiring: &ExpiringVariable,
extra_args: Option<Json<Box<RawValue>>>,
) -> windmill_common::error::Result<(Uuid, Transaction<'c, Postgres>)> {
// Serialized whole so a new `ExpiringVariable` field reaches handlers. Any writer of the
// variable controls these, so only they are neutralized: the admin's `extra_args` keep their
// `$res:` references, which is how the handler gets its token.
let mut extra = omit_untrusted_interpolation(
serde_json::from_value(serde_json::to_value(expiring)?)?,
"The following resource has been omitted for security reasons: ",
);
if let Some(args_v) = extra_args {
if let Ok(args_m) = serde_json::from_str::<HashMap<String, Box<RawValue>>>(args_v.get()) {
extra.extend(args_m);
} else {
return Err(error::Error::ExecutionErr(
"args of scripts needs to be dict".to_string(),
));
}
}
let (email, permissioned_as) = if let Some(on_behalf_of) = on_behalf_of.as_ref() {
(
on_behalf_of.email.as_str(),
on_behalf_of.permissioned_as.clone(),
)
} else {
(
VARIABLE_EXPIRATION_HANDLER_USER_EMAIL,
VARIABLE_EXPIRATION_HANDLER_USER_GROUP.to_string(),
)
};
let empty: HashMap<String, Box<RawValue>> = HashMap::new();
let args = PushArgs { extra: Some(extra), args: &empty };
push(
db,
PushIsolationLevel::Transaction(tx),
w_id,
payload,
args,
VARIABLE_EXPIRATION_HANDLER_USERNAME,
email,
permissioned_as,
Some(&format!(
"variable.expiration.{}",
expiring.variable_path.replace('/', ".")
)),
None,
None,
None,
None,
None,
None,
None,
false,
false,
None,
true,
tag,
None,
None,
None,
None,
false,
None,
None,
None,
None,
)
.await
}
pub async fn push_success_handler<'a, 'c, T: Serialize + Send + Sync>(
db: &Pool<Postgres>,
job_id: Uuid,
@@ -6357,6 +6454,7 @@ async fn push_inner<'c, 'd>(
if !team_plan_status.premium
&& email != ERROR_HANDLER_USER_EMAIL
&& email != SCHEDULE_ERROR_HANDLER_USER_EMAIL
&& email != VARIABLE_EXPIRATION_HANDLER_USER_EMAIL
&& email != recovery_email
&& email != "worker@windmill.dev"
&& email != SUPERADMIN_SECRET_EMAIL
@@ -8886,7 +8984,8 @@ mod preset_handler_path_tests {
let hub_paths: std::collections::HashMap<String, String> =
serde_json::from_str(include_str!("../../../frontend/src/lib/hubPaths.json")).unwrap();
for (name, path) in hub_paths {
if path.contains("-handler-") {
// Variable expiration handlers run as their own group whatever their path.
if path.contains("-handler-") && !path.contains("variable-expiration") {
assert!(
is_preset_handler_path(&path),
"{name}: {path} is not pinned"
backend/windmill-queue/src/lib.rs
+1
View File
@@ -34,6 +34,7 @@ pub mod jobs;
pub mod jobs_ee;
pub mod jobs_oss;
pub mod schedule;
pub mod variable_expiration;
pub use jobs::*;
pub mod flow_status;
pub mod tags;
backend/windmill-queue/src/variable_expiration.rs
+309
View File
@@ -0,0 +1,309 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
//! Dispatches the workspace variable expiration handler for variables coming due.
use std::collections::{HashMap, HashSet};
use serde_json::value::RawValue;
use sqlx::types::Json;
use windmill_common::jobs::get_payload_tag_from_prefixed_path;
use windmill_common::utils::{report_critical_error, report_recovered_critical_error};
use windmill_common::DB;
use crate::jobs::{push_variable_expiration_handler, ExpiringVariable};
/// Lead time when the handler setting has no `notify_before_secs`.
const VARIABLE_EXPIRATION_DEFAULT_NOTIFY_BEFORE_SECS: i64 = 24 * 3600;
/// Subtracted from `remind_every_secs` so a reminder whose interval is a multiple of the
/// sweep's period is not skipped by tick jitter.
const VARIABLE_EXPIRATION_REMINDER_SLACK_SECS: i64 = 30;
/// Advisory lock id ensuring only one server replica sweeps at a time. Must not reuse one of
/// the other monitor pass locks (737_483_920..=737_483_925): both sides try-lock and skip.
const VARIABLE_EXPIRATION_LOCK_ID: i64 = 737_483_926;
/// System caller only: dispatches across every workspace. A handler job that fails is not
/// re-dispatched before its next reminder, as re-running it would repeat its side effects
/// against the third party it rotates; retrying is the handler's job.
pub async fn dispatch_expiring_variables(db: &DB) {
// Transaction-scoped, not session-scoped: a session lock would go back to the pool still
// held if this future were dropped mid-pass, wedging the sweep on every replica.
let mut lock_tx = match db.begin().await {
Ok(tx) => tx,
Err(e) => {
tracing::error!("variable expiration: failed to begin lock tx: {e:#}");
return;
}
};
let locked: bool = match sqlx::query_scalar("SELECT pg_try_advisory_xact_lock($1)")
.bind(VARIABLE_EXPIRATION_LOCK_ID)
.fetch_one(&mut *lock_tx)
.await
{
Ok(v) => v,
Err(e) => {
tracing::error!("variable expiration: advisory lock failed: {e:#}");
return;
}
};
if !locked {
return;
}
dispatch_expiring_variables_inner(db).await;
if let Err(e) = lock_tx.rollback().await {
tracing::error!("variable expiration: releasing lock failed: {e:#}");
}
}
async fn dispatch_expiring_variables_inner(db: &DB) {
resolve_removed_handlers(db).await;
// No lower bound on `value_valid_until`, so a downed instance catches up. No cap either:
// everything due is tried every pass, so no workspace or kind of notice can be crowded out.
let due = sqlx::query!(
r#"SELECT v.workspace_id, v.path, v.description, v.value_valid_until AS "value_valid_until!",
v.is_secret, v.expiration_dispatched_for, v.expiration_notified_at,
ws.variable_expiration_handler->>'path' AS "handler_path!",
(ws.variable_expiration_handler->'extra_args')::text::json
AS "extra_args: Json<Box<RawValue>>"
FROM variable v
JOIN workspace_settings ws ON ws.workspace_id = v.workspace_id
JOIN workspace w ON w.id = v.workspace_id
WHERE NOT w.deleted
AND v.value_valid_until IS NOT NULL
AND (v.value_valid_until IS DISTINCT FROM v.expiration_dispatched_for
OR (v.value_valid_until > now()
AND v.expiration_notified_at <= now() - make_interval(secs =>
NULLIF((ws.variable_expiration_handler->>'remind_every_secs')::bigint, 0) - $2)))
AND v.value_valid_until <= now() + make_interval(secs =>
COALESCE((ws.variable_expiration_handler->>'notify_before_secs')::bigint, $1))
AND ws.variable_expiration_handler->>'path' IS NOT NULL
AND (NOT COALESCE((ws.variable_expiration_handler->>'muted_on_user_path')::boolean, false)
OR v.path NOT LIKE 'u/%')"#,
VARIABLE_EXPIRATION_DEFAULT_NOTIFY_BEFORE_SECS,
VARIABLE_EXPIRATION_REMINDER_SLACK_SECS,
)
.fetch_all(db)
.await;
let due = match due {
Ok(due) => due,
Err(e) => {
tracing::error!("Error listing expiring variables: {e:#}");
return;
}
};
let mut dispatched = 0usize;
// A handler fails alike for every variable of its workspace: try it once per pass.
let mut failed_this_pass: HashSet<String> = HashSet::new();
// Hub handlers are shared by every workspace using them, so a hub outage is fetched once
// per pass rather than once per workspace.
let mut failed_hub_handlers: HashMap<String, String> = HashMap::new();
for row in due {
if failed_this_pass.contains(&row.workspace_id) {
continue;
}
// Resolved before the claim's transaction opens, so a pass never holds one connection
// while waiting on another.
let handler = match failed_hub_handlers.get(&row.handler_path) {
Some(error) => Err(error.clone()),
None => get_payload_tag_from_prefixed_path(&row.handler_path, db, &row.workspace_id)
.await
.map_err(|e| {
let error = format!("{e:#}");
if row.handler_path.starts_with("script/hub/")
|| row.handler_path.starts_with("flow/hub/")
{
failed_hub_handlers.insert(row.handler_path.clone(), error.clone());
}
error
}),
};
let handler = match handler {
Ok(handler) => handler,
Err(error) => {
failed_this_pass.insert(row.workspace_id.clone());
record_failure(db, &row.workspace_id, &row.handler_path, &error).await;
continue;
}
};
// Claim and push in one transaction. The claim matches the row as it was read, so it
// loses both to a concurrent re-arm and to whichever replica got there first.
let mut tx = match db.begin().await {
Ok(tx) => tx,
Err(e) => {
tracing::error!("Error opening tx for expiring variable {}: {e:#}", row.path);
continue;
}
};
let claimed = sqlx::query_scalar!(
"UPDATE variable SET expiration_dispatched_for = $3, expiration_notified_at = now()
WHERE workspace_id = $1 AND path = $2 AND value_valid_until = $3
AND expiration_dispatched_for IS NOT DISTINCT FROM $4
AND expiration_notified_at IS NOT DISTINCT FROM $5
RETURNING 1",
row.workspace_id,
row.path,
row.value_valid_until,
row.expiration_dispatched_for,
row.expiration_notified_at,
)
.fetch_optional(&mut *tx)
.await;
match claimed {
Ok(Some(_)) => {}
Ok(None) => continue,
Err(e) => {
tracing::error!("Error claiming expiring variable {}: {e:#}", row.path);
continue;
}
}
let expiring = ExpiringVariable {
workspace_id: row.workspace_id.clone(),
variable_path: row.path.clone(),
description: row.description,
value_valid_until: row.value_valid_until,
is_secret: row.is_secret,
};
let pushed = push_variable_expiration_handler(
db,
tx,
&row.workspace_id,
handler,
&expiring,
row.extra_args,
)
.await;
let tx = match pushed {
Ok((_job_id, tx)) => tx,
Err(e) => {
// Dropping the tx un-claims the variable, so the next pass retries it.
failed_this_pass.insert(row.workspace_id.clone());
record_failure(db, &row.workspace_id, &row.handler_path, &format!("{e:#}")).await;
continue;
}
};
if let Err(e) = tx.commit().await {
tracing::error!("Error committing dispatch for {}: {e:#}", row.path);
continue;
}
windmill_common::feature_usage::log_feature_usage(
"variable_expiration",
"dispatched",
if row.is_secret { "secret" } else { "plain" },
);
dispatched += 1;
recover(db, &row.workspace_id, &row.handler_path).await;
}
if dispatched > 0 {
tracing::info!("Dispatched {dispatched} variable expiration handler job(s)");
}
}
/// Alerts once per failure, not per pass: the alert is resolved when the handler next starts.
async fn record_failure(db: &DB, w_id: &str, handler_path: &str, error: &str) {
let newly_failing = sqlx::query_scalar!(
"UPDATE workspace_settings SET variable_expiration_failed_at = now()
WHERE workspace_id = $1 AND variable_expiration_failed_at IS NULL
RETURNING 1",
w_id,
)
.fetch_optional(db)
.await;
match newly_failing {
Ok(Some(_)) => {
report_critical_error(
format!(
"The variable expiration handler {handler_path} of workspace {w_id} failed to \
start, so variables in this workspace are not being notified. It is retried \
on every check, and this alert is resolved once it starts: {error}"
),
db.clone(),
Some(w_id),
Some(&alert_resource(w_id)),
)
.await
}
Ok(None) => {}
Err(e) => {
tracing::error!("Error recording variable expiration handler failure of {w_id}: {e:#}")
}
}
}
/// Tags a failing handler's critical alerts, so they can be resolved together.
fn alert_resource(w_id: &str) -> String {
format!("variable_expiration_handler:{w_id}")
}
/// A removed handler has nothing left to retry, so its failure would never be resolved.
async fn resolve_removed_handlers(db: &DB) {
let removed = sqlx::query_scalar!(
"UPDATE workspace_settings SET variable_expiration_failed_at = NULL
WHERE variable_expiration_failed_at IS NOT NULL AND variable_expiration_handler IS NULL
RETURNING workspace_id",
)
.fetch_all(db)
.await;
match removed {
Ok(removed) => {
for w_id in removed {
report_recovered_critical_error(
format!("The variable expiration handler of workspace {w_id} was removed"),
db.clone(),
Some(&w_id),
Some(&alert_resource(&w_id)),
)
.await;
}
}
Err(e) => tracing::error!("Error resolving removed variable expiration handlers: {e:#}"),
}
}
async fn recover(db: &DB, w_id: &str, handler_path: &str) {
let cleared = sqlx::query_scalar!(
"UPDATE workspace_settings SET variable_expiration_failed_at = NULL
WHERE workspace_id = $1 AND variable_expiration_failed_at IS NOT NULL
RETURNING 1",
w_id,
)
.fetch_optional(db)
.await;
match cleared {
Ok(Some(_)) => {
report_recovered_critical_error(
format!(
"The variable expiration handler {handler_path} of workspace {w_id} started again"
),
db.clone(),
Some(w_id),
Some(&alert_resource(w_id)),
)
.await
}
Ok(None) => {}
Err(e) => {
tracing::error!("Error clearing variable expiration handler failure of {w_id}: {e:#}")
}
}
}
backend/windmill-store/src/variables.rs
+29 -4
View File
@@ -149,6 +149,7 @@ async fn list_variables(
"resource.path IS NOT NULL as is_linked",
"account.refresh_token != '' as is_refreshed",
"variable.expires_at",
"variable.value_valid_until",
"variable.labels",
"folder_labels(variable.workspace_id, variable.path) as inherited_labels",
"ws_specific.path IS NOT NULL as ws_specific",
@@ -245,7 +246,7 @@ async fn list_variables(
for row in draft_only_rows {
let v: serde_json::Value =
serde_json::from_str(row.value.0.get()).unwrap_or(serde_json::Value::Null);
// VariableEditor's `VariableState`: { path, variable: { value, is_secret, description }, labels?, wsSpecific }
// VariableEditor's `VariableState`: { path, variable: { value, is_secret, description }, labels?, wsSpecific, value_valid_until? }
let path = v
.get("path")
.and_then(|s| s.as_str())
@@ -284,6 +285,11 @@ async fn list_variables(
})
});
let ws_specific = v.get("wsSpecific").and_then(|x| x.as_bool());
let value_valid_until = v
.get("value_valid_until")
.and_then(|x| x.as_str())
.and_then(|s| chrono::DateTime::parse_from_rfc3339(s).ok())
.map(|dt| dt.with_timezone(&chrono::Utc));
rows.push(ListableVariable {
workspace_id: w_id.clone(),
@@ -299,6 +305,7 @@ async fn list_variables(
refresh_error: None,
is_linked: None,
expires_at: None,
value_valid_until,
labels,
// No deployed row to inherit folder labels from.
inherited_labels: None,
@@ -339,7 +346,7 @@ async fn get_variable(
let variable_o = sqlx::query_as::<_, ListableVariable>(
"SELECT variable.workspace_id, variable.path, variable.value::text AS value, variable.is_secret,
variable.description, variable.extra_perms, variable.account, variable.is_oauth,
variable.expires_at, variable.labels,
variable.expires_at, variable.value_valid_until, variable.labels,
folder_labels(variable.workspace_id, variable.path) as inherited_labels,
variable.edited_at, variable.edited_by,
(now() > account.expires_at) as is_expired, account.refresh_error,
@@ -712,8 +719,8 @@ pub async fn create_variable_with_account(
sqlx::query!(
"INSERT INTO variable
(workspace_id, path, value, is_secret, description, account, is_oauth, expires_at, labels, edited_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)",
(workspace_id, path, value, is_secret, description, account, is_oauth, expires_at, value_valid_until, labels, edited_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)",
&w_id,
variable.path,
value,
@@ -722,6 +729,7 @@ pub async fn create_variable_with_account(
account_id,
variable.is_oauth.unwrap_or(false),
variable.expires_at,
variable.value_valid_until,
variable.labels.as_deref() as Option<&[String]>,
&authed.username
)
@@ -1122,6 +1130,13 @@ struct EditVariable {
description: Option<String>,
labels: Option<Vec<String>>,
ws_specific: Option<bool>,
/// Absent leaves the stored expiry alone and `null` clears it, so a partial update
/// never clears it by omission.
#[serde(
default,
deserialize_with = "windmill_common::more_serde::double_option"
)]
value_valid_until: Option<Option<chrono::DateTime<chrono::Utc>>>,
}
#[derive(Deserialize)]
@@ -1216,6 +1231,16 @@ async fn update_variable(
has_sql_updates = true;
}
if let Some(nvalue_valid_until) = ns.value_valid_until {
// Nothing to reset: the sweep compares the date with `expiration_dispatched_for`, so
// moving it re-arms the handler and rewriting the same date does not.
match nvalue_valid_until {
Some(ts) => sqlb.set("value_valid_until", "?".bind(&ts.to_rfc3339())),
None => sqlb.set("value_valid_until", "NULL"),
};
has_sql_updates = true;
}
if let Some(nbool) = ns.is_secret {
let old_secret = sqlx::query_scalar!(
"SELECT is_secret from variable WHERE path = $1 AND workspace_id = $2",
cli/src/commands/variable/variable.ts
+10 -1
View File
@@ -98,6 +98,7 @@ export interface VariableFile {
description: string;
account?: number;
is_oauth?: boolean;
value_valid_until?: string;
extra_perms?: Record<string, boolean>;
}
@@ -162,7 +163,14 @@ export async function pushVariable(
const { extra_perms: localPerms, ...localVariableBody } = localVariable;
if (variable) {
if (isSuperset(localVariableBody, variable)) {
// An expiry removed from the file is sent as `null` (absent keeps the stored date), and
// `isSuperset` cannot see a removed key, so the clear is detected here.
const clearsValueValidUntil =
localVariableBody.value_valid_until === undefined &&
variable.value_valid_until !== undefined &&
variable.value_valid_until !== null;
if (!clearsValueValidUntil && isSuperset(localVariableBody, variable)) {
log.debug(`Variable ${remotePath} is up-to-date`);
} else {
log.debug(`Variable ${remotePath} is not up-to-date, updating`);
@@ -188,6 +196,7 @@ export async function pushVariable(
...localVariableBody,
is_secret: nextIsSecret,
...(wsSpecific !== undefined ? { ws_specific: wsSpecific } : {}),
...(clearsValueValidUntil ? { value_valid_until: null } : {}),
},
});
}
cli/src/core/settings.ts
+10
View File
@@ -34,12 +34,21 @@ export interface SuccessHandlerConfig {
extra_args?: any;
}
export interface VariableExpirationHandlerConfig {
path?: string;
extra_args?: any;
muted_on_user_path?: boolean;
notify_before_secs?: number;
remind_every_secs?: number;
}
export interface SimplifiedSettings {
// Grouped format (current). Explicit `null` on error_handler / success_handler
// signals "clear the remote value"; `undefined` means "not managed by git".
auto_invite?: AutoInviteConfig;
error_handler?: ErrorHandlerConfig | null;
success_handler?: SuccessHandlerConfig | null;
variable_expiration_handler?: VariableExpirationHandlerConfig | null;
// Other fields
webhook?: string;
@@ -85,6 +94,7 @@ export const SETTINGS_MANAGED_ONLY_WHEN_PRESENT = [
"guest_jwt_jwks_url",
"error_handler_fallback_to_instance_alerts",
"add_admins_and_developers_to_forks",
"variable_expiration_handler",
] as const;
// Legacy settings interface for reading old settings.yaml files
cli/test/deploy_variable_expiry_unit.test.ts
+60
View File
@@ -0,0 +1,60 @@
import { expect, test } from "bun:test";
import { deployItem } from "../windmill-utils-internal/src/deploy.ts";
// Workspace deploy builds the variable body field by field, so a forgotten field is
// dropped silently on promotion, and a source with no expiry must send `null`.
function recordingProvider(
captured: [string, any][],
exists: boolean,
value_valid_until: string | null,
) {
return {
existsVariable: async () => exists,
getVariable: async () => ({
path: "f/x/v",
value: "secret",
is_secret: true,
description: "d",
value_valid_until,
}),
createVariable: async (p: any) =>
void captured.push(["createVariable", p.requestBody]),
updateVariable: async (p: any) =>
void captured.push(["updateVariable", p.requestBody]),
} as any;
}
test("deployItem: carries value_valid_until, and clears it explicitly", async () => {
const captured: [string, any][] = [];
// Written once per branch: create and update.
for (const exists of [false, true]) {
await deployItem(
recordingProvider(captured, exists, "2027-03-15T08:00:00Z"),
"variable" as any,
"f/x/v",
"src",
"dst",
);
await deployItem(
recordingProvider(captured, exists, null),
"variable" as any,
"f/x/v",
"src",
"dst",
);
}
expect(captured.map(([fn]) => fn)).toEqual([
"createVariable",
"createVariable",
"updateVariable",
"updateVariable",
]);
expect(captured.map(([, body]) => body.value_valid_until)).toEqual([
"2027-03-15T08:00:00Z",
null,
"2027-03-15T08:00:00Z",
null,
]);
});
cli/test/variable_resource_push.test.ts
+64
View File
@@ -133,6 +133,70 @@ describe("variable", () => {
});
});
test("push clears value_valid_until when the key is dropped from the file", async () => {
await withTestBackend(async (backend, tempDir) => {
await setupWorkspaceProfile(backend);
const uniqueId = Date.now();
const path = `f/test/expiry_var_${uniqueId}`;
const createResp = await backend.apiRequest!(
`/api/w/${backend.workspace}/variables/create`,
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
path,
value: "original_value",
is_secret: false,
description: "Expiring",
value_valid_until: "2027-03-15T08:00:00Z",
}),
}
);
expect(createResp.status).toBeLessThan(300);
await createResp.text();
const created = await backend.apiRequest!(
`/api/w/${backend.workspace}/variables/get/${path}`
);
expect((await created.json()).value_valid_until).toBe("2027-03-15T08:00:00Z");
await writeFile(
join(tempDir, "wmill.yaml"),
`defaultTs: bun\nincludes:\n - "**"\nexcludes: []\n`,
"utf-8"
);
await mkdir(join(tempDir, "f", "test"), { recursive: true });
// No value_valid_until: the CLI must send an explicit null to clear it.
await writeFile(
join(tempDir, `${path}.variable.yaml`),
`value: "original_value"\nis_secret: false\ndescription: "Expiring"\n`,
"utf-8"
);
const pushResult = await backend.runCLICommand(
["sync", "push", "--yes", "--includes", `${path}**`],
tempDir
);
expect(pushResult.code).toEqual(0);
const apiResp = await backend.apiRequest!(
`/api/w/${backend.workspace}/variables/get/${path}`
);
expect(apiResp.status).toEqual(200);
const varData = await apiResp.json();
expect(varData.value_valid_until ?? null).toBeNull();
// A clear that did not land leaves the diff pending, so push success is not proof.
const secondPush = await backend.runCLICommand(
["sync", "push", "--yes", "--includes", `${path}**`],
tempDir
);
expect(secondPush.code).toEqual(0);
expect(secondPush.stdout).toContain("0 changes to apply");
});
});
test("add creates secret by default and preserves fields on update", async () => {
await withTestBackend(async (backend, tempDir) => {
await setupWorkspaceProfile(backend);
cli/windmill-utils-internal/package-lock.json
Generated
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "windmill-utils-internal",
"version": "1.8.3",
"version": "1.8.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "windmill-utils-internal",
"version": "1.8.3",
"version": "1.8.4",
"license": "Apache 2.0",
"devDependencies": {
"@types/node": "^24.2.0",
cli/windmill-utils-internal/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "windmill-utils-internal",
"version": "1.8.3",
"version": "1.8.4",
"description": "Internal utility functions for Windmill",
"main": "dist/cjs/index.js",
"module": "dist/esm/index.js",
cli/windmill-utils-internal/src/deploy.ts
+4
View File
@@ -617,6 +617,8 @@ export async function deployItem(
is_secret: variable.is_secret,
description: variable.description ?? "",
labels: variable.labels,
// `null` clears; absent would keep the target's stored date.
value_valid_until: variable.value_valid_until ?? null,
},
alreadyEncrypted: false,
});
@@ -629,6 +631,7 @@ export async function deployItem(
is_secret: variable.is_secret,
description: variable.description ?? "",
labels: variable.labels,
value_valid_until: variable.value_valid_until ?? null,
},
});
}
@@ -930,6 +933,7 @@ export async function getItemValue(
value: variable.value,
is_secret: variable.is_secret,
description: variable.description,
value_valid_until: variable.value_valid_until ?? undefined,
};
} else if (kind === "resource") {
const resource = await provider.getResource({ workspace, path });
docs/feature-telemetry.md
+5 -3
View File
@@ -4,10 +4,12 @@
anonymous usage-stats payload. It answers "does anyone use this, and which variant do they pick"
without any identifying data leaving the instance.
It currently carries 62 registered actions across twenty-one features (`ai_session`, `ai_chat`,
It currently carries 78 registered actions across twenty-nine features (`ai_session`, `ai_chat`,
`ai_fix`, `ai_agent`, `ai_decision`, `ai_agent_eval`, `app_sandbox`, `datatable`, `db_manager`, `flow_editor`,
`flow_run`, `flow_step`, `home`, `run_form`, `debugger`, `trigger`, `command_script`,
`hub_script`, `usage_meter`, `sso_groups_claim`, `cloud_trial_offer`). Nearly all of the
`flow_run`, `flow_step`, `home`, `header_nav`, `run_form`, `debugger`, `trigger`, `command_script`,
`hub_script`, `usage_meter`, `sso_groups_claim`, `cloud_trial_offer`, `job_token_scopes`,
`worker_draining`, `worker_group`, `worker_resource_stats`, `dev_workspace_isolation`,
`slack_agent`, `variable_expiration`). Nearly all of the
product is uninstrumented, so new user-facing work is the opportunity to change that.
## When to instrument
frontend/package-lock.json
Generated
+4 -4
View File
@@ -110,7 +110,7 @@
"windmill-parser-wasm-wac": "1.830.0",
"windmill-parser-wasm-yaml": "1.770.0",
"windmill-sql-datatype-parser-wasm": "1.512.0",
"windmill-utils-internal": "1.8.3",
"windmill-utils-internal": "1.8.4",
"xterm": "^5.3.0",
"xterm-readline": "^1.1.2",
"y-monaco": "^0.1.4",
@@ -14719,9 +14719,9 @@
"integrity": "sha512-uHNL8F72/Tf96xF3hOHnPDjkEyqXw7fNjcPJiUhth9sTQkcwUIoJMOdwm8/cs+j9kKVRJ4tgNYMHEBLylazp6g=="
},
"node_modules/windmill-utils-internal": {
"version": "1.8.3",
"resolved": "https://registry.npmjs.org/windmill-utils-internal/-/windmill-utils-internal-1.8.3.tgz",
"integrity": "sha512-Lp8fVM/uHUvO6q01rIODIR46hTQ9D9R9BEiKoZHQqw9NJoy1OxhcsvX4usBbkXff/GbHlkWiQfZXCOn0ozJeKQ==",
"version": "1.8.4",
"resolved": "https://registry.npmjs.org/windmill-utils-internal/-/windmill-utils-internal-1.8.4.tgz",
"integrity": "sha512-s27Di5z/vonhMLd52P4i3iILQyty6ZIXYc89CCdkKZzseE6WiX/VMwdTF4nvNCjV3xN0WhAtXR+/PBhzarU2GQ==",
"license": "Apache 2.0"
},
"node_modules/word-wrap": {
frontend/package.json
+1 -1
View File
@@ -177,7 +177,7 @@
"windmill-parser-wasm-wac": "1.830.0",
"windmill-parser-wasm-yaml": "1.770.0",
"windmill-sql-datatype-parser-wasm": "1.512.0",
"windmill-utils-internal": "1.8.3",
"windmill-utils-internal": "1.8.4",
"xterm": "^5.3.0",
"xterm-readline": "^1.1.2",
"y-monaco": "^0.1.4",
frontend/src/lib/components/CompareWorkspaces.svelte
+2 -1
View File
@@ -290,7 +290,8 @@
path,
value: v.value ?? '',
is_secret: v.is_secret ?? false,
description: v.description ?? ''
description: v.description ?? '',
value_valid_until: v.value_valid_until ?? undefined
}
})
}
frontend/src/lib/components/ErrorOrRecoveryHandler.svelte
+30 -7
View File
@@ -16,6 +16,15 @@
'trigger_path'
]
/** Arguments the variable expiration sweep always sends. */
const variableExpirationHandlerArgs = [
'workspace_id',
'variable_path',
'description',
'value_valid_until',
'is_secret'
]
export const slackErrorHandlerHubPathEnding = '/workspace-or-schedule-error-handler-slack'
</script>
@@ -78,6 +87,7 @@
/** Offer the instance critical alert channels as a destination. Workspace-level
* error handling only: schedules and triggers have no such setting. */
showInstanceAlerts?: boolean
customRequiresEE?: boolean
}
let {
@@ -90,9 +100,15 @@
customTabTooltip,
noMargin = false,
workspace = undefined,
showInstanceAlerts = false
showInstanceAlerts = false,
customRequiresEE = true
}: Props = $props()
let isVariableExpiration = $derived(errorOrRecovery === 'variableExpiration')
let handlerArgs = $derived(
isVariableExpiration ? variableExpirationHandlerArgs : errorHandlerArgs
)
let effectiveWorkspace = $derived(workspace ?? $operatingWorkspace)
// Carry the workspace onto the "create from template" route, so the handler script is
// created where this handler is saved. `customScriptTemplate` already has a query string.
@@ -156,7 +172,11 @@
let submitted_job = await testJobFunction({
workspace: effectiveWorkspace!,
requestBody: {
hub_script_path: draft[platform].path,
// Ours are not test-runnable presets; the error handler's checks the same
// token and channel.
hub_script_path: isVariableExpiration
? builtinHubPath('error', platform)
: draft[platform].path,
channel: channel,
test_msg: `This is a notification to test the connection between ${platform} and Windmill workspace '${effectiveWorkspace!}'`
}
@@ -250,7 +270,7 @@
$effect(() => {
const { path, kind } = draft.custom
path &&
loadHandlerScriptArgs(path, kind, errorHandlerArgs).then(
loadHandlerScriptArgs(path, kind, handlerArgs).then(
(schema) => (customHandlerSchema = schema)
)
})
@@ -258,7 +278,7 @@
$effect(() => {
const path = draft.slack.path
path &&
loadHandlerScriptArgs(path, 'script', [...errorHandlerArgs, 'slack']).then(
loadHandlerScriptArgs(path, 'script', [...handlerArgs, 'slack']).then(
(schema) => (slackHandlerSchema = schema)
)
})
@@ -269,7 +289,9 @@
{#snippet children({ item })}
<ToggleButton label="Slack" value="slack" {item} disabled={!isEditable} />
<ToggleButton label="Teams" value="teams" {item} disabled={!isEditable} />
<ToggleButton label="Email" value="email" {item} disabled={!isEditable} />
{#if !isVariableExpiration}
<ToggleButton label="Email" value="email" {item} disabled={!isEditable} />
{/if}
<ToggleButton
label="Custom"
value="custom"
@@ -294,7 +316,7 @@
<div class="flex flex-col gap-1">
<div class="flex flex-row">
<ScriptPicker
disabled={!isEditable || !$enterpriseLicense}
disabled={!isEditable || (customRequiresEE && !$enterpriseLicense)}
kinds={['script', 'failure']}
allowFlow={true}
bind:scriptPath={draft.custom.path}
@@ -343,7 +365,7 @@
/>
{/await}
{#if customHandlerSchema && customHandlerSchema.properties && Object.keys(customHandlerSchema.properties).length === 0}
<div class="text-xs text-secondary">This error handler takes no extra arguments</div>
<div class="text-xs text-secondary">This handler takes no extra arguments</div>
{/if}
</div>
{/if}
@@ -381,6 +403,7 @@
}}
bind:args={draft.slack.args}
shouldHideNoInputs
compact
className="text-xs"
/>
{/await}
frontend/src/lib/components/VariableEditor.svelte
+45 -21
View File
@@ -1,5 +1,5 @@
<script lang="ts">
import { VariableService, WorkspaceService } from '$lib/gen'
import { VariableService, WorkspaceService, type ListableVariable } from '$lib/gen'
import { createEventDispatcher, onDestroy, untrack } from 'svelte'
import { Button } from './common'
import Drawer from './common/drawer/Drawer.svelte'
@@ -29,17 +29,37 @@
import { useDraftConflictSession } from '$lib/draftConflictSession.svelte'
import { isEncryptedDraftValue } from '$lib/encryptedDraft'
import { setLocalDraftHint } from '$lib/localDraftHints.svelte'
import { logFeatureUsage } from '$lib/utils/featureUsage'
import { useOperatingWorkspace } from '$lib/components/operatingWorkspace.svelte'
const operatingWorkspace = useOperatingWorkspace()
const dispatch = createEventDispatcher()
// Subset of VariableDraftState (copilot/chat/global/workspaceItems.ts) — the same
// localStorage rows back both, so a field this shape omits is dropped on save.
type VariableState = {
path: string
variable: { value: string; is_secret: boolean; description: string }
labels: string[] | undefined
wsSpecific: boolean
value_valid_until?: string
}
// The deployed baseline every dirty check and save diff runs against: a field it
// leaves out reads as unchanged, so its edits are never sent.
function deployedVariableState(v: ListableVariable): VariableState {
return {
path: v.path,
variable: {
value: v.value ?? '',
is_secret: v.is_secret,
description: v.description ?? ''
},
labels: v.labels ?? undefined,
wsSpecific: v.ws_specific ?? false,
value_valid_until: v.value_valid_until ?? undefined
}
}
// The "current" workspace this editor defaults New/Edit actions to. Session
@@ -79,6 +99,7 @@
let initialStates: Record<string, VariableState> = $state({})
let existedInitially: Record<string, boolean> = $state({})
let extraPerms: Record<string, Record<string, boolean>> = $state({})
let oauthLinked: Record<string, boolean> = $state({})
let selected: string | undefined = $state(undefined)
let pathError = $state('')
const acting = useActingUser(() => selected)
@@ -224,16 +245,7 @@
decryptSecret: false,
getDraft: true
})
const deployedState: VariableState = {
path: v.path,
variable: {
value: v.value ?? '',
is_secret: v.is_secret,
description: v.description ?? ''
},
labels: v.labels ?? undefined,
wsSpecific: v.ws_specific ?? false
}
const deployedState = deployedVariableState(v)
// Everything below writes shared editor state, so first make sure it is still this
// variable's: the drawer stays closable while the read is out, and another variable
// opened meanwhile would otherwise get this one's baseline — and with it this one's
@@ -257,6 +269,7 @@
// displayed would leave those deciding on the version the user just replaced.
existedInitially[ws] = !(v as any).no_deployed
extraPerms[ws] = v.extra_perms ?? {}
oauthLinked[ws] = v.account != null
UserDraftDbSyncer.recordRemoteSync(query, (v as any).draft_saved_at)
UserDraft.seed(
'variable',
@@ -332,16 +345,7 @@
const savedDraftState = (v as any).draft as VariableState | undefined
// Deployed baseline as the dirty-check reference, so the banner
// compares draft-vs-deployed and fires immediately when a draft exists.
const deployedState: VariableState = {
path: v.path,
variable: {
value: v.value ?? '',
is_secret: v.is_secret,
description: v.description ?? ''
},
labels: v.labels ?? undefined,
wsSpecific: v.ws_specific ?? false
}
const deployedState = deployedVariableState(v)
// A refused save leaves this tab's own version parked. `.draft` is the version
// that refused it, so opening on that would quietly drop the edit the alert is
// about and leave "Keep mine" offering to keep the other one.
@@ -363,6 +367,7 @@
// CREATE, not update (update 404s).
existedInitially[ws] = !(v as any).no_deployed
extraPerms[ws] = v.extra_perms ?? {}
oauthLinked[ws] = v.account != null
})
})
})
@@ -376,6 +381,7 @@
initialStates = {}
existedInitially = {}
extraPerms = {}
oauthLinked = {}
pathError = ''
acting.forgetFailures()
}
@@ -435,6 +441,11 @@
async function save(): Promise<void> {
const dirty = dirtyWorkspaces
const savedPath = (curWs ? states[curWs]?.draft?.path : undefined) ?? editPath ?? ''
// Before the loop rewrites the baselines; once per save, not per workspace.
const expiryChanged = dirty.filter(
(ws) => initialStates[ws]?.value_valid_until !== states[ws].draft?.value_valid_until
)
const expiryNowSet = expiryChanged.some((ws) => !!states[ws].draft?.value_valid_until)
try {
for (const ws of dirty) {
const s = states[ws].draft!
@@ -452,6 +463,11 @@
ini.variable.description != s.variable.description
? s.variable.description
: undefined,
// Tri-state: absent leaves the stored date alone, `null` clears it.
value_valid_until:
ini.value_valid_until != s.value_valid_until
? (s.value_valid_until ?? null)
: undefined,
labels: s.labels,
ws_specific: s.wsSpecific
}
@@ -464,6 +480,7 @@
value: s.variable.value,
is_secret: s.variable.is_secret,
description: s.variable.description,
value_valid_until: s.value_valid_until,
labels: s.labels,
ws_specific: s.wsSpecific
}
@@ -480,6 +497,11 @@
// it shows up immediately instead of after the 60s TTL.
invalidateWorkspacePaths(ws)
}
if (expiryChanged.length > 0) {
logFeatureUsage('variable_expiration', 'expiry_edited', {
key: expiryNowSet ? 'set' : 'cleared'
})
}
sendUserToast(edit ? `Updated variable in ${dirty.length} workspace(s)` : `Created variable`)
dispatch('create')
onSaved?.(savedPath)
@@ -567,6 +589,7 @@
bind:variable={current.variable}
bind:labels={current.labels}
bind:wsSpecific={current.wsSpecific}
bind:valueValidUntil={current.value_valid_until}
{initialPath}
deployTo={deployTo.current}
can_write={can_write === true}
@@ -578,6 +601,7 @@
!!initialStates[selected]?.variable.is_secret}
workspace={selected}
actingUser={acting.in(selected) ?? null}
oauthLinked={!!selected && !!oauthLinked[selected]}
/>
{/key}
{/if}
frontend/src/lib/components/VariableForm.svelte
+58 -2
View File
@@ -3,7 +3,7 @@
import LabelsInput from './LabelsInput.svelte'
import Toggle from './Toggle.svelte'
import Alert from './common/alert/Alert.svelte'
import { Button } from './common'
import { Button, Section } from './common'
import Tooltip from './Tooltip.svelte'
import Label from './Label.svelte'
import ToggleButtonGroup from './common/toggleButton-v2/ToggleButtonGroup.svelte'
@@ -15,6 +15,7 @@
import { isOwner } from '$lib/utils'
import { isEncryptedDraftValue } from '$lib/encryptedDraft'
import EncryptedDraftField from './EncryptedDraftField.svelte'
import DateTimeInput from './DateTimeInput.svelte'
import { useOperatingWorkspace } from '$lib/components/operatingWorkspace.svelte'
const operatingWorkspace = useOperatingWorkspace()
@@ -32,6 +33,7 @@
variable: Variable
labels: string[] | undefined
wsSpecific: boolean
valueValidUntil: string | undefined
deployTo: string | undefined
can_write: boolean
edit: boolean
@@ -47,6 +49,8 @@
* refuses, rather than answering with the navigation user's rights in another
* workspace. */
actingUser: UserExt | null
/** Linked to an OAuth account, whose token Windmill refreshes on its own. */
oauthLinked?: boolean
}
let {
@@ -56,6 +60,7 @@
variable = $bindable(),
labels = $bindable(),
wsSpecific = $bindable(),
valueValidUntil = $bindable(),
deployTo,
can_write,
edit,
@@ -63,7 +68,8 @@
onSecretChange,
deployedSecret = false,
workspace = undefined,
actingUser
actingUser,
oauthLinked = false
}: Props = $props()
let ws = $derived(workspace ?? $operatingWorkspace)
@@ -74,6 +80,15 @@
)
)
// Enabling seeds a date: an empty DateTimeInput still shows 12:00, which reads as set.
const DEFAULT_EXPIRY_DAYS = 90
function defaultValueExpiry(): string {
const d = new Date()
d.setDate(d.getDate() + DEFAULT_EXPIRY_DAYS)
return d.toISOString()
}
// Loading the deployed secret overwrites the draft row this form shares with the AI
// chat, so every path that would trigger it has to be blocked while that row stages a
// value — otherwise the staged one is replaced and the next deploy carries the old one.
@@ -243,3 +258,44 @@ stretch the hit area over the whole row and the alert. The toggle carries its na
<textarea rows="4" use:autosize bind:value={variable.description} placeholder="Used for X"
></textarea>
</label>
{#if !oauthLinked}
<Section label="Advanced" small collapsable initiallyCollapsed={!valueValidUntil}>
<Label
label="Value expiration"
tooltip="When the value stored here stops working, such as the expiry date of an API key. Ahead of it (1 day by default), Windmill runs the workspace's variable expiration handler if one is set, so the value can be rotated. The variable itself is never deleted."
>
<Toggle
size="sm"
checked={Boolean(valueValidUntil)}
disabled={!can_write}
on:change={() => (valueValidUntil = valueValidUntil ? undefined : defaultValueExpiry())}
options={{ right: 'Set a date on which this value expires' }}
/>
{#if valueValidUntil}
<span class="text-xs font-semibold text-emphasis leading-none mt-2">Expires on</span>
<!-- Unbound: DateTimeInput writes `null` when emptied, which reads as an edit,
so empty states become `undefined`. Not `clearable`: the toggle means "no expiry". -->
<DateTimeInput
value={valueValidUntil}
useDropdown
disabled={!can_write}
on:change={(e) => (valueValidUntil = e.detail ?? undefined)}
/>
<span class="text-2xs font-normal text-hint">
{#if new Date(valueValidUntil).getTime() <= Date.now()}
This date has passed. Set a new one once the value is rotated.
{:else}
Runs the workspace's <a
href="/workspace_settings?tab=variable_expiration&workspace={ws}"
>variable expiration handler</a
>
ahead of this date, if the workspace has one{path.startsWith('u/')
? ' and does not skip u/ paths'
: ''}.
{/if}
</span>
{/if}
</Label>
</Section>
{/if}
frontend/src/lib/components/common/seconds/SecondsInput.svelte
+40 -30
View File
@@ -23,6 +23,10 @@
size?: ButtonType.UnifiedSize
clearable?: boolean
defaultValue?: number | undefined
/** For a value consumed at a coarser granularity, where seconds would have no effect. */
hideSeconds?: boolean
/** Hides the "= N seconds" caption below the input. */
hideTotal?: boolean
onfocus?: (e: FocusEvent) => void
}
@@ -33,6 +37,8 @@
size = 'md',
clearable = false,
defaultValue = undefined,
hideSeconds = false,
hideTotal = false,
onfocus
}: Props = $props()
@@ -193,30 +199,32 @@
><span class="invisible">mins</span><span>{min && min > 1 ? 'mins' : 'min'}</span></span
>
</div>
<div class="flex items-baseline">
<input
type="number"
class={segmentClass}
value={hasValue ? sec || 0 : ''}
placeholder="_ _"
min="0"
max="59"
{disabled}
oninput={(e) => handleInput(e, (v) => (sec = v), 59)}
onkeydown={(e) =>
handleKeydown(
e,
() => sec,
(v) => (sec = v),
59
)}
onfocus={handleFocus}
onblur={handleBlur}
/>
<span class="text-secondary text-2xs inline-grid [&>*]:col-start-1 [&>*]:row-start-1"
><span class="invisible">secs</span><span>{sec && sec > 1 ? 'secs' : 'sec'}</span></span
>
</div>
{#if !hideSeconds}
<div class="flex items-baseline">
<input
type="number"
class={segmentClass}
value={hasValue ? sec || 0 : ''}
placeholder="_ _"
min="0"
max="59"
{disabled}
oninput={(e) => handleInput(e, (v) => (sec = v), 59)}
onkeydown={(e) =>
handleKeydown(
e,
() => sec,
(v) => (sec = v),
59
)}
onfocus={handleFocus}
onblur={handleBlur}
/>
<span class="text-secondary text-2xs inline-grid [&>*]:col-start-1 [&>*]:row-start-1"
><span class="invisible">secs</span><span>{sec && sec > 1 ? 'secs' : 'sec'}</span></span
>
</div>
{/if}
</div>
{#if clearable && !disabled && seconds != null && seconds !== defaultValue}
<CloseButton
@@ -229,11 +237,13 @@
/>
{/if}
</div>
{#if seconds != null}
<span class="text-2xs text-hint">
= {seconds.toLocaleString()} second{seconds === 1 ? '' : 's'}
</span>
{:else}
<span class="text-2xs text-hint"> no time set </span>
{#if !hideTotal}
{#if seconds != null}
<span class="text-2xs text-hint">
= {seconds.toLocaleString()} second{seconds === 1 ? '' : 's'}
</span>
{:else}
<span class="text-2xs text-hint"> no time set </span>
{/if}
{/if}
</div>
frontend/src/lib/components/copilot/chat/global/core.test.ts
+34
View File
@@ -1370,6 +1370,40 @@ describe('global AI tools', () => {
expect(localStorageSnapshot()).not.toContain('new-secret-token')
})
// The draft has no null, so a cleared expiry must deploy as an explicit one.
it('clears a variable value expiry through to the deploy body', async () => {
vi.mocked(VariableService.existsVariable).mockResolvedValueOnce(true) // write probe
vi.mocked(VariableService.existsVariable).mockResolvedValueOnce(true) // deploy probe
vi.mocked(VariableService.getVariable).mockResolvedValueOnce({
path: 'f/secrets/api_key',
value: 'tok',
is_secret: false,
description: 'd',
value_valid_until: '2027-03-15T08:30:00Z'
} as any)
await callGlobalTool('write_variable', {
path: 'f/secrets/api_key',
value_valid_until: null
})
expect(
getBackendDraft<any>('variable', 'f/secrets/api_key', { workspace: WORKSPACE })
?.value_valid_until
).toBeUndefined()
await callGlobalTool('deploy_workspace_item', {
type: 'variable',
path: 'f/secrets/api_key'
})
expect(VariableService.updateVariable).toHaveBeenCalledWith(
expect.objectContaining({
requestBody: expect.objectContaining({ value_valid_until: null })
})
)
})
// '' means "no value staged", so there is nothing to create the secret with.
it('does not create a secret variable draft that stages no value', async () => {
seedBackendDraft(
frontend/src/lib/components/copilot/chat/global/core.ts
+27 -7
View File
@@ -747,6 +747,15 @@ const writeVariableSchema = variableRequestSchema.extend({
.string()
.optional()
.describe('The description of the variable. Omit it to keep the current description.'),
// Nullable, unlike the generated schema, so a cleared expiry differs from an omitted one.
value_valid_until: z
.string()
.datetime({ offset: true })
.nullable()
.optional()
.describe(
"When the value stored in this variable stops working. Triggers the workspace's variable expiration handler ahead of it (1 day by default, set on the handler); never causes the variable to be deleted. Omit it to keep the current expiry, or pass null to clear one — for instance once the value has been rotated."
),
override: draftOverrideField
})
@@ -2879,7 +2888,7 @@ function getResourceInstructions(): string {
- Global mode writes complete draft payloads only; it does not save, deploy, run, scaffold local files, or generate metadata.
- A resource draft is a workspace item: \`{ type: 'resource', path, summary?, value, isDraft }\`. \`value\` is a CreateResource body: \`{ path, value, description?, resource_type, labels? }\` where the inner \`value\` is the resource type's data shape.
- Reading a variable returns \`{ type: 'variable', path, summary?, isSecret, isDraft }\` — never its value, secret or not. \`isSecret\` tells you whether the value is encrypted.
- \`write_variable\` takes \`{ path, value?, is_secret?, description?, is_oauth?, expires_at?, labels? }\`. Creating a variable needs \`value\` and \`is_secret\`; editing one needs only the fields you are changing. Omitting \`value\` keeps the stored value, which is the only way to edit a secret variable — you cannot read its value, so passing any \`value\` you did not get from the user destroys it.
- \`write_variable\` takes \`{ path, value?, is_secret?, description?, is_oauth?, expires_at?, value_valid_until?, labels? }\`. Creating a variable needs \`value\` and \`is_secret\`; editing one needs only the fields you are changing. Omitting \`value\` keeps the stored value, which is the only way to edit a secret variable — you cannot read its value, so passing any \`value\` you did not get from the user destroys it.
- For secret fields in a resource value, create the variable with \`write_variable\` and \`is_secret: true\`, and deploy it before the resource (see "Secrets" in the reference below).
- Reference formats inside resource values: \`$var:g/all/name\` (global), \`$var:u/user/name\` (user), \`$var:f/folder/name\` (folder). Reference another resource with \`$res:path/to/resource\`. The same strings are also how a resource or variable is passed as a run argument (see the run-argument rule in the resource reference below); what they are never valid as is a variable's own value.
- Use \`search_resource_types\` to discover valid \`resource_type\` names and their JSON Schemas. Match the resource value to that schema.
@@ -5781,7 +5790,8 @@ function variableToDraftState(variable: ListableVariable): VariableDraftState {
// reviews. Undefined drops out of the JSON instead.
account: variable.account ?? undefined,
is_oauth: variable.is_oauth ?? undefined,
expires_at: variable.expires_at ?? undefined
expires_at: variable.expires_at ?? undefined,
value_valid_until: variable.value_valid_until ?? undefined
}
}
@@ -5851,7 +5861,11 @@ function createVariableToDraftState(
wsSpecific: args.ws_specific ?? base?.wsSpecific ?? false,
account: args.account ?? base?.account,
is_oauth: args.is_oauth ?? base?.is_oauth,
expires_at: args.expires_at ?? base?.expires_at
expires_at: args.expires_at ?? base?.expires_at,
// `null` clears and omitted keeps the base's. The draft stores a cleared expiry as
// `undefined`, since a stored draft goes through a JSON round-trip.
value_valid_until:
args.value_valid_until === null ? undefined : (args.value_valid_until ?? base?.value_valid_until)
}
}
@@ -5870,10 +5884,16 @@ function buildVariableCreateRequestBody(draftValue: CreateVariable): CreateVaria
// endpoint, and a draft stores '' when it stages no new value — so omitting `value` in
// that case is what leaves the stored one untouched. A staged value is sent as-is: the
// endpoint decrypts an `$encrypted:` marker and encrypts plaintext.
function buildVariableUpdateRequestBody(
draftValue: CreateVariable
): Omit<CreateVariable, 'value'> & { value?: string } {
const { value, ...rest } = structuredClone(draftValue)
function buildVariableUpdateRequestBody(draftValue: CreateVariable): Omit<
CreateVariable,
'value' | 'value_valid_until'
> & {
value?: string
value_valid_until?: string | null
} {
const { value, value_valid_until, ...others } = structuredClone(draftValue)
// The draft holds the full state, so an absent expiry was cleared: send `null`.
const rest = { ...others, value_valid_until: value_valid_until ?? null }
// A value the draft is never allowed to carry — a secret's, and an OAuth-managed one
// dropped by `variableToDraftState` — reaches here as '' whenever this edit staged
// none, so sending it would blank the stored value or wipe a rotating token.
frontend/src/lib/components/copilot/chat/global/diffSnapshot.test.ts
+14
View File
@@ -573,6 +573,20 @@ describe('fork mode', () => {
expect(entry?.valueMasked).toBe(true)
})
it('sees an expiry-only variable change', async () => {
mockComparison([comparisonDiff({ kind: 'variable', path: 'f/a/plain' })])
vi.mocked(VariableService.getVariable).mockImplementation(async ({ workspace }: any) => ({
path: 'f/a/plain',
is_secret: false,
value: 'same',
description: 'same',
value_valid_until: workspace === FORK ? '2027-09-09T10:00:00Z' : '2027-03-01T10:00:00Z'
})) as any
const entry = await readForkDiffEntryOne(FORK, PARENT, ['variable'], 'f/a/plain')
expect(entry?.status).toBe('modified')
expect(entry?.patch).toContain('2027-09-09')
})
it('sees flow schema-only changes the shared projection drops', async () => {
mockComparison([comparisonDiff({ kind: 'flow', path: 'f/a/fl' })])
vi.mocked(FlowService.getFlowByPath).mockImplementation(async ({ workspace }: any) => ({
frontend/src/lib/components/copilot/chat/global/diffSnapshot.ts
+1
View File
@@ -904,6 +904,7 @@ async function fetchForkSideValue(
value: {
description: variable.description,
is_secret: variable.is_secret,
value_valid_until: variable.value_valid_until,
value: VARIABLE_VALUE_PLACEHOLDER
},
valueMasked: true
frontend/src/lib/components/copilot/chat/global/userDraftAdapter.ts
+1
View File
@@ -239,6 +239,7 @@ function variableDraftToWorkspaceItem(path: string, draft: VariableDraftState):
account: draft.account,
is_oauth: draft.is_oauth,
expires_at: draft.expires_at,
value_valid_until: draft.value_valid_until,
labels: draft.labels,
ws_specific: draft.wsSpecific
},
frontend/src/lib/components/copilot/chat/global/workspaceItems.ts
+1
View File
@@ -108,6 +108,7 @@ export type VariableDraftState = {
account?: number
is_oauth?: boolean
expires_at?: string
value_valid_until?: string
}
export type WorkspaceItem = {
frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts
Generated
+2 -1
View File
@@ -490,7 +490,8 @@ export const variableRequestSchema = z.object({
"description": z.string().describe("The description of the variable"),
"account": z.number().int().describe("Ignored. A variable is linked to an OAuth account only when `createAccount` creates both.").optional(),
"is_oauth": z.boolean().describe("Whether the variable is an OAuth variable").optional(),
"expires_at": z.string().datetime({ offset: true }).describe("The expiration date of the variable").optional(),
"expires_at": z.string().datetime({ offset: true }).describe("When to garbage-collect the variable. Used for ephemeral secrets; the variable is deleted once this is reached. Not the value's expiry \u2014 see value_valid_until.\n").optional(),
"value_valid_until": z.string().datetime({ offset: true }).describe("When the value stored in this variable stops working. Triggers the workspace's variable expiration handler ahead of it, by the handler's `notify_before_secs` (1 day by default); never causes the variable to be deleted.\n").optional(),
"labels": z.array(z.string()).optional(),
"ws_specific": z.boolean().optional()
})
frontend/src/lib/components/handlerDraft.svelte.ts
+5 -1
View File
@@ -2,7 +2,7 @@ import type { ErrorHandler } from '$lib/gen'
import { WORKSPACE_SLACK_BOT_TOKEN_PATH } from '$lib/consts'
import hubPaths from '$lib/hubPaths.json'
export type HandlerMode = 'error' | 'recovery' | 'success'
export type HandlerMode = 'error' | 'recovery' | 'success' | 'variableExpiration'
type BuiltinDestination = 'slack' | 'teams' | 'email'
export type HandlerSlot = { path?: string; args: Record<string, any> }
@@ -32,6 +32,10 @@ const BUILTIN_SUFFIXES: Record<HandlerMode, Record<'slack' | 'teams', string>> =
success: {
slack: '/schedule-success-handler-slack',
teams: '/schedule-success-handler-teams'
},
variableExpiration: {
slack: '/variable-expiration-handler-slack',
teams: '/variable-expiration-handler-teams'
}
}
frontend/src/lib/components/variables/VariablesList.svelte
+17 -1
View File
@@ -50,6 +50,7 @@
Shield,
Trash,
Building,
CalendarClock,
DollarSign,
EyeOff,
Circle,
@@ -441,7 +442,7 @@
</tr>
</Head>
<tbody class="divide-y">
{#each filteredItems as { path, value, is_secret, description, extra_perms, canWrite, account, is_refreshed, is_expired, refresh_error, is_linked, labels, inherited_labels, ws_specific, draft_only, is_draft }}
{#each filteredItems as { path, value, is_secret, description, extra_perms, canWrite, account, is_refreshed, is_expired, refresh_error, is_linked, labels, inherited_labels, ws_specific, draft_only, is_draft, value_valid_until }}
{@const hasDraft =
getLocalDraftHint($operatingWorkspace, 'variable', path) ?? is_draft}
<Row>
@@ -522,6 +523,21 @@
{/snippet}
</Popover>
{/if}
{#if value_valid_until}
{@const expired = new Date(value_valid_until).getTime() <= Date.now()}
<Popover notClickable>
<CalendarClock
size={16}
class={expired ? 'text-red-600' : 'text-secondary'}
/>
{#snippet text()}
<div>
The value {expired ? 'expired' : 'expires'} on
{new Date(value_valid_until).toLocaleString()}.
</div>
{/snippet}
</Popover>
{/if}
{#if account}
<Popover notClickable>
<RefreshCw size={16} />
frontend/src/lib/hub.ts
+2
View File
@@ -86,6 +86,8 @@ type HubPaths = {
teamsErrorHandler: string
teamsRecoveryHandler: string
teamsSuccessHandler: string
slackVariableExpirationHandler: string
teamsVariableExpirationHandler: string
emailErrorHandler: string
cloneRepoToS3forGitRepoViewer: string
appReport: string
frontend/src/lib/hubPaths.json
+2
View File
@@ -8,6 +8,8 @@
"teamsErrorHandler": "hub/19742/workspace-or-schedule-error-handler-teams",
"teamsRecoveryHandler": "hub/11593/schedule-recovery-handler-teams",
"teamsSuccessHandler": "hub/11596/schedule-success-handler-teams",
"slackVariableExpirationHandler": "hub/30049/variable-expiration-handler-slack",
"teamsVariableExpirationHandler": "hub/30048/variable-expiration-handler-teams",
"slackReport": "hub/28792/slack",
"discordReport": "hub/9085/discord",
"smtpReport": "hub/28242/smtp",
frontend/src/lib/mcpEndpointTools.ts
+12 -1
View File
@@ -193,7 +193,12 @@ export const mcpEndpointTools: EndpointTool[] = [
},
"expires_at": {
"type": "string",
"description": "The expiration date of the variable",
"description": "When to garbage-collect the variable. Used for ephemeral secrets; the variable is deleted once this is reached. Not the value's expiry \u2014 see value_valid_until.\n",
"format": "date-time"
},
"value_valid_until": {
"type": "string",
"description": "When the value stored in this variable stops working. Triggers the workspace's variable expiration handler ahead of it, by the handler's `notify_before_secs` (1 day by default); never causes the variable to be deleted.\n",
"format": "date-time"
},
"labels": {
@@ -281,6 +286,12 @@ export const mcpEndpointTools: EndpointTool[] = [
"type": "string",
"description": "The new description of the variable"
},
"value_valid_until": {
"type": "string",
"nullable": true,
"description": "When the value stored in this variable stops working. Omit to leave it unchanged; send null to clear it. Changing it re-arms the workspace's variable expiration handler.\n",
"format": "date-time"
},
"labels": {
"type": "array",
"items": {
frontend/src/lib/utils_draft_deploy.test.ts
+58 -2
View File
@@ -6,7 +6,7 @@ vi.mock('$lib/gen', () => ({
FlowService: { getFlowByPath: vi.fn(), createFlow: vi.fn(), updateFlow: vi.fn() },
DraftService: { deleteDraft: vi.fn() },
AppService: {},
VariableService: {},
VariableService: { getVariable: vi.fn(), createVariable: vi.fn(), updateVariable: vi.fn() },
ResourceService: { getResource: vi.fn(), updateResource: vi.fn(), createResource: vi.fn() },
ScheduleService: {},
HttpTriggerService: {},
@@ -32,7 +32,7 @@ vi.mock('$lib/components/raw_apps/utils', () => ({ canonicalRawAppDiffValue: vi.
vi.mock('$lib/appDiffSides', () => ({ classicAppDraftParts: vi.fn() }))
vi.mock('$lib/utils_deployable', () => ({ TRIGGER_RUNTIME_IGNORE: [] }))
import { ScriptService, FlowService, ResourceService } from '$lib/gen'
import { ScriptService, FlowService, VariableService, ResourceService } from '$lib/gen'
import { UserDraftDbSyncer } from '$lib/userDraftDbSyncer.svelte'
// draftBaseIsStale compares a draft's base pointer against the deployed head
@@ -143,6 +143,62 @@ describe('deployDraft preserves on_behalf_of', () => {
})
})
// The deploy bodies are built field by field, so a forgotten field is dropped silently.
describe('deployDraft variable value_valid_until', () => {
beforeEach(() => vi.clearAllMocks())
const DATE = '2027-03-15T08:30:00.000Z'
function mockVariable(deployedExpiry: string | undefined, draftExpiry: string | undefined) {
vi.mocked(VariableService.getVariable).mockResolvedValueOnce({
path: 'u/admin/key',
value: 'v',
is_secret: false,
description: '',
value_valid_until: deployedExpiry,
draft: {
path: 'u/admin/key',
variable: { value: 'v', is_secret: false, description: '' },
value_valid_until: draftExpiry
}
} as any)
}
const updateBody = () =>
vi.mocked(VariableService.updateVariable).mock.calls[0][0].requestBody as any
it('sends a date the draft set on a variable that had none', async () => {
mockVariable(undefined, DATE)
await deployDraft('variable', 'u/admin/key', 'ws')
expect(updateBody().value_valid_until).toBe(DATE)
})
it('sends null when the draft cleared the deployed date', async () => {
mockVariable(DATE, undefined)
await deployDraft('variable', 'u/admin/key', 'ws')
expect(updateBody().value_valid_until).toBeNull()
})
it('omits it when the draft leaves the deployed date alone', async () => {
mockVariable(DATE, DATE)
await deployDraft('variable', 'u/admin/key', 'ws')
expect(updateBody().value_valid_until).toBeUndefined()
})
it('omits it when the draft spells the deployed moment differently', async () => {
mockVariable('2027-03-15T08:30:00Z', DATE)
await deployDraft('variable', 'u/admin/key', 'ws')
expect(updateBody().value_valid_until).toBeUndefined()
})
it('carries the date onto a draft-only variable', async () => {
mockVariable(undefined, DATE)
await deployDraft('variable', 'u/admin/key', 'ws', { draftOnly: true })
const body = vi.mocked(VariableService.createVariable).mock.calls[0][0].requestBody as any
expect(body.value_valid_until).toBe(DATE)
})
})
// The backend skips the dependency job when `lock` is set, so a draft that inherited the
// deployed lock would deploy new content against the old lockfile.
describe('deployDraft: script lock', () => {
frontend/src/lib/utils_draft_deploy.ts
+15 -4
View File
@@ -219,14 +219,15 @@ function stripScheduleTriggerRuntime(row: any): Record<string, unknown> {
function canonicalizeDraftDiffValue(kind: DraftKind, raw: any, isDraft: boolean): unknown {
if (!raw || typeof raw !== 'object') return raw ?? {}
if (kind === 'variable') {
// draft: { variable: { value, is_secret, description }, labels, wsSpecific }
// deployed row: { value, is_secret, description, labels, ws_specific }
// draft: { variable: { value, is_secret, description }, value_valid_until, labels, wsSpecific }
// deployed row: { value, is_secret, description, value_valid_until, labels, ws_specific }
const v = isDraft ? (raw.variable ?? {}) : raw
const is_secret = !!v.is_secret
return {
value: is_secret ? '<secret>' : (v.value ?? ''),
is_secret,
description: v.description ?? '',
value_valid_until: raw.value_valid_until ?? undefined,
labels: raw.labels ?? undefined,
ws_specific: (isDraft ? raw.wsSpecific : raw.ws_specific) ?? undefined
}
@@ -446,6 +447,10 @@ async function deployDraftTriggers(
await deployTriggers(triggers, workspace, isAdmin, writable<string[]>([]), path, isNew)
}
function timeOf(date: string | null | undefined): number | undefined {
return date ? new Date(date).getTime() : undefined
}
/**
* Promote a draft to deployed by replaying the editor's create/update call with
* the stored draft value. The matching draft row is deleted server-side by the
@@ -594,7 +599,7 @@ export async function deployDraft(
} else if (kind === 'variable') {
const { deployed, draft: d } = splitOverlay(await OVERLAY_GETTERS.variable!(workspace, path))
// VariableEditor's `VariableState` draft shape:
// { path, variable: { value, is_secret, description }, labels?, wsSpecific }
// { path, variable: { value, is_secret, description }, value_valid_until?, labels?, wsSpecific }
if (draftOnly) {
await VariableService.createVariable({
workspace,
@@ -603,6 +608,7 @@ export async function deployDraft(
value: d.variable?.value ?? '',
is_secret: !!d.variable?.is_secret,
description: d.variable?.description ?? '',
value_valid_until: d.value_valid_until,
labels: d.labels,
ws_specific: d.wsSpecific
}
@@ -617,12 +623,17 @@ export async function deployDraft(
value: d.variable?.value === '' ? undefined : d.variable?.value,
is_secret: d.variable?.is_secret,
description: d.variable?.description,
// `null` clears, absent keeps. Diffed against the deployed row by moment, since
// an absent date in the draft means "no expiry", not "unchanged".
value_valid_until:
timeOf(deployed.value_valid_until) !== timeOf(d.value_valid_until)
? (d.value_valid_until ?? null)
: undefined,
labels: d.labels,
ws_specific: d.wsSpecific
}
})
}
void deployed
} else if (kind === 'resource') {
const overlay = await OVERLAY_GETTERS.resource!(workspace, path)
// Adopt the row this promote is based on as the baseline for the delete below. Without one
frontend/src/routes/(root)/(logged)/workspace_settings/+page.svelte
+267 -1
View File
@@ -3,7 +3,7 @@
import { page } from '$app/stores'
import { isCloudHosted } from '$lib/cloud'
import CenteredPage from '$lib/components/CenteredPage.svelte'
import { Alert, Button, Section, Skeleton, Tab, Tabs } from '$lib/components/common'
import { Alert, Button, SecondsInput, Section, Skeleton, Tab, Tabs } from '$lib/components/common'
import ToggleButtonGroup from '$lib/components/common/toggleButton-v2/ToggleButtonGroup.svelte'
import ToggleButton from '$lib/components/common/toggleButton-v2/ToggleButton.svelte'
@@ -60,6 +60,7 @@
import PremiumInfo from '$lib/components/settings/PremiumInfo.svelte'
import Toggle from '$lib/components/Toggle.svelte'
import Popover from '$lib/components/Popover.svelte'
import SlackChannelAgents from '$lib/components/workspaceSettings/SlackChannelAgents.svelte'
import ChangeWorkspaceName from '$lib/components/settings/ChangeWorkspaceName.svelte'
@@ -222,6 +223,21 @@
let errorHandlerMutedOnCancel: boolean | undefined = $state(undefined)
let errorHandlerMutedOnUserPath: boolean | undefined = $state(undefined)
let successHandlerScriptPath: string | undefined = $state(undefined)
const noVariableExpirationHandler = handlerDraftFromStored('variableExpiration', undefined)
let variableExpirationDraft: HandlerDraft = $state(cloneHandlerDraft(noVariableExpirationHandler))
let variableExpirationMutedOnUserPath: boolean = $state(false)
// The sweep's default when the stored handler has no `notify_before_secs`.
const defaultNotifyBeforeSecs = 24 * 3600
let variableExpirationNotifyBeforeSecs: number = $state(defaultNotifyBeforeSecs)
let variableExpirationRemindEverySecs: number | undefined = $state(undefined)
// The sweep runs every 5 minutes, so shorter intervals count as 5 minutes.
let variableExpirationRuns = $derived(
variableExpirationRemindEverySecs
? Math.ceil(
variableExpirationNotifyBeforeSecs / Math.max(variableExpirationRemindEverySecs, 300)
)
: 1
)
let criticalAlertUIMuted: boolean | undefined = $state(undefined)
let initialCriticalAlertUIMuted: boolean | undefined = $state(undefined)
let publicAppRateLimitPerMinute: number | undefined = $state(undefined)
@@ -291,6 +307,14 @@
// Track initial success handler for unsaved changes detection
let initialSuccessHandlerScriptPath: string | undefined = $state(undefined)
// Track initial variable expiration handler for unsaved changes detection
let initialVariableExpirationDraft: HandlerDraft = $state(
cloneHandlerDraft(noVariableExpirationHandler)
)
let initialVariableExpirationMutedOnUserPath: boolean = $state(false)
let initialVariableExpirationNotifyBeforeSecs: number = $state(defaultNotifyBeforeSecs)
let initialVariableExpirationRemindEverySecs: number | undefined = $state(undefined)
let s3ResourceSettings: S3ResourceSettings = $state({
resourceType: 's3',
resourcePath: undefined,
@@ -348,6 +372,43 @@
)
})
// The mute flag is stored inside the handler setting, which saving with no handler
// clears, so the toggle is gated on having one.
let variableExpirationHasHandler = $derived(!!handlerDraftToStored(variableExpirationDraft).path)
// The stored handler is nested because `hasUnsavedChanges` drops a top-level `path`.
function variableExpirationSettingsValue(
draft: HandlerDraft,
mutedOnUserPath: boolean,
notifyBeforeSecs: number,
remindEverySecs: number | undefined
) {
return {
handler: handlerDraftToStored(draft),
mutedOnUserPath,
notifyBeforeSecs,
remindEverySecs: remindEverySecs || undefined
}
}
let hasVariableExpirationChanges = $derived.by(() => {
if (tab !== 'variable_expiration') return false
return hasUnsavedChanges(
variableExpirationSettingsValue(
initialVariableExpirationDraft,
initialVariableExpirationMutedOnUserPath,
initialVariableExpirationNotifyBeforeSecs,
initialVariableExpirationRemindEverySecs
),
variableExpirationSettingsValue(
variableExpirationDraft,
variableExpirationMutedOnUserPath,
variableExpirationNotifyBeforeSecs,
variableExpirationRemindEverySecs
)
)
})
// Derived state for checking unsaved changes in critical alert mute setting
let hasCriticalAlertMuteChanges = $derived.by(() => {
if (tab !== 'critical_alerts') return false
@@ -491,6 +552,7 @@
| 'dev_workspace'
| 'error_handler'
| 'success_handler'
| 'variable_expiration'
| 'critical_alerts'
| 'ai'
| 'windmill_data_tables'
@@ -809,6 +871,20 @@
| { path?: string; extra_args?: any }
| undefined
successHandlerScriptPath = (successHandler?.path ?? '').split('/').slice(1).join('/')
const variableExpirationHandler = settings.variable_expiration_handler
const [variableExpirationKind, ...variableExpirationPath] = (
variableExpirationHandler?.path ?? ''
).split('/')
variableExpirationDraft = handlerDraftFromStored(
'variableExpiration',
variableExpirationPath.join('/'),
variableExpirationKind === 'flow' ? 'flow' : 'script',
variableExpirationHandler?.extra_args
)
variableExpirationMutedOnUserPath = variableExpirationHandler?.muted_on_user_path ?? false
variableExpirationNotifyBeforeSecs =
variableExpirationHandler?.notify_before_secs ?? defaultNotifyBeforeSecs
variableExpirationRemindEverySecs = variableExpirationHandler?.remind_every_secs || undefined
workspaceDefaultAppPath = settings.default_app
initialWorkspaceDefaultAppPath = settings.default_app
@@ -860,6 +936,12 @@
// Store initial success handler state for unsaved changes detection
initialSuccessHandlerScriptPath = successHandlerScriptPath
// Store initial variable expiration handler state for unsaved changes detection
initialVariableExpirationDraft = cloneHandlerDraft(variableExpirationDraft)
initialVariableExpirationMutedOnUserPath = variableExpirationMutedOnUserPath
initialVariableExpirationNotifyBeforeSecs = variableExpirationNotifyBeforeSecs
initialVariableExpirationRemindEverySecs = variableExpirationRemindEverySecs
loadedSettings = true
}
@@ -1066,6 +1148,46 @@
initialErrorHandlerMutedOnUserPath = errorHandlerMutedOnUserPath
}
async function editVariableExpirationHandler() {
const handler = handlerDraftToStored(variableExpirationDraft)
await WorkspaceService.editVariableExpirationHandler({
workspace: $workspaceStore!,
requestBody: {
path: handler.path ? `${handler.kind}/${handler.path}` : undefined,
extra_args: handler.args,
muted_on_user_path: variableExpirationMutedOnUserPath,
notify_before_secs: variableExpirationNotifyBeforeSecs,
remind_every_secs: variableExpirationRemindEverySecs || undefined
}
})
sendUserToast(
handler.path
? `Variable expiration handler set to ${handler.path}`
: handlerDraftToStored(initialVariableExpirationDraft).path
? 'Variable expiration handler removed'
: 'Variable expiration settings saved'
)
// Rebuilt from what was stored, so Discard also drops the other tabs' unsaved edits.
initialVariableExpirationDraft = handlerDraftFromStored(
'variableExpiration',
handler.path,
handler.kind,
handler.args,
{ emptySelected: variableExpirationDraft.selected }
)
initialVariableExpirationMutedOnUserPath = variableExpirationMutedOnUserPath
initialVariableExpirationNotifyBeforeSecs = variableExpirationNotifyBeforeSecs
initialVariableExpirationRemindEverySecs = variableExpirationRemindEverySecs
}
function discardVariableExpirationChanges() {
variableExpirationDraft = cloneHandlerDraft(initialVariableExpirationDraft)
variableExpirationMutedOnUserPath = initialVariableExpirationMutedOnUserPath
variableExpirationNotifyBeforeSecs = initialVariableExpirationNotifyBeforeSecs
variableExpirationRemindEverySecs = initialVariableExpirationRemindEverySecs
}
async function editSuccessHandler() {
if (successHandlerScriptPath) {
await WorkspaceService.editSuccessHandler({
@@ -1330,6 +1452,21 @@
}
}
}
case 'variable_expiration':
return {
savedValue: variableExpirationSettingsValue(
initialVariableExpirationDraft,
initialVariableExpirationMutedOnUserPath,
initialVariableExpirationNotifyBeforeSecs,
initialVariableExpirationRemindEverySecs
),
modifiedValue: variableExpirationSettingsValue(
variableExpirationDraft,
variableExpirationMutedOnUserPath,
variableExpirationNotifyBeforeSecs,
variableExpirationRemindEverySecs
)
}
case 'critical_alerts':
return {
savedValue: { criticalAlertUIMuted: initialCriticalAlertUIMuted },
@@ -1374,6 +1511,9 @@
discardErrorHandlerSettingsChanges()
successHandlerScriptPath = initialSuccessHandlerScriptPath
break
case 'variable_expiration':
discardVariableExpirationChanges()
break
case 'critical_alerts':
criticalAlertUIMuted = initialCriticalAlertUIMuted
break
@@ -1481,6 +1621,13 @@
aiDescription: 'Error and success handler workspace settings',
isEE: true
},
{
id: 'variable_expiration',
label: 'Variable expiration',
aiId: 'workspace-settings-variable-expiration',
aiDescription:
'Runnable executed when a variable value is about to expire, so the value can be rotated'
},
{
id: 'critical_alerts',
label: 'Critical alerts',
@@ -2293,6 +2440,125 @@ export async function main(
disabled={!$enterpriseLicense}
/>
</Section>
{:else if tab == 'variable_expiration'}
<SettingsPageHeader
title="Variable expiration handler"
description="Run a script or flow ahead of the date the value of a workspace variable expires, so it can be rotated before it stops working."
link="https://www.windmill.dev/docs/core_concepts/variables_and_secrets"
/>
{#if !$enterpriseLicense && variableExpirationDraft.selected !== 'custom'}
<Alert type="warning" title="Slack and Teams handlers are an EE feature">
Connecting a workspace to Slack or Teams is a Windmill EE feature.
</Alert>
{/if}
<Section label="Handler">
<div class="flex flex-col gap-6">
<ErrorOrRecoveryHandler
noMargin
isEditable={true}
errorOrRecovery="variableExpiration"
customRequiresEE={false}
toggleText="Notify when a variable is about to expire"
bind:draft={variableExpirationDraft}
customScriptTemplate="/scripts/add?hub=hub%2F30050%2Fwindmill%2Fworkspace_variable_expiration_handler_template"
/>
<SettingCard
label="Notify before expiry"
description="How long before a variable's value expires the handler runs"
class="gap-2"
>
<SecondsInput
bind:seconds={variableExpirationNotifyBeforeSecs}
disabled={!variableExpirationHasHandler}
hideSeconds
hideTotal
/>
<div class="flex flex-col gap-1 mt-4">
<span class="text-xs font-semibold text-emphasis">Remind every</span>
<span class="text-xs text-secondary">
Run the handler again at this interval until the value expires. Expiries are
checked every 5 minutes, so shorter intervals act as 5 minutes. Leave empty
or at 0 for a single notice.
</span>
<SecondsInput
bind:seconds={variableExpirationRemindEverySecs}
disabled={!variableExpirationHasHandler}
clearable
hideSeconds
hideTotal
/>
{#if variableExpirationRemindEverySecs && variableExpirationRemindEverySecs >= variableExpirationNotifyBeforeSecs}
<span class="text-xs text-yellow-600 dark:text-yellow-400">
No reminder will be sent: the interval must be shorter than Notify before
expiry.
</span>
{:else if variableExpirationRuns >= 24}
<span class="text-xs text-yellow-600 dark:text-yellow-400">
This runs the handler up to {variableExpirationRuns} times before the value
expires.
</span>
{/if}
</div>
</SettingCard>
<SettingCard class="gap-2">
<!-- `w-fit`: the trigger is a flex item of the card, so it would otherwise
stretch the full width and anchor the popup away from the toggle. -->
<Popover
notClickable
disablePopup={variableExpirationHasHandler}
placement="bottom-start"
class="w-fit"
>
<Toggle
bind:checked={variableExpirationMutedOnUserPath}
disabled={!variableExpirationHasHandler}
options={{
right: 'Do not run the handler for variables in u/ paths'
}}
/>
{#snippet text()}
Select a handler first
{/snippet}
</Popover>
</SettingCard>
<Alert type="info" title="Runs as g/variable_expiration_handler">
{#if variableExpirationDraft.selected === 'custom'}
The handler is executed by the automatically created group
g/variable_expiration_handler. Rotating a variable means writing to it, so
grant that group write access to the variables it is meant to rotate. It
starts with none. A script that carries an on-behalf-of identity is the
exception: it runs as that user, with their permissions rather than the
group's.
{:else}
The handler is executed by the automatically created group
g/variable_expiration_handler. It only posts a message, so it needs no access
to your variables: it is given the path and the expiry date, never the value.
{#if variableExpirationDraft.selected === 'slack'}
Access to the Slack bot token is granted for you.
{/if}
{/if}
</Alert>
<span class="text-xs text-secondary">
If the handler fails to start, a critical alert is raised once and the handler
is tried again at every check. The alert is resolved as soon as it starts.
</span>
</div>
<SettingsFooter
class="mt-6"
inline
hasUnsavedChanges={hasVariableExpirationChanges}
onSave={editVariableExpirationHandler}
onDiscard={discardVariableExpirationChanges}
saveLabel="Save variable expiration handler"
disabled={!$enterpriseLicense && variableExpirationDraft.selected !== 'custom'}
/>
</Section>
{:else if tab == 'critical_alerts'}
<SettingsPageHeader
title="Workspace critical alerts"
rust-client/src/client.rs
+2
View File
@@ -247,6 +247,7 @@ impl Windmill {
account: None,
is_oauth: None,
expires_at: None,
value_valid_until: None,
labels: None,
ws_specific: None,
},
@@ -263,6 +264,7 @@ impl Windmill {
value: Some(value),
is_secret: None,
description: None,
value_valid_until: None,
labels: None,
ws_specific: None,
},