Files
windmill/cli/windmill-utils-internal/src/deploy.ts
T
7a29abc7ee feat: run a workspace handler when a variable's value expires (#10837)
* feat: run a workspace handler when a secret variable expires

Secret variables gain an optional `value_expires_at`, and a workspace can point
at a script or flow to run once that date passes, so a rotation is driven by the
platform instead of tracked out of band.

The secret never leaves the database. The sweep selects the path, description and
expiry and nothing else, so the handler is given a reference to rotate rather than
a value to handle. Job results are stored in cleartext and shown in Runs, which is
what makes that a boundary rather than a preference; a test asserts the job's args
never contain the value.

Each row is claimed by writing `expiration_dispatched_for` in the same transaction
as the push, so a failed push un-claims it and a lost race claims nothing. Moving
the date re-arms the row; re-saving the same date does not. A pass takes at most
100 rows under a transaction-scoped advisory lock, and orders so that one broken
workspace cannot starve the rest.

Dispatch runs as `g/variable_expiration_handler`, exempt from the CE group limit
and from non-premium exec metering the way the error and schedule handlers already
are. A handler that keeps failing backs off and raises a critical alert instead of
retrying every pass, and the back-off ends when the handler is repointed.

The expiry travels with the variable through workspace clone, tarball export and
workspace comparison. The dispatch marker deliberately does not, so a fork re-arms
on its own copy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(frontend): expose variable value expiration in the UI

The variable drawer takes an optional expiry on a secret value, and the variables
list badges it, turning red once the date has passed. A workspace settings tab
picks the handler, its extra args, and whether variables under `u/` are skipped.

That mute flag is stored inside the handler setting, so the toggle stays disabled
until a handler is picked: saving it on its own would clear the setting it lives
in and still report success.

The expiry is carried as a tri-state through the draft deploy and copilot paths —
absent leaves the remote value alone, an explicit null clears it — so deploying a
draft cannot silently drop a date someone set elsewhere.

The settings tab names the one case where the configured identity does not apply:
a handler script carrying `on_behalf_of` runs as that user rather than as
`g/variable_expiration_handler`, which matters because the same tab asks the admin
to grant that group write access to the variables it rotates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(cli): sync variable expiry and the workspace expiration handler

`wmill sync` carries `value_expires_at` on variables in both directions, and
pushes the workspace handler alongside the error and success handlers in the
grouped settings format. Dropping the key from a variable's yaml clears the
remote expiry, so the file stays canonical rather than accumulating fields that
can only ever be added.

The handler follows the same rule, with the caveat recorded where the type is
declared: an omitted handler only reaches the clear when some other setting
differs, because the push short-circuits on a comparison that walks the local
object's keys and so never sees a missing one. That gate is shared with the two
existing handlers and is left as it is here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(frontend): open the variable expiration handler template in a new tab

The "Create a handler" button hands its script template to /scripts/add through
the URL hash, which `load` reads from `window.location`. A client-side navigation
leaves the previous page's hash in place, so the payload was dropped and the
editor opened on the default template at a `u/` path. `target="_blank"` forces a
full page load, matching the success handler's "Create from template" button.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(frontend): make the variable expiry field state its own status

The expiry field rendered a DateTimeInput whose time defaults to 12:00 when it
holds no value, so a variable with no expiry looked like one already configured,
and "cleared" looked identical to "never set". Setting an expiry is optional, so
the field has to say which it is.

Rebuild it on the shape ScriptAdvancedSettings already uses for Timeout, Cache and
Concurrency limits: a toggle whose checked state is the stored value being present,
seeding a date on enable rather than revealing an empty picker, with the input
under its own label. Drop `clearable` with it, so the toggle is the single way to
mean "no expiry" instead of competing with an X.

Stop the surrounding copy promising a run that may not happen. A handler is only
dispatched if the workspace has one configured and does not skip u/ paths, so the
list tooltip now states the date alone, and the form hedges both conditions,
naming the u/ mute only on personal paths where it can apply.

Also anchor the mute toggle's tooltip: Popover's trigger is a span around a
block-level label, which stretched to the full width of its SettingCard and left
the popup floating beside the card rather than under the control.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(rust-client): initialize value_expires_at in the variable models

The generated `CreateVariable` / `EditVariable` structs gain the field from
openapi.yaml, and client.rs builds both with exhaustive literals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(frontend): drop extra_args when the variable expiration handler is cleared

Clearing the handler nulls the whole setting server-side, extra_args included,
but the tab kept its loaded copy. A handler picked afterwards without reloading
was persisted with the cleared one's arguments, which the tab does not display.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: back the variable expiration handler's built-in Slack and Teams destinations

Backfills the Slack bot token grant for g/variable_expiration_handler, mirroring
20231016142659 for g/error_handler: the slack_bot folder's extra_perms is written
once at Slack connect and never refreshed, and disconnecting leaves the folder
behind, so workspaces that already connected Slack cannot pick the grant up from
the creation-time list alone.

Adds run_variable_expiration_test_job so an admin can exercise the handler on
demand rather than waiting for a real expiry, which may be months away. It goes
through push_variable_expiration_handler so the test run proves what matters:
the handler resolves, and the group can read what it needs, under the same
identity and argument shape the sweep uses.

Also covers extra_args on the edit endpoint, which the settings tab now writes
for the built-ins' channel. The endpoint rebuilds the setting from the request,
so an omitted extra_args erases it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* feat(frontend): offer Slack and Teams as variable expiration handlers

Gives the variable expiration tab the same no-code destinations the workspace
error handler has, so "tell #ops before this credential expires" no longer
requires writing a script.

A sibling component rather than a fourth mode of ErrorOrRecoveryHandler: that
one is shared with the workspace error handler and the three schedule handlers,
and its stripped-args list, hub path suffixes and Email tab are all
error-shaped. This reuses the pieces that are already generic
(SlackConnectionStatus, TeamsConnectionStatus, ChannelSelector, ScriptPicker,
SchemaForm) and leaves those five call sites untouched.

extra_args stops being an API/CLI-only escape hatch now that the built-ins write
their channel into it, so it joins the unsaved-changes comparison and the
discard path, normalised through the existing helper that drops the two
auto-managed keys.

The two hub ids are placeholders until the scripts are pushed to the hub.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* fix(frontend): scope the handler identity notice to the chosen destination

The notice told every admin to grant g/variable_expiration_handler write access
to the variables it rotates. That is right for a custom handler and wrong for
the built-ins, which only post a message and are granted the Slack bot token by
migration. Following it for a Slack handler would hand a group write access to
the workspace's secrets for no reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* feat(frontend): flag the Slack and Teams handlers as EE

Connecting a workspace to Slack or Teams is EE-only (oauth2_oss.rs returns
"Slack only available on enterprise"), so on CE those two tabs can never be
completed. Says so up front, the way the error/success handler tab does,
instead of leaving the user at a "not connected" badge with no working path.

Only on those two tabs: a custom script or flow handler is CE.

Drops the "Connect Slack first" alert, now that the banner and the connection
badge above it already say it twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* chore(backend): regenerate the sqlx cache after the merge

The merged workspace_settings column list in change_workspace_id is a new
query text, so it needs a new cached entry. Drops the two pre-merge entries
it supersedes; both had been left with conflict markers by the merge, since
rename/rename markers are eight characters and escape the usual scan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* fix(cli): narrow the expiry-clear check to a variable that exists remotely

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Do7wJTnL6MG6ngFpCGW4DG

* feat(frontend): point the built-in handlers at their published hub scripts

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(backend): regenerate the sqlx cache after the merge

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): disable the Slack and Teams handler controls without a license

Matches the workspace error handler, which greys out its built-in destinations
and hides the test button on an unlicensed instance. The custom destination stays
editable: unlike the error handler, the variable expiration handler itself is not
an EE feature.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: record which destination a variable expiration handler uses

handler_configured now counts slack, teams, custom or cleared instead of
set or cleared, so usage shows whether the built-in destinations are picked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(frontend): let the variable expiration handler template run on any worker

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: guard the variable expiration test endpoint and correct its help text

* fix: reuse the error handler's slack arg and built-in path helpers

* docs: trim variable expiration sweep and backfill comments

* feat: put the variable value expiration under an advanced section

* fix: restrict the expiration test job to built-ins and keep expiry on conflict reload

* fix: link the expiry hint to the variable's workspace and document built-in test paths

* fix: give the expiration sweep its own lock id and reuse double_option

* fix: pin the variable expiration presets so the hub-paths test accepts them

* feat: let the variable expiration handler set how far ahead it runs

* fix: give the notify before expiry setting its own card

* feat: remind the variable expiration handler at a set interval

* fix: tighten the gap between the Slack channel field and its test button

* feat: refuse a value expiry on variables linked to an OAuth account

* refactor: reuse the shared handler form for variable expiration

* refactor: sweep expiring variables round-robin and check the handler exists

* chore: trim comments to the constraints they record

* fix: compare expiry dates by moment when deploying a variable draft

* refactor: rename value_expires_at to value_valid_until

* fix: neutralize variable fields in expiration handler args

* fix: reserve the variable_expiration_handler group name for admins

* fix: send first expiry notices before reminders

* test: pin that a handler failing to start leaves its rows unclaimed

* chore: say any hub handler path skips the existence check

* fix: empty a pre-existing handler group before granting it the Slack token

* docs: correct the registered feature usage counts

* fix: let a workspace whose handler failed to start sit out for an hour

* fix: send first notices before reminders across workspaces

* fix: end a failed handler's wait as soon as it is redeployed

* fix: show an expiry-only variable change in the AI fork diff

* chore: regenerate workspace tool schemas

* test: keep one variable expiration guard per behavior

* fix: use the renamed hub expiration handlers and template

* feat: carry the variable expiry in windmill-utils-internal 1.8.4

* fix: keep the variable expiry through MCP drafts

* chore: bump windmill-utils-internal to 1.8.4

* fix: keep the variable expiry through mcp deploys

* fix: disable saving a slack or teams expiry handler without ee

* fix: skip the failure mark when the expiry handler was saved mid-sweep

* fix: count expiry telemetry only where the expiry changed

* test: assert the exact variable expiry in export and cli push tests

* fix: stamp an expiry handler failure with the pass's read time

* fix: wait for a pending handler lock instead of marking it failed

* test: isolate expiry handler tests from the deployed-script cache

* fix: retry a failing expiry handler every pass instead of waiting an hour

* chore: update ee-repo-ref to b6e23d229966cf0019c0a1926a254fda658d8def

This commit updates the EE repository reference after PR #744 was merged in windmill-ee-private.

Previous ee-repo-ref: c52c06d145b13a6fb04459e85ca5818bf3c1bff4

New ee-repo-ref: b6e23d229966cf0019c0a1926a254fda658d8def

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
2026-10-23 18:12:55 +02:00

1066 lines
36 KiB
TypeScript

/**
* Shared deploy logic for workspace fork/merge operations.
*
* Used by both the CLI (`wmill workspace merge`) and the frontend
* (`CompareWorkspaces.svelte`). The caller provides a {@link DeployProvider}
* that wraps the concrete API client (class-based for the frontend,
* standalone functions for the CLI).
*/
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
export type DeployKind =
| "script"
| "flow"
| "app"
| "raw_app"
| "resource"
| "variable"
| "resource_type"
| "folder"
| "schedule"
| "datatable_migration"
| "http_trigger"
| "websocket_trigger"
| "kafka_trigger"
| "nats_trigger"
| "postgres_trigger"
| "mqtt_trigger"
| "sqs_trigger"
| "gcp_trigger"
| "azure_trigger"
| "email_trigger";
export const TRIGGER_KINDS = [
"http_trigger",
"websocket_trigger",
"kafka_trigger",
"nats_trigger",
"postgres_trigger",
"mqtt_trigger",
"sqs_trigger",
"gcp_trigger",
"azure_trigger",
"email_trigger",
] as const satisfies readonly DeployKind[];
export type TriggerDeployKind = (typeof TRIGGER_KINDS)[number];
export function isTriggerKind(kind: string): kind is TriggerDeployKind {
return (TRIGGER_KINDS as readonly string[]).includes(kind);
}
/** True for any kind that deploys via a trigger or schedule API. */
export function isTriggerOrScheduleKind(
kind: string
): kind is TriggerDeployKind | "schedule" {
return kind === "schedule" || isTriggerKind(kind);
}
export interface DeployResult {
success: boolean;
error?: string;
}
/**
* Abstraction over the generated API client.
* Both the frontend (class-based services) and the CLI (standalone functions)
* can satisfy this interface with a thin adapter.
*/
export interface DeployProvider {
// Existence checks
existsFlowByPath(p: { workspace: string; path: string }): Promise<boolean>;
existsScriptByPath(p: { workspace: string; path: string }): Promise<boolean>;
existsApp(p: { workspace: string; path: string }): Promise<boolean>;
existsVariable(p: { workspace: string; path: string }): Promise<boolean>;
existsResource(p: { workspace: string; path: string }): Promise<boolean>;
existsResourceType(p: { workspace: string; path: string }): Promise<boolean>;
existsFolder(p: { workspace: string; name: string }): Promise<boolean>;
// Flows
getFlowByPath(p: { workspace: string; path: string }): Promise<any>;
createFlow(p: { workspace: string; requestBody: any }): Promise<any>;
updateFlow(p: {
workspace: string;
path: string;
requestBody: any;
}): Promise<any>;
archiveFlowByPath(p: {
workspace: string;
path: string;
requestBody: any;
}): Promise<any>;
// Scripts
getScriptByPath(p: { workspace: string; path: string }): Promise<any>;
createScript(p: { workspace: string; requestBody: any }): Promise<any>;
archiveScriptByPath(p: {
workspace: string;
path: string;
}): Promise<any>;
// Apps
getAppByPath(p: { workspace: string; path: string }): Promise<any>;
createApp(p: { workspace: string; requestBody: any }): Promise<any>;
updateApp(p: {
workspace: string;
path: string;
requestBody: any;
}): Promise<any>;
createAppRaw(p: { workspace: string; formData: any }): Promise<any>;
updateAppRaw(p: {
workspace: string;
path: string;
formData: any;
}): Promise<any>;
getPublicSecretOfLatestVersionOfApp(p: {
workspace: string;
path: string;
}): Promise<any>;
getRawAppData(p: {
secretWithExtension: string;
workspace: string;
}): Promise<any>;
deleteApp(p: { workspace: string; path: string }): Promise<any>;
// Variables
getVariable(p: {
workspace: string;
path: string;
decryptSecret?: boolean;
}): Promise<any>;
createVariable(p: { workspace: string; requestBody: any }): Promise<any>;
updateVariable(p: {
workspace: string;
path: string;
requestBody: any;
alreadyEncrypted?: boolean;
}): Promise<any>;
deleteVariable(p: { workspace: string; path: string }): Promise<any>;
// Resources
getResource(p: { workspace: string; path: string }): Promise<any>;
createResource(p: { workspace: string; requestBody: any }): Promise<any>;
updateResource(p: {
workspace: string;
path: string;
requestBody: any;
}): Promise<any>;
deleteResource(p: { workspace: string; path: string }): Promise<any>;
// Resource types
getResourceType(p: { workspace: string; path: string }): Promise<any>;
createResourceType(p: { workspace: string; requestBody: any }): Promise<any>;
updateResourceType(p: {
workspace: string;
path: string;
requestBody: any;
}): Promise<any>;
deleteResourceType(p: { workspace: string; path: string }): Promise<any>;
// Folders
getFolder(p: { workspace: string; name: string }): Promise<any>;
createFolder(p: { workspace: string; requestBody: any }): Promise<any>;
updateFolder(p: {
workspace: string;
name: string;
requestBody: any;
}): Promise<any>;
deleteFolder(p: { workspace: string; name: string }): Promise<any>;
// Datatable migrations. In the diff, an item's `path` is
// `<datatable>/<timestamp>_<name>` (see `parseDatatableMigrationDeployPath`).
listDatatableMigrations(p: { workspace: string }): Promise<any>;
upsertDatatableMigration(p: {
workspace: string;
datatableName: string;
requestBody: {
timestamp: number;
name: string;
code_up: string;
code_down?: string;
};
}): Promise<any>;
deleteDatatableMigration(p: {
workspace: string;
datatableName: string;
timestamp: number;
}): Promise<any>;
// Triggers — per-kind dispatch is delegated to the implementor so the shared
// module doesn't need to know about each of the 9 trigger services.
existsTriggerByKind(
kind: TriggerDeployKind,
p: { workspace: string; path: string }
): Promise<boolean>;
/**
* Get a trigger as it should be sent to create/update. Includes any kind-specific
* transforms the implementor wants to apply before deploy (e.g. wiping GCP
* subscription_id so the target workspace can create its own subscription).
*
* Operational-state handling (strip `mode`/`enabled` on update, pass through
* on create) is applied by `deployItem` at the dispatch layer — implementors
* should return the full row including `mode`/`enabled` and let the shared
* dispatch decide.
*/
getTriggerForDeploy(
kind: TriggerDeployKind,
p: { workspace: string; path: string; onBehalfOf?: string }
): Promise<any>;
createTriggerByKind(
kind: TriggerDeployKind,
p: { workspace: string; requestBody: any }
): Promise<any>;
updateTriggerByKind(
kind: TriggerDeployKind,
p: { workspace: string; path: string; requestBody: any }
): Promise<any>;
deleteTriggerByKind(
kind: TriggerDeployKind,
p: { workspace: string; path: string }
): Promise<any>;
/** Stripped trigger row used for the diff drawer (config fields only). */
getTriggerValue(
kind: TriggerDeployKind,
p: { workspace: string; path: string }
): Promise<unknown>;
/** Returns the trigger's `permissioned_as` for `--preserve-on-behalf-of`. */
getTriggerPermissionedAs(
kind: TriggerDeployKind,
p: { workspace: string; path: string }
): Promise<string | undefined>;
// Schedules
existsSchedule(p: { workspace: string; path: string }): Promise<boolean>;
getSchedule(p: { workspace: string; path: string }): Promise<any>;
createSchedule(p: { workspace: string; requestBody: any }): Promise<any>;
updateSchedule(p: {
workspace: string;
path: string;
requestBody: any;
}): Promise<any>;
deleteSchedule(p: { workspace: string; path: string }): Promise<any>;
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/** Folder diff paths carry the `f/` prefix; folder API endpoints expect just the name. */
export function folderName(path: string): string {
return path.replace(/^f\//, "");
}
/**
* Strip operational state (`mode`, `enabled`) from a trigger/schedule payload
* when deploying as an update; pass through unchanged on create.
*
* On update the strip lets the backend preserve the target row's existing
* state via `is_mode_unspecified()` (triggers) or via `EditSchedule` lacking
* `enabled` (schedules). On create there's no target state to preserve, so
* the source's flag is sent through — fork-only items land with whatever
* state the fork creator chose, and missing flags fall back to the backend
* default of enabled (`BaseTriggerData::mode()` / schedule insert default).
*/
export function stripOperationalStateOnUpdate<T extends Record<string, any>>(
payload: T,
alreadyExists: boolean
): T {
if (!alreadyExists) return payload;
const { mode: _mode, enabled: _enabled, ...rest } = payload;
return rest as T;
}
function getSubModules(flowModule: any): any[][] {
const type = flowModule?.value?.type;
if (type === "forloopflow" || type === "whileloopflow") {
return [flowModule.value.modules ?? []];
} else if (type === "branchall") {
return (flowModule.value.branches ?? []).map(
(branch: any) => branch.modules ?? []
);
} else if (type === "branchone") {
return [
...(flowModule.value.branches ?? []).map((b: any) => b.modules ?? []),
flowModule.value.default ?? [],
];
} else if (type === "aiagent") {
if (flowModule.value.tools) {
return [
flowModule.value.tools
.filter(
(t: any) =>
t.value?.type === "script" || t.value?.type === "flow"
)
.map((t: any) => ({
id: t.id,
value: t.value,
summary: t.summary,
})),
];
}
}
return [];
}
function getAllSubmodules(flowModule: any): any[] {
return getSubModules(flowModule)
.map((modules) => modules.flatMap((m: any) => [m, ...getAllSubmodules(m)]))
.flat();
}
/** Recursively collect all modules from a flow definition, including the failure module. */
export function getAllModules(
flowModules: any[],
failureModule?: any
): any[] {
return [
...flowModules,
...flowModules.flatMap((x) => getAllSubmodules(x)),
...(failureModule ? [failureModule] : []),
];
}
function toError(e: unknown): string {
const err = e as { body?: string; message?: string };
return err.body || err.message || String(e);
}
// A datatable-migration diff item's path is `<datatable>/<timestamp>_<name>`
// (mirrors the backend, e.g. `mydt/20260101000001_create_users`).
export function parseDatatableMigrationDeployPath(path: string): {
datatable: string;
timestamp: number;
name: string;
} {
const slash = path.indexOf("/");
const underscore = slash >= 0 ? path.indexOf("_", slash + 1) : -1;
if (slash < 0 || underscore < 0) {
throw new Error(`Invalid datatable migration path: ${path}`);
}
const datatable = path.slice(0, slash);
const timestamp = Number(path.slice(slash + 1, underscore));
const name = path.slice(underscore + 1);
if (!datatable || !Number.isFinite(timestamp) || !name) {
throw new Error(`Invalid datatable migration path: ${path}`);
}
return { datatable, timestamp, name };
}
// The backend rejects `upsertDatatableMigration` when the target data table
// hasn't opted in to migrations. Turn that opaque 400 into an explicit,
// deploy-context message (falls back to the original error otherwise).
function asMigrationsDisabledError(e: unknown, datatable: string): unknown {
const msg = (e as { body?: string; message?: string })?.body ?? ''
if (typeof msg === "string" && /migrations are not enabled/i.test(msg)) {
return new Error(
`Data table '${datatable}' has not opted in to migrations on the target workspace; enable migrations for it there before deploying its migrations.`
);
}
return e;
}
// ---------------------------------------------------------------------------
// checkItemExists
// ---------------------------------------------------------------------------
export async function checkItemExists(
provider: DeployProvider,
kind: DeployKind,
path: string,
workspace: string
): Promise<boolean> {
if (kind === "flow") {
return provider.existsFlowByPath({ workspace, path });
} else if (kind === "script") {
return provider.existsScriptByPath({ workspace, path });
} else if (kind === "app" || kind === "raw_app") {
return provider.existsApp({ workspace, path });
} else if (kind === "variable") {
return provider.existsVariable({ workspace, path });
} else if (kind === "resource") {
return provider.existsResource({ workspace, path });
} else if (kind === "resource_type") {
return provider.existsResourceType({ workspace, path });
} else if (kind === "folder") {
return provider.existsFolder({ workspace, name: folderName(path) });
} else if (kind === "schedule") {
return provider.existsSchedule({ workspace, path });
} else if (kind === "datatable_migration") {
const { datatable, timestamp } = parseDatatableMigrationDeployPath(path);
const migrations = await provider.listDatatableMigrations({ workspace });
return (migrations as { datatable: string; timestamp: number }[]).some(
(m) => m.datatable === datatable && m.timestamp === timestamp
);
} else if (isTriggerKind(kind)) {
return provider.existsTriggerByKind(kind, { workspace, path });
}
throw new Error(`Unknown kind: ${kind}`);
}
/**
* Fetch one part of a raw app's compiled bundle.
*
* A bundle with no styles may have no `css` blob stored at all, which older
* backends serve as a 404. Absent means empty, not broken — letting that reject
* makes such an app permanently un-deployable. Only `.css` and only a 404 are
* forgiven: swallowing auth/network failures would silently deploy the app with
* its styles stripped, and a missing `.js` is a genuinely broken bundle.
*/
export async function getRawAppBundlePart(
provider: DeployProvider,
secret: string,
ext: "js" | "css",
workspace: string
): Promise<any> {
try {
return await provider.getRawAppData({
secretWithExtension: `${secret}.${ext}`,
workspace,
});
} catch (e: any) {
if (ext === "css" && e?.status === 404) return "";
throw e;
}
}
// ---------------------------------------------------------------------------
// deployItem
// ---------------------------------------------------------------------------
export async function deployItem(
provider: DeployProvider,
kind: DeployKind,
path: string,
workspaceFrom: string,
workspaceTo: string,
onBehalfOf?: string
): Promise<DeployResult> {
const preserveOnBehalfOf = onBehalfOf !== undefined;
try {
const alreadyExists = await checkItemExists(
provider,
kind,
path,
workspaceTo
);
if (kind === "flow") {
const flow = await provider.getFlowByPath({
workspace: workspaceFrom,
path,
});
// Clear inline script hashes so the target workspace resolves by path
getAllModules(
flow.value?.modules ?? [],
flow.value?.failure_module
).forEach((x: any) => {
if (x.value?.type === "script" && x.value.hash != undefined) {
x.value.hash = undefined;
}
});
if (alreadyExists) {
await provider.updateFlow({
workspace: workspaceTo,
path,
requestBody: {
...flow,
// Sent even when unset: the backend keeps a restriction an omitted field leaves out.
job_token_scopes: flow.job_token_scopes ?? null,
preserve_on_behalf_of: preserveOnBehalfOf,
on_behalf_of_email: onBehalfOf,
// Usernames are per-workspace, so the source's principal names nobody in
// the target (or names a different person). Clearing it lets the backend
// derive the target's own principal from the email above.
on_behalf_of: undefined,
},
});
} else {
await provider.createFlow({
workspace: workspaceTo,
requestBody: {
...flow,
// Sent even when unset: the backend keeps a restriction an omitted field leaves out.
job_token_scopes: flow.job_token_scopes ?? null,
preserve_on_behalf_of: preserveOnBehalfOf,
on_behalf_of_email: onBehalfOf,
// Usernames are per-workspace, so the source's principal names nobody in
// the target (or names a different person). Clearing it lets the backend
// derive the target's own principal from the email above.
on_behalf_of: undefined,
},
});
}
} else if (kind === "script") {
const script = await provider.getScriptByPath({
workspace: workspaceFrom,
path,
});
let parentHash: string | undefined;
if (alreadyExists) {
const existing = await provider.getScriptByPath({
workspace: workspaceTo,
path,
});
parentHash = existing.hash;
}
await provider.createScript({
workspace: workspaceTo,
requestBody: {
...script,
lock: script.lock,
job_token_scopes: script.job_token_scopes ?? null,
parent_hash: parentHash,
preserve_on_behalf_of: preserveOnBehalfOf,
on_behalf_of_email: onBehalfOf,
// See the flow branch: a source-workspace principal is never valid here.
on_behalf_of: undefined,
},
});
} else if (kind === "app" || kind === "raw_app") {
const rawApp = await provider.getAppByPath({
workspace: workspaceFrom,
path,
});
// See the flow branch: a source-workspace principal is never valid here, and the
// policy carries the app's in `on_behalf_of`. Clearing it lets the backend derive
// the target's own from the address. A group travels as its synthetic
// `group-*@windmill.dev` address, which an admin-created account holding it would
// win: known and accepted, see `users::permissioned_as_from_email` in the backend.
const app = {
...rawApp,
policy: {
...rawApp.policy,
on_behalf_of: undefined,
on_behalf_of_email: onBehalfOf,
},
};
if (alreadyExists) {
if (app.raw_app) {
const secret = await provider.getPublicSecretOfLatestVersionOfApp({
workspace: workspaceFrom,
path: app.path,
});
const js = await getRawAppBundlePart(
provider,
secret,
"js",
workspaceFrom
);
const css = await getRawAppBundlePart(
provider,
secret,
"css",
workspaceFrom
);
await provider.updateAppRaw({
workspace: workspaceTo,
path,
formData: {
app: { ...app, preserve_on_behalf_of: preserveOnBehalfOf },
css,
js,
},
});
} else {
await provider.updateApp({
workspace: workspaceTo,
path,
requestBody: {
...app,
preserve_on_behalf_of: preserveOnBehalfOf,
},
});
}
} else {
if (app.raw_app) {
const secret = await provider.getPublicSecretOfLatestVersionOfApp({
workspace: workspaceFrom,
path: app.path,
});
const js = await getRawAppBundlePart(
provider,
secret,
"js",
workspaceFrom
);
const css = await getRawAppBundlePart(
provider,
secret,
"css",
workspaceFrom
);
await provider.createAppRaw({
workspace: workspaceTo,
formData: {
app: { ...app, preserve_on_behalf_of: preserveOnBehalfOf },
css,
js,
},
});
} else {
await provider.createApp({
workspace: workspaceTo,
requestBody: {
...app,
preserve_on_behalf_of: preserveOnBehalfOf,
},
});
}
}
} else if (kind === "variable") {
const variable = await provider.getVariable({
workspace: workspaceFrom,
path,
decryptSecret: true,
});
if (alreadyExists) {
await provider.updateVariable({
workspace: workspaceTo,
path,
requestBody: {
path,
value: variable.value ?? "",
is_secret: variable.is_secret,
description: variable.description ?? "",
labels: variable.labels,
// `null` clears; absent would keep the target's stored date.
value_valid_until: variable.value_valid_until ?? null,
},
alreadyEncrypted: false,
});
} else {
await provider.createVariable({
workspace: workspaceTo,
requestBody: {
path,
value: variable.value ?? "",
is_secret: variable.is_secret,
description: variable.description ?? "",
labels: variable.labels,
value_valid_until: variable.value_valid_until ?? null,
},
});
}
} else if (kind === "resource") {
const resource = await provider.getResource({
workspace: workspaceFrom,
path,
});
let value = resource.value ?? "";
// An agent's value carries the principal it runs as, as a trigger's row does: the source's
// names nobody in the target, so it is replaced by the chosen one or dropped, leaving the
// backend to make it the deployer's.
const agent = resource.resource_type === "ai_agent" && typeof value === "object";
if (agent) {
const { on_behalf_of: _, ...rest } = value;
value = onBehalfOf ? { ...rest, on_behalf_of: onBehalfOf } : rest;
}
const agentPreserve = agent ? { preserve_on_behalf_of: preserveOnBehalfOf } : {};
if (alreadyExists) {
await provider.updateResource({
workspace: workspaceTo,
path,
requestBody: {
path,
value,
description: resource.description ?? "",
labels: resource.labels,
...agentPreserve,
},
});
} else {
await provider.createResource({
workspace: workspaceTo,
requestBody: {
path,
value,
resource_type: resource.resource_type,
description: resource.description ?? "",
labels: resource.labels,
...agentPreserve,
},
});
}
} else if (kind === "resource_type") {
const rt = await provider.getResourceType({
workspace: workspaceFrom,
path,
});
if (alreadyExists) {
await provider.updateResourceType({
workspace: workspaceTo,
path,
requestBody: {
schema: rt.schema,
description: rt.description ?? "",
},
});
} else {
await provider.createResourceType({
workspace: workspaceTo,
requestBody: {
name: rt.name,
schema: rt.schema,
description: rt.description ?? "",
},
});
}
} else if (kind === "folder") {
const name = folderName(path);
const folder = await provider.getFolder({
workspace: workspaceFrom,
name,
});
if (alreadyExists) {
await provider.updateFolder({
workspace: workspaceTo,
name,
requestBody: {
owners: folder.owners,
extra_perms: folder.extra_perms,
summary: folder.summary ?? undefined,
// A folder stores cleared labels as NULL and reads back without the field, so an
// absent value has to be sent as [] or the target keeps labels the source removed.
labels: folder.labels ?? [],
},
});
} else {
await provider.createFolder({
workspace: workspaceTo,
requestBody: {
name,
owners: folder.owners,
extra_perms: folder.extra_perms,
summary: folder.summary ?? undefined,
labels: folder.labels,
},
});
}
} else if (kind === "schedule") {
const schedule = await provider.getSchedule({
workspace: workspaceFrom,
path,
});
// Operational-state handling — same shape as triggers below:
// - Update: strip `enabled` so the target's existing state is preserved
// (`EditSchedule` lacks `enabled` server-side, so this is also
// enforced by the type, but stripping keeps the intent explicit).
// - Create: pass `enabled` through so a fork-only schedule lands with
// the state the fork creator chose.
const baseBody = stripOperationalStateOnUpdate(schedule, alreadyExists);
const requestBody = {
...baseBody,
permissioned_as: onBehalfOf,
preserve_permissioned_as: preserveOnBehalfOf,
};
if (alreadyExists) {
await provider.updateSchedule({
workspace: workspaceTo,
path,
requestBody,
});
} else {
await provider.createSchedule({
workspace: workspaceTo,
requestBody,
});
}
} else if (isTriggerKind(kind)) {
const triggerBody = await provider.getTriggerForDeploy(kind, {
workspace: workspaceFrom,
path,
onBehalfOf,
});
// Strip operational state on update; pass through on create. The
// implementor's `getTriggerForDeploy` is responsible only for kind-specific
// transforms (e.g. GCP subscription_id wipe) — the operational-state
// strip lives here so it stays consistent with the schedule branch and
// with the legacy `kind === 'trigger'` path in the frontend.
const requestBody = stripOperationalStateOnUpdate(triggerBody, alreadyExists);
if (alreadyExists) {
await provider.updateTriggerByKind(kind, {
workspace: workspaceTo,
path,
requestBody,
});
} else {
await provider.createTriggerByKind(kind, {
workspace: workspaceTo,
requestBody,
});
}
} else if (kind === "datatable_migration") {
const { datatable, timestamp } = parseDatatableMigrationDeployPath(path);
const migrations = await provider.listDatatableMigrations({
workspace: workspaceFrom,
});
const migration = (
migrations as {
datatable: string;
timestamp: number;
name: string;
code_up: string;
code_down?: string;
}[]
).find((m) => m.datatable === datatable && m.timestamp === timestamp);
if (!migration) {
throw new Error(
`Datatable migration ${path} not found in ${workspaceFrom}`
);
}
try {
await provider.upsertDatatableMigration({
workspace: workspaceTo,
datatableName: datatable,
requestBody: {
timestamp: migration.timestamp,
name: migration.name,
code_up: migration.code_up,
...(migration.code_down != null
? { code_down: migration.code_down }
: {}),
},
});
} catch (e) {
throw asMigrationsDisabledError(e, datatable);
}
} else {
throw new Error(`Unknown kind: ${kind}`);
}
return { success: true };
} catch (e: unknown) {
return { success: false, error: toError(e) };
}
}
// ---------------------------------------------------------------------------
// deleteItemInWorkspace
// ---------------------------------------------------------------------------
/**
* Delete/archive an item in a workspace.
* Scripts and flows are archived (reversible). Other types are deleted.
*/
export async function deleteItemInWorkspace(
provider: DeployProvider,
kind: DeployKind,
path: string,
workspace: string
): Promise<DeployResult> {
try {
if (kind === "script") {
await provider.archiveScriptByPath({ workspace, path });
} else if (kind === "flow") {
await provider.archiveFlowByPath({
workspace,
path,
requestBody: { archived: true },
});
} else if (kind === "app" || kind === "raw_app") {
await provider.deleteApp({ workspace, path });
} else if (kind === "variable") {
await provider.deleteVariable({ workspace, path });
} else if (kind === "resource") {
await provider.deleteResource({ workspace, path });
} else if (kind === "resource_type") {
await provider.deleteResourceType({ workspace, path });
} else if (kind === "folder") {
await provider.deleteFolder({ workspace, name: folderName(path) });
} else if (kind === "schedule") {
await provider.deleteSchedule({ workspace, path });
} else if (kind === "datatable_migration") {
const { datatable, timestamp } = parseDatatableMigrationDeployPath(path);
await provider.deleteDatatableMigration({
workspace,
datatableName: datatable,
timestamp,
});
} else if (isTriggerKind(kind)) {
await provider.deleteTriggerByKind(kind, { workspace, path });
} else {
throw new Error(`Deletion not supported for kind: ${kind}`);
}
return { success: true };
} catch (e: unknown) {
return { success: false, error: toError(e) };
}
}
// ---------------------------------------------------------------------------
// getOnBehalfOf
// ---------------------------------------------------------------------------
/**
* Get the value of an item for diff comparison.
* Returns a normalized representation suitable for JSON comparison.
*/
export async function getItemValue(
provider: DeployProvider,
kind: DeployKind,
path: string,
workspace: string
): Promise<unknown> {
try {
if (kind === "flow") {
const flow = await provider.getFlowByPath({ workspace, path });
getAllModules(flow.value?.modules ?? [], flow.value?.failure_module).forEach(
(x: any) => {
if (x.value?.type === "script" && x.value.hash != undefined) {
x.value.hash = undefined;
}
}
);
return {
summary: flow.summary,
description: flow.description,
value: flow.value,
job_token_scopes: flow.job_token_scopes ?? null,
};
} else if (kind === "script") {
const script = await provider.getScriptByPath({ workspace, path });
return {
content: script.content,
lock: script.lock,
schema: script.schema,
summary: script.summary,
language: script.language,
job_token_scopes: script.job_token_scopes ?? null,
};
} else if (kind === "app" || kind === "raw_app") {
return await provider.getAppByPath({ workspace, path });
} else if (kind === "variable") {
const variable = await provider.getVariable({
workspace,
path,
decryptSecret: true,
});
return {
value: variable.value,
is_secret: variable.is_secret,
description: variable.description,
value_valid_until: variable.value_valid_until ?? undefined,
};
} else if (kind === "resource") {
const resource = await provider.getResource({ workspace, path });
// Each workspace resolves its own agent identity, which a deploy never copies.
if (resource.resource_type === "ai_agent" && resource.value) {
const { on_behalf_of: _, ...value } = resource.value;
return value;
}
return resource.value;
} else if (kind === "resource_type") {
const rt = await provider.getResourceType({ workspace, path });
return rt.schema;
} else if (kind === "folder") {
const folder = await provider.getFolder({
workspace,
name: folderName(path),
});
return {
name: folder.name,
owners: folder.owners,
extra_perms: folder.extra_perms,
summary: folder.summary,
};
} else if (kind === "schedule") {
// Mirror the runtime-fields-ignore set used server-side so the diff drawer
// matches the merge UI's "did config change?" semantics.
const schedule = await provider.getSchedule({ workspace, path });
return stripTriggerOrScheduleRuntimeFields(schedule);
} else if (isTriggerKind(kind)) {
const trigger = await provider.getTriggerValue(kind, { workspace, path });
return stripTriggerOrScheduleRuntimeFields(trigger);
} else if (kind === "datatable_migration") {
// Surface the migration SQL so the diff drawer shows the up/down bodies a
// reviewer needs to inspect before deploying.
const { datatable, timestamp } = parseDatatableMigrationDeployPath(path);
const migrations = (await provider.listDatatableMigrations({
workspace,
})) as {
datatable: string;
timestamp: number;
name: string;
code_up: string;
code_down?: string;
}[];
const migration = migrations.find(
(m) => m.datatable === datatable && m.timestamp === timestamp
);
if (migration) {
return {
name: migration.name,
code_up: migration.code_up,
code_down: migration.code_down ?? null,
};
}
}
} catch {
// Item may not exist
}
return {};
}
/**
* Strip runtime fields from a trigger or schedule row so the diff drawer
* shows only config differences. Mirrors the backend's
* `TRIGGER_COMPARE_IGNORE` constant.
*/
function stripTriggerOrScheduleRuntimeFields(row: unknown): unknown {
if (!row || typeof row !== "object") return row;
const ignore = new Set([
"workspace_id",
"edited_by",
"edited_at",
"email",
"error",
"enabled",
"mode",
"server_id",
"last_server_ping",
"extra_perms",
"permissioned_as",
// Server-managed (kept in sync with backend `TRIGGER_COMPARE_IGNORE`).
"subscription_id",
"push_auth_config",
]);
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(row as Record<string, unknown>)) {
if (!ignore.has(k)) out[k] = v;
}
return out;
}
/**
* Fetch the on_behalf_of value for a deployable item.
* Returns an email for flows/scripts/apps, a principal (u/username, g/group) for
* triggers/schedules and agents, or undefined if not applicable.
*/
export async function getOnBehalfOf(
provider: DeployProvider,
kind: DeployKind,
path: string,
workspace: string
): Promise<string | undefined> {
try {
if (kind === "flow") {
const flow = await provider.getFlowByPath({ workspace, path });
return flow.on_behalf_of_email;
} else if (kind === "script") {
const script = await provider.getScriptByPath({ workspace, path });
return script.on_behalf_of_email;
} else if (kind === "app" || kind === "raw_app") {
const app = await provider.getAppByPath({ workspace, path });
return app.policy?.on_behalf_of_email;
} else if (kind === "resource") {
// Only an agent runs as anyone, and like a trigger it records the principal.
const resource = await provider.getResource({ workspace, path });
return resource.resource_type === "ai_agent"
? resource.value?.on_behalf_of
: undefined;
} else if (kind === "schedule") {
const schedule = await provider.getSchedule({ workspace, path });
return schedule.permissioned_as;
} else if (isTriggerKind(kind)) {
return await provider.getTriggerPermissionedAs(kind, { workspace, path });
}
} catch {
// Item may not exist
}
return undefined;
}