mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-10 16:05:58 +00:00
* docs: add design doc for automatic git-to-windmill pull sync
* docs: add migration plan and implementation phases to git-sync pull design
* feat(git-sync): add auto_pull settings schema and pull enqueue primitive
Adds AutoPullSettings/AutoPullMode/AutoPullStatus on GitRepositorySettings
(workspace_settings.git_sync JSONB), the GIT_SYNC_PULL_SCRIPT_PATH constant,
and should_pull/effective_poll_interval_s helpers with unit tests. Exports the
EE enqueue_git_pull_job primitive. Foundation for repo→Windmill auto-pull.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): poll repos and auto-pull new commits into the workspace
Phase 1 of automatic repo → Windmill sync. A monitor task (EE-licensed,
single-replica via advisory lock) git ls-remotes each auto-pull-enabled
repository ~every minute and enqueues a pull when the tracked branch moves,
reusing the {workspace_id}:git_sync concurrency key so pulls serialize with
in-flight push commits.
- windmill-store: background (no-authed) resolver get_git_repo_head_for_autopull
that resolves the repo resource (incl. $var: refs) and ls-remotes; GitHub-App
repos are skipped here and will sync via webhooks (phase 2).
- monitor.rs: poll/reconcile/persist with optimistic sha advance and failure
status; targeted jsonb update so concurrent settings edits aren't clobbered.
- edit_git_sync_repository: preserve server-owned auto_pull state on UI save.
- openapi: AutoPullSettings/AutoPullMode/AutoPullStatus + auto_pull field.
- frontend: per-repo "Automatically deploy changes from Git" toggle with last
sync status; demote the GitHub Actions link to an advanced CI option.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): wire webhook lifecycle + receiver; share reconcile logic
OSS side of phase 2 auto-pull webhooks:
- edit_git_sync_repository creates/removes the repo webhook on save (EE-gated,
best-effort → falls back to polling).
- monitor poller now delegates to the shared windmill_git_sync reconcile/persist
helpers (also used by the webhook receiver), removing duplicated logic.
- export the shared reconcile/persist/failure helpers; bump EE ref.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref for phase 3 in-app PR creation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): show webhook vs polling status on the auto-pull toggle
When a repo has an active webhook (auto_pull.webhook_id set), the status line
reads "instant via webhook"; otherwise it reads the ~1-minute polling cadence.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(git-sync): post PR diff check on dry-run completion (phase 4)
Worker completion hook in process_completed_job: when a DeploymentCallback job
carrying the __git_sync_pr_check marker finishes, parse the dry-run SyncResponse
and patch the GitHub check run with the diff summary (success/neutral/failure).
Export enqueue_git_pull_dry_run; bump EE ref.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref (drop unused GHES webhook_secret)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* revert(git-sync): defer phase 4 PR diff checks (OSS side)
Remove the worker completion hook that posted the PR check run, drop the
enqueue_git_pull_dry_run re-export and the orphaned sqlx cache, bump EE ref.
Phases 1-3 (polling, webhooks, in-app PR creation) are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Revert "revert(git-sync): defer phase 4 PR diff checks (OSS side)"
This reverts commit 0137d3ca48.
* chore(git-sync): point EE ref at restored phase 4 commit
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref for clone_ref dry-run
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump init-repository hub script to v28784
Picks up the clone_ref param (windmill-integrations#158) so the phase 4 PR-check
dry-run can clone the PR head. Backward compatible; manual pull/push and the
automated pull/poller/webhook all move to the same published version.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref for auto-pull admin-permissioning fix
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(git-sync): bump EE ref for superadmin pull fallback
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(git-sync): refresh auto-pull tooltip; bump EE ref for webhook secret encryption
The auto-pull toggle tooltip claimed GitHub App repos would sync via
webhooks "in a future update"; webhook delivery now works, so describe
the webhook-vs-polling behavior accurately. Bump the EE ref to pick up
encrypting the webhook HMAC secret at rest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(git-sync): poll app-backed repos in auto/polling mode
The auto-pull poller skipped app-backed repos (the ls-remote head check
can't authenticate a tokenless URL), so auto- and polling-mode app repos
never synced when their webhook wasn't live. Wire the poller to fetch the
head via the GitHub API for app repos and reconcile. Bump the EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(git-sync): auto-pull UI — direction split, delivery mode, fallback notice
Reorganize the repository card into two clearly labeled directions:
"Push to Git on deploy (Windmill → Git)" and "Pull from Git (Git →
Windmill)". In the pull section:
- new connections default to auto-pull enabled (webhook with polling
fallback); existing repos load with auto-pull off and are unchanged
- a Delivery selector chooses "Webhook with polling fallback" or
"Polling only (air-gapped)"
- a notice surfaces webhook_error when delivery falls back to polling
- a reminder to remove any pre-existing GitHub Action that pushed into
Windmill, to avoid conflicting double-syncs
Adds the webhook_error field to AutoPullSettings (+ openapi) and bumps
the EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(git-sync): clearer push indicator + gate webhook delivery to app repos
- Push-on-deploy is shown with a check icon + concise line (via the
shared GitSyncModeDisplay, restyled from the oversized "Sync:" text);
the setup wizard reuses it without the check (pre-save preview).
- The delivery-mode selector only shows for GitHub App-backed repos;
token-based repos show a "webhooks require the GitHub App (managed or
GHES)" note with a docs link and poll instead. Bumps the EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(git-sync): fork auto-sync (phase 5) + live deploy check (phase 6)
Phase 5 — fork auto-sync configured at the parent (replaces the *-to-forks
GitHub Actions):
- Add fork_open_prs + fork_pull_sync to GitRepositorySettings (openapi + UI).
- UI: two "Forks of this workspace" toggles in the repo card, gated on
app-backed and not-a-fork; serialize the flags on save.
- On fork creation, strip the inherited auto_pull block (and fork_* flags) from
the copied git_sync repo: a fork must not carry the parent's webhook id (it
would delete the parent's hook on disable) or self-poll on top of the parent's
fan-out. Push-direction config + installation are still inherited unchanged.
Phase 6 — live deploy status check on the commit (Cloudflare-style): an
in-progress "Windmill" check on the head commit that flips to "Deployed N
changes"; completion handled by the generalized git-sync check hook.
Bump EE ref for the phase 5-6 EE implementation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* chore(git-sync): bump EE ref for PAT auto-pull mode normalization
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): address PR review findings
- webhook_secret: redact from the settings API response and Debug output (still
persisted encrypted); it's a server-only HMAC key the UI never needs.
- poller: honor each repo's effective poll interval (relaxed ~10 min when a
webhook is live) instead of probing every ~60s tick.
- settings save: roll back a just-created webhook if the settings transaction
doesn't commit, so a failed save can't orphan a hook.
- auto-pull head check: fail SSH remotes with an actionable message (background
polling has no SSH identity) instead of a confusing ls-remote error.
- deploy/PR check summary: a pull result carrying neither changes nor a settings
diff now falls back to the unsummarized path instead of a false "in sync".
- UI: reset isGithubApp on resource change / failed fetch so webhook + fork
controls can't show for the wrong repo.
- tests: cover parse_git_sync_changes and format_change_list edge cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): correct feature gating for OSS builds
- monitor.rs: keep the AUTO_PULL_LAST_POLL static, slack const, and
poll_git_auto_pull_inner all behind #[cfg(feature = "private")] (an inserted
static had split the cfg off the function, ungating it in OSS builds).
- edit_git_sync_repository: the webhook create/rollback block references
windmill_common::git_sync_ee (private module), so gate it on
all(enterprise, private) instead of enterprise only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* chore(sqlx): cache workspace_diff query pulled in from origin/main
Re-merged origin/main (advanced past the earlier merge); regenerate the offline
sqlx entry for the new workspace_comparison test query so SQLX_OFFLINE builds
(cargo_test) pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): address Codex review findings (webhook cleanup on delete)
- Deleting a git-sync repository now tears down its managed GitHub webhook
(deletion bypassed the sync_repo_webhook lifecycle, orphaning the hook so
GitHub kept delivering to the instance).
- Worker completion hook rolls back the optimistic auto-pull sha on job failure
(OSS side of the EE change) + caches the new marker query. Bump EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): delete repo webhook after the removal commits
Codex re-review nits:
- delete_git_sync_repository deleted the webhook before the settings transaction
committed; a failed save would then leave the repo pointing at a hook that no
longer exists (sync_repo_webhook treats a set webhook_id as live and won't
recreate it). Capture the hook id, commit the DB removal, then delete the hook.
- Reword a fork-copy comment to drop drafting-history wording per AGENTS.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): reconcile the edit-path webhook after the settings commit
Codex nit: edit_git_sync_repository ran sync_repo_webhook before the transaction
committed. The rollback only covered created hooks, but sync_repo_webhook also
deletes a hook on disable/switch-to-polling — a commit failure then left the DB
with a webhook_id whose hook was already gone (and it wouldn't be recreated).
Save + commit first, then reconcile the webhook against the durable config and
persist any hook id/secret change (best-effort). Bump EE ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): preserve webhook secret on whole-config save + default on visible add
Codex nits:
- edit_git_sync_config saved the client config verbatim, so the webhook_secret
redacted from the GET response would be dropped (breaking delivery). Preserve
server-owned auto-pull state (webhook id/secret, synced sha, last status) per
repo from the existing settings, matching edit_git_sync_repository.
- addSyncRepository (the visible add path) didn't set the auto_pull default, so
new sync repos added from the UI came up with auto-deploy off. Match
addRepository's default (webhook + polling fallback).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* refactor(git-sync): drop fork_pull_sync (parent-level keep-forks-in-sync)
Removes the "Keep forks in sync with the tracked branch" toggle and its
fan-out. Pulling the tracked branch straight into every fork was the
inconsistent piece; the consistent model is per-fork branch sync (each
fork tracks its own wm-fork/** branch), which is a separate follow-up.
fork_open_prs is kept. Also tightens the fork toggle-section spacing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): detect dev workspaces in CLI fork branch derivation
isForkWorkspace / computeGitSyncDeployBranch keyed off the wm-fork- id
prefix. Dev workspaces are forks with a custom, prefix-less id, so their
wm-fork/** branch was never derived or created. Detect them via
parent_workspace_id too (which the backend already passes), mirroring the
backend's `parent.is_some() || wm-fork- prefix` rule.
Pairs with the hub-script clone-flag fix (windmill-integrations#163); both
take effect once the CLI is released and the pinned version is bumped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): reconcile webhooks on full-config save
edit_git_sync_config preserved server-owned webhook fields but never
created or deleted the managed GitHub webhook, so enabling auto-pull
through the whole-config endpoint only polled, and disabling or removing
a repo left an orphan hook still delivering. Mirror the per-repository
endpoint: after the commit is durable, reconcile every saved repo's
webhook (sync_repo_webhook) and delete the hooks of repos the save
removed, including the clear-whole-config case. Addresses the Codex nit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): address Codex nits (webhook orphan on cleared auto_pull, fork detection)
- edit_git_sync_config: also delete a repo's old webhook when the save drops
the repo OR clears its auto_pull. Webhook fields are only preserved onto a
Some auto_pull, so a save that present-but-clears a repo would otherwise
orphan its hook.
- GitSyncRepositoryCard: isFork now uses parent_workspace_id OR the wm-fork-
prefix (was AND), matching the backend/CLI rule, so prefix-less dev
workspaces are detected as forks and don't show the parent fork-PR toggle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* docs(git-sync): update design doc for the dropped fork_pull_sync
Phase 5 documented "Keep forks in sync with the tracked branch"
(fork_pull_sync) and its fan-out as implemented; that feature was removed.
Rewrite the section to reflect what ships (fork_open_prs), note the drop +
the per-fork-branch follow-up, and remove the stale fan-out mentions
elsewhere. Addresses the Codex nit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): redact webhook secrets from workspace export; fix doc endpoints
- Export (P1): strip the server-owned auto_pull state (webhook secret/id/error
+ synced sha + last pull status) from git_sync before it is written into an
export's settings.json for both settings formats. The HMAC webhook secret
must never leave the server (matching the GET-settings redaction), and a
re-imported workspace must not inherit another install's hook/sync state.
- Docs: the webhook receiver is a single per-workspace endpoint
/api/w/{workspace}/github_app/webhook (host-aware for managed + self-managed);
update the stale push_webhook/{id} and instance-global /api/github_app/webhook
references.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): skip deleted/archived workspaces in the auto-pull poller
The poller scanned workspace_settings directly, so an archived (soft-deleted)
or renamed-away workspace — whose settings row persists — kept polling and
could enqueue a pull into a dead workspace. Join workspace and require
NOT deleted. The EE webhook receiver gets the same filter (ee ref bumped).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): never trust client-supplied server-owned auto-pull fields
Both write endpoints (edit_git_sync_repository, edit_git_sync_config)
persisted caller-supplied auto_pull.webhook_id / webhook_secret /
webhook_error / last_synced_sha / last_pull_status when adding a repo or
newly enabling auto-pull, letting a client inject a webhook id/secret or
fake sync state. Strip those server-owned fields from the request up front;
existing repos re-derive them from the DB (carried over), new ones start
clean and the server (re)creates the webhook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): merge webhook fields post-commit instead of clobbering the row
The post-commit webhook reconcile in edit_git_sync_repository and
edit_git_sync_config wrote the whole pre-reconcile git_sync snapshot back
after the main save committed. A concurrent git-sync edit or poller status
write that landed in the gap could then be dropped by the stale snapshot.
Re-read the current row and merge only the reconciled webhook id/secret/error
for the repos the reconcile actually changed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): parent-managed fork sync + PR-on-deploy toggles
Fork sync (push-on-merge-to-forks parity): a parent-level
auto_pull.sync_forks toggle routes changes on each fork's wm-fork/** branch
into that fork workspace, via the parent's existing webhook and one extra
fork-heads listing per poll tick (git ls-remote pattern for token repos,
git/matching-refs for app-backed). Fork state is a server-written
status-only auto_pull blob on the fork's own repo entry; the fork's card
shows a read-only "managed in the parent workspace" line with its branch
and last pull status. Dev workspaces (prefix-less ids) use the same branch
parsing (unit-tested in windmill-common).
PR-on-deploy: opening PRs for Windmill-pushed branches moves into the
deploy pipeline, per repo toggle (promotion_open_prs on the promotion
repo; parent-level fork_open_prs for fork deploys). The push job carries a
marker and the job-completion hook derives the pushed branch (helper
unit-tested against the CLI formula) and opens the PR outbound, so it
works without inbound webhooks; the webhook-side wm_deploy PR arm is
removed. The documented open-pr-* GitHub Actions remain valid alternatives
(PR creation is idempotent).
Fork guards: promotion mode, enabled auto-pull, and fork_open_prs are
rejected on fork workspaces (they are parent-managed; a fork's deploys
always target its wm-fork/** branch) and the promotion card is hidden in a
fork's settings. Enabling auto-pull now also requires EE, and the
post-commit webhook reconcile persists the normalized delivery mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): dev workspaces sync with their environment-label branch
A dev workspace's git branch is its environment label verbatim (dev/
staging, default dev) — a first-class env branch like the documented
push-on-merge-staging layout — instead of the wm-fork/** form. The label
rides the deploy job args (backend → hub script → CLI
--dev-workspace-label), the PR completion hook derives the same head, the
webhook/poller route label branches into the matching dev-workspace child
(poller lists them alongside wm-fork/* via extra ls-remote refs / per-label
API lookups), and manual pulls from the UI pass clone_ref accordingly. The
CLI refuses to deploy when the label branch equals the checked-out tracked
branch, which would otherwise commit fork content straight to it.
Because the branch is keyed on the label, the label is now immutable after
creation: set at create/attach only, the set_dev_workspace_label endpoint
is removed and the settings tab shows it read-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): nested fork routing + fork-of-dev branch rooting
A fork of a dev workspace now roots its wm-fork/** branch on the dev's
environment-label branch (the content it diverged from) and its PR merges
back into that branch: the backend passes parent_dev_workspace_label with
the deploy (parent row joined in both enqueue paths), the CLI gains
--parent-dev-workspace-label and checks it before the wm-fork- prefix
fallback when rooting a fork-of-a-fork branch, and the PR completion hook
uses it as the PR base.
Fork sync routing covers the whole live descendant chain of the
webhook/poller workspace (recursive, depth-capped) instead of direct
children only, and fork_open_prs is resolved at the root ancestor — only
the root can hold auto-pull config, so grandchild forks sync through it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): PR deploy-preview comment, clearer check copy, app-only hints
- The PR diff completion hook maintains one managed comment on the PR
(Cloudflare deploy-preview style: workspace, status, commit, collapsible
change list), upserted per synchronize via a hidden marker. The check run
stays for required-check gating.
- A settings difference in the diff summary is worded by cause: the PR
changes wmill.yaml, vs pre-existing drift between the repo's wmill.yaml
and the workspace, vs undetermined (neutral wording).
- Deploy-status check titles name the target workspace ("Deployed 2
change(s) to staging"), since GitHub shows a head commit's checks on any
PR containing it and a bare "Deployed" read as if the PR had deployed.
- Token-based repos see a hint pointing at the open-pr-on-commit /
open-pr-on-fork-commit workflows where the app-only PR toggles would be;
an API-set toggle on a non-app repo now logs a warning naming the
fallback; the design doc lists app-only features and their degradation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): EE-gate auto-pull UI, fork pull clone_ref, no-op push PR gate
- CE: the auto-pull and fork-PR toggles are disabled with an EE badge, and
new sync repos only default them on when licensed (basic git sync is
available on CE since #8493, but auto-pull is EE and the backend rejects it)
- The pull modal passes clone_ref for wm-fork- forks (wm-fork/<tracked>/<id>)
so a manual pull fetches the fork branch instead of the tracked branch head
- PR-on-deploy skips no-op pushes: when the push script reports pushed=false
(e.g. the deploy was caused by an auto-pull), the completion hook no longer
ensures a PR, so closed PRs aren't recreated by the sync loop
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* chore: refresh package-lock after main merge (windmill-utils-internal 1.8.2)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* test: auto-pull e2e integration tests; fix PR comment table formatting
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): runtime license gate for auto-pull saves; user/group promotion-branch parity
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): explain in-sync PR verdicts with the repo's sync filter scope
A PR that only touches files outside the repository's include paths gets
"In sync", which reads as a wrong verdict; the check summary (and managed
comment) now name the filters, e.g. "Only files matching this repository's
sync filters deploy on merge: `f/**` (excluding `f/pat/**`)."
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): clearer card copy/structure; surface PR-creation failures
- Fork sync toggle renamed and kept in the pull section; the fork PR toggle
moves to the push section with a note that push settings apply to forks
- Fork/dev workspaces' push section names their actual branch instead of the
tracked-branch line; promotion repos hide the pull direction (promotion
pushes deploy branches on top of a sync-mode setup)
- Promotion mode line describes the wm_deploy/** branch + merge-to-promote
flow; workflow-fallback hints lead with the how-to and link to the docs;
test connection button demoted from accent per brand guidelines
- New server-owned open_pr_error on repo settings: the deploy completion hook
records why a PR couldn't be opened (e.g. app permission not yet approved)
and clears it on the next success; shown as a warning under the PR toggles
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix: cfg-gate scope-note helper (dead code on OSS builds)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): license-gate preserved auto-pull; attach strips parent-only settings
- edit_git_sync_repository re-checks the runtime Enterprise gate against the
EFFECTIVE repo state after preservation: the older-client arm copies the
existing auto_pull back, which the request-side check never saw
- attach_dev_workspace now mirrors the fork-creation copy on the attached
workspace's own git sync: promotion repos dropped, auto_pull/fork PRs/PR
error stripped, and any managed webhook deleted after commit (the attached
workspace is parent-managed and must not keep pulling its old tracked branch)
- integration test: attaching an auto-pull-enabled workspace strips it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): detach clears standalone parent; reject label == tracked branch
- detach_dev_workspace clears parent_workspace_id for prefix-less (attached
standalone) workspaces so they stop classifying as forks and deploying to
wm-fork/** branches; wm-fork- re-designated forks keep their parent; cache
invalidations mirror attach
- dev-workspace create/attach reject an environment label that equals a
git-sync repository's tracked branch (prod's or the candidate's): deploys
would target the very branch the repo syncs from, and the CLI guard would
fail every push job after the fact
- CLI unit tests: prefix-less fork beats wm_deploy derivation; isForkWorkspace
parent-id argument
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* chore(git-sync): bump hub script pins (push 28786, pull 28785)
Published from windmill-integrations #163 with windmill-cli@1.753.1-gitsync.0:
dev-workspace label deploys, fork-of-dev rooting, fork checkout on the
existing remote branch, and the pushed-flag result.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): no parent-only defaults on fork repos; rename strips webhook state
- addSyncRepository skips the auto_pull/fork_open_prs defaults on fork/dev
workspaces where the backend rejects them (saving a new sync repo from an
EE fork 400'd deterministically)
- change_workspace_id strips webhook id/secret/error from the copied git_sync
and deletes the stale GitHub hooks post-commit: they deliver to the old
(archived) workspace URL, so the new workspace would report a live webhook
while polling at the relaxed interval; next save re-registers cleanly
- EE: PR diff checks for contributor-fork PRs clone the synthetic
pull/<n>/head ref (head.ref doesn't exist in the base repo)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* chore(git-sync): bump pull script pin to hub/28787 (synthetic PR ref support)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): targeted jsonb update for open_pr_error (no full-blob clobber)
The full read-modify-write raced the poller's concurrent last_synced_sha /
last_pull_status writes on the same column; mirror the EE status writer and
update only the matching repository element's open_pr_error key.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* style(git-sync): inline EE badge on gated toggles (matches settings nav)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* copy(git-sync): both directions in page/card descriptions; clearer promotion flow
- Page header and sync-card description mention the pull direction, not only
push-on-deploy
- Promotion description walks the actual flow (wm_deploy/** branch, merge to
promote, sync the target workspace) and points at the PR toggle / workflow;
the Git Promotion docs link now also shows on configured cards, not only in
the empty state
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): resolve branch-less resources' default branch for fork sync
A git resource without an explicit branch polled as the bare "HEAD" ref,
which the fork/dev-label fan-out cannot scope (wm-fork/<branch>/*), so fork
sync silently never ran on polling-only repos. Resolve the remote's default
branch name with `ls-remote --symref HEAD` (one call for name + head sha);
"HEAD" only remains when resolution fails. The polling e2e test now uses a
branch-less resource to cover this shape.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): runtime license gate for in-app PR creation
promotion_open_prs/fork_open_prs are rejected on save without an Enterprise
plan (like auto_pull), and the deploy completion hook re-checks the plan
before opening PRs so flags stored while licensed stop driving GitHub calls
after a lapse.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): app-aware pull defaults, always webhook delivery, token-repo guidance
- Pull-from-Git defaults on only for app-backed repos (applied when the
selected resource resolves); polling is opt-in for token repositories,
with a warning alert recommending the GitHub App (instant pull + in-app
PRs) or the sync GitHub workflow
- App repos always use webhook delivery with polling fallback: the delivery
selector is gone and a stored polling mode is normalized back to auto
- Post-save modal reflects the auto-pull state instead of telling the user
to turn on a toggle that is already on
- Non-app PR hints recommend the GitHub App explicitly
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* copy(git-sync): single info box for token-repo pull guidance
Merges the instant-pull recommendation with the GitHub Action conflict note,
shown only for non-app repos; app repos need neither, and the redundant
'instant webhook sync requires' line is gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* copy(git-sync): keep the GitHub Action conflict note on app repos
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* copy(git-sync): anchor docs links to their exact sections
GitHub App references point at integrations/git_repository#github-app, the
workflow hints at deploy_gh_gl#github-actions-setup, and the sync workflow
at git_sync#github-actions (all anchors verified against the live docs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* copy(git-sync): fork workflow hint links to git_sync#github-actions
open-pr-on-fork-commit is documented on the git_sync page, not deploy_gh_gl.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* fix(git-sync): fork PRs are opt-in on new connections too
Only auto-pull and fork sync default on for new app-backed connections;
opening pull requests stays a deliberate per-repo decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* feat(git-sync): default the managed PR on for new app-backed promotion repos
A promotion deploy's wm_deploy/** branch exists to be merged; without a PR
it's an orphaned branch. Fork PRs stay opt-in. Also scope the sync-repo
auto-pull default to sync mode so promotion repos can't pick it up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* docs(git-sync): GHES self-managed app permission setup
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* docs(git-sync): frame permission update against GitHub Actions, not polling
Existing installations don't have polling; their git-to-Windmill direction
runs on GitHub Actions today, so the approval text describes the update as
replacing those workflows and notes every feature is opt-in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm
* copy(workspaces): drop 'cosmetic' qualifier from dev-workspace label UI
* chore: update ee-repo-ref to 9b2a6375f838436cf68cff449cc9bc621cca5281
This commit updates the EE repository reference after PR #632 was merged in windmill-ee-private.
Previous ee-repo-ref: 99eef24e2f0402b9a997cde5f67be52ee5d54b0e
New ee-repo-ref: 9b2a6375f838436cf68cff449cc9bc621cca5281
Automated by sync-ee-ref workflow.
* fix(git-sync): reject '/' in fork and dev workspace ids
* fix(git-sync): bound auto-pull git probes with a per-command timeout
* fix(git-sync): persist webhook reconcile via targeted jsonb updates
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
1686 lines
64 KiB
Rust
1686 lines
64 KiB
Rust
/*
|
|
* Author: Ruben Fiszel
|
|
* Copyright: Windmill Labs, Inc 2022
|
|
* This file and its contents are licensed under the AGPLv3 License.
|
|
* Please see the included NOTICE for copyright information and
|
|
* LICENSE-AGPL for a copy of the license.
|
|
*/
|
|
|
|
use std::collections::HashMap;
|
|
|
|
use crate::db::ApiAuthed;
|
|
|
|
use crate::{apps::AppWithLastVersion, db::DB, folders::Folder};
|
|
|
|
use windmill_api_auth::check_scopes;
|
|
|
|
#[cfg(any(
|
|
feature = "http_trigger",
|
|
feature = "websocket",
|
|
feature = "postgres_trigger",
|
|
feature = "mqtt_trigger",
|
|
all(
|
|
feature = "enterprise",
|
|
any(
|
|
feature = "kafka",
|
|
feature = "sqs_trigger",
|
|
feature = "gcp_trigger",
|
|
feature = "azure_trigger",
|
|
feature = "nats",
|
|
feature = "smtp",
|
|
),
|
|
feature = "private"
|
|
)
|
|
))]
|
|
use crate::triggers::TriggerCrud;
|
|
|
|
use axum::{
|
|
extract::{Extension, Path, Query},
|
|
response::IntoResponse,
|
|
};
|
|
|
|
use http::HeaderName;
|
|
use itertools::Itertools;
|
|
|
|
use windmill_common::runnable_settings::{ConcurrencySettings, DebouncingSettings};
|
|
use windmill_common::scripts::ScriptRunnableSettingsHandle;
|
|
use windmill_common::utils::require_admin;
|
|
use windmill_common::variables::decrypt;
|
|
use windmill_common::worker::WINDMILL_DIR;
|
|
use windmill_common::{
|
|
db::UserDB,
|
|
error::{to_anyhow, Error, Result},
|
|
flows::Flow,
|
|
schedule::Schedule,
|
|
scripts::{Schema, Script, ScriptLang},
|
|
variables::{build_crypt, ExportableListableVariable},
|
|
workspace_dependencies::WorkspaceDependencies,
|
|
};
|
|
|
|
use hyper::header;
|
|
use serde::{Deserialize, Serialize};
|
|
use serde_json::Value;
|
|
use tempfile::TempDir;
|
|
use tokio::fs::File;
|
|
use tokio_util::io::ReaderStream;
|
|
use windmill_store::resources::{Resource, ResourceType};
|
|
|
|
#[derive(Serialize)]
|
|
struct ScriptMetadata {
|
|
summary: String,
|
|
description: String,
|
|
schema: Option<Schema>,
|
|
lock: Option<String>,
|
|
kind: String,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
envs: Option<Vec<String>>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
cache_ttl: Option<i32>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
dedicated_worker: Option<bool>,
|
|
#[serde(skip_serializing_if = "is_none_or_false")]
|
|
ws_error_handler_muted: Option<bool>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
priority: Option<i16>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
tag: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub timeout: Option<i32>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub delete_after_secs: Option<i32>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub restart_unless_cancelled: Option<bool>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub visible_to_runner_only: Option<bool>,
|
|
// auto_kind is intentionally excluded from export — it is auto-detected by the
|
|
// parser at deploy time from the script content (workflow/task patterns for "wac",
|
|
// no main function for "lib").
|
|
#[serde(skip_serializing)]
|
|
#[allow(dead_code)]
|
|
pub auto_kind: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub codebase: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub has_preprocessor: Option<bool>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub on_behalf_of_email: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub modules: Option<std::collections::HashMap<String, windmill_common::scripts::ScriptModule>>,
|
|
#[serde(flatten)]
|
|
pub concurrency_settings: ConcurrencySettings,
|
|
#[serde(flatten)]
|
|
pub debouncing_settings: DebouncingSettings,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub labels: Option<Vec<String>>,
|
|
#[serde(skip_serializing_if = "is_empty_extra_perms")]
|
|
pub extra_perms: serde_json::Value,
|
|
}
|
|
|
|
fn is_empty_extra_perms(value: &serde_json::Value) -> bool {
|
|
value.as_object().is_some_and(|o| o.is_empty()) || value.is_null()
|
|
}
|
|
|
|
pub fn is_none_or_false(val: &Option<bool>) -> bool {
|
|
match val {
|
|
Some(val) => !val,
|
|
None => true,
|
|
}
|
|
}
|
|
|
|
/// A fork's git-sync export rewrites each trigger's `mode` (and each schedule's
|
|
/// `enabled`) to the *parent* workspace's value, instead of emitting the fork's
|
|
/// own (clone-disabled / locally-toggled) state. This keeps the fork's synced
|
|
/// file byte-identical to the parent on the operational-state field, so a
|
|
/// normal-git PR merge has nothing to resolve — no dropped `mode:` line, no
|
|
/// flipped parent trigger. Fork-only paths (absent from the parent) keep the
|
|
/// fork's own value: there's no parent state to defer to, so the trigger lands
|
|
/// with whatever the fork creator set. The write half of the same rule lives in
|
|
/// `windmill-trigger::handler::workspace_is_fork`.
|
|
///
|
|
/// Maps trigger `path` → parent `mode` (as the lowercase enum text that matches
|
|
/// `TriggerMode`'s serde representation). Empty when not a fork.
|
|
#[cfg(any(
|
|
feature = "http_trigger",
|
|
feature = "websocket",
|
|
feature = "postgres_trigger",
|
|
feature = "mqtt_trigger",
|
|
feature = "native_trigger",
|
|
all(
|
|
feature = "enterprise",
|
|
any(
|
|
feature = "kafka",
|
|
feature = "sqs_trigger",
|
|
feature = "gcp_trigger",
|
|
feature = "azure_trigger",
|
|
feature = "nats",
|
|
feature = "smtp",
|
|
),
|
|
feature = "private"
|
|
)
|
|
))]
|
|
async fn fork_parent_trigger_modes(
|
|
db: &DB,
|
|
table_name: &str,
|
|
parent_workspace_id: Option<&str>,
|
|
) -> Result<HashMap<String, String>> {
|
|
let Some(parent) = parent_workspace_id else {
|
|
return Ok(HashMap::new());
|
|
};
|
|
// Read the parent's rows on the non-RLS pool (like `workspace_is_fork`): the
|
|
// substitution must be complete regardless of the exporter's folder perms,
|
|
// otherwise a parent path the exporter can't read would fall back to the
|
|
// fork's own value and silently re-introduce the divergence we're fixing.
|
|
// No leak: only values for paths the fork already has (it's a clone) are used.
|
|
// SAFETY: `table_name` is a compile-time `TriggerCrud::TABLE_NAME` constant.
|
|
let rows: Vec<(String, String)> = sqlx::query_as(&format!(
|
|
"SELECT path, mode::text FROM {} WHERE workspace_id = $1",
|
|
table_name
|
|
))
|
|
.bind(parent)
|
|
.fetch_all(db)
|
|
.await?;
|
|
Ok(rows.into_iter().collect())
|
|
}
|
|
|
|
/// Build the `{ "mode": <parent value> }` override for a single trigger, or
|
|
/// `None` (keep the fork's own value) when the path is fork-only.
|
|
#[cfg(any(
|
|
feature = "http_trigger",
|
|
feature = "websocket",
|
|
feature = "postgres_trigger",
|
|
feature = "mqtt_trigger",
|
|
feature = "native_trigger",
|
|
all(
|
|
feature = "enterprise",
|
|
any(
|
|
feature = "kafka",
|
|
feature = "sqs_trigger",
|
|
feature = "gcp_trigger",
|
|
feature = "azure_trigger",
|
|
feature = "nats",
|
|
feature = "smtp",
|
|
),
|
|
feature = "private"
|
|
)
|
|
))]
|
|
fn trigger_mode_override(
|
|
parent_modes: &HashMap<String, String>,
|
|
path: &str,
|
|
) -> Option<serde_json::Map<String, Value>> {
|
|
parent_modes.get(path).map(|mode| {
|
|
let mut o = serde_json::Map::new();
|
|
o.insert("mode".to_string(), Value::String(mode.clone()));
|
|
o
|
|
})
|
|
}
|
|
|
|
/// Schedule analog of [`fork_parent_trigger_modes`]: maps schedule `path` →
|
|
/// parent `enabled`. Empty when not a fork.
|
|
async fn fork_parent_schedule_enabled(
|
|
db: &DB,
|
|
parent_workspace_id: Option<&str>,
|
|
) -> Result<HashMap<String, bool>> {
|
|
let Some(parent) = parent_workspace_id else {
|
|
return Ok(HashMap::new());
|
|
};
|
|
// Non-RLS pool, same rationale as `fork_parent_trigger_modes`.
|
|
let rows: Vec<(String, bool)> =
|
|
sqlx::query_as("SELECT path, enabled FROM schedule WHERE workspace_id = $1")
|
|
.bind(parent)
|
|
.fetch_all(db)
|
|
.await?;
|
|
Ok(rows.into_iter().collect())
|
|
}
|
|
|
|
enum ArchiveImpl {
|
|
#[cfg(feature = "zip")]
|
|
Zip(async_zip::tokio::write::ZipFileWriter<tokio::fs::File>),
|
|
Tar(tokio_tar::Builder<File>),
|
|
}
|
|
|
|
impl ArchiveImpl {
|
|
async fn write_to_archive(&mut self, content: &str, path: &str) -> Result<()> {
|
|
match self {
|
|
ArchiveImpl::Tar(t) => {
|
|
let bytes = content.as_bytes();
|
|
let mut header = tokio_tar::Header::new_gnu();
|
|
header.set_size(bytes.len() as u64);
|
|
header.set_mtime(0);
|
|
header.set_uid(0);
|
|
header.set_gid(0);
|
|
header.set_mode(0o777);
|
|
header.set_cksum();
|
|
t.append_data(&mut header, path, bytes).await?;
|
|
}
|
|
#[cfg(feature = "zip")]
|
|
ArchiveImpl::Zip(z) => {
|
|
let header =
|
|
async_zip::ZipEntryBuilder::new(path.into(), async_zip::Compression::Deflate)
|
|
.last_modification_date(Default::default())
|
|
.unix_permissions(0o777)
|
|
.build();
|
|
z.write_entry_whole(header, content.as_bytes())
|
|
.await
|
|
.map_err(to_anyhow)?;
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
async fn finish(self) -> Result<()> {
|
|
match self {
|
|
ArchiveImpl::Tar(t) => t.into_inner().await?,
|
|
#[cfg(feature = "zip")]
|
|
ArchiveImpl::Zip(z) => z.close().await.map_err(to_anyhow)?.into_inner(),
|
|
}
|
|
.sync_all()
|
|
.await?;
|
|
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
#[derive(Deserialize)]
|
|
pub(crate) struct ArchiveQueryParams {
|
|
archive_type: Option<String>,
|
|
plain_secret: Option<bool>,
|
|
plain_secrets: Option<bool>,
|
|
skip_secrets: Option<bool>,
|
|
skip_variables: Option<bool>,
|
|
skip_resources: Option<bool>,
|
|
skip_resource_types: Option<bool>,
|
|
include_schedules: Option<bool>,
|
|
include_triggers: Option<bool>,
|
|
include_users: Option<bool>,
|
|
include_groups: Option<bool>,
|
|
include_settings: Option<bool>,
|
|
include_key: Option<bool>,
|
|
include_workspace_dependencies: Option<bool>,
|
|
default_ts: Option<String>,
|
|
/// Settings format version: "v1" (default) returns legacy flat format, "v2" returns grouped format
|
|
settings_version: Option<String>,
|
|
/// Opt-in: include `extra_perms` on flow / script / app rows. Default `false`
|
|
/// so cross-workspace tarball imports do not carry over ACLs referring to
|
|
/// identities that may not exist in the target workspace. `wmill sync pull`
|
|
/// passes `true` to surface ACLs in the git-tracked yaml.
|
|
preserve_extra_perms: Option<bool>,
|
|
}
|
|
|
|
/// How to handle `extra_perms` in the serialized output.
|
|
///
|
|
/// * `Drop` — strip the field unconditionally (legacy behavior for
|
|
/// types that have never carried ACLs in source).
|
|
/// * `KeepEvenEmpty` — always keep the field, even when `{}`. Matches the
|
|
/// pre-existing serialization for folders and groups so
|
|
/// no customer sees a one-time noisy diff on upgrade.
|
|
/// * `KeepIfNonEmpty` — keep when there is at least one entry, drop when `{}`
|
|
/// or null. New surface for flow / script / app, which
|
|
/// never carried ACLs in source before this change.
|
|
#[derive(Clone, Copy)]
|
|
pub enum ExtraPermsBehavior {
|
|
Drop,
|
|
KeepEvenEmpty,
|
|
KeepIfNonEmpty,
|
|
}
|
|
|
|
#[inline]
|
|
pub fn to_string_without_metadata<T>(
|
|
value: &T,
|
|
extra_perms: ExtraPermsBehavior,
|
|
ignore_keys: Option<Vec<&str>>,
|
|
) -> Result<String>
|
|
where
|
|
T: ?Sized + Serialize,
|
|
{
|
|
to_string_without_metadata_inner(value, extra_perms, ignore_keys, None)
|
|
}
|
|
|
|
/// Like [`to_string_without_metadata`] but additionally lets the caller
|
|
/// override top-level keys after stripping. Used for fork trigger/schedule
|
|
/// exports, where `mode`/`enabled` is rewritten to the *parent* workspace's
|
|
/// value so the fork's synced file is byte-identical to the parent on those
|
|
/// fields — a clean 3-way git merge instead of a dropped line. See the write
|
|
/// half of the rule in `windmill-trigger::handler::workspace_is_fork`.
|
|
#[inline]
|
|
pub fn to_string_without_metadata_inner<T>(
|
|
value: &T,
|
|
extra_perms: ExtraPermsBehavior,
|
|
ignore_keys: Option<Vec<&str>>,
|
|
overrides: Option<&serde_json::Map<String, Value>>,
|
|
) -> Result<String>
|
|
where
|
|
T: ?Sized + Serialize,
|
|
{
|
|
let mut value = serde_json::to_value(value).map_err(to_anyhow)?;
|
|
value
|
|
.as_object_mut()
|
|
.map(|obj| {
|
|
let keys = [
|
|
vec![
|
|
"workspace_id",
|
|
"path",
|
|
"name",
|
|
"versions",
|
|
"id",
|
|
"created_at",
|
|
"updated_at",
|
|
"created_by",
|
|
"updated_by",
|
|
"edited_at",
|
|
"edited_by",
|
|
"permissioned_as",
|
|
"archived",
|
|
"error",
|
|
"last_server_ping",
|
|
"server_id",
|
|
"raw_app",
|
|
],
|
|
ignore_keys.unwrap_or(vec![]),
|
|
]
|
|
.concat();
|
|
|
|
for key in keys {
|
|
if obj.contains_key(key) {
|
|
obj.remove(key);
|
|
}
|
|
}
|
|
|
|
if let Some(o2) = obj.get_mut("policy").and_then(|x| x.as_object_mut()) {
|
|
o2.remove("on_behalf_of");
|
|
o2.remove("on_behalf_of_email");
|
|
}
|
|
if obj.contains_key("extra_perms") {
|
|
let is_empty_extra_perms = obj
|
|
.get("extra_perms")
|
|
.map(|v| v.as_object().is_some_and(|o| o.is_empty()) || v.is_null())
|
|
.unwrap_or(true);
|
|
let drop = match extra_perms {
|
|
ExtraPermsBehavior::Drop => true,
|
|
ExtraPermsBehavior::KeepEvenEmpty => false,
|
|
ExtraPermsBehavior::KeepIfNonEmpty => is_empty_extra_perms,
|
|
};
|
|
if drop {
|
|
obj.remove("extra_perms");
|
|
}
|
|
}
|
|
if obj
|
|
.get("default_permissioned_as")
|
|
.and_then(|v| v.as_array())
|
|
.is_some_and(|a| a.is_empty())
|
|
{
|
|
obj.remove("default_permissioned_as");
|
|
}
|
|
|
|
if let Some(overrides) = overrides {
|
|
for (k, v) in overrides {
|
|
obj.insert(k.clone(), v.clone());
|
|
}
|
|
}
|
|
|
|
serde_json::to_string_pretty(&obj).ok()
|
|
})
|
|
.flatten()
|
|
.ok_or_else(|| Error::BadRequest("Impossible to serialize value".to_string()))
|
|
}
|
|
|
|
#[derive(Serialize)]
|
|
struct SimplifiedUser {
|
|
username: String,
|
|
role: String,
|
|
disabled: bool,
|
|
email: String,
|
|
}
|
|
|
|
#[derive(Serialize)]
|
|
struct SimplifiedGroup {
|
|
name: String,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
summary: Option<String>,
|
|
members: Vec<String>,
|
|
admins: Vec<String>,
|
|
}
|
|
|
|
// V2 format: New grouped format
|
|
#[derive(Serialize)]
|
|
struct SimplifiedSettings {
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
auto_invite: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
webhook: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
deploy_to: Option<String>,
|
|
// Always serialize (including as `null`) so that `wmill sync pull` emits
|
|
// these fields in settings.yaml unconditionally. Makes round-trip
|
|
// bijective: YAML is the source of truth, absence/null = "clear remote",
|
|
// mirroring every other workspace setting.
|
|
error_handler: Option<Value>,
|
|
success_handler: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
ai_config: Option<serde_json::Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
large_file_storage: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
git_sync: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
default_app: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
default_scripts: Option<Value>,
|
|
name: String,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
mute_critical_alerts: Option<bool>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
color: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
operator_settings: Option<serde_json::Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
datatable: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
slack_team_id: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
slack_name: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
slack_command_script: Option<String>,
|
|
// Always serialize (see note above on error_handler / success_handler).
|
|
slack_oauth_client_id: Option<String>,
|
|
slack_oauth_client_secret: Option<String>,
|
|
}
|
|
|
|
// V1 format: Legacy flat format for backward compatibility (matches main branch exactly)
|
|
#[derive(Serialize)]
|
|
struct SimplifiedSettingsLegacy {
|
|
auto_invite_enabled: bool,
|
|
auto_invite_as: String,
|
|
auto_invite_mode: String,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
webhook: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
deploy_to: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
error_handler: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
error_handler_extra_args: Option<Value>,
|
|
error_handler_muted_on_cancel: bool,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
ai_config: Option<serde_json::Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
large_file_storage: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
git_sync: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
default_app: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
default_scripts: Option<Value>,
|
|
name: String,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
mute_critical_alerts: Option<bool>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
color: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
operator_settings: Option<serde_json::Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
datatable: Option<Value>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
slack_team_id: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
slack_name: Option<String>,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
slack_command_script: Option<String>,
|
|
}
|
|
|
|
// Internal struct for querying database
|
|
#[derive(sqlx::FromRow)]
|
|
struct SettingsRow {
|
|
auto_invite: Option<Value>,
|
|
webhook: Option<String>,
|
|
deploy_to: Option<String>,
|
|
error_handler: Option<Value>,
|
|
success_handler: Option<Value>,
|
|
ai_config: Option<serde_json::Value>,
|
|
large_file_storage: Option<Value>,
|
|
git_sync: Option<Value>,
|
|
default_app: Option<String>,
|
|
default_scripts: Option<Value>,
|
|
name: Option<String>,
|
|
mute_critical_alerts: Option<bool>,
|
|
color: Option<String>,
|
|
operator_settings: Option<serde_json::Value>,
|
|
datatable: Option<Value>,
|
|
slack_team_id: Option<String>,
|
|
slack_name: Option<String>,
|
|
slack_command_script: Option<String>,
|
|
slack_oauth_client_id: Option<String>,
|
|
slack_oauth_client_secret: Option<String>,
|
|
}
|
|
|
|
pub(crate) async fn tarball_workspace(
|
|
authed: ApiAuthed,
|
|
Extension(user_db): Extension<UserDB>,
|
|
Extension(db): Extension<DB>,
|
|
Path(w_id): Path<String>,
|
|
Query(ArchiveQueryParams {
|
|
archive_type,
|
|
plain_secret,
|
|
plain_secrets,
|
|
skip_resources,
|
|
skip_resource_types,
|
|
skip_secrets,
|
|
skip_variables,
|
|
include_schedules,
|
|
include_triggers,
|
|
include_users,
|
|
include_groups,
|
|
include_settings,
|
|
include_key,
|
|
include_workspace_dependencies,
|
|
default_ts,
|
|
settings_version,
|
|
preserve_extra_perms,
|
|
}): Query<ArchiveQueryParams>,
|
|
) -> Result<([(HeaderName, String); 2], impl IntoResponse)> {
|
|
tracing::info!(
|
|
"tarball_workspace called for workspace {}: include_workspace_dependencies={:?}, skip_variables={:?}, skip_resources={:?}",
|
|
w_id,
|
|
include_workspace_dependencies,
|
|
skip_variables,
|
|
skip_resources
|
|
);
|
|
|
|
// The route is gated by workspaces:read, but exporting DECRYPTED secrets is a
|
|
// variable-read capability beyond workspace metadata. Require variables:read
|
|
// only on the plaintext-secret path: ordinary tarball pulls (structure and
|
|
// encrypted-only values) keep working with workspaces:read, and the workspace
|
|
// key itself stays admin-only (include_key). No-op for unscoped tokens.
|
|
if plain_secret.or(plain_secrets).unwrap_or(false)
|
|
&& !skip_secrets.unwrap_or(false)
|
|
&& !skip_variables.unwrap_or(false)
|
|
{
|
|
check_scopes(&authed, || "variables:read".to_string())?;
|
|
}
|
|
|
|
// Opt-in behavior for surfacing per-resource ACLs on flow/app rows.
|
|
// Folder and group rows have always carried `extra_perms` in source and
|
|
// continue to do so unconditionally (`KeepEvenEmpty`) so existing
|
|
// customer git repos see no one-time noisy diff.
|
|
let new_kinds_extra_perms = if preserve_extra_perms.unwrap_or(false) {
|
|
ExtraPermsBehavior::KeepIfNonEmpty
|
|
} else {
|
|
ExtraPermsBehavior::Drop
|
|
};
|
|
|
|
// Resolve workspace dependencies on the pool *before* opening the RLS
|
|
// transaction: fetching them mid-transaction would hold a second
|
|
// simultaneous connection while `tx` is still checked out.
|
|
let workspace_dependencies = if include_workspace_dependencies.unwrap_or(false)
|
|
&& require_admin(authed.is_admin, &authed.username).is_ok()
|
|
{
|
|
Some(WorkspaceDependencies::list(&w_id, &db).await?)
|
|
} else {
|
|
None
|
|
};
|
|
|
|
let mut tx = user_db.begin(&authed).await?;
|
|
|
|
// Exporting decrypted secrets in bulk is the same capability as a per-item
|
|
// secret read, so record it for parity with variables.decrypt_secret.
|
|
if plain_secret.or(plain_secrets).unwrap_or(false)
|
|
&& !skip_variables.unwrap_or(false)
|
|
&& !skip_secrets.unwrap_or(false)
|
|
{
|
|
windmill_audit::audit_oss::audit_log(
|
|
&mut *tx,
|
|
&authed,
|
|
"variables.decrypt_secret",
|
|
windmill_audit::ActionKind::Execute,
|
|
&w_id,
|
|
Some("workspace_tarball_export"),
|
|
None,
|
|
)
|
|
.await?;
|
|
}
|
|
|
|
// Source-of-truth for fork-ness: the workspace's parent_workspace_id column.
|
|
// The wm-fork-* prefix is a creation-time naming convention that could in
|
|
// principle drift (rename, manual SQL); the column is the contract that
|
|
// matches what the conflict-warning gates read. The id is also the workspace
|
|
// whose trigger `mode` / schedule `enabled` a fork export defers to.
|
|
let parent_workspace_id: Option<String> = sqlx::query_scalar::<_, Option<String>>(
|
|
"SELECT parent_workspace_id FROM workspace WHERE id = $1",
|
|
)
|
|
.bind(&w_id)
|
|
.fetch_optional(&mut *tx)
|
|
.await?
|
|
.flatten();
|
|
|
|
let tmp_dir = TempDir::new_in(&*WINDMILL_DIR)?;
|
|
|
|
let name = match archive_type.as_deref() {
|
|
Some("tar") | None => Ok(format!("windmill-{w_id}.tar")),
|
|
Some("zip") => Ok(format!("windmill-{w_id}.zip")),
|
|
Some(t) => Err(Error::BadRequest(format!("Invalid Archive Type {t}"))),
|
|
}?;
|
|
let file_path = tmp_dir.path().join(&name);
|
|
let mut archive = match archive_type.as_deref() {
|
|
Some("tar") | None => {
|
|
let file = File::create(&file_path).await?;
|
|
Ok(ArchiveImpl::Tar(tokio_tar::Builder::new(file)))
|
|
}
|
|
#[cfg(feature = "zip")]
|
|
Some("zip") => {
|
|
let file = tokio::fs::File::create(&file_path).await?;
|
|
Ok(ArchiveImpl::Zip(
|
|
async_zip::tokio::write::ZipFileWriter::with_tokio(file),
|
|
))
|
|
}
|
|
Some(t) => Err(Error::BadRequest(format!("Invalid Archive Type {t}"))),
|
|
}?;
|
|
{
|
|
let folders = sqlx::query_as::<_, Folder>("SELECT name, workspace_id, display_name, owners, extra_perms, summary, edited_at, created_by, default_permissioned_as, labels FROM folder WHERE workspace_id = $1")
|
|
.bind(&w_id)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for folder in folders {
|
|
archive
|
|
.write_to_archive(
|
|
&to_string_without_metadata(&folder, ExtraPermsBehavior::KeepEvenEmpty, None)
|
|
.unwrap(),
|
|
&format!("f/{}/folder.meta.json", folder.name),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
{
|
|
let scripts = sqlx::query_as::<_, Script<ScriptRunnableSettingsHandle>>(&format!(
|
|
"SELECT {} FROM script as o WHERE workspace_id = $1 AND archived = false
|
|
AND created_at = (select max(created_at) from script where path = o.path AND \
|
|
workspace_id = $1)",
|
|
windmill_common::scripts::SCRIPT_COLUMNS,
|
|
))
|
|
.bind(&w_id)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for script in scripts {
|
|
let script = windmill_common::scripts::prefetch_cached_script(script, &db).await?;
|
|
let ext = match script.language {
|
|
ScriptLang::Python3 => "py",
|
|
ScriptLang::Deno => {
|
|
if default_ts.as_ref().is_some_and(|x| x == "bun") {
|
|
"deno.ts"
|
|
} else {
|
|
"ts"
|
|
}
|
|
}
|
|
ScriptLang::Go => "go",
|
|
ScriptLang::Bash => "sh",
|
|
ScriptLang::Powershell => "ps1",
|
|
ScriptLang::Postgresql => "pg.sql",
|
|
ScriptLang::Mysql => "my.sql",
|
|
ScriptLang::Bigquery => "bq.sql",
|
|
ScriptLang::Snowflake => "sf.sql",
|
|
ScriptLang::Mssql => "ms.sql",
|
|
ScriptLang::DuckDb => "duckdb.sql",
|
|
ScriptLang::Graphql => "gql",
|
|
ScriptLang::Nativets => "fetch.ts",
|
|
ScriptLang::Bun | ScriptLang::Bunnative => {
|
|
if default_ts.as_ref().is_some_and(|x| x == "bun") {
|
|
"ts"
|
|
} else {
|
|
"bun.ts"
|
|
}
|
|
}
|
|
ScriptLang::Php => "php",
|
|
ScriptLang::Rust => "rs",
|
|
ScriptLang::Ansible => "playbook.yml",
|
|
ScriptLang::CSharp => "cs",
|
|
ScriptLang::Nu => "nu",
|
|
ScriptLang::OracleDB => "odb.sql",
|
|
ScriptLang::Java => "java",
|
|
ScriptLang::Ruby => "rb",
|
|
ScriptLang::Rlang => "r",
|
|
// for related places search: ADD_NEW_LANG
|
|
};
|
|
archive
|
|
.write_to_archive(&script.content, &format!("{}.{}", script.path, ext))
|
|
.await?;
|
|
|
|
let metadata = ScriptMetadata {
|
|
summary: script.summary,
|
|
description: script.description,
|
|
schema: script.schema,
|
|
kind: script.kind.to_string(),
|
|
lock: script.lock,
|
|
envs: script.envs,
|
|
concurrency_settings: script.runnable_settings.concurrency_settings,
|
|
debouncing_settings: script.runnable_settings.debouncing_settings,
|
|
cache_ttl: script.cache_ttl,
|
|
dedicated_worker: script.dedicated_worker,
|
|
ws_error_handler_muted: script.ws_error_handler_muted,
|
|
priority: script.priority,
|
|
tag: script.tag,
|
|
timeout: script.timeout,
|
|
delete_after_secs: script.delete_after_secs,
|
|
restart_unless_cancelled: script.restart_unless_cancelled,
|
|
visible_to_runner_only: script.visible_to_runner_only,
|
|
auto_kind: script.auto_kind,
|
|
codebase: script.codebase,
|
|
has_preprocessor: script.has_preprocessor,
|
|
on_behalf_of_email: script.on_behalf_of_email,
|
|
modules: script.modules,
|
|
labels: script.labels,
|
|
// Same opt-in contract as flow/app: the tarball only surfaces
|
|
// ACLs when `?preserve_extra_perms=true`. Passing `Null` lets the
|
|
// `is_empty_extra_perms` skip-serializer drop the field entirely.
|
|
extra_perms: if matches!(new_kinds_extra_perms, ExtraPermsBehavior::KeepIfNonEmpty)
|
|
{
|
|
script.extra_perms
|
|
} else {
|
|
serde_json::Value::Null
|
|
},
|
|
};
|
|
let metadata_str = serde_json::to_string_pretty(&metadata).unwrap();
|
|
archive
|
|
.write_to_archive(&metadata_str, &format!("{}.script.json", script.path))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if !skip_resources.unwrap_or(false) {
|
|
let resources = sqlx::query_as!(
|
|
Resource,
|
|
"SELECT workspace_id, path, value, description, resource_type, extra_perms, created_by, edited_at, labels FROM resource WHERE workspace_id = $1 AND resource_type != 'state' AND resource_type != 'cache'",
|
|
&w_id
|
|
)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for resource in resources {
|
|
let resource_str =
|
|
&to_string_without_metadata(&resource, ExtraPermsBehavior::Drop, None).unwrap();
|
|
archive
|
|
.write_to_archive(&resource_str, &format!("{}.resource.json", resource.path))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if !skip_resource_types.unwrap_or(false) {
|
|
let resource_types = sqlx::query_as!(
|
|
ResourceType,
|
|
"SELECT workspace_id, name, schema, description, created_by, edited_at, format_extension, is_fileset FROM resource_type WHERE workspace_id = $1",
|
|
&w_id
|
|
)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for resource_type in resource_types {
|
|
let resource_str =
|
|
&to_string_without_metadata(&resource_type, ExtraPermsBehavior::Drop, None)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&resource_str,
|
|
&format!("{}.resource-type.json", resource_type.name),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
{
|
|
let flows = sqlx::query_as::<_, Flow>(
|
|
"SELECT flow.workspace_id, flow.path, flow.summary, flow.description, flow.archived, flow.extra_perms, flow.dedicated_worker, flow.tag, flow.ws_error_handler_muted, flow.timeout, flow.visible_to_runner_only, flow.on_behalf_of_email, flow.labels, flow_version.schema, flow_version.value, flow_version.created_at as edited_at, flow_version.created_by as edited_by
|
|
FROM flow
|
|
LEFT JOIN flow_version ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]
|
|
WHERE flow.workspace_id = $1 AND flow.archived = false",
|
|
)
|
|
.bind(&w_id)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for flow in flows {
|
|
let flow_str = &to_string_without_metadata(&flow, new_kinds_extra_perms, None).unwrap();
|
|
archive
|
|
.write_to_archive(&flow_str, &format!("{}.flow.json", flow.path))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if !skip_variables.unwrap_or(false) {
|
|
let variables =
|
|
sqlx::query_as::<_, ExportableListableVariable>(if !skip_secrets.unwrap_or(false) {
|
|
"SELECT workspace_id, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at, labels FROM variable WHERE workspace_id = $1 AND expires_at IS NULL"
|
|
} else {
|
|
"SELECT workspace_id, path, value, is_secret, description, extra_perms, account, is_oauth, expires_at, labels FROM variable WHERE workspace_id = $1 AND is_secret = false AND expires_at IS NULL"
|
|
})
|
|
.bind(&w_id)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
let mc = build_crypt(&db, &w_id).await?;
|
|
|
|
for mut var in variables {
|
|
if plain_secret.or(plain_secrets).unwrap_or(false)
|
|
&& var.value.is_some()
|
|
&& var.is_secret
|
|
{
|
|
var.value = Some(decrypt(&mc, var.value.unwrap()).map_err(|e| {
|
|
Error::internal_err(format!("Error decrypting variable {}: {}", var.path, e))
|
|
})?);
|
|
}
|
|
let var_str =
|
|
&to_string_without_metadata(&var, ExtraPermsBehavior::Drop, None).unwrap();
|
|
archive
|
|
.write_to_archive(&var_str, &format!("{}.variable.json", var.path))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
{
|
|
let apps = sqlx::query_as::<_, AppWithLastVersion>(
|
|
"SELECT app.id, app.path, app.summary, app.versions, app.policy, app.custom_path,
|
|
app.extra_perms, app_version.value,
|
|
app_version.created_at, app_version.created_by, app_version.raw_app, app.labels from app, app_version
|
|
WHERE app.workspace_id = $1 AND app_version.id = app.versions[array_upper(app.versions, 1)]",
|
|
)
|
|
.bind(&w_id)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for app in apps {
|
|
let app_str = &to_string_without_metadata(&app, new_kinds_extra_perms, None).unwrap();
|
|
let kind = if app.raw_app { "raw_app" } else { "app" };
|
|
archive
|
|
.write_to_archive(&app_str, &format!("{}.{}.json", app.path, kind))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if let Some(workspace_dependencies) = workspace_dependencies {
|
|
tracing::info!("Including workspace dependencies in tarball export");
|
|
tracing::info!(
|
|
"Found {} workspace dependencies",
|
|
workspace_dependencies.len()
|
|
);
|
|
for dep in workspace_dependencies {
|
|
// let dep_str = &to_string_without_metadata(&dep, ExtraPermsBehavior::Drop, None).unwrap();
|
|
let filename = WorkspaceDependencies::to_path(&dep.name, dep.language)?;
|
|
tracing::info!(
|
|
"Adding workspace dependency: name={:?}, language={:?}, filename={}",
|
|
dep.name,
|
|
dep.language,
|
|
filename
|
|
);
|
|
archive.write_to_archive(&dep.content, &filename).await?;
|
|
}
|
|
} else {
|
|
tracing::info!(
|
|
"Skipping workspace dependencies: include_workspace_dependencies={:?}",
|
|
include_workspace_dependencies
|
|
);
|
|
}
|
|
|
|
if include_schedules.unwrap_or(false) {
|
|
// Managed ducklake-maintenance schedules are excluded: they are
|
|
// derived from the workspace ducklake settings (and admins bypass the
|
|
// RLS that hides them), so exporting them would drag unsyncable rows
|
|
// into git.
|
|
let schedules = sqlx::query_as::<_, Schedule>(
|
|
"SELECT workspace_id, path, edited_by, edited_at, schedule, timezone, enabled, script_path, is_flow, args, extra_perms, email, permissioned_as, error, on_failure, on_failure_times, on_failure_exact, on_failure_extra_args, on_recovery, on_recovery_times, on_recovery_extra_args, on_success, on_success_extra_args, ws_error_handler_muted, retry, no_flow_overlap, summary, description, tag, paused_until, cron_version, dynamic_skip, labels FROM schedule
|
|
WHERE workspace_id = $1 AND NOT starts_with(path, $2)",
|
|
)
|
|
.bind(&w_id)
|
|
.bind(windmill_common::workspaces::DUCKLAKE_MAINTENANCE_PATH_PREFIX)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
// For a fork, defer each schedule's `enabled` to the parent so the
|
|
// synced file matches the parent and the merge doesn't flip it.
|
|
let parent_enabled =
|
|
fork_parent_schedule_enabled(&db, parent_workspace_id.as_deref()).await?;
|
|
for schedule in schedules {
|
|
let enabled_override = parent_enabled.get(&schedule.path).map(|enabled| {
|
|
let mut o = serde_json::Map::new();
|
|
o.insert("enabled".to_string(), Value::Bool(*enabled));
|
|
o
|
|
});
|
|
let app_str = &to_string_without_metadata_inner(
|
|
&schedule,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
enabled_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(&app_str, &format!("{}.schedule.json", schedule.path))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if include_triggers.unwrap_or(false) {
|
|
#[cfg(feature = "http_trigger")]
|
|
{
|
|
use crate::triggers::http::HttpTrigger;
|
|
let handler = HttpTrigger;
|
|
let http_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<HttpTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in http_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.http_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "websocket")]
|
|
{
|
|
use crate::triggers::websocket::WebsocketTrigger;
|
|
let handler = WebsocketTrigger;
|
|
let websocket_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<WebsocketTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in websocket_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.websocket_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(all(feature = "enterprise", feature = "kafka", feature = "private"))]
|
|
{
|
|
use crate::triggers::kafka::KafkaTrigger;
|
|
let handler = KafkaTrigger;
|
|
let kafka_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<KafkaTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in kafka_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.kafka_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(all(feature = "enterprise", feature = "sqs_trigger", feature = "private"))]
|
|
{
|
|
use crate::triggers::sqs::SqsTrigger;
|
|
let handler = SqsTrigger;
|
|
let sqs_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<SqsTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in sqs_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.sqs_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(all(feature = "enterprise", feature = "gcp_trigger", feature = "private"))]
|
|
{
|
|
use crate::triggers::gcp::GcpTrigger;
|
|
let handler = GcpTrigger;
|
|
let gcp_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<GcpTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in gcp_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.gcp_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(all(feature = "enterprise", feature = "azure_trigger", feature = "private"))]
|
|
{
|
|
use crate::triggers::azure::AzureTrigger;
|
|
let handler = AzureTrigger;
|
|
let azure_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<AzureTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in azure_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.azure_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(all(feature = "enterprise", feature = "nats", feature = "private"))]
|
|
{
|
|
use crate::triggers::nats::NatsTrigger;
|
|
let handler = NatsTrigger;
|
|
let nats_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<NatsTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in nats_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str: &String = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.nats_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "postgres_trigger")]
|
|
{
|
|
use crate::triggers::postgres::PostgresTrigger;
|
|
let handler = PostgresTrigger;
|
|
let postgres_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<PostgresTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in postgres_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.postgres_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "mqtt_trigger")]
|
|
{
|
|
use crate::triggers::mqtt::MqttTrigger;
|
|
let handler = MqttTrigger;
|
|
let mqtt_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<MqttTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in mqtt_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.mqtt_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(all(feature = "enterprise", feature = "smtp", feature = "private"))]
|
|
{
|
|
use crate::triggers::email::EmailTrigger;
|
|
let handler = EmailTrigger;
|
|
let email_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?;
|
|
let parent_modes = fork_parent_trigger_modes(
|
|
&db,
|
|
<EmailTrigger as TriggerCrud>::TABLE_NAME,
|
|
parent_workspace_id.as_deref(),
|
|
)
|
|
.await?;
|
|
|
|
for trigger in email_triggers {
|
|
let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path);
|
|
let trigger_str = &to_string_without_metadata_inner(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
mode_override.as_ref(),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!("{}.email_trigger.json", trigger.base.path),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "native_trigger")]
|
|
{
|
|
use crate::native_triggers::{list_native_triggers, ServiceName};
|
|
use strum::IntoEnumIterator;
|
|
|
|
for service_name in ServiceName::iter() {
|
|
let native_triggers =
|
|
list_native_triggers(&mut *tx, &w_id, service_name, None, None, None, None)
|
|
.await?;
|
|
|
|
// Native triggers (Nextcloud, Google Drive, GitHub) are never
|
|
// cloned into a fork — a fork only has one if its owner created
|
|
// it there, so it's always "fork-only" and keeps its own mode.
|
|
// No parent-value substitution applies; we only strip the
|
|
// webhook token hash.
|
|
let native_ignore_keys = vec!["webhook_token_hash"];
|
|
|
|
for trigger in native_triggers {
|
|
let trigger_str = &to_string_without_metadata(
|
|
&trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
Some(native_ignore_keys.clone()),
|
|
)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(
|
|
&trigger_str,
|
|
&format!(
|
|
"{}.{}.{}.{}_native_trigger.json",
|
|
trigger.script_path,
|
|
if trigger.is_flow { "flow" } else { "script" },
|
|
trigger.external_id,
|
|
service_name.as_str()
|
|
),
|
|
)
|
|
.await?;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
if include_users.unwrap_or(false) {
|
|
let users = sqlx::query!(
|
|
"SELECT workspace_id, username, email, is_admin, created_at, operator, disabled, role, added_via FROM usr
|
|
WHERE workspace_id = $1",
|
|
&w_id
|
|
)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for user in users {
|
|
let user = SimplifiedUser {
|
|
username: user.username,
|
|
role: if user.is_admin {
|
|
"admin".to_string()
|
|
} else if user.operator {
|
|
"operator".to_string()
|
|
} else {
|
|
"developer".to_string()
|
|
},
|
|
disabled: user.disabled,
|
|
email: user.email,
|
|
};
|
|
let user_str =
|
|
&to_string_without_metadata(&user, ExtraPermsBehavior::Drop, Some(vec!["email"]))
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(&user_str, &format!("users/{}.user.json", user.email))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if include_groups.unwrap_or(false) {
|
|
let groups = sqlx::query!(
|
|
r#"SELECT g_.workspace_id, name, summary, extra_perms, array_agg(u2g.usr) filter (where u2g.usr is not null) as members
|
|
FROM usr u
|
|
JOIN usr_to_group u2g ON u2g.usr = u.username AND u2g.workspace_id = u.workspace_id
|
|
RIGHT JOIN group_ g_ ON g_.workspace_id = u.workspace_id AND g_.name = u2g.group_
|
|
WHERE g_.workspace_id = $1 AND g_.name != 'all'
|
|
GROUP BY g_.workspace_id, name, summary, extra_perms"#,
|
|
&w_id
|
|
)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
|
|
for group in groups {
|
|
let extra_perms: HashMap<String, bool> = serde_json::from_value(group.extra_perms)
|
|
.map_err(|e| {
|
|
Error::internal_err(format!(
|
|
"Error parsing extra_perms for group {}: {}",
|
|
group.name, e
|
|
))
|
|
})?;
|
|
tracing::info!("{:?}", extra_perms);
|
|
let members = group.members.unwrap_or(vec![]);
|
|
let admins: Vec<String> = extra_perms
|
|
.iter()
|
|
.filter_map(|(k, v)| {
|
|
// only consider extra_perms that concern actual members of the group
|
|
if members.contains(&k[2..].to_string()) && *v {
|
|
Some(k.clone())
|
|
} else {
|
|
None
|
|
}
|
|
})
|
|
.sorted()
|
|
.collect();
|
|
let group = SimplifiedGroup {
|
|
name: group.name,
|
|
summary: group.summary,
|
|
members: members
|
|
.iter()
|
|
.filter_map(|x| {
|
|
// remove members that are also admins as they are already in the admins list
|
|
let full_name = format!("u/{}", x);
|
|
if !admins.contains(&full_name) {
|
|
Some(full_name)
|
|
} else {
|
|
None
|
|
}
|
|
})
|
|
.collect(),
|
|
admins,
|
|
};
|
|
|
|
let group_str =
|
|
&to_string_without_metadata(&group, ExtraPermsBehavior::KeepEvenEmpty, None)
|
|
.unwrap();
|
|
archive
|
|
.write_to_archive(&group_str, &format!("groups/{}.group.json", group.name))
|
|
.await?;
|
|
}
|
|
}
|
|
|
|
if include_settings.unwrap_or(false) {
|
|
let row = sqlx::query_as::<_, SettingsRow>(
|
|
r#"SELECT
|
|
auto_invite,
|
|
webhook,
|
|
deploy_to,
|
|
error_handler,
|
|
success_handler,
|
|
ai_config,
|
|
large_file_storage,
|
|
git_sync,
|
|
default_app,
|
|
default_scripts,
|
|
workspace.name as name,
|
|
mute_critical_alerts,
|
|
color,
|
|
operator_settings,
|
|
datatable,
|
|
slack_team_id,
|
|
slack_name,
|
|
slack_command_script,
|
|
slack_oauth_client_id,
|
|
slack_oauth_client_secret
|
|
FROM workspace_settings
|
|
LEFT JOIN workspace ON workspace.id = workspace_settings.workspace_id
|
|
WHERE workspace_id = $1"#,
|
|
)
|
|
.bind(&w_id)
|
|
.fetch_one(&mut *tx)
|
|
.await?;
|
|
|
|
// Use v2 format only if explicitly requested, otherwise use v1 (legacy) for backward compatibility
|
|
// Server-owned auto-pull state (the HMAC webhook secret + hook id/error and
|
|
// the synced-sha / last-pull status) must never leave the server: keep it out
|
|
// of export archives and synced repos, and don't let a re-imported workspace
|
|
// inherit another install's hook/sync state. Mirrors the GET-settings redaction.
|
|
fn redact_git_sync_for_export(git_sync: Option<Value>) -> Option<Value> {
|
|
let mut git_sync = git_sync?;
|
|
if let Some(repos) = git_sync
|
|
.get_mut("repositories")
|
|
.and_then(|r| r.as_array_mut())
|
|
{
|
|
for repo in repos {
|
|
if let Some(auto_pull) =
|
|
repo.get_mut("auto_pull").and_then(|a| a.as_object_mut())
|
|
{
|
|
for field in [
|
|
"webhook_secret",
|
|
"webhook_id",
|
|
"webhook_error",
|
|
"last_synced_sha",
|
|
"last_pull_status",
|
|
] {
|
|
auto_pull.remove(field);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
Some(git_sync)
|
|
}
|
|
|
|
let settings_str = if settings_version.as_deref() == Some("v2") {
|
|
let settings = SimplifiedSettings {
|
|
auto_invite: row.auto_invite,
|
|
webhook: row.webhook,
|
|
deploy_to: row.deploy_to,
|
|
error_handler: row.error_handler,
|
|
success_handler: row.success_handler,
|
|
ai_config: row.ai_config,
|
|
large_file_storage: row.large_file_storage,
|
|
git_sync: redact_git_sync_for_export(row.git_sync),
|
|
default_app: row.default_app,
|
|
default_scripts: row.default_scripts,
|
|
name: row.name.clone().unwrap_or_default(),
|
|
mute_critical_alerts: row.mute_critical_alerts,
|
|
color: row.color.clone(),
|
|
operator_settings: row.operator_settings.clone(),
|
|
datatable: row.datatable.clone(),
|
|
slack_team_id: row.slack_team_id.clone(),
|
|
slack_name: row.slack_name.clone(),
|
|
slack_command_script: row.slack_command_script.clone(),
|
|
slack_oauth_client_id: row.slack_oauth_client_id.clone(),
|
|
// Mirror the non-admin redaction in `get_settings`: the OAuth
|
|
// client secret is admin-only and must not leak via tarball.
|
|
slack_oauth_client_secret: if authed.is_admin {
|
|
row.slack_oauth_client_secret.clone()
|
|
} else {
|
|
None
|
|
},
|
|
};
|
|
serde_json::to_value(settings)
|
|
.map(|v| serde_json::to_string_pretty(&v).ok())
|
|
.ok()
|
|
.flatten()
|
|
} else {
|
|
// V1 (legacy) format: convert JSONB to flat fields (matches main branch exactly)
|
|
let (auto_invite_enabled, auto_invite_as, auto_invite_mode) =
|
|
if let Some(ref ai) = row.auto_invite {
|
|
let enabled = ai.get("enabled").and_then(|v| v.as_bool()).unwrap_or(false);
|
|
let operator = ai
|
|
.get("operator")
|
|
.and_then(|v| v.as_bool())
|
|
.unwrap_or(false);
|
|
let mode = ai.get("mode").and_then(|v| v.as_str()).unwrap_or("invite");
|
|
(
|
|
enabled,
|
|
if operator {
|
|
"operator".to_string()
|
|
} else {
|
|
"developer".to_string()
|
|
},
|
|
mode.to_string(),
|
|
)
|
|
} else {
|
|
(false, "developer".to_string(), "invite".to_string())
|
|
};
|
|
|
|
let (error_handler, error_handler_extra_args, error_handler_muted_on_cancel) =
|
|
if let Some(ref eh) = row.error_handler {
|
|
let path = eh.get("path").and_then(|v| v.as_str()).map(String::from);
|
|
let extra_args = eh.get("extra_args").cloned();
|
|
let muted_on_cancel = eh
|
|
.get("muted_on_cancel")
|
|
.and_then(|v| v.as_bool())
|
|
.unwrap_or(false);
|
|
(path, extra_args, muted_on_cancel)
|
|
} else {
|
|
(None, None, false)
|
|
};
|
|
|
|
let settings = SimplifiedSettingsLegacy {
|
|
auto_invite_enabled,
|
|
auto_invite_as,
|
|
auto_invite_mode,
|
|
webhook: row.webhook,
|
|
deploy_to: row.deploy_to,
|
|
error_handler,
|
|
error_handler_extra_args,
|
|
error_handler_muted_on_cancel,
|
|
ai_config: row.ai_config,
|
|
large_file_storage: row.large_file_storage,
|
|
git_sync: redact_git_sync_for_export(row.git_sync),
|
|
default_app: row.default_app,
|
|
default_scripts: row.default_scripts,
|
|
name: row.name.unwrap_or_default(),
|
|
mute_critical_alerts: row.mute_critical_alerts,
|
|
color: row.color,
|
|
operator_settings: row.operator_settings,
|
|
datatable: row.datatable,
|
|
slack_team_id: row.slack_team_id,
|
|
slack_name: row.slack_name,
|
|
slack_command_script: row.slack_command_script,
|
|
};
|
|
serde_json::to_value(settings)
|
|
.map(|v| serde_json::to_string_pretty(&v).ok())
|
|
.ok()
|
|
.flatten()
|
|
}
|
|
.ok_or_else(|| Error::internal_err("Error serializing settings".to_string()))?;
|
|
|
|
archive
|
|
.write_to_archive(&settings_str, "settings.json")
|
|
.await?;
|
|
}
|
|
|
|
if include_key.unwrap_or(false) {
|
|
require_admin(authed.is_admin, &authed.username)?;
|
|
|
|
let key = sqlx::query_scalar!(
|
|
"SELECT key FROM workspace_key WHERE workspace_id = $1",
|
|
&w_id
|
|
)
|
|
.fetch_one(&mut *tx)
|
|
.await?;
|
|
|
|
let key_json = serde_json::to_value(key)
|
|
.map(|v| serde_json::to_string_pretty(&v).ok())
|
|
.ok()
|
|
.flatten()
|
|
.ok_or_else(|| Error::internal_err("Error serializing enryption key".to_string()))?;
|
|
archive
|
|
.write_to_archive(&key_json, "encryption_key.json")
|
|
.await?;
|
|
}
|
|
|
|
{
|
|
// Data table migrations live in the `datatable_migrations` table; surface
|
|
// them in the export as `migrations/datatable/<datatable>/<version>_<name>`
|
|
// .up.sql (and .down.sql when present) so `wmill sync` treats them like any
|
|
// other workspace item.
|
|
let migrations = sqlx::query!(
|
|
"SELECT datatable, timestamp, name, code_up, code_down FROM datatable_migrations \
|
|
WHERE workspace_id = $1 ORDER BY datatable, timestamp",
|
|
&w_id
|
|
)
|
|
.fetch_all(&mut *tx)
|
|
.await?;
|
|
for m in migrations {
|
|
let base = format!(
|
|
"migrations/datatable/{}/{}_{}",
|
|
m.datatable, m.timestamp, m.name
|
|
);
|
|
archive
|
|
.write_to_archive(&m.code_up, &format!("{base}.up.sql"))
|
|
.await?;
|
|
if let Some(code_down) = m.code_down {
|
|
archive
|
|
.write_to_archive(&code_down, &format!("{base}.down.sql"))
|
|
.await?;
|
|
}
|
|
}
|
|
}
|
|
|
|
archive.finish().await?;
|
|
|
|
let file = tokio::fs::File::open(&file_path).await?;
|
|
|
|
let stream = ReaderStream::new(file);
|
|
let body = axum::body::Body::from_stream(stream);
|
|
|
|
let headers = [
|
|
(header::CONTENT_TYPE, "application/x-tar".to_string()),
|
|
(
|
|
header::CONTENT_DISPOSITION,
|
|
format!("attachment; filename=\"{name}\""),
|
|
),
|
|
];
|
|
Ok((headers, body))
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod fork_export_tests {
|
|
use super::*;
|
|
use serde_json::json;
|
|
|
|
/// A fork export rewrites `mode` to the parent's value: the serialized file
|
|
/// carries the parent's state (`enabled`), not the fork's clone-disabled DB
|
|
/// value — so a normal-git merge sees no change on that line.
|
|
#[test]
|
|
fn override_substitutes_parent_mode() {
|
|
let fork_trigger = json!({
|
|
"path": "f/triggers/x",
|
|
"script_path": "f/scripts/x",
|
|
"mode": "disabled", // fork's local (clone-disabled) state
|
|
"is_flow": false,
|
|
});
|
|
let mut overrides = serde_json::Map::new();
|
|
overrides.insert("mode".to_string(), Value::String("enabled".to_string()));
|
|
|
|
let out = to_string_without_metadata_inner(
|
|
&fork_trigger,
|
|
ExtraPermsBehavior::Drop,
|
|
None,
|
|
Some(&overrides),
|
|
)
|
|
.unwrap();
|
|
let parsed: Value = serde_json::from_str(&out).unwrap();
|
|
|
|
assert_eq!(parsed["mode"], json!("enabled"), "parent mode substituted");
|
|
// `path` is in the metadata strip list, so it should be removed.
|
|
assert!(parsed.get("path").is_none());
|
|
}
|
|
|
|
/// A fork-only trigger (no parent counterpart, so no override) keeps the
|
|
/// fork creator's chosen state.
|
|
#[test]
|
|
fn no_override_keeps_fork_value() {
|
|
let fork_only = json!({ "mode": "enabled", "script_path": "f/scripts/x" });
|
|
let out =
|
|
to_string_without_metadata_inner(&fork_only, ExtraPermsBehavior::Drop, None, None)
|
|
.unwrap();
|
|
let parsed: Value = serde_json::from_str(&out).unwrap();
|
|
assert_eq!(parsed["mode"], json!("enabled"));
|
|
}
|
|
|
|
/// `trigger_mode_override` builds an override only when the parent has the
|
|
/// path; fork-only paths return `None` (keep the fork's own value).
|
|
#[cfg(feature = "http_trigger")]
|
|
#[test]
|
|
fn trigger_mode_override_defers_to_parent_or_self() {
|
|
let mut parent_modes = HashMap::new();
|
|
parent_modes.insert("f/triggers/shared".to_string(), "enabled".to_string());
|
|
|
|
let shared = trigger_mode_override(&parent_modes, "f/triggers/shared");
|
|
assert_eq!(
|
|
shared.as_ref().and_then(|o| o.get("mode")),
|
|
Some(&Value::String("enabled".to_string())),
|
|
);
|
|
|
|
// Fork-only path: no parent entry → no override → keep fork's own value.
|
|
assert!(trigger_mode_override(&parent_modes, "f/triggers/fork_only").is_none());
|
|
}
|
|
}
|