feat(git-sync): gate GitHub PRs on Windmill CI test results (WIN-2051) (#10096)

* docs: add design doc for automatic git-to-windmill pull sync

* docs: add migration plan and implementation phases to git-sync pull design

* feat(git-sync): add auto_pull settings schema and pull enqueue primitive

Adds AutoPullSettings/AutoPullMode/AutoPullStatus on GitRepositorySettings
(workspace_settings.git_sync JSONB), the GIT_SYNC_PULL_SCRIPT_PATH constant,
and should_pull/effective_poll_interval_s helpers with unit tests. Exports the
EE enqueue_git_pull_job primitive. Foundation for repo→Windmill auto-pull.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): poll repos and auto-pull new commits into the workspace

Phase 1 of automatic repo → Windmill sync. A monitor task (EE-licensed,
single-replica via advisory lock) git ls-remotes each auto-pull-enabled
repository ~every minute and enqueues a pull when the tracked branch moves,
reusing the {workspace_id}:git_sync concurrency key so pulls serialize with
in-flight push commits.

- windmill-store: background (no-authed) resolver get_git_repo_head_for_autopull
  that resolves the repo resource (incl. $var: refs) and ls-remotes; GitHub-App
  repos are skipped here and will sync via webhooks (phase 2).
- monitor.rs: poll/reconcile/persist with optimistic sha advance and failure
  status; targeted jsonb update so concurrent settings edits aren't clobbered.
- edit_git_sync_repository: preserve server-owned auto_pull state on UI save.
- openapi: AutoPullSettings/AutoPullMode/AutoPullStatus + auto_pull field.
- frontend: per-repo "Automatically deploy changes from Git" toggle with last
  sync status; demote the GitHub Actions link to an advanced CI option.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): wire webhook lifecycle + receiver; share reconcile logic

OSS side of phase 2 auto-pull webhooks:
- edit_git_sync_repository creates/removes the repo webhook on save (EE-gated,
  best-effort → falls back to polling).
- monitor poller now delegates to the shared windmill_git_sync reconcile/persist
  helpers (also used by the webhook receiver), removing duplicated logic.
- export the shared reconcile/persist/failure helpers; bump EE ref.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for phase 3 in-app PR creation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): show webhook vs polling status on the auto-pull toggle

When a repo has an active webhook (auto_pull.webhook_id set), the status line
reads "instant via webhook"; otherwise it reads the ~1-minute polling cadence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): post PR diff check on dry-run completion (phase 4)

Worker completion hook in process_completed_job: when a DeploymentCallback job
carrying the __git_sync_pr_check marker finishes, parse the dry-run SyncResponse
and patch the GitHub check run with the diff summary (success/neutral/failure).
Export enqueue_git_pull_dry_run; bump EE ref.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref (drop unused GHES webhook_secret)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* revert(git-sync): defer phase 4 PR diff checks (OSS side)

Remove the worker completion hook that posted the PR check run, drop the
enqueue_git_pull_dry_run re-export and the orphaned sqlx cache, bump EE ref.
Phases 1-3 (polling, webhooks, in-app PR creation) are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Revert "revert(git-sync): defer phase 4 PR diff checks (OSS side)"

This reverts commit 0137d3ca48.

* chore(git-sync): point EE ref at restored phase 4 commit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for clone_ref dry-run

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump init-repository hub script to v28784

Picks up the clone_ref param (windmill-integrations#158) so the phase 4 PR-check
dry-run can clone the PR head. Backward compatible; manual pull/push and the
automated pull/poller/webhook all move to the same published version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for auto-pull admin-permissioning fix

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for superadmin pull fallback

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(git-sync): refresh auto-pull tooltip; bump EE ref for webhook secret encryption

The auto-pull toggle tooltip claimed GitHub App repos would sync via
webhooks "in a future update"; webhook delivery now works, so describe
the webhook-vs-polling behavior accurately. Bump the EE ref to pick up
encrypting the webhook HMAC secret at rest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(git-sync): poll app-backed repos in auto/polling mode

The auto-pull poller skipped app-backed repos (the ls-remote head check
can't authenticate a tokenless URL), so auto- and polling-mode app repos
never synced when their webhook wasn't live. Wire the poller to fetch the
head via the GitHub API for app repos and reconcile. Bump the EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(git-sync): auto-pull UI — direction split, delivery mode, fallback notice

Reorganize the repository card into two clearly labeled directions:
"Push to Git on deploy (Windmill → Git)" and "Pull from Git (Git →
Windmill)". In the pull section:
- new connections default to auto-pull enabled (webhook with polling
  fallback); existing repos load with auto-pull off and are unchanged
- a Delivery selector chooses "Webhook with polling fallback" or
  "Polling only (air-gapped)"
- a notice surfaces webhook_error when delivery falls back to polling
- a reminder to remove any pre-existing GitHub Action that pushed into
  Windmill, to avoid conflicting double-syncs

Adds the webhook_error field to AutoPullSettings (+ openapi) and bumps
the EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(git-sync): clearer push indicator + gate webhook delivery to app repos

- Push-on-deploy is shown with a check icon + concise line (via the
  shared GitSyncModeDisplay, restyled from the oversized "Sync:" text);
  the setup wizard reuses it without the check (pre-save preview).
- The delivery-mode selector only shows for GitHub App-backed repos;
  token-based repos show a "webhooks require the GitHub App (managed or
  GHES)" note with a docs link and poll instead. Bumps the EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(git-sync): fork auto-sync (phase 5) + live deploy check (phase 6)

Phase 5 — fork auto-sync configured at the parent (replaces the *-to-forks
GitHub Actions):
- Add fork_open_prs + fork_pull_sync to GitRepositorySettings (openapi + UI).
- UI: two "Forks of this workspace" toggles in the repo card, gated on
  app-backed and not-a-fork; serialize the flags on save.
- On fork creation, strip the inherited auto_pull block (and fork_* flags) from
  the copied git_sync repo: a fork must not carry the parent's webhook id (it
  would delete the parent's hook on disable) or self-poll on top of the parent's
  fan-out. Push-direction config + installation are still inherited unchanged.

Phase 6 — live deploy status check on the commit (Cloudflare-style): an
in-progress "Windmill" check on the head commit that flips to "Deployed N
changes"; completion handled by the generalized git-sync check hook.

Bump EE ref for the phase 5-6 EE implementation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(git-sync): bump EE ref for PAT auto-pull mode normalization

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): address PR review findings

- webhook_secret: redact from the settings API response and Debug output (still
  persisted encrypted); it's a server-only HMAC key the UI never needs.
- poller: honor each repo's effective poll interval (relaxed ~10 min when a
  webhook is live) instead of probing every ~60s tick.
- settings save: roll back a just-created webhook if the settings transaction
  doesn't commit, so a failed save can't orphan a hook.
- auto-pull head check: fail SSH remotes with an actionable message (background
  polling has no SSH identity) instead of a confusing ls-remote error.
- deploy/PR check summary: a pull result carrying neither changes nor a settings
  diff now falls back to the unsummarized path instead of a false "in sync".
- UI: reset isGithubApp on resource change / failed fetch so webhook + fork
  controls can't show for the wrong repo.
- tests: cover parse_git_sync_changes and format_change_list edge cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): correct feature gating for OSS builds

- monitor.rs: keep the AUTO_PULL_LAST_POLL static, slack const, and
  poll_git_auto_pull_inner all behind #[cfg(feature = "private")] (an inserted
  static had split the cfg off the function, ungating it in OSS builds).
- edit_git_sync_repository: the webhook create/rollback block references
  windmill_common::git_sync_ee (private module), so gate it on
  all(enterprise, private) instead of enterprise only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(sqlx): cache workspace_diff query pulled in from origin/main

Re-merged origin/main (advanced past the earlier merge); regenerate the offline
sqlx entry for the new workspace_comparison test query so SQLX_OFFLINE builds
(cargo_test) pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): address Codex review findings (webhook cleanup on delete)

- Deleting a git-sync repository now tears down its managed GitHub webhook
  (deletion bypassed the sync_repo_webhook lifecycle, orphaning the hook so
  GitHub kept delivering to the instance).
- Worker completion hook rolls back the optimistic auto-pull sha on job failure
  (OSS side of the EE change) + caches the new marker query. Bump EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): delete repo webhook after the removal commits

Codex re-review nits:
- delete_git_sync_repository deleted the webhook before the settings transaction
  committed; a failed save would then leave the repo pointing at a hook that no
  longer exists (sync_repo_webhook treats a set webhook_id as live and won't
  recreate it). Capture the hook id, commit the DB removal, then delete the hook.
- Reword a fork-copy comment to drop drafting-history wording per AGENTS.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): reconcile the edit-path webhook after the settings commit

Codex nit: edit_git_sync_repository ran sync_repo_webhook before the transaction
committed. The rollback only covered created hooks, but sync_repo_webhook also
deletes a hook on disable/switch-to-polling — a commit failure then left the DB
with a webhook_id whose hook was already gone (and it wouldn't be recreated).
Save + commit first, then reconcile the webhook against the durable config and
persist any hook id/secret change (best-effort). Bump EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): preserve webhook secret on whole-config save + default on visible add

Codex nits:
- edit_git_sync_config saved the client config verbatim, so the webhook_secret
  redacted from the GET response would be dropped (breaking delivery). Preserve
  server-owned auto-pull state (webhook id/secret, synced sha, last status) per
  repo from the existing settings, matching edit_git_sync_repository.
- addSyncRepository (the visible add path) didn't set the auto_pull default, so
  new sync repos added from the UI came up with auto-deploy off. Match
  addRepository's default (webhook + polling fallback).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* refactor(git-sync): drop fork_pull_sync (parent-level keep-forks-in-sync)

Removes the "Keep forks in sync with the tracked branch" toggle and its
fan-out. Pulling the tracked branch straight into every fork was the
inconsistent piece; the consistent model is per-fork branch sync (each
fork tracks its own wm-fork/** branch), which is a separate follow-up.
fork_open_prs is kept. Also tightens the fork toggle-section spacing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): detect dev workspaces in CLI fork branch derivation

isForkWorkspace / computeGitSyncDeployBranch keyed off the wm-fork- id
prefix. Dev workspaces are forks with a custom, prefix-less id, so their
wm-fork/** branch was never derived or created. Detect them via
parent_workspace_id too (which the backend already passes), mirroring the
backend's `parent.is_some() || wm-fork- prefix` rule.

Pairs with the hub-script clone-flag fix (windmill-integrations#163); both
take effect once the CLI is released and the pinned version is bumped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): reconcile webhooks on full-config save

edit_git_sync_config preserved server-owned webhook fields but never
created or deleted the managed GitHub webhook, so enabling auto-pull
through the whole-config endpoint only polled, and disabling or removing
a repo left an orphan hook still delivering. Mirror the per-repository
endpoint: after the commit is durable, reconcile every saved repo's
webhook (sync_repo_webhook) and delete the hooks of repos the save
removed, including the clear-whole-config case. Addresses the Codex nit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): address Codex nits (webhook orphan on cleared auto_pull, fork detection)

- edit_git_sync_config: also delete a repo's old webhook when the save drops
  the repo OR clears its auto_pull. Webhook fields are only preserved onto a
  Some auto_pull, so a save that present-but-clears a repo would otherwise
  orphan its hook.
- GitSyncRepositoryCard: isFork now uses parent_workspace_id OR the wm-fork-
  prefix (was AND), matching the backend/CLI rule, so prefix-less dev
  workspaces are detected as forks and don't show the parent fork-PR toggle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* docs(git-sync): update design doc for the dropped fork_pull_sync

Phase 5 documented "Keep forks in sync with the tracked branch"
(fork_pull_sync) and its fan-out as implemented; that feature was removed.
Rewrite the section to reflect what ships (fork_open_prs), note the drop +
the per-fork-branch follow-up, and remove the stale fan-out mentions
elsewhere. Addresses the Codex nit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): redact webhook secrets from workspace export; fix doc endpoints

- Export (P1): strip the server-owned auto_pull state (webhook secret/id/error
  + synced sha + last pull status) from git_sync before it is written into an
  export's settings.json for both settings formats. The HMAC webhook secret
  must never leave the server (matching the GET-settings redaction), and a
  re-imported workspace must not inherit another install's hook/sync state.
- Docs: the webhook receiver is a single per-workspace endpoint
  /api/w/{workspace}/github_app/webhook (host-aware for managed + self-managed);
  update the stale push_webhook/{id} and instance-global /api/github_app/webhook
  references.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): skip deleted/archived workspaces in the auto-pull poller

The poller scanned workspace_settings directly, so an archived (soft-deleted)
or renamed-away workspace — whose settings row persists — kept polling and
could enqueue a pull into a dead workspace. Join workspace and require
NOT deleted. The EE webhook receiver gets the same filter (ee ref bumped).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): never trust client-supplied server-owned auto-pull fields

Both write endpoints (edit_git_sync_repository, edit_git_sync_config)
persisted caller-supplied auto_pull.webhook_id / webhook_secret /
webhook_error / last_synced_sha / last_pull_status when adding a repo or
newly enabling auto-pull, letting a client inject a webhook id/secret or
fake sync state. Strip those server-owned fields from the request up front;
existing repos re-derive them from the DB (carried over), new ones start
clean and the server (re)creates the webhook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): merge webhook fields post-commit instead of clobbering the row

The post-commit webhook reconcile in edit_git_sync_repository and
edit_git_sync_config wrote the whole pre-reconcile git_sync snapshot back
after the main save committed. A concurrent git-sync edit or poller status
write that landed in the gap could then be dropped by the stale snapshot.
Re-read the current row and merge only the reconciled webhook id/secret/error
for the repos the reconcile actually changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): parent-managed fork sync + PR-on-deploy toggles

Fork sync (push-on-merge-to-forks parity): a parent-level
auto_pull.sync_forks toggle routes changes on each fork's wm-fork/** branch
into that fork workspace, via the parent's existing webhook and one extra
fork-heads listing per poll tick (git ls-remote pattern for token repos,
git/matching-refs for app-backed). Fork state is a server-written
status-only auto_pull blob on the fork's own repo entry; the fork's card
shows a read-only "managed in the parent workspace" line with its branch
and last pull status. Dev workspaces (prefix-less ids) use the same branch
parsing (unit-tested in windmill-common).

PR-on-deploy: opening PRs for Windmill-pushed branches moves into the
deploy pipeline, per repo toggle (promotion_open_prs on the promotion
repo; parent-level fork_open_prs for fork deploys). The push job carries a
marker and the job-completion hook derives the pushed branch (helper
unit-tested against the CLI formula) and opens the PR outbound, so it
works without inbound webhooks; the webhook-side wm_deploy PR arm is
removed. The documented open-pr-* GitHub Actions remain valid alternatives
(PR creation is idempotent).

Fork guards: promotion mode, enabled auto-pull, and fork_open_prs are
rejected on fork workspaces (they are parent-managed; a fork's deploys
always target its wm-fork/** branch) and the promotion card is hidden in a
fork's settings. Enabling auto-pull now also requires EE, and the
post-commit webhook reconcile persists the normalized delivery mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): dev workspaces sync with their environment-label branch

A dev workspace's git branch is its environment label verbatim (dev/
staging, default dev) — a first-class env branch like the documented
push-on-merge-staging layout — instead of the wm-fork/** form. The label
rides the deploy job args (backend → hub script → CLI
--dev-workspace-label), the PR completion hook derives the same head, the
webhook/poller route label branches into the matching dev-workspace child
(poller lists them alongside wm-fork/* via extra ls-remote refs / per-label
API lookups), and manual pulls from the UI pass clone_ref accordingly. The
CLI refuses to deploy when the label branch equals the checked-out tracked
branch, which would otherwise commit fork content straight to it.

Because the branch is keyed on the label, the label is now immutable after
creation: set at create/attach only, the set_dev_workspace_label endpoint
is removed and the settings tab shows it read-only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): nested fork routing + fork-of-dev branch rooting

A fork of a dev workspace now roots its wm-fork/** branch on the dev's
environment-label branch (the content it diverged from) and its PR merges
back into that branch: the backend passes parent_dev_workspace_label with
the deploy (parent row joined in both enqueue paths), the CLI gains
--parent-dev-workspace-label and checks it before the wm-fork- prefix
fallback when rooting a fork-of-a-fork branch, and the PR completion hook
uses it as the PR base.

Fork sync routing covers the whole live descendant chain of the
webhook/poller workspace (recursive, depth-capped) instead of direct
children only, and fork_open_prs is resolved at the root ancestor — only
the root can hold auto-pull config, so grandchild forks sync through it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): PR deploy-preview comment, clearer check copy, app-only hints

- The PR diff completion hook maintains one managed comment on the PR
  (Cloudflare deploy-preview style: workspace, status, commit, collapsible
  change list), upserted per synchronize via a hidden marker. The check run
  stays for required-check gating.
- A settings difference in the diff summary is worded by cause: the PR
  changes wmill.yaml, vs pre-existing drift between the repo's wmill.yaml
  and the workspace, vs undetermined (neutral wording).
- Deploy-status check titles name the target workspace ("Deployed 2
  change(s) to staging"), since GitHub shows a head commit's checks on any
  PR containing it and a bare "Deployed" read as if the PR had deployed.
- Token-based repos see a hint pointing at the open-pr-on-commit /
  open-pr-on-fork-commit workflows where the app-only PR toggles would be;
  an API-set toggle on a non-app repo now logs a warning naming the
  fallback; the design doc lists app-only features and their degradation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): EE-gate auto-pull UI, fork pull clone_ref, no-op push PR gate

- CE: the auto-pull and fork-PR toggles are disabled with an EE badge, and
  new sync repos only default them on when licensed (basic git sync is
  available on CE since #8493, but auto-pull is EE and the backend rejects it)
- The pull modal passes clone_ref for wm-fork- forks (wm-fork/<tracked>/<id>)
  so a manual pull fetches the fork branch instead of the tracked branch head
- PR-on-deploy skips no-op pushes: when the push script reports pushed=false
  (e.g. the deploy was caused by an auto-pull), the completion hook no longer
  ensures a PR, so closed PRs aren't recreated by the sync loop

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore: refresh package-lock after main merge (windmill-utils-internal 1.8.2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* test: auto-pull e2e integration tests; fix PR comment table formatting

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): runtime license gate for auto-pull saves; user/group promotion-branch parity

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): explain in-sync PR verdicts with the repo's sync filter scope

A PR that only touches files outside the repository's include paths gets
"In sync", which reads as a wrong verdict; the check summary (and managed
comment) now name the filters, e.g. "Only files matching this repository's
sync filters deploy on merge: `f/**` (excluding `f/pat/**`)."

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): clearer card copy/structure; surface PR-creation failures

- Fork sync toggle renamed and kept in the pull section; the fork PR toggle
  moves to the push section with a note that push settings apply to forks
- Fork/dev workspaces' push section names their actual branch instead of the
  tracked-branch line; promotion repos hide the pull direction (promotion
  pushes deploy branches on top of a sync-mode setup)
- Promotion mode line describes the wm_deploy/** branch + merge-to-promote
  flow; workflow-fallback hints lead with the how-to and link to the docs;
  test connection button demoted from accent per brand guidelines
- New server-owned open_pr_error on repo settings: the deploy completion hook
  records why a PR couldn't be opened (e.g. app permission not yet approved)
  and clears it on the next success; shown as a warning under the PR toggles

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix: cfg-gate scope-note helper (dead code on OSS builds)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): license-gate preserved auto-pull; attach strips parent-only settings

- edit_git_sync_repository re-checks the runtime Enterprise gate against the
  EFFECTIVE repo state after preservation: the older-client arm copies the
  existing auto_pull back, which the request-side check never saw
- attach_dev_workspace now mirrors the fork-creation copy on the attached
  workspace's own git sync: promotion repos dropped, auto_pull/fork PRs/PR
  error stripped, and any managed webhook deleted after commit (the attached
  workspace is parent-managed and must not keep pulling its old tracked branch)
- integration test: attaching an auto-pull-enabled workspace strips it

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): detach clears standalone parent; reject label == tracked branch

- detach_dev_workspace clears parent_workspace_id for prefix-less (attached
  standalone) workspaces so they stop classifying as forks and deploying to
  wm-fork/** branches; wm-fork- re-designated forks keep their parent; cache
  invalidations mirror attach
- dev-workspace create/attach reject an environment label that equals a
  git-sync repository's tracked branch (prod's or the candidate's): deploys
  would target the very branch the repo syncs from, and the CLI guard would
  fail every push job after the fact
- CLI unit tests: prefix-less fork beats wm_deploy derivation; isForkWorkspace
  parent-id argument

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(git-sync): bump hub script pins (push 28786, pull 28785)

Published from windmill-integrations #163 with windmill-cli@1.753.1-gitsync.0:
dev-workspace label deploys, fork-of-dev rooting, fork checkout on the
existing remote branch, and the pushed-flag result.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): no parent-only defaults on fork repos; rename strips webhook state

- addSyncRepository skips the auto_pull/fork_open_prs defaults on fork/dev
  workspaces where the backend rejects them (saving a new sync repo from an
  EE fork 400'd deterministically)
- change_workspace_id strips webhook id/secret/error from the copied git_sync
  and deletes the stale GitHub hooks post-commit: they deliver to the old
  (archived) workspace URL, so the new workspace would report a live webhook
  while polling at the relaxed interval; next save re-registers cleanly
- EE: PR diff checks for contributor-fork PRs clone the synthetic
  pull/<n>/head ref (head.ref doesn't exist in the base repo)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(git-sync): bump pull script pin to hub/28787 (synthetic PR ref support)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): targeted jsonb update for open_pr_error (no full-blob clobber)

The full read-modify-write raced the poller's concurrent last_synced_sha /
last_pull_status writes on the same column; mirror the EE status writer and
update only the matching repository element's open_pr_error key.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* style(git-sync): inline EE badge on gated toggles (matches settings nav)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): both directions in page/card descriptions; clearer promotion flow

- Page header and sync-card description mention the pull direction, not only
  push-on-deploy
- Promotion description walks the actual flow (wm_deploy/** branch, merge to
  promote, sync the target workspace) and points at the PR toggle / workflow;
  the Git Promotion docs link now also shows on configured cards, not only in
  the empty state

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): resolve branch-less resources' default branch for fork sync

A git resource without an explicit branch polled as the bare "HEAD" ref,
which the fork/dev-label fan-out cannot scope (wm-fork/<branch>/*), so fork
sync silently never ran on polling-only repos. Resolve the remote's default
branch name with `ls-remote --symref HEAD` (one call for name + head sha);
"HEAD" only remains when resolution fails. The polling e2e test now uses a
branch-less resource to cover this shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): runtime license gate for in-app PR creation

promotion_open_prs/fork_open_prs are rejected on save without an Enterprise
plan (like auto_pull), and the deploy completion hook re-checks the plan
before opening PRs so flags stored while licensed stop driving GitHub calls
after a lapse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): app-aware pull defaults, always webhook delivery, token-repo guidance

- Pull-from-Git defaults on only for app-backed repos (applied when the
  selected resource resolves); polling is opt-in for token repositories,
  with a warning alert recommending the GitHub App (instant pull + in-app
  PRs) or the sync GitHub workflow
- App repos always use webhook delivery with polling fallback: the delivery
  selector is gone and a stored polling mode is normalized back to auto
- Post-save modal reflects the auto-pull state instead of telling the user
  to turn on a toggle that is already on
- Non-app PR hints recommend the GitHub App explicitly

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): single info box for token-repo pull guidance

Merges the instant-pull recommendation with the GitHub Action conflict note,
shown only for non-app repos; app repos need neither, and the redundant
'instant webhook sync requires' line is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): keep the GitHub Action conflict note on app repos

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): anchor docs links to their exact sections

GitHub App references point at integrations/git_repository#github-app, the
workflow hints at deploy_gh_gl#github-actions-setup, and the sync workflow
at git_sync#github-actions (all anchors verified against the live docs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): fork workflow hint links to git_sync#github-actions

open-pr-on-fork-commit is documented on the git_sync page, not deploy_gh_gl.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): fork PRs are opt-in on new connections too

Only auto-pull and fork sync default on for new app-backed connections;
opening pull requests stays a deliberate per-repo decision.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): default the managed PR on for new app-backed promotion repos

A promotion deploy's wm_deploy/** branch exists to be merged; without a PR
it's an orphaned branch. Fork PRs stay opt-in. Also scope the sync-repo
auto-pull default to sync mode so promotion repos can't pick it up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* docs(git-sync): GHES self-managed app permission setup

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* docs(git-sync): frame permission update against GitHub Actions, not polling

Existing installations don't have polling; their git-to-Windmill direction
runs on GitHub Actions today, so the approval text describes the update as
replacing those workflows and notes every feature is opt-in.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): gate GitHub PRs on Windmill CI test results (WIN-2051)

Add a "Windmill CI tests" GitHub check run on any PR against the tracked
branch, so a workspace-fork customer can mark it a required status check and
have Windmill's own CI tests (the `// test:` annotation) block the PR —
replacing the GitHub Action that polls the CI-test results API.

Driven by the pull_request webhook (same event as the Phase 4 diff check): a
new git_sync_ci_test_check table tracks one check per (fork workspace, PR head
commit); a per-ci_test-job completion hook (result_processor) and the git-sync
poller sweep (monitor) conclude it. Reuses the Checks: write grant from #9552;
token repos keep the documented Action. See docs Phase 7.

EE logic lives in windmill-ee-private (git_sync_ee.rs); see ee-repo-ref.txt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(sqlx): drop two cache entries orphaned by the merge

Both queries were rewritten on main; nothing in the sources produces them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(migration): describe the CI-test check as webhook-opened, not deploy-opened

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: bump ee-repo-ref (skipped CI-test check on unmapped PRs)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(git-sync): skipped check for unmapped PRs; correct the installation rationale

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): cascade CI-test check rows with their workspaces; review fixes (EE ref)

Deleting a fork (the normal end of the one-fork-per-PR flow) or its parent
must not fail on the check table's foreign keys.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI-test check waits for the head's deploy; ignore retired tests (EE ref)

Design doc: GitLab scope, pending-while-deploying and retired-test rules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI-test verdict matches jobs against current test references (EE ref)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): record the pushed head as synced; CI-test check waits for it (EE ref)

The push completion hook records the commit the deploy push script reports as
the branch's synced sha, mirroring what a pull records. The check row stores
head_ref. Requires the hub push script version that reports the sha; the pin
is bumped once it is published.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI-test check row keeps the newest check run when creates race (EE ref)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): queue CI tests before a deploy or dependency job completes

The dispatch was detached, so a finished deploy did not yet imply its tests
existed. The git-sync PR check reads "deploy settled" as "tests queued", so the
dispatch is awaited at all three sites; errors are still only logged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(ci-tests): sqlx cache for the pull-success readiness clause; bump EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): readiness waits for pull success, failed deploy fails the check; sqlx cache; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): queue resource-triggered CI tests before the update is acknowledged

Same ordering guarantee as scripts and flows: the git-sync PR check
treats a finished pull as "the deploy's tests exist". Also carries the
sqlx cache for the tested-item liveness filter and the EE ref.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): dispatch on flow/resource create, sweep outside the auto-pull lock, pushed-head map

- create_flow and create_resource queue CI tests the way the update paths do
- the check sweep runs after the advisory unlock so GitHub calls never extend
  the auto-pull tick
- AutoPullSettings gains last_pushed_sha (settings round-trips keep it); the
  EE push hook writes it instead of last_synced_sha so auto-pull still pulls a
  commit someone else pushed under ours

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): last_pushed_sha is server-owned auto-pull state

Clear it from client-supplied settings, carry it over from the existing
entry on save, and keep it out of workspace exports, like the other sync
state fields. Correct the two doc comments that still described the
pushed head as the synced head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(git-sync): say which pull toggles the PR checks depend on

The repo webhook created by the pull toggle is what delivers pull request
events, and fork sync is what lets a fork PR's externally pushed commits
reach the fork workspace before its CI tests check is read.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(git-sync): CI check readiness reads a synced-head event log

New git_sync_synced_head table: one row per commit a workspace reflects
on a branch, written by the pull completion hook on success (from the
pull marker's branch and sha) and by the push completion hook from the
push script's reported sha. The check reads that instead of inferring
sync state from the auto-pull settings jsonb, so AutoPullSettings gets
no pushed-sha field and the settings gates for it go away. Pull success
now also runs the auto-pull completion hook.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): queue CI tests before the deploy push is enqueued; schema summary

The synced-head row a push writes is what makes the PR check read the
fork's results, so the tests must be queued before the push job exists.
Reordered at the script create, lockfile, and resource sites (flows
already push from their dependency job). Lists the two new tables in
summarized_schema.txt.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): newest synced head decides readiness; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): newest synced head decides readiness

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): synced heads keyed by repository; check row keeps the repo resource path

Both tables are unreleased, so their migrations gain the column in place.
The push hook reads the push job's repository from its args again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI check rows keyed by repository; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(git-sync): pin the deploy push script to hub/28956

The version whose main returns the push outcome {pushed, sha, branch},
which the completion hook records as the branch's synced head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(ci-tests): tooltips name GitHub for the CI check; drop the sanitization test; docs and EE ref

The sanitization test guarded a field that no longer exists and now only
re-exercised pre-existing behavior. The design doc carries the repository
key and the row-before-create order.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): re-fire resets the run id; pinned-script reason on timeout; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): record the commit a pull checked out; skip a rebased push

The pull script now reports the sha and branch its clone landed on, so
the synced head is what was applied rather than the head observed when
the pull was enqueued (the marker stays the fallback). A push the script
had to rebase sits on commits the workspace has not pulled, so it is not
recorded; the pull those commits trigger records the head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(git-sync): pin the push script to hub/28958 and the pull script to hub/28957

The versions whose results report a rebased push and the commit a pull
checked out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): queued-pull wait and kept failure on re-fire; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(ci-tests): sqlx cache for the prune and skipped-guard queries; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): a failed head passes only on newer runs; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ci-tests): the PR check runs the head's own suite

The synced-head row records the CI test runs dispatched for the head
(tests_dispatched_at, ci_test_job_ids); the check reads exactly those.
windmill-git-sync now owns the check lifecycle (it can enqueue jobs) and
depends on windmill-dep-map for the dispatcher. The deploy-site changes
that ordered per-item dispatch before the push existed only for the old
workspace-wide verdict and are restored to main's form; flow and resource
creation keep the per-item dispatch they lacked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): suite runs are not debounced, run as the workspace admin, and a dead dispatch claim is retaken

EE ref f30eaf8: the head's suite no longer shares debounce keys with
deploy-triggered runs (a superseded run dropped out of the verdict), runs as
the workspace admin an auto pull resolves rather than the sync job's identity
(which retention could remove), and a dispatch claim that never recorded ids
is retaken after 5 minutes. sqlx cache and design doc updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): the head's suite expands wildcard references

EE ref 53d77c3; sqlx cache updated for the reference and script queries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): bound the failed-deployment gate; keep a current head's check row

EE ref 63523d5; sqlx cache updated for the two changed queries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): the head's suite runs as the fork's owner; poster_workspace_id

EE ref a74af94. The unreleased check-table migration renames
github_workspace_id to poster_workspace_id (the workspace whose credential
posts the check); sqlx cache, schema summary and design doc follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(git-sync): the PR check's suite runs as the fork's owner

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): CE-only build, UI pull-script pin, detached sweep

- The four check-lifecycle re-exports exist only on enterprise builds; gating
  them on private alone broke the CE image (private without enterprise).
- hubPaths.json's gitInitRepo follows the server's pull-script pin (28957),
  as every previous bump did and as the git-sync integration test asserts.
- The monitor spawns the CI-check sweep detached (EE ref f96aa31).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: bump EE ref to a1d7a1e

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): advance the PR check off the worker's completion loop

Concluding a check calls GitHub; the completion loop finishes jobs serially,
so the evaluation is spawned detached (idempotent, retried by the poller).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(git-sync): check-row-before-create order, retarget events; EE ref 40079e0

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: update ee-repo-ref to a4da009a5eae72bd55f34de41ba7929b53d53c9b

This commit updates the EE repository reference after PR #662 was merged in windmill-ee-private.

Previous ee-repo-ref: 4845e823bfaa28d3258b4c2fb1e9e2781dde382f

New ee-repo-ref: a4da009a5eae72bd55f34de41ba7929b53d53c9b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
This commit is contained in:
hugocasa
2026-09-14 11:42:57 +02:00
committed by GitHub
co-authored by Claude Fable 5.1 windmill-internal-app[bot]
parent d8d7332eb6
commit 80eba80d6e
45 changed files with 1047 additions and 16 deletions
@@ -0,0 +1,19 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO git_sync_ci_test_check\n (workspace_id, poster_workspace_id, head_sha, head_ref, repo_url,\n repo_resource_path, check_run_id,\n created_at, concluded, conclusion, concluded_at, github_posted)\n VALUES ($1, $2, $3, $4, $5, $6, NULL, now(), false, NULL, NULL, false)\n ON CONFLICT (workspace_id, repo_resource_path, head_sha) DO UPDATE SET\n poster_workspace_id = EXCLUDED.poster_workspace_id,\n head_ref = EXCLUDED.head_ref,\n repo_url = EXCLUDED.repo_url,\n check_run_id = NULL,\n created_at = now(),\n concluded = false,\n conclusion = NULL,\n concluded_at = NULL,\n github_posted = false",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Text",
"Varchar"
]
},
"nullable": []
},
"hash": "16c6e24ae06b52feed597a0c3d299107f989d50ea651546e964670cf99fd2de6"
}
@@ -0,0 +1,40 @@
{
"db_name": "PostgreSQL",
"query": "SELECT test_script_path, tested_item_path, tested_item_kind, has_wildcard AS \"has_wildcard!\" FROM ci_test_reference WHERE workspace_id = $1 ORDER BY test_script_path, tested_item_kind, tested_item_path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "test_script_path",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "tested_item_path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "tested_item_kind",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "has_wildcard!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false,
false,
true
]
},
"hash": "18ba139acef81d4de18bf21755fa8605c3851b48bab4964c380538ada93f06a9"
}
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT u.username, u.email FROM workspace w JOIN usr u ON u.workspace_id = w.id AND u.email = w.owner WHERE w.id = $1 AND NOT u.disabled",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "username",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "email",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "1b5f6620d35dd74b32ce6325891be02fe144a2da1ebf446b0df92944da791fa7"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT r->>'script_path' as \"script_path\"\n FROM workspace_settings ws,\n jsonb_array_elements(\n CASE WHEN jsonb_typeof(ws.git_sync->'repositories') = 'array'\n THEN ws.git_sync->'repositories' END\n ) r\n WHERE ws.workspace_id = $1\n AND r->>'git_repo_resource_path' IN ($2, '$res:' || $2)\n LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "script_path",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "35783f52031d7ba14142108480b3599f083dee7415acdbe4410412309dfa2ca1"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_ci_test_check\n SET concluded = true, github_posted = true, concluded_at = now(),\n conclusion = COALESCE(conclusion, 'failure')\n WHERE (check_run_id IS NULL AND NOT concluded\n AND created_at < now() - make_interval(secs => $1))\n OR (concluded AND NOT github_posted\n AND concluded_at < now() - make_interval(secs => $2))",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Float8",
"Float8"
]
},
"nullable": []
},
"hash": "422490f2f91b4d97331e87da135884932eab27f53171c12291cca15a6ec33586"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_synced_head SET tests_dispatched_at = NULL\n WHERE workspace_id = $1 AND repo_resource_path = $2 AND branch = $3 AND sha = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "45d0e716fa402a63b0bf6877c21c0fa50b81d46845c400d569d6d8e49e503b59"
}
@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_synced_head SET tests_dispatched_at = now()\n WHERE workspace_id = $1 AND repo_resource_path = $2 AND branch = $3 AND sha = $4\n AND (tests_dispatched_at IS NULL\n OR (ci_test_job_ids IS NULL\n AND tests_dispatched_at < now() - make_interval(secs => $5)))\n RETURNING true as \"claimed!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "claimed!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text",
"Float8"
]
},
"nullable": [
null
]
},
"hash": "4dde939e92f5b8a9cc165c9ea383a456eef081d16c2a58e7262f86d80289c2da"
}
@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "SELECT (\n SELECT h.sha FROM git_sync_synced_head h\n WHERE h.workspace_id = $1 AND h.repo_resource_path = $4 AND h.branch = $3\n ORDER BY h.synced_at DESC LIMIT 1\n ) = $2 AND NOT EXISTS (\n SELECT 1\n FROM v2_job_queue q\n JOIN v2_job j ON j.id = q.id\n WHERE q.workspace_id = $1\n AND j.kind = 'deploymentcallback'\n AND j.args->'__git_sync_auto_pull'->>'branch' = $3\n AND j.args->'__git_sync_auto_pull'->>'repo_resource_path'\n IN ($4, '$res:' || $4)\n ) AND NOT EXISTS (\n SELECT 1\n FROM v2_job_queue q\n JOIN v2_job j ON j.id = q.id\n WHERE q.workspace_id = $1\n AND j.kind IN ('dependencies', 'flowdependencies', 'appdependencies')\n ) as \"ready\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "ready",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "4e4b31e97f0cc946f26cc0faf9a09de2846c43ee48f047e0f2ea9ee7a6502c81"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_synced_head SET ci_test_job_ids = NULL, tests_dispatched_at = NULL\n WHERE workspace_id = $1 AND repo_resource_path = $2 AND branch = $3 AND sha = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "5fcaf17e24fa00ffafdc5f0f425fe4c1c745457d463d8ea61627ebe61ba8ab2c"
}
@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_synced_head SET ci_test_job_ids = $5\n WHERE workspace_id = $1 AND repo_resource_path = $2 AND branch = $3 AND sha = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text",
"UuidArray"
]
},
"nullable": []
},
"hash": "6e3cd83ad7eef0dddacf9359f662193e094ca44642342778fb4281a711263385"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_ci_test_check SET github_posted = true\n WHERE workspace_id = $1 AND repo_resource_path = $4 AND head_sha = $2\n AND check_run_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "7a0ddb6821d8f628bcf85f786e5864e09c7a5a421ba99647570c2b557d53aa51"
}
@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_ci_test_check\n SET concluded = true, conclusion = $3, concluded_at = now()\n WHERE workspace_id = $1 AND repo_resource_path = $4 AND head_sha = $2 AND NOT concluded\n RETURNING check_run_id, poster_workspace_id, repo_url",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "check_run_id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "poster_workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "repo_url",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": [
true,
false,
false
]
},
"hash": "89a7f413f6f37aeb7e777faeebb0c4e1612928737a5fb2b4ff1336d8827f788b"
}
@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "SELECT ci_test_job_ids\n FROM git_sync_synced_head\n WHERE workspace_id = $1 AND repo_resource_path = $2 AND branch = $3 AND sha = $4",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "ci_test_job_ids",
"type_info": "UuidArray"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "8efe5509034327c202cb3fdd409ce00cc1dbd4921a9ccee8f906515286190b8f"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_synced_head SET tests_dispatched_at = NULL\n WHERE workspace_id = $1 AND repo_resource_path = $2 AND branch = $3 AND sha = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "98b036be15cbd5efbaf2420feb56dd175aeab87f6377f6ac7b51956ce6d5f039"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM git_sync_ci_test_check c\n WHERE c.concluded AND c.github_posted\n AND c.concluded_at < now() - make_interval(secs => $1)\n AND NOT EXISTS (\n SELECT 1 FROM git_sync_synced_head h\n WHERE h.workspace_id = c.workspace_id\n AND h.repo_resource_path = c.repo_resource_path\n AND h.sha = c.head_sha\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Float8"
]
},
"nullable": []
},
"hash": "a7d5a7b6b3bb88f5f7926da577f2cc25020b11882fb359b136466c82b040f8a1"
}
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT repo_resource_path, head_sha FROM git_sync_ci_test_check\n WHERE workspace_id = $1 AND NOT concluded",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "repo_resource_path",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "head_sha",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "a970bbf4d3d064614bc47d438a0447eff928c4518fd2f6800145bf1211086352"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE git_sync_ci_test_check\n SET github_posted = github_posted\n AND check_run_id IS NOT DISTINCT FROM GREATEST(check_run_id, $3),\n check_run_id = GREATEST(check_run_id, $3)\n WHERE workspace_id = $1 AND repo_resource_path = $4 AND head_sha = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "b7c72ecebf6818d4e60a02edb986c920aa2e14365e63ef8c28c7f65da7c6c9ab"
}
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS (\n SELECT 1 FROM git_sync_ci_test_check\n WHERE poster_workspace_id = $1 AND repo_resource_path = $2 AND head_sha = $3\n ) as \"exists!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "bafee32cbff8bb7fff26a241d9ad203ea689c1e4b094a2c90eb90c8e4b6e0dff"
}
@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) as \"count!\"\n FROM git_sync_synced_head h\n JOIN v2_job_completed pc ON pc.id = h.job_id\n JOIN v2_job j ON j.workspace_id = h.workspace_id\n AND j.kind IN ('dependencies', 'flowdependencies', 'appdependencies')\n AND j.created_at >= pc.started_at\n AND j.created_at <= COALESCE(h.tests_dispatched_at, now())\n JOIN v2_job_completed c ON c.id = j.id AND c.status IN ('failure', 'canceled')\n WHERE h.workspace_id = $1 AND h.repo_resource_path = $4\n AND h.branch = $3 AND h.sha = $2\n AND h.source = 'pull'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "c38a1cf8d2a8fd89008a98f03ab87a438b619eb1919c028c6c80e972b4ae438d"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT args->>'repo_url_resource_path' FROM v2_job WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "cfbe6784b3d108f935ba884fe2d3b36afde836fbc6db046279b015bea6f70201"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT path FROM script WHERE workspace_id = $1 AND deleted = false AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "d644b9cd3407e58f235cc2e97558257c07785c8b3123f13d59ee361b4ee0bc0a"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) as \"count!\" FROM v2_job WHERE id = ANY($1::uuid[])",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"UuidArray"
]
},
"nullable": [
null
]
},
"hash": "dfffd6573a1eab11c0515805f85b95a942f71651a4f34e671f25ba6816a506c8"
}
@@ -0,0 +1,50 @@
{
"db_name": "PostgreSQL",
"query": "SELECT workspace_id, poster_workspace_id, head_sha, repo_url, repo_resource_path,\n check_run_id\n FROM git_sync_ci_test_check\n WHERE NOT concluded OR NOT github_posted",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "poster_workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "head_sha",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "repo_url",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "repo_resource_path",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "check_run_id",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false,
false,
false,
false,
true
]
},
"hash": "e64ec4941cbbee016c14d958b7220ddfc414c7e741a171fb80673c23644e3619"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM git_sync_synced_head h\n WHERE h.synced_at < now() - make_interval(secs => $1)\n AND EXISTS (\n SELECT 1 FROM git_sync_synced_head n\n WHERE n.workspace_id = h.workspace_id\n AND n.repo_resource_path = h.repo_resource_path\n AND n.branch = h.branch\n AND n.synced_at > h.synced_at\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Float8"
]
},
"nullable": []
},
"hash": "e6f2a6fa47bf3b5c774d6bc6060ca5a99addffab3dd44529ce9f40de115b0a3c"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COALESCE(jc.status::text, 'running') as \"status!\"\n FROM unnest($1::uuid[]) AS run(id)\n LEFT JOIN v2_job_completed jc ON jc.id = run.id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "status!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"UuidArray"
]
},
"nullable": [
null
]
},
"hash": "ea397add5eb6555457883e5b6bdc67efbe6b0559adb891dba65d8b8e1430f357"
}
@@ -0,0 +1,19 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO git_sync_synced_head\n (workspace_id, repo_resource_path, branch, sha, source, job_id)\n VALUES ($1, $2, $3, $4, $5, $6)\n ON CONFLICT (workspace_id, repo_resource_path, branch, sha)\n DO UPDATE SET source = EXCLUDED.source, job_id = EXCLUDED.job_id, synced_at = now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Uuid"
]
},
"nullable": []
},
"hash": "f1282393a95b499f1a9fce5939205879d507b9697eed7e5e232f8e6cb95c2bd0"
}
@@ -0,0 +1,66 @@
{
"db_name": "PostgreSQL",
"query": "SELECT repo_url, check_run_id, poster_workspace_id, conclusion,\n created_at, concluded, github_posted, head_ref\n FROM git_sync_ci_test_check\n WHERE workspace_id = $1 AND repo_resource_path = $3 AND head_sha = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "repo_url",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "check_run_id",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "poster_workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "conclusion",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "created_at",
"type_info": "Timestamptz"
},
{
"ordinal": 5,
"name": "concluded",
"type_info": "Bool"
},
{
"ordinal": 6,
"name": "github_posted",
"type_info": "Bool"
},
{
"ordinal": 7,
"name": "head_ref",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
false,
true,
false,
true,
false,
false,
false,
false
]
},
"hash": "f6a2a8fbc22c69fd5da86626372f84d0ec7b6cb9375e30b10415605cba9b2fcb"
}
+1
View File
@@ -15727,6 +15727,7 @@ dependencies = [
"tracing",
"uuid",
"windmill-common",
"windmill-dep-map",
"windmill-queue",
]
+1 -1
View File
@@ -1 +1 @@
be7262ca144933128cc7924e418c88ffe4e5a6ef
a4da009a5eae72bd55f34de41ba7929b53d53c9b
@@ -0,0 +1 @@
DROP TABLE IF EXISTS git_sync_ci_test_check;
@@ -0,0 +1,39 @@
-- One "Windmill CI tests" GitHub check run per (fork workspace, repository, PR head commit):
-- the pull_request webhook opens the check in_progress and it is concluded once
-- the fork's CI tests settle, so the results can gate a GitHub PR.
CREATE TABLE git_sync_ci_test_check (
-- The fork workspace whose CI tests gate the PR: keys the row, and its `ci_test`
-- jobs are what the check reflects.
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id) ON DELETE CASCADE,
head_sha VARCHAR(64) NOT NULL,
-- The PR's head branch: the check waits until the fork's synced state for this
-- branch (written by its pushes and pulls alike) names `head_sha`.
head_ref VARCHAR(255) NOT NULL,
-- The workspace whose git host credential posts the check: the one that received
-- the pull request webhook (the parent owning the repo hook).
poster_workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id) ON DELETE CASCADE,
repo_url TEXT NOT NULL,
-- The fork's copy of the repository resource: keys the synced-head lookup, since a
-- fork syncing two repositories names its branch identically in both.
repo_resource_path VARCHAR(255) NOT NULL,
-- NULL when the GitHub check-run creation failed; the poller retries the create.
check_run_id BIGINT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
concluded BOOLEAN NOT NULL DEFAULT false,
conclusion TEXT,
concluded_at TIMESTAMPTZ,
-- Decoupled from `concluded` so a failed check-run PATCH is retried by the
-- poller instead of hanging a required check on GitHub.
github_posted BOOLEAN NOT NULL DEFAULT false,
PRIMARY KEY (workspace_id, repo_resource_path, head_sha)
);
-- Rows still needing action (create retry, conclusion, timeout, delivery retry).
-- A row drops out only once it is both concluded and delivered to GitHub, so the
-- per-job conclusion hook and the poller sweeper both scan a small live set.
CREATE INDEX idx_git_sync_ci_test_check_pending
ON git_sync_ci_test_check (workspace_id)
WHERE NOT concluded OR NOT github_posted;
GRANT ALL ON git_sync_ci_test_check TO windmill_user;
GRANT ALL ON git_sync_ci_test_check TO windmill_admin;
@@ -0,0 +1 @@
DROP TABLE IF EXISTS git_sync_synced_head;
@@ -0,0 +1,24 @@
-- One row per commit a workspace has come to reflect on a branch, written when a
-- pull of that commit succeeds or a deploy push produces it. The "Windmill CI
-- tests" PR check reads it to know when a workspace reflects a PR head, and
-- records the head's CI test runs on it. Kept apart from `workspace_settings.git_sync.auto_pull.last_synced_sha`,
-- which decides whether the next poll pulls and is client-round-tripped settings.
CREATE TABLE git_sync_synced_head (
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id) ON DELETE CASCADE,
-- Repository resource path without its `$res:` prefix.
repo_resource_path VARCHAR(255) NOT NULL,
branch VARCHAR(255) NOT NULL,
sha VARCHAR(64) NOT NULL,
-- 'pull' rows name the pull job; 'push' rows the deploy push job.
source VARCHAR(4) NOT NULL CHECK (source IN ('pull', 'push')),
job_id UUID,
synced_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-- The head's own CI test suite, dispatched once the workspace reflects it and its
-- dependency jobs settled: the "Windmill CI tests" check reads exactly these runs.
tests_dispatched_at TIMESTAMPTZ,
ci_test_job_ids UUID[],
PRIMARY KEY (workspace_id, repo_resource_path, branch, sha)
);
GRANT ALL ON git_sync_synced_head TO windmill_user;
GRANT ALL ON git_sync_synced_head TO windmill_admin;
+8
View File
@@ -4711,6 +4711,14 @@ pub async fn poll_git_auto_pull(db: &Pool<Postgres>) {
{
tracing::error!("git auto-pull: advisory unlock failed: {e:#}");
}
// Backstop for the "Windmill CI tests" checks: retry a failed GitHub create or
// delivery, conclude checks whose tests settled, time out stuck ones, prune old
// rows. Detached and outside the advisory lock: its writes are guarded (claimed
// conclude, greatest-id upsert), it is single-flight, and its GitHub calls must not
// count against the monitor pass's budget.
let db = db.clone();
tokio::spawn(async move { windmill_git_sync::sweep_ci_test_checks(&db).await });
}
#[cfg(feature = "private")]
+5
View File
@@ -113,6 +113,11 @@ folder: name(char), workspace_id(char), display_name(char), owners(char), extra_
folder_permission_history: id(bigint), workspace_id(char), folder_name(char), changed_by(char), changed_at(ts), change_type(char), affected(char)
FK: (workspace_id, folder_name) -> folder(workspace_id, name)
gcp_trigger: gcp_resource_path(char), topic_id(char), subscription_id(char), delivery_type(delivery_mode), delivery_config(jsonb), path(char), script_path(char), is_flow(bool), workspace_id(char), edited_by(char), email(char), edited_at(ts), extra_perms(jsonb), server_id(char), last_server_ping(ts), error(text), subscription_mode(gcp_subscription_mode), error_handler_path(char), error_handler_args(jsonb), retry(jsonb), auto_acknowledge_msg(bool), ack_deadline(int), mode(trigger_mode), labels(text[])
git_sync_ci_test_check: workspace_id(char), head_sha(char), head_ref(char), poster_workspace_id(char), repo_url(text), repo_resource_path(char), check_run_id(bigint), created_at(timestamptz), concluded(bool), conclusion(text), concluded_at(timestamptz), github_posted(bool)
FK: (workspace_id) -> workspace(id)
FK: (poster_workspace_id) -> workspace(id)
git_sync_synced_head: workspace_id(char), repo_resource_path(char), branch(char), sha(char), source(char), job_id(uuid), synced_at(timestamptz), tests_dispatched_at(timestamptz), ci_test_job_ids(uuid[])
FK: (workspace_id) -> workspace(id)
global_settings: name(char), value(jsonb), updated_at(ts)
guest_activity: email(char), workspace_id(char), day(date), last_seen_at(timestamptz), jwt_entry(bool)
group_: workspace_id(char), name(char), summary(text), extra_perms(jsonb)
+23
View File
@@ -827,6 +827,29 @@ async fn create_flow(
WebhookMessage::CreateFlow { workspace: w_id.clone(), path: nf.path.clone() },
);
// Trigger CI tests for items that reference this flow
{
let db2 = db.clone();
let w_id2 = w_id.clone();
let flow_path2 = nf.path.clone();
let email2 = authed.email.clone();
let username2 = authed.username.clone();
tokio::spawn(async move {
if let Err(e) = windmill_dep_map::ci_tests::trigger_ci_tests_for_item(
&db2,
&w_id2,
&flow_path2,
"flow",
&email2,
&username2,
)
.await
{
tracing::error!(%e, "error triggering CI tests after flow creation");
}
});
}
Ok((StatusCode::CREATED, nf.path.to_string()))
}
+5 -4
View File
@@ -191,7 +191,7 @@ pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28958/sync-script-to-git-repo
/// ignores the slug, so the slug is kept free of characters that would be
/// percent-encoded into the run URL (a `:` becomes `%3A`, which some hardened
/// reverse proxies reject as double-encoding when the client re-encodes it).
pub const GIT_SYNC_PULL_SCRIPT_PATH: &str = "hub/28948/git-sync-init-repository-windmill";
pub const GIT_SYNC_PULL_SCRIPT_PATH: &str = "hub/28957/git-sync-init-repository-windmill";
/// Prefix used to identify fork workspaces. A workspace whose id starts with this string is a
/// fork of another workspace.
@@ -559,9 +559,10 @@ impl AutoPullSettings {
/// Whether a freshly observed `(git_ref, head_sha)` warrants enqueuing a pull.
///
/// A trigger (poll or webhook) is only a hint: we pull when auto-pull is
/// enabled and the observed head differs from the last sha we synced for
/// that ref. Re-observing the same head (e.g. a redundant poll, or the
/// commit our own deploy callback just pushed back) is a no-op.
/// enabled and the observed head differs from the last sha we pulled for
/// that ref. Re-observing the same head (a redundant poll) is a no-op. A
/// commit our own deploy pushed is not: pushes never write here, so the pull
/// it triggers picks up anything pushed under it.
pub fn should_pull(&self, git_ref: &str, head_sha: &str) -> bool {
self.enabled && self.last_synced_sha.get(git_ref).map(String::as_str) != Some(head_sha)
}
+10
View File
@@ -19,3 +19,13 @@ pub async fn trigger_ci_tests_for_item(
) -> error::Result<Vec<Uuid>> {
Ok(vec![])
}
#[cfg(not(feature = "private"))]
pub async fn trigger_all_ci_tests(
_db: &sqlx::Pool<sqlx::Postgres>,
_w_id: &str,
_email: &str,
_username: &str,
) -> error::Result<Vec<Uuid>> {
Ok(vec![])
}
+2 -1
View File
@@ -9,7 +9,7 @@ name = "windmill_git_sync"
path = "./src/lib.rs"
[features]
private = ["windmill-common/private"]
private = ["windmill-common/private", "windmill-dep-map/private"]
enterprise = ["windmill-queue/enterprise", "windmill-common/enterprise"]
all_sqlx_features = ["enterprise"]
default = []
@@ -22,5 +22,6 @@ serde_json.workspace = true
tracing.workspace = true
windmill-common = { workspace = true, default-features = false }
windmill-queue.workspace = true
windmill-dep-map.workspace = true
regex = "1.10.3"
tokio = { workspace = true, features = ["full"] }
+10 -1
View File
@@ -17,7 +17,16 @@ pub use git_sync_ee::{
clear_auto_pull_failure, enqueue_git_pull_dry_run, enqueue_git_pull_job,
handle_deployment_metadata, handle_deployment_metadata_batch, handle_fork_branch_creation,
persist_auto_pull_state, reconcile_and_enqueue_pull, reconcile_fork_branch_pull,
record_auto_pull_failure, tally_deployed_object_changes,
record_auto_pull_failure, record_synced_head, sweep_ci_test_checks,
tally_deployed_object_changes,
};
// The CI-test check exists only on enterprise builds; `private` alone (the CE image)
// compiles git_sync_ee without them.
#[cfg(all(feature = "private", feature = "enterprise"))]
pub use git_sync_ee::{
ensure_ci_test_check_for_pr, evaluate_and_conclude_ci_test_checks,
post_ci_test_check_not_applicable, resolve_pr_head_workspace,
};
#[cfg(not(feature = "private"))]
+18 -1
View File
@@ -1297,9 +1297,26 @@ async fn create_resource(
webhook.send_message(
w_id.clone(),
WebhookMessage::CreateResource { workspace: w_id, path: resource.path.clone() },
WebhookMessage::CreateResource { workspace: w_id.clone(), path: resource.path.clone() },
);
// Trigger CI tests for items that reference this resource
{
let db2 = db.clone();
let path2 = resource.path.clone();
let email2 = authed.email.clone();
let username2 = authed.username.clone();
tokio::spawn(async move {
if let Err(e) = windmill_dep_map::ci_tests::trigger_ci_tests_for_item(
&db2, &w_id, &path2, "resource", &email2, &username2,
)
.await
{
tracing::error!(%e, "error triggering CI tests after resource creation");
}
});
}
Ok((
StatusCode::CREATED,
format!("resource {} created", resource.path),
+140 -5
View File
@@ -983,19 +983,21 @@ mod git_sync_check_tests {
}
}
/// When an auto-pull job (carrying `__git_sync_auto_pull`) fails, roll the
/// When an auto-pull job (carrying `__git_sync_auto_pull`) completes: on success,
/// record the commit as a head the workspace reflects; on failure, roll the
/// optimistic `last_synced_sha` advance back to the pre-pull value so the commit
/// is retried instead of being silently treated as synced, and record the failure.
/// The recorded commit is the one the pull script reports having checked out
/// (`{sha, branch}` in its result): the branch can move between the observation
/// the marker holds and the clone. A result without it falls back to the marker.
#[cfg(all(feature = "enterprise", feature = "private"))]
async fn maybe_reconcile_git_sync_auto_pull(
db: &DB,
job_id: &uuid::Uuid,
workspace_id: &str,
success: bool,
result: &str,
) {
if success {
return; // the optimistic synced state is already correct
}
let marker: Option<serde_json::Value> = match sqlx::query_scalar!(
"SELECT args->'__git_sync_auto_pull' FROM v2_job WHERE id = $1",
job_id
@@ -1015,12 +1017,50 @@ async fn maybe_reconcile_git_sync_auto_pull(
#[derive(serde::Deserialize)]
struct AutoPullMarker {
repo_resource_path: String,
branch: Option<String>,
head_sha: Option<String>,
#[serde(default)]
prev_synced: std::collections::HashMap<String, String>,
}
let Ok(m) = serde_json::from_value::<AutoPullMarker>(marker) else {
return;
};
if success {
// The optimistic synced state is already correct; record that the workspace
// now reflects the commit, which the PR CI-test check waits for.
#[derive(serde::Deserialize)]
struct PullResult {
sha: Option<String>,
branch: Option<String>,
}
let applied = serde_json::from_str::<PullResult>(result).ok();
let branch = applied
.as_ref()
.and_then(|r| r.branch.as_deref())
.or(m.branch.as_deref());
let sha = applied
.as_ref()
.and_then(|r| r.sha.as_deref())
.or(m.head_sha.as_deref());
if let (Some(branch), Some(sha)) = (branch, sha) {
if let Err(e) = windmill_git_sync::record_synced_head(
db,
workspace_id,
&m.repo_resource_path,
branch,
sha,
"pull",
Some(*job_id),
)
.await
{
tracing::warn!(
"git auto-pull: failed to record synced head {sha} on {branch}: {e:#}"
);
}
}
return;
}
windmill_git_sync::record_auto_pull_failure(
db,
workspace_id,
@@ -1109,6 +1149,70 @@ fn git_sync_push_result_pushed(result: &str) -> Option<bool> {
.as_bool()
}
/// When a git-sync push job pushed a commit, record it as a head the workspace
/// reflects, the way a successful pull records the commit it applied. The PR
/// CI-test check waits for that record. Best-effort: failures are logged, never
/// propagated.
#[cfg(all(feature = "enterprise", feature = "private"))]
async fn maybe_record_git_sync_pushed_head(
db: &DB,
job_id: &uuid::Uuid,
workspace_id: &str,
result: &str,
) {
#[derive(serde::Deserialize)]
struct PushResult {
pushed: bool,
sha: Option<String>,
branch: Option<String>,
#[serde(default)]
rebased: bool,
}
let Ok(PushResult { pushed: true, sha: Some(sha), branch: Some(branch), rebased }) =
serde_json::from_str::<PushResult>(result)
else {
return;
};
// A push that had to rebase sits on commits this workspace has not pulled, so the
// pushed head is not something it reflects yet; the pull those commits trigger
// records the head once they are in.
if rebased {
tracing::info!(
"git sync push: {sha} on {branch} was rebased onto unpulled commits; not recording it as synced for {workspace_id}"
);
return;
}
let repo_path = match sqlx::query_scalar!(
"SELECT args->>'repo_url_resource_path' FROM v2_job WHERE id = $1",
job_id
)
.fetch_optional(db)
.await
{
Ok(Some(Some(p))) => p,
Ok(_) => return,
Err(e) => {
tracing::error!("git sync push: failed to read job args: {e:#}");
return;
}
};
if let Err(e) = windmill_git_sync::record_synced_head(
db,
workspace_id,
&repo_path,
&branch,
&sha,
"push",
Some(*job_id),
)
.await
{
tracing::warn!(
"git sync push: failed to record pushed head {sha} on {branch} for {workspace_id}/{repo_path}: {e:#}"
);
}
}
/// When a git-sync push job carrying `__git_sync_open_pr` succeeds, open (or
/// reopen) the PR for the branch it pushed: `wm-fork/<base>/<id>` for a fork
/// deploy, `wm_deploy/**` for a promotion deploy. Runs outbound with the
@@ -1707,8 +1811,26 @@ pub async fn process_completed_job(
#[cfg(all(feature = "enterprise", feature = "private"))]
if job.kind == JobKind::DeploymentCallback {
maybe_post_git_sync_check(db, &job_id, &workspace_id, true, result.get()).await;
maybe_reconcile_git_sync_auto_pull(db, &job_id, &workspace_id, true, result.get())
.await;
maybe_record_git_sync_pushed_head(db, &job_id, &workspace_id, result.get()).await;
maybe_open_git_sync_deploy_pr(db, &job_id, &workspace_id, result.get()).await;
}
// A CI test job just finished: advance any open "Windmill CI tests" PR check for
// its workspace. Detached, since concluding a check calls GitHub and this loop
// completes jobs serially; the evaluation is idempotent and the poller retries.
#[cfg(all(feature = "enterprise", feature = "private"))]
if job
.trigger_kind
.as_ref()
.is_some_and(|k| k.is(windmill_common::jobs::JobTriggerKind::CiTest))
{
let db = db.clone();
let w_id = workspace_id.clone();
tokio::spawn(async move {
windmill_git_sync::evaluate_and_conclude_ci_test_checks(&db, &w_id).await
});
}
// Asset-trigger fan-out: best-effort, never propagates errors.
// Internal eligibility checks gate to top-level Script/Preview runs;
@@ -1819,7 +1941,20 @@ pub async fn process_completed_job(
#[cfg(all(feature = "enterprise", feature = "private"))]
if job.kind == JobKind::DeploymentCallback {
maybe_post_git_sync_check(db, &job.id, &job.workspace_id, false, result.get()).await;
maybe_reconcile_git_sync_auto_pull(db, &job.id, &job.workspace_id, false).await;
maybe_reconcile_git_sync_auto_pull(db, &job.id, &job.workspace_id, false, "").await;
}
// A failed CI test job also settles its check; same detached advance as on success.
#[cfg(all(feature = "enterprise", feature = "private"))]
if job
.trigger_kind
.as_ref()
.is_some_and(|k| k.is(windmill_common::jobs::JobTriggerKind::CiTest))
{
let db = db.clone();
let w_id = job.workspace_id.clone();
tokio::spawn(async move {
windmill_git_sync::evaluate_and_conclude_ci_test_checks(&db, &w_id).await
});
}
if job.is_flow_step() {
if let Some(parent_job) = job.parent_job {
+87
View File
@@ -618,6 +618,93 @@ repo's **Environments** timeline ("Production → Deployed"). Needs
opt-in / later. The check-run version is the cheap default and matches the visual
Cloudflare parity without a new permission.
### Phase 7 — CI test results check (WIN-2051) — implemented
Surfaces Windmill's own CI tests (the `// test: script/...` annotation) as a
**"Windmill CI tests"** check run on **any PR** against the tracked branch, so a customer
can mark it a **required status check** and have Windmill CI results gate the PR —
replacing the documented GitHub Action that polls `ci_test_results_batch`. GitHub App-backed
only; reuses the Phase 4 `Checks: write` grant, so no new permission. Token repos keep the
Action, and GitLab merge requests get no CI-test surface for the same reason the Phase 4
preview lives in a note there (a commit status would fail the project's own pipeline).
Driven by the **`pull_request` webhook** — the same event Phase 4 already reacts to —
rather than the deploy push/pull, so it's uniform across how the PR's commit came to exist
(a fork deploy that pushes `wm-fork/**` and opens the PR, or an external push that gets
pulled in). CI tests run as separate async `ci_test` jobs in the **fork workspace** the PR
corresponds to; the check reflects that fork's current results on the PR head.
- **State** — `git_sync_ci_test_check(workspace_id, repo_resource_path, head_sha)` (new
table). `workspace_id` is the **fork** whose `ci_test` jobs the check reflects;
`repo_resource_path` the repository (a fork can sync several, and two can hold the same
commit); `poster_workspace_id` is the **parent** whose GitHub-App installation posts the
run (the workspace that received the webhook and owns the repo hook). Plus `repo_url`,
`head_ref`, `check_run_id` (NULL until the create succeeds, and reset to NULL by a re-fired event
for the same head: the row is written first so a create that never gets recorded cannot
strand an in-progress run, and the poller retries any row without an id), `created_at`,
`concluded`, `conclusion`, `concluded_at`, `github_posted`.
Partial index `(workspace_id) WHERE NOT concluded OR NOT github_posted` (the live set the
hook + poller scan).
- **Open** — in the `pull_request` handler (opened/synchronize/reopened, or edited with a
base change, base = tracked): when the head lives in the base repo, resolve the fork
workspace from the head ref (reusing the fork-branch routing;
`resolve_pr_head_workspace`), persist the intent row with a null check-run id, then
`create_check_run` in_progress on `head_sha` via the parent's installation and adopt the
id (the poller retries the create from the row if it failed), then evaluate. An earlier head's open check is left to conclude on its
own (fork verdict or timeout): a late-delivered event for an old head must never touch
the current head's check.
- **Conclude** — the verdict is the head's own suite. Once the fork reflects the head (below)
and its dependency jobs settled, every CI test the fork declares is dispatched once, one
run per `ci_test_reference` row the way a deploy of that item would (`trigger_all_ci_tests`,
as the fork's owner, the user who created it, with no more reach than they have; a
missing or disabled owner concludes the check as failure; and without the per-item
debounce so a deploy-triggered run of the same test cannot supersede a suite run), and the job ids are
recorded on the synced-head row (`ci_test_job_ids`; `tests_dispatched_at` claims the
dispatch so the per-job hook and the poller queue it once, and a claim that never recorded
ids is retaken after 5 min). The verdict is exactly those runs: fail-fast on any
failed/canceled; `success` once all settle ("No CI tests" when the fork declares none);
`skipped` ignored. Nothing older, newer or workspace-wide stands in
for a head's runs, so a re-fired event reads the same runs and gets the same answer, a
test-only change is run because the suite runs on every head, and a deploy in flight in
the fork cannot feed another head's check. Runs purged by job retention reset the row so
the head is re-tested.
- **Readiness** — the suite is dispatched only once the fork reflects the head, so it does
not matter which webhook GitHub delivers first. The evidence is `git_sync_synced_head`: the
pull completion hook writes a row when a pull job succeeds (the pull script reports the
commit its clone checked out; the enqueue-time marker is the fallback), and the push
completion hook writes one from the deploy push script's `{pushed, sha, branch, rebased}`
result (a rebased push sits on unpulled commits and is not recorded). The head is ready
when the repository branch's newest row names it, so a branch reset to an older commit
waits for its re-pull; the prune keeps each repository branch's newest row so a PR reopened
at an unchanged head stays ready. This is a sync event log, deliberately apart from
`auto_pull.last_synced_sha`: that map decides whether the next poll pulls (a push must
never write it, or a commit someone else pushed under ours would be skipped) and it is
client-round-tripped settings. The check row stores `head_ref` for the lookup. Dispatch
also waits while a dependency job in the fork or a pull of the repository branch is queued
(a deploy push lands on whatever the remote held when it cloned, so a commit pushed there
from outside is in the workspace only once its pull ran), and the check fails outright if
a dependency job failed after the head's pull started (the item deployed nothing
runnable). A commit the fork never comes to reflect times out; a timeout on a repository
pinned to a sync script older than the one that reports pushed commits names that as the
reason. Needs the hub script versions that report the sha (`LATEST_GIT_SYNC_SCRIPT_PATH`,
`GIT_SYNC_PULL_SCRIPT_PATH`).
- **Drivers** — a per-`ci_test`-job completion hook (low latency) and the git-sync poller
(the backstop: retries the GitHub create/deliver, times stuck checks out after 30 min,
prunes old rows; runs after the auto-pull advisory lock is released so its GitHub calls
never extend the tick). Both call one idempotent `evaluate_and_conclude`, which claims the
decision with a guarded `UPDATE ... WHERE NOT concluded RETURNING` (exactly-once) and
decouples GitHub delivery via `github_posted` so a failed PATCH is retried, not hung.
Invariants: only a head in the base repo can map to a workspace (a contributor fork's
branch names mean nothing here); the webhook's workspace posts through its own
installation; the timeout stops a hung test job from blocking a required check forever;
rows cascade away with either workspace. A plain feature-branch or
contributor-fork PR resolves to no fork workspace and gets an already-concluded `skipped`
check (branch protection counts `skipped` as passing, so requiring the check does not block
those PRs). A timeout on a repository pinned to a sync script older than the one that reports
pushed commits names that as the reason. Known limit (accepted for v1): the fork's status is workspace-wide (all its
tested items), which for the one-fork-per-PR model equals the PR's scope.
## 16. Alternatives considered
**Portal as webhook proxy (the rejected "option 2").** Subscribe the managed app
@@ -949,7 +949,7 @@
options={{
right: 'Automatically deploy changes from Git',
rightTooltip:
'Windmill deploys new commits from the tracked branch into this workspace. Repositories Windmill holds a credential for sync instantly via webhooks with a polling fallback; other token-based repositories are checked about every minute.'
'Windmill deploys new commits from the tracked branch into this workspace. Repositories Windmill holds a credential for sync instantly via webhooks with a polling fallback; other token-based repositories are checked about every minute. The webhook also delivers pull request events, which is what posts the diff check on pull requests and, for GitHub repositories, the Windmill CI tests check.'
}}
on:change={(e) => setAutoPullEnabled(e.detail)}
>
@@ -964,7 +964,7 @@
options={{
right: 'Automatically sync forks with git branches',
rightTooltip: repo.auto_pull?.enabled
? "When a fork's wm-fork/** branch changes in the repository (for example after merging the tracked branch into it), Windmill deploys those commits into the fork workspace. Configured once here, applied to every fork of this workspace."
? "When a fork's wm-fork/** branch changes in the repository (for example after merging the tracked branch into it), Windmill deploys those commits into the fork workspace. On GitHub, needed for the Windmill CI tests check on a fork pull request whose commits were pushed outside Windmill. Configured once here, applied to every fork of this workspace."
: 'Requires automatic deploy from Git to be enabled above.'
}}
on:change={(e) => setSyncForks(e.detail)}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"gitSyncTest": "hub/28950/git-repo-test-read-write-windmill",
"gitInitRepo": "hub/28948/git-sync-init-repository-windmill",
"gitInitRepo": "hub/28957/git-sync-init-repository-windmill",
"slackErrorHandler": "hub/28794/workspace-or-schedule-error-handler-slack",
"emailErrorHandler": "hub/19795/workspace-or-error-handler-email",
"slackRecoveryHandler": "hub/28791/slack/schedule-recovery-handler-slack",