feat: bring gitlab repositories to parity for git sync (#10938)

* feat: track and rotate gitlab git-sync repository tokens

* chore: point ee-repo-ref at the gitlab credential branch

* fix: strip server-owned credential status and correct expiry copy

* fix: gate credential maintenance on enterprise and alert on stalled renewal

* fix: alert on an auto-renewed token only once it has actually expired

* feat: receive gitlab push webhooks for instant git sync pull

* feat: open gitlab merge requests and post diff previews on them

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep gitlab merge request previews out of the project's own pipeline

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: bound the credential maintenance pass and gate the gitlab picker on a license

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: create the gitlab picker's variable in the edited workspace

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: make the gitlab picker's variable path collision-resistant

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: state the gitlab scope and rotation facts the code relies on

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: resolve the check marker's repository from its path, not a stored url

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: refuse to finish a check whose repository has been repointed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: trust a check marker's captured url when it carries no identity

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: recreate a missing webhook from credential maintenance

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: state that relative-url gitlab installs are out of scope

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep credential status out of exports and clear stale webhook warnings

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: refuse an unprovable check and guard the picker on the stored repository

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: re-check the picker's target path at the moment it is written

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: snapshot the picker's inputs before it starts writing

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: recommend a project access token per repository

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* [ee] feat: keep the git-sync credential in workspace settings

* [ee] fix: drop a removed repo's credential and honor the workspace override

* [ee] fix: resolve a fork's git credential from its whole ancestry

* [ee] refactor: reuse fork_ancestor_chain instead of a second ancestry walk

* [ee] fix: resolve an app installation from the whole ancestry, not the parent

* [ee] revert: keep the app installation fallback at one level

* fix: store the git credential only once the resource is saved

* fix: keep a repository's credential when it leaves git sync settings

* docs: cut the gitlab picker's token guidance down to what it needs

* feat: mark a repository whose credential windmill holds

* fix: ignore the managed-credential marker when the url carries a token

* docs: drop the picker's setup alert for a line by the token field

* feat: replace a repository's stored token from its resource

* fix: store a picked credential for its own workspace, before the resource

* refactor: key a stored git credential by its repository, not its resource

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: refresh the sqlx cache for the repository-keyed credential queries

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: gate the credential pass budget on the features that use it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: decide credential rotation ownership by repository, not resource path

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: renew only the credentials windmill holds, not tokens in a repo url

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: authenticate the fork-branch poll and correct the renewal guidance

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: do not claim a managed credential for a url the client cannot resolve

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: define the credential facade for private builds without enterprise

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: pin the listed token before the await and name the real renewal blocker

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: pin the token the replace flow checked, and derive the scope test once

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: classify the renewal state once so the card cannot contradict itself

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: ask only whether the token gets renewed, not why it does not

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: replace the managed-credential marker with a server answer

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: read renewal from the credential and its origin, not a removed field

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: read the provider for url-token repos, await the origin before defaulting, and visit unchecked repos last

The maintenance pass sorted repositories with no recorded check first on
the premise that they cost nothing, but a token-in-URL remote on a host
that is not GitLab is probed every pass and never records a check, so it
held the head of the list ahead of the tokens that expire. Such
repositories now sort last.

The card decided its delivery defaults before the origin lookup landed,
so a freshly picked GitLab repository never got webhook delivery; the two
lookups are awaited together. The resource editor offers to replace a
token only where it is held, not in a fork that borrows it, and the
replace flow refuses a URL it cannot parse instead of keying the token to
it. Attaching a stored credential to a commit-hash probe now requires
admin, matching the installation credential beside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: describe the gitlab listing token the way the picker and the setup guide do

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: a token in the repository url is a plain remote, not a tracked credential

Drops the status fingerprint that told one URL token from another, the
docs' promise that such a token's expiry is reported, and the test's
expectation that a URL-token repository declares a host.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: the card reads the credential origin for managed controls and honours the licence for a borrowed token

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: bump the ee ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: hide a repository's credential line once nothing is held for the repository it names

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: describe the exported credential status as it is

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: run the credential maintenance pass as its own task, without a budget

The pass ran inside the monitor's join, whose deadline cancels every
future in it, and a rotation cancelled between GitLab issuing a token and
Windmill storing it loses the token family. A wall-clock budget with a
least-recently-checked ordering kept it under the deadline. Spawning the
pass instead makes the deadline irrelevant, so the budget, the ordering
and the counter go; the advisory lock keeps a slow pass from overlapping
the next, as it already did.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: say what detaching the maintenance pass buys, and what it does not

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: run git sync on the hub script version that reads a stored credential

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: run the deploy push and the connection test on the hub versions that read a stored credential

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep App repositories and plain remotes out of the stored-credential paths

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: host-neutral deploy preview wording, drop the project filter from the GitLab picker

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: bump ee ref, rotation no longer retains a second connection per repository

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: bump ee ref, the rotation write-back holds a single connection

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: hold the credential maintenance lock in a transaction so a dead sweep releases it

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: describe the credential-stored callback as it fires

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep the credential maintenance lock past the pool's idle-in-transaction timeout

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: update ee-repo-ref to e092518ee60e33160fee9ae91a4d109566f7b0ee

This commit updates the EE repository reference after PR #771 was merged in windmill-ee-private.

Previous ee-repo-ref: 74481f7cc345757aebb2a8b04d3a22978328c348

New ee-repo-ref: e092518ee60e33160fee9ae91a4d109566f7b0ee

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
This commit is contained in:
hugocasa
2026-09-08 11:54:43 +02:00
committed by GitHub
co-authored by Claude Fable 5.1 windmill-internal-app[bot]
parent 6860521b4e
commit 9444049d60
39 changed files with 1975 additions and 99 deletions
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "1d8ccd32266637d7f7915f92a8483dce0c9986f3847227c2f66daa84d4109d7d"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT (\n SELECT elem\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' = $2\n )\n FROM workspace_settings WHERE workspace_id = $1 FOR UPDATE\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "elem",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "1fc9a0aeabb0a33efc37167be3de72dc33ca421338e04277ac91c745351f89f3"
}
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT elem->'credential'\n FROM workspace_settings, jsonb_array_elements(git_sync->'repositories') AS elem\n WHERE workspace_id = $1 AND elem->>'git_repo_resource_path' IN ($2, $3)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "48055203c97499ab4fc4dcbe9271d3f9e80375b52748c458854a4883a6ecc8f6"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_credentials = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' IS DISTINCT FROM $2\n ) || jsonb_build_array($3::jsonb)\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Jsonb"
]
},
"nullable": []
},
"hash": "64f72d7477f7c82d1596560ff6c31c79281d5cb2b0e81f1e832eac6eb3cc32e2"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "73a3417f5331032a9b2afcf5b1c260d5ead34a6a3783dd21db08c7ee049eb232"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_credentials = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' IS DISTINCT FROM $2\n ) || jsonb_build_array($3::jsonb)\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Jsonb"
]
},
"nullable": []
},
"hash": "9c0ebee945eee03d667a4f38d18a4d446b8ea00cf51e02138a06c245ab33252e"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_credentials = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' IS DISTINCT FROM $2\n ) || jsonb_build_array($3::jsonb)\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Jsonb"
]
},
"nullable": []
},
"hash": "af5609185c6218d12fcb4a4234cfd2a14b2e9e613bcf4bc80abbfd495637ab2f"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_advisory_unlock(hashtext($1)::bigint)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "pg_advisory_unlock",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "b2f91eb32edd8db1605aeb5a17b6e96cd3a980411d80c6b10bae949cf2c42385"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_sync = jsonb_set(git_sync, '{repositories}',\n COALESCE((SELECT jsonb_agg(\n CASE WHEN elem->>'git_repo_resource_path' IN ($2, $3)\n THEN CASE WHEN $4::jsonb = 'null'::jsonb\n THEN elem - 'credential'\n ELSE jsonb_set(elem, '{credential}', $4) END\n ELSE elem END)\n FROM jsonb_array_elements(git_sync->'repositories') AS elem), '[]'::jsonb)\n )\n WHERE workspace_id = $1\n AND jsonb_typeof(git_sync->'repositories') = 'array'\n AND EXISTS (\n SELECT 1 FROM jsonb_array_elements(git_sync->'repositories') AS e\n WHERE e->>'git_repo_resource_path' IN ($2, $3)\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Jsonb"
]
},
"nullable": []
},
"hash": "bbb8331348216892e4714e4338c496c9448cffba5786b51e748bec0165a6519c"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT elem\n FROM workspace_settings, jsonb_array_elements(git_credentials) AS elem\n WHERE workspace_id = $1 AND elem->>'repo_identity' = $2\n LIMIT 1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "elem",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "ce3340a43a141cc7527211db16905d53bc756d00c8d28462097717502d82cc15"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_try_advisory_lock(hashtext($1)::bigint)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "pg_try_advisory_lock",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "d0eeb992a826d26376e9802a8c4ced12d26f67f1b059c7c2afe66f8e4b3a9749"
}
@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "SELECT ws.workspace_id, ws.git_sync\n FROM workspace_settings ws\n JOIN workspace w ON w.id = ws.workspace_id\n WHERE NOT w.deleted\n AND ws.git_sync IS NOT NULL\n AND jsonb_typeof(ws.git_sync->'repositories') = 'array'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "git_sync",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
true
]
},
"hash": "ddd41de79b23b0436bbc7997751c90d7d82bd20ea30f1dc08f2e2066fb8d6f4b"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT (\n SELECT elem\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' = $2\n )\n FROM workspace_settings WHERE workspace_id = $1 FOR UPDATE\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "elem",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "f3a27f7781d986d9917a7022e1118313f0434758413dbf83f8575d09de98fc25"
}
+1 -1
View File
@@ -1 +1 @@
d33ea730c550cdbc7d050aeb6d40dcef3d134e07
e092518ee60e33160fee9ae91a4d109566f7b0ee
@@ -0,0 +1 @@
ALTER TABLE workspace_settings DROP COLUMN IF EXISTS git_credentials;
@@ -0,0 +1,8 @@
-- Server-owned git-sync credentials, one entry per repository the workspace
-- holds a token for, keyed by that repository rather than by a resource naming
-- it: a resource's URL is writable, and its path is not settled while it is
-- being created.
-- Kept out of `git_sync` because that column is copied into forks and returned
-- by the workspace settings API; this one is copied by neither.
ALTER TABLE workspace_settings
ADD COLUMN IF NOT EXISTS git_credentials JSONB NOT NULL DEFAULT '[]'::jsonb;
+178 -11
View File
@@ -354,7 +354,9 @@ pub async fn initial_load(
)
}
});
pass.action(windmill_common::min_version::store_min_keep_alive_version(db));
pass.action(windmill_common::min_version::store_min_keep_alive_version(
db,
));
pass.setting(
windmill_common::global_settings::INSTANCE_EVENTS_WEBHOOK_SETTING,
false,
@@ -709,7 +711,6 @@ pub async fn initial_load(
pass.run(conn).await;
}
pub fn apply_metrics_enabled(value: Option<serde_json::Value>) {
if let Some(serde_json::Value::Bool(t)) = value {
METRICS_ENABLED.store(t, Ordering::Relaxed)
@@ -1066,8 +1067,8 @@ pub fn apply_fork_workspace_tag_append_fork_suffix(value: Option<serde_json::Val
}
pub async fn reload_critical_alert_mute_ui_setting(conn: &Connection) -> error::Result<()> {
let v =
load_value_from_global_settings_with_conn(conn, CRITICAL_ALERT_MUTE_UI_SETTING, true).await?;
let v = load_value_from_global_settings_with_conn(conn, CRITICAL_ALERT_MUTE_UI_SETTING, true)
.await?;
apply_critical_alert_mute_ui_setting(v);
Ok(())
}
@@ -2732,7 +2733,6 @@ pub async fn reload_timeout_wait_result_setting(conn: &Connection) {
.await;
}
pub async fn reload_extra_pip_index_url_setting(conn: &Connection) {
reload_option_setting_with_tracing(
conn,
@@ -2823,7 +2823,6 @@ pub async fn reload_bunfig_install_scopes_setting(conn: &Connection) {
.await;
}
pub async fn reload_nuget_config_setting(conn: &Connection) {
reload_option_setting_with_tracing(
conn,
@@ -2931,7 +2930,6 @@ pub async fn reload_ruby_repos_setting(conn: &Connection) {
.await;
}
pub async fn reload_workspace_registries_setting(conn: &Connection) {
match load_value_from_global_settings_with_conn(
conn,
@@ -3185,7 +3183,6 @@ pub async fn apply_job_isolation_setting(value: Option<serde_json::Value>) {
}
}
async fn resolve_license_key_value(conn: &Connection, quiet: bool) -> anyhow::Result<String> {
let q = load_value_from_global_settings_with_conn(conn, LICENSE_KEY_SETTING, true)
.await
@@ -3480,7 +3477,10 @@ impl<'a> SettingsPass<'a> {
// on compile-time defaults until the next full reload. Only the single-query transport
// can fail this way; over HTTP the batch already is the per-setting read.
if matches!(conn, Connection::Sql(_)) && values.is_empty() && !names.is_empty() {
tracing::warn!("Falling back to per-setting reads for {} settings", names.len());
tracing::warn!(
"Falling back to per-setting reads for {} settings",
names.len()
);
values = fetch_settings_individually(conn, &names).await;
}
for (name, http) in &declared {
@@ -3872,7 +3872,6 @@ pub fn parse_setting_value<T: FromStr + DeserializeOwned + Display>(
value
}
#[cfg(feature = "prometheus")]
pub async fn monitor_pool(db: &DB) {
if METRICS_ENABLED.load(Ordering::Relaxed) {
@@ -4316,6 +4315,24 @@ pub async fn monitor_db(
}
};
// Re-check what each git-sync repository's own credential says about its expiry,
// and rotate the ones close to it. Every ~40 min: the values move over days, and
// `should_run` counts iterations in a u8. Spawned rather than joined: the join
// below is cancelled at its deadline, which a long sweep would reach, and a
// rotation cut off between GitLab issuing a token and Windmill storing it
// loses the token family. Detached, only process shutdown can cut it off,
// which a rotation almost never coincides with. The pass's advisory lock
// keeps a slow one from overlapping the next.
let git_credential_maintenance_f = async {
#[cfg(all(feature = "enterprise", feature = "private"))]
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(240) {
if let Some(db) = conn.as_sql() {
let db = db.clone();
tokio::spawn(async move { maintain_git_credentials(&db).await });
}
}
};
// run every 2 iterations (~20s at the default LISTEN_NEW_EVENTS_INTERVAL_SEC).
// Enterprise feature: the active `// freshness` backstop lives in
// windmill-queue's `freshness_watchdog` (`private`); OSS gets a no-op stub.
@@ -4369,6 +4386,7 @@ pub async fn monitor_db(
export_audit_logs_to_object_store_f,
cleanup_scheduled_job_deletions_f,
git_auto_pull_f,
git_credential_maintenance_f,
pipeline_freshness_watchdog_f,
reconcile_unarmed_schedules_f,
);
@@ -4691,6 +4709,156 @@ lazy_static::lazy_static! {
#[cfg(feature = "private")]
const AUTO_PULL_POLL_SLACK_S: i64 = 30;
/// Advisory lock id ensuring only one server replica maintains git credentials at
/// a time (adjacent to GIT_AUTO_PULL_LOCK_ID).
#[cfg(all(feature = "enterprise", feature = "private"))]
const GIT_CREDENTIAL_LOCK_ID: i64 = 737_483_923;
/// Refresh every git-sync repository's credential status and rotate the ones near
/// expiry, so a token dies visibly (and usually not at all) rather than taking
/// sync down on its expiry date.
#[cfg(all(feature = "enterprise", feature = "private"))]
async fn maintain_git_credentials(db: &Pool<Postgres>) {
use windmill_common::ee_oss::{get_license_plan, LicensePlan};
if !matches!(get_license_plan().await, LicensePlan::Enterprise) {
return;
}
// Transaction-scoped advisory lock, as for the schedule reconcile above: a
// session lock on a pooled connection would ride back into the pool still
// held if the sweep died before unlocking, and wedge the pass on every
// replica until a restart. The transaction only owns the lock; the sweep
// commits each status and each rotated token on its own as it goes.
let mut lock_tx = match db.begin().await {
Ok(tx) => tx,
Err(e) => {
tracing::error!("git credentials: failed to begin lock tx: {e:#}");
return;
}
};
// The transaction stays idle while the sweep talks to git hosts, and the
// pool's ten-minute idle-in-transaction timeout would end it, lock included,
// partway through a sweep over enough slow hosts. Lifted for this
// transaction only; it dies with the connection either way.
if let Err(e) = sqlx::query("SET LOCAL idle_in_transaction_session_timeout = 0")
.execute(&mut *lock_tx)
.await
{
tracing::error!("git credentials: failed to lift the idle timeout: {e:#}");
return;
}
let locked: bool = match sqlx::query_scalar("SELECT pg_try_advisory_xact_lock($1)")
.bind(GIT_CREDENTIAL_LOCK_ID)
.fetch_one(&mut *lock_tx)
.await
{
Ok(v) => v,
Err(e) => {
tracing::error!("git credentials: advisory lock failed: {e:#}");
return;
}
};
if !locked {
return;
}
if let Err(e) = maintain_git_credentials_inner(db).await {
tracing::error!("git credentials: maintenance error: {e:#}");
}
drop(lock_tx);
}
#[cfg(all(feature = "enterprise", feature = "private"))]
async fn maintain_git_credentials_inner(db: &Pool<Postgres>) -> error::Result<()> {
use windmill_common::workspaces::WorkspaceGitSyncSettings;
// Same deleted/archived exclusion as the auto-pull poller: a dead workspace's
// settings row survives, and rotating a token for one would be pure damage.
let rows = sqlx::query!(
r#"SELECT ws.workspace_id, ws.git_sync
FROM workspace_settings ws
JOIN workspace w ON w.id = ws.workspace_id
WHERE NOT w.deleted
AND ws.git_sync IS NOT NULL
AND jsonb_typeof(ws.git_sync->'repositories') = 'array'"#
)
.fetch_all(db)
.await?;
for row in rows {
let Some(git_sync) = row.git_sync else {
continue;
};
let settings: WorkspaceGitSyncSettings = match serde_json::from_value(git_sync) {
Ok(s) => s,
Err(e) => {
tracing::warn!(
"git credentials: invalid git_sync settings for workspace {}: {e}",
row.workspace_id
);
continue;
}
};
for repo in settings.repositories.iter() {
let path = &repo.git_repo_resource_path;
// This refreshes and records the status on every repository it looks at,
// rotating only the ones near expiry, so it is the whole maintenance pass
// rather than just the rotation half.
if let Err(e) = windmill_common::git_sync_ee::rotate_git_credential_if_due(
db,
&row.workspace_id,
path,
)
.await
{
tracing::error!(
"git credentials: maintenance failed for {path} in workspace {}: {e:#}",
row.workspace_id
);
}
// A repository that wants webhook delivery but holds no hook never
// gets one otherwise: the reconcile runs on a settings save, so a
// credential that was unusable when the hook should have been created
// would leave it missing until an admin saved again. Checking stored
// state costs nothing, and only the repositories actually missing a
// hook reach the host.
use windmill_common::workspaces::AutoPullMode;
// Also when a hook exists but carries a warning: a save during a GitLab
// outage keeps the hook and records why it could not be confirmed, and
// that warning is only cleared by a reconcile that confirms it again.
// Only repositories with a checked credential of their own: for
// everything else a settings save stays the one place hooks are
// reconciled, so an App repository or a plain remote is never touched
// here, and a recorded delivery mode nobody saved is never normalized.
let needs_hook = repo.credential.is_some()
&& repo.auto_pull.as_ref().is_some_and(|a| {
a.enabled
&& matches!(a.mode, AutoPullMode::Auto | AutoPullMode::Webhook)
&& (a.webhook_id.is_none() || a.webhook_error.is_some())
});
if needs_hook {
let mut repo = repo.clone();
if let Err(e) = windmill_common::git_sync_ee::sync_repo_webhook(
db,
&row.workspace_id,
&mut repo,
)
.await
{
tracing::warn!(
"git credentials: could not reconcile the webhook for {path} in workspace {}: {e:#}",
row.workspace_id
);
}
}
}
}
Ok(())
}
#[cfg(feature = "private")]
async fn poll_git_auto_pull_inner(db: &Pool<Postgres>) -> error::Result<()> {
use windmill_common::workspaces::{AutoPullMode, WorkspaceGitSyncSettings};
@@ -6588,7 +6756,6 @@ pub async fn reload_critical_alerts_on_db_oversize(conn: &DB) -> error::Result<(
Ok(())
}
pub async fn reload_jwt_secret_setting(db: &DB) -> error::Result<()> {
let v = load_value_from_global_settings(db, JWT_SECRET_SETTING).await?;
apply_jwt_secret_setting(db, v).await
+55
View File
@@ -0,0 +1,55 @@
-- A parent whose git-sync repository has a recorded credential, and the workspace
-- shapes the credential lookup and the qualification predicate have to tell apart.
--
-- The credential itself is shared down the fork chain; the recorded *status* is
-- not, because it describes one repository and a fork can repoint its copy of the
-- resource. Forks get a status by fork creation copying it, which no fixture here
-- simulates, so a fork without one is a workspace nothing has checked yet.
INSERT INTO workspace (id, name, owner, parent_workspace_id) VALUES
('parent-ws', 'parent-ws', 'test-user', NULL),
('fork-ws', 'fork-ws', 'test-user', 'parent-ws'),
-- A fork of a fork: the shape a fork of a dev workspace takes, and the one a
-- parent-only lookup misses.
('deep-fork-ws', 'deep-fork-ws', 'test-user', 'fork-ws'),
('errored-fork-ws', 'errored-fork-ws', 'test-user', 'parent-ws'),
('orphan-ws', 'orphan-ws', 'test-user', NULL);
-- A stored credential is encrypted with its own workspace's key.
INSERT INTO workspace_key (workspace_id, kind, key) VALUES
('parent-ws', 'cloud', 'parent-key'),
('fork-ws', 'cloud', 'fork-key'),
('deep-fork-ws', 'cloud', 'deep-fork-key'),
('errored-fork-ws', 'cloud', 'errored-fork-key'),
('orphan-ws', 'cloud', 'orphan-key');
-- The parent holds the credential.
INSERT INTO workspace_settings (workspace_id, git_sync) VALUES
('parent-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo",
"credential":{"provider":"gitlab","rotatable":true,"checked_at":1788500000}}]}'),
-- A fork inherits the repository but not the credential: this is what
-- clone_workspace_data leaves behind.
('fork-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo"}]}'),
-- Two levels down, so neither the credential nor its status is one hop away.
('deep-fork-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo"}]}'),
-- A fork whose own credential has since failed. Its own standing must win over
-- the parent's healthy record rather than being papered over.
('errored-fork-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo",
"credential":{"provider":"gitlab","rotatable":false,"checked_at":1788500000,
"error":"GitLab no longer accepts this token"}}]}'),
-- No credential and no parent to borrow one from.
('orphan-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo"}]}');
-- The resource each repository entry names, all pointing at the same repository.
-- The errored fork carries its token in the URL, the way a repository configured
-- by hand does: a plain remote, whatever the chain above it holds.
INSERT INTO resource (workspace_id, path, value, resource_type) VALUES
('parent-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository'),
('fork-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository'),
('deep-fork-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository'),
('errored-fork-ws', 'u/admin/repo', '{"url":"https://oauth2:glpat-inline@gitlab.com/grp/proj.git"}', 'git_repository'),
('orphan-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository');
+252
View File
@@ -0,0 +1,252 @@
//! A fork reaches the git credential held above it in its fork chain.
//!
//! Fork creation copies the parent's git-sync repositories but not the credential,
//! which is stored per workspace so that rotation has one owner. Chains nest (a
//! fork of a dev workspace, a fork of that), so the depth-2 cases here are what
//! keep the lookup from regressing to the parent.
//!
//! The recorded *status* is not shared the same way: it describes one repository,
//! and a fork can repoint its copy of the resource, so each workspace answers from
//! its own record and gets one by fork creation copying it down.
#![cfg(all(feature = "enterprise", feature = "private"))]
use sqlx::{Pool, Postgres};
use windmill_common::git_sync_ee::{
git_credential_for_url, repo_provider, repo_supports_managed_git_features, set_git_credential,
GitProvider,
};
use windmill_common::workspaces::GitCredentialProvider;
const REPO: &str = "$res:u/admin/repo";
const URL: &str = "https://gitlab.com/grp/proj.git";
/// A repository is managed when a credential is held for the repository its
/// URL names now and the last check found it healthy. The recorded status is
/// keyed by resource path, so alone it would outlive a repoint; the held
/// credential alone says nothing about whether the host still accepts it.
#[sqlx::test(fixtures("git_sync_fork_credential"))]
async fn credential_status_is_a_workspaces_own(db: Pool<Postgres>) -> anyhow::Result<()> {
assert!(
!repo_supports_managed_git_features(&db, "parent-ws", REPO).await,
"a healthy status with nothing held behind it does not qualify"
);
set_git_credential(
&db,
"parent-ws",
URL,
"glpat-secret",
GitCredentialProvider::Gitlab,
)
.await?;
assert!(
repo_supports_managed_git_features(&db, "parent-ws", REPO).await,
"the workspace holding both the credential and the recorded status qualifies"
);
assert!(
!repo_supports_managed_git_features(&db, "fork-ws", REPO).await,
"a fork borrowing the credential with no record of its own does not: the \
status describes one repository, and this fork's resource could name another"
);
assert!(
!repo_supports_managed_git_features(&db, "errored-fork-ws", REPO).await,
"a workspace whose own credential failed stays disqualified"
);
Ok(())
}
/// The host a repository talks to is declared when its credential is stored, and
/// travels with the credential down the fork chain.
///
/// Read from the recorded status instead, a fork answered with the default
/// provider until its own check ran, which is long enough to register a webhook
/// against the wrong receiver.
#[sqlx::test(fixtures("git_sync_fork_credential"))]
async fn the_provider_comes_from_the_credential_and_reaches_forks(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
assert_eq!(
repo_provider(&db, "parent-ws", REPO).await,
GitProvider::GitHub,
"with nothing stored there is no declaration to read, so the default stands"
);
set_git_credential(
&db,
"parent-ws",
URL,
"glpat-secret",
GitCredentialProvider::Gitlab,
)
.await?;
assert_eq!(
repo_provider(&db, "parent-ws", REPO).await,
GitProvider::GitLab,
"the workspace that stored it reads its own declaration"
);
assert_eq!(
repo_provider(&db, "fork-ws", REPO).await,
GitProvider::GitLab,
"and a fork resolving that credential reads it too, without a check of its own"
);
assert_eq!(
repo_provider(&db, "deep-fork-ws", REPO).await,
GitProvider::GitLab,
"two levels down as well"
);
assert_eq!(
repo_provider(&db, "orphan-ws", REPO).await,
GitProvider::GitHub,
"a workspace outside the chain resolves no credential and no declaration"
);
assert_eq!(
repo_provider(&db, "errored-fork-ws", REPO).await,
GitProvider::GitHub,
"a token written into the URL makes the repository a plain remote: the \
parent's credential is not consulted and no host is declared"
);
Ok(())
}
/// The stored credential is shared with forks and keyed by one repository.
///
/// Both properties are the point of keeping it in `workspace_settings` under the
/// repository's identity: sharing is what stops a rotation from stranding every
/// fork on a revoked token, and the key is what stops a rewritten resource URL
/// from carrying the token to a host of the writer's choosing.
#[sqlx::test(fixtures("git_sync_fork_credential"))]
async fn a_fork_reads_an_ancestors_credential_for_the_bound_repository_only(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
set_git_credential(
&db,
"parent-ws",
URL,
"glpat-secret",
GitCredentialProvider::Gitlab,
)
.await?;
assert_eq!(
git_credential_for_url(&db, "parent-ws", URL)
.await?
.as_deref(),
Some("glpat-secret"),
"the workspace that stored it reads it back"
);
assert_eq!(
git_credential_for_url(&db, "fork-ws", URL)
.await?
.as_deref(),
Some("glpat-secret"),
"a fork stores none of its own and resolves the parent's"
);
assert_eq!(
git_credential_for_url(&db, "deep-fork-ws", URL)
.await?
.as_deref(),
Some("glpat-secret"),
"a fork of a fork resolves the root's, two levels up"
);
assert_eq!(
git_credential_for_url(&db, "fork-ws", "https://evil.example/grp/proj.git").await?,
None,
"a resource repointed at another repository asks for that one's \
credential and finds none"
);
assert_eq!(
git_credential_for_url(&db, "orphan-ws", URL).await?,
None,
"a workspace with no credential and no parent resolves nothing"
);
Ok(())
}
/// One repository's credential is untouched by another's.
///
/// The key is the repository, so picking a second repository stores beside the
/// first rather than over it. Keyed by the resource instead, a workspace editing
/// one repository's resource to point somewhere else would replace the token the
/// original repository was still syncing with.
#[sqlx::test(fixtures("git_sync_fork_credential"))]
async fn each_repository_keeps_its_own_credential(db: Pool<Postgres>) -> anyhow::Result<()> {
const OTHER_URL: &str = "https://gitlab.com/grp/other.git";
set_git_credential(
&db,
"parent-ws",
URL,
"glpat-first",
GitCredentialProvider::Gitlab,
)
.await?;
set_git_credential(
&db,
"parent-ws",
OTHER_URL,
"glpat-second",
GitCredentialProvider::Gitlab,
)
.await?;
assert_eq!(
git_credential_for_url(&db, "parent-ws", URL)
.await?
.as_deref(),
Some("glpat-first"),
"storing a second repository's token leaves the first's in place"
);
assert_eq!(
git_credential_for_url(&db, "parent-ws", OTHER_URL)
.await?
.as_deref(),
Some("glpat-second")
);
set_git_credential(
&db,
"parent-ws",
URL,
"glpat-replacement",
GitCredentialProvider::Gitlab,
)
.await?;
assert_eq!(
git_credential_for_url(&db, "parent-ws", URL)
.await?
.as_deref(),
Some("glpat-replacement"),
"storing the same repository again replaces rather than duplicates"
);
assert_eq!(
git_credential_for_url(&db, "parent-ws", OTHER_URL)
.await?
.as_deref(),
Some("glpat-second"),
"and still leaves the other repository alone"
);
Ok(())
}
/// A credential issued for `https` is not served for the `http` spelling.
///
/// The resource holding the URL is writable by anyone with write on its path, so
/// without the scheme in the key that edit would send the token over cleartext.
#[sqlx::test(fixtures("git_sync_fork_credential"))]
async fn a_credential_is_not_served_over_a_downgraded_transport(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
set_git_credential(
&db,
"parent-ws",
URL,
"glpat-secret",
GitCredentialProvider::Gitlab,
)
.await?;
assert_eq!(
git_credential_for_url(&db, "parent-ws", "http://gitlab.com/grp/proj.git").await?,
None
);
Ok(())
}
@@ -3986,6 +3986,7 @@ async fn edit_git_sync_config(
clear_client_supplied_auto_pull_state(ap);
}
repo.open_pr_error = None;
repo.credential = None;
}
reject_parent_only_git_sync_settings_on_fork(
&db,
@@ -4086,6 +4087,7 @@ async fn edit_git_sync_config(
continue;
};
repo.open_pr_error = old.open_pr_error.clone();
repo.credential = old.credential.clone();
if let (Some(new_ap), Some(old_ap)) =
(repo.auto_pull.as_mut(), old.auto_pull.as_ref())
{
@@ -4131,6 +4133,7 @@ async fn edit_git_sync_config(
.flatten()
.and_then(|v| serde_json::from_value(v).ok());
let removed_webhooks: Vec<(String, i64)> = existing
.as_ref()
.map(|e| {
e.repositories
.iter()
@@ -4164,6 +4167,19 @@ async fn edit_git_sync_config(
// `sync_repo_webhook` writes back the webhook fields it changes itself:
// the remote hook and the record of it have to move together, so
// persisting them out here would let one land without the other.
// Before the webhook reconcile, which decides whether this repo can have
// one from the credential this records. Also puts a short-lived or
// under-scoped token in front of the operator while they are still on the
// settings page, rather than when it expires.
if let Err(e) = windmill_common::git_sync_ee::refresh_git_credential_status(
&db,
&w_id,
&repo.git_repo_resource_path,
)
.await
{
tracing::warn!("git credential check error: {}", e);
}
if let Err(e) = windmill_common::git_sync_ee::sync_repo_webhook(&db, &w_id, repo).await
{
tracing::warn!("git auto-pull: webhook sync error: {}", e);
@@ -4217,6 +4233,7 @@ async fn edit_git_sync_repository(
clear_client_supplied_auto_pull_state(ap);
}
new_config.repository.open_pr_error = None;
new_config.repository.credential = None;
reject_parent_only_git_sync_settings_on_fork(
&db,
&w_id,
@@ -4341,6 +4358,7 @@ async fn edit_git_sync_repository(
// from the UI cannot revert what the poller/webhook layer wrote.
let mut updated = new_config.repository;
updated.open_pr_error = existing_repo.open_pr_error.clone();
updated.credential = existing_repo.credential.clone();
match (updated.auto_pull.as_mut(), existing_repo.auto_pull.as_ref()) {
(Some(new_ap), Some(old_ap)) => {
new_ap.last_synced_sha = old_ap.last_synced_sha.clone();
@@ -4398,6 +4416,15 @@ async fn edit_git_sync_repository(
.iter_mut()
.find(|r| r.git_repo_resource_path == new_config.git_repo_resource_path)
{
if let Err(e) = windmill_common::git_sync_ee::refresh_git_credential_status(
&db,
&w_id,
&repo.git_repo_resource_path,
)
.await
{
tracing::warn!("git credential check error: {}", e);
}
if let Err(e) = windmill_common::git_sync_ee::sync_repo_webhook(&db, &w_id, repo).await {
tracing::warn!("git auto-pull: webhook sync error: {}", e);
}
@@ -4537,6 +4564,10 @@ async fn delete_git_sync_repository(
}
}
// The stored credential is deliberately left alone: it belongs to the git
// repository resource, which this endpoint does not delete, and the resource
// still authenticates with it for connection tests and commit lookups.
// Trigger git sync for repository deletion
handle_deployment_metadata(
&authed.email,
@@ -6306,9 +6337,10 @@ async fn update_workspace_settings(
// Auto-pull and fork PRs are parent-owned and must not be inherited:
// the fork would otherwise carry the parent's webhook id (turning off
// auto-pull on the fork would delete the parent's webhook). A fork
// still inherits the push-direction config and the installation.
// Repo → fork sync is driven by the parent's webhook/poller
// (`sync_forks`), which routes the fork's `wm-fork/**` branch into it.
// still inherits the push-direction config, the installation, and the
// recorded credential status, which describes the repository rather
// than belonging to either workspace and would otherwise leave the
// fork unqualified for managed features until its first check.
r.auto_pull = None;
r.fork_open_prs = false;
r.open_pr_error = None;
@@ -113,7 +113,7 @@ pub(crate) async fn change_workspace_id(
// Duplicate workspace settings (keep copy in old workspace for reference)
info!("Duplicating workspace_settings table");
sqlx::query!(
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
+176 -2
View File
@@ -2741,7 +2741,7 @@ paths:
/w/{workspace}/github_app/token:
post:
summary: get github app token
summary: get the git credential for a git-sync job (GitHub App token, or the credential stored for the repository)
operationId: getGithubAppToken
tags:
- workspace
@@ -2761,7 +2761,7 @@ paths:
- job_token
responses:
"200":
description: github app token
description: git credential
content:
application/json:
schema:
@@ -2905,6 +2905,131 @@ paths:
"200":
description: Successfully imported the installation
/w/{workspace}/git_sync/gitlab/projects:
post:
tags:
- Git Sync
summary: List the GitLab projects a token can sync
description: >-
Lists the projects the supplied GitLab token can push to, so a git
repository resource can be filled in without hand-writing a project
path. The token is used for this call only and is never stored.
Requires workspace admin.
operationId: listGitlabProjects
parameters:
- $ref: "#/components/parameters/WorkspaceId"
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
base_url:
type: string
description: The GitLab instance, e.g. https://gitlab.com
token:
type: string
description: A project access token with the api scope, or a group token that reaches the project
search:
type: string
description: Narrow the list to projects matching this text
required:
- base_url
- token
responses:
"200":
description: the projects the token can sync
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/GitlabProject"
/w/{workspace}/git_sync/credential/origin:
get:
tags:
- Git Sync
summary: Where a repository's credential comes from
description: >-
Whether Windmill holds this repository's access token, and which host it
talks to. `held` means this workspace stores it, `borrowed` means an
ancestor does and it is not this workspace's to replace. Both absent
means the repository authenticates with whatever its URL carries.
Returns no secret. Requires workspace admin.
operationId: getCredentialOrigin
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: path
in: query
required: true
description: >-
Path of the git repository resource, with or without the `$res:`
prefix. A path rather than a URL, because a resource URL may carry a
token and a URL in a query string lands in logs.
schema:
type: string
responses:
"200":
description: where the credential comes from
content:
application/json:
schema:
type: object
properties:
origin:
type: string
enum:
- held
- borrowed
provider:
type: string
enum:
- gitlab
/w/{workspace}/git_sync/credential:
post:
tags:
- Git Sync
summary: Store the credential for a git repository
description: >-
Stores the access token a git repository authenticates with, so it does
not have to be written into the repository URL or a workspace variable.
The token is write-only: it is served only to a git-sync job that
presents its own job token, and a fork of this workspace reads this
copy instead of holding one of its own. Requires workspace admin.
operationId: setGitCredential
parameters:
- $ref: "#/components/parameters/WorkspaceId"
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
repo_url:
type: string
description: >-
The repository the credential is for, and the key it is
stored under. It is served for this repository and no other,
so repointing a resource elsewhere cannot carry the token
along.
token:
type: string
description: The access token, as pasted
required:
- repo_url
- token
responses:
"200":
description: the credential was stored
content:
text/plain:
schema:
type: string
/w/{workspace}/github_app/ghes_installation_callback:
post:
summary: GHES installation callback
@@ -34484,9 +34609,58 @@ components:
open_pr_error:
type: string
description: server-owned, last failure opening a PR for a deploy branch of this repo
credential:
$ref: "#/components/schemas/GitCredentialStatus"
required:
- git_repo_resource_path
GitCredentialStatus:
type: object
description: server-owned, what the repo's own credential reports about itself
properties:
provider:
type: string
enum:
- gitlab
token_id:
type: integer
format: int64
expires_at:
type: string
format: date
description: absent for a non-expiring token
scopes:
type: array
items:
type: string
rotatable:
type: boolean
description: whether this workspace renews the credential itself
checked_at:
type: integer
format: int64
error:
type: string
required:
- provider
- rotatable
- checked_at
GitlabProject:
type: object
description: a GitLab project a token can sync, as the resource form needs it
properties:
path_with_namespace:
type: string
description: nested group path plus project name, which is also GitLab's project id
http_url_to_repo:
type: string
default_branch:
type: string
required:
- path_with_namespace
- http_url_to_repo
AutoPullMode:
type: string
enum:
+5
View File
@@ -10,6 +10,11 @@ pub fn workspaced_service() -> Router {
Router::new()
}
#[cfg(not(feature = "private"))]
pub fn workspaced_git_sync_service() -> Router {
Router::new()
}
#[cfg(not(feature = "private"))]
pub fn global_service() -> Router {
Router::new()
+9
View File
@@ -959,6 +959,15 @@ pub async fn run_server(
#[cfg(not(feature = "enterprise"))]
Router::new()
})
.nest("/w/{workspace_id}/git_sync", {
#[cfg(feature = "enterprise")]
{
git_sync_oss::workspaced_git_sync_service()
}
#[cfg(not(feature = "enterprise"))]
Router::new()
})
.nest(
"/w/{workspace_id}/resources_u",
public_service().layer(cors.clone()),
+12 -4
View File
@@ -1586,10 +1586,11 @@ pub(crate) async fn tarball_workspace(
.await?;
// Use v2 format only if explicitly requested, otherwise use v1 (legacy) for backward compatibility
// Server-owned auto-pull state (the HMAC webhook secret + hook id/error and
// the synced-sha / last-pull status) must never leave the server: keep it out
// of export archives and synced repos, and don't let a re-imported workspace
// inherit another install's hook/sync state. Mirrors the GET-settings redaction.
// Server-owned state (the HMAC webhook secret + hook id/error, the
// synced-sha / last-pull status, and what the credential check observed)
// must never leave the server: keep it out of export archives and synced
// repos, and don't let a re-imported workspace inherit another install's
// hook/sync state. Mirrors the GET-settings redaction.
fn redact_git_sync_for_export(git_sync: Option<Value>) -> Option<Value> {
let mut git_sync = git_sync?;
if let Some(repos) = git_sync
@@ -1610,6 +1611,13 @@ pub(crate) async fn tarball_workspace(
auto_pull.remove(field);
}
}
// What this install observed about its own credential: a token
// id and expiry, and a `checked_at` that moves on its own.
// None of it describes the workspace, and in a git-synced
// `wmill.yaml` it would churn the file for no reason.
if let Some(repo) = repo.as_object_mut() {
repo.remove("credential");
}
}
}
Some(git_sync)
+22 -2
View File
@@ -1,11 +1,16 @@
#[cfg(feature = "private")]
#[allow(unused)]
pub use crate::git_sync_ee::*;
#[cfg(not(feature = "private"))]
#[cfg(not(all(feature = "private", feature = "enterprise")))]
use sqlx::{Pool, Postgres};
use url::Url;
#[cfg(not(feature = "private"))]
/// Gated on the pair to match [`with_stored_credential`] below, whose callers
/// reach it through this facade un-gated and so depend on it. Nothing routes
/// here today (the one caller imports the enterprise item directly), so this is
/// for uniformity: the next plain caller would otherwise find no definition
/// under `private` without `enterprise`.
#[cfg(not(all(feature = "private", feature = "enterprise")))]
pub async fn get_github_app_token_internal(
_db: &Pool<Postgres>,
_job_token: &str,
@@ -15,6 +20,21 @@ pub async fn get_github_app_token_internal(
));
}
/// Server-held git credentials are an enterprise feature, so on this build a
/// repository URL authenticates with whatever it already carries.
///
/// Gated on the pair rather than on `private` alone: `private` does not imply
/// `enterprise`, and the callers are plain (no `#[cfg]`), so a build with one
/// and not the other would find neither this nor the enterprise definition.
#[cfg(not(all(feature = "private", feature = "enterprise")))]
pub async fn with_stored_credential(
_db: &Pool<Postgres>,
_w_id: &str,
url: String,
) -> crate::error::Result<String> {
Ok(url)
}
lazy_static::lazy_static! {
/// Matches a `user:password@` (or `user@`) userinfo component right after the URL scheme.
static ref GIT_URL_USERINFO_RE: regex::Regex =
+48 -5
View File
@@ -183,7 +183,7 @@ pub enum ObjectType {
DatatableMigration,
}
pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28931/sync-script-to-git-repo-windmill";
pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28949/sync-script-to-git-repo-windmill";
/// Hub script that applies a repository's state back into a workspace
/// (the repo → Windmill / "pull" direction). Same script the UI runs from
@@ -191,7 +191,7 @@ pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28931/sync-script-to-git-repo
/// ignores the slug, so the slug is kept free of characters that would be
/// percent-encoded into the run URL (a `:` becomes `%3A`, which some hardened
/// reverse proxies reject as double-encoding when the client re-encodes it).
pub const GIT_SYNC_PULL_SCRIPT_PATH: &str = "hub/28930/git-sync-init-repository-windmill";
pub const GIT_SYNC_PULL_SCRIPT_PATH: &str = "hub/28948/git-sync-init-repository-windmill";
/// Prefix used to identify fork workspaces. A workspace whose id starts with this string is a
/// fork of another workspace.
@@ -345,13 +345,14 @@ pub struct GitRepositorySettings {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub auto_pull: Option<AutoPullSettings>,
/// Open a PR when a deploy pushes a `wm_deploy/**` branch of this promotion
/// repo (app-backed only; runs from the deploy callback so it works without
/// repo (needs a credential the server holds — a GitHub App installation or
/// a checked GitLab token; runs from the deploy callback so it works without
/// inbound webhooks). Off by default so upgrades don't change behavior.
#[serde(default, skip_serializing_if = "is_false")]
pub promotion_open_prs: bool,
/// Parent-level: open a PR when a fork of this workspace deploys to its
/// `wm-fork/**` branch (app-backed only; the fork's deploy callback reads
/// this from the parent). Off by default.
/// `wm-fork/**` branch (needs a credential the server holds; the fork's
/// deploy callback reads this from the parent). Off by default.
#[serde(default, skip_serializing_if = "is_false")]
pub fork_open_prs: bool,
/// Server-owned: the last failure opening a PR for a deploy branch of this
@@ -360,6 +361,10 @@ pub struct GitRepositorySettings {
/// successful PR; never accepted from clients.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub open_pr_error: Option<String>,
/// Server-owned: what the repo's credential says about its own expiry and
/// scopes. Written by the credential check, never accepted from clients.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub credential: Option<GitCredentialStatus>,
}
impl GitRepositorySettings {
@@ -405,6 +410,44 @@ pub enum AutoPullMode {
Polling,
}
/// Host whose credential lifecycle Windmill can manage from the repo URL.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum GitCredentialProvider {
Gitlab,
}
/// What the repo's own credential says about itself, refreshed by asking the
/// host. Server-owned: written by the credential check, never accepted from a
/// client.
///
/// Absent means the check has not run or the repo carries no credential we can
/// introspect (a GitHub App repo mints tokens per call and has nothing to expire).
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct GitCredentialStatus {
pub provider: GitCredentialProvider,
/// Changes on every rotation, so it identifies the current token, not the
/// credential's whole history.
#[serde(skip_serializing_if = "Option::is_none")]
pub token_id: Option<i64>,
/// `None` is a non-expiring token, which only self-managed GitLab can issue
/// (and only for a service account). It means no warning and no rotation.
#[serde(skip_serializing_if = "Option::is_none")]
pub expires_at: Option<chrono::NaiveDate>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub scopes: Vec<String>,
/// Whether *this workspace* renews the credential. That needs a scope which
/// permits it (`api` or `self_rotate`) and a credential this workspace holds:
/// a token carried in the repository URL is the operator's to manage, and one
/// resolved from an ancestor is the ancestor's, so neither is renewed here.
pub rotatable: bool,
/// Unix timestamp (seconds) of the last check.
pub checked_at: i64,
/// Why the last check or rotation failed, cleared by the next success.
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
}
/// Outcome of the most recent auto-pull attempt, surfaced in the UI.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct AutoPullStatus {
+19
View File
@@ -3629,6 +3629,16 @@ async fn get_git_commit_hash(
})?;
git_resource.url =
resolve_azure_devops_url(&db_with_opt_authed, &w_id, &git_resource.url, false).await?;
// A credential is stored under the repository it was issued for, so a
// resource repointed elsewhere finds none. Which credential can be attached
// is bounded by that; who may use it is bounded here, on the same terms as
// the installation credential above.
let plain_url = git_resource.url.clone();
git_resource.url =
windmill_common::git_sync_oss::with_stored_credential(&db, &w_id, git_resource.url).await?;
if git_resource.url != plain_url {
require_admin(authed.is_admin, &authed.username)?;
}
let identities: Vec<String> = query
.git_ssh_identity
@@ -4307,6 +4317,10 @@ pub async fn get_git_repo_head_for_autopull(
}
git_resource.url =
resolve_azure_devops_url(&git_sync_system_dba(db), w_id, &git_resource.url, true).await?;
// A repo whose credential Windmill holds carries none in its URL, so the
// poller has to attach it here or every probe would be unauthenticated.
git_resource.url =
windmill_common::git_sync_oss::with_stored_credential(db, w_id, git_resource.url).await?;
if let Some(branch) = git_resource.branch.as_deref().filter(|s| !s.is_empty()) {
let branch = branch.to_string();
@@ -4413,6 +4427,11 @@ pub async fn get_git_repo_fork_heads_for_autopull(
));
}
git_resource.url = resolve_azure_devops_url(&dba, w_id, &git_resource.url, true).await?;
// Same reason as the head probe above: a repository whose credential Windmill
// holds carries none in its URL, and listing the fork branches is the half of
// polling that would otherwise go out unauthenticated.
git_resource.url =
windmill_common::git_sync_oss::with_stored_credential(db, w_id, git_resource.url).await?;
validate_git_url(&git_resource.url).await?;
validate_git_ref(base_branch)?;
+90 -40
View File
@@ -816,8 +816,19 @@ pub async fn handle_receive_completed_job(
#[cfg(all(feature = "enterprise", feature = "private"))]
#[derive(serde::Deserialize)]
struct GitSyncCheck {
check_run_id: i64,
repo_url: String,
/// Absent when the repository's host has no check surface (GitLab): the
/// result then reaches the pull request through the managed comment alone.
#[serde(default)]
check_run_id: Option<i64>,
/// Only markers written before the repository URL moved out of job args
/// carry one; the resource path on the job is what is used now.
#[serde(default)]
repo_url: Option<String>,
/// Host and path of the repository the check was created on, with no
/// credential in it. The resource path is mutable, so this is what proves
/// the resource still points where the check lives.
#[serde(default)]
repo: Option<String>,
#[serde(default)]
pr_number: Option<i64>,
#[serde(default)]
@@ -1162,18 +1173,18 @@ async fn maybe_open_git_sync_deploy_pr(
};
// Base = the tracked branch (resource branch, else the repo default). Also
// acts as the app-backed gate: PR creation needs the installation token.
let base = match windmill_common::git_sync_ee::get_app_repo_head_for_autopull(
// acts as the gate: PR creation needs a credential the server itself holds.
let base = match windmill_common::git_sync_ee::managed_pr_base_branch(
db,
workspace_id,
&repo_path,
)
.await
{
Ok(Some((branch, _))) => branch,
Ok(Some(branch)) => branch,
Ok(None) => {
tracing::warn!(
"git sync PR: repo {repo_path} in {workspace_id} has a PR-on-deploy toggle set but is not GitHub-App-backed; skipping (connect the repo through the GitHub App, or use the open-pr-on-commit workflow)"
"git sync PR: repo {repo_path} in {workspace_id} has a PR-on-deploy toggle set but the server holds no credential for it; skipping (connect the repo through the GitHub App or a GitLab token, or use the open-pr-on-commit workflow)"
);
return;
}
@@ -1373,22 +1384,59 @@ async fn maybe_post_git_sync_check(
(None, Some(deploy)) => (true, deploy),
(None, None) => return,
};
let Ok(mut check) = serde_json::from_value::<GitSyncCheck>(marker) else {
let Ok(check) = serde_json::from_value::<GitSyncCheck>(marker) else {
return;
};
// Markers carry the literal resource URL (job args are persisted, so a
// `$var:`-resolved URL must not land there); interpolate before calling
// GitHub.
check.repo_url =
match windmill_common::variables::get_variable_or_self(check.repo_url, db, workspace_id)
.await
{
Ok(u) => u,
Err(e) => {
tracing::error!("git sync-check: cannot interpolate repo url: {e:#}");
return;
}
};
// Job args are persisted, so the repository URL is not among them: it is
// re-resolved here from the resource path the pull job carries. A marker
// written before that change still has the URL, and is honoured until the
// last such job has drained.
// The resource path is mutable, so it is only trusted when the marker also
// carries the identity to check it against. A marker written before that
// identity existed keeps using the URL it captured at enqueue, which cannot
// have been repointed since.
let repo_url = match (
check.repo.is_some(),
row.repo_path.as_deref(),
check.repo_url.clone(),
) {
// The resource path is mutable, so following it is only safe when the
// marker also carries the identity to check the result against.
(true, Some(path), _) => {
windmill_common::git_sync_ee::resolve_repo_url_interpolated(db, workspace_id, path)
.await
}
// A marker written before that identity existed captured the URL itself,
// which cannot have been repointed since.
(_, _, Some(url)) => {
windmill_common::variables::get_variable_or_self(url, db, workspace_id).await
}
// Neither: nothing here can prove which repository this check belongs to,
// and resolving the path anyway is how a preview reaches the wrong one.
// Leaving the check unfinished is the safe failure.
_ => {
tracing::error!(
"git sync-check: the marker carries neither a repository identity nor a url; not acting on it"
);
return;
}
};
let repo_url = match repo_url {
Ok(u) => u,
Err(e) => {
tracing::error!("git sync-check: cannot resolve repo url: {e:#}");
return;
}
};
// A resource repointed while the diff was running would otherwise close a
// check, or post a preview, on a repository that has nothing to do with it.
if check.repo.is_some() && windmill_common::git_sync_ee::repo_identity(&repo_url) != check.repo
{
tracing::warn!(
"git sync-check: the repository moved since the check was created; leaving it alone"
);
return;
}
// "In sync" on a PR that visibly changes files reads as a bug when those
// files are outside the repo's sync filters — say what the scope is.
let scope_note = if !is_deploy && success {
@@ -1451,7 +1499,7 @@ async fn maybe_post_git_sync_check(
(
"failure",
"Merge conflicts with the base branch".to_string(),
"This PR cannot be merged cleanly, so there is no deploy diff to compute. Resolve the conflicts and push again to re-run this check."
"This branch cannot be merged cleanly, so there is no deploy diff to compute. Resolve the conflicts and push again to re-run this check."
.to_string(),
)
} else if pr_check_error.as_deref() == Some("PR_HEAD_REF_UNAVAILABLE") {
@@ -1461,7 +1509,7 @@ async fn maybe_post_git_sync_check(
(
"neutral",
"Could not compute the deploy diff".to_string(),
"Windmill could not fetch this PR's head or enough history from GitHub to compute its merge with the base. Push again to re-run this check."
"Windmill could not fetch this branch's head, or enough history, to compute its merge with the base. Push again to re-run this check."
.to_string(),
)
} else if pr_check_error.is_some() {
@@ -1484,20 +1532,20 @@ async fn maybe_post_git_sync_check(
"success",
"In sync".to_string(),
format!(
"Merging this PR would make no changes to the workspace.{}",
"Merging this branch would make no changes to the workspace.{}",
scope_note.as_deref().unwrap_or_default()
),
),
Some((changes, settings_changed)) => {
let mut lines = vec![format!(
"Merging this PR would apply {} change(s) to the workspace:\n",
"Merging this branch would apply {} change(s) to the workspace:\n",
changes.len()
)];
lines.extend(format_change_list(&changes));
if settings_changed {
lines.push(match check.wmill_yaml_changed {
Some(true) => "\nThis PR changes wmill.yaml: pulling also applies the updated workspace settings.".to_string(),
Some(false) => "\nIndependent of this PR, the workspace's git-sync settings differ from the repo's wmill.yaml and a pull updates them to match.".to_string(),
Some(true) => "\nThis branch changes wmill.yaml: pulling also applies the updated workspace settings.".to_string(),
Some(false) => "\nIndependent of this branch, the workspace's git-sync settings differ from the repo's wmill.yaml and a pull updates them to match.".to_string(),
None => "\nA pull also updates the workspace's git-sync settings to match the repo's wmill.yaml.".to_string(),
});
}
@@ -1520,19 +1568,21 @@ async fn maybe_post_git_sync_check(
Some(url) => format!("{summary}\n\n[See the job in Windmill]({url})"),
None => summary.clone(),
};
if let Err(e) = windmill_common::git_sync_ee::update_check_run(
db,
workspace_id,
&check.repo_url,
check.check_run_id,
conclusion,
&title,
&check_summary,
job_url.as_deref(),
)
.await
{
tracing::error!("git sync-check: failed to update check run: {e:#}");
if let Some(check_run_id) = check.check_run_id {
if let Err(e) = windmill_common::git_sync_ee::update_check_run(
db,
workspace_id,
&repo_url,
check_run_id,
conclusion,
&title,
&check_summary,
job_url.as_deref(),
)
.await
{
tracing::error!("git sync-check: failed to update check run: {e:#}");
}
}
// Phase 4 also maintains ONE managed comment on the PR (Cloudflare
@@ -1556,7 +1606,7 @@ async fn maybe_post_git_sync_check(
if let Err(e) = windmill_common::git_sync_ee::upsert_pr_comment(
db,
workspace_id,
&check.repo_url,
&repo_url,
pr_number,
marker,
&body,
+170
View File
@@ -0,0 +1,170 @@
# Git sync with GitLab
GitLab has no equivalent of a GitHub App, so there is nothing to install and no
consent screen. What Windmill needs instead is one credential you create in
GitLab and paste once. With it, a GitLab repository gets the same managed
features an app-backed GitHub repository has: instant pull over a webhook, merge
requests opened on deploy, and a diff preview posted onto the merge request.
## The credential
Create a **project access token** on the project you are syncing (Settings →
Access tokens). It is a bot identity that outlives the person who created it,
which is what you want for a credential the instance uses unattended, and it
reaches exactly the one project.
| | |
| --- | --- |
| Scope | `api` |
| Role | Developer to push deploy branches; **Maintainer** to also manage the webhook and open merge requests |
| Expiry | Required. A group service account PAT can be non-expiring on self-managed (see below); an access token cannot |
**Use a separate token per repository.** A group access token works too and
reaches every project in the group, which is convenient for a lot of
repositories — but Windmill stores the credential per repository, and renewal
rewrites the repository it renewed for. Any other repository holding that same
token keeps the revoked one and stops syncing until you paste a new token there.
Each stranded repository says so on its card, so it is visible rather than
silent, but a token per repository avoids it entirely.
`api` is a superset: it authorizes Git over HTTPS as well, so no separate
`write_repository` is needed to clone and push, and it is also what makes the
token renewable. A `write_repository`-only token can still push, but Windmill
cannot inspect or renew it and reports that in the workspace's git sync settings.
### The identity Windmill acts as
GitLab issues an access token to a bot user it creates for it — `project_<id>_bot_…`
for a project token, `group_<id>_bot_…` for a group one — and the bot's display
name is **the name you gave the token**. That name is the byline on everything
Windmill does: the author of deploy commits, of the merge requests it opens, and
of the preview notes it writes. Name it for what it is, `windmill-sync` or
similar, rather than something only you will recognise.
Each token you create adds another bot member to the project or group. Renewal
does not — it keeps the same bot — so a repository accumulates one bot, not one
per year.
Renewal goes through GitLab's own self-rotation endpoint. Both kinds of access
token are held as their bot user's personal access token, so the token rotates
itself and Windmill never needs a credential with rights over the project or
group.
## Connecting a repository
In the resource form for a `git_repository` resource, use the **GitLab** button:
paste the instance URL and the token, pick a project from the list, and Windmill
keeps the token for you. The resource itself gets the plain remote URL
(`"url": "https://gitlab.com/group/project.git"`), with no credential in it.
The token is stored encrypted on the workspace, keyed by the repository it was
issued for rather than by the resource naming it, the same way a GitHub App
installation is held against the account it covers. Nothing reads it back out
over the API: the server attaches it when it talks to GitLab, and a sync job
receives it only against its own job token. Repointing a resource's `url` asks
for a different repository's token and finds none, so the edit carries nothing
with it; a repository that genuinely moved needs its token entered again.
Because the repository is the key, the token is stored the moment you pick the
project, before the resource is saved. Renaming the resource later keeps it, and
cancelling the edit leaves a stored token that nothing uses until some resource
points at that repository again.
Forks of the workspace read this one copy rather than getting their own, so
renewal reaches all of them at once and the token is not duplicated into every
descendant workspace.
Treat workspace admin as equivalent to holding the token. An admin of the
workspace, or of any fork below it, can point a repository at a sync script they
wrote and have that job request the credential, exactly as they can for a GitHub
App installation token. Storing it this way keeps it out of the variables API and
out of every fork's own storage; it is not a boundary against the admins of those
workspaces.
A URL with the token written into it, in the resource or in a secret variable
the resource points at (`"url": "$var:..."`), is a plain git remote: it syncs on
deploy and by polling, and nothing else. Windmill does not know the token is
there, so it registers no webhook, opens no merge request, and neither reports
nor renews its expiry. Use the **GitLab** button to hand the token to Windmill
if you want any of that.
## Expiry and renewal
Windmill reads `expires_at` from the token it holds and shows it on the
repository in the workspace's git sync settings. Within three weeks of expiry it
rotates the token through GitLab's own
`POST /personal_access_tokens/self/rotate`, stores the replacement, and verifies
it. Only the token can rotate itself, so one without `api` (or `self_rotate`) is
a permanent warning rather than something Windmill can fix.
Only the workspace that holds a credential renews it, so one renewal serves the
whole fork chain instead of each fork racing to renew the same token. A fork
reads the parent's without holding one, so it never renews; the parent does, and
every fork sees the replacement at once.
Rotation is deliberately never retried. GitLab revokes the old token the instant
it issues the replacement, and presenting an already-rotated token to `/rotate`
again is treated as reuse: it revokes **the whole token family, including the
live replacement**. So a rotation that succeeded at GitLab but failed to persist
is surfaced as an error to act on, not retried.
Non-expiring tokens are possible only for a **group service account PAT** on
self-managed, with `require_personal_access_token_expiry` turned off in the
instance's application settings. A group access token is always rejected without
an `expires_at`.
## What each managed feature needs
| Feature | Needs |
| --- | --- |
| Instant pull | A project hook Windmill creates, so Maintainer; and a Windmill base URL GitLab can reach |
| Merge requests on deploy | Developer, plus the `api` scope |
| Diff preview on a merge request | The project hook, plus permission to post merge request notes |
Instant pull falls back to checking the tracked branch about every minute when
the hook cannot be created or delivered, so nothing silently stops syncing.
## Self-managed differences
**Webhooks to a private network are blocked by default.** GitLab refuses to
create a hook pointing at a private or local address until an administrator
enables *Allow requests to the local network from webhooks and integrations*
(Admin → Settings → Network → Outbound requests,
`allow_local_requests_from_web_hooks_and_services`). A Windmill instance on the
same private network as GitLab needs this; without it, hook creation fails with a
"blocked" error and the repository keeps polling.
**A relative-URL install is not supported.** GitLab can be served under a path
prefix (`https://example.com/gitlab`), and that prefix cannot be told apart from
a group of the same name: `example.com/a/b/c.git` is either group `a/b` project
`c`, or prefix `a` with group `b` project `c`. Windmill reads it as the nested
group, so on a relative-URL install it derives the wrong API base and the managed
features stay unavailable. Such a repository still syncs through its token URL,
which needs no API base.
Everything else is identical: Windmill talks to `<your-gitlab>/api/v4` and needs
no inbound access of its own beyond the hook deliveries.
## The deploy preview is a note, not a pipeline status
On GitHub the preview is a check run: its own object, advisory unless the
repository makes it required. GitLab has no equivalent. Its only comparable
primitive is a commit status, and posting one has side effects Windmill will not
impose on a project:
- GitLab files the status **as a job inside whatever pipeline already covers that
commit**, so a failed Windmill status fails the project's own pipeline, and its
reviewers see their test suite as failed.
- `allow_failure` is ignored on the commit-status endpoint, so the status cannot
be made advisory.
- On a commit with no pipeline it creates an `external` pipeline instead, which
then gates merging under *Pipelines must succeed*, including while it is still
running.
So on GitLab the preview lives entirely in a **merge request note** that Windmill
keeps up to date: it carries the workspace, the status line, the commit, a link
to the job, and the full list of changes merging would deploy. A note cannot
block a merge or change what the project's own CI reports.
The note is upserted rather than appended, so a merge request accumulates one
Windmill comment however many times it is pushed to.
@@ -12,6 +12,7 @@
import { Loader2 } from 'lucide-svelte'
import { untrack } from 'svelte'
import GitHubAppIntegration from './GitHubAppIntegration.svelte'
import GitLabIntegration from './GitLabIntegration.svelte'
import BedrockCredentialsCheck from './BedrockCredentialsCheck.svelte'
import { isCloudHosted } from '$lib/cloud'
import ResourceGen from './copilot/ResourceGen.svelte'
@@ -28,6 +29,13 @@
isValid?: boolean
linkedSecretCandidates?: string[] | undefined
description?: string | undefined
/** Workspace the resource is being saved into, which is not always the one
* being navigated. The GitLab picker has to store the credential where the
* resource will look for it. */
workspace?: string
/** Fired once the GitLab picker has stored the picked project's token, so a
* form that would otherwise file the URL as a secret knows it holds none. */
onCredentialStored?: () => void
onSynced?: () => void
}
@@ -39,6 +47,8 @@
isValid = $bindable(true),
linkedSecretCandidates = undefined,
description = $bindable(undefined),
workspace = undefined,
onCredentialStored,
onSynced = undefined
}: Props = $props()
@@ -249,6 +259,19 @@
}}
onDescriptionUpdate={(newDescription) => (description = newDescription)}
/>
<!-- Last in a `flex-row-reverse` row, so it lands beside the GitHub App
button without splitting it from its own refresh control. -->
<GitLabIntegration
{resourceType}
{args}
{workspace}
{onCredentialStored}
onArgsUpdate={(newArgs) => {
args = newArgs
rawCode = JSON.stringify(args, null, 2)
rawCodeEditor?.setCode(rawCode)
}}
/>
</div>
{#if resourceType?.includes('bedrock') && !isCloudHosted()}
<BedrockCredentialsCheck />
@@ -1373,6 +1373,15 @@
{linkedSecretCandidates}
{resourceType}
{resourceTypeInfo}
workspace={effectiveWorkspace}
onCredentialStored={() => {
// `forceSecretValue` files a git_repository's `url` in a secret
// variable, for the URLs that carry a token in them. The picker's
// does not: the token is stored separately, so that variable would
// hold nothing secret and add a second place to keep in step with
// the resource.
linkedSecrets = linkedSecrets.filter((f) => f !== 'url')
}}
bind:args
bind:isValid
onSynced={getResourceTypeInfo}
@@ -180,7 +180,7 @@
{#if !githubState.loadingGithubInstallations}
<Button
variant="default"
size="xs"
unifiedSize="sm"
on:click={handleRefreshInstallations}
disabled={!$enterpriseLicense}
startIcon={{ icon: RotateCw }}
@@ -198,7 +198,7 @@
{#snippet trigger()}
<Button
variant="default"
size="xs"
unifiedSize="sm"
disabled={!$enterpriseLicense || githubState.loadingGithubInstallations}
startIcon={{
icon: githubState.loadingGithubInstallations ? Loader2 : Github,
@@ -0,0 +1,221 @@
<script lang="ts">
import { workspaceStore, userStore, enterpriseLicense } from '$lib/stores'
import { GitSyncService, type GitlabProject } from '$lib/gen'
import { sendUserToast } from '$lib/toast'
import Popover from './meltComponents/Popover.svelte'
import Button from './common/button/Button.svelte'
import { Alert } from './common'
import TextInput from './text_input/TextInput.svelte'
import Select from './select/Select.svelte'
import { GitBranch, Gitlab, Loader2 } from 'lucide-svelte'
interface Props {
resourceType: string
args?: Record<string, any>
/** The workspace the resource is being edited in, which is not always the
* one being navigated: the credential has to land where the resource will
* look for it. */
workspace?: string
/** Fired once the picked project's token is stored, so a form that would
* otherwise file the URL as a secret knows it no longer holds one. */
onCredentialStored?: () => void
onArgsUpdate?: (args: Record<string, any>) => void
}
let {
resourceType,
args = {},
workspace = undefined,
onCredentialStored,
onArgsUpdate
}: Props = $props()
let ws = $derived(workspace ?? $workspaceStore)
let baseUrl = $state('https://gitlab.com')
let token = $state('')
let projects: GitlabProject[] = $state([])
let selectedProject: string | undefined = $state(undefined)
let loading = $state(false)
let listError: string | undefined = $state(undefined)
let applying = $state(false)
let applyError: string | undefined = $state(undefined)
/** The token the current listing was made with. Editing the token afterwards
* leaves projects on screen that were never checked against it, and applying
* would store the new token for a project chosen under the old one. */
let listedToken = $state('')
// Shown alongside the GitHub App button and on the same terms, so the two
// read as one choice rather than one option and one absence.
let show = $derived(
resourceType === 'git_repository' &&
!!ws &&
($userStore?.is_admin || $userStore?.is_super_admin)
)
// The project listing is served by an enterprise-only route, so on a build
// without it the form's first request would 404. The button still shows,
// disabled and labelled, because a missing button reads as "GitLab is not
// supported" rather than "this needs a licence".
let enabled = $derived(!!$enterpriseLicense)
let project = $derived(projects.find((p) => p.path_with_namespace === selectedProject))
let staleListing = $derived(projects.length > 0 && token !== listedToken)
async function listProjects() {
if (!ws) return
loading = true
listError = undefined
try {
// Captured before the await, not after: the field stays editable while the
// request is in flight, and reading it on the way back would record a
// token this listing was never checked against.
const listedWith = token
projects = await GitSyncService.listGitlabProjects({
workspace: ws,
requestBody: { base_url: baseUrl, token: listedWith }
})
listedToken = listedWith
selectedProject = projects[0]?.path_with_namespace
if (projects.length === 0) {
listError = 'The token can see no project with at least the Developer role'
}
} catch (err) {
listError = err?.body ?? err?.message ?? String(err)
projects = []
selectedProject = undefined
} finally {
loading = false
}
}
async function apply(close: (_: any) => void) {
if (!project || !token || applying || staleListing) return
const chosen = project
const url = chosen.http_url_to_repo
applying = true
applyError = undefined
try {
// Stored against the project it was issued for, the way a GitHub App
// installation is stored against the account it covers. Nothing waits on
// the resource: its path is not settled while it is being created, and
// deferring the write would tie one repository's token to another
// repository's save succeeding.
await GitSyncService.setGitCredential({
workspace: ws!,
requestBody: { repo_url: url, token }
})
} catch (err) {
applyError = err?.body ?? err?.message ?? String(err)
return
} finally {
applying = false
}
onCredentialStored?.()
onArgsUpdate?.({
...args,
url,
is_github_app: false,
branch: args.branch || chosen.default_branch || undefined
})
token = ''
listedToken = ''
projects = []
selectedProject = undefined
sendUserToast(`${chosen.path_with_namespace} selected and its token stored`)
close(null)
}
</script>
{#if show}
<Popover
documentationLink="https://www.windmill.dev/docs/integrations/git_repository"
disabled={!enabled}
contentClasses="overflow-auto"
>
{#snippet trigger()}
<Button
variant="default"
unifiedSize="sm"
disabled={!enabled}
startIcon={{ icon: Gitlab }}
nonCaptureEvent
>
{enabled ? 'GitLab' : 'GitLab (ee only)'}
</Button>
{/snippet}
{#snippet content({ close })}
<div class="block text-primary p-4">
<div class="flex flex-col gap-4 w-[600px]">
<div class="flex flex-col gap-y-1">
<div class="text-xs font-semibold text-emphasis">GitLab instance</div>
<TextInput bind:value={baseUrl} size="sm" />
</div>
<div class="flex flex-col gap-y-1">
<div class="text-xs font-semibold text-emphasis">Project access token</div>
<div class="text-xs font-normal text-secondary">
Create it on the project you are syncing, with the <code>api</code> scope and the
<code>Maintainer</code> role.
</div>
<TextInput bind:value={token} size="sm" inputProps={{ type: 'password' }} />
<div class="text-2xs font-normal text-hint">
Windmill stores it and renews it before it expires. Use one token per project: a group
token covers the group, but renewal replaces it for one project at a time.
</div>
</div>
<div>
<Button
variant="default"
unifiedSize="sm"
disabled={!token || !baseUrl || loading}
startIcon={{
icon: loading ? Loader2 : GitBranch,
classes: loading ? 'animate-spin' : ''
}}
onclick={listProjects}
>
List projects
</Button>
</div>
{#if listError}
<Alert type="error" title="Could not list projects" size="xs">{listError}</Alert>
{/if}
{#if staleListing}
<div class="text-xs font-normal text-secondary">
The token changed. List the projects again to choose one it can reach.
</div>
{/if}
{#if projects.length > 0 && !staleListing}
<div class="flex flex-col gap-y-1">
<div class="text-xs font-semibold text-emphasis">Project</div>
<Select
items={projects.map((p) => ({
label: p.path_with_namespace,
value: p.path_with_namespace
}))}
bind:value={selectedProject}
clearable={false}
/>
</div>
{#if applyError}
<Alert type="error" title="Could not store the token" size="xs">{applyError}</Alert>
{/if}
<div class="flex justify-end">
<Button
variant="accent"
unifiedSize="sm"
disabled={!project || !token || applying}
startIcon={{
icon: applying ? Loader2 : GitBranch,
classes: applying ? 'animate-spin' : ''
}}
onclick={() => apply(close)}
>
Use this project
</Button>
</div>
{/if}
</div>
</div>
{/snippet}
</Popover>
{/if}
@@ -1,12 +1,19 @@
<script lang="ts">
import type { Schema } from '$lib/common'
import { ResourceService, WorkspaceService, type Resource, type ResourceType } from '$lib/gen'
import {
GitSyncService,
ResourceService,
WorkspaceService,
type Resource,
type ResourceType
} from '$lib/gen'
import { canWrite } from '$lib/utils'
import { createEventDispatcher, onDestroy, untrack } from 'svelte'
import { userStore, workspaceStore } from '$lib/stores'
import { sendUserToast } from '$lib/toast'
import { clearJsonSchemaResourceCache } from './schema/jsonSchemaResource.svelte'
import ResourceForm from './ResourceForm.svelte'
import ReplaceGitCredential from './git_sync/ReplaceGitCredential.svelte'
import { invalidateWorkspacePaths } from './PathNameAutocomplete.svelte'
import Alert from './common/alert/Alert.svelte'
import { resource } from 'runed'
@@ -194,6 +201,40 @@
let loadingSchema = $derived(resourceTypeResource.loading)
let current = $derived(selected ? states[selected]?.draft : undefined)
// The saved URL, not the draft's: a credential is bound to the repository it
// is issued for, so binding one to an edit that has not landed yet would tie
// it to something the resource does not point at.
let deployedUrl = $derived(
selected ? ((fetchedResources[selected]?.value as any)?.url as string | undefined) : undefined
)
// The deployed path, for the same reason as the deployed URL: the server
// answers about what is stored, and an unsaved rename names nothing yet.
let deployedPath = $derived(selected ? (initialStates[selected]?.path ?? initialPath) : undefined)
// Asked of the server rather than read off the resource: the resource is
// client-editable, exported and copied into forks, so nothing written on it
// stays true. Re-asked when the saved URL moves, since that is a different
// repository. The answer is for admins, who are the only ones who could act
// on it, so nobody else asks.
const credentialOrigin = resource(
[
() => selected,
() => deployedPath,
() => deployedUrl,
() => resource_type,
() => (selected ? (perWsUser[selected] ?? $userStore)?.is_admin : undefined)
],
async ([ws, path, _url, type, admin]) =>
ws && path && type === 'git_repository' && admin
? await GitSyncService.getCredentialOrigin({ workspace: ws, path }).catch(() => undefined)
: undefined
)
// Only a credential this workspace holds is its to replace: a fork borrows
// its ancestor's, and storing a replacement here would split it in two.
let holdsCredential = $derived(credentialOrigin.current?.origin === 'held')
// Only an unsaved *URL* blocks replacing the token, not any unsaved change:
// opening the drawer materialises schema defaults (`folder: ""`), so a whole-
// resource dirty check would disable it the moment the drawer opens.
let urlDirty = $derived(!!deployedUrl && current?.args?.url !== deployedUrl)
let resourceToEdit: Resource | undefined = $derived(
selected ? fetchedResources[selected] : undefined
)
@@ -495,6 +536,25 @@
</Alert>
{/if}
{#if holdsCredential && selected}
<Alert type="info" title="Windmill holds this repository's access token">
<div class="flex flex-col items-start gap-2">
<div>
The URL carries no credential. Windmill stores the token and renews it before it
expires.
{#if urlDirty}
Save your URL change to replace the token.
{/if}
</div>
<ReplaceGitCredential
workspace={selected}
repoUrl={deployedUrl ?? ''}
disabled={urlDirty || !deployedUrl}
/>
</div>
</Alert>
{/if}
{#if current}
{#key current}
<ResourceForm
@@ -19,6 +19,7 @@
import GfmMarkdown from './GfmMarkdown.svelte'
import TestTriggerConnection from './triggers/TestTriggerConnection.svelte'
import GitHubAppIntegration from './GitHubAppIntegration.svelte'
import GitLabIntegration from './GitLabIntegration.svelte'
import Button from './common/button/Button.svelte'
import ResourceGen from './copilot/ResourceGen.svelte'
import SyncResourceTypes from './SyncResourceTypes.svelte'
@@ -47,6 +48,9 @@
/** Workspace the path is validated against and the connection is tested in;
* defaults to the nav workspace. */
workspace?: string | undefined
/** Fired once the GitLab picker has stored the picked project's token, so a
* form that would otherwise file the URL as a secret knows it holds none. */
onCredentialStored?: () => void
}
let {
@@ -68,7 +72,8 @@
loadingSchema,
resourceToEdit,
onLoadResourceType,
workspace = undefined
workspace = undefined,
onCredentialStored
}: Props = $props()
let ws = $derived(workspace ?? $workspaceStore)
@@ -248,12 +253,30 @@
{description}
onArgsUpdate={(newArgs) => {
args = newArgs
if (viewJsonSchema) {
// The raw editor is also what a workspace missing the resource type
// gets, and it holds its own copy of the value: without this the
// picker fills in a URL nothing on screen ever shows.
if (viewJsonSchema || !resourceSchema) {
rawCode = JSON.stringify(args, null, 2)
}
}}
onDescriptionUpdate={(newDescription) => (description = newDescription)}
/>
<GitLabIntegration
resourceType={resource_type}
{args}
workspace={ws}
{onCredentialStored}
onArgsUpdate={(newArgs) => {
args = newArgs
// The raw editor is also what a workspace missing the resource type
// gets, and it holds its own copy of the value: without this the
// picker fills in a URL nothing on screen ever shows.
if (viewJsonSchema || !resourceSchema) {
rawCode = JSON.stringify(args, null, 2)
}
}}
/>
{/if}
</div>
@@ -23,7 +23,7 @@
import GitSyncModeDisplay from './GitSyncModeDisplay.svelte'
import Toggle from '$lib/components/Toggle.svelte'
import EEOnly from '$lib/components/EEOnly.svelte'
import { ResourceService, VariableService } from '$lib/gen'
import { GitSyncService, ResourceService, VariableService } from '$lib/gen'
let {
idx = null,
@@ -151,6 +151,91 @@
let loadingResourceInfo = $state(false)
// Only GitHub App-backed repos can register webhooks; PAT repos poll only.
let isGithubApp = $state(false)
/** Where the credential Windmill uses for this repository lives, answered by
* the server rather than inferred from the resource: `held` when this
* workspace stores it, `borrowed` when an ancestor does. A borrowed one is not
* this workspace's to renew or replace, which is what keeps a fork from
* warning about a token it must not touch. `undefined` until the lookup
* lands, and when nothing in the chain holds one. */
let credentialOrigin = $state<'held' | 'borrowed' | undefined>(undefined)
// Whether Windmill itself holds a credential for the repository, which is
// what the managed features (webhooks, pull requests, commit checks) need.
// A GitHub App installation qualifies, and so does a token the server keeps.
// Not the recorded status: that is keyed by resource path and outlives a
// repoint, while the origin follows the repository the URL names now.
let hasManagedCredential = $derived(isGithubApp || credentialOrigin !== undefined)
const MS_PER_DAY = 86_400_000
/**
* Whole days until the repository's own token expires, or undefined when it
* never expires and when nothing has checked it yet.
*
* Counted between calendar dates, not instants: GitLab expires a token on a
* date, so measuring from "now" would call a token expiring later today
* expired, and one expiring tomorrow today's problem.
*/
const credentialDaysLeft = $derived.by(() => {
const expiresAt = repo.credential?.expires_at
if (!expiresAt) return undefined
const expiry = new Date(`${expiresAt}T00:00:00Z`).getTime()
const now = new Date()
const today = Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate())
return Math.round((expiry - today) / MS_PER_DAY)
})
/**
* Only raised when a person has to act. A token Windmill renews on its own is
* reported in the quiet status line instead, so the alert keeps meaning
* "this needs you".
*/
const credentialAlert = $derived.by(() => {
const credential = repo.credential
// The status describes the repository the resource named when it was
// checked; once nothing is held for the one it names now, it is stale.
if (!credential || !hasManagedCredential) return undefined
if (credential.error) {
return {
type: 'error' as const,
title: 'Repository token needs attention',
body: credential.error
}
}
const days = credentialDaysLeft
if (days === undefined) return undefined
const when =
days <= 0 ? 'has expired' : days === 1 ? 'expires tomorrow' : `expires in ${days} days`
// Renewed here, or by the workspace above that holds it. Either way this
// workspace has nothing to do, and telling a fork to replace a borrowed
// token would split the credential in two. Renewal is licensed per
// instance, so a fork knows as well as its parent when nothing renews.
if ((credential.rotatable || credentialOrigin === 'borrowed') && $enterpriseLicense) {
// A token Windmill renews needs no countdown: a renewal that fails records
// an error, which is handled above. Reaching the expiry date anyway is the
// one state that proves renewal never happened, and it is the only one
// worth raising here — picking an earlier threshold would just be guessing
// at the server's renewal window from the client.
if (days > 0) return undefined
return {
type: 'error' as const,
title: 'Repository token has expired',
body:
credentialOrigin === 'borrowed'
? 'The workspace that holds this token renews it, but it has expired anyway. Replace it there to restore sync.'
: 'Windmill renews this token automatically but has not managed to. Check that the instance can reach GitLab, then replace the token to restore sync.'
}
}
if (days > 30) return undefined
const where =
credentialOrigin === 'borrowed'
? 'in the workspace that holds it'
: `on the ${repo?.git_repo_resource_path?.replace(/^\$res:/, '') ?? 'repository'} resource`
return {
type: days <= 7 ? ('error' as const) : days <= 14 ? ('warning' as const) : ('info' as const),
title: `Repository token ${when}`,
body: `Windmill does not renew this token. Replace it ${where}${days <= 0 ? ' to restore sync.' : ' before it expires.'}`
}
})
// Update target branch when repository changes
$effect(() => {
@@ -187,12 +272,27 @@
// Clear stale app state up front so a resource change or a failed
// fetch can't leave webhook/fork controls showing for the wrong repo.
isGithubApp = false
credentialOrigin = undefined
try {
const resource = await ResourceService.getResource({
workspace: $workspaceStore,
path: repo.git_repo_resource_path
})
// The server answers whether it holds this repository's credential;
// the resource cannot, being client-editable, exported and copied
// into forks. Best-effort: a failure here must not hide the URL
// below. Awaited alongside the resource because the defaults below
// read the answer.
const [origin, resource] = await Promise.all([
GitSyncService.getCredentialOrigin({
workspace: $workspaceStore,
path: repo.git_repo_resource_path
}).catch(() => undefined),
ResourceService.getResource({
workspace: $workspaceStore,
path: repo.git_repo_resource_path
})
])
if (!abortController.signal.aborted) {
credentialOrigin = origin?.origin
}
if (!abortController.signal.aborted && resource?.value) {
// Extract git URL from resource value
const value = resource.value as Record<string, any>
@@ -205,7 +305,7 @@
if (
repoMode === 'sync' &&
repo.isUnsavedConnection &&
isGithubApp &&
hasManagedCredential &&
!isFork &&
$enterpriseLicense &&
repo.auto_pull === undefined
@@ -219,7 +319,7 @@
if (
repoMode === 'promotion' &&
repo.isUnsavedConnection &&
isGithubApp &&
hasManagedCredential &&
$enterpriseLicense &&
repo.promotion_open_prs === undefined
) {
@@ -292,6 +392,7 @@
} else {
resourceInfo = null
isGithubApp = false
credentialOrigin = undefined
}
}
@@ -537,6 +638,29 @@
</div>
{/if}
{#if credentialAlert}
<Alert type={credentialAlert.type} title={credentialAlert.title} size="xs">
{credentialAlert.body}
</Alert>
{:else if hasManagedCredential && repo.credential && !repo.credential.error}
<div class="text-xs text-secondary">
{#if credentialDaysLeft === undefined}
Repository token does not expire.
{:else if credentialOrigin === 'borrowed' && $enterpriseLicense}
Repository token expires on {repo.credential.expires_at}, and the workspace that holds it
renews it.
{:else if repo.credential.rotatable && $enterpriseLicense}
Repository token expires on {repo.credential.expires_at}, and Windmill renews it
automatically.
{:else if repo.credential.rotatable || credentialOrigin === 'borrowed'}
Repository token expires on {repo.credential.expires_at}. Renewing it automatically
requires an enterprise license.
{:else}
Repository token expires on {repo.credential.expires_at}, and Windmill does not renew it.
{/if}
</div>
{/if}
{#if !emptyString(repo.git_repo_resource_path)}
<!-- Validation and Test Status -->
{#if validation?.isDuplicate}
@@ -671,7 +795,7 @@
{/if}
</div>
{/if}
{#if repoMode === 'promotion' && isGithubApp}
{#if repoMode === 'promotion' && hasManagedCredential}
<div class="mt-2">
<!-- Locked while the dev promotion toggle's save is in flight: this
toggle is revealed by that save, and an edit made mid-save would be
@@ -700,7 +824,12 @@
href="https://www.windmill.dev/docs/integrations/git_repository#github-app"
target="_blank"
class="text-blue-500 hover:underline">GitHub App</a
> and Windmill opens them automatically.
>, or give a GitLab repository a
<a
href="https://www.windmill.dev/docs/integrations/git_repository"
target="_blank"
class="text-blue-500 hover:underline">project access token</a
>, and Windmill opens them automatically.
</div>
{/if}
{#if repoMode === 'sync' && isFork}
@@ -715,7 +844,7 @@
fork is pushed to the fork's own
<span class="font-mono">wm-fork/…</span> branch instead of the tracked branch.
</div>
{#if isGithubApp}
{#if hasManagedCredential}
<div class="mt-2">
<Toggle
checked={repo.fork_open_prs ?? false}
@@ -740,11 +869,12 @@
target="_blank"
class="text-blue-500 hover:underline font-mono">open-pr-on-fork-commit</a
>
workflow in the repository. Recommended: connect the repository through the
workflow in the repository. Recommended: connect the repository through the GitHub
App, or give a GitLab repository a
<a
href="https://www.windmill.dev/docs/integrations/git_repository#github-app"
href="https://www.windmill.dev/docs/integrations/git_repository"
target="_blank"
class="text-blue-500 hover:underline">GitHub App</a
class="text-blue-500 hover:underline">project access token</a
> and Windmill opens them automatically.
</div>
{/if}
@@ -806,7 +936,7 @@
options={{
right: 'Automatically deploy changes from Git',
rightTooltip:
'Windmill deploys new commits from the tracked branch into this workspace. Repositories connected through the GitHub App sync instantly via webhooks with a polling fallback; token-based repositories are checked about every minute.'
'Windmill deploys new commits from the tracked branch into this workspace. Repositories Windmill holds a credential for sync instantly via webhooks with a polling fallback; other token-based repositories are checked about every minute.'
}}
on:change={(e) => setAutoPullEnabled(e.detail)}
>
@@ -828,7 +958,7 @@
/>
</div>
{/if}
{#if !isGithubApp && !loadingResourceInfo}
{#if !hasManagedCredential && !loadingResourceInfo}
<div class="mt-2">
<Alert type="info" title="Instant pull recommended" size="xs">
Pull for this repository checks the tracked branch about every minute; longer
@@ -838,25 +968,30 @@
href="https://www.windmill.dev/docs/integrations/git_repository#github-app"
target="_blank"
class="text-blue-500 hover:underline">GitHub App</a
>, or give a GitLab repository a
<a
href="https://www.windmill.dev/docs/integrations/git_repository"
target="_blank"
class="text-blue-500 hover:underline">project access token</a
>
(which also lets Windmill manage pull requests), or push changes into Windmill
(either also lets Windmill manage pull requests), or push changes into Windmill
with the
<a
href="https://www.windmill.dev/docs/advanced/git_sync#github-actions"
target="_blank"
class="text-blue-500 hover:underline">sync GitHub workflow</a
>. If you already push changes with a GitHub Action, keep either the Action or
automatic pull, not both, so they don't fight over deploys.
>. If you already push changes from CI, keep either that or automatic pull,
not both, so they don't fight over deploys.
</Alert>
</div>
{/if}
{#if repo.auto_pull?.enabled}
{@const viaWebhook = repo.auto_pull?.webhook_id != null}
{#if isGithubApp}
{#if hasManagedCredential}
<div class="mt-2">
<Alert type="info" title="Already pulling with a GitHub Action?" size="xs">
If you previously set up a GitHub Action to push changes into Windmill,
remove it now so the two don't fight over deploys.
<Alert type="info" title="Already pulling with a CI job?" size="xs">
If you previously set up a CI job to push changes into Windmill, remove it
now so the two don't fight over deploys.
</Alert>
</div>
{/if}
@@ -882,7 +1017,7 @@
: 'Checking the tracked branch about every minute. New commits deploy automatically.'}
{/if}
</div>
{#if isGithubApp && repo.auto_pull?.webhook_error}
{#if hasManagedCredential && repo.auto_pull?.webhook_error}
<div class="mt-2">
<Alert type="warning" title="Falling back to polling" size="xs">
{repo.auto_pull.webhook_error}
@@ -0,0 +1,134 @@
<script lang="ts">
import { GitSyncService } from '$lib/gen'
import { sendUserToast } from '$lib/toast'
import Popover from '../meltComponents/Popover.svelte'
import Button from '../common/button/Button.svelte'
import { Alert } from '../common'
import TextInput from '../text_input/TextInput.svelte'
import { KeyRound, Loader2 } from 'lucide-svelte'
interface Props {
workspace: string
/** The repository as currently saved, which is the key the token is stored
* under. A URL edited but not yet saved would file the token against a
* repository the resource does not point at, so the caller disables this
* until it is saved. */
repoUrl: string
disabled?: boolean
onReplaced?: () => void
}
let { workspace, repoUrl, disabled = false, onReplaced }: Props = $props()
let token = $state('')
let saving = $state(false)
let error: string | undefined = $state(undefined)
/** The instance and project a repository URL names, for checking the pasted
* token against the repository it is meant for. */
function repoParts(url: string): { base: string; project: string } | undefined {
try {
const u = new URL(url)
const project = u.pathname.replace(/^\/+/, '').replace(/\.git$/, '')
return project ? { base: `${u.protocol}//${u.host}`, project } : undefined
} catch {
return undefined
}
}
async function replace(close: (_: any) => void) {
if (!token || saving) return
saving = true
error = undefined
// Pinned before the first await, all three: the field stays editable and the
// props follow the drawer's selected workspace and deployed URL, so
// re-reading any of them afterwards would store the token against something
// the check never validated.
const candidate = token
const forRepo = repoUrl
const inWorkspace = workspace
try {
// Check the token before storing it. The server binds a credential to its
// repository but only refuses it when something tries to use it, so a
// wrong token would otherwise be accepted here and surface as a failed
// sync later.
// A URL that does not parse names no project to check, and would be
// stored as the credential's key verbatim: a `$var:` reference here
// keys the token to a repository that does not exist.
const parts = repoParts(forRepo)
if (!parts) {
error = 'The resource URL must name the repository directly to replace its token here.'
return
}
const projects = await GitSyncService.listGitlabProjects({
workspace: inWorkspace,
// Searched by name rather than listed whole: the listing is one
// capped page, so a token that reaches more projects than fit
// would not show this one and a working token would be refused.
requestBody: {
base_url: parts.base,
token: candidate,
search: parts.project.split('/').pop()
}
})
if (!projects.some((p) => p.path_with_namespace === parts.project)) {
error = `That token cannot push to ${parts.project}. Check its role and that it belongs to this project.`
return
}
await GitSyncService.setGitCredential({
workspace: inWorkspace,
requestBody: { repo_url: forRepo, token: candidate }
})
token = ''
sendUserToast('Token replaced')
onReplaced?.()
close(null)
} catch (err) {
error = err?.body ?? err?.message ?? String(err)
} finally {
saving = false
}
}
</script>
<Popover contentClasses="overflow-auto" {disabled}>
{#snippet trigger()}
<Button
variant="default"
unifiedSize="xs"
{disabled}
startIcon={{ icon: KeyRound }}
nonCaptureEvent
>
Replace token
</Button>
{/snippet}
{#snippet content({ close })}
<div class="block text-primary p-4">
<div class="flex flex-col gap-3 w-[420px]">
<div class="flex flex-col gap-y-1">
<div class="text-xs font-semibold text-emphasis">New access token</div>
<div class="text-xs font-normal text-secondary">
For the same repository. Windmill stores it in place of the current one and renews it
from then on.
</div>
<TextInput bind:value={token} size="sm" inputProps={{ type: 'password' }} />
</div>
{#if error}
<Alert type="error" title="Could not replace the token" size="xs">{error}</Alert>
{/if}
<div class="flex justify-end">
<Button
variant="accent"
unifiedSize="sm"
disabled={!token || saving}
startIcon={{ icon: saving ? Loader2 : KeyRound, classes: saving ? 'animate-spin' : '' }}
onclick={() => replace(close)}
>
Replace
</Button>
</div>
</div>
</div>
{/snippet}
</Popover>
+2 -2
View File
@@ -1,6 +1,6 @@
{
"gitSyncTest": "hub/28184/git-repo-test-read-write-windmill",
"gitInitRepo": "hub/28930/git-sync-init-repository-windmill",
"gitSyncTest": "hub/28950/git-repo-test-read-write-windmill",
"gitInitRepo": "hub/28948/git-sync-init-repository-windmill",
"slackErrorHandler": "hub/28794/workspace-or-schedule-error-handler-slack",
"emailErrorHandler": "hub/19795/workspace-or-error-handler-email",
"slackRecoveryHandler": "hub/28791/slack/schedule-recovery-handler-slack",