mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-09 00:04:10 +00:00
feat: bring gitlab repositories to parity for git sync (#10938)
* feat: track and rotate gitlab git-sync repository tokens * chore: point ee-repo-ref at the gitlab credential branch * fix: strip server-owned credential status and correct expiry copy * fix: gate credential maintenance on enterprise and alert on stalled renewal * fix: alert on an auto-renewed token only once it has actually expired * feat: receive gitlab push webhooks for instant git sync pull * feat: open gitlab merge requests and post diff previews on them Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep gitlab merge request previews out of the project's own pipeline Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: bound the credential maintenance pass and gate the gitlab picker on a license Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: create the gitlab picker's variable in the edited workspace Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: make the gitlab picker's variable path collision-resistant Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: state the gitlab scope and rotation facts the code relies on Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: resolve the check marker's repository from its path, not a stored url Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: refuse to finish a check whose repository has been repointed Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: trust a check marker's captured url when it carries no identity Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: recreate a missing webhook from credential maintenance Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: state that relative-url gitlab installs are out of scope Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep credential status out of exports and clear stale webhook warnings Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: refuse an unprovable check and guard the picker on the stored repository Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: re-check the picker's target path at the moment it is written Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: snapshot the picker's inputs before it starts writing Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: recommend a project access token per repository Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * [ee] feat: keep the git-sync credential in workspace settings * [ee] fix: drop a removed repo's credential and honor the workspace override * [ee] fix: resolve a fork's git credential from its whole ancestry * [ee] refactor: reuse fork_ancestor_chain instead of a second ancestry walk * [ee] fix: resolve an app installation from the whole ancestry, not the parent * [ee] revert: keep the app installation fallback at one level * fix: store the git credential only once the resource is saved * fix: keep a repository's credential when it leaves git sync settings * docs: cut the gitlab picker's token guidance down to what it needs * feat: mark a repository whose credential windmill holds * fix: ignore the managed-credential marker when the url carries a token * docs: drop the picker's setup alert for a line by the token field * feat: replace a repository's stored token from its resource * fix: store a picked credential for its own workspace, before the resource * refactor: key a stored git credential by its repository, not its resource Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: refresh the sqlx cache for the repository-keyed credential queries Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: gate the credential pass budget on the features that use it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: decide credential rotation ownership by repository, not resource path Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: renew only the credentials windmill holds, not tokens in a repo url Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: authenticate the fork-branch poll and correct the renewal guidance Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: do not claim a managed credential for a url the client cannot resolve Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: define the credential facade for private builds without enterprise Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: pin the listed token before the await and name the real renewal blocker Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: pin the token the replace flow checked, and derive the scope test once Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: classify the renewal state once so the card cannot contradict itself Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: ask only whether the token gets renewed, not why it does not Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: replace the managed-credential marker with a server answer Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: read renewal from the credential and its origin, not a removed field Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: read the provider for url-token repos, await the origin before defaulting, and visit unchecked repos last The maintenance pass sorted repositories with no recorded check first on the premise that they cost nothing, but a token-in-URL remote on a host that is not GitLab is probed every pass and never records a check, so it held the head of the list ahead of the tokens that expire. Such repositories now sort last. The card decided its delivery defaults before the origin lookup landed, so a freshly picked GitLab repository never got webhook delivery; the two lookups are awaited together. The resource editor offers to replace a token only where it is held, not in a fork that borrows it, and the replace flow refuses a URL it cannot parse instead of keying the token to it. Attaching a stored credential to a commit-hash probe now requires admin, matching the installation credential beside it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: describe the gitlab listing token the way the picker and the setup guide do Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: a token in the repository url is a plain remote, not a tracked credential Drops the status fingerprint that told one URL token from another, the docs' promise that such a token's expiry is reported, and the test's expectation that a URL-token repository declares a host. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: the card reads the credential origin for managed controls and honours the licence for a borrowed token Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: bump the ee ref Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: hide a repository's credential line once nothing is held for the repository it names Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: describe the exported credential status as it is Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: run the credential maintenance pass as its own task, without a budget The pass ran inside the monitor's join, whose deadline cancels every future in it, and a rotation cancelled between GitLab issuing a token and Windmill storing it loses the token family. A wall-clock budget with a least-recently-checked ordering kept it under the deadline. Spawning the pass instead makes the deadline irrelevant, so the budget, the ordering and the counter go; the advisory lock keeps a slow pass from overlapping the next, as it already did. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: say what detaching the maintenance pass buys, and what it does not Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: run git sync on the hub script version that reads a stored credential Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: run the deploy push and the connection test on the hub versions that read a stored credential Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep App repositories and plain remotes out of the stored-credential paths Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: host-neutral deploy preview wording, drop the project filter from the GitLab picker Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: bump ee ref, rotation no longer retains a second connection per repository Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: bump ee ref, the rotation write-back holds a single connection Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: hold the credential maintenance lock in a transaction so a dead sweep releases it Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: describe the credential-stored callback as it fires Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep the credential maintenance lock past the pool's idle-in-transaction timeout Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: update ee-repo-ref to e092518ee60e33160fee9ae91a4d109566f7b0ee This commit updates the EE repository reference after PR #771 was merged in windmill-ee-private. Previous ee-repo-ref: 74481f7cc345757aebb2a8b04d3a22978328c348 New ee-repo-ref: e092518ee60e33160fee9ae91a4d109566f7b0ee Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
windmill-internal-app[bot]
parent
6860521b4e
commit
9444049d60
+15
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "1d8ccd32266637d7f7915f92a8483dce0c9986f3847227c2f66daa84d4109d7d"
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT (\n SELECT elem\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' = $2\n )\n FROM workspace_settings WHERE workspace_id = $1 FOR UPDATE\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "elem",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "1fc9a0aeabb0a33efc37167be3de72dc33ca421338e04277ac91c745351f89f3"
|
||||
}
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT elem->'credential'\n FROM workspace_settings, jsonb_array_elements(git_sync->'repositories') AS elem\n WHERE workspace_id = $1 AND elem->>'git_repo_resource_path' IN ($2, $3)\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "?column?",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "48055203c97499ab4fc4dcbe9271d3f9e80375b52748c458854a4883a6ecc8f6"
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE workspace_settings\n SET git_credentials = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' IS DISTINCT FROM $2\n ) || jsonb_build_array($3::jsonb)\n WHERE workspace_id = $1\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "64f72d7477f7c82d1596560ff6c31c79281d5cb2b0e81f1e832eac6eb3cc32e2"
|
||||
}
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts FROM workspace_settings WHERE workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "73a3417f5331032a9b2afcf5b1c260d5ead34a6a3783dd21db08c7ee049eb232"
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE workspace_settings\n SET git_credentials = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' IS DISTINCT FROM $2\n ) || jsonb_build_array($3::jsonb)\n WHERE workspace_id = $1\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "9c0ebee945eee03d667a4f38d18a4d446b8ea00cf51e02138a06c245ab33252e"
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE workspace_settings\n SET git_credentials = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' IS DISTINCT FROM $2\n ) || jsonb_build_array($3::jsonb)\n WHERE workspace_id = $1\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "af5609185c6218d12fcb4a4234cfd2a14b2e9e613bcf4bc80abbfd495637ab2f"
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT pg_advisory_unlock(hashtext($1)::bigint)",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "pg_advisory_unlock",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "b2f91eb32edd8db1605aeb5a17b6e96cd3a980411d80c6b10bae949cf2c42385"
|
||||
}
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE workspace_settings\n SET git_sync = jsonb_set(git_sync, '{repositories}',\n COALESCE((SELECT jsonb_agg(\n CASE WHEN elem->>'git_repo_resource_path' IN ($2, $3)\n THEN CASE WHEN $4::jsonb = 'null'::jsonb\n THEN elem - 'credential'\n ELSE jsonb_set(elem, '{credential}', $4) END\n ELSE elem END)\n FROM jsonb_array_elements(git_sync->'repositories') AS elem), '[]'::jsonb)\n )\n WHERE workspace_id = $1\n AND jsonb_typeof(git_sync->'repositories') = 'array'\n AND EXISTS (\n SELECT 1 FROM jsonb_array_elements(git_sync->'repositories') AS e\n WHERE e->>'git_repo_resource_path' IN ($2, $3)\n )\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "bbb8331348216892e4714e4338c496c9448cffba5786b51e748bec0165a6519c"
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT elem\n FROM workspace_settings, jsonb_array_elements(git_credentials) AS elem\n WHERE workspace_id = $1 AND elem->>'repo_identity' = $2\n LIMIT 1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "elem",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "ce3340a43a141cc7527211db16905d53bc756d00c8d28462097717502d82cc15"
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT pg_try_advisory_lock(hashtext($1)::bigint)",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "pg_try_advisory_lock",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "d0eeb992a826d26376e9802a8c4ced12d26f67f1b059c7c2afe66f8e4b3a9749"
|
||||
}
|
||||
+26
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT ws.workspace_id, ws.git_sync\n FROM workspace_settings ws\n JOIN workspace w ON w.id = ws.workspace_id\n WHERE NOT w.deleted\n AND ws.git_sync IS NOT NULL\n AND jsonb_typeof(ws.git_sync->'repositories') = 'array'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "workspace_id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "git_sync",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "ddd41de79b23b0436bbc7997751c90d7d82bd20ea30f1dc08f2e2066fb8d6f4b"
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT (\n SELECT elem\n FROM jsonb_array_elements(git_credentials) AS elem\n WHERE elem->>'repo_identity' = $2\n )\n FROM workspace_settings WHERE workspace_id = $1 FOR UPDATE\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "elem",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "f3a27f7781d986d9917a7022e1118313f0434758413dbf83f8575d09de98fc25"
|
||||
}
|
||||
@@ -1 +1 @@
|
||||
d33ea730c550cdbc7d050aeb6d40dcef3d134e07
|
||||
e092518ee60e33160fee9ae91a4d109566f7b0ee
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE workspace_settings DROP COLUMN IF EXISTS git_credentials;
|
||||
@@ -0,0 +1,8 @@
|
||||
-- Server-owned git-sync credentials, one entry per repository the workspace
|
||||
-- holds a token for, keyed by that repository rather than by a resource naming
|
||||
-- it: a resource's URL is writable, and its path is not settled while it is
|
||||
-- being created.
|
||||
-- Kept out of `git_sync` because that column is copied into forks and returned
|
||||
-- by the workspace settings API; this one is copied by neither.
|
||||
ALTER TABLE workspace_settings
|
||||
ADD COLUMN IF NOT EXISTS git_credentials JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||
+178
-11
@@ -354,7 +354,9 @@ pub async fn initial_load(
|
||||
)
|
||||
}
|
||||
});
|
||||
pass.action(windmill_common::min_version::store_min_keep_alive_version(db));
|
||||
pass.action(windmill_common::min_version::store_min_keep_alive_version(
|
||||
db,
|
||||
));
|
||||
pass.setting(
|
||||
windmill_common::global_settings::INSTANCE_EVENTS_WEBHOOK_SETTING,
|
||||
false,
|
||||
@@ -709,7 +711,6 @@ pub async fn initial_load(
|
||||
pass.run(conn).await;
|
||||
}
|
||||
|
||||
|
||||
pub fn apply_metrics_enabled(value: Option<serde_json::Value>) {
|
||||
if let Some(serde_json::Value::Bool(t)) = value {
|
||||
METRICS_ENABLED.store(t, Ordering::Relaxed)
|
||||
@@ -1066,8 +1067,8 @@ pub fn apply_fork_workspace_tag_append_fork_suffix(value: Option<serde_json::Val
|
||||
}
|
||||
|
||||
pub async fn reload_critical_alert_mute_ui_setting(conn: &Connection) -> error::Result<()> {
|
||||
let v =
|
||||
load_value_from_global_settings_with_conn(conn, CRITICAL_ALERT_MUTE_UI_SETTING, true).await?;
|
||||
let v = load_value_from_global_settings_with_conn(conn, CRITICAL_ALERT_MUTE_UI_SETTING, true)
|
||||
.await?;
|
||||
apply_critical_alert_mute_ui_setting(v);
|
||||
Ok(())
|
||||
}
|
||||
@@ -2732,7 +2733,6 @@ pub async fn reload_timeout_wait_result_setting(conn: &Connection) {
|
||||
.await;
|
||||
}
|
||||
|
||||
|
||||
pub async fn reload_extra_pip_index_url_setting(conn: &Connection) {
|
||||
reload_option_setting_with_tracing(
|
||||
conn,
|
||||
@@ -2823,7 +2823,6 @@ pub async fn reload_bunfig_install_scopes_setting(conn: &Connection) {
|
||||
.await;
|
||||
}
|
||||
|
||||
|
||||
pub async fn reload_nuget_config_setting(conn: &Connection) {
|
||||
reload_option_setting_with_tracing(
|
||||
conn,
|
||||
@@ -2931,7 +2930,6 @@ pub async fn reload_ruby_repos_setting(conn: &Connection) {
|
||||
.await;
|
||||
}
|
||||
|
||||
|
||||
pub async fn reload_workspace_registries_setting(conn: &Connection) {
|
||||
match load_value_from_global_settings_with_conn(
|
||||
conn,
|
||||
@@ -3185,7 +3183,6 @@ pub async fn apply_job_isolation_setting(value: Option<serde_json::Value>) {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
async fn resolve_license_key_value(conn: &Connection, quiet: bool) -> anyhow::Result<String> {
|
||||
let q = load_value_from_global_settings_with_conn(conn, LICENSE_KEY_SETTING, true)
|
||||
.await
|
||||
@@ -3480,7 +3477,10 @@ impl<'a> SettingsPass<'a> {
|
||||
// on compile-time defaults until the next full reload. Only the single-query transport
|
||||
// can fail this way; over HTTP the batch already is the per-setting read.
|
||||
if matches!(conn, Connection::Sql(_)) && values.is_empty() && !names.is_empty() {
|
||||
tracing::warn!("Falling back to per-setting reads for {} settings", names.len());
|
||||
tracing::warn!(
|
||||
"Falling back to per-setting reads for {} settings",
|
||||
names.len()
|
||||
);
|
||||
values = fetch_settings_individually(conn, &names).await;
|
||||
}
|
||||
for (name, http) in &declared {
|
||||
@@ -3872,7 +3872,6 @@ pub fn parse_setting_value<T: FromStr + DeserializeOwned + Display>(
|
||||
value
|
||||
}
|
||||
|
||||
|
||||
#[cfg(feature = "prometheus")]
|
||||
pub async fn monitor_pool(db: &DB) {
|
||||
if METRICS_ENABLED.load(Ordering::Relaxed) {
|
||||
@@ -4316,6 +4315,24 @@ pub async fn monitor_db(
|
||||
}
|
||||
};
|
||||
|
||||
// Re-check what each git-sync repository's own credential says about its expiry,
|
||||
// and rotate the ones close to it. Every ~40 min: the values move over days, and
|
||||
// `should_run` counts iterations in a u8. Spawned rather than joined: the join
|
||||
// below is cancelled at its deadline, which a long sweep would reach, and a
|
||||
// rotation cut off between GitLab issuing a token and Windmill storing it
|
||||
// loses the token family. Detached, only process shutdown can cut it off,
|
||||
// which a rotation almost never coincides with. The pass's advisory lock
|
||||
// keeps a slow one from overlapping the next.
|
||||
let git_credential_maintenance_f = async {
|
||||
#[cfg(all(feature = "enterprise", feature = "private"))]
|
||||
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(240) {
|
||||
if let Some(db) = conn.as_sql() {
|
||||
let db = db.clone();
|
||||
tokio::spawn(async move { maintain_git_credentials(&db).await });
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// run every 2 iterations (~20s at the default LISTEN_NEW_EVENTS_INTERVAL_SEC).
|
||||
// Enterprise feature: the active `// freshness` backstop lives in
|
||||
// windmill-queue's `freshness_watchdog` (`private`); OSS gets a no-op stub.
|
||||
@@ -4369,6 +4386,7 @@ pub async fn monitor_db(
|
||||
export_audit_logs_to_object_store_f,
|
||||
cleanup_scheduled_job_deletions_f,
|
||||
git_auto_pull_f,
|
||||
git_credential_maintenance_f,
|
||||
pipeline_freshness_watchdog_f,
|
||||
reconcile_unarmed_schedules_f,
|
||||
);
|
||||
@@ -4691,6 +4709,156 @@ lazy_static::lazy_static! {
|
||||
#[cfg(feature = "private")]
|
||||
const AUTO_PULL_POLL_SLACK_S: i64 = 30;
|
||||
|
||||
/// Advisory lock id ensuring only one server replica maintains git credentials at
|
||||
/// a time (adjacent to GIT_AUTO_PULL_LOCK_ID).
|
||||
#[cfg(all(feature = "enterprise", feature = "private"))]
|
||||
const GIT_CREDENTIAL_LOCK_ID: i64 = 737_483_923;
|
||||
|
||||
/// Refresh every git-sync repository's credential status and rotate the ones near
|
||||
/// expiry, so a token dies visibly (and usually not at all) rather than taking
|
||||
/// sync down on its expiry date.
|
||||
#[cfg(all(feature = "enterprise", feature = "private"))]
|
||||
async fn maintain_git_credentials(db: &Pool<Postgres>) {
|
||||
use windmill_common::ee_oss::{get_license_plan, LicensePlan};
|
||||
|
||||
if !matches!(get_license_plan().await, LicensePlan::Enterprise) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Transaction-scoped advisory lock, as for the schedule reconcile above: a
|
||||
// session lock on a pooled connection would ride back into the pool still
|
||||
// held if the sweep died before unlocking, and wedge the pass on every
|
||||
// replica until a restart. The transaction only owns the lock; the sweep
|
||||
// commits each status and each rotated token on its own as it goes.
|
||||
let mut lock_tx = match db.begin().await {
|
||||
Ok(tx) => tx,
|
||||
Err(e) => {
|
||||
tracing::error!("git credentials: failed to begin lock tx: {e:#}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
// The transaction stays idle while the sweep talks to git hosts, and the
|
||||
// pool's ten-minute idle-in-transaction timeout would end it, lock included,
|
||||
// partway through a sweep over enough slow hosts. Lifted for this
|
||||
// transaction only; it dies with the connection either way.
|
||||
if let Err(e) = sqlx::query("SET LOCAL idle_in_transaction_session_timeout = 0")
|
||||
.execute(&mut *lock_tx)
|
||||
.await
|
||||
{
|
||||
tracing::error!("git credentials: failed to lift the idle timeout: {e:#}");
|
||||
return;
|
||||
}
|
||||
let locked: bool = match sqlx::query_scalar("SELECT pg_try_advisory_xact_lock($1)")
|
||||
.bind(GIT_CREDENTIAL_LOCK_ID)
|
||||
.fetch_one(&mut *lock_tx)
|
||||
.await
|
||||
{
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
tracing::error!("git credentials: advisory lock failed: {e:#}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
if !locked {
|
||||
return;
|
||||
}
|
||||
|
||||
if let Err(e) = maintain_git_credentials_inner(db).await {
|
||||
tracing::error!("git credentials: maintenance error: {e:#}");
|
||||
}
|
||||
drop(lock_tx);
|
||||
}
|
||||
|
||||
#[cfg(all(feature = "enterprise", feature = "private"))]
|
||||
async fn maintain_git_credentials_inner(db: &Pool<Postgres>) -> error::Result<()> {
|
||||
use windmill_common::workspaces::WorkspaceGitSyncSettings;
|
||||
|
||||
// Same deleted/archived exclusion as the auto-pull poller: a dead workspace's
|
||||
// settings row survives, and rotating a token for one would be pure damage.
|
||||
let rows = sqlx::query!(
|
||||
r#"SELECT ws.workspace_id, ws.git_sync
|
||||
FROM workspace_settings ws
|
||||
JOIN workspace w ON w.id = ws.workspace_id
|
||||
WHERE NOT w.deleted
|
||||
AND ws.git_sync IS NOT NULL
|
||||
AND jsonb_typeof(ws.git_sync->'repositories') = 'array'"#
|
||||
)
|
||||
.fetch_all(db)
|
||||
.await?;
|
||||
|
||||
for row in rows {
|
||||
let Some(git_sync) = row.git_sync else {
|
||||
continue;
|
||||
};
|
||||
let settings: WorkspaceGitSyncSettings = match serde_json::from_value(git_sync) {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"git credentials: invalid git_sync settings for workspace {}: {e}",
|
||||
row.workspace_id
|
||||
);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
for repo in settings.repositories.iter() {
|
||||
let path = &repo.git_repo_resource_path;
|
||||
// This refreshes and records the status on every repository it looks at,
|
||||
// rotating only the ones near expiry, so it is the whole maintenance pass
|
||||
// rather than just the rotation half.
|
||||
if let Err(e) = windmill_common::git_sync_ee::rotate_git_credential_if_due(
|
||||
db,
|
||||
&row.workspace_id,
|
||||
path,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::error!(
|
||||
"git credentials: maintenance failed for {path} in workspace {}: {e:#}",
|
||||
row.workspace_id
|
||||
);
|
||||
}
|
||||
|
||||
// A repository that wants webhook delivery but holds no hook never
|
||||
// gets one otherwise: the reconcile runs on a settings save, so a
|
||||
// credential that was unusable when the hook should have been created
|
||||
// would leave it missing until an admin saved again. Checking stored
|
||||
// state costs nothing, and only the repositories actually missing a
|
||||
// hook reach the host.
|
||||
use windmill_common::workspaces::AutoPullMode;
|
||||
// Also when a hook exists but carries a warning: a save during a GitLab
|
||||
// outage keeps the hook and records why it could not be confirmed, and
|
||||
// that warning is only cleared by a reconcile that confirms it again.
|
||||
// Only repositories with a checked credential of their own: for
|
||||
// everything else a settings save stays the one place hooks are
|
||||
// reconciled, so an App repository or a plain remote is never touched
|
||||
// here, and a recorded delivery mode nobody saved is never normalized.
|
||||
let needs_hook = repo.credential.is_some()
|
||||
&& repo.auto_pull.as_ref().is_some_and(|a| {
|
||||
a.enabled
|
||||
&& matches!(a.mode, AutoPullMode::Auto | AutoPullMode::Webhook)
|
||||
&& (a.webhook_id.is_none() || a.webhook_error.is_some())
|
||||
});
|
||||
if needs_hook {
|
||||
let mut repo = repo.clone();
|
||||
if let Err(e) = windmill_common::git_sync_ee::sync_repo_webhook(
|
||||
db,
|
||||
&row.workspace_id,
|
||||
&mut repo,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!(
|
||||
"git credentials: could not reconcile the webhook for {path} in workspace {}: {e:#}",
|
||||
row.workspace_id
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "private")]
|
||||
async fn poll_git_auto_pull_inner(db: &Pool<Postgres>) -> error::Result<()> {
|
||||
use windmill_common::workspaces::{AutoPullMode, WorkspaceGitSyncSettings};
|
||||
@@ -6588,7 +6756,6 @@ pub async fn reload_critical_alerts_on_db_oversize(conn: &DB) -> error::Result<(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
pub async fn reload_jwt_secret_setting(db: &DB) -> error::Result<()> {
|
||||
let v = load_value_from_global_settings(db, JWT_SECRET_SETTING).await?;
|
||||
apply_jwt_secret_setting(db, v).await
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
-- A parent whose git-sync repository has a recorded credential, and the workspace
|
||||
-- shapes the credential lookup and the qualification predicate have to tell apart.
|
||||
--
|
||||
-- The credential itself is shared down the fork chain; the recorded *status* is
|
||||
-- not, because it describes one repository and a fork can repoint its copy of the
|
||||
-- resource. Forks get a status by fork creation copying it, which no fixture here
|
||||
-- simulates, so a fork without one is a workspace nothing has checked yet.
|
||||
|
||||
INSERT INTO workspace (id, name, owner, parent_workspace_id) VALUES
|
||||
('parent-ws', 'parent-ws', 'test-user', NULL),
|
||||
('fork-ws', 'fork-ws', 'test-user', 'parent-ws'),
|
||||
-- A fork of a fork: the shape a fork of a dev workspace takes, and the one a
|
||||
-- parent-only lookup misses.
|
||||
('deep-fork-ws', 'deep-fork-ws', 'test-user', 'fork-ws'),
|
||||
('errored-fork-ws', 'errored-fork-ws', 'test-user', 'parent-ws'),
|
||||
('orphan-ws', 'orphan-ws', 'test-user', NULL);
|
||||
|
||||
-- A stored credential is encrypted with its own workspace's key.
|
||||
INSERT INTO workspace_key (workspace_id, kind, key) VALUES
|
||||
('parent-ws', 'cloud', 'parent-key'),
|
||||
('fork-ws', 'cloud', 'fork-key'),
|
||||
('deep-fork-ws', 'cloud', 'deep-fork-key'),
|
||||
('errored-fork-ws', 'cloud', 'errored-fork-key'),
|
||||
('orphan-ws', 'cloud', 'orphan-key');
|
||||
|
||||
-- The parent holds the credential.
|
||||
INSERT INTO workspace_settings (workspace_id, git_sync) VALUES
|
||||
('parent-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo",
|
||||
"credential":{"provider":"gitlab","rotatable":true,"checked_at":1788500000}}]}'),
|
||||
|
||||
-- A fork inherits the repository but not the credential: this is what
|
||||
-- clone_workspace_data leaves behind.
|
||||
('fork-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo"}]}'),
|
||||
|
||||
-- Two levels down, so neither the credential nor its status is one hop away.
|
||||
('deep-fork-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo"}]}'),
|
||||
|
||||
-- A fork whose own credential has since failed. Its own standing must win over
|
||||
-- the parent's healthy record rather than being papered over.
|
||||
('errored-fork-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo",
|
||||
"credential":{"provider":"gitlab","rotatable":false,"checked_at":1788500000,
|
||||
"error":"GitLab no longer accepts this token"}}]}'),
|
||||
|
||||
-- No credential and no parent to borrow one from.
|
||||
('orphan-ws', '{"repositories":[{"git_repo_resource_path":"$res:u/admin/repo"}]}');
|
||||
|
||||
-- The resource each repository entry names, all pointing at the same repository.
|
||||
-- The errored fork carries its token in the URL, the way a repository configured
|
||||
-- by hand does: a plain remote, whatever the chain above it holds.
|
||||
INSERT INTO resource (workspace_id, path, value, resource_type) VALUES
|
||||
('parent-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository'),
|
||||
('fork-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository'),
|
||||
('deep-fork-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository'),
|
||||
('errored-fork-ws', 'u/admin/repo', '{"url":"https://oauth2:glpat-inline@gitlab.com/grp/proj.git"}', 'git_repository'),
|
||||
('orphan-ws', 'u/admin/repo', '{"url":"https://gitlab.com/grp/proj.git"}', 'git_repository');
|
||||
@@ -0,0 +1,252 @@
|
||||
//! A fork reaches the git credential held above it in its fork chain.
|
||||
//!
|
||||
//! Fork creation copies the parent's git-sync repositories but not the credential,
|
||||
//! which is stored per workspace so that rotation has one owner. Chains nest (a
|
||||
//! fork of a dev workspace, a fork of that), so the depth-2 cases here are what
|
||||
//! keep the lookup from regressing to the parent.
|
||||
//!
|
||||
//! The recorded *status* is not shared the same way: it describes one repository,
|
||||
//! and a fork can repoint its copy of the resource, so each workspace answers from
|
||||
//! its own record and gets one by fork creation copying it down.
|
||||
#![cfg(all(feature = "enterprise", feature = "private"))]
|
||||
|
||||
use sqlx::{Pool, Postgres};
|
||||
use windmill_common::git_sync_ee::{
|
||||
git_credential_for_url, repo_provider, repo_supports_managed_git_features, set_git_credential,
|
||||
GitProvider,
|
||||
};
|
||||
use windmill_common::workspaces::GitCredentialProvider;
|
||||
|
||||
const REPO: &str = "$res:u/admin/repo";
|
||||
const URL: &str = "https://gitlab.com/grp/proj.git";
|
||||
|
||||
/// A repository is managed when a credential is held for the repository its
|
||||
/// URL names now and the last check found it healthy. The recorded status is
|
||||
/// keyed by resource path, so alone it would outlive a repoint; the held
|
||||
/// credential alone says nothing about whether the host still accepts it.
|
||||
#[sqlx::test(fixtures("git_sync_fork_credential"))]
|
||||
async fn credential_status_is_a_workspaces_own(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
assert!(
|
||||
!repo_supports_managed_git_features(&db, "parent-ws", REPO).await,
|
||||
"a healthy status with nothing held behind it does not qualify"
|
||||
);
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
URL,
|
||||
"glpat-secret",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
assert!(
|
||||
repo_supports_managed_git_features(&db, "parent-ws", REPO).await,
|
||||
"the workspace holding both the credential and the recorded status qualifies"
|
||||
);
|
||||
assert!(
|
||||
!repo_supports_managed_git_features(&db, "fork-ws", REPO).await,
|
||||
"a fork borrowing the credential with no record of its own does not: the \
|
||||
status describes one repository, and this fork's resource could name another"
|
||||
);
|
||||
assert!(
|
||||
!repo_supports_managed_git_features(&db, "errored-fork-ws", REPO).await,
|
||||
"a workspace whose own credential failed stays disqualified"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The host a repository talks to is declared when its credential is stored, and
|
||||
/// travels with the credential down the fork chain.
|
||||
///
|
||||
/// Read from the recorded status instead, a fork answered with the default
|
||||
/// provider until its own check ran, which is long enough to register a webhook
|
||||
/// against the wrong receiver.
|
||||
#[sqlx::test(fixtures("git_sync_fork_credential"))]
|
||||
async fn the_provider_comes_from_the_credential_and_reaches_forks(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
assert_eq!(
|
||||
repo_provider(&db, "parent-ws", REPO).await,
|
||||
GitProvider::GitHub,
|
||||
"with nothing stored there is no declaration to read, so the default stands"
|
||||
);
|
||||
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
URL,
|
||||
"glpat-secret",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
|
||||
assert_eq!(
|
||||
repo_provider(&db, "parent-ws", REPO).await,
|
||||
GitProvider::GitLab,
|
||||
"the workspace that stored it reads its own declaration"
|
||||
);
|
||||
assert_eq!(
|
||||
repo_provider(&db, "fork-ws", REPO).await,
|
||||
GitProvider::GitLab,
|
||||
"and a fork resolving that credential reads it too, without a check of its own"
|
||||
);
|
||||
assert_eq!(
|
||||
repo_provider(&db, "deep-fork-ws", REPO).await,
|
||||
GitProvider::GitLab,
|
||||
"two levels down as well"
|
||||
);
|
||||
assert_eq!(
|
||||
repo_provider(&db, "orphan-ws", REPO).await,
|
||||
GitProvider::GitHub,
|
||||
"a workspace outside the chain resolves no credential and no declaration"
|
||||
);
|
||||
assert_eq!(
|
||||
repo_provider(&db, "errored-fork-ws", REPO).await,
|
||||
GitProvider::GitHub,
|
||||
"a token written into the URL makes the repository a plain remote: the \
|
||||
parent's credential is not consulted and no host is declared"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The stored credential is shared with forks and keyed by one repository.
|
||||
///
|
||||
/// Both properties are the point of keeping it in `workspace_settings` under the
|
||||
/// repository's identity: sharing is what stops a rotation from stranding every
|
||||
/// fork on a revoked token, and the key is what stops a rewritten resource URL
|
||||
/// from carrying the token to a host of the writer's choosing.
|
||||
#[sqlx::test(fixtures("git_sync_fork_credential"))]
|
||||
async fn a_fork_reads_an_ancestors_credential_for_the_bound_repository_only(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
URL,
|
||||
"glpat-secret",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "parent-ws", URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-secret"),
|
||||
"the workspace that stored it reads it back"
|
||||
);
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "fork-ws", URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-secret"),
|
||||
"a fork stores none of its own and resolves the parent's"
|
||||
);
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "deep-fork-ws", URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-secret"),
|
||||
"a fork of a fork resolves the root's, two levels up"
|
||||
);
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "fork-ws", "https://evil.example/grp/proj.git").await?,
|
||||
None,
|
||||
"a resource repointed at another repository asks for that one's \
|
||||
credential and finds none"
|
||||
);
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "orphan-ws", URL).await?,
|
||||
None,
|
||||
"a workspace with no credential and no parent resolves nothing"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// One repository's credential is untouched by another's.
|
||||
///
|
||||
/// The key is the repository, so picking a second repository stores beside the
|
||||
/// first rather than over it. Keyed by the resource instead, a workspace editing
|
||||
/// one repository's resource to point somewhere else would replace the token the
|
||||
/// original repository was still syncing with.
|
||||
#[sqlx::test(fixtures("git_sync_fork_credential"))]
|
||||
async fn each_repository_keeps_its_own_credential(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
const OTHER_URL: &str = "https://gitlab.com/grp/other.git";
|
||||
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
URL,
|
||||
"glpat-first",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
OTHER_URL,
|
||||
"glpat-second",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "parent-ws", URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-first"),
|
||||
"storing a second repository's token leaves the first's in place"
|
||||
);
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "parent-ws", OTHER_URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-second")
|
||||
);
|
||||
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
URL,
|
||||
"glpat-replacement",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "parent-ws", URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-replacement"),
|
||||
"storing the same repository again replaces rather than duplicates"
|
||||
);
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "parent-ws", OTHER_URL)
|
||||
.await?
|
||||
.as_deref(),
|
||||
Some("glpat-second"),
|
||||
"and still leaves the other repository alone"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A credential issued for `https` is not served for the `http` spelling.
|
||||
///
|
||||
/// The resource holding the URL is writable by anyone with write on its path, so
|
||||
/// without the scheme in the key that edit would send the token over cleartext.
|
||||
#[sqlx::test(fixtures("git_sync_fork_credential"))]
|
||||
async fn a_credential_is_not_served_over_a_downgraded_transport(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
set_git_credential(
|
||||
&db,
|
||||
"parent-ws",
|
||||
URL,
|
||||
"glpat-secret",
|
||||
GitCredentialProvider::Gitlab,
|
||||
)
|
||||
.await?;
|
||||
assert_eq!(
|
||||
git_credential_for_url(&db, "parent-ws", "http://gitlab.com/grp/proj.git").await?,
|
||||
None
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
@@ -3986,6 +3986,7 @@ async fn edit_git_sync_config(
|
||||
clear_client_supplied_auto_pull_state(ap);
|
||||
}
|
||||
repo.open_pr_error = None;
|
||||
repo.credential = None;
|
||||
}
|
||||
reject_parent_only_git_sync_settings_on_fork(
|
||||
&db,
|
||||
@@ -4086,6 +4087,7 @@ async fn edit_git_sync_config(
|
||||
continue;
|
||||
};
|
||||
repo.open_pr_error = old.open_pr_error.clone();
|
||||
repo.credential = old.credential.clone();
|
||||
if let (Some(new_ap), Some(old_ap)) =
|
||||
(repo.auto_pull.as_mut(), old.auto_pull.as_ref())
|
||||
{
|
||||
@@ -4131,6 +4133,7 @@ async fn edit_git_sync_config(
|
||||
.flatten()
|
||||
.and_then(|v| serde_json::from_value(v).ok());
|
||||
let removed_webhooks: Vec<(String, i64)> = existing
|
||||
.as_ref()
|
||||
.map(|e| {
|
||||
e.repositories
|
||||
.iter()
|
||||
@@ -4164,6 +4167,19 @@ async fn edit_git_sync_config(
|
||||
// `sync_repo_webhook` writes back the webhook fields it changes itself:
|
||||
// the remote hook and the record of it have to move together, so
|
||||
// persisting them out here would let one land without the other.
|
||||
// Before the webhook reconcile, which decides whether this repo can have
|
||||
// one from the credential this records. Also puts a short-lived or
|
||||
// under-scoped token in front of the operator while they are still on the
|
||||
// settings page, rather than when it expires.
|
||||
if let Err(e) = windmill_common::git_sync_ee::refresh_git_credential_status(
|
||||
&db,
|
||||
&w_id,
|
||||
&repo.git_repo_resource_path,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!("git credential check error: {}", e);
|
||||
}
|
||||
if let Err(e) = windmill_common::git_sync_ee::sync_repo_webhook(&db, &w_id, repo).await
|
||||
{
|
||||
tracing::warn!("git auto-pull: webhook sync error: {}", e);
|
||||
@@ -4217,6 +4233,7 @@ async fn edit_git_sync_repository(
|
||||
clear_client_supplied_auto_pull_state(ap);
|
||||
}
|
||||
new_config.repository.open_pr_error = None;
|
||||
new_config.repository.credential = None;
|
||||
reject_parent_only_git_sync_settings_on_fork(
|
||||
&db,
|
||||
&w_id,
|
||||
@@ -4341,6 +4358,7 @@ async fn edit_git_sync_repository(
|
||||
// from the UI cannot revert what the poller/webhook layer wrote.
|
||||
let mut updated = new_config.repository;
|
||||
updated.open_pr_error = existing_repo.open_pr_error.clone();
|
||||
updated.credential = existing_repo.credential.clone();
|
||||
match (updated.auto_pull.as_mut(), existing_repo.auto_pull.as_ref()) {
|
||||
(Some(new_ap), Some(old_ap)) => {
|
||||
new_ap.last_synced_sha = old_ap.last_synced_sha.clone();
|
||||
@@ -4398,6 +4416,15 @@ async fn edit_git_sync_repository(
|
||||
.iter_mut()
|
||||
.find(|r| r.git_repo_resource_path == new_config.git_repo_resource_path)
|
||||
{
|
||||
if let Err(e) = windmill_common::git_sync_ee::refresh_git_credential_status(
|
||||
&db,
|
||||
&w_id,
|
||||
&repo.git_repo_resource_path,
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::warn!("git credential check error: {}", e);
|
||||
}
|
||||
if let Err(e) = windmill_common::git_sync_ee::sync_repo_webhook(&db, &w_id, repo).await {
|
||||
tracing::warn!("git auto-pull: webhook sync error: {}", e);
|
||||
}
|
||||
@@ -4537,6 +4564,10 @@ async fn delete_git_sync_repository(
|
||||
}
|
||||
}
|
||||
|
||||
// The stored credential is deliberately left alone: it belongs to the git
|
||||
// repository resource, which this endpoint does not delete, and the resource
|
||||
// still authenticates with it for connection tests and commit lookups.
|
||||
|
||||
// Trigger git sync for repository deletion
|
||||
handle_deployment_metadata(
|
||||
&authed.email,
|
||||
@@ -6306,9 +6337,10 @@ async fn update_workspace_settings(
|
||||
// Auto-pull and fork PRs are parent-owned and must not be inherited:
|
||||
// the fork would otherwise carry the parent's webhook id (turning off
|
||||
// auto-pull on the fork would delete the parent's webhook). A fork
|
||||
// still inherits the push-direction config and the installation.
|
||||
// Repo → fork sync is driven by the parent's webhook/poller
|
||||
// (`sync_forks`), which routes the fork's `wm-fork/**` branch into it.
|
||||
// still inherits the push-direction config, the installation, and the
|
||||
// recorded credential status, which describes the repository rather
|
||||
// than belonging to either workspace and would otherwise leave the
|
||||
// fork unqualified for managed features until its first check.
|
||||
r.auto_pull = None;
|
||||
r.fork_open_prs = false;
|
||||
r.open_pr_error = None;
|
||||
|
||||
@@ -113,7 +113,7 @@ pub(crate) async fn change_workspace_id(
|
||||
// Duplicate workspace settings (keep copy in old workspace for reference)
|
||||
info!("Duplicating workspace_settings table");
|
||||
sqlx::query!(
|
||||
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
|
||||
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
|
||||
&rw.new_id,
|
||||
&old_id
|
||||
)
|
||||
|
||||
@@ -2741,7 +2741,7 @@ paths:
|
||||
|
||||
/w/{workspace}/github_app/token:
|
||||
post:
|
||||
summary: get github app token
|
||||
summary: get the git credential for a git-sync job (GitHub App token, or the credential stored for the repository)
|
||||
operationId: getGithubAppToken
|
||||
tags:
|
||||
- workspace
|
||||
@@ -2761,7 +2761,7 @@ paths:
|
||||
- job_token
|
||||
responses:
|
||||
"200":
|
||||
description: github app token
|
||||
description: git credential
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
@@ -2905,6 +2905,131 @@ paths:
|
||||
"200":
|
||||
description: Successfully imported the installation
|
||||
|
||||
/w/{workspace}/git_sync/gitlab/projects:
|
||||
post:
|
||||
tags:
|
||||
- Git Sync
|
||||
summary: List the GitLab projects a token can sync
|
||||
description: >-
|
||||
Lists the projects the supplied GitLab token can push to, so a git
|
||||
repository resource can be filled in without hand-writing a project
|
||||
path. The token is used for this call only and is never stored.
|
||||
Requires workspace admin.
|
||||
operationId: listGitlabProjects
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
base_url:
|
||||
type: string
|
||||
description: The GitLab instance, e.g. https://gitlab.com
|
||||
token:
|
||||
type: string
|
||||
description: A project access token with the api scope, or a group token that reaches the project
|
||||
search:
|
||||
type: string
|
||||
description: Narrow the list to projects matching this text
|
||||
required:
|
||||
- base_url
|
||||
- token
|
||||
responses:
|
||||
"200":
|
||||
description: the projects the token can sync
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/GitlabProject"
|
||||
|
||||
/w/{workspace}/git_sync/credential/origin:
|
||||
get:
|
||||
tags:
|
||||
- Git Sync
|
||||
summary: Where a repository's credential comes from
|
||||
description: >-
|
||||
Whether Windmill holds this repository's access token, and which host it
|
||||
talks to. `held` means this workspace stores it, `borrowed` means an
|
||||
ancestor does and it is not this workspace's to replace. Both absent
|
||||
means the repository authenticates with whatever its URL carries.
|
||||
Returns no secret. Requires workspace admin.
|
||||
operationId: getCredentialOrigin
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- name: path
|
||||
in: query
|
||||
required: true
|
||||
description: >-
|
||||
Path of the git repository resource, with or without the `$res:`
|
||||
prefix. A path rather than a URL, because a resource URL may carry a
|
||||
token and a URL in a query string lands in logs.
|
||||
schema:
|
||||
type: string
|
||||
responses:
|
||||
"200":
|
||||
description: where the credential comes from
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
origin:
|
||||
type: string
|
||||
enum:
|
||||
- held
|
||||
- borrowed
|
||||
provider:
|
||||
type: string
|
||||
enum:
|
||||
- gitlab
|
||||
|
||||
/w/{workspace}/git_sync/credential:
|
||||
post:
|
||||
tags:
|
||||
- Git Sync
|
||||
summary: Store the credential for a git repository
|
||||
description: >-
|
||||
Stores the access token a git repository authenticates with, so it does
|
||||
not have to be written into the repository URL or a workspace variable.
|
||||
The token is write-only: it is served only to a git-sync job that
|
||||
presents its own job token, and a fork of this workspace reads this
|
||||
copy instead of holding one of its own. Requires workspace admin.
|
||||
operationId: setGitCredential
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
repo_url:
|
||||
type: string
|
||||
description: >-
|
||||
The repository the credential is for, and the key it is
|
||||
stored under. It is served for this repository and no other,
|
||||
so repointing a resource elsewhere cannot carry the token
|
||||
along.
|
||||
token:
|
||||
type: string
|
||||
description: The access token, as pasted
|
||||
required:
|
||||
- repo_url
|
||||
- token
|
||||
responses:
|
||||
"200":
|
||||
description: the credential was stored
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/github_app/ghes_installation_callback:
|
||||
post:
|
||||
summary: GHES installation callback
|
||||
@@ -34484,9 +34609,58 @@ components:
|
||||
open_pr_error:
|
||||
type: string
|
||||
description: server-owned, last failure opening a PR for a deploy branch of this repo
|
||||
credential:
|
||||
$ref: "#/components/schemas/GitCredentialStatus"
|
||||
required:
|
||||
- git_repo_resource_path
|
||||
|
||||
GitCredentialStatus:
|
||||
type: object
|
||||
description: server-owned, what the repo's own credential reports about itself
|
||||
properties:
|
||||
provider:
|
||||
type: string
|
||||
enum:
|
||||
- gitlab
|
||||
token_id:
|
||||
type: integer
|
||||
format: int64
|
||||
expires_at:
|
||||
type: string
|
||||
format: date
|
||||
description: absent for a non-expiring token
|
||||
scopes:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
rotatable:
|
||||
type: boolean
|
||||
description: whether this workspace renews the credential itself
|
||||
checked_at:
|
||||
type: integer
|
||||
format: int64
|
||||
error:
|
||||
type: string
|
||||
required:
|
||||
- provider
|
||||
- rotatable
|
||||
- checked_at
|
||||
|
||||
GitlabProject:
|
||||
type: object
|
||||
description: a GitLab project a token can sync, as the resource form needs it
|
||||
properties:
|
||||
path_with_namespace:
|
||||
type: string
|
||||
description: nested group path plus project name, which is also GitLab's project id
|
||||
http_url_to_repo:
|
||||
type: string
|
||||
default_branch:
|
||||
type: string
|
||||
required:
|
||||
- path_with_namespace
|
||||
- http_url_to_repo
|
||||
|
||||
AutoPullMode:
|
||||
type: string
|
||||
enum:
|
||||
|
||||
@@ -10,6 +10,11 @@ pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "private"))]
|
||||
pub fn workspaced_git_sync_service() -> Router {
|
||||
Router::new()
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "private"))]
|
||||
pub fn global_service() -> Router {
|
||||
Router::new()
|
||||
|
||||
@@ -959,6 +959,15 @@ pub async fn run_server(
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
Router::new()
|
||||
})
|
||||
.nest("/w/{workspace_id}/git_sync", {
|
||||
#[cfg(feature = "enterprise")]
|
||||
{
|
||||
git_sync_oss::workspaced_git_sync_service()
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
Router::new()
|
||||
})
|
||||
.nest(
|
||||
"/w/{workspace_id}/resources_u",
|
||||
public_service().layer(cors.clone()),
|
||||
|
||||
@@ -1586,10 +1586,11 @@ pub(crate) async fn tarball_workspace(
|
||||
.await?;
|
||||
|
||||
// Use v2 format only if explicitly requested, otherwise use v1 (legacy) for backward compatibility
|
||||
// Server-owned auto-pull state (the HMAC webhook secret + hook id/error and
|
||||
// the synced-sha / last-pull status) must never leave the server: keep it out
|
||||
// of export archives and synced repos, and don't let a re-imported workspace
|
||||
// inherit another install's hook/sync state. Mirrors the GET-settings redaction.
|
||||
// Server-owned state (the HMAC webhook secret + hook id/error, the
|
||||
// synced-sha / last-pull status, and what the credential check observed)
|
||||
// must never leave the server: keep it out of export archives and synced
|
||||
// repos, and don't let a re-imported workspace inherit another install's
|
||||
// hook/sync state. Mirrors the GET-settings redaction.
|
||||
fn redact_git_sync_for_export(git_sync: Option<Value>) -> Option<Value> {
|
||||
let mut git_sync = git_sync?;
|
||||
if let Some(repos) = git_sync
|
||||
@@ -1610,6 +1611,13 @@ pub(crate) async fn tarball_workspace(
|
||||
auto_pull.remove(field);
|
||||
}
|
||||
}
|
||||
// What this install observed about its own credential: a token
|
||||
// id and expiry, and a `checked_at` that moves on its own.
|
||||
// None of it describes the workspace, and in a git-synced
|
||||
// `wmill.yaml` it would churn the file for no reason.
|
||||
if let Some(repo) = repo.as_object_mut() {
|
||||
repo.remove("credential");
|
||||
}
|
||||
}
|
||||
}
|
||||
Some(git_sync)
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
#[cfg(feature = "private")]
|
||||
#[allow(unused)]
|
||||
pub use crate::git_sync_ee::*;
|
||||
#[cfg(not(feature = "private"))]
|
||||
#[cfg(not(all(feature = "private", feature = "enterprise")))]
|
||||
use sqlx::{Pool, Postgres};
|
||||
use url::Url;
|
||||
|
||||
#[cfg(not(feature = "private"))]
|
||||
/// Gated on the pair to match [`with_stored_credential`] below, whose callers
|
||||
/// reach it through this facade un-gated and so depend on it. Nothing routes
|
||||
/// here today (the one caller imports the enterprise item directly), so this is
|
||||
/// for uniformity: the next plain caller would otherwise find no definition
|
||||
/// under `private` without `enterprise`.
|
||||
#[cfg(not(all(feature = "private", feature = "enterprise")))]
|
||||
pub async fn get_github_app_token_internal(
|
||||
_db: &Pool<Postgres>,
|
||||
_job_token: &str,
|
||||
@@ -15,6 +20,21 @@ pub async fn get_github_app_token_internal(
|
||||
));
|
||||
}
|
||||
|
||||
/// Server-held git credentials are an enterprise feature, so on this build a
|
||||
/// repository URL authenticates with whatever it already carries.
|
||||
///
|
||||
/// Gated on the pair rather than on `private` alone: `private` does not imply
|
||||
/// `enterprise`, and the callers are plain (no `#[cfg]`), so a build with one
|
||||
/// and not the other would find neither this nor the enterprise definition.
|
||||
#[cfg(not(all(feature = "private", feature = "enterprise")))]
|
||||
pub async fn with_stored_credential(
|
||||
_db: &Pool<Postgres>,
|
||||
_w_id: &str,
|
||||
url: String,
|
||||
) -> crate::error::Result<String> {
|
||||
Ok(url)
|
||||
}
|
||||
|
||||
lazy_static::lazy_static! {
|
||||
/// Matches a `user:password@` (or `user@`) userinfo component right after the URL scheme.
|
||||
static ref GIT_URL_USERINFO_RE: regex::Regex =
|
||||
|
||||
@@ -183,7 +183,7 @@ pub enum ObjectType {
|
||||
DatatableMigration,
|
||||
}
|
||||
|
||||
pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28931/sync-script-to-git-repo-windmill";
|
||||
pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28949/sync-script-to-git-repo-windmill";
|
||||
|
||||
/// Hub script that applies a repository's state back into a workspace
|
||||
/// (the repo → Windmill / "pull" direction). Same script the UI runs from
|
||||
@@ -191,7 +191,7 @@ pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28931/sync-script-to-git-repo
|
||||
/// ignores the slug, so the slug is kept free of characters that would be
|
||||
/// percent-encoded into the run URL (a `:` becomes `%3A`, which some hardened
|
||||
/// reverse proxies reject as double-encoding when the client re-encodes it).
|
||||
pub const GIT_SYNC_PULL_SCRIPT_PATH: &str = "hub/28930/git-sync-init-repository-windmill";
|
||||
pub const GIT_SYNC_PULL_SCRIPT_PATH: &str = "hub/28948/git-sync-init-repository-windmill";
|
||||
|
||||
/// Prefix used to identify fork workspaces. A workspace whose id starts with this string is a
|
||||
/// fork of another workspace.
|
||||
@@ -345,13 +345,14 @@ pub struct GitRepositorySettings {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub auto_pull: Option<AutoPullSettings>,
|
||||
/// Open a PR when a deploy pushes a `wm_deploy/**` branch of this promotion
|
||||
/// repo (app-backed only; runs from the deploy callback so it works without
|
||||
/// repo (needs a credential the server holds — a GitHub App installation or
|
||||
/// a checked GitLab token; runs from the deploy callback so it works without
|
||||
/// inbound webhooks). Off by default so upgrades don't change behavior.
|
||||
#[serde(default, skip_serializing_if = "is_false")]
|
||||
pub promotion_open_prs: bool,
|
||||
/// Parent-level: open a PR when a fork of this workspace deploys to its
|
||||
/// `wm-fork/**` branch (app-backed only; the fork's deploy callback reads
|
||||
/// this from the parent). Off by default.
|
||||
/// `wm-fork/**` branch (needs a credential the server holds; the fork's
|
||||
/// deploy callback reads this from the parent). Off by default.
|
||||
#[serde(default, skip_serializing_if = "is_false")]
|
||||
pub fork_open_prs: bool,
|
||||
/// Server-owned: the last failure opening a PR for a deploy branch of this
|
||||
@@ -360,6 +361,10 @@ pub struct GitRepositorySettings {
|
||||
/// successful PR; never accepted from clients.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub open_pr_error: Option<String>,
|
||||
/// Server-owned: what the repo's credential says about its own expiry and
|
||||
/// scopes. Written by the credential check, never accepted from clients.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub credential: Option<GitCredentialStatus>,
|
||||
}
|
||||
|
||||
impl GitRepositorySettings {
|
||||
@@ -405,6 +410,44 @@ pub enum AutoPullMode {
|
||||
Polling,
|
||||
}
|
||||
|
||||
/// Host whose credential lifecycle Windmill can manage from the repo URL.
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum GitCredentialProvider {
|
||||
Gitlab,
|
||||
}
|
||||
|
||||
/// What the repo's own credential says about itself, refreshed by asking the
|
||||
/// host. Server-owned: written by the credential check, never accepted from a
|
||||
/// client.
|
||||
///
|
||||
/// Absent means the check has not run or the repo carries no credential we can
|
||||
/// introspect (a GitHub App repo mints tokens per call and has nothing to expire).
|
||||
#[derive(Serialize, Deserialize, Debug, Clone)]
|
||||
pub struct GitCredentialStatus {
|
||||
pub provider: GitCredentialProvider,
|
||||
/// Changes on every rotation, so it identifies the current token, not the
|
||||
/// credential's whole history.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub token_id: Option<i64>,
|
||||
/// `None` is a non-expiring token, which only self-managed GitLab can issue
|
||||
/// (and only for a service account). It means no warning and no rotation.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub expires_at: Option<chrono::NaiveDate>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub scopes: Vec<String>,
|
||||
/// Whether *this workspace* renews the credential. That needs a scope which
|
||||
/// permits it (`api` or `self_rotate`) and a credential this workspace holds:
|
||||
/// a token carried in the repository URL is the operator's to manage, and one
|
||||
/// resolved from an ancestor is the ancestor's, so neither is renewed here.
|
||||
pub rotatable: bool,
|
||||
/// Unix timestamp (seconds) of the last check.
|
||||
pub checked_at: i64,
|
||||
/// Why the last check or rotation failed, cleared by the next success.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
/// Outcome of the most recent auto-pull attempt, surfaced in the UI.
|
||||
#[derive(Serialize, Deserialize, Debug, Clone)]
|
||||
pub struct AutoPullStatus {
|
||||
|
||||
@@ -3629,6 +3629,16 @@ async fn get_git_commit_hash(
|
||||
})?;
|
||||
git_resource.url =
|
||||
resolve_azure_devops_url(&db_with_opt_authed, &w_id, &git_resource.url, false).await?;
|
||||
// A credential is stored under the repository it was issued for, so a
|
||||
// resource repointed elsewhere finds none. Which credential can be attached
|
||||
// is bounded by that; who may use it is bounded here, on the same terms as
|
||||
// the installation credential above.
|
||||
let plain_url = git_resource.url.clone();
|
||||
git_resource.url =
|
||||
windmill_common::git_sync_oss::with_stored_credential(&db, &w_id, git_resource.url).await?;
|
||||
if git_resource.url != plain_url {
|
||||
require_admin(authed.is_admin, &authed.username)?;
|
||||
}
|
||||
|
||||
let identities: Vec<String> = query
|
||||
.git_ssh_identity
|
||||
@@ -4307,6 +4317,10 @@ pub async fn get_git_repo_head_for_autopull(
|
||||
}
|
||||
git_resource.url =
|
||||
resolve_azure_devops_url(&git_sync_system_dba(db), w_id, &git_resource.url, true).await?;
|
||||
// A repo whose credential Windmill holds carries none in its URL, so the
|
||||
// poller has to attach it here or every probe would be unauthenticated.
|
||||
git_resource.url =
|
||||
windmill_common::git_sync_oss::with_stored_credential(db, w_id, git_resource.url).await?;
|
||||
|
||||
if let Some(branch) = git_resource.branch.as_deref().filter(|s| !s.is_empty()) {
|
||||
let branch = branch.to_string();
|
||||
@@ -4413,6 +4427,11 @@ pub async fn get_git_repo_fork_heads_for_autopull(
|
||||
));
|
||||
}
|
||||
git_resource.url = resolve_azure_devops_url(&dba, w_id, &git_resource.url, true).await?;
|
||||
// Same reason as the head probe above: a repository whose credential Windmill
|
||||
// holds carries none in its URL, and listing the fork branches is the half of
|
||||
// polling that would otherwise go out unauthenticated.
|
||||
git_resource.url =
|
||||
windmill_common::git_sync_oss::with_stored_credential(db, w_id, git_resource.url).await?;
|
||||
validate_git_url(&git_resource.url).await?;
|
||||
validate_git_ref(base_branch)?;
|
||||
|
||||
|
||||
@@ -816,8 +816,19 @@ pub async fn handle_receive_completed_job(
|
||||
#[cfg(all(feature = "enterprise", feature = "private"))]
|
||||
#[derive(serde::Deserialize)]
|
||||
struct GitSyncCheck {
|
||||
check_run_id: i64,
|
||||
repo_url: String,
|
||||
/// Absent when the repository's host has no check surface (GitLab): the
|
||||
/// result then reaches the pull request through the managed comment alone.
|
||||
#[serde(default)]
|
||||
check_run_id: Option<i64>,
|
||||
/// Only markers written before the repository URL moved out of job args
|
||||
/// carry one; the resource path on the job is what is used now.
|
||||
#[serde(default)]
|
||||
repo_url: Option<String>,
|
||||
/// Host and path of the repository the check was created on, with no
|
||||
/// credential in it. The resource path is mutable, so this is what proves
|
||||
/// the resource still points where the check lives.
|
||||
#[serde(default)]
|
||||
repo: Option<String>,
|
||||
#[serde(default)]
|
||||
pr_number: Option<i64>,
|
||||
#[serde(default)]
|
||||
@@ -1162,18 +1173,18 @@ async fn maybe_open_git_sync_deploy_pr(
|
||||
};
|
||||
|
||||
// Base = the tracked branch (resource branch, else the repo default). Also
|
||||
// acts as the app-backed gate: PR creation needs the installation token.
|
||||
let base = match windmill_common::git_sync_ee::get_app_repo_head_for_autopull(
|
||||
// acts as the gate: PR creation needs a credential the server itself holds.
|
||||
let base = match windmill_common::git_sync_ee::managed_pr_base_branch(
|
||||
db,
|
||||
workspace_id,
|
||||
&repo_path,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(Some((branch, _))) => branch,
|
||||
Ok(Some(branch)) => branch,
|
||||
Ok(None) => {
|
||||
tracing::warn!(
|
||||
"git sync PR: repo {repo_path} in {workspace_id} has a PR-on-deploy toggle set but is not GitHub-App-backed; skipping (connect the repo through the GitHub App, or use the open-pr-on-commit workflow)"
|
||||
"git sync PR: repo {repo_path} in {workspace_id} has a PR-on-deploy toggle set but the server holds no credential for it; skipping (connect the repo through the GitHub App or a GitLab token, or use the open-pr-on-commit workflow)"
|
||||
);
|
||||
return;
|
||||
}
|
||||
@@ -1373,22 +1384,59 @@ async fn maybe_post_git_sync_check(
|
||||
(None, Some(deploy)) => (true, deploy),
|
||||
(None, None) => return,
|
||||
};
|
||||
let Ok(mut check) = serde_json::from_value::<GitSyncCheck>(marker) else {
|
||||
let Ok(check) = serde_json::from_value::<GitSyncCheck>(marker) else {
|
||||
return;
|
||||
};
|
||||
// Markers carry the literal resource URL (job args are persisted, so a
|
||||
// `$var:`-resolved URL must not land there); interpolate before calling
|
||||
// GitHub.
|
||||
check.repo_url =
|
||||
match windmill_common::variables::get_variable_or_self(check.repo_url, db, workspace_id)
|
||||
.await
|
||||
{
|
||||
Ok(u) => u,
|
||||
Err(e) => {
|
||||
tracing::error!("git sync-check: cannot interpolate repo url: {e:#}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
// Job args are persisted, so the repository URL is not among them: it is
|
||||
// re-resolved here from the resource path the pull job carries. A marker
|
||||
// written before that change still has the URL, and is honoured until the
|
||||
// last such job has drained.
|
||||
// The resource path is mutable, so it is only trusted when the marker also
|
||||
// carries the identity to check it against. A marker written before that
|
||||
// identity existed keeps using the URL it captured at enqueue, which cannot
|
||||
// have been repointed since.
|
||||
let repo_url = match (
|
||||
check.repo.is_some(),
|
||||
row.repo_path.as_deref(),
|
||||
check.repo_url.clone(),
|
||||
) {
|
||||
// The resource path is mutable, so following it is only safe when the
|
||||
// marker also carries the identity to check the result against.
|
||||
(true, Some(path), _) => {
|
||||
windmill_common::git_sync_ee::resolve_repo_url_interpolated(db, workspace_id, path)
|
||||
.await
|
||||
}
|
||||
// A marker written before that identity existed captured the URL itself,
|
||||
// which cannot have been repointed since.
|
||||
(_, _, Some(url)) => {
|
||||
windmill_common::variables::get_variable_or_self(url, db, workspace_id).await
|
||||
}
|
||||
// Neither: nothing here can prove which repository this check belongs to,
|
||||
// and resolving the path anyway is how a preview reaches the wrong one.
|
||||
// Leaving the check unfinished is the safe failure.
|
||||
_ => {
|
||||
tracing::error!(
|
||||
"git sync-check: the marker carries neither a repository identity nor a url; not acting on it"
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let repo_url = match repo_url {
|
||||
Ok(u) => u,
|
||||
Err(e) => {
|
||||
tracing::error!("git sync-check: cannot resolve repo url: {e:#}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
// A resource repointed while the diff was running would otherwise close a
|
||||
// check, or post a preview, on a repository that has nothing to do with it.
|
||||
if check.repo.is_some() && windmill_common::git_sync_ee::repo_identity(&repo_url) != check.repo
|
||||
{
|
||||
tracing::warn!(
|
||||
"git sync-check: the repository moved since the check was created; leaving it alone"
|
||||
);
|
||||
return;
|
||||
}
|
||||
// "In sync" on a PR that visibly changes files reads as a bug when those
|
||||
// files are outside the repo's sync filters — say what the scope is.
|
||||
let scope_note = if !is_deploy && success {
|
||||
@@ -1451,7 +1499,7 @@ async fn maybe_post_git_sync_check(
|
||||
(
|
||||
"failure",
|
||||
"Merge conflicts with the base branch".to_string(),
|
||||
"This PR cannot be merged cleanly, so there is no deploy diff to compute. Resolve the conflicts and push again to re-run this check."
|
||||
"This branch cannot be merged cleanly, so there is no deploy diff to compute. Resolve the conflicts and push again to re-run this check."
|
||||
.to_string(),
|
||||
)
|
||||
} else if pr_check_error.as_deref() == Some("PR_HEAD_REF_UNAVAILABLE") {
|
||||
@@ -1461,7 +1509,7 @@ async fn maybe_post_git_sync_check(
|
||||
(
|
||||
"neutral",
|
||||
"Could not compute the deploy diff".to_string(),
|
||||
"Windmill could not fetch this PR's head or enough history from GitHub to compute its merge with the base. Push again to re-run this check."
|
||||
"Windmill could not fetch this branch's head, or enough history, to compute its merge with the base. Push again to re-run this check."
|
||||
.to_string(),
|
||||
)
|
||||
} else if pr_check_error.is_some() {
|
||||
@@ -1484,20 +1532,20 @@ async fn maybe_post_git_sync_check(
|
||||
"success",
|
||||
"In sync".to_string(),
|
||||
format!(
|
||||
"Merging this PR would make no changes to the workspace.{}",
|
||||
"Merging this branch would make no changes to the workspace.{}",
|
||||
scope_note.as_deref().unwrap_or_default()
|
||||
),
|
||||
),
|
||||
Some((changes, settings_changed)) => {
|
||||
let mut lines = vec![format!(
|
||||
"Merging this PR would apply {} change(s) to the workspace:\n",
|
||||
"Merging this branch would apply {} change(s) to the workspace:\n",
|
||||
changes.len()
|
||||
)];
|
||||
lines.extend(format_change_list(&changes));
|
||||
if settings_changed {
|
||||
lines.push(match check.wmill_yaml_changed {
|
||||
Some(true) => "\nThis PR changes wmill.yaml: pulling also applies the updated workspace settings.".to_string(),
|
||||
Some(false) => "\nIndependent of this PR, the workspace's git-sync settings differ from the repo's wmill.yaml and a pull updates them to match.".to_string(),
|
||||
Some(true) => "\nThis branch changes wmill.yaml: pulling also applies the updated workspace settings.".to_string(),
|
||||
Some(false) => "\nIndependent of this branch, the workspace's git-sync settings differ from the repo's wmill.yaml and a pull updates them to match.".to_string(),
|
||||
None => "\nA pull also updates the workspace's git-sync settings to match the repo's wmill.yaml.".to_string(),
|
||||
});
|
||||
}
|
||||
@@ -1520,19 +1568,21 @@ async fn maybe_post_git_sync_check(
|
||||
Some(url) => format!("{summary}\n\n[See the job in Windmill]({url})"),
|
||||
None => summary.clone(),
|
||||
};
|
||||
if let Err(e) = windmill_common::git_sync_ee::update_check_run(
|
||||
db,
|
||||
workspace_id,
|
||||
&check.repo_url,
|
||||
check.check_run_id,
|
||||
conclusion,
|
||||
&title,
|
||||
&check_summary,
|
||||
job_url.as_deref(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::error!("git sync-check: failed to update check run: {e:#}");
|
||||
if let Some(check_run_id) = check.check_run_id {
|
||||
if let Err(e) = windmill_common::git_sync_ee::update_check_run(
|
||||
db,
|
||||
workspace_id,
|
||||
&repo_url,
|
||||
check_run_id,
|
||||
conclusion,
|
||||
&title,
|
||||
&check_summary,
|
||||
job_url.as_deref(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
tracing::error!("git sync-check: failed to update check run: {e:#}");
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 4 also maintains ONE managed comment on the PR (Cloudflare
|
||||
@@ -1556,7 +1606,7 @@ async fn maybe_post_git_sync_check(
|
||||
if let Err(e) = windmill_common::git_sync_ee::upsert_pr_comment(
|
||||
db,
|
||||
workspace_id,
|
||||
&check.repo_url,
|
||||
&repo_url,
|
||||
pr_number,
|
||||
marker,
|
||||
&body,
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
# Git sync with GitLab
|
||||
|
||||
GitLab has no equivalent of a GitHub App, so there is nothing to install and no
|
||||
consent screen. What Windmill needs instead is one credential you create in
|
||||
GitLab and paste once. With it, a GitLab repository gets the same managed
|
||||
features an app-backed GitHub repository has: instant pull over a webhook, merge
|
||||
requests opened on deploy, and a diff preview posted onto the merge request.
|
||||
|
||||
## The credential
|
||||
|
||||
Create a **project access token** on the project you are syncing (Settings →
|
||||
Access tokens). It is a bot identity that outlives the person who created it,
|
||||
which is what you want for a credential the instance uses unattended, and it
|
||||
reaches exactly the one project.
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| Scope | `api` |
|
||||
| Role | Developer to push deploy branches; **Maintainer** to also manage the webhook and open merge requests |
|
||||
| Expiry | Required. A group service account PAT can be non-expiring on self-managed (see below); an access token cannot |
|
||||
|
||||
**Use a separate token per repository.** A group access token works too and
|
||||
reaches every project in the group, which is convenient for a lot of
|
||||
repositories — but Windmill stores the credential per repository, and renewal
|
||||
rewrites the repository it renewed for. Any other repository holding that same
|
||||
token keeps the revoked one and stops syncing until you paste a new token there.
|
||||
Each stranded repository says so on its card, so it is visible rather than
|
||||
silent, but a token per repository avoids it entirely.
|
||||
|
||||
`api` is a superset: it authorizes Git over HTTPS as well, so no separate
|
||||
`write_repository` is needed to clone and push, and it is also what makes the
|
||||
token renewable. A `write_repository`-only token can still push, but Windmill
|
||||
cannot inspect or renew it and reports that in the workspace's git sync settings.
|
||||
|
||||
### The identity Windmill acts as
|
||||
|
||||
GitLab issues an access token to a bot user it creates for it — `project_<id>_bot_…`
|
||||
for a project token, `group_<id>_bot_…` for a group one — and the bot's display
|
||||
name is **the name you gave the token**. That name is the byline on everything
|
||||
Windmill does: the author of deploy commits, of the merge requests it opens, and
|
||||
of the preview notes it writes. Name it for what it is, `windmill-sync` or
|
||||
similar, rather than something only you will recognise.
|
||||
|
||||
Each token you create adds another bot member to the project or group. Renewal
|
||||
does not — it keeps the same bot — so a repository accumulates one bot, not one
|
||||
per year.
|
||||
|
||||
Renewal goes through GitLab's own self-rotation endpoint. Both kinds of access
|
||||
token are held as their bot user's personal access token, so the token rotates
|
||||
itself and Windmill never needs a credential with rights over the project or
|
||||
group.
|
||||
|
||||
## Connecting a repository
|
||||
|
||||
In the resource form for a `git_repository` resource, use the **GitLab** button:
|
||||
paste the instance URL and the token, pick a project from the list, and Windmill
|
||||
keeps the token for you. The resource itself gets the plain remote URL
|
||||
(`"url": "https://gitlab.com/group/project.git"`), with no credential in it.
|
||||
|
||||
The token is stored encrypted on the workspace, keyed by the repository it was
|
||||
issued for rather than by the resource naming it, the same way a GitHub App
|
||||
installation is held against the account it covers. Nothing reads it back out
|
||||
over the API: the server attaches it when it talks to GitLab, and a sync job
|
||||
receives it only against its own job token. Repointing a resource's `url` asks
|
||||
for a different repository's token and finds none, so the edit carries nothing
|
||||
with it; a repository that genuinely moved needs its token entered again.
|
||||
|
||||
Because the repository is the key, the token is stored the moment you pick the
|
||||
project, before the resource is saved. Renaming the resource later keeps it, and
|
||||
cancelling the edit leaves a stored token that nothing uses until some resource
|
||||
points at that repository again.
|
||||
|
||||
Forks of the workspace read this one copy rather than getting their own, so
|
||||
renewal reaches all of them at once and the token is not duplicated into every
|
||||
descendant workspace.
|
||||
|
||||
Treat workspace admin as equivalent to holding the token. An admin of the
|
||||
workspace, or of any fork below it, can point a repository at a sync script they
|
||||
wrote and have that job request the credential, exactly as they can for a GitHub
|
||||
App installation token. Storing it this way keeps it out of the variables API and
|
||||
out of every fork's own storage; it is not a boundary against the admins of those
|
||||
workspaces.
|
||||
|
||||
A URL with the token written into it, in the resource or in a secret variable
|
||||
the resource points at (`"url": "$var:..."`), is a plain git remote: it syncs on
|
||||
deploy and by polling, and nothing else. Windmill does not know the token is
|
||||
there, so it registers no webhook, opens no merge request, and neither reports
|
||||
nor renews its expiry. Use the **GitLab** button to hand the token to Windmill
|
||||
if you want any of that.
|
||||
|
||||
## Expiry and renewal
|
||||
|
||||
Windmill reads `expires_at` from the token it holds and shows it on the
|
||||
repository in the workspace's git sync settings. Within three weeks of expiry it
|
||||
rotates the token through GitLab's own
|
||||
`POST /personal_access_tokens/self/rotate`, stores the replacement, and verifies
|
||||
it. Only the token can rotate itself, so one without `api` (or `self_rotate`) is
|
||||
a permanent warning rather than something Windmill can fix.
|
||||
|
||||
Only the workspace that holds a credential renews it, so one renewal serves the
|
||||
whole fork chain instead of each fork racing to renew the same token. A fork
|
||||
reads the parent's without holding one, so it never renews; the parent does, and
|
||||
every fork sees the replacement at once.
|
||||
|
||||
Rotation is deliberately never retried. GitLab revokes the old token the instant
|
||||
it issues the replacement, and presenting an already-rotated token to `/rotate`
|
||||
again is treated as reuse: it revokes **the whole token family, including the
|
||||
live replacement**. So a rotation that succeeded at GitLab but failed to persist
|
||||
is surfaced as an error to act on, not retried.
|
||||
|
||||
Non-expiring tokens are possible only for a **group service account PAT** on
|
||||
self-managed, with `require_personal_access_token_expiry` turned off in the
|
||||
instance's application settings. A group access token is always rejected without
|
||||
an `expires_at`.
|
||||
|
||||
## What each managed feature needs
|
||||
|
||||
| Feature | Needs |
|
||||
| --- | --- |
|
||||
| Instant pull | A project hook Windmill creates, so Maintainer; and a Windmill base URL GitLab can reach |
|
||||
| Merge requests on deploy | Developer, plus the `api` scope |
|
||||
| Diff preview on a merge request | The project hook, plus permission to post merge request notes |
|
||||
|
||||
Instant pull falls back to checking the tracked branch about every minute when
|
||||
the hook cannot be created or delivered, so nothing silently stops syncing.
|
||||
|
||||
## Self-managed differences
|
||||
|
||||
**Webhooks to a private network are blocked by default.** GitLab refuses to
|
||||
create a hook pointing at a private or local address until an administrator
|
||||
enables *Allow requests to the local network from webhooks and integrations*
|
||||
(Admin → Settings → Network → Outbound requests,
|
||||
`allow_local_requests_from_web_hooks_and_services`). A Windmill instance on the
|
||||
same private network as GitLab needs this; without it, hook creation fails with a
|
||||
"blocked" error and the repository keeps polling.
|
||||
|
||||
**A relative-URL install is not supported.** GitLab can be served under a path
|
||||
prefix (`https://example.com/gitlab`), and that prefix cannot be told apart from
|
||||
a group of the same name: `example.com/a/b/c.git` is either group `a/b` project
|
||||
`c`, or prefix `a` with group `b` project `c`. Windmill reads it as the nested
|
||||
group, so on a relative-URL install it derives the wrong API base and the managed
|
||||
features stay unavailable. Such a repository still syncs through its token URL,
|
||||
which needs no API base.
|
||||
|
||||
Everything else is identical: Windmill talks to `<your-gitlab>/api/v4` and needs
|
||||
no inbound access of its own beyond the hook deliveries.
|
||||
|
||||
## The deploy preview is a note, not a pipeline status
|
||||
|
||||
On GitHub the preview is a check run: its own object, advisory unless the
|
||||
repository makes it required. GitLab has no equivalent. Its only comparable
|
||||
primitive is a commit status, and posting one has side effects Windmill will not
|
||||
impose on a project:
|
||||
|
||||
- GitLab files the status **as a job inside whatever pipeline already covers that
|
||||
commit**, so a failed Windmill status fails the project's own pipeline, and its
|
||||
reviewers see their test suite as failed.
|
||||
- `allow_failure` is ignored on the commit-status endpoint, so the status cannot
|
||||
be made advisory.
|
||||
- On a commit with no pipeline it creates an `external` pipeline instead, which
|
||||
then gates merging under *Pipelines must succeed*, including while it is still
|
||||
running.
|
||||
|
||||
So on GitLab the preview lives entirely in a **merge request note** that Windmill
|
||||
keeps up to date: it carries the workspace, the status line, the commit, a link
|
||||
to the job, and the full list of changes merging would deploy. A note cannot
|
||||
block a merge or change what the project's own CI reports.
|
||||
|
||||
The note is upserted rather than appended, so a merge request accumulates one
|
||||
Windmill comment however many times it is pushed to.
|
||||
@@ -12,6 +12,7 @@
|
||||
import { Loader2 } from 'lucide-svelte'
|
||||
import { untrack } from 'svelte'
|
||||
import GitHubAppIntegration from './GitHubAppIntegration.svelte'
|
||||
import GitLabIntegration from './GitLabIntegration.svelte'
|
||||
import BedrockCredentialsCheck from './BedrockCredentialsCheck.svelte'
|
||||
import { isCloudHosted } from '$lib/cloud'
|
||||
import ResourceGen from './copilot/ResourceGen.svelte'
|
||||
@@ -28,6 +29,13 @@
|
||||
isValid?: boolean
|
||||
linkedSecretCandidates?: string[] | undefined
|
||||
description?: string | undefined
|
||||
/** Workspace the resource is being saved into, which is not always the one
|
||||
* being navigated. The GitLab picker has to store the credential where the
|
||||
* resource will look for it. */
|
||||
workspace?: string
|
||||
/** Fired once the GitLab picker has stored the picked project's token, so a
|
||||
* form that would otherwise file the URL as a secret knows it holds none. */
|
||||
onCredentialStored?: () => void
|
||||
onSynced?: () => void
|
||||
}
|
||||
|
||||
@@ -39,6 +47,8 @@
|
||||
isValid = $bindable(true),
|
||||
linkedSecretCandidates = undefined,
|
||||
description = $bindable(undefined),
|
||||
workspace = undefined,
|
||||
onCredentialStored,
|
||||
onSynced = undefined
|
||||
}: Props = $props()
|
||||
|
||||
@@ -249,6 +259,19 @@
|
||||
}}
|
||||
onDescriptionUpdate={(newDescription) => (description = newDescription)}
|
||||
/>
|
||||
<!-- Last in a `flex-row-reverse` row, so it lands beside the GitHub App
|
||||
button without splitting it from its own refresh control. -->
|
||||
<GitLabIntegration
|
||||
{resourceType}
|
||||
{args}
|
||||
{workspace}
|
||||
{onCredentialStored}
|
||||
onArgsUpdate={(newArgs) => {
|
||||
args = newArgs
|
||||
rawCode = JSON.stringify(args, null, 2)
|
||||
rawCodeEditor?.setCode(rawCode)
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
{#if resourceType?.includes('bedrock') && !isCloudHosted()}
|
||||
<BedrockCredentialsCheck />
|
||||
|
||||
@@ -1373,6 +1373,15 @@
|
||||
{linkedSecretCandidates}
|
||||
{resourceType}
|
||||
{resourceTypeInfo}
|
||||
workspace={effectiveWorkspace}
|
||||
onCredentialStored={() => {
|
||||
// `forceSecretValue` files a git_repository's `url` in a secret
|
||||
// variable, for the URLs that carry a token in them. The picker's
|
||||
// does not: the token is stored separately, so that variable would
|
||||
// hold nothing secret and add a second place to keep in step with
|
||||
// the resource.
|
||||
linkedSecrets = linkedSecrets.filter((f) => f !== 'url')
|
||||
}}
|
||||
bind:args
|
||||
bind:isValid
|
||||
onSynced={getResourceTypeInfo}
|
||||
|
||||
@@ -180,7 +180,7 @@
|
||||
{#if !githubState.loadingGithubInstallations}
|
||||
<Button
|
||||
variant="default"
|
||||
size="xs"
|
||||
unifiedSize="sm"
|
||||
on:click={handleRefreshInstallations}
|
||||
disabled={!$enterpriseLicense}
|
||||
startIcon={{ icon: RotateCw }}
|
||||
@@ -198,7 +198,7 @@
|
||||
{#snippet trigger()}
|
||||
<Button
|
||||
variant="default"
|
||||
size="xs"
|
||||
unifiedSize="sm"
|
||||
disabled={!$enterpriseLicense || githubState.loadingGithubInstallations}
|
||||
startIcon={{
|
||||
icon: githubState.loadingGithubInstallations ? Loader2 : Github,
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
<script lang="ts">
|
||||
import { workspaceStore, userStore, enterpriseLicense } from '$lib/stores'
|
||||
import { GitSyncService, type GitlabProject } from '$lib/gen'
|
||||
import { sendUserToast } from '$lib/toast'
|
||||
import Popover from './meltComponents/Popover.svelte'
|
||||
import Button from './common/button/Button.svelte'
|
||||
import { Alert } from './common'
|
||||
import TextInput from './text_input/TextInput.svelte'
|
||||
import Select from './select/Select.svelte'
|
||||
import { GitBranch, Gitlab, Loader2 } from 'lucide-svelte'
|
||||
|
||||
interface Props {
|
||||
resourceType: string
|
||||
args?: Record<string, any>
|
||||
/** The workspace the resource is being edited in, which is not always the
|
||||
* one being navigated: the credential has to land where the resource will
|
||||
* look for it. */
|
||||
workspace?: string
|
||||
/** Fired once the picked project's token is stored, so a form that would
|
||||
* otherwise file the URL as a secret knows it no longer holds one. */
|
||||
onCredentialStored?: () => void
|
||||
onArgsUpdate?: (args: Record<string, any>) => void
|
||||
}
|
||||
|
||||
let {
|
||||
resourceType,
|
||||
args = {},
|
||||
workspace = undefined,
|
||||
onCredentialStored,
|
||||
onArgsUpdate
|
||||
}: Props = $props()
|
||||
|
||||
let ws = $derived(workspace ?? $workspaceStore)
|
||||
|
||||
let baseUrl = $state('https://gitlab.com')
|
||||
let token = $state('')
|
||||
let projects: GitlabProject[] = $state([])
|
||||
let selectedProject: string | undefined = $state(undefined)
|
||||
let loading = $state(false)
|
||||
let listError: string | undefined = $state(undefined)
|
||||
let applying = $state(false)
|
||||
let applyError: string | undefined = $state(undefined)
|
||||
/** The token the current listing was made with. Editing the token afterwards
|
||||
* leaves projects on screen that were never checked against it, and applying
|
||||
* would store the new token for a project chosen under the old one. */
|
||||
let listedToken = $state('')
|
||||
|
||||
// Shown alongside the GitHub App button and on the same terms, so the two
|
||||
// read as one choice rather than one option and one absence.
|
||||
let show = $derived(
|
||||
resourceType === 'git_repository' &&
|
||||
!!ws &&
|
||||
($userStore?.is_admin || $userStore?.is_super_admin)
|
||||
)
|
||||
// The project listing is served by an enterprise-only route, so on a build
|
||||
// without it the form's first request would 404. The button still shows,
|
||||
// disabled and labelled, because a missing button reads as "GitLab is not
|
||||
// supported" rather than "this needs a licence".
|
||||
let enabled = $derived(!!$enterpriseLicense)
|
||||
|
||||
let project = $derived(projects.find((p) => p.path_with_namespace === selectedProject))
|
||||
let staleListing = $derived(projects.length > 0 && token !== listedToken)
|
||||
|
||||
async function listProjects() {
|
||||
if (!ws) return
|
||||
loading = true
|
||||
listError = undefined
|
||||
try {
|
||||
// Captured before the await, not after: the field stays editable while the
|
||||
// request is in flight, and reading it on the way back would record a
|
||||
// token this listing was never checked against.
|
||||
const listedWith = token
|
||||
projects = await GitSyncService.listGitlabProjects({
|
||||
workspace: ws,
|
||||
requestBody: { base_url: baseUrl, token: listedWith }
|
||||
})
|
||||
listedToken = listedWith
|
||||
selectedProject = projects[0]?.path_with_namespace
|
||||
if (projects.length === 0) {
|
||||
listError = 'The token can see no project with at least the Developer role'
|
||||
}
|
||||
} catch (err) {
|
||||
listError = err?.body ?? err?.message ?? String(err)
|
||||
projects = []
|
||||
selectedProject = undefined
|
||||
} finally {
|
||||
loading = false
|
||||
}
|
||||
}
|
||||
|
||||
async function apply(close: (_: any) => void) {
|
||||
if (!project || !token || applying || staleListing) return
|
||||
const chosen = project
|
||||
const url = chosen.http_url_to_repo
|
||||
applying = true
|
||||
applyError = undefined
|
||||
try {
|
||||
// Stored against the project it was issued for, the way a GitHub App
|
||||
// installation is stored against the account it covers. Nothing waits on
|
||||
// the resource: its path is not settled while it is being created, and
|
||||
// deferring the write would tie one repository's token to another
|
||||
// repository's save succeeding.
|
||||
await GitSyncService.setGitCredential({
|
||||
workspace: ws!,
|
||||
requestBody: { repo_url: url, token }
|
||||
})
|
||||
} catch (err) {
|
||||
applyError = err?.body ?? err?.message ?? String(err)
|
||||
return
|
||||
} finally {
|
||||
applying = false
|
||||
}
|
||||
onCredentialStored?.()
|
||||
onArgsUpdate?.({
|
||||
...args,
|
||||
url,
|
||||
is_github_app: false,
|
||||
branch: args.branch || chosen.default_branch || undefined
|
||||
})
|
||||
token = ''
|
||||
listedToken = ''
|
||||
projects = []
|
||||
selectedProject = undefined
|
||||
sendUserToast(`${chosen.path_with_namespace} selected and its token stored`)
|
||||
close(null)
|
||||
}
|
||||
</script>
|
||||
|
||||
{#if show}
|
||||
<Popover
|
||||
documentationLink="https://www.windmill.dev/docs/integrations/git_repository"
|
||||
disabled={!enabled}
|
||||
contentClasses="overflow-auto"
|
||||
>
|
||||
{#snippet trigger()}
|
||||
<Button
|
||||
variant="default"
|
||||
unifiedSize="sm"
|
||||
disabled={!enabled}
|
||||
startIcon={{ icon: Gitlab }}
|
||||
nonCaptureEvent
|
||||
>
|
||||
{enabled ? 'GitLab' : 'GitLab (ee only)'}
|
||||
</Button>
|
||||
{/snippet}
|
||||
{#snippet content({ close })}
|
||||
<div class="block text-primary p-4">
|
||||
<div class="flex flex-col gap-4 w-[600px]">
|
||||
<div class="flex flex-col gap-y-1">
|
||||
<div class="text-xs font-semibold text-emphasis">GitLab instance</div>
|
||||
<TextInput bind:value={baseUrl} size="sm" />
|
||||
</div>
|
||||
<div class="flex flex-col gap-y-1">
|
||||
<div class="text-xs font-semibold text-emphasis">Project access token</div>
|
||||
<div class="text-xs font-normal text-secondary">
|
||||
Create it on the project you are syncing, with the <code>api</code> scope and the
|
||||
<code>Maintainer</code> role.
|
||||
</div>
|
||||
<TextInput bind:value={token} size="sm" inputProps={{ type: 'password' }} />
|
||||
<div class="text-2xs font-normal text-hint">
|
||||
Windmill stores it and renews it before it expires. Use one token per project: a group
|
||||
token covers the group, but renewal replaces it for one project at a time.
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<Button
|
||||
variant="default"
|
||||
unifiedSize="sm"
|
||||
disabled={!token || !baseUrl || loading}
|
||||
startIcon={{
|
||||
icon: loading ? Loader2 : GitBranch,
|
||||
classes: loading ? 'animate-spin' : ''
|
||||
}}
|
||||
onclick={listProjects}
|
||||
>
|
||||
List projects
|
||||
</Button>
|
||||
</div>
|
||||
{#if listError}
|
||||
<Alert type="error" title="Could not list projects" size="xs">{listError}</Alert>
|
||||
{/if}
|
||||
{#if staleListing}
|
||||
<div class="text-xs font-normal text-secondary">
|
||||
The token changed. List the projects again to choose one it can reach.
|
||||
</div>
|
||||
{/if}
|
||||
{#if projects.length > 0 && !staleListing}
|
||||
<div class="flex flex-col gap-y-1">
|
||||
<div class="text-xs font-semibold text-emphasis">Project</div>
|
||||
<Select
|
||||
items={projects.map((p) => ({
|
||||
label: p.path_with_namespace,
|
||||
value: p.path_with_namespace
|
||||
}))}
|
||||
bind:value={selectedProject}
|
||||
clearable={false}
|
||||
/>
|
||||
</div>
|
||||
{#if applyError}
|
||||
<Alert type="error" title="Could not store the token" size="xs">{applyError}</Alert>
|
||||
{/if}
|
||||
<div class="flex justify-end">
|
||||
<Button
|
||||
variant="accent"
|
||||
unifiedSize="sm"
|
||||
disabled={!project || !token || applying}
|
||||
startIcon={{
|
||||
icon: applying ? Loader2 : GitBranch,
|
||||
classes: applying ? 'animate-spin' : ''
|
||||
}}
|
||||
onclick={() => apply(close)}
|
||||
>
|
||||
Use this project
|
||||
</Button>
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
{/snippet}
|
||||
</Popover>
|
||||
{/if}
|
||||
@@ -1,12 +1,19 @@
|
||||
<script lang="ts">
|
||||
import type { Schema } from '$lib/common'
|
||||
import { ResourceService, WorkspaceService, type Resource, type ResourceType } from '$lib/gen'
|
||||
import {
|
||||
GitSyncService,
|
||||
ResourceService,
|
||||
WorkspaceService,
|
||||
type Resource,
|
||||
type ResourceType
|
||||
} from '$lib/gen'
|
||||
import { canWrite } from '$lib/utils'
|
||||
import { createEventDispatcher, onDestroy, untrack } from 'svelte'
|
||||
import { userStore, workspaceStore } from '$lib/stores'
|
||||
import { sendUserToast } from '$lib/toast'
|
||||
import { clearJsonSchemaResourceCache } from './schema/jsonSchemaResource.svelte'
|
||||
import ResourceForm from './ResourceForm.svelte'
|
||||
import ReplaceGitCredential from './git_sync/ReplaceGitCredential.svelte'
|
||||
import { invalidateWorkspacePaths } from './PathNameAutocomplete.svelte'
|
||||
import Alert from './common/alert/Alert.svelte'
|
||||
import { resource } from 'runed'
|
||||
@@ -194,6 +201,40 @@
|
||||
let loadingSchema = $derived(resourceTypeResource.loading)
|
||||
|
||||
let current = $derived(selected ? states[selected]?.draft : undefined)
|
||||
// The saved URL, not the draft's: a credential is bound to the repository it
|
||||
// is issued for, so binding one to an edit that has not landed yet would tie
|
||||
// it to something the resource does not point at.
|
||||
let deployedUrl = $derived(
|
||||
selected ? ((fetchedResources[selected]?.value as any)?.url as string | undefined) : undefined
|
||||
)
|
||||
// The deployed path, for the same reason as the deployed URL: the server
|
||||
// answers about what is stored, and an unsaved rename names nothing yet.
|
||||
let deployedPath = $derived(selected ? (initialStates[selected]?.path ?? initialPath) : undefined)
|
||||
// Asked of the server rather than read off the resource: the resource is
|
||||
// client-editable, exported and copied into forks, so nothing written on it
|
||||
// stays true. Re-asked when the saved URL moves, since that is a different
|
||||
// repository. The answer is for admins, who are the only ones who could act
|
||||
// on it, so nobody else asks.
|
||||
const credentialOrigin = resource(
|
||||
[
|
||||
() => selected,
|
||||
() => deployedPath,
|
||||
() => deployedUrl,
|
||||
() => resource_type,
|
||||
() => (selected ? (perWsUser[selected] ?? $userStore)?.is_admin : undefined)
|
||||
],
|
||||
async ([ws, path, _url, type, admin]) =>
|
||||
ws && path && type === 'git_repository' && admin
|
||||
? await GitSyncService.getCredentialOrigin({ workspace: ws, path }).catch(() => undefined)
|
||||
: undefined
|
||||
)
|
||||
// Only a credential this workspace holds is its to replace: a fork borrows
|
||||
// its ancestor's, and storing a replacement here would split it in two.
|
||||
let holdsCredential = $derived(credentialOrigin.current?.origin === 'held')
|
||||
// Only an unsaved *URL* blocks replacing the token, not any unsaved change:
|
||||
// opening the drawer materialises schema defaults (`folder: ""`), so a whole-
|
||||
// resource dirty check would disable it the moment the drawer opens.
|
||||
let urlDirty = $derived(!!deployedUrl && current?.args?.url !== deployedUrl)
|
||||
let resourceToEdit: Resource | undefined = $derived(
|
||||
selected ? fetchedResources[selected] : undefined
|
||||
)
|
||||
@@ -495,6 +536,25 @@
|
||||
</Alert>
|
||||
{/if}
|
||||
|
||||
{#if holdsCredential && selected}
|
||||
<Alert type="info" title="Windmill holds this repository's access token">
|
||||
<div class="flex flex-col items-start gap-2">
|
||||
<div>
|
||||
The URL carries no credential. Windmill stores the token and renews it before it
|
||||
expires.
|
||||
{#if urlDirty}
|
||||
Save your URL change to replace the token.
|
||||
{/if}
|
||||
</div>
|
||||
<ReplaceGitCredential
|
||||
workspace={selected}
|
||||
repoUrl={deployedUrl ?? ''}
|
||||
disabled={urlDirty || !deployedUrl}
|
||||
/>
|
||||
</div>
|
||||
</Alert>
|
||||
{/if}
|
||||
|
||||
{#if current}
|
||||
{#key current}
|
||||
<ResourceForm
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
import GfmMarkdown from './GfmMarkdown.svelte'
|
||||
import TestTriggerConnection from './triggers/TestTriggerConnection.svelte'
|
||||
import GitHubAppIntegration from './GitHubAppIntegration.svelte'
|
||||
import GitLabIntegration from './GitLabIntegration.svelte'
|
||||
import Button from './common/button/Button.svelte'
|
||||
import ResourceGen from './copilot/ResourceGen.svelte'
|
||||
import SyncResourceTypes from './SyncResourceTypes.svelte'
|
||||
@@ -47,6 +48,9 @@
|
||||
/** Workspace the path is validated against and the connection is tested in;
|
||||
* defaults to the nav workspace. */
|
||||
workspace?: string | undefined
|
||||
/** Fired once the GitLab picker has stored the picked project's token, so a
|
||||
* form that would otherwise file the URL as a secret knows it holds none. */
|
||||
onCredentialStored?: () => void
|
||||
}
|
||||
|
||||
let {
|
||||
@@ -68,7 +72,8 @@
|
||||
loadingSchema,
|
||||
resourceToEdit,
|
||||
onLoadResourceType,
|
||||
workspace = undefined
|
||||
workspace = undefined,
|
||||
onCredentialStored
|
||||
}: Props = $props()
|
||||
|
||||
let ws = $derived(workspace ?? $workspaceStore)
|
||||
@@ -248,12 +253,30 @@
|
||||
{description}
|
||||
onArgsUpdate={(newArgs) => {
|
||||
args = newArgs
|
||||
if (viewJsonSchema) {
|
||||
// The raw editor is also what a workspace missing the resource type
|
||||
// gets, and it holds its own copy of the value: without this the
|
||||
// picker fills in a URL nothing on screen ever shows.
|
||||
if (viewJsonSchema || !resourceSchema) {
|
||||
rawCode = JSON.stringify(args, null, 2)
|
||||
}
|
||||
}}
|
||||
onDescriptionUpdate={(newDescription) => (description = newDescription)}
|
||||
/>
|
||||
<GitLabIntegration
|
||||
resourceType={resource_type}
|
||||
{args}
|
||||
workspace={ws}
|
||||
{onCredentialStored}
|
||||
onArgsUpdate={(newArgs) => {
|
||||
args = newArgs
|
||||
// The raw editor is also what a workspace missing the resource type
|
||||
// gets, and it holds its own copy of the value: without this the
|
||||
// picker fills in a URL nothing on screen ever shows.
|
||||
if (viewJsonSchema || !resourceSchema) {
|
||||
rawCode = JSON.stringify(args, null, 2)
|
||||
}
|
||||
}}
|
||||
/>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
import GitSyncModeDisplay from './GitSyncModeDisplay.svelte'
|
||||
import Toggle from '$lib/components/Toggle.svelte'
|
||||
import EEOnly from '$lib/components/EEOnly.svelte'
|
||||
import { ResourceService, VariableService } from '$lib/gen'
|
||||
import { GitSyncService, ResourceService, VariableService } from '$lib/gen'
|
||||
|
||||
let {
|
||||
idx = null,
|
||||
@@ -151,6 +151,91 @@
|
||||
let loadingResourceInfo = $state(false)
|
||||
// Only GitHub App-backed repos can register webhooks; PAT repos poll only.
|
||||
let isGithubApp = $state(false)
|
||||
/** Where the credential Windmill uses for this repository lives, answered by
|
||||
* the server rather than inferred from the resource: `held` when this
|
||||
* workspace stores it, `borrowed` when an ancestor does. A borrowed one is not
|
||||
* this workspace's to renew or replace, which is what keeps a fork from
|
||||
* warning about a token it must not touch. `undefined` until the lookup
|
||||
* lands, and when nothing in the chain holds one. */
|
||||
let credentialOrigin = $state<'held' | 'borrowed' | undefined>(undefined)
|
||||
// Whether Windmill itself holds a credential for the repository, which is
|
||||
// what the managed features (webhooks, pull requests, commit checks) need.
|
||||
// A GitHub App installation qualifies, and so does a token the server keeps.
|
||||
// Not the recorded status: that is keyed by resource path and outlives a
|
||||
// repoint, while the origin follows the repository the URL names now.
|
||||
let hasManagedCredential = $derived(isGithubApp || credentialOrigin !== undefined)
|
||||
|
||||
const MS_PER_DAY = 86_400_000
|
||||
|
||||
/**
|
||||
* Whole days until the repository's own token expires, or undefined when it
|
||||
* never expires and when nothing has checked it yet.
|
||||
*
|
||||
* Counted between calendar dates, not instants: GitLab expires a token on a
|
||||
* date, so measuring from "now" would call a token expiring later today
|
||||
* expired, and one expiring tomorrow today's problem.
|
||||
*/
|
||||
const credentialDaysLeft = $derived.by(() => {
|
||||
const expiresAt = repo.credential?.expires_at
|
||||
if (!expiresAt) return undefined
|
||||
const expiry = new Date(`${expiresAt}T00:00:00Z`).getTime()
|
||||
const now = new Date()
|
||||
const today = Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate())
|
||||
return Math.round((expiry - today) / MS_PER_DAY)
|
||||
})
|
||||
|
||||
/**
|
||||
* Only raised when a person has to act. A token Windmill renews on its own is
|
||||
* reported in the quiet status line instead, so the alert keeps meaning
|
||||
* "this needs you".
|
||||
*/
|
||||
const credentialAlert = $derived.by(() => {
|
||||
const credential = repo.credential
|
||||
// The status describes the repository the resource named when it was
|
||||
// checked; once nothing is held for the one it names now, it is stale.
|
||||
if (!credential || !hasManagedCredential) return undefined
|
||||
if (credential.error) {
|
||||
return {
|
||||
type: 'error' as const,
|
||||
title: 'Repository token needs attention',
|
||||
body: credential.error
|
||||
}
|
||||
}
|
||||
const days = credentialDaysLeft
|
||||
if (days === undefined) return undefined
|
||||
const when =
|
||||
days <= 0 ? 'has expired' : days === 1 ? 'expires tomorrow' : `expires in ${days} days`
|
||||
// Renewed here, or by the workspace above that holds it. Either way this
|
||||
// workspace has nothing to do, and telling a fork to replace a borrowed
|
||||
// token would split the credential in two. Renewal is licensed per
|
||||
// instance, so a fork knows as well as its parent when nothing renews.
|
||||
if ((credential.rotatable || credentialOrigin === 'borrowed') && $enterpriseLicense) {
|
||||
// A token Windmill renews needs no countdown: a renewal that fails records
|
||||
// an error, which is handled above. Reaching the expiry date anyway is the
|
||||
// one state that proves renewal never happened, and it is the only one
|
||||
// worth raising here — picking an earlier threshold would just be guessing
|
||||
// at the server's renewal window from the client.
|
||||
if (days > 0) return undefined
|
||||
return {
|
||||
type: 'error' as const,
|
||||
title: 'Repository token has expired',
|
||||
body:
|
||||
credentialOrigin === 'borrowed'
|
||||
? 'The workspace that holds this token renews it, but it has expired anyway. Replace it there to restore sync.'
|
||||
: 'Windmill renews this token automatically but has not managed to. Check that the instance can reach GitLab, then replace the token to restore sync.'
|
||||
}
|
||||
}
|
||||
if (days > 30) return undefined
|
||||
const where =
|
||||
credentialOrigin === 'borrowed'
|
||||
? 'in the workspace that holds it'
|
||||
: `on the ${repo?.git_repo_resource_path?.replace(/^\$res:/, '') ?? 'repository'} resource`
|
||||
return {
|
||||
type: days <= 7 ? ('error' as const) : days <= 14 ? ('warning' as const) : ('info' as const),
|
||||
title: `Repository token ${when}`,
|
||||
body: `Windmill does not renew this token. Replace it ${where}${days <= 0 ? ' to restore sync.' : ' before it expires.'}`
|
||||
}
|
||||
})
|
||||
|
||||
// Update target branch when repository changes
|
||||
$effect(() => {
|
||||
@@ -187,12 +272,27 @@
|
||||
// Clear stale app state up front so a resource change or a failed
|
||||
// fetch can't leave webhook/fork controls showing for the wrong repo.
|
||||
isGithubApp = false
|
||||
credentialOrigin = undefined
|
||||
try {
|
||||
const resource = await ResourceService.getResource({
|
||||
workspace: $workspaceStore,
|
||||
path: repo.git_repo_resource_path
|
||||
})
|
||||
// The server answers whether it holds this repository's credential;
|
||||
// the resource cannot, being client-editable, exported and copied
|
||||
// into forks. Best-effort: a failure here must not hide the URL
|
||||
// below. Awaited alongside the resource because the defaults below
|
||||
// read the answer.
|
||||
const [origin, resource] = await Promise.all([
|
||||
GitSyncService.getCredentialOrigin({
|
||||
workspace: $workspaceStore,
|
||||
path: repo.git_repo_resource_path
|
||||
}).catch(() => undefined),
|
||||
ResourceService.getResource({
|
||||
workspace: $workspaceStore,
|
||||
path: repo.git_repo_resource_path
|
||||
})
|
||||
])
|
||||
|
||||
if (!abortController.signal.aborted) {
|
||||
credentialOrigin = origin?.origin
|
||||
}
|
||||
if (!abortController.signal.aborted && resource?.value) {
|
||||
// Extract git URL from resource value
|
||||
const value = resource.value as Record<string, any>
|
||||
@@ -205,7 +305,7 @@
|
||||
if (
|
||||
repoMode === 'sync' &&
|
||||
repo.isUnsavedConnection &&
|
||||
isGithubApp &&
|
||||
hasManagedCredential &&
|
||||
!isFork &&
|
||||
$enterpriseLicense &&
|
||||
repo.auto_pull === undefined
|
||||
@@ -219,7 +319,7 @@
|
||||
if (
|
||||
repoMode === 'promotion' &&
|
||||
repo.isUnsavedConnection &&
|
||||
isGithubApp &&
|
||||
hasManagedCredential &&
|
||||
$enterpriseLicense &&
|
||||
repo.promotion_open_prs === undefined
|
||||
) {
|
||||
@@ -292,6 +392,7 @@
|
||||
} else {
|
||||
resourceInfo = null
|
||||
isGithubApp = false
|
||||
credentialOrigin = undefined
|
||||
}
|
||||
}
|
||||
|
||||
@@ -537,6 +638,29 @@
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
{#if credentialAlert}
|
||||
<Alert type={credentialAlert.type} title={credentialAlert.title} size="xs">
|
||||
{credentialAlert.body}
|
||||
</Alert>
|
||||
{:else if hasManagedCredential && repo.credential && !repo.credential.error}
|
||||
<div class="text-xs text-secondary">
|
||||
{#if credentialDaysLeft === undefined}
|
||||
Repository token does not expire.
|
||||
{:else if credentialOrigin === 'borrowed' && $enterpriseLicense}
|
||||
Repository token expires on {repo.credential.expires_at}, and the workspace that holds it
|
||||
renews it.
|
||||
{:else if repo.credential.rotatable && $enterpriseLicense}
|
||||
Repository token expires on {repo.credential.expires_at}, and Windmill renews it
|
||||
automatically.
|
||||
{:else if repo.credential.rotatable || credentialOrigin === 'borrowed'}
|
||||
Repository token expires on {repo.credential.expires_at}. Renewing it automatically
|
||||
requires an enterprise license.
|
||||
{:else}
|
||||
Repository token expires on {repo.credential.expires_at}, and Windmill does not renew it.
|
||||
{/if}
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
{#if !emptyString(repo.git_repo_resource_path)}
|
||||
<!-- Validation and Test Status -->
|
||||
{#if validation?.isDuplicate}
|
||||
@@ -671,7 +795,7 @@
|
||||
{/if}
|
||||
</div>
|
||||
{/if}
|
||||
{#if repoMode === 'promotion' && isGithubApp}
|
||||
{#if repoMode === 'promotion' && hasManagedCredential}
|
||||
<div class="mt-2">
|
||||
<!-- Locked while the dev promotion toggle's save is in flight: this
|
||||
toggle is revealed by that save, and an edit made mid-save would be
|
||||
@@ -700,7 +824,12 @@
|
||||
href="https://www.windmill.dev/docs/integrations/git_repository#github-app"
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline">GitHub App</a
|
||||
> and Windmill opens them automatically.
|
||||
>, or give a GitLab repository a
|
||||
<a
|
||||
href="https://www.windmill.dev/docs/integrations/git_repository"
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline">project access token</a
|
||||
>, and Windmill opens them automatically.
|
||||
</div>
|
||||
{/if}
|
||||
{#if repoMode === 'sync' && isFork}
|
||||
@@ -715,7 +844,7 @@
|
||||
fork is pushed to the fork's own
|
||||
<span class="font-mono">wm-fork/…</span> branch instead of the tracked branch.
|
||||
</div>
|
||||
{#if isGithubApp}
|
||||
{#if hasManagedCredential}
|
||||
<div class="mt-2">
|
||||
<Toggle
|
||||
checked={repo.fork_open_prs ?? false}
|
||||
@@ -740,11 +869,12 @@
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline font-mono">open-pr-on-fork-commit</a
|
||||
>
|
||||
workflow in the repository. Recommended: connect the repository through the
|
||||
workflow in the repository. Recommended: connect the repository through the GitHub
|
||||
App, or give a GitLab repository a
|
||||
<a
|
||||
href="https://www.windmill.dev/docs/integrations/git_repository#github-app"
|
||||
href="https://www.windmill.dev/docs/integrations/git_repository"
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline">GitHub App</a
|
||||
class="text-blue-500 hover:underline">project access token</a
|
||||
> and Windmill opens them automatically.
|
||||
</div>
|
||||
{/if}
|
||||
@@ -806,7 +936,7 @@
|
||||
options={{
|
||||
right: 'Automatically deploy changes from Git',
|
||||
rightTooltip:
|
||||
'Windmill deploys new commits from the tracked branch into this workspace. Repositories connected through the GitHub App sync instantly via webhooks with a polling fallback; token-based repositories are checked about every minute.'
|
||||
'Windmill deploys new commits from the tracked branch into this workspace. Repositories Windmill holds a credential for sync instantly via webhooks with a polling fallback; other token-based repositories are checked about every minute.'
|
||||
}}
|
||||
on:change={(e) => setAutoPullEnabled(e.detail)}
|
||||
>
|
||||
@@ -828,7 +958,7 @@
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
{#if !isGithubApp && !loadingResourceInfo}
|
||||
{#if !hasManagedCredential && !loadingResourceInfo}
|
||||
<div class="mt-2">
|
||||
<Alert type="info" title="Instant pull recommended" size="xs">
|
||||
Pull for this repository checks the tracked branch about every minute; longer
|
||||
@@ -838,25 +968,30 @@
|
||||
href="https://www.windmill.dev/docs/integrations/git_repository#github-app"
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline">GitHub App</a
|
||||
>, or give a GitLab repository a
|
||||
<a
|
||||
href="https://www.windmill.dev/docs/integrations/git_repository"
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline">project access token</a
|
||||
>
|
||||
(which also lets Windmill manage pull requests), or push changes into Windmill
|
||||
(either also lets Windmill manage pull requests), or push changes into Windmill
|
||||
with the
|
||||
<a
|
||||
href="https://www.windmill.dev/docs/advanced/git_sync#github-actions"
|
||||
target="_blank"
|
||||
class="text-blue-500 hover:underline">sync GitHub workflow</a
|
||||
>. If you already push changes with a GitHub Action, keep either the Action or
|
||||
automatic pull, not both, so they don't fight over deploys.
|
||||
>. If you already push changes from CI, keep either that or automatic pull,
|
||||
not both, so they don't fight over deploys.
|
||||
</Alert>
|
||||
</div>
|
||||
{/if}
|
||||
{#if repo.auto_pull?.enabled}
|
||||
{@const viaWebhook = repo.auto_pull?.webhook_id != null}
|
||||
{#if isGithubApp}
|
||||
{#if hasManagedCredential}
|
||||
<div class="mt-2">
|
||||
<Alert type="info" title="Already pulling with a GitHub Action?" size="xs">
|
||||
If you previously set up a GitHub Action to push changes into Windmill,
|
||||
remove it now so the two don't fight over deploys.
|
||||
<Alert type="info" title="Already pulling with a CI job?" size="xs">
|
||||
If you previously set up a CI job to push changes into Windmill, remove it
|
||||
now so the two don't fight over deploys.
|
||||
</Alert>
|
||||
</div>
|
||||
{/if}
|
||||
@@ -882,7 +1017,7 @@
|
||||
: 'Checking the tracked branch about every minute. New commits deploy automatically.'}
|
||||
{/if}
|
||||
</div>
|
||||
{#if isGithubApp && repo.auto_pull?.webhook_error}
|
||||
{#if hasManagedCredential && repo.auto_pull?.webhook_error}
|
||||
<div class="mt-2">
|
||||
<Alert type="warning" title="Falling back to polling" size="xs">
|
||||
{repo.auto_pull.webhook_error}
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
<script lang="ts">
|
||||
import { GitSyncService } from '$lib/gen'
|
||||
import { sendUserToast } from '$lib/toast'
|
||||
import Popover from '../meltComponents/Popover.svelte'
|
||||
import Button from '../common/button/Button.svelte'
|
||||
import { Alert } from '../common'
|
||||
import TextInput from '../text_input/TextInput.svelte'
|
||||
import { KeyRound, Loader2 } from 'lucide-svelte'
|
||||
|
||||
interface Props {
|
||||
workspace: string
|
||||
/** The repository as currently saved, which is the key the token is stored
|
||||
* under. A URL edited but not yet saved would file the token against a
|
||||
* repository the resource does not point at, so the caller disables this
|
||||
* until it is saved. */
|
||||
repoUrl: string
|
||||
disabled?: boolean
|
||||
onReplaced?: () => void
|
||||
}
|
||||
|
||||
let { workspace, repoUrl, disabled = false, onReplaced }: Props = $props()
|
||||
|
||||
let token = $state('')
|
||||
let saving = $state(false)
|
||||
let error: string | undefined = $state(undefined)
|
||||
|
||||
/** The instance and project a repository URL names, for checking the pasted
|
||||
* token against the repository it is meant for. */
|
||||
function repoParts(url: string): { base: string; project: string } | undefined {
|
||||
try {
|
||||
const u = new URL(url)
|
||||
const project = u.pathname.replace(/^\/+/, '').replace(/\.git$/, '')
|
||||
return project ? { base: `${u.protocol}//${u.host}`, project } : undefined
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
async function replace(close: (_: any) => void) {
|
||||
if (!token || saving) return
|
||||
saving = true
|
||||
error = undefined
|
||||
// Pinned before the first await, all three: the field stays editable and the
|
||||
// props follow the drawer's selected workspace and deployed URL, so
|
||||
// re-reading any of them afterwards would store the token against something
|
||||
// the check never validated.
|
||||
const candidate = token
|
||||
const forRepo = repoUrl
|
||||
const inWorkspace = workspace
|
||||
try {
|
||||
// Check the token before storing it. The server binds a credential to its
|
||||
// repository but only refuses it when something tries to use it, so a
|
||||
// wrong token would otherwise be accepted here and surface as a failed
|
||||
// sync later.
|
||||
// A URL that does not parse names no project to check, and would be
|
||||
// stored as the credential's key verbatim: a `$var:` reference here
|
||||
// keys the token to a repository that does not exist.
|
||||
const parts = repoParts(forRepo)
|
||||
if (!parts) {
|
||||
error = 'The resource URL must name the repository directly to replace its token here.'
|
||||
return
|
||||
}
|
||||
const projects = await GitSyncService.listGitlabProjects({
|
||||
workspace: inWorkspace,
|
||||
// Searched by name rather than listed whole: the listing is one
|
||||
// capped page, so a token that reaches more projects than fit
|
||||
// would not show this one and a working token would be refused.
|
||||
requestBody: {
|
||||
base_url: parts.base,
|
||||
token: candidate,
|
||||
search: parts.project.split('/').pop()
|
||||
}
|
||||
})
|
||||
if (!projects.some((p) => p.path_with_namespace === parts.project)) {
|
||||
error = `That token cannot push to ${parts.project}. Check its role and that it belongs to this project.`
|
||||
return
|
||||
}
|
||||
await GitSyncService.setGitCredential({
|
||||
workspace: inWorkspace,
|
||||
requestBody: { repo_url: forRepo, token: candidate }
|
||||
})
|
||||
token = ''
|
||||
sendUserToast('Token replaced')
|
||||
onReplaced?.()
|
||||
close(null)
|
||||
} catch (err) {
|
||||
error = err?.body ?? err?.message ?? String(err)
|
||||
} finally {
|
||||
saving = false
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<Popover contentClasses="overflow-auto" {disabled}>
|
||||
{#snippet trigger()}
|
||||
<Button
|
||||
variant="default"
|
||||
unifiedSize="xs"
|
||||
{disabled}
|
||||
startIcon={{ icon: KeyRound }}
|
||||
nonCaptureEvent
|
||||
>
|
||||
Replace token
|
||||
</Button>
|
||||
{/snippet}
|
||||
{#snippet content({ close })}
|
||||
<div class="block text-primary p-4">
|
||||
<div class="flex flex-col gap-3 w-[420px]">
|
||||
<div class="flex flex-col gap-y-1">
|
||||
<div class="text-xs font-semibold text-emphasis">New access token</div>
|
||||
<div class="text-xs font-normal text-secondary">
|
||||
For the same repository. Windmill stores it in place of the current one and renews it
|
||||
from then on.
|
||||
</div>
|
||||
<TextInput bind:value={token} size="sm" inputProps={{ type: 'password' }} />
|
||||
</div>
|
||||
{#if error}
|
||||
<Alert type="error" title="Could not replace the token" size="xs">{error}</Alert>
|
||||
{/if}
|
||||
<div class="flex justify-end">
|
||||
<Button
|
||||
variant="accent"
|
||||
unifiedSize="sm"
|
||||
disabled={!token || saving}
|
||||
startIcon={{ icon: saving ? Loader2 : KeyRound, classes: saving ? 'animate-spin' : '' }}
|
||||
onclick={() => replace(close)}
|
||||
>
|
||||
Replace
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{/snippet}
|
||||
</Popover>
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"gitSyncTest": "hub/28184/git-repo-test-read-write-windmill",
|
||||
"gitInitRepo": "hub/28930/git-sync-init-repository-windmill",
|
||||
"gitSyncTest": "hub/28950/git-repo-test-read-write-windmill",
|
||||
"gitInitRepo": "hub/28948/git-sync-init-repository-windmill",
|
||||
"slackErrorHandler": "hub/28794/workspace-or-schedule-error-handler-slack",
|
||||
"emailErrorHandler": "hub/19795/workspace-or-error-handler-email",
|
||||
"slackRecoveryHandler": "hub/28791/slack/schedule-recovery-handler-slack",
|
||||
|
||||
Reference in New Issue
Block a user