hugocasa 5099f405d4 feat: make the Git Repo Viewer work with GitHub App repositories (#10765) ...

* fix: resolve the head commit of GitHub App repos in the git repo viewer

`get_git_commit_hash` ran `git ls-remote` against the raw resource URL.
A GitHub-App-backed repository stores a tokenless URL, so the probe failed
with "could not read Username" and the viewer never got past its first
step. Resolve the head over the GitHub REST API with a server-side
installation token instead, reusing the lookup the auto-pull poller
already uses for app repos. Non-app repositories keep the ls-remote path.

Also picks up the EE-side allowlist fix that lets the clone hub script
request an installation token.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 63c67e2a2db198af26a0334f5be14af7d9987eb1

This commit updates the EE repository reference after PR #732 was merged in windmill-ee-private.

Previous ee-repo-ref: 2a260961fa0a9bb5631c17e2f718cb8efb4f9aa2

New ee-repo-ref: 63c67e2a2db198af26a0334f5be14af7d9987eb1

Automated by sync-ee-ref workflow.

* fix: honour the app-repo head lookup's not-app-backed result

`get_app_repo_head_for_autopull` documents `Ok(None)` as "this repo is not
app-backed, use the ls-remote path", which is what the other two callers do.
Fall through to `ls-remote` on `None` instead of turning it into a 500, and
drop the handler's own `is_github_app` read now that the callee's answer is
honoured.

Also bumps ee-repo-ref to pick up route-safe ref handling in that lookup.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: serve GitHub App repositories as an archive instead of a token

The viewer's clone script asked the server for an installation token and put
it in the clone URL. That token is installation-wide and carries the App's
full permissions, so minting one requires a workspace admin, and the viewer
was therefore admin-only for app-backed repositories.

The server now streams a tarball of the commit instead, authorized by read
access to the git_repository resource, so no GitHub credential reaches the
job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: run delegate_to_git_repo playbooks from GitHub App repositories

An Ansible job's runnable_path is the user's own script, which no entry in
the git-sync script allowlist can match, so `delegate_to_git_repo` could
never obtain a token for an app-backed repo. It also gave up entirely on
agent workers, whose connection has no database to mint one from.

A playbook run only reads a working tree: the clone is followed by one
rev-parse for a log line, and nothing after that touches git. So take the
same archive route the viewer uses, extracting the commit's tarball into the
job's repository directory. No GitHub credential reaches the worker, and
agent workers work because the route is HTTP.

Archive entries are joined onto the target by hand so a crafted archive
cannot write outside the job directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: drop the now-immutable secret_url binding

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: point the repo viewer at the archive-based clone script

hub/28905 reads app-backed repositories through the server's archive route
instead of minting an installation token, which the backend in this release
no longer grants it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: stream repository archives to disk rather than into memory

The archive download went through `AuthedClient::get`, whose client caps a
request at 20 seconds and whose response was then buffered whole. A
repository is arbitrarily large, so that cut off slow downloads and put every
job on the worker at risk of running the process out of memory.

Add `get_streaming`, the read counterpart to the streaming upload path, and
write the response out chunk by chunk.

Extraction now creates each entry's parent directory: a tar carries directory
entries only by convention, and the traversal guard now has tests, one of
which caught the missing parent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: require admin to read an app-backed repository

A `git_repository` resource names the repository rather than holding a
credential for it, so read access to one authorizes nothing: anyone who can
write a resource path can point one at any repository the GitHub App
installation reaches, then read their own resource. The head lookup now
requires admin for app-backed repos, matching the archive route and the
repository picker, which already limits itself to workspaces where the
caller is an admin.

Repos that aren't app-backed are untouched and stay open to any reader.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: describe the repo viewer's hub script as it stands

The file read as a patch waiting to be applied, against a hub version two
releases stale. Describe what the published script does, including the
archive route app-backed repositories now take.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: run the archive fetch under the job poller, off the job directory

Three defects in the delegate path's fetch:

The download and extraction ran outside the job poller that the git clone
paths go through, so a cancelled or timed-out run kept streaming and
extracting an arbitrarily large repository while holding the worker. There is
no wall-clock bound on the download itself, by design, which is exactly why
it needs the poller.

The archive was written to a fixed name inside the job directory, where
`create_file_resources` has already laid down the run's own files at paths
the playbook chooses. A run naming a file `repo_archive.tar.gz` had it
truncated and then deleted. It goes to a per-job temp path now.

Link entries were unpacked with their target unchecked. `Entry::unpack`
writes the link verbatim, so a link out of the tree plus a later entry
descending through it writes wherever it points. Targets now face the same
containment check as entry paths.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: keep repo symlinks, refuse only writes that go through them

The link check rejected any target containing `..`, which is ordinary in a
repository — `docs/x -> ../README.md` resolves inside the tree, and a git
checkout keeps it. Rejecting it failed the whole extraction for repositories
the clone path handles, and app-backed repos have no clone path to fall back
to.

Targets are preserved as git preserves them. What would let one escape is a
later entry written at or underneath the link, so that is what is refused.

Extraction also polls an abort flag now: a `spawn_blocking` task outlives the
join handle its caller drops, so a cancelled job left it unpacking in the
background.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: refuse hard links in a repository archive

Leaving link targets verbatim is right for symlinks — git checks them out
that way, and an escape needs a second entry descending through the link,
which is refused. A hard link is not like that: unpacking one creates it
against a target resolved there and then, so an escaping target is useful on
its own.

No git tree can express a hard link, so an archive carrying one did not come
from a repository. Refuse it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: update ee-repo-ref to 21f79bbbd39ae89665d1a89738630978616aa309

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: update ee-repo-ref to 37695a769b25d16b34107eedc1076793a8b388c8

This commit updates the EE repository reference after PR #737 was merged in windmill-ee-private.

Previous ee-repo-ref: 21f79bbbd39ae89665d1a89738630978616aa309

New ee-repo-ref: 37695a769b25d16b34107eedc1076793a8b388c8

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

2026-08-20 22:05:24 +02:00

1008 KiB

Raw Blame History