Fixes silent timeouts and partial-tree rendering when loading large git
repositories into the in-app viewer (Tony Hoang report: 400+ host_vars
files, 32 roles).
Three coordinated fixes:
1. Frontend (GitRepoViewer.svelte): drop the 60s clone timeout. Long-poll
getJobUpdates until the job completes, with a 30 min hard cap and a
user-cancel button. Stream live job logs into the viewer with a link
to the full job page. After success, verify the
.windmill_clone_complete marker before flipping pathExists, so a
partial S3 directory is no longer rendered as a complete tree.
2. Backend (check_s3_folder_exists, EE): new optional marker_file query
param. When set, the handler short-circuits to head() on the marker
object instead of "any object under prefix exists". The frontend now
always passes marker_file=.windmill_clone_complete.
3. Hub script: cloneRepoToS3forGitRepoViewer points at hub/28216, which
uploads files via a bounded-concurrency pool (16 workers), emits
throttled progress logs, and writes .windmill_clone_complete as its
last action. docs/clone_repo_and_upload_to_instance_storage.bun.ts is
the source for that hub publish; docs/git-repo-viewer-hub-script.md
explains the change.
Also drops three unused legacy hubPaths entries
(cloneRepoToS3forGitRepoViewer_0..2) — none were referenced from
anywhere in the codebase.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruben Fiszel