Files
windmill/backend/windmill-object-store
Diego Imbert 386e115daa feat: lazily expand s3 explorer folders one level at a time (#10420)
* feat: wire paged object storage listing module

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* feat: document list_stored_files_paged endpoint in openapi

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* feat: lazily expand s3 explorer folders one level at a time

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: pin ee-repo-ref to the paged listing branch

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: share object_store credential resolution and surface listing errors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sqq2LhmWaGwP11Cf3UqWxe

* Chevron is cool

* page size 5000

* feat: make the load more row full-width, secondary and chevron-led

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sqq2LhmWaGwP11Cf3UqWxe

* fix: render newly loaded flat pages inside already-expanded folders

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: address review findings in the lazy s3 explorer

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: address review nits in the lazy s3 explorer

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref after merging main

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: document ambient credential contract and constrain max_keys schema

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: treat an exhausted page token as exhausted, not as a continuation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref for canonical prefix validation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref for prefix scoping and opaque cursors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: invalidate a folder's in-flight load when deleting from it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: discard a stale folder page after its level is invalidated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref for bounded local listing

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: label folders whose final path segment is empty

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* feat: search files by any part of their path, not just folder prefix

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* feat: search files by path prefix instead of a full-bucket scan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* fix: guard stale search responses and describe prefix search accurately

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref for the search prefix fallback fix

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: regenerate the served openapi specs

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref for the search cursor fallback fix

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqAc8mz6Gu698kBbJJVwcT

* chore: bump ee-repo-ref for the bounded search scan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: surface a failed flat listing instead of spinning forever

The flat branch of loadFiles was awaited without a catch, and loadFlatFiles
clears its loading flags only on the success tail. Every caller reaches it
un-awaited, so a rejected listing left the drawer on "Loading content" with
nothing reported. Routing the filter box through this arm made it reachable
per keystroke rather than once per open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: give back the flat cursor when a page fails to load

"Load more" advanced `page` before requesting it, so a failed page left the
cursor pointing at a `listMarkers` slot that was never filled. The retry sent
no marker at all and silently replayed the first page, and the
`listMarkers.length == page` guard kept it there until the listing was reset.

Only reachable now that a failed page is retryable rather than a permanent
spinner.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: scope the flat cursor rollback to its own listing

The rollback matched on the page number alone, so a page that failed after a
filter or storage change could roll back the *replacement* listing once it had
reached the same number, stranding its cursor. Tie it to the generation the
request was issued under.

The delete replay loop had the mirrored problem: it re-drove `page` by hand and
carried on past a failed page, leaving `page` ahead of `listMarkers` for good.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: skip the delete replay when the fresh listing itself failed

clearAndLoadFiles dropped the result it already computes, so a failed
post-delete listing still ran the replay loop: each page advanced `page` with
an empty `listMarkers`, which never recovers because the marker-length guard
only pushes when the two agree. Every later "Load more" then replayed page one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a superseded lazy load from writing into the search that replaced it

loadFolderPage resolves rather than throwing once its generation is stale, so a
filter change that switches the picker to the flat listing mid-flight left the
lazy branch free to expand a preselected file into the search's results and to
clear the search's loading flags. Guard both on the generation it started under,
as the flat branch already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: check the listing generation throughout the reveal walk

Revealing a preselected key is a chain of round trips, so checking once at entry
left the rest of the walk free to keep loading after a filter change had already
switched the picker to the search — under the replacement generation, so the
per-level guards inside loadFolderPage saw nothing wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: let a late metadata failure clear only its own preview

The handler blanked fileMetadata and filePreview without checking that its
request still owned the pane, so selecting a second file while the first was
still loading meant the first's rejection wiped the second's preview.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: key preview ownership on the request, not the selected key

Comparing the selected key let an older request speak for a newer one when both
targeted the same key, which a storage switch does, and made a request whose
selection had moved to something with no metadata return early with the spinner
still up — the case the handler exists to prevent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: clear the preview when the previewed file is deleted

The lazy branch refetches only the affected level and returns, so it never
reached the reset that the flat refresh gets from clearAndLoadFiles. The pane
renders from fileMetadata rather than from the selection, leaving the deleted
file previewed with working download, move and delete actions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: retire the in-flight preview load when its file is deleted

Clearing the pane was not enough: a metadata response computed before the DELETE
landed still repopulated it, restoring the deleted file's preview and its
download, move and delete actions. Deleting now retires the owning request, and
the success and preview writes honour that the same way the failure path does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: clear the preview loading flag when the delete retires its request

Retiring the in-flight metadata load left nobody to report its outcome, so in
lazy mode the pane sat on "Loading..." instead of falling back to the empty
state. The delete owns the flag once it has retired the request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: drop the regenerated openapi deref artifacts

They are generated files that CI only syntax-validates, never checks against
openapi.yaml, and the committed copies already differ from the spec they derive
from by ~9.7k lines. Regenerating here imported that pre-existing drift into a
feature diff, burying ~800 lines of actual change under ~17k lines of other
changes' staleness. Regenerating them is its own chore.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: state the flat cursor invariant once, where the cursor lives

It was spelled out at four sites, which is what AGENTS.md asks not to do. The
rule now sits on the declaration it constrains and the guards reference it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* nit ui

* fix: add the paged listing to the served openapi json

openapi_json() embeds openapi-deref.json via include_str!, and the Docker build
regenerates only the yaml artifact, so the json is served exactly as committed —
leaving the new operation out of the Scalar API reference.

Spliced in the operation and the two schemas it references rather than
regenerating, which would have re-imported ~7k lines of pre-existing drift
between the committed artifact and the spec it derives from.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref for the filesystem symlink boundary

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 0373b4bfdaf8dd51533552e2e4de63ceb3c18b4d

This commit updates the EE repository reference after PR #697 was merged in windmill-ee-private.

Previous ee-repo-ref: eb1a765bb9b29e0c94a6e4942c304934fa15406e

New ee-repo-ref: 0373b4bfdaf8dd51533552e2e4de63ceb3c18b4d

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-08-04 01:41:06 +02:00
..