Files
windmill/frontend
Tristan TRandClaude Opus 5 8bd144ccb1 keep crawlers off the login page (#11057)
* fix(frontend): keep crawlers off the login page

Every page on the hub links to /user/login with itself in `rd`, so a crawler
sees one login URL per hub page — 4,311 of them in Search Console, all
rendering this same form and flagged as duplicates without a canonical. Nothing
about a login page belongs in an index, on any instance.

Mark the page noindex, as public_run already is, and ship a robots.txt that
keeps crawlers out of /user/ and /api/. The frontend is embedded as static
assets with an index.html fallback, which is why /robots.txt answered with the
app shell until now; a real file in static/ is served as itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): let crawlers fetch the login page so the noindex is seen

robots.txt disallowed /user/, which stopped a crawler fetching /user/login at
all — and a page that is never fetched never shows its noindex. The two halves
cancelled: the URLs would have moved from "duplicate" to "blocked" rather than
out of the index.

Drop the disallow, keeping /api/. And since the app is client-rendered, the
meta tag only exists after a render pass; send X-Robots-Tag on /user/* from
serve_path as well, which a crawler sees on the first fetch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 18:42:01 +00:00
..

Windmill frontend

The Windmill frontend written in Svelte 5 + Tailwind CSS

The frontend is under AGPL, see the LICENSE file at the root of this repo