Files
centdix 9d61e4e59e feat: self-host docs search for chat, mcp, cli; drop inkeep (#9772)
* feat: self-host docs search for chat, mcp and cli; remove inkeep

Embed a vendored docs snapshot (llms.txt/llms-full.txt) in the backend and
serve ranking + page rendering from GET /api/docs/{search,page}. The AI chat,
the MCP searchDocs/readDocsPage tools, and 'wmill docs' all consume it, so docs
search works with no runtime egress and is no longer EE-gated. Removes the
inkeep proxy. EE companion deletes inkeep_ee.rs (ee-repo-ref bumped).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: name read_docs_page param `url` instead of `path`

search_docs returns each hit's `Source` URL, so the read tool now takes a
`url` argument to match — the AI/MCP loop reads "search gives a Source URL,
read takes that url" rather than copying a `Source:` URL into a `path` slot.
A bare `/docs/...` path is still accepted and canonicalized before lookup.

Regenerated openapi-deref, the MCP endpoint tools, and the frontend client.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: add scheduled workflow to refresh the vendored docs snapshot

The backend embeds docs_snapshot/*.gz at build time, so the in-product docs
corpus is otherwise only as fresh as the last manual fetch.sh run. This adds a
weekly (and manually dispatchable) job that re-runs fetch.sh, sanity-checks the
result against truncation/garbage, and opens a PR via the internal app when the
snapshot changed — so a human reviews the docs diff before it rides into the
next release build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor: make docs tool-result strings caller-neutral

The search/page endpoints back three differently-named consumers (the AI chat
`read_docs_page` tool, the MCP `readDocsPage` tool, and the `wmill docs` CLI),
so the shared rendered text shouldn't name one of them. Refer to "the docs
page-reading tool" and its `url` argument instead, and add tests pinning the
caller-neutral follow-up guidance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: point ee-repo-ref at inkeep-removal companion rebased on EE main

The companion branch now carries only the inkeep_ee.rs deletion on top of EE
main (was based on the native-job-retry EE line, which polluted the EE PR diff).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(docs): expose docs:read in token catalog; precompute lowercased corpus

Addresses two review nits on the self-hosted docs PR:

- docs:read was enforced (ScopeDomain::Docs) but missing from the token scope
  catalog (token.rs ALL_SCOPES), so it couldn't be selected when creating a
  standard scoped token in the UI — leaving scope-restricted CLI/MCP docs use
  effectively ungrantable. Add a read-only "Documentation" group (no write
  surface) and a test asserting it is exposed.
- search ran page.body.to_lowercase() on the whole corpus per query. Lowercase
  body/title/description once at parse time (into the OnceLock corpus) and scan
  the precomputed copies instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: update ee-repo-ref to 27a4f41b8e5603d6e444efcfc420bd1c44a07eed

This commit updates the EE repository reference after PR #630 was merged in windmill-ee-private.

Previous ee-repo-ref: c7ec3a0c2fa38d4cb5e50bf0265eef4710de4860

New ee-repo-ref: 27a4f41b8e5603d6e444efcfc420bd1c44a07eed

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-06-25 16:32:07 +02:00
..