mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 16:02:10 +00:00
centdix
9d61e4e59e
* feat: self-host docs search for chat, mcp and cli; remove inkeep
Embed a vendored docs snapshot (llms.txt/llms-full.txt) in the backend and
serve ranking + page rendering from GET /api/docs/{search,page}. The AI chat,
the MCP searchDocs/readDocsPage tools, and 'wmill docs' all consume it, so docs
search works with no runtime egress and is no longer EE-gated. Removes the
inkeep proxy. EE companion deletes inkeep_ee.rs (ee-repo-ref bumped).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: name read_docs_page param `url` instead of `path`
search_docs returns each hit's `Source` URL, so the read tool now takes a
`url` argument to match — the AI/MCP loop reads "search gives a Source URL,
read takes that url" rather than copying a `Source:` URL into a `path` slot.
A bare `/docs/...` path is still accepted and canonicalized before lookup.
Regenerated openapi-deref, the MCP endpoint tools, and the frontend client.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci: add scheduled workflow to refresh the vendored docs snapshot
The backend embeds docs_snapshot/*.gz at build time, so the in-product docs
corpus is otherwise only as fresh as the last manual fetch.sh run. This adds a
weekly (and manually dispatchable) job that re-runs fetch.sh, sanity-checks the
result against truncation/garbage, and opens a PR via the internal app when the
snapshot changed — so a human reviews the docs diff before it rides into the
next release build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: make docs tool-result strings caller-neutral
The search/page endpoints back three differently-named consumers (the AI chat
`read_docs_page` tool, the MCP `readDocsPage` tool, and the `wmill docs` CLI),
so the shared rendered text shouldn't name one of them. Refer to "the docs
page-reading tool" and its `url` argument instead, and add tests pinning the
caller-neutral follow-up guidance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: point ee-repo-ref at inkeep-removal companion rebased on EE main
The companion branch now carries only the inkeep_ee.rs deletion on top of EE
main (was based on the native-job-retry EE line, which polluted the EE PR diff).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(docs): expose docs:read in token catalog; precompute lowercased corpus
Addresses two review nits on the self-hosted docs PR:
- docs:read was enforced (ScopeDomain::Docs) but missing from the token scope
catalog (token.rs ALL_SCOPES), so it couldn't be selected when creating a
standard scoped token in the UI — leaving scope-restricted CLI/MCP docs use
effectively ungrantable. Add a read-only "Documentation" group (no write
surface) and a test asserting it is exposed.
- search ran page.body.to_lowercase() on the whole corpus per query. Lowercase
body/title/description once at parse time (into the OnceLock corpus) and scan
the precomputed copies instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: update ee-repo-ref to 27a4f41b8e5603d6e444efcfc420bd1c44a07eed
This commit updates the EE repository reference after PR #630 was merged in windmill-ee-private.
Previous ee-repo-ref: c7ec3a0c2fa38d4cb5e50bf0265eef4710de4860
New ee-repo-ref: 27a4f41b8e5603d6e444efcfc420bd1c44a07eed
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
53 lines
2.2 KiB
YAML
53 lines
2.2 KiB
YAML
name: Refresh docs snapshot
|
|
|
|
# The backend embeds a vendored docs snapshot (backend/windmill-api/docs_snapshot/*.gz)
|
|
# so in-product docs search works with no runtime egress. This job re-fetches it from
|
|
# windmill.dev on a schedule and opens a PR when it changed, keeping the embedded docs
|
|
# fresh independently of the release cadence (the binary embeds whatever is on the
|
|
# source tree at build time, so a merged refresh rides into the next release build).
|
|
on:
|
|
schedule:
|
|
- cron: "0 6 * * 1" # Mondays 06:00 UTC
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
refresh:
|
|
runs-on: ubicloud
|
|
permissions:
|
|
contents: write
|
|
pull-requests: write
|
|
steps:
|
|
- uses: actions/create-github-app-token@v2
|
|
id: app
|
|
with:
|
|
app-id: ${{ vars.INTERNAL_APP_ID }}
|
|
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
token: ${{ steps.app.outputs.token }}
|
|
- name: Fetch + re-gzip docs snapshot
|
|
run: cd backend/windmill-api/docs_snapshot && ./fetch.sh
|
|
- name: Sanity-check the fetched corpus
|
|
# curl -f in fetch.sh rejects HTTP errors, but not a valid-but-garbage 200
|
|
# (truncated file, error page). Guard against embedding a broken snapshot.
|
|
run: |
|
|
cd backend/windmill-api/docs_snapshot
|
|
test "$(wc -c < llms-full.txt.gz)" -gt 100000
|
|
test "$(wc -c < llms.txt.gz)" -gt 1000
|
|
pages=$(gzip -dc llms-full.txt.gz | grep -c '^Source:' || true)
|
|
echo "pages in snapshot: $pages"
|
|
test "${pages:-0}" -ge 200
|
|
- uses: peter-evans/create-pull-request@v6
|
|
with:
|
|
token: ${{ steps.app.outputs.token }}
|
|
branch: chore/refresh-docs-snapshot
|
|
add-paths: backend/windmill-api/docs_snapshot/*.gz
|
|
commit-message: "chore: refresh vendored docs snapshot"
|
|
title: "chore: refresh vendored docs snapshot"
|
|
body: |
|
|
Automated refresh of the embedded docs snapshot
|
|
(`backend/windmill-api/docs_snapshot/*.gz`) from windmill.dev.
|
|
|
|
Review the diff for unexpected churn (a bad upstream docs deploy would
|
|
show up as a large drop in pages or content) before merging.
|