8 Commits

Author SHA1 Message Date
Ruben Fiszel 3394546657 feat(npm-proxy): keep package files on disk and in the object store (#10638)
* docs: design for where the npm proxy keeps cached registry content

* feat(npm-proxy): keep package files on disk and in the object store

* fix(npm-proxy): degrade when the cache is unwritable, stream and bound it

* fix(npm-proxy): keep the happy path off the heap and isolate pull scratch

* fix(npm-proxy): bound the upload, verify pulled trees, keep oversized manifests

* fix(npm-proxy): protect live scratch, bound uploads by parts, refuse traversals

* fix: let the blocking unpack own the scratch it writes into

* fix: replace a cache directory that is not a package instead of deferring to it

* fix: evict by moving a package off the live path, not by deleting it in place

* fix: leave a package the sweep cannot move rather than deleting it in place

* fix: take one registry snapshot through a cache miss

* fix: stamp a pulled package as used so the sweep does not evict it first
2026-08-11 16:40:53 +00:00
Ruben Fiszel 6967804f5a chore(npm-proxy): count file paths and retained entries separately (#10637) 2026-08-11 12:38:39 +00:00
Ruben Fiszel a8816b896d feat(ata): prefer the npm proxy when the instance configures a registry (#10632)
* feat(ata): prefer the npm proxy when the instance configures a registry

* fix(npm-proxy): cap tarball extraction and stop pinning a failed config probe

* fix(npm-proxy): keep large packages cacheable by using a single shard

* fix(npm-proxy): cache the archive so a large package is served, not refused

* fix(npm-proxy): read archives off the runtime, keeping only what types need

* fix(npm-proxy): charge a retained entry for what it allocates, not its bytes

* fix(npm-proxy): size retention for real packages and read the manifest back

* fix(npm-proxy): charge path bytes and pin the manifest read-back

* fix(npm-proxy): stop retaining past the budget instead of refusing the package
2026-08-11 12:31:04 +00:00
Ruben Fiszel ceacc17014 fix(raw-apps): respect the instance .npmrc in the raw app editor (#10629)
* feat(raw-apps): route in-browser npm installs through the npm proxy

* fix(npm-proxy): follow npm range semantics and cache packuments

* fix(npm-proxy): bound the packument cache by bytes and stream tarballs

* fix(npm-proxy): keep a v-prefixed pin exact and read the tarball once

* chore(raw-apps): bump the ui_builder pin to the npm-proxy installer
2026-08-11 12:30:40 +02:00
Ruben Fiszel 0389d9601c chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add route reachability tests for ~80 previously untested endpoints

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update new trash routes to axum 0.8 path syntax

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to latest EE commit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: upgrade route tests to assert 2xx responses with proper data setup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: restore npm_proxy and ai_routes tests using local echo servers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate workspace fork test behind enterprise feature flag

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review findings from axum 0.8 upgrade

- Use cookie value_trimmed() instead of value() for cookie 0.18 compat
- Update comments still referencing old :workspace_id syntax

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1

This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private.

Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac

New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1

Automated by sync-ee-ref workflow.

* test: add test for new get_imports endpoint

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused import in raw_apps test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 09:55:04 +00:00
Ruben Fiszel 86182ed2e9 fix: validate tarball URL host against registry to prevent SSRF and token exfiltration (#8153)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 22:03:19 +00:00
Ruben Fiszel 9eb15312f6 feat: add .npmrc support for private npm registries (#8039)
* feat: add .npmrc support for private npm registries

Add a new `npmrc` instance setting that accepts full .npmrc file content
for configuring private npm registries. Works with bun (native .npmrc
support since 1.1.18), deno (native .npmrc support in 2.x), and the npm
proxy (parses default registry + auth token from .npmrc).

Legacy `npm_config_registry` and `bunfig_install_scopes` fields are now
hidden when empty, so new users only see the .npmrc field. Also fixes a
pre-existing race condition where gen_bunfig was called after
start_child_process.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:29:36 +00:00
Ruben Fiszel e26e437dd1 refactor: extract 12 leaf crates from windmill-api (#7899)
* feat(backend): extract 12 leaf crates from windmill-api to improve incremental compilation

Extract independent modules from windmill-api (90k LOC monolith) into
separate leaf crates to reduce incremental compilation times. Modules
extracted: assets, configs, debug, flow-conversations, inputs,
npm-proxy, openapi, schedule, settings, workers, agent-workers, and
alerting (from windmill-common).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add windmill-api-settings dep to root crate, make ee_oss public

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add agent_workers integration tests

8 tests covering agent worker lifecycle:
- Simple script execution (bun)
- Script with arguments
- Script with logs (verified in job_logs table)
- Script failure handling
- Complex result (nested objects/arrays)
- Agent token creation via API
- Token creation + Initial/MainLoop ping cycle
- Multiple sequential job execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:31:03 +00:00