mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-26 00:01:37 +00:00
5cc4cd55dfd67908f2e5dd2d56971f53e308a942
56 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
552ad9c859 |
fix: reflect custom tag add/remove in the manage-tags drawer immediately (#10526)
* fix: reflect custom tag add/remove in the manage-tags drawer immediately Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: do not fail the custom_tags write when the cache refresh errors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
318c9f0073 |
feat(git-sync): dedicated base url for GitHub webhook delivery (#10411)
* feat(git-sync): let GitHub webhooks register a dedicated base url Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): validate the webhook base url and apply it on change Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: pin ee ref for the git-sync webhook base url change Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): validate and reconcile the webhook base url on every write path Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): route every declarative settings writer through the same rules Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): let the reconciler own the webhook field write-back Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): make the webhook base url validators agree across UI and server Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): lock the workspace row across git_sync read-modify-writes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: pin the webhook base url validator to its server counterpart Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): retry a failed webhook move on every re-apply of the setting Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): retry pending webhook moves on every declarative re-apply Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): reject non-string webhook base urls and bound the sweep Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): reject credential-bearing webhook base urls Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): keep credentials out of webhook base url validation errors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): redact through the last authority @ when reporting a bad url Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): stop echoing unparsed webhook base urls instead of scrubbing them Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): never echo a submitted webhook base url in validation errors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): keep the submitted scheme out of validation errors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(git-sync): drop the webhook sweep, surface stale receivers in settings Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): refresh the stale webhook list when settings are saved Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): mark registered_url nullable and drop the duplicated field error Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: pin ee ref after dropping the reconcile lock and CAS Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(git-sync): refresh the stale webhook list on category saves too Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to aa05ca8e97fc8265cd724753a80db37f83243254 This commit updates the EE repository reference after PR #695 was merged in windmill-ee-private. Previous ee-repo-ref: 3e6cd9226b68707233ae2434511fe5131dce808b New ee-repo-ref: aa05ca8e97fc8265cd724753a80db37f83243254 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
fbf9f04e10 |
fix: surface the real postgres error when data table migrations fail (#10371)
* fix: surface the real postgres error when data table migrations fail Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review nits on the data table migration error fix Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: name the exact grant a data table migration needs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: quote both identifiers in the data table grant hint Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: add a data table connection and privilege check to workspace settings Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report data table privileges from the capability fields, not the grant list Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read grant targets from the server and drop the public schema guess Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: render the search_path suggestion server-side and pin the granted database Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: key the connection check on request identity, not the data table name Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: declare the data table check schema field nullable and required Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
1b6b2aa859 |
fix(datatable): provision the replication user on managed postgres (#10375)
* fix(datatable): provision the replication user on managed postgres * fix(datatable): serialize replication user provisioning and sync config schema * fix(datatable): keep replication cleanup best-effort and self-heal a null password |
||
|
Ruben Fiszel
|
78e115bee5 |
fix: datatable full schema hangs behind a transaction-pooling postgres proxy (#10352)
* fix: datatable full schema hangs behind a transaction-pooling postgres proxy * test: pause the clock in the pg connection shutdown test |
||
 Diego Imbert
|
68daed8501 |
refactor: custom-instance datatable connection handling (#10271)
Attach custom-instance datatables in the DuckDB executor through a DuckDB secret instead of an inline connection string, and route postgres triggers on custom-instance datatables through a dedicated custom_instance_replication_user role (with its own auto-generated password in global_settings). Normalize custom_instance_user attributes on server boot. Claude-Session: https://claude.ai/code/session_01Tp6NNNinCB8dwWqGaFXDRF Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
07d4b674f1 |
fix: manual resource type sync fetches from hub first, cache as fallback (#10269)
* fix: manual resource type sync fetches from hub first, cache as fallback The superadmin "Synchronize resource types" endpoint (POST /api/settings/sync_cached_resource_types) was cache-first: it read the on-disk hub_rt cache and only fell back to the hub when no cache file existed. Since that cache is refreshed by a daily cron, a newly-published hub type could not be pulled on demand, the button replayed the stale cache and reported "Synced 0", so the type never landed in the admins workspace. The manual endpoint is now hub-first: it fetches the live list and upserts it into admins, falling back to reading the on-disk cache only when the hub is unreachable (airgapped install / network error), logging which path it took. The startup/offline sync in main.rs (SYNC_CACHED_RT + the cache-rt cron) stays cache-based and owns writing the cache, so this endpoint never touches it. Adds an optional `name` query param: when a specific type is requested and is still absent from the hub after syncing, the endpoint returns an explicit not-found instead of a silent "Synced 0". The three not-found frontend call sites (ResourceForm, AppConnectInner, ApiConnectForm) thread the type name through SyncResourceTypes; the global instance-settings button stays name-less. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: run whole-list sync before the targeted not-found check, word 404 by source Address review nits: the optional `name` check ran before the upsert loop, so a `?name=<absent>` request skipped the whole-list refresh; move it after the loop so the sync always happens. Also word the not-found 404 by source, the cache-fallback path (hub unreachable) no longer claims it checked the hub. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
ec6324409d |
fix(s3): support instance-policy credentials in object storage tests (#10238)
* fix(s3): cache ambient aws credentials and surface credential chain errors Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SCczeVBDhDnLXWqxPWw6od * fix(frontend): clarify that object storage connection tests run on the server Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SCczeVBDhDnLXWqxPWw6od * fix: render test connection tooltip in popup and harden cache test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SCczeVBDhDnLXWqxPWw6od * better doc * fix(frontend): correct tooltip wording, tests run from the executing worker Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SCczeVBDhDnLXWqxPWw6od * fix(s3): single-flight ambient credential provider creation * fix(frontend): clarify object storage tests run on the server process * fix(frontend): make instance object storage test tooltip provider-neutral --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszel
|
ff774c46bf |
feat: add per-workspace job-retention override (#10050)
* feat: add per-workspace job-retention override (EE) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 2ba6a2a75b6fc97858b306b2c98ada481e363c10 This commit updates the EE repository reference after PR #658 was merged in windmill-ee-private. Previous ee-repo-ref: e7fb36acd813cd717bcf05f5aafbf81de271d618 New ee-repo-ref: 2ba6a2a75b6fc97858b306b2c98ada481e363c10 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
76a9523009 |
feat: use derived username instead of email for non-member superadmins (#9857)
* feat: use derived username instead of email for non-member superadmins Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address review - drop redundant username cache, guard whoami membership by email Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: use explicit non_member boolean instead of role string for superadmin banner Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve email from password table for non-member superadmin permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin drafts via shared username->email resolver Adds resolve_username_to_email (usr, then super_admin password fallback for both derived-username and email modes) and uses it in get_email_from_permissioned_as and the drafts get/list endpoints, so a non-member superadmin's drafts resolve and no email leaks into the drafts payload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: superadmin-not-in-workspace schedule uses derived username as permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin identity in draft owner-circles, username_to_email, and home filter Applies the password-fallback username resolution to the script/flow/app/draft owner-circle subqueries and the username_to_email endpoint (was an admins-workspace 'username == email' hack), and switches the home items-list user-folder filter to the non_member flag instead of the now-broken username-contains-@ heuristic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: backfill non-member superadmin favorites from email to derived username Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: propagate DB errors in username resolution instead of leaking email (CI review) Addresses cubic-dev-ai P2: get_instance_username_or_fallback_to_email now returns Result and only falls back to the email for a genuine 'no derived username'; a query error propagates so callers fail closed rather than leaking the raw email as the acting username. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: clarify non-member superadmin popover (username used + admin permissions) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep username_to_email endpoint member-only to not disclose non-member superadmin email (CI review) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: forbid disabling automate_username_creation once usernames assigned (CI review) Makes the setting effectively one-way once instance-wide usernames exist, so the global-uniqueness invariant that keeps stored u/<username> identities (schedules/triggers/drafts/superadmin ownership) unambiguous can never be dropped back to workspace-local uniqueness. Re-saving false on an already-disabled instance stays a no-op. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
83ed011e26 |
feat(object-store): make GCS service account key optional for Workload Identity (#9842)
build_gcs_client always called `.with_service_account_key(...)`, so an
absent key (the settings UI stores "no key" as the empty JSON object `{}`)
was handed to the builder and failed to parse instead of falling through
to the object_store crate's InstanceCredentialProvider. Skip the call when
the key is blank so GCS uses the instance's ambient credentials (GKE
Workload Identity / the GCP metadata server).
"Blank" (empty/whitespace/`{}`/`null`) is centralized in a shared
`gcs_service_account_key_is_blank` predicate so the build path and the
non-super-admin connectivity-test SSRF guard (`validate_object_storage_test`)
agree on what counts as "no key" — otherwise a blank key would bypass the
guard yet still trigger the ambient-credential fallback, letting an
untrusted caller probe arbitrary buckets with the server's instance role.
Also clarify the settings UI hint that the key may be left empty for
ambient credentials, and add regression tests for the blank-key build path
and the guard.
Fixes WIN-2110
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
577ceeee86 |
perf(audit): re-anchor S3 audit export on enable + opt-in backfill (#9818)
* [ee] perf(audit): re-anchor S3 audit export on enable + opt-in backfill
The S3/GCS audit-log export's steady-state query filters by `age(xmin)`
(unindexable), so the only scan bound is the timestamp floor. On a fresh
enable the floor was epoch, and on a re-enable the cursor resumed from its
pre-disable position — either way the first run scanned the whole
`audit_partitioned` table. Under a `statement_timeout` (e.g. Aiven) that scan
never completes: the cursor never advances, nothing is exported, and the
repeated full scans saturate the database.
Re-anchor on enable (EE companion, windmill-ee-private#634):
- New trigger migration records a recent timestamp floor instead of the epoch
sentinel and `DO UPDATE`s the cursor to the current snapshot xmin on
re-enable, so the export always resumes from ~now and never rescans history.
Includes a one-time fixup for legacy epoch-sentinel checkpoints on upgrade.
Opt-in historical backfill (new `audit_logs_s3_backfill` module + endpoints):
- Exports a chosen `[from, to)` window on demand, scanning strictly by
`timestamp` (the partition key) in bounded keyset pages — each query is an
index scan capped at one page (verified via EXPLAIN: later partitions
`never executed`, ~11ms/page), so it stays well under any statement timeout
regardless of window size. Writes alongside the steady-state objects under
logs/audit/, without touching the xmin cursor.
- POST /settings/audit_logs_s3_backfill {from,to} (super-admin + Enterprise),
GET /settings/audit_logs_s3_backfill_status.
Also repurposes the status endpoint's `bootstrapping` flag to mean "draining a
backlog" (the cursor is capped and catching up), and updates the setting
description to point operators at the backfill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): heartbeat backfill lease per object; bump EE ref
Address review (cubic): persist progress (refreshing the lease heartbeat) after
every object PUT in the backfill page loop, not only once per page, so the gap
between heartbeats stays well under STALE_HEARTBEAT_SECS even on slow uploads
and another replica can't re-claim mid-page and run a concurrent backfill.
Bumps ee-repo-ref.txt to pull in the EE test-race fix (folding the backlog-drain
regression into the single audit e2e test).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): reject unstable backfill windows; bump EE ref
Address review (P1): the backfill keyset-pages over rows visible at scan time
and declares completion when the scan runs dry, but a row's `timestamp` is its
inserting transaction's `xact_start`. A window whose upper bound is recent or in
the future could silently omit a transaction that started inside `[from, to)`
but commits after the scan passed that timestamp. `try_start` now rejects any
`to` newer than the oldest in-flight `xact_start` (everything strictly older
than the oldest running transaction is committed and stable), using the same
trustworthy stats gating as the exporter's floor (restricted role / 2PC → a
7-day-old cutoff).
Bumps ee-repo-ref.txt for the EE monotonic-checkpoint fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): re-anchor legacy epoch checkpoints instead of synthetic floor
Address review (P1): the legacy-checkpoint fixup stamped last_oldest_inflight_ts
to now()-7d while leaving the old last_xmin in place. On an instance that
enabled export on the old code >7 days ago and got stuck before the first
successful batch, the next run would filter post-enable rows older than 7 days
out via `timestamp >= ts_floor` while still advancing last_xmin over the
interval — silently dropping them (the same floor-vs-cursor loss class fixed
elsewhere in this PR), and contradicting the "nothing committed after enabling
is skipped" guarantee.
A stuck epoch-sentinel checkpoint cannot be safely resumed (its backlog can be
arbitrarily old, so any recent floor prunes rows the cursor then skips, and an
epoch floor reintroduces the full scan). Re-anchor it to the migration's current
snapshot xmin instead — exactly like a fresh enable — so the export resumes
cleanly from ~now and the never-exported pre-upgrade window is recovered via the
opt-in backfill rather than silently dropped. Reword the setting description so
it no longer implies the disabled/legacy window is covered by the cursor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(audit): end-to-end integration tests for the object-store backfill
The backfill previously had only SQL-level/EXPLAIN validation. Add real
integration tests (in-memory object store, sqlx::test) exercising the public
path:
- backfill_exports_window_in_pages: with the page size forced to 2 rows, a
settled 3-day window is exported across multiple keyset pages; asserts every
in-window row lands exactly once, rows outside [from,to) are excluded, a day
that straddles a page boundary yields more than one object, progress counts
match, and a re-run is idempotent (deterministic keys overwritten, no dupes).
- backfill_rejects_unstable_window: a future/live `to` is rejected as unstable,
a window safely in the past is accepted.
Adds a test-only PAGE_ROWS override so multi-page behaviour is exercised with a
handful of rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(audit): note backfill scope is audit_partitioned only
Make explicit that, like the steady-state export, the backfill reads only
audit_partitioned; the pre-partitioning `audit` table is intentionally out of
scope (not a missed case).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): reject backfill windows before the partitioned boundary
Address review (Codex P1): the backfill reads only audit_partitioned, but
pre-partitioning history lives in the legacy `audit` table (still read by audit
list/get via UNION ALL, and retained for the configured period — 365 days by
default on EE). Since the setting text points operators at this API for
"pre-existing history", a window overlapping legacy rows would report completion
while silently omitting them.
Per the decision to not export the legacy table, reject instead of silently
omit: try_start now rejects a `from` earlier than the oldest audit_partitioned
timestamp (every legacy row predates the partition cutover, so a `from` at/after
that boundary can never overlap them). Reworded the setting text to scope the
backfill to the partitioned era. Added a regression test, plus an RAII guard
(cubic P2) so the test-only globals are restored even if an assertion panics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): backfill object keys per-window; require trustworthy settled cutoff
Address review (two P1s):
- Object-key overwrite loss: keys were `dt=<day>/audit_backfill_<min_id>.ndjson`.
A narrower, overlapping backfill can start a day's page at the same first row
(same min_id) but hold fewer rows, and `put` would overwrite a broader run's
object — silently dropping the rows only that object held. Include the
requested window in the key so different ranges write disjoint objects (same
window re-runs stay idempotent; consumers dedupe overlapping rows by id). New
regression test (verified red→green).
- Untrustworthy settled cutoff: when min(xact_start) isn't trustworthy (role
lacks pg_read_all_stats/superuser, or a prepared 2PC txn exists), the old
now()-7d fallback could still let an old transaction commit rows inside an
accepted window after the scan, so a "complete" backfill silently missed them.
Since a backfill asserts completeness, reject in those cases instead of
falling back. (The continuous exporter keeps its 7-day fallback — it only
claims bounded lag.)
Also makes the tests robust under the parallel runner: run_backfill takes the
store as a param, so tests pass a local in-memory store (no global
OBJECT_STORE_SETTINGS race) and serialize on the PAGE_ROWS override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): reject backfill overlapping legacy table; regen deref openapi; trim migration comment
Address review (1 P1 + 2 P2):
- Empty-partition backfill (P1): the min(audit_partitioned) guard no-ops when
audit_partitioned is empty, so an upgraded instance with legacy `audit` rows
but no partitioned rows yet would accept a window and complete with zero rows,
silently omitting the legacy rows. Check the legacy `audit` table directly:
reject any window that overlaps a legacy row (subsumes the boundary check and
covers the empty-partitioned case). Test updated accordingly.
- openapi-deref (P2): regenerate openapi-deref.yaml/json (served via include_str!)
so /openapi.{yaml,json} expose the new backfill endpoints.
- Migration comment (P2): trim the PR-history narration to the durable
constraints (why a recent floor and a monotonic cursor are required), per
AGENTS.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to b821fecccbcba2efed544890576bf2b84321d70d
This commit updates the EE repository reference after PR #634 was merged in windmill-ee-private.
Previous ee-repo-ref: 6b191b77aabcf77658ad4f9031576e0d7b66bf89
New ee-repo-ref: b821fecccbcba2efed544890576bf2b84321d70d
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Diego Imbert
|
170cd79aaf |
fix: allow hyphens in postgresql database name validation (#9782)
* fix: allow hyphens in postgresql database name validation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: cover hyphen acceptance in validate_dbname Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 hugocasa
|
24446e8009 |
fix: allow object storage test for non-super-admins, harden on cloud (#9739)
* fix: allow non-super-admin object storage test, harden SSRF surface on cloud Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: validate effective object storage host to close region/bucket SSRF bypass Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: validate gcs_base_url/token_uri in GCS service account key to close SSRF bypass Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: match url scheme case-insensitively in object storage host validation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
fb44fe7af2 |
fix: require super admin for object storage config test endpoint (#9683)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
a9e5140995 |
feat: warn when custom instance db is shared across workspaces (#9359)
* feat: warn when custom instance db is shared across workspaces * Fix leaking workspace names * sqlx prepare |
||
|
Ruben Fiszel
|
8bf7fd2c92 | feat(queue): stochastic admission + EE availability of workspace fairness algorithm (#9321) | ||
|
Ruben Fiszel
|
e218d60919 |
skip workspaced-route duplicate checks on cloud (#9305)
* fix(settings): skip workspaced-route duplicate checks on cloud The pre-write validation hooks for `app_workspaced_route` and `http_route_workspaced_route` query the DB for cross-workspace duplicates and fail the save when any are found. On cloud both `custom_path_exists` (apps) and `route_path_key_exists` (HTTP triggers) already scope lookups by `workspace_id` regardless of these settings, so duplicates across workspaces are expected and the validation has no runtime meaning. The result was that any cloud super-admin attempting to save instance settings with these toggles set to false received `Duplicate HTTP route paths detected` even though the setting has no effect on cloud routing. Fixes WIN-1983 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(error): render JsonErr as readable text and return 400 `Error::JsonErr` previously rendered through `#[error("Error: {0:#?}")]`, leaking Rust's `Debug` output (`Object { "error": String(...), "details": Array [...] }`) into the HTTP response body, and was bucketed into the catch-all 500 branch in `IntoResponse`. The result was a 500 status with a wall of Rust debug syntax in the toast — confusing and user-hostile. - Bucket `JsonErr` into 400 (Bad Request): every current call site (workspaced-route duplicate checks, OAuth client errors, etc.) is a client/validation issue, not an internal server fault. - Add `format_json_err_message` which surfaces the `error` field as the headline, summarises `details` (with a `- key=value` per entry), and pretty-prints the rest as JSON for unknown shapes. The frontend toast now reads e.g. Duplicate HTTP route paths detected - route_path=a, workspace_id=admins, http_method=post - route_path=a, workspace_id=starter, http_method=post Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(toast): preserve newlines and escape HTML in multi-line errors The toast renders via `{@html processMessage(message)}`, so server-side error bodies that span multiple lines (e.g. the duplicate-route response from the settings endpoint) collapsed into a single line because HTML treats consecutive whitespace (including `\n`) as a single space. When the message contains a newline, escape HTML first (defends against injected markup in server error bodies) and convert `\n` to `<br />` so multi-line errors stay readable in the toast. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup: address CI review feedback - toast.ts: escape HTML unconditionally. The previous gate on `\n` left single-line server error bodies unsafe under {@html}, which cubic flagged as P0. The path regex below only inserts a `<span>` around a `u/...` or `f/...` capture that can't contain HTML metacharacters, so escaping the whole input is the simpler and correct fix. - error.rs: add unit tests pinning the rendered shape of `format_json_err_message` (error+details, error-only, truncation cap, non-object fallback to pretty JSON). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
de2e243313 |
feat(queue): per-workspace fairness cap on the shared cloud worker pool (#9303)
* feat(queue): cloud-only per-workspace fairness cap on the shared worker pool
On `app.windmill.dev` the cluster runs a single default worker group, so a
single workspace flooding the queue can degrade quality of service for
everyone else. This adds an opt-in mechanism that caps any single workspace
at a configurable share of the shared worker pool when it has been
dominating cluster activity for more than a configurable window.
Detection signal counts both currently-running jobs and jobs completed in
the rolling window, so it catches workspaces hogging slots with long jobs
**and** workspaces spamming many tiny jobs (where no individual job's
started_at is old, but throughput share dominates).
Refresh is coordinated cluster-wide via a single UPDATE on
`background_task_state`: the `WHERE updated_at < now() - interval` predicate
combined with row-level locking means only one process per refresh cycle
actually runs the aggregation, regardless of fleet size. Every other
process gets the freshly written value in the same round trip via
`UNION ALL ... LIMIT 1`. Heavy aggregation rate stays at ~0.2-0.5 qps for
the whole cluster.
Pull queries are split: the existing query string and its bind shape stay
bit-identical to today, so the planner keeps using the same indexes when
fairness is off or no workspace is currently capped. A separate
`WORKER_PULL_QUERIES_FAIRNESS` adds `AND workspace_id <> ALL($2::text[])`
and is only materialized while the feature is enabled.
Hard-gated to `CLOUD_HOSTED=true` + BASE_URL host == app.windmill.dev at
three layers: frontend `cloudonly: true`, API setter rejection in
`set_global_setting_internal`, runtime check in `fairness_active`. Settings
are exposed under Jobs in the instance-settings UI; defaults are off so
the change is a no-op for self-hosted.
Two-pass pull guarantees no worker idling: if every queued job belongs to
a capped workspace, the second pass uses the unmodified pull queries.
Cap re-asserts on the next refresh.
Fixes WIN-1982
* fix(queue): address CI review findings on workspace fairness
Six fixes from the four-reviewer cross-check on #9303:
1. **Aggregation evaluation (Codex P1).** The previous `INSERT ... ON CONFLICT
DO UPDATE WHERE updated_at < ...` had the heavy `v2_job_queue ∪
v2_job_completed` aggregation inlined into `VALUES`, which Postgres
evaluates for every contender to build the proposed row — losing the
"one heavy aggregation per cycle cluster-wide" property the design
advertises. Split into three small statements: (a) cheap claim with
constant `VALUES`, (b) winner-only `UPDATE ... SET value = jsonb_build_object('overloaded', <agg>)`
(Postgres only evaluates `SET` per row matching `WHERE`, so losers never
compute the aggregation), (c) read for everyone. Heavy query now truly
runs ~0.2-0.5 qps cluster-wide regardless of fleet size.
2. **Numeric setting wraparound (cubic P1).** `u64 as u32` and downstream
`u32 as i32` could silently flip sign and feed `make_interval(secs => -N)`,
making `now() - interval` a future timestamp and disabling the
completed-jobs half of the activity signal. Clamp `duration_secs` to
[1, 86400] and `min_total_jobs` to [0, u32::MAX] before storing.
3. **`/instance_config` bypass (cubic/Claude/Codex P2).** Bulk config endpoint
sidestepped `set_global_setting_internal`'s gate; a self-hosted superadmin
could persist `workspace_fairness_*` rows via the bulk path. Mirror the
per-key check in `set_instance_config` upsert flow.
4. **DB error coerced to false (Claude P2).** `load_workspace_fairness_enabled`
collapsed `Err(_)` to `false` and unconditionally swapped the atomic — a
transient DB blip during notify-event propagation toggled the feature off
cluster-wide (and triggered a `store_pull_query` rebuild precisely when load
is highest). Now propagates the error so the atomic stays at its prior value.
5. **Refresh failure cooldown (Claude P2).** Storing `0` removed the rate
limit entirely; every subsequent pull spawned a new refresh task. Leave
`LAST_REFRESH_MICROS` at `now_us` (already written by the CAS) so the
natural interval acts as the cooldown.
6. **Visibility + duplication (Pi P2).** Mark `make_pull_query_fairness` as
`pub(crate)`. Move the duplicated `BASE_URL host == app.windmill.dev`
parser into `windmill-common::worker::is_cloud_production_host` and share
it between the API setter and the runtime path.
Verified locally:
- `POST /api/settings/global/workspace_fairness_enabled` → 400 (per-key gate)
- `PUT /api/settings/instance_config` with fairness key → 400 (bulk gate)
- `cargo check --workspace --features=private,enterprise,quickjs` — clean
Refs WIN-1982.
* fix(queue): second round of CI review nits on workspace fairness
Three issues raised by the Codex/Claude re-review of commit
|
||
|
Ruben Fiszel
|
ba6fb7021b |
feat: export audit logs to a dedicated object store folder (#9207)
* feat: export audit logs to dedicated object store folder * fix: gap-free audit export via snapshot-xmin gate and stable object keys * test: add integration test for audit log object store exporter * fix: cursor audit export on snapshot xmin to prevent id-leapfrog loss * fix: protect audit s3 checkpoint from config sync and bound export interval * fix: anchor audit s3 checkpoint at enable time to not skip first-window rows * fix: anchor first audit export at the enable transaction's xid * fix: use epoch timestamp floor on first audit export run to not drop old backlog * fix: anchor audit export at startup for env-var enable path * fix: anchor audit export via enabling-txn snapshot xmin trigger * fix: bound the bootstrap audit export to MAX_XID_INTERVAL per tick * refactor: store audit export cursor in background_task_state, add status endpoint * docs: align store_audit_logs_s3 setting text with the actual enable-boundary contract * [ee] refactor: move audit s3 export core logic to EE, gate on Enterprise license * chore: update ee-repo-ref to ec3cd353245e1cdf6a290528dbd7f2ac2498386c This commit updates the EE repository reference after PR #579 was merged in windmill-ee-private. Previous ee-repo-ref: 4ffc6d5f874e64d7dc4a147b4e73baa6c44867a5 New ee-repo-ref: ec3cd353245e1cdf6a290528dbd7f2ac2498386c Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
9c6cd8c852 |
offline (URL-bound) license keys (#9089)
* [ee] feat(license): offline (URL-bound) license keys Offline keys are a 4-segment variant for air-gapped customers — no phone-home, embedded seat/CU caps, locked to the instance's base_url. Existing 3-segment online keys are unchanged. Companion PRs: - windmill-labs/windmill-ee-private (full design + EE impl) - windmill-labs/windmill-customer-service (issuance + portal) - windmill-labs/windmill-cf-worker-keygen (signing) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] refactor(license): bind offline keys via instance hash; simpler CU enforcement - /settings/license_status now surfaces an `instance_hash` superadmins share with support when requesting an offline key - OfflineMetadata: `hash` replaces `base_url`; OfflineCapStatus reports `current_cu` (last 2min) and drops the grace-period fields - verify_license_key now takes a db so EE can recheck the hash - InstanceSetting.svelte: hash copy-block + simpler status panel - Bump ee-repo-ref Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] chore(license): bump ee-repo-ref Pulls in the current_cu clamp + prod public key restoration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] refactor(license): split instance_hash endpoint; minimal cap UI; restore workers expiry toast - `instance_hash` is no longer part of /settings/license_status responses; it lives at GET /settings/instance_hash (super-admin only) so it isn't re-emitted on every status poll. The UI doesn't show it — admins fetch it explicitly when requesting a key from support. - InstanceSetting offline cap UI is now two compact green/red status lines (Seats X.X/Y and CUs X.X/Y) placed above the action buttons, matching the existing "Latest key renewal" badge style. The block-panel is gone. - "Latest key renewal" line and the "Renew key" button are now hidden when an offline key is loaded (renewal is server-disabled for offline keys). - Restore parseLicenseKey + checkLicenseExpiration toast on /workers (works for both 3- and 4-segment keys). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] chore(license): bump ee-repo-ref Pulls in the plain-SHA256 instance hash + stats_ee revert. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] chore(license): bump ee-repo-ref Picks up the alert wording change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] chore(license): bump ee-repo-ref Picks up the instance_uid cache so the periodic verify_license_key cycle no longer hits global_settings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] refactor(license): rename /settings/license_status → /offline_license_status The endpoint was only used by the offline-license UI; the other fields it returned (license_key_id, license_key_valid, kind, offline metadata) were unused. Rename to clarify scope and flatten the response — it now returns just the OfflineCapStatus (or null when no offline license is loaded). Frontend uses `offlineCapStatus != null` as the "is offline" check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] fix(ci): regenerate sqlx cache for the inline worker_ping query After reverting unused stats_ee helpers (fetch_worker_pings*), the inline `sqlx::query_as!(WorkerPingRecord, ...)` in get_stats_payload lost its cache entry — CI's check_ee_full + cargo_test were failing under SQLX_OFFLINE=true with E0282 type-inference errors. Re-running update_sqlx.sh regenerates the cache file under its current hash and prunes a couple of stale entries. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] fix(license): address cubic-bot review - get_offline_license_status: propagate enforce_offline_caps errors as 500 instead of swallowing into a "no offline license" (Option::None) response - canonical_base_url: rewrite the doc to match the actual fallback behavior (lowercase + trailing-slash strip on URL parse failure); the original cross-service contract is gone since the customer-service no longer canonicalizes (treats the instance hash as opaque) - check_seat_cap_for_new_user: take an email and short-circuit when the email is already in `usr ∪ workspace_invite` so net-zero invite upserts and invite→user transitions aren't spuriously blocked at cap. Mirrors the dedup rule the count itself uses. - Bump ee-repo-ref to pull in the EE-side change Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] chore(license): bump ee-repo-ref Picks up the exact-delta seat-cap check (replaces the simple existence short-circuit). Regenerates the new sqlx cache for the bool_and query. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [ee] fix(license): propagate get_instance_hash errors; bump ee-repo-ref - get_instance_hash: replace `.ok().flatten()` with map_err+? so DB errors during instance_uid lookup surface as 500 instead of silently returning `{"instance_hash": null}` (same pattern get_offline_license_status already uses) - Bump ee-repo-ref to pull in the enforce_offline_caps cached-state preservation Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to c6cd1afe2d9e04809b30751cd1687b28a65e62b1 This commit updates the EE repository reference after PR #566 was merged in windmill-ee-private. Previous ee-repo-ref: a6d91016ae0d43c46604313aecae3aa9c778c8e0 New ee-repo-ref: c6cd1afe2d9e04809b30751cd1687b28a65e62b1 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
392888d113 |
docs: add SAFETY comments to all dynamic SQL call sites (#9009)
* docs: add SAFETY comments to all dynamic SQL call sites Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: address review feedback on SAFETY comments - Fix missed comment for obo_triggers loop in offboarding.rs - Fix variable name in comment (table -> table_name) in offboarding.rs - Fix api-settings comment to reference inline VALID_NAME regex, not validate_dbname() - Add SAFETY comments to batch_execute calls in api-settings - Fix db.rs comment: PG_SCHEMA is env var, not compile-time constant - Add doc comments on RunnableSettingsTraitInternal constants * docs: remove misleading SAFETY comment on static SQL --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
 centdix
|
26a6d1e4ce |
refactor: create windmill-ai crate (part 1 — types, traits, base modules) (#8530)
* refactor: create windmill-ai crate and move base AI types from windmill-common Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: move worker AI types to windmill-ai crate Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: move QueryBuilder trait and StreamEventSink abstraction to windmill-ai Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add base64 dependency to windmill-ai for bedrock PDF support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add windmill-ai refactor plan Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — remove dead bedrock feature, add boxed_sink helper, move plan to docs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
8514347784 |
fix: serve populated jwks at /.well-known/jwks.json for vault (#8865)
* fix: serve populated jwks at /.well-known/jwks.json for vault Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: gate jwks route on private feature and use oidc_oss Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
3f5841f84d |
feat: instance-level ruff config auto-pulled by LSP container (#8803)
* feat: add instance-level ruff config auto-pulled by LSP container Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: move ruff config to new LSP tab in instance settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
3d43d31aba |
fix: refresh custom instance user password if auth failed (#8787)
* Refresh custom instance user pwd if connection failed * No longer need to check on startup * nit: unneeded inner function * fix |
||
|
Ruben Fiszel
|
ec9cec1d02 |
fix: treat empty global setting strings as unset (#8793)
* fix: treat empty global setting strings as unset Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: close protected-setting whitespace gap in diff and preserve empty ws override Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
3d4f4c6c38 |
feat: Fork datatables (#8339)
* export_datatable_schema * Propose to fork the datatable on ws fork * dump datatable * Dockerfile * Fix import_datatable_dump * datatable schema fork works! * Option to copy both schema and data * Datatable fork behavior * nit ui * use psql instead * remove fork_datatable route * feat: add fork_pg_database and export_pg_schema routes with DB Manager UI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: pluralize "schema" to "schemas" in DB Manager export/import UI Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add import mode select (schema only vs schema + data) to DB Manager import Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Select schema or schema+data when important database * fix: prepend $res: prefix to resource paths in DB Manager import/export Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: dynamic import button label based on selected mode Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * nits * feat: add warning alert when schema+data import mode is selected Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * nit hide on cloud hosted * refactor: remove fork_behavior from datatable settings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: split CreateWorkspace into layout wrapper and CreateWorkspaceInner Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: instantiate CreateWorkspaceInner in globalForkModal Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * nit icons * Data table fork UI * feat: pass per-datatable fork behaviors from UI to backend during workspace fork Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix fork overwriting all datatables * UI nits * custom instance db refactor * custom instance db wizard btn for all in dropdown * nit * Delete custom instance database button * Disable forking for resource datatables * Big import buttons when db empty * Revert "Disable forking for resource datatables" This reverts commit |
||
|
Ruben Fiszel
|
09bbc18bb7 |
feat: add AWS Secrets Manager as secret storage backend (Beta) (#8734)
* feat: add AWS KMS as secret backend (EE) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: switch from AWS KMS to AWS Secrets Manager as secret backend Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add AWS Secrets Manager integration tests (requires LocalStack) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: mark AWS Secrets Manager as beta Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove leftover KMS handler functions from api-settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to include AWS Secrets Manager EE impl Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use full commit hash in ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * sqlx --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f703fba1ef |
fix: log cleanup scans S3 orphans and works cross-server (#8729)
* fix: log cleanup scans S3 orphans and works cross-server Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: don't skip service log orphan scan when job retention is disabled Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: time-based heartbeat + flag partial folder sizes on list errors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: move background_task module from common to api-settings Only log_cleanup and storage_usage use it today, both in windmill-api-settings. Keeping it in the consumer crate narrows the blast radius; if workers or indexer later need cross-server lease+progress coordination they can move it back to common then. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
02d0ee9198 | feat: add object storage usage view and manual log cleanup (#8724) | ||
|
Ruben Fiszel
|
dcd615fdc3 |
feat: add Azure Key Vault as secret storage backend (#8704)
* feat: add --main flag to write_latest_ee_ref.sh to point to latest EE main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add Azure Key Vault as secret storage backend (EE) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt to azure-key-vault-support branch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add token auth, insecure TLS for emulator, and integration tests Adds optional `token` field to AzureKeyVaultSettings for direct Bearer auth (bypasses OAuth2), enables self-signed cert acceptance in token mode, and includes 4 integration tests against the Azure KV emulator. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: handle Azure KV soft-delete and emulator quirks - Purge soft-deleted secrets after delete to allow name reuse - Retry set_secret on 409 Conflict (purge stale soft-deleted secret) - Accept self-signed certs when using static token (emulator mode) - Work around emulator version-ordering bug in CRUD test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 47b0d9d5d163efdab1e145ee012bdb2eb1373b78 This commit updates the EE repository reference after PR #511 was merged in windmill-ee-private. Previous ee-repo-ref: d432d78bda151d611d8065162de7c1b7edce92e9 New ee-repo-ref: 47b0d9d5d163efdab1e145ee012bdb2eb1373b78 Automated by sync-ee-ref workflow. * fix: accept token OR client_secret in Azure KV validation, add token UI field - isAzureKvConfigValid() now accepts either client_secret or token - Added token input field to the Azure KV config form for emulator/dev use Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
f0437eba19 |
feat: add endpoint to restart workers in a worker group (#8659)
* feat: add endpoint to restart workers in a worker group Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx query cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing modules field to RawCode in tests and regenerate sqlx cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update sqlx * fix: use require_devops_role for restart worker group endpoint Matches the permission level of the clean cache endpoint (update_config), allowing both superadmin and devops role users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback for restart worker group - Fix OpenAPI description to say "devops role" instead of "superadmin" - Add dispatch('reload') after restart to refresh worker list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: only dispatch reload on successful restart Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
0389d9601c |
chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add route reachability tests for ~80 previously untested endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update new trash routes to axum 0.8 path syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: upgrade route tests to assert 2xx responses with proper data setup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: restore npm_proxy and ai_routes tests using local echo servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate workspace fork test behind enterprise feature flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings from axum 0.8 upgrade - Use cookie value_trimmed() instead of value() for cookie 0.18 compat - Update comments still referencing old :workspace_id syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1 This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private. Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1 Automated by sync-ee-ref workflow. * test: add test for new get_imports endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused import in raw_apps test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
9b3e558d84 |
feat: add instance setting to enforce workspace prefix for HTTP routes (#8528)
* feat: add instance-level setting to enforce workspace prefix for HTTP routes
Add `http_route_workspaced_route` instance setting that forces all HTTP routes
to use workspace prefix (`/api/r/{workspace_id}/{route}`), mirroring the existing
`app_workspaced_route` setting for apps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: bump http trigger version on setting change to invalidate route cache
The route cache is version-based, not TTL-based. Without bumping the
version sequence when the instance setting changes, cached routes would
continue serving with the old prefix behavior until a route is
created/updated/deleted or the server restarts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: immediately refresh HTTP routers on setting change
The route cache polls every 60 seconds, but bumping the version sequence
only makes the next poll pick up changes. Explicitly call refresh_routers
after the setting reload so routes are rebuilt immediately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
34cf0a0324 |
show sync resource types button when resource type is missing (#8514)
* feat: show sync resource types button when resource type is missing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show prominent error message when resource type is not found Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use sync_cached_resource_types endpoint instead of hub_sync script Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fallback to fetching resource types from hub when cache file missing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
centdix
|
db5e03610d |
feat: add instance-level AI settings (#8453)
* feat: add instance-level AI settings with workspace fallback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add AI step to onboarding setup wizard Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: thread workspace prop through resource editor and disable chat offset Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Revert "fix: thread workspace prop through resource editor and disable chat offset" This reverts commit 9fea9cc0c239f6432d1fef1487c45e74ab752e21. * fix: set workspace store and disable chat offset during AI setup step Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: thread workspace and disableChatOffset props through resource editors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: populate workspace and user stores for AI step path component Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: initialize AI clients for test key during onboarding Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract AI config state into InstanceAISettings component Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: move AI config state ownership into AISettings component Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Persist instance AI settings before navigation * Reload effective workspace AI state after save * Scope AI key tests to the rendered workspace * Add post-create AI onboarding for new workspaces * Unify instance AI settings header * Fix instance AI drawer offset on workspace selection * Add instance AI fallback settings behavior * Update sqlx metadata * Update sqlx metadata * Clarify active instance AI in workspace settings * Refresh workspace AI state after instance AI save * Declare instance AI summary in API schema * Normalize empty instance AI config handling * Clean up workspace AI settings UI * Unify AI config provider checks * Split AI settings metadata from effective config * Propagate instance AI cache invalidation across servers * Fix AI settings dirty state tracking * Update sqlx metadata --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Diego Imbert
|
446afb5b36 |
fix: fix datatable setup on RDS (#8450)
* Fix Datatable setup on RDS * nit * unused import * add replication |
||
|
hugocasa
|
8c769aebbf |
improve analytics (#8418)
* [ee] improve analytics: add git sync & AI chat telemetry, HMAC-signed download
- Add ai_chat_usage table to track chat sessions (session_id, provider, model, mode, message_count)
- Add POST /w/{workspace}/workspaces/log_chat endpoint with upsert on session_id
- Frontend fires logAiChat on every sendRequest, using HistoryManager's existing chat ID
- EE stats: add git_sync_usage (sync vs promotion repo count) and ai_chat_usage (30-day aggregates)
- Replace RSA+AES-GCM encrypted telemetry download with plaintext JSON + HMAC-SHA256 signature
- Signature (12 hex chars) included in download filename for verification
- Update instance settings telemetry descriptions for both EE and CE
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make StatsDownload struct pub to fix private-interfaces error
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 878cc2044717e0177228529a50433fe2768e70b5
This commit updates the EE repository reference after PR #464 was merged in windmill-ee-private.
Previous ee-repo-ref: 33eb863b6b881bd54ed69a540e0c65d5fe125024
New ee-repo-ref: 878cc2044717e0177228529a50433fe2768e70b5
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Ruben Fiszel
|
372023e995 |
feat: add ws_base_url instance setting for WebSocket URL override (#8405)
* feat: add ws_base_url instance setting to override WebSocket base URL Allow deployments behind reverse proxies to route WebSocket traffic (LSP, debugger, multiplayer) to a different host/port than the main frontend via a new instance setting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: move ws_base_url to Advanced section with toggle and connectivity test - Move setting from Core to Advanced > WebSocket section - Render as toggle "Custom websocket base url from frontend to multiplayer/lsp/debugger" with conditional URL text field - Add Test connectivity button (always visible) that checks HTTP health and WebSocket ping for all three services (LSP, Multiplayer, Debugger) - Add /ws/ping and /ws/health endpoints to LSP service - Add /ws_mp/health HTTP and __ping__ WS handlers to multiplayer service - Add /ping WS handler to debugger service - Add CORS headers to health endpoints for cross-origin testing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: toggle enabled check and testWs promise resolution - Fix enabled derived to check only for null (not empty string), otherwise the toggle never turns on since toggleEnabled sets '' - Fix testWs onclose handler to resolve(false) so the promise doesn't hang if the server closes without sending a message Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: make connectivity test work with existing services - HTTP test: accept plain text "ok"/"okay" (old services) in addition to JSON {"status": "ok"} (new services), reject HTML (SPA fallback) - WS test: resolve on onopen (connection established) instead of waiting for a specific pong message, so the test works even with services that don't have the new /ping handler yet Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
fe1519f128 |
feat: support minimal telemetry mode (#8243)
* feat: support minimal telemetry mode for EE When EE customers disable telemetry, send a reduced payload with only license-compliance data instead of ignoring the setting. Job usage data is excluded in minimal mode. The telemetry settings UI now shows in EE with context-appropriate descriptions for both CE and EE. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee-repo-ref for telemetry-minimal Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: make telemetry toggle label and description license-aware Show "Minimal telemetry" with EE-specific description on EE, and "Disable telemetry" with CE-specific description on CE. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update commit hash in ee-repo-ref.txt * Update reference hash in ee-repo-ref.txt * chore: update ee-repo-ref to 2f52c015bc6c81391234fa87b27ee1d4cd3a48a3 This commit updates the EE repository reference after PR #440 was merged in windmill-ee-private. Previous ee-repo-ref: 3628ed51426d8d29b3d5c62864ba256b7f9eab17 New ee-repo-ref: 2f52c015bc6c81391234fa87b27ee1d4cd3a48a3 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Diego Imbert
|
53ac43f5ee |
fix: resync custom_instance_user password on startup (#8297)
On backend startup, verify the custom_instance_user can connect to the database with the stored password. If the connection fails, automatically refresh the password by calling refresh_custom_instance_user_pwd_inner(). Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
6a0473c578 | fix: redact secrets in set_global_setting log line (#8270) | ||
|
hugocasa
|
63ebae8829 |
feat: replace hub error toasts with warning alerts and add disable hub setting (#8225)
* feat: replace hub error toasts with warning alerts and add disable hub setting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: guard hub script cache refresh when hub is disabled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
424ca59dfe |
feat: make WINDMILL_DIR configurable via environment variable (#8215)
* fix: auto-heal corrupted python runtime cache on remote workers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "fix: auto-heal corrupted python runtime cache on remote workers"
This reverts commit
|
||
|
Diego Imbert
|
8e7ba9b33d |
feat: Data table as pg resource / trigger (#8088)
* Enable running pg scripts with datatable database input * Postgres triggers for data tables * REPLICATION attribute on custom_instance_user * disable edit for datatables * Update backend/windmill-trigger-postgres/src/replication_message.rs Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> |
||
 Alexander Petric
|
adfd8b4df0 | allow devops user to see workers page (#8023) | ||
|
Ruben Fiszel
|
6bf544f507 |
refactor: extract object store into dedicated crate with filesystem backend (#7996)
* refactor: extract object store code into windmill-object-store crate with filesystem backend Consolidate all object_store-dependent code from windmill-common into a new windmill-object-store crate. Add a filesystem-backed object store implementation using LocalFileSystem for dev/testing without cloud credentials. Includes 30 comprehensive tests covering render_endpoint, lfs_to_object_store_resource, duckdb_connection_settings, error mapping, and filesystem-backed integration tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * all * all * fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
b3eeee4131 |
feat: show all settings in YAML UI and protect from empty overwrites (#7976)
- Show custom_instance_pg_databases, ducklake_settings, ducklake_user_pg_pwd and rsa_keys in frontend YAML editor (remove from excludedKeys) - Redact sensitive values: add ducklake_user_pg_pwd and rsa_keys to sensitiveKeys, add custom_instance_pg_databases.user_pwd to nestedSensitiveFields - Remove rsa_keys from HIDDEN_SETTINGS so it appears in YAML export - Hide automate_username_creation from export (add to HIDDEN_SETTINGS) - Add ducklake_user_pg_pwd and rsa_keys to SENSITIVE_SETTINGS for log redaction - Generalize empty/null protection for all PROTECTED_SETTINGS: operator diff skips empty values when DB has existing data, direct API rejects delete/empty for protected settings Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
2019aecf42 |
fix: improve operator ConfigMap settings handling (#7975)
* feat: improve operator ConfigMap settings handling - Protect jwt_secret and min_keep_alive_version from deletion (add to PROTECTED_SETTINGS) - Expose jwt_secret in config exports (remove from HIDDEN_SETTINGS) - Reject empty/null jwt_secret values with warning - Clamp retention_period_secs to 30 days max on CE builds - Improve apply_settings_diff logging: distinguish Created/Updated/Deleted with from/to values and unchanged count summary - Add sensitive value masking in logs with partial redaction (prefix/suffix) for top-level secrets and nested sub-field masking for oauths, smtp, object_store_cache_config, custom_instance_pg_databases - Sort global_settings keys alphabetically in YAML export - Order worker_configs with "default" and "native" first in YAML export - Add tests for sorted YAML serializer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback - Fix redact_string panic on multi-byte UTF-8 by using chars() instead of byte-length slicing - Protect jwt_secret from deletion via direct API (set_global_setting_internal rejects empty/null with BadRequest) - Add code comment documenting jwt_secret visibility trade-off Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |