Compare commits

..
Author SHA1 Message Date
Ruben Fiszelandrubenfiszel 01b38320ea chore(main): release 1.815.0 (#11212)
* chore(main): release 1.815.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-18 15:28:25 +02:00
hugocasaandClaude Opus 5 37e493ae66 feat: add an instance setting to refuse a token in MCP URLs (#11162)
* feat: add an instance setting to refuse a token in MCP URLs

MCP clients are commonly configured with the token in the URL
(`/api/mcp/w/{workspace}/mcp?token=...`). A URL-borne credential ends up in
browser history, proxy logs and referrers, so an instance can now turn that
channel off with the `mcp_disable_token_query_param` global setting and leave
the Authorization header as the only way in, which sends MCP clients through
the OAuth flow the endpoints already advertise.

The rejection is a middleware on both the workspaced and the gateway MCP
mounts, layered outside everything that reads a token and inside the
WWW-Authenticate layer, so the 401 carries the resource pointer a client needs
to start OAuth discovery.

Off by default. With it on, the token drawer and the home connect drawer stop
offering to mint a token for an MCP URL and hand over the bare URL instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: read the MCP URL policy when a URL is asked for, and drop the all-workspaces option

Two review findings on the token drawer:

The policy was read once per page load and cached for the browser session, so a
superadmin turning the setting on left every open tab handing out `?token=` URLs
the server now refuses. Both entry points now read it when the user actually asks
for an MCP URL: when MCP mode is entered, and when the connect drawer opens.

The workspace picker offered "All workspaces / Multi-workspace", but the gateway's
consent screen binds the token it issues to the one workspace picked there, so
OAuth has no multi-workspace grant to hand out. That entry is now token-only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: don't guess the MCP URL policy, and say where the switch lands on restart

Review findings:

The comment on the settings load claimed `MODE=mcp` as the target deployment, but
that mode joins no monitor loop, so the startup pass is its only read and a change
lands on restart. That is true of every global setting there, `base_url` included;
the comment now says so, and the setting description tells an operator running
dedicated MCP servers what to expect.

A failed settings probe resolved to "tokens allowed", so with the switch on the
drawer would mint a non-expiring token and hand over a URL the server refuses for
as long as it exists. The probe now propagates its error and the panel reports it
with a retry, creating nothing until the answer is known.

The test passed a valid token, so it could not tell a rejection before
authentication from one after it. It now also sends a token that was never valid
and asserts the middleware's own message, which fails if the layer moves inward.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: withhold the MCP URL until a workspace is picked

With no persisted workspace the store starts undefined, so opening the drawer from
/user/workspaces before choosing one rendered a copyable
`/api/mcp/w/undefined/mcp`. It reads like a real URL and a client pointed at it
would never connect. The panel now asks for a workspace instead, matching the guard
the token branch already has on its generate button.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: drop the coverage-status note from the MCP switch test

It documented what the test does not reach rather than a constraint the next
reader could break; that belongs in the PR, not the module doc. The layer-order
rationale, which is what a future edit would break, stays.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: fall back to the bare MCP URL instead of alerting on a failed read

When the setting read fails, show the bare URL rather than an error with a retry.
It works whichever way the setting is, so no alert is needed, and it still never
mints a token for a URL the server may refuse. The connect drawer's wording falls
back the same way so the blurb matches the panel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: move the MCP URL token setting to Core

It sat in the Auth/OAuth/SAML list, which the settings sidebar shows under SSO,
suggesting a dependency on SSO that does not exist: MCP OAuth has Windmill act as
the authorization server, and any login method, password included, completes it.
It is an instance-wide credential policy, so it now lives with the other ones in
Core, kept out of quick setup like its neighbours.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 15:24:00 +02:00
f0d66a42eb fix: re-encrypt git sync secrets on workspace key rotation (#11218)
* fix: re-encrypt git sync credentials and webhook secrets on workspace key rotation

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: point ee-repo-ref at the git sync key rotation companion

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: update ee-repo-ref to bc3ef08c8e4233508c023e6ee847a3cd0b8be43b

This commit updates the EE repository reference after PR #814 was merged in windmill-ee-private.

Previous ee-repo-ref: 8121eac421c5d026f36e2edec140f3c79b23d2cd

New ee-repo-ref: bc3ef08c8e4233508c023e6ee847a3cd0b8be43b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-18 15:21:07 +02:00
hugocasaandClaude Opus 5 9d335de87a feat: add a workspace toggle that adds its admins and developers to new forks (#11215)
* feat: add a workspace toggle that adds its admins and developers to new forks

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: add members copied into a fork as manual members, not instance-group ones

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: keep the fork members copy comment at the query and drop the raw spacer

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 15:15:35 +02:00
hugocasaandClaude Opus 5 53afecd458 fix: register the job token with the sensitive log masking system (#10943)
* fix(worker): register the job token with the log masking system

The masking system covered secrets fetched through `get_value_internal` and
`$encrypted:` args, but not the job's own token, so a script that echoed
`$WM_TOKEN` wrote it verbatim into logs that are persisted to the database
and, when configured, to object storage.

`run_worker` now registers the token for the job it just pulled, alongside
the existing `register_running_job` call, so it is redacted like any other
registered secret.

That makes every job carry at least one registered value, where before the
per-batch mask snapshot was skipped entirely for the majority of jobs that
touched no secret. Cache the compiled Aho-Corasick automaton per job and
invalidate it when a new secret is registered, so a chatty job no longer
rebuilds it once per log batch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(nativets): mask secrets in the in-process log path

NativeTS hands `console.log` output to a task that drains a channel into
`append_logs`, so it never reaches the masking in `handle_child::write_lines`
and a script logging `$WM_TOKEN` persisted the raw JWT.

That drain can still be flushing after the job is unregistered, so a plain
per-line `snapshot` would leave the tail unmasked. `JobMasker` keeps the last
masks it saw for exactly that window, and refreshes while the job is alive so
secrets fetched mid-run are covered too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(nativets): seed the job masker at construction

A `JobMasker` that only looked up its masks on the first `mask` call had the
same hole at the head of the log that its retention closes at the tail: if the
drain task's first productive poll landed after the job was unregistered, the
registry was already gone and every line was written raw.

`new` now takes the snapshot, and its callers construct it from the job's own
execution while the job is still registered.

Also cover the nativets sink with an integration test, gated on `deno_core` the
way the CI test build is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(nativets): mask on the producing side of the log channel

Masking as the drain task wrote to `append_logs` left two holes, because that
task is detached and outlives the job: a secret registered mid-run could still
be queued when the job was unregistered and would then be written raw, and the
`windmill:job_log` tracing emission that EE forwards job logs on never went
through the mask at all.

Mask where the line is produced instead. That loop is joined before the job
completes, so the job's secrets are always still registered, and one call now
covers both the tracing mirror and the channel. The result stream keeps reading
the raw text, the way `handle_child` keeps its raw `line` for results.

`JobMasker` is no longer load-bearing for the post-unregistration window, so it
is documented for what it now does: keep the security notice to once per set of
secrets for a sink that masks line by line.

Also drop the nativets test's tag override — `DEFAULT_TAGS` does advertise
`nativets`, so the comment justifying it was wrong — and pin the automaton
cache invalidation, whose failure mode is an unmasked secret.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(worker): hold the log-masking lifecycle at the job boundary

Registering the job around the poller's call left every other way of running a
job uncovered: the interactive worker shell and inline AI agent tools both call
`handle_queued_job` directly, and a script logging `$WM_TOKEN` from either
persisted the live credential. Register from inside `handle_queued_job` instead,
under a drop guard, so each path is covered by construction rather than by
remembering to add a call. Nothing is lost by unregistering earlier: the writes
that follow go through `append_logs`, which never consulted the registry.

In nativets, decide the stream/log routing before masking. `MaskSnapshot`'s
notice is one-shot, so a secret-bearing `WM_STREAM:` chunk used to spend it on
text that is then discarded, leaving later redactions in `job_logs` unexplained.

Restore the masker's post-unregistration test: the memory-limit path never joins
the producing loop, so that fallback is still load-bearing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* docs: correct the nativets masking comments

The producer loop is not joined on the memory-limit path, so it does not
"always" run while the job is registered — say normally, which is what
`JobMasker`'s fallback is there for. Name the reason a stream chunk stays raw
everywhere it goes, including the tracing mirror: it is result data that no log
sink persists, so masking it would be masking a result. State the masker test's
invariant without asserting a mechanism behind it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(masking): keep the security notice on a line of its own

`mask` appended the notice as a newline plus the notice text, which
assumes the caller hands it a bare log line. nativets hands it a chunk
that already ends in a newline, and its sink concatenates chunks
verbatim, so the notice arrived after a blank line and the next log line
was welded onto the end of it.

Emit the notice as its own line for either shape. `handle_child` is
unaffected: its input never ends in a newline, so it keeps the original
path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 15:15:02 +02:00
GuilhemandClaude Opus 5 3b4e13d1c5 fix: key the large root font size on screen width, not window width (#11216)
* fix: key the large root font size on screen width, not window width

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: move the flow preview button offset to the screen-width query

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 14:51:32 +02:00
Alexander PetricandClaude Fable 5.1 0cb92a7dcf chore(security): resolve Dependabot alerts in python-client/wmill/uv.lock (#11180)
Targeted lock bumps only (idna, pytest, pygments); requires-python kept at >=3.14.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 14:47:15 +02:00
Alexander PetricandClaude Fable 5.1 2a390c173d chore: remove two lockfiles that nothing consumes (#11183)
benchmarks/pulumi/package-lock.json: the directory is used with bun (bun.lockb); no workflow,
Dockerfile or script installs from this lockfile.

rust-client/Cargo.lock: library crate whose Cargo.toml is generated by dev.nu; the lockfile is
not consumed by downstream crates.io users and was stale (pinned 1.518.2). Ignored from now on.

Both only generated Dependabot alerts about dependency trees that never ship.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 14:45:37 +02:00
hugocasaandClaude Opus 5 a08992834d feat: render an AI agent result as its answer, not as raw JSON (#11051)
* feat: render an AI agent result as its answer, not as raw JSON

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: sanitize agent markdown through the shared plugin chain

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: fold agent stream events incrementally per poll

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: separate the agent meta line from the result toggle group

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: add a transcript view of an agent run's conversation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: retire AIAgentLogViewer in favour of the transcript

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: show the partial transcript a max-iterations failure carries

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: move the agent meta line and system prompt below the conversation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: show an agent run as what it did, not as a conversation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: name the agent run breakdown a trace

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): label the save-as-agent form fields per the guidelines

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): reuse the resource form's path and description fields

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): keep the action tags on a max-iterations failure

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): keep offline replay inert and the streamed answer to one turn

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): reset the streamed answer on providers that skip tool_call

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: carry the event type narrowing through the stream parser

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: survive a malformed message rather than take the viewer down

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: coerce agent messages once at the parse boundary

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: show an agent run as one scroll ending in its output

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep every streamed turn instead of dropping the narration

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: share the chat divider and drop the unsafe run auto-scroll

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: give the labelled divider a border colour and the standard pretty icon

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: pad the agent run below its badges as well as above

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: follow a streaming run's pane without moving the page

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: share the chat's stick-to-bottom mechanics with the agent run

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep the answer's citations and end a turn's reasoning with the turn

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: read the agent stream through the windmill-chat sdk parser

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: show an agent run's thinking instead of falling back to raw json

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a stream the fold cannot use from claiming the run pane

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: identify an agent run by more than the job id the replay withholds

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: drop the tests and comment lines that were not earning their place

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a streamed turn's text shifting when a tool call closes it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 14:38:46 +02:00
hugocasaandClaude Opus 5 48f00259c5 feat: support $flow_expr[...] dynamic tags on flow steps (#11170)
* feat: resolve $flow_expr[...] dynamic tags on flow steps

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: run an unresolvable $flow_expr tag on the default tag and give it the step input scope

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: resolve $flow_expr tags by path lookup instead of expression evaluation

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: say a $flow_expr tag fails to resolve, not to evaluate

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): word the $flow_expr help like the other dynamic tag lines

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: reject a malformed $flow_expr placeholder instead of queueing its literal tag

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: render $flow_expr tag values through the $args path lookup

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: pin interpolate_args through the shared tag path rendering

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:37:50 +02:00
hugocasaandClaude Opus 5 9320312eac feat: cap user token expiration with an instance setting (#11159)
* feat: cap user token expiration with an instance setting

Adds `max_token_expiration_days`, an instance-wide ceiling on how far ahead a
token created through `POST /users/tokens/create` may expire. With it set, that
route refuses a token with no expiration and one that expires past the window;
absent or non-positive, nothing changes.

Only the user-facing handler enforces it. Server-side mints (native trigger
webhook tokens, app embed tokens, sessions) pick a lifetime the caller never
chooses and go straight to `create_token_internal`, so they stay uncapped, as
does the superadmin `impersonate` route.

Service accounts are exempt, in the workspace the token targets or in any
workspace for a global token, so unattended automation can keep longer-lived
credentials.

The token form now surfaces the API error instead of only logging it, and offers
"Expires In" in MCP mode as well: that mode always sent no expiration, which the
cap refuses, leaving MCP URLs impossible to generate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: shorten over-long token expirations instead of refusing them

Refusing a non-compliant request breaks the callers that cannot comply. The CLI
authorization page, `wmill user create-token` and the editor's language-server
token each pick a lifetime — usually none at all — with no way to read the
setting, so a cap made browser login hang and the editor lose its LSP root
rather than stopping the long-lived tokens the setting is aimed at.

`cap_token_expiration` now returns the expiration to store, shortening a request
that asks for too long or for none. The policy still holds absolutely, no caller
can break, and there is no clock-skew boundary where an expiration exactly at the
ceiling flips to an error. The token form needed no changes at all, so its MCP
and error-toast edits are gone with it.

Also drops the Enterprise badge on the setting, which nothing enforced, notes the
mint paths in docs/auth-surface.md, and pins that `tokens/impersonate` and the
second-workspace case stay outside the exemption.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: drop the unintended token-form change and correct the exemption docs

The token form needed no change once the ceiling shortens rather than refuses,
but the earlier revert restored from the index, which already held the staged
edit, so the MCP expiration field and the error toast stayed on the branch with
a comment justifying them by a refusal that no longer happens.

docs/auth-surface.md claimed a service-account row in any workspace exempts
outright; that only holds for a workspace-less token, which has no workspace to
match. A ceiling written as a string, which the YAML instance config and config
sync can both produce, now has a test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: offer only expirations within the ceiling in the token form

The server shortens a token that asks for longer than `max_token_expiration_days`
or for no expiration, which the token form could not tell anyone: a user picking
"No expiration" got the ceiling silently. The form now reads the setting and,
with one set, drops "No expiration" and every choice above it, adds the ceiling
itself as "N days (maximum)" and selects it, and says the instance limits tokens
to N days.

MCP mode hides the expiration field and always sent none, so with a ceiling the
field now shows there too and keeps its value across the toggle. Without a
ceiling the form is unchanged.

Reading it needs no superadmin: the setting joins the keys any logged-in user
can read through `GET /settings/global/{key}`. It holds a policy, not a secret.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: make the token form and the server agree on what counts as a ceiling

The form parsed `max_token_expiration_days` more loosely than
`cap_token_expiration`, so the two could disagree on whether a ceiling exists
at all. A `7.0` from the YAML instance config, or a string such as "7.0" or
"1e1", made the form hide "No expiration" and announce a 7-day limit while the
server capped nothing; a value between chrono's and JavaScript's date limits
preselected an expiration the server could not parse.

Both now read the same thing as a ceiling: a whole number of days from 1 to
1,000,000, stored as an integer, an integral float or a string of digits.
`parseMaxTokenExpirationDays` holds the frontend's copy, and the instance
settings validation uses it too, so the settings page no longer accepts a value
the server would ignore. The bound replaces the date-range guard on both sides.

Also corrects the rationale for shortening rather than refusing: the setting is
now readable by any logged-in user, so those callers do not read it rather than
cannot, and CLIs already installed never will.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: cap service-account tokens like everyone else's

The ticket exempted service accounts from `max_token_expiration_days`, but
their tokens are the long-lived ones a rotation policy is meant to bound, and
the exemption let any workspace admin get an uncapped token by impersonating
one. It also left the token form unable to agree with the server: an admin
impersonating a service account was offered only capped choices while the
server would have kept any.

`cap_token_expiration` now takes just the requested expiration, with no
per-caller lookup, and the service-account query and its cache entry are gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: cap superadmin impersonation tokens and pin the frontend parser

`POST /users/tokens/impersonate` wrote its own token row with whatever
expiration the superadmin sent, so it was the one route left that could mint a
token that never expires with `max_token_expiration_days` set. The ceiling only
decides the stored expiration (the auth lookup never reads the setting), so
leaving it uncapped meant exactly that. It now goes through
`cap_token_expiration` like `create_token`; nothing in Windmill calls it, so no
caller changes.

Also adds `tokenExpiration.test.ts`, pinning which stored values
`parseMaxTokenExpirationDays` reads as a ceiling against the server's reading,
and documents that tokens existing when the setting is turned on or lowered
keep their expiration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: reject a max_token_expiration_days the token routes cannot read

The settings API and config sync stored any value for the key, and the token
routes can only read an unparseable one as no ceiling. A typo such as `7.5` or
"7.0" was accepted and silently turned the policy off.

`parse_max_token_expiration_days` in windmill-common is now the single server
reading of the setting: null or empty clears it, a whole number of days within
the bound is the ceiling, anything else is an error. The settings write hook and
`sync_global_settings_declarative` reject that error, and `cap_token_expiration`
reads through the same function, logging a value written around both.

Tests: the parser's accept/clear/reject table (the same table as the frontend
parser's), the settings API refusing 7.5, and config sync refusing "7.0".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: reserve the CLI login token label so its expiry does not email the user

With a token expiration ceiling, the token the CLI authorization page mints now
expires, so every `wmill` login earned an "expiring soon" and an "expired and
deleted" email and critical alert. The CLI already signs in again on its own
when that token stops working, so those notifications ask the user to do
nothing.

The page now labels it `cli-login:<username>` (previously `cli-<username>`),
reserved in `is_user_token` and its SQL and Svelte mirrors: no expiry
notifications, and the label cannot be edited. A colon-terminated namespace
like `embed_app:` and `impersonation:` keeps hand-made labels clear of it. Not
in `is_server_minted_label`, since the page mints through `/users/tokens/create`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: skip the expiring-soon warning for tokens that were short-lived from the start

A token whose whole lifetime fits in the 7-day warning window got its
"expiring soon" email (and critical alert, when enabled) minutes after it was
created, about a lifetime its creator had just picked. With an expiration
ceiling of 7 days or less that is every token created from the form or
`wmill token create`.

`register_token_expiry_notification` no longer queues a warning for such a
token. The window is now `TOKEN_EXPIRY_WARNING_DAYS`, shared with
`check_expiring_tokens`, so shortening the warning window can never leave
tokens of an intermediate lifetime with no warning at all. The "expired and
deleted" notice still goes out for every user token.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: exempt service-account tokens from the expiration ceiling again

Service accounts are the identity automation that needs a long-lived credential
runs as, so their tokens are exempt from `max_token_expiration_days` once more:
a service account in the workspace the token names, or in any workspace for a
workspace-less token. `tokens/impersonate` checks the impersonated account, so
a superadmin minting a token for a service account gets the same exemption.

The token form applies the same rule for the account it is running as, when
that account is a service account in the current workspace, which is what an
admin impersonating one sees; otherwise it would offer only capped choices
while the server keeps any.

Any workspace admin can create and impersonate a service account to hold an
uncapped token, so the ceiling bounds personal tokens; the doc comment and
docs/auth-surface.md say so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: decide the token form's service-account exemption from the token's workspace

The form treated the account as a service account only when it was one in the
workspace the app was on, while the server checks the workspace the token is
for, or any workspace for a workspace-less token. With an email that is a
service account in one workspace and an ordinary member of another, picking the
other workspace in MCP mode offered "No expiration" and the server silently
stored the ceiling; the reverse hid the exemption.

`GET /workspaces/users` now returns each membership's `is_service_account`
(its query already joins the `usr` row), and the form applies the server's rule
to the token's own workspace. The selection becomes a derived value held within
the ceiling, so switching to a capped workspace never leaves an unoffered choice
selected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:37:39 +02:00
hugocasaandClaude Opus 5 9690c4462c fix: re-point cloned fork identities that name nobody in the fork (#11161)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:37:25 +02:00
Alexander PetricandClaude Fable 5.1 de4818d905 chore(security): resolve Dependabot alerts in frontend, windmill-yaml-validator, typescript-client and cli lockfiles (#11181)
Targeted dependency bumps only; no behavior change intended.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 14:36:33 +02:00
Alexander PetricandClaude Fable 5.1 1015874f7c install the multiplayer server from its lockfile (#11190)
DockerfileExtra copied multiplayer/package.json alone and ran `npm install`,
so the image shipped whatever npm resolved at build time and the committed
multiplayer/package-lock.json described nothing that ran in production:
Dependabot alerts on it were meaningless either way. Copy the lockfile and
install with `npm ci --omit=dev` so the image is reproducible and the lock is
the source of truth for what ships.

The lock is refreshed with `npm update <dep>` per direct dependency, which
moves each to the newest version inside its existing caret range without
touching package.json. That is exactly what a lockfile-less `npm install`
resolves today (verified: a fresh `npm install --package-lock-only` from the
same package.json produces identical versions), so the image does not regress:

  ws           8.19.0  -> 8.21.3  (covers the open alerts on ws < 8.21.0)
  y-websocket  3.0.0   -> 3.1.0
  yjs          13.6.29 -> 13.6.32
  lib0 0.2.117, y-protocols 1.0.7, isomorphic.js 0.2.5 unchanged

package.json has no devDependencies and the lock has no `dev: true` entries,
so `--omit=dev` changes nothing today and only guards against future ones.

Validation:
- `npm ci` on the pre-update lock and `npm ci --omit=dev` on the updated lock
  both succeed; `node --check server.mjs gateway.mjs` passes.
- Smoke start: server.mjs listens on PORT=3999 and logs "Multiplayer server
  running"; gateway.mjs listens on PORT=3998 and logs its route table.
- The same COPY/RUN lines built on node:22-slim with the repo root as build
  context (the context build-extra-image.yml uses) install the six locked
  packages and pass `node --check`. A full DockerfileExtra build was skipped:
  it is a single stage on an uncached multi-GB base image.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 14:35:23 +02:00
Alexander PetricandClaude Fable 5.1 813e486e16 fix(frontend): inline only the package version, not the whole package.json (#11191)
`vite.config.js` had `define: { __pkg__: version }` where `version` was
the entire parsed `package.json`. Vite's `define` substitutes the full
object literal at every site that reads `__pkg__.version`, so each of
those sites became `{name:..., version:..., scripts:{...},
devDependencies:{...}, ...}.version` in the client bundle and the
minifier kept the object. On origin/main that is 11 copies of a
22,093-byte object (npm scripts, dependency/devDependency lists,
overrides) across three chunks — 243,023 bytes of package.json shipped
to every browser, for a single version string.

The only property ever read is `__pkg__.version` (script_helpers.ts,
apps/editor/component/default-codes.ts, flows/content/s3Scripts/deno.ts),
and the declarations in src/app.d.ts, src/global.d.ts and
sharedUtils/sharedUtils.d.ts already type it as `{ version: string }`,
so define the dotted expression `__pkg__.version` to the JSON-encoded
version string instead. Any new property read would now fail
`npm run check` rather than silently compile to `undefined`.

Verification (npm ci, generate-backend-client, check, build):
- `npm run check`: 0 errors, 82 warnings.
- `grep -l '"generate-backend-client"' build/_app/immutable/**/*.js`:
  3 chunks before, none after.
- `grep -c 'windmill-labs/components' -r build/_app/immutable`: 11 → 0.
- `npm:windmill-client@1.813.0` still appears in the chunks that build
  the default Deno/pgsql/S3 snippets (3 chunks, 11 sites).
- build/_app/immutable apparent size: 81,815 KB → 81,578 KB
  (JS bytes 54,202,793 → 53,959,705, -243,088 bytes).

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-18 14:34:11 +02:00
Diego Imbert ecd0a6c77b fix: show the New menu's description panel only on hover (#11199)
* fix: show the New menu's description panel only on hover

* fix: keep the New menu's submenus reachable with the mouse
2026-09-18 14:33:24 +02:00
df61dea5fa fix: keep instance groups when editing auto-invite (#11217)
* fix: keep instance groups when editing auto-invite and push them from sync

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: compare settings.yaml without undeclared instance groups on push

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: read a missing auto_invite as empty when diffing settings.yaml on push

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: update ee-repo-ref to f2fced19fcae81de7f6dac545010ce404c052e1b

This commit updates the EE repository reference after PR #813 was merged in windmill-ee-private.

Previous ee-repo-ref: cf4258c1232720ca3b82db2b22ea1fb4bca9533e

New ee-repo-ref: f2fced19fcae81de7f6dac545010ce404c052e1b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-18 14:32:55 +02:00
hugocasaandClaude Opus 5 5639187fec feat(auth): 2 h login links and a click-to-sign-in page for emailed ones (#11203)
* feat(auth): 2 h login links and a click-to-sign-in page for emailed ones

Raise the login link cap from 15 min to 2 h, so a link sent by email still
works when it is read.

A link minted with `confirm: true` is a /user/login_link page instead of
the API path. Loading the page does nothing; its button POSTs to
/api/auth/login_link/{token}, which spends the link and answers where to go.
Mail scanners that open links on delivery no longer burn them. Links minted
without `confirm` still sign in on open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(auth): keep the login link page to design-system components

A tokenless visit bounced off a raw <p>; send it to the page a spent link
already bounces to, and show the modal's own spinner while it goes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:31:16 +02:00
AlexRV12andClaude Opus 5 c8c06d8f79 feat(ai-chat): tell the chat which kind of app it is looking at (#11208)
* feat(ai-chat): tell the chat which kind of app it is looking at

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018r5ppn8YmHsM1MaAwY7xgB

* refactor(ai-chat): drop a stale comment and hoist the prefix local

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018r5ppn8YmHsM1MaAwY7xgB

* test(ai-chat): add a global eval for refusing to edit a drag-and-drop app

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018r5ppn8YmHsM1MaAwY7xgB

* test(ai-chat): make the app-kind eval require editing the code app too

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018r5ppn8YmHsM1MaAwY7xgB

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 14:19:06 +02:00
AlexRV12andClaude Opus 5 6f9c4dc294 feat(ai-chat): render get_run through the run tools' card (#11204)
* feat(ai-chat): render get_run through the run tools' card

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-chat): retry unreadable logs on reopen and drop the loading tint

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(ai-chat): inline two single-use deriveds in the run card

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(ai-chat): record why an inspected run's logs need the dedicated endpoint

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-chat): key an inspected run's fetch to the tool call, not the job

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(ai-chat): say an inspected run's logs are a tail, not the whole log

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 14:16:52 +02:00
Henri CourdentandRuben Fiszel 041ac95dc4 Auth providers capital (#11209)
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-18 13:59:58 +02:00
209 changed files with 7090 additions and 14260 deletions
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "1.814.0"
".": "1.815.0"
}
+34
View File
@@ -1,5 +1,39 @@
# Changelog
## [1.815.0](https://github.com/windmill-labs/windmill/compare/v1.814.0...v1.815.0) (2026-09-18)
### Features
* add a workspace toggle that adds its admins and developers to new forks ([#11215](https://github.com/windmill-labs/windmill/issues/11215)) ([9d335de](https://github.com/windmill-labs/windmill/commit/9d335de87a4dbaa51038d55afe8d980761dcdfaf))
* add an instance setting to refuse a token in MCP URLs ([#11162](https://github.com/windmill-labs/windmill/issues/11162)) ([37e493a](https://github.com/windmill-labs/windmill/commit/37e493ae66ed5c000ecac492d60fc0fdf4bda71f))
* **ai-chat:** render get_run through the run tools' card ([#11204](https://github.com/windmill-labs/windmill/issues/11204)) ([6f9c4dc](https://github.com/windmill-labs/windmill/commit/6f9c4dc29455d13b0e64af05c2d6aa8bd5ff4fd6))
* **ai-chat:** tell the chat which kind of app it is looking at ([#11208](https://github.com/windmill-labs/windmill/issues/11208)) ([c8c06d8](https://github.com/windmill-labs/windmill/commit/c8c06d8f79774abf109192e71a8b6fc37c7937ba))
* attach files to a flow chat message ([#11185](https://github.com/windmill-labs/windmill/issues/11185)) ([c4c9677](https://github.com/windmill-labs/windmill/commit/c4c9677982b75c63d98ebf85b1904e0c341ba957))
* **auth:** 2 h login links and a click-to-sign-in page for emailed ones ([#11203](https://github.com/windmill-labs/windmill/issues/11203)) ([5639187](https://github.com/windmill-labs/windmill/commit/5639187fec6d517a72e82df49d63d7438301127c))
* cap user token expiration with an instance setting ([#11159](https://github.com/windmill-labs/windmill/issues/11159)) ([9320312](https://github.com/windmill-labs/windmill/commit/9320312eac56f944c4d31504601293ab4e816ccc))
* flow chat job-backed detail, smooth streaming and answer chrome ([#11186](https://github.com/windmill-labs/windmill/issues/11186)) ([e2a91ca](https://github.com/windmill-labs/windmill/commit/e2a91ca2b141a0bee64ddfafe3cbf3ff86b6fb35))
* put a data table's connection under Postgres roles ([#11020](https://github.com/windmill-labs/windmill/issues/11020)) ([0e807fb](https://github.com/windmill-labs/windmill/commit/0e807fb1dd80d7536ec144cd49445abc7961e504))
* render an AI agent result as its answer, not as raw JSON ([#11051](https://github.com/windmill-labs/windmill/issues/11051)) ([a089928](https://github.com/windmill-labs/windmill/commit/a08992834d45d0211336f4fc32c3421646ca47c5))
* support $flow_expr[...] dynamic tags on flow steps ([#11170](https://github.com/windmill-labs/windmill/issues/11170)) ([48f0025](https://github.com/windmill-labs/windmill/commit/48f00259c5e7361d3553dbcb809461e1cde96f8f))
### Bug Fixes
* **frontend:** inline only the package version, not the whole package.json ([#11191](https://github.com/windmill-labs/windmill/issues/11191)) ([813e486](https://github.com/windmill-labs/windmill/commit/813e486e166ac6215364817a7732b66f8dc1d463))
* keep instance groups when editing auto-invite ([#11217](https://github.com/windmill-labs/windmill/issues/11217)) ([df61dea](https://github.com/windmill-labs/windmill/commit/df61dea5fa8b18d1e0044dc0db6702b053d5119f))
* key the large root font size on screen width, not window width ([#11216](https://github.com/windmill-labs/windmill/issues/11216)) ([3b4e13d](https://github.com/windmill-labs/windmill/commit/3b4e13d1c564c6195e30b55b0671f7533e3ce408))
* re-encrypt git sync secrets on workspace key rotation ([#11218](https://github.com/windmill-labs/windmill/issues/11218)) ([f0d66a4](https://github.com/windmill-labs/windmill/commit/f0d66a42eba27502767423af0c08b2ca1f16e8df))
* re-point cloned fork identities that name nobody in the fork ([#11161](https://github.com/windmill-labs/windmill/issues/11161)) ([9690c44](https://github.com/windmill-labs/windmill/commit/9690c4462cf264a5577b87d07d465b5442b4e09d))
* register the job token with the sensitive log masking system ([#10943](https://github.com/windmill-labs/windmill/issues/10943)) ([53afecd](https://github.com/windmill-labs/windmill/commit/53afecd4588247bc1812d3e68a30db1f3c3b2724))
* show the New menu's description panel only on hover ([#11199](https://github.com/windmill-labs/windmill/issues/11199)) ([ecd0a6c](https://github.com/windmill-labs/windmill/commit/ecd0a6c77bc3a057b8072dbb0aca731e8bd3d882))
* stop picker listing a draft twice when drafts share a friendly path ([#11214](https://github.com/windmill-labs/windmill/issues/11214)) ([72507d5](https://github.com/windmill-labs/windmill/commit/72507d52a021de8101794ee8f4a3392be1ba20c7))
### Performance Improvements
* count completed jobs on the v2_job_completed index with a timeout ([#11211](https://github.com/windmill-labs/windmill/issues/11211)) ([d1a2536](https://github.com/windmill-labs/windmill/commit/d1a25360b070994a31fb033707dbfde1fb553939))
## [1.814.0](https://github.com/windmill-labs/windmill/compare/v1.813.0...v1.814.0) (2026-09-17)
+8 -4
View File
@@ -49,12 +49,16 @@ export interface BenchmarkWorkspaceFlow {
export interface BenchmarkWorkspaceApp {
path: string
summary: string
/** Defaults to true. Set false for a drag-and-drop app, which the chat can list
* and read but has no tool to edit — its value is a grid, not files. */
rawApp?: boolean
value: {
files: Record<string, string>
runnables: Record<string, unknown>
files?: Record<string, string>
runnables?: Record<string, unknown>
data?: unknown
policy?: unknown
custom_path?: unknown
[key: string]: unknown
}
}
@@ -994,7 +998,7 @@ function buildBenchmarkListableApp(app: BenchmarkWorkspaceApp): ListableApp {
extra_perms: {},
edited_at: BENCHMARK_TIMESTAMP,
execution_mode: 'viewer',
raw_app: true
raw_app: app.rawApp ?? true
}
}
@@ -1012,7 +1016,7 @@ function buildBenchmarkApp(app: BenchmarkWorkspaceApp): AppWithLastVersion {
execution_mode: 'viewer',
extra_perms: {},
custom_path: app.value.custom_path as string | undefined,
raw_app: true
raw_app: app.rawApp ?? true
}
}
+37
View File
@@ -2611,3 +2611,40 @@
judgeChecklist:
- runs the existing script rather than rewriting it
- passes the GitHub resource as the bare string $res:f/evals/global/github_main
- id: global-drag-and-drop-app-not-editable
prompt: |-
Add a refresh button to the ops console app, and the same to the sales board app.
initial: ai_evals/fixtures/frontend/global/initial/apps_code_and_drag_and_drop.json
runtime:
maxTurns: 12
validate:
# One request, two apps, only one of them editable: the code app must come back with a
# draft and the drag-and-drop one must not. Refusing both, or editing both, fails here —
# which is what makes this a test of the distinction rather than of caution.
draftCountExactly: 1
requiredDrafts:
- type: app
path: f/evals/global/ops_console
forbiddenDrafts:
- type: app
path: f/evals/global/sales_board
toolExpect:
# Deliberately not constraining write_app_file/patch_app_file by argument: an entry there
# fails when its tool was never called, so naming both would fail on whichever the model
# did not pick. The draft assertions above cover the same ground, tool-agnostically.
forbiddenToolsUsed:
- init_app
- deploy_workspace_item
- delete_app_file
- delete_app_runnable
assistantExpect:
# A refusal leaves no draft for the judge to read, so the explanation is checked here.
# Only the app kind: substring tests cannot see paraphrase, and every wording of "I can't
# edit it" defeats a fixed list.
requiredMentionsAnyOf:
- - drag-and-drop
- drag and drop
- low-code
- no-code
skipJudge: true
@@ -0,0 +1,30 @@
{
"user": {
"username": "admin",
"is_admin": true,
"folders": ["evals"],
"folders_read": ["evals"]
},
"workspace": {
"apps": [
{
"path": "f/evals/global/sales_board",
"summary": "Sales board",
"rawApp": false,
"value": {
"grid": []
}
},
{
"path": "f/evals/global/ops_console",
"summary": "Ops console",
"value": {
"files": {
"/App.tsx": "export default function App() {\n\treturn <div>Ops console</div>\n}\n"
},
"runnables": {}
}
}
]
}
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "1d8ccd32266637d7f7915f92a8483dce0c9986f3847227c2f66daa84d4109d7d"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET auto_invite = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Text"
]
},
"nullable": []
},
"hash": "255ba68caa78d0e814ea817693a319f769907025f9c7d1150d45e8ecb1bff4ab"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via)\n SELECT $1, username, email, is_admin, created_at, operator, disabled, role, is_service_account,\n CASE WHEN $3 THEN NULL ELSE added_via END\n FROM usr WHERE workspace_id = $2\n AND (NOT $3 OR (NOT operator AND NOT disabled AND NOT is_service_account))\n ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text",
"Bool"
]
},
"nullable": []
},
"hash": "2d6b117324eaf076a0ed06d2cb0ce73279957d6a39fdc6b1ecb6e0a1e02f921f"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT add_admins_and_developers_to_forks FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "add_admins_and_developers_to_forks",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "2f39fce0ee700117f3e4c066e0b56ee979e4c392970278304e80368b770bb7b4"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE token SET label = $1\n WHERE email = $2 AND token_prefix = $3\n AND (label IS NULL OR (\n label <> 'session'\n AND label <> 'guest_session'\n AND lower(label) NOT LIKE 'ephemeral%'\n AND label <> 'debugger-token'\n AND label NOT LIKE 'mcp-oauth-%'\n AND NOT starts_with(label, 'embed_app:')\n AND NOT starts_with(label, 'sdk_app:')\n AND NOT starts_with(label, 'impersonation:')\n ))\n RETURNING token_prefix",
"query": "UPDATE token SET label = $1\n WHERE email = $2 AND token_prefix = $3\n AND (label IS NULL OR (\n label <> 'session'\n AND label <> 'guest_session'\n AND lower(label) NOT LIKE 'ephemeral%'\n AND label <> 'debugger-token'\n AND label NOT LIKE 'mcp-oauth-%'\n AND NOT starts_with(label, 'embed_app:')\n AND NOT starts_with(label, 'sdk_app:')\n AND NOT starts_with(label, 'impersonation:')\n AND NOT starts_with(label, 'cli-login:')\n ))\n RETURNING token_prefix",
"describe": {
"columns": [
{
@@ -20,5 +20,5 @@
false
]
},
"hash": "d631a26e5531589ff37e677f91a4f1d9f850e3e46c17130dd580426cda7a9f65"
"hash": "383c80239525d9c4ee90e2f5cb6e46c8a0983c52ad15db083f84a9e344dc4c99"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET\n ai_config = source_ws.ai_config,\n large_file_storage = source_ws.large_file_storage,\n ducklake = source_ws.ducklake,\n dbt_warehouses = source_ws.dbt_warehouses,\n datatable = source_ws.datatable,\n git_app_installations = source_ws.git_app_installations\n FROM workspace_settings source_ws\n WHERE source_ws.workspace_id = $1\n AND workspace_settings.workspace_id = $2\n ",
"query": "\n UPDATE workspace_settings\n SET\n ai_config = source_ws.ai_config,\n large_file_storage = source_ws.large_file_storage,\n ducklake = source_ws.ducklake,\n dbt_warehouses = source_ws.dbt_warehouses,\n datatable = source_ws.datatable,\n git_app_installations = source_ws.git_app_installations,\n add_admins_and_developers_to_forks = source_ws.add_admins_and_developers_to_forks\n FROM workspace_settings source_ws\n WHERE source_ws.workspace_id = $1\n AND workspace_settings.workspace_id = $2\n ",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "a6decdebcd9750691f20f874b66a9c6f2ede08c33605c6a3c3dfc213a3eda76a"
"hash": "5ccfbd0f345b9b86ca356008def6cc1011a49fc8d8ad046ac572e81cf8938995"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM token_expiry_notification n\n USING token t\n WHERE n.token_hash = t.token_hash\n AND n.expiration > now()\n AND n.expiration <= now() + interval '7 days'\n RETURNING t.token_prefix, t.label, t.email, t.workspace_id",
"query": "DELETE FROM token_expiry_notification n\n USING token t\n WHERE n.token_hash = t.token_hash\n AND n.expiration > now()\n AND n.expiration <= now() + make_interval(days => $1)\n RETURNING t.token_prefix, t.label, t.email, t.workspace_id",
"describe": {
"columns": [
{
@@ -25,7 +25,9 @@
}
],
"parameters": {
"Left": []
"Left": [
"Int4"
]
},
"nullable": [
false,
@@ -34,5 +36,5 @@
true
]
},
"hash": "94fd0a57cfc9341b2e9deae60506c6c06aa6934b87200da14231f12f65149cd3"
"hash": "6c57c46c5a0462f379ed6a22ae97e363009177fdc45d795447a920f455f193cc"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM usr WHERE email = $1 AND is_service_account IS true\n AND ($2::varchar IS NULL OR workspace_id = $2))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Varchar"
]
},
"nullable": [
null
]
},
"hash": "75e6b5cd52d63ac094c90abd1524bc7a919692a96b076b8aaf24d812723531c8"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT workspace.id, workspace.name, usr.username, workspace_settings.color, workspace.parent_workspace_id,\n workspace.is_dev_workspace, workspace.dev_workspace_label,\n workspace.owner AS \"created_by?\",\n CASE WHEN usr.operator THEN workspace_settings.operator_settings ELSE NULL END as operator_settings,\n usr.disabled\n FROM workspace\n JOIN usr ON usr.workspace_id = workspace.id\n JOIN workspace_settings ON workspace_settings.workspace_id = workspace.id\n WHERE usr.email = $1 AND workspace.deleted = false",
"query": "SELECT workspace.id, workspace.name, usr.username, workspace_settings.color, workspace.parent_workspace_id,\n workspace.is_dev_workspace, workspace.dev_workspace_label,\n workspace.owner AS \"created_by?\",\n CASE WHEN usr.operator THEN workspace_settings.operator_settings ELSE NULL END as operator_settings,\n usr.disabled, usr.is_service_account\n FROM workspace\n JOIN usr ON usr.workspace_id = workspace.id\n JOIN workspace_settings ON workspace_settings.workspace_id = workspace.id\n WHERE usr.email = $1 AND workspace.deleted = false",
"describe": {
"columns": [
{
@@ -52,6 +52,11 @@
"ordinal": 9,
"name": "disabled",
"type_info": "Bool"
},
{
"ordinal": 10,
"name": "is_service_account",
"type_info": "Bool"
}
],
"parameters": {
@@ -69,8 +74,9 @@
true,
false,
null,
false,
false
]
},
"hash": "88a134e4ca82d5ce0334977c7713021ae3e99a5a61ea1c944c1df1368746dfa5"
"hash": "7b33adb5cf051bc123340982b97a80a96ee43e1c91d9907d50cc2ee14111623e"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n customer_id,\n plan,\n webhook,\n ai_config,\n dbt_warehouses,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_invite,\n error_handler,\n success_handler,\n public_app_execution_limit_per_minute,\n error_handler_fallback_to_instance_alerts,\n guest_access_enabled,\n guest_jwt_public_key,\n guest_jwt_jwks_url\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n customer_id,\n plan,\n webhook,\n ai_config,\n dbt_warehouses,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_invite,\n error_handler,\n success_handler,\n public_app_execution_limit_per_minute,\n error_handler_fallback_to_instance_alerts,\n guest_access_enabled,\n guest_jwt_public_key,\n guest_jwt_jwks_url,\n add_admins_and_developers_to_forks\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
@@ -177,6 +177,11 @@
"ordinal": 34,
"name": "guest_jwt_jwks_url",
"type_info": "Text"
},
{
"ordinal": 35,
"name": "add_admins_and_developers_to_forks",
"type_info": "Bool"
}
],
"parameters": {
@@ -219,8 +224,9 @@
false,
false,
true,
true
true,
false
]
},
"hash": "dc4a57df3becc610f631ef22c116450390addbfae85fecc61c991d94167e6e99"
"hash": "8ebe054b41793f1a7b85f1f8d29cd21d12fa1207b58bfc46249c250cdcdb5363"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET add_admins_and_developers_to_forks = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Text"
]
},
"nullable": []
},
"hash": "9bd1995747f0073b3a866d1238e78f7e6dfa9185056f7731547ff05b8176b271"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET auto_invite = (COALESCE(auto_invite, '{}'::jsonb) - 'domain') || $1::jsonb WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Text"
]
},
"nullable": []
},
"hash": "a1bcf143135ecb9f32709a56a353d3246dc238920c3e155caa9cf767246fa6fb"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via)\n SELECT $1, username, email, is_admin, created_at, operator, disabled, role, is_service_account, added_via\n FROM usr WHERE workspace_id = $2\n ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "b98844926ff127c528ed3e7bc63bf1ebed0192be267983e8fbd18f79997e6142"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n slack_name,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n mute_critical_alerts,\n guest_access_enabled,\n deploy_ui,\n large_file_storage,\n datatable\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n slack_name,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n mute_critical_alerts,\n guest_access_enabled,\n add_admins_and_developers_to_forks,\n deploy_ui,\n large_file_storage,\n datatable\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
@@ -45,16 +45,21 @@
},
{
"ordinal": 8,
"name": "add_admins_and_developers_to_forks",
"type_info": "Bool"
},
{
"ordinal": 9,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 9,
"ordinal": 10,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "datatable",
"type_info": "Jsonb"
}
@@ -73,10 +78,11 @@
true,
true,
false,
false,
true,
true,
true
]
},
"hash": "ede15bff96152f209aff756830cbc76b5afa1af6ed324376989117b1054c3447"
"hash": "e6e31fdf705896c81f9a0f45d47c1b93db0406aaf278bccba07be00e9f937e2a"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "eefa0588a6a927fd9b3f65e1df652fb2b4cf7983049d2c490940df360c7e2b30"
}
+85 -86
View File
@@ -2311,9 +2311,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.4.6"
version = "1.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3eb0f42d6c360dc3f8a821f6bf2fdea7f72bfd36b3076eb0e6d1e9e0752fff4"
checksum = "54413ede23c2daf518f35156dfde027feb2374004d63bd497f983c8db9c0e313"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -5055,9 +5055,9 @@ dependencies = [
[[package]]
name = "find-msvc-tools"
version = "0.1.12"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d"
checksum = "ef25905e51abafe4dcea6c15fec58c57b601cdbd0ee53d22ea1d3016c587d39b"
[[package]]
name = "fixedbitset"
@@ -5497,9 +5497,9 @@ dependencies = [
[[package]]
name = "generator"
version = "0.8.9"
version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3b854b0e584ead1a33f18b2fcad7cf7be18b3875c78816b753639aa501513ae"
checksum = "54ade96dc9003043bce7c035c85a9df5a858bfb2039c5a2e6fdf00f324f6c551"
dependencies = [
"cc",
"cfg-if",
@@ -14793,7 +14793,7 @@ dependencies = [
[[package]]
name = "windmill"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-nats",
@@ -14881,7 +14881,7 @@ dependencies = [
[[package]]
name = "windmill-ai"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"async-stream",
"async-trait",
@@ -14915,7 +14915,7 @@ dependencies = [
[[package]]
name = "windmill-alerting"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -14928,7 +14928,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"argon2",
@@ -15068,7 +15068,7 @@ dependencies = [
[[package]]
name = "windmill-api-agent-workers"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15091,7 +15091,7 @@ dependencies = [
[[package]]
name = "windmill-api-assets"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15108,7 +15108,7 @@ dependencies = [
[[package]]
name = "windmill-api-auth"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15134,7 +15134,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -15144,7 +15144,7 @@ dependencies = [
[[package]]
name = "windmill-api-configs"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15161,7 +15161,7 @@ dependencies = [
[[package]]
name = "windmill-api-debug"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"base64 0.22.1",
@@ -15183,7 +15183,7 @@ dependencies = [
[[package]]
name = "windmill-api-embeddings"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15206,7 +15206,7 @@ dependencies = [
[[package]]
name = "windmill-api-flow-conversations"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15222,7 +15222,7 @@ dependencies = [
[[package]]
name = "windmill-api-flows"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15244,7 +15244,7 @@ dependencies = [
[[package]]
name = "windmill-api-groups"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15266,7 +15266,7 @@ dependencies = [
[[package]]
name = "windmill-api-inputs"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15280,7 +15280,7 @@ dependencies = [
[[package]]
name = "windmill-api-integration-tests"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-nats",
@@ -15315,7 +15315,7 @@ dependencies = [
[[package]]
name = "windmill-api-jobs"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15340,7 +15340,7 @@ dependencies = [
[[package]]
name = "windmill-api-npm-proxy"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15368,7 +15368,7 @@ dependencies = [
[[package]]
name = "windmill-api-openapi"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15390,7 +15390,7 @@ dependencies = [
[[package]]
name = "windmill-api-schedule"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15410,7 +15410,7 @@ dependencies = [
[[package]]
name = "windmill-api-scripts"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15448,7 +15448,7 @@ dependencies = [
[[package]]
name = "windmill-api-settings"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15477,7 +15477,7 @@ dependencies = [
[[package]]
name = "windmill-api-sse"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"lazy_static",
"serde",
@@ -15489,7 +15489,7 @@ dependencies = [
[[package]]
name = "windmill-api-users"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"argon2",
"axum 0.8.9",
@@ -15513,7 +15513,7 @@ dependencies = [
[[package]]
name = "windmill-api-workers"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15527,7 +15527,7 @@ dependencies = [
[[package]]
name = "windmill-api-workspaces"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15562,7 +15562,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"chrono",
"lazy_static",
@@ -15576,7 +15576,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15595,7 +15595,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"aes-gcm",
"aho-corasick",
@@ -15653,7 +15653,6 @@ dependencies = [
"pin-project-lite",
"pkcs1",
"postgres-native-tls 0.5.3",
"postgres-protocol",
"prometheus",
"quick_cache",
"rand 0.9.0",
@@ -15703,7 +15702,7 @@ dependencies = [
[[package]]
name = "windmill-dep-map"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"chrono",
"futures",
@@ -15723,7 +15722,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"regex",
"serde",
@@ -15740,7 +15739,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -15767,7 +15766,7 @@ dependencies = [
[[package]]
name = "windmill-jseval"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"futures",
@@ -15784,7 +15783,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15800,7 +15799,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -15821,7 +15820,7 @@ dependencies = [
[[package]]
name = "windmill-native-triggers"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -15852,7 +15851,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"arc-swap",
@@ -15877,7 +15876,7 @@ dependencies = [
[[package]]
name = "windmill-object-store"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-stream",
@@ -15912,7 +15911,7 @@ dependencies = [
[[package]]
name = "windmill-operator"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"futures",
@@ -15930,7 +15929,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15939,7 +15938,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15951,7 +15950,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15963,7 +15962,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"gosyn",
@@ -15975,7 +15974,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15987,7 +15986,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15999,7 +15998,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -16010,7 +16009,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16021,7 +16020,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16033,7 +16032,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-asset"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -16044,7 +16043,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16066,7 +16065,7 @@ dependencies = [
[[package]]
name = "windmill-parser-r"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16078,7 +16077,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16092,7 +16091,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16109,7 +16108,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16122,7 +16121,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql-asset"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde",
@@ -16134,7 +16133,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16152,7 +16151,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts-asset"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde-wasm-bindgen",
@@ -16168,7 +16167,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wac"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -16184,7 +16183,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16198,7 +16197,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16237,7 +16236,7 @@ dependencies = [
[[package]]
name = "windmill-runtime-nativets"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"const_format",
@@ -16277,7 +16276,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"getrandom 0.3.4",
"wasm-bindgen",
@@ -16288,7 +16287,7 @@ dependencies = [
[[package]]
name = "windmill-store"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16323,7 +16322,7 @@ dependencies = [
[[package]]
name = "windmill-test-utils"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16347,7 +16346,7 @@ dependencies = [
[[package]]
name = "windmill-trigger"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16380,7 +16379,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-amqp"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16407,7 +16406,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-azure"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16440,7 +16439,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-email"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16460,7 +16459,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-gcp"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16494,7 +16493,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-http"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16530,7 +16529,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-kafka"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16553,7 +16552,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-mqtt"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16577,7 +16576,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-nats"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-nats",
@@ -16601,7 +16600,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-postgres"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16636,7 +16635,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-sqs"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16664,7 +16663,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-websocket"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16689,7 +16688,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"bitflags 2.13.2",
@@ -16708,7 +16707,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-once-cell",
@@ -16826,7 +16825,7 @@ dependencies = [
[[package]]
name = "windmill-worker-volumes"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"bytes",
"futures",
+2 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.814.0"
version = "1.815.0"
authors.workspace = true
edition.workspace = true
@@ -88,7 +88,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
[workspace.package]
version = "1.814.0"
version = "1.815.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -624,7 +624,6 @@ wasm-bindgen-test = "^0"
convert_case = "0.6.0"
getrandom = "0.2"
tokio-postgres = {version = "^0.7", features = ["array-impls", "with-serde_json-1", "with-chrono-0_4", "with-uuid-1", "with-bit-vec-0_6"]}
postgres-protocol = "0.6"
rust-postgres = { package = "tokio-postgres", git = "https://github.com/MaterializeInc/rust-postgres", rev = "78c1222577bb091d69bc22b1bc7ad01c14675abe"}
rust-postgres-native-tls = { package = "postgres-native-tls", git = "https://github.com/MaterializeInc/rust-postgres", features = ["runtime"], rev = "78c1222577bb091d69bc22b1bc7ad01c14675abe" }
bit-vec = "=0.6.3"
+1 -1
View File
@@ -1 +1 @@
220301c27272b6c9e38401884c45cd959bd02a09
bc3ef08c8e4233508c023e6ee847a3cd0b8be43b
@@ -0,0 +1 @@
ALTER TABLE workspace_settings DROP COLUMN add_admins_and_developers_to_forks;
@@ -0,0 +1 @@
ALTER TABLE workspace_settings ADD COLUMN add_admins_and_developers_to_forks BOOLEAN NOT NULL DEFAULT false;
+24 -24
View File
@@ -6191,7 +6191,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill-common"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"aho-corasick",
"anyhow",
@@ -6274,7 +6274,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"proc-macro2",
"quote",
@@ -6286,7 +6286,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"convert_case",
"serde",
@@ -6295,7 +6295,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6307,7 +6307,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6319,7 +6319,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"gosyn",
@@ -6331,7 +6331,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6343,7 +6343,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6355,7 +6355,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -6366,7 +6366,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -6377,7 +6377,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -6389,7 +6389,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-asset"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -6400,7 +6400,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -6422,7 +6422,7 @@ dependencies = [
[[package]]
name = "windmill-parser-r"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6434,7 +6434,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6448,7 +6448,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"convert_case",
@@ -6465,7 +6465,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6478,7 +6478,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql-asset"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde",
@@ -6490,7 +6490,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6508,7 +6508,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts-asset"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"serde-wasm-bindgen",
@@ -6524,7 +6524,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wac"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -6540,7 +6540,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wasm"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"getrandom 0.2.17",
@@ -6572,7 +6572,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6586,7 +6586,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.814.0"
version = "1.815.0"
dependencies = [
"anyhow",
"bitflags",
@@ -12,7 +12,7 @@ resolver = "2"
members = ["."]
[workspace.package]
version = "1.814.0"
version = "1.815.0"
edition = "2021"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
+7 -2
View File
@@ -52,7 +52,8 @@ use windmill_common::{
INSTANCE_EVENTS_WEBHOOK_SETTING, INSTANCE_PYTHON_VERSION_SETTING,
JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING, JWT_SECRET_SETTING,
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING, MAVEN_SETTINGS_XML_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING,
NO_DEFAULT_MAVEN_SETTING,
NPM_CONFIG_REGISTRY_SETTING, NSJAIL_TMPFS_SIZE_MB_SETTING, NSJAIL_TMP_BACKING_SETTING,
NUGET_CONFIG_SETTING, OAUTH_SETTING, OTEL_SETTING, OTEL_TRACES_RETENTION_SECS_SETTING,
OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
@@ -126,7 +127,8 @@ use windmill_worker::{
use crate::monitor::{
initial_load, load_concurrency_key_max_queued, load_disable_password_login,
load_fork_workspace_tag_append_fork_suffix, load_keep_job_dir, load_metrics_debug_enabled,
load_fork_workspace_tag_append_fork_suffix, load_keep_job_dir,
load_mcp_disable_token_query_param, load_metrics_debug_enabled,
load_preview_tags_override, load_require_preexisting_user, load_retention_period_overrides,
load_tag_per_workspace_enabled, load_tag_per_workspace_workspaces,
load_workspace_fairness_duration_secs, load_workspace_fairness_enabled,
@@ -2164,6 +2166,9 @@ async fn process_notify_event(
DISABLE_PASSWORD_LOGIN_SETTING => {
load_disable_password_login(db).await;
}
MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING => {
load_mcp_disable_token_query_param(db).await;
}
EXPOSE_METRICS_SETTING => {
tracing::info!("Metrics setting changed, restarting");
spawn_graceful_killpill(tx, db, 30, "metrics setting change", server_mode)
+34 -3
View File
@@ -45,7 +45,10 @@ use windmill_common::otel_oss::{
use windmill_common::{
agent_workers::DECODED_AGENT_TOKEN,
apps::APP_WORKSPACED_ROUTE,
auth::{create_token_for_owner, ephemeral_script_token_label, job_token_expiry_secs},
auth::{
create_token_for_owner, ephemeral_script_token_label, job_token_expiry_secs,
TOKEN_EXPIRY_WARNING_DAYS,
},
ee_oss::CriticalErrorChannel,
email_oss::send_email_if_possible,
error,
@@ -62,6 +65,7 @@ use windmill_common::{
FORK_WORKSPACE_TAG_APPEND_FORK_SUFFIX_SETTING, HUB_API_SECRET_SETTING,
HUB_BASE_URL_SETTING, INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING,
JOB_ISOLATION_SETTING, JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING,
MCP_DISABLE_TOKEN_QUERY_PARAM, MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NPMRC_SETTING, NPM_CONFIG_REGISTRY_SETTING,
NSJAIL_TMPFS_SIZE_MB_SETTING, NSJAIL_TMP_BACKING_SETTING, NUGET_CONFIG_SETTING,
OTEL_SETTING, OTEL_TRACES_RETENTION_SECS_SETTING, OTEL_TRACING_PROXY_SETTING,
@@ -285,6 +289,15 @@ pub async fn initial_load(
);
if let Some(db) = conn.as_sql() {
// Outside the `server_mode` block below: a `MODE=mcp` process serves the MCP routes
// with `server_mode` false and would otherwise never read this at all. That mode
// joins no monitor loop, so there — as for every global setting, `base_url`
// included — this pass is the only read, and a change lands on restart.
pass.setting(
MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING,
false,
|v| async move { apply_mcp_disable_token_query_param(v) },
);
pass.setting(DEFAULT_TAGS_PER_WORKSPACE_SETTING, false, |v| async move {
apply_tag_per_workspace_enabled(v)
});
@@ -1614,6 +1627,23 @@ pub fn apply_disable_password_login(value: Option<serde_json::Value>) {
};
}
pub async fn load_mcp_disable_token_query_param(db: &DB) {
match load_value_from_global_settings(db, MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING).await {
Ok(v) => apply_mcp_disable_token_query_param(v),
Err(e) => tracing::error!("Error loading mcp_disable_token_query_param setting: {e:#}"),
};
}
pub fn apply_mcp_disable_token_query_param(value: Option<serde_json::Value>) {
match value {
Some(serde_json::Value::Bool(t)) => {
MCP_DISABLE_TOKEN_QUERY_PARAM.store(t, Ordering::Relaxed)
}
None => MCP_DISABLE_TOKEN_QUERY_PARAM.store(false, Ordering::Relaxed),
_ => (),
};
}
struct LogFile {
file_path: String,
hostname: String,
@@ -2198,7 +2228,7 @@ async fn cleanup_scheduled_job_deletions(db: &Pool<Postgres>) {
}
pub async fn check_expiring_tokens(db: &DB) {
// Find tokens expiring within 7 days that still have a pending notification row.
// Find tokens expiring within the warning window that still have a pending notification row.
// The notification table stores token_hash (not plaintext) so the join works
// even after the hash migration makes token.token nullable.
let expiring_tokens_r = sqlx::query_as!(
@@ -2207,8 +2237,9 @@ pub async fn check_expiring_tokens(db: &DB) {
USING token t
WHERE n.token_hash = t.token_hash
AND n.expiration > now()
AND n.expiration <= now() + interval '7 days'
AND n.expiration <= now() + make_interval(days => $1)
RETURNING t.token_prefix, t.label, t.email, t.workspace_id",
TOKEN_EXPIRY_WARNING_DAYS,
)
.fetch_all(db)
.await;
+1 -1
View File
@@ -234,7 +234,7 @@ workspace_protection_rule: workspace_id(char), name(char), rules(int), bypass_gr
FK: (workspace_id) -> workspace(id)
workspace_runnable_dependencies: flow_path(char), runnable_path(char), script_hash(bigint), runnable_is_flow(bool), workspace_id(char), app_path(char), id(bigint), runnable_is_agent(bool)
FK: (app_path, workspace_id) -> app(path, workspace_id) | (flow_path, workspace_id) -> flow(path, workspace_id)
workspace_settings: workspace_id(char), slack_team_id(char), slack_name(char), slack_command_script(char), slack_email(char), customer_id(char), plan(char), webhook(text), ai_config(jsonb), large_file_storage(jsonb), git_sync(jsonb), default_app(char), default_scripts(jsonb), deploy_ui(jsonb), mute_critical_alerts(bool), color(char), operator_settings(jsonb), teams_command_script(text), teams_team_id(text), teams_team_name(text), git_app_installations(jsonb), ducklake(jsonb), slack_oauth_client_id(char), slack_oauth_client_secret(char), datatable(jsonb), teams_team_guid(text), auto_invite(jsonb), error_handler(jsonb), success_handler(jsonb), public_app_execution_limit_per_minute(int), dbt_warehouses(jsonb), guest_access_enabled(bool), guest_jwt_public_key(text), guest_jwt_jwks_url(text), ai_sessions_backup_generation(int)
workspace_settings: workspace_id(char), slack_team_id(char), slack_name(char), slack_command_script(char), slack_email(char), customer_id(char), plan(char), webhook(text), ai_config(jsonb), large_file_storage(jsonb), git_sync(jsonb), default_app(char), default_scripts(jsonb), deploy_ui(jsonb), mute_critical_alerts(bool), color(char), operator_settings(jsonb), teams_command_script(text), teams_team_id(text), teams_team_name(text), git_app_installations(jsonb), ducklake(jsonb), slack_oauth_client_id(char), slack_oauth_client_secret(char), datatable(jsonb), teams_team_guid(text), auto_invite(jsonb), error_handler(jsonb), success_handler(jsonb), public_app_execution_limit_per_minute(int), dbt_warehouses(jsonb), guest_access_enabled(bool), guest_jwt_public_key(text), guest_jwt_jwks_url(text), ai_sessions_backup_generation(int), add_admins_and_developers_to_forks(bool)
FK: (workspace_id) -> workspace(id)
zombie_job_counter: job_id(uuid), counter(int)
FK: (job_id) -> v2_job(id)
+57
View File
@@ -3312,3 +3312,60 @@ export function main(i: number) {
Ok(())
}
// A `$flow_expr[...]` step tag is resolved from the flow's state before the step is pushed, and
// one that cannot be resolved fails the step instead of queueing it on a tag no worker serves.
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_flow_expr_step_tag(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let step = |id: &str, tag: Option<&str>| {
flow_module(
id,
FlowModuleValue::RawScript {
input_transforms: Default::default(),
language: ScriptLang::Deno,
content: "export function main() { return { lang: 'bun' } }".to_string(),
path: None,
lock: None,
tag: tag.map(str::to_string),
concurrency_settings: Default::default(),
is_trigger: None,
assets: None,
},
)
};
let flow = FlowValue {
modules: vec![
step("a", None),
step("b", Some("$flow_expr[results.a.lang]")),
step("c", Some("nobody-serves-$flow_expr[a.lang]")),
],
same_worker: false,
..Default::default()
};
let job = RunJob::from(JobPayload::RawFlow { value: flow, path: None, restarted_from: None })
.run_until_complete(&db, false, server.addr.port())
.await;
let b_tag = sqlx::query_scalar::<_, String>(
"SELECT tag FROM v2_job WHERE parent_job = $1 AND flow_step_id = 'b'",
)
.bind(job.id)
.fetch_one(&db)
.await?;
assert_eq!(b_tag, "bun");
assert!(!job.success);
let result = job.json_result().unwrap();
let message = result["error"]["message"].as_str().unwrap_or_default();
assert!(
message.contains("Could not resolve the step tag `nobody-serves-$flow_expr[a.lang]`"),
"got {result:?}"
);
Ok(())
}
+25
View File
@@ -1530,6 +1530,31 @@ async fn declarative_sync_rejects_an_unusable_instance_banner(db: Pool<Postgres>
);
}
#[sqlx::test(fixtures("base"))]
async fn declarative_sync_rejects_a_malformed_max_token_expiration(db: Pool<Postgres>) {
clear_settings_and_configs(&db).await;
let before = count_global_settings(&db).await;
let mut desired = BTreeMap::new();
desired.insert(
"max_token_expiration_days".to_string(),
serde_json::json!("7.0"),
);
let err = windmill_common::instance_config::sync_global_settings_declarative(
&db,
&BTreeMap::new(),
&desired,
)
.await
.expect_err("a ceiling the token routes cannot read must fail the sync");
assert!(
err.to_string().contains("max_token_expiration_days"),
"the error should name the offending setting, got: {err}"
);
assert_eq!(count_global_settings(&db).await, before);
}
#[sqlx::test(fixtures("base"))]
async fn declarative_sync_rejects_an_unusable_default_allowed_origins(db: Pool<Postgres>) {
// The declarative writers (the sync-config CLI, the operator's ConfigMap
+108
View File
@@ -0,0 +1,108 @@
/*
* The job's own token (`$WM_TOKEN`) stays valid well past the job it was minted
* for, and job logs are persisted to `job_logs` and optionally to object storage,
* so a script that echoes the token would otherwise park a live credential in
* durable storage. `run_worker` registers the token with `sensitive_log_masks`
* for the job it pulled; this pins that the persisted log carries the masked form.
*/
use sqlx::{Pool, Postgres};
use windmill_common::{
jobs::{JobPayload, RawCode},
scripts::ScriptLang,
};
use windmill_test_utils::*;
/// Prefix of a serialized job token: `jwt_` plus the base64 of a JWT header.
/// The masked form keeps only `jwt` + the last three characters, so it never matches.
const RAW_TOKEN_PREFIX: &str = "jwt_ey";
#[sqlx::test(fixtures("base"))]
async fn test_job_token_masked_in_persisted_logs(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let job = RunJob::from(JobPayload::Code(RawCode {
hash: None,
content: "echo \"running with --token $WM_TOKEN\"".to_string(),
path: None,
lock: None,
language: ScriptLang::Bash,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
tag: None,
}))
.run_until_complete(&db, false, port)
.await;
assert!(job.success, "job should have succeeded");
let logs =
sqlx::query_scalar::<_, Option<String>>("SELECT logs FROM job_logs WHERE job_id = $1")
.bind(job.id)
.fetch_one(&db)
.await?
.unwrap_or_default();
assert!(
!logs.contains(RAW_TOKEN_PREFIX),
"an unmasked job token reached the persisted logs: {logs}"
);
assert!(
logs.contains("secret value was masked"),
"expected the masking notice in logs: {logs}"
);
Ok(())
}
/// nativets runs V8 in-process and persists `console.log` output through its own
/// channel, so it is masked by a different mechanism than the bash case above and
/// needs its own guard.
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_job_token_masked_in_nativets_logs(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let job = RunJob::from(JobPayload::Code(RawCode {
hash: None,
content: "export async function main() {\n console.log('running with --token ' + process.env.WM_TOKEN);\n return 'ok';\n}".to_string(),
path: None,
lock: None,
language: ScriptLang::Nativets,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
tag: None,
}))
.run_until_complete(&db, false, port)
.await;
assert!(job.success, "job should have succeeded");
let logs =
sqlx::query_scalar::<_, Option<String>>("SELECT logs FROM job_logs WHERE job_id = $1")
.bind(job.id)
.fetch_one(&db)
.await?
.unwrap_or_default();
assert!(
!logs.contains(RAW_TOKEN_PREFIX),
"an unmasked job token reached the persisted logs: {logs}"
);
assert!(
logs.contains("secret value was masked"),
"expected the masking notice in logs: {logs}"
);
Ok(())
}
+4
View File
@@ -1249,6 +1249,10 @@ pub async fn register_token_expiry_notification(
if !windmill_common::auth::is_user_token(label) {
return;
}
let warning_days = windmill_common::auth::TOKEN_EXPIRY_WARNING_DAYS;
if expiration <= chrono::Utc::now() + chrono::Duration::days(warning_days.into()) {
return;
}
if let Err(e) = sqlx::query!(
"INSERT INTO token_expiry_notification (token_hash, expiration) VALUES ($1, $2) ON CONFLICT DO NOTHING",
token_hash,
@@ -1,108 +0,0 @@
//! Who may read and change a data table's grants and owners. On the Enterprise Edition: its
//! administrators, from the workspace that governs it. Without it: nobody. Each refusal is decided
//! before anything connects to the data table, so the fixture's database never has to exist.
use serde_json::{json, Value};
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
fn grant_select_on_public() -> Value {
json!({
"target": {"kind": "schema", "schema": "public"},
"change": {"type": "grant", "role": "analytics", "privileges": ["SELECT"],
"scope": "all_tables"},
"statements": [r#"GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO "analytics""#]
})
}
async fn post_acl(
port: u16,
w_id: &str,
action: &str,
token: &str,
) -> anyhow::Result<reqwest::Response> {
Ok(reqwest::Client::new()
.post(format!(
"http://localhost:{port}/api/w/{w_id}/workspaces/datatable_acl/main/{action}"
))
.header("Authorization", format!("Bearer {token}"))
.json(&grant_select_on_public())
.send()
.await?)
}
/// A fork reaches the data table through a pointer: it may use it, never change what each role may
/// touch on it — not even as an admin of the fork.
#[cfg(all(feature = "private", feature = "enterprise"))]
#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
async fn a_fork_cannot_change_access_on_the_data_table_it_points_at(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
for action in ["plan", "apply"] {
let resp = post_acl(port, "wm-fork-dt", action, "SECRET_TOKEN_2").await?;
assert_eq!(resp.status(), 401, "{action}: {}", resp.text().await?);
}
Ok(())
}
#[cfg(all(feature = "private", feature = "enterprise"))]
#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
async fn a_member_who_is_not_an_admin_cannot_change_access(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
for action in ["plan", "apply"] {
let resp = post_acl(port, "test-workspace", action, "SECRET_TOKEN_2").await?;
assert_eq!(resp.status(), 401, "{action}: {}", resp.text().await?);
}
Ok(())
}
/// Not even reading, and not even on a data table that is not under roles — which any member
/// reaches, so only the edition stands between them and the instance's credentials.
#[cfg(not(all(feature = "private", feature = "enterprise")))]
#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
async fn only_the_enterprise_edition_has_the_access_editor(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query(
"UPDATE workspace_settings
SET datatable = datatable #- '{datatables,main,permissions}'
WHERE workspace_id = 'test-workspace'",
)
.execute(&db)
.await?;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let read = reqwest::Client::new()
.get(format!(
"http://localhost:{port}/api/w/test-workspace/workspaces/datatable_acl/main?kind=database"
))
.header("Authorization", "Bearer SECRET_TOKEN")
.send()
.await?;
let mut responses = vec![("read", read)];
for action in ["plan", "apply"] {
responses.push((
action,
post_acl(port, "test-workspace", action, "SECRET_TOKEN").await?,
));
}
for (action, resp) in responses {
assert_eq!(resp.status(), 400, "{action}");
let body = resp.text().await?;
assert!(
body.contains("Data table roles are a Windmill Enterprise Edition feature"),
"{action}: {body}"
);
}
Ok(())
}
@@ -912,18 +912,6 @@ async fn a_stored_name_containing_a_question_mark_resolves_as_itself(
resolve("main?dt").await.is_err(),
"an unknown parameter was ignored"
);
sqlx::query(
"UPDATE workspace_settings
SET datatable = jsonb_set(datatable, '{datatables,main?role=analytics}', datatable->'datatables'->'main')
WHERE workspace_id = 'test-workspace'",
)
.execute(&db)
.await?;
assert!(
resolve("main?role=analytics").await.is_err(),
"a reference naming both a stored data table and a role on another resolved to one of them"
);
Ok(())
}
@@ -1021,70 +1009,6 @@ async fn an_entry_without_roles_cannot_newly_reach_a_database_under_roles(
Ok(())
}
/// Browsing names the role it connects as, and a role the caller may not use is refused rather
/// than quietly listed as the default. The refusal is decided before connecting, so the fixture's
/// database never has to exist.
#[cfg(all(feature = "private", feature = "enterprise"))]
#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
async fn browsing_as_a_role_the_caller_may_not_use_is_refused(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
// `test-user-2` is a tenant of `analytics` only.
let resp = authed(
client().get(format!(
"{base}/list_datatable_tables?role_for=main&role=admin"
)),
"SECRET_TOKEN_2",
)
.send()
.await?;
assert_eq!(resp.status(), 200);
let body: Value = resp.json().await?;
let entry = body
.as_array()
.and_then(|a| a.iter().find(|e| e["datatable_name"] == "main"))
.expect("main is listed");
assert_eq!(entry["usable_roles"], json!(["analytics"]), "{entry}");
assert_eq!(entry["default_role"], "analytics", "{entry}");
assert_eq!(entry["permissioned"], true, "{entry}");
assert_eq!(entry["instance"], true, "{entry}");
let error = entry["error"].as_str().unwrap_or_default();
assert!(
error.contains("Not allowed to use role 'admin'"),
"listed as another role than the one asked for: {entry}"
);
let resp = authed(
client().get(format!(
"{base}/get_datatable_table_schema?datatable_name=main&schema_name=public&table_name=t&role=admin"
)),
"SECRET_TOKEN_2",
)
.send()
.await?;
let status = resp.status();
let text = resp.text().await?;
assert!(
text.contains("Not allowed to use role 'admin'"),
"{status}: {text}"
);
// A role means nothing without the data table it belongs to.
let resp = authed(
client().get(format!("{base}/list_datatable_tables?role=analytics")),
"SECRET_TOKEN_2",
)
.send()
.await?;
assert_eq!(resp.status(), 400, "{}", resp.text().await?);
Ok(())
}
#[sqlx::test(migrations = "../migrations", fixtures("base", "datatable_roles"))]
async fn an_alias_saved_elsewhere_waits_for_roles_going_on_for_its_database(
db: Pool<Postgres>,
@@ -311,3 +311,121 @@ async fn test_fork_keeps_only_resolvable_on_behalf_of(db: Pool<Postgres>) -> any
Ok(())
}
/// Apps, schedules, triggers and their drafts cannot drop an identity the way scripts and flows
/// do, so one naming nobody in the fork goes to its creator while one that still resolves stays.
/// Forked as an admin, whose app policies the clone otherwise keeps.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_fork_repoints_unresolvable_identities(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let base_url = format!("http://localhost:{}/api", server.addr.port());
let stranger = json!({
"on_behalf_of": "u/test-user-2",
"on_behalf_of_email": "test2@windmill.dev",
"execution_mode": "publisher",
});
sqlx::query(
"INSERT INTO app (workspace_id, path, summary, policy, versions)
VALUES ('test-workspace', 'u/test-user/stranger', '', $1, '{}'),
('test-workspace', 'u/test-user/group', '', $2, '{}')",
)
.bind(&stranger)
.bind(json!({
"on_behalf_of": "g/all",
"on_behalf_of_email": "group-all@windmill.dev",
"execution_mode": "publisher",
}))
.execute(&db)
.await?;
// The clone re-aggregates `versions` from `app_version`, and the column is NOT NULL.
sqlx::query(
"WITH v AS (
INSERT INTO app_version (app_id, value, created_by)
SELECT id, '{}'::json, 'test-user' FROM app WHERE workspace_id = 'test-workspace'
RETURNING id, app_id
)
UPDATE app SET versions = ARRAY[v.id] FROM v WHERE app.id = v.app_id",
)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO draft (workspace_id, path, typ, value, created_at, email)
VALUES ('test-workspace', 'u/test-user/stranger', 'raw_app', $1::json, NOW(), 'test@windmill.dev'),
('test-workspace', 'u/test-user/stranger', 'trigger_websocket', $2::json, NOW(), 'test@windmill.dev'),
('test-workspace', 'u/test-user/nul', 'raw_app', $3::json, NOW(), 'test@windmill.dev')",
)
.bind(json!({ "policy": stranger }))
.bind(json!({ "permissioned_as": "u/test-user-2" }))
// Saved before drafts were stripped of NULs: any jsonb parse of it raises, so it must be
// skipped rather than abort the fork. Built from parts because a NUL escape can't sit in source.
.bind(format!(
r#"{{"policy":{{"on_behalf_of":"u/test-user-2"}},"files":{{"f":"a{}u0000"}}}}"#,
"\\"
))
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO schedule (workspace_id, path, edited_by, schedule, script_path, email, permissioned_as, enabled)
VALUES ('test-workspace', 'u/test-user/stranger', 'test-user', '0 0 * * * *', 'u/test-user/s', 'test2@windmill.dev', 'u/test-user-2', false)",
)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO websocket_trigger (workspace_id, path, url, script_path, is_flow, edited_by, permissioned_as, mode)
VALUES ('test-workspace', 'u/test-user/stranger', 'ws://localhost', 'u/test-user/s', false, 'test-user', 'u/test-user-2', 'disabled')",
)
.execute(&db)
.await?;
let resp = reqwest::Client::new()
.post(format!(
"{base_url}/w/test-workspace/workspaces/create_fork"
))
.header("Authorization", "Bearer SECRET_TOKEN")
.json(&json!({ "id": "wm-fork-repoint", "name": "Fork", "color": "#0000ff" }))
.send()
.await?;
assert!(
resp.status().is_success(),
"creating the fork: {}",
resp.text().await?
);
let text = |sql: &'static str| sqlx::query_scalar::<_, String>(sql).fetch_one(&db);
assert_eq!(
text("SELECT (policy->>'on_behalf_of') || ' ' || (policy->>'on_behalf_of_email') FROM app WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/stranger'").await?,
"u/test-user test@windmill.dev"
);
assert_eq!(
text("SELECT policy->>'on_behalf_of' FROM app WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/group'").await?,
"g/all"
);
assert_eq!(
text("SELECT value->'policy'->>'on_behalf_of' FROM draft WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/stranger' AND typ = 'raw_app'").await?,
"u/test-user"
);
assert_eq!(
text("SELECT CASE WHEN strpos(value::text, 'u/test-user-2') > 0 THEN 'kept' ELSE 'rewritten' END FROM draft WHERE workspace_id = 'wm-fork-repoint' AND path = 'u/test-user/nul'").await?,
"kept"
);
assert_eq!(
text("SELECT value->>'permissioned_as' FROM draft WHERE workspace_id = 'wm-fork-repoint' AND typ = 'trigger_websocket'").await?,
"u/test-user"
);
assert_eq!(
text("SELECT permissioned_as || ' ' || email FROM schedule WHERE workspace_id = 'wm-fork-repoint'").await?,
"u/test-user test@windmill.dev"
);
assert_eq!(
text(
"SELECT permissioned_as FROM websocket_trigger WHERE workspace_id = 'wm-fork-repoint'"
)
.await?,
"u/test-user"
);
Ok(())
}
@@ -0,0 +1,75 @@
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
/// With `add_admins_and_developers_to_forks` on, a fork starts with the parent's admins and
/// developers at their parent role, even when a developer forks it; operators are left out. The
/// copies are manual members: a parent membership that came from an instance group must not carry
/// that provenance into a fork that does not configure the group.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_fork_adds_parent_admins_and_developers(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let base_url = format!(
"http://localhost:{}/api/w/test-workspace/workspaces",
server.addr.port()
);
let client = reqwest::Client::new();
sqlx::query(
"UPDATE usr SET operator = true WHERE workspace_id = 'test-workspace' AND username = 'test-user-3'",
)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO usr (workspace_id, email, username, is_admin, added_via)
VALUES ('test-workspace', 'test4@windmill.dev', 'test-user-4', false,
'{\"source\": \"instance_group\", \"group\": \"devs\"}')",
)
.execute(&db)
.await?;
let resp = client
.post(format!(
"{base_url}/edit_add_admins_and_developers_to_forks"
))
.header("Authorization", "Bearer SECRET_TOKEN")
.json(&json!({ "add_admins_and_developers_to_forks": true }))
.send()
.await?;
assert!(
resp.status().is_success(),
"enabling the setting: {}",
resp.text().await?
);
let resp = client
.post(format!("{base_url}/create_fork"))
.header("Authorization", "Bearer SECRET_TOKEN_2")
.json(&json!({ "id": "wm-fork-team", "name": "Team fork" }))
.send()
.await?;
assert!(
resp.status().is_success(),
"creating the fork: {}",
resp.text().await?
);
let members: Vec<(String, bool, bool)> = sqlx::query_as(
"SELECT username, is_admin, added_via IS NULL FROM usr
WHERE workspace_id = 'wm-fork-team' ORDER BY username",
)
.fetch_all(&db)
.await?;
assert_eq!(
members,
vec![
("test-user".to_string(), true, true),
("test-user-2".to_string(), false, true),
("test-user-4".to_string(), false, true),
]
);
Ok(())
}
@@ -115,6 +115,60 @@ async fn login_link_is_single_use_and_same_origin(db: Pool<Postgres>) -> anyhow:
Ok(())
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn confirmed_login_link_is_spent_by_the_click_not_the_page(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api");
let resp = client()
.post(format!("{base}/users/login_links"))
.header("Authorization", "Bearer SECRET_TOKEN")
.json(&json!({"email": "test2@windmill.dev", "confirm": true}))
.send()
.await?;
assert_eq!(resp.status(), 201);
let link = resp.json::<serde_json::Value>().await?;
// The URL handed out is the frontend page, not the API path that signs in on a GET.
let token = link["url"]
.as_str()
.unwrap()
.split_once("/user/login_link?token=")
.expect("confirmation page url")
.1
.to_string();
let confirm = || {
client()
.post(format!("{base}/auth/login_link/{token}"))
.send()
};
let resp = confirm().await?;
assert_eq!(resp.status(), 200);
assert!(resp
.headers()
.get_all("set-cookie")
.iter()
.any(|c| c.to_str().unwrap().starts_with("token=")));
assert_eq!(
resp.json::<serde_json::Value>().await?["location"],
"/user/workspaces"
);
let resp = confirm().await?;
assert_eq!(resp.status(), 200);
assert!(resp.headers().get("set-cookie").is_none());
assert_eq!(
resp.json::<serde_json::Value>().await?["location"],
"/user/login_link_expired?reason=used"
);
Ok(())
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn login_link_mint_can_require_a_login_type(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
@@ -0,0 +1,239 @@
//! `max_token_expiration_days`: the instance-wide ceiling on how far ahead a token a caller
//! picks the lifetime of may expire, and the service-account exemption.
use serde_json::json;
use sqlx::types::chrono::{DateTime, Utc};
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
const DAY: u64 = 24 * 60 * 60;
fn client() -> reqwest::Client {
reqwest::Client::new()
}
fn from_now(secs: u64) -> DateTime<Utc> {
Utc::now() + std::time::Duration::from_secs(secs)
}
async fn set_max(db: &Pool<Postgres>, value: serde_json::Value) {
sqlx::query(
"INSERT INTO global_settings (name, value) VALUES ('max_token_expiration_days', $1)
ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
)
.bind(value)
.execute(db)
.await
.unwrap();
}
/// Mints as `test2@windmill.dev`, a plain member of `test-workspace`.
async fn create_token(port: u16, body: serde_json::Value) -> reqwest::Response {
client()
.post(format!("http://localhost:{port}/api/users/tokens/create"))
.header("Authorization", "Bearer SECRET_TOKEN_2")
.json(&body)
.send()
.await
.unwrap()
}
async fn stored_expiration(db: &Pool<Postgres>, label: &str) -> Option<DateTime<Utc>> {
sqlx::query_scalar::<_, Option<DateTime<Utc>>>("SELECT expiration FROM token WHERE label = $1")
.bind(label)
.fetch_one(db)
.await
.unwrap()
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_max_token_expiration_days_shortens_user_tokens(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let resp = create_token(port, json!({ "label": "unset" })).await;
assert_eq!(resp.status(), 201);
assert_eq!(
stored_expiration(&db, "unset").await,
None,
"with no setting a token may still have no expiration"
);
// Refused at write time: the token routes can only read a value they cannot parse as no
// ceiling at all.
let resp = client()
.post(format!(
"http://localhost:{port}/api/settings/global/max_token_expiration_days"
))
.header("Authorization", "Bearer SECRET_TOKEN")
.json(&json!({ "value": 7.5 }))
.send()
.await?;
assert_eq!(resp.status(), 400);
set_max(&db, json!(7)).await;
// The token form reads the ceiling as whoever is creating the token, usually not a
// superadmin, so it can offer only expirations the server would keep.
let resp = client()
.get(format!(
"http://localhost:{port}/api/settings/global/max_token_expiration_days"
))
.header("Authorization", "Bearer SECRET_TOKEN_2")
.send()
.await?;
assert_eq!(resp.status(), 200);
assert_eq!(resp.text().await?, "7");
let resp = create_token(port, json!({ "label": "none asked" })).await;
assert_eq!(resp.status(), 201);
let expiration = stored_expiration(&db, "none asked")
.await
.expect("a token asking for no expiration gets the ceiling");
assert!(
expiration > from_now(6 * DAY) && expiration <= from_now(7 * DAY),
"expected the 7 day ceiling, got {expiration}"
);
let resp = create_token(
port,
json!({ "label": "past the ceiling", "expiration": from_now(30 * DAY) }),
)
.await;
assert_eq!(resp.status(), 201);
let expiration = stored_expiration(&db, "past the ceiling").await.unwrap();
assert!(
expiration > from_now(6 * DAY) && expiration <= from_now(7 * DAY),
"expected an expiration past the ceiling to be shortened to it, got {expiration}"
);
let resp = create_token(
port,
json!({ "label": "within", "expiration": from_now(3 * DAY) }),
)
.await;
assert_eq!(resp.status(), 201);
let expiration = stored_expiration(&db, "within").await.unwrap();
assert!(
expiration <= from_now(3 * DAY),
"an expiration within the ceiling must be kept, got {expiration}"
);
// The settings UI stores an integer, but the YAML instance config and config sync can write
// the same whole number as a string or as `5.0`. Reading either as "unset" would silently
// drop the ceiling, while the token form (`parseMaxTokenExpirationDays`) still showed it.
for (stored, label) in [
(json!("5"), "string setting"),
(json!(5.0), "float setting"),
] {
set_max(&db, stored).await;
let resp = create_token(port, json!({ "label": label })).await;
assert_eq!(resp.status(), 201);
let expiration = stored_expiration(&db, label).await;
assert!(
expiration.is_some_and(|e| e > from_now(4 * DAY) && e <= from_now(5 * DAY)),
"{label}: expected the 5 day ceiling, got {expiration:?}"
);
}
// A superadmin impersonating a user picks the lifetime too, so the ceiling applies there;
// left out, it would be the one way to mint a token that never expires.
let resp = client()
.post(format!(
"http://localhost:{port}/api/users/tokens/impersonate"
))
.header("Authorization", "Bearer SECRET_TOKEN")
.json(&json!({ "label": "impersonated", "impersonate_email": "test3@windmill.dev" }))
.send()
.await?;
assert_eq!(resp.status(), 201);
assert!(
stored_expiration(&db, "impersonated").await.is_some(),
"an impersonation token asking for no expiration gets the ceiling"
);
Ok(())
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_service_accounts_are_exempt(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
set_max(&db, json!(7)).await;
// The same email is a service account in one workspace and an ordinary user in another.
sqlx::query(
"UPDATE usr SET is_service_account = true
WHERE email = 'test2@windmill.dev' AND workspace_id = 'test-workspace'",
)
.execute(&db)
.await?;
sqlx::query("INSERT INTO workspace (id, name, owner) VALUES ('other', 'other', 'test-user')")
.execute(&db)
.await?;
sqlx::query("INSERT INTO workspace_settings (workspace_id) VALUES ('other')")
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO usr (workspace_id, email, username, is_admin, role)
VALUES ('other', 'test2@windmill.dev', 'test-user-2', false, 'User')",
)
.execute(&db)
.await?;
// The token form decides the exemption from this list, so it must carry each membership's flag.
let memberships: serde_json::Value = client()
.get(format!("http://localhost:{port}/api/workspaces/users"))
.header("Authorization", "Bearer SECRET_TOKEN_2")
.send()
.await?
.json()
.await?;
for (workspace, is_service_account) in [("test-workspace", true), ("other", false)] {
let membership = memberships["workspaces"]
.as_array()
.and_then(|ws| ws.iter().find(|w| w["id"] == workspace))
.unwrap_or_else(|| panic!("{workspace} missing from {memberships}"));
assert_eq!(membership["is_service_account"], json!(is_service_account));
}
for (label, workspace_id, exempt) in [
("own workspace", Some("test-workspace"), true),
("other workspace", Some("other"), false),
// A workspace-less token has no workspace to match, so a service account anywhere counts.
("global", None, true),
] {
let resp = create_token(
port,
json!({ "label": label, "workspace_id": workspace_id }),
)
.await;
assert_eq!(resp.status(), 201);
assert_eq!(
stored_expiration(&db, label).await.is_none(),
exempt,
"{label}: expected exempt = {exempt}"
);
}
// Impersonation checks the impersonated account, not the superadmin minting the token.
let resp = client()
.post(format!(
"http://localhost:{port}/api/users/tokens/impersonate"
))
.header("Authorization", "Bearer SECRET_TOKEN")
.json(&json!({ "label": "impersonated service account", "impersonate_email": "test2@windmill.dev" }))
.send()
.await?;
assert_eq!(resp.status(), 201);
assert_eq!(
stored_expiration(&db, "impersonated service account").await,
None
);
Ok(())
}
@@ -0,0 +1,102 @@
//! The `mcp_disable_token_query_param` switch closes the URL-borne credential path.
//!
//! The rejection is a middleware layered between the `WWW-Authenticate` decorator and
//! everything that reads a token, on both the workspaced and the gateway mount. Each half of
//! that sandwich is pinned: the `WWW-Authenticate` header on the refusal catches the layer
//! being moved outward (a client would lose the pointer that starts OAuth discovery), and
//! refusing a token that was never valid catches it being moved inward past authentication
//! (the URL-borne token would be hashed and looked up before anything refused it).
#![cfg(feature = "mcp")]
use std::sync::atomic::Ordering;
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_common::global_settings::MCP_DISABLE_TOKEN_QUERY_PARAM;
use windmill_test_utils::*;
/// Workspace-less with an `mcp:` scope, which is what the gateway mount requires; the
/// workspaced mount takes its workspace from the path, so one token reaches both.
async fn insert_mcp_token(db: &Pool<Postgres>) -> anyhow::Result<()> {
sqlx::query(
"INSERT INTO token (token_hash, token_prefix, token, email, label, super_admin, scopes)
VALUES (encode(sha256('MCP_TOKEN'::bytea), 'hex'), 'MCP_TOK', 'MCP_TOKEN', 'test@windmill.dev', 'mcp token', true, ARRAY['mcp:all'])",
)
.execute(db)
.await?;
Ok(())
}
/// A token that is not in `token` at all. Authentication would refuse it on its own, so a
/// refusal carrying the middleware's own wording is evidence nothing looked it up first.
const BOGUS_TOKEN: &str = "NOT_A_REAL_TOKEN";
async fn tools_list(url: &str) -> anyhow::Result<reqwest::Response> {
Ok(reqwest::Client::new()
.post(url)
.header("Accept", "application/json, text/event-stream")
.json(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {} }))
.send()
.await?)
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_mcp_token_query_param_switch(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
set_jwt_secret().await;
insert_mcp_token(&db).await?;
let server = ApiServer::start_mcp(db.clone()).await?;
let port = server.addr.port();
let workspaced =
format!("http://localhost:{port}/api/mcp/w/test-workspace/mcp?token=MCP_TOKEN");
let gateway = format!("http://localhost:{port}/api/mcp/gateway?token=MCP_TOKEN");
assert_eq!(
tools_list(&workspaced).await?.status(),
200,
"a URL-borne token is the documented default and must keep working while the switch is off"
);
assert_eq!(tools_list(&gateway).await?.status(), 200);
MCP_DISABLE_TOKEN_QUERY_PARAM.store(true, Ordering::Relaxed);
for url in [&workspaced, &gateway] {
let resp = tools_list(url).await?;
assert_eq!(
resp.status(),
401,
"{url} still admitted a token in the URL"
);
// What sends the client into the OAuth flow rather than leaving it stuck on a 401.
assert!(
resp.headers().contains_key("www-authenticate"),
"{url} rejected without pointing at the authorization server"
);
}
// Refused before authentication, not after: an invalid token gets the middleware's own
// message rather than the generic 401 that looking it up would produce.
let resp = tools_list(&format!(
"http://localhost:{port}/api/mcp/w/test-workspace/mcp?token={BOGUS_TOKEN}"
))
.await?;
assert_eq!(resp.status(), 401);
assert!(
resp.text().await?.contains("does not accept a token in the MCP URL"),
"an invalid URL token was answered by authentication, so the token was read before \
the switch refused it"
);
// The header stays open: it is the channel the OAuth flow itself hands tokens over on.
let resp = reqwest::Client::new()
.post(format!("http://localhost:{port}/api/mcp/gateway"))
.header("Accept", "application/json, text/event-stream")
.header("Authorization", "Bearer MCP_TOKEN")
.json(&json!({ "jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {} }))
.send()
.await?;
assert_eq!(resp.status(), 200);
Ok(())
}
@@ -0,0 +1,49 @@
//! Which user tokens get an "expiring soon" warning queued when they are created.
use serde_json::json;
use sqlx::types::chrono::Utc;
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
const DAY: u64 = 24 * 60 * 60;
async fn warning_queued(db: &Pool<Postgres>, label: &str) -> bool {
sqlx::query_scalar::<_, bool>(
"SELECT EXISTS(SELECT 1 FROM token_expiry_notification n
JOIN token t ON t.token_hash = n.token_hash WHERE t.label = $1)",
)
.bind(label)
.fetch_one(db)
.await
.unwrap()
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_short_lived_tokens_get_no_expiry_warning(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
for (label, lifetime) in [("short", DAY), ("long", 30 * DAY)] {
let resp = reqwest::Client::new()
.post(format!("http://localhost:{port}/api/users/tokens/create"))
.header("Authorization", "Bearer SECRET_TOKEN_2")
.json(&json!({
"label": label,
"expiration": Utc::now() + std::time::Duration::from_secs(lifetime),
}))
.send()
.await?;
assert_eq!(resp.status(), 201);
}
assert!(
!warning_queued(&db, "short").await,
"a token whose whole lifetime fits in the warning window must not be warned about"
);
assert!(
warning_queued(&db, "long").await,
"a longer-lived token still gets its warning"
);
Ok(())
}
@@ -275,6 +275,67 @@ async fn test_encryption_key_rotation_dispatches_batched_git_sync(
Ok(())
}
/// Stored repository tokens and webhook secrets are encrypted under the
/// workspace key but never synced, so a rotation has to carry them over even
/// when the caller skips re-encrypting variables.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_encryption_key_rotation_reencrypts_git_sync_secrets(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_common::variables::{build_crypt, crypt_from_key_with_suffix, decrypt, encrypt};
initialize_tracing().await;
create_folder(&db, "28103").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28103/test_sync_script_git_secrets";
create_sync_script(&db, sync_script_path).await?;
setup_git_sync_config(&db, sync_script_path).await?;
let mc = build_crypt(&db, "test-workspace").await?;
sqlx::query(
r#"
UPDATE workspace_settings SET
git_credentials = jsonb_build_array(jsonb_build_object(
'token', $1::text, 'repo_identity', 'https://gitlab.example.com/grp/proj')),
git_sync = jsonb_set(git_sync, '{repositories,0,auto_pull}', jsonb_build_object(
'enabled', true, 'mode', 'webhook', 'webhook_id', 1, 'webhook_secret', $2::text))
WHERE workspace_id = 'test-workspace'
"#,
)
.bind(encrypt(&mc, "stored-token"))
.bind(encrypt(&mc, "hook-secret"))
.execute(&db)
.await?;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
let new_key = "c".repeat(64);
let resp = authed(client().post(format!("{base}/encryption_key")))
.json(&json!({"new_key": new_key, "skip_reencrypt": true}))
.send()
.await?;
assert_eq!(
resp.status(),
200,
"set_encryption_key failed: {}",
resp.text().await?
);
let (token, secret): (String, String) = sqlx::query_as(
"SELECT git_credentials->0->>'token', git_sync#>>'{repositories,0,auto_pull,webhook_secret}'
FROM workspace_settings WHERE workspace_id = 'test-workspace'",
)
.fetch_one(&db)
.await?;
let new_mc = crypt_from_key_with_suffix(&new_key, "");
assert_eq!(decrypt(&new_mc, token)?, "stored-token");
assert_eq!(decrypt(&new_mc, secret)?, "hook-secret");
Ok(())
}
/// Regression test for the non-debouncing fallback: a workspace whose sync
/// script predates hub version 28103 must still receive git-sync jobs for the
/// encryption_key entry and every re-encrypted secret. Before the fallback was
@@ -1166,3 +1166,47 @@ async fn test_create_service_account_drops_orphaned_group_memberships(
Ok(())
}
#[cfg(feature = "private")]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_edit_auto_invite_preserves_instance_groups(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query(
r#"UPDATE workspace_settings
SET auto_invite = '{"instance_groups": ["eng"], "instance_groups_roles": {"eng": "developer"}}'
WHERE workspace_id = 'test-workspace'"#,
)
.execute(&db)
.await?;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
// enable, then disable
for body in [
json!({"operator": false, "invite_all": true, "auto_add": false}),
json!({}),
] {
let resp = authed(client().post(format!("{base}/edit_auto_invite")))
.json(&body)
.send()
.await?;
assert_eq!(resp.status(), 200, "{body}: {}", resp.text().await?);
let auto_invite: serde_json::Value = sqlx::query_scalar(
"SELECT auto_invite FROM workspace_settings WHERE workspace_id = 'test-workspace'",
)
.fetch_one(&db)
.await?;
assert_eq!(auto_invite["instance_groups"], json!(["eng"]), "{body}");
assert_eq!(
auto_invite["instance_groups_roles"],
json!({"eng": "developer"}),
"{body}"
);
}
Ok(())
}
+25 -201
View File
@@ -42,6 +42,7 @@ use axum::{
routing::{get, post},
Json, Router,
};
use serde_json::json;
use serde::{Deserialize, Serialize};
use windmill_ai::ai_cache::bump_instance_ai_config_revision;
@@ -59,10 +60,11 @@ use windmill_common::{
global_settings::{
AI_CONFIG_SETTING, APP_WORKSPACED_ROUTE_SETTING, AUTOMATE_USERNAME_CREATION_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING,
DISABLE_HUB_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS, EXTERNAL_INSTANCE_PG_SETTING,
DISABLE_HUB_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
GITHUB_APP_WEBHOOK_BASE_URL_SETTING, HTTP_ROUTE_DEFAULT_ALLOWED_ORIGINS_SETTING,
HTTP_ROUTE_WORKSPACED_ROUTE_SETTING, HUB_ACCESSIBLE_URL_SETTING, HUB_BASE_URL_SETTING,
INSTANCE_BANNER_SETTING, MAX_RETENTION_OVERRIDE_WORKSPACES,
MAX_TOKEN_EXPIRATION_DAYS_SETTING, MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING,
RETENTION_PERIOD_SECS_OVERRIDES_SETTING, RUFF_CONFIG_SETTING, UNIQUE_ID_SETTING,
WORKSPACE_FAIRNESS_DURATION_SECS_SETTING, WORKSPACE_FAIRNESS_ENABLED_SETTING,
WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING, WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING,
@@ -167,22 +169,6 @@ pub fn global_service() -> Router {
"/refresh_custom_instance_user_pwd",
post(refresh_custom_instance_user_pwd),
)
.route(
"/external_instance_pg/status",
get(get_external_instance_pg_status),
)
.route(
"/external_instance_pg/setup",
post(setup_external_instance_pg),
)
.route(
"/external_instance_pg/databases",
get(list_external_instance_pg_databases),
)
.route(
"/external_instance_pg/databases/{name}",
post(create_external_instance_pg_database).delete(drop_external_instance_pg_database),
)
.route(
"/setup_custom_instance_pg_database/{name}",
post(setup_custom_instance_pg_database),
@@ -892,14 +878,6 @@ pub async fn set_global_setting_internal(
)));
}
if key == EXTERNAL_INSTANCE_PG_SETTING {
return windmill_common::external_instance_pg::write_external_instance_pg_setting(
db,
Some(&value),
)
.await;
}
run_setting_pre_write_hook(db, &key, &value).await?;
match value {
@@ -1218,6 +1196,12 @@ async fn run_setting_pre_write_hook(
}
}
}
MAX_TOKEN_EXPIRATION_DAYS_SETTING => {
windmill_common::global_settings::parse_max_token_expiration_days(Some(value))
.map_err(|e| {
error::Error::BadRequest(format!("{MAX_TOKEN_EXPIRATION_DAYS_SETTING}: {e}"))
})?;
}
INSTANCE_BANNER_SETTING => {
match value {
// Clearing (delete row) is handled by the caller; allow it through.
@@ -1281,7 +1265,7 @@ async fn set_instance_config(
let desired_map = desired.global_settings.to_settings_map();
if !desired_map.is_empty() {
let current_map = current.global_settings.to_settings_map();
let mut settings_diff =
let settings_diff =
instance_config::diff_global_settings(&current_map, &desired_map, ApplyMode::Merge);
let ai_config_changed = settings_diff
.upserts
@@ -1310,15 +1294,8 @@ async fn set_instance_config(
}
for (key, value) in &settings_diff.upserts {
if key != EXTERNAL_INSTANCE_PG_SETTING {
run_setting_pre_write_hook(&db, key, value).await?;
}
run_setting_pre_write_hook(&db, key, value).await?;
}
windmill_common::external_instance_pg::write_external_instance_pg_from_diff(
&db,
&mut settings_diff,
)
.await?;
instance_config::apply_settings_diff(&db, &settings_diff)
.await
@@ -1386,6 +1363,12 @@ pub async fn get_global_setting(
&& key != HTTP_ROUTE_DEFAULT_ALLOWED_ORIGINS_SETTING
&& key != WS_BASE_URL_SETTING
&& key != INSTANCE_BANNER_SETTING
// The token form reads it to stop offering expirations the server would shorten.
&& key != MAX_TOKEN_EXPIRATION_DAYS_SETTING
// Whoever is wiring up an MCP client reads it to know whether a URL-borne token
// would be refused, and they are usually not a superadmin. Not a secret: pointing
// any MCP client at the instance discovers the same answer.
&& key != MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING
{
require_super_admin(&db, &authed).await?;
}
@@ -1677,8 +1660,6 @@ struct CustomInstanceDb {
tag: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
used_by_workspaces: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
workspace_id: Option<String>,
}
#[derive(Deserialize, Debug, Serialize, Default)]
@@ -1719,15 +1700,7 @@ async fn list_custom_instance_pg_databases(
))
})?;
if !windmill_api_auth::is_super_admin_authed(&db, &authed).await? {
// Which workspace reserved a fork copy is nobody else's business: it would enumerate every
// pending fork on the instance.
for entry in result.values_mut() {
entry.workspace_id = None;
}
return Ok(Json(result));
}
{
if windmill_api_auth::is_super_admin_authed(&db, &authed).await? {
// Enrich each database with the list of workspaces referencing it through
// either a ducklake catalog or a datatable database whose resource_type is
// 'instance'. Not stored in DB to avoid drift.
@@ -1783,135 +1756,6 @@ async fn refresh_custom_instance_user_pwd(
Ok(Json(()))
}
async fn get_external_instance_pg_status(
authed: ApiAuthed,
Extension(db): Extension<DB>,
) -> JsonResult<windmill_common::external_instance_pg::ExternalInstancePgStatus> {
require_super_admin(&db, &authed).await?;
Ok(Json(
windmill_common::external_instance_pg::external_instance_pg_status(&db).await?,
))
}
#[derive(Deserialize)]
struct SetupExternalInstancePgBody {
#[serde(default)]
rotate_passwords: bool,
}
async fn setup_external_instance_pg(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Json(body): Json<SetupExternalInstancePgBody>,
) -> JsonResult<windmill_common::external_instance_pg::ExternalInstancePgSetupReport> {
require_super_admin(&db, &authed).await?;
let report = windmill_common::external_instance_pg::setup_external_instance_pg_unchecked(
&db,
body.rotate_passwords,
)
.await?;
let rotated = body.rotate_passwords.to_string();
let success = report.success.to_string();
windmill_audit::audit_oss::audit_log(
&db,
&authed,
"settings.setup_external_instance_pg",
windmill_audit::ActionKind::Update,
"global",
Some(&authed.email),
Some(
[
("rotate_passwords", rotated.as_str()),
("success", success.as_str()),
]
.into(),
),
)
.await?;
Ok(Json(report))
}
#[derive(Serialize)]
struct ExternalInstancePgDatabase {
#[serde(flatten)]
status: windmill_common::instance_config::CustomInstanceDb,
used_by_workspaces: Vec<String>,
}
async fn list_external_instance_pg_databases(
authed: ApiAuthed,
Extension(db): Extension<DB>,
) -> JsonResult<std::collections::BTreeMap<String, ExternalInstancePgDatabase>> {
require_super_admin(&db, &authed).await?;
let databases = windmill_common::external_instance_pg::external_instance_databases(&db).await?;
let mut usages =
windmill_common::external_instance_pg::external_instance_database_usages(&db).await?;
Ok(Json(
databases
.into_iter()
.map(|(name, status)| {
let used_by_workspaces = usages.remove(&name).unwrap_or_default();
(
name,
ExternalInstancePgDatabase {
status,
used_by_workspaces: used_by_workspaces.into_iter().collect(),
},
)
})
.collect(),
))
}
async fn create_external_instance_pg_database(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Path(dbname): Path<String>,
Json(body): Json<SetupCustomInstanceDbBody>,
) -> JsonResult<()> {
require_super_admin(&db, &authed).await?;
let tag = body.tag.as_deref().unwrap_or("datatable");
windmill_common::external_instance_pg::create_external_instance_database_unchecked(
&db, &dbname, tag, None,
)
.await?;
windmill_audit::audit_oss::audit_log(
&db,
&authed,
"settings.create_external_instance_pg_database",
windmill_audit::ActionKind::Create,
"global",
Some(&authed.email),
Some([("dbname", dbname.as_str()), ("tag", tag)].into()),
)
.await?;
Ok(Json(()))
}
async fn drop_external_instance_pg_database(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Path(dbname): Path<String>,
) -> JsonResult<()> {
require_super_admin(&db, &authed).await?;
// A data table naming a dropped database fails on every job, far from the drop that caused it.
windmill_common::external_instance_pg::drop_external_instance_database_unchecked(
&db, &dbname, None,
)
.await?;
windmill_audit::audit_oss::audit_log(
&db,
&authed,
"settings.drop_external_instance_pg_database",
windmill_audit::ActionKind::Delete,
"global",
Some(&authed.email),
Some([("dbname", dbname.as_str())].into()),
)
.await?;
Ok(Json(()))
}
#[derive(Deserialize)]
struct SetupCustomInstanceDbBody {
tag: Option<String>,
@@ -1923,38 +1767,18 @@ async fn setup_custom_instance_pg_database(
Path(dbname): Path<String>,
Json(body): Json<SetupCustomInstanceDbBody>,
) -> JsonResult<CustomInstanceDb> {
// Before anything is recorded: the status written below replaces the registry entry, and with it
// the workspace a fork copy is reserved for.
require_super_admin(&db, &authed).await?;
let mut logs = CustomInstanceDbLogs::default();
let result = setup_custom_instance_pg_database_inner(authed, &db, &dbname, &mut logs).await;
let success = result.is_ok();
let error = result.err().map(|e| e.to_string());
let status = CustomInstanceDb {
logs,
success,
error,
tag: body.tag,
used_by_workspaces: vec![],
workspace_id: None,
};
let status =
CustomInstanceDb { logs, success, error, tag: body.tag, used_by_workspaces: vec![] };
let status_json = serde_json::to_value(&status).map_err(to_anyhow)?;
// The fork reservation is carried over inside the write, from whatever the row holds then: a
// rename migrating it while the setup above ran would otherwise be overwritten with the value
// this request started from, stranding the copy under the archived workspace.
let saved = sqlx::query_scalar::<_, serde_json::Value>(
r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}',
COALESCE(value->'databases', '{}'::jsonb)
|| jsonb_build_object($1::text, $2::jsonb || jsonb_build_object(
'workspace_id', value->'databases'->$1::text->'workspace_id')))
WHERE name = 'custom_instance_pg_databases'
RETURNING value->'databases'->$1::text"#,
)
.bind(&dbname)
.bind(&status_json)
.fetch_one(&db)
.await?;
let status: CustomInstanceDb = serde_json::from_value(saved).map_err(to_anyhow)?;
// Save that the database was setup successfully
sqlx::query!(
r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}', (COALESCE(value->'databases', '{}'::jsonb) || to_jsonb($1::json))) WHERE name = 'custom_instance_pg_databases'"#,
json!({ dbname: status_json })
).execute(&db).await?;
Ok(Json(status))
}
+127 -22
View File
@@ -47,7 +47,10 @@ use windmill_audit::audit_oss::audit_log;
use windmill_audit::ActionKind;
use windmill_common::audit::AuditAuthor;
use windmill_common::auth::{hash_token, safe_token_prefix, TOKEN_PREFIX_LEN};
use windmill_common::global_settings::AUTOMATE_USERNAME_CREATION_SETTING;
use windmill_common::global_settings::{
load_value_from_global_settings, parse_max_token_expiration_days,
AUTOMATE_USERNAME_CREATION_SETTING, MAX_TOKEN_EXPIRATION_DAYS_SETTING,
};
use windmill_common::oauth2::InstanceEvent;
use windmill_common::per_minute_counter::PerMinuteCounter;
use windmill_common::users::truncate_token;
@@ -168,7 +171,10 @@ pub fn make_unauthed_service() -> Router {
.route("/logout", post(logout).get(logout))
.route("/is_first_time_setup", get(is_first_time_setup))
.route("/request_password_reset", post(request_password_reset))
.route("/login_link/{token}", get(consume_login_link))
.route(
"/login_link/{token}",
get(consume_login_link).post(confirm_login_link),
)
.route("/is_smtp_configured", get(is_smtp_configured))
.route(
"/is_password_login_disabled",
@@ -3092,11 +3098,68 @@ pub async fn create_guest_session_token<'c>(
// create_token_internal is re-exported from windmill-api-auth above
/// Applies the instance-wide ceiling on how long a token a caller picks the lifetime of may
/// live (`create_token`, and `impersonate` for superadmins), returning the expiration to store:
/// the requested one while it fits, the ceiling otherwise, and the ceiling as well when none was
/// requested. Only the stored expiration is capped: tokens already stored when the setting is
/// turned on or lowered keep theirs, since the auth lookup never reads the setting.
///
/// It shortens rather than refuses because most callers do not comply on their own. The CLI
/// authorization page, `wmill user create-token` and the editor's language-server token each
/// pick a lifetime, often none at all, without reading the setting (and CLIs already installed
/// never will), so refusing would break logging in and the editor instead of the long-lived
/// tokens the setting is aimed at.
///
/// Read from `global_settings` on each call rather than cached: token creation is rare
/// enough that the round trip costs nothing, and the ceiling is then never served stale.
///
/// A token owned by a service account is exempt: in the workspace the token names, or in any
/// workspace for a workspace-less token, which has none to match. Service accounts are the
/// identity automation that needs a long-lived credential runs as. The cost is that any
/// workspace admin can create and impersonate one to hold an uncapped token, so the ceiling
/// bounds personal tokens rather than what an admin can obtain.
async fn cap_token_expiration(
db: &DB,
owner_email: &str,
workspace_id: Option<&str>,
requested: Option<chrono::DateTime<chrono::Utc>>,
) -> Result<Option<chrono::DateTime<chrono::Utc>>> {
let value = load_value_from_global_settings(db, MAX_TOKEN_EXPIRATION_DAYS_SETTING).await?;
let max_days = match parse_max_token_expiration_days(value.as_ref()) {
Ok(Some(max_days)) => max_days,
Ok(None) => return Ok(requested),
// Both write paths reject this, so only a row written around them gets here.
Err(e) => {
tracing::warn!("ignoring {MAX_TOKEN_EXPIRATION_DAYS_SETTING}: {e}");
return Ok(requested);
}
};
let max = chrono::Utc::now() + chrono::Duration::days(max_days);
let is_service_account = sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM usr WHERE email = $1 AND is_service_account IS true
AND ($2::varchar IS NULL OR workspace_id = $2))",
owner_email,
workspace_id,
)
.fetch_one(db)
.await?
.unwrap_or(false);
if is_service_account {
return Ok(requested);
}
Ok(Some(match requested {
Some(expiration) if expiration < max => expiration,
_ => max,
}))
}
async fn create_token(
Extension(db): Extension<DB>,
authed: ApiAuthed,
OptJobAuthed { job_id, .. }: OptJobAuthed,
Json(token_config): Json<NewToken>,
Json(mut token_config): Json<NewToken>,
) -> Result<(StatusCode, String)> {
forbid_elevated_job_token(&db, &authed.email, job_id).await?;
check_token_create_rate_limit(&authed.username)?;
@@ -3118,6 +3181,14 @@ async fn create_token(
windmill_api_auth::ensure_scopes_within_caller(&authed, token_config.scopes.as_deref())?;
token_config.expiration = cap_token_expiration(
&db,
&authed.email,
token_config.workspace_id.as_deref(),
token_config.expiration,
)
.await?;
let mut tx = db.begin().await?;
let token = create_token_internal(&mut *tx, &db, &authed, token_config).await?;
@@ -3174,6 +3245,7 @@ async fn impersonate(
.fetch_optional(&db)
.await?
.unwrap_or(false);
let expiration = cap_token_expiration(&db, &impersonated, None, new_token.expiration).await?;
let mut tx = db.begin().await?;
sqlx::query!(
@@ -3185,7 +3257,7 @@ async fn impersonate(
plaintext as Option<&str>,
impersonated,
new_token.label,
new_token.expiration,
expiration,
is_super_admin
)
.execute(&mut *tx)
@@ -3195,7 +3267,7 @@ async fn impersonate(
&mut *tx,
&t_hash,
new_token.label.as_deref(),
new_token.expiration,
expiration,
)
.await;
@@ -3215,9 +3287,12 @@ async fn impersonate(
}
const LOGIN_LINK_DEFAULT_TTL_S: u32 = 600;
const LOGIN_LINK_MAX_TTL_S: u32 = 900;
// Long enough for a link sent by email to still work when it is read. `require_login_type` is
// only checked at mint, so a much longer cap would need re-checking it when the link is opened.
const LOGIN_LINK_MAX_TTL_S: u32 = 7200;
const LOGIN_LINK_DEFAULT_RD: &str = "/user/workspaces";
const LOGIN_LINK_EXPIRED_PAGE: &str = "/user/login_link_expired";
const LOGIN_LINK_CONFIRM_PAGE: &str = "/user/login_link";
#[derive(Deserialize)]
pub struct NewLoginLink {
@@ -3228,6 +3303,9 @@ pub struct NewLoginLink {
/// account it created can require `pending_oauth`, so the link stops working once the
/// owner has set a password or signed in with a provider.
pub require_login_type: Option<String>,
/// Hand out a page that signs in only when its button is clicked. Mail scanners open links
/// on delivery, and opening the plain link spends it, so a link sent by email sets this.
pub confirm: Option<bool>,
}
#[derive(Serialize)]
@@ -3378,11 +3456,12 @@ async fn create_login_link(
.await?;
tx.commit().await?;
let url = format!(
"{}/api/auth/login_link/{}",
(**BASE_URL.load()).clone(),
token
);
let base_url = (**BASE_URL.load()).clone();
let url = if nl.confirm.unwrap_or(false) {
format!("{base_url}{LOGIN_LINK_CONFIRM_PAGE}?token={token}")
} else {
format!("{base_url}/api/auth/login_link/{token}")
};
Ok((StatusCode::CREATED, Json(LoginLink { url, expires_at })))
}
@@ -3628,19 +3707,45 @@ async fn consume_login_link(
Path(token): Path<String>,
Query(query): Query<LoginLinkQuery>,
) -> Result<Response> {
let bounce = |reason: &str| {
Ok(login_link_redirect(format!(
"{LOGIN_LINK_EXPIRED_PAGE}?reason={reason}"
)))
};
let location = redeem_login_link(&headers, cookies, &db, &token, query.rd).await?;
Ok(login_link_redirect(location))
}
#[derive(Serialize)]
struct LoginLinkLocation {
location: String,
}
/// The confirmation page's click. It answers with where to go rather than redirecting, and the
/// page navigates there itself.
async fn confirm_login_link(
headers: axum::http::HeaderMap,
cookies: Cookies,
Extension(db): Extension<DB>,
Path(token): Path<String>,
) -> JsonResult<LoginLinkLocation> {
let location = redeem_login_link(&headers, cookies, &db, &token, None).await?;
Ok(Json(LoginLinkLocation { location }))
}
/// Spends the link and sets the session cookie, returning the post-login destination; or
/// returns the explanation page, with no session, when the link cannot be used.
async fn redeem_login_link(
headers: &axum::http::HeaderMap,
cookies: Cookies,
db: &DB,
token: &str,
requested_rd: Option<String>,
) -> Result<String> {
let bounce = |reason: &str| Ok(format!("{LOGIN_LINK_EXPIRED_PAGE}?reason={reason}"));
if token.len() != 32 {
return bounce("invalid");
}
let t_hash = hash_token(&token);
let t_hash = hash_token(token);
// The account is unknown until the row is read, so only the global and per-IP tiers
// apply here; a 32-char random token leaves nothing for the per-account tier to guard.
windmill_common::login_rate_limit::check_and_increment_login_attempt(
&headers,
headers,
&t_hash[..TOKEN_PREFIX_LEN],
)?;
@@ -3707,11 +3812,10 @@ async fn consume_login_link(
.await?;
tx.commit().await?;
let rd = link
Ok(link
.rd
.or_else(|| same_origin_rd(query.rd))
.unwrap_or_else(|| LOGIN_LINK_DEFAULT_RD.to_string());
Ok(login_link_redirect(rd))
.or_else(|| same_origin_rd(requested_rd))
.unwrap_or_else(|| LOGIN_LINK_DEFAULT_RD.to_string()))
}
#[derive(Deserialize)]
@@ -3945,6 +4049,7 @@ async fn update_token_label(
AND NOT starts_with(label, 'embed_app:')
AND NOT starts_with(label, 'sdk_app:')
AND NOT starts_with(label, 'impersonation:')
AND NOT starts_with(label, 'cli-login:')
))
RETURNING token_prefix",
req.label.as_deref(),
File diff suppressed because it is too large Load Diff
@@ -1,44 +0,0 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
//! Where the ACL planner comes from: the enterprise one, or a refusal.
//!
//! Data table roles are an Enterprise Edition feature, and so is everything here — reading who
//! owns what included. `private` alone is not that edition — community builds carry it — so the
//! planner is behind `enterprise` as well.
#[cfg(all(feature = "private", feature = "enterprise"))]
pub(crate) use crate::datatable_acl_ee::plan_statements;
#[cfg(all(feature = "private", feature = "enterprise"))]
pub(crate) fn ensure_datatable_acl_available() -> windmill_common::error::Result<()> {
Ok(())
}
#[cfg(not(all(feature = "private", feature = "enterprise")))]
use {
crate::datatable_acl::{AclChange, AclPlan, AclTarget, CatalogFacts},
windmill_common::{datatable_roles_oss::datatable_roles_unavailable, error::Result},
};
/// Checked first by every ACL route, before anything is read or connected to.
#[cfg(not(all(feature = "private", feature = "enterprise")))]
pub(crate) fn ensure_datatable_acl_available() -> Result<()> {
Err(datatable_roles_unavailable())
}
#[cfg(not(all(feature = "private", feature = "enterprise")))]
pub(crate) fn plan_statements(
_target: &AclTarget,
_change: &AclChange,
_dbname: &str,
_pg_role: &str,
_facts: &CatalogFacts,
) -> Result<AclPlan> {
Err(datatable_roles_unavailable())
}
@@ -11,7 +11,7 @@
//! to keep that file focused on core workspace configuration.
use crate::workspaces::{
managed_datatable_kind, pg_dump_database, strip_unreplayable_dump_lines, ItemComparison,
is_instance_datatable, pg_dump_database, strip_unreplayable_dump_lines, ItemComparison,
PgDumpOptions,
};
@@ -1556,9 +1556,7 @@ async fn generate_initial_datatable_migration(
// without what a replay elsewhere cannot run: the replaying user owns none of this
// database's objects, and the grants Windmill plants in an instance database (`ALTER
// DEFAULT PRIVILEGES FOR ROLE ...`) fail even replaying onto the same server.
let no_acl = managed_datatable_kind(&db, &w_id, &datatable_name)
.await?
.is_some();
let no_acl = is_instance_datatable(&db, &w_id, &datatable_name).await?;
let dump_file = pg_dump_database(
&pg_db,
PgDumpOptions {
@@ -65,15 +65,3 @@ pub(crate) async fn ensure_reaches_datatable(
) -> Result<()> {
roles::ensure_reaches_datatable(db, w_id, datatable_name, authed).await
}
/// [`ensure_reaches_datatable`] against an entry already resolved, for a caller that goes on to
/// connect from that same entry.
pub(crate) async fn ensure_reaches_governing_datatable(
db: &DB,
w_id: &str,
datatable_name: &str,
governing: &GoverningDatatable,
authed: &ApiAuthed,
) -> Result<()> {
roles::ensure_reaches_governing_datatable(db, w_id, datatable_name, governing, authed).await
}
@@ -12,9 +12,8 @@
#[cfg(all(feature = "private", feature = "enterprise"))]
pub(crate) use crate::datatable_permissions_ee::{
ensure_governs_datatable, ensure_reaches_datatable, ensure_reaches_governing_datatable,
get_datatable_permissions, list_usable_datatable_roles, set_datatable_permissions,
usable_datatable_roles,
ensure_governs_datatable, ensure_reaches_datatable, get_datatable_permissions,
list_usable_datatable_roles, set_datatable_permissions,
};
#[cfg(not(all(feature = "private", feature = "enterprise")))]
@@ -57,20 +56,6 @@ mod ce {
}
}
pub(crate) async fn ensure_reaches_governing_datatable(
_db: &DB,
_w_id: &str,
_datatable_name: &str,
governing: &GoverningDatatable,
_authed: &ApiAuthed,
) -> Result<()> {
if governing.datatable.permissions.is_none() {
Ok(())
} else {
Err(unavailable())
}
}
// The routes stay registered so the API has one shape; each answers after authentication,
// before anything is read.
@@ -85,28 +70,4 @@ mod ce {
pub(crate) async fn list_usable_datatable_roles(_authed: ApiAuthed) -> Result<String> {
Err(unavailable())
}
pub(crate) struct UsableDatatableRoles {
pub(crate) permissioned: bool,
pub(crate) roles: Vec<String>,
pub(crate) default_role: String,
}
/// A data table not under roles is used as `admin`, as before roles existed. One under roles
/// is refused: no role of it can be connected as.
pub(crate) async fn usable_datatable_roles(
_db: &DB,
_authed: &ApiAuthed,
_w_id: &str,
governing: &GoverningDatatable,
) -> Result<UsableDatatableRoles> {
if governing.datatable.permissions.is_some() {
return Err(unavailable());
}
Ok(UsableDatatableRoles {
permissioned: false,
roles: vec![],
default_role: windmill_common::datatable_roles::ADMIN_DATATABLE_ROLE.to_string(),
})
}
}
@@ -1,8 +1,6 @@
#[cfg(feature = "parquet")]
pub mod ai_session_backups;
pub mod data_metrics;
pub mod datatable_acl;
pub mod datatable_acl_oss;
pub mod datatable_migrations;
pub mod datatable_permissions;
pub mod datatable_permissions_oss;
@@ -14,8 +12,5 @@ pub mod workspaces_oss;
#[cfg(feature = "private")]
pub mod workspaces_ee;
#[cfg(all(feature = "private", feature = "enterprise"))]
pub mod datatable_acl_ee;
#[cfg(all(feature = "private", feature = "enterprise"))]
pub mod datatable_permissions_ee;
File diff suppressed because it is too large Load Diff
@@ -110,25 +110,10 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
// A fork copy reserved for the old id would otherwise be unreachable: its creator cannot
// import into it or finish its fork under the new id, and nothing else would ever drop it.
sqlx::query(
r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}', (
SELECT COALESCE(jsonb_object_agg(k, CASE WHEN v->>'workspace_id' = $1
THEN jsonb_set(v, '{workspace_id}', to_jsonb($2::text)) ELSE v END), '{}'::jsonb)
FROM jsonb_each(COALESCE(value->'databases', '{}'::jsonb)) AS e(k, v)
))
WHERE name = 'custom_instance_pg_databases'"#,
)
.bind(&old_id)
.bind(&rw.new_id)
.execute(&mut *tx)
.await?;
// Duplicate workspace settings (keep copy in old workspace for reference)
info!("Duplicating workspace_settings table");
sqlx::query!(
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url FROM workspace_settings WHERE workspace_id = $2",
"INSERT INTO workspace_settings (workspace_id, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks) SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, git_credentials, ducklake, dbt_warehouses, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler, public_app_execution_limit_per_minute, error_handler_fallback_to_instance_alerts, guest_access_enabled, guest_jwt_public_key, guest_jwt_jwks_url, add_admins_and_developers_to_forks FROM workspace_settings WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
@@ -1424,7 +1409,9 @@ pub async fn drop_forked_datatable_databases(
_ => continue,
};
if database.resource_type.is_windmill_managed() {
if database.resource_type
== windmill_common::workspaces::DataTableCatalogResourceType::Instance
{
let db_to_drop = &database.resource_path;
if !db_to_drop.starts_with("wm_fork_") {
errors.push(format!(
@@ -1433,64 +1420,7 @@ pub async fn drop_forked_datatable_databases(
));
continue;
}
// The fork's own entry is what is going away; anything else still reaching the copy,
// a child fork's pointer at this entry included, keeps it. The lock keeps a child fork
// from gaining such a pointer before the drop.
let dropped = async {
let mut tx = db.begin().await?;
// The three locks a settings save takes, in its order: this workspace's data
// tables, its settings row, and the database itself. Without them a save could
// rename this entry, or point another one here, either side of the check below.
windmill_common::workspaces::lock_fork_datatables(&mut tx, &w_id).await?;
sqlx::query("SELECT 1 FROM workspace_settings WHERE workspace_id = $1 FOR UPDATE")
.bind(&w_id)
.fetch_optional(&mut *tx)
.await?;
windmill_common::datatable_roles::lock_instance_databases_governance(
&mut tx,
[db_to_drop.as_str()],
)
.await?;
if database.resource_type
== windmill_common::workspaces::DataTableCatalogResourceType::ExternalInstance
{
windmill_common::external_instance_pg::drop_external_instance_database_unchecked(
&db,
db_to_drop,
Some((&w_id, dt_name)),
)
.await?;
} else {
let uses = windmill_common::workspaces::managed_database_uses(
&mut tx,
windmill_common::workspaces::DataTableCatalogResourceType::Instance,
db_to_drop,
Some((&w_id, dt_name)),
)
.await?;
if !uses.is_empty() {
return Err(Error::BadRequest(format!(
"it is still used by {}",
uses.join(", ")
)));
}
windmill_common::drop_custom_instance_database(&db, db_to_drop).await?;
}
// The entry goes with the database: a fork this one is cloned into afterwards must
// not inherit a pointer at a data table whose database is gone.
sqlx::query(
"UPDATE workspace_settings SET datatable = datatable #- ARRAY['datatables', $2]
WHERE workspace_id = $1",
)
.bind(&w_id)
.bind(dt_name)
.execute(&mut *tx)
.await?;
tx.commit().await?;
Ok::<_, Error>(())
}
.await;
if let Err(e) = dropped {
if let Err(e) = windmill_common::drop_custom_instance_database(&db, db_to_drop).await {
errors.push(format!(
"Could not drop instance database '{}' for datatable://{}: {}",
db_to_drop, dt_name, e
+78 -535
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.814.0
version: 1.815.0
title: Windmill API
contact:
@@ -529,6 +529,30 @@ paths:
responses:
"302":
description: redirected to the post-login destination, or to /user/login_link_expired when the link is used, expired or unknown
post:
security: []
summary: consume a single-use login link from its confirmation page, set the session cookie and answer where to go
operationId: confirmLoginLink
tags:
- user
parameters:
- name: token
in: path
required: true
schema:
type: string
responses:
"200":
description: the post-login destination, or /user/login_link_expired when the link is used, expired or unknown
content:
application/json:
schema:
type: object
required:
- location
properties:
location:
type: string
/auth/reset_password:
post:
@@ -1572,104 +1596,6 @@ paths:
schema:
type: object
/settings/external_instance_pg/status:
get:
summary: Returns whether the external instance cluster is configured and how its last setup went
operationId: getExternalInstancePgStatus
tags:
- setting
responses:
"200":
description: external instance cluster status
content:
application/json:
schema:
$ref: "#/components/schemas/ExternalInstancePgStatus"
/settings/external_instance_pg/setup:
post:
summary: Sets up the external instance cluster with its saved admin login, optionally rotating the passwords Windmill manages on it (enterprise edition only)
operationId: setupExternalInstancePg
tags:
- setting
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
rotate_passwords:
type: boolean
responses:
"200":
description: the setup report, also stored as the last setup
content:
application/json:
schema:
$ref: "#/components/schemas/ExternalInstancePgSetupReport"
/settings/external_instance_pg/databases:
get:
summary: Lists the databases Windmill created on the external instance cluster, with the workspaces whose data tables use each
operationId: listExternalInstancePgDatabases
tags:
- setting
responses:
"200":
description: databases by name
content:
application/json:
schema:
type: object
additionalProperties:
$ref: "#/components/schemas/CustomInstanceDb"
/settings/external_instance_pg/databases/{name}:
post:
summary: Creates a database on the external instance cluster (enterprise edition only)
operationId: createExternalInstancePgDatabase
tags:
- setting
parameters:
- name: name
in: path
required: true
schema:
type: string
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
tag:
$ref: "#/components/schemas/CustomInstanceDbTag"
responses:
"200":
description: database created
content:
application/json:
schema: {}
delete:
summary: Drops a database Windmill created on the external instance cluster, refused while a data table uses it
operationId: dropExternalInstancePgDatabase
tags:
- setting
parameters:
- name: name
in: path
required: true
schema:
type: string
responses:
"200":
description: database dropped
content:
application/json:
schema: {}
/settings/list_custom_instance_pg_databases:
post:
summary: Returns the set-up statuses of custom instance pg databases
@@ -4164,9 +4090,13 @@ paths:
guest_access_enabled:
type: boolean
description: Whether this workspace admits guest sessions. An app's own `guest` execution mode is inert while this is false.
add_admins_and_developers_to_forks:
type: boolean
description: Whether every new fork of this workspace starts with its admins and developers as members, keeping their role.
required:
- workspace_id
- guest_access_enabled
- add_admins_and_developers_to_forks
/w/{workspace}/workspaces/get_settings:
get:
@@ -4257,6 +4187,9 @@ paths:
guest_jwt_jwks_url:
type: string
description: JWKS URL a guest JWT (`jwt_guest_`) is verified against for this workspace. Mutually exclusive with `guest_jwt_public_key`.
add_admins_and_developers_to_forks:
type: boolean
description: Whether every new fork of this workspace starts with its admins and developers as members, keeping their role.
/w/{workspace}/workspaces/get_deploy_to:
get:
@@ -5328,7 +5261,7 @@ paths:
type: string
resource_type:
type: string
enum: [postgres, instance, external_instance]
enum: [postgres, instance]
resource_path:
type: string
governing_workspace_id:
@@ -5392,97 +5325,6 @@ paths:
schema:
type: string
/w/{workspace}/workspaces/datatable_acl/{datatable_name}:
get:
summary: read the owner and grants of an instance data table's database, schema or table
operationId: getDatatableAcl
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: datatable_name
in: path
required: true
schema:
type: string
- name: kind
in: query
required: true
schema:
type: string
enum: [database, schema, table]
- name: schema
in: query
required: false
schema:
type: string
- name: table
in: query
required: false
schema:
type: string
responses:
"200":
description: owner and grants
content:
application/json:
schema:
$ref: "#/components/schemas/DatatableAclInfo"
/w/{workspace}/workspaces/datatable_acl/{datatable_name}/plan:
post:
summary: preview the SQL an ownership or grant change would run (data table administrators only)
operationId: planDatatableAcl
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: datatable_name
in: path
required: true
schema:
type: string
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/AclChangeRequest"
responses:
"200":
description: statements that would run, in a single transaction
content:
application/json:
schema:
$ref: "#/components/schemas/AclPlan"
/w/{workspace}/workspaces/datatable_acl/{datatable_name}/apply:
post:
summary: run an ownership or grant change exactly as planned (data table administrators only)
operationId: applyDatatableAcl
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: datatable_name
in: path
required: true
schema:
type: string
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/AclChangeRequest"
responses:
"200":
description: change applied
content:
text/plain:
schema:
type: string
/w/{workspace}/workspaces/datatable_usable_roles/{datatable_name}:
get:
summary: list the data table roles the caller may connect as
@@ -5586,21 +5428,6 @@ paths:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: datatable_name
in: query
description: list only this data table; each listed data table opens a connection to its database
schema:
type: string
- name: role_for
in: query
description: the data table `role` applies to; every other one is listed as its default role
schema:
type: string
- name: role
in: query
description: the role to list `role_for` as; refused, in that entry's `error`, if the caller may not use it
schema:
type: string
responses:
"200":
description: table metadata of all datatables
@@ -5634,11 +5461,6 @@ paths:
required: true
schema:
type: string
- name: role
in: query
description: the data table role to read the table as; defaults to the data table's default role
schema:
type: string
responses:
"200":
description: schema of one datatable table
@@ -6498,6 +6320,39 @@ paths:
schema:
type: string
/w/{workspace}/workspaces/edit_add_admins_and_developers_to_forks:
post:
summary: choose whether new forks of this workspace start with its admins and developers
description: >-
When on, every fork created from this workspace gets the workspace's admins and
developers as members, with the role they hold here; operators, disabled users and
service accounts are left out. The setting is copied into each fork, so forks of a
fork follow it too. Off by default. Workspace-admin gated.
operationId: editAddAdminsAndDevelopersToForks
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
requestBody:
description: Whether new forks start with this workspace's admins and developers
required: true
content:
application/json:
schema:
type: object
properties:
add_admins_and_developers_to_forks:
type: boolean
required:
- add_admins_and_developers_to_forks
responses:
"200":
description: status
content:
text/plain:
schema:
type: string
/w/{workspace}/workspaces/edit_guest_jwt_key:
post:
summary: set the key guest JWTs are verified against for this workspace
@@ -6962,7 +6817,7 @@ paths:
type: string
expires_in_s:
type: integer
description: link lifetime in seconds, at most 900 (default 600)
description: link lifetime in seconds, at most 7200 (default 600)
rd:
type: string
description: same-origin path the browser lands on after login (default /user/workspaces)
@@ -6972,6 +6827,12 @@ paths:
mint only while the account still has this login type (for example
pending_oauth), so a link stops working once the owner has set a password
or signed in with a provider
confirm:
type: boolean
description: >-
return a /user/login_link page that signs in only when its button is
clicked, instead of a link spent by opening it; set it for links sent by
email, which mail scanners open on delivery (default false)
responses:
"201":
description: login link minted
@@ -33798,44 +33659,6 @@ components:
- ducklake
- datatable
ExternalInstancePgSetupStep:
type: object
required: [name, status, message]
properties:
name:
type: string
status:
type: string
enum: [ok, warning, error]
message:
type: string
ExternalInstancePgSetupReport:
type: object
required: [success, finished_at, steps]
properties:
success:
type: boolean
description: no step failed; warnings leave it true
finished_at:
type: string
format: date-time
steps:
type: array
items:
$ref: "#/components/schemas/ExternalInstancePgSetupStep"
ExternalInstancePgStatus:
type: object
required: [configured, database_count]
properties:
configured:
type: boolean
database_count:
type: integer
last_setup:
$ref: "#/components/schemas/ExternalInstancePgSetupReport"
InstanceDatatableRole:
type: object
required: [id, name, enabled]
@@ -33896,255 +33719,6 @@ components:
datatable:
type: string
AclTarget:
description: what access is read or changed on
oneOf:
- $ref: "#/components/schemas/AclTargetDatabase"
- $ref: "#/components/schemas/AclTargetSchema"
- $ref: "#/components/schemas/AclTargetTable"
discriminator:
propertyName: kind
mapping:
database: "#/components/schemas/AclTargetDatabase"
schema: "#/components/schemas/AclTargetSchema"
table: "#/components/schemas/AclTargetTable"
AclTargetDatabase:
type: object
required: [kind]
properties:
kind:
type: string
enum: [database]
AclTargetSchema:
type: object
required: [kind, schema]
properties:
kind:
type: string
enum: [schema]
schema:
type: string
AclTargetTable:
type: object
required: [kind, schema, table]
properties:
kind:
type: string
enum: [table]
schema:
type: string
table:
type: string
AclChange:
description: one change to plan or apply
oneOf:
- $ref: "#/components/schemas/AclChangeSetOwner"
- $ref: "#/components/schemas/AclChangeGrant"
- $ref: "#/components/schemas/AclChangeRevoke"
discriminator:
propertyName: type
mapping:
set_owner: "#/components/schemas/AclChangeSetOwner"
grant: "#/components/schemas/AclChangeGrant"
revoke: "#/components/schemas/AclChangeRevoke"
AclChangeSetOwner:
type: object
description: >-
hands the target to role — for a schema, with everything already in it but an extension's
members, which stay with the extension
required: [type, role]
properties:
type:
type: string
enum: [set_owner]
role:
type: string
description: a data table role of the instance, or admin
AclChangeGrant:
type: object
required: [type, role, privileges, scope]
properties:
type:
type: string
enum: [grant]
role:
type: string
description: a data table role of the instance, or admin
privileges:
type: array
items:
type: string
scope:
$ref: "#/components/schemas/AclGrantScope"
AclChangeRevoke:
type: object
required: [type, role, privileges, scope]
properties:
type:
type: string
enum: [revoke]
role:
type: string
description: a data table role of the instance, other than admin
privileges:
type: array
items:
type: string
scope:
$ref: "#/components/schemas/AclGrantScope"
objects:
type: array
description: >-
objects inside the target the revoke covers, empty for the target itself. Only with the
target scope; a revoke on all objects of a kind is refused, since it cannot say which
grants it takes back.
items:
$ref: "#/components/schemas/AclObject"
AclGrantScope:
type: string
enum:
[
target,
all_tables,
all_sequences,
all_functions,
future_tables,
future_sequences,
future_functions,
]
AclChangeRequest:
type: object
required: [target, change]
properties:
target:
$ref: "#/components/schemas/AclTarget"
change:
$ref: "#/components/schemas/AclChange"
statements:
type: array
description: >-
The statements the plan showed. Required to apply, which plans again and refuses if
the result differs.
items:
type: string
AclPlan:
type: object
required: [statements, warnings]
properties:
statements:
type: array
items:
type: string
warnings:
type: array
items:
type: string
AclObject:
type: object
required: [name, kind]
properties:
name:
type: string
kind:
type: string
description: >-
TABLE, SEQUENCE, FUNCTION, PROCEDURE or TYPE — what the object is. A revoke turns it
into the keyword it takes, ROUTINE for both routine kinds; a type's grants are read
only.
args:
type: string
description: identity arguments of a routine, which is what tells two of the same name apart
AclGrant:
type: object
required: [grantee, privileges, sources]
properties:
grantee:
type: string
privileges:
type: array
items:
type: string
object:
$ref: "#/components/schemas/AclObject"
future:
type: string
description: >-
set for a default privilege, naming the kind of object it covers (TABLES, SEQUENCES,
FUNCTIONS, TYPES, or SCHEMAS). On a schema, the defaults set in that schema; on the
database, the ones set database-wide, which apply in every schema and which no schema's
own defaults take back.
sources:
type: array
description: >-
the roles the grant comes from, each once — who granted it, or for a default privilege
the role whose future objects it covers. A revoke of some of the grant's privileges
takes them back from every source that gave them.
items:
$ref: "#/components/schemas/AclSource"
AclSource:
type: object
required: [role, privileges, reachable]
properties:
role:
type: string
privileges:
type: array
description: >-
what role gave of the grant's privileges. A revoke is held back only by a source out of
reach that gave some of what it takes back.
items:
type: string
reachable:
type: boolean
description: >-
whether the data table's connection can take back what role gave. On an object that is
the owner, when the connection acts for the owner, and otherwise the connection itself;
for a default privilege, a creating role the connection acts for. What a source out of
reach gave is not revocable from here; privileges only other sources gave still are.
DatatableAclInfo:
type: object
required: [owner, roles, editable, supports_maintain, dbname, grants, children]
properties:
owner:
type: string
roles:
type: array
description: the roles a change may name; empty unless the caller may change anything
items:
type: string
editable:
type: boolean
description: whether the caller may plan and apply changes
supports_maintain:
type: boolean
description: whether the server is Postgres 17+, which added the MAINTAIN table privilege
dbname:
type: string
description: the database the target lives in
grants:
type: array
items:
$ref: "#/components/schemas/AclGrant"
children:
type: array
description: a database's schemas, or a schema's tables
items:
type: string
CustomInstanceDb:
type: object
required:
@@ -34168,10 +33742,7 @@ components:
type: array
items:
type: string
description: Workspaces that reference this database through a ducklake catalog or a datatable database of the kind being listed — 'instance' for the instance databases endpoint, 'external_instance' for the external cluster one. Computed at request time, not persisted, and only returned to superadmins.
workspace_id:
type: string
description: The workspace a member created this database for as a fork copy. Only that workspace can import into it or point a fork at it.
description: Workspaces that reference this database via a ducklake catalog or datatable database with resource_type 'instance'. Computed at request time, not persisted.
NewSqsTrigger:
type: object
@@ -35230,6 +34801,9 @@ components:
nullable: true
disabled:
type: boolean
is_service_account:
type: boolean
description: Whether this membership is a service account.
required:
- id
- name
@@ -36208,7 +35782,6 @@ components:
enum:
- postgresql
- instance
- external_instance
resource_path:
type: string
required:
@@ -36292,17 +35865,7 @@ components:
DataTableTables:
type: object
required:
[
datatable_name,
schemas,
instance,
permissioned,
usable_roles,
default_role,
can_create_schema,
creatable_schemas,
]
required: [datatable_name, schemas]
properties:
datatable_name:
type: string
@@ -36315,26 +35878,6 @@ components:
type: string
error:
type: string
instance:
type: boolean
description: on the instance database, the only kind that can be under roles or have its access edited
permissioned:
type: boolean
usable_roles:
type: array
description: the roles the caller may connect as, by name; empty when not under roles
items:
type: string
default_role:
type: string
can_create_schema:
type: boolean
description: whether the role the listing connected as may create schemas
creatable_schemas:
type: array
description: the schemas the role the listing connected as may create in
items:
type: string
DataTableTableSchema:
type: object
+1 -6
View File
@@ -8723,12 +8723,7 @@ fn register_potential_assets_on_inline_execution(
.as_ref()
.and_then(|args| args.get("database"))
.map(|v| v.get().trim_matches('"'))
.and_then(|dt| dt.strip_prefix("datatable://"))
// `?role=` picks the connection, not the data table. Anything else after a `?` may be
// part of a name stored before names were restricted, so it stays.
.map(|dt| {
windmill_common::workspaces::parse_datatable_ref(dt).map_or(dt, |(name, _)| name)
});
.and_then(|dt| dt.strip_prefix("datatable://"));
if let Some(datatable) = datatable {
let re = regex::Regex::new(r#"SET search_path TO "([^"]+)";"#).unwrap();
let (schema, content) = if let Some(captures) = re.captures(&preview.content) {
+5 -3
View File
@@ -560,7 +560,7 @@ pub async fn run_server(
if server_mode || mcp_mode {
use mcp::{
add_www_authenticate_header, add_www_authenticate_header_gateway,
extract_workspace_from_token,
extract_workspace_from_token, reject_token_query_param,
};
let (mcp_router, mcp_cancellation_token) = setup_mcp_server(
db.clone(),
@@ -573,15 +573,17 @@ pub async fn run_server(
let workspaced_mcp_router = mcp_router
.clone()
.route_layer(from_extractor::<ApiAuthed>())
.layer(axum::middleware::from_fn(reject_token_query_param))
.layer(axum::middleware::from_fn(add_www_authenticate_header))
.layer(axum::middleware::from_fn(extract_and_store_workspace_id));
// Gateway MCP router — resolves workspace from token
let gateway_mcp_router = mcp_router
.route_layer(from_extractor::<ApiAuthed>())
.layer(axum::middleware::from_fn(extract_workspace_from_token))
.layer(axum::middleware::from_fn(reject_token_query_param))
.layer(axum::middleware::from_fn(
add_www_authenticate_header_gateway,
))
.layer(axum::middleware::from_fn(extract_workspace_from_token));
));
(
workspaced_mcp_router,
gateway_mcp_router,
+28 -2
View File
@@ -43,11 +43,14 @@ use axum::{
extract::{Extension, Path},
http::Request,
middleware::Next,
response::Response,
response::{IntoResponse, Response},
routing::get,
Json, Router,
};
use windmill_common::{auth::hash_token, db::GatewayWorkspaceId, error::JsonResult};
use windmill_common::{
auth::hash_token, db::GatewayWorkspaceId, error::JsonResult,
global_settings::MCP_DISABLE_TOKEN_QUERY_PARAM,
};
// McpAuth impl for ApiAuthed is in windmill-api-auth (same crate as the type)
@@ -446,6 +449,29 @@ pub async fn add_www_authenticate_header(
}
}
/// Middleware refusing a credential carried in the MCP URL once the instance sets
/// `mcp_disable_token_query_param`. Sits outside everything that reads the token, so neither
/// the gateway lookup nor `ApiAuthed` ever sees it, and inside the `WWW-Authenticate` layer,
/// whose header is what sends the client into the OAuth flow instead. Refused rather than
/// ignored: the URL leaked the token whether or not the request used it.
pub async fn reject_token_query_param(request: Request<axum::body::Body>, next: Next) -> Response {
let carries_token = MCP_DISABLE_TOKEN_QUERY_PARAM.load(std::sync::atomic::Ordering::Relaxed)
&& request
.uri()
.query()
.is_some_and(|q| url::form_urlencoded::parse(q.as_bytes()).any(|(k, _)| k == "token"));
if carries_token {
return (
axum::http::StatusCode::UNAUTHORIZED,
"This instance does not accept a token in the MCP URL. Remove the token query \
parameter and let your client sign in through OAuth, or send the token in an \
Authorization header.",
)
.into_response();
}
next.run(request).await
}
/// Extract the bearer token from either the `Authorization` header or the
/// `?token=` query parameter (MCP clients commonly pass it in the URL).
fn extract_gateway_token(request: &Request<axum::body::Body>) -> Option<String> {
+1 -1
View File
@@ -12,5 +12,5 @@ pub mod oauth_server;
pub use core::{
add_www_authenticate_header, add_www_authenticate_header_gateway,
extract_and_store_workspace_id, extract_workspace_from_token, list_tools_service,
setup_mcp_server,
reject_token_query_param, setup_mcp_server,
};
-1
View File
@@ -76,7 +76,6 @@ bitflags.workspace = true
once_cell.workspace = true
phf.workspace = true
tokio-postgres.workspace = true
postgres-protocol.workspace = true
postgres-native-tls.workspace = true
native-tls.workspace = true
+25 -8
View File
@@ -20,7 +20,7 @@ use crate::{
/// Whether `label` denotes a user-created token rather than a system token
/// (`session`, `guest_session`, `ephemeral*`, `debugger-token`, `mcp-oauth-*`,
/// `embed_app:*`, `sdk_app:*`, `impersonation:*`). System-token labels are load-bearing —
/// `embed_app:*`, `sdk_app:*`, `impersonation:*`, `cli-login:*`). System-token labels are load-bearing —
/// session cleanup, super_admin propagation, expiry notifications and username overrides
/// all key off them — so they must not be user-editable. `None` (no label) is treated as
/// a user token.
@@ -47,10 +47,17 @@ pub fn is_user_token(label: Option<&str>) -> bool {
&& !l.starts_with(APP_EMBED_TOKEN_LABEL_PREFIX)
&& !l.starts_with(RAW_APP_SDK_TOKEN_LABEL_PREFIX)
&& !l.starts_with("impersonation:")
&& !l.starts_with(CLI_LOGIN_TOKEN_LABEL_PREFIX)
}
}
}
/// How far ahead of a user token's expiration its owner is warned (`check_expiring_tokens` in
/// the monitor). A token whose whole lifetime fits in this window gets no warning at all: it
/// would arrive minutes after creation, about a lifetime its creator just picked. Its
/// "expired and deleted" notice still goes out.
pub const TOKEN_EXPIRY_WARNING_DAYS: i32 = 7;
/// Label prefix, followed by the app path, of the token an app viewer's sandboxed iframe
/// runs with. Reserved in [`is_user_token`], whose SQL and frontend mirrors spell it out.
pub const APP_EMBED_TOKEN_LABEL_PREFIX: &str = "embed_app:";
@@ -59,6 +66,13 @@ pub const APP_EMBED_TOKEN_LABEL_PREFIX: &str = "embed_app:";
/// frontend SDK. Reserved in [`is_user_token`], whose SQL and frontend mirrors spell it out.
pub const RAW_APP_SDK_TOKEN_LABEL_PREFIX: &str = "sdk_app:";
/// Label prefix, followed by the username, of the token the CLI authorization page mints for
/// `wmill` logins. Reserved in [`is_user_token`], whose SQL and frontend mirrors spell it out:
/// the CLI signs in again on its own once that token expires, so an expiry email for it asks
/// the user to do nothing. Not in [`is_server_minted_label`], since the page mints it through
/// `/users/tokens/create`.
pub const CLI_LOGIN_TOKEN_LABEL_PREFIX: &str = "cli-login:";
/// Whether `label` belongs to a namespace only the server mints, and which therefore must be
/// rejected by `create_token`. Narrower than [`is_user_token`], which also drives label
/// editability and expiry notifications and can afford to reserve more: `Ephemeral lsp token`,
@@ -475,14 +489,16 @@ async fn fetch_authed_from_permissioned_as_inner(
// principal — a username freed and reassigned while its previous holder keeps a privileged
// account — would mix one account's role with another's instance privileges.
let member = match permissioned_as.split_once('/') {
Some(("u", name)) => sqlx::query!(
"SELECT is_admin, operator, email FROM usr where username = $1 AND \
Some(("u", name)) => {
sqlx::query!(
"SELECT is_admin, operator, email FROM usr where username = $1 AND \
workspace_id = $2 AND disabled = false",
name,
&w_id
)
.fetch_optional(&mut *conn)
.await?,
name,
&w_id
)
.fetch_optional(&mut *conn)
.await?
}
_ => None,
};
let resolved_email;
@@ -980,6 +996,7 @@ mod tests {
assert!(!is_user_token(Some("embed_app:f/team/dashboard")));
assert!(!is_user_token(Some("sdk_app:u/admin/raw app")));
assert!(!is_user_token(Some("impersonation:admin@windmill.dev")));
assert!(!is_user_token(Some("cli-login:admin")));
}
#[test]
@@ -15,8 +15,7 @@
use crate::error::Error;
/// What every roles path answers without the Enterprise Edition. The frontend matches this exact
/// sentence (`datatableUsableRoles.ts`) to read the refusal as "not under roles": reword both.
/// What every roles path answers without the Enterprise Edition.
pub fn datatable_roles_unavailable() -> Error {
Error::BadRequest("Data table roles are a Windmill Enterprise Edition feature".to_string())
}
@@ -1,412 +0,0 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
//! The external Postgres cluster behind `external_instance` data tables and Ducklake catalogs.
//!
//! Windmill administers that cluster itself, logged in as the user in
//! [`EXTERNAL_INSTANCE_PG_SETTING`]. It creates `custom_instance_user` and
//! `custom_instance_replication_user` there, with passwords it generates and keeps in
//! [`EXTERNAL_INSTANCE_PG_STATE_SETTING`]. They share their names with the roles on Windmill's own
//! cluster, but they are different roles with different passwords.
//!
//! The cluster may hold data Windmill did not create. Two Windmill instances sharing one is not
//! supported: each would keep resetting the passwords the other depends on.
use std::collections::{BTreeMap, BTreeSet};
use serde::{Deserialize, Serialize};
use crate::{
error::{Error, Result},
global_settings::{EXTERNAL_INSTANCE_PG_SETTING, EXTERNAL_INSTANCE_PG_STATE_SETTING},
instance_config::{CustomInstanceDb, ExternalInstancePg},
DB,
};
/// What Windmill keeps about the external cluster. Server-managed and hidden: never part of the
/// instance config, never readable by an agent worker. No `Debug`: it carries live passwords.
#[derive(Serialize, Deserialize, Clone, Default)]
pub struct ExternalInstancePgState {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub user_pwd: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub replication_pwd: Option<String>,
/// The databases Windmill created on the cluster. It only ever drops one of these.
#[serde(default)]
pub databases: BTreeMap<String, CustomInstanceDb>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub last_setup: Option<ExternalInstancePgSetupReport>,
/// The cluster ([`external_instance_pg_address`]) the last successful setup converged. Databases
/// are only created on a cluster setup succeeded on: the passwords above exist as soon as setup
/// first runs, whether or not the cluster accepted them.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub set_up_for: Option<String>,
}
/// What identifies the cluster a configuration points at. Other fields (admin login, sslmode) can
/// change without it becoming another cluster.
pub fn external_instance_pg_address(config: &ExternalInstancePg) -> String {
format!(
"{}:{}",
config.host.trim().to_lowercase(),
config.port.unwrap_or(5432)
)
}
#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ExternalInstancePgSetupReport {
/// No step failed. Warnings leave it true.
pub success: bool,
pub finished_at: chrono::DateTime<chrono::Utc>,
pub steps: Vec<ExternalInstancePgSetupStep>,
}
#[derive(Serialize, Deserialize, Clone, Debug)]
pub struct ExternalInstancePgSetupStep {
pub name: String,
pub status: SetupStepStatus,
pub message: String,
}
#[derive(Serialize, Deserialize, Clone, Copy, Debug, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
pub enum SetupStepStatus {
Ok,
Warning,
Error,
}
/// The status the settings page shows without running anything.
#[derive(Serialize, Debug)]
pub struct ExternalInstancePgStatus {
pub configured: bool,
pub database_count: usize,
#[serde(skip_serializing_if = "Option::is_none")]
pub last_setup: Option<ExternalInstancePgSetupReport>,
}
/// Authorization: returns the cluster's admin password and checks nothing. Callers MUST be
/// superadmin or an internal server path.
pub(crate) async fn read_external_instance_pg_config<'c>(
executor: impl sqlx::PgExecutor<'c>,
) -> Result<Option<ExternalInstancePg>> {
let value = sqlx::query_scalar!(
"SELECT value FROM global_settings WHERE name = $1",
EXTERNAL_INSTANCE_PG_SETTING
)
.fetch_optional(executor)
.await?;
value
.map(|v| {
serde_json::from_value(v).map_err(|e| {
Error::internal_err(format!("reading {EXTERNAL_INSTANCE_PG_SETTING}: {e}"))
})
})
.transpose()
}
/// Authorization: returns the passwords Windmill generated on the cluster and checks nothing.
/// Callers MUST be superadmin or an internal server path.
pub(crate) async fn read_external_instance_pg_state<'c>(
executor: impl sqlx::PgExecutor<'c>,
) -> Result<ExternalInstancePgState> {
let value = sqlx::query_scalar!(
"SELECT value FROM global_settings WHERE name = $1",
EXTERNAL_INSTANCE_PG_STATE_SETTING
)
.fetch_optional(executor)
.await?;
match value {
None => Ok(ExternalInstancePgState::default()),
Some(v) => serde_json::from_value(v).map_err(|e| {
Error::internal_err(format!("reading {EXTERNAL_INSTANCE_PG_STATE_SETTING}: {e}"))
}),
}
}
pub async fn external_instance_pg_status(db: &DB) -> Result<ExternalInstancePgStatus> {
let configured = read_external_instance_pg_config(db).await?.is_some();
let state = read_external_instance_pg_state(db).await?;
Ok(ExternalInstancePgStatus {
configured,
database_count: state.databases.len(),
last_setup: state.last_setup,
})
}
/// The databases Windmill created on the external cluster, without the passwords kept beside them.
///
/// Authorization: names every database across all workspaces, and the workspace each fork copy is
/// reserved for, and checks nothing. Callers MUST be superadmin or an internal authorization or
/// lifecycle path that does not return the names to a workspace caller.
pub async fn external_instance_databases(db: &DB) -> Result<BTreeMap<String, CustomInstanceDb>> {
Ok(read_external_instance_pg_state(db).await?.databases)
}
/// The workspaces whose data tables name each database on the external cluster.
///
/// Authorization: reads every workspace's settings and checks nothing. Callers MUST be superadmin
/// or an internal lifecycle path.
pub async fn external_instance_database_usages<'c>(
db: impl sqlx::PgExecutor<'c>,
) -> Result<BTreeMap<String, BTreeSet<String>>> {
let rows = sqlx::query_as::<_, (String, String)>(
"SELECT ws.workspace_id, entry->'database'->>'resource_path'
FROM workspace_settings ws
CROSS JOIN LATERAL jsonb_each(
CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
THEN ws.datatable->'datatables'
ELSE '{}'::jsonb END
) AS dt(k, entry)
WHERE entry->'database'->>'resource_type' = 'external_instance'
AND entry->'database'->>'resource_path' IS NOT NULL",
)
.fetch_all(db)
.await?;
let mut usages: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
for (workspace_id, dbname) in rows {
usages.entry(dbname).or_default().insert(workspace_id);
}
Ok(usages)
}
/// Refuse to unset the cluster while Windmill still has databases on it, or a workspace still
/// points at one: every data table there would stop resolving. Allowed on every edition, so a
/// downgraded instance can still clear a setting it no longer uses.
pub async fn ensure_external_instance_pg_removable(conn: &mut sqlx::PgConnection) -> Result<()> {
let state = read_external_instance_pg_state(&mut *conn).await?;
let usages = external_instance_database_usages(&mut *conn).await?;
if state.databases.is_empty() && usages.is_empty() {
return Ok(());
}
let names = state
.databases
.keys()
.chain(usages.keys())
.collect::<BTreeSet<_>>()
.into_iter()
.cloned()
.collect::<Vec<_>>()
.join(", ");
Err(Error::BadRequest(format!(
"The external instance cluster still holds databases in use ({names}). Drop them and \
repoint the data tables using them before removing {EXTERNAL_INSTANCE_PG_SETTING}."
)))
}
/// Refuse a workspace setting that newly names an `external_instance` database on an edition
/// without them.
pub fn ensure_external_instance_available() -> Result<()> {
crate::external_instance_pg_oss::ensure_external_instance_available()
}
/// The connection an `external_instance` database resolves to: `custom_instance_user`, or the
/// replication user, on the external cluster.
///
/// Authorization: returns live credentials and checks nothing. Callers MUST have authorized access
/// to the data table that names `dbname`.
pub async fn external_instance_connection_unchecked(
db: &DB,
dbname: &str,
replication: bool,
) -> Result<crate::PgDatabase> {
crate::external_instance_pg_oss::external_instance_connection_unchecked(db, dbname, replication)
.await
}
/// Create `dbname` on the external cluster and register it. Refuses a name already taken there,
/// whoever took it.
///
/// Authorization: checks nothing. Callers MUST be superadmin, or be cloning a data table they may
/// fork into a `wm_fork_` database.
pub async fn create_external_instance_database_unchecked(
db: &DB,
dbname: &str,
tag: &str,
for_workspace: Option<&str>,
) -> Result<()> {
crate::external_instance_pg_oss::create_external_instance_database_unchecked(
db,
dbname,
tag,
for_workspace,
)
.await
}
/// Drop `dbname` from the external cluster: only a database Windmill registered creating, and still
/// carries the mark it set there. Refused while anything uses it
/// ([`crate::workspaces::managed_database_uses`]), except the `exempt` data table entry: the fork
/// copy being cleaned up.
///
/// Authorization: checks nothing. Callers MUST be superadmin, or be deleting the fork that owns
/// this `wm_fork_` database.
pub async fn drop_external_instance_database_unchecked(
db: &DB,
dbname: &str,
exempt: Option<(&str, &str)>,
) -> Result<()> {
crate::external_instance_pg_oss::drop_external_instance_database_unchecked(db, dbname, exempt)
.await
}
/// Serializes everything that changes which databases exist on the external cluster, or which data
/// tables name them: setup, creates, drops, and data table saves. Held until `tx` ends.
pub async fn lock_external_instance_pg_state(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
) -> Result<()> {
sqlx::query("SELECT pg_advisory_xact_lock(hashtext($1))")
.bind(EXTERNAL_INSTANCE_PG_STATE_SETTING)
.execute(&mut **tx)
.await?;
Ok(())
}
/// Refuse a data table naming `dbname` unless Windmill created it on the external cluster. Takes
/// the lock drops take, so none can remove the database before `tx`, which saves the data table,
/// commits.
///
/// Authorization: its refusal says whether Windmill created a database of that name, which is
/// instance-wide knowledge. Callers MUST have authorized the caller as superadmin first.
pub async fn ensure_external_instance_database_registered(
tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
dbname: &str,
) -> Result<()> {
lock_external_instance_pg_state(tx).await?;
if read_external_instance_pg_state(&mut **tx)
.await?
.databases
.contains_key(dbname)
{
return Ok(());
}
Err(Error::BadRequest(format!(
"Windmill did not create a database named '{dbname}' on the external instance cluster. \
Create it from the instance settings first."
)))
}
/// Write [`EXTERNAL_INSTANCE_PG_SETTING`]: `None`, null or an empty string unsets it. Every writer
/// of global settings goes through this for that key — the per-key and bulk endpoints as well as
/// the declarative sync — instead of writing the row itself.
///
/// The checks and the write share one transaction holding [`lock_external_instance_pg_state`]. A
/// check taken outside it could pass while a database create still reads the old cluster, which
/// would then register a database there after the setting names another one.
///
/// Authorization: checks nothing. Callers MUST be superadmin, or the declarative instance config
/// sync, which applies what the operator deployed.
pub async fn write_external_instance_pg_setting(
db: &DB,
value: Option<&serde_json::Value>,
) -> Result<()> {
let value = match value {
None | Some(serde_json::Value::Null) => None,
Some(serde_json::Value::String(s)) if s.trim().is_empty() => None,
Some(value) => Some(value),
};
let mut tx = db.begin().await?;
lock_external_instance_pg_state(&mut tx).await?;
// Every check runs on this transaction's own connection: it holds the advisory lock, and
// taking a second connection from the pool while other writers queue on that lock is how a
// small pool deadlocks.
match value {
None => {
ensure_external_instance_pg_removable(&mut tx).await?;
sqlx::query("DELETE FROM global_settings WHERE name = $1")
.bind(EXTERNAL_INSTANCE_PG_SETTING)
.execute(&mut *tx)
.await?;
}
Some(value) => {
crate::external_instance_pg_oss::validate_external_instance_pg_setting(value)?;
ensure_external_instance_pg_not_repointed(&mut tx, value).await?;
sqlx::query(
"INSERT INTO global_settings (name, value) VALUES ($1, $2)
ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value, updated_at = now()",
)
.bind(EXTERNAL_INSTANCE_PG_SETTING)
.bind(value)
.execute(&mut *tx)
.await?;
}
}
tx.commit().await?;
tracing::info!(
"{} global setting {EXTERNAL_INSTANCE_PG_SETTING}",
if value.is_some() { "Set" } else { "Unset" }
);
Ok(())
}
/// [`write_external_instance_pg_setting`] for a settings diff: writes the key if the diff touches
/// it, and takes it out of the diff so the generic apply does not write it again.
///
/// Authorization: checks nothing. Callers MUST be superadmin, or the declarative instance config
/// sync, which applies what the operator deployed.
pub async fn write_external_instance_pg_from_diff(
db: &DB,
diff: &mut crate::instance_config::SettingsDiff,
) -> Result<()> {
if let Some(value) = diff.upserts.remove(EXTERNAL_INSTANCE_PG_SETTING) {
write_external_instance_pg_setting(db, Some(&value)).await?;
}
if let Some(i) = diff
.deletes
.iter()
.position(|k| k == EXTERNAL_INSTANCE_PG_SETTING)
{
diff.deletes.remove(i);
write_external_instance_pg_setting(db, None).await?;
}
Ok(())
}
/// Refuse pointing the setting at another host or port while databases live on the current one.
/// Data tables name databases, not clusters, so they would silently resolve to whatever the new
/// cluster holds under the same names. Other fields (admin login, sslmode) may change freely.
async fn ensure_external_instance_pg_not_repointed(
conn: &mut sqlx::PgConnection,
value: &serde_json::Value,
) -> Result<()> {
let Some(current) = read_external_instance_pg_config(&mut *conn).await? else {
return Ok(());
};
let Ok(desired) = serde_json::from_value::<ExternalInstancePg>(value.clone()) else {
return Ok(());
};
if external_instance_pg_address(&current) == external_instance_pg_address(&desired) {
return Ok(());
}
let state = read_external_instance_pg_state(&mut *conn).await?;
let usages = external_instance_database_usages(&mut *conn).await?;
if state.databases.is_empty() && usages.is_empty() {
return Ok(());
}
Err(Error::BadRequest(format!(
"The external instance cluster at {}:{} still holds databases in use. Drop them and repoint \
what uses them before pointing {EXTERNAL_INSTANCE_PG_SETTING} at another cluster.",
current.host.trim(),
current.port.unwrap_or(5432)
)))
}
/// Converge the external cluster on the configured login: check what it can do, create or update
/// Windmill's two roles with the stored passwords, and report anything that would get in the way.
/// With `rotate_passwords`, generate new passwords first. Safe to run again; running it again is
/// how a failed rotation is repaired.
///
/// Authorization: administers the external cluster with its admin credentials and checks nothing.
/// Callers MUST be superadmin.
pub async fn setup_external_instance_pg_unchecked(
db: &DB,
rotate_passwords: bool,
) -> Result<ExternalInstancePgSetupReport> {
crate::external_instance_pg_oss::setup_external_instance_pg_unchecked(db, rotate_passwords)
.await
}
@@ -1,81 +0,0 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
//! Where the external instance cluster comes from: the enterprise implementation, or a refusal.
//! `private` alone is not that edition: community builds carry it.
use crate::error::Error;
pub fn external_instance_pg_unavailable() -> Error {
Error::BadRequest(
"External instance databases are a Windmill Enterprise Edition feature".to_string(),
)
}
#[cfg(all(feature = "private", feature = "enterprise"))]
pub(crate) use crate::external_instance_pg_ee::{
create_external_instance_database_unchecked, drop_external_instance_database_unchecked,
external_instance_connection_unchecked, setup_external_instance_pg_unchecked,
validate_external_instance_pg_setting,
};
#[cfg(all(feature = "private", feature = "enterprise"))]
pub(crate) fn ensure_external_instance_available() -> crate::error::Result<()> {
Ok(())
}
#[cfg(not(all(feature = "private", feature = "enterprise")))]
pub(crate) use ce::*;
#[cfg(not(all(feature = "private", feature = "enterprise")))]
mod ce {
use super::external_instance_pg_unavailable as unavailable;
use crate::{
error::Result, external_instance_pg::ExternalInstancePgSetupReport, PgDatabase, DB,
};
pub(crate) fn validate_external_instance_pg_setting(_value: &serde_json::Value) -> Result<()> {
Err(unavailable())
}
pub(crate) fn ensure_external_instance_available() -> Result<()> {
Err(unavailable())
}
pub(crate) async fn setup_external_instance_pg_unchecked(
_db: &DB,
_rotate_passwords: bool,
) -> Result<ExternalInstancePgSetupReport> {
Err(unavailable())
}
pub(crate) async fn external_instance_connection_unchecked(
_db: &DB,
_dbname: &str,
_replication: bool,
) -> Result<PgDatabase> {
Err(unavailable())
}
pub(crate) async fn create_external_instance_database_unchecked(
_db: &DB,
_dbname: &str,
_tag: &str,
_for_workspace: Option<&str>,
) -> Result<()> {
Err(unavailable())
}
pub(crate) async fn drop_external_instance_database_unchecked(
_db: &DB,
_dbname: &str,
_exempt: Option<(&str, &str)>,
) -> Result<()> {
Err(unavailable())
}
}
+93 -5
View File
@@ -57,8 +57,6 @@ pub const SAML_METADATA_SETTING: &str = "saml_metadata";
pub const SMTP_SETTING: &str = "smtp_settings";
pub const TEAMS_SETTING: &str = "teams";
pub const INDEXER_SETTING: &str = "indexer_settings";
pub const EXTERNAL_INSTANCE_PG_SETTING: &str = "external_instance_pg";
pub const EXTERNAL_INSTANCE_PG_STATE_SETTING: &str = "external_instance_pg_state";
pub const TIMEOUT_WAIT_RESULT_SETTING: &str = "timeout_wait_result";
pub const UNIQUE_ID_SETTING: &str = "uid";
@@ -104,6 +102,49 @@ pub const HUB_API_SECRET_SETTING: &str = "hub_api_secret";
pub const AUTOMATE_USERNAME_CREATION_SETTING: &str = "automate_username_creation";
pub const DISABLE_WORKSPACE_INVITE_EMAILS_SETTING: &str = "disable_workspace_invite_emails";
pub const DISABLE_PASSWORD_LOGIN_SETTING: &str = "disable_password_login";
/// Refuse `?token=` on the MCP endpoints, leaving the `Authorization` header as the only way
/// in. A URL-borne credential ends up in browser history, proxy logs and referrers, so an
/// instance that cares sends MCP clients through the OAuth flow instead.
pub const MCP_DISABLE_TOKEN_QUERY_PARAM_SETTING: &str = "mcp_disable_token_query_param";
/// Ceiling, in days, on how far ahead a token minted through `POST /users/tokens/create` or
/// `POST /users/tokens/impersonate` may expire; a request asking for more, or for no
/// expiration at all, is shortened to it rather than refused. On those routes only: server-side
/// mints (webhook tokens, app embed tokens, sessions) choose a lifetime the caller never picks
/// and go straight to `create_token_internal`. Read and validated by
/// [`parse_max_token_expiration_days`].
pub const MAX_TOKEN_EXPIRATION_DAYS_SETTING: &str = "max_token_expiration_days";
/// Largest `max_token_expiration_days` read as a ceiling, about 2,700 years. The token form
/// applies the same bound (`frontend/src/lib/tokenExpiration.ts`) so that it and the server
/// agree on whether a ceiling exists.
pub const MAX_TOKEN_EXPIRATION_DAYS_BOUND: i64 = 1_000_000;
/// Reads a stored `max_token_expiration_days`: `Ok(None)` when unset or cleared (null or an
/// empty string), the ceiling for a whole number of days within
/// `1..=MAX_TOKEN_EXPIRATION_DAYS_BOUND` stored as an integer, an integral float or a string of
/// digits, and an error for anything else.
///
/// The settings API and config sync both reject the error at write time: the token routes can
/// only read an unparseable value as no ceiling, so accepting a typo would silently turn the
/// policy off. `parseMaxTokenExpirationDays` in the frontend must accept exactly the same values.
pub fn parse_max_token_expiration_days(
value: Option<&serde_json::Value>,
) -> Result<Option<i64>, String> {
let days = match value {
None | Some(serde_json::Value::Null) => return Ok(None),
Some(serde_json::Value::String(s)) if s.trim().is_empty() => return Ok(None),
Some(serde_json::Value::Number(n)) => n
.as_i64()
.or_else(|| n.as_f64().filter(|f| f.fract() == 0.0).map(|f| f as i64)),
Some(serde_json::Value::String(s)) => s.trim().parse::<i64>().ok(),
Some(_) => None,
};
match days {
Some(days) if (1..=MAX_TOKEN_EXPIRATION_DAYS_BOUND).contains(&days) => Ok(Some(days)),
_ => Err(format!(
"must be a whole number of days from 1 to {MAX_TOKEN_EXPIRATION_DAYS_BOUND}, or empty for no limit"
)),
}
}
pub const AUTO_LOGIN_PROVIDER_SETTING: &str = "auto_login_provider";
/// Name of the SAML attribute or OIDC userinfo claim carrying the user's IdP groups. Unset or
/// empty leaves instance-group membership entirely to SCIM.
@@ -357,9 +398,6 @@ pub const AGENT_WORKER_BLOCKED_SETTINGS: &[&str] = &[
// resolve datatable connections through the dedicated datatable endpoints, never these.
"custom_instance_pg_databases",
"custom_instance_replication_pwd",
// The external cluster's admin login, and the passwords Windmill generated on it.
EXTERNAL_INSTANCE_PG_SETTING,
EXTERNAL_INSTANCE_PG_STATE_SETTING,
];
/// Whether an agent worker may read the given global setting over HTTP.
@@ -373,6 +411,7 @@ use std::sync::atomic::AtomicBool;
lazy_static::lazy_static! {
pub static ref HTTP_ROUTE_WORKSPACED_ROUTE: AtomicBool = AtomicBool::new(false);
pub static ref DISABLE_PASSWORD_LOGIN: AtomicBool = AtomicBool::new(false);
pub static ref MCP_DISABLE_TOKEN_QUERY_PARAM: AtomicBool = AtomicBool::new(false);
/// Origins HTTP routes allow cross-origin when they configure none of their
/// own. Empty means unset, which keeps the historical `*`.
pub static ref HTTP_ROUTE_DEFAULT_ALLOWED_ORIGINS: arc_swap::ArcSwap<Vec<String>> =
@@ -816,6 +855,55 @@ pub fn workspace_integration_auth_endpoint(client_name: &str, base_url: &str) ->
mod tests {
use super::*;
// `frontend/src/lib/tokenExpiration.test.ts` holds the same table for the token form's
// parser; the two must stay in step.
#[test]
fn max_token_expiration_days_accepts_only_whole_days_within_the_bound() {
use serde_json::json;
for (stored, days) in [
(json!(7), 7),
(json!(7.0), 7),
(json!("7"), 7),
(json!(" 30 "), 30),
(json!("+7"), 7),
(
json!(MAX_TOKEN_EXPIRATION_DAYS_BOUND),
MAX_TOKEN_EXPIRATION_DAYS_BOUND,
),
] {
assert_eq!(
parse_max_token_expiration_days(Some(&stored)),
Ok(Some(days)),
"{stored}"
);
}
for cleared in [json!(null), json!(""), json!(" ")] {
assert_eq!(
parse_max_token_expiration_days(Some(&cleared)),
Ok(None),
"{cleared}"
);
}
assert_eq!(parse_max_token_expiration_days(None), Ok(None));
for bad in [
json!(7.5),
json!(0),
json!(-3),
json!("7.0"),
json!("1e1"),
json!("0x7"),
json!(MAX_TOKEN_EXPIRATION_DAYS_BOUND + 1),
json!("99999999999999999999"),
json!(true),
json!([7]),
] {
assert!(
parse_max_token_expiration_days(Some(&bad)).is_err(),
"{bad} must be rejected"
);
}
}
#[test]
fn webhook_base_url_errors_never_echo_credentials() {
// These strings reach sync-config output and operator logs, so no branch may
+36 -71
View File
@@ -350,8 +350,6 @@ pub struct GlobalSettings {
pub ducklake_settings: Option<DucklakeSettings>,
#[serde(skip_serializing_if = "Option::is_none")]
pub custom_instance_pg_databases: Option<CustomInstancePgDatabases>,
#[serde(skip_serializing_if = "Option::is_none")]
pub external_instance_pg: Option<ExternalInstancePg>,
// Opaque settings (EE-private structs or no clear schema)
#[serde(skip_serializing_if = "Option::is_none")]
@@ -811,9 +809,6 @@ pub struct CustomInstanceDb {
pub error: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tag: Option<String>,
/// The workspace a member created this fork copy for. Absent when a superadmin created it.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub workspace_id: Option<String>,
}
/// Setup log entries for a custom instance database.
@@ -838,36 +833,6 @@ pub struct CustomInstanceDbLogs {
pub replication_user_error: Option<String>,
}
// ---------------------------------------------------------------------------
// External instance PG cluster
// ---------------------------------------------------------------------------
/// The external Postgres cluster Windmill manages for `external_instance` data tables and Ducklake
/// catalogs. `user` logs in as the cluster's administrator: it needs `CREATEDB` and `CREATEROLE`.
/// `dbname` is only where that login connects to run cluster-wide statements.
///
/// Every field defaults rather than being required: this deserializes as part of the whole
/// instance config, and one malformed row must not make every other setting unreadable. The
/// write path and every use reject an incomplete value instead.
#[derive(Deserialize, Serialize, Clone, Debug, Default)]
#[cfg_attr(feature = "instance_config_schema", derive(schemars::JsonSchema))]
pub struct ExternalInstancePg {
#[serde(default)]
pub host: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub port: Option<u16>,
#[serde(default)]
pub user: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub password: Option<StringOrSecretRef>,
#[serde(skip_serializing_if = "Option::is_none")]
pub dbname: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub sslmode: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub root_certificate_pem: Option<String>,
}
// ---------------------------------------------------------------------------
// Autoscaling (worker config)
// ---------------------------------------------------------------------------
@@ -1004,7 +969,6 @@ pub const PROTECTED_SETTINGS: &[&str] = &[
"ducklake_settings",
"custom_instance_pg_databases",
"custom_instance_replication_pwd",
"external_instance_pg_state",
"uid",
"rsa_keys",
"jwt_secret",
@@ -1030,8 +994,6 @@ pub const HIDDEN_SETTINGS: &[&str] = &[
// Server-only (written by setup/refresh via direct SQL), never operator-authored —
// hidden so the config machinery can't read, rewrite, or drop it.
"custom_instance_replication_pwd",
// Same for the passwords and database registry Windmill keeps for the external cluster.
"external_instance_pg_state",
];
/// Top-level settings whose entire value is sensitive and must be fully redacted in logs.
@@ -1043,7 +1005,6 @@ const SENSITIVE_SETTINGS: &[&str] = &[
"license_key",
"ducklake_user_pg_pwd",
"custom_instance_replication_pwd",
"external_instance_pg_state",
"pip_index_url",
"pip_extra_index_url",
"npm_config_registry",
@@ -1069,7 +1030,6 @@ const NESTED_SENSITIVE_FIELDS: &[(&str, &[&str])] = &[
&["secret_key", "serviceAccountKey"],
),
("custom_instance_pg_databases", &["user_pwd"]),
("external_instance_pg", &["password"]),
];
fn redact_json_value(value: &serde_json::Value) -> serde_json::Value {
@@ -1398,8 +1358,11 @@ pub async fn sync_global_settings_declarative(
crate::global_settings::parse_allowed_origins_setting(desired.get(origins_key))
.map_err(|e| anyhow::anyhow!("{origins_key}: {e}"))?;
let mut diff = diff_global_settings(current, desired, ApplyMode::Replace);
crate::external_instance_pg::write_external_instance_pg_from_diff(db, &mut diff).await?;
let max_expiration_key = crate::global_settings::MAX_TOKEN_EXPIRATION_DAYS_SETTING;
crate::global_settings::parse_max_token_expiration_days(desired.get(max_expiration_key))
.map_err(|e| anyhow::anyhow!("{max_expiration_key}: {e}"))?;
let diff = diff_global_settings(current, desired, ApplyMode::Replace);
apply_settings_diff(db, &diff).await?;
Ok(())
@@ -1532,10 +1495,6 @@ pub fn resolve_env_refs(settings: &mut GlobalSettings) -> Result<(), String> {
resolve_env_option(&mut pg.user_pwd)?;
}
if let Some(pg) = &mut settings.external_instance_pg {
resolve_env_option(&mut pg.password)?;
}
Ok(())
}
@@ -2505,33 +2464,39 @@ mod tests {
}
#[test]
fn server_generated_db_passwords_are_isolated_from_config() {
// These hold passwords the server generates: written by setup/refresh via direct SQL,
// never operator-authored. They must stay out of the declarative config surface
// (hidden on read) and be undeletable, so config sync can't read, rewrite, or drop them.
for key in [
"custom_instance_replication_pwd",
"external_instance_pg_state",
] {
assert!(HIDDEN_SETTINGS.contains(&key), "{key}");
assert!(PROTECTED_SETTINGS.contains(&key), "{key}");
assert!(SENSITIVE_SETTINGS.contains(&key), "{key}");
fn custom_instance_replication_pwd_is_isolated_from_config() {
// The replication-role password is server-only: written by setup/refresh via direct
// SQL, never operator-authored. It must stay out of the declarative config surface
// (hidden on read) and be undeletable, so config sync can't read, rewrite, or drop it.
assert!(HIDDEN_SETTINGS.contains(&"custom_instance_replication_pwd"));
assert!(PROTECTED_SETTINGS.contains(&"custom_instance_replication_pwd"));
assert!(SENSITIVE_SETTINGS.contains(&"custom_instance_replication_pwd"));
// A stray desired value (e.g. flattened into `extra`) is ignored, not upserted.
let mut desired = BTreeMap::new();
desired.insert(key.to_string(), serde_json::json!("attacker-set"));
let diff = diff_global_settings(&BTreeMap::new(), &desired, ApplyMode::Merge);
assert!(diff.upserts.is_empty(), "{key} must not be upserted");
// A stray desired value (e.g. flattened into `extra`) is ignored, not upserted.
let mut desired = BTreeMap::new();
desired.insert(
"custom_instance_replication_pwd".to_string(),
serde_json::json!("attacker-set"),
);
let diff = diff_global_settings(&BTreeMap::new(), &desired, ApplyMode::Merge);
assert!(
diff.upserts.is_empty(),
"hidden setting must not be upserted"
);
// A current value is never deleted by a Replace that omits it.
let mut current = BTreeMap::new();
current.insert(key.to_string(), serde_json::json!("live"));
let diff = diff_global_settings(&current, &BTreeMap::new(), ApplyMode::Replace);
assert!(
!diff.deletes.contains(&key.to_string()),
"{key} must not be deleted"
);
}
// A current value is never deleted by a Replace that omits it.
let mut current = BTreeMap::new();
current.insert(
"custom_instance_replication_pwd".to_string(),
serde_json::json!("live"),
);
let diff = diff_global_settings(&current, &BTreeMap::new(), ApplyMode::Replace);
assert!(
!diff
.deletes
.contains(&"custom_instance_replication_pwd".to_string()),
"hidden setting must not be deleted"
);
}
#[test]
+3 -77
View File
@@ -58,10 +58,6 @@ pub mod ee_oss;
pub mod email_ee;
pub mod email_oss;
pub mod error;
pub mod external_instance_pg;
#[cfg(all(feature = "private", feature = "enterprise"))]
mod external_instance_pg_ee;
pub mod external_instance_pg_oss;
pub mod external_ip;
#[cfg(feature = "private")]
pub mod feature_usage_ee;
@@ -1028,20 +1024,6 @@ impl Future for TokioPgConnection {
}
}
impl TokioPgConnection {
/// Drive the connection and hand back what the server sends outside of a query's response —
/// notices above all, which driving it as a future silently discards.
pub fn poll_message(
&mut self,
cx: &mut core::task::Context<'_>,
) -> core::task::Poll<Option<Result<tokio_postgres::AsyncMessage, tokio_postgres::Error>>> {
match self {
TokioPgConnection::Tls(conn) => conn.poll_message(cx),
TokioPgConnection::NoTls(conn) => conn.poll_message(cx),
}
}
}
impl PgDatabase {
/// The role the connection logs in as, whichever way it authenticates.
pub fn login_name(&self) -> &str {
@@ -1086,13 +1068,7 @@ impl PgDatabase {
if err_str.contains("password authentication failed for user")
&& err_str.contains("custom_instance_user")
{
// The external instance cluster has a `custom_instance_user` of its own, whose
// password setup manages. Rotating the local one would break every instance
// data table and fix nothing.
let local = PgDatabase::parse_uri(&get_database_url().await?.as_str().await)?;
let on_local_cluster = local.host == self.host
&& local.port.unwrap_or(5432) == self.port.unwrap_or(5432);
if let Some(db) = main_db.filter(|_| on_local_cluster) {
if let Some(db) = main_db {
tracing::warn!(
"custom_instance_user password auth failed, refreshing and retrying..."
);
@@ -1578,13 +1554,11 @@ pub async fn ensure_instance_db_grant_options_unchecked(
}
/// Create a custom instance database: CREATE DATABASE, grant permissions, register in global_settings.
/// The `tag` is stored in global_settings metadata (e.g. "datatable" or "ducklake"). `for_workspace`
/// is the workspace a member creates a fork copy for; see [`ensure_fork_database_available_to`].
/// The `tag` is stored in global_settings metadata (e.g. "datatable" or "ducklake").
pub async fn create_custom_instance_database(
db: &DB,
dbname: &str,
tag: &str,
for_workspace: Option<&str>,
) -> error::Result<()> {
let dbname = dbname.trim();
validate_dbname(dbname)?;
@@ -1638,8 +1612,7 @@ pub async fn create_custom_instance_database(
},
"success": true,
"error": null,
"tag": tag,
"workspace_id": for_workspace,
"tag": tag
});
sqlx::query!(
r#"UPDATE global_settings SET value = jsonb_set(value, '{databases}', (COALESCE(value->'databases', '{}'::jsonb) || to_jsonb($1::json))) WHERE name = 'custom_instance_pg_databases'"#,
@@ -1659,53 +1632,6 @@ pub async fn create_custom_instance_database(
Ok(())
}
/// Refuse a workspace member writing a fork copy into, or pointing a fork at, the managed database
/// `dbname` of `kind`, unless `w_id` created it for that ([`create_custom_instance_database`], or
/// its external instance counterpart) and nothing uses it yet. The `wm_fork_` prefix is no
/// authorization: every database of a cluster answers to the same `custom_instance_user`, so a name
/// is all it takes to reach another workspace's copy.
///
/// Authorization: reads the global registries and every workspace's settings, and names other
/// workspaces in its refusal. Callers MUST have authorized `w_id` for the caller first — a member
/// of it forking or importing there — and MUST NOT call it on a workspace the caller is not in.
pub async fn ensure_fork_database_available_to(
db: &DB,
kind: workspaces::DataTableCatalogResourceType,
dbname: &str,
w_id: &str,
) -> error::Result<()> {
let created_for = match kind {
workspaces::DataTableCatalogResourceType::ExternalInstance => {
external_instance_pg::external_instance_databases(db)
.await?
.remove(dbname)
.and_then(|entry| entry.workspace_id)
}
_ => sqlx::query_scalar::<_, Option<String>>(
"SELECT value->'databases'->$1->>'workspace_id' FROM global_settings
WHERE name = 'custom_instance_pg_databases'",
)
.bind(dbname)
.fetch_optional(db)
.await?
.flatten(),
};
if created_for.as_deref() != Some(w_id) {
return Err(Error::BadRequest(format!(
"Database '{dbname}' was not created for a fork of workspace '{w_id}'"
)));
}
let uses =
workspaces::managed_database_uses(&mut *db.acquire().await?, kind, dbname, None).await?;
if !uses.is_empty() {
return Err(Error::BadRequest(format!(
"Database '{dbname}' is already in use: {}",
uses.join(", ")
)));
}
Ok(())
}
/// Connection options parsed from a database URL.
///
/// The only place a database URL becomes `PgConnectOptions`. Providers that mint the password
+1 -36
View File
@@ -329,7 +329,6 @@ pub fn try_expand_internal_db_query(
"ALTER_TABLE" => expand_alter_table(json_str, db_type).map(ExpandedQuery::sql),
"CREATE_SCHEMA" => expand_create_schema(json_str, db_type).map(ExpandedQuery::sql),
"DROP_SCHEMA" => expand_drop_schema(json_str, db_type).map(ExpandedQuery::sql),
"RENAME_SCHEMA" => expand_rename_schema(json_str, db_type).map(ExpandedQuery::sql),
// Metadata queries
"LOAD_TABLE_METADATA" => expand_load_table_metadata(json_str, db_type),
"FOREIGN_KEYS" => expand_foreign_keys(json_str, db_type).map(ExpandedQuery::sql),
@@ -1717,13 +1716,6 @@ struct DropSchemaPayload {
ducklake: Option<String>,
}
#[derive(Deserialize)]
struct RenameSchemaPayload {
schema: String,
new_schema: String,
ducklake: Option<String>,
}
#[derive(Debug, Clone, Deserialize)]
struct TableEditorColumn {
name: String,
@@ -2012,23 +2004,6 @@ fn expand_drop_schema(json_str: &str, db_type: DbType) -> Result<String, String>
Ok(maybe_wrap_ducklake(query, p.ducklake.as_deref()))
}
fn expand_rename_schema(json_str: &str, db_type: DbType) -> Result<String, String> {
let p: RenameSchemaPayload = serde_json::from_str(json_str)
.map_err(|e| format!("Invalid RENAME_SCHEMA payload: {}", e))?;
if !matches!(db_type, DbType::Postgresql | DbType::Snowflake) || p.ducklake.is_some() {
return Err(format!(
"Renaming a schema is not supported on {:?}",
db_type
));
}
let query = format!(
"ALTER SCHEMA {} RENAME TO {};",
qi(&p.schema, db_type),
qi(&p.new_schema, db_type)
);
Ok(query)
}
fn expand_create_table(json_str: &str, db_type: DbType) -> Result<String, String> {
let p: CreateTablePayload = serde_json::from_str(json_str)
.map_err(|e| format!("Invalid CREATE_TABLE payload: {}", e))?;
@@ -2623,9 +2598,7 @@ WHERE table_catalog = current_database()",
)
} else {
(
// pg_catalog is readable by everyone: without the privilege check this lists
// tables of schemas the connection's role cannot even enter.
"\nWHERE c.relkind = 'r' AND a.attnum > 0 AND NOT a.attisdropped\n AND ns.nspname != 'pg_catalog' AND ns.nspname != 'information_schema'\n AND has_schema_privilege(ns.oid, 'USAGE')".to_string(),
"\nWHERE c.relkind = 'r' AND a.attnum > 0 AND NOT a.attisdropped\n AND ns.nspname != 'pg_catalog' AND ns.nspname != 'information_schema'".to_string(),
",\n ns.nspname AS schema_name,\n c.relname AS table_name".to_string(),
"\nJOIN pg_catalog.pg_class c ON a.attrelid = c.oid\nJOIN pg_catalog.pg_namespace ns ON c.relnamespace = ns.oid".to_string(),
"ns.nspname, c.relname, a.attnum".to_string(),
@@ -4128,13 +4101,6 @@ mod tests {
assert_eq!(sql, "DROP SCHEMA \"old_schema\" CASCADE;");
}
#[test]
fn test_expand_rename_schema() {
let marker = r#"-- WM_INTERNAL_DB_RENAME_SCHEMA {"schema":"old","new_schema":"new"}"#;
let sql = expand_code(marker, &ScriptLang::Postgresql);
assert_eq!(sql, "ALTER SCHEMA \"old\" RENAME TO \"new\";");
}
#[test]
fn test_expand_create_schema_with_ducklake() {
let marker = r#"-- WM_INTERNAL_DB_CREATE_SCHEMA {"schema":"s","ducklake":"lake"}"#;
@@ -4502,7 +4468,6 @@ mod tests {
assert!(sql.contains("schema_name"));
assert!(sql.contains("table_name"));
assert!(sql.contains("c.relkind = 'r'"));
assert!(sql.contains("has_schema_privilege(ns.oid, 'USAGE')"));
}
#[test]
@@ -10,7 +10,7 @@
use std::borrow::Cow;
use std::collections::{HashMap, HashSet};
use std::sync::RwLock;
use std::sync::{Arc, RwLock};
use uuid::Uuid;
/// Minimum length for a secret to be registered for masking.
@@ -20,9 +20,27 @@ const MIN_SECRET_LENGTH: usize = 8;
const MASKED_NOTICE: &str =
"[windmill] secret value was masked for security reasons, use string transformations to display full value";
/// The secrets registered for one job, plus the automaton compiled from them.
#[derive(Default)]
struct JobMasks {
secrets: HashSet<String>,
/// Built on the first `snapshot` after a change and shared by every later
/// snapshot. Every job registers at least its own token, so without this
/// cache each log batch of each job would rebuild the automaton.
compiled: Option<Arc<CompiledMasks>>,
}
/// Aho-Corasick automaton for O(m) multi-pattern matching in a single pass,
/// regardless of the number of secrets registered, with the replacement
/// strings indexed to match the automaton's pattern order.
struct CompiledMasks {
ac: aho_corasick::AhoCorasick,
replacements: Vec<String>,
}
lazy_static::lazy_static! {
/// Map of job_id -> set of secret values that should be masked in that job's logs.
static ref SENSITIVE_MASKS: RwLock<HashMap<Uuid, HashSet<String>>> =
/// Map of job_id -> secret values that should be masked in that job's logs.
static ref SENSITIVE_MASKS: RwLock<HashMap<Uuid, JobMasks>> =
RwLock::new(HashMap::new());
/// Set of currently running job IDs on this worker process.
@@ -32,13 +50,8 @@ lazy_static::lazy_static! {
}
/// A lock-free snapshot of secrets for a job, taken once per log batch.
/// Uses Aho-Corasick for O(m) multi-pattern matching in a single pass,
/// regardless of the number of secrets registered.
pub struct MaskSnapshot {
/// Aho-Corasick automaton for fast matching.
ac: aho_corasick::AhoCorasick,
/// Replacement strings, indexed to match the automaton's pattern order.
replacements: Vec<String>,
compiled: Arc<CompiledMasks>,
/// Whether the security notice has already been appended for this snapshot.
/// Tracked locally to avoid a global write lock on every masked line.
notice_shown: std::cell::Cell<bool>,
@@ -53,34 +66,104 @@ impl MaskSnapshot {
}
// Single-pass check + replace using the pre-built automaton
if !self.ac.is_match(text) {
if !self.compiled.ac.is_match(text) {
return Cow::Borrowed(text);
}
let mut result = self.ac.replace_all(text, &self.replacements);
let mut result = self
.compiled
.ac
.replace_all(text, &self.compiled.replacements);
// Append the notice only once per snapshot (i.e. per batch)
// Append the notice only once per snapshot (i.e. per batch), as its own line.
// Callers pass either a bare line (`handle_child`) or a chunk that already ends
// in a newline (nativets), and the sinks concatenate what they get verbatim:
// assuming either shape welds the notice onto a neighbouring line.
if !self.notice_shown.get() {
self.notice_shown.set(true);
result.push('\n');
result.push_str(MASKED_NOTICE);
if result.ends_with('\n') {
result.push_str(MASKED_NOTICE);
result.push('\n');
} else {
result.push('\n');
result.push_str(MASKED_NOTICE);
}
}
Cow::Owned(result)
}
}
/// A masker for sinks that mask line by line rather than in batches, like nativets
/// masking each `console.log` chunk as V8 produces it. `snapshot` per line would
/// re-arm the security notice on every one; this keeps it to once per distinct set
/// of secrets while still picking up secrets registered mid-run.
///
/// Masks by job id alone — the caller is the one that knows the text it passes
/// belongs to that job.
pub struct JobMasker {
job_id: Uuid,
snapshot: Option<MaskSnapshot>,
}
impl JobMasker {
pub fn new(job_id: Uuid) -> Self {
JobMasker { job_id, snapshot: snapshot(&job_id) }
}
/// Mask every secret registered for the job. Returns `Cow::Borrowed` when no match.
/// Falls back to the masks it last saw once the job is unregistered, so a sink
/// still draining past the end of a run does not start emitting secrets.
pub fn mask<'a>(&mut self, text: &'a str) -> Cow<'a, str> {
if let Some(fresh) = snapshot(&self.job_id) {
// Replacing an equivalent snapshot would re-arm the notice, so only take
// one built from a secret set we have not seen.
let unchanged = self
.snapshot
.as_ref()
.is_some_and(|cur| Arc::ptr_eq(&cur.compiled, &fresh.compiled));
if !unchanged {
self.snapshot = Some(fresh);
}
}
match self.snapshot.as_ref() {
Some(snapshot) => snapshot.mask(text),
None => Cow::Borrowed(text),
}
}
}
/// Take a snapshot of the current secrets for a job. Returns `None` if no secrets
/// are registered (the caller can then skip masking entirely for the whole batch).
///
/// Call this once per log batch in `write_lines`, not per line.
pub fn snapshot(job_id: &Uuid) -> Option<MaskSnapshot> {
let masks = SENSITIVE_MASKS.read().unwrap_or_else(|e| e.into_inner());
let secrets = masks.get(job_id)?;
if secrets.is_empty() {
return None;
{
let masks = SENSITIVE_MASKS.read().unwrap_or_else(|e| e.into_inner());
let job = masks.get(job_id)?;
if job.secrets.is_empty() {
return None;
}
if let Some(compiled) = job.compiled.as_ref() {
return Some(MaskSnapshot {
compiled: compiled.clone(),
notice_shown: std::cell::Cell::new(false),
});
}
}
let mut masks = SENSITIVE_MASKS.write().unwrap_or_else(|e| e.into_inner());
let job = masks.get_mut(job_id)?;
if job.secrets.is_empty() {
return None;
}
let compiled = job
.compiled
.get_or_insert_with(|| Arc::new(compile(&job.secrets)));
Some(MaskSnapshot { compiled: compiled.clone(), notice_shown: std::cell::Cell::new(false) })
}
fn compile(secrets: &HashSet<String>) -> CompiledMasks {
// Sort longest-first so longer secrets are matched before shorter substrings
let mut sorted: Vec<&String> = secrets.iter().collect();
sorted.sort_by(|a, b| b.len().cmp(&a.len()));
@@ -106,7 +189,7 @@ pub fn snapshot(job_id: &Uuid) -> Option<MaskSnapshot> {
.build(sorted.iter().map(|s| s.as_str()))
.expect("failed to build aho-corasick automaton");
Some(MaskSnapshot { ac, replacements, notice_shown: std::cell::Cell::new(false) })
CompiledMasks { ac, replacements }
}
/// Register a job as currently running. Call this before `handle_queued_job`.
@@ -148,20 +231,110 @@ pub fn register_secret_for_all_running_jobs(secret: &str) {
let mut masks = SENSITIVE_MASKS.write().unwrap_or_else(|e| e.into_inner());
for job_id in job_ids {
if let Some(set) = masks.get_mut(&job_id) {
set.insert(secret.to_string());
if let Some(job) = masks.get_mut(&job_id) {
if job.secrets.insert(secret.to_string()) {
job.compiled = None;
}
}
}
}
/// Register a secret value for a specific job.
/// Used for `$encrypted:` args where we know the job ID.
/// Used for the job's own token and for `$encrypted:` args, where we know the job ID.
pub fn register_secret_for_job(job_id: Uuid, secret: &str) {
if secret.len() < MIN_SECRET_LENGTH {
return;
}
let mut masks = SENSITIVE_MASKS.write().unwrap_or_else(|e| e.into_inner());
if let Some(set) = masks.get_mut(&job_id) {
set.insert(secret.to_string());
if let Some(job) = masks.get_mut(&job_id) {
if job.secrets.insert(secret.to_string()) {
job.compiled = None;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The compiled automaton is cached per job, so a secret registered after the
/// first snapshot only gets masked if the cache is invalidated.
#[test]
fn snapshot_rebuilds_after_a_new_secret_is_registered() {
let job_id = Uuid::new_v4();
register_running_job(job_id);
register_secret_for_job(job_id, "firstsecretvalue");
let _ = snapshot(&job_id)
.expect("secret registered")
.mask("firstsecretvalue");
register_secret_for_job(job_id, "secondsecretvalue");
let snap = snapshot(&job_id).expect("secrets registered");
let masked = snap.mask("firstsecretvalue then secondsecretvalue");
assert!(!masked.contains("firstsecretvalue"), "{masked}");
assert!(!masked.contains("secondsecretvalue"), "{masked}");
unregister_running_job(job_id);
}
/// A line-by-line sink must not repeat the notice on every line, and must still
/// pick up a secret registered after the masker was built.
#[test]
fn job_masker_notices_once_per_secret_set() {
let job_id = Uuid::new_v4();
register_running_job(job_id);
register_secret_for_job(job_id, "firstsecretvalue");
let mut masker = JobMasker::new(job_id);
let first = masker.mask("saw firstsecretvalue").into_owned();
assert!(!first.contains("firstsecretvalue"), "{first}");
assert!(first.contains(MASKED_NOTICE), "{first}");
let second = masker.mask("saw firstsecretvalue again").into_owned();
assert!(!second.contains("firstsecretvalue"), "{second}");
assert!(!second.contains(MASKED_NOTICE), "{second}");
register_secret_for_job(job_id, "secondsecretvalue");
let third = masker.mask("saw secondsecretvalue").into_owned();
assert!(!third.contains("secondsecretvalue"), "{third}");
unregister_running_job(job_id);
}
/// Unregistration must not turn masking off under a sink that is still emitting:
/// the masker keeps working off the masks it last saw rather than going quiet.
#[test]
fn job_masker_masks_after_the_job_is_unregistered() {
let job_id = Uuid::new_v4();
register_running_job(job_id);
register_secret_for_job(job_id, "supersecretvalue");
let mut masker = JobMasker::new(job_id);
unregister_running_job(job_id);
let masked = masker.mask("logged supersecretvalue here");
assert!(!masked.contains("supersecretvalue"), "{masked}");
}
/// The notice has to end up on a line of its own for both shapes callers pass:
/// a bare line (`handle_child`) and a newline-terminated chunk (nativets). The
/// sinks concatenate what they are given verbatim, so getting this wrong welds
/// the notice onto whichever line follows it.
#[test]
fn notice_lands_on_its_own_line_for_both_caller_shapes() {
let job_id = Uuid::new_v4();
register_running_job(job_id);
register_secret_for_job(job_id, "supersecretvalue");
let line = snapshot(&job_id)
.expect("secret registered")
.mask("tok supersecretvalue");
assert_eq!(line, format!("tok s*****e\n{MASKED_NOTICE}"));
let chunk = snapshot(&job_id)
.expect("secret registered")
.mask("tok supersecretvalue\n");
assert_eq!(chunk, format!("tok s*****e\n{MASKED_NOTICE}\n"));
unregister_running_job(job_id);
}
}
+7 -108
View File
@@ -1397,18 +1397,6 @@ pub enum DataTableCatalogResourceType {
#[strum(serialize = "postgres")]
Postgresql,
Instance,
/// On the external instance cluster ([`crate::external_instance_pg`]). Enterprise Edition.
#[serde(rename = "external_instance")]
#[strum(serialize = "external_instance")]
ExternalInstance,
}
impl DataTableCatalogResourceType {
/// A database Windmill created and administers, on its own cluster or the external one, as
/// opposed to one a user brought as a resource.
pub fn is_windmill_managed(self) -> bool {
matches!(self, Self::Instance | Self::ExternalInstance)
}
}
/// Build a self-teaching error for an unresolved `datatable://<name>` reference.
@@ -1483,73 +1471,6 @@ pub struct GoverningDatatable {
pub datatable: DataTable,
}
/// Everything still using the Windmill-managed database `dbname`, one description per use: data
/// table entries naming it, fork entries pointing at those, Ducklake catalogs on it, and fork
/// Ducklake metadata schemas there that cleanup has not dropped yet. `exempt` is the one data table
/// entry, `(workspace_id, name)`, the caller is about to stop using it through; pointers at that
/// entry still count, since dropping the database would leave them resolving to nothing.
///
/// Authorization: reads every workspace's settings and checks nothing. Callers MUST only turn the
/// answer into a refusal for someone allowed to administer `dbname`.
pub async fn managed_database_uses(
conn: &mut sqlx::PgConnection,
kind: DataTableCatalogResourceType,
dbname: &str,
exempt: Option<(&str, &str)>,
) -> Result<Vec<String>> {
let (exempt_workspace, exempt_name) = exempt.unzip();
Ok(sqlx::query_scalar::<_, String>(
"WITH entries AS (
SELECT ws.workspace_id::text AS workspace_id, dt.key AS name, dt.value
FROM workspace_settings ws
CROSS JOIN LATERAL jsonb_each(
CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
THEN ws.datatable->'datatables' ELSE '{}'::jsonb END) dt
), naming AS (
SELECT workspace_id, name FROM entries
WHERE value->'database'->>'resource_type' = $1
AND value->'database'->>'resource_path' = $2
)
SELECT format('data table ''%s'' in workspace ''%s''', name, workspace_id) FROM naming
WHERE $3::text IS NULL OR NOT (workspace_id = $3 AND name = $4)
UNION ALL
SELECT format('data table ''%s'' in workspace ''%s'', which points at the one in ''%s''',
e.name, e.workspace_id, n.workspace_id)
FROM entries e JOIN naming n
ON e.value->'reference'->>'workspace_id' = n.workspace_id
AND e.value->'reference'->>'datatable' = n.name
UNION ALL
SELECT format('Ducklake ''%s'' in workspace ''%s''', dl.key, ws.workspace_id)
FROM workspace_settings ws
CROSS JOIN LATERAL jsonb_each(
CASE WHEN jsonb_typeof(ws.ducklake->'ducklakes') = 'object'
THEN ws.ducklake->'ducklakes' ELSE '{}'::jsonb END) dl
WHERE dl.value->'catalog'->>'resource_type' = $1
AND dl.value->'catalog'->>'resource_path' = $2
UNION ALL
SELECT format('the Ducklake namespace of fork ''%s'', not cleaned up yet', workspace_id)
FROM fork_ducklake_namespace
WHERE catalog = $1 || ':' || $2 AND NOT schema_dropped
ORDER BY 1",
)
.bind(kind.as_ref())
.bind(dbname)
.bind(exempt_workspace)
.bind(exempt_name)
.fetch_all(&mut *conn)
.await?)
}
/// Held by fork cleanup of `w_id`'s data tables and by forking `w_id`, which can hand the new fork
/// pointers at them, so a pointer cannot appear between cleanup's check and its drop.
pub async fn lock_fork_datatables(conn: &mut sqlx::PgConnection, w_id: &str) -> Result<()> {
sqlx::query("SELECT pg_advisory_xact_lock(hashtext('fork_datatables:' || $1))")
.bind(w_id)
.execute(&mut *conn)
.await?;
Ok(())
}
impl GoverningDatatable {
/// Backed by the Windmill instance's own Postgres, which is the only substrate data table
/// roles apply to.
@@ -1692,8 +1613,7 @@ pub async fn resolve_workspace_governing_datatables(
}
/// Build the `admin` connection for a governing entry: `custom_instance_user` for an instance
/// database, on Windmill's cluster or the external one; the user's own resource for a BYO-postgres
/// one.
/// database, the user's own resource for a BYO-postgres one.
async fn resolve_datatable_connection_unchecked(
db: &DB,
governing: &GoverningDatatable,
@@ -1704,16 +1624,7 @@ async fn resolve_datatable_connection_unchecked(
.database
.as_ref()
.expect("a governing entry owns a database");
if database.resource_type == DataTableCatalogResourceType::ExternalInstance {
let pg_creds = crate::external_instance_pg::external_instance_connection_unchecked(
db,
&database.resource_path,
replication,
)
.await?;
serde_json::to_value(&pg_creds)
.map_err(|e| Error::internal_err(format!("Error serializing pg creds: {}", e)))
} else if database.resource_type == DataTableCatalogResourceType::Instance {
if database.resource_type == DataTableCatalogResourceType::Instance {
let mut pg_creds = PgDatabase::parse_uri(&get_database_url().await?.as_str().await)?;
pg_creds.dbname = database.resource_path.clone();
if replication {
@@ -2056,8 +1967,6 @@ pub fn strip_datatable_permissions(
/// As [`parse_datatable_ref`], except that an entry whose stored name itself contains `?` — which
/// names could before they were restricted — resolves by that exact name, without a role. It is
/// looked up first, so `sales?role=x` never reaches a different entry than the one stored so.
/// When `sales` is stored too, the reference means either one, and is refused rather than
/// resolved to whichever is looked up first.
///
/// Authorization: checks nothing, and its answer reveals whether `w_id` stores that exact name.
/// Callers MUST already act for `w_id` — a job of it, or a caller authenticated into it — and
@@ -2068,26 +1977,16 @@ pub async fn parse_datatable_ref_for(
reference: &str,
) -> Result<(String, Option<String>)> {
if reference.contains('?') {
let role_target = parse_datatable_ref(reference)
.ok()
.and_then(|(name, role)| role.map(|_| name));
let (exists, target_exists) = sqlx::query_as::<_, (Option<bool>, Option<bool>)>(
"SELECT (datatable->'datatables') ? $2, (datatable->'datatables') ? $3
FROM workspace_settings WHERE workspace_id = $1",
let exists = sqlx::query_scalar::<_, Option<bool>>(
"SELECT (datatable->'datatables') ? $2 FROM workspace_settings WHERE workspace_id = $1",
)
.bind(w_id)
.bind(reference)
.bind(role_target)
.fetch_optional(db)
.await?
.unwrap_or((None, None));
if exists.unwrap_or(false) {
if let (Some(name), Some(true)) = (role_target, target_exists) {
return Err(Error::BadRequest(format!(
"Data table reference '{reference}' names both the data table '{reference}' \
and a role on the data table '{name}'. Rename '{reference}' to use either."
)));
}
.flatten()
.unwrap_or(false);
if exists {
return Ok((reference.to_string(), None));
}
}
+76 -35
View File
@@ -4650,6 +4650,30 @@ pub fn tag_reads_args(tag: &str) -> bool {
RE_ARG_TAG.is_match(tag)
}
/// Whether the tag reads the flow's state (`$flow_expr[results.a.foo]`), which only the flow
/// runtime can resolve, right before pushing the step. A malformed placeholder counts too, so it
/// is rejected or dropped instead of queueing the job on its literal text.
pub fn tag_reads_flow_expr(tag: &str) -> bool {
tag.contains("$flow_expr[")
}
/// Renders the value at the dotted `path` below `root` as a dynamic tag component, shared by
/// `$args[...]` and `$flow_expr[...]`: its JSON text with surrounding quotes trimmed, and empty
/// once a segment is missing. Only object keys are followed, never array indexes.
pub fn render_tag_path(root: Option<&RawValue>, path: &str) -> String {
let mut value = root.map(|x| x.get()).unwrap_or_default().to_string();
for part in path.split('.').filter(|p| !p.is_empty()) {
match serde_json::from_str::<serde_json::Value>(&value) {
Ok(obj) => value = obj.get(part).map(|v| v.to_string()).unwrap_or_default(),
Err(_) => {
value = String::new();
break;
}
}
}
value.trim_matches('"').to_string()
}
pub fn interpolate_args(x: String, args: &PushArgs, workspace_id: &str) -> String {
// Save this value to avoid parsing twice
let workspaced = x.as_str().replace("$workspace", workspace_id).to_string();
@@ -4657,40 +4681,12 @@ pub fn interpolate_args(x: String, args: &PushArgs, workspace_id: &str) -> Strin
let mut interpolated = workspaced.clone();
for cap in RE_ARG_TAG.captures_iter(&workspaced) {
let arg_name = cap.get(1).unwrap().as_str();
let arg_value = if arg_name.contains('.') {
let parts: Vec<&str> = arg_name.split('.').collect();
let root = parts[0];
let mut value = args
.args
.get(root)
.or(args.extra.as_ref().and_then(|x| x.get(root)))
.map(|x| x.get())
.unwrap_or_default()
.to_string();
for part in parts.iter().skip(1) {
if let Ok(obj) = serde_json::from_str::<serde_json::Value>(&value) {
value = obj
.get(part)
.and_then(|v| Some(v.to_string()))
.unwrap_or_default()
.as_str()
.to_string();
} else {
value = "".to_string(); // Invalid JSON or missing field
break;
}
}
value.trim_matches('"').to_string()
} else {
args.args
.get(arg_name)
.or(args.extra.as_ref().and_then(|x| x.get(arg_name)))
.map(|x| x.get())
.unwrap_or_default()
.trim_matches('"')
.to_string()
};
let (root, rest) = arg_name.split_once('.').unwrap_or((arg_name, ""));
let root_value = args
.args
.get(root)
.or(args.extra.as_ref().and_then(|x| x.get(root)));
let arg_value = render_tag_path(root_value.map(|x| &**x), rest);
interpolated =
interpolated.replace(format!("$args[{}]", arg_name).as_str(), &arg_value);
}
@@ -5269,6 +5265,8 @@ pub fn empty_result() -> Box<RawValue> {
lazy_static::lazy_static! {
pub static ref RE_ARG_TAG: Regex = Regex::new(r#"\$args\[((?:\w+\.)*\w+)\]"#).unwrap();
pub static ref RE_FLOW_EXPR_TAG: Regex =
Regex::new(r#"\$flow_expr\[((?:\w+\.)*\w+)\]"#).unwrap();
}
#[cfg(feature = "cloud")]
@@ -6537,7 +6535,10 @@ async fn push_inner<'c, 'd>(
);
windmill_common::worker::dedicated_worker_tag(workspace_id, &full_path)
} else {
if tag == Some("".to_string()) {
// The flow runtime resolves a step's `$flow_expr[...]` before pushing it, so one still here
// was pushed with no flow state to read (a step test, a dependency job) and would name a
// queue no worker serves: the job runs on its default tag instead.
if tag == Some("".to_string()) || tag.as_deref().is_some_and(tag_reads_flow_expr) {
tag = None;
}
@@ -7922,3 +7923,43 @@ mod result_metadata_tests {
assert_eq!(meta.wm_failure.as_deref(), Some("boom"));
}
}
#[cfg(test)]
mod render_tag_path_tests {
use super::{interpolate_args, render_tag_path, PushArgs};
use serde_json::value::RawValue;
use std::collections::HashMap;
fn render(root: &str, path: &str) -> String {
render_tag_path(
Some(&RawValue::from_string(root.to_string()).unwrap()),
path,
)
}
// Existing `$args[...]` tags route on exactly these renderings.
#[test]
fn renders_like_args_tags() {
assert_eq!(render(r#""eu""#, ""), "eu");
assert_eq!(render(r#"{"a": {"b": "eu"}}"#, "a.b"), "eu");
assert_eq!(render(r#"{"n": 4}"#, "n"), "4");
assert_eq!(render("null", ""), "null");
assert_eq!(render(r#"{"a": 1}"#, "b.c"), "");
assert_eq!(render(r#"{"a": ["eu"]}"#, "a.0"), "");
assert_eq!(render_tag_path(None, "a"), "");
let args = HashMap::from([("cfg".to_string(), raw(r#"{"lang": "eu"}"#))]);
let push_args = PushArgs {
args: &args,
extra: Some(HashMap::from([("e".to_string(), raw(r#""x""#))])),
};
assert_eq!(
interpolate_args("w-$args[cfg.lang]-$args[e]".to_string(), &push_args, "ws"),
"w-eu-x"
);
}
fn raw(json: &str) -> Box<RawValue> {
RawValue::from_string(json.to_string()).unwrap()
}
}
+19 -4
View File
@@ -838,6 +838,11 @@ pub async fn eval_fetch_timeout(
}
}
let w_id_for_tracing = w_id_for_tracing;
// nativets delivers logs in-process, so they never reach the masking in
// `handle_child::write_lines` and a `console.log` of `$WM_TOKEN` would be
// persisted verbatim. Mask here rather than in the detached task draining into
// `append_logs`: this loop normally runs while the job is still registered.
let mut masker = windmill_common::sensitive_log_masks::JobMasker::new(job_id);
let handle = tokio::spawn(async move {
let mut result_stream = String::new();
let mut is_stream = false;
@@ -845,10 +850,20 @@ pub async fn eval_fetch_timeout(
use windmill_common::result_stream::extract_stream_from_logs;
use windmill_common::tracing_init::{OTEL_JOB_LOGS, OTEL_PREFIX};
let stream = extract_stream_from_logs(&log.trim_end_matches("\n"));
// A stream chunk is result data, not a log line — it never reaches
// `job_logs`, and `merge_result_stream` can make it the job's result —
// so it stays raw wherever it goes, here and in the mirror below.
// Deliberately unlike `handle_child`, which streams the masked text.
// Routed before masking because the notice is one-shot: spent on a chunk
// no sink persists, a later redaction in `job_logs` would go unexplained.
let logged = stream.is_none().then(|| masker.mask(&log).into_owned());
// Mirror `process_streaming_log_lines` (EE) + the OTEL_JOB_LOGS
// hook from handle_child.rs, neither of which runs for nativets
// since nativets delivers logs in-process via the log channel.
for line in log.lines() {
for line in logged.as_deref().unwrap_or(&log).lines() {
tracing::info!(
target: "windmill:job_log",
job_id = ?job_id,
@@ -862,7 +877,7 @@ pub async fn eval_fetch_timeout(
}
}
if let Some(stream) = extract_stream_from_logs(&log.trim_end_matches("\n")) {
if let Some(stream) = stream {
if !is_stream {
is_stream = true;
if let Some(ref f) = stream_notifier_update {
@@ -874,8 +889,8 @@ pub async fn eval_fetch_timeout(
if let Err(e) = result_stream_sender.send(stream) {
tracing::error!("failed to send result stream: {e}");
}
} else {
if let Err(e) = append_logs_sender.send(log) {
} else if let Some(logged) = logged {
if let Err(e) = append_logs_sender.send(logged) {
tracing::error!("failed to send log: {e}");
}
}
+45 -5
View File
@@ -79,6 +79,15 @@ lazy_static::lazy_static! {
const DEFAULT_MAX_AGENT_ITERATIONS: usize = 10;
const HARD_MAX_AGENT_ITERATIONS: usize = 1000;
/// What a run stopped by `max_iterations` reports back. `Message` rather than
/// `OpenAIMessage` is load-bearing: `agent_action` is `skip_serializing` on the
/// latter and reaches JSON only through this wrapper, so serializing these raw
/// drops every tool name and job id and leaves the partial run unreadable.
#[derive(serde::Serialize)]
struct MaxIterPartialResult<'a> {
messages: Vec<Message<'a>>,
}
fn strip_system_messages(messages: &[OpenAIMessage]) -> Vec<OpenAIMessage> {
messages
.iter()
@@ -1785,10 +1794,6 @@ pub async fn run_agent(
step_id: Option<&'a str>,
result: MaxIterPartialResult<'a>,
}
#[derive(serde::Serialize)]
struct MaxIterPartialResult<'a> {
messages: &'a [OpenAIMessage],
}
return Err(Error::ExecutionRawError(
serde_json::value::to_raw_value(&MaxIterError {
message: format!(
@@ -1797,7 +1802,15 @@ pub async fn run_agent(
),
name: "ExecutionErr",
step_id: effective_flow_step_id,
result: MaxIterPartialResult { messages: &messages },
result: MaxIterPartialResult {
messages: messages
.iter()
.map(|m| Message {
message: m,
agent_action: m.agent_action.as_ref(),
})
.collect(),
},
})?,
));
}
@@ -2322,6 +2335,33 @@ mod tests {
assert!(!streaming_requested(Some(false)));
}
#[test]
fn max_iterations_partial_result_keeps_the_action_tags() {
let messages = vec![OpenAIMessage {
role: "tool".to_string(),
content: Some(OpenAIContent::Text("{\"rows\":2}".to_string())),
tool_call_id: Some("call_1".to_string()),
agent_action: Some(AgentAction::ToolCall {
job_id: uuid::Uuid::nil(),
function_name: "list_payouts".to_string(),
module_id: "b".to_string(),
}),
..Default::default()
}];
let partial = MaxIterPartialResult {
messages: messages
.iter()
.map(|m| Message { message: m, agent_action: m.agent_action.as_ref() })
.collect(),
};
let json = serde_json::to_value(&partial).unwrap();
let action = &json["messages"][0]["agent_action"];
assert_eq!(action["type"], "tool_call");
assert_eq!(action["function_name"], "list_payouts");
}
/// Over 64 characters OpenAI rejects the key outright, which costs a wasted round
/// trip per run and silently leaves that step with no prompt caching at all.
#[test]
+26 -4
View File
@@ -3859,8 +3859,6 @@ pub async fn run_worker(
let arc_job = Arc::new(job);
windmill_common::sensitive_log_masks::register_running_job(arc_job.id);
let span = create_span_with_name(&arc_job, &worker_name, Some(hostname), "job");
let log_ctx = log_context_for_job(&arc_job, &worker_name, Some(hostname));
@@ -3976,8 +3974,6 @@ pub async fn run_worker(
_ => {}
}
windmill_common::sensitive_log_masks::unregister_running_job(job_id);
#[cfg(feature = "prometheus")]
if let Some(duration) = _timer.map(|x| x.stop_and_record()) {
register_metric(
@@ -4512,6 +4508,30 @@ async fn detect_and_store_runtime_assets_from_job_args(
}
}
/// Holds a job's entry in the log-masking registry for as long as it executes, so
/// that secrets it fetches can be registered against it, and masks the job's own
/// token from the start: `$WM_TOKEN` stays valid well past the run, and a script
/// that echoes it would otherwise leave a live credential in the persisted logs.
///
/// Lives here rather than at the call sites so that every way of running a job —
/// the poller, the interactive worker shell, an inline AI agent tool — is covered
/// by construction.
struct RunningJobMasks(Uuid);
impl RunningJobMasks {
fn register(job_id: Uuid, token: &str) -> Self {
windmill_common::sensitive_log_masks::register_running_job(job_id);
windmill_common::sensitive_log_masks::register_secret_for_job(job_id, token);
RunningJobMasks(job_id)
}
}
impl Drop for RunningJobMasks {
fn drop(&mut self) {
windmill_common::sensitive_log_masks::unregister_running_job(self.0);
}
}
pub async fn handle_queued_job(
job: Arc<MiniPulledJob>,
raw_code: Option<String>,
@@ -4533,6 +4553,8 @@ pub async fn handle_queued_job(
flow_runners: Option<Arc<FlowRunners>>,
#[cfg(feature = "benchmark")] _bench: &mut BenchmarkIter,
) -> windmill_common::error::Result<JobOutcome> {
let _masks = RunningJobMasks::register(job.id, &client.token);
if job.canceled_by.is_some() {
return Err(Error::JsonErr(canceled_job_to_result(&job)));
}
+103 -9
View File
@@ -69,10 +69,10 @@ use windmill_common::{
use windmill_queue::schedule::get_schedule_opt;
use windmill_queue::{
add_completed_job, add_completed_job_error, append_logs, get_mini_pulled_job,
insert_concurrency_key_capped, interpolate_args,
insert_concurrency_key_capped, interpolate_args, render_tag_path,
report_error_to_workspace_handler_or_critical_side_channel, tag_reads_args,
try_schedule_next_job, CanceledBy, FlowRunners, MiniCompletedJob, MiniPulledJob, PushArgs,
PushIsolationLevel, SameWorkerPayload, WrappedError,
tag_reads_flow_expr, try_schedule_next_job, CanceledBy, FlowRunners, MiniCompletedJob,
MiniPulledJob, PushArgs, PushIsolationLevel, SameWorkerPayload, WrappedError, RE_FLOW_EXPR_TAG,
};
use windmill_audit::audit_oss::audit_log;
@@ -3115,6 +3115,66 @@ fn resolve_flow_step_tag(
}
}
/// Resolves each `$flow_expr[root.key.path]` of a step tag by reading `key.path` from `results`
/// (where `key` is a step id), `flow_input` or `flow_env`, rendered as `$args[key.path]` would be.
async fn interpolate_flow_expr_tag(
tag: &str,
db: &DB,
flow_job: &MiniPulledJob,
flow_input: &HashMap<String, Box<RawValue>>,
flow_env: Option<&HashMap<String, Box<RawValue>>>,
) -> error::Result<String> {
if RE_FLOW_EXPR_TAG
.replace_all(tag, "")
.contains("$flow_expr[")
{
return Err(Error::ExecutionErr(format!(
"Could not resolve the step tag `{tag}`: each `$flow_expr[...]` must hold a dotted \
path such as `results.a.b.c`"
)));
}
let mut rendered: HashMap<&str, String> = HashMap::new();
for cap in RE_FLOW_EXPR_TAG.captures_iter(tag) {
let path = cap.get(1).unwrap().as_str();
if rendered.contains_key(path) {
continue;
}
let (root, key_path) = path.split_once('.').unwrap_or((path, ""));
let (key, rest) = key_path.split_once('.').unwrap_or((key_path, ""));
if key.is_empty() || !matches!(root, "results" | "flow_input" | "flow_env") {
return Err(Error::ExecutionErr(format!(
"Could not resolve the step tag `{tag}`: `{path}` must start with \
`results.<step_id>`, `flow_input.<key>` or `flow_env.<key>`"
)));
}
let value = match root {
"flow_input" => render_tag_path(flow_input.get(key).map(|x| &**x), rest),
"flow_env" => render_tag_path(flow_env.and_then(|e| e.get(key)).map(|x| &**x), rest),
_ => match windmill_queue::get_result_by_id(
db.clone(),
flow_job.workspace_id.clone(),
flow_job.id,
key.to_string(),
None,
)
.await
{
Ok(result) => render_tag_path(Some(&*result), rest),
Err(Error::NotFound(_)) => String::new(),
Err(e) => {
return Err(Error::ExecutionErr(format!(
"Could not resolve the step tag `{tag}`: {e}"
)))
}
},
};
rendered.insert(path, value);
}
Ok(RE_FLOW_EXPR_TAG
.replace_all(tag, |cap: &regex::Captures| rendered[&cap[1]].clone())
.into_owned())
}
#[cfg(test)]
mod tag_resolution_tests {
use super::resolve_flow_step_tag;
@@ -4193,6 +4253,10 @@ async fn push_next_flow_job(
None
};
// The `flow_input` the step's input transforms read, which a `$flow_expr[flow_input...]`
// tag must read too: the body of a simple for-loop also sees `iter` there.
let mut step_flow_input = arc_flow_job_args.clone();
let marc;
let me;
let args = match &next_status {
@@ -4217,8 +4281,10 @@ async fn push_next_flow_job(
if let Some(input_transforms) = simple_input_transforms {
//previous id is none because we do not want to use previous id if we are in a for loop
let ctx = get_transform_context(&flow_job, "", &status);
let args = Marc::new(args);
step_flow_input = args.clone();
let ti = transform_input(
Marc::new(args),
args,
flow_env,
arc_last_job_result.clone(),
input_transforms,
@@ -4397,17 +4463,21 @@ async fn push_next_flow_job(
payload_tag.tag.as_deref(),
);
// `push_args` is empty once the input transforms failed, so a tag reading `$args[...]`
// interpolates to a queue nobody serves and the step sits there instead of reporting
// the error. Send it to the flow's tag, which a worker is provably serving right now.
// A step whose inputs failed to evaluate, or whose `$flow_expr[...]` tag failed to resolve,
// is pushed only to report the error, and a computed tag can then name a queue nobody
// serves (`push_args` is empty, so `$args[...]` reads nothing), leaving the step stuck
// instead. Send it to the flow's tag, which a worker is provably serving right now.
//
// A step handed over by id, or one whose tag `push` replaces, never reaches a worker
// through its tag, so rewriting theirs would be noise.
let step_is_pulled_by_tag = !continue_on_same_worker
&& !continue_with_runners
&& !payload_tag.payload.is_dedicated_worker();
let reroute_to_flow_tag =
err.is_some() && step_is_pulled_by_tag && tag.as_deref().is_some_and(tag_reads_args);
let reroute_to_flow_tag = err.is_some()
&& step_is_pulled_by_tag
&& tag
.as_deref()
.is_some_and(|t| tag_reads_args(t) || tag_reads_flow_expr(t));
let tag = if reroute_to_flow_tag {
Some(flow_job.tag.clone())
} else {
@@ -4449,6 +4519,30 @@ async fn push_next_flow_job(
.await?;
}
// Resolved only after the check: CUSTOM_TAGS allows the template, so its value may name
// any queue, as the value of an `$args[...]` tag does.
let mut tag_err = None;
let tag = match tag {
Some(t) if err.is_none() && tag_reads_flow_expr(&t) => {
match interpolate_flow_expr_tag(&t, db, &flow_job, &step_flow_input, flow_env)
.warn_after_seconds(3)
.await
{
Ok(resolved) => Some(resolved),
Err(e) => {
tag_err = Some(e);
Some(if step_is_pulled_by_tag {
flow_job.tag.clone()
} else {
t
})
}
}
}
t => t,
};
let err = err.or(tag_err.as_ref());
let evaluated_timeout = if let Some(timeout_transform) = &module.timeout {
let ctx = get_transform_context(&flow_job, &previous_id, &status);
+1 -1
View File
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.814.0";
export const VERSION = "v1.815.0";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({
-2465
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "windmill-chat",
"version": "1.814.0",
"version": "1.815.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "windmill-chat",
"version": "1.814.0",
"version": "1.815.0",
"license": "Apache-2.0",
"devDependencies": {
"@ai-sdk/react": "^4.0.102",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "windmill-chat",
"description": "Build chat interfaces on Windmill flows deployed in chat mode, from any frontend or raw app",
"version": "1.814.0",
"version": "1.815.0",
"author": "Ruben Fiszel",
"license": "Apache-2.0",
"homepage": "https://github.com/windmill-labs/windmill/tree/main/chat-sdk#readme",
+1 -1
View File
@@ -72,7 +72,7 @@ const packageJson = {
url: "https://github.com/windmill-labs/windmill/issues",
},
dependencies: {
esbuild: "0.28.0",
esbuild: "0.28.2",
...Object.fromEntries(parserPackages.map(p => [p, cliDeps[p] ?? "*"])),
},
optionalDependencies: {
+30 -30
View File
@@ -11,7 +11,7 @@
"@cliffy/table": "npm:@jsr/cliffy__table@1.0.0",
"@windmill-labs/shared-utils": "^1.0.13",
"diff": "^5.2.0",
"esbuild": "0.28.0",
"esbuild": "0.28.2",
"get-port": "7.1.0",
"jszip": "3.8.0",
"minimatch": "^10.0.0",
@@ -34,7 +34,7 @@
"windmill-parser-wasm-ts": "1.695.0",
"windmill-parser-wasm-yaml": "1.770.0",
"windmill-yaml-validator": "1.1.1",
"ws": "8.18.0",
"ws": "8.21.3",
"yaml": "^2.7.0",
},
"devDependencies": {
@@ -57,57 +57,57 @@
"@cliffy/table": ["@jsr/cliffy__table@1.0.0", "https://npm.jsr.io/~/11/@jsr/cliffy__table/1.0.0.tgz", { "dependencies": { "@jsr/std__fmt": "^1.0.9" } }, "sha512-VoLxH0DjofHWPWKUc5N+oCwXB6O6e+carnhp23yJTa7qokBb+SCrTIABEgQdIe/p0bxgmZhz17xt2efaAxXvbQ=="],
"@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.0", "", { "os": "aix", "cpu": "ppc64" }, "sha512-lhRUCeuOyJQURhTxl4WkpFTjIsbDayJHih5kZC1giwE+MhIzAb7mEsQMqMf18rHLsrb5qI1tafG20mLxEWcWlA=="],
"@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ=="],
"@esbuild/android-arm": ["@esbuild/android-arm@0.28.0", "", { "os": "android", "cpu": "arm" }, "sha512-wqh0ByljabXLKHeWXYLqoJ5jKC4XBaw6Hk08OfMrCRd2nP2ZQ5eleDZC41XHyCNgktBGYMbqnrJKq/K/lzPMSQ=="],
"@esbuild/android-arm": ["@esbuild/android-arm@0.28.2", "", { "os": "android", "cpu": "arm" }, "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg=="],
"@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.0", "", { "os": "android", "cpu": "arm64" }, "sha512-+WzIXQOSaGs33tLEgYPYe/yQHf0WTU0X42Jca3y8NWMbUVhp7rUnw+vAsRC/QiDrdD31IszMrZy+qwPOPjd+rw=="],
"@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.2", "", { "os": "android", "cpu": "arm64" }, "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A=="],
"@esbuild/android-x64": ["@esbuild/android-x64@0.28.0", "", { "os": "android", "cpu": "x64" }, "sha512-+VJggoaKhk2VNNqVL7f6S189UzShHC/mR9EE8rDdSkdpN0KflSwWY/gWjDrNxxisg8Fp1ZCD9jLMo4m0OUfeUA=="],
"@esbuild/android-x64": ["@esbuild/android-x64@0.28.2", "", { "os": "android", "cpu": "x64" }, "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q=="],
"@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-0T+A9WZm+bZ84nZBtk1ckYsOvyA3x7e2Acj1KdVfV4/2tdG4fzUp91YHx+GArWLtwqp77pBXVCPn2We7Letr0Q=="],
"@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw=="],
"@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-fyzLm/DLDl/84OCfp2f/XQ4flmORsjU7VKt8HLjvIXChJoFFOIL6pLJPH4Yhd1n1gGFF9mPwtlN5Wf82DZs+LQ=="],
"@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw=="],
"@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.0", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-l9GeW5UZBT9k9brBYI+0WDffcRxgHQD8ShN2Ur4xWq/NFzUKm3k5lsH4PdaRgb2w7mI9u61nr2gI2mLI27Nh3Q=="],
"@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.2", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw=="],
"@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-BXoQai/A0wPO6Es3yFJ7APCiKGc1tdAEOgeTNy3SsB491S3aHn4S4r3e976eUnPdU+NbdtmBuLncYir2tMU9Nw=="],
"@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg=="],
"@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.0", "", { "os": "linux", "cpu": "arm" }, "sha512-CjaaREJagqJp7iTaNQjjidaNbCKYcd4IDkzbwwxtSvjI7NZm79qiHc8HqciMddQ6CKvJT6aBd8lO9kN/ZudLlw=="],
"@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.2", "", { "os": "linux", "cpu": "arm" }, "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w=="],
"@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-RVyzfb3FWsGA55n6WY0MEIEPURL1FcbhFE6BffZEMEekfCzCIMtB5yyDcFnVbTnwk+CLAgTujmV/Lgvih56W+A=="],
"@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug=="],
"@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.0", "", { "os": "linux", "cpu": "ia32" }, "sha512-KBnSTt1kxl9x70q+ydterVdl+Cn0H18ngRMRCEQfrbqdUuntQQ0LoMZv47uB97NljZFzY6HcfqEZ2SAyIUTQBQ=="],
"@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.2", "", { "os": "linux", "cpu": "ia32" }, "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ=="],
"@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.0", "", { "os": "linux", "cpu": "none" }, "sha512-zpSlUce1mnxzgBADvxKXX5sl8aYQHo2ezvMNI8I0lbblJtp8V4odlm3Yzlj7gPyt3T8ReksE6bK+pT3WD+aJRg=="],
"@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ=="],
"@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.0", "", { "os": "linux", "cpu": "none" }, "sha512-2jIfP6mmjkdmeTlsX/9vmdmhBmKADrWqN7zcdtHIeNSCH1SqIoNI63cYsjQR8J+wGa4Y5izRcSHSm8K3QWmk3w=="],
"@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA=="],
"@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-bc0FE9wWeC0WBm49IQMPSPILRocGTQt3j5KPCA8os6VprfuJ7KD+5PzESSrJ6GmPIPJK965ZJHTUlSA6GNYEhg=="],
"@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ=="],
"@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.0", "", { "os": "linux", "cpu": "none" }, "sha512-SQPZOwoTTT/HXFXQJG/vBX8sOFagGqvZyXcgLA3NhIqcBv1BJU1d46c0rGcrij2B56Z2rNiSLaZOYW5cUk7yLQ=="],
"@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA=="],
"@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-SCfR0HN8CEEjnYnySJTd2cw0k9OHB/YFzt5zgJEwa+wL/T/raGWYMBqwDNAC6dqFKmJYZoQBRfHjgwLHGSrn3Q=="],
"@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg=="],
"@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.0", "", { "os": "linux", "cpu": "x64" }, "sha512-us0dSb9iFxIi8srnpl931Nvs65it/Jd2a2K3qs7fz2WfGPHqzfzZTfec7oxZJRNPXPnNYZtanmRc4AL/JwVzHQ=="],
"@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.2", "", { "os": "linux", "cpu": "x64" }, "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ=="],
"@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.0", "", { "os": "none", "cpu": "arm64" }, "sha512-CR/RYotgtCKwtftMwJlUU7xCVNg3lMYZ0RzTmAHSfLCXw3NtZtNpswLEj/Kkf6kEL3Gw+BpOekRX0BYCtklhUw=="],
"@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw=="],
"@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.0", "", { "os": "none", "cpu": "x64" }, "sha512-nU1yhmYutL+fQ71Kxnhg8uEOdC0pwEW9entHykTgEbna2pw2dkbFSMeqjjyHZoCmt8SBkOSvV+yNmm94aUrrqw=="],
"@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.2", "", { "os": "none", "cpu": "x64" }, "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw=="],
"@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.0", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-cXb5vApOsRsxsEl4mcZ1XY3D4DzcoMxR/nnc4IyqYs0rTI8ZKmW6kyyg+11Z8yvgMfAEldKzP7AdP64HnSC/6g=="],
"@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.2", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ=="],
"@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.0", "", { "os": "openbsd", "cpu": "x64" }, "sha512-8wZM2qqtv9UP3mzy7HiGYNH/zjTA355mpeuA+859TyR+e+Tc08IHYpLJuMsfpDJwoLo1ikIJI8jC3GFjnRClzA=="],
"@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.2", "", { "os": "openbsd", "cpu": "x64" }, "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw=="],
"@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.0", "", { "os": "none", "cpu": "arm64" }, "sha512-FLGfyizszcef5C3YtoyQDACyg95+dndv79i2EekILBofh5wpCa1KuBqOWKrEHZg3zrL3t5ouE5jgr94vA+Wb2w=="],
"@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q=="],
"@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.0", "", { "os": "sunos", "cpu": "x64" }, "sha512-1ZgjUoEdHZZl/YlV76TSCz9Hqj9h9YmMGAgAPYd+q4SicWNX3G5GCyx9uhQWSLcbvPW8Ni7lj4gDa1T40akdlw=="],
"@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.2", "", { "os": "sunos", "cpu": "x64" }, "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g=="],
"@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-Q9StnDmQ/enxnpxCCLSg0oo4+34B9TdXpuyPeTedN/6+iXBJ4J+zwfQI28u/Jl40nOYAxGoNi7mFP40RUtkmUA=="],
"@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ=="],
"@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-zF3ag/gfiCe6U2iczcRzSYJKH1DCI+ByzSENHlM2FcDbEeo5Zd2C86Aq0tKUYAJJ1obRP84ymxIAksZUcdztHA=="],
"@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.2", "", { "os": "win32", "cpu": "ia32" }, "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA=="],
"@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.0", "", { "os": "win32", "cpu": "x64" }, "sha512-pEl1bO9mfAmIC+tW5btTmrKaujg3zGtUmWNdCw/xs70FBjwAL3o9OEKNHvNmnyylD6ubxUERiEhdsL0xBQ9efw=="],
"@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.2", "", { "os": "win32", "cpu": "x64" }, "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g=="],
"@jridgewell/gen-mapping": ["@jridgewell/gen-mapping@0.3.13", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.0", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA=="],
@@ -211,7 +211,7 @@
"diff": ["diff@5.2.2", "", {}, "sha512-vtcDfH3TOjP8UekytvnHH1o1P4FcUdt4eQ1Y+Abap1tk/OB2MWQvcwS2ClCd1zuIhc3JKOx6p3kod8Vfys3E+A=="],
"esbuild": ["esbuild@0.28.0", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.0", "@esbuild/android-arm": "0.28.0", "@esbuild/android-arm64": "0.28.0", "@esbuild/android-x64": "0.28.0", "@esbuild/darwin-arm64": "0.28.0", "@esbuild/darwin-x64": "0.28.0", "@esbuild/freebsd-arm64": "0.28.0", "@esbuild/freebsd-x64": "0.28.0", "@esbuild/linux-arm": "0.28.0", "@esbuild/linux-arm64": "0.28.0", "@esbuild/linux-ia32": "0.28.0", "@esbuild/linux-loong64": "0.28.0", "@esbuild/linux-mips64el": "0.28.0", "@esbuild/linux-ppc64": "0.28.0", "@esbuild/linux-riscv64": "0.28.0", "@esbuild/linux-s390x": "0.28.0", "@esbuild/linux-x64": "0.28.0", "@esbuild/netbsd-arm64": "0.28.0", "@esbuild/netbsd-x64": "0.28.0", "@esbuild/openbsd-arm64": "0.28.0", "@esbuild/openbsd-x64": "0.28.0", "@esbuild/openharmony-arm64": "0.28.0", "@esbuild/sunos-x64": "0.28.0", "@esbuild/win32-arm64": "0.28.0", "@esbuild/win32-ia32": "0.28.0", "@esbuild/win32-x64": "0.28.0" }, "bin": "bin/esbuild" }, "sha512-sNR9MHpXSUV/XB4zmsFKN+QgVG82Cc7+/aaxJ8Adi8hyOac+EXptIp45QBPaVyX3N70664wRbTcLTOemCAnyqw=="],
"esbuild": ["esbuild@0.28.2", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.2", "@esbuild/android-arm": "0.28.2", "@esbuild/android-arm64": "0.28.2", "@esbuild/android-x64": "0.28.2", "@esbuild/darwin-arm64": "0.28.2", "@esbuild/darwin-x64": "0.28.2", "@esbuild/freebsd-arm64": "0.28.2", "@esbuild/freebsd-x64": "0.28.2", "@esbuild/linux-arm": "0.28.2", "@esbuild/linux-arm64": "0.28.2", "@esbuild/linux-ia32": "0.28.2", "@esbuild/linux-loong64": "0.28.2", "@esbuild/linux-mips64el": "0.28.2", "@esbuild/linux-ppc64": "0.28.2", "@esbuild/linux-riscv64": "0.28.2", "@esbuild/linux-s390x": "0.28.2", "@esbuild/linux-x64": "0.28.2", "@esbuild/netbsd-arm64": "0.28.2", "@esbuild/netbsd-x64": "0.28.2", "@esbuild/openbsd-arm64": "0.28.2", "@esbuild/openbsd-x64": "0.28.2", "@esbuild/openharmony-arm64": "0.28.2", "@esbuild/sunos-x64": "0.28.2", "@esbuild/win32-arm64": "0.28.2", "@esbuild/win32-ia32": "0.28.2", "@esbuild/win32-x64": "0.28.2" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA=="],
"esm-env": ["esm-env@1.2.2", "", {}, "sha512-Epxrv+Nr/CaL4ZcFGPJIYLWFom+YeV1DqMLHJoEd9SYRxNbaFruBwfEX/kkHUJf55j2+TUbmDcmuilbP1TmXHA=="],
@@ -321,7 +321,7 @@
"windmill-yaml-validator": ["windmill-yaml-validator@1.1.1", "", { "dependencies": { "@stoplight/yaml": "^4.3.0", "ajv": "^8.17.1" } }, "sha512-CVgAwEoBdJhF39q2N012QffhlGPRIyIWd8gj7NnfG+/lMWgH2k5CBLtKIt6cPF8Bxz+6DGC3st1ARSsecDtbTg=="],
"ws": ["ws@8.18.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-8VbfWfHLbbwu3+N6OKsOMpBdT4kXPDDB9cJk2bJ6mh9ucxdlnNvH1e+roYkKmN9Nxw2yjz7VzeO9oOz2zJ04Pw=="],
"ws": ["ws@8.21.3", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw=="],
"wsl-utils": ["wsl-utils@0.1.0", "", { "dependencies": { "is-wsl": "^3.1.0" } }, "sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw=="],
+2 -2
View File
@@ -23,7 +23,7 @@
"@cliffy/table": "npm:@jsr/cliffy__table@1.0.0",
"@windmill-labs/shared-utils": "^1.0.13",
"diff": "^5.2.0",
"esbuild": "0.28.0",
"esbuild": "0.28.2",
"get-port": "7.1.0",
"jszip": "3.8.0",
"minimatch": "^10.0.0",
@@ -46,7 +46,7 @@
"windmill-parser-wasm-ts": "1.695.0",
"windmill-parser-wasm-yaml": "1.770.0",
"windmill-yaml-validator": "1.1.1",
"ws": "8.18.0",
"ws": "8.21.3",
"yaml": "^2.7.0"
},
"devDependencies": {
-2
View File
@@ -55,8 +55,6 @@ export interface AppFile {
tables?: string[];
datatable?: string;
schema?: string;
/** The role the app uses each data table through, by data table name. */
roles?: Record<string, string>;
};
// Mirrors granular ACLs on the raw_app path. Synced via /acls/* by
// applyExtraPermsDiff — never through update_app_raw — so a perm-only
+17 -3
View File
@@ -2546,9 +2546,9 @@ export function preservePendingScriptLocks(
}
// `sync push` never applies the workspace's display name from settings.yaml and
// applies its color only when the local file carries one (see
// pushWorkspaceSettings), so on a push the fields it would not apply must
// compare equal, or the row is listed on every run.
// applies its color and auto_invite.instance_groups only when the local file
// carries them (see pushWorkspaceSettings), so on a push the fields it would not
// apply must compare equal, or the row is listed on every run.
const isWorkspaceSettingsFile = (p: string) =>
/^settings(\.[^./\\]+)?\.(yaml|json)$/.test(p);
function stripUnappliedSettingsFields(local: any, remote: any) {
@@ -2558,6 +2558,20 @@ function stripUnappliedSettingsFields(local: any, remote: any) {
delete local?.color;
delete remote?.color;
}
// push reads a missing auto_invite as {} on both sides
if (local) local.auto_invite ??= {};
if (remote) remote.auto_invite ??= {};
const localInvite = local?.auto_invite;
const remoteInvite = remote?.auto_invite;
if (localInvite?.instance_groups == null) {
for (const invite of [localInvite, remoteInvite]) {
delete invite?.instance_groups;
delete invite?.instance_groups_roles;
}
} else {
localInvite.instance_groups_roles ??= {};
if (remoteInvite) remoteInvite.instance_groups_roles ??= {};
}
}
export async function compareDynFSElement(
+1 -1
View File
@@ -10,4 +10,4 @@ export const WM_FORK_PREFIX = "wm-fork";
// (e.g. utils.ts) can read it without importing main.ts and creating a circular
// dependency (main → workspace → utils → main) that triggers a TDZ.
// Re-exported from main.ts for backwards compatibility.
export const VERSION = "1.814.0";
export const VERSION = "1.815.0";
+27 -2
View File
@@ -239,8 +239,19 @@ export async function pushWorkspaceSettings(
});
}
// Handle auto_invite using grouped format
if (!deepEqual(localSettings.auto_invite, settings.auto_invite)) {
// Handle auto_invite using grouped format. The domain invite and the instance groups
// are applied by separate endpoints, each rewriting only its own keys.
const {
instance_groups: localGroups,
instance_groups_roles: localGroupRoles,
...localDomainInvite
} = localSettings.auto_invite ?? {};
const {
instance_groups: remoteGroups,
instance_groups_roles: remoteGroupRoles,
...remoteDomainInvite
} = settings.auto_invite ?? {};
if (!deepEqual(localDomainInvite, remoteDomainInvite)) {
log.debug(`Updating auto invite...`);
const localAutoInvite = localSettings.auto_invite;
@@ -278,6 +289,20 @@ export async function pushWorkspaceSettings(
}
}
// Only when settings.yaml declares instance_groups: clearing a group removes the
// workspace members it granted, so an absent key must never clear it.
if (
localGroups != undefined &&
(!deepEqual(localGroups, remoteGroups) ||
!deepEqual(localGroupRoles ?? {}, remoteGroupRoles ?? {}))
) {
log.debug(`Updating instance groups...`);
await wmill.editInstanceGroups({
workspace,
requestBody: { groups: localGroups, roles: localGroupRoles ?? {} },
});
}
if (!deepEqual(localSettings.ai_config, settings.ai_config)) {
log.debug(`Updating copilot settings...`);
await wmill.editCopilotConfig({
-4
View File
@@ -5826,8 +5826,6 @@ data:
tables:
- main/users # Table in public schema
- main/app_schema:items # Table in specific schema
roles: # Optional: the role the app uses each datatable through
main: analyst
\`\`\`
**Table reference formats:**
@@ -5835,8 +5833,6 @@ data:
- \`<datatable>/<table>\` — Specific table in public schema
- \`<datatable>/<schema>:<table>\` — Table in specific schema
**Roles:** when a datatable is under roles, its queries run as a role, which only reaches what it was granted. \`roles\` records the role the app uses each datatable through; the app's code must pass the same role: \`wmill.datatable('main', { role: 'analyst' })\` in TypeScript, \`wmill.datatable('main', role='analyst')\` in Python. A datatable without an entry is used as its default role.
## SQL Migrations (sql_to_apply/)
The \`sql_to_apply/\` folder is for creating/modifying database tables during development.
+1 -1
View File
@@ -25,7 +25,7 @@ type Esbuild = typeof import("esbuild");
// Version to fall back to if the native host's version can't be read. Keep in
// sync with the "esbuild" pin in cli/package.json.
const FALLBACK_VERSION = "0.28.0";
const FALLBACK_VERSION = "0.28.2";
let cached: Esbuild | undefined;
let inFlight: Promise<Esbuild> | undefined;
@@ -320,3 +320,29 @@ test("push: settings.yaml differing only by name or an unset color is not a chan
});
expect(await diff(otherColor, remote, skips)).toEqual(["edited settings.yaml"]);
});
// A push applies auto_invite.instance_groups only when the local file declares
// them (see pushWorkspaceSettings).
test("push: settings.yaml without instance_groups is not a change", async () => {
const remote = local({
"settings.yaml":
"name: prod\nauto_invite:\n enabled: false\n instance_groups:\n - eng\n instance_groups_roles:\n eng: developer\n",
});
const undeclared = local({
"settings.yaml": "name: prod\nauto_invite:\n enabled: false\n",
});
const skips = { includeSettings: true };
expect(await diff(undeclared, remote, skips)).toEqual([]);
const groupsOnlyRemote = local({
"settings.yaml": "name: prod\nauto_invite:\n instance_groups:\n - eng\n",
});
const noAutoInvite = local({ "settings.yaml": "name: prod\n" });
expect(await diff(noAutoInvite, groupsOnlyRemote, skips)).toEqual([]);
const otherGroups = local({
"settings.yaml":
"name: prod\nauto_invite:\n enabled: false\n instance_groups: []\n",
});
expect(await diff(otherGroups, remote, skips)).toEqual(["edited settings.yaml"]);
});
@@ -0,0 +1,82 @@
/**
* Regression guard: `sync push` (pushWorkspaceSettings) applies the domain invite and
* the instance groups of `auto_invite` through their own endpoints, and never clears
* instance groups that settings.yaml does not declare.
*/
import { expect, test, describe, beforeEach, mock } from "bun:test";
let editAutoInviteCalls: unknown[] = [];
let editInstanceGroupsCalls: unknown[] = [];
const remoteAutoInvite = {
enabled: true,
domain: "*",
operator: false,
mode: "invite",
instance_groups: ["eng"],
instance_groups_roles: { eng: "developer" },
};
// Every wmill.* call reachable from pushWorkspaceSettings is stubbed: bun shares one
// mocked module across test files, and names missing from whichever mock loads first
// stay missing for the others.
mock.module("../gen/services.gen.ts", () => ({
getSettings: async () => ({ auto_invite: remoteAutoInvite }),
getWorkspaceName: async () => "phoenix",
changeWorkspaceName: async () => {},
changeWorkspaceColor: async () => {},
editWebhook: async () => {},
editAutoInvite: async (a: unknown) => {
editAutoInviteCalls.push(a);
},
editInstanceGroups: async (a: unknown) => {
editInstanceGroupsCalls.push(a);
},
editErrorHandler: async () => {},
editSuccessHandler: async () => {},
editCopilotConfig: async () => {},
editLargeFileStorageConfig: async () => {},
editWorkspaceGitSyncConfig: async () => {},
editWorkspaceDefaultApp: async () => {},
editDefaultScripts: async () => {},
workspaceMuteCriticalAlertsUi: async () => {},
updateOperatorSettings: async () => {},
editDataTableConfig: async () => {},
editSlackCommand: async () => {},
setWorkspaceSlackOauthConfig: async () => {},
deleteWorkspaceSlackOauthConfig: async () => {},
}));
const { pushWorkspaceSettings } = await import("../src/core/settings.ts");
describe("pushWorkspaceSettings auto_invite", () => {
beforeEach(() => {
editAutoInviteCalls = [];
editInstanceGroupsCalls = [];
});
test("an instance-group-only change updates the groups and leaves the domain invite", async () => {
await pushWorkspaceSettings("phoenix", "settings", undefined, {
name: "phoenix",
auto_invite: { ...remoteAutoInvite, instance_groups_roles: { eng: "admin" } },
});
expect(editAutoInviteCalls.length).toBe(0);
expect(editInstanceGroupsCalls).toEqual([
{
workspace: "phoenix",
requestBody: { groups: ["eng"], roles: { eng: "admin" } },
},
]);
});
test("a settings.yaml without instance_groups does not clear them", async () => {
const { instance_groups: _g, instance_groups_roles: _r, ...domainInvite } =
remoteAutoInvite;
await pushWorkspaceSettings("phoenix", "settings", undefined, {
name: "phoenix",
auto_invite: { ...domainInvite, operator: true },
});
expect(editAutoInviteCalls.length).toBe(1);
expect(editInstanceGroupsCalls.length).toBe(0);
});
});
@@ -31,6 +31,7 @@ mock.module("../gen/services.gen.ts", () => ({
editWebhookCalls.push(a);
},
editAutoInvite: async () => {},
editInstanceGroups: async () => {},
editErrorHandler: async () => {},
editSuccessHandler: async () => {},
editCopilotConfig: async () => {},
+2 -1
View File
@@ -110,11 +110,12 @@ WORKDIR /multiplayer
# Copy multiplayer server files
COPY multiplayer/package.json .
COPY multiplayer/package-lock.json .
COPY multiplayer/server.mjs .
COPY multiplayer/gateway.mjs .
# Install dependencies
RUN npm install
RUN npm ci --omit=dev
# ============================================================================
# Entrypoint Setup
+34 -10
View File
@@ -12,14 +12,36 @@ Symbols, not line numbers, are cited: they drift less.
by `create_session_token` (`windmill-api-users/src/users.rs`). `GET /api/users/refresh_token`
mints one for any non-job token but returns plain text, no redirect.
- **`tokens/impersonate`** (superadmin) returns a multi-use token and sets no cookie.
- **A token's label decides whether its expiry raises alerts.** When `delete_expired_items`
removes an expired `token` row, the monitor emails the owner and raises a critical alert (if
enabled); rows registered by `register_token_expiry_notification` also get an "expiring soon"
warning first. Neither happens when `is_user_token` (`windmill-common/src/auth.rs`) reserves
the label, so a token the system mints for itself, whether from the backend or from the frontend
through `tokens/create`, needs a reserved label. An `ephemeral-` prefix needs no other change
(keep it clear of `is_server_minted_label` if minted through `tokens/create`); a new prefix
also goes into the SQL and Svelte mirrors that function's doc lists.
- **`max_token_expiration_days`** caps `POST /users/tokens/create` and `tokens/impersonate`, by
shortening the stored expiration (`cap_token_expiration`), never by refusing: the CLI
authorization page, `wmill user create-token` and the editor's language-server token all pick a
lifetime without reading the setting, and CLIs already installed never will. The CLI signs in
again on its own when its token expires, which is why the authorization page labels it
`cli-login:<username>`, reserved in `is_user_token` so its expiry does not email the user. A token
owned by a service account is exempt: one in the workspace the token names, or in any workspace
for a workspace-less token (for `tokens/impersonate`, the impersonated account). Any workspace
admin can therefore create and impersonate a service account to hold an uncapped token, so the
ceiling bounds personal tokens only. Only the stored expiration is capped: the auth lookup never
reads the setting, so tokens that exist when it is turned on or lowered keep theirs, including
none. Deliberately outside it: server-side mints (`create_token_internal` callers such as native
trigger webhook tokens, which never expire for GitHub and Nextcloud), and tokens with their own
fixed lifetime that outlive a short ceiling: sessions (`MAX_SESSION_VALIDITY_SECONDS`, 3 days, and
re-mintable through `GET /users/refresh_token`) and MCP OAuth access tokens (7 days, with a
rotating 30-day refresh token). Any logged-in user can read the setting through `GET
/settings/global/{key}`, which the token form uses to offer only expirations within it. The
settings API and config sync reject any value `parse_max_token_expiration_days` cannot read, since
the token routes would read it as no ceiling; `parseMaxTokenExpirationDays` in the frontend must
accept exactly the same values.
- **A token's label decides whether its expiry raises alerts.** When `delete_expired_items` removes
an expired `token` row, the monitor emails the owner and raises a critical alert (if enabled);
rows registered by `register_token_expiry_notification` also get an "expiring soon" warning first,
except a token whose whole lifetime fits in `TOKEN_EXPIRY_WARNING_DAYS` (7), which gets no row
since the warning would arrive minutes after it was created. Neither happens when `is_user_token`
(`windmill-common/src/auth.rs`) reserves the label, so a token the system mints for itself,
whether from the backend or from the frontend through `tokens/create`, needs a reserved label. An
`ephemeral-` prefix needs no other change (keep it clear of `is_server_minted_label` if minted
through `tokens/create`); a new prefix also goes into the SQL and Svelte mirrors that function's
doc lists.
- **Every superadmin route refuses a job token**: `require_super_admin`
(`windmill-api-auth/src/lib.rs`) errors on `authed.job_id.is_some()`. A script that needs
`users/create`, `tokens/impersonate`, `set_login_type`, … must use a dedicated superadmin user
@@ -29,8 +51,10 @@ Symbols, not line numbers, are cited: they drift less.
the account into a `password` one in the same statement (an account created ahead of its owner
gets its first credential that way, or through the OAuth claim below).
- **Login links** (`login_link` table, `POST /users/login_links` superadmin-only,
`GET /auth/login_link/{token}` unauthenticated): single-use, ≤15 min, a session cookie and a
302 to a same-origin `rd`. `require_login_type` on the mint refuses (409) an account whose
`GET /auth/login_link/{token}` unauthenticated): single-use, ≤2 h, a session cookie and a
302 to a same-origin `rd`. A link minted with `confirm` is the `/user/login_link` page
instead, which spends it only on a click (`POST` to the same path, answering `{location}`), so
a mail scanner opening it does not. `require_login_type` on the mint refuses (409) an account whose
`login_type` has moved on — the way a caller re-entering an account it created stops being
able to once the owner has a password or a provider.
- **Pre-approved trial offer** (`cloud_trial_offer`, cloud-only routes under
+389 -329
View File
File diff suppressed because it is too large Load Diff
+14 -12
View File
@@ -1,6 +1,6 @@
{
"name": "@windmill-labs/components",
"version": "1.814.0",
"version": "1.815.0",
"scripts": {
"dev": "vite dev",
"dev:ui-builder": "mv static/ui_builder static/ui_builder.dev-disabled 2>/dev/null || true ; trap 'mv static/ui_builder.dev-disabled static/ui_builder 2>/dev/null || true' EXIT ; vite dev",
@@ -28,14 +28,13 @@
"@melt-ui/svelte": "^0.86.2",
"@playwright/test": "^1.57.0",
"@sveltejs/adapter-static": "^3.0.6",
"@sveltejs/kit": "^2.53.4",
"@sveltejs/kit": "^2.69.1",
"@sveltejs/package": "^2.5.7",
"@sveltejs/vite-plugin-svelte": "^7.0.0",
"@tailwindcss/forms": "^0.5.3",
"@tailwindcss/typography": "^0.5.8",
"@types/d3": "^7.4.0",
"@types/d3-zoom": "^3.0.3",
"@types/diff": "^7.0.1",
"@types/lodash": "^4.14.195",
"@types/vscode": "^1.83.5",
"@typescript-eslint/eslint-plugin": "^5.59.8",
@@ -59,7 +58,7 @@
"prettier-plugin-svelte": "^3.3.3",
"style-to-object": "^0.4.1",
"stylelint-config-recommended": "^13.0.0",
"svelte": "^5.53.5",
"svelte": "^5.55.7",
"svelte-awesome-color-picker": "^3.0.4",
"svelte-check": "^4.4.3",
"svelte-fast-check": "^0.4.5",
@@ -70,19 +69,22 @@
"svelte-range-slider-pips": "^2.3.1",
"svelte-splitpanes": "^8.0.9",
"tailwindcss": "^3.4.1",
"tar": "^7.5.4",
"tar": "^7.5.18",
"tslib": "^2.6.1",
"typescript": "^5.5.0",
"vite": "^8.2.0",
"vite-plugin-mkcert": "^2.0.0",
"vitest": "^4.1.0",
"vitest": "^4.1.11",
"vitest-browser-svelte": "^2.0.1"
},
"overrides": {
"monaco-graphql": {
"monaco-editor": "$monaco-editor"
},
"tar": "$tar"
"tar": "$tar",
"dompurify": "$dompurify",
"cookie": "^0.7.0",
"handlebars": "^4.7.9"
},
"type": "module",
"dependencies": {
@@ -117,8 +119,8 @@
"clone": "^2.1.2",
"d3-zoom": "^3.0.0",
"date-fns": "^2.30.0",
"diff": "^7.0.0",
"dompurify": "^3.3.1",
"diff": "^8.0.3",
"dompurify": "^3.4.13",
"driver.js": "^1.3.0",
"esm-env": "^1.0.0",
"fast-equals": "^5.0.1",
@@ -130,8 +132,8 @@
"lru-cache": "^11.1.0",
"lucide-svelte": "^0.540.0",
"mdast-util-find-and-replace": "^3.0.2",
"mermaid": "^11.15.0",
"minimatch": "^10.0.1",
"mermaid": "^11.16.1",
"minimatch": "^10.2.3",
"modern-screenshot": "^4.7.0",
"monaco-editor": "npm:@codingame/monaco-vscode-editor-api@=25.0.0",
"monaco-languageclient": "10.6.0",
@@ -178,7 +180,7 @@
"xterm-readline": "^1.1.2",
"y-monaco": "^0.1.4",
"y-websocket": "^1.5.4",
"yaml": "^2.8.0",
"yaml": "^2.8.3",
"yjs": "^13.6.7",
"zod": "^4.1.10"
},
+4 -1
View File
@@ -3,7 +3,10 @@
@tailwind components;
@tailwind utilities;
@media (min-width: 1760px) {
/* Keyed on the screen, not the window: a viewport query rescales the whole app
whenever the window is resized or split. Mirrored in editorFontSize.svelte.ts,
TextInput.svelte's leading classes and FlowGraphPreviewButton.svelte. */
@media (min-device-width: 1760px) {
:root {
font-size: 18px;
}
@@ -1,281 +0,0 @@
<script lang="ts">
import type { GraphModuleState } from './graph'
import {
JobService,
type CompletedJob,
type FlowModule,
type FlowStatusModule,
type Job
} from '$lib/gen'
import { workspaceStore } from '$lib/stores'
import FlowLogViewerWrapper from './FlowLogViewerWrapper.svelte'
import { z } from 'zod'
import { untrack } from 'svelte'
import type { AgentTool } from './flows/agentToolUtils'
type AgentActionWithContent = NonNullable<FlowStatusModule['agent_actions']>[number] & {
content?: unknown
}
const resultSchema = z.object({
messages: z.array(
z.object({
role: z.string(),
content: z.unknown(),
agent_action: z
.union([
z.object({
type: z.literal('tool_call'),
job_id: z.string(),
module_id: z.string(),
function_name: z.string()
}),
z.object({
type: z.literal('mcp_tool_call'),
call_id: z.string(),
function_name: z.string(),
resource_path: z.string(),
arguments: z.record(z.any(), z.any()).optional()
}),
z.object({
type: z.literal('message')
}),
z.object({
type: z.literal('web_search')
})
])
.optional()
})
)
})
interface Props {
tools: AgentTool[]
agentJob: Partial<CompletedJob> & Pick<CompletedJob, 'id'> & { type: 'CompletedJob' }
workspaceId?: string | undefined
storedToolCallJobs?: Record<number, Job>
onToolJobLoaded?: (job: Job, idx: number) => void
noPadding?: boolean
}
let {
tools,
agentJob,
workspaceId,
onToolJobLoaded,
storedToolCallJobs,
noPadding = false
}: Props = $props()
const fakeModuleStates: Record<string, GraphModuleState> = $state({})
async function loadMissingJobs(
agentActions: AgentActionWithContent[],
gen: number
): Promise<Record<string, GraphModuleState>> {
const states: Record<string, GraphModuleState> = {}
const promises = agentActions.map(async (toolCall, idx) => {
if (toolCall.type === 'tool_call') {
let job: Job | undefined = storedToolCallJobs?.[idx]
if (!job || job.type !== 'CompletedJob') {
job = await JobService.getJob({
id: toolCall.job_id,
workspace: workspaceId ?? $workspaceStore!
})
}
states[idx.toString()] = {
args: job.args,
type: job['success'] ? 'Success' : 'Failure',
logs: job.logs,
result: job['result'],
job_id: toolCall.job_id
}
// Keyed by index in the parent's cache, so a superseded run must not write into it.
if (gen === loadGen) {
onToolJobLoaded?.(job, idx)
}
} else if (toolCall.type === 'mcp_tool_call') {
states[idx.toString()] = {
type: 'Success',
args: toolCall.arguments ?? {},
logs: '',
result: toolCall.content
}
} else if (toolCall.type === 'web_search') {
states[idx.toString()] = {
type: 'Success',
args: {},
logs: '',
result: toolCall.content
}
} else {
states[idx.toString()] = {
type: 'Success',
args: {},
logs: '',
result: toolCall.content
}
}
})
await Promise.all(promises)
return states
}
let job: Partial<Job> | undefined = $state(undefined)
// Every prop change starts another load; only the newest may write the shared view, else a
// slower reload for a previous run restores its logs over the one now selected.
let loadGen = 0
async function loadToolCalls(agentJob: Props['agentJob'], tools: AgentTool[]) {
const gen = ++loadGen
let parsedResult = resultSchema.safeParse(agentJob.result)
if (!parsedResult.success) {
console.error('Invalid result', parsedResult.error)
// A failed agent job has no parseable action list. Drop the view rather than leave the
// previously selected step's tool tree rendered under this one's header.
if (gen === loadGen) {
job = undefined
for (const key of Object.keys(fakeModuleStates)) {
delete fakeModuleStates[key]
}
}
return
}
let agentActions = parsedResult.data.messages
.map(
(m) =>
(m.agent_action?.type === 'message'
? {
type: 'message',
content: m.content
}
: m.agent_action?.type === 'tool_call'
? {
type: 'tool_call',
job_id: m.agent_action.job_id,
module_id: m.agent_action.module_id,
function_name: m.agent_action.function_name
}
: m.agent_action?.type === 'mcp_tool_call'
? {
type: 'mcp_tool_call',
content: m.content,
call_id: m.agent_action.call_id,
function_name: m.agent_action.function_name,
arguments: m.agent_action.arguments
}
: m.agent_action?.type === 'web_search'
? {
type: 'web_search',
content: m.content
}
: undefined) as AgentActionWithContent | undefined
)
.filter((m) => m !== undefined)
const states = await loadMissingJobs(agentActions, gen)
if (gen !== loadGen) {
return
}
for (const key of Object.keys(fakeModuleStates)) {
delete fakeModuleStates[key]
}
Object.assign(fakeModuleStates, states)
job = {
...agentJob,
raw_flow: {
modules: agentActions
.map((toolCall, idx) => {
if (toolCall.type === 'message') {
return {
id: idx.toString(),
value: {
type: 'identity' as const
}
}
} else if (toolCall.type === 'mcp_tool_call') {
return {
id: idx.toString(),
value: {
type: 'identity' as const
},
summary: toolCall.function_name,
arguments: toolCall.arguments
}
} else if (toolCall.type === 'web_search') {
return {
id: idx.toString(),
value: {
type: 'identity' as const
},
summary: 'Web Search'
}
} else {
const module = tools.find((m) => m.summary === toolCall.function_name)
// A definition can be missing for a call that did run: the tool was renamed or
// removed since, or it belongs to a linked agent whose resource is no longer
// readable. Keep the recorded call — its args, logs and result come from the
// child job — rather than dropping it from the history.
return module
? ({
...module,
id: idx.toString()
} as FlowModule)
: ({
id: idx.toString(),
value: { type: 'identity' as const },
summary: toolCall.function_name
} as FlowModule)
}
})
}
}
}
// Identity, not a summary digest: a refreshed resource can change a tool's path, code or id while
// keeping its name and count. The store swaps the array only when its contents actually differ,
// so one version per array instance tracks that exactly. An empty list is always the same key,
// since callers hand out a fresh [] for it on every render.
const toolsVersions = new WeakMap<object, number>()
let nextToolsVersion = 0
function toolsIdentity(list: AgentTool[]): string {
if (list.length === 0) {
return 'empty'
}
let version = toolsVersions.get(list)
if (version === undefined) {
version = ++nextToolsVersion
toolsVersions.set(list, version)
}
return String(version)
}
// Rebuild when the inputs change, not only on mount: a linked agent's tools resolve
// asynchronously after the first render, and switching between completed runs reuses this
// component — either would otherwise keep the first snapshot. Keyed by value, because callers
// rebuild the `agentJob` object on every render and identity alone would reload in a loop.
let reloadKey = $derived(`${agentJob?.id ?? ''}|${toolsIdentity(tools)}`)
$effect(() => {
reloadKey
untrack(() => {
if (agentJob) {
loadToolCalls(agentJob, tools)
}
})
})
</script>
{#if job}
<div class={noPadding ? '' : 'p-2'}>
<FlowLogViewerWrapper
{job}
localModuleStates={fakeModuleStates}
{workspaceId}
render={true}
onSelectedIteration={async () => {}}
mode="aiagent"
/>
</div>
{/if}
@@ -0,0 +1,116 @@
<script lang="ts">
import type { Snippet } from 'svelte'
import { Badge } from '$lib/components/common'
import GfmMarkdown from './GfmMarkdown.svelte'
import AgentTrace from './AgentTrace.svelte'
import ChatCollapsibleCard from './copilot/chat/ChatCollapsibleCard.svelte'
import LabeledDivider from './LabeledDivider.svelte'
import WebSearchSourcesDisplay from './copilot/chat/WebSearchSourcesDisplay.svelte'
import { buildAgentTrace, splitFinalAnswer } from './agentTrace'
import { runPane } from './agentScroll'
import { createBottomSticker } from './stickToBottom'
import { formatTokenCount, summarizeAgentResult, type AgentResult } from './aiAgentResult'
interface Props {
result: AgentResult
workspaceId?: string
/** Identifies the run, so a reused viewer lands on the new one's output. */
runKey?: string
/**
* How to render an output that is not text: whatever the result viewer would do
* with that object on its own, a table for rows or the file viewer for an S3
* object. Passed in rather than imported so this does not reach back into the
* viewer rendering it.
*/
structuredOutput: Snippet<[unknown]>
}
let { result, workspaceId, runKey, structuredOutput }: Props = $props()
let summary = $derived(summarizeAgentResult(result))
let textOutput = $derived(typeof result.output === 'string' ? result.output : undefined)
// The turn that produced `output` is not a trace row: the output block below is
// that same text, and printing it twice in one scroll reads as the agent having
// answered itself. Its citations move down with it.
let answer = $derived(splitFinalAnswer(buildAgentTrace(result.messages), result.output))
let trace = $derived(answer.trace)
let reasoning = $derived(result.reasoning?.trim())
let reasoningExpanded = $state(false)
let anchor: HTMLElement | undefined = $state()
const sticker = createBottomSticker()
$effect(() => {
// Also on arrival from a stream: the run finishing adds the output separator,
// so the end has moved from wherever the stream had the reader parked.
runKey
trace.length
reasoning
sticker.scrollToEnd(runPane(anchor))
})
</script>
<div bind:this={anchor} class="flex flex-col w-full py-3">
{#if trace.length > 0}
<AgentTrace entries={trace} {workspaceId} />
{/if}
{#if reasoning}
<!-- The whole run's thinking, which the worker hands back joined rather than
per iteration, so it reads as one block above the answer it led to. -->
<ChatCollapsibleCard
label="Thinking"
expanded={reasoningExpanded}
onToggle={() => (reasoningExpanded = !reasoningExpanded)}
contentClass="font-main"
>
<GfmMarkdown md={reasoning} prose="xs" noPadding />
</ChatCollapsibleCard>
{/if}
{#if trace.length > 0 || reasoning}
<LabeledDivider class="my-3">
<span class="text-2xs text-hint">Output</span>
</LabeledDivider>
{/if}
<div>
{#if textOutput !== undefined}
{#if textOutput === ''}
<span class="text-tertiary text-xs">The agent returned no answer</span>
{:else}
<!-- A model writes this, and what it writes is steerable by whatever
reached its context — a user message, a tool's output. So it is
untrusted input and goes through the shared sanitizing chain, which
is also what makes it inert on the public replay page. -->
<GfmMarkdown md={textOutput} noPadding />
{/if}
{:else}
{@render structuredOutput(result.output)}
{/if}
{#if answer.sources}
<div class="mt-2">
<WebSearchSourcesDisplay sources={answer.sources} />
</div>
{/if}
</div>
<!-- What the run cost, as a footnote to what it produced. -->
<div class="flex items-center gap-2 flex-wrap text-xs mt-3">
{#if summary.toolCalls > 0}
<Badge color="blue">
{summary.toolCalls}
{summary.toolCalls === 1 ? 'tool call' : 'tool calls'}
</Badge>
{/if}
{#if summary.webSearches > 0}
<Badge color="blue">
{summary.webSearches}
{summary.webSearches === 1 ? 'web search' : 'web searches'}
</Badge>
{/if}
{#if summary.tokens !== undefined}
<Badge color="gray">{formatTokenCount(summary.tokens)} tokens</Badge>
{/if}
{#if summary.cachedTokens}
<Badge color="gray">{formatTokenCount(summary.cachedTokens)} cached</Badge>
{/if}
</div>
</div>
@@ -0,0 +1,111 @@
<script lang="ts">
import { untrack } from 'svelte'
import ChatCollapsibleCard from './copilot/chat/ChatCollapsibleCard.svelte'
import GfmMarkdown from './GfmMarkdown.svelte'
import { runPane } from './agentScroll'
import { createBottomSticker } from './stickToBottom'
import {
advanceAgentStream,
emptyAgentStreamProgress,
type AgentStreamProgress
} from './aiAgentResult'
interface Props {
/** The raw `result_stream` buffer, which only ever grows for a given run. */
raw: string
/** Identifies the run, so a viewer reused for another one starts over. */
streamKey?: string
}
let { raw, streamKey }: Props = $props()
let progress: AgentStreamProgress & { key?: string } = $state(emptyAgentStreamProgress())
$effect(() => {
raw
streamKey
untrack(() => {
// A shorter buffer than what was already folded in cannot be a longer
// version of the same stream, so treat it as a different one.
const continues = progress.key === streamKey && raw.length >= progress.consumed
const base = continues ? progress : emptyAgentStreamProgress()
progress = { ...advanceAgentStream(raw, base), key: streamKey }
if (!continues) {
// Another run, so the reader's decision to stop following the previous
// one does not carry over: this one starts at its latest output.
following = true
}
})
})
let stream = $derived(progress.stream)
let anchor: HTMLElement | undefined = $state()
// Carry the reader along as the text is written, unless they have scrolled up
// to read something — then the pane is theirs until they come back to the end.
let following = true
const sticker = createBottomSticker()
// The listener goes on the pane, not on this element: a scroll event fires on
// whatever actually scrolled and does not bubble, so a handler here would never
// run and the reader would be dragged back on every poll.
$effect(() => {
const pane = runPane(anchor)
if (!pane) {
return
}
const onScroll = () => {
if (!sticker.isOwnScroll()) {
following = sticker.isAtEnd(pane)
}
}
pane.addEventListener('scroll', onScroll, { passive: true })
return () => pane.removeEventListener('scroll', onScroll)
})
$effect(() => {
stream.current
stream.reasoning
stream.entries.length
if (following) {
sticker.scrollToEnd(runPane(anchor))
}
})
</script>
<!-- One gap on the container rather than margins per child, and the same gap the
finished trace uses: a turn's text must not shift when a tool call turns it from
the live text into a row. The text at the bottom is deliberately unlabelled — a
turn that goes on to call a tool was narration, and the run's end settles which. -->
<div bind:this={anchor} class="flex flex-col w-full py-3 gap-1">
{#each stream.entries as entry, index (entry.kind === 'tool' ? entry.callId : index)}
{#if entry.kind === 'tool'}
<ChatCollapsibleCard
label={entry.name}
expanded={false}
toggleable={false}
shimmer={entry.running}
onToggle={() => {}}
labelClass={entry.success === false ? 'text-red-500' : ''}
/>
{:else}
<div>
<GfmMarkdown md={entry.content} noPadding />
</div>
{/if}
{/each}
{#if stream.current !== ''}
<div>
<!-- Same sanitizing chain as a finished output: a partial answer is written
by the same model and is no more trusted for arriving in pieces. -->
<GfmMarkdown md={stream.current} noPadding />
</div>
{:else if stream.reasoning !== ''}
<!-- Reasoning arrives before the text, so on its own it means the model is
still thinking rather than that this run has nothing to say. -->
<div class="text-secondary">
<GfmMarkdown md={stream.reasoning} prose="xs" noPadding />
</div>
{/if}
</div>
@@ -0,0 +1,135 @@
<script lang="ts">
import { ExternalLink, Globe } from 'lucide-svelte'
import { JobService, type Job } from '$lib/gen'
import { base } from '$lib/base'
import { workspaceStore } from '$lib/stores'
import { msToReadableTimeShort } from '$lib/utils'
import ChatCollapsibleCard from './copilot/chat/ChatCollapsibleCard.svelte'
import ToolContentDisplay from './copilot/chat/ToolContentDisplay.svelte'
import WebSearchSourcesDisplay from './copilot/chat/WebSearchSourcesDisplay.svelte'
import GfmMarkdown from './GfmMarkdown.svelte'
import type { AgentTraceEntry } from './agentTrace'
import { SvelteMap, SvelteSet } from 'svelte/reactivity'
interface Props {
entries: AgentTraceEntry[]
workspaceId?: string
}
let { entries, workspaceId }: Props = $props()
let expanded = new SvelteSet<number>()
// Row state is keyed by position, which means nothing once the viewer is handed
// another run: row 0 would stay open showing the previous run's job.
$effect(() => {
entries
expanded.clear()
jobs.clear()
})
// A tool's own job holds what the envelope does not: its logs, how long it
// took, and whether it succeeded. Fetched when a row is opened rather than
// upfront, so a run with twenty calls does not issue twenty requests to draw
// a list of names.
let jobs = new SvelteMap<string, Job | 'loading' | 'failed'>()
async function loadJob(jobId: string) {
if (jobs.has(jobId)) {
return
}
jobs.set(jobId, 'loading')
try {
jobs.set(
jobId,
await JobService.getJob({ id: jobId, workspace: workspaceId ?? $workspaceStore! })
)
} catch {
// A tool job can be gone (retention) or unreadable. The row still has its
// arguments and result from the envelope, so only the extras are lost.
jobs.set(jobId, 'failed')
}
}
function toggle(index: number, entry: AgentTraceEntry) {
if (expanded.delete(index)) {
return
}
expanded.add(index)
if (entry.kind === 'tool' && entry.jobId) {
loadJob(entry.jobId)
}
}
function jobOf(jobId: string | undefined): Job | undefined {
if (!jobId) return undefined
const job = jobs.get(jobId)
return typeof job === 'object' ? job : undefined
}
function toolLabel(entry: Extract<AgentTraceEntry, { kind: 'tool' }>): string {
const job = jobOf(entry.jobId)
const duration = job?.['duration_ms']
return duration === undefined
? entry.name
: `${entry.name} · ${msToReadableTimeShort(duration)}`
}
</script>
<div class="flex flex-col w-full min-w-0 gap-1">
{#each entries as entry, index (index)}
{#if entry.kind === 'assistant'}
<div class="min-w-0">
<GfmMarkdown md={entry.content} noPadding />
{#if entry.sources}
<div class="mt-2">
<WebSearchSourcesDisplay sources={entry.sources} />
</div>
{/if}
</div>
{:else if entry.kind === 'search'}
<!-- Nothing to reveal: the worker records only that a search ran, and its
citations render under the assistant turn that follows. -->
<ChatCollapsibleCard
label="Web search"
expanded={false}
toggleable={false}
onToggle={() => {}}
/>
{:else}
{@const job = jobOf(entry.jobId)}
<ChatCollapsibleCard
label={toolLabel(entry)}
expanded={expanded.has(index)}
onToggle={() => toggle(index, entry)}
contentClass="space-y-3"
>
{#if entry.args}
<ToolContentDisplay title="Parameters" content={entry.args} toolName={entry.name} />
{/if}
{#if job?.logs}
<ToolContentDisplay title="Logs" content={job.logs} />
{/if}
<ToolContentDisplay
title="Result"
content={entry.result}
error={job?.type === 'CompletedJob' && !job.success ? entry.result : undefined}
/>
{#if entry.resourcePath}
<div class="text-2xs text-hint flex items-center gap-1">
<Globe size={11} />
{entry.resourcePath}
</div>
{:else if entry.jobId}
<a
class="text-2xs text-accent inline-flex items-center gap-1 w-fit hover:underline"
href="{base}/run/{entry.jobId}?workspace={workspaceId ?? $workspaceStore}"
target="_blank"
rel="noreferrer"
>
<ExternalLink size={11} />
Open job
</a>
{/if}
</ChatCollapsibleCard>
{/if}
{/each}
</div>

Some files were not shown because too many files have changed in this diff Show More