1977 Commits
Author SHA1 Message Date
Ruben FiszelandClaude Opus 5.5 d4a423f567 fix: bound the pg cached-connection reset probe so a vanished server fails fast (#11505)
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 11:14:36 +02:00
Ruben FiszelandClaude Opus 5.5 013b302d48 perf: keep a pg executor connection per database (#11492)
* fix: stop the pg executor cache from pinning a pooler slot

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: retry while the evicted pg backend exits

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf: keep a pg executor connection per database

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: close a pg connection whose query was cut off by a timeout or cancel

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: keep a pg connection only after its query read every result

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: free idle pg connections and retry when a connect is refused

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: retry a pg connect only when a connection cap refused it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: detect pg connection caps by SQLSTATE and label cloud connections as new

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 09:34:10 +02:00
e7fc1b2e2e feat: restricted job tokens per script and flow (#11484)
* feat: restricted job tokens (job_token_scopes on scripts and flows)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: admit flow-run reads, skip dedicated workers, gate on worker version

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep restricted jobs off flow runners, preserve scopes on rename and promotion

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep restricted jobs off every dedicated handoff, confine progress flow id

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: exclude restricted runnables from dedicated worker startup

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: gate restrictions on the release after 1.821.0

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: store per-job scopes on job_perms instead of v2_job, pin inline runs to the checked version

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: step-level job_token_scopes for flow steps and agent tools

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: fail closed on perms read errors, refuse restricted queue imports, gate step scopes in previews

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: carry a job's scopes on its completion so a re-run keeps the caller's cap

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: carry a zombie job's scopes into its completion

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: leave a zombie for the next sweep when its scopes cannot be read

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* docs: correct the QueuedJobV2 completion comment

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: validate step scopes in batch flows, fail closed on unvalidated step scopes

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: refuse flows with step or tool restrictions at push while an older worker is live

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: apply the step-scope worker gate to flow restarts

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: list the job token toggle with the other step and flow settings

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* chore: pin the EE companion merged with EE main

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* perf: skip scope lookups for unrestricted jobs; list job token setting last

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* style: rustfmt scopes tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* fix: confine restricted job tokens to their own run lineage; drop remaining extra lookups

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FHGE3ynoAu6yrgeg4kLwL

* chore: update ee-repo-ref to 259ad3bfeef5285ba80eedc86309b11dca001220

This commit updates the EE repository reference after PR #843 was merged in windmill-ee-private.

Previous ee-repo-ref: 2b77c0225dca441235daf7bf0a06ba968df0c927

New ee-repo-ref: 259ad3bfeef5285ba80eedc86309b11dca001220

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-10-03 09:33:44 +02:00
Ruben FiszelandClaude Opus 5.5 b7f74cd562 fix: never reuse a pg connection a script left inside a transaction (#11497)
* fix: never reuse a pg connection a script left inside a transaction

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: close a pg connection whose job failed instead of caching it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* docs: describe every caller of PgConnectionLease::discard

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 09:03:44 +02:00
Ruben FiszelandClaude Opus 5.5 d7227726be feat: log pg job progress, warn on stalls, detect dead db connections (#11491)
* feat: log pg job progress, warn on stalls, detect dead db connections

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: log only slow pg steps to keep job log writes off fast statements

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: say when a slow connect included a failed cached-connection reset

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 22:26:37 +02:00
Ruben FiszelandClaude Opus 5.5 eed7e7d9e6 fix: stop the pg executor cache from pinning a pooler slot (#11490)
* fix: stop the pg executor cache from pinning a pooler slot

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: retry while the evicted pg backend exits

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 22:21:58 +02:00
Ruben FiszelandClaude Opus 5.5 14801fdd68 fix: unstick postgres jobs on large results with custom-typed columns (#11475)
* fix: unstick postgres jobs on large results with custom-typed columns

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: carry the parked-rows delivery fix in the postgres fork

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: describe postgres queries before streaming instead of parking rows

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf: skip the postgres describe for statements that return no rows

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: look for returning in the whole postgres statement

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read the leading postgres keyword past nested block comments

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: end leading postgres line comments at CR too

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 17:21:20 +02:00
Ruben FiszelandClaude Opus 5.5 ae093b7666 fix: read inet and cidr results with their prefix, including arrays (#11476)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 15:56:11 +02:00
Ruben FiszelandClaude Opus 5.5 c25bb1ed90 feat: add //no_network annotation for native scripts (#11445)
* feat: add //no_network annotation to deny all network access in native scripts

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: close no_network bypasses (quic dns, header parsing, token, non-native)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: refuse //no_network on deno regardless of //native, derive disabled ops from deno_net

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: scope //no_network to native scripts only, drop bun/deno refusal

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 10:47:28 +02:00
hugocasa 94ac34a489 fix: await async def main in python scripts (#11437)
* fix: await async main and preprocessor in python scripts

* fix: reuse one event loop per python process for async main

* fix: reuse the event loop a python script sets at import

* docs: clarify python await helper comment
2026-09-30 13:02:30 +02:00
Ruben FiszelandClaude Opus 5.5 cec8eaeadb test: fix CLI and Windows cargo test failures on 1.820.0 (#11429)
* test(cli): stop CLI tests resolving a just-published windmill-client

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: quote the Windows cert path in the pg attach assertion

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(cli): note when the windmill-client pin must move

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 22:36:32 +02:00
hugocasa 039f49ef12 feat: dispatch ai agent tools through the worker queue (#11227)
* feat: dispatch ai agent tools through the worker queue

* fix: respect worker tags when executing ai agent tools locally

* fix: cancel only direct-child tool jobs on agent failure

* fix: cancel queued descendant tool jobs on agent failure

* fix: route ai agent tools by the flow's tag rule
2026-09-29 22:30:50 +02:00
Ruben FiszelandClaude Opus 5.5 c5f59d6ccc fix: drop caller-supplied _MODULES at push for every job (#11418)
* fix: drop caller-supplied _MODULES at push for every job

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: never interpolate _MODULES into a tag and drop unused mut

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 20:17:32 +02:00
d1260d7c5a feat: refuse OIDC tokens to previews that impersonate a path their user cannot write (#11396)
* feat: add provenance claims to job OIDC tokens and mark preview sub

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: require a flow or script job's version to belong to its path for deployed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: derive app script paths server-side and test job provenance in CE

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: count an app script as deployed only when a deployed app run stamped it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: state the deployed condition for the preview sub prefix

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: keep the plain OIDC sub for previews by users who can write the path

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: refuse OIDC tokens to previews by users who cannot write the path

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: keep OIDC token issuance unchanged, leaving provenance to the claims

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: name the root job's trigger claim root_trigger_kind

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: report the request-supplied paths in a job's provenance

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: count a worker-pushed preview under deployed code as deployed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: drop the claimed parent of a restarted flow preview

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: only read a job's modules from its args when it is a preview

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: never count a preview whose modules came in its args as deployed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to dba91044a174f8c9275fefea2619dc966b0f9250

This commit updates the EE repository reference after PR #832 was merged in windmill-ee-private.

Previous ee-repo-ref: a703cb17776d0858255fbfd817dce132377d3e80

New ee-repo-ref: dba91044a174f8c9275fefea2619dc966b0f9250

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-29 19:09:48 +02:00
c1b59f70dd feat(ai-agent): add compaction memory that summarizes older context (#10928)
* feat(ai-agent): add autocompacted memory that summarizes older context

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): compact on final-answer turns and count what a turn appended

Address the pre-push review findings on the compaction path:

- A turn the model answers without a tool call left the agent loop on its first
  iteration, so a chat-shaped step never compacted and reloaded the whole
  conversation on every later turn. Compaction now also runs after the loop.
- The trigger measured only the last request, so a single large tool result
  could carry the next one past the window without ever crossing 80%.
- The summarization call re-sent the usage-tracking request shape on endpoints
  the loop had already learned to drop it for.
- The flat 8000-token summary reserve swallowed the whole target on a small
  context window, leaving one message in the tail and summarizing the rest.
- A response cut off inside the <analysis> scratchpad was accepted as a summary.
- The chat-mode memory default was a shared object the step form edited in place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): keep Anthropic prompt counts and compact once per response

Address the first CI review round on the compaction path:

- Anthropic's streaming parser dropped `message_start`, the only event carrying
  the prompt-side counts, so a native Anthropic run reported no input tokens at
  all and compaction fell back to a character estimate.
- A loop that exits without issuing another request — a structured-output turn
  does — reached the post-loop pass still holding the previous measurement and
  compacted a second time, or retried a failure with nothing changed.
- The summarization call inherited the step's `max_completion_tokens`; a low one
  truncates the summary inside its scratchpad, which counts as a failure and
  disables compaction after three of them.
- A fired trigger that found nothing to summarize said nothing.
- Memory already over the window — a lowered `context_window`, or a step moved
  over from `auto` — had no way back, since compaction only ran after an
  accepted request. It now also runs once before the first one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): state the summary's own completion cap and drop the pre-flight pass

- The summarization call asked for no completion cap at all, which is "uncapped"
  only on the OpenAI-shaped providers: Anthropic substitutes 64000, over several
  Claude models' output ceiling, and Bedrock leaves the model's own small default,
  short enough to cut the response off inside its scratchpad. It now asks for the
  reserve the split already set aside, raised to the step's cap when that is larger.
- Compaction no longer runs before the first request. The fallbacks the loop learns
  from a rejection are not known that early, so on exactly the endpoints that need
  them the summarization was malformed by construction: it failed, spent a strike,
  and the first agent request still carried the oversized conversation. A memory
  already past the window is repaired on the turn after a request the endpoint
  accepts, rather than by a pass that cannot succeed there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): ask the summary for exactly the room the split reserved

The split scales its reserve down on a small window while the request asked for
a flat 8000, so the two diverged below an 80k window: on a 4k/8k model the cap
alone exceeded the window and every summarization was refused, and on a 20k one
a full-length summary could land the conversation back over the trigger and
compact its own previous summary on the next response. Both now read one
`summary_reserve_tokens`.

The call also no longer inherits the step's reasoning effort. Every provider
counts thinking against that same budget, so a high-effort model could spend the
whole reserve before writing anything and return a summary cut off inside its
scratchpad; the compaction prompt asks for an `<analysis>` block, which is the
reasoning this call needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): charge the compaction budget for tools and the system prompt

The tail budget was the whole target, but a request also carries the system
prompt compaction keeps and the tool definitions, which are not in the message
list at all. On a small window those are most of it: a tail sized to the full
target left the next request back over the trigger, compacting again every
response, and the no-usage estimate missed the tool schemas entirely so it could
fail to trigger at all. Both now account for them.

The reserve also gains a floor. It is the summary's output cap as well as the
room the split leaves, and scaled down without one a small window gave a
structured nine-section summary a few hundred tokens — truncated inside its
scratchpad every time, which is discarded, which switches the mode off after
three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): count Gemini's tool-use prompt tokens in an agent step's usage

Gemini splits a tool-using turn's input across `promptTokenCount` and a disjoint
`toolUsePromptTokenCount`, and its thinking apart from `candidatesTokenCount`.
The agent step's parser read only the headline fields, so every tool-using turn
under-reported both — and the compaction trigger, which runs off the reported
prompt, could not see the tool results that grew it. It now goes through the same
helpers the proxy path already used.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): calibrate the compaction estimate against the measured prompt

Two rounds running, the finding was "the character estimate cannot see input X"
— tool schemas, then S3 attachments, which are short paths in the message list
and whole images by the time a provider counts them. Enumerating those is a list
that only grows, so the estimate is now scaled to the one number that is ground
truth: what the provider charged for the last request. Attachments, tokenizer
drift and whatever comes next fall out of that, because the estimate is only
ever used relative to itself.

Also stop the Gemini helpers turning an absent count into `Some(0)`. Downstream,
absent means "fall back to estimating the conversation" while zero reads as an
empty prompt and would hold the trigger below its threshold for the whole run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): charge attachments what they cost and let a heavy short prefix compact

The calibration conserved the conversation's total cost but spread it by
character count, so an attachment — a short S3 path in the message list, a whole
image or PDF once a provider expands it — was charged to the text messages around
it and stayed nearly free in the split. It now carries a nominal cost of its own,
which the calibration corrects a residual on rather than the whole gap.

The four-message minimum also refused exactly the case that fix is for: an
attachment arriving on the first or second turn can pass the trigger before four
removable messages exist, and summarizing even one of them saves most of the
prompt. A prefix worth a quarter of the window is now enough on its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): never summarize a prefix holding only a previous summary

The message-count floor was carrying a second job: a fresh summary sits in a one
or two message prefix, so requiring four declined it. The share threshold added
last commit admits it, and a summary is reserve-sized by construction — so the
post-compaction shape could spend one summarization per response swapping a
summary for another the same size, shrinking nothing and losing fidelity each
time. A previous summary no longer counts towards that threshold.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): take the context window from the model and drop the estimate calibration

Brings compaction in line with how the AI session does the same job, which had
already answered these three questions.

- The window is looked up from the model. `MODEL_CONTEXT_WINDOWS` in
  `windmill-ai/src/model_context.rs` mirrors the session's table in
  `copilot/modelConfig.ts`, entry for entry and with the same matching rules;
  each side points at the other, since a model added to one and not the other
  compacts at two different sizes. A step's `context_window` becomes the
  override for what the lookup cannot serve, and chat mode writes none.
- Provider usage is normalized where the provider's quirk is, not at the
  consumer. `TokenUsage::with_cache_beside_input` raises `input_tokens` to the
  whole prompt for Anthropic and Bedrock, which report their cached prefix
  beside it; the OpenAI shape already counts it inside. `prompt_tokens()` is
  then just `input_tokens`, rather than inferring the shape from whether a
  write count is present.
- The estimator is no longer calibrated against the measured prompt. The
  session uses the provider's count when it has one and a chars/4 estimate
  otherwise, with nothing in between, and a tail sized a little wrong only
  compacts again a turn later.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* feat(ai-agent): summarize memory down to what the database can store

Without an instance object store, memory is a 100KB database row cut
from its oldest message, the summary included, so compaction on a
mainstream model never got to keep anything across runs. A step that
persists there now runs its post-loop compaction pass against the
smaller of the model's window and the cap at chars/4, about 25k tokens:
the loop keeps the whole window, and what is written is a summary plus a
tail that fits. The run logs when that pass summarizes, and how many
messages the write dropped when one still overshoots.

The editor's storage warning on the option is removed: nothing exposes
the instance storage to it, so it keyed on the workspace S3 setting,
which is unrelated to where memory goes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): get a complete, billed summary out of every provider

Compaction against the real providers turned up four things the stub
could not: Gemini and OpenAI's reasoning models think by default and
bill it against the same cap the summary must fit in, so the
summarization request now asks them for their least (none, low); an
OpenAI Responses call that hits max_output_tokens ends in
response.incomplete, whose usage the parser dropped, so that
summarization went unbilled; a summary that quotes </summary> when it
describes its own instruction was cut off at the quote, on the agent
step and the AI session alike; and the prefix could end on an unanswered
user message, after which the instruction reads as part of that turn
(Anthropic merges the two outright). The tail now starts on a user
message, and both prompts tell the model the instruction is not part of
the conversation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): compact down to half the window, on the agent step and the AI session

The gap between the 80% trigger and the target is what one compaction
buys, and every summarization request carries most of the window. At a
70% target a 128k model summarized about 13k tokens of prefix for a
summary of up to 8k, so each ~100k-token request bought a few turns of
room before the next one re-summarized the previous summary. At 50% the
same request frees about 30k.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): drop the workspace-S3 memory hint and state the database bound in the tooltip

The memory field warned that memory is kept in the database whenever the
workspace had no S3 storage. That setting has no bearing on where memory
goes: the instance object store decides, and nothing exposes it to the
editor. The field's tooltip now describes both memory kinds and states
the database bound unconditionally; the run log says what happened.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): send the summarizer its tool history as text

The summarization request carries no tool definitions, and Bedrock's
Converse API rejects toolUse/toolResult blocks that arrive without them,
so on Bedrock every summarization of a prefix holding a tool call failed
silently until the breaker tripped. The prefix's tool calls and results
now reach the summarizer rendered as text, on the agent step and in the
AI session's compaction, which goes through the same proxy.

Also drops the TokenUsage::prompt_tokens accessor, which had become a
plain read of the normalized input_tokens, and shortens the context
window field's description.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): price attachments from the provider count, bound storage in bytes, effort per pro model

Addresses two Codex rounds and a leftovers audit.

- Attachments were priced at a flat 1500 tokens in the split, so a
  multi-page PDF (tens of thousands of tokens to the provider, a short
  S3 path in the message list) could be kept in the tail or leave no
  prefix worth summarizing. They are now priced from the provider's
  count for the request that carried them, less that request's text,
  with the 1500 floor where nothing was counted.
- The database storage bound measured the provider's token count, but
  the 100KB cap is bytes and repetitive text packs several characters
  per token. The persist pass now measures the serialized conversation.
- The summarizer forced `low` on every reasoning model, which the pro
  variants reject (gpt-5-pro takes only high, gpt-5.2-pro starts at
  medium); they now get no effort.
- Dropped the unused prompt_tokens accessor and its orphaned assert, an
  unused PartialEq, a needlessly public lookup, and fully-qualified
  Gemini calls; refreshed stale comments and the memory_id schema doc;
  regenerated the flow schema artifacts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): evict a heavy attachment into the summarized prefix, not the tail

Pricing attachments from the provider count was not enough on its own: a
leading attachment is a user message, and the boundary rule pulled the
last unanswered user turn back into the kept tail to keep it with its
answer. For a heavy attachment that dragged it into the tail — or, at
the front, emptied the prefix — so it was never summarized and rode
every request. The boundary now moves forward instead, keeping that
user turn and its answer in the summarized prefix. Verified on the
running instance: a 25k-token PDF on a 30k window is summarized out on
the turn it overflows, and later turns drop from 26k to ~1.5k tokens.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): keep the forward boundary move off tool results and the prefix start

The forward move that keeps an unanswered user turn out of the tail had
two edges the third Codex round found: advancing past the user could
land the boundary on a tool result (its tool_calls then summarized away,
orphaning it), and with no system prompt the summarizable prefix starts
at 0, so a trigger firing while the tail estimate fit everything indexed
below the start and panicked the task. The forward scan now skips
tool-opening boundaries, and the move is guarded above the prefix start.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): drop the step temperature from the summary request

OpenAI's reasoning models (gpt-5-mini, gpt-5.1, gpt-5.2) reject
`temperature` alongside any reasoning effort but their own default, so a
step configured with a temperature made every summarization fail once
the summarizer forced a low effort — history then grew unchecked. The
internal summary call now omits the step's temperature: a structured
extraction does not need a set one, and omitting it sidesteps each
provider's temperature-versus-reasoning rules. Confirmed against the API
that low + temperature is refused on those models.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): compact an oversized loaded memory before the first request

Compaction was reactive, taken only after a request the endpoint
accepted, so the fallbacks the loop learns from a rejection are known
first. But a memory loaded from an earlier run can already exceed this
run's window — the step was switched to a smaller model, or a run under
a wider one persisted more than fits — and that first request then
overflows and fails the run, with every retry reloading the same
history and failing again. A pass is now taken up front, off the
character estimate, before the first request. It uses the default
request shape; an endpoint needing a fallback may reject this one
summary, which is non-fatal, and mainstream providers need none.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): under the storage bound, trigger on the max of bytes and model tokens

The storage-bound pass measured only the serialized row size, so an
attachment — a few bytes as an S3 path but nearly the whole model
context — read as tiny and the pass skipped a compaction the model
needed. It now takes the larger of the byte measure and the model's
token count, since repetitive text is few tokens but many bytes and an
attachment is the reverse; either being over must fire a pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): drop oldest turns when a summary cannot fit the window, as the AI session does

An oversized loaded memory (a step switched to a smaller model, or an
object-store run that persisted more than a later model's window holds)
left a prefix larger than the summarizer's own window, so the summary
request overflowed and failed, the memory was untouched, and every
retry failed the same way. The AI session handles this by falling back
from summarization to dropping the oldest turns down to the target;
compaction here now does the same. When a summary cannot run — it
failed, the breaker is tripped, or nothing is worth folding — the oldest
turns are dropped until the conversation fits and opens on a user
message, keeping the newest turn. The next request then always fits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): drop whole turns only, keep the storage pass to bytes, refresh the count after a rewrite

Three edges the seventh Codex round found, all in the drop-oldest
fallback and the storage-bound measure:

- drop_oldest_to_fit dropped to any point that freed enough, which could
  strand a tool result whose tool_calls went with the messages before
  it. It now drops whole turns only, always landing the boundary on a
  user message and never splitting the newest turn; a lone turn too big
  for the window is left whole rather than broken.
- The storage-bound pass measured the whole model prompt against the
  shrunk 25k window, so a large tool roster and the system prompt —
  neither written to the row — tripped it on a conversation the row
  easily held. It measures the serialized bytes alone now; the model's
  own window is enforced by the in-loop passes and the pre-first-request
  pass, so the persisted size is all this pass is for.
- A compaction rewrites the message list, so the provider's count for
  the request that produced it no longer lines up. The count is now
  cleared after any pass that rewrites the conversation, so a later pass
  measures the estimate over the actual messages instead of a stale,
  larger prompt (which could decline a summary that already fit and then
  drop it). The step temperature, no longer sent to the summarizer on
  any path, is dropped from the request struct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): measure only the persisted messages against the storage cap

Persistence strips the system prompt before writing the memory row, but
the storage pass was serializing every message including it, so a large
system prompt with a tiny conversation reported far over the storage
trigger, and the fallback dropped the one real turn, run after run. The
storage measure now serializes only the non-system messages, matching
what the row actually holds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix(ai-agent): run the model-window pass before the storage-bytes pass post-loop

A turn the model answered without a tool call broke before the in-loop
compaction check, so on database-backed memory its only pass was the
storage one, which measures bytes. An attachment fills the model context
but is a few bytes in the row, so that turn never compacted and a
follow-up could overflow the model. The post-loop now runs a
model-window pass first, off the provider's count, then the
storage-bytes pass when the row is smaller than the model — both limits
enforced for a chat-shaped step, not just the one that happens to bind.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix: simplify agent compaction and preserve execution history

* fix: remove unused compaction history setting

* fix: preserve answers and recover rejected agent context

* refactor: make agent compaction transactional

* fix: skip agent summaries that cannot fit retained context

* fix: explain skipped agent context compaction

* fix: retain recent agent memory when storage compaction cannot fit

* fix: start retained agent memory at a user turn

* fix: reject unsafe agent memory truncation on storage fallback

* docs: clarify agent context window override scope

* fix: keep recent turns verbatim when compaction memory outgrows storage

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix: keep the compaction summary out of the agent's answers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* fix: shorten the agent context window help text

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ViJyjUmidDYV2m6ifQdLeH

* chore: update ee-repo-ref to 942d4013f36edac1fc9a9addbdb02198db1c7a05

This commit updates the EE repository reference after PR #812 was merged in windmill-ee-private.

Previous ee-repo-ref: 8ca1682ce6106ba6ea96894fbe606dac64102eb6

New ee-repo-ref: 942d4013f36edac1fc9a9addbdb02198db1c7a05

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-29 19:01:53 +02:00
hugocasaandClaude Opus 5 cf5c49c3dc feat: restart perpetual runs on the version a deploy makes runnable (#11200)
* feat: opt-in move of perpetual runs to a newly deployed script version

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: keep the perpetual-run opt-in across relocks and check the new version's tag

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: pin the tag check on a perpetual version switch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: count perpetual runs past the first queue page in the deploy prompt

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: restart perpetual runs on the version a deploy makes runnable

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: claim a perpetual run and queue its replacement in one transaction

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: honour a cancel that lands after the worker last read its queue row

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: leave the lost-cancel fix to its own PR and match the scale down to 0 wording

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: run a preprocessor the deployed version adds over the arguments carried over

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: shorten the wording of the modal's argument warning

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: never preprocess a restarted perpetual run, as every other restart does

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: preprocess for a replacement whose run had not been through one

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: push a perpetual replacement without the deployed debounce settings

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: name the runs the deploy button restarts

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* perf: skip the perpetual restart lookups on a deploy that is not perpetual

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: move perpetual runs before anything that can fail after the deploy commits

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: report cancellation on the queue listing so the deploy prompt can skip it

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: pass the tag workspace to the availability check after the merge

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: say which arguments are defined differently and which values are kept

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: resolve a dynamic tag before checking it for a restarted run

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: pin that a deployed dynamic tag is checked as it resolves

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-29 18:03:43 +02:00
8741843d2e feat: external instance cluster for data tables and Ducklake catalogs (#11197)
* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): data table roles in the DB manager and raw apps

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: never add a role to the reference of a data table whose name contains '?'

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: read the roles of a data table whose name contains '?'

The generated client leaves a '?' in a path param unencoded, so the lookup
404'd and the raw-app picker blocked Start on such a data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): refuse a reference naming both a legacy data table and a role

When a workspace stores both `sales` and a legacy `sales?role=analytics`, the
reference resolved to the legacy entry without a role, so browsing `sales` as
`analytics` reached another data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: declare the default role in migrations written for a data table whose name contains '?'

Such a data table connects as its default role without naming it, so the
migrations the manager wrote for it declared no role and ran as admin.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): set up an external instance cluster for data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): send external cluster passwords as SCRAM verifiers

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): scope external cluster credential readers to the crate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [ee] feat(datatables): external_instance data tables on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): compare the external cluster settings under a row lock before storing setup

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): only drop external databases Windmill marked, and check use under the lock

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): put a data table's connection under Postgres roles

A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): gate the paths that reach a whole database as admin

Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.

All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping

A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.

Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.

Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): fail loudly where a role or a pointer can be left half-recorded

Three ways the feature could end up in a state nobody could see or undo.

Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.

Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.

`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.

Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): unbreak two operator messages and two comments that described other code

The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.

`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract

The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.

Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.

The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.

Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.

Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): give the role catalog its own row, out of reach of the config machinery

Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.

`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.

Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.

Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.

Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced

The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".

The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing

The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.

Three ways a save could leave something resolving to nothing:

A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.

Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.

Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.

Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* refactor(datatables): put the role catalog in its own table, not in global_settings

Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.

The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.

One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.

Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): validate a rename against the save it describes, and re-check under the locks

Three from the round, all about deciding on state that could already have moved.

A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.

Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.

The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.

Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically

The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".

The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.

Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse to copy a data table that is under roles

pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.

Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse the clone's database too, not only its data

A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.

Refuse in both, so the clone stops before a database exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* nit worker error msg

* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): Ducklake catalogs on the external instance cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): never grant CREATEROLE to custom_instance_user on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): state the authorization contract of external database usage lookups

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the parent's settings while a fork points at its data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse repointing the external cluster while it is in use, and keep verify-ca working for pg_dump

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): protect external databases pending fork cleanup, and describe Ducklake usage in the API

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): per-cluster data table role catalogs, with roles on the external instance cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): write the external cluster setting under the lifecycle lock, and check fork targets are registered

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): register external fork catalogs under the lifecycle lock, and keep certificate verification in DuckDB attaches

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep certificate verification when DuckDB attaches an external data table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep certificate verification when DuckDB attaches an external data table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse fork cleanup of an external database another workspace uses

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse rolling back while external data tables are under roles, and type external_instance in the CLI

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): stop counting storage-only fork cleanup rows as uses of an external database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bind fork database copies to their workspace, and count every use before dropping one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): authenticate instance database setup before writing its status, and keep a fork reservation across it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): create external databases only on a cluster setup succeeded on, and document the registry reader

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep only the most recently used DuckDB root certificate files

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): migrate fork reservations on workspace rename, and lock the parent's data tables for the whole fork

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): take the fork data table lock once, before the external cluster's

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): describe the external instance cluster and how to run one locally

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep fork reservations private, drop a cleaned-up entry with its database, and serialize cleanup with settings saves

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the fork lock across a fork import, and carry the reservation inside the setup write

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check the external cluster setting on its own transaction, and gate the registry probe

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59

This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.

Previous ee-repo-ref: 0edd40979cf36bfba59323f3f6a0811ae1369cf5

New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59

Automated by sync-ee-ref workflow.

* fix(datatables): keep DuckDB root certificate files in the job directory

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: drop the unused json import from the settings crate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop the serde_json::json import left unused by the fork setup write

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): accept external_instance data tables in the settings form type

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize fork database reservations, cleanup, setup and Ducklake saves on the database

- A fork copy is created and registered under its workspace's fork lock, and refused once the
  workspace is archived; a rename re-migrates reservations under that lock after archiving.
- Ducklake saves lock every instance database they newly name, as data table saves do.
- Instance database setup holds the database's lock until its entry is written.
- Fork import also holds the database's lock across the restore.
- Cleanup re-reads the entry under its locks before dropping anything.
- Non-superadmins no longer see fork copies reserved for workspaces they are not in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): state the authorization contract of the external cluster status and write check

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): lock the fork copy at finalization, and order cleanup's registry write after its settings row

- Fork finalization holds the copy's database lock from its availability check to the commit.
- Cleanup removes the registry entry in its own transaction, in a task of its own, instead of on a
  second connection; a rename migrates reservations after its settings rewrites, so both take the
  settings rows before the registry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the fork lock across a rename's settings copy

Fork cleanup of the old id could otherwise drop a copy the renamed workspace goes on using. Also
states the authorization contract of the instance database drop helpers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse moving the external cluster to another host or port while it holds databases

Every settings writer goes through the same check as removal: the login, TLS and maintenance
database may still change, the cluster may not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): clone a data table under roles with its owners and grants (#11120)

Claude-Session: https://claude.ai/code/session_01UbrtwiYNfayrmqouBJHwGV

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: open the raw app data table drawer when the workspace has none

Selecting the first data table of an empty list passed undefined to the name
check, which threw instead of opening the drawer on no data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): offer cloning a data table under roles where its grants can be replayed

The server clones such a data table and replays the source's owners and grants,
which only the Enterprise Edition does, so the fork wizard hid both clone
options everywhere instead of on a build that cannot replay them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name the placeholder the empty raw app data drawer renders

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): replace a job directory file at the DuckDB root certificate path instead of trusting it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin the enterprise refusal the role pickers read as 'not under roles'

The server's sentence and the frontend's copy of it were coupled by nothing,
so rewording either one turned every role picker on a community build into a
failed lookup.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): type Ducklake catalogs on the external instance cluster in the settings form

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): read a roles answer only for the workspace it was asked in

A fork and its parent each have their own roles on a data table of the same
name, so an answer stamped with the name alone settled the role from the
workspace the editor was acting on before.

Also derive the AI table creation flag from the data replaced into the editor:
data naming no data table left the flag on from before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep an instance database a settings save is waiting to name

Cleanup for a database whose setup failed took the lock first, read no user,
and dropped it while a save blocked on that same lock was about to commit a
reference to it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check the workspace stamp in the default database selector too

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): read what a save racing instance-database cleanup committed

A transaction blocked on the lock may still roll back, so keeping the database
for it stranded one whose name then blocks every retry: it is let through and
its outcome read instead. The waiter query also matches this database's locks
only, since pg_locks spans the cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): tell a waiting request apart from the workspaces using a database

Both callers render what cleanup returns as the workspaces that keep the
database, so a waiting request's pid read as one of them. Each now words that
case itself, and the give-up comment names where the kept name actually goes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): take the cleanup lock on a connection the pool cannot reclaim

A session lock outlives the future holding it, so a cancellation between
taking it and releasing it handed a locked session back to the pool, where
every later settings save waits on it. Detached, the connection closes when it
is dropped and the server releases the lock with the session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): close the cleanup connection on drop instead of detaching it

Detaching released the pool permit while the session stayed alive, so
concurrent cleanups waiting on their locks could open as many connections as
they liked. Closing on drop covers the same cancellation and keeps them
counted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 179d2454a063ee818a9387a1eafdd354a217a16c

This commit updates the EE repository reference after PR #802 was merged in windmill-ee-private.

Previous ee-repo-ref: c2e43f5b5ff753d339b70e232fd17b6ffecb054d

New ee-repo-ref: 179d2454a063ee818a9387a1eafdd354a217a16c

Automated by sync-ee-ref workflow.

* chore: update ee-repo-ref to fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3

This commit updates the EE repository reference after PR #798 was merged in windmill-ee-private.

Previous ee-repo-ref: 3145e422d61d580f0a82804f075285c112879da0

New ee-repo-ref: fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3

Automated by sync-ee-ref workflow.

* fix(datatables): classify a fork import's target from the resolution that built its connection

A second read could see the entry flipped to a resource and skip the reservation check while the
connection already built still reached the instance cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): run external database lifecycle on the caller's transaction, one lock order for fork cleanup

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check a fork copy's reservation on the locked transaction's connection

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): make and drop fork copies of external data tables on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a fork whose external copy something already names

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check an external fork copy's uses before removing its entry, so child fork pointers still count

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): migrate external cluster fork reservations on workspace rename

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): configure the external instance cluster and pick its databases from the UI

Instance settings gets an External Postgres tab: the cluster's connection, the setup run
with its report, and the databases Windmill created there. Data table and Ducklake settings
offer the external_instance kind, with a picker over those databases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the external cluster form out of the instance settings the page bulk-saves

The form seeded its key into the settings store after the page snapshotted them, so merely
opening the tab made the page send a setting the validator refuses, failing an admin's
unrelated save. The form is local state now, and only the setup writes the key.

The database picker also treats a superadmin-only listing as authoritative: a workspace
admin, who cannot list, no longer sees a saved database as missing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(tests): app paths are validated, so a guest app cannot be renamed to one with a space

Path validation on apps landed after this test, which still expected a space to be accepted
and created its fixture on a ':' path. The scopable-path guard it exists for is kept by
planting that path on the row, which is now the only way an app can hold one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): one Managed Postgres tab, and a switch for Windmill's own database

The instance settings tab covers both substrates Windmill administers: the external cluster,
which can now be disabled once nothing sits on it, and Windmill's own database, which an
operator can turn off so the cluster is the only one a workspace may newly name. A save that
names it then refuses, as the create endpoint does.

Workspace settings name them the way an admin meets them: Postgres Resource, and Managed
instance, qualified as Internal or External only while both are on offer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): offer the external cluster in the add-data-table wizard, and give Ducklake its room back

The wizard gets the external cluster as a fourth substrate: pick a database it already holds
or name a new one, which the run creates before writing the entry, so Try again does not trip
on a database the last attempt made.

Ducklake's maintenance column moves into the settings popover next to the extra args, which
the wider catalog select had squeezed the name box out of.

A setting a component saves itself now also moves the page's baseline, or Discard would
restore what it replaced and a later save would send that stale value back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): only a database this wizard run made makes its retry idempotent

A retry skipped the create whenever the name was registered, which let a run adopt a database
another superadmin had made and share their data without the warning the existing-database
branch shows. The run tracks what it created instead, and reports it so Discard can say the
database is still there. Creating one now refreshes the registry the next run's default name
and its validation read, and that default counts the external cluster's databases rather than
the instance's.

Row ownership carries the substrate: "instance" and "external_instance" can hold the same name,
and a row repointed between them while a run probes must not read as that run's own.

Also fixes the merge leftovers in the DuckDB executor's tests, which cargo check never compiles:
the new PgDatabase field and the attach helper's job directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): shorten the picked managed kind, and say what an empty database list means

The closed select has room for the qualifier, not the whole name, so a picked managed kind
reads "Managed (Internal)". An external entry gets the tooltip its internal counterpart has,
and a database picker with nothing in it says to type a name rather than "No items found".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the wizard's external-database bookkeeping optional and parked

The run deps gained a required field, which every existing caller of runSetup -- the sibling
test suite included -- does not pass. It defaults to empty instead, the parked payload carries
it across a Supabase redirect as it claims to, and a resumed run counts it as something left
behind. Two regressions pin the create: made on a first attempt, skipped only for a database
this run made.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): review an external-cluster database as one, not as a Postgres resource

The review step fell through to the resource branch for the new provider, so it announced a
connection already in the workspace for a database that does not exist yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): show the server's refusal instead of an object in the toast

Saving data table settings passed the whole error to the toast, which rendered the request
and response as JSON, and the instance-database wizard replaced it with "check console".
Both surface the message the API sent, through the helper that exists for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: point at the EE commit dropping the unused import

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): define a role on the cluster its data table sits on

The permissions drawer created and listed roles without naming a cluster, so a role defined
from an external data table became a login on Windmill's own, and the refresh then replaced
the correct catalog with the internal one. The drawer passes the data table's cluster to both.

The wizard also read a database it had just created as a name collision, which blocked
returning to the failed attempt and finishing under the same name.

And the merge had left the CONNECT-grant pass and its log in both the caller and the callee.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): manage the external cluster's role catalog, and name the cluster in the copy

The roles drawer offered one catalog, so the external cluster's roles could only be reached
through a data table sitting on it — and became unreachable once the last one was gone, while
the cluster could not be unset until they were dropped. Opened from the page it now offers the
clusters the instance has; a data table's own drawer still pins its cluster, and its copy names
that cluster rather than "the instance".

A DuckDB attach also decides whether to verify certificates the way every other Postgres
connection does, so a resource carrying a root certificate is no longer downgraded to require
here alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bind the roles list to the cluster it was read for

Switching catalogs left the previous cluster's rows on screen and applied whichever response
landed last, so a slower read could seat one cluster's logins under the other's heading while
every control acted on the wrong id — invisible where a name exists on both. The switch clears
the rows, a token discards a response the selection has moved past, and the controls stay inert
while a catalog is being read.

The drop confirmation also names the cluster whose databases it reaches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56

This commit updates the EE repository reference after PR #810 was merged in windmill-ee-private.

Previous ee-repo-ref: 30db1b33bda0446f5fc5dfe353fbb226d57a26d4

New ee-repo-ref: dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-29 14:45:47 +02:00
hugocasaandClaude Opus 5.5 797147ea7a fix: report a worker's last job when it ran under one poll interval (#11399)
* fix: report a worker's last job when it ran under one poll interval

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: share the unreported job slot with the interactive worker shell

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: pin that a main-loop ping without a job keeps the last one

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 14:27:23 +02:00
Ruben FiszelandClaude Opus 5.5 649c43e7c1 fix: run an AI agent tool on the worker its own tag selects (#11370)
* fix: run an AI agent tool on the worker its own tag selects

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: run a tagged agent tool inline when this worker serves its tag

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: give an inline agent tool a job token of its own

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: report a lost tool wait to the model and cancel tools on agent timeout

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 17:32:25 +00:00
Ruben FiszelandClaude Opus 5.5 65cba2dbb7 perf: advance a flow step with one v2_job_status update (#11357)
* perf: advance a flow step with one v2_job_status update

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: state what advance_flow_status returning None means

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the merged flow advance identical for rows without a status row or with a malformed status

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FbA4shbpCnRsUxir1GEfm

* docs: note the JSON null invariant behind the empty-path no-op

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FbA4shbpCnRsUxir1GEfm

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 00:18:59 +02:00
Ruben FiszelandClaude Opus 5.5 53a5cfd17a perf: skip job-start pings and checkpoint read for short non-WAC jobs (#11356)
* perf: skip job-start pings and checkpoint read for short non-WAC jobs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: pin the wac language gate alongside is_wac_v2

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the start memory sample for jobs shorter than one poll tick

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 18:39:08 +02:00
e14da5c6bc feat(bedrock): add OIDC role assumption as a fourth auth mode (#10936)
* feat(bedrock): add OIDC role assumption as a fourth auth mode

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YCVb91fZp3dqM14KRPzoEn

* fix(bedrock): gate the OIDC cache correctly and assume the role once per job

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YCVb91fZp3dqM14KRPzoEn

* refactor(bedrock): check the OIDC region before minting a token

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YCVb91fZp3dqM14KRPzoEn

* fix(bedrock): keep OIDC session names collision-resistant, gate the copy on EE

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YCVb91fZp3dqM14KRPzoEn

* fix(bedrock): check the OIDC region before reusing cached credentials

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YCVb91fZp3dqM14KRPzoEn

* fix(bedrock): clear assumed-role sessions when AI settings change

invalidate_ai_request_cache_for_workspace cleared AI_REQUEST_CACHE only, so a
workspace's AI settings edit reset one cache and left the assumed-role sessions
keyed on the old config in place until STS expired them.

Also name the region requirement in the credentials-check hint, so following it
does not land on the OIDC path's region guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YCVb91fZp3dqM14KRPzoEn

* chore: update ee-repo-ref to de73db2bacfdc3eaa2e63b1827178bc198d54e5c

This commit updates the EE repository reference after PR #770 was merged in windmill-ee-private.

Previous ee-repo-ref: c43dab1e69b1cb3f685e6df07bff634dc2a0b734

New ee-repo-ref: de73db2bacfdc3eaa2e63b1827178bc198d54e5c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-25 18:19:38 +02:00
hugocasaandClaude Opus 5 d3d5392917 feat: add an options field to the postgresql resource (#11223)
* feat: add an options field to the postgresql resource

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: keep a literal plus in postgres connection string parameters

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: pass postgres options to trigger connections

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: read DATABASE_URL options the way sqlx does

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: include postgres options in databaseUrlFromResource

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-25 17:40:13 +02:00
Ruben FiszelandClaude Opus 5.5 bc4f872f10 perf: skip the flow_env ancestor walk for sub-flows with nothing to inherit (#11349)
* perf: skip the flow_env ancestor walk for sub-flows with nothing to inherit

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the flow_env mark out of sub-flow definitions replayed by restarts

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: align the guest scopable-path test with app path validation

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 10:30:43 +00:00
Ruben FiszelandClaude Opus 5 69fafb9262 feat: check dynamic worker tags on the tag they resolve to (#11271)
* feat: check dynamic worker tags on the tag they resolve to

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin the tag check after a flow preprocessor resolves the tag

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep tied tag placeholders exact and drop an unresolvable flow tag

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: describe tied tag placeholders in the custom tags editor

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: check workflow-as-code task tags on the tag they resolve to

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: judge a preprocessed flow's tag as written when it is run

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: check a preprocessed flow's own tag, as written only where it reads args

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: explain dynamic tags in the worker tag picker

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: look up worker availability for typed static tags

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: poll worker availability with the tag list as it is at each run

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: warn on custom tags with nothing fixed at their start or end

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: open the broad custom tag warning from the keyboard

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 07:30:48 +00:00
Ruben FiszelandClaude Opus 5 2699d5065e fix: check workflow-as-code task tags against CUSTOM_TAGS (#11273)
* fix: check workflow-as-code task tags against CUSTOM_TAGS

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: resolve and check every WAC child before the parent parks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: build parent-code WAC children at push time, not in the pre-pass

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 00:54:34 +02:00
hugocasaandClaude Opus 5 53afecd458 fix: register the job token with the sensitive log masking system (#10943)
* fix(worker): register the job token with the log masking system

The masking system covered secrets fetched through `get_value_internal` and
`$encrypted:` args, but not the job's own token, so a script that echoed
`$WM_TOKEN` wrote it verbatim into logs that are persisted to the database
and, when configured, to object storage.

`run_worker` now registers the token for the job it just pulled, alongside
the existing `register_running_job` call, so it is redacted like any other
registered secret.

That makes every job carry at least one registered value, where before the
per-batch mask snapshot was skipped entirely for the majority of jobs that
touched no secret. Cache the compiled Aho-Corasick automaton per job and
invalidate it when a new secret is registered, so a chatty job no longer
rebuilds it once per log batch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(nativets): mask secrets in the in-process log path

NativeTS hands `console.log` output to a task that drains a channel into
`append_logs`, so it never reaches the masking in `handle_child::write_lines`
and a script logging `$WM_TOKEN` persisted the raw JWT.

That drain can still be flushing after the job is unregistered, so a plain
per-line `snapshot` would leave the tail unmasked. `JobMasker` keeps the last
masks it saw for exactly that window, and refreshes while the job is alive so
secrets fetched mid-run are covered too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(nativets): seed the job masker at construction

A `JobMasker` that only looked up its masks on the first `mask` call had the
same hole at the head of the log that its retention closes at the tail: if the
drain task's first productive poll landed after the job was unregistered, the
registry was already gone and every line was written raw.

`new` now takes the snapshot, and its callers construct it from the job's own
execution while the job is still registered.

Also cover the nativets sink with an integration test, gated on `deno_core` the
way the CI test build is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(nativets): mask on the producing side of the log channel

Masking as the drain task wrote to `append_logs` left two holes, because that
task is detached and outlives the job: a secret registered mid-run could still
be queued when the job was unregistered and would then be written raw, and the
`windmill:job_log` tracing emission that EE forwards job logs on never went
through the mask at all.

Mask where the line is produced instead. That loop is joined before the job
completes, so the job's secrets are always still registered, and one call now
covers both the tracing mirror and the channel. The result stream keeps reading
the raw text, the way `handle_child` keeps its raw `line` for results.

`JobMasker` is no longer load-bearing for the post-unregistration window, so it
is documented for what it now does: keep the security notice to once per set of
secrets for a sink that masks line by line.

Also drop the nativets test's tag override — `DEFAULT_TAGS` does advertise
`nativets`, so the comment justifying it was wrong — and pin the automaton
cache invalidation, whose failure mode is an unmasked secret.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(worker): hold the log-masking lifecycle at the job boundary

Registering the job around the poller's call left every other way of running a
job uncovered: the interactive worker shell and inline AI agent tools both call
`handle_queued_job` directly, and a script logging `$WM_TOKEN` from either
persisted the live credential. Register from inside `handle_queued_job` instead,
under a drop guard, so each path is covered by construction rather than by
remembering to add a call. Nothing is lost by unregistering earlier: the writes
that follow go through `append_logs`, which never consulted the registry.

In nativets, decide the stream/log routing before masking. `MaskSnapshot`'s
notice is one-shot, so a secret-bearing `WM_STREAM:` chunk used to spend it on
text that is then discarded, leaving later redactions in `job_logs` unexplained.

Restore the masker's post-unregistration test: the memory-limit path never joins
the producing loop, so that fallback is still load-bearing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* docs: correct the nativets masking comments

The producer loop is not joined on the memory-limit path, so it does not
"always" run while the job is registered — say normally, which is what
`JobMasker`'s fallback is there for. Name the reason a stream chunk stays raw
everywhere it goes, including the tracing mirror: it is result data that no log
sink persists, so masking it would be masking a result. State the masker test's
invariant without asserting a mechanism behind it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

* fix(masking): keep the security notice on a line of its own

`mask` appended the notice as a newline plus the notice text, which
assumes the caller hands it a bare log line. nativets hands it a chunk
that already ends in a newline, and its sink concatenates chunks
verbatim, so the notice arrived after a blank line and the next log line
was welded onto the end of it.

Emit the notice as its own line for either shape. `handle_child` is
unaffected: its input never ends in a newline, so it keeps the original
path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P316wKe2QCYNcdsx1PwAJ3

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 15:15:02 +02:00
hugocasaandClaude Opus 5 a08992834d feat: render an AI agent result as its answer, not as raw JSON (#11051)
* feat: render an AI agent result as its answer, not as raw JSON

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: sanitize agent markdown through the shared plugin chain

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: fold agent stream events incrementally per poll

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: separate the agent meta line from the result toggle group

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: add a transcript view of an agent run's conversation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: retire AIAgentLogViewer in favour of the transcript

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: show the partial transcript a max-iterations failure carries

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: move the agent meta line and system prompt below the conversation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: show an agent run as what it did, not as a conversation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: name the agent run breakdown a trace

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): label the save-as-agent form fields per the guidelines

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): reuse the resource form's path and description fields

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): keep the action tags on a max-iterations failure

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): keep offline replay inert and the streamed answer to one turn

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(ai-agent): reset the streamed answer on providers that skip tool_call

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: carry the event type narrowing through the stream parser

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: survive a malformed message rather than take the viewer down

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: coerce agent messages once at the parse boundary

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: show an agent run as one scroll ending in its output

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep every streamed turn instead of dropping the narration

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: share the chat divider and drop the unsafe run auto-scroll

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: give the labelled divider a border colour and the standard pretty icon

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: pad the agent run below its badges as well as above

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: follow a streaming run's pane without moving the page

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: share the chat's stick-to-bottom mechanics with the agent run

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep the answer's citations and end a turn's reasoning with the turn

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: read the agent stream through the windmill-chat sdk parser

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: show an agent run's thinking instead of falling back to raw json

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a stream the fold cannot use from claiming the run pane

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: identify an agent run by more than the job id the replay withholds

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: drop the tests and comment lines that were not earning their place

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a streamed turn's text shifting when a tool call closes it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 14:38:46 +02:00
hugocasaandClaude Opus 5 48f00259c5 feat: support $flow_expr[...] dynamic tags on flow steps (#11170)
* feat: resolve $flow_expr[...] dynamic tags on flow steps

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: run an unresolvable $flow_expr tag on the default tag and give it the step input scope

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: resolve $flow_expr tags by path lookup instead of expression evaluation

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: say a $flow_expr tag fails to resolve, not to evaluate

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(frontend): word the $flow_expr help like the other dynamic tag lines

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: reject a malformed $flow_expr placeholder instead of queueing its literal tag

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: render $flow_expr tag values through the $args path lookup

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: pin interpolate_args through the shared tag path rendering

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:37:50 +02:00
0e807fb1dd feat: put a data table's connection under Postgres roles (#11020)
* feat(datatables): put a data table's connection under Postgres roles

A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): gate the paths that reach a whole database as admin

Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.

All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping

A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.

Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.

Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): fail loudly where a role or a pointer can be left half-recorded

Three ways the feature could end up in a state nobody could see or undo.

Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.

Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.

`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.

Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): unbreak two operator messages and two comments that described other code

The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.

`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract

The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.

Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.

The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.

Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.

Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): give the role catalog its own row, out of reach of the config machinery

Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.

`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.

Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.

Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.

Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced

The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".

The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing

The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.

Three ways a save could leave something resolving to nothing:

A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.

Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.

Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.

Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* refactor(datatables): put the role catalog in its own table, not in global_settings

Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.

The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.

One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.

Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): validate a rename against the save it describes, and re-check under the locks

Three from the round, all about deciding on state that could already have moved.

A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.

Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.

The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.

Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically

The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".

The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.

Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse to copy a data table that is under roles

pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.

Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse the clone's database too, not only its data

A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.

Refuse in both, so the clone stops before a database exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* nit worker error msg

* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the parent's settings while a fork points at its data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59

This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.

Previous ee-repo-ref: 38d6fcf2aeb39cfdac21814bbdbbcc02911e566a

New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-18 13:56:41 +02:00
GuilhemandClaude Opus 5 a571117f3f feat: store mcp tool call, result and reasoning on flow conversation rows (#11176)
* feat: store mcp tool call, result and reasoning on flow conversation rows

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: keep the failure reason and web search citations on tool rows

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: persist a structured answer as its own row and keep reasoning-only rows from closing a turn

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: keep a failed Windmill tool's error on its conversation row

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: let a structured answer row claim its streamed thinking

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: scope a structured answer's claim to its own turn and document the row fields as stored

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: claim a textless assistant row only within the newest turn

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor: store the thinking that led to a tool call on the tool row

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep tool calls in stream order and scope a structured answer to its turn

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: store the files a user message carried as object-storage references on its row

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(chat-sdk): read a message's attachments and build their download URL

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(chat-sdk): carry a loaded user message's attachments in AI SDK metadata

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: store the model's call and what it got back on every tool row

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: pass no extras in the orphaned-conversation test's message insert

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(chat-sdk): keep a stored JSON null tool result instead of the row text

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: label a tool call the turn finished without as not finished

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep a structured answer's streamed call out of the did-not-finish label

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: write a turn's conversation rows in order and describe stored tool calls

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: await the image answer row like the agent loop's other rows

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(chat-sdk): place a row nothing streamed by its sequence, not at the end

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(chat-sdk): place only tool rows by sequence, keep a closing row last

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(flow chat): show a failed tool's error, and no Retry on a running or stopped turn

A failed tool card showed the row label in place of the error the row now
stores. The live tool result now reports failures, so a failed call briefly
flagged a running turn as failed; a stopped turn, whose last row is the
failed tool or the cancelled flow's failure, offered Retry too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-17 15:35:18 +02:00
hugocasaandClaude Opus 5 c297ed0052 feat: managed memory with an inherited or custom memory id per step (#11118)
* feat: split ai agent memory into agent policy, run memory id and step history

* fix: scope string memory ids to workspace and flow, keep nested tool history inputs

* chore: update sqlx cache for the flow context query

* docs: describe memory id scoping as collision-free rather than isolated

* chore: regenerate openflow json after merging main

* fix: offer no memory id for legacy manual memory, document linked history inputs

* fix: seed provided messages from legacy manual memory and hide its note once set

* fix: bypass memory when a provided messages expression evaluates to null

* fix: require a user message when provided messages are empty

* chore: keep the empty messages comment within the line width

* docs: name the history inputs wherever linked steps list their flow-local inputs

* docs: keep the memory storage path on one line

* feat: managed memory with an inherited or custom memory id per step

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: list a custom memory id in the test run form and name where an inherited one comes from

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: keep memory id out of the add-field menu and drop the memory id telemetry

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: keep legacy auto memory without an id working after an untouched redeploy

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: rename step messages to previous_messages and address review

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: rewrap comments and docs lines lengthened by the previous_messages rename

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: read agent memory as either a legacy shape or the current one

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: name the memory setting in ignored-input notes and keep conversions honest

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: keep a legacy memory count unset on open and read a cleared count as off

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: address review on cleared test history and zero-count memory

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: drop flow-local keys from a linked agent resource before interpolating it

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: keep a linked resource's own inputs as fallbacks and note ignored history on image runs

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: restore the linked agent draft tests and log ignored history on every image run

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: resolve the one-of variant from the value when the selected one leaves the list

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: treat zero-count managed memory as off when enabling chat mode and shorten comments

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: stop requiring user_message in the openflow agent contract when previous messages are the prompt

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 11:55:43 +02:00
hugocasaandClaude Opus 5 02e47de8b4 feat: allow git sync auto-pull, promotion and PRs on Pro licenses (#11173)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 17:28:40 +02:00
hugocasaandClaude Opus 5 b51c0eabbe feat: stream reasoning summaries in AI agent Responses API steps (#11124)
* feat: stream reasoning summaries in AI agent Responses API steps

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: retry without a reasoning summary a strict gateway rejects

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: request the reasoning summary whether or not the step streams

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: return the OpenAI reasoning summary in the agent step's reasoning

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: retry without a reasoning summary a gateway rejects with 422

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 10:46:15 +02:00
hugocasa 57a99f66a8 feat: rename saved agents from the agent editor and flag broken links (#11147)
* feat: list flows that link a saved agent and flag broken agent links

* feat: rename saved agents from the agent editor and repoint the flow

* fix: show an unreadable linked agent as not accessible, not missing

* fix: address review nits on agent rename and missing-agent state

* fix: open content search above modals and keep Escape for it

* fix: register content search on the opener's overlay stack

* docs: scope the global search z-index comment to the bases it clears

* refactor: show linked agents' rename warning as for scripts and flows

* fix: keep the failed-lookup rename warning to resources
2026-09-16 10:45:01 +02:00
Ruben Fiszel e8078f2a96 fix: dispatch workflow-as-code tasks from a deployed flow's inline step (#11146)
* fix: dispatch workflow-as-code tasks from a deployed flow's inline step

* fix: give a workflow-as-code task its own result-cache key

* fix: key a cached workflow-as-code task on its name and arguments

* fix: hash a cached workflow-as-code task's arguments like any job's

* chore: regenerate system prompts for the task cache_ttl docs

* fix: key a cached workflow-as-code task on its step key, not its name

* fix: key a cached workflow-as-code task on a fingerprint of its code

* fix: keep the task() doc attached to task()

* fix: key a cached inline task on its step key and the workflow input

* docs: cache_ttl has no effect on a taskFlow target
2026-09-16 09:58:29 +02:00
Ruben Fiszel 9a8a9c480c fix: stop reading an array job result as wm_failure or http response (#11154)
* fix: only read wm_failure and wm_labels from an object job result

* fix: serve an array sync result as json, not a composite response
2026-09-15 23:21:53 +02:00
GuilhemandClaude Fable 5.1 c4e878e831 feat: return an ai agent step's thinking in its job result (#11140)
* feat: return an ai agent step's thinking in its job result

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs: say when an agent result carries no reasoning

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 14:28:48 +02:00
Ruben FiszelandClaude Fable 5.1 31c43255fd fix(worker): bound cache transfers and import fetches in bun jobs (#11138)
* fix(worker): bound object-store cache transfers and relative import fetches

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(worker): bound the codebase download and label slow-step warnings

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(worker): log a stalled cache transfer once and ignore a zero cache timeout

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 13:02:44 +02:00
a78beff743 feat: dynamic AI agent toolsets (#11050)
* feat: dynamic ai agent toolsets, and memory as a step input

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address review round 1 on dynamic ai agent toolsets

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: tag enabled_tools and drop the memory step input

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: let an mcp server entry be named by the path the roster shows

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep $res: out of the tool names the enabled tools picker offers

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: name an mcp server by its bare path on the one side that can hold it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: count the enabled tool names that matched nothing instead of logging them

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: narrow an agent's roster in one pass, by whole entries

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin that an mcp summary is rejected against a name that is not

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: regenerate the copilot flow schema after the merge

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: shorten the enabled tools list hint

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: take enabled_tools back to a plain list of tool names

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: keep the enabled tools add-menu hint describing the unset field

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: name a websearch tool that carries no summary of its own

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: reserve the name web search is enabled by so no tool can share it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: spell the reserved web search name with a hyphen

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: reserve __wm_web_search as the name web search is enabled by

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: advance ee-repo-ref past the git sync ci check work

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: shorten the enabled tools description the run form shows

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to e4c1b794d6c5e6e390987341b2840587bbb40348

This commit updates the EE repository reference after PR #785 was merged in windmill-ee-private.

Previous ee-repo-ref: af668462f0f06b02a5f4e0c22e6156858487a518

New ee-repo-ref: e4c1b794d6c5e6e390987341b2840587bbb40348

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-15 11:34:30 +02:00
Ruben FiszelandClaude Fable 5.1 96963080f1 fix(python): parse wheel RECORD paths as RFC 4180 csv fields (#11133)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 09:45:50 +02:00
Ruben FiszelandClaude Fable 5.1 0b1e9c0dda fix: wake a WAC parent from every path that completes its child (#11119)
* fix: wake a WAC parent from every path that completes its child

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L2ibGNBxNd8oa3uQZLHsXn

* fix: park a WAC parent before writing its checkpoint so lock order matches child completion

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L2ibGNBxNd8oa3uQZLHsXn

* fix: check the parent-child link before touching a WAC parent, wrap the fallback error, keep inline checkpoints in lock order

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L2ibGNBxNd8oa3uQZLHsXn

* docs: say the zombie fallback keeps the WAC parent notification in its transaction

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 18:51:21 +00:00
Davide Modolo 56e21bce83 fix(flows): stop re-evaluating skip_if once a loop is in progress (#11008)
* fix(flows): stop re-evaluating skip_if once a loop is in progress

skip_if is a one-time entry gate, but the flow stays at the same step
for a loop's whole lifetime, so it gets re-evaluated on every
iteration. previous_id stays pinned to the module preceding the loop,
but once the loop is InProgress the last completed job is an inner
iteration, and the results proxy in windmill-jseval aliases
results.<previous_id> to that job's result. skip_if then reads the
wrong value and can flip the loop's module to skipped after one
iteration.

Skip the check once status_module is already InProgress.

* fix(flows): match skip_if gate to sibling entry-state allowlists

Rewrite the skip_if gate as a positive allowlist (WaitingForPriorSteps
| WaitingForEvents | WaitingForExecutor), matching the shape already
used by the BranchOne/BranchAll predicate gates, instead of a negative
filter on InProgress. Restart-at-iteration also enters as InProgress;
document it as a separate case rather than folding it into the
aliasing reason, which does not apply there.

Add a regression test pinning skip_if to run once at while-loop entry.
2026-09-14 19:05:45 +02:00
80eba80d6e feat(git-sync): gate GitHub PRs on Windmill CI test results (WIN-2051) (#10096)
* docs: add design doc for automatic git-to-windmill pull sync

* docs: add migration plan and implementation phases to git-sync pull design

* feat(git-sync): add auto_pull settings schema and pull enqueue primitive

Adds AutoPullSettings/AutoPullMode/AutoPullStatus on GitRepositorySettings
(workspace_settings.git_sync JSONB), the GIT_SYNC_PULL_SCRIPT_PATH constant,
and should_pull/effective_poll_interval_s helpers with unit tests. Exports the
EE enqueue_git_pull_job primitive. Foundation for repo→Windmill auto-pull.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): poll repos and auto-pull new commits into the workspace

Phase 1 of automatic repo → Windmill sync. A monitor task (EE-licensed,
single-replica via advisory lock) git ls-remotes each auto-pull-enabled
repository ~every minute and enqueues a pull when the tracked branch moves,
reusing the {workspace_id}:git_sync concurrency key so pulls serialize with
in-flight push commits.

- windmill-store: background (no-authed) resolver get_git_repo_head_for_autopull
  that resolves the repo resource (incl. $var: refs) and ls-remotes; GitHub-App
  repos are skipped here and will sync via webhooks (phase 2).
- monitor.rs: poll/reconcile/persist with optimistic sha advance and failure
  status; targeted jsonb update so concurrent settings edits aren't clobbered.
- edit_git_sync_repository: preserve server-owned auto_pull state on UI save.
- openapi: AutoPullSettings/AutoPullMode/AutoPullStatus + auto_pull field.
- frontend: per-repo "Automatically deploy changes from Git" toggle with last
  sync status; demote the GitHub Actions link to an advanced CI option.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): wire webhook lifecycle + receiver; share reconcile logic

OSS side of phase 2 auto-pull webhooks:
- edit_git_sync_repository creates/removes the repo webhook on save (EE-gated,
  best-effort → falls back to polling).
- monitor poller now delegates to the shared windmill_git_sync reconcile/persist
  helpers (also used by the webhook receiver), removing duplicated logic.
- export the shared reconcile/persist/failure helpers; bump EE ref.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for phase 3 in-app PR creation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): show webhook vs polling status on the auto-pull toggle

When a repo has an active webhook (auto_pull.webhook_id set), the status line
reads "instant via webhook"; otherwise it reads the ~1-minute polling cadence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(git-sync): post PR diff check on dry-run completion (phase 4)

Worker completion hook in process_completed_job: when a DeploymentCallback job
carrying the __git_sync_pr_check marker finishes, parse the dry-run SyncResponse
and patch the GitHub check run with the diff summary (success/neutral/failure).
Export enqueue_git_pull_dry_run; bump EE ref.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref (drop unused GHES webhook_secret)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* revert(git-sync): defer phase 4 PR diff checks (OSS side)

Remove the worker completion hook that posted the PR check run, drop the
enqueue_git_pull_dry_run re-export and the orphaned sqlx cache, bump EE ref.
Phases 1-3 (polling, webhooks, in-app PR creation) are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Revert "revert(git-sync): defer phase 4 PR diff checks (OSS side)"

This reverts commit 0137d3ca48.

* chore(git-sync): point EE ref at restored phase 4 commit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for clone_ref dry-run

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump init-repository hub script to v28784

Picks up the clone_ref param (windmill-integrations#158) so the phase 4 PR-check
dry-run can clone the PR head. Backward compatible; manual pull/push and the
automated pull/poller/webhook all move to the same published version.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for auto-pull admin-permissioning fix

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(git-sync): bump EE ref for superadmin pull fallback

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(git-sync): refresh auto-pull tooltip; bump EE ref for webhook secret encryption

The auto-pull toggle tooltip claimed GitHub App repos would sync via
webhooks "in a future update"; webhook delivery now works, so describe
the webhook-vs-polling behavior accurately. Bump the EE ref to pick up
encrypting the webhook HMAC secret at rest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(git-sync): poll app-backed repos in auto/polling mode

The auto-pull poller skipped app-backed repos (the ls-remote head check
can't authenticate a tokenless URL), so auto- and polling-mode app repos
never synced when their webhook wasn't live. Wire the poller to fetch the
head via the GitHub API for app repos and reconcile. Bump the EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(git-sync): auto-pull UI — direction split, delivery mode, fallback notice

Reorganize the repository card into two clearly labeled directions:
"Push to Git on deploy (Windmill → Git)" and "Pull from Git (Git →
Windmill)". In the pull section:
- new connections default to auto-pull enabled (webhook with polling
  fallback); existing repos load with auto-pull off and are unchanged
- a Delivery selector chooses "Webhook with polling fallback" or
  "Polling only (air-gapped)"
- a notice surfaces webhook_error when delivery falls back to polling
- a reminder to remove any pre-existing GitHub Action that pushed into
  Windmill, to avoid conflicting double-syncs

Adds the webhook_error field to AutoPullSettings (+ openapi) and bumps
the EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(git-sync): clearer push indicator + gate webhook delivery to app repos

- Push-on-deploy is shown with a check icon + concise line (via the
  shared GitSyncModeDisplay, restyled from the oversized "Sync:" text);
  the setup wizard reuses it without the check (pre-save preview).
- The delivery-mode selector only shows for GitHub App-backed repos;
  token-based repos show a "webhooks require the GitHub App (managed or
  GHES)" note with a docs link and poll instead. Bumps the EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(git-sync): fork auto-sync (phase 5) + live deploy check (phase 6)

Phase 5 — fork auto-sync configured at the parent (replaces the *-to-forks
GitHub Actions):
- Add fork_open_prs + fork_pull_sync to GitRepositorySettings (openapi + UI).
- UI: two "Forks of this workspace" toggles in the repo card, gated on
  app-backed and not-a-fork; serialize the flags on save.
- On fork creation, strip the inherited auto_pull block (and fork_* flags) from
  the copied git_sync repo: a fork must not carry the parent's webhook id (it
  would delete the parent's hook on disable) or self-poll on top of the parent's
  fan-out. Push-direction config + installation are still inherited unchanged.

Phase 6 — live deploy status check on the commit (Cloudflare-style): an
in-progress "Windmill" check on the head commit that flips to "Deployed N
changes"; completion handled by the generalized git-sync check hook.

Bump EE ref for the phase 5-6 EE implementation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(git-sync): bump EE ref for PAT auto-pull mode normalization

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): address PR review findings

- webhook_secret: redact from the settings API response and Debug output (still
  persisted encrypted); it's a server-only HMAC key the UI never needs.
- poller: honor each repo's effective poll interval (relaxed ~10 min when a
  webhook is live) instead of probing every ~60s tick.
- settings save: roll back a just-created webhook if the settings transaction
  doesn't commit, so a failed save can't orphan a hook.
- auto-pull head check: fail SSH remotes with an actionable message (background
  polling has no SSH identity) instead of a confusing ls-remote error.
- deploy/PR check summary: a pull result carrying neither changes nor a settings
  diff now falls back to the unsummarized path instead of a false "in sync".
- UI: reset isGithubApp on resource change / failed fetch so webhook + fork
  controls can't show for the wrong repo.
- tests: cover parse_git_sync_changes and format_change_list edge cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): correct feature gating for OSS builds

- monitor.rs: keep the AUTO_PULL_LAST_POLL static, slack const, and
  poll_git_auto_pull_inner all behind #[cfg(feature = "private")] (an inserted
  static had split the cfg off the function, ungating it in OSS builds).
- edit_git_sync_repository: the webhook create/rollback block references
  windmill_common::git_sync_ee (private module), so gate it on
  all(enterprise, private) instead of enterprise only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(sqlx): cache workspace_diff query pulled in from origin/main

Re-merged origin/main (advanced past the earlier merge); regenerate the offline
sqlx entry for the new workspace_comparison test query so SQLX_OFFLINE builds
(cargo_test) pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): address Codex review findings (webhook cleanup on delete)

- Deleting a git-sync repository now tears down its managed GitHub webhook
  (deletion bypassed the sync_repo_webhook lifecycle, orphaning the hook so
  GitHub kept delivering to the instance).
- Worker completion hook rolls back the optimistic auto-pull sha on job failure
  (OSS side of the EE change) + caches the new marker query. Bump EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): delete repo webhook after the removal commits

Codex re-review nits:
- delete_git_sync_repository deleted the webhook before the settings transaction
  committed; a failed save would then leave the repo pointing at a hook that no
  longer exists (sync_repo_webhook treats a set webhook_id as live and won't
  recreate it). Capture the hook id, commit the DB removal, then delete the hook.
- Reword a fork-copy comment to drop drafting-history wording per AGENTS.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): reconcile the edit-path webhook after the settings commit

Codex nit: edit_git_sync_repository ran sync_repo_webhook before the transaction
committed. The rollback only covered created hooks, but sync_repo_webhook also
deletes a hook on disable/switch-to-polling — a commit failure then left the DB
with a webhook_id whose hook was already gone (and it wouldn't be recreated).
Save + commit first, then reconcile the webhook against the durable config and
persist any hook id/secret change (best-effort). Bump EE ref.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): preserve webhook secret on whole-config save + default on visible add

Codex nits:
- edit_git_sync_config saved the client config verbatim, so the webhook_secret
  redacted from the GET response would be dropped (breaking delivery). Preserve
  server-owned auto-pull state (webhook id/secret, synced sha, last status) per
  repo from the existing settings, matching edit_git_sync_repository.
- addSyncRepository (the visible add path) didn't set the auto_pull default, so
  new sync repos added from the UI came up with auto-deploy off. Match
  addRepository's default (webhook + polling fallback).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* refactor(git-sync): drop fork_pull_sync (parent-level keep-forks-in-sync)

Removes the "Keep forks in sync with the tracked branch" toggle and its
fan-out. Pulling the tracked branch straight into every fork was the
inconsistent piece; the consistent model is per-fork branch sync (each
fork tracks its own wm-fork/** branch), which is a separate follow-up.
fork_open_prs is kept. Also tightens the fork toggle-section spacing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): detect dev workspaces in CLI fork branch derivation

isForkWorkspace / computeGitSyncDeployBranch keyed off the wm-fork- id
prefix. Dev workspaces are forks with a custom, prefix-less id, so their
wm-fork/** branch was never derived or created. Detect them via
parent_workspace_id too (which the backend already passes), mirroring the
backend's `parent.is_some() || wm-fork- prefix` rule.

Pairs with the hub-script clone-flag fix (windmill-integrations#163); both
take effect once the CLI is released and the pinned version is bumped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): reconcile webhooks on full-config save

edit_git_sync_config preserved server-owned webhook fields but never
created or deleted the managed GitHub webhook, so enabling auto-pull
through the whole-config endpoint only polled, and disabling or removing
a repo left an orphan hook still delivering. Mirror the per-repository
endpoint: after the commit is durable, reconcile every saved repo's
webhook (sync_repo_webhook) and delete the hooks of repos the save
removed, including the clear-whole-config case. Addresses the Codex nit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): address Codex nits (webhook orphan on cleared auto_pull, fork detection)

- edit_git_sync_config: also delete a repo's old webhook when the save drops
  the repo OR clears its auto_pull. Webhook fields are only preserved onto a
  Some auto_pull, so a save that present-but-clears a repo would otherwise
  orphan its hook.
- GitSyncRepositoryCard: isFork now uses parent_workspace_id OR the wm-fork-
  prefix (was AND), matching the backend/CLI rule, so prefix-less dev
  workspaces are detected as forks and don't show the parent fork-PR toggle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* docs(git-sync): update design doc for the dropped fork_pull_sync

Phase 5 documented "Keep forks in sync with the tracked branch"
(fork_pull_sync) and its fan-out as implemented; that feature was removed.
Rewrite the section to reflect what ships (fork_open_prs), note the drop +
the per-fork-branch follow-up, and remove the stale fan-out mentions
elsewhere. Addresses the Codex nit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): redact webhook secrets from workspace export; fix doc endpoints

- Export (P1): strip the server-owned auto_pull state (webhook secret/id/error
  + synced sha + last pull status) from git_sync before it is written into an
  export's settings.json for both settings formats. The HMAC webhook secret
  must never leave the server (matching the GET-settings redaction), and a
  re-imported workspace must not inherit another install's hook/sync state.
- Docs: the webhook receiver is a single per-workspace endpoint
  /api/w/{workspace}/github_app/webhook (host-aware for managed + self-managed);
  update the stale push_webhook/{id} and instance-global /api/github_app/webhook
  references.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): skip deleted/archived workspaces in the auto-pull poller

The poller scanned workspace_settings directly, so an archived (soft-deleted)
or renamed-away workspace — whose settings row persists — kept polling and
could enqueue a pull into a dead workspace. Join workspace and require
NOT deleted. The EE webhook receiver gets the same filter (ee ref bumped).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): never trust client-supplied server-owned auto-pull fields

Both write endpoints (edit_git_sync_repository, edit_git_sync_config)
persisted caller-supplied auto_pull.webhook_id / webhook_secret /
webhook_error / last_synced_sha / last_pull_status when adding a repo or
newly enabling auto-pull, letting a client inject a webhook id/secret or
fake sync state. Strip those server-owned fields from the request up front;
existing repos re-derive them from the DB (carried over), new ones start
clean and the server (re)creates the webhook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): merge webhook fields post-commit instead of clobbering the row

The post-commit webhook reconcile in edit_git_sync_repository and
edit_git_sync_config wrote the whole pre-reconcile git_sync snapshot back
after the main save committed. A concurrent git-sync edit or poller status
write that landed in the gap could then be dropped by the stale snapshot.
Re-read the current row and merge only the reconciled webhook id/secret/error
for the repos the reconcile actually changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): parent-managed fork sync + PR-on-deploy toggles

Fork sync (push-on-merge-to-forks parity): a parent-level
auto_pull.sync_forks toggle routes changes on each fork's wm-fork/** branch
into that fork workspace, via the parent's existing webhook and one extra
fork-heads listing per poll tick (git ls-remote pattern for token repos,
git/matching-refs for app-backed). Fork state is a server-written
status-only auto_pull blob on the fork's own repo entry; the fork's card
shows a read-only "managed in the parent workspace" line with its branch
and last pull status. Dev workspaces (prefix-less ids) use the same branch
parsing (unit-tested in windmill-common).

PR-on-deploy: opening PRs for Windmill-pushed branches moves into the
deploy pipeline, per repo toggle (promotion_open_prs on the promotion
repo; parent-level fork_open_prs for fork deploys). The push job carries a
marker and the job-completion hook derives the pushed branch (helper
unit-tested against the CLI formula) and opens the PR outbound, so it
works without inbound webhooks; the webhook-side wm_deploy PR arm is
removed. The documented open-pr-* GitHub Actions remain valid alternatives
(PR creation is idempotent).

Fork guards: promotion mode, enabled auto-pull, and fork_open_prs are
rejected on fork workspaces (they are parent-managed; a fork's deploys
always target its wm-fork/** branch) and the promotion card is hidden in a
fork's settings. Enabling auto-pull now also requires EE, and the
post-commit webhook reconcile persists the normalized delivery mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): dev workspaces sync with their environment-label branch

A dev workspace's git branch is its environment label verbatim (dev/
staging, default dev) — a first-class env branch like the documented
push-on-merge-staging layout — instead of the wm-fork/** form. The label
rides the deploy job args (backend → hub script → CLI
--dev-workspace-label), the PR completion hook derives the same head, the
webhook/poller route label branches into the matching dev-workspace child
(poller lists them alongside wm-fork/* via extra ls-remote refs / per-label
API lookups), and manual pulls from the UI pass clone_ref accordingly. The
CLI refuses to deploy when the label branch equals the checked-out tracked
branch, which would otherwise commit fork content straight to it.

Because the branch is keyed on the label, the label is now immutable after
creation: set at create/attach only, the set_dev_workspace_label endpoint
is removed and the settings tab shows it read-only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): nested fork routing + fork-of-dev branch rooting

A fork of a dev workspace now roots its wm-fork/** branch on the dev's
environment-label branch (the content it diverged from) and its PR merges
back into that branch: the backend passes parent_dev_workspace_label with
the deploy (parent row joined in both enqueue paths), the CLI gains
--parent-dev-workspace-label and checks it before the wm-fork- prefix
fallback when rooting a fork-of-a-fork branch, and the PR completion hook
uses it as the PR base.

Fork sync routing covers the whole live descendant chain of the
webhook/poller workspace (recursive, depth-capped) instead of direct
children only, and fork_open_prs is resolved at the root ancestor — only
the root can hold auto-pull config, so grandchild forks sync through it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): PR deploy-preview comment, clearer check copy, app-only hints

- The PR diff completion hook maintains one managed comment on the PR
  (Cloudflare deploy-preview style: workspace, status, commit, collapsible
  change list), upserted per synchronize via a hidden marker. The check run
  stays for required-check gating.
- A settings difference in the diff summary is worded by cause: the PR
  changes wmill.yaml, vs pre-existing drift between the repo's wmill.yaml
  and the workspace, vs undetermined (neutral wording).
- Deploy-status check titles name the target workspace ("Deployed 2
  change(s) to staging"), since GitHub shows a head commit's checks on any
  PR containing it and a bare "Deployed" read as if the PR had deployed.
- Token-based repos see a hint pointing at the open-pr-on-commit /
  open-pr-on-fork-commit workflows where the app-only PR toggles would be;
  an API-set toggle on a non-app repo now logs a warning naming the
  fallback; the design doc lists app-only features and their degradation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): EE-gate auto-pull UI, fork pull clone_ref, no-op push PR gate

- CE: the auto-pull and fork-PR toggles are disabled with an EE badge, and
  new sync repos only default them on when licensed (basic git sync is
  available on CE since #8493, but auto-pull is EE and the backend rejects it)
- The pull modal passes clone_ref for wm-fork- forks (wm-fork/<tracked>/<id>)
  so a manual pull fetches the fork branch instead of the tracked branch head
- PR-on-deploy skips no-op pushes: when the push script reports pushed=false
  (e.g. the deploy was caused by an auto-pull), the completion hook no longer
  ensures a PR, so closed PRs aren't recreated by the sync loop

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore: refresh package-lock after main merge (windmill-utils-internal 1.8.2)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* test: auto-pull e2e integration tests; fix PR comment table formatting

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): runtime license gate for auto-pull saves; user/group promotion-branch parity

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): explain in-sync PR verdicts with the repo's sync filter scope

A PR that only touches files outside the repository's include paths gets
"In sync", which reads as a wrong verdict; the check summary (and managed
comment) now name the filters, e.g. "Only files matching this repository's
sync filters deploy on merge: `f/**` (excluding `f/pat/**`)."

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): clearer card copy/structure; surface PR-creation failures

- Fork sync toggle renamed and kept in the pull section; the fork PR toggle
  moves to the push section with a note that push settings apply to forks
- Fork/dev workspaces' push section names their actual branch instead of the
  tracked-branch line; promotion repos hide the pull direction (promotion
  pushes deploy branches on top of a sync-mode setup)
- Promotion mode line describes the wm_deploy/** branch + merge-to-promote
  flow; workflow-fallback hints lead with the how-to and link to the docs;
  test connection button demoted from accent per brand guidelines
- New server-owned open_pr_error on repo settings: the deploy completion hook
  records why a PR couldn't be opened (e.g. app permission not yet approved)
  and clears it on the next success; shown as a warning under the PR toggles

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix: cfg-gate scope-note helper (dead code on OSS builds)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): license-gate preserved auto-pull; attach strips parent-only settings

- edit_git_sync_repository re-checks the runtime Enterprise gate against the
  EFFECTIVE repo state after preservation: the older-client arm copies the
  existing auto_pull back, which the request-side check never saw
- attach_dev_workspace now mirrors the fork-creation copy on the attached
  workspace's own git sync: promotion repos dropped, auto_pull/fork PRs/PR
  error stripped, and any managed webhook deleted after commit (the attached
  workspace is parent-managed and must not keep pulling its old tracked branch)
- integration test: attaching an auto-pull-enabled workspace strips it

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): detach clears standalone parent; reject label == tracked branch

- detach_dev_workspace clears parent_workspace_id for prefix-less (attached
  standalone) workspaces so they stop classifying as forks and deploying to
  wm-fork/** branches; wm-fork- re-designated forks keep their parent; cache
  invalidations mirror attach
- dev-workspace create/attach reject an environment label that equals a
  git-sync repository's tracked branch (prod's or the candidate's): deploys
  would target the very branch the repo syncs from, and the CLI guard would
  fail every push job after the fact
- CLI unit tests: prefix-less fork beats wm_deploy derivation; isForkWorkspace
  parent-id argument

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(git-sync): bump hub script pins (push 28786, pull 28785)

Published from windmill-integrations #163 with windmill-cli@1.753.1-gitsync.0:
dev-workspace label deploys, fork-of-dev rooting, fork checkout on the
existing remote branch, and the pushed-flag result.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): no parent-only defaults on fork repos; rename strips webhook state

- addSyncRepository skips the auto_pull/fork_open_prs defaults on fork/dev
  workspaces where the backend rejects them (saving a new sync repo from an
  EE fork 400'd deterministically)
- change_workspace_id strips webhook id/secret/error from the copied git_sync
  and deletes the stale GitHub hooks post-commit: they deliver to the old
  (archived) workspace URL, so the new workspace would report a live webhook
  while polling at the relaxed interval; next save re-registers cleanly
- EE: PR diff checks for contributor-fork PRs clone the synthetic
  pull/<n>/head ref (head.ref doesn't exist in the base repo)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* chore(git-sync): bump pull script pin to hub/28787 (synthetic PR ref support)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): targeted jsonb update for open_pr_error (no full-blob clobber)

The full read-modify-write raced the poller's concurrent last_synced_sha /
last_pull_status writes on the same column; mirror the EE status writer and
update only the matching repository element's open_pr_error key.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* style(git-sync): inline EE badge on gated toggles (matches settings nav)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): both directions in page/card descriptions; clearer promotion flow

- Page header and sync-card description mention the pull direction, not only
  push-on-deploy
- Promotion description walks the actual flow (wm_deploy/** branch, merge to
  promote, sync the target workspace) and points at the PR toggle / workflow;
  the Git Promotion docs link now also shows on configured cards, not only in
  the empty state

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): resolve branch-less resources' default branch for fork sync

A git resource without an explicit branch polled as the bare "HEAD" ref,
which the fork/dev-label fan-out cannot scope (wm-fork/<branch>/*), so fork
sync silently never ran on polling-only repos. Resolve the remote's default
branch name with `ls-remote --symref HEAD` (one call for name + head sha);
"HEAD" only remains when resolution fails. The polling e2e test now uses a
branch-less resource to cover this shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): runtime license gate for in-app PR creation

promotion_open_prs/fork_open_prs are rejected on save without an Enterprise
plan (like auto_pull), and the deploy completion hook re-checks the plan
before opening PRs so flags stored while licensed stop driving GitHub calls
after a lapse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): app-aware pull defaults, always webhook delivery, token-repo guidance

- Pull-from-Git defaults on only for app-backed repos (applied when the
  selected resource resolves); polling is opt-in for token repositories,
  with a warning alert recommending the GitHub App (instant pull + in-app
  PRs) or the sync GitHub workflow
- App repos always use webhook delivery with polling fallback: the delivery
  selector is gone and a stored polling mode is normalized back to auto
- Post-save modal reflects the auto-pull state instead of telling the user
  to turn on a toggle that is already on
- Non-app PR hints recommend the GitHub App explicitly

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): single info box for token-repo pull guidance

Merges the instant-pull recommendation with the GitHub Action conflict note,
shown only for non-app repos; app repos need neither, and the redundant
'instant webhook sync requires' line is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): keep the GitHub Action conflict note on app repos

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): anchor docs links to their exact sections

GitHub App references point at integrations/git_repository#github-app, the
workflow hints at deploy_gh_gl#github-actions-setup, and the sync workflow
at git_sync#github-actions (all anchors verified against the live docs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* copy(git-sync): fork workflow hint links to git_sync#github-actions

open-pr-on-fork-commit is documented on the git_sync page, not deploy_gh_gl.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* fix(git-sync): fork PRs are opt-in on new connections too

Only auto-pull and fork sync default on for new app-backed connections;
opening pull requests stays a deliberate per-repo decision.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): default the managed PR on for new app-backed promotion repos

A promotion deploy's wm_deploy/** branch exists to be merged; without a PR
it's an orphaned branch. Fork PRs stay opt-in. Also scope the sync-repo
auto-pull default to sync mode so promotion repos can't pick it up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* docs(git-sync): GHES self-managed app permission setup

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* docs(git-sync): frame permission update against GitHub Actions, not polling

Existing installations don't have polling; their git-to-Windmill direction
runs on GitHub Actions today, so the approval text describes the update as
replacing those workflows and notes every feature is opt-in.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm

* feat(git-sync): gate GitHub PRs on Windmill CI test results (WIN-2051)

Add a "Windmill CI tests" GitHub check run on any PR against the tracked
branch, so a workspace-fork customer can mark it a required status check and
have Windmill's own CI tests (the `// test:` annotation) block the PR —
replacing the GitHub Action that polls the CI-test results API.

Driven by the pull_request webhook (same event as the Phase 4 diff check): a
new git_sync_ci_test_check table tracks one check per (fork workspace, PR head
commit); a per-ci_test-job completion hook (result_processor) and the git-sync
poller sweep (monitor) conclude it. Reuses the Checks: write grant from #9552;
token repos keep the documented Action. See docs Phase 7.

EE logic lives in windmill-ee-private (git_sync_ee.rs); see ee-repo-ref.txt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(sqlx): drop two cache entries orphaned by the merge

Both queries were rewritten on main; nothing in the sources produces them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(migration): describe the CI-test check as webhook-opened, not deploy-opened

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: bump ee-repo-ref (skipped CI-test check on unmapped PRs)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(git-sync): skipped check for unmapped PRs; correct the installation rationale

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): cascade CI-test check rows with their workspaces; review fixes (EE ref)

Deleting a fork (the normal end of the one-fork-per-PR flow) or its parent
must not fail on the check table's foreign keys.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI-test check waits for the head's deploy; ignore retired tests (EE ref)

Design doc: GitLab scope, pending-while-deploying and retired-test rules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI-test verdict matches jobs against current test references (EE ref)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): record the pushed head as synced; CI-test check waits for it (EE ref)

The push completion hook records the commit the deploy push script reports as
the branch's synced sha, mirroring what a pull records. The check row stores
head_ref. Requires the hub push script version that reports the sha; the pin
is bumped once it is published.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI-test check row keeps the newest check run when creates race (EE ref)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): queue CI tests before a deploy or dependency job completes

The dispatch was detached, so a finished deploy did not yet imply its tests
existed. The git-sync PR check reads "deploy settled" as "tests queued", so the
dispatch is awaited at all three sites; errors are still only logged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(ci-tests): sqlx cache for the pull-success readiness clause; bump EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): readiness waits for pull success, failed deploy fails the check; sqlx cache; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): queue resource-triggered CI tests before the update is acknowledged

Same ordering guarantee as scripts and flows: the git-sync PR check
treats a finished pull as "the deploy's tests exist". Also carries the
sqlx cache for the tested-item liveness filter and the EE ref.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): dispatch on flow/resource create, sweep outside the auto-pull lock, pushed-head map

- create_flow and create_resource queue CI tests the way the update paths do
- the check sweep runs after the advisory unlock so GitHub calls never extend
  the auto-pull tick
- AutoPullSettings gains last_pushed_sha (settings round-trips keep it); the
  EE push hook writes it instead of last_synced_sha so auto-pull still pulls a
  commit someone else pushed under ours

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): last_pushed_sha is server-owned auto-pull state

Clear it from client-supplied settings, carry it over from the existing
entry on save, and keep it out of workspace exports, like the other sync
state fields. Correct the two doc comments that still described the
pushed head as the synced head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(git-sync): say which pull toggles the PR checks depend on

The repo webhook created by the pull toggle is what delivers pull request
events, and fork sync is what lets a fork PR's externally pushed commits
reach the fork workspace before its CI tests check is read.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(git-sync): CI check readiness reads a synced-head event log

New git_sync_synced_head table: one row per commit a workspace reflects
on a branch, written by the pull completion hook on success (from the
pull marker's branch and sha) and by the push completion hook from the
push script's reported sha. The check reads that instead of inferring
sync state from the auto-pull settings jsonb, so AutoPullSettings gets
no pushed-sha field and the settings gates for it go away. Pull success
now also runs the auto-pull completion hook.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): queue CI tests before the deploy push is enqueued; schema summary

The synced-head row a push writes is what makes the PR check read the
fork's results, so the tests must be queued before the push job exists.
Reordered at the script create, lockfile, and resource sites (flows
already push from their dependency job). Lists the two new tables in
summarized_schema.txt.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): newest synced head decides readiness; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): newest synced head decides readiness

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): synced heads keyed by repository; check row keeps the repo resource path

Both tables are unreleased, so their migrations gain the column in place.
The push hook reads the push job's repository from its args again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): CI check rows keyed by repository; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(git-sync): pin the deploy push script to hub/28956

The version whose main returns the push outcome {pushed, sha, branch},
which the completion hook records as the branch's synced head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(ci-tests): tooltips name GitHub for the CI check; drop the sanitization test; docs and EE ref

The sanitization test guarded a field that no longer exists and now only
re-exercised pre-existing behavior. The design doc carries the repository
key and the row-before-create order.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): re-fire resets the run id; pinned-script reason on timeout; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(git-sync): record the commit a pull checked out; skip a rebased push

The pull script now reports the sha and branch its clone landed on, so
the synced head is what was applied rather than the head observed when
the pull was enqueued (the marker stays the fallback). A push the script
had to rebase sits on commits the workspace has not pulled, so it is not
recorded; the pull those commits trigger records the head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(git-sync): pin the push script to hub/28958 and the pull script to hub/28957

The versions whose results report a rebased push and the commit a pull
checked out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): queued-pull wait and kept failure on re-fire; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(ci-tests): sqlx cache for the prune and skipped-guard queries; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(ci-tests): a failed head passes only on newer runs; EE ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ci-tests): the PR check runs the head's own suite

The synced-head row records the CI test runs dispatched for the head
(tests_dispatched_at, ci_test_job_ids); the check reads exactly those.
windmill-git-sync now owns the check lifecycle (it can enqueue jobs) and
depends on windmill-dep-map for the dispatcher. The deploy-site changes
that ordered per-item dispatch before the push existed only for the old
workspace-wide verdict and are restored to main's form; flow and resource
creation keep the per-item dispatch they lacked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): suite runs are not debounced, run as the workspace admin, and a dead dispatch claim is retaken

EE ref f30eaf8: the head's suite no longer shares debounce keys with
deploy-triggered runs (a superseded run dropped out of the verdict), runs as
the workspace admin an auto pull resolves rather than the sync job's identity
(which retention could remove), and a dispatch claim that never recorded ids
is retaken after 5 minutes. sqlx cache and design doc updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): the head's suite expands wildcard references

EE ref 53d77c3; sqlx cache updated for the reference and script queries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): bound the failed-deployment gate; keep a current head's check row

EE ref 63523d5; sqlx cache updated for the two changed queries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): the head's suite runs as the fork's owner; poster_workspace_id

EE ref a74af94. The unreleased check-table migration renames
github_workspace_id to poster_workspace_id (the workspace whose credential
posts the check); sqlx cache, schema summary and design doc follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(git-sync): the PR check's suite runs as the fork's owner

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): CE-only build, UI pull-script pin, detached sweep

- The four check-lifecycle re-exports exist only on enterprise builds; gating
  them on private alone broke the CE image (private without enterprise).
- hubPaths.json's gitInitRepo follows the server's pull-script pin (28957),
  as every previous bump did and as the git-sync integration test asserts.
- The monitor spawns the CI-check sweep detached (EE ref f96aa31).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: bump EE ref to a1d7a1e

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(ci-tests): advance the PR check off the worker's completion loop

Concluding a check calls GitHub; the completion loop finishes jobs serially,
so the evaluation is spawned detached (idempotent, retried by the poller).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(git-sync): check-row-before-create order, retarget events; EE ref 40079e0

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: update ee-repo-ref to a4da009a5eae72bd55f34de41ba7929b53d53c9b

This commit updates the EE repository reference after PR #662 was merged in windmill-ee-private.

Previous ee-repo-ref: 4845e823bfaa28d3258b4c2fb1e9e2781dde382f

New ee-repo-ref: a4da009a5eae72bd55f34de41ba7929b53d53c9b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-14 11:42:57 +02:00
Ruben FiszelandClaude Opus 5 c90d1d95c2 refactor: make the app policy's principal the authority for its identity (#10440)
* refactor: make the app policy's principal the authority for its identity

* fix: align the app backfill with the sibling migration and audit the uncached address

* chore: refresh the sqlx cache after rebasing onto the merged base

* fix: resolve the app execution address uncached, it decides the job's authorization

* chore: cache the EE queries at the ref this branch pins

* chore: cache the EE queries at the ref this branch pins

* fix: derive the app draft's on-behalf-of address on read

* chore: cache the query the draft derivation test added

* fix: derive the app identity on the draft-table and version reads too

* docs: state the draft resolver's authorization contract

* fix: resolve a draft's principal against workspace membership only

* chore: cache the membership lookup the draft resolver added

* fix: drop an unresolvable draft's address instead of leaving it stale

* perf: evict the address cache on change so app dispatch can read it

* fix: evict on superadmin role changes, not only address changes

* refactor: make the app policy's address optional instead of derived on read

* fix: follow an external superadmin's rename into the apps that name them

* docs: state the removal gate once, and correctly

* refactor: drop the app-policy version constant that gated nothing

* docs: drop the last reference to the removed constant

* perf: read the address cache everywhere now that eviction reaches every replica

* fix: keep persisted addresses off the cache the poller evicts asynchronously

* docs: state where the cached address is accepted and where it is not

* docs: keep the cache rule in one place and drop the stale premise

* docs: sort the two lookups by how long a wrong answer lives

* fix: resolve the schedule address uncached where it is written to the row

* docs: name the release this actually ships in

* perf: evict a superadmin's key per workspace instead of the whole cache

* fix: evict every alias a superadmin principal can be spelled as

* docs: describe the trigger as it is

* docs: cover the round-tripped read in the cache rule

* docs: record why a stale dispatch address cannot escalate

* fix: validate a dispatch address against the principal's live binding

* fix: carry the validated address through to the job row and token

* fix: record the validated address on the job row, not the one handed in

* test: run the substep tag check as the non-superadmin it means to test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: rewrite a stored app address that disagrees with its principal

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: record the accepted staleness window of the cached dispatch address

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: record the validated address on the job's audit row

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: record the accepted rename race of pre-transaction identity resolution

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: separate the app's stored address from the derived one in the resolver doc

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: describe the job identity fast path the push comments skipped

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: backfill a legacy group-prefixed username as the group it names

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: resolve a schedule edit's identity before opening its transaction

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: never resolve a disabled member to a same-named superadmin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: state what the email-change notify buys, and rewrap two comment lines

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: keep a group's runnables when offboarding a legacy group-prefixed member

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* fix: read the app author from the stored address, as execution does

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: record the rename race's full consequence as a known, accepted limitation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

* docs: record the keep-target group address case as a known, accepted limitation

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JY4bBCR1q2c5XB8s2r7Ysc

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 09:10:15 +02:00
Ruben FiszelandClaude Opus 5 4afb9aa677 fix: bundle deployed bun scripts whose only pin is on a dynamic import (#11096)
* fix: bundle deployed bun scripts whose only pin is on a dynamic import

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EH42obCk6WJnc7N4Fa25JH

* fix: retry the no-db prebundle too, and guard bundles bun builds as written

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EH42obCk6WJnc7N4Fa25JH

* fix: name the bundle retry after the import specifiers it unpins

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EH42obCk6WJnc7N4Fa25JH

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 00:02:38 +02:00
Ruben FiszelandClaude Opus 5 9fc50a23fb feat: make snowflake_oauth work as a dbt warehouse on every engine (#11095)
* feat: make snowflake_oauth work as a dbt warehouse on every engine

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* fix: scope the early token refresh to dbt, never follow jail symlinks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* fix: lock early token refreshes per account, keep the profile until the new one renders

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* fix: hold the refresh lock until the new token is written

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* fix: poll the refresh lock with a bound instead of pinning a connection

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* refactor: drop the early OAuth refresh from the dbt warehouse route

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* fix: keep endpoint keys like token_uri in the profile identity

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

* fix: mask access key ids with the secrets they pair with

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gD7E1cPVhX1kLsB4fAH3b

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 00:00:17 +02:00
Ruben FiszelandClaude Opus 5 30ffdbecc1 fix: unpin only the specifiers in the bundle a bun modules run executes (#11083)
* fix: keep version pins from imported scripts in bun lockfiles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* fix: strip version pins from the bundle a bun modules run executes

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* fix: unpin only module specifiers, not matching text elsewhere in the script

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* docs: name the raw endpoint lock generation fetches imports through

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* fix: leave require calls alone and skip spans not on a quote pair when unpinning

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* test: create the bun bundle cache dir a dependency job saves into

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* test: drop the lock test #11082's module test already covers

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* fix: narrow the change to a fail-open strip of the modules-run bundle

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

* fix: unpin only the specifiers in the modules-run bundle, and log a parse fallback

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WCq4nkUjiZBnPPMuGYCo6w

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:09:45 +00:00
Ruben FiszelandClaude Opus 5 d539e8674f fix(dbt): stop dbt sending anonymous usage stats from workers (#11091)
Claude-Session: https://claude.ai/code/session_01KJZKmyRJZWUmSS7H1vV6Hs

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:01:20 +00:00
hugocasaandClaude Opus 5 57f8b0826a fix: keep pinned import versions of imported scripts in bun lockfiles (#11082)
* fix: keep pinned import versions of imported scripts in bun lockfiles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: guard pinned imports through an unlocked multi-file bun run

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 08:48:02 +00:00