mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-23 16:03:33 +00:00
7516a3ef0b49ffbe616e765bf0af5a1eabf032c3
51
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Ruben FiszelandClaude Opus 5.5
|
cedd6dc901 |
fix: hold interpolated references and captures to the token path scopes (#11391)
* fix: hold interpolated references and captures to the token path scopes Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: let a resource read cover its own linked secret variable Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: resolve policy-granted app upload resources on the viewer's rls Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: cover multi-secret linked variables and keep capture paths out of refusals Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
 
|
0e807fb1dd |
feat: put a data table's connection under Postgres roles (#11020)
* feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.
A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.
Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.
A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.
Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): gate the paths that reach a whole database as admin
Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.
All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping
A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.
Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.
Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): fail loudly where a role or a pointer can be left half-recorded
Three ways the feature could end up in a state nobody could see or undo.
Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.
Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.
`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.
Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): unbreak two operator messages and two comments that described other code
The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.
`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract
The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.
Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.
The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.
Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.
Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): give the role catalog its own row, out of reach of the config machinery
Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.
`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.
Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.
Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.
Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced
The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".
The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing
The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.
Three ways a save could leave something resolving to nothing:
A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.
Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.
Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.
Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* refactor(datatables): put the role catalog in its own table, not in global_settings
Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.
The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.
One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.
Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): validate a rename against the save it describes, and re-check under the locks
Three from the round, all about deciding on state that could already have moved.
A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.
Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.
The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.
Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically
The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".
The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.
Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse to copy a data table that is under roles
pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.
Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse the clone's database too, not only its data
A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.
Refuse in both, so the clone stops before a database exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* nit worker error msg
* fix pg_dump stuck on version 17 on nix
* fix(datatables): refuse a malformed role annotation instead of ignoring it
`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.
A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.
Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse a malformed role query string instead of ignoring it
`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.
The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): carry the role annotation into the row_to_json retry
The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".
Carry the leading comment block over. The retry itself is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore(datatables): don't mount the roles UI until the ACL editor lands
Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.
The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test
Two review findings, both real.
`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.
`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): clone a fork's pointer instead of failing after the copy
Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.
Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.
Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse to roll back the catalog while roles exist
The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.
Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* refactor(datatables): settle clonability in one place, before anything is created
A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.
That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.
Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): let a retried clone reclaim its own leftover database
A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.
`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Revert "fix(datatables): let a retried clone reclaim its own leftover database"
This reverts commit
|
||
 hugocasaandClaude Opus 5
|
40f0cab2ad |
fix: scope capture deletion to the workspace in the request path (#10795)
* fix: scope capture deletion to the workspace in the request path Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: layer the capture fixture on base instead of duplicating it Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
8e508ea01a |
feat: support application default credentials for gcp pub/sub triggers (#10778)
* feat: support application default credentials for gcp pub/sub triggers Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address review findings on gcp application default credentials Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address review nits on gcp application default credentials Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: key the gcp credential-mode permission off the loaded mode Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: gate enabling an ADC gcp trigger on workspace admin Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: lock the gcp trigger row while authorizing a mode change Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: skip admin-only gcp listing when the caller cannot use those credentials Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 54bf630681000c8ed87a7067e357118e015123b1 This commit updates the EE repository reference after PR #738 was merged in windmill-ee-private. Previous ee-repo-ref: 91d0e228a0ad226625278b400c64f96a61404a10 New ee-repo-ref: 54bf630681000c8ed87a7067e357118e015123b1 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
|
68debab877 |
feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230)
* feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library Fixes WIN-2214 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish Revert the amqp_trigger additions to the shared windmill-utils-internal TriggerDeployKind and the frontend cross-workspace deploy adapter: the frontend installs the published npm package, which lacks the new kind until a release is cut. AMQP create/edit/delete/list/sync/capture are unaffected (they use local types); only cross-workspace deploy/merge of AMQP triggers waits on the package bump. Also document the at-most-once ack in the consumer loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts - ack AMQP deliveries only after successful dispatch; nack+requeue on failure - add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on workspace deletion (and the listener stops) - fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to WorkspaceDiffRow.kind - register AMQP in the generated workspace trigger tool (create_trigger) - drop banned $bindable defaults on optional props in the config section - add build_uri unit tests (encoding, ports, vhost) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): stop AMQP poison-message loop and reconnect on transient drops Chaos testing against a live RabbitMQ broker showed the previous nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000 critical-error reports/sec) on a poison message, and any connection blip permanently disabled the trigger (lapin has no built-in reconnect). - on dispatch failure: nack+requeue then stop consuming; the listener framework re-lists the trigger after its ping goes stale (~15s), backing redelivery off to that cadence instead of a tight loop (verified: rate dropped from ~1000/s to ~1 per ~26s, message preserved) - on connection/stream error: stop and let the framework reconnect instead of disabling; persistent failures are still disabled via get_consumer (verified: a forced connection close now auto-reconnects and resumes) - finish the AI create-trigger action wiring for AMQP: add amqp to CreatedResourceTriggerKind, the action-card registry, and the drawer registry so the result card renders and its "Open" action works Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP frontend registries and defer merge rows - add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES - wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union, writeTriggerSchema, triggerServices, and the draft adapter - stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred until windmill-utils-internal is published — avoids a deploy that fails with "Unknown kind: amqp_trigger" - use design-system TextInput instead of raw <input> in the config section Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP session/draft registries and constrain prefetch - add amqp to the session-deploy, draft-compare, preview-router, and copilot workspace-item registries so AMQP drafts/deploys/nav/path resolution work - include amqp_count in the MoveDrawer attached-trigger rename warning - replace the raw prefetch <input> with a design-system TextInput bounded to an integer 1-65535 (backend u16) and block save on invalid values Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now mirrors the Kafka trigger's explicit reconnect loop instead of relying on the framework re-list (which disabled the trigger once get_consumer failed on a sustained outage): - get_consumer returns cheaply; consume owns a (re)connect loop that retries with a 30s backoff, reports a critical error every 10 failed attempts, and reports a recovered critical error once it reconnects — never disabling the trigger on a connectivity failure - a consumer/stream error breaks out to reconnect rather than disabling - dispatch failure still nacks+requeues (at-least-once) with a short backoff to avoid a tight poison-message loop, keeping the connection alive Verified against a live RabbitMQ broker: killing the broker keeps the trigger enabled and retrying (attempt N), and restarting it auto-reconnects (logs "reconnected after N attempts") and resumes dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP capture registries and constrain prefetch contract - add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor from a capture button no longer throws "Unknown TriggerKind: amqp" - register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the script/flow CaptureButton menu - bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust u16) and regenerate clients/prompts - require a non-empty exchange name when the exchange binding is enabled - build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types - asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas - frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu, node presentation, event-trigger set, annotation keywords, and the editor/service registrations - git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync deployment stages only its .amqp_trigger.* file, not an unrelated same-path object - preprocessor starters: add the AMQP event to the generated TS/Python/PHP trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/ delivery_tag) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists - fix a stray edit that corrupted the pre-existing MqttTriggerEditor import ($lib/... path) in PipelineTriggerEditors.svelte - reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0 as unlimited) and defensively skip basic_qos(0) in build_consumer (covers the capture path that bypasses CRUD validation) - recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never runs an AMQP-only node as a manual root without an event - add amqp to the preprocessor intro lists and both pipeline AI instructions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists - extract a shared validate_amqp_options used by both CRUD validate_config and build_consumer, so capture configs (which bypass CRUD validation) also reject prefetch 0 instead of silently connecting with an unlimited buffer (+ unit tests for 0/1/65535/None) - add AMQP to the main script-writing preprocessor-sources prompt and the CLI triggers-skill guidance list Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text - keep the zero-prefetch rationale only on the shared validate_amqp_options doc; drop the redundant call-site comments - correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved") Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore(migrations): consolidate the four AMQP migrations into one The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind, draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger table doesn't reference those enum types, so they can share a single migration instead of four. Verified applying cleanly in a single transaction on a fresh DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
|
d6c642b170 |
feat: add Azure Event Grid triggers (#8888)
* feat: add Azure Event Grid triggers (EE)
Introduces a new enterprise trigger kind `azure` that supports three
modes via a single unified trigger type:
- basic_push: Azure Event Grid basic — custom topics, system topics
(Storage, Resource Manager, Key Vault, etc.), domains (push only)
- namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push)
- namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token
ack/reject for dead-lettering)
Auth uses a Service Principal resource (tenant_id, client_id,
client_secret, subscription_id). Subscriptions are created in
CloudEvents 1.0 schema so the push webhook handler and the pull listener
share one payload parser.
Backend
- New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from
windmill-ee-private)
- Migration `azure_trigger` table with CHECK constraints enforcing
mode/columns coherence
- `TriggerKind::Azure`, `JobTriggerKind::Azure`,
`DeployedObject::AzureTrigger` variants
- Push route `/api/azure/w/{workspace}/*path` handles classic
Event Grid SubscriptionValidation handshake and CloudEvents 1.0
abuse-protection OPTIONS handshake
- Optional inbound JWT validation (audience check only for v1)
- Feature flag `azure_trigger` propagated through windmill-api,
windmill-store (resource helper), and added to ee_core
Frontend
- `triggers/azure/` editor with mode toggle (basic/namespace-push/
namespace-pull) and per-mode config (topic ARM id / namespace +
topic name / subscription / filters / push auth / pull options)
- Registered in icon map, display names, save functions, badge,
wrapper, editor, add-trigger menu
OpenAPI
- `AzureTrigger`, `AzureTriggerData`, `AzureMode`,
`AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection`
schemas; `/azure_triggers/*` endpoints; client regenerated
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8
This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private.
Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9
New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8
Automated by sync-ee-ref workflow.
* feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity
Frontend:
- Split mode selector into Namespace/Basic + Pull/Push
- ARM resource dropdowns (namespaces, Basic topics, namespace topics)
populated from the service principal; cascade with stale-selection
reset on SP / edition change
- Remove stale authenticate toggle + audience input (server-managed
push_auth_config has replaced them)
- Azure listing page: "Create from template" button; "Also delete Azure
subscription" toggle in the delete modal; simplified trigger label
falling back to path
- AzureCapture.svelte: "Test subscription name" with -wm-capture suffix
- CompareWorkspaces.svelte: wire Azure for fork/compare
- Drop Trigger-deployed/event-loss warning (capture subscription is
isolated with -wm-capture)
Backend:
- Shared-secret push auth (see EE crate for detail)
- JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)]
so clients/CLI/exports never see it
- Drop redundant enabled column; mode supersedes
- Azure capture infra: AzureTriggerConfig + set_azure_trigger_config +
azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on
capture subscriptions so they bound storage after tab close
- Granular ACLs, users offboarding, trash, git-sync deployed-object:
all include azure_trigger
CLI:
- Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath,
trigger commands (get/update/create/list/template), sync delete
switch + regex; e2e test for `trigger new --kind azure`
- system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger;
auto-generated/* regenerated
Skill:
- .claude/skills/adding-a-trigger/ checklist covering every file that
needs editing when wiring a new trigger type (learned from this PR)
ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts
- frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger'
to the Kind type so the listing page's "Permissions" action compiles
(ts2345 — caught by npm_check on CI, missed by fast-check locally).
- system_prompts/auto-generated/: regenerate to drop the stale
delivery_config / AzureDeliveryConfig fields from the Azure schema
(check-freshness on CI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(azure-trigger): use workspace constant_time_eq crate
Drop hand-rolled constant-time compare in favour of the workspace
constant_time_eq crate (same one used by http_trigger_auth).
ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): pass placeholder + disabled via inputProps
`TextInput`'s `placeholder` and `disabled` go through its `inputProps`
prop — CI's `npm run check` caught the stale top-level passing that
`npm run check:fast` missed. Align with the DefaultEmailConfigSection
pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213
The hub deploy of the azure-aware sync-script is version 28213, not
28214. Backend was pinning a non-existent hub script, which broke the
git_sync_e2e suite (every deploy's sync step 404'd).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): add azure_triggers to token scope selector + skill
- windmill-api/src/token.rs: `build_trigger_scope_domains` was missing
`("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope
selector didn't surface azure_triggers:read/write. Backend already had
`ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it.
- .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related
files under the hardcoded-arrays section so future triggers don't miss
the UI surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(adding-a-trigger-skill): clarify token.rs scope effect
Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS
just means the scope works via API/CLI but has no UI checkbox.
* docs(adding-a-trigger-skill): trim token.rs bullet
* fix(azure-trigger): regen openapi-deref + swap textarea for TextInput
- Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the
12 azure_triggers paths + schemas. These files are served by the
runtime (include_str! in windmill-api/src/lib.rs) to external SDK
consumers; without this regen the new endpoints wouldn't be advertised.
- Replace the raw <textarea> for event type filters with the
design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw
HTML elements).
Addresses cubic + claude PR review items.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
|
0389d9601c |
chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add route reachability tests for ~80 previously untested endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update new trash routes to axum 0.8 path syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: upgrade route tests to assert 2xx responses with proper data setup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: restore npm_proxy and ai_routes tests using local echo servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate workspace fork test behind enterprise feature flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings from axum 0.8 upgrade - Use cookie value_trimmed() instead of value() for cookie 0.18 compat - Update comments still referencing old :workspace_id syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1 This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private. Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1 Automated by sync-ee-ref workflow. * test: add test for new get_imports endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused import in raw_apps test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
|
ec20d76216 |
feat: add auto_commit option to Kafka triggers with advanced UI badges (#8317)
* feat: add auto_commit option to Kafka triggers with manual commit API
Add ability to disable auto-commit on Kafka triggers so users can
manually commit offsets after processing messages. This prevents
message loss when processing fails.
Changes:
- Add `auto_commit` column to kafka_trigger table (default true)
- Add POST /kafka_triggers/commit_offsets/{path} endpoint using
BaseConsumer with manual assign() to avoid rebalance
- Enrich trigger_info payload with partition and offset fields
- Conditionally commit based on auto_commit setting
- Add auto-commit toggle to frontend Kafka trigger config
- Add commitKafkaOffsets helpers to Python and TypeScript SDKs
- Add integration tests for auto_commit DB defaults
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: use DB-based pending commits for kafka manual offset commit
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: pass trigger_path to all v2 preprocessors, secure commit_offsets endpoint, fix commit semantics
- Add trigger_path to v2 preprocessor event for all trigger types (kafka, nats, sqs, mqtt, gcp, postgres, websocket, http, email)
- Secure commit_offsets endpoint: infer trigger from job token (OptJobAuthed) instead of requiring trigger path parameter
- Fix auto_commit: only commit offset after successful job push
- Fix pending commits: commit offset+1 (Kafka semantics) and use CommitMode::Sync
- Update TS/Python clients and frontend preprocessor templates
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add advanced section badges and reorganize kafka trigger settings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove dead wm_trigger assertions from kafka e2e test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* sqlx
* refactor: remove unused advancedCollapsed state from all trigger editors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* update ref
* chore: update ee-repo-ref to ed2c9d360e6fab866b9744cc79f50038d1fc7152
This commit updates the EE repository reference after PR #452 was merged in windmill-ee-private.
Previous ee-repo-ref: 5b31116a1d5a042c6a780732901cfd89584d1773
New ee-repo-ref: ed2c9d360e6fab866b9744cc79f50038d1fc7152
Automated by sync-ee-ref workflow.
* fix: use path-based auth for kafka commit_offsets endpoint
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref to fcd3ea52b0cc94fbe1159baf662a38da947456de
This commit updates the EE repository reference after PR #457 was merged in windmill-ee-private.
Previous ee-repo-ref: b3a5c33c92cb1b2caf7a65986d71da291ff72a35
New ee-repo-ref: fcd3ea52b0cc94fbe1159baf662a38da947456de
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
|
||
|
hugocasa
|
f0bbdd4aa8 | feat(backend): pass all headers and query to webhook preprocessor (#6931) | ||
 dieriba
|
426065efee |
fix: add ack deadline gcp (#6625)
* update ref * add ack deadline * update ref |
||
|
dieriba
|
e61477fcbc |
refactor: trigger listener (#6401)
* base * base crate * websocket updated * crud refactored * fix and nits * fix compiler warning, nits and update repo ref * fix oss * fix compilation * update ref * fix * update feature * listener base * add listener * refactor logic done and implemented for postgres * fix capture * websocket * implem for all triggers * update sqlx,repo ref and fix * fix oss * unify struct fix websocket * nits and fix oss runtime axum error * perf cache query * add listener module * update .sqlx and cargo.toml * fix import * fix * fix * update repo ref * fix * update ref * update ref * fix * fix * revert * update ref |
||
|
36bbde6239 |
feat: email triggers (#6548)
* feat: email triggers
* Change down migration to drop email_trigger table
* email triggers UI
* bug fix
* Apply suggestion from @ellipsis-dev[bot]
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* cli and git sync
* Revert "cli and git sync"
This reverts commit
|
||
|
dieriba
|
f3fd1e90b0 |
feat: refactor trigger crud (#6472)
* base * base crate * websocket updated * crud refactored * fix and nits * fix compiler warning, nits and update repo ref * fix oss * fix compilation * update ref * fix * update feature * listener base * add listener * refactor logic done and implemented for postgres * fix capture * websocket * implem for all triggers * update sqlx,repo ref and fix * fix oss * unify struct fix websocket * nits and fix oss runtime axum error * perf cache query * update .sqlx * update .sqlx * fix * fix unused * fix * fix * nits and fix http handler update endpoint * update .sqlx * update repo ref * nits * fix * update ref * fix * update .sqlx |
||
|
dieribaandRuben Fiszel
|
852bf064dc |
feat: togglable manual acknowledgement for gcp trigger (#6321)
* update * done * update repo ref * nits * update repo ref * fix * all --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
dieriba
|
aba8c01d7f |
feat: windmill http triggers and webhooks to openapi spec (#5918)
* add sum and description to http routes * add to openapi spec * add subsection * add collaspable and action button * added path rendering, * migrate logic to backend * remover server and handle different format generation * add filter for http route and document generated * remove print * handled webhook and integrated server component * done * add download and copy * nits * update .sqlx * remove .vscode * fix npm check * add summary description and fix toggle * added security handling and nits * update .sqlx * nits and rename key * remove unused code * update ref * nits rename var * nits * nits * nits * nits * nits * add token generation for cURL command * create token * update label and remove section * update repo ref * clean * let brower start download before cleaning up resource * handle sync/async for webhook * format fix * nits * nits * reset sum and description |
||
|
dieribaandellipsis-dev[bot]
|
39dbd646b9 |
feat: use rust-postgres client instead of sqlx for postgres trigger (#5853)
* use rust-postgres client instead of sqlx * fix * Update backend/windmill-api/src/postgres_triggers/mod.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix import --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> |
||
|
Diego Imbert
|
0e316239dd |
EE Refactor (#5844)
* app compiles with every ee substituted
* Replace all oss files content
* Revert "Replace all oss files content"
This reverts commit
|
||
|
dieriba
|
4cbcbdb960 |
feat: fix backward compatibility pg 14 for postgres trigger (#5851)
* remove macro query when interacting with remote db * fix compatibility and nits * fix bug save for publication tracking all table * fix infinite loop svelte 5 * nits on (gcp/postgres) trigger+ refactor on postgres trigger * update .sqlx * update repo ref * uodate repo ref * fix * fix * nits + extend postgres capture * fix import gcp * fix reactivity issue * add postgres transaction for successive write operation * fix isValid state edge cases * fix reorder transaction * reorder transaction * remove unused import * update test connection * add missing props * fix import/bugs and nits * nits and fix small bugs * add missing colon |
||
|
30edcdfe0e |
fix: workspace preprocessor improvements (#5784)
* add tracing to get of authed client * internal: Trigger claude when commenting with /aider (#5783) * add claude instructions files * call claude too when using aider * fix * add draft for linear claude integration * fix: workspace preprocessor fixes * tmp ee ref * fix build * update ee ref * fix: hub script preprocessor handling * fix build * good ref --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: centdix <40307056+centdix@users.noreply.github.com> |
||
|
Ruben Fiszel
|
b15964b8a3 | fix | ||
|
dieriba
|
9e9514b9af |
fix: http trigger signature validation (#5753)
* fix * fix import * nits: force raw string on signature authentication * fix |
||
|
Ruben Fiszel
|
9579e14c8a | small fix | ||
|
dieriba
|
6d84f8c17b |
fix gcp on ee private ref (#5735)
* fix * update repo ref |
||
|
dieriba
|
54b4815df2 |
nits: use generic deserializer for empty data (#5727)
* use generic deserializer for empty data * update repo ref * fix import |
||
|
HugoCasa
|
254c3cf8ef |
feat: preprocessor refactor (#5629)
* feat: preprocessor refactor * nit * oneOf kind support + new capture format + fix build * fix build * fix build * update preprocessor templates * cache runnable version info * update ee ref * update hub scripts && ee ref |
||
|
494d662b2d |
fix cli gcp types (#5644)
* update types * fix types cli * fix ci * fix gcp types and fix used triggers * update repo ref revert .env and nits * fix build * Update frontend/src/lib/utils.ts Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> * fix type and deploy to * update ee-repo-ref * update .sqlx * add missing fields * fix: add sqlx derive macro * hanfle missing field, fix deploy to other workspace and update ee-repo-ref * update repo ref --------- Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
|
5b123b01a1 |
feat: handle different aws auth resource type (#5637)
* feat: oidc auth with sqs * rafctor * update: add id token struct * update has_expired function * fix duration * fix typo * Update backend/windmill-common/src/auth.rs Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> * Update backend/windmill-common/src/auth.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * update ref * update sqlx * update resource type * update repo ref * Update settings.json * add new resource type * update repo ref and clear resource path on resource type change * update .sqlx * add missing key --------- Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> |
||
|
|
6339775404 |
feat: add gcp trigger (#5501)
* update: add migration for gcp_trigger table, add cli method gcp and add gcp_trigger args for github ci * update: add gcp module and routing it in lib.rs * update: added gcp type to TRIGGER KIND type in db, add new feature condition to try_get_fn and added gcp_trigger feature in cargo file * update: add ui for gcp trigger, added missing triggers type for job kin, add gcp trigger in deploy to functionallity * adding google cloud crate * update: handle pull and push delivery * update: handle pull and push delivery * update: handle push event, changed front ui * update: ui * fix: capture for gcp * update: automatically manage pub sub subscription and refactoring * capture done * update cli types * fix: wrong func argument and type openapi * fix: missing import * update .sqlx * update: svg color and remove pulse button for capture push gcp * update: handle deployto * update script helper and link to hub gcp script/flow * update gcp representation * update: add confirmation modal * add unique index to mitigate same subscriber id, fix `zombie worker`, update test connection logic * Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update cli/gen/services.gen.ts Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * nits * update repo ref * fix feature function * update dependencies * update .sqlx * fix missing import * update: handle existing subscription and creating new one or update it * fix ee build * handle existing config properly * update .sqlx * update .sqlx * remove unused * update documentation link fix ci * use on instance of empty_string_as_none fn * update refo ref * updat script_helper * update ui * update migration * add missing arguments * fix and nits * update repo ref * remove unused * nits * add doc links * nits * Update frontend/src/lib/components/triggers/gcp/GcpTriggerPanel.svelte Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * use trigger path as default route path * update .sqlx * update repo ref * update ref * update route path * update types * fix deploy to * nits * update repo ref * update .sqlx * Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorConfigSection.svelte Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * nits * nits * nits * remove dupliacte fn and update repo ref * remove unused import * fix missing import * fix: wong name var * update script helper and template script * Update frontend/src/lib/script_helpers.ts Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update ee-repo-ref.txt --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: HugoCasa <hugo@casademont.ch> |
||
|
|
e920101107 |
feat: api key/basic/hmac auth for http triggers (#5476)
* feat: add webhook configuration in front and start handling logic in back * feat: add generic implementation * update: add stripe provider working on auth * merge main * handling new webhook, add generic implem * feat: add auth method and api key and handling custom authentication * nits * added generic implem for webhook provider * feat: added resource webhook_auth to handle http route auth, refactored backend code * update sqlx * fix: zoom and twitch challenge request event, call verify function * removing webhook name to make it more general, properly added custom auth option alongside windmill_auth option * update .sqlx * update deply to for http routes * update .sqlx * add different auth method for http routes * add custom script method and custom signature fixed bug with webhook auth * update auth options * update .sqlx * add new auth method * update .sqlx * refacto function and fix frontend bug * fix typo, update .sqlx and handle pototential error * nits * nits * add script template flow * Update frontend/src/lib/components/triggers/http/utils.ts Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * nits * update: remove wrong trigger type and use right icon for sqs trigger * nits * update: .sqlx * update: reunite flow and script logic for script placeholder value replacement * Update backend/windmill-api/src/http_trigger_auth.rs Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> * delete vite config * remove darks mode observer * update: add new option to capture panel for http route * nits * revert * nits * typo --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> Co-authored-by: HugoCasa <hugo@casademont.ch> |
||
|
Ruben Fiszel
|
880db319e8 |
feat: filter by worker + backend perf opt (#5489)
* all * all * all * refactor * sqlx * ref * impr * fix * fix * fix * ee private |
||
|
HugoCasa
|
4f43b1984f | feat: keep captures across drafts and deploys (#5482) | ||
|
5c39037aea |
feat: MQTT triggers (#5277)
* first commit * update migration mqtt * chore: update dependencies * add cli support and update schema for mqtt * update: casing style * update: rename get_resource function and moving to resource.rs * feat: * feat: refactored * feat: add ready endpoints for workers to enterprise * feat: handle connection to server * fix: main.rs * feat: added support v5 * feat: 🚧 updated .github, handle more case for v5 and handle v3 * feat: handled client, update script helper * update .sqlx * refactor: added generic implement and trait to remove redundant code, misses client impleme * feat: remove optinal property update test function * feat: done * handle client certificate * minor fix * fix: tls and mtls * fix: min topic len in frontend * Update frontend/src/lib/components/triggers/mqtt/MqttEditorConfigSection.svelte Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-api/openapi.yaml Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-api/src/mqtt_triggers.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * update openapi, and script_helper.ts * fix: toggle button and update mqtt migration * nits: remove postgres type and only relies on serde * fix: timeout connection * fix typo and captures * fix: capture table reactivity * handle more mqtt v5 option * nits: remove clone method * update: better naming * nits: display tooltip qos * fix: hanle tls connection with natively * update: add documentation links for qos and topics * fix tooltip and add default value * update: shoz preprocessor only if selected in capture table * nits: show mqtt icon in resource related componenets * update hub link and repo ref * fix: unused var * update ee ref * fix: give good reference id for script hub --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Clement Zhang <clement.zhang2@gmail.com> |
||
|
HugoCasa
|
7bf9e25ede |
feat: track workspace runnables used in flows (#5369)
* feat: track workspace runnables used in flows * track script hash * weird * do it with lock * Revert "feat: add support for | None and Optional in python (#5361)" This reverts commit |
||
|
|
58a67a3ac0 |
feat: SQS triggers (#5182)
* feat: first commit * fix: npm check * fix: openapi file * feat: update openapi and migration * feat: basic implementation done * fix: fix: no used function when no feature * feat: capture done * Update capture.rs * nits: change sqs trigger * fix: make migration great again * feat: add message attributes * feat: nits: fix error messages, remove console.log and add try catch * update sqs icon and ee feature for sqs_trigger * update: change sqs name casing and added test connection button * nits: update Icon and add create from template button * fix: ci build and error compilation * update migration type sqs * update link on create from template button for sqs, add archive in workspace export and update sqlx * fix: ci * Update SqsTriggerEditorInner.svelte * add link to docs, use generic function for resource and fix import error * chore: update .github ci * nits: remove empty * update to match ee repo changement * Update backend/windmill-api/src/resources.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * add sqs handling for the cli and refacoring sqsEditorInner * Update cli/sync.ts Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix: add break to switch statement for sqs case * fix: display aws_resource_path when retrieve or create a new trigger * rework sqs ui, fix postgres optional port * fix: ci * update ui for trigger * update repo ref and specific * feat: add ready endpoints for workers to enterprise * update ref * Update frontend/src/lib/script_helpers.ts Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> --------- Co-authored-by: HugoCasa <hugo@casademont.ch> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
57cfa4045b |
feat: add postgres trigger captures (#5165)
* feat: 🚧 add postgres trigger config in back * feat: add capture section for postgres * feat: update postgres trigger panel, update capture migration * fix: ci and casing style * feat: capture done * fix: update sqlx * fix: remove unused * feat: handle persisting data * fix: add feature flag * feat: capture done * fix: feature flag * feat: capture done * nits: add plus into postgres capture label * feat: trigger done * Update PostgresTriggerEditorInner.svelte * Update script_helpers.ts * nit * chore: update to meet pr comment requirement * refactor: * fix: * fix: miss closing curly braces * fix: * fix: ci * Update mod.rs * polishing postgres trigger * update minor changes on UI * nits: remove log and update function parameters * fix: remove wrong name * nits: change icon * fix: nit migration * nits: refacto and remove alert * nits: change variable name * update workspaces_export * fix: ci --------- Co-authored-by: HugoCasa <hugo@casademont.ch> Co-authored-by: Guilhem <guilhemlemouel@gmail.com> |
||
|
Ruben Fiszel
|
90ba65ae20 | improve error messages for internal err | ||
|
HugoCasa
|
5d8260e90e | add single capture endpoint (#5170) | ||
|
HugoCasaandRuben Fiszel
|
d17397b070 |
feat: test trigger connection (#5145)
* feat: test trigger connection * fix build * fix build * update ee ref * nit --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
HugoCasa
|
b7eeed0729 |
triggers improvements (#5074)
* triggers improvements * nits * nits * update ee ref |
||
|
ebfde197fc |
fix(ui): capture v2 UX (#4954)
* feat: captures * flow UI and improvements * fix: build * fix sqlx * Move Capture WIP * Add capture to webhook and websocket * Move connection status viewer in the head * change trigger section label * Add popover capture picker using melt ui * Add shortcut to triggers capture from input form * remove capture tab in input * Allways show capture * remove useless logs * Add email capture * Add kafka capture into triggers * Add edit option in capture table * use light header for arg input * Add prefilled group id * Change button label * fix logic * Open resource drawer if prototype has fields * fix default completion * Change name Prototype * Dissociate Editor Mode for triggers * Fix bug for script * Fix apply args * fix apply schema * Add capture table to script * fix apply args * Add capture button for script * Delete capture tab * Set capture on when opening triggers capture * fix connection indicator * fix minor issues * Add preprocessor logic * Use slot in log Panel for captures * handle capture refresh in script * Delete capture tab from script editor * reset kafka resource on toggle static * fix minor issue * Allow resource in kafka capture * use simple capture button in flow * Remove capture panel * Polish route trigger editor * Fix resource saving * Remove excessive padding * merge nits * Add history tab # Conflicts: # frontend/src/lib/components/EditableSchemaForm.svelte * add workflow_dispatch to build * add workflow_dispatch to build * Move input copy from in a panel * better capture UI * Add smoothing animation on open input editor * fix bad wrapping in input editor menu * clean code * restore captureTable * clean * fix sqlx * fix build * fix build * build * make initial_messages optional in line with db * fix npm check * better handle args for capture webhook and http * add capture pagination * Add capture table * Add table for history inputs * nit * simplify capture table * stop capturing on closing capture section * show only 10 items per page for captures * set focus to logs on script test * Fix collapsable section * nit * open captures when applying args from triggers * update edit input drawer * fix all EditableSchemaForm * fix panel init animation * Add capture popover * change open tab button * clean code * fix bad table display * Fix JSON input bad sync * make capture icon bigger * nit * nit * revert unwanted change * Change capture name and nits * infinite scroll and cleaning * Add toast when failing to retreive first step input * add icons to dropdown edit input list * remove unused log * Add click outside to all schema pickers * fix click oustide * Fix migrations * solve run button not refreshing properly * Add a capture drawer * change apply args and schema logic * change apply args and schema logic * fix toggle display * fix schema display * improve preview mode * Add first step panel * nit * clean * Add hint for captures * clean * nit * nit * fix script node page height * improve animation on redirecting to capture * improve saved input picker * nit * add Json placeholder * nit * add list animations * delete useless logs * add animated list to captures history and saved inputs * Delete unused component * clean * nit * clean code * improve tab scroll experience * nit * nits * nits * fix script args update * nit * nit: include preview in history inputs in edit mode --------- Co-authored-by: HugoCasa <hugo@casademont.ch> Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
HugoCasa
|
e66fcf9270 |
feat: nats triggers (#5039)
* feat: nats triggers * missing files * fix build * update feature flags * event stream fixes * update ee ref |
||
|
|
01c6f84520 |
feat: captures (#4807)
* feat: captures * flow UI and improvements * fix: build * fix sqlx * Move Capture WIP * Add capture to webhook and websocket * Move connection status viewer in the head * change trigger section label * Add popover capture picker using melt ui * Add shortcut to triggers capture from input form * remove capture tab in input * Allways show capture * remove useless logs * Add email capture * Add kafka capture into triggers * Add edit option in capture table * use light header for arg input * Add prefilled group id * Change button label * fix logic * Open resource drawer if prototype has fields * fix default completion * Change name Prototype * Dissociate Editor Mode for triggers * Fix bug for script * Fix apply args * fix apply schema * Add capture table to script * fix apply args * Add capture button for script * Delete capture tab * Set capture on when opening triggers capture * fix connection indicator * fix minor issues * Add preprocessor logic * Use slot in log Panel for captures * handle capture refresh in script * Delete capture tab from script editor * reset kafka resource on toggle static * fix minor issue * Allow resource in kafka capture * use simple capture button in flow * Remove capture panel * Polish route trigger editor * Fix resource saving * Remove excessive padding * merge nits * add workflow_dispatch to build * add workflow_dispatch to build * better capture UI * fix sqlx * fix build * fix build * build * make initial_messages optional in line with db * fix npm check * better handle args for capture webhook and http * improve migration + http capture fixes * fix sqlx * update ee ref --------- Co-authored-by: Guilhem <guilhemlemouel@gmail.com> Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
HugoCasa
|
563a492008 |
feat: accept direct file upload for webhook/http (s3) (#4903)
* feat: accept direct file upload for webhook/http (s3) * nit * nit |
||
|
Ruben Fiszel
|
2812792326 | fix: memory optimization for flow with big args (#4019) | ||
|
Ruben Fiszel
|
c43d04b95d |
feat: reduce memory usage (#3768)
* all * all * optimize deserialize * all * all * fix issue * fix compile * fix compile * fix compile * fix compile * fix compile * fix compile * fix compile * fix compile * fix compile * fix compile |
||
 Ruben Fiszelandgbouv
|
2ee57f3f84 |
feat: refactor entirely json processing in favor or rawjson to handle larger payloads (#2446)
* raw json * raw json * raw json * raw json * the big rawjson rewrite * resolve merge conflicts * all * progress * progress * all compile * remove mut * remove mut * remove mut * arc queuedjob * finish * finish * finish * fixes * progress * fix result * small fixes * small fixes * fix tests * fix tests * fix tests * optimize eval timeout * optimize eval timeout * optimize eval timeout * handle big args and result in get jobs * improve args * fix * fix * fix tests * fix tests * done * add bigquery to native picker * add bigquery --------- Co-authored-by: gbouv <guillaume@windmill.dev> |
||
|
Ruben Fiszel
|
7cd563596d | align capture endpoint with run endpoint | ||
|
Ruben Fiszel
|
c77d38563d | remove unecessary postgres transactions and improve performance | ||
|
223b80768a |
chore(backend): upgrade sqlx to ^0.7 (#1865)
* DRAFT chore(backend): upgrade sqlx to ^0.7 related to: * https://github.com/windmill-labs/windmill/pull/1858 * https://github.com/launchbadge/sqlx/issues/1163#issuecomment-1627685514 * (vol. 2) in 0.7, `Transaction` can no longer implement `Executor` directly ref: https://github.com/launchbadge/sqlx/blob/afb6b1066e61f8e3875f530d96cfb5a299f13fda/examples/postgres/transaction/src/main.rs#L14-L17 notice that I'm temporarly using my custom patch https://github.com/mrl5/sqlx/commit/16e4c9a8f380214ad61b1744d3e1f5fa866ffbcc it's related to https://github.com/launchbadge/sqlx/issues/2611 * post git rebase chores * use upstream fix from 0.7.1 * fix compile --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com> |
||
|
Ruben Fiszel
|
68e8435409 | fix(backend): improve result retrieval |