Commit Graph
8 Commits
Author SHA1 Message Date
Diego ImbertandClaude Opus 5 6a3a2ed13c fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved
Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 15:14:29 +02:00
Diego ImbertandClaude Opus 5 791bb8d843 refactor(datatables): keep Postgres triggers and data table roles apart
A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 15:14:29 +02:00
Diego ImbertandClaude Opus 5 22b3e69c07 feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-16 15:14:28 +02:00
Ruben Fiszel 5e52346242 fix: surface postgres publication errors as 400 instead of 500 (#10376) 2026-07-28 12:25:08 +02:00
Diego ImbertandClaude Fable 5 68daed8501 refactor: custom-instance datatable connection handling (#10271)
Attach custom-instance datatables in the DuckDB executor through a DuckDB
secret instead of an inline connection string, and route postgres triggers on
custom-instance datatables through a dedicated custom_instance_replication_user
role (with its own auto-generated password in global_settings). Normalize
custom_instance_user attributes on server boot.


Claude-Session: https://claude.ai/code/session_01Tp6NNNinCB8dwWqGaFXDRF

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 18:57:31 +02:00
Diego Imbertandclaude[bot] 8e7ba9b33d feat: Data table as pg resource / trigger (#8088)
* Enable running pg scripts with datatable database input

* Postgres triggers for data tables

* REPLICATION attribute on custom_instance_user

* disable edit for datatables

* Update backend/windmill-trigger-postgres/src/replication_message.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-02-25 15:06:12 +00:00
Ruben FiszelandClaude Opus 4.6 2eafe6df36 test: add comprehensive test coverage for extracted backend crates
Add 296 tests across unit and integration test suites to cover
the newly extracted crates from the recent refactor commits.

Unit tests (270):
- windmill-trigger-postgres (96): hex codec, bool parsing, type
  conversion, relation tracking, replication message parsing,
  publication data validation
- windmill-trigger-http (92): HMAC signature verification for
  GitHub/Slack/Stripe/TikTok/Twitch/Zoom webhooks, API key auth,
  Basic Auth, route validation, HTTP method/request type serde
- windmill-api-jobs (39): SQL query builder for job listing/counting
  with filters, pagination, label handling
- windmill-trigger (31): TriggerMode serde, query pagination,
  BaseTriggerData backward compat, HandlerAction, ServerState
- windmill-common webhook (7): WebhookMessage serialization tags
- worker nativets/postgresql (5): nativets job execution with
  args/objects/datetime, postgresql query execution

Integration tests (26):
- backend/tests/triggers.rs: capture config CRUD, capture payload
  operations, capture API endpoints, HTTP trigger CRUD with mode
  filtering, all trigger types DB schema validation (websocket,
  kafka, postgres, nats, sqs), schedule operations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 08:58:24 +00:00
Ruben FiszelandClaude Opus 4.6 9ff8a85af6 refactor: extract windmill-api into subcrates for parallel compilation (#7845)
* refactor: extract windmill-api into 4 subcrates (api-auth, store, api-sse, api-jobs)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: eliminate refresh_token OnceLock bridge in windmill-store

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: eliminate FromRequestParts OnceLock bridge in windmill-api-auth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: wire subcrates into workspace and clean up unused re-exports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve cargo check --all-features errors in subcrate wiring

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* all

* chore: update ee-repo-ref for warning fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract windmill-trigger crate and expand windmill-api-jobs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-kafka crate from windmill-api

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-postgres crate from windmill-api

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-websocket and windmill-trigger-mqtt crates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-nats, sqs, gcp, and email crates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-trigger-http crate from windmill-api

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: move token creation and permission helpers to windmill-api-auth

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract windmill-native-triggers crate from windmill-api

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* all

* refactor: extract windmill-api-embeddings crate and fix CI warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve type mismatch in oauth2_oss and remaining warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use correct HTTP_CLIENT config in embeddings crate (30s timeout, cert override)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* fix: gate oauth_refresh_ee on oauth2 feature to fix warnings

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:12:55 +00:00