fix(datatables): a tenant and a resource path follow the principal they name

A data table role names its tenants — `u/alice`, `g/devs`, `f/team` — and a
data table names its database by resource path. Both are strings that outlive
what they point at, and the flows that free those names left them behind:

- Deleting a workspace user, a group or a folder left its tenant on every role
  it could run as. The name is free afterwards, so whoever takes it next
  inherits the role without an admin granting it. All three now drop it in the
  same transaction that removes the principal.
- Offboarding reassigns a user's resources to a folder without the data table
  following, which both stops it resolving and frees the path it named — a new
  resource there points it at another database with its roles intact. The
  username rename had the same gap. Both move the config with the resource.

The walk both need is one function each now, with the table test that pins
which tenant forms it touches.

Saving permissions also takes a lock per data table, held from the read that
plans to the write that persists: two saves interleaved each planned against
what the other was leaving, and the one that persisted last stored roles the
other had already dropped. The ACL apply takes the same lock, since it reads
the same config and catalog.

backend/windmill-api-workspaces/src/datatable_acl.rs

@@ -1041,6 +1041,13 @@ async fn apply_datatable_acl(
     Path((w_id, datatable_name)): Path<(String, String)>,
     Json(req): Json<AclChangeRequest>,
 ) -> Result<String> {
+    // Taken for the same reason the role save takes it: the plan below is read
+    // off the catalog and the config, and a role save running at the same time
+    // is what changes both under it. Held to the end of this handler.
+    let mut lock_tx = db.begin().await?;
+    crate::datatable_permissions::lock_datatable_permissions(&mut lock_tx, &w_id, &datatable_name)
+        .await?;
+
     // Authorization first: the repair below opens a connection as the instance's
     // own Postgres user, which is not something a request that is about to be
     // refused gets to reach.
@@ -1076,7 +1083,7 @@ async fn apply_datatable_acl(
     })?;
 
     audit_log(
-        &db,
+        &mut *lock_tx,
         &authed,
         "datatables.acl",
         ActionKind::Update,
@@ -1085,6 +1092,7 @@ async fn apply_datatable_acl(
         Some([("target", format!("{:?}", req.target).as_str())].into()),
     )
     .await?;
+    lock_tx.commit().await?;
 
     Ok(format!("Updated access on {}", req.target.label(&dbname)))
 }

backend/windmill-api-workspaces/src/datatable_permissions.rs

@@ -137,6 +137,27 @@ pub(crate) struct RolePlan {
     pub(crate) warnings: Vec<String>,
 }
 
+/// Serialize everything that changes one data table's roles or their access.
+///
+/// Both the role save and an ACL apply read the config, plan against it and run
+/// the result on the data table's own database; two of them at once plan against
+/// a state the other is leaving. Keyed per data table, and released when the
+/// caller's transaction ends.
+pub(crate) async fn lock_datatable_permissions(
+    tx: &mut sqlx::Transaction<'_, sqlx::Postgres>,
+    w_id: &str,
+    datatable_name: &str,
+) -> Result<()> {
+    sqlx::query!(
+        "SELECT pg_advisory_xact_lock(hashtext('datatable_permissions:' || $1), hashtext($2))",
+        w_id,
+        datatable_name,
+    )
+    .execute(&mut **tx)
+    .await?;
+    Ok(())
+}
+
 /// Refuse to plan a change on an enterprise binary whose plan does not cover it.
 /// A build that is not enterprise has no planner at all — see
 /// [`crate::datatable_permissions_oss`] — so this only has the licensed
@@ -571,6 +592,14 @@ async fn set_datatable_permissions(
 ) -> Result<String> {
     require_admin(authed.is_admin, &authed.username)?;
 
+    // Reading the config, planning against it, running the plan and persisting
+    // it are one operation: two saves of the same data table interleaved would
+    // each plan against the state the other is leaving, and the one that
+    // persists last would store roles the other already dropped. Held to commit,
+    // so the whole sequence below is inside it.
+    let mut tx = db.begin().await?;
+    lock_datatable_permissions(&mut tx, &w_id, &datatable_name).await?;
+
     // The roles about to be created are handed privileges by this connection,
     // which cannot pass on what it holds without the grant option.
     ensure_instance_db_can_delegate(&db, &w_id, &datatable_name).await;
@@ -588,7 +617,6 @@ async fn set_datatable_permissions(
     let permissions = serde_json::to_value(&plan.permissions)
         .map_err(|e| Error::internal_err(format!("Failed to serialize permissions: {e}")))?;
 
-    let mut tx = db.begin().await?;
     // Written at the permissions path only, so a concurrent edit of the data
     // table's own settings is not clobbered.
     let updated = sqlx::query_scalar!(