docs: qualify the any-language claim, and pin the dbt-script refusal

`AssetKind::Dbt`'s contract (both enums), the two runtime guides and the deploy
comment said a script of any language may declare a `dbt://` write, which the
dbt-script refusal added last round contradicts. They now say "any language but
dbt's own", with the reason: a project's writes are read from its manifest.

The deploy-contract integration test covers that refusal for both annotations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ruben Fiszel
2026-09-04 18:20:08 +02:00
co-authored by Claude Opus 5
parent addcb4c525
commit 6564f5a814
7 changed files with 47 additions and 16 deletions
@@ -31,8 +31,8 @@ pub enum AssetKind {
Volume,
/// A warehouse relation, `dbt://<warehouse>/<schema>/<name>`, the warehouse
/// named as the workspace configures it. The scheme names the namespace dbt
/// made — a script of any language can declare a write to one — and the path
/// stays the relation. See `windmill_types::AssetKind::Dbt`.
/// made — a script in any language but dbt's own can declare a write to one —
/// and the path stays the relation. See `windmill_types::AssetKind::Dbt`.
Dbt,
}
@@ -131,5 +131,31 @@ async fn test_dbt_materialize_target_deploy_contract(db: Pool<Postgres>) -> anyh
assert_eq!(resp.status(), 400);
assert!(resp.text().await?.contains("u/test-user/project"));
// Neither annotation is accepted on a dbt script: the graph ingest
// republishes that path's asset and trigger rows wholesale, so either would
// deploy something the dependency job then silently removes.
for content in [
"# materialize manual dbt://main/analytics/orders\nprofile:\n warehouse: main\n",
"# on dbt://main/analytics/orders\nprofile:\n warehouse: main\n",
] {
let resp = authed(client().post(format!(
"http://localhost:{port}/api/w/test-workspace/scripts/create"
)))
.json(&json!({
"path": "u/test-user/dbt_project",
"summary": "",
"description": "",
"content": content,
"language": "dbt",
"modules": { "dbt_project.yml": { "content": "name: p\n", "language": "dbt" } },
"schema": { "type": "object", "properties": {}, "required": [] }
}))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 400);
assert!(resp.text().await?.contains("a dbt script cannot"));
}
Ok(())
}
+4 -3
View File
@@ -1573,9 +1573,10 @@ async fn create_script_internal<'c>(
// A non-DuckDB script never reaches that executor.
// • `dbt://<warehouse>/<schema>/<name>` — a warehouse relation. Nothing
// generates warehouse DDL, so the declaration is track-only (`manual`)
// and any language may make it: the script writes the relation, the
// worker records the materialization, and the relation's asset node is
// shared with whatever dbt model reads it.
// and any language but dbt's own may make it: the script writes the
// relation, the worker records the materialization, and the relation's
// asset node is shared with whatever dbt model reads it. A dbt project's
// own writes are read from its manifest, so it may not declare one.
// Any other kind would deploy, register a producer in the asset graph, then
// silently no-op at run time (`build_materialized_query` returns `Ok(None)`).
// The managed-only checks (single trailing SELECT, no SQL args) come after —
+4 -3
View File
@@ -18,9 +18,10 @@ pub enum AssetKind {
/// `<warehouse>` is the name the workspace configures it under.
///
/// The SCHEME names the namespace dbt made rather than an exclusive
/// producer: dbt is what derives these relations from a project, and a
/// script of any language can DECLARE one it writes
/// (`// materialize manual dbt://…`). The PATH stays the physical relation,
/// producer: dbt is what derives these relations from a project, and a script
/// in any language but dbt's own can DECLARE one it writes
/// (`// materialize manual dbt://…`) — a project's writes are read from its
/// manifest, never annotated. The PATH stays the physical relation,
/// because that is what two producers agree on: a mart one builds is a
/// `source` the next reads, and their dbt `unique_id`s differ
/// (`model.a.orders` vs `source.b.analytics.orders`) where the relation does
+2 -1
View File
@@ -5564,7 +5564,8 @@ async fn handle_code_execution_job(
/// Nothing generates warehouse DDL, so the script issues its own write and this
/// is the only thing that turns it into a `materialized_partition` row — the
/// relation's last writer on the run page and the graph. Language-agnostic on
/// purpose: the DuckLake write engine is DuckDB's, this declaration is anyone's.
/// purpose the DuckLake write engine is DuckDB's, this declaration is anyone's
/// — except dbt's own, which is refused at deploy.
///
/// Best-effort, and it can be: the cascade fans out from the deploy-time `asset`
/// rows, not from this one, so a lost row costs the relation its last writer and
+7 -5
View File
@@ -47,7 +47,7 @@ the dominant way dbt is orchestrated today.
| 22 | Naming | Match Cosmos field names; importer deferred |
| 23 | Descriptor | `wm_dbt.yaml` inside the project, OPTIONAL. See below |
| 24 | Warehouse | Configured on the workspace by name, `main` by default. See below |
| 25 | Cascade direction | Into a relation, not out of a run: `// materialize manual dbt://…` declares a write from any language and wakes `# on dbt://…` subscribers; a finished dbt run still does not dispatch. See "No cascade *from* dbt" |
| 25 | Cascade direction | Into a relation, not out of a run: `// materialize manual dbt://…` declares a write from any language but dbt's own and wakes `# on dbt://…` subscribers; a finished dbt run still does not dispatch. See "No cascade *from* dbt" |
## Decision 1: engine toggle, and why the shipped default is not Fusion yet
@@ -163,9 +163,10 @@ put warehouse relations in the asset graph and is what derives them from a
project; no other language *infers* one, and calling the kind something generic
promised a parity with native Snowflake and BigQuery scripts that does not exist.
A script can nonetheless DECLARE that it writes one — `// materialize manual
dbt://<warehouse>/<schema>/<name>`, in any language — and that declaration lands
on the same node the dbt model reading the relation does, because identity is the
relation rather than the tool. See "No cascade *from* dbt" below.
dbt://<warehouse>/<schema>/<name>`, in any language but dbt's own, whose writes
come from its manifest — and that declaration lands on the same node the dbt model
reading the relation does, because identity is the relation rather than the tool.
See "No cascade *from* dbt" below.
The PATH is the physical relation, and that is the load-bearing half. dbt-core
has no cross-project `ref()`: two projects meet when one materializes a mart and
@@ -690,7 +691,8 @@ mode: nothing generates warehouse DDL, so the script issues its own write and
Windmill records the outcome — the same `materialized_partition` row a DuckLake
target lands, so the relation carries a last writer on the run page and the graph.
It is language-agnostic (the DuckLake write ENGINE is DuckDB's; this declaration
is anyone's), and the recording happens in the generic job path
is anyone's but a dbt project's, whose writes are read from its manifest), and the
recording happens in the generic job path
(`record_declared_warehouse_write`) rather than in an executor, for the same
reason. Identity is unchanged — the physical relation — so the ingestion script
and the dbt model reading it are one node, and a `source` declared on the relation
+2 -2
View File
@@ -608,8 +608,8 @@ examples) lives in windmilldocs `core_concepts/63_pipelines` → "Ingestion
- **A `ducklake://` `// materialize` is DuckDB-only** (deploy-rejected
elsewhere, managed and `manual` alike — `windmill-api-scripts/src/scripts.rs`;
a `dbt://` warehouse-relation target is the one any language may declare, and
it is track-only — see `docs/dbt-runtime.md`), and the SDK
a `dbt://` warehouse-relation target is the one any language but dbt's own may
declare, and it is track-only — see `docs/dbt-runtime.md`), and the SDK
materialize helpers (`upsert_partition` / `upsertPartition`) build their SQL
inside the SDK, so the asset parsers cannot see the write. A polyglot node
that "writes the lake directly" therefore deploys with **no output edge**