Files
windmill/backend/windmill-duckdb-ffi-internal/README_DEV.md
T
f9513e82ee feat: run duckdb jobs out of process under job isolation, drop the engine fork, bump to 1.5.6 (#11780)
* feat: prototype running duckdb jobs out of process inside nsjail

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: bound runner output, honor force install outside the jail, set oom score before exec

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: gate the duckdb runner bin behind a feature and decode pre-install errors

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: allowlist-only policy inside the duckdb jail, with its own extension cache

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: run duckdb jobs out of process under job isolation and drop the engine fork

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: runner everywhere, stock duckdb engine, no sql fence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: update ee-repo-ref to 24bdc6615b3f651141fb740e2cacf9c3c2d81252

This commit updates the EE repository reference after PR #884 was merged in windmill-ee-private.

Previous ee-repo-ref: 73452bbc280cc9ca33cf0fe325d26c2fbf3aace4

New ee-repo-ref: 24bdc6615b3f651141fb740e2cacf9c3c2d81252

Automated by sync-ee-ref workflow.

* fix: pass the windows system env and TZ to the duckdb runner

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: cap a jailed duckdb job's disk use from the worker, 6GiB by default on cloud

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: keep a disk-backed jail tmp under the duckdb disk cap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop the benchmark scaffolding and bump duckdb to 1.5.6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-10-15 16:49:50 +02:00

832 B

This crate is compiled separately from the backend workspace because the bundled DuckDB's libc++ clashes with deno_core's when both live in one process.

The symptom was opaque errors. DuckDB errors worked, except when attached to a DuckLake with the deno_core feature on:

ATTACH 'ducklake' AS dl; USE dl;
CREATE TABLE IF NOT EXISTS t (x string not null);
INSERT INTO t VALUES (NULL);

gave Unknown exception in ExecutorTask::Execute instead of Constraint Error: NOT NULL constraint failed: t.x. The opaque error comes from the C++ DuckDB library: https://github.com/duckdb/duckdb/blob/f99fed1e0b16a842573f9dad529f6c170a004f6e/src/parallel/executor_task.cpp#L58

The worker never links this crate: it runs windmill_duckdb_runner (src/bin) as a separate process for every job. See docs/duckdb-execution.md.