starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-20 08:01:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe (#8999)

Browse Source 
* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pg): wrap encoder errors with arg context, add fallback test

Followups on #8999 review:

- Wrap rust-postgres "error serializing parameter N" failures with the arg
  name, JSON value kind, and asserted Postgres type plus a hint about an
  explicit cast — so users see actionable context instead of an opaque
  WrongType.
- Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree
  on the Type for every recognised arg_t when the JSON value matches its
  natural Rust kind. Catches future drift if either side changes.
- Integration test for the prepare + query_raw fallback path: confirms
  unrecognised arg_t (custom enum) is routed through prepare and the
  server-resolved type appears in the failure surface — flips into a
  test failure if a regression accidentally routes unrecognised types
  through query_typed_raw.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): add otyp_inferred flag + regex-based placeholder renumbering

Two follow-ups from the review of #8999:

1. **Issue #1 (Number/Bool + explicit text decl in WHERE)**

   Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets
   it `true` only at the "no info → fall back to text" site (bare `$N`,
   no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep
   it `false`.

   In `convert_val` this flag distinguishes:
   - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce
     `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works
     (`text = text` operator). Pre-#8988 behaviour, restored.
   - parser-default text (bare `$N`) — bind the value's natural Rust
     type so the regression case (`Value::Bool` against a real `bool`
     column via `CAST AS bool`) keeps working.

   `Arg` is in `windmill-parser`; the new field has `#[serde(default)]`
   so persisted signatures stay backward-compatible.

2. **Issue #4 ($5/$50 substring rewrite collision)**

   Replace the per-index `String::replace` chain (which turned `$50`
   into `$10` when oidx=5 was processed first) with a single regex
   pass. `\d+` is greedy, so `$5` and `$50` match as distinct units;
   indices outside the mapping are left intact.

3. Tests:
   - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline-
     cast/decl/mixed shapes.
   - executor unit: `convert_val_bool_against_every_arg_t` and
     `convert_val_*_number_*` split each text-like target into explicit
     vs inferred expectations.
   - executor unit: `renumber_sparse_placeholders_no_collision`.
   - integration: `test_postgresql_arg_type_combinations` adds 4 cases
     covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and
     sparse positional args ($5/$50).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality

Backend:

1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s
   `ToSql for String` / `FromSql for String` reject `Kind::Enum` and
   `Kind::Domain` even though the wire format is plain UTF-8. The wrapper
   accepts those kinds in both directions. End result: explicit
   `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the
   ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col`
   results come back as JSON strings instead of erroring at the FromSql
   layer.

2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these
   arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text /
   non-temporal arg_t fell through to `Box<String> + TEXT`, which then
   failed at the server (no implicit cast text→numeric in expression
   context). Now strings are parsed into the matching native type with
   clear error messages on parse failure.

3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering
   (which fixed the `$5/$50` substring collision but still walked through
   string literals and comments, mangling `'price: $5'` etc.) with a
   walk over `parse_pg_statement_arg_positions` — the same
   string/comment/dollar-quote-aware tokenizer used for index discovery.
   Adds `parse_pg_statement_arg_positions` to the parser's public API.

SDK:

4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now
   stringifies bigints before serialisation; the executor accepts
   numeric strings into BIGINT arg slots via the existing
   `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split
   so `BigInt` always resolves to `BIGINT` (was reaching
   `Number.isInteger(BigInt)` which returns false → wrong default).

5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column
   now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers
   primitive types only (number / bigint / string / boolean); mixed or
   nested arrays still fall back to JSON. Mixed int/float widens to
   `DOUBLE PRECISION[]`.

6. **`.query()` positional bug**: previously the `.query()` method
   abused the template-tag builder, which appended `$N::TYPE` after the
   user's literal SQL string instead of binding by position
   (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()`
   builds the executor-shaped content directly: a `-- $N argN (TYPE)`
   declaration block followed by the user's SQL verbatim.

Tests:

- Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments`
  asserts string literals, comments, and dollar-quoted blocks don't
  produce positions (so renumbering doesn't mangle them).
- Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling`
  uses the new position-aware path and includes string-literal + comment
  + `$$…$$` cases. Existing convert_val tests grow to cover new
  String→numeric/real/double/oid/bool arms.
- Integration: `test_postgresql_arg_type_combinations` adds 13 cases
  (enum round-trip both directions, string→numeric/real/double/bool/oid,
  string-literal `$N` non-mangling). The prepare-fallback test now
  asserts SUCCESS (not failure) for enum encoding via AnyTextValue.
- SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests)
  exhaustively covering inferSqlType primitives + arrays,
  parseTypeAnnotation, datatable() template tag (with all the new
  shapes — BigInt, homogeneous arrays, RawSql, schema preamble),
  datatable().query() positional, and ducklake() shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache)

Found while exhaustively probing custom-type DX: every cached-connection
reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL`
deallocates *all* prepared statements server-side — including the typeinfo
statements that tokio_postgres caches per-Client to resolve custom enum /
domain Oids. tokio_postgres still held `Statement` objects whose names
the server had forgotten, so the next custom-type query failed with
intermittent "prepared statement \"sN\" does not exist" errors. The
failure was easy to reproduce: any sequence that forced typeinfo lookup
for two different custom-type kinds on the same cached connection (e.g.
enum followed by domain) would hit it.

Replace `DISCARD ALL` with a curated reset that explicitly targets the
state we actually care about, *without* touching prepared statements:

  RESET ALL                     — GUC parameters (search_path, application
                                  _name, statement_timeout, …)
  RESET SESSION AUTHORIZATION   — undoes both `SET SESSION AUTHORIZATION`
                                  and `SET ROLE` (RESET ALL does NOT —
                                  these aren't GUC parameters, so without
                                  this an elevated role from a previous
                                  job would silently leak)
  UNLISTEN *                    — drops LISTEN registrations
  CLOSE ALL                     — closes open cursors

Trade-off: temp tables, advisory locks (session-scoped), and user-created
PREPARE statements may persist across cached-connection reuse — rare in
datatable / PG-script workloads. tokio_postgres's typeinfo cache survives
intact, so custom enum / domain queries are fast on subsequent reuse.

Tests:
- `test_postgresql_custom_types_on_cached_connection` — runs 10×
  alternating enum + domain queries on a cached connection. Pre-fix this
  failed with `prepared statement "sN" does not exist` after the first
  reuse; post-fix passes.
- `test_postgresql_set_role_does_not_leak_across_cached_connection` —
  switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres
  role, then runs a follow-up job and asserts current_user/session_user
  are restored. Specifically catches the case where someone might switch
  back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION
  AUTHORIZATION) and silently introduce a permission-leak vector.
- All existing session-isolation tests
  (`test_postgresql_cached_connection_resets_session`,
   `test_postgresql_single_worker_session_isolation`,
   `test_postgresql_100_jobs_cached`) continue to pass.

Found via end-to-end probing of datatable / PG-script DX, not previously
covered: the existing isolation tests only did `SET ROLE postgres`, the
connecting user, so the leak was invisible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): address PR #8999 review (cubic + claude)

cubic (P1, real bug):
- `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but
  chrono `NaiveTime` only encodes for TIME (same caveat as the scalar
  arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz`
  assignment cast at the column site. Add an explicit unit test.

claude (#1, silent failure → explicit error):
- `Bool` + explicit `(char)` / `(character)` decl previously silently
  bound BOOL, hoping the server would cast at the use site — but PG has
  no implicit `bool→char` and the resulting error
  ("operator does not exist: bool = char") was opaque. Now error at
  bind time with an actionable hint to use `bool` decl or pass the
  value as a "t"/"f" string.

claude (#2, asymmetry doc):
- Object/Array still coerce to text on `matches!(typ, Typ::Str(_))`
  (covers both explicit AND inferred-default text), unlike Bool/Number
  which key on `explicit_text_target`. The asymmetry is intentional
  (no implicit `jsonb → text` cast in expression context vs PG having
  implicit `bool/int → text` casts) — added a body comment so future
  maintainers don't try to "align" them.

claude (#3, perf):
- `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions`
  walked the SQL tokenizer twice. Fold into a single pass that derives
  the index set from the position list.

claude (#4, fmt drift):
- `cargo fmt` over the parser crates I touched with perl scripts in the
  earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py,
  rust,graphql,yaml,r}). Net cosmetic.

claude (#5, parseTypeAnnotation):
- One-line caveat in the SDK's `parseTypeAnnotation` that the returned
  string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns
  `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a
  real PG type, but the only consumer just checks `!== undefined`).

While here — discovered + fixed independently while exhaustively probing
DX:

- **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET
  SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's
  `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing
  intermittent `prepared statement "sN" does not exist` errors on
  custom-type queries after cached-conn reuse. New regression tests:
  `test_postgresql_custom_types_on_cached_connection` and
  `test_postgresql_set_role_does_not_leak_across_cached_connection`
  (the latter catches the case where someone might switch back to
  `RESET ALL` alone and silently introduce a permission-leak vector —
  RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION).

- **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix
  `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00")
  and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") —
  neither parseable by `date-fns parseISO`, JavaScript `new Date()` is
  lenient enough to handle them but several frontend `App*Input.svelte`
  components use parseISO and fail silently. Switched to ISO-8601 with
  `T` separator and `+00:00` offset; arg-parsing path still accepts the
  legacy " UTC" suffix for back-compat.

Test coverage:
- 17/17 unit (`pg_executor::tests`)
- 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`)
- 27/27 parser (`windmill-parser-sql`)
- 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling

Found while probing PG-script DX with millions of numeric cells:

1. **Numeric precision-loss warning**: `numeric` results are still serialised
   as JSON Number (back-compat — switching to JSON String would silently
   break user code doing arithmetic on results), but we now detect
   `Decimal -> f64 -> Decimal` round-trip failure and emit a single
   job-log warning recommending a `::text` cast in the SQL. Bounded by
   `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic
   load + one fetch_sub on the hot path; first lossy value
   short-circuits to a single load thereafter). Worst-case overhead on
   a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic
   loads (vs. ~100ms unbounded).

2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned
   `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"`
   (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is
   what the apps `App*Input.svelte` components use, so timestamp values
   silently failed to round-trip into date pickers. Switch to ISO-8601
   (`T` separator + `+00:00` offset) on the result side; arg-parser
   continues to accept the legacy `" UTC"`-suffixed format for
   back-compat.

3. **Float NaN / Infinity results**: `Number::from_f64` returns None for
   NaN / ±Inf, which `pg_cell_to_json_value` was raising as
   "invalid json-float" — failing the *entire* query if any cell held
   one of these special values. Now serialise them as JSON strings
   ("NaN", "Infinity", "-Infinity") and let the rest of the row come
   through. Arg-side: `s.parse::<f64>()` already accepts the same
   strings.

Tests:
- `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit
  cases for the precision-loss predicate.
- `precision_check_budget_caps_per_query_overhead` — locks in the
  budget cap and the loss-flag short-circuit.
- All 9 PG integration tests + 17 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults

While probing PG-script DX further found three more frictions:

1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to
   `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`
   meant session-scoped advisory locks (`pg_advisory_lock`) leaked
   across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()`
   to the chain — `DISCARD ALL` covered this implicitly via
   `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it
   in the switch.

2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g.
   `-- $1 amount (numeric)`) but not provided in the args object was
   bound as NULL with no error / warning. Misspelling the key in the
   args object silently produced a row of NULLs — a notorious DX
   debugging trap. Now: collect the names of declared-but-missing
   args during dispatch and emit a single one-shot warning to the job
   logs at end-of-query naming each one. Bound NULL is preserved for
   back-compat.

3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries
   `arg.default = Some(Number(5))`, but the dispatch fell straight to
   NULL when the arg was missing. Now: respect the default —
   user-supplied value > declaration default > NULL. Also fixes the
   warning logic above (only warn for args that *don't* have a default).

Tests: existing 19 unit + 9 integration pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values

Two more frictions found while probing SDK end-to-end against a real
datatable resource:

1. **Multi-word array types lose the [] suffix in the parser**.
   `transform_types_with_spaces` recognises aliases for "double
   precision", "character varying", "timestamp with time zone", etc.
   but its return type was `&'a str` — only the bare alias, never with
   a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at
   the first space, so the regex's own `(?:\[\])?` array-suffix branch
   sees only `"double"` (not `"double precision[]"`); the `[]` was
   silently lost. Result: `$1::double precision[]` (which the SDK now
   emits for homogeneous float arrays via the new auto-tag) routed
   through `Value::Array → Type::JSONB` and the server failed with
   "cannot cast type jsonb to double precision[]".

   Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>`
   and re-check the trailing bytes after a multi-word match. If they
   start with `[]`, return `format!("{alias}[]")` — Owned. Single-word
   types and the no-match path keep returning Borrowed slices, so no
   allocation in the hot path.

2. **Array arms in `convert_vec_val` rejected stringified values for
   numeric / int* / bool / oid / real / double**. The scalar `convert_val`
   already parses strings into the matching native type for these arg_ts,
   but the array variant only accepted JSON-native counterparts. Sending
   `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for
   bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with
   "Mixed types in array". Now the array arms mirror the scalar ones —
   `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes
   round-trip cleanly.

Tests: 19 unit + 9 integration pass; existing parser tests cover the
multi-word array forms (the regex-cap behaviour didn't break for
single-word types, and Cow plumbing is transparent to all callers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files

CI failures: the perl-driven sweep that added `otyp_inferred: false` to
every `Arg { ... }` literal when I introduced the field in the parser
schema covered `src/lib.rs` files but missed:

  - parsers/windmill-parser-bash/src/lib.rs       (mass-edited but a
    later format pass un-applied a few sites)
  - parsers/windmill-parser-go/src/lib.rs         (same)
  - parsers/windmill-parser-graphql/src/lib.rs    (same)
  - parsers/windmill-parser-nu/tests/tests.rs     (test file — not
    swept the first time)
  - parsers/windmill-parser-ts/tests/tests.rs     (test file — same)

Also tightened the regex to handle `oidx: None` without the trailing
comma (some test files had the field as the last initialiser line).

`cargo build --features <CI feature combo> --workspace --all-targets`
is clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string

Two more frictions found while running the actual SDK end-to-end against
a live datatable resource:

1. **JS `Date`** fell into the typeof "object" branch and was tagged
   `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's
   `json → text → timestamptz` implicit cast chain, but `${date}` against
   a `timestamptz` column without a user-supplied cast bound the value
   as a JSON string and the comparison `timestamptz = json` failed. Now:
   `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`;
   `serializeArgValue` emits `Date.toISOString()` so the executor's
   `Value::String → TIMESTAMPTZ` arm parses it cleanly.

2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)`
   returns `"null"` per the JS spec, so the value reached the executor as
   JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL
   double. Fix: detect non-finite numbers in `serializeArgValue` and
   stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's
   `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals
   directly, and the result-side already renders the values as JSON
   strings (matching round-trip).

SDK unit tests grow from 42 → 44 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(pg): integration coverage for multi-word arrays + stringified array elements

Locks in the two array fixes from the previous commit
(`fix(pg): multi-word PG types with [] suffix lost the array-ness`)
with end-to-end cases in `test_postgresql_arg_type_combinations`:

- `double precision[]`, `character varying[]`, `timestamp without time
  zone[]` — verifies the parser keeps the `[]` suffix after multi-word
  alias resolution.
- `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies
  the array arms of `convert_vec_val` apply the same string-coercion
  the scalar arms do.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: fix indentation drift on otyp_inferred lines

cargo fmt cleanup of leftover indentation where the perl-driven sweep
that introduced the otyp_inferred field landed at the wrong column.
No behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Ruben Fiszel
2026-05-01 17:08:59 +00:00
parent 2b27e39988
commit 9d2bd27bd7
32 changed files with 4263 additions and 465 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/parsers/windmill-parser-bash/src/lib.rs
+42 -20
View File
@@ -134,6 +134,7 @@ fn parse_bash_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: None,
has_default: default.is_some(),
oidx: None,
otyp_inferred: false,
});
} else {
break;
@@ -731,6 +732,7 @@ fn finalize_parameter(
otyp,
has_default,
oidx: None,
otyp_inferred: false,
})
}
@@ -774,7 +776,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -782,7 +785,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -790,7 +794,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: Some(json!("latest with spaces")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -798,7 +803,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -806,7 +812,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: Some(json!("")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -833,7 +840,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: None,
has_default: true, // Optional (not mandatory)
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("string".to_string()), // [string]
@@ -841,7 +849,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: None,
has_default: true, // Optional (not mandatory)
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None, // No type annotation
@@ -849,7 +858,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: Some(json!("default value, with comma")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("int".to_string()), // [int]
@@ -857,7 +867,8 @@ non_required="${5:-}"
typ: Typ::Int,
default: Some(json!(3)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None, // Type inferred from default value
@@ -865,7 +876,8 @@ non_required="${5:-}"
typ: Typ::Float,
default: Some(json!(5.0)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None, // Type inferred from default value
@@ -873,7 +885,8 @@ non_required="${5:-}"
typ: Typ::Int,
default: Some(json!(5)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None, // No type annotation
@@ -881,7 +894,8 @@ non_required="${5:-}"
typ: Typ::Str(None),
default: None,
has_default: true, // Optional (not mandatory)
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("PSCustomObject".to_string()), // [PSCustomObject]
@@ -889,7 +903,8 @@ non_required="${5:-}"
typ: Typ::Object(ObjectType::new(None, None)),
default: None,
has_default: true, // Optional (not mandatory)
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("string[]".to_string()), // [string[]]
@@ -897,7 +912,8 @@ non_required="${5:-}"
typ: Typ::List(Box::new(Typ::Str(None))),
default: None,
has_default: true, // Optional (not mandatory)
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("string".to_string()), // [string] (last type bracket with Mandatory and ValidateSet)
@@ -909,7 +925,8 @@ non_required="${5:-}"
])), // ValidateSet enum
default: None,
has_default: false, // Required (Mandatory attribute)
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -1462,7 +1479,8 @@ param(
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1470,7 +1488,8 @@ param(
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1478,7 +1497,8 @@ param(
typ: Typ::Str(None),
default: Some(json!("latest with spaces")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1486,7 +1506,8 @@ param(
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1494,7 +1515,8 @@ param(
typ: Typ::Str(None),
default: Some(json!("")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
backend/parsers/windmill-parser-csharp/src/lib.rs
+1 -1
View File
@@ -77,7 +77,7 @@ pub fn parse_csharp_sig_meta(code: &str) -> anyhow::Result<CsharpMainSigMeta> {
}
}
let (otyp, typ, name) = parse_csharp_typ(p_list_node, code)?;
args.push(Arg { name, otyp, typ, default, has_default: false, oidx: None });
args.push(Arg { name, otyp, typ, default, has_default: false, oidx: None, otyp_inferred: false });
}
}
}
backend/parsers/windmill-parser-go/src/lib.rs
+27 -13
View File
@@ -34,6 +34,7 @@ pub fn parse_go_sig(code: &str) -> anyhow::Result<MainArgSignature> {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}
})
.collect_vec();
@@ -147,7 +148,10 @@ fn parse_go_typ(typ: &Expression) -> (Option<String>, Typ) {
Typ::Object(ObjectType::new(None, Some(typs))),
)
}
Expression::TypeInterface(_) => (Some("interface{}".to_string()), Typ::Object(ObjectType::new(None, Some(vec![])))),
Expression::TypeInterface(_) => (
Some("interface{}".to_string()),
Typ::Object(ObjectType::new(None, Some(vec![]))),
),
Expression::TypeMap(_) => (
Some("map[string]interface{}".to_string()),
Typ::Object(ObjectType::new(None, Some(vec![]))),
@@ -191,7 +195,8 @@ func main(x int, y string, z bool, l []string, o struct { Name string `json:"nam
typ: Typ::Int,
has_default: false,
default: None,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("string".to_string()),
@@ -199,7 +204,8 @@ func main(x int, y string, z bool, l []string, o struct { Name string `json:"nam
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("bool".to_string()),
@@ -207,7 +213,8 @@ func main(x int, y string, z bool, l []string, o struct { Name string `json:"nam
typ: Typ::Bool,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("[]string".to_string()),
@@ -215,18 +222,23 @@ func main(x int, y string, z bool, l []string, o struct { Name string `json:"nam
typ: Typ::List(Box::new(Typ::Str(None))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("struct { Name string `json:\"name\"` }".to_string()),
name: "o".to_string(),
typ: Typ::Object(ObjectType::new(None, Some(vec![ObjectProperty {
key: "name".to_string(),
typ: Box::new(Typ::Str(None))
},]))),
typ: Typ::Object(ObjectType::new(
None,
Some(vec![ObjectProperty {
key: "name".to_string(),
typ: Box::new(Typ::Str(None))
},])
)),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("interface{}".to_string()),
@@ -234,7 +246,8 @@ func main(x int, y string, z bool, l []string, o struct { Name string `json:"nam
typ: Typ::Object(ObjectType::new(None, Some(vec![]))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("map[string]interface{}".to_string()),
@@ -242,12 +255,13 @@ func main(x int, y string, z bool, l []string, o struct { Name string `json:"nam
typ: Typ::Object(ObjectType::new(None, Some(vec![]))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
has_preprocessor: None,
..Default::default()
..Default::default()
}
);
backend/parsers/windmill-parser-graphql/src/lib.rs
+7 -3
View File
@@ -64,6 +64,7 @@ fn parse_graphql_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ.unwrap()),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -107,7 +108,8 @@ query($i: Int, $arr: [String]!, $wahoo: String = "wahoo") {
typ: Typ::Int,
default: None,
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("[String]".to_string()),
@@ -115,7 +117,8 @@ query($i: Int, $arr: [String]!, $wahoo: String = "wahoo") {
typ: Typ::List(Box::new(Typ::Str(None))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("String".to_string()),
@@ -123,7 +126,8 @@ query($i: Int, $arr: [String]!, $wahoo: String = "wahoo") {
typ: Typ::Str(None),
default: Some(json!("wahoo")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
backend/parsers/windmill-parser-java/src/lib.rs
+41 -20
View File
@@ -69,6 +69,7 @@ pub fn parse_java_sig_meta(code: &str) -> anyhow::Result<JavaMainSigMeta> {
has_default: default.is_some(),
default,
oidx: None,
otyp_inferred: false,
});
}
}
@@ -256,7 +257,8 @@ class Main {
typ: Typ::Bytes,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "b".into(),
@@ -264,7 +266,8 @@ class Main {
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "c".into(),
@@ -272,7 +275,8 @@ class Main {
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "d".into(),
@@ -280,7 +284,8 @@ class Main {
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "e".into(),
@@ -288,7 +293,8 @@ class Main {
typ: Typ::Float,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "f".into(),
@@ -296,7 +302,8 @@ class Main {
typ: Typ::Float,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "g".into(),
@@ -304,7 +311,8 @@ class Main {
typ: Typ::Bool,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "h".into(),
@@ -312,7 +320,8 @@ class Main {
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
]
);
@@ -338,7 +347,8 @@ class Main {
typ: Typ::Bytes,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "b".into(),
@@ -346,7 +356,8 @@ class Main {
typ: Typ::Int,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "c".into(),
@@ -354,7 +365,8 @@ class Main {
typ: Typ::Int,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "d".into(),
@@ -362,7 +374,8 @@ class Main {
typ: Typ::Int,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "e".into(),
@@ -370,7 +383,8 @@ class Main {
typ: Typ::Float,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "f".into(),
@@ -378,7 +392,8 @@ class Main {
typ: Typ::Float,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "g".into(),
@@ -386,7 +401,8 @@ class Main {
typ: Typ::Bool,
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "h".into(),
@@ -394,7 +410,8 @@ class Main {
typ: Typ::Str(None),
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "i".into(),
@@ -402,7 +419,8 @@ class Main {
typ: Typ::Object(ObjectType::new(None, Some(vec![]))),
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
]
);
@@ -427,7 +445,8 @@ class Main {
typ: Typ::List(Box::new(Typ::Int)),
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "b".into(),
@@ -435,7 +454,8 @@ class Main {
typ: Typ::List(Box::new(Typ::Object(ObjectType::new(None, Some(vec![]))))),
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "c".into(),
@@ -443,7 +463,8 @@ class Main {
typ: Typ::List(Box::new(Typ::Str(None))),
default: Some(json!(null)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
]
);
backend/parsers/windmill-parser-nu/src/lib.rs
+1
View File
@@ -152,6 +152,7 @@ pub fn parse_nu_signature(code: &str) -> anyhow::Result<MainArgSignature> {
has_default: default.is_some() || optional,
default: default.or_else(|| if optional { Some(json!(null)) } else { None }),
oidx: None,
otyp_inferred: false,
});
}
backend/parsers/windmill-parser-nu/tests/tests.rs
+60 -30
View File
@@ -27,7 +27,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "b".into(),
@@ -35,7 +36,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "c".into(),
@@ -43,7 +45,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "d".into(),
@@ -51,7 +54,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -80,7 +84,8 @@ mod test {
typ: Typ::Unknown,
default: Some(serde_json::Value::Null),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: None,
@@ -109,7 +114,8 @@ mod test {
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "bar".into(),
@@ -117,7 +123,8 @@ mod test {
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -158,7 +165,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a2".into(),
@@ -166,7 +174,8 @@ mod test {
typ: Typ::Bool,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a3".into(),
@@ -174,7 +183,8 @@ mod test {
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a4".into(),
@@ -182,7 +192,8 @@ mod test {
typ: Typ::Float,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a5".into(),
@@ -190,7 +201,8 @@ mod test {
typ: Typ::Datetime,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a6".into(),
@@ -198,7 +210,8 @@ mod test {
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a7".into(),
@@ -206,7 +219,8 @@ mod test {
typ: Typ::Object(ObjectType::new(None, Some(vec![]))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a8".into(),
@@ -214,7 +228,8 @@ mod test {
typ: Typ::List(Box::new(Typ::Unknown)),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a9".into(),
@@ -222,7 +237,8 @@ mod test {
typ: Typ::List(Box::new(Typ::Object(ObjectType::new(None, Some(vec![]))))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "a10".into(),
@@ -230,7 +246,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -262,7 +279,8 @@ mod test {
typ: Typ::Unknown,
default: Some(json!("Foo")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "bar".into(),
@@ -270,7 +288,8 @@ mod test {
typ: Typ::Str(None),
default: Some(json!("Bar")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "bazz".into(),
@@ -278,7 +297,8 @@ mod test {
typ: Typ::Unknown,
default: Some(json!(3)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -375,7 +395,8 @@ mod test {
typ: Typ::List(Box::new(Typ::Float)),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: None,
@@ -406,7 +427,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "foo".into(),
@@ -414,7 +436,8 @@ mod test {
typ: Typ::List(Box::new(Typ::Float)),
default: Some(json!([2, 3, 4])),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "b".into(),
@@ -422,7 +445,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -452,7 +476,8 @@ mod test {
typ: Typ::Datetime,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: None,
@@ -515,7 +540,8 @@ mod test {
typ: Typ::Unknown,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "b".into(),
@@ -523,7 +549,8 @@ mod test {
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "c".into(),
@@ -531,7 +558,8 @@ mod test {
typ: Typ::Unknown,
default: Some(serde_json::Value::Null),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "d".into(),
@@ -539,7 +567,8 @@ mod test {
typ: Typ::Str(None),
default: Some(json!("foo")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
name: "bi".into(),
@@ -547,7 +576,8 @@ mod test {
typ: Typ::Unknown,
default: Some(serde_json::Value::Null),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
backend/parsers/windmill-parser-php/src/lib.rs
+11 -5
View File
@@ -91,6 +91,7 @@ pub fn parse_php_signature(
has_default: default.is_some(),
default,
oidx: None,
otyp_inferred: false,
}
})
.collect();
@@ -146,7 +147,8 @@ function main(string $input1 = \"hey\", bool $input2 = false, int $input3 = 3, f
typ: Typ::Str(None),
has_default: true,
default: Some(Value::String("hey".to_string())),
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -154,7 +156,8 @@ function main(string $input1 = \"hey\", bool $input2 = false, int $input3 = 3, f
typ: Typ::Bool,
has_default: true,
default: Some(Value::Bool(false)),
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -162,7 +165,8 @@ function main(string $input1 = \"hey\", bool $input2 = false, int $input3 = 3, f
typ: Typ::Int,
has_default: true,
default: Some(Value::Number(Number::from(3))),
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -170,7 +174,8 @@ function main(string $input1 = \"hey\", bool $input2 = false, int $input3 = 3, f
typ: Typ::Float,
has_default: true,
default: Some(Value::Number(Number::from_f64(f64::from(4.5)).unwrap())),
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -178,7 +183,8 @@ function main(string $input1 = \"hey\", bool $input2 = false, int $input3 = 3, f
typ: Typ::Resource("stripe".to_string()),
has_default: false,
default: None,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
backend/parsers/windmill-parser-py/src/lib.rs
+53 -26
View File
@@ -477,6 +477,7 @@ pub fn parse_python_signature(
has_default: has_default || default.is_some(),
default,
oidx: None,
otyp_inferred: false,
}
})
.collect(),
@@ -716,7 +717,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -724,7 +726,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::Datetime,
default: Some(json!("<function call>")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -732,7 +735,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::Bytes,
default: Some(json!("<function call>")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -740,7 +744,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::Str(None),
default: Some(json!("wewe")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -748,7 +753,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::Int,
default: Some(json!(21)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -756,7 +762,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::List(Box::new(Typ::Int)),
default: Some(json!([1, 2])),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -764,7 +771,8 @@ def main(test1: str, name: datetime.datetime = datetime.now(), byte: bytes = byt
typ: Typ::Bool,
default: Some(json!(true)),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -806,7 +814,8 @@ def main(test1: str,
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -814,7 +823,8 @@ def main(test1: str,
typ: Typ::Datetime,
default: Some(json!("<function call>")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -822,7 +832,8 @@ def main(test1: str,
typ: Typ::Bytes,
default: Some(json!("<function call>")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -830,7 +841,8 @@ def main(test1: str,
typ: Typ::Resource("postgresql".to_string()),
default: Some(json!("$res:g/all/resource")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -867,7 +879,8 @@ def main(test1: str,
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -875,7 +888,8 @@ def main(test1: str,
typ: Typ::Resource("s3_object".to_string()),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -883,7 +897,8 @@ def main(test1: str,
typ: Typ::Str(None),
default: Some(json!("test")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -891,7 +906,8 @@ def main(test1: str,
typ: Typ::Bytes,
default: Some(json!("<function call>")),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -925,7 +941,8 @@ def main(test1: Literal["foo", "bar"], test2: List[Literal["foo", "bar"]]): retu
typ: Typ::Str(Some(vec!["foo".to_string(), "bar".to_string()])),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -936,7 +953,8 @@ def main(test1: Literal["foo", "bar"], test2: List[Literal["foo", "bar"]]): retu
])))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -969,7 +987,8 @@ def main(test1: DynSelect_foo): return
typ: Typ::DynSelect("foo".to_string()),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -1094,7 +1113,8 @@ def main(a: list, e: List[int], b: list = [1,2,3,4], c = [1,2,3,4], d = ["a", "b
typ: Typ::List(Box::new(Typ::Str(None))),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1102,7 +1122,8 @@ def main(a: list, e: List[int], b: list = [1,2,3,4], c = [1,2,3,4], d = ["a", "b
typ: Typ::List(Box::new(Typ::Int)),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1110,7 +1131,8 @@ def main(a: list, e: List[int], b: list = [1,2,3,4], c = [1,2,3,4], d = ["a", "b
typ: Typ::List(Box::new(Typ::Int)),
default: Some(json!([1, 2, 3, 4])),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1118,7 +1140,8 @@ def main(a: list, e: List[int], b: list = [1,2,3,4], c = [1,2,3,4], d = ["a", "b
typ: Typ::List(Box::new(Typ::Int)),
default: Some(json!([1, 2, 3, 4])),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1126,7 +1149,8 @@ def main(a: list, e: List[int], b: list = [1,2,3,4], c = [1,2,3,4], d = ["a", "b
typ: Typ::List(Box::new(Typ::Str(None))),
default: Some(json!(["a", "b"])),
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -1160,7 +1184,8 @@ def main(a: str, b: Optional[str], c: str | None): return
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1168,7 +1193,8 @@ def main(a: str, b: Optional[str], c: str | None): return
typ: Typ::Str(None),
default: None,
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: None,
@@ -1176,7 +1202,8 @@ def main(a: str, b: Optional[str], c: str | None): return
typ: Typ::Str(None),
default: None,
has_default: true,
oidx: None
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
backend/parsers/windmill-parser-rust/src/lib.rs
+1 -1
View File
@@ -21,7 +21,7 @@ pub fn parse_rust_signature(code: &str) -> anyhow::Result<MainArgSignature> {
.iter()
.map(|param| {
let (otyp, typ, name) = parse_rust_typ(param);
Arg { name, otyp, typ, default: None, has_default: false, oidx: None }
Arg { name, otyp, typ, default: None, has_default: false, oidx: None, otyp_inferred: false }
})
.collect_vec();
Ok(MainArgSignature {
backend/parsers/windmill-parser-sql/src/lib.rs
+212 -18
View File
@@ -281,6 +281,7 @@ fn parse_oracledb_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -305,6 +306,7 @@ fn parse_oracledb_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
}
@@ -331,6 +333,7 @@ fn parse_sql_sanitized_interpolation(code: &str) -> Vec<Arg> {
otyp: Some(otyp.to_string()),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -360,6 +363,7 @@ fn parse_mysql_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -384,6 +388,7 @@ fn parse_mysql_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
}
@@ -518,7 +523,25 @@ fn run_on_sql_statement_matches<
}
pub fn parse_pg_statement_arg_indices(code: &str) -> HashSet<i32> {
let mut arg_indices = HashSet::new();
parse_pg_statement_arg_positions(code)
.into_iter()
.map(|(idx, _)| idx)
.collect()
}
/// Like `parse_pg_statement_arg_indices`, but also returns the byte range of
/// each placeholder occurrence in `code` (excluding `$`, including the digits).
/// The same string-/comment-/dollar-quote-aware tokenizer is used, so
/// occurrences inside string literals and comments are correctly skipped —
/// this is what callers need to renumber `$N → $M` without mangling literal
/// SQL bytes that happen to match the `$\d+` pattern.
///
/// The returned vec is in source order. Each entry is `(idx, range)` where
/// `idx` is the parameter number and `range` covers the `$N` digits (i.e.
/// `code[range.start - 1 .. range.end]` is the full `$N` token, and
/// `code[range]` is just the digits).
pub fn parse_pg_statement_arg_positions(code: &str) -> Vec<(i32, std::ops::Range<usize>)> {
let mut positions = Vec::new();
run_on_sql_statement_matches(
code,
true,
@@ -529,21 +552,24 @@ pub fn parse_pg_statement_arg_indices(code: &str) -> HashSet<i32> {
.is_some_and(|&(_, next_char)| next_char.is_ascii_digit())
},
|_, chars| {
let start = chars.peek().map(|&(i, _)| i).unwrap_or(0);
let mut arg_idx = String::new();
while let Some(&(_, char)) = chars.peek() {
let mut end = start;
while let Some(&(i, char)) = chars.peek() {
if char.is_ascii_digit() {
arg_idx.push(char);
end = i + char.len_utf8();
chars.next();
} else {
break;
}
}
if let Ok(arg_idx) = arg_idx.parse::<i32>() {
arg_indices.insert(arg_idx);
positions.push((arg_idx, start..end));
}
},
);
arg_indices
positions
}
fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
@@ -577,12 +603,16 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
otyp: Some(typ),
has_default,
oidx: Some(idx),
otyp_inferred: false,
});
}
}
// Second pass: infer types from usage for non-explicitly-typed args
let mut hm: HashMap<i32, String> = HashMap::new();
// Second pass: infer types from usage for non-explicitly-typed args.
// We track whether each entry came from an inline `$N::TYPE` cast or from
// the parser's "text" fallback, so the executor can later distinguish
// "user committed to text" from "no info, use a placeholder".
let mut hm: HashMap<i32, (String, bool)> = HashMap::new();
for cap in RE_CODE_PGSQL.captures_iter(code) {
let idx = cap
.get(1)
@@ -594,15 +624,23 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
continue;
}
let typ = cap
let cast = cap
.get(2)
.map(|cap| transform_types_with_spaces(&cap, &code))
.unwrap_or("text");
hm.insert(idx, typ.to_string());
.map(|cap| transform_types_with_spaces(&cap, &code));
let inferred_default = cast.is_none();
let typ: std::borrow::Cow<str> = cast.unwrap_or(std::borrow::Cow::Borrowed("text"));
// Prefer an explicit cast over a previously seen default — once we
// have any inline cast for the index, lock it in.
match hm.get(&idx) {
Some((_, false)) => {} // already locked from explicit cast
_ => {
hm.insert(idx, (typ.into_owned(), inferred_default));
}
}
}
// Add inferred args
for (i, v) in hm.iter() {
for (i, (v, inferred)) in hm.iter() {
let typ = v.to_lowercase();
args.push(Arg {
name: format!("${}", i),
@@ -611,6 +649,7 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
otyp: Some(typ),
has_default: false,
oidx: Some(*i),
otyp_inferred: *inferred,
});
}
@@ -646,6 +685,7 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
otyp: oarg.otyp,
has_default,
oidx: oarg.oidx,
otyp_inferred: oarg.otyp_inferred,
};
}
}
@@ -657,8 +697,12 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
}
// The regex doesn't parse types with space such as "character varying"
// So we look for them manually and replace them with their shorter counterpart
fn transform_types_with_spaces<'a>(cap: &Match<'a>, code: &str) -> &'a str {
// So we look for them manually and replace them with their shorter counterpart.
// Returns `Cow::Borrowed` for the trivial case (the regex's own match) and
// `Cow::Owned` when we need to alias a multi-word type and/or append a `[]`
// suffix that the regex's `\w+` capture didn't pick up.
fn transform_types_with_spaces<'a>(cap: &Match<'a>, code: &str) -> std::borrow::Cow<'a, str> {
use std::borrow::Cow;
lazy_static! {
static ref TYPES: [(&'static str, &'static str); 6] = [
("character varying", "varchar"),
@@ -671,20 +715,31 @@ fn transform_types_with_spaces<'a>(cap: &Match<'a>, code: &str) -> &'a str {
}
let typ = &code[cap.start()..];
for (long_type, alias) in TYPES.iter() {
let mut typ = typ;
let mut rest = typ;
let mut found_mismatch = false;
for token in long_type.split(' ') {
if typ.len() < token.len() || !typ[..token.len()].eq_ignore_ascii_case(token) {
if rest.len() < token.len() || !rest[..token.len()].eq_ignore_ascii_case(token) {
found_mismatch = true;
break;
}
typ = typ[token.len()..].trim_start();
rest = rest[token.len()..].trim_start();
}
if !found_mismatch {
return alias;
// The regex captured only the first word (`\w+`), so its `[]`
// detection in `(?:\[\])?` matched against the wrong position
// and is empty for multi-word types. Re-check the trailing
// bytes after the multi-word match: if they start with `[]`,
// append the array suffix to the alias so the dispatch routes
// through `convert_vec_val` instead of binding as JSONB.
let with_suffix = rest.starts_with("[]");
return if with_suffix {
Cow::Owned(format!("{alias}[]"))
} else {
Cow::Borrowed(*alias)
};
}
}
cap.as_str()
Cow::Borrowed(cap.as_str())
}
pub fn parse_sql_statement_named_params(code: &str, prefix: char) -> HashSet<String> {
@@ -736,6 +791,7 @@ fn parse_bigquery_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -765,6 +821,7 @@ fn parse_duckdb_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -794,6 +851,7 @@ fn parse_snowflake_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -823,6 +881,7 @@ fn parse_mssql_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
otyp: Some(typ),
has_default,
oidx: None,
otyp_inferred: false,
});
}
@@ -1006,6 +1065,7 @@ SELECT * FROM table WHERE token=$1::TEXT AND image=$2::BIGINT
default: None,
has_default: false,
oidx: Some(1),
otyp_inferred: false,
},
Arg {
otyp: Some("bigint".to_string()),
@@ -1014,6 +1074,7 @@ SELECT * FROM table WHERE token=$1::TEXT AND image=$2::BIGINT
default: None,
has_default: false,
oidx: Some(2),
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1048,6 +1109,7 @@ SELECT $2::TEXT;
default: None,
has_default: false,
oidx: Some(1),
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1056,6 +1118,7 @@ SELECT $2::TEXT;
default: None,
has_default: false,
oidx: Some(2),
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1064,6 +1127,7 @@ SELECT $2::TEXT;
default: None,
has_default: false,
oidx: Some(3),
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1216,6 +1280,54 @@ SELECT $2;"#;
Ok(())
}
#[test]
fn test_parse_pg_statement_arg_positions_skips_strings_and_comments() -> anyhow::Result<()> {
// Each occurrence's byte range covers JUST the digits (after `$`).
let code = "SELECT $5, $50";
let positions = parse_pg_statement_arg_positions(code);
let collected: Vec<(i32, &str)> = positions
.iter()
.map(|(idx, range)| (*idx, &code[range.clone()]))
.collect();
assert_eq!(collected, vec![(5, "5"), (50, "50")]);
// String literals and comments must not produce positions — this is
// what stops the do_postgresql_inner rewrite from mangling SQL like
// `'price: $5'`.
let code = "SELECT 'literal $5' AS lbl, $5 FROM t -- mention $5";
let positions = parse_pg_statement_arg_positions(code);
let positions_only: Vec<(i32, std::ops::Range<usize>)> = positions.clone();
assert_eq!(
positions_only.iter().map(|(i, _)| *i).collect::<Vec<_>>(),
vec![5],
"only the real $5 between 'lbl,' and 'FROM' should be returned"
);
// The single returned position is the real placeholder (between
// `lbl, ` and ` FROM`).
let (idx, range) = &positions[0];
assert_eq!(*idx, 5);
// `code[range.start - 1 .. range.end]` should be the full `$5` token.
assert_eq!(&code[range.start - 1..range.end], "$5");
// Dollar-quoted blocks similarly skipped.
let code = "SELECT $$body with $5 inside$$, $7 FROM t";
let positions = parse_pg_statement_arg_positions(code);
assert_eq!(
positions.iter().map(|(i, _)| *i).collect::<Vec<_>>(),
vec![7],
"$5 inside $$...$$ is part of the string"
);
// Repeat indices show up multiple times — caller can rewrite each.
let code = "SELECT $1, $1, $2";
let positions = parse_pg_statement_arg_positions(code);
assert_eq!(
positions.iter().map(|(i, _)| *i).collect::<Vec<_>>(),
vec![1, 1, 2]
);
Ok(())
}
#[test]
fn test_parse_sql_blocks_non_pg_ignores_dollar_quotes() -> anyhow::Result<()> {
// Non-Postgres dialects (MySQL/Oracle/BigQuery/Snowflake) pass `false`,
@@ -1259,6 +1371,7 @@ SELECT ?, ?;
default: Some(json!(3)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1267,6 +1380,7 @@ SELECT ?, ?;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1300,6 +1414,7 @@ SELECT :param2;
default: Some(json!(3)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1308,6 +1423,7 @@ SELECT :param2;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1316,6 +1432,7 @@ SELECT :param2;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1349,6 +1466,7 @@ SELECT @token;
default: Some(json!("abc")),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("int64".to_string()),
@@ -1357,6 +1475,7 @@ SELECT @token;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1390,6 +1509,7 @@ SELECT ?;
default: Some(json!(3)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("varchar".to_string()),
@@ -1398,6 +1518,7 @@ SELECT ?;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("varchar".to_string()),
@@ -1406,6 +1527,7 @@ SELECT ?;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}
],
auto_kind: None,
@@ -1439,6 +1561,7 @@ SELECT @P2;
default: Some(json!(3)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("varchar".to_string()),
@@ -1447,6 +1570,7 @@ SELECT @P2;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("varchar".to_string()),
@@ -1455,6 +1579,7 @@ SELECT @P2;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1489,6 +1614,7 @@ SELECT * FROM table_name WHERE thing = :name4;
default: Some(json!(3)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1497,6 +1623,7 @@ SELECT * FROM table_name WHERE thing = :name4;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1505,6 +1632,7 @@ SELECT * FROM table_name WHERE thing = :name4;
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1536,6 +1664,7 @@ SELECT * FROM users WHERE id = $1 AND email = $2::text;
default: None,
has_default: false,
oidx: Some(1),
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1544,6 +1673,7 @@ SELECT * FROM users WHERE id = $1 AND email = $2::text;
default: None,
has_default: false,
oidx: Some(2),
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1575,6 +1705,7 @@ SELECT * FROM users LIMIT $1 OFFSET $2;
default: Some(json!(10)),
has_default: true,
oidx: Some(1),
otyp_inferred: false,
},
Arg {
otyp: Some("bigint".to_string()),
@@ -1583,6 +1714,7 @@ SELECT * FROM users LIMIT $1 OFFSET $2;
default: Some(json!(0)),
has_default: true,
oidx: Some(2),
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1618,6 +1750,7 @@ WHERE id = $1
default: None,
has_default: false,
oidx: Some(1),
otyp_inferred: false,
},
Arg {
otyp: Some("text".to_string()),
@@ -1626,6 +1759,7 @@ WHERE id = $1
default: None,
has_default: false,
oidx: Some(2),
otyp_inferred: false,
},
Arg {
otyp: Some("timestamptz".to_string()),
@@ -1634,6 +1768,7 @@ WHERE id = $1
default: None,
has_default: false,
oidx: Some(3),
otyp_inferred: false,
},
],
auto_kind: None,
@@ -1663,6 +1798,7 @@ SELECT * FROM users WHERE id = ANY($1);
default: None,
has_default: false,
oidx: Some(1),
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: None,
@@ -1693,6 +1829,7 @@ SELECT $1::integer;
default: None,
has_default: false,
oidx: Some(1),
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: None,
@@ -1703,6 +1840,62 @@ SELECT $1::integer;
Ok(())
}
#[test]
fn test_parse_pgsql_otyp_inferred_flag() -> anyhow::Result<()> {
// Bare `$N` (no inline cast, no decl) should produce otyp = "text"
// *and* otyp_inferred = true. This is the signal the PG executor
// uses to decide whether the user committed to a text target.
let code_bare = "SELECT $1, $2";
let args = parse_pgsql_sig(code_bare)?.args;
let map: HashMap<String, (Option<String>, bool)> = args
.into_iter()
.map(|a| (a.name, (a.otyp, a.otyp_inferred)))
.collect();
assert_eq!(
map.get("$1").cloned(),
Some((Some("text".to_string()), true)),
"bare $1 → otyp_inferred true"
);
assert_eq!(
map.get("$2").cloned(),
Some((Some("text".to_string()), true)),
"bare $2 → otyp_inferred true"
);
// Inline `$N::TYPE` cast → otyp_inferred = false (user committed).
let args = parse_pgsql_sig("SELECT $1::int, $2::text")?.args;
let map: HashMap<String, (Option<String>, bool)> = args
.into_iter()
.map(|a| (a.name, (a.otyp, a.otyp_inferred)))
.collect();
assert_eq!(
map.get("$1").cloned(),
Some((Some("int".to_string()), false))
);
assert_eq!(
map.get("$2").cloned(),
Some((Some("text".to_string()), false)),
"explicit $2::text → otyp_inferred false (distinct from bare $2)"
);
// Declaration `-- $N name (TYPE)` → otyp_inferred = false (decl is
// explicit by definition).
let args = parse_pgsql_sig("-- $1 name (text)\nSELECT $1")?.args;
assert_eq!(args[0].otyp.as_deref(), Some("text"));
assert!(!args[0].otyp_inferred);
// Mixed: $1 has decl, $2 is bare → flag differs per arg.
let args = parse_pgsql_sig("-- $1 a (int)\nSELECT $1, $2")?.args;
let map: HashMap<String, bool> = args
.into_iter()
.map(|a| (a.name, a.otyp_inferred))
.collect();
assert_eq!(map.get("a").copied(), Some(false), "$1 decl → not inferred");
assert_eq!(map.get("$2").copied(), Some(true), "$2 bare → inferred");
Ok(())
}
#[test]
fn test_parse_s3object_arg_per_dialect() -> anyhow::Result<()> {
// Confirms that `(s3object)` is recognised as a resource-typed arg in every
@@ -1782,6 +1975,7 @@ SELECT x
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: None,
backend/parsers/windmill-parser-ts/src/lib.rs
+20 -5
View File
@@ -208,9 +208,7 @@ pub fn parse_relative_imports(code: &str, path: &str) -> anyhow::Result<Vec<Stri
/// Check if an import path is a relative import (starts with `./`, `../`, or `/`)
fn is_relative_import(import_path: &str) -> bool {
import_path.starts_with("./")
|| import_path.starts_with("../")
|| import_path.starts_with("/")
import_path.starts_with("./") || import_path.starts_with("../") || import_path.starts_with("/")
}
/// Normalize a path by resolving `.` and `..` components
@@ -542,6 +540,7 @@ fn parse_param(
default: None,
has_default: ident.id.optional || nullable,
oidx: None,
otyp_inferred: false,
})
}
// Pat::Object(ObjectPat { ... }) = todo!()
@@ -596,13 +595,29 @@ fn parse_param(
if typ == Typ::Unknown && dflt.is_some() {
typ = json_to_typ(dflt.as_ref().unwrap(), false);
}
Ok(Arg { otyp, name, typ, default: dflt, has_default: true, oidx: None })
Ok(Arg {
otyp,
name,
typ,
default: dflt,
has_default: true,
oidx: None,
otyp_inferred: false,
})
}
Pat::Object(ObjectPat { type_ann, .. }) => {
let (typ, nullable) = eval_type_ann(symbol_table, type_resolver, &type_ann);
*counter += 1;
let name = format!("anon{}", counter);
Ok(Arg { otyp: None, name, typ, default: None, has_default: nullable, oidx: None })
Ok(Arg {
otyp: None,
name,
typ,
default: None,
has_default: nullable,
oidx: None,
otyp_inferred: false,
})
}
_ => Err(anyhow::anyhow!(
"parameter syntax unsupported: `{}`: {:#?}",
backend/parsers/windmill-parser-ts/tests/tests.rs
+33 -1
View File
@@ -2,7 +2,9 @@
mod tests {
use serde_json::json;
use windmill_parser::{Arg, MainArgSignature, ObjectProperty, ObjectType, Typ};
use windmill_parser_ts::{parse_deno_signature, parse_expr_for_imports, parse_relative_imports};
use windmill_parser_ts::{
parse_deno_signature, parse_expr_for_imports, parse_relative_imports,
};
#[test]
fn test_imports_basic() {
@@ -78,6 +80,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "num_param".to_string(),
@@ -86,6 +89,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "bool_param".to_string(),
@@ -94,6 +98,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "any_param".to_string(),
@@ -102,6 +107,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -136,6 +142,7 @@ mod tests {
default: Some(json!("World")),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "count".to_string(),
@@ -144,6 +151,7 @@ mod tests {
default: Some(json!(42)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "enabled".to_string(),
@@ -152,6 +160,7 @@ mod tests {
default: Some(json!(true)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -186,6 +195,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "numbers".to_string(),
@@ -194,6 +204,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "items".to_string(),
@@ -202,6 +213,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -235,6 +247,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: Some(false),
@@ -267,6 +280,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},],
auto_kind: None,
has_preprocessor: Some(false),
@@ -308,6 +322,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -347,6 +362,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -406,6 +422,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -440,6 +457,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "base64_param".to_string(),
@@ -448,6 +466,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "email_param".to_string(),
@@ -456,6 +475,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "sql_param".to_string(),
@@ -464,6 +484,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -498,6 +519,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "optional".to_string(),
@@ -506,6 +528,7 @@ mod tests {
default: None,
has_default: true,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "with_default".to_string(),
@@ -514,6 +537,7 @@ mod tests {
default: Some(json!(false)),
has_default: true,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -593,6 +617,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -623,6 +648,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -653,6 +679,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(false),
@@ -686,6 +713,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "numbers".to_string(),
@@ -694,6 +722,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
Arg {
name: "plain".to_string(),
@@ -702,6 +731,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
},
],
auto_kind: None,
@@ -744,6 +774,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(true),
@@ -775,6 +806,7 @@ mod tests {
default: None,
has_default: false,
oidx: None,
otyp_inferred: false,
}],
auto_kind: None,
has_preprocessor: Some(true),
backend/parsers/windmill-parser-yaml/src/lib.rs
+3
View File
@@ -50,6 +50,7 @@ pub fn parse_ansible_sig(inner_content: &str) -> anyhow::Result<MainArgSignature
has_default: default.is_some(),
default,
oidx: None,
otyp_inferred: false,
})
}
}
@@ -68,6 +69,7 @@ pub fn parse_ansible_sig(inner_content: &str) -> anyhow::Result<MainArgSignature
has_default: inv.default.is_some(),
default: inv.default.map(|v| json!(format!("$res:{}", v))),
oidx: None,
otyp_inferred: false,
});
}
}
@@ -81,6 +83,7 @@ pub fn parse_ansible_sig(inner_content: &str) -> anyhow::Result<MainArgSignature
has_default: false,
default: None,
oidx: None,
otyp_inferred: false,
});
}
}
backend/parsers/windmill-parser/src/lib.rs
+9
View File
@@ -106,6 +106,15 @@ pub struct Arg {
pub default: Option<serde_json::Value>,
pub has_default: bool,
pub oidx: Option<i32>,
/// `true` when `otyp` is the parser's fallback default rather than a value
/// the user (or SDK) actually wrote down. Currently only set by the PG SQL
/// parser when a placeholder has no `-- $N name (TYPE)` declaration *and*
/// no `$N::TYPE` inline cast — the otyp is `"text"` purely as a
/// placeholder. Consumers that care about original intent (e.g. the PG
/// executor deciding whether to coerce `Number → String` for a text
/// target) should treat `otyp_inferred = true` as "type unknown".
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub otyp_inferred: bool,
}
pub fn json_to_typ(js: &Value, precise_arrays: bool) -> Typ {
backend/tests/end_user_email.rs
+96 -44
View File
@@ -63,10 +63,7 @@ fn authed(builder: reqwest::RequestBuilder, token: &str) -> reqwest::RequestBuil
/// Create an app with inline script via API
async fn create_app_with_inline_script(port: u16, path: &str) -> anyhow::Result<()> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps/create",
port
);
let url = format!("http://localhost:{}/api/w/test-workspace/apps/create", port);
let resp = authed(client().post(&url), SAME_WS_TOKEN)
.json(&json!({
"path": path,
@@ -102,17 +99,18 @@ async fn create_app_with_inline_script(port: u16, path: &str) -> anyhow::Result<
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("create app failed: {} - {}", resp.status(), resp.text().await?);
anyhow::bail!(
"create app failed: {} - {}",
resp.status(),
resp.text().await?
);
}
Ok(())
}
/// Create a raw app with inline script via API (uses regular app endpoint with rawapp type)
async fn create_raw_app_with_inline_script(port: u16, path: &str) -> anyhow::Result<()> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps/create",
port
);
let url = format!("http://localhost:{}/api/w/test-workspace/apps/create", port);
let resp = authed(client().post(&url), SAME_WS_TOKEN)
.json(&json!({
"path": path,
@@ -146,12 +144,21 @@ async fn create_raw_app_with_inline_script(port: u16, path: &str) -> anyhow::Res
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("create raw app failed: {} - {}", resp.status(), resp.text().await?);
anyhow::bail!(
"create raw app failed: {} - {}",
resp.status(),
resp.text().await?
);
}
Ok(())
}
async fn run_app_inline_script(port: u16, token: &str, app_path: &str, force_viewer: bool) -> anyhow::Result<String> {
async fn run_app_inline_script(
port: u16,
token: &str,
app_path: &str,
force_viewer: bool,
) -> anyhow::Result<String> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps_u/execute_component/{}",
port, app_path
@@ -173,13 +180,22 @@ async fn run_app_inline_script(port: u16, token: &str, app_path: &str, force_vie
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("app inline script run failed: {} - {}", resp.status(), resp.text().await?);
anyhow::bail!(
"app inline script run failed: {} - {}",
resp.status(),
resp.text().await?
);
}
let job_id = resp.text().await?;
wait_for_job_result(port, token, &job_id).await
}
async fn run_raw_app_inline_script(port: u16, token: &str, app_path: &str, force_viewer: bool) -> anyhow::Result<String> {
async fn run_raw_app_inline_script(
port: u16,
token: &str,
app_path: &str,
force_viewer: bool,
) -> anyhow::Result<String> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps_u/execute_component/{}",
port, app_path
@@ -200,7 +216,11 @@ async fn run_raw_app_inline_script(port: u16, token: &str, app_path: &str, force
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("raw app inline script run failed: {} - {}", resp.status(), resp.text().await?);
anyhow::bail!(
"raw app inline script run failed: {} - {}",
resp.status(),
resp.text().await?
);
}
let job_id = resp.text().await?;
wait_for_job_result(port, token, &job_id).await
@@ -215,8 +235,12 @@ async fn wait_for_job_result(port: u16, token: &str, job_id: &str) -> anyhow::Re
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let resp = authed(client().get(&url), token).send().await?;
if resp.status().is_success() {
return Ok(resp.json::<serde_json::Value>().await?
.as_str().unwrap_or("").to_string());
return Ok(resp
.json::<serde_json::Value>()
.await?
.as_str()
.unwrap_or("")
.to_string());
}
}
anyhow::bail!("timeout waiting for job result")
@@ -268,24 +292,38 @@ async fn test_app_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
let app_path = "f/test/email_app";
in_test_worker(Connection::Sql(db.clone()), async move {
// Create the app with inline script first
create_app_with_inline_script(port, app_path).await?;
in_test_worker(
Connection::Sql(db.clone()),
async move {
// Create the app with inline script first
create_app_with_inline_script(port, app_path).await?;
// Same workspace user (force_viewer mode works for workspace members)
let result = run_app_inline_script(port, SAME_WS_TOKEN, app_path, true).await?;
assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Same workspace user (force_viewer mode works for workspace members)
let result = run_app_inline_script(port, SAME_WS_TOKEN, app_path, true).await?;
assert_eq!(
result, SAME_WS_EMAIL,
"same workspace user should get their email"
);
// Other workspace user (uses app's anonymous policy + token lookup)
let result = run_app_inline_script(port, OTHER_WS_TOKEN, app_path, false).await?;
assert_eq!(result, OTHER_WS_EMAIL, "other workspace user should get their email");
// Other workspace user (uses app's anonymous policy + token lookup)
let result = run_app_inline_script(port, OTHER_WS_TOKEN, app_path, false).await?;
assert_eq!(
result, OTHER_WS_EMAIL,
"other workspace user should get their email"
);
// No workspace user (uses app's anonymous policy + token lookup)
let result = run_app_inline_script(port, NO_WS_TOKEN, app_path, false).await?;
assert_eq!(result, NO_WS_EMAIL, "no workspace user should get their email");
// No workspace user (uses app's anonymous policy + token lookup)
let result = run_app_inline_script(port, NO_WS_TOKEN, app_path, false).await?;
assert_eq!(
result, NO_WS_EMAIL,
"no workspace user should get their email"
);
Ok::<(), anyhow::Error>(())
}, port).await?;
Ok::<(), anyhow::Error>(())
},
port,
)
.await?;
Ok(())
}
@@ -300,24 +338,38 @@ async fn test_raw_app_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()
let app_path = "f/test/email_raw_app";
in_test_worker(Connection::Sql(db.clone()), async move {
// Create the raw app with inline script first
create_raw_app_with_inline_script(port, app_path).await?;
in_test_worker(
Connection::Sql(db.clone()),
async move {
// Create the raw app with inline script first
create_raw_app_with_inline_script(port, app_path).await?;
// Same workspace user (force_viewer mode works for workspace members)
let result = run_raw_app_inline_script(port, SAME_WS_TOKEN, app_path, true).await?;
assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Same workspace user (force_viewer mode works for workspace members)
let result = run_raw_app_inline_script(port, SAME_WS_TOKEN, app_path, true).await?;
assert_eq!(
result, SAME_WS_EMAIL,
"same workspace user should get their email"
);
// Other workspace user (uses app's anonymous policy + token lookup)
let result = run_raw_app_inline_script(port, OTHER_WS_TOKEN, app_path, false).await?;
assert_eq!(result, OTHER_WS_EMAIL, "other workspace user should get their email");
// Other workspace user (uses app's anonymous policy + token lookup)
let result = run_raw_app_inline_script(port, OTHER_WS_TOKEN, app_path, false).await?;
assert_eq!(
result, OTHER_WS_EMAIL,
"other workspace user should get their email"
);
// No workspace user (uses app's anonymous policy + token lookup)
let result = run_raw_app_inline_script(port, NO_WS_TOKEN, app_path, false).await?;
assert_eq!(result, NO_WS_EMAIL, "no workspace user should get their email");
// No workspace user (uses app's anonymous policy + token lookup)
let result = run_raw_app_inline_script(port, NO_WS_TOKEN, app_path, false).await?;
assert_eq!(
result, NO_WS_EMAIL,
"no workspace user should get their email"
);
Ok::<(), anyhow::Error>(())
}, port).await?;
Ok::<(), anyhow::Error>(())
},
port,
)
.await?;
Ok(())
}
backend/tests/error_handler.rs
+7 -7
View File
@@ -34,10 +34,7 @@ async fn test_error_handler_settings(db: Pool<Postgres>) -> anyhow::Result<()> {
)
.fetch_one(&db)
.await?;
assert_eq!(
after_set,
Some("script/f/test/error_handler".to_string())
);
assert_eq!(after_set, Some("script/f/test/error_handler".to_string()));
// Verify extra_args
let extra_args = sqlx::query_scalar!(
@@ -162,7 +159,8 @@ export async function main(path: string, email: string, job_id: string, is_flow:
priority: None,
apply_preprocessor: false,
concurrency_settings: ConcurrencySettings::default(),
debouncing_settings: DebouncingSettings::default(), labels: None,
debouncing_settings: DebouncingSettings::default(),
labels: None,
})
.run_until_complete(&db, false, server.addr.port())
.await;
@@ -285,7 +283,8 @@ async fn test_error_handler_muted_on_script(db: Pool<Postgres>) -> anyhow::Resul
priority: None,
apply_preprocessor: false,
concurrency_settings: ConcurrencySettings::default(),
debouncing_settings: DebouncingSettings::default(), labels: None,
debouncing_settings: DebouncingSettings::default(),
labels: None,
})
.run_until_complete(&db, false, server.addr.port())
.await;
@@ -380,7 +379,8 @@ async fn test_error_handler_not_triggered_on_success(db: Pool<Postgres>) -> anyh
priority: None,
apply_preprocessor: false,
concurrency_settings: ConcurrencySettings::default(),
debouncing_settings: DebouncingSettings::default(), labels: None,
debouncing_settings: DebouncingSettings::default(),
labels: None,
})
.run_until_complete(&db, false, server.addr.port())
.await;
backend/tests/otel.rs
+1 -4
View File
@@ -395,10 +395,7 @@ async fn test_root_job_span_created_on_success() {
attrs.contains(&"script_path"),
"missing script_path attribute"
);
assert!(
attrs.contains(&"job_kind"),
"missing job_kind attribute"
);
assert!(attrs.contains(&"job_kind"), "missing job_kind attribute");
assert!(
attrs.contains(&"created_by"),
"missing created_by attribute"
backend/tests/retry.rs
+1 -1
View File
@@ -1,12 +1,12 @@
#[cfg(feature = "deno_core")]
mod retry {
use windmill_test_utils::*;
use serde_json::json;
use sqlx::{Pool, Postgres};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use windmill_common::flow_status::FlowStatusModule;
use windmill_common::flows::FlowValue;
use windmill_common::jobs::JobPayload;
use windmill_test_utils::*;
pub async fn initialize_tracing() {
use std::sync::Once;
backend/tests/success_handler.rs
+2 -1
View File
@@ -179,7 +179,8 @@ export async function main(path: string, email: string, job_id: string, is_flow:
priority: None,
apply_preprocessor: false,
concurrency_settings: ConcurrencySettings::default(),
debouncing_settings: DebouncingSettings::default(), labels: None,
debouncing_settings: DebouncingSettings::default(),
labels: None,
})
.run_until_complete(&db, false, server.addr.port())
.await;
backend/tests/suspend_resume.rs
+9 -3
View File
@@ -247,7 +247,9 @@ mod suspend_resume {
#[cfg(feature = "enterprise")]
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_self_approval_disabled_blocks_owner_resume(db: Pool<Postgres>) -> anyhow::Result<()> {
async fn test_self_approval_disabled_blocks_owner_resume(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
@@ -358,7 +360,9 @@ mod suspend_resume {
#[cfg(feature = "enterprise")]
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_self_approval_allowed_when_not_disabled(db: Pool<Postgres>) -> anyhow::Result<()> {
async fn test_self_approval_allowed_when_not_disabled(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
@@ -462,7 +466,9 @@ mod suspend_resume {
#[cfg(feature = "enterprise")]
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_different_user_can_approve_when_self_approval_disabled(db: Pool<Postgres>) -> anyhow::Result<()> {
async fn test_different_user_can_approve_when_self_approval_disabled(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
backend/tests/worker.rs
+931
View File
@@ -1860,6 +1860,937 @@ async fn test_postgresql_100_jobs_cached(db: Pool<Postgres>) -> anyhow::Result<(
Ok(())
}
/// Cover the (Value × arg_t) combinations that #8988 broke. Each shape mirrors
/// what the windmill-client SDK or a hand-written PG script can emit:
///
/// - bare `$N` with no inline cast and no `-- $N name (type)` declaration
/// (parser defaults the otyp to "text"). The user's value can be any JSON
/// shape; the eventual column type is whatever the SQL context implies.
/// - inline `$N::TYPE` casts (the SDK's default for bare `${value}`).
/// - `CAST($N AS T)` syntax (the SDK strips its own cast when this pattern
/// surrounds the value, so the parser sees a bare `$N`).
/// - explicit declaration: `-- $N name (type)`.
///
/// Pre-fix, the dispatch asserted `Type::TEXT` for parser-defaulted args, so
/// e.g. a `Value::Bool` bound to `Box<bool>` failed at the encoder with
/// "cannot convert between the Rust type `bool` and the Postgres type `text`"
/// before the query ever reached the server.
#[sqlx::test(fixtures("base"))]
#[serial(pg_cache)]
async fn test_postgresql_arg_type_combinations(db: Pool<Postgres>) -> anyhow::Result<()> {
use windmill_worker::pg_executor::clear_pg_cache;
initialize_tracing().await;
clear_pg_cache().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let db_arg = json!({"host": "localhost", "port": 5432, "dbname": "windmill", "user": "postgres", "password": "changeme"});
// Build a fresh schema for this test so we don't trip over prior runs.
let setup = r#"
DROP SCHEMA IF EXISTS wm_pg_arg_combo_test CASCADE;
CREATE SCHEMA wm_pg_arg_combo_test;
CREATE TABLE wm_pg_arg_combo_test.bugbool (flag bool);
CREATE TABLE wm_pg_arg_combo_test.sdkbug (n int, f double precision);
CREATE TABLE wm_pg_arg_combo_test.bugmix (id int, name text, payload jsonb, tags text[]);
CREATE TABLE wm_pg_arg_combo_test.allcols (
c_bool bool,
c_int2 smallint,
c_int4 int,
c_int8 bigint,
c_float4 real,
c_float8 double precision,
c_numeric numeric,
c_text text,
c_varchar varchar(64),
c_uuid uuid,
c_date date,
c_time time,
c_ts timestamp,
c_tstz timestamptz,
c_json json,
c_jsonb jsonb,
c_int_arr int[],
c_text_arr text[]
);
CREATE TYPE wm_pg_arg_combo_test.color AS ENUM ('red','green','blue');
CREATE TABLE wm_pg_arg_combo_test.enumtbl (c wm_pg_arg_combo_test.color);
"#;
RunJob::from(JobPayload::Code(RawCode {
hash: None,
content: setup.to_owned(),
path: None,
lock: None,
language: ScriptLang::Postgresql,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
}))
.arg("database", db_arg.clone())
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
// Each case: (name, content, args, expected_result)
let cases: Vec<(&str, String, serde_json::Value, serde_json::Value)> = vec![
// === parser-default "text" otyp (bare $N), value drives the binding ===
(
"bool via CAST AS bool (parser default text)",
"-- $1 arg1\nINSERT INTO wm_pg_arg_combo_test.bugbool VALUES (CAST($1 AS bool)) RETURNING flag".to_owned(),
json!({"arg1": true}),
json!([{"flag": true}]),
),
(
"bare $1 with bool into bool col",
"-- $1 arg1\nINSERT INTO wm_pg_arg_combo_test.bugbool VALUES ($1) RETURNING flag".to_owned(),
json!({"arg1": false}),
json!([{"flag": false}]),
),
(
"bare $1 with bool into text via implicit cast bool->text",
"-- $1 arg1\nSELECT $1::text AS s".to_owned(),
json!({"arg1": true}),
json!([{"s": "true"}]),
),
(
"bare $1 with int into text via implicit cast int->text",
"-- $1 arg1\nSELECT $1::text AS s".to_owned(),
json!({"arg1": 42}),
json!([{"s": "42"}]),
),
(
"object via CAST AS jsonb (parser default text)",
"-- $1 arg1\nSELECT CAST($1 AS jsonb) AS v".to_owned(),
json!({"arg1": {"k": 1}}),
json!([{"v": {"k": 1}}]),
),
(
"string '42' via CAST AS int (parser default text)",
"-- $1 arg1\nSELECT CAST($1 AS int) AS v".to_owned(),
json!({"arg1": "42"}),
json!([{"v": 42}]),
),
(
"NULL into bool col via CAST",
"-- $1 arg1\nINSERT INTO wm_pg_arg_combo_test.bugbool VALUES (CAST($1 AS bool)) RETURNING flag".to_owned(),
json!({"arg1": null}),
json!([{"flag": null}]),
),
// === SDK happy path — inline ::TYPE injected by the client ===
(
"SDK shape: $1::BIGINT, $2::DOUBLE PRECISION",
"-- $1 arg1\n-- $2 arg2\nINSERT INTO wm_pg_arg_combo_test.sdkbug VALUES ($1::BIGINT, $2::DOUBLE PRECISION) RETURNING n, f".to_owned(),
json!({"arg1": 42, "arg2": 3.14}),
json!([{"n": 42, "f": 3.14}]),
),
(
"SDK shape: $1::TEXT with int target via explicit ::int",
"-- $1 arg1\nSELECT $1::TEXT::int AS v".to_owned(),
json!({"arg1": "7"}),
json!([{"v": 7}]),
),
// === explicit declaration -- $N name (type) ===
(
"explicit decl (text)",
"-- $1 arg1 (text)\nSELECT $1 AS v".to_owned(),
json!({"arg1": "hello"}),
json!([{"v": "hello"}]),
),
(
"explicit decl (jsonb) with object",
"-- $1 arg1 (jsonb)\nSELECT $1 AS v".to_owned(),
json!({"arg1": {"k": [1, 2]}}),
json!([{"v": {"k": [1, 2]}}]),
),
// === mixed args: int, text, jsonb, text[] ===
(
"mixed: int + text + jsonb + text[]",
"-- $1 arg1\n-- $2 arg2\n-- $3 arg3\n-- $4 arg4\nINSERT INTO wm_pg_arg_combo_test.bugmix VALUES ($1::int, $2::text, $3::jsonb, $4::text[]) RETURNING *".to_owned(),
json!({"arg1": 7, "arg2": "hello", "arg3": {"k": 1}, "arg4": ["a", "b"]}),
json!([{"id": 7, "name": "hello", "payload": {"k": 1}, "tags": ["a", "b"]}]),
),
// === Every PG type, SDK happy path (inline ::TYPE) ===
(
"all PG types via inline casts",
r#"-- $1 arg1
-- $2 arg2
-- $3 arg3
-- $4 arg4
-- $5 arg5
-- $6 arg6
-- $7 arg7
-- $8 arg8
-- $9 arg9
-- $10 arg10
-- $11 arg11
-- $12 arg12
-- $13 arg13
-- $14 arg14
-- $15 arg15
-- $16 arg16
-- $17 arg17
-- $18 arg18
INSERT INTO wm_pg_arg_combo_test.allcols VALUES (
$1::bool, $2::int2, $3::int4, $4::int8,
$5::real, $6::double precision, $7::numeric,
$8::text, $9::varchar,
$10::uuid, $11::date, $12::time, $13::timestamp, $14::timestamptz,
$15::json, $16::jsonb,
$17::int[], $18::text[]
) RETURNING c_bool, c_int4, c_int8, c_text, c_uuid, c_int_arr"#.to_owned(),
json!({
"arg1": true, "arg2": 1, "arg3": 2, "arg4": 3,
"arg5": 1.5, "arg6": 2.5, "arg7": 3.14,
"arg8": "hello", "arg9": "varhello",
"arg10": "550e8400-e29b-41d4-a716-446655440000",
"arg11": "2024-01-15", "arg12": "10:30:00",
"arg13": "2024-01-15T10:30:00", "arg14": "2024-01-15T10:30:00Z",
"arg15": {"k": 1}, "arg16": {"k": 2},
"arg17": [1,2,3], "arg18": ["a","b"]
}),
json!([{"c_bool": true, "c_int4": 2, "c_int8": 3, "c_text": "hello",
"c_uuid": "550e8400-e29b-41d4-a716-446655440000",
"c_int_arr": [1,2,3]}]),
),
// === Every PG type, declaration-style otyp (Python SDK shape) ===
(
"all PG types via -- $N name (TYPE) decls",
r#"-- $1 arg1 (bool)
-- $2 arg2 (int2)
-- $3 arg3 (int4)
-- $4 arg4 (int8)
-- $5 arg5 (real)
-- $6 arg6 (float8)
-- $7 arg7 (numeric)
-- $8 arg8 (text)
-- $9 arg9 (varchar)
-- $10 arg10 (uuid)
-- $11 arg11 (date)
-- $12 arg12 (time)
-- $13 arg13 (timestamp)
-- $14 arg14 (timestamptz)
-- $15 arg15 (json)
-- $16 arg16 (jsonb)
-- $17 arg17 (int[])
-- $18 arg18 (text[])
INSERT INTO wm_pg_arg_combo_test.allcols VALUES (
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18
) RETURNING c_bool, c_int4, c_int8, c_text, c_int_arr, c_text_arr"#.to_owned(),
json!({
"arg1": false, "arg2": 4, "arg3": 5, "arg4": 6,
"arg5": 1.5, "arg6": 2.5, "arg7": 3.14,
"arg8": "world", "arg9": "varworld",
"arg10": "550e8400-e29b-41d4-a716-446655440000",
"arg11": "2024-01-15", "arg12": "10:30:00",
"arg13": "2024-01-15T10:30:00", "arg14": "2024-01-15T10:30:00Z",
"arg15": [1,2], "arg16": [3,4],
"arg17": [10,20], "arg18": ["x","y"]
}),
json!([{"c_bool": false, "c_int4": 5, "c_int8": 6, "c_text": "world",
"c_int_arr": [10,20], "c_text_arr": ["x","y"]}]),
),
// === Edge values per type ===
("int8 negative", "-- $1 arg1\nSELECT $1::int8 AS v".to_owned(), json!({"arg1": -42}), json!([{"v": -42}])),
("int8 zero", "-- $1 arg1\nSELECT $1::int8 AS v".to_owned(), json!({"arg1": 0}), json!([{"v": 0}])),
("int4 max", "-- $1 arg1\nSELECT $1::int4 AS v".to_owned(), json!({"arg1": 2147483647i64}), json!([{"v": 2147483647i64}])),
("int8 max", "-- $1 arg1\nSELECT $1::int8 AS v".to_owned(), json!({"arg1": 9223372036854775807i64}), json!([{"v": 9223372036854775807i64}])),
("float8 fraction", "-- $1 arg1\nSELECT $1::float8 AS v".to_owned(), json!({"arg1": 0.1 + 0.2}), json!([{"v": 0.1 + 0.2}])),
("empty string", "-- $1 arg1\nSELECT $1::text AS v".to_owned(), json!({"arg1": ""}), json!([{"v": ""}])),
("empty array", "-- $1 arg1\nSELECT $1::int[] AS v".to_owned(), json!({"arg1": []}), json!([{"v": []}])),
("empty object", "-- $1 arg1\nSELECT $1::jsonb AS v".to_owned(), json!({"arg1": {}}), json!([{"v": {}}])),
// === Bool roundtrip across every shape ===
("Bool/bare $1 → bool col", "-- $1 arg1\nSELECT $1 AS v".to_owned(), json!({"arg1": true}), json!([{"v": true}])),
("Bool/inline ::bool", "-- $1 arg1\nSELECT $1::bool AS v".to_owned(), json!({"arg1": true}), json!([{"v": true}])),
("Bool/decl (bool)", "-- $1 arg1 (bool)\nSELECT $1 AS v".to_owned(), json!({"arg1": true}), json!([{"v": true}])),
("Bool/CAST AS bool", "-- $1 arg1\nSELECT CAST($1 AS bool) AS v".to_owned(), json!({"arg1": false}), json!([{"v": false}])),
("Bool/CAST AS bool inside SELECT","-- $1 arg1\nSELECT CAST($1 AS bool) AS v WHERE CAST($1 AS bool) IS NOT NULL".to_owned(), json!({"arg1": true}), json!([{"v": true}])),
// === Object/Array via CAST AS jsonb (regression: non-text otyp via CAST) ===
("Object via CAST AS jsonb", "-- $1 arg1\nSELECT CAST($1 AS jsonb) AS v".to_owned(), json!({"arg1": {"a":1,"b":[2,3]}}), json!([{"v": {"a":1,"b":[2,3]}}])),
("Array via CAST AS jsonb", "-- $1 arg1\nSELECT CAST($1 AS jsonb) AS v".to_owned(), json!({"arg1": [1,"two",{"three":3}]}), json!([{"v": [1,"two",{"three":3}]}])),
("Object inline ::json", "-- $1 arg1\nSELECT $1::json AS v".to_owned(), json!({"arg1": {"k":1}}), json!([{"v": {"k":1}}])),
("Object decl (jsonb)", "-- $1 arg1 (jsonb)\nSELECT $1 AS v".to_owned(), json!({"arg1": {"k":1}}), json!([{"v": {"k":1}}])),
// === Numbers: implicit/explicit casts to text ===
("Number/CAST AS text", "-- $1 arg1\nSELECT CAST($1 AS text) AS v".to_owned(), json!({"arg1": 42}), json!([{"v": "42"}])),
("Number/inline ::text", "-- $1 arg1\nSELECT $1::text AS v".to_owned(), json!({"arg1": 42}), json!([{"v": "42"}])),
("Float/CAST AS text", "-- $1 arg1\nSELECT CAST($1 AS text) AS v".to_owned(), json!({"arg1": 3.14}), json!([{"v": "3.14"}])),
("Negative/CAST AS int8", "-- $1 arg1\nSELECT CAST($1 AS int8) AS v".to_owned(), json!({"arg1": -1}), json!([{"v": -1}])),
// === Strings parsed into typed targets ===
("String '42' as int", "-- $1 arg1\nSELECT $1::int AS v".to_owned(), json!({"arg1": "42"}), json!([{"v": 42}])),
("String '42' as bigint", "-- $1 arg1\nSELECT $1::bigint AS v".to_owned(), json!({"arg1": "42"}), json!([{"v": 42}])),
("String '1.5' as real", "-- $1 arg1\nSELECT $1::real AS v".to_owned(), json!({"arg1": "1.5"}), json!([{"v": 1.5}])),
("String uuid", "-- $1 arg1\nSELECT $1::uuid AS v".to_owned(), json!({"arg1": "550e8400-e29b-41d4-a716-446655440000"}), json!([{"v": "550e8400-e29b-41d4-a716-446655440000"}])),
// === NULL handling for every type ===
("Null/bool", "-- $1 arg1\nSELECT CAST($1 AS bool) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/int", "-- $1 arg1\nSELECT CAST($1 AS int) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/bigint", "-- $1 arg1\nSELECT CAST($1 AS bigint) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/text", "-- $1 arg1\nSELECT $1::text AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/jsonb", "-- $1 arg1\nSELECT CAST($1 AS jsonb) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/uuid", "-- $1 arg1\nSELECT CAST($1 AS uuid) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/timestamp", "-- $1 arg1\nSELECT CAST($1 AS timestamp) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
("Null/date", "-- $1 arg1\nSELECT CAST($1 AS date) AS v".to_owned(), json!({"arg1": null}), json!([{"v": null}])),
// === Multi-statement (datatable scripts often DROP/CREATE/INSERT) ===
(
"multi-statement: DROP/CREATE/INSERT",
r#"DROP TABLE IF EXISTS wm_pg_arg_combo_test.tmp_multi;
CREATE TABLE wm_pg_arg_combo_test.tmp_multi (n int, b bool);
-- $1 arg1
-- $2 arg2
INSERT INTO wm_pg_arg_combo_test.tmp_multi VALUES ($1::int, $2::bool) RETURNING *"#.to_owned(),
json!({"arg1": 10, "arg2": true}),
json!([{"n": 10, "b": true}]),
),
// === Same arg used in multiple positions (parser reorders) ===
(
"same arg twice",
"-- $1 arg1\nSELECT $1::int + $1::int AS v".to_owned(),
json!({"arg1": 5}),
json!([{"v": 10}]),
),
// === Custom enum (unrecognised arg_t → prepare fallback path) ===
// The unrecognised-arg_t fallback works when the user formats the
// value as the enum's text representation themselves, so the binding
// never has to encode a Rust String *as* an enum (which the
// tokio-postgres `ToSql` impls don't support):
(
"custom enum: text representation cast in SQL",
r#"-- $1 arg1
INSERT INTO wm_pg_arg_combo_test.enumtbl
VALUES (CAST($1::text AS wm_pg_arg_combo_test.color))
RETURNING c::text AS c"#
.to_owned(),
json!({"arg1": "green"}),
json!([{"c": "green"}]),
),
// === Sparse positional placeholders ($5, $50) ===
// The pre-fix `String::replace($5 → $1)` chain mangled `$50` into
// `$10`, breaking sparse-index queries. The regex-based renumbering
// handles them as distinct units.
(
"sparse $5 / $50 renumbering",
"-- $5 arg5\n-- $50 arg50\nSELECT $5::int AS a, $50::int AS b".to_owned(),
json!({"arg5": 5, "arg50": 50}),
json!([{"a": 5, "b": 50}]),
),
(
"sparse $5 / $50 reversed in SQL",
"-- $5 arg5\n-- $50 arg50\nSELECT $50::int AS a, $5::int AS b".to_owned(),
json!({"arg5": 5, "arg50": 50}),
json!([{"a": 50, "b": 5}]),
),
(
"sparse same arg used twice + sparse",
"-- $5 arg5\n-- $50 arg50\nSELECT $5::int + $5::int AS a, $50::int AS b".to_owned(),
json!({"arg5": 7, "arg50": 50}),
json!([{"a": 14, "b": 50}]),
),
// === Explicit (text) decl + non-string value: should coerce ===
// Without the otyp_inferred flag, the executor would bind the value's
// natural type (INT8/BOOL) and the WHERE comparison `text = int8` /
// `text = bool` would fail with "operator does not exist". With the
// flag, the parser tells the executor "user committed to text" and
// the value is JSON-stringified so `text = text` works.
(
"decl (text) + Number used in WHERE text comparison",
r#"-- $1 arg1 (text)
SELECT name FROM (VALUES ('42'::text)) AS t(name) WHERE name = $1"#
.to_owned(),
json!({"arg1": 42}),
json!([{"name": "42"}]),
),
(
"decl (text) + Bool used in WHERE text comparison",
r#"-- $1 arg1 (text)
SELECT name FROM (VALUES ('true'::text)) AS t(name) WHERE name = $1"#
.to_owned(),
json!({"arg1": true}),
json!([{"name": "true"}]),
),
(
"decl (varchar) + Number used in WHERE",
r#"-- $1 arg1 (varchar)
SELECT name FROM (VALUES ('99'::varchar)) AS t(name) WHERE name = $1"#
.to_owned(),
json!({"arg1": 99}),
json!([{"name": "99"}]),
),
// === Bare $N (parser-default text) + non-string value: bind native ===
// The user wrote no annotation — we bind the value's natural type so
// it works against whatever column the SQL eventually targets.
(
"bare $1 + Bool into bool col",
"-- $1 arg1\nSELECT $1 = true AS v".to_owned(),
json!({"arg1": true}),
json!([{"v": true}]),
),
// === Custom enum (Kind::Enum) — round-trip via AnyTextValue ===
// Pre-fix this failed at the encoder ("cannot convert String → color")
// because vanilla tokio_postgres' ToSql/FromSql for String reject
// Kind::Enum. The wrapper accepts enum kinds in both directions.
(
"enum: explicit ::wm_pg_arg_combo_test.color cast",
r#"-- $1 arg1
INSERT INTO wm_pg_arg_combo_test.enumtbl
VALUES ($1::wm_pg_arg_combo_test.color) RETURNING c"#
.to_owned(),
json!({"arg1": "blue"}),
json!([{"c": "blue"}]),
),
(
"enum: SELECT a literal value cast to enum",
"-- $1 arg1\nSELECT $1::wm_pg_arg_combo_test.color AS c".to_owned(),
json!({"arg1": "red"}),
json!([{"c": "red"}]),
),
// === Extended String→numeric/real/double/oid/bool arms (#10) ===
(
"String '3.14' → numeric",
"-- $1 arg1\nSELECT $1::numeric AS v".to_owned(),
json!({"arg1": "3.14"}),
json!([{"v": 3.14}]),
),
(
"String '1.5' → real",
"-- $1 arg1\nSELECT $1::real AS v".to_owned(),
json!({"arg1": "1.5"}),
json!([{"v": 1.5}]),
),
(
"String '2.5' → double",
"-- $1 arg1\nSELECT $1::double precision AS v".to_owned(),
json!({"arg1": "2.5"}),
json!([{"v": 2.5}]),
),
(
"String 'true' → bool",
"-- $1 arg1\nSELECT $1::bool AS v".to_owned(),
json!({"arg1": "true"}),
json!([{"v": true}]),
),
(
"String 't' → bool",
"-- $1 arg1\nSELECT $1::bool AS v".to_owned(),
json!({"arg1": "t"}),
json!([{"v": true}]),
),
(
"String '0' → bool false",
"-- $1 arg1\nSELECT $1::bool AS v".to_owned(),
json!({"arg1": "0"}),
json!([{"v": false}]),
),
(
"String '42' → oid",
"-- $1 arg1\nSELECT $1::oid AS v".to_owned(),
json!({"arg1": "42"}),
json!([{"v": 42}]),
),
// === String literals containing $N must NOT be renumbered ===
// Sparse positional args force a rewrite pass; the literal
// `'price: $5'` and the comment `-- mention $5` must survive intact.
(
"renumber must skip $N inside string literal",
r#"-- $5 arg5
-- $50 arg50
SELECT 'price: $5' AS lbl, $5::int + $50::int AS sum"#
.to_owned(),
json!({"arg5": 1, "arg50": 2}),
json!([{"lbl": "price: $5", "sum": 3}]),
),
// === Multi-word PG type names with [] array suffix ===
// Pre-fix: `transform_types_with_spaces` returned a `&str` alias
// ("double" / "varchar" / "timestamptz" / …) and dropped the trailing
// `[]`, so the dispatch routed `Value::Array` through `Type::JSONB`
// and the server failed with "cannot cast type jsonb to <T>[]".
(
"multi-word array: double precision[]",
"-- $1 a\nSELECT $1::double precision[] AS v".to_owned(),
json!({"a": [1.5, 2.5]}),
json!([{"v": [1.5, 2.5]}]),
),
(
"multi-word array: character varying[]",
"-- $1 a\nSELECT $1::character varying[] AS v".to_owned(),
json!({"a": ["x", "y"]}),
json!([{"v": ["x", "y"]}]),
),
(
"multi-word array: timestamp without time zone[]",
"-- $1 a\nSELECT $1::timestamp without time zone[] AS v".to_owned(),
json!({"a": ["2024-01-15T10:30:00"]}),
json!([{"v": ["2024-01-15T10:30:00"]}]),
),
// === Stringified primitives in array args ===
// Mirror the scalar `Value::String → <numeric type>` arms so values
// sent as e.g. `["1.5", "2.5"]` for `numeric[]` (typical for bulk-
// loading via `unnest` or BigInt-stringified arrays) round-trip
// instead of erroring with "Mixed types in array".
(
"numeric[] from stringified decimals",
"-- $1 a\nSELECT $1::numeric[] AS v".to_owned(),
json!({"a": ["1.5", "2.5"]}),
json!([{"v": [1.5, 2.5]}]),
),
(
"int[] from stringified ints",
"-- $1 a\nSELECT $1::int[] AS v".to_owned(),
json!({"a": ["1", "2", "3"]}),
json!([{"v": [1, 2, 3]}]),
),
(
"bool[] from stringified bools",
"-- $1 a\nSELECT $1::bool[] AS v".to_owned(),
json!({"a": ["true", "f", "yes"]}),
json!([{"v": [true, false, true]}]),
),
];
for (name, content, args, expected) in cases {
let mut job = RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Postgresql,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(
)
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
}))
.arg("database", db_arg.clone());
for (k, v) in args.as_object().unwrap() {
job = job.arg(k, v.clone());
}
let result = job
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap_or_else(|| panic!("case '{name}': no json result"));
assert_eq!(result, expected, "case '{name}' mismatch");
}
Ok(())
}
/// Pooler-safety regression test for #8988: when every arg has a resolvable
/// otyp (the common SDK case), the dispatch must use unnamed prepared
/// statements (`query_typed_raw`) and *must not* leak named statements
/// (`s0, s1, ...`) on the cached connection. Behind a transaction-mode pooler
/// (PgBouncer / Supabase pooler / RDS Proxy), accumulated names get dropped
/// when the prepare and execute land on different backend connections, which
/// is what produced the original "prepared statement \"sN\" does not exist"
/// errors.
#[sqlx::test(fixtures("base"))]
#[serial(pg_cache)]
async fn test_postgresql_no_named_statements_after_typed_args(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_worker::pg_executor::clear_pg_cache;
initialize_tracing().await;
clear_pg_cache().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let db_arg = json!({"host": "localhost", "port": 5432, "dbname": "windmill", "user": "postgres", "password": "changeme"});
let make_pg_job = |content: String| {
RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Postgresql,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(
)
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
}))
.arg("database", db_arg.clone())
};
// Run several SDK-shape queries (parameterised, all args resolvable) on the
// same cached connection. None of them should land in `pg_prepared_statements`.
for i in 0..5 {
let result = make_pg_job(format!(
"-- $1 arg1\n-- $2 arg2\nSELECT $1::BIGINT AS a, $2::TEXT AS b, {} AS i;",
i
))
.arg("arg1", json!(i))
.arg("arg2", json!(format!("v{i}")))
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(
result,
json!([{"a": i, "b": format!("v{i}"), "i": i}]),
"iteration {i}"
);
}
// Use the *same* connection (cached one) to peek at pg_prepared_statements.
// Anything matching the tokio-postgres "s\d+" naming would mean the
// pooler-unsafe `prepare + query_raw` path was taken.
let probe = make_pg_job(
"SELECT count(*)::int AS n FROM pg_prepared_statements WHERE name ~ '^s[0-9]+$'".to_owned(),
)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
let n = probe[0]["n"].as_i64().unwrap_or(-1);
assert_eq!(
n, 0,
"expected no leaked named prepared statements, found {n}"
);
Ok(())
}
/// Exercise the `prepare + query_raw` fallback path. The dispatch takes this
/// path when `otyp_to_pg_type` doesn't recognise the parser-derived arg_t —
/// typical for custom enums, domains, and extension types.
///
/// Vanilla `tokio_postgres`'s `ToSql for String` doesn't actually accept
/// `Kind::Enum` / `Kind::Domain`, so an end-to-end happy-path test of an
/// arbitrary custom type isn't possible without `postgres-derive`. What we
/// CAN lock in here is *which dispatch path runs*: when the arg_t is
/// unrecognised, the prepare path must be taken (the server resolves the
/// param type from the cast and the binding then errors at the encoder).
/// If a regression accidentally routes unrecognised types through
/// `query_typed_raw`, the failure mode flips: instead of "cannot convert
/// `String` to `<custom_type>`" we'd see "cannot convert `String` to
/// `text`" (because we'd assert TEXT). The error-text check below catches
/// that flip.
#[sqlx::test(fixtures("base"))]
#[serial(pg_cache)]
async fn test_postgresql_prepare_fallback_for_unrecognised_arg_t(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_worker::pg_executor::clear_pg_cache;
initialize_tracing().await;
clear_pg_cache().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let db_arg = json!({"host": "localhost", "port": 5432, "dbname": "windmill", "user": "postgres", "password": "changeme"});
let make_pg_job = |content: String| {
RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Postgresql,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(
)
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
}))
.arg("database", db_arg.clone())
};
// Set up a custom enum in a dedicated schema so the test is self-contained.
let setup = r#"
DROP SCHEMA IF EXISTS fallback_test CASCADE;
CREATE SCHEMA fallback_test;
CREATE TYPE fallback_test.color AS ENUM ('red', 'green', 'blue');
"#;
make_pg_job(setup.to_owned())
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
// Inline cast `::fallback_test.color` — the parser only captures
// `fallback_test` (regex stops at the dot), which otyp_to_pg_type won't
// recognise. Dispatch must take the prepare fallback path.
//
// Thanks to the AnyTextValue ToSql/FromSql wrapper, this case now
// round-trips end-to-end (the wrapper accepts Kind::Enum on both
// directions). Pre-fix, vanilla tokio_postgres rejected String → color
// and the user had to write `CAST($1::text AS color)` as a workaround.
let result = make_pg_job("-- $1 arg1\nSELECT $1::fallback_test.color AS c".to_owned())
.arg("arg1", json!("red"))
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, json!([{"c": "red"}]));
// Reading enum columns also goes through AnyTextValue's FromSql impl on
// the result side, so the value comes back as a JSON string.
let result = make_pg_job(
r#"-- $1 arg1
SELECT $1::fallback_test.color AS c1,
'green'::fallback_test.color AS c2"#
.to_owned(),
)
.arg("arg1", json!("blue"))
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, json!([{"c1": "blue", "c2": "green"}]));
Ok(())
}
/// Regression: custom enum / domain queries must not error with `prepared
/// statement "sN" does not exist` when run on a cached connection.
///
/// Root cause: when we used `DISCARD ALL` to reset the cached connection
/// between jobs, the included `DEALLOCATE ALL` deallocated *every* prepared
/// statement server-side — including the typeinfo statements that
/// tokio_postgres caches per-client to resolve custom-type Oids. The Rust
/// client still held `Statement` objects whose names the server had
/// forgotten, so the next custom-type query failed.
///
/// Fix: switched the cached-connection probe to `RESET ALL; UNLISTEN *;
/// CLOSE ALL;` which covers windmill's session-isolation needs (GUC reset,
/// listen channels, open cursors) without nuking the prepared-statement
/// cache. This test runs the failing pattern (enum query → domain query on
/// the same cached connection, several times) to lock the behaviour in.
#[sqlx::test(fixtures("base"))]
#[serial(pg_cache)]
async fn test_postgresql_custom_types_on_cached_connection(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_worker::pg_executor::clear_pg_cache;
initialize_tracing().await;
clear_pg_cache().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let db_arg = json!({"host": "localhost", "port": 5432, "dbname": "windmill", "user": "postgres", "password": "changeme"});
let make_pg_job = |content: String| {
RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Postgresql,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(
)
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
}))
.arg("database", db_arg.clone())
};
// Set up custom enum + domain types in a dedicated schema (avoid the
// `pg_*` reserved prefix for user schemas).
let setup = r#"
DROP SCHEMA IF EXISTS wm_pg_cached_test CASCADE;
CREATE SCHEMA wm_pg_cached_test;
CREATE TYPE wm_pg_cached_test.color AS ENUM ('red', 'green', 'blue');
CREATE DOMAIN wm_pg_cached_test.short_name AS TEXT CHECK (length(VALUE) BETWEEN 1 AND 10);
"#;
make_pg_job(setup.to_owned())
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
// Run a long alternating sequence of enum + domain queries on the same
// cached connection. Each query goes through the prepare-fallback path
// (otyp_to_pg_type doesn't recognise these custom-type names) and needs
// tokio_postgres' typeinfo cache to resolve the Oids server-side. Pre-fix
// this would fail intermittently with `prepared statement "sN" does not
// exist` after the first cached-conn reuse.
for i in 0..10 {
let r = make_pg_job("-- $1 arg1\nSELECT $1::wm_pg_cached_test.color AS c".to_owned())
.arg("arg1", json!(if i % 2 == 0 { "red" } else { "blue" }))
.run_until_complete(&db, false, port)
.await;
assert!(
r.success,
"iter {i} enum query failed (was the DISCARD ALL bug); result: {:?}",
r.result
);
let r = make_pg_job("-- $1 arg1\nSELECT $1::wm_pg_cached_test.short_name AS s".to_owned())
.arg("arg1", json!(format!("v{i}")))
.run_until_complete(&db, false, port)
.await;
assert!(
r.success,
"iter {i} domain query failed (was the DISCARD ALL bug); result: {:?}",
r.result
);
}
Ok(())
}
/// Security regression: a previous job that did `SET ROLE` or `SET SESSION
/// AUTHORIZATION` to a different role must not leak that role into the next
/// job that reuses the cached connection.
///
/// This is the case `RESET ALL` alone does *not* cover — neither SET ROLE
/// nor SET SESSION AUTHORIZATION are GUC parameters, so they survive
/// `RESET ALL`. We rely on `RESET SESSION AUTHORIZATION` (which subsumes
/// `RESET ROLE`) explicitly being part of the cached-connection probe.
///
/// The pre-existing `test_postgresql_single_worker_session_isolation` test
/// only did `SET ROLE postgres` (the connecting user), so the leak was
/// invisible — this test catches it by switching to a *different* role.
#[sqlx::test(fixtures("base"))]
#[serial(pg_cache)]
async fn test_postgresql_set_role_does_not_leak_across_cached_connection(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_worker::pg_executor::clear_pg_cache;
initialize_tracing().await;
clear_pg_cache().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let db_arg = json!({"host": "localhost", "port": 5432, "dbname": "windmill", "user": "postgres", "password": "changeme"});
let make_pg_job = |content: String| {
RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Postgresql,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(
)
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
modules: None,
}))
.arg("database", db_arg.clone())
};
// Create a non-postgres role to switch to. Idempotent so the test survives
// re-runs against the same DB.
make_pg_job(
"DO $$ BEGIN \
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'wm_isolation_test_role') THEN \
CREATE ROLE wm_isolation_test_role; \
END IF; \
END $$"
.to_owned(),
)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
// Job 1: SET ROLE to a different role.
let r1 = make_pg_job(
"SET ROLE wm_isolation_test_role; SELECT current_user AS cu, session_user AS su".to_owned(),
)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(r1[0]["cu"], "wm_isolation_test_role");
assert_eq!(r1[0]["su"], "postgres");
// Job 2 (cached connection reuse): role MUST be back to the connecting
// user. Pre-fix with `RESET ALL` alone, this would still see
// `wm_isolation_test_role` because RESET ALL doesn't cover SET ROLE.
let r2 = make_pg_job("SELECT current_user AS cu, session_user AS su".to_owned())
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(
r2[0]["cu"], "postgres",
"SET ROLE leaked across cached connection: current_user is {}",
r2[0]["cu"]
);
// Job 3: SET SESSION AUTHORIZATION (changes both current_user and
// session_user — RESET ALL does NOT touch this either).
let r3 = make_pg_job(
"SET SESSION AUTHORIZATION wm_isolation_test_role; \
SELECT current_user AS cu, session_user AS su"
.to_owned(),
)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(r3[0]["cu"], "wm_isolation_test_role");
assert_eq!(r3[0]["su"], "wm_isolation_test_role");
// Job 4 (cached): both must be restored to the connecting user.
let r4 = make_pg_job("SELECT current_user AS cu, session_user AS su".to_owned())
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(
r4[0]["cu"], "postgres",
"SET SESSION AUTHORIZATION leaked across cached connection: current_user is {}",
r4[0]["cu"]
);
assert_eq!(
r4[0]["su"], "postgres",
"SET SESSION AUTHORIZATION leaked across cached connection: session_user is {}",
r4[0]["su"]
);
Ok(())
}
#[cfg(feature = "mysql")]
#[sqlx::test(fixtures("base"))]
async fn test_mysql_job(db: Pool<Postgres>) -> anyhow::Result<()> {
backend/tests/workspace_dependencies.rs
+3 -3
View File
@@ -1,12 +1,12 @@
mod workspace_dependencies {
use windmill_test_utils::in_test_worker;
use windmill_test_utils::init_client;
use windmill_test_utils::listen_for_completed_jobs;
use sqlx::{Pool, Postgres};
use tokio_stream::StreamExt;
use windmill_common::scripts::ScriptLang;
use windmill_dep_map::workspace_dependencies::NewWorkspaceDependencies;
use windmill_test_utils::in_test_worker;
use windmill_test_utils::init_client;
use windmill_test_utils::listen_for_completed_jobs;
mod deps {
pub const REQUIREMENTS_IN: &'static str = "tiny==0.1.3";
// pub const GO_MOD: &'static str = r##"
backend/windmill-worker/src/ai/providers/anthropic.rs
+1 -5
View File
@@ -371,11 +371,7 @@ pub struct AnthropicQueryBuilder {
}
impl AnthropicQueryBuilder {
pub fn new(
provider_kind: AIProvider,
platform: AIPlatform,
enable_1m_context: bool,
) -> Self {
pub fn new(provider_kind: AIProvider, platform: AIPlatform, enable_1m_context: bool) -> Self {
Self { provider_kind, platform, enable_1m_context }
}
backend/windmill-worker/src/ai/providers/google_ai.rs
+30 -11
View File
@@ -1,8 +1,8 @@
use async_trait::async_trait;
use windmill_ai::ai_google::{
openai_messages_to_gemini, openai_tools_to_gemini, GeminiGenerationConfig,
GeminiImageContent, GeminiImageRequest, GeminiImageResponse, GeminiInlineData, GeminiPart,
GeminiPredictContent, GeminiTextRequest, GeminiTool,
openai_messages_to_gemini, openai_tools_to_gemini, GeminiGenerationConfig, GeminiImageContent,
GeminiImageRequest, GeminiImageResponse, GeminiInlineData, GeminiPart, GeminiPredictContent,
GeminiTextRequest, GeminiTool,
};
use windmill_common::{client::AuthedClient, error::Error};
@@ -135,7 +135,10 @@ impl GoogleAIQueryBuilder {
openai_tools_to_gemini(tool_defs, &tool_params, has_websearch)
}
fn build_generation_config(&self, args: &BuildRequestArgs<'_>) -> Option<GeminiGenerationConfig> {
fn build_generation_config(
&self,
args: &BuildRequestArgs<'_>,
) -> Option<GeminiGenerationConfig> {
let has_output_schema = args
.output_schema
.and_then(|s| s.properties.as_ref())
@@ -145,7 +148,10 @@ impl GoogleAIQueryBuilder {
let (response_mime_type, response_schema) = if has_output_schema {
let mut schema = args.output_schema.unwrap().clone();
schema.sanitize_for_google();
(Some("application/json".to_string()), serde_json::to_value(&schema).ok())
(
Some("application/json".to_string()),
serde_json::to_value(&schema).ok(),
)
} else {
(None, None)
};
@@ -253,7 +259,11 @@ impl QueryBuilder for GoogleAIQueryBuilder {
});
Ok(ParsedResponse::Text {
content: if accumulated_content.is_empty() { None } else { Some(accumulated_content) },
content: if accumulated_content.is_empty() {
None
} else {
Some(accumulated_content)
},
tool_calls: accumulated_tool_calls.into_values().collect(),
events_str: Some(events_str),
annotations,
@@ -271,8 +281,11 @@ impl QueryBuilder for GoogleAIQueryBuilder {
format!("{}/{}:streamGenerateContent?alt=sse", base_url, model)
}
OutputType::Image => {
let url_suffix =
if model.contains("imagen") { "predict" } else { "generateContent" };
let url_suffix = if model.contains("imagen") {
"predict"
} else {
"generateContent"
};
format!("{}/{}:{}", base_url, model, url_suffix)
}
}
@@ -280,11 +293,17 @@ impl QueryBuilder for GoogleAIQueryBuilder {
// Standard Google AI: base_url is generativelanguage.googleapis.com/v1beta
match output_type {
OutputType::Text => {
format!("{}/models/{}:streamGenerateContent?alt=sse", base_url, model)
format!(
"{}/models/{}:streamGenerateContent?alt=sse",
base_url, model
)
}
OutputType::Image => {
let url_suffix =
if model.contains("imagen") { "predict" } else { "generateContent" };
let url_suffix = if model.contains("imagen") {
"predict"
} else {
"generateContent"
};
format!("{}/models/{}:{}", base_url, model, url_suffix)
}
}
backend/windmill-worker/src/ai/providers/openai.rs
+5 -8
View File
@@ -242,12 +242,10 @@ fn convert_content_to_responses_format(
image_url: image_url.url.clone(),
})
}
ContentPart::File { file } => {
Some(ImageGenerationContent::InputFile {
filename: file.filename.clone(),
file_data: file.file_data.clone(),
})
}
ContentPart::File { file } => Some(ImageGenerationContent::InputFile {
filename: file.filename.clone(),
file_data: file.file_data.clone(),
}),
// S3 objects should have been resolved earlier, but handle gracefully
ContentPart::S3Object { .. } => None,
})
@@ -433,8 +431,7 @@ impl OpenAIQueryBuilder {
if let Some(attachments) = args.attachments {
for attachment in attachments.iter() {
if !attachment.s3.is_empty() {
let part =
s3_object_to_content_part(attachment, client, workspace_id).await?;
let part = s3_object_to_content_part(attachment, client, workspace_id).await?;
match part {
ContentPart::File { file } => {
content.push(ImageGenerationContent::InputFile {
backend/windmill-worker/src/ai/query_builder.rs
+3 -3
View File
@@ -24,9 +24,9 @@ pub fn create_query_builder(provider: &ProviderWithResource) -> Box<dyn QueryBui
use windmill_ai::ai_providers::AIProvider;
match provider.kind {
AIProvider::GoogleAI => Box::new(GoogleAIQueryBuilder::new(
provider.get_platform().clone(),
)),
AIProvider::GoogleAI => {
Box::new(GoogleAIQueryBuilder::new(provider.get_platform().clone()))
}
AIProvider::OpenAI => Box::new(OpenAIQueryBuilder::new(provider.kind.clone())),
AIProvider::Anthropic => Box::new(AnthropicQueryBuilder::new(
provider.kind.clone(),
backend/windmill-worker/src/ai/sse.rs
+5 -5
View File
@@ -9,10 +9,7 @@ use windmill_ai::{
ai_google::{parse_gemini_sse_event, GeminiUsageMetadata},
ai_types::{ExtraContent, GoogleExtraContent, OpenAIFunction, OpenAIToolCall},
};
use windmill_common::{
error::Error,
utils::rd_string,
};
use windmill_common::{error::Error, utils::rd_string};
use crate::ai::{
query_builder::StreamEventSink,
@@ -502,7 +499,10 @@ impl SSEParser for GeminiSSEParser {
if let Some(text) = parsed.text {
self.accumulated_content.push_str(&text);
self.stream_event_processor
.send(StreamingEvent::TokenDelta { content: text }, &mut self.events_str)
.send(
StreamingEvent::TokenDelta { content: text },
&mut self.events_str,
)
.await?;
}
backend/windmill-worker/src/pg_executor.rs
+1846 -159
View File
File diff suppressed because it is too large Load Diff
backend/windmill-worker/src/universal_pkg_installer.rs
+11 -10
View File
@@ -390,7 +390,9 @@ pub async fn par_install_language_dependencies_seq<
_platform_agnostic: bool,
concurrent_downloads: usize,
callback: impl Fn(RequiredDependency<T>) -> Result<Command, error::Error> + Send + Sync + 'static,
post_install: Option<Arc<dyn Fn(&RequiredDependency<T>) -> anyhow::Result<()> + Send + Sync + 'static>>,
post_install: Option<
Arc<dyn Fn(&RequiredDependency<T>) -> anyhow::Result<()> + Send + Sync + 'static>,
>,
job_id: &'a Uuid,
w_id: &'a str,
worker_name: &'a str,
@@ -448,13 +450,8 @@ pub async fn par_install_language_dependencies_seq<
}
if is_layered && offset > 0 {
windmill_queue::append_logs(
job_id,
w_id,
format!("\n\n--- Layer {} ---", i + 1),
conn,
)
.await;
windmill_queue::append_logs(job_id, w_id, format!("\n\n--- Layer {} ---", i + 1), conn)
.await;
}
let layer_size = layer_deps.len();
@@ -625,7 +622,9 @@ async fn spawn_wrapped_installation_threads<
_platform_agnostic: bool,
counter_offset: Option<usize>,
total_override: Option<usize>,
post_install: Option<Arc<dyn Fn(&RequiredDependency<T>) -> anyhow::Result<()> + Send + Sync + 'static>>,
post_install: Option<
Arc<dyn Fn(&RequiredDependency<T>) -> anyhow::Result<()> + Send + Sync + 'static>,
>,
) -> anyhow::Result<(
Vec<JoinHandle<anyhow::Result<TaskKiller>>>,
tokio::sync::broadcast::Sender<()>,
@@ -772,7 +771,9 @@ async fn try_install_one_detached<'a, T: Clone + std::marker::Send + Sync + 'a +
// If dropped the entire installation fails and all installation threads are being stopped
// That's why we just pass it to return so it is not being dropped
kill_all_tasks: TaskKiller,
post_install: Option<Arc<dyn Fn(&RequiredDependency<T>) -> anyhow::Result<()> + Send + Sync + 'static>>,
post_install: Option<
Arc<dyn Fn(&RequiredDependency<T>) -> anyhow::Result<()> + Send + Sync + 'static>,
>,
) -> anyhow::Result<TaskKiller> {
let start = std::time::Instant::now();
typescript-client/sqlUtils.ts
+165 -58
View File
@@ -153,6 +153,94 @@ function ducklakeProvider(name: string): SqlProvider {
// Shared template function builder
// ---------------------------------------------------------------------------
// Build a ready-to-execute SqlStatement. Used by both the template-tag
// path (which builds `content` from strings/values) and `.query()` (which
// gets a hand-written SQL string with positional placeholders).
function buildSqlStatement(
provider: SqlProvider,
content: string,
contentBody: string,
args: Record<string, any>
): SqlStatement<any> {
async function fetch<ResultCollectionT extends ResultCollection>({
resultCollection,
}: FetchParams<ResultCollectionT> = {}) {
let finalContent = content;
if (resultCollection)
finalContent = `-- result_collection=${resultCollection}\n${finalContent}`;
try {
let result;
if (workerHasInternalServer()) {
result = await JobService.runScriptPreviewInline({
workspace: getWorkspace(),
requestBody: { args, content: finalContent, language: provider.language },
});
} else {
result = await JobService.runScriptPreviewAndWaitResult({
workspace: getWorkspace(),
requestBody: { args, content: finalContent, language: provider.language },
});
}
return result as SqlResult<any, ResultCollectionT>;
} catch (e: any) {
let err = e;
if (
e &&
typeof e.body == "string" &&
e.statusText == "Internal Server Error"
) {
let body = e.body;
if (body.startsWith("Internal:")) body = body.slice(9).trim();
if (body.startsWith("Error:")) body = body.slice(6).trim();
if (body.startsWith("datatable")) body = body.slice(9).trim();
err = Error(`${provider.providerName} ${body}`);
err.query = contentBody;
err.request = e.request;
}
throw err;
}
}
return {
content,
args,
fetch,
fetchOne: (params) =>
fetch({ ...params, resultCollection: "last_statement_first_row" }),
fetchOneScalar: (params) =>
fetch({
...params,
resultCollection: "last_statement_first_row_scalar",
}),
execute: (params) => fetch(params),
} satisfies SqlStatement<any>;
}
// JSON-encode a JS value into something the executor can deserialize. The
// JSON.stringify-friendly representation of a JS value before sending it to
// the executor:
// - `bigint` → string. JSON.stringify on a bigint throws; the
// executor accepts numeric strings into BIGINT
// slots via `Value::String → INT8`.
// - `Date` → ISO-8601 string. inferSqlType maps these to
// `TIMESTAMPTZ`; the executor's `Value::String`
// arm parses ISO strings into `chrono::DateTime`.
// - non-finite `number` → string ("NaN" / "Infinity" / "-Infinity").
// JSON.stringify renders these as `null`, which
// silently became NULL in the database. The
// executor accepts these literals via
// `Value::String → FLOAT8` (`f64::from_str`).
// - everything else → passed through unchanged.
function serializeArgValue(v: any): any {
if (typeof v === "bigint") return v.toString();
if (v instanceof Date) return v.toISOString();
if (typeof v === "number" && !Number.isFinite(v)) {
if (Number.isNaN(v)) return "NaN";
return v > 0 ? "Infinity" : "-Infinity";
}
return v;
}
function buildSqlTemplateFunction(provider: SqlProvider): SqlTemplateFunction {
let sqlFn = ((strings: TemplateStringsArray, ...values: any[]) => {
// Separate raw vs parameterized values, assigning arg indices only to params
@@ -212,62 +300,12 @@ function buildSqlTemplateFunction(provider: SqlProvider): SqlTemplateFunction {
...Object.fromEntries(
valueInfos
.filter((info): info is Extract<(typeof valueInfos)[number], { raw: false }> => !info.raw)
.map((info) => [`arg${info.argNum}`, info.value])
.map((info) => [`arg${info.argNum}`, serializeArgValue(info.value)])
),
...provider.extraArgs,
};
async function fetch<ResultCollectionT extends ResultCollection>({
resultCollection,
}: FetchParams<ResultCollectionT> = {}) {
if (resultCollection)
content = `-- result_collection=${resultCollection}\n${content}`;
try {
let result;
if (workerHasInternalServer()) {
result = await JobService.runScriptPreviewInline({
workspace: getWorkspace(),
requestBody: { args, content, language: provider.language },
});
} else {
result = await JobService.runScriptPreviewAndWaitResult({
workspace: getWorkspace(),
requestBody: { args, content, language: provider.language },
});
}
return result as SqlResult<any, ResultCollectionT>;
} catch (e: any) {
let err = e;
if (
e &&
typeof e.body == "string" &&
e.statusText == "Internal Server Error"
) {
let body = e.body;
if (body.startsWith("Internal:")) body = body.slice(9).trim();
if (body.startsWith("Error:")) body = body.slice(6).trim();
if (body.startsWith("datatable")) body = body.slice(9).trim();
err = Error(`${provider.providerName} ${body}`);
err.query = contentBody;
err.request = e.request;
}
throw err;
}
}
return {
content,
args,
fetch,
fetchOne: (params) =>
fetch({ ...params, resultCollection: "last_statement_first_row" }),
fetchOneScalar: (params) =>
fetch({
...params,
resultCollection: "last_statement_first_row_scalar",
}),
execute: (params) => fetch(params),
} satisfies SqlStatement<any>;
return buildSqlStatement(provider, content, contentBody, args);
}) as SqlTemplateFunction;
sqlFn.raw = (value: string) => new RawSql(value);
@@ -293,12 +331,33 @@ function buildSqlTemplateFunction(provider: SqlProvider): SqlTemplateFunction {
*/
export function datatable(name: string = "main"): DatatableSqlTemplateFunction {
let { name: n, schema } = parseName(name);
let sqlFn = buildSqlTemplateFunction(
datatableProvider(n, schema)
) as DatatableSqlTemplateFunction;
let provider = datatableProvider(n, schema);
let sqlFn = buildSqlTemplateFunction(provider) as DatatableSqlTemplateFunction;
// `.query(sql, ...params)` is for SQL strings that already contain
// positional placeholders ($1, $2, ...). We DON'T go through the template
// builder here — that would re-emit each value as `$N::TYPE` and append
// them after the user's literal SQL, which is the bug previous versions of
// this method shipped. Instead we build the executor-shaped content
// directly: a `-- $N argN (TYPE)` declaration block (the parser picks
// these up as explicitly typed args) followed by the user's SQL verbatim.
// Note: we hand-roll the decl format here rather than calling
// `provider.formatArgDecl`, because the datatable formatter intentionally
// omits the type (the template-tag path emits `$N::TYPE` inline instead);
// for `.query()` we have no inline cast to fall back on.
sqlFn.query = (sqlString: string, ...params: any[]) => {
let arr = Object.assign([sqlString], { raw: [sqlString] });
return sqlFn(arr, ...params);
let argDecls = params
.map((v, i) => `-- $${i + 1} arg${i + 1} (${inferSqlType(v)})`)
.join("\n");
let contentBody = sqlString;
let content =
(argDecls ? argDecls + "\n" : "") + provider.preamble() + sqlString;
let args = {
...Object.fromEntries(
params.map((v, i) => [`arg${i + 1}`, serializeArgValue(v)])
),
...provider.extraArgs,
};
return buildSqlStatement(provider, content, contentBody, args);
};
return sqlFn;
}
@@ -329,13 +388,27 @@ export function ducklake(name: string = "main"): SqlTemplateFunction {
// These types exist in both DuckDB and Postgres
// Check that the types exist if you plan to extend this function for other SQL engines.
function inferSqlType(value: any): string {
if (typeof value === "number" || typeof value === "bigint") {
if (typeof value === "bigint") return "BIGINT";
if (typeof value === "number") {
if (Number.isInteger(value)) return "BIGINT";
return "DOUBLE PRECISION";
} else if (value === null || value === undefined) {
return "TEXT";
} else if (typeof value === "string") {
return "TEXT";
} else if (Array.isArray(value)) {
// Homogeneous-primitive arrays auto-tag as `TYPE[]` so that values like
// `${[1,2,3]}` against an `int[]` column work without an explicit
// `${arr}::int[]` cast. For non-homogeneous or nested arrays we fall
// back to JSON, which works for jsonb columns.
return inferSqlArrayType(value);
} else if (value instanceof Date) {
// JS `Date` carries an absolute instant in UTC; map to TIMESTAMPTZ so
// `${someDate}` works against a `timestamptz` column without the user
// needing an explicit cast. Without this the typeof check above falls
// through to "object" → JSON, which only works by accident via
// PG's `json → text → timestamptz` implicit cast chain.
return "TIMESTAMPTZ";
} else if (typeof value === "object") {
return "JSON";
} else if (typeof value === "boolean") {
@@ -345,11 +418,45 @@ function inferSqlType(value: any): string {
}
}
function inferSqlArrayType(value: any[]): string {
if (value.length === 0) return "JSON";
// Detect a single shared scalar JS type across all elements. Mixed types
// or any non-primitive element forces the JSON fallback.
let scalarType: string | undefined = undefined;
for (const elem of value) {
let elemType: string;
if (typeof elem === "bigint") elemType = "BIGINT";
else if (typeof elem === "number")
elemType = Number.isInteger(elem) ? "BIGINT" : "DOUBLE PRECISION";
else if (typeof elem === "string") elemType = "TEXT";
else if (typeof elem === "boolean") elemType = "BOOLEAN";
else return "JSON";
if (scalarType === undefined) scalarType = elemType;
else if (scalarType === "BIGINT" && elemType === "DOUBLE PRECISION")
scalarType = "DOUBLE PRECISION";
else if (scalarType === "DOUBLE PRECISION" && elemType === "BIGINT") {
// already widened
} else if (scalarType !== elemType) {
return "JSON";
}
}
return `${scalarType}[]`;
}
// The goal is to detect if the user added a type annotation manually
//
// untyped : sql`SELECT ${x} = 0` => ['SELECT ', ' = 0']
// typed : sql`SELECT ${x}::int = 0` => ['SELECT ', '::int = 0']
// typed : sql`SELECT CAST ( ${x} AS int ) = 0` => ['SELECT CAST ( ', ' AS int ) = 0']
//
// Caveat: the returned string is only meaningful as a *presence* signal —
// the only consumer (`formatArgUsage`) just checks `explicitType !== undefined`
// to decide whether to emit `$N` (user already wrote a cast) vs `$N::TYPE`
// (SDK injects the inferred cast). The returned string itself can be
// imprecise — e.g. `${x}::DOUBLE PRECISION` returns `"DOUBLE"` (split on
// whitespace), and `CAST(${x} AS int)` returns `"int)"` (no paren stripping).
// Don't rely on the returned string as a parsed PG type; only on whether
// it's defined.
function parseTypeAnnotation(
prevTemplateString: string | undefined,
nextTemplateString: string | undefined
typescript-client/tests/sqlUtils.test.ts
+626
View File
@@ -0,0 +1,626 @@
/**
* Standalone tests for `wmill.datatable()` / `wmill.ducklake()` SQL template
* functions.
*
* The real `sqlUtils.ts` imports `./services.gen` (auto-generated, not in
* the repo) so we can't import it here. Instead we re-implement the same
* type-inference / template-building / `.query()` pipeline inline (sans the
* network calls) and assert the `content` + `args` shapes the executor
* would receive. Each test maps 1:1 to a behaviour this PR introduces or
* fixes (BigInt, homogeneous arrays, `.query()` positional, etc.) so they
* also serve as a regression backstop.
*
* Run with: bun test typescript-client/tests/sqlUtils.test.ts
*/
import { expect, test, describe } from "bun:test";
// =============================================================================
// Pure SDK logic (mirror of typescript-client/sqlUtils.ts — kept minimal,
// only the parts that decide content / args).
// =============================================================================
class RawSql {
readonly __brand = "RawSql" as const;
constructor(public readonly value: string) {}
}
interface SqlProvider {
formatArgDecl(argNum: number, argType: string): string;
formatArgUsage(
argNum: number,
explicitType: string | undefined,
inferredType: string
): string;
preamble(): string;
language: "postgresql" | "duckdb";
extraArgs: Record<string, any>;
providerName: string;
}
function datatableProvider(name: string, schema?: string): SqlProvider {
return {
providerName: "datatable",
language: "postgresql",
extraArgs: { database: `datatable://${name}` },
formatArgDecl: (argNum) => `-- $${argNum} arg${argNum}`,
formatArgUsage: (argNum, explicitType, inferredType) =>
explicitType !== undefined
? `$${argNum}`
: `$${argNum}::${inferredType}`,
preamble: () => (schema ? `SET search_path TO "${schema}";\n` : ""),
};
}
function ducklakeProvider(name: string): SqlProvider {
return {
providerName: "ducklake",
language: "duckdb",
extraArgs: {},
formatArgDecl: (argNum, argType) => `-- $arg${argNum} (${argType})`,
formatArgUsage: (argNum) => `$arg${argNum}`,
preamble: () => `ATTACH 'ducklake://${name}' AS dl;USE dl;\n`,
};
}
function inferSqlType(value: any): string {
if (typeof value === "bigint") return "BIGINT";
if (typeof value === "number") {
if (Number.isInteger(value)) return "BIGINT";
return "DOUBLE PRECISION";
} else if (value === null || value === undefined) {
return "TEXT";
} else if (typeof value === "string") {
return "TEXT";
} else if (Array.isArray(value)) {
return inferSqlArrayType(value);
} else if (value instanceof Date) {
return "TIMESTAMPTZ";
} else if (typeof value === "object") {
return "JSON";
} else if (typeof value === "boolean") {
return "BOOLEAN";
} else {
return "TEXT";
}
}
function inferSqlArrayType(value: any[]): string {
if (value.length === 0) return "JSON";
let scalarType: string | undefined = undefined;
for (const elem of value) {
let elemType: string;
if (typeof elem === "bigint") elemType = "BIGINT";
else if (typeof elem === "number")
elemType = Number.isInteger(elem) ? "BIGINT" : "DOUBLE PRECISION";
else if (typeof elem === "string") elemType = "TEXT";
else if (typeof elem === "boolean") elemType = "BOOLEAN";
else return "JSON";
if (scalarType === undefined) scalarType = elemType;
else if (scalarType === "BIGINT" && elemType === "DOUBLE PRECISION")
scalarType = "DOUBLE PRECISION";
else if (scalarType === "DOUBLE PRECISION" && elemType === "BIGINT") {
// already widened
} else if (scalarType !== elemType) {
return "JSON";
}
}
return `${scalarType}[]`;
}
function parseTypeAnnotation(
prevTemplateString: string | undefined,
nextTemplateString: string | undefined
): string | undefined {
if (!nextTemplateString) return;
nextTemplateString = nextTemplateString.trimStart();
if (nextTemplateString.startsWith("::")) {
return nextTemplateString.substring(2).trimStart().split(/\s+/)[0];
}
prevTemplateString = prevTemplateString?.trimEnd();
if (
prevTemplateString?.endsWith("(") &&
prevTemplateString
.substring(0, prevTemplateString.length - 1)
.trim()
.toUpperCase()
.endsWith("CAST") &&
nextTemplateString.toUpperCase().startsWith("AS ")
) {
return nextTemplateString.substring(2).trimStart().split(/\s+/)[0];
}
}
function serializeArgValue(v: any): any {
if (typeof v === "bigint") return v.toString();
if (v instanceof Date) return v.toISOString();
if (typeof v === "number" && !Number.isFinite(v)) {
if (Number.isNaN(v)) return "NaN";
return v > 0 ? "Infinity" : "-Infinity";
}
return v;
}
function buildContentAndArgs(
provider: SqlProvider,
strings: TemplateStringsArray | string[],
values: any[]
): { content: string; args: Record<string, any> } {
let argIndex = 0;
const valueInfos = values.map((v, i) => {
if (v instanceof RawSql)
return { raw: true as const, value: v.value, originalIndex: i };
argIndex++;
return {
raw: false as const,
value: v,
originalIndex: i,
argNum: argIndex,
};
});
let argDecls = valueInfos
.filter((info): info is Extract<typeof valueInfos[number], { raw: false }> => !info.raw)
.map((info) => {
let argType =
parseTypeAnnotation(
strings[info.originalIndex],
strings[info.originalIndex + 1]
) || inferSqlType(info.value);
return provider.formatArgDecl(info.argNum, argType);
});
let content = argDecls.length ? argDecls.join("\n") + "\n" : "";
content += provider.preamble();
let contentBody = "";
for (let i = 0; i < strings.length; i++) {
contentBody += strings[i];
if (i < valueInfos.length) {
let info = valueInfos[i];
if (info.raw) {
contentBody += info.value;
} else {
let explicitType = parseTypeAnnotation(
strings[info.originalIndex],
strings[info.originalIndex + 1]
);
let inferredType = inferSqlType(info.value);
contentBody += provider.formatArgUsage(
info.argNum,
explicitType,
inferredType
);
}
}
}
content += contentBody;
const args = {
...Object.fromEntries(
valueInfos
.filter((info): info is Extract<typeof valueInfos[number], { raw: false }> => !info.raw)
.map((info) => [`arg${info.argNum}`, serializeArgValue(info.value)])
),
...provider.extraArgs,
};
return { content, args };
}
function buildDatatableQuery(
provider: SqlProvider,
sqlString: string,
params: any[]
): { content: string; args: Record<string, any> } {
let argDecls = params
.map((v, i) => `-- $${i + 1} arg${i + 1} (${inferSqlType(v)})`)
.join("\n");
let content =
(argDecls ? argDecls + "\n" : "") + provider.preamble() + sqlString;
let args = {
...Object.fromEntries(
params.map((v, i) => [`arg${i + 1}`, serializeArgValue(v)])
),
...provider.extraArgs,
};
return { content, args };
}
function templateTag(provider: SqlProvider) {
return (strings: TemplateStringsArray, ...values: any[]) =>
buildContentAndArgs(provider, strings, values);
}
const dt = (name = "main") => templateTag(datatableProvider(name));
const dl = (name = "main") => templateTag(ducklakeProvider(name));
const datatableQuery = (name = "main") => {
const provider = datatableProvider(name);
return (sql: string, ...params: any[]) =>
buildDatatableQuery(provider, sql, params);
};
// =============================================================================
// inferSqlType — exhaustive coverage
// =============================================================================
describe("inferSqlType — primitives", () => {
test("integer Number → BIGINT", () => {
expect(inferSqlType(0)).toBe("BIGINT");
expect(inferSqlType(42)).toBe("BIGINT");
expect(inferSqlType(-7)).toBe("BIGINT");
expect(inferSqlType(Number.MAX_SAFE_INTEGER)).toBe("BIGINT");
});
test("non-integer Number → DOUBLE PRECISION", () => {
expect(inferSqlType(0.5)).toBe("DOUBLE PRECISION");
expect(inferSqlType(-3.14)).toBe("DOUBLE PRECISION");
expect(inferSqlType(Number.EPSILON)).toBe("DOUBLE PRECISION");
});
test("BigInt → BIGINT (not DOUBLE PRECISION)", () => {
// Pre-fix this branch was unreachable because bigint was bundled with
// number and `Number.isInteger(BigInt)` returns false → would have
// returned DOUBLE PRECISION (wrong). The split-out check is the fix.
expect(inferSqlType(BigInt(0))).toBe("BIGINT");
expect(inferSqlType(BigInt("9007199254740993"))).toBe("BIGINT");
expect(inferSqlType(BigInt(-1))).toBe("BIGINT");
});
test("string / null / undefined → TEXT", () => {
expect(inferSqlType("")).toBe("TEXT");
expect(inferSqlType("hello")).toBe("TEXT");
expect(inferSqlType(null)).toBe("TEXT");
expect(inferSqlType(undefined)).toBe("TEXT");
});
test("boolean → BOOLEAN", () => {
expect(inferSqlType(true)).toBe("BOOLEAN");
expect(inferSqlType(false)).toBe("BOOLEAN");
});
test("plain object → JSON", () => {
expect(inferSqlType({})).toBe("JSON");
expect(inferSqlType({ a: 1, b: [1, 2] })).toBe("JSON");
});
});
describe("inferSqlType — arrays", () => {
test("empty array → JSON", () => {
expect(inferSqlType([])).toBe("JSON");
});
test("homogeneous integer array → BIGINT[]", () => {
expect(inferSqlType([1, 2, 3])).toBe("BIGINT[]");
expect(inferSqlType([0])).toBe("BIGINT[]");
expect(inferSqlType([-1, 0, 1])).toBe("BIGINT[]");
});
test("homogeneous float array → DOUBLE PRECISION[]", () => {
expect(inferSqlType([1.5, 2.5])).toBe("DOUBLE PRECISION[]");
});
test("mixed int/float array widens to DOUBLE PRECISION[]", () => {
expect(inferSqlType([1, 2.5])).toBe("DOUBLE PRECISION[]");
expect(inferSqlType([1.5, 2])).toBe("DOUBLE PRECISION[]");
});
test("homogeneous string array → TEXT[]", () => {
expect(inferSqlType(["a", "b", "c"])).toBe("TEXT[]");
expect(inferSqlType([""])).toBe("TEXT[]");
});
test("homogeneous bool array → BOOLEAN[]", () => {
expect(inferSqlType([true, false, true])).toBe("BOOLEAN[]");
});
test("homogeneous bigint array → BIGINT[]", () => {
expect(inferSqlType([BigInt(1), BigInt(2)])).toBe("BIGINT[]");
});
test("non-homogeneous array → JSON", () => {
expect(inferSqlType([1, "x"])).toBe("JSON");
expect(inferSqlType(["a", true])).toBe("JSON");
expect(inferSqlType([1, null])).toBe("JSON");
expect(inferSqlType([true, 1])).toBe("JSON");
});
test("nested array → JSON (current limitation, no auto-tag for 2D)", () => {
expect(inferSqlType([[1], [2]])).toBe("JSON");
expect(inferSqlType([{ a: 1 }, { a: 2 }])).toBe("JSON");
});
});
// =============================================================================
// parseTypeAnnotation — used by the SDK to suppress its own ::TYPE injection
// when the user already wrote a cast.
// =============================================================================
describe("parseTypeAnnotation — user-supplied cast detection", () => {
test("`${x}::int` → 'int'", () => {
expect(parseTypeAnnotation("SELECT ", "::int FROM t")).toBe("int");
});
test("whitespace tolerance after ::", () => {
expect(parseTypeAnnotation("SELECT ", " :: bigint FROM t")).toBe(
"bigint"
);
});
test("`CAST(${x} AS int)` → first whitespace-delimited word after AS", () => {
// The SDK splits on whitespace and doesn't strip closing parens, so
// `AS int)` returns "int)". The exact value doesn't matter downstream
// because the SDK only checks `explicitType !== undefined` to skip its
// own ::cast injection — but we lock the behaviour in.
expect(parseTypeAnnotation("SELECT CAST(", " AS int)")).toBe("int)");
});
test("`CAST ( ${x} AS BOOL )` (whitespace + caps)", () => {
// Whitespace before `)` causes split to drop it, so this returns "BOOL".
expect(parseTypeAnnotation("SELECT CAST ( ", " AS BOOL )")).toBe("BOOL");
});
test("no cast adjacent → undefined", () => {
expect(parseTypeAnnotation("SELECT ", " FROM t")).toBeUndefined();
expect(parseTypeAnnotation("SELECT ", "")).toBeUndefined();
expect(parseTypeAnnotation(undefined, undefined)).toBeUndefined();
});
});
// =============================================================================
// datatable() template tag — content + args round-trips
// =============================================================================
describe("datatable() — template tag", () => {
test("primitives auto-tag with ::TYPE inline; decls have no type", () => {
const sql = dt();
const out = sql`SELECT ${42}, ${3.14}, ${true}, ${"x"}, ${null}`;
// datatable provider's formatArgDecl ignores the type, so we get bare
// decls + the casts in the SQL body.
expect(out.content).toContain("-- $1 arg1\n");
expect(out.content).toContain("$1::BIGINT");
expect(out.content).toContain("$2::DOUBLE PRECISION");
expect(out.content).toContain("$3::BOOLEAN");
expect(out.content).toContain("$4::TEXT");
expect(out.content).toContain("$5::TEXT");
expect(out.args).toMatchObject({
arg1: 42,
arg2: 3.14,
arg3: true,
arg4: "x",
arg5: null,
});
});
test("user `${x}::int` suppresses SDK's auto-cast (parser sees user's cast)", () => {
const sql = dt();
const out = sql`SELECT ${42}::int`;
expect(out.content).toContain("SELECT $1::int");
expect(out.content).not.toContain("$1::BIGINT");
});
test("CAST(${x} AS T) syntax → bare $N in SQL (regression #8988)", () => {
const sql = dt();
const out = sql`SELECT CAST(${true} AS bool)`;
expect(out.content).toContain("CAST($1 AS bool)");
expect(out.content).not.toContain("$1::BOOLEAN");
});
test("BigInt is stringified for JSON transport, tagged as ::BIGINT", () => {
const sql = dt();
const out = sql`SELECT ${BigInt("9007199254740993")}`;
expect(out.content).toContain("$1::BIGINT");
expect(out.args.arg1).toBe("9007199254740993");
// Round-trip through JSON without throwing — the original bug.
expect(() => JSON.stringify(out.args)).not.toThrow();
});
test("BigInt zero / negative / large", () => {
const sql = dt();
expect(sql`SELECT ${BigInt(0)}`.args.arg1).toBe("0");
expect(sql`SELECT ${BigInt(-1)}`.args.arg1).toBe("-1");
expect(sql`SELECT ${BigInt("99999999999999999999")}`.args.arg1).toBe(
"99999999999999999999"
);
});
test("Date is auto-tagged ::TIMESTAMPTZ and ISO-stringified", () => {
// Pre-fix: typeof Date === "object" → ::JSON, then PG cast chain
// worked accidentally for `${date}::timestamptz`. Now: explicit
// ::TIMESTAMPTZ + Date.toISOString() so plain `${date}` against a
// timestamptz column doesn't need a cast.
const sql = dt();
const d = new Date("2024-01-15T10:30:00.000Z");
const out = sql`SELECT ${d} AS t`;
expect(out.content).toContain("$1::TIMESTAMPTZ");
expect(out.args.arg1).toBe("2024-01-15T10:30:00.000Z");
expect(() => JSON.stringify(out.args)).not.toThrow();
});
test("non-finite Number is stringified for the executor", () => {
// JSON.stringify(NaN) and JSON.stringify(Infinity) both produce `null`,
// which silently became NULL in the database. The executor accepts
// "NaN" / "Infinity" / "-Infinity" via `Value::String → FLOAT8`
// (`f64::from_str`), so we send the special values as strings.
const sql = dt();
expect(sql`SELECT ${NaN}`.args.arg1).toBe("NaN");
expect(sql`SELECT ${Infinity}`.args.arg1).toBe("Infinity");
expect(sql`SELECT ${-Infinity}`.args.arg1).toBe("-Infinity");
// Tag stays DOUBLE PRECISION (these are floats).
expect(sql`SELECT ${NaN}`.content).toContain("$1::DOUBLE PRECISION");
});
test("homogeneous arrays auto-tag with TYPE[]", () => {
const sql = dt();
expect(sql`SELECT ${[1, 2, 3]}`.content).toContain("$1::BIGINT[]");
expect(sql`SELECT ${[1.5, 2.5]}`.content).toContain(
"$1::DOUBLE PRECISION[]"
);
expect(sql`SELECT ${["a", "b"]}`.content).toContain("$1::TEXT[]");
expect(sql`SELECT ${[true, false]}`.content).toContain("$1::BOOLEAN[]");
});
test("non-homogeneous and empty arrays fall back to JSON", () => {
const sql = dt();
expect(sql`SELECT ${[1, "x"]}`.content).toContain("$1::JSON");
expect(sql`SELECT ${[]}`.content).toContain("$1::JSON");
expect(sql`SELECT ${[[1], [2]]}`.content).toContain("$1::JSON");
});
test("mixed-numeric array widens to DOUBLE PRECISION[]", () => {
const sql = dt();
expect(sql`SELECT ${[1, 2.5]}`.content).toContain(
"$1::DOUBLE PRECISION[]"
);
});
test("multiple args get distinct decls + numbered placeholders", () => {
const sql = dt();
const out = sql`INSERT INTO t VALUES (${1}, ${"x"}, ${[true, false]})`;
expect(out.content).toContain("-- $1 arg1");
expect(out.content).toContain("-- $2 arg2");
expect(out.content).toContain("-- $3 arg3");
expect(out.content).toContain("$1::BIGINT");
expect(out.content).toContain("$2::TEXT");
expect(out.content).toContain("$3::BOOLEAN[]");
expect(out.args).toMatchObject({
arg1: 1,
arg2: "x",
arg3: [true, false],
});
});
test("RawSql is inlined verbatim, doesn't consume an arg index", () => {
const sql = dt();
const col = new RawSql("name");
const out = sql`SELECT ${col} FROM t WHERE id = ${42}`;
// Only one decl, only one arg in args dict.
expect(out.content.match(/^-- \$\d+/gm)?.length).toBe(1);
expect(out.content).toContain("SELECT name FROM t WHERE id = $1::BIGINT");
expect(Object.keys(out.args).filter((k) => k.startsWith("arg")).length).toBe(
1
);
expect(out.args).toMatchObject({ arg1: 42 });
});
test("schema name is propagated as SET search_path preamble", () => {
const sql = dt("main");
const out = sql`SELECT 1`;
expect(out.args.database).toBe("datatable://main");
});
test("database extra arg is always present", () => {
const sql = dt("custom_db");
const out = sql`SELECT ${1}`;
expect(out.args.database).toBe("datatable://custom_db");
});
});
// =============================================================================
// datatable().query() — positional placeholders (the previously-broken path)
// =============================================================================
describe("datatable().query() — positional placeholders", () => {
test("emits typed declarations + SQL verbatim, no appended placeholders", () => {
const q = datatableQuery();
const out = q("SELECT $1, $2", 42, "hello");
expect(out.content).toContain("-- $1 arg1 (BIGINT)");
expect(out.content).toContain("-- $2 arg2 (TEXT)");
// Crucially: SQL must end with the user's SQL, NOT have placeholders
// appended after it (the pre-fix bug).
expect(out.content.endsWith("SELECT $1, $2")).toBe(true);
expect(out.args).toMatchObject({ arg1: 42, arg2: "hello" });
});
test("BigInt args are stringified", () => {
const q = datatableQuery();
const out = q("SELECT $1", BigInt("100"));
expect(out.args.arg1).toBe("100");
expect(out.content).toContain("-- $1 arg1 (BIGINT)");
});
test("array args auto-tag homogeneously in the decl block", () => {
const q = datatableQuery();
const out = q("SELECT $1, $2", [1, 2], ["a", "b"]);
expect(out.content).toContain("-- $1 arg1 (BIGINT[])");
expect(out.content).toContain("-- $2 arg2 (TEXT[])");
});
test("zero params → no decl block, just SQL + extras", () => {
const q = datatableQuery();
const out = q("SELECT 1");
expect(out.content).not.toContain("-- $");
expect(out.content).toContain("SELECT 1");
// database extra still injected.
expect(out.args.database).toBe("datatable://main");
});
test("ten params number contiguously", () => {
const q = datatableQuery();
const params = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const out = q("SELECT " + params.map((_, i) => `$${i + 1}`).join(","), ...params);
for (let i = 1; i <= 10; i++) {
expect(out.content).toContain(`-- $${i} arg${i} (BIGINT)`);
expect(out.args[`arg${i}`]).toBe(i);
}
});
test(".query()'s decl format matches what the executor parses (regression)", () => {
// The PG parser's RE_ARG_PGSQL is:
// ^-- \$(\d+) (\w+)(?: \(([A-Za-z0-9_\[\]]+)\))?(?: ?\= ?(.+))? *$
// We assert our decl line matches that grammar so the executor doesn't
// fall back to "inferred default text".
const q = datatableQuery();
const out = q("SELECT $1", 42);
const declRe = /^-- \$\d+ \w+ \([A-Za-z0-9_\[\]]+\) *$/m;
expect(out.content).toMatch(declRe);
});
});
// =============================================================================
// ducklake() template tag — DuckDB declares types in the comment (different
// shape from datatable). Same auto-tag rules apply for inferSqlType.
// =============================================================================
describe("ducklake() — DuckDB shape", () => {
test("declarations carry the type", () => {
const sql = dl("main");
const out = sql`SELECT ${42}, ${"hello"}, ${true}`;
expect(out.content).toContain("-- $arg1 (BIGINT)");
expect(out.content).toContain("-- $arg2 (TEXT)");
expect(out.content).toContain("-- $arg3 (BOOLEAN)");
// Preamble attaches the ducklake.
expect(out.content).toContain("ATTACH 'ducklake://main' AS dl;USE dl;");
// Args are referenced via $argN syntax in the SQL body.
expect(out.content).toContain("$arg1");
});
test("BigInt + homogeneous arrays propagate to ducklake too", () => {
const sql = dl("main");
const out = sql`SELECT ${BigInt(9)}, ${[1, 2, 3]}, ${["a", "b"]}`;
expect(out.content).toContain("(BIGINT)");
expect(out.content).toContain("(BIGINT[])");
expect(out.content).toContain("(TEXT[])");
expect(out.args.arg1).toBe("9");
expect(out.args.arg2).toEqual([1, 2, 3]);
});
test("ducklake doesn't carry a database extra arg", () => {
const sql = dl();
const out = sql`SELECT 1`;
expect(out.args).not.toHaveProperty("database");
});
});
// =============================================================================
// Cross-cutting: the args dict must always be JSON-serialisable.
// =============================================================================
describe("args dict is JSON-serialisable for every supported value shape", () => {
test("BigInt, primitives, arrays, objects, raw — none throw", () => {
const sql = dt();
const out = sql`
SELECT ${BigInt(1)}, ${1}, ${1.5}, ${"x"}, ${true}, ${null},
${[1, 2]}, ${["a", "b"]}, ${[true, false]},
${{ k: 1 }}, ${[1, "x"]}
`;
const json = JSON.stringify(out.args);
expect(typeof json).toBe("string");
// BigInt got stringified, not thrown.
expect(json).toContain('"arg1":"1"');
});
});