mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-30 16:17:25 +00:00
* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(pg): wrap encoder errors with arg context, add fallback test Followups on #8999 review: - Wrap rust-postgres "error serializing parameter N" failures with the arg name, JSON value kind, and asserted Postgres type plus a hint about an explicit cast — so users see actionable context instead of an opaque WrongType. - Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree on the Type for every recognised arg_t when the JSON value matches its natural Rust kind. Catches future drift if either side changes. - Integration test for the prepare + query_raw fallback path: confirms unrecognised arg_t (custom enum) is routed through prepare and the server-resolved type appears in the failure surface — flips into a test failure if a regression accidentally routes unrecognised types through query_typed_raw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add otyp_inferred flag + regex-based placeholder renumbering Two follow-ups from the review of #8999: 1. **Issue #1 (Number/Bool + explicit text decl in WHERE)** Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets it `true` only at the "no info → fall back to text" site (bare `$N`, no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep it `false`. In `convert_val` this flag distinguishes: - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works (`text = text` operator). Pre-#8988 behaviour, restored. - parser-default text (bare `$N`) — bind the value's natural Rust type so the regression case (`Value::Bool` against a real `bool` column via `CAST AS bool`) keeps working. `Arg` is in `windmill-parser`; the new field has `#[serde(default)]` so persisted signatures stay backward-compatible. 2. **Issue #4 ($5/$50 substring rewrite collision)** Replace the per-index `String::replace` chain (which turned `$50` into `$10` when oidx=5 was processed first) with a single regex pass. `\d+` is greedy, so `$5` and `$50` match as distinct units; indices outside the mapping are left intact. 3. Tests: - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline- cast/decl/mixed shapes. - executor unit: `convert_val_bool_against_every_arg_t` and `convert_val_*_number_*` split each text-like target into explicit vs inferred expectations. - executor unit: `renumber_sparse_placeholders_no_collision`. - integration: `test_postgresql_arg_type_combinations` adds 4 cases covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and sparse positional args ($5/$50). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality Backend: 1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s `ToSql for String` / `FromSql for String` reject `Kind::Enum` and `Kind::Domain` even though the wire format is plain UTF-8. The wrapper accepts those kinds in both directions. End result: explicit `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col` results come back as JSON strings instead of erroring at the FromSql layer. 2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text / non-temporal arg_t fell through to `Box<String> + TEXT`, which then failed at the server (no implicit cast text→numeric in expression context). Now strings are parsed into the matching native type with clear error messages on parse failure. 3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering (which fixed the `$5/$50` substring collision but still walked through string literals and comments, mangling `'price: $5'` etc.) with a walk over `parse_pg_statement_arg_positions` — the same string/comment/dollar-quote-aware tokenizer used for index discovery. Adds `parse_pg_statement_arg_positions` to the parser's public API. SDK: 4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now stringifies bigints before serialisation; the executor accepts numeric strings into BIGINT arg slots via the existing `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split so `BigInt` always resolves to `BIGINT` (was reaching `Number.isInteger(BigInt)` which returns false → wrong default). 5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers primitive types only (number / bigint / string / boolean); mixed or nested arrays still fall back to JSON. Mixed int/float widens to `DOUBLE PRECISION[]`. 6. **`.query()` positional bug**: previously the `.query()` method abused the template-tag builder, which appended `$N::TYPE` after the user's literal SQL string instead of binding by position (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()` builds the executor-shaped content directly: a `-- $N argN (TYPE)` declaration block followed by the user's SQL verbatim. Tests: - Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments` asserts string literals, comments, and dollar-quoted blocks don't produce positions (so renumbering doesn't mangle them). - Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling` uses the new position-aware path and includes string-literal + comment + `$$…$$` cases. Existing convert_val tests grow to cover new String→numeric/real/double/oid/bool arms. - Integration: `test_postgresql_arg_type_combinations` adds 13 cases (enum round-trip both directions, string→numeric/real/double/bool/oid, string-literal `$N` non-mangling). The prepare-fallback test now asserts SUCCESS (not failure) for enum encoding via AnyTextValue. - SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests) exhaustively covering inferSqlType primitives + arrays, parseTypeAnnotation, datatable() template tag (with all the new shapes — BigInt, homogeneous arrays, RawSql, schema preamble), datatable().query() positional, and ducklake() shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache) Found while exhaustively probing custom-type DX: every cached-connection reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL` deallocates *all* prepared statements server-side — including the typeinfo statements that tokio_postgres caches per-Client to resolve custom enum / domain Oids. tokio_postgres still held `Statement` objects whose names the server had forgotten, so the next custom-type query failed with intermittent "prepared statement \"sN\" does not exist" errors. The failure was easy to reproduce: any sequence that forced typeinfo lookup for two different custom-type kinds on the same cached connection (e.g. enum followed by domain) would hit it. Replace `DISCARD ALL` with a curated reset that explicitly targets the state we actually care about, *without* touching prepared statements: RESET ALL — GUC parameters (search_path, application _name, statement_timeout, …) RESET SESSION AUTHORIZATION — undoes both `SET SESSION AUTHORIZATION` and `SET ROLE` (RESET ALL does NOT — these aren't GUC parameters, so without this an elevated role from a previous job would silently leak) UNLISTEN * — drops LISTEN registrations CLOSE ALL — closes open cursors Trade-off: temp tables, advisory locks (session-scoped), and user-created PREPARE statements may persist across cached-connection reuse — rare in datatable / PG-script workloads. tokio_postgres's typeinfo cache survives intact, so custom enum / domain queries are fast on subsequent reuse. Tests: - `test_postgresql_custom_types_on_cached_connection` — runs 10× alternating enum + domain queries on a cached connection. Pre-fix this failed with `prepared statement "sN" does not exist` after the first reuse; post-fix passes. - `test_postgresql_set_role_does_not_leak_across_cached_connection` — switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres role, then runs a follow-up job and asserts current_user/session_user are restored. Specifically catches the case where someone might switch back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION AUTHORIZATION) and silently introduce a permission-leak vector. - All existing session-isolation tests (`test_postgresql_cached_connection_resets_session`, `test_postgresql_single_worker_session_isolation`, `test_postgresql_100_jobs_cached`) continue to pass. Found via end-to-end probing of datatable / PG-script DX, not previously covered: the existing isolation tests only did `SET ROLE postgres`, the connecting user, so the leak was invisible. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): address PR #8999 review (cubic + claude) cubic (P1, real bug): - `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but chrono `NaiveTime` only encodes for TIME (same caveat as the scalar arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz` assignment cast at the column site. Add an explicit unit test. claude (#1, silent failure → explicit error): - `Bool` + explicit `(char)` / `(character)` decl previously silently bound BOOL, hoping the server would cast at the use site — but PG has no implicit `bool→char` and the resulting error ("operator does not exist: bool = char") was opaque. Now error at bind time with an actionable hint to use `bool` decl or pass the value as a "t"/"f" string. claude (#2, asymmetry doc): - Object/Array still coerce to text on `matches!(typ, Typ::Str(_))` (covers both explicit AND inferred-default text), unlike Bool/Number which key on `explicit_text_target`. The asymmetry is intentional (no implicit `jsonb → text` cast in expression context vs PG having implicit `bool/int → text` casts) — added a body comment so future maintainers don't try to "align" them. claude (#3, perf): - `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions` walked the SQL tokenizer twice. Fold into a single pass that derives the index set from the position list. claude (#4, fmt drift): - `cargo fmt` over the parser crates I touched with perl scripts in the earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py, rust,graphql,yaml,r}). Net cosmetic. claude (#5, parseTypeAnnotation): - One-line caveat in the SDK's `parseTypeAnnotation` that the returned string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a real PG type, but the only consumer just checks `!== undefined`). While here — discovered + fixed independently while exhaustively probing DX: - **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing intermittent `prepared statement "sN" does not exist` errors on custom-type queries after cached-conn reuse. New regression tests: `test_postgresql_custom_types_on_cached_connection` and `test_postgresql_set_role_does_not_leak_across_cached_connection` (the latter catches the case where someone might switch back to `RESET ALL` alone and silently introduce a permission-leak vector — RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION). - **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00") and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") — neither parseable by `date-fns parseISO`, JavaScript `new Date()` is lenient enough to handle them but several frontend `App*Input.svelte` components use parseISO and fail silently. Switched to ISO-8601 with `T` separator and `+00:00` offset; arg-parsing path still accepts the legacy " UTC" suffix for back-compat. Test coverage: - 17/17 unit (`pg_executor::tests`) - 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`) - 27/27 parser (`windmill-parser-sql`) - 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling Found while probing PG-script DX with millions of numeric cells: 1. **Numeric precision-loss warning**: `numeric` results are still serialised as JSON Number (back-compat — switching to JSON String would silently break user code doing arithmetic on results), but we now detect `Decimal -> f64 -> Decimal` round-trip failure and emit a single job-log warning recommending a `::text` cast in the SQL. Bounded by `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic load + one fetch_sub on the hot path; first lossy value short-circuits to a single load thereafter). Worst-case overhead on a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic loads (vs. ~100ms unbounded). 2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"` (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is what the apps `App*Input.svelte` components use, so timestamp values silently failed to round-trip into date pickers. Switch to ISO-8601 (`T` separator + `+00:00` offset) on the result side; arg-parser continues to accept the legacy `" UTC"`-suffixed format for back-compat. 3. **Float NaN / Infinity results**: `Number::from_f64` returns None for NaN / ±Inf, which `pg_cell_to_json_value` was raising as "invalid json-float" — failing the *entire* query if any cell held one of these special values. Now serialise them as JSON strings ("NaN", "Infinity", "-Infinity") and let the rest of the row come through. Arg-side: `s.parse::<f64>()` already accepts the same strings. Tests: - `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit cases for the precision-loss predicate. - `precision_check_budget_caps_per_query_overhead` — locks in the budget cap and the loss-flag short-circuit. - All 9 PG integration tests + 17 unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults While probing PG-script DX further found three more frictions: 1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;` meant session-scoped advisory locks (`pg_advisory_lock`) leaked across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()` to the chain — `DISCARD ALL` covered this implicitly via `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it in the switch. 2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g. `-- $1 amount (numeric)`) but not provided in the args object was bound as NULL with no error / warning. Misspelling the key in the args object silently produced a row of NULLs — a notorious DX debugging trap. Now: collect the names of declared-but-missing args during dispatch and emit a single one-shot warning to the job logs at end-of-query naming each one. Bound NULL is preserved for back-compat. 3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries `arg.default = Some(Number(5))`, but the dispatch fell straight to NULL when the arg was missing. Now: respect the default — user-supplied value > declaration default > NULL. Also fixes the warning logic above (only warn for args that *don't* have a default). Tests: existing 19 unit + 9 integration pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values Two more frictions found while probing SDK end-to-end against a real datatable resource: 1. **Multi-word array types lose the [] suffix in the parser**. `transform_types_with_spaces` recognises aliases for "double precision", "character varying", "timestamp with time zone", etc. but its return type was `&'a str` — only the bare alias, never with a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at the first space, so the regex's own `(?:\[\])?` array-suffix branch sees only `"double"` (not `"double precision[]"`); the `[]` was silently lost. Result: `$1::double precision[]` (which the SDK now emits for homogeneous float arrays via the new auto-tag) routed through `Value::Array → Type::JSONB` and the server failed with "cannot cast type jsonb to double precision[]". Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>` and re-check the trailing bytes after a multi-word match. If they start with `[]`, return `format!("{alias}[]")` — Owned. Single-word types and the no-match path keep returning Borrowed slices, so no allocation in the hot path. 2. **Array arms in `convert_vec_val` rejected stringified values for numeric / int* / bool / oid / real / double**. The scalar `convert_val` already parses strings into the matching native type for these arg_ts, but the array variant only accepted JSON-native counterparts. Sending `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with "Mixed types in array". Now the array arms mirror the scalar ones — `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes round-trip cleanly. Tests: 19 unit + 9 integration pass; existing parser tests cover the multi-word array forms (the regex-cap behaviour didn't break for single-word types, and Cow plumbing is transparent to all callers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files CI failures: the perl-driven sweep that added `otyp_inferred: false` to every `Arg { ... }` literal when I introduced the field in the parser schema covered `src/lib.rs` files but missed: - parsers/windmill-parser-bash/src/lib.rs (mass-edited but a later format pass un-applied a few sites) - parsers/windmill-parser-go/src/lib.rs (same) - parsers/windmill-parser-graphql/src/lib.rs (same) - parsers/windmill-parser-nu/tests/tests.rs (test file — not swept the first time) - parsers/windmill-parser-ts/tests/tests.rs (test file — same) Also tightened the regex to handle `oidx: None` without the trailing comma (some test files had the field as the last initialiser line). `cargo build --features <CI feature combo> --workspace --all-targets` is clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string Two more frictions found while running the actual SDK end-to-end against a live datatable resource: 1. **JS `Date`** fell into the typeof "object" branch and was tagged `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's `json → text → timestamptz` implicit cast chain, but `${date}` against a `timestamptz` column without a user-supplied cast bound the value as a JSON string and the comparison `timestamptz = json` failed. Now: `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`; `serializeArgValue` emits `Date.toISOString()` so the executor's `Value::String → TIMESTAMPTZ` arm parses it cleanly. 2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)` returns `"null"` per the JS spec, so the value reached the executor as JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL double. Fix: detect non-finite numbers in `serializeArgValue` and stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals directly, and the result-side already renders the values as JSON strings (matching round-trip). SDK unit tests grow from 42 → 44 passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(pg): integration coverage for multi-word arrays + stringified array elements Locks in the two array fixes from the previous commit (`fix(pg): multi-word PG types with [] suffix lost the array-ness`) with end-to-end cases in `test_postgresql_arg_type_combinations`: - `double precision[]`, `character varying[]`, `timestamp without time zone[]` — verifies the parser keeps the `[]` suffix after multi-word alias resolution. - `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies the array arms of `convert_vec_val` apply the same string-coercion the scalar arms do. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: fix indentation drift on otyp_inferred lines cargo fmt cleanup of leftover indentation where the perl-driven sweep that introduced the otyp_inferred field landed at the wrong column. No behaviour change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
263 lines
8.7 KiB
Rust
263 lines
8.7 KiB
Rust
#![cfg_attr(target_arch = "wasm32", feature(c_variadic))]
|
|
|
|
use anyhow::{anyhow, bail};
|
|
use nu_parser::lex;
|
|
|
|
use serde_json::{json, Value};
|
|
use windmill_parser::{Arg, MainArgSignature, ObjectType, Typ};
|
|
|
|
pub fn parse_nu_signature(code: &str) -> anyhow::Result<MainArgSignature> {
|
|
let (tokens, ..) = lex(code.as_bytes(), 0, &[], &[], true);
|
|
let src = code.to_owned();
|
|
#[derive(Debug)]
|
|
enum LastToken {
|
|
None,
|
|
Def,
|
|
Main,
|
|
Args(String),
|
|
}
|
|
let mut last_token = LastToken::None;
|
|
for token in tokens {
|
|
let s = token.span;
|
|
let cont = src.get(s.start..s.end).ok_or(anyhow!("Parsing error"))?;
|
|
last_token = match last_token {
|
|
LastToken::None if cont == "def" => LastToken::Def,
|
|
LastToken::Def if cont == "main" => LastToken::Main,
|
|
LastToken::Main => {
|
|
LastToken::Args(cont.get(1..(cont.len() - 1)).unwrap_or("Error").to_owned())
|
|
}
|
|
LastToken::Args(_) => break,
|
|
_ => LastToken::None,
|
|
};
|
|
}
|
|
|
|
let LastToken::Args(args) = last_token else {
|
|
bail!("Cannot find main function.");
|
|
};
|
|
|
|
let mut sig = MainArgSignature::default();
|
|
sig.auto_kind = None;
|
|
|
|
let batches = args
|
|
.lines()
|
|
.filter_map(|el| {
|
|
if el.trim_start().starts_with('#') {
|
|
None
|
|
} else {
|
|
Some(
|
|
el.split(',')
|
|
.map(|el| el.trim())
|
|
.filter(|el| el != &"")
|
|
.collect::<Vec<&str>>(),
|
|
)
|
|
}
|
|
})
|
|
.flatten()
|
|
.collect::<Vec<&str>>();
|
|
|
|
let mut compensate_lookahead = 0;
|
|
for (i, batch) in batches.iter().enumerate() {
|
|
// parse_default can lookahead and if it does we need to compensate
|
|
// otherwise we would try to parse data already parsed but not yielded by parse_default
|
|
if compensate_lookahead > 0 {
|
|
compensate_lookahead -= 1;
|
|
continue;
|
|
}
|
|
|
|
let type_start = batch.find(":");
|
|
let default_start = batch.find("=");
|
|
|
|
let (name, typ, default) = match (type_start, default_start) {
|
|
(None, None) => (batch.trim(), None, None),
|
|
(None, Some(d)) => (
|
|
batch
|
|
.get(0..d)
|
|
.ok_or(anyhow!("Cannot parse argument ident"))?
|
|
.trim(),
|
|
None,
|
|
Some(parse_default(
|
|
&batch
|
|
.get(d..)
|
|
.ok_or(anyhow!("Cannot parse default value for argument"))?,
|
|
&batches,
|
|
i,
|
|
&mut compensate_lookahead,
|
|
)?),
|
|
),
|
|
(Some(t), None) => (
|
|
batch
|
|
.get(0..t)
|
|
.ok_or(anyhow!("Cannot parse argument ident"))?
|
|
.trim(),
|
|
Some(parse_type(
|
|
&batch
|
|
.get(t..)
|
|
.ok_or(anyhow!("Cannot parse type of argument"))?,
|
|
)?),
|
|
None,
|
|
),
|
|
(Some(t), Some(d)) => {
|
|
if t < d {
|
|
(
|
|
batch
|
|
.get(0..t)
|
|
.ok_or(anyhow!("Cannot parse argument ident"))?
|
|
.trim(),
|
|
Some(parse_type(
|
|
&batch
|
|
.get(t..d)
|
|
.ok_or(anyhow!("Cannot parse type of argument"))?,
|
|
)?),
|
|
Some(parse_default(
|
|
&batch
|
|
.get(d..)
|
|
.ok_or(anyhow!("Cannot parse default value of argument"))?,
|
|
&batches,
|
|
i,
|
|
&mut compensate_lookahead,
|
|
)?),
|
|
)
|
|
} else {
|
|
bail!("Parsing error `:` should be before `=`\nit likely means you are trying to set default value to record or table which is not supported at the moment.")
|
|
}
|
|
}
|
|
};
|
|
|
|
// Check if it is optional
|
|
let optional = {
|
|
let Some(element) = name.chars().last() else {
|
|
bail!("Internal error, cannot check if argument is optional")
|
|
};
|
|
element == '?'
|
|
};
|
|
|
|
// Rest parameters are not supported
|
|
if matches!(name.get(0..3), Some("...")) {
|
|
bail!("Rest (...) parameters are not supported")
|
|
}
|
|
|
|
// Flags are not supported
|
|
if matches!(name.get(0..2), Some("--")) {
|
|
bail!("Flags are not supported")
|
|
}
|
|
|
|
sig.args.push(Arg {
|
|
name: if optional {
|
|
name.get(..name.len() - 1).unwrap_or("Error").to_owned()
|
|
} else {
|
|
name.to_owned()
|
|
},
|
|
typ: typ.unwrap_or(Typ::Unknown),
|
|
otyp: None,
|
|
has_default: default.is_some() || optional,
|
|
default: default.or_else(|| if optional { Some(json!(null)) } else { None }),
|
|
oidx: None,
|
|
otyp_inferred: false,
|
|
});
|
|
}
|
|
|
|
fn parse_type(content: &str) -> anyhow::Result<Typ> {
|
|
let c = content.replace(":", "").trim().to_owned();
|
|
let typ = match c.as_str() {
|
|
"string" => Typ::Str(None),
|
|
"int" => Typ::Int,
|
|
"float" => Typ::Float,
|
|
"number" => Typ::Float,
|
|
"record" => Typ::Object(ObjectType::new(None, Some(vec![]))),
|
|
"table" => Typ::List(Box::new(Typ::Object(ObjectType::new(None, Some(vec![]))))),
|
|
"nothing" => Typ::Unknown,
|
|
// TODO: needs additional work on literal parsing
|
|
// "binary" => Typ::Bytes,
|
|
"datetime" => Typ::Datetime,
|
|
"any" => Typ::Unknown,
|
|
"bool" => Typ::Bool,
|
|
// Lists
|
|
"list" | "list<any>" | "list<nothing>" => Typ::List(Box::new(Typ::Unknown)),
|
|
"list<number>" => Typ::List(Box::new(Typ::Float)),
|
|
"list<bool>" => Typ::List(Box::new(Typ::Bool)),
|
|
"list<string>" => Typ::List(Box::new(Typ::Str(None))),
|
|
// list<float/int> is not supported
|
|
// Records and Tables
|
|
// TODO: Support in V1?
|
|
s if s.contains("record<") => {
|
|
bail!("typed records are not supported, use `ident: record`")
|
|
}
|
|
s if s.contains("table<") => {
|
|
bail!("typed tables are not supported, use `ident: table`")
|
|
}
|
|
s => bail!("{s} is not supported"),
|
|
};
|
|
Ok(typ)
|
|
}
|
|
fn parse_default(
|
|
content: &str,
|
|
ctx: &[&str],
|
|
i: usize,
|
|
skip: &mut usize,
|
|
) -> anyhow::Result<Value> {
|
|
let mut c = content.replace("=", "").trim().to_owned();
|
|
|
|
fn parse_object_literal(
|
|
(open, close): (char, char),
|
|
mut c_2: String,
|
|
ctx: &[&str],
|
|
i: usize,
|
|
skip: &mut usize,
|
|
) -> anyhow::Result<String> {
|
|
// It is list
|
|
// if c.contains("[") {
|
|
let mut closed = false;
|
|
if c_2 != open.to_string() {
|
|
// [a ~ , ~ ...
|
|
// Add ^
|
|
c_2 += ",";
|
|
}
|
|
// else {
|
|
// [
|
|
// a < Do not add ","
|
|
// ...
|
|
// }
|
|
let remainder = &ctx
|
|
.iter()
|
|
.skip(i + 1)
|
|
.map_while(|el| {
|
|
let el = el.trim();
|
|
|
|
if closed {
|
|
None
|
|
} else {
|
|
if el.chars().last() == Some(close) {
|
|
closed = true;
|
|
}
|
|
*skip += 1;
|
|
Some(el)
|
|
}
|
|
})
|
|
.collect::<Vec<&str>>()
|
|
.join(",");
|
|
|
|
if remainder.contains(&['{', '[']) {
|
|
bail!("Nesting is not supported")
|
|
}
|
|
|
|
Ok((c_2 + remainder)
|
|
// Remove trailing comma if there is any
|
|
.replace(&format!(",{close}"), &close.to_string()))
|
|
}
|
|
// It is list
|
|
if c.contains("[") {
|
|
if c.chars().last() != Some(']') {
|
|
c = parse_object_literal(('[', ']'), c.clone(), ctx, i, skip)?;
|
|
}
|
|
}
|
|
// It is record
|
|
if c.contains("{") {
|
|
if c.chars().last() != Some('}') {
|
|
c = parse_object_literal(('{', '}'), c.clone(), ctx, i, skip)?;
|
|
}
|
|
}
|
|
Ok(serde_json::from_str(&c)?)
|
|
}
|
|
Ok(sig)
|
|
}
|