Files
Ruben FiszelandClaude Opus 4.6 31d6660d56 feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing modules field to RawCode in bun_executor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* feat: enrich WAC templates with checkpoint and replay semantics

Add prominent comments explaining that all computation must happen
inside task/step/taskScript or it will be replayed on resume/retry.
Clarify that waitForApproval does not hold a worker and that
approve/reject URLs are available in the timeline step details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): script module sync idempotency, per-module hash tracking, and preview support

- Fix pull→push idempotency: use `??` instead of `||` for module lock
  field so empty strings are preserved (matches API's `lock: ""`)
- Add per-module hash tracking in wmill-lock.yaml following the flow
  inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes)
- Selective module lock regeneration: only regenerate locks for modules
  whose content actually changed, not all modules
- Use unfiltered rawWorkspaceDependencies for module hashes to match
  what updateModuleLocks passes to fetchScriptLock
- Show changed module names in stale script output for clarity
- Add module support to `script preview` command: read modules from
  __mod/ folder and pass them in the preview API request
- Add preview tests for taskScript pattern (flat and folder layout)
- Update test assertion for module stale detection output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation

- Reorder WAC template buttons: TypeScript before Python in
  ScriptBuilder, CreateActionsScript, and CreateActionsFlow
- Remove dropdown items from +Script button (simplify to direct link)
- Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer
- Add module tab rename: pencil icon on hover opens popover with
  validation, fixed-width icon container prevents layout shift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: remaining module-mode changes from working branch

- Backend parser updates for WAC detection
- CLI sync/types updates for raw app path and module support
- Frontend UI polish (Dev.svelte, ScriptRow, script hash page)
- Test fixture updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add test for module modification detection in generate-metadata

Verifies that modifying a single module file re-triggers stale
detection and only the changed module is listed, not all modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): critical fixes from PR review

- Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use
  "windmill-client" import instead of absolute filesystem path
- Fix missed no_main_func → auto_kind rename in parser TS test
- Add modules column to clone_script SQL (windmill-common and
  windmill-api-workspaces) so cloned scripts retain their modules
- Add modules: None to RawCode structs in worker tests
- Restore complete sqlx cache (merge main's cache + our new queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix clone warning treated as error in CI

Change `.clone()` on double reference to `*k` dereference in
scripts.rs hash implementation. Update sqlx cache with new query
hashes from modified clone_script SQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use published parser wasm versions for CI build

The local file:// paths for windmill-parser-wasm-py and
windmill-parser-wasm-ts don't exist in the Cloudflare Pages build
environment. Revert to published npm versions (1.655.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): update parser wasm packages to 1.657.2

Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py
v1.657.2 which include auto_kind/WAC detection changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): regenerate package-lock.json for npm ci compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use main's lockfile as base, update only parser wasm packages

Regenerating package-lock.json from scratch pulled different dependency
versions causing svelte-check type errors. Instead, start from main's
lockfile and only update the two changed packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): add modules column to fetch_script_for_update query

The Script<SR> struct has a modules field (FromRow), but
fetch_script_for_update didn't SELECT modules, causing a runtime
error "no column found for name: modules" when the worker processed
dependency jobs. This was the root cause of the relock_skip test
timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix script module execution for Python and Bun

- Fix modules not passed through job queue: inject _MODULES into
  PushArgs.extra when pushing Code jobs so worker can extract them
- Fix Python module imports: use relative imports (from .helper)
  and add sys.path.insert for module directory in wrapper
- Fix Python tests: use relative imports and empty lock to prevent
  pip from resolving module names as packages
- Add local file check in Bun loader for module resolution
- Ignore Bun module test (bundle mode loader integration tracked
  separately)
- Add missing modules column to fetch_script_for_update query

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): remove unnecessary empty lock in Python module tests

Relative imports (from .helper) are not parsed as pip packages,
so the empty lock workaround is not needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix module execution for Python and Bun — all tests pass

Python modules:
- Use relative imports (from .helper import greet) since scripts run
  as packages
- Add sys.path.insert for module directory in wrapper to ensure local
  modules take precedence over pip packages with same name

Bun modules:
- Use bundled output (./out/main.js) as wrapper import when modules
  are present — the bundled output has module content inlined by
  Bun.build, avoiding runtime loader resolution issues
- Add local file check in loader.bun.js onResolve to short-circuit
  API URL resolution for module files on disk

Job queue:
- Inject _MODULES into PushArgs.extra when pushing Code jobs so
  the worker can extract them at execution time

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — simplify, fix correctness, remove dead code

Critical fixes:
- Replace all CLI `no_main_func` references with `auto_kind` (string)
  to match the backend migration and API changes
- Remove duplicated `compute_python_module_dir` in worker.rs, use
  the canonical version from python_executor.rs

High priority:
- Auto-create `__init__.py` in intermediate directories for nested
  Python modules so imports like `from .utils.math import add` work
  without users manually creating __init__.py files
- Remove redundant `sys_path_insert` — relative imports use Python's
  package system, not sys.path

Medium:
- Fix lock file base name extraction: use regex to strip only the
  final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")`
  which breaks for files like `helper.test.ts`

Simplification:
- Remove dead `{#if false}` Popover block in ScriptEditor.svelte
- Guard loader.bun.js local file check to only run for relative paths
  (matching the Windows loader pattern)
- Add clarifying comment on Bun dual mechanism (build + run phases)
- Add maintenance comment on manual Hash impl for NewScript

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: final review fixes — stale cleanup, baseName, auto_kind export

- Fix sync.ts baseName extraction using indexOf(".") → regex
  (same fix as script.ts/metadata.ts, missed this instance)
- Add stale module file cleanup in writeModulesToDisk: removes files
  from __mod/ that are no longer in the modules map before writing,
  fixing the pull→push cycle that couldn't delete modules
- Log warning when _MODULES serialization fails in job push instead
  of silently dropping modules
- Use strict equality (===) for auto_kind comparison
- Exclude auto_kind from workspace export — it is auto-detected by
  the parser at deploy time from script content

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): remove auto_kind from push, comparison, and metadata

auto_kind is auto-detected by the parser at deploy time, so the CLI
should not send it, compare it, or write it to script.yaml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove erroneously added backend/backend/.sqlx directory

Duplicate .sqlx cache was committed at the wrong nested path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback + fix CI dead_code warning

Frontend (ScriptEditor.svelte):
- Fix switchToMain() missing lastSyncedCode update — prevents stale
  code sync on external changes while editing a module tab
- Fix formatAction saving module code to main script's localStorage
  draft — now saves main code when on a module tab
- Fix non-null assertion on inferModuleLang in renameModule — fall
  back to original language instead of force unwrap
- Remove redundant activeModuleTab truthy check in runTest

CLI (script.ts):
- Clean up empty directories after removing stale module files in
  writeModulesToDisk

Backend:
- Add path traversal guard in write_module_files — reject module
  paths containing ".."
- Fix dead_code warning on auto_kind field in workspace export struct

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): improve auto_kind UX + address review findings

- Rename "Include without main function" toggle to "Include library
  scripts" in script list (ItemsList.svelte)
- Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip
- Filter module file extensions by main script language — Python
  scripts only allow .py modules, TypeScript only .ts, etc.
- Split flushModuleState into flushModuleContent (no UI side-effect)
  and flushModuleState (flush + reset tab), reducing duplication
- Dynamic placeholder and hint text in add module popover based on
  main script language

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 01:20:09 +00:00

262 lines
8.7 KiB
Rust

#![cfg_attr(target_arch = "wasm32", feature(c_variadic))]
use anyhow::{anyhow, bail};
use nu_parser::lex;
use serde_json::{json, Value};
use windmill_parser::{Arg, MainArgSignature, ObjectType, Typ};
pub fn parse_nu_signature(code: &str) -> anyhow::Result<MainArgSignature> {
let (tokens, ..) = lex(code.as_bytes(), 0, &[], &[], true);
let src = code.to_owned();
#[derive(Debug)]
enum LastToken {
None,
Def,
Main,
Args(String),
}
let mut last_token = LastToken::None;
for token in tokens {
let s = token.span;
let cont = src.get(s.start..s.end).ok_or(anyhow!("Parsing error"))?;
last_token = match last_token {
LastToken::None if cont == "def" => LastToken::Def,
LastToken::Def if cont == "main" => LastToken::Main,
LastToken::Main => {
LastToken::Args(cont.get(1..(cont.len() - 1)).unwrap_or("Error").to_owned())
}
LastToken::Args(_) => break,
_ => LastToken::None,
};
}
let LastToken::Args(args) = last_token else {
bail!("Cannot find main function.");
};
let mut sig = MainArgSignature::default();
sig.auto_kind = None;
let batches = args
.lines()
.filter_map(|el| {
if el.trim_start().starts_with('#') {
None
} else {
Some(
el.split(',')
.map(|el| el.trim())
.filter(|el| el != &"")
.collect::<Vec<&str>>(),
)
}
})
.flatten()
.collect::<Vec<&str>>();
let mut compensate_lookahead = 0;
for (i, batch) in batches.iter().enumerate() {
// parse_default can lookahead and if it does we need to compensate
// otherwise we would try to parse data already parsed but not yielded by parse_default
if compensate_lookahead > 0 {
compensate_lookahead -= 1;
continue;
}
let type_start = batch.find(":");
let default_start = batch.find("=");
let (name, typ, default) = match (type_start, default_start) {
(None, None) => (batch.trim(), None, None),
(None, Some(d)) => (
batch
.get(0..d)
.ok_or(anyhow!("Cannot parse argument ident"))?
.trim(),
None,
Some(parse_default(
&batch
.get(d..)
.ok_or(anyhow!("Cannot parse default value for argument"))?,
&batches,
i,
&mut compensate_lookahead,
)?),
),
(Some(t), None) => (
batch
.get(0..t)
.ok_or(anyhow!("Cannot parse argument ident"))?
.trim(),
Some(parse_type(
&batch
.get(t..)
.ok_or(anyhow!("Cannot parse type of argument"))?,
)?),
None,
),
(Some(t), Some(d)) => {
if t < d {
(
batch
.get(0..t)
.ok_or(anyhow!("Cannot parse argument ident"))?
.trim(),
Some(parse_type(
&batch
.get(t..d)
.ok_or(anyhow!("Cannot parse type of argument"))?,
)?),
Some(parse_default(
&batch
.get(d..)
.ok_or(anyhow!("Cannot parse default value of argument"))?,
&batches,
i,
&mut compensate_lookahead,
)?),
)
} else {
bail!("Parsing error `:` should be before `=`\nit likely means you are trying to set default value to record or table which is not supported at the moment.")
}
}
};
// Check if it is optional
let optional = {
let Some(element) = name.chars().last() else {
bail!("Internal error, cannot check if argument is optional")
};
element == '?'
};
// Rest parameters are not supported
if matches!(name.get(0..3), Some("...")) {
bail!("Rest (...) parameters are not supported")
}
// Flags are not supported
if matches!(name.get(0..2), Some("--")) {
bail!("Flags are not supported")
}
sig.args.push(Arg {
name: if optional {
name.get(..name.len() - 1).unwrap_or("Error").to_owned()
} else {
name.to_owned()
},
typ: typ.unwrap_or(Typ::Unknown),
otyp: None,
has_default: default.is_some() || optional,
default: default.or_else(|| if optional { Some(json!(null)) } else { None }),
oidx: None,
});
}
fn parse_type(content: &str) -> anyhow::Result<Typ> {
let c = content.replace(":", "").trim().to_owned();
let typ = match c.as_str() {
"string" => Typ::Str(None),
"int" => Typ::Int,
"float" => Typ::Float,
"number" => Typ::Float,
"record" => Typ::Object(ObjectType::new(None, Some(vec![]))),
"table" => Typ::List(Box::new(Typ::Object(ObjectType::new(None, Some(vec![]))))),
"nothing" => Typ::Unknown,
// TODO: needs additional work on literal parsing
// "binary" => Typ::Bytes,
"datetime" => Typ::Datetime,
"any" => Typ::Unknown,
"bool" => Typ::Bool,
// Lists
"list" | "list<any>" | "list<nothing>" => Typ::List(Box::new(Typ::Unknown)),
"list<number>" => Typ::List(Box::new(Typ::Float)),
"list<bool>" => Typ::List(Box::new(Typ::Bool)),
"list<string>" => Typ::List(Box::new(Typ::Str(None))),
// list<float/int> is not supported
// Records and Tables
// TODO: Support in V1?
s if s.contains("record<") => {
bail!("typed records are not supported, use `ident: record`")
}
s if s.contains("table<") => {
bail!("typed tables are not supported, use `ident: table`")
}
s => bail!("{s} is not supported"),
};
Ok(typ)
}
fn parse_default(
content: &str,
ctx: &[&str],
i: usize,
skip: &mut usize,
) -> anyhow::Result<Value> {
let mut c = content.replace("=", "").trim().to_owned();
fn parse_object_literal(
(open, close): (char, char),
mut c_2: String,
ctx: &[&str],
i: usize,
skip: &mut usize,
) -> anyhow::Result<String> {
// It is list
// if c.contains("[") {
let mut closed = false;
if c_2 != open.to_string() {
// [a ~ , ~ ...
// Add ^
c_2 += ",";
}
// else {
// [
// a < Do not add ","
// ...
// }
let remainder = &ctx
.iter()
.skip(i + 1)
.map_while(|el| {
let el = el.trim();
if closed {
None
} else {
if el.chars().last() == Some(close) {
closed = true;
}
*skip += 1;
Some(el)
}
})
.collect::<Vec<&str>>()
.join(",");
if remainder.contains(&['{', '[']) {
bail!("Nesting is not supported")
}
Ok((c_2 + remainder)
// Remove trailing comma if there is any
.replace(&format!(",{close}"), &close.to_string()))
}
// It is list
if c.contains("[") {
if c.chars().last() != Some(']') {
c = parse_object_literal(('[', ']'), c.clone(), ctx, i, skip)?;
}
}
// It is record
if c.contains("{") {
if c.chars().last() != Some('}') {
c = parse_object_literal(('{', '}'), c.clone(), ctx, i, skip)?;
}
}
Ok(serde_json::from_str(&c)?)
}
Ok(sig)
}