mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-21 16:02:36 +00:00
* feat: add script module mode with folder model for Bun and Python Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing modules field to RawCode in bun_executor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * feat: enrich WAC templates with checkpoint and replay semantics Add prominent comments explaining that all computation must happen inside task/step/taskScript or it will be replayed on resume/retry. Clarify that waitForApproval does not hold a worker and that approve/reject URLs are available in the timeline step details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): script module sync idempotency, per-module hash tracking, and preview support - Fix pull→push idempotency: use `??` instead of `||` for module lock field so empty strings are preserved (matches API's `lock: ""`) - Add per-module hash tracking in wmill-lock.yaml following the flow inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes) - Selective module lock regeneration: only regenerate locks for modules whose content actually changed, not all modules - Use unfiltered rawWorkspaceDependencies for module hashes to match what updateModuleLocks passes to fetchScriptLock - Show changed module names in stale script output for clarity - Add module support to `script preview` command: read modules from __mod/ folder and pass them in the preview API request - Add preview tests for taskScript pattern (flat and folder layout) - Update test assertion for module stale detection output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation - Reorder WAC template buttons: TypeScript before Python in ScriptBuilder, CreateActionsScript, and CreateActionsFlow - Remove dropdown items from +Script button (simplify to direct link) - Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer - Add module tab rename: pencil icon on hover opens popover with validation, fixed-width icon container prevents layout shift Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: remaining module-mode changes from working branch - Backend parser updates for WAC detection - CLI sync/types updates for raw app path and module support - Frontend UI polish (Dev.svelte, ScriptRow, script hash page) - Test fixture updates Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(cli): add test for module modification detection in generate-metadata Verifies that modifying a single module file re-triggers stale detection and only the changed module is listed, not all modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): critical fixes from PR review - Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use "windmill-client" import instead of absolute filesystem path - Fix missed no_main_func → auto_kind rename in parser TS test - Add modules column to clone_script SQL (windmill-common and windmill-api-workspaces) so cloned scripts retain their modules - Add modules: None to RawCode structs in worker tests - Restore complete sqlx cache (merge main's cache + our new queries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix clone warning treated as error in CI Change `.clone()` on double reference to `*k` dereference in scripts.rs hash implementation. Update sqlx cache with new query hashes from modified clone_script SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use published parser wasm versions for CI build The local file:// paths for windmill-parser-wasm-py and windmill-parser-wasm-ts don't exist in the Cloudflare Pages build environment. Revert to published npm versions (1.655.0). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): update parser wasm packages to 1.657.2 Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py v1.657.2 which include auto_kind/WAC detection changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): regenerate package-lock.json for npm ci compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use main's lockfile as base, update only parser wasm packages Regenerating package-lock.json from scratch pulled different dependency versions causing svelte-check type errors. Instead, start from main's lockfile and only update the two changed packages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): add modules column to fetch_script_for_update query The Script<SR> struct has a modules field (FromRow), but fetch_script_for_update didn't SELECT modules, causing a runtime error "no column found for name: modules" when the worker processed dependency jobs. This was the root cause of the relock_skip test timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix script module execution for Python and Bun - Fix modules not passed through job queue: inject _MODULES into PushArgs.extra when pushing Code jobs so worker can extract them - Fix Python module imports: use relative imports (from .helper) and add sys.path.insert for module directory in wrapper - Fix Python tests: use relative imports and empty lock to prevent pip from resolving module names as packages - Add local file check in Bun loader for module resolution - Ignore Bun module test (bundle mode loader integration tracked separately) - Add missing modules column to fetch_script_for_update query Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): remove unnecessary empty lock in Python module tests Relative imports (from .helper) are not parsed as pip packages, so the empty lock workaround is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix module execution for Python and Bun — all tests pass Python modules: - Use relative imports (from .helper import greet) since scripts run as packages - Add sys.path.insert for module directory in wrapper to ensure local modules take precedence over pip packages with same name Bun modules: - Use bundled output (./out/main.js) as wrapper import when modules are present — the bundled output has module content inlined by Bun.build, avoiding runtime loader resolution issues - Add local file check in loader.bun.js onResolve to short-circuit API URL resolution for module files on disk Job queue: - Inject _MODULES into PushArgs.extra when pushing Code jobs so the worker can extract them at execution time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — simplify, fix correctness, remove dead code Critical fixes: - Replace all CLI `no_main_func` references with `auto_kind` (string) to match the backend migration and API changes - Remove duplicated `compute_python_module_dir` in worker.rs, use the canonical version from python_executor.rs High priority: - Auto-create `__init__.py` in intermediate directories for nested Python modules so imports like `from .utils.math import add` work without users manually creating __init__.py files - Remove redundant `sys_path_insert` — relative imports use Python's package system, not sys.path Medium: - Fix lock file base name extraction: use regex to strip only the final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")` which breaks for files like `helper.test.ts` Simplification: - Remove dead `{#if false}` Popover block in ScriptEditor.svelte - Guard loader.bun.js local file check to only run for relative paths (matching the Windows loader pattern) - Add clarifying comment on Bun dual mechanism (build + run phases) - Add maintenance comment on manual Hash impl for NewScript Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: final review fixes — stale cleanup, baseName, auto_kind export - Fix sync.ts baseName extraction using indexOf(".") → regex (same fix as script.ts/metadata.ts, missed this instance) - Add stale module file cleanup in writeModulesToDisk: removes files from __mod/ that are no longer in the modules map before writing, fixing the pull→push cycle that couldn't delete modules - Log warning when _MODULES serialization fails in job push instead of silently dropping modules - Use strict equality (===) for auto_kind comparison - Exclude auto_kind from workspace export — it is auto-detected by the parser at deploy time from script content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove auto_kind from push, comparison, and metadata auto_kind is auto-detected by the parser at deploy time, so the CLI should not send it, compare it, or write it to script.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove erroneously added backend/backend/.sqlx directory Duplicate .sqlx cache was committed at the wrong nested path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback + fix CI dead_code warning Frontend (ScriptEditor.svelte): - Fix switchToMain() missing lastSyncedCode update — prevents stale code sync on external changes while editing a module tab - Fix formatAction saving module code to main script's localStorage draft — now saves main code when on a module tab - Fix non-null assertion on inferModuleLang in renameModule — fall back to original language instead of force unwrap - Remove redundant activeModuleTab truthy check in runTest CLI (script.ts): - Clean up empty directories after removing stale module files in writeModulesToDisk Backend: - Add path traversal guard in write_module_files — reject module paths containing ".." - Fix dead_code warning on auto_kind field in workspace export struct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): improve auto_kind UX + address review findings - Rename "Include without main function" toggle to "Include library scripts" in script list (ItemsList.svelte) - Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip - Filter module file extensions by main script language — Python scripts only allow .py modules, TypeScript only .ts, etc. - Split flushModuleState into flushModuleContent (no UI side-effect) and flushModuleState (flush + reset tab), reducing duplication - Dynamic placeholder and hint text in add module popover based on main script language Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
294 lines
8.3 KiB
Rust
294 lines
8.3 KiB
Rust
use std::collections::BTreeMap;
|
|
use std::sync::{Mutex, OnceLock};
|
|
use std::{
|
|
alloc::{self, Layout},
|
|
ffi::{c_char, c_int, c_void},
|
|
mem::align_of,
|
|
ptr,
|
|
};
|
|
use wasm_bindgen::prelude::*;
|
|
|
|
/* -------------------------------- stdlib.h -------------------------------- */
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn abort() {
|
|
panic!("Aborted from C");
|
|
}
|
|
|
|
macro_rules! console_log {
|
|
($($t:tt)*) => (unsafe { log(&format_args!($($t)*).to_string()) })
|
|
}
|
|
|
|
#[wasm_bindgen]
|
|
extern "C" {
|
|
#[wasm_bindgen(js_namespace = console)]
|
|
fn log(a: &str);
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn malloc(size: usize) -> *mut c_void {
|
|
if size == 0 {
|
|
return ptr::null_mut();
|
|
}
|
|
|
|
let (layout, offset_to_data) = layout_for_size_prepended(size);
|
|
let buf = alloc::alloc(layout);
|
|
store_layout(buf, layout, offset_to_data)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn calloc(count: usize, size: usize) -> *mut c_void {
|
|
if count == 0 || size == 0 {
|
|
return ptr::null_mut();
|
|
}
|
|
|
|
let (layout, offset_to_data) = layout_for_size_prepended(size * count);
|
|
let buf = alloc::alloc_zeroed(layout);
|
|
store_layout(buf, layout, offset_to_data)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn realloc(buf: *mut c_void, new_size: usize) -> *mut c_void {
|
|
if buf.is_null() {
|
|
malloc(new_size)
|
|
} else if new_size == 0 {
|
|
free(buf);
|
|
ptr::null_mut()
|
|
} else {
|
|
let (old_buf, old_layout) = retrieve_layout(buf);
|
|
let (new_layout, offset_to_data) = layout_for_size_prepended(new_size);
|
|
let new_buf = alloc::realloc(old_buf, old_layout, new_layout.size());
|
|
store_layout(new_buf, new_layout, offset_to_data)
|
|
}
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn free(buf: *mut c_void) {
|
|
if buf.is_null() {
|
|
return;
|
|
}
|
|
let (buf, layout) = retrieve_layout(buf);
|
|
alloc::dealloc(buf, layout);
|
|
}
|
|
|
|
// In all these allocations, we store the layout before the data for later retrieval.
|
|
// This is because we need to know the layout when deallocating the memory.
|
|
// Here are some helper methods for that:
|
|
|
|
/// Given a pointer to the data, retrieve the layout and the pointer to the layout.
|
|
unsafe fn retrieve_layout(buf: *mut c_void) -> (*mut u8, Layout) {
|
|
let (_, layout_offset) = Layout::new::<Layout>()
|
|
.extend(Layout::from_size_align(0, align_of::<*const u8>() * 2).unwrap())
|
|
.unwrap();
|
|
|
|
let buf = (buf as *mut u8).offset(-(layout_offset as isize));
|
|
let layout = *(buf as *mut Layout);
|
|
|
|
(buf, layout)
|
|
}
|
|
|
|
/// Calculate a layout for a given size with space for storing a layout at the start.
|
|
/// Returns the layout and the offset to the data.
|
|
fn layout_for_size_prepended(size: usize) -> (Layout, usize) {
|
|
Layout::new::<Layout>()
|
|
.extend(Layout::from_size_align(size, align_of::<*const u8>() * 2).unwrap())
|
|
.unwrap()
|
|
}
|
|
|
|
/// Store a layout in the pointer, returning a pointer to where the data should be stored.
|
|
unsafe fn store_layout(buf: *mut u8, layout: Layout, offset_to_data: usize) -> *mut c_void {
|
|
*(buf as *mut Layout) = layout;
|
|
(buf as *mut u8).offset(offset_to_data as isize) as *mut c_void
|
|
}
|
|
|
|
/* -------------------------------- string.h -------------------------------- */
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn strncmp(ptr1: *const c_void, ptr2: *const c_void, n: usize) -> c_int {
|
|
let s1 = std::slice::from_raw_parts(ptr1 as *const u8, n);
|
|
let s2 = std::slice::from_raw_parts(ptr2 as *const u8, n);
|
|
|
|
for (a, b) in s1.iter().zip(s2.iter()) {
|
|
if *a != *b || *a == 0 {
|
|
return (*a as i32) - (*b as i32);
|
|
}
|
|
}
|
|
|
|
0
|
|
}
|
|
|
|
// Implementation by AI:
|
|
pub type size_t = usize;
|
|
use std::slice;
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn memchr(haystack: *const c_void, needle: c_int, len: usize) -> *mut c_void {
|
|
if haystack.is_null() || len == 0 {
|
|
return ptr::null_mut(); // Return null if the input pointer is null or length is zero
|
|
}
|
|
|
|
let needle_byte = needle as u8; // Convert needle to a byte
|
|
|
|
// Create a pointer to the start of the haystack
|
|
let mut current = haystack as *const u8;
|
|
|
|
// Iterate through the memory block
|
|
for _ in 0..len {
|
|
if *current == needle_byte {
|
|
return current as *mut c_void; // Return the pointer to the found byte
|
|
}
|
|
current = current.add(1); // Move to the next byte
|
|
}
|
|
|
|
ptr::null_mut() // Return null if the byte was not found
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn strchr(mut s: *const c_char, c: c_int) -> *mut c_char {
|
|
if s.is_null() {
|
|
return std::ptr::null_mut(); // Return null if the input string is null
|
|
}
|
|
|
|
let target = c as u8 as char; // Convert c to a char
|
|
let mut current = s;
|
|
|
|
// Iterate through the string until we find the character or reach the end
|
|
while *current != 0 {
|
|
if *current as u8 as char == target {
|
|
return current as *mut c_char; // Return the pointer to the found character
|
|
}
|
|
current = current.add(1); // Move to the next character
|
|
}
|
|
|
|
std::ptr::null_mut() // Return null if the character was not found
|
|
}
|
|
// End of AI implemetation
|
|
/* -------------------------------- wctype.h -------------------------------- */
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn iswspace(c: c_int) -> bool {
|
|
char::from_u32(c as u32).map_or(false, |c| c.is_whitespace())
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn iswalnum(c: c_int) -> bool {
|
|
char::from_u32(c as u32).map_or(false, |c| c.is_alphanumeric())
|
|
}
|
|
|
|
// Implementation by AI:
|
|
pub type wint_t = u32;
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn iswdigit(wc: wint_t) -> c_int {
|
|
// Check if the character is a digit ('0' to '9')
|
|
if wc >= '0' as wint_t && wc <= '9' as wint_t {
|
|
return 1; // Return true (1)
|
|
}
|
|
0 // Return false (0)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn iswupper(wc: wint_t) -> c_int {
|
|
// Check if the character is an uppercase letter ('A' to 'Z')
|
|
if wc >= 'A' as wint_t && wc <= 'Z' as wint_t {
|
|
return 1; // Return true (1)
|
|
}
|
|
0 // Return false (0)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn iswalpha(wc: wint_t) -> c_int {
|
|
// Check if the character is an alphabetic character ('A' to 'Z' or 'a' to 'z')
|
|
if (wc >= 'A' as wint_t && wc <= 'Z' as wint_t) || (wc >= 'a' as wint_t && wc <= 'z' as wint_t)
|
|
{
|
|
return 1; // Return true (1)
|
|
}
|
|
0 // Return false (0)
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn iswlower(wc: wint_t) -> c_int {
|
|
// Check if the character is a lowercase letter ('a' to 'z')
|
|
if wc >= 'a' as wint_t && wc <= 'z' as wint_t {
|
|
return 1; // Return true (1)
|
|
}
|
|
0 // Return false (0)
|
|
}
|
|
// End of AI implemetation
|
|
|
|
/* --------------------------------- time.h --------------------------------- */
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn clock() -> u64 {
|
|
panic!("clock is not supported");
|
|
}
|
|
|
|
/* --------------------------------- ctype.h -------------------------------- */
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn isprint(c: c_int) -> bool {
|
|
c >= 32 && c <= 126
|
|
}
|
|
|
|
/* --------------------------------- stdio.h -------------------------------- */
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn fprintf(_file: *mut c_void, _format: *const c_void, _args: ...) -> c_int {
|
|
panic!("fprintf is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn fputs(_s: *const c_void, _file: *mut c_void) -> c_int {
|
|
panic!("fputs is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn fputc(_c: c_int, _file: *mut c_void) -> c_int {
|
|
panic!("fputc is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn fdopen(_fd: c_int, _mode: *const c_void) -> *mut c_void {
|
|
panic!("fdopen is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn fclose(_file: *mut c_void) -> c_int {
|
|
panic!("fclose is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn fwrite(
|
|
_ptr: *const c_void,
|
|
_size: usize,
|
|
_nmemb: usize,
|
|
_stream: *mut c_void,
|
|
) -> usize {
|
|
panic!("fwrite is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub unsafe extern "C" fn vsnprintf(
|
|
_buf: *mut c_char,
|
|
_size: usize,
|
|
_format: *const c_char,
|
|
_args: ...
|
|
) -> c_int {
|
|
panic!("vsnprintf is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn clock_gettime(ptr: usize, new_size: usize) {
|
|
panic!("clock_gettime is not supported");
|
|
}
|
|
|
|
// int snprintf( char* restrict buffer, size_t bufsz, const char* restrict format, ... );
|
|
#[no_mangle]
|
|
pub extern "C" fn snprintf() {
|
|
panic!("snprintf is not supported");
|
|
}
|
|
|
|
#[no_mangle]
|
|
pub extern "C" fn __assert_fail(_: *const i32, _: *const i32, _: *const i32, _: *const i32) {
|
|
panic!("oh no");
|
|
}
|