Commit Graph
35 Commits
Author SHA1 Message Date
Ruben FiszelandClaude Opus 4.6 52a04d210f fix: preserve flow notes/groups and field ordering in generate-metadata (#8641) (#8642)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:48:22 +00:00
Ruben FiszelandClaude Opus 4.6 cc67fd9e46 refactor: move fs-backed cache under WINDMILL_DIR (#8537)
* refactor: move fs-backed cache under WINDMILL_DIR

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add WINDMILL_CACHE_PREFIX env var for per-session cache isolation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: auto-use WEBMUX_BRANCH as cache prefix for session isolation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 08:58:50 +00:00
Ruben FiszelandClaude Opus 4.6 167084a0eb feat: show groups and notes in flow status viewer (#8535)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 08:18:01 +00:00
GuilhemandClaude Opus 4.6 81eb446eee feat: flow group nodes with collapsible groups (#8075)
* feat: add flow group nodes core infrastructure

Add group data model (start_id/end_id boundary pairs), GroupEditor for
CRUD operations, groupDetectionUtils for membership computation and
validation, GroupedModulesProxy for reactive sync, and compound layout
support. Update openflow.openapi.yaml with group schema.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add group UI components and rendering

Add GroupOverlay with bounding box and z-ordering, GroupHeader with
StepCountTab and ellipsis menu, GroupNodeCard, GroupNoteArea for inline
markdown notes, CollapsedGroupNode/CollapsedSubflowNode for collapsed
rendering, GroupEndNode/GroupHeadNode boundary markers, and group
actions in NodeContextMenu and SelectionBoundingBox.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: integrate groups into flow graph, builder, and existing components

Wire group support into FlowGraphV2 (overlays, collapsed rendering,
group-aware layout), graphBuilder (GroupedModule tree, container
collapse/expand, group boundary nodes), BaseEdge (drop targets for
group operations), ModuleNode (collapsed container rendering), and
flow map components (schema item grouping). Remove SubflowBound in
favor of CollapsedSubflowNode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove banned $bindable(default) pattern and dead ternary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: decouple collapse state from grouped module tree

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: pass groups prop to FlowGraphV2 and use GroupDisplayState via graphContext

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove group membership system, compute nesting depth from visual bounds

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: simplify GroupOverlay bounds, remove unused headerY and showNotes prop

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: populate innerNodeIds for expanded subflow overlay

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove expanded subflow overlay feature for separate PR

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: flatten groups in getContainerModules to prevent crash on collapsed containers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add drag-to-move support for group nodes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: derive group boundaries from expanded membership to prevent splitting existing groups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: catch group validation errors and display as flow graph alert

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add unit tests for group validation in buildGroupedModules

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: reject virtual nodes (Input, Result, Trigger) from groups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add virtual node rejection tests for buildGroupedModules

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: exclude preprocessor and failure module from groups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: disable Create group button when preprocessor is selected

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: reject selection entirely when it contains excluded nodes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove unnecessary excludeIds from buildGroupedModules

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove debug console.log from FlowGraphV2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use cross-browser CSS grid trick for group summary input auto-sizing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide group boundary edges and reformat GroupNoteArea

Hide edges between group header and first node, and between last node
and group-end, keeping them in the DOM but visually hidden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: stop FlowGraphV2 from reading groups via groupEditorContext

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show module previews with status, selection, and suspend popover in collapsed groups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract collapsible implicit containers to separate branch

Remove collapse/expand functionality for implicit containers (forloops,
while loops, branches) from this branch. Backed up as
collapsible-implicit-containers-backup for later rebase.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: use original reactive modules for graph node data instead of proxy snapshots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prevent node loss when moving into forloop inside a group

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: replace GroupedModule proxy with structure-only FlowStructureNode tree

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use "group-" prefix for group IDs instead of "note-"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update group boundaries when renaming a module ID

When a module at a group boundary (start_id or end_id) is renamed,
the group definitions now get updated before the reactive rebuild,
preventing stale references that would break the flow structure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update graph layout when removing a group note

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add opaque background behind test run button to prevent see-through

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: detect and reject duplicate group IDs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify group creation validation with early marker normalization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use $state.raw in MiniFlowGraph to avoid xyflow performance warning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address code review feedback

- Revert backend traverse_modules change (not part of this feature)
- Use Map for node lookup in GroupOverlay (O(1) vs O(n) per group)
- Extract computeNodeExtraSpace to nodeExtraSpace.ts for testability

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review feedback

- Compute group depths from structure tree O(n) instead of O(n²) bounds comparison
- Remove unnecessary $derived(groups) in GroupOverlay
- Remove unused collapsed field from container types in OpenAPI spec
- Use NODE.width constant in GroupNodeCard instead of hardcoded 275px
- Add comment explaining intentional stale preservation in rebuild()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: preserve flow groups during dependency job re-serialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve Svelte state_referenced_locally warnings in GroupHeader and FlowGraphV2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show subflow groups when expanding a subflow in the graph

- Store both modules and groups when expanding a subflow
- Pass groups to buildStructureTree so group nodes render
- Include subflow groups in overlay rendering and collapse tracking
- Clone modules for prefix rewriting to avoid state_unsafe_mutation
- Register expanded subflow modules in moduleMap before prefix rewriting
- Disable group editing in expanded subflows and read-only views

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: restore accidentally removed code from main

- Restore subflowBound selection handling in selectionUtils
- Restore comments in SelectionBoundingBox
- Restore deletable={false} in FirstStepInputs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove redundant adjacency check from MoveManager

The disableMoveIds check already prevents all invalid drop targets,
making the adjacencySourceId/adjacencyTargetId fields unnecessary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate auto-generated files after OpenAPI schema change

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate cli skills after main merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: include groups in view_graph localStorage state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: centralize canCreateGroup and replace group note with group creation

- Add canCreateGroup StateStore to GroupEditorContext, computed in FlowGraphV2
- Replace "Create group note" with "Create group" in FlowSelectionPanel
- Remove "Add note" from selection bounding box dropdown
- Remove unused NodeContextMenu component
- Wire createGroup through FlowModuleSchemaMap → FlowGraphV2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: reject groups spanning parallel branches and surface ill-formed group errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate auto-generated files after main merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: ensure modules appears before groups in YAML export

Svelte 5's $state proxy registers groups as a tracked property before
it's explicitly set, causing it to appear before modules in Object.keys
iteration. Reorder the value object at export time for readable YAML.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address second round of PR review feedback

- Add comment explaining duplicateMultiple bypasses structure tree
- Add warning log for inverted ranges in computeGroupModuleIds
- Use NODE.width constant in CollapsedGroupNode instead of hardcoded 275px
- Simplify redundant condition in getGroupsEmptiedBy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove stored group ID, derive ephemeral key from start_id:end_id

Groups no longer store an `id` field. Instead, a `groupKey(g)` helper
derives an ephemeral key from `${start_id}:${end_id}` at read time.
This simplifies the schema while preserving all runtime functionality.

When boundaries shift (module deletion), runtime state (collapse,
note heights) is remapped to the new key via GroupDisplayState.remapGroupKey.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add note button, save/cancel hints, and rename collapsed_by_default to autocollapse

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: propagate selection from collapsed group badges to external listeners

Pass eventHandlers to GroupModuleIcons so clicking a module badge
calls both selectionManager.selectId (visual highlight) and
eventHandlers.select (side panel propagation via onSelect).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate auto-generated files after main merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide In/Out popovers and actions during click-to-move

Replace isDragging with isMoving derived that covers both drag-move
and click-move states, disabling popovers, delete button, and test
run button during any move operation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:47:33 +00:00
9643006f1e feat(cli): better stale scripts detection #3 (#8480)
* fix

Signed-off-by: pyranota <pyra@duck.com>

* reduce tests

Signed-off-by: pyranota <pyra@duck.com>

* update

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* update

Signed-off-by: pyranota <pyra@duck.com>

* WIP: stash changes after merge with origin/main

* Delete backend/parsers/windmill-parser-wasm/Cargo.lock

* reset cargo.toml

* feat(cli): integrate dependency tree into generate-metadata command

- Add isDirectlyStale field to DependencyNode for staleness tracking
- Update addScript to accept itemType, folder, isRawApp, isDirectlyStale
- Update propagateStaleness to use isDirectlyStale field instead of parameter
- Handlers now determine staleness and pass it to tree.addScript
- generate-metadata calls propagateStaleness() and populates staleItems from tree
- Pass legacyBehaviour=false and tree to handlers during generation phase

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): store originalPath in tree for correct handler invocation

Scripts need the path with extension to be passed to the handler.
Added originalPath field to DependencyNode to track this.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix parsers

Signed-off-by: pyranota <pyra@duck.com>

* rever sqlx removal

* update sqlx

* feat: make py-imports parser WASM-compatible and add as separate WASM package

Gate heavy deps (sqlx, windmill-common, async-recursion, toml, pep440_rs,
tracing) behind cfg(not(wasm32)). Make parse_code_for_imports,
parse_relative_imports, NImport, and ImportPin public. Remove duplicate
import_parser from parser-py (reset to origin/main). Add py-imports-parser
feature to windmill-parser-wasm and py-imports target to build.nu.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* safer return

* update

* fix: CLI metadata fixes - folder filter, staleness detection, WASM py-imports setup

- Fix lazy_static cfg gating for WASM compatibility (split into separate blocks)
- Fix folder argument filter to match specific file paths (not just directories)
- Fix staleness detection to use checkHash with conf (includes module hashes)
- Convert relative_imports_skip tests from Deno to bun APIs
- Add windmill-parser-wasm-py-imports to CLI and build-npm dependencies
- Relax module stale test to not require per-module change detail in output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore temp_script_refs parameter in parse_python_imports

Re-adds the temp_script_refs parameter that was lost when resetting
py-imports crate to origin/main. This enables resolving relative imports
from not-yet-deployed scripts during CLI lock generation.

* fixes

* extend testsuit

* update ee repo ref

* fix: diff endpoint bytea cast, upload only mismatched scripts

- Add POST /scripts/raw_temp/diff endpoint to batch-compare local content
  hashes against deployed versions using Postgres sha256()
- Use convert_to(content, 'UTF8') instead of content::bytea to avoid
  failure on scripts containing backslash sequences (e.g. \n)
- CLI now diffs all scripts against deployed, uploads only mismatched ones
- propagateStaleness no longer deletes non-stale nodes (needed for diff)
- Suppress verbose log.info messages during metadata generation
- Add E2E tests for locally modified and unpushed helper scripts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* rework

* sqlx

* fixes

* add index

* expand tests

* fix flows

* archive script before executing

* disable tests for ci

* skip Python-dependent E2E tests on CI

Tests requiring the python backend feature are skipped when
CI_MINIMAL_FEATURES=true since CI builds with zip-only features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: make flow fixture lock optional and reset nonDottedPaths after tests

Flow fixtures no longer emit an empty lock file by default. The lockContent
parameter controls whether a lock: "!inline ..." line appears in flow.yaml.
This prevents flows from appearing "up-to-date" when they should be processed
by generate-metadata.

Also adds afterAll to reset setNonDottedPaths(false) so global state doesn't
leak between test files when run together.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: add error logging in withTestBackend to diagnose CI failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: add --bail 1 to CI test runner to show full error on first failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: include CLI stdout/stderr in assertion message for workspace deps test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: set WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES in test backend

The workspace deps feature requires workers to report their version, but
in test/CI there are no separate workers (standalone mode). The version
check fails because workers haven't had time to ping yet. Setting this
env var bypasses the version check.

Also reverts --bail 1 from CI workflow now that the root cause is fixed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: add --bail 1 to Windows CI and assertion messages for Windows failure diagnosis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace TEMP_SCRIPT_REFS_PLACEHOLDER in bun builder tests

The loader.bun.js now includes a TEMP_SCRIPT_REFS_PLACEHOLDER that must
be replaced before execution. The builder tests were missing this
replacement, causing all 6 bun_builder_tests to fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use cdirFwd in Windows loader filterLoad regex

Raw cdir (with backslashes) interpolated into RegExp causes \r to
become carriage return and \w to become word-char, so filterLoad
never matches main.ts. This prevents replaceRelativeImports from
running, leaving bare relative imports like "./script_b" in the
bundled output, which scanImports then misparses as package ".".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Windows filterLoad regex + graceful fallback for old backends

- Fix filterLoad in loader.bun.windows.js to match both native backslash
  and forward-slash paths from Bun's resolver by escaping cdir for regex
- Wrap uploadScripts in try/catch so generate-metadata degrades gracefully
  when the backend lacks /raw_temp endpoints (locks use deployed versions)
- Add TODO for missing TEMP_SCRIPT_REFS support in Windows loader

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: add loader/builder debug logging for Windows CI diagnosis

Temporary console.log statements to understand:
- What path Bun passes to onLoad for main.ts
- Whether filterLoad regex matches
- Whether replaceRelativeImports fires
- What the bundled output contains
- What imports scanImports extracts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: trigger CI for cli path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: trigger CI via workflow file change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add TEMP_SCRIPT_REFS to Windows loader, use .ts extensions in test imports

- Add TEMP_SCRIPT_REFS_PLACEHOLDER support to loader.bun.windows.js
  (mirrors loader.bun.js) so CLI lock generation can resolve imports
  from locally-modified scripts on Windows
- Use .ts extensions in all test relative imports to work around the
  Windows filterLoad regex bug (replaceRelativeImports doesn't fire
  on Windows, so extensionless imports fail)
- Remove unused uploadSucceeded variable

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove debug logging from loader_builder.bun.js

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove windmill-parser-wasm-py-imports from frontend package.json

This dependency is only needed by the CLI, not the frontend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: add temp_script_refs logging for Windows CI investigation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: remove --bail 1 from Windows CLI tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: normalize backslashes in folder filter treePath lookup (Windows)

On Windows, item.path (originalPath) uses backslashes but tree keys
use forward slashes. The isRelevant filter's touchesFolder call
passed the unnormalized path to traverseTransitive, which couldn't
find the node. This caused cross-folder importers to be excluded
from generate-metadata when a folder argument was specified.

Also removes debug logging from previous commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update cli-tests.yml

* fix: normalize backslashes in strict-folder-boundaries warning message (Windows)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to fe8f0d1d7448464c98474d994e6492c0a45e8e38

This commit updates the EE repository reference after PR #467 was merged in windmill-ee-private.

Previous ee-repo-ref: 03e6eaf950776c96b9581848a583af9ad735be60

New ee-repo-ref: fe8f0d1d7448464c98474d994e6492c0a45e8e38

Automated by sync-ee-ref workflow.

* revert cli-tests.yml

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-23 18:20:19 +00:00
Ruben FiszelandClaude Opus 4.6 31d6660d56 feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing modules field to RawCode in bun_executor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* feat: enrich WAC templates with checkpoint and replay semantics

Add prominent comments explaining that all computation must happen
inside task/step/taskScript or it will be replayed on resume/retry.
Clarify that waitForApproval does not hold a worker and that
approve/reject URLs are available in the timeline step details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): script module sync idempotency, per-module hash tracking, and preview support

- Fix pull→push idempotency: use `??` instead of `||` for module lock
  field so empty strings are preserved (matches API's `lock: ""`)
- Add per-module hash tracking in wmill-lock.yaml following the flow
  inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes)
- Selective module lock regeneration: only regenerate locks for modules
  whose content actually changed, not all modules
- Use unfiltered rawWorkspaceDependencies for module hashes to match
  what updateModuleLocks passes to fetchScriptLock
- Show changed module names in stale script output for clarity
- Add module support to `script preview` command: read modules from
  __mod/ folder and pass them in the preview API request
- Add preview tests for taskScript pattern (flat and folder layout)
- Update test assertion for module stale detection output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation

- Reorder WAC template buttons: TypeScript before Python in
  ScriptBuilder, CreateActionsScript, and CreateActionsFlow
- Remove dropdown items from +Script button (simplify to direct link)
- Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer
- Add module tab rename: pencil icon on hover opens popover with
  validation, fixed-width icon container prevents layout shift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: remaining module-mode changes from working branch

- Backend parser updates for WAC detection
- CLI sync/types updates for raw app path and module support
- Frontend UI polish (Dev.svelte, ScriptRow, script hash page)
- Test fixture updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add test for module modification detection in generate-metadata

Verifies that modifying a single module file re-triggers stale
detection and only the changed module is listed, not all modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): critical fixes from PR review

- Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use
  "windmill-client" import instead of absolute filesystem path
- Fix missed no_main_func → auto_kind rename in parser TS test
- Add modules column to clone_script SQL (windmill-common and
  windmill-api-workspaces) so cloned scripts retain their modules
- Add modules: None to RawCode structs in worker tests
- Restore complete sqlx cache (merge main's cache + our new queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix clone warning treated as error in CI

Change `.clone()` on double reference to `*k` dereference in
scripts.rs hash implementation. Update sqlx cache with new query
hashes from modified clone_script SQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use published parser wasm versions for CI build

The local file:// paths for windmill-parser-wasm-py and
windmill-parser-wasm-ts don't exist in the Cloudflare Pages build
environment. Revert to published npm versions (1.655.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): update parser wasm packages to 1.657.2

Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py
v1.657.2 which include auto_kind/WAC detection changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): regenerate package-lock.json for npm ci compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use main's lockfile as base, update only parser wasm packages

Regenerating package-lock.json from scratch pulled different dependency
versions causing svelte-check type errors. Instead, start from main's
lockfile and only update the two changed packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): add modules column to fetch_script_for_update query

The Script<SR> struct has a modules field (FromRow), but
fetch_script_for_update didn't SELECT modules, causing a runtime
error "no column found for name: modules" when the worker processed
dependency jobs. This was the root cause of the relock_skip test
timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix script module execution for Python and Bun

- Fix modules not passed through job queue: inject _MODULES into
  PushArgs.extra when pushing Code jobs so worker can extract them
- Fix Python module imports: use relative imports (from .helper)
  and add sys.path.insert for module directory in wrapper
- Fix Python tests: use relative imports and empty lock to prevent
  pip from resolving module names as packages
- Add local file check in Bun loader for module resolution
- Ignore Bun module test (bundle mode loader integration tracked
  separately)
- Add missing modules column to fetch_script_for_update query

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): remove unnecessary empty lock in Python module tests

Relative imports (from .helper) are not parsed as pip packages,
so the empty lock workaround is not needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix module execution for Python and Bun — all tests pass

Python modules:
- Use relative imports (from .helper import greet) since scripts run
  as packages
- Add sys.path.insert for module directory in wrapper to ensure local
  modules take precedence over pip packages with same name

Bun modules:
- Use bundled output (./out/main.js) as wrapper import when modules
  are present — the bundled output has module content inlined by
  Bun.build, avoiding runtime loader resolution issues
- Add local file check in loader.bun.js onResolve to short-circuit
  API URL resolution for module files on disk

Job queue:
- Inject _MODULES into PushArgs.extra when pushing Code jobs so
  the worker can extract them at execution time

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — simplify, fix correctness, remove dead code

Critical fixes:
- Replace all CLI `no_main_func` references with `auto_kind` (string)
  to match the backend migration and API changes
- Remove duplicated `compute_python_module_dir` in worker.rs, use
  the canonical version from python_executor.rs

High priority:
- Auto-create `__init__.py` in intermediate directories for nested
  Python modules so imports like `from .utils.math import add` work
  without users manually creating __init__.py files
- Remove redundant `sys_path_insert` — relative imports use Python's
  package system, not sys.path

Medium:
- Fix lock file base name extraction: use regex to strip only the
  final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")`
  which breaks for files like `helper.test.ts`

Simplification:
- Remove dead `{#if false}` Popover block in ScriptEditor.svelte
- Guard loader.bun.js local file check to only run for relative paths
  (matching the Windows loader pattern)
- Add clarifying comment on Bun dual mechanism (build + run phases)
- Add maintenance comment on manual Hash impl for NewScript

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: final review fixes — stale cleanup, baseName, auto_kind export

- Fix sync.ts baseName extraction using indexOf(".") → regex
  (same fix as script.ts/metadata.ts, missed this instance)
- Add stale module file cleanup in writeModulesToDisk: removes files
  from __mod/ that are no longer in the modules map before writing,
  fixing the pull→push cycle that couldn't delete modules
- Log warning when _MODULES serialization fails in job push instead
  of silently dropping modules
- Use strict equality (===) for auto_kind comparison
- Exclude auto_kind from workspace export — it is auto-detected by
  the parser at deploy time from script content

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): remove auto_kind from push, comparison, and metadata

auto_kind is auto-detected by the parser at deploy time, so the CLI
should not send it, compare it, or write it to script.yaml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove erroneously added backend/backend/.sqlx directory

Duplicate .sqlx cache was committed at the wrong nested path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback + fix CI dead_code warning

Frontend (ScriptEditor.svelte):
- Fix switchToMain() missing lastSyncedCode update — prevents stale
  code sync on external changes while editing a module tab
- Fix formatAction saving module code to main script's localStorage
  draft — now saves main code when on a module tab
- Fix non-null assertion on inferModuleLang in renameModule — fall
  back to original language instead of force unwrap
- Remove redundant activeModuleTab truthy check in runTest

CLI (script.ts):
- Clean up empty directories after removing stale module files in
  writeModulesToDisk

Backend:
- Add path traversal guard in write_module_files — reject module
  paths containing ".."
- Fix dead_code warning on auto_kind field in workspace export struct

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): improve auto_kind UX + address review findings

- Rename "Include without main function" toggle to "Include library
  scripts" in script list (ItemsList.svelte)
- Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip
- Filter module file extensions by main script language — Python
  scripts only allow .py modules, TypeScript only .ts, etc.
- Split flushModuleState into flushModuleContent (no UI side-effect)
  and flushModuleState (flush + reset tab), reducing duplication
- Dynamic placeholder and hint text in add module popover based on
  main script language

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 01:20:09 +00:00
d6c72df99a fix: conditionally skip relock on dep job (#7860)
* perf: conditionally skip relock on dep job

Signed-off-by: pyranota <pyra@duck.com>

* fix

* stage

Signed-off-by: pyranota <pyra@duck.com>

* final

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* add index

Signed-off-by: pyranota <pyra@duck.com>

* fix bug

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* final (for real)

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* merge migrations into one

Signed-off-by: pyranota <pyra@duck.com>

* add test

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* remove unused

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* increase timeout

Signed-off-by: pyranota <pyra@duck.com>

* optimize the hell out of tests

Signed-off-by: pyranota <pyra@duck.com>

* simplify test

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-12 23:02:36 +00:00
Pyra df0ae90a2c fix(bun): //native not using workspace dependencies (#7833)
* fix: `//native` not using workspace dependencies

Signed-off-by: pyranota <pyra@duck.com>

* proper fix

Signed-off-by: pyranota <pyra@duck.com>

* make code cleaner

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2026-02-06 15:04:36 +00:00
Pyraandwindmill-internal-app[bot] 69c550bca6 fix(workspace-dependencies): implement better caching (#7273)
* fix(workspace-dependencies): implement better caching

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* nits

Signed-off-by: pyranota <pyra@duck.com>

* spawn non blocking

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix issue

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-02 12:48:14 +00:00
Ruben Fiszel e8fd36e2e7 fix(cli): support better esm mode for codebases 2025-11-27 17:21:04 +00:00
GuilhemandRuben Fiszel cfeb294308 feat(frontend): add notes to flow (#6628)
* Add note component

* save note size and position

* move add note button up

* nit

* Add markdown support

* wip

* fix add sticky note button

* fix text update

* Add sticky note to saved flow data

* add note color picker

* Introduce node multiselect

* Add group notes

* Adapt layout to group node

* create a note manager class

* clean reactivity

* clean

* improve adaptive layout to group note

* modify layout based on cached text height

* fined grained graph rendering for notes

* separate noteManager into editor and render

* separate noteManager into editor and render

* create a note change observer

* render note node from context

* simplify note state managment

* show note in flow viewer

* clean dirty changes

* clean selection manager

* fix layout check

* improve bg surface select

* Handle z-index for stacked group notes

* clean selection manager

* exclude notes from rect select

* Allow switch between selection modes with keyboard keys

* improve selection box styling

* prevent dragging note when editing

* nit

* Simplify selection using svelte flow built in feature

* handle note selection separately

* Add min size for notes

* improve selection toggle

* improve mode switch

* make size and position optional for group notes

* Improve initial viewport position

* Add context menu for the canevas

* nit

* Add node context menu

* improve note select

* use clickoutside for note deselect

* use pointerdown outside to close context menu

* nit

* fix selection issues

* make edges non selectable

* improve color palette

* fix backend

* fix backend check

* cargo lock restore

* Add toggle to display notes

* fix note selection

* nit

* account for css offset in for loop

* fix multiple selection pannel styling

* clear flow selection when creating note

* Improve placeholder and note default text

* Escape note edit mode when pressing Esc

* Allow note edition in local dev

* clean

* Handle subflow selection

* prevent group note resizing

* nit

* allow notes in flow expand

* Improve multi select panel

* Allow context menu in note mode

* Add event listenner to fix pane click deselect

* prevent zoom in text area in notes

* improve bounding box styling

* Use control for box selection for non mac users

* nit

* clean notes groups

* nit

* use portal for note actions

* handle assets node when computing note layout

* Simplify layout compute for notes

* use smart color choice for notes

* Switch display note when adding a new note

* clean code

* improve group note bound size calculation

* simplify AI tool nodes and asset handling

* nit

* nit

* improve flow centering

* create group note button

* Improve selection of nodes

* Revert "Improve selection of nodes"

This reverts commit d2c40d82b1.

* refert backend changes

* nit

* improve graph selection

* clean

* make backend work except job runs

* fix notSelectable

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-19 21:25:57 +00:00
Diego Imbert c079155e3e SQL result_collection flag (#7041)
* Return array of results duckdb

* Migration script to add result_collection=legacy

* migration script fixes

* app_version_lite not necessary

* New annotations macro that supports custom types

* pass unit tests

* nit style

* result_collection almost works for postgres

* fix last_statement

* frontend suggesitons

* fix column_order making columns disappear

* added version check for duckdb FFI lib to avoid crashes when changing FFI interface on Windows agent workers

* result_collection for duckdb

* Correct legacy behavior in DuckDB

* mysql result_collection

* mssql collection_strategy

* result_collection for oracle

* snowflake result_collection

* fix errors

* mistake, .clone() causing deadlock

* fix frontend errors on oracle / mssql

* fix legacy behavior on mssql

* add exception catch in migration

* missing app_script update

* change cache layout to avoid worker pulling old scripts
2025-11-12 15:32:07 +00:00
hugocasa 3b5c962473 fix(backend): revert flow node opti for ai agents (#6840)
* Revert "feat(backend): use flow nodes opti for ai agent steps (#6808)"

This reverts commit 8d5acda340.

* fix(backend): revert flow node opti for ai agents

* keep standard base64
2025-10-16 14:05:06 +00:00
hugocasa 8d5acda340 feat(backend): use flow nodes opti for ai agent steps (#6808) 2025-10-14 18:06:27 +00:00
Alexander Petricandwindmill-internal-app[bot] ae8d37fc34 feat: add dynamic skip for schedules (#6739)
* feat: add dynamic skip for schedules

* npm check

* accidental rename

* fixing tests

* Update SQLx metadata

* simplify api / queries

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-08 09:18:15 +00:00
dieriba f3fd1e90b0 feat: refactor trigger crud (#6472)
* base

* base crate

* websocket updated

* crud refactored

* fix and nits

* fix compiler warning, nits and update repo ref

* fix oss

* fix compilation

* update ref

* fix

* update feature

* listener base

* add listener

* refactor logic done and implemented for postgres

* fix capture

* websocket

* implem for all triggers

* update sqlx,repo ref and fix

* fix oss

* unify struct fix websocket

* nits and fix oss runtime axum error

* perf cache query

* update .sqlx

* update .sqlx

* fix

* fix unused

* fix

* fix

* nits and fix http handler update endpoint

* update .sqlx

* update repo ref

* nits

* fix

* update ref

* fix

* update .sqlx
2025-09-02 11:28:06 +00:00
Ruben Fiszel 97ed4a539b nits cleanup + faster script index #6450 2025-08-23 22:42:39 +00:00
+1 161dbbc6d6 feat: add native result streaming (#6242)
* feat: add stream output feature to SSE job updates

Adds stream_output field to JobUpdate struct that extracts log lines
starting with '[wm_stream]:' from job logs. Regular logs now exclude
stream lines, which are captured separately for specialized handling.

- Added stream_output: Option<String> field to JobUpdate struct
- Created extract_stream_output_from_logs() function to filter stream lines
- Modified get_job_update_data() to use stream extraction logic
- SSE clients now receive both new_logs and stream_output in job updates

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* feat: rename stream_output to stream and handle newlines as \n

- Renamed `stream_output` field to `stream` in JobUpdate struct
- Updated extract_stream_output_from_logs to extract_stream_from_logs
- Changed stream output to join with literal \n instead of actual newlines
- Stream lines are properly excluded from regular new_logs

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* decision tree nits

* push ee ref

* push ee ref

* fix: fix id renaming in apps

* remove duplicate caching (#6285)

* feat: migrate audit log ids to bigints (blocking migration for EE)

* fix(mcp): add proper check for mcp routes (#6282)

* add proper check for mcp routes

* cleaner

* apply to flow

* fix add checks scopes

---------

Co-authored-by: dieriba <dieriba.pro@gmail.com>

* chore(main): release 1.514.0 (#6283)

* chore(main): release 1.514.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: pin tokio to 1.46.1 and aws-sdks-ts

* pin rustls to 0.23.29 + pin aws-sdk

* chore(main): release 1.514.1 (#6288)

* chore(main): release 1.514.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: improve docker logs collection in docker mode

* support $res: string in form inputs of arrays

* fix import nit

* fix: fix DynSelect

* nits

* fix: resource-type-ts-parser (#6289)

* fix: resource types as arg in typescript handle imported defined types

* Update nix flake (#6291)

* merge

* Small UI fixes (#6294)

* fix step history not refreshing with staticInputs

* fix array of obj not showing up in json editor in test this step

* datatable scales correctly in DisplayResult and scrolling is much more usable

* avoid next button disapearing and changing layout / hurting ux

* nits

* fix bug when renaming module A to B then module C to A, C takes the schema of A

* fix bug with comments in sql repl

* fix aggrid theme randomly not loading

* bindable script

* better delete button in db manager

* property select doesnt exist

* fix all warnings

* delete $flowStateStore[id] on delete

* feat(cli): generate cursor rules on init (#6270)

* create cursor rules on init

* change gen

* add missing resource-type command

* add resource type command in guidance

* add schema option

* revert

* nit

* nit

* add flow guidance

* nit

* chore(main): release 1.515.0 (#6292)

* chore(main): release 1.515.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: improved logs for script

* nits logs

* chore(main): release 1.515.1 (#6295)

* chore(main): release 1.515.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* merge

* even more indexer tracings

* add more tracing logs

* feat: prevent too large results (>500Mb) from OOMing database

* nit naming

* feat: add CA certificate update at startup via environment variable (#6280)

* feat: add CA certificate update at startup via environment variable

Add support for running 'update-ca-certificates' at binary startup
when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true".

- Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup
- Execute update-ca-certificates command if env var is set to "true"
- Log success/failure appropriately with tracing
- Continue startup even if CA certificate update fails
- Non-blocking implementation with proper error handling

Fixes #6279

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: extract CA certificate update logic into separate function

Extract the CA certificate update logic from windmill_main() into a
dedicated update_ca_certificates_if_requested() function for better
code organization and maintainability.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* improvements

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>

* fix: indexer collection of job logs before indexing (#6300)

* Add flume as dependecy for indexer

* Update ee-repo-ref

* Remove flags from cargo.toml

* Update ee-repo-ref

* Update ee-repo-ref

* fix rust sdk build error (#6305)

Signed-off-by: pyranota <pyra@duck.com>

* fix broken audit logs filter (#6304)

* rename to from to

* goto fix

* default to false if field not present operator settings (#6301)

* git sync UI improvements (#6303)

* ui improvements round 1

* modal cleanup

* init

* UI refactor

* UI cleanup + refactor

* legacy cleanup

* success model -> github actions, non-ee warnings

* sqlx

* npm check

* ee warning everywhere

* last comments

* formatting

* no hardcoded theme

* claude review improvemenets

* fix: no process relative imports for scripts with codebase

* fix: sqs oidc authentication disconnect #6307

* handle metadata for new scripts happen after commit

* handle_deployment_metadata in a task

* nits

* chore: add windmill-utils-internal package (#6299)

* add utils package

* naming

* cleaning

* add docs

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* add docs

* chore(main): release 1.516.0 (#6298)

* chore(main): release 1.516.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* merge

* indexer improvements

* upgrade tantivy to 0.24.2

* use tantivy fork

* nit warnings

* fix oss build

* improve indexer

* chore: use windmill-utils-internal for cli (#6297)

* add utils package

* naming

* cleaning

* simplify assignPath

* rename old files

* same for locks

* create on confirm

* default true

* use replaceinlinescripts from utils

* use extractscriptfromflows

* make it compile

* cleaning

* use argsigtojson

* fix

* fix missing await

* cleaner

* cleaning

* cleaning

* use in frontend

* add docs

* testing

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* adapt usage

* draft

* better build script

* fix build

* revert to default creation

* add docs

* remove and rename

* make everything work

* add await

* only if not installed

* add vs code setting

* add to publish action

* fix bc

* safer use of sep

* fix

* do not rename on push

* no publish on release

* use published package on frontend

* nit

* Add dependencies to run sqlx prepare to nix flake (#6309)

* feat(cli): wmill-lock.yaml v2 for easier git merge diffs

* merge

* merge

* all

* all

* rm warnings

* fix styling on aichatinput (#6312)

* fix: use with_capacity back presusre for tantivy directory multipart writes (#6313)

* use with capacity for tantivy directory multi part uploads

* Update ee repo ref

* Update ee-repo-ref

* Update ee-repo-ref

* chore(main): release 1.517.0 (#6310)

* chore(main): release 1.517.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix typo on cli build (#6314)

* cleanup

* feat(utils): add flow.yaml validation function (#6316)

* add validateflow function

* cleaner code

* preprocess json

* cleaning

* create specific package

* cleaning

* add tests

* fix: cleanup concurrency_counter automatically + remove orphans keys automatically

* fix: add disabled support to resource picker in schema forms

* fix: add wm_labels to tracing spans

* all

* merge

* all

* fix: delete empty git connection (#6318)

* fix checks

* bun handling

* all

* all?

* all

* all

* update

* all

* update

* check

* fix history

* all

* all

* all

* Remove leftover debug tracing statements

- Remove commented debug trace in jobs.rs for stream output
- Remove commented debug trace in result_stream.rs for stream processing

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* fix test

* all

* handle iter

* fix

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: dieriba <dieriba.pro@gmail.com>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>
Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com>
2025-08-06 22:40:42 +00:00
Ruben Fiszel 60018aadf6 fix: improve agent workers for deployed scripts 2025-04-29 19:35:52 +02:00
Ruben Fiszel 63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
wendrul 6634c82e20 feat: backend arg schema validation (#5455)
* Make schema validation struct

Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).

* Make other validator builder

* Fail dependency job like with lockfile failing for schema validator

* Add last types + tests

* Remove unused dependency

* fix typos

* Migration ID was colliding with another, changed it manually

* Add Oneof + other fixes

* fix: cache for querying scripts correclty handles ScriptMetadata

* Add cache for schema validation from main arg sig

* Prepare sqlx

* Remove default features

* Feature flags

* Fix down migration table name

* cleanup: put validation logic inside a function

* Refactor to cache the should_validate boolean

Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.

Also pass the schema when avaialble to construct the schema validator

* Add other job kinds to u8 cache key just in case

* Only cache if not preview
2025-03-19 19:41:10 +01:00
wendrul 6dd2502d70 fix: cache for querying scripts correclty handles ScriptMetadata (#5466) 2025-03-11 20:04:23 +01:00
Lucas Abel d51c3080f7 backend: use v2 tables through views where possible (v2 phase 3) (#5119) 2025-02-06 12:43:24 +01:00
Ruben Fiszel 90ba65ae20 improve error messages for internal err 2025-02-05 16:32:01 +01:00
Ruben Fiszel b1155a9012 fix: fix codebase script execution 2025-01-30 19:06:55 +01:00
Ruben Fiszel c8cc9d2941 fix: improve codebase support 2025-01-30 18:25:08 +01:00
Lucas Abel f316045483 cache: clear the cache between tests (#4958)
otherwise tests aren't correctly isolated
2024-12-20 20:03:56 +09:00
Lucas Abel 83d24cb9ab cache: better cache system (#4951)
* cache: consider source as untrusted

* cache: better scoped cache

* cache: non-lazy flow value parse error

* fixup! cache: consider source as untrusted

* fixup! cache: consider source as untrusted

* fixup! cache: better scoped cache
2024-12-20 11:19:57 +01:00
Lucas Abel a040513791 cache: disable caching for tests (#4943)
As `#[sqlx::test]` spawn a database per test, the cache
could yield unexpected results
2024-12-18 09:57:31 +01:00
Lucas Abel 51b74e713c cache: improve error reporting (#4940) 2024-12-17 19:44:51 +00:00
Lucas Abel 977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
Lucas Abel 2bc4934c4f feat: add db storage for app inline scripts (#4837) 2024-12-06 13:37:45 +01:00
Lucas Abel 3c4408e3db feat(cache): refurbish fs backed cache (#4863)
* feat(cache): refurbish fs backed cache

* add `cached` exemple
2024-12-06 12:26:36 +01:00
Lucas Abel c443c2b3a8 nit: format cache.rs (#4845) 2024-12-05 11:10:30 +01:00
Lucas Abel 3fbb2bfc8a feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00