feat: runner groups for shared-process multi-script dedicated workers (#8434)

* feat: add runner groups for shared-process multi-script dedicated workers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: unify dedicated worker and runner group wrappers into single multi-script wrapper

Replace per-language single-script wrappers with the unified load/exec/exec_preprocess/end
protocol. Each start_worker() now writes scripts to scripts/<safe_name>/ and uses
generate_multi_script_wrapper(). handle_dedicated_process() sends load: on start and
exec: per job instead of raw JSON args.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: merge runner groups into dedicated workers with inline arg metadata

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to match EE branch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate EE-only functions behind cfg(feature = "private") to fix OSS dead_code errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: auto-detect runner groups from workspace dependency annotations

- New endpoint GET /scripts/list_dedicated_with_deps: returns dedicated
  scripts with parsed workspace dependency names from content annotations
- Frontend: show dep badges in DedicatedWorkersSelector with links to
  workspace settings, warn when referenced dep doesn't exist, group
  scripts sharing deps into "Shared runner" sections
- Remove manual "Runner groups" tab and RunnerGroupSelector component
- Remove runner_groups from WorkerConfigOpt/WorkerConfig (auto-detected)
- Fix Node.js single dedicated workers: transpile main.ts -> main.js via
  Bun.build so the multi-script wrapper's dynamic import() works under Node
- Add package.json with type:module in scripts dir to silence Node warning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: unify dedicated worker wrappers with baked-in codegen and routing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add e2e tests for multi-script dedicated worker routing (bun, deno, python)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dead generate_dedicated_worker_wrapper function

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add dependency installation to runner groups + make dep functions pub(crate)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prevent bun loader from intercepting absolute paths within cwd

When a plugin's onResolve returns an absolute path, Bun re-invokes
the resolver with that path. The loader was then routing it through
the remote URL resolver, breaking runner group script imports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use _wm_ prefix for runner group scripts to avoid bun loader interception

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract DENO_UNSTABLE_ARGS constant to avoid repeating flags

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate private-only exports behind cfg(feature = "private") for OSS build

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move format strings before handle_dedicated_process to fix lifetime

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate sqlx offline cache

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix sqlx

* fix: skip empty lines in deno e2e tests (double newline from console.log + '\n')

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use dict() instead of {{}} in python wrapper to avoid set literal

{{{{}}}} in format!() produces {{}} which Python interprets as an
empty set, not a dict. Use dict() which is unambiguous.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove deno from runner groups and associated tests

Deno resolves dependencies at runtime via URLs/import maps, so there's
no shared node_modules/pip install to benefit from runner groups.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: revert deno wrapper to inline old-style with exec: protocol

Since deno doesn't support runner groups, the unified multi-script
wrapper is unnecessary. Reverted to the old inline wrapper from main
but adapted to use the exec:<path>:<args> protocol.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract deno wrapper into reusable function and add e2e tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use codebase presence (not nodejs annotation) to determine wrapper import extension

On main, codebase scripts import ./main.js (pre-bundled JS).
The wrapper_ext was incorrectly based on annotation.nodejs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: improve dedicated workers UI - combine lists, better badges, tooltips

- Merge shared runners section with selected tags into one unified list
- Move language tag to right side of selector for alignment
- Change dep badge color from dark-gray to indigo
- Add tooltip on yellow warning badge explaining missing workspace dep

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: group shared runners visually in dedicated workers list

- Runner groups shown with a header (Shared runner · language · dep badge)
- Scripts in the same group nested under the header
- Standalone scripts/flows shown after groups
- Used Svelte snippet for reusable tag row rendering

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve visual separation between shared runner groups and standalone items

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: give standalone runners same header style as shared runners

- Each standalone script/flow gets its own header row with bg-surface-secondary
- Header shows "Dedicated runner" / "Flow runner" label, dep link, language badge
- Shared runner header: swapped language and dep badge positions
- Dep shown as inline link instead of badge in headers for cleaner look

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: inline standalone runner path in header, language badge on right edge, no max height

- Standalone items: path shown directly in header row (no sub-row)
- Language badge placed after flex-1 spacer (right-aligned)
- Removed max-h-64 overflow constraint from the list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: consistent badges across runner list - dep+language on right, depBadge snippet

- Shared runner scripts: show (workspace) and language badge on right
- Standalone items: dep badges and language badge on right (after flex-1)
- Shared runner header: dep badge and language badge on right
- Extract depBadge snippet to deduplicate dep badge rendering
- Picker selector also uses depBadge snippet

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show language badge on standalone items, hide from shared runner sub-items

- Fetch script language from API when not available from workspace deps
- Hide dep+language badges from tagRow when script is inside a runner group
  (already shown in the group header)
- Standalone items now always show language badge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: differentiate badge colors - gray for language, indigo for workspace deps

Matches codebase convention: gray for metadata (like script hashes),
indigo for linkable features/entities.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use transparent (bordered) badge for language - visible on all backgrounds

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use gray badge for language everywhere

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert skills.ts and AI files, add _wm_ exclusion to Windows loader

- Revert cli/src/guidance/skills.ts to main (not our change)
- Revert AI provider formatting changes (not our change)
- Add _wm_ prefix exclusion to loader.bun.windows.js filterResolve

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update ee-repo-ref and regenerate system prompts after merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: use DISTINCT ON in list_dedicated_with_deps to dedup at DB level

Avoids fetching all script versions and deduplicating in Rust.
Addresses PR review feedback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use sqlx query! macro for list_dedicated_with_deps and regenerate cache

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker review fixes and test coverage

- Fix Python relative imports in dedicated workers (write loader.py, add
  import loader to wrapper when needed)
- Move Python colon parsing inside try/except to prevent crashes on
  malformed stdin
- Add indexOf guard in Bun/Deno wrappers for malformed protocol messages
- Add stderr logging for unrecognized stdin commands in all wrappers
- Remove asyncio handling from Python wrapper (consistent with normal path)
- Add exec_preprocess protocol tests for Bun, Deno, and Python
- Add argument transformation tests (dates, bytes, kwargs, sentinel)
- Add relative import detection test for Python wrapper
- Add PreprocessedArgs variant to DedicatedWorkerResult test helper

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove symlink from git and gate has_relative_imports behind private feature

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update ee-repo-ref for dedicated_worker_ee.rs changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add mixed exec+preprocess test to use ProtocolCmd::Exec variant

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove hanging deno missing-preprocessor test

The Deno wrapper only generates the exec_preprocess handler when the
script has a preprocessor function. Without one, the message is
unrecognized and the test hangs reading stdout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 182943e5ad9bf2a905ccdf07d4e346437fb329a9

This commit updates the EE repository reference after PR #466 was merged in windmill-ee-private.

Previous ee-repo-ref: 995f701fe3754be6260fc6b679e5de8fc636e68a

New ee-repo-ref: 182943e5ad9bf2a905ccdf07d4e346437fb329a9

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

backend/windmill-api-scripts/src/scripts.rs

@@ -239,6 +239,7 @@ pub fn workspaced_service() -> Router {
             "/history_update/h/:hash/p/*path",
             post(update_script_history),
         )
+        .route("/list_dedicated_with_deps", get(list_dedicated_with_deps))
         // Temporary raw script storage for CLI lock generation
         .route("/raw_temp/store", post(store_raw_script_temp))
         .route("/raw_temp/diff", post(diff_raw_scripts_with_deployed))
@@ -2480,6 +2481,62 @@ async fn guard_script_from_debounce_data(ns: &NewScript) -> Result<()> {
     }
 }
 
+#[derive(Serialize)]
+struct DedicatedScriptDeps {
+    path: String,
+    language: ScriptLang,
+    workspace_dep_names: Vec<String>,
+}
+
+async fn list_dedicated_with_deps(
+    authed: ApiAuthed,
+    Extension(user_db): Extension<UserDB>,
+    Path(w_id): Path<String>,
+) -> JsonResult<Vec<DedicatedScriptDeps>> {
+    let mut tx = user_db.begin(&authed).await?;
+
+    let rows = sqlx::query!(
+        "SELECT DISTINCT ON (path) path, language AS \"language: ScriptLang\", content FROM script
+         WHERE workspace_id = $1
+           AND archived = false
+           AND dedicated_worker = true
+           AND language = ANY($2::SCRIPT_LANG[])
+         ORDER BY path, created_at DESC",
+        &w_id,
+        &[
+            ScriptLang::Python3,
+            ScriptLang::Bun,
+            ScriptLang::Bunnative,
+            ScriptLang::Deno,
+        ] as &[ScriptLang],
+    )
+    .fetch_all(&mut *tx)
+    .await?;
+
+    tx.commit().await?;
+
+    let result = rows
+        .into_iter()
+        .map(|row| {
+            let dep_names =
+                windmill_common::scripts::extract_workspace_dependencies_annotated_refs(
+                    &row.language,
+                    &row.content,
+                    &row.path,
+                )
+                .map(|refs| refs.external)
+                .unwrap_or_default();
+            DedicatedScriptDeps {
+                path: row.path,
+                language: row.language,
+                workspace_dep_names: dep_names,
+            }
+        })
+        .collect();
+
+    Ok(Json(result))
+}
+
 // ============================================================================
 // Temporary Raw Script Storage for CLI Lock Generation
 // ============================================================================
@@ -2508,11 +2565,9 @@ async fn store_raw_script_temp(
     .await?;
 
     // Clean up old entries (1 week TTL)
-    sqlx::query!(
-        "DELETE FROM raw_script_temp WHERE created_at < NOW() - INTERVAL '1 week'"
-    )
-    .execute(&db)
-    .await?;
+    sqlx::query!("DELETE FROM raw_script_temp WHERE created_at < NOW() - INTERVAL '1 week'")
+        .execute(&db)
+        .await?;
 
     Ok(Json(hash))
 }
@@ -2560,7 +2615,7 @@ async fn diff_raw_scripts_with_deployed(
                FROM script s \
                WHERE s.path = local.path AND s.workspace_id = $3 AND s.archived = false \
                ORDER BY s.created_at DESC LIMIT 1 \
-             ) deployed ON deployed.deployed_hash = local.hash"
+             ) deployed ON deployed.deployed_hash = local.hash",
         )
         .bind(&paths)
         .bind(&hashes)
@@ -2582,7 +2637,7 @@ async fn diff_raw_scripts_with_deployed(
                  AND wd.language = $3::SCRIPT_LANG \
                  AND wd.name IS NOT DISTINCT FROM $4 \
                  AND encode(sha256(convert_to(wd.content, 'UTF8')), 'hex') = $5 \
-             )"
+             )",
         )
         .bind(&dep.path)
         .bind(&w_id)