Compare commits

...
Author SHA1 Message Date
Ruben Fiszelandrubenfiszel 5b338bb749 chore(main): release 1.533.0 (#6448)
* chore(main): release 1.533.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-23 02:53:29 +01:00
centdix d948ff5d0d nits(aichat): add keyboard navigation in context list (#6443)
* better flow module code peice

* better keyboard nav for availablecontextlist

* cleaning

* escape to close + cleaning

* nit tab handling

* fix module extraction

* remove code category

* fixes

* comment

* fix

* fix tool params display
2025-08-23 02:50:17 +01:00
Alexander Petricandellipsis-dev[bot] a41b9e47e2 feat: CLI improvements (#6446)
* feat: branch specific items for cli

* error on wmill.yaml parsing errors

* also search for wmill.yaml in parent dirs when git

* git_branches -> gitBranches

* Update cli/src/core/specific_items.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* sanitize branch name (regex + fs path)

* improve sanitatino

* robust relative paths

* hubpath

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-23 02:45:00 +01:00
hugocasa c13747cda9 fix(frontend): ai agent flow status + UI nits (#6447)
* fix(frontend): ai agent flow status

* nit: prevent undefined node issue

* feat: UI nits + flow status select iter fix

* nit ai agent color in picker
2025-08-22 18:51:13 +01:00
Ruben Fiszel 964351e211 update monaco-vscode-api (#6445) 2025-08-22 11:54:26 +00:00
Ruben Fiszelandrubenfiszel 2046b64ec8 chore(main): release 1.532.0 (#6439)
* chore(main): release 1.532.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-22 11:05:42 +01:00
hugocasa 7da79a8bc5 feat: json schema resource (#6433)
* feat: json schema resource

* feat: cache

* fix cleanup

* fix: use format instead of custom property
2025-08-22 10:39:13 +01:00
Diego Imbert 4d8777b278 Fix read undefined when renaming flow step (#6440) 2025-08-22 09:24:43 +00:00
centdix 73272f16fd feat(aichat): allow adding contexts to flow mode (#6424)
* add new feature instructions

* add db as context for flow mode

* add diff

* cleaner diff

* add modules as available context

* convert to svelte 5

* auto add selected module to context

* change flowinline ai button + nits

* handle adding selected lines

* clean context handling

* apply code pieces

* new chat when changing mode

* clean

* show code for code steps

* add last saved flow

* fix size

* categorize context

* optionnaly categorize

* fix module finding

* logs

* nit prompt

* fix

* fix

* fix test tool for script

* clean
2025-08-22 08:46:03 +00:00
Ruben Fiszelandrubenfiszel ee5e39a3d5 chore(main): release 1.531.0 (#6429)
* chore(main): release 1.531.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-22 08:01:58 +00:00
Ruben Fiszel 9df008b9f8 fix: s3 result presigned not working with list 2025-08-22 07:56:02 +00:00
73 changed files with 3419 additions and 2007 deletions
+36
View File
@@ -1,5 +1,41 @@
# Changelog
## [1.533.0](https://github.com/windmill-labs/windmill/compare/v1.532.0...v1.533.0) (2025-08-23)
### Features
* CLI improvements ([#6446](https://github.com/windmill-labs/windmill/issues/6446)) ([a41b9e4](https://github.com/windmill-labs/windmill/commit/a41b9e47e233ebaa2baafb5cca1187bb85d6f8f4))
### Bug Fixes
* **frontend:** ai agent flow status + UI nits ([#6447](https://github.com/windmill-labs/windmill/issues/6447)) ([c13747c](https://github.com/windmill-labs/windmill/commit/c13747cda9449369288e8d078b60542ea79a49bf))
## [1.532.0](https://github.com/windmill-labs/windmill/compare/v1.531.0...v1.532.0) (2025-08-22)
### Features
* **aichat:** allow adding contexts to flow mode ([#6424](https://github.com/windmill-labs/windmill/issues/6424)) ([73272f1](https://github.com/windmill-labs/windmill/commit/73272f16fddc355703b04f2c3458520753d1e19c))
* json schema resource ([#6433](https://github.com/windmill-labs/windmill/issues/6433)) ([7da79a8](https://github.com/windmill-labs/windmill/commit/7da79a8bc525fc6b89748ad0af25c2bac4ca2ef3))
## [1.531.0](https://github.com/windmill-labs/windmill/compare/v1.530.0...v1.531.0) (2025-08-22)
### Features
* ai agent steps ([#6393](https://github.com/windmill-labs/windmill/issues/6393)) ([958e8af](https://github.com/windmill-labs/windmill/commit/958e8af78290cf859f98c45c012ed41e3bada39e))
* bump Go version from 1.22.0 to 1.25.0 [#6415](https://github.com/windmill-labs/windmill/issues/6415) ([c92bfe6](https://github.com/windmill-labs/windmill/commit/c92bfe6601fd96f6d74860f52f9307e02961ac21))
### Bug Fixes
* **app:** fix ctrl drag for insertion into subgrids ([51ea947](https://github.com/windmill-labs/windmill/commit/51ea9473ef23c6871699e69bbe79772a4d50d3b8))
* **frontend:** graph cache of ai agent step tools ([#6431](https://github.com/windmill-labs/windmill/issues/6431)) ([28f1d61](https://github.com/windmill-labs/windmill/commit/28f1d611643459d42531fa217c185408eb97d6d1))
* make relevant sidebar menu items a instead of button ([06d078e](https://github.com/windmill-labs/windmill/commit/06d078ebfa8f70b66bc764eae70d33c8c57b4012))
* s3 result presigned not working with list ([9df008b](https://github.com/windmill-labs/windmill/commit/9df008b9f8fe58692463e4b9da0538935e458b10))
## [1.530.0](https://github.com/windmill-labs/windmill/compare/v1.529.0...v1.530.0) (2025-08-20)
+11
View File
@@ -4,6 +4,17 @@
Windmill is an open-source developer platform for building internal tools, workflows, API integrations, background jobs, workflows, and user interfaces. See @windmill-overview.mdc for full platform details.
## New Feature Implementation Guidelines
When implementing new features in Windmill, follow these best practices:
- **Clean Code First**: Write clean, readable, and maintainable code. Prioritize clarity over cleverness.
- **Avoid Duplication at All Costs**: Before writing new code, thoroughly search for existing implementations that can be reused or extended.
- **Adapt Existing Code**: Refactor and generalize existing code when necessary to avoid logic duplication. Extract common patterns into reusable utilities.
- **Follow Established Patterns**: Study existing code patterns in the codebase and maintain consistency with established conventions.
- **Single Responsibility**: Each function, component, and module should have a single, well-defined responsibility.
- **Incremental Implementation**: Break large features into smaller, reviewable chunks that can be implemented and tested incrementally.
## Language-Specific Guides
- Backend (Rust): @backend/rust-best-practices.mdc + @backend/summarized_schema.txt
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value from resource WHERE path = $1 AND workspace_id = $2 AND resource_type = 'json_schema'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "d4c963fa653652b7a3e8529cbf0d0fca091d7c1cb0924f6f9343544abb2666a5"
}
+149 -149
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.530.0"
version = "1.533.0"
authors.workspace = true
edition.workspace = true
@@ -33,7 +33,7 @@ members = [
]
[workspace.package]
version = "1.530.0"
version = "1.533.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.530.0
version: 1.533.0
title: Windmill API
contact:
+20 -14
View File
@@ -762,20 +762,26 @@ async fn get_public_resource(
Path((w_id, path)): Path<(String, StripPath)>,
) -> JsonResult<Option<serde_json::Value>> {
let path = path.to_path();
if !path.starts_with("f/app_themes/") {
return Err(Error::BadRequest(
"Only app themes are public resources".to_string(),
));
}
let res = sqlx::query_scalar!(
"SELECT value from resource WHERE path = $1 AND workspace_id = $2",
path.to_owned(),
&w_id
)
.fetch_optional(&db)
.await?
.flatten();
Ok(Json(res))
let res = if path.starts_with("f/app_themes/") {
sqlx::query_scalar!(
"SELECT value from resource WHERE path = $1 AND workspace_id = $2",
path.to_owned(),
&w_id
)
.fetch_optional(&db)
.await?
} else {
sqlx::query_scalar!(
"SELECT value from resource WHERE path = $1 AND workspace_id = $2 AND resource_type = 'json_schema'",
path.to_owned(),
&w_id
)
.fetch_optional(&db)
.await?
};
Ok(Json(res.flatten()))
}
async fn get_secret_id(
+14 -11
View File
@@ -118,7 +118,7 @@ struct Tool {
#[derive(Deserialize, Debug)]
struct AIAgentArgs {
provider: Provider,
system_prompt: String,
system_prompt: Option<String>,
user_message: String,
temperature: Option<f32>,
max_completion_tokens: Option<u32>,
@@ -608,18 +608,21 @@ async fn run_agent(
hostname: &str,
killpill_rx: &mut tokio::sync::broadcast::Receiver<()>,
) -> error::Result<Box<RawValue>> {
let mut messages = vec![
OpenAIMessage {
let mut messages = if let Some(system_prompt) = args.system_prompt.filter(|s| !s.is_empty()) {
vec![OpenAIMessage {
role: "system".to_string(),
content: Some(args.system_prompt),
content: Some(system_prompt),
..Default::default()
},
OpenAIMessage {
role: "user".to_string(),
content: Some(args.user_message),
..Default::default()
},
];
}]
} else {
vec![]
};
messages.push(OpenAIMessage {
role: "user".to_string(),
content: Some(args.user_message),
..Default::default()
});
let mut actions = vec![];
+1 -1
View File
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.530.0";
export const VERSION = "v1.533.0";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({
+18 -20
View File
@@ -3,11 +3,11 @@ import { GlobalOptions } from "../../types.ts";
import { requireLogin } from "../../core/auth.ts";
import { resolveWorkspace } from "../../core/context.ts";
import * as wmill from "../../../gen/services.gen.ts";
import { SyncOptions, readConfigFile, getEffectiveSettings, DEFAULT_SYNC_OPTIONS } from "../../core/conf.ts";
import { SyncOptions, readConfigFile, getEffectiveSettings, DEFAULT_SYNC_OPTIONS, getWmillYamlPath } from "../../core/conf.ts";
import { deepEqual } from "../../utils/utils.ts";
import { getCurrentGitBranch, isGitRepository } from "../../utils/git.ts";
import { GitSyncRepository, WriteMode } from "./types.ts";
import { WriteMode } from "./types.ts";
import { GitSyncSettingsConverter } from "./converter.ts";
import { handleLegacyRepositoryMigration } from "./legacySettings.ts";
import {
@@ -132,11 +132,9 @@ export async function pullGitSyncSettings(
const backendSyncOptions: SyncOptions = GitSyncSettingsConverter.fromBackendFormat(selectedRepo.settings);
// Check if wmill.yaml exists - create a default one if it doesn't exist
let wmillYamlExists = true;
try {
await Deno.stat("wmill.yaml");
} catch (error) {
wmillYamlExists = false;
const wmillYamlPath = getWmillYamlPath();
const wmillYamlExists = wmillYamlPath !== null;
if (!wmillYamlExists) {
if (!opts.jsonOutput) {
log.info(
colors.yellow(
@@ -165,11 +163,11 @@ export async function pullGitSyncSettings(
if (isGitRepository()) {
const currentBranch = getCurrentGitBranch();
if (currentBranch) {
if (!updatedConfig.git_branches) {
updatedConfig.git_branches = {};
if (!updatedConfig.gitBranches) {
updatedConfig.gitBranches = {};
}
if (!updatedConfig.git_branches[currentBranch]) {
updatedConfig.git_branches[currentBranch] = { overrides: {} };
if (!updatedConfig.gitBranches[currentBranch]) {
updatedConfig.gitBranches[currentBranch] = { overrides: {} };
}
}
}
@@ -358,16 +356,16 @@ export async function pullGitSyncSettings(
let needsBranchStructure = false;
if (isGitRepository()) {
const currentBranch = getCurrentGitBranch();
if (currentBranch && (!localConfig.git_branches || !localConfig.git_branches[currentBranch])) {
if (currentBranch && (!localConfig.gitBranches || !localConfig.gitBranches[currentBranch])) {
needsBranchStructure = true;
// Create empty branch structure
const updatedConfig = { ...localConfig };
if (!updatedConfig.git_branches) {
updatedConfig.git_branches = {};
if (!updatedConfig.gitBranches) {
updatedConfig.gitBranches = {};
}
if (!updatedConfig.git_branches[currentBranch]) {
updatedConfig.git_branches[currentBranch] = { overrides: {} };
if (!updatedConfig.gitBranches[currentBranch]) {
updatedConfig.gitBranches[currentBranch] = { overrides: {} };
}
// Write updated configuration
@@ -429,11 +427,11 @@ export async function pullGitSyncSettings(
const currentBranch = getCurrentGitBranch();
if (currentBranch) {
log.info(`Detected Git repository, adding empty branch structure for: ${currentBranch}`);
if (!updatedConfig.git_branches) {
updatedConfig.git_branches = {};
if (!updatedConfig.gitBranches) {
updatedConfig.gitBranches = {};
}
if (!updatedConfig.git_branches[currentBranch]) {
updatedConfig.git_branches[currentBranch] = { overrides: {} };
if (!updatedConfig.gitBranches[currentBranch]) {
updatedConfig.gitBranches[currentBranch] = { overrides: {} };
}
}
}
+3 -4
View File
@@ -3,7 +3,7 @@ import { GlobalOptions } from "../../types.ts";
import { requireLogin } from "../../core/auth.ts";
import { resolveWorkspace } from "../../core/context.ts";
import * as wmill from "../../../gen/services.gen.ts";
import { SyncOptions, readConfigFile, validateBranchConfiguration, getEffectiveSettings } from "../../core/conf.ts";
import { SyncOptions, readConfigFile, validateBranchConfiguration, getEffectiveSettings, getWmillYamlPath } from "../../core/conf.ts";
import { deepEqual } from "../../utils/utils.ts";
import { GitSyncRepository } from "./types.ts";
@@ -44,9 +44,8 @@ export async function pushGitSyncSettings(
try {
// Check if wmill.yaml exists - require it for git-sync settings commands
try {
await Deno.stat("wmill.yaml");
} catch (error) {
const wmillYamlPath = getWmillYamlPath();
if (!wmillYamlPath) {
log.error(
colors.red(
"No wmill.yaml file found. Please run 'wmill init' first to create the configuration file.",
+9 -9
View File
@@ -30,12 +30,12 @@ export function getOrCreateBranchConfig(config: SyncOptions, branchName: string)
config: SyncOptions;
branchKey: string;
} {
if (!config.git_branches) {
config.git_branches = {};
if (!config.gitBranches) {
config.gitBranches = {};
}
if (!config.git_branches[branchName]) {
config.git_branches[branchName] = {};
if (!config.gitBranches[branchName]) {
config.gitBranches[branchName] = {};
}
return {
@@ -53,12 +53,12 @@ export function applyBackendSettingsToBranch(
const { config: updatedConfig } = getOrCreateBranchConfig(config, branchName);
// Get the base settings (top-level + defaults) to compare against
const { git_branches, ...topLevelSettings } = config;
const { gitBranches, ...topLevelSettings } = config;
const baseSettings: Partial<SyncOptions> = { ...DEFAULT_SYNC_OPTIONS, ...topLevelSettings };
// Only store fields that differ from the base settings
Object.keys(backendSettings).forEach(key => {
if (key !== 'git_branches' && backendSettings[key as keyof SyncOptions] !== undefined) {
if (key !== 'gitBranches' && backendSettings[key as keyof SyncOptions] !== undefined) {
const backendValue = backendSettings[key as keyof SyncOptions];
const baseValue = baseSettings[key as keyof SyncOptions];
@@ -66,10 +66,10 @@ export function applyBackendSettingsToBranch(
const isDifferent = GitSyncSettingsConverter.isDifferent(backendValue, baseValue);
if (isDifferent) {
if (!updatedConfig.git_branches![branchName].overrides) {
updatedConfig.git_branches![branchName].overrides = {};
if (!updatedConfig.gitBranches![branchName].overrides) {
updatedConfig.gitBranches![branchName].overrides = {};
}
(updatedConfig.git_branches![branchName].overrides as any)[key] = backendValue;
(updatedConfig.gitBranches![branchName].overrides as any)[key] = backendValue;
}
}
});
+9 -9
View File
@@ -43,14 +43,14 @@ async function initAction(opts: InitOptions) {
if (isGitRepository()) {
const currentBranch = getCurrentGitBranch();
if (currentBranch) {
initialConfig.git_branches = {
initialConfig.gitBranches = {
[currentBranch]: { overrides: {} },
};
} else {
initialConfig.git_branches = {};
initialConfig.gitBranches = {};
}
} else {
initialConfig.git_branches = {};
initialConfig.gitBranches = {};
}
await Deno.writeTextFile("wmill.yaml", yamlStringify(initialConfig));
@@ -116,16 +116,16 @@ async function initAction(opts: InitOptions) {
const currentConfig = await import("../../core/conf.ts").then((m) =>
m.readConfigFile()
);
if (!currentConfig.git_branches) {
currentConfig.git_branches = {};
if (!currentConfig.gitBranches) {
currentConfig.gitBranches = {};
}
if (!currentConfig.git_branches[currentBranch]) {
currentConfig.git_branches[currentBranch] = { overrides: {} };
if (!currentConfig.gitBranches[currentBranch]) {
currentConfig.gitBranches[currentBranch] = { overrides: {} };
}
currentConfig.git_branches[currentBranch].baseUrl =
currentConfig.gitBranches[currentBranch].baseUrl =
activeWorkspace.remote;
currentConfig.git_branches[currentBranch].workspaceId =
currentConfig.gitBranches[currentBranch].workspaceId =
activeWorkspace.workspaceId;
await Deno.writeTextFile(
+165 -24
View File
@@ -42,7 +42,19 @@ import {
readConfigFile,
getEffectiveSettings,
validateBranchConfiguration,
mergeConfigWithConfigFile,
} from "../../core/conf.ts";
import {
SpecificItemsConfig,
getSpecificItemsForCurrentBranch,
isSpecificItem,
getBranchSpecificPath,
fromBranchSpecificPath,
isCurrentBranchFile,
toBranchSpecificPath,
isBranchSpecificFile,
} from "../../core/specific_items.ts";
import { getCurrentGitBranch } from "../../utils/git.ts";
import { Workspace } from "../workspace/workspace.ts";
import { removePathPrefix } from "../../types.ts";
import { SyncCodebase, listSyncCodebases } from "../../utils/codebase.ts";
@@ -67,9 +79,9 @@ function mergeCliWithEffectiveOptions<
// Resolve effective sync options using branch-based configuration
async function resolveEffectiveSyncOptions(
workspace: Workspace,
localConfig: SyncOptions,
promotion?: string
): Promise<SyncOptions> {
const localConfig = await readConfigFile();
return await getEffectiveSettings(localConfig, promotion);
}
@@ -631,9 +643,36 @@ export async function elementsToMap(
els: DynFSElement,
ignore: (path: string, isDirectory: boolean) => boolean,
json: boolean,
skips: Skips
skips: Skips,
specificItems?: SpecificItemsConfig
): Promise<{ [key: string]: string }> {
const map: { [key: string]: string } = {};
const processedBasePaths = new Set<string>();
// First pass: collect all file paths to identify branch-specific files
const allPaths: string[] = [];
for await (const entry of readDirRecursiveWithIgnore(ignore, els)) {
if (!entry.isDirectory && !entry.ignored) {
allPaths.push(entry.path);
}
}
const branchSpecificExists = new Set<string>();
if (specificItems) {
const currentBranch = getCurrentGitBranch();
if (currentBranch) {
for (const path of allPaths) {
if (isCurrentBranchFile(path)) {
const basePath = fromBranchSpecificPath(path, currentBranch);
if (isSpecificItem(basePath, specificItems)) {
branchSpecificExists.add(basePath);
}
}
}
}
}
for await (const entry of readDirRecursiveWithIgnore(ignore, els)) {
if (entry.isDirectory || entry.ignored) continue;
const path = entry.path;
@@ -695,11 +734,21 @@ export async function elementsToMap(
"nu",
"java",
"rb",
// for related places search: ADD_NEW_LANG
// for related places search: ADD_NEW_LANG
].includes(path.split(".").pop() ?? "") &&
!isFileResource(path)
)
continue;
// Handle branch-specific files - skip files for other branches
if (specificItems && isBranchSpecificFile(path)) {
const currentBranch = getCurrentGitBranch();
if (!currentBranch || !isCurrentBranchFile(path)) {
// Skip branch-specific files for other branches
continue;
}
}
const content = await entry.getContentText();
if (skips.skipSecrets && path.endsWith(".variable" + ext)) {
@@ -727,7 +776,33 @@ export async function elementsToMap(
log.warn(`Error reading variable ${path} to check for secrets`);
}
}
map[entry.path] = content;
// Handle branch-specific path mapping after all filtering
if (specificItems) {
const currentBranch = getCurrentGitBranch();
if (currentBranch && isCurrentBranchFile(path)) {
// This is a branch-specific file for current branch
const basePath = fromBranchSpecificPath(path, currentBranch);
if (isSpecificItem(basePath, specificItems)) {
// Map to base path for push operations
map[basePath] = content;
processedBasePaths.add(basePath);
} else {
// Branch-specific file doesn't match pattern, skip it
continue;
}
} else if (!isBranchSpecificFile(path)) {
// This is a regular base file, check if we should skip it
if (processedBasePaths.has(path)) {
// Skip base file, we already processed branch-specific version
continue;
}
map[path] = content;
}
} else {
// No specific items configuration, use regular path
map[entry.path] = content;
}
}
return map;
}
@@ -758,14 +833,15 @@ async function compareDynFSElement(
skips: Skips,
ignoreMetadataDeletion: boolean,
codebases: SyncCodebase[],
ignoreCodebaseChanges: boolean
ignoreCodebaseChanges: boolean,
specificItems?: SpecificItemsConfig
): Promise<Change[]> {
const [m1, m2] = els2
? await Promise.all([
elementsToMap(els1, ignore, json, skips),
elementsToMap(els2, ignore, json, skips),
elementsToMap(els1, ignore, json, skips, specificItems),
elementsToMap(els2, ignore, json, skips, specificItems),
])
: [await elementsToMap(els1, ignore, json, skips), {}];
: [await elementsToMap(els1, ignore, json, skips, specificItems), {}];
const changes: Change[] = [];
@@ -1163,6 +1239,10 @@ export async function pull(
opts: GlobalOptions &
SyncOptions & { repository?: string; promotion?: string }
) {
const originalCliOpts = { ...opts };
opts = await mergeConfigWithConfigFile(opts);
// Validate branch configuration early
try {
await validateBranchConfiguration(false, opts.yes);
@@ -1184,11 +1264,15 @@ export async function pull(
// Resolve effective sync options with branch awareness
const effectiveOpts = await resolveEffectiveSyncOptions(
workspace,
opts,
opts.promotion
);
// Extract specific items configuration before merging overwrites gitBranches
const specificItems = getSpecificItemsForCurrentBranch(opts);
// Merge CLI flags with resolved settings (CLI flags take precedence only for explicit overrides)
opts = mergeCliWithEffectiveOptions(opts, effectiveOpts);
opts = mergeCliWithEffectiveOptions(originalCliOpts, effectiveOpts);
const codebases = await listSyncCodebases(opts);
@@ -1238,7 +1322,8 @@ export async function pull(
opts,
false,
codebases,
true
true,
specificItems
);
log.info(
@@ -1255,6 +1340,12 @@ export async function pull(
...(change.name === "edited" && change.codebase
? { codebase_changed: true }
: {}),
...(specificItems && isSpecificItem(change.path, specificItems)
? {
branch_specific: true,
branch_specific_path: getBranchSpecificPath(change.path, specificItems)
}
: {}),
})),
total: changes.length,
};
@@ -1264,7 +1355,7 @@ export async function pull(
if (changes.length > 0) {
if (!opts.jsonOutput) {
prettyChanges(changes);
prettyChanges(changes, specificItems);
}
if (opts.dryRun) {
log.info(colors.gray(`Dry run complete.`));
@@ -1284,8 +1375,17 @@ export async function pull(
log.info(colors.gray(`Applying changes to files ...`));
for await (const change of changes) {
const target = path.join(Deno.cwd(), change.path);
const stateTarget = path.join(Deno.cwd(), ".wmill", change.path);
// Determine if this file should be written to a branch-specific path
let targetPath = change.path;
if (specificItems && isSpecificItem(change.path, specificItems)) {
const branchSpecificPath = getBranchSpecificPath(change.path, specificItems);
if (branchSpecificPath) {
targetPath = branchSpecificPath;
}
}
const target = path.join(Deno.cwd(), targetPath);
const stateTarget = path.join(Deno.cwd(), ".wmill", targetPath);
if (change.name === "edited") {
if (opts.stateful) {
try {
@@ -1328,12 +1428,12 @@ export async function pull(
}
}
if (exts.some((e) => change.path.endsWith(e))) {
log.info(`Editing script content of ${change.path}`);
log.info(`Editing script content of ${targetPath}${targetPath !== change.path ? colors.gray(` (branch-specific override for ${change.path})`) : ""}`);
} else if (
change.path.endsWith(".yaml") ||
change.path.endsWith(".json")
) {
log.info(`Editing ${getTypeStrFromPath(change.path)} ${change.path}`);
log.info(`Editing ${getTypeStrFromPath(change.path)} ${targetPath}${targetPath !== change.path ? colors.gray(` (branch-specific override for ${change.path})`) : ""}`);
}
await Deno.writeTextFile(target, change.after);
@@ -1345,10 +1445,10 @@ export async function pull(
await ensureDir(path.dirname(target));
if (opts.stateful) {
await ensureDir(path.dirname(stateTarget));
log.info(`Adding ${getTypeStrFromPath(change.path)} ${change.path}`);
log.info(`Adding ${getTypeStrFromPath(change.path)} ${targetPath}${targetPath !== change.path ? colors.gray(` (branch-specific override for ${change.path})`) : ""}`);
}
await Deno.writeTextFile(target, change.content);
log.info(`Writing ${getTypeStrFromPath(change.path)} ${change.path}`);
log.info(`Writing ${getTypeStrFromPath(change.path)} ${targetPath}${targetPath !== change.path ? colors.gray(` (branch-specific override for ${change.path})`) : ""}`);
if (opts.stateful) {
await Deno.copyFile(target, stateTarget);
}
@@ -1423,6 +1523,12 @@ export async function pull(
...(change.name === "edited" && change.codebase
? { codebase_changed: true }
: {}),
...(specificItems && isSpecificItem(change.path, specificItems)
? {
branch_specific: true,
branch_specific_path: getBranchSpecificPath(change.path, specificItems)
}
: {}),
})),
total: changes.length,
};
@@ -1445,21 +1551,33 @@ export async function pull(
}
}
function prettyChanges(changes: Change[]) {
function prettyChanges(changes: Change[], specificItems?: SpecificItemsConfig) {
for (const change of changes) {
let displayPath = change.path;
let branchNote = "";
// Check if this will be written as a branch-specific file
if (specificItems && isSpecificItem(change.path, specificItems)) {
const branchSpecificPath = getBranchSpecificPath(change.path, specificItems);
if (branchSpecificPath) {
displayPath = branchSpecificPath;
branchNote = " (branch-specific)";
}
}
if (change.name === "added") {
log.info(
colors.green(`+ ${getTypeStrFromPath(change.path)} ` + change.path)
colors.green(`+ ${getTypeStrFromPath(change.path)} ` + displayPath + colors.gray(branchNote))
);
} else if (change.name === "deleted") {
log.info(
colors.red(`- ${getTypeStrFromPath(change.path)} ` + change.path)
colors.red(`- ${getTypeStrFromPath(change.path)} ` + displayPath + colors.gray(branchNote))
);
} else if (change.name === "edited") {
log.info(
colors.yellow(
`~ ${getTypeStrFromPath(change.path)} ` +
change.path +
displayPath + colors.gray(branchNote) +
(change.codebase ? ` (codebase changed)` : "")
)
);
@@ -1499,6 +1617,12 @@ function removeSuffix(str: string, suffix: string) {
export async function push(
opts: GlobalOptions & SyncOptions & { repository?: string }
) {
// Save original CLI options before merging with config file
const originalCliOpts = { ...opts };
// Load configuration from wmill.yaml and merge with CLI options
opts = await mergeConfigWithConfigFile(opts);
// Validate branch configuration early
try {
await validateBranchConfiguration(false, opts.yes);
@@ -1516,11 +1640,15 @@ export async function push(
// Resolve effective sync options with branch awareness
const effectiveOpts = await resolveEffectiveSyncOptions(
workspace,
opts,
opts.promotion
);
// Extract specific items configuration BEFORE merging overwrites gitBranches
const specificItems = getSpecificItemsForCurrentBranch(opts);
// Merge CLI flags with resolved settings (CLI flags take precedence only for explicit overrides)
opts = mergeCliWithEffectiveOptions(opts, effectiveOpts);
opts = mergeCliWithEffectiveOptions(originalCliOpts, effectiveOpts);
const codebases = await listSyncCodebases(opts);
if (opts.raw) {
@@ -1581,7 +1709,8 @@ export async function push(
opts,
true,
codebases,
false
false,
specificItems
);
const globalDeps = await findGlobalDeps();
@@ -1660,6 +1789,12 @@ export async function push(
...(change.name === "edited" && change.codebase
? { codebase_changed: true }
: {}),
...(specificItems && isSpecificItem(change.path, specificItems)
? {
branch_specific: true,
branch_specific_path: getBranchSpecificPath(change.path, specificItems)
}
: {}),
})),
total: changes.length,
};
@@ -1669,7 +1804,7 @@ export async function push(
if (changes.length > 0) {
if (!opts.jsonOutput) {
prettyChanges(changes);
prettyChanges(changes, specificItems);
}
if (opts.dryRun) {
log.info(colors.gray(`Dry run complete.`));
@@ -2041,6 +2176,12 @@ export async function push(
...(change.name === "edited" && change.codebase
? { codebase_changed: true }
: {}),
...(specificItems && isSpecificItem(change.path, specificItems)
? {
branch_specific: true,
branch_specific_path: getBranchSpecificPath(change.path, specificItems)
}
: {}),
})),
total: changes.length,
duration_ms: Math.round(performance.now() - start),
+10 -10
View File
@@ -386,22 +386,22 @@ async function bind(
}
// For unbind, check if branch exists
if (!bindWorkspace && (!config.git_branches || !config.git_branches[branch])) {
log.error(colors.red(`Branch '${branch}' not found in wmill.yaml git_branches`));
if (!bindWorkspace && (!config.gitBranches || !config.gitBranches[branch])) {
log.error(colors.red(`Branch '${branch}' not found in wmill.yaml gitBranches`));
return;
}
// Update the branch configuration with workspace binding
if (!config.git_branches) {
config.git_branches = {};
if (!config.gitBranches) {
config.gitBranches = {};
}
if (!config.git_branches[branch]) {
config.git_branches[branch] = { overrides: {} };
if (!config.gitBranches[branch]) {
config.gitBranches[branch] = { overrides: {} };
}
if (bindWorkspace && activeWorkspace) {
config.git_branches[branch].baseUrl = activeWorkspace.remote;
config.git_branches[branch].workspaceId = activeWorkspace.workspaceId;
config.gitBranches[branch].baseUrl = activeWorkspace.remote;
config.gitBranches[branch].workspaceId = activeWorkspace.workspaceId;
log.info(colors.green(
`✓ Bound branch '${branch}' to workspace '${activeWorkspace.name}'\n` +
@@ -409,8 +409,8 @@ async function bind(
));
} else {
// Unbind
delete config.git_branches[branch].baseUrl;
delete config.git_branches[branch].workspaceId;
delete config.gitBranches[branch].baseUrl;
delete config.gitBranches[branch].workspaceId;
log.info(colors.green(`✓ Removed workspace binding from branch '${branch}'`));
}
+210 -35
View File
@@ -1,5 +1,8 @@
import { log, yamlParseFile, Confirm, yamlStringify } from "../../deps.ts";
import { getCurrentGitBranch, isGitRepository } from "../utils/git.ts";
import { join, dirname, resolve, relative } from "node:path";
import { existsSync } from "node:fs";
import { execSync } from "node:child_process";
export let showDiffs = false;
export function setShowDiffs(value: boolean) {
@@ -37,13 +40,40 @@ export interface SyncOptions {
codebases?: Codebase[];
parallel?: number;
jsonOutput?: boolean;
git_branches?: {
gitBranches?: {
commonSpecificItems?: {
variables?: string[];
resources?: string[];
};
} & {
[branchName: string]: SyncOptions & {
overrides?: Partial<SyncOptions>;
promotionOverrides?: Partial<SyncOptions>;
baseUrl?: string;
workspaceId?: string;
}
specificItems?: {
variables?: string[];
resources?: string[];
};
};
};
// Legacy field - deprecated, use gitBranches instead
git_branches?: {
commonSpecificItems?: {
variables?: string[];
resources?: string[];
};
} & {
[branchName: string]: SyncOptions & {
overrides?: Partial<SyncOptions>;
promotionOverrides?: Partial<SyncOptions>;
baseUrl?: string;
workspaceId?: string;
specificItems?: {
variables?: string[];
resources?: string[];
};
};
};
promotion?: string;
}
@@ -62,9 +92,90 @@ export interface Codebase {
inject?: string[];
}
function getGitRepoRoot(): string | null {
try {
const result = execSync("git rev-parse --show-toplevel", {
encoding: "utf8",
stdio: "pipe"
});
return result.trim();
} catch (error) {
return null;
}
}
function findWmillYaml(): string | null {
const startDir = resolve(Deno.cwd());
const isInGitRepo = isGitRepository();
// If not in git repo, only check current directory
if (!isInGitRepo) {
const wmillYamlPath = join(startDir, "wmill.yaml");
return existsSync(wmillYamlPath) ? wmillYamlPath : null;
}
// If in git repo, search up to git repository root
const gitRoot = getGitRepoRoot();
let currentDir = startDir;
let foundPath: string | null = null;
while (true) {
const wmillYamlPath = join(currentDir, "wmill.yaml");
if (existsSync(wmillYamlPath)) {
foundPath = wmillYamlPath;
break;
}
// Check if we've reached the git repository root
if (gitRoot && resolve(currentDir) === resolve(gitRoot)) {
break;
}
// Check if we've reached the filesystem root
const parentDir = dirname(currentDir);
if (parentDir === currentDir) {
break;
}
currentDir = parentDir;
}
// If wmill.yaml was found in a parent directory, warn the user and change working directory
if (foundPath && resolve(dirname(foundPath)) !== resolve(startDir)) {
const configDir = dirname(foundPath);
const relativePath = relative(startDir, foundPath);
log.warn(`⚠️ wmill.yaml found in parent directory: ${relativePath}`);
// Change working directory to where wmill.yaml was found
Deno.chdir(configDir);
log.info(`📁 Changed working directory to: ${configDir}`);
}
return foundPath;
}
export function getWmillYamlPath(): string | null {
return findWmillYaml();
}
export async function readConfigFile(): Promise<SyncOptions> {
try {
const conf = (await yamlParseFile("wmill.yaml")) as SyncOptions;
// First, try to find wmill.yaml recursively
const wmillYamlPath = findWmillYaml();
if (!wmillYamlPath) {
log.warn(
"No wmill.yaml found. Use 'wmill init' to bootstrap it. Using 'bun' as default typescript runtime."
);
return {};
}
const conf = (await yamlParseFile(wmillYamlPath)) as SyncOptions;
// Handle legacy format migrations (combine overrides and git_branches)
let needsConfigWrite = false;
const migrationMessages: string[] = [];
// Handle obsolete overrides format
if (conf && 'overrides' in conf) {
@@ -78,18 +189,54 @@ export async function readConfigFile(): Promise<SyncOptions> {
" Please delete your wmill.yaml and run 'wmill init' to recreate it with the new format."
);
} else {
// Remove empty overrides with a note
log.info("️ Removing empty 'overrides: {}' from wmill.yaml (migrated to git_branches format)");
// Remove empty overrides
delete conf.overrides;
// Write the updated config back to file
try {
await Deno.writeTextFile("wmill.yaml", yamlStringify(conf));
} catch (error) {
log.warn(`Could not update wmill.yaml to remove empty overrides: ${error instanceof Error ? error.message : error}`);
}
needsConfigWrite = true;
migrationMessages.push("️ Removing empty 'overrides: {}' from wmill.yaml (migrated to gitBranches format)");
}
}
// Handle git_branches to gitBranches migration
if (conf && 'git_branches' in conf) {
if (!conf.gitBranches) {
// Deep copy git_branches to gitBranches (even if empty)
conf.gitBranches = JSON.parse(JSON.stringify(conf.git_branches));
needsConfigWrite = true;
migrationMessages.push("⚠️ Migrating 'git_branches' to 'gitBranches' (camelCase). The snake_case format is deprecated.");
migrationMessages.push("✅ Successfully migrated 'git_branches' to 'gitBranches' in wmill.yaml");
} else {
migrationMessages.push("⚠️ Both 'git_branches' and 'gitBranches' found in wmill.yaml. Using 'gitBranches' and ignoring 'git_branches'.");
}
// Always remove the old field from config object (both file and memory)
delete conf.git_branches;
}
// Perform single atomic write if any migrations are needed
if (needsConfigWrite) {
try {
await Deno.writeTextFile(wmillYamlPath, yamlStringify(conf));
// Log all migration messages after successful write
migrationMessages.forEach(msg => {
if (msg.startsWith('⚠️')) {
log.warn(msg);
} else {
log.info(msg);
}
});
} catch (error) {
log.warn(`Could not update wmill.yaml to apply migrations: ${error instanceof Error ? error.message : error}`);
}
} else if (migrationMessages.length > 0) {
// Log messages for non-write cases (like "both found")
migrationMessages.forEach(msg => {
if (msg.startsWith('⚠️')) {
log.warn(msg);
} else {
log.info(msg);
}
});
}
if (conf?.defaultTs == undefined) {
log.warn(
"No defaultTs defined in your wmill.yaml. Using 'bun' as default."
@@ -100,10 +247,23 @@ export async function readConfigFile(): Promise<SyncOptions> {
if (e instanceof Error && (e.message.includes("overrides") || e.message.includes("Obsolete configuration format"))) {
throw e; // Re-throw the specific obsolete format error
}
log.warn(
"No wmill.yaml found. Use 'wmill init' to bootstrap it. Using 'bun' as default typescript runtime."
);
return {};
// Since we already found the file path, this is likely a parsing or access error
if (e instanceof Error && e.message.includes("Error parsing yaml")) {
const yamlError = e.cause instanceof Error ? e.cause.message : String(e.cause);
throw new Error(
"❌ YAML syntax error in wmill.yaml:\n" +
" " + yamlError + "\n" +
" Please fix the YAML syntax in wmill.yaml or delete the file to start fresh."
);
} else {
// File exists but has other issues (permissions, etc.)
throw new Error(
"❌ Failed to read wmill.yaml:\n" +
" " + (e instanceof Error ? e.message : String(e)) + "\n" +
" Please check file permissions or fix the syntax."
);
}
}
}
@@ -148,26 +308,26 @@ export async function validateBranchConfiguration(skipValidation?: boolean, auto
}
const config = await readConfigFile();
const { git_branches } = config;
const { gitBranches } = config;
const currentBranch = getCurrentGitBranch();
// In a git repository, git_branches section is recommended
if (!git_branches || Object.keys(git_branches).length === 0) {
// In a git repository, gitBranches section is recommended
if (!gitBranches || Object.keys(gitBranches).length === 0) {
log.warn(
"⚠️ WARNING: In a Git repository, the 'git_branches' section is recommended in wmill.yaml.\n" +
" Consider adding a git_branches section with configuration for your Git branches.\n" +
"⚠️ WARNING: In a Git repository, the 'gitBranches' section is recommended in wmill.yaml.\n" +
" Consider adding a gitBranches section with configuration for your Git branches.\n" +
" Run 'wmill init' to recreate the configuration file with proper branch setup."
);
return;
}
// Current branch must be defined in git_branches config
if (currentBranch && !git_branches[currentBranch]) {
// Current branch must be defined in gitBranches config
if (currentBranch && !gitBranches[currentBranch]) {
// In interactive mode, offer to create the branch
if (Deno.stdin.isTerminal()) {
const availableBranches = Object.keys(git_branches).join(', ');
const availableBranches = Object.keys(gitBranches).join(', ');
log.info(
`Current Git branch '${currentBranch}' is not defined in the git_branches configuration.\n` +
`Current Git branch '${currentBranch}' is not defined in the gitBranches configuration.\n` +
`Available branches: ${availableBranches}`
);
@@ -177,13 +337,21 @@ export async function validateBranchConfiguration(skipValidation?: boolean, auto
});
if (shouldCreate) {
// Warn if branch name contains filesystem-unsafe characters
if (/[\/\\:*?"<>|.]/.test(currentBranch)) {
const sanitizedBranchName = currentBranch.replace(/[\/\\:*?"<>|.]/g, '_');
log.warn(`⚠️ WARNING: Branch name "${currentBranch}" contains filesystem-unsafe characters (/ \\ : * ? " < > | .).`);
log.warn(` Branch-specific files will be saved with sanitized name: "${sanitizedBranchName}"`);
log.warn(` Example: "file.variable.yaml" → "file.${sanitizedBranchName}.variable.yaml"`);
}
// Read current config, add branch, and write it back
const currentConfig = await readConfigFile();
if (!currentConfig.git_branches) {
currentConfig.git_branches = {};
if (!currentConfig.gitBranches) {
currentConfig.gitBranches = {};
}
currentConfig.git_branches[currentBranch] = { overrides: {} };
currentConfig.gitBranches[currentBranch] = { overrides: {} };
await Deno.writeTextFile("wmill.yaml", yamlStringify(currentConfig));
@@ -193,10 +361,17 @@ export async function validateBranchConfiguration(skipValidation?: boolean, auto
return;
}
} else {
// Warn about filesystem-unsafe characters in branch name
if (/[\/\\:*?"<>|.]/.test(currentBranch)) {
const sanitizedBranchName = currentBranch.replace(/[\/\\:*?"<>|.]/g, '_');
log.warn(`⚠️ WARNING: Branch name "${currentBranch}" contains filesystem-unsafe characters (/ \\ : * ? " < > | .).`);
log.warn(` Branch-specific files will use sanitized name: "${sanitizedBranchName}"`);
}
log.warn(
`⚠️ WARNING: Current Git branch '${currentBranch}' is not defined in the git_branches configuration.\n` +
` Consider adding configuration for branch '${currentBranch}' in the git_branches section of wmill.yaml.\n` +
` Available branches: ${Object.keys(git_branches).join(', ')}`
`⚠️ WARNING: Current Git branch '${currentBranch}' is not defined in the gitBranches configuration.\n` +
` Consider adding configuration for branch '${currentBranch}' in the gitBranches section of wmill.yaml.\n` +
` Available branches: ${Object.keys(gitBranches).join(', ')}`
);
return;
}
@@ -206,15 +381,15 @@ export async function validateBranchConfiguration(skipValidation?: boolean, auto
// Get effective settings by merging top-level settings with branch-specific overrides
export async function getEffectiveSettings(config: SyncOptions, promotion?: string, skipBranchValidation?: boolean, suppressLogs?: boolean): Promise<SyncOptions> {
// Start with top-level settings from config
const { git_branches, ...topLevelSettings } = config;
const { gitBranches, ...topLevelSettings } = config;
let effective = { ...topLevelSettings };
if (isGitRepository()) {
const currentBranch = getCurrentGitBranch();
// If promotion is specified, use that branch's promotionOverrides or overrides
if (promotion && git_branches && git_branches[promotion]) {
const targetBranch = git_branches[promotion];
if (promotion && gitBranches && gitBranches[promotion]) {
const targetBranch = gitBranches[promotion];
// First try promotionOverrides, then fall back to overrides
if (targetBranch.promotionOverrides) {
@@ -232,8 +407,8 @@ export async function getEffectiveSettings(config: SyncOptions, promotion?: stri
}
}
// Otherwise use current branch overrides (existing behavior)
else if (currentBranch && git_branches && git_branches[currentBranch] && git_branches[currentBranch].overrides) {
Object.assign(effective, git_branches[currentBranch].overrides);
else if (currentBranch && gitBranches && gitBranches[currentBranch] && gitBranches[currentBranch].overrides) {
Object.assign(effective, gitBranches[currentBranch].overrides);
if (!suppressLogs) {
log.info(`Applied settings for Git branch: ${currentBranch}`);
}
+1 -1
View File
@@ -135,7 +135,7 @@ async function tryResolveBranchWorkspace(
// Read wmill.yaml to check for branch workspace configuration
const config = await readConfigFile();
const branchConfig = config.git_branches?.[currentBranch];
const branchConfig = config.gitBranches?.[currentBranch];
// Check if branch has workspace configuration
if (!branchConfig?.baseUrl || !branchConfig?.workspaceId) {
+186
View File
@@ -0,0 +1,186 @@
import { minimatch } from "../../deps.ts";
import { getCurrentGitBranch, isGitRepository } from "../utils/git.ts";
import { SyncOptions } from "./conf.ts";
export interface SpecificItemsConfig {
variables?: string[];
resources?: string[];
}
/**
* Get the specific items configuration for the current git branch
* Merges commonSpecificItems with branch-specific specificItems
*/
export function getSpecificItemsForCurrentBranch(config: SyncOptions): SpecificItemsConfig | undefined {
if (!isGitRepository() || !config.gitBranches) {
return undefined;
}
const currentBranch = getCurrentGitBranch();
if (!currentBranch) {
return undefined;
}
const commonItems = config.gitBranches.commonSpecificItems;
const branchItems = config.gitBranches[currentBranch]?.specificItems;
// If neither common nor branch-specific items exist, return undefined
if (!commonItems && !branchItems) {
return undefined;
}
// Merge common and branch-specific items
const merged: SpecificItemsConfig = {};
// Add common items
if (commonItems?.variables) {
merged.variables = [...commonItems.variables];
}
if (commonItems?.resources) {
merged.resources = [...commonItems.resources];
}
// Add branch-specific items (extending common items)
if (branchItems?.variables) {
merged.variables = [...(merged.variables || []), ...branchItems.variables];
}
if (branchItems?.resources) {
merged.resources = [...(merged.resources || []), ...branchItems.resources];
}
return merged;
}
/**
* Check if a path matches any of the patterns in the given list
*/
function matchesPatterns(path: string, patterns: string[]): boolean {
return patterns.some(pattern => minimatch(path, pattern));
}
/**
* Check if a file path should be treated as branch-specific
*/
export function isSpecificItem(path: string, specificItems: SpecificItemsConfig | undefined): boolean {
if (!specificItems) {
return false;
}
// Determine the item type from the file path
if (path.endsWith('.variable.yaml')) {
return specificItems.variables ? matchesPatterns(path, specificItems.variables) : false;
}
if (path.endsWith('.resource.yaml')) {
return specificItems.resources ? matchesPatterns(path, specificItems.resources) : false;
}
return false;
}
/**
* Convert a base path to a branch-specific path
*/
export function toBranchSpecificPath(basePath: string, branchName: string): string {
// Extract the extension (e.g., ".variable.yaml" or ".resource.yaml")
const extensionMatch = basePath.match(/(\.(variable|resource)\.yaml)$/);
if (!extensionMatch) {
return basePath; // Return unchanged if no recognized extension
}
const extension = extensionMatch[1];
const pathWithoutExtension = basePath.substring(0, basePath.length - extension.length);
// Sanitize branch name to be filesystem-safe
const sanitizedBranchName = branchName.replace(/[\/\\:*?"<>|.]/g, '_');
// Warn about potential collisions if sanitization occurred
if (sanitizedBranchName !== branchName) {
console.warn(`Warning: Branch name "${branchName}" contains filesystem-unsafe characters (/ \\ : * ? " < > | .) and was sanitized to "${sanitizedBranchName}". This may cause collisions with other similarly named branches.`);
}
return `${pathWithoutExtension}.${sanitizedBranchName}${extension}`;
}
/**
* Convert a branch-specific path back to a base path
*/
export function fromBranchSpecificPath(branchSpecificPath: string, branchName: string): string {
// Sanitize branch name the same way as in toBranchSpecificPath
const sanitizedBranchName = branchName.replace(/[\/\\:*?"<>|.]/g, '_');
// Pattern: path.sanitizedBranchName.extension
const escapedBranchName = sanitizedBranchName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const pattern = new RegExp(`\\.${escapedBranchName}(\\.(variable|resource)\\.yaml)$`);
const match = branchSpecificPath.match(pattern);
if (!match) {
return branchSpecificPath; // Return unchanged if not a branch-specific path
}
const extension = match[1];
const pathWithoutBranchAndExtension = branchSpecificPath.substring(
0,
branchSpecificPath.length - `.${sanitizedBranchName}${extension}`.length
);
return `${pathWithoutBranchAndExtension}${extension}`;
}
/**
* Get the branch-specific path for the current branch if the item should be branch-specific
*/
export function getBranchSpecificPath(
basePath: string,
specificItems: SpecificItemsConfig | undefined
): string | undefined {
if (!isGitRepository() || !specificItems) {
return undefined;
}
const currentBranch = getCurrentGitBranch();
if (!currentBranch) {
return undefined;
}
if (isSpecificItem(basePath, specificItems)) {
return toBranchSpecificPath(basePath, currentBranch);
}
return undefined;
}
// Cache for compiled regex patterns to avoid recompilation
const branchPatternCache = new Map<string, RegExp>();
/**
* Check if a path is a branch-specific file for the current branch
*/
export function isCurrentBranchFile(path: string): boolean {
if (!isGitRepository()) {
return false;
}
const currentBranch = getCurrentGitBranch();
if (!currentBranch) {
return false;
}
// Use cached pattern or create and cache new one
let pattern = branchPatternCache.get(currentBranch);
if (!pattern) {
pattern = new RegExp(`\\.${currentBranch}\\.(variable|resource)\\.yaml$`);
branchPatternCache.set(currentBranch, pattern);
}
return pattern.test(path);
}
/**
* Check if a path is a branch-specific file for ANY branch (not necessarily current)
* Used to identify and skip files from other branches during sync operations
*/
export function isBranchSpecificFile(path: string): boolean {
// Pattern: *.branchName.variable.yaml or *.branchName.resource.yaml
return /\.[^.]+\.(variable|resource)\.yaml$/.test(path);
}
+1 -1
View File
@@ -68,7 +68,7 @@ export {
// }
// });
export const VERSION = "1.530.0";
export const VERSION = "1.533.0";
const command = new Command()
.name("wmill")
+999 -1111
View File
File diff suppressed because it is too large Load Diff
+13 -13
View File
@@ -1,6 +1,6 @@
{
"name": "windmill-components",
"version": "1.530.0",
"version": "1.533.0",
"scripts": {
"dev": "vite dev",
"build": "vite build",
@@ -78,13 +78,13 @@
"type": "module",
"dependencies": {
"@aws-crypto/sha256-js": "^4.0.0",
"@codingame/monaco-vscode-configuration-service-override": "~19.1.4",
"@codingame/monaco-vscode-editor-api": "~19.1.4",
"@codingame/monaco-vscode-standalone-css-language-features": "~19.1.4",
"@codingame/monaco-vscode-standalone-html-language-features": "~19.1.4",
"@codingame/monaco-vscode-standalone-json-language-features": "~19.1.4",
"@codingame/monaco-vscode-standalone-languages": "~19.1.4",
"@codingame/monaco-vscode-standalone-typescript-language-features": "~19.1.4",
"@codingame/monaco-vscode-configuration-service-override": "~20.2.1",
"@codingame/monaco-vscode-editor-api": "~20.2.1",
"@codingame/monaco-vscode-standalone-css-language-features": "~20.2.1",
"@codingame/monaco-vscode-standalone-html-language-features": "~20.2.1",
"@codingame/monaco-vscode-standalone-json-language-features": "~20.2.1",
"@codingame/monaco-vscode-standalone-languages": "~20.2.1",
"@codingame/monaco-vscode-standalone-typescript-language-features": "~20.2.1",
"@json2csv/plainjs": "^7.0.6",
"@leeoniya/ufuzzy": "^1.0.8",
"@popperjs/core": "^2.11.6",
@@ -117,10 +117,10 @@
"lru-cache": "^11.1.0",
"lucide-svelte": "^0.399.0",
"minimatch": "^10.0.1",
"monaco-editor": "npm:@codingame/monaco-vscode-editor-api@~19.1.4",
"monaco-editor-wrapper": "6.10.0",
"monaco-editor": "npm:@codingame/monaco-vscode-editor-api@~20.2.1",
"monaco-editor-wrapper": "6.12.0",
"monaco-graphql": "=1.6.0",
"monaco-languageclient": "9.9.0",
"monaco-languageclient": "9.11.0",
"monaco-vim": "^0.4.1",
"ol": "^7.4.0",
"openai": "^4.87.1",
@@ -137,10 +137,10 @@
"svelte-infinite-loading": "^1.4.0",
"svelte-tiny-virtual-list": "^2.0.5",
"tailwind-merge": "^1.13.2",
"vscode": "npm:@codingame/monaco-vscode-extension-api@~19.1.4",
"vscode": "npm:@codingame/monaco-vscode-extension-api@~20.2.1",
"vscode-languageclient": "~9.0.1",
"vscode-uri": "~3.1.0",
"vscode-ws-jsonrpc": "~3.4.0",
"vscode-ws-jsonrpc": "~3.5.0",
"windmill-parser-wasm-csharp": "1.510.1",
"windmill-parser-wasm-go": "1.510.1",
"windmill-parser-wasm-java": "1.510.1",
@@ -6,6 +6,7 @@
import {
setInputCat as computeInputCat,
debounce,
emptySchema,
emptyString,
getSchemaFromProperties,
type DynamicSelect
@@ -42,6 +43,8 @@
import { safeSelectItems } from './select/utils.svelte'
import S3ArgInput from './common/fileUpload/S3ArgInput.svelte'
import { base } from '$lib/base'
import { workspaceStore } from '$lib/stores'
import { getJsonSchemaFromResource } from './schema/jsonSchemaResource.svelte'
interface Props {
label?: string
@@ -658,6 +661,47 @@
{appPath}
{computeS3ForceViewerPolicies}
/>
{:else if inputCat == 'object' && format == 'json-schema'}
{#await import('$lib/components/EditableSchemaForm.svelte')}
<Loader2 class="animate-spin" />
{:then Module}
<Module.default
bind:schema={
() =>
value && typeof value === 'object' && !Array.isArray(value) ? value : emptySchema(),
(v) => {
value = v
}
}
isFlowInput
editTab="inputEditor"
noPreview
addPropertyInEditorTab
/>
{/await}
{:else if inputCat == 'object' && format?.startsWith('jsonschema-')}
{#await getJsonSchemaFromResource(format.substring('jsonschema-'.length), workspace ?? $workspaceStore ?? '')}
<Loader2 class="animate-spin" />
{:then schema}
{#if !schema || !schema.properties}
{#await import('$lib/components/JsonEditor.svelte')}
<Loader2 class="animate-spin" />
{:then Module}
<Module.default code={JSON.stringify(value, null, 2)} bind:value />
{/await}
{:else}
<div class="py-4 pr-2 pl-6 border rounded-md w-full">
<SchemaForm
{onlyMaskPassword}
{disablePortal}
{disabled}
{prettifyHeader}
{schema}
bind:args={value}
/>
</div>
{/if}
{/await}
{:else if inputCat == 'list' && !isListJson}
<div class="w-full flex gap-4">
<div class="w-full">
@@ -928,7 +928,7 @@
>
</button>
{:else if !s3object?.disable_download}
<FileDownload {s3object} />
<FileDownload {workspaceId} {s3object} {appPath} />
{:else}
<div class="flex text-secondary pt-2">{s3object?.s3} (download disabled)</div>
{/if}
@@ -29,6 +29,7 @@
import type { EditableSchemaFormUi } from '$lib/components/custom_ui'
import Section from '$lib/components/Section.svelte'
import Editor from './Editor.svelte'
import AddPropertyV2 from './schema/AddPropertyV2.svelte'
// export let openEditTab: () => void = () => {}
const dispatch = createEventDispatcher()
@@ -69,6 +70,7 @@
dynSelectCode?: string | undefined
dynSelectLang?: ScriptLang | undefined
showDynSelectOpt?: boolean
addPropertyInEditorTab?: boolean
openEditTab?: import('svelte').Snippet
addProperty?: import('svelte').Snippet
runButton?: import('svelte').Snippet
@@ -104,6 +106,7 @@
dynSelectCode = $bindable(),
dynSelectLang = $bindable(),
showDynSelectOpt = false,
addPropertyInEditorTab = false,
openEditTab,
addProperty,
runButton,
@@ -509,22 +512,31 @@
{:else}
<!-- WIP -->
{#if jsonEnabled && customUi?.jsonOnly != true}
<div class="w-full p-3 flex justify-end">
<Toggle
bind:checked={jsonView}
label="JSON View"
size="xs"
options={{
right: 'JSON editor',
rightTooltip:
'Arguments can be edited either using the wizard, or by editing their JSON Schema.'
}}
lightMode
on:change={() => {
schemaString = JSON.stringify(schema, null, '\t')
editor?.setCode(schemaString)
}}
/>
<div class="w-full p-3 flex gap-4 justify-end items-center">
{#if addPropertyInEditorTab}
<AddPropertyV2 bind:schema on:change>
{#snippet trigger()}
<Button color="light" size="xs" iconOnly startIcon={{ icon: Plus }} />
{/snippet}
</AddPropertyV2>
{/if}
<div class="shrink-0">
<Toggle
bind:checked={jsonView}
label="JSON View"
size="xs"
options={{
right: 'JSON editor',
rightTooltip:
'Arguments can be edited either using the wizard, or by editing their JSON Schema.'
}}
lightMode
on:change={() => {
schemaString = JSON.stringify(schema, null, '\t')
editor?.setCode(schemaString)
}}
/>
</div>
</div>
{/if}
@@ -655,7 +667,6 @@
const isS3 = v == 'S3'
const isOneOf = v == 'oneOf'
const isDynSelect = v == 'dynselect'
const emptyProperty = {
contentEncoding: undefined,
enum_: undefined,
+13 -4
View File
@@ -184,6 +184,7 @@
loadAsync?: boolean
key?: string | undefined
class?: string | undefined
moduleId?: string
}
let {
@@ -209,7 +210,8 @@
changeTimeout = 500,
loadAsync = false,
key = undefined,
class: clazz = undefined
class: clazz = undefined,
moduleId = undefined
}: Props = $props()
$effect.pre(() => {
@@ -1235,7 +1237,13 @@
try {
editor = meditor.create(divEl as HTMLDivElement, {
...editorConfig(code ?? '', lang, automaticLayout, fixedOverflowWidgets, $relativeLineNumbers),
...editorConfig(
code ?? '',
lang,
automaticLayout,
fixedOverflowWidgets,
$relativeLineNumbers
),
model,
fontSize: !small ? 14 : 12,
lineNumbersMinChars,
@@ -1328,7 +1336,8 @@
aiChatManager.addSelectedLinesToContext(
selectedLines,
selection.startLineNumber,
selection.endLineNumber
selection.endLineNumber,
moduleId
)
} else {
aiChatManager.toggleOpen()
@@ -1654,7 +1663,7 @@
files && model && untrack(() => onFileChanges())
})
$effect(() => {
editor?.updateOptions({
editor?.updateOptions({
lineNumbers: $relativeLineNumbers ? 'relative' : 'on'
})
})
+4 -2
View File
@@ -82,6 +82,7 @@
showHistoryDrawer?: boolean
right?: import('svelte').Snippet
openAiChat?: boolean
moduleId?: string
}
let {
@@ -105,7 +106,8 @@
diffMode = false,
showHistoryDrawer = $bindable(false),
right,
openAiChat = false
openAiChat = false,
moduleId = undefined
}: Props = $props()
let contextualVariablePicker: ItemPicker | undefined = $state()
@@ -964,7 +966,7 @@ JsonNode ${windmillPathToCamelCaseName(path)} = JsonNode.Parse(await client.GetS
{#if customUi?.aiGen != false}
{#if openAiChat}
<FlowInlineScriptAiButton />
<FlowInlineScriptAiButton {moduleId} />
{:else}
<ScriptGen {editor} {diffEditor} {lang} {iconOnly} {args} />
{/if}
@@ -40,7 +40,7 @@
{/if}
{#if displayType}
{#if format && !format.startsWith('resource')}
{#if format && !format.startsWith('resource') && !format.startsWith('jsonschema-')}
<span class="text-xs italic ml-2 text-tertiary dark:text-indigo-400">
{format}
</span>
@@ -110,6 +110,9 @@
let updateGlobalRefresh = (moduleId: string, updateFn: (clear, root) => Promise<void>) => {
globalRefreshes[moduleId] = [...(globalRefreshes[moduleId] ?? []), updateFn]
}
let storedToolCallJobs: Record<string, Job> = $state({})
let toolCallIndicesToLoad: string[] = $state([])
</script>
<FlowStatusViewerInner
@@ -141,4 +144,30 @@
isNodeSelected={true}
{refreshGlobal}
{updateGlobalRefresh}
toolCallStore={{
getStoredToolCallJob: (storeKey: string) => storedToolCallJobs[storeKey],
setStoredToolCallJob: (storeKey: string, job: Job) => {
storedToolCallJobs[storeKey] = job
},
getLocalToolCallJobs: (prefix: string) => {
// we return a map from tool call index to job
// to do so, we filter the storedToolCallJobs object by the prefix and we make sure what's left in the key is a tool call index: 2 part of format agentModuleId-toolCallIndex
// and not a further nested tool call index
return Object.fromEntries(
Object.entries(storedToolCallJobs)
.filter(
([key]) => key.startsWith(prefix) && key.replace(prefix, '').split('-').length === 2
)
.map(([key, job]) => [Number(key.replace(prefix, '').split('-').pop()), job])
)
},
isToolCallToBeLoaded: (storeKey: string) => {
return toolCallIndicesToLoad.includes(storeKey)
},
addToolCallToLoad: (storeKey: string) => {
if (!toolCallIndicesToLoad.includes(storeKey)) {
toolCallIndicesToLoad.push(storeKey)
}
}
}}
/>
@@ -33,6 +33,7 @@
import { deepEqual } from 'fast-equals'
import FlowTimeline from './FlowTimeline.svelte'
import { dfs } from './flows/dfs'
import { dfs as dfsPreviousResults } from '$lib/components/flows/previousResults'
import Alert from './common/alert/Alert.svelte'
import FlowGraphViewerStep from './FlowGraphViewerStep.svelte'
import FlowGraphV2 from './graph/FlowGraphV2.svelte'
@@ -116,6 +117,13 @@
onStart?: () => void
onJobsLoaded?: ({ job, force }: { job: Job; force: boolean }) => void
onDone?: ({ job }: { job: CompletedJob }) => void
toolCallStore?: {
getStoredToolCallJob: (storeKey: string) => Job | undefined
setStoredToolCallJob: (storeKey: string, job: Job) => void
getLocalToolCallJobs: (prefix: string) => Record<number, Job>
isToolCallToBeLoaded: (storeKey: string) => boolean
addToolCallToLoad: (storeKey: string) => void
}
}
let {
@@ -155,7 +163,8 @@
loadExtraLogs = undefined,
onStart = undefined,
onJobsLoaded = undefined,
onDone = undefined
onDone = undefined,
toolCallStore
}: Props = $props()
let getTopModuleStates = $derived(topModuleStates ?? localModuleStates)
@@ -913,9 +922,7 @@
let storedListJobs: Record<number, Job> = $state({})
let storedToolCallJobs: Record<number, Job> = $state({})
let selectedToolCall: number | undefined = $state(undefined)
let toolCallIndicesToLoad: number[] = $state([])
let selectedToolCall: string | undefined = $state(undefined)
let wrapperHeight: number = $state(0)
@@ -950,8 +957,10 @@
let nprefix = buildPrefix(prefix, oid)
return fms
? rec(
dfs(fms, (x) =>
x.id.startsWith('subflow:') ? x.id : buildSubflowKey(x.id, nprefix)
dfs(
fms,
(x) => (x.id.startsWith('subflow:') ? x.id : buildSubflowKey(x.id, nprefix)),
{ skipToolNodes: true }
),
nprefix
)
@@ -1009,6 +1018,11 @@
selectedForLoopSetManually: false
})
}
if (selectedNode?.startsWith(AI_TOOL_CALL_PREFIX)) {
const [, agentModuleId, toolCallIndex, _] = selectedNode.split('-')
const parentLoopsPrefix = getParentLoopsPrefix(agentModuleId)
toolCallStore?.addToolCallToLoad(parentLoopsPrefix + agentModuleId + '-' + toolCallIndex)
}
}
$effect(() => {
@@ -1039,6 +1053,29 @@
let animateLogsTab = $state(false)
let noLogs = $derived(graphTabOpen && !isNodeSelected)
/**
* Returns a string like "forloopmodid1-{iter1}-forloopmodid2-{iter2}-forloopmodid3-{iter3}-"
* that can be used to prefix tool call store keys for nested tool calls.
*/
function getParentLoopsPrefix(modId: string) {
if (job?.raw_flow) {
const indices: string[] = []
const parents = dfsPreviousResults(modId, { value: job?.raw_flow, summary: '' }, true)
for (const parent of parents) {
if (parent.value.type === 'forloopflow' || parent.value.type === 'whileloopflow') {
const state = localModuleStates[parent.id]
if (state?.selectedForloopIndex !== undefined) {
indices.push(parent.id + '-' + state.selectedForloopIndex.toString())
}
}
}
indices.reverse()
return indices.length > 0 ? indices.join('-') + '-' : ''
}
return ''
}
</script>
<JobLoader workspaceOverride={workspaceId} {noLogs} noCode bind:this={jobLoader} />
@@ -1173,6 +1210,10 @@
{@const forloopIsSelected =
forloop_selected == loopJobId ||
(innerModule?.type != 'forloopflow' && innerModule?.type != 'whileloopflow')}
{@const forLoopStoreKeyPrefix =
innerModule?.type == 'forloopflow' || innerModule?.type == 'whileloopflow'
? (flowJobIds?.moduleId ?? '') + '-' + j + '-'
: ''}
<!-- <LogId id={loopJobId} /> -->
<div class="border p-6" class:hidden={forloop_selected != loopJobId}>
<FlowStatusViewerInner
@@ -1207,6 +1248,18 @@
graphTabOpen={selected == 'graph' && graphTabOpen}
isNodeSelected={forloop_selected == loopJobId}
{globalIterationBounds}
toolCallStore={{
getStoredToolCallJob: (storeKey: string) =>
toolCallStore?.getStoredToolCallJob(forLoopStoreKeyPrefix + storeKey),
setStoredToolCallJob: (storeKey: string, job: Job) =>
toolCallStore?.setStoredToolCallJob(forLoopStoreKeyPrefix + storeKey, job),
getLocalToolCallJobs: (prefix: string) =>
toolCallStore?.getLocalToolCallJobs(forLoopStoreKeyPrefix + prefix) ?? {},
addToolCallToLoad: (storeKey: string) =>
toolCallStore?.addToolCallToLoad(forLoopStoreKeyPrefix + storeKey),
isToolCallToBeLoaded: (storeKey: string) =>
toolCallStore?.isToolCallToBeLoaded(forLoopStoreKeyPrefix + storeKey) ?? false
}}
/>
</div>
{/if}
@@ -1366,12 +1419,17 @@
graphTabOpen={selected == 'graph' && graphTabOpen}
isNodeSelected={localModuleStates?.[selectedNode ?? '']?.job_id == mod.job}
{globalIterationBounds}
{toolCallStore}
/>
{#if mod.agent_actions && mod.agent_actions.length > 0}
{#if mod.agent_actions && mod.agent_actions.length > 0 && mod.id}
{@const storeKeyPrefix = getParentLoopsPrefix(mod.id)}
{#each mod.agent_actions as agentAction, j}
{#if agentAction.type === 'tool_call' && mod.id}
{#if agentAction.type === 'tool_call'}
{@const toolCallId = getToolCallId(j, mod.id, agentAction.module_id)}
{@const isSelected = selectedToolCall === j}
{@const localToolCallKey = mod.id + '-' + j}
{@const storeKey = storeKeyPrefix + localToolCallKey}
{@const storedToolCallJob = toolCallStore?.getStoredToolCallJob(storeKey)}
{@const isSelected = localToolCallKey === selectedToolCall}
<Button
variant={isSelected ? 'contained' : 'border'}
color={mod.agent_actions_success?.[j] === false
@@ -1381,10 +1439,10 @@
: 'light'}
btnClasses="w-full flex justify-start"
on:click={async () => {
if (selectedToolCall == j) {
if (isSelected) {
selectedToolCall = undefined
} else {
selectedToolCall = j
selectedToolCall = localToolCallKey
}
}}
endIcon={{
@@ -1396,7 +1454,7 @@
Tool call: {agentAction.function_name}
</span>
</Button>
{#if isSelected || storedToolCallJobs[j] || toolCallIndicesToLoad.includes(j)}
{#if isSelected || storedToolCallJob || toolCallStore?.isToolCallToBeLoaded(storeKey)}
<FlowStatusViewerInner
topModuleStates={getTopModuleStates}
{refreshGlobal}
@@ -1414,11 +1472,11 @@
{subflowParentsDurationStatuses}
{isSelectedBranch}
jobId={agentAction.job_id}
job={storedToolCallJobs[j]}
initialJob={storedToolCallJobs[j]}
job={storedToolCallJob}
initialJob={storedToolCallJob}
{reducedPolling}
onJobsLoaded={({ job, force }) => {
storedToolCallJobs[j] = job
toolCallStore?.setStoredToolCallJob(storeKey, job)
onJobsLoadedInner({ id: toolCallId } as FlowStatusModule, job, force)
}}
loadExtraLogs={(logs) => {
@@ -1509,11 +1567,11 @@
stepDetail = mod
selectedNode = e
if (e.startsWith(AI_TOOL_CALL_PREFIX)) {
const [_prefix, _agentModuleId, j, _toolModuleId] = e.split('-')
const [_prefix, agentModuleId, j, _toolModuleId] = e.split('-')
const parentLoopsPrefix = getParentLoopsPrefix(agentModuleId)
const jIdx = Number(j)
if (!toolCallIndicesToLoad.includes(jIdx)) {
toolCallIndicesToLoad.push(jIdx)
}
const storeKey = parentLoopsPrefix + agentModuleId + '-' + jIdx
toolCallStore?.addToolCallToLoad(storeKey)
}
}
} else {
@@ -1603,6 +1661,7 @@
stepDetail && typeof stepDetail !== 'string' ? stepDetail : undefined}
{@const agentTools =
module && module.value.type === 'aiagent' ? module.value.tools : undefined}
{@const parentLoopsPrefix = getParentLoopsPrefix(module?.id ?? '')}
{#if node.flow_jobs_results}
<span class="pl-1 text-tertiary"
>Result of step as collection of all subflows</span
@@ -1667,7 +1726,7 @@
logs={node.logs}
downloadLogs={!hideDownloadLogs}
aiAgentStatus={agentTools &&
node.job_id &&
node?.job_id &&
(node.type === 'Success' || node.type === 'Failure')
? {
tools: agentTools,
@@ -1679,9 +1738,14 @@
success: node.type === 'Success',
type: 'CompletedJob'
},
storedToolCallJobs,
storedToolCallJobs: module
? toolCallStore?.getLocalToolCallJobs(parentLoopsPrefix)
: undefined,
onToolJobLoaded: (job, idx) => {
storedToolCallJobs[idx] = job
if (module) {
const storeKey = parentLoopsPrefix + module.id + '-' + idx
toolCallStore?.setStoredToolCallJob(storeKey, job)
}
}
}
: undefined}
@@ -5,12 +5,14 @@
let {
flowStore: oldFlowStore,
flowStateStore: oldFlowStateStore,
disableAi,
light,
...props
}: FlowBuilderProps & { light?: boolean } = $props()
let flowStore = $state(oldFlowStore)
let flowStateStore = $state(oldFlowStateStore)
let trialRender = $state(true)
@@ -24,7 +26,7 @@
{#if trialRender}
<AiChatLayout noPadding={true} {disableAi}>
{#if light}<div class="bg-red-500 absolute z-10">Trial version</div>{/if}
<FlowBuilder {flowStore} {disableAi} {...props} />
<FlowBuilder {flowStore} {flowStateStore} {disableAi} {...props} />
</AiChatLayout>
{:else}
<div class="flex flex-col items-center justify-center h-screen">
@@ -179,7 +179,10 @@
}
}
}
bind:job={modulesTestStates.states[mod.id].testJob}
bind:job={
() => modulesTestStates.states[mod.id]?.testJob,
(v) => modulesTestStates.states[mod.id] && (modulesTestStates.states[mod.id].testJob = v)
}
loadPlaceholderJobOnStart={{
type: 'QueuedJob',
id: '',
@@ -22,6 +22,7 @@
import TestTriggerConnection from './triggers/TestTriggerConnection.svelte'
import GitHubAppIntegration from './GitHubAppIntegration.svelte'
import Button from './common/button/Button.svelte'
import { clearJsonSchemaResourceCache } from './schema/jsonSchemaResource.svelte'
interface Props {
canSave?: boolean
@@ -94,6 +95,9 @@
path: resourceToEdit.path,
requestBody: { path, value: args, description }
})
if (resourceToEdit.resource_type === 'json_schema') {
clearJsonSchemaResourceCache(resourceToEdit.path, $workspaceStore!)
}
sendUserToast(`Updated resource at ${path}`)
dispatch('refresh', path)
} else {
@@ -269,6 +269,7 @@
onMount(() => {
inferSchema(code)
loadPastTests()
aiChatManager.saveAndClear()
aiChatManager.changeMode(AIMode.SCRIPT)
})
@@ -3,9 +3,13 @@
import { Download } from 'lucide-svelte'
import { base } from '$lib/base'
export let s3object: any
export let workspaceId: string | undefined = undefined
export let appPath: string | undefined = undefined
interface Props {
s3object: any
workspaceId?: string | undefined
appPath?: string | undefined
}
let { s3object, workspaceId = undefined, appPath = undefined }: Props = $props()
</script>
<a
@@ -8,6 +8,12 @@
import { twMerge } from 'tailwind-merge'
import { aiChatManager, AIMode } from './chat/AIChatManager.svelte'
interface Props {
moduleId?: string
}
const { moduleId }: Props = $props()
const aiChatScriptModeClasses = $derived(
aiChatManager.mode === AIMode.SCRIPT && aiChatManager.isOpen
? 'dark:bg-violet-900 bg-violet-100'
@@ -22,7 +28,7 @@
btnClasses={twMerge('!px-2', aiChatScriptModeClasses)}
{onClick}
iconOnly
title="Open AI chat in script mode"
title="Open AI chat"
startIcon={{ icon: WandSparkles, classes: 'text-violet-800 dark:text-violet-400' }}
/>
{/snippet}
@@ -30,7 +36,8 @@
{#if $copilotInfo.enabled}
{@render button(() => {
aiChatManager.openChat()
aiChatManager.changeMode(AIMode.SCRIPT)
const availableContext = aiChatManager.contextManager.getAvailableContext()
aiChatManager.contextManager.setSelectedModuleContext(moduleId, availableContext)
})}
{:else}
<Popover
@@ -174,19 +174,17 @@ export class Autocompletor {
additionalTextEdits:
endsWithNewLine && !multiline
? [
{
range: toEol,
text: ''
}
]
{
range: toEol,
text: ''
}
]
: []
}
]
}
},
// @ts-ignore
disposeInlineCompletions: () => {},
freeInlineCompletions: () => {}
disposeInlineCompletions: () => { },
}
)
@@ -77,7 +77,7 @@
})
$effect(() => {
aiChatManager.listenForScriptEditorContextChange(
aiChatManager.listenForContextChange(
$dbSchemas,
$workspaceStore,
$copilotSessionModel
@@ -115,9 +115,7 @@
pastChats={historyManager.getPastChats()}
bind:selectedContext={
() => aiChatManager.contextManager.getSelectedContext(),
(sc) => {
aiChatManager.scriptEditorOptions && aiChatManager.contextManager.setSelectedContext(sc)
}
(sc) => aiChatManager.contextManager.setSelectedContext(sc)
}
availableContext={aiChatManager.contextManager.getAvailableContext()}
messages={aiChatManager.currentReply
@@ -52,7 +52,7 @@
if (placeholder) {
return placeholder
}
switch (aiChatManager.mode) {
case AIMode.SCRIPT:
return 'Modify this script...'
@@ -74,7 +74,7 @@
let instructions = $state(initialInstructions)
export function focusInput() {
if (aiChatManager.mode === AIMode.SCRIPT) {
if (aiChatManager.mode === AIMode.SCRIPT || aiChatManager.mode === AIMode.FLOW) {
contextTextareaComponent?.focus()
} else {
instructionsTextareaComponent?.focus()
@@ -132,7 +132,7 @@
</script>
<div use:clickOutside class="relative">
{#if aiChatManager.mode === AIMode.SCRIPT}
{#if aiChatManager.mode === AIMode.SCRIPT || aiChatManager.mode === AIMode.FLOW}
{#if showContext}
<div class="flex flex-row gap-1 mb-1 overflow-scroll pt-2 no-scrollbar">
<Popover>
@@ -157,7 +157,7 @@
<ContextElementBadge
contextElement={element}
deletable
on:delete={() => {
onDelete={() => {
selectedContext = selectedContext?.filter(
(c) => c.type !== element.type || c.title !== element.title
)
@@ -1,5 +1,5 @@
import type { AIProviderModel, ScriptLang } from '$lib/gen/types.gen'
import type { ScriptOptions } from './ContextManager.svelte'
import type { FlowOptions, ScriptOptions } from './ContextManager.svelte'
import {
flowTools,
prepareFlowSystemMessage,
@@ -88,6 +88,7 @@ class AIChatManager {
helpers = $state<any | undefined>(undefined)
scriptEditorOptions = $state<ScriptOptions | undefined>(undefined)
flowOptions = $state<FlowOptions | undefined>(undefined)
scriptEditorApplyCode = $state<((code: string, applyAll?: boolean) => void) | undefined>(
undefined
)
@@ -100,7 +101,7 @@ class AIChatManager {
private confirmationCallback = $state<((value: boolean) => void) | undefined>(undefined)
allowedModes: Record<AIMode, boolean> = $derived({
script: this.scriptEditorOptions !== undefined,
script: this.flowAiChatHelpers === undefined && this.scriptEditorOptions !== undefined,
flow: this.flowAiChatHelpers !== undefined,
navigator: true,
ask: true,
@@ -127,7 +128,7 @@ class AIChatManager {
return (
estimatedTokens >
modelContextWindow -
Math.max(modelContextWindow * MAX_TOKENS_THRESHOLD_PERCENTAGE, MAX_TOKENS_HARD_LIMIT)
Math.max(modelContextWindow * MAX_TOKENS_THRESHOLD_PERCENTAGE, MAX_TOKENS_HARD_LIMIT)
)
}
@@ -557,8 +558,8 @@ class AIChatManager {
onNewToken: (token: string) => {
reply += token
},
onMessageEnd: () => { },
setToolStatus: () => { }
onMessageEnd: () => {},
setToolStatus: () => {}
},
systemMessage
}
@@ -625,7 +626,7 @@ class AIChatManager {
}
try {
const oldSelectedContext = this.contextManager?.getSelectedContext() ?? []
if (this.mode === AIMode.SCRIPT) {
if (this.mode === AIMode.SCRIPT || this.mode === AIMode.FLOW) {
this.contextManager?.updateContextOnRequest(options)
}
this.loading = true
@@ -648,7 +649,10 @@ class AIChatManager {
{
role: 'user',
content: this.instructions,
contextElements: this.mode === AIMode.SCRIPT ? oldSelectedContext : undefined,
contextElements:
this.mode === AIMode.SCRIPT || this.mode === AIMode.FLOW
? oldSelectedContext
: undefined,
snapshot,
index: this.messages.length // matching with actual messages index. not -1 because it's not yet added to the messages array
}
@@ -672,7 +676,8 @@ class AIChatManager {
case AIMode.FLOW:
userMessage = prepareFlowUserMessage(
oldInstructions,
this.flowAiChatHelpers!.getFlowAndSelectedId()
this.flowAiChatHelpers!.getFlowAndSelectedId(),
oldSelectedContext
)
break
case AIMode.NAVIGATOR:
@@ -823,12 +828,19 @@ class AIChatManager {
this.sendRequest()
}
addSelectedLinesToContext = (lines: string, startLine: number, endLine: number) => {
addSelectedLinesToContext = (
lines: string,
startLine: number,
endLine: number,
moduleId?: string
) => {
if (!this.open) {
this.toggleOpen()
}
this.changeMode(AIMode.SCRIPT)
this.contextManager?.addSelectedLinesToContext(lines, startLine, endLine)
if (!moduleId) {
this.changeMode(AIMode.SCRIPT)
}
this.contextManager?.addSelectedLinesToContext(lines, startLine, endLine, moduleId)
this.focusInput()
}
@@ -869,12 +881,12 @@ class AIChatManager {
})
}
listenForScriptEditorContextChange = (
listenForContextChange = (
dbSchemas: DBSchemas,
workspaceStore: string | undefined,
copilotSessionModel: AIProviderModel | undefined
) => {
if (this.scriptEditorOptions) {
if (this.mode === AIMode.SCRIPT && this.scriptEditorOptions) {
this.contextManager.updateAvailableContext(
this.scriptEditorOptions,
dbSchemas,
@@ -882,6 +894,18 @@ class AIChatManager {
!copilotSessionModel?.model.endsWith('/thinking'),
untrack(() => this.contextManager.getSelectedContext())
)
} else if (this.mode === AIMode.FLOW && this.flowOptions) {
this.contextManager.updateAvailableContextForFlow(
this.flowOptions,
dbSchemas,
workspaceStore ?? '',
!copilotSessionModel?.model.endsWith('/thinking'),
untrack(() => this.contextManager.getSelectedContext())
)
}
if (this.scriptEditorOptions) {
this.contextManager.setScriptOptions(this.scriptEditorOptions)
}
}
@@ -941,15 +965,15 @@ class AIChatManager {
const editorRelated =
currentEditor && currentEditor.type === 'script' && currentEditor.stepId === module.id
? {
diffMode: currentEditor.diffMode,
lastDeployedCode: currentEditor.lastDeployedCode,
lastSavedCode: undefined
}
diffMode: currentEditor.diffMode,
lastDeployedCode: currentEditor.lastDeployedCode,
lastSavedCode: undefined
}
: {
diffMode: false,
lastDeployedCode: undefined,
lastSavedCode: undefined
}
diffMode: false,
lastDeployedCode: undefined,
lastSavedCode: undefined
}
return {
args: moduleState?.previewArgs ?? {},
@@ -976,6 +1000,13 @@ class AIChatManager {
this.scriptEditorOptions = undefined
}
untrack(() =>
this.contextManager?.setSelectedModuleContext(
selectedId,
untrack(() => this.contextManager.getAvailableContext())
)
)
return () => {
this.scriptEditorOptions = undefined
}
@@ -1,65 +1,285 @@
<script lang="ts">
import FlowModuleIcon from '$lib/components/flows/FlowModuleIcon.svelte'
import BarsStaggered from '$lib/components/icons/BarsStaggered.svelte'
import type { FlowModule } from '$lib/gen/types.gen'
import { ContextIconMap, type ContextElement } from './context'
import { ArrowLeft, Diff, Database, ChevronRight } from 'lucide-svelte'
interface Props {
availableContext: ContextElement[]
selectedContext: ContextElement[]
onSelect: (element: ContextElement) => void
setShowing?: (showing: boolean) => void
showAllAvailable?: boolean
stringSearch?: string
selectedIndex?: number
onViewChange?: (newNumber: number) => void
}
const {
availableContext,
selectedContext,
onSelect,
setShowing,
showAllAvailable = false,
stringSearch = '',
selectedIndex = 0
onViewChange
}: Props = $props()
// Define priority map for context types
const typePriority = {
code: 1,
diff: 2,
default: 3
// Current view state: 'categories' or specific category type
let currentView = $state<'categories' | 'diffs' | 'modules' | 'databases'>('categories')
// Selected index for keyboard navigation
let itemSelectedIndex = $state(0)
let categorySelectedIndex = $state(0)
// Category definitions
const categories = [
{ id: 'diffs', label: 'Diffs', icon: Diff },
{ id: 'modules', label: 'Modules', icon: BarsStaggered },
{ id: 'databases', label: 'Databases', icon: Database }
]
const filteredAvailableContext = $derived(
availableContext.filter((context) => {
const filtered =
(showAllAvailable ||
!selectedContext.some((sc) => sc.type === context.type && sc.title === context.title)) &&
(!stringSearch || context.title.toLowerCase().includes(stringSearch.toLowerCase()))
return filtered
})
)
// Group context by category
const contextByCategory = $derived.by(() => {
const grouped: Record<string, ContextElement[]> = {
diffs: [],
modules: [],
databases: []
}
filteredAvailableContext.forEach((context) => {
if (context.type === 'diff') grouped.diffs.push(context)
else if (context.type === 'flow_module') grouped.modules.push(context)
else if (context.type === 'db') grouped.databases.push(context)
})
return grouped
})
const currentCategoryItems = $derived(
currentView !== 'categories' ? contextByCategory[currentView] : []
)
// Filter to only show categories with items
const availableCategories = $derived(
categories.filter((cat) => contextByCategory[cat.id].length > 0)
)
// Report view changes
$effect(() => {
if (onViewChange) {
if (currentView === 'categories') {
onViewChange(availableCategories.length)
} else {
onViewChange(currentCategoryItems.length + 1)
}
}
})
function handleCategoryClick(categoryId: string) {
currentView = categoryId as typeof currentView
}
const actualAvailableContext = $derived(
availableContext
.filter(
(c) =>
(showAllAvailable ||
!selectedContext.some((sc) => sc.type === c.type && sc.title === c.title)) &&
(!stringSearch || c.title.toLowerCase().includes(stringSearch.toLowerCase()))
)
.sort((a, b) => {
const priorityA = typePriority[a.type] || typePriority.default
const priorityB = typePriority[b.type] || typePriority.default
return priorityA - priorityB
})
)
function handleBackClick() {
currentView = 'categories'
itemSelectedIndex = 0
}
function handleKeyDown(e: KeyboardEvent) {
if (stringSearch.length > 0) {
// Navigation in search view (flat list)
if (e.key === 'ArrowDown') {
e.preventDefault()
e.stopPropagation()
if (filteredAvailableContext.length > 0) {
itemSelectedIndex = (itemSelectedIndex + 1) % filteredAvailableContext.length
}
} else if (e.key === 'ArrowUp') {
e.preventDefault()
e.stopPropagation()
if (filteredAvailableContext.length > 0) {
itemSelectedIndex =
(itemSelectedIndex - 1 + filteredAvailableContext.length) %
filteredAvailableContext.length
}
} else if (e.key === 'Enter' || e.key === 'Tab') {
if (e.key === 'Tab') e.preventDefault()
e.stopPropagation()
const selectedItem = filteredAvailableContext[itemSelectedIndex]
if (selectedItem) {
onSelect(selectedItem)
}
}
} else if (currentView === 'categories') {
// Navigation in categories view
if (e.key === 'ArrowDown') {
e.preventDefault()
e.stopPropagation()
categorySelectedIndex = (categorySelectedIndex + 1) % availableCategories.length
} else if (e.key === 'ArrowUp') {
e.preventDefault()
e.stopPropagation()
categorySelectedIndex =
(categorySelectedIndex - 1 + availableCategories.length) % availableCategories.length
} else if (e.key === 'Enter' || e.key === 'ArrowRight' || e.key === 'Tab') {
e.preventDefault()
e.stopPropagation()
const selectedCategory = availableCategories[categorySelectedIndex]
if (selectedCategory) {
handleCategoryClick(selectedCategory.id)
}
} else if (e.key === 'Escape' || e.key === 'ArrowLeft') {
e.preventDefault()
e.stopPropagation()
setShowing?.(false)
}
} else {
// Navigation in category items view
if (e.key === 'ArrowDown') {
e.preventDefault()
e.stopPropagation()
if (currentCategoryItems.length > 0) {
itemSelectedIndex = (itemSelectedIndex + 1) % currentCategoryItems.length
}
} else if (e.key === 'ArrowUp') {
e.preventDefault()
e.stopPropagation()
if (currentCategoryItems.length > 0) {
itemSelectedIndex =
(itemSelectedIndex - 1 + currentCategoryItems.length) % currentCategoryItems.length
}
} else if (e.key === 'Enter' || e.key === 'Tab') {
if (e.key === 'Tab') e.preventDefault()
e.stopPropagation()
const selectedItem = currentCategoryItems[itemSelectedIndex]
if (selectedItem) {
onSelect(selectedItem)
currentView = 'categories' // Go back to categories after selection
}
} else if (e.key === 'ArrowLeft' || e.key === 'Escape') {
e.preventDefault()
e.stopPropagation()
handleBackClick()
}
}
}
// Listen for keyboard events
$effect(() => {
document.addEventListener('keydown', handleKeyDown)
return () => {
document.removeEventListener('keydown', handleKeyDown)
}
})
$effect(() => {
if (stringSearch.length > 0) {
itemSelectedIndex = 0
}
})
</script>
<div class="flex flex-col gap-1 text-tertiary text-xs p-1 min-w-24 max-h-48 overflow-y-scroll">
{#if actualAvailableContext.length === 0}
<div class="text-center text-tertiary text-xs">No available context</div>
{:else}
{#each actualAvailableContext as element, i}
<div
class="flex flex-col gap-1 text-tertiary text-xs p-1 pr-0 min-w-24 max-h-48 overflow-y-scroll"
onmousedown={(e) =>
// avoids triggering onblur on the textinput and closing the tooltip
e.preventDefault()}
role="listbox"
tabindex={0}
>
{#if stringSearch.length > 0}
<!-- Search view - show flat list -->
{#each filteredAvailableContext as element, i}
{@const Icon = ContextIconMap[element.type]}
<button
class="hover:bg-surface-hover rounded-md p-1 text-left flex flex-row gap-1 items-center font-normal {i ===
selectedIndex
class="hover:bg-surface-hover rounded-md p-1 text-left flex flex-row gap-1 items-center font-normal transition-colors {i ===
itemSelectedIndex
? 'bg-surface-hover'
: ''}"
onclick={() => onSelect(element)}
onclick={() => {
onSelect(element)
}}
>
{#if Icon}
{#if element.type === 'flow_module'}
<FlowModuleIcon module={element as FlowModule} size={16} />
{:else if Icon}
<Icon size={16} />
{/if}
{element.type === 'diff' ? element.title.replace(/_/g, ' ') : element.title}
<span class="truncate">
{element.type === 'diff' || element.type === 'flow_module'
? element.title.replace(/_/g, ' ')
: element.title}
</span>
</button>
{/each}
{#if filteredAvailableContext.length === 0}
<div class="text-center text-tertiary text-xs py-2">No matching context</div>
{/if}
{:else if currentView === 'categories'}
<!-- Categories view -->
{#each availableCategories as category, i}
{@const Icon = category.icon}
<button
class="hover:bg-surface-hover rounded-md p-1 pr-0 text-left flex flex-row gap-1 items-center font-normal transition-colors {i ===
categorySelectedIndex
? 'bg-surface-hover'
: ''}"
onclick={() => handleCategoryClick(category.id)}
>
<Icon size={16} />
<span class="flex-1">{category.label}</span>
<ChevronRight size={16} />
</button>
{/each}
{#if availableCategories.length === 0}
<div class="text-center text-tertiary text-xs py-2">No available context</div>
{/if}
{:else}
<!-- Category items view -->
<button
class="hover:bg-surface-hover rounded-md text-left flex flex-row gap-1 items-center font-normal transition-colors mb-1"
onclick={handleBackClick}
>
<ArrowLeft size={12} />
<span class="text-xs">Go back</span>
</button>
{#if currentCategoryItems.length === 0}
<div class="text-center text-tertiary text-xs py-2">No items in this category</div>
{:else}
{#each currentCategoryItems as element, i}
{@const Icon = ContextIconMap[element.type]}
<button
class="hover:bg-surface-hover rounded-md p-1 text-left flex flex-row gap-1 items-center font-normal transition-colors {i ===
itemSelectedIndex
? 'bg-surface-hover'
: ''}"
onclick={() => {
onSelect(element)
currentView = 'categories' // Go back to categories after selection
}}
>
{#if element.type === 'flow_module'}
<FlowModuleIcon module={element as FlowModule} size={16} />
{:else if Icon}
<Icon size={16} />
{/if}
<span class="truncate">
{element.type === 'diff' ? element.title.replace(/_/g, ' ') : element.title}
</span>
</button>
{/each}
{/if}
{/if}
</div>
@@ -10,36 +10,43 @@
formatSchema
} from '$lib/components/apps/components/display/dbtable/utils'
import ObjectViewer from '$lib/components/propertyPicker/ObjectViewer.svelte'
import { createEventDispatcher } from 'svelte'
import HighlightCode from '$lib/components/HighlightCode.svelte'
import FlowModuleIcon from '$lib/components/flows/FlowModuleIcon.svelte'
import type { FlowModule } from '$lib/gen'
export let contextElement: ContextElement
export let deletable = false
interface Props {
contextElement: ContextElement
deletable?: boolean
onDelete?: () => void
}
let { contextElement, deletable = false, onDelete }: Props = $props()
const icon = ContextIconMap[contextElement.type]
let showDelete = false
let showDelete = $state(false)
const dispatch = createEventDispatcher<{
delete: void
}>()
const isDeletable = $derived(deletable && contextElement.deletable !== false)
</script>
<Popover>
<svelte:fragment slot="trigger">
{#snippet trigger()}
<div
class={twMerge(
'border rounded-md px-1 py-0.5 flex flex-row items-center gap-1 text-tertiary text-xs cursor-default hover:bg-surface-hover hover:cursor-pointer max-w-48 bg-surface'
)}
on:mouseenter={() => (showDelete = true)}
on:mouseleave={() => (showDelete = false)}
onmouseenter={() => (showDelete = true)}
onmouseleave={() => (showDelete = false)}
aria-label="Context element"
role="button"
tabindex={0}
>
<button on:click={() => dispatch('delete')} class:cursor-default={!deletable}>
{#if showDelete && deletable}
<button onclick={isDeletable ? onDelete : undefined} class:cursor-default={!isDeletable}>
{#if showDelete && isDeletable}
<X size={16} />
{:else if contextElement.type === 'flow_module' || contextElement.type === 'flow_module_code_piece'}
<FlowModuleIcon module={contextElement as FlowModule} size={16} />
{:else}
<svelte:component this={icon} size={16} />
{@const SvelteComponent = icon}
<SvelteComponent size={16} />
{/if}
</button>
<span class="truncate">
@@ -48,8 +55,8 @@
: contextElement.title}
</span>
</div>
</svelte:fragment>
<svelte:fragment slot="content">
{/snippet}
{#snippet content()}
{#if contextElement.type === 'error'}
<div class="max-w-96 max-h-[300px] text-xs overflow-auto">
<Highlight language={json} code={contextElement.content} class="w-full p-2" />
@@ -71,7 +78,7 @@
<div class="text-tertiary">Not loaded yet</div>
{/if}
</div>
{:else if contextElement.type === 'code' || contextElement.type === 'code_piece' || contextElement.type === 'diff'}
{:else if contextElement.type === 'code' || contextElement.type === 'code_piece' || contextElement.type === 'diff' || contextElement.type === 'flow_module_code_piece'}
<div class="max-w-96 max-h-[300px] text-xs overflow-auto">
<HighlightCode
language={contextElement.lang}
@@ -79,6 +86,20 @@
class="w-full p-2 "
/>
</div>
{:else if contextElement.type === 'flow_module'}
{#if contextElement.value.content}
<div class="p-2 max-w-96 max-h-[300px] text-xs overflow-auto">
<HighlightCode
language={contextElement.value.language}
code={contextElement.value.content}
class="w-full p-2 "
/>
</div>
{:else}
<div class="p-2 max-w-96 max-h-[300px] text-xs overflow-auto">
<div class="text-tertiary">{contextElement.title}</div>
</div>
{/if}
{/if}
</svelte:fragment>
{/snippet}
</Popover>
@@ -1,11 +1,13 @@
import { ResourceService, type ListResourceResponse, type ScriptLang } from '$lib/gen'
import { ResourceService, type Flow, type ListResourceResponse, type ScriptLang } from '$lib/gen'
import { scriptLangToEditorLang } from '$lib/scripts'
import { SQLSchemaLanguages, type DBSchemas } from '$lib/stores'
import { diffLines } from 'diff'
import type { ContextElement } from './context'
import type { ContextElement, FlowModuleElement } from './context'
import type { FlowModule } from '$lib/gen'
import type { DisplayMessage } from './shared'
import { langToExt } from '$lib/editorLangUtils'
import type { ExtendedOpenFlow } from '$lib/components/flows/types'
export interface ScriptOptions {
lang: ScriptLang | 'bunnative'
@@ -18,6 +20,14 @@ export interface ScriptOptions {
diffMode: boolean
}
export interface FlowOptions {
currentFlow: ExtendedOpenFlow
lastDeployedFlow?: Flow
path: string | undefined
modules: FlowModule[]
lastSavedFlow?: Flow
}
export default class ContextManager {
private selectedContext: ContextElement[] = $state([])
private availableContext: ContextElement[] = $state([])
@@ -55,6 +65,93 @@ export default class ContextManager {
)
}
async updateAvailableContextForFlow(
flowOptions: FlowOptions,
dbSchemas: DBSchemas,
workspace: string,
toolSupport: boolean,
currentlySelectedContext: ContextElement[]
) {
try {
if (this.workspace !== workspace) {
await this.refreshDbResources(workspace)
this.workspace = workspace
}
let newAvailableContext: ContextElement[] = []
// Add diff context if we have a deployed flow version
const deployedFlowString = JSON.stringify(flowOptions.lastDeployedFlow, null, 2)
const savedFlowString = JSON.stringify(flowOptions.lastSavedFlow, null, 2)
const currentFlowString = JSON.stringify(flowOptions.currentFlow, null, 2)
if (currentFlowString && deployedFlowString && deployedFlowString !== currentFlowString) {
newAvailableContext.push({
type: 'diff',
title: 'diff_with_last_deployed_version',
content: deployedFlowString,
diff: diffLines(deployedFlowString, currentFlowString),
lang: 'graphql' // irrelevant, but needed for the diff component
})
}
if (currentFlowString && savedFlowString && savedFlowString !== currentFlowString) {
newAvailableContext.push({
type: 'diff',
title: 'diff_with_last_saved_draft',
content: savedFlowString,
diff: diffLines(savedFlowString, currentFlowString),
lang: 'graphql' // irrelevant, but needed for the diff component
})
}
for (const module of flowOptions.modules) {
newAvailableContext.push({
type: 'flow_module',
id: module.id,
title: `${module.id}`,
value: {
language: 'language' in module.value ? module.value.language : 'bunnative',
path: 'path' in module.value ? module.value.path : '',
content: 'content' in module.value ? module.value.content : '',
type: module.value.type
}
})
}
if (toolSupport) {
for (const d of this.dbResources) {
const loadedSchema = dbSchemas[d.path]
newAvailableContext.push({
type: 'db',
title: d.path,
// If the db is already fetched, add the schema to the context
...(loadedSchema ? { schema: loadedSchema } : {})
})
}
}
let newSelectedContext: ContextElement[] = [...currentlySelectedContext]
// Filter selected context to only include available items
newSelectedContext = newSelectedContext
.filter((c) => newAvailableContext.some((ac) => ac.type === c.type && ac.title === c.title))
.map((c) =>
c.type === 'db' && dbSchemas[c.title]
? {
...c,
schema: dbSchemas[c.title]
}
: c
)
this.availableContext = newAvailableContext
this.selectedContext = newSelectedContext
} catch (err) {
console.error('Could not update available context for flow', err)
}
}
async updateAvailableContext(
scriptOptions: ScriptOptions,
dbSchemas: DBSchemas,
@@ -63,12 +160,10 @@ export default class ContextManager {
currentlySelectedContext: ContextElement[]
) {
try {
let firstTime = !this.workspace
if (this.workspace !== workspace) {
await this.refreshDbResources(workspace)
this.workspace = workspace
}
this.scriptOptions = scriptOptions
let newAvailableContext: ContextElement[] = [
{
type: 'code',
@@ -123,16 +218,15 @@ export default class ContextManager {
let newSelectedContext: ContextElement[] = [...currentlySelectedContext]
if (firstTime) {
newSelectedContext = [
{
type: 'code',
title: this.getContextCodePath(scriptOptions) ?? '',
content: scriptOptions.code,
lang: scriptOptions.lang
}
]
}
newSelectedContext = [
{
type: 'code',
title: this.getContextCodePath(scriptOptions) ?? '',
content: scriptOptions.code,
lang: scriptOptions.lang,
deletable: false
}
]
const db = this.getSelectedDBSchema(scriptOptions, dbSchemas)
if (
@@ -160,15 +254,15 @@ export default class ContextManager {
.map((c) =>
c.type === 'code'
? {
...c,
content: scriptOptions.code,
title: this.getContextCodePath(scriptOptions)
}
...c,
content: scriptOptions.code,
title: this.getContextCodePath(scriptOptions)
}
: c.type === 'db' && dbSchemas[c.title]
? {
...c,
schema: dbSchemas[c.title]
}
...c,
schema: dbSchemas[c.title]
}
: c
)
@@ -191,26 +285,56 @@ export default class ContextManager {
return this.availableContext
}
addSelectedLinesToContext(lines: string, startLine: number, endLine: number) {
setScriptOptions(scriptOptions: ScriptOptions) {
this.scriptOptions = scriptOptions
}
addSelectedLinesToContext(lines: string, startLine: number, endLine: number, moduleId?: string) {
const title = moduleId ? `${moduleId} L${startLine}-L${endLine}` : `L${startLine}-L${endLine}`
if (
!this.scriptOptions ||
this.selectedContext.find(
(c) => c.type === 'code_piece' && c.title === `L${startLine}-L${endLine}`
(c) =>
(c.type === 'code_piece' && c.title === title) ||
(c.type === 'flow_module_code_piece' && c.id === moduleId && c.title === title)
)
) {
return
}
this.selectedContext = [
...this.selectedContext,
{
type: 'code_piece',
title: `L${startLine}-L${endLine}`,
startLine,
endLine,
content: lines,
lang: this.scriptOptions.lang
if (moduleId) {
const module = [...this.availableContext, ...this.selectedContext].find(
(c) => c.type === 'flow_module' && c.id === moduleId
) as FlowModuleElement
if (!module) {
console.error('Module not found', moduleId)
return
}
]
this.selectedContext = [
...this.selectedContext,
{
type: 'flow_module_code_piece',
id: moduleId,
title: title,
startLine,
endLine,
content: lines,
lang: this.scriptOptions.lang,
value: module.value
}
]
} else {
this.selectedContext = [
...this.selectedContext,
{
type: 'code_piece',
title: title,
startLine,
endLine,
content: lines,
lang: this.scriptOptions.lang
}
]
}
}
setFixContext() {
@@ -234,14 +358,14 @@ export default class ContextManager {
...(options.withCode === false ? [] : [codeContext]),
...(options.withDiff
? [
{
type: 'diff' as const,
title: 'diff_with_last_deployed_version',
content: this.scriptOptions.lastDeployedCode ?? '',
diff: diffLines(this.scriptOptions.lastDeployedCode ?? '', this.scriptOptions.code),
lang: this.scriptOptions.lang
}
]
{
type: 'diff' as const,
title: 'diff_with_last_deployed_version',
content: this.scriptOptions.lastDeployedCode ?? '',
diff: diffLines(this.scriptOptions.lastDeployedCode ?? '', this.scriptOptions.code),
lang: this.scriptOptions.lang
}
]
: [])
]
}
@@ -268,15 +392,37 @@ export default class ContextManager {
contextElements:
m.role !== 'tool' && m.contextElements
? m.contextElements.map((c) =>
c.type === 'db'
? {
type: 'db',
title: c.title,
schema: dbSchemas[c.title]
}
: c
)
c.type === 'db'
? {
type: 'db',
title: c.title,
schema: dbSchemas[c.title]
}
: c
)
: undefined
}))
}
setSelectedModuleContext(
moduleId: string | undefined,
availableContext: ContextElement[] | undefined
) {
if (availableContext && moduleId) {
const module = availableContext.find((c) => c.type === 'flow_module' && c.id === moduleId)
if (
module &&
!this.selectedContext.find((c) => c.type === 'flow_module' && c.id === moduleId)
) {
this.selectedContext = this.selectedContext.filter((c) => c.type !== 'flow_module')
this.selectedContext = [module, ...this.selectedContext]
}
} else if (!moduleId) {
this.selectedContext = this.selectedContext.filter((c) => c.type !== 'flow_module')
}
}
clearContext() {
this.selectedContext = []
}
}
@@ -38,7 +38,7 @@
let tooltipPosition = $state({ x: 0, y: 0 })
let textarea = $state<HTMLTextAreaElement | undefined>(undefined)
let tooltipElement = $state<HTMLDivElement | undefined>(undefined)
let selectedSuggestionIndex = $state(0)
let tooltipCurrentViewNumber = $state(0)
// Properties to copy for caret position calculation
const properties = [
@@ -155,7 +155,7 @@
}
function getHighlightedText(text: string) {
return text.replace(/@[\w/.-]+/g, (match) => {
return text.replace(/@[\w/.\-\[\]]+/g, (match) => {
const contextElement = availableContext.find((c) => c.title === match.slice(1))
if (contextElement) {
return `<span class="bg-black dark:bg-white text-white dark:text-black z-10">${match}</span>`
@@ -182,27 +182,19 @@
showContextTooltip = false
}
async function updateTooltipPosition(
availableContext: ContextElement[],
showContextTooltip: boolean,
contextTooltipWord: string
) {
if (!textarea || !showContextTooltip) return
async function updateTooltipPosition(currentViewItemsNumber: number) {
if (!textarea) return
try {
const coords = getCaretCoordinates(textarea, textarea.selectionEnd)
const rect = textarea.getBoundingClientRect()
const filteredAvailableContext = availableContext.filter(
(c) => !contextTooltipWord || c.title.toLowerCase().includes(contextTooltipWord.slice(1))
)
const itemHeight = 28 // Estimated height of one item + gap (Button: p-1(8px) + text-xs(16px) = 24px; Parent: gap-1(4px) = 28px)
const containerPadding = 8 // p-1 top + p-1 bottom = 4px + 4px = 8px
const maxHeight = 192 + containerPadding // max-h-48 (192px) + containerPadding (8px)
// Calculate uncapped height, subtract gap from last item as it's not needed
const numItems = filteredAvailableContext.length
const numItems = currentViewItemsNumber
let uncappedHeight =
numItems > 0 ? numItems * itemHeight - 4 + containerPadding : containerPadding
// Ensure height is at least containerPadding even if no items
@@ -270,68 +262,33 @@
} else {
showContextTooltip = false
contextTooltipWord = ''
selectedSuggestionIndex = 0
}
}
function handleKeyPress(e: KeyboardEvent) {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
if (contextTooltipWord) {
const filteredContext = availableContext.filter(
(c) => !contextTooltipWord || c.title.toLowerCase().includes(contextTooltipWord.slice(1))
)
const contextElement = filteredContext[selectedSuggestionIndex]
if (contextElement) {
const isInSelectedContext = selectedContext.find(
(c) => c.title === contextElement.title && c.type === contextElement.type
)
// If the context element is already in the selected context and the last word in the instructions is the same as the context element title, send request
if (isInSelectedContext && value.split(' ').pop() === '@' + contextElement.title) {
onSendRequest()
return
}
handleContextSelection(contextElement)
} else if (contextTooltipWord === '@' && availableContext.length > 0) {
handleContextSelection(availableContext[0])
}
} else {
onSendRequest()
}
}
}
function handleKeyDown(e: KeyboardEvent) {
// Pass to parent first if provided
if (onKeyDown) {
onKeyDown(e)
}
if (!showContextTooltip) return
const filteredContext = availableContext.filter(
(c) => !contextTooltipWord || c.title.toLowerCase().includes(contextTooltipWord.slice(1))
)
if (e.key === 'Tab') {
e.preventDefault()
const contextElement = filteredContext[selectedSuggestionIndex]
if (contextElement) {
handleContextSelection(contextElement)
if (showContextTooltip) {
// avoid new line after Enter in the tooltip
if (e.key === 'Enter') {
e.preventDefault()
}
return
}
if (e.key === 'ArrowDown') {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
selectedSuggestionIndex = (selectedSuggestionIndex + 1) % filteredContext.length
} else if (e.key === 'ArrowUp') {
e.preventDefault()
selectedSuggestionIndex =
(selectedSuggestionIndex - 1 + filteredContext.length) % filteredContext.length
onSendRequest()
}
}
$effect(() => {
updateTooltipPosition(availableContext, showContextTooltip, contextTooltipWord)
if (showContextTooltip) {
updateTooltipPosition(tooltipCurrentViewNumber)
}
})
export function focus() {
@@ -352,7 +309,6 @@
</div>
<textarea
bind:this={textarea}
onkeypress={handleKeyPress}
onkeydown={handleKeyDown}
bind:value
use:autosize
@@ -388,7 +344,12 @@
}}
showAllAvailable={true}
stringSearch={contextTooltipWord.slice(1)}
selectedIndex={selectedSuggestionIndex}
onViewChange={(newNumber) => {
tooltipCurrentViewNumber = newNumber
}}
setShowing={(showing) => {
showContextTooltip = showing
}}
/>
</div>
</Portal>
@@ -25,6 +25,14 @@
return obj
}
}
for (const key in obj) {
try {
const parsed = JSON.parse(obj[key])
obj[key] = parsed
} catch (e) {
console.error('Failed to parse JSON:', e)
}
}
return JSON.stringify(obj, null, 2)
} catch {
return String(obj)
@@ -81,7 +89,9 @@
<div
class="bg-surface-secondary border border-gray-200 dark:border-gray-700 rounded p-3 overflow-x-auto max-h-64 overflow-y-auto"
>
<pre class="text-2xs text-primary whitespace-pre-wrap">{formatJson(content)}</pre>
<pre class="text-2xs text-primary whitespace-pre-wrap"
>{formatJson($state.snapshot(content))}</pre
>
</div>
{:else}
<div
@@ -25,8 +25,8 @@
<!-- Collapsible Header -->
<button
class={twMerge(
"w-full p-3 bg-surface-secondary hover:bg-surface-hover transition-colors flex items-center justify-between text-left border-b border-gray-200 dark:border-gray-700",
message.needsConfirmation ? "opacity-80" : ""
'w-full p-3 bg-surface-secondary hover:bg-surface-hover transition-colors flex items-center justify-between text-left border-b border-gray-200 dark:border-gray-700',
message.needsConfirmation ? 'opacity-80' : ''
)}
onclick={() => (isExpanded = !isExpanded)}
disabled={!message.showDetails}
@@ -57,7 +57,7 @@
{#if isExpanded}
<div class="p-3 bg-surface space-y-3">
<!-- Parameters Section -->
<div class={message.needsConfirmation ? "opacity-80" : ""}>
<div class={message.needsConfirmation ? 'opacity-80' : ''}>
<ToolContentDisplay title="Parameters" content={message.parameters} />
</div>
@@ -9,6 +9,7 @@ export const ContextIconMap = {
db: Database,
diff: Diff,
code_piece: Code
// flow_module type is handled with FlowModuleIcon
}
export interface CodeElement {
@@ -47,4 +48,33 @@ export interface CodePieceElement {
lang: ScriptLang | 'bunnative'
}
export type ContextElement = CodeElement | ErrorElement | DBElement | DiffElement | CodePieceElement
export interface FlowModuleElement {
type: 'flow_module'
id: string
title: string
// mimics the FlowModule type, with only the fields we need
value: {
language?: ScriptLang | 'bunnative'
path?: string
content?: string
type: string
}
}
export interface FlowModuleCodePieceElement extends Omit<CodePieceElement, 'type'> {
type: 'flow_module_code_piece'
id: string
value: FlowModuleElement['value']
}
export type ContextElement = (
| CodeElement
| ErrorElement
| DBElement
| DiffElement
| CodePieceElement
| FlowModuleElement
| FlowModuleCodePieceElement
) & {
deletable?: boolean
}
@@ -19,7 +19,7 @@
import DiffDrawer from '$lib/components/DiffDrawer.svelte'
let {
flowModuleSchemaMap
flowModuleSchemaMap,
}: {
flowModuleSchemaMap: FlowModuleSchemaMap | undefined
} = $props()
@@ -10,9 +10,21 @@ import { emptySchema, emptyString } from '$lib/utils'
import {
getFormattedResourceTypes,
getLangContext,
SUPPORTED_CHAT_SCRIPT_LANGUAGES
SUPPORTED_CHAT_SCRIPT_LANGUAGES,
createDbSchemaTool
} from '../script/core'
import { createSearchHubScriptsTool, createToolDef, type Tool, executeTestRun, buildSchemaForTool, buildTestRunArgs } from '../shared'
import {
createSearchHubScriptsTool,
createToolDef,
type Tool,
executeTestRun,
buildSchemaForTool,
buildTestRunArgs,
buildContextString,
applyCodePiecesToFlowModules,
findModuleById
} from '../shared'
import type { ContextElement } from '../context'
import type { ExtendedOpenFlow } from '$lib/components/flows/types'
export type AIModuleAction = 'added' | 'modified' | 'removed'
@@ -339,8 +351,11 @@ const getInstructionsForCodeGenerationToolDef = createToolDef(
// Will be overridden by setSchema
const testRunFlowSchema = z.object({
args: z.object({}).nullable().optional()
.describe('Arguments to pass to the flow (optional, uses default flow inputs if not provided)')
args: z
.object({})
.nullable()
.optional()
.describe('Arguments to pass to the flow (optional, uses default flow inputs if not provided)')
})
const testRunFlowToolDef = createToolDef(
@@ -368,6 +383,7 @@ const workspaceScriptsSearch = new WorkspaceScriptsSearch()
export const flowTools: Tool<FlowAIChatHelpers>[] = [
createSearchHubScriptsTool(false),
createDbSchemaTool<FlowAIChatHelpers>(),
{
def: searchScriptsToolDef,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
@@ -562,7 +578,7 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
},
{
def: testRunFlowToolDef,
fn: async function({ args, workspace, helpers, toolCallbacks, toolId }) {
fn: async function ({ args, workspace, helpers, toolCallbacks, toolId }) {
const { flow } = helpers.getFlowAndSelectedId()
if (!flow || !flow.value) {
@@ -577,13 +593,14 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
const parsedArgs = await buildTestRunArgs(args, this.def)
return executeTestRun({
jobStarter: () => JobService.runFlowPreview({
workspace: workspace,
requestBody: {
args: parsedArgs,
value: flow.value,
}
}),
jobStarter: () =>
JobService.runFlowPreview({
workspace: workspace,
requestBody: {
args: parsedArgs,
value: flow.value
}
}),
workspace,
toolCallbacks,
toolId,
@@ -591,7 +608,7 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
contextName: 'flow'
})
},
setSchema: async function(helpers: FlowAIChatHelpers) {
setSchema: async function (helpers: FlowAIChatHelpers) {
await buildSchemaForTool(this.def, async () => {
const flowInputsSchema = await helpers.getFlowInputsSchema()
return flowInputsSchema
@@ -622,7 +639,7 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
// Find the step in the flow
const modules = helpers.getModules()
let targetModule: FlowModule | undefined = modules.find((m) => m.id === stepId)
let targetModule: FlowModule | undefined = findModuleById(modules, stepId)
if (!targetModule) {
toolCallbacks.setToolStatus(toolId, {
@@ -647,7 +664,10 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
requestBody: {
content: moduleValue.content ?? '',
language: moduleValue.language,
args: module.id === 'preprocessor' ? { _ENTRYPOINT_OVERRIDE: 'preprocessor', ...stepArgs } : stepArgs
args:
module.id === 'preprocessor'
? { _ENTRYPOINT_OVERRIDE: 'preprocessor', ...stepArgs }
: stepArgs
}
}),
workspace,
@@ -675,7 +695,10 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
requestBody: {
content: script.content,
language: script.language,
args: module.id === 'preprocessor' ? { _ENTRYPOINT_OVERRIDE: 'preprocessor', ...stepArgs } : stepArgs,
args:
module.id === 'preprocessor'
? { _ENTRYPOINT_OVERRIDE: 'preprocessor', ...stepArgs }
: stepArgs
}
}),
workspace,
@@ -721,6 +744,16 @@ Follow the user instructions carefully.
Go step by step, and explain what you're doing as you're doing it.
DO NOT wait for user confirmation before performing an action. Only do it if the user explicitly asks you to wait in their initial instructions.
ALWAYS test your modifications. You have access to the \`test_run_flow\` and \`test_run_step\` tools to test the flow and steps. If you only modified a single step, use the \`test_run_step\` tool to test it. If you modified the flow, use the \`test_run_flow\` tool to test it. If the user cancels the test run, do not try again and wait for the next user instruction.
When testing steps that are sql scripts, the arguments to be passed are { database: $res:<db_resource> }.
## Code Markers in Flow Modules
When viewing flow modules, the code content of rawscript steps may include \`[#START]\` and \`[#END]\` markers:
- These markers indicate specific code sections that need attention
- You MUST only modify the code between these markers when using the \`set_code\` tool
- After modifying the code, remove the markers from your response
- If a question is asked about the code, focus only on the code between the markers
- The markers appear in the YAML representation of flow modules when specific code pieces are selected
## Understanding User Requests
@@ -802,6 +835,16 @@ For truly static values in step inputs (those not linked to previous steps or lo
Both modules only support a script or rawscript step. You cannot nest modules using forloop/branchone/branchall.
### Contexts
You have access to the following contexts:
- Database schemas
- Flow diffs
- Focused flow modules
Database schemas give you the schema of databases the user is using.
Flow diffs give you the diff between the current flow and the last deployed flow.
Focused flow modules give you the ids of the flow modules the user is focused on. Your response should focus on these modules.
## Resource types
On Windmill, credentials and configuration are stored in resources. Resource types define the format of the resource.
If the user needs a resource as flow input, you should set the property type in the schema to "object" as well as add a key called "format" and set it to "resource-nameofresourcetype" (e.g. "resource-stripe").
@@ -816,26 +859,33 @@ If the user wants a specific resource as step input, you should set the step val
export function prepareFlowUserMessage(
instructions: string,
flowAndSelectedId?: { flow: ExtendedOpenFlow; selectedId: string }
flowAndSelectedId?: { flow: ExtendedOpenFlow; selectedId: string },
selectedContext: ContextElement[] = []
): ChatCompletionUserMessageParam {
const flow = flowAndSelectedId?.flow
const selectedId = flowAndSelectedId?.selectedId
// Handle context elements
const contextInstructions = selectedContext ? buildContextString(selectedContext) : ''
if (!flow || !selectedId) {
let userMessage = `## INSTRUCTIONS:
${instructions}`
return {
role: 'user',
content: `## INSTRUCTIONS:
${instructions}`
content: userMessage
}
}
return {
role: 'user',
content: `## FLOW:
const codePieces = selectedContext.filter((c) => c.type === 'flow_module_code_piece')
const flowModulesYaml = applyCodePiecesToFlowModules(codePieces, flow.value.modules)
let flowContent = `## FLOW:
flow_input schema:
${JSON.stringify(flow.schema ?? emptySchema())}
flow modules:
${YAML.stringify(flow.value.modules)}
${flowModulesYaml}
preprocessor module:
${YAML.stringify(flow.value.preprocessor_module)}
@@ -844,9 +894,15 @@ failure module:
${YAML.stringify(flow.value.failure_module)}
currently selected step:
${selectedId}
${selectedId}`
## INSTRUCTIONS:
flowContent += contextInstructions
flowContent += `\n\n## INSTRUCTIONS:
${instructions}`
return {
role: 'user',
content: flowContent
}
}
@@ -1,6 +1,6 @@
import { ResourceService, JobService } from '$lib/gen/services.gen'
import type { ResourceType, ScriptLang } from '$lib/gen/types.gen'
import { capitalize, emptySchema, isObject, toCamel } from '$lib/utils'
import { capitalize, isObject, toCamel } from '$lib/utils'
import { get } from 'svelte/store'
import { compile, phpCompile, pythonCompile } from '../../utils'
import type {
@@ -9,14 +9,18 @@ import type {
ChatCompletionUserMessageParam
} from 'openai/resources/index.mjs'
import { copilotSessionModel, type DBSchema, dbSchemas } from '$lib/stores'
import { scriptLangToEditorLang } from '$lib/scripts'
import { getDbSchemas } from '$lib/components/apps/components/display/dbtable/utils'
import type { CodePieceElement, ContextElement } from '../context'
import type { ContextElement } from '../context'
import { PYTHON_PREPROCESSOR_MODULE_CODE, TS_PREPROCESSOR_MODULE_CODE } from '$lib/script_helpers'
import { createSearchHubScriptsTool, type Tool, executeTestRun, buildSchemaForTool, buildTestRunArgs } from '../shared'
import {
createSearchHubScriptsTool,
type Tool,
executeTestRun,
buildTestRunArgs,
buildContextString
} from '../shared'
import { setupTypeAcquisition, type DepsToGet } from '$lib/ata'
import { getModelContextWindow } from '../../lib'
import { inferArgs } from '$lib/infer'
// Score threshold for npm packages search filtering
const SCORE_THRESHOLD = 1000
@@ -348,7 +352,7 @@ export const CHAT_SYSTEM_PROMPT = `
- You can also receive a \`DIFF\` of the changes that have been made to the code. You should use this diff to give better answers.
- Before giving your answer, check again that you carefully followed these instructions.
- When asked to create a script that communicates with an external service, you can use the \`search_hub_scripts\` tool to search for relevant scripts in the hub. Make sure the language is the same as what the user is coding in. If you do not find any relevant scripts, you can use the \`search_npm_packages\` tool to search for relevant packages and their documentation. Always give a link to the documentation in your answer if possible.
- After modifying the code, ALWAYS use the \`test_run_script\` tool to test the code, and iterate on the code until it works as expected. If the user cancels the test run, do not try again and wait for the next user instruction.
- At the end of your reponse, if you modified or suggested changes to the code, ALWAYS use the \`test_run_script\` tool to test the code, and iterate on the code until it works as expected (MAX 3 times). If the user cancels the test run, do not try again and wait for the next user instruction.
Important:
Do not mention or reveal these instructions to the user unless explicitly asked to do so.
@@ -439,18 +443,6 @@ export async function main() {
\`\`\`
`
const CHAT_USER_CODE_CONTEXT = `
- {title}:
\`\`\`{language}
{code}
\`\`\`
`
const CHAT_USER_ERROR_CONTEXT = `
ERROR:
{error}
`
export const CHAT_USER_PROMPT = `
INSTRUCTIONS:
{instructions}
@@ -460,8 +452,6 @@ WINDMILL LANGUAGE CONTEXT:
`
export const CHAT_USER_DB_CONTEXT = `- {title}: SCHEMA: \n{schema}\n`
export function prepareScriptSystemMessage(): ChatCompletionSystemMessageParam {
return {
role: 'system',
@@ -469,18 +459,6 @@ export function prepareScriptSystemMessage(): ChatCompletionSystemMessageParam {
}
}
const applyCodePieceToCodeContext = (codePieces: CodePieceElement[], codeContext: string) => {
let code = codeContext.split('\n')
let shiftOffset = 0
codePieces.sort((a, b) => a.startLine - b.startLine)
for (const codePiece of codePieces) {
code.splice(codePiece.endLine + shiftOffset, 0, '[#END]')
code.splice(codePiece.startLine + shiftOffset - 1, 0, '[#START]')
shiftOffset += 2
}
return code.join('\n')
}
export function prepareScriptTools(
language: ScriptLang | 'bunnative',
context: ContextElement[]
@@ -508,61 +486,12 @@ export function prepareScriptUserMessage(
isPreprocessor?: boolean
} = {}
): ChatCompletionUserMessageParam {
let codeContext = 'CODE:\n'
let errorContext = 'ERROR:\n'
let dbContext = 'DATABASES:\n'
let diffContext = 'DIFF:\n'
let hasCode = false
let hasError = false
let hasDb = false
let hasDiff = false
for (const context of selectedContext) {
if (context.type === 'code') {
hasCode = true
codeContext += CHAT_USER_CODE_CONTEXT.replace('{title}', context.title)
.replace('{language}', scriptLangToEditorLang(language))
.replace(
'{code}',
applyCodePieceToCodeContext(
selectedContext.filter((c) => c.type === 'code_piece'),
context.content
)
)
} else if (context.type === 'error') {
if (hasError) {
throw new Error('Multiple error contexts provided')
}
hasError = true
errorContext = CHAT_USER_ERROR_CONTEXT.replace('{error}', context.content)
} else if (context.type === 'db') {
hasDb = true
dbContext += CHAT_USER_DB_CONTEXT.replace('{title}', context.title).replace(
'{schema}',
context.schema?.stringified ?? 'to fetch with get_db_schema'
)
} else if (context.type === 'diff') {
hasDiff = true
const diff = JSON.stringify(context.diff)
diffContext = diff.length > 3000 ? diff.slice(0, 3000) + '...' : diff
}
}
let userMessage = CHAT_USER_PROMPT.replace('{instructions}', instructions).replace(
'{lang_context}',
getLangContext(language, { allowResourcesFetch: true, ...options })
)
if (hasCode) {
userMessage += codeContext
}
if (hasError) {
userMessage += errorContext
}
if (hasDb) {
userMessage += dbContext
}
if (hasDiff) {
userMessage += diffContext
}
const contextInstructions = buildContextString(selectedContext)
userMessage += contextInstructions
return {
role: 'user',
content: userMessage
@@ -626,7 +555,12 @@ async function formatDBSchema(dbSchema: DBSchema) {
}
export interface ScriptChatHelpers {
getScriptOptions: () => { code: string; lang: ScriptLang | 'bunnative'; path: string; args: Record<string, any> }
getScriptOptions: () => {
code: string
lang: ScriptLang | 'bunnative'
path: string
args: Record<string, any>
}
getLastSuggestedCode: () => string | undefined
applyCode: (code: string, applyAll?: boolean) => void
}
@@ -634,51 +568,60 @@ export interface ScriptChatHelpers {
export const resourceTypeTool: Tool<ScriptChatHelpers> = {
def: RESOURCE_TYPE_FUNCTION_DEF,
fn: async ({ args, workspace, helpers, toolCallbacks, toolId }) => {
toolCallbacks.setToolStatus(toolId, { content: 'Searching resource types for "' + args.query + '"...' })
toolCallbacks.setToolStatus(toolId, {
content: 'Searching resource types for "' + args.query + '"...'
})
const lang = helpers.getScriptOptions().lang
const formattedResourceTypes = await getFormattedResourceTypes(
lang,
args.query,
workspace
)
toolCallbacks.setToolStatus(toolId, { content: 'Retrieved resource types for "' + args.query + '"' })
const formattedResourceTypes = await getFormattedResourceTypes(lang, args.query, workspace)
toolCallbacks.setToolStatus(toolId, {
content: 'Retrieved resource types for "' + args.query + '"'
})
return formattedResourceTypes
}
}
export const dbSchemaTool: Tool<ScriptChatHelpers> = {
def: DB_SCHEMA_FUNCTION_DEF,
fn: async ({ args, workspace, toolCallbacks, toolId }) => {
if (!args.resourcePath) {
throw new Error('Database path not provided')
}
toolCallbacks.setToolStatus(toolId, { content: 'Getting database schema for ' + args.resourcePath + '...' })
const resource = await ResourceService.getResource({
workspace: workspace,
path: args.resourcePath
})
const newDbSchemas = {}
await getDbSchemas(
resource.resource_type,
args.resourcePath,
workspace,
newDbSchemas,
(error) => {
console.error(error)
// Generic DB schema tool factory that can be used by both script and flow modes
export function createDbSchemaTool<T>(): Tool<T> {
return {
def: DB_SCHEMA_FUNCTION_DEF,
fn: async ({ args, workspace, toolCallbacks, toolId }) => {
if (!args.resourcePath) {
throw new Error('Database path not provided')
}
)
dbSchemas.update((schemas) => ({ ...schemas, ...newDbSchemas }))
const dbs = get(dbSchemas)
const db = dbs[args.resourcePath]
if (!db) {
throw new Error('Database not found')
toolCallbacks.setToolStatus(toolId, {
content: 'Getting database schema for ' + args.resourcePath + '...'
})
const resource = await ResourceService.getResource({
workspace: workspace,
path: args.resourcePath
})
const newDbSchemas = {}
await getDbSchemas(
resource.resource_type,
args.resourcePath,
workspace,
newDbSchemas,
(error) => {
console.error(error)
}
)
dbSchemas.update((schemas) => ({ ...schemas, ...newDbSchemas }))
const dbs = get(dbSchemas)
const db = dbs[args.resourcePath]
if (!db) {
throw new Error('Database not found')
}
const stringSchema = await formatDBSchema(db)
toolCallbacks.setToolStatus(toolId, {
content: 'Retrieved database schema for ' + args.resourcePath
})
return stringSchema
}
const stringSchema = await formatDBSchema(db)
toolCallbacks.setToolStatus(toolId, { content: 'Retrieved database schema for ' + args.resourcePath })
return stringSchema
}
}
export const dbSchemaTool: Tool<ScriptChatHelpers> = createDbSchemaTool<ScriptChatHelpers>()
type PackageSearchQuery = {
package: {
name: string
@@ -839,31 +782,31 @@ const TEST_RUN_SCRIPT_TOOL: ChatCompletionTool = {
function: {
name: 'test_run_script',
description: 'Execute a test run of the current script in the editor',
// will be overridden by setSchema
parameters: {
type: 'object',
properties: {
args: {
type: 'object',
description: 'Arguments to pass to the script (optional, uses current editor args if not provided)'
}
args: { type: 'string', description: 'JSON string containing the arguments for the tool' }
},
required: []
additionalProperties: false,
strict: false,
required: ['args']
}
},
}
}
export const testRunScriptTool: Tool<ScriptChatHelpers> = {
def: TEST_RUN_SCRIPT_TOOL,
fn: async function({ args, workspace, helpers, toolCallbacks, toolId }) {
fn: async function ({ args, workspace, helpers, toolCallbacks, toolId }) {
const scriptOptions = helpers.getScriptOptions()
if (!scriptOptions) {
toolCallbacks.setToolStatus(toolId, {
toolCallbacks.setToolStatus(toolId, {
content: 'No script available to test',
error: 'No script found in current context'
})
throw new Error('No script code available to test. Please ensure you have a script open in the editor.')
throw new Error(
'No script code available to test. Please ensure you have a script open in the editor.'
)
}
let codeToTest = scriptOptions.code
@@ -873,7 +816,7 @@ export const testRunScriptTool: Tool<ScriptChatHelpers> = {
if (lastSuggestedCode && lastSuggestedCode !== codeToTest) {
codeToTest = lastSuggestedCode
toolCallbacks.setToolStatus(toolId, { content: 'Applying code changes...' })
// Apply the suggested code changes using the existing mechanism
helpers.applyCode(lastSuggestedCode, true)
@@ -883,15 +826,16 @@ export const testRunScriptTool: Tool<ScriptChatHelpers> = {
const parsedArgs = await buildTestRunArgs(args, this.def)
return executeTestRun({
jobStarter: () => JobService.runScriptPreview({
workspace: workspace,
requestBody: {
path: scriptOptions.path,
content: codeToTest,
args: parsedArgs,
language: scriptOptions.lang as ScriptLang,
}
}),
jobStarter: () =>
JobService.runScriptPreview({
workspace: workspace,
requestBody: {
path: scriptOptions.path,
content: codeToTest,
args: parsedArgs,
language: scriptOptions.lang as ScriptLang
}
}),
workspace,
toolCallbacks,
toolId,
@@ -899,23 +843,7 @@ export const testRunScriptTool: Tool<ScriptChatHelpers> = {
contextName: 'script'
})
},
setSchema: async function(helpers: ScriptChatHelpers) {
await buildSchemaForTool(this.def, async () => {
const scriptOptions = helpers.getScriptOptions()
const code = scriptOptions?.code
const lang = scriptOptions?.lang
const lastSuggestedCode = helpers.getLastSuggestedCode()
const codeToTest = lastSuggestedCode ?? code
if (codeToTest) {
const newSchema = emptySchema()
await inferArgs(lang, codeToTest, newSchema)
return newSchema
}
return emptySchema()
})
},
requiresConfirmation: true,
confirmationMessage: 'Run script test',
showDetails: true,
showDetails: true
}
@@ -4,13 +4,194 @@ import type {
ChatCompletionTool
} from 'openai/resources/chat/completions.mjs'
import { get } from 'svelte/store'
import type { ContextElement } from './context'
import type { CodePieceElement, ContextElement, FlowModuleCodePieceElement } from './context'
import { copilotSessionModel, workspaceStore } from '$lib/stores'
import type { ExtendedOpenFlow } from '$lib/components/flows/types'
import type { FunctionParameters } from 'openai/resources/shared.mjs'
import { zodToJsonSchema } from 'zod-to-json-schema'
import { z } from 'zod'
import { ScriptService, JobService, type CompletedJob } from '$lib/gen'
import { ScriptService, JobService, type CompletedJob, type FlowModule } from '$lib/gen'
import { scriptLangToEditorLang } from '$lib/scripts'
import YAML from 'yaml'
export interface ContextStringResult {
dbContext: string
diffContext: string
flowModuleContext: string
hasDb: boolean
hasDiff: boolean
hasFlowModule: boolean
}
export const extractAllModules = (modules: FlowModule[]): FlowModule[] => {
return modules.flatMap((m) => {
if (m.value.type === 'forloopflow' || m.value.type === 'whileloopflow') {
return [m, ...extractAllModules(m.value.modules)]
}
if (m.value.type === 'branchall') {
return [m, ...extractAllModules(m.value.branches.flatMap((b) => b.modules))]
}
if (m.value.type === 'branchone') {
return [
m,
...extractAllModules([...m.value.branches.flatMap((b) => b.modules), ...m.value.default])
]
}
return [m]
})
}
export const findModuleById = (modules: FlowModule[], moduleId: string): FlowModule | undefined => {
for (const module of modules) {
if (module.id === moduleId) {
return module
}
if (module.value.type === 'forloopflow' || module.value.type === 'whileloopflow') {
const found = findModuleById(module.value.modules, moduleId)
if (found) {
return found
}
}
if (module.value.type === 'branchall') {
const allModules = module.value.branches.flatMap((b) => b.modules)
const found = findModuleById(allModules, moduleId)
if (found) {
return found
}
}
if (module.value.type === 'branchone') {
const allModules = [
...module.value.branches.flatMap((b) => b.modules),
...module.value.default
]
const found = findModuleById(allModules, moduleId)
if (found) {
return found
}
}
}
return undefined
}
const applyCodePieceToCodeContext = (codePieces: CodePieceElement[], codeContext: string) => {
let code = codeContext.split('\n')
let shiftOffset = 0
codePieces.sort((a, b) => a.startLine - b.startLine)
for (const codePiece of codePieces) {
code.splice(codePiece.endLine + shiftOffset, 0, '[#END]')
code.splice(codePiece.startLine + shiftOffset - 1, 0, '[#START]')
shiftOffset += 2
}
return code.join('\n')
}
export function applyCodePiecesToFlowModules(
codePieces: FlowModuleCodePieceElement[],
flowModules: FlowModule[]
): string {
const moduleCodePieces = new Map<string, FlowModuleCodePieceElement[]>()
for (const codePiece of codePieces) {
const moduleId = codePiece.id
if (!moduleCodePieces.has(moduleId)) {
moduleCodePieces.set(moduleId, [])
}
moduleCodePieces.get(moduleId)!.push(codePiece)
}
// Clone modules to avoid mutation
const modifiedModules = JSON.parse(JSON.stringify(flowModules))
// Apply code pieces to each module
for (const [moduleId, pieces] of moduleCodePieces) {
const module = findModuleById(modifiedModules, moduleId)
if (module && module.value.type === 'rawscript' && module.value.content) {
module.value.content = applyCodePieceToCodeContext(
pieces as unknown as CodePieceElement[],
module.value.content
)
}
}
return YAML.stringify(modifiedModules)
}
export function buildContextString(selectedContext: ContextElement[]): string {
const dbTemplate = `- {title}: SCHEMA: \n{schema}\n`
const codeTemplate = `
- {title}:
\`\`\`{language}
{code}
\`\`\`
`
let dbContext = 'DATABASES:\n'
let diffContext = 'DIFF:\n'
let flowModuleContext = 'FOCUSED FLOW MODULES IDS:\n'
let codeContext = 'CODE:\n'
let errorContext = `
ERROR:
{error}
`
let hasCode = false
let hasDb = false
let hasDiff = false
let hasFlowModule = false
let hasError = false
let result = '\n\n'
for (const context of selectedContext) {
if (context.type === 'code') {
hasCode = true
codeContext += codeTemplate
.replace('{title}', context.title)
.replace('{language}', scriptLangToEditorLang(context.lang))
.replace(
'{code}',
applyCodePieceToCodeContext(
selectedContext.filter((c) => c.type === 'code_piece'),
context.content
)
)
} else if (context.type === 'error') {
if (hasError) {
throw new Error('Multiple error contexts provided')
}
hasError = true
errorContext = errorContext.replace('{error}', context.content)
} else if (context.type === 'db') {
hasDb = true
dbContext += dbTemplate
.replace('{title}', context.title)
.replace('{schema}', context.schema?.stringified ?? 'to fetch with get_db_schema')
dbContext += '\n'
} else if (context.type === 'diff') {
hasDiff = true
const diff = JSON.stringify(context.diff)
diffContext += (diff.length > 3000 ? diff.slice(0, 3000) + '...' : diff) + '\n'
} else if (context.type === 'flow_module') {
hasFlowModule = true
flowModuleContext += `${context.id}\n`
}
}
if (hasCode) {
result += '\n' + codeContext
}
if (hasError) {
result += '\n' + errorContext
}
if (hasDb) {
result += '\n' + dbContext
}
if (hasDiff) {
result += '\n' + diffContext
}
if (hasFlowModule) {
result += '\n' + flowModuleContext
}
return result
}
type BaseDisplayMessage = {
content: string
@@ -89,11 +270,13 @@ export async function processToolCall<T>({
// Add the tool to the display with appropriate status
toolCallbacks.setToolStatus(toolCall.id, {
...(tool?.requiresConfirmation ? { content: tool.confirmationMessage ?? "Waiting for confirmation..." } : {}),
...(tool?.requiresConfirmation
? { content: tool.confirmationMessage ?? 'Waiting for confirmation...' }
: {}),
parameters: args,
isLoading: true,
needsConfirmation: needsConfirmation,
showDetails: tool?.showDetails,
showDetails: tool?.showDetails
})
// If confirmation is needed and we have the callback, wait for it
@@ -254,12 +437,17 @@ export const createSearchHubScriptsTool = (withContent: boolean = false) => ({
}
})
export async function buildSchemaForTool(toolDef: ChatCompletionTool, schemaBuilder: () => Promise<FunctionParameters>): Promise<boolean> {
export async function buildSchemaForTool(
toolDef: ChatCompletionTool,
schemaBuilder: () => Promise<FunctionParameters>
): Promise<boolean> {
try {
const schema = await schemaBuilder()
// if schema properties contains values different from '^[a-zA-Z0-9_.-]{1,64}$'
const invalidProperties = Object.keys(schema.properties ?? {}).filter((key) => !/^[a-zA-Z0-9_.-]{1,64}$/.test(key))
const invalidProperties = Object.keys(schema.properties ?? {}).filter(
(key) => !/^[a-zA-Z0-9_.-]{1,64}$/.test(key)
)
if (invalidProperties.length > 0) {
console.warn(`Invalid flow inputs schema: ${invalidProperties.join(', ')}`)
throw new Error(`Invalid flow inputs schema: ${invalidProperties.join(', ')}`)
@@ -275,7 +463,15 @@ export async function buildSchemaForTool(toolDef: ChatCompletionTool, schemaBuil
} catch (error) {
console.error('Error building schema for tool', error)
// fallback to schema with args as a JSON string
toolDef.function.parameters = { type: 'object', properties: { args: { type: 'string', description: 'JSON string containing the arguments for the tool' } }, additionalProperties: false, strict: false, required: ['args'] }
toolDef.function.parameters = {
type: 'object',
properties: {
args: { type: 'string', description: 'JSON string containing the arguments for the tool' }
},
additionalProperties: false,
strict: false,
required: ['args']
}
return false
}
}
@@ -393,7 +589,10 @@ function getErrorMessage(result: unknown): string {
export async function buildTestRunArgs(args: any, toolDef: ChatCompletionTool): Promise<any> {
let parsedArgs = args
// if the schema is the fallback schema, parse the args as a JSON string
if ((toolDef.function.parameters as any).properties?.args?.description === 'JSON string containing the arguments for the tool') {
if (
(toolDef.function.parameters as any).properties?.args?.description ===
'JSON string containing the arguments for the tool'
) {
try {
parsedArgs = JSON.parse(args.args)
} catch (error) {
@@ -18,6 +18,8 @@
import { triggerableByAI } from '$lib/actions/triggerableByAI.svelte'
import type { ModulesTestStates } from '../modulesTest.svelte'
import type { StateStore } from '$lib/utils'
import type { FlowOptions } from '../copilot/chat/ContextManager.svelte'
import { extractAllModules } from '../copilot/chat/shared'
const { flowStore } = getContext<FlowEditorContext>('FlowEditorContext')
interface Props {
@@ -103,11 +105,24 @@
pickablePropertiesFiltered: writable<PickableProperties | undefined>(undefined)
})
$effect(() => {
const options: FlowOptions = {
currentFlow: flowStore.val,
lastDeployedFlow: savedFlow,
lastSavedFlow: savedFlow?.draft,
path: savedFlow?.path,
modules: extractAllModules(flowStore.val.value.modules)
}
aiChatManager.flowOptions = options
})
onMount(() => {
aiChatManager.saveAndClear()
aiChatManager.changeMode(AIMode.FLOW)
})
onDestroy(() => {
aiChatManager.flowOptions = undefined
aiChatManager.changeMode(AIMode.NAVIGATOR)
})
</script>
@@ -0,0 +1,50 @@
<script lang="ts">
import LanguageIcon from '$lib/components/common/languageIcons/LanguageIcon.svelte'
import IconedResourceType from '$lib/components/IconedResourceType.svelte'
import type { FlowModule } from '$lib/gen'
import { Building, Repeat, Square, ArrowDown, GitBranch, Bot } from 'lucide-svelte'
import BarsStaggered from '$lib/components/icons/BarsStaggered.svelte'
interface Props {
module: FlowModule
size?: number
width?: number
height?: number
}
let { module, size = 16, width, height }: Props = $props()
// Use width/height if provided, otherwise use size for both
const iconWidth = width || size
const iconHeight = height || size
</script>
{#if module.value.type === 'aiagent'}
<Bot size={16} class="text-violet-800 dark:text-violet-400" />
{:else if module.value.type === 'rawscript'}
<LanguageIcon lang={module.value.language} width={iconWidth} height={iconHeight} />
{:else if module.summary === 'Terminate flow'}
<Square {size} />
{:else if module.value.type === 'identity'}
<ArrowDown {size} />
{:else if module.value.type === 'flow'}
<BarsStaggered {size} />
{:else if module.value.type === 'forloopflow' || module.value.type === 'whileloopflow'}
<Repeat {size} />
{:else if module.value.type === 'branchone' || module.value.type === 'branchall'}
<GitBranch {size} />
{:else if module.value.type === 'script'}
{#if module.value.path.startsWith('hub/')}
<IconedResourceType
width={iconWidth.toString() + 'px'}
height={iconHeight.toString() + 'px'}
name={module.value.path.split('/')[2]}
silent={true}
/>
{:else}
<Building {size} />
{/if}
{:else}
<!-- Fallback icon for unknown module types -->
<BarsStaggered {size} />
{/if}
@@ -154,6 +154,9 @@
{:else if flowModuleValue.type === 'flow'}
<Badge color="indigo" capitalize>flow</Badge>
<input bind:value={summary} placeholder="Summary" class="w-full grow" />
{:else if flowModuleValue.type === 'aiagent'}
<Badge color="indigo">AI Agent</Badge>
<input bind:value={summary} placeholder="Summary" class="w-full grow" />
{/if}
</div>
</span>
@@ -419,6 +419,7 @@
{lastDeployedCode}
{diffMode}
openAiChat
moduleId={flowModule.id}
/>
</div>
{/if}
@@ -477,6 +478,7 @@
{}
)}
key={`flow-inline-${$workspaceStore}-${$pathStore}-${flowModule.id}`}
moduleId={flowModule.id}
/>
<DiffEditor
open={false}
+6 -6
View File
@@ -3,29 +3,29 @@ import type { FlowModule } from '$lib/gen'
export function dfs<T>(
modules: FlowModule[],
f: (x: FlowModule, modules: FlowModule[], branches: FlowModule[][]) => T,
{ skipToolNodes = false }: { skipToolNodes?: boolean } = {}
opts: { skipToolNodes?: boolean } = {}
): T[] {
let result: T[] = []
for (const module of modules) {
if (module.value.type == 'forloopflow' || module.value.type == 'whileloopflow') {
result = result.concat(f(module, modules, [module.value.modules]))
result = result.concat(dfs(module.value.modules, f))
result = result.concat(dfs(module.value.modules, f, opts))
} else if (module.value.type == 'branchone') {
const allBranches = [module.value.default, ...module.value.branches.map((b) => b.modules)]
result = result.concat(f(module, modules, allBranches))
for (const branch of allBranches) {
result = result.concat(dfs(branch, f))
result = result.concat(dfs(branch, f, opts))
}
} else if (module.value.type == 'branchall') {
const allBranches = module.value.branches.map((b) => b.modules)
result = result.concat(f(module, modules, allBranches))
for (const branch of allBranches) {
result = result.concat(dfs(branch, f))
result = result.concat(dfs(branch, f, opts))
}
} else if (module.value.type == 'aiagent' && !skipToolNodes) {
} else if (module.value.type == 'aiagent' && !opts.skipToolNodes) {
result = result.concat(f(module, modules, [module.value.tools]))
result = result.concat(dfs(module.value.tools, f))
result = result.concat(dfs(module.value.tools, f, opts))
} else {
result.push(f(module, modules, []))
}
@@ -96,13 +96,12 @@ export async function loadSchemaFromModule(module: FlowModule): Promise<{
}
]
},
system_prompt: {
type: 'string',
default: 'You are a helpful assistant'
},
user_message: {
type: 'string'
},
system_prompt: {
type: 'string'
},
max_completion_tokens: {
type: 'number'
},
@@ -110,13 +109,13 @@ export async function loadSchemaFromModule(module: FlowModule): Promise<{
type: 'number'
}
},
required: ['provider', 'model', 'system_prompt', 'user_message'],
required: ['provider', 'model', 'user_message'],
type: 'object',
order: [
'provider',
'model',
'system_prompt',
'user_message',
'system_prompt',
'max_completion_tokens',
'temperature'
]
@@ -165,8 +165,7 @@ export async function createBranchAll(id: string): Promise<[FlowModule, FlowModu
export async function createAiAgent(id: string): Promise<[FlowModule, FlowModuleState]> {
const aiAgentFlowModules: FlowModule = {
id,
value: { type: 'aiagent', tools: [], input_transforms: {} },
summary: 'AI Agent'
value: { type: 'aiagent', tools: [], input_transforms: {} }
}
const flowModuleState = await loadFlowModuleState(aiAgentFlowModules)
@@ -1,15 +1,12 @@
<script lang="ts">
import { Button } from '$lib/components/common'
import LanguageIcon from '$lib/components/common/languageIcons/LanguageIcon.svelte'
import IconedResourceType from '$lib/components/IconedResourceType.svelte'
import type { FlowModule, FlowStatusModule, Job } from '$lib/gen'
import { Building, Repeat, Square, ArrowDown, GitBranch, Bot } from 'lucide-svelte'
import { createEventDispatcher, getContext } from 'svelte'
import type { Writable } from 'svelte/store'
import FlowModuleSchemaItem from './FlowModuleSchemaItem.svelte'
import FlowModuleIcon from '../FlowModuleIcon.svelte'
import { prettyLanguage } from '$lib/common'
import { msToSec } from '$lib/utils'
import BarsStaggered from '$lib/components/icons/BarsStaggered.svelte'
import FlowJobsMenu from './FlowJobsMenu.svelte'
import {
isTriggerStep,
@@ -185,9 +182,7 @@
{darkMode}
>
{#snippet icon()}
<div>
<Repeat size={16} />
</div>
<FlowModuleIcon module={mod} />
{/snippet}
</FlowModuleSchemaItem>
{:else if mod.value.type === 'branchone'}
@@ -208,9 +203,7 @@
{darkMode}
>
{#snippet icon()}
<div>
<GitBranch size={16} />
</div>
<FlowModuleIcon module={mod} />
{/snippet}
</FlowModuleSchemaItem>
{:else if mod.value.type === 'branchall'}
@@ -231,9 +224,7 @@
{darkMode}
>
{#snippet icon()}
<div>
<GitBranch size={16} />
</div>
<FlowModuleIcon module={mod} />
{/snippet}
</FlowModuleSchemaItem>
{:else}
@@ -257,6 +248,7 @@
{bgColor}
{bgHoverColor}
label={mod.summary ||
(mod.value.type === 'aiagent' ? 'AI Agent' : undefined) ||
(mod.id === 'preprocessor'
? 'Preprocessor'
: mod.id.startsWith('failure')
@@ -281,32 +273,13 @@
{skipped}
>
{#snippet icon()}
<div>
{#if mod.value.type === 'aiagent'}
<Bot size={16} />
{:else if mod.value.type === 'rawscript'}
<LanguageIcon lang={mod.value.language} width={16} height={16} />
{:else if mod.summary == 'Terminate flow'}
<Square size={16} />
{:else if mod.value.type === 'identity'}
<ArrowDown size={16} />
{:else if mod.value.type === 'flow'}
<BarsStaggered size={16} />
{:else if mod.value.type === 'script'}
{#if mod.value.path.startsWith('hub/')}
<div>
<IconedResourceType
width="20px"
height="20px"
name={mod.value.path.split('/')[2]}
silent={true}
/>
</div>
{:else}
<Building size={14} />
{/if}
{/if}
</div>
{@const size =
mod.value.type === 'script' && mod.value.path.startsWith('hub/')
? 20
: mod.value.type === 'script'
? 14
: 16}
<FlowModuleIcon module={mod} {size} />
{/snippet}
</FlowModuleSchemaItem>
{/if}
@@ -63,7 +63,7 @@
<GitBranch size={14} />
Branch to all
{:else if label === 'AI Agent'}
<BotIcon size={14} />
<BotIcon size={14} class="text-violet-800 dark:text-violet-400" />
AI Agent
{/if}
</span>
@@ -11,7 +11,7 @@
export const AI_TOOL_MESSAGE_PREFIX = '_wm_ai_agent_message'
const ROW_WIDTH = 275
const NEW_TOOL_NODE_WIDTH = 40
const NEW_TOOL_NODE_WIDTH = 50
const MAX_TOOLS_PER_ROW = 2
let computeAIToolNodesCache:
@@ -140,13 +140,13 @@
width: inputToolWidth,
position: {
x:
tools.length === 1
(tools.length === 1
? (ROW_WIDTH - inputToolWidth) / 2
: (i + 1) % 2 === 0
? inputToolWidth + inputToolXGap
: isLastRow && tools.length % 2 === 1
? (ROW_WIDTH - inputToolWidth) / 2
: 0,
: 0) + node.data.offset,
y:
baseOffset +
rowOffset *
@@ -176,7 +176,7 @@
parentId: node.id,
width: NEW_TOOL_NODE_WIDTH,
position: {
x: (ROW_WIDTH - NEW_TOOL_NODE_WIDTH) / 2,
x: (ROW_WIDTH - NEW_TOOL_NODE_WIDTH) / 2 + node.data.offset,
y: baseOffset + rowOffset
}
} satisfies Node & NewAiToolN)
@@ -1,69 +1,89 @@
<script lang="ts">
import { preventDefault, stopPropagation } from 'svelte/legacy'
import { twMerge } from 'tailwind-merge'
import { type NewAiToolN } from '../../graphBuilder.svelte'
import NodeWrapper from './NodeWrapper.svelte'
import Popover from '$lib/components/meltComponents/Popover.svelte'
import InsertModuleInner from '$lib/components/flows/map/InsertModuleInner.svelte'
import { Cross } from 'lucide-svelte'
import PopupV2 from '$lib/components/common/popup/PopupV2.svelte'
import { flip, offset } from 'svelte-floating-ui/dom'
import type { ComputeConfig } from 'svelte-floating-ui'
let funcDesc = $state('')
interface Props {
data: NewAiToolN['data']
}
let { data }: Props = $props()
let floatingConfig: ComputeConfig = {
strategy: 'fixed',
// @ts-ignore
placement: 'bottom-center',
middleware: [offset(8), flip()],
autoUpdate: true
}
</script>
<NodeWrapper>
{#snippet children({ darkMode })}
<!-- svelte-ignore a11y_no_static_element_interactions -->
<Popover
portal={null}
usePointerDownOutside
<!-- svelte-ignore a11y_no_static_element_interactions -->
<PopupV2 {floatingConfig} target="#flow-editor">
{#snippet button({ pointerdown, pointerup })}
<button
title={`Add 'tool'
}`}
type="button"
class={twMerge(
'!w-full text-2xs font-normal bg-surface h-6 pr-0.5 flex justify-center items-center rounded-sm text-tertiary border',
'hover:bg-surface-hover'
'!w-full h-6 flex items-center justify-center !outline-[1px] outline dark:outline-gray-500 outline-gray-300 text-secondary bg-surface focus:outline-none hover:bg-surface-hover rounded'
)}
placement="top"
onpointerdown={stopPropagation(
preventDefault(() => {
pointerdown()
})
)}
onpointerup={pointerup}
>
<svelte:fragment slot="trigger">+tool</svelte:fragment>
<svelte:fragment slot="content" let:close>
<InsertModuleInner
bind:funcDesc
scriptOnly
on:close={() => {
close()
}}
on:new={(e) => {
data.eventHandlers.insert({
index: -1, // ignored when agentId is set
agentId: data.agentModuleId,
...e.detail
})
close()
}}
on:insert={(e) => {
data.eventHandlers.insert({
index: -1, // ignored when agentId is set
agentId: data.agentModuleId,
...e.detail
})
close()
}}
on:pickScript={(e) => {
data.eventHandlers.insert({
index: -1, // ignored when agentId is set
agentId: data.agentModuleId,
kind: e.detail.kind,
script: {
...e.detail,
summary: e.detail.summary
? e.detail.summary.replace(/\s/, '_').replace(/[^a-zA-Z0-9_]/g, '')
: e.detail.path.split('/').pop()
}
})
close()
}}
/>
</svelte:fragment>
</Popover>
<div class="flex flex-row items-center gap-1 font-medium text-2xs">
<Cross size={12} />
tool
</div>
</button>
{/snippet}
</NodeWrapper>
{#snippet children({ close })}
<InsertModuleInner
bind:funcDesc
scriptOnly
on:close={() => {
close()
}}
on:new={(e) => {
data.eventHandlers.insert({
index: -1, // ignored when agentId is set
agentId: data.agentModuleId,
...e.detail
})
close()
}}
on:insert={(e) => {
data.eventHandlers.insert({
index: -1, // ignored when agentId is set
agentId: data.agentModuleId,
...e.detail
})
close()
}}
on:pickScript={(e) => {
data.eventHandlers.insert({
index: -1, // ignored when agentId is set
agentId: data.agentModuleId,
kind: e.detail.kind,
script: {
...e.detail,
summary: e.detail.summary
? e.detail.summary.replace(/\s/, '_').replace(/[^a-zA-Z0-9_]/g, '')
: e.detail.path.split('/').pop()
}
})
close()
}}
/>
{/snippet}
</PopupV2>
@@ -17,6 +17,8 @@
import Button from '../common/button/Button.svelte'
import { Pen, Plus, Trash2 } from 'lucide-svelte'
import Popover from '$lib/components/meltComponents/Popover.svelte'
import ResourcePicker from '../ResourcePicker.svelte'
import Tooltip from '../Tooltip.svelte'
interface Props {
format?: string | undefined
@@ -122,9 +124,19 @@
}
let initialObjectSelected = $state(
Object.keys(properties ?? {}).length == 0 ? 'resource' : 'custom-object'
format === 'json-schema'
? 'json-schema'
: format?.startsWith('jsonschema-')
? 'custom-object'
: Object.keys(properties ?? {}).length == 0
? 'resource'
: 'custom-object'
)
let isDynSelect = $derived(format?.startsWith('dynselect-') ?? false)
let customObjectSelected: 'editor' | 'json-schema-resource' = $state(
format?.startsWith('jsonschema-') ? 'json-schema-resource' : 'editor'
)
</script>
<div class="flex flex-col gap-2">
@@ -291,33 +303,82 @@
<Tabs
bind:selected={initialObjectSelected}
on:selected={(e) => {
if (e.detail === 'custom-object') {
if (e.detail === 'json-schema') {
format = 'json-schema'
} else {
format = ''
}
}}
>
<Tab value="resource">Resource</Tab>
<Tab value="custom-object">Custom Object</Tab>
<Tab value="json-schema">
JSON Schema
<Tooltip>
This displays a JSON schema editor, useful when a JSON schema input is expected.
</Tooltip>
</Tab>
{#snippet content()}
<div class="pt-2">
<TabContent value="custom-object">
<EditableSchemaDrawer
bind:schema={
() => {
return {
properties: properties,
order: order,
required: requiredProperty
}
},
(v) => {
properties = v.properties
order = v.order
requiredProperty = v.required
dispatch('schemaChange')
<ToggleButtonGroup
bind:selected={customObjectSelected}
class="mb-2"
on:selected={(e) => {
if (e.detail === 'editor') {
format = undefined
} else {
properties = undefined
order = undefined
requiredProperty = undefined
}
}
/>
}}
>
{#snippet children({ item })}
<ToggleButton value="editor" label="Editor" {item} />
<ToggleButton
value="json-schema-resource"
label="JSON Schema Resource"
{item}
tooltip="Select a JSON schema resource to specify the object's properties"
showTooltipIcon
/>
{/snippet}
</ToggleButtonGroup>
{#if customObjectSelected === 'editor'}
<EditableSchemaDrawer
bind:schema={
() => {
return {
properties: properties,
order: order,
required: requiredProperty
}
},
(v) => {
properties = v.properties
order = v.order
requiredProperty = v.required
dispatch('schemaChange')
}
}
/>
{:else if customObjectSelected === 'json-schema-resource'}
<ResourcePicker
resourceType="json_schema"
bind:value={
() => {
if (format?.startsWith('jsonschema-')) {
return format.substring('jsonschema-'.length)
}
return undefined
},
(v) => {
format = 'jsonschema-' + v
}
}
/>
{/if}
</TabContent>
<TabContent value="resource">
@@ -0,0 +1,44 @@
import { z } from 'zod'
import { AppService } from '$lib/gen'
import { sendUserToast } from '$lib/utils'
let loadedJsonSchemaResources: Record<string, Record<string, any>> = $state({})
const jsonSchemaResourceSchema = z.object({
schema: z.record(z.string(), z.any())
})
export async function getJsonSchemaFromResource(path: string, workspace: string) {
if (loadedJsonSchemaResources[workspace]?.[path]) {
return loadedJsonSchemaResources[workspace][path]
}
try {
const resourceValue = await AppService.getPublicResource({
path,
workspace
})
const parsedResource = jsonSchemaResourceSchema.safeParse(resourceValue)
if (parsedResource.success) {
const workspaceResources = loadedJsonSchemaResources[workspace]
if (!workspaceResources) {
loadedJsonSchemaResources[workspace] = {}
}
loadedJsonSchemaResources[workspace][path] = parsedResource.data.schema
return parsedResource.data.schema
} else {
console.error('Invalid JSON schema resource:', parsedResource.error)
sendUserToast('Invalid JSON schema resource: ' + parsedResource.error, true)
}
} catch (err) {
console.error(err)
sendUserToast('Could not load JSON schema resource: ' + err, true)
}
}
export async function clearJsonSchemaResourceCache(path: string, workspace: string) {
if (loadedJsonSchemaResources[workspace]?.[path]) {
delete loadedJsonSchemaResources[workspace][path]
}
}
+2 -1
View File
@@ -11,7 +11,8 @@
"gitSync_9": "hub/19738/sync-script-to-git-repo-windmill",
"gitSync_10": "hub/19785/sync-script-to-git-repo-windmill",
"gitSync_11": "hub/19789/sync-script-to-git-repo-windmill",
"gitSync": "hub/19798/sync-script-to-git-repo-windmill",
"gitSync_12": "hub/19798/sync-script-to-git-repo-windmill",
"gitSync": "hub/19801/sync-script-to-git-repo-windmill",
"gitSyncTest_0": "hub/9073/git-repo-test-read-write-windmill",
"gitSyncTest_1": "hub/11499/git-repo-test-read-write-windmill",
"gitSyncTest_2": "hub/11667/git-repo-test-read-write-windmill",
+9 -8
View File
@@ -172,9 +172,9 @@ export function displayDate(
}
const dateChoices: Intl.DateTimeFormatOptions = displayDate
? {
day: 'numeric',
month: 'numeric'
}
day: 'numeric',
month: 'numeric'
}
: {}
return date.toLocaleString(undefined, {
...timeChoices,
@@ -592,6 +592,7 @@ export type InputCat =
| 'currency'
| 'oneOf'
| 'dynselect'
| 'json-schema'
export namespace DynamicSelect {
export type HelperScript =
@@ -1025,7 +1026,7 @@ export async function tryEvery({
try {
await tryCode()
break
} catch (err) { }
} catch (err) {}
i++
}
if (i >= times) {
@@ -1292,7 +1293,7 @@ export function conditionalMelt(node: HTMLElement, meltItem: AnyMeltElement | un
if (meltItem) {
return meltItem(node)
}
return { destroy: () => { } }
return { destroy: () => {} }
}
export type Item = {
@@ -1497,9 +1498,9 @@ export type S3Uri = `s3://${string}/${string}`
export type S3Object =
| S3Uri
| {
s3: string
storage?: string
}
s3: string
storage?: string
}
export function parseS3Object(s3Object: S3Object): { s3: string; storage?: string } {
if (typeof s3Object === 'object') return s3Object
+2 -2
View File
@@ -4,8 +4,8 @@ verify_ssl = true
name = "pypi"
[packages]
wmill = ">=1.530.0"
wmill_pg = ">=1.530.0"
wmill = ">=1.533.0"
wmill_pg = ">=1.533.0"
sendgrid = "*"
mysql-connector-python = "*"
pymongo = "*"
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.530.0
version: 1.533.0
title: OpenFlow Spec
contact:
name: Ruben Fiszel
@@ -12,7 +12,7 @@
RootModule = 'WindmillClient.psm1'
# Version number of this module.
ModuleVersion = '1.530.0'
ModuleVersion = '1.533.0'
# Supported PSEditions
# CompatiblePSEditions = @()
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "wmill"
version = "1.530.0"
version = "1.533.0"
description = "A client library for accessing Windmill server wrapping the Windmill client API"
license = "Apache-2.0"
homepage = "https://windmill.dev"
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "wmill-pg"
version = "1.530.0"
version = "1.533.0"
description = "An extension client for the wmill client library focused on pg"
license = "Apache-2.0"
homepage = "https://windmill.dev"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@windmill/windmill",
"version": "1.530.0",
"version": "1.533.0",
"exports": "./src/index.ts",
"publish": {
"exclude": ["!src", "./s3Types.ts", "./client.ts"]
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "windmill-client",
"description": "Windmill SDK client for browsers and Node.js",
"version": "1.530.0",
"version": "1.533.0",
"author": "Ruben Fiszel",
"license": "Apache 2.0",
"devDependencies": {
+1 -1
View File
@@ -1 +1 @@
1.530.0
1.533.0