Commit Graph
1675 Commits
Author SHA1 Message Date
Ruben Fiszel 0dc84254fc fix(app): improve copy paste of tables with sub-components 2025-08-12 10:01:13 +00:00
Ruben Fiszel 322f68053a fix: improve app decision tree behavior 2025-08-11 15:29:17 +00:00
Ruben Fiszel 1fef46027b remove debug numbers 2025-08-11 12:21:15 +00:00
Ruben Fiszel 13bf33f83c fix: improve app component loading speed 2025-08-11 12:09:45 +00:00
Ruben Fiszel a47463e053 fix(app): improve carousel list recursive error 2025-08-11 10:04:09 +00:00
Ruben Fiszel 47c6386d0f fix(app): improve id handling for transformers 2025-08-11 06:17:47 +00:00
Ruben Fiszel ea2f71d8be fix: improve validate ID for id editors 2025-08-10 23:14:16 +00:00
Ruben Fiszel 2648520b53 fix(app): handle inline script of components with underscore in apps 2025-08-10 23:10:15 +00:00
dieriba 414f099188 feat: add instance-wide workspace prefix option for custom app (#6180) 2025-08-07 17:55:26 +00:00
+1 161dbbc6d6 feat: add native result streaming (#6242)
* feat: add stream output feature to SSE job updates

Adds stream_output field to JobUpdate struct that extracts log lines
starting with '[wm_stream]:' from job logs. Regular logs now exclude
stream lines, which are captured separately for specialized handling.

- Added stream_output: Option<String> field to JobUpdate struct
- Created extract_stream_output_from_logs() function to filter stream lines
- Modified get_job_update_data() to use stream extraction logic
- SSE clients now receive both new_logs and stream_output in job updates

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* feat: rename stream_output to stream and handle newlines as \n

- Renamed `stream_output` field to `stream` in JobUpdate struct
- Updated extract_stream_output_from_logs to extract_stream_from_logs
- Changed stream output to join with literal \n instead of actual newlines
- Stream lines are properly excluded from regular new_logs

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* decision tree nits

* push ee ref

* push ee ref

* fix: fix id renaming in apps

* remove duplicate caching (#6285)

* feat: migrate audit log ids to bigints (blocking migration for EE)

* fix(mcp): add proper check for mcp routes (#6282)

* add proper check for mcp routes

* cleaner

* apply to flow

* fix add checks scopes

---------

Co-authored-by: dieriba <dieriba.pro@gmail.com>

* chore(main): release 1.514.0 (#6283)

* chore(main): release 1.514.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: pin tokio to 1.46.1 and aws-sdks-ts

* pin rustls to 0.23.29 + pin aws-sdk

* chore(main): release 1.514.1 (#6288)

* chore(main): release 1.514.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: improve docker logs collection in docker mode

* support $res: string in form inputs of arrays

* fix import nit

* fix: fix DynSelect

* nits

* fix: resource-type-ts-parser (#6289)

* fix: resource types as arg in typescript handle imported defined types

* Update nix flake (#6291)

* merge

* Small UI fixes (#6294)

* fix step history not refreshing with staticInputs

* fix array of obj not showing up in json editor in test this step

* datatable scales correctly in DisplayResult and scrolling is much more usable

* avoid next button disapearing and changing layout / hurting ux

* nits

* fix bug when renaming module A to B then module C to A, C takes the schema of A

* fix bug with comments in sql repl

* fix aggrid theme randomly not loading

* bindable script

* better delete button in db manager

* property select doesnt exist

* fix all warnings

* delete $flowStateStore[id] on delete

* feat(cli): generate cursor rules on init (#6270)

* create cursor rules on init

* change gen

* add missing resource-type command

* add resource type command in guidance

* add schema option

* revert

* nit

* nit

* add flow guidance

* nit

* chore(main): release 1.515.0 (#6292)

* chore(main): release 1.515.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: improved logs for script

* nits logs

* chore(main): release 1.515.1 (#6295)

* chore(main): release 1.515.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* merge

* even more indexer tracings

* add more tracing logs

* feat: prevent too large results (>500Mb) from OOMing database

* nit naming

* feat: add CA certificate update at startup via environment variable (#6280)

* feat: add CA certificate update at startup via environment variable

Add support for running 'update-ca-certificates' at binary startup
when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true".

- Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup
- Execute update-ca-certificates command if env var is set to "true"
- Log success/failure appropriately with tracing
- Continue startup even if CA certificate update fails
- Non-blocking implementation with proper error handling

Fixes #6279

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: extract CA certificate update logic into separate function

Extract the CA certificate update logic from windmill_main() into a
dedicated update_ca_certificates_if_requested() function for better
code organization and maintainability.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* improvements

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>

* fix: indexer collection of job logs before indexing (#6300)

* Add flume as dependecy for indexer

* Update ee-repo-ref

* Remove flags from cargo.toml

* Update ee-repo-ref

* Update ee-repo-ref

* fix rust sdk build error (#6305)

Signed-off-by: pyranota <pyra@duck.com>

* fix broken audit logs filter (#6304)

* rename to from to

* goto fix

* default to false if field not present operator settings (#6301)

* git sync UI improvements (#6303)

* ui improvements round 1

* modal cleanup

* init

* UI refactor

* UI cleanup + refactor

* legacy cleanup

* success model -> github actions, non-ee warnings

* sqlx

* npm check

* ee warning everywhere

* last comments

* formatting

* no hardcoded theme

* claude review improvemenets

* fix: no process relative imports for scripts with codebase

* fix: sqs oidc authentication disconnect #6307

* handle metadata for new scripts happen after commit

* handle_deployment_metadata in a task

* nits

* chore: add windmill-utils-internal package (#6299)

* add utils package

* naming

* cleaning

* add docs

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* add docs

* chore(main): release 1.516.0 (#6298)

* chore(main): release 1.516.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* merge

* indexer improvements

* upgrade tantivy to 0.24.2

* use tantivy fork

* nit warnings

* fix oss build

* improve indexer

* chore: use windmill-utils-internal for cli (#6297)

* add utils package

* naming

* cleaning

* simplify assignPath

* rename old files

* same for locks

* create on confirm

* default true

* use replaceinlinescripts from utils

* use extractscriptfromflows

* make it compile

* cleaning

* use argsigtojson

* fix

* fix missing await

* cleaner

* cleaning

* cleaning

* use in frontend

* add docs

* testing

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* adapt usage

* draft

* better build script

* fix build

* revert to default creation

* add docs

* remove and rename

* make everything work

* add await

* only if not installed

* add vs code setting

* add to publish action

* fix bc

* safer use of sep

* fix

* do not rename on push

* no publish on release

* use published package on frontend

* nit

* Add dependencies to run sqlx prepare to nix flake (#6309)

* feat(cli): wmill-lock.yaml v2 for easier git merge diffs

* merge

* merge

* all

* all

* rm warnings

* fix styling on aichatinput (#6312)

* fix: use with_capacity back presusre for tantivy directory multipart writes (#6313)

* use with capacity for tantivy directory multi part uploads

* Update ee repo ref

* Update ee-repo-ref

* Update ee-repo-ref

* chore(main): release 1.517.0 (#6310)

* chore(main): release 1.517.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix typo on cli build (#6314)

* cleanup

* feat(utils): add flow.yaml validation function (#6316)

* add validateflow function

* cleaner code

* preprocess json

* cleaning

* create specific package

* cleaning

* add tests

* fix: cleanup concurrency_counter automatically + remove orphans keys automatically

* fix: add disabled support to resource picker in schema forms

* fix: add wm_labels to tracing spans

* all

* merge

* all

* fix: delete empty git connection (#6318)

* fix checks

* bun handling

* all

* all?

* all

* all

* update

* all

* update

* check

* fix history

* all

* all

* all

* Remove leftover debug tracing statements

- Remove commented debug trace in jobs.rs for stream output
- Remove commented debug trace in result_stream.rs for stream processing

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* fix test

* all

* handle iter

* fix

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: dieriba <dieriba.pro@gmail.com>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>
Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com>
2025-08-06 22:40:42 +00:00
e6f1211d31 feat: Ducklake native support (#6268)
* upgrade duckdb

* basic ducklake works

* ducklake works with custom db catalogs

* fix: pwsh skip already installed modules outside of cache (#6037)

* improve query performance of user stats

* separate ducklake_catalog db

* ducklake settings

* DucklakeSettings frontend

* Ducklake ws settings saved in backend

* fetch ducklake catalog resource

* Ducklake works with configured s3 storage

* Ducklake as asset

* ducklake asset icon

* Fix duckdb array and object args not working properly (#6254)

* Fix bug with comments in duckdb

* Avoid multiple queries when doing ATTACH ducklake

* trunc sig no longer needed now that comments are trimmed

* cache DuckdbConnectionSettingsResponse

* duplicated code

* transform_attach_ducklake contributes to duckdb_connection_settings_cache

* eliminate the need for used_storages

* nit

* cleaner management of the bigquery credentials file

* DBManagerDrawer refactor to prepare for Ducklake

* get ducklake schema

* implement delete for ducklake

* load column metadata for ducklake

* Select query works for ducklake, basic db explorer works !

* duckdb count query

* Support all db ops for ducklake

* clean migrations

* SQL repl for Ducklake

* fix broken database studio

* nit

* assert function

* Ducklake in Editor Bar

* default ducklake syntax + allow extra args

* DucklakeCatalogWizard UI

* nit + remove extra $

* modal when databases do not exist

* cannot be windmill

* Ducklake works safely with instance database

* Avoid sending instance db credentials on network

* resource leak security

* remove fetch_attach_db_conn_str

* prevent instance pg password leak

* hide asset usage count when not available

* case unsensitivity duckdb

* warnings

* disable instance catalog

* use shorthand syntax when inserting with EditorBar

* Instance ducklake catalog is now safe to use

* use safer argon2 pwd

* update package json parsers

* update package json

* better msgs

* tooltips

* disable explore button until saved

* nit

* fix warnings

* better ducklake_user password management

* nit

* Sanitize passwords from errors in ducklake

* DisplayResult broken in job result

* remove superadmin requirement to check databases_exist

* duckdb_connection_settings_v2_inner

* Ducklake works on agent worker (finally)

* ci

* #[allow(dead_code)]

* fix openapi missing response

* Separate +Database button for DuckDB in EditorBar

* Fix dropdown in ducklake settings

* Attempt to fix migration race condition in CI

* update sqlx failing for some offline queries

* avoid temp password for ducklake_user

* nits

* ducklake settings nits

* update duckdb default script

* fix sql repl resetting text on refresh

* avoid pgcrypto extension

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-06 13:55:36 +00:00
centdix 17ae1ee160 chore: use windmill-utils-internal for cli (#6297)
* add utils package

* naming

* cleaning

* simplify assignPath

* rename old files

* same for locks

* create on confirm

* default true

* use replaceinlinescripts from utils

* use extractscriptfromflows

* make it compile

* cleaning

* use argsigtojson

* fix

* fix missing await

* cleaner

* cleaning

* cleaning

* use in frontend

* add docs

* testing

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* adapt usage

* draft

* better build script

* fix build

* revert to default creation

* add docs

* remove and rename

* make everything work

* add await

* only if not installed

* add vs code setting

* add to publish action

* fix bc

* safer use of sep

* fix

* do not rename on push

* no publish on release

* use published package on frontend

* nit
2025-07-31 13:23:39 +00:00
Diego Imbert 5af47d29e7 Small UI fixes (#6294)
* fix step history not refreshing with staticInputs

* fix array of obj not showing up in json editor in test this step

* datatable scales correctly in DisplayResult and scrolling is much more usable

* avoid next button disapearing and changing layout / hurting ux

* nits

* fix bug when renaming module A to B then module C to A, C takes the schema of A

* fix bug with comments in sql repl

* fix aggrid theme randomly not loading

* bindable script

* better delete button in db manager

* property select doesnt exist

* fix all warnings

* delete $flowStateStore[id] on delete
2025-07-29 19:28:54 +00:00
Ruben Fiszel 8a2cd1c468 fix import nit 2025-07-29 14:26:55 +00:00
Ruben Fiszel a999bc7b28 fix: fix id renaming in apps 2025-07-27 23:15:19 +00:00
Ruben Fiszel ed1bee9f66 nit 2025-07-24 17:17:59 +00:00
Ruben Fiszel f71104e9f6 fix: improve aggrid columnDefs reactivity 2025-07-24 17:11:36 +00:00
Ruben Fiszel e02f06bf68 decision tree nits 2025-07-23 13:45:41 +00:00
Ruben Fiszel a57df1f781 fix: fix decision tree graph editor 2025-07-23 13:42:04 +00:00
4f993c82b5 feat: implement SSE for job updates polling (#6174)
* feat: implement SSE for job updates polling

- Add /getupdate_sse/:id endpoint for real-time job updates via Server-Sent Events
- SSE streams job status, logs, progress, and flow status updates
- Auto-stops streaming when job completes
- Frontend uses EventSource with graceful fallback to polling on errors
- Reduces server load and improves real-time responsiveness
- Update OpenAPI spec with new SSE endpoint definition

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* all

* all

* v1

* rename test job loader

* iter2

* nit

* all

* all

* improve

* upgrade

* try catch assets json parse

* only display loader of custom component if custom component is rendered

* Fix required properties (#6221)

* whitelabelling nits

* remove fdsfs (#6222)

* add refresh on worker tag select

* whitelabel nits

* merge

* merge

* done

* nit fix

* all

* app db

* all

* all

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Tammo Ippen <tammo.ippen@posteo.de>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
2025-07-18 22:48:51 +00:00
hugocasa 6a012a87ed fix: app s3 multi upload policy (#6228) 2025-07-18 13:14:55 +00:00
Ruben Fiszel 2a739cb04b only display loader of custom component if custom component is rendered 2025-07-18 08:43:31 +00:00
Ruben Fiszel cca8e748aa fix: only close app dropdown when action is done 2025-07-15 13:51:45 +00:00
HugoCasa b58c46a504 feat(frontend): app static and user resource picker default values (#6179) 2025-07-15 09:52:54 +00:00
Ruben Fiszel d0d680bc16 close dropdown on click 2025-07-14 09:08:08 +00:00
Ruben Fiszel 11cf60f4a5 improve edit history behavior (#6166) 2025-07-11 18:19:47 +00:00
Ruben Fiszel 566a9c45d0 fix: fix frontend scripts in app editor copying their content 2025-07-07 15:41:32 +00:00
Ruben Fiszel 3fbd3ec4f9 fix: tag select in script builder top bar (#6136)
* all

* update

* react 18

* update

* update

* update
2025-07-07 10:27:09 +00:00
Alexander Petric 7284c51762 fix: carousel app component, expose current index (#6120)
* feat: carousel app component, expose current index

* minimal diff
2025-07-02 16:31:43 +00:00
Ruben Fiszel 53f615ca0b lazy load nits 2025-06-30 00:07:35 +00:00
Ruben Fiszel a7e78f01f1 fix: optimize public apps rendering 2025-06-29 22:46:27 +00:00
Ruben Fiszel 52ac19642d finish migrating app editor to svelte 5 (#6090)
* all

* all

* all

* all
2025-06-29 19:01:06 +00:00
Ruben Fiszel 6d2c118b3e recompute all fix 2025-06-29 10:09:16 +00:00
Ruben Fiszel 5ac5c873fd app svelte 5 improvements 2025-06-29 09:29:28 +00:00
Ruben Fiszel 8a6426bb4e improve arraystaticinputeditor 2025-06-29 09:03:35 +00:00
Ruben Fiszel 2c7705b78c more app svelte 5 migrations 2025-06-29 08:43:37 +00:00
Ruben Fiszel 021742b9ba fix appDisplayComponentByJobId 2025-06-29 08:18:46 +00:00
Ruben Fiszel c511ad28d0 fix delete issue of app 2025-06-29 07:54:41 +00:00
Ruben Fiszel d293dbca22 svelte 5 nits 2025-06-29 07:43:36 +00:00
Ruben Fiszel 123b17dec9 svelte 5 nits 2025-06-28 21:46:49 +00:00
Ruben Fiszel e6a9f2d593 nits svelte 5 migration 2025-06-28 20:13:55 +02:00
Ruben Fiszel ca368aba7a fix: public url in app menu 2025-06-28 19:41:12 +02:00
Ruben Fiszel 42f56c431f fix resource select broken 2025-06-27 21:46:30 +02:00
ded54f2e68 fix(frontend): improve flow editor settings bar UX (#6049)
* move settings and static inputs into top node

* Move test button in the top nodes

* Revert "Move test button in the top nodes"

This reverts commit 1c8648a538.

* Add error handler to top toolbar

* nit

* polishing

* add flow settings to topbar dropdown

* remove unused files

* progress

* progress

* fixes

* fix

* fix

* fix

* fix

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-06-27 15:26:03 +00:00
Alexander Petric aaf6e9d693 center icon if no label present (#6080) 2025-06-26 21:59:03 +00:00
Ruben Fiszel 18ee03a323 fix: improve ordering of forms for non complete ordering + array schema fix 2025-06-25 20:58:58 +02:00
b21a8da6c6 save editor cursor positions (#6039)
* save global css panel cursor position

* save editor cursor positions globally

* better initialCursorPos + avoid flicker

* unused var err

* editorPositionMap global

* handle cursor pos saving in editors

* better editor keys

* feat: add workspaceStore to all editor key props for better isolation

- Added workspaceStore import to InlineScriptEditor.svelte
- Updated all editor keys to include workspace prefix:
  - CssSettings: `app-global-css-editor-${$workspaceStore}-${$appPath}`
  - InlineScriptEditor: `app-inline-${$workspaceStore}-${$appPath}-${id}`
  - FlowModuleComponent: `flow-inline-${$workspaceStore}-${$pathStore}-${flowModule.id}`

This ensures cursor positions are isolated per workspace for multi-workspace scenarios.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-06-24 18:42:46 +02:00
Ruben Fiszel 4ab7f2919e nit fixes app 2025-06-24 18:21:06 +02:00
HugoCasaandellipsis-dev[bot] 5722014651 feat: ai flow chat prompt and UX improvements (#5942)
* feat: ai flow chat prompt and UX improvements

* fix scroll + better stop button + snapshots base

* tmp

* feat: apply reject

* feat: warning modal + retry last + nits + use flow preview for diff/test/export

* Update frontend/src/lib/components/copilot/chat/flow/core.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* typo

* feat: step summary by AI

* nits

* reactivity nits flow builder

* nits

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-06-24 09:29:38 +02:00
Ruben Fiszel d6a0c026d4 app nits 2025-06-24 09:02:32 +02:00