Commit Graph

2390 Commits

Author SHA1 Message Date
hugocasa dfd857e57c fix(backend): email triggers error handler and retry (#6601) 2025-09-12 20:51:25 +02:00
dieriba 0e82962545 refactor: trigger listener (#6401)
* base

* base crate

* websocket updated

* crud refactored

* fix and nits

* fix compiler warning, nits and update repo ref

* fix oss

* fix compilation

* update ref

* fix

* update feature

* listener base

* add listener

* refactor logic done and implemented for postgres

* fix capture

* websocket

* implem for all triggers

* update sqlx,repo ref and fix

* fix oss

* unify struct fix websocket

* nits and fix oss runtime axum error

* perf cache query

* add listener module

* update .sqlx and cargo.toml

* fix import

* fix

* fix

* update repo ref

* fix

* update ref

* update ref

* fix

* fix

* revert

* update ref
2025-09-12 18:10:58 +00:00
Ruben Fiszel 69222c62c4 fix: custom tag helper 2025-09-12 13:16:00 +00:00
Ruben Fiszel e852925b23 chore(main): release 1.542.3 (#6593)
* chore(main): release 1.542.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-11 21:58:45 +00:00
Ruben Fiszel c48f7ab754 nit 2025-09-11 21:57:56 +00:00
Ruben Fiszel 851ba1590e fix: catchPanicLayer to handle axum panics more gracefully + onFailure tracing 2025-09-11 21:54:28 +00:00
Ruben Fiszel 3fb7765c3e fix(perf): improve perf and reliablity using tcp_nodelay and content-length for intra worker requests 2025-09-11 21:39:27 +00:00
Ruben Fiszel 4b6c057033 chore(main): release 1.542.2 (#6592)
* chore(main): release 1.542.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-11 18:13:38 +00:00
Ruben Fiszel f0c3d82077 fix: archive by hash workspace specificity 2025-09-11 18:01:52 +00:00
Ruben Fiszel ece0cea4d9 chore(main): release 1.542.1 (#6581)
* chore(main): release 1.542.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-11 13:13:19 +02:00
Ruben Fiszel 23079ecd9f chore(main): release 1.542.0 (#6576)
* chore(main): release 1.542.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-11 00:36:46 +00:00
claude[bot] 404d7b8a4f feat: show position of job in queue when waiting for executor (#6554)
* feat: show position of job in queue when waiting for executor

- Added new API endpoint /queue/position/:id to get job's position in queue
- Modified DisplayResult.svelte to fetch and display queue position
- Shows 'Waiting for executor (position X in queue)' when job is queued
- Refreshes position every 2 seconds while waiting

Fixes #6553

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* iterate

* iterate

* all

* all

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-09-11 00:13:52 +00:00
wendrul 94045bb6fb workspace forks: fix compilation issue (#6578) 2025-09-10 21:50:03 +00:00
wendrul 0bf200d26c feat: forkables workspaces v0 (#6479)
* Add create_ephemeral workspace endpoint

* Add cli devShell

* List ephemeral workspaces + improve endpoint

* Add postgres function to clone a workspace (to be revisited)

* Clone workspace using the postgres function

* Add first iteration of ephemeral workspaces command

* Update display of forked workspaces

* Remove SQLX_OFFLINE

* Add UI to create ephemeral workspace

* Add option to exclude repository from being inherited to forks

* WIP: reworking cloning logic

* Fix cloning

* Fix redirect after creating fork

* Clean up cloning behaviour

* Rename ephemeral to fork

* emove ephemeral_workspaces table in favour of columns in  workspaces

* Fix display of forked workspaces

* Fix skip inherit git sync repo setting

* Fix fork invite display + creating fork as user

* Fix SideMenu bug

* Fix alignment

* Simplify migrations

* Update deletion of workspaces

* Delete forked workspace from cli

* Deleting fork workspaces from the UI as non-admin

* Update cli sync and fork creation to adapt to branches and forks

* Update fork prefix

* Remove skip tracking toggle

* Fix npm check warnings

* Fix last npm check

* fix: force stdin to Stdio::null for all user code execution (#6575)

Set stdin to Stdio::null for all Commands that execute user code across all supported languages to prevent unwanted input consumption. This affects Python, Deno, Bash, PowerShell, Go, Rust, PHP, Ruby, Java, C#, Ansible, Nu, and Bun executors.

The dedicated worker handler was intentionally left unchanged as it requires stdin for inter-process communication.

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Update ee-repo ref

* Update SQLx metadata

* Fix typos

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-09-10 17:42:45 +00:00
Ruben Fiszel d07311baa3 chore(main): release 1.541.1 (#6569)
* chore(main): release 1.541.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-10 15:54:55 +00:00
hugocasa f1ca3caf1c cli and git sync (#6568) 2025-09-10 12:22:28 +00:00
Ruben Fiszel 4c41cc0dc2 fix: revoke tokens on demotions of superadmins 2025-09-10 09:20:11 +00:00
Ruben Fiszel c3eeefe1ce chore(main): release 1.541.0 (#6566)
* chore(main): release 1.541.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-09 22:51:19 +00:00
Ruben Fiszel d26a30fffd warn for variable values delays 2025-09-09 22:16:51 +00:00
Ruben Fiszel e5d3823d2b warn for variable values delays 2025-09-09 22:06:14 +00:00
Ruben Fiszel c9f0cff550 fix: variable perf and oauth refresh improvements (#6567)
* all

* improve

* improve

* improve

* improve
2025-09-09 22:03:53 +00:00
hugocasa 5f5968f2de feat: email triggers (#6548)
* feat: email triggers

* Change down migration to drop email_trigger table

* email triggers UI

* bug fix

* Apply suggestion from @ellipsis-dev[bot]

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* cli and git sync

* Revert "cli and git sync"

This reverts commit 220fd50d13.

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-09-09 21:04:23 +00:00
Ruben Fiszel 4e1b611560 chore(main): release 1.540.2 (#6562)
* chore(main): release 1.540.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-09 14:42:33 +00:00
Ruben Fiszel 1f65851c03 debug raw script endpoints 2025-09-09 14:38:47 +00:00
Ruben Fiszel 2f3cab16e2 fix scope test 2025-09-09 13:56:14 +00:00
Ruben Fiszel a68564e404 chore(main): release 1.540.1 (#6558)
* chore(main): release 1.540.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-09 15:27:11 +02:00
Ruben Fiszel acf0d958d5 fix(jwt): implement scopes run for executing apps components 2025-09-09 12:45:16 +00:00
Ruben Fiszel 9b0c1f5b14 chore(main): release 1.540.0 (#6556)
* chore(main): release 1.540.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-09 07:55:28 +00:00
hugocasa 6eecc5e85b feat(backend): prefix matching for list_paths_from_workspace_runnable + bulk delete endpoints for scripts/variables/resources (#6542)
* feat(backend): accept wildcard for list_paths_from_workspace_runnable + bulk delete endpoints for scripts/variables

* nit

* better result

* safer runnables list

* nit

* nits

* bulk delete resources as well

* batch sql queries

* nits
2025-09-08 13:41:01 +02:00
Ruben Fiszel c7d2f45a60 chore(main): release 1.539.1 (#6550)
* chore(main): release 1.539.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-06 14:20:59 +00:00
Ruben Fiszel 17d4525ac7 chore(main): release 1.539.0 (#6506)
* chore(main): release 1.539.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-05 17:11:18 +00:00
pyranota 668283a859 fix(flows): fix relative imports cache invalidation (#6519)
* fix(flows): fix relative imports cache invalidation

Signed-off-by: pyranota <pyra@duck.com>

* update flow_versions in the end

Signed-off-by: pyranota <pyra@duck.com>

* comms + fallback

Signed-off-by: pyranota <pyra@duck.com>

* add warn if fallback is used

Signed-off-by: pyranota <pyra@duck.com>

* include referenced scripts' hashes in flow_node id

Signed-off-by: pyranota <pyra@duck.com>

* improve fallback + TODOs

Signed-off-by: pyranota <pyra@duck.com>

* perf: optimize trigger

Signed-off-by: pyranota <pyra@duck.com>

* reduce TODOs

Signed-off-by: pyranota <pyra@duck.com>

* remove more TODOs

Signed-off-by: pyranota <pyra@duck.com>

* test thing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* drop old flow trigger

Signed-off-by: pyranota <pyra@duck.com>

* add comments to migration

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-09-05 16:06:51 +00:00
Ruben Fiszel c2ce888319 fix flow push schedule on flow update (#6538)
* fix flow schedule

* fix flow schedule
2025-09-05 10:09:17 +00:00
centdix ac62b4da1c feat(flows): allow all providers for ai agent steps (#6529)
* add ui for structured output

* implement backend logic for openai models

* simulate having non required props

* cleaning

* implement logic for anthropic

* cleaning

* cleaning

* cleaning

* avoid name clash

* return object

* focus new field when adding

* fix condition

* small opti

* use box raw value

* avoid unnecessary parsing + return error if parsing fails

* update ui to show all providers

* cleaning

* adapt backend

* fix

* styling fix

* fix for anthropic with openrouter

* nit styling

* avoid race condition

* fix merge

* fix

* fix import

* make one component for whole provider selection

* fix resource logic

* rename

* clean

* remove console log

* fix bad copy
2025-09-05 08:11:23 +00:00
Ruben Fiszel e3b6563014 fix: check perms more tightly for running jobs (#6532)
* all

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* all

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* masterpiece

* nits
2025-09-05 06:07:45 +00:00
hugocasa 563f033467 fix s3 proxy dev build (#6535) 2025-09-04 22:05:22 +00:00
dieriba 6320ba45c7 feat: conditional retry workflow (#6461)
* feat retry

* fix

* fix

* fix and nits

* fix query

* fix test

* nits and perf

* update sqlx

* once

* remove

* update sqlx
2025-09-03 20:24:38 +00:00
Ruben Fiszel cef6113c6e fix(app): fix apps_u routes scopes 2025-09-03 17:59:21 +00:00
Diego Imbert cd02586ba2 feat: use S3 Proxy for duckdb instead of direct connection (#6505)
* s3 proxy works with get (no auth yet)

* nit

* support s3:// syntax

* Support s3:// syntax and fix vite api proxy normalizing double slashes in URI

* s3 checks authed

* nit

* PUT works

* delete file works

* Derive the JWT signature from the backend

* Authorize s3 correctly (JWT signature is never sent in cleartext)

* convert object store error to wmill error for correct status code

* stash

* fix

* POST first request proxy works

* s3 put for duckdb

* factor out direct proxy code

* Fix Issue with backend proxy and wrong signature due to Host header mismatch

* Add _default_ syntax to solve URI normalization issues with signing

* restricted to user paths toggle

* user path restriction works !

* change restriction to allow

* fix

* factor out code

* better permissions UX in object storage settings

* Revert to restrict_to_user_paths

* check permissions in old s3 api

* DuckDB now uses S3 Proxy and no longer needs LFS query

* implement todo

* fix hardcoded w_id

* s3 proxy size limit

* s3_proxy is ee

* nit

* add Google Cloud Storage as option to secondary storage

* GCS secret in duckdb

* fix toolchain compile

* Remove user permissions for v0

* fix ci 2

* fix CI OSS

* fix missing feature flag

* fix unused warning

* integration test fails bc rustc 1.85.0

* ee ref

* fix ci ...

* update ee ref
2025-09-03 15:03:44 +00:00
Ruben Fiszel 89e8062991 test logs 2025-09-02 21:32:21 +00:00
Ruben Fiszel 88646fd1ee fix extra migration startup issue 2025-09-02 20:15:37 +00:00
Ruben Fiszel ad5fbd0a66 nit 2025-09-02 16:28:57 +00:00
Ruben Fiszel a86681edd0 add variable caching api 2025-09-02 14:34:46 +00:00
dieriba 7953d3c09f feat: refactor trigger crud (#6472)
* base

* base crate

* websocket updated

* crud refactored

* fix and nits

* fix compiler warning, nits and update repo ref

* fix oss

* fix compilation

* update ref

* fix

* update feature

* listener base

* add listener

* refactor logic done and implemented for postgres

* fix capture

* websocket

* implem for all triggers

* update sqlx,repo ref and fix

* fix oss

* unify struct fix websocket

* nits and fix oss runtime axum error

* perf cache query

* update .sqlx

* update .sqlx

* fix

* fix unused

* fix

* fix

* nits and fix http handler update endpoint

* update .sqlx

* update repo ref

* nits

* fix

* update ref

* fix

* update .sqlx
2025-09-02 11:28:06 +00:00
hugocasa de47d61d71 fix(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs (#6490)
* feat(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs

* nit

* nits

* set root_job only if additional information meaning if not equal to innermost or parent

* nit

* feat: add root job env var

* nits

* fix build

* nit

* fix build

* nit

* sqlx
2025-09-02 09:44:09 +00:00
centdix 8c6053b575 feat(aichat): allow custom system prompt for each mode (#6500)
* add ui to add custom system prompt

* implement backend changes

* add custom prompt in system prompt

* add in openapi

* add max length

* add backend validation
2025-09-02 09:27:37 +00:00
claude[bot] 02d5dab51b feat(perf): add 60-second local cache for variables and resources(#6511)
* feat: add 60-second cache for variables and resources with PostgreSQL invalidation

- Add new var_resource_cache module with 60-second TTL
- Implement PostgreSQL NOTIFY/LISTEN for immediate cache invalidation
- Cache get_variable() for non-secret, non-encrypted reads
- Cache get_resource_value() for all read operations
- Add database triggers on variable/resource table changes
- Initialize cache system in main.rs after database connection
- Add Clone derive to ListableVariable for cache compatibility

Performance benefits:
- Avoids database queries on cache hits
- Immediate invalidation ensures data consistency
- Selective caching respects security constraints

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: move cache to API layer with allow_cache query param

- Move variable/resource cache from windmill-common to windmill-api
- Add allow_cache query parameter to variables and resources endpoints
- Follow raw script cache pattern with timestamp + value structure
- Create proper database migration for notification triggers
- Include encrypted values in cache when appropriate
- Only activate caching when explicitly requested

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: move cache to API layer with allow_cache query param

Move PostgreSQL LISTEN logic to main.rs following established pattern
- Remove custom listener initialization from cache module
- Add cache invalidation channels to centralized notification handler
- Simplified cache module to only handle cache operations
- Follow raw script cache pattern for notification handling

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-09-02 07:48:03 +00:00
centdix ded248ff0b fix(mcp): fix path transformation (#6508)
* fix path transformation

* fix mcp url copy

* safer code
2025-09-02 06:50:14 +00:00
Ruben Fiszel 3cb9eb8f8e remove pin ambiguity on cache 2025-09-01 14:59:25 +00:00
Ruben Fiszel 57754a8919 feat: cache relative imports (#6504)
* all

* all

* update
2025-09-01 14:55:08 +00:00