Commit Graph
34 Commits
Author SHA1 Message Date
a689c0b0f1 feat: Batch re-run (#5553)
* runs on svelte 5

* Line component from svelte-chartjs

* Replaced all svelte-chartjs occurrences with custom wrapper

* Fix props mistake

* Fix illegal table structures

* self-closing-tags fix

* aria labels

* Fixed trivial warnings and errors

* @tanstack/svelte-table fix

* upgrade to vite 6

* svelte-kit sync before running svelte-check

* Remove on:clear which is actually on:removeAll and already handled by on:change

* fix worker tags not displaying in Autoscaling

* Try to fix svelte-kit sync not working during CI

* remove warnings

* Fix add flow page crashing

* access worldStore before assignment fix

* fix infinite recursions in App Editor

* Replaced JSON.stringify with proper deepEqual

* component mount api changed (no longer classes)

* fix ci errors

* Fix infinite loops in background runnable panel

* factored effect on deep equal logic in onObjChange

* fix "Add" not working in AgGrid Table

* Replaced legacy component.$set api

* Fix multiselect infinite value reaction

* Fix flow input fields resetting when opening their edit tab

* fix date input resetting when typing year

* Remove !p-0 affecting subgrid dotted borders

* fix missing debounceTemplate causing hundreds of updates

* Fix AgGrid action refreshes and disppearing

* resolve getItems generating random ids every rerun

* fix cannot access items before init

* fix sort lambda arguments being undefined

* Revert "Remove !p-0 affecting subgrid dotted borders"

This reverts commit c62809bb45d682a48376b071680645ed4e1c601b.

* fix input not updating in decision tree editor

* Update frontend/src/lib/components/schema/EditableSchemaWrapper.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Re-added padding affecting subgrid dotted borders (#5479)

* remove !p-0 in preset components

* removed extra padding on accordion tabs subgrid

* Fix non-reactive SchemaForm

* dirty fix for the oneOf bug

* feat: add nu-lang support (#5217)

* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* merge

* update wasm

* use MiniPulledJob

* update cli

* change cli wasm schema

* change cli

* update deno.json

* make wasm modules load lazily

* regenerate parsers

* remove leftover

* update cargo.lock

* clean up dnt.ts

* add docs to cli/test.nu

* add schema validation option

* add Nu to try_validate_schema

* reference frontend to new parser version

* feat: unsafe parameters for sql queries (table names, column names) (#5488)

* Make schema validation struct

Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).

* Make other validator builder

* Fail dependency job like with lockfile failing for schema validator

* Add last types + tests

* Remove unused dependency

* fix typos

* Migration ID was colliding with another, changed it manually

* Add Oneof + other fixes

* fix: cache for querying scripts correclty handles ScriptMetadata

* Add cache for schema validation from main arg sig

* Prepare sqlx

* Remove default features

* Feature flags

* WIP: unsafe sql params for sql langauges

* Fix down migration table name

* cleanup: put validation logic inside a function

* Refactor to cache the should_validate boolean

Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.

Also pass the schema when avaialble to construct the schema validator

* Add other job kinds to u8 cache key just in case

* Change sql languages to all get arguments as Values instead of RawValue

* Only cache if not preview

* Add last sql languages and some CI fixes

* Rename after typo on `sanitized`

* Finish rename

* Remove unused import

* Fix wrong test

* Add newly published regex parser version

* Remove default features from cargo.toml

* Change to a cleaner syntax for the interpolated args

* Update republished parser

* fix win build (#5494)

* add sysinfoapi feature flag for winapi dependency

* add ff

* add ff at the right place

* fix(frontend): use stable path for capture tables + nits (#5495)

* add missing capture move on first time deploy (#5496)

* avoid regen client as build step

* perf: cache workspace env variables to avoid one query (#5499)

* perf: optimize number of queries needed for job run (#5504)

* optPerf

* update sqlx

* update sqlx

* fix: improve cancel for flows with many substeps

* feat: list references upon renaming a script or a flow (#5487)

* Refactored flow_workspace_runnables to more generic workspace_runnable_dependencies

* list flows referencing an item upon renaming it

* Refactor with two exclusive columns to avoid breaking FK constraints

* Show apps depending on item upon renaming

* sqlx prepare

* list-disc instead of •

* on delete and on update cascade

* displayPathChangedWarning oneOf check instead noneOf

* combine migrations + add "on update cascade" to flow fk

* unique index on app dependencies to avoid duplicates

* create new workspace_runnable_dependencies instead of renaming old table

* Add "looking for references" loading msg

* Revert "create new workspace_runnable_dependencies instead of renaming old table"

This reverts commit 015c38ca8f.

* flow_workspace_runnables view for backwards compatibility

* Add warning for script imports on rename

* support import dependency tracking in deno

* number of using scripts / flows / apps tooltip

* forgot sqlx prepare

* delete app-related rows in down migration

* Made selection more generic

* RunsBatchActionsDropdown refactor

* started BatchReRunOptionsPane

* fix overflow quirk

* fetch schema

* refactor to group jobs by (kind, path)

* auto select

* computePropertyMap

* InputTransformForm works

* Pickable properties

* remove PropPickerWrapper and make it optional in InputTransformForm

* hide help btn

* available expressions info alert

* extraLib for editor linting

* fix selected group not updating

* nit

* Refactor async logic in script tag

* persist changes in state

* correct typing

* count for each (path, kind) group

* support flows

* use dot operator when possible

* count jobs and fix wrong number

* fix selectedJobs recomputing periodically

* (v0) individual api requests to re-run jobs

* move batchReRunChangedArgs state upwards

* Support static arg

* mistake

* Single confirmation modal + removed unnecessary state

* change confirmation modal color

* use runes in confirmation modal

* listSelectedJobsSchema API endpoint

* refactored batch rerun pane for listSelectedJobsSchema

* eliminated selectedJobs

* batch rerun works backend (v0 same args)

* Static input transforms

* simpler list_selected_jobs_schemas sql query with coalesce

* use latest schema UI + refactor

* run latest version in backend

* add deno_core dependency to windmill-api

* stream jobs from db

* basic js evaluation

* sqlx prepare

* add id path and hash in editor lint

* js works with job object!

* moved deno_core logic to separate function

* openapi yaml mistake

* unnecessary bind

* fix date as string

* Stream re-ran uuids

* handle SSE multiple values at once

* don't select all by default on batch action

* nit ui

* check that schema has property backend

* Better JobGroup query + cache

* handle multi type properties

* Notify user on error

* stupid mistake

* Fix warnings and update svelte-exmarkdown for svelte 5

* regen package-lock to fix crash on vite preview

* batch re-run all filtered jobs

* merge schemas to common type

* more explicit tooltips

* changed sse counter ui

* typos

* fix tutorial first part

* nit mistake

* package lock + elipsis nit

* fix: latest_schema option still checked on the job original schema

* always gotta forget sqlx prepare

* fix flashing loading screen

* fix batch re-run select all filtered

* better tooltip

* fix batch actions btn growing on wide screen

* revert disableBatchActions

* fix selectable step jobs

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-04-14 16:25:41 +02:00
HugoCasa c4d8eeec07 fix: improve app image picker UX (#5589) 2025-04-08 22:41:49 +00:00
HugoCasa 198f813cf5 Update S3FilePicker.svelte (#5459) 2025-03-07 19:49:26 +01:00
HugoCasa 28568e30c5 fix: s3 file picker delete + better s3 path handling (#5454)
* fix: s3 file picker delete + better s3 path handling

* update ee ref
2025-03-06 17:28:22 +01:00
HugoCasa 538d00ef9f fix(frontend): fix many s3 file picker bugs (#5428)
* fix(frontend): fix many s3 file picker bugs

* missing

* nit
2025-03-06 10:39:37 +01:00
HugoCasa e2186e704b feat: serve static websites (#5218)
* feat: serve static websites

* nit

* sqlx

* nit

* nits

* nits

* nits

* nits
2025-02-06 17:38:12 +01:00
HugoCasa 1453b9bfe6 feat: pdf file preview (#4753)
also fixes s3 missing content-length header
2024-11-19 19:09:20 +01:00
HugoCasa c686cc437d feat: http custom routes for static assets (#4666)
* feat: http custom routes for static assets

* update ee ref + fix build

* fix build

* nit
2024-11-07 20:16:14 +01:00
HugoCasa 4671e97038 feat: s3 image preview (#4262)
* feat: s3 image preview

* fix: use url in src directly

* update ee ref
2024-08-26 13:36:07 +02:00
Ruben Fiszel c9300bce2b fix sveltekit conf 2024-07-22 22:36:56 +02:00
Ruben Fiszel 7b1798f9ef fix: fix BASE_URL build conf 2024-07-22 21:53:08 +02:00
Ruben Fiszel b1a3244032 fix: fix BASE_URL build conf 2024-07-22 20:56:13 +02:00
Ruben Fiszel d7b5796fd8 small nit BASE_URL 2024-07-22 19:34:33 +02:00
e21a814712 fix: Allow deploying frontend in a sub directory (#3867)
* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow depolying frontend in a sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Expose base path from $lib/navigation

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow deploying frontend in sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow deploying frontend in sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow deploying frontend in sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow deploying frontend in sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* fix: Allow deploying frontend in sub directory

Signed-off-by: Marcel Klehr <mklehr@gmx.net>

* update

* update

---------

Signed-off-by: Marcel Klehr <mklehr@gmx.net>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-07-22 19:26:17 +02:00
Ruben Fiszel 1100d1d99a fix: handle csv with different separators in csv preview 2024-06-04 23:53:45 +02:00
Ruben Fiszel 0e7a25c0bd feat: support multiple object storage + parquet_csv + polars -> datafusion" (#3853)
* multiple storage

* all

* rm symlinks

* all

* all
2024-05-31 11:59:51 +02:00
Henri Courdentandellipsis-dev[bot] d4d64973d2 Links update & stroke width icons (#3833)
* Links update & stroke width icons

* Update frontend/src/lib/components/StringTypeNarrowing.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-05-28 16:05:56 +02:00
Ruben Fiszel bd6337b0ae fix: improve s3file browser tester 2024-02-24 17:25:07 +01:00
Guillaume Bouvignies bbda7cf268 feat: Support for Azure Blob storage (#3130) 2024-02-02 19:55:07 +01:00
0003673e8d feat: migrate s3 client to object_store (#3116)
* feat: migrate s3 client to object_store

* remove multipart from open API

* Remove multipart in favor of a single stream

* progress report

* add progress reader on server side

* small nit fix

* fix read chunk

* Fix TS and python SDK

* Fix download button

* Fix download button object viewer

* fix list

* Better errors

* export loadS3FileContent

* revert changes SDK

* fix browser

* small file list unavailable fix

* Old endpoints throws informative error messages

* Typescript SDK uses raw fetch

* update python SDK

* Error if uploaded file > 50Mb

* revert python SDL changes

* Update python SDK method docs

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-02-01 16:03:30 +01:00
Guillaume Bouvignies ef76a90e34 fix: Better UI for S3 download and S3 TS SDK endpoints (#3065)
* Better UI for S3 download and S3 TS SDK endpoints

* gitignore typescript client node_modules

* USe Windmill BE upload endpoint for TS SDK

* Use WM backend endpoint in Python upload SDK endpoint

* revert changes

* Add expiration for Python

* Add downaload endpoint

* Add toggle for public S3 resource

* revert changes to package.json

* Add link to doc page

* fix unauthorized bug
2024-01-25 15:34:40 +01:00
Guillaume Bouvignies 4f4b59c77a feat: Download button for s3 files (#3059)
* feat: Download button for s3 files

* Add S3 load and write endpoint to Python and TS SDK

* fix
2024-01-22 21:40:30 +01:00
Guillaume Bouvignies 326eac46fc feat: File path is option when uploading a file to S3 (#3029)
* feat: File path is option when uploading a file to S3

* Add frontend

* Fix lock

* Drag and drop file upload UI
2024-01-19 14:17:42 +01:00
Guillaume Bouvignies daaeb83002 feat: Browse s3 bucket content from workspace settings page (#3013) 2024-01-16 18:28:46 +01:00
Guillaume Bouvignies cd48fdce2e feat: move S3 file within bucket (#2913) 2023-12-22 15:02:16 +01:00
Guillaume Bouvignies d238fc366a feat: S3 delete file and upload new (#2909) 2023-12-22 08:38:09 +01:00
Guillaume Bouvignies b484a8dc27 feat: Add download button to S3 file picker (#2857) 2023-12-14 15:45:37 +01:00
Guillaume Bouvignies e6e8b95042 fix: Array of S3 files now shows links to the S3 explorer (#2783)
* fix: array of s3 object result

* Add more text file extensions

* Fix bug when adding a s3object param
2023-12-05 15:44:10 +01:00
Guillaume Bouvignies ba5af7d069 feat: Various minor improvements for S3 DX (#2760)
* feat: Various minor improvements for S3 DX

* Fix openAPI yaml

* Remove unused imports
2023-12-04 13:20:04 +01:00
Faton Ramadani 16f5f34127 fix(frontend): Improve error message + fix overflow when file name is too long (#2691) 2023-11-28 11:31:33 +01:00
Guillaume Bouvignies 9c2125bb89 fix: S3 bucket browser small improvements and fixes (#2700)
* fix: S3 bucket browser small improvements and fixes

* Bump file limits to 20 in CE and add a message in the FE
2023-11-24 20:18:02 +01:00
Guillaume Bouvignies 0882279055 fix: Various fixes and improvements for Windmill S3 capabilities (#2674) 2023-11-22 12:29:50 +01:00
Guillaume Bouvignies 90618274bb feat: Expanding an s3object result now opens the S3 file browser (#2656)
* feat: Expanding an s3object result now opens the S3 file browser

* Add s3 explorer link

* Revert changes in ArgInfo
2023-11-20 17:07:52 +01:00
Guillaume Bouvignies 9aea209226 feat: S3 file picker as a drawer (#2640)
* feat: Pick a S3 file drawer

* Add endpoint to generate simple file preview

* FE prototype for file preview

* More info on the preview

* Small fix

* Fix BE compile

* fix BE compile

* remove weird vite.config.js file

* Fix fronte NodeJS.Timer -> NodeJS.Timeout and rename dataset to file for S3

* Add EE restrictions and polish FE

* Fix merge conflicst

* replace dataset with s3 object

* Update wording

* BE throws an error when bucket too large in CE

* regenerate Cargo.lock

* Fix typo in error message
2023-11-20 10:26:36 +01:00