Commit Graph
16 Commits
Author SHA1 Message Date
e6f1211d31 feat: Ducklake native support (#6268)
* upgrade duckdb

* basic ducklake works

* ducklake works with custom db catalogs

* fix: pwsh skip already installed modules outside of cache (#6037)

* improve query performance of user stats

* separate ducklake_catalog db

* ducklake settings

* DucklakeSettings frontend

* Ducklake ws settings saved in backend

* fetch ducklake catalog resource

* Ducklake works with configured s3 storage

* Ducklake as asset

* ducklake asset icon

* Fix duckdb array and object args not working properly (#6254)

* Fix bug with comments in duckdb

* Avoid multiple queries when doing ATTACH ducklake

* trunc sig no longer needed now that comments are trimmed

* cache DuckdbConnectionSettingsResponse

* duplicated code

* transform_attach_ducklake contributes to duckdb_connection_settings_cache

* eliminate the need for used_storages

* nit

* cleaner management of the bigquery credentials file

* DBManagerDrawer refactor to prepare for Ducklake

* get ducklake schema

* implement delete for ducklake

* load column metadata for ducklake

* Select query works for ducklake, basic db explorer works !

* duckdb count query

* Support all db ops for ducklake

* clean migrations

* SQL repl for Ducklake

* fix broken database studio

* nit

* assert function

* Ducklake in Editor Bar

* default ducklake syntax + allow extra args

* DucklakeCatalogWizard UI

* nit + remove extra $

* modal when databases do not exist

* cannot be windmill

* Ducklake works safely with instance database

* Avoid sending instance db credentials on network

* resource leak security

* remove fetch_attach_db_conn_str

* prevent instance pg password leak

* hide asset usage count when not available

* case unsensitivity duckdb

* warnings

* disable instance catalog

* use shorthand syntax when inserting with EditorBar

* Instance ducklake catalog is now safe to use

* use safer argon2 pwd

* update package json parsers

* update package json

* better msgs

* tooltips

* disable explore button until saved

* nit

* fix warnings

* better ducklake_user password management

* nit

* Sanitize passwords from errors in ducklake

* DisplayResult broken in job result

* remove superadmin requirement to check databases_exist

* duckdb_connection_settings_v2_inner

* Ducklake works on agent worker (finally)

* ci

* #[allow(dead_code)]

* fix openapi missing response

* Separate +Database button for DuckDB in EditorBar

* Fix dropdown in ducklake settings

* Attempt to fix migration race condition in CI

* update sqlx failing for some offline queries

* avoid temp password for ducklake_user

* nits

* ducklake settings nits

* update duckdb default script

* fix sql repl resetting text on refresh

* avoid pgcrypto extension

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-06 13:55:36 +00:00
Ruben Fiszel b92fcb90a1 add tagLabel in flow editor 2025-07-23 09:21:17 +00:00
Ruben Fiszel ae1793ecd1 whitelabel nits 2025-07-18 10:22:27 +00:00
Diego ImbertandRuben Fiszel 433341b295 feat: assets as a primary concept (#6125)
* assets migration

* parse assets (duckdb)

* iterate on assets

* S3 object Preview

* remove pagination

* filterText

* better occurence list

* tweak

* assets in JobPreview

* clone impl

* AssetsDetectedBadge

* improve DbManagerButton + asset dropdown button

* edit resource btn

* warning when incorrect resource

* +Resource in DuckDB

* +S3 Object editor bar

* nit fix rename

* flow asset badge

* More Generic OnChange

* Highlight assets used in modules

* Show occurence count in flow

* Better UX, avoid moving parts

* nit

* Asset nodes

* move to dedicated Asset ctx

* fix layoutNodes not handling first assetsMap

* explore asset btn in flow asset node

* correct offset

* single computeAssetNodes function

* Fix y positioning of nodes with assets

* resource editor

* write mode node (ui)

* accessType in ctx + fix insert button positioning

* right positioning when mixing read and write nodes

* right positioning when mixing R and W assets

* Better layout fix algorithm

* listAssetsByUsage and asset nodes on transitive usages

* refactor + remove linkAssets

* Refactor to allow for custom R/W modes

* AssetsDropdownButton in flow script editor

* R/W/RW selection and changes node pos in flow

* layoutNodes doesnt need recompute now

* fix wrong assumption that nodes recompute when assets change

* r/w/rw multi toggle

* MultiToggle cool animation + clearable

* rename + 1px nit

* remove mini toggle button group, use ToggleButtonGroup

* Combinator parser that detects R / W asset context

* nit fix missing flex-1

* missing order by

* better ui indication for access type

* special x offset case when only one asset node for clarity

* parse getResource in TS with swc ecma parser

* support load and write s3 detection in TS

* Python asset parser

* support wmill api calls without special $res: or s3:// syntax

* detect out of context asset uris python

* do not use access type override when not ambiguous in flow graph

* parse_assets match case in rust

* AsRef<str> refactor

* From impl

* Save flow assets

* Save script asset usages + fixes + save fallback access types

* asset sub icon

* max total asset node width to avoid overlap

* small refactor

* don't parse comments in duckdb assets

* fix assets clearing on parse error

* fix script asset save in wrong place

* load initial asset fallback access types

* support variables

* ui fixes

* Support S3Object as URI in TS client

* support new syntax in python client

* Support +S3Object in EditorBar for TS and python

* Reduce resource requests in assets page

* import windmill client when necessary

* update s3Types.d.ts

* nit fix

* Show input resources and s3 objects as assets

* improve asset icons

* DarkModeObserver refactor

* asset page tabs

* Moved resource variables and s3object pages to assets tabs

* fetch resource usages

* Get variables usages

* move assets usage dropdown to component

* Revert "move assets usage dropdown to component"

This reverts commit 622ea4ab12.

* Revert "Get variables usages"

This reverts commit b11ced4e29.

* Revert "fetch resource usages"

This reverts commit aa5187ad4b.

* Revert "Moved resource variables and s3object pages to assets tabs"

This reverts commit 4430487be4.

* Revert "asset page tabs"

This reverts commit dacc2f0da5.

* move assets usage dropdown to component

* asset icon in asset pages

* tooltip

* details

* Storage selector in S3 File Picker

* make edge less opaque

* Refactor computeAssetNodes to separate in and out nodes

* AssetsOverflowedNode

* nits

* fix assets not being parsed in flows sometimes

* show asset kind and resource_type

* ui nits

* support res:// in duckdb

* add banner for old deployments

* Fix permissionning

* fix broken disable /enable all

* assets page view permission for operators

* Disable ExploreAssetButton for operators

* asset kind as subtitle

* do not spam getResource in assets page. prob. revert fail

* update assets page on workspace change

* reload storage names on ws change

* delete assets on archive / deletion

* sqlx prepare

* missing update when updating user

* add indexes on asset

* better message

* missing loadInit: false

* dead code

* use transaction

* typo

* update package.json

* update package.json

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-11 09:56:53 +00:00
Ruben Fiszel 3fbd3ec4f9 fix: tag select in script builder top bar (#6136)
* all

* update

* react 18

* update

* update

* update
2025-07-07 10:27:09 +00:00
ded54f2e68 fix(frontend): improve flow editor settings bar UX (#6049)
* move settings and static inputs into top node

* Move test button in the top nodes

* Revert "Move test button in the top nodes"

This reverts commit 1c8648a538.

* Add error handler to top toolbar

* nit

* polishing

* add flow settings to topbar dropdown

* remove unused files

* progress

* progress

* fixes

* fix

* fix

* fix

* fix

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-06-27 15:26:03 +00:00
e118d2cd5f feat: Implement sending diff to ai (#5510)
* allow mentioning specific files in instructions

* remove not working highlight implementation

* make highlighted text work

* fix tooltip position

* clean code

* cleaning

* use lib for tooltip positioning

* fix logic

* draft for db in context

* use tools for db in context

* fixes

* cleaning and bug fixes

* fix

* cleaning

* fix when script is db type

* simplify logic

* put schema in context if already here

* fix imports

* fix tooltip position and make it scrollable

* remove console logs

* check if selected is in available

* fix tooltip list

* add back lost logic

* last fix

* fix type errors

* use loaded schema from dbSchemas

* fix typing, content and lang are always there

* remove from context if not available anymore

* add not loaded yet mention if schema not loaded

* add missing callback logic

* fix prompt

* fix usage of updateselectedContext function

* fix styling for white theme

* handle tab and arrows

* fix schemas not being refreshed on contexts

* also refresh displayMessages when dbschemas change

* fix duplicate available contexts

* fix logic for new scripts

* fix new lines inside text area

* implement sending diff in context

* add button in deploy options to ask ai about diff

* also visualize change when asking for diff

* better prompt

* add limit to diff size

* put diff mode toggle in editor bar

* add button to see history from editor

* adjustements

* put see diff button in dropdown

* fixes

* better styling

* highlight if diff mode

* format files

* change buttons based on diffmode

* remove diff after sending message

* fix type error

* smaller buttons

* draft

* use existing editor in diff editor

* fix number of db resources fetches

* fix apply and add buttons on diff mode

* cleaning

* undo ai gen button show

* better buttons

* styling asjustements + show diff in badge

* styling

* fix deployed code check

* cleaning and styling

* better quick actions

* dont send code when analyzing

* remove apply in chat if only code and no diff

* fix bad code refactor

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-04-03 12:15:22 +00:00
Ruben Fiszelandsmuun 72109b01d7 whitelabel improvements (#5405)
* hide resource

* more customizations

* disable tooltips globally & hide other elemetns

* hide variable picker in autogenerated ui too

* change to disableX and context

* all

* all

* all

* all

* all

* all

* fix

* fix

* fix

---------

Co-authored-by: smuun <silas@athenaintelligence.ai>
2025-03-03 22:35:05 +01:00
Ruben Fiszel 995b1bf319 diff 2025-01-16 17:02:10 +01:00
Ruben Fiszel 8f56456701 fix: global timeout not set correctly on load 2024-11-28 16:02:52 +01:00
Ruben Fiszel 070a32a89e improve init script assignment 2024-10-10 02:25:25 +02:00
Ruben Fiszel 7546dade58 nit customUi 2024-09-03 09:41:57 +02:00
Ruben Fiszel 95202211f5 nits 2024-09-03 00:49:44 +02:00
Ruben Fiszel 2b3e86e341 feat: add vim support for monaco/webeditor 2024-08-26 22:31:23 +02:00
Ruben Fiszel b78249e7f7 improve modularity for whitelabelling 2024-08-21 15:37:26 +02:00
Ruben Fiszel 21db2eb340 whitelabel custom settings 2024-08-02 13:47:28 +02:00