Commit Graph

5272 Commits

Author SHA1 Message Date
Diego Imbert 699331d158 Merge remote-tracking branch 'origin/main' into di/new-duckdb-asset-parser 2025-12-08 14:51:46 +01:00
Ruben Fiszel 0e63dd301f fix: raw app v0.5 (#7310) 2025-12-06 16:06:30 +00:00
Diego Imbert 9bbab3321e feat: Data tables (#7226)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* go to settings button

* ee repo ref

* fix compilation

* wording nit
2025-12-05 23:08:58 +00:00
hugocasa 8897dab282 feat: configurable max ai agent iterations (#7302)
* feat: configurable max ai agent iterations

* nit

* increase limit
2025-12-05 18:31:36 +00:00
Diego Imbert 7b1a8b0dce fix: improve S3 etag caching (#7301) 2025-12-05 18:01:27 +00:00
Ruben Fiszel cb44d94bc6 improve folder error 2025-12-05 17:12:31 +00:00
Diego Imbert 60c5f40c7d stash 2025-12-05 17:07:00 +01:00
Ruben Fiszel 3ccdd51b9a chore(main): release 1.589.3 (#7305)
* chore(main): release 1.589.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-05 13:29:56 +00:00
Diego Imbert bbf1da8d26 Flate2 1.1.6 wouldn't compile (#7307) 2025-12-05 13:26:07 +00:00
Ruben Fiszel d6d093d342 chore(main): release 1.589.2 (#7295)
* chore(main): release 1.589.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-05 07:11:46 +00:00
Pyra 1dfb0f0078 refactor: move concurrency and debouncing settings to their own structs (#7286)
* move concurrency and debouncing settings to it's own structs

Signed-off-by: pyranota <pyra@duck.com>

* cleanup queue::jobs::pull

Signed-off-by: pyranota <pyra@duck.com>

* done

Signed-off-by: pyranota <pyra@duck.com>

* migrated the rest

Signed-off-by: pyranota <pyra@duck.com>

* implement todo!

* impmenet todo! v2

* more cleanup

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove unrelated code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* do not serialize rawcode

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-04 10:11:54 +00:00
claude[bot] e9f13065bf fix: show related job when deployment is in progress (#7294)
* feat: show related job when deployment is in progress

- Added job_id column to deployment_metadata table to track current deployment jobs
- Updated backend to store job_id when creating dependency jobs for scripts and flows
- Modified deployment status API endpoints to include job_id in responses
- Updated frontend to display clickable job link in "Deployment in progress" badge
- Added OpenAPI schema updates for new job_id field

Resolves #7293

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* update

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-04 09:50:07 +00:00
Ruben Fiszel fcde732485 nit warning concurrency limit for deps 2025-12-04 08:22:31 +00:00
Diego Imbert e936890a53 Merge branch 'main' into di/data-tables 2025-12-03 22:01:50 +01:00
Diego Imbert ea8d493bd4 Box pin in test_workflow_as_code to fix stack overflow 2025-12-03 21:53:31 +01:00
Diego Imbert 62c4e735ac sqlx prepare 2025-12-03 20:27:08 +01:00
Ruben Fiszel 0c74253319 chore(main): release 1.589.1 (#7287)
* chore(main): release 1.589.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-03 19:22:40 +00:00
Diego Imbert adfb7e809a fix column order set with s3 mode (#7289) 2025-12-03 19:09:18 +00:00
Ruben Fiszel 752e43e653 fix(backend): fix broken cancel selection/all 2025-12-03 18:05:31 +00:00
Ruben Fiszel d4c1be5b8b chore(main): release 1.589.0 (#7277)
* chore(main): release 1.589.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-03 17:51:21 +00:00
Diego Imbert 4f604b3fcd Revert "duckdb asset parser var_identifiers"
This reverts commit 88068b1a77.
2025-12-03 16:41:29 +01:00
Diego Imbert 88068b1a77 duckdb asset parser var_identifiers 2025-12-03 15:58:44 +01:00
Diego Imbert 8ae9455278 datatable in duckdb asset parser 2025-12-03 15:32:43 +01:00
Diego Imbert 80270b77f8 DuckDB supports datatables 2025-12-03 15:31:01 +01:00
Pyra e15af78d93 feat: agent workers min version support (#7284)
* feat: agent workers min version support

Signed-off-by: pyranota <pyra@duck.com>

* update oapi

Signed-off-by: pyranota <pyra@duck.com>

* fix what claude wrote

Signed-off-by: pyranota <pyra@duck.com>

* final cleanup

Signed-off-by: pyranota <pyra@duck.com>

* change a bit

Signed-off-by: pyranota <pyra@duck.com>

* remove unsed import

Signed-off-by: pyranota <pyra@duck.com>

* run update_min_version periodically

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-03 13:48:07 +00:00
Diego Imbert 0010df8b1a fork ducklakes and datatables 2025-12-03 13:41:51 +01:00
Diego Imbert 4b779b6b7e ducklake_user doesnt exist in unit tests 2025-12-03 11:04:51 +01:00
Diego Imbert 634bfd92e1 Merge remote-tracking branch 'origin/main' into di/data-tables 2025-12-03 10:40:42 +01:00
Diego Imbert 5fc5a95c9c ee repo ref 2025-12-03 10:40:29 +01:00
Ruben Fiszel 60d5ea8579 fix: get flow by id doesn't require path (#7278) 2025-12-02 19:45:30 +00:00
wendrul 2a841bb0e3 fix: Prevent running git sync on promotion mode repos for forks (#7276) 2025-12-02 17:26:58 +00:00
Diego Imbert f4b39866ab manual fix, somehow sqlx prepare won't do it 2025-12-02 15:29:48 +01:00
Diego Imbert 8812016162 Merge remote-tracking branch 'origin/main' into di/data-tables 2025-12-02 14:37:50 +01:00
Diego Imbert 5fb3f034d2 sqlx prepare fail 2025-12-02 14:36:41 +01:00
Ruben Fiszel 82d4e5edf1 chore(main): release 1.588.0 (#7270)
* chore(main): release 1.588.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-02 13:08:57 +00:00
Devdatta Talele 764e1e15f5 fix: configurable timeout for AI requests (#6497) (#7267)
* fix: configurable timeout for AI requests

Add AI_REQUEST_TIMEOUT_SECONDS environment variable (default 3600s)
to fix timeout issues with slow AI models like self-hosted Ollama.

Previously hardcoded at 300 seconds, causing legitimate long-running
requests to fail.

Fixes #6497

* docs(ai): add critical NGINX configuration warning

Add comprehensive documentation about reverse proxy timeout requirements.
Without proper NGINX/proxy configuration, connections will still timeout
at the proxy layer regardless of backend timeout settings.

Enhanced documentation includes:
- CRITICAL warning about proxy configuration requirement
- Example NGINX configuration snippet
- Explanation of proxy vs backend timeout interaction

This addresses the root cause in issue #6497 where logs showed
"upstream prematurely closed connection" indicating proxy-level timeout.

Part of #6497
2025-12-02 13:01:50 +00:00
Pyra 69c550bca6 fix(workspace-dependencies): implement better caching (#7273)
* fix(workspace-dependencies): implement better caching

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* nits

Signed-off-by: pyranota <pyra@duck.com>

* spawn non blocking

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix issue

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-02 12:48:14 +00:00
Diego Imbert 01bd5ef620 still register ducklake/datatable if not used as unknown R/W 2025-12-02 13:30:53 +01:00
Diego Imbert fe8dbc2fed Python parser successfully infers R/W/RW from ducklake / datatable 2025-12-02 13:03:09 +01:00
Diego Imbert da0302ada8 detect data table assigns in var_identifiers 2025-12-02 12:36:25 +01:00
Diego Imbert 53feb9239e remove ducklake/datatable and default 2025-12-02 12:28:09 +01:00
Diego Imbert 7e98d4f08f Revert "claude step 1: detect var declarations"
This reverts commit f866f4819d.
2025-12-02 12:23:45 +01:00
Diego Imbert cbd634bebc Revert "claude step 2: detect when saved vars are queried"
This reverts commit 1e1f930568.
2025-12-02 12:23:30 +01:00
Diego Imbert 1e1f930568 claude step 2: detect when saved vars are queried 2025-12-02 12:01:51 +01:00
Diego Imbert 20ffb3a2c9 move detect_sql_access_type to common mod 2025-12-02 12:01:21 +01:00
Diego Imbert f866f4819d claude step 1: detect var declarations 2025-12-02 11:54:27 +01:00
Diego Imbert 8349aaf3c9 ee repo ref 2025-12-01 17:10:39 +01:00
Diego Imbert f18085bc9b clearer comment 2025-12-01 16:39:20 +01:00
Diego Imbert 10f7b0095f unnecessary security (admin is required) 2025-12-01 16:34:20 +01:00
Pyra 16ef690cac nit: do not use wildcard for clone_script (#7269)
* nit: do not use wildcard for clone_script

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* reorder fields in query

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* test: make is_template optional

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* gave on using macro

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* remove dbg!

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-01 15:21:43 +00:00