starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-10-12 08:02:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into breadcrumb-leaf-navigation-popover

Browse Source
This commit is contained in:
Guilhem Lemouel
2026-10-09 07:47:55 -07:00
parent 3c74ce0847 92af345d05
commit 266fce74d9
296 changed files with 15057 additions and 3596 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.release-please-manifest.json
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "1.827.0"
".": "1.829.0"
}
CHANGELOG.md
+53
View File
@@ -1,5 +1,58 @@
# Changelog
## [1.829.0](https://github.com/windmill-labs/windmill/compare/v1.828.0...v1.829.0) (2026-10-09)
### Features
* add an api base url instance setting for displayed urls ([#11628](https://github.com/windmill-labs/windmill/issues/11628)) ([6acf185](https://github.com/windmill-labs/windmill/commit/6acf1858540e83b65b5218c50c98af692092d22e))
* add Pi template to the AI sandbox picker ([#11619](https://github.com/windmill-labs/windmill/issues/11619)) ([a85d0b7](https://github.com/windmill-labs/windmill/commit/a85d0b76cc494aaa4a484a273ec534bdc2958102))
* agent nits: multi-select, memory-off notice, agent step label ([#11622](https://github.com/windmill-labs/windmill/issues/11622)) ([c52ba2f](https://github.com/windmill-labs/windmill/commit/c52ba2fd30e19986522420ff02fae224a075b4f6))
* export worker cpu, memory and occupancy metrics ([#11629](https://github.com/windmill-labs/windmill/issues/11629)) ([5d90f96](https://github.com/windmill-labs/windmill/commit/5d90f96863b64e9bc88ed35429f8068a6a788d1e))
* let a job see that its worker is draining ([#11612](https://github.com/windmill-labs/windmill/issues/11612)) ([588d846](https://github.com/windmill-labs/windmill/commit/588d8463dfa1c8936b8753b5f0fd794ecb115203))
* let workspace admins delete other users' drafts ([#11644](https://github.com/windmill-labs/windmill/issues/11644)) ([68376fc](https://github.com/windmill-labs/windmill/commit/68376fc011ef6406a820c69217058316514d3f8d))
* per-script resource rollup and heaviest scripts table ([#11631](https://github.com/windmill-labs/windmill/issues/11631)) ([442357d](https://github.com/windmill-labs/windmill/commit/442357dbe8db0c52995572acd7385dedf4efdc42))
* pipeline editor: assets-only view, Fix actions and draft triggers ([#11440](https://github.com/windmill-labs/windmill/issues/11440)) ([3dae553](https://github.com/windmill-labs/windmill/commit/3dae553576ba23b2c1f56e4deac01305c4645c01))
### Bug Fixes
* advertise the actual signing algorithm in OIDC discovery ([#11637](https://github.com/windmill-labs/windmill/issues/11637)) ([98fc65a](https://github.com/windmill-labs/windmill/commit/98fc65a16bbe2bc9515cdd5443304053c4322274))
* ask a workspace for its session backups before pushing to it ([#11641](https://github.com/windmill-labs/windmill/issues/11641)) ([2ea846f](https://github.com/windmill-labs/windmill/commit/2ea846f563f5a77cbd3eae2eddee9e38a3700d00))
* keep instance settings users sub-tabs from being squashed ([#11633](https://github.com/windmill-labs/windmill/issues/11633)) ([fb22e5c](https://github.com/windmill-labs/windmill/commit/fb22e5ce368eba9146e0d51b8fdabb52ca9f68f0))
* keep raw app deploy panel settings shared with the editor ([#11616](https://github.com/windmill-labs/windmill/issues/11616)) ([0a95ea4](https://github.com/windmill-labs/windmill/commit/0a95ea47facaede2545cc4741d4a70c1c847e11a))
* load schedules once per page load, poll critical alerts every 30s ([#11639](https://github.com/windmill-labs/windmill/issues/11639)) ([1b4e686](https://github.com/windmill-labs/windmill/commit/1b4e686b2f5de3138df0c3b6ed627ea0fe0a2f7d))
* name the open workspace settings tab in the page header breadcrumb ([#11617](https://github.com/windmill-labs/windmill/issues/11617)) ([e233908](https://github.com/windmill-labs/windmill/commit/e233908c1e99a246dc7c94fa86ea82c5cbe9d6b3))
* **python:** add missing import-to-package mappings for common modules ([#11638](https://github.com/windmill-labs/windmill/issues/11638)) ([ab8606f](https://github.com/windmill-labs/windmill/commit/ab8606f37921913ef1410da735abb5742c025225))
* send hub script completions to the workspace handlers ([#11613](https://github.com/windmill-labs/windmill/issues/11613)) ([d685332](https://github.com/windmill-labs/windmill/commit/d685332771dcdfc718d7a2077fe1cdb3b323909b))
* set WM_END_USER_EMAIL on direct script and flow runs ([#11636](https://github.com/windmill-labs/windmill/issues/11636)) ([2bdeceb](https://github.com/windmill-labs/windmill/commit/2bdeceba7fb6815cf3150ecb856afb8eb3872e44))
* store a WAC parent's first start and wall duration ([#11625](https://github.com/windmill-labs/windmill/issues/11625)) ([8f28a58](https://github.com/windmill-labs/windmill/commit/8f28a5860595dbe558e6777a06bb97b6ec6a3e67))
* sync back volume files rewritten to the same size ([#11620](https://github.com/windmill-labs/windmill/issues/11620)) ([2758920](https://github.com/windmill-labs/windmill/commit/2758920f1182ddd12e6473a644a4a966a435522e))
* uniform decrypt error and clear stored credentials on logout ([#11632](https://github.com/windmill-labs/windmill/issues/11632)) ([987c321](https://github.com/windmill-labs/windmill/commit/987c321ebcd305df79fa945e941abc2b2e1d3e95))
## [1.828.0](https://github.com/windmill-labs/windmill/compare/v1.827.0...v1.828.0) (2026-10-08)
### Features
* add gpt-6.1-sol to the chat, agent steps and weekly ai evals ([#11584](https://github.com/windmill-labs/windmill/issues/11584)) ([8f7e0c7](https://github.com/windmill-labs/windmill/commit/8f7e0c72848b56fa97f6f0141d930c7d15044864))
* add wmill user list and document post-suspend approvers context ([#11600](https://github.com/windmill-labs/windmill/issues/11600)) ([1c2667c](https://github.com/windmill-labs/windmill/commit/1c2667cd5530925893cf69fd622938b6ffa55d1d))
* ask before minting the frontend sdk token in the raw app editor ([#11604](https://github.com/windmill-labs/windmill/issues/11604)) ([6e948b0](https://github.com/windmill-labs/windmill/commit/6e948b07d13b85e9251c3a200bf149f167649bc4))
* improve data table roles and grants UX ([#11562](https://github.com/windmill-labs/windmill/issues/11562)) ([d7d18f0](https://github.com/windmill-labs/windmill/commit/d7d18f00af4e2df8fcb1362c9439d2250d0ce4b6))
* one page header across routes, and one sidebar for operators and developers ([#11310](https://github.com/windmill-labs/windmill/issues/11310)) ([41162b2](https://github.com/windmill-labs/windmill/commit/41162b270c70ddac6a256bde9acf31a629990e24))
### Bug Fixes
* add retry to runScriptByPathAsync and runScriptByHashAsync ([#11485](https://github.com/windmill-labs/windmill/issues/11485)) ([3f3b1c2](https://github.com/windmill-labs/windmill/commit/3f3b1c2c308ddaba377db45308711e7c660d8775))
* describe exponential retry as multiplier × seconds^attempt ([#11546](https://github.com/windmill-labs/windmill/issues/11546)) ([a2e51cd](https://github.com/windmill-labs/windmill/commit/a2e51cdbaafb7d71b8e8bbd818e40c472098f9ba))
* keep capturing job logs after a non-UTF-8 output line ([#11595](https://github.com/windmill-labs/windmill/issues/11595)) ([0e7d974](https://github.com/windmill-labs/windmill/commit/0e7d974ce9b468353ecb6618dace710704690687))
### Performance Improvements
* load public raw apps without the loading screen ([#11608](https://github.com/windmill-labs/windmill/issues/11608)) ([c09764f](https://github.com/windmill-labs/windmill/commit/c09764fa6516badddbdbb520af32e5d1541cf996))
## [1.827.0](https://github.com/windmill-labs/windmill/compare/v1.826.0...v1.827.0) (2026-10-07)
backend/.sqlx/query-1333436e3d0466eccdaf99fb70aadad56b50619036fab3f2c1bdbbf280a80217.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "WITH prev AS (SELECT started_at FROM v2_job_queue WHERE id = $1)\n UPDATE v2_job_queue q SET running = false, started_at = null,\n extras = CASE WHEN q.started_at IS NULL OR q.extras ? 'wac_first_started_at' THEN q.extras\n ELSE jsonb_set(COALESCE(q.extras, '{}'::jsonb), '{wac_first_started_at}', to_jsonb(q.started_at))\n END\n FROM prev WHERE q.id = $1\n RETURNING (extract(epoch FROM now() - prev.started_at) * 1000)::bigint",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "int8",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "1333436e3d0466eccdaf99fb70aadad56b50619036fab3f2c1bdbbf280a80217"
}
backend/.sqlx/query-1ae4aab40783b4cd588e5a4da57eabca765789cd706ad682e8094c8960bc46a8.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM runnable_job_stats s WHERE s.hour >= $1 AND NOT EXISTS (SELECT 1 FROM workspace w WHERE w.id = s.workspace_id)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": []
},
"hash": "1ae4aab40783b4cd588e5a4da57eabca765789cd706ad682e8094c8960bc46a8"
}
backend/.sqlx/query-1e3407ad2ec46bd965f2aeede0d3cd4a3f9226a5e0c6b15c2580ddd1dba6199b.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT wp.draining FROM v2_job_queue q JOIN worker_ping wp ON wp.worker = q.worker\n WHERE q.id = $1 AND q.workspace_id = $2 AND q.running AND q.started_at IS NOT NULL",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "draining",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "1e3407ad2ec46bd965f2aeede0d3cd4a3f9226a5e0c6b15c2580ddd1dba6199b"
}
backend/.sqlx/query-24cfd719a45af5beb4eb264939290c01e1b684525bae4cb329515989cd341875.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO runnable_job_stats\n (hour, workspace_id, runnable_path, worker_group, job_count, total_duration_ms,\n max_memory_peak, sum_memory_peak, memory_sample_count, total_cpu_ms)\n SELECT s.hour, s.workspace_id, s.runnable_path, s.worker_group, s.job_count,\n s.total_duration_ms, s.max_memory_peak, s.sum_memory_peak,\n s.memory_sample_count, s.total_cpu_ms\n FROM UNNEST($1::bigint[], $2::text[], $3::text[], $4::text[], $5::int[], $6::bigint[],\n $7::int[], $8::bigint[], $9::int[], $10::bigint[])\n AS s(hour, workspace_id, runnable_path, worker_group, job_count, total_duration_ms,\n max_memory_peak, sum_memory_peak, memory_sample_count, total_cpu_ms)\n JOIN workspace w ON w.id = s.workspace_id\n ORDER BY s.hour, s.workspace_id, s.runnable_path, s.worker_group\n ON CONFLICT (hour, workspace_id, runnable_path, worker_group) DO UPDATE SET\n job_count = runnable_job_stats.job_count + EXCLUDED.job_count,\n total_duration_ms = runnable_job_stats.total_duration_ms + EXCLUDED.total_duration_ms,\n max_memory_peak = GREATEST(runnable_job_stats.max_memory_peak, EXCLUDED.max_memory_peak),\n sum_memory_peak = runnable_job_stats.sum_memory_peak + EXCLUDED.sum_memory_peak,\n memory_sample_count = runnable_job_stats.memory_sample_count + EXCLUDED.memory_sample_count,\n total_cpu_ms = runnable_job_stats.total_cpu_ms + EXCLUDED.total_cpu_ms\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8Array",
"TextArray",
"TextArray",
"TextArray",
"Int4Array",
"Int8Array",
"Int4Array",
"Int8Array",
"Int4Array",
"Int8Array"
]
},
"nullable": []
},
"hash": "24cfd719a45af5beb4eb264939290c01e1b684525bae4cb329515989cd341875"
}
backend/.sqlx/query-71c9e0c0f1d9eba0f24c8e410ca2f2a64dc559e3228d3b763f68b1ad3491ae6e.json
Generated
+45
View File
@@ -0,0 +1,45 @@
{
"db_name": "PostgreSQL",
"query": "WITH deleted AS (\n DELETE FROM v2_job_queue WHERE id = $1\n RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason,\n (extras->>'wac_first_started_at')::timestamptz AS first_started_at,\n COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(started_at, now()))))*1000)::bigint AS run_ms\n ), completed AS (\n INSERT INTO v2_job_completed AS cj\n ( workspace_id\n , id\n , started_at\n , duration_ms\n , extras\n , result\n , result_columns\n , canceled_by\n , canceled_reason\n , flow_status\n , workflow_as_code_status\n , memory_peak\n , status\n , worker\n )\n SELECT d.workspace_id, d.id, COALESCE(d.first_started_at, d.started_at),\n CASE WHEN d.first_started_at IS NULL THEN d.run_ms\n ELSE EXTRACT('epoch' FROM (now() - d.first_started_at))*1000 END,\n CASE WHEN d.first_started_at IS NOT NULL\n THEN jsonb_build_object('wac_last_segment_ms', d.run_ms) END,\n $3::text::jsonb, $10,\n CASE WHEN $4::BOOL THEN $5 ELSE d.canceled_by END,\n CASE WHEN $4::BOOL THEN $6 WHEN d.canceled_by IS NOT NULL THEN d.canceled_reason END,\n s.flow_status, s.workflow_as_code_status, $8,\n CASE WHEN $4::BOOL OR d.canceled_by IS NOT NULL THEN 'canceled'::job_status\n WHEN $7::BOOL THEN 'skipped'::job_status\n WHEN $2::BOOL THEN 'success'::job_status\n ELSE 'failure'::job_status END,\n d.worker\n FROM deleted d LEFT JOIN v2_job_status s ON s.id = d.id\n ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, result = $3::text::jsonb,\n canceled_by = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_by ELSE cj.canceled_by END,\n canceled_reason = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_reason ELSE cj.canceled_reason END\n RETURNING cj.started_at, cj.duration_ms,\n COALESCE((cj.extras->>'wac_last_segment_ms')::bigint, cj.duration_ms) AS last_run_ms\n ), parent_ping AS (\n UPDATE v2_job_runtime r SET ping = now()\n FROM v2_job_queue q\n WHERE r.id = $11 AND q.id = r.id AND q.workspace_id = $12\n AND EXISTS (SELECT 1 FROM completed)\n )\n SELECT started_at AS \"started_at?\", duration_ms AS \"duration_ms!\",\n last_run_ms AS \"last_run_ms!\" FROM completed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "started_at?",
"type_info": "Timestamptz"
},
{
"ordinal": 1,
"name": "duration_ms!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "last_run_ms!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Uuid",
"Bool",
"Text",
"Bool",
"Varchar",
"Text",
"Bool",
"Int4",
"Int8",
"TextArray",
"Uuid",
"Text"
]
},
"nullable": [
true,
false,
null
]
},
"hash": "71c9e0c0f1d9eba0f24c8e410ca2f2a64dc559e3228d3b763f68b1ad3491ae6e"
}
backend/.sqlx/query-7963eff5f7109874b94c0940eaef7d4b418a9e82be56dfe7dd65d160e0dcfd13.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT wp.draining FROM v2_job j\n JOIN v2_job_queue q ON q.id = j.id\n JOIN worker_ping wp ON wp.worker = q.worker\n WHERE j.id = $1 AND j.workspace_id = $2 AND q.running AND q.started_at IS NOT NULL\n AND ($3::text[] IS NULL OR j.tag = ANY($3))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "draining",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid",
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "7963eff5f7109874b94c0940eaef7d4b418a9e82be56dfe7dd65d160e0dcfd13"
}
backend/.sqlx/query-9212902d4eb2834ab3b429968a7a85c53c49e4ef0f42c146324d5c71fbd1e21d.json
Generated
-31
View File
@@ -1,31 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH deleted AS (\n DELETE FROM v2_job_queue WHERE id = $1\n RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason\n ), completed AS (\n INSERT INTO v2_job_completed AS cj\n ( workspace_id\n , id\n , started_at\n , duration_ms\n , result\n , result_columns\n , canceled_by\n , canceled_reason\n , flow_status\n , workflow_as_code_status\n , memory_peak\n , status\n , worker\n )\n SELECT d.workspace_id, d.id, d.started_at,\n COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(d.started_at, now()))))*1000),\n $3::text::jsonb, $10,\n CASE WHEN $4::BOOL THEN $5 ELSE d.canceled_by END,\n CASE WHEN $4::BOOL THEN $6 WHEN d.canceled_by IS NOT NULL THEN d.canceled_reason END,\n s.flow_status, s.workflow_as_code_status, $8,\n CASE WHEN $4::BOOL OR d.canceled_by IS NOT NULL THEN 'canceled'::job_status\n WHEN $7::BOOL THEN 'skipped'::job_status\n WHEN $2::BOOL THEN 'success'::job_status\n ELSE 'failure'::job_status END,\n d.worker\n FROM deleted d LEFT JOIN v2_job_status s ON s.id = d.id\n ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, result = $3::text::jsonb,\n canceled_by = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_by ELSE cj.canceled_by END,\n canceled_reason = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_reason ELSE cj.canceled_reason END\n RETURNING duration_ms\n )\n SELECT duration_ms AS \"duration_ms!\" FROM completed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "duration_ms!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Uuid",
"Bool",
"Text",
"Bool",
"Varchar",
"Text",
"Bool",
"Int4",
"Int8",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "9212902d4eb2834ab3b429968a7a85c53c49e4ef0f42c146324d5c71fbd1e21d"
}
backend/.sqlx/query-975158ae72ec12f3f933fba1d9623bf1c925feebe746299cd1f0eda3f8b949d5.json
Generated
-33
View File
@@ -1,33 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH deleted AS (\n DELETE FROM v2_job_queue WHERE id = $1\n RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason\n ), completed AS (\n INSERT INTO v2_job_completed AS cj\n ( workspace_id\n , id\n , started_at\n , duration_ms\n , result\n , result_columns\n , canceled_by\n , canceled_reason\n , flow_status\n , workflow_as_code_status\n , memory_peak\n , status\n , worker\n )\n SELECT d.workspace_id, d.id, d.started_at,\n COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(d.started_at, now()))))*1000),\n $3::text::jsonb, $10,\n CASE WHEN $4::BOOL THEN $5 ELSE d.canceled_by END,\n CASE WHEN $4::BOOL THEN $6 WHEN d.canceled_by IS NOT NULL THEN d.canceled_reason END,\n s.flow_status, s.workflow_as_code_status, $8,\n CASE WHEN $4::BOOL OR d.canceled_by IS NOT NULL THEN 'canceled'::job_status\n WHEN $7::BOOL THEN 'skipped'::job_status\n WHEN $2::BOOL THEN 'success'::job_status\n ELSE 'failure'::job_status END,\n d.worker\n FROM deleted d LEFT JOIN v2_job_status s ON s.id = d.id\n ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, result = $3::text::jsonb,\n canceled_by = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_by ELSE cj.canceled_by END,\n canceled_reason = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_reason ELSE cj.canceled_reason END\n RETURNING duration_ms\n ), parent_ping AS (\n UPDATE v2_job_runtime r SET ping = now()\n FROM v2_job_queue q\n WHERE r.id = $11 AND q.id = r.id AND q.workspace_id = $12\n AND EXISTS (SELECT 1 FROM completed)\n )\n SELECT duration_ms AS \"duration_ms!\" FROM completed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "duration_ms!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Uuid",
"Bool",
"Text",
"Bool",
"Varchar",
"Text",
"Bool",
"Int4",
"Int8",
"TextArray",
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "975158ae72ec12f3f933fba1d9623bf1c925feebe746299cd1f0eda3f8b949d5"
}
backend/.sqlx/query-97d8a9835344b8c72d59afc99d76b681c5120ef29eb09b8626a3fbc43a32cc0e.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM runnable_job_stats WHERE hour < $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": []
},
"hash": "97d8a9835344b8c72d59afc99d76b681c5120ef29eb09b8626a3fbc43a32cc0e"
}
backend/.sqlx/query-9e7a5e1ebba42217092971dd340cd9182a4812b14735e502958eaafeb4c6b33d.json
Generated
+17
View File
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_queue\n SET suspend = $3, suspend_until = now() + make_interval(secs => $4), started_at = null,\n extras = CASE WHEN started_at IS NULL OR extras ? 'wac_first_started_at' THEN extras\n ELSE jsonb_set(COALESCE(extras, '{}'::jsonb), '{wac_first_started_at}', to_jsonb(started_at))\n END\n WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Text",
"Int4",
"Float8"
]
},
"nullable": []
},
"hash": "9e7a5e1ebba42217092971dd340cd9182a4812b14735e502958eaafeb4c6b33d"
}
backend/.sqlx/query-a2f6ca89faaa8d8739f67cd1aae571de793b377dd6058065f51b12c974e5b665.json
Generated
-28
View File
@@ -1,28 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT usr.email, usage.executions\n FROM usr, LATERAL (\n SELECT COALESCE(SUM(c.duration_ms + 1000)/1000 , 0)::BIGINT executions\n FROM v2_job_completed c JOIN v2_job j USING (id)\n WHERE j.workspace_id = $1\n AND j.kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow')\n AND j.permissioned_as_email = usr.email\n AND now() - '1 week'::interval < j.created_at\n ) usage\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "executions",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
null
]
},
"hash": "a2f6ca89faaa8d8739f67cd1aae571de793b377dd6058065f51b12c974e5b665"
}
backend/.sqlx/query-c5971326bdbfaa7dff74689d1365db4f26c41fa56b326dbd02e604c793a13477.json
Generated
+74
View File
@@ -0,0 +1,74 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT workspace_id, runnable_path, worker_group,\n SUM(job_count)::bigint AS \"job_count!\",\n SUM(total_duration_ms)::bigint AS \"total_duration_ms!\",\n MAX(max_memory_peak) AS \"max_memory_peak!\",\n SUM(sum_memory_peak)::bigint AS \"sum_memory_peak!\",\n SUM(memory_sample_count)::bigint AS \"memory_sample_count!\",\n SUM(total_cpu_ms)::bigint AS \"total_cpu_ms!\"\n FROM runnable_job_stats\n WHERE hour >= $1\n AND ($2::text IS NULL OR worker_group = $2)\n AND ($3::text IS NULL OR workspace_id = $3)\n GROUP BY workspace_id, runnable_path, worker_group\n ORDER BY CASE $4::text\n WHEN 'max_memory' THEN MAX(max_memory_peak)::numeric\n WHEN 'cpu' THEN SUM(total_cpu_ms)\n ELSE SUM(total_duration_ms)\n END DESC, workspace_id, runnable_path, worker_group\n LIMIT $5\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "worker_group",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "job_count!",
"type_info": "Int8"
},
{
"ordinal": 4,
"name": "total_duration_ms!",
"type_info": "Int8"
},
{
"ordinal": 5,
"name": "max_memory_peak!",
"type_info": "Int4"
},
{
"ordinal": 6,
"name": "sum_memory_peak!",
"type_info": "Int8"
},
{
"ordinal": 7,
"name": "memory_sample_count!",
"type_info": "Int8"
},
{
"ordinal": 8,
"name": "total_cpu_ms!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8",
"Text",
"Text",
"Text",
"Int8"
]
},
"nullable": [
false,
false,
false,
null,
null,
null,
null,
null,
null
]
},
"hash": "c5971326bdbfaa7dff74689d1365db4f26c41fa56b326dbd02e604c793a13477"
}
backend/.sqlx/query-c748617e060bc41b5922df4b433f20b6971b993e0671b6ccb45db5ef028550bc.json
Generated
-33
View File
@@ -1,33 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation, native_mode) VALUES ($1, $2, COALESCE($3, 'NO IP'), $4, $5, $6, $7, $8, $9, $10, $11, $12) ON CONFLICT (worker)\n DO UPDATE set ping_at = now(), worker_instance = EXCLUDED.worker_instance, ip = COALESCE($3, worker_ping.ip), custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_worker = EXCLUDED.dedicated_worker, dedicated_workers = EXCLUDED.dedicated_workers, wm_version = EXCLUDED.wm_version, vcpus = COALESCE(EXCLUDED.vcpus, worker_ping.vcpus), memory = COALESCE(EXCLUDED.memory, worker_ping.memory), job_isolation = EXCLUDED.job_isolation, native_mode = EXCLUDED.native_mode, current_job_id = NULL, current_job_workspace_id = NULL\n RETURNING jobs_executed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "jobs_executed",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text",
"TextArray",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Int8",
"Int8",
"Text",
"Bool"
]
},
"nullable": [
false
]
},
"hash": "c748617e060bc41b5922df4b433f20b6971b993e0671b6ccb45db5ef028550bc"
}
backend/.sqlx/query-ce39fd9650500b16f7247c8173a40c7e5d2d8a51aea2d36f2bc8c6f86d53a9b8.json
Generated
+43
View File
@@ -0,0 +1,43 @@
{
"db_name": "PostgreSQL",
"query": "WITH deleted AS (\n DELETE FROM v2_job_queue WHERE id = $1\n RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason,\n (extras->>'wac_first_started_at')::timestamptz AS first_started_at,\n COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(started_at, now()))))*1000)::bigint AS run_ms\n ), completed AS (\n INSERT INTO v2_job_completed AS cj\n ( workspace_id\n , id\n , started_at\n , duration_ms\n , extras\n , result\n , result_columns\n , canceled_by\n , canceled_reason\n , flow_status\n , workflow_as_code_status\n , memory_peak\n , status\n , worker\n )\n SELECT d.workspace_id, d.id, COALESCE(d.first_started_at, d.started_at),\n CASE WHEN d.first_started_at IS NULL THEN d.run_ms\n ELSE EXTRACT('epoch' FROM (now() - d.first_started_at))*1000 END,\n CASE WHEN d.first_started_at IS NOT NULL\n THEN jsonb_build_object('wac_last_segment_ms', d.run_ms) END,\n $3::text::jsonb, $10,\n CASE WHEN $4::BOOL THEN $5 ELSE d.canceled_by END,\n CASE WHEN $4::BOOL THEN $6 WHEN d.canceled_by IS NOT NULL THEN d.canceled_reason END,\n s.flow_status, s.workflow_as_code_status, $8,\n CASE WHEN $4::BOOL OR d.canceled_by IS NOT NULL THEN 'canceled'::job_status\n WHEN $7::BOOL THEN 'skipped'::job_status\n WHEN $2::BOOL THEN 'success'::job_status\n ELSE 'failure'::job_status END,\n d.worker\n FROM deleted d LEFT JOIN v2_job_status s ON s.id = d.id\n ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, result = $3::text::jsonb,\n canceled_by = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_by ELSE cj.canceled_by END,\n canceled_reason = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL\n THEN EXCLUDED.canceled_reason ELSE cj.canceled_reason END\n RETURNING cj.started_at, cj.duration_ms,\n COALESCE((cj.extras->>'wac_last_segment_ms')::bigint, cj.duration_ms) AS last_run_ms\n )\n SELECT started_at AS \"started_at?\", duration_ms AS \"duration_ms!\",\n last_run_ms AS \"last_run_ms!\" FROM completed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "started_at?",
"type_info": "Timestamptz"
},
{
"ordinal": 1,
"name": "duration_ms!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "last_run_ms!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Uuid",
"Bool",
"Text",
"Bool",
"Varchar",
"Text",
"Bool",
"Int4",
"Int8",
"TextArray"
]
},
"nullable": [
true,
false,
null
]
},
"hash": "ce39fd9650500b16f7247c8173a40c7e5d2d8a51aea2d36f2bc8c6f86d53a9b8"
}
backend/.sqlx/query-d2fbaaaea6c542342862d8b535673d19dc4ca721ab656a39330bad5ea48999d8.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM runnable_job_stats WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "d2fbaaaea6c542342862d8b535673d19dc4ca721ab656a39330bad5ea48999d8"
}
backend/.sqlx/query-e12f852fa196d16862151ab490b05e6f5c25e23b7c41ce1c0e5a83bf7b118bfd.json
Generated
-22
View File
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH prev AS (SELECT started_at FROM v2_job_queue WHERE id = $1)\n UPDATE v2_job_queue q SET running = false, started_at = null\n FROM prev WHERE q.id = $1\n RETURNING (extract(epoch FROM now() - prev.started_at) * 1000)::bigint",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "int8",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "e12f852fa196d16862151ab490b05e6f5c25e23b7c41ce1c0e5a83bf7b118bfd"
}
backend/.sqlx/query-e4b9c0e8a46dbfe29bace8d8b12123a98ec175f9d3dc72e40a1cb40e14dec919.json
Generated
+14
View File
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE worker_ping SET draining = true WHERE worker = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "e4b9c0e8a46dbfe29bace8d8b12123a98ec175f9d3dc72e40a1cb40e14dec919"
}
backend/.sqlx/query-e7a8c0ee68b8626eb45a7eacdd364360a5d5c783d6ca0a3c4123ea126ed0e60b.json
Generated
+34
View File
@@ -0,0 +1,34 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation, native_mode, draining) VALUES ($1, $2, COALESCE($3, 'NO IP'), $4, $5, $6, $7, $8, $9, $10, $11, $12, COALESCE($13, false)) ON CONFLICT (worker)\n DO UPDATE set ping_at = now(), worker_instance = EXCLUDED.worker_instance, ip = COALESCE($3, worker_ping.ip), custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_worker = EXCLUDED.dedicated_worker, dedicated_workers = EXCLUDED.dedicated_workers, wm_version = EXCLUDED.wm_version, vcpus = COALESCE(EXCLUDED.vcpus, worker_ping.vcpus), memory = COALESCE(EXCLUDED.memory, worker_ping.memory), job_isolation = EXCLUDED.job_isolation, native_mode = EXCLUDED.native_mode, current_job_id = NULL, current_job_workspace_id = NULL, draining = COALESCE($13, worker_ping.draining)\n RETURNING jobs_executed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "jobs_executed",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text",
"TextArray",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Int8",
"Int8",
"Text",
"Bool",
"Bool"
]
},
"nullable": [
false
]
},
"hash": "e7a8c0ee68b8626eb45a7eacdd364360a5d5c783d6ca0a3c4123ea126ed0e60b"
}
backend/.sqlx/query-e8779f1cddc475554dbbd2e3122db8a21a2eaacf971bf046cef1ee14cf04e323.json
Generated
+28
View File
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT usr.email, usage.executions\n FROM usr, LATERAL (\n SELECT COALESCE(SUM(COALESCE((c.extras->>'wac_last_segment_ms')::bigint, c.duration_ms) + 1000)/1000 , 0)::BIGINT executions\n FROM v2_job_completed c JOIN v2_job j USING (id)\n WHERE j.workspace_id = $1\n AND j.kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow')\n AND j.permissioned_as_email = usr.email\n AND now() - '1 week'::interval < j.created_at\n ) usage\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "executions",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
null
]
},
"hash": "e8779f1cddc475554dbbd2e3122db8a21a2eaacf971bf046cef1ee14cf04e323"
}
backend/.sqlx/query-f0498c9bddc0d4d8948167f3ed849bf8b3523d8dde1ef6b8f1950423830cc6ed.json
Generated
-17
View File
@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_queue\n SET suspend = $3, suspend_until = now() + make_interval(secs => $4), started_at = null\n WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Text",
"Int4",
"Float8"
]
},
"nullable": []
},
"hash": "f0498c9bddc0d4d8948167f3ed849bf8b3523d8dde1ef6b8f1950423830cc6ed"
}
backend/.sqlx/query-f338741f0866dc717ccfcf6b7d46b4e55ca07e472610b29fad24b64f45f98e18.json
Generated
+59
View File
@@ -0,0 +1,59 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM draft\n WHERE workspace_id = $1 AND path = $2 AND typ = $3\n AND ($4::text IS NULL OR email = $4)\n RETURNING 1 as \"one!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "one!",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "draft_kind",
"kind": {
"Enum": [
"script",
"flow",
"app",
"raw_app",
"resource",
"variable",
"trigger_schedule",
"trigger_webhook",
"trigger_default_email",
"trigger_email",
"trigger_http",
"trigger_websocket",
"trigger_postgres",
"trigger_kafka",
"trigger_nats",
"trigger_mqtt",
"trigger_sqs",
"trigger_gcp",
"trigger_azure",
"trigger_poll",
"trigger_cli",
"trigger_nextcloud",
"trigger_google",
"trigger_github",
"data_pipeline",
"trigger_amqp"
]
}
}
},
"Text"
]
},
"nullable": [
null
]
},
"hash": "f338741f0866dc717ccfcf6b7d46b4e55ca07e472610b29fad24b64f45f98e18"
}
backend/.sqlx/query-68bca8f839e47705b11d312ee874eceaa3d1d24d9053ad4aea94b9f8465585ca.json → backend/.sqlx/query-fe22c4609d8b2472c0f83fedb92eceec2b6f1ae1a32fcf375161d04419404d9c.json
Generated
+8 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT worker, worker_instance, EXTRACT(EPOCH FROM (now() - ping_at))::integer as last_ping, started_at, ip, jobs_executed,\n CASE WHEN $4 IS TRUE THEN current_job_id ELSE NULL END as last_job_id, CASE WHEN $4 IS TRUE THEN current_job_workspace_id ELSE NULL END as last_job_workspace_id,\n custom_tags, worker_group, wm_version, occupancy_rate, occupancy_rate_15s, occupancy_rate_5m, occupancy_rate_30m, memory, vcpus, memory_usage, wm_memory_usage, job_isolation, native_mode\n FROM worker_ping\n WHERE ($1::integer IS NULL AND ping_at > now() - interval '5 minute') OR (ping_at > now() - ($1 || ' seconds')::interval)\n ORDER BY ping_at desc LIMIT $2 OFFSET $3",
"query": "SELECT worker, worker_instance, EXTRACT(EPOCH FROM (now() - ping_at))::integer as last_ping, started_at, ip, jobs_executed,\n CASE WHEN $4 IS TRUE THEN current_job_id ELSE NULL END as last_job_id, CASE WHEN $4 IS TRUE THEN current_job_workspace_id ELSE NULL END as last_job_workspace_id,\n custom_tags, worker_group, wm_version, occupancy_rate, occupancy_rate_15s, occupancy_rate_5m, occupancy_rate_30m, memory, vcpus, memory_usage, wm_memory_usage, job_isolation, native_mode, draining\n FROM worker_ping\n WHERE ($1::integer IS NULL AND ping_at > now() - interval '5 minute') OR (ping_at > now() - ($1 || ' seconds')::interval)\n ORDER BY ping_at desc LIMIT $2 OFFSET $3",
"describe": {
"columns": [
{
@@ -107,6 +107,11 @@
"ordinal": 20,
"name": "native_mode",
"type_info": "Bool"
},
{
"ordinal": 21,
"name": "draining",
"type_info": "Bool"
}
],
"parameters": {
@@ -138,8 +143,9 @@
true,
true,
true,
false,
false
]
},
"hash": "68bca8f839e47705b11d312ee874eceaa3d1d24d9053ad4aea94b9f8465585ca"
"hash": "fe22c4609d8b2472c0f83fedb92eceec2b6f1ae1a32fcf375161d04419404d9c"
}
backend/Cargo.lock
Generated
+97 -97
View File
@@ -2903,9 +2903,9 @@ dependencies = [
[[package]]
name = "curl-sys"
version = "0.4.90+curl-8.21.0"
version = "0.4.91+curl-8.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97799a0d220bfb3361e0fe4936966ff8c4b24d65c3f06dfc70d7b680b44e7897"
checksum = "cd39579123e45bf762716bb893d43886ed1cabe6289455be29a4f1d4e37afb06"
dependencies = [
"cc",
"libc",
@@ -6685,9 +6685,9 @@ checksum = "8bb03732005da905c88227371639bf1ad885cc712789c011c31c5fb3ab3ccf02"
[[package]]
name = "inventory"
version = "0.3.24"
version = "0.3.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4f0c30c76f2f4ccee3fe55a2435f691ca00c0e4bd87abe4f4a851b1d4dac39b"
checksum = "6928282826c822ad91bf1c9a1cb90a30ba1c26770749929b4656cd6be829cd7c"
dependencies = [
"rustversion",
]
@@ -9617,7 +9617,7 @@ version = "3.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f"
dependencies = [
"toml_edit 0.25.15+spec-1.1.0",
"toml_edit 0.25.17+spec-1.1.0",
]
[[package]]
@@ -13622,9 +13622,9 @@ dependencies = [
[[package]]
name = "toml_datetime"
version = "1.1.1+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7"
checksum = "2b86d767906c6c42421dcba507eb9d203e779497710a47782a224bb871653053"
dependencies = [
"serde_core",
]
@@ -13644,21 +13644,21 @@ dependencies = [
[[package]]
name = "toml_edit"
version = "0.25.15+spec-1.1.0"
version = "0.25.17+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1340ea94a5856333492c9064b02c778b191dd2c853778d9609debdcdfea3a614"
checksum = "e3641d5bbb5349a79e1020a242d251efbc546ad8048d133958323ce9c40a9c9c"
dependencies = [
"indexmap 2.14.2",
"toml_datetime 1.1.1+spec-1.1.0",
"toml_datetime 1.1.2+spec-1.1.0",
"toml_parser",
"winnow 1.0.4",
]
[[package]]
name = "toml_parser"
version = "1.1.3+spec-1.1.0"
version = "1.1.5+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d38ac1cf9b95face32296c0a3ede1fdc270627c9d9c02a7274dd6d960dc4d56"
checksum = "baa693a8032d7e1cada7d0041e96126df243179ff061456783ac7f12bda4744c"
dependencies = [
"winnow 1.0.4",
]
@@ -13973,9 +13973,9 @@ dependencies = [
[[package]]
name = "tree-sitter-language"
version = "0.1.8"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca0d1bf6fdd806e43ae5198f82f527056d359def39e54e67a0f478ac09dac081"
checksum = "d0af592be68c579aa78a16846bd19422978c3c52e438523d45ff5d1bff1f9d4a"
[[package]]
name = "tree-sitter-r"
@@ -14829,7 +14829,7 @@ dependencies = [
[[package]]
name = "windmill"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-nats",
@@ -14918,7 +14918,7 @@ dependencies = [
[[package]]
name = "windmill-ai"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"async-stream",
"async-trait",
@@ -14952,7 +14952,7 @@ dependencies = [
[[package]]
name = "windmill-alerting"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -14965,7 +14965,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"argon2",
@@ -15105,7 +15105,7 @@ dependencies = [
[[package]]
name = "windmill-api-agent-workers"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15128,7 +15128,7 @@ dependencies = [
[[package]]
name = "windmill-api-assets"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15145,7 +15145,7 @@ dependencies = [
[[package]]
name = "windmill-api-auth"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15171,7 +15171,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -15181,7 +15181,7 @@ dependencies = [
[[package]]
name = "windmill-api-configs"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15198,7 +15198,7 @@ dependencies = [
[[package]]
name = "windmill-api-debug"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"base64 0.22.1",
@@ -15220,7 +15220,7 @@ dependencies = [
[[package]]
name = "windmill-api-embeddings"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15243,7 +15243,7 @@ dependencies = [
[[package]]
name = "windmill-api-flow-conversations"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15259,7 +15259,7 @@ dependencies = [
[[package]]
name = "windmill-api-flows"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15281,7 +15281,7 @@ dependencies = [
[[package]]
name = "windmill-api-groups"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15303,7 +15303,7 @@ dependencies = [
[[package]]
name = "windmill-api-inputs"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15317,7 +15317,7 @@ dependencies = [
[[package]]
name = "windmill-api-integration-tests"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-nats",
@@ -15353,7 +15353,7 @@ dependencies = [
[[package]]
name = "windmill-api-jobs"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15379,7 +15379,7 @@ dependencies = [
[[package]]
name = "windmill-api-npm-proxy"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15407,7 +15407,7 @@ dependencies = [
[[package]]
name = "windmill-api-openapi"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15429,7 +15429,7 @@ dependencies = [
[[package]]
name = "windmill-api-schedule"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15449,7 +15449,7 @@ dependencies = [
[[package]]
name = "windmill-api-scripts"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15491,7 +15491,7 @@ dependencies = [
[[package]]
name = "windmill-api-settings"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15520,7 +15520,7 @@ dependencies = [
[[package]]
name = "windmill-api-sse"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"lazy_static",
"serde",
@@ -15532,7 +15532,7 @@ dependencies = [
[[package]]
name = "windmill-api-users"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"argon2",
"axum 0.8.9",
@@ -15556,7 +15556,7 @@ dependencies = [
[[package]]
name = "windmill-api-workers"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15570,7 +15570,7 @@ dependencies = [
[[package]]
name = "windmill-api-workspaces"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"axum 0.8.9",
"chrono",
@@ -15609,7 +15609,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"chrono",
"lazy_static",
@@ -15624,7 +15624,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"axum 0.8.9",
@@ -15643,7 +15643,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"aes-gcm",
"aho-corasick",
@@ -15757,7 +15757,7 @@ dependencies = [
[[package]]
name = "windmill-dep-map"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"chrono",
"futures",
@@ -15777,7 +15777,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"regex",
"serde",
@@ -15794,7 +15794,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -15821,7 +15821,7 @@ dependencies = [
[[package]]
name = "windmill-jseval"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"futures",
@@ -15838,7 +15838,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15854,7 +15854,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -15875,7 +15875,7 @@ dependencies = [
[[package]]
name = "windmill-native-triggers"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -15906,7 +15906,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"arc-swap",
@@ -15931,7 +15931,7 @@ dependencies = [
[[package]]
name = "windmill-object-store"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-stream",
@@ -15967,7 +15967,7 @@ dependencies = [
[[package]]
name = "windmill-operator"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"futures",
@@ -15985,7 +15985,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15994,7 +15994,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16006,7 +16006,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16018,7 +16018,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"gosyn",
@@ -16030,7 +16030,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16042,7 +16042,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16054,7 +16054,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -16065,7 +16065,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16076,7 +16076,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16088,7 +16088,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-asset"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -16099,7 +16099,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16121,7 +16121,7 @@ dependencies = [
[[package]]
name = "windmill-parser-r"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16133,7 +16133,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16147,7 +16147,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16164,7 +16164,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16177,7 +16177,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql-asset"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde",
@@ -16189,7 +16189,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16207,7 +16207,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts-asset"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde-wasm-bindgen",
@@ -16223,7 +16223,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wac"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -16239,7 +16239,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16253,7 +16253,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16292,7 +16292,7 @@ dependencies = [
[[package]]
name = "windmill-runtime-nativets"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"const_format",
@@ -16332,7 +16332,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"getrandom 0.3.4",
"wasm-bindgen",
@@ -16343,7 +16343,7 @@ dependencies = [
[[package]]
name = "windmill-store"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16378,7 +16378,7 @@ dependencies = [
[[package]]
name = "windmill-test-utils"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16402,7 +16402,7 @@ dependencies = [
[[package]]
name = "windmill-trigger"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16435,7 +16435,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-amqp"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16462,7 +16462,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-azure"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16495,7 +16495,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-email"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16515,7 +16515,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-gcp"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16549,7 +16549,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-http"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16586,7 +16586,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-kafka"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16609,7 +16609,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-mqtt"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16633,7 +16633,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-nats"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-nats",
@@ -16657,7 +16657,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-postgres"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16692,7 +16692,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-sqs"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16720,7 +16720,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-websocket"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16745,7 +16745,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"bitflags 2.13.2",
@@ -16764,7 +16764,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-once-cell",
@@ -16883,7 +16883,7 @@ dependencies = [
[[package]]
name = "windmill-worker-volumes"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"bytes",
"futures",
@@ -17592,18 +17592,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.61"
version = "0.8.62"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "879fb705ce98c32e41ebdb970fbe1204f8492423b314c6ab0354c3e7b5542866"
checksum = "86502bf56ac7c77571a32e2647bb2a15894565e981fb2a48d7bde2d91c965a9d"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.61"
version = "0.8.62"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "708882a28301d604fa039cc7727607a98d04c4b86dc76ec9cc683f805d709759"
checksum = "5457206954b06561e2608c7e19cf58b1926586d999c246eebe4502f7e2039d1a"
dependencies = [
"proc-macro2",
"quote",
backend/Cargo.toml
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.827.0"
version = "1.829.0"
authors.workspace = true
edition.workspace = true
@@ -88,7 +88,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
[workspace.package]
version = "1.827.0"
version = "1.829.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
backend/ee-repo-ref.txt
+1 -1
View File
@@ -1 +1 @@
e5689bf58a125ff46aabb6b5d0f030a7e5c715d3
5f5d03a47e58d1ea1e3970ad141ec24bd6c5f068
backend/migrations/20261008070335_worker_ping_draining.down.sql
+1
View File
@@ -0,0 +1 @@
ALTER TABLE worker_ping DROP COLUMN IF EXISTS draining;
backend/migrations/20261008070335_worker_ping_draining.up.sql
+1
View File
@@ -0,0 +1 @@
ALTER TABLE worker_ping ADD COLUMN IF NOT EXISTS draining BOOLEAN NOT NULL DEFAULT FALSE;
backend/migrations/20261008165042_runnable_job_stats.down.sql
+1
View File
@@ -0,0 +1 @@
DROP TABLE IF EXISTS runnable_job_stats;
backend/migrations/20261008165042_runnable_job_stats.up.sql
+17
View File
@@ -0,0 +1,17 @@
-- Hourly resource rollup per runnable and worker group, fed by the workers.
-- Memory figures are in kB, like v2_job_completed.memory_peak.
-- No foreign key to workspace: every worker upserts here, and the key would cost each row
-- a lookup and a lock on its workspace. Workspace deletion removes the rows itself.
CREATE TABLE IF NOT EXISTS runnable_job_stats (
hour BIGINT NOT NULL,
workspace_id VARCHAR(50) NOT NULL,
runnable_path VARCHAR(255) NOT NULL,
worker_group TEXT NOT NULL,
job_count INTEGER NOT NULL DEFAULT 0,
total_duration_ms BIGINT NOT NULL DEFAULT 0,
max_memory_peak INTEGER NOT NULL DEFAULT 0,
sum_memory_peak BIGINT NOT NULL DEFAULT 0,
memory_sample_count INTEGER NOT NULL DEFAULT 0,
total_cpu_ms BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (hour, workspace_id, runnable_path, worker_group)
);
backend/parsers/windmill-parser-py-imports/src/mapping.rs
+5
View File
@@ -383,5 +383,10 @@ pub static SHORT_IMPORTS_MAP: PyMap = phf_map! {
"vt" => "vt-py",
"grpc" => "grpcio",
"serpapi" => "google-search-results",
"pptx" => "python-pptx",
"OpenSSL" => "pyOpenSSL",
"win32api" => "pywin32",
"ruamel" => "ruamel.yaml",
"speech-recognition" => "SpeechRecognition",
// Add new entry here ^
};
backend/parsers/windmill-parser-wasm/Cargo.lock
Generated
+24 -24
View File
@@ -6216,7 +6216,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill-common"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"aho-corasick",
"anyhow",
@@ -6302,7 +6302,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"proc-macro2",
"quote",
@@ -6314,7 +6314,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"convert_case",
"serde",
@@ -6323,7 +6323,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6335,7 +6335,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6347,7 +6347,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"gosyn",
@@ -6359,7 +6359,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6371,7 +6371,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6383,7 +6383,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -6394,7 +6394,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -6405,7 +6405,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -6417,7 +6417,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-asset"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -6428,7 +6428,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -6450,7 +6450,7 @@ dependencies = [
[[package]]
name = "windmill-parser-r"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6462,7 +6462,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6476,7 +6476,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"convert_case",
@@ -6493,7 +6493,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6506,7 +6506,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql-asset"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde",
@@ -6518,7 +6518,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6536,7 +6536,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts-asset"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"serde-wasm-bindgen",
@@ -6552,7 +6552,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wac"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -6568,7 +6568,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wasm"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"getrandom 0.2.17",
@@ -6600,7 +6600,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6614,7 +6614,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.827.0"
version = "1.829.0"
dependencies = [
"anyhow",
"bitflags",
backend/parsers/windmill-parser-wasm/Cargo.toml
+1 -1
View File
@@ -12,7 +12,7 @@ resolver = "2"
members = ["."]
[workspace.package]
version = "1.827.0"
version = "1.829.0"
edition = "2021"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
backend/src/monitor.rs
+14
View File
@@ -4136,6 +4136,20 @@ pub async fn monitor_db(
}
_ => {}
}
match windmill_common::runnable_job_stats::cleanup_old_runnable_job_stats(
db,
windmill_common::runnable_job_stats::RETENTION_DAYS,
)
.await
{
Ok(count) if count > 0 => {
tracing::info!("Deleted {} old runnable job stats rows", count);
}
Err(e) => {
tracing::error!("Error cleaning up runnable job stats: {:?}", e);
}
_ => {}
}
}
}
};
backend/summarized_schema.txt
+2 -1
View File
@@ -180,6 +180,7 @@ resource_type: workspace_id(char), name(char), schema(jsonb), description(text),
FK: (workspace_id) -> workspace(id)
resume_job: id(uuid), job(uuid), flow(uuid), created_at(ts), value(jsonb), approver(char), resume_id(int), approved(bool)
FK: (flow) -> v2_job_queue(id)
runnable_job_stats: hour(bigint), workspace_id(char), runnable_path(char), worker_group(text), job_count(int), total_duration_ms(bigint), max_memory_peak(int), sum_memory_peak(bigint), memory_sample_count(int), total_cpu_ms(bigint)
runnable_settings: hash(bigint), debouncing_settings(bigint), concurrency_settings(bigint)
schedule: workspace_id(char), path(char), edited_by(char), edited_at(ts), schedule(char), enabled(bool), script_path(char), args(jsonb), extra_perms(jsonb), is_flow(bool), email(char), error(text), timezone(char), on_failure(char), on_recovery(char), on_failure_times(int), on_failure_exact(bool), on_failure_extra_args(jsonb), on_recovery_times(int), on_recovery_extra_args(jsonb), ws_error_handler_muted(bool), retry(jsonb), summary(char), no_flow_overlap(bool), tag(char), paused_until(ts), on_success(char), on_success_extra_args(jsonb), cron_version(text), description(text), dynamic_skip(char), labels(text[])
FK: (workspace_id) -> workspace(id)
@@ -219,7 +220,7 @@ websocket_trigger: path(char), url(char), script_path(char), is_flow(bool), work
windmill_migrations: name(text), created_at(ts)
worker_group_job_stats: hour(bigint), worker_group(text), script_lang(char), workspace_id(char), job_count(int), total_duration_ms(bigint)
FK: (workspace_id) -> workspace(id)
worker_ping: worker(char), worker_instance(char), ping_at(ts), started_at(ts), ip(char), jobs_executed(int), custom_tags(text[]), worker_group(char), dedicated_worker(char), wm_version(char), current_job_id(uuid), current_job_workspace_id(char), vcpus(bigint), memory(bigint), occupancy_rate(float), memory_usage(bigint), wm_memory_usage(bigint), occupancy_rate_15s(float), occupancy_rate_5m(float), occupancy_rate_30m(float), job_isolation(text), dedicated_workers(text[])
worker_ping: worker(char), worker_instance(char), ping_at(ts), started_at(ts), ip(char), jobs_executed(int), custom_tags(text[]), worker_group(char), dedicated_worker(char), wm_version(char), current_job_id(uuid), current_job_workspace_id(char), vcpus(bigint), memory(bigint), occupancy_rate(float), memory_usage(bigint), wm_memory_usage(bigint), occupancy_rate_15s(float), occupancy_rate_5m(float), occupancy_rate_30m(float), job_isolation(text), dedicated_workers(text[]), draining(bool)
workspace: id(char), name(char), owner(char), deleted(bool), premium(bool), parent_workspace_id(char), is_dev_workspace(bool)
FK: (parent_workspace_id) -> workspace(id)
workspace_dependencies: id(bigint), name(char), content(text), language(script_lang), description(text), archived(bool), workspace_id(char), created_at(ts)
backend/tests/drafts_delete_for_user.rs
+85
View File
@@ -0,0 +1,85 @@
//! Only a workspace admin can delete a draft that belongs to another user.
//!
//! A discard through `drafts/update` is scoped to the caller's own row, so it
//! never reaches someone else's draft. `drafts/delete_for_user` is the admin
//! route for that, and it must stay closed to everyone else.
use serde_json::{json, Value};
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
#[sqlx::test(fixtures("base"))]
async fn test_only_admin_deletes_another_users_draft(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/drafts");
let client = reqwest::Client::new();
let path = "script/u/test-user-2/theirs";
let save = |token: &'static str| {
client
.post(format!("{base}/update/{path}"))
.header("Authorization", format!("Bearer {token}"))
.json(&json!({ "value": { "path": "u/test-user-2/theirs", "content": "" } }))
.send()
};
let resp = save("SECRET_TOKEN_2").await?;
assert_eq!(resp.status(), 200, "{}", resp.text().await?);
let delete = |token: &'static str| {
client
.post(format!(
"{base}/delete_for_user/{path}?username=test-user-2"
))
.header("Authorization", format!("Bearer {token}"))
.send()
};
let own_draft = || async {
client
.get(format!("{base}/get_own/{path}"))
.header("Authorization", "Bearer SECRET_TOKEN_2")
.send()
.await?
.json::<Value>()
.await
};
let resp = delete("SECRET_TOKEN_3").await?;
assert_eq!(resp.status(), 403, "{}", resp.text().await?);
assert!(
!own_draft().await?.is_null(),
"a non-admin deleted the draft"
);
let resp = delete("SECRET_TOKEN").await?;
assert_eq!(resp.status(), 200, "{}", resp.text().await?);
assert!(own_draft().await?.is_null(), "the draft is still there");
let resp = delete("SECRET_TOKEN").await?;
assert_eq!(resp.status(), 404, "{}", resp.text().await?);
// Without a username the delete takes every draft at the path.
for token in ["SECRET_TOKEN_2", "SECRET_TOKEN"] {
let resp = save(token).await?;
assert_eq!(resp.status(), 200, "{}", resp.text().await?);
}
let delete_all = |token: &'static str| {
client
.post(format!("{base}/delete_for_user/{path}"))
.header("Authorization", format!("Bearer {token}"))
.send()
};
let resp = delete_all("SECRET_TOKEN_3").await?;
assert_eq!(resp.status(), 403, "{}", resp.text().await?);
let resp = delete_all("SECRET_TOKEN").await?;
assert_eq!(resp.status(), 200, "{}", resp.text().await?);
let left: i64 = sqlx::query_scalar(
"SELECT count(*) FROM draft WHERE workspace_id = 'test-workspace' AND path = 'u/test-user-2/theirs'",
)
.fetch_one(&db)
.await?;
assert_eq!(left, 0, "drafts survived a path-wide delete");
Ok(())
}
backend/tests/end_user_email.rs
+106 -72
View File
@@ -1,10 +1,8 @@
//! Tests for WM_END_USER_EMAIL environment variable.
//!
//! These tests verify that WM_END_USER_EMAIL is populated with the authenticated
//! user's email when executing app components.
//!
//! TODO: Add tests for scripts and flows once public execution endpoints are identified.
//! Currently only apps support non-workspace-member execution via OptAuthed + token lookup.
//! user's email when executing app components and when running scripts and flows directly.
//! Only apps support non-workspace-member execution via OptAuthed + token lookup.
use serde_json::json;
use sqlx::{Pool, Postgres};
@@ -27,40 +25,6 @@ fn authed(builder: reqwest::RequestBuilder, token: &str) -> reqwest::RequestBuil
builder.header("Authorization", format!("Bearer {}", token))
}
// TODO: Script tests - need to identify public execution endpoints for non-workspace-members
// async fn run_script(port: u16, token: &str) -> anyhow::Result<String> {
// let url = format!(
// "http://localhost:{}/api/w/test-workspace/jobs/run_wait_result/p/f/test/get_end_user_email",
// port
// );
// let resp = authed(client().post(&url), token)
// .json(&json!({}))
// .send()
// .await?;
// if !resp.status().is_success() {
// anyhow::bail!("script run failed: {} - {}", resp.status(), resp.text().await?);
// }
// Ok(resp.json::<serde_json::Value>().await?
// .as_str().unwrap_or("").to_string())
// }
// TODO: Flow tests - need to identify public execution endpoints for non-workspace-members
// async fn run_flow(port: u16, token: &str) -> anyhow::Result<String> {
// let url = format!(
// "http://localhost:{}/api/w/test-workspace/jobs/run_wait_result/f/f/test/get_end_user_email_flow",
// port
// );
// let resp = authed(client().post(&url), token)
// .json(&json!({}))
// .send()
// .await?;
// if !resp.status().is_success() {
// anyhow::bail!("flow run failed: {} - {}", resp.status(), resp.text().await?);
// }
// Ok(resp.json::<serde_json::Value>().await?
// .as_str().unwrap_or("").to_string())
// }
/// Create an app with inline script via API
async fn create_app_with_inline_script(port: u16, path: &str) -> anyhow::Result<()> {
let url = format!("http://localhost:{}/api/w/test-workspace/apps/create", port);
@@ -312,41 +276,111 @@ async fn wait_for_job_result(port: u16, token: &str, job_id: &str) -> anyhow::Re
anyhow::bail!("timeout waiting for job result")
}
// TODO: Script tests - need to identify public execution endpoints for non-workspace-members
// #[cfg(feature = "deno_core")]
// #[sqlx::test(fixtures("base", "end_user_email"))]
// async fn test_script_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
// initialize_tracing().await;
// set_jwt_secret().await;
// let server = ApiServer::start(db.clone()).await?;
// let port = server.addr.port();
//
// in_test_worker(Connection::Sql(db.clone()), async move {
// let result = run_script(port, SAME_WS_TOKEN).await?;
// assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Ok::<(), anyhow::Error>(())
// }, port).await?;
//
// Ok(())
// }
async fn run_direct(port: u16, token: &str, route: &str) -> anyhow::Result<String> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/jobs/run/{}",
port, route
);
let resp = authed(client().post(&url), token)
.json(&json!({}))
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!(
"run {route} failed: {} - {}",
resp.status(),
resp.text().await?
);
}
let job_id = resp.text().await?;
wait_for_job_result(port, token, &job_id).await
}
// TODO: Flow tests - need to identify public execution endpoints for non-workspace-members
// #[cfg(feature = "deno_core")]
// #[sqlx::test(fixtures("base", "end_user_email"))]
// async fn test_flow_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
// initialize_tracing().await;
// set_jwt_secret().await;
// let server = ApiServer::start(db.clone()).await?;
// let port = server.addr.port();
//
// in_test_worker(Connection::Sql(db.clone()), async move {
// let result = run_flow(port, SAME_WS_TOKEN).await?;
// assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Ok::<(), anyhow::Error>(())
// }, port).await?;
//
// Ok(())
// }
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "end_user_email"))]
async fn test_direct_run_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
set_jwt_secret().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
in_test_worker(
Connection::Sql(db.clone()),
async move {
let result = run_direct(port, SAME_WS_TOKEN, "p/f/test/get_end_user_email").await?;
assert_eq!(result, SAME_WS_EMAIL, "a script run should see its caller");
let result =
run_direct(port, SAME_WS_TOKEN, "f/f/test/get_end_user_email_flow").await?;
assert_eq!(
result, SAME_WS_EMAIL,
"every flow step should see the flow's caller"
);
Ok::<(), anyhow::Error>(())
},
port,
)
.await?;
Ok(())
}
/// A job's `WM_TOKEN` names the identity the job runs as, which differs from its end user when
/// the job runs on behalf of another account. A run it starts must keep the job's end user.
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "end_user_email"))]
async fn test_job_token_run_keeps_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
set_jwt_secret().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let calling_job = uuid::Uuid::new_v4();
sqlx::query(
"INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email)
VALUES ($1, 'test2@windmill.dev', 'test-user-2', false, false, '{}', '{}', 'test-workspace', $2)",
)
.bind(calling_job)
.bind(SAME_WS_EMAIL)
.execute(&db)
.await?;
let wm_token = windmill_common::auth::create_jwt_token(
windmill_common::db::Authed {
email: "test2@windmill.dev".to_string(),
username: "test-user-2".to_string(),
is_admin: false,
is_operator: false,
groups: vec!["all".to_string()],
folders: vec![],
scopes: None,
token_prefix: None,
},
"test-workspace",
3600,
Some(calling_job),
None,
None,
None,
)
.await?;
in_test_worker(
Connection::Sql(db.clone()),
async move {
let result = run_direct(port, &wm_token, "p/f/test/get_end_user_email").await?;
assert_eq!(
result, SAME_WS_EMAIL,
"the calling job's end user, not its run-as identity"
);
Ok::<(), anyhow::Error>(())
},
port,
)
.await?;
Ok(())
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "end_user_email"))]
backend/tests/error_handler.rs
+67
View File
@@ -411,3 +411,70 @@ async fn test_error_handler_not_triggered_on_success(db: Pool<Postgres>) -> anyh
Ok(())
}
/// A hub script run at the top level (a schedule, a trigger, the API) is a `script_hub` job,
/// and its failure must reach the workspace error handler like a workspace script's.
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(fixtures("base"))]
async fn test_error_handler_triggered_on_hub_script_failure(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_common::jobs::JobPayload;
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let version = "990000002";
let hub_dir = &*windmill_common::worker::HUB_CACHE_DIR;
tokio::fs::create_dir_all(hub_dir).await?;
tokio::fs::write(
format!("{hub_dir}/{version}"),
r#"{"content":"exit 1","lockfile":null,"language":"bash","schema":{},"summary":null}"#,
)
.await?;
// The workspace handler settings are cached per workspace id across the tests of this
// binary, so this reuses the handler path every other test here configures.
sqlx::query(
"INSERT INTO script (workspace_id, hash, path, content, language, kind, created_by, schema, summary, description, lock)
VALUES ('test-workspace', 1111111112, 'f/test/error_handler', 'echo handled', 'bash', 'script', 'test-user', '{}', '', '', '')",
)
.execute(&db)
.await?;
sqlx::query(
r#"UPDATE workspace_settings SET error_handler = '{"path": "script/f/test/error_handler"}'::jsonb
WHERE workspace_id = 'test-workspace'"#,
)
.execute(&db)
.await?;
let completed_job = RunJob::from(JobPayload::ScriptHub {
path: format!("hub/{version}/test/fail"),
apply_preprocessor: false,
})
.run_until_complete(&db, false, server.addr.port())
.await;
assert!(!completed_job.success, "Hub script should have failed");
let mut handler_parent = None;
for _ in 0..50 {
handler_parent = sqlx::query_scalar::<_, Option<uuid::Uuid>>(
"SELECT parent_job FROM v2_job
WHERE workspace_id = 'test-workspace' AND permissioned_as_email = 'error_handler@windmill.dev'",
)
.fetch_optional(&db)
.await?;
if handler_parent.is_some() {
break;
}
tokio::time::sleep(tokio::time::Duration::from_millis(100)).await;
}
assert_eq!(
handler_parent,
Some(Some(completed_job.id)),
"Workspace error handler should handle the failed hub script"
);
Ok(())
}
backend/tests/otel.rs
+26 -1
View File
@@ -176,6 +176,10 @@ async fn test_all_metrics_e2e() {
otel_set_worker_uptime("w-uptime", 3600.0);
otel_set_health_status_phase("healthy");
otel_set_health_db_unresponsive(true);
otel_set_worker_memory("w-res", "gpu", Some(512), Some(128), Some(1024));
otel_add_worker_cpu("w-res", "gpu", 1.5, 40, 7, 0.35);
otel_set_worker_occupancy_rate("w-res", "gpu", "5m", 0.25);
otel_record_job_memory_peak("python3", 100 * 1024 * 1024);
// ── Histograms ──────────────────────────────────────────────────
@@ -189,7 +193,7 @@ async fn test_all_metrics_e2e() {
let metrics = flush_and_get_metrics(&state);
let names = metric_names(&metrics);
// ── Verify all 21 metric names are present ──────────────────────
// ── Verify all metric names are present ─────────────────────────
let expected = [
"windmill.queue.push_count",
@@ -213,6 +217,15 @@ async fn test_all_metrics_e2e() {
"windmill.worker.uptime",
"windmill.health.status",
"windmill.health.db_unresponsive",
"windmill.worker.memory_usage",
"windmill.worker.windmill_memory_usage",
"windmill.worker.memory_limit",
"windmill.worker.cpu_usage",
"windmill.worker.cpu_periods",
"windmill.worker.cpu_throttled_periods",
"windmill.worker.cpu_throttled_time",
"windmill.worker.occupancy_rate",
"windmill.worker.job_memory_peak",
];
for name in expected {
assert!(
@@ -290,6 +303,18 @@ async fn test_all_metrics_e2e() {
.expect("worker.busy data point with worker=worker-test-1 not found");
assert_eq!(dp.1, 1);
let m = find_metric(&metrics, "windmill.worker.memory_limit").unwrap();
let values = gauge_i64_values(m);
let dp = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "worker_group" && kv.value.as_str() == "gpu")
})
.expect("worker.memory_limit data point with worker_group=gpu not found");
assert_eq!(dp.1, 1024);
let m = find_metric(&metrics, "windmill.db.pool.active").unwrap();
assert_eq!(gauge_i64_values(m)[0].1, 5);
let m = find_metric(&metrics, "windmill.db.pool.idle").unwrap();
backend/tests/success_handler.rs
+67
View File
@@ -239,3 +239,70 @@ export async function main(path: string, email: string, job_id: string, is_flow:
Ok(())
}
/// A hub script run at the top level (a schedule, a trigger, the API) is a `script_hub` job,
/// and its success must reach the workspace success handler like a workspace script's.
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(fixtures("base"))]
async fn test_success_handler_triggered_on_hub_script_success(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
use windmill_common::jobs::JobPayload;
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let version = "990000003";
let hub_dir = &*windmill_common::worker::HUB_CACHE_DIR;
tokio::fs::create_dir_all(hub_dir).await?;
tokio::fs::write(
format!("{hub_dir}/{version}"),
r#"{"content":"echo ok","lockfile":null,"language":"bash","schema":{},"summary":null}"#,
)
.await?;
// The workspace handler settings are cached per workspace id across the tests of this
// binary, so this reuses the handler path every other test here configures.
sqlx::query(
"INSERT INTO script (workspace_id, hash, path, content, language, kind, created_by, schema, summary, description, lock)
VALUES ('test-workspace', 1234567891, 'f/test/success_handler', 'echo handled', 'bash', 'script', 'test-user', '{}', '', '', '')",
)
.execute(&db)
.await?;
sqlx::query(
r#"UPDATE workspace_settings SET success_handler = '{"path": "script/f/test/success_handler"}'::jsonb
WHERE workspace_id = 'test-workspace'"#,
)
.execute(&db)
.await?;
let completed_job = RunJob::from(JobPayload::ScriptHub {
path: format!("hub/{version}/test/ok"),
apply_preprocessor: false,
})
.run_until_complete(&db, false, server.addr.port())
.await;
assert!(completed_job.success, "Hub script should have succeeded");
let mut handler_parent = None;
for _ in 0..50 {
handler_parent = sqlx::query_scalar::<_, Option<uuid::Uuid>>(
"SELECT parent_job FROM v2_job
WHERE workspace_id = 'test-workspace' AND permissioned_as_email = 'success_handler@windmill.dev'",
)
.fetch_optional(&db)
.await?;
if handler_parent.is_some() {
break;
}
tokio::time::sleep(tokio::time::Duration::from_millis(100)).await;
}
assert_eq!(
handler_parent,
Some(Some(completed_job.id)),
"Workspace success handler should handle the successful hub script"
);
Ok(())
}
backend/tests/wac_suspend_started_at.rs
+88
View File
@@ -109,3 +109,91 @@ async fn wac_suspend_stands_down_for_a_cancel(db: Pool<Postgres>) -> anyhow::Res
Ok(())
}
/// Each park clears `started_at`, so the completion would otherwise store only the last
/// segment: a workflow that waited an hour on its tasks would read as a sub-second job that
/// started an hour late. The completed row spans the workflow from its first start, and
/// keeps the last segment aside: that is what is metered and what worker-time readers sum.
#[sqlx::test]
async fn wac_completion_spans_the_workflow_and_keeps_the_last_segment(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
let job_id = Uuid::new_v4();
sqlx::query(
"INSERT INTO v2_job (id, workspace_id, created_by, created_at, permissioned_as, \
permissioned_as_email, kind, script_lang, runnable_path, tag, visible_to_owner) \
VALUES ($1, 'test-workspace', 'test-user', now() - interval '2 hours', 'u/test-user', \
'test@windmill.dev', 'script', 'bun', 'u/test-user/wac', 'bun', true)",
)
.bind(job_id)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, running, tag, started_at) \
VALUES ($1, 'test-workspace', now(), true, 'bun', now() - interval '1 hour')",
)
.bind(job_id)
.execute(&db)
.await?;
// Two parks, each followed by the pull re-stamping `started_at`: the second park must
// not replace the first start with its own segment's.
for resumed_ago in ["10 minutes", "3 seconds"] {
let mut tx = db.begin().await?;
suspend_wac_parent(&mut tx, &job_id, "test-workspace", 1, 3600.0).await?;
tx.commit().await?;
sqlx::query(
"UPDATE v2_job_queue SET suspend = 0, started_at = now() - $2::text::interval \
WHERE id = $1",
)
.bind(job_id)
.bind(resumed_ago)
.execute(&db)
.await?;
}
let job = windmill_queue::get_mini_completed_job(&job_id, "test-workspace", &db)
.await?
.unwrap();
let result = serde_json::value::to_raw_value(&serde_json::json!("done"))?;
let (_, span) = windmill_queue::add_completed_job(
&db,
&job,
true,
false,
sqlx::types::Json(&result),
None,
0,
None,
false,
Some(3000),
false,
)
.await?;
let (started_ago_s, duration_ms, last_segment_ms): (f64, i64, Option<i64>) = sqlx::query_as(
"SELECT extract(epoch FROM now() - started_at)::float8, duration_ms, \
(extras->>'wac_last_segment_ms')::bigint \
FROM v2_job_completed WHERE id = $1",
)
.bind(job_id)
.fetch_one(&db)
.await?;
assert!(
(started_ago_s - 3600.0).abs() < 60.0,
"started_at must be the first start, got {started_ago_s}s ago"
);
assert!(
(duration_ms - 3_600_000).abs() < 60_000,
"duration_ms must be the wall time since the first start, got {duration_ms}"
);
assert_eq!(last_segment_ms, Some(3000));
assert_eq!(
span.duration_ms, duration_ms,
"a flow records the step with the span the row holds"
);
assert!(span.outlasts_run(job.started_at));
assert_eq!(span.last_run_ms, 3000, "only the last segment is left to meter");
Ok(())
}
backend/tests/worker_ping_ip.rs
+31 -1
View File
@@ -2,10 +2,19 @@ use sqlx::{Pool, Postgres};
use uuid::Uuid;
use windmill_common::{
external_ip::UNKNOWN_IP,
worker::{insert_ping_query, update_worker_ping_main_loop_query},
worker::{insert_ping_query, set_worker_draining_query, update_worker_ping_main_loop_query},
};
async fn insert_ping(db: &Pool<Postgres>, worker: &str, ip: Option<&str>) -> anyhow::Result<()> {
register(db, worker, ip, None).await
}
async fn register(
db: &Pool<Postgres>,
worker: &str,
ip: Option<&str>,
draining: Option<bool>,
) -> anyhow::Result<()> {
insert_ping_query(
"test-instance",
worker,
@@ -19,12 +28,33 @@ async fn insert_ping(db: &Pool<Postgres>, worker: &str, ip: Option<&str>) -> any
None,
None,
false,
draining,
db,
)
.await?;
Ok(())
}
/// A worker re-registers mid-life once its external IP resolves. That registration must not
/// clear a draining flag the same process already published; only a worker starting up under
/// a reclaimed name resets it.
#[sqlx::test]
async fn only_a_starting_worker_resets_draining(db: Pool<Postgres>) -> anyhow::Result<()> {
let draining = || async {
sqlx::query_scalar::<_, bool>("SELECT draining FROM worker_ping WHERE worker = 'wk-drain'")
.fetch_one(&db)
.await
};
register(&db, "wk-drain", None, Some(false)).await?;
set_worker_draining_query("wk-drain", &db).await?;
register(&db, "wk-drain", Some("1.2.3.4"), None).await?;
assert!(draining().await?);
register(&db, "wk-drain", None, Some(false)).await?;
assert!(!draining().await?);
Ok(())
}
/// The external IP resolves in the background, so the initial ping often has none yet. That must
/// not blank the address a previous process wrote to the row this one reclaims — worker names are
/// stable across restarts under EXIT_AFTER_N_JOBS.
backend/windmill-api-auth/src/scopes.rs
+1
View File
@@ -996,6 +996,7 @@ pub fn is_own_job_runtime_route(route_path: &str, http_method: &str, job_id: uui
["job_metrics", "set_progress", ..] => (http_method == "POST", 2),
["job_metrics", "get_progress", ..] => (http_method == "GET", 2),
["jobs" | "jobs_u", "resume_urls" | "wac_approval_urls", ..] => (http_method == "GET", 2),
["jobs", "worker_is_draining", ..] => (http_method == "GET", 2),
["jobs", "wac", "inline_checkpoint", ..] => (http_method == "POST", 3),
["jobs", "run", "workflow_as_code", ..] => (http_method == "POST", 3),
_ => return false,
backend/windmill-api-integration-tests/tests/jobs_authed.rs
+48
View File
@@ -702,3 +702,51 @@ async fn test_list_filtered_job_uuids_resolved_excludes_queue(
Ok(())
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_worker_is_draining_follows_the_job_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let job = Uuid::new_v4();
sqlx::query(
"INSERT INTO v2_job (id, workspace_id, created_by, permissioned_as, kind, tag, args)
VALUES ($1, 'test-workspace', 'test-user', 'u/test-user', 'script', 'deno', '{}'::jsonb)",
)
.bind(job)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, tag, running, started_at, worker)
VALUES ($1, 'test-workspace', now(), 'deno', true, now(), 'wk-drain-test')",
)
.bind(job)
.execute(&db)
.await?;
sqlx::query(
"INSERT INTO worker_ping (worker, worker_instance, ip, worker_group, wm_version)
VALUES ('wk-drain-test', 'host', 'NO IP', 'default', 'v0')",
)
.execute(&db)
.await?;
let url = format!("http://localhost:{port}/api/w/test-workspace/jobs/worker_is_draining/{job}");
let draining = || async {
let resp = authed(client().get(&url)).send().await?;
anyhow::Ok(resp.json::<bool>().await?)
};
assert!(!draining().await?);
windmill_common::worker::set_worker_draining_query("wk-drain-test", &db).await?;
assert!(draining().await?);
// A parked job keeps its last worker on the queue row without running on it.
sqlx::query("UPDATE v2_job_queue SET started_at = NULL WHERE id = $1")
.bind(job)
.execute(&db)
.await?;
assert!(!draining().await?);
Ok(())
}
backend/windmill-api-jobs/src/execution.rs
+25 -2
View File
@@ -245,6 +245,27 @@ pub async fn unclaimable_run_lineage(
Ok(referenced)
}
/// `WM_END_USER_EMAIL` of a run started through a `/jobs/run*` route. `WM_EMAIL` names the
/// runnable's `on_behalf_of` identity when it has one, so this is the only place the caller's
/// address reaches the job. A job's own `WM_TOKEN` passes on the end user of that job, and a run
/// a trigger fired has no end user.
pub async fn run_end_user_email(
db: &DB,
w_id: &str,
authed: &ApiAuthed,
trigger: Option<&TriggerMetadata>,
) -> error::Result<Option<String>> {
if trigger.is_some() {
return Ok(None);
}
match authed.job_id {
Some(job_id) => Ok(windmill_common::auth::get_job_perms(db, &job_id, w_id)
.await?
.and_then(|p| p.end_user_email)),
None => Ok(Some(authed.email.clone())),
}
}
#[cfg(feature = "enterprise")]
pub async fn check_license_key_valid() -> error::Result<()> {
use windmill_common::ee_oss::LICENSE_KEY_VALID;
@@ -1013,6 +1034,7 @@ pub async fn run_flow<'c>(
)
};
let end_user_email = run_end_user_email(db, w_id, authed, trigger.as_ref()).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(db, authed).await?;
let (uuid, mut tx) = push(
&db,
@@ -1048,7 +1070,7 @@ pub async fn run_flow<'c>(
None,
push_authed.as_ref(),
false,
None,
end_user_email,
authed.trigger_or_fallback(trigger),
run_query.suspended_mode,
scope_ceiling.as_deref(),
@@ -1244,6 +1266,7 @@ pub async fn push_script_job_by_path_into_queue<'c>(
)
};
let end_user_email = run_end_user_email(&db, &w_id, &authed, trigger.as_ref()).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(&db, &authed).await?;
let (uuid, tx) = push(
&db,
@@ -1277,7 +1300,7 @@ pub async fn push_script_job_by_path_into_queue<'c>(
},
push_authed.as_ref(),
false,
None,
end_user_email,
authed.trigger_or_fallback(trigger),
run_query.suspended_mode,
scope_ceiling.as_deref(),
backend/windmill-api-settings/src/lib.rs
+17 -17
View File
@@ -58,10 +58,10 @@ use windmill_common::{
error::{self, pg_error_message, JsonResult, Result},
get_database_url,
global_settings::{
ACCENT_COLOR_SETTING, AI_CONFIG_SETTING, APP_WORKSPACED_ROUTE_SETTING,
AUTOMATE_USERNAME_CREATION_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING, CUSTOM_TAGS_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, DISABLE_HUB_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
EXTERNAL_INSTANCE_PG_SETTING,
ACCENT_COLOR_SETTING, AI_CONFIG_SETTING, API_BASE_URL_SETTING,
APP_WORKSPACED_ROUTE_SETTING, AUTOMATE_USERNAME_CREATION_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING,
DISABLE_HUB_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS, EXTERNAL_INSTANCE_PG_SETTING,
GITHUB_APP_WEBHOOK_BASE_URL_SETTING, HTTP_ROUTE_DEFAULT_ALLOWED_ORIGINS_SETTING,
HTTP_ROUTE_WORKSPACED_ROUTE_SETTING, HUB_ACCESSIBLE_URL_SETTING, HUB_BASE_URL_SETTING,
INSTANCE_BANNER_SETTING, MAX_RETENTION_OVERRIDE_WORKSPACES,
@@ -1238,26 +1238,22 @@ async fn run_setting_pre_write_hook(
}
}
}
GITHUB_APP_WEBHOOK_BASE_URL_SETTING => {
// A bad value here yields a webhook GitHub can never deliver to, and the
GITHUB_APP_WEBHOOK_BASE_URL_SETTING | API_BASE_URL_SETTING => {
// A bad webhook base yields a webhook GitHub can never deliver to, and the
// failure only shows up much later as "falling back to polling" on a
// repository — so reject it at the boundary instead.
// repository; a bad API base puts an unreachable URL in front of every user.
// So reject both at the boundary instead.
match value {
// Clearing (delete row) is handled by the caller; allow it through.
serde_json::Value::Null => {}
serde_json::Value::String(s) if s.trim().is_empty() => {}
serde_json::Value::String(s) => {
windmill_common::global_settings::validate_webhook_base_url(s).map_err(
|e| {
error::Error::BadRequest(format!(
"{GITHUB_APP_WEBHOOK_BASE_URL_SETTING}: {e}"
))
},
)?;
windmill_common::global_settings::validate_webhook_base_url(s)
.map_err(|e| error::Error::BadRequest(format!("{key}: {e}")))?;
}
_ => {
return Err(error::Error::BadRequest(format!(
"{GITHUB_APP_WEBHOOK_BASE_URL_SETTING} must be a URL string"
"{key} must be a URL string"
)));
}
}
@@ -1394,8 +1390,12 @@ async fn set_instance_config(
// Applied and audited one setting at a time: the writes are not transactional, so a
// batch that fails midway must not leave the settings it did write unrecorded.
let instance_config::SettingsDiff { upserts, deletes, mut previous_values, unchanged_count } =
settings_diff;
let instance_config::SettingsDiff {
upserts,
deletes,
mut previous_values,
unchanged_count,
} = settings_diff;
let steps = upserts
.into_iter()
.map(|(k, v)| (k, Some(v)))
backend/windmill-api-users/src/users.rs
+1 -1
View File
@@ -490,7 +490,7 @@ async fn list_user_usage(
"
SELECT usr.email, usage.executions
FROM usr, LATERAL (
SELECT COALESCE(SUM(c.duration_ms + 1000)/1000 , 0)::BIGINT executions
SELECT COALESCE(SUM(COALESCE((c.extras->>'wac_last_segment_ms')::bigint, c.duration_ms) + 1000)/1000 , 0)::BIGINT executions
FROM v2_job_completed c JOIN v2_job j USING (id)
WHERE j.workspace_id = $1
AND j.kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow')
backend/windmill-api-workers/src/lib.rs
+78 -1
View File
@@ -41,6 +41,7 @@ pub fn global_service() -> Router {
.route("/queue_metrics", get(get_queue_metrics))
.route("/queue_metrics_series", get(get_queue_metrics_series))
.route("/queue_status", get(get_queue_status))
.route("/runnable_stats", get(get_runnable_stats))
.route("/queue_counts", get(get_queue_counts))
.route("/queue_running_counts", get(get_queue_running_counts))
.route(
@@ -86,6 +87,7 @@ struct WorkerPing {
job_isolation: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
native_mode: Option<bool>,
draining: bool,
}
// #[derive(Serialize, Deserialize)]
@@ -118,7 +120,7 @@ async fn list_worker_pings(
WorkerPing,
"SELECT worker, worker_instance, EXTRACT(EPOCH FROM (now() - ping_at))::integer as last_ping, started_at, ip, jobs_executed,
CASE WHEN $4 IS TRUE THEN current_job_id ELSE NULL END as last_job_id, CASE WHEN $4 IS TRUE THEN current_job_workspace_id ELSE NULL END as last_job_workspace_id,
custom_tags, worker_group, wm_version, occupancy_rate, occupancy_rate_15s, occupancy_rate_5m, occupancy_rate_30m, memory, vcpus, memory_usage, wm_memory_usage, job_isolation, native_mode
custom_tags, worker_group, wm_version, occupancy_rate, occupancy_rate_15s, occupancy_rate_5m, occupancy_rate_30m, memory, vcpus, memory_usage, wm_memory_usage, job_isolation, native_mode, draining
FROM worker_ping
WHERE ($1::integer IS NULL AND ping_at > now() - interval '5 minute') OR (ping_at > now() - ($1 || ' seconds')::interval)
ORDER BY ping_at desc LIMIT $2 OFFSET $3",
@@ -298,6 +300,81 @@ async fn get_queue_metrics(
Ok(Json(queue_metrics))
}
#[derive(Deserialize)]
struct RunnableStatsQuery {
window_secs: Option<i64>,
worker_group: Option<String>,
workspace: Option<String>,
order_by: Option<String>,
limit: Option<i64>,
}
#[derive(Serialize)]
struct RunnableStats {
workspace_id: String,
runnable_path: String,
worker_group: String,
job_count: i64,
total_duration_ms: i64,
max_memory_peak: i32,
sum_memory_peak: i64,
memory_sample_count: i64,
total_cpu_ms: i64,
}
async fn get_runnable_stats(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Query(query): Query<RunnableStatsQuery>,
) -> JsonResult<Vec<RunnableStats>> {
require_devops_role(&db, &authed).await?;
let window = query.window_secs.unwrap_or(24 * 3600).clamp(
3600,
windmill_common::runnable_job_stats::RETENTION_DAYS * 24 * 3600,
);
// Rows are hourly buckets keyed by their start, so the bucket the window begins in
// is included whole.
let from_hour = (chrono::Utc::now().timestamp() - window) / 3600 * 3600;
let order_by = match query.order_by.as_deref() {
Some("max_memory") => "max_memory",
Some("cpu") => "cpu",
_ => "total_duration",
};
let rows = sqlx::query_as!(
RunnableStats,
r#"
SELECT workspace_id, runnable_path, worker_group,
SUM(job_count)::bigint AS "job_count!",
SUM(total_duration_ms)::bigint AS "total_duration_ms!",
MAX(max_memory_peak) AS "max_memory_peak!",
SUM(sum_memory_peak)::bigint AS "sum_memory_peak!",
SUM(memory_sample_count)::bigint AS "memory_sample_count!",
SUM(total_cpu_ms)::bigint AS "total_cpu_ms!"
FROM runnable_job_stats
WHERE hour >= $1
AND ($2::text IS NULL OR worker_group = $2)
AND ($3::text IS NULL OR workspace_id = $3)
GROUP BY workspace_id, runnable_path, worker_group
ORDER BY CASE $4::text
WHEN 'max_memory' THEN MAX(max_memory_peak)::numeric
WHEN 'cpu' THEN SUM(total_cpu_ms)
ELSE SUM(total_duration_ms)
END DESC, workspace_id, runnable_path, worker_group
LIMIT $5
"#,
from_hour,
query.worker_group,
query.workspace,
order_by,
query.limit.unwrap_or(100).clamp(1, 1000),
)
.fetch_all(&db)
.await?;
Ok(Json(rows))
}
#[derive(Deserialize)]
struct QueueMetricsSeriesQuery {
window_secs: Option<i64>,
backend/windmill-api-workspaces/src/workspaces_extra.rs
+7
View File
@@ -1299,6 +1299,13 @@ pub(crate) async fn delete_workspace(
.execute(&mut *tx)
.await?;
sqlx::query!(
"DELETE FROM runnable_job_stats WHERE workspace_id = $1",
&w_id
)
.execute(&mut *tx)
.await?;
// If this workspace is itself a dev workspace, deleting it dissolves the pairing, so also drop
// the parent prod's reserved dev_workspace_lock (mirrors detach_dev_workspace) — otherwise prod
// stays locked against direct deploy/forking with no dev workspace left to make changes in.
backend/windmill-api/openapi.yaml
+146 -3
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.827.0
version: 1.829.0
title: Windmill API
contact:
@@ -1919,7 +1919,7 @@ paths:
/settings/instance_ui:
get:
summary: get the instance-wide announcement banner and accent color
summary: get the instance-wide announcement banner, accent color and API base url
description: |
Readable by any authenticated user; every page load reads it. Each value
is the raw stored setting, or null when unset.
@@ -1928,7 +1928,7 @@ paths:
- setting
responses:
"200":
description: the instance_banner and accent_color settings
description: the instance_banner, accent_color and api_base_url settings
content:
application/json:
schema:
@@ -1936,6 +1936,7 @@ paths:
properties:
instance_banner: {}
accent_color: {}
api_base_url: {}
/settings_u/ruff_config:
get:
@@ -10924,6 +10925,37 @@ paths:
description: "`saved` only, upsert or delete: where the write landed. Differs from the URL path when the item had moved away from it; the editor follows it there. Absent when a delete found nothing to remove and the caller cannot read the path it moved to."
required: [status, current_timestamp]
/w/{workspace}/drafts/delete_for_user/{kind}/{path}:
post:
summary: delete another user's draft, or every draft at a path (admin only)
description: Delete the draft a given user holds at a path, or, without `username`, every draft at that path (the legacy ownerless one and the caller's own included). Workspace admins / superadmins only, and only for script, flow, app and raw_app drafts. Unlike a discard of one's own draft through `drafts/update`, this returns 404 when nothing was deleted.
operationId: deleteDraftForUser
tags:
- draft
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: kind
in: path
required: true
schema:
$ref: "#/components/schemas/UserDraftItemKind"
- $ref: "#/components/parameters/ScriptPath"
- name: username
in: query
required: false
description: Workspace username of the draft owner. Omit to delete every user's draft at the path.
schema:
type: string
responses:
"200":
description: draft(s) deleted
content:
text/plain:
schema:
type: string
"404":
description: no matching draft at that path
/w/{workspace}/drafts/move/{kind}/{path}:
post:
summary: move the current user's draft-only item to another path
@@ -17322,6 +17354,27 @@ paths:
schema:
type: string
/w/{workspace}/jobs/worker_is_draining/{id}:
get:
summary: whether the worker running this job is draining
description: |
True once the worker running the job has received its shutdown signal. The worker
still waits for the job to finish, so a long-running job can poll this to exit early.
False when the job is not currently running on a worker.
operationId: workerIsDraining
tags:
- job
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/JobId"
responses:
"200":
description: whether the worker running this job is draining
content:
application/json:
schema:
type: boolean
/w/{workspace}/jobs_u/get_logs/{id}:
get:
summary: get job logs
@@ -23502,6 +23555,54 @@ paths:
- id
- values
/workers/runnable_stats:
get:
summary: get resource usage per runnable and worker group over a time window (require devops role)
operationId: getRunnableStats
tags:
- worker
parameters:
- name: window_secs
in: query
required: false
description: how far back to read, in seconds (defaults to one day, capped at the 30-day retention)
schema:
type: integer
- name: worker_group
in: query
required: false
description: only the jobs run by this worker group
schema:
type: string
- name: workspace
in: query
required: false
description: only the jobs of this workspace
schema:
type: string
- name: order_by
in: query
required: false
description: "what the rows are ranked by, descending (default: total_duration)"
schema:
type: string
enum: [total_duration, max_memory, cpu]
- name: limit
in: query
required: false
description: number of rows to return (default 100, at most 1000)
schema:
type: integer
responses:
"200":
description: resource usage per runnable and worker group, heaviest first
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/RunnableStats"
/workers/queue_metrics_series:
get:
summary: get the queue metrics of a time window, as a bounded line per tag
@@ -35853,6 +35954,45 @@ components:
`u/<username>`, `g/<groupname>`, or an email that exists in this
workspace.
RunnableStats:
type: object
description: >
Resource usage of the jobs of one runnable on one worker group. Memory and CPU are
only known for jobs run as a subprocess; memory is the peak of the job's main process.
properties:
workspace_id:
type: string
runnable_path:
type: string
description: path of the script or flow step; `<adhoc>` gathers previews and other runs without a stable path
worker_group:
type: string
job_count:
type: integer
total_duration_ms:
type: integer
max_memory_peak:
type: integer
description: highest peak memory of a job, in kB
sum_memory_peak:
type: integer
description: sum of the peak memory of the jobs that have one, in kB
memory_sample_count:
type: integer
description: number of jobs that have a peak memory reading
total_cpu_ms:
type: integer
required:
- workspace_id
- runnable_path
- worker_group
- job_count
- total_duration_ms
- max_memory_peak
- sum_memory_peak
- memory_sample_count
- total_cpu_ms
WorkerPing:
type: object
properties:
@@ -35901,6 +36041,9 @@ components:
type: string
native_mode:
type: boolean
draining:
type: boolean
description: the worker received its shutdown signal and exits once its current job is done
required:
- worker
- worker_instance
backend/windmill-api/src/drafts.rs
+85 -1
View File
@@ -15,13 +15,14 @@ use axum::{
};
use serde::{Deserialize, Serialize};
use windmill_api_flows::flows::validate_operator_flow;
use windmill_audit::{audit_oss::audit_log, ActionKind};
use windmill_common::{
db::UserDB,
error::{Error, Result},
flows::FlowValue,
user_drafts::{DraftUserRef, UserDraftItemKind, ENCRYPTED_DRAFT_PREFIX},
users::resolve_username_to_email,
utils::{check_proper_path, strip_json_nul},
utils::{check_proper_path, require_admin, strip_json_nul},
variables::{build_crypt, encrypt},
workspaces::operator_can_build_flows,
};
@@ -32,6 +33,10 @@ pub fn workspaced_service() -> Router {
.route("/get/{kind}/{*path}", get(get_draft_for_user))
.route("/get_own/{kind}/{*path}", get(get_own_draft))
.route("/update/{kind}/{*path}", post(update_draft))
.route(
"/delete_for_user/{kind}/{*path}",
post(delete_draft_for_user),
)
.route("/move/{kind}/{*path}", post(move_draft))
.route("/migrate_legacy/{kind}/{*path}", post(migrate_legacy_draft))
}
@@ -1000,6 +1005,85 @@ async fn migrate_legacy_draft(
}
}
#[derive(Deserialize, Debug)]
pub struct DeleteDraftForUserQuery {
/// Workspace username of the draft owner. Absent: every draft at the path,
/// the legacy ownerless one and the caller's own included.
pub username: Option<String>,
}
/// Delete ANOTHER user's draft at a path, or every draft there, for workspace
/// admins (and superadmins, which carry `is_admin` in a workspace). Covers the
/// kinds whose authors are visible to others (`shares_drafts_across_users`); the
/// owner is named by workspace username, as on `/drafts/get`.
async fn delete_draft_for_user(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Path((w_id, kind, path)): Path<(String, UserDraftItemKind, windmill_common::utils::StripPath)>,
Query(query): Query<DeleteDraftForUserQuery>,
) -> Result<String> {
require_admin(authed.is_admin, &authed.username)?;
if !kind.shares_drafts_across_users() {
return Err(Error::NotFound(
"drafts for this item kind are private to their owner".to_string(),
));
}
let path = path.to_path();
let username = query.username.as_deref();
let owner_email = match username {
Some(username) => Some(
resolve_username_to_email(&w_id, username, &db)
.await?
.ok_or_else(|| {
Error::NotFound(format!("no user with username {username} in workspace"))
})?,
),
None => None,
};
let mut tx = db.begin().await?;
let deleted = sqlx::query_scalar!(
r#"DELETE FROM draft
WHERE workspace_id = $1 AND path = $2 AND typ = $3
AND ($4::text IS NULL OR email = $4)
RETURNING 1 as "one!""#,
&w_id,
path,
kind as UserDraftItemKind,
owner_email.as_deref(),
)
.fetch_all(&mut *tx)
.await?
.len();
if deleted == 0 {
return Err(Error::NotFound(match username {
Some(username) => format!("no draft for {username} at {path}"),
None => format!("no draft at {path}"),
}));
}
audit_log(
&mut *tx,
&authed,
"drafts.delete_for_user",
ActionKind::Delete,
&w_id,
Some(path),
Some(
[
("kind", kind.as_str()),
("username", username.unwrap_or("*")),
]
.into(),
),
)
.await?;
tx.commit().await?;
Ok(match username {
Some(username) => format!("Deleted {username}'s draft at {path}"),
None => format!("Deleted {deleted} draft(s) at {path}"),
})
}
/// For variable-kind drafts with `variable.is_secret == true`, encrypt
/// `variable.value` with the workspace crypt key and mark it
/// `$encrypted:<base64>` so the secret never persists in plaintext at rest.
backend/windmill-api/src/jobs.rs
+61 -5
View File
@@ -378,6 +378,10 @@ pub fn workspaced_service() -> Router {
"/wac_approval_urls/{job_id}/{step_key}",
get(get_wac_approval_urls).layer(cors.clone()),
)
.route(
"/worker_is_draining/{id}",
get(worker_is_draining).layer(cors.clone()),
)
.route(
"/result_by_id/{job_id}/{node_id}",
get(get_result_by_id).layer(cors.clone()),
@@ -613,6 +617,53 @@ async fn get_root_job(
Ok(Json(res))
}
/// Whether the worker running this job has received its shutdown signal. A draining worker
/// never interrupts its job, so a long-running script polls this to exit on its own terms.
async fn worker_is_draining(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Extension(user_db): Extension<UserDB>,
Path((w_id, id)): Path<(String, Uuid)>,
) -> windmill_common::error::JsonResult<bool> {
// A job always may ask about itself. Anyone else must be able to see the job: the
// caller's row policies on v2_job decide, narrowed by the token's tag filter.
let draining = if authed.job_id == Some(id) {
sqlx::query_scalar!(
"SELECT wp.draining FROM v2_job_queue q JOIN worker_ping wp ON wp.worker = q.worker
WHERE q.id = $1 AND q.workspace_id = $2 AND q.running AND q.started_at IS NOT NULL",
id,
w_id
)
.fetch_optional(&db)
.await?
} else {
let tags = get_scope_tags(&authed)
.map(|tags| tags.into_iter().map(str::to_string).collect::<Vec<_>>());
let mut tx = user_db.begin(&authed).await?;
let draining = sqlx::query_scalar!(
"SELECT wp.draining FROM v2_job j
JOIN v2_job_queue q ON q.id = j.id
JOIN worker_ping wp ON wp.worker = q.worker
WHERE j.id = $1 AND j.workspace_id = $2 AND q.running AND q.started_at IS NOT NULL
AND ($3::text[] IS NULL OR j.tag = ANY($3))",
id,
w_id,
tags.as_deref()
)
.fetch_optional(&mut *tx)
.await?;
tx.commit().await?;
draining
}
.unwrap_or(false);
windmill_common::feature_usage::log_feature_usage(
"worker_draining",
"check",
if draining { "draining" } else { "not_draining" },
);
Ok(Json(draining))
}
async fn compute_root_job_for_flow(db: &DB, w_id: &str, job_id: Uuid) -> error::Result<String> {
let root_job = sqlx::query_scalar!(
r#"SELECT COALESCE(root_job, flow_innermost_root_job, parent_job, id) as "root_job!" FROM v2_job WHERE id = $1 AND workspace_id = $2"#,
@@ -7775,6 +7826,7 @@ pub async fn run_workflow_as_code(
)
};
let end_user_email = run_end_user_email(&db, &w_id, &authed, None).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(&db, &authed).await?;
let (uuid, mut tx) = push(
&db,
@@ -7803,7 +7855,7 @@ pub async fn run_workflow_as_code(
None,
push_authed.as_ref(),
false,
None,
end_user_email,
None,
None,
scope_ceiling.as_deref(),
@@ -8090,6 +8142,7 @@ pub async fn run_wait_result_job_by_path_get(
)
};
let end_user_email = run_end_user_email(&db, &w_id, &authed, None).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(&db, &authed).await?;
let (uuid, tx) = push(
&db,
@@ -8118,7 +8171,7 @@ pub async fn run_wait_result_job_by_path_get(
None,
push_authed.as_ref(),
false,
None,
end_user_email,
authed.trigger_or_fallback(None),
run_query.suspended_mode,
scope_ceiling.as_deref(),
@@ -8240,6 +8293,7 @@ pub async fn run_wait_result_script_by_path_internal(
)
};
let end_user_email = run_end_user_email(&db, &w_id, &authed, None).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(&db, &authed).await?;
let (uuid, tx) = push(
&db,
@@ -8268,7 +8322,7 @@ pub async fn run_wait_result_script_by_path_internal(
None,
push_authed.as_ref(),
false,
None,
end_user_email,
authed.trigger_or_fallback(None),
run_query.suspended_mode,
scope_ceiling.as_deref(),
@@ -8358,6 +8412,7 @@ pub async fn run_wait_result_script_by_hash(
)
};
let end_user_email = run_end_user_email(&db, &w_id, &authed, None).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(&db, &authed).await?;
let (uuid, tx) = push(
&db,
@@ -8400,7 +8455,7 @@ pub async fn run_wait_result_script_by_hash(
None,
push_authed.as_ref(),
false,
None,
end_user_email,
authed.trigger_or_fallback(None),
run_query.suspended_mode,
scope_ceiling.as_deref(),
@@ -10505,6 +10560,7 @@ pub async fn run_job_by_hash_inner(
};
let job_payload = with_run_retry(&run_query, job_payload, &push_args, &tag)?;
let end_user_email = run_end_user_email(&db, &w_id, &authed, trigger.as_ref()).await?;
let scope_ceiling = windmill_api_auth::caller_scope_ceiling(&db, &authed).await?;
let (uuid, tx) = push(
&db,
@@ -10533,7 +10589,7 @@ pub async fn run_job_by_hash_inner(
None,
push_authed.as_ref(),
false,
None,
end_user_email,
authed.trigger_or_fallback(trigger),
run_query.suspended_mode,
scope_ceiling.as_deref(),
backend/windmill-common/src/global_settings.rs
+34 -7
View File
@@ -181,6 +181,12 @@ pub const GITHUB_ENTERPRISE_APP_SETTING: &str = "github_enterprise_app";
/// `base_url` when unset; set it when the browser-facing URL is not reachable
/// from GitHub and a separate ingress fronts the API for inbound webhooks.
pub const GITHUB_APP_WEBHOOK_BASE_URL_SETTING: &str = "github_app_webhook_base_url";
/// Base URL the UI shows for the endpoints external clients call (webhooks, HTTP
/// routes, push trigger endpoints, the CLI), instead of the origin the UI is browsed on.
/// For deployments that serve the API on a separate domain or behind a gateway.
/// EE only. Readable by any authenticated user, like [`INSTANCE_BANNER_SETTING`]:
/// every user is shown URLs built from it.
pub const API_BASE_URL_SETTING: &str = "api_base_url";
/// Instance-wide announcement rendered above every page of the app (maintenance
/// windows, incidents). Readable by any authenticated user, unlike most settings:
/// the banner exists to be shown to everyone, so it must never hold anything the
@@ -313,12 +319,17 @@ pub fn validate_accent_color(value: &serde_json::Value) -> Result<(), String> {
pub struct InstanceUi {
pub instance_banner: Option<serde_json::Value>,
pub accent_color: Option<serde_json::Value>,
pub api_base_url: Option<serde_json::Value>,
}
pub async fn get_instance_ui(db: &Pool<Postgres>) -> error::Result<InstanceUi> {
let rows = sqlx::query!(
"SELECT name, value FROM global_settings WHERE name = ANY($1)",
&[INSTANCE_BANNER_SETTING, ACCENT_COLOR_SETTING] as &[&str]
&[
INSTANCE_BANNER_SETTING,
ACCENT_COLOR_SETTING,
API_BASE_URL_SETTING
] as &[&str]
)
.fetch_all(db)
.await?;
@@ -327,17 +338,22 @@ pub async fn get_instance_ui(db: &Pool<Postgres>) -> error::Result<InstanceUi> {
match row.name.as_str() {
INSTANCE_BANNER_SETTING => ui.instance_banner = Some(row.value),
ACCENT_COLOR_SETTING => ui.accent_color = Some(row.value),
API_BASE_URL_SETTING => ui.api_base_url = Some(row.value),
_ => {}
}
}
Ok(ui)
}
/// Validate a [`GITHUB_APP_WEBHOOK_BASE_URL_SETTING`] value.
const SHELL_ACTIVE_CHARS: [char; 16] = [
'$', '`', '\'', '"', '\\', ';', '&', '|', '<', '>', '(', ')', '{', '}', '*', '!',
];
/// Validate a [`GITHUB_APP_WEBHOOK_BASE_URL_SETTING`] or [`API_BASE_URL_SETTING`] value.
///
/// The receiver path is appended to it verbatim, so anything that doesn't
/// concatenate into a URL GitHub can POST to must be rejected at write time
/// rather than silently producing an unreachable hook: a wrong scheme
/// A path is appended to it verbatim, so anything that doesn't concatenate into a
/// URL an external caller can reach must be rejected at write time rather than
/// silently producing an unreachable hook or endpoint: a wrong scheme
/// (`httpss://`), a missing host, embedded whitespace, or a query/fragment
/// (appending a path after `?`/`#` keeps it inside the query/fragment).
///
@@ -353,7 +369,7 @@ pub fn validate_webhook_base_url(value: &str) -> Result<(), String> {
url::Url::parse(value).map_err(|e| format!("must be an absolute http(s) URL: {e}"))?;
if !url.username().is_empty() || url.password().is_some() {
return Err(
"must not embed a username or password: the receiver URL is stored in workspace settings, where it is readable by workspace admins".to_string(),
"must not embed a username or password: the URL is readable by users who are not instance admins".to_string(),
);
}
if !matches!(url.scheme(), "http" | "https") {
@@ -366,13 +382,21 @@ pub fn validate_webhook_base_url(value: &str) -> Result<(), String> {
}
if url.query().is_some() || url.fragment().is_some() {
return Err(
"must not include a query string or fragment, since the webhook path is appended to it"
"must not include a query string or fragment, since a path is appended to it"
.to_string(),
);
}
if value.chars().any(char::is_whitespace) {
return Err("must not contain whitespace".to_string());
}
// The UI pastes the URL into shell commands users copy, where double quotes
// do not stop `$(...)` or backticks from running.
if value.contains(SHELL_ACTIVE_CHARS) {
return Err(
"must not contain shell metacharacters, since it is shown inside commands users copy"
.to_string(),
);
}
// Matches the sibling `base_url` / `hub_base_url` convention. The receiver
// builder trims it anyway, so this is about keeping the stored value canonical
// rather than about reachability.
@@ -1101,6 +1125,9 @@ mod tests {
"https://hooks.example.com#",
"https://user:password@hooks.example.com",
"https://user@hooks.example.com",
"https://hooks.example.com/$(uname)",
"https://hooks.example.com/`uname`",
"https://hooks.example.com/a;b",
];
for v in accept {
assert!(
backend/windmill-common/src/instance_config.rs
+33 -25
View File
@@ -281,6 +281,8 @@ pub struct GlobalSettings {
#[serde(skip_serializing_if = "Option::is_none")]
pub github_app_webhook_base_url: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub api_base_url: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub email_domain: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub hub_base_url: Option<String>,
@@ -1375,8 +1377,8 @@ pub fn diff_worker_configs(
}
/// Declaratively replace the global settings, rejecting a
/// `github_app_webhook_base_url` or `http_route_default_allowed_origins` the
/// API would reject.
/// `github_app_webhook_base_url`, `api_base_url` or
/// `http_route_default_allowed_origins` the API would reject.
///
/// Every declarative writer (the `sync-config` CLI, the Kubernetes operator's
/// ConfigMap sync) MUST go through this rather than calling
@@ -1399,29 +1401,35 @@ pub async fn sync_global_settings_declarative(
current: &BTreeMap<String, serde_json::Value>,
desired: &BTreeMap<String, serde_json::Value>,
) -> anyhow::Result<()> {
let webhook_key = crate::global_settings::GITHUB_APP_WEBHOOK_BASE_URL_SETTING;
// Non-string shapes are rejected rather than ignored: `as_str()` alone would let a
// bool/number/object through as if the key were absent, and the diff below would
// then persist it — where the HTTP path answers "must be a URL string".
match desired.get(webhook_key) {
None | Some(serde_json::Value::Null) => {}
Some(serde_json::Value::String(s)) if s.trim().is_empty() => {}
Some(serde_json::Value::String(s)) => crate::global_settings::validate_webhook_base_url(s)
.map_err(|e| anyhow::anyhow!("{webhook_key}: {e}"))?,
// Names the JSON kind rather than printing it: this is the last message on
// this path that could report submitted content, and an object or array could
// carry a secret into `sync-config` output and operator logs.
Some(other) => {
let kind = match other {
serde_json::Value::Bool(_) => "a boolean",
serde_json::Value::Number(_) => "a number",
serde_json::Value::Array(_) => "an array",
serde_json::Value::Object(_) => "an object",
_ => "a non-string value",
};
return Err(anyhow::anyhow!(
"{webhook_key} must be a URL string, got {kind}"
));
for url_key in [
crate::global_settings::GITHUB_APP_WEBHOOK_BASE_URL_SETTING,
crate::global_settings::API_BASE_URL_SETTING,
] {
// Non-string shapes are rejected rather than ignored: `as_str()` alone would let a
// bool/number/object through as if the key were absent, and the diff below would
// then persist it — where the HTTP path answers "must be a URL string".
match desired.get(url_key) {
None | Some(serde_json::Value::Null) => {}
Some(serde_json::Value::String(s)) if s.trim().is_empty() => {}
Some(serde_json::Value::String(s)) => {
crate::global_settings::validate_webhook_base_url(s)
.map_err(|e| anyhow::anyhow!("{url_key}: {e}"))?
}
// Names the JSON kind rather than printing it: this is the last message on
// this path that could report submitted content, and an object or array could
// carry a secret into `sync-config` output and operator logs.
Some(other) => {
let kind = match other {
serde_json::Value::Bool(_) => "a boolean",
serde_json::Value::Number(_) => "a number",
serde_json::Value::Array(_) => "an array",
serde_json::Value::Object(_) => "an object",
_ => "a non-string value",
};
return Err(anyhow::anyhow!(
"{url_key} must be a URL string, got {kind}"
));
}
}
}
backend/windmill-common/src/lib.rs
+1
View File
@@ -123,6 +123,7 @@ pub mod query_builders;
pub mod queue;
pub mod queue_metrics;
pub mod result_stream;
pub mod runnable_job_stats;
pub mod runnable_settings;
pub mod runnables;
pub mod schedule;
backend/windmill-common/src/otel_oss.rs
+33
View File
@@ -132,3 +132,36 @@ pub fn otel_set_health_status_phase(_phase: &str) {}
#[cfg(not(feature = "private"))]
pub fn otel_set_health_db_unresponsive(_unresponsive: bool) {}
#[cfg(not(feature = "private"))]
pub fn otel_set_worker_memory(
_worker: &str,
_worker_group: &str,
_usage: Option<i64>,
_windmill_usage: Option<i64>,
_limit: Option<i64>,
) {
}
#[cfg(not(feature = "private"))]
pub fn otel_add_worker_cpu(
_worker: &str,
_worker_group: &str,
_usage_secs: f64,
_periods: u64,
_throttled_periods: u64,
_throttled_secs: f64,
) {
}
#[cfg(not(feature = "private"))]
pub fn otel_set_worker_occupancy_rate(
_worker: &str,
_worker_group: &str,
_window: &'static str,
_rate: f64,
) {
}
#[cfg(not(feature = "private"))]
pub fn otel_record_job_memory_peak(_tag: &str, _bytes: u64) {}
backend/windmill-common/src/runnable_job_stats.rs
+288
View File
@@ -0,0 +1,288 @@
//! Hourly resource rollup per runnable and worker group (`runnable_job_stats`).
//!
//! Workers accumulate completed jobs in memory and upsert periodically, so that the
//! "heaviest scripts" view never has to aggregate `v2_job_completed`.
//!
//! What a row leaves out: executors that run in the worker process (SQL, native
//! TypeScript, ...) have no CPU reading, and for memory report nothing or, for some SQL
//! ones, the size of their result, as the job's `memory_peak` does. Memory is otherwise
//! the peak of the job's main process only, and CPU counts that process and the children
//! it waited for, and nothing for a job that was killed.
//!
//! Nothing here checks authorization: these functions write and prune every workspace's
//! rows, and are for the worker's completion path and the server's monitor only. Reads
//! go through the devops-gated API handler.
use std::{collections::HashMap, sync::Mutex};
use sqlx::{Pool, Postgres};
use crate::{jobs::JobKind, worker_group_job_stats::get_current_hour};
/// Every run without a stable path is counted under this one, per workspace and worker
/// group: previews and other ad-hoc runs carry caller-chosen paths that would otherwise
/// grow the table without bound.
pub const ADHOC_RUNNABLE_PATH: &str = "<adhoc>";
/// Also the furthest back the API reads.
pub const RETENTION_DAYS: i64 = 30;
#[derive(Debug, Default, Clone, Copy, PartialEq)]
struct RunnableStats {
job_count: i32,
total_duration_ms: i64,
max_memory_peak: i32,
sum_memory_peak: i64,
memory_sample_count: i32,
total_cpu_ms: i64,
}
impl RunnableStats {
/// `mem_peak` is in kB and non-positive when the job had no reading.
fn add_job(&mut self, duration_ms: i64, mem_peak: i32, cpu_time_ms: Option<i64>) {
self.job_count += 1;
self.total_duration_ms += duration_ms;
if mem_peak > 0 {
self.max_memory_peak = self.max_memory_peak.max(mem_peak);
self.sum_memory_peak += mem_peak as i64;
self.memory_sample_count += 1;
}
self.total_cpu_ms += cpu_time_ms.unwrap_or(0).max(0);
}
fn merge(&mut self, other: &RunnableStats) {
self.job_count += other.job_count;
self.total_duration_ms += other.total_duration_ms;
self.max_memory_peak = self.max_memory_peak.max(other.max_memory_peak);
self.sum_memory_peak += other.sum_memory_peak;
self.memory_sample_count += other.memory_sample_count;
self.total_cpu_ms += other.total_cpu_ms;
}
}
/// (hour, workspace_id, runnable_path, worker_group)
type StatsKey = (i64, String, String, String);
lazy_static::lazy_static! {
static ref STATS: Mutex<HashMap<StatsKey, RunnableStats>> = Mutex::new(HashMap::new());
}
/// The path a job is rolled up under: its own for the kinds whose path names something
/// deployed (a flow step's is `<flow path>/<step id>`), the ad-hoc bucket otherwise.
pub fn stats_path(kind: JobKind, runnable_path: Option<&str>) -> &str {
match (kind, runnable_path) {
(
JobKind::Script
| JobKind::Script_Hub
| JobKind::FlowScript
| JobKind::FlowNode
| JobKind::AIAgent,
Some(path),
) if !path.is_empty() => path,
_ => ADHOC_RUNNABLE_PATH,
}
}
pub fn accumulate_runnable_job_stats(
workspace_id: &str,
kind: JobKind,
runnable_path: Option<&str>,
worker_group: &str,
duration_ms: i64,
mem_peak: i32,
cpu_time_ms: Option<i64>,
) {
let key = (
get_current_hour(),
workspace_id.to_string(),
stats_path(kind, runnable_path).to_string(),
worker_group.to_string(),
);
STATS
.lock()
.unwrap()
.entry(key)
.or_default()
.add_job(duration_ms, mem_peak, cpu_time_ms);
}
/// Counts the time spent on a round of a job that is not completing, without counting a
/// job: a suspended workflow-as-code round replays its finished steps for free when it
/// resumes, so the time they took would otherwise never reach the rollup. A row can
/// therefore hold time and no job until the workflow completes.
pub fn accumulate_runnable_round(
workspace_id: &str,
kind: JobKind,
runnable_path: Option<&str>,
worker_group: &str,
duration_ms: i64,
cpu_time_ms: Option<i64>,
) {
let key = (
get_current_hour(),
workspace_id.to_string(),
stats_path(kind, runnable_path).to_string(),
worker_group.to_string(),
);
let mut stats = STATS.lock().unwrap();
let entry = stats.entry(key).or_default();
entry.total_duration_ms += duration_ms;
entry.total_cpu_ms += cpu_time_ms.unwrap_or(0).max(0);
}
pub async fn flush_runnable_job_stats(db: &Pool<Postgres>) -> Result<(), sqlx::Error> {
let drained: Vec<(StatsKey, RunnableStats)> = {
let mut stats = STATS.lock().unwrap();
if stats.is_empty() {
return Ok(());
}
stats.drain().collect()
};
let mut hours = Vec::with_capacity(drained.len());
let mut workspace_ids = Vec::with_capacity(drained.len());
let mut runnable_paths = Vec::with_capacity(drained.len());
let mut worker_groups = Vec::with_capacity(drained.len());
let mut job_counts = Vec::with_capacity(drained.len());
let mut durations = Vec::with_capacity(drained.len());
let mut max_peaks = Vec::with_capacity(drained.len());
let mut sum_peaks = Vec::with_capacity(drained.len());
let mut sample_counts = Vec::with_capacity(drained.len());
let mut cpus = Vec::with_capacity(drained.len());
for ((hour, workspace_id, runnable_path, worker_group), s) in &drained {
hours.push(*hour);
workspace_ids.push(workspace_id.clone());
runnable_paths.push(runnable_path.clone());
worker_groups.push(worker_group.clone());
job_counts.push(s.job_count);
durations.push(s.total_duration_ms);
max_peaks.push(s.max_memory_peak);
sum_peaks.push(s.sum_memory_peak);
sample_counts.push(s.memory_sample_count);
cpus.push(s.total_cpu_ms);
}
// The join drops rows of a workspace deleted since its jobs ran. The ORDER BY makes
// every worker lock the rows they share in one order: without it two concurrent
// flushes can deadlock, and the join is free to undo an ordering of the arrays.
let res = sqlx::query!(
r#"
INSERT INTO runnable_job_stats
(hour, workspace_id, runnable_path, worker_group, job_count, total_duration_ms,
max_memory_peak, sum_memory_peak, memory_sample_count, total_cpu_ms)
SELECT s.hour, s.workspace_id, s.runnable_path, s.worker_group, s.job_count,
s.total_duration_ms, s.max_memory_peak, s.sum_memory_peak,
s.memory_sample_count, s.total_cpu_ms
FROM UNNEST($1::bigint[], $2::text[], $3::text[], $4::text[], $5::int[], $6::bigint[],
$7::int[], $8::bigint[], $9::int[], $10::bigint[])
AS s(hour, workspace_id, runnable_path, worker_group, job_count, total_duration_ms,
max_memory_peak, sum_memory_peak, memory_sample_count, total_cpu_ms)
JOIN workspace w ON w.id = s.workspace_id
ORDER BY s.hour, s.workspace_id, s.runnable_path, s.worker_group
ON CONFLICT (hour, workspace_id, runnable_path, worker_group) DO UPDATE SET
job_count = runnable_job_stats.job_count + EXCLUDED.job_count,
total_duration_ms = runnable_job_stats.total_duration_ms + EXCLUDED.total_duration_ms,
max_memory_peak = GREATEST(runnable_job_stats.max_memory_peak, EXCLUDED.max_memory_peak),
sum_memory_peak = runnable_job_stats.sum_memory_peak + EXCLUDED.sum_memory_peak,
memory_sample_count = runnable_job_stats.memory_sample_count + EXCLUDED.memory_sample_count,
total_cpu_ms = runnable_job_stats.total_cpu_ms + EXCLUDED.total_cpu_ms
"#,
&hours,
&workspace_ids,
&runnable_paths,
&worker_groups,
&job_counts,
&durations,
&max_peaks,
&sum_peaks,
&sample_counts,
&cpus,
)
.execute(db)
.await;
if let Err(e) = res {
let mut stats = STATS.lock().unwrap();
for (key, s) in drained {
stats.entry(key).or_default().merge(&s);
}
return Err(e);
}
Ok(())
}
/// Also removes the rows of workspaces that no longer exist: a flush that overlaps a
/// workspace's deletion still sees the workspace and can write after the deletion's own
/// cleanup ran. Such a row is at most a couple of hours old, so only the last day is
/// searched, which keeps both statements on a range of the primary key.
pub async fn cleanup_old_runnable_job_stats(
db: &Pool<Postgres>,
retention_days: i64,
) -> Result<u64, sqlx::Error> {
let current_hour = get_current_hour();
let expired = sqlx::query!(
"DELETE FROM runnable_job_stats WHERE hour < $1",
current_hour - retention_days * 24 * 3600
)
.execute(db)
.await?;
let orphaned = sqlx::query!(
"DELETE FROM runnable_job_stats s WHERE s.hour >= $1 \
AND NOT EXISTS (SELECT 1 FROM workspace w WHERE w.id = s.workspace_id)",
current_hour - 24 * 3600
)
.execute(db)
.await?;
Ok(expired.rows_affected() + orphaned.rows_affected())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn only_deployed_runnables_keep_their_path() {
assert_eq!(
stats_path(JobKind::Script, Some("f/etl/load")),
"f/etl/load"
);
assert_eq!(
stats_path(JobKind::FlowScript, Some("f/etl/flow/a")),
"f/etl/flow/a"
);
// A preview carries the path being edited, which is not a stable identity.
assert_eq!(
stats_path(JobKind::Preview, Some("f/etl/load")),
ADHOC_RUNNABLE_PATH
);
assert_eq!(
stats_path(JobKind::FlowPreview, Some("f/etl/flow")),
ADHOC_RUNNABLE_PATH
);
assert_eq!(
stats_path(JobKind::Dependencies, Some("f/etl/load")),
ADHOC_RUNNABLE_PATH
);
assert_eq!(stats_path(JobKind::Script, None), ADHOC_RUNNABLE_PATH);
}
#[test]
fn jobs_without_a_memory_reading_do_not_dilute_it() {
let mut s = RunnableStats::default();
s.add_job(1000, 2048, Some(700));
s.add_job(500, -3, None);
s.add_job(250, 0, Some(-5));
s.add_job(250, 1024, Some(50));
assert_eq!(
s,
RunnableStats {
job_count: 4,
total_duration_ms: 2000,
max_memory_peak: 2048,
sum_memory_peak: 3072,
memory_sample_count: 2,
total_cpu_ms: 750,
}
);
}
}
backend/windmill-common/src/variables.rs
+35 -9
View File
@@ -14,7 +14,7 @@ use crate::utils::WarnAfterExt;
use crate::worker::Connection;
use crate::{worker::WORKER_GROUP, BASE_URL, DB};
use chrono::{SecondsFormat, Utc};
use magic_crypt::{MagicCrypt256, MagicCryptError, MagicCryptTrait};
use magic_crypt::{MagicCrypt256, MagicCryptTrait};
use quick_cache::sync::Cache;
use serde::{Deserialize, Serialize};
@@ -375,13 +375,15 @@ pub fn encrypt(mc: &MagicCrypt256, value: &str) -> String {
mc.encrypt_str_to_base64(value)
}
const DECRYPT_FAILED: &str =
"Could not decrypt value. The value may have been encrypted with a different key.";
/// Every failure returns the same message so responses carry no signal about why decryption
/// failed (padding, base64, UTF-8); the cause is only logged.
pub fn decrypt(mc: &MagicCrypt256, value: String) -> error::Result<String> {
mc.decrypt_base64_to_string(value).map_err(|e| match e {
MagicCryptError::DecryptError(_) => error::Error::internal_err(
"Could not decrypt value. The value may have been encrypted with a different key."
.to_string(),
),
_ => error::Error::internal_err(e.to_string()),
mc.decrypt_base64_to_string(value).map_err(|e| {
tracing::warn!("decryption failed: {e}");
error::Error::internal_err(DECRYPT_FAILED.to_string())
})
}
@@ -595,7 +597,7 @@ pub async fn get_reserved_variables(
ContextualVariable {
name: "WM_END_USER_EMAIL".to_string(),
value: end_user_email.unwrap_or_else(|| "".to_string()),
description: "Email of the end user that executed the current script, propagated to flow steps. Only set when the run was triggered from an app (empty otherwise).".to_string(),
description: "Email of the end user that executed the current script, even when it runs on behalf of another identity, propagated to flow steps and child jobs. Set for app and API runs (for a webhook, the owner of the token it was called with), empty for runs fired by a schedule or by a trigger such as an HTTP route or Kafka.".to_string(),
is_custom: false,
},
ContextualVariable {
@@ -746,7 +748,31 @@ pub async fn get_variable_or_self_as<T: Authable + Sync>(
#[cfg(test)]
mod tests {
use super::{can_bind_as_prologue_const, escape_js_single_quoted, is_valid_js_identifier};
use super::{
can_bind_as_prologue_const, decrypt, escape_js_single_quoted, is_valid_js_identifier,
DECRYPT_FAILED,
};
use crate::error::Error;
use magic_crypt::MagicCryptTrait;
#[test]
fn decrypt_failures_all_return_the_same_message() {
let mc = magic_crypt::new_magic_crypt!("workspace-key", 256);
let other = magic_crypt::new_magic_crypt!("another-key", 256);
let failures = [
"not base64 at all!".to_string(),
mc.encrypt_bytes_to_base64(&[0xff, 0xfe, 0xfd]),
other.encrypt_str_to_base64("secret"),
];
for value in failures {
match decrypt(&mc, value.clone()) {
Err(Error::InternalErrLoc { message, .. }) => {
assert_eq!(message, DECRYPT_FAILED, "for {value:?}")
}
other => panic!("expected the uniform decrypt error for {value:?}, got {other:?}"),
}
}
}
#[test]
fn is_valid_js_identifier_gates_prologue_injection() {
backend/windmill-common/src/worker.rs
+115 -2
View File
@@ -1783,6 +1783,73 @@ pub fn get_cpu_period() -> Option<i64> {
Some(100000)
}
/// Bucket bounds in bytes, 16 MiB to 32 GiB, shared by the Prometheus and OTel job
/// peak-memory histograms so the two exports stay comparable.
pub const JOB_MEMORY_PEAK_BUCKETS: [f64; 12] = {
let mut buckets = [0.0; 12];
let mut i = 0;
while i < 12 {
buckets[i] = (16u64 << (20 + i)) as f64;
i += 1;
}
buckets
};
/// Cumulative CPU accounting of the worker's cgroup. The throttling fields stay at 0
/// when the cgroup has no CPU controller, which is also what an unthrottled one reports.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct CpuStat {
pub usage_usec: u64,
pub nr_periods: u64,
pub nr_throttled: u64,
pub throttled_usec: u64,
}
fn cpu_stat_field(stat: &str, key: &str) -> Option<u64> {
stat.lines().find_map(|line| {
let (k, v) = line.split_once(' ')?;
(k == key).then(|| v.trim().parse().ok()).flatten()
})
}
fn parse_cgroup_v2_cpu_stat(stat: &str) -> Option<CpuStat> {
Some(CpuStat {
usage_usec: cpu_stat_field(stat, "usage_usec")?,
nr_periods: cpu_stat_field(stat, "nr_periods").unwrap_or(0),
nr_throttled: cpu_stat_field(stat, "nr_throttled").unwrap_or(0),
throttled_usec: cpu_stat_field(stat, "throttled_usec").unwrap_or(0),
})
}
/// cgroup v1 splits the same figures over two controllers and counts time in nanoseconds.
fn parse_cgroup_v1_cpu_stat(cpuacct_usage: &str, stat: &str) -> Option<CpuStat> {
Some(CpuStat {
usage_usec: cpuacct_usage.trim().parse::<u64>().ok()? / 1000,
nr_periods: cpu_stat_field(stat, "nr_periods").unwrap_or(0),
nr_throttled: cpu_stat_field(stat, "nr_throttled").unwrap_or(0),
throttled_usec: cpu_stat_field(stat, "throttled_time").unwrap_or(0) / 1000,
})
}
#[cfg(not(windows))]
pub fn get_cpu_stat() -> Option<CpuStat> {
const V1_USAGE: &str = "/sys/fs/cgroup/cpuacct/cpuacct.usage";
if Path::new(V1_USAGE).exists() {
parse_cgroup_v1_cpu_stat(
&std::fs::read_to_string(V1_USAGE).ok()?,
&std::fs::read_to_string("/sys/fs/cgroup/cpu/cpu.stat").unwrap_or_default(),
)
} else {
let cgroup_path = get_cgroupv2_path()?;
parse_cgroup_v2_cpu_stat(&std::fs::read_to_string(format!("{cgroup_path}/cpu.stat")).ok()?)
}
}
#[cfg(windows)]
pub fn get_cpu_stat() -> Option<CpuStat> {
None
}
/// CPUs the process is allowed to run on, ignoring any bandwidth quota — the count
/// Go's `NumCPU` reports. `available_parallelism` cannot stand in for it: that folds
/// the quota in, so a fraction of a CPU makes it report a single-core machine.
@@ -1952,6 +2019,7 @@ pub enum PingType {
MainLoop,
Job,
InitScript,
Draining,
}
#[derive(Serialize, Deserialize)]
pub struct Ping {
@@ -1974,6 +2042,8 @@ pub struct Ping {
pub occupancy_rate_30m: Option<f32>,
pub job_isolation: Option<String>,
pub native_mode: Option<bool>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub draining: Option<bool>,
pub ping_type: PingType,
}
pub async fn update_ping_http(
@@ -2026,6 +2096,7 @@ pub async fn update_ping_http(
insert_ping.memory,
insert_ping.job_isolation,
insert_ping.native_mode.unwrap_or(false),
insert_ping.draining,
db,
)
.await?;
@@ -2054,10 +2125,21 @@ pub async fn update_ping_http(
)
.await?
}
PingType::Draining => set_worker_draining_query(worker_name, db).await?,
}
Ok(())
}
pub async fn set_worker_draining_query(worker_name: &str, db: &DB) -> anyhow::Result<()> {
sqlx::query!(
"UPDATE worker_ping SET draining = true WHERE worker = $1",
worker_name
)
.execute(db)
.await?;
Ok(())
}
#[derive(Debug, Serialize, Deserialize)]
pub struct JobCancelled {
pub canceled_by: String,
@@ -2156,6 +2238,9 @@ pub async fn fetch_raw_script_from_app_query(
/// `started_at` and `jobs_executed` are the only two columns carried over, being the
/// continuity itself — plus `ip` for as long as `ip` is `None`, which means the external IP
/// lookup has not resolved yet and the predecessor's address is still the best guess.
///
/// `draining` is `Some(false)` from a worker starting up and `None` from one re-registering
/// mid-life, which must leave alone a flag its own shutdown may have set in the meantime.
pub async fn insert_ping_query(
worker_instance: &str,
worker_name: &str,
@@ -2169,6 +2254,7 @@ pub async fn insert_ping_query(
memory: Option<i64>,
job_isolation: Option<String>,
native_mode: bool,
draining: Option<bool>,
db: &DB,
) -> anyhow::Result<i32> {
// A NULL `ip` means the external IP lookup is still in flight; a later ping fills it in, and
@@ -2176,8 +2262,8 @@ pub async fn insert_ping_query(
// lookup that has failed reports `external_ip::UNRETRIEVABLE_IP`, which does overwrite it. The
// literal below must stay equal to `external_ip::UNKNOWN_IP`.
let previous_jobs_executed = sqlx::query_scalar!(
"INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation, native_mode) VALUES ($1, $2, COALESCE($3, 'NO IP'), $4, $5, $6, $7, $8, $9, $10, $11, $12) ON CONFLICT (worker)
DO UPDATE set ping_at = now(), worker_instance = EXCLUDED.worker_instance, ip = COALESCE($3, worker_ping.ip), custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_worker = EXCLUDED.dedicated_worker, dedicated_workers = EXCLUDED.dedicated_workers, wm_version = EXCLUDED.wm_version, vcpus = COALESCE(EXCLUDED.vcpus, worker_ping.vcpus), memory = COALESCE(EXCLUDED.memory, worker_ping.memory), job_isolation = EXCLUDED.job_isolation, native_mode = EXCLUDED.native_mode, current_job_id = NULL, current_job_workspace_id = NULL
"INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation, native_mode, draining) VALUES ($1, $2, COALESCE($3, 'NO IP'), $4, $5, $6, $7, $8, $9, $10, $11, $12, COALESCE($13, false)) ON CONFLICT (worker)
DO UPDATE set ping_at = now(), worker_instance = EXCLUDED.worker_instance, ip = COALESCE($3, worker_ping.ip), custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_worker = EXCLUDED.dedicated_worker, dedicated_workers = EXCLUDED.dedicated_workers, wm_version = EXCLUDED.wm_version, vcpus = COALESCE(EXCLUDED.vcpus, worker_ping.vcpus), memory = COALESCE(EXCLUDED.memory, worker_ping.memory), job_isolation = EXCLUDED.job_isolation, native_mode = EXCLUDED.native_mode, current_job_id = NULL, current_job_workspace_id = NULL, draining = COALESCE($13, worker_ping.draining)
RETURNING jobs_executed",
worker_instance,
worker_name,
@@ -2191,6 +2277,7 @@ pub async fn insert_ping_query(
memory,
job_isolation.as_deref(),
native_mode,
draining,
)
.fetch_one(db)
.await?;
@@ -2899,6 +2986,32 @@ mod tests {
use super::*;
use std::collections::HashMap;
#[test]
fn cgroup_cpu_stat_is_normalized_to_microseconds() {
let v2 = "usage_usec 2500000\nuser_usec 2000000\nsystem_usec 500000\n\
nr_periods 40\nnr_throttled 7\nthrottled_usec 350000\nnr_bursts 0\n";
let expected = CpuStat {
usage_usec: 2_500_000,
nr_periods: 40,
nr_throttled: 7,
throttled_usec: 350_000,
};
assert_eq!(parse_cgroup_v2_cpu_stat(v2), Some(expected));
let v1_stat = "nr_periods 40\nnr_throttled 7\nthrottled_time 350000000\n";
assert_eq!(
parse_cgroup_v1_cpu_stat("2500000000\n", v1_stat),
Some(expected)
);
// No CPU controller: usage is still reported, throttling reads as zero.
assert_eq!(
parse_cgroup_v2_cpu_stat("usage_usec 12\nuser_usec 8\nsystem_usec 4\n"),
Some(CpuStat { usage_usec: 12, ..Default::default() })
);
assert_eq!(parse_cgroup_v2_cpu_stat("nr_periods 40\n"), None);
}
#[test]
fn datatable_role_is_read_from_the_leading_comment_block() {
let role = |code| SqlAnnotations::datatable_role(code);
backend/windmill-queue/src/jobs.rs
+113 -32
View File
@@ -182,6 +182,15 @@ pub struct CanceledBy {
pub reason: Option<String>,
}
/// What only the worker that ran a job knows about it, for the per-runnable rollup: an
/// agent worker's completion is processed by the server, whose own worker group is not
/// the one the job ran in.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct JobResourceUsage {
pub worker_group: String,
pub cpu_time_ms: Option<i64>,
}
#[derive(Debug, Serialize, Deserialize)]
pub struct JobCompleted {
pub job: MiniCompletedJob,
@@ -196,6 +205,8 @@ pub struct JobCompleted {
pub duration: Option<i64>,
pub has_stream: Option<bool>,
pub from_cache: Option<bool>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub resource_usage: Option<JobResourceUsage>,
#[serde(skip)]
pub flow_runners: Option<Arc<FlowRunners>>,
#[serde(skip)]
@@ -1221,7 +1232,7 @@ pub async fn add_completed_job_pre_shaped_failure<T: Serialize + Send + Sync + V
worker_name: &str,
flow_is_done: bool,
duration: Option<i64>,
) -> Result<(), Error> {
) -> Result<CompletedSpan, Error> {
record_failure_metrics(completed_job, worker_name).await;
tracing::error!(
@@ -1229,7 +1240,7 @@ pub async fn add_completed_job_pre_shaped_failure<T: Serialize + Send + Sync + V
completed_job.id,
completed_job.workspace_id,
);
let _ = add_completed_job(
let (_, span) = add_completed_job(
db,
completed_job,
false,
@@ -1244,7 +1255,7 @@ pub async fn add_completed_job_pre_shaped_failure<T: Serialize + Send + Sync + V
)
.warn_after_seconds(10)
.await?;
Ok(())
Ok(span)
}
pub async fn add_completed_job_error(
@@ -1257,6 +1268,31 @@ pub async fn add_completed_job_error(
flow_is_done: bool,
duration: Option<i64>,
) -> Result<WrappedError, Error> {
add_completed_job_error_with_span(
db,
completed_job,
mem_peak,
canceled_by,
e,
worker_name,
flow_is_done,
duration,
)
.await
.map(|(wrapped, _)| wrapped)
}
/// `add_completed_job_error`, also handing back what the completed row holds.
pub async fn add_completed_job_error_with_span(
db: &Pool<Postgres>,
completed_job: &MiniCompletedJob,
mem_peak: i32,
canceled_by: Option<CanceledBy>,
e: serde_json::Value,
worker_name: &str,
flow_is_done: bool,
duration: Option<i64>,
) -> Result<(WrappedError, CompletedSpan), Error> {
record_failure_metrics(completed_job, worker_name).await;
let result = WrappedError { error: e };
@@ -1266,7 +1302,7 @@ pub async fn add_completed_job_error(
completed_job.workspace_id,
serde_json::to_string(&result).unwrap_or_else(|_| "".to_string())
);
let _ = add_completed_job(
let (_, span) = add_completed_job(
db,
&completed_job,
false,
@@ -1281,7 +1317,7 @@ pub async fn add_completed_job_error(
)
.warn_after_seconds(10)
.await?;
Ok(result)
Ok((result, span))
}
lazy_static::lazy_static! {
@@ -1320,7 +1356,7 @@ pub async fn add_completed_job<T: Serialize + Send + Sync + ValidableJson>(
flow_is_done: bool,
duration: Option<i64>,
from_cache: bool,
) -> Result<(Uuid, i64), Error> {
) -> Result<(Uuid, CompletedSpan), Error> {
// tracing::error!("Start");
// let start = tokio::time::Instant::now();
@@ -1354,7 +1390,7 @@ pub async fn add_completed_job<T: Serialize + Send + Sync + ValidableJson>(
};
let result_columns = result_columns.as_ref();
let (opt_uuid, duration, _skip_downstream_error_handlers, wac_parent_ready) = (|| {
let (opt_uuid, span, _skip_downstream_error_handlers, wac_parent_ready) = (|| {
commit_completed_job(
db,
completed_job,
@@ -1395,7 +1431,7 @@ pub async fn add_completed_job<T: Serialize + Send + Sync + ValidableJson>(
// if scheduling next job failed, return the job_id early to ensure the job get retried after a timeout
if let Some(job_id) = opt_uuid {
return Ok((job_id, duration));
return Ok((job_id, span));
}
// Auto-resolve a retry chain that ultimately worked, from whichever of the two
@@ -1436,7 +1472,7 @@ pub async fn add_completed_job<T: Serialize + Send + Sync + ValidableJson>(
}
#[cfg(feature = "cloud")]
apply_completed_job_cloud_usage(db, completed_job, duration);
apply_completed_job_cloud_usage(db, completed_job, span.last_run_ms);
#[cfg(all(feature = "enterprise", feature = "private"))]
crate::jobs_ee::apply_completed_job_error_handlers(
@@ -1453,7 +1489,7 @@ pub async fn add_completed_job<T: Serialize + Send + Sync + ValidableJson>(
// tracing::error!("4 {:?}", start.elapsed());
Ok((completed_job.id, duration))
Ok((completed_job.id, span))
}
async fn commit_completed_job<T: Serialize + Send + Sync + ValidableJson>(
@@ -1471,7 +1507,7 @@ async fn commit_completed_job<T: Serialize + Send + Sync + ValidableJson>(
// True when a native script retry was enqueued for this failed attempt, i.e.
// this is not the terminal attempt — schedule completion handlers must wait.
retry_pending: bool,
) -> windmill_common::error::Result<(Option<Uuid>, i64, bool, bool)> {
) -> windmill_common::error::Result<(Option<Uuid>, CompletedSpan, bool, bool)> {
// let start = std::time::Instant::now();
let job_id = completed_job.id;
@@ -1538,11 +1574,11 @@ async fn commit_completed_job<T: Serialize + Send + Sync + ValidableJson>(
&& monitor_parent.is_none()
&& !is_scheduled
{
let Some(duration) = completion.execute(&mut *db.acquire().await?).await? else {
let Some(completed) = completion.execute(&mut *db.acquire().await?).await? else {
return Err(not_in_queue_error(db, job_id).await);
};
log_completed_job(completed_job, duration, success);
return Ok((None, duration, false, false));
log_completed_job(completed_job, completed.duration_ms, success);
return Ok((None, completed, false, false));
}
let mut tx = db.begin().warn_after_seconds(10).await?;
@@ -1550,7 +1586,8 @@ async fn commit_completed_job<T: Serialize + Send + Sync + ValidableJson>(
// The parent's rows are locked ahead of the child's own queue row (see
// `record_child_completion` for the order this must keep), so the duration it stamps is read
// before the completion: the one the completed row will hold. `now()` is fixed for the
// transaction, and a completed row already there keeps its own duration.
// transaction, and a completed row already there keeps its own duration. For a child that
// itself parked it is the last run's, matching the last start its timeline entry carries.
let mut wac_parent_ready = false;
if let Some(parent_job) = wac_parent {
let Some(duration) = sqlx::query_scalar!(
@@ -1577,7 +1614,7 @@ async fn commit_completed_job<T: Serialize + Send + Sync + ValidableJson>(
.await?;
}
let Some(duration) = completion.execute(&mut *tx).await? else {
let Some(completed) = completion.execute(&mut *tx).await? else {
return Err(not_in_queue_error(&mut *tx, job_id).await);
};
@@ -1787,16 +1824,35 @@ async fn commit_completed_job<T: Serialize + Send + Sync + ValidableJson>(
tx.commit().warn_after_seconds(10).await?;
log_completed_job(completed_job, duration, success);
log_completed_job(completed_job, completed.duration_ms, success);
// tracing::info!("completed job: {:?}", start.elapsed().as_micros());
Ok((
None,
duration,
completed,
_skip_downstream_error_handlers,
wac_parent_ready,
))
}
/// What a job's completion wrote. `started_at` and `duration_ms` are the completed row's.
/// `last_run_ms` is the time the job's last run held a worker, which is what gets metered;
/// it differs from `duration_ms` only for a job that parked.
#[derive(Debug, Clone, Copy)]
pub struct CompletedSpan {
pub started_at: Option<chrono::DateTime<chrono::Utc>>,
pub duration_ms: i64,
pub last_run_ms: i64,
}
impl CompletedSpan {
/// Whether the row spans more than the run that started at `last_started_at`, i.e. the
/// job parked and resumed. A flow records such a step with the row's span, so the step
/// reads like the job does.
pub fn outlasts_run(&self, last_started_at: Option<chrono::DateTime<chrono::Utc>>) -> bool {
matches!((self.started_at, last_started_at), (Some(first), Some(last)) if first < last)
}
}
/// What a job's completion writes, for `Completion::execute`.
struct Completion<'a> {
completed_job: &'a MiniCompletedJob,
@@ -1813,6 +1869,11 @@ impl Completion<'_> {
/// Moves the job from the queue to the completed jobs and refreshes a flow step's parent
/// ping, as one statement. Returns `None` when the job was no longer in the queue.
///
/// For a job that parked (a Workflow-as-Code parent, see `suspend_wac_parent`) the row's
/// `started_at` and `duration_ms` span the whole workflow from its first start, and the
/// last run goes to `extras.wac_last_segment_ms` for readers that sum worker time from
/// completed rows.
///
/// The completion takes its cancellation from the queue row it deletes, not only from
/// `canceled_by`: that is what the worker last read, and the delete waits for a cancel still
/// being written, so the deleted row is the final word on whether the job was canceled.
@@ -1820,7 +1881,7 @@ impl Completion<'_> {
/// It locks the queue row before the completed row's key. Any other writer completing a job
/// (the monitor's zombie fallback, debounce) must take them in the same order, or the two
/// deadlock.
async fn execute(&self, conn: &mut sqlx::PgConnection) -> error::Result<Option<i64>> {
async fn execute(&self, conn: &mut sqlx::PgConnection) -> error::Result<Option<CompletedSpan>> {
let Completion {
completed_job,
success,
@@ -1851,16 +1912,20 @@ impl Completion<'_> {
.then_some(completed_job.parent_job)
.flatten()
else {
return sqlx::query_scalar!(
return sqlx::query_as!(
CompletedSpan,
"WITH deleted AS (
DELETE FROM v2_job_queue WHERE id = $1
RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason
RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason,
(extras->>'wac_first_started_at')::timestamptz AS first_started_at,
COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(started_at, now()))))*1000)::bigint AS run_ms
), completed AS (
INSERT INTO v2_job_completed AS cj
( workspace_id
, id
, started_at
, duration_ms
, extras
, result
, result_columns
, canceled_by
@@ -1871,8 +1936,11 @@ impl Completion<'_> {
, status
, worker
)
SELECT d.workspace_id, d.id, d.started_at,
COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(d.started_at, now()))))*1000),
SELECT d.workspace_id, d.id, COALESCE(d.first_started_at, d.started_at),
CASE WHEN d.first_started_at IS NULL THEN d.run_ms
ELSE EXTRACT('epoch' FROM (now() - d.first_started_at))*1000 END,
CASE WHEN d.first_started_at IS NOT NULL
THEN jsonb_build_object('wac_last_segment_ms', d.run_ms) END,
$3::text::jsonb, $10,
CASE WHEN $4::BOOL THEN $5 ELSE d.canceled_by END,
CASE WHEN $4::BOOL THEN $6 WHEN d.canceled_by IS NOT NULL THEN d.canceled_reason END,
@@ -1888,9 +1956,11 @@ impl Completion<'_> {
THEN EXCLUDED.canceled_by ELSE cj.canceled_by END,
canceled_reason = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL
THEN EXCLUDED.canceled_reason ELSE cj.canceled_reason END
RETURNING duration_ms
RETURNING cj.started_at, cj.duration_ms,
COALESCE((cj.extras->>'wac_last_segment_ms')::bigint, cj.duration_ms) AS last_run_ms
)
SELECT duration_ms AS \"duration_ms!\" FROM completed",
SELECT started_at AS \"started_at?\", duration_ms AS \"duration_ms!\",
last_run_ms AS \"last_run_ms!\" FROM completed",
/* $1 */ completed_job.id,
/* $2 */ success,
/* $3 */ result,
@@ -1910,16 +1980,20 @@ impl Completion<'_> {
// A canceled flow is pinged too: it is completed by its next transition like any other
// flow, and the zombie flow monitor needs the ping to finish the cancel if that
// transition is lost.
sqlx::query_scalar!(
sqlx::query_as!(
CompletedSpan,
"WITH deleted AS (
DELETE FROM v2_job_queue WHERE id = $1
RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason
RETURNING id, workspace_id, started_at, worker, canceled_by, canceled_reason,
(extras->>'wac_first_started_at')::timestamptz AS first_started_at,
COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(started_at, now()))))*1000)::bigint AS run_ms
), completed AS (
INSERT INTO v2_job_completed AS cj
( workspace_id
, id
, started_at
, duration_ms
, extras
, result
, result_columns
, canceled_by
@@ -1930,8 +2004,11 @@ impl Completion<'_> {
, status
, worker
)
SELECT d.workspace_id, d.id, d.started_at,
COALESCE($9::bigint, (EXTRACT('epoch' FROM (now())) - EXTRACT('epoch' FROM (COALESCE(d.started_at, now()))))*1000),
SELECT d.workspace_id, d.id, COALESCE(d.first_started_at, d.started_at),
CASE WHEN d.first_started_at IS NULL THEN d.run_ms
ELSE EXTRACT('epoch' FROM (now() - d.first_started_at))*1000 END,
CASE WHEN d.first_started_at IS NOT NULL
THEN jsonb_build_object('wac_last_segment_ms', d.run_ms) END,
$3::text::jsonb, $10,
CASE WHEN $4::BOOL THEN $5 ELSE d.canceled_by END,
CASE WHEN $4::BOOL THEN $6 WHEN d.canceled_by IS NOT NULL THEN d.canceled_reason END,
@@ -1947,14 +2024,16 @@ impl Completion<'_> {
THEN EXCLUDED.canceled_by ELSE cj.canceled_by END,
canceled_reason = CASE WHEN NOT $4::BOOL AND EXCLUDED.canceled_by IS NOT NULL
THEN EXCLUDED.canceled_reason ELSE cj.canceled_reason END
RETURNING duration_ms
RETURNING cj.started_at, cj.duration_ms,
COALESCE((cj.extras->>'wac_last_segment_ms')::bigint, cj.duration_ms) AS last_run_ms
), parent_ping AS (
UPDATE v2_job_runtime r SET ping = now()
FROM v2_job_queue q
WHERE r.id = $11 AND q.id = r.id AND q.workspace_id = $12
AND EXISTS (SELECT 1 FROM completed)
)
SELECT duration_ms AS \"duration_ms!\" FROM completed",
SELECT started_at AS \"started_at?\", duration_ms AS \"duration_ms!\",
last_run_ms AS \"last_run_ms!\" FROM completed",
/* $1 */ completed_job.id,
/* $2 */ success,
/* $3 */ result,
@@ -2021,7 +2100,7 @@ async fn check_result_size<T: ValidableJson>(
db: &Pool<Postgres>,
queued_job: &MiniCompletedJob,
result: Json<&T>,
) -> Option<Result<(Option<Uuid>, i64, bool, bool), Error>> {
) -> Option<Result<(Option<Uuid>, CompletedSpan, bool, bool), Error>> {
let result_size = result.size() / 1024 / 1024;
if result_size > 2 {
if result_size > *MAX_RESULT_SIZE_MB {
@@ -4252,6 +4331,7 @@ impl PulledJobResult {
))),
result_columns: None,
mem_peak: 0,
resource_usage: None,
cached_res_path: None,
token: "".to_string(),
canceled_by: None,
@@ -4273,6 +4353,7 @@ impl PulledJobResult {
}))),
result_columns: None,
mem_peak: 0,
resource_usage: None,
cached_res_path: None,
token: "".to_string(),
canceled_by: None,
backend/windmill-worker-volumes/src/lib.rs
+34
View File
@@ -54,11 +54,45 @@ pub struct VolumeMount {
pub target: String,
}
/// A file's size and last change time. Sync-back trusts a pulled file's stored hash only while
/// its stamp is the one taken at hand-off: size alone misses a rewrite of the same length. A
/// write within the filesystem's timestamp granularity of the hand-off is not seen.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct FileStamp {
size: u64,
changed: Option<(i64, i64)>,
}
impl FileStamp {
/// Whether the file is as it was at `earlier`. An unreadable change time never matches.
pub fn unchanged_since(&self, earlier: &FileStamp) -> bool {
self.changed.is_some() && self == earlier
}
pub fn of(meta: &std::fs::Metadata) -> Self {
// ctime rather than mtime: a tool that restores mtime (cp -p, tar, rsync -t) cannot set it.
#[cfg(unix)]
let changed = {
use std::os::unix::fs::MetadataExt;
Some((meta.ctime(), meta.ctime_nsec()))
};
#[cfg(not(unix))]
let changed = meta
.modified()
.ok()
.and_then(|t| t.duration_since(std::time::UNIX_EPOCH).ok())
.map(|d| (d.as_secs() as i64, d.subsec_nanos() as i64));
Self { size: meta.len(), changed }
}
}
pub struct VolumeState {
pub mount: VolumeMount,
pub local_dir: PathBuf,
pub manifest: HashMap<String, FileEntry>,
pub symlinks: HashMap<String, String>,
/// Stamp of each pulled file as the job received it, keyed like `manifest`.
pub stamps: HashMap<String, FileStamp>,
}
pub struct DownloadStats {
backend/windmill-worker/src/bun_executor.rs
+8 -4
View File
@@ -3273,9 +3273,9 @@ pub async fn handle_wac_v2_output(
.or(if own_runnable { None } else { job.priority }),
None, // authed
false, // running
None, // end_user_email
None, // trigger
None, // suspended_mode
job.permissioned_as_end_user_email.clone(),
None, // trigger
None, // suspended_mode
job.job_token_scopes.as_deref(),
)
.await?;
@@ -3731,7 +3731,10 @@ pub async fn handle_wac_v2_output(
// status row, the order every child completion takes.
let segment_ms = sqlx::query_scalar!(
"WITH prev AS (SELECT started_at FROM v2_job_queue WHERE id = $1)
UPDATE v2_job_queue q SET running = false, started_at = null
UPDATE v2_job_queue q SET running = false, started_at = null,
extras = CASE WHEN q.started_at IS NULL OR q.extras ? 'wac_first_started_at' THEN q.extras
ELSE jsonb_set(COALESCE(q.extras, '{}'::jsonb), '{wac_first_started_at}', to_jsonb(q.started_at))
END
FROM prev WHERE q.id = $1
RETURNING (extract(epoch FROM now() - prev.started_at) * 1000)::bigint",
job.id,
@@ -4083,6 +4086,7 @@ async fn handle_dedicated_bunnative(
result,
result_columns: None,
mem_peak: 0,
resource_usage: None,
canceled_by: None,
success,
cached_res_path: None,
backend/windmill-worker/src/common.rs
+1
View File
@@ -643,6 +643,7 @@ pub async fn update_worker_ping_for_failed_init_script(
wm_memory_usage: None,
job_isolation: None,
native_mode: None,
draining: None,
ping_type: PingType::InitScript,
},
)
backend/windmill-worker/src/handle_child.rs
+75 -1
View File
@@ -59,6 +59,56 @@ use windmill_common::tracing_init::{OTEL_JOB_LOGS, OTEL_PREFIX, QUIET_MODE, VERB
lazy_static::lazy_static! {
pub static ref SLOW_LOGS: bool = std::env::var("SLOW_LOGS").ok().is_some_and(|x| x == "1" || x == "true");
// A job runs several children in turn (install, build, run): their CPU time adds up
// here until the job completes.
static ref JOB_CPU_TIME_MS: std::sync::Mutex<std::collections::HashMap<Uuid, i64>> = Default::default();
}
/// CPU time of the children run so far for the job, which stops being tracked.
pub(crate) fn take_job_cpu_time_ms(job_id: &Uuid) -> Option<i64> {
JOB_CPU_TIME_MS.lock().unwrap().remove(job_id)
}
/// CPU time in ms from the content of `/proc/<pid>/stat`: the process's own user and
/// system time plus that of the children it waited for, which is where the time of a
/// job run under a wrapper (nsjail, unshare) ends up.
#[cfg_attr(not(target_os = "linux"), allow(dead_code))]
fn parse_proc_stat_cpu_ms(stat: &str) -> Option<i64> {
// The values are in USER_HZ ticks, which Linux fixes at 100 per second.
const MS_PER_TICK: i64 = 10;
// `comm` is free text in parentheses: the numbered fields resume after its last `)`,
// with the state as field 3 and utime, stime, cutime, cstime as fields 14 to 17.
let fields: Vec<&str> = stat.rsplit_once(')')?.1.split_ascii_whitespace().collect();
let ticks = fields
.get(11..15)?
.iter()
.map(|f| f.parse::<i64>().ok())
.sum::<Option<i64>>()?;
Some(ticks * MS_PER_TICK)
}
/// Resolves once the child has exited, to its CPU time. The child is left un-reaped:
/// `/proc/<pid>/stat` is only readable until then, so the caller reaps afterwards.
#[cfg(target_os = "linux")]
async fn wait_for_exit_cpu_time_ms(pid: u32) -> Option<i64> {
use nix::sys::wait::{waitid, Id, WaitPidFlag};
let (tx, rx) = tokio::sync::oneshot::channel();
// A plain thread rather than `spawn_blocking`: the wait cannot be cancelled, and a
// blocking task still parked on a live child would hold up the runtime's shutdown.
std::thread::Builder::new()
.name("job-cpu-time".to_string())
.spawn(move || {
let cpu = waitid(
Id::Pid(Pid::from_raw(pid as i32)),
WaitPidFlag::WEXITED | WaitPidFlag::WNOWAIT,
)
.ok()
.and_then(|_| std::fs::read_to_string(format!("/proc/{pid}/stat")).ok())
.and_then(|stat| parse_proc_stat_cpu_ms(&stat));
let _ = tx.send(cpu);
})
.ok()?;
rx.await.ok().flatten()
}
// - kill windows process along with all child processes
@@ -267,7 +317,15 @@ pub async fn handle_child(
let wait_on_child = async {
let kill_reason = tokio::select! {
biased;
result = Box::into_pin(child.wait()) => return result.map(Ok),
result = async {
#[cfg(target_os = "linux")]
if let Some(pid) = pid.filter(|_| job_id != Uuid::nil()) {
if let Some(cpu) = wait_for_exit_cpu_time_ms(pid).await {
*JOB_CPU_TIME_MS.lock().unwrap().entry(job_id).or_insert(0) += cpu;
}
}
Box::into_pin(child.wait()).await
} => return result.map(Ok),
Ok(()) = too_many_logs.changed() => KillReason::TooManyLogs,
_ = sleep(timeout_duration) => KillReason::Timeout {
phase: child_name.to_string(),
@@ -1049,6 +1107,7 @@ where
let wm_memory_usage = get_windmill_memory_usage();
tracing::info!("job {job_id} on {worker_name} in {w_id} worker memory snapshot {}kB/{}kB", memory_usage.unwrap_or_default()/1024, wm_memory_usage.unwrap_or_default()/1024);
let occupancy = occupancy_metrics.as_mut().map(|x| x.update_occupancy_metrics());
crate::resource_metrics::record_worker_resources(worker_name, memory_usage, wm_memory_usage, occupancy.as_ref());
if job_id != Uuid::nil() {
if let Err(err) = update_worker_ping_from_job(&conn, &job_id, w_id, worker_name, memory_usage, wm_memory_usage, occupancy).await {
tracing::error!("Unable to update worker ping for job {} in workspace {}. Error was: {:?}", job_id, w_id, err);
@@ -1213,6 +1272,21 @@ pub fn process_status(
mod tests {
use super::*;
#[test]
fn proc_stat_cpu_time_survives_a_hostile_process_name() {
// utime 150, stime 50, cutime 700, cstime 100 ticks: 1000 ticks, 10 s.
let tail = "Z 1 42 42 0 -1 4194560 100 200 0 0 150 50 700 100 20 0 1 0 12345 0 0";
assert_eq!(
parse_proc_stat_cpu_ms(&format!("42 (bun) {tail}")),
Some(10_000)
);
assert_eq!(
parse_proc_stat_cpu_ms(&format!("42 (a) R 9 (9 9 9 9 9 9 9 9 9) {tail}")),
Some(10_000)
);
assert_eq!(parse_proc_stat_cpu_ms("42 (bun) Z 1 42"), None);
}
// A non-positive timeout means "unset" to `resolve_job_timeout`, so an
// exhausted budget that reported 0 would hand the next phase no deadline at
// all — the opposite of what running out of time should do.
backend/windmill-worker/src/lib.rs
+1
View File
@@ -81,6 +81,7 @@ mod prepare_deps;
mod python_executor;
#[cfg(feature = "python")]
mod python_versions;
mod resource_metrics;
pub mod result_processor;
#[cfg(feature = "rust")]
mod rust_executor;
backend/windmill-worker/src/resource_metrics.rs
+235
View File
@@ -0,0 +1,235 @@
use std::sync::{atomic::Ordering, Mutex, OnceLock};
use windmill_common::{
otel_oss::{
otel_add_worker_cpu, otel_record_job_memory_peak, otel_set_worker_memory,
otel_set_worker_occupancy_rate,
},
worker::{get_cpu_stat, get_memory, CpuStat, WORKER_GROUP},
OTEL_METRICS_ENABLED,
};
use crate::common::OccupancyResult;
#[cfg(feature = "prometheus")]
struct PromMetrics {
memory_usage: prometheus::IntGaugeVec,
windmill_memory_usage: prometheus::IntGaugeVec,
memory_limit: prometheus::IntGaugeVec,
cpu_usage: prometheus::CounterVec,
cpu_periods: prometheus::IntCounterVec,
cpu_throttled_periods: prometheus::IntCounterVec,
cpu_throttled_seconds: prometheus::CounterVec,
occupancy_rate: prometheus::GaugeVec,
job_memory_peak: prometheus::HistogramVec,
}
#[cfg(feature = "prometheus")]
lazy_static::lazy_static! {
static ref PROM_METRICS: Option<PromMetrics> =
if windmill_common::METRICS_ENABLED.load(Ordering::Relaxed) {
const WORKER: &[&str] = &["name", "worker_group"];
Some(PromMetrics {
memory_usage: prometheus::register_int_gauge_vec!(
"worker_memory_usage_bytes",
"Memory used by the worker's container, page cache excluded",
WORKER
)
.unwrap(),
windmill_memory_usage: prometheus::register_int_gauge_vec!(
"worker_windmill_memory_usage_bytes",
"Resident memory of the worker process itself",
WORKER
)
.unwrap(),
memory_limit: prometheus::register_int_gauge_vec!(
"worker_memory_limit_bytes",
"Memory limit of the worker's container",
WORKER
)
.unwrap(),
cpu_usage: prometheus::register_counter_vec!(
"worker_cpu_usage_seconds_total",
"CPU time consumed by the worker's container",
WORKER
)
.unwrap(),
cpu_periods: prometheus::register_int_counter_vec!(
"worker_cpu_periods_total",
"CPU enforcement periods elapsed for the worker's container",
WORKER
)
.unwrap(),
cpu_throttled_periods: prometheus::register_int_counter_vec!(
"worker_cpu_throttled_periods_total",
"CPU enforcement periods in which the worker's container was throttled",
WORKER
)
.unwrap(),
cpu_throttled_seconds: prometheus::register_counter_vec!(
"worker_cpu_throttled_seconds_total",
"Time the worker's container spent throttled by its CPU limit",
WORKER
)
.unwrap(),
occupancy_rate: prometheus::register_gauge_vec!(
"worker_occupancy_rate",
"Share of time the worker spent executing jobs, over the window",
&["name", "worker_group", "window"]
)
.unwrap(),
job_memory_peak: prometheus::register_histogram_vec!(
"worker_job_memory_peak_bytes",
"Peak memory of a job's main process",
&["tag"],
windmill_common::worker::JOB_MEMORY_PEAK_BUCKETS.to_vec()
)
.unwrap(),
})
} else {
None
};
}
fn metrics_enabled() -> bool {
#[cfg(feature = "prometheus")]
if PROM_METRICS.is_some() {
return true;
}
OTEL_METRICS_ENABLED.load(Ordering::Relaxed)
}
// Memory and CPU are read from the cgroup, which every worker of a process shares
// (native mode runs several). Only the first worker to report exports them, so that
// summing the series over workers does not count one container several times.
static CONTAINER_SERIES_OWNER: OnceLock<String> = OnceLock::new();
static MEMORY_LIMIT: Mutex<Option<Option<i64>>> = Mutex::new(None);
/// Replaces the exported memory limit, for the ping that re-reads the cgroup limits.
pub(crate) fn set_memory_limit(limit: Option<i64>) {
*MEMORY_LIMIT.lock().unwrap() = Some(limit);
}
static LAST_CPU_STAT: Mutex<CpuStat> =
Mutex::new(CpuStat { usage_usec: 0, nr_periods: 0, nr_throttled: 0, throttled_usec: 0 });
/// Exports the worker's resource readings. Meant for the ping ticks, which have already
/// read memory usage and occupancy; the only extra work is the cgroup `cpu.stat` read.
pub(crate) fn record_worker_resources(
worker_name: &str,
memory_usage: Option<i64>,
wm_memory_usage: Option<i64>,
occupancy: Option<&OccupancyResult>,
) {
if !metrics_enabled() {
return;
}
let worker_group = WORKER_GROUP.as_str();
if let Some(occupancy) = occupancy {
let windows = [
("total", Some(occupancy.occupancy_rate)),
("15s", occupancy.occupancy_rate_15s),
("5m", occupancy.occupancy_rate_5m),
("30m", occupancy.occupancy_rate_30m),
];
for (window, rate) in windows {
let Some(rate) = rate.filter(|r| r.is_finite()) else {
continue;
};
#[cfg(feature = "prometheus")]
if let Some(m) = PROM_METRICS.as_ref() {
m.occupancy_rate
.with_label_values(&[worker_name, worker_group, window])
.set(rate as f64);
}
otel_set_worker_occupancy_rate(worker_name, worker_group, window, rate as f64);
}
}
if CONTAINER_SERIES_OWNER.get_or_init(|| worker_name.to_string()) != worker_name {
return;
}
let memory_limit = *MEMORY_LIMIT.lock().unwrap().get_or_insert_with(get_memory);
#[cfg(feature = "prometheus")]
if let Some(m) = PROM_METRICS.as_ref() {
let labels = &[worker_name, worker_group];
if let Some(v) = memory_usage {
m.memory_usage.with_label_values(labels).set(v);
}
if let Some(v) = wm_memory_usage {
m.windmill_memory_usage.with_label_values(labels).set(v);
}
if let Some(v) = memory_limit {
m.memory_limit.with_label_values(labels).set(v);
}
}
otel_set_worker_memory(
worker_name,
worker_group,
memory_usage,
wm_memory_usage,
memory_limit,
);
let Some(stat) = get_cpu_stat() else {
return;
};
let delta = {
let mut last = LAST_CPU_STAT.lock().unwrap();
// A reading below the previous one means the cgroup changed under the process:
// restart from it rather than exporting a negative increase.
let delta = if stat.usage_usec < last.usage_usec {
CpuStat::default()
} else {
CpuStat {
usage_usec: stat.usage_usec - last.usage_usec,
nr_periods: stat.nr_periods.saturating_sub(last.nr_periods),
nr_throttled: stat.nr_throttled.saturating_sub(last.nr_throttled),
throttled_usec: stat.throttled_usec.saturating_sub(last.throttled_usec),
}
};
*last = stat;
delta
};
let usage_secs = delta.usage_usec as f64 / 1e6;
let throttled_secs = delta.throttled_usec as f64 / 1e6;
#[cfg(feature = "prometheus")]
if let Some(m) = PROM_METRICS.as_ref() {
let labels = &[worker_name, worker_group];
m.cpu_usage.with_label_values(labels).inc_by(usage_secs);
m.cpu_periods
.with_label_values(labels)
.inc_by(delta.nr_periods);
m.cpu_throttled_periods
.with_label_values(labels)
.inc_by(delta.nr_throttled);
m.cpu_throttled_seconds
.with_label_values(labels)
.inc_by(throttled_secs);
}
otel_add_worker_cpu(
worker_name,
worker_group,
usage_secs,
delta.nr_periods,
delta.nr_throttled,
throttled_secs,
);
}
/// `mem_peak_kb` is the job's `mem_peak`: non-positive when the executor had no process
/// to sample or the reading failed, and those are not observations.
pub(crate) fn record_job_memory_peak(tag: &str, mem_peak_kb: i32) {
if mem_peak_kb <= 0 {
return;
}
let bytes = mem_peak_kb as u64 * 1024;
#[cfg(feature = "prometheus")]
if let Some(m) = PROM_METRICS.as_ref() {
m.job_memory_peak
.with_label_values(&[tag])
.observe(bytes as f64);
}
otel_record_job_memory_peak(tag, bytes);
}
backend/windmill-worker/src/result_processor.rs
+74 -15
View File
@@ -21,6 +21,7 @@ use windmill_common::{
error::{self, Error},
flow_status::FlowJobDuration,
jobs::JobKind,
runnable_job_stats::{accumulate_runnable_job_stats, flush_runnable_job_stats},
utils::WarnAfterExt,
worker::{error_to_value, to_raw_value, Connection, WORKER_GROUP},
worker_group_job_stats::{accumulate_job_stats, flush_stats_to_db, JobStatsMap},
@@ -32,8 +33,8 @@ use windmill_common::bench::{BenchmarkInfo, BenchmarkIter};
use windmill_queue::{
append_logs, asset_dispatch, get_mini_completed_job, is_pre_shaped_wm_failure_result,
parse_result_object, CanceledBy, FlowRunners, JobCompleted, MiniCompletedJob, MiniPulledJob,
ValidableJson, WrappedError, INIT_SCRIPT_TAG, MANUAL_FAILURE_ERROR_NAME,
parse_result_object, CanceledBy, FlowRunners, JobCompleted, JobResourceUsage, MiniCompletedJob,
MiniPulledJob, ValidableJson, WrappedError, INIT_SCRIPT_TAG, MANUAL_FAILURE_ERROR_NAME,
};
use serde_json::{json, value::RawValue, Value};
@@ -231,6 +232,10 @@ async fn process_jc(
let duration_ms = jc.duration.clone();
let script_lang = jc.job.script_lang.clone();
let workspace_id = jc.job.workspace_id.clone();
let job_kind = jc.job.kind;
let runnable_path = jc.job.runnable_path.clone();
let mem_peak = jc.mem_peak;
let resource_usage = jc.resource_usage.take();
let root_job = handle_receive_completed_job(
jc,
@@ -269,6 +274,17 @@ async fn process_jc(
duration_ms,
)
.await;
accumulate_runnable_job_stats(
&workspace_id,
job_kind,
runnable_path.as_deref(),
resource_usage
.as_ref()
.map_or(WORKER_GROUP.as_str(), |u| u.worker_group.as_str()),
duration_ms,
mem_peak,
resource_usage.as_ref().and_then(|u| u.cpu_time_ms),
);
}
success
@@ -317,6 +333,14 @@ pub fn start_background_processor(
let flush_handle = tokio::spawn(async move {
let mut interval = tokio::time::interval(tokio::time::Duration::from_secs(900)); // Flush every 15 min
// Once per hourly bucket, at an offset of its own per worker so that a fleet
// started together does not write the same rows in the same second.
let runnable_period = tokio::time::Duration::from_secs(3600);
let mut runnable_interval = tokio::time::interval_at(
tokio::time::Instant::now() + runnable_period.mul_f64(rand::random::<f64>()),
runnable_period,
);
loop {
tokio::select! {
_ = interval.tick() => {
@@ -324,6 +348,11 @@ pub fn start_background_processor(
tracing::error!("Failed to flush worker group job stats: {}", e);
}
}
_ = runnable_interval.tick() => {
if let Err(e) = flush_runnable_job_stats(&db_clone).await {
tracing::error!("Failed to flush runnable job stats: {}", e);
}
}
_ = killpill_rx_clone.recv() => {
tracing::info!("bg processor received killpill signal, flushing remaining stats");
break;
@@ -524,6 +553,18 @@ pub fn start_background_processor(
Ok(Err(join_err)) => tracing::error!("Stats flush task failed: {}", join_err),
Err(_) => tracing::error!("Stats flush timed out after 10 seconds"),
}
// The periodic task stops at the killpill, before the jobs drained above were
// accumulated: without this a short-lived worker would lose its whole rollup.
match tokio::time::timeout(
std::time::Duration::from_secs(10),
flush_runnable_job_stats(&db),
)
.await
{
Ok(Ok(())) => {}
Ok(Err(e)) => tracing::error!("Failed to flush runnable job stats: {}", e),
Err(_) => tracing::error!("Runnable job stats flush timed out after 10 seconds"),
}
job_completed_processor_is_done.store(true, Ordering::SeqCst);
@@ -555,6 +596,7 @@ pub async fn process_result(
job_dir: &str,
job_completed_tx: JobCompletedSender,
mem_peak: i32,
cpu_time_ms: Option<i64>,
canceled_by: Option<CanceledBy>,
cached_res_path: Option<String>,
token: &str,
@@ -565,6 +607,7 @@ pub async fn process_result(
has_stream: bool,
flow_runners: Option<Arc<FlowRunners>>,
) -> error::Result<crate::worker::JobOutcome> {
let resource_usage = Some(JobResourceUsage { worker_group: WORKER_GROUP.clone(), cpu_time_ms });
match result {
Ok(result) => {
send_job_completed(
@@ -575,6 +618,7 @@ pub async fn process_result(
result,
result_columns,
mem_peak,
resource_usage,
canceled_by,
success: true,
cached_res_path,
@@ -658,6 +702,7 @@ pub async fn process_result(
result_columns: None,
preprocessed_args: None,
mem_peak,
resource_usage,
canceled_by,
success: false,
cached_res_path,
@@ -1776,7 +1821,7 @@ pub async fn process_completed_job(
add_time!(bench, "pre add_completed_job");
let (_, duration) = add_completed_job(
let (_, span) = add_completed_job(
db,
&job,
true,
@@ -1835,7 +1880,10 @@ pub async fn process_completed_job(
true,
canceled_by,
result,
started_at.map(|x| FlowJobDuration { started_at: x, duration_ms: duration }),
span.started_at.or(started_at).map(|x| FlowJobDuration {
started_at: x,
duration_ms: span.duration_ms,
}),
StepFailureKind::Normal,
&same_worker_tx.expect(SAME_WORKER_REQUIREMENTS).to_owned(),
&worker_dir,
@@ -1867,9 +1915,10 @@ pub async fn process_completed_job(
// `wm_failure` field) so a real runtime failure whose raw
// result happens to contain a `wm_failure` field still goes
// through the standard `WrappedError { error: ... }` wrap path.
let downstream_result: Arc<Box<RawValue>> = if is_pre_shaped_wm_failure_result(result.get())
{
windmill_queue::add_completed_job_pre_shaped_failure(
let (downstream_result, span): (Arc<Box<RawValue>>, _) = if is_pre_shaped_wm_failure_result(
result.get(),
) {
let span = windmill_queue::add_completed_job_pre_shaped_failure(
db,
&job,
mem_peak.to_owned(),
@@ -1880,9 +1929,9 @@ pub async fn process_completed_job(
None,
)
.await?;
result.clone()
(result.clone(), span)
} else {
let wrapped = add_completed_job_error(
let (wrapped, span) = windmill_queue::add_completed_job_error_with_span(
db,
&job,
mem_peak.to_owned(),
@@ -1895,7 +1944,10 @@ pub async fn process_completed_job(
None,
)
.await?;
Arc::new(serde_json::value::to_raw_value(&wrapped).unwrap())
(
Arc::new(serde_json::value::to_raw_value(&wrapped).unwrap()),
span,
)
};
#[cfg(all(feature = "enterprise", feature = "private"))]
if job.kind == JobKind::DeploymentCallback {
@@ -1927,12 +1979,19 @@ pub async fn process_completed_job(
false,
canceled_by,
downstream_result,
duration.and_then(|d| {
job.started_at.map(|started_at| FlowJobDuration {
started_at: started_at,
duration_ms: d,
if span.outlasts_run(job.started_at) {
span.started_at.map(|started_at| FlowJobDuration {
started_at,
duration_ms: span.duration_ms,
})
}),
} else {
duration.and_then(|d| {
job.started_at.map(|started_at| FlowJobDuration {
started_at: started_at,
duration_ms: d,
})
})
},
StepFailureKind::Normal,
&same_worker_tx.expect(SAME_WORKER_REQUIREMENTS).to_owned(),
&worker_dir,
backend/windmill-worker/src/wac_executor.rs
+9 -1
View File
@@ -113,6 +113,11 @@ pub enum WacPark {
/// handler — falls back to `now() - started_at`. Left pointing at the first segment,
/// that fallback reports the whole sleep or approval wait as execution time.
///
/// The first park keeps that first start in `extras.wac_first_started_at`, so the
/// completed row can still say when the workflow began and how long it took overall
/// (`Completion::execute`). The queue row's `started_at` stays per segment. The inline
/// checkpoint requeue clears `started_at` on its own and keeps the first start the same way.
///
/// Call it before any write to the parent's `v2_job_status` row in the same
/// transaction: a child's completion locks the queue row and then the status row
/// (`record_child_completion`), and taking them the other way round here can
@@ -156,7 +161,10 @@ pub async fn suspend_wac_parent(
sqlx::query!(
"UPDATE v2_job_queue
SET suspend = $3, suspend_until = now() + make_interval(secs => $4), started_at = null
SET suspend = $3, suspend_until = now() + make_interval(secs => $4), started_at = null,
extras = CASE WHEN started_at IS NULL OR extras ? 'wac_first_started_at' THEN extras
ELSE jsonb_set(COALESCE(extras, '{}'::jsonb), '{wac_first_started_at}', to_jsonb(started_at))
END
WHERE id = $1 AND workspace_id = $2",
job_id,
w_id,
backend/windmill-worker/src/worker.rs
+46 -3
View File
@@ -155,7 +155,7 @@ use crate::{
handle_app_dependency_job, handle_dependency_job, handle_flow_dependency_job,
tally_unfinished_dependency_deploy,
},
worker_utils::{insert_ping, queue_vacuum, update_worker_ping_full},
worker_utils::{insert_ping, mark_worker_draining, queue_vacuum, update_worker_ping_full},
};
#[cfg(feature = "rust")]
@@ -2247,6 +2247,7 @@ pub async fn handle_all_job_kind_error(
))),
result_columns: None,
mem_peak: 0,
resource_usage: None,
canceled_by: None,
success: false,
cached_res_path: None,
@@ -2713,7 +2714,7 @@ pub async fn run_worker(
let mut last_ping = Instant::now() - Duration::from_secs(NUM_SECS_PING + 1);
let mut reported_ip = cached_ip();
let previous_jobs_executed = insert_ping(hostname, &worker_name, reported_ip, conn)
let previous_jobs_executed = insert_ping(hostname, &worker_name, reported_ip, true, conn)
.await
.expect("initial ping could be sent");
@@ -3160,6 +3161,31 @@ pub async fn run_worker(
let mut killpill_rx2 = killpill_rx.resubscribe();
// The loop below only sees the killpill once the running job is done, which is too late
// for that job to learn its worker is going away.
{
let mut killpill_rx = killpill_rx.resubscribe();
let conn = conn.clone();
let worker_name = worker_name.clone();
tokio::spawn(async move {
if matches!(
killpill_rx.recv().await,
Err(broadcast::error::RecvError::Closed)
) {
return;
}
for attempt in 1..=3 {
match mark_worker_draining(&conn, &worker_name).await {
Ok(()) => return,
Err(e) => {
tracing::warn!(worker = %worker_name, "failed to mark worker as draining (attempt {attempt}/3): {e:#}");
tokio::time::sleep(Duration::from_secs(2)).await;
}
}
}
});
}
loop {
let last_processing_duration_secs = last_processing_duration.load(Ordering::SeqCst);
if last_processing_duration_secs > 5 {
@@ -3235,7 +3261,7 @@ pub async fn run_worker(
// which costs at most the last job's id here: no job of this worker is in flight at this
// point in the loop, and the next one refills them.
if ip_just_resolved && conn.as_sql().is_none() {
if let Err(e) = insert_ping(hostname, &worker_name, ip, &conn).await {
if let Err(e) = insert_ping(hostname, &worker_name, ip, false, &conn).await {
tracing::warn!(
worker = %worker_name, hostname = %hostname,
"failed to re-register with the resolved external IP: {e}"
@@ -3747,6 +3773,7 @@ pub async fn run_worker(
result: Arc::new(empty_result()),
result_columns: None,
mem_peak: 0,
resource_usage: None,
cached_res_path: None,
token: "".to_string(),
canceled_by: None,
@@ -4738,6 +4765,7 @@ pub async fn handle_queued_job(
result,
result_columns: None,
mem_peak: 0,
resource_usage: None,
canceled_by: None,
success: true,
cached_res_path: None,
@@ -5087,6 +5115,8 @@ pub async fn handle_queued_job(
let cjob = MiniCompletedJob::from(job.to_owned());
drop(job);
// Taken ahead of the returns below so that none of them leaves the entry behind.
let cpu_time_ms = crate::handle_child::take_job_cpu_time_ms(&cjob.id);
//it's a test job, no need to update the db
if cjob.workspace_id == "" {
return Ok(JobOutcome::Completed);
@@ -5103,14 +5133,27 @@ pub async fn handle_queued_job(
.is_err_and(|err| matches!(err, &Error::WacSuspended(_)))
{
// WAC v2 job suspended while waiting for child jobs — don't complete it
// Only a worker with a database connection flushes the rollup.
if let Connection::Sql(_) = conn {
windmill_common::runnable_job_stats::accumulate_runnable_round(
&cjob.workspace_id,
cjob.kind,
cjob.runnable_path.as_deref(),
&WORKER_GROUP,
started.elapsed().as_millis() as i64,
cpu_time_ms,
);
}
return Ok(JobOutcome::Completed);
}
crate::resource_metrics::record_job_memory_peak(&cjob.tag, mem_peak);
process_result(
cjob,
result.map(|x| Arc::new(x)),
job_dir,
job_completed_tx,
mem_peak,
cpu_time_ms,
canceled_by,
cached_res_path,
&client.token,
backend/windmill-worker/src/worker_flow.rs
+4 -4
View File
@@ -2097,7 +2097,7 @@ pub async fn update_flow_status_after_job_completion_internal(
)
.await?;
let duration = if success {
let (_, duration) = add_completed_job(
let (_, span) = add_completed_job(
db,
&cflow_job,
true,
@@ -2111,9 +2111,9 @@ pub async fn update_flow_status_after_job_completion_internal(
false,
)
.await?;
duration
span.duration_ms
} else {
let (_, duration) = add_completed_job(
let (_, span) = add_completed_job(
db,
&cflow_job,
false,
@@ -2131,7 +2131,7 @@ pub async fn update_flow_status_after_job_completion_internal(
false,
)
.await?;
duration
span.duration_ms
};
flow_job_duration = flow_job
.started_at
backend/windmill-worker/src/worker_utils.rs
+58 -5
View File
@@ -7,9 +7,9 @@ use windmill_common::{
external_ip::UNKNOWN_IP,
worker::{
get_memory, get_vcpus, get_windmill_memory_usage, get_worker_memory_usage,
insert_ping_query, update_job_ping_query, update_worker_ping_from_job_query,
update_worker_ping_main_loop_query, Connection, Ping, PingType, NATIVE_MODE_RESOLVED,
WORKER_CONFIG, WORKER_GROUP,
insert_ping_query, set_worker_draining_query, update_job_ping_query,
update_worker_ping_from_job_query, update_worker_ping_main_loop_query, Connection, Ping,
PingType, NATIVE_MODE_RESOLVED, WORKER_CONFIG, WORKER_GROUP,
},
KillpillSender, DB,
};
@@ -39,17 +39,26 @@ pub(crate) async fn update_worker_ping_full(
let wm_memory_usage = get_windmill_memory_usage();
let (vcpus, memory) = if read_cgroups {
(get_vcpus(), get_memory())
let memory = get_memory();
crate::resource_metrics::set_memory_limit(memory);
(get_vcpus(), memory)
} else {
(None, None)
};
let occupancy = occupancy_metrics.update_occupancy_metrics();
crate::resource_metrics::record_worker_resources(
worker_name,
memory_usage,
wm_memory_usage,
Some(&occupancy),
);
let OccupancyResult {
occupancy_rate,
occupancy_rate_15s,
occupancy_rate_5m,
occupancy_rate_30m,
} = occupancy_metrics.update_occupancy_metrics();
} = occupancy;
let ping_start = std::time::Instant::now();
if let Err(e) = (|| {
@@ -164,6 +173,7 @@ async fn update_worker_ping_full_inner(
wm_memory_usage: get_windmill_memory_usage(),
job_isolation: None,
native_mode: Some(native_mode),
draining: None,
ping_type: PingType::MainLoop,
},
)
@@ -179,6 +189,7 @@ pub async fn insert_ping(
worker_instance: &str,
worker_name: &str,
ip: Option<&str>,
reset_draining: bool,
db: &Connection,
) -> anyhow::Result<i32> {
let (tags, dw, dws, native_mode) = {
@@ -224,6 +235,7 @@ pub async fn insert_ping(
memory,
job_isolation,
native_mode,
reset_draining.then_some(false),
db,
)
.await;
@@ -256,6 +268,7 @@ pub async fn insert_ping(
wm_memory_usage: get_windmill_memory_usage(),
job_isolation,
native_mode: Some(native_mode),
draining: reset_draining.then_some(false),
ping_type: PingType::Initial,
},
)
@@ -314,6 +327,7 @@ pub async fn update_worker_ping_from_job(
&Ping {
last_job_executed: Some(job_id.clone()),
last_job_workspace_id: Some(w_id.to_string()),
draining: None,
ping_type: PingType::Job,
worker_instance: None,
ip: None,
@@ -342,6 +356,45 @@ pub async fn update_worker_ping_from_job(
Ok(())
}
/// Publishes on the worker's ping row that it has received its killpill, so the job it is
/// still running can find out and exit early.
pub async fn mark_worker_draining(conn: &Connection, worker_name: &str) -> anyhow::Result<()> {
match conn {
Connection::Sql(db) => set_worker_draining_query(worker_name, db).await,
Connection::Http(client) => {
client
.post::<Ping, ()>(
UPDATE_PING_URL,
None,
&Ping {
last_job_executed: None,
last_job_workspace_id: None,
draining: None,
ping_type: PingType::Draining,
worker_instance: None,
ip: None,
tags: None,
dw: None,
dws: None,
version: None,
vcpus: None,
memory: None,
memory_usage: None,
wm_memory_usage: None,
jobs_executed: None,
occupancy_rate: None,
occupancy_rate_15s: None,
occupancy_rate_5m: None,
occupancy_rate_30m: None,
job_isolation: None,
native_mode: None,
},
)
.await
}
}
}
pub async fn ping_job_status(
conn: &Connection,
job_id: &Uuid,
benchmarks/lib.ts
+1 -1
View File
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.827.0";
export const VERSION = "v1.829.0";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({
chat-sdk/package-lock.json
Generated
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "windmill-chat",
"version": "1.827.0",
"version": "1.829.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "windmill-chat",
"version": "1.827.0",
"version": "1.829.0",
"license": "Apache-2.0",
"devDependencies": {
"@ai-sdk/react": "^4.0.102",
chat-sdk/package.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "windmill-chat",
"description": "Build chat interfaces on Windmill flows deployed in chat mode, from any frontend or raw app",
"version": "1.827.0",
"version": "1.829.0",
"author": "Ruben Fiszel",
"license": "Apache-2.0",
"homepage": "https://github.com/windmill-labs/windmill/tree/main/chat-sdk#readme",
cli/src/core/constants.ts
+1 -1
View File
@@ -10,4 +10,4 @@ export const WM_FORK_PREFIX = "wm-fork";
// (e.g. utils.ts) can read it without importing main.ts and creating a circular
// dependency (main → workspace → utils → main) that triggers a TDZ.
// Re-exported from main.ts for backwards compatibility.
export const VERSION = "1.827.0";
export const VERSION = "1.829.0";
cli/src/guidance/skills.gen.ts
Generated
+76 -2
View File
@@ -839,6 +839,14 @@ async setProgress(percent: number, jobId?: any): Promise<void>
*/
async getProgress(jobId?: any): Promise<number | null>
/**
* Whether the worker running this job has received its shutdown signal.
* A draining worker waits for its current job to finish before exiting, so a
* long-running script can poll this and return early to let the worker be replaced.
* @param jobId? Job to check, defaults to the current job
*/
async workerIsDraining(jobId?: string): Promise<boolean>
/**
* Set a flow user state
* @param key key of the state
@@ -1674,6 +1682,14 @@ async setProgress(percent: number, jobId?: any): Promise<void>
*/
async getProgress(jobId?: any): Promise<number | null>
/**
* Whether the worker running this job has received its shutdown signal.
* A draining worker waits for its current job to finish before exiting, so a
* long-running script can poll this and return early to let the worker be replaced.
* @param jobId? Job to check, defaults to the current job
*/
async workerIsDraining(jobId?: string): Promise<boolean>
/**
* Set a flow user state
* @param key key of the state
@@ -2615,6 +2631,14 @@ async setProgress(percent: number, jobId?: any): Promise<void>
*/
async getProgress(jobId?: any): Promise<number | null>
/**
* Whether the worker running this job has received its shutdown signal.
* A draining worker waits for its current job to finish before exiting, so a
* long-running script can poll this and return early to let the worker be replaced.
* @param jobId? Job to check, defaults to the current job
*/
async workerIsDraining(jobId?: string): Promise<boolean>
/**
* Set a flow user state
* @param key key of the state
@@ -4620,6 +4644,18 @@ def set_progress(value: int, job_id: Optional[str] = None)
# Progress value (0-100) or None if not set
def get_progress(job_id: Optional[str] = None) -> Any
# Whether the worker running the job has received its shutdown signal.
#
# A draining worker waits for its current job to finish before exiting. A long-running
# script can poll this and return early so the worker can be replaced.
#
# Args:
# job_id: Job ID (defaults to current WM_JOB_ID)
#
# Returns:
# True if the worker running the job is draining
def worker_is_draining(job_id: Optional[str] = None) -> bool
# Set the user state of a flow at a given key
def set_flow_user_state(key: str, value: Any) -> None
@@ -7879,10 +7915,48 @@ A **data pipeline** is NOT a flow. A flow is one runnable that orchestrates step
A pipeline node that produces a table should almost always be a **\`duckdb\`** node that materializes its output into a **DuckLake** table with \`-- materialize ducklake://<name>/<table>\` (in a DuckDB node the annotation uses SQL \`--\` comment syntax; write the body as a bare \`SELECT\` and let the runtime do the write). DuckLake is the default lakehouse store for pipelines and is the shape the pipeline editor is built around, so prefer it unless the work specifically calls for something else:
- \`postgresql\` / data tables — only for row-level, OLTP-style mutations against an existing Postgres data table (frequent single-row upserts/updates, transactional reads that an app queries live).
- \`bun\` / \`python3\` — only for non-tabular work that doesn't map to SQL: calling an external API, wrangling files, arbitrary glue. When such a node still produces tabular data for downstream steps, land it in DuckLake (write it with the wmill SDK / ducklake helpers) rather than inventing a parallel store.
- \`bun\` / \`python3\` — only for work that doesn't map to SQL: calling an external API, wrangling files, arbitrary glue. When such a node produces tabular data for downstream steps, write it **straight into DuckLake** (or a data table) from the same script — see "Ingesting from Python / TypeScript" below — rather than inventing a parallel store.
Do not spread a pipeline across postgres, S3, and DuckLake when one DuckLake lake would do; a consistent DuckLake lakehouse is the goal.
## Ingesting from Python / TypeScript
A node that fetches data in code (an API, a SaaS export, a scraper) writes the rows **directly** into their DuckLake table with \`wmill.ducklake()\`, or into a data table with \`wmill.datatable()\`. DuckLake already stores every table as parquet files in the workspace object storage, so **never stage the raw payload first** — no intermediate JSON / CSV / parquet file in S3 and no second "load the raw file" node. One ingestion node, one table.
Pass the rows as a query argument (a list of dicts / array of objects is sent as JSON) and let SQL unnest them; declare the column types so the table gets a real schema:
\`\`\`python
# pipeline
# on schedule
import wmill
def main():
rows = fetch_charges() # list[dict]
lake = wmill.ducklake()
lake.query("""
CREATE OR REPLACE TABLE stripe_charges AS
SELECT unnest(from_json($rows, '[{"id":"VARCHAR","amount":"BIGINT","created":"BIGINT"}]'), recursive := true)
""", rows=rows).fetch()
\`\`\`
\`\`\`ts
// pipeline
// on schedule
import * as wmill from "windmill-client"
export async function main() {
const rows = await fetchCharges()
const lake = wmill.ducklake()
await lake\`CREATE OR REPLACE TABLE stripe_charges AS
SELECT unnest(from_json(\${rows}, '[{"id":"VARCHAR","amount":"BIGINT","created":"BIGINT"}]'), recursive := true)\`.fetch()
}
\`\`\`
- Use \`INSERT INTO <table> SELECT ...\` instead of \`CREATE OR REPLACE\` to append (an incremental pull), with a one-time \`CREATE TABLE IF NOT EXISTS\` first.
- Assign the client to a variable (\`lake = wmill.ducklake()\`) and keep the SQL a **string literal** (Python) or a template without \`sql.raw\` (TS): that is what lets the graph record the node's write to \`ducklake://main/<table>\`. A chained \`wmill.ducklake().query(...)\` or SQL built in a variable records no output, so the downstream edge never forms.
- \`wmill.ducklake("<name>")\` targets a non-default catalog; \`wmill.datatable("<name>")\` writes a Postgres data table the same way (Python \`db.query(sql, *args)\` with \`$1\` placeholders, TS the same template form) — use it only when the data belongs in Postgres (see above).
- The downstream node triggers on the table itself: \`-- on ducklake://main/stripe_charges\`.
## Storage prerequisites
A DuckLake pipeline only runs once the workspace has **object storage** (S3 / Azure Blob / GCS) **and a DuckLake catalog** configured — DuckLake tables and \`s3://\` assets can't be materialized or read without it. Check which DuckLake catalogs the workspace has before you build.
@@ -7926,7 +8000,7 @@ The key must be a **string literal** — the graph parser is static and cannot f
## Materialize (the managed output)
> **A MANAGED \`// materialize\` is DuckDB-only**, and its target must be a DuckLake table (\`ducklake://<name>/<table>\`). Deploy **rejects** a \`ducklake://\` \`// materialize\` on any other language (\`python3\`, \`bun\`, \`postgresql\`). For a non-DuckDB node writing the lake, do **not** use \`// materialize\` — write the output via the SDK (\`wmill.writeS3File(...)\`, ducklake helpers, …) and let it be inferred. Use \`duckdb\` when a node should materialize a DuckLake table.
> **A MANAGED \`// materialize\` is DuckDB-only**, and its target must be a DuckLake table (\`ducklake://<name>/<table>\`). Deploy **rejects** a \`ducklake://\` \`// materialize\` on any other language (\`python3\`, \`bun\`, \`postgresql\`). For a non-DuckDB node writing the lake, do **not** use \`// materialize\` — write the table itself with \`wmill.ducklake()\` (see "Ingesting from Python / TypeScript") and let the output be inferred. Use \`duckdb\` when a node should materialize a DuckLake table.
>
> The one target ANY language may declare (except a dbt script, whose writes come from its manifest) is a **warehouse relation**: \`// materialize manual dbt://<warehouse>/<schema>/<name>\`, where \`<warehouse>\` is a warehouse the workspace configures under Settings → dbt. \`manual\` is the only mode it has — nothing generates warehouse DDL, so the node issues its own write (a postgresql \`CREATE TABLE\` / \`INSERT\`, an SDK load, …) and the annotation records the outcome. Use it on an ingestion node whose output a dbt project reads as a \`source\`: the declared relation and the dbt model land on ONE graph node, and a downstream \`// on dbt://<warehouse>/<schema>/<name>\` fires when the ingestion node completes.
frontend/package-lock.json
Generated
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@windmill-labs/components",
"version": "1.827.0",
"version": "1.829.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@windmill-labs/components",
"version": "1.827.0",
"version": "1.829.0",
"hasInstallScript": true,
"license": "AGPL-3.0",
"dependencies": {
frontend/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@windmill-labs/components",
"version": "1.827.0",
"version": "1.829.0",
"scripts": {
"dev": "vite dev",
"dev:ui-builder": "mv static/ui_builder static/ui_builder.dev-disabled 2>/dev/null || true ; trap 'mv static/ui_builder.dev-disabled static/ui_builder 2>/dev/null || true' EXIT ; vite dev",
frontend/src/lib/apiBaseUrl.svelte.ts
+22
View File
@@ -0,0 +1,22 @@
import { fromStore } from 'svelte/store'
import { base } from '$lib/base'
import { instanceUi } from '$lib/instanceUi'
const ui = fromStore(instanceUi)
/**
* Base of every URL shown for an external client to call (webhooks, HTTP routes, push
* trigger endpoints, the CLI remote): the `api_base_url` instance setting, else the
* origin being browsed. Append `/api/...` to it.
*
* Not for URLs the editor itself calls or opens (captures, previews, OAuth popups, links
* to pages): those must stay on the browsing origin, where the session cookie lives.
*
* Reactive: the setting lands after page load, so read it inside a `$derived` or the
* template rather than caching the result.
*/
export function apiBaseUrl(): string {
const setting = ui.current?.api_base_url
const configured = typeof setting === 'string' ? setting.trim() : ''
return configured || `${window.location.origin}${base}`
}
frontend/src/lib/components/AutosaveIndicator.svelte
+7 -1
View File
@@ -142,7 +142,13 @@
let hintTimer: ReturnType<typeof setTimeout> | undefined
const kindLabel = $derived(
itemKind === 'flow' ? 'flow' : itemKind === 'app' || itemKind === 'raw_app' ? 'app' : 'script'
itemKind === 'flow'
? 'flow'
: itemKind === 'app' || itemKind === 'raw_app'
? 'app'
: itemKind === 'data_pipeline'
? 'pipeline'
: 'script'
)
// Each false → true transition fires a hint (others > loaded). Seeded to
frontend/src/lib/components/CliHelpBox.svelte
+2 -1
View File
@@ -2,10 +2,11 @@
import ClipboardPanel from './details/ClipboardPanel.svelte'
import Section from './Section.svelte'
import { useOperatingWorkspace } from '$lib/components/operatingWorkspace.svelte'
import { apiBaseUrl } from '$lib/apiBaseUrl.svelte'
const operatingWorkspace = useOperatingWorkspace()
let url = $derived(`${window.location.protocol}//${window.location.hostname}/`)
let url = $derived(`${apiBaseUrl()}/`)
</script>
<Section label="CLI quick setup">
frontend/src/lib/components/CronBuilder.svelte
+1 -1
View File
@@ -15,7 +15,7 @@
contentClasses="p-4"
>
{#snippet trigger()}
<Button variant="accent" size="xs" nonCaptureEvent={true} startIcon={{ icon: Clock }}>
<Button variant="default" unifiedSize="sm" nonCaptureEvent={true} startIcon={{ icon: Clock }}>
Cron builder
</Button>
{/snippet}
frontend/src/lib/components/DatatablePicker.svelte
+8 -1
View File
@@ -17,6 +17,9 @@
selectInputClass?: string
class?: string
onClear?: () => void
/** Size the input to its content, like Path's folder picker. */
useContentEditable?: boolean
RightIcon?: any
}
let {
@@ -27,7 +30,9 @@
placeholder = undefined,
selectInputClass = '',
class: className = '',
onClear = undefined
onClear = undefined,
useContentEditable = false,
RightIcon = undefined
}: Props = $props()
let datatables = usePromise(() =>
@@ -47,6 +52,8 @@
{placeholder}
inputClass={selectInputClass}
{onClear}
{useContentEditable}
{RightIcon}
/>
{#if showSchemaExplorer && value && assetCanBeExplored({ kind: 'datatable', path: value })}
<ExploreAssetButton class="mt-1 w-fit" asset={{ kind: 'datatable', path: value }} />
frontend/src/lib/components/DucklakePicker.svelte
+8 -1
View File
@@ -17,6 +17,9 @@
selectInputClass?: string
class?: string
onClear?: () => void
/** Size the input to its content, like Path's folder picker. */
useContentEditable?: boolean
RightIcon?: any
}
let {
@@ -27,7 +30,9 @@
placeholder = undefined,
selectInputClass = '',
class: className = '',
onClear = undefined
onClear = undefined,
useContentEditable = false,
RightIcon = undefined
}: Props = $props()
let ducklakes = usePromise(() =>
@@ -45,6 +50,8 @@
{placeholder}
inputClass={selectInputClass}
{onClear}
{useContentEditable}
{RightIcon}
/>
{#if showSchemaExplorer && value && assetCanBeExplored({ kind: 'ducklake', path: value })}
<ExploreAssetButton class="mt-1 w-fit" asset={{ kind: 'ducklake', path: value }} />
frontend/src/lib/components/Editor.svelte
+3
View File
@@ -1523,6 +1523,9 @@
} catch (e) {
console.log('error initializing services', e)
}
// Unmounted while the services loaded: the container is detached, and
// creating the editor on it throws.
if (destroyed || !divEl?.isConnected) return
// vscode.languages.registerDefinitionProvider('*', {
// provideDefinition(document, position, token) {
frontend/src/lib/components/EditorBar.svelte
+1
View File
@@ -105,6 +105,7 @@
| 'powershell'
| 'bunnative'
| 'claudesandbox'
| 'pisandbox'
| 'wac_python'
| 'wac_typescript'
| 'ci_test_bun'
frontend/src/lib/components/FlowStatusViewerInner.svelte
+5 -1
View File
@@ -69,6 +69,7 @@
retainLinkedToolsScope
} from './flows/linkedAgentToolsStore.svelte'
import { useOperatingWorkspace } from '$lib/components/operatingWorkspace.svelte'
import { AGENT_STEP_ID, agentStepLabel } from './flows/conversations/agentEditorChat'
const operatingWorkspace = useOperatingWorkspace()
@@ -1238,6 +1239,8 @@
id: string
type: FlowModuleValue['type']
suspend?: boolean
/** Shown in place of the id. */
label?: string
}
function allModulesForTimeline(
@@ -1250,7 +1253,8 @@
({
id: x.id,
type: x.value.type,
suspend: x.suspend != undefined
suspend: x.suspend != undefined,
label: x.id === AGENT_STEP_ID ? agentStepLabel(job?.script_path) : undefined
}) as FlowModuleForTimeline,
{
skipToolNodes: true
frontend/src/lib/components/FlowTimeline.svelte
+2 -2
View File
@@ -179,12 +179,12 @@
/>
</div>
{/if}
{#each flowModules as { id: k, type: typ, suspend: isSuspend } (k)}
{#each flowModules as { id: k, type: typ, suspend: isSuspend, label } (k)}
{@const subItems = items?.[k]?.filter((x) => x.created_at && x.started_at)}
<div class="relative px-3 py-1.5">
<div class="flex items-center justify-between mb-0.5">
<div class="text-xs font-medium"
>{k.startsWith('subflow:') ? k.substring(8) : k}
>{label ?? (k.startsWith('subflow:') ? k.substring(8) : k)}
{#if localModuleStates[k]?.selectedForloop && (typ == 'forloopflow' || typ == 'whileloopflow')}
<span class="text-xs font-mono font-medium inline-flex items-center -my-2">
<button onclick={(e) => e.stopPropagation()}>
frontend/src/lib/components/HeaviestScriptsDrawer.svelte
+201
View File
@@ -0,0 +1,201 @@
<script lang="ts">
import { WorkerService } from '$lib/gen'
import { resource } from 'runed'
import { Alert, Drawer, DrawerContent, Section, Skeleton } from './common'
import TextInput from './text_input/TextInput.svelte'
import ToggleButtonGroup from './common/toggleButton-v2/ToggleButtonGroup.svelte'
import ToggleButton from './common/toggleButton-v2/ToggleButton.svelte'
import Select from './select/Select.svelte'
import DataTable from './table/DataTable.svelte'
import Head from './table/Head.svelte'
import Cell from './table/Cell.svelte'
import { formatMemory, msToReadableTime } from '$lib/utils'
import { logFeatureUsage } from '$lib/utils/featureUsage'
interface Props {
workerGroups: string[]
}
let { workerGroups }: Props = $props()
const WINDOWS = {
'24h': 24 * 3600,
'7d': 7 * 24 * 3600,
'30d': 30 * 24 * 3600
}
const ORDERS = {
total_duration: 'Total runtime',
max_memory: 'Max memory',
cpu: 'CPU time'
} as const
type OrderBy = keyof typeof ORDERS
// The server gathers every run without a stable path (previews, dependency jobs) under this one.
const ADHOC_PATH = '<adhoc>'
let drawer: Drawer | undefined = $state()
let opened = $state(false)
let windowKey: keyof typeof WINDOWS = $state('24h')
let orderBy: OrderBy = $state('total_duration')
let workerGroup: string | undefined = $state()
let workspace = $state('')
export function openDrawer(group?: string) {
workerGroup = group
opened = true
drawer?.openDrawer()
logFeatureUsage('worker_resource_stats', 'opened')
}
const stats = resource(
() => ({ opened, windowKey, orderBy, workerGroup, workspace: workspace.trim() }),
async (q, _, { signal }) => {
if (!q.opened) return undefined
try {
return await WorkerService.getRunnableStats({
windowSecs: WINDOWS[q.windowKey],
orderBy: q.orderBy,
workerGroup: q.workerGroup || undefined,
workspace: q.workspace || undefined
})
} finally {
// A slower answer for filters no longer selected must not replace the current one:
// `resource` drops the abort error thrown in its place.
signal.throwIfAborted()
}
},
{ debounce: 200 }
)
// Many scripts run for well under a second, which `msToReadableTime` rounds to "0.0s".
function duration(ms: number) {
return ms < 1000 ? `${Math.round(ms)}ms` : msToReadableTime(ms, 1)
}
function cores(cpuMs: number, durationMs: number) {
return durationMs > 0 ? (cpuMs / durationMs).toFixed(2) : '-'
}
</script>
<Drawer bind:this={drawer} size="1100px" on:close={() => (opened = false)}>
<DrawerContent title="Heaviest scripts" on:close={drawer?.closeDrawer}>
<Section
label="Resource usage per script"
tooltip="Aggregated by the workers once an hour, per script and worker group, so the last hour is incomplete. Memory is the peak of the job's main process. CPU time counts that process and the child processes it waited for. Scripts run inside the worker process (SQL, native TypeScript) and jobs of dedicated workers report neither, and a killed job reports no CPU time."
>
{#snippet action()}
<ToggleButtonGroup bind:selected={windowKey} noWFull>
{#snippet children({ item })}
{#each Object.keys(WINDOWS) as key (key)}
<ToggleButton value={key} label={key} size="sm" {item} />
{/each}
{/snippet}
</ToggleButtonGroup>
{/snippet}
<div class="flex flex-wrap items-center gap-4 pb-4">
<div class="flex items-center gap-2">
<span class="text-xs text-secondary">Sort by</span>
<ToggleButtonGroup
bind:selected={
() => orderBy,
(v) => {
orderBy = v
logFeatureUsage('worker_resource_stats', 'sort', { key: v })
}
}
noWFull
>
{#snippet children({ item })}
{#each Object.entries(ORDERS) as [value, label] (value)}
<ToggleButton {value} {label} size="sm" {item} />
{/each}
{/snippet}
</ToggleButtonGroup>
</div>
<div class="w-48">
<Select
items={workerGroups.map((g) => ({ label: g, value: g }))}
bind:value={workerGroup}
placeholder="All worker groups"
clearable
size="sm"
/>
</div>
<div class="w-48">
<TextInput
bind:value={workspace}
inputProps={{ placeholder: 'Workspace id' }}
size="sm"
/>
</div>
</div>
{#if stats.error}
<Alert type="error" title="Failed to load the script statistics">
{stats.error.message}
</Alert>
{:else if stats.current === undefined}
<Skeleton layout={[[8]]} />
{:else if stats.current.length === 0}
<p class="text-secondary text-xs">No jobs were recorded for this selection.</p>
{:else}
<DataTable size="sm" noBorder={false} rounded={true}>
<Head>
<tr>
<Cell head first>Script</Cell>
<Cell head>Workspace</Cell>
<Cell head>Worker group</Cell>
<Cell head numeric>Jobs</Cell>
<Cell head numeric>Total runtime</Cell>
<Cell head numeric>Avg runtime</Cell>
<Cell head numeric>Max memory</Cell>
<Cell head numeric>Avg memory</Cell>
<Cell head numeric>CPU time</Cell>
<Cell head numeric last>Avg CPUs</Cell>
</tr>
</Head>
<tbody>
{#each stats.current as s (s.workspace_id + '\0' + s.runnable_path + '\0' + s.worker_group)}
<tr class="border-b last:border-b-0">
<Cell first class="text-xs text-primary">
{#if s.runnable_path === ADHOC_PATH}
<span class="text-secondary">Previews and other ad-hoc runs</span>
{:else}
<span class="font-mono">{s.runnable_path}</span>
{/if}
</Cell>
<Cell class="text-xs text-primary">{s.workspace_id}</Cell>
<Cell class="text-xs text-primary">{s.worker_group}</Cell>
<Cell numeric class="text-xs text-primary">{s.job_count}</Cell>
<Cell numeric class="text-xs text-primary">
{duration(s.total_duration_ms)}
</Cell>
<Cell numeric class="text-xs text-primary">
<!-- A workflow suspended and not yet completed has time recorded and no job. -->
{s.job_count > 0 ? duration(s.total_duration_ms / s.job_count) : '-'}
</Cell>
{#if s.memory_sample_count > 0}
<Cell numeric class="text-xs text-primary">
{formatMemory(s.max_memory_peak)}
</Cell>
<Cell numeric class="text-xs text-primary">
{formatMemory(Math.round(s.sum_memory_peak / s.memory_sample_count))}
</Cell>
{:else}
<Cell numeric class="text-xs text-secondary">-</Cell>
<Cell numeric class="text-xs text-secondary">-</Cell>
{/if}
<Cell numeric class="text-xs text-primary">
{duration(s.total_cpu_ms)}
</Cell>
<Cell numeric last class="text-xs text-primary">
{cores(s.total_cpu_ms, s.total_duration_ms)}
</Cell>
</tr>
{/each}
</tbody>
</DataTable>
{/if}
</Section>
</DrawerContent>
</Drawer>
frontend/src/lib/components/JobArgs.svelte
+2 -1
View File
@@ -15,6 +15,7 @@
import { isWindmillTooBigObject } from './job_args'
import { downloadViaClient, shouldDownloadViaClient } from '$lib/utils/downloadFile'
import { appendViewToken } from '$lib/viewToken'
import { apiBaseUrl } from '$lib/apiBaseUrl.svelte'
interface Props {
id?: string | undefined
@@ -246,7 +247,7 @@ ${Object.entries(displayArgs)
If using the wmill client in your code, set the following env variables:
<pre
><code
>BASE_URL="{window.location.origin}"
>BASE_URL="{apiBaseUrl()}"
WM_TOKEN="{'<TOKEN>'}"</code
></pre
>
frontend/src/lib/components/NavBreadcrumb.svelte
+11 -1
View File
@@ -302,12 +302,13 @@ not this component) — so the lead can run the whole line.
{createMenu}
usePointerDownOutside
placement="bottom-start"
class="h-auto"
contentStyle="margin-left: {-nameWidth}px"
>
{#snippet triggr({ trigger })}
<MeltButton
meltElement={trigger}
class="flex items-center p-1.5 rounded text-tertiary hover:bg-surface-hover hover:text-primary transition-colors"
class="flex items-center h-9 px-1.5 rounded text-tertiary hover:bg-surface-hover hover:text-primary transition-colors"
title="Switch workspace"
>
<ChevronDown size={ICON} class="flex-shrink-0" />
@@ -406,6 +407,15 @@ not this component) — so the lead can run the whole line.
{@render navPicker()}
</div>
{:else if section}
{#if section.parent}
<!-- Gives width back with the workspace part, before the section's own name does. -->
<span
class={twMerge(SEGMENT, LEAD, 'shrink-[999] hover:bg-transparent hover:text-secondary')}
>
<span class="truncate">{section.parent}</span>
</span>
{@render slash()}
{/if}
<!-- A section's own widget lays out as a row: its title, and whatever belongs with it. -->
<div class="flex items-center min-w-0">
<span
frontend/src/lib/components/PageHeaderBar.svelte
+14 -3
View File
@@ -33,6 +33,7 @@ The row's height matches the sidebar's own header row, so the two read as one ba
const item = $derived(content?.item)
const section = $derived(content?.section)
const hasSummary = $derived(!!item && !!(item.summaryContent || item.summary))
// Below this the trail holds a tighter cap on the path it draws. The names in it truncate on
// their own at any width (see NavBreadcrumb), so this is not a breakpoint the look changes at —
@@ -136,8 +137,18 @@ The row's height matches the sidebar's own header row, so the two read as one ba
where the user is.
An embed (`navHidden`) keeps the page's own name but drops the workspace part around it:
the trail would offer to navigate the host's workspace, while the name is what says which
page the controls beside it belong to. -->
<div class="flex shrink-[0.3] min-w-[5rem]">
page the controls beside it belong to.
The 0.3 only holds against a summary or an actions box that shrinks too: alone, a factor
sum below 1 makes flex hand out only that share of the overflow, and the bar overflows
instead. -->
<div
class={twMerge(
'flex shrink-[0.3] min-w-[5rem]',
!hasSummary &&
(actions.length === 0 || (phone && !content?.actionsFlexible && !content?.actionsFill)) &&
'shrink'
)}
>
<!-- Bridged like the actions below: what a page hangs off its own name renders here, out of
the tree that named it, and the rename editor in there asks that tree who the acting user
is before it offers to rename anything. -->
@@ -162,7 +173,7 @@ The row's height matches the sidebar's own header row, so the two read as one ba
</svelte:boundary>
</div>
{#if item && (item.summaryContent || item.summary)}
{#if item && hasSummary}
<!-- No kind icon: the page below is the item, and saying "this is a flow" above a flow
editor tells the reader what they can already see. -->
<div class={twMerge('flex items-center gap-1 min-w-0', phone && 'shrink')}>
frontend/src/lib/components/ScriptBuilder.svelte
+20 -2
View File
@@ -596,6 +596,7 @@
| 'powershell'
| 'bunnative'
| 'claudesandbox'
| 'pisandbox'
| 'wac_python'
| 'wac_typescript'
| 'ci_test_bun'
@@ -1543,10 +1544,12 @@
(lang == script.language &&
template != 'bunnative' &&
template != 'docker' &&
template != 'claudesandbox') ||
template != 'claudesandbox' &&
template != 'pisandbox') ||
(template == 'bunnative' && lang == 'bunnative') ||
(template == 'docker' && lang == 'docker') ||
(template == 'claudesandbox' && lang == 'bun')}
((template == 'claudesandbox' || template == 'pisandbox') &&
lang == 'bun')}
<Popover
disablePopup={!enterpriseLangs.includes(lang) || !!$enterpriseLicense}
>
@@ -1597,6 +1600,21 @@
>
Claude Sandbox
</Button>
<Button
unifiedSize="xs"
variant="default"
startIcon={{
icon: LanguageIcon,
props: { lang: 'pisandbox', width: 16, height: 16 }
} as ButtonType.Icon}
onclick={() => {
template = 'pisandbox'
script.language = 'bun'
initContent('bun', script.kind, template)
}}
>
Pi Sandbox
</Button>
<span title="Workflow-as-Code">
<Button
size="xs2"
frontend/src/lib/components/ScriptEditor.svelte
+35 -37
View File
@@ -150,6 +150,7 @@
| 'powershell'
| 'bunnative'
| 'claudesandbox'
| 'pisandbox'
| 'wac_python'
| 'wac_typescript'
| 'ci_test_bun'
@@ -2100,43 +2101,15 @@
</div>
{/if}
{#if customUi?.previewPanel?.hideArgs}
<!-- Compact preview layout used by the pipeline editor:
no args column (the script is known to take no
inputs), LogPanel takes the full width, with a
small Test/Cancel button at the top-left of the
preview band. The earlier `-translate-y-1/2`
"float onto the editor" version was clipped in
Firefox (transform + overflow: visible interaction
on the parent splitpane Pane), so the button is
now positioned inside the panel — `top-1` keeps
it visually pinned to the top edge without
relying on cross-browser overflow behaviour. -->
<div class="relative h-full pt-9 flex flex-col">
{#if testJob?.id && testJob.type === 'CompletedJob' && opWs}
<!-- Right-side affordances when we're displaying a *completed*
job (either the user just ran a test, or the on-mount
last-run loader populated the panel). The job-id link
opens the full run page in a new tab; the dispatch
button mounts a popover that shows what downstream
jobs this run triggered (rendered only when the run
actually dispatched anything). -->
<div class="absolute top-1 right-2 z-10 flex items-center gap-2">
<a
class="text-3xs text-blue-600 hover:underline font-mono"
href={`${base}/run/${testJob.id}?workspace=${opWs}`}
target="_blank"
rel="noopener noreferrer"
title="Open this run"
>
{testJob.id.slice(0, 8)}… ↗
</a>
<DispatchEventsButton
workspace={testJob.workspace_id ?? opWs}
jobId={testJob.id}
/>
</div>
{/if}
<div class="absolute top-1 left-2 z-10">
<!-- Compact preview layout used by the pipeline editor: no args
column (most pipeline scripts take no inputs) and a full-width
LogPanel. The controls sit in normal flow rather than floating
onto the editor band, which Firefox clipped (transform +
overflow: visible on the parent splitpane Pane). -->
<div class="h-full flex flex-col">
<!-- Test sits at the top right, the last run's link under it. -->
<div class="shrink-0 flex flex-col items-end gap-1 px-2 pt-2 pb-1">
<div>
{#if testIsLoading}
{@render cancelTestButton('sm', 'shadow-md')}
{:else if (customUi?.previewPanel?.downstreamSubscribers ?? 0) > 0 || customUi?.previewPanel?.onBoundedRun}
@@ -2270,6 +2243,31 @@
{@render runTestButton('sm', 'shadow-md')}
{/if}
</div>
{#if testJob?.id && testJob.type === 'CompletedJob' && opWs}
<!-- Right-side affordances when we're displaying a *completed*
job (either the user just ran a test, or the on-mount
last-run loader populated the panel). The job-id link
opens the full run page in a new tab; the dispatch
button mounts a popover that shows what downstream
jobs this run triggered (rendered only when the run
actually dispatched anything). -->
<div class="flex items-center gap-2">
<a
class="text-3xs text-blue-600 hover:underline font-mono"
href={`${base}/run/${testJob.id}?workspace=${opWs}`}
target="_blank"
rel="noopener noreferrer"
title="Open this run"
>
{testJob.id.slice(0, 8)}… ↗
</a>
<DispatchEventsButton
workspace={testJob.workspace_id ?? opWs}
jobId={testJob.id}
/>
</div>
{/if}
</div>
{#if customUi?.previewPanel?.argsAboveLogs && schema?.properties && Object.keys(schema.properties).length > 0}
<div
class="shrink-0 overflow-auto max-h-[45%] border-b border-gray-200 dark:border-gray-700 px-4 py-3"
frontend/src/lib/components/SummaryPathDisplay.svelte
+22 -1
View File
@@ -31,6 +31,13 @@
/** What is being renamed. An agent is a resource, so the path field is scoped to one and
* the write goes through the agent branch of the rename manager. */
kind?: 'flow' | 'script' | 'agent'
/** Saves instead of the API, for an item that exists only locally (a draft).
* Returns an error message to keep the popover open. */
saveOverride?: (next: {
path: string
summary: string
labels: string[] | undefined
}) => string | undefined
/** Header variant: no path line (the host shows it) and a lighter summary. */
compact?: boolean
/** Rendered inside the trigger in place of the summary, for a host that hangs a second copy
@@ -51,6 +58,7 @@
editable = false,
onSaved,
kind = 'flow',
saveOverride = undefined,
compact = false,
label,
triggerClass = 'block min-w-0 max-w-full px-1 py-0.5 rounded text-left cursor-pointer hover:bg-surface-hover transition-colors',
@@ -67,7 +75,9 @@
let popoverOpen = $state(false)
// Resolved before the popover opens, not on opening it: the path field only exists for an owner,
// and `openFocus` looks for it in the same flush the popover renders in.
const own = $derived(isOwner(path ?? '', actingUser, $operatingWorkspace))
const own = $derived(
saveOverride !== undefined || isOwner(path ?? '', actingUser, $operatingWorkspace)
)
let onBehalfOfEmail = $state<string | undefined>(undefined)
let summaryInput: ReturnType<typeof TextInput> | undefined = $state()
let labelsDirty = $state(false)
@@ -101,6 +111,17 @@
async function save(close: () => void) {
const initialPath = path ?? ''
const newPath = own ? editPath : initialPath
if (saveOverride) {
const error = saveOverride({ path: newPath, summary: editSummary, labels })
if (error) {
sendUserToast(error, true)
return
}
labelsDirty = false
close()
onSaved?.(newPath)
return
}
try {
await updateItemPathAndSummary({
frontend/src/lib/components/SuperadminSettingsInner.svelte
+1 -1
View File
@@ -372,7 +372,7 @@
</div>
{/if}
<Tabs bind:selected={usersSubTab} class="mb-4">
<Tabs bind:selected={usersSubTab} class="mb-4" wrapperClass="shrink-0">
<Tab value="users" label="Users" />
{#if extJwtTokens.length > 0}
<Tab value="ext_jwt" label="External JWTs" />
frontend/src/lib/components/WorkerGroup.svelte
+2 -1
View File
@@ -55,6 +55,7 @@
import TagList from './TagList.svelte'
import DedicatedWorkersSelector from './DedicatedWorkersSelector.svelte'
import { computeHashedTag } from './dedicated_worker'
import { apiBaseUrl } from '$lib/apiBaseUrl.svelte'
import ObjectStoreConfigSettings, {
type ObjectStoreConfig
} from './ObjectStoreConfigSettings.svelte'
@@ -1238,7 +1239,7 @@
<p class="text-xs text-tertiary">Requires a superadmin token.</p>
<pre
class="mt-1 p-2 bg-surface-secondary rounded text-xs overflow-x-auto whitespace-pre-wrap break-all"
>curl -X POST '{window.location.origin}/api/configs/update/worker__{name}' \
>curl -X POST '{apiBaseUrl()}/api/configs/update/worker__{name}' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer &lt;token&gt;' \
-d '{JSON.stringify(cleanWorkerGroupConfig(nconfig ?? {}), null, 2)}'</pre
frontend/src/lib/components/assets/AssetGraph/AddDataSourceMenu.svelte
+150
View File
@@ -0,0 +1,150 @@
<!--
@component
The menu that adds a pipeline script started by a data source (a schedule, a
webhook, a queue…): the add node's, and the canvas's right-click one.
-->
<script lang="ts">
import type { Snippet } from 'svelte'
import PipelineInsertMenu, {
type PipelineInsertOptions,
type PipelineInsertPick
} from './PipelineInsertMenu.svelte'
import {
Clock,
Webhook,
Mail,
Zap,
Radio,
MessageSquare,
Database,
Send,
CloudCog,
Upload
} from 'lucide-svelte'
import type { ScriptLang } from '$lib/gen'
import type { NativeTriggerKind } from './types'
import { PIPELINE_LANGUAGES } from './pipelineLanguages'
import type { PipelineOutputKind } from './pipelineTemplates'
interface Props {
onAddPipelineScript: (
language: ScriptLang,
path: string,
source: { kind: NativeTriggerKind; path: string | undefined },
outputKind: PipelineOutputKind,
aiPrompt?: string,
options?: PipelineInsertOptions
) => void
pathPrefix: string
defaultPathSuffix: string
trigger: Snippet<[{ open: boolean }]>
/** Opens the menu each time it changes. */
openSignal?: number
}
let {
onAddPipelineScript,
pathPrefix,
defaultPathSuffix,
trigger,
openSignal
}: Props = $props()
function handlePick(pick: PipelineInsertPick) {
if (!pick.language || !pick.path) return
// Native trigger annotation is marker-only — the binding lives on the
// trigger row's own `script_path`, which is created separately.
onAddPipelineScript(
pick.language as ScriptLang,
pick.path,
{ kind: pick.kindId as NativeTriggerKind, path: undefined },
(pick.outputKind ?? 'none') as PipelineOutputKind,
pick.aiPrompt,
{ schedule: pick.schedule, outputAsset: pick.outputAsset }
)
}
</script>
<PipelineInsertMenu
kinds={[
{
id: 'data_upload',
label: 'On data upload',
description: 'UI-first: run by uploading a file via the S3 picker',
icon: Upload
},
{
id: 'schedule',
label: 'On schedule',
description: 'Triggered by a schedule you create',
icon: Clock
},
{
id: 'webhook',
label: 'On webhook',
description: 'Triggered by an HTTP webhook',
icon: Webhook
},
{
id: 'email',
configuredAfterCreate: true,
label: 'On email',
description: 'Triggered by incoming email',
icon: Mail
},
{
id: 'kafka',
configuredAfterCreate: true,
label: 'On Kafka',
description: 'Triggered by a Kafka message',
icon: Zap
},
{
id: 'mqtt',
configuredAfterCreate: true,
label: 'On MQTT',
description: 'Triggered by an MQTT message',
icon: Radio
},
{
id: 'amqp',
configuredAfterCreate: true,
label: 'On AMQP',
description: 'Triggered by an AMQP (RabbitMQ) message',
icon: Radio
},
{
id: 'nats',
configuredAfterCreate: true,
label: 'On NATS',
description: 'Triggered by a NATS message',
icon: MessageSquare
},
{
id: 'postgres',
configuredAfterCreate: true,
label: 'On Postgres',
description: 'Triggered by a Postgres event',
icon: Database
},
{
id: 'sqs',
configuredAfterCreate: true,
label: 'On SQS',
description: 'Triggered by an SQS message',
icon: Send
},
{
id: 'gcp',
configuredAfterCreate: true,
label: 'On GCP Pub/Sub',
description: 'Triggered by a Pub/Sub message',
icon: CloudCog
}
]}
languages={PIPELINE_LANGUAGES as any}
{pathPrefix}
{defaultPathSuffix}
onPick={handlePick}
{openSignal}
{trigger}
/>
frontend/src/lib/components/assets/AssetGraph/AddDownstreamMenu.svelte
+73
View File
@@ -0,0 +1,73 @@
<!--
@component
The menu adding a pipeline script downstream of an asset: opened from the asset's
"+", or where that "+" was dropped on the blank canvas.
-->
<script lang="ts" module>
import type { ScriptLang } from '$lib/gen'
import type { AssetKind } from '$lib/components/assets/lib'
import type { PipelineInsertOptions } from './PipelineInsertMenu.svelte'
import type { PipelineOutputKind } from './pipelineTemplates'
export type AddScriptForAsset = (
asset: { kind: AssetKind; path: string },
language: ScriptLang,
scriptPath: string,
outputKind: PipelineOutputKind,
aiPrompt?: string,
options?: PipelineInsertOptions
) => void
</script>
<script lang="ts">
import { Code2 } from 'lucide-svelte'
import type { Snippet } from 'svelte'
import PipelineInsertMenu, { type PipelineInsertPick } from './PipelineInsertMenu.svelte'
import { PIPELINE_LANGUAGES } from './pipelineLanguages'
let {
asset,
onAddScript,
pathPrefix = '',
defaultPathSuffix = '',
openSignal,
trigger
}: {
asset: { kind: AssetKind; path: string }
onAddScript: AddScriptForAsset
pathPrefix?: string
defaultPathSuffix?: string
openSignal?: number
trigger: Snippet<[{ open: boolean }]>
} = $props()
function handlePick(pick: PipelineInsertPick) {
if (pick.kindId === 'pipeline_script' && pick.language && pick.path) {
onAddScript(
asset,
pick.language as ScriptLang,
pick.path,
(pick.outputKind ?? 'none') as PipelineOutputKind,
pick.aiPrompt,
{ outputAsset: pick.outputAsset }
)
}
}
</script>
<PipelineInsertMenu
kinds={[
{
id: 'pipeline_script',
label: 'Add downstream pipeline script',
description: 'Triggered when this asset changes',
icon: Code2
}
]}
languages={PIPELINE_LANGUAGES as any}
{pathPrefix}
{defaultPathSuffix}
{openSignal}
onPick={handlePick}
{trigger}
/>

Some files were not shown because too many files have changed in this diff Show More