starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-09-05 08:02:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into glm/frontend/add_notes_to_flow

Browse Source 
# Conflicts:
#	backend/windmill-common/src/flows.rs
#	backend/windmill-queue/src/jobs.rs
#	frontend/src/lib/components/FlowBuilder.svelte
#	frontend/src/lib/components/flows/content/FlowEditorPanel.svelte
#	frontend/src/lib/components/flows/utils.svelte.ts
This commit is contained in:
Guilhem
2025-11-12 23:26:57 +01:00
parent eb4b524c6c 77316cbd0e
commit bf2f8355c5
262 changed files with 6789 additions and 3247 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/git-commands.yaml
+2 -1
View File
@@ -40,11 +40,12 @@ jobs:
with:
github-token: ${{ steps.app.outputs.token }}
script: |
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: 'Starting sqlx update...'
body: `Starting sqlx update...\n\n[View workflow run](${runUrl})`
})
- name: Checkout repository
.github/workflows/weekly-pr-summary.yml
+14 -18
View File
@@ -49,14 +49,14 @@ jobs:
- **Bug Fixes**: PRs with titles starting with "fix:", "bug:", or containing "fix", "resolve", "patch"
- **Other**: All remaining PRs (improvements, refactors, docs, chores, etc.)
4. **Gather Details**: For each merged PR, include:
4. **Gather Details**: For each feature and bug fix merged PR, include:
- Full PR title (NO truncation, NO links)
- Author (extract login from author.login in JSON)
- Brief summary: Use `gh pr view <number> --json body` to get PR description, then extract first paragraph or key points (1-2 sentences max)
5. **Character Limit Enforcement**:
- The final summary MUST be under 6000 characters
- If the summary exceeds 6000 characters, truncate PR descriptions (NOT titles) and add at the end: "_and X more PRs_" where X is the count of omitted PRs
- The final summary MUST be under 5000 characters
- If the summary exceeds 5000 characters, truncate PR descriptions (NOT titles) and add at the end: "_and X more PRs_" where X is the count of omitted PRs
6. **Save Summary to Markdown File**: Write the summary to a file for webhook delivery:
- Save the complete formatted markdown to: `summary.md`
@@ -65,23 +65,19 @@ jobs:
## Output Format:
```markdown
#### 📈 Weekly overview
### 📈 Weekly overview
- **Total merged**: X
- **Features**: Y
- **Bug Fixes**: Z
- **Other**: W
#### ✨ Features (Y)
• **[Full PR Title]** by @username - [brief impact description]
• **[Full PR Title]** by @username - [brief impact description]
### ✨ Features (Y)
- **[Full PR Title]** by @username - [brief impact description]
- **[Full PR Title]** by @username - [brief impact description]
#### 🐛 Bug Fixes (Z)
• **[Full PR Title]** by @username - [brief impact description]
• **[Full PR Title]** by @username - [brief impact description]
#### 🔧 Other (W)
• **[Full PR Title]** by @username - [brief impact description]
• **[Full PR Title]** by @username - [brief impact description]
### 🐛 Bug Fixes (Z)
- **[Full PR Title]** by @username - [brief impact description]
- **[Full PR Title]** by @username - [brief impact description]
_and X more PRs_
```
@@ -89,9 +85,9 @@ jobs:
## Important Notes:
- **CRITICAL**: ONLY include PRs with state "merged" from the last 7 days
- **CRITICAL**: EXCLUDE all PRs with titles starting with "chore: release" or "chore(release)"
- **CRITICAL**: Total character count MUST be under 6000 characters
- Only use #### markdown headers for major sections and emoji indicators
- Use bullet points (•) for individual PR entries - more compact than paragraphs
- **CRITICAL**: Total character count MUST be under 5000 characters
- Count the number of "Other" PRs but do not include a section for them in the output
- Only use ### markdown headers for major sections and emoji indicators
- NO links to PRs
- NO merged date in output
- NEVER truncate PR titles - show full titles
@@ -100,7 +96,7 @@ jobs:
- If a PR has no description, write "(No description provided)"
- Extract meaningful summary from PR body - look for the first paragraph or key bullet points
- Parse JSON responses carefully using `jq` or similar tools
- If summary exceeds 6000 chars, shorten PR descriptions and add "_and X more PRs_" at the end
- If summary exceeds 5000 chars, shorten PR descriptions and add "_and X more PRs_" at the end
- Count PRs in each category and display in both overview and section headers
## Saving the Markdown Output:
CHANGELOG.md
+146
View File
@@ -1,5 +1,151 @@
# Changelog
## [1.575.0](https://github.com/windmill-labs/windmill/compare/v1.574.3...v1.575.0) (2025-11-12)
### Features
* SQL result_collection flag ([#7041](https://github.com/windmill-labs/windmill/issues/7041)) ([14e41ba](https://github.com/windmill-labs/windmill/commit/14e41babfe6321abb22081b1b3601e9009451257))
### Bug Fixes
* allow --yes in fork command ([#7122](https://github.com/windmill-labs/windmill/issues/7122)) ([1280617](https://github.com/windmill-labs/windmill/commit/1280617a283ef7bcf53c17d833f1413ffe69dbf8))
* **backend:** scim email filter ([#7120](https://github.com/windmill-labs/windmill/issues/7120)) ([7a8fd86](https://github.com/windmill-labs/windmill/commit/7a8fd865e289522fd2d5eaaa9af471f01193a03d))
* **flow chat:** fix chat in edit mode + cleaner code ([#7118](https://github.com/windmill-labs/windmill/issues/7118)) ([9791eb4](https://github.com/windmill-labs/windmill/commit/9791eb4568aa9548174941f1bbfb6a338c3bef00))
## [1.574.3](https://github.com/windmill-labs/windmill/compare/v1.574.2...v1.574.3) (2025-11-11)
### Bug Fixes
* **backend:** add sslmode parameter to DuckLake PostgreSQL connections ([ea5b3e6](https://github.com/windmill-labs/windmill/commit/ea5b3e6b9269a0c631c7601313ab97c4bbdc2137))
* use teams internal_id rather than id ([#7113](https://github.com/windmill-labs/windmill/issues/7113)) ([25c36c1](https://github.com/windmill-labs/windmill/commit/25c36c1e0a6cfc56e079ff5a18980ea67403d4f9))
## [1.574.2](https://github.com/windmill-labs/windmill/compare/v1.574.1...v1.574.2) (2025-11-10)
### Bug Fixes
* **backend:** improve pwsh param block parsing ([#7096](https://github.com/windmill-labs/windmill/issues/7096)) ([51cba95](https://github.com/windmill-labs/windmill/commit/51cba95d3961ec637fd5060556f0e7a8a8bdbb88))
* **backend:** improve sse parsing ([#7100](https://github.com/windmill-labs/windmill/issues/7100)) ([2e245ac](https://github.com/windmill-labs/windmill/commit/2e245accc5f641f64a2cd28fc181e6a806c407e2))
* fix multiselect in list for apps ([9e4882c](https://github.com/windmill-labs/windmill/commit/9e4882c0a919303480f5a431db3c05b1d5855d37))
* make ai chat works with unicode messages ([2d54dfb](https://github.com/windmill-labs/windmill/commit/2d54dfbf05f21faa14655b7124daa291a9d8d66d))
* preproccessor ui and expanding preprocessor support ([#6872](https://github.com/windmill-labs/windmill/issues/6872)) ([fc5034e](https://github.com/windmill-labs/windmill/commit/fc5034e94d513bab2feb0ebb2aff822bbb42e4aa))
* proper error handling in pulled job preprocessor ([#7098](https://github.com/windmill-labs/windmill/issues/7098)) ([84992cd](https://github.com/windmill-labs/windmill/commit/84992cd8ffa94e009aeb6b25db224a3001581e15))
## [1.574.1](https://github.com/windmill-labs/windmill/compare/v1.574.0...v1.574.1) (2025-11-08)
### Bug Fixes
* make get_logs work even for partial flow jobs ([d6421c2](https://github.com/windmill-labs/windmill/commit/d6421c2ea79993ef7815c50cf035d3b3425e271e))
## [1.574.0](https://github.com/windmill-labs/windmill/compare/v1.573.5...v1.574.0) (2025-11-07)
### Features
* env var in flow ([#6852](https://github.com/windmill-labs/windmill/issues/6852)) ([c59183f](https://github.com/windmill-labs/windmill/commit/c59183f5c39f853d9679c00dd5aa755ef171d735))
### Bug Fixes
* **cli:** add automatic handler of .node files for codebase bundler ([62ffe9f](https://github.com/windmill-labs/windmill/commit/62ffe9ffce6d070c81794e8483ab83b431aee9ea))
* teams selector svelte5 ([#7087](https://github.com/windmill-labs/windmill/issues/7087)) ([6045f0c](https://github.com/windmill-labs/windmill/commit/6045f0c40654a88e93be688bcdfab874cfc0b267))
## [1.573.5](https://github.com/windmill-labs/windmill/compare/v1.573.4...v1.573.5) (2025-11-07)
### Bug Fixes
* dep jobs improvements ([#7081](https://github.com/windmill-labs/windmill/issues/7081)) ([ad43680](https://github.com/windmill-labs/windmill/commit/ad43680bb979f80db3bdb182ccb5bc61ca385ceb))
* modal action in App AgGrid ([#7085](https://github.com/windmill-labs/windmill/issues/7085)) ([b07c35f](https://github.com/windmill-labs/windmill/commit/b07c35fe1c9dbc4955ce47766ef92bd5cfdac4c8))
## [1.573.4](https://github.com/windmill-labs/windmill/compare/v1.573.3...v1.573.4) (2025-11-07)
### Bug Fixes
* **bun:** support relative imports without .ts but starting with /u/ or /f/ ([9e3ea76](https://github.com/windmill-labs/windmill/commit/9e3ea7657568d6561abcdfd23cc8a210432e4050))
* create git branch right before creating the workspace fork to catch errors and have a coherent fork point ([#7073](https://github.com/windmill-labs/windmill/issues/7073)) ([192fecc](https://github.com/windmill-labs/windmill/commit/192fecc86f54cf2a69ec7dddc466139b40933ac9))
* improve priority for child jobs ran from api ([5f7e0b7](https://github.com/windmill-labs/windmill/commit/5f7e0b7244e27533d9a8d1551e8affeb17eb94c0))
* make it impossible to have non linear script history even in edge condition ([3bda6da](https://github.com/windmill-labs/windmill/commit/3bda6da1683589180cad0780dd87b391a04851f8))
## [1.573.3](https://github.com/windmill-labs/windmill/compare/v1.573.2...v1.573.3) (2025-11-06)
### Bug Fixes
* job streaming improvement after compaction ([f16231d](https://github.com/windmill-labs/windmill/commit/f16231d7c9837efa3873034498abc9bfa87575d5))
## [1.573.2](https://github.com/windmill-labs/windmill/compare/v1.573.1...v1.573.2) (2025-11-06)
### Bug Fixes
* authentik frontend baseurl field empty when loaded from db ([#7065](https://github.com/windmill-labs/windmill/issues/7065)) ([8cb8650](https://github.com/windmill-labs/windmill/commit/8cb8650460856a1975c6eee429a2771f784a1555))
* oidc token issue ([75e056b](https://github.com/windmill-labs/windmill/commit/75e056bbce7e00334a36175eed00eb8eba332fbc))
## [1.573.1](https://github.com/windmill-labs/windmill/compare/v1.573.0...v1.573.1) (2025-11-05)
### Bug Fixes
* nits hub search and telemetry ([#7063](https://github.com/windmill-labs/windmill/issues/7063)) ([3338a3e](https://github.com/windmill-labs/windmill/commit/3338a3e3c48cbf05c48f0af6754b4c536bcb25f6))
## [1.573.0](https://github.com/windmill-labs/windmill/compare/v1.572.2...v1.573.0) (2025-11-05)
### Features
* add groups to user endpoint + set displayname when group created via igroup api ([#7061](https://github.com/windmill-labs/windmill/issues/7061)) ([8667024](https://github.com/windmill-labs/windmill/commit/86670240a773792f4e84cb15af476b9cd2e79c07))
* hub actions and better search ([#7056](https://github.com/windmill-labs/windmill/issues/7056)) ([1edfdce](https://github.com/windmill-labs/windmill/commit/1edfdce0ddd521819e4cca9dd2e57a20757ad400))
### Bug Fixes
* incorrect query arg in hub link ([2303f2c](https://github.com/windmill-labs/windmill/commit/2303f2cca0f65bdf69f246cdad2e0fe0bedb3e6e))
* **lsp:** fix ruff integration ([0271a9f](https://github.com/windmill-labs/windmill/commit/0271a9f51dfdbda950600d41f59ea18c0dafe7a0))
## [1.572.2](https://github.com/windmill-labs/windmill/compare/v1.572.1...v1.572.2) (2025-11-04)
### Bug Fixes
* **aiagent:** force structured tool usage for claude ([#7052](https://github.com/windmill-labs/windmill/issues/7052)) ([5ed41c6](https://github.com/windmill-labs/windmill/commit/5ed41c6d132a55713eadae26371a8f8080af1b6d))
* redirect first time user ([#7057](https://github.com/windmill-labs/windmill/issues/7057)) ([2add278](https://github.com/windmill-labs/windmill/commit/2add2785640c18db65e6642f37e210577f0fb056))
## [1.572.1](https://github.com/windmill-labs/windmill/compare/v1.572.0...v1.572.1) (2025-11-04)
### Bug Fixes
* add workspace error handler cache for improved performance ([4a849ca](https://github.com/windmill-labs/windmill/commit/4a849ca9b96e981bbace020fac7ad3123f330ad3))
* pass whitelist env vars to bun install ([#7047](https://github.com/windmill-labs/windmill/issues/7047)) ([4220582](https://github.com/windmill-labs/windmill/commit/4220582daf23a5367d534c98fb48f0f54ebaab20))
* preprocessor schema type ([#7049](https://github.com/windmill-labs/windmill/issues/7049)) ([0ae27a3](https://github.com/windmill-labs/windmill/commit/0ae27a3fe88d6aac046137220aab23e8c0d21590))
* **ruby:** propagate error correctly ([#7046](https://github.com/windmill-labs/windmill/issues/7046)) ([6052714](https://github.com/windmill-labs/windmill/commit/605271483365d9a9f59e7196b7364200ad14d33a))
## [1.572.0](https://github.com/windmill-labs/windmill/compare/v1.571.0...v1.572.0) (2025-11-03)
### Features
* **flow:** Add graph diff visualizer ([#6948](https://github.com/windmill-labs/windmill/issues/6948)) ([04d2ef4](https://github.com/windmill-labs/windmill/commit/04d2ef419dfe64c99da55ecead545cb1cc5cf185))
### Bug Fixes
* **backend:** add 404 error when not found in resource delete endpoints ([#7036](https://github.com/windmill-labs/windmill/issues/7036)) ([aaadf60](https://github.com/windmill-labs/windmill/commit/aaadf60d8f975df66bb9837d8fe21eaeaaf2e70b))
* consider duckdb as a normal tag ([#7035](https://github.com/windmill-labs/windmill/issues/7035)) ([16317a4](https://github.com/windmill-labs/windmill/commit/16317a471446e444ada4a67d56be36b146c4dfcf))
* fix rebuild_dependency_map ([#7026](https://github.com/windmill-labs/windmill/issues/7026)) ([f661caf](https://github.com/windmill-labs/windmill/commit/f661caf2b1bb9cb9a2f388684f94cb3055291da5))
* include missing tags from default/native consts ([#7034](https://github.com/windmill-labs/windmill/issues/7034)) ([c04489c](https://github.com/windmill-labs/windmill/commit/c04489c463f6b277452b7480659fa65cef1ab1ba))
### Performance Improvements
* parse flow value only if needed ([#7025](https://github.com/windmill-labs/windmill/issues/7025)) ([b5e341f](https://github.com/windmill-labs/windmill/commit/b5e341fde79d6335c0e27d0adbe0febd49a09d36))
## [1.571.0](https://github.com/windmill-labs/windmill/compare/v1.570.0...v1.571.0) (2025-11-01)
backend/.sqlx/query-0084c1246d1391d106da2e67a394eafc6695257632406ed9a2111dba1dd106c7.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\" FROM v2_job WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "args: sqlx::types::Json<HashMap<String, Box<RawValue>>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
true
]
},
"hash": "0084c1246d1391d106da2e67a394eafc6695257632406ed9a2111dba1dd106c7"
}
backend/.sqlx/query-0186c1058f147e012b8120c342caf8688a6d1643747be3ec4f784c3029a59e52.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT j.id\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE r.ping < now() - ($1 || ' seconds')::interval\n AND q.running = true AND j.kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow') AND j.same_worker = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "0186c1058f147e012b8120c342caf8688a6d1643747be3ec4f784c3029a59e52"
}
backend/.sqlx/query-0c0f3909b80c35210fc64c685905308621f9135c2c45a2fa0531ea750387da1f.json
Generated
+25
View File
@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT \n CASE \n WHEN flow_version.id IS NOT NULL THEN\n (flow_version.value -> 'flow_env' -> $3) #> $4\n ELSE\n (root_job.raw_flow -> 'flow_env' -> $3) #> $4\n END AS \"flow_env: sqlx::types::Json<Box<RawValue>>\"\n FROM \n v2_job current_job\n JOIN \n v2_job root_job ON root_job.id = COALESCE(current_job.root_job, current_job.flow_innermost_root_job, current_job.parent_job, current_job.id)\n AND root_job.workspace_id = current_job.workspace_id\n LEFT JOIN\n flow_version ON flow_version.id = root_job.runnable_id\n AND flow_version.path = root_job.runnable_path\n AND flow_version.workspace_id = root_job.workspace_id\n WHERE \n current_job.id = $1 AND \n current_job.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "flow_env: sqlx::types::Json<Box<RawValue>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid",
"Text",
"Text",
"TextArray"
]
},
"nullable": [
null
]
},
"hash": "0c0f3909b80c35210fc64c685905308621f9135c2c45a2fa0531ea750387da1f"
}
backend/.sqlx/query-1d9498226b3d962688558d8ab77f88de5dc8ae5321fa7ff0c7632e628556f991.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM resource_type WHERE name = $1 AND workspace_id = $2 RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "1d9498226b3d962688558d8ab77f88de5dc8ae5321fa7ff0c7632e628556f991"
}
backend/.sqlx/query-1fc43faf3a7ea15be82b1666273836901cc6ba17c7d5e194b50b9204f264de27.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT 1 FROM script WHERE hash = $1 AND workspace_id = $2 FOR UPDATE",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Int8",
"Text"
]
},
"nullable": [
null
]
},
"hash": "1fc43faf3a7ea15be82b1666273836901cc6ba17c7d5e194b50b9204f264de27"
}
backend/.sqlx/query-acc0b67c8e768b524b5cfb309e4307daeb0c095e07063c57b26ab94211bf6359.json → backend/.sqlx/query-24d302b8215d49a289bedd14a5791e9366d1f6d3d2aa485e0f50c6f2d85693dd.json
Generated
+3 -3
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id AS \"id!\" FROM v2_as_queue WHERE id = ANY($1) AND schedule_path IS NULL AND ($2::text[] IS NULL OR tag = ANY($2))",
"query": "SELECT j.id AS \"id!\" FROM v2_job j WHERE j.id = ANY($1) AND j.trigger_kind != 'schedule'::job_trigger_kind AND ($2::text[] IS NULL OR j.tag = ANY($2))",
"describe": {
"columns": [
{
@@ -16,8 +16,8 @@
]
},
"nullable": [
true
false
]
},
"hash": "acc0b67c8e768b524b5cfb309e4307daeb0c095e07063c57b26ab94211bf6359"
"hash": "24d302b8215d49a289bedd14a5791e9366d1f6d3d2aa485e0f50c6f2d85693dd"
}
backend/.sqlx/query-255415e4b1e891ce43b92ebde534f116c4ea41ccd1bc994d59c1cd56aae71334.json
Generated
-23
View File
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT f.value as \"value: Json<FlowValue>\" FROM v2_job j JOIN flow_version f ON f.id = j.runnable_id WHERE j.id = $1 AND j.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value: Json<FlowValue>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "255415e4b1e891ce43b92ebde534f116c4ea41ccd1bc994d59c1cd56aae71334"
}
backend/.sqlx/query-25cba74bec5959e6752265cd7b6f84846f74d468f0073f02f81122895e86c364.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT COALESCE(s.flow_status, s.workflow_as_code_status)->'user_states'->$1\n FROM v2_job_queue q LEFT JOIN v2_job_status s USING (id)\n WHERE q.id = $2 AND q.workspace_id = $3\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Uuid",
"Text"
]
},
"nullable": [
null
]
},
"hash": "25cba74bec5959e6752265cd7b6f84846f74d468f0073f02f81122895e86c364"
}
backend/.sqlx/query-2729c73b53605908e2fa26b4e4bbbd75d6d5bb77e6a92bf187d978a059d7af4a.json
Generated
+20
View File
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM v2_job_queue q WHERE q.canceled_by IS NULL AND (q.scheduled_for <= now()\n OR (q.suspend_until IS NOT NULL\n AND (q.suspend <= 0 OR q.suspend_until <= now())))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "2729c73b53605908e2fa26b4e4bbbd75d6d5bb77e6a92bf187d978a059d7af4a"
}
backend/.sqlx/query-280a361076d1c6317610765960f543252891c53351bdc98da66cc30ffc895866.json
Generated
-22
View File
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT script_path FROM v2_as_completed_job WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "script_path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
true
]
},
"hash": "280a361076d1c6317610765960f543252891c53351bdc98da66cc30ffc895866"
}
backend/.sqlx/query-2bfa1ffb3d5869fc3038049ba77890203332e398c865c47aaf019dd5721d59f9.json → backend/.sqlx/query-29785f22ee7092e8cebeae3757caf0563e3f98fff1f100616c33e8dc95fbff99.json
Generated
+3 -3
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by AS \"created_by!\", args as \"args: sqlx::types::Json<Box<RawValue>>\"\n FROM v2_as_completed_job\n WHERE id = $1 AND workspace_id = $2 AND ($3::text[] IS NULL OR tag = ANY($3))",
"query": "SELECT j.created_by AS \"created_by!\", j.args as \"args: sqlx::types::Json<Box<RawValue>>\"\n FROM v2_job j\n WHERE j.id = $1 AND j.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
"describe": {
"columns": [
{
@@ -22,9 +22,9 @@
]
},
"nullable": [
true,
false,
true
]
},
"hash": "2bfa1ffb3d5869fc3038049ba77890203332e398c865c47aaf019dd5721d59f9"
"hash": "29785f22ee7092e8cebeae3757caf0563e3f98fff1f100616c33e8dc95fbff99"
}
backend/.sqlx/query-2ae44acae7ac80e191b37071baa26e57488489177fe596940d3698d843a93666.json → backend/.sqlx/query-371d652fbb1d34d56f645c75d12852ff982821ed7b24574cdd09a06eac0d628b.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT script_path FROM v2_as_queue WHERE id = $1",
"query": "SELECT runnable_path as script_path FROM v2_job WHERE id = $1",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
true
]
},
"hash": "2ae44acae7ac80e191b37071baa26e57488489177fe596940d3698d843a93666"
"hash": "371d652fbb1d34d56f645c75d12852ff982821ed7b24574cdd09a06eac0d628b"
}
backend/.sqlx/query-2ea447f9e644554d415367b91042687ee8690d475b8ed31c48e31180689a278f.json → backend/.sqlx/query-37285436c16684449b33810d97d0a2611dab30faff2891fc3a7f00ee8c120950.json
Generated
+3 -3
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by AS \"created_by!\", CONCAT(coalesce(v2_as_queue.logs, ''), coalesce(job_logs.logs, '')) as logs, coalesce(job_logs.log_offset, 0) as log_offset, job_logs.log_file_index\n FROM v2_as_queue\n LEFT JOIN job_logs ON job_logs.job_id = v2_as_queue.id\n WHERE v2_as_queue.id = $1 AND v2_as_queue.workspace_id = $2 AND ($3::text[] IS NULL OR v2_as_queue.tag = ANY($3))",
"query": "SELECT j.created_by AS \"created_by!\", CONCAT(coalesce(job_logs.logs, '')) as logs, coalesce(job_logs.log_offset, 0) as log_offset, job_logs.log_file_index\n FROM v2_job j\n LEFT JOIN job_logs ON job_logs.job_id = j.id\n WHERE j.id = $1 AND j.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
"describe": {
"columns": [
{
@@ -32,11 +32,11 @@
]
},
"nullable": [
true,
false,
null,
null,
true
]
},
"hash": "2ea447f9e644554d415367b91042687ee8690d475b8ed31c48e31180689a278f"
"hash": "37285436c16684449b33810d97d0a2611dab30faff2891fc3a7f00ee8c120950"
}
backend/.sqlx/query-38baa6cf7d1c2532d38486a01e6d27ac1a58540a51e97dec5613e9bc0791e890.json
Generated
+17
View File
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO ai_agent_memory (workspace_id, conversation_id, step_id, messages, created_at, updated_at)\n VALUES ($1, $2, $3, $4, NOW(), NOW())\n ON CONFLICT (workspace_id, conversation_id, step_id)\n DO UPDATE SET\n messages = EXCLUDED.messages,\n updated_at = NOW()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Uuid",
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "38baa6cf7d1c2532d38486a01e6d27ac1a58540a51e97dec5613e9bc0791e890"
}
backend/.sqlx/query-3d38720e807b379645d8f3ab61c6a968143d42c3014152608f7d1b252cd8085c.json
Generated
-22
View File
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version\n (app_id, value, created_by, raw_app)\n SELECT app_id, value, created_by, raw_app\n FROM app_version WHERE id = $1\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "3d38720e807b379645d8f3ab61c6a968143d42c3014152608f7d1b252cd8085c"
}
backend/.sqlx/query-4545e1e0953cde730272353e78044744910a17af23f8201d853f0a4f8404fc73.json
Generated
+29
View File
@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT j.tag as \"tag!\", COUNT(*) as \"count!\"\n FROM v2_job_completed c JOIN v2_job j USING (id)\n WHERE c.started_at > NOW() - make_interval(secs => $1) AND ($2::text IS NULL OR j.workspace_id = $2)\n GROUP BY j.tag\n ORDER BY \"count!\" DESC\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "tag!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Float8",
"Text"
]
},
"nullable": [
false,
null
]
},
"hash": "4545e1e0953cde730272353e78044744910a17af23f8201d853f0a4f8404fc73"
}
backend/.sqlx/query-7af1cf089022fc1c3597b270b69aa669a153ab0c0bb2807cd4f7fd405afa6f69.json → backend/.sqlx/query-48a41e1ad0ad8fc51624af4de34964c72fe4b3cf2497644d514c5c00d6d71bf3.json
Generated
+4 -4
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n script_path, script_hash AS \"script_hash: ScriptHash\",\n job_kind AS \"job_kind!: JobKind\",\n flow_status AS \"flow_status: Json<Box<RawValue>>\",\n raw_flow AS \"raw_flow: Json<Box<RawValue>>\"\n FROM v2_as_completed_job WHERE id = $1 and workspace_id = $2",
"query": "SELECT\n j.runnable_path as script_path, j.runnable_id AS \"script_hash: ScriptHash\",\n j.kind AS \"job_kind!: JobKind\",\n COALESCE(c.flow_status, c.workflow_as_code_status) AS \"flow_status: Json<Box<RawValue>>\",\n j.raw_flow AS \"raw_flow: Json<Box<RawValue>>\"\n FROM v2_job_completed c JOIN v2_job j USING (id) WHERE j.id = $1 and j.workspace_id = $2",
"describe": {
"columns": [
{
@@ -65,10 +65,10 @@
"nullable": [
true,
true,
true,
true,
false,
null,
true
]
},
"hash": "7af1cf089022fc1c3597b270b69aa669a153ab0c0bb2807cd4f7fd405afa6f69"
"hash": "48a41e1ad0ad8fc51624af4de34964c72fe4b3cf2497644d514c5c00d6d71bf3"
}
backend/.sqlx/query-2456fc71fc7a0758a4c1fbe77d72fbac2fead0e1bff4e909fd7fb1a41bc35d8f.json → backend/.sqlx/query-49f7e7481019d27b44d5c8a167f9d5d3309a31b195b3147fa32f2ca8a6c9c90e.json
Generated
+3 -3
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n script_path, args AS \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n tag AS \"tag!\", priority\n FROM v2_as_completed_job\n WHERE id = $1 and workspace_id = $2",
"query": "SELECT\n j.runnable_path as script_path, j.args AS \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n j.tag AS \"tag!\", j.priority\n FROM v2_job j\n WHERE j.id = $1 and j.workspace_id = $2",
"describe": {
"columns": [
{
@@ -33,9 +33,9 @@
"nullable": [
true,
true,
true,
false,
true
]
},
"hash": "2456fc71fc7a0758a4c1fbe77d72fbac2fead0e1bff4e909fd7fb1a41bc35d8f"
"hash": "49f7e7481019d27b44d5c8a167f9d5d3309a31b195b3147fa32f2ca8a6c9c90e"
}
backend/.sqlx/query-519f4f76649947f036a2129c11e92ef0ad30e39eec59c27bae8cb0622062c8fb.json
Generated
-20
View File
@@ -1,20 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM v2_as_queue WHERE canceled = false AND (scheduled_for <= now()\n OR (suspend_until IS NOT NULL\n AND ( suspend <= 0\n OR suspend_until <= now())))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "519f4f76649947f036a2129c11e92ef0ad30e39eec59c27bae8cb0622062c8fb"
}
backend/.sqlx/query-528cdbb75f1c5135170a58fce3fda464be138272487639d0ffbbbe6961ec5c37.json
Generated
-18
View File
@@ -1,18 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_logs SET logs = $1, log_offset = $2, \n log_file_index = array_append(coalesce(log_file_index, array[]::text[]), $3) \n WHERE workspace_id = $4 AND job_id = $5",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Int4",
"Text",
"Text",
"Uuid"
]
},
"nullable": []
},
"hash": "528cdbb75f1c5135170a58fce3fda464be138272487639d0ffbbbe6961ec5c37"
}
backend/.sqlx/query-0cb0e912bc942af2b1ef784455f3f073a79e300f3dd48f14122d1782eee663cd.json → backend/.sqlx/query-57bd445bfdc667089f847acb9dc484028ce8a077308979a5ba8ef252e19aa825.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT coalesce(COUNT(*) FILTER(WHERE suspend = 0 AND running = false), 0) as \"database_length!\", coalesce(COUNT(*) FILTER(WHERE suspend > 0), 0) as \"suspended!\" FROM v2_as_queue WHERE (workspace_id = $1 OR $2) AND scheduled_for <= now() AND ($3::text[] IS NULL OR tag = ANY($3))",
"query": "SELECT coalesce(COUNT(*) FILTER(WHERE q.suspend = 0 AND q.running = false), 0) as \"database_length!\", coalesce(COUNT(*) FILTER(WHERE q.suspend > 0), 0) as \"suspended!\" FROM v2_job_queue q JOIN v2_job j USING (id) WHERE (j.workspace_id = $1 OR $2) AND q.scheduled_for <= now() AND ($3::text[] IS NULL OR j.tag = ANY($3))",
"describe": {
"columns": [
{
@@ -26,5 +26,5 @@
null
]
},
"hash": "0cb0e912bc942af2b1ef784455f3f073a79e300f3dd48f14122d1782eee663cd"
"hash": "57bd445bfdc667089f847acb9dc484028ce8a077308979a5ba8ef252e19aa825"
}
backend/.sqlx/query-ca5d9a9d8d18da970c7fd6eab41ecbb3a5c7c29803e4c38b8a0b2ca3790e52f9.json → backend/.sqlx/query-5e7cadffbee74b11e224b60322b102b9899b4a97b8e557692c0085a9b472b8a7.json
Generated
+5 -5
View File
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by AS \"created_by!\", CONCAT(coalesce(v2_as_completed_job.logs, ''), coalesce(job_logs.logs, '')) as logs, job_logs.log_offset, job_logs.log_file_index\n FROM v2_as_completed_job\n LEFT JOIN job_logs ON job_logs.job_id = v2_as_completed_job.id\n WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2 AND ($3::text[] IS NULL OR v2_as_completed_job.tag = ANY($3))",
"query": "SELECT j.created_by AS \"created_by\", coalesce(job_logs.logs, '') as logs, COALESCE(job_logs.log_offset, 0) as log_offset, job_logs.log_file_index\n FROM v2_job j\n LEFT JOIN job_logs ON job_logs.job_id = j.id\n WHERE j.id = $1 AND j.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "created_by!",
"name": "created_by",
"type_info": "Varchar"
},
{
@@ -32,11 +32,11 @@
]
},
"nullable": [
true,
null,
false,
null,
null,
true
]
},
"hash": "ca5d9a9d8d18da970c7fd6eab41ecbb3a5c7c29803e4c38b8a0b2ca3790e52f9"
"hash": "5e7cadffbee74b11e224b60322b102b9899b4a97b8e557692c0085a9b472b8a7"
}
backend/.sqlx/query-f3571e1d2b57011e5f6a38725eb42d909456d28a98563923cca43e760862e5e0.json → backend/.sqlx/query-6a1005b8ae5326c5d5955534e9228b6bd524253971ae09cbe100c0d6d42e6cdc.json
Generated
+4 -4
View File
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT flow_status->'user_states'->$1\n FROM v2_as_queue\n WHERE id = $2 AND workspace_id = $3\n ",
"query": "SELECT messages FROM ai_agent_memory\n WHERE workspace_id = $1 AND conversation_id = $2 AND step_id = $3",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"name": "messages",
"type_info": "Jsonb"
}
],
@@ -17,8 +17,8 @@
]
},
"nullable": [
null
false
]
},
"hash": "f3571e1d2b57011e5f6a38725eb42d909456d28a98563923cca43e760862e5e0"
"hash": "6a1005b8ae5326c5d5955534e9228b6bd524253971ae09cbe100c0d6d42e6cdc"
}
backend/.sqlx/query-d4d83d8177144c91aa489b5a42a45c83f8b069a52f681f14afb4931ac77baf45.json → backend/.sqlx/query-6b9ea0059ad1037a77f67b70d5de13776870612e8faa8d8fa577f8e7b7309f76.json
Generated
+5 -5
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT id AS \"id!\", flow_status, suspend AS \"suspend!\", script_path\n FROM v2_as_queue\n WHERE id = $1\n ",
"query": "\n SELECT j.id AS \"id!\", COALESCE(s.flow_status, s.workflow_as_code_status) as flow_status, q.suspend AS \"suspend!\", j.runnable_path as script_path\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1\n ",
"describe": {
"columns": [
{
@@ -30,11 +30,11 @@
]
},
"nullable": [
true,
true,
true,
false,
null,
false,
true
]
},
"hash": "d4d83d8177144c91aa489b5a42a45c83f8b069a52f681f14afb4931ac77baf45"
"hash": "6b9ea0059ad1037a77f67b70d5de13776870612e8faa8d8fa577f8e7b7309f76"
}
backend/.sqlx/query-6cc922a5bbd348c938a9d1431aaa0f24f078ea814b429d44403aca1e5002e750.json
Generated
+216
View File
@@ -0,0 +1,216 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"name": "started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 5,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 6,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 7,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 8,
"name": "kind!: JobKind",
"type_info": {
"Custom": {
"name": "job_kind",
"kind": {
"Enum": [
"script",
"preview",
"flow",
"dependencies",
"flowpreview",
"script_hub",
"identity",
"flowdependencies",
"http",
"graphql",
"postgresql",
"noop",
"appdependencies",
"deploymentcallback",
"singlestepflow",
"flowscript",
"flownode",
"appscript",
"aiagent"
]
}
}
}
},
{
"ordinal": 9,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 10,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 11,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby"
]
}
}
}
},
{
"ordinal": 12,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 13,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 14,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
"name": "job_trigger_kind",
"kind": {
"Enum": [
"webhook",
"http",
"websocket",
"kafka",
"email",
"nats",
"schedule",
"app",
"ui",
"postgres",
"sqs",
"gcp",
"mqtt"
]
}
}
}
},
{
"ordinal": 15,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 16,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 17,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 18,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 19,
"name": "cache_ttl",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false,
false,
true,
false,
true,
true,
true,
true,
false,
false,
false,
true,
false,
true,
true,
true,
true,
true,
false,
true
]
},
"hash": "6cc922a5bbd348c938a9d1431aaa0f24f078ea814b429d44403aca1e5002e750"
}
backend/.sqlx/query-6ec1e81e1698c754ca8a660cf6a3ae66e75b487901befd647839dd2bc2233bf4.json
Generated
+16
View File
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_logs SET logs = $1 WHERE workspace_id = $2 AND job_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Uuid"
]
},
"nullable": []
},
"hash": "6ec1e81e1698c754ca8a660cf6a3ae66e75b487901befd647839dd2bc2233bf4"
}
backend/.sqlx/query-e8e33f599eae064011232f9f715e676d0c8ae31982865cb2c8103ed735c42c69.json → backend/.sqlx/query-6f49a47b0b77720b0e8a2ddbd46e45e414dd646c912387ee63609d9f5611690f.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO deployment_metadata (workspace_id, path, flow_version, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, path, flow_version) WHERE flow_version IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"query": "INSERT INTO deployment_metadata (workspace_id, path, flow_version, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, path, flow_version) WHERE flow_version IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"describe": {
"columns": [],
"parameters": {
@@ -14,5 +14,5 @@
},
"nullable": []
},
"hash": "e8e33f599eae064011232f9f715e676d0c8ae31982865cb2c8103ed735c42c69"
"hash": "6f49a47b0b77720b0e8a2ddbd46e45e414dd646c912387ee63609d9f5611690f"
}
backend/.sqlx/query-70a6880960d17218bc5bf05287e2a6d9a6393c6bb1783ab8903d87dd099e236b.json
Generated
+58
View File
@@ -0,0 +1,58 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n j.id AS \"id!\", j.workspace_id AS \"workspace_id!\", j.parent_job, j.flow_step_id IS NOT NULL AS \"is_flow_step?\",\n COALESCE(s.flow_status, s.workflow_as_code_status) AS \"flow_status: Box<str>\", r.ping AS last_ping, j.same_worker AS \"same_worker?\"\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE q.running = true AND q.suspend = 0 AND q.suspend_until IS null AND q.scheduled_for <= now()\n AND (j.kind = 'flow' OR j.kind = 'flowpreview' OR j.kind = 'flownode')\n AND r.ping IS NOT NULL AND r.ping < NOW() - ($1 || ' seconds')::interval\n AND q.canceled_by IS NULL\n \n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "workspace_id!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 3,
"name": "is_flow_step?",
"type_info": "Bool"
},
{
"ordinal": 4,
"name": "flow_status: Box<str>",
"type_info": "Jsonb"
},
{
"ordinal": 5,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 6,
"name": "same_worker?",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false,
true,
null,
null,
true,
false
]
},
"hash": "70a6880960d17218bc5bf05287e2a6d9a6393c6bb1783ab8903d87dd099e236b"
}
backend/.sqlx/query-72956f508f66312807738399b57aa01a048fa4f9281327cf5b78111178424b43.json
Generated
-58
View File
@@ -1,58 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n id AS \"id!\", workspace_id AS \"workspace_id!\", parent_job, is_flow_step,\n flow_status AS \"flow_status: Box<str>\", last_ping, same_worker\n FROM v2_as_queue\n WHERE running = true AND suspend = 0 AND suspend_until IS null AND scheduled_for <= now()\n AND (job_kind = 'flow' OR job_kind = 'flowpreview' OR job_kind = 'flownode')\n AND last_ping IS NOT NULL AND last_ping < NOW() - ($1 || ' seconds')::interval\n AND canceled = false\n \n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "workspace_id!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 3,
"name": "is_flow_step",
"type_info": "Bool"
},
{
"ordinal": 4,
"name": "flow_status: Box<str>",
"type_info": "Jsonb"
},
{
"ordinal": 5,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 6,
"name": "same_worker",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true,
true,
true,
true,
true,
true,
true
]
},
"hash": "72956f508f66312807738399b57aa01a048fa4f9281327cf5b78111178424b43"
}
backend/.sqlx/query-6ab112fa42a9ae332bfa30427b70fa742351c5c180ac3de106df54f7badb494c.json → backend/.sqlx/query-7a4e79bcae7c7d69e4bb71c8540b2adb9ca8bd15098038e3ccf8b16164944951.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(id) FROM v2_as_queue WHERE email = $1",
"query": "SELECT COUNT(j.id) FROM v2_job_queue q JOIN v2_job j USING (id) WHERE j.permissioned_as_email = $1",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
null
]
},
"hash": "6ab112fa42a9ae332bfa30427b70fa742351c5c180ac3de106df54f7badb494c"
"hash": "7a4e79bcae7c7d69e4bb71c8540b2adb9ca8bd15098038e3ccf8b16164944951"
}
backend/.sqlx/query-83232f2db5eb1b6fef744998e60420ef920d472286cf4c1f78452446a4bcb604.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version\n (app_id, value, created_by, raw_app)\n SELECT app_id, value, created_by, raw_app\n FROM app_version WHERE id = $1\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "83232f2db5eb1b6fef744998e60420ef920d472286cf4c1f78452446a4bcb604"
}
backend/.sqlx/query-85ec6c6384ab77df102d05caf82079e65a5300a52a292b9143b755087b9f9c4c.json
Generated
+23
View File
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM resource WHERE path = $1 AND workspace_id = $2 RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "85ec6c6384ab77df102d05caf82079e65a5300a52a292b9143b755087b9f9c4c"
}
backend/.sqlx/query-89940a53f29b173b6a8717f057a81a4f2edad152581950fdd80d758a0d242c17.json
Generated
-28
View File
@@ -1,28 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT usr.email, usage.executions\n FROM usr\n , LATERAL (\n SELECT COALESCE(SUM(duration_ms + 1000)/1000 , 0)::BIGINT executions\n FROM v2_as_completed_job\n WHERE workspace_id = $1\n AND job_kind NOT IN ('flow', 'flowpreview', 'flownode')\n AND email = usr.email\n AND now() - '1 week'::interval < created_at\n ) usage\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "executions",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
null
]
},
"hash": "89940a53f29b173b6a8717f057a81a4f2edad152581950fdd80d758a0d242c17"
}
backend/.sqlx/query-2e131a019051bdac7c9c65f7c504cfba31cfdd64a0f68001e123694ed5cde5ed.json → backend/.sqlx/query-8e750d4b3af9b5844c11b1b92741f2ee9d2d3412d4a2c96c6ccc87ec1c382384.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5) \n ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5) \n ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"describe": {
"columns": [],
"parameters": {
@@ -14,5 +14,5 @@
},
"nullable": []
},
"hash": "2e131a019051bdac7c9c65f7c504cfba31cfdd64a0f68001e123694ed5cde5ed"
"hash": "8e750d4b3af9b5844c11b1b92741f2ee9d2d3412d4a2c96c6ccc87ec1c382384"
}
backend/.sqlx/query-90fbb9430ab03ce3aadd95cc263e5a3d1a91ea02de7608676575e1c03023ed71.json
Generated
-29
View File
@@ -1,29 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT tag as \"tag!\", COUNT(*) as \"count!\"\n FROM v2_as_completed_job\n WHERE started_at > NOW() - make_interval(secs => $1) AND ($2::text IS NULL OR workspace_id = $2)\n GROUP BY tag\n ORDER BY \"count!\" DESC\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "tag!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Float8",
"Text"
]
},
"nullable": [
true,
null
]
},
"hash": "90fbb9430ab03ce3aadd95cc263e5a3d1a91ea02de7608676575e1c03023ed71"
}
backend/.sqlx/query-99c289a8bcf87588ecb89575f66e1fbfce74dd6b69a8a039714a02ce7558a1b3.json
Generated
+28
View File
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT instance_group.id, COALESCE(instance_group.scim_display_name, instance_group.name) as display_name\n FROM email_to_igroup\n JOIN instance_group ON instance_group.name = email_to_igroup.igroup\n WHERE email_to_igroup.email = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "display_name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true,
null
]
},
"hash": "99c289a8bcf87588ecb89575f66e1fbfce74dd6b69a8a039714a02ce7558a1b3"
}
backend/.sqlx/query-a0af7dc507778ab23d5b1615e2a03278de05f4cd90fb1c31d7281f8681c7f85d.json
Generated
+18
View File
@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_logs SET logs = $1, log_offset = $2,\n log_file_index = array_append(coalesce(log_file_index, array[]::text[]), $3)\n WHERE workspace_id = $4 AND job_id = $5",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Int4",
"Text",
"Text",
"Uuid"
]
},
"nullable": []
},
"hash": "a0af7dc507778ab23d5b1615e2a03278de05f4cd90fb1c31d7281f8681c7f85d"
}
backend/.sqlx/query-11db65c493990f6935103033b2fbb0c08ae6d91b05b2f3f7c89a990d1d5a5f8a.json → backend/.sqlx/query-a3d18ae5e5125940ae0d6af315e2dbb17c739415850e7c67f8575fb983c295fe.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(id) FROM v2_as_queue WHERE running = true AND email = $1",
"query": "SELECT COUNT(j.id) FROM v2_job_queue q JOIN v2_job j USING (id) WHERE q.running = true AND j.permissioned_as_email = $1",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
null
]
},
"hash": "11db65c493990f6935103033b2fbb0c08ae6d91b05b2f3f7c89a990d1d5a5f8a"
"hash": "a3d18ae5e5125940ae0d6af315e2dbb17c739415850e7c67f8575fb983c295fe"
}
backend/.sqlx/query-a6a973dcd92d2e40fd9a1c1be42052fcd350bd47ee4f63832448b6e6f0f472f0.json
Generated
-24
View File
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_version\n (workspace_id, path, value, schema, created_by)\n\n SELECT workspace_id, path, value, schema, created_by\n FROM flow_version WHERE path = $1 AND workspace_id = $2 AND id = $3\n\n RETURNING id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8"
]
},
"nullable": [
false
]
},
"hash": "a6a973dcd92d2e40fd9a1c1be42052fcd350bd47ee4f63832448b6e6f0f472f0"
}
backend/.sqlx/query-a72e7fb55d7268fe1ea40015a4a84b12dd961ba732772d8f6c59d18fe05f285d.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS (\n SELECT 1 FROM v2_job_completed c JOIN v2_job j USING (id)\n WHERE j.workspace_id = $2\n AND (j.kind = 'appscript' OR j.kind = 'preview')\n AND j.created_by = 'anonymous'\n AND c.started_at > now() - interval '3 hours'\n AND j.runnable_path LIKE $3 || '/%'\n AND c.result @> ('{\"s3\":\"' || $1 || '\"}')::jsonb\n )",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "a72e7fb55d7268fe1ea40015a4a84b12dd961ba732772d8f6c59d18fe05f285d"
}
backend/.sqlx/query-be00ac55e8668a0ed3befda7d8595c7cda0cba0b119d4fdb8a0dea1b28a1d560.json
Generated
+22
View File
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n (elem->>'installation_id')::bigint as installation_id\n FROM workspace_settings,\n LATERAL jsonb_array_elements(git_app_installations) AS elem\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "installation_id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "be00ac55e8668a0ed3befda7d8595c7cda0cba0b119d4fdb8a0dea1b28a1d560"
}
backend/.sqlx/query-d1d9cb0a7043760564171f481dd73f04eaa2c7d5c1ddc0501bc70c3fb34a07be.json
Generated
+15
View File
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM ai_agent_memory\n WHERE workspace_id = $1 AND conversation_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Uuid"
]
},
"nullable": []
},
"hash": "d1d9cb0a7043760564171f481dd73f04eaa2c7d5c1ddc0501bc70c3fb34a07be"
}
backend/.sqlx/query-d48c9a748746080e9b6cf0366b2ca3516ac5c1d32a98946196b1a42e3f103efd.json
Generated
+239
View File
@@ -0,0 +1,239 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, q.workspace_id, j.runnable_id as \"runnable_id: ScriptHash\", scheduled_for, parent_job, flow_innermost_root_job, runnable_path, kind as \"kind: JobKind\", started_at, permissioned_as, created_by, script_lang as \"script_lang: ScriptLang\", \n permissioned_as_email, flow_step_id, trigger_kind as \"trigger_kind: JobTriggerKind\", trigger, q.priority, concurrent_limit, q.tag, cache_ttl, r.ping as last_ping, worker, memory_peak, running\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 5,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 6,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "kind: JobKind",
"type_info": {
"Custom": {
"name": "job_kind",
"kind": {
"Enum": [
"script",
"preview",
"flow",
"dependencies",
"flowpreview",
"script_hub",
"identity",
"flowdependencies",
"http",
"graphql",
"postgresql",
"noop",
"appdependencies",
"deploymentcallback",
"singlestepflow",
"flowscript",
"flownode",
"appscript",
"aiagent"
]
}
}
}
},
{
"ordinal": 8,
"name": "started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 9,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 10,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 11,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby"
]
}
}
}
},
{
"ordinal": 12,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 13,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 14,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
"name": "job_trigger_kind",
"kind": {
"Enum": [
"webhook",
"http",
"websocket",
"kafka",
"email",
"nats",
"schedule",
"app",
"ui",
"postgres",
"sqs",
"gcp",
"mqtt"
]
}
}
}
},
{
"ordinal": 15,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 16,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 17,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 18,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 19,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 20,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 21,
"name": "worker",
"type_info": "Varchar"
},
{
"ordinal": 22,
"name": "memory_peak",
"type_info": "Int4"
},
{
"ordinal": 23,
"name": "running",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false,
false,
true,
false,
true,
true,
true,
false,
true,
false,
false,
true,
false,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
false
]
},
"hash": "d48c9a748746080e9b6cf0366b2ca3516ac5c1d32a98946196b1a42e3f103efd"
}
backend/.sqlx/query-d5d97005eebcb2760216dbd10872acdb42868fd5f7a27f71836e7a6ff1c69856.json
Generated
-15
View File
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM resource_type WHERE name = $1 AND workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "d5d97005eebcb2760216dbd10872acdb42868fd5f7a27f71836e7a6ff1c69856"
}
backend/.sqlx/query-ec9980f80a8bfa4b09225035e8a1f78d7d61fbf83dda9b39bb22e0f9584d221b.json
Generated
-24
View File
@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS (\n SELECT 1 FROM v2_as_completed_job\n WHERE workspace_id = $2\n AND (job_kind = 'appscript' OR job_kind = 'preview')\n AND created_by = 'anonymous'\n AND started_at > now() - interval '3 hours'\n AND script_path LIKE $3 || '/%'\n AND result @> ('{\"s3\":\"' || $1 || '\"}')::jsonb\n )",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "ec9980f80a8bfa4b09225035e8a1f78d7d61fbf83dda9b39bb22e0f9584d221b"
}
backend/.sqlx/query-edd57b3d59ddc21b99212b5fabd4a8b793c4ae618a04220b3609c8c3c168f8fd.json
Generated
+28
View File
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT usr.email, usage.executions\n FROM usr, LATERAL (\n SELECT COALESCE(SUM(c.duration_ms + 1000)/1000 , 0)::BIGINT executions\n FROM v2_job_completed c JOIN v2_job j USING (id)\n WHERE j.workspace_id = $1\n AND j.kind NOT IN ('flow', 'flowpreview', 'flownode')\n AND j.permissioned_as_email = usr.email\n AND now() - '1 week'::interval < j.created_at\n ) usage\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "executions",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
null
]
},
"hash": "edd57b3d59ddc21b99212b5fabd4a8b793c4ae618a04220b3609c8c3c168f8fd"
}
backend/.sqlx/query-f0efa383f2025158de160577ad839ae72faf0c8fe097e6ad6d309aee9a8aede2.json
Generated
+24
View File
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_version\n (workspace_id, path, value, schema, created_by)\n\n SELECT workspace_id, path, value, schema, created_by\n FROM flow_version WHERE path = $1 AND workspace_id = $2 AND id = $3\n\n RETURNING id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8"
]
},
"nullable": [
false
]
},
"hash": "f0efa383f2025158de160577ad839ae72faf0c8fe097e6ad6d309aee9a8aede2"
}
backend/.sqlx/query-e572fa64eec9188368d7c271ac7ecd6b45dc161423abab22cddb2b13f6fb9833.json → backend/.sqlx/query-f844026597d2e1c59a80dc8e1ddd069c0ad82b27b41171825928a938cfa8316f.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO deployment_metadata (workspace_id, path, app_version, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, path, app_version) WHERE app_version IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"query": "INSERT INTO deployment_metadata (workspace_id, path, app_version, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, path, app_version) WHERE app_version IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"describe": {
"columns": [],
"parameters": {
@@ -14,5 +14,5 @@
},
"nullable": []
},
"hash": "e572fa64eec9188368d7c271ac7ecd6b45dc161423abab22cddb2b13f6fb9833"
"hash": "f844026597d2e1c59a80dc8e1ddd069c0ad82b27b41171825928a938cfa8316f"
}
backend/.sqlx/query-3f08ffbb5c71b873a9e164ecb0b10fffb37599f2a703885ee723cb9290fed13e.json → backend/.sqlx/query-fbd38c9c4f4ecba2d3e3d79433327fd4280466eec443aff4f1e8c1ab810cca7e.json
Generated
+2 -2
View File
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "WITH job_info AS (\n -- Query for Teams (running jobs)\n SELECT\n parent.job_kind AS \"job_kind!: JobKind\",\n parent.script_hash AS \"script_hash: ScriptHash\",\n parent.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n child.parent_job AS \"parent_job: Uuid\",\n parent.created_at AS \"created_at!: chrono::NaiveDateTime\",\n parent.created_by AS \"created_by!\",\n parent.script_path,\n parent.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n FROM v2_as_queue child\n JOIN v2_as_queue parent ON parent.id = child.parent_job\n WHERE child.id = $1 AND child.workspace_id = $2\n UNION ALL\n -- Query for Slack (completed jobs)\n SELECT\n v2_as_queue.job_kind AS \"job_kind!: JobKind\",\n v2_as_queue.script_hash AS \"script_hash: ScriptHash\",\n v2_as_queue.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n v2_as_completed_job.parent_job AS \"parent_job: Uuid\",\n v2_as_completed_job.created_at AS \"created_at!: chrono::NaiveDateTime\",\n v2_as_completed_job.created_by AS \"created_by!\",\n v2_as_queue.script_path,\n v2_as_queue.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n FROM v2_as_queue\n JOIN v2_as_completed_job ON v2_as_completed_job.parent_job = v2_as_queue.id\n WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2\n )\n SELECT * FROM job_info LIMIT 1",
"query": "WITH job_info AS (\n -- Query for Teams (running jobs)\n SELECT\n parent_j.kind AS \"job_kind!: JobKind\",\n parent_j.runnable_id AS \"script_hash: ScriptHash\",\n parent_j.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n child_j.parent_job AS \"parent_job: Uuid\",\n parent_j.created_at AS \"created_at!: chrono::NaiveDateTime\",\n parent_j.created_by AS \"created_by!\",\n parent_j.runnable_path as script_path,\n parent_j.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n FROM v2_job_queue child_q JOIN v2_job child_j USING (id)\n JOIN v2_job parent_j ON parent_j.id = child_j.parent_job\n WHERE child_j.id = $1 AND child_j.workspace_id = $2\n UNION ALL\n -- Query for Slack (completed jobs)\n SELECT\n parent_j.kind AS \"job_kind!: JobKind\",\n parent_j.runnable_id AS \"script_hash: ScriptHash\",\n parent_j.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n completed_j.parent_job AS \"parent_job: Uuid\",\n completed_j.created_at AS \"created_at!: chrono::NaiveDateTime\",\n completed_j.created_by AS \"created_by!\",\n parent_j.runnable_path as script_path,\n parent_j.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n FROM v2_job_completed completed_c JOIN v2_job completed_j USING (id)\n JOIN v2_job parent_j ON parent_j.id = completed_j.parent_job\n WHERE completed_j.id = $1 AND completed_j.workspace_id = $2\n )\n SELECT * FROM job_info LIMIT 1",
"describe": {
"columns": [
{
@@ -88,5 +88,5 @@
null
]
},
"hash": "3f08ffbb5c71b873a9e164ecb0b10fffb37599f2a703885ee723cb9290fed13e"
"hash": "fbd38c9c4f4ecba2d3e3d79433327fd4280466eec443aff4f1e8c1ab810cca7e"
}
backend/Cargo.lock
Generated
+216 -203
View File
File diff suppressed because it is too large Load Diff
backend/Cargo.toml
+3 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.571.0"
version = "1.575.0"
authors.workspace = true
edition.workspace = true
@@ -33,7 +33,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.571.0"
version = "1.575.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -252,6 +252,7 @@ urlencoding = "^2"
url = { version = "^2" , features = ["serde"]}
async-oauth2 = "0.5.1"
reqwest = { version = "^0.12", features = ["json", "stream", "gzip", "multipart"] }
eventsource-stream = "0.2.3"
time = "^0"
serde_urlencoded = "^0"
tokio-tar = "^0"
backend/ee-repo-ref.txt
+1 -1
View File
@@ -1 +1 @@
6a3c688d51480b764d780bb02368c4b1da56caff
7cdb62d968feef22046b12a1217b711af457cced
backend/migrations/20251104135208_ai_agent_memory.down.sql
+4
View File
@@ -0,0 +1,4 @@
-- Add down migration script here
-- Drop table
DROP TABLE IF EXISTS ai_agent_memory;
backend/migrations/20251104135208_ai_agent_memory.up.sql
+16
View File
@@ -0,0 +1,16 @@
-- Add up migration script here
-- Create ai_agent_memory table for storing AI agent step memory when S3 is unavailable
CREATE TABLE ai_agent_memory (
workspace_id VARCHAR(50) NOT NULL,
conversation_id UUID NOT NULL,
step_id VARCHAR(255) NOT NULL,
messages JSONB NOT NULL,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
PRIMARY KEY (workspace_id, conversation_id, step_id)
);
-- Grant permissions
GRANT ALL ON ai_agent_memory TO windmill_admin;
GRANT ALL ON ai_agent_memory TO windmill_user;
backend/migrations/20251105100125_legacy_sql_result_flag.down.sql
+1
View File
@@ -0,0 +1 @@
-- Add down migration script here
backend/migrations/20251105100125_legacy_sql_result_flag.up.sql
+99
View File
@@ -0,0 +1,99 @@
CREATE OR REPLACE FUNCTION update_string(s text)
RETURNS text
LANGUAGE plpgsql
AS $$
DECLARE
prefix TEXT := '-- https://www.windmill.dev/docs/getting_started/scripts_quickstart/sql#result-collection
-- result_collection=legacy
';
BEGIN
RETURN prefix || s;
END;
$$;
CREATE OR REPLACE FUNCTION update_all_modules(obj jsonb)
RETURNS jsonb
LANGUAGE plpgsql
AS $$
DECLARE
result jsonb;
k text;
v jsonb;
BEGIN
IF jsonb_typeof(obj) = 'object' THEN
result := '{}'::jsonb;
FOR k, v IN SELECT * FROM jsonb_each(obj)
LOOP
IF k = 'content' and jsonb_typeof(v) = 'string' AND obj->>'language' IN ('bigquery', 'postgresql', 'duckdb', 'mssql', 'oracledb', 'snowflake', 'mysql') THEN
result := result || jsonb_build_object('content', update_string(obj->>'content'));
ELSE
result := result || jsonb_build_object(k, update_all_modules(v));
END IF;
END LOOP;
RETURN result;
ELSIF jsonb_typeof(obj) = 'array' AND jsonb_array_length(obj) > 0 THEN
SELECT jsonb_agg(update_all_modules(elem))
INTO result
FROM jsonb_array_elements(obj) elem;
RETURN result;
ELSE
RETURN obj;
END IF;
END;
$$;
-- Run on a flow_version_lite jsonb value. Returns an array of flow_node ids whose languages are SQL.
CREATE OR REPLACE FUNCTION find_sql_flow_nodes_ids(obj jsonb)
RETURNS BIGINT[]
LANGUAGE plpgsql
AS $$
DECLARE
result BIGINT[] := '{}';
k text;
v jsonb;
BEGIN
IF jsonb_typeof(obj) = 'object' THEN
IF obj->>'language' IN ('bigquery', 'postgresql', 'duckdb', 'mssql', 'oracledb', 'snowflake', 'mysql') AND jsonb_typeof(obj->'id') = 'number' THEN
result := result || (obj->>'id')::BIGINT;
END IF;
FOR k, v IN SELECT * FROM jsonb_each(obj)
LOOP
result := result || find_sql_flow_nodes_ids(v);
END LOOP;
ELSIF jsonb_typeof(obj) = 'array' AND jsonb_array_length(obj) > 0 THEN
SELECT array_agg(result_ids)
INTO result
FROM jsonb_array_elements(obj) elem, unnest(find_sql_flow_nodes_ids(elem)) as result_ids;
END IF;
RETURN result;
END;
$$;
DO $$
BEGIN
UPDATE app_version SET value = update_all_modules(value::jsonb)::json;
UPDATE draft SET value = update_all_modules(value::jsonb)::json;
UPDATE flow SET value = update_all_modules(value);
UPDATE flow_version SET value = update_all_modules(value);
UPDATE flow_node SET code = update_string(code) WHERE id IN (
SELECT v FROM flow_version_lite, unnest(find_sql_flow_nodes_ids(value)) as v
);
UPDATE app_script SET code = update_string(code) WHERE id IN (
SELECT v FROM app_version_lite, unnest(find_sql_flow_nodes_ids(value)) as v
);
UPDATE script SET content = update_string(content) WHERE language IN ('bigquery', 'postgresql', 'duckdb', 'mssql', 'oracledb', 'snowflake', 'mysql');
EXCEPTION WHEN OTHERS THEN
-- ✅ LOG ERROR WITHOUT STOPPING THE MIGRATION
RAISE WARNING 'Migration failed: %', SQLERRM;
END;
$$;
DROP FUNCTION IF EXISTS update_all_modules(jsonb);
DROP FUNCTION IF EXISTS update_string(text);
DROP FUNCTION IF EXISTS find_sql_flow_nodes_ids(jsonb);
backend/parsers/windmill-parser-bash/src/lib.rs
+255 -4
View File
@@ -47,7 +47,6 @@ pub fn parse_powershell_sig(code: &str) -> anyhow::Result<MainArgSignature> {
lazy_static::lazy_static! {
static ref RE_BASH: Regex = Regex::new(r#"(?m)^(\w+)="\$(?:(\d+)|\{(\d+)\}|\{(\d+):-(.*)\})"(?:[\t ]*)?(?:#.*)?\r?$"#).unwrap();
pub static ref RE_POWERSHELL_PARAM: Regex = Regex::new(r#"(?m)param[\t ]*\(([^)]*)\)"#).unwrap();
static ref RE_POWERSHELL_ARGS: Regex = Regex::new(r#"(?:\[([\w\[\]]+)\])?\$(\w+)[\t ]*(?:=[\t ]*(?:(?:(?:"|')([^"\n\r\$]*)(?:"|'))|([\d.]+)))?\r?"#).unwrap();
}
@@ -85,6 +84,136 @@ fn parse_bash_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
Ok(Some(args))
}
/// Extract a PowerShell param() block, handling nested parentheses.
///
/// # Arguments
/// * `code` - The PowerShell code to extract from
/// * `include_keyword` - If true, returns the full block including "param(...)"
/// If false, returns only the contents between the parentheses
///
/// # Returns
/// The extracted param block or contents, or None if not found.
pub fn extract_powershell_param_block(code: &str, include_keyword: bool) -> Option<&str> {
// Find "param" keyword (case-insensitive)
let lower_code = code.to_lowercase();
let param_start = lower_code.find("param")?;
// Verify that only comments and whitespace appear before "param"
let before_param = &code[..param_start];
let mut chars = before_param.chars().peekable();
let mut in_block_comment = false;
while let Some(ch) = chars.next() {
if in_block_comment {
// Check for end of block comment: #>
if ch == '#' && chars.peek() == Some(&'>') {
chars.next(); // consume '>'
in_block_comment = false;
}
} else {
match ch {
// Start of block comment: <#
'<' if chars.peek() == Some(&'#') => {
chars.next(); // consume '#'
in_block_comment = true;
}
// Single-line comment: consume until end of line
'#' => {
while let Some(&next_ch) = chars.peek() {
if next_ch == '\n' || next_ch == '\r' {
break;
}
chars.next();
}
}
// Whitespace is allowed
c if c.is_whitespace() => {}
// Any other character means there's code before param
_ => return None,
}
}
}
// If we're still in a block comment at the end, it's unclosed - invalid
if in_block_comment {
return None;
}
// Skip whitespace and tabs after "param"
let mut chars = code[param_start + 5..].char_indices();
let mut paren_offset = param_start + 5;
// Skip whitespace to find opening paren
while let Some((idx, ch)) = chars.next() {
if ch == '(' {
paren_offset += idx;
break;
} else if !ch.is_whitespace() && ch != '\t' {
// Found non-whitespace, non-paren character - not a valid param block
return None;
}
}
// Now parse from the opening parenthesis
let remaining = &code[paren_offset..];
let mut chars = remaining.char_indices();
// Skip the opening '('
if let Some((_, ch)) = chars.next() {
if ch != '(' {
return None;
}
} else {
return None;
}
let mut depth = 1;
let mut in_single_quote = false;
let mut in_double_quote = false;
let mut escape_next = false;
let content_start = paren_offset + 1; // Start after the opening '('
for (idx, ch) in chars {
if escape_next {
escape_next = false;
continue;
}
match ch {
'`' if in_double_quote => {
// PowerShell escape character
escape_next = true;
}
'\'' if !in_double_quote => {
in_single_quote = !in_single_quote;
}
'"' if !in_single_quote => {
in_double_quote = !in_double_quote;
}
'(' if !in_single_quote && !in_double_quote => {
depth += 1;
}
')' if !in_single_quote && !in_double_quote => {
depth -= 1;
if depth == 0 {
// Found the matching closing parenthesis
// idx is the position of ')' relative to paren_offset
if include_keyword {
// Return full block including "param" keyword and closing paren
return Some(&code[param_start..paren_offset + idx + 1]);
} else {
// Return only contents between parentheses
return Some(&code[content_start..paren_offset + idx]);
}
}
}
_ => {}
}
}
None
}
enum ParserState {
Normal,
InSingleQuote,
@@ -137,10 +266,9 @@ fn parse_powershell_single_typ(typ: &str) -> Typ {
}
fn parse_powershell_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
let param_wrapper = RE_POWERSHELL_PARAM.captures(code);
let param_wrapper = extract_powershell_param_block(code, false);
let mut args = vec![];
if let Some(param_wrapper) = param_wrapper {
let param_wrapper = param_wrapper.get(1).unwrap().as_str();
let params = split_pwsh_args(param_wrapper);
for param in params {
if let Some(cap) = RE_POWERSHELL_ARGS.captures(param) {
@@ -266,7 +394,7 @@ non_required="${5:-}"
#[test]
fn test_parse_powershell_sig() -> anyhow::Result<()> {
let code = r#"param($Msg, [string]$Msg2, $Dflt = "default value, with comma", [int]$Nb = 3 , $Nb2 = 5.0, $Nb3 = 5, $Wahoo = $env:WAHOO, [PSCustomObject]$Obj, [string[]]$Arr)"#;
let code = r#"param($Msg, [string]$Msg2, $Dflt = "default value, with comma", [int]$Nb = 3 , $Nb2 = 5.0, $Nb3 = 5, $Wahoo = $env:WAHOO, [PSCustomObject]$Obj, [string[]]$Arr, [Parameter(Mandatory)][ValidateSet('Green', 'Blue', 'Red')][string]$Message)"#;
assert_eq!(
parse_powershell_sig(code)?,
MainArgSignature {
@@ -344,6 +472,14 @@ non_required="${5:-}"
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "Message".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
}
],
no_main_func: None,
@@ -353,6 +489,121 @@ non_required="${5:-}"
Ok(())
}
#[test]
fn test_extract_powershell_param_block() {
// Basic cases
assert_eq!(
extract_powershell_param_block("param($Name, $Age)", true),
Some("param($Name, $Age)")
);
assert_eq!(
extract_powershell_param_block("param($Name, $Age)", false),
Some("$Name, $Age")
);
// Case insensitive and whitespace
assert_eq!(
extract_powershell_param_block("PARAM ($Value)", false),
Some("$Value")
);
// Nested parentheses
assert_eq!(
extract_powershell_param_block("param([ValidateScript({$_ -gt 0})]$Count)", false),
Some("[ValidateScript({$_ -gt 0})]$Count")
);
// Strings with parentheses
assert_eq!(
extract_powershell_param_block("param([string]$Path = 'C:\\file(1).txt')", false),
Some("[string]$Path = 'C:\\file(1).txt'")
);
assert_eq!(
extract_powershell_param_block(r#"param($Msg = "Hello (world)")"#, false),
Some(r#"$Msg = "Hello (world)""#)
);
assert_eq!(
extract_powershell_param_block(
r#"param([Parameter(Mandatory)][ValidateSet('Green', 'Blue', 'Red')][string]$Message)"#,
false
),
Some(r#"[Parameter(Mandatory)][ValidateSet('Green', 'Blue', 'Red')][string]$Message"#)
);
// Escaped quotes
assert_eq!(
extract_powershell_param_block("param($Text = 'don''t')", false),
Some("$Text = 'don''t'")
);
assert_eq!(
extract_powershell_param_block(r#"param($Text = "He said `"Hi`"")"#, false),
Some(r#"$Text = "He said `"Hi`"""#)
);
// Multiline
let multiline = "param(\n [string]$Name,\n [int]$Age\n)";
assert!(extract_powershell_param_block(multiline, false).is_some());
// Invalid cases
assert_eq!(extract_powershell_param_block("$x = 5", false), None);
assert_eq!(extract_powershell_param_block("param", false), None);
assert_eq!(extract_powershell_param_block("param($x", false), None);
// Valid: param at beginning with single-line comments before
assert_eq!(
extract_powershell_param_block("# This is a comment\nparam($Name)", false),
Some("$Name")
);
assert_eq!(
extract_powershell_param_block("# Comment 1\n# Comment 2\n\nparam($Name)", false),
Some("$Name")
);
// Valid: param at beginning with block comment before
assert_eq!(
extract_powershell_param_block("<# Block comment #>\nparam($Name)", false),
Some("$Name")
);
assert_eq!(
extract_powershell_param_block(
"<#\n Multi-line\n block comment\n#>\nparam($Name)",
false
),
Some("$Name")
);
// Valid: mixed comments and whitespace
assert_eq!(
extract_powershell_param_block(
"# Line comment\n<# Block comment #>\n\nparam($Name)",
false
),
Some("$Name")
);
// Invalid: code before param
assert_eq!(
extract_powershell_param_block("$x = 5\nparam($Name)", false),
None
);
assert_eq!(
extract_powershell_param_block("Write-Host 'test'\nparam($Name)", false),
None
);
// Invalid: unclosed block comment
assert_eq!(
extract_powershell_param_block("<# Unclosed comment\nparam($Name)", false),
None
);
// Invalid: unclosed block comment
assert_eq!(
extract_powershell_param_block("function test-x{ param($Name)\n}", false),
None
);
}
#[test]
fn test_parse_bash_sig_with_crlf() -> anyhow::Result<()> {
// Test with CRLF line endings (Windows-style)
backend/parsers/windmill-parser-php/src/lib.rs
+31 -13
View File
@@ -44,23 +44,41 @@ fn parse_default_expr(e: Expression) -> Option<Value> {
pub fn parse_php_signature(
code: &str,
override_main: Option<String>,
override_entrypoint: Option<String>,
) -> anyhow::Result<MainArgSignature> {
let main_name = override_main.unwrap_or("main".to_string());
let entrypoint_fn_name = override_entrypoint.unwrap_or("main".to_string());
let ast = parser::parse(code)
.map_err(|e| anyhow::anyhow!("Error parsing code: {}", e.to_string()))?;
let params = ast.into_iter().find_map(|x| match x {
Statement::Function(FunctionStatement {
name,
parameters: FunctionParameterList { parameters, .. },
..
}) if name.to_string() == main_name => Some(parameters),
_ => None,
});
let mut entrypoint_params = None;
let mut has_preprocessor = None;
for node in ast.into_iter() {
match node {
Statement::Function(FunctionStatement {
name,
parameters: FunctionParameterList { parameters, .. },
..
}) => {
let fn_name = name.to_string();
if let Some(params) = params {
if has_preprocessor.is_none() && fn_name == "preprocessor" {
has_preprocessor = Some(true);
}
if entrypoint_params.is_none() && fn_name == entrypoint_fn_name {
entrypoint_params = Some(parameters);
}
if has_preprocessor.is_some() && entrypoint_params.is_some() {
break;
}
}
_ => {}
};
}
if let Some(params) = entrypoint_params {
let args = params
.into_iter()
.map(|x| {
@@ -82,7 +100,7 @@ pub fn parse_php_signature(
star_kwargs: false,
args,
no_main_func: Some(false),
has_preprocessor: None,
has_preprocessor,
})
} else {
Ok(MainArgSignature {
@@ -90,7 +108,7 @@ pub fn parse_php_signature(
star_kwargs: false,
args: vec![],
no_main_func: Some(true),
has_preprocessor: None,
has_preprocessor,
})
}
}
backend/parsers/windmill-parser-wasm/src/lib.rs
+5 -2
View File
@@ -134,8 +134,11 @@ pub fn parse_graphql(code: &str) -> String {
#[cfg(feature = "php-parser")]
#[wasm_bindgen]
pub fn parse_php(code: &str) -> String {
wrap_sig(windmill_parser_php::parse_php_signature(code, None))
pub fn parse_php(code: &str, main_override: Option<String>) -> String {
wrap_sig(windmill_parser_php::parse_php_signature(
code,
main_override,
))
}
#[cfg(feature = "rust-parser")]
backend/src/monitor.rs
+33 -42
View File
@@ -59,7 +59,6 @@ use windmill_common::{
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
},
indexer::load_indexer_config,
jobs::QueuedJob,
jwt::JWT_SECRET,
oauth2::REQUIRE_PREEXISTING_USER_FOR_OAUTH,
server::load_smtp_config,
@@ -80,7 +79,7 @@ use windmill_common::{
OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED, SERVICE_LOG_RETENTION_SECS,
};
use windmill_common::{client::AuthedClient, global_settings::APP_WORKSPACED_ROUTE_SETTING};
use windmill_queue::{cancel_job, MiniPulledJob, SameWorkerPayload};
use windmill_queue::{SameWorkerPayload, cancel_job, get_queued_job_v2};
use windmill_worker::{
handle_job_error, JobCompletedSender, SameWorkerSender, BUNFIG_INSTALL_SCOPES,
INSTANCE_PYTHON_VERSION, JOB_DEFAULT_TIMEOUT, KEEP_JOB_DIR, MAVEN_REPOS, NO_DEFAULT_MAVEN,
@@ -2047,7 +2046,7 @@ async fn cancel_stale_job(
const RESTART_LIMIT: i32 = 3;
async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker_name: &str) {
async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, node_name: &str) {
let mut zombie_jobs_uuid_restart_limit_reached = vec![];
if *RESTART_ZOMBIE_JOBS {
@@ -2220,24 +2219,18 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
);
}
let jobs = sqlx::query_as::<_, QueuedJob>(
"SELECT *, null as workflow_as_code_status FROM v2_as_queue WHERE id = ANY($1)",
)
.bind(&timeouts[..])
.fetch_all(db)
.await
.map_err(|e| tracing::error!("Error fetching same worker jobs: {:?}", e))
.unwrap_or_default();
jobs
timeouts
};
let non_restartable_jobs = if *RESTART_ZOMBIE_JOBS {
vec![]
} else {
sqlx::query_as::<_, QueuedJob>("SELECT *, null as workflow_as_code_status FROM v2_as_queue WHERE last_ping < now() - ($1 || ' seconds')::interval
AND running = true AND job_kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow') AND same_worker = false")
.bind(ZOMBIE_JOB_TIMEOUT.as_str())
sqlx::query_scalar!("SELECT j.id
FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)
WHERE r.ping < now() - ($1 || ' seconds')::interval
AND q.running = true AND j.kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow') AND j.same_worker = false",
ZOMBIE_JOB_TIMEOUT.as_str())
.fetch_all(db)
.await
.ok()
@@ -2260,14 +2253,6 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
}
}
let zombie_jobs_restart_limit_reached = sqlx::query_as::<_, QueuedJob>(
"SELECT *, null as workflow_as_code_status FROM v2_as_queue WHERE id = ANY($1)",
)
.bind(&zombie_jobs_uuid_restart_limit_reached[..])
.fetch_all(db)
.await
.ok()
.unwrap_or_else(|| vec![]);
let timeouts = non_restartable_jobs
.into_iter()
@@ -2278,7 +2263,7 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
.map(|x| (x, ErrorMessage::SameWorker)),
)
.chain(
zombie_jobs_restart_limit_reached
zombie_jobs_uuid_restart_limit_reached
.into_iter()
.map(|x| (x, ErrorMessage::RestartLimit)),
)
@@ -2289,7 +2274,7 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
QUEUE_ZOMBIE_DELETE_COUNT.inc_by(timeouts.len() as _);
}
for (job, error_kind) in timeouts {
for (job_id, error_kind) in timeouts {
// since the job is unrecoverable, the same worker queue should never be sent anything
let (same_worker_tx_never_used, _same_worker_rx_never_used) =
mpsc::channel::<SameWorkerPayload>(1);
@@ -2298,6 +2283,12 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
let (send_result_never_used, _send_result_rx_never_used) =
JobCompletedSender::new_never_used();
let job = get_queued_job_v2(db, &job_id).await;
if let Err(e) = job {
tracing::error!("Error getting queued job: {:?}", e);
continue;
}
if let Some(job) = job.unwrap() {
let label = if job.permissioned_as != format!("u/{}", job.created_by)
&& job.permissioned_as != job.created_by
{
@@ -2311,7 +2302,7 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
&job.permissioned_as,
&label,
*SCRIPT_TOKEN_EXPIRY,
&job.email,
&job.permissioned_as_email,
&job.id,
None,
Some(format!("handle_zombie_jobs")),
@@ -2326,32 +2317,32 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, worker
None,
);
let last_ping = job.last_ping.clone();
let error_message = format!(
"Job timed out after no ping from job since {} (ZOMBIE_JOB_TIMEOUT: {}, reason: {:?})",
last_ping
.map(|x| x.to_string())
.unwrap_or_else(|| "no ping".to_string()),
"Job timed out after no ping from job since {} (ZOMBIE_JOB_TIMEOUT: {}, reason: {:?}).\nThis likely means that the job died on worker {}, OOM are a common reason for worker crashes.\nCheck the workers around the time of the last ping and the exit code if any.",
job.last_ping.unwrap_or_default(),
*ZOMBIE_JOB_TIMEOUT,
error_kind.to_string()
error_kind.to_string(),
job.worker.clone().unwrap_or_default(),
);
let memory_peak = job.memory_peak.unwrap_or(0);
let _ = handle_job_error(
db,
&client,
&MiniPulledJob::from(&job),
0,
&windmill_queue::MiniCompletedJob::from(job),
memory_peak,
None,
error::Error::ExecutionErr(error_message),
true,
Some(&same_worker_tx_never_used),
"",
worker_name,
node_name,
send_result_never_used,
#[cfg(feature = "benchmark")]
&mut windmill_common::bench::BenchmarkIter::new(),
)
.await;
}
}
}
async fn cleanup_concurrency_counters_orphaned_keys(db: &DB) -> error::Result<()> {
@@ -2457,13 +2448,13 @@ async fn handle_zombie_flows(db: &DB) -> error::Result<()> {
let flows = sqlx::query!(
r#"
SELECT
id AS "id!", workspace_id AS "workspace_id!", parent_job, is_flow_step,
flow_status AS "flow_status: Box<str>", last_ping, same_worker
FROM v2_as_queue
WHERE running = true AND suspend = 0 AND suspend_until IS null AND scheduled_for <= now()
AND (job_kind = 'flow' OR job_kind = 'flowpreview' OR job_kind = 'flownode')
AND last_ping IS NOT NULL AND last_ping < NOW() - ($1 || ' seconds')::interval
AND canceled = false
j.id AS "id!", j.workspace_id AS "workspace_id!", j.parent_job, j.flow_step_id IS NOT NULL AS "is_flow_step?",
COALESCE(s.flow_status, s.workflow_as_code_status) AS "flow_status: Box<str>", r.ping AS last_ping, j.same_worker AS "same_worker?"
FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)
WHERE q.running = true AND q.suspend = 0 AND q.suspend_until IS null AND q.scheduled_for <= now()
AND (j.kind = 'flow' OR j.kind = 'flowpreview' OR j.kind = 'flownode')
AND r.ping IS NOT NULL AND r.ping < NOW() - ($1 || ' seconds')::interval
AND q.canceled_by IS NULL
"#,
FLOW_ZOMBIE_TRANSITION_TIMEOUT.as_str()
backend/substitute_ee_code.sh
+1 -1
View File
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
set -euo pipefail
script_dirpath="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
root_dirpath="$(cd "${script_dirpath}/.." && pwd)"
backend/tests/common/mod.rs
+11 -1
View File
@@ -394,7 +394,17 @@ pub async fn listen_for_uuid_on(
pub async fn completed_job(uuid: Uuid, db: &Pool<Postgres>) -> CompletedJob {
sqlx::query_as::<_, CompletedJob>(
"SELECT *, result->'wm_labels' as labels FROM v2_as_completed_job WHERE id = $1",
"SELECT j.id, j.workspace_id, j.parent_job, j.created_by, j.created_at, c.duration_ms,
c.status = 'success' OR c.status = 'skipped' AS success, j.runnable_id AS script_hash, j.runnable_path AS script_path,
j.args, c.result, FALSE AS deleted, j.raw_code, c.status = 'canceled' AS canceled,
c.canceled_by, c.canceled_reason, j.kind AS job_kind,
CASE WHEN j.trigger_kind = 'schedule'::job_trigger_kind THEN j.trigger END AS schedule_path,
j.permissioned_as, COALESCE(c.flow_status, c.workflow_as_code_status) AS flow_status, j.raw_flow,
j.flow_step_id IS NOT NULL AS is_flow_step, j.script_lang AS language, c.started_at,
c.status = 'skipped' AS is_skipped, j.raw_lock, j.permissioned_as_email AS email, j.visible_to_owner,
c.memory_peak AS mem_peak, j.tag, j.priority, NULL::TEXT AS logs, c.result_columns,
j.script_entrypoint_override, j.preprocessed, c.result->'wm_labels' as labels
FROM v2_job_completed c JOIN v2_job j USING (id) WHERE j.id = $1",
)
.bind(uuid)
.fetch_one(db)
backend/tests/worker.rs
+4 -4
View File
@@ -2395,7 +2395,7 @@ async fn test_script_schedule_handlers(db: Pool<Postgres>) -> anyhow::Result<()>
let uuid = uuid.unwrap().unwrap();
let completed_job = sqlx::query!(
"SELECT script_path FROM v2_as_completed_job WHERE id = $1",
"SELECT runnable_path as script_path FROM v2_job WHERE id = $1",
uuid
)
.fetch_one(&db2)
@@ -2466,7 +2466,7 @@ async fn test_script_schedule_handlers(db: Pool<Postgres>) -> anyhow::Result<()>
let uuid = uuid.unwrap().unwrap();
let completed_job =
sqlx::query!("SELECT script_path FROM v2_as_completed_job WHERE id = $1", uuid)
sqlx::query!("SELECT runnable_path as script_path FROM v2_job WHERE id = $1", uuid)
.fetch_one(&db2)
.await
.unwrap();
@@ -2553,7 +2553,7 @@ async fn test_flow_schedule_handlers(db: Pool<Postgres>) -> anyhow::Result<()> {
let uuid = uuid.unwrap().unwrap();
let completed_job = sqlx::query!(
"SELECT script_path FROM v2_as_completed_job WHERE id = $1",
"SELECT runnable_path as script_path FROM v2_job WHERE id = $1",
uuid
)
.fetch_one(&db2)
@@ -2625,7 +2625,7 @@ async fn test_flow_schedule_handlers(db: Pool<Postgres>) -> anyhow::Result<()> {
let uuid = uuid.unwrap().unwrap();
let completed_job =
sqlx::query!("SELECT script_path FROM v2_as_completed_job WHERE id = $1", uuid)
sqlx::query!("SELECT runnable_path as script_path FROM v2_job WHERE id = $1", uuid)
.fetch_one(&db2)
.await
.unwrap();
backend/update_sqlx.sh
-5
View File
@@ -2,9 +2,6 @@
set -e
# Default directory
EE_DIR="../windmill-ee-private"
# Parse arguments
while [[ "$#" -gt 0 ]]; do
case $1 in
@@ -14,8 +11,6 @@ while [[ "$#" -gt 0 ]]; do
shift
done
./substitute_ee_code.sh --dir "$EE_DIR"
# Check if running on macOS
if [[ "$(uname)" == "Darwin" ]]; then
echo "Running on macOS - substituting samael..."
backend/windmill-api/openapi.yaml
+64 -4
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.571.0
version: 1.575.0
title: Windmill API
contact:
@@ -629,12 +629,13 @@ paths:
tags:
- Git Sync
parameters:
- name: search
- name: page
in: query
description: Search repositories by name
description: Page number for pagination (default 1)
required: false
schema:
type: string
type: integer
default: 1
responses:
"200":
description: connected repositories
@@ -727,6 +728,32 @@ paths:
schema:
type: string
/w/{workspace}/workspaces/create_workspace_fork_branch:
post:
summary: create forked workspace branch with git sync
operationId: createWorkspaceForkGitBranch
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
requestBody:
description: new forked workspace
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/CreateWorkspaceFork"
responses:
"201":
description: forked workspace branch created
content:
application/json:
schema:
type: array
items:
type: string
format: uuid
/w/{workspace}/workspaces/create_fork:
post:
summary: create forked workspace
@@ -4871,6 +4898,27 @@ paths:
- content
- language
/scripts/hub/pick/{path}:
get:
summary: record hub script pick
operationId: pickHubScriptByPath
tags:
- script
parameters:
- $ref: "#/components/parameters/ScriptPath"
responses:
"200":
description: script pick recorded
content:
application/json:
schema:
type: object
properties:
success:
type: boolean
required:
- success
/scripts/hub/top:
get:
summary: get top hub scripts
@@ -7946,6 +7994,12 @@ paths:
schema:
type: boolean
- $ref: "#/components/parameters/NewJobId"
- name: memory_id
description: memory ID for chat-enabled flows
in: query
schema:
type: string
format: uuid
requestBody:
description: preview
@@ -7972,6 +8026,12 @@ paths:
- job
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: memory_id
description: memory ID for chat-enabled flows
in: query
schema:
type: string
format: uuid
requestBody:
description: preview
backend/windmill-api/src/approvals.rs
+22 -22
View File
@@ -204,31 +204,31 @@ pub async fn get_approval_form_details(
"WITH job_info AS (
-- Query for Teams (running jobs)
SELECT
parent.job_kind AS \"job_kind!: JobKind\",
parent.script_hash AS \"script_hash: ScriptHash\",
parent.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
child.parent_job AS \"parent_job: Uuid\",
parent.created_at AS \"created_at!: chrono::NaiveDateTime\",
parent.created_by AS \"created_by!\",
parent.script_path,
parent.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
FROM v2_as_queue child
JOIN v2_as_queue parent ON parent.id = child.parent_job
WHERE child.id = $1 AND child.workspace_id = $2
parent_j.kind AS \"job_kind!: JobKind\",
parent_j.runnable_id AS \"script_hash: ScriptHash\",
parent_j.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
child_j.parent_job AS \"parent_job: Uuid\",
parent_j.created_at AS \"created_at!: chrono::NaiveDateTime\",
parent_j.created_by AS \"created_by!\",
parent_j.runnable_path as script_path,
parent_j.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
FROM v2_job_queue child_q JOIN v2_job child_j USING (id)
JOIN v2_job parent_j ON parent_j.id = child_j.parent_job
WHERE child_j.id = $1 AND child_j.workspace_id = $2
UNION ALL
-- Query for Slack (completed jobs)
SELECT
v2_as_queue.job_kind AS \"job_kind!: JobKind\",
v2_as_queue.script_hash AS \"script_hash: ScriptHash\",
v2_as_queue.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
v2_as_completed_job.parent_job AS \"parent_job: Uuid\",
v2_as_completed_job.created_at AS \"created_at!: chrono::NaiveDateTime\",
v2_as_completed_job.created_by AS \"created_by!\",
v2_as_queue.script_path,
v2_as_queue.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
FROM v2_as_queue
JOIN v2_as_completed_job ON v2_as_completed_job.parent_job = v2_as_queue.id
WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2
parent_j.kind AS \"job_kind!: JobKind\",
parent_j.runnable_id AS \"script_hash: ScriptHash\",
parent_j.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
completed_j.parent_job AS \"parent_job: Uuid\",
completed_j.created_at AS \"created_at!: chrono::NaiveDateTime\",
completed_j.created_by AS \"created_by!\",
parent_j.runnable_path as script_path,
parent_j.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
FROM v2_job_completed completed_c JOIN v2_job completed_j USING (id)
JOIN v2_job parent_j ON parent_j.id = completed_j.parent_job
WHERE completed_j.id = $1 AND completed_j.workspace_id = $2
)
SELECT * FROM job_info LIMIT 1",
job_id,
backend/windmill-api/src/apps.rs
+7 -7
View File
@@ -2530,13 +2530,13 @@ async fn check_if_allowed_to_access_s3_file_from_app(
|| {
sqlx::query_scalar!(
r#"SELECT EXISTS (
SELECT 1 FROM v2_as_completed_job
WHERE workspace_id = $2
AND (job_kind = 'appscript' OR job_kind = 'preview')
AND created_by = 'anonymous'
AND started_at > now() - interval '3 hours'
AND script_path LIKE $3 || '/%'
AND result @> ('{"s3":"' || $1 || '"}')::jsonb
SELECT 1 FROM v2_job_completed c JOIN v2_job j USING (id)
WHERE j.workspace_id = $2
AND (j.kind = 'appscript' OR j.kind = 'preview')
AND j.created_by = 'anonymous'
AND c.started_at > now() - interval '3 hours'
AND j.runnable_path LIKE $3 || '/%'
AND c.result @> ('{"s3":"' || $1 || '"}')::jsonb
)"#,
file_query.s3,
w_id,
backend/windmill-api/src/embeddings.rs
+12 -2
View File
@@ -441,7 +441,7 @@ impl EmbeddingsDb {
&query_embedding,
limit.unwrap_or(10) as usize,
Some(&filter),
Some(0.75),
Some(0.8),
);
let results: Result<Vec<_>> = results
@@ -480,7 +480,17 @@ impl EmbeddingsDb {
})
.collect();
results
let mut results = results?;
if results.len() > 1 {
let top_score = results[0].score;
results = results
.into_iter()
.take_while(|r| (top_score - r.score) / top_score <= 0.05)
.collect();
}
Ok(results)
}
pub async fn query_resource_types(
backend/windmill-api/src/flow_conversations.rs
+9 -4
View File
@@ -11,7 +11,7 @@ use uuid::Uuid;
use crate::db::ApiAuthed;
use windmill_common::{
db::UserDB,
db::{UserDB, DB},
error::{JsonResult, Result},
flow_conversations::MessageType,
utils::{not_found_if_none, paginate, Pagination},
@@ -154,6 +154,7 @@ pub async fn get_or_create_conversation_with_id(
async fn delete_conversation(
authed: ApiAuthed,
Extension(user_db): Extension<UserDB>,
Extension(db): Extension<DB>,
Path((w_id, conversation_id)): Path<(String, Uuid)>,
) -> Result<String> {
let mut tx = user_db.clone().begin(&authed).await?;
@@ -185,10 +186,14 @@ async fn delete_conversation(
// Delete associated memory in background (non-blocking cleanup)
let w_id_clone = w_id.clone();
let db_clone = db.clone();
tokio::spawn(async move {
if let Err(e) =
windmill_worker::memory_oss::delete_conversation_memory(&w_id_clone, conversation_id)
.await
if let Err(e) = windmill_worker::memory_oss::delete_conversation_memory(
&db_clone,
&w_id_clone,
conversation_id,
)
.await
{
tracing::error!(
"Failed to delete memory for conversation {} in workspace {}: {:?}",
backend/windmill-api/src/flows.rs
+1
View File
@@ -1634,6 +1634,7 @@ mod tests {
early_return: None,
concurrency_key: None,
chat_input_enabled: None,
flow_env: None,
debounce_key: None,
debounce_delay_s: None,
ui: None,
backend/windmill-api/src/jobs.rs
+143 -131
View File
@@ -281,6 +281,10 @@ pub fn workspaced_service() -> Router {
"/result_by_id/:job_id/:node_id",
get(get_result_by_id).layer(cors.clone()),
)
.route(
"/flow_env_by_flow_job_id/:flow_job_id/:var_name",
get(get_flow_env_by_flow_job_id).layer(cors.clone()),
)
.route("/run/dependencies", post(run_dependencies_job))
.route("/run/flow_dependencies", post(run_flow_dependencies_job))
.route(
@@ -376,6 +380,59 @@ async fn get_root_job(
Ok(Json(res))
}
async fn get_flow_env_by_flow_job_id(
authed: ApiAuthed,
tokened: Tokened,
Extension(db): Extension<DB>,
Path((w_id, flow_job_id, var_name)): Path<(String, Uuid, String)>,
Query(JsonPath { json_path, .. }): Query<JsonPath>,
) -> windmill_common::error::JsonResult<Box<JsonRawValue>> {
let flow_env = sqlx::query_scalar!(
r#"
SELECT
CASE
WHEN flow_version.id IS NOT NULL THEN
(flow_version.value -> 'flow_env' -> $3) #> $4
ELSE
(root_job.raw_flow -> 'flow_env' -> $3) #> $4
END AS "flow_env: sqlx::types::Json<Box<RawValue>>"
FROM
v2_job current_job
JOIN
v2_job root_job ON root_job.id = COALESCE(current_job.root_job, current_job.flow_innermost_root_job, current_job.parent_job, current_job.id)
AND root_job.workspace_id = current_job.workspace_id
LEFT JOIN
flow_version ON flow_version.id = root_job.runnable_id
AND flow_version.path = root_job.runnable_path
AND flow_version.workspace_id = root_job.workspace_id
WHERE
current_job.id = $1 AND
current_job.workspace_id = $2"#,
flow_job_id,
w_id,
var_name,
json_path
.as_ref()
.map(|x| x.split(".").collect::<Vec<_>>())
.unwrap_or_default() as Vec<&str>,
)
.fetch_optional(&db)
.await?
.map(|r| r.map(|x| x.0))
.flatten()
.unwrap_or_else(|| to_raw_value(&serde_json::Value::Null));
log_job_view(
&db,
Some(&authed),
Some(&tokened.token),
&w_id,
&flow_job_id,
)
.await?;
Ok(Json(flow_env))
}
async fn compute_root_job_for_flow(db: &DB, w_id: &str, job_id: Uuid) -> error::Result<String> {
let root_job = sqlx::query_scalar!(
r#"SELECT COALESCE(root_job, flow_innermost_root_job, parent_job, id) as "root_job!" FROM v2_job WHERE id = $1 AND workspace_id = $2"#,
@@ -841,101 +898,6 @@ macro_rules! get_job_query {
}
}
// CREATE OR REPLACE VIEW v2_as_queue AS
// SELECT
// j.id,
// j.workspace_id,
// j.parent_job,
// j.created_by,
// j.created_at,
// q.started_at,
// q.scheduled_for,
// q.running,
// j.runnable_id AS script_hash,
// j.runnable_path AS script_path,
// j.args,
// j.raw_code,
// q.canceled_by IS NOT NULL AS canceled,
// q.canceled_by,
// q.canceled_reason,
// r.ping AS last_ping,
// j.kind AS job_kind,
// CASE WHEN j.trigger_kind = 'schedule'::job_trigger_kind THEN j.trigger END
// AS schedule_path,
// j.permissioned_as,
// COALESCE(s.flow_status, s.workflow_as_code_status) AS flow_status,
// j.raw_flow,
// j.flow_step_id IS NOT NULL AS is_flow_step,
// j.script_lang AS language,
// q.suspend,
// q.suspend_until,
// j.same_worker,
// j.raw_lock,
// j.pre_run_error,
// j.permissioned_as_email AS email,
// j.visible_to_owner,
// r.memory_peak AS mem_peak,
// j.flow_innermost_root_job AS root_job,
// s.flow_leaf_jobs AS leaf_jobs,
// j.tag,
// j.concurrent_limit,
// j.concurrency_time_window_s,
// j.timeout,
// j.flow_step_id,
// j.cache_ttl,
// j.priority,
// NULL::TEXT AS logs,
// j.script_entrypoint_override,
// j.preprocessed
// FROM v2_job_queue q
// JOIN v2_job j USING (id)
// LEFT JOIN v2_job_runtime r USING (id)
// LEFT JOIN v2_job_status s USING (id)
// ;
// -- Add up migration script here
// CREATE OR REPLACE VIEW v2_as_completed_job AS
// SELECT
// j.id,
// j.workspace_id,
// j.parent_job,
// j.created_by,
// j.created_at,
// c.duration_ms,
// c.status = 'success' OR c.status = 'skipped' AS success,
// j.runnable_id AS script_hash,
// j.runnable_path AS script_path,
// j.args,
// c.result,
// FALSE AS deleted,
// j.raw_code,
// c.status = 'canceled' AS canceled,
// c.canceled_by,
// c.canceled_reason,
// j.kind AS job_kind,
// CASE WHEN j.trigger_kind = 'schedule'::job_trigger_kind THEN j.trigger END
// AS schedule_path,
// j.permissioned_as,
// COALESCE(c.flow_status, c.workflow_as_code_status) AS flow_status,
// j.raw_flow,
// j.flow_step_id IS NOT NULL AS is_flow_step,
// j.script_lang AS language,
// c.started_at,
// c.status = 'skipped' AS is_skipped,
// j.raw_lock,
// j.permissioned_as_email AS email,
// j.visible_to_owner,
// c.memory_peak AS mem_peak,
// j.tag,
// j.priority,
// NULL::TEXT AS logs,
// c.result_columns,
// j.script_entrypoint_override,
// j.preprocessed
// FROM v2_job_completed c
// JOIN v2_job j USING (id)
// ;
#[derive(Copy, Clone)]
struct GetQuery<'a> {
with_logs: bool,
@@ -1506,10 +1468,10 @@ async fn get_job_logs(
.flatten();
let record = sqlx::query!(
"SELECT created_by AS \"created_by!\", CONCAT(coalesce(v2_as_completed_job.logs, ''), coalesce(job_logs.logs, '')) as logs, job_logs.log_offset, job_logs.log_file_index
FROM v2_as_completed_job
LEFT JOIN job_logs ON job_logs.job_id = v2_as_completed_job.id
WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2 AND ($3::text[] IS NULL OR v2_as_completed_job.tag = ANY($3))",
"SELECT j.created_by AS \"created_by\", coalesce(job_logs.logs, '') as logs, COALESCE(job_logs.log_offset, 0) as log_offset, job_logs.log_file_index
FROM v2_job j
LEFT JOIN job_logs ON job_logs.job_id = j.id
WHERE j.id = $1 AND j.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
id,
w_id,
tags.as_ref().map(|v| v.as_slice())
@@ -1535,11 +1497,21 @@ async fn get_job_logs(
.await?;
#[cfg(all(feature = "enterprise", feature = "parquet"))]
if let Some(r) = get_logs_from_store(record.log_offset, &logs, &record.log_file_index).await
if let Some(r) = get_logs_from_store(
record.log_offset.unwrap_or(0),
&logs,
&record.log_file_index,
)
.await
{
return r.map(content_plain);
}
if let Some(r) = get_logs_from_disk(record.log_offset, &logs, &record.log_file_index).await
if let Some(r) = get_logs_from_disk(
record.log_offset.unwrap_or(0),
&logs,
&record.log_file_index,
)
.await
{
return r.map(content_plain);
}
@@ -1554,10 +1526,10 @@ async fn get_job_logs(
Ok(content_plain(Body::from(logs)))
} else {
let text = sqlx::query!(
"SELECT created_by AS \"created_by!\", CONCAT(coalesce(v2_as_queue.logs, ''), coalesce(job_logs.logs, '')) as logs, coalesce(job_logs.log_offset, 0) as log_offset, job_logs.log_file_index
FROM v2_as_queue
LEFT JOIN job_logs ON job_logs.job_id = v2_as_queue.id
WHERE v2_as_queue.id = $1 AND v2_as_queue.workspace_id = $2 AND ($3::text[] IS NULL OR v2_as_queue.tag = ANY($3))",
"SELECT j.created_by AS \"created_by!\", CONCAT(coalesce(job_logs.logs, '')) as logs, coalesce(job_logs.log_offset, 0) as log_offset, job_logs.log_file_index
FROM v2_job j
LEFT JOIN job_logs ON job_logs.job_id = j.id
WHERE j.id = $1 AND j.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
id,
w_id,
tags.as_ref().map(|v| v.as_slice())
@@ -1613,9 +1585,9 @@ async fn get_args(
.map(|authed| get_scope_tags(authed))
.flatten();
let record = sqlx::query!(
"SELECT created_by AS \"created_by!\", args as \"args: sqlx::types::Json<Box<RawValue>>\"
FROM v2_as_completed_job
WHERE id = $1 AND workspace_id = $2 AND ($3::text[] IS NULL OR tag = ANY($3))",
"SELECT j.created_by AS \"created_by!\", j.args as \"args: sqlx::types::Json<Box<RawValue>>\"
FROM v2_job j
WHERE j.id = $1 AND j.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
id,
&w_id,
tags.as_ref().map(|v| v.as_slice()) as Option<&[&str]>,
@@ -2170,7 +2142,7 @@ async fn cancel_selection(
let mut tx = user_db.begin(&authed).await?;
let tags = get_scope_tags(&authed).map(|v| v.iter().map(|s| s.to_string()).collect_vec());
let jobs_to_cancel = sqlx::query_scalar!(
"SELECT id AS \"id!\" FROM v2_as_queue WHERE id = ANY($1) AND schedule_path IS NULL AND ($2::text[] IS NULL OR tag = ANY($2))",
"SELECT j.id AS \"id!\" FROM v2_job j WHERE j.id = ANY($1) AND j.trigger_kind != 'schedule'::job_trigger_kind AND ($2::text[] IS NULL OR j.tag = ANY($2))",
&jobs,
tags.as_ref().map(|v| v.as_slice())
)
@@ -2274,7 +2246,7 @@ async fn count_queue_jobs(
Ok(Json(
sqlx::query_as!(
QueueStats,
"SELECT coalesce(COUNT(*) FILTER(WHERE suspend = 0 AND running = false), 0) as \"database_length!\", coalesce(COUNT(*) FILTER(WHERE suspend > 0), 0) as \"suspended!\" FROM v2_as_queue WHERE (workspace_id = $1 OR $2) AND scheduled_for <= now() AND ($3::text[] IS NULL OR tag = ANY($3))",
"SELECT coalesce(COUNT(*) FILTER(WHERE q.suspend = 0 AND q.running = false), 0) as \"database_length!\", coalesce(COUNT(*) FILTER(WHERE q.suspend > 0), 0) as \"suspended!\" FROM v2_job_queue q JOIN v2_job j USING (id) WHERE (j.workspace_id = $1 OR $2) AND q.scheduled_for <= now() AND ($3::text[] IS NULL OR j.tag = ANY($3))",
w_id,
w_id == "admins" && cq.all_workspaces.unwrap_or(false),
tags.as_ref().map(|v| v.as_slice())
@@ -2717,9 +2689,9 @@ async fn get_suspended_flow_info<'c>(
let flow = sqlx::query_as!(
FlowInfo,
r#"
SELECT id AS "id!", flow_status, suspend AS "suspend!", script_path
FROM v2_as_queue
WHERE id = $1
SELECT j.id AS "id!", COALESCE(s.flow_status, s.workflow_as_code_status) as flow_status, q.suspend AS "suspend!", j.runnable_path as script_path
FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_status s USING (id)
WHERE j.id = $1
"#,
job_id,
)
@@ -2933,9 +2905,9 @@ pub async fn get_flow_user_state(
let mut tx = user_db.begin(&authed).await?;
let r = sqlx::query_scalar!(
r#"
SELECT flow_status->'user_states'->$1
FROM v2_as_queue
WHERE id = $2 AND workspace_id = $3
SELECT COALESCE(s.flow_status, s.workflow_as_code_status)->'user_states'->$1
FROM v2_job_queue q LEFT JOIN v2_job_status s USING (id)
WHERE q.id = $2 AND q.workspace_id = $3
"#,
key,
job_id,
@@ -3864,6 +3836,7 @@ async fn batch_rerun_handle_job(
batch_rerun_compute_js_expression(expr.clone(), job.clone()).await?,
);
}
InputTransform::Ai => {}
}
}
@@ -4166,10 +4139,10 @@ pub async fn restart_flow(
let mut tx = user_db.clone().begin(&authed).await?;
let completed_job = sqlx::query!(
"SELECT
script_path, args AS \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",
tag AS \"tag!\", priority
FROM v2_as_completed_job
WHERE id = $1 and workspace_id = $2",
j.runnable_path as script_path, j.args AS \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",
j.tag AS \"tag!\", j.priority
FROM v2_job j
WHERE j.id = $1 and j.workspace_id = $2",
job_id,
&w_id,
)
@@ -4321,7 +4294,12 @@ pub async fn run_script_by_path_inner(
tag,
timeout,
None,
None,
// If the job has a parent job, set priority to 2 as it may be ran synchronously and block a current worker until being executed. Flow steps have a priority of 1 so this is higher.
if run_query.parent_job.is_some() || run_query.root_job.is_some() {
Some(2)
} else {
None
},
push_authed.as_ref(),
false,
None,
@@ -4819,10 +4797,9 @@ pub async fn delete_job_metadata_after_use(db: &DB, job_uuid: Uuid) -> Result<()
pub async fn check_queue_too_long(db: &DB, queue_limit: Option<i64>) -> error::Result<()> {
if let Some(limit) = queue_limit {
let count = sqlx::query_scalar!(
"SELECT COUNT(*) FROM v2_as_queue WHERE canceled = false AND (scheduled_for <= now()
OR (suspend_until IS NOT NULL
AND ( suspend <= 0
OR suspend_until <= now())))",
"SELECT COUNT(*) FROM v2_job_queue q WHERE q.canceled_by IS NULL AND (q.scheduled_for <= now()
OR (q.suspend_until IS NOT NULL
AND (q.suspend <= 0 OR q.suspend_until <= now())))",
)
.fetch_one(db)
.await?
@@ -6352,7 +6329,15 @@ async fn run_preview_flow_job(
check_tag_available_for_workspace(&db, &w_id, &tag, &authed).await?;
let tx = PushIsolationLevel::Isolated(user_db.clone(), authed.clone().into());
let (uuid, tx) = push(
let chat_input_enabled = raw_flow.value.chat_input_enabled.unwrap_or(false);
let flow_path = raw_flow.path.clone().unwrap_or_default();
let user_message = raw_flow
.args
.as_ref()
.and_then(|args| args.get("user_message"))
.cloned();
let (uuid, mut tx) = push(
&db,
tx,
&w_id,
@@ -6386,6 +6371,25 @@ async fn run_preview_flow_job(
None,
)
.await?;
// Set memory_id if provided (for agent memory)
if let Some(memory_id) = run_query.memory_id {
set_flow_memory_id(&mut tx, uuid, memory_id).await?;
}
// Handle conversation messages for chat-enabled flows
if chat_input_enabled {
handle_chat_conversation_messages(
&mut tx,
&authed,
&w_id,
&flow_path,
&run_query,
user_message.as_ref(),
)
.await?;
}
tx.commit().await?;
Ok((StatusCode::CREATED, uuid.to_string()))
@@ -7119,6 +7123,11 @@ pub fn start_job_update_sse_stream(
// Update log offset if available
if let Some(new_offset) = update.log_offset {
if new_offset != log_offset.unwrap_or(0) {
// let logs = update.new_logs.clone().unwrap_or_default();
// tracing::error!(
// "new_offset: {new_offset}; log_offset: {log_offset:?}; {} {logs}",
// logs.len()
// );
log_offset = Some(new_offset);
} else {
update.log_offset = None;
@@ -7460,7 +7469,10 @@ async fn get_job_update_data(
}
let job = if record.completed.unwrap_or(false) && get_full_job_on_completion {
let get = GetQuery::new().with_auth(&opt_authed).without_logs();
let get = GetQuery::new()
.with_auth(&opt_authed)
.without_logs()
.without_code();
Some(get.fetch(&db, job_id, &w_id).await?)
} else {
None
@@ -7963,10 +7975,10 @@ async fn count_by_tag(
let counts = sqlx::query_as!(
TagCount,
r#"
SELECT tag as "tag!", COUNT(*) as "count!"
FROM v2_as_completed_job
WHERE started_at > NOW() - make_interval(secs => $1) AND ($2::text IS NULL OR workspace_id = $2)
GROUP BY tag
SELECT j.tag as "tag!", COUNT(*) as "count!"
FROM v2_job_completed c JOIN v2_job j USING (id)
WHERE c.started_at > NOW() - make_interval(secs => $1) AND ($2::text IS NULL OR j.workspace_id = $2)
GROUP BY j.tag
ORDER BY "count!" DESC
"#,
horizon as f64,
backend/windmill-api/src/lib.rs
+14 -1
View File
@@ -62,7 +62,10 @@ use tower_http::{
};
use windmill_common::db::UserDB;
use windmill_common::worker::CLOUD_HOSTED;
use windmill_common::{utils::{configure_client, GIT_VERSION}, BASE_URL, INSTANCE_NAME};
use windmill_common::{
utils::{configure_client, GIT_VERSION},
BASE_URL, INSTANCE_NAME,
};
use crate::scim_oss::has_scim_token;
use windmill_common::error::AppError;
@@ -180,6 +183,7 @@ pub mod workspaces_ee;
mod workspaces_export;
mod workspaces_extra;
mod workspaces_oss;
mod public_app_layer;
#[cfg(feature = "mcp")]
mod mcp;
@@ -699,6 +703,15 @@ pub async fn run_server(
)
};
let app = if let Some(domain) = public_app_layer::PUBLIC_APP_DOMAIN.as_ref() {
tracing::info!("Public app domain filter enabled for domain: {}", domain);
app.layer(axum::middleware::from_fn(
public_app_layer::public_app_domain_filter,
))
} else {
app
};
let app = app.layer(CatchPanicLayer::custom(|err| {
tracing::error!("panic in handler, returning 500: {:?}", err);
Response::builder()
backend/windmill-api/src/public_app_layer.rs
+229
View File
@@ -0,0 +1,229 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
use axum::body::Body;
use axum::response::Response;
use http::StatusCode;
lazy_static::lazy_static! {
pub static ref PUBLIC_APP_DOMAIN: Option<String> = std::env::var("PUBLIC_APP_DOMAIN").ok();
}
/// Middleware to restrict public app domain to whitelisted routes
pub async fn public_app_domain_filter(
req: axum::extract::Request,
next: axum::middleware::Next,
) -> Response {
if let Some(public_domain) = PUBLIC_APP_DOMAIN.as_ref() {
if let Some(host) = req.headers().get(http::header::HOST) {
if let Ok(host_str) = host.to_str() {
// Extract hostname without port
let hostname = host_str.split(':').next().unwrap_or(host_str);
if hostname == public_domain {
let path = req.uri().path();
// Check if route is whitelisted
let is_whitelisted = is_public_route_whitelisted(path);
if !is_whitelisted {
tracing::warn!(
"Rejected request to {} on public app domain {}",
path,
public_domain
);
return Response::builder()
.status(StatusCode::FORBIDDEN)
.body(Body::from(
"Access forbidden: route not allowed on public app domain",
))
.unwrap();
}
}
}
}
}
next.run(req).await
}
fn is_public_route_whitelisted(path: &str) -> bool {
// Whitelisted route patterns for public app domain
let whitelist = [
// Public app routes
"/api/ee_license",
"/api/w/*/users/whoami",
"/api/w/*/apps_u/*",
"/api/w/*/resources/list",
"/api/w/*/jobs_u/getupdate_sse/*",
"/api/auth/login",
"/api/w/*/folders/listnames",
"/api/w/*/resources/exists/*",
"/api/w/*/resources/type/get/*",
"/api/w/*/resources/type/listnames",
"/api/oauth/login/*",
"/api/oauth/connect/*",
"/oauth/callback/*",
"/user/login_callback/*",
"/api/workspaces/users",
"/api/users/whoami",
"/api/oauth/list_connects",
"/api/oauth/list_logins",
"/public/*",
"/a/*",
"/Inter-Variable.woff2",
];
for pattern in &whitelist {
if matches_pattern(path, pattern) {
return true;
}
}
false
}
fn matches_pattern(path: &str, pattern: &str) -> bool {
// Handle exact match
if path == pattern {
return true;
}
// Handle wildcard patterns
if !pattern.contains('*') {
return false;
}
let parts: Vec<&str> = pattern.split('*').collect();
// Start from the beginning - path must start with first part
if !path.starts_with(parts[0]) {
return false;
}
let mut remaining = &path[parts[0].len()..];
// Check each part in between wildcards
for i in 1..parts.len() - 1 {
let part = parts[i];
if let Some(pos) = remaining.find(part) {
remaining = &remaining[pos + part.len()..];
} else {
return false;
}
}
// Check the last part
let last = parts[parts.len() - 1];
if last.is_empty() {
// Wildcard at the end, already matched
return true;
} else {
// Must end with the last part
return remaining.ends_with(last);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_matches_pattern_exact() {
assert!(matches_pattern("/api/version", "/api/version"));
assert!(!matches_pattern("/api/versions", "/api/version"));
}
#[test]
fn test_matches_pattern_wildcard_middle() {
assert!(matches_pattern(
"/api/w/my-workspace/apps_u/my-app",
"/api/w/*/apps_u/*"
));
assert!(matches_pattern(
"/api/w/workspace123/jobs_u/job456",
"/api/w/*/jobs_u/*"
));
assert!(!matches_pattern(
"/api/w/workspace/apps/my-app",
"/api/w/*/apps_u/*"
));
}
#[test]
fn test_matches_pattern_wildcard_end() {
assert!(matches_pattern("/api/r/some/path", "/api/r/*"));
assert!(matches_pattern("/api/auth/login", "/api/auth/*"));
assert!(matches_pattern("/public/asset.css", "/public/*"));
assert!(!matches_pattern("/api/admin/users", "/api/r/*"));
}
#[test]
fn test_is_public_route_whitelisted() {
// Whitelisted routes based on current whitelist
assert!(is_public_route_whitelisted("/api/ee_license"));
assert!(is_public_route_whitelisted(
"/api/w/my-workspace/users/whoami"
));
assert!(is_public_route_whitelisted(
"/api/w/my-workspace/apps_u/my-app"
));
assert!(is_public_route_whitelisted(
"/api/w/workspace/resources/list"
));
// Non-whitelisted routes
assert!(!is_public_route_whitelisted("/api/version"));
assert!(!is_public_route_whitelisted("/api/uptodate"));
assert!(!is_public_route_whitelisted("/api/w/workspace/scripts"));
assert!(!is_public_route_whitelisted("/api/w/workspace/flows"));
assert!(!is_public_route_whitelisted(
"/api/w/workspace/apps/private"
));
assert!(!is_public_route_whitelisted("/api/admin/settings"));
assert!(!is_public_route_whitelisted("/api/users"));
}
#[test]
fn test_matches_pattern_complex_multiple_wildcards() {
// Test pattern with two wildcards like "/api/w/*/foo/*"
assert!(matches_pattern(
"/api/w/workspace1/s3_proxy/file.txt",
"/api/w/*/s3_proxy/*"
));
assert!(matches_pattern(
"/api/w/my-workspace/s3_proxy/path/to/file.jpg",
"/api/w/*/s3_proxy/*"
));
assert!(matches_pattern(
"/api/w/test/apps_u/myapp",
"/api/w/*/apps_u/*"
));
// Should not match if middle part is different
assert!(!matches_pattern(
"/api/w/workspace1/other/file.txt",
"/api/w/*/s3_proxy/*"
));
assert!(!matches_pattern(
"/api/w/workspace/apps/myapp",
"/api/w/*/apps_u/*"
));
// Edge cases
assert!(matches_pattern(
"/api/w/ws/s3_proxy/a",
"/api/w/*/s3_proxy/*"
));
// Path must have content after the last wildcard segment
assert!(matches_pattern(
"/api/w/ws/s3_proxy/",
"/api/w/*/s3_proxy/*"
));
}
}
backend/windmill-api/src/resources.rs
+10 -6
View File
@@ -798,13 +798,14 @@ async fn delete_resource(
check_scopes(&authed, || format!("resources:write:{}", path))?;
let mut tx = user_db.begin(&authed).await?;
sqlx::query!(
"DELETE FROM resource WHERE path = $1 AND workspace_id = $2",
let deleted_path = sqlx::query_scalar!(
"DELETE FROM resource WHERE path = $1 AND workspace_id = $2 RETURNING path",
path,
w_id
)
.execute(&mut *tx)
.fetch_optional(&mut *tx)
.await?;
not_found_if_none(deleted_path, "Resource", &path)?;
sqlx::query!(
"DELETE FROM variable WHERE path = $1 AND workspace_id = $2",
path,
@@ -1248,13 +1249,16 @@ async fn delete_resource_type(
let mut tx = user_db.begin(&authed).await?;
sqlx::query!(
"DELETE FROM resource_type WHERE name = $1 AND workspace_id = $2",
let deleted_name = sqlx::query_scalar!(
"DELETE FROM resource_type WHERE name = $1 AND workspace_id = $2 RETURNING name",
name,
w_id
)
.execute(&mut *tx)
.fetch_optional(&mut *tx)
.await?;
not_found_if_none(deleted_name, "ResourceType", &name)?;
audit_log(
&mut *tx,
&authed,
backend/windmill-api/src/scripts.rs
+38 -1
View File
@@ -131,6 +131,7 @@ pub fn global_service() -> Router {
.route("/hub/top", get(get_top_hub_scripts))
.route("/hub/get/*path", get(get_hub_script_by_path))
.route("/hub/get_full/*path", get(get_full_hub_script_by_path))
.route("/hub/pick/*path", get(pick_hub_script_by_path))
}
pub fn global_unauthed_service() -> Router {
@@ -606,8 +607,10 @@ async fn create_script_internal<'c>(
Ok(None)
}
(Some(p_hash), o) => {
// Lock the parent row to prevent concurrent updates with the same parent_hash
// This ensures linear lineage - only one script can have a given parent at a time
if sqlx::query_scalar!(
"SELECT 1 FROM script WHERE hash = $1 AND workspace_id = $2",
"SELECT 1 FROM script WHERE hash = $1 AND workspace_id = $2 FOR UPDATE",
p_hash.0,
&w_id
)
@@ -1112,6 +1115,40 @@ pub async fn get_full_hub_script_by_path(
))
}
pub async fn pick_hub_script_by_path(
Path(path): Path<StripPath>,
Extension(db): Extension<DB>,
) -> impl IntoResponse {
let path_str = path.to_path();
// Extract version_id from path (format: {hub}/{version_id}/{summary})
let version_id = path_str.split('/').nth(1).unwrap_or("");
let hub_base_url = HUB_BASE_URL.read().await.clone();
// Determine which hub to use based on version_id
// If version_id < PRIVATE_HUB_MIN_VERSION, use default hub
let target_hub_url = if version_id
.parse::<i32>()
.is_ok_and(|v| v < windmill_common::PRIVATE_HUB_MIN_VERSION)
{
windmill_common::DEFAULT_HUB_BASE_URL
} else {
&hub_base_url
};
// Call the hub's pick endpoint: /scripts/{version_id}/pick
let (status_code, headers, response) = query_elems_from_hub(
&HTTP_CLIENT,
&format!("{}/scripts/{}/pick", target_hub_url, version_id),
None,
&db,
)
.await?;
Ok::<_, Error>((status_code, headers, response))
}
async fn get_script_by_path(
authed: ApiAuthed,
Extension(user_db): Extension<UserDB>,
backend/windmill-api/src/triggers/trigger_helpers.rs
+6 -2
View File
@@ -46,7 +46,7 @@ struct ScriptInfo {
#[derive(Debug, Deserialize)]
struct PropertyDefinition {
r#type: Option<String>,
r#type: Option<Box<RawValue>>,
}
#[derive(Debug, Deserialize)]
@@ -119,7 +119,11 @@ fn runnable_format_from_schema_without_preprocessor(
if schema.as_ref().is_some_and(|schema| {
schema.properties.as_ref().is_some_and(|properties| {
properties.iter().any(|(key, def)| {
key == "payload" && def.r#type.as_ref().is_some_and(|t| t == "array")
key == "payload"
&& def.r#type.as_ref().is_some_and(|t| {
let typ = t.get().trim();
typ == "array" || (typ.starts_with('[') && typ.ends_with(']'))
})
})
})
}) =>
backend/windmill-api/src/users.rs
+7 -8
View File
@@ -494,14 +494,13 @@ async fn list_user_usage(
UserWithUsage,
"
SELECT usr.email, usage.executions
FROM usr
, LATERAL (
SELECT COALESCE(SUM(duration_ms + 1000)/1000 , 0)::BIGINT executions
FROM v2_as_completed_job
WHERE workspace_id = $1
AND job_kind NOT IN ('flow', 'flowpreview', 'flownode')
AND email = usr.email
AND now() - '1 week'::interval < created_at
FROM usr, LATERAL (
SELECT COALESCE(SUM(c.duration_ms + 1000)/1000 , 0)::BIGINT executions
FROM v2_job_completed c JOIN v2_job j USING (id)
WHERE j.workspace_id = $1
AND j.kind NOT IN ('flow', 'flowpreview', 'flownode')
AND j.permissioned_as_email = usr.email
AND now() - '1 week'::interval < j.created_at
) usage
WHERE workspace_id = $1
",
backend/windmill-api/src/workspaces.rs
+21 -6
View File
@@ -50,7 +50,7 @@ use windmill_common::{
oauth2::WORKSPACE_SLACK_BOT_TOKEN_PATH,
utils::{paginate, rd_string, require_admin, Pagination},
};
use windmill_git_sync::{handle_deployment_metadata, DeployedObject};
use windmill_git_sync::{handle_fork_branch_creation, handle_deployment_metadata, DeployedObject};
use windmill_worker::scoped_dependency_map::{DependencyMap, ScopedDependencyMap};
#[cfg(feature = "enterprise")]
@@ -163,6 +163,7 @@ pub fn workspaced_service() -> Router {
post(acknowledge_all_critical_alerts),
)
.route("/critical_alerts/mute", post(mute_critical_alerts))
.route("/create_workspace_fork_branch", post(create_workspace_fork_branch))
.route("/operator_settings", post(update_operator_settings));
#[cfg(all(feature = "stripe", feature = "enterprise"))]
@@ -3026,19 +3027,33 @@ async fn deprecated_create_workspace_fork(_authed: ApiAuthed) -> Result<String>
return Err(Error::BadRequest("This API endpoint has been relocated. Your Windmill CLI version is outdated and needs to be updated.".to_string()));
}
/// Return the uuids of the git sync jobs to create the branch before creating the fork
async fn create_workspace_fork_branch(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Path(w_id): Path<String>,
Json(nw): Json<CreateWorkspaceFork>,
) -> JsonResult<Vec<Uuid>> {
if *CLOUD_HOSTED {
return Err(Error::BadRequest(format!(
"Forking workspaces is not available on app.windmill.dev"
)));
}
Ok(Json(
handle_fork_branch_creation(&authed.email, &authed.username, &db, &w_id, &nw.id).await?,
))
}
async fn create_workspace_fork(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Path(parent_workspace_id): Path<String>,
Json(nw): Json<CreateWorkspaceFork>,
) -> Result<String> {
// if *CREATE_WORKSPACE_REQUIRE_SUPERADMIN {
// require_super_admin(&db, &authed.email).await?;
// }
if *CLOUD_HOSTED {
return Err(Error::BadRequest(format!(
"Forking workspaces is not available on Cloud"
"Forking workspaces is not available on app.windmill.dev"
)));
}
backend/windmill-common/src/cache.rs
+4 -4
View File
@@ -433,9 +433,9 @@ pub mod flow {
make_static! {
/// Flow node cache.
/// FIXME: Use `Arc<Node>` for cheap cloning.
static ref NODES: { FlowNodeId => RawData } in "flow" <= 1000;
static ref NODES: { FlowNodeId => RawData } in "flow_2" <= 1000;
/// Flow version value cache (version id => value).
static ref FLOWS: { i64 => Entry<FlowData> } in "flows" <= 1000;
static ref FLOWS: { i64 => Entry<FlowData> } in "flows_2" <= 1000;
/// Flow version lite value cache (version id => value).
static ref FLOWS_LITE: { i64 => Entry<FlowData> } in "flowslite" <= 1000;
}
@@ -567,7 +567,7 @@ pub mod script {
make_static! {
/// Scripts cache.
/// FIXME: Use `Arc<Val>` for cheap cloning.
static ref CACHE: { ScriptHash => ScriptFull } in "script" <= 1000;
static ref CACHE: { ScriptHash => ScriptFull } in "script_2" <= 1000;
}
/// Clear the script cache.
@@ -670,7 +670,7 @@ pub mod app {
make_static! {
/// App scripts cache.
static ref CACHE: { AppScriptId => Entry<ScriptData> } in "app" <= 1000;
static ref CACHE: { AppScriptId => Entry<ScriptData> } in "app_2" <= 1000;
}
/// Clear the app scripts cache.
backend/windmill-common/src/client.rs
+69 -50
View File
@@ -49,14 +49,7 @@ impl AuthedClient {
"{}/api/w/{}/oidc/token/{}",
self.base_internal_url, self.workspace, audience
);
let response = self.get(&url, vec![]).await?;
match response.status().as_u16() {
200u16 => Ok(response
.json::<String>()
.await
.context("decoding oidc token as json string")?),
_ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
}
make_basic_get_request(self, &url, None, Some("decoding oidc token as json string")).await
}
pub async fn get_resource_value<T: DeserializeOwned>(&self, path: &str) -> anyhow::Result<T> {
@@ -64,14 +57,7 @@ impl AuthedClient {
"{}/api/w/{}/resources/get_value/{}",
self.base_internal_url, self.workspace, path
);
let response = self.get(&url, vec![]).await?;
match response.status().as_u16() {
200u16 => Ok(response
.json::<T>()
.await
.context("decoding resource value as json")?),
_ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
}
make_basic_get_request(self, &url, None, Some("decoding resource value as json")).await
}
pub async fn get_variable_value(&self, path: &str) -> anyhow::Result<String> {
@@ -79,14 +65,7 @@ impl AuthedClient {
"{}/api/w/{}/variables/get_value/{}",
self.base_internal_url, self.workspace, path
);
let response = self.get(&url, vec![]).await?;
match response.status().as_u16() {
200u16 => Ok(response
.json::<String>()
.await
.context("decoding variable value as json")?),
_ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
}
make_basic_get_request(self, &url, None, Some("decoding variable value as json")).await
}
pub async fn get_resource_value_interpolated<T: DeserializeOwned>(
@@ -121,19 +100,34 @@ impl AuthedClient {
"{}/api/w/{}/jobs_u/completed/get_result/{}",
self.base_internal_url, self.workspace, path
);
let query = if let Some(json_path) = json_path {
vec![("json_path", json_path)]
} else {
vec![]
};
let response = self.get(&url, query).await?;
match response.status().as_u16() {
200u16 => Ok(response
.json::<T>()
.await
.context("decoding completed job result as json")?),
_ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
}
let query = query_from_json_path(json_path);
make_basic_get_request(
self,
&url,
Some(query),
Some("decoding completed job result as json"),
)
.await
}
pub async fn get_flow_env_by_flow_job_id<T: DeserializeOwned>(
&self,
root_job_id: &str,
var_name: &str,
json_path: Option<String>,
) -> anyhow::Result<T> {
let url = format!(
"{}/api/w/{}/jobs/flow_env_by_flow_job_id/{}/{}",
self.base_internal_url, self.workspace, root_job_id, var_name
);
let query = query_from_json_path(json_path);
make_basic_get_request(
self,
&url,
Some(query),
Some("decoding flow env variable as json"),
)
.await
}
pub async fn get_result_by_id<T: DeserializeOwned>(
@@ -146,19 +140,14 @@ impl AuthedClient {
"{}/api/w/{}/jobs/result_by_id/{}/{}",
self.base_internal_url, self.workspace, flow_job_id, node_id
);
let query = if let Some(json_path) = json_path {
vec![("json_path", json_path)]
} else {
vec![]
};
let response = self.get(&url, query).await?;
match response.status().as_u16() {
200u16 => Ok(response
.json::<T>()
.await
.context("decoding result by id as json")?),
_ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
}
let query = query_from_json_path(json_path);
make_basic_get_request(
self,
&url,
Some(query),
Some("decoding result by id as json"),
)
.await
}
pub async fn upload_s3_file<S>(
@@ -245,3 +234,33 @@ impl AuthedClient {
}
}
}
#[inline]
fn query_from_json_path(json_path: Option<String>) -> Vec<(&'static str, String)> {
json_path
.map(|json_path| vec![("json_path", json_path)])
.unwrap_or_else(|| Vec::new())
}
#[inline]
async fn make_basic_get_request<T: DeserializeOwned>(
client: &AuthedClient,
url: &str,
query: Option<Vec<(&'static str, String)>>,
context: Option<&'static str>,
) -> anyhow::Result<T> {
let response = client
.get(&url, query.unwrap_or_else(|| Vec::new()))
.await?;
match response.status().as_u16() {
200u16 => {
let json_body = response
.json::<T>()
.await
.context(context.unwrap_or("error decoding body as json"))?;
Ok(json_body)
}
_ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
}
}
backend/windmill-common/src/flows.rs
+14 -2
View File
@@ -193,6 +193,8 @@ pub struct FlowValue {
#[serde(skip_serializing_if = "Option::is_none")]
pub chat_input_enabled: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub flow_env: Option<HashMap<String, Box<RawValue>>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub notes: Option<Vec<FlowNote>>,
}
@@ -565,6 +567,10 @@ impl FlowModule {
.map_err(crate::error::to_anyhow)
}
pub fn is_ai_agent(&self) -> bool {
self.get_type().is_ok_and(|x| x == "aiagent")
}
pub fn is_simple(&self) -> bool {
//todo: flow modules could also be simple execpt for the fact that the case of having single parallel flow approval step is not handled well (Create SuspendedTimeout)
self.get_type()
@@ -677,6 +683,7 @@ pub enum InputTransform {
#[serde(default = "default_empty_string")]
expr: String,
},
Ai,
}
impl InputTransform {
@@ -695,6 +702,7 @@ impl TryFrom<UntaggedInputTransform> for InputTransform {
let input_transform = match value.type_.as_str() {
"static" => InputTransform::new_static_value(value.value.unwrap_or_else(default_null)),
"javascript" => InputTransform::new_javascript_expr(&value.expr.unwrap_or_default()),
"ai" => InputTransform::Ai,
other => {
return Err(anyhow::anyhow!(
"got value: {other} for field `type`, expected value: `static` or `javascript`"
@@ -850,6 +858,10 @@ pub struct McpToolValue {
pub exclude_tools: Vec<String>,
}
fn is_none_or_empty_vec<T>(expr: &Option<Vec<T>>) -> bool {
expr.is_none() || expr.as_ref().unwrap().is_empty()
}
#[derive(Serialize, Debug, Clone)]
#[serde(
tag = "type",
@@ -941,7 +953,7 @@ pub enum FlowModuleValue {
concurrency_time_window_s: Option<i32>,
#[serde(skip_serializing_if = "Option::is_none")]
is_trigger: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(skip_serializing_if = "is_none_or_empty_vec")]
assets: Option<Vec<AssetWithAltAccessType>>,
},
@@ -967,7 +979,7 @@ pub enum FlowModuleValue {
concurrency_time_window_s: Option<i32>,
#[serde(skip_serializing_if = "Option::is_none")]
is_trigger: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(skip_serializing_if = "is_none_or_empty_vec")]
assets: Option<Vec<AssetWithAltAccessType>>,
},
backend/windmill-common/src/lib.rs
+2
View File
@@ -53,6 +53,7 @@ pub mod job_metrics;
pub mod job_s3_helpers_ee;
#[cfg(feature = "parquet")]
pub mod job_s3_helpers_oss;
pub mod lockfiles;
#[cfg(feature = "private")]
pub mod git_sync_ee;
@@ -97,6 +98,7 @@ pub const DEFAULT_MAX_CONNECTIONS_WORKER: u32 = 5;
pub const DEFAULT_MAX_CONNECTIONS_INDEXER: u32 = 5;
pub const DEFAULT_HUB_BASE_URL: &str = "https://hub.windmill.dev";
pub const PRIVATE_HUB_MIN_VERSION: i32 = 10_000_000;
pub const SERVICE_LOG_RETENTION_SECS: i64 = 60 * 60 * 24 * 14; // 2 weeks retention period for logs
#[macro_export]
backend/windmill-common/src/lockfiles.rs
+17
View File
@@ -0,0 +1,17 @@
use crate::scripts::ScriptLang;
pub const LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT: &str = "# from requirements.txt";
pub fn is_generated_from_raw_requirements(
lang: &Option<ScriptLang>,
lock: &Option<String>,
) -> bool {
(lang.is_some_and(|v| v == ScriptLang::Bun)
&& lock
.as_ref()
.is_some_and(|v| v.contains("generatedFromPackageJson")))
|| (lang.is_some_and(|v| v == ScriptLang::Python3)
&& lock
.as_ref()
.is_some_and(|v| v.starts_with(LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT)))
}
backend/windmill-common/src/scripts.rs
+27 -14
View File
@@ -17,7 +17,7 @@ use crate::{
assets::AssetWithAltAccessType,
error::{to_anyhow, Error},
utils::http_get_from_hub,
DB, DEFAULT_HUB_BASE_URL, HUB_BASE_URL,
DB, DEFAULT_HUB_BASE_URL, HUB_BASE_URL, PRIVATE_HUB_MIN_VERSION,
};
use crate::worker::HUB_CACHE_DIR;
@@ -585,7 +585,7 @@ pub async fn get_hub_script_by_path(
&& path
.split("/")
.next()
.is_some_and(|x| x.parse::<i32>().is_ok_and(|x| x < 10_000_000))
.is_some_and(|x| x.parse::<i32>().is_ok_and(|x| x < PRIVATE_HUB_MIN_VERSION))
{
tracing::info!(
"Not found on private hub, fallback to default hub for {}",
@@ -670,10 +670,9 @@ async fn get_full_hub_script_by_path_inner(
Ok(response) => Ok(response),
Err(e) => {
if hub_base_url != DEFAULT_HUB_BASE_URL
&& path
.split("/")
.next()
.is_some_and(|x| x.parse::<i32>().is_ok_and(|x| x < 10_000_000))
&& path.split("/").next().is_some_and(|x| {
x.parse::<i32>().is_ok_and(|x| x < PRIVATE_HUB_MIN_VERSION)
})
{
// TODO: should only fallback to default hub if status is 404 (hub returns 500 currently)
tracing::info!(
@@ -733,18 +732,32 @@ pub fn hash_script(ns: &NewScript) -> i64 {
dh.finish() as i64
}
pub struct ClonedScript {
pub old_script: NewScript,
pub new_hash: i64,
}
pub async fn clone_script<'c>(
base_hash: ScriptHash,
w_id: &str,
deployment_message: Option<String>,
tx: &mut sqlx::Transaction<'c, sqlx::Postgres>,
) -> crate::error::Result<i64> {
let s =
sqlx::query_as::<_, Script>("SELECT * FROM script WHERE hash = $1 AND workspace_id = $2")
.bind(base_hash.0)
.bind(w_id)
.fetch_one(&mut **tx)
.await?;
) -> crate::error::Result<ClonedScript> {
let s = sqlx::query_as::<_, Script>(
"SELECT * FROM script WHERE hash = $1 AND workspace_id = $2 AND archived = false FOR UPDATE",
)
.bind(base_hash.0)
.bind(w_id)
.fetch_optional(&mut **tx)
.await?;
let s = if let Some(s) = s {
s
} else {
return Err(crate::error::Error::NotFound(format!(
"Non-archived script with hash {} not found",
base_hash.0
)));
};
let ns = NewScript {
path: s.path.clone(),
@@ -820,5 +833,5 @@ pub async fn clone_script<'c>(
.execute(&mut **tx)
.await?;
Ok(new_hash)
Ok(ClonedScript { old_script: ns, new_hash })
}
backend/windmill-common/src/variables.rs
+6 -3
View File
@@ -401,8 +401,7 @@ pub async fn get_reserved_variables(
value,
description: "Custom workspace environment variable".to_string(),
is_custom: true,
})
).collect()
})).collect()
}
async fn get_cached_workspace_envs(conn: &Connection, w_id: &str) -> Vec<(String, String)> {
@@ -439,7 +438,11 @@ async fn get_cached_workspace_envs(conn: &Connection, w_id: &str) -> Vec<(String
custom_envs
}
pub async fn get_variable_or_self(path: String, db: &DB, w_id: &str) -> crate::error::Result<String> {
pub async fn get_variable_or_self(
path: String,
db: &DB,
w_id: &str,
) -> crate::error::Result<String> {
if !path.starts_with("$var:") {
return Ok(path);
}
backend/windmill-common/src/worker.rs
+170 -6
View File
@@ -7,7 +7,7 @@ use regex::Regex;
use reqwest_middleware::ClientWithMiddleware;
use semver::Version;
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use serde_json::value::RawValue;
use serde_json::{json, value::RawValue};
use sqlx::{types::Json, Pool, Postgres};
use std::{
cmp::Reverse,
@@ -498,7 +498,6 @@ pub async fn store_pull_query(wc: &WorkerConfig) {
pub const TMP_DIR: &str = "/tmp/windmill";
pub const TMP_LOGS_DIR: &str = concatcp!(TMP_DIR, "/logs");
pub const TMP_MEMORY_DIR: &str = concatcp!(TMP_DIR, "/memory");
pub const HUB_CACHE_DIR: &str = concatcp!(ROOT_CACHE_DIR, "hub");
@@ -664,13 +663,11 @@ fn parse_file<T: FromStr>(path: &str) -> Option<T> {
.flatten()
}
#[derive(Copy, Clone)]
#[annotations("#")]
pub struct RubyAnnotations {
pub verbose: bool,
}
#[derive(Copy, Clone)]
#[annotations("#")]
pub struct PythonAnnotations {
pub no_cache: bool,
@@ -683,7 +680,6 @@ pub struct PythonAnnotations {
pub py313: bool,
}
#[derive(Copy, Clone)]
#[annotations("//")]
pub struct GoAnnotations {
pub go1_22_compat: bool,
@@ -699,13 +695,174 @@ pub struct TypeScriptAnnotations {
#[annotations("--")]
pub struct SqlAnnotations {
pub return_last_result: bool,
pub return_last_result: bool, // deprecated, use result_collection instead
pub result_collection: SqlResultCollectionStrategy,
}
#[annotations("#")]
pub struct BashAnnotations {
pub docker: bool,
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum SqlResultCollectionStrategy {
LastStatementAllRows,
LastStatementFirstRow,
LastStatementAllRowsScalar,
LastStatementFirstRowScalar,
AllStatementsAllRows,
AllStatementsFirstRow,
AllStatementsAllRowsScalar,
AllStatementsFirstRowScalar,
Legacy,
}
impl SqlResultCollectionStrategy {
pub fn parse(s: &str) -> Self {
use SqlResultCollectionStrategy::*;
match s {
"last_statement_all_rows" => LastStatementAllRows,
"last_statement_first_row" => LastStatementFirstRow,
"last_statement_all_rows_scalar" => LastStatementAllRowsScalar,
"last_statement_first_row_scalar" => LastStatementFirstRowScalar,
"all_statements_all_rows" => AllStatementsAllRows,
"all_statements_first_row" => AllStatementsFirstRow,
"all_statements_all_rows_scalar" => AllStatementsAllRowsScalar,
"all_statements_first_row_scalar" => AllStatementsFirstRowScalar,
"legacy" => Legacy,
_ => SqlResultCollectionStrategy::default(),
}
}
pub fn collect_last_statement_only(&self, query_count: usize) -> bool {
use SqlResultCollectionStrategy::*;
match self {
LastStatementAllRows
| LastStatementFirstRow
| LastStatementFirstRowScalar
| LastStatementAllRowsScalar => true,
Legacy => query_count == 1,
_ => false,
}
}
pub fn collect_first_row_only(&self) -> bool {
use SqlResultCollectionStrategy::*;
match self {
LastStatementFirstRow
| LastStatementFirstRowScalar
| AllStatementsFirstRow
| AllStatementsFirstRowScalar => true,
_ => false,
}
}
pub fn collect_scalar(&self) -> bool {
use SqlResultCollectionStrategy::*;
match self {
LastStatementFirstRowScalar
| AllStatementsFirstRowScalar
| LastStatementAllRowsScalar
| AllStatementsAllRowsScalar => true,
_ => false,
}
}
// This function transforms the shape (e.g Row[][] -> Row)
// It is the responsibility of the executor to avoid fetching unnecessary statements/rows
pub fn collect(
&self,
values: Vec<Vec<Box<serde_json::value::RawValue>>>,
) -> error::Result<Box<serde_json::value::RawValue>> {
let null = || serde_json::value::RawValue::from_string("null".to_string()).unwrap();
let values = if self.collect_last_statement_only(values.len()) {
values.into_iter().rev().take(1).collect()
} else {
values
};
let values = if self.collect_first_row_only() {
values
.into_iter()
.map(|rows| rows.into_iter().take(1).collect())
.collect()
} else {
values
};
let values = if self.collect_scalar() {
values
.into_iter()
.map(|rows| {
rows.into_iter()
.map(|row| {
// Take the first value in the object
let record =
match serde_json::from_str(row.get()) {
Ok(serde_json::Value::Object(record)) => record,
Ok(_) => return Err(error::Error::ExecutionErr(
"Could not collect sql scalar value from non-object row"
.to_string(),
)),
Err(e) => {
return Err(error::Error::ExecutionErr(format!(
"Could not collect sql scalar value (failed to parse row): {}",
e
)))
}
};
let Some((_, value)) = record.iter().next() else {
return Err(error::Error::ExecutionErr(
"Could not collect sql scalar value from empty row".to_string(),
));
};
Ok(serde_json::value::RawValue::from_string(
serde_json::to_string(value).map_err(to_anyhow)?,
)
.map_err(to_anyhow)?)
})
.collect::<error::Result<Vec<_>>>()
})
.collect::<error::Result<Vec<_>>>()?
} else {
values
};
match (
self.collect_last_statement_only(values.len()),
self.collect_first_row_only(),
) {
(true, true) => {
match values
.into_iter()
.last()
.map(|rows| rows.into_iter().next())
{
Some(Some(row)) => Ok(row.clone()),
_ => Ok(null()),
}
}
(true, false) => match values.into_iter().last() {
Some(rows) => Ok(to_raw_value(&rows)),
None => Ok(null()),
},
(false, true) => {
let values = values
.into_iter()
.map(|rows| rows.into_iter().next().unwrap_or_else(null))
.collect::<Vec<_>>();
Ok(to_raw_value(&values))
}
(false, false) => Ok(to_raw_value(&values)),
}
}
}
impl Default for SqlResultCollectionStrategy {
fn default() -> Self {
SqlResultCollectionStrategy::LastStatementAllRows
}
}
/// length = 5
/// value = "foo"
/// output = "foo "
@@ -1702,6 +1859,13 @@ pub fn load_env_vars(
.collect()
}
pub fn error_to_value(err: &error::Error) -> serde_json::Value {
match err {
error::Error::JsonErr(err) => err.clone(),
_ => json!({"message": err.to_string(), "name": err.name()}),
}
}
#[derive(Clone, PartialEq, Debug)]
pub struct WorkspacedPath {
pub workspace_id: String,
backend/windmill-duckdb-ffi-internal/Cargo.lock
Generated
+2 -4
View File
@@ -416,8 +416,7 @@ checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5"
[[package]]
name = "duckdb"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a093eed1c714143b257b95fa323e38527fabf05fbf02bb0d5d2045275ffdaef"
source = "git+https://github.com/diegoimbert/duckdb-rs?branch=main#0df52a6941c9996d7ec60d585eaf6430db8c48cf"
dependencies = [
"arrow",
"cast",
@@ -704,8 +703,7 @@ checksum = "6a82ae493e598baaea5209805c49bbf2ea7de956d50d7da0da1164f9c6d28543"
[[package]]
name = "libduckdb-sys"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b93c3ff279601516f01531cadf2ccba50394fbb5f7bf685c6e6b9b07c8dca6f"
source = "git+https://github.com/diegoimbert/duckdb-rs?branch=main#0df52a6941c9996d7ec60d585eaf6430db8c48cf"
dependencies = [
"cc",
"flate2",
backend/windmill-duckdb-ffi-internal/Cargo.toml
+1 -1
View File
@@ -5,7 +5,7 @@ edition = "2024"
[dependencies]
chrono = "0.4.41"
duckdb = { version = "^1.4.1", features = ["bundled"] }
duckdb = { git = "https://github.com/diegoimbert/duckdb-rs", branch = "main", features = ["bundled"] }
rust_decimal = "1.37.2"
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
backend/windmill-duckdb-ffi-internal/README_DEV.md
+1 -1
View File
@@ -12,4 +12,4 @@ INSERT INTO t VALUES (NULL);
causes `Constraint Error: NOT NULL constraint failed: t.x` normally, but here we see `Unknown exception in ExecutorTask::Execute`. This opaque errors comes directly from the C++ DuckDB library : https://github.com/duckdb/duckdb/blob/f99fed1e0b16a842573f9dad529f6c170a004f6e/src/parallel/executor_task.cpp#L58
To solve this, we compile duckdb separately from the main backend crate and call it with FFI
To solve this, we compile duckdb separately from the main backend crate and call it with FFI. It has to be loaded dynamically, if it is loaded statically it will still share lib c++ with deno_core and cause issues.
backend/windmill-duckdb-ffi-internal/src/lib.rs
+137 -90
View File
@@ -1,11 +1,11 @@
use std::{
collections::HashMap,
ffi::{c_char, CStr, CString},
ffi::{CStr, CString, c_char, c_uint},
ptr::null_mut,
};
use duckdb::{params_from_iter, types::TimeUnit, Row};
use rust_decimal::{prelude::FromPrimitive, Decimal};
use duckdb::{Row, params_from_iter, types::TimeUnit};
use rust_decimal::{Decimal, prelude::FromPrimitive};
use serde::Deserialize;
use serde_json::value::RawValue;
@@ -27,6 +27,14 @@ pub extern "C" fn free_cstr(string: *mut c_char) -> () {
}
}
#[unsafe(no_mangle)]
pub extern "C" fn get_version() -> c_uint {
// Increment when making breaking changes to the FFI interface.
// The windmill worker will check that the version matches or else refuse to call
// the FFI functions to avoid undefined behavior.
return 1;
}
#[unsafe(no_mangle)]
pub extern "C" fn run_duckdb_ffi(
query_block_list: *const *const c_char,
@@ -36,6 +44,8 @@ pub extern "C" fn run_duckdb_ffi(
base_internal_url: *const c_char,
w_id: *const c_char,
column_order_ptr: *mut *mut c_char,
collect_last_only: bool,
collect_first_row_only: bool,
) -> *mut c_char {
let (r, column_order) = match convert_args(
query_block_list,
@@ -54,6 +64,8 @@ pub extern "C" fn run_duckdb_ffi(
token,
base_internal_url,
w_id,
collect_last_only,
collect_first_row_only,
)
},
) {
@@ -153,6 +165,8 @@ fn run_duckdb_internal<'a>(
token: &str,
base_internal_url: &str,
w_id: &str,
collect_last_only: bool,
collect_first_row_only: bool,
) -> Result<(String, Option<Vec<String>>), String> {
let conn = duckdb::Connection::open_in_memory().map_err(|e| e.to_string())?;
@@ -189,23 +203,23 @@ fn run_duckdb_internal<'a>(
))
.map_err(|e| format!("Error setting up S3 secret: {}", e.to_string()))?;
let mut result: Option<Box<RawValue>> = None;
let mut results: Vec<Vec<Box<RawValue>>> = vec![];
let mut column_order = None;
for (query_block_index, query_block) in query_block_list.enumerate() {
result = Some(
do_duckdb_inner(
&conn,
query_block,
&job_args,
query_block_index != query_block_list_count - 1,
&mut column_order,
)
.map_err(|e| e.to_string())?,
);
let result = do_duckdb_inner(
&conn,
query_block,
&job_args,
collect_last_only && query_block_index != query_block_list_count - 1,
collect_first_row_only,
&mut column_order,
)
.map_err(|e| e.to_string())?;
results.push(result);
}
let result = result.unwrap_or_else(|| RawValue::from_string("[]".to_string()).unwrap());
Ok((result.get().to_string(), column_order))
let results = serde_json::value::to_raw_value(&results).map_err(|e| e.to_string())?;
Ok((results.get().to_string(), column_order))
}
fn do_duckdb_inner(
@@ -213,8 +227,9 @@ fn do_duckdb_inner(
query: &str,
job_args: &HashMap<String, duckdb::types::Value>,
skip_collect: bool,
collect_first_row_only: bool,
column_order: &mut Option<Vec<String>>,
) -> Result<Box<RawValue>, String> {
) -> Result<Vec<Box<RawValue>>, String> {
let mut rows_vec = vec![];
let (query, job_args) = interpolate_named_args(query, &job_args);
@@ -226,17 +241,18 @@ fn do_duckdb_inner(
.map_err(|e| e.to_string())?;
if skip_collect {
return Ok(RawValue::from_string("[]".to_string()).unwrap());
return Ok(vec![]);
}
// Statement needs to be stepped at least once or stmt.column_names() will panic
let mut column_names = None;
let mut type_aliases = None;
loop {
let row = rows.next();
match row {
Ok(Some(row)) => {
// Set column names if not already set
let stmt = row.as_ref();
let column_names = match column_names.as_ref() {
Some(column_names) => column_names,
None => {
@@ -244,8 +260,20 @@ fn do_duckdb_inner(
column_names.as_ref().unwrap()
}
};
let type_aliases = match type_aliases.as_ref() {
Some(type_aliases) => type_aliases,
None => {
type_aliases = Some(
(0..stmt.column_count())
.map(|i| stmt.column_logical_type(i).get_alias())
.collect::<Vec<_>>(),
);
type_aliases.as_ref().unwrap()
}
};
let row = row_to_value(row, &column_names.as_slice()).map_err(|e| e.to_string())?;
let row = row_to_value(row, &column_names.as_slice(), &type_aliases.as_slice())
.map_err(|e| e.to_string())?;
rows_vec.push(row);
}
Ok(None) => break,
@@ -253,11 +281,14 @@ fn do_duckdb_inner(
return Err(e.to_string());
}
}
if collect_first_row_only {
break;
}
}
*column_order = column_names;
serde_json::value::to_raw_value(&rows_vec).map_err(|e| e.to_string())
Ok(rows_vec)
}
// duckdb-rs does not support named parameters,
@@ -282,83 +313,99 @@ fn interpolate_named_args<'a>(
(query, values)
}
fn row_to_value(row: &Row<'_>, column_names: &[String]) -> Result<Box<RawValue>, String> {
fn row_to_value(
row: &Row<'_>,
column_names: &[String],
type_aliases: &[Option<String>],
) -> Result<Box<RawValue>, String> {
let mut obj = serde_json::Map::new();
for (i, key) in column_names.iter().enumerate() {
let value: duckdb::types::Value = row.get(i).map_err(|e| e.to_string())?;
let json_value = match value {
duckdb::types::Value::Null => serde_json::Value::Null,
duckdb::types::Value::Boolean(b) => serde_json::Value::Bool(b),
duckdb::types::Value::TinyInt(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::SmallInt(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::Int(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::BigInt(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::HugeInt(i) => serde_json::Value::String(i.to_string()),
duckdb::types::Value::UTinyInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::USmallInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::UInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::UBigInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::Float(f) => serde_json::Value::Number(
serde_json::Number::from_f64(f as f64)
.ok_or_else(|| "Could not convert to f64".to_string())?,
),
duckdb::types::Value::Double(f) => serde_json::Value::Number(
serde_json::Number::from_f64(f)
.ok_or_else(|| "Could not convert to f64".to_string())?,
),
duckdb::types::Value::Decimal(d) => serde_json::Value::String(d.to_string()),
duckdb::types::Value::Timestamp(_, ts) => serde_json::Value::String(ts.to_string()),
duckdb::types::Value::Text(s) => serde_json::Value::String(s),
duckdb::types::Value::Blob(b) => serde_json::Value::Array(
b.into_iter()
.map(|byte| serde_json::Value::Number(byte.into()))
.collect(),
),
duckdb::types::Value::Date32(d) => serde_json::Value::Number(d.into()),
duckdb::types::Value::Time64(_, t) => serde_json::Value::String(t.to_string()),
duckdb::types::Value::Interval { months, days, nanos } => serde_json::json!({
"months": months,
"days": days,
"nanos": nanos
}),
duckdb::types::Value::List(values) => serde_json::Value::Array(
values
.into_iter()
.map(|v| serde_json::Value::String(format!("{:?}", v)))
.collect(),
),
duckdb::types::Value::Enum(e) => serde_json::Value::String(e),
duckdb::types::Value::Struct(fields) => serde_json::Value::Object(
fields
.iter()
.map(|(k, v)| (k.clone(), serde_json::Value::String(format!("{:?}", v))))
.collect(),
),
duckdb::types::Value::Array(values) => serde_json::Value::Array(
values
.into_iter()
.map(|v| serde_json::Value::String(format!("{:?}", v)))
.collect(),
),
duckdb::types::Value::Map(map) => serde_json::Value::Object(
map.iter()
.map(|(k, v)| {
(
format!("{:?}", k),
serde_json::Value::String(format!("{:?}", v)),
)
})
.collect(),
),
duckdb::types::Value::Union(value) => {
serde_json::Value::String(format!("{:?}", *value))
}
};
let type_alias = &type_aliases[i];
let json_value = duckdb_value_to_json_value(value, type_alias)?;
obj.insert(key.clone(), json_value);
}
serde_json::value::to_raw_value(&obj).map_err(|e| e.to_string())
}
fn duckdb_value_to_json_value(
value: duckdb::types::Value,
type_alias: &Option<String>,
) -> Result<serde_json::Value, String> {
let json_value = match value {
duckdb::types::Value::Null => serde_json::Value::Null,
duckdb::types::Value::Boolean(b) => serde_json::Value::Bool(b),
duckdb::types::Value::TinyInt(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::SmallInt(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::Int(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::BigInt(i) => serde_json::Value::Number(i.into()),
duckdb::types::Value::HugeInt(i) => serde_json::Value::String(i.to_string()),
duckdb::types::Value::UTinyInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::USmallInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::UInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::UBigInt(u) => serde_json::Value::Number(u.into()),
duckdb::types::Value::Float(f) => serde_json::Value::Number(
serde_json::Number::from_f64(f as f64)
.ok_or_else(|| "Could not convert to f64".to_string())?,
),
duckdb::types::Value::Double(f) => serde_json::Value::Number(
serde_json::Number::from_f64(f)
.ok_or_else(|| "Could not convert to f64".to_string())?,
),
duckdb::types::Value::Decimal(d) => serde_json::Value::String(d.to_string()),
duckdb::types::Value::Timestamp(_, ts) => serde_json::Value::String(ts.to_string()),
duckdb::types::Value::Text(s) if type_alias.as_deref().unwrap_or_default() == "JSON" => {
serde_json::from_str(&s)
.map_err(|e| format!("Error parsing JSON text: {}", e.to_string()))?
}
duckdb::types::Value::Text(s) => serde_json::Value::String(s),
duckdb::types::Value::Blob(b) => serde_json::Value::Array(
b.into_iter()
.map(|byte| serde_json::Value::Number(byte.into()))
.collect(),
),
duckdb::types::Value::Date32(d) => serde_json::Value::Number(d.into()),
duckdb::types::Value::Time64(_, t) => serde_json::Value::String(t.to_string()),
duckdb::types::Value::Interval { months, days, nanos } => serde_json::json!({
"months": months,
"days": days,
"nanos": nanos
}),
duckdb::types::Value::List(values) => serde_json::Value::Array(
values
.into_iter()
.map(|v| duckdb_value_to_json_value(v, &None))
.collect::<Result<Vec<_>, _>>()?,
),
duckdb::types::Value::Enum(e) => serde_json::Value::String(e),
duckdb::types::Value::Struct(fields) => serde_json::Value::Object(
fields
.iter()
.map(|(k, v)| duckdb_value_to_json_value(v.clone(), &None).map(|v| (k.clone(), v)))
.collect::<Result<serde_json::Map<_, _>, _>>()?,
),
duckdb::types::Value::Array(values) => serde_json::Value::Array(
values
.into_iter()
.map(|v| duckdb_value_to_json_value(v, &None))
.collect::<Result<Vec<_>, _>>()?,
),
duckdb::types::Value::Map(map) => serde_json::Value::Object(
map.iter()
.map(|(k, v)| {
let k = match k {
duckdb::types::Value::Text(s) | duckdb::types::Value::Enum(s) => s.clone(),
_ => format!("{:?}", k),
};
duckdb_value_to_json_value(v.clone(), &None).map(|v| (k, v))
})
.collect::<Result<serde_json::Map<_, _>, _>>()?,
),
duckdb::types::Value::Union(value) => serde_json::Value::String(format!("{:?}", *value)),
};
Ok(json_value)
}
fn json_value_to_duckdb_value(
json_value: &serde_json::Value,
arg_type: &str,
backend/windmill-git-sync/src/git_sync_oss.rs
+11
View File
@@ -21,3 +21,14 @@ pub async fn handle_deployment_metadata<'c>(
// Git sync is an enterprise feature and not part of the open-source version
return Ok(());
}
#[cfg(not(feature = "private"))]
pub async fn handle_fork_branch_creation<'c>(
_email: &str,
_created_by: &str,
_db: &DB,
_w_id: &str,
_fork_workspace_id: &str,
) -> Result<Vec<uuid::Uuid>> {
return Ok(vec![]);
}
backend/windmill-git-sync/src/lib.rs
+1 -1
View File
@@ -12,7 +12,7 @@ use windmill_common::{scripts::ScriptHash, DB};
pub mod git_sync_ee;
pub mod git_sync_oss;
pub use git_sync_oss::handle_deployment_metadata;
pub use git_sync_oss::{handle_deployment_metadata, handle_fork_branch_creation};
#[derive(Clone, Debug)]
pub enum DeployedObject {
backend/windmill-macros/src/lib.rs
+56 -66
View File
@@ -1,93 +1,83 @@
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, Ident, ItemStruct, Lit};
use syn::{parse_macro_input, ItemStruct, Lit, Type};
fn is_bool_type(ty: &Type) -> bool {
if let Type::Path(type_path) = ty {
if let Some(segment) = type_path.path.segments.last() {
return segment.ident == "bool";
}
}
false
}
#[proc_macro_attribute]
pub fn annotations(attr: TokenStream, item: TokenStream) -> TokenStream {
let input = parse_macro_input!(item as ItemStruct);
let name = input.ident.clone();
let fields = input
.fields
.iter()
.map(|f| f.ident.clone().unwrap())
.collect::<Vec<Ident>>();
// Match on the literal to extract the string value
// Separate fields by type
let mut bool_fields = Vec::new();
let mut custom_fields = Vec::new();
for field in input.fields.iter() {
let field_name = field.ident.clone().unwrap();
let field_type = &field.ty;
if is_bool_type(field_type) {
bool_fields.push(field_name);
} else {
custom_fields.push((field_name, field_type));
}
}
let (custom_field_names, custom_field_types): (Vec<_>, Vec<_>) =
custom_fields.into_iter().unzip();
// Parse comment literal
let comm_lit = match parse_macro_input!(attr as Lit) {
Lit::Str(lit_str) => lit_str.value(), // This will give "#" without quotes
Lit::Str(lit_str) => lit_str.value(),
_ => panic!("Expected a string literal"),
};
// Generate regex
let mut reg = format!("^{}|", &comm_lit);
{
for field in fields.iter() {
reg.push_str(&(field.to_string()));
reg.push_str("\\b");
}
reg.push_str(r#"|\w+"#);
}
// Example of generated regex:
// ^#
// |ann1\b|ann2\b|ann3\b|ann4\b
// |\w+
TokenStream::from(quote! {
#[derive(Default, Debug)]
#[derive(Default, Debug, Copy, Clone)]
#input
impl std::ops::BitOrAssign for #name{
fn bitor_assign(&mut self, rhs: Self) {
// Unfold fields
// Read more: https://docs.rs/quote/latest/quote/macro.quote.html#interpolation
#( self.#fields |= rhs.#fields; )*
}
}
impl #name {
/// Autogenerated by windmill-macros
pub fn parse(inner_content: &str) -> Self{
pub fn parse(code: &str) -> Self {
let mut res = Self::default();
lazy_static::lazy_static! {
static ref RE: regex::Regex = regex::Regex::new(#reg).unwrap();
}
// Create lines stream
let mut lines = inner_content.lines();
'outer: while let Some(line) = lines.next() {
// If comment sign(s) on the right place
let mut comms = false;
// New instance
// We will apply it if in line only annotations
let mut new = Self::default();
'inner: for (i, mat) in RE.find_iter(line).enumerate() {
let mut lines = code.lines();
match mat.as_str(){
#comm_lit if i == 0 => {
comms = true;
continue 'inner;
},
// Will expand into something like:
// "ann1" => new.ann1 = true,
// "ann2" => new.ann2 = true,
// "ann3" => new.ann3 = true,
#( stringify!(#fields) => new.#fields = true, )*
// Non annotations
_ => continue 'outer,
};
while let Some(line) = lines.next() {
if !line.starts_with(#comm_lit) {
break;
}
let line = line[#comm_lit.len()..].trim();
let (key, value) = line.split_once('=').unwrap_or((line, ""));
if !comms {
// We dont want to continue if line does not start with #
return res;
match key {
#(
stringify!(#custom_field_names) => {
if value.is_empty() {
continue;
}
res.#custom_field_names = #custom_field_types::parse(value);
}
)*
#(
stringify!(#bool_fields) => {
res.#bool_fields = true;
}
)*
_ => {
// Unknown key=value annotation
continue;
}
}
// Apply changes
res |= new;
}
res
}
}
backend/windmill-macros/tests/annotations.rs
+3 -3
View File
@@ -24,7 +24,7 @@ mod annotations_tests {
}
#[annotations("#")]
#[derive(Eq, PartialEq, Copy, Clone)]
#[derive(Eq, PartialEq)]
pub struct Annotations {
pub ann1: bool,
pub ann2: bool,
@@ -34,7 +34,7 @@ mod annotations_tests {
}
#[annotations("//")]
#[derive(Eq, PartialEq, Copy, Clone)]
#[derive(Eq, PartialEq)]
pub struct SlashedAnnotations {
pub ann1: bool,
pub ann2: bool,
@@ -43,7 +43,7 @@ mod annotations_tests {
}
#[annotations("--")]
#[derive(Eq, PartialEq, Copy, Clone)]
#[derive(Eq, PartialEq)]
pub struct MinusedAnnotations {
pub ann1: bool,
pub ann2: bool,
backend/windmill-queue/Cargo.toml
+2
View File
@@ -45,3 +45,5 @@ axum.workspace = true
serde_urlencoded.workspace = true
regex.workspace = true
backon.workspace = true
quick_cache.workspace = true
thiserror.workspace = true
backend/windmill-queue/src/jobs.rs
+659 -442
View File
File diff suppressed because it is too large Load Diff
backend/windmill-worker/Cargo.toml
+1
View File
@@ -115,6 +115,7 @@ nix.workspace = true
bytes.workspace = true
reqwest.workspace = true
reqwest-middleware.workspace = true
eventsource-stream.workspace = true
mime_guess.workspace = true
hex.workspace = true
tiberius = { workspace = true, optional = true }

Some files were not shown because too many files have changed in this diff Show More