mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-21 08:02:38 +00:00
* feat: column-level lineage for dbt from the engine's parquet index `manifest.json` carries no column-to-column edges, which is why decision 14 recorded column lineage as unavailable. The edges live in a different artifact: `dbt compile --static-analysis strict --write-index` writes `target/index/`, whose `dbt.column_lineage.parquet` holds them and whose `dbt.node_columns.parquet` holds every column of every node, typed and ordered rather than only the ones an author documented. Strict analysis rejects SQL the default accepts, so this is a separate compile with its own `--target-path`, opt-in per project via `column_lineage: true`, and best-effort throughout: a project it cannot analyze keeps exactly the graph it had, with the engine's own diagnostics in the job log. Storage mirrors `dbt_edge`: `dbt_column_edge` keyed by (path, version, job) with the same composite FK to `script` and the same sweeps. The typed column list lands in `dbt_node.column_schema`, beside `columns` rather than merged into it, so `columns` stays what the author declared. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * fix: address review findings on the dbt column-lineage pass - The workspace fork copied every other dbt sidecar table and not this one, so a fork lost its column lineage silently and could not recover it: the cloned digest covers the column edges, so a dynamic run in the fork matched it and stored nothing. - The parquet was collected whole before the edge cap applied, which is exactly the input the cap exists for — a project whose `scan` lineage is quadratic in its widest model could take the worker process down. Decoded a row at a time with the bound enforced during the decode. - The pass swallowed every error from the runner, including the job poller's cancellation and deadline, so a run that blew its timeout inside an optional annotation could still publish a graph and report success. `run_captured` now carries the exit status in its value, so only a failed COMPILE is downgraded, and the pass may spend at most half the remaining wall clock so it cannot starve the build that follows it. - `scan` edges are stored but no longer served: they are most of a project's lineage, nothing renders them, and the graph endpoint is polled by the run page. They are also the first thing the storage cap gives up now, rather than evicting the direct edges the trace draws. - `column_schema` and the column edges take the same gate as the model's SQL. A column-level view is the shape of what the author wrote, one level finer than the `ref()` graph, which is ungated only because it draws relations the caller already sees. - `graph_digest` hashes the new section only when it has edges, so a project that never asked for the pass keeps the digest it has instead of re-snapshotting on every dynamic run until it is redeployed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * fix: the editor buffer's column lineage, and three bounds that were wrong Round-2 review found four defects, all of them introduced by the round-1 fixes. - The `script_visible` gate on the column edges was copied from the node query without its `script_hash IS NULL` arm. `= NULL` is never true, so every version-less row was filtered out and an editor buffer's parse rendered its typed columns and none of their lineage — the one place the feature is meant to be used. Pinned by an assertion in `dbt_pinned_graph.rs`, which is where this class of bug already had a home. - The phase budget was handed to the poller, whose expiry is an `Err` indistinguishable from a cancellation or the job's own deadline, so a slow but valid analysis aborted the build it exists to annotate. The runner gets the full deadline again — those two must still fail the job — and the budget is a race around the whole pass, where expiring is this budget and nothing else. - The decode cap counted parquet ROWS, so `scan` and out-of-graph rows could spend it before a single drawn edge was read. It now counts what is kept, takes direct kinds in a first pass, and is handed the graph's own nodes so the budget cannot go on rows that could never be stored. - Hashing the new digest section conditionally did not preserve old digests, because an absent `column_schema` still serialized as `null` inside the nodes. It is skipped when absent instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * refactor: split the lineage pass by error contract, and read it in one query Round 3's findings were all consequences of round 1 and 2's fixes, clustered in the same two files, so this reshapes those two seams rather than patching again. The worker pass was one function being three things at once — a subprocess runner with job-lifecycle error semantics, a bounded decoder, and a best-effort degrader — which is why each fix to one perturbed another. It is now `compile_index`, which owns the JOB's semantics (only a cancellation or the job's deadline can `Err`; a non-zero exit, the output ceiling and the phase budget are outcomes), and `read_index`, which owns the ARTIFACT's and knows nothing about the job. The budget wraps the compile alone, so a decode can no longer outlive the timeout that reported the build would get the rest. The output ceiling likewise becomes a value rather than a job error, for the caller that can carry on without the tail of a compile's stdout. The column edges were read by a fourth hand-written copy of the `live`/`chosen` CTEs and the version/editor-buffer join conditions, and copying them is what dropped the `script_hash IS NULL` arm and hid every buffer parse's lineage. Both kinds of edge now come from ONE statement over a `UNION ALL`'d edge source, so those conditions exist once. The union is at the source rather than a join because column lineage can name a node pair `dbt_edge` has no row for: a model reading `{{ this }}` gets edges from itself to itself, and `parent_map` has no self-loop. The cap on the column half now sits after the scope filter, the visibility check and the graph joins — the scope moved into SQL via the existing `ScopePathFilter` — so a row the caller may not read can no longer spend it and leave an allowed project's trace short. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * refactor: serve dbt column lineage from its own endpoint The column edges rode on the folder-wide asset graph, which a run page polls, while the trace is drawn for one selected relation. That needed a cap, and a cap has to be applied after every filter that can drop a row. Keyed to the asset there is no cap: `assets/column_lineage` answers for one relation, and the caller's `scripts:read` scope and the project's visibility are decided once, for the script that owns it. Pinning to a run's snapshot or the editor's parse of its buffer costs the job-read gate, so that form is `jobs/dbt_column_lineage/{id}` — the same shape `jobs/dbt_graph/{id}` has. The worker's decode now bounds work and memory separately, and a compile stopped by the output ceiling reports as truncated rather than complete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve the owning dbt version the way the graph does The unpinned arm picked the newest live version at the path without narrowing to dbt, so a path since redeployed in another language answered with no lineage while the graph beside it still drew that project's stale nodes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate pinned column lineage on reading the project, and answer the component Four things round 5 found, three of them in code this branch rewrote: - The pinned arm resolved the version from the job and stopped there, so a share-link viewer entitled to a run got the project's column names and edges while the graph beside it still redacted `raw_code` and `column_schema`. Resolving WHICH version answers is not deciding whether the caller may read it; the version-less editor buffer keeps its exemption, having no `script` row to ask. - The answer was the whole owning project's edges. The canvas lays out the connected component of the selected relation's columns, so the rest was unrenderable weight; a recursive walk over both directions returns exactly what is drawn, and the project key travels with it so a `unique_id` two projects share cannot walk from one graph into the other. - The decode had no exit but the 4M-row backstop once its buckets were full, spending wall clock the build below does not get. - An unreadable index was reported as a missing one, sending the reader to look at their engine rather than at the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stitch the two column graphs, and walk the component in Rust Round 6's two findings, both regressions this branch introduced: - The decode returned `Continue` on the edge that FILLED the direct-edge budget, so a `scan`-only tail after it decoded to the 4M-row backstop with nowhere to put anything. The read now ends on that edge. - Seam 3 made the pipeline page choose between the dbt graph and the producer one. They share node ids — `// column total <- dbt://wh/analytics/orders.amount` mints the same `(dbt, path, column)` node dbt's own lineage does — so choosing ended a trace at the boundary in both directions. They are merged again, and a ducklake selection asks about the dbt relation its producers name so the chain continues past it. The dbt editor gets the same merge. Also: the component is walked in Rust rather than by a recursive CTE. A CTE has no index, so the recursive term rescanned the doubled edge set once per level — 1243ms against 59ms for the query alone on a 3000-model project, 11.7M rows in the plan. Same answers, same tests; end to end 1.48s to 0.73s there and 1.60s to 0.26s on a 1000-deep chain. The client stops re-asking for a component it already holds, which is most clicks within one project. The four doc sites that described a whole-project answer are rewritten around what it now is, rather than edited where they disagreed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: expand every dbt boundary a selection reaches, and only skip what was asked Round 7's findings, all in the frontend seam this branch added: - A ducklake selection seeded the dbt fetch from the FIRST boundary relation it found, so a table derived from two unconnected dbt relations expanded one and left the other a leaf — the same "stops at the boundary" symptom the round-6 fix removed, one hop further along. Every distinct boundary is fetched now and the components merged. - The component cache skipped a relation merely PRESENT in the graph in hand. A relation two projects describe has an owner row in each, and a component fetched for one carries it as an endpoint without the other's half, so that skipped the request that would have resolved the second owner. Only a relation actually asked about under this pin is skipped. - A comment still called the producer graph gated to ducklake selections after it was widened to dbt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: land dbt column lineage as storage and ingest only The API surface that draws a column trace moves to a follow-up PR, on `dbt-column-lineage-surface`. It kept generating findings — a client cache whose premise was wrong for a two-owner relation, then staleness and a lost retry from tightening it, and a seed walk that stopped at the first boundary — and the fix for the last of them is a transitive owner expansion, which has to re-apply the caller's gate to every newly discovered project. That is the same shape as the leak four reviewers caught in the pinned arm, and it wants its own review rather than being the fourth fix at the end of this one. What lands here stands on its own: the analysis pass, `dbt_column_edge`, `dbt_node.column_schema`, the engine gating and the error-contract split — plus the one user-visible half, the typed and ordered column list, which rides the asset graph the details pane already fetches and replaces a panel that could only show the columns an author had documented. Also fixes a real bug in the pass, found in review: it compiled without the build's `--full-refresh`. `is_incremental()` branches on that flag, so an incremental model reading `{{ this }}` compiles its self-join — and any `ref()` inside that branch — only when the flag is absent, and the pass was storing lineage for SQL a full-refresh run never executed. The flag now comes from one place shared with the build, and a run that overrides it gets its own graph rather than standing as the version's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: say why direct kinds get the budget without naming a view The bucketing comments explained the priority by what a trace draws, which is a forward reference now that the surface moved out. The reason stands on its own: `copy`/`mod` say the value travelled, `scan` says the column was read to produce the row and so reaches every output column of its model. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: round-9 findings on the descoped PR - The `full_refresh` helper was inserted between `selection_is_overridden` and its doc comment, so thirteen lines about `select`/`exclude` echoes documented the wrong function and the one they were written for had none. Moved below it. - The parse path ran the analysis compile and the parquet decode BEFORE the guard that returns when there is no warehouse identity, paying for both and dropping the result. Moved after it. - Three sites still described a `/column_lineage` endpoint this branch no longer has, and two user-facing strings promised a column trace it no longer renders: the panel's hint and the descriptor template now say what the flag actually buys, which is the typed column schema. - Dropped test scaffolding the removed suite left behind: a `raw_orders` node and `dbt_edge` whose only assertion re-tested pre-existing graph behaviour, and a second editor-buffer node nothing asserts on. Documented rather than fixed: an incremental model has two shapes, and which one the index holds depends on whether the target existed when the pass ran. `is_incremental()` is false with no target as well as under `--full-refresh`, and dbt has no mode that emits both — so a version's graph describes the compile that produced it, and only a re-ingesting run describes its own run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep lineage_kind in the edge key, and one answer for --full-refresh - Both unique indexes omitted `lineage_kind`, so a column that is projected AND used as a predicate for the same output column — an ordinary shape — had its `copy` and `scan` edges collapse under `ON CONFLICT DO NOTHING`, while the digest counted both. The kind is part of the fact, so it is part of the key. Edited in the migration rather than added as a second one: it has not landed. - `full_refresh` was shared between the build and the analysis pass without the `command != "test"` condition that sat at the build's call site, so the two disagreed for exactly the runs that build nothing. The condition moved inside the function, which is the point of sharing it, and the command is threaded to the pass. - The "what a trace draws" rewrite missed the copy in `dbt_manifest.rs`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the unreachable full_refresh threading, test the uniqueness key `DBT_COMMANDS` is `["build", "retry", "show", "parse"]` and `default_command` returns `build` in every arm, so `command == "test"` cannot happen — the guard the last commit moved into `full_refresh` was already inert where it came from. Threading the command through five signatures to preserve it bought nothing, so it is gone; the build and the pass call one function of the descriptor and the invocation, which is what the sharing was for. The uniqueness-key fix now has a test: a column projected AND used as a predicate for the same output column stores both its `copy` and its `scan` row. Verified against the old key, where it returns 1 instead of 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: restore the dbt test --full-refresh guard I removed on a wrong premise The previous commit removed it after reading `DBT_COMMANDS` and concluding `"test"` was unreachable. That is only true of the command a CALLER can name: `run_dbt` is invoked with `"test"` directly for the `after_all` test phase, so an `after_all` project with `full_refresh: true` reached it — and dbt rejects `--full-refresh` on `test`, failing the phase. Both reviewers caught it. The guard is back inside the shared function, where the build and the pass get one answer, and its doc now records why reading the allowlist alone is misleading. The test covering the `test` case is restored with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: notice a job that ended during the decode, and name truncation as the cause - The parquet decode runs on a blocking thread with no poller watching it, so a cancellation or an expired deadline during it was invisible: `dbt_dep` went on to publish the graph and the job returned success. The job's state is checked once the decode returns, before the caller publishes anything, and an ended job `Err`s — which this module may always do for the job's own semantics. - A compile stopped by the output ceiling could leave no artifact, and the log then blamed the engine's capability, sending the reader to check their adapter rather than the ceiling. Truncation now names itself in the missing and unreadable branches too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read cancellation from the DB after the decode, not from a poller's field `ctx.canceled_by` is only ever written by a poller, and no poller runs during the blocking decode — which is the exact window the check was added for. So the guard caught only a cancellation already observed before it, and the comment beside it claimed more than it did. It now queries `v2_job_queue` directly, the same probe `worker_lockfiles` uses before it overwrites a flow. A failed probe answers "still running": this decides whether to discard work already done, so an unreachable database must not be the reason a healthy deploy loses its graph. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: reuse job_is_canceled rather than a second copy of it The probe added last round was `job_is_canceled` from the same file, retyped — same query, same `Connection::Http` behaviour. Reused instead. Its doc said a non-database connection was "a failed probe", which reads as an error path. It is not: it is the agent worker, and on one there is no database to ask, so only the deadline answers and a cancel issued during the decode is not observable. The retry path avoids that by refusing to run on an agent worker at all — which an optional annotation has no business doing — so the gap is recorded at both ends instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: close the agent-worker cancellation gap instead of documenting it The previous commit said a cancel issued during the decode is not observable on an agent worker. It is: `ping_job_status` returns `canceled_by` over both connection kinds, and is how the poller itself notices one there. So the check asks through the ping rather than querying `v2_job_queue` directly, and holds on an agent worker, where a direct query reaches no database at all. `job_is_canceled` goes back to private and its doc to what it said before — the retry that calls it still refuses to run on an agent worker for its own reasons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: decode the index under the job poller instead of checking after it Two findings with one cause: the decode was the only phase of this pass with no subprocess behind it, so nothing heartbeated while it ran. A large index left the worker silent for as long as it took, which the zombie sweep reads as a dead job and restarts — and the cancellation check bolted on afterwards could only ever report what had already happened, while dropping the ping's `already_completed`, so a force-cancelled deploy still published its graph. Running it under `run_future_with_polling_update_job_poller` answers all of it: the poller pings throughout, and ends the phase with an `Err` on cancellation, `AlreadyCompleted` or the phase timeout. The bespoke probe is gone with it. Verified on a live deploy: 32 edges and 4 typed schemas ingested through the polled decode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop a cancelled decode, and say what the read phase can now do Putting the decode under the poller heartbeats it and ends the phase when the job does, but dropping a `JoinHandle` detaches a blocking task rather than cancelling it — so a cancelled job left a thread decoding up to four million rows for a job that was over. The row loop reads an abandonment flag that a drop guard on the awaiting future sets, so the decode stops at its next row. That same change made the read phase able to `Err`, and three places still said it could not — decision 14 in as many words. The distinction that holds is narrower: nothing the ARTIFACT does or fails to do can fail a job, so absent, unreadable and partial are all values; the JOB can still end the phase the read runs in. Stated that way in the module doc, the `Artifact` doc, `MAX_INDEX_ROWS` and the decision. Verified on a live deploy: 32 edges and 4 typed schemas. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: share AbortOnDrop, and stop citing a hazard that is now handled `Abandon` was `ansible_executor`'s `AbortOnDrop` retyped — same struct, same reason, same `spawn_blocking` shape. Moved to `common` and used from both. The paragraph explaining why the phase budget wraps the compile alone gave as its reason "a decode still running on a blocking thread", which is exactly what the abandonment flag now prevents. The reason that survives is the one that was always the point: the budget exists to leave the build its share of the clock, and only the compile can spend that share unboundedly. The decode's end is the job's, through the poller it runs under. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: put both doc comments back on the items they describe Moving AbortOnDrop orphaned a doc at each end: it landed between `raw_to_string`'s doc and `raw_to_string`, and the doc of the struct it replaced stayed behind to prefix `fetch_repo_archive`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: name the binding the row loop actually reads `Abandoned` was neither the type nor the binding; the flag is `abandoned`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1848 lines
54 KiB
TypeScript
1848 lines
54 KiB
TypeScript
import { type Script } from './gen'
|
|
|
|
import type { SupportedLanguage } from './common'
|
|
|
|
import CLAUDE_SANDBOX_INIT_CODE from './templates/claude_sandbox.ts.template?raw'
|
|
import WAC_PYTHON_INIT_CODE from './templates/wac_python.py.template?raw'
|
|
import WAC_TYPESCRIPT_INIT_CODE from './templates/wac_typescript.ts.template?raw'
|
|
import CI_TEST_BUN_INIT_CODE from './templates/ci_test_bun.ts.template?raw'
|
|
import CI_TEST_PYTHON_INIT_CODE from './templates/ci_test_python.py.template?raw'
|
|
|
|
const PYTHON_FAILURE_MODULE_CODE = `import os
|
|
|
|
def main(message: str, name: str, step_id: str):
|
|
flow_id = os.environ.get("WM_ROOT_FLOW_JOB_ID")
|
|
print("message", message)
|
|
print("name", name)
|
|
print("step_id", step_id)
|
|
return { "message": message, "flow_id": flow_id, "step_id": step_id, "recover": False }`
|
|
|
|
const PYTHON_INIT_CODE_CLEAR = `# import wmill
|
|
|
|
|
|
def main(x: str):
|
|
return x`
|
|
|
|
const PYTHON_INIT_CODE_TRIGGER = `import wmill
|
|
|
|
|
|
def main():
|
|
# A common trigger script would follow this pattern:
|
|
# 1. Get the last saved state
|
|
# state = wmill.get_state()
|
|
# 2. Get the actual state from the external service
|
|
# newState = ...
|
|
# 3. Compare the two states and update the internal state
|
|
# wmill.setState(newState)
|
|
# 4. Return the new rows
|
|
# return range from (state to newState)
|
|
#
|
|
# For more complex states, consider using Data Tables:
|
|
# https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables
|
|
return [1, 2, 3]`
|
|
|
|
const PYTHON_INIT_CODE = `import os
|
|
import wmill
|
|
|
|
# You can import any PyPi package.
|
|
# See here for more info: https://www.windmill.dev/docs/advanced/dependencies_in_python
|
|
|
|
# you can use typed resources by doing a type alias to dict
|
|
#postgresql = dict
|
|
|
|
def main(
|
|
no_default: str,
|
|
#db: postgresql,
|
|
name="Nicolas Bourbaki",
|
|
age=42,
|
|
obj: dict = {"even": "dicts"},
|
|
l: list = ["or", "lists!"],
|
|
file_: bytes = bytes(0),
|
|
):
|
|
|
|
print(f"Hello World and a warm welcome especially to {name}")
|
|
print("and its acolytes..", age, obj, l, len(file_))
|
|
|
|
# retrieve variables, resources, states using the wmill client
|
|
try:
|
|
secret = wmill.get_variable("f/examples/secret")
|
|
except:
|
|
secret = "No secret yet at f/examples/secret !"
|
|
print(f"The variable at \`f/examples/secret\`: {secret}")
|
|
|
|
# Get last state of this script execution by the same trigger/user
|
|
last_state = wmill.get_state()
|
|
new_state = {"foo": 42} if last_state is None else last_state
|
|
new_state["foo"] += 1
|
|
wmill.set_state(new_state)
|
|
|
|
# fetch context variables
|
|
user = os.environ.get("WM_USERNAME")
|
|
|
|
# return value is converted to JSON
|
|
return {"splitted": name.split(), "user": user, "state": new_state}`
|
|
|
|
const NATIVETS_INIT_CODE = `// Fetch-only script, no imports allowed (except windmill) but benefits from a dedicated highly efficient runtime
|
|
//import * as wmill from './windmill.ts'
|
|
|
|
export async function main(example_input: number = 3) {
|
|
// "3" is the default value of example_input, it can be overriden with code or using the UI
|
|
const res = await fetch(\`https://jsonplaceholder.typicode.com/todos/\${example_input}\`, {
|
|
headers: { "Content-Type": "application/json" },
|
|
});
|
|
return res.json();
|
|
}
|
|
`
|
|
|
|
const BUNNATIVE_INIT_CODE = `//native
|
|
//you can add proxy support using //proxy http(s)://host:port
|
|
|
|
// native scripts are bun scripts that are executed on native workers and can be parallelized
|
|
// only fetch is allowed, but imports will work as long as they also use only fetch and the standard lib
|
|
|
|
//import * as wmill from "windmill-client"
|
|
|
|
export async function main(example_input: number = 3) {
|
|
// "3" is the default value of example_input, it can be overriden with code or using the UI
|
|
const res = await fetch(\`https://jsonplaceholder.typicode.com/todos/\${example_input}\`, {
|
|
headers: { "Content-Type": "application/json" },
|
|
});
|
|
return res.json();
|
|
}
|
|
`
|
|
|
|
const NATIVETS_INIT_CODE_CLEAR = `// Fetch-only script, no imports allowed (except windmill) but benefits from a dedicated highly efficient runtime
|
|
//import * as wmill from './windmill.ts'
|
|
|
|
export async function main() {
|
|
const res = await fetch("https://jsonplaceholder.typicode.com/todos/1", {
|
|
headers: { "Content-Type": "application/json" },
|
|
});
|
|
return res.json();
|
|
}
|
|
`
|
|
|
|
const DENO_INIT_BLOCK = `// Ctrl/CMD+. to cache dependencies on imports hover.
|
|
|
|
// Deno uses "npm:" prefix to import from npm (https://deno.land/manual@v1.36.3/node/npm_specifiers)
|
|
// import * as wmill from "npm:windmill-client@${__pkg__.version}"
|
|
|
|
// fill the type, or use the +Resource type to get a type-safe reference to a resource
|
|
// type Postgresql = object`
|
|
|
|
const DENO_INIT_CODE =
|
|
DENO_INIT_BLOCK +
|
|
`
|
|
|
|
export async function main(
|
|
a: number,
|
|
b: "my" | "enum",
|
|
//c: Postgresql,
|
|
d = "inferred type string from default arg",
|
|
e = { nested: "object" },
|
|
//e: wmill.Base64
|
|
) {
|
|
// let x = await wmill.getVariable('u/user/foo')
|
|
return { foo: a };
|
|
}
|
|
`
|
|
|
|
const BUN_INIT_BLOCK = `// there are multiple modes to add as header: //nobundling //native //npm //nodejs
|
|
// https://www.windmill.dev/docs/getting_started/scripts_quickstart/typescript#modes
|
|
|
|
// import { toWords } from "number-to-words@1"
|
|
import * as wmill from "windmill-client"
|
|
|
|
// fill the type, or use the +Resource type to get a type-safe reference to a resource
|
|
// type Postgresql = object`
|
|
|
|
const BUN_INIT_CODE =
|
|
BUN_INIT_BLOCK +
|
|
`
|
|
|
|
|
|
export async function main(
|
|
a: number,
|
|
b: "my" | "enum",
|
|
//c: Postgresql,
|
|
//d: wmill.S3Object, // https://www.windmill.dev/docs/core_concepts/persistent_storage/large_data_files
|
|
//d: DynSelect_foo, // https://www.windmill.dev/docs/core_concepts/json_schema_and_parsing#dynamic-select
|
|
e = "inferred type string from default arg",
|
|
f = { nested: "object" },
|
|
g: {
|
|
label: "Variant 1",
|
|
foo: string
|
|
} | {
|
|
label: "Variant 2",
|
|
bar: number
|
|
}
|
|
) {
|
|
// let x = await wmill.getVariable('u/user/foo')
|
|
return { foo: a };
|
|
}
|
|
`
|
|
|
|
const GO_INIT_CODE = `package inner
|
|
|
|
import (
|
|
"fmt"
|
|
"rsc.io/quote"
|
|
// wmill "github.com/windmill-labs/windmill-go-client"
|
|
)
|
|
|
|
// Pin dependencies partially in go.mod with a comment starting with "//require":
|
|
//require rsc.io/quote v1.5.1
|
|
|
|
// the main must return (interface{}, error)
|
|
|
|
func main(x string, nested struct {
|
|
Foo string \`json:"foo"\`
|
|
}) (interface{}, error) {
|
|
fmt.Println("Hello, World")
|
|
fmt.Println(nested.Foo)
|
|
fmt.Println(quote.Opt())
|
|
// v, _ := wmill.GetVariable("f/examples/secret")
|
|
return x, nil
|
|
}
|
|
`
|
|
|
|
const GO_FAILURE_MODULE_CODE = `package inner
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
)
|
|
|
|
// connect the error parameter to 'previous_result.error'
|
|
|
|
func main(message string, name string) (interface{}, error) {
|
|
fmt.Println(message)
|
|
fmt.Println(name)
|
|
fmt.Println("flow id that failed", os.Getenv("WM_FLOW_JOB_ID"))
|
|
return message, nil
|
|
}
|
|
`
|
|
|
|
const DENO_INIT_CODE_CLEAR = `// import * as wmill from "npm:windmill-client@${__pkg__.version}"
|
|
|
|
export async function main(x: string) {
|
|
return x
|
|
}
|
|
`
|
|
|
|
const BUN_INIT_CODE_CLEAR = `// import * as wmill from "windmill-client"
|
|
|
|
export async function main(x: string) {
|
|
return x
|
|
}
|
|
`
|
|
|
|
const DENO_FAILURE_MODULE_CODE = `
|
|
export async function main(message: string, name: string, step_id: string) {
|
|
const flow_id = Deno.env.get("WM_ROOT_FLOW_JOB_ID")
|
|
console.log("message", message)
|
|
console.log("name",name)
|
|
console.log("step_id", step_id)
|
|
return { message, flow_id, step_id, recover: false }
|
|
}
|
|
`
|
|
|
|
const BUN_FAILURE_MODULE_CODE = `
|
|
export async function main(message: string, name: string, step_id: string) {
|
|
const flow_id = process.env.WM_ROOT_FLOW_JOB_ID
|
|
console.log("message", message)
|
|
console.log("name",name)
|
|
console.log("step_id", step_id)
|
|
return { message, flow_id, step_id, recover: false }
|
|
}
|
|
`
|
|
|
|
const POSTGRES_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- to pin the database use '-- database f/your/path'
|
|
-- to stream a large query result to your workspace storage use '-- s3'
|
|
-- to feed an S3Object (json/jsonl/parquet/csv) as a parameter, declare it as (s3object):
|
|
-- -- $5 input_file (s3object)
|
|
-- INSERT INTO demo SELECT * FROM jsonb_to_recordset(\$5::jsonb) AS x(id INT, name TEXT);
|
|
-- $1 name1 = default arg
|
|
-- $2 name2
|
|
-- $3 name3
|
|
-- $4 name4
|
|
INSERT INTO demo VALUES (\$1::TEXT, \$2::INT, \$3::TEXT[]) RETURNING *;
|
|
UPDATE demo SET col2 = \$4::INT WHERE col2 = \$2::INT;
|
|
`
|
|
|
|
const MYSQL_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- to pin the database use '-- database f/your/path'
|
|
-- to stream a large query result to your workspace storage use '-- s3'
|
|
-- to feed an S3Object (json/jsonl/parquet/csv) as a parameter, declare it as (s3object):
|
|
-- -- :input_file (s3object)
|
|
-- INSERT INTO demo SELECT * FROM JSON_TABLE(:input_file, '$[*]' COLUMNS (id INT PATH '$.id', name VARCHAR(255) PATH '$.name')) AS x;
|
|
-- :name1 (text) = default arg
|
|
-- :name2 (int)
|
|
-- :name3 (int)
|
|
INSERT INTO demo VALUES (:name1, :name2);
|
|
UPDATE demo SET col2 = :name3 WHERE col2 = :name2;
|
|
`
|
|
|
|
const BIGQUERY_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- to pin the database use '-- database f/your/path'
|
|
-- to stream a large query result to your workspace storage use '-- s3'
|
|
-- to feed an S3Object (json/jsonl/parquet/csv) as a parameter, declare it as (s3object):
|
|
-- -- @input_file (s3object)
|
|
-- SELECT * FROM UNNEST(JSON_QUERY_ARRAY(@input_file)) AS row;
|
|
-- @name1 (string) = default arg
|
|
-- @name2 (integer)
|
|
-- @name3 (string[])
|
|
-- @name4 (integer)
|
|
INSERT INTO \`demodb.demo\` VALUES (@name1, @name2, @name3);
|
|
UPDATE \`demodb.demo\` SET col2 = @name4 WHERE col2 = @name2;
|
|
`
|
|
|
|
const ORACLEDB_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- to pin the database use '-- database f/your/path'
|
|
-- to stream a large query result to your workspace storage use '-- s3'
|
|
-- :name1 (text) = default arg
|
|
-- :name2 (int)
|
|
-- :name3 (int)
|
|
INSERT INTO demo VALUES (:name1, :name2);
|
|
UPDATE demo SET col2 = :name3 WHERE col2 = :name2;
|
|
`
|
|
|
|
const SNOWFLAKE_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- to pin the database use '-- database f/your/path'
|
|
-- to stream a large query result to your workspace storage use '-- s3'
|
|
-- to feed an S3Object (json/jsonl/parquet/csv) as a parameter, declare it as (s3object):
|
|
-- -- ? input_file (s3object)
|
|
-- SELECT v.value:id::int AS id, v.value:name::string AS name
|
|
-- FROM TABLE(FLATTEN(input => PARSE_JSON(?))) v;
|
|
-- ? name1 (varchar) = default arg
|
|
-- ? name2 (int)
|
|
INSERT INTO demo VALUES (?, ?);
|
|
-- ? name3 (int)
|
|
-- ? name2 (int)
|
|
UPDATE demo SET col2 = ? WHERE col2 = ?;
|
|
`
|
|
|
|
const MSSQL_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- to pin the database use '-- database f/your/path'
|
|
-- to stream a large query result to your workspace storage use '-- s3'
|
|
-- to feed an S3Object (json/jsonl/parquet/csv) as a parameter, declare it as (s3object):
|
|
-- -- @P4 input_file (s3object)
|
|
-- INSERT INTO demo
|
|
-- SELECT id, name FROM OPENJSON(@P4) WITH (id INT '$.id', name NVARCHAR(255) '$.name');
|
|
-- @P1 name1 (varchar) = default arg
|
|
-- @P2 name2 (int)
|
|
-- @P3 name3 (int)
|
|
INSERT INTO demo VALUES (@P1, @P2);
|
|
UPDATE demo SET col2 = @P3 WHERE col2 = @P2;
|
|
`
|
|
|
|
const DUCKDB_INIT_CODE = `-- result_collection=last_statement_all_rows
|
|
-- $name (text) = Ben
|
|
-- $age (text) = 20
|
|
-- -- $friends_csv (s3object)
|
|
|
|
-- Click the +Database button to connect to a database
|
|
-- https://www.windmill.dev/docs/getting_started/scripts_quickstart/sql#duckdb-1
|
|
--
|
|
-- ATTACH '$res:u/demo/amazed_postgresql' AS db (TYPE postgres);
|
|
-- SELECT * FROM db.public.friends;
|
|
|
|
-- Click the +Ducklake button to use a ducklake
|
|
-- https://www.windmill.dev/docs/core_concepts/persistent_storage/ducklake
|
|
--
|
|
-- ATTACH 'ducklake' AS dl;
|
|
-- USE dl;
|
|
-- SELECT * FROM customers;
|
|
|
|
CREATE TABLE friends (
|
|
name text,
|
|
age int
|
|
);
|
|
|
|
INSERT INTO friends VALUES ($name, $age);
|
|
-- INSERT INTO friends
|
|
-- SELECT name, age FROM read_csv($friends_csv);
|
|
|
|
SELECT * FROM friends;
|
|
`
|
|
|
|
const GRAPHQL_INIT_CODE = `query($name4: String, $name2: Int, $name3: [String]) {
|
|
demo(name1: $name1, name2: $name2, name3: $name3) {
|
|
name1,
|
|
name2,
|
|
name3
|
|
}
|
|
}
|
|
`
|
|
|
|
const PHP_INIT_CODE = `<?php
|
|
|
|
// remove the first // of the following lines to specify packages to install using composer
|
|
// // require:
|
|
// // monolog/monolog@3.6.0
|
|
// // stripe/stripe-php
|
|
|
|
function main(
|
|
// Postgresql $a,
|
|
// array $b,
|
|
// object $c,
|
|
int $d = 123,
|
|
string $e = "default value",
|
|
float $f = 3.5,
|
|
bool $g = true,
|
|
) {
|
|
return $d;
|
|
}
|
|
`
|
|
|
|
const RUST_INIT_CODE = `//! Add dependencies in the following partial Cargo.toml manifest
|
|
//!
|
|
//! \`\`\`cargo
|
|
//! [dependencies]
|
|
//! anyhow = "1.0.86"
|
|
//! rand = "0.7.2"
|
|
//! \`\`\`
|
|
//!
|
|
//! Note that serde is used by default with the \`derive\` feature.
|
|
//! You can still reimport it if you need additional features.
|
|
|
|
use anyhow::anyhow;
|
|
use rand::seq::SliceRandom;
|
|
use serde::Serialize;
|
|
|
|
#[derive(Serialize, Debug)]
|
|
struct Ret {
|
|
msg: String,
|
|
number: i8,
|
|
}
|
|
|
|
fn main(who_to_greet: String, numbers: Vec<i8>) -> anyhow::Result<Ret> {
|
|
println!(
|
|
"Person to greet: {} - numbers to choose: {:?}",
|
|
who_to_greet, numbers
|
|
);
|
|
Ok(Ret {
|
|
msg: format!("Greetings {}!", who_to_greet),
|
|
number: *numbers
|
|
.choose(&mut rand::thread_rng())
|
|
.ok_or(anyhow!("There should be some numbers to choose from"))?,
|
|
})
|
|
}
|
|
`
|
|
|
|
const CSHARP_INIT_CODE = `#r "nuget: Humanizer, 2.14.1"
|
|
|
|
using System;
|
|
using System.Linq;
|
|
using Humanizer;
|
|
|
|
|
|
class Script
|
|
{
|
|
public static int Main(string[] extraWords, string word = "clue", int highNumberThreshold = 50)
|
|
{
|
|
Console.WriteLine("Hello, World!");
|
|
|
|
Console.WriteLine("Your chosen words are pluralized here:");
|
|
|
|
string[] newWordArray = extraWords.Concat(new[] { word }).ToArray();
|
|
|
|
foreach (var s in newWordArray)
|
|
{
|
|
Console.WriteLine($" {s.Pluralize()}");
|
|
}
|
|
|
|
var random = new Random();
|
|
int randomNumber = random.Next(1, 101);
|
|
|
|
Console.WriteLine($"Random number: {randomNumber}");
|
|
|
|
string greeting = randomNumber > highNumberThreshold ? "High number!" : "Low number!";
|
|
greeting += " (according to the threshold parameter)";
|
|
Console.WriteLine(greeting);
|
|
// Humanize a timespan
|
|
var timespan = TimeSpan.FromMinutes(90);
|
|
Console.WriteLine($"Timespan: {timespan.Humanize()}");
|
|
|
|
// Humanize numbers into words
|
|
int number = 123;
|
|
Console.WriteLine($"Number: {number.ToWords()}");
|
|
|
|
// Pluralize words
|
|
string singular = "apple";
|
|
|
|
// Humanize date difference
|
|
var date = DateTime.UtcNow.AddDays(-3);
|
|
Console.WriteLine($"Date: {date.Humanize()}");
|
|
return 2;
|
|
}
|
|
}
|
|
`
|
|
|
|
const NU_INIT_CODE = `use std assert
|
|
|
|
# Nushell
|
|
# A new type of shell
|
|
def main [
|
|
no_default: string,
|
|
name = "Nicolas Bourbaki",
|
|
age: int = 42,
|
|
date_of_birth?: datetime,
|
|
obj: record = {"records": "included"},
|
|
l: list<string> = ["or", "lists!"],
|
|
tables?: table,
|
|
enable_kill_mode?: bool = true,
|
|
] {
|
|
# Test
|
|
# https://www.nushell.sh/book/testing.html
|
|
assert ($age == 42)
|
|
|
|
print $"Hello World and a warm welcome especially to ($name)"
|
|
print "and its acolytes.." $age $obj $l
|
|
print $tables
|
|
|
|
let secret = try {
|
|
get_variable f/examples/secret
|
|
} catch {
|
|
'No secret yet at f/examples/secret !'
|
|
};
|
|
|
|
print $"The variable at \`f/examples/secret\`: ($secret)"
|
|
# fetch context variables
|
|
let user = $env.WM_USERNAME
|
|
|
|
# Nu pipelines
|
|
ls | where size > 1kb | sort-by modified | print "ls:" $in
|
|
|
|
# Nu works with existing data
|
|
# Nu speaks JSON, YAML, SQLite, Excel, and more out of the box.
|
|
# It's easy to bring data into a Nu pipeline whether it's in a file, a database, or a web API:
|
|
let nu_license = http get https://api.github.com/repos/nushell/nushell | get license
|
|
|
|
return { splitted: ($name | split words), user: $user, nu_license: $nu_license}
|
|
# Interested in learning more?
|
|
# https://www.nushell.sh/book/getting_started.html
|
|
}
|
|
`
|
|
|
|
const FETCH_INIT_CODE = `export async function main(
|
|
url: string | undefined,
|
|
method: 'GET' | 'POST' | 'PUT' | 'DELETE' | 'PATCH' | 'OPTIONS' = 'GET',
|
|
body: Object = {},
|
|
headers: Record<string, string> = {}
|
|
): Promise<Response | null> {
|
|
if (!url) {
|
|
console.error('Error: URL is undefined')
|
|
return null
|
|
}
|
|
|
|
const requestOptions: RequestInit = {
|
|
method: method || 'GET',
|
|
headers: headers || {}
|
|
}
|
|
|
|
if (requestOptions.method !== 'GET' && requestOptions.method !== 'HEAD' && body !== undefined) {
|
|
requestOptions.body = JSON.stringify(body)
|
|
requestOptions.headers = {
|
|
'Content-Type': 'application/json',
|
|
...requestOptions.headers
|
|
}
|
|
}
|
|
|
|
return await fetch(url, requestOptions)
|
|
.then((res) => res.json())
|
|
.catch(() => {
|
|
throw new Error('An error occurred')
|
|
})
|
|
}`
|
|
|
|
const BASH_INIT_CODE = `# shellcheck shell=bash
|
|
# arguments of the form X="$I" are parsed as parameters X of type string
|
|
msg="$1"
|
|
dflt="\${2:-default value}"
|
|
|
|
# the last line of the stdout is the return value
|
|
# unless you write json to './result.json' or a string to './result.out'
|
|
echo "Hello $msg"
|
|
`
|
|
|
|
const DENO_INIT_CODE_TRIGGER = `import * as wmill from "npm:windmill-client@${__pkg__.version}"
|
|
|
|
export async function main() {
|
|
|
|
// A common trigger script would follow this pattern:
|
|
// 1. Get the last saved state
|
|
// const state = await wmill.getState()
|
|
// 2. Get the actual state from the external service
|
|
// const newState = await (await fetch('https://hacker-news.firebaseio.com/v0/topstories.json')).json()
|
|
// 3. Compare the two states and update the internal state
|
|
// await wmill.setState(newState)
|
|
// 4. Return the new rows
|
|
// return range from (state to newState)
|
|
//
|
|
// For more complex states, consider using Data Tables:
|
|
// https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables
|
|
|
|
return [1,2,3]
|
|
|
|
// In subsequent scripts, you may refer to each row/value returned by the trigger script using
|
|
// 'flow_input.iter.value'
|
|
}
|
|
`
|
|
|
|
const BUN_INIT_CODE_TRIGGER = `import * as wmill from "windmill-client"
|
|
|
|
export async function main() {
|
|
|
|
// A common trigger script would follow this pattern:
|
|
// 1. Get the last saved state
|
|
// const state = await wmill.getState()
|
|
// 2. Get the actual state from the external service
|
|
// const newState = await (await fetch('https://hacker-news.firebaseio.com/v0/topstories.json')).json()
|
|
// 3. Compare the two states and update the internal state
|
|
// await wmill.setState(newState)
|
|
// 4. Return the new rows
|
|
// return range from (state to newState)
|
|
//
|
|
// For more complex states, consider using Data Tables:
|
|
// https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables
|
|
|
|
return [1,2,3]
|
|
|
|
// In subsequent scripts, you may refer to each row/value returned by the trigger script using
|
|
// 'flow_input.iter.value'
|
|
}
|
|
`
|
|
|
|
const GO_INIT_CODE_TRIGGER = `package inner
|
|
|
|
import (
|
|
wmill "github.com/windmill-labs/windmill-go-client"
|
|
)
|
|
|
|
func main() (interface{}, error) {
|
|
|
|
// A common trigger script would follow this pattern:
|
|
// 1. Get the last saved state
|
|
state, _ := wmill.GetState()
|
|
// 2. Get the actual state from the external service
|
|
// newState := ...
|
|
// 3. Compare the two states and update the internal state
|
|
wmill.SetState(4)
|
|
// 4. Return the new rows
|
|
//
|
|
// For more complex states, consider using Data Tables:
|
|
// https://www.windmill.dev/docs/core_concepts/persistent_storage/data_tables
|
|
|
|
return state, nil
|
|
|
|
// In subsequent scripts, you may refer to each row/value returned by the trigger script using
|
|
// 'flow_input.iter.value'
|
|
}
|
|
`
|
|
|
|
const DENO_INIT_CODE_APPROVAL = `import * as wmill from "npm:windmill-client@^1.158.2"
|
|
|
|
export async function main(approver?: string) {
|
|
const urls = await wmill.getResumeUrls(approver)
|
|
// send the urls to their intended recipients
|
|
|
|
return {
|
|
// if the resumeUrls are part of the response, they will be available to any persons having access
|
|
// to the run page and allowed to be approved from there, even from non owners of the flow
|
|
// self-approval is disableable in the suspend options
|
|
...urls,
|
|
|
|
// to have prompts (self-approvable steps), clude instead the resume url in the returned payload of the step
|
|
// the UX will automatically adapt and show the prompt to the operator when running the flow. e.g:
|
|
// resume: urls['resume'],
|
|
|
|
default_args: {},
|
|
enums: {},
|
|
description: undefined
|
|
// supports all formats from rich display rendering such as simple strings,
|
|
// but also markdown, html, images, tables, maps, render_all, etc...
|
|
// https://www.windmill.dev/docs/core_concepts/rich_display_rendering
|
|
}
|
|
}
|
|
|
|
// add a form in Advanced - Suspend
|
|
// all on approval steps: https://www.windmill.dev/docs/flows/flow_approval`
|
|
|
|
const BUN_INIT_CODE_APPROVAL = `import * as wmill from "windmill-client@^1.158.2"
|
|
|
|
export async function main(approver?: string) {
|
|
const urls = await wmill.getResumeUrls(approver)
|
|
// send the urls to their intended recipients
|
|
|
|
return {
|
|
// if the resumeUrls are part of the response, they will be available to any persons having access
|
|
// to the run page and allowed to be approved from there, even from non owners of the flow
|
|
// self-approval is disableable in the suspend options
|
|
...urls,
|
|
|
|
// to have prompts (self-approvable steps), clude instead the resume url in the returned payload of the step
|
|
// the UX will automatically adapt and show the prompt to the operator when running the flow. e.g:
|
|
// resume: urls['resume'],
|
|
|
|
default_args: {},
|
|
enums: {},
|
|
description: undefined
|
|
// supports all formats from rich display rendering such as simple strings,
|
|
// but also markdown, html, images, tables, maps, render_all, etc...
|
|
// https://www.windmill.dev/docs/core_concepts/rich_display_rendering
|
|
}
|
|
}
|
|
|
|
// add a form in Advanced - Suspend
|
|
// all on approval steps: https://www.windmill.dev/docs/flows/flow_approval`
|
|
|
|
export const TS_PREPROCESSOR_SCRIPT_INTRO = `/**
|
|
* Trigger preprocessor
|
|
*
|
|
* ⚠️ This function runs BEFORE the main function.
|
|
*
|
|
* It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email)
|
|
* before passing it to \`main\`. This separates the trigger logic from the main logic and keeps the auto-generated runnable UI clean.
|
|
*
|
|
* The returned object defines the parameter values passed to \`main()\`.
|
|
* e.g., { b: 1, a: 2 } → Calls \`main(2, 1)\`, assuming \`main\` is defined as \`main(a: number, b: number)\`.
|
|
* Ensure that the parameter names in \`main\` match the keys in the returned object.
|
|
*
|
|
* Learn more: https://www.windmill.dev/docs/core_concepts/preprocessors
|
|
*/\n`
|
|
|
|
export const TS_PREPROCESSOR_FLOW_INTRO = `/**
|
|
* Trigger preprocessor
|
|
*
|
|
* It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email)
|
|
* before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean.
|
|
*
|
|
* The returned object determines the parameter values passed to the flow.
|
|
* e.g., \`{ b: 1, a: 2 }\` → Calls the flow with \`a = 2\` and \`b = 1\`, assuming the flow has two inputs called \`a\` and \`b\`.
|
|
* Ensure that the input names of the flow match the keys in the returned object.
|
|
*
|
|
* Learn more: https://www.windmill.dev/docs/core_concepts/preprocessors
|
|
*/\n`
|
|
|
|
export const TS_PREPROCESSOR_MODULE_CODE = `export async function preprocessor(event: TriggerEvent) {
|
|
return {
|
|
// return the args to be passed to the runnable
|
|
};
|
|
}
|
|
|
|
type TriggerEvent =
|
|
| {
|
|
kind: "webhook";
|
|
body: any;
|
|
raw_string: string | null;
|
|
query: Record<string, string>;
|
|
headers: Record<string, string>;
|
|
}
|
|
| {
|
|
kind: "http";
|
|
trigger_path: string;
|
|
body: any;
|
|
raw_string: string | null;
|
|
route: string;
|
|
path: string;
|
|
method: string;
|
|
params: Record<string, string>;
|
|
query: Record<string, string>;
|
|
headers: Record<string, string>;
|
|
}
|
|
| {
|
|
kind: "email";
|
|
trigger_path: string;
|
|
parsed_email: any;
|
|
raw_email: string;
|
|
email_extra_args?: Record<string, string>;
|
|
}
|
|
| { kind: "websocket"; trigger_path: string; msg: string; url: string }
|
|
| {
|
|
kind: "kafka";
|
|
trigger_path: string;
|
|
payload: string;
|
|
brokers: string[];
|
|
topic: string;
|
|
partition: number;
|
|
offset: number;
|
|
group_id: string;
|
|
}
|
|
| {
|
|
kind: "nats";
|
|
trigger_path: string;
|
|
payload: string;
|
|
servers: string[];
|
|
subject: string;
|
|
headers?: Record<string, string[]>;
|
|
status?: number;
|
|
description?: string;
|
|
length: number;
|
|
}
|
|
| {
|
|
kind: "sqs";
|
|
trigger_path: string;
|
|
msg: string;
|
|
queue_url: string;
|
|
message_id?: string;
|
|
receipt_handle?: string;
|
|
attributes: Record<string, string>;
|
|
message_attributes?: Record<
|
|
string,
|
|
{ string_value?: string; data_type: string }
|
|
>;
|
|
}
|
|
| {
|
|
kind: "mqtt";
|
|
trigger_path: string;
|
|
payload: string;
|
|
topic: string;
|
|
retain: boolean;
|
|
pkid: number;
|
|
qos: number;
|
|
v5?: {
|
|
payload_format_indicator?: number;
|
|
topic_alias?: number;
|
|
response_topic?: string;
|
|
correlation_data?: Array<number>;
|
|
user_properties?: Array<[string, string]>;
|
|
subscription_identifiers?: Array<number>;
|
|
content_type?: string;
|
|
};
|
|
}
|
|
| {
|
|
kind: "amqp";
|
|
trigger_path: string;
|
|
payload: string;
|
|
exchange: string;
|
|
routing_key: string;
|
|
queue_name: string;
|
|
redelivered: boolean;
|
|
delivery_tag: number;
|
|
}
|
|
| {
|
|
kind: "gcp";
|
|
trigger_path: string;
|
|
payload: string;
|
|
message_id: string;
|
|
subscription: string;
|
|
ordering_key?: string;
|
|
attributes?: Record<string, string>;
|
|
delivery_type: "push" | "pull";
|
|
headers?: Record<string, string>;
|
|
publish_time?: string;
|
|
ack_id?: string;
|
|
}
|
|
| {
|
|
kind: "postgres";
|
|
trigger_path: string;
|
|
transaction_type: "insert" | "update" | "delete";
|
|
schema_name: string;
|
|
table_name: string;
|
|
old_row?: Record<string, any>;
|
|
row: Record<string, any>;
|
|
};
|
|
`
|
|
|
|
const PYTHON_INIT_CODE_APPROVAL = `import wmill
|
|
|
|
def main():
|
|
urls = wmill.get_resume_urls()
|
|
# send the urls to their intended recipients
|
|
|
|
return {
|
|
# if the get_resume_urls are part of the response, they will be available to any persons having access
|
|
# to the run page and allowed to be approved from there, even from non owners of the flow
|
|
# self-approval is disableable in the suspend options
|
|
**urls,
|
|
|
|
# to have prompts (self-approvable steps), clude instead the resume url in the returned payload of the step
|
|
# the UX will automatically adapt and show the prompt to the operator when running the flow. e.g:
|
|
# "resume": urls["resume"],
|
|
|
|
"default_args": {},
|
|
"enums": {},
|
|
"description": None,
|
|
# supports all formats from rich display rendering such as simple strings,
|
|
# but also markdown, html, images, tables, maps, render_all, etc...
|
|
# https://www.windmill.dev/docs/core_concepts/rich_display_rendering
|
|
}
|
|
|
|
# add a form in Advanced - Suspend
|
|
# all on approval steps: https://www.windmill.dev/docs/flows/flow_approval`
|
|
|
|
export const PYTHON_PREPROCESSOR_SCRIPT_INTRO = `# Trigger preprocessor
|
|
#
|
|
# ⚠️ This function runs BEFORE the main function.
|
|
#
|
|
# It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email)
|
|
# before passing it to \`main\`. This separates the trigger logic from the main logic and keeps the auto-generated UI clean.
|
|
#
|
|
# The returned object defines the parameter values passed to \`main()\`.
|
|
# e.g., { b: 1, a: 2 } → Calls \`main(2, 1)\`, assuming \`main\` is defined as \`main(a: int, b: int)\`.
|
|
# Ensure that the parameter names in \`main\` match the keys in the returned object.
|
|
#
|
|
# Learn more: https://www.windmill.dev/docs/core_concepts/preprocessors\n\n`
|
|
|
|
export const PYTHON_PREPROCESSOR_FLOW_INTRO = `# Trigger preprocessor
|
|
#
|
|
# It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email)
|
|
# before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean.
|
|
#
|
|
# The returned object determines the parameter values passed to the flow.
|
|
# e.g., \`{ b: 1, a: 2 }\` → Calls the flow with \`a = 2\` and \`b = 1\`, assuming the flow has two inputs called \`a\` and \`b\`.
|
|
# Ensure that the input names of the flow match the keys in the returned object.
|
|
#
|
|
# Learn more: https://www.windmill.dev/docs/core_concepts/preprocessors\n\n`
|
|
|
|
export const PYTHON_PREPROCESSOR_MODULE_CODE = `from typing import TypedDict, Literal, Optional, Union
|
|
|
|
|
|
class WebhookEvent(TypedDict):
|
|
kind: Literal["webhook"]
|
|
body: dict
|
|
raw_string: Optional[str]
|
|
query: dict[str, str]
|
|
headers: dict[str, str]
|
|
|
|
|
|
class HttpEvent(TypedDict):
|
|
kind: Literal["http"]
|
|
trigger_path: str
|
|
body: dict
|
|
raw_string: Optional[str]
|
|
route: str
|
|
path: str
|
|
method: str
|
|
params: dict[str, str]
|
|
query: dict[str, str]
|
|
headers: dict[str, str]
|
|
|
|
|
|
class EmailEvent(TypedDict):
|
|
kind: Literal["email"]
|
|
trigger_path: str
|
|
parsed_email: dict
|
|
raw_email: str
|
|
email_extra_args: Optional[dict[str, str]]
|
|
|
|
|
|
class WebsocketEvent(TypedDict):
|
|
kind: Literal["websocket"]
|
|
trigger_path: str
|
|
msg: str
|
|
url: str
|
|
|
|
|
|
class KafkaEvent(TypedDict):
|
|
kind: Literal["kafka"]
|
|
trigger_path: str
|
|
payload: str
|
|
brokers: list[str]
|
|
topic: str
|
|
partition: int
|
|
offset: int
|
|
group_id: str
|
|
|
|
|
|
class NatsEvent(TypedDict):
|
|
kind: Literal["nats"]
|
|
trigger_path: str
|
|
payload: str
|
|
servers: list[str]
|
|
subject: str
|
|
headers: Optional[dict[str, list[str]]]
|
|
status: Optional[int]
|
|
description: Optional[str]
|
|
length: int
|
|
|
|
|
|
class MessageAttribute(TypedDict):
|
|
string_value: Optional[str]
|
|
data_type: str
|
|
|
|
|
|
class SqsEvent(TypedDict):
|
|
kind: Literal["sqs"]
|
|
trigger_path: str
|
|
msg: str
|
|
queue_url: str
|
|
message_id: Optional[str]
|
|
receipt_handle: Optional[str]
|
|
attributes: dict[str, str]
|
|
message_attributes: Optional[dict[str, MessageAttribute]]
|
|
|
|
|
|
class MqttV5Properties(TypedDict, total=False):
|
|
payload_format_indicator: Optional[int]
|
|
topic_alias: Optional[int]
|
|
response_topic: Optional[str]
|
|
correlation_data: Optional[list[int]]
|
|
user_properties: Optional[list[tuple[str, str]]]
|
|
subscription_identifiers: Optional[list[int]]
|
|
content_type: Optional[str]
|
|
|
|
|
|
class MqttEvent(TypedDict):
|
|
kind: Literal["mqtt"]
|
|
trigger_path: str
|
|
payload: str
|
|
topic: str
|
|
retain: bool
|
|
pkid: int
|
|
qos: int
|
|
v5: Optional[MqttV5Properties]
|
|
|
|
|
|
class AmqpEvent(TypedDict):
|
|
kind: Literal["amqp"]
|
|
trigger_path: str
|
|
payload: str
|
|
exchange: str
|
|
routing_key: str
|
|
queue_name: str
|
|
redelivered: bool
|
|
delivery_tag: int
|
|
|
|
|
|
class GcpEvent(TypedDict):
|
|
kind: Literal["gcp"]
|
|
trigger_path: str
|
|
payload: str
|
|
message_id: str
|
|
subscription: str
|
|
ordering_key: Optional[str]
|
|
attributes: Optional[dict[str, str]]
|
|
delivery_type: Literal["push", "pull"]
|
|
headers: Optional[dict[str, str]]
|
|
publish_time: Optional[str]
|
|
ack_id: Optional[str]
|
|
|
|
|
|
class PostgresEvent(TypedDict):
|
|
kind: Literal["postgres"]
|
|
trigger_path: str
|
|
transaction_type: Literal["insert", "update", "delete"]
|
|
schema_name: str
|
|
table_name: str
|
|
old_row: Optional[dict[str, any]]
|
|
row: dict[str, any]
|
|
|
|
|
|
Event = Union[
|
|
WebhookEvent,
|
|
HttpEvent,
|
|
EmailEvent,
|
|
WebsocketEvent,
|
|
KafkaEvent,
|
|
NatsEvent,
|
|
SqsEvent,
|
|
MqttEvent,
|
|
AmqpEvent,
|
|
GcpEvent,
|
|
PostgresEvent,
|
|
]
|
|
|
|
|
|
def preprocessor(event: Event):
|
|
return {
|
|
# return the args to be passed to the runnable
|
|
}
|
|
`
|
|
|
|
export const PHP_PREPROCESSOR_SCRIPT_INTRO = `<?php
|
|
/**
|
|
* Trigger preprocessor
|
|
*
|
|
* ⚠️ This function runs BEFORE the main function.
|
|
*
|
|
* It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email)
|
|
* before passing it to \`main\`. This separates the trigger logic from the main logic and keeps the auto-generated runnable UI clean.
|
|
*
|
|
* The returned object defines the parameter values passed to \`main()\`.
|
|
* e.g., ['b' => 1, 'a' => 2] → Calls \`main(2, 1)\`, assuming \`main\` is defined as \`main($a, $b)\`.
|
|
* Ensure that the parameter names in \`main\` match the keys in the returned array.
|
|
*
|
|
* Learn more: https://www.windmill.dev/docs/core_concepts/preprocessors
|
|
*/
|
|
|
|
`
|
|
|
|
export const PHP_PREPROCESSOR_FLOW_INTRO = `<?php
|
|
/**
|
|
* Trigger preprocessor
|
|
*
|
|
* It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email)
|
|
* before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean.
|
|
*
|
|
* The returned object determines the parameter values passed to the flow.
|
|
* e.g., ['b' => 1, 'a' => 2] → Calls the flow with \`a = 2\` and \`b = 1\`, assuming the flow has two inputs called \`a\` and \`b\`.
|
|
* Ensure that the input names of the flow match the keys in the returned array.
|
|
*
|
|
* Learn more: https://www.windmill.dev/docs/core_concepts/preprocessors
|
|
*/
|
|
|
|
`
|
|
|
|
export const PHP_PREPROCESSOR_MODULE_CODE = `function preprocessor(object $event) {
|
|
// $event can be one of the following types:
|
|
//
|
|
// All events (except webhook) include 'trigger_path' => '...' (the path of the trigger in Windmill)
|
|
//
|
|
// Webhook event:
|
|
// ['kind' => 'webhook', 'body' => [...], 'raw_string' => '...', 'query' => [...], 'headers' => [...]]
|
|
//
|
|
// HTTP event:
|
|
// ['kind' => 'http', 'trigger_path' => '...', 'body' => [...], 'raw_string' => '...', 'route' => '...', 'path' => '...',
|
|
// 'method' => '...', 'params' => [...], 'query' => [...], 'headers' => [...]]
|
|
//
|
|
// Email event:
|
|
// ['kind' => 'email', 'trigger_path' => '...', 'parsed_email' => [...], 'raw_email' => '...', 'email_extra_args' => [...]]
|
|
//
|
|
// WebSocket event:
|
|
// ['kind' => 'websocket', 'trigger_path' => '...', 'msg' => '...', 'url' => '...']
|
|
//
|
|
// Kafka event:
|
|
// ['kind' => 'kafka', 'trigger_path' => '...', 'payload' => '...', 'brokers' => [...], 'topic' => '...',
|
|
// 'partition' => 0, 'offset' => 0, 'group_id' => '...']
|
|
//
|
|
// NATS event:
|
|
// ['kind' => 'nats', 'trigger_path' => '...', 'payload' => '...', 'servers' => [...], 'subject' => '...',
|
|
// 'headers' => [...], 'status' => 200, 'description' => '...', 'length' => 100]
|
|
//
|
|
// SQS event:
|
|
// ['kind' => 'sqs', 'trigger_path' => '...', 'msg' => '...', 'queue_url' => '...', 'message_id' => '...',
|
|
// 'receipt_handle' => '...', 'attributes' => [...], 'message_attributes' => [...]]
|
|
//
|
|
// MQTT event:
|
|
// ['kind' => 'mqtt', 'trigger_path' => '...', 'payload' => '...', 'topic' => '...', 'retain' => true, 'pkid' => 1,
|
|
// 'qos' => 1, 'v5' => [...]]
|
|
//
|
|
// AMQP event:
|
|
// ['kind' => 'amqp', 'trigger_path' => '...', 'payload' => '...', 'exchange' => '...', 'routing_key' => '...',
|
|
// 'queue_name' => '...', 'redelivered' => false, 'delivery_tag' => 1]
|
|
//
|
|
// GCP event:
|
|
// ['kind' => 'gcp', 'trigger_path' => '...', 'payload' => '...', 'message_id' => '...', 'subscription' => '...',
|
|
// 'ordering_key' => '...', 'attributes' => [...], 'delivery_type' => 'push',
|
|
// 'headers' => [...], 'publish_time' => '...', 'ack_id' => '...']
|
|
//
|
|
// Postgres event:
|
|
// ['kind' => 'postgres', 'trigger_path' => '...', 'transaction_type' => 'insert', 'schema_name' => '...',
|
|
// 'table_name' => '...', 'old_row' => [...], 'row' => [...]]
|
|
|
|
return [
|
|
// return the args to be passed to the runnable
|
|
];
|
|
}
|
|
`
|
|
|
|
const DOCKER_INIT_CODE = `# shellcheck shell=bash
|
|
# sandbox alpine:latest
|
|
# The "# sandbox <image>" annotation runs this script INSIDE the image above,
|
|
# sandboxed via nsjail: the image's rootfs is extracted (rootless podman) and the
|
|
# body runs chrooted in it, inheriting the job's confinement. The body runs with
|
|
# the image's /bin/sh and windmill args bind positionally as $1, $2, ...
|
|
# Daemonless — no docker run/-d/exec/build and no host -v bind mounts.
|
|
# (A bare "# docker" still uses the legacy daemon runtime instead.)
|
|
|
|
msg="\${1:-world}"
|
|
|
|
echo "Hello $msg"
|
|
cat /etc/os-release | head -1
|
|
`
|
|
|
|
const POWERSHELL_INIT_CODE = `param($Msg, [string[]]$Names, [PSCustomObject]$Obj, $Dflt = "default value", [int]$Nb = 3)
|
|
|
|
# Import-Module MyModule
|
|
|
|
# Import-Module WindmillClient
|
|
# Connect-Windmill
|
|
# Get-WindmillVariable -Path 'u/user/foo'
|
|
|
|
# the last line of the stdout is the return value
|
|
Write-Output "Hello $Msg"`
|
|
|
|
const ANSIBLE_PLAYBOOK_INIT_CODE = `---
|
|
inventory:
|
|
- resource_type: ansible_inventory
|
|
# You can pin an inventory to this script by hardcoding the resource path:
|
|
# resource: u/user/your_resource
|
|
# - name: hcloud.yml
|
|
# resource_type: dynamic_inventory
|
|
|
|
options:
|
|
- verbosity: vvv
|
|
|
|
|
|
# File resources will be written in the relative \`target\` location before
|
|
# running the playbook
|
|
# files:
|
|
# - resource: u/user/fabulous_jinja_template
|
|
# target: ./config_template.j2
|
|
# - variable: u/user/ssh_key
|
|
# target: ./ssh_key
|
|
# mode: '0600'
|
|
|
|
# Define the arguments of the windmill script
|
|
extra_vars:
|
|
world_qualifier:
|
|
type: string
|
|
|
|
# If using Ansible Vault:
|
|
# vault_password: u/user/ansible_vault_password
|
|
|
|
dependencies:
|
|
galaxy:
|
|
collections:
|
|
- name: community.general
|
|
- name: community.vmware
|
|
roles:
|
|
python:
|
|
- jmespath
|
|
---
|
|
- name: Echo
|
|
hosts: 127.0.0.1
|
|
connection: local
|
|
vars:
|
|
my_result:
|
|
a: 2
|
|
b: true
|
|
c: "Hello"
|
|
|
|
tasks:
|
|
- name: Print debug message
|
|
debug:
|
|
msg: "Hello, {{world_qualifier}} world!"
|
|
- name: Write variable my_result to result.json
|
|
delegate_to: localhost
|
|
copy:
|
|
content: "{{ my_result | to_json }}"
|
|
dest: result.json
|
|
`
|
|
const JAVA_INIT_CODE = `//requirements:
|
|
//com.google.code.gson:gson:2.8.9
|
|
//com.github.ricksbrown:cowsay:1.1.0
|
|
|
|
import com.google.gson.Gson;
|
|
import com.google.gson.GsonBuilder;
|
|
import com.github.ricksbrown.cowsay.Cowsay;
|
|
import com.github.ricksbrown.cowsay.plugin.CowExecutor;
|
|
|
|
public class Main {
|
|
public static class Person {
|
|
private String name;
|
|
private int age;
|
|
|
|
// Constructor
|
|
public Person(String name, int age) {
|
|
this.name = name;
|
|
this.age = age;
|
|
}
|
|
}
|
|
|
|
public static Object main(
|
|
// Primitive
|
|
int a,
|
|
float b,
|
|
// Objects
|
|
Integer age,
|
|
Float d,
|
|
Object e,
|
|
String name,
|
|
// Lists
|
|
String[] f
|
|
// No trailing commas!
|
|
){
|
|
Gson gson = new Gson();
|
|
|
|
// Get resources
|
|
var theme = Wmill.getResource("f/app_themes/theme_0");
|
|
System.out.println("Theme: " + theme);
|
|
|
|
// Create a Person object
|
|
Person person = new Person( (name == "") ? "Alice" : name, (age == null) ? 30 : age);
|
|
|
|
// Serialize the Person object to JSON
|
|
String json = gson.toJson(person);
|
|
System.out.println("Serialized JSON: " + json);
|
|
|
|
// Use cowsay
|
|
String[] args = new String[]{"-f", "dragon", json };
|
|
String result = Cowsay.say(args);
|
|
return result;
|
|
}
|
|
}
|
|
`
|
|
const RUBY_INIT_CODE = `require 'windmill/inline'
|
|
require 'windmill/mini'
|
|
|
|
# Dependency management: declare gems in gemfile block for automatic installation
|
|
# Windmill uses bundler/inline compatible syntax with automatic requiring
|
|
gemfile do
|
|
source 'https://rubygems.org'
|
|
gem 'amazing_print', '~> 1.6'
|
|
end
|
|
|
|
def main(
|
|
no_default,
|
|
name = "Nicolas Bourbaki",
|
|
age = 42,
|
|
obj = { "even": "hashes" },
|
|
list = ["or", "arrays!"]
|
|
)
|
|
puts "Hello World and a warm welcome especially to #{name}"
|
|
puts "and its acolytes.. #{age} #{obj} #{list}"
|
|
|
|
# Retrieve variables using the Windmill mini client
|
|
begin
|
|
secret = get_variable("f/examples/secret")
|
|
rescue => e
|
|
secret = "No secret yet at f/examples/secret!"
|
|
end
|
|
puts "The variable at 'f/examples/secret': #{secret}"
|
|
|
|
# Get typed resources using the mini client
|
|
# database = get_resource("u/user/my_postgresql")
|
|
|
|
# Access environment variables provided by Windmill
|
|
user = ENV['WM_USERNAME']
|
|
|
|
# Pretty print results using amazing_print (automatically required from gemfile)
|
|
result = {
|
|
"splitted" => name.split,
|
|
"user" => user,
|
|
"age" => age,
|
|
"obj" => obj,
|
|
"list" => list
|
|
}
|
|
|
|
ap result
|
|
|
|
# Return value is automatically converted to JSON
|
|
return result
|
|
end
|
|
`
|
|
const R_INIT_CODE = `library(dplyr)
|
|
library(jsonlite)
|
|
|
|
main <- function(
|
|
x,
|
|
name = "default",
|
|
age = 25,
|
|
data = list(1, 2, 3),
|
|
flag = TRUE
|
|
) {
|
|
# Use Windmill helpers:
|
|
# var <- get_variable("f/my_var")
|
|
# res <- get_resource("f/my_resource")
|
|
|
|
df <- tibble(name = name, age = age, x = x)
|
|
result <- df %>% mutate(greeting = paste("Hello", name))
|
|
|
|
return(toJSON(result, auto_unbox = TRUE))
|
|
}
|
|
`
|
|
|
|
// A dbt script is a whole dbt project: the descriptor below is the script's
|
|
// content, and the project's own files live in its module bundle (the
|
|
// `<script>__dbt/` folder the CLI syncs). Field names track dbt's vocabulary.
|
|
const DBT_INIT_CODE = `# dbt-core-1x (default) | dbt-core-2x | fusion
|
|
engine: dbt-core-1x
|
|
profile:
|
|
# A warehouse configured on this workspace (Settings -> dbt), by name. Omitted
|
|
# takes 'main'. The NAME is also the warehouse's identity in the asset graph:
|
|
# every model becomes dbt://<warehouse>/<schema>/<name>, which is what gives
|
|
# this project its lineage, and lets a native script reading one of those
|
|
# tables share the node. A dbt run does not trigger such a script.
|
|
# warehouse: main
|
|
# target: prod
|
|
# Alternatively, keep the project's own file. It runs unchanged, but names a
|
|
# warehouse only to say where its assets belong:
|
|
# profiles_yml: profiles.yml
|
|
# Passed to dbt verbatim — this is dbt's selector grammar, not Windmill's
|
|
select: []
|
|
exclude: []
|
|
# build (models and tests interleaved) | after_all | none
|
|
test_behavior: build
|
|
vars: {}
|
|
threads: 4
|
|
full_refresh: false
|
|
# Resolve a ref() this run does not build through the state the last successful
|
|
# run of this environment published, instead of through the schema it writes
|
|
# into. The default for the run form's toggle: the run that publishes the state
|
|
# and the run that defers to it are two invocations of this one script.
|
|
defer: false
|
|
# Rebuild the nodes a failed build left failed or skipped, in this same job,
|
|
# before reporting failure. dbt confines a failure to its own subtree, so a
|
|
# transient warehouse error costs those nodes rather than the whole project.
|
|
# Not available on agent workers, whose wait could not observe a cancellation.
|
|
# retry_failed_nodes:
|
|
# attempts: 2
|
|
# delay_seconds: 30
|
|
# Extra env for the project's own {{ env_var() }} lookups. A $var: value is
|
|
# resolved to that Windmill variable, so secrets stay out of this file.
|
|
# env:
|
|
# DBT_PASSWORD: $var:u/user/my_warehouse_password
|
|
# Real column schemas — every column typed and in the order the model produces
|
|
# it — from the engine's static analysis, which also records column-level
|
|
# lineage for a later view. Opt-in because it runs a separate dbt compile under
|
|
# --static-analysis strict, which rejects SQL the default accepts; a project it
|
|
# cannot analyze keeps the graph it has. Needs an engine that computes it.
|
|
# column_lineage: true
|
|
`
|
|
// for related places search: ADD_NEW_LANG
|
|
export const INITIAL_CODE = {
|
|
bun: {
|
|
scriptInitCodeBlock: BUN_INIT_BLOCK,
|
|
script: BUN_INIT_CODE,
|
|
trigger: BUN_INIT_CODE_TRIGGER,
|
|
approval: BUN_INIT_CODE_APPROVAL,
|
|
failure: BUN_FAILURE_MODULE_CODE,
|
|
preprocessor: TS_PREPROCESSOR_FLOW_INTRO + TS_PREPROCESSOR_MODULE_CODE,
|
|
clear: BUN_INIT_CODE_CLEAR
|
|
},
|
|
python3: {
|
|
script: PYTHON_INIT_CODE,
|
|
trigger: PYTHON_INIT_CODE_TRIGGER,
|
|
approval: PYTHON_INIT_CODE_APPROVAL,
|
|
failure: PYTHON_FAILURE_MODULE_CODE,
|
|
preprocessor: PYTHON_PREPROCESSOR_FLOW_INTRO + PYTHON_PREPROCESSOR_MODULE_CODE,
|
|
clear: PYTHON_INIT_CODE_CLEAR
|
|
},
|
|
deno: {
|
|
scriptInitCodeBlock: DENO_INIT_BLOCK,
|
|
script: DENO_INIT_CODE,
|
|
trigger: DENO_INIT_CODE_TRIGGER,
|
|
approval: DENO_INIT_CODE_APPROVAL,
|
|
failure: DENO_FAILURE_MODULE_CODE,
|
|
preprocessor: TS_PREPROCESSOR_FLOW_INTRO + TS_PREPROCESSOR_MODULE_CODE,
|
|
fetch: FETCH_INIT_CODE,
|
|
clear: DENO_INIT_CODE_CLEAR
|
|
},
|
|
go: {
|
|
script: GO_INIT_CODE,
|
|
trigger: GO_INIT_CODE_TRIGGER,
|
|
failure: GO_FAILURE_MODULE_CODE
|
|
},
|
|
bash: {
|
|
script: BASH_INIT_CODE
|
|
},
|
|
powershell: {
|
|
script: POWERSHELL_INIT_CODE
|
|
},
|
|
nativets: {
|
|
script: NATIVETS_INIT_CODE
|
|
},
|
|
postgresql: {
|
|
script: POSTGRES_INIT_CODE
|
|
},
|
|
mysql: {
|
|
script: MYSQL_INIT_CODE
|
|
},
|
|
bigquery: {
|
|
script: BIGQUERY_INIT_CODE
|
|
},
|
|
snowflake: {
|
|
script: SNOWFLAKE_INIT_CODE
|
|
},
|
|
mssql: {
|
|
script: MSSQL_INIT_CODE
|
|
},
|
|
duckdb: {
|
|
script: DUCKDB_INIT_CODE
|
|
},
|
|
graphql: {
|
|
script: GRAPHQL_INIT_CODE
|
|
},
|
|
oracledb: {
|
|
script: ORACLEDB_INIT_CODE
|
|
},
|
|
php: {
|
|
script: PHP_INIT_CODE,
|
|
preprocessor: PHP_PREPROCESSOR_FLOW_INTRO + PHP_PREPROCESSOR_MODULE_CODE
|
|
},
|
|
rust: {
|
|
script: RUST_INIT_CODE
|
|
},
|
|
ansible: {
|
|
script: ANSIBLE_PLAYBOOK_INIT_CODE
|
|
},
|
|
csharp: {
|
|
script: CSHARP_INIT_CODE
|
|
},
|
|
nu: {
|
|
script: NU_INIT_CODE
|
|
},
|
|
docker: {
|
|
script: DOCKER_INIT_CODE
|
|
},
|
|
bunnative: {
|
|
script: BUNNATIVE_INIT_CODE
|
|
},
|
|
java: {
|
|
script: JAVA_INIT_CODE
|
|
},
|
|
ruby: {
|
|
script: RUBY_INIT_CODE
|
|
},
|
|
rlang: {
|
|
script: R_INIT_CODE
|
|
},
|
|
dbt: {
|
|
script: DBT_INIT_CODE
|
|
},
|
|
claudesandbox: {
|
|
script: CLAUDE_SANDBOX_INIT_CODE
|
|
},
|
|
wac_python: {
|
|
script: WAC_PYTHON_INIT_CODE
|
|
},
|
|
wac_typescript: {
|
|
script: WAC_TYPESCRIPT_INIT_CODE
|
|
},
|
|
ci_test_bun: {
|
|
script: CI_TEST_BUN_INIT_CODE
|
|
},
|
|
ci_test_python: {
|
|
script: CI_TEST_PYTHON_INIT_CODE
|
|
}
|
|
// for related places search: ADD_NEW_LANG
|
|
}
|
|
|
|
/**
|
|
* Whether a bash script body runs inside a custom container image that does not
|
|
* ship the `wmill` CLI (nor `jq`), namely `# sandbox <image>` or `# docker`. In
|
|
* that case editor snippets must fall back to a plain HTTP client instead of `wmill`.
|
|
*
|
|
* Mirrors the worker's annotation grammar (backend/windmill-common/src/worker.rs,
|
|
* `BashAnnotations`): only leading comment lines are scanned, stopping at the first
|
|
* non-comment line. A bare `# sandbox` (no image) is the nsjail-bash modifier that
|
|
* still runs on the worker rootfs where `wmill` is available, so it is excluded.
|
|
*/
|
|
export function bashRunsInCustomImage(code: string): boolean {
|
|
for (const line of code.split('\n')) {
|
|
const trimmed = line.trim()
|
|
if (trimmed === '') continue
|
|
if (!trimmed.startsWith('#')) break
|
|
const tokens = trimmed.slice(1).trim().split(/\s+/)
|
|
// `# sandbox <image>` selects a container; bare `# sandbox` does not.
|
|
if (tokens[0] === 'sandbox' && tokens[1]) return true
|
|
// `# docker` (v1 daemon runtime) runs in the referenced image, no wmill.
|
|
if (tokens[0] === 'docker' && tokens.length === 1) return true
|
|
}
|
|
return false
|
|
}
|
|
|
|
export function isInitialCode(content: string): boolean {
|
|
for (const lang of Object.values(INITIAL_CODE)) {
|
|
for (const code of Object.values(lang)) {
|
|
if (content === code) {
|
|
return true
|
|
}
|
|
}
|
|
}
|
|
return false
|
|
}
|
|
|
|
export function initialCode(
|
|
language: SupportedLanguage | 'bunnative' | undefined,
|
|
kind: Script['kind'] | undefined,
|
|
subkind:
|
|
| 'pgsql'
|
|
| 'mysql'
|
|
| 'flow'
|
|
| 'script'
|
|
| 'fetch'
|
|
| 'docker'
|
|
| 'powershell'
|
|
| 'bunnative'
|
|
| 'claudesandbox'
|
|
| 'wac_python'
|
|
| 'wac_typescript'
|
|
| 'ci_test_bun'
|
|
| 'ci_test_python'
|
|
| undefined,
|
|
templateScript?: boolean
|
|
): string {
|
|
if (!kind) {
|
|
kind = 'script'
|
|
}
|
|
if (language === 'deno') {
|
|
if (kind === 'trigger') {
|
|
return INITIAL_CODE.deno.trigger
|
|
} else if (kind === 'script') {
|
|
if (subkind === 'flow') {
|
|
return INITIAL_CODE.deno.clear
|
|
} else if (subkind === 'pgsql') {
|
|
return INITIAL_CODE.postgresql.script
|
|
} else if (subkind === 'mysql') {
|
|
return INITIAL_CODE.mysql.script
|
|
} else if (subkind === 'fetch') {
|
|
return INITIAL_CODE.deno.fetch
|
|
} else {
|
|
return INITIAL_CODE.deno.script
|
|
}
|
|
} else if (kind === 'failure') {
|
|
return INITIAL_CODE.deno.failure
|
|
} else if (kind === 'approval') {
|
|
return INITIAL_CODE.deno.approval
|
|
} else if (kind === 'preprocessor') {
|
|
return INITIAL_CODE.deno.preprocessor
|
|
} else {
|
|
return INITIAL_CODE.deno.script
|
|
}
|
|
} else if (subkind === 'ci_test_bun') {
|
|
return INITIAL_CODE.ci_test_bun.script
|
|
} else if (subkind === 'ci_test_python') {
|
|
return INITIAL_CODE.ci_test_python.script
|
|
} else if (subkind === 'wac_python') {
|
|
return INITIAL_CODE.wac_python.script
|
|
} else if (subkind === 'wac_typescript') {
|
|
return INITIAL_CODE.wac_typescript.script
|
|
} else if (language === 'python3') {
|
|
if (kind === 'trigger') {
|
|
return INITIAL_CODE.python3.trigger
|
|
} else if (kind === 'approval') {
|
|
return INITIAL_CODE.python3.approval
|
|
} else if (kind === 'failure') {
|
|
return INITIAL_CODE.python3.failure
|
|
} else if (kind === 'preprocessor') {
|
|
return INITIAL_CODE.python3.preprocessor
|
|
} else if (subkind === 'flow') {
|
|
return INITIAL_CODE.python3.clear
|
|
} else {
|
|
return INITIAL_CODE.python3.script
|
|
}
|
|
} else if (language == 'bash') {
|
|
if (subkind === 'docker') {
|
|
return INITIAL_CODE.docker.script
|
|
} else {
|
|
return INITIAL_CODE.bash.script
|
|
}
|
|
} else if (language == 'powershell') {
|
|
return INITIAL_CODE.powershell.script
|
|
} else if (language == 'nativets') {
|
|
return INITIAL_CODE.nativets.script
|
|
} else if (language == 'postgresql') {
|
|
return INITIAL_CODE.postgresql.script
|
|
} else if (language == 'mysql') {
|
|
return INITIAL_CODE.mysql.script
|
|
} else if (language == 'bigquery') {
|
|
return INITIAL_CODE.bigquery.script
|
|
} else if (language == 'oracledb') {
|
|
return INITIAL_CODE.oracledb.script
|
|
} else if (language == 'snowflake') {
|
|
return INITIAL_CODE.snowflake.script
|
|
} else if (language == 'mssql') {
|
|
return INITIAL_CODE.mssql.script
|
|
} else if (language == 'graphql') {
|
|
return INITIAL_CODE.graphql.script
|
|
} else if (language == 'duckdb') {
|
|
return INITIAL_CODE.duckdb.script
|
|
} else if (language == 'php') {
|
|
if (kind == 'preprocessor') {
|
|
return INITIAL_CODE.php.preprocessor
|
|
}
|
|
return INITIAL_CODE.php.script
|
|
} else if (language == 'rust') {
|
|
return INITIAL_CODE.rust.script
|
|
} else if (language == 'ansible') {
|
|
return INITIAL_CODE.ansible.script
|
|
} else if (language == 'csharp') {
|
|
return INITIAL_CODE.csharp.script
|
|
} else if (language == 'nu') {
|
|
return INITIAL_CODE.nu.script
|
|
} else if (language == 'java') {
|
|
return INITIAL_CODE.java.script
|
|
} else if (language == 'ruby') {
|
|
return INITIAL_CODE.ruby.script
|
|
} else if (language == 'rlang') {
|
|
return INITIAL_CODE.rlang.script
|
|
} else if (language == 'dbt') {
|
|
return INITIAL_CODE.dbt.script
|
|
// for related places search: ADD_NEW_LANG
|
|
} else if (language == 'bun' || language == 'bunnative') {
|
|
if (subkind === 'claudesandbox') {
|
|
return INITIAL_CODE.claudesandbox.script
|
|
} else if (kind == 'trigger') {
|
|
return INITIAL_CODE.bun.trigger
|
|
} else if (language == 'bunnative' || subkind === 'bunnative') {
|
|
return INITIAL_CODE.bunnative.script
|
|
} else if (kind === 'approval') {
|
|
return INITIAL_CODE.bun.approval
|
|
} else if (kind === 'failure') {
|
|
return INITIAL_CODE.bun.failure
|
|
} else if (kind === 'preprocessor') {
|
|
return INITIAL_CODE.bun.preprocessor
|
|
} else if (templateScript == true) {
|
|
return INITIAL_CODE.bun.scriptInitCodeBlock
|
|
} else if (subkind === 'flow') {
|
|
return INITIAL_CODE.bun.clear
|
|
}
|
|
|
|
return INITIAL_CODE.bun.script
|
|
} else {
|
|
if (kind === 'failure') {
|
|
return INITIAL_CODE.go.failure
|
|
} else if (kind === 'trigger') {
|
|
return INITIAL_CODE.go.trigger
|
|
} else {
|
|
return INITIAL_CODE.go.script
|
|
}
|
|
}
|
|
}
|
|
|
|
export function getResetCode(
|
|
language: SupportedLanguage | 'bunnative' | undefined,
|
|
kind: Script['kind'] | undefined,
|
|
subkind:
|
|
| 'pgsql'
|
|
| 'mysql'
|
|
| 'flow'
|
|
| 'script'
|
|
| 'fetch'
|
|
| 'docker'
|
|
| 'powershell'
|
|
| 'bunnative'
|
|
| 'claudesandbox'
|
|
| 'wac_python'
|
|
| 'wac_typescript'
|
|
| 'ci_test_bun'
|
|
| 'ci_test_python'
|
|
| undefined
|
|
) {
|
|
// Every *_INIT_CODE_CLEAR below is a `main` stub, which cannot run under the preprocessor
|
|
// entrypoint. Preprocessors must go through initialCode to keep theirs.
|
|
if (kind === 'preprocessor') {
|
|
return initialCode(language, kind, subkind)
|
|
}
|
|
if (language === 'deno') {
|
|
return DENO_INIT_CODE_CLEAR
|
|
} else if (language === 'python3') {
|
|
return PYTHON_INIT_CODE_CLEAR
|
|
} else if (language === 'nativets') {
|
|
return NATIVETS_INIT_CODE_CLEAR
|
|
} else if (language === 'bun') {
|
|
return BUN_INIT_CODE_CLEAR
|
|
} else if (language === 'bunnative') {
|
|
return BUNNATIVE_INIT_CODE
|
|
} else {
|
|
return initialCode(language, kind, subkind)
|
|
}
|
|
}
|
|
|
|
export const PREPROCESSOR_SUPPORTED_LANGUAGES = [
|
|
'typescript',
|
|
'python',
|
|
'python3',
|
|
'deno',
|
|
'bun',
|
|
'php'
|
|
] as const
|
|
|
|
export function canHavePreprocessor(language: string | undefined): boolean {
|
|
if (!language) {
|
|
return false
|
|
}
|
|
|
|
return PREPROCESSOR_SUPPORTED_LANGUAGES.includes(language as any)
|
|
}
|
|
|
|
export function canHaveTrigger(language: SupportedLanguage | undefined): boolean {
|
|
if (!language) {
|
|
return false
|
|
}
|
|
|
|
return ['python3', 'bun', 'deno', 'go'].includes(language)
|
|
}
|
|
|
|
export function canHaveApproval(language: SupportedLanguage | undefined): boolean {
|
|
if (!language) {
|
|
return false
|
|
}
|
|
|
|
return ['python3', 'bun'].includes(language)
|
|
}
|
|
|
|
export function canHaveFailure(language: SupportedLanguage | undefined): boolean {
|
|
if (!language) {
|
|
return false
|
|
}
|
|
|
|
return ['python3', 'bun', 'deno', 'go'].includes(language)
|
|
}
|
|
|
|
export function getPreprocessorIntro(
|
|
language: SupportedLanguage | 'docker' | 'bunnative' | undefined,
|
|
isFlow: boolean = false
|
|
): string {
|
|
if (!language || !PREPROCESSOR_SUPPORTED_LANGUAGES.includes(language as any)) {
|
|
return ''
|
|
}
|
|
|
|
switch (language) {
|
|
case 'python3':
|
|
return isFlow ? PYTHON_PREPROCESSOR_FLOW_INTRO : PYTHON_PREPROCESSOR_SCRIPT_INTRO
|
|
case 'deno':
|
|
case 'bun':
|
|
return isFlow ? TS_PREPROCESSOR_FLOW_INTRO : TS_PREPROCESSOR_SCRIPT_INTRO
|
|
case 'php':
|
|
return isFlow ? PHP_PREPROCESSOR_FLOW_INTRO : PHP_PREPROCESSOR_SCRIPT_INTRO
|
|
default:
|
|
return ''
|
|
}
|
|
}
|
|
|
|
export function getPreprocessorModuleCode(
|
|
language: SupportedLanguage | 'docker' | 'bunnative' | undefined
|
|
): string {
|
|
if (!language || !PREPROCESSOR_SUPPORTED_LANGUAGES.includes(language as any)) {
|
|
return ''
|
|
}
|
|
|
|
switch (language) {
|
|
case 'python3':
|
|
return PYTHON_PREPROCESSOR_MODULE_CODE
|
|
case 'deno':
|
|
case 'bun':
|
|
return TS_PREPROCESSOR_MODULE_CODE
|
|
case 'php':
|
|
return PHP_PREPROCESSOR_MODULE_CODE
|
|
default:
|
|
return ''
|
|
}
|
|
}
|
|
|
|
export function getPreprocessorFullCode(
|
|
language: SupportedLanguage | 'docker' | 'bunnative' | undefined,
|
|
isFlow: boolean = false
|
|
): string {
|
|
const intro = getPreprocessorIntro(language, isFlow)
|
|
const moduleCode = getPreprocessorModuleCode(language)
|
|
return intro + moduleCode
|
|
}
|
|
|
|
export function getMainFunctionPattern(
|
|
language: SupportedLanguage | 'docker' | 'bunnative' | undefined
|
|
): string {
|
|
if (!language) {
|
|
return ''
|
|
}
|
|
|
|
switch (language) {
|
|
case 'python3':
|
|
return 'def main'
|
|
case 'deno':
|
|
case 'bun':
|
|
case 'nativets':
|
|
return 'export async function main'
|
|
case 'php':
|
|
return 'function main'
|
|
default:
|
|
return 'main'
|
|
}
|
|
}
|