81b23a2ba0 feat: make the fork lineage the only deploy relationship (#10410)
* feat: make the fork lineage the only deploy relationship

`workspace_settings.deploy_to` (2023) and `workspace.parent_workspace_id` (2025)
both expressed "which workspace does this one deploy into". Fork creation and
dev-workspace attach seeded both, but nothing kept them in agreement, so every
reader picked one and they disagreed.

Drop `deploy_to`. A migration folds surviving pairs into the lineage: a sole
claimant on a target with no dev workspace becomes that target's dev workspace
and keeps its own job tags, while many-to-one pairs become plain forks. Pairs
that the lineage cannot express -- dangling target, self-reference, chain,
mutual -- are reported and left unlinked.

Job tags were never lineage-aware: `per_workspace_tag` mapped any parented
workspace to its parent while `$workspace` interpolated the raw id, so a fork
running a script tagged `<tag>-$workspace` produced a tag no worker serves and
the job queued forever. Both paths now resolve to the nearest ancestor whose id
an admin would provision workers for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: preserve unconvertible deploy links and sweep tag caches on reparent

Review findings on the deploy_to unification:

- convert chains instead of discarding them, and keep whatever the lineage
  cannot express in workspace_deploy_to_unmigrated so the down migration can
  restore it
- ignore soft-deleted workspaces when choosing between a dev workspace and a
  plain fork; an archived claimant was demoting live pairs
- mirror attach_dev_workspace's git-sync strip, which the migration skipped
- sweep the tag cache over whole subtrees on rename and delete: tag resolution
  now walks ancestors, so a nested fork kept a tag nothing serves
- call a dev workspace a dev workspace in the settings copy
- redirect a root away from ?tab=deploy_to instead of rendering an empty target

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: detect lineage cycles and record archived links in the deploy_to migration

Second review round on the unification:

- detect cycles over the lineage as it would exist after conversion, not over
  the deploy_to graph alone: a root whose target was one of its own forks
  closed a loop that no deploy_to edge revealed
- record an archived source's link instead of filtering it out entirely, which
  dropped it with the column
- treat a fork whose deploy_to merely repeats its parent as redundant rather
  than reporting every pre-existing fork as unmigrated
- read the row count from the lineage update rather than the git-sync one
- sweep the tag cache when archiving a dev workspace, the last site that
  mutates is_dev_workspace without one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: resolve $workspace on preprocessed flow tags regardless of $args

Third review round on the unification:

- a flow tag containing only `$workspace` skipped interpolation entirely on the
  preprocessed path, because the branch that ran it keys on `$args`. The raw
  tag was written back and named a queue no worker serves. Resolve `$workspace`
  before the branch and leave `$args` to it.
- record the new table's foreign key in the schema summary
- describe what the archive tag sweep actually does: the dev flag is cleared for
  any archived workspace, which is why it is unconditional

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep the deploy_to leftovers table only when it holds something

* fix: sweep tag caches on archive only where the dev flag actually changes

* feat: broadcast lineage changes and walk ws_specific ancestors only

- propagate tag-cache invalidation across processes over notify_events: the
  cache is per-process, so replicas kept resolving stale lineage for the TTL.
  The listener clears the whole cache rather than tracking ids, since a single
  mutation invalidates an unbounded set of descendants and lineage changes are
  rare admin actions.
- narrow list_ws_specific_versions to ancestors: walking down as well made a
  root fan out over its entire live fork subtree, and each member costs an
  identity lookup plus an RLS switch and probe. Ancestors are bounded by the
  fork depth limit.
- probe the leftovers table unqualified so rollback restores on a PG_SCHEMA
  install, where search_path is not public
- drop the nativets client method for the removed edit_deploy_to endpoint

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: let a prod see its dev workspace in ws_specific, and stop the walk oscillating

Descending into plain forks made a root fan out over its whole live fork
subtree, but a dev workspace is the paired editable environment rather than a
throwaway copy, so a prod should still see it. There is at most one per parent
and attach rejects nested dev chains, so that edge stays bounded.

The edges run both ways, so the recursion never converged: it bounced
parent<->dev until the depth cap on every call, 33 rows for a two-member set.
A visited-path guard ends the walk when nothing new is reachable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep dev pairings unnested, gate the delete broadcast, cover the ws_specific walk

Fifth review round:

- a root that already owns a dev workspace no longer converts: linking it under
  its deploy target would leave that dev nested beneath a fork, the shape
  attach_dev_workspace refuses to create. The link is preserved instead.
- broadcast a lineage change on delete only when descendants are orphaned.
  Deleting a leaf, which ephemeral fork churn does constantly, changes nobody
  else's resolution and was making every replica drop its whole tag cache.
- call list_ws_specific_versions in a test. plpgsql defers everything past a raw
  parse to the first call, so replaying the migration only proved it parses.
- use unwrap_or_default for the descendant sweeps, which run after the
  transaction has committed; a transient failure must not fail the request
- trim the traversal comment to the four-line limit

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: cache the renamed tally query and clear instance alerts on conversion

The integration test's query was never cached: `cargo sqlx prepare` without
--all-targets skips test targets entirely, and renaming its fixture workspace
changed the query text. Regenerated with --all-targets --features
all_sqlx_features,private, which is what lets the EE-gated otel test compile.

Also from review:
- clear error_handler_fallback_to_instance_alerts on converted workspaces.
  Dispatch ignores it once a parent exists, but the settings page keeps
  submitting the stored true, which the API rejects on a fork.
- restore the schema summary row to the file's name: columns format and put it
  back in alphabetical order

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: never cache an unresolvable tag workspace, and unadvertise the removed endpoint

- lookup_tag_workspace cached a "no row" result as self-resolution. A rename
  resolves the new id before its row lands, so a fork could be pinned to its own
  wm-fork-* id -- which nothing serves -- for the whole TTL, and its schedules
  kept re-pushing onto that dead tag. Fall back for the call without caching,
  matching how the error path already behaved.
- change_workspace_id swept its children but never itself. Sweep the new and old
  ids and broadcast unconditionally, since a rename always changes lineage.
- openapi-deref.{json,yaml} are served to clients via include_str!, so they were
  advertising edit_deploy_to after it started 404ing. The audit-action enum
  keeps the entry: historical rows still carry it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: align the served YAML spec with the JSON one and correct two comments

- the YAML deref lost the removed path but kept deploy_to on get_settings,
  so the two served specs disagreed. Both are now identical.
- the rename-sweep comment blamed cached-unresolvable lookups, which the same
  commit stopped caching. The real reason is that workspace ids are
  reclaimable, so a new id can carry a previous occupant's resolution.
- the instance-alert comment claimed the settings page submits the stored true
  and gets a 400. It hides the option on a fork and sends false; the hazard is
  the value outliving the pairing and re-enabling alerts after a detach.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 82da6cb2bafeda18acd6b70c599013a12117ecb0

This commit updates the EE repository reference after PR #694 was merged in windmill-ee-private.

Previous ee-repo-ref: f9ddf6a75aa13d1c13a3d7216a361a96f75ca435

New ee-repo-ref: 82da6cb2bafeda18acd6b70c599013a12117ecb0

Automated by sync-ee-ref workflow.

* fix: grant the deploy_to preservation table to the windmill roles

* test: drop the one-shot migration tests, keep the ws_specific execution guard

The two conversion tests replayed the migration against the fully-migrated
schema, which is not how it runs -- in production it runs mid-sequence against
the schema as of that point. A later migration touching workspace or
workspace_settings would break them without breaking anything real, and sqlx
checksums already freeze a released migration. They earned their keep finding
the archived-claimant and nested-dev cases during development; there is nothing
left for them to guard.

list_ws_specific_versions is different: it is live, no caller exercises it, and
plpgsql only parses a function body until first call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: invalidate a reclaimed fork id cluster-wide without flushing every entry

Gating the delete broadcast on orphaned descendants stopped leaf churn flushing
every replica, but fork ids are reclaimable: the deleting process invalidated
locally while every other replica kept the old parent for the TTL, so a job
pushed in a recreated fork routed to the previous parent's tag.

The broadcast payload now carries meaning. A workspace id drops that one entry,
used for leaf deletion where exactly one id changed what it denotes. The `*`
sentinel drops everything, used for attach, detach, archive, rename and
deletions that orphan descendants -- reshaping a subtree no single id names.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name the right broadcast for each invalidation case

* docs: attach does invalidate the tag cache; the resolver walks the whole chain

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-07-30 14:20:27 +00:00
2024-02-08 16:09:11 +01:00
2026-02-03 17:50:56 +00:00
2026-07-29 12:12:07 +02:00
2023-08-26 09:17:07 +02:00
2025-04-21 16:53:23 +02:00
2026-04-16 06:26:25 -07:00
2025-12-08 10:34:16 +00:00
2025-03-07 15:08:19 +01:00
2022-05-05 04:25:58 +02:00
2026-02-16 23:36:41 +00:00
2024-02-08 16:09:11 +01:00

windmill.dev

Open-source developer platform for internal code: APIs, background jobs, workflows and UIs. Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.

Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported languages: Python, TypeScript, Go, Bash, SQL, GraphQL, PowerShell, Rust, and more.

Package version Docker Image CI Package version

Commit activity Discord Shield

Try it - Website - Docs - Discord - Hub - Contributor's guide

Windmill - Developer platform for APIs, background jobs, workflows and UIs

Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instances and commercial support and licenses.

Windmill Diagram

https://github.com/user-attachments/assets/d80de1d9-64de-4d89-aacd-6df23fa81fc4

Main Concepts

  1. Define a minimal and generic script in Python, TypeScript, Go or Bash that solves a specific task. The code can be defined in the provided Web IDE or synchronized with your own GitHub repo (e.g. through VS Code extension): provided Web IDE or synchronized with your own GitHub repo (e.g. through VS Code extension):

Step 1

  1. Your scripts parameters are automatically parsed and generate a frontend.

Step 2

Step 3

  1. Make it flow! You can chain your scripts or scripts made by the community shared on WindmillHub.

Step 3

  1. Build complex UIs on top of your scripts and flows.

Step 4

Scripts and flows can be triggered by schedules, webhooks, HTTP routes, Kafka, WebSockets, emails, and more.

Build your entire infra on top of Windmill!

Show me some actual script code

//import any dependency  from npm
import * as wmill from "windmill-client";
import * as cowsay from "cowsay@1.5.0";

// fill the type, or use the +Resource type to get a type-safe reference to a resource
type Postgresql = {
  host: string;
  port: number;
  user: string;
  dbname: string;
  sslmode: string;
  password: string;
};

export async function main(
  a: number,
  b: "my" | "enum",
  c: Postgresql,
  d = "inferred type string from default arg",
  e = { nested: "object" }
  //f: wmill.Base64
) {
  const email = process.env["WM_EMAIL"];
  // variables are permissioned and by path
  let variable = await wmill.getVariable("f/company-folder/my_secret");
  const lastTimeRun = await wmill.getState();
  // logs are printed and always inspectable
  console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun }));
  await wmill.setState(Date.now());

  // return is serialized as JSON
  return { foo: d, variable };
}

Local Development

Windmill supports multiple ways to develop locally and sync with your instance:

Tool Description
CLI Sync scripts from local files or GitHub, run scripts/flows from the command line
VS Code Extension Edit and test scripts & flows directly from VS Code / Cursor with full IDE support
Git Sync Two-way sync between Windmill and your Git repository
Claude Code AI-assisted development with Claude for scripts, flows, and apps

https://github.com/user-attachments/assets/c541c326-e9ae-4602-a09a-1989aaded1e9

You can run scripts locally by passing the right environment variables for the wmill client library to fetch resources and variables from your instance. See local development docs.

Stack

  • Database: Postgres (compatible with Aurora, Cloud SQL, Neon, Azure PostgreSQL)
  • Backend: Rust - stateless API servers and workers pulling jobs from a Postgres queue
  • Frontend: Svelte 5
  • Sandboxing: nsjail and PID namespace isolation
  • Runtimes:
    • TypeScript/JavaScript: Bun (default) and Deno
    • Python: python3 with uv for dependency management
    • Go, Bash, PowerShell, PHP, Rust, C#, Java, Ansible

Fastest Self-Hostable Workflow Engine

We have compared Windmill to other self-hostable workflow engines (Airflow, Prefect & Temporal) and Windmill is the most performant solution for both benchmarks: one flow composed of 40 lightweight tasks & one flow composed of 10 long-running tasks.

All methodology & results on our Benchmarks page.

Fastest workflow engine

Security

  • Sandboxing: nsjail for filesystem/resource isolation, and PID namespace isolation (enabled by default) to prevent jobs from accessing worker process memory
  • Secrets: One encryption key per workspace for credentials stored in Windmill's K/V store. We recommend encrypting the Postgres database as well.

See Security documentation for details.

Performance

Once a job started, there is no overhead compared to running the same script on the node with its corresponding runner (Deno/Go/Python/Bash). The added latency from a job being pulled from the queue, started, and then having its result sent back to the database is ~50ms. A typical lightweight deno job will take around 100ms total.

Architecture

How to self-host

For detailed setup options, see Self-Host documentation.

Docker compose

Deploy Windmill with 3 files (docker-compose.yml, Caddyfile, .env):

curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env

docker compose up -d

Go to http://localhost - default credentials: admin@windmill.dev / changeme

Using an external database: Set DATABASE_URL in .env to point to your managed Postgres (AWS RDS, GCP Cloud SQL, Azure, Neon, etc.) and set db replicas to 0.

Kubernetes (Helm charts)

helm repo add windmill https://windmill-labs.github.io/windmill-helm-charts/
helm install windmill-chart windmill/windmill --namespace=windmill --create-namespace

See windmill-helm-charts for configuration options.

Cloud providers

Windmill works on AWS (EKS/ECS), GCP, Azure, Ubicloud, Fly.io, Render.com, Hetzner, Digital Ocean, and others. Rule of thumb: 1 worker per 1vCPU and 1-2 GB RAM.

OAuth, SSO & SMTP

Configure OAuth and SSO (Google Workspace, Microsoft/Azure, Okta) directly from the superadmin UI. See documentation.

License

The Community Edition is free to use internally. For commercial redistribution or managed services, contact sales@windmill.dev. See LICENSE and Pricing for details.

The "Community Edition" of Windmill available in the docker images hosted under ghcr.io/windmill-labs/windmill and the github binary releases contains the files under the AGPLv3 and Apache 2 sources but also includes proprietary and non-public code and features which are not open source and under the following terms: Windmill Labs, Inc. grants a right to use all the features of the "Community Edition" for free without restrictions other than the limits and quotas set in the software and a right to distribute the community edition as is but not to sell, resell, serve Windmill as a managed service, modify or wrap under any form without an explicit agreement.

The binary compilable from source code in this repository without the "enterprise" feature flag is open-source under the LICENSE-AGPLv3 License terms and conditions.

To re-expose directly any Windmill parts to your users as a feature of your product, with the exception of iframed public Windmill "apps", or to build a feature on top of "Windmill Community Edition" that you sell commercially or embed in a distributable product or binary, you must get a commercial license. Contact us at sales@windmill.dev if you have any questions. To do the same from the binary compiled from the source code in this repository without the "enterprise" feature flag, you must comply with the AGPLv3 license terms and conditions or get a commercial license from Windmill Labs, Inc.

To use Windmill "Community Edition" as is internally in your organization, or to use its APIs as is, you do NOT need a commercial license.

Integrations

In Windmill, integrations are referred to as resources and resource types. Each Resource has a Resource Type that defines the schema that the resource needs to implement.

On self-hosted instances, you might want to import all the approved resource types from WindmillHub. A setup script will prompt you to have it being synced automatically everyday.

Environment Variables

Environment Variable name Default Description Api Server/Worker/All
DATABASE_URL The Postgres database url. All
WORKER_GROUP default The worker group the worker belongs to and get its configuration pulled from Worker
MODE standalone The mode if the binary. Possible values: standalone, worker, server, agent All
METRICS_ADDR None (ee only) The socket addr at which to expose Prometheus metrics at the /metrics path. Set to "true" to expose it on port 8001 All
JSON_FMT false Output the logs in json format instead of logfmt All
BASE_URL http://localhost:8000 The base url that is exposed publicly to access your instance. Is overriden by the instance settings if any. Server
ZOMBIE_JOB_TIMEOUT 30 The timeout after which a job is considered to be zombie if the worker did not send pings about processing the job (every server check for zombie jobs every 30s) Server
RESTART_ZOMBIE_JOBS true If true then a zombie job is restarted (in-place with the same uuid and some logs), if false the zombie job is failed Server
NATIVE_MODE false Enable native mode: sets NUM_WORKERS=8, rejects non-native jobs (nativets, postgresql, mysql, etc.) Worker
SLEEP_QUEUE 50 The number of ms to sleep in between the last check for new jobs in the DB. It is multiplied by NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. Worker
KEEP_JOB_DIR false Keep the job directory after the job is done. Useful for debugging. Worker
LICENSE_KEY (EE only) None License key checked at startup for the Enterprise Edition of Windmill Worker
SLACK_SIGNING_SECRET None The signing secret of your Slack app. See Slack documentation Server
COOKIE_DOMAIN None The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin Server
DENO_PATH /usr/bin/deno The path to the deno binary. Worker
PYTHON_PATH The path to the python binary if wanting to not have it managed by uv. Worker
GO_PATH /usr/bin/go The path to the go binary. Worker
GOPRIVATE The GOPRIVATE env variable to use private go modules Worker
GOPROXY The GOPROXY env variable to use Worker
NETRC The netrc content to use a private go registry Worker
PY_CONCURRENT_DOWNLOADS 20 Sets the maximum number of in-flight concurrent python downloads that windmill will perform at any given time. Worker
PATH None The path environment variable, usually inherited Worker
HOME None The home directory to use for Go and Bash , usually inherited Worker
DATABASE_CONNECTIONS 50 (Server)/3 (Worker) The max number of connections in the database connection pool All
SUPERADMIN_SECRET None A token that would let the caller act as a virtual superadmin superadmin@windmill.dev Server
TIMEOUT_WAIT_RESULT 20 The number of seconds to wait before timeout on the 'run_wait_result' endpoint Worker
QUEUE_LIMIT_WAIT_RESULT None The number of max jobs in the queue before rejecting immediately the request in 'run_wait_result' endpoint. Takes precedence on the query arg. If none is specified, there are no limit. Worker
DENO_AUTH_TOKENS None Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules Worker
DISABLE_RESPONSE_LOGS false Disable response logs Server
CREATE_WORKSPACE_REQUIRE_SUPERADMIN true If true, only superadmins can create new workspaces Server
MIN_FREE_DISK_SPACE_MB 15000 Minimum amount of free space on worker. Sends critical alert if worker has less free space. Worker
RUN_UPDATE_CA_CERTIFICATE_AT_START false If true, runs CA certificate update command at startup before other initialization All
RUN_UPDATE_CA_CERTIFICATE_PATH /usr/sbin/update-ca-certificates Path to the CA certificate update command/script to run when RUN_UPDATE_CA_CERTIFICATE_AT_START is true All

Run a local dev setup

We recommend using Nix. See ./frontend/README_DEV.md for all options.

Frontend only

Uses the backend of https://app.windmill.dev with local frontend (hot-reload):

cd frontend
npm install
npm run generate-backend-client  # or generate-backend-client-mac on Mac
npm run dev

Windmill available at http://localhost/

Backend + Frontend

See the ./frontend/README_DEV.md file for all running options.

  1. Start a local Postgres database using for instance the start-dev-db.sh script which will make a database available at postgres://postgres:changeme@localhost:5432/windmill Then run the migrations using the following command:
    cargo install sqlx-cli
    env DATABASE_URL=<YOUR_DATABASE_URL> sqlx migrate run
    
    This will also avoid compile time issue with sqlx's query! macro.
  2. (optional, linux only) Install nsjail and have it accessible in your PATH
  3. Install bun, deno and python3 (+ any languages you want to use), have the bins at /usr/bin/bun,/usr/bin/deno, and /usr/local/bin/python3 or set the corresponding environment variables.
  4. (optional) Install the lld linker
  5. Go to frontend/:
    1. npm install, npm run generate-backend-client then REMOTE=http://localhost:8000 npm run dev
    2. You might need to set some extra heap space for the node runtime export NODE_OPTIONS="--max-old-space-size=4096"
    3. Create an empty frontend/build folder using mkdir frontend/build
  6. Go to backend/:
    1. env DATABASE_URL=<YOUR_DATABASE_URL> RUST_LOG=info cargo run
    2. You can specify any feature flag you want to enable, for example cargo run --features python to enable the python executor.
  7. Windmill should be available at http://localhost:3000

Contributors

© 2023-2026 Windmill Labs, Inc.

S
Description
Open-source developer platform to power your entire infra and turn scripts into webhooks, workflows and UIs. Fastest workflow engine (13x vs Airflow). Open-source alternative to Retool and Temporal.
Readme
926 MiB
Languages
Rust 34.6%
TypeScript 25.6%
Svelte 23.9%
HTML 12.1%
JavaScript 1.5%
Other 2.1%