hugocasaandClaude Opus 5 d54a66f15c fix(cli): say where a sync push deleted variable or resource went (#10851)
* fix(cli): keep variables and resources a sync push repo never tracked

`wmill sync push` archives a script it no longer finds locally, but hard-deletes
a variable or a resource: the credentials go for good. A remote-only one is
equally a deletion being deployed and one the repository never had, provisioned
on the instance or written by a script at runtime, and reading the second as a
deletion is unrecoverable.

Committed history tells them apart. A push whose changeset deletes a variable or
resource now asks what this branch has ever tracked at `*.variable.*` /
`*.resource.*`; anything it has never recorded is kept on the remote (prompted
for on a TTY), and a real deletion, recorded before the commit that removed it,
still applies. Where the history cannot be read (shallow clone, sparse checkout,
no repository) there is no evidence either way, so the deletion stands as before
with a warning naming the remedy — the git-sync "Pull from repo" job runs in a
depth-1 clone and must keep deploying the deletions it always has.

`--delete-untracked-secrets` / `deleteUntrackedSecrets` opts a mirror-semantics
pipeline back into deleting them unattended.

Fixes GIT-980

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): classify secret-bearing deletions the way the push itself does

Three ways the suffix match missed:

- A fileset child can be any file, `inner.resource.yaml` included, and its
  deletion re-pushes the parent rather than deleting anything. Classifying with
  the push's own `getTypeStrFromPath`, behind the same fileset exclusion the
  apply loop uses, keeps the two in step.
- Deleting `f/x.resource.file.ini` deletes the resource `f/x` outright, so
  without that file in the pathspecs every file resource walked past the check.
  Its two files now count as the one resource they delete.
- A `specificItems` item is committed as `y.<workspace>.variable.yaml` while
  `elementsToMap` collapses it to the base path the changeset carries, so a
  deletion the user did commit read as never tracked. The history is searched
  under both names.

`gitRecordedPaths` also reads its history with `core.quotePath=false`: a path
with a non-ASCII byte came back C-quoted and matched nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): judge held-back deletions per object, not per file

`DELETE /variables/delete` takes the resource at the same path down with it, and
`DELETE /resources/delete` does the same to the variables its value references,
so a tracked deletion could destroy an untracked object the push had just
reported it was keeping. A file resource had the same shape from the other end:
two files for one resource, either survivor deleting it.

The unit is the server-side object. One file left unaccounted for by history now
holds the whole object back, so nothing in a group reported as kept is deleted.
The residual is a resource whose value references a variable at another path,
which stays possible and is called out in the PR.

Also corrects what the messages claim. Deleting a variable or resource is not
irrecoverable: both move to the workspace trash, which keeps them for three days
(migrations/20260326000000_trashbin.up.sql, CE since v1.665.0). The asymmetry
with a script is real but narrower, and the prompt defaults to No, so it should
say what it actually costs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): warn when sync push deletes variables the repo never tracked

`sync push` deletes a remote variable or resource that has no local file. An
object the repository has never tracked was provisioned outside it, by hand or
by a script at runtime, rather than deleted from it, and the change list said
nothing to tell the two apart.

The push still deploys every deletion — that is what the repo-is-the-mirror
contract means, and a shallow clone (the git-sync "Pull from repo" job, a
default actions/checkout) could not tell them apart anyway. What changes is that
the preview names the ones this branch's history has no record of, before the
prompt that confirms them, and points at the excludes that stop them recurring.

Deleting is also not final, which the CLI was alone in not saying: both handlers
move the item to the workspace trash first, restorable for three days
(migrations/20260326000000_trashbin.up.sql, CE since v1.665.0). A push that
deleted any now says so.

This replaces the earlier hold-back design. Keeping objects back changed what a
push deploys, needed a flag and a wmill.yaml key to opt out of, and could claim
to keep an object that a linked deletion then cascaded onto. Reporting cannot.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): name the paths the untracked-deletion warning is about

A push can delete a tracked and a never-tracked resource together, where "1
resource" identified neither. The warning lists the paths instead of counting
them, so the reader knows which one to exclude.

Outside a git checkout it no longer opens "This branch's history", which
contradicted the reason it went on to give, and it drops the pronouns that
disagreed with a plural count. The history walk is skipped under --json-output,
where both notices are silenced and its result had no reader.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): vouch for an object with any file in history, not just deleted ones

The tracked set was built from the deletions being judged, so a companion file
the push was not deleting could not vouch for its object: a file resource whose
`.resource.yaml` stays while its content file goes was reported as never
tracked, though the repository plainly owned it. It is built from the whole
history now, which also turns the workspace-specific lookup around — history is
normalized to base paths, the form the changeset already carries, instead of
each candidate being searched for under two names.

The warning also prints one line per server-side object rather than per file, so
a file resource is the one deletion it is rather than two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): name both kinds at a shared path, gate the history path conversion

Three from review:

A variable and a resource at one path are judged together, since deleting either
takes both, but they are two objects to name — keying the printed lines by path
alone dropped one of them.

`fromWorkspaceSpecificPath` strips a `.<workspace>` segment wherever it finds
one, so a history entry that merely looks workspace-suffixed was re-keyed onto a
different object, whose history then vouched for it. Only a path `specificItems`
claims is converted now. Not reachable from a server object (the backend rejects
`.` in paths), but history holds whatever was committed.

`secretBearingKey` lost its last caller when the tracked set moved to object
paths; removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): identify a secret-bearing object by kind as well as path

A variable and a resource can share a path and are still two backend objects:
`DELETE /variables/delete` drops the same-path resource unconditionally, while
`DELETE /resources/delete` drops only the variables its value references. Keying
tracked history by path alone let a committed variable vouch for a resource the
repository never had, which then went unmentioned. The cascade is a reason to
report both, not to treat them as one. A file resource's two files keep one id.

`git log HEAD` also fails on a repository with no commits, which was reported as
"its history could not be read. Check that git runs correctly in this directory"
— true of neither.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(cli): tighten the comments on the untracked-deletion warning

Halves the prose without dropping a constraint: the trashbin retention is
stated where the message says it rather than twice more in doc comments, and the
two stacked comments at the print site had come to contradict each other, one
still describing a same-path variable and resource as judged together.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): say where a deleted variable or resource went

`sync push` deletes a remote variable or resource that has no local file, and
said nothing more. Both handlers move the item to the workspace trash first,
restorable for three days (migrations/20260326000000_trashbin.up.sql, CE since
v1.665.0), and the CLI was the one surface never to mention it — the report
behind this concluded the deletion was final and there was nothing to restore.

A push that deleted any now ends with where they went and how long they have.

Drops the untracked-deletion warning this branch carried: distinguishing a
deletion the repo deployed from an object it never owned needs the branch's git
history, and roughly 130 lines to read it and be right about the answer, for a
claim the trash already softens. Two fixes it turned up in the data-table
migration guard, which reads history the same way, are kept: a path with a
non-ASCII byte came back C-quoted and matched nothing, and a repository with no
commits was reported as one where git does not run.

Fixes GIT-980

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): name the trashbin correctly and say who can restore

The tab is labelled Trashbin, not Trash, and `restore_trash_item` requires
admin, so a non-admin reading the old line would go looking for a control they
do not have.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(cli): move the migration-guard git fixes to their own PR

They fix `gitRecordedDatatableMigrationPaths`, which this PR no longer touches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(cli): don't count a .lock deletion the push skips

The apply loop `continue`s past a non-raw-app, non-dbt `.lock` deletion before
reaching the delete switch, so nothing happens on the server. The classifier did
not mirror that, and `f/x.resource.file.lock` reaches it as a resource through
`isFileResource` — a resource type whose format_extension is literally `lock`
would have the notice announce a deletion the push never performed.

A raw-app or dbt `.lock`, the two that loop does not skip, classifies as its
bundle's own kind well before the file-resource check, so a suffix test is
enough.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(cli): state what the classification tests protect

The header described the change rather than the invariant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 19:07:33 +02:00
2024-02-08 16:09:11 +01:00
2026-02-03 17:50:56 +00:00
2026-09-13 18:40:23 +00:00
2023-08-26 09:17:07 +02:00
2026-08-19 14:18:37 +00:00
2025-04-21 16:53:23 +02:00
2026-04-16 06:26:25 -07:00
2025-12-08 10:34:16 +00:00
2025-03-07 15:08:19 +01:00
2022-05-05 04:25:58 +02:00
2024-02-08 16:09:11 +01:00

windmill.dev

Open-source developer platform for internal code: APIs, background jobs, workflows and UIs. Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.

Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported languages: Python, TypeScript, Go, Bash, SQL, GraphQL, PowerShell, Rust, and more.

Package version Docker Image CI Package version

Commit activity Discord Shield

Try it - Website - Docs - Discord - Hub - Contributing

Windmill - Developer platform for APIs, background jobs, workflows and UIs

Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instances and commercial support and licenses.

Windmill Diagram

https://github.com/user-attachments/assets/d80de1d9-64de-4d89-aacd-6df23fa81fc4

Main Concepts

  1. Define a minimal and generic script in Python, TypeScript, Go or Bash that solves a specific task. The code can be defined in the provided Web IDE or synchronized with your own GitHub repo (e.g. through VS Code extension): provided Web IDE or synchronized with your own GitHub repo (e.g. through VS Code extension):

Step 1

  1. Your scripts parameters are automatically parsed and generate a frontend.

Step 2

Step 3

  1. Make it flow! You can chain your scripts or scripts made by the community shared on WindmillHub.

Step 3

  1. Build complex UIs on top of your scripts and flows.

Step 4

Scripts and flows can be triggered by schedules, webhooks, HTTP routes, Kafka, WebSockets, emails, and more.

Build your entire infra on top of Windmill!

Show me some actual script code

//import any dependency  from npm
import * as wmill from "windmill-client";
import * as cowsay from "cowsay@1.5.0";

// fill the type, or use the +Resource type to get a type-safe reference to a resource
type Postgresql = {
  host: string;
  port: number;
  user: string;
  dbname: string;
  sslmode: string;
  password: string;
};

export async function main(
  a: number,
  b: "my" | "enum",
  c: Postgresql,
  d = "inferred type string from default arg",
  e = { nested: "object" }
  //f: wmill.Base64
) {
  const email = process.env["WM_EMAIL"];
  // variables are permissioned and by path
  let variable = await wmill.getVariable("f/company-folder/my_secret");
  const lastTimeRun = await wmill.getState();
  // logs are printed and always inspectable
  console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun }));
  await wmill.setState(Date.now());

  // return is serialized as JSON
  return { foo: d, variable };
}

Local Development

Windmill supports multiple ways to develop locally and sync with your instance:

Tool Description
CLI Sync scripts from local files or GitHub, run scripts/flows from the command line
VS Code Extension Edit and test scripts & flows directly from VS Code / Cursor with full IDE support
Git Sync Two-way sync between Windmill and your Git repository
Claude Code AI-assisted development with Claude for scripts, flows, and apps

https://github.com/user-attachments/assets/c541c326-e9ae-4602-a09a-1989aaded1e9

You can run scripts locally by passing the right environment variables for the wmill client library to fetch resources and variables from your instance. See local development docs.

Stack

  • Database: Postgres (compatible with Aurora, Cloud SQL, Neon, Azure PostgreSQL)
  • Backend: Rust - stateless API servers and workers pulling jobs from a Postgres queue
  • Frontend: Svelte 5
  • Sandboxing: nsjail and PID namespace isolation
  • Runtimes:
    • TypeScript/JavaScript: Bun (default) and Deno
    • Python: python3 with uv for dependency management
    • Go, Bash, PowerShell, PHP, Rust, C#, Java, Ansible

Fastest Self-Hostable Workflow Engine

We have compared Windmill to other self-hostable workflow engines (Airflow, Prefect & Temporal) and Windmill is the most performant solution for both benchmarks: one flow composed of 40 lightweight tasks & one flow composed of 10 long-running tasks.

All methodology & results on our Benchmarks page.

Fastest workflow engine

Security

  • Sandboxing: nsjail for filesystem/resource isolation, and PID namespace isolation (enabled by default) to prevent jobs from accessing worker process memory
  • Secrets: One encryption key per workspace for credentials stored in Windmill's K/V store. We recommend encrypting the Postgres database as well.

See Security documentation for details.

Performance

Once a job started, there is no overhead compared to running the same script on the node with its corresponding runner (Deno/Go/Python/Bash). The added latency from a job being pulled from the queue, started, and then having its result sent back to the database is ~50ms. A typical lightweight deno job will take around 100ms total.

Architecture

How to self-host

For detailed setup options, see Self-Host documentation.

Docker compose

Deploy Windmill with 3 files (docker-compose.yml, Caddyfile, .env):

curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env

docker compose up -d

Go to http://localhost - default credentials: admin@windmill.dev / changeme

Using an external database: Set DATABASE_URL in .env to point to your managed Postgres (AWS RDS, GCP Cloud SQL, Azure, Neon, etc.) and set db replicas to 0.

Kubernetes (Helm charts)

helm repo add windmill https://windmill-labs.github.io/windmill-helm-charts/
helm install windmill-chart windmill/windmill --namespace=windmill --create-namespace

See windmill-helm-charts for configuration options.

Cloud providers

Windmill works on AWS (EKS/ECS), GCP, Azure, Ubicloud, Fly.io, Render.com, Hetzner, Digital Ocean, and others. Rule of thumb: 1 worker per 1vCPU and 1-2 GB RAM.

OAuth, SSO & SMTP

Configure OAuth and SSO (Google Workspace, Microsoft/Azure, Okta) directly from the superadmin UI. See documentation.

License

The Community Edition is free to use internally. For commercial redistribution or managed services, contact sales@windmill.dev. See LICENSE and Pricing for details.

The "Community Edition" of Windmill available in the docker images hosted under ghcr.io/windmill-labs/windmill and the github binary releases contains the files under the AGPLv3 and Apache 2 sources but also includes proprietary and non-public code and features which are not open source and under the following terms: Windmill Labs, Inc. grants a right to use all the features of the "Community Edition" for free without restrictions other than the limits and quotas set in the software and a right to distribute the community edition as is but not to sell, resell, serve Windmill as a managed service, modify or wrap under any form without an explicit agreement.

The binary compilable from source code in this repository without the "enterprise" feature flag is open-source under the LICENSE-AGPLv3 License terms and conditions.

To re-expose directly any Windmill parts to your users as a feature of your product, with the exception of iframed public Windmill "apps", or to build a feature on top of "Windmill Community Edition" that you sell commercially or embed in a distributable product or binary, you must get a commercial license. Contact us at sales@windmill.dev if you have any questions. To do the same from the binary compiled from the source code in this repository without the "enterprise" feature flag, you must comply with the AGPLv3 license terms and conditions or get a commercial license from Windmill Labs, Inc.

To use Windmill "Community Edition" as is internally in your organization, or to use its APIs as is, you do NOT need a commercial license.

Integrations

In Windmill, integrations are referred to as resources and resource types. Each Resource has a Resource Type that defines the schema that the resource needs to implement.

On self-hosted instances, you might want to import all the approved resource types from WindmillHub. A setup script will prompt you to have it being synced automatically everyday.

Environment Variables

Environment Variable name Default Description Api Server/Worker/All
DATABASE_URL The Postgres database url. All
WORKER_GROUP default The worker group the worker belongs to and get its configuration pulled from Worker
MODE standalone The mode if the binary. Possible values: standalone, worker, server, agent All
METRICS_ADDR None (ee only) The socket addr at which to expose Prometheus metrics at the /metrics path. Set to "true" to expose it on port 8001 All
JSON_FMT false Output the logs in json format instead of logfmt All
BASE_URL http://localhost:8000 The base url that is exposed publicly to access your instance. Is overriden by the instance settings if any. Server
ZOMBIE_JOB_TIMEOUT 30 The timeout after which a job is considered to be zombie if the worker did not send pings about processing the job (every server check for zombie jobs every 30s) Server
RESTART_ZOMBIE_JOBS true If true then a zombie job is restarted (in-place with the same uuid and some logs), if false the zombie job is failed Server
NATIVE_MODE false Enable native mode: sets NUM_WORKERS=8, rejects non-native jobs (nativets, postgresql, mysql, etc.) Worker
SLEEP_QUEUE 50 The number of ms to sleep in between the last check for new jobs in the DB. It is multiplied by NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. Worker
KEEP_JOB_DIR false Keep the job directory after the job is done. Useful for debugging. Worker
EXIT_AFTER_N_JOBS None Exit the worker process after it has executed that many jobs, so that a supervisor restarts it and no process runs more than that many, bar the steps of a same-worker flow it has started, which it always finishes (set it to 1 for a process per job; jobs handed to a dedicated worker, and the worker's own init and periodic scripts, do not count). Not counting the init and periodic scripts means they run again on every restart: an init script's runtime is added to the latency of every batch of that many jobs, and a periodic script fires once per process start whatever its interval says. The worker's shell in the workers page also starts backed off rather than after the two minutes it otherwise takes, since a process due to be recycled cannot count on living that long: the first command of a session can wait up to 15s, later ones are immediate. For deployments that isolate executions by process lifetime rather than with nsjail; note that a container restart resets the process, not the container filesystem, so caches and /tmp survive it. The worker name is then derived from the hostname instead of being random, so the restarted worker keeps its row in the workers list (an agent worker keeps the row but restarts its job count). Use one worker per process: workers of one process share its environment, so the first to reach the limit shuts the others down too. Worker
WORKER_SUFFIX None Pins the last part of the worker name, which is otherwise random, so that a restarted worker keeps its row in the workers list. Only needed when several worker processes of the same worker group run on one host, since the name is derived from the hostname: give each of them a distinct value, as two processes sharing one must never happen. At most 64 letters, digits and underscores; anything else is refused at startup. Worker
LICENSE_KEY (EE only) None License key checked at startup for the Enterprise Edition of Windmill Worker
SLACK_SIGNING_SECRET None The signing secret of your Slack app. See Slack documentation Server
COOKIE_DOMAIN None The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin Server
DENO_PATH /usr/bin/deno The path to the deno binary. Worker
PYTHON_PATH The path to the python binary if wanting to not have it managed by uv. Worker
GO_PATH /usr/bin/go The path to the go binary. Worker
GOPRIVATE The GOPRIVATE env variable to use private go modules Worker
GOPROXY The GOPROXY env variable to use Worker
NETRC The netrc content to use a private go registry Worker
PY_CONCURRENT_DOWNLOADS 20 Sets the maximum number of in-flight concurrent python downloads that windmill will perform at any given time. Worker
PATH None The path environment variable, usually inherited Worker
HOME None The home directory to use for Go and Bash , usually inherited Worker
DATABASE_CONNECTIONS 50 (Server)/3 (Worker) The max number of connections in the database connection pool All
SUPERADMIN_SECRET None A token that would let the caller act as a virtual superadmin superadmin@windmill.dev Server
TIMEOUT_WAIT_RESULT 20 The number of seconds to wait before timeout on the 'run_wait_result' endpoint Worker
QUEUE_LIMIT_WAIT_RESULT None The number of max jobs in the queue before rejecting immediately the request in 'run_wait_result' endpoint. Takes precedence on the query arg. If none is specified, there are no limit. Worker
DENO_AUTH_TOKENS None Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules Worker
DISABLE_RESPONSE_LOGS false Disable response logs Server
CREATE_WORKSPACE_REQUIRE_SUPERADMIN true If true, only superadmins can create new workspaces Server
MIN_FREE_DISK_SPACE_MB 15000 Minimum amount of free space on worker. Sends critical alert if worker has less free space. Worker
RUN_UPDATE_CA_CERTIFICATE_AT_START false If true, runs CA certificate update command at startup before other initialization All
RUN_UPDATE_CA_CERTIFICATE_PATH /usr/sbin/update-ca-certificates Path to the CA certificate update command/script to run when RUN_UPDATE_CA_CERTIFICATE_AT_START is true All
GOOGLE_APPLICATION_CREDENTIALS None (ee only) Credentials file for GCP Pub/Sub triggers that authenticate as the instance rather than through a gcloud resource (workspace admins only). Application default credentials also resolve the gcloud well-known file and the GCE metadata server. Workload Identity Federation files work with the file, url and aws credential sources; the executable source is not supported. Server

Run a local dev setup

We recommend using Nix. See ./frontend/README_DEV.md for all options.

Frontend only

Uses the backend of https://app.windmill.dev with local frontend (hot-reload):

cd frontend
npm install
npm run generate-backend-client  # or generate-backend-client-mac on Mac
npm run dev

Windmill available at http://localhost/

Backend + Frontend

See the ./frontend/README_DEV.md file for all running options.

  1. Start a local Postgres database using for instance the start-dev-db.sh script which will make a database available at postgres://postgres:changeme@localhost:5432/windmill Then run the migrations using the following command:
    cargo install sqlx-cli
    env DATABASE_URL=<YOUR_DATABASE_URL> sqlx migrate run
    
    This will also avoid compile time issue with sqlx's query! macro.
  2. (optional, linux only) Install nsjail and have it accessible in your PATH
  3. Install bun, deno and python3 (+ any languages you want to use), have the bins at /usr/bin/bun,/usr/bin/deno, and /usr/local/bin/python3 or set the corresponding environment variables.
  4. (optional) Install the lld linker
  5. Go to frontend/:
    1. npm install, npm run generate-backend-client then REMOTE=http://localhost:8000 npm run dev
    2. You might need to set some extra heap space for the node runtime export NODE_OPTIONS="--max-old-space-size=4096"
    3. Create an empty frontend/build folder using mkdir frontend/build
  6. Go to backend/:
    1. env DATABASE_URL=<YOUR_DATABASE_URL> RUST_LOG=info cargo run
    2. You can specify any feature flag you want to enable, for example cargo run --features python to enable the python executor.
  7. Windmill should be available at http://localhost:3000

Contributing

At this time, we are not seeking outside contribution. Bug reports and feature requests remain very welcome, and small, trivially-verified PRs that fix a problem are still accepted. See CONTRIBUTING.md for the full policy.

Contributors

© 2023-2026 Windmill Labs, Inc.

S
Description
Open-source developer platform to power your entire infra and turn scripts into webhooks, workflows and UIs. Fastest workflow engine (13x vs Airflow). Open-source alternative to Retool and Temporal.
Readme
882 MiB
Languages
Rust 34.1%
TypeScript 25.4%
Svelte 24.1%
HTML 12.6%
JavaScript 1.4%
Other 2.2%