Deploying Windmill on AWS ECS ============================= This folder contains terraform files to deploy a modular Windmill stack on AWS ECS with just a few commands. ### Pre-requisite ##### AWS terraform user An AWS user with the AWS managed `AdministratorAccess` policy (arn: `arn:aws:iam::aws:policy/AdministratorAccess`) is necessary. This will be the user terraform uses to deploy the components to AWS. Generate an AWS access key ID and secret, and append the following block to your `~/.aws/credentials`: ``` [terraform] aws_access_key_id = aws_secret_access_key = ``` ##### Creating an ecs task execution role Terraform will reference the role `ecsTaskExecutionRole` to launch tasks in the ECS cluster. SInce it is a common role, this Terraform does not create it, it need to be present in your AWS account. If it's not, just create it in you IAM console and attach it the following AWS managed policy: `AmazonECSTaskExecutionRolePolicy` (arn: `arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy`) ##### Database password Windmill relies on a PostgreSQL database which will be created by Terraform. The only think you need to manually provide is a strong password. It should be set in [terraform.tfvars](./terraform.tfvars). ### Windmill stack The following stack will be deployed by this terraform as it is provided: - 2 Windmill servers (each requiring 1 CPU and 1.5GiB of Memory) - 2 multi-purpose Windmill workers (each requiring 2 CPU and 3GiB of Memory) - Windmill Extra: LSP and Multiplayer, plus the optional Debugger (requiring 1 CPU and 1.5GiB of Memory) - 1 native Windmill worker (requiring 2 CPU and 3GiB of Memory) - 1 high performance Windmill worker (requiring 4 CPU and 15GiB of Memory) The ECS cluster will be composed of 1 autoscaling group of `t3.medium` instances, sized by the ECS capacity provider up to `max_size` (10) in [ecs_cluster.tf](./ecs_cluster.tf). In our test deployment it settled on 5 instances for this load; the headroom covers rolling upgrades. This will host all the services except the high performance workers. Those will be deployed on a separate auto-scaling group composed of up-to 2 `t3.xlarge` instances. Of course the above is provided as an example, it should be tuned for your own needs, both in terms of instance specs, but also in terms of service architecture. For example, you might not need high performance workers, in which case you won't need the second autoscaling group at all. Same for the native worker, or even Windmill Extra (though its LSP is highly recommended for a better coding experience). Its services are switched on and off with the `ENABLE_LSP`, `ENABLE_MULTIPLAYER` (Enterprise Edition) and `ENABLE_DEBUGGER` variables in [windmill_extra.tf](./windmill_extra.tf). The only strictly required components are: - At least 1 Windmill server - At least 1 multi-purpose Windmill worker This terraform has been assembled to easily remove components. Each component mentioned in the above list is entirely contained in a `.tf` file. If you don't want it, just remove the file. And if you want to tune it, you know where to look. - _REQUIRED_ Windmill server -> [windmill_server.tf](./windmill_server.tf) - _REQUIRED_ Multi-purpose windmill worker -> [windmill_worker_basic.tf](./windmill_worker_basic.tf) - _OPTIONAL_ Windmill Extra (LSP, Multiplayer, Debugger) -> [windmill_extra.tf](./windmill_extra.tf) - _OPTIONAL_ Windmill Native worker -> [windmill_worker_native.tf](./windmill_worker_native.tf) - _OPTIONAL_ Windmill High Performace worker -> [windmill_worker_high_performance.tf](./windmill_worker_high_performance.tf) - this one is a lot longer because it deploys a new auto scaling group ### Network The network deployed here is a single VPC, composed on 4 subnets spread across 2 availability zones (1 public and 1 private subnet per zone). All the services are behind a load balancer routing the request to Windmill server or Windmill Extra (`/ws/*`, `/ws_mp/*`, `/ws_debug/*`). A single security group is used, allowing HTTP traffic on port 80 (it is not deploying custom certificates and therefore does not use HTTPS). All this is provided as an example. It can be refined depending on your needs. All the network components are defined in [vpc.tf](./vpc.tf), except the security group which is is [security_group.tf](./security_group.tf) and load balancer which is defined in [load_balancer.tf](./load_balancer.tf) ### RDS Database Windmill heavily relies on PostgreSQL database. This terraform deploys a standalone RDS instance having 100GiB of storage, which can be scaled up to 1000GiB. For production use cases, we recommend deploying a Multi-AZ instance, or even a Multi-AZ DB cluster. The RDS definition is in [rds.tf](./rds.tf) ### Upgrading a stack deployed before Windmill Extra Earlier versions of this example ran the LSP and Multiplayer as two separate services (`windmill_lsp.tf` and `windmill_multiplayer.tf`). A plain `terraform apply` of this version migrates such a stack: the `moved` blocks in [load_balancer.tf](./load_balancer.tf) keep the listener rule in place, and the old services and target groups are removed. While the new `windmill-extra` task starts (about 2.5 minutes in our test), the `/ws/*` and `/ws_mp/*` routes answer 503, so code intelligence and multiplayer are unavailable in the editor until it is healthy. The debugger (`/ws_debug/*`) stays off in this example unless you set `ENABLE_DEBUGGER=true` in [windmill_extra.tf](./windmill_extra.tf). The Windmill servers and workers are not affected. ### Ready? REMINDER: don't forget to edit [terraform.tfvars](./terraform.tfvars) before deploying the stack. Once you're ready, simply run: ```bash # for a dry-run terraform plan ``` and then ```bash terraform apply ``` ![Windmill screenshot](./assets/windmill.png)