mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 16:02:10 +00:00
centdix
74 lines
2.7 KiB
Markdown
74 lines
2.7 KiB
Markdown
# Testing Guide for Windmill CLI
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
# Run unit tests only (fast — no backend, no database, no cargo build)
|
|
bun run test:unit
|
|
|
|
# Run all tests (unit + integration — requires PostgreSQL + cargo)
|
|
DATABASE_URL=postgres://postgres:changeme@localhost:5432 bun run test
|
|
|
|
# Run specific test files
|
|
bun test test/sync_pull_push.test.ts
|
|
bun test test/workspace_conflicts_unit.test.ts
|
|
```
|
|
|
|
## Test Categories
|
|
|
|
### Unit tests (`*_unit.test.ts`)
|
|
|
|
Pure local tests — no backend, no database. Uses `bunfig.unit.toml` (no preload).
|
|
|
|
Examples: `git_unit`, `lint_command_unit`, `tar_creation_unit`, `workspace_conflicts_unit`
|
|
|
|
### Integration tests
|
|
|
|
Require a running backend and PostgreSQL. The `setup.ts` preload builds the backend
|
|
binary and starts a shared backend instance.
|
|
|
|
Examples: `sync_pull_push`, `dev_server`, `standalone_commands`
|
|
|
|
## AI Benchmark Caveats
|
|
|
|
The repo-level benchmark CLI lives under `ai_evals/`, but it currently depends on
|
|
mocked frontend flow execution in a few places. Treat `flow` benchmark passes as
|
|
artifact-shape signal, not full runtime correctness, when either of these apply:
|
|
|
|
- deterministic flow validation does not currently reject syntactically invalid
|
|
`rawscript` module bodies
|
|
- frontend benchmark calls to `test_run_flow` and `test_run_step` return mocked
|
|
completed jobs for `mock-job-id-*` workspaces instead of executing the flow
|
|
|
|
If a prompt change depends on flow wiring or script runtime behavior, verify it
|
|
with additional validation or a real run before trusting the benchmark result.
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Purpose | Default |
|
|
|----------|---------|---------|
|
|
| `DATABASE_URL` | PostgreSQL connection string (without database name) | `postgres://postgres:changeme@localhost:5432` |
|
|
| `TEST_BACKEND` | `cargo` or `docker` | `cargo` |
|
|
| `CI_MINIMAL_FEATURES` | `true` for CI mode (zip-only features) | unset |
|
|
| `EE_LICENSE_KEY` | Enterprise license for EE feature tests | unset |
|
|
| `TEST_FEATURES` | Additional cargo features (comma-separated) | unset |
|
|
| `TEST_CLI_RUNTIME` | `node` to test npm package | unset |
|
|
| `UNIT_ONLY` | `1` to skip backend setup in preload (used by `test:unit`) | unset |
|
|
| `VERBOSE` | `1` for backend process output | unset |
|
|
|
|
## Cleanup
|
|
|
|
Stale test databases (`windmill_test_*`) and orphaned backend processes from
|
|
previous crashed runs are automatically cleaned up when starting a new test run.
|
|
|
|
To manually check for leftovers:
|
|
|
|
```bash
|
|
# Check for stale test databases
|
|
psql postgres://postgres:changeme@localhost:5432/postgres -c \
|
|
"SELECT datname FROM pg_database WHERE datname LIKE 'windmill_test_%';"
|
|
|
|
# Check for orphaned backend processes
|
|
ps aux | grep "target/debug/windmill" | grep -v grep
|
|
```
|