mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-23 00:00:33 +00:00
Ruben Fiszel
82e5f6de48
* Add windmill-operator crate for Kubernetes CRD-based instance config Introduces a new `windmill-operator` crate that enables declarative management of Windmill instance configuration via a Kubernetes `WindmillInstance` CRD. The operator watches CRD resources and performs full declarative sync of global_settings and worker configs to the database, supporting GitOps workflows for instance-level configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add tests for windmill-operator CRD and db_sync - 9 unit tests for CRD serialization, deserialization, metadata, and status field behavior - 15 integration tests for db_sync using #[sqlx::test] with full declarative sync coverage: upsert, delete, protected keys, idempotency, worker config prefix handling, and end-to-end sync Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Replace untyped BTreeMap CRD fields with typed structs for schema validation GlobalSettings, SmtpSettings, IndexerSettings, and WorkerGroupConfig now have explicit typed fields with serde(flatten) catch-all for forward compatibility. The generated CRD YAML includes a full OpenAPI v3 schema that Kubernetes validates on kubectl apply. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Type opaque serde_json::Value CRD fields with real structs Replace most remaining serde_json::Value fields in WindmillInstance CRD with properly typed structs derived from the codebase: - oauths: BTreeMap<String, OAuthClient> - otel: OtelSettings - otel_tracing_proxy: OtelTracingProxySettings with ScriptLang enum - critical_error_channels: Vec<CriticalErrorChannel> (untagged enum) - critical_alerts_on_db_oversize: DbOversizeAlert - ducklake_settings: DucklakeSettings with nested catalog/storage types - custom_instance_pg_databases: CustomInstancePgDatabases - autoscaling (worker config): AutoscalingConfig with integration struct - custom_tags, default_tags_workspaces: Vec<String> - default_tags_per_workspace: bool Still opaque (serde_json::Value): object_store_cache_config (kube-core can't generate schemas for internally-tagged enums), secret_backend (EE-private), slack, teams (no clear struct definitions). Regenerated CRD YAML with full OpenAPI schema (352→703 lines). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Unify instance config types and add bulk GET/PUT API Move all typed settings (GlobalSettings, WorkerGroupConfig, etc.) from windmill-operator/crd.rs into windmill-common/instance_config.rs so both the API server and operator share a single source of truth. Add diff/apply logic (Merge mode for UI, Replace mode for operator) and InstanceConfig::from_db(). Add GET/PUT /settings/instance_config endpoints so the frontend loads all settings in 1 call instead of 42, and saves with a single bulk PUT. The backend handles the diff internally, running pre-write hooks for changed keys. Refactor windmill-operator/db_sync.rs to use the shared diff+apply functions and slim crd.rs down to the CRD wrapper with re-exports. Includes 32 unit tests and 30 integration tests covering serialization, diff logic, DB roundtrips, protected settings, and edge cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add Form/YAML toggle to instance settings UI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: show Form/YAML toggle regardless of hideTabs prop Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: replace toggle button group with simple YAML toggle Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: mask sensitive fields in YAML view with show/hide toggle Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: hide internal settings and mask sensitive fields in YAML view Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: hide jwt_secret and min_keep_alive_version from API and config exports Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * feat: add secretKeyRef support for sensitive fields in operator CRD Allow sensitive fields (license_key, hub_api_secret, scim_token, smtp_password, OAuthClient.secret, custom PG user_pwd) to reference Kubernetes Secrets via the standard secretKeyRef pattern instead of inlining values as plaintext YAML. The reconciler resolves all refs by reading K8s Secrets before syncing to the database. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * all * fix: merge main and update dev environment docs Resolve merge conflicts from origin/main, fix duplicate UV_INDEX_STRATEGY_SETTING import, and add Playwright MCP testing instructions to CLAUDE.md. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * fix: init tracing for CLI subcommands and deduplicate setting side-effects Initialize tracing subscriber before early-return CLI paths (sync-config, operator) so tracing calls are not silently dropped. Refactor set_global_setting_internal to call run_setting_pre_write_hook instead of duplicating the side-effect logic. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add `wmill instance get-config` CLI command Dumps the current instance config (global settings + worker configs) as YAML. Supports --output-file to write to a file instead of stdout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
69 lines
4.2 KiB
Markdown
69 lines
4.2 KiB
Markdown
# Windmill Development Guide
|
|
|
|
## Overview
|
|
|
|
Windmill is an open-source developer platform for building internal tools, workflows, API integrations, background jobs, workflows, and user interfaces. See @windmill-overview.mdc for full platform details.
|
|
|
|
## New Feature Implementation Guidelines
|
|
|
|
When implementing new features in Windmill, follow these best practices:
|
|
|
|
- **Clean Code First**: Write clean, readable, and maintainable code. Prioritize clarity over cleverness.
|
|
- **Avoid Duplication at All Costs**: Before writing new code, thoroughly search for existing implementations that can be reused or extended.
|
|
- **Adapt Existing Code**: Refactor and generalize existing code when necessary to avoid logic duplication. Extract common patterns into reusable utilities.
|
|
- **Follow Established Patterns**: Study existing code patterns in the codebase and maintain consistency with established conventions.
|
|
- **Single Responsibility**: Each function, component, and module should have a single, well-defined responsibility.
|
|
- **Incremental Implementation**: Break large features into smaller, reviewable chunks that can be implemented and tested incrementally.
|
|
|
|
## Language-Specific Guides
|
|
|
|
- Backend (Rust): see `backend/CLAUDE.md` and the `rust-backend` skill: `.claude/skills/rust-backend/SKILL.md`
|
|
- Frontend (Svelte 5): see `frontend/CLAUDE.md` and the `svelte-frontend` skill: `.claude/skills/svelte-frontend/SKILL.md`
|
|
|
|
## Dev Environment
|
|
|
|
- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
|
|
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/`
|
|
- The `REMOTE` env var configures the Vite proxy target. Without it, API calls proxy to `https://app.windmill.dev` instead of the local backend.
|
|
- The dev server starts on port 3000 (or 3001+ if 3000 is in use).
|
|
- **Default login**: `admin@windmill.dev` / `changeme`
|
|
- **Instance settings**: navigate to `/#superadmin-settings` (opens the drawer overlay)
|
|
|
|
## UI Testing with Playwright MCP
|
|
|
|
When testing the frontend with the Playwright MCP tools:
|
|
|
|
1. **Start servers**: Launch backend (`cargo run`) and frontend (`REMOTE=http://localhost:8000 npm run dev`) as background tasks
|
|
2. **Wait for readiness**: Backend takes ~60s to compile; check output for `health check completed`. Frontend starts in ~5s.
|
|
3. **Login flow**: Navigate to `/user/login`, click "Log in without third-party", fill email/password, submit
|
|
4. **Instance settings drawer**: Navigate to `/#superadmin-settings` to open the drawer directly
|
|
5. **Toggle components**: The YAML toggle uses a custom `<Toggle>` component where the checkbox is visually hidden (`sr-only`). Click the wrapper `<label>` element (the parent container with `cursor=pointer`), not the checkbox ref directly.
|
|
6. **Console errors to ignore**: `critical_alerts` 404s are expected on CE builds (EE-only endpoint). VSCode worker 404s are dev-mode artifacts.
|
|
|
|
## Code Validation (MUST DO)
|
|
|
|
After making code changes, you MUST run the appropriate checks and fix all errors before considering the work done:
|
|
|
|
- **Backend**: Run `cargo check` from the `backend/` directory. Only enable the feature flags needed for the code you changed — check `backend/Cargo.toml` `[features]` section to identify which flags gate the crates/modules you modified. For example: `cargo check --features enterprise,parquet` if you only touched enterprise and parquet code.
|
|
- **Frontend**: Run `npm run check` from the `frontend/` directory.
|
|
|
|
## Querying the Database
|
|
|
|
`backend/summarized_schema.txt` provides a compact overview of all tables, columns, types, ENUMs, and foreign keys. Use it to quickly understand the data model and relationships. Note: this file is a simplified summary — it omits indexes, constraints details, and other metadata.
|
|
|
|
For exact table definitions (indexes, constraints, column defaults, etc.), query the database directly:
|
|
|
|
```bash
|
|
psql postgres://postgres:changeme@localhost:5432/windmill
|
|
```
|
|
|
|
Useful psql commands:
|
|
- `\d <table_name>` — full table definition with indexes and constraints
|
|
- `\di <table_name>*` — list indexes for a table
|
|
- `\d+ <table_name>` — extended table info including storage and descriptions
|
|
|
|
This is also helpful for:
|
|
- Inspecting database state during development
|
|
- Testing queries before implementing them in Rust
|
|
- Debugging data-related issues
|