mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-11 08:07:15 +00:00
merge: sync with main
Resolved conflict in FlowModuleSchemaMap.svelte, taking main's refactored move logic with affectedGroups/commit pattern. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Guilhem Lemouel
Claude Opus 4.6
@@ -0,0 +1,59 @@
|
||||
---
|
||||
name: commit
|
||||
description: Create a git commit with conventional commit format. MUST use anytime you want to commit changes.
|
||||
---
|
||||
|
||||
# Git Commit Skill
|
||||
|
||||
Create a focused, single-line commit following conventional commit conventions.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Analyze changes**: Run `git status` and `git diff` to understand what was modified
|
||||
2. **Stage only modified files**: Add files individually by name. NEVER use `git add -A` or `git add .`
|
||||
3. **Write commit message**: Follow the conventional commit format as a single line
|
||||
|
||||
## Conventional Commit Format
|
||||
|
||||
```
|
||||
<type>: <description>
|
||||
```
|
||||
|
||||
### Types
|
||||
- `feat`: New feature or capability
|
||||
- `fix`: Bug fix
|
||||
- `refactor`: Code change that neither fixes a bug nor adds a feature
|
||||
- `docs`: Documentation only changes
|
||||
- `style`: Formatting, missing semicolons, etc (no code change)
|
||||
- `test`: Adding or correcting tests
|
||||
- `chore`: Maintenance tasks, dependency updates, etc
|
||||
- `perf`: Performance improvement
|
||||
|
||||
### Rules
|
||||
- Message MUST be a single line (no multi-line messages)
|
||||
- Description should be lowercase, imperative mood ("add" not "added")
|
||||
- No period at the end
|
||||
- Keep under 72 characters total
|
||||
|
||||
### Examples
|
||||
```
|
||||
feat: add token usage tracking for AI providers
|
||||
fix: resolve null pointer in job executor
|
||||
refactor: extract common validation logic
|
||||
docs: update API endpoint documentation
|
||||
chore: upgrade sqlx to 0.7
|
||||
```
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Run `git status` to see all changes
|
||||
2. Run `git diff` to understand the changes in detail
|
||||
3. Run `git log --oneline -5` to see recent commit style
|
||||
4. Stage ONLY the modified/relevant files: `git add <file1> <file2> ...`
|
||||
5. Create the commit with conventional format:
|
||||
```bash
|
||||
git commit -m "<type>: <description>
|
||||
|
||||
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
|
||||
```
|
||||
6. Run `git status` to verify the commit succeeded
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
name: local-review
|
||||
description: Code review a pull request for bugs and CLAUDE.md compliance. MUST use when asked to review code.
|
||||
---
|
||||
|
||||
# Local Code Review Skill
|
||||
|
||||
Review a pull request for real bugs and CLAUDE.md compliance violations. This review targets HIGH SIGNAL issues only.
|
||||
|
||||
## Review Philosophy
|
||||
|
||||
- **Only flag issues you are certain about.** If you are not sure an issue is real, do not flag it. False positives erode trust and waste reviewer time.
|
||||
- Think like a senior engineer doing a final review — flag things that would cause incidents, not things that are merely imperfect.
|
||||
|
||||
## What to Flag
|
||||
|
||||
- Code that won't compile or parse (syntax errors, type errors, missing imports)
|
||||
- Code that will definitely produce wrong results regardless of inputs
|
||||
- Clear, unambiguous CLAUDE.md violations (quote the exact rule being violated)
|
||||
- Security issues in introduced code (injection, auth bypass, data exposure)
|
||||
- Incorrect logic that will fail in production
|
||||
|
||||
## What NOT to Flag
|
||||
|
||||
- Code style or quality concerns
|
||||
- Potential issues that depend on specific inputs or runtime state
|
||||
- Subjective suggestions or improvements
|
||||
- Pre-existing issues not introduced by this PR
|
||||
- Pedantic nitpicks a senior engineer wouldn't flag
|
||||
- Issues a linter or type checker will catch
|
||||
- General quality concerns unless explicitly prohibited in CLAUDE.md
|
||||
- Issues silenced via lint ignore comments
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. **Determine the PR scope**:
|
||||
- If an argument is provided, use it as the PR number or branch
|
||||
- Otherwise, detect from the current branch vs main
|
||||
- Run `gh pr view` if a PR exists, or use `git diff main...HEAD`
|
||||
|
||||
2. **Find relevant CLAUDE.md files**:
|
||||
- Read the root `CLAUDE.md`
|
||||
- Check for CLAUDE.md files in directories containing changed files
|
||||
|
||||
3. **Get the diff and metadata**:
|
||||
- `gh pr diff` or `git diff main...HEAD` for the full diff
|
||||
- `gh pr view` or `git log main..HEAD --oneline` for context
|
||||
|
||||
4. **Read changed files** where the diff alone is insufficient to understand context
|
||||
|
||||
5. **Review for**:
|
||||
- CLAUDE.md compliance — check each rule against the changed code
|
||||
- Bugs and logic errors — will this code work correctly?
|
||||
- Security issues — injection, auth, data exposure in new code
|
||||
|
||||
6. **Self-validate each finding**: Before reporting, ask yourself:
|
||||
- "Is this definitely a real issue, not a false positive?"
|
||||
- "Would a senior engineer flag this in review?"
|
||||
- If the answer to either is no, discard the finding
|
||||
|
||||
7. **Output findings** to the terminal (default) or post as PR comments (with `--comment` flag)
|
||||
|
||||
## Output Format
|
||||
|
||||
```
|
||||
## Code review
|
||||
|
||||
Found N issues:
|
||||
|
||||
1. <description> (<reason: CLAUDE.md adherence | bug | security>)
|
||||
<file_path:line_number>
|
||||
|
||||
2. <description> (<reason>)
|
||||
<file_path:line_number>
|
||||
```
|
||||
|
||||
If no issues are found:
|
||||
|
||||
```
|
||||
## Code review
|
||||
|
||||
No issues found. Checked for bugs and CLAUDE.md compliance.
|
||||
```
|
||||
|
||||
## Posting Comments (--comment flag)
|
||||
|
||||
If the user passes `--comment`, post findings as inline PR comments using:
|
||||
|
||||
```bash
|
||||
gh pr review --comment --body "<summary>"
|
||||
```
|
||||
|
||||
Or for inline comments on specific lines:
|
||||
|
||||
```bash
|
||||
gh api repos/{owner}/{repo}/pulls/{pr}/reviews -f body="<summary>" -f event="COMMENT" -f comments="[...]"
|
||||
```
|
||||
@@ -0,0 +1,777 @@
|
||||
# Skill: Adding Native Trigger Services
|
||||
|
||||
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The native trigger system consists of:
|
||||
|
||||
1. **Database Layer** - PostgreSQL tables and enum types
|
||||
2. **Backend Rust Implementation** - Core trait, handlers, and service modules in the `windmill-native-triggers` crate
|
||||
3. **Frontend Svelte Components** - Configuration forms and UI components
|
||||
|
||||
### Key Files
|
||||
|
||||
| Component | Path |
|
||||
|-----------|------|
|
||||
| Core module with `External` trait | `backend/windmill-native-triggers/src/lib.rs` |
|
||||
| Generic CRUD handlers | `backend/windmill-native-triggers/src/handler.rs` |
|
||||
| Background sync logic | `backend/windmill-native-triggers/src/sync.rs` |
|
||||
| OAuth/workspace integration | `backend/windmill-native-triggers/src/workspace_integrations.rs` |
|
||||
| Re-export shim (windmill-api) | `backend/windmill-api/src/native_triggers/mod.rs` |
|
||||
| TriggerKind enum | `backend/windmill-common/src/triggers.rs` |
|
||||
| JobTriggerKind enum | `backend/windmill-common/src/jobs.rs` |
|
||||
| Frontend service registry | `frontend/src/lib/components/triggers/native/utils.ts` |
|
||||
| Frontend trigger utilities | `frontend/src/lib/components/triggers/utils.ts` |
|
||||
| Trigger badges (icons + counts) | `frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte` |
|
||||
| Workspace integrations UI | `frontend/src/lib/components/workspaceSettings/WorkspaceIntegrations.svelte` |
|
||||
| OAuth config form component | `frontend/src/lib/components/workspaceSettings/OAuthClientConfig.svelte` |
|
||||
| OpenAPI spec | `backend/windmill-api/openapi.yaml` |
|
||||
| Reference: Nextcloud module | `backend/windmill-native-triggers/src/nextcloud/` |
|
||||
| Reference: Google module | `backend/windmill-native-triggers/src/google/` |
|
||||
|
||||
### Crate Structure
|
||||
|
||||
The native trigger code lives in the `windmill-native-triggers` crate (`backend/windmill-native-triggers/`). The `windmill-api` crate re-exports everything via a shim:
|
||||
|
||||
```rust
|
||||
// backend/windmill-api/src/native_triggers/mod.rs
|
||||
pub use windmill_native_triggers::*;
|
||||
```
|
||||
|
||||
All new service modules go in `backend/windmill-native-triggers/src/`.
|
||||
|
||||
---
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### The `External` Trait
|
||||
|
||||
Every native trigger service implements the `External` trait defined in `lib.rs`:
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
pub trait External: Send + Sync + 'static {
|
||||
// Associated types:
|
||||
type ServiceConfig: Debug + DeserializeOwned + Serialize + Send + Sync;
|
||||
type TriggerData: Debug + Serialize + Send + Sync;
|
||||
type OAuthData: DeserializeOwned + Serialize + Clone + Send + Sync;
|
||||
type CreateResponse: DeserializeOwned + Send + Sync;
|
||||
|
||||
// Constants:
|
||||
const SUPPORT_WEBHOOK: bool;
|
||||
const SERVICE_NAME: ServiceName;
|
||||
const DISPLAY_NAME: &'static str;
|
||||
const TOKEN_ENDPOINT: &'static str;
|
||||
const REFRESH_ENDPOINT: &'static str;
|
||||
const AUTH_ENDPOINT: &'static str;
|
||||
|
||||
// Required methods:
|
||||
async fn create(&self, w_id, oauth_data, webhook_token, data, db, tx) -> Result<Self::CreateResponse>;
|
||||
async fn update(&self, w_id, oauth_data, external_id, webhook_token, data, db, tx) -> Result<serde_json::Value>;
|
||||
async fn get(&self, w_id, oauth_data, external_id, db, tx) -> Result<Self::TriggerData>;
|
||||
async fn delete(&self, w_id, oauth_data, external_id, db, tx) -> Result<()>;
|
||||
async fn exists(&self, w_id, oauth_data, external_id, db, tx) -> Result<bool>;
|
||||
async fn maintain_triggers(&self, db, workspace_id, triggers, oauth_data, synced, errors);
|
||||
fn external_id_and_metadata_from_response(&self, resp) -> (String, Option<serde_json::Value>);
|
||||
|
||||
// Methods with defaults:
|
||||
async fn prepare_webhook(&self, db, w_id, headers, body, script_path, is_flow) -> Result<PushArgsOwned>;
|
||||
fn service_config_from_create_response(&self, data, resp) -> Option<serde_json::Value>;
|
||||
fn additional_routes(&self) -> axum::Router;
|
||||
async fn http_client_request<T, B>(&self, url, method, workspace_id, tx, db, headers, body) -> Result<T>;
|
||||
}
|
||||
```
|
||||
|
||||
Key design points:
|
||||
- **`update()` returns `serde_json::Value`** - the resolved service_config to store. Each service is responsible for building the final config.
|
||||
- **`maintain_triggers()`** - periodic background maintenance. Each service implements its own strategy (Nextcloud: reconcile with external state; Google: renew expiring channels).
|
||||
- **No `list_all()` in the trait** - services that need it (Nextcloud) implement it privately; services that don't (Google) use different maintenance strategies.
|
||||
- **No `get_external_id_from_trigger_data()` or `extract_service_config_from_trigger_data()`** - removed in favor of the `maintain_triggers` pattern.
|
||||
|
||||
### Create Lifecycle: Two Paths
|
||||
|
||||
The `create_native_trigger` handler in `handler.rs` supports two creation flows, controlled by `service_config_from_create_response()`:
|
||||
|
||||
**Path A: Short (Google pattern)** - `service_config_from_create_response()` returns `Some(config)`:
|
||||
1. `create()` registers on external service
|
||||
2. `external_id_and_metadata_from_response()` extracts the ID
|
||||
3. `service_config_from_create_response()` builds the config directly from input data + response metadata
|
||||
4. Stores trigger in DB -- done, no extra round-trip
|
||||
|
||||
Use this when the external_id is known before the create call (e.g., Google generates the channel_id as a UUID upfront and includes it in the webhook URL).
|
||||
|
||||
**Path B: Long (Nextcloud pattern)** - `service_config_from_create_response()` returns `None` (default):
|
||||
1. `create()` registers on external service (webhook URL has no external_id yet)
|
||||
2. `external_id_and_metadata_from_response()` extracts the ID
|
||||
3. `update()` is called to fix the webhook URL with the now-known external_id
|
||||
4. `update()` returns the resolved service_config
|
||||
5. Stores trigger in DB
|
||||
|
||||
Use this when the external_id is assigned by the remote service and the webhook URL needs to be corrected after creation.
|
||||
|
||||
### OAuth Token Storage (Three-Table Pattern)
|
||||
|
||||
OAuth tokens are stored across three tables, NOT in `workspace_integrations.oauth_data` directly:
|
||||
|
||||
| Table | What's Stored |
|
||||
|-------|---------------|
|
||||
| `workspace_integrations` | `oauth_data` JSON with `base_url`, `client_id`, `client_secret`, `instance_shared` flag; `resource_path` pointing to the variable |
|
||||
| `variable` | Encrypted `access_token` (at the path stored in `resource_path`), linked to `account` via `account` column |
|
||||
| `account` | `refresh_token`, keyed by `workspace_id` + `client` (service name) + `is_workspace_integration = true` |
|
||||
|
||||
The `decrypt_oauth_data()` function in `lib.rs` assembles these into a unified struct:
|
||||
```rust
|
||||
pub struct OAuthConfig {
|
||||
pub base_url: String,
|
||||
pub access_token: String, // decrypted from variable
|
||||
pub refresh_token: Option<String>, // from account table
|
||||
pub client_id: String, // from oauth_data or instance settings
|
||||
pub client_secret: String, // from oauth_data or instance settings
|
||||
}
|
||||
```
|
||||
|
||||
Instance-level sharing: when `oauth_data.instance_shared == true`, `client_id` and `client_secret` are read from global settings instead of workspace_integrations.
|
||||
|
||||
### URL Resolution
|
||||
|
||||
The `resolve_endpoint()` helper handles both absolute and relative OAuth URLs:
|
||||
|
||||
```rust
|
||||
pub fn resolve_endpoint(base_url: &str, endpoint: &str) -> String {
|
||||
if endpoint.starts_with("http://") || endpoint.starts_with("https://") {
|
||||
endpoint.to_string() // Google: absolute URLs
|
||||
} else {
|
||||
format!("{}{}", base_url, endpoint) // Nextcloud: relative paths
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### ServiceName Methods
|
||||
|
||||
`ServiceName` is the central registry enum. Each variant must implement these match arms:
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `as_str()` | Lowercase identifier (e.g., `"google"`) |
|
||||
| `as_trigger_kind()` | Maps to `TriggerKind` enum |
|
||||
| `as_job_trigger_kind()` | Maps to `JobTriggerKind` enum |
|
||||
| `token_endpoint()` | OAuth token endpoint (relative or absolute) |
|
||||
| `auth_endpoint()` | OAuth authorization endpoint |
|
||||
| `oauth_scopes()` | Space-separated OAuth scopes |
|
||||
| `resource_type()` | Resource type for token storage (e.g., `"gworkspace"`) |
|
||||
| `extra_auth_params()` | Extra OAuth params (e.g., Google needs `access_type=offline`, `prompt=consent`) |
|
||||
| `integration_service()` | Maps to the workspace integration service (usually `*self`) |
|
||||
| `TryFrom<String>` | Parse from string |
|
||||
| `Display` | Delegates to `as_str()` |
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Implementation Guide
|
||||
|
||||
### Step 1: Database Migration
|
||||
|
||||
Create a new migration file: `backend/migrations/YYYYMMDDHHMMSS_newservice_trigger.up.sql`
|
||||
|
||||
```sql
|
||||
-- Add the service to the native_trigger_service enum
|
||||
ALTER TYPE native_trigger_service ADD VALUE IF NOT EXISTS 'newservice';
|
||||
|
||||
-- Add to TRIGGER_KIND enum (used for trigger tracking)
|
||||
ALTER TYPE TRIGGER_KIND ADD VALUE IF NOT EXISTS 'newservice';
|
||||
|
||||
-- Add to job_trigger_kind enum (used for job tracking)
|
||||
ALTER TYPE job_trigger_kind ADD VALUE IF NOT EXISTS 'newservice';
|
||||
```
|
||||
|
||||
Also create the corresponding down migration.
|
||||
|
||||
### Step 2: Update windmill-common Enums
|
||||
|
||||
#### `backend/windmill-common/src/triggers.rs`
|
||||
|
||||
Add variant to `TriggerKind` enum, and update `to_key()` and `fmt()` implementations.
|
||||
|
||||
#### `backend/windmill-common/src/jobs.rs`
|
||||
|
||||
Add variant to `JobTriggerKind` enum and update the `Display` implementation.
|
||||
|
||||
### Step 3: Backend Service Module
|
||||
|
||||
Create a new directory: `backend/windmill-native-triggers/src/newservice/`
|
||||
|
||||
#### `mod.rs` - Type Definitions
|
||||
|
||||
```rust
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub mod external;
|
||||
// pub mod routes; // Only if you need additional service-specific routes
|
||||
|
||||
/// OAuth data deserialized from the three-table pattern.
|
||||
/// The actual structure is built by decrypt_oauth_data() from variable + account + workspace_integrations.
|
||||
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||
pub struct NewServiceOAuthData {
|
||||
pub base_url: String, // from workspace_integrations.oauth_data
|
||||
pub access_token: String, // decrypted from variable table
|
||||
pub refresh_token: Option<String>, // from account table
|
||||
// Note: client_id and client_secret are in OAuthConfig, not here
|
||||
// unless the service needs them at runtime for API calls
|
||||
}
|
||||
|
||||
/// Configuration provided by user when creating/updating a trigger.
|
||||
/// Stored as JSON in native_trigger.service_config.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct NewServiceConfig {
|
||||
// Service-specific configuration fields
|
||||
pub folder_path: String,
|
||||
pub file_filter: Option<String>,
|
||||
}
|
||||
|
||||
/// Data retrieved from the external service about a trigger.
|
||||
/// Returned by the get() method and shown in the UI.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct NewServiceTriggerData {
|
||||
pub folder_path: String,
|
||||
pub file_filter: Option<String>,
|
||||
// Fields that shouldn't affect service_config comparison should use #[serde(skip_serializing)]
|
||||
}
|
||||
|
||||
/// Response from external service when creating a trigger/webhook.
|
||||
#[derive(Debug, Deserialize)]
|
||||
pub struct CreateTriggerResponse {
|
||||
pub id: String,
|
||||
}
|
||||
|
||||
/// Handler struct (stateless, used for routing)
|
||||
#[derive(Copy, Clone)]
|
||||
pub struct NewService;
|
||||
```
|
||||
|
||||
#### `external.rs` - External Trait Implementation
|
||||
|
||||
```rust
|
||||
use async_trait::async_trait;
|
||||
use reqwest::Method;
|
||||
use sqlx::PgConnection;
|
||||
use std::collections::HashMap;
|
||||
use windmill_common::{
|
||||
error::{Error, Result},
|
||||
BASE_URL, DB,
|
||||
};
|
||||
|
||||
use crate::{
|
||||
generate_webhook_service_url, External, NativeTrigger, NativeTriggerData, ServiceName,
|
||||
sync::{SyncError, TriggerSyncInfo},
|
||||
};
|
||||
use super::{NewService, NewServiceConfig, NewServiceOAuthData, NewServiceTriggerData, CreateTriggerResponse};
|
||||
|
||||
#[async_trait]
|
||||
impl External for NewService {
|
||||
type ServiceConfig = NewServiceConfig;
|
||||
type TriggerData = NewServiceTriggerData;
|
||||
type OAuthData = NewServiceOAuthData;
|
||||
type CreateResponse = CreateTriggerResponse;
|
||||
|
||||
const SERVICE_NAME: ServiceName = ServiceName::NewService;
|
||||
const DISPLAY_NAME: &'static str = "New Service";
|
||||
const SUPPORT_WEBHOOK: bool = true;
|
||||
const TOKEN_ENDPOINT: &'static str = "/oauth/token";
|
||||
const REFRESH_ENDPOINT: &'static str = "/oauth/token";
|
||||
const AUTH_ENDPOINT: &'static str = "/oauth/authorize";
|
||||
|
||||
async fn create(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
webhook_token: &str,
|
||||
data: &NativeTriggerData<Self::ServiceConfig>,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<Self::CreateResponse> {
|
||||
let base_url = &*BASE_URL.read().await;
|
||||
|
||||
// external_id is None during create (we get it from the response)
|
||||
let webhook_url = generate_webhook_service_url(
|
||||
base_url, w_id, &data.script_path, data.is_flow,
|
||||
None, Self::SERVICE_NAME, webhook_token,
|
||||
);
|
||||
|
||||
let url = format!("{}/api/webhooks/create", oauth_data.base_url);
|
||||
let payload = serde_json::json!({
|
||||
"callback_url": webhook_url,
|
||||
"folder_path": data.service_config.folder_path,
|
||||
});
|
||||
|
||||
let response: CreateTriggerResponse = self
|
||||
.http_client_request(&url, Method::POST, w_id, tx, db, None, Some(&payload))
|
||||
.await?;
|
||||
|
||||
Ok(response)
|
||||
}
|
||||
|
||||
/// Update returns the resolved service_config as JSON.
|
||||
/// For services using the update+get pattern, call self.get() and serialize.
|
||||
async fn update(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
webhook_token: &str,
|
||||
data: &NativeTriggerData<Self::ServiceConfig>,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<serde_json::Value> {
|
||||
let base_url = &*BASE_URL.read().await;
|
||||
|
||||
let webhook_url = generate_webhook_service_url(
|
||||
base_url, w_id, &data.script_path, data.is_flow,
|
||||
Some(external_id), Self::SERVICE_NAME, webhook_token,
|
||||
);
|
||||
|
||||
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
|
||||
let payload = serde_json::json!({
|
||||
"callback_url": webhook_url,
|
||||
"folder_path": data.service_config.folder_path,
|
||||
});
|
||||
|
||||
let _: serde_json::Value = self
|
||||
.http_client_request(&url, Method::PUT, w_id, tx, db, None, Some(&payload))
|
||||
.await?;
|
||||
|
||||
// Fetch back the updated state to get the resolved config
|
||||
let trigger_data = self.get(w_id, oauth_data, external_id, db, tx).await?;
|
||||
serde_json::to_value(&trigger_data)
|
||||
.map_err(|e| Error::InternalErr(format!("Failed to serialize trigger data: {}", e)))
|
||||
}
|
||||
|
||||
async fn get(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<Self::TriggerData> {
|
||||
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
|
||||
self.http_client_request::<_, ()>(&url, Method::GET, w_id, tx, db, None, None).await
|
||||
}
|
||||
|
||||
async fn delete(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<()> {
|
||||
let url = format!("{}/api/webhooks/{}", oauth_data.base_url, external_id);
|
||||
let _: serde_json::Value = self
|
||||
.http_client_request::<_, ()>(&url, Method::DELETE, w_id, tx, db, None, None)
|
||||
.await
|
||||
.or_else(|e| match &e {
|
||||
Error::InternalErr(msg) if msg.contains("404") => Ok(serde_json::Value::Null),
|
||||
_ => Err(e),
|
||||
})?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn exists(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &Self::OAuthData,
|
||||
external_id: &str,
|
||||
db: &DB,
|
||||
tx: &mut PgConnection,
|
||||
) -> Result<bool> {
|
||||
match self.get(w_id, oauth_data, external_id, db, tx).await {
|
||||
Ok(_) => Ok(true),
|
||||
Err(Error::NotFound(_)) => Ok(false),
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
/// Background maintenance. Choose the right pattern for your service:
|
||||
/// - For services with queryable external state: use reconcile_with_external_state()
|
||||
/// - For channel-based services with expiration: implement renewal logic
|
||||
async fn maintain_triggers(
|
||||
&self,
|
||||
db: &DB,
|
||||
workspace_id: &str,
|
||||
triggers: &[NativeTrigger],
|
||||
oauth_data: &Self::OAuthData,
|
||||
synced: &mut Vec<TriggerSyncInfo>,
|
||||
errors: &mut Vec<SyncError>,
|
||||
) {
|
||||
// Option A: Reconcile with external state (Nextcloud pattern)
|
||||
// Fetch all triggers from external service and compare with DB
|
||||
let external_triggers = match self.list_all(workspace_id, oauth_data, db).await {
|
||||
Ok(triggers) => triggers,
|
||||
Err(e) => {
|
||||
errors.push(SyncError {
|
||||
resource_path: format!("workspace:{}", workspace_id),
|
||||
error_message: format!("Failed to list triggers: {}", e),
|
||||
error_type: "api_error".to_string(),
|
||||
});
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
// Convert to (external_id, config_json) pairs
|
||||
let external_pairs: Vec<(String, serde_json::Value)> = external_triggers
|
||||
.into_iter()
|
||||
.map(|t| (t.id.clone(), serde_json::to_value(&t).unwrap_or_default()))
|
||||
.collect();
|
||||
|
||||
crate::sync::reconcile_with_external_state(
|
||||
db, workspace_id, Self::SERVICE_NAME, triggers, &external_pairs, synced, errors,
|
||||
).await;
|
||||
}
|
||||
|
||||
fn external_id_and_metadata_from_response(
|
||||
&self,
|
||||
resp: &Self::CreateResponse,
|
||||
) -> (String, Option<serde_json::Value>) {
|
||||
(resp.id.clone(), None)
|
||||
}
|
||||
|
||||
// service_config_from_create_response: NOT overridden (returns None).
|
||||
// This means the handler uses the update+get pattern after create.
|
||||
// Override and return Some(...) to skip the update+get cycle (Google pattern).
|
||||
}
|
||||
|
||||
impl NewService {
|
||||
/// Private helper to list all triggers from the external service.
|
||||
async fn list_all(
|
||||
&self,
|
||||
w_id: &str,
|
||||
oauth_data: &<Self as External>::OAuthData,
|
||||
db: &DB,
|
||||
) -> Result<Vec<<Self as External>::TriggerData>> {
|
||||
// Implementation depends on the external service's API
|
||||
todo!()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Update lib.rs Registry
|
||||
|
||||
In `backend/windmill-native-triggers/src/lib.rs`:
|
||||
|
||||
```rust
|
||||
// Service modules - add new services here:
|
||||
#[cfg(feature = "native_trigger")]
|
||||
pub mod newservice; // <-- Add this
|
||||
|
||||
// ServiceName enum - add variant:
|
||||
pub enum ServiceName {
|
||||
Nextcloud,
|
||||
Google,
|
||||
NewService, // <-- Add this
|
||||
}
|
||||
|
||||
// Then add match arms in ALL ServiceName methods:
|
||||
// as_str(), as_trigger_kind(), as_job_trigger_kind(), token_endpoint(),
|
||||
// auth_endpoint(), oauth_scopes(), resource_type(), extra_auth_params(),
|
||||
// integration_service(), TryFrom<String>, Display
|
||||
```
|
||||
|
||||
### Step 5: Update handler.rs Routes
|
||||
|
||||
In `backend/windmill-native-triggers/src/handler.rs`:
|
||||
|
||||
```rust
|
||||
pub fn generate_native_trigger_routers() -> Router {
|
||||
// ...
|
||||
#[cfg(feature = "native_trigger")]
|
||||
{
|
||||
use crate::newservice::NewService;
|
||||
return router
|
||||
.nest("/nextcloud", service_routes(NextCloud))
|
||||
.nest("/google", service_routes(Google))
|
||||
.nest("/newservice", service_routes(NewService)); // <-- Add this
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 6: Update sync.rs
|
||||
|
||||
In `backend/windmill-native-triggers/src/sync.rs`:
|
||||
|
||||
```rust
|
||||
pub async fn sync_all_triggers(db: &DB) -> Result<BackgroundSyncResult> {
|
||||
// ...
|
||||
#[cfg(feature = "native_trigger")]
|
||||
{
|
||||
use crate::newservice::NewService;
|
||||
|
||||
// ... existing service syncs ...
|
||||
|
||||
// New service sync
|
||||
let (service_name, result) = sync_service_triggers(db, NewService).await;
|
||||
total_synced += result.synced_triggers.len();
|
||||
total_errors += result.errors.len();
|
||||
service_results.insert(service_name, result);
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 7: Frontend Service Registry
|
||||
|
||||
In `frontend/src/lib/components/triggers/native/utils.ts`:
|
||||
|
||||
Add to `NATIVE_TRIGGER_SERVICES`, `getTriggerIconName()`, and `getServiceIcon()`.
|
||||
|
||||
### Step 8: Frontend Trigger Form Component
|
||||
|
||||
Create: `frontend/src/lib/components/triggers/native/services/newservice/NewServiceTriggerForm.svelte`
|
||||
|
||||
### Step 9: Frontend Icon Component
|
||||
|
||||
Create: `frontend/src/lib/components/icons/NewServiceIcon.svelte`
|
||||
|
||||
### Step 10: Update NativeTriggerEditor
|
||||
|
||||
Check `frontend/src/lib/components/triggers/native/NativeTriggerEditor.svelte` to ensure it dynamically loads form components based on service name.
|
||||
|
||||
### Step 11: Workspace Integration UI
|
||||
|
||||
Add your service to the `supportedServices` map in `frontend/src/lib/components/workspaceSettings/WorkspaceIntegrations.svelte`:
|
||||
|
||||
```typescript
|
||||
const supportedServices: Record<string, ServiceConfig> = {
|
||||
// ... existing services ...
|
||||
newservice: {
|
||||
name: 'newservice',
|
||||
displayName: 'New Service',
|
||||
description: 'Connect to New Service for triggers',
|
||||
icon: NewServiceIcon,
|
||||
docsUrl: 'https://www.windmill.dev/docs/integrations/newservice',
|
||||
requiresBaseUrl: false, // false for cloud services, true for self-hosted
|
||||
setupInstructions: [
|
||||
'Step 1: Create an OAuth app on the service',
|
||||
'Step 2: Configure the redirect URI shown below',
|
||||
'Step 3: Enter the client credentials below'
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 12: Update `frontend/src/lib/components/triggers/utils.ts`
|
||||
|
||||
Update ALL of these maps/functions:
|
||||
1. `triggerIconMap` - import and add icon
|
||||
2. `triggerDisplayNamesMap` - add display name
|
||||
3. `triggerTypeOrder` in `sortTriggers()` - add type
|
||||
4. `getLightConfig()` - add case for your service
|
||||
5. `getTriggerLabel()` - add case for your service
|
||||
6. `jobTriggerKinds` - add to array
|
||||
7. `countPropertyMap` - add count property
|
||||
8. `triggerSaveFunctions` - add save function
|
||||
|
||||
### Step 13: Update TriggersBadge Component
|
||||
|
||||
In `frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte`:
|
||||
|
||||
1. Import the icon
|
||||
2. Add to `baseConfig` with `countKey` (the dynamic `availableNativeServices` loop does NOT set `countKey`)
|
||||
3. Add to the `allTypes` array
|
||||
|
||||
### Step 14: Update TriggersWrapper.svelte
|
||||
|
||||
In `frontend/src/lib/components/triggers/TriggersWrapper.svelte`:
|
||||
|
||||
Add a `{:else if selectedTrigger.type === 'yourservice'}` case that renders `<NativeTriggersPanel service="yourservice" ...>` with the same props pattern as the existing native trigger cases (e.g., `nextcloud`).
|
||||
|
||||
### Step 15: Update AddTriggersButton.svelte
|
||||
|
||||
In `frontend/src/lib/components/triggers/AddTriggersButton.svelte`:
|
||||
|
||||
1. Add `yourserviceAvailable` state variable
|
||||
2. Add `setYourserviceState()` async function using `isServiceAvailable('yourservice', $workspaceStore!)`
|
||||
3. Call it at module level
|
||||
4. Add a dropdown entry to `addTriggerItems` with `hidden: !yourserviceAvailable`
|
||||
|
||||
### Step 16: Update TriggersEditor.svelte Delete Handling
|
||||
|
||||
In `frontend/src/lib/components/triggers/TriggersEditor.svelte`:
|
||||
|
||||
Add your service to the `nativeTriggerServices` map in `deleteDeployedTrigger()`. Native triggers use `NativeTriggerService.deleteNativeTrigger({ workspace, serviceName, externalId })` instead of the standard `path`-based delete.
|
||||
|
||||
### Step 17: Update OpenAPI Spec and Regenerate Types
|
||||
|
||||
Add to `JobTriggerKind` enum in `backend/windmill-api/openapi.yaml`, then:
|
||||
|
||||
```bash
|
||||
cd frontend && npm run generate-backend-client
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Special Patterns
|
||||
|
||||
### Unified Service with `trigger_type` (Google Pattern)
|
||||
|
||||
When a single service handles multiple trigger types (e.g., Google Drive + Calendar share OAuth and API patterns), use a single `ServiceName` variant with a discriminator field:
|
||||
|
||||
```rust
|
||||
pub enum GoogleTriggerType { Drive, Calendar }
|
||||
|
||||
pub struct GoogleServiceConfig {
|
||||
pub trigger_type: GoogleTriggerType,
|
||||
// Drive-specific fields (only used when trigger_type = Drive)
|
||||
pub resource_id: Option<String>,
|
||||
pub resource_name: Option<String>,
|
||||
// Calendar-specific fields (only used when trigger_type = Calendar)
|
||||
pub calendar_id: Option<String>,
|
||||
pub calendar_name: Option<String>,
|
||||
// Metadata set after creation
|
||||
pub google_resource_id: Option<String>,
|
||||
pub expiration: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Branch in trait methods based on `trigger_type`. Frontend uses a `ToggleButtonGroup` to switch between types. This keeps the codebase simpler (one service, one OAuth flow, one set of routes).
|
||||
|
||||
See `backend/windmill-native-triggers/src/google/` for the reference implementation.
|
||||
|
||||
### Skipping update+get After Create (Google Pattern)
|
||||
|
||||
Override `service_config_from_create_response()` to return `Some(config)` when the external_id is known before the create call:
|
||||
|
||||
```rust
|
||||
fn service_config_from_create_response(
|
||||
&self,
|
||||
data: &NativeTriggerData<Self::ServiceConfig>,
|
||||
resp: &Self::CreateResponse,
|
||||
) -> Option<serde_json::Value> {
|
||||
// Clone input config, add metadata from response
|
||||
let mut config = data.service_config.clone();
|
||||
config.google_resource_id = Some(resp.resource_id.clone());
|
||||
config.expiration = Some(resp.expiration.clone());
|
||||
Some(serde_json::to_value(&config).unwrap())
|
||||
}
|
||||
```
|
||||
|
||||
### Services with Absolute OAuth Endpoints (Google)
|
||||
|
||||
Unlike self-hosted services where OAuth endpoints are relative paths appended to `base_url`, services like Google have absolute URLs:
|
||||
|
||||
```rust
|
||||
// Nextcloud: relative paths
|
||||
ServiceName::Nextcloud => "/apps/oauth2/api/v1/token",
|
||||
// Google: absolute URLs
|
||||
ServiceName::Google => "https://oauth2.googleapis.com/token",
|
||||
```
|
||||
|
||||
The `resolve_endpoint()` function handles both. For services with absolute endpoints:
|
||||
- `base_url` can be empty
|
||||
- `requiresBaseUrl: false` in the frontend workspace integration config
|
||||
- Add `extra_auth_params()` if needed (Google requires `access_type=offline` and `prompt=consent`)
|
||||
|
||||
### Channel-Based Push Notifications with Renewal (Google Pattern)
|
||||
|
||||
For services using expiring watch channels instead of persistent webhooks:
|
||||
|
||||
1. Store expiration in `service_config` (as part of `ServiceConfig`)
|
||||
2. In `maintain_triggers()`, implement renewal logic instead of using `reconcile_with_external_state()`:
|
||||
```rust
|
||||
async fn maintain_triggers(&self, db, workspace_id, triggers, oauth_data, synced, errors) {
|
||||
for trigger in triggers {
|
||||
if should_renew_channel(trigger) {
|
||||
self.renew_channel(db, trigger, oauth_data).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
3. Renewal: best-effort stop old channel, create new one with same external_id, update service_config with new expiration
|
||||
4. Google example: Drive channels expire in 24h (renew when <1h left), Calendar channels expire in 7 days (renew when <1 day left)
|
||||
|
||||
### reconcile_with_external_state (Nextcloud Pattern)
|
||||
|
||||
The reusable function in `sync.rs` compares external triggers with DB state:
|
||||
- Triggers missing externally: sets error "Trigger no longer exists on external service"
|
||||
- Triggers present externally: clears errors, updates service_config if it differs
|
||||
|
||||
Usage in `maintain_triggers()`:
|
||||
```rust
|
||||
let external_pairs: Vec<(String, serde_json::Value)> = /* fetch from external */;
|
||||
crate::sync::reconcile_with_external_state(
|
||||
db, workspace_id, Self::SERVICE_NAME, triggers, &external_pairs, synced, errors,
|
||||
).await;
|
||||
```
|
||||
|
||||
### Webhook Payload Processing
|
||||
|
||||
Override `prepare_webhook()` to parse service-specific payloads into script/flow args:
|
||||
|
||||
```rust
|
||||
async fn prepare_webhook(&self, db, w_id, headers, body, script_path, is_flow) -> Result<PushArgsOwned> {
|
||||
let mut args = HashMap::new();
|
||||
args.insert("event_type".to_string(), Box::new(headers.get("x-event-type").cloned()) as _);
|
||||
args.insert("payload".to_string(), Box::new(serde_json::from_str::<serde_json::Value>(&body)?) as _);
|
||||
Ok(PushArgsOwned { extra: None, args })
|
||||
}
|
||||
```
|
||||
|
||||
Then register in `prepare_native_trigger_args()` in `lib.rs`:
|
||||
```rust
|
||||
pub async fn prepare_native_trigger_args(service_name, db, w_id, headers, body) -> Result<Option<PushArgsOwned>> {
|
||||
match service_name {
|
||||
ServiceName::Google => { /* ... */ Ok(Some(args)) }
|
||||
ServiceName::NewService => { /* ... */ Ok(Some(args)) }
|
||||
ServiceName::Nextcloud => Ok(None), // Uses default body parsing
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Instance-Level OAuth Credentials
|
||||
|
||||
When `workspace_integrations.oauth_data.instance_shared == true`, `decrypt_oauth_data()` reads `client_id` and `client_secret` from instance-level global settings instead of workspace-level. This allows admins to share OAuth app credentials across workspaces.
|
||||
|
||||
The frontend handles this via the `generate_instance_connect_url` endpoint in `workspace_integrations.rs`.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Database migration runs successfully
|
||||
- [ ] `cargo check -p windmill-native-triggers --features native_trigger` passes
|
||||
- [ ] `npx svelte-check --threshold error` passes (in frontend/)
|
||||
- [ ] Service appears in workspace integrations list
|
||||
- [ ] OAuth flow completes successfully
|
||||
- [ ] Can create a new trigger
|
||||
- [ ] Can view trigger details
|
||||
- [ ] Can update trigger configuration
|
||||
- [ ] Can delete trigger
|
||||
- [ ] Webhook receives and processes payloads
|
||||
- [ ] Background sync works correctly (reconciliation or channel renewal)
|
||||
- [ ] Error handling works (expired tokens, service unavailable)
|
||||
|
||||
---
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
### Nextcloud (Self-Hosted, Update+Get Pattern)
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `nextcloud/mod.rs` | Types: NextCloudOAuthData, NextcloudServiceConfig, NextCloudTriggerData |
|
||||
| `nextcloud/external.rs` | External trait: uses update+get pattern, reconcile_with_external_state for sync |
|
||||
| `nextcloud/routes.rs` | Additional route: `GET /events` |
|
||||
|
||||
Key patterns: relative OAuth endpoints, base_url required, list_all + reconcile for sync, update returns JSON from get().
|
||||
|
||||
### Google (Cloud, Unified Service, Short Create)
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `google/mod.rs` | Types: GoogleServiceConfig with trigger_type discriminator, GoogleTriggerType enum |
|
||||
| `google/external.rs` | External trait: overrides service_config_from_create_response, channel renewal for sync |
|
||||
| `google/routes.rs` | Additional routes: `GET /calendars`, `GET /drive/files`, `GET /drive/shared_drives` |
|
||||
|
||||
Key patterns: absolute OAuth endpoints, empty base_url, trigger_type for Drive/Calendar, expiring watch channels with renewal, service_config_from_create_response skips update+get, get() reconstructs data from stored service_config (no external "get channel" API).
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
name: pr
|
||||
description: Open a draft pull request on GitHub. MUST use when you want to create/open a PR.
|
||||
---
|
||||
|
||||
# Pull Request Skill
|
||||
|
||||
Create a draft pull request with a clear title and explicit description of changes.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Analyze branch changes**: Understand all commits since diverging from main
|
||||
2. **Push to remote**: Ensure all commits are pushed
|
||||
3. **Create draft PR**: Always open as draft for review before merging
|
||||
|
||||
## PR Title Format
|
||||
|
||||
Follow conventional commit format for the PR title:
|
||||
```
|
||||
<type>: <description>
|
||||
```
|
||||
|
||||
### Types
|
||||
- `feat`: New feature or capability
|
||||
- `fix`: Bug fix
|
||||
- `refactor`: Code restructuring
|
||||
- `docs`: Documentation changes
|
||||
- `chore`: Maintenance tasks
|
||||
- `perf`: Performance improvements
|
||||
|
||||
### Title Rules
|
||||
- Keep under 70 characters
|
||||
- Use lowercase, imperative mood
|
||||
- No period at the end
|
||||
- If `*_ee.rs` files were modified, prefix with `[ee]`: `[ee] <type>: <description>`
|
||||
|
||||
## PR Body Format
|
||||
|
||||
The body MUST be explicit about what changed. Structure:
|
||||
|
||||
```markdown
|
||||
## Summary
|
||||
<Clear description of what this PR does and why>
|
||||
|
||||
## Changes
|
||||
- <Specific change 1>
|
||||
- <Specific change 2>
|
||||
- <Specific change 3>
|
||||
|
||||
## Test plan
|
||||
- [ ] <How to verify change 1>
|
||||
- [ ] <How to verify change 2>
|
||||
|
||||
---
|
||||
Generated with [Claude Code](https://claude.com/claude-code)
|
||||
```
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Run `git status` to check for uncommitted changes
|
||||
2. Run `git log main..HEAD --oneline` to see all commits in this branch
|
||||
3. Run `git diff main...HEAD` to see the full diff against main
|
||||
4. Check if remote branch exists and is up to date:
|
||||
```bash
|
||||
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
|
||||
```
|
||||
5. Push to remote if needed: `git push -u origin HEAD`
|
||||
6. Create draft PR using gh CLI:
|
||||
```bash
|
||||
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
|
||||
## Summary
|
||||
<description>
|
||||
|
||||
## Changes
|
||||
- <change 1>
|
||||
- <change 2>
|
||||
|
||||
## Test plan
|
||||
- [ ] <test 1>
|
||||
- [ ] <test 2>
|
||||
|
||||
---
|
||||
Generated with [Claude Code](https://claude.com/claude-code)
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
7. Return the PR URL to the user
|
||||
|
||||
## EE Companion PR (when `*_ee.rs` files were modified)
|
||||
|
||||
The `*_ee.rs` files in the windmill repo are **symlinks** to `windmill-ee-private` — changes won't appear in `git diff` of the windmill repo. Instead, check the EE repo for uncommitted or unpushed changes.
|
||||
|
||||
Follow the full EE PR workflow in `docs/enterprise.md`. The key PR-specific details:
|
||||
|
||||
1. Find the EE repo/worktree: see "Finding the EE Repo" in `docs/enterprise.md`
|
||||
2. Check for changes: `git -C <ee-path> status --short`
|
||||
- If there are no changes in the EE repo, skip this entire section
|
||||
3. Follow steps 1–5 from the "EE PR Workflow" in `docs/enterprise.md`
|
||||
4. Create the companion PR (title does NOT get the `[ee]` prefix):
|
||||
```bash
|
||||
gh pr create --draft --repo windmill-labs/windmill-ee-private --title "<type>: <description>" --body "$(cat <<'EOF'
|
||||
Companion PR for windmill-labs/windmill#<PR_NUMBER>
|
||||
|
||||
---
|
||||
Generated with [Claude Code](https://claude.com/claude-code)
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
5. Commit `ee-repo-ref.txt` and push the updated windmill branch
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: refine
|
||||
description: End-of-session reflection. Reviews friction encountered during the session and proposes updates to docs/ to capture lessons learned.
|
||||
---
|
||||
|
||||
# Refine Skill
|
||||
|
||||
Reflect on the current session and update documentation with lessons learned.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Identify friction**: Review what happened in this session:
|
||||
- Run `git diff main...HEAD --stat` to see what files were touched
|
||||
- Think about: what was slow, what failed, what required multiple attempts, what information was missing or hard to find
|
||||
|
||||
2. **Read current docs**: Read the docs that were relevant to this session:
|
||||
- `docs/validation.md`
|
||||
- `docs/enterprise.md`
|
||||
- `docs/autonomous-mode.md`
|
||||
- Any skills that were invoked
|
||||
|
||||
3. **Propose updates**: For each piece of friction, decide if it warrants a doc update:
|
||||
- **Missing knowledge**: Information you had to discover that should be documented
|
||||
- **Wrong guidance**: Instructions that led you astray
|
||||
- **Missing validation rule**: A check that should be in the validation matrix
|
||||
- **New pattern**: A codebase pattern worth capturing for next time
|
||||
|
||||
4. **Apply updates**: Edit the relevant `docs/` files. Keep changes minimal and specific — add only what would have saved time this session.
|
||||
|
||||
5. **Report**: Summarize what was added/changed and why.
|
||||
|
||||
## Rules
|
||||
|
||||
- Only add knowledge confirmed by this session — no speculative additions
|
||||
- Keep docs concise — add a line or two, not a paragraph
|
||||
- If a whole new doc is needed, create it in `docs/` and add a pointer in `CLAUDE.md`
|
||||
- Don't update skills unless a coding pattern was genuinely wrong
|
||||
- Don't add things Claude already knows — only Windmill-specific knowledge
|
||||
@@ -0,0 +1,107 @@
|
||||
---
|
||||
name: rust-backend
|
||||
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
|
||||
---
|
||||
|
||||
# Windmill Rust Patterns
|
||||
|
||||
Apply these Windmill-specific patterns when writing Rust code in `backend/`.
|
||||
|
||||
## Error Handling
|
||||
|
||||
Use `Error` from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>`:
|
||||
|
||||
```rust
|
||||
use windmill_common::error::{Error, Result};
|
||||
|
||||
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
|
||||
sqlx::query_as!(Job, "SELECT id, workspace_id FROM v2_job WHERE id = $1", id)
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
|
||||
}
|
||||
```
|
||||
|
||||
Never panic in library code. Reserve `.unwrap()` for compile-time guarantees.
|
||||
|
||||
## SQLx Patterns
|
||||
|
||||
**Never use `SELECT *`** — always list columns explicitly. Critical for backwards compatibility when workers lag behind API version:
|
||||
|
||||
```rust
|
||||
// Correct
|
||||
sqlx::query_as!(Job, "SELECT id, workspace_id, path FROM v2_job WHERE id = $1", id)
|
||||
|
||||
// Wrong — breaks when columns are added
|
||||
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", id)
|
||||
```
|
||||
|
||||
Use batch operations to avoid N+1:
|
||||
|
||||
```rust
|
||||
// Preferred — single query with IN clause
|
||||
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
|
||||
```
|
||||
|
||||
Use transactions for multi-step operations. Parameterize all queries.
|
||||
|
||||
## JSON Handling
|
||||
|
||||
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when storing/passing JSON without inspection:
|
||||
|
||||
```rust
|
||||
pub struct Job {
|
||||
pub args: Option<Box<serde_json::value::RawValue>>,
|
||||
}
|
||||
```
|
||||
|
||||
Only use `serde_json::Value` when you need to inspect or modify the JSON.
|
||||
|
||||
## Serde Optimizations
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub struct Job {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub parent_job: Option<Uuid>,
|
||||
#[serde(skip_serializing_if = "Vec::is_empty")]
|
||||
pub tags: Vec<String>,
|
||||
#[serde(default)]
|
||||
pub priority: i32,
|
||||
}
|
||||
```
|
||||
|
||||
## Async & Concurrency
|
||||
|
||||
Never block the async runtime. Use `spawn_blocking` for CPU-intensive work:
|
||||
|
||||
```rust
|
||||
let result = tokio::task::spawn_blocking(move || expensive_computation(&data)).await?;
|
||||
```
|
||||
|
||||
**Mutex selection**: Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) for data protection. Only use `tokio::sync::Mutex` when holding locks across `.await` points.
|
||||
|
||||
Use `tokio::sync::mpsc` (bounded) for channels. Avoid `std::thread::sleep` in async contexts.
|
||||
|
||||
## Module Structure & Visibility
|
||||
|
||||
- Use `pub(crate)` instead of `pub` when possible
|
||||
- Place new code in the appropriate crate based on functionality
|
||||
- API endpoints go in `windmill-api/src/` organized by domain
|
||||
- Shared functionality goes in `windmill-common/src/`
|
||||
|
||||
## Code Navigation
|
||||
|
||||
Always use rust-analyzer LSP for go-to-definition, find-references, and type info. Do not guess at module paths.
|
||||
|
||||
## Axum Handlers
|
||||
|
||||
Destructure extractors directly in function signatures:
|
||||
|
||||
```rust
|
||||
async fn process_job(
|
||||
Extension(db): Extension<DB>,
|
||||
Path((workspace, job_id)): Path<(String, Uuid)>,
|
||||
Query(pagination): Query<Pagination>,
|
||||
) -> Result<Json<Job>> { ... }
|
||||
```
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
name: svelte-frontend
|
||||
description: Svelte coding guidelines for the Windmill frontend. MUST use when writing or modifying code in the frontend directory.
|
||||
---
|
||||
|
||||
# Windmill Svelte Patterns
|
||||
|
||||
Apply these Windmill-specific patterns when writing Svelte code in `frontend/`. For general Svelte 5 syntax (runes, snippets, event handling), use the Svelte MCP server.
|
||||
|
||||
## Windmill UI Components (MUST use)
|
||||
|
||||
Always use Windmill's design-system components. Never use raw HTML elements.
|
||||
|
||||
### Buttons — `<Button>`
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { Button } from '$lib/components/common'
|
||||
import { ChevronLeft } from 'lucide-svelte'
|
||||
</script>
|
||||
|
||||
<Button variant="default" onclick={handleClick}>Label</Button>
|
||||
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prev} />
|
||||
```
|
||||
|
||||
Props: `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`, `unifiedSize?: 'sm' | 'md' | 'lg'`, `startIcon?: { icon: SvelteComponent }`, `iconOnly?: boolean`, `disabled?: boolean`
|
||||
|
||||
### Text inputs — `<TextInput>`
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { TextInput } from '$lib/components/common'
|
||||
</script>
|
||||
|
||||
<TextInput bind:value={val} placeholder="Enter value" />
|
||||
```
|
||||
|
||||
Props: `value?: string | number` (bindable), `placeholder?: string`, `disabled?: boolean`, `error?: string | boolean`, `size?: 'sm' | 'md' | 'lg'`
|
||||
|
||||
### Selects — `<Select>`
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import Select from '$lib/components/select/Select.svelte'
|
||||
</script>
|
||||
|
||||
<Select items={[{ label: 'Jan', value: 1 }]} bind:value={selected} />
|
||||
```
|
||||
|
||||
Props: `items?: Array<{ label?: string; value: any }>`, `value` (bindable), `placeholder?: string`, `clearable?: boolean`, `size?: 'sm' | 'md' | 'lg'`
|
||||
|
||||
### Icons — `lucide-svelte`
|
||||
|
||||
Never write inline SVGs. Import from `lucide-svelte`:
|
||||
|
||||
```svelte
|
||||
<script>
|
||||
import { ChevronLeft, X } from 'lucide-svelte'
|
||||
</script>
|
||||
<ChevronLeft size={16} />
|
||||
```
|
||||
|
||||
## Form Components
|
||||
|
||||
Form components (TextInput, Toggle, Select, etc.) should use the unified size system when placed together.
|
||||
|
||||
## Styling
|
||||
|
||||
- Use Tailwind CSS for all styling — no custom CSS
|
||||
- Use Windmill's theming classes for colors/surfaces (see `frontend/brand-guidelines.md`)
|
||||
- Read component props JSDoc before using them
|
||||
|
||||
## Svelte MCP Server
|
||||
|
||||
Use the Svelte MCP tools when working on Svelte code:
|
||||
|
||||
1. **list-sections**: Call first to discover available docs
|
||||
2. **get-documentation**: Fetch relevant sections based on use_cases
|
||||
3. **svelte-autofixer**: MUST use on all Svelte code before finalizing — keep calling until no issues
|
||||
4. **playground-link**: Only after user confirms and code was NOT written to project files
|
||||
@@ -13,10 +13,10 @@ on:
|
||||
jobs:
|
||||
check-membership:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/ai')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/ai')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/ai')) ||
|
||||
(github.event_name == 'issues' && contains(github.event.issue.body, '/ai'))
|
||||
(github.event_name == 'issue_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && startsWith(github.event.comment.body, '/ai') && !startsWith(github.event.comment.body, '/ai-fast')) ||
|
||||
(github.event_name == 'pull_request_review' && startsWith(github.event.review.body, '/ai') && !startsWith(github.event.review.body, '/ai-fast')) ||
|
||||
(github.event_name == 'issues' && startsWith(github.event.issue.body, '/ai') && !startsWith(github.event.issue.body, '/ai-fast'))
|
||||
uses: ./.github/workflows/check-org-membership.yml
|
||||
secrets:
|
||||
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
|
||||
+6
-3
@@ -55,11 +55,13 @@ profiles:
|
||||
- id: backend
|
||||
kind: command
|
||||
split: right
|
||||
command: ROOT="$(git rev-parse --show-toplevel)"; cd "$ROOT/backend" && cargo watch -x "run ${CARGO_FEATURES:+--features $CARGO_FEATURES}"
|
||||
workingDir: backend
|
||||
command: PORT=${BACKEND_PORT:-8000} cargo watch -x "run ${CARGO_FEATURES:+--features $CARGO_FEATURES}"
|
||||
- id: frontend
|
||||
kind: command
|
||||
split: bottom
|
||||
command: ROOT="$(git rev-parse --show-toplevel)"; cd "$ROOT/frontend" && npm run generate-backend-client && npm run dev -- --host 0.0.0.0
|
||||
workingDir: frontend
|
||||
command: npm run generate-backend-client && REMOTE=${REMOTE:-http://localhost:${BACKEND_PORT:-8000}} npm run dev -- --port ${FRONTEND_PORT:-3000} --host 0.0.0.0
|
||||
|
||||
frontendOnly:
|
||||
runtime: host
|
||||
@@ -82,7 +84,8 @@ profiles:
|
||||
- id: frontend
|
||||
kind: command
|
||||
split: right
|
||||
command: ROOT="$(git rev-parse --show-toplevel)"; cd "$ROOT/frontend" && npm run generate-backend-client && npm run dev -- --host 0.0.0.0
|
||||
workingDir: frontend
|
||||
command: npm run generate-backend-client && npm run dev -- --port ${FRONTEND_PORT:-3000} --host 0.0.0.0
|
||||
|
||||
agentOnly:
|
||||
runtime: host
|
||||
|
||||
@@ -1,5 +1,45 @@
|
||||
# Changelog
|
||||
|
||||
## [1.664.0](https://github.com/windmill-labs/windmill/compare/v1.663.0...v1.664.0) (2026-03-24)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* add instance-level AI settings ([#8453](https://github.com/windmill-labs/windmill/issues/8453)) ([db5e036](https://github.com/windmill-labs/windmill/commit/db5e03610da325288d53afdbca94b9cbfc7ceace))
|
||||
* add selfApproval option to WAC + inline approval buttons ([#8440](https://github.com/windmill-labs/windmill/issues/8440)) ([d578e40](https://github.com/windmill-labs/windmill/commit/d578e40101a838d3dffda14157cf72ee4d5a93c0))
|
||||
* flow group nodes with collapsible groups ([#8075](https://github.com/windmill-labs/windmill/issues/8075)) ([81eb446](https://github.com/windmill-labs/windmill/commit/81eb446eee359f44374b81320690e5345fd08c15))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* add GIT_SSL_CAINFO to tracing proxy env vars ([#8502](https://github.com/windmill-labs/windmill/issues/8502)) ([bdfd5d5](https://github.com/windmill-labs/windmill/commit/bdfd5d57261a4bb760fc57ad41ee56aff9b9c0af))
|
||||
* create parent dirs and accept 'python' alias in script bootstrap ([#8497](https://github.com/windmill-labs/windmill/issues/8497)) ([7f27d99](https://github.com/windmill-labs/windmill/commit/7f27d996accb3c3b471d1c50df397867d89c738a))
|
||||
|
||||
## [1.663.0](https://github.com/windmill-labs/windmill/compare/v1.662.0...v1.663.0) (2026-03-24)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* add summary field for native triggers ([#8476](https://github.com/windmill-labs/windmill/issues/8476)) ([5089a45](https://github.com/windmill-labs/windmill/commit/5089a458819abbc6f241bc354bebb91520bd1a52))
|
||||
* add typed request body to OpenAPI spec generation ([#8481](https://github.com/windmill-labs/windmill/issues/8481)) ([37ebaf4](https://github.com/windmill-labs/windmill/commit/37ebaf4d0ac342703498733f97778a552f979f6a))
|
||||
* **cli:** better stale scripts detection [#3](https://github.com/windmill-labs/windmill/issues/3) ([#8480](https://github.com/windmill-labs/windmill/issues/8480)) ([9643006](https://github.com/windmill-labs/windmill/commit/9643006f1e90b991b334bb58caf62301bc26d09d))
|
||||
* Debounce node ([#8324](https://github.com/windmill-labs/windmill/issues/8324)) ([5d1c54d](https://github.com/windmill-labs/windmill/commit/5d1c54d9b33d6ff6f2c98481a2740d1e7629cdfa))
|
||||
* surface permissioned_as selector in trigger editor UI ([#8475](https://github.com/windmill-labs/windmill/issues/8475)) ([f035b53](https://github.com/windmill-labs/windmill/commit/f035b538bbd786445526339f88be8f33a3628105))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* clean up stale dependency map entries for renamed scripts ([#8492](https://github.com/windmill-labs/windmill/issues/8492)) ([47c0c36](https://github.com/windmill-labs/windmill/commit/47c0c363f4fc1d9af7efd07ea172e32989ce50d2))
|
||||
* **cli:** add Svelte 5 event delegation guidance and safe push to raw-app skill ([#8466](https://github.com/windmill-labs/windmill/issues/8466)) ([911df95](https://github.com/windmill-labs/windmill/commit/911df958e78d2dab9823dfa7d7e5c9824fc2d565))
|
||||
* Fix worker panic when job_isolation changed to unshare at runtime ([#8490](https://github.com/windmill-labs/windmill/issues/8490)) ([cbe47c0](https://github.com/windmill-labs/windmill/commit/cbe47c0b6c22f79452d020777e481ee26970f25b))
|
||||
* improve SQS retries ([3c8d351](https://github.com/windmill-labs/windmill/commit/3c8d351c9722a089133871019d27cf3bc3cdc159))
|
||||
* Move database manager SQL queries to backend ([#8306](https://github.com/windmill-labs/windmill/issues/8306)) ([aa30fd2](https://github.com/windmill-labs/windmill/commit/aa30fd252dcf40233d191c43a6293fb9feabf010))
|
||||
* prevent SQL injection in job query parameters ([#8494](https://github.com/windmill-labs/windmill/issues/8494)) ([54f5a19](https://github.com/windmill-labs/windmill/commit/54f5a19377e9df712e18f85f896e21b1776981ed))
|
||||
* respect NO_COLOR env variable for stdout log output ([#8483](https://github.com/windmill-labs/windmill/issues/8483)) ([f329ee7](https://github.com/windmill-labs/windmill/commit/f329ee7aaefbae0ad344743c40825440a936bd30))
|
||||
* show effective isolation level on workers page ([#8491](https://github.com/windmill-labs/windmill/issues/8491)) ([37886ed](https://github.com/windmill-labs/windmill/commit/37886edda1443293806a9b1b810196b72e076b12))
|
||||
* skip debounce arg accumulation when batch table is empty (CE) ([#8485](https://github.com/windmill-labs/windmill/issues/8485)) ([010753c](https://github.com/windmill-labs/windmill/commit/010753c73ac85237af50acadf9c08567b1bc993c))
|
||||
* stop_after_if with empty error_message prevents flow from stopping ([#8464](https://github.com/windmill-labs/windmill/issues/8464)) ([1503bf9](https://github.com/windmill-labs/windmill/commit/1503bf948e3340b8a6933d71885f8f2cb8dc1867))
|
||||
|
||||
## [1.662.0](https://github.com/windmill-labs/windmill/compare/v1.661.0...v1.662.0) (2026-03-20)
|
||||
|
||||
|
||||
|
||||
+4
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n INSERT INTO native_trigger (\n external_id,\n workspace_id,\n service_name,\n script_path,\n is_flow,\n webhook_token_hash,\n service_config\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7\n )\n ON CONFLICT (external_id, workspace_id, service_name)\n DO UPDATE SET script_path = $4, is_flow = $5, webhook_token_hash = $6, service_config = $7, error = NULL, updated_at = NOW()\n ",
|
||||
"query": "\n INSERT INTO native_trigger (\n external_id,\n workspace_id,\n service_name,\n script_path,\n is_flow,\n webhook_token_hash,\n service_config,\n summary\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8\n )\n ON CONFLICT (external_id, workspace_id, service_name)\n DO UPDATE SET script_path = $4, is_flow = $5, webhook_token_hash = $6, service_config = $7, summary = $8, error = NULL, updated_at = NOW()\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -21,10 +21,11 @@
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Jsonb"
|
||||
"Jsonb",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "6f9386dfcb4c201525722aee3caa25bf2f3a35d90f7354c7d3aef8a3538a03a7"
|
||||
"hash": "1048d1c95270ce1f36c02bce31a2bc8a88935c613bd213b7156299811377db8e"
|
||||
}
|
||||
+8
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT email, login_type::text, verified, super_admin, devops, name, company, username, NULL::bool as operator_only, first_time_user, role_source FROM password ORDER BY super_admin DESC, devops DESC, email LIMIT $1 OFFSET $2",
|
||||
"query": "SELECT email, login_type::text, verified, super_admin, devops, name, company, username, NULL::bool as operator_only, first_time_user, role_source, disabled FROM password ORDER BY super_admin DESC, devops DESC, email LIMIT $1 OFFSET $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -57,6 +57,11 @@
|
||||
"ordinal": 10,
|
||||
"name": "role_source",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 11,
|
||||
"name": "disabled",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -76,8 +81,9 @@
|
||||
true,
|
||||
null,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "05027983ffdb11824190543754d0be922e1463d2046753cf80377369a90013ab"
|
||||
"hash": "115a9cb44d0a41952c08dc36e0331410d32a8d672cfa4929e9e3763c51daa1bc"
|
||||
}
|
||||
+9
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n external_id,\n workspace_id,\n service_name AS \"service_name!: ServiceName\",\n script_path,\n is_flow,\n webhook_token_hash,\n service_config,\n error,\n created_at,\n updated_at\n FROM\n native_trigger\n WHERE\n workspace_id = $1\n AND service_name = $2\n AND external_id = $3\n ",
|
||||
"query": "\n SELECT\n external_id,\n workspace_id,\n service_name AS \"service_name!: ServiceName\",\n script_path,\n is_flow,\n webhook_token_hash,\n service_config,\n error,\n created_at,\n updated_at,\n summary\n FROM\n native_trigger\n WHERE\n workspace_id = $1\n AND service_name = $2\n AND external_id = $3\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -62,6 +62,11 @@
|
||||
"ordinal": 9,
|
||||
"name": "updated_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 10,
|
||||
"name": "summary",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -91,8 +96,9 @@
|
||||
true,
|
||||
true,
|
||||
false,
|
||||
false
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "bac545933a627a62b7845d8aab80702443285e4d1d11e5a0f4cd2a3d4add51bb"
|
||||
"hash": "15014ce696cf2af4f719a537a4e3ca5b322cc130a35a91f8b8854f5ebdf25ad2"
|
||||
}
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "DELETE FROM token WHERE email = $1",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "192ddae8c3c82a8f099a4944483024d9826a328bf0416c22daf06fff5ced08f6"
|
||||
}
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT ws.default_app AS default_app_path, av.raw_app AS \"default_app_raw: Option<bool>\"\n FROM workspace_settings ws\n LEFT JOIN app ON app.path = ws.default_app AND app.workspace_id = ws.workspace_id\n LEFT JOIN app_version av ON av.id = app.versions[array_upper(app.versions, 1)]\n WHERE ws.workspace_id = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "default_app_path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "default_app_raw: Option<bool>",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "1bc77ad29b9c68b1d339b85158bc3592deb61d1111d1430ddd2879b72e6424ef"
|
||||
}
|
||||
+21
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n route_path,\n http_method AS \"http_method: _\",\n request_type AS \"request_type: _\",\n workspaced_route,\n summary,\n description,\n authentication_method AS \"authentication_method: _\",\n authentication_resource_path\n FROM\n http_trigger\n WHERE\n path ~ ANY($1) AND\n route_path ~ ANY($2) AND\n workspace_id = $3\n ",
|
||||
"query": "\n SELECT\n route_path,\n http_method AS \"http_method: _\",\n request_type AS \"request_type: _\",\n workspaced_route,\n summary,\n description,\n authentication_method AS \"authentication_method: _\",\n authentication_resource_path,\n script_path,\n is_flow,\n wrap_body\n FROM\n http_trigger\n WHERE\n path ~ ANY($1) AND\n route_path ~ ANY($2) AND\n workspace_id = $3\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -80,6 +80,21 @@
|
||||
"ordinal": 7,
|
||||
"name": "authentication_resource_path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 8,
|
||||
"name": "script_path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 9,
|
||||
"name": "is_flow",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 10,
|
||||
"name": "wrap_body",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -97,8 +112,11 @@
|
||||
true,
|
||||
true,
|
||||
false,
|
||||
true
|
||||
true,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "9360d00990822f153ff09c7905ae3180f07d02f38ac12d07a5664d93f160e7ee"
|
||||
"hash": "1cb21a66ffc89ebe53fd8f58690eec4cf11cb4b7816738202b474e8d3ffef427"
|
||||
}
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT email, disabled FROM password WHERE email = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "email",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "disabled",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "23b9c862d050b00aaa332527b62ef901cd3c417b9f3af03f35009213143bd443"
|
||||
}
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "DELETE FROM raw_script_temp WHERE created_at < NOW() - INTERVAL '1 week'",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "25ac66a1022c41267df199a95f532b0f778c25fbe0f7a7f9734c1f7e536ed6ce"
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO raw_script_temp (workspace_id, hash, content, created_at)\n VALUES ($1, $2, $3, NOW())\n ON CONFLICT (workspace_id, hash) DO UPDATE SET created_at = NOW()",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Bpchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "2d523cd0d5b7107b15846b885fa40af492d4c8a8871cef972980150f319fe6ff"
|
||||
}
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n f.schema AS \"schema: serde_json::Value\",\n fv.value->>'preprocessor_module' IS NOT NULL AS \"has_preprocessor: bool\"\n FROM flow f\n LEFT JOIN flow_version fv ON fv.id = f.versions[array_length(f.versions, 1)]\n AND fv.workspace_id = f.workspace_id\n WHERE f.path = $1 AND f.workspace_id = $2 AND NOT f.archived",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "schema: serde_json::Value",
|
||||
"type_info": "Json"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "has_preprocessor: bool",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "372ec62fad93831b00d44f6c52a148b6bfa6008e49b2a80bec06c76d9432ec77"
|
||||
}
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n WITH step_index AS (\n SELECT idx::text AS idx\n FROM v2_job_status,\n jsonb_array_elements(flow_status->'modules') WITH ORDINALITY arr(elem, idx)\n WHERE id = $1\n AND elem->>'id' = $5\n LIMIT 1\n ), completed AS (\n INSERT INTO v2_job_completed\n (workspace_id, id, started_at, duration_ms, result,\n flow_status, workflow_as_code_status, status, worker)\n SELECT\n q.workspace_id, q.id, q.started_at,\n (EXTRACT('epoch' FROM now()) - EXTRACT('epoch' FROM COALESCE(q.started_at, now()))) * 1000,\n $3::text::jsonb,\n CASE WHEN si.idx IS NOT NULL\n THEN jsonb_set(\n s.flow_status,\n ARRAY['modules', (si.idx::int - 1)::text],\n $6::jsonb\n )\n ELSE s.flow_status\n END,\n s.workflow_as_code_status,\n 'skipped'::job_status,\n q.worker\n FROM v2_job_queue q\n LEFT JOIN v2_job_status s ON s.id = q.id\n LEFT JOIN step_index si ON true\n WHERE q.id = $1\n ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, result = EXCLUDED.result\n RETURNING 1 AS x\n ), _deleted AS (\n DELETE FROM v2_job_queue WHERE id = $1\n ), _logged AS (\n INSERT INTO job_logs (logs, job_id, workspace_id)\n VALUES ($4, $1, $2)\n ON CONFLICT (job_id) DO UPDATE SET logs = concat(job_logs.logs, EXCLUDED.logs)\n )\n SELECT x FROM completed\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "x",
|
||||
"type_info": "Int4"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Varchar",
|
||||
"Text",
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "4461fe84370e7f07b4423ea5e71b913dd6ee9c655f9ec7087cc0fec9b9c3099a"
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, parent_job)\n VALUES ($1, 'noop', 'deno', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Varchar",
|
||||
"Uuid"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "4538bea4159677d8e653159d7d01649cae08e6ffef668a7cbac49b312bf30766"
|
||||
}
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n INSERT INTO debounce_key (job_id, key)\n VALUES ($1, $2)\n ON CONFLICT (key)\n DO UPDATE SET\n previous_job_id = debounce_key.job_id,\n job_id = EXCLUDED.job_id,\n debounced_times = debounce_key.debounced_times + 1\n RETURNING\n debounced_times,\n first_started_at,\n previous_job_id AS job_id_to_debounce\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "debounced_times",
|
||||
"type_info": "Int4"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "first_started_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "job_id_to_debounce",
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "45de6be332f4ec89482782e3b1640649ed1a6f6d4f1e1b636c71bdd36c31b618"
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT \n path,\n summary,\n description\n FROM\n flow\n WHERE\n path ~ ANY($1) AND\n workspace_id = $2 AND\n archived is FALSE\n ",
|
||||
"query": "SELECT\n path,\n summary,\n description\n FROM\n flow\n WHERE\n path ~ ANY($1) AND\n workspace_id = $2 AND\n archived is FALSE\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -31,5 +31,5 @@
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "33367c42e87e78ae987c0966dc4d445c5eff75b2e2843ffd7a46b03cbaea9ae8"
|
||||
"hash": "4615b37cb848f9589622426d291c721e532b230c527deb701e24605c7027e38b"
|
||||
}
|
||||
+1
-1
@@ -15,7 +15,7 @@
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"
|
||||
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT debounced_times FROM debounce_key WHERE key = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "debounced_times",
|
||||
"type_info": "Int4"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "6009cf60c43608bb1c7924dbd0a9cdc01986d787eb24f0fffe4550d939851e22"
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT \n path,\n summary,\n description\n FROM\n script\n WHERE\n path ~ ANY($1) AND\n workspace_id = $2 AND\n archived is FALSE\n ",
|
||||
"query": "SELECT\n path,\n summary,\n description\n FROM\n script\n WHERE\n path ~ ANY($1) AND\n workspace_id = $2 AND\n archived is FALSE\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -31,5 +31,5 @@
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "dc36b46b9eb80cb7c92fa72519d117eda99a6f482a073ccd36a6431ef689a3fd"
|
||||
"hash": "69b44efb0144fececccafc8a77d040649bb17e239e5afbc27b915efc4c95d54e"
|
||||
}
|
||||
+9
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n external_id,\n workspace_id,\n service_name AS \"service_name!: ServiceName\",\n script_path,\n is_flow,\n webhook_token_hash,\n service_config,\n error,\n created_at,\n updated_at\n FROM\n native_trigger\n WHERE\n workspace_id = $1\n AND service_name = $2\n AND script_path = $3\n AND is_flow = $4\n LIMIT 1\n ",
|
||||
"query": "\n SELECT\n external_id,\n workspace_id,\n service_name AS \"service_name!: ServiceName\",\n script_path,\n is_flow,\n webhook_token_hash,\n service_config,\n error,\n created_at,\n updated_at,\n summary\n FROM\n native_trigger\n WHERE\n workspace_id = $1\n AND service_name = $2\n AND script_path = $3\n AND is_flow = $4\n LIMIT 1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -62,6 +62,11 @@
|
||||
"ordinal": 9,
|
||||
"name": "updated_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 10,
|
||||
"name": "summary",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -92,8 +97,9 @@
|
||||
true,
|
||||
true,
|
||||
false,
|
||||
false
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "1a69ef11a3f361f105c2a8af7b7fa182f3953150ade1756259b31a50e9308fce"
|
||||
"hash": "6dafcc89668fb0e5740f23264b515b1724c36031da39d53ae6c329a479bdf8aa"
|
||||
}
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n WITH dk AS (\n INSERT INTO debounce_key (job_id, key)\n VALUES ($1, $2)\n ON CONFLICT (key)\n DO UPDATE SET\n previous_job_id = debounce_key.job_id,\n job_id = EXCLUDED.job_id,\n debounced_times = debounce_key.debounced_times + 1\n RETURNING\n debounced_times,\n first_started_at,\n previous_job_id AS job_id_to_debounce\n ), _batch AS (\n INSERT INTO v2_job_debounce_batch (id, debounce_batch)\n SELECT\n $1,\n COALESCE(\n (SELECT debounce_batch FROM v2_job_debounce_batch WHERE id = dk.job_id_to_debounce LIMIT 1),\n nextval('debounce_batch_seq')\n )\n FROM dk\n )\n SELECT debounced_times, first_started_at, job_id_to_debounce FROM dk\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "debounced_times",
|
||||
"type_info": "Int4"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "first_started_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "job_id_to_debounce",
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "8657c21ace89a9bafe4d184b30e3fc104a2c83f698f7dd657d6e6c95c4ff1f3b"
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT content FROM raw_script_temp WHERE hash = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "content",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Bpchar"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "88ec0ddcc86fb67089b551ccbce7b932800e33cd462a651f7e6716929ee9b6f2"
|
||||
}
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE password SET disabled = $1 WHERE email = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Bool",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "8bd266705fc8272f3d8941922ad7d18161eb6f5ec1ba9f1b55feffe8b6518c67"
|
||||
}
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n schema AS \"schema: serde_json::Value\",\n has_preprocessor\n FROM script\n WHERE path = $1 AND workspace_id = $2\n AND NOT archived AND NOT deleted\n ORDER BY created_at DESC\n LIMIT 1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "schema: serde_json::Value",
|
||||
"type_info": "Json"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "has_preprocessor",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "909a095d679ee7b8266fc93fe0c598e4b6583daa2b0d7c593fc72ff9b8d1d06b"
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT content FROM script WHERE path = $1 AND workspace_id = $2\n AND archived = false ORDER BY created_at DESC LIMIT 1\n ",
|
||||
"query": "\n SELECT content FROM script WHERE path = $1 AND workspace_id = $2\n AND archived = false ORDER BY created_at DESC LIMIT 1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -19,5 +19,5 @@
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "c7cae4cf872fce0a989cf89aa35929218a9d459ee1c2b36a28b110e9741ab623"
|
||||
"hash": "96f6163a164b9ffb4ec52372d7fea158db101f54f0908551b5a4f5e6655e122b"
|
||||
}
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT value FROM global_settings WHERE name = 'ai_config'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "value",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "975099ff6b07718ea94bcb5f84a4414c59199964cee53ef2ee6b35a78cf0c49a"
|
||||
}
|
||||
-35
@@ -1,35 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n WITH dk AS (\n INSERT INTO debounce_key (job_id, key)\n VALUES ($1, $2)\n ON CONFLICT (key)\n DO UPDATE SET\n previous_job_id = debounce_key.job_id,\n job_id = EXCLUDED.job_id,\n debounced_times = debounce_key.debounced_times + 1\n RETURNING\n debounced_times,\n first_started_at,\n previous_job_id AS job_id_to_debounce\n ), _batch AS (\n INSERT INTO v2_job_debounce_batch (id, debounce_batch)\n SELECT\n $1,\n COALESCE(\n (SELECT debounce_batch FROM v2_job_debounce_batch WHERE id = dk.job_id_to_debounce LIMIT 1),\n nextval('debounce_batch_seq')\n )\n FROM dk\n )\n SELECT debounced_times, first_started_at, job_id_to_debounce FROM dk\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "debounced_times",
|
||||
"type_info": "Int4"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "first_started_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "job_id_to_debounce",
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "98033aae3182bde22d5b2ff08ef6e8a4f8f3a9bf04238b33e9caf46836df73d9"
|
||||
}
|
||||
-17
@@ -1,17 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, deployment_msg) VALUES ($1, $2, $3, $4) ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET deployment_msg = EXCLUDED.deployment_msg",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Int8",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "9f07510019ebe6f0c5fa17bf31c2d14755474cba82b3b388a47585a8bb325b1a"
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT hash FROM script WHERE path = $1 AND workspace_id = $2 AND deleted = false ORDER BY created_at DESC LIMIT 1",
|
||||
"query": "SELECT hash FROM script WHERE path = $1 AND workspace_id = $2 AND deleted = false AND archived = false ORDER BY created_at DESC LIMIT 1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -19,5 +19,5 @@
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "d5661c7557cf3a8dee7cf799cd364d21d38edb827d2c08b0ca7d72311b78d574"
|
||||
"hash": "a32d7ba43745226fd65328475731526e0b20ea6eeafeb937eb01cdc2cdfcb859"
|
||||
}
|
||||
+8
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "WITH active_users AS (SELECT distinct username as email FROM (SELECT username, timestamp, operation FROM audit_partitioned UNION ALL SELECT username, timestamp, operation FROM audit) AS a WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n authors as (SELECT distinct email FROM usr WHERE usr.operator IS false)\n SELECT email, email NOT IN (SELECT email FROM authors) as operator_only, login_type::text, verified, super_admin, devops, name, company, username, first_time_user, role_source\n FROM password\n WHERE email IN (SELECT email FROM active_users)\n ORDER BY super_admin DESC, devops DESC\n LIMIT $1 OFFSET $2",
|
||||
"query": "WITH active_users AS (SELECT distinct username as email FROM (SELECT username, timestamp, operation FROM audit_partitioned UNION ALL SELECT username, timestamp, operation FROM audit) AS a WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n authors as (SELECT distinct email FROM usr WHERE usr.operator IS false)\n SELECT email, email NOT IN (SELECT email FROM authors) as operator_only, login_type::text, verified, super_admin, devops, name, company, username, first_time_user, role_source, disabled\n FROM password\n WHERE email IN (SELECT email FROM active_users)\n ORDER BY super_admin DESC, devops DESC\n LIMIT $1 OFFSET $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -57,6 +57,11 @@
|
||||
"ordinal": 10,
|
||||
"name": "role_source",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 11,
|
||||
"name": "disabled",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -76,8 +81,9 @@
|
||||
true,
|
||||
true,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "60118de85463098220b1c74f667b6fedb0f3f0040844c3774145e8f1f4c023ce"
|
||||
"hash": "a5fd115e7be5129d623543bbfa7b5b31f0efc6d8ef73f691009c73f833dcee10"
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT scheduled_for FROM v2_job_queue WHERE id = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "scheduled_for",
|
||||
"type_info": "Timestamptz"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "a9c2019c7bafb172dfb353870238c02673113b806bc3c7a71354c34255ecd31c"
|
||||
}
|
||||
+9
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n nt.external_id,\n nt.workspace_id,\n nt.service_name AS \"service_name!: ServiceName\",\n nt.script_path,\n nt.is_flow,\n nt.webhook_token_hash,\n nt.service_config,\n nt.error,\n nt.created_at,\n nt.updated_at\n FROM\n native_trigger nt\n WHERE\n nt.workspace_id = $1 AND\n nt.service_name = $2 AND\n ($5::text IS NULL OR nt.script_path = $5) AND\n ($6::bool IS NULL OR nt.is_flow = $6) AND\n (\n (nt.is_flow = false AND EXISTS (\n SELECT 1 FROM script s\n WHERE s.workspace_id = nt.workspace_id\n AND s.path = nt.script_path\n ))\n OR\n (nt.is_flow = true AND EXISTS (\n SELECT 1 FROM flow f\n WHERE f.workspace_id = nt.workspace_id\n AND f.path = nt.script_path\n ))\n )\n LIMIT $3\n OFFSET $4\n ",
|
||||
"query": "\n SELECT\n nt.external_id,\n nt.workspace_id,\n nt.service_name AS \"service_name!: ServiceName\",\n nt.script_path,\n nt.is_flow,\n nt.webhook_token_hash,\n nt.service_config,\n nt.error,\n nt.created_at,\n nt.updated_at,\n nt.summary\n FROM\n native_trigger nt\n WHERE\n nt.workspace_id = $1 AND\n nt.service_name = $2 AND\n ($5::text IS NULL OR nt.script_path = $5) AND\n ($6::bool IS NULL OR nt.is_flow = $6) AND\n (\n (nt.is_flow = false AND EXISTS (\n SELECT 1 FROM script s\n WHERE s.workspace_id = nt.workspace_id\n AND s.path = nt.script_path\n ))\n OR\n (nt.is_flow = true AND EXISTS (\n SELECT 1 FROM flow f\n WHERE f.workspace_id = nt.workspace_id\n AND f.path = nt.script_path\n ))\n )\n LIMIT $3\n OFFSET $4\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -62,6 +62,11 @@
|
||||
"ordinal": 9,
|
||||
"name": "updated_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 10,
|
||||
"name": "summary",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -94,8 +99,9 @@
|
||||
true,
|
||||
true,
|
||||
false,
|
||||
false
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "a115d8ea786907561afdbbc07d11dc715d80b00c0e79b61b0057a3ae3886a85e"
|
||||
"hash": "b0775af41a9b54cce040bf37cae770e63dab12a1383a0855d105c061e4e4ca48"
|
||||
}
|
||||
+4
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE native_trigger\n SET script_path = $1, is_flow = $2, webhook_token_hash = $3, service_config = $4, error = NULL, updated_at = NOW()\n WHERE\n workspace_id = $5\n AND service_name = $6\n AND external_id = $7\n ",
|
||||
"query": "\n UPDATE native_trigger\n SET script_path = $1, is_flow = $2, webhook_token_hash = $3, service_config = $4, summary = $8, error = NULL, updated_at = NOW()\n WHERE\n workspace_id = $5\n AND service_name = $6\n AND external_id = $7\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -21,10 +21,11 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"Text"
|
||||
"Text",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "40a8bf6a5a42c275d73221bc5f386f2e18cb911352551d0a34bf1933e558674e"
|
||||
"hash": "bf224f6441c36187f1402f9f01bfe15bb9edfa1dc9052f8a829e486b7334d708"
|
||||
}
|
||||
-35
@@ -1,35 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n WITH dk AS (\n INSERT INTO debounce_key (job_id, key)\n VALUES ($1, $2)\n ON CONFLICT (key)\n DO UPDATE SET\n previous_job_id = debounce_key.job_id,\n job_id = EXCLUDED.job_id,\n debounced_times = debounce_key.debounced_times + 1\n RETURNING\n debounced_times,\n first_started_at,\n previous_job_id AS job_id_to_debounce\n ), _batch AS (\n INSERT INTO v2_job_debounce_batch (id, debounce_batch)\n SELECT\n $1,\n COALESCE(\n (SELECT debounce_batch FROM v2_job_debounce_batch WHERE id = dk.job_id_to_debounce LIMIT 1),\n nextval('debounce_batch_seq')\n )\n FROM dk\n )\n SELECT debounced_times, first_started_at, job_id_to_debounce FROM dk\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "debounced_times",
|
||||
"type_info": "Int4"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "first_started_at",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "job_id_to_debounce",
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "c2347460b73ae9d3167031c263032e97ebefb46be9e58bd3da9067748075311b"
|
||||
}
|
||||
+40
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n WITH RECURSIVE chain AS (\n SELECT\n j.id,\n j.parent_job,\n j.flow_step_id,\n 1 AS depth\n FROM v2_job j\n WHERE j.id = $1\n UNION ALL\n SELECT\n pj.id,\n pj.parent_job,\n pj.flow_step_id,\n c.depth + 1\n FROM chain c\n JOIN v2_job pj ON pj.id = c.parent_job\n WHERE c.parent_job IS NOT NULL\n )\n SELECT\n c.id,\n c.parent_job,\n c.flow_step_id,\n EXISTS(SELECT 1 FROM v2_job_queue q WHERE q.id = c.parent_job) AS \"parent_in_queue!\"\n FROM chain c\n WHERE c.depth >= 1\n ORDER BY c.depth ASC\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "id",
|
||||
"type_info": "Uuid"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "parent_job",
|
||||
"type_info": "Uuid"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "flow_step_id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "parent_in_queue!",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "c79501b30fc28a3ae761579d05f2296e848554e62a00ff7c164109bf3d97f44f"
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT super_admin FROM password WHERE email = $1 AND disabled = false",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "super_admin",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "ccc49a2a6e11f874825365de758bdc0e1934d67d3f2b14047d434b77d370af21"
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO password (email, login_type, verified, username, name) VALUES ($1, 'saml', true, $2, $3) ON CONFLICT DO NOTHING",
|
||||
"query": "INSERT INTO password (email, login_type, verified, username, name) VALUES ($1, 'saml', true, $2, $3) ON CONFLICT (email) DO UPDATE SET disabled = false",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -12,5 +12,5 @@
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "638d3c2ba1198dce5b5b0e47df59a92ff8011e19fbefcc3960d6f0fe167e55b6"
|
||||
"hash": "daa1a6bf3d4a1001da88301932a7ac9019767074158e0c027988e5b0d51a3656"
|
||||
}
|
||||
-22
@@ -1,22 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT default_app FROM workspace_settings WHERE workspace_id = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "default_app",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "ed1a053c7b22d9cb69767be40d33f3be67b6160cd258c86b8e8f22a6d601afd0"
|
||||
}
|
||||
+8
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT email, login_type::TEXT, super_admin, devops, verified, name, company, username, NULL::bool as operator_only, first_time_user, role_source FROM password WHERE email = $1",
|
||||
"query": "SELECT email, login_type::TEXT, super_admin, devops, verified, name, company, username, NULL::bool as operator_only, first_time_user, role_source, disabled FROM password WHERE email = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -57,6 +57,11 @@
|
||||
"ordinal": 10,
|
||||
"name": "role_source",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 11,
|
||||
"name": "disabled",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -75,8 +80,9 @@
|
||||
true,
|
||||
null,
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "65c59e224e460351c2f88261f8b1b1e7ce2bb160270b59c0f359b7952453b2b9"
|
||||
"hash": "f0c9c54740cc1c0c2a6fa4e79d4d504b7b5cb7a39538ab9abeb44f781c711493"
|
||||
}
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT DISTINCT ON (path) path, language AS \"language: ScriptLang\", content FROM script\n WHERE workspace_id = $1\n AND archived = false\n AND dedicated_worker = true\n AND language = ANY($2::SCRIPT_LANG[])\n ORDER BY path, created_at DESC",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "language: ScriptLang",
|
||||
"type_info": {
|
||||
"Custom": {
|
||||
"name": "script_lang",
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"python3",
|
||||
"deno",
|
||||
"go",
|
||||
"bash",
|
||||
"postgresql",
|
||||
"nativets",
|
||||
"bun",
|
||||
"mysql",
|
||||
"bigquery",
|
||||
"snowflake",
|
||||
"graphql",
|
||||
"powershell",
|
||||
"mssql",
|
||||
"php",
|
||||
"bunnative",
|
||||
"rust",
|
||||
"ansible",
|
||||
"csharp",
|
||||
"oracledb",
|
||||
"nu",
|
||||
"java",
|
||||
"duckdb",
|
||||
"ruby"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "content",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
{
|
||||
"Custom": {
|
||||
"name": "script_lang[]",
|
||||
"kind": {
|
||||
"Array": {
|
||||
"Custom": {
|
||||
"name": "script_lang",
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"python3",
|
||||
"deno",
|
||||
"go",
|
||||
"bash",
|
||||
"postgresql",
|
||||
"nativets",
|
||||
"bun",
|
||||
"mysql",
|
||||
"bigquery",
|
||||
"snowflake",
|
||||
"graphql",
|
||||
"powershell",
|
||||
"mssql",
|
||||
"php",
|
||||
"bunnative",
|
||||
"rust",
|
||||
"ansible",
|
||||
"csharp",
|
||||
"oracledb",
|
||||
"nu",
|
||||
"java",
|
||||
"duckdb",
|
||||
"ruby"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "f2fa27ed5020aa9c085176b25466be1bceb79c82e7b5542f17b23b6d70cc02d6"
|
||||
}
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE v2_job_queue SET scheduled_for = $1 WHERE id = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Timestamptz",
|
||||
"Uuid"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "f9eabfab66ae102c8a61ae5e6349b5b167ff7939828a747ad09bb06c9d6c6d8d"
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT disabled FROM password WHERE email = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "disabled",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "fc6c6310ae8ac5eb351d7e2af1678447d0aa3d143e94e49924ff7ac8b7abf924"
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO token\n (token_hash, token_prefix, token, label, super_admin, email)\n VALUES ($1, $2, $3, $4, $5, $6)",
|
||||
"query": "INSERT INTO token\n (token_hash, token_prefix, token, label, super_admin, email)\n VALUES ($1, $2, $3, $4, $5, $6)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -15,5 +15,5 @@
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "d05f20431cd08f737bfbf904efedfdf104e3d77b0725c5355305d19f67359e90"
|
||||
"hash": "fd4c5391107af34a3bf9b83b0c3f7d5ee9490240a627b20a1037444845e39c5f"
|
||||
}
|
||||
Generated
+262
-258
File diff suppressed because it is too large
Load Diff
+5
-5
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "windmill"
|
||||
version = "1.662.0"
|
||||
version = "1.664.0"
|
||||
authors.workspace = true
|
||||
edition.workspace = true
|
||||
|
||||
@@ -82,7 +82,7 @@ members = [
|
||||
exclude = ["./windmill-duckdb-ffi-internal"]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.662.0"
|
||||
version = "1.664.0"
|
||||
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
|
||||
edition = "2021"
|
||||
|
||||
@@ -373,7 +373,7 @@ tower = "^0"
|
||||
tower-http = { version = "^0.6", features = ["trace", "cors", "catch-panic"] }
|
||||
tower-cookies = "^0.10"
|
||||
#stuck because of swc for now
|
||||
serde = "=1.0.219"
|
||||
serde = "=1.0.220"
|
||||
serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
|
||||
serde_yml = "0.0.12"
|
||||
uuid = { version = "^1", features = ["serde", "v4", "js"] }
|
||||
@@ -510,7 +510,7 @@ native-tls = ">=0.2, <0.2.17"
|
||||
# samael will break compilation on MacOS. Use this fork instead to make it work
|
||||
# samael = { git="https://github.com/njaremko/samael", rev="464d015e3ae393e4b5dd00b4d6baa1b617de0dd6", features = ["xmlsec"] }
|
||||
libxml = { version = "=0.3.3" }
|
||||
samael = { version="0.0.14", features = ["xmlsec"] }
|
||||
samael = { git="https://github.com/njaremko/samael", rev="f879f1942ec1b34b6d3027ce7e4724ad95d15dfa", features = ["xmlsec"] }
|
||||
gcp_auth = "0.9.0"
|
||||
rust_decimal = { version = "^1", features = ["db-postgres", "serde-float"]}
|
||||
jsonwebtoken = "8.3.0"
|
||||
@@ -587,7 +587,7 @@ tikv-jemalloc-ctl = { version = "^0.5" }
|
||||
triomphe = "^0"
|
||||
pin-project-lite = "^0"
|
||||
|
||||
tantivy = { git="https://github.com/windmill-labs/tantivy", rev="6a24621231202ccd77bec90d8787e2281fb94e4e" }
|
||||
tantivy = { git="https://github.com/windmill-labs/tantivy", rev="6ae7c70bc603b8e69e27f3240e08bd00a93fb12c" }
|
||||
|
||||
backon = "1.3.0"
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
563877bf1c8b4184f638bab51be89b1c0aec6dad
|
||||
a1274aa11a83f608eacc32c0d449ca3527d98c15
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
DROP INDEX IF EXISTS idx_raw_script_temp_created_at;
|
||||
DROP TABLE IF EXISTS raw_script_temp;
|
||||
@@ -0,0 +1,11 @@
|
||||
-- Temporary storage for raw script content during CLI lock generation
|
||||
-- Content is stored with hash as key, cleaned up after 1 week
|
||||
CREATE TABLE raw_script_temp (
|
||||
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id),
|
||||
hash CHAR(64) NOT NULL,
|
||||
content TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
PRIMARY KEY (workspace_id, hash)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_raw_script_temp_created_at ON raw_script_temp (created_at);
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE native_trigger DROP COLUMN IF EXISTS summary;
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE native_trigger ADD COLUMN summary VARCHAR(1000);
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE password DROP COLUMN IF EXISTS disabled;
|
||||
@@ -0,0 +1 @@
|
||||
ALTER TABLE password ADD COLUMN disabled BOOLEAN NOT NULL DEFAULT false;
|
||||
@@ -0,0 +1,3 @@
|
||||
-- Revoke grants for app_bundles table
|
||||
REVOKE ALL ON app_bundles FROM windmill_user;
|
||||
REVOKE ALL ON app_bundles FROM windmill_admin;
|
||||
@@ -0,0 +1,3 @@
|
||||
-- Add grants for app_bundles table
|
||||
GRANT ALL ON app_bundles TO windmill_user;
|
||||
GRANT ALL ON app_bundles TO windmill_admin;
|
||||
@@ -13,21 +13,19 @@ regex-lite.workspace = true
|
||||
|
||||
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
|
||||
regex.workspace = true
|
||||
|
||||
[dependencies]
|
||||
windmill-parser.workspace = true
|
||||
windmill-common.workspace = true
|
||||
rustpython-parser.workspace = true
|
||||
malachite.workspace = true
|
||||
malachite-bigint.workspace = true
|
||||
phf.workspace = true
|
||||
itertools.workspace = true
|
||||
serde_json.workspace = true
|
||||
anyhow.workspace = true
|
||||
lazy_static.workspace = true
|
||||
sqlx.workspace = true
|
||||
async-recursion.workspace = true
|
||||
toml.workspace = true
|
||||
serde.workspace = true
|
||||
pep440_rs.workspace = true
|
||||
tracing.workspace = true
|
||||
|
||||
[dependencies]
|
||||
windmill-parser.workspace = true
|
||||
rustpython-parser.workspace = true
|
||||
phf.workspace = true
|
||||
itertools.workspace = true
|
||||
serde_json.workspace = true
|
||||
anyhow.workspace = true
|
||||
lazy_static.workspace = true
|
||||
|
||||
@@ -8,10 +8,14 @@
|
||||
|
||||
mod mapping;
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use async_recursion::async_recursion;
|
||||
use itertools::Itertools;
|
||||
use lazy_static::lazy_static;
|
||||
use std::{collections::HashMap, str::FromStr};
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use std::str::FromStr;
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use std::collections::HashMap;
|
||||
|
||||
use mapping::{FULL_IMPORTS_MAP, SHORT_IMPORTS_MAP};
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
@@ -24,7 +28,9 @@ use rustpython_parser::{
|
||||
text_size::TextRange,
|
||||
Parse,
|
||||
};
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use sqlx::{Pool, Postgres};
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
use windmill_common::{
|
||||
error::{self, to_anyhow},
|
||||
worker::{
|
||||
@@ -46,10 +52,14 @@ fn replace_full_import(x: &str) -> Option<String> {
|
||||
FULL_IMPORTS_MAP.get(x).map(|x| (*x).to_owned())
|
||||
}
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
lazy_static! {
|
||||
static ref RE: Regex = Regex::new(r"^\#\s?(\S+)\s*$").unwrap();
|
||||
static ref PIN_RE: Regex = Regex::new(r"(?:\s*#\s*(pin|repin):\s*)(\S*)").unwrap();
|
||||
static ref PKG_RE: Regex = Regex::new(r"^([^!=<>]+)(?:[!=<>]|$)").unwrap();
|
||||
}
|
||||
|
||||
lazy_static! {
|
||||
static ref PIN_RE: Regex = Regex::new(r"(?:\s*#\s*(pin|repin):\s*)(\S*)").unwrap();
|
||||
// Regex to properly match main function definition at line start,
|
||||
// capturing both sync and async variants
|
||||
static ref DEF_MAIN_RE: Regex = Regex::new(r"(?m)^(async\s+)?def\s+main\s*\(").unwrap();
|
||||
@@ -82,7 +92,7 @@ fn process_import(module: Option<String>, path: &str, level: usize) -> Vec<NImpo
|
||||
}
|
||||
}
|
||||
|
||||
pub fn parse_relative_imports(code: &str, path: &str) -> error::Result<Vec<String>> {
|
||||
pub fn parse_relative_imports(code: &str, path: &str) -> anyhow::Result<Vec<String>> {
|
||||
let nimports = parse_code_for_imports(code, path)?;
|
||||
return Ok(nimports
|
||||
.into_iter()
|
||||
@@ -94,7 +104,7 @@ pub fn parse_relative_imports(code: &str, path: &str) -> error::Result<Vec<Strin
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||
enum NImport {
|
||||
pub enum NImport {
|
||||
// Order matters! First we want to resolve all repins
|
||||
|
||||
// manually repinned requirement
|
||||
@@ -134,6 +144,8 @@ enum NImport {
|
||||
// Relative imports
|
||||
Relative(String),
|
||||
}
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||
enum NImportResolved {
|
||||
Repin { pin: ImportPin, key: String },
|
||||
@@ -142,12 +154,12 @@ enum NImportResolved {
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||
struct ImportPin {
|
||||
pkg: String,
|
||||
path: String,
|
||||
pub struct ImportPin {
|
||||
pub pkg: String,
|
||||
pub path: String,
|
||||
}
|
||||
|
||||
fn parse_code_for_imports(code: &str, path: &str) -> error::Result<Vec<NImport>> {
|
||||
pub fn parse_code_for_imports(code: &str, path: &str) -> anyhow::Result<Vec<NImport>> {
|
||||
// Use regex to safely find the main function definition
|
||||
let mut code = DEF_MAIN_RE
|
||||
.split(code)
|
||||
@@ -175,7 +187,7 @@ fn parse_code_for_imports(code: &str, path: &str) -> error::Result<Vec<NImport>>
|
||||
let code_with_fake_main = format!("{}\n\ndef main(): pass", code);
|
||||
|
||||
let ast = Suite::parse(&code_with_fake_main, "main.py").map_err(|e| {
|
||||
error::Error::ExecutionErr(format!("Error parsing code for imports: {}", e.to_string()))
|
||||
anyhow::anyhow!("Error parsing code for imports: {}", e.to_string())
|
||||
})?;
|
||||
// Note: We're still using the original code for finding pins,
|
||||
// as the TextRange values from the parsed AST would be based on code_with_fake_main
|
||||
@@ -256,6 +268,7 @@ fn parse_code_for_imports(code: &str, path: &str) -> error::Result<Vec<NImport>>
|
||||
return Ok(nimports);
|
||||
}
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub async fn parse_python_imports(
|
||||
code: &str,
|
||||
w_id: &str,
|
||||
@@ -264,6 +277,7 @@ pub async fn parse_python_imports(
|
||||
version_specifiers: &mut Vec<pep440_rs::VersionSpecifier>,
|
||||
locked_v: &mut Option<pep440_rs::Version>,
|
||||
raw_workspace_dependencies_o: &Option<RawWorkspaceDependencies>,
|
||||
temp_script_refs: &Option<HashMap<String, String>>,
|
||||
) -> error::Result<(Vec<String>, Option<String>)> {
|
||||
let mut compile_error_hint: Option<String> = None;
|
||||
let mut imports = parse_python_imports_inner(
|
||||
@@ -276,6 +290,7 @@ pub async fn parse_python_imports(
|
||||
&mut None,
|
||||
locked_v,
|
||||
raw_workspace_dependencies_o,
|
||||
temp_script_refs,
|
||||
)
|
||||
.await?
|
||||
.into_values()
|
||||
@@ -313,6 +328,7 @@ pub async fn parse_python_imports(
|
||||
Ok((imports, compile_error_hint))
|
||||
}
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
fn extract_pkg_name(requirement: &str) -> String {
|
||||
PKG_RE
|
||||
.captures(requirement)
|
||||
@@ -320,6 +336,7 @@ fn extract_pkg_name(requirement: &str) -> String {
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
#[async_recursion]
|
||||
async fn parse_python_imports_inner(
|
||||
code: &str,
|
||||
@@ -331,6 +348,7 @@ async fn parse_python_imports_inner(
|
||||
path_where_annotated_pyv: &mut Option<String>,
|
||||
locked_v: &mut Option<pep440_rs::Version>,
|
||||
raw_workspace_dependencies_o: &Option<RawWorkspaceDependencies>,
|
||||
temp_script_refs: &Option<HashMap<String, String>>,
|
||||
) -> error::Result<HashMap<String, NImportResolved>> {
|
||||
tracing::debug!("Parsing python imports for path: {}", path);
|
||||
let PythonAnnotations { py310, py311, py312, py313, .. } = PythonAnnotations::parse(&code);
|
||||
@@ -494,17 +512,37 @@ async fn parse_python_imports_inner(
|
||||
for n in nimports.into_iter() {
|
||||
let mut nested = match n {
|
||||
NImport::Relative(rpath) => {
|
||||
let code = sqlx::query_scalar!(
|
||||
r#"
|
||||
SELECT content FROM script WHERE path = $1 AND workspace_id = $2
|
||||
AND archived = false ORDER BY created_at DESC LIMIT 1
|
||||
"#,
|
||||
&rpath,
|
||||
w_id
|
||||
)
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
.unwrap_or_else(|| "".to_string());
|
||||
// First try to get content from temp_script_refs cache if available
|
||||
let code_from_cache = if let Some(hash) = temp_script_refs.as_ref().and_then(|dt| dt.get(&rpath)) {
|
||||
tracing::debug!("Found relative import '{}' in temp_script_refs with hash '{}'", rpath, hash);
|
||||
match windmill_common::cache::raw_script_temp::load(hash.clone(), db).await {
|
||||
Ok(content) => Some(content),
|
||||
Err(e) => {
|
||||
tracing::warn!("temp_script_refs hash '{}' not found in cache: {}, falling back to deployed script", hash, e);
|
||||
None
|
||||
}
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
// Use cached content if available, otherwise fall back to deployed script
|
||||
let code = match code_from_cache {
|
||||
Some(content) => content,
|
||||
None => {
|
||||
sqlx::query_scalar!(
|
||||
r#"
|
||||
SELECT content FROM script WHERE path = $1 AND workspace_id = $2
|
||||
AND archived = false ORDER BY created_at DESC LIMIT 1
|
||||
"#,
|
||||
&rpath,
|
||||
w_id
|
||||
)
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
.unwrap_or_else(|| "".to_string())
|
||||
}
|
||||
};
|
||||
|
||||
if already_visited.contains(&rpath) {
|
||||
vec![]
|
||||
@@ -522,6 +560,7 @@ async fn parse_python_imports_inner(
|
||||
path_where_annotated_pyv,
|
||||
locked_v,
|
||||
raw_workspace_dependencies_o,
|
||||
temp_script_refs,
|
||||
)
|
||||
.await?
|
||||
.into_values()
|
||||
@@ -646,6 +685,7 @@ async fn parse_python_imports_inner(
|
||||
Ok(final_imports)
|
||||
}
|
||||
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
fn extract_nimports_from_content(
|
||||
content: &str,
|
||||
hm: &mut HashMap<String, NImportResolved>,
|
||||
|
||||
@@ -26,6 +26,7 @@ def main():
|
||||
&mut vec![],
|
||||
&mut None,
|
||||
&None,
|
||||
&None,
|
||||
)
|
||||
.await?;
|
||||
// println!("{}", serde_json::to_string(&r)?);
|
||||
@@ -67,6 +68,7 @@ def main():
|
||||
&mut vec![],
|
||||
&mut None,
|
||||
&None,
|
||||
&None,
|
||||
)
|
||||
.await?;
|
||||
println!("{}", serde_json::to_string(&r)?);
|
||||
@@ -98,6 +100,7 @@ def main():
|
||||
&mut vec![],
|
||||
&mut None,
|
||||
&None,
|
||||
&None,
|
||||
)
|
||||
.await?;
|
||||
println!("{}", serde_json::to_string(&r)?);
|
||||
|
||||
@@ -16,7 +16,6 @@ regex.workspace = true
|
||||
|
||||
[dependencies]
|
||||
windmill-parser.workspace = true
|
||||
windmill-types.workspace = true
|
||||
anyhow.workspace = true
|
||||
lazy_static.workspace = true
|
||||
serde_json.workspace = true
|
||||
|
||||
@@ -15,7 +15,7 @@ use std::{
|
||||
iter::Peekable,
|
||||
str::CharIndices,
|
||||
};
|
||||
pub use windmill_parser::{Arg, MainArgSignature, ObjectType, Typ};
|
||||
pub use windmill_parser::{s3_mode_extension, Arg, MainArgSignature, ObjectType, S3ModeFormat, Typ};
|
||||
|
||||
pub const SANITIZED_ENUM_STR: &str = "__sanitized_enum__";
|
||||
pub const SANITIZED_RAW_STRING_STR: &str = "__sanitized_raw_string__";
|
||||
@@ -143,7 +143,6 @@ pub fn parse_db_resource(code: &str) -> Option<String> {
|
||||
cap.map(|x| x.get(1).map(|x| x.as_str().to_string()).unwrap())
|
||||
}
|
||||
|
||||
pub use windmill_types::s3::{s3_mode_extension, S3ModeFormat};
|
||||
pub struct S3ModeArgs {
|
||||
pub prefix: Option<String>,
|
||||
pub storage: Option<String>,
|
||||
|
||||
@@ -117,6 +117,12 @@ impl Visit for ImportsFinder {
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse TypeScript/JavaScript code and extract all import paths as raw strings.
|
||||
///
|
||||
/// Returns import paths exactly as written in the code (e.g., `"./module"`, `"../utils"`, `"lodash"`).
|
||||
/// Does not resolve relative paths to absolute Windmill paths.
|
||||
///
|
||||
/// See also: [`parse_relative_imports`] for resolved absolute paths.
|
||||
pub fn parse_expr_for_imports(code: &str, skip_type_only: bool) -> anyhow::Result<Vec<String>> {
|
||||
let cm: Lrc<SourceMap> = Default::default();
|
||||
let fm = cm.new_source_file(FileName::Custom("main.d.ts".into()).into(), code.into());
|
||||
@@ -151,6 +157,82 @@ pub fn parse_expr_for_imports(code: &str, skip_type_only: bool) -> anyhow::Resul
|
||||
Ok(imports)
|
||||
}
|
||||
|
||||
/// Parse TypeScript/JavaScript code and extract relative imports resolved to absolute Windmill paths.
|
||||
///
|
||||
/// Takes the script's Windmill path (e.g., `"f/folder/script"`) and resolves relative imports
|
||||
/// like `"./module"` or `"../utils"` to absolute paths like `"f/folder/module"` or `"f/utils"`.
|
||||
///
|
||||
/// Only returns relative imports (those starting with `./`, `../`, or `/`).
|
||||
/// External package imports (e.g., `"lodash"`) are filtered out.
|
||||
///
|
||||
/// See also: [`parse_expr_for_imports`] for raw import strings without resolution.
|
||||
///
|
||||
/// # Arguments
|
||||
/// * `code` - The TypeScript/JavaScript source code
|
||||
/// * `path` - The Windmill path of the script (e.g., `"f/folder/script"`)
|
||||
///
|
||||
/// # Returns
|
||||
/// A sorted, deduplicated list of resolved absolute Windmill paths.
|
||||
///
|
||||
/// # Examples
|
||||
/// ```ignore
|
||||
/// // Script at "f/folder/script" with: import { x } from "../utils"
|
||||
/// // Returns: ["f/utils"]
|
||||
/// ```
|
||||
pub fn parse_relative_imports(code: &str, path: &str) -> anyhow::Result<Vec<String>> {
|
||||
let imports = parse_expr_for_imports(code, false)?;
|
||||
let script_dir = path.rsplit_once('/').map(|(dir, _)| dir).unwrap_or("");
|
||||
|
||||
let mut resolved: Vec<String> = imports
|
||||
.into_iter()
|
||||
.filter(|imp| is_relative_import(imp))
|
||||
.map(|imp| {
|
||||
// Remove .ts extension if present
|
||||
let imp = imp.strip_suffix(".ts").unwrap_or(&imp);
|
||||
|
||||
if imp.starts_with("/") {
|
||||
// Absolute path (e.g., /f/folder/script) - remove leading slash
|
||||
imp[1..].to_string()
|
||||
} else {
|
||||
// Relative path (e.g., ./script or ../folder/script)
|
||||
let combined = format!("{}/{}", script_dir, imp);
|
||||
normalize_path(&combined)
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
resolved.sort();
|
||||
resolved.dedup();
|
||||
Ok(resolved)
|
||||
}
|
||||
|
||||
/// Check if an import path is a relative import (starts with `./`, `../`, or `/`)
|
||||
fn is_relative_import(import_path: &str) -> bool {
|
||||
import_path.starts_with("./")
|
||||
|| import_path.starts_with("../")
|
||||
|| import_path.starts_with("/")
|
||||
}
|
||||
|
||||
/// Normalize a path by resolving `.` and `..` components
|
||||
fn normalize_path(input_path: &str) -> String {
|
||||
let parts: Vec<&str> = input_path.split('/').filter(|p| !p.is_empty()).collect();
|
||||
let mut result: Vec<&str> = Vec::new();
|
||||
|
||||
for part in parts {
|
||||
if part == "." {
|
||||
continue;
|
||||
} else if part == ".." {
|
||||
if !result.is_empty() {
|
||||
result.pop();
|
||||
}
|
||||
} else {
|
||||
result.push(part);
|
||||
}
|
||||
}
|
||||
|
||||
result.join("/")
|
||||
}
|
||||
|
||||
struct OutputFinder {
|
||||
idents: HashSet<(String, String)>,
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
mod tests {
|
||||
use serde_json::json;
|
||||
use windmill_parser::{Arg, MainArgSignature, ObjectProperty, ObjectType, Typ};
|
||||
use windmill_parser_ts::{parse_deno_signature, parse_expr_for_imports};
|
||||
use windmill_parser_ts::{parse_deno_signature, parse_expr_for_imports, parse_relative_imports};
|
||||
|
||||
#[test]
|
||||
fn test_imports_basic() {
|
||||
@@ -798,7 +798,7 @@ mod tests {
|
||||
// Test case where there are exports but no preprocessor
|
||||
let code = r#"
|
||||
export { foo, bar } from "./utils";
|
||||
|
||||
|
||||
export async function main(param: string) {
|
||||
return param;
|
||||
}
|
||||
@@ -806,4 +806,84 @@ mod tests {
|
||||
let sig = parse_deno_signature(code, false, false, None).unwrap();
|
||||
assert_eq!(sig.has_preprocessor, Some(false));
|
||||
}
|
||||
|
||||
// ==========================================================================
|
||||
// Tests for parse_relative_imports
|
||||
// ==========================================================================
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_dot() {
|
||||
let code = r#"
|
||||
import { helper } from "./helper";
|
||||
export async function main() { return helper(); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/folder/script").unwrap();
|
||||
assert_eq!(result, vec!["f/folder/helper"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_double_dot() {
|
||||
let code = r#"
|
||||
import { utils } from "../utils/helper";
|
||||
export async function main() { return utils(); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/folder/subfolder/script").unwrap();
|
||||
assert_eq!(result, vec!["f/folder/utils/helper"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_absolute_path() {
|
||||
let code = r#"
|
||||
import { shared } from "/f/shared/utils";
|
||||
export async function main() { return shared(); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/folder/script").unwrap();
|
||||
assert_eq!(result, vec!["f/shared/utils"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_mixed() {
|
||||
let code = r#"
|
||||
import { helper } from "./helper";
|
||||
import { utils } from "../utils";
|
||||
import { shared } from "/f/shared/lib";
|
||||
import lodash from "lodash";
|
||||
export async function main() { return helper() + utils() + shared(); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/folder/script").unwrap();
|
||||
// Should only include relative imports, not external packages like lodash
|
||||
assert_eq!(result, vec!["f/folder/helper", "f/shared/lib", "f/utils"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_with_ts_extension() {
|
||||
let code = r#"
|
||||
import { helper } from "./helper.ts";
|
||||
export async function main() { return helper(); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/folder/script").unwrap();
|
||||
assert_eq!(result, vec!["f/folder/helper"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_external_only() {
|
||||
let code = r#"
|
||||
import lodash from "lodash";
|
||||
import { something } from "@scope/package";
|
||||
export async function main() { return lodash.map([]); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/folder/script").unwrap();
|
||||
assert!(result.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_relative_imports_deeply_nested() {
|
||||
let code = r#"
|
||||
import { a } from "../../a";
|
||||
import { b } from "../../../b";
|
||||
export async function main() { return a() + b(); }
|
||||
"#;
|
||||
let result = parse_relative_imports(code, "f/one/two/three/script").unwrap();
|
||||
assert_eq!(result, vec!["f/b", "f/one/a"]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -40,6 +40,7 @@ java-parser = [ "dep:windmill-parser-java"]
|
||||
ruby-parser = [ "dep:windmill-parser-ruby"]
|
||||
wac-parser = [ "dep:windmill-parser-wac"]
|
||||
asset-parser = [ "dep:windmill-parser-ts-asset", "dep:windmill-parser-py-asset", "dep:windmill-parser-sql-asset"]
|
||||
py-imports-parser = [ "dep:windmill-parser-py-imports"]
|
||||
|
||||
[dependencies]
|
||||
anyhow.workspace = true
|
||||
@@ -61,6 +62,7 @@ windmill-parser-wac = { workspace = true, optional = true }
|
||||
windmill-parser-ts-asset = { workspace = true, optional = true }
|
||||
windmill-parser-py-asset = { workspace = true, optional = true }
|
||||
windmill-parser-sql-asset = { workspace = true, optional = true }
|
||||
windmill-parser-py-imports = { workspace = true, optional = true }
|
||||
wasm-bindgen.workspace = true
|
||||
|
||||
serde_json.workspace = true
|
||||
|
||||
@@ -67,6 +67,12 @@ const targets = [
|
||||
features: "asset-parser",
|
||||
env: "default",
|
||||
},
|
||||
{
|
||||
ident: "py-imports",
|
||||
desc: "Python imports"
|
||||
features: "py-imports-parser",
|
||||
env: "default",
|
||||
},
|
||||
# ^^^ Add new entry here ^^^
|
||||
];
|
||||
# NOTE: This is legacy command for building all, but it is not more used
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env nu
|
||||
|
||||
# Build in debug mode specified lang parser to wasm
|
||||
|
||||
# Build in debug mode specified lang parser to wasm
|
||||
# and perform installation to frontend
|
||||
def "main" [
|
||||
lang: string # Example: nu
|
||||
@@ -9,4 +9,7 @@ def "main" [
|
||||
(
|
||||
cd ../../../frontend; npm install ../backend/parsers/windmill-parser-wasm/pkg-($lang)
|
||||
)
|
||||
(
|
||||
cd ../../../cli; bun install ../backend/parsers/windmill-parser-wasm/pkg-($lang)
|
||||
)
|
||||
}
|
||||
|
||||
@@ -36,3 +36,6 @@ popd
|
||||
|
||||
pushd "pkg-asset" && npm publish ${args}
|
||||
popd
|
||||
|
||||
pushd "pkg-py-imports" && npm publish ${args}
|
||||
popd
|
||||
|
||||
@@ -38,6 +38,8 @@ pub fn parse_outputs(code: &str) -> String {
|
||||
return serde_json::to_string(&r).unwrap();
|
||||
}
|
||||
|
||||
/// Parse TypeScript imports and return raw import strings.
|
||||
/// See [`parse_ts_relative_imports`] for resolved absolute paths.
|
||||
#[cfg(feature = "ts-parser")]
|
||||
#[wasm_bindgen]
|
||||
pub fn parse_ts_imports(code: &str) -> String {
|
||||
@@ -50,6 +52,15 @@ pub fn parse_ts_imports(code: &str) -> String {
|
||||
return serde_json::to_string(&r).unwrap();
|
||||
}
|
||||
|
||||
/// Parse TypeScript imports and return relative imports resolved to absolute Windmill paths.
|
||||
/// Throws JS error on parse failure.
|
||||
/// See [`parse_ts_imports`] for raw import strings.
|
||||
#[cfg(feature = "ts-parser")]
|
||||
#[wasm_bindgen]
|
||||
pub fn parse_ts_relative_imports(code: &str, path: &str) -> Result<Vec<String>, String> {
|
||||
windmill_parser_ts::parse_relative_imports(code, path).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
#[cfg(feature = "bash-parser")]
|
||||
#[wasm_bindgen]
|
||||
pub fn parse_bash(code: &str) -> String {
|
||||
@@ -214,6 +225,14 @@ pub fn parse_assets_py(code: &str) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse Python imports and return relative imports resolved to absolute Windmill paths.
|
||||
/// Throws JS error on parse failure.
|
||||
#[cfg(feature = "py-imports-parser")]
|
||||
#[wasm_bindgen]
|
||||
pub fn parse_py_relative_imports(code: &str, path: &str) -> Result<Vec<String>, String> {
|
||||
windmill_parser_py_imports::parse_relative_imports(code, path).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
#[cfg(feature = "ansible-parser")]
|
||||
#[wasm_bindgen]
|
||||
pub fn parse_assets_ansible(code: &str) -> String {
|
||||
|
||||
@@ -14,6 +14,23 @@ use serde_json::Value;
|
||||
|
||||
pub mod asset_parser;
|
||||
|
||||
/// S3 output format for SQL queries (moved here to avoid pulling sqlx into WASM via windmill-types)
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
pub enum S3ModeFormat {
|
||||
Json,
|
||||
Csv,
|
||||
Parquet,
|
||||
}
|
||||
|
||||
/// Returns the file extension for the given S3 mode format
|
||||
pub fn s3_mode_extension(format: S3ModeFormat) -> &'static str {
|
||||
match format {
|
||||
S3ModeFormat::Json => "json",
|
||||
S3ModeFormat::Csv => "csv",
|
||||
S3ModeFormat::Parquet => "parquet",
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Serialize, Debug, PartialEq, Default)]
|
||||
pub struct MainArgSignature {
|
||||
pub star_args: bool,
|
||||
|
||||
+8
-2
@@ -36,9 +36,10 @@ use windmill_common::ee_oss::{
|
||||
|
||||
use windmill_common::{
|
||||
agent_workers::AgentConfig,
|
||||
ai_cache::bump_instance_ai_config_revision,
|
||||
global_settings::{
|
||||
APP_WORKSPACED_ROUTE_SETTING, AUDIT_LOG_RETENTION_DAYS_SETTING, BASE_URL_SETTING,
|
||||
BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
|
||||
AI_CONFIG_SETTING, APP_WORKSPACED_ROUTE_SETTING, AUDIT_LOG_RETENTION_DAYS_SETTING,
|
||||
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
|
||||
CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
|
||||
CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
|
||||
DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
|
||||
@@ -312,6 +313,7 @@ async fn cache_hub_scripts(file_path: Option<String>) -> anyhow::Result<()> {
|
||||
"cache_init",
|
||||
"",
|
||||
&mut None,
|
||||
&None,
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -1812,6 +1814,10 @@ async fn process_notify_event(
|
||||
tracing::error!(error = %e, "Could not reload app workspaced route setting");
|
||||
}
|
||||
}
|
||||
AI_CONFIG_SETTING => {
|
||||
tracing::info!("AI config setting changed, bumping instance AI cache revision");
|
||||
bump_instance_ai_config_revision();
|
||||
}
|
||||
OTEL_SETTING => {
|
||||
tracing::info!("OTEL setting changed, restarting");
|
||||
send_delayed_killpill(tx, 4, "OTEL setting change").await;
|
||||
|
||||
@@ -98,7 +98,7 @@ use windmill_worker::{
|
||||
CARGO_REGISTRIES, INSTANCE_PYTHON_VERSION, JAVA_HOME_DIR, JOB_DEFAULT_TIMEOUT, JOB_ISOLATION,
|
||||
KEEP_JOB_DIR, MAVEN_REPOS, MAVEN_SETTINGS_XML, NO_DEFAULT_MAVEN, NPMRC, NPM_CONFIG_REGISTRY,
|
||||
NSJAIL_AVAILABLE, NUGET_CONFIG, OTEL_TRACING_PROXY_SETTINGS, PIP_EXTRA_INDEX_URL,
|
||||
PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, UV_INDEX_STRATEGY,
|
||||
PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, UNSHARE_PATH, UV_INDEX_STRATEGY,
|
||||
WORKSPACE_REGISTRIES,
|
||||
};
|
||||
|
||||
@@ -1701,6 +1701,12 @@ pub async fn reload_job_isolation_setting(conn: &Connection) {
|
||||
All jobs will fail until nsjail is installed or the setting is changed."
|
||||
);
|
||||
}
|
||||
if value == JobIsolationLevel::Unshare && UNSHARE_PATH.is_none() {
|
||||
tracing::error!(
|
||||
"job_isolation is set to unshare but the unshare binary is not available on this worker. \
|
||||
Jobs will run without isolation until unshare is installed or the setting is changed."
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn reload_request_size(conn: &Connection) {
|
||||
|
||||
+741
-30
@@ -891,25 +891,34 @@ mod dedicated_worker_protocol {
|
||||
use std::process::{Command, Stdio};
|
||||
use windmill_test_utils::{parse_dedicated_worker_line, DedicatedWorkerResult};
|
||||
use windmill_worker::{
|
||||
build_loader, generate_dedicated_worker_wrapper, LoaderMode, BUN_DEDICATED_WORKER_ARGS,
|
||||
BUN_PATH, NODE_BIN_PATH,
|
||||
build_loader, compute_ts_codegen, generate_multi_script_wrapper, LoaderMode, TsScriptEntry,
|
||||
BUN_DEDICATED_WORKER_ARGS, BUN_PATH, NODE_BIN_PATH,
|
||||
};
|
||||
|
||||
const TEST_SCRIPT_PATH: &str = "f/test/script";
|
||||
|
||||
/// Creates test worker files and optionally bundles for Node.js (like production)
|
||||
/// Returns the path to the wrapper file to execute
|
||||
fn create_test_worker_files(
|
||||
dir: &std::path::Path,
|
||||
script: &str,
|
||||
arg_names: &[&str],
|
||||
bundle_for_node: bool,
|
||||
) -> std::path::PathBuf {
|
||||
let dir_str = dir.to_str().unwrap();
|
||||
// Write main.ts at root (like production single-script)
|
||||
std::fs::write(dir.join("main.ts"), script).unwrap();
|
||||
|
||||
let codegen = compute_ts_codegen(script);
|
||||
let ext = if bundle_for_node { "js" } else { "ts" };
|
||||
let scripts = [TsScriptEntry {
|
||||
import_name: "main",
|
||||
original_path: TEST_SCRIPT_PATH,
|
||||
codegen: &codegen,
|
||||
}];
|
||||
let wrapper = generate_multi_script_wrapper(&scripts, ext);
|
||||
|
||||
if bundle_for_node {
|
||||
// For Node.js: bundle to JavaScript first (like production's build_loader with LoaderMode::Node)
|
||||
let wrapper = generate_dedicated_worker_wrapper(arg_names, "./main.js", None, None);
|
||||
std::fs::write(dir.join("wrapper.mjs"), wrapper).unwrap();
|
||||
std::fs::write(dir.join("wrapper.mjs"), &wrapper).unwrap();
|
||||
|
||||
// Use the exact same build_loader function as production
|
||||
tokio::runtime::Runtime::new()
|
||||
@@ -919,8 +928,9 @@ mod dedicated_worker_protocol {
|
||||
"http://localhost:8000",
|
||||
"test_token",
|
||||
"test-workspace",
|
||||
"f/test/script",
|
||||
TEST_SCRIPT_PATH,
|
||||
LoaderMode::Node,
|
||||
&None,
|
||||
))
|
||||
.expect("build_loader failed");
|
||||
|
||||
@@ -944,10 +954,8 @@ mod dedicated_worker_protocol {
|
||||
std::fs::rename(&bundled_path, &output_path).unwrap();
|
||||
output_path
|
||||
} else {
|
||||
// For Bun: use TypeScript directly (like production)
|
||||
let wrapper = generate_dedicated_worker_wrapper(arg_names, "./main.ts", None, None);
|
||||
let wrapper_path = dir.join("wrapper.mjs");
|
||||
std::fs::write(&wrapper_path, wrapper).unwrap();
|
||||
std::fs::write(&wrapper_path, &wrapper).unwrap();
|
||||
wrapper_path
|
||||
}
|
||||
}
|
||||
@@ -956,14 +964,12 @@ mod dedicated_worker_protocol {
|
||||
fn run_worker_test(
|
||||
runtime: &str,
|
||||
script: &str,
|
||||
arg_names: &[&str],
|
||||
jobs: Vec<serde_json::Value>,
|
||||
) -> Vec<Result<serde_json::Value, String>> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
|
||||
// Create files and get the wrapper path (bundled for node, raw for bun)
|
||||
let wrapper_path =
|
||||
create_test_worker_files(temp_dir.path(), script, arg_names, runtime == "node");
|
||||
let wrapper_path = create_test_worker_files(temp_dir.path(), script, runtime == "node");
|
||||
let wrapper_str = wrapper_path.to_str().unwrap();
|
||||
|
||||
// Build args matching production behavior
|
||||
@@ -1007,7 +1013,8 @@ mod dedicated_worker_protocol {
|
||||
let mut results = Vec::new();
|
||||
|
||||
for job_args in jobs {
|
||||
writeln!(stdin, "{}", job_args.to_string()).unwrap();
|
||||
// Protocol: exec:<script_path>:<json_args>
|
||||
writeln!(stdin, "exec:{}:{}", TEST_SCRIPT_PATH, job_args.to_string()).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
let mut response = String::new();
|
||||
@@ -1042,12 +1049,7 @@ export function main(x: number, y: number): number {
|
||||
return x + y;
|
||||
}
|
||||
"#;
|
||||
let results = run_worker_test(
|
||||
"node",
|
||||
script,
|
||||
&["x", "y"],
|
||||
vec![serde_json::json!({"x": 5, "y": 3})],
|
||||
);
|
||||
let results = run_worker_test("node", script, vec![serde_json::json!({"x": 5, "y": 3})]);
|
||||
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(8)));
|
||||
@@ -1061,7 +1063,7 @@ export function main(n: number): number {
|
||||
}
|
||||
"#;
|
||||
let jobs: Vec<serde_json::Value> = (1..=5).map(|i| serde_json::json!({"n": i})).collect();
|
||||
let results = run_worker_test("node", script, &["n"], jobs);
|
||||
let results = run_worker_test("node", script, jobs);
|
||||
|
||||
assert_eq!(results.len(), 5);
|
||||
for (i, result) in results.iter().enumerate() {
|
||||
@@ -1080,7 +1082,6 @@ export function main(msg: string): never {
|
||||
let results = run_worker_test(
|
||||
"node",
|
||||
script,
|
||||
&["msg"],
|
||||
vec![serde_json::json!({"msg": "test error"})],
|
||||
);
|
||||
|
||||
@@ -1098,12 +1099,7 @@ export function main(x: number, y: number): number {
|
||||
return x + y;
|
||||
}
|
||||
"#;
|
||||
let results = run_worker_test(
|
||||
"bun",
|
||||
script,
|
||||
&["x", "y"],
|
||||
vec![serde_json::json!({"x": 5, "y": 3})],
|
||||
);
|
||||
let results = run_worker_test("bun", script, vec![serde_json::json!({"x": 5, "y": 3})]);
|
||||
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(8)));
|
||||
@@ -1117,7 +1113,7 @@ export function main(n: number): number {
|
||||
}
|
||||
"#;
|
||||
let jobs: Vec<serde_json::Value> = (1..=5).map(|i| serde_json::json!({"n": i})).collect();
|
||||
let results = run_worker_test("bun", script, &["n"], jobs);
|
||||
let results = run_worker_test("bun", script, jobs);
|
||||
|
||||
assert_eq!(results.len(), 5);
|
||||
for (i, result) in results.iter().enumerate() {
|
||||
@@ -1136,7 +1132,6 @@ export function main(msg: string): never {
|
||||
let results = run_worker_test(
|
||||
"bun",
|
||||
script,
|
||||
&["msg"],
|
||||
vec![serde_json::json!({"msg": "test error"})],
|
||||
);
|
||||
|
||||
@@ -1144,6 +1139,721 @@ export function main(msg: string): never {
|
||||
assert!(results[0].is_err());
|
||||
assert_eq!(results[0], Err("test error".to_string()));
|
||||
}
|
||||
|
||||
// ==================== Multi-Script (Runner Group) Tests ====================
|
||||
|
||||
/// Job to send to a specific script in a multi-script wrapper
|
||||
struct MultiScriptJob {
|
||||
script_path: String,
|
||||
args: serde_json::Value,
|
||||
}
|
||||
|
||||
/// Creates a multi-script wrapper with multiple scripts as flat files, returns the wrapper path
|
||||
fn create_multi_script_worker_files(
|
||||
dir: &std::path::Path,
|
||||
scripts: &[(&str, &str)], // (original_path, script_content)
|
||||
) -> std::path::PathBuf {
|
||||
let mut entries_data = Vec::new();
|
||||
for (path, content) in scripts {
|
||||
let safe_name = format!("_wm_{}", path.replace('/', "__"));
|
||||
std::fs::write(dir.join(format!("{safe_name}.ts")), content).unwrap();
|
||||
entries_data.push((safe_name, path.to_string(), compute_ts_codegen(content)));
|
||||
}
|
||||
|
||||
let entries: Vec<TsScriptEntry<'_>> = entries_data
|
||||
.iter()
|
||||
.map(|(safe, path, cg)| TsScriptEntry {
|
||||
import_name: safe.as_str(),
|
||||
original_path: path.as_str(),
|
||||
codegen: cg,
|
||||
})
|
||||
.collect();
|
||||
|
||||
let wrapper = generate_multi_script_wrapper(&entries, "ts");
|
||||
let wrapper_path = dir.join("wrapper.mjs");
|
||||
std::fs::write(&wrapper_path, &wrapper).unwrap();
|
||||
wrapper_path
|
||||
}
|
||||
|
||||
/// Helper to run a multi-script dedicated worker test
|
||||
fn run_multi_script_worker_test(
|
||||
scripts: &[(&str, &str)],
|
||||
jobs: Vec<MultiScriptJob>,
|
||||
) -> Vec<Result<serde_json::Value, String>> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
let wrapper_path = create_multi_script_worker_files(temp_dir.path(), scripts);
|
||||
let wrapper_str = wrapper_path.to_str().unwrap();
|
||||
|
||||
let mut cmd_args: Vec<&str> = BUN_DEDICATED_WORKER_ARGS.to_vec();
|
||||
cmd_args.push(wrapper_str);
|
||||
|
||||
let mut child = Command::new(BUN_PATH.as_str())
|
||||
.args(cmd_args)
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.current_dir(temp_dir.path())
|
||||
.spawn()
|
||||
.expect("Failed to spawn worker process");
|
||||
|
||||
let mut stdin = child.stdin.take().unwrap();
|
||||
let stdout = child.stdout.take().unwrap();
|
||||
let mut reader = BufReader::new(stdout);
|
||||
|
||||
// Wait for "start" signal
|
||||
let mut start_line = String::new();
|
||||
reader.read_line(&mut start_line).unwrap();
|
||||
assert_eq!(
|
||||
parse_dedicated_worker_line(start_line.trim()),
|
||||
DedicatedWorkerResult::Start,
|
||||
"Expected 'start', got: {}",
|
||||
start_line.trim()
|
||||
);
|
||||
|
||||
let mut results = Vec::new();
|
||||
|
||||
for job in &jobs {
|
||||
writeln!(stdin, "exec:{}:{}", job.script_path, job.args.to_string()).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
let mut response = String::new();
|
||||
reader.read_line(&mut response).unwrap();
|
||||
|
||||
match parse_dedicated_worker_line(response.trim()) {
|
||||
DedicatedWorkerResult::Success(value) => results.push(Ok(value)),
|
||||
DedicatedWorkerResult::Error(err) => {
|
||||
let msg = err["message"]
|
||||
.as_str()
|
||||
.unwrap_or("Unknown error")
|
||||
.to_string();
|
||||
results.push(Err(msg));
|
||||
}
|
||||
other => panic!("Unexpected response: {:?}", other),
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(stdin, "end").unwrap();
|
||||
stdin.flush().unwrap();
|
||||
let _ = child.wait().expect("Worker process failed to exit");
|
||||
|
||||
results
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_multi_script_routing_basic() {
|
||||
let script_add = r#"
|
||||
export function main(a: number, b: number): number {
|
||||
return a + b;
|
||||
}
|
||||
"#;
|
||||
let script_mul = r#"
|
||||
export function main(x: number, y: number): number {
|
||||
return x * y;
|
||||
}
|
||||
"#;
|
||||
let results = run_multi_script_worker_test(
|
||||
&[("f/math/add", script_add), ("f/math/mul", script_mul)],
|
||||
vec![
|
||||
MultiScriptJob {
|
||||
script_path: "f/math/add".to_string(),
|
||||
args: serde_json::json!({"a": 3, "b": 4}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/math/mul".to_string(),
|
||||
args: serde_json::json!({"x": 5, "y": 6}),
|
||||
},
|
||||
// Route back to add
|
||||
MultiScriptJob {
|
||||
script_path: "f/math/add".to_string(),
|
||||
args: serde_json::json!({"a": 10, "b": 20}),
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(7))); // 3 + 4
|
||||
assert_eq!(results[1], Ok(serde_json::json!(30))); // 5 * 6
|
||||
assert_eq!(results[2], Ok(serde_json::json!(30))); // 10 + 20
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_multi_script_interleaved_jobs() {
|
||||
let script_upper = r#"
|
||||
export function main(s: string): string {
|
||||
return s.toUpperCase();
|
||||
}
|
||||
"#;
|
||||
let script_len = r#"
|
||||
export function main(s: string): number {
|
||||
return s.length;
|
||||
}
|
||||
"#;
|
||||
let results = run_multi_script_worker_test(
|
||||
&[("f/str/upper", script_upper), ("f/str/len", script_len)],
|
||||
vec![
|
||||
MultiScriptJob {
|
||||
script_path: "f/str/upper".to_string(),
|
||||
args: serde_json::json!({"s": "hello"}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/str/len".to_string(),
|
||||
args: serde_json::json!({"s": "hello"}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/str/upper".to_string(),
|
||||
args: serde_json::json!({"s": "world"}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/str/len".to_string(),
|
||||
args: serde_json::json!({"s": "ab"}),
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
assert_eq!(results.len(), 4);
|
||||
assert_eq!(results[0], Ok(serde_json::json!("HELLO")));
|
||||
assert_eq!(results[1], Ok(serde_json::json!(5)));
|
||||
assert_eq!(results[2], Ok(serde_json::json!("WORLD")));
|
||||
assert_eq!(results[3], Ok(serde_json::json!(2)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_multi_script_unknown_path_error() {
|
||||
let script = r#"
|
||||
export function main(x: number): number {
|
||||
return x;
|
||||
}
|
||||
"#;
|
||||
let results = run_multi_script_worker_test(
|
||||
&[("f/known", script)],
|
||||
vec![MultiScriptJob {
|
||||
script_path: "f/unknown".to_string(),
|
||||
args: serde_json::json!({"x": 1}),
|
||||
}],
|
||||
);
|
||||
|
||||
assert_eq!(results.len(), 1);
|
||||
assert!(results[0].is_err());
|
||||
assert!(results[0]
|
||||
.as_ref()
|
||||
.unwrap_err()
|
||||
.contains("Script not found"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_multi_script_error_doesnt_break_other_scripts() {
|
||||
let script_ok = r#"
|
||||
export function main(x: number): number {
|
||||
return x * 2;
|
||||
}
|
||||
"#;
|
||||
let script_err = r#"
|
||||
export function main(msg: string): never {
|
||||
throw new Error(msg);
|
||||
}
|
||||
"#;
|
||||
let results = run_multi_script_worker_test(
|
||||
&[("f/ok", script_ok), ("f/err", script_err)],
|
||||
vec![
|
||||
MultiScriptJob {
|
||||
script_path: "f/ok".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/err".to_string(),
|
||||
args: serde_json::json!({"msg": "boom"}),
|
||||
},
|
||||
// Should still work after error in other script
|
||||
MultiScriptJob {
|
||||
script_path: "f/ok".to_string(),
|
||||
args: serde_json::json!({"x": 10}),
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(10)));
|
||||
assert!(results[1].is_err());
|
||||
assert_eq!(results[1], Err("boom".to_string()));
|
||||
assert_eq!(results[2], Ok(serde_json::json!(20)));
|
||||
}
|
||||
|
||||
// ==================== exec_preprocess Tests ====================
|
||||
|
||||
/// Raw protocol command to send to a dedicated worker
|
||||
enum ProtocolCmd {
|
||||
Exec { path: String, args: serde_json::Value },
|
||||
ExecPreprocess { path: String, args: serde_json::Value },
|
||||
}
|
||||
|
||||
/// Run a multi-script worker test with raw protocol commands, returning all protocol lines
|
||||
fn run_raw_protocol_test(
|
||||
scripts: &[(&str, &str)],
|
||||
commands: Vec<ProtocolCmd>,
|
||||
) -> Vec<DedicatedWorkerResult> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
let wrapper_path = create_multi_script_worker_files(temp_dir.path(), scripts);
|
||||
let wrapper_str = wrapper_path.to_str().unwrap();
|
||||
|
||||
let mut cmd_args: Vec<&str> = BUN_DEDICATED_WORKER_ARGS.to_vec();
|
||||
cmd_args.push(wrapper_str);
|
||||
|
||||
let mut child = Command::new(BUN_PATH.as_str())
|
||||
.args(cmd_args)
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.current_dir(temp_dir.path())
|
||||
.spawn()
|
||||
.expect("Failed to spawn worker process");
|
||||
|
||||
let mut stdin = child.stdin.take().unwrap();
|
||||
let stdout = child.stdout.take().unwrap();
|
||||
let mut reader = BufReader::new(stdout);
|
||||
|
||||
let mut start_line = String::new();
|
||||
reader.read_line(&mut start_line).unwrap();
|
||||
assert_eq!(
|
||||
parse_dedicated_worker_line(start_line.trim()),
|
||||
DedicatedWorkerResult::Start,
|
||||
);
|
||||
|
||||
let mut results = Vec::new();
|
||||
|
||||
for cmd in &commands {
|
||||
let line = match cmd {
|
||||
ProtocolCmd::Exec { path, args } => format!("exec:{}:{}", path, args),
|
||||
ProtocolCmd::ExecPreprocess { path, args } => {
|
||||
format!("exec_preprocess:{}:{}", path, args)
|
||||
}
|
||||
};
|
||||
writeln!(stdin, "{}", line).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
// exec_preprocess produces 2 response lines (preprocessed_args + success/error)
|
||||
// exec produces 1 response line (success/error)
|
||||
let expected_lines = match cmd {
|
||||
ProtocolCmd::ExecPreprocess { .. } => 2,
|
||||
ProtocolCmd::Exec { .. } => 1,
|
||||
};
|
||||
|
||||
for _ in 0..expected_lines {
|
||||
let mut response = String::new();
|
||||
reader.read_line(&mut response).unwrap();
|
||||
let parsed = parse_dedicated_worker_line(response.trim());
|
||||
// If it's an error, stop reading more lines for this command
|
||||
if matches!(parsed, DedicatedWorkerResult::Error(_)) {
|
||||
results.push(parsed);
|
||||
break;
|
||||
}
|
||||
results.push(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(stdin, "end").unwrap();
|
||||
stdin.flush().unwrap();
|
||||
let _ = child.wait().expect("Worker process failed to exit");
|
||||
|
||||
results
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bun_exec_preprocess() {
|
||||
let script = r#"
|
||||
export function preprocessor(x: number) {
|
||||
return { x: x * 10 };
|
||||
}
|
||||
export function main(x: number): number {
|
||||
return x + 1;
|
||||
}
|
||||
"#;
|
||||
let results = run_raw_protocol_test(
|
||||
&[("f/test/pre", script)],
|
||||
vec![ProtocolCmd::ExecPreprocess {
|
||||
path: "f/test/pre".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
}],
|
||||
);
|
||||
// Should get preprocessed_args then success
|
||||
assert_eq!(results.len(), 2);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
DedicatedWorkerResult::PreprocessedArgs(serde_json::json!({"x": 50}))
|
||||
);
|
||||
// main(50) => 51
|
||||
assert_eq!(
|
||||
results[1],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(51))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bun_exec_preprocess_missing_preprocessor() {
|
||||
let script = r#"
|
||||
export function main(x: number): number {
|
||||
return x;
|
||||
}
|
||||
"#;
|
||||
let results = run_raw_protocol_test(
|
||||
&[("f/test/nopre", script)],
|
||||
vec![ProtocolCmd::ExecPreprocess {
|
||||
path: "f/test/nopre".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
}],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert!(matches!(results[0], DedicatedWorkerResult::Error(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bun_exec_preprocess_then_exec() {
|
||||
let script = r#"
|
||||
export function preprocessor(x: number) {
|
||||
return { x: x * 2 };
|
||||
}
|
||||
export function main(x: number): number {
|
||||
return x + 100;
|
||||
}
|
||||
"#;
|
||||
let results = run_raw_protocol_test(
|
||||
&[("f/test/mixed", script)],
|
||||
vec![
|
||||
ProtocolCmd::ExecPreprocess {
|
||||
path: "f/test/mixed".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
},
|
||||
ProtocolCmd::Exec {
|
||||
path: "f/test/mixed".to_string(),
|
||||
args: serde_json::json!({"x": 7}),
|
||||
},
|
||||
],
|
||||
);
|
||||
// preprocess: preprocessor(5) => {"x":10}, main(10) => 110
|
||||
// exec: main(7) => 107
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
DedicatedWorkerResult::PreprocessedArgs(serde_json::json!({"x": 10}))
|
||||
);
|
||||
assert_eq!(
|
||||
results[1],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(110))
|
||||
);
|
||||
assert_eq!(
|
||||
results[2],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(107))
|
||||
);
|
||||
}
|
||||
|
||||
// ==================== Argument Transformation Tests ====================
|
||||
|
||||
#[test]
|
||||
fn test_bun_date_arg_transformation() {
|
||||
let script = r#"
|
||||
export function main(d: Date): string {
|
||||
return d instanceof Date ? d.toISOString() : typeof d;
|
||||
}
|
||||
"#;
|
||||
let results = run_worker_test(
|
||||
"bun",
|
||||
script,
|
||||
vec![serde_json::json!({"d": "2024-01-15T10:30:00.000Z"})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
Ok(serde_json::json!("2024-01-15T10:30:00.000Z"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bun_null_and_undefined_args() {
|
||||
let script = r#"
|
||||
export function main(x?: number): string {
|
||||
return x === null ? "null" : x === undefined ? "undefined" : String(x);
|
||||
}
|
||||
"#;
|
||||
let results = run_worker_test(
|
||||
"bun",
|
||||
script,
|
||||
vec![
|
||||
serde_json::json!({"x": null}),
|
||||
serde_json::json!({"x": 42}),
|
||||
serde_json::json!({}),
|
||||
],
|
||||
);
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(results[0], Ok(serde_json::json!("null")));
|
||||
assert_eq!(results[1], Ok(serde_json::json!("42")));
|
||||
// Missing arg should be undefined
|
||||
assert_eq!(results[2], Ok(serde_json::json!("undefined")));
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Deno Dedicated Worker Protocol Tests
|
||||
// ============================================================================
|
||||
|
||||
mod dedicated_worker_protocol_deno {
|
||||
use std::io::{BufRead, BufReader, Write};
|
||||
use std::process::{Command, Stdio};
|
||||
use windmill_test_utils::{parse_dedicated_worker_line, DedicatedWorkerResult};
|
||||
use windmill_worker::{generate_deno_dedicated_worker_wrapper, DENO_PATH};
|
||||
|
||||
const TEST_SCRIPT_PATH: &str = "f/test/script";
|
||||
|
||||
fn run_deno_worker_test(
|
||||
script: &str,
|
||||
jobs: Vec<serde_json::Value>,
|
||||
) -> Vec<Result<serde_json::Value, String>> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
std::fs::write(temp_dir.path().join("main.ts"), script).unwrap();
|
||||
|
||||
let wrapper = generate_deno_dedicated_worker_wrapper(script).unwrap();
|
||||
std::fs::write(temp_dir.path().join("wrapper.ts"), &wrapper).unwrap();
|
||||
|
||||
let mut child = Command::new(DENO_PATH.as_str())
|
||||
.args([
|
||||
"run",
|
||||
"--no-check",
|
||||
"--unstable-unsafe-proto",
|
||||
"--unstable-bare-node-builtins",
|
||||
"-A",
|
||||
"wrapper.ts",
|
||||
])
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.current_dir(temp_dir.path())
|
||||
.spawn()
|
||||
.expect("Failed to spawn deno process");
|
||||
|
||||
let mut stdin = child.stdin.take().unwrap();
|
||||
let stdout = child.stdout.take().unwrap();
|
||||
let mut reader = BufReader::new(stdout);
|
||||
|
||||
// Wait for "start" — deno outputs 'start\n' via console.log which adds
|
||||
// its own newline, producing double newlines. Skip empty lines.
|
||||
loop {
|
||||
let mut line = String::new();
|
||||
reader.read_line(&mut line).unwrap();
|
||||
if line.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
assert_eq!(
|
||||
parse_dedicated_worker_line(line.trim()),
|
||||
DedicatedWorkerResult::Start,
|
||||
"Expected 'start', got: {}",
|
||||
line.trim()
|
||||
);
|
||||
break;
|
||||
}
|
||||
|
||||
let mut results = Vec::new();
|
||||
for job_args in jobs {
|
||||
writeln!(stdin, "exec:{}:{}", TEST_SCRIPT_PATH, job_args.to_string()).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
loop {
|
||||
let mut response = String::new();
|
||||
reader.read_line(&mut response).unwrap();
|
||||
let trimmed = response.trim();
|
||||
if trimmed.is_empty() {
|
||||
continue;
|
||||
}
|
||||
match parse_dedicated_worker_line(trimmed) {
|
||||
DedicatedWorkerResult::Success(value) => results.push(Ok(value)),
|
||||
DedicatedWorkerResult::Error(err) => {
|
||||
let msg = err["message"]
|
||||
.as_str()
|
||||
.unwrap_or("Unknown error")
|
||||
.to_string();
|
||||
results.push(Err(msg));
|
||||
}
|
||||
other => panic!("Unexpected response: {:?}", other),
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(stdin, "end").unwrap();
|
||||
stdin.flush().unwrap();
|
||||
let _ = child.wait().expect("Worker process failed to exit");
|
||||
results
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deno_dedicated_worker_simple() {
|
||||
let script = r#"
|
||||
export function main(x: number, y: number): number {
|
||||
return x + y;
|
||||
}
|
||||
"#;
|
||||
let results = run_deno_worker_test(script, vec![serde_json::json!({"x": 5, "y": 3})]);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(8)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deno_dedicated_worker_multiple_jobs() {
|
||||
let script = r#"
|
||||
export function main(n: number): number {
|
||||
return n * 2;
|
||||
}
|
||||
"#;
|
||||
let jobs: Vec<serde_json::Value> = (1..=5).map(|i| serde_json::json!({"n": i})).collect();
|
||||
let results = run_deno_worker_test(script, jobs);
|
||||
assert_eq!(results.len(), 5);
|
||||
for (i, result) in results.iter().enumerate() {
|
||||
assert_eq!(*result, Ok(serde_json::json!(((i + 1) * 2) as i64)));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deno_dedicated_worker_error() {
|
||||
let script = r#"
|
||||
export function main(msg: string): never {
|
||||
throw new Error(msg);
|
||||
}
|
||||
"#;
|
||||
let results = run_deno_worker_test(script, vec![serde_json::json!({"msg": "test error"})]);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert!(results[0].is_err());
|
||||
assert_eq!(results[0], Err("test error".to_string()));
|
||||
}
|
||||
|
||||
// ==================== exec_preprocess Tests ====================
|
||||
|
||||
/// Run a raw deno protocol test, reading all output lines per command
|
||||
fn run_deno_raw_protocol_test(
|
||||
script: &str,
|
||||
commands: Vec<(&str, serde_json::Value)>, // ("exec" or "exec_preprocess", args)
|
||||
) -> Vec<DedicatedWorkerResult> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
std::fs::write(temp_dir.path().join("main.ts"), script).unwrap();
|
||||
|
||||
let wrapper = generate_deno_dedicated_worker_wrapper(script).unwrap();
|
||||
std::fs::write(temp_dir.path().join("wrapper.ts"), &wrapper).unwrap();
|
||||
|
||||
let mut child = Command::new(DENO_PATH.as_str())
|
||||
.args([
|
||||
"run",
|
||||
"--no-check",
|
||||
"--unstable-unsafe-proto",
|
||||
"--unstable-bare-node-builtins",
|
||||
"-A",
|
||||
"wrapper.ts",
|
||||
])
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.current_dir(temp_dir.path())
|
||||
.spawn()
|
||||
.expect("Failed to spawn deno process");
|
||||
|
||||
let mut stdin = child.stdin.take().unwrap();
|
||||
let stdout = child.stdout.take().unwrap();
|
||||
let mut reader = BufReader::new(stdout);
|
||||
|
||||
// Wait for start, skip empty lines
|
||||
loop {
|
||||
let mut line = String::new();
|
||||
reader.read_line(&mut line).unwrap();
|
||||
if line.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
assert_eq!(
|
||||
parse_dedicated_worker_line(line.trim()),
|
||||
DedicatedWorkerResult::Start,
|
||||
);
|
||||
break;
|
||||
}
|
||||
|
||||
let mut results = Vec::new();
|
||||
|
||||
for (cmd, args) in &commands {
|
||||
writeln!(stdin, "{}:{}:{}", cmd, TEST_SCRIPT_PATH, args).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
let expected_lines = if *cmd == "exec_preprocess" { 2 } else { 1 };
|
||||
|
||||
for _ in 0..expected_lines {
|
||||
loop {
|
||||
let mut response = String::new();
|
||||
reader.read_line(&mut response).unwrap();
|
||||
if response.trim().is_empty() {
|
||||
continue;
|
||||
}
|
||||
let parsed = parse_dedicated_worker_line(response.trim());
|
||||
if matches!(parsed, DedicatedWorkerResult::Error(_)) {
|
||||
results.push(parsed);
|
||||
break;
|
||||
}
|
||||
results.push(parsed);
|
||||
break;
|
||||
}
|
||||
// If last result was an error, don't read more lines for this command
|
||||
if matches!(results.last(), Some(DedicatedWorkerResult::Error(_))) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(stdin, "end").unwrap();
|
||||
stdin.flush().unwrap();
|
||||
let _ = child.wait().expect("Worker process failed to exit");
|
||||
|
||||
results
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_deno_exec_preprocess() {
|
||||
let script = r#"
|
||||
export function preprocessor(x: number) {
|
||||
return { x: x * 10 };
|
||||
}
|
||||
export function main(x: number): number {
|
||||
return x + 1;
|
||||
}
|
||||
"#;
|
||||
let results = run_deno_raw_protocol_test(
|
||||
script,
|
||||
vec![("exec_preprocess", serde_json::json!({"x": 5}))],
|
||||
);
|
||||
assert_eq!(results.len(), 2);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
DedicatedWorkerResult::PreprocessedArgs(serde_json::json!({"x": 50}))
|
||||
);
|
||||
assert_eq!(
|
||||
results[1],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(51))
|
||||
);
|
||||
}
|
||||
|
||||
// Note: no "missing preprocessor" test for Deno because the wrapper only generates
|
||||
// the exec_preprocess handler when the script actually has a preprocessor function.
|
||||
// Without one, exec_preprocess messages are unrecognized (by design — Rust never sends them).
|
||||
|
||||
// ==================== Argument Transformation Tests ====================
|
||||
|
||||
#[test]
|
||||
fn test_deno_date_arg_transformation() {
|
||||
let script = r#"
|
||||
export function main(d: Date): string {
|
||||
return d instanceof Date ? d.toISOString() : typeof d;
|
||||
}
|
||||
"#;
|
||||
let results = run_deno_worker_test(
|
||||
script,
|
||||
vec![serde_json::json!({"d": "2024-01-15T10:30:00.000Z"})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
Ok(serde_json::json!("2024-01-15T10:30:00.000Z"))
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
@@ -1299,6 +2009,7 @@ mod bun_builder_tests {
|
||||
// Write build.js using the loader and builder constants directly
|
||||
// Parameters are dummy values since tests don't use Windmill relative imports
|
||||
let loader = RELATIVE_BUN_LOADER
|
||||
.replace("TEMP_SCRIPT_REFS_PLACEHOLDER", "{}")
|
||||
.replace("W_ID", "test-workspace")
|
||||
.replace("BASE_INTERNAL_URL", "http://localhost:8000")
|
||||
.replace("TOKEN", "test-token")
|
||||
|
||||
@@ -48,6 +48,7 @@ fn flow_module(id: &str, value: FlowModuleValue) -> FlowModule {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -32,6 +32,7 @@ mod prewarmed_isolate_tests {
|
||||
"test-workspace",
|
||||
"f/test/script",
|
||||
LoaderMode::BrowserBundle,
|
||||
&None,
|
||||
)
|
||||
.await
|
||||
.expect("build_loader failed");
|
||||
|
||||
@@ -1,9 +1,470 @@
|
||||
use serde_json::json;
|
||||
#[cfg(feature = "python")]
|
||||
use sqlx::postgres::Postgres;
|
||||
#[cfg(feature = "python")]
|
||||
use sqlx::Pool;
|
||||
#[cfg(feature = "python")]
|
||||
use windmill_common::scripts::ScriptLang;
|
||||
use windmill_test_utils::*;
|
||||
|
||||
// ============================================================================
|
||||
// Dedicated Worker Protocol Tests (Python)
|
||||
// ============================================================================
|
||||
|
||||
#[cfg(feature = "python")]
|
||||
mod dedicated_worker_protocol_python {
|
||||
use std::io::{BufRead, BufReader, Write};
|
||||
use std::process::{Command, Stdio};
|
||||
use windmill_test_utils::{parse_dedicated_worker_line, DedicatedWorkerResult};
|
||||
use windmill_worker::{compute_py_codegen, generate_py_multi_script_wrapper, PyScriptEntry};
|
||||
|
||||
struct MultiScriptJob {
|
||||
script_path: String,
|
||||
args: serde_json::Value,
|
||||
}
|
||||
|
||||
/// Creates a multi-script Python wrapper, writes scripts to proper module paths
|
||||
fn create_py_worker_files(
|
||||
dir: &std::path::Path,
|
||||
scripts: &[(&str, &str)], // (original_path, content)
|
||||
) -> std::path::PathBuf {
|
||||
let mut codegens = Vec::new();
|
||||
for (path, content) in scripts {
|
||||
let cg = compute_py_codegen(content, path);
|
||||
let module_dir = dir.join(&cg.dirs);
|
||||
std::fs::create_dir_all(&module_dir).unwrap();
|
||||
std::fs::write(module_dir.join(format!("{}.py", cg.module_name)), content).unwrap();
|
||||
codegens.push((path.to_string(), cg));
|
||||
}
|
||||
|
||||
let entries: Vec<PyScriptEntry<'_>> = codegens
|
||||
.iter()
|
||||
.map(|(path, cg)| PyScriptEntry { original_path: path.as_str(), codegen: cg })
|
||||
.collect();
|
||||
|
||||
let wrapper = generate_py_multi_script_wrapper(&entries, false, false);
|
||||
let wrapper_path = dir.join("wrapper.py");
|
||||
std::fs::write(&wrapper_path, &wrapper).unwrap();
|
||||
wrapper_path
|
||||
}
|
||||
|
||||
fn run_py_multi_script_test(
|
||||
scripts: &[(&str, &str)],
|
||||
jobs: Vec<MultiScriptJob>,
|
||||
) -> Vec<Result<serde_json::Value, String>> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
create_py_worker_files(temp_dir.path(), scripts);
|
||||
|
||||
let mut child = Command::new("python3")
|
||||
.args(["-u", "-m", "wrapper"])
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.current_dir(temp_dir.path())
|
||||
.spawn()
|
||||
.expect("Failed to spawn python3 process");
|
||||
|
||||
let mut stdin = child.stdin.take().unwrap();
|
||||
let stdout = child.stdout.take().unwrap();
|
||||
let mut reader = BufReader::new(stdout);
|
||||
|
||||
let mut start_line = String::new();
|
||||
reader.read_line(&mut start_line).unwrap();
|
||||
assert_eq!(
|
||||
parse_dedicated_worker_line(start_line.trim()),
|
||||
DedicatedWorkerResult::Start,
|
||||
"Expected 'start', got: {}",
|
||||
start_line.trim()
|
||||
);
|
||||
|
||||
let mut results = Vec::new();
|
||||
for job in &jobs {
|
||||
writeln!(stdin, "exec:{}:{}", job.script_path, job.args.to_string()).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
let mut response = String::new();
|
||||
reader.read_line(&mut response).unwrap();
|
||||
|
||||
match parse_dedicated_worker_line(response.trim()) {
|
||||
DedicatedWorkerResult::Success(value) => results.push(Ok(value)),
|
||||
DedicatedWorkerResult::Error(err) => {
|
||||
let msg = err["message"]
|
||||
.as_str()
|
||||
.unwrap_or("Unknown error")
|
||||
.to_string();
|
||||
results.push(Err(msg));
|
||||
}
|
||||
other => panic!("Unexpected response: {:?}", other),
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(stdin, "end").unwrap();
|
||||
stdin.flush().unwrap();
|
||||
let _ = child.wait().expect("Worker process failed to exit");
|
||||
results
|
||||
}
|
||||
|
||||
fn run_py_single_script_test(
|
||||
script_path: &str,
|
||||
content: &str,
|
||||
jobs: Vec<serde_json::Value>,
|
||||
) -> Vec<Result<serde_json::Value, String>> {
|
||||
run_py_multi_script_test(
|
||||
&[(script_path, content)],
|
||||
jobs.into_iter()
|
||||
.map(|args| MultiScriptJob { script_path: script_path.to_string(), args })
|
||||
.collect(),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_dedicated_worker_simple() {
|
||||
let results = run_py_single_script_test(
|
||||
"f/test/add",
|
||||
"def main(a: int, b: int):\n return a + b\n",
|
||||
vec![serde_json::json!({"a": 3, "b": 4})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(7)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_dedicated_worker_multiple_jobs() {
|
||||
let results = run_py_single_script_test(
|
||||
"f/test/double",
|
||||
"def main(n: int):\n return n * 2\n",
|
||||
(1..=5).map(|i| serde_json::json!({"n": i})).collect(),
|
||||
);
|
||||
assert_eq!(results.len(), 5);
|
||||
for (i, result) in results.iter().enumerate() {
|
||||
assert_eq!(*result, Ok(serde_json::json!(((i + 1) * 2) as i64)));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_multi_script_routing() {
|
||||
let results = run_py_multi_script_test(
|
||||
&[
|
||||
(
|
||||
"f/math/add",
|
||||
"def main(a: int, b: int):\n return a + b\n",
|
||||
),
|
||||
(
|
||||
"f/math/mul",
|
||||
"def main(x: int, y: int):\n return x * y\n",
|
||||
),
|
||||
],
|
||||
vec![
|
||||
MultiScriptJob {
|
||||
script_path: "f/math/add".to_string(),
|
||||
args: serde_json::json!({"a": 3, "b": 4}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/math/mul".to_string(),
|
||||
args: serde_json::json!({"x": 5, "y": 6}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/math/add".to_string(),
|
||||
args: serde_json::json!({"a": 10, "b": 20}),
|
||||
},
|
||||
],
|
||||
);
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(7)));
|
||||
assert_eq!(results[1], Ok(serde_json::json!(30)));
|
||||
assert_eq!(results[2], Ok(serde_json::json!(30)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_multi_script_error_isolation() {
|
||||
let results = run_py_multi_script_test(
|
||||
&[
|
||||
("f/ok", "def main(x: int):\n return x * 2\n"),
|
||||
("f/err", "def main(msg: str):\n raise Exception(msg)\n"),
|
||||
],
|
||||
vec![
|
||||
MultiScriptJob {
|
||||
script_path: "f/ok".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/err".to_string(),
|
||||
args: serde_json::json!({"msg": "boom"}),
|
||||
},
|
||||
MultiScriptJob {
|
||||
script_path: "f/ok".to_string(),
|
||||
args: serde_json::json!({"x": 10}),
|
||||
},
|
||||
],
|
||||
);
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(10)));
|
||||
assert!(results[1].is_err());
|
||||
assert_eq!(results[1], Err("boom".to_string()));
|
||||
assert_eq!(results[2], Ok(serde_json::json!(20)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_multi_script_unknown_path() {
|
||||
let results = run_py_multi_script_test(
|
||||
&[("f/known", "def main(x: int):\n return x\n")],
|
||||
vec![MultiScriptJob {
|
||||
script_path: "f/unknown".to_string(),
|
||||
args: serde_json::json!({"x": 1}),
|
||||
}],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert!(results[0].is_err());
|
||||
assert!(results[0]
|
||||
.as_ref()
|
||||
.unwrap_err()
|
||||
.contains("Script not found"));
|
||||
}
|
||||
|
||||
// ==================== exec_preprocess Tests ====================
|
||||
|
||||
/// Raw protocol command for Python
|
||||
enum ProtocolCmd {
|
||||
Exec { path: String, args: serde_json::Value },
|
||||
ExecPreprocess { path: String, args: serde_json::Value },
|
||||
}
|
||||
|
||||
/// Run a Python worker test with raw protocol commands
|
||||
fn run_py_raw_protocol_test(
|
||||
scripts: &[(&str, &str)],
|
||||
commands: Vec<ProtocolCmd>,
|
||||
) -> Vec<DedicatedWorkerResult> {
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
create_py_worker_files(temp_dir.path(), scripts);
|
||||
|
||||
let mut child = Command::new("python3")
|
||||
.args(["-u", "-m", "wrapper"])
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::piped())
|
||||
.current_dir(temp_dir.path())
|
||||
.spawn()
|
||||
.expect("Failed to spawn python3 process");
|
||||
|
||||
let mut stdin = child.stdin.take().unwrap();
|
||||
let stdout = child.stdout.take().unwrap();
|
||||
let mut reader = BufReader::new(stdout);
|
||||
|
||||
let mut start_line = String::new();
|
||||
reader.read_line(&mut start_line).unwrap();
|
||||
assert_eq!(
|
||||
parse_dedicated_worker_line(start_line.trim()),
|
||||
DedicatedWorkerResult::Start,
|
||||
);
|
||||
|
||||
let mut results = Vec::new();
|
||||
|
||||
for cmd in &commands {
|
||||
let line = match cmd {
|
||||
ProtocolCmd::Exec { path, args } => format!("exec:{}:{}", path, args),
|
||||
ProtocolCmd::ExecPreprocess { path, args } => {
|
||||
format!("exec_preprocess:{}:{}", path, args)
|
||||
}
|
||||
};
|
||||
writeln!(stdin, "{}", line).unwrap();
|
||||
stdin.flush().unwrap();
|
||||
|
||||
let expected_lines = match cmd {
|
||||
ProtocolCmd::ExecPreprocess { .. } => 2,
|
||||
ProtocolCmd::Exec { .. } => 1,
|
||||
};
|
||||
|
||||
for _ in 0..expected_lines {
|
||||
let mut response = String::new();
|
||||
reader.read_line(&mut response).unwrap();
|
||||
let parsed = parse_dedicated_worker_line(response.trim());
|
||||
if matches!(parsed, DedicatedWorkerResult::Error(_)) {
|
||||
results.push(parsed);
|
||||
break;
|
||||
}
|
||||
results.push(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(stdin, "end").unwrap();
|
||||
stdin.flush().unwrap();
|
||||
let _ = child.wait().expect("Worker process failed to exit");
|
||||
|
||||
results
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_exec_preprocess() {
|
||||
let script = r#"
|
||||
def preprocessor(x: int):
|
||||
return {"x": x * 10}
|
||||
|
||||
def main(x: int):
|
||||
return x + 1
|
||||
"#;
|
||||
let results = run_py_raw_protocol_test(
|
||||
&[("f/test/pre", script)],
|
||||
vec![ProtocolCmd::ExecPreprocess {
|
||||
path: "f/test/pre".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
}],
|
||||
);
|
||||
assert_eq!(results.len(), 2);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
DedicatedWorkerResult::PreprocessedArgs(serde_json::json!({"x": 50}))
|
||||
);
|
||||
// main(50) => 51
|
||||
assert_eq!(
|
||||
results[1],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(51))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_exec_preprocess_missing_preprocessor() {
|
||||
let script = "def main(x: int):\n return x\n";
|
||||
let results = run_py_raw_protocol_test(
|
||||
&[("f/test/nopre", script)],
|
||||
vec![ProtocolCmd::ExecPreprocess {
|
||||
path: "f/test/nopre".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
}],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert!(matches!(results[0], DedicatedWorkerResult::Error(_)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_exec_preprocess_then_exec() {
|
||||
let script = r#"
|
||||
def preprocessor(x: int):
|
||||
return {"x": x * 2}
|
||||
|
||||
def main(x: int):
|
||||
return x + 100
|
||||
"#;
|
||||
let results = run_py_raw_protocol_test(
|
||||
&[("f/test/mixed", script)],
|
||||
vec![
|
||||
ProtocolCmd::ExecPreprocess {
|
||||
path: "f/test/mixed".to_string(),
|
||||
args: serde_json::json!({"x": 5}),
|
||||
},
|
||||
ProtocolCmd::Exec {
|
||||
path: "f/test/mixed".to_string(),
|
||||
args: serde_json::json!({"x": 7}),
|
||||
},
|
||||
],
|
||||
);
|
||||
// preprocess: preprocessor(5) => {"x":10}, main(10) => 110
|
||||
// exec: main(7) => 107
|
||||
assert_eq!(results.len(), 3);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
DedicatedWorkerResult::PreprocessedArgs(serde_json::json!({"x": 10}))
|
||||
);
|
||||
assert_eq!(
|
||||
results[1],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(110))
|
||||
);
|
||||
assert_eq!(
|
||||
results[2],
|
||||
DedicatedWorkerResult::Success(serde_json::json!(107))
|
||||
);
|
||||
}
|
||||
|
||||
// ==================== Argument Transformation Tests ====================
|
||||
|
||||
#[test]
|
||||
fn test_python_datetime_arg_transformation() {
|
||||
let script = r#"
|
||||
from datetime import datetime
|
||||
|
||||
def main(d: datetime):
|
||||
return d.isoformat()
|
||||
"#;
|
||||
let results = run_py_single_script_test(
|
||||
"f/test/dt",
|
||||
script,
|
||||
vec![serde_json::json!({"d": "2024-01-15T10:30:00+00:00"})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(
|
||||
results[0],
|
||||
Ok(serde_json::json!("2024-01-15T10:30:00+00:00"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_bytes_arg_transformation() {
|
||||
let script = r#"
|
||||
def main(data: bytes):
|
||||
return len(data)
|
||||
"#;
|
||||
// base64 of "hello" is "aGVsbG8="
|
||||
let results = run_py_single_script_test(
|
||||
"f/test/bytes",
|
||||
script,
|
||||
vec![serde_json::json!({"data": "aGVsbG8="})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(5)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_kwargs_filtering() {
|
||||
// Test that extra kwargs are filtered out and only declared args are passed
|
||||
let script = "def main(a: int, b: int):\n return a + b\n";
|
||||
let results = run_py_single_script_test(
|
||||
"f/test/kwargs",
|
||||
script,
|
||||
vec![serde_json::json!({"a": 1, "b": 2, "extra": 99})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
assert_eq!(results[0], Ok(serde_json::json!(3)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_python_function_call_sentinel_removal() {
|
||||
// Test that '<function call>' sentinel values are removed from args
|
||||
let script = "def main(a: int, b: int = 10):\n return a + b\n";
|
||||
let results = run_py_single_script_test(
|
||||
"f/test/sentinel",
|
||||
script,
|
||||
vec![serde_json::json!({"a": 5, "b": "<function call>"})],
|
||||
);
|
||||
assert_eq!(results.len(), 1);
|
||||
// b should be removed (sentinel), default 10 used
|
||||
assert_eq!(results[0], Ok(serde_json::json!(15)));
|
||||
}
|
||||
|
||||
// ==================== Relative Import Tests ====================
|
||||
|
||||
#[test]
|
||||
fn test_python_dedicated_worker_with_relative_import_detection() {
|
||||
// Test that the wrapper includes 'import loader' when scripts have relative imports
|
||||
let script_with_relative = "from f.helper import util\ndef main(x: int):\n return x\n";
|
||||
let cg = compute_py_codegen(script_with_relative, "f/test/rel");
|
||||
let entries = [PyScriptEntry { original_path: "f/test/rel", codegen: &cg }];
|
||||
let wrapper = generate_py_multi_script_wrapper(&entries, false, true);
|
||||
assert!(
|
||||
wrapper.contains("import loader"),
|
||||
"wrapper should contain 'import loader' when any_relative_imports=true"
|
||||
);
|
||||
|
||||
// Without relative imports
|
||||
let script_no_relative = "def main(x: int):\n return x\n";
|
||||
let cg2 = compute_py_codegen(script_no_relative, "f/test/norel");
|
||||
let entries2 = [PyScriptEntry { original_path: "f/test/norel", codegen: &cg2 }];
|
||||
let wrapper2 = generate_py_multi_script_wrapper(&entries2, false, false);
|
||||
assert!(
|
||||
!wrapper2.contains("import loader"),
|
||||
"wrapper should NOT contain 'import loader' when any_relative_imports=false"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "python")]
|
||||
#[sqlx::test(fixtures("base", "lockfile_python"))]
|
||||
async fn test_requirements_python(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
|
||||
@@ -209,6 +209,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
FlowModule {
|
||||
id: "b".to_string(),
|
||||
@@ -255,6 +256,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
}],
|
||||
modules_node: None,
|
||||
}
|
||||
@@ -275,6 +277,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
],
|
||||
same_worker: false,
|
||||
@@ -389,6 +392,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
FlowModule {
|
||||
id: "b".to_string(),
|
||||
@@ -444,6 +448,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
FlowModule {
|
||||
id: "e".to_string(),
|
||||
@@ -485,6 +490,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
],
|
||||
modules_node: None,
|
||||
@@ -505,6 +511,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
FlowModule {
|
||||
id: "c".to_string(),
|
||||
@@ -552,6 +559,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
],
|
||||
same_worker: true,
|
||||
|
||||
@@ -33,8 +33,10 @@ use sqlx::{FromRow, Postgres, Transaction};
|
||||
use windmill_audit::audit_oss::{audit_log, AuditAuthorable};
|
||||
use windmill_audit::ActionKind;
|
||||
use windmill_common::assets::{clear_static_asset_usage, AssetUsageKind};
|
||||
use windmill_common::flows::FlowModule;
|
||||
use windmill_common::min_version::{
|
||||
MIN_VERSION_SUPPORTS_DEBOUNCING, MIN_VERSION_SUPPORTS_DEBOUNCING_V2,
|
||||
MIN_VERSION_SUPPORTS_NODE_DEBOUNCING,
|
||||
};
|
||||
use windmill_common::runnable_settings::RunnableSettingsTrait;
|
||||
use windmill_common::utils::query_elems_from_hub;
|
||||
@@ -1576,30 +1578,54 @@ async fn archive_flow_by_path(
|
||||
/// Validates that flow debouncing configuration is supported by all workers
|
||||
/// Returns an error if debouncing is configured but workers are behind required version
|
||||
async fn guard_flow_from_debounce_data(nf: &NewFlow) -> Result<()> {
|
||||
if !MIN_VERSION_SUPPORTS_DEBOUNCING.met().await
|
||||
&& !nf.parse_flow_value()?.debouncing_settings.is_default()
|
||||
let flow_value = nf.parse_flow_value()?;
|
||||
|
||||
if !MIN_VERSION_SUPPORTS_DEBOUNCING.met().await && !flow_value.debouncing_settings.is_default()
|
||||
{
|
||||
tracing::warn!(
|
||||
"Flow debouncing configuration rejected: workers are behind minimum required version for debouncing feature"
|
||||
);
|
||||
Err(Error::WorkersAreBehind { feature: "Debouncing".into(), min_version: "1.566.0".into() })
|
||||
} else if !MIN_VERSION_SUPPORTS_DEBOUNCING_V2.met().await
|
||||
&& !nf
|
||||
.parse_flow_value()?
|
||||
.debouncing_settings
|
||||
.is_legacy_compatible()
|
||||
return Err(Error::WorkersAreBehind {
|
||||
feature: "Debouncing".into(),
|
||||
min_version: "1.566.0".into(),
|
||||
});
|
||||
}
|
||||
|
||||
if !MIN_VERSION_SUPPORTS_DEBOUNCING_V2.met().await
|
||||
&& !flow_value.debouncing_settings.is_legacy_compatible()
|
||||
&& !*WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT
|
||||
{
|
||||
tracing::warn!(
|
||||
"Flow debouncing configuration rejected: workers are behind minimum required version for debouncing feature"
|
||||
);
|
||||
Err(Error::WorkersAreBehind {
|
||||
return Err(Error::WorkersAreBehind {
|
||||
feature: "V2 Debouncing".into(),
|
||||
min_version: "1.597.0".into(),
|
||||
})
|
||||
} else {
|
||||
Ok(())
|
||||
});
|
||||
}
|
||||
|
||||
// Check node-level debouncing on all modules (including nested branches/loops)
|
||||
let mut has_node_debouncing = false;
|
||||
let check_result = FlowModule::traverse_modules(&flow_value.modules, &mut |m| {
|
||||
if m.debouncing
|
||||
.as_ref()
|
||||
.is_some_and(|d| d.debounce_delay_s.is_some_and(|s| s > 0))
|
||||
{
|
||||
has_node_debouncing = true;
|
||||
}
|
||||
Ok(())
|
||||
});
|
||||
if let Err(e) = check_result {
|
||||
tracing::warn!("Failed to traverse flow modules for debounce guard: {e}");
|
||||
}
|
||||
if has_node_debouncing && !MIN_VERSION_SUPPORTS_NODE_DEBOUNCING.met().await {
|
||||
return Err(Error::WorkersAreBehind {
|
||||
feature: "Flow node debouncing".into(),
|
||||
min_version: "1.658.0".into(),
|
||||
});
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -1768,6 +1794,7 @@ mod tests {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
FlowModule {
|
||||
id: "b".to_string(),
|
||||
@@ -1801,6 +1828,7 @@ mod tests {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
FlowModule {
|
||||
id: "c".to_string(),
|
||||
@@ -1834,6 +1862,7 @@ mod tests {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
},
|
||||
],
|
||||
failure_module: Some(Box::new(FlowModule {
|
||||
@@ -1866,6 +1895,7 @@ mod tests {
|
||||
skip_if: None,
|
||||
apply_preprocessor: None,
|
||||
pass_flow_input_directly: None,
|
||||
debouncing: None,
|
||||
})),
|
||||
preprocessor_module: None,
|
||||
same_worker: false,
|
||||
|
||||
@@ -18,7 +18,6 @@ use regex::Regex;
|
||||
use windmill_api_auth::{check_scopes, ApiAuthed, AuthCache, Tokened};
|
||||
use windmill_audit::audit_oss::{audit_log, AuditAuthorable};
|
||||
use windmill_audit::ActionKind;
|
||||
use windmill_common::{error::Error, webhook::{WebhookMessage, WebhookShared}, workspaces::{check_user_against_rule, ProtectionRuleKind, RuleCheckResult}};
|
||||
use windmill_common::DB;
|
||||
use windmill_common::{
|
||||
db::UserDB,
|
||||
@@ -26,6 +25,11 @@ use windmill_common::{
|
||||
users::username_to_permissioned_as,
|
||||
utils::{not_found_if_none, paginate, Pagination},
|
||||
};
|
||||
use windmill_common::{
|
||||
error::Error,
|
||||
webhook::{WebhookMessage, WebhookShared},
|
||||
workspaces::{check_user_against_rule, ProtectionRuleKind, RuleCheckResult},
|
||||
};
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sqlx::{FromRow, Postgres, Transaction};
|
||||
@@ -716,13 +720,14 @@ async fn add_owner(
|
||||
.await?;
|
||||
|
||||
validate_owner(&owner)?;
|
||||
sqlx::query(&format!(
|
||||
"UPDATE folder SET extra_perms = jsonb_set(extra_perms, '{{\"{owner}\"}}', to_jsonb($1), \
|
||||
true) WHERE name = $2 AND workspace_id = $3 RETURNING extra_perms"
|
||||
))
|
||||
sqlx::query(
|
||||
"UPDATE folder SET extra_perms = jsonb_set(extra_perms, array[$4]::text[], to_jsonb($1), \
|
||||
true) WHERE name = $2 AND workspace_id = $3 RETURNING extra_perms",
|
||||
)
|
||||
.bind(true)
|
||||
.bind(&name)
|
||||
.bind(&w_id)
|
||||
.bind(&owner)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?;
|
||||
|
||||
@@ -787,14 +792,15 @@ async fn remove_owner(
|
||||
}
|
||||
|
||||
if let Some(write) = write {
|
||||
let old_write = sqlx::query_scalar::<_, Option<bool>>(&format!(
|
||||
"UPDATE folder SET extra_perms = jsonb_set(extra_perms, '{{\"{owner}\"}}', to_jsonb($1), \
|
||||
true) FROM (SELECT (extra_perms->>'{owner}')::boolean as old_val FROM folder WHERE name = $2 AND workspace_id = $3) old \
|
||||
let old_write = sqlx::query_scalar::<_, Option<bool>>(
|
||||
"UPDATE folder SET extra_perms = jsonb_set(extra_perms, array[$4]::text[], to_jsonb($1), \
|
||||
true) FROM (SELECT (extra_perms->>$4)::boolean as old_val FROM folder WHERE name = $2 AND workspace_id = $3) old \
|
||||
WHERE name = $2 AND workspace_id = $3 RETURNING old.old_val"
|
||||
))
|
||||
)
|
||||
.bind(write)
|
||||
.bind(&name)
|
||||
.bind(&w_id)
|
||||
.bind(&owner)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?
|
||||
.flatten();
|
||||
|
||||
@@ -422,6 +422,7 @@ async fn test_delete_integration_full_cascade(db: Pool<Postgres>) -> anyhow::Res
|
||||
"ext-1",
|
||||
&trigger_config,
|
||||
json!({"triggerType": "drive"}),
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
|
||||
@@ -511,6 +512,7 @@ async fn test_cleanup_preserves_triggers(db: Pool<Postgres>) -> anyhow::Result<(
|
||||
"ext-1",
|
||||
&trigger_config,
|
||||
json!({"triggerType": "drive"}),
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
|
||||
|
||||
@@ -82,12 +82,10 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
assert_eq!(resp.status(), 200);
|
||||
|
||||
// --- allowed_domain_auto_invite ---
|
||||
let resp = authed(client().get(format!(
|
||||
"{global_base}/allowed_domain_auto_invite"
|
||||
)))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
let resp = authed(client().get(format!("{global_base}/allowed_domain_auto_invite")))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), 200);
|
||||
resp.json::<bool>().await?;
|
||||
|
||||
@@ -213,12 +211,10 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
resp.json::<Vec<String>>().await?;
|
||||
|
||||
// --- get_dependents (empty, no dependencies exist) ---
|
||||
let resp = authed(client().get(format!(
|
||||
"{base}/get_dependents/u/test-user/nonexistent"
|
||||
)))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
let resp = authed(client().get(format!("{base}/get_dependents/u/test-user/nonexistent")))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), 200);
|
||||
let dependents = resp.json::<Vec<serde_json::Value>>().await?;
|
||||
assert!(dependents.is_empty());
|
||||
@@ -425,13 +421,11 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
);
|
||||
|
||||
// --- edit_large_file_storage_config ---
|
||||
let resp = authed(client().post(format!(
|
||||
"{base}/edit_large_file_storage_config"
|
||||
)))
|
||||
.json(&json!({"large_file_storage": null}))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
let resp = authed(client().post(format!("{base}/edit_large_file_storage_config")))
|
||||
.json(&json!({"large_file_storage": null}))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
resp.status(),
|
||||
200,
|
||||
@@ -532,9 +526,7 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
.unwrap();
|
||||
let invites = resp.json::<Vec<serde_json::Value>>().await?;
|
||||
assert!(
|
||||
invites
|
||||
.iter()
|
||||
.any(|i| i["email"] == "invited@example.com"),
|
||||
invites.iter().any(|i| i["email"] == "invited@example.com"),
|
||||
"invite not found: {:?}",
|
||||
invites
|
||||
);
|
||||
@@ -549,12 +541,7 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
resp.status(),
|
||||
201,
|
||||
"delete_invite: {}",
|
||||
resp.text().await?
|
||||
);
|
||||
assert_eq!(resp.status(), 201, "delete_invite: {}", resp.text().await?);
|
||||
|
||||
// ===== Critical alerts (EE-gated, returns 404 in OSS) =====
|
||||
|
||||
@@ -624,12 +611,7 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
resp.status(),
|
||||
200,
|
||||
"create_fork: {}",
|
||||
resp.text().await?
|
||||
);
|
||||
assert_eq!(resp.status(), 200, "create_fork: {}", resp.text().await?);
|
||||
|
||||
// verify fork exists
|
||||
let resp = authed(client().post(format!("{global_base}/exists")))
|
||||
@@ -702,13 +684,122 @@ async fn test_workspace_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
assert_eq!(resp.json::<bool>().await?, false);
|
||||
|
||||
// --- create_workspace_require_superadmin ---
|
||||
let resp = authed(client().get(format!(
|
||||
"{global_base}/create_workspace_require_superadmin"
|
||||
)))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
let resp = authed(client().get(format!("{global_base}/create_workspace_require_superadmin")))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), 200);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_get_copilot_settings_state_reports_instance_ai_fallback_flags(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
|
||||
|
||||
let instance_ai_config = json!({
|
||||
"providers": {
|
||||
"openai": {
|
||||
"resource_path": "u/test-user/openai_instance",
|
||||
"models": ["gpt-4o-mini"]
|
||||
}
|
||||
}
|
||||
});
|
||||
let workspace_ai_config = json!({
|
||||
"providers": {
|
||||
"anthropic": {
|
||||
"resource_path": "u/test-user/anthropic_workspace",
|
||||
"models": ["claude-3-5-haiku-latest"]
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
sqlx::query("UPDATE workspace_settings SET ai_config = NULL WHERE workspace_id = $1")
|
||||
.bind("test-workspace")
|
||||
.execute(&db)
|
||||
.await?;
|
||||
sqlx::query(
|
||||
"INSERT INTO global_settings (name, value) VALUES ($1, $2) \
|
||||
ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
|
||||
)
|
||||
.bind("ai_config")
|
||||
.bind(instance_ai_config)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
let resp = authed(client().get(format!("{base}/get_copilot_settings_state")))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), 200);
|
||||
let settings = resp.json::<serde_json::Value>().await?;
|
||||
assert_eq!(settings["has_instance_ai_config"], true);
|
||||
assert_eq!(settings["uses_instance_ai_config"], true);
|
||||
assert_eq!(
|
||||
settings["instance_ai_summary"]["providers"][0]["provider"],
|
||||
"openai"
|
||||
);
|
||||
assert_eq!(
|
||||
settings["instance_ai_summary"]["providers"][0]["models"][0],
|
||||
"gpt-4o-mini"
|
||||
);
|
||||
|
||||
sqlx::query("UPDATE workspace_settings SET ai_config = $1 WHERE workspace_id = $2")
|
||||
.bind(workspace_ai_config)
|
||||
.bind("test-workspace")
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
let resp = authed(client().get(format!("{base}/get_copilot_settings_state")))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), 200);
|
||||
let settings = resp.json::<serde_json::Value>().await?;
|
||||
assert_eq!(settings["has_instance_ai_config"], true);
|
||||
assert_eq!(settings["uses_instance_ai_config"], false);
|
||||
assert_eq!(
|
||||
settings["instance_ai_summary"]["providers"][0]["provider"],
|
||||
"openai"
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_get_copilot_info_ignores_empty_instance_ai_row(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
|
||||
|
||||
sqlx::query("UPDATE workspace_settings SET ai_config = NULL WHERE workspace_id = $1")
|
||||
.bind("test-workspace")
|
||||
.execute(&db)
|
||||
.await?;
|
||||
sqlx::query(
|
||||
"INSERT INTO global_settings (name, value) VALUES ($1, $2) \
|
||||
ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value",
|
||||
)
|
||||
.bind("ai_config")
|
||||
.bind(json!({}))
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
let resp = authed(client().get(format!("{base}/get_copilot_info")))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), 200);
|
||||
let settings = resp.json::<serde_json::Value>().await?;
|
||||
assert!(settings["providers"].is_null());
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -50,11 +50,11 @@ pub fn filter_list_queue_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let p = v.replace("*", "%").replace("'", "''");
|
||||
let p = v.replace("*", "%");
|
||||
if w.negated {
|
||||
format!("v2_job_queue.worker NOT LIKE '{p}'")
|
||||
format!("v2_job_queue.worker NOT LIKE {}", quote(&p))
|
||||
} else {
|
||||
format!("v2_job_queue.worker LIKE '{p}'")
|
||||
format!("v2_job_queue.worker LIKE {}", quote(&p))
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -77,11 +77,11 @@ pub fn filter_list_queue_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let e = v.replace("'", "''");
|
||||
let p = format!("{}%", v);
|
||||
if ps.negated {
|
||||
format!("runnable_path NOT LIKE '{e}%'")
|
||||
format!("runnable_path NOT LIKE {}", quote(&p))
|
||||
} else {
|
||||
format!("runnable_path LIKE '{e}%'")
|
||||
format!("runnable_path LIKE {}", quote(&p))
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -123,11 +123,11 @@ pub fn filter_list_queue_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let p = v.replace("*", "%").replace("'", "''");
|
||||
let p = v.replace("*", "%");
|
||||
if t.negated {
|
||||
format!("v2_job.tag NOT LIKE '{p}'")
|
||||
format!("v2_job.tag NOT LIKE {}", quote(&p))
|
||||
} else {
|
||||
format!("v2_job.tag LIKE '{p}'")
|
||||
format!("v2_job.tag LIKE {}", quote(&p))
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -287,14 +287,14 @@ pub fn filter_list_completed_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let p = v.replace("*", "%").replace("'", "''");
|
||||
let p = v.replace("*", "%");
|
||||
if label.negated {
|
||||
format!(
|
||||
"NOT EXISTS (SELECT 1 FROM jsonb_array_elements_text(result->'wm_labels') lbl WHERE jsonb_typeof(result->'wm_labels') = 'array' AND lbl LIKE '{p}')"
|
||||
"NOT EXISTS (SELECT 1 FROM jsonb_array_elements_text(result->'wm_labels') lbl WHERE jsonb_typeof(result->'wm_labels') = 'array' AND lbl LIKE {})", quote(&p)
|
||||
)
|
||||
} else {
|
||||
format!(
|
||||
"EXISTS (SELECT 1 FROM jsonb_array_elements_text(result->'wm_labels') lbl WHERE jsonb_typeof(result->'wm_labels') = 'array' AND lbl LIKE '{p}')"
|
||||
"EXISTS (SELECT 1 FROM jsonb_array_elements_text(result->'wm_labels') lbl WHERE jsonb_typeof(result->'wm_labels') = 'array' AND lbl LIKE {})", quote(&p)
|
||||
)
|
||||
}
|
||||
})
|
||||
@@ -308,14 +308,14 @@ pub fn filter_list_completed_query(
|
||||
let clauses: Vec<_> = label
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| format!("NOT (result->'wm_labels' ? '{}')", v.replace("'", "''")))
|
||||
.map(|v| format!("NOT (result->'wm_labels' ? {})", quote(v)))
|
||||
.collect();
|
||||
sqlb.and_where(format!("({})", clauses.join(" AND ")));
|
||||
} else {
|
||||
let clauses: Vec<_> = label
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| format!("result->'wm_labels' ? '{}'", v.replace("'", "''")))
|
||||
.map(|v| format!("result->'wm_labels' ? {}", quote(v)))
|
||||
.collect();
|
||||
sqlb.and_where("result ? 'wm_labels'");
|
||||
sqlb.and_where(format!("({})", clauses.join(" OR ")));
|
||||
@@ -329,11 +329,11 @@ pub fn filter_list_completed_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let p = v.replace("*", "%").replace("'", "''");
|
||||
let p = v.replace("*", "%");
|
||||
if worker.negated {
|
||||
format!("v2_job_completed.worker NOT LIKE '{p}'")
|
||||
format!("v2_job_completed.worker NOT LIKE {}", quote(&p))
|
||||
} else {
|
||||
format!("v2_job_completed.worker LIKE '{p}'")
|
||||
format!("v2_job_completed.worker LIKE {}", quote(&p))
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -366,11 +366,11 @@ pub fn filter_list_completed_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let e = v.replace("'", "''");
|
||||
let p = format!("{}%", v);
|
||||
if ps.negated {
|
||||
format!("runnable_path NOT LIKE '{e}%'")
|
||||
format!("runnable_path NOT LIKE {}", quote(&p))
|
||||
} else {
|
||||
format!("runnable_path LIKE '{e}%'")
|
||||
format!("runnable_path LIKE {}", quote(&p))
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -400,11 +400,11 @@ pub fn filter_list_completed_query(
|
||||
.values
|
||||
.iter()
|
||||
.map(|v| {
|
||||
let p = v.replace("*", "%").replace("'", "''");
|
||||
let p = v.replace("*", "%");
|
||||
if t.negated {
|
||||
format!("v2_job.tag NOT LIKE '{p}'")
|
||||
format!("v2_job.tag NOT LIKE {}", quote(&p))
|
||||
} else {
|
||||
format!("v2_job.tag LIKE '{p}'")
|
||||
format!("v2_job.tag LIKE {}", quote(&p))
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
@@ -449,11 +449,7 @@ pub fn filter_list_completed_query(
|
||||
}
|
||||
if let Some(dt) = &lq.created_or_started_after {
|
||||
let ts = dt.to_rfc3339();
|
||||
sqlb.and_where(format!(
|
||||
"(created_at >= '{}' OR started_at >= '{}')",
|
||||
ts.replace("'", "''"),
|
||||
ts.replace("'", "''")
|
||||
));
|
||||
sqlb.and_where("(created_at >= ? OR started_at >= ?)".bind(&ts).bind(&ts));
|
||||
}
|
||||
|
||||
if let Some(dt) = &lq.created_before {
|
||||
|
||||
@@ -23,4 +23,5 @@ serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
serde_yml.workspace = true
|
||||
sqlx.workspace = true
|
||||
tracing.workspace = true
|
||||
url.workspace = true
|
||||
|
||||
@@ -164,6 +164,7 @@ pub struct FuturePath {
|
||||
summary: Option<String>,
|
||||
description: Option<String>,
|
||||
security_scheme: Option<SecurityScheme>,
|
||||
args_schema: Option<Value>,
|
||||
}
|
||||
|
||||
impl FuturePath {
|
||||
@@ -174,8 +175,17 @@ impl FuturePath {
|
||||
summary: Option<String>,
|
||||
description: Option<String>,
|
||||
security_scheme: Option<SecurityScheme>,
|
||||
args_schema: Option<Value>,
|
||||
) -> FuturePath {
|
||||
FuturePath { route_path, kind, request_type, summary, description, security_scheme }
|
||||
FuturePath {
|
||||
route_path,
|
||||
kind,
|
||||
request_type,
|
||||
summary,
|
||||
description,
|
||||
security_scheme,
|
||||
args_schema,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -397,7 +407,21 @@ fn generate_paths(
|
||||
);
|
||||
|
||||
if method != Method::GET {
|
||||
method_map.insert("requestBody", generate_default_request());
|
||||
if let Some(schema) = &path.args_schema {
|
||||
method_map.insert(
|
||||
"requestBody",
|
||||
serde_json::json!({
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": schema
|
||||
}
|
||||
}
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
method_map.insert("requestBody", generate_default_request());
|
||||
}
|
||||
} else if is_webhook {
|
||||
method_map.insert(
|
||||
"parameters",
|
||||
@@ -655,6 +679,76 @@ struct GenerateOpenAPI {
|
||||
openapi_spec_format: Format,
|
||||
}
|
||||
|
||||
fn clean_schema_for_openapi(schema: Value) -> Value {
|
||||
if let Value::Object(mut obj) = schema {
|
||||
obj.remove("$schema");
|
||||
obj.remove("order");
|
||||
Value::Object(obj)
|
||||
} else {
|
||||
schema
|
||||
}
|
||||
}
|
||||
|
||||
async fn get_runnable_schema(
|
||||
db: &DB,
|
||||
w_id: &str,
|
||||
script_path: &str,
|
||||
is_flow: bool,
|
||||
) -> Option<Value> {
|
||||
if is_flow {
|
||||
let row = sqlx::query!(
|
||||
r#"SELECT
|
||||
f.schema AS "schema: serde_json::Value",
|
||||
fv.value->>'preprocessor_module' IS NOT NULL AS "has_preprocessor: bool"
|
||||
FROM flow f
|
||||
LEFT JOIN flow_version fv ON fv.id = f.versions[array_length(f.versions, 1)]
|
||||
AND fv.workspace_id = f.workspace_id
|
||||
WHERE f.path = $1 AND f.workspace_id = $2 AND NOT f.archived"#,
|
||||
script_path,
|
||||
w_id,
|
||||
)
|
||||
.fetch_optional(db)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
tracing::warn!("Failed to fetch flow schema for {script_path}: {e}");
|
||||
e
|
||||
})
|
||||
.ok()
|
||||
.flatten()?;
|
||||
|
||||
if row.has_preprocessor.unwrap_or(false) {
|
||||
return None;
|
||||
}
|
||||
row.schema.map(clean_schema_for_openapi)
|
||||
} else {
|
||||
let row = sqlx::query!(
|
||||
r#"SELECT
|
||||
schema AS "schema: serde_json::Value",
|
||||
has_preprocessor
|
||||
FROM script
|
||||
WHERE path = $1 AND workspace_id = $2
|
||||
AND NOT archived AND NOT deleted
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 1"#,
|
||||
script_path,
|
||||
w_id,
|
||||
)
|
||||
.fetch_optional(db)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
tracing::warn!("Failed to fetch script schema for {script_path}: {e}");
|
||||
e
|
||||
})
|
||||
.ok()
|
||||
.flatten()?;
|
||||
|
||||
if row.has_preprocessor.unwrap_or(false) {
|
||||
return None;
|
||||
}
|
||||
row.schema.map(clean_schema_for_openapi)
|
||||
}
|
||||
}
|
||||
|
||||
async fn http_routes_to_future_paths(
|
||||
db: &DB,
|
||||
user_db: UserDB,
|
||||
@@ -691,6 +785,9 @@ async fn http_routes_to_future_paths(
|
||||
description: Option<String>,
|
||||
authentication_method: AuthenticationMethod,
|
||||
authentication_resource_path: Option<String>,
|
||||
script_path: String,
|
||||
is_flow: bool,
|
||||
wrap_body: bool,
|
||||
}
|
||||
|
||||
http_routes = sqlx::query_as!(
|
||||
@@ -704,7 +801,10 @@ async fn http_routes_to_future_paths(
|
||||
summary,
|
||||
description,
|
||||
authentication_method AS "authentication_method: _",
|
||||
authentication_resource_path
|
||||
authentication_resource_path,
|
||||
script_path,
|
||||
is_flow,
|
||||
wrap_body
|
||||
FROM
|
||||
http_trigger
|
||||
WHERE
|
||||
@@ -764,6 +864,12 @@ async fn http_routes_to_future_paths(
|
||||
HttpMethod::Delete => Method::DELETE,
|
||||
};
|
||||
|
||||
let args_schema = if !http_route.wrap_body {
|
||||
get_runnable_schema(db, w_id, &http_route.script_path, http_route.is_flow).await
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let future_path = FuturePath::new(
|
||||
route_path,
|
||||
Kind::HttpRoute(HttpRouteConfig::new(method)),
|
||||
@@ -771,6 +877,7 @@ async fn http_routes_to_future_paths(
|
||||
http_route.summary,
|
||||
http_route.description,
|
||||
auth_method,
|
||||
args_schema,
|
||||
);
|
||||
|
||||
openapi_future_paths.push(future_path);
|
||||
@@ -780,6 +887,7 @@ async fn http_routes_to_future_paths(
|
||||
}
|
||||
|
||||
async fn webhook_to_future_paths(
|
||||
db: &DB,
|
||||
pg_pool: &mut PgConnection,
|
||||
webhook_filters: Option<&[WebhookFilter]>,
|
||||
w_id: &str,
|
||||
@@ -805,7 +913,7 @@ async fn webhook_to_future_paths(
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Clone, Hash)]
|
||||
#[derive(Debug, Deserialize, Clone)]
|
||||
struct MinifiedWebhook {
|
||||
path: String,
|
||||
description: Option<String>,
|
||||
@@ -814,7 +922,7 @@ async fn webhook_to_future_paths(
|
||||
|
||||
let webhook_scripts = sqlx::query_as!(
|
||||
MinifiedWebhook,
|
||||
r#"SELECT
|
||||
r#"SELECT
|
||||
path,
|
||||
summary,
|
||||
description
|
||||
@@ -833,7 +941,7 @@ async fn webhook_to_future_paths(
|
||||
|
||||
let webhook_flows = sqlx::query_as!(
|
||||
MinifiedWebhook,
|
||||
r#"SELECT
|
||||
r#"SELECT
|
||||
path,
|
||||
summary,
|
||||
description
|
||||
@@ -853,6 +961,7 @@ async fn webhook_to_future_paths(
|
||||
openapi_future_paths.reserve_exact(webhook_scripts.len() + webhook_flows.len());
|
||||
|
||||
for webhook in webhook_scripts {
|
||||
let args_schema = get_runnable_schema(db, w_id, &webhook.path, false).await;
|
||||
openapi_future_paths.push(FuturePath::new(
|
||||
webhook.path,
|
||||
Kind::Webhook(WebhookConfig::new(RunnableKind::Script)),
|
||||
@@ -860,10 +969,12 @@ async fn webhook_to_future_paths(
|
||||
webhook.summary,
|
||||
webhook.description,
|
||||
Some(SecurityScheme::BearerJwt),
|
||||
args_schema,
|
||||
));
|
||||
}
|
||||
|
||||
for webhook in webhook_flows {
|
||||
let args_schema = get_runnable_schema(db, w_id, &webhook.path, true).await;
|
||||
openapi_future_paths.push(FuturePath::new(
|
||||
webhook.path,
|
||||
Kind::Webhook(WebhookConfig::new(RunnableKind::Flow)),
|
||||
@@ -871,6 +982,7 @@ async fn webhook_to_future_paths(
|
||||
webhook.summary,
|
||||
webhook.description,
|
||||
Some(SecurityScheme::BearerJwt),
|
||||
args_schema,
|
||||
));
|
||||
}
|
||||
}
|
||||
@@ -898,7 +1010,7 @@ async fn generate_openapi_future_path(
|
||||
http_routes_to_future_paths(db, user_db, authed, &mut tx, http_route_filters, w_id).await?;
|
||||
|
||||
openapi_future_paths
|
||||
.append(&mut webhook_to_future_paths(&mut tx, webhook_filters, w_id).await?);
|
||||
.append(&mut webhook_to_future_paths(db, &mut tx, webhook_filters, w_id).await?);
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
|
||||
@@ -239,6 +239,10 @@ pub fn workspaced_service() -> Router {
|
||||
"/history_update/h/:hash/p/*path",
|
||||
post(update_script_history),
|
||||
)
|
||||
.route("/list_dedicated_with_deps", get(list_dedicated_with_deps))
|
||||
// Temporary raw script storage for CLI lock generation
|
||||
.route("/raw_temp/store", post(store_raw_script_temp))
|
||||
.route("/raw_temp/diff", post(diff_raw_scripts_with_deployed))
|
||||
}
|
||||
|
||||
#[derive(Serialize, FromRow)]
|
||||
@@ -1614,6 +1618,9 @@ struct RawScriptByPathQuery {
|
||||
cache_key: Option<String>,
|
||||
// used specifically for python to cache folders on import success to avoid extra db calls on package fetch
|
||||
cache_folders: Option<bool>,
|
||||
// If provided, load content from raw_script_temp table using this hash instead of deployed script.
|
||||
// Used by CLI lock generation to resolve imports from not-yet-deployed scripts.
|
||||
temp_script_hash: Option<String>,
|
||||
}
|
||||
|
||||
struct StringWithLength(String);
|
||||
@@ -1672,6 +1679,16 @@ async fn raw_script_by_path_internal(
|
||||
) -> Result<String> {
|
||||
let path = path.to_path();
|
||||
check_scopes(&authed, || format!("scripts:read:{}", path))?;
|
||||
|
||||
// If temp_script_hash is provided, try loading from temp storage first.
|
||||
// This is used by CLI lock generation to resolve imports from not-yet-deployed scripts.
|
||||
// Falls back to the normal deployed script lookup if not found in temp storage.
|
||||
if let Some(hash) = query.temp_script_hash {
|
||||
if let Ok(content) = windmill_common::cache::raw_script_temp::load(hash, &db).await {
|
||||
return Ok(content);
|
||||
}
|
||||
}
|
||||
|
||||
let cache_path = query
|
||||
.cache_key
|
||||
.map(|x| format!("{w_id}:{path}:{x}{}", if unpin { ":unpinned" } else { "" }));
|
||||
@@ -2463,3 +2480,183 @@ async fn guard_script_from_debounce_data(ns: &NewScript) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct DedicatedScriptDeps {
|
||||
path: String,
|
||||
language: ScriptLang,
|
||||
workspace_dep_names: Vec<String>,
|
||||
}
|
||||
|
||||
async fn list_dedicated_with_deps(
|
||||
authed: ApiAuthed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path(w_id): Path<String>,
|
||||
) -> JsonResult<Vec<DedicatedScriptDeps>> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
let rows = sqlx::query!(
|
||||
"SELECT DISTINCT ON (path) path, language AS \"language: ScriptLang\", content FROM script
|
||||
WHERE workspace_id = $1
|
||||
AND archived = false
|
||||
AND dedicated_worker = true
|
||||
AND language = ANY($2::SCRIPT_LANG[])
|
||||
ORDER BY path, created_at DESC",
|
||||
&w_id,
|
||||
&[
|
||||
ScriptLang::Python3,
|
||||
ScriptLang::Bun,
|
||||
ScriptLang::Bunnative,
|
||||
ScriptLang::Deno,
|
||||
] as &[ScriptLang],
|
||||
)
|
||||
.fetch_all(&mut *tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
let result = rows
|
||||
.into_iter()
|
||||
.map(|row| {
|
||||
let dep_names =
|
||||
windmill_common::scripts::extract_workspace_dependencies_annotated_refs(
|
||||
&row.language,
|
||||
&row.content,
|
||||
&row.path,
|
||||
)
|
||||
.map(|refs| refs.external)
|
||||
.unwrap_or_default();
|
||||
DedicatedScriptDeps {
|
||||
path: row.path,
|
||||
language: row.language,
|
||||
workspace_dep_names: dep_names,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
Ok(Json(result))
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Temporary Raw Script Storage for CLI Lock Generation
|
||||
// ============================================================================
|
||||
|
||||
/// Store raw script content temporarily for CLI lock generation.
|
||||
async fn store_raw_script_temp(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(content): Json<String>,
|
||||
) -> Result<Json<String>> {
|
||||
check_scopes(&authed, || "scripts:write".to_string())?;
|
||||
|
||||
let hash = windmill_common::cache::raw_script_temp::compute_hash(&w_id, &content);
|
||||
|
||||
// Store to DB
|
||||
sqlx::query!(
|
||||
"INSERT INTO raw_script_temp (workspace_id, hash, content, created_at)
|
||||
VALUES ($1, $2, $3, NOW())
|
||||
ON CONFLICT (workspace_id, hash) DO UPDATE SET created_at = NOW()",
|
||||
&w_id,
|
||||
&hash,
|
||||
&content
|
||||
)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
// Clean up old entries (1 week TTL)
|
||||
sqlx::query!("DELETE FROM raw_script_temp WHERE created_at < NOW() - INTERVAL '1 week'")
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
Ok(Json(hash))
|
||||
}
|
||||
|
||||
/// Compare local script content hashes with deployed versions.
|
||||
/// Receives a map of path → SHA256(content), returns paths where the hash
|
||||
/// differs from the deployed script (or the script doesn't exist on remote).
|
||||
/// Hash comparison is done entirely in Postgres to avoid transferring content.
|
||||
#[derive(Deserialize)]
|
||||
struct WorkspaceDepDiff {
|
||||
path: String,
|
||||
language: ScriptLang,
|
||||
name: Option<String>,
|
||||
hash: String,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct DiffRequest {
|
||||
scripts: std::collections::HashMap<String, String>,
|
||||
#[serde(default)]
|
||||
workspace_deps: Vec<WorkspaceDepDiff>,
|
||||
}
|
||||
|
||||
async fn diff_raw_scripts_with_deployed(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(req): Json<DiffRequest>,
|
||||
) -> Result<Json<Vec<String>>> {
|
||||
check_scopes(&authed, || "scripts:read".to_string())?;
|
||||
|
||||
let mut matching_set: std::collections::HashSet<String> = std::collections::HashSet::new();
|
||||
let mut all_paths: Vec<String> = Vec::new();
|
||||
|
||||
// --- Scripts ---
|
||||
if !req.scripts.is_empty() {
|
||||
let paths: Vec<String> = req.scripts.keys().cloned().collect();
|
||||
let hashes: Vec<String> = paths.iter().map(|p| req.scripts[p].clone()).collect();
|
||||
|
||||
let matching: Vec<String> = sqlx::query_scalar(
|
||||
"SELECT local.path FROM \
|
||||
unnest($1::text[], $2::text[]) AS local(path, hash) \
|
||||
INNER JOIN LATERAL ( \
|
||||
SELECT encode(sha256(convert_to(s.content, 'UTF8')), 'hex') AS deployed_hash \
|
||||
FROM script s \
|
||||
WHERE s.path = local.path AND s.workspace_id = $3 AND s.archived = false \
|
||||
ORDER BY s.created_at DESC LIMIT 1 \
|
||||
) deployed ON deployed.deployed_hash = local.hash",
|
||||
)
|
||||
.bind(&paths)
|
||||
.bind(&hashes)
|
||||
.bind(&w_id)
|
||||
.fetch_all(&db)
|
||||
.await?;
|
||||
|
||||
matching_set.extend(matching);
|
||||
all_paths.extend(paths);
|
||||
}
|
||||
|
||||
// --- Workspace dependencies ---
|
||||
for dep in &req.workspace_deps {
|
||||
let matching: Option<String> = sqlx::query_scalar(
|
||||
"SELECT $1::text \
|
||||
WHERE EXISTS ( \
|
||||
SELECT 1 FROM workspace_dependencies wd \
|
||||
WHERE wd.workspace_id = $2 AND wd.archived = false \
|
||||
AND wd.language = $3::SCRIPT_LANG \
|
||||
AND wd.name IS NOT DISTINCT FROM $4 \
|
||||
AND encode(sha256(convert_to(wd.content, 'UTF8')), 'hex') = $5 \
|
||||
)",
|
||||
)
|
||||
.bind(&dep.path)
|
||||
.bind(&w_id)
|
||||
.bind(dep.language.as_str())
|
||||
.bind(&dep.name)
|
||||
.bind(&dep.hash)
|
||||
.fetch_optional(&db)
|
||||
.await?;
|
||||
|
||||
if let Some(path) = matching {
|
||||
matching_set.insert(path);
|
||||
}
|
||||
all_paths.push(dep.path.clone());
|
||||
}
|
||||
|
||||
let mismatched: Vec<String> = all_paths
|
||||
.into_iter()
|
||||
.filter(|p| !matching_set.contains(p))
|
||||
.collect();
|
||||
|
||||
Ok(Json(mismatched))
|
||||
}
|
||||
|
||||
@@ -38,11 +38,12 @@ use windmill_common::ee_oss::{send_critical_alert, CriticalAlertKind, CriticalEr
|
||||
#[cfg(all(feature = "private", feature = "enterprise"))]
|
||||
use windmill_common::secret_backend::{SecretMigrationReport, VaultSettings};
|
||||
use windmill_common::{
|
||||
ai_cache::bump_instance_ai_config_revision,
|
||||
email_oss::send_email_plain_text,
|
||||
error::{self, JsonResult, Result},
|
||||
get_database_url,
|
||||
global_settings::{
|
||||
APP_WORKSPACED_ROUTE_SETTING, AUTOMATE_USERNAME_CREATION_SETTING,
|
||||
AI_CONFIG_SETTING, APP_WORKSPACED_ROUTE_SETTING, AUTOMATE_USERNAME_CREATION_SETTING,
|
||||
CRITICAL_ALERT_MUTE_UI_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING, DISABLE_HUB_SETTING,
|
||||
EMAIL_DOMAIN_SETTING, ENV_SETTINGS, HUB_ACCESSIBLE_URL_SETTING, HUB_BASE_URL_SETTING,
|
||||
WS_BASE_URL_SETTING,
|
||||
@@ -284,6 +285,7 @@ pub async fn set_global_setting_internal(
|
||||
key: String,
|
||||
value: serde_json::Value,
|
||||
) -> error::Result<()> {
|
||||
let should_bump_instance_ai_revision = key == AI_CONFIG_SETTING;
|
||||
let value = if key == "retention_period_secs" {
|
||||
instance_config::clamp_retention_period(value)
|
||||
} else {
|
||||
@@ -325,6 +327,10 @@ pub async fn set_global_setting_internal(
|
||||
}
|
||||
};
|
||||
|
||||
if should_bump_instance_ai_revision {
|
||||
bump_instance_ai_config_revision();
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -471,6 +477,10 @@ async fn set_instance_config(
|
||||
let current_map = current.global_settings.to_settings_map();
|
||||
let settings_diff =
|
||||
instance_config::diff_global_settings(¤t_map, &desired_map, ApplyMode::Merge);
|
||||
let ai_config_changed = settings_diff
|
||||
.upserts
|
||||
.iter()
|
||||
.any(|(key, _)| key == AI_CONFIG_SETTING);
|
||||
|
||||
for (key, value) in &settings_diff.upserts {
|
||||
run_setting_pre_write_hook(&db, key, value).await?;
|
||||
@@ -479,6 +489,10 @@ async fn set_instance_config(
|
||||
instance_config::apply_settings_diff(&db, &settings_diff)
|
||||
.await
|
||||
.map_err(|e| error::Error::internal_err(e.to_string()))?;
|
||||
|
||||
if ai_config_changed {
|
||||
bump_instance_ai_config_revision();
|
||||
}
|
||||
}
|
||||
|
||||
if !desired.worker_configs.is_empty() {
|
||||
@@ -1110,6 +1124,59 @@ struct CachedResourceType {
|
||||
description: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(serde::Deserialize)]
|
||||
struct HubResourceTypeRaw {
|
||||
id: i64,
|
||||
name: String,
|
||||
schema: Option<String>,
|
||||
app: String,
|
||||
description: Option<String>,
|
||||
}
|
||||
|
||||
async fn fetch_resource_types_from_hub() -> error::Result<Vec<CachedResourceType>> {
|
||||
let response = HTTP_CLIENT
|
||||
.get(format!(
|
||||
"{}/resource_types/list",
|
||||
windmill_common::DEFAULT_HUB_BASE_URL
|
||||
))
|
||||
.header("Accept", "application/json")
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| error::Error::InternalErr(format!("Failed to fetch from hub: {}", e)))?;
|
||||
|
||||
if !response.status().is_success() {
|
||||
return Err(error::Error::InternalErr(format!(
|
||||
"Hub returned status {}",
|
||||
response.status()
|
||||
)));
|
||||
}
|
||||
|
||||
let raw_types: Vec<HubResourceTypeRaw> = response
|
||||
.json()
|
||||
.await
|
||||
.map_err(|e| error::Error::InternalErr(format!("Failed to parse hub response: {}", e)))?;
|
||||
|
||||
Ok(raw_types
|
||||
.into_iter()
|
||||
.filter_map(|rt| {
|
||||
let schema = match rt.schema {
|
||||
Some(s) => match serde_json::from_str(&s) {
|
||||
Ok(v) => Some(v),
|
||||
Err(_) => return None,
|
||||
},
|
||||
None => None,
|
||||
};
|
||||
Some(CachedResourceType {
|
||||
id: rt.id,
|
||||
name: rt.name,
|
||||
schema,
|
||||
app: rt.app,
|
||||
description: rt.description,
|
||||
})
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
async fn sync_cached_resource_types(
|
||||
Extension(db): Extension<DB>,
|
||||
authed: ApiAuthed,
|
||||
@@ -1119,16 +1186,12 @@ async fn sync_cached_resource_types(
|
||||
use windmill_common::worker::HUB_RT_CACHE_DIR;
|
||||
let cache_path = format!("{}/resource_types.json", *HUB_RT_CACHE_DIR);
|
||||
|
||||
let content = tokio::fs::read_to_string(&cache_path).await.map_err(|e| {
|
||||
error::Error::NotFound(format!(
|
||||
"No cached resource types found at {}: {}",
|
||||
cache_path, e
|
||||
))
|
||||
})?;
|
||||
|
||||
let cached_types: Vec<CachedResourceType> = serde_json::from_str(&content).map_err(|e| {
|
||||
error::Error::InternalErr(format!("Failed to parse cached resource types: {}", e))
|
||||
})?;
|
||||
let cached_types = match tokio::fs::read_to_string(&cache_path).await {
|
||||
Ok(content) => serde_json::from_str::<Vec<CachedResourceType>>(&content).map_err(|e| {
|
||||
error::Error::InternalErr(format!("Failed to parse cached resource types: {}", e))
|
||||
})?,
|
||||
Err(_) => fetch_resource_types_from_hub().await?,
|
||||
};
|
||||
|
||||
let mut synced_count = 0;
|
||||
|
||||
|
||||
@@ -34,3 +34,4 @@ time.workspace = true
|
||||
tokio.workspace = true
|
||||
tower-cookies.workspace = true
|
||||
tracing.workspace = true
|
||||
url.workspace = true
|
||||
|
||||
@@ -49,13 +49,13 @@ use windmill_common::users::truncate_token;
|
||||
use windmill_common::users::COOKIE_NAME;
|
||||
use windmill_common::utils::paginate;
|
||||
use windmill_common::worker::CLOUD_HOSTED;
|
||||
use windmill_common::BASE_URL;
|
||||
use windmill_common::{
|
||||
auth::{get_folders_for_user, get_groups_for_user},
|
||||
db::UserDB,
|
||||
error::{self, Error, JsonResult, Result},
|
||||
utils::{not_found_if_none, rd_string, require_admin, Pagination, StripPath},
|
||||
};
|
||||
use windmill_common::{BASE_URL, HUB_BASE_URL};
|
||||
use windmill_git_sync::handle_deployment_metadata;
|
||||
|
||||
const COOKIE_PATH: &str = "/";
|
||||
@@ -157,6 +157,7 @@ pub struct GlobalUserInfo {
|
||||
operator_only: Option<bool>,
|
||||
first_time_user: bool,
|
||||
role_source: String,
|
||||
disabled: bool,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Debug)]
|
||||
@@ -213,6 +214,7 @@ pub struct EditUser {
|
||||
pub is_super_admin: Option<bool>,
|
||||
pub is_devops: Option<bool>,
|
||||
pub name: Option<String>,
|
||||
pub disabled: Option<bool>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
@@ -396,7 +398,7 @@ async fn list_users_as_super_admin(
|
||||
GlobalUserInfo,
|
||||
"WITH active_users AS (SELECT distinct username as email FROM (SELECT username, timestamp, operation FROM audit_partitioned UNION ALL SELECT username, timestamp, operation FROM audit) AS a WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),
|
||||
authors as (SELECT distinct email FROM usr WHERE usr.operator IS false)
|
||||
SELECT email, email NOT IN (SELECT email FROM authors) as operator_only, login_type::text, verified, super_admin, devops, name, company, username, first_time_user, role_source
|
||||
SELECT email, email NOT IN (SELECT email FROM authors) as operator_only, login_type::text, verified, super_admin, devops, name, company, username, first_time_user, role_source, disabled
|
||||
FROM password
|
||||
WHERE email IN (SELECT email FROM active_users)
|
||||
ORDER BY super_admin DESC, devops DESC
|
||||
@@ -409,7 +411,7 @@ async fn list_users_as_super_admin(
|
||||
} else {
|
||||
sqlx::query_as!(
|
||||
GlobalUserInfo,
|
||||
"SELECT email, login_type::text, verified, super_admin, devops, name, company, username, NULL::bool as operator_only, first_time_user, role_source FROM password ORDER BY super_admin DESC, devops DESC, email LIMIT \
|
||||
"SELECT email, login_type::text, verified, super_admin, devops, name, company, username, NULL::bool as operator_only, first_time_user, role_source, disabled FROM password ORDER BY super_admin DESC, devops DESC, email LIMIT \
|
||||
$1 OFFSET $2",
|
||||
per_page as i32,
|
||||
offset as i32
|
||||
@@ -577,12 +579,44 @@ async fn logout(
|
||||
}
|
||||
tx.commit().await?;
|
||||
if let Some(rd) = rd {
|
||||
Ok((StatusCode::TEMPORARY_REDIRECT, [(LOCATION, rd)]).into_response())
|
||||
if is_valid_logout_redirect(&rd).await {
|
||||
Ok((StatusCode::TEMPORARY_REDIRECT, [(LOCATION, rd)]).into_response())
|
||||
} else {
|
||||
tracing::warn!("Blocked logout redirect to non-whitelisted URL: {}", rd);
|
||||
Ok((StatusCode::OK, "logged out successfully".to_string()).into_response())
|
||||
}
|
||||
} else {
|
||||
Ok((StatusCode::OK, "logged out successfully".to_string()).into_response())
|
||||
}
|
||||
}
|
||||
|
||||
async fn is_valid_logout_redirect(rd: &str) -> bool {
|
||||
// Allow relative paths (same-origin redirects)
|
||||
if rd.starts_with('/') && !rd.starts_with("//") {
|
||||
return true;
|
||||
}
|
||||
let parsed = match url::Url::parse(rd) {
|
||||
Ok(u) => u,
|
||||
Err(_) => return false,
|
||||
};
|
||||
let host: &str = match parsed.host_str() {
|
||||
Some(h) => h,
|
||||
None => return false,
|
||||
};
|
||||
if host == "windmill.dev" || host.ends_with(".windmill.dev") {
|
||||
return true;
|
||||
}
|
||||
let hub_url = HUB_BASE_URL.read().await.clone();
|
||||
if let Ok(hub_parsed) = url::Url::parse(&hub_url) {
|
||||
if let Some(hub_host) = hub_parsed.host_str() {
|
||||
if host == hub_host {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
async fn whoami(
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
@@ -625,7 +659,7 @@ async fn global_whoami(
|
||||
) -> JsonResult<GlobalUserInfo> {
|
||||
let user = sqlx::query_as!(
|
||||
GlobalUserInfo,
|
||||
"SELECT email, login_type::TEXT, super_admin, devops, verified, name, company, username, NULL::bool as operator_only, first_time_user, role_source FROM password WHERE \
|
||||
"SELECT email, login_type::TEXT, super_admin, devops, verified, name, company, username, NULL::bool as operator_only, first_time_user, role_source, disabled FROM password WHERE \
|
||||
email = $1",
|
||||
email
|
||||
)
|
||||
@@ -648,6 +682,7 @@ async fn global_whoami(
|
||||
operator_only: None,
|
||||
first_time_user: false,
|
||||
role_source: "manual".to_string(),
|
||||
disabled: false,
|
||||
}))
|
||||
} else {
|
||||
Err(user.unwrap_err())
|
||||
@@ -1407,6 +1442,22 @@ async fn update_user(
|
||||
.await?;
|
||||
}
|
||||
|
||||
if let Some(d) = eu.disabled {
|
||||
sqlx::query_scalar!(
|
||||
"UPDATE password SET disabled = $1 WHERE email = $2",
|
||||
d,
|
||||
&email_to_update
|
||||
)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
if d {
|
||||
// Delete all tokens for immediate session revocation
|
||||
sqlx::query!("DELETE FROM token WHERE email = $1", &email_to_update)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
}
|
||||
|
||||
audit_log(
|
||||
&mut *tx,
|
||||
&authed,
|
||||
@@ -1429,6 +1480,9 @@ async fn delete_user(
|
||||
require_super_admin(&db, &authed.email).await?;
|
||||
let mut tx = db.begin().await?;
|
||||
|
||||
sqlx::query!("DELETE FROM token WHERE email = $1", &email_to_delete)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
sqlx::query!("DELETE FROM password WHERE email = $1", &email_to_delete)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
@@ -1687,7 +1741,7 @@ async fn login(
|
||||
};
|
||||
let email_w_h: Option<(String, String, bool)> = sqlx::query_as(
|
||||
"SELECT email, password_hash, super_admin FROM password WHERE email = $1 AND login_type = \
|
||||
'password'",
|
||||
'password' AND disabled = false",
|
||||
)
|
||||
.bind(&email)
|
||||
.fetch_optional(&mut *tx)
|
||||
@@ -1776,7 +1830,7 @@ async fn refresh_token(
|
||||
}
|
||||
|
||||
let super_admin = sqlx::query_scalar!(
|
||||
"SELECT super_admin FROM password WHERE email = $1",
|
||||
"SELECT super_admin FROM password WHERE email = $1 AND disabled = false",
|
||||
&authed.email
|
||||
)
|
||||
.fetch_optional(&mut *tx)
|
||||
|
||||
@@ -14,9 +14,11 @@ enterprise = ["windmill-common/enterprise"]
|
||||
private = ["windmill-common/private"]
|
||||
cloud = ["windmill-common/cloud"]
|
||||
no_auth = ["windmill-api-auth/no_auth"]
|
||||
parquet = ["windmill-object-store/parquet"]
|
||||
|
||||
[dependencies]
|
||||
windmill-common = { workspace = true, default-features = false }
|
||||
windmill-object-store = { workspace = true, optional = true }
|
||||
windmill-types.workspace = true
|
||||
windmill-api-auth.workspace = true
|
||||
windmill-api-users.workspace = true
|
||||
|
||||
@@ -35,7 +35,6 @@ use windmill_common::variables::{
|
||||
build_crypt, decrypt, encrypt, SECRET_SALT, WORKSPACE_CRYPT_CACHE,
|
||||
};
|
||||
use windmill_common::worker::{to_raw_value, CLOUD_HOSTED};
|
||||
#[cfg(feature = "enterprise")]
|
||||
use windmill_common::workspaces::GitRepositorySettings;
|
||||
#[cfg(feature = "enterprise")]
|
||||
use windmill_common::workspaces::WorkspaceDeploymentUISettings;
|
||||
@@ -82,6 +81,10 @@ pub fn workspaced_service() -> Router {
|
||||
.route("/get_dependents/*imported_path", get(get_dependents))
|
||||
.route("/get_dependents_amounts", post(get_dependents_amounts))
|
||||
.route("/get_settings", get(get_settings))
|
||||
.route(
|
||||
"/get_copilot_settings_state",
|
||||
get(get_copilot_settings_state),
|
||||
)
|
||||
.route("/get_deploy_to", get(get_deploy_to))
|
||||
.route("/edit_slack_command", post(edit_slack_command))
|
||||
.route(
|
||||
@@ -111,6 +114,7 @@ pub fn workspaced_service() -> Router {
|
||||
.route("/list_datatables", get(list_datatables))
|
||||
.route("/list_datatable_schemas", get(list_datatable_schemas))
|
||||
.route("/edit_datatable_config", post(edit_datatable_config))
|
||||
.route("/git_sync_enabled", get(get_git_sync_enabled))
|
||||
.route("/edit_git_sync_config", post(edit_git_sync_config))
|
||||
.route("/edit_git_sync_repository", post(edit_git_sync_repository))
|
||||
.route(
|
||||
@@ -257,6 +261,35 @@ pub struct WorkspaceSettings {
|
||||
pub public_app_execution_limit_per_minute: Option<i32>,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
|
||||
pub struct CopilotSettingsState {
|
||||
pub has_instance_ai_config: bool,
|
||||
pub uses_instance_ai_config: bool,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub instance_ai_summary: Option<InstanceAISummary>,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
|
||||
pub struct InstanceAIProviderSummary {
|
||||
pub provider: String,
|
||||
pub models: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
|
||||
pub struct InstanceAIModelSummary {
|
||||
pub provider: String,
|
||||
pub model: String,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
|
||||
pub struct InstanceAISummary {
|
||||
pub providers: Vec<InstanceAIProviderSummary>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub default_model: Option<InstanceAIModelSummary>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub code_completion_model: Option<InstanceAIModelSummary>,
|
||||
}
|
||||
|
||||
/// #[derive(sqlx::Type, Serialize, Deserialize, Debug)]
|
||||
// #[sqlx(type_name = "WORKSPACE_KEY_KIND", rename_all = "lowercase")]
|
||||
// pub enum WorkspaceKeyKind {
|
||||
@@ -608,15 +641,106 @@ async fn get_settings(
|
||||
.await
|
||||
.map_err(|e| Error::internal_err(format!("getting settings: {e:#}")))?;
|
||||
|
||||
let mut settings = not_found_if_none(settings, "workspace settings", &w_id)?;
|
||||
tx.commit().await?;
|
||||
|
||||
let mut settings = not_found_if_none(settings, "workspace settings", &w_id)?;
|
||||
if !authed.is_admin {
|
||||
settings.slack_oauth_client_secret = None;
|
||||
}
|
||||
Ok(Json(settings))
|
||||
}
|
||||
|
||||
async fn get_copilot_settings_state(
|
||||
authed: ApiAuthed,
|
||||
Path(w_id): Path<String>,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
) -> JsonResult<CopilotSettingsState> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let workspace_ai_config = sqlx::query_scalar!(
|
||||
"SELECT ai_config FROM workspace_settings WHERE workspace_id = $1",
|
||||
&w_id
|
||||
)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await
|
||||
.map_err(|e| Error::internal_err(format!("getting workspace ai settings: {e:#}")))?;
|
||||
let workspace_ai_config = not_found_if_none(workspace_ai_config, "workspace settings", &w_id)?;
|
||||
let instance_ai_config: Option<serde_json::Value> =
|
||||
sqlx::query_scalar("SELECT value FROM global_settings WHERE name = 'ai_config'")
|
||||
.fetch_optional(&mut *tx)
|
||||
.await
|
||||
.map_err(|e| Error::internal_err(format!("getting instance ai settings: {e:#}")))?;
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Json(build_copilot_settings_state(
|
||||
has_ai_providers(workspace_ai_config.as_ref()),
|
||||
instance_ai_config.as_ref(),
|
||||
)))
|
||||
}
|
||||
|
||||
pub fn has_ai_providers(config: Option<&serde_json::Value>) -> bool {
|
||||
config
|
||||
.and_then(|value| value.get("providers"))
|
||||
.and_then(|providers| providers.as_object())
|
||||
.map(|providers| !providers.is_empty())
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
pub fn build_copilot_settings_state(
|
||||
has_workspace_ai_config: bool,
|
||||
instance_ai_config: Option<&serde_json::Value>,
|
||||
) -> CopilotSettingsState {
|
||||
let has_instance_ai_config = has_ai_providers(instance_ai_config);
|
||||
CopilotSettingsState {
|
||||
has_instance_ai_config,
|
||||
uses_instance_ai_config: !has_workspace_ai_config && has_instance_ai_config,
|
||||
instance_ai_summary: build_instance_ai_summary(instance_ai_config),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn build_instance_ai_summary(config: Option<&serde_json::Value>) -> Option<InstanceAISummary> {
|
||||
let config = config?;
|
||||
if !has_ai_providers(Some(config)) {
|
||||
return None;
|
||||
}
|
||||
let providers = config.get("providers")?.as_object()?;
|
||||
|
||||
let mut provider_summaries = providers
|
||||
.iter()
|
||||
.map(|(provider, provider_config)| InstanceAIProviderSummary {
|
||||
provider: provider.clone(),
|
||||
models: provider_config
|
||||
.get("models")
|
||||
.and_then(|models| models.as_array())
|
||||
.map(|models| {
|
||||
models
|
||||
.iter()
|
||||
.filter_map(|model| model.as_str().map(ToOwned::to_owned))
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.unwrap_or_default(),
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
provider_summaries.sort_by(|left, right| left.provider.cmp(&right.provider));
|
||||
|
||||
Some(InstanceAISummary {
|
||||
providers: provider_summaries,
|
||||
default_model: extract_instance_ai_model_summary(config, "default_model"),
|
||||
code_completion_model: extract_instance_ai_model_summary(config, "code_completion_model"),
|
||||
})
|
||||
}
|
||||
|
||||
fn extract_instance_ai_model_summary(
|
||||
config: &serde_json::Value,
|
||||
key: &str,
|
||||
) -> Option<InstanceAIModelSummary> {
|
||||
let model_config = config.get(key)?.as_object()?;
|
||||
Some(InstanceAIModelSummary {
|
||||
provider: model_config.get("provider")?.as_str()?.to_owned(),
|
||||
model: model_config.get("model")?.as_str()?.to_owned(),
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct DeployTo {
|
||||
deploy_to: Option<String>,
|
||||
@@ -1471,24 +1595,20 @@ async fn edit_datatable_config(
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub struct EditGitSyncConfig {
|
||||
#[cfg(feature = "enterprise")]
|
||||
pub git_sync_settings: Option<WorkspaceGitSyncSettings>,
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub struct EditGitSyncRepository {
|
||||
pub git_repo_resource_path: String,
|
||||
pub repository: GitRepositorySettings,
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub struct DeleteGitSyncRepositoryRequest {
|
||||
pub git_repo_resource_path: String,
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
fn validate_git_repo_resource_path(path: &str) -> Result<()> {
|
||||
// Resource paths should follow the pattern: $res:f/<folder>/<name> or $res:u/<username>/<name>
|
||||
if path.is_empty() {
|
||||
@@ -1537,7 +1657,6 @@ fn validate_git_repo_resource_path(path: &str) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
fn cleanup_legacy_git_sync_settings_in_memory(
|
||||
git_sync_settings: &mut windmill_common::workspaces::WorkspaceGitSyncSettings,
|
||||
workspace_id: &str,
|
||||
@@ -1564,18 +1683,72 @@ fn cleanup_legacy_git_sync_settings_in_memory(
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
async fn edit_git_sync_config(
|
||||
_authed: ApiAuthed,
|
||||
Extension(_db): Extension<DB>,
|
||||
Path(_w_id): Path<String>,
|
||||
Json(_new_config): Json<EditGitSyncConfig>,
|
||||
) -> Result<String> {
|
||||
return Err(Error::BadRequest(
|
||||
"Git sync is only available on Windmill Enterprise Edition".to_string(),
|
||||
));
|
||||
const CE_GIT_SYNC_MAX_USERS: i64 = 2;
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
async fn check_git_sync_access(_db: &DB, _w_id: &str) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
async fn check_git_sync_access(db: &DB, w_id: &str) -> Result<()> {
|
||||
let user_count: i64 = sqlx::query_scalar!(
|
||||
"SELECT COUNT(*) FROM usr WHERE workspace_id = $1 AND disabled = false",
|
||||
w_id
|
||||
)
|
||||
.fetch_one(db)
|
||||
.await?
|
||||
.unwrap_or(0);
|
||||
|
||||
if user_count > CE_GIT_SYNC_MAX_USERS {
|
||||
return Err(Error::BadRequest(format!(
|
||||
"Git sync is available for workspaces with up to {} members. \
|
||||
Upgrade to Windmill Enterprise Edition for unlimited workspace members.",
|
||||
CE_GIT_SYNC_MAX_USERS
|
||||
)));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
async fn get_git_sync_enabled(
|
||||
_authed: ApiAuthed,
|
||||
Extension(_db): Extension<DB>,
|
||||
Path(_w_id): Path<String>,
|
||||
) -> JsonResult<serde_json::Value> {
|
||||
Ok(Json(serde_json::json!({
|
||||
"enabled": true,
|
||||
"reason": "enterprise",
|
||||
"max_repos": null,
|
||||
"user_count": null,
|
||||
"max_users": null,
|
||||
})))
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
async fn get_git_sync_enabled(
|
||||
_authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
) -> JsonResult<serde_json::Value> {
|
||||
let user_count: i64 = sqlx::query_scalar!(
|
||||
"SELECT COUNT(*) FROM usr WHERE workspace_id = $1 AND disabled = false",
|
||||
&w_id
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await?
|
||||
.unwrap_or(0);
|
||||
|
||||
let enabled = user_count <= CE_GIT_SYNC_MAX_USERS;
|
||||
Ok(Json(serde_json::json!({
|
||||
"enabled": enabled,
|
||||
"reason": if enabled { Some("free_tier") } else { None::<&str> },
|
||||
"max_repos": if enabled { Some(1) } else { None::<i32> },
|
||||
"user_count": user_count,
|
||||
"max_users": CE_GIT_SYNC_MAX_USERS,
|
||||
})))
|
||||
}
|
||||
|
||||
async fn edit_git_sync_config(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
@@ -1584,6 +1757,7 @@ async fn edit_git_sync_config(
|
||||
Json(new_config): Json<EditGitSyncConfig>,
|
||||
) -> Result<String> {
|
||||
require_admin(is_admin, &username)?;
|
||||
check_git_sync_access(&db, &w_id).await?;
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
|
||||
@@ -1640,19 +1814,6 @@ async fn edit_git_sync_config(
|
||||
Ok(format!("Edit git sync config for workspace {}", &w_id))
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
async fn edit_git_sync_repository(
|
||||
_authed: ApiAuthed,
|
||||
Extension(_db): Extension<DB>,
|
||||
Path(_w_id): Path<String>,
|
||||
Json(_new_config): Json<serde_json::Value>,
|
||||
) -> Result<String> {
|
||||
return Err(Error::BadRequest(
|
||||
"Git sync is only available on Windmill Enterprise Edition".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
async fn edit_git_sync_repository(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
@@ -1661,10 +1822,19 @@ async fn edit_git_sync_repository(
|
||||
Json(new_config): Json<EditGitSyncRepository>,
|
||||
) -> Result<String> {
|
||||
require_admin(is_admin, &username)?;
|
||||
check_git_sync_access(&db, &w_id).await?;
|
||||
|
||||
// Validate the resource path format
|
||||
validate_git_repo_resource_path(&new_config.git_repo_resource_path)?;
|
||||
|
||||
// Promotion mode: EE only
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
if new_config.repository.use_individual_branch.unwrap_or(false) {
|
||||
return Err(Error::BadRequest(
|
||||
"Promotion mode is an Enterprise Edition feature".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
|
||||
// First, get the current git sync settings
|
||||
@@ -1686,6 +1856,20 @@ async fn edit_git_sync_repository(
|
||||
WorkspaceGitSyncSettings::default()
|
||||
};
|
||||
|
||||
// Multi-repo: EE only
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
{
|
||||
let is_new = !git_sync_settings
|
||||
.repositories
|
||||
.iter()
|
||||
.any(|r| r.git_repo_resource_path == new_config.git_repo_resource_path);
|
||||
if is_new && !git_sync_settings.repositories.is_empty() {
|
||||
return Err(Error::BadRequest(
|
||||
"Multiple git sync repositories is an Enterprise Edition feature".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
// Audit log before we move the repository
|
||||
audit_log(
|
||||
&mut *tx,
|
||||
@@ -1769,19 +1953,6 @@ async fn edit_git_sync_repository(
|
||||
))
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "enterprise"))]
|
||||
async fn delete_git_sync_repository(
|
||||
_authed: ApiAuthed,
|
||||
Extension(_db): Extension<DB>,
|
||||
Path(_w_id): Path<String>,
|
||||
Json(_request): Json<serde_json::Value>,
|
||||
) -> Result<String> {
|
||||
return Err(Error::BadRequest(
|
||||
"Git sync is only available on Windmill Enterprise Edition".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
#[cfg(feature = "enterprise")]
|
||||
async fn delete_git_sync_repository(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
@@ -1791,7 +1962,7 @@ async fn delete_git_sync_repository(
|
||||
) -> Result<String> {
|
||||
require_admin(is_admin, &username)?;
|
||||
|
||||
// For deletion, only validate that path is not empty to allow cleanup of malformed entries
|
||||
// No check_git_sync_access here — admins should always be able to delete/clean up repos
|
||||
if request.git_repo_resource_path.is_empty() {
|
||||
return Err(Error::BadRequest(
|
||||
"Resource path cannot be empty".to_string(),
|
||||
@@ -2109,22 +2280,29 @@ async fn edit_default_app(
|
||||
#[derive(Serialize)]
|
||||
struct WorkspaceDefaultApp {
|
||||
pub default_app_path: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub default_app_raw: Option<bool>,
|
||||
}
|
||||
async fn get_default_app(
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
) -> JsonResult<WorkspaceDefaultApp> {
|
||||
let mut tx = db.begin().await?;
|
||||
let default_app_path = sqlx::query_scalar!(
|
||||
"SELECT default_app FROM workspace_settings WHERE workspace_id = $1",
|
||||
let row = sqlx::query!(
|
||||
"SELECT ws.default_app AS default_app_path, av.raw_app AS \"default_app_raw: Option<bool>\"
|
||||
FROM workspace_settings ws
|
||||
LEFT JOIN app ON app.path = ws.default_app AND app.workspace_id = ws.workspace_id
|
||||
LEFT JOIN app_version av ON av.id = app.versions[array_upper(app.versions, 1)]
|
||||
WHERE ws.workspace_id = $1",
|
||||
&w_id
|
||||
)
|
||||
.fetch_one(&mut *tx)
|
||||
.fetch_one(&db)
|
||||
.await
|
||||
.map_err(|err| Error::internal_err(format!("getting default_app: {err}")))?;
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Json(WorkspaceDefaultApp { default_app_path }))
|
||||
Ok(Json(WorkspaceDefaultApp {
|
||||
default_app_path: row.default_app_path,
|
||||
default_app_raw: row.default_app_raw,
|
||||
}))
|
||||
}
|
||||
|
||||
async fn edit_error_handler(
|
||||
@@ -3296,6 +3474,11 @@ async fn clone_apps(
|
||||
.fetch_all(&mut **tx)
|
||||
.await?;
|
||||
|
||||
let mut cloned_from_db: std::collections::HashSet<(i64, String)> = HashSet::new();
|
||||
for bundle in &bundles {
|
||||
cloned_from_db.insert((bundle.app_version_id, bundle.file_type.clone()));
|
||||
}
|
||||
|
||||
for bundle in bundles {
|
||||
if let Some(&new_version_id) = version_id_mapping.get(&bundle.app_version_id) {
|
||||
sqlx::query!(
|
||||
@@ -3310,6 +3493,66 @@ async fn clone_apps(
|
||||
.await?;
|
||||
}
|
||||
}
|
||||
|
||||
// Clone bundles from S3 for versions not found in DB
|
||||
#[cfg(all(feature = "enterprise", feature = "parquet"))]
|
||||
{
|
||||
let object_store = windmill_object_store::get_object_store().await;
|
||||
if let Some(os) = object_store {
|
||||
for (&old_version_id, &new_version_id) in &version_id_mapping {
|
||||
for file_type in &["js", "css"] {
|
||||
if cloned_from_db.contains(&(old_version_id, file_type.to_string())) {
|
||||
continue;
|
||||
}
|
||||
let src_path = format!(
|
||||
"/app_bundles/{}/{}.{}",
|
||||
source_workspace_id, old_version_id, file_type
|
||||
);
|
||||
let get_result = os
|
||||
.get(&windmill_object_store::object_store_reexports::Path::from(
|
||||
src_path,
|
||||
))
|
||||
.await;
|
||||
match get_result {
|
||||
Ok(result) => {
|
||||
let data = result.bytes().await.map_err(
|
||||
windmill_object_store::object_store_error_to_error,
|
||||
)?;
|
||||
let dst_path = format!(
|
||||
"/app_bundles/{}/{}.{}",
|
||||
target_workspace_id, new_version_id, file_type
|
||||
);
|
||||
os.put(
|
||||
&windmill_object_store::object_store_reexports::Path::from(
|
||||
dst_path.clone(),
|
||||
),
|
||||
data.into(),
|
||||
)
|
||||
.await
|
||||
.map_err(
|
||||
windmill_object_store::object_store_error_to_error,
|
||||
)?;
|
||||
tracing::info!(
|
||||
"Cloned app bundle from S3: {}.{} -> {}.{}",
|
||||
old_version_id,
|
||||
file_type,
|
||||
new_version_id,
|
||||
file_type
|
||||
);
|
||||
}
|
||||
Err(windmill_object_store::object_store_reexports::ObjectStoreError::NotFound { .. }) => {
|
||||
// No bundle in S3 for this version/type, skip
|
||||
}
|
||||
Err(e) => {
|
||||
return Err(
|
||||
windmill_object_store::object_store_error_to_error(e),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Update app versions arrays
|
||||
|
||||
@@ -18,7 +18,7 @@ agent_worker_server = ["dep:windmill-worker", "dep:windmill-api-agent-workers"]
|
||||
enterprise_saml = ["dep:samael", "dep:libxml"]
|
||||
benchmark = []
|
||||
embedding = ["windmill-api-embeddings/embedding"]
|
||||
parquet = ["dep:datafusion", "windmill-common/parquet", "windmill-object-store/parquet", "windmill-worker?/parquet", "windmill-api-users/parquet", "windmill-api-settings/parquet", "dep:aws-sigv4", "dep:aws-sdk-config"]
|
||||
parquet = ["dep:datafusion", "windmill-common/parquet", "windmill-object-store/parquet", "windmill-worker?/parquet", "windmill-api-users/parquet", "windmill-api-settings/parquet", "windmill-api-workspaces/parquet", "dep:aws-sigv4", "dep:aws-sdk-config"]
|
||||
prometheus = ["windmill-common/prometheus", "windmill-queue/prometheus", "dep:prometheus", "windmill-worker?/prometheus"]
|
||||
openidconnect = ["dep:openidconnect", "windmill-common/openidconnect", "windmill-store/openidconnect"]
|
||||
tantivy = ["dep:windmill-indexer"]
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: "3.0.3"
|
||||
|
||||
info:
|
||||
version: 1.662.0
|
||||
version: 1.664.0
|
||||
title: Windmill API
|
||||
|
||||
contact:
|
||||
@@ -588,6 +588,8 @@ paths:
|
||||
type: boolean
|
||||
name:
|
||||
type: string
|
||||
disabled:
|
||||
type: boolean
|
||||
responses:
|
||||
"200":
|
||||
description: user updated
|
||||
@@ -3191,9 +3193,49 @@ paths:
|
||||
"200":
|
||||
description: status
|
||||
content:
|
||||
text/plain:
|
||||
application/json:
|
||||
schema:
|
||||
type: string
|
||||
type: object
|
||||
properties:
|
||||
effective_ai_config:
|
||||
$ref: "#/components/schemas/AIConfig"
|
||||
has_instance_ai_config:
|
||||
type: boolean
|
||||
uses_instance_ai_config:
|
||||
type: boolean
|
||||
instance_ai_summary:
|
||||
$ref: "#/components/schemas/InstanceAISummary"
|
||||
required:
|
||||
- effective_ai_config
|
||||
- has_instance_ai_config
|
||||
- uses_instance_ai_config
|
||||
|
||||
/w/{workspace}/workspaces/get_copilot_settings_state:
|
||||
get:
|
||||
summary: get copilot settings state
|
||||
operationId: getCopilotSettingsState
|
||||
tags:
|
||||
- workspace
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
|
||||
responses:
|
||||
"200":
|
||||
description: status
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
has_instance_ai_config:
|
||||
type: boolean
|
||||
uses_instance_ai_config:
|
||||
type: boolean
|
||||
instance_ai_summary:
|
||||
$ref: "#/components/schemas/InstanceAISummary"
|
||||
required:
|
||||
- has_instance_ai_config
|
||||
- uses_instance_ai_config
|
||||
|
||||
/w/{workspace}/workspaces/get_copilot_info:
|
||||
get:
|
||||
@@ -3390,6 +3432,37 @@ paths:
|
||||
application/json:
|
||||
schema: {}
|
||||
|
||||
/w/{workspace}/workspaces/git_sync_enabled:
|
||||
get:
|
||||
summary: Check if git sync is available for this workspace
|
||||
operationId: getGitSyncEnabled
|
||||
tags:
|
||||
- workspace
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
responses:
|
||||
"200":
|
||||
description: Git sync availability status
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
enabled:
|
||||
type: boolean
|
||||
reason:
|
||||
type: string
|
||||
nullable: true
|
||||
max_repos:
|
||||
type: integer
|
||||
nullable: true
|
||||
user_count:
|
||||
type: integer
|
||||
nullable: true
|
||||
max_users:
|
||||
type: integer
|
||||
nullable: true
|
||||
|
||||
/w/{workspace}/workspaces/edit_git_sync_config:
|
||||
post:
|
||||
summary: edit workspace git sync settings
|
||||
@@ -3662,6 +3735,8 @@ paths:
|
||||
properties:
|
||||
default_app_path:
|
||||
type: string
|
||||
default_app_raw:
|
||||
type: boolean
|
||||
|
||||
/w/{workspace}/workspaces/usage:
|
||||
get:
|
||||
@@ -6867,6 +6942,60 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/scripts/list_dedicated_with_deps:
|
||||
get:
|
||||
summary: list dedicated worker scripts with workspace dependency annotations
|
||||
operationId: listDedicatedWithDeps
|
||||
tags:
|
||||
- script
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
responses:
|
||||
"200":
|
||||
description: list of dedicated scripts with their workspace dependency names
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
path:
|
||||
type: string
|
||||
language:
|
||||
type: string
|
||||
enum:
|
||||
- python3
|
||||
- deno
|
||||
- go
|
||||
- bash
|
||||
- powershell
|
||||
- postgresql
|
||||
- mysql
|
||||
- bigquery
|
||||
- snowflake
|
||||
- mssql
|
||||
- graphql
|
||||
- nativets
|
||||
- bun
|
||||
- bunnative
|
||||
- php
|
||||
- rust
|
||||
- ansible
|
||||
- csharp
|
||||
- oracledb
|
||||
- duckdb
|
||||
- java
|
||||
- ruby
|
||||
workspace_dep_names:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
required:
|
||||
- path
|
||||
- language
|
||||
- workspace_dep_names
|
||||
|
||||
/w/{workspace}/scripts/raw/p/{path}:
|
||||
get:
|
||||
summary: raw script by path
|
||||
@@ -6988,6 +7117,83 @@ paths:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
/w/{workspace}/scripts/raw_temp/store:
|
||||
post:
|
||||
summary: store raw script content temporarily for CLI lock generation
|
||||
operationId: storeRawScriptTemp
|
||||
tags:
|
||||
- script
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
requestBody:
|
||||
description: script content to store
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: string
|
||||
responses:
|
||||
"200":
|
||||
description: hash of stored content
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/scripts/raw_temp/diff:
|
||||
post:
|
||||
summary: diff local script hashes against deployed versions
|
||||
operationId: diffRawScriptsWithDeployed
|
||||
tags:
|
||||
- script
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
requestBody:
|
||||
description: scripts and workspace deps to diff against deployed versions
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required:
|
||||
- scripts
|
||||
properties:
|
||||
scripts:
|
||||
description: map of script path to SHA256 content hash
|
||||
type: object
|
||||
additionalProperties:
|
||||
type: string
|
||||
workspace_deps:
|
||||
description: workspace dependencies to diff
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
required:
|
||||
- path
|
||||
- language
|
||||
- hash
|
||||
properties:
|
||||
path:
|
||||
description: CLI path (e.g. dependencies/package.json)
|
||||
type: string
|
||||
language:
|
||||
$ref: "#/components/schemas/ScriptLang"
|
||||
name:
|
||||
description: named workspace dependency (null for default)
|
||||
type: string
|
||||
hash:
|
||||
description: SHA256 content hash
|
||||
type: string
|
||||
responses:
|
||||
"200":
|
||||
description: list of paths that differ from deployed versions
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/jobs/list_selected_job_groups:
|
||||
# We use post because sending a huge array as a query param can produce
|
||||
# URLs that may be too long
|
||||
@@ -10494,6 +10700,10 @@ paths:
|
||||
in: query
|
||||
schema:
|
||||
type: boolean
|
||||
- name: fast
|
||||
in: query
|
||||
schema:
|
||||
type: boolean
|
||||
|
||||
responses:
|
||||
"200":
|
||||
@@ -10971,6 +11181,129 @@ paths:
|
||||
"200":
|
||||
description: Interactive slack approval message sent successfully
|
||||
|
||||
/w/{workspace}/jobs_u/flow/resume_suspended/{job_id}:
|
||||
post:
|
||||
summary: resume or cancel a suspended flow/WAC job
|
||||
description: >
|
||||
Resume or cancel a suspended flow/WAC job. Uses approval rules to
|
||||
determine authorization. Either a valid approval_token or an
|
||||
authenticated session is required.
|
||||
operationId: resumeSuspended
|
||||
tags:
|
||||
- job
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- name: job_id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
payload:
|
||||
description: payload to send to the resumed job
|
||||
approval_token:
|
||||
type: string
|
||||
description: approval token for unauthenticated access
|
||||
approved:
|
||||
type: boolean
|
||||
description: whether to approve (true) or cancel (false) the job
|
||||
default: true
|
||||
responses:
|
||||
"201":
|
||||
description: job resumed
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/jobs_u/flow/approval_info/{job_id}:
|
||||
get:
|
||||
summary: get approval info for a suspended flow/WAC job
|
||||
description: >
|
||||
Get approval info for a suspended flow/WAC job. Returns form schema,
|
||||
approval rules, and whether the current user can approve. Either a
|
||||
valid token query parameter or an authenticated session is required.
|
||||
operationId: getApprovalInfo
|
||||
tags:
|
||||
- job
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- name: job_id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- name: token
|
||||
in: query
|
||||
required: false
|
||||
schema:
|
||||
type: string
|
||||
description: approval token for unauthenticated access
|
||||
responses:
|
||||
"200":
|
||||
description: approval info
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required:
|
||||
- flow_id
|
||||
- can_approve
|
||||
- user_auth_required
|
||||
- approvers
|
||||
properties:
|
||||
flow_id:
|
||||
type: string
|
||||
format: uuid
|
||||
form_schema:
|
||||
description: form schema for the approval step
|
||||
description:
|
||||
description: description of the approval step
|
||||
approval_conditions:
|
||||
type: object
|
||||
properties:
|
||||
user_auth_required:
|
||||
type: boolean
|
||||
user_groups_required:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
self_approval_disabled:
|
||||
type: boolean
|
||||
required:
|
||||
- user_auth_required
|
||||
- user_groups_required
|
||||
- self_approval_disabled
|
||||
can_approve:
|
||||
type: boolean
|
||||
description: whether the current user/token holder can approve
|
||||
user_auth_required:
|
||||
type: boolean
|
||||
description: whether user authentication is required to approve
|
||||
hide_cancel:
|
||||
type: boolean
|
||||
description: whether to hide the cancel button in the UI
|
||||
approvers:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
required:
|
||||
- resume_id
|
||||
- approver
|
||||
properties:
|
||||
resume_id:
|
||||
type: integer
|
||||
approver:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/jobs_u/resume/{id}/{resume_id}/{signature}:
|
||||
get:
|
||||
summary: resume a job for a suspended flow
|
||||
@@ -17187,6 +17520,27 @@ paths:
|
||||
description: count of log lines that matched the query per hostname
|
||||
type: object
|
||||
|
||||
/srch/index/storage/disk:
|
||||
get:
|
||||
summary: Get index disk storage sizes from the indexer.
|
||||
operationId: getIndexDiskStorageSizes
|
||||
tags:
|
||||
- indexSearch
|
||||
responses:
|
||||
"200":
|
||||
description: disk storage sizes for each index
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
job_index_disk_size_bytes:
|
||||
type: integer
|
||||
nullable: true
|
||||
log_index_disk_size_bytes:
|
||||
type: integer
|
||||
nullable: true
|
||||
|
||||
/indexer/delete/{idx_name}:
|
||||
delete:
|
||||
summary: Clear an index and restart the indexer.
|
||||
@@ -18715,6 +19069,33 @@ components:
|
||||
minimum: 1
|
||||
maximum: 2000000
|
||||
|
||||
InstanceAIProviderSummary:
|
||||
type: object
|
||||
properties:
|
||||
provider:
|
||||
$ref: "#/components/schemas/AIProvider"
|
||||
models:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
required:
|
||||
- provider
|
||||
- models
|
||||
|
||||
InstanceAISummary:
|
||||
type: object
|
||||
properties:
|
||||
providers:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/InstanceAIProviderSummary"
|
||||
default_model:
|
||||
$ref: "#/components/schemas/AIProviderModel"
|
||||
code_completion_model:
|
||||
$ref: "#/components/schemas/AIProviderModel"
|
||||
required:
|
||||
- providers
|
||||
|
||||
Alert:
|
||||
type: object
|
||||
properties:
|
||||
@@ -23091,6 +23472,8 @@ components:
|
||||
role_source:
|
||||
type: string
|
||||
enum: ["manual", "instance_group"]
|
||||
disabled:
|
||||
type: boolean
|
||||
|
||||
required:
|
||||
- email
|
||||
@@ -23099,6 +23482,7 @@ components:
|
||||
- verified
|
||||
- first_time_user
|
||||
- role_source
|
||||
- disabled
|
||||
|
||||
Flow:
|
||||
allOf:
|
||||
@@ -24515,6 +24899,10 @@ components:
|
||||
type: string
|
||||
nullable: true
|
||||
description: Error message if the trigger is in an error state
|
||||
summary:
|
||||
type: string
|
||||
nullable: true
|
||||
description: Short summary to be displayed when listed
|
||||
required:
|
||||
- external_id
|
||||
- workspace_id
|
||||
@@ -24549,6 +24937,10 @@ components:
|
||||
type: string
|
||||
nullable: true
|
||||
description: Error message if the trigger is in an error state
|
||||
summary:
|
||||
type: string
|
||||
nullable: true
|
||||
description: Short summary to be displayed when listed
|
||||
external_data:
|
||||
type: object
|
||||
description: Configuration data from the external service
|
||||
@@ -24641,6 +25033,10 @@ components:
|
||||
type: object
|
||||
description: Service-specific configuration (e.g., event types, filters)
|
||||
additionalProperties: true
|
||||
summary:
|
||||
type: string
|
||||
nullable: true
|
||||
description: Short summary to be displayed when listed
|
||||
required:
|
||||
- script_path
|
||||
- is_flow
|
||||
|
||||
+169
-44
@@ -16,6 +16,7 @@ use serde_json::{json, value::RawValue};
|
||||
use std::collections::HashMap;
|
||||
use std::time::Duration;
|
||||
use windmill_audit::{audit_oss::audit_log, ActionKind};
|
||||
use windmill_common::ai_cache::current_instance_ai_config_revision;
|
||||
use windmill_common::ai_providers::{
|
||||
empty_string_as_none, AIPlatform, AIProvider, ProviderConfig, ProviderModel,
|
||||
};
|
||||
@@ -127,6 +128,10 @@ lazy_static::lazy_static! {
|
||||
};
|
||||
}
|
||||
|
||||
pub(crate) fn invalidate_ai_request_cache_for_workspace(workspace_id: &str) {
|
||||
AI_REQUEST_CACHE.retain(|(cached_workspace_id, _), _| cached_workspace_id != workspace_id);
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
struct AIOAuthResource {
|
||||
client_id: String,
|
||||
@@ -373,8 +378,7 @@ impl AIRequestConfig {
|
||||
|
||||
let is_azure = provider.is_azure_openai(base_url);
|
||||
let is_anthropic = matches!(provider, AIProvider::Anthropic);
|
||||
let is_anthropic_vertex =
|
||||
is_anthropic && self.platform == AIPlatform::GoogleVertexAi;
|
||||
let is_anthropic_vertex = is_anthropic && self.platform == AIPlatform::GoogleVertexAi;
|
||||
let is_anthropic_sdk = headers.get("X-Anthropic-SDK").is_some();
|
||||
let is_google_ai = matches!(provider, AIProvider::GoogleAI);
|
||||
|
||||
@@ -483,18 +487,27 @@ impl AIRequestConfig {
|
||||
pub struct ExpiringAIRequestConfig {
|
||||
config: AIRequestConfig,
|
||||
expires_at: std::time::Instant,
|
||||
instance_ai_config_revision: Option<u64>,
|
||||
}
|
||||
|
||||
impl ExpiringAIRequestConfig {
|
||||
fn new(config: AIRequestConfig) -> Self {
|
||||
Self { config, expires_at: std::time::Instant::now() + std::time::Duration::from_secs(60) }
|
||||
fn new(config: AIRequestConfig, instance_ai_config_revision: Option<u64>) -> Self {
|
||||
Self {
|
||||
config,
|
||||
expires_at: std::time::Instant::now() + std::time::Duration::from_secs(60),
|
||||
instance_ai_config_revision,
|
||||
}
|
||||
}
|
||||
|
||||
fn is_expired(&self) -> bool {
|
||||
self.expires_at < std::time::Instant::now()
|
||||
|| self
|
||||
.instance_ai_config_revision
|
||||
.is_some_and(|revision| revision != current_instance_ai_config_revision())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug)]
|
||||
#[derive(Serialize, Deserialize, Debug, Default)]
|
||||
pub struct AIConfig {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub providers: Option<HashMap<AIProvider, ProviderConfig>>,
|
||||
@@ -508,6 +521,14 @@ pub struct AIConfig {
|
||||
pub max_tokens_per_model: Option<HashMap<String, i32>>,
|
||||
}
|
||||
|
||||
impl AIConfig {
|
||||
pub fn has_providers(&self) -> bool {
|
||||
self.providers
|
||||
.as_ref()
|
||||
.is_some_and(|providers| !providers.is_empty())
|
||||
}
|
||||
}
|
||||
|
||||
/// Anthropic API version for Google Vertex AI
|
||||
const ANTHROPIC_VERSION_VERTEX: &str = "vertex-2023-10-16";
|
||||
|
||||
@@ -762,47 +783,76 @@ async fn proxy(
|
||||
request_cache.config
|
||||
}
|
||||
_ => {
|
||||
let (resource_path, save_to_cache) = if let Some(resource_path) = forced_resource_path {
|
||||
// forced resource path
|
||||
(resource_path, false)
|
||||
} else {
|
||||
let ai_config = sqlx::query_scalar!(
|
||||
"SELECT ai_config FROM workspace_settings WHERE workspace_id = $1",
|
||||
&w_id
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await?;
|
||||
let (resource_path, save_to_cache, resource_workspace, instance_ai_config_revision) =
|
||||
if let Some(resource_path) = forced_resource_path {
|
||||
// forced resource path
|
||||
(resource_path, false, w_id.clone(), None)
|
||||
} else {
|
||||
let workspace_ai_config = sqlx::query_scalar!(
|
||||
"SELECT ai_config FROM workspace_settings WHERE workspace_id = $1",
|
||||
&w_id
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await?;
|
||||
|
||||
if ai_config.is_none() {
|
||||
return Err(Error::internal_err(
|
||||
"AI resource not configured".to_string(),
|
||||
));
|
||||
}
|
||||
let (ai_config_value, resource_workspace, instance_ai_config_revision) = {
|
||||
let ws_has_config = workspace_ai_config
|
||||
.as_ref()
|
||||
.and_then(|v| serde_json::from_value::<AIConfig>(v.clone()).ok())
|
||||
.is_some_and(|config| config.has_providers());
|
||||
|
||||
let mut ai_config = serde_json::from_value::<AIConfig>(ai_config.unwrap())
|
||||
.map_err(|e| Error::BadRequest(e.to_string()))?;
|
||||
if ws_has_config {
|
||||
(workspace_ai_config.unwrap(), w_id.clone(), None)
|
||||
} else {
|
||||
let instance_config = sqlx::query_scalar!(
|
||||
"SELECT value FROM global_settings WHERE name = 'ai_config'"
|
||||
)
|
||||
.fetch_optional(&db)
|
||||
.await?;
|
||||
|
||||
let provider_config = ai_config
|
||||
.providers
|
||||
.as_mut()
|
||||
.map(|providers| providers.remove(&provider))
|
||||
.flatten()
|
||||
.ok_or_else(|| {
|
||||
Error::BadRequest(format!("Provider {:?} not configured", provider))
|
||||
})?;
|
||||
match instance_config {
|
||||
Some(config) => (
|
||||
config,
|
||||
"admins".to_string(),
|
||||
Some(current_instance_ai_config_revision()),
|
||||
),
|
||||
None => {
|
||||
return Err(Error::internal_err(
|
||||
"AI resource not configured".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
if provider_config.resource_path.is_empty() {
|
||||
return Err(Error::BadRequest("Resource path is empty".to_string()));
|
||||
}
|
||||
let mut ai_config = serde_json::from_value::<AIConfig>(ai_config_value)
|
||||
.map_err(|e| Error::BadRequest(e.to_string()))?;
|
||||
|
||||
(provider_config.resource_path, true)
|
||||
};
|
||||
let provider_config = ai_config
|
||||
.providers
|
||||
.as_mut()
|
||||
.and_then(|providers| providers.remove(&provider))
|
||||
.ok_or_else(|| {
|
||||
Error::BadRequest(format!("Provider {:?} not configured", provider))
|
||||
})?;
|
||||
|
||||
let resource= sqlx::query_scalar!(
|
||||
"SELECT value as \"value: sqlx::types::Json<Box<RawValue>>\" FROM resource WHERE path = $1 AND workspace_id = $2",
|
||||
&resource_path,
|
||||
&w_id
|
||||
if provider_config.resource_path.is_empty() {
|
||||
return Err(Error::BadRequest("Resource path is empty".to_string()));
|
||||
}
|
||||
|
||||
(
|
||||
provider_config.resource_path,
|
||||
true,
|
||||
resource_workspace,
|
||||
instance_ai_config_revision,
|
||||
)
|
||||
};
|
||||
|
||||
let resource = sqlx::query_scalar::<_, Option<sqlx::types::Json<Box<RawValue>>>>(
|
||||
"SELECT value FROM resource WHERE path = $1 AND workspace_id = $2",
|
||||
)
|
||||
.bind(&resource_path)
|
||||
.bind(&resource_workspace)
|
||||
.fetch_optional(&db)
|
||||
.await?
|
||||
.ok_or_else(|| Error::NotFound(format!("Could not find the resource {}, update the resource path in the workspace settings", resource_path)))?
|
||||
@@ -811,11 +861,15 @@ async fn proxy(
|
||||
let resource = serde_json::from_str::<AIResource>(resource.0.get())
|
||||
.map_err(|e| Error::BadRequest(e.to_string()))?;
|
||||
|
||||
let request_config = AIRequestConfig::new(&provider, &db, &w_id, resource).await?;
|
||||
let request_config =
|
||||
AIRequestConfig::new(&provider, &db, &resource_workspace, resource).await?;
|
||||
if save_to_cache {
|
||||
AI_REQUEST_CACHE.insert(
|
||||
(w_id.clone(), provider.clone()),
|
||||
ExpiringAIRequestConfig::new(request_config.clone()),
|
||||
ExpiringAIRequestConfig::new(
|
||||
request_config.clone(),
|
||||
instance_ai_config_revision,
|
||||
),
|
||||
);
|
||||
}
|
||||
request_config
|
||||
@@ -858,9 +912,7 @@ async fn proxy(
|
||||
"chat/completions" => {
|
||||
crate::google::handle_google_ai_chat(&body, api_key, base_url, is_vertex).await
|
||||
}
|
||||
"models" => {
|
||||
crate::google::handle_google_ai_models(api_key, base_url, is_vertex).await
|
||||
}
|
||||
"models" => crate::google::handle_google_ai_models(api_key, base_url, is_vertex).await,
|
||||
_ => Err(Error::BadRequest(format!(
|
||||
"Unsupported Google AI path: {}",
|
||||
ai_path
|
||||
@@ -1005,3 +1057,76 @@ async fn proxy(
|
||||
};
|
||||
Ok((status_code, headers, body))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::sync::{LazyLock, Mutex};
|
||||
use windmill_common::ai_cache::bump_instance_ai_config_revision;
|
||||
use windmill_common::ai_providers::AIPlatform;
|
||||
|
||||
static TEST_LOCK: LazyLock<Mutex<()>> = LazyLock::new(|| Mutex::new(()));
|
||||
|
||||
fn sample_request_config() -> AIRequestConfig {
|
||||
AIRequestConfig {
|
||||
base_url: "https://example.com".to_string(),
|
||||
api_key: None,
|
||||
access_token: None,
|
||||
organization_id: None,
|
||||
user: None,
|
||||
region: None,
|
||||
aws_access_key_id: None,
|
||||
aws_secret_access_key: None,
|
||||
aws_session_token: None,
|
||||
platform: AIPlatform::Standard,
|
||||
enable_1m_context: false,
|
||||
custom_headers: HashMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalidates_all_cached_providers_for_workspace() {
|
||||
let _guard = TEST_LOCK.lock().unwrap();
|
||||
AI_REQUEST_CACHE.clear();
|
||||
AI_REQUEST_CACHE.insert(
|
||||
("workspace-a".to_string(), AIProvider::OpenAI),
|
||||
ExpiringAIRequestConfig::new(sample_request_config(), None),
|
||||
);
|
||||
AI_REQUEST_CACHE.insert(
|
||||
("workspace-a".to_string(), AIProvider::Anthropic),
|
||||
ExpiringAIRequestConfig::new(sample_request_config(), None),
|
||||
);
|
||||
AI_REQUEST_CACHE.insert(
|
||||
("workspace-b".to_string(), AIProvider::OpenAI),
|
||||
ExpiringAIRequestConfig::new(sample_request_config(), None),
|
||||
);
|
||||
|
||||
invalidate_ai_request_cache_for_workspace("workspace-a");
|
||||
|
||||
assert!(AI_REQUEST_CACHE
|
||||
.get(&("workspace-a".to_string(), AIProvider::OpenAI))
|
||||
.is_none());
|
||||
assert!(AI_REQUEST_CACHE
|
||||
.get(&("workspace-a".to_string(), AIProvider::Anthropic))
|
||||
.is_none());
|
||||
assert!(AI_REQUEST_CACHE
|
||||
.get(&("workspace-b".to_string(), AIProvider::OpenAI))
|
||||
.is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn instance_backed_cache_entries_expire_when_revision_changes() {
|
||||
let _guard = TEST_LOCK.lock().unwrap();
|
||||
AI_REQUEST_CACHE.clear();
|
||||
|
||||
let cached = ExpiringAIRequestConfig::new(
|
||||
sample_request_config(),
|
||||
Some(current_instance_ai_config_revision()),
|
||||
);
|
||||
assert!(!cached.is_expired());
|
||||
|
||||
bump_instance_ai_config_revision();
|
||||
|
||||
assert!(cached.is_expired());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -440,17 +440,29 @@ async fn get_raw_app_data(
|
||||
#[cfg(all(feature = "enterprise", feature = "parquet"))]
|
||||
if let Some(os) = object_store {
|
||||
let path = format!("/app_bundles/{}/{}.{}", w_id, id, file_type);
|
||||
let stream = os
|
||||
match os
|
||||
.get(&windmill_object_store::object_store_reexports::Path::from(
|
||||
path,
|
||||
))
|
||||
.await
|
||||
.map_err(windmill_object_store::object_store_error_to_error)?
|
||||
.bytes()
|
||||
.await
|
||||
.map_err(windmill_object_store::object_store_error_to_error)?;
|
||||
tracing::info!("stream: {}", stream.len());
|
||||
body = Some(Body::from(stream));
|
||||
{
|
||||
Ok(result) => {
|
||||
let stream = result
|
||||
.bytes()
|
||||
.await
|
||||
.map_err(windmill_object_store::object_store_error_to_error)?;
|
||||
tracing::info!("stream: {}", stream.len());
|
||||
body = Some(Body::from(stream));
|
||||
}
|
||||
Err(windmill_object_store::object_store_reexports::ObjectStoreError::NotFound {
|
||||
..
|
||||
}) => {
|
||||
// S3 key not found, fall through to DB lookup below
|
||||
}
|
||||
Err(e) => {
|
||||
return Err(windmill_object_store::object_store_error_to_error(e));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if body.is_none() {
|
||||
@@ -2053,6 +2065,7 @@ async fn execute_component(
|
||||
.triggerables_v2
|
||||
.as_ref()
|
||||
.ok_or_else(|| Error::BadRequest(format!("Policy is missing triggerables")))?;
|
||||
|
||||
let policy_triggerables = triggerables_v2
|
||||
.get(path) // start with `path` in case we can avoid the next` format!`.
|
||||
.or_else(|| triggerables_v2.get(&format!("{}:{}", payload.component, &path)))
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
use crate::db::ApiAuthed;
|
||||
use axum::{extract::Path, routing::post, Json, Router};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use windmill_common::{
|
||||
error::{Error, Result},
|
||||
query_builders::try_expand_internal_db_query,
|
||||
scripts::ScriptLang,
|
||||
};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new().route("/expand_marker", post(expand_marker))
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct ExpandMarkerRequest {
|
||||
language: ScriptLang,
|
||||
content: String,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct ExpandMarkerResponse {
|
||||
code: String,
|
||||
}
|
||||
|
||||
async fn expand_marker(
|
||||
_authed: ApiAuthed,
|
||||
Path(_w_id): Path<String>,
|
||||
Json(req): Json<ExpandMarkerRequest>,
|
||||
) -> Result<Json<ExpandMarkerResponse>> {
|
||||
match try_expand_internal_db_query(&req.content, &req.language) {
|
||||
Some(Ok(expanded)) => Ok(Json(ExpandMarkerResponse { code: expanded.code })),
|
||||
Some(Err(msg)) => Err(Error::BadRequest(msg)),
|
||||
None => Err(Error::BadRequest(
|
||||
"Content is not a WM_INTERNAL_DB marker".to_string(),
|
||||
)),
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user