From 68debab877c6dc8ee3732e0c23d467db85fd4584 Mon Sep 17 00:00:00 2001 From: Ruben Fiszel Date: Tue, 21 Jul 2026 17:10:29 +0200 Subject: [PATCH] feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library Fixes WIN-2214 Co-Authored-By: Claude Opus 4.8 (1M context) * chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish Revert the amqp_trigger additions to the shared windmill-utils-internal TriggerDeployKind and the frontend cross-workspace deploy adapter: the frontend installs the published npm package, which lacks the new kind until a release is cut. AMQP create/edit/delete/list/sync/capture are unaffected (they use local types); only cross-workspace deploy/merge of AMQP triggers waits on the package bump. Also document the at-most-once ack in the consumer loop. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts - ack AMQP deliveries only after successful dispatch; nack+requeue on failure - add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on workspace deletion (and the listener stops) - fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to WorkspaceDiffRow.kind - register AMQP in the generated workspace trigger tool (create_trigger) - drop banned $bindable defaults on optional props in the config section - add build_uri unit tests (encoding, ports, vhost) Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): stop AMQP poison-message loop and reconnect on transient drops Chaos testing against a live RabbitMQ broker showed the previous nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000 critical-error reports/sec) on a poison message, and any connection blip permanently disabled the trigger (lapin has no built-in reconnect). - on dispatch failure: nack+requeue then stop consuming; the listener framework re-lists the trigger after its ping goes stale (~15s), backing redelivery off to that cadence instead of a tight loop (verified: rate dropped from ~1000/s to ~1 per ~26s, message preserved) - on connection/stream error: stop and let the framework reconnect instead of disabling; persistent failures are still disabled via get_consumer (verified: a forced connection close now auto-reconnects and resumes) - finish the AI create-trigger action wiring for AMQP: add amqp to CreatedResourceTriggerKind, the action-card registry, and the drawer registry so the result card renders and its "Open" action works Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): complete AMQP frontend registries and defer merge rows - add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES - wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union, writeTriggerSchema, triggerServices, and the draft adapter - stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred until windmill-utils-internal is published — avoids a deploy that fails with "Unknown kind: amqp_trigger" - use design-system TextInput instead of raw in the config section Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): complete AMQP session/draft registries and constrain prefetch - add amqp to the session-deploy, draft-compare, preview-router, and copilot workspace-item registries so AMQP drafts/deploys/nav/path resolution work - include amqp_count in the MoveDrawer attached-trigger rename warning - replace the raw prefetch with a design-system TextInput bounded to an integer 1-65535 (backend u16) and block save on invalid values Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now mirrors the Kafka trigger's explicit reconnect loop instead of relying on the framework re-list (which disabled the trigger once get_consumer failed on a sustained outage): - get_consumer returns cheaply; consume owns a (re)connect loop that retries with a 30s backoff, reports a critical error every 10 failed attempts, and reports a recovered critical error once it reconnects — never disabling the trigger on a connectivity failure - a consumer/stream error breaks out to reconnect rather than disabling - dispatch failure still nacks+requeues (at-least-once) with a short backoff to avoid a tight poison-message loop, keeping the connection alive Verified against a live RabbitMQ broker: killing the broker keeps the trigger enabled and retrying (attempt N), and restarting it auto-reconnects (logs "reconnected after N attempts") and resumes dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): complete AMQP capture registries and constrain prefetch contract - add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor from a capture button no longer throws "Unknown TriggerKind: amqp" - register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the script/flow CaptureButton menu - bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust u16) and regenerate clients/prompts - require a non-empty exchange name when the exchange binding is enabled - build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests) Co-Authored-By: Claude Opus 4.8 (1M context) * feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types - asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas - frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu, node presentation, event-trigger set, annotation keywords, and the editor/service registrations - git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync deployment stages only its .amqp_trigger.* file, not an unrelated same-path object - preprocessor starters: add the AMQP event to the generated TS/Python/PHP trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/ delivery_tag) Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists - fix a stray edit that corrupted the pre-existing MqttTriggerEditor import ($lib/... path) in PipelineTriggerEditors.svelte - reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0 as unlimited) and defensively skip basic_qos(0) in build_consumer (covers the capture path that bypasses CRUD validation) - recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never runs an AMQP-only node as a manual root without an event - add amqp to the preprocessor intro lists and both pipeline AI instructions Co-Authored-By: Claude Opus 4.8 (1M context) * fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists - extract a shared validate_amqp_options used by both CRUD validate_config and build_consumer, so capture configs (which bypass CRUD validation) also reject prefetch 0 instead of silently connecting with an unlimited buffer (+ unit tests for 0/1/65535/None) - add AMQP to the main script-writing preprocessor-sources prompt and the CLI triggers-skill guidance list Co-Authored-By: Claude Opus 4.8 (1M context) * docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text - keep the zero-prefetch rationale only on the shared validate_amqp_options doc; drop the redundant call-site comments - correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved") Co-Authored-By: Claude Opus 4.8 (1M context) * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore(migrations): consolidate the four AMQP migrations into one The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind, draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger table doesn't reference those enum types, so they can share a single migration instead of four. Verified applying cleanly in a single transaction on a fresh DB. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: windmill-internal-app[bot] --- ...281a22467010d2c4a0611c8ae02742336f433.json | 38 + ...d3be7a970d602d3809396d9253d7e5dc88152.json | 41 + ...bc7e9ee85289ff5c52af43166928337c257e7.json | 3 +- ...7b23d6dcf2e5364852a5e85b3c41c417bace2.json | 3 +- ...66f38ffe2f3a0958adea53e67757c6ac463ec.json | 3 +- ...69c3592218d34e58d617253e8b563bc661a7a.json | 17 + ...adbb2860ce52d37a56b2861de1215260ecff8.json | 6 +- ...fbe7db4887c4ac5295052c796bd78a7edc50b.json | 3 +- ...f518b42f4cdb33e034441015494bd26c172d2.json | 3 +- ...fa25bfb67dd641153d1ca23f7d0d2ae73624e.json | 3 +- ...cb3adcf0175d0a71013f2497397c1c8ccc619.json | 3 +- ...cad6d8995242dc20a4e14c1a8fae4fc6a9fd2.json | 3 +- ...2f7175cc6f2e02d9c565bb5159cae3e0b6835.json | 3 +- ...70f68bc0421f1aa228f80157adda63191d33b.json | 3 +- ...c23caf312ec377dd91e5307f00d3fb8ec325d.json | 3 +- ...08532a1020cabaf5e22325a1e05f0491d785c.json | 3 +- ...ac91ee793ad6f5ceed5b3eda9b48617c5434d.json | 3 +- ...e35adf640d44d607e62a1fb296e38ac3fdd36.json | 6 +- ...e6ecfcdec3d505d28cc65b5920dcf87c0014f.json | 3 +- ...2d82c350857166fc27fe9eecc88bcc4b229bc.json | 3 +- ...6e490d5dfcdd7a94dcbe5d05c9e5af37ac4a4.json | 3 +- ...9393023e79f948976e6ffc0e9bfc94623393c.json | 27 + ...1a803e1bced2f8e1a12f8933b7598ee85a147.json | 3 +- ...04af5a7eb488c76f783e690af0c2153b1d6a8.json | 3 +- ...a55a9f8fa854b29edcea83710a9170124edf0.json | 3 +- ...4f412c57af931f80621aee5012e9cb3535626.json | 3 +- ...8d92a983470adcd3cc850734960c905e61e83.json | 3 +- ...a22e4276c1da653475628889ce17dc904fbaa.json | 3 +- ...6eb9016207975af1e37042d81df8cb6ae2c53.json | 3 +- ...506bf08c28efe82b68b3d44bafbd3d0e91c29.json | 3 +- ...c749524f1db10a51f3de6cadd4826e6c1d422.json | 3 +- ...68644853987e62cbcedc8d3a46626a60c9fb0.json | 15 + ...d2393e297043e4e58555d872b5c4dd89c196a.json | 3 +- ...3a960b89e02ec40b292f3b5198baf2a1d3dbf.json | 3 +- ...fbafe5dc3cdecc17a3d5a7476b160c1b6e7e1.json | 3 +- ...c408eda4323ec12ade20ec0da565051128f26.json | 100 +++ ...e33244fe8ab9a803248fc23f334034d24aad4.json | 3 +- ...bb6a8f961c119516e4c33dc928cd3b4f4edbc.json | 3 +- ...9e637e9e156cd85b69126c87dfff89a47711d.json | 3 +- ...4267d67f58f9cbff0d164b4df45de885b0e9d.json | 3 +- ...77429c51e6166c414bcc5ce2e97fac25bcd77.json | 3 +- ...9746147aa9b8d0a81642a7b7cb4738a0cad66.json | 3 +- ...d434f860922fbc021185beffb9827647feb8e.json | 3 +- ...d288eb60f6c6644a73cdf949de0159215a7f7.json | 3 +- ...13f1f882b695f896f94e5cf91d205633355a1.json | 3 +- ...e1ea2fe96a1b94f5a92953fb1c1642d15c016.json | 3 +- ...120e5b0f629f9f04e08a2982df33ff23ac7a9.json | 3 +- ...1a8a9cc35f362197c0766a5787436af141106.json | 3 +- ...5e59e5edda1faea72f39603814afb6a3cd596.json | 3 +- ...acd0068c51ca21018d6751cc1044e38c06730.json | 15 + ...9352d4d1e3cb3a8a08f1c9c03055a1cab1235.json | 3 +- ...f03f72ef81db302d727e18e63a307ac902022.json | 3 +- ...c35c7342ac9a814c73f6290e1d6ebd3a55423.json | 3 +- ...50d6a6935420fbedc83aa3ad1e9be7465c8dd.json | 3 +- backend/Cargo.lock | 718 +++++++++++++++++- backend/Cargo.toml | 10 +- backend/ee-repo-ref.txt | 2 +- .../20260721082249_add_amqp_trigger.down.sql | 2 + .../20260721082249_add_amqp_trigger.up.sql | 83 ++ .../windmill-parser/src/asset_parser.rs | 7 +- .../tests/pipeline_annotations_parity.rs | 1 + backend/windmill-api-assets/src/lib.rs | 9 + backend/windmill-api-auth/src/scopes.rs | 3 + .../windmill-api-groups/src/granular_acls.rs | 7 +- backend/windmill-api-users/src/users.rs | 1 + .../windmill-api-workspaces/src/workspaces.rs | 19 + .../src/workspaces_extra.rs | 9 + backend/windmill-api/Cargo.toml | 6 +- backend/windmill-api/openapi.yaml | 394 ++++++++++ backend/windmill-api/src/capture.rs | 12 + backend/windmill-api/src/offboarding.rs | 5 +- backend/windmill-api/src/token.rs | 1 + backend/windmill-api/src/trash.rs | 1 + backend/windmill-api/src/triggers/amqp/mod.rs | 1 + backend/windmill-api/src/triggers/handler.rs | 23 + backend/windmill-api/src/triggers/listener.rs | 8 + backend/windmill-api/src/triggers/mod.rs | 2 + backend/windmill-api/src/workspaces_export.rs | 33 + backend/windmill-common/src/assets.rs | 1 + backend/windmill-common/src/triggers.rs | 3 +- backend/windmill-common/src/user_drafts.rs | 6 +- backend/windmill-git-sync/src/lib.rs | 11 + backend/windmill-store/Cargo.toml | 1 + backend/windmill-store/src/resources.rs | 1 + backend/windmill-trigger-amqp/Cargo.toml | 37 + backend/windmill-trigger-amqp/src/handler.rs | 203 +++++ backend/windmill-trigger-amqp/src/lib.rs | 341 +++++++++ backend/windmill-trigger-amqp/src/listener.rs | 255 +++++++ .../windmill-trigger/src/trigger_helpers.rs | 2 +- backend/windmill-types/src/jobs.rs | 2 + backend/windmill-types/src/triggers.rs | 3 + cli/src/commands/pipeline/boundedCascade.ts | 1 + cli/src/commands/pipeline/localGraph.ts | 1 + cli/src/commands/sync/sync.ts | 10 +- cli/src/commands/trigger/trigger.ts | 21 +- cli/src/guidance/core.ts | 2 +- cli/src/guidance/skills.gen.ts | 121 ++- cli/src/types.ts | 5 + cli/src/utils/git.ts | 2 + cli/test/git_unit.test.ts | 3 + .../src/lib/components/CompareDrafts.svelte | 2 + .../lib/components/CompareWorkspaces.svelte | 1 + frontend/src/lib/components/MoveDrawer.svelte | 1 + frontend/src/lib/components/Path.svelte | 7 + .../assets/AssetGraph/AddNode.svelte | 6 + .../AssetGraph/PipelineTriggerEditors.svelte | 11 + .../assets/AssetGraph/TriggerNode.svelte | 1 + .../assets/AssetGraph/boundedCascade.ts | 1 + .../AssetGraph/parsePipelineAnnotations.ts | 1 + .../assets/AssetGraph/pipelineTemplates.ts | 1 + .../lib/components/assets/AssetGraph/types.ts | 1 + .../lib/components/common/table/Row.svelte | 1 + .../components/common/table/RowIcon.svelte | 6 + .../chat/CreatedResourceActionDrawers.svelte | 4 + .../copilot/chat/ToolMessageActions.svelte | 2 + .../components/copilot/chat/global/core.ts | 11 + .../copilot/chat/global/userDraftAdapter.ts | 2 + .../copilot/chat/global/workspaceItems.ts | 3 + .../components/copilot/chat/pipeline/core.ts | 2 +- .../src/lib/components/copilot/chat/shared.ts | 1 + .../copilot/chat/workspaceItems.svelte.ts | 3 + .../components/copilot/chat/workspaceTools.ts | 13 +- .../copilot/chat/workspaceToolsZod.gen.ts | 40 + .../renderers/triggers/TriggersBadge.svelte | 11 +- .../src/lib/components/icons/AmqpIcon.svelte | 22 + frontend/src/lib/components/icons/index.ts | 3 + .../src/lib/components/offboarding-utils.ts | 2 + .../search/GlobalSearchModal.svelte | 17 +- .../lib/components/sessions/previewRouter.ts | 2 + .../components/sessions/sessionDeployModel.ts | 1 + .../components/sidebar/OperatorMenu.svelte | 1 + .../components/sidebar/SidebarContent.svelte | 10 + frontend/src/lib/components/triggers.ts | 3 + .../triggers/AddTriggersButton.svelte | 6 + .../components/triggers/CaptureButton.svelte | 6 + .../components/triggers/CaptureTable.svelte | 3 +- .../components/triggers/CaptureWrapper.svelte | 29 +- .../triggers/TestTriggerConnection.svelte | 22 +- .../components/triggers/TriggersEditor.svelte | 10 + .../triggers/TriggersWrapper.svelte | 10 + .../triggers/amqp/AmqpCapture.svelte | 58 ++ .../amqp/AmqpEditorConfigSection.svelte | 175 +++++ .../triggers/amqp/AmqpTriggerEditor.svelte | 29 + .../amqp/AmqpTriggerEditorInner.svelte | 615 +++++++++++++++ .../triggers/amqp/AmqpTriggersPanel.svelte | 61 ++ .../src/lib/components/triggers/amqp/utils.ts | 56 ++ .../components/triggers/triggers.svelte.ts | 28 + frontend/src/lib/components/triggers/utils.ts | 19 + frontend/src/lib/script_helpers.ts | 38 +- frontend/src/lib/userDraft.svelte.ts | 1 + frontend/src/lib/userDraftDbMigration.ts | 1 + frontend/src/lib/utils_deployable.ts | 31 + frontend/src/lib/utils_draft_deploy.ts | 6 + .../src/routes/(root)/(logged)/+layout.svelte | 4 + .../(root)/(logged)/amqp_triggers/+page.js | 5 + .../(logged)/amqp_triggers/+page.svelte | 550 ++++++++++++++ .../auto-generated/cli/cli-commands.md | 6 +- system_prompts/auto-generated/prompts.ts | 10 +- .../schemas/amqp_trigger.schema.yaml | 113 +++ system_prompts/auto-generated/script.md | 2 +- .../skills/cli-commands/SKILL.md | 6 +- system_prompts/base/pipeline-base.md | 2 +- system_prompts/base/script-base.md | 2 +- system_prompts/generate.py | 4 + system_prompts/utils.py | 1 + 165 files changed, 4753 insertions(+), 138 deletions(-) create mode 100644 backend/.sqlx/query-02e39bb9957d3ccfe3d573994cd281a22467010d2c4a0611c8ae02742336f433.json create mode 100644 backend/.sqlx/query-05843e117361d6b7cd9da652596d3be7a970d602d3809396d9253d7e5dc88152.json create mode 100644 backend/.sqlx/query-1cd16f93bdc90b24ceb1e86495069c3592218d34e58d617253e8b563bc661a7a.json create mode 100644 backend/.sqlx/query-5f2e66e53166840c68efbc521fe9393023e79f948976e6ffc0e9bfc94623393c.json create mode 100644 backend/.sqlx/query-90da952a912de746abee641cc4268644853987e62cbcedc8d3a46626a60c9fb0.json create mode 100644 backend/.sqlx/query-a47f7d625a96ad47277609ae173c408eda4323ec12ade20ec0da565051128f26.json create mode 100644 backend/.sqlx/query-d9d997591a163f25a9be5134bffacd0068c51ca21018d6751cc1044e38c06730.json create mode 100644 backend/migrations/20260721082249_add_amqp_trigger.down.sql create mode 100644 backend/migrations/20260721082249_add_amqp_trigger.up.sql create mode 100644 backend/windmill-api/src/triggers/amqp/mod.rs create mode 100644 backend/windmill-trigger-amqp/Cargo.toml create mode 100644 backend/windmill-trigger-amqp/src/handler.rs create mode 100644 backend/windmill-trigger-amqp/src/lib.rs create mode 100644 backend/windmill-trigger-amqp/src/listener.rs create mode 100644 frontend/src/lib/components/icons/AmqpIcon.svelte create mode 100644 frontend/src/lib/components/triggers/amqp/AmqpCapture.svelte create mode 100644 frontend/src/lib/components/triggers/amqp/AmqpEditorConfigSection.svelte create mode 100644 frontend/src/lib/components/triggers/amqp/AmqpTriggerEditor.svelte create mode 100644 frontend/src/lib/components/triggers/amqp/AmqpTriggerEditorInner.svelte create mode 100644 frontend/src/lib/components/triggers/amqp/AmqpTriggersPanel.svelte create mode 100644 frontend/src/lib/components/triggers/amqp/utils.ts create mode 100644 frontend/src/routes/(root)/(logged)/amqp_triggers/+page.js create mode 100644 frontend/src/routes/(root)/(logged)/amqp_triggers/+page.svelte create mode 100644 system_prompts/auto-generated/schemas/amqp_trigger.schema.yaml diff --git a/backend/.sqlx/query-02e39bb9957d3ccfe3d573994cd281a22467010d2c4a0611c8ae02742336f433.json b/backend/.sqlx/query-02e39bb9957d3ccfe3d573994cd281a22467010d2c4a0611c8ae02742336f433.json new file mode 100644 index 0000000000..9219641ce9 --- /dev/null +++ b/backend/.sqlx/query-02e39bb9957d3ccfe3d573994cd281a22467010d2c4a0611c8ae02742336f433.json @@ -0,0 +1,38 @@ +{ + "db_name": "PostgreSQL", + "query": "\n INSERT INTO amqp_trigger (\n amqp_resource_path,\n queue_name,\n exchange,\n options,\n workspace_id,\n path,\n script_path,\n is_flow,\n permissioned_as,\n mode,\n edited_by,\n error_handler_path,\n error_handler_args,\n retry\n )\n VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14\n )", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Varchar", + "Jsonb", + "Jsonb", + "Varchar", + "Varchar", + "Varchar", + "Bool", + "Varchar", + { + "Custom": { + "name": "trigger_mode", + "kind": { + "Enum": [ + "enabled", + "disabled", + "suspended" + ] + } + } + }, + "Varchar", + "Varchar", + "Jsonb", + "Jsonb" + ] + }, + "nullable": [] + }, + "hash": "02e39bb9957d3ccfe3d573994cd281a22467010d2c4a0611c8ae02742336f433" +} diff --git a/backend/.sqlx/query-05843e117361d6b7cd9da652596d3be7a970d602d3809396d9253d7e5dc88152.json b/backend/.sqlx/query-05843e117361d6b7cd9da652596d3be7a970d602d3809396d9253d7e5dc88152.json new file mode 100644 index 0000000000..950fe7bcdf --- /dev/null +++ b/backend/.sqlx/query-05843e117361d6b7cd9da652596d3be7a970d602d3809396d9253d7e5dc88152.json @@ -0,0 +1,41 @@ +{ + "db_name": "PostgreSQL", + "query": "\n SELECT kind, path, script_path, is_flow FROM (\n SELECT 'schedule' AS kind, path, script_path, is_flow FROM schedule\n WHERE workspace_id = $1\n AND script_path IS NOT NULL\n UNION ALL\n SELECT 'email', path, script_path, is_flow FROM email_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'kafka', path, script_path, is_flow FROM kafka_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'mqtt', path, script_path, is_flow FROM mqtt_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'amqp', path, script_path, is_flow FROM amqp_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'nats', path, script_path, is_flow FROM nats_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'postgres', path, script_path, is_flow FROM postgres_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'sqs', path, script_path, is_flow FROM sqs_trigger\n WHERE workspace_id = $1\n UNION ALL\n SELECT 'gcp', path, script_path, is_flow FROM gcp_trigger\n WHERE workspace_id = $1\n ) t\n WHERE ($2::text IS NULL OR script_path LIKE $2)\n ", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "kind", + "type_info": "Text" + }, + { + "ordinal": 1, + "name": "path", + "type_info": "Varchar" + }, + { + "ordinal": 2, + "name": "script_path", + "type_info": "Varchar" + }, + { + "ordinal": 3, + "name": "is_flow", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text", + "Text" + ] + }, + "nullable": [ + null, + null, + null, + null + ] + }, + "hash": "05843e117361d6b7cd9da652596d3be7a970d602d3809396d9253d7e5dc88152" +} diff --git a/backend/.sqlx/query-089d7bc7acdbb97cf477159e111bc7e9ee85289ff5c52af43166928337c257e7.json b/backend/.sqlx/query-089d7bc7acdbb97cf477159e111bc7e9ee85289ff5c52af43166928337c257e7.json index 6efb66005d..a762af65c9 100644 --- a/backend/.sqlx/query-089d7bc7acdbb97cf477159e111bc7e9ee85289ff5c52af43166928337c257e7.json +++ b/backend/.sqlx/query-089d7bc7acdbb97cf477159e111bc7e9ee85289ff5c52af43166928337c257e7.json @@ -36,7 +36,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-0f7e01b613a94b29784aae6d7b17b23d6dcf2e5364852a5e85b3c41c417bace2.json b/backend/.sqlx/query-0f7e01b613a94b29784aae6d7b17b23d6dcf2e5364852a5e85b3c41c417bace2.json index 0e0fbca58e..9333553d8e 100644 --- a/backend/.sqlx/query-0f7e01b613a94b29784aae6d7b17b23d6dcf2e5364852a5e85b3c41c417bace2.json +++ b/backend/.sqlx/query-0f7e01b613a94b29784aae6d7b17b23d6dcf2e5364852a5e85b3c41c417bace2.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-16d438374b03a9c515f4c2d638366f38ffe2f3a0958adea53e67757c6ac463ec.json b/backend/.sqlx/query-16d438374b03a9c515f4c2d638366f38ffe2f3a0958adea53e67757c6ac463ec.json index 0e966467a9..9efcf8fc44 100644 --- a/backend/.sqlx/query-16d438374b03a9c515f4c2d638366f38ffe2f3a0958adea53e67757c6ac463ec.json +++ b/backend/.sqlx/query-16d438374b03a9c515f4c2d638366f38ffe2f3a0958adea53e67757c6ac463ec.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-1cd16f93bdc90b24ceb1e86495069c3592218d34e58d617253e8b563bc661a7a.json b/backend/.sqlx/query-1cd16f93bdc90b24ceb1e86495069c3592218d34e58d617253e8b563bc661a7a.json new file mode 100644 index 0000000000..f2327d1700 --- /dev/null +++ b/backend/.sqlx/query-1cd16f93bdc90b24ceb1e86495069c3592218d34e58d617253e8b563bc661a7a.json @@ -0,0 +1,17 @@ +{ + "db_name": "PostgreSQL", + "query": "WITH t1 AS (UPDATE websocket_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), t2 AS (UPDATE kafka_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), t3 AS (UPDATE postgres_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), t4 AS (UPDATE mqtt_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), t5 AS (UPDATE nats_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), t6 AS (UPDATE sqs_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), t7 AS (UPDATE amqp_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4) UPDATE gcp_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Text", + "Text", + "Bool" + ] + }, + "nullable": [] + }, + "hash": "1cd16f93bdc90b24ceb1e86495069c3592218d34e58d617253e8b563bc661a7a" +} diff --git a/backend/.sqlx/query-212553c83e4dcdc6d045eb2fe2dadbb2860ce52d37a56b2861de1215260ecff8.json b/backend/.sqlx/query-212553c83e4dcdc6d045eb2fe2dadbb2860ce52d37a56b2861de1215260ecff8.json index 704883d4f1..831dce83d7 100644 --- a/backend/.sqlx/query-212553c83e4dcdc6d045eb2fe2dadbb2860ce52d37a56b2861de1215260ecff8.json +++ b/backend/.sqlx/query-212553c83e4dcdc6d045eb2fe2dadbb2860ce52d37a56b2861de1215260ecff8.json @@ -40,7 +40,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } @@ -79,7 +80,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-22e0e8a1aa48f8b21763452bd36fbe7db4887c4ac5295052c796bd78a7edc50b.json b/backend/.sqlx/query-22e0e8a1aa48f8b21763452bd36fbe7db4887c4ac5295052c796bd78a7edc50b.json index 49a2977cf8..54f9e57c2f 100644 --- a/backend/.sqlx/query-22e0e8a1aa48f8b21763452bd36fbe7db4887c4ac5295052c796bd78a7edc50b.json +++ b/backend/.sqlx/query-22e0e8a1aa48f8b21763452bd36fbe7db4887c4ac5295052c796bd78a7edc50b.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-23419adcd74c326d716527293eff518b42f4cdb33e034441015494bd26c172d2.json b/backend/.sqlx/query-23419adcd74c326d716527293eff518b42f4cdb33e034441015494bd26c172d2.json index 5c84181bdb..6aa714182f 100644 --- a/backend/.sqlx/query-23419adcd74c326d716527293eff518b42f4cdb33e034441015494bd26c172d2.json +++ b/backend/.sqlx/query-23419adcd74c326d716527293eff518b42f4cdb33e034441015494bd26c172d2.json @@ -43,7 +43,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-2bed492ef32edf36e60e8a03268fa25bfb67dd641153d1ca23f7d0d2ae73624e.json b/backend/.sqlx/query-2bed492ef32edf36e60e8a03268fa25bfb67dd641153d1ca23f7d0d2ae73624e.json index 241778d359..920a463d01 100644 --- a/backend/.sqlx/query-2bed492ef32edf36e60e8a03268fa25bfb67dd641153d1ca23f7d0d2ae73624e.json +++ b/backend/.sqlx/query-2bed492ef32edf36e60e8a03268fa25bfb67dd641153d1ca23f7d0d2ae73624e.json @@ -43,7 +43,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-3c84781704b84b8a927ecce5a3fcb3adcf0175d0a71013f2497397c1c8ccc619.json b/backend/.sqlx/query-3c84781704b84b8a927ecce5a3fcb3adcf0175d0a71013f2497397c1c8ccc619.json index f5ee767768..257e1e528f 100644 --- a/backend/.sqlx/query-3c84781704b84b8a927ecce5a3fcb3adcf0175d0a71013f2497397c1c8ccc619.json +++ b/backend/.sqlx/query-3c84781704b84b8a927ecce5a3fcb3adcf0175d0a71013f2497397c1c8ccc619.json @@ -129,7 +129,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-42b4b73e9d60348e2d90fcade9dcad6d8995242dc20a4e14c1a8fae4fc6a9fd2.json b/backend/.sqlx/query-42b4b73e9d60348e2d90fcade9dcad6d8995242dc20a4e14c1a8fae4fc6a9fd2.json index ad5613b8fd..a2174020cd 100644 --- a/backend/.sqlx/query-42b4b73e9d60348e2d90fcade9dcad6d8995242dc20a4e14c1a8fae4fc6a9fd2.json +++ b/backend/.sqlx/query-42b4b73e9d60348e2d90fcade9dcad6d8995242dc20a4e14c1a8fae4fc6a9fd2.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-45024b932383199974616bba1fc2f7175cc6f2e02d9c565bb5159cae3e0b6835.json b/backend/.sqlx/query-45024b932383199974616bba1fc2f7175cc6f2e02d9c565bb5159cae3e0b6835.json index e61104d62d..a5db66e7c6 100644 --- a/backend/.sqlx/query-45024b932383199974616bba1fc2f7175cc6f2e02d9c565bb5159cae3e0b6835.json +++ b/backend/.sqlx/query-45024b932383199974616bba1fc2f7175cc6f2e02d9c565bb5159cae3e0b6835.json @@ -33,7 +33,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-46f00a75b2e7e4ac70758a9687070f68bc0421f1aa228f80157adda63191d33b.json b/backend/.sqlx/query-46f00a75b2e7e4ac70758a9687070f68bc0421f1aa228f80157adda63191d33b.json index c269f7f340..dcd3573e92 100644 --- a/backend/.sqlx/query-46f00a75b2e7e4ac70758a9687070f68bc0421f1aa228f80157adda63191d33b.json +++ b/backend/.sqlx/query-46f00a75b2e7e4ac70758a9687070f68bc0421f1aa228f80157adda63191d33b.json @@ -42,7 +42,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-4a43d4df6c5b2e8dda4308dcb88c23caf312ec377dd91e5307f00d3fb8ec325d.json b/backend/.sqlx/query-4a43d4df6c5b2e8dda4308dcb88c23caf312ec377dd91e5307f00d3fb8ec325d.json index 7474818bf3..8849c7f0fa 100644 --- a/backend/.sqlx/query-4a43d4df6c5b2e8dda4308dcb88c23caf312ec377dd91e5307f00d3fb8ec325d.json +++ b/backend/.sqlx/query-4a43d4df6c5b2e8dda4308dcb88c23caf312ec377dd91e5307f00d3fb8ec325d.json @@ -81,7 +81,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-4d272cf4a77aab7007a5b35589e08532a1020cabaf5e22325a1e05f0491d785c.json b/backend/.sqlx/query-4d272cf4a77aab7007a5b35589e08532a1020cabaf5e22325a1e05f0491d785c.json index 99d6e818ea..6425204e0e 100644 --- a/backend/.sqlx/query-4d272cf4a77aab7007a5b35589e08532a1020cabaf5e22325a1e05f0491d785c.json +++ b/backend/.sqlx/query-4d272cf4a77aab7007a5b35589e08532a1020cabaf5e22325a1e05f0491d785c.json @@ -40,7 +40,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-4efaa4c3c14987fe95a508cc0cbac91ee793ad6f5ceed5b3eda9b48617c5434d.json b/backend/.sqlx/query-4efaa4c3c14987fe95a508cc0cbac91ee793ad6f5ceed5b3eda9b48617c5434d.json index bdb39d134f..3a9c2ffae7 100644 --- a/backend/.sqlx/query-4efaa4c3c14987fe95a508cc0cbac91ee793ad6f5ceed5b3eda9b48617c5434d.json +++ b/backend/.sqlx/query-4efaa4c3c14987fe95a508cc0cbac91ee793ad6f5ceed5b3eda9b48617c5434d.json @@ -47,7 +47,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-4f547c0fd54f3bc57212ce87810e35adf640d44d607e62a1fb296e38ac3fdd36.json b/backend/.sqlx/query-4f547c0fd54f3bc57212ce87810e35adf640d44d607e62a1fb296e38ac3fdd36.json index dea8f68e4f..ee1034e845 100644 --- a/backend/.sqlx/query-4f547c0fd54f3bc57212ce87810e35adf640d44d607e62a1fb296e38ac3fdd36.json +++ b/backend/.sqlx/query-4f547c0fd54f3bc57212ce87810e35adf640d44d607e62a1fb296e38ac3fdd36.json @@ -35,7 +35,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } @@ -76,7 +77,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-54ad5cc89563fdbbacdd6bfde9be6ecfcdec3d505d28cc65b5920dcf87c0014f.json b/backend/.sqlx/query-54ad5cc89563fdbbacdd6bfde9be6ecfcdec3d505d28cc65b5920dcf87c0014f.json index d81d6ebd94..6513bbbf21 100644 --- a/backend/.sqlx/query-54ad5cc89563fdbbacdd6bfde9be6ecfcdec3d505d28cc65b5920dcf87c0014f.json +++ b/backend/.sqlx/query-54ad5cc89563fdbbacdd6bfde9be6ecfcdec3d505d28cc65b5920dcf87c0014f.json @@ -36,7 +36,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-560539adbad0ecfa57fa477c3b82d82c350857166fc27fe9eecc88bcc4b229bc.json b/backend/.sqlx/query-560539adbad0ecfa57fa477c3b82d82c350857166fc27fe9eecc88bcc4b229bc.json index 91c941593f..81ca75de73 100644 --- a/backend/.sqlx/query-560539adbad0ecfa57fa477c3b82d82c350857166fc27fe9eecc88bcc4b229bc.json +++ b/backend/.sqlx/query-560539adbad0ecfa57fa477c3b82d82c350857166fc27fe9eecc88bcc4b229bc.json @@ -43,7 +43,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-5884ce1906015f6b96231f311226e490d5dfcdd7a94dcbe5d05c9e5af37ac4a4.json b/backend/.sqlx/query-5884ce1906015f6b96231f311226e490d5dfcdd7a94dcbe5d05c9e5af37ac4a4.json index 0c445c1ae5..9a18371f1f 100644 --- a/backend/.sqlx/query-5884ce1906015f6b96231f311226e490d5dfcdd7a94dcbe5d05c9e5af37ac4a4.json +++ b/backend/.sqlx/query-5884ce1906015f6b96231f311226e490d5dfcdd7a94dcbe5d05c9e5af37ac4a4.json @@ -36,7 +36,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-5f2e66e53166840c68efbc521fe9393023e79f948976e6ffc0e9bfc94623393c.json b/backend/.sqlx/query-5f2e66e53166840c68efbc521fe9393023e79f948976e6ffc0e9bfc94623393c.json new file mode 100644 index 0000000000..2a9c1c119e --- /dev/null +++ b/backend/.sqlx/query-5f2e66e53166840c68efbc521fe9393023e79f948976e6ffc0e9bfc94623393c.json @@ -0,0 +1,27 @@ +{ + "db_name": "PostgreSQL", + "query": "\n UPDATE\n amqp_trigger\n SET\n amqp_resource_path = $1,\n queue_name = $2,\n exchange = $3,\n options = $4,\n is_flow = $5,\n edited_by = $6,\n permissioned_as = $7,\n script_path = $8,\n path = $9,\n edited_at = now(),\n error = NULL,\n server_id = NULL,\n error_handler_path = $12,\n error_handler_args = $13,\n retry = $14\n WHERE\n workspace_id = $10 AND\n path = $11\n ", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Varchar", + "Jsonb", + "Jsonb", + "Bool", + "Varchar", + "Varchar", + "Varchar", + "Varchar", + "Text", + "Text", + "Varchar", + "Jsonb", + "Jsonb" + ] + }, + "nullable": [] + }, + "hash": "5f2e66e53166840c68efbc521fe9393023e79f948976e6ffc0e9bfc94623393c" +} diff --git a/backend/.sqlx/query-66a0e51cf149ba532463e29dd361a803e1bced2f8e1a12f8933b7598ee85a147.json b/backend/.sqlx/query-66a0e51cf149ba532463e29dd361a803e1bced2f8e1a12f8933b7598ee85a147.json index 68d9a1fc3b..8870f18d44 100644 --- a/backend/.sqlx/query-66a0e51cf149ba532463e29dd361a803e1bced2f8e1a12f8933b7598ee85a147.json +++ b/backend/.sqlx/query-66a0e51cf149ba532463e29dd361a803e1bced2f8e1a12f8933b7598ee85a147.json @@ -38,7 +38,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-67e25a7c19ea0ffaf7ea5303fcd04af5a7eb488c76f783e690af0c2153b1d6a8.json b/backend/.sqlx/query-67e25a7c19ea0ffaf7ea5303fcd04af5a7eb488c76f783e690af0c2153b1d6a8.json index fcc16e9a7c..0eb26d5dc0 100644 --- a/backend/.sqlx/query-67e25a7c19ea0ffaf7ea5303fcd04af5a7eb488c76f783e690af0c2153b1d6a8.json +++ b/backend/.sqlx/query-67e25a7c19ea0ffaf7ea5303fcd04af5a7eb488c76f783e690af0c2153b1d6a8.json @@ -162,7 +162,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-6b9348e60cc1ce158314a93fc7aa55a9f8fa854b29edcea83710a9170124edf0.json b/backend/.sqlx/query-6b9348e60cc1ce158314a93fc7aa55a9f8fa854b29edcea83710a9170124edf0.json index 43d3b6d7f6..9dbeab4c76 100644 --- a/backend/.sqlx/query-6b9348e60cc1ce158314a93fc7aa55a9f8fa854b29edcea83710a9170124edf0.json +++ b/backend/.sqlx/query-6b9348e60cc1ce158314a93fc7aa55a9f8fa854b29edcea83710a9170124edf0.json @@ -47,7 +47,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-7065f23d04e26831664048f2cfc4f412c57af931f80621aee5012e9cb3535626.json b/backend/.sqlx/query-7065f23d04e26831664048f2cfc4f412c57af931f80621aee5012e9cb3535626.json index 046d8ad6bd..e4eea7fc8f 100644 --- a/backend/.sqlx/query-7065f23d04e26831664048f2cfc4f412c57af931f80621aee5012e9cb3535626.json +++ b/backend/.sqlx/query-7065f23d04e26831664048f2cfc4f412c57af931f80621aee5012e9cb3535626.json @@ -32,7 +32,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-73fdd01bad58b8be1a52f89faef8d92a983470adcd3cc850734960c905e61e83.json b/backend/.sqlx/query-73fdd01bad58b8be1a52f89faef8d92a983470adcd3cc850734960c905e61e83.json index 7b7e68ad45..23dd32b094 100644 --- a/backend/.sqlx/query-73fdd01bad58b8be1a52f89faef8d92a983470adcd3cc850734960c905e61e83.json +++ b/backend/.sqlx/query-73fdd01bad58b8be1a52f89faef8d92a983470adcd3cc850734960c905e61e83.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-7fbf72d9059fcd77e4c1112fa4fa22e4276c1da653475628889ce17dc904fbaa.json b/backend/.sqlx/query-7fbf72d9059fcd77e4c1112fa4fa22e4276c1da653475628889ce17dc904fbaa.json index cd5e361f15..19264cfdfe 100644 --- a/backend/.sqlx/query-7fbf72d9059fcd77e4c1112fa4fa22e4276c1da653475628889ce17dc904fbaa.json +++ b/backend/.sqlx/query-7fbf72d9059fcd77e4c1112fa4fa22e4276c1da653475628889ce17dc904fbaa.json @@ -30,7 +30,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-836bac47d89113d90bd03a471446eb9016207975af1e37042d81df8cb6ae2c53.json b/backend/.sqlx/query-836bac47d89113d90bd03a471446eb9016207975af1e37042d81df8cb6ae2c53.json index 1c0e5a0f1b..c2d9b482b5 100644 --- a/backend/.sqlx/query-836bac47d89113d90bd03a471446eb9016207975af1e37042d81df8cb6ae2c53.json +++ b/backend/.sqlx/query-836bac47d89113d90bd03a471446eb9016207975af1e37042d81df8cb6ae2c53.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-87564a196a1662f524407d853db506bf08c28efe82b68b3d44bafbd3d0e91c29.json b/backend/.sqlx/query-87564a196a1662f524407d853db506bf08c28efe82b68b3d44bafbd3d0e91c29.json index 9b3fd8f205..71d71e7843 100644 --- a/backend/.sqlx/query-87564a196a1662f524407d853db506bf08c28efe82b68b3d44bafbd3d0e91c29.json +++ b/backend/.sqlx/query-87564a196a1662f524407d853db506bf08c28efe82b68b3d44bafbd3d0e91c29.json @@ -38,7 +38,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-8f163ee5adf4caaaa12a5698e68c749524f1db10a51f3de6cadd4826e6c1d422.json b/backend/.sqlx/query-8f163ee5adf4caaaa12a5698e68c749524f1db10a51f3de6cadd4826e6c1d422.json index 90031c14d6..4e63ce0324 100644 --- a/backend/.sqlx/query-8f163ee5adf4caaaa12a5698e68c749524f1db10a51f3de6cadd4826e6c1d422.json +++ b/backend/.sqlx/query-8f163ee5adf4caaaa12a5698e68c749524f1db10a51f3de6cadd4826e6c1d422.json @@ -36,7 +36,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-90da952a912de746abee641cc4268644853987e62cbcedc8d3a46626a60c9fb0.json b/backend/.sqlx/query-90da952a912de746abee641cc4268644853987e62cbcedc8d3a46626a60c9fb0.json new file mode 100644 index 0000000000..a67728cc90 --- /dev/null +++ b/backend/.sqlx/query-90da952a912de746abee641cc4268644853987e62cbcedc8d3a46626a60c9fb0.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "UPDATE amqp_trigger SET workspace_id = $1 WHERE workspace_id = $2", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Text" + ] + }, + "nullable": [] + }, + "hash": "90da952a912de746abee641cc4268644853987e62cbcedc8d3a46626a60c9fb0" +} diff --git a/backend/.sqlx/query-940b6d78bab940a37a42492f030d2393e297043e4e58555d872b5c4dd89c196a.json b/backend/.sqlx/query-940b6d78bab940a37a42492f030d2393e297043e4e58555d872b5c4dd89c196a.json index c76a069f49..ea5a2946f7 100644 --- a/backend/.sqlx/query-940b6d78bab940a37a42492f030d2393e297043e4e58555d872b5c4dd89c196a.json +++ b/backend/.sqlx/query-940b6d78bab940a37a42492f030d2393e297043e4e58555d872b5c4dd89c196a.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-9c50e3a136a8ee3ec56e083f26d3a960b89e02ec40b292f3b5198baf2a1d3dbf.json b/backend/.sqlx/query-9c50e3a136a8ee3ec56e083f26d3a960b89e02ec40b292f3b5198baf2a1d3dbf.json index 87655ad8c1..52590474a3 100644 --- a/backend/.sqlx/query-9c50e3a136a8ee3ec56e083f26d3a960b89e02ec40b292f3b5198baf2a1d3dbf.json +++ b/backend/.sqlx/query-9c50e3a136a8ee3ec56e083f26d3a960b89e02ec40b292f3b5198baf2a1d3dbf.json @@ -35,7 +35,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-9ecb404e46a4eac55f977f05a3afbafe5dc3cdecc17a3d5a7476b160c1b6e7e1.json b/backend/.sqlx/query-9ecb404e46a4eac55f977f05a3afbafe5dc3cdecc17a3d5a7476b160c1b6e7e1.json index ab01730c02..fc1e28642b 100644 --- a/backend/.sqlx/query-9ecb404e46a4eac55f977f05a3afbafe5dc3cdecc17a3d5a7476b160c1b6e7e1.json +++ b/backend/.sqlx/query-9ecb404e46a4eac55f977f05a3afbafe5dc3cdecc17a3d5a7476b160c1b6e7e1.json @@ -36,7 +36,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-a47f7d625a96ad47277609ae173c408eda4323ec12ade20ec0da565051128f26.json b/backend/.sqlx/query-a47f7d625a96ad47277609ae173c408eda4323ec12ade20ec0da565051128f26.json new file mode 100644 index 0000000000..9361eb51aa --- /dev/null +++ b/backend/.sqlx/query-a47f7d625a96ad47277609ae173c408eda4323ec12ade20ec0da565051128f26.json @@ -0,0 +1,100 @@ +{ + "db_name": "PostgreSQL", + "query": "\n SELECT\n EXISTS(SELECT 1 FROM websocket_trigger WHERE workspace_id = $1) AS \"websocket_used!\",\n EXISTS(SELECT 1 FROM http_trigger WHERE workspace_id = $1) AS \"http_routes_used!\",\n EXISTS(SELECT 1 FROM kafka_trigger WHERE workspace_id = $1) as \"kafka_used!\",\n EXISTS(SELECT 1 FROM nats_trigger WHERE workspace_id = $1) as \"nats_used!\",\n EXISTS(SELECT 1 FROM postgres_trigger WHERE workspace_id = $1) AS \"postgres_used!\",\n EXISTS(SELECT 1 FROM mqtt_trigger WHERE workspace_id = $1) AS \"mqtt_used!\",\n EXISTS(SELECT 1 FROM amqp_trigger WHERE workspace_id = $1) AS \"amqp_used!\",\n EXISTS(SELECT 1 FROM sqs_trigger WHERE workspace_id = $1) AS \"sqs_used!\",\n EXISTS(SELECT 1 FROM gcp_trigger WHERE workspace_id = $1) AS \"gcp_used!\",\n EXISTS(SELECT 1 FROM azure_trigger WHERE workspace_id = $1) AS \"azure_used!\",\n EXISTS(SELECT 1 FROM email_trigger WHERE workspace_id = $1) AS \"email_used!\",\n EXISTS(SELECT 1 FROM native_trigger WHERE workspace_id = $1 AND service_name = 'nextcloud'::native_trigger_service) AS \"nextcloud_used!\",\n EXISTS(SELECT 1 FROM native_trigger WHERE workspace_id = $1 AND service_name = 'google'::native_trigger_service) AS \"google_used!\",\n EXISTS(SELECT 1 FROM native_trigger WHERE workspace_id = $1 AND service_name = 'github'::native_trigger_service) AS \"github_used!\"\n ", + "describe": { + "columns": [ + { + "ordinal": 0, + "name": "websocket_used!", + "type_info": "Bool" + }, + { + "ordinal": 1, + "name": "http_routes_used!", + "type_info": "Bool" + }, + { + "ordinal": 2, + "name": "kafka_used!", + "type_info": "Bool" + }, + { + "ordinal": 3, + "name": "nats_used!", + "type_info": "Bool" + }, + { + "ordinal": 4, + "name": "postgres_used!", + "type_info": "Bool" + }, + { + "ordinal": 5, + "name": "mqtt_used!", + "type_info": "Bool" + }, + { + "ordinal": 6, + "name": "amqp_used!", + "type_info": "Bool" + }, + { + "ordinal": 7, + "name": "sqs_used!", + "type_info": "Bool" + }, + { + "ordinal": 8, + "name": "gcp_used!", + "type_info": "Bool" + }, + { + "ordinal": 9, + "name": "azure_used!", + "type_info": "Bool" + }, + { + "ordinal": 10, + "name": "email_used!", + "type_info": "Bool" + }, + { + "ordinal": 11, + "name": "nextcloud_used!", + "type_info": "Bool" + }, + { + "ordinal": 12, + "name": "google_used!", + "type_info": "Bool" + }, + { + "ordinal": 13, + "name": "github_used!", + "type_info": "Bool" + } + ], + "parameters": { + "Left": [ + "Text" + ] + }, + "nullable": [ + null, + null, + null, + null, + null, + null, + null, + null, + null, + null, + null, + null, + null, + null + ] + }, + "hash": "a47f7d625a96ad47277609ae173c408eda4323ec12ade20ec0da565051128f26" +} diff --git a/backend/.sqlx/query-a4b6371d33206010b2f3ffd2b09e33244fe8ab9a803248fc23f334034d24aad4.json b/backend/.sqlx/query-a4b6371d33206010b2f3ffd2b09e33244fe8ab9a803248fc23f334034d24aad4.json index 405904604a..4824bba526 100644 --- a/backend/.sqlx/query-a4b6371d33206010b2f3ffd2b09e33244fe8ab9a803248fc23f334034d24aad4.json +++ b/backend/.sqlx/query-a4b6371d33206010b2f3ffd2b09e33244fe8ab9a803248fc23f334034d24aad4.json @@ -192,7 +192,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc.json b/backend/.sqlx/query-b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc.json index 092d15e592..3bbd0f043e 100644 --- a/backend/.sqlx/query-b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc.json +++ b/backend/.sqlx/query-b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc.json @@ -167,7 +167,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-b3f0595cacba194e08b9a3e244d9e637e9e156cd85b69126c87dfff89a47711d.json b/backend/.sqlx/query-b3f0595cacba194e08b9a3e244d9e637e9e156cd85b69126c87dfff89a47711d.json index 7325a2a313..6efaff7069 100644 --- a/backend/.sqlx/query-b3f0595cacba194e08b9a3e244d9e637e9e156cd85b69126c87dfff89a47711d.json +++ b/backend/.sqlx/query-b3f0595cacba194e08b9a3e244d9e637e9e156cd85b69126c87dfff89a47711d.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-b6cfb752675a3f36975e6cc6c454267d67f58f9cbff0d164b4df45de885b0e9d.json b/backend/.sqlx/query-b6cfb752675a3f36975e6cc6c454267d67f58f9cbff0d164b4df45de885b0e9d.json index da7e3d5787..98bcf5a2c2 100644 --- a/backend/.sqlx/query-b6cfb752675a3f36975e6cc6c454267d67f58f9cbff0d164b4df45de885b0e9d.json +++ b/backend/.sqlx/query-b6cfb752675a3f36975e6cc6c454267d67f58f9cbff0d164b4df45de885b0e9d.json @@ -47,7 +47,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-bcfa34cf80abea05f0c24883b9e77429c51e6166c414bcc5ce2e97fac25bcd77.json b/backend/.sqlx/query-bcfa34cf80abea05f0c24883b9e77429c51e6166c414bcc5ce2e97fac25bcd77.json index 419ab26383..fb35992691 100644 --- a/backend/.sqlx/query-bcfa34cf80abea05f0c24883b9e77429c51e6166c414bcc5ce2e97fac25bcd77.json +++ b/backend/.sqlx/query-bcfa34cf80abea05f0c24883b9e77429c51e6166c414bcc5ce2e97fac25bcd77.json @@ -81,7 +81,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-be6d2c92a62b7b284651c45af809746147aa9b8d0a81642a7b7cb4738a0cad66.json b/backend/.sqlx/query-be6d2c92a62b7b284651c45af809746147aa9b8d0a81642a7b7cb4738a0cad66.json index b9d33a6b5f..809deb1a55 100644 --- a/backend/.sqlx/query-be6d2c92a62b7b284651c45af809746147aa9b8d0a81642a7b7cb4738a0cad66.json +++ b/backend/.sqlx/query-be6d2c92a62b7b284651c45af809746147aa9b8d0a81642a7b7cb4738a0cad66.json @@ -112,7 +112,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-c3b1152b554812d65eb27f95b1fd434f860922fbc021185beffb9827647feb8e.json b/backend/.sqlx/query-c3b1152b554812d65eb27f95b1fd434f860922fbc021185beffb9827647feb8e.json index 8a6ab29126..2bc2bdf950 100644 --- a/backend/.sqlx/query-c3b1152b554812d65eb27f95b1fd434f860922fbc021185beffb9827647feb8e.json +++ b/backend/.sqlx/query-c3b1152b554812d65eb27f95b1fd434f860922fbc021185beffb9827647feb8e.json @@ -34,7 +34,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-c8fb2a1491f90951a6d2e4e9c56d288eb60f6c6644a73cdf949de0159215a7f7.json b/backend/.sqlx/query-c8fb2a1491f90951a6d2e4e9c56d288eb60f6c6644a73cdf949de0159215a7f7.json index 161b9d36f9..52830d5c73 100644 --- a/backend/.sqlx/query-c8fb2a1491f90951a6d2e4e9c56d288eb60f6c6644a73cdf949de0159215a7f7.json +++ b/backend/.sqlx/query-c8fb2a1491f90951a6d2e4e9c56d288eb60f6c6644a73cdf949de0159215a7f7.json @@ -43,7 +43,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-ccef7a1bde5cac6c362c5fedb6c13f1f882b695f896f94e5cf91d205633355a1.json b/backend/.sqlx/query-ccef7a1bde5cac6c362c5fedb6c13f1f882b695f896f94e5cf91d205633355a1.json index f567c462f8..0cf9c77f62 100644 --- a/backend/.sqlx/query-ccef7a1bde5cac6c362c5fedb6c13f1f882b695f896f94e5cf91d205633355a1.json +++ b/backend/.sqlx/query-ccef7a1bde5cac6c362c5fedb6c13f1f882b695f896f94e5cf91d205633355a1.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-d41ea93fd58381b89e151c965eae1ea2fe96a1b94f5a92953fb1c1642d15c016.json b/backend/.sqlx/query-d41ea93fd58381b89e151c965eae1ea2fe96a1b94f5a92953fb1c1642d15c016.json index d97c02d26b..2660ca05ef 100644 --- a/backend/.sqlx/query-d41ea93fd58381b89e151c965eae1ea2fe96a1b94f5a92953fb1c1642d15c016.json +++ b/backend/.sqlx/query-d41ea93fd58381b89e151c965eae1ea2fe96a1b94f5a92953fb1c1642d15c016.json @@ -112,7 +112,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-d4211392e174a0e8f89c7fcebdf120e5b0f629f9f04e08a2982df33ff23ac7a9.json b/backend/.sqlx/query-d4211392e174a0e8f89c7fcebdf120e5b0f629f9f04e08a2982df33ff23ac7a9.json index 052d83fcd9..0bd066afd1 100644 --- a/backend/.sqlx/query-d4211392e174a0e8f89c7fcebdf120e5b0f629f9f04e08a2982df33ff23ac7a9.json +++ b/backend/.sqlx/query-d4211392e174a0e8f89c7fcebdf120e5b0f629f9f04e08a2982df33ff23ac7a9.json @@ -252,7 +252,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-d495c94b580fd34d5ae90615ef21a8a9cc35f362197c0766a5787436af141106.json b/backend/.sqlx/query-d495c94b580fd34d5ae90615ef21a8a9cc35f362197c0766a5787436af141106.json index eb340e6cc7..049856cbde 100644 --- a/backend/.sqlx/query-d495c94b580fd34d5ae90615ef21a8a9cc35f362197c0766a5787436af141106.json +++ b/backend/.sqlx/query-d495c94b580fd34d5ae90615ef21a8a9cc35f362197c0766a5787436af141106.json @@ -28,7 +28,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-d4e0da9f9653d532770066310f85e59e5edda1faea72f39603814afb6a3cd596.json b/backend/.sqlx/query-d4e0da9f9653d532770066310f85e59e5edda1faea72f39603814afb6a3cd596.json index e60f5cc187..317f33de07 100644 --- a/backend/.sqlx/query-d4e0da9f9653d532770066310f85e59e5edda1faea72f39603814afb6a3cd596.json +++ b/backend/.sqlx/query-d4e0da9f9653d532770066310f85e59e5edda1faea72f39603814afb6a3cd596.json @@ -48,7 +48,8 @@ "trigger_nextcloud", "trigger_google", "trigger_github", - "data_pipeline" + "data_pipeline", + "trigger_amqp" ] } } diff --git a/backend/.sqlx/query-d9d997591a163f25a9be5134bffacd0068c51ca21018d6751cc1044e38c06730.json b/backend/.sqlx/query-d9d997591a163f25a9be5134bffacd0068c51ca21018d6751cc1044e38c06730.json new file mode 100644 index 0000000000..d5e5970903 --- /dev/null +++ b/backend/.sqlx/query-d9d997591a163f25a9be5134bffacd0068c51ca21018d6751cc1044e38c06730.json @@ -0,0 +1,15 @@ +{ + "db_name": "PostgreSQL", + "query": "INSERT INTO amqp_trigger (\n amqp_resource_path, queue_name, exchange, options, path, script_path, is_flow,\n workspace_id, edited_by, edited_at, extra_perms, server_id, last_server_ping,\n error, error_handler_path, error_handler_args, retry, mode, permissioned_as, labels\n )\n SELECT\n amqp_resource_path, queue_name, exchange, options, path, script_path, is_flow,\n $1, edited_by, edited_at, extra_perms, NULL, NULL,\n NULL, error_handler_path, error_handler_args, retry, 'disabled'::TRIGGER_MODE, permissioned_as, labels\n FROM amqp_trigger WHERE workspace_id = $2", + "describe": { + "columns": [], + "parameters": { + "Left": [ + "Varchar", + "Text" + ] + }, + "nullable": [] + }, + "hash": "d9d997591a163f25a9be5134bffacd0068c51ca21018d6751cc1044e38c06730" +} diff --git a/backend/.sqlx/query-e4d71278fb80126a7a9da73f1889352d4d1e3cb3a8a08f1c9c03055a1cab1235.json b/backend/.sqlx/query-e4d71278fb80126a7a9da73f1889352d4d1e3cb3a8a08f1c9c03055a1cab1235.json index a35373a959..ad34773e36 100644 --- a/backend/.sqlx/query-e4d71278fb80126a7a9da73f1889352d4d1e3cb3a8a08f1c9c03055a1cab1235.json +++ b/backend/.sqlx/query-e4d71278fb80126a7a9da73f1889352d4d1e3cb3a8a08f1c9c03055a1cab1235.json @@ -192,7 +192,8 @@ "github", "azure", "asset", - "freshness" + "freshness", + "amqp" ] } } diff --git a/backend/.sqlx/query-e80177f3ffd4c1f52cdb4757483f03f72ef81db302d727e18e63a307ac902022.json b/backend/.sqlx/query-e80177f3ffd4c1f52cdb4757483f03f72ef81db302d727e18e63a307ac902022.json index 3f403a8f95..cb0b92a822 100644 --- a/backend/.sqlx/query-e80177f3ffd4c1f52cdb4757483f03f72ef81db302d727e18e63a307ac902022.json +++ b/backend/.sqlx/query-e80177f3ffd4c1f52cdb4757483f03f72ef81db302d727e18e63a307ac902022.json @@ -34,7 +34,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-eac595e19e5c8e70f1514ef29dec35c7342ac9a814c73f6290e1d6ebd3a55423.json b/backend/.sqlx/query-eac595e19e5c8e70f1514ef29dec35c7342ac9a814c73f6290e1d6ebd3a55423.json index b48f144f8e..461b7fc037 100644 --- a/backend/.sqlx/query-eac595e19e5c8e70f1514ef29dec35c7342ac9a814c73f6290e1d6ebd3a55423.json +++ b/backend/.sqlx/query-eac595e19e5c8e70f1514ef29dec35c7342ac9a814c73f6290e1d6ebd3a55423.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/.sqlx/query-ed8facbf29ebb670d05fe8aa34b50d6a6935420fbedc83aa3ad1e9be7465c8dd.json b/backend/.sqlx/query-ed8facbf29ebb670d05fe8aa34b50d6a6935420fbedc83aa3ad1e9be7465c8dd.json index c28997d702..902a07854d 100644 --- a/backend/.sqlx/query-ed8facbf29ebb670d05fe8aa34b50d6a6935420fbedc83aa3ad1e9be7465c8dd.json +++ b/backend/.sqlx/query-ed8facbf29ebb670d05fe8aa34b50d6a6935420fbedc83aa3ad1e9be7465c8dd.json @@ -27,7 +27,8 @@ "nextcloud", "google", "github", - "azure" + "azure", + "amqp" ] } } diff --git a/backend/Cargo.lock b/backend/Cargo.lock index 0bed7274d7..84014f30e9 100644 --- a/backend/Cargo.lock +++ b/backend/Cargo.lock @@ -128,6 +128,54 @@ version = "0.2.21" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923" +[[package]] +name = "amq-protocol" +version = "7.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "587d313f3a8b4a40f866cc84b6059fe83133bf172165ac3b583129dd211d8e1c" +dependencies = [ + "amq-protocol-tcp", + "amq-protocol-types", + "amq-protocol-uri", + "cookie-factory", + "nom", + "serde", +] + +[[package]] +name = "amq-protocol-tcp" +version = "7.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc707ab9aa964a85d9fc25908a3fdc486d2e619406883b3105b48bf304a8d606" +dependencies = [ + "amq-protocol-uri", + "tcp-stream", + "tracing", +] + +[[package]] +name = "amq-protocol-types" +version = "7.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf99351d92a161c61ec6ecb213bc7057f5b837dd4e64ba6cb6491358efd770c4" +dependencies = [ + "cookie-factory", + "nom", + "serde", + "serde_json", +] + +[[package]] +name = "amq-protocol-uri" +version = "7.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f89f8273826a676282208e5af38461a07fe939def57396af6ad5997fcf56577d" +dependencies = [ + "amq-protocol-types", + "percent-encoding", + "url", +] + [[package]] name = "android_system_properties" version = "0.1.5" @@ -472,7 +520,7 @@ version = "0.6.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5493c3bedbacf7fd7382c6346bbd66687d12bbaad3a89a2d2c303ee6cf20b048" dependencies = [ - "asn1-rs-derive", + "asn1-rs-derive 0.5.1", "asn1-rs-impl", "displaydoc", "nom", @@ -482,6 +530,22 @@ dependencies = [ "time", ] +[[package]] +name = "asn1-rs" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8" +dependencies = [ + "asn1-rs-derive 0.6.0", + "asn1-rs-impl", + "displaydoc", + "nom", + "num-traits", + "rusticata-macros", + "thiserror 2.0.19", + "time", +] + [[package]] name = "asn1-rs-derive" version = "0.5.1" @@ -494,6 +558,18 @@ dependencies = [ "synstructure", ] +[[package]] +name = "asn1-rs-derive" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + [[package]] name = "asn1-rs-impl" version = "0.2.0" @@ -555,6 +631,18 @@ dependencies = [ "futures-core", ] +[[package]] +name = "async-channel" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +dependencies = [ + "concurrent-queue", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + [[package]] name = "async-compression" version = "0.4.19" @@ -574,6 +662,92 @@ dependencies = [ "zstd-safe", ] +[[package]] +name = "async-executor" +version = "1.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a" +dependencies = [ + "async-task", + "concurrent-queue", + "fastrand 2.5.0", + "futures-lite 2.6.1", + "pin-project-lite", + "slab", +] + +[[package]] +name = "async-global-executor" +version = "3.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13f937e26114b93193065fd44f507aa2e9169ad0cdabbb996920b1fe1ddea7ba" +dependencies = [ + "async-channel 2.5.0", + "async-executor", + "async-io 2.6.0", + "async-lock 3.4.2", + "blocking", + "futures-lite 2.6.1", +] + +[[package]] +name = "async-global-executor-trait" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9af57045d58eeb1f7060e7025a1631cbc6399e0a1d10ad6735b3d0ea7f8346ce" +dependencies = [ + "async-global-executor", + "async-trait", + "executor-trait", +] + +[[package]] +name = "async-io" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fc5b45d93ef0529756f812ca52e44c221b35341892d3dcc34132ac02f3dd2af" +dependencies = [ + "async-lock 2.8.0", + "autocfg", + "cfg-if", + "concurrent-queue", + "futures-lite 1.13.0", + "log", + "parking", + "polling 2.8.0", + "rustix 0.37.28", + "slab", + "socket2 0.4.10", + "waker-fn", +] + +[[package]] +name = "async-io" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc" +dependencies = [ + "autocfg", + "cfg-if", + "concurrent-queue", + "futures-io", + "futures-lite 2.6.1", + "parking", + "polling 3.11.0", + "rustix 1.1.4", + "slab", + "windows-sys 0.61.2", +] + +[[package]] +name = "async-lock" +version = "2.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "287272293e9d8c41773cec55e365490fe034813a2f172f502d6ddcf75b2f582b" +dependencies = [ + "event-listener 2.5.3", +] + [[package]] name = "async-lock" version = "3.4.2" @@ -646,6 +820,18 @@ version = "0.5.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4288f83726785267c6f2ef073a3d83dc3f9b81464e9f99898240cced85fce35a" +[[package]] +name = "async-reactor-trait" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a6012d170ad00de56c9ee354aef2e358359deb1ec504254e0e5a3774771de0e" +dependencies = [ + "async-io 1.13.0", + "async-trait", + "futures-core", + "reactor-trait", +] + [[package]] name = "async-recursion" version = "1.1.1" @@ -679,6 +865,12 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "async-task" +version = "4.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" + [[package]] name = "async-trait" version = "0.1.91" @@ -699,7 +891,7 @@ dependencies = [ "async-compression", "chrono", "crc32fast", - "futures-lite", + "futures-lite 2.6.1", "pin-project", "thiserror 1.0.69", "tokio", @@ -759,7 +951,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "hex", "http 1.4.2", "ring 0.17.14", @@ -823,7 +1015,7 @@ dependencies = [ "aws-types", "bytes", "bytes-utils", - "fastrand", + "fastrand 2.5.0", "http 1.4.2", "http-body 1.1.0", "percent-encoding", @@ -849,7 +1041,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "http 1.4.2", "regex-lite", @@ -875,7 +1067,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "hyper 0.14.32", "regex-lite", @@ -898,7 +1090,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "once_cell", "regex-lite", @@ -924,7 +1116,7 @@ dependencies = [ "aws-smithy-types", "aws-smithy-xml", "aws-types", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "http 1.4.2", "regex-lite", @@ -949,7 +1141,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "http 1.4.2", "regex-lite", @@ -972,7 +1164,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "regex-lite", "tracing", @@ -995,7 +1187,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "http 1.4.2", "regex-lite", @@ -1019,7 +1211,7 @@ dependencies = [ "aws-smithy-types", "aws-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "http 1.4.2", "regex-lite", @@ -1043,7 +1235,7 @@ dependencies = [ "aws-smithy-types", "aws-smithy-xml", "aws-types", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "regex-lite", "tracing", @@ -1217,7 +1409,7 @@ dependencies = [ "aws-smithy-runtime-api", "aws-smithy-types", "bytes", - "fastrand", + "fastrand 2.5.0", "http 0.2.12", "http 1.4.2", "http-body 0.4.6", @@ -1429,7 +1621,7 @@ version = "1.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cffb0e931875b666fc4fcb20fee52e9bbd1ef836fd9e9e04ec21555f9f85f7ef" dependencies = [ - "fastrand", + "fastrand 2.5.0", "gloo-timers", "tokio", ] @@ -1690,6 +1882,19 @@ dependencies = [ "generic-array", ] +[[package]] +name = "blocking" +version = "1.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e83f8d02be6967315521be875afa792a316e28d57b5a2d401897e2a7921b7f21" +dependencies = [ + "async-channel 2.5.0", + "async-task", + "futures-io", + "futures-lite 2.6.1", + "piper", +] + [[package]] name = "bollard" version = "0.18.1" @@ -2252,6 +2457,18 @@ dependencies = [ "cc", ] +[[package]] +name = "cms" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b77c319abfd5219629c45c34c89ba945ed3c5e49fcde9d16b6c3885f118a730" +dependencies = [ + "const-oid", + "der", + "spki", + "x509-cert", +] + [[package]] name = "colorchoice" version = "1.0.5" @@ -2419,6 +2636,12 @@ dependencies = [ "version_check", ] +[[package]] +name = "cookie-factory" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9885fa71e26b8ab7855e2ec7cae6e9b380edff76cd052e07c683a0319d51b3a2" + [[package]] name = "core-foundation" version = "0.9.4" @@ -3854,7 +4077,7 @@ version = "0.4.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7bc6b10059f0ccb14c6e0319c5275f0407fb2f9ffe405cd555700561999ea4bf" dependencies = [ - "fastrand", + "fastrand 2.5.0", "futures-channel", "libc", "windows-sys 0.59.0", @@ -3975,6 +4198,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" dependencies = [ "const-oid", + "der_derive", + "flagset", "pem-rfc7468", "zeroize", ] @@ -3985,7 +4210,7 @@ version = "9.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5cd0a5c643689626bec213c4d8bd4d96acc8ffdb4ad4bb6bc16abf27d5f4b553" dependencies = [ - "asn1-rs", + "asn1-rs 0.6.2", "displaydoc", "nom", "num-bigint", @@ -3993,6 +4218,31 @@ dependencies = [ "rusticata-macros", ] +[[package]] +name = "der-parser" +version = "10.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6" +dependencies = [ + "asn1-rs 0.7.2", + "displaydoc", + "nom", + "num-bigint", + "num-traits", + "rusticata-macros", +] + +[[package]] +name = "der_derive" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8034092389675178f570469e6c3b0465d3d30b4505c294a6550db47f3c17ad18" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "deranged" version = "0.5.8" @@ -4117,6 +4367,15 @@ dependencies = [ "opaque-debug", ] +[[package]] +name = "des" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ffdd80ce8ce993de27e9f063a444a4d53ce8e8db4c1f00cc03af5ad5a9867a1e" +dependencies = [ + "cipher 0.4.4", +] + [[package]] name = "digest" version = "0.9.0" @@ -4255,6 +4514,12 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "doc-comment" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "780955b8b195a21ab8e4ac6b60dd1dbdcec1dc6c51c0617964b08c81785e12c9" + [[package]] name = "dotenv" version = "0.15.0" @@ -4584,6 +4849,15 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "executor-trait" +version = "2.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c39dff9342e4e0e16ce96be751eb21a94e94a87bb2f6e63ad1961c2ce109bf" +dependencies = [ + "async-trait", +] + [[package]] name = "fallible-iterator" version = "0.2.0" @@ -4618,6 +4892,15 @@ version = "0.4.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9afc2bd4d5a73106dd53d10d73d3401c2f32730ba2c0b93ddb888a8983680471" +[[package]] +name = "fastrand" +version = "1.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e51093e27b0797c359783294ca4f0a911c270184cb10f85783b118614a1501be" +dependencies = [ + "instant", +] + [[package]] name = "fastrand" version = "2.5.0" @@ -4668,6 +4951,12 @@ version = "0.5.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99" +[[package]] +name = "flagset" +version = "0.4.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7ac824320a75a52197e8f2d787f6a38b6718bb6897a35142d749af3c0e8f4fe" + [[package]] name = "flatbuffers" version = "25.12.19" @@ -4874,13 +5163,28 @@ version = "0.3.33" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4577ecaa3c4f96589d473f679a71b596316f6641bc350038b962a5daf0085d7a" +[[package]] +name = "futures-lite" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49a9d51ce47660b1e808d3c990b4709f2f415d928835a17dfd16991515c46bce" +dependencies = [ + "fastrand 1.9.0", + "futures-core", + "futures-io", + "memchr", + "parking", + "pin-project-lite", + "waker-fn", +] + [[package]] name = "futures-lite" version = "2.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" dependencies = [ - "fastrand", + "fastrand 2.5.0", "futures-core", "futures-io", "parking", @@ -5301,7 +5605,7 @@ version = "0.30.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bebc6e7327e49a66ffb40508c673b7643191bd6b509530193bda97f09272cdcf" dependencies = [ - "async-channel", + "async-channel 1.9.0", "async-stream", "google-cloud-auth", "google-cloud-gax", @@ -5513,6 +5817,12 @@ version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" +[[package]] +name = "hermit-abi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d231dfb89cfffdbc30e7fc41579ed6066ad03abda9e567ccafae602b97ec5024" + [[package]] name = "hermit-abi" version = "0.5.2" @@ -6157,6 +6467,15 @@ dependencies = [ "generic-array", ] +[[package]] +name = "instant" +version = "0.1.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222" +dependencies = [ + "cfg-if", +] + [[package]] name = "integer-encoding" version = "3.0.4" @@ -6172,6 +6491,17 @@ dependencies = [ "rustversion", ] +[[package]] +name = "io-lifetimes" +version = "1.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eae7b9aee968036d54dce06cebaefd919e4472e753296daccd6d344e3e2df0c2" +dependencies = [ + "hermit-abi 0.3.9", + "libc", + "windows-sys 0.48.0", +] + [[package]] name = "io-uring" version = "0.7.13" @@ -6577,6 +6907,28 @@ version = "0.20.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "507460a910eb7b32ee961886ff48539633b788a36b65692b95f225b844c82553" +[[package]] +name = "lapin" +version = "2.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02d2aa4725b9607915fa1a73e940710a3be6af508ce700e56897cbe8847fbb07" +dependencies = [ + "amq-protocol", + "async-global-executor-trait", + "async-reactor-trait", + "async-trait", + "executor-trait", + "flume", + "futures-core", + "futures-io", + "parking_lot", + "pinky-swear", + "reactor-trait", + "serde", + "tracing", + "waker-fn", +] + [[package]] name = "lazy_static" version = "1.5.0" @@ -6774,6 +7126,12 @@ version = "0.5.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0717cef1bc8b636c6e1c1bbdefc09e6322da8a9321966e8928ef80d20f7f770f" +[[package]] +name = "linux-raw-sys" +version = "0.3.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ef53942eb7bf7ff43a617b3e2c1c4a5ecf5944a7c1bc12d7ee39bbb15e5c1519" + [[package]] name = "linux-raw-sys" version = "0.4.15" @@ -6932,7 +7290,7 @@ dependencies = [ "base64 0.21.7", "block-modes", "crc-any", - "des", + "des 0.7.0", "digest 0.9.0", "md-5 0.9.1", "sha2 0.9.9", @@ -7238,7 +7596,7 @@ version = "0.12.15" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "957228ad12042ee839f93c8f257b62b4c0ab5eaae1d4fa60de53b27c9d7c5046" dependencies = [ - "async-lock", + "async-lock 3.4.2", "crossbeam-channel", "crossbeam-epoch", "crossbeam-utils", @@ -7749,7 +8107,7 @@ version = "1.17.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "91df4bbde75afed763b708b7eee1e8e7651e02d97f6d5dd763e89367e957b23b" dependencies = [ - "hermit-abi", + "hermit-abi 0.5.2", "libc", ] @@ -7862,7 +8220,16 @@ version = "0.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a8d8034d9489cdaf79228eb9f6a3b8d7bb32ba00d6645ebd48eef4077ceb5bd9" dependencies = [ - "asn1-rs", + "asn1-rs 0.6.2", +] + +[[package]] +name = "oid-registry" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7" +dependencies = [ + "asn1-rs 0.7.2", ] [[package]] @@ -8273,6 +8640,28 @@ version = "4.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d211803b9b6b570f68772237e415a029d5a50c65d382910b879fb19d3271f94d" +[[package]] +name = "p12-keystore" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3cae83056e7cb770211494a0ecf66d9fa7eba7d00977e5bb91f0e925b40b937f" +dependencies = [ + "cbc", + "cms", + "der", + "des 0.8.1", + "hex", + "hmac", + "pkcs12", + "pkcs5", + "rand 0.9.0", + "rc2", + "sha1", + "sha2 0.10.9", + "thiserror 2.0.19", + "x509-parser 0.17.0", +] + [[package]] name = "p256" version = "0.13.2" @@ -8414,6 +8803,16 @@ version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "df94ce210e5bc13cb6651479fa48d14f601d9858cfe0467f43ae157023b938d3" +[[package]] +name = "pbkdf2" +version = "0.12.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8ed6a7761f76e3b9f92dfb0a60a6a6477c61024b775147ff0973a02653abaf2" +dependencies = [ + "digest 0.10.7", + "hmac", +] + [[package]] name = "pem" version = "1.1.1" @@ -8654,6 +9053,29 @@ version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" +[[package]] +name = "pinky-swear" +version = "6.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1ea6e230dd3a64d61bcb8b79e597d3ab6b4c94ec7a234ce687dd718b4f2e657" +dependencies = [ + "doc-comment", + "flume", + "parking_lot", + "tracing", +] + +[[package]] +name = "piper" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" +dependencies = [ + "atomic-waker", + "fastrand 2.5.0", + "futures-io", +] + [[package]] name = "pkcs1" version = "0.7.5" @@ -8665,6 +9087,36 @@ dependencies = [ "spki", ] +[[package]] +name = "pkcs12" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "695b3df3d3cc1015f12d70235e35b6b79befc5fa7a9b95b951eab1dd07c9efc2" +dependencies = [ + "cms", + "const-oid", + "der", + "digest 0.10.7", + "spki", + "x509-cert", + "zeroize", +] + +[[package]] +name = "pkcs5" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e847e2c91a18bfa887dd028ec33f2fe6f25db77db3619024764914affe8b69a6" +dependencies = [ + "aes 0.8.3", + "cbc", + "der", + "pbkdf2", + "scrypt", + "sha2 0.10.9", + "spki", +] + [[package]] name = "pkcs8" version = "0.10.2" @@ -8687,6 +9139,36 @@ version = "0.2.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6" +[[package]] +name = "polling" +version = "2.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b2d323e8ca7996b3e23126511a523f7e62924d93ecd5ae73b333815b0eb3dce" +dependencies = [ + "autocfg", + "bitflags 1.3.2", + "cfg-if", + "concurrent-queue", + "libc", + "log", + "pin-project-lite", + "windows-sys 0.48.0", +] + +[[package]] +name = "polling" +version = "3.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218" +dependencies = [ + "cfg-if", + "concurrent-queue", + "hermit-abi 0.5.2", + "pin-project-lite", + "rustix 1.1.4", + "windows-sys 0.61.2", +] + [[package]] name = "polyval" version = "0.6.2" @@ -9334,6 +9816,15 @@ dependencies = [ "crossbeam-utils", ] +[[package]] +name = "rc2" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "62c64daa8e9438b84aaae55010a93f396f8e60e3911590fcba770d04643fc1dd" +dependencies = [ + "cipher 0.4.4", +] + [[package]] name = "rcgen" version = "0.13.2" @@ -9344,7 +9835,7 @@ dependencies = [ "ring 0.17.14", "rustls-pki-types", "time", - "x509-parser", + "x509-parser 0.16.0", "yasna", ] @@ -9382,6 +9873,17 @@ dependencies = [ "sasl2-sys", ] +[[package]] +name = "reactor-trait" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "438a4293e4d097556730f4711998189416232f009c137389e0f961d2bc0ddc58" +dependencies = [ + "async-trait", + "futures-core", + "futures-io", +] + [[package]] name = "reborrow" version = "0.5.5" @@ -9804,7 +10306,7 @@ version = "0.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b8bf7840285c321c3ab20e752a9afb95548c75cd7f4632a0627cea3507e310c1" dependencies = [ - "async-lock", + "async-lock 3.4.2", "hashbrown 0.16.1", "relative-path", "rquickjs-sys", @@ -9984,6 +10486,20 @@ dependencies = [ "nom", ] +[[package]] +name = "rustix" +version = "0.37.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "519165d378b97752ca44bbe15047d5d3409e875f39327546b42ac81d7e18c1b6" +dependencies = [ + "bitflags 1.3.2", + "errno", + "io-lifetimes", + "libc", + "linux-raw-sys 0.3.8", + "windows-sys 0.48.0", +] + [[package]] name = "rustix" version = "0.38.44" @@ -10052,6 +10568,19 @@ dependencies = [ "zeroize", ] +[[package]] +name = "rustls-connector" +version = "0.20.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70cc376c6ba1823ae229bacf8ad93c136d93524eab0e4e5e0e4f96b9c4e5b212" +dependencies = [ + "log", + "rustls 0.23.35", + "rustls-native-certs 0.7.3", + "rustls-pki-types", + "rustls-webpki 0.103.13", +] + [[package]] name = "rustls-native-certs" version = "0.6.3" @@ -10275,6 +10804,15 @@ dependencies = [ "serde_json", ] +[[package]] +name = "salsa20" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "97a22f5af31f73a954c10289c93e8a50cc23d971e80ee446f1f6f7137a088213" +dependencies = [ + "cipher 0.4.4", +] + [[package]] name = "samael" version = "0.0.20" @@ -10411,6 +10949,17 @@ version = "1.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" +[[package]] +name = "scrypt" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0516a385866c09368f0b5bcd1caff3366aace790fcd46e2bb032697bb172fd1f" +dependencies = [ + "pbkdf2", + "salsa20", + "sha2 0.10.9", +] + [[package]] name = "sct" version = "0.7.1" @@ -10978,6 +11527,16 @@ version = "1.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "199905e6153d6405f9728fe44daace35f8f837bbf830bb6e85fbd5828709a886" +[[package]] +name = "socket2" +version = "0.4.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7916fc008ca5542385b89a3d3ce689953c143e9304a9bf8beec1de48994c0d" +dependencies = [ + "libc", + "winapi", +] + [[package]] name = "socket2" version = "0.5.10" @@ -12137,6 +12696,18 @@ dependencies = [ "xattr", ] +[[package]] +name = "tcp-stream" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "495b0abdce3dc1f8fd27240651c9e68890c14e9d9c61527b1ce44d8a5a7bd3d5" +dependencies = [ + "cfg-if", + "p12-keystore", + "rustls-connector", + "rustls-pemfile 2.2.0", +] + [[package]] name = "temp_deno_which" version = "0.1.0" @@ -12152,7 +12723,7 @@ version = "3.27.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" dependencies = [ - "fastrand", + "fastrand 2.5.0", "getrandom 0.4.3", "once_cell", "rustix 1.1.4", @@ -12462,6 +13033,17 @@ dependencies = [ "windows-sys 0.52.0", ] +[[package]] +name = "tokio-executor-trait" +version = "2.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6278565f9fd60c2d205dfbc827e8bb1236c2b1a57148708e95861eff7a6b3bad" +dependencies = [ + "async-trait", + "executor-trait", + "tokio", +] + [[package]] name = "tokio-graceful" version = "0.1.6" @@ -12521,6 +13103,20 @@ dependencies = [ "whoami", ] +[[package]] +name = "tokio-reactor-trait" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e9481a72f36bd9cbb8d6dd349227c4783e234e4332cfe806225bc929c4b92486" +dependencies = [ + "async-trait", + "futures-core", + "futures-io", + "reactor-trait", + "tokio", + "tokio-stream", +] + [[package]] name = "tokio-retry2" version = "0.5.8" @@ -13551,6 +14147,12 @@ dependencies = [ "memchr", ] +[[package]] +name = "waker-fn" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "317211a0dc0ceedd78fb2ca9a44aed3d7b9b26f81870d485c07122b4350673b7" + [[package]] name = "walkdir" version = "2.5.0" @@ -14129,6 +14731,7 @@ dependencies = [ "windmill-queue", "windmill-store", "windmill-trigger", + "windmill-trigger-amqp", "windmill-trigger-azure", "windmill-trigger-email", "windmill-trigger-gcp", @@ -15425,6 +16028,33 @@ dependencies = [ "windmill-queue", ] +[[package]] +name = "windmill-trigger-amqp" +version = "1.764.0" +dependencies = [ + "anyhow", + "async-trait", + "axum 0.8.9", + "base64 0.22.1", + "futures", + "itertools 0.14.0", + "lapin", + "serde", + "serde_json", + "sqlx", + "thiserror 2.0.19", + "tokio", + "tokio-executor-trait", + "tokio-reactor-trait", + "tracing", + "urlencoding", + "windmill-api-auth", + "windmill-common", + "windmill-git-sync", + "windmill-store", + "windmill-trigger", +] + [[package]] name = "windmill-trigger-azure" version = "1.764.0" @@ -15836,7 +16466,7 @@ dependencies = [ "windmill-types", "windmill-worker-volumes", "windows 0.61.3", - "x509-parser", + "x509-parser 0.16.0", "yaml-rust", ] @@ -16488,24 +17118,52 @@ dependencies = [ "zeroize", ] +[[package]] +name = "x509-cert" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1301e935010a701ae5f8655edc0ad17c44bad3ac5ce8c39185f75453b720ae94" +dependencies = [ + "const-oid", + "der", + "spki", +] + [[package]] name = "x509-parser" version = "0.16.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fcbc162f30700d6f3f82a24bf7cc62ffe7caea42c0b2cba8bf7f3ae50cf51f69" dependencies = [ - "asn1-rs", + "asn1-rs 0.6.2", "data-encoding", - "der-parser", + "der-parser 9.0.0", "lazy_static", "nom", - "oid-registry", + "oid-registry 0.7.1", "ring 0.17.14", "rusticata-macros", "thiserror 1.0.69", "time", ] +[[package]] +name = "x509-parser" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4569f339c0c402346d4a75a9e39cf8dad310e287eef1ff56d4c68e5067f53460" +dependencies = [ + "asn1-rs 0.7.2", + "data-encoding", + "der-parser 10.0.0", + "lazy_static", + "nom", + "oid-registry 0.8.1", + "rusticata-macros", + "thiserror 2.0.19", + "time", +] + [[package]] name = "xattr" version = "1.6.1" diff --git a/backend/Cargo.toml b/backend/Cargo.toml index 5bb6069461..638e267c04 100644 --- a/backend/Cargo.toml +++ b/backend/Cargo.toml @@ -22,6 +22,7 @@ members = [ "./windmill-trigger-kafka", "./windmill-trigger-postgres", "./windmill-trigger-mqtt", + "./windmill-trigger-amqp", "./windmill-trigger-websocket", "./windmill-trigger-email", "./windmill-trigger-nats", @@ -141,6 +142,7 @@ postgres_trigger = ["windmill-api/postgres_trigger"] mcp = ["windmill-ai/mcp", "windmill-api/mcp", "windmill-worker/mcp"] bedrock = ["windmill-ai/bedrock", "windmill-api/bedrock", "windmill-worker/bedrock"] mqtt_trigger = ["windmill-api/mqtt_trigger"] +amqp_trigger = ["windmill-api/amqp_trigger"] native_trigger = ["windmill-api/native_trigger"] sqs_trigger = ["windmill-api/sqs_trigger", "windmill-common/aws_auth", "windmill-api/openidconnect"] gcp_trigger = ["windmill-api/gcp_trigger"] @@ -180,7 +182,7 @@ run_inline = ["windmill-api/run_inline"] oss_core = [ "embedding", "parquet", "openidconnect", "license", "http_trigger", "zip", "oauth2", "postgres_trigger", - "mqtt_trigger", "websocket", "smtp", "native_trigger", + "mqtt_trigger", "amqp_trigger", "websocket", "smtp", "native_trigger", "static_frontend", "mcp", "bedrock", "run_inline", "quickjs" ] @@ -201,7 +203,7 @@ ee_rhel = ["ce_core", "ee_core", "kafka-gssapi", "all_languages"] ee_windows = ["ce_core", "ee_core", "all_languages_windows"] all_sqlx_features = ["all_languages", "enterprise", "enterprise_saml", "embedding", "parquet", "prometheus", "flow_testing", "openidconnect", "cloud", "jemalloc", "tantivy", "sqlx", "kafka", "kafka-gssapi", "nats", "otel", "dind", "websocket", "http_trigger", - "postgres_trigger", "mcp", "mqtt_trigger", "sqs_trigger", "gcp_trigger", "azure_trigger", "smtp", "stripe", + "postgres_trigger", "mcp", "mqtt_trigger", "amqp_trigger", "sqs_trigger", "gcp_trigger", "azure_trigger", "smtp", "stripe", "license", "oauth2", "zip", "static_frontend", "scoped_cache", "agent_worker_server", "bedrock", "native_trigger", "quickjs", "windmill-git-sync/all_sqlx_features"] @@ -361,6 +363,7 @@ windmill-trigger = { path = "./windmill-trigger" } windmill-trigger-kafka = { path = "./windmill-trigger-kafka" } windmill-trigger-postgres = { path = "./windmill-trigger-postgres" } windmill-trigger-mqtt = { path = "./windmill-trigger-mqtt" } +windmill-trigger-amqp = { path = "./windmill-trigger-amqp" } windmill-trigger-websocket = { path = "./windmill-trigger-websocket" } windmill-trigger-email = { path = "./windmill-trigger-email" } windmill-trigger-nats = { path = "./windmill-trigger-nats" } @@ -678,6 +681,9 @@ tree-sitter-ruby = "=0.23.1" tree-sitter-r = "=1.2.0" oracle = { version = "0.6.3", features = ["chrono"] } rumqttc = { version = "0.24.0", features = ["use-native-tls"]} +lapin = "2.5" +tokio-executor-trait = "2.1" +tokio-reactor-trait = "1.1" strum = { version = "0.27", features = ["derive"] } strum_macros = "0.27" hudsucker = { version = "0.22", features = ["rcgen-ca", "native-tls-client"] } diff --git a/backend/ee-repo-ref.txt b/backend/ee-repo-ref.txt index 8bc1d06cad..d0497c1e02 100644 --- a/backend/ee-repo-ref.txt +++ b/backend/ee-repo-ref.txt @@ -1 +1 @@ -eb3690a34b41134c91b9cb6f90b11daa541ad171 \ No newline at end of file +aaa6cb89b05b76139252c64f057e53b94d12ac60 diff --git a/backend/migrations/20260721082249_add_amqp_trigger.down.sql b/backend/migrations/20260721082249_add_amqp_trigger.down.sql new file mode 100644 index 0000000000..1ab9baecb8 --- /dev/null +++ b/backend/migrations/20260721082249_add_amqp_trigger.down.sql @@ -0,0 +1,2 @@ +-- Add down migration script here +DROP TABLE amqp_trigger; diff --git a/backend/migrations/20260721082249_add_amqp_trigger.up.sql b/backend/migrations/20260721082249_add_amqp_trigger.up.sql new file mode 100644 index 0000000000..2f2da55c00 --- /dev/null +++ b/backend/migrations/20260721082249_add_amqp_trigger.up.sql @@ -0,0 +1,83 @@ +-- Add up migration script here +CREATE TABLE amqp_trigger ( + amqp_resource_path VARCHAR(255) NOT NULL, + queue_name VARCHAR(255) NOT NULL, + exchange JSONB NULL, + options JSONB NULL, + path VARCHAR(255) NOT NULL, + script_path VARCHAR(255) NOT NULL, + is_flow BOOLEAN NOT NULL, + workspace_id VARCHAR(50) NOT NULL, + edited_by VARCHAR(50) NOT NULL, + edited_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + extra_perms JSONB NOT NULL DEFAULT '{}', + server_id VARCHAR(50) NULL, + last_server_ping TIMESTAMPTZ NULL, + error TEXT NULL, + error_handler_path VARCHAR(255) NULL, + error_handler_args JSONB NULL, + retry JSONB NULL, + mode TRIGGER_MODE NOT NULL DEFAULT 'enabled'::TRIGGER_MODE, + permissioned_as VARCHAR(255) NOT NULL, + labels TEXT[] NULL, + PRIMARY KEY (path, workspace_id), + FOREIGN KEY (workspace_id) REFERENCES workspace(id) ON DELETE CASCADE +); + +CREATE INDEX idx_amqp_trigger_labels ON amqp_trigger USING gin (labels) WHERE labels IS NOT NULL; + +GRANT ALL ON amqp_trigger TO windmill_user; +GRANT ALL ON amqp_trigger TO windmill_admin; + +ALTER TABLE amqp_trigger ENABLE ROW LEVEL SECURITY; + +CREATE POLICY admin_policy ON amqp_trigger FOR ALL TO windmill_admin USING (true); + +CREATE POLICY see_folder_extra_perms_user_select ON amqp_trigger FOR SELECT TO windmill_user +USING (SPLIT_PART(amqp_trigger.path, '/', 1) = 'f' AND SPLIT_PART(amqp_trigger.path, '/', 2) = any((select regexp_split_to_array(current_setting('session.folders_read'), ','))::text[])); +CREATE POLICY see_folder_extra_perms_user_insert ON amqp_trigger FOR INSERT TO windmill_user +WITH CHECK (SPLIT_PART(amqp_trigger.path, '/', 1) = 'f' AND SPLIT_PART(amqp_trigger.path, '/', 2) = any((select regexp_split_to_array(current_setting('session.folders_write'), ','))::text[])); +CREATE POLICY see_folder_extra_perms_user_update ON amqp_trigger FOR UPDATE TO windmill_user +USING (SPLIT_PART(amqp_trigger.path, '/', 1) = 'f' AND SPLIT_PART(amqp_trigger.path, '/', 2) = any((select regexp_split_to_array(current_setting('session.folders_write'), ','))::text[])); +CREATE POLICY see_folder_extra_perms_user_delete ON amqp_trigger FOR DELETE TO windmill_user +USING (SPLIT_PART(amqp_trigger.path, '/', 1) = 'f' AND SPLIT_PART(amqp_trigger.path, '/', 2) = any((select regexp_split_to_array(current_setting('session.folders_write'), ','))::text[])); + +CREATE POLICY see_own ON amqp_trigger FOR ALL TO windmill_user +USING (SPLIT_PART(amqp_trigger.path, '/', 1) = 'u' AND SPLIT_PART(amqp_trigger.path, '/', 2) = (select current_setting('session.user'))); +CREATE POLICY see_member ON amqp_trigger FOR ALL TO windmill_user +USING (SPLIT_PART(amqp_trigger.path, '/', 1) = 'g' AND SPLIT_PART(amqp_trigger.path, '/', 2) = any((select regexp_split_to_array(current_setting('session.groups'), ','))::text[])); + +CREATE POLICY see_extra_perms_user_select ON amqp_trigger FOR SELECT TO windmill_user +USING (extra_perms ? (select concat('u/', current_setting('session.user')))); +CREATE POLICY see_extra_perms_user_insert ON amqp_trigger FOR INSERT TO windmill_user +WITH CHECK ((extra_perms ->> (select concat('u/', current_setting('session.user'))))::boolean); +CREATE POLICY see_extra_perms_user_update ON amqp_trigger FOR UPDATE TO windmill_user +USING ((extra_perms ->> (select concat('u/', current_setting('session.user'))))::boolean); +CREATE POLICY see_extra_perms_user_delete ON amqp_trigger FOR DELETE TO windmill_user +USING ((extra_perms ->> (select concat('u/', current_setting('session.user'))))::boolean); + +CREATE POLICY see_extra_perms_groups_select ON amqp_trigger FOR SELECT TO windmill_user +USING (extra_perms ?| (select regexp_split_to_array(current_setting('session.pgroups'), ','))::text[]); +CREATE POLICY see_extra_perms_groups_insert ON amqp_trigger FOR INSERT TO windmill_user +WITH CHECK (exists( + SELECT key, value FROM jsonb_each_text(extra_perms) + WHERE SPLIT_PART(key, '/', 1) = 'g' AND key = ANY((select regexp_split_to_array(current_setting('session.pgroups'), ','))::text[]) + AND value::boolean)); +CREATE POLICY see_extra_perms_groups_update ON amqp_trigger FOR UPDATE TO windmill_user +USING (exists( + SELECT key, value FROM jsonb_each_text(extra_perms) + WHERE SPLIT_PART(key, '/', 1) = 'g' AND key = ANY((select regexp_split_to_array(current_setting('session.pgroups'), ','))::text[]) + AND value::boolean)); +CREATE POLICY see_extra_perms_groups_delete ON amqp_trigger FOR DELETE TO windmill_user +USING (exists( + SELECT key, value FROM jsonb_each_text(extra_perms) + WHERE SPLIT_PART(key, '/', 1) = 'g' AND key = ANY((select regexp_split_to_array(current_setting('session.pgroups'), ','))::text[]) + AND value::boolean)); + +-- Enum values for the new trigger kind. ALTER TYPE ... ADD VALUE runs inside the +-- migration transaction on PG >= 14 (Windmill's minimum) as long as the value +-- isn't used in the same transaction — the amqp_trigger table above does not +-- reference these enum types. +ALTER TYPE TRIGGER_KIND ADD VALUE IF NOT EXISTS 'amqp'; +ALTER TYPE job_trigger_kind ADD VALUE IF NOT EXISTS 'amqp'; +ALTER TYPE draft_kind ADD VALUE IF NOT EXISTS 'trigger_amqp'; diff --git a/backend/parsers/windmill-parser/src/asset_parser.rs b/backend/parsers/windmill-parser/src/asset_parser.rs index 4cf0c22a41..211c205579 100644 --- a/backend/parsers/windmill-parser/src/asset_parser.rs +++ b/backend/parsers/windmill-parser/src/asset_parser.rs @@ -172,6 +172,7 @@ pub enum TriggerSpec { Email, Kafka, Mqtt, + Amqp, Nats, Postgres, Sqs, @@ -1527,6 +1528,7 @@ fn parse_trigger_spec(s: &str) -> Option { ("email", TriggerSpec::Email), ("kafka", TriggerSpec::Kafka), ("mqtt", TriggerSpec::Mqtt), + ("amqp", TriggerSpec::Amqp), ("nats", TriggerSpec::Nats), ("postgres", TriggerSpec::Postgres), ("sqs", TriggerSpec::Sqs), @@ -1722,7 +1724,10 @@ mod pipeline_annotation_tests { ); // A non-ASCII body must not panic on a byte offset that is not a char // boundary. - assert_eq!(split_measure_filter("sum(amount) + π"), ("sum(amount) + π", None)); + assert_eq!( + split_measure_filter("sum(amount) + π"), + ("sum(amount) + π", None) + ); assert_eq!( split_measure_filter("sum(π) where region = 'π'"), ("sum(π)", Some("region = 'π'".to_string())) diff --git a/backend/parsers/windmill-parser/tests/pipeline_annotations_parity.rs b/backend/parsers/windmill-parser/tests/pipeline_annotations_parity.rs index fff818edb2..1936f7ff70 100644 --- a/backend/parsers/windmill-parser/tests/pipeline_annotations_parity.rs +++ b/backend/parsers/windmill-parser/tests/pipeline_annotations_parity.rs @@ -121,6 +121,7 @@ fn native_str(t: &TriggerSpec) -> Option<&'static str> { TriggerSpec::Email => "email", TriggerSpec::Kafka => "kafka", TriggerSpec::Mqtt => "mqtt", + TriggerSpec::Amqp => "amqp", TriggerSpec::Nats => "nats", TriggerSpec::Postgres => "postgres", TriggerSpec::Sqs => "sqs", diff --git a/backend/windmill-api-assets/src/lib.rs b/backend/windmill-api-assets/src/lib.rs index 997879e155..3a2bf83097 100644 --- a/backend/windmill-api-assets/src/lib.rs +++ b/backend/windmill-api-assets/src/lib.rs @@ -784,6 +784,11 @@ enum TriggerEdge { runnable_kind: AssetUsageKind, runnable_path: String, }, + Amqp { + path: String, + runnable_kind: AssetUsageKind, + runnable_path: String, + }, Nats { path: String, runnable_kind: AssetUsageKind, @@ -928,6 +933,9 @@ async fn asset_graph( SELECT 'mqtt', path, script_path, is_flow FROM mqtt_trigger WHERE workspace_id = $1 UNION ALL + SELECT 'amqp', path, script_path, is_flow FROM amqp_trigger + WHERE workspace_id = $1 + UNION ALL SELECT 'nats', path, script_path, is_flow FROM nats_trigger WHERE workspace_id = $1 UNION ALL @@ -1210,6 +1218,7 @@ async fn asset_graph( "email" => TriggerEdge::Email { path, runnable_kind, runnable_path: script_path }, "kafka" => TriggerEdge::Kafka { path, runnable_kind, runnable_path: script_path }, "mqtt" => TriggerEdge::Mqtt { path, runnable_kind, runnable_path: script_path }, + "amqp" => TriggerEdge::Amqp { path, runnable_kind, runnable_path: script_path }, "nats" => TriggerEdge::Nats { path, runnable_kind, runnable_path: script_path }, "postgres" => TriggerEdge::Postgres { path, runnable_kind, runnable_path: script_path }, "sqs" => TriggerEdge::Sqs { path, runnable_kind, runnable_path: script_path }, diff --git a/backend/windmill-api-auth/src/scopes.rs b/backend/windmill-api-auth/src/scopes.rs index 3b594fe79c..75cbc104ac 100644 --- a/backend/windmill-api-auth/src/scopes.rs +++ b/backend/windmill-api-auth/src/scopes.rs @@ -261,6 +261,7 @@ pub enum ScopeDomain { KafkaTriggers, NatsTriggers, MqttTriggers, + AmqpTriggers, SqsTriggers, GcpTriggers, AzureTriggers, @@ -323,6 +324,7 @@ impl ScopeDomain { Self::KafkaTriggers => "kafka_triggers", Self::NatsTriggers => "nats_triggers", Self::MqttTriggers => "mqtt_triggers", + Self::AmqpTriggers => "amqp_triggers", Self::SqsTriggers => "sqs_triggers", Self::GcpTriggers => "gcp_triggers", Self::AzureTriggers => "azure_triggers", @@ -378,6 +380,7 @@ impl ScopeDomain { "kafka_triggers" => Some(Self::KafkaTriggers), "nats_triggers" => Some(Self::NatsTriggers), "mqtt_triggers" => Some(Self::MqttTriggers), + "amqp_triggers" => Some(Self::AmqpTriggers), "sqs_triggers" => Some(Self::SqsTriggers), "gcp_triggers" => Some(Self::GcpTriggers), "azure_triggers" => Some(Self::AzureTriggers), diff --git a/backend/windmill-api-groups/src/granular_acls.rs b/backend/windmill-api-groups/src/granular_acls.rs index 663644e8e0..e413165825 100644 --- a/backend/windmill-api-groups/src/granular_acls.rs +++ b/backend/windmill-api-groups/src/granular_acls.rs @@ -46,13 +46,13 @@ fn audit_action_prefix_for_acl_kind(kind: &str) -> Option<&'static str> { "variable" => Some("variables"), "schedule" => Some("schedules"), "http_trigger" | "websocket_trigger" | "kafka_trigger" | "nats_trigger" - | "postgres_trigger" | "mqtt_trigger" | "gcp_trigger" | "azure_trigger" | "sqs_trigger" - | "email_trigger" => Some("triggers"), + | "postgres_trigger" | "mqtt_trigger" | "amqp_trigger" | "gcp_trigger" + | "azure_trigger" | "sqs_trigger" | "email_trigger" => Some("triggers"), _ => None, } } -const KINDS: [&str; 20] = [ +const KINDS: [&str; 21] = [ "script", "group_", "resource", @@ -68,6 +68,7 @@ const KINDS: [&str; 20] = [ "nats_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "gcp_trigger", "azure_trigger", "sqs_trigger", diff --git a/backend/windmill-api-users/src/users.rs b/backend/windmill-api-users/src/users.rs index b98bc6f414..0e22f4a09d 100644 --- a/backend/windmill-api-users/src/users.rs +++ b/backend/windmill-api-users/src/users.rs @@ -1751,6 +1751,7 @@ pub async fn delete_workspace_user_internal( "kafka_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "nats_trigger", "sqs_trigger", "gcp_trigger", diff --git a/backend/windmill-api-workspaces/src/workspaces.rs b/backend/windmill-api-workspaces/src/workspaces.rs index 83fdc51e38..99fb33e417 100644 --- a/backend/windmill-api-workspaces/src/workspaces.rs +++ b/backend/windmill-api-workspaces/src/workspaces.rs @@ -4444,6 +4444,7 @@ struct UsedTriggers { pub nats_used: bool, pub postgres_used: bool, pub mqtt_used: bool, + pub amqp_used: bool, pub sqs_used: bool, pub gcp_used: bool, pub azure_used: bool, @@ -4469,6 +4470,7 @@ async fn get_used_triggers( EXISTS(SELECT 1 FROM nats_trigger WHERE workspace_id = $1) as "nats_used!", EXISTS(SELECT 1 FROM postgres_trigger WHERE workspace_id = $1) AS "postgres_used!", EXISTS(SELECT 1 FROM mqtt_trigger WHERE workspace_id = $1) AS "mqtt_used!", + EXISTS(SELECT 1 FROM amqp_trigger WHERE workspace_id = $1) AS "amqp_used!", EXISTS(SELECT 1 FROM sqs_trigger WHERE workspace_id = $1) AS "sqs_used!", EXISTS(SELECT 1 FROM gcp_trigger WHERE workspace_id = $1) AS "gcp_used!", EXISTS(SELECT 1 FROM azure_trigger WHERE workspace_id = $1) AS "azure_used!", @@ -5146,6 +5148,23 @@ async fn clone_triggers_and_schedules( .execute(&mut **tx) .await?; + sqlx::query!( + r#"INSERT INTO amqp_trigger ( + amqp_resource_path, queue_name, exchange, options, path, script_path, is_flow, + workspace_id, edited_by, edited_at, extra_perms, server_id, last_server_ping, + error, error_handler_path, error_handler_args, retry, mode, permissioned_as, labels + ) + SELECT + amqp_resource_path, queue_name, exchange, options, path, script_path, is_flow, + $1, edited_by, edited_at, extra_perms, NULL, NULL, + NULL, error_handler_path, error_handler_args, retry, 'disabled'::TRIGGER_MODE, permissioned_as, labels + FROM amqp_trigger WHERE workspace_id = $2"#, + target_workspace_id, + source_workspace_id, + ) + .execute(&mut **tx) + .await?; + sqlx::query!( r#"INSERT INTO sqs_trigger ( path, queue_url, aws_resource_path, message_attributes, script_path, diff --git a/backend/windmill-api-workspaces/src/workspaces_extra.rs b/backend/windmill-api-workspaces/src/workspaces_extra.rs index 7ef55be497..1166d04fe3 100644 --- a/backend/windmill-api-workspaces/src/workspaces_extra.rs +++ b/backend/windmill-api-workspaces/src/workspaces_extra.rs @@ -279,6 +279,15 @@ pub(crate) async fn change_workspace_id( .execute(&mut *tx) .await?; + info!("Updating amqp_trigger table"); + sqlx::query!( + "UPDATE amqp_trigger SET workspace_id = $1 WHERE workspace_id = $2", + &rw.new_id, + &old_id + ) + .execute(&mut *tx) + .await?; + info!("Updating gcp_trigger table"); sqlx::query!( "UPDATE gcp_trigger SET workspace_id = $1 WHERE workspace_id = $2", diff --git a/backend/windmill-api/Cargo.toml b/backend/windmill-api/Cargo.toml index fa9ed3f2e5..b764e6ff83 100644 --- a/backend/windmill-api/Cargo.toml +++ b/backend/windmill-api/Cargo.toml @@ -10,8 +10,8 @@ path = "src/lib.rs" [features] default = [] -private = ["windmill-audit/private", "windmill-common/private", "windmill-api-auth/private", "windmill-store/private", "windmill-api-users/private", "windmill-api-workspaces/private", "windmill-api-groups/private", "windmill-api-configs/private", "windmill-api-settings/private", "windmill-api-assets/private", "windmill-api-agent-workers?/private", "windmill-trigger-kafka?/private", "windmill-trigger-postgres?/private", "windmill-trigger-mqtt?/private", "windmill-trigger-websocket?/private", "windmill-trigger-nats?/private", "windmill-trigger-sqs?/private", "windmill-trigger-gcp?/private", "windmill-trigger-azure?/private", "windmill-trigger-email?/private", "windmill-git-sync/private", "windmill-autoscaling?/private", "windmill-object-store/private"] -enterprise = ["windmill-queue/enterprise", "windmill-audit/enterprise", "windmill-git-sync/enterprise", "windmill-common/enterprise", "windmill-worker?/enterprise", "windmill-api-auth/enterprise", "windmill-store/enterprise", "windmill-api-jobs/enterprise", "windmill-api-scripts/enterprise", "windmill-api-flows/enterprise", "windmill-api-users/enterprise", "windmill-api-workspaces/enterprise", "windmill-api-groups/enterprise", "windmill-api-configs/enterprise", "windmill-api-settings/enterprise", "windmill-api-schedule/enterprise", "windmill-api-agent-workers?/enterprise", "windmill-trigger/enterprise", "windmill-trigger-kafka?/enterprise", "windmill-trigger-postgres?/enterprise", "windmill-trigger-mqtt?/enterprise", "windmill-trigger-websocket?/enterprise", "windmill-trigger-email?/enterprise", "windmill-trigger-nats?/enterprise", "windmill-trigger-sqs?/enterprise", "windmill-trigger-gcp?/enterprise", "windmill-trigger-azure?/enterprise", "windmill-trigger-http?/enterprise", "windmill-native-triggers?/enterprise", "dep:windmill-autoscaling", "windmill-autoscaling/enterprise", "license"] +private = ["windmill-audit/private", "windmill-common/private", "windmill-api-auth/private", "windmill-store/private", "windmill-api-users/private", "windmill-api-workspaces/private", "windmill-api-groups/private", "windmill-api-configs/private", "windmill-api-settings/private", "windmill-api-assets/private", "windmill-api-agent-workers?/private", "windmill-trigger-kafka?/private", "windmill-trigger-postgres?/private", "windmill-trigger-mqtt?/private", "windmill-trigger-amqp?/private", "windmill-trigger-websocket?/private", "windmill-trigger-nats?/private", "windmill-trigger-sqs?/private", "windmill-trigger-gcp?/private", "windmill-trigger-azure?/private", "windmill-trigger-email?/private", "windmill-git-sync/private", "windmill-autoscaling?/private", "windmill-object-store/private"] +enterprise = ["windmill-queue/enterprise", "windmill-audit/enterprise", "windmill-git-sync/enterprise", "windmill-common/enterprise", "windmill-worker?/enterprise", "windmill-api-auth/enterprise", "windmill-store/enterprise", "windmill-api-jobs/enterprise", "windmill-api-scripts/enterprise", "windmill-api-flows/enterprise", "windmill-api-users/enterprise", "windmill-api-workspaces/enterprise", "windmill-api-groups/enterprise", "windmill-api-configs/enterprise", "windmill-api-settings/enterprise", "windmill-api-schedule/enterprise", "windmill-api-agent-workers?/enterprise", "windmill-trigger/enterprise", "windmill-trigger-kafka?/enterprise", "windmill-trigger-postgres?/enterprise", "windmill-trigger-mqtt?/enterprise", "windmill-trigger-amqp?/enterprise", "windmill-trigger-websocket?/enterprise", "windmill-trigger-email?/enterprise", "windmill-trigger-nats?/enterprise", "windmill-trigger-sqs?/enterprise", "windmill-trigger-gcp?/enterprise", "windmill-trigger-azure?/enterprise", "windmill-trigger-http?/enterprise", "windmill-native-triggers?/enterprise", "dep:windmill-autoscaling", "windmill-autoscaling/enterprise", "license"] stripe = [] run_inline = ["dep:windmill-worker", "windmill-api-configs/run_inline"] agent_worker_server = ["dep:windmill-worker", "dep:windmill-api-agent-workers"] @@ -34,6 +34,7 @@ http_trigger = ["dep:matchit", "dep:windmill-trigger-http", "windmill-store/http static_frontend = ["dep:rust-embed"] postgres_trigger = ["dep:windmill-trigger-postgres", "windmill-store/postgres_trigger"] mqtt_trigger = ["dep:windmill-trigger-mqtt", "windmill-store/mqtt_trigger"] +amqp_trigger = ["dep:windmill-trigger-amqp", "windmill-store/amqp_trigger"] native_trigger = ["dep:windmill-native-triggers", "windmill-native-triggers/native_trigger", "dep:strum", "oauth2"] sqs_trigger = ["dep:windmill-trigger-sqs", "windmill-store/sqs_trigger"] gcp_trigger = ["dep:windmill-trigger-gcp", "windmill-store/gcp_trigger"] @@ -141,6 +142,7 @@ matchit = { workspace = true, optional = true } windmill-trigger-kafka = { workspace = true, optional = true } windmill-trigger-postgres = { workspace = true, optional = true } windmill-trigger-mqtt = { workspace = true, optional = true } +windmill-trigger-amqp = { workspace = true, optional = true } windmill-trigger-websocket = { workspace = true, optional = true } windmill-trigger-email = { workspace = true, optional = true } windmill-trigger-nats = { workspace = true, optional = true } diff --git a/backend/windmill-api/openapi.yaml b/backend/windmill-api/openapi.yaml index 6b50450af9..81ebb3c72f 100644 --- a/backend/windmill-api/openapi.yaml +++ b/backend/windmill-api/openapi.yaml @@ -5629,6 +5629,8 @@ paths: type: boolean mqtt_used: type: boolean + amqp_used: + type: boolean gcp_used: type: boolean azure_used: @@ -5650,6 +5652,7 @@ paths: - nats_used - postgres_used - mqtt_used + - amqp_used - gcp_used - azure_used - sqs_used @@ -17457,6 +17460,211 @@ paths: schema: type: string + /w/{workspace}/amqp_triggers/create: + post: + summary: create amqp trigger + operationId: createAmqpTrigger + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + requestBody: + description: new amqp trigger + required: true + content: + application/json: + schema: + $ref: "#/components/schemas/NewAmqpTrigger" + responses: + "201": + description: amqp trigger created + content: + text/plain: + schema: + type: string + + /w/{workspace}/amqp_triggers/update/{path}: + post: + summary: update amqp trigger + operationId: updateAmqpTrigger + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + - $ref: "#/components/parameters/Path" + requestBody: + description: updated trigger + required: true + content: + application/json: + schema: + $ref: "#/components/schemas/EditAmqpTrigger" + responses: + "200": + description: amqp trigger updated + content: + text/plain: + schema: + type: string + + /w/{workspace}/amqp_triggers/delete/{path}: + delete: + summary: delete amqp trigger + operationId: deleteAmqpTrigger + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + - $ref: "#/components/parameters/Path" + responses: + "200": + description: amqp trigger deleted + content: + text/plain: + schema: + type: string + + /w/{workspace}/amqp_triggers/get/{path}: + get: + summary: get amqp trigger + operationId: getAmqpTrigger + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + - $ref: "#/components/parameters/Path" + - $ref: "#/components/parameters/GetDraft" + responses: + "200": + description: amqp trigger retrieved + content: + application/json: + schema: + allOf: + - $ref: "#/components/schemas/AmqpTrigger" + - $ref: "#/components/schemas/UserDraftOverlay" + + /w/{workspace}/amqp_triggers/list: + get: + summary: list amqp triggers + operationId: listAmqpTriggers + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + required: true + - $ref: "#/components/parameters/Page" + - $ref: "#/components/parameters/PerPage" + - name: path + description: filter by path + in: query + schema: + type: string + - name: is_flow + in: query + schema: + type: boolean + - name: path_start + in: query + schema: + type: string + - name: label + in: query + required: false + schema: + type: string + description: Filter by label + - $ref: "#/components/parameters/IncludeDraftOnly" + responses: + "200": + description: amqp trigger list + content: + application/json: + schema: + type: array + items: + $ref: "#/components/schemas/AmqpTrigger" + + /w/{workspace}/amqp_triggers/exists/{path}: + get: + summary: does amqp trigger exists + operationId: existsAmqpTrigger + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + - $ref: "#/components/parameters/Path" + responses: + "200": + description: amqp trigger exists + content: + application/json: + schema: + type: boolean + + /w/{workspace}/amqp_triggers/setmode/{path}: + post: + summary: set enabled amqp trigger + operationId: setAmqpTriggerMode + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + - $ref: "#/components/parameters/Path" + requestBody: + description: updated amqp trigger enable + required: true + content: + application/json: + schema: + type: object + properties: + mode: + $ref: "#/components/schemas/TriggerMode" + force: + type: boolean + description: > + Bypass the parent-state conflict warning when enabling a + trigger in a fork whose parent has the same path enabled. + required: + - mode + responses: + "200": + description: amqp trigger enabled set + content: + text/plain: + schema: + type: string + + /w/{workspace}/amqp_triggers/test: + post: + summary: test amqp connection + operationId: testAmqpConnection + tags: + - amqp_trigger + parameters: + - $ref: "#/components/parameters/WorkspaceId" + requestBody: + description: test amqp connection + required: true + content: + application/json: + schema: + type: object + properties: + amqp_resource_path: + type: string + description: Path to the AMQP resource containing broker connection configuration + required: + - amqp_resource_path + responses: + "200": + description: successfully connected to amqp + content: + text/plain: + schema: + type: string + /w/{workspace}/gcp_triggers/create: post: summary: create gcp trigger @@ -19966,6 +20174,7 @@ paths: nats_trigger, postgres_trigger, mqtt_trigger, + amqp_trigger, gcp_trigger, azure_trigger, sqs_trigger, @@ -20013,6 +20222,7 @@ paths: nats_trigger, postgres_trigger, mqtt_trigger, + amqp_trigger, gcp_trigger, azure_trigger, sqs_trigger, @@ -20071,6 +20281,7 @@ paths: nats_trigger, postgres_trigger, mqtt_trigger, + amqp_trigger, gcp_trigger, azure_trigger, sqs_trigger, @@ -23229,6 +23440,7 @@ components: - trigger_kafka - trigger_nats - trigger_mqtt + - trigger_amqp - trigger_sqs - trigger_gcp - trigger_azure @@ -26415,6 +26627,7 @@ components: - kafka - nats - mqtt + - amqp - sqs - gcp - azure @@ -26919,6 +27132,8 @@ components: type: number mqtt_count: type: number + amqp_count: + type: number gcp_count: type: number azure_count: @@ -27435,6 +27650,183 @@ components: - subscribe_topics - mqtt_resource_path + AmqpExchange: + type: object + properties: + exchange_name: + type: string + description: Name of the exchange to bind the consumed queue to + routing_keys: + type: array + items: + type: string + description: Routing keys used to bind the queue to the exchange + required: + - exchange_name + + AmqpOptions: + type: object + properties: + declare_queue: + type: boolean + description: Declare the queue (durable) before consuming; when false the queue is declared passively and must already exist + prefetch_count: + type: integer + format: int32 + minimum: 1 + maximum: 65535 + description: Maximum number of unacknowledged messages the broker delivers at once (1-65535) + + AmqpTrigger: + allOf: + - $ref: "#/components/schemas/TriggerExtraProperty" + type: object + properties: + amqp_resource_path: + type: string + description: Path to the AMQP resource containing broker connection configuration + queue_name: + type: string + description: Name of the queue to consume messages from + exchange: + $ref: "#/components/schemas/AmqpExchange" + nullable: true + description: Optional exchange binding for the consumed queue + options: + $ref: "#/components/schemas/AmqpOptions" + nullable: true + description: Optional consumer options (queue declaration, prefetch) + server_id: + type: string + description: ID of the server currently handling this trigger (internal) + last_server_ping: + type: string + format: date-time + description: Timestamp of last server heartbeat (internal) + error: + type: string + description: Last error message if the trigger failed + error_handler_path: + type: string + description: Path to a script or flow to run when the triggered job fails + error_handler_args: + $ref: "#/components/schemas/ScriptArgs" + description: Arguments to pass to the error handler + retry: + $ref: "../../openflow.openapi.yaml#/components/schemas/Retry" + description: Retry configuration for failed executions + required: + - amqp_resource_path + - queue_name + + NewAmqpTrigger: + type: object + properties: + amqp_resource_path: + type: string + description: Path to the AMQP resource containing broker connection configuration + queue_name: + type: string + description: Name of the queue to consume messages from + exchange: + nullable: true + $ref: "#/components/schemas/AmqpExchange" + description: Optional exchange binding for the consumed queue + options: + nullable: true + $ref: "#/components/schemas/AmqpOptions" + description: Optional consumer options (queue declaration, prefetch) + path: + type: string + description: The unique Windmill path for this trigger. Must be of the form `u//` or `f//`. + script_path: + type: string + description: Path to the script or flow to execute when a message is received + is_flow: + type: boolean + description: True if script_path points to a flow, false if it points to a script + mode: + $ref: "#/components/schemas/TriggerMode" + error_handler_path: + type: string + description: Path to a script or flow to run when the triggered job fails + error_handler_args: + $ref: "#/components/schemas/ScriptArgs" + description: Arguments to pass to the error handler + retry: + $ref: "../../openflow.openapi.yaml#/components/schemas/Retry" + description: Retry configuration for failed executions + permissioned_as: + type: string + description: The user or group this trigger runs as. Used during deployment to preserve the original trigger owner. + preserve_permissioned_as: + type: boolean + description: "When true and the caller is a member of the 'wm_deployers' group, preserves the original permissioned_as value instead of overwriting it." + labels: + type: array + items: + type: string + required: + - path + - script_path + - is_flow + - amqp_resource_path + - queue_name + + EditAmqpTrigger: + type: object + properties: + amqp_resource_path: + type: string + description: Path to the AMQP resource containing broker connection configuration + queue_name: + type: string + description: Name of the queue to consume messages from + exchange: + nullable: true + $ref: "#/components/schemas/AmqpExchange" + description: Optional exchange binding for the consumed queue + options: + nullable: true + $ref: "#/components/schemas/AmqpOptions" + description: Optional consumer options (queue declaration, prefetch) + path: + type: string + description: The unique Windmill path for this trigger. Must be of the form `u//` or `f//`. + script_path: + type: string + description: Path to the script or flow to execute when a message is received + is_flow: + type: boolean + description: True if script_path points to a flow, false if it points to a script + mode: + $ref: "#/components/schemas/TriggerMode" + error_handler_path: + type: string + description: Path to a script or flow to run when the triggered job fails + error_handler_args: + $ref: "#/components/schemas/ScriptArgs" + description: Arguments to pass to the error handler + retry: + $ref: "../../openflow.openapi.yaml#/components/schemas/Retry" + description: Retry configuration for failed executions + permissioned_as: + type: string + description: The user or group this trigger runs as. Used during deployment to preserve the original trigger owner. + preserve_permissioned_as: + type: boolean + description: "When true and the caller is a member of the 'wm_deployers' group, preserves the original permissioned_as value instead of overwriting it." + labels: + type: array + items: + type: string + required: + - path + - script_path + - is_flow + - amqp_resource_path + - queue_name + DeliveryType: type: string enum: @@ -30393,6 +30785,7 @@ components: postgres, sqs, mqtt, + amqp, gcp, azure, email, @@ -30575,6 +30968,7 @@ components: "nats_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "sqs_trigger", "gcp_trigger", "azure_trigger", diff --git a/backend/windmill-api/src/capture.rs b/backend/windmill-api/src/capture.rs index bf98ae460d..1eed038c25 100644 --- a/backend/windmill-api/src/capture.rs +++ b/backend/windmill-api/src/capture.rs @@ -48,6 +48,8 @@ use windmill_common::error::Error; #[cfg(all(feature = "enterprise", feature = "kafka", feature = "private"))] use crate::triggers::kafka::KafkaTriggerConfigConnection; +#[cfg(feature = "amqp_trigger")] +use crate::triggers::amqp::{AmqpOptions, ExchangeConfig}; #[cfg(feature = "mqtt_trigger")] use crate::triggers::mqtt::{MqttClientVersion, MqttV3Config, MqttV5Config, SubscribeTopic}; @@ -234,6 +236,14 @@ pub struct MqttTriggerConfig { pub client_version: Option, pub client_id: Option, } +#[cfg(feature = "amqp_trigger")] +#[derive(Debug, Serialize, Deserialize)] +pub struct AmqpTriggerConfig { + pub amqp_resource_path: String, + pub queue_name: String, + pub exchange: Option, + pub options: Option, +} #[cfg(feature = "postgres_trigger")] #[derive(Serialize, Deserialize, Debug)] pub struct PostgresTriggerConfig { @@ -271,6 +281,8 @@ enum TriggerConfig { Nats(NatsTriggerConfig), #[cfg(feature = "mqtt_trigger")] Mqtt(MqttTriggerConfig), + #[cfg(feature = "amqp_trigger")] + Amqp(AmqpTriggerConfig), #[cfg(all(feature = "enterprise", feature = "gcp_trigger", feature = "private"))] Gcp(GcpTriggerConfig), #[cfg(all(feature = "enterprise", feature = "azure_trigger", feature = "private"))] diff --git a/backend/windmill-api/src/offboarding.rs b/backend/windmill-api/src/offboarding.rs index 24b6d3f10d..f9bf0e1449 100644 --- a/backend/windmill-api/src/offboarding.rs +++ b/backend/windmill-api/src/offboarding.rs @@ -186,6 +186,7 @@ async fn get_offboard_preview( "kafka_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "nats_trigger", "sqs_trigger", "gcp_trigger", @@ -750,6 +751,7 @@ async fn check_path_conflicts( "kafka_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "nats_trigger", "sqs_trigger", "gcp_trigger", @@ -1017,13 +1019,14 @@ async fn offboard_user_from_workspace<'c>( &new_permissioned_as, username, w_id ).execute(&mut **tx).await?; - // ---- triggers (all 9 types with path/permissioned_as) ---- + // ---- triggers (all 10 types with path/permissioned_as) ---- let trigger_tables = [ "http_trigger", "websocket_trigger", "kafka_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "nats_trigger", "sqs_trigger", "gcp_trigger", diff --git a/backend/windmill-api/src/token.rs b/backend/windmill-api/src/token.rs index 8bcd02d2fc..c7229fe056 100644 --- a/backend/windmill-api/src/token.rs +++ b/backend/windmill-api/src/token.rs @@ -24,6 +24,7 @@ fn build_trigger_scope_domains() -> Vec { ("kafka_triggers", "Kafka"), ("nats_triggers", "NATS"), ("mqtt_triggers", "MQTT"), + ("amqp_triggers", "AMQP"), ("sqs_triggers", "AWS SQS"), ("gcp_triggers", "GCP Pub/Sub"), ("azure_triggers", "Azure Event Grid"), diff --git a/backend/windmill-api/src/trash.rs b/backend/windmill-api/src/trash.rs index d0d7ddc6c2..416546b850 100644 --- a/backend/windmill-api/src/trash.rs +++ b/backend/windmill-api/src/trash.rs @@ -467,6 +467,7 @@ async fn restore_trigger(tx: &mut sqlx::PgConnection, item: &TrashItemWithData) "nats_trigger", "postgres_trigger", "mqtt_trigger", + "amqp_trigger", "sqs_trigger", "gcp_trigger", "azure_trigger", diff --git a/backend/windmill-api/src/triggers/amqp/mod.rs b/backend/windmill-api/src/triggers/amqp/mod.rs new file mode 100644 index 0000000000..81aaca3aa8 --- /dev/null +++ b/backend/windmill-api/src/triggers/amqp/mod.rs @@ -0,0 +1 @@ +pub use windmill_trigger_amqp::*; diff --git a/backend/windmill-api/src/triggers/handler.rs b/backend/windmill-api/src/triggers/handler.rs index 35b025965d..d86912d8d0 100644 --- a/backend/windmill-api/src/triggers/handler.rs +++ b/backend/windmill-api/src/triggers/handler.rs @@ -61,6 +61,16 @@ pub fn generate_trigger_routers() -> Router { ); } + #[cfg(feature = "amqp_trigger")] + { + use crate::triggers::amqp::AmqpTrigger; + + router = router.nest( + AmqpTrigger::ROUTE_PREFIX, + complete_trigger_routes(AmqpTrigger), + ); + } + #[cfg(all(feature = "enterprise", feature = "sqs_trigger", feature = "private"))] { use crate::triggers::sqs::SqsTrigger; @@ -143,6 +153,7 @@ pub struct TriggersCount { nats_count: i64, postgres_count: i64, mqtt_count: i64, + amqp_count: i64, sqs_count: i64, gcp_count: i64, azure_count: i64, @@ -245,6 +256,17 @@ pub async fn get_triggers_count_internal( #[cfg(not(feature = "mqtt_trigger"))] let mqtt_count = 0; + #[cfg(feature = "amqp_trigger")] + let amqp_count = { + use crate::triggers::amqp::AmqpTrigger; + let count = AmqpTrigger + .trigger_count(&mut tx, w_id, is_flow, path) + .await; + count + }; + #[cfg(not(feature = "amqp_trigger"))] + let amqp_count = 0; + #[cfg(all(feature = "sqs_trigger", feature = "enterprise", feature = "private"))] let sqs_count = { use crate::triggers::sqs::SqsTrigger; @@ -362,6 +384,7 @@ pub async fn get_triggers_count_internal( nats_count, postgres_count, mqtt_count, + amqp_count, gcp_count, azure_count, sqs_count, diff --git a/backend/windmill-api/src/triggers/listener.rs b/backend/windmill-api/src/triggers/listener.rs index 43cabf13e3..d12ff7a1e2 100644 --- a/backend/windmill-api/src/triggers/listener.rs +++ b/backend/windmill-api/src/triggers/listener.rs @@ -51,6 +51,14 @@ pub fn start_all_listeners(db: DB, killpill_rx: &tokio::sync::broadcast::Receive listen_to(MqttTrigger, db.clone(), mqtt_killpill_rx) } + #[cfg(feature = "amqp_trigger")] + { + let amqp_killpill_rx = killpill_rx.resubscribe(); + use crate::triggers::amqp::AmqpTrigger; + + listen_to(AmqpTrigger, db.clone(), amqp_killpill_rx) + } + #[cfg(feature = "websocket")] { let mqtt_killpill_rx = killpill_rx.resubscribe(); diff --git a/backend/windmill-api/src/triggers/mod.rs b/backend/windmill-api/src/triggers/mod.rs index bc69c66b2b..10f45cbd7f 100644 --- a/backend/windmill-api/src/triggers/mod.rs +++ b/backend/windmill-api/src/triggers/mod.rs @@ -1,4 +1,6 @@ // Concrete trigger submodules (feature-gated) +#[cfg(feature = "amqp_trigger")] +pub mod amqp; #[cfg(all(feature = "azure_trigger", feature = "enterprise", feature = "private"))] pub mod azure; #[cfg(all(feature = "smtp", feature = "private"))] diff --git a/backend/windmill-api/src/workspaces_export.rs b/backend/windmill-api/src/workspaces_export.rs index 5cf198c08b..c869d87780 100644 --- a/backend/windmill-api/src/workspaces_export.rs +++ b/backend/windmill-api/src/workspaces_export.rs @@ -19,6 +19,7 @@ use windmill_api_auth::check_scopes; feature = "websocket", feature = "postgres_trigger", feature = "mqtt_trigger", + feature = "amqp_trigger", all( feature = "enterprise", any( @@ -144,6 +145,7 @@ pub fn is_none_or_false(val: &Option) -> bool { feature = "websocket", feature = "postgres_trigger", feature = "mqtt_trigger", + feature = "amqp_trigger", feature = "native_trigger", all( feature = "enterprise", @@ -189,6 +191,7 @@ async fn fork_parent_trigger_modes( feature = "websocket", feature = "postgres_trigger", feature = "mqtt_trigger", + feature = "amqp_trigger", feature = "native_trigger", all( feature = "enterprise", @@ -1226,6 +1229,36 @@ pub(crate) async fn tarball_workspace( } } + #[cfg(feature = "amqp_trigger")] + { + use crate::triggers::amqp::AmqpTrigger; + let handler = AmqpTrigger; + let amqp_triggers = handler.list_triggers(&mut *tx, &w_id, None, None).await?; + let parent_modes = fork_parent_trigger_modes( + &db, + ::TABLE_NAME, + parent_workspace_id.as_deref(), + ) + .await?; + + for trigger in amqp_triggers { + let mode_override = trigger_mode_override(&parent_modes, &trigger.base.path); + let trigger_str = &to_string_without_metadata_inner( + &trigger, + ExtraPermsBehavior::Drop, + None, + mode_override.as_ref(), + ) + .unwrap(); + archive + .write_to_archive( + &trigger_str, + &format!("{}.amqp_trigger.json", trigger.base.path), + ) + .await?; + } + } + #[cfg(all(feature = "enterprise", feature = "smtp", feature = "private"))] { use crate::triggers::email::EmailTrigger; diff --git a/backend/windmill-common/src/assets.rs b/backend/windmill-common/src/assets.rs index 62f6d32070..d1bbed1a18 100644 --- a/backend/windmill-common/src/assets.rs +++ b/backend/windmill-common/src/assets.rs @@ -632,6 +632,7 @@ pub fn trigger_spec_to_row(spec: &TriggerSpec) -> Option<(ScriptTriggerKind, Str | TriggerSpec::Email | TriggerSpec::Kafka | TriggerSpec::Mqtt + | TriggerSpec::Amqp | TriggerSpec::Nats | TriggerSpec::Postgres | TriggerSpec::Sqs diff --git a/backend/windmill-common/src/triggers.rs b/backend/windmill-common/src/triggers.rs index b4f8f472cc..d667363ebb 100644 --- a/backend/windmill-common/src/triggers.rs +++ b/backend/windmill-common/src/triggers.rs @@ -39,7 +39,8 @@ pub async fn update_triggers_script_path( t3 AS (UPDATE postgres_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), \ t4 AS (UPDATE mqtt_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), \ t5 AS (UPDATE nats_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), \ - t6 AS (UPDATE sqs_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4) \ + t6 AS (UPDATE sqs_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4), \ + t7 AS (UPDATE amqp_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4) \ UPDATE gcp_trigger SET script_path = $1, server_id = NULL WHERE script_path = $2 AND workspace_id = $3 AND is_flow = $4", new_path, old_path, diff --git a/backend/windmill-common/src/user_drafts.rs b/backend/windmill-common/src/user_drafts.rs index 7e0c1f3f55..84a0e8c3ea 100644 --- a/backend/windmill-common/src/user_drafts.rs +++ b/backend/windmill-common/src/user_drafts.rs @@ -45,6 +45,7 @@ pub enum UserDraftItemKind { TriggerKafka, TriggerNats, TriggerMqtt, + TriggerAmqp, TriggerSqs, TriggerGcp, TriggerAzure, @@ -80,6 +81,7 @@ impl UserDraftItemKind { UserDraftItemKind::TriggerKafka => "trigger_kafka", UserDraftItemKind::TriggerNats => "trigger_nats", UserDraftItemKind::TriggerMqtt => "trigger_mqtt", + UserDraftItemKind::TriggerAmqp => "trigger_amqp", UserDraftItemKind::TriggerSqs => "trigger_sqs", UserDraftItemKind::TriggerGcp => "trigger_gcp", UserDraftItemKind::TriggerAzure => "trigger_azure", @@ -94,7 +96,7 @@ impl UserDraftItemKind { /// Every variant, for code that must enumerate kinds (e.g. generating /// the `draft_only` existence SQL). - pub const ALL: [UserDraftItemKind; 25] = [ + pub const ALL: [UserDraftItemKind; 26] = [ UserDraftItemKind::Script, UserDraftItemKind::Flow, UserDraftItemKind::App, @@ -111,6 +113,7 @@ impl UserDraftItemKind { UserDraftItemKind::TriggerKafka, UserDraftItemKind::TriggerNats, UserDraftItemKind::TriggerMqtt, + UserDraftItemKind::TriggerAmqp, UserDraftItemKind::TriggerSqs, UserDraftItemKind::TriggerGcp, UserDraftItemKind::TriggerAzure, @@ -144,6 +147,7 @@ impl UserDraftItemKind { TriggerKafka => Some("kafka_trigger"), TriggerNats => Some("nats_trigger"), TriggerMqtt => Some("mqtt_trigger"), + TriggerAmqp => Some("amqp_trigger"), TriggerSqs => Some("sqs_trigger"), TriggerGcp => Some("gcp_trigger"), TriggerAzure => Some("azure_trigger"), diff --git a/backend/windmill-git-sync/src/lib.rs b/backend/windmill-git-sync/src/lib.rs index b7c714e5c8..cf000244e5 100644 --- a/backend/windmill-git-sync/src/lib.rs +++ b/backend/windmill-git-sync/src/lib.rs @@ -93,6 +93,10 @@ pub enum DeployedObject { path: String, parent_path: Option, }, + AmqpTrigger { + path: String, + parent_path: Option, + }, SqsTrigger { path: String, parent_path: Option, @@ -144,6 +148,7 @@ impl DeployedObject { DeployedObject::NatsTrigger { path, .. } => path.to_owned(), DeployedObject::PostgresTrigger { path, .. } => path.to_owned(), DeployedObject::MqttTrigger { path, .. } => path.to_owned(), + DeployedObject::AmqpTrigger { path, .. } => path.to_owned(), DeployedObject::SqsTrigger { path, .. } => path.to_owned(), DeployedObject::GcpTrigger { path, .. } => path.to_owned(), DeployedObject::AzureTrigger { path, .. } => path.to_owned(), @@ -186,6 +191,7 @@ impl DeployedObject { DeployedObject::NatsTrigger { parent_path, .. } => parent_path.to_owned(), DeployedObject::PostgresTrigger { parent_path, .. } => parent_path.to_owned(), DeployedObject::MqttTrigger { parent_path, .. } => parent_path.to_owned(), + DeployedObject::AmqpTrigger { parent_path, .. } => parent_path.to_owned(), DeployedObject::SqsTrigger { parent_path, .. } => parent_path.to_owned(), DeployedObject::GcpTrigger { parent_path, .. } => parent_path.to_owned(), DeployedObject::AzureTrigger { parent_path, .. } => parent_path.to_owned(), @@ -216,6 +222,7 @@ impl DeployedObject { DeployedObject::NatsTrigger { .. } => "nats_trigger", DeployedObject::PostgresTrigger { .. } => "postgres_trigger", DeployedObject::MqttTrigger { .. } => "mqtt_trigger", + DeployedObject::AmqpTrigger { .. } => "amqp_trigger", DeployedObject::SqsTrigger { .. } => "sqs_trigger", DeployedObject::GcpTrigger { .. } => "gcp_trigger", DeployedObject::AzureTrigger { .. } => "azure_trigger", @@ -446,6 +453,10 @@ mod tests { DeployedObject::MqttTrigger { path: "t".to_string(), parent_path: None }.get_kind(), "mqtt_trigger" ); + assert_eq!( + DeployedObject::AmqpTrigger { path: "t".to_string(), parent_path: None }.get_kind(), + "amqp_trigger" + ); assert_eq!( DeployedObject::SqsTrigger { path: "t".to_string(), parent_path: None }.get_kind(), "sqs_trigger" diff --git a/backend/windmill-store/Cargo.toml b/backend/windmill-store/Cargo.toml index fdd82a7e5d..1409e30ebf 100644 --- a/backend/windmill-store/Cargo.toml +++ b/backend/windmill-store/Cargo.toml @@ -19,6 +19,7 @@ mcp = ["dep:windmill-mcp", "windmill-mcp/server", "windmill-mcp/auth"] http_trigger = [] postgres_trigger = [] mqtt_trigger = [] +amqp_trigger = [] sqs_trigger = [] gcp_trigger = [] azure_trigger = [] diff --git a/backend/windmill-store/src/resources.rs b/backend/windmill-store/src/resources.rs index 5ea1638baf..7a3ca50e73 100644 --- a/backend/windmill-store/src/resources.rs +++ b/backend/windmill-store/src/resources.rs @@ -2386,6 +2386,7 @@ async fn update_resource_type( feature = "http_trigger", feature = "postgres_trigger", feature = "mqtt_trigger", + feature = "amqp_trigger", all( feature = "enterprise", any( diff --git a/backend/windmill-trigger-amqp/Cargo.toml b/backend/windmill-trigger-amqp/Cargo.toml new file mode 100644 index 0000000000..4c5ce41941 --- /dev/null +++ b/backend/windmill-trigger-amqp/Cargo.toml @@ -0,0 +1,37 @@ +[package] +name = "windmill-trigger-amqp" +version.workspace = true +authors.workspace = true +edition.workspace = true + +[lib] +name = "windmill_trigger_amqp" +path = "src/lib.rs" + +[features] +default = [] +enterprise = ["windmill-common/enterprise", "windmill-store/enterprise", "windmill-trigger/enterprise"] +private = ["windmill-common/private", "windmill-store/private"] + +[dependencies] +windmill-common = { workspace = true, default-features = false } +windmill-api-auth.workspace = true +windmill-store = { workspace = true, features = ["amqp_trigger"] } +windmill-trigger.workspace = true +windmill-git-sync.workspace = true +lapin.workspace = true +tokio-executor-trait.workspace = true +tokio-reactor-trait.workspace = true +futures.workspace = true +urlencoding.workspace = true +axum.workspace = true +serde.workspace = true +serde_json.workspace = true +sqlx.workspace = true +tokio.workspace = true +tracing.workspace = true +async-trait.workspace = true +itertools.workspace = true +base64.workspace = true +anyhow.workspace = true +thiserror.workspace = true diff --git a/backend/windmill-trigger-amqp/src/handler.rs b/backend/windmill-trigger-amqp/src/handler.rs new file mode 100644 index 0000000000..48710224df --- /dev/null +++ b/backend/windmill-trigger-amqp/src/handler.rs @@ -0,0 +1,203 @@ +use async_trait::async_trait; +use sqlx::{types::Json as SqlxJson, PgConnection}; +use windmill_api_auth::ApiAuthed; +use windmill_common::DB; +use windmill_common::{ + db::UserDB, + error::{Error, Result}, +}; +use windmill_git_sync::DeployedObject; +use windmill_store::resources::try_get_resource_from_db_as; +use windmill_trigger::{Trigger, TriggerCrud, TriggerData}; + +use super::{ + AmqpClientBuilder, AmqpConfig, AmqpConfigRequest, AmqpOptions, AmqpResource, AmqpTrigger, + ExchangeConfig, TestAmqpConfig, +}; + +#[async_trait] +impl TriggerCrud for AmqpTrigger { + type TriggerConfig = AmqpConfig; + type Trigger = Trigger; + type TriggerConfigRequest = AmqpConfigRequest; + type TestConnectionConfig = TestAmqpConfig; + + const TABLE_NAME: &'static str = "amqp_trigger"; + const TRIGGER_TYPE: &'static str = "amqp"; + const DRAFT_KIND: windmill_common::user_drafts::UserDraftItemKind = + windmill_common::user_drafts::UserDraftItemKind::TriggerAmqp; + const SUPPORTS_SERVER_STATE: bool = true; + const SUPPORTS_TEST_CONNECTION: bool = true; + const ROUTE_PREFIX: &'static str = "/amqp_triggers"; + const DEPLOYMENT_NAME: &'static str = "AMQP trigger"; + const ADDITIONAL_SELECT_FIELDS: &[&'static str] = + &["amqp_resource_path", "queue_name", "exchange", "options"]; + const IS_ALLOWED_ON_CLOUD: bool = false; + + fn get_deployed_object(path: String, parent_path: Option) -> DeployedObject { + DeployedObject::AmqpTrigger { path, parent_path } + } + + async fn validate_config( + &self, + _db: &DB, + config: &Self::TriggerConfigRequest, + _workspace_id: &str, + ) -> Result<()> { + if config.amqp_resource_path.trim().is_empty() { + return Err(Error::BadRequest( + "AMQP resource path cannot be empty".to_string(), + )); + } + + if config.queue_name.trim().is_empty() { + return Err(Error::BadRequest("Queue name cannot be empty".to_string())); + } + + super::validate_amqp_options(config.options.as_ref()).map_err(Error::BadRequest)?; + + Ok(()) + } + + async fn create_trigger( + &self, + _db: &DB, + tx: &mut PgConnection, + authed: &ApiAuthed, + w_id: &str, + trigger: TriggerData, + ) -> Result<()> { + let resolved_edited_by = trigger.base.resolve_edited_by(authed); + let resolved_permissioned_as = trigger.base.resolve_permissioned_as(authed); + let exchange = trigger.config.exchange.map(SqlxJson); + let options = trigger.config.options.map(SqlxJson); + + sqlx::query!( + r#" + INSERT INTO amqp_trigger ( + amqp_resource_path, + queue_name, + exchange, + options, + workspace_id, + path, + script_path, + is_flow, + permissioned_as, + mode, + edited_by, + error_handler_path, + error_handler_args, + retry + ) + VALUES ( + $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14 + )"#, + trigger.config.amqp_resource_path, + trigger.config.queue_name, + exchange as Option>, + options as Option>, + w_id, + trigger.base.path, + trigger.base.script_path, + trigger.base.is_flow, + resolved_permissioned_as, + trigger.base.mode() as _, + &resolved_edited_by, + trigger.error_handling.error_handler_path, + trigger.error_handling.error_handler_args as _, + trigger.error_handling.retry as _ + ) + .execute(tx) + .await?; + + Ok(()) + } + + async fn update_trigger( + &self, + _db: &DB, + tx: &mut PgConnection, + authed: &ApiAuthed, + workspace_id: &str, + path: &str, + trigger: TriggerData, + ) -> Result<()> { + let resolved_edited_by = trigger.base.resolve_edited_by(authed); + let resolved_permissioned_as = trigger.base.resolve_permissioned_as(authed); + let exchange = trigger.config.exchange.map(SqlxJson); + let options = trigger.config.options.map(SqlxJson); + + // Important to set server_id to NULL to stop the current amqp listener + sqlx::query!( + r#" + UPDATE + amqp_trigger + SET + amqp_resource_path = $1, + queue_name = $2, + exchange = $3, + options = $4, + is_flow = $5, + edited_by = $6, + permissioned_as = $7, + script_path = $8, + path = $9, + edited_at = now(), + error = NULL, + server_id = NULL, + error_handler_path = $12, + error_handler_args = $13, + retry = $14 + WHERE + workspace_id = $10 AND + path = $11 + "#, + trigger.config.amqp_resource_path, + trigger.config.queue_name, + exchange as Option>, + options as Option>, + trigger.base.is_flow, + &resolved_edited_by, + resolved_permissioned_as, + trigger.base.script_path, + trigger.base.path, + workspace_id, + path, + trigger.error_handling.error_handler_path, + trigger.error_handling.error_handler_args as _, + trigger.error_handling.retry as _ + ) + .execute(tx) + .await?; + + Ok(()) + } + + async fn test_connection( + &self, + db: &DB, + authed: &ApiAuthed, + user_db: &UserDB, + workspace_id: &str, + config: Self::TestConnectionConfig, + ) -> Result<()> { + let amqp_resource = try_get_resource_from_db_as::( + authed, + Some(user_db.clone()), + db, + &config.amqp_resource_path, + workspace_id, + ) + .await?; + + let client_builder = AmqpClientBuilder::new(amqp_resource, "", None, None); + + client_builder + .test_connection() + .await + .map_err(|err| Error::BadConfig(format!("Error connecting to AMQP broker: {}", err)))?; + + Ok(()) + } +} diff --git a/backend/windmill-trigger-amqp/src/lib.rs b/backend/windmill-trigger-amqp/src/lib.rs new file mode 100644 index 0000000000..24ca19036d --- /dev/null +++ b/backend/windmill-trigger-amqp/src/lib.rs @@ -0,0 +1,341 @@ +use base64::engine; +use base64::prelude::*; +use lapin::{ + options::{BasicConsumeOptions, BasicQosOptions, QueueBindOptions, QueueDeclareOptions}, + types::FieldTable, + Channel, Connection, ConnectionProperties, Consumer, +}; +use serde::{Deserialize, Serialize}; +use serde_json::value::RawValue; +use sqlx::{types::Json as SqlxJson, FromRow}; +use std::collections::HashMap; +use windmill_common::{error::Error, triggers::TriggerKind, worker::to_raw_value}; + +use windmill_trigger::trigger_helpers::TriggerJobArgs; + +pub mod handler; +pub mod listener; + +#[derive(Clone, Copy)] +pub struct AmqpTrigger; + +impl TriggerJobArgs for AmqpTrigger { + type Payload = Vec; + const TRIGGER_KIND: TriggerKind = TriggerKind::Amqp; + + fn v1_payload_fn(payload: &Self::Payload) -> HashMap> { + HashMap::from([("payload".to_string(), to_raw_value(&payload))]) + } + + fn v2_payload_fn(payload: &Self::Payload) -> HashMap> { + let base64_payload = engine::general_purpose::STANDARD.encode(payload); + HashMap::from([("payload".to_string(), to_raw_value(&base64_payload))]) + } +} + +#[derive(Debug, Deserialize)] +pub struct AmqpResource { + pub host: String, + pub port: Option, + pub username: Option, + pub password: Option, + pub vhost: Option, + pub tls: Option, +} + +/// Binding of the consumed queue to an exchange. When present, the queue is bound +/// to `exchange_name` for each routing key so messages published to the exchange +/// are routed to it. +#[derive(Clone, Debug, Serialize, Deserialize)] +pub struct ExchangeConfig { + pub exchange_name: String, + #[serde(default)] + pub routing_keys: Vec, +} + +#[derive(Clone, Debug, Default, Serialize, Deserialize)] +pub struct AmqpOptions { + /// Declare the queue (durable) before consuming. When false the queue is + /// declared passively, i.e. it must already exist on the broker. + pub declare_queue: Option, + /// Maximum number of unacknowledged messages the broker delivers at once. + pub prefetch_count: Option, +} + +/// Shared validation for AMQP options used by both the CRUD handler and the +/// consumer builder (which also covers capture configs, that bypass CRUD +/// validation). RabbitMQ treats prefetch 0 as unlimited (unbounded consumer +/// buffer), so a set prefetch must be at least 1. +pub fn validate_amqp_options(options: Option<&AmqpOptions>) -> Result<(), String> { + if options.and_then(|o| o.prefetch_count) == Some(0) { + return Err("Prefetch count must be at least 1".to_string()); + } + Ok(()) +} + +#[derive(Debug, Clone, FromRow, Serialize, Deserialize)] +pub struct AmqpConfig { + pub amqp_resource_path: String, + pub queue_name: String, + pub exchange: Option>, + pub options: Option>, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AmqpConfigRequest { + pub amqp_resource_path: String, + pub queue_name: String, + pub exchange: Option, + pub options: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TestAmqpConfig { + pub amqp_resource_path: String, +} + +#[derive(Debug, thiserror::Error)] +pub enum AmqpError { + #[error("{0}")] + Common(#[from] Error), + #[error("{0}")] + Lapin(#[from] lapin::Error), +} + +/// Consumer bundle. The connection and channel are kept alive for as long as the +/// consumer stream is polled: dropping them would tear down the AMQP consumer. +pub struct AmqpConsumer { + pub connection: Connection, + pub channel: Channel, + pub consumer: Consumer, + pub queue_name: String, +} + +pub const CONSUMER_TAG: &str = "windmill"; + +fn build_uri(resource: &AmqpResource) -> String { + let tls = resource.tls.unwrap_or(false); + let scheme = if tls { "amqps" } else { "amqp" }; + let port = resource.port.unwrap_or(if tls { 5671 } else { 5672 }); + + let credentials = match (resource.username.as_deref(), resource.password.as_deref()) { + (Some(user), password) if !user.is_empty() => format!( + "{}:{}@", + urlencoding::encode(user), + urlencoding::encode(password.unwrap_or("")), + ), + _ => String::new(), + }; + + // The URI path is the virtual host and must be percent-encoded; the default + // vhost "/" therefore becomes "%2F". An empty vhost also falls back to "/". + let vhost = match resource.vhost.as_deref() { + Some(v) if !v.is_empty() => v, + _ => "/", + }; + let vhost_encoded = urlencoding::encode(vhost); + + // Bracket an IPv6 literal host so `host:port` parses correctly. + let host = if resource.host.contains(':') && !resource.host.starts_with('[') { + format!("[{}]", resource.host) + } else { + resource.host.clone() + }; + + format!( + "{}://{}{}:{}/{}", + scheme, credentials, host, port, vhost_encoded + ) +} + +fn connection_properties() -> ConnectionProperties { + ConnectionProperties::default() + .with_executor(tokio_executor_trait::Tokio::current()) + .with_reactor(tokio_reactor_trait::Tokio) +} + +pub struct AmqpClientBuilder<'client> { + resource: AmqpResource, + queue_name: &'client str, + exchange: Option<&'client ExchangeConfig>, + options: Option<&'client AmqpOptions>, +} + +impl<'client> AmqpClientBuilder<'client> { + pub fn new( + resource: AmqpResource, + queue_name: &'client str, + exchange: Option<&'client ExchangeConfig>, + options: Option<&'client AmqpOptions>, + ) -> Self { + Self { resource, queue_name, exchange, options } + } + + async fn connect(&self) -> Result { + let uri = build_uri(&self.resource); + let connection = Connection::connect(&uri, connection_properties()).await?; + Ok(connection) + } + + /// Establish a connection and open a channel to verify the broker is reachable + /// with the provided credentials. + pub async fn test_connection(&self) -> Result<(), AmqpError> { + let connection = self.connect().await?; + connection.create_channel().await?; + Ok(()) + } + + pub async fn build_consumer(&self) -> Result { + let connection = self.connect().await?; + let channel = connection.create_channel().await?; + + validate_amqp_options(self.options).map_err(|e| AmqpError::Common(Error::BadConfig(e)))?; + if let Some(prefetch_count) = self.options.and_then(|o| o.prefetch_count) { + channel + .basic_qos(prefetch_count, BasicQosOptions::default()) + .await?; + } + + let declare_queue = self.options.and_then(|o| o.declare_queue).unwrap_or(true); + + let queue_declare_options = QueueDeclareOptions { + passive: !declare_queue, + durable: declare_queue, + exclusive: false, + auto_delete: false, + nowait: false, + }; + + channel + .queue_declare( + self.queue_name, + queue_declare_options, + FieldTable::default(), + ) + .await?; + + if let Some(exchange) = self.exchange { + if !exchange.exchange_name.trim().is_empty() { + // Bind the queue for every routing key; an empty list binds once + // with an empty routing key (fanout exchanges ignore it anyway). + let routing_keys = if exchange.routing_keys.is_empty() { + vec![String::new()] + } else { + exchange.routing_keys.clone() + }; + for routing_key in routing_keys { + channel + .queue_bind( + self.queue_name, + &exchange.exchange_name, + &routing_key, + QueueBindOptions::default(), + FieldTable::default(), + ) + .await?; + } + } + } + + let consumer = channel + .basic_consume( + self.queue_name, + CONSUMER_TAG, + BasicConsumeOptions::default(), + FieldTable::default(), + ) + .await?; + + Ok(AmqpConsumer { connection, channel, consumer, queue_name: self.queue_name.to_string() }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn resource( + host: &str, + port: Option, + username: Option<&str>, + password: Option<&str>, + vhost: Option<&str>, + tls: Option, + ) -> AmqpResource { + AmqpResource { + host: host.to_string(), + port, + username: username.map(str::to_string), + password: password.map(str::to_string), + vhost: vhost.map(str::to_string), + tls, + } + } + + #[test] + fn build_uri_defaults_encode_root_vhost_and_pick_plaintext_port() { + let uri = build_uri(&resource("broker", None, None, None, None, None)); + assert_eq!(uri, "amqp://broker:5672/%2F"); + } + + #[test] + fn build_uri_tls_picks_amqps_and_5671() { + let uri = build_uri(&resource("broker", None, None, None, None, Some(true))); + assert_eq!(uri, "amqps://broker:5671/%2F"); + } + + #[test] + fn build_uri_encodes_credentials_and_custom_vhost() { + let uri = build_uri(&resource( + "broker", + Some(5673), + Some("us er"), + Some("p@ss/word"), + Some("my/vhost"), + None, + )); + assert_eq!(uri, "amqp://us%20er:p%40ss%2Fword@broker:5673/my%2Fvhost"); + } + + #[test] + fn build_uri_omits_credentials_when_username_empty() { + let uri = build_uri(&resource( + "broker", + None, + Some(""), + Some("secret"), + None, + None, + )); + assert_eq!(uri, "amqp://broker:5672/%2F"); + } + + #[test] + fn build_uri_blank_vhost_falls_back_to_root() { + let uri = build_uri(&resource("broker", None, None, None, Some(""), None)); + assert_eq!(uri, "amqp://broker:5672/%2F"); + } + + #[test] + fn build_uri_brackets_ipv6_host() { + let uri = build_uri(&resource("::1", Some(5672), None, None, None, None)); + assert_eq!(uri, "amqp://[::1]:5672/%2F"); + } + + fn options(prefetch: Option) -> AmqpOptions { + AmqpOptions { declare_queue: None, prefetch_count: prefetch } + } + + #[test] + fn validate_amqp_options_rejects_zero_prefetch() { + assert!(validate_amqp_options(Some(&options(Some(0)))).is_err()); + } + + #[test] + fn validate_amqp_options_accepts_valid_prefetch_and_none() { + assert!(validate_amqp_options(Some(&options(Some(1)))).is_ok()); + assert!(validate_amqp_options(Some(&options(Some(65535)))).is_ok()); + assert!(validate_amqp_options(Some(&options(None))).is_ok()); + assert!(validate_amqp_options(None).is_ok()); + } +} diff --git a/backend/windmill-trigger-amqp/src/listener.rs b/backend/windmill-trigger-amqp/src/listener.rs new file mode 100644 index 0000000000..347eb94b28 --- /dev/null +++ b/backend/windmill-trigger-amqp/src/listener.rs @@ -0,0 +1,255 @@ +use std::{collections::HashMap, sync::Arc, time::Duration}; + +use async_trait::async_trait; +use futures::StreamExt; +use lapin::options::{BasicAckOptions, BasicNackOptions}; +use tokio::sync::RwLock; +use windmill_common::{ + db::UserDB, + error::{Error, Result}, + jobs::JobTriggerKind, + utils::{report_critical_error, report_recovered_critical_error}, + worker::to_raw_value, + DB, +}; + +use windmill_store::resources::try_get_resource_from_db_as; +use windmill_trigger::listener::ListeningTrigger; +use windmill_trigger::trigger_helpers::TriggerJobArgs; +use windmill_trigger::Listener; + +use super::{AmqpClientBuilder, AmqpConfig, AmqpConsumer, AmqpResource, AmqpTrigger}; + +// lapin (like rdkafka) has no transparent reconnect, so — mirroring the Kafka +// trigger — the connection is (re)established in `consume` with a backoff retry +// loop rather than disabling the trigger on a transient broker outage. +const RECONNECT_BACKOFF_SECS: u64 = 30; +// Back off after a failed dispatch so a poison message that always fails can't +// spin a tight redelivery loop; the connection stays up for other messages. +const DISPATCH_FAILURE_BACKOFF_SECS: u64 = 5; + +impl AmqpTrigger { + async fn build_amqp_consumer( + &self, + db: &DB, + listening_trigger: &ListeningTrigger, + ) -> Result { + let AmqpConfig { amqp_resource_path, queue_name, exchange, options } = + &listening_trigger.trigger_config; + + let authed = listening_trigger + .authed(db, &Self::TRIGGER_KIND.to_string()) + .await?; + + let amqp_resource = try_get_resource_from_db_as::( + &authed, + Some(UserDB::new(db.clone())), + db, + amqp_resource_path, + &listening_trigger.workspace_id, + ) + .await?; + + let client_builder = AmqpClientBuilder::new( + amqp_resource, + queue_name, + exchange.as_ref().map(|e| &e.0), + options.as_ref().map(|o| &o.0), + ); + + client_builder + .build_consumer() + .await + .map_err(|e| Error::BadConfig(format!("Failed to build AMQP consumer: {}", e))) + } +} + +#[async_trait] +impl Listener for AmqpTrigger { + type Consumer = (); + type Extra = (); + type ExtraState = (); + const JOB_TRIGGER_KIND: JobTriggerKind = JobTriggerKind::Amqp; + + async fn get_consumer( + &self, + _db: &DB, + _listening_trigger: &ListeningTrigger, + _err_message: Arc>>, + _killpill_rx: tokio::sync::broadcast::Receiver<()>, + ) -> Result> { + // The connection is established (and re-established) in `consume` so that + // a transient broker outage retries with backoff instead of disabling the + // trigger — mirroring the Kafka trigger, whose client also lacks a + // transparent reconnect. + Ok(Some(())) + } + + async fn consume( + &self, + db: &DB, + _consumer: Self::Consumer, + listening_trigger: &ListeningTrigger, + err_message: Arc>>, + _killpill_rx: tokio::sync::broadcast::Receiver<()>, + _extra_state: Option<&Self::ExtraState>, + ) { + let path = &listening_trigger.path; + let workspace_id = &listening_trigger.workspace_id; + let alert_id = format!("amqp_trigger:{}", path); + let mut tries = 0_usize; + + // (Re)connect loop: retries forever with backoff; the framework's + // `select!` around `consume` cancels it on killpill, and + // `update_ping_and_loop_ping_status` returning None (trigger removed / + // disabled / capture stopped) breaks us out. + loop { + let mut consumer = match self.build_amqp_consumer(db, listening_trigger).await { + Ok(consumer) => consumer, + Err(e) => { + let status = format!( + "Failed to connect (attempt {}), retrying in {}s: {}", + tries + 1, + RECONNECT_BACKOFF_SECS, + e + ); + if self + .update_ping_and_loop_ping_status( + db, + listening_trigger, + err_message.clone(), + Some(status), + ) + .await + .is_none() + { + return; + } + tracing::error!( + "AMQP trigger {} failed to connect (attempt {}): {}", + path, + tries + 1, + e + ); + if tries % 10 == 0 && listening_trigger.trigger_mode { + report_critical_error( + format!( + "Failed to connect AMQP trigger {} (attempt {}), retrying every {}s. This alert repeats every 10 failed attempts. Error: {}", + path, tries + 1, RECONNECT_BACKOFF_SECS, e + ), + db.clone(), + Some(workspace_id), + Some(&alert_id), + ) + .await; + } + tries += 1; + tokio::time::sleep(Duration::from_secs(RECONNECT_BACKOFF_SECS)).await; + continue; + } + }; + + // Connected: clear any "reconnecting" status. + if self + .update_ping_and_loop_ping_status(db, listening_trigger, err_message.clone(), None) + .await + .is_none() + { + return; + } + if tries > 0 { + tracing::info!("AMQP trigger {} reconnected after {} attempts", path, tries); + if listening_trigger.trigger_mode { + report_recovered_critical_error( + format!("AMQP trigger {} reconnected", path), + db.clone(), + Some(workspace_id), + Some(&alert_id), + ) + .await; + } + tries = 0; + } + + // Consume until the stream errors, then break out to reconnect. + loop { + match consumer.consumer.next().await { + Some(Ok(delivery)) => { + let trigger_info = HashMap::from([ + ( + "exchange".to_string(), + to_raw_value(&delivery.exchange.as_str()), + ), + ( + "routing_key".to_string(), + to_raw_value(&delivery.routing_key.as_str()), + ), + ("queue_name".to_string(), to_raw_value(&consumer.queue_name)), + ( + "redelivered".to_string(), + to_raw_value(&delivery.redelivered), + ), + ( + "delivery_tag".to_string(), + to_raw_value(&delivery.delivery_tag), + ), + ]); + + let dispatched = self + .handle_event( + db, + listening_trigger, + delivery.data.clone(), + trigger_info, + None, + ) + .await; + + // Only ack once the job/capture was dispatched. On failure + // nack with requeue so the broker redelivers rather than + // dropping the message (at-least-once). + let dispatch_failed = dispatched.is_err(); + let ack_result = if dispatch_failed { + delivery + .acker + .nack(BasicNackOptions { requeue: true, multiple: false }) + .await + } else { + delivery.acker.ack(BasicAckOptions::default()).await + }; + + if let Err(err) = ack_result { + // Channel is gone; break out to reconnect. + tracing::warn!( + "AMQP trigger {} ack/nack failed, reconnecting: {}", + path, + err + ); + break; + } + + if dispatch_failed { + // The message was requeued: back off before consuming + // again so a poison message can't spin a tight + // redelivery loop, while keeping the connection alive. + tokio::time::sleep(Duration::from_secs(DISPATCH_FAILURE_BACKOFF_SECS)) + .await; + } + } + Some(Err(err)) => { + tracing::warn!( + "AMQP trigger {} consumer error, reconnecting: {}", + path, + err + ); + break; + } + None => { + tracing::warn!("AMQP trigger {} consumer stream ended, reconnecting", path); + break; + } + } + } + } + } +} diff --git a/backend/windmill-trigger/src/trigger_helpers.rs b/backend/windmill-trigger/src/trigger_helpers.rs index 1427f7f102..e0bfaf3d34 100644 --- a/backend/windmill-trigger/src/trigger_helpers.rs +++ b/backend/windmill-trigger/src/trigger_helpers.rs @@ -140,7 +140,7 @@ fn runnable_format_from_schema_without_preprocessor( schema: Option>, ) -> RunnableFormat { match trigger_kind { - TriggerKind::Mqtt + TriggerKind::Mqtt | TriggerKind::Amqp if schema.as_ref().is_some_and(|schema| { schema.properties.as_ref().is_some_and(|properties| { properties.iter().any(|(key, def)| { diff --git a/backend/windmill-types/src/jobs.rs b/backend/windmill-types/src/jobs.rs index 95ac69bc2c..79b33f86ac 100644 --- a/backend/windmill-types/src/jobs.rs +++ b/backend/windmill-types/src/jobs.rs @@ -32,6 +32,7 @@ pub enum JobTriggerKind { Email, Nats, Mqtt, + Amqp, Sqs, Postgres, Schedule, @@ -66,6 +67,7 @@ impl std::fmt::Display for JobTriggerKind { JobTriggerKind::Email => "email", JobTriggerKind::Nats => "nats", JobTriggerKind::Mqtt => "mqtt", + JobTriggerKind::Amqp => "amqp", JobTriggerKind::Sqs => "sqs", JobTriggerKind::Postgres => "postgres", JobTriggerKind::Schedule => "schedule", diff --git a/backend/windmill-types/src/triggers.rs b/backend/windmill-types/src/triggers.rs index 8ad1d5e8a7..e7c398b1f2 100644 --- a/backend/windmill-types/src/triggers.rs +++ b/backend/windmill-types/src/triggers.rs @@ -16,6 +16,7 @@ pub enum TriggerKind { Email, Nats, Mqtt, + Amqp, Sqs, Postgres, Gcp, @@ -36,6 +37,7 @@ impl TriggerKind { TriggerKind::DefaultEmail => "email".to_string(), TriggerKind::Nats => "nats".to_string(), TriggerKind::Mqtt => "mqtt".to_string(), + TriggerKind::Amqp => "amqp".to_string(), TriggerKind::Sqs => "sqs".to_string(), TriggerKind::Postgres => "postgres".to_string(), TriggerKind::Gcp => "gcp".to_string(), @@ -58,6 +60,7 @@ impl fmt::Display for TriggerKind { TriggerKind::DefaultEmail => "default_email", TriggerKind::Nats => "nats", TriggerKind::Mqtt => "mqtt", + TriggerKind::Amqp => "amqp", TriggerKind::Sqs => "sqs", TriggerKind::Postgres => "postgres", TriggerKind::Gcp => "gcp", diff --git a/cli/src/commands/pipeline/boundedCascade.ts b/cli/src/commands/pipeline/boundedCascade.ts index 69166726c5..57130ba129 100644 --- a/cli/src/commands/pipeline/boundedCascade.ts +++ b/cli/src/commands/pipeline/boundedCascade.ts @@ -71,6 +71,7 @@ const assetNodeId = (kind: string, path: string): string => `${kind}:${path}`; const NON_AUTORUN_TRIGGER_KINDS = new Set([ "kafka", "mqtt", + "amqp", "nats", "postgres", "sqs", diff --git a/cli/src/commands/pipeline/localGraph.ts b/cli/src/commands/pipeline/localGraph.ts index eb3350c818..be5c8f1567 100644 --- a/cli/src/commands/pipeline/localGraph.ts +++ b/cli/src/commands/pipeline/localGraph.ts @@ -259,6 +259,7 @@ const NATIVE_KINDS = new Set([ "email", "kafka", "mqtt", + "amqp", "nats", "postgres", "sqs", diff --git a/cli/src/commands/sync/sync.ts b/cli/src/commands/sync/sync.ts index 7a78623870..29d83862e4 100644 --- a/cli/src/commands/sync/sync.ts +++ b/cli/src/commands/sync/sync.ts @@ -1717,6 +1717,7 @@ export async function elementsToMap( path.endsWith(".nats_trigger" + ext) || path.endsWith(".postgres_trigger" + ext) || path.endsWith(".mqtt_trigger" + ext) || + path.endsWith(".amqp_trigger" + ext) || path.endsWith(".sqs_trigger" + ext) || path.endsWith(".gcp_trigger" + ext) || path.endsWith(".azure_trigger" + ext) || @@ -2470,6 +2471,7 @@ function getOrderFromPath(p: string) { typ == "nats_trigger" || typ == "postgres_trigger" || typ == "mqtt_trigger" || + typ == "amqp_trigger" || typ == "sqs_trigger" || typ == "gcp_trigger" || typ == "azure_trigger" || @@ -4237,7 +4239,7 @@ export async function push( } } const rules = folderRulesCache.get(folderName)!; - const remotePath = change.path.replace(/\.(script|schedule|http_trigger|websocket_trigger|kafka_trigger|nats_trigger|postgres_trigger|mqtt_trigger|sqs_trigger|gcp_trigger|azure_trigger|email_trigger)\.(yaml|json)$/, "").replace(/(\.flow|__flow)\/flow\.(yaml|json)$/, "").replace(/\.(app|raw_app)(\/app\.(yaml|json))?$/, ""); + const remotePath = change.path.replace(/\.(script|schedule|http_trigger|websocket_trigger|kafka_trigger|nats_trigger|postgres_trigger|mqtt_trigger|amqp_trigger|sqs_trigger|gcp_trigger|azure_trigger|email_trigger)\.(yaml|json)$/, "").replace(/(\.flow|__flow)\/flow\.(yaml|json)$/, "").replace(/\.(app|raw_app)(\/app\.(yaml|json))?$/, ""); const relative = remotePath.slice(`f/${folderName}/`.length); if (!relative) continue; for (const rule of rules) { @@ -4931,6 +4933,12 @@ export async function push( path: removeSuffix(target, ".mqtt_trigger.json"), }); break; + case "amqp_trigger": + await wmill.deleteAmqpTrigger({ + workspace: workspaceId, + path: removeSuffix(target, ".amqp_trigger.json"), + }); + break; case "sqs_trigger": await wmill.deleteSqsTrigger({ workspace: workspaceId, diff --git a/cli/src/commands/trigger/trigger.ts b/cli/src/commands/trigger/trigger.ts index 1cadc8d9d2..83b62f9eb4 100644 --- a/cli/src/commands/trigger/trigger.ts +++ b/cli/src/commands/trigger/trigger.ts @@ -9,6 +9,7 @@ import { HttpTrigger, KafkaTrigger, MqttTrigger, + AmqpTrigger, NatsTrigger, PostgresTrigger, SqsTrigger, @@ -48,6 +49,7 @@ type Trigger = { nats: NatsTrigger; postgres: PostgresTrigger; mqtt: MqttTrigger; + amqp: AmqpTrigger; sqs: SqsTrigger; gcp: GcpTrigger; azure: AzureTrigger; @@ -84,6 +86,7 @@ async function getTrigger( nats: wmill.getNatsTrigger, postgres: wmill.getPostgresTrigger, mqtt: wmill.getMqttTrigger, + amqp: wmill.getAmqpTrigger, sqs: wmill.getSqsTrigger, gcp: wmill.getGcpTrigger, azure: wmill.getAzureTrigger, @@ -114,6 +117,7 @@ async function updateTrigger( nats: wmill.updateNatsTrigger, postgres: wmill.updatePostgresTrigger, mqtt: wmill.updateMqttTrigger, + amqp: wmill.updateAmqpTrigger, sqs: wmill.updateSqsTrigger, gcp: wmill.updateGcpTrigger, azure: wmill.updateAzureTrigger, @@ -142,6 +146,7 @@ async function createTrigger( nats: wmill.createNatsTrigger, postgres: wmill.createPostgresTrigger, mqtt: wmill.createMqttTrigger, + amqp: wmill.createAmqpTrigger, sqs: wmill.createSqsTrigger, gcp: wmill.createGcpTrigger, azure: wmill.createAzureTrigger, @@ -381,6 +386,13 @@ const triggerTemplates: Record> = { subscribe_topics: [], enabled: false, }, + amqp: { + script_path: "", + is_flow: false, + amqp_resource_path: "", + queue_name: "", + enabled: false, + }, sqs: { script_path: "", is_flow: false, @@ -536,6 +548,7 @@ async function list(opts: GlobalOptions & { json?: boolean }) { natsTriggers, postgresTriggers, mqttTriggers, + amqpTriggers, sqsTriggers, gcpTriggers, azureTriggers, @@ -547,6 +560,7 @@ async function list(opts: GlobalOptions & { json?: boolean }) { listOrEmpty(() => wmill.listNatsTriggers({ workspace: ws })), listOrEmpty(() => wmill.listPostgresTriggers({ workspace: ws })), listOrEmpty(() => wmill.listMqttTriggers({ workspace: ws })), + listOrEmpty(() => wmill.listAmqpTriggers({ workspace: ws })), listOrEmpty(() => wmill.listSqsTriggers({ workspace: ws })), listOrEmpty(() => wmill.listGcpTriggers({ workspace: ws })), listOrEmpty(() => wmill.listAzureTriggers({ workspace: ws })), @@ -559,6 +573,7 @@ async function list(opts: GlobalOptions & { json?: boolean }) { ...natsTriggers.map((x) => ({ path: x.path, kind: "nats" })), ...postgresTriggers.map((x) => ({ path: x.path, kind: "postgres" })), ...mqttTriggers.map((x) => ({ path: x.path, kind: "mqtt" })), + ...amqpTriggers.map((x) => ({ path: x.path, kind: "amqp" })), ...sqsTriggers.map((x) => ({ path: x.path, kind: "sqs" })), ...gcpTriggers.map((x) => ({ path: x.path, kind: "gcp" })), ...azureTriggers.map((x) => ({ path: x.path, kind: "azure" })), @@ -643,11 +658,11 @@ const command = new Command() .command("get", "get a trigger's details") .arguments("") .option("--json", "Output as JSON (for piping to jq)") - .option("--kind ", "Trigger kind (http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email). Recommended for faster lookup") + .option("--kind ", "Trigger kind (http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email). Recommended for faster lookup") .action(get as any) .command("new", "create a new trigger locally") .arguments("") - .option("--kind ", "Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email)") + .option("--kind ", "Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email)") .action(newTrigger as any) .command( "push", @@ -662,7 +677,7 @@ const command = new Command() .arguments(" ") .option( "--kind ", - "Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email)" + "Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email)" ) .action((async (opts: any, triggerPath: string, email: string) => { const workspace = await resolveWorkspace(opts); diff --git a/cli/src/guidance/core.ts b/cli/src/guidance/core.ts index ae8a3f20ee..c43ff8c6c0 100644 --- a/cli/src/guidance/core.ts +++ b/cli/src/guidance/core.ts @@ -89,7 +89,7 @@ When a new app needs to be created, YOU run \`wmill app new\` yourself with \`-- ## Triggers -You MUST use the \`triggers\` skill to configure HTTP routes, WebSocket, Kafka, NATS, SQS, MQTT, GCP, Azure, Email, or Postgres CDC triggers. +You MUST use the \`triggers\` skill to configure HTTP routes, WebSocket, Kafka, NATS, SQS, MQTT, AMQP, GCP, Azure, Email, or Postgres CDC triggers. ## Schedules diff --git a/cli/src/guidance/skills.gen.ts b/cli/src/guidance/skills.gen.ts index bdeeda6bf3..69e4338d22 100644 --- a/cli/src/guidance/skills.gen.ts +++ b/cli/src/guidance/skills.gen.ts @@ -7232,12 +7232,12 @@ trigger related commands - \`--json\` - Output as JSON (for piping to jq) - \`trigger get \` - get a trigger's details - \`--json\` - Output as JSON (for piping to jq) - - \`--kind \` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email). Recommended for faster lookup + - \`--kind \` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email). Recommended for faster lookup - \`trigger new \` - create a new trigger locally - - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) - \`trigger push \` - push a local trigger spec. This overrides any remote versions. - \`trigger set-permissioned-as \` - Set the email (run-as user) for a trigger (requires admin or wm_deployers group) - - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) ### user @@ -7509,6 +7509,120 @@ Both print the job result, are safe to run yourself, and don't deploy. // YAML schema content for triggers and schedules export const SCHEMAS: Record = { + "amqp_trigger": `type: object +properties: + script_path: + type: string + description: Path to the script or flow to execute when triggered + permissioned_as: + type: string + description: The user or group this trigger runs as (permissioned_as) + is_flow: + type: boolean + description: True if script_path points to a flow, false if it points to a script + labels: + type: array + items: + type: string + draft_only: + type: boolean + description: 'True when this row is a per-user draft with no deployed + + trigger at the same path. Set by list endpoints when + + \`include_draft_only=true\` synthesizes the row from the + + draft. Frontend renders a "Draft" badge. + + ' + is_draft: + type: boolean + description: 'True when the authed user has a per-user draft at this path + + (over a deployed row or a synthesized draft-only row). + + Frontend appends a \`*\` to the displayed name. + + ' + amqp_resource_path: + type: string + description: Path to the AMQP resource containing broker connection configuration + queue_name: + type: string + description: Name of the queue to consume messages from + exchange: + type: object + properties: + exchange_name: + type: string + description: Name of the exchange to bind the consumed queue to + routing_keys: + type: array + items: + type: string + description: Routing keys used to bind the queue to the exchange + options: + type: object + properties: + declare_queue: + type: boolean + description: Declare the queue (durable) before consuming; when false the + queue is declared passively and must already exist + prefetch_count: + type: integer + format: int32 + minimum: 1 + maximum: 65535 + description: Maximum number of unacknowledged messages the broker delivers + at once (1-65535) + error_handler_path: + type: string + description: Path to a script or flow to run when the triggered job fails + error_handler_args: + type: object + description: The arguments to pass to the script or flow + retry: + type: object + properties: + constant: + type: object + description: Retry with constant delay between attempts + properties: + attempts: + type: integer + description: Number of retry attempts + seconds: + type: integer + description: Seconds to wait between retries + exponential: + type: object + description: Retry with exponential backoff (delay doubles each time) + properties: + attempts: + type: integer + description: Number of retry attempts + multiplier: + type: integer + description: Multiplier for exponential backoff + seconds: + type: integer + minimum: 1 + description: Initial delay in seconds + random_factor: + type: integer + minimum: 0 + maximum: 100 + description: Random jitter percentage (0-100) to avoid thundering herd + retry_if: + $ref: '#/components/schemas/RetryIf' + description: Retry configuration for failed module executions +required: +- script_path +- permissioned_as +- is_flow +- amqp_resource_path +- queue_name +`, "azure_trigger": `type: object properties: script_path: @@ -8815,6 +8929,7 @@ export const SCHEMA_MAPPINGS: Record = { { name: "NatsTrigger", schemaKey: "nats_trigger", filePattern: "*.nats_trigger.yaml" }, { name: "PostgresTrigger", schemaKey: "postgres_trigger", filePattern: "*.postgres_trigger.yaml" }, { name: "MqttTrigger", schemaKey: "mqtt_trigger", filePattern: "*.mqtt_trigger.yaml" }, + { name: "AmqpTrigger", schemaKey: "amqp_trigger", filePattern: "*.amqp_trigger.yaml" }, { name: "SqsTrigger", schemaKey: "sqs_trigger", filePattern: "*.sqs_trigger.yaml" }, { name: "GcpTrigger", schemaKey: "gcp_trigger", filePattern: "*.gcp_trigger.yaml" }, { name: "AzureTrigger", schemaKey: "azure_trigger", filePattern: "*.azure_trigger.yaml" }, diff --git a/cli/src/types.ts b/cli/src/types.ts index c125ae2c2e..f0cd58abbb 100644 --- a/cli/src/types.ts +++ b/cli/src/types.ts @@ -63,6 +63,7 @@ export const TRIGGER_TYPES = [ "nats", "postgres", "mqtt", + "amqp", "sqs", "gcp", "azure", @@ -243,6 +244,8 @@ export async function pushObj( await pushTrigger("postgres", workspace, p, befObj, newObj, permissionedAsContext); } else if (typeEnding === "mqtt_trigger") { await pushTrigger("mqtt", workspace, p, befObj, newObj, permissionedAsContext); + } else if (typeEnding === "amqp_trigger") { + await pushTrigger("amqp", workspace, p, befObj, newObj, permissionedAsContext); } else if (typeEnding === "sqs_trigger") { await pushTrigger("sqs", workspace, p, befObj, newObj, permissionedAsContext); } else if (typeEnding === "gcp_trigger") { @@ -335,6 +338,7 @@ export function getTypeStrFromPath( | "nats_trigger" | "postgres_trigger" | "mqtt_trigger" + | "amqp_trigger" | "sqs_trigger" | "gcp_trigger" | "azure_trigger" @@ -419,6 +423,7 @@ export function getTypeStrFromPath( typeEnding === "nats_trigger" || typeEnding === "postgres_trigger" || typeEnding === "mqtt_trigger" || + typeEnding === "amqp_trigger" || typeEnding === "sqs_trigger" || typeEnding === "gcp_trigger" || typeEnding === "azure_trigger" || diff --git a/cli/src/utils/git.ts b/cli/src/utils/git.ts index 342222ed1b..441133afe4 100644 --- a/cli/src/utils/git.ts +++ b/cli/src/utils/git.ts @@ -285,6 +285,8 @@ export function gitSyncIncludePattern( return `${path}.postgres_trigger.*`; case "mqtttrigger": return `${path}.mqtt_trigger.*`; + case "amqptrigger": + return `${path}.amqp_trigger.*`; case "sqstrigger": return `${path}.sqs_trigger.*`; case "gcptrigger": diff --git a/cli/test/git_unit.test.ts b/cli/test/git_unit.test.ts index 4b6cf464e2..140da33866 100644 --- a/cli/test/git_unit.test.ts +++ b/cli/test/git_unit.test.ts @@ -274,6 +274,9 @@ describe("gitSyncIncludePattern", () => { expect(gitSyncIncludePattern("gcptrigger", "f/t")).toBe( "f/t.gcp_trigger.*" ); + expect(gitSyncIncludePattern("amqptrigger", "f/t")).toBe( + "f/t.amqp_trigger.*" + ); }); }); diff --git a/frontend/src/lib/components/CompareDrafts.svelte b/frontend/src/lib/components/CompareDrafts.svelte index b7970c90e1..32be6caf35 100644 --- a/frontend/src/lib/components/CompareDrafts.svelte +++ b/frontend/src/lib/components/CompareDrafts.svelte @@ -118,6 +118,7 @@ 'nats_trigger', 'postgres_trigger', 'mqtt_trigger', + 'amqp_trigger', 'sqs_trigger', 'gcp_trigger', 'azure_trigger', @@ -483,6 +484,7 @@ trigger_kafka: '/kafka_triggers', trigger_nats: '/nats_triggers', trigger_mqtt: '/mqtt_triggers', + trigger_amqp: '/amqp_triggers', trigger_sqs: '/sqs_triggers', trigger_gcp: '/gcp_triggers', trigger_azure: '/azure_triggers', diff --git a/frontend/src/lib/components/CompareWorkspaces.svelte b/frontend/src/lib/components/CompareWorkspaces.svelte index a2453add27..ac4fcfad99 100644 --- a/frontend/src/lib/components/CompareWorkspaces.svelte +++ b/frontend/src/lib/components/CompareWorkspaces.svelte @@ -1001,6 +1001,7 @@ nats_trigger: 'NATS trigger', postgres_trigger: 'Postgres trigger', mqtt_trigger: 'MQTT trigger', + amqp_trigger: 'AMQP trigger', sqs_trigger: 'SQS trigger', gcp_trigger: 'GCP trigger', azure_trigger: 'Azure trigger', diff --git a/frontend/src/lib/components/MoveDrawer.svelte b/frontend/src/lib/components/MoveDrawer.svelte index 212e45d58d..21b155fce3 100644 --- a/frontend/src/lib/components/MoveDrawer.svelte +++ b/frontend/src/lib/components/MoveDrawer.svelte @@ -48,6 +48,7 @@ push('schedule', c.schedule_count) push('kafka', c.kafka_count) push('mqtt', c.mqtt_count) + push('amqp', c.amqp_count) push('nats', c.nats_count) push('postgres', c.postgres_count) push('sqs', c.sqs_count) diff --git a/frontend/src/lib/components/Path.svelte b/frontend/src/lib/components/Path.svelte index 27d6b45e6c..284677ce7c 100644 --- a/frontend/src/lib/components/Path.svelte +++ b/frontend/src/lib/components/Path.svelte @@ -20,6 +20,7 @@ PostgresTriggerService, NatsTriggerService, MqttTriggerService, + AmqpTriggerService, SqsTriggerService, GcpTriggerService, AzureTriggerService, @@ -58,6 +59,7 @@ | 'postgres_trigger' | 'nats_trigger' | 'mqtt_trigger' + | 'amqp_trigger' | 'sqs_trigger' | 'gcp_trigger' | 'azure_trigger' @@ -302,6 +304,11 @@ workspace: ws!, path: path }) + } else if (kind === 'amqp_trigger') { + return await AmqpTriggerService.existsAmqpTrigger({ + workspace: ws!, + path: path + }) } else if (kind == 'sqs_trigger') { return await SqsTriggerService.existsSqsTrigger({ workspace: ws!, diff --git a/frontend/src/lib/components/assets/AssetGraph/AddNode.svelte b/frontend/src/lib/components/assets/AssetGraph/AddNode.svelte index 4745742e20..880abb685a 100644 --- a/frontend/src/lib/components/assets/AssetGraph/AddNode.svelte +++ b/frontend/src/lib/components/assets/AssetGraph/AddNode.svelte @@ -94,6 +94,12 @@ description: 'Triggered by an MQTT message', icon: Radio }, + { + id: 'amqp', + label: 'On AMQP', + description: 'Triggered by an AMQP (RabbitMQ) message', + icon: Radio + }, { id: 'nats', label: 'On NATS', diff --git a/frontend/src/lib/components/assets/AssetGraph/PipelineTriggerEditors.svelte b/frontend/src/lib/components/assets/AssetGraph/PipelineTriggerEditors.svelte index 0a32929720..e364b29883 100644 --- a/frontend/src/lib/components/assets/AssetGraph/PipelineTriggerEditors.svelte +++ b/frontend/src/lib/components/assets/AssetGraph/PipelineTriggerEditors.svelte @@ -8,6 +8,7 @@ GcpTriggerService, KafkaTriggerService, MqttTriggerService, + AmqpTriggerService, NatsTriggerService, PostgresTriggerService, ScheduleService, @@ -15,6 +16,7 @@ } from '$lib/gen' import KafkaTriggerEditor from '$lib/components/triggers/kafka/KafkaTriggerEditor.svelte' import MqttTriggerEditor from '$lib/components/triggers/mqtt/MqttTriggerEditor.svelte' + import AmqpTriggerEditor from '$lib/components/triggers/amqp/AmqpTriggerEditor.svelte' import NatsTriggerEditor from '$lib/components/triggers/nats/NatsTriggerEditor.svelte' import PostgresTriggerEditor from '$lib/components/triggers/postgres/PostgresTriggerEditor.svelte' import SqsTriggerEditor from '$lib/components/triggers/sqs/SqsTriggerEditor.svelte' @@ -44,6 +46,7 @@ let kafkaEditor: KafkaTriggerEditor | undefined = $state() let mqttEditor: MqttTriggerEditor | undefined = $state() + let amqpEditor: AmqpTriggerEditor | undefined = $state() let natsEditor: NatsTriggerEditor | undefined = $state() let postgresEditor: PostgresTriggerEditor | undefined = $state() let sqsEditor: SqsTriggerEditor | undefined = $state() @@ -63,6 +66,8 @@ return kafkaEditor?.openNew(false, scriptPath) case 'mqtt': return mqttEditor?.openNew(false, scriptPath) + case 'amqp': + return amqpEditor?.openNew(false, scriptPath) case 'nats': return natsEditor?.openNew(false, scriptPath) case 'postgres': @@ -87,6 +92,8 @@ return kafkaEditor?.openEdit(triggerPath, false, scriptPath) case 'mqtt': return mqttEditor?.openEdit(triggerPath, false, scriptPath) + case 'amqp': + return amqpEditor?.openEdit(triggerPath, false, scriptPath) case 'nats': return natsEditor?.openEdit(triggerPath, false, scriptPath) case 'postgres': @@ -135,6 +142,9 @@ case 'mqtt': await MqttTriggerService.deleteMqttTrigger({ workspace, path: triggerPath }) break + case 'amqp': + await AmqpTriggerService.deleteAmqpTrigger({ workspace, path: triggerPath }) + break case 'nats': await NatsTriggerService.deleteNatsTrigger({ workspace, path: triggerPath }) break @@ -195,6 +205,7 @@ gated off the canvas outside edit mode. --> + diff --git a/frontend/src/lib/components/assets/AssetGraph/TriggerNode.svelte b/frontend/src/lib/components/assets/AssetGraph/TriggerNode.svelte index 98307013c4..0e279112cd 100644 --- a/frontend/src/lib/components/assets/AssetGraph/TriggerNode.svelte +++ b/frontend/src/lib/components/assets/AssetGraph/TriggerNode.svelte @@ -48,6 +48,7 @@ email: { icon: Mail, label: 'email', ...MUTED }, kafka: { icon: Zap, label: 'kafka', ...MUTED }, mqtt: { icon: Radio, label: 'mqtt', ...MUTED }, + amqp: { icon: Radio, label: 'amqp', ...MUTED }, nats: { icon: MessageSquare, label: 'nats', ...MUTED }, postgres: { icon: Database, label: 'postgres', ...MUTED }, sqs: { icon: Send, label: 'sqs', ...MUTED }, diff --git a/frontend/src/lib/components/assets/AssetGraph/boundedCascade.ts b/frontend/src/lib/components/assets/AssetGraph/boundedCascade.ts index 6fcabbe71f..810ab36cc1 100644 --- a/frontend/src/lib/components/assets/AssetGraph/boundedCascade.ts +++ b/frontend/src/lib/components/assets/AssetGraph/boundedCascade.ts @@ -51,6 +51,7 @@ export function assetUriToNodeId(uri: string): string | undefined { const EVENT_TRIGGER_KINDS: ReadonlySet = new Set([ 'kafka', 'mqtt', + 'amqp', 'nats', 'postgres', 'sqs', diff --git a/frontend/src/lib/components/assets/AssetGraph/parsePipelineAnnotations.ts b/frontend/src/lib/components/assets/AssetGraph/parsePipelineAnnotations.ts index f9e53494ff..34b9ac2eab 100644 --- a/frontend/src/lib/components/assets/AssetGraph/parsePipelineAnnotations.ts +++ b/frontend/src/lib/components/assets/AssetGraph/parsePipelineAnnotations.ts @@ -31,6 +31,7 @@ const NATIVE_TRIGGER_KEYWORDS: NativeTriggerKind[] = [ 'email', 'kafka', 'mqtt', + 'amqp', 'nats', 'postgres', 'sqs', diff --git a/frontend/src/lib/components/assets/AssetGraph/pipelineTemplates.ts b/frontend/src/lib/components/assets/AssetGraph/pipelineTemplates.ts index 02607d5f35..bf0c14d16e 100644 --- a/frontend/src/lib/components/assets/AssetGraph/pipelineTemplates.ts +++ b/frontend/src/lib/components/assets/AssetGraph/pipelineTemplates.ts @@ -309,6 +309,7 @@ export type DraftTriggerSource = | 'email' | 'kafka' | 'mqtt' + | 'amqp' | 'nats' | 'postgres' | 'sqs' diff --git a/frontend/src/lib/components/assets/AssetGraph/types.ts b/frontend/src/lib/components/assets/AssetGraph/types.ts index 0e055abca2..d9ef2dafbf 100644 --- a/frontend/src/lib/components/assets/AssetGraph/types.ts +++ b/frontend/src/lib/components/assets/AssetGraph/types.ts @@ -110,6 +110,7 @@ export type NativeTriggerKind = | 'email' | 'kafka' | 'mqtt' + | 'amqp' | 'nats' | 'postgres' | 'sqs' diff --git a/frontend/src/lib/components/common/table/Row.svelte b/frontend/src/lib/components/common/table/Row.svelte index d2d15ecfd5..df0e4ce023 100644 --- a/frontend/src/lib/components/common/table/Row.svelte +++ b/frontend/src/lib/components/common/table/Row.svelte @@ -45,6 +45,7 @@ | 'nats_trigger' | 'postgres_trigger' | 'mqtt_trigger' + | 'amqp_trigger' | 'sqs_trigger' | 'gcp_trigger' | 'azure_trigger' diff --git a/frontend/src/lib/components/common/table/RowIcon.svelte b/frontend/src/lib/components/common/table/RowIcon.svelte index b2deb47443..7ef3543073 100644 --- a/frontend/src/lib/components/common/table/RowIcon.svelte +++ b/frontend/src/lib/components/common/table/RowIcon.svelte @@ -3,6 +3,7 @@ import KafkaIcon from '$lib/components/icons/KafkaIcon.svelte' import NatsIcon from '$lib/components/icons/NatsIcon.svelte' import MqttIcon from '$lib/components/icons/MqttIcon.svelte' + import AmqpIcon from '$lib/components/icons/AmqpIcon.svelte' import AwsIcon from '$lib/components/icons/AwsIcon.svelte' import AzureIcon from '$lib/components/icons/AzureIcon.svelte' import GoogleCloudIcon from '$lib/components/icons/GoogleCloudIcon.svelte' @@ -41,6 +42,7 @@ | 'kafka' | 'nats' | 'mqtt' + | 'amqp' | 'sqs' | 'gcp' | 'emails' @@ -50,6 +52,7 @@ | 'nats_trigger' | 'postgres_trigger' | 'mqtt_trigger' + | 'amqp_trigger' | 'sqs_trigger' | 'gcp_trigger' | 'azure_trigger' @@ -75,6 +78,7 @@ nats_trigger: 'nats', postgres_trigger: 'postgres', mqtt_trigger: 'mqtt', + amqp_trigger: 'amqp', sqs_trigger: 'sqs', gcp_trigger: 'gcp', azure_trigger: 'azure', @@ -117,6 +121,8 @@ {:else if effectiveKind === 'mqtt'} + {:else if effectiveKind === 'amqp'} + {:else if effectiveKind === 'sqs'} {:else if effectiveKind === 'gcp'} diff --git a/frontend/src/lib/components/copilot/chat/CreatedResourceActionDrawers.svelte b/frontend/src/lib/components/copilot/chat/CreatedResourceActionDrawers.svelte index b73c8dc6bc..e25df2cc40 100644 --- a/frontend/src/lib/components/copilot/chat/CreatedResourceActionDrawers.svelte +++ b/frontend/src/lib/components/copilot/chat/CreatedResourceActionDrawers.svelte @@ -70,6 +70,10 @@ label: 'MQTT trigger', load: () => import('$lib/components/triggers/mqtt/MqttTriggerEditorInner.svelte') }, + amqp: { + label: 'AMQP trigger', + load: () => import('$lib/components/triggers/amqp/AmqpTriggerEditorInner.svelte') + }, sqs: { label: 'SQS trigger', load: () => import('$lib/components/triggers/sqs/SqsTriggerEditorInner.svelte') diff --git a/frontend/src/lib/components/copilot/chat/ToolMessageActions.svelte b/frontend/src/lib/components/copilot/chat/ToolMessageActions.svelte index a40f244019..58ebfe2bb8 100644 --- a/frontend/src/lib/components/copilot/chat/ToolMessageActions.svelte +++ b/frontend/src/lib/components/copilot/chat/ToolMessageActions.svelte @@ -26,6 +26,7 @@ import GoogleCloudIcon from '$lib/components/icons/GoogleCloudIcon.svelte' import KafkaIcon from '$lib/components/icons/KafkaIcon.svelte' import MqttIcon from '$lib/components/icons/MqttIcon.svelte' + import AmqpIcon from '$lib/components/icons/AmqpIcon.svelte' import NatsIcon from '$lib/components/icons/NatsIcon.svelte' import { runToolDisplayAction } from './createdResourceActions.svelte' import type { CreatedResourceTriggerKind, ToolDisplayAction } from './shared' @@ -54,6 +55,7 @@ kafka: { title: 'Kafka trigger', icon: KafkaIcon }, nats: { title: 'NATS trigger', icon: NatsIcon }, mqtt: { title: 'MQTT trigger', icon: MqttIcon }, + amqp: { title: 'AMQP trigger', icon: AmqpIcon }, sqs: { title: 'SQS trigger', icon: AwsIcon }, gcp: { title: 'GCP Pub/Sub trigger', icon: GoogleCloudIcon }, azure: { title: 'Azure Event Grid trigger', icon: AzureIcon }, diff --git a/frontend/src/lib/components/copilot/chat/global/core.ts b/frontend/src/lib/components/copilot/chat/global/core.ts index 86b95bfa89..c45f5b129b 100644 --- a/frontend/src/lib/components/copilot/chat/global/core.ts +++ b/frontend/src/lib/components/copilot/chat/global/core.ts @@ -8,6 +8,7 @@ import { JobService, KafkaTriggerService, MqttTriggerService, + AmqpTriggerService, NatsTriggerService, PostgresTriggerService, ResourceService, @@ -462,6 +463,7 @@ const writeTriggerSchema = z.object({ triggerRequestSchemas.nats, triggerRequestSchemas.postgres, triggerRequestSchemas.mqtt, + triggerRequestSchemas.amqp, triggerRequestSchemas.sqs, triggerRequestSchemas.gcp, triggerRequestSchemas.azure @@ -1553,6 +1555,14 @@ const triggerServices: Record = { update: (a) => MqttTriggerService.updateMqttTrigger(a), delete: (a) => MqttTriggerService.deleteMqttTrigger(a) }, + amqp: { + exists: (a) => AmqpTriggerService.existsAmqpTrigger(a), + get: (a) => AmqpTriggerService.getAmqpTrigger(a), + list: (a) => AmqpTriggerService.listAmqpTriggers(a), + create: (a) => AmqpTriggerService.createAmqpTrigger(a), + update: (a) => AmqpTriggerService.updateAmqpTrigger(a), + delete: (a) => AmqpTriggerService.deleteAmqpTrigger(a) + }, sqs: { exists: (a) => SqsTriggerService.existsSqsTrigger(a), get: (a) => SqsTriggerService.getSqsTrigger(a), @@ -4858,6 +4868,7 @@ const triggerLabels: Record = { nats: 'NATS trigger', postgres: 'Postgres trigger', mqtt: 'MQTT trigger', + amqp: 'AMQP trigger', sqs: 'SQS trigger', gcp: 'GCP Pub/Sub trigger', azure: 'Azure Event Grid trigger' diff --git a/frontend/src/lib/components/copilot/chat/global/userDraftAdapter.ts b/frontend/src/lib/components/copilot/chat/global/userDraftAdapter.ts index 32cf08d229..eae27fb5b4 100644 --- a/frontend/src/lib/components/copilot/chat/global/userDraftAdapter.ts +++ b/frontend/src/lib/components/copilot/chat/global/userDraftAdapter.ts @@ -21,6 +21,7 @@ const TRIGGER_DRAFT_KIND_BY_TRIGGER_KIND = { nats: 'trigger_nats', postgres: 'trigger_postgres', mqtt: 'trigger_mqtt', + amqp: 'trigger_amqp', sqs: 'trigger_sqs', gcp: 'trigger_gcp', azure: 'trigger_azure' @@ -44,6 +45,7 @@ const GLOBAL_DRAFT_KINDS = [ 'trigger_nats', 'trigger_postgres', 'trigger_mqtt', + 'trigger_amqp', 'trigger_sqs', 'trigger_gcp', 'trigger_azure', diff --git a/frontend/src/lib/components/copilot/chat/global/workspaceItems.ts b/frontend/src/lib/components/copilot/chat/global/workspaceItems.ts index c17e84cc8f..31f9261227 100644 --- a/frontend/src/lib/components/copilot/chat/global/workspaceItems.ts +++ b/frontend/src/lib/components/copilot/chat/global/workspaceItems.ts @@ -7,6 +7,7 @@ import type { NewHttpTrigger, NewKafkaTrigger, NewMqttTrigger, + NewAmqpTrigger, NewNatsTrigger, NewPostgresTrigger, NewSchedule, @@ -38,6 +39,7 @@ export const TRIGGER_KINDS = [ 'nats', 'postgres', 'mqtt', + 'amqp', 'sqs', 'gcp', 'azure' @@ -52,6 +54,7 @@ export type TriggerRequestBody = | NewNatsTrigger | NewPostgresTrigger | NewMqttTrigger + | NewAmqpTrigger | NewSqsTrigger | GcpTriggerData | AzureTriggerData diff --git a/frontend/src/lib/components/copilot/chat/pipeline/core.ts b/frontend/src/lib/components/copilot/chat/pipeline/core.ts index 31d74880a5..795b2983fe 100644 --- a/frontend/src/lib/components/copilot/chat/pipeline/core.ts +++ b/frontend/src/lib/components/copilot/chat/pipeline/core.ts @@ -319,7 +319,7 @@ export function getPipelinePromptSection(ctx: PipelineContext): string { Data Pipeline editor (ACTIVE): - The user has the /pipeline/${ctx.folder} editor open. A pipeline is a DAG of scripts (nodes) connected by storage assets (DuckLake tables, data tables, S3 objects, volumes, resources) and execution triggers. - Annotations are top-of-file comments in the NODE'S OWN comment syntax: \`--\` for SQL (duckdb/postgresql), \`#\` for python3/bash, \`//\` for bun/TS. The \`//\` shown below is the TS form — translate it (a \`// pipeline\` line in a SQL node is a syntax error that won't deploy). -- A script becomes a pipeline node when its source starts with the \`// pipeline\` annotation. Declare execution-DAG inputs with \`// on \` (e.g. \`// on ducklake://main/orders\`). Outputs are inferred from what the body writes (wmill SDK calls / SQL CREATE TABLE / writeS3File); declare a managed output with \`// materialize \`. Optional badges: \`// partitioned \`, \`// freshness \`, \`// tag \`, \`// retry [delay]\`, \`// data_test ...\`, \`// measure = [where ]\`, \`// dimension = \`. +- A script becomes a pipeline node when its source starts with the \`// pipeline\` annotation. Declare execution-DAG inputs with \`// on \` (e.g. \`// on ducklake://main/orders\`). Outputs are inferred from what the body writes (wmill SDK calls / SQL CREATE TABLE / writeS3File); declare a managed output with \`// materialize \`. Optional badges: \`// partitioned \`, \`// freshness \`, \`// tag \`, \`// retry [delay]\`, \`// data_test ...\`, \`// measure = [where ]\`, \`// dimension = \`. - \`materialize\` (the managed output): \`// materialize \` means the runtime writes the node's output table FOR you — write the body as a single SELECT and the runtime wraps it in the create/replace, so do NOT also write your own CREATE TABLE / INSERT. IMPORTANT: \`// materialize\` is **DuckDB-only** and its target MUST be a DuckLake table (\`ducklake:///\`) — deploy rejects it on any other language or target. For a \`python3\`/\`bun\`/\`postgresql\` node, do NOT use \`// materialize\`; write the output via the SDK instead (e.g. \`wmill.writeS3File(...)\`, a \`CREATE TABLE\` in postgresql, or \`wmill.databaseUrlFromResource\`/ducklake helpers) and let the output be inferred. Reach for \`duckdb\` when a node should materialize a DuckLake table. Write strategy: with no option it REPLACES the whole table each run (full refresh; the only mode whose output columns may change); \`// materialize append\` INSERT-appends rows (incremental); \`// materialize key=\` merges/upserts on \`\`. \`// materialize manual \` opts OUT of managed writes — the script writes its own DDL and the annotation only records the output asset for lineage. \`materialize\` is paired with partitioning for incremental pipelines: a \`// partitioned \` node runs once per partition (append/merge into a fixed-schema table), and the \`{partition}\` token — usable in any asset URI AND in the body SQL — is substituted with the current partition's IDENTITY string at run time. To filter the source to the active slice on a time grain, use the runtime-injected macro: \`WHERE wm_partition() = {partition}\`. \`wm_partition(ts)\` buckets a timestamp with the exact identity format the runtime used (daily/hourly/weekly/monthly), so it always matches and you never hand-write a \`strftime\` format. Do NOT write \`= TIMESTAMP {partition}\`: the identity string is not a valid timestamp literal for hourly/weekly/monthly and errors at runtime. For \`dynamic\` partitioning the identity is your caller-supplied key (not a timestamp, no macro), so filter on it directly: \`WHERE = {partition}\`. \`materialize\` is an output DECLARATION on the node — it is not a command; there is no "materialize run". - \`measure\` / \`dimension\` (declared metrics): on a node that materializes a DuckLake table, \`// measure = [where ]\` names the canonical way to aggregate that table (e.g. \`// measure revenue = sum(amount) where not is_refund\`), and \`// dimension = \` names a way to slice it (e.g. \`// dimension region = region\`, \`// dimension month = date_trunc('month', ordered_at)\`). They execute nothing: they are catalogued at deploy so the editor and other agents can reuse the definition instead of re-deriving it and silently disagreeing. Keep the predicate in the \`where\` clause rather than folding it into the aggregate: it is rendered as \` FILTER (WHERE )\`, which is what lets two measures with different predicates sit under one GROUP BY. DuckLake-only, and only meaningful next to \`// materialize\`. Declare one when a number carries a judgement call someone else would get wrong (refunds excluded, test rows dropped, which column is the amount); do NOT blanket every table with measures, an obvious \`count(*)\` earns nothing. To USE a metric another node declares, read that node with read_pipeline_node and reuse its exact expression rather than guessing it. - Use get_pipeline_graph to see the current nodes/assets/triggers, and read_pipeline_node before editing one. diff --git a/frontend/src/lib/components/copilot/chat/shared.ts b/frontend/src/lib/components/copilot/chat/shared.ts index a1f79f181e..2e3f49de5f 100644 --- a/frontend/src/lib/components/copilot/chat/shared.ts +++ b/frontend/src/lib/components/copilot/chat/shared.ts @@ -482,6 +482,7 @@ export type CreatedResourceTriggerKind = | 'nats' | 'postgres' | 'mqtt' + | 'amqp' | 'sqs' | 'gcp' | 'azure' diff --git a/frontend/src/lib/components/copilot/chat/workspaceItems.svelte.ts b/frontend/src/lib/components/copilot/chat/workspaceItems.svelte.ts index 8a7932b8d9..b1a6f695d3 100644 --- a/frontend/src/lib/components/copilot/chat/workspaceItems.svelte.ts +++ b/frontend/src/lib/components/copilot/chat/workspaceItems.svelte.ts @@ -7,6 +7,7 @@ import { HttpTriggerService, KafkaTriggerService, MqttTriggerService, + AmqpTriggerService, NatsTriggerService, PostgresTriggerService, ResourceService, @@ -35,6 +36,7 @@ export type WindmillItemKind = | 'nats_trigger' | 'postgres_trigger' | 'mqtt_trigger' + | 'amqp_trigger' | 'sqs_trigger' | 'gcp_trigger' | 'azure_trigger' @@ -123,6 +125,7 @@ const workspaceItemLoaders: Array<{ list: (workspace) => PostgresTriggerService.listPostgresTriggers({ workspace }) }, { kind: 'mqtt_trigger', list: (workspace) => MqttTriggerService.listMqttTriggers({ workspace }) }, + { kind: 'amqp_trigger', list: (workspace) => AmqpTriggerService.listAmqpTriggers({ workspace }) }, { kind: 'sqs_trigger', list: (workspace) => SqsTriggerService.listSqsTriggers({ workspace }) }, { kind: 'gcp_trigger', list: (workspace) => GcpTriggerService.listGcpTriggers({ workspace }) }, { diff --git a/frontend/src/lib/components/copilot/chat/workspaceTools.ts b/frontend/src/lib/components/copilot/chat/workspaceTools.ts index 5609ab1321..a976b7dab6 100644 --- a/frontend/src/lib/components/copilot/chat/workspaceTools.ts +++ b/frontend/src/lib/components/copilot/chat/workspaceTools.ts @@ -4,6 +4,7 @@ import { HttpTriggerService, KafkaTriggerService, MqttTriggerService, + AmqpTriggerService, NatsTriggerService, PostgresTriggerService, ScheduleService, @@ -14,6 +15,7 @@ import { type NewHttpTrigger, type NewKafkaTrigger, type NewMqttTrigger, + type NewAmqpTrigger, type NewNatsTrigger, type NewPostgresTrigger, type NewSchedule, @@ -45,6 +47,7 @@ type TriggerRequestByKind = { nats: NewNatsTrigger postgres: NewPostgresTrigger mqtt: NewMqttTrigger + amqp: NewAmqpTrigger sqs: NewSqsTrigger gcp: GcpTriggerData azure: AzureTriggerData @@ -155,6 +158,12 @@ const triggerConfigs = { create: (data: { workspace: string; requestBody: NewMqttTrigger }) => MqttTriggerService.createMqttTrigger(data) }, + amqp: { + label: 'AMQP trigger', + requestSchema: triggerRequestSchemas.amqp as z.ZodType, + create: (data: { workspace: string; requestBody: NewAmqpTrigger }) => + AmqpTriggerService.createAmqpTrigger(data) + }, sqs: { label: 'SQS trigger', requestSchema: triggerRequestSchemas.sqs as z.ZodType, @@ -302,7 +311,9 @@ const createScheduleTool: Tool = { }) return JSON.stringify(toolResult) } catch (error) { - throw new Error(`Failed to create schedule "${requestBody.path}": ${formatToolError(error)}`) + throw new Error( + `Failed to create schedule "${requestBody.path}": ${formatToolError(error)}` + ) } } catch (error) { return setToolError(toolCallbacks, toolId, error) diff --git a/frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts b/frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts index d989412bb1..79883d2dd9 100644 --- a/frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts +++ b/frontend/src/lib/components/copilot/chat/workspaceToolsZod.gen.ts @@ -293,6 +293,43 @@ export const mqttTriggerRequestSchema = z.object({ "labels": z.array(z.string()).optional() }) +export const amqpTriggerRequestSchema = z.object({ + "amqp_resource_path": z.string().describe("Path to the AMQP resource containing broker connection configuration"), + "queue_name": z.string().describe("Name of the queue to consume messages from"), + "exchange": z.object({ + "exchange_name": z.string().describe("Name of the exchange to bind the consumed queue to"), + "routing_keys": z.array(z.string()).describe("Routing keys used to bind the queue to the exchange").optional() + }).describe("Optional exchange binding for the consumed queue").nullable().optional(), + "options": z.object({ + "declare_queue": z.boolean().describe("Declare the queue (durable) before consuming; when false the queue is declared passively and must already exist").optional(), + "prefetch_count": z.number().int().gte(1).lte(65535).describe("Maximum number of unacknowledged messages the broker delivers at once (1-65535)").optional() + }).describe("Optional consumer options (queue declaration, prefetch)").nullable().optional(), + "path": z.string().describe("The unique Windmill path for this trigger. Must be of the form `u//` or `f//`."), + "script_path": z.string().describe("Path to the script or flow to execute when a message is received"), + "is_flow": z.boolean().describe("True if script_path points to a flow, false if it points to a script"), + "mode": z.enum(["enabled", "disabled", "suspended"]).describe("job trigger mode").optional(), + "error_handler_path": z.string().describe("Path to a script or flow to run when the triggered job fails").optional(), + "error_handler_args": z.record(z.string(), z.any()).describe("Arguments to pass to the error handler").optional(), + "retry": z.object({ + "constant": z.object({ + "attempts": z.number().int().describe("Number of retry attempts").optional(), + "seconds": z.number().int().describe("Seconds to wait between retries").optional() + }).describe("Retry with constant delay between attempts").optional(), + "exponential": z.object({ + "attempts": z.number().int().describe("Number of retry attempts").optional(), + "multiplier": z.number().int().describe("Multiplier for exponential backoff").optional(), + "seconds": z.number().int().gte(1).describe("Initial delay in seconds").optional(), + "random_factor": z.number().int().gte(0).lte(100).describe("Random jitter percentage (0-100) to avoid thundering herd").optional() + }).describe("Retry with exponential backoff (delay doubles each time)").optional(), + "retry_if": z.object({ + "expr": z.string().describe("JavaScript expression that returns true to retry. Has access to 'result' and 'error' variables") + }).describe("Conditional retry based on error or result").optional() + }).describe("Retry configuration for failed executions").optional(), + "permissioned_as": z.string().describe("The user or group this trigger runs as. Used during deployment to preserve the original trigger owner.").optional(), + "preserve_permissioned_as": z.boolean().describe("When true and the caller is a member of the 'wm_deployers' group, preserves the original permissioned_as value instead of overwriting it.").optional(), + "labels": z.array(z.string()).optional() +}) + export const sqsTriggerRequestSchema = z.object({ "queue_url": z.string().describe("The full URL of the AWS SQS queue to poll for messages"), "aws_auth_resource_type": z.enum(["oidc", "credentials"]).describe("Authentication type - 'credentials' for access key/secret, 'oidc' for OpenID Connect"), @@ -425,6 +462,7 @@ export const triggerRequestSchemas = { nats: natsTriggerRequestSchema, postgres: postgresTriggerRequestSchema, mqtt: mqttTriggerRequestSchema, + amqp: amqpTriggerRequestSchema, sqs: sqsTriggerRequestSchema, gcp: gcpTriggerRequestSchema, azure: azureTriggerRequestSchema, @@ -440,6 +478,7 @@ export const createTriggerToolSchema = z.object({ "nats", "postgres", "mqtt", + "amqp", "sqs", "gcp", "azure", @@ -452,6 +491,7 @@ export const createTriggerToolSchema = z.object({ natsTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), postgresTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), mqttTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), + amqpTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), sqsTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), gcpTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), azureTriggerRequestSchema.omit({ path: true, script_path: true, is_flow: true }), diff --git a/frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte b/frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte index d7340e053a..ac41827d6c 100644 --- a/frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte +++ b/frontend/src/lib/components/graph/renderers/triggers/TriggersBadge.svelte @@ -5,7 +5,14 @@ import { getContext } from 'svelte' import { type TriggerContext } from '$lib/components/triggers' import { enterpriseLicense } from '$lib/stores' - import { MqttIcon, NatsIcon, KafkaIcon, AwsIcon, GoogleCloudIcon } from '$lib/components/icons' + import { + MqttIcon, + AmqpIcon, + NatsIcon, + KafkaIcon, + AwsIcon, + GoogleCloudIcon + } from '$lib/components/icons' import AzureIcon from '$lib/components/icons/AzureIcon.svelte' import { type Trigger, type TriggerType } from '$lib/components/triggers/utils' import { Menu, Menubar, MeltButton, MenuItem, Tooltip } from '$lib/components/meltComponents' @@ -69,6 +76,7 @@ email: { icon: Mail, countKey: 'email_count' }, nats: { icon: NatsIcon, countKey: 'nats_count', disabled: !$enterpriseLicense }, mqtt: { icon: MqttIcon, countKey: 'mqtt_count', disabled: !$enterpriseLicense }, + amqp: { icon: AmqpIcon, countKey: 'amqp_count' }, sqs: { icon: AwsIcon, countKey: 'sqs_count', disabled: !$enterpriseLicense }, gcp: { icon: GoogleCloudIcon, countKey: 'gcp_count', disabled: !$enterpriseLicense }, azure: { icon: AzureIcon, countKey: 'azure_count', disabled: !$enterpriseLicense }, @@ -103,6 +111,7 @@ 'default_email', 'nats', 'mqtt', + 'amqp', 'sqs', 'gcp', 'azure', diff --git a/frontend/src/lib/components/icons/AmqpIcon.svelte b/frontend/src/lib/components/icons/AmqpIcon.svelte new file mode 100644 index 0000000000..41987f704f --- /dev/null +++ b/frontend/src/lib/components/icons/AmqpIcon.svelte @@ -0,0 +1,22 @@ + + + + + diff --git a/frontend/src/lib/components/icons/index.ts b/frontend/src/lib/components/icons/index.ts index b39df4310e..7774bb50f0 100644 --- a/frontend/src/lib/components/icons/index.ts +++ b/frontend/src/lib/components/icons/index.ts @@ -104,6 +104,7 @@ import XeroIcon from './XeroIcon.svelte' import KafkaIcon from './KafkaIcon.svelte' import NatsIcon from './NatsIcon.svelte' import MqttIcon from './MqttIcon.svelte' +import AmqpIcon from './AmqpIcon.svelte' import ApifyIcon from './ApifyIcon.svelte' import McpIcon from './McpIcon.svelte' import SageIcon from './SageIcon.svelte' @@ -333,6 +334,7 @@ export const APP_TO_ICON_COMPONENT = { kafka: KafkaIcon, nats: NatsIcon, mqtt: MqttIcon, + amqp: AmqpIcon, apify: ApifyIcon, mcp: McpIcon, zoho: ZohoIcon, @@ -550,6 +552,7 @@ export { KafkaIcon, NatsIcon, MqttIcon, + AmqpIcon, ApifyIcon, McpIcon, ZohoIcon, diff --git a/frontend/src/lib/components/offboarding-utils.ts b/frontend/src/lib/components/offboarding-utils.ts index f6f41bcc68..11f266e101 100644 --- a/frontend/src/lib/components/offboarding-utils.ts +++ b/frontend/src/lib/components/offboarding-utils.ts @@ -28,6 +28,7 @@ const TRIGGER_TABLE_TO_ROUTE: Record = { kafka_trigger: 'kafka_triggers', postgres_trigger: 'postgres_triggers', mqtt_trigger: 'mqtt_triggers', + amqp_trigger: 'amqp_triggers', nats_trigger: 'nats_triggers', sqs_trigger: 'sqs_triggers', gcp_trigger: 'gcp_triggers', @@ -41,6 +42,7 @@ const TRIGGER_TABLE_TO_LABEL: Record = { kafka_trigger: 'kafka trigger', postgres_trigger: 'postgres trigger', mqtt_trigger: 'mqtt trigger', + amqp_trigger: 'amqp trigger', nats_trigger: 'nats trigger', sqs_trigger: 'sqs trigger', gcp_trigger: 'gcp trigger', diff --git a/frontend/src/lib/components/search/GlobalSearchModal.svelte b/frontend/src/lib/components/search/GlobalSearchModal.svelte index 3e631a55e9..d2b27635da 100644 --- a/frontend/src/lib/components/search/GlobalSearchModal.svelte +++ b/frontend/src/lib/components/search/GlobalSearchModal.svelte @@ -40,7 +40,15 @@ import { Alert } from '../common' import Popover from '../Popover.svelte' import Logs from 'lucide-svelte/icons/logs' - import { AwsIcon, AzureIcon, GoogleCloudIcon, KafkaIcon, MqttIcon, NatsIcon } from '../icons' + import { + AwsIcon, + AzureIcon, + GoogleCloudIcon, + KafkaIcon, + MqttIcon, + AmqpIcon, + NatsIcon + } from '../icons' import RunsSearch from './RunsSearch.svelte' import AskAiButton from '../copilot/AskAiButton.svelte' @@ -152,6 +160,13 @@ icon: MqttIcon, disabled: $userStore?.operator }, + { + search_id: 'nav:amqp_triggers', + label: 'Go to AMQP triggers', + action: (newtab: boolean = false) => gotoPage('/amqp_triggers', newtab), + icon: AmqpIcon, + disabled: $userStore?.operator + }, { search_id: 'nav:email_triggers', label: 'Go to Email triggers', diff --git a/frontend/src/lib/components/sessions/previewRouter.ts b/frontend/src/lib/components/sessions/previewRouter.ts index f6a3cc8be4..1a876491db 100644 --- a/frontend/src/lib/components/sessions/previewRouter.ts +++ b/frontend/src/lib/components/sessions/previewRouter.ts @@ -57,6 +57,7 @@ export type TriggerKind = | 'gcp' | 'azure' | 'mqtt' + | 'amqp' | 'email' export const TRIGGER_PAGES: Record = { @@ -69,6 +70,7 @@ export const TRIGGER_PAGES: Record> = { trigger_nats: 'nats_trigger', trigger_postgres: 'postgres_trigger', trigger_mqtt: 'mqtt_trigger', + trigger_amqp: 'amqp_trigger', trigger_sqs: 'sqs_trigger', trigger_gcp: 'gcp_trigger', trigger_azure: 'azure_trigger', diff --git a/frontend/src/lib/components/sidebar/OperatorMenu.svelte b/frontend/src/lib/components/sidebar/OperatorMenu.svelte index b3b30aa710..e6519ffafc 100644 --- a/frontend/src/lib/components/sidebar/OperatorMenu.svelte +++ b/frontend/src/lib/components/sidebar/OperatorMenu.svelte @@ -158,6 +158,7 @@ kind: 'azure' }, { label: 'MQTT triggers', id: 'triggers', href: `${base}/mqtt_triggers`, kind: 'mqtt' }, + { label: 'AMQP triggers', id: 'triggers', href: `${base}/amqp_triggers`, kind: 'amqp' }, { label: 'Email triggers', id: 'triggers', href: `${base}/email_triggers`, kind: 'email' } ] as TriggerMenuLink[] ).filter(filterLink) diff --git a/frontend/src/lib/components/sidebar/SidebarContent.svelte b/frontend/src/lib/components/sidebar/SidebarContent.svelte index 524c11072a..2fffd7b22c 100644 --- a/frontend/src/lib/components/sidebar/SidebarContent.svelte +++ b/frontend/src/lib/components/sidebar/SidebarContent.svelte @@ -58,6 +58,7 @@ import KafkaIcon from '../icons/KafkaIcon.svelte' import NatsIcon from '../icons/NatsIcon.svelte' import MqttIcon from '../icons/MqttIcon.svelte' + import AmqpIcon from '../icons/AmqpIcon.svelte' import AwsIcon from '../icons/AwsIcon.svelte' import { getAvailableNativeTriggerServices, @@ -359,6 +360,15 @@ aiId: 'sidebar-menu-link-mqtt', aiDescription: 'Button to navigate to MQTT triggers' }, + { + label: 'AMQP', + href: '/amqp_triggers', + icon: AmqpIcon, + disabled: $userStore?.operator, + kind: 'amqp', + aiId: 'sidebar-menu-link-amqp', + aiDescription: 'Button to navigate to AMQP triggers' + }, { label: 'Email', href: '/email_triggers', diff --git a/frontend/src/lib/components/triggers.ts b/frontend/src/lib/components/triggers.ts index a870b9a622..9f41d7ced5 100644 --- a/frontend/src/lib/components/triggers.ts +++ b/frontend/src/lib/components/triggers.ts @@ -54,6 +54,7 @@ export type TriggerKind = | 'nats' | 'postgres' | 'mqtt' + | 'amqp' | 'sqs' | 'gcp' | 'azure' @@ -78,6 +79,8 @@ export function captureTriggerKindToTriggerKind(kind: CaptureTriggerKind): Trigg return 'nats' case 'mqtt': return 'mqtt' + case 'amqp': + return 'amqp' case 'sqs': return 'sqs' case 'postgres': diff --git a/frontend/src/lib/components/triggers/AddTriggersButton.svelte b/frontend/src/lib/components/triggers/AddTriggersButton.svelte index 19daa64489..901eb37244 100644 --- a/frontend/src/lib/components/triggers/AddTriggersButton.svelte +++ b/frontend/src/lib/components/triggers/AddTriggersButton.svelte @@ -92,6 +92,12 @@ icon: triggerIconMap.mqtt, extra: cloudHosted ? extra : undefined }, + { + displayName: 'AMQP', + action: () => onAddDraftTrigger?.('amqp'), + icon: triggerIconMap.amqp, + extra: cloudHosted ? extra : undefined + }, { displayName: 'SQS', action: () => onAddDraftTrigger?.('sqs'), diff --git a/frontend/src/lib/components/triggers/CaptureButton.svelte b/frontend/src/lib/components/triggers/CaptureButton.svelte index 995be6e1f5..cf148d19f4 100644 --- a/frontend/src/lib/components/triggers/CaptureButton.svelte +++ b/frontend/src/lib/components/triggers/CaptureButton.svelte @@ -11,6 +11,7 @@ import AwsIcon from '../icons/AwsIcon.svelte' import DropdownV2 from '$lib/components/DropdownV2.svelte' import MqttIcon from '../icons/MqttIcon.svelte' + import AmqpIcon from '../icons/AmqpIcon.svelte' import GoogleCloudIcon from '../icons/GoogleCloudIcon.svelte' interface Props { @@ -61,6 +62,11 @@ displayName: 'MQTT', action: () => handleClick('mqtt') }, + { + icon: AmqpIcon, + displayName: 'AMQP', + action: () => handleClick('amqp') + }, { icon: Database, displayName: 'Postgres', diff --git a/frontend/src/lib/components/triggers/CaptureTable.svelte b/frontend/src/lib/components/triggers/CaptureTable.svelte index 18ab7d7e91..2d20ac02e1 100644 --- a/frontend/src/lib/components/triggers/CaptureTable.svelte +++ b/frontend/src/lib/components/triggers/CaptureTable.svelte @@ -17,7 +17,7 @@ import { isObject, sendUserToast } from '$lib/utils' import SchemaPickerRow from '$lib/components/schema/SchemaPickerRow.svelte' import type { Capture } from '$lib/gen' - import { AwsIcon, MqttIcon } from '../icons' + import { AwsIcon, MqttIcon, AmqpIcon } from '../icons' import GoogleCloudIcon from '../icons/GoogleCloudIcon.svelte' interface Props { @@ -218,6 +218,7 @@ websocket: Unplug, kafka: KafkaIcon, mqtt: MqttIcon, + amqp: AmqpIcon, sqs: AwsIcon, postgres: DatabaseIcon, gcp: GoogleCloudIcon diff --git a/frontend/src/lib/components/triggers/CaptureWrapper.svelte b/frontend/src/lib/components/triggers/CaptureWrapper.svelte index 2c56681230..bfffbfc56b 100644 --- a/frontend/src/lib/components/triggers/CaptureWrapper.svelte +++ b/frontend/src/lib/components/triggers/CaptureWrapper.svelte @@ -13,6 +13,7 @@ import KafkaCapture from './kafka/KafkaCapture.svelte' import NatsCapture from './nats/NatsCapture.svelte' import MqttCapture from './mqtt/MqttCapture.svelte' + import AmqpCapture from './amqp/AmqpCapture.svelte' import SqsCapture from './sqs/SqsCapture.svelte' import GcpCapture from './gcp/GcpCapture.svelte' import AzureCapture from './azure/AzureCapture.svelte' @@ -78,9 +79,17 @@ if (captureType === 'azure' && args.azure_mode !== 'namespace_pull') { return false } - return ['mqtt', 'sqs', 'websocket', 'postgres', 'kafka', 'nats', 'gcp', 'azure'].includes( - captureType - ) + return [ + 'mqtt', + 'amqp', + 'sqs', + 'websocket', + 'postgres', + 'kafka', + 'nats', + 'gcp', + 'azure' + ].includes(captureType) } async function getCaptureConfigs() { @@ -301,6 +310,20 @@ on:captureToggle={handleCapture} on:testWithArgs /> + {:else if captureType === 'amqp'} + {:else if captureType === 'sqs'} ; - noButton?: boolean; - testLoading?: boolean; + kind: 'websocket' | 'nats' | 'kafka' | 'postgres' | 'sqs' | 'mqtt' | 'amqp' | 'gcp' + args: Record + noButton?: boolean + testLoading?: boolean } - let { - kind, - args, - noButton = false, - testLoading = $bindable(false) - }: Props = $props(); + let { kind, args, noButton = false, testLoading = $bindable(false) }: Props = $props() const triggerWs = getTriggerWorkspace() const wsId = $derived(triggerWs?.() ?? $workspaceStore) @@ -37,6 +33,7 @@ sqs: 'SQS', postgres: 'Postgres', mqtt: 'MQTT broker', + amqp: 'AMQP broker', gcp: 'Google Cloud Pub/Sub' } @@ -69,6 +66,11 @@ workspace: wsId!, requestBody: args as any }) + } else if (kind === 'amqp') { + promise = AmqpTriggerService.testAmqpConnection({ + workspace: wsId!, + requestBody: args as any + }) } else if (kind === 'sqs') { promise = SqsTriggerService.testSqsConnection({ workspace: wsId!, diff --git a/frontend/src/lib/components/triggers/TriggersEditor.svelte b/frontend/src/lib/components/triggers/TriggersEditor.svelte index 3f886be4ee..44f565d086 100644 --- a/frontend/src/lib/components/triggers/TriggersEditor.svelte +++ b/frontend/src/lib/components/triggers/TriggersEditor.svelte @@ -26,6 +26,7 @@ KafkaTriggerService, NatsTriggerService, MqttTriggerService, + AmqpTriggerService, HttpTriggerService, GcpTriggerService, AzureTriggerService, @@ -116,6 +117,7 @@ azure: () => AzureTriggerService.deleteAzureTrigger, sqs: () => SqsTriggerService.deleteSqsTrigger, mqtt: () => MqttTriggerService.deleteMqttTrigger, + amqp: () => AmqpTriggerService.deleteAmqpTrigger, http: () => HttpTriggerService.deleteHttpTrigger, email: () => EmailTriggerService.deleteEmailTrigger } @@ -255,6 +257,14 @@ isFlow, $userStore ) + } else if (triggerType === 'amqp') { + await triggersState.fetchAmqpTriggers( + triggersCount, + $workspaceStore, + currentPath, + isFlow, + $userStore + ) } else if (triggerType === 'http') { await triggersState.fetchHttpTriggers( triggersCount, diff --git a/frontend/src/lib/components/triggers/TriggersWrapper.svelte b/frontend/src/lib/components/triggers/TriggersWrapper.svelte index 6b9399c6a2..72e96c9150 100644 --- a/frontend/src/lib/components/triggers/TriggersWrapper.svelte +++ b/frontend/src/lib/components/triggers/TriggersWrapper.svelte @@ -8,6 +8,7 @@ import KafkaTriggerPanel from './kafka/KafkaTriggersPanel.svelte' import NatsTriggersPanel from './nats/NatsTriggersPanel.svelte' import MqttTriggerPanel from './mqtt/MqttTriggersPanel.svelte' + import AmqpTriggerPanel from './amqp/AmqpTriggersPanel.svelte' import SqsTriggerPanel from './sqs/SqsTriggerPanel.svelte' import GcpTriggerPanel from './gcp/GcpTriggerPanel.svelte' import AzureTriggerPanel from './azure/AzureTriggerPanel.svelte' @@ -143,6 +144,15 @@ {customLabel} {...props} /> +{:else if selectedTrigger.type === 'amqp'} + {:else if selectedTrigger.type === 'sqs'} + import { fade } from 'svelte/transition' + import type { CaptureInfo } from '../CaptureSection.svelte' + import CaptureSection from '../CaptureSection.svelte' + + interface Props { + captureInfo?: CaptureInfo | undefined + isValid?: boolean | undefined + hasPreprocessor?: boolean + isFlow?: boolean + captureLoading?: boolean + triggerDeployed?: boolean + } + + let { + captureInfo = undefined, + isValid = undefined, + hasPreprocessor = false, + isFlow = false, + captureLoading = false, + triggerDeployed = false + }: Props = $props() + + +{#if captureInfo} + + {#snippet description()} + {#if captureInfo.active} + {#if captureInfo.connectionInfo?.connected} +

+ Listening to AMQP messages... +

+ {:else} +

+ Connecting to amqp... +

+ {/if} + {:else} +

+ Start capturing to listen to AMQP messages. +

+ {/if} + {/snippet} + +{/if} diff --git a/frontend/src/lib/components/triggers/amqp/AmqpEditorConfigSection.svelte b/frontend/src/lib/components/triggers/amqp/AmqpEditorConfigSection.svelte new file mode 100644 index 0000000000..72b6f6360a --- /dev/null +++ b/frontend/src/lib/components/triggers/amqp/AmqpEditorConfigSection.svelte @@ -0,0 +1,175 @@ + + +
+
+ {#snippet header()} + {#if showTestingBadge} + + {/if} + {/snippet} +
+ + + {#if !emptyStringTrimmed(amqp_resource_path)} + + {/if} + + + +

+ Name of the queue to consume messages from +

+ + + + + { + if (ev.detail) { + exchange = { exchange_name: '', routing_keys: [] } + } else { + exchange = undefined + } + }} + options={{ + right: 'Bind the queue to an exchange', + rightTooltip: + 'Declare the queue and bind it to an exchange for each routing key, so messages published to the exchange are routed to it.' + }} + class="py-1" + /> + + {#if exchange} +
+ + + + Routing keys + The routing keys used to bind the queue to the exchange. + + {#each exchange.routing_keys ?? [] as _, i} +
+ + +
+ {/each} +
+ +
+
+ {/if} + +
+
+
diff --git a/frontend/src/lib/components/triggers/amqp/AmqpTriggerEditor.svelte b/frontend/src/lib/components/triggers/amqp/AmqpTriggerEditor.svelte new file mode 100644 index 0000000000..cc181951bf --- /dev/null +++ b/frontend/src/lib/components/triggers/amqp/AmqpTriggerEditor.svelte @@ -0,0 +1,29 @@ + + +{#if open} + +{/if} diff --git a/frontend/src/lib/components/triggers/amqp/AmqpTriggerEditorInner.svelte b/frontend/src/lib/components/triggers/amqp/AmqpTriggerEditorInner.svelte new file mode 100644 index 0000000000..6383db2a39 --- /dev/null +++ b/frontend/src/lib/components/triggers/amqp/AmqpTriggerEditorInner.svelte @@ -0,0 +1,615 @@ + + +{#if mode === 'suspended'} + +{/if} + +{#if useDrawer} + + + {#snippet actions()} + {@render actionsSnippet()} + {/snippet} + {#snippet banner()} + draftSync.deployed} + reserveSpace={draftSync.hasBaseline} + getCurrent={() => draftSync.current} + onDiscard={() => draftSync.resetToDeployed(initialPath)} + disabled={!can_write} + /> + {/snippet} + {@render config()} + + +{:else} +
+ {#snippet header()} + {#if customLabel} + {@render customLabel()} + {/if} + {/snippet} + {#snippet action()} + {@render actionsSnippet()} + {/snippet} + {@render config()} +
+{/if} + +{#snippet actionsSnippet()} + {#if !drawerLoading} + + {/if} +{/snippet} + +{#snippet config()} + {#if drawerLoading} + {#if showLoading} + + {/if} + {:else} + { + selectedPermissionedAs = pa + preservePermissionedAs = preserve + }} + /> +
+ {#if description} + {@render description()} + {/if} + {#if !hideTooltips} + + {#if edit} + Changes can take up to 30 seconds to take effect. + {:else} + New AMQP triggers can take up to 30 seconds to start listening. + {/if} + + {/if} +
+
+ {#if mode === 'suspended'} + + {/if} +
+ +
+ + {#if !hideTarget} +
+ +
+ {/if} + + + +
+ {#snippet header()} + + {/snippet} +
+
+ + + + + +
+ {#if optionTabSelected === 'connection_options'} +
+ setDeclareQueue(ev.detail)} + options={{ + right: 'Declare queue', + rightTooltip: + 'Declare the queue as durable before consuming. When disabled, the queue is declared passively and must already exist on the broker.' + }} + class="py-1" + /> + +
+ setPrefetchEnabled(ev.detail)} + options={{ + right: 'Prefetch count', + rightTooltip: + 'Maximum number of unacknowledged messages the broker delivers at once (QoS).' + }} + class="py-1" + /> + {#if enablePrefetch && options} + + {/if} +
+
+ {:else} + + {/if} +
+
+
+
+
+
+ {/if} +{/snippet} diff --git a/frontend/src/lib/components/triggers/amqp/AmqpTriggersPanel.svelte b/frontend/src/lib/components/triggers/amqp/AmqpTriggersPanel.svelte new file mode 100644 index 0000000000..40a68566de --- /dev/null +++ b/frontend/src/lib/components/triggers/amqp/AmqpTriggersPanel.svelte @@ -0,0 +1,61 @@ + + +
+ + {#snippet description()} + {#if cloudDisabled} + + AMQP triggers are disabled in the multi-tenant cloud. + + {:else} + + AMQP triggers allow you to execute scripts and flows in response to messages consumed from + an AMQP (RabbitMQ) queue. + + {/if} + {/snippet} + +
diff --git a/frontend/src/lib/components/triggers/amqp/utils.ts b/frontend/src/lib/components/triggers/amqp/utils.ts new file mode 100644 index 0000000000..f82fe2ebf7 --- /dev/null +++ b/frontend/src/lib/components/triggers/amqp/utils.ts @@ -0,0 +1,56 @@ +import { AmqpTriggerService, type EditAmqpTrigger } from '$lib/gen' +import { sendUserToast } from '$lib/toast' +import { get, type Writable } from 'svelte/store' + +export async function saveAmqpTriggerFromCfg( + initialPath: string, + cfg: Record, + edit: boolean, + workspace: string, + usedTriggerKinds: Writable +): Promise { + const errorHandlerAndRetries = !cfg.is_flow + ? { + error_handler_path: cfg.error_handler_path, + error_handler_args: cfg.error_handler_path ? cfg.error_handler_args : undefined, + retry: cfg.retry + } + : {} + const requestBody: EditAmqpTrigger = { + amqp_resource_path: cfg.amqp_resource_path, + queue_name: cfg.queue_name, + exchange: cfg.exchange, + options: cfg.options, + path: cfg.path, + script_path: cfg.script_path, + is_flow: cfg.is_flow, + mode: cfg.mode, + permissioned_as: cfg.permissioned_as, + preserve_permissioned_as: cfg.preserve_permissioned_as, + ...errorHandlerAndRetries + } + try { + if (edit) { + await AmqpTriggerService.updateAmqpTrigger({ + workspace, + path: initialPath, + requestBody + }) + sendUserToast(`AMQP trigger ${cfg.path} updated`) + } else { + await AmqpTriggerService.createAmqpTrigger({ + workspace, + requestBody: { ...requestBody, mode: 'enabled' } + }) + sendUserToast(`AMQP trigger ${cfg.path} created`) + } + + if (!get(usedTriggerKinds).includes('amqp')) { + usedTriggerKinds.update((t) => [...t, 'amqp']) + } + return true + } catch (error) { + sendUserToast(error.body || error.message, true) + return false + } +} diff --git a/frontend/src/lib/components/triggers/triggers.svelte.ts b/frontend/src/lib/components/triggers/triggers.svelte.ts index c849b24c79..cd25328ff6 100644 --- a/frontend/src/lib/components/triggers/triggers.svelte.ts +++ b/frontend/src/lib/components/triggers/triggers.svelte.ts @@ -1,6 +1,7 @@ import { KafkaTriggerService, MqttTriggerService, + AmqpTriggerService, NatsTriggerService, PostgresTriggerService, ScheduleService, @@ -349,6 +350,32 @@ export class Triggers { } } + async fetchAmqpTriggers( + triggersCountStore: Writable, + workspaceId: string | undefined, + path: string, + isFlow: boolean, + user: UserExt | undefined = undefined + ): Promise { + if (!workspaceId) return + try { + const amqpTriggers = await AmqpTriggerService.listAmqpTriggers({ + workspace: workspaceId, + path, + isFlow + }) + const amqpCount = this.updateTriggers(amqpTriggers, 'amqp', user) + triggersCountStore.update((triggersCount) => { + return { + ...(triggersCount ?? {}), + amqp_count: amqpCount + } + }) + } catch (error) { + console.error('Failed to fetch AMQP triggers:', error) + } + } + async fetchSqsTriggers( triggersCountStore: Writable, workspaceId: string | undefined, @@ -533,6 +560,7 @@ export class Triggers { this.fetchWebsocketTriggers(triggersCountStore, workspaceId, path, isFlow, user), this.fetchPostgresTriggers(triggersCountStore, workspaceId, path, isFlow, user), this.fetchMqttTriggers(triggersCountStore, workspaceId, path, isFlow, user), + this.fetchAmqpTriggers(triggersCountStore, workspaceId, path, isFlow, user), this.fetchNativeTriggers(triggersCountStore, 'nextcloud', workspaceId, path, isFlow, user), this.fetchNativeTriggers(triggersCountStore, 'google', workspaceId, path, isFlow, user), this.fetchNativeTriggers(triggersCountStore, 'github', workspaceId, path, isFlow, user), diff --git a/frontend/src/lib/components/triggers/utils.ts b/frontend/src/lib/components/triggers/utils.ts index d8066eb18a..6ff2ea3ea0 100644 --- a/frontend/src/lib/components/triggers/utils.ts +++ b/frontend/src/lib/components/triggers/utils.ts @@ -13,6 +13,7 @@ import { import KafkaIcon from '$lib/components/icons/KafkaIcon.svelte' import NatsIcon from '$lib/components/icons/NatsIcon.svelte' import MqttIcon from '$lib/components/icons/MqttIcon.svelte' +import AmqpIcon from '$lib/components/icons/AmqpIcon.svelte' import AwsIcon from '$lib/components/icons/AwsIcon.svelte' import GoogleCloudIcon from '$lib/components/icons/GoogleCloudIcon.svelte' import AzureIcon from '$lib/components/icons/AzureIcon.svelte' @@ -35,6 +36,7 @@ import { saveKafkaTriggerFromCfg } from './kafka/utils' import { saveSqsTriggerFromCfg } from './sqs/utils' import { saveNatsTriggerFromCfg } from './nats/utils' import { saveMqttTriggerFromCfg } from './mqtt/utils' +import { saveAmqpTriggerFromCfg } from './amqp/utils' import { saveGcpTriggerFromCfg } from './gcp/utils' import { saveAzureTriggerFromCfg } from './azure/utils' import type { Triggers } from './triggers.svelte' @@ -50,6 +52,7 @@ export const CLOUD_DISABLED_TRIGGER_TYPES = [ 'kafka', 'sqs', 'mqtt', + 'amqp', 'gcp', 'azure', 'websocket', @@ -67,6 +70,7 @@ export type TriggerType = | 'kafka' | 'nats' | 'mqtt' + | 'amqp' | 'sqs' | 'gcp' | 'azure' @@ -86,6 +90,7 @@ export const jobTriggerKinds: JobTriggerKind[] = [ 'email', 'nats', 'mqtt', + 'amqp', 'sqs', 'postgres', 'schedule', @@ -123,6 +128,7 @@ export const triggerIconMap = { kafka: KafkaIcon, nats: NatsIcon, mqtt: MqttIcon, + amqp: AmqpIcon, sqs: AwsIcon, gcp: GoogleCloudIcon, azure: AzureIcon, @@ -149,6 +155,7 @@ export const triggerDisplayNamesMap = { kafka: 'Kafka', nats: 'NATS', mqtt: 'MQTT', + amqp: 'AMQP', sqs: 'SQS', gcp: 'GCP Pub/Sub', azure: 'Azure Event Grid', @@ -184,6 +191,7 @@ export function triggerTypeToCaptureKind(triggerType: TriggerType): CaptureTrigg 'kafka', 'nats', 'mqtt', + 'amqp', 'sqs', 'gcp', 'azure', @@ -215,6 +223,7 @@ export function updateTriggersCount( kafka: 'kafka_count', nats: 'nats_count', mqtt: 'mqtt_count', + amqp: 'amqp_count', sqs: 'sqs_count', gcp: 'gcp_count', azure: 'azure_count', @@ -286,6 +295,8 @@ export function triggerKindToTriggerType(kind: TriggerKind): TriggerType | undef return 'nats' case 'mqtt': return 'mqtt' + case 'amqp': + return 'amqp' case 'sqs': return 'sqs' case 'gcp': @@ -381,6 +392,14 @@ export async function deployTriggers( workspaceId, usedTriggerKinds ), + amqp: (trigger: Trigger) => + saveAmqpTriggerFromCfg( + trigger.path ?? trigger.draftConfig?.path ?? '', + trigger.draftConfig ?? {}, + !trigger.isDraft, + workspaceId, + usedTriggerKinds + ), sqs: (trigger: Trigger) => saveSqsTriggerFromCfg( trigger.path ?? trigger.draftConfig?.path ?? '', diff --git a/frontend/src/lib/script_helpers.ts b/frontend/src/lib/script_helpers.ts index 4a85107b70..91e310f5c9 100644 --- a/frontend/src/lib/script_helpers.ts +++ b/frontend/src/lib/script_helpers.ts @@ -703,7 +703,7 @@ export const TS_PREPROCESSOR_SCRIPT_INTRO = `/** * * ⚠️ This function runs BEFORE the main function. * - * It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) + * It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) * before passing it to \`main\`. This separates the trigger logic from the main logic and keeps the auto-generated runnable UI clean. * * The returned object defines the parameter values passed to \`main()\`. @@ -716,7 +716,7 @@ export const TS_PREPROCESSOR_SCRIPT_INTRO = `/** export const TS_PREPROCESSOR_FLOW_INTRO = `/** * Trigger preprocessor * - * It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) + * It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) * before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. * * The returned object determines the parameter values passed to the flow. @@ -812,6 +812,16 @@ type TriggerEvent = content_type?: string; }; } + | { + kind: "amqp"; + trigger_path: string; + payload: string; + exchange: string; + routing_key: string; + queue_name: string; + redelivered: boolean; + delivery_tag: number; + } | { kind: "gcp"; trigger_path: string; @@ -867,7 +877,7 @@ export const PYTHON_PREPROCESSOR_SCRIPT_INTRO = `# Trigger preprocessor # # ⚠️ This function runs BEFORE the main function. # -# It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) +# It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) # before passing it to \`main\`. This separates the trigger logic from the main logic and keeps the auto-generated UI clean. # # The returned object defines the parameter values passed to \`main()\`. @@ -878,7 +888,7 @@ export const PYTHON_PREPROCESSOR_SCRIPT_INTRO = `# Trigger preprocessor export const PYTHON_PREPROCESSOR_FLOW_INTRO = `# Trigger preprocessor # -# It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) +# It processes raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) # before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. # # The returned object determines the parameter values passed to the flow. @@ -986,6 +996,17 @@ class MqttEvent(TypedDict): v5: Optional[MqttV5Properties] +class AmqpEvent(TypedDict): + kind: Literal["amqp"] + trigger_path: str + payload: str + exchange: str + routing_key: str + queue_name: str + redelivered: bool + delivery_tag: int + + class GcpEvent(TypedDict): kind: Literal["gcp"] trigger_path: str @@ -1019,6 +1040,7 @@ Event = Union[ NatsEvent, SqsEvent, MqttEvent, + AmqpEvent, GcpEvent, PostgresEvent, ] @@ -1036,7 +1058,7 @@ export const PHP_PREPROCESSOR_SCRIPT_INTRO = ` 'mqtt', 'trigger_path' => '...', 'payload' => '...', 'topic' => '...', 'retain' => true, 'pkid' => 1, // 'qos' => 1, 'v5' => [...]] // + // AMQP event: + // ['kind' => 'amqp', 'trigger_path' => '...', 'payload' => '...', 'exchange' => '...', 'routing_key' => '...', + // 'queue_name' => '...', 'redelivered' => false, 'delivery_tag' => 1] + // // GCP event: // ['kind' => 'gcp', 'trigger_path' => '...', 'payload' => '...', 'message_id' => '...', 'subscription' => '...', // 'ordering_key' => '...', 'attributes' => [...], 'delivery_type' => 'push', diff --git a/frontend/src/lib/userDraft.svelte.ts b/frontend/src/lib/userDraft.svelte.ts index 6ecacbce22..24572ee1ab 100644 --- a/frontend/src/lib/userDraft.svelte.ts +++ b/frontend/src/lib/userDraft.svelte.ts @@ -27,6 +27,7 @@ export const USER_DRAFT_ITEM_KINDS = [ 'trigger_kafka', 'trigger_nats', 'trigger_mqtt', + 'trigger_amqp', 'trigger_sqs', 'trigger_gcp', 'trigger_azure', diff --git a/frontend/src/lib/userDraftDbMigration.ts b/frontend/src/lib/userDraftDbMigration.ts index 4c3079ea94..1496e595cb 100644 --- a/frontend/src/lib/userDraftDbMigration.ts +++ b/frontend/src/lib/userDraftDbMigration.ts @@ -46,6 +46,7 @@ const ITEM_KINDS = [ 'trigger_kafka', 'trigger_nats', 'trigger_mqtt', + 'trigger_amqp', 'trigger_sqs', 'trigger_gcp', 'trigger_azure', diff --git a/frontend/src/lib/utils_deployable.ts b/frontend/src/lib/utils_deployable.ts index 556ac0a3c7..7d517c3bb4 100644 --- a/frontend/src/lib/utils_deployable.ts +++ b/frontend/src/lib/utils_deployable.ts @@ -6,6 +6,7 @@ import { HttpTriggerService, KafkaTriggerService, MqttTriggerService, + AmqpTriggerService, NatsTriggerService, PostgresTriggerService, ScheduleService, @@ -36,6 +37,7 @@ export type Kind = | 'nats_trigger' | 'postgres_trigger' | 'mqtt_trigger' + | 'amqp_trigger' | 'sqs_trigger' | 'gcp_trigger' | 'azure_trigger' @@ -94,6 +96,8 @@ export async function existsTrigger( return await KafkaTriggerService.existsKafkaTrigger(data) } else if (triggerKind === 'mqtt') { return await MqttTriggerService.existsMqttTrigger(data) + } else if (triggerKind === 'amqp') { + return await AmqpTriggerService.existsAmqpTrigger(data) } else if (triggerKind === 'postgres') { return await PostgresTriggerService.existsPostgresTrigger(data) } else if (triggerKind === 'sqs') { @@ -196,6 +200,21 @@ export async function getTriggersDeployData( createFn: MqttTriggerService.createMqttTrigger, updateFn: MqttTriggerService.updateMqttTrigger } + } else if (kind === 'amqp') { + const amqpTrigger = await AmqpTriggerService.getAmqpTrigger({ + workspace: workspace!, + path: path + }) + + return { + data: { + ...amqpTrigger, + permissioned_as: onBehalfOf, + preserve_permissioned_as: preservePermissionedAs + }, + createFn: AmqpTriggerService.createAmqpTrigger, + updateFn: AmqpTriggerService.updateAmqpTrigger + } } else if (kind === 'nats') { const natsTrigger = await NatsTriggerService.getNatsTrigger({ workspace: workspace!, @@ -372,6 +391,8 @@ export async function getTriggerValue(kind: TriggerKind, path: string, workspace trigger = await KafkaTriggerService.getKafkaTrigger({ workspace, path }) } else if (kind === 'mqtt') { trigger = await MqttTriggerService.getMqttTrigger({ workspace, path }) + } else if (kind === 'amqp') { + trigger = await AmqpTriggerService.getAmqpTrigger({ workspace, path }) } else if (kind === 'nats') { trigger = await NatsTriggerService.getNatsTrigger({ workspace, path }) } else if (kind === 'postgres') { @@ -412,6 +433,9 @@ export async function getTriggerPermissionedAs( } else if (kind === 'mqtt') { const trigger = await MqttTriggerService.getMqttTrigger({ workspace, path }) return trigger.permissioned_as + } else if (kind === 'amqp') { + const trigger = await AmqpTriggerService.getAmqpTrigger({ workspace, path }) + return trigger.permissioned_as } else if (kind === 'nats') { const trigger = await NatsTriggerService.getNatsTrigger({ workspace, path }) return trigger.permissioned_as @@ -493,6 +517,13 @@ export async function getTriggerDependency(kind: TriggerKind, path: string, work }) result = retrieveKindsValues({ resource_path: mqtt_resource_path, script_path, is_flow }) + } else if (kind === 'amqp') { + const { amqp_resource_path, script_path, is_flow } = await AmqpTriggerService.getAmqpTrigger({ + workspace: workspace!, + path: path + }) + + result = retrieveKindsValues({ resource_path: amqp_resource_path, script_path, is_flow }) } else if (kind === 'nats') { const { nats_resource_path, script_path, is_flow } = await NatsTriggerService.getNatsTrigger({ workspace: workspace!, diff --git a/frontend/src/lib/utils_draft_deploy.ts b/frontend/src/lib/utils_draft_deploy.ts index ee8538e4db..6068753c4c 100644 --- a/frontend/src/lib/utils_draft_deploy.ts +++ b/frontend/src/lib/utils_draft_deploy.ts @@ -27,6 +27,7 @@ import { KafkaTriggerService, NatsTriggerService, MqttTriggerService, + AmqpTriggerService, SqsTriggerService, GcpTriggerService, AzureTriggerService, @@ -49,6 +50,7 @@ import { savePostgresTriggerFromCfg } from '$lib/components/triggers/postgres/ut import { saveKafkaTriggerFromCfg } from '$lib/components/triggers/kafka/utils' import { saveNatsTriggerFromCfg } from '$lib/components/triggers/nats/utils' import { saveMqttTriggerFromCfg } from '$lib/components/triggers/mqtt/utils' +import { saveAmqpTriggerFromCfg } from '$lib/components/triggers/amqp/utils' import { saveSqsTriggerFromCfg } from '$lib/components/triggers/sqs/utils' import { saveGcpTriggerFromCfg } from '$lib/components/triggers/gcp/utils' import { saveAzureTriggerFromCfg } from '$lib/components/triggers/azure/utils' @@ -79,6 +81,8 @@ const OVERLAY_GETTERS: Partial< NatsTriggerService.getNatsTrigger({ workspace, path, getDraft: true }), trigger_mqtt: (workspace, path) => MqttTriggerService.getMqttTrigger({ workspace, path, getDraft: true }), + trigger_amqp: (workspace, path) => + AmqpTriggerService.getAmqpTrigger({ workspace, path, getDraft: true }), trigger_sqs: (workspace, path) => SqsTriggerService.getSqsTrigger({ workspace, path, getDraft: true }), trigger_gcp: (workspace, path) => @@ -589,6 +593,8 @@ const TRIGGER_SAVERS: Partial< saveNatsTriggerFromCfg(p, cfg, edit, ws, writable([])), trigger_mqtt: (p, cfg, edit, ws) => saveMqttTriggerFromCfg(p, cfg, edit, ws, writable([])), + trigger_amqp: (p, cfg, edit, ws) => + saveAmqpTriggerFromCfg(p, cfg, edit, ws, writable([])), trigger_sqs: (p, cfg, edit, ws) => saveSqsTriggerFromCfg(p, cfg, edit, ws, writable([])), trigger_gcp: (p, cfg, edit, ws) => diff --git a/frontend/src/routes/(root)/(logged)/+layout.svelte b/frontend/src/routes/(root)/(logged)/+layout.svelte index 9d6accc4e2..d36569ae95 100644 --- a/frontend/src/routes/(root)/(logged)/+layout.svelte +++ b/frontend/src/routes/(root)/(logged)/+layout.svelte @@ -520,6 +520,7 @@ nats_used, sqs_used, mqtt_used, + amqp_used, gcp_used, azure_used, email_used, @@ -547,6 +548,9 @@ if (mqtt_used) { usedKinds.push('mqtt') } + if (amqp_used) { + usedKinds.push('amqp') + } if (sqs_used) { usedKinds.push('sqs') } diff --git a/frontend/src/routes/(root)/(logged)/amqp_triggers/+page.js b/frontend/src/routes/(root)/(logged)/amqp_triggers/+page.js new file mode 100644 index 0000000000..42bd37a2fd --- /dev/null +++ b/frontend/src/routes/(root)/(logged)/amqp_triggers/+page.js @@ -0,0 +1,5 @@ +export function load() { + return { + stuff: { title: 'AMQP triggers' } + } +} diff --git a/frontend/src/routes/(root)/(logged)/amqp_triggers/+page.svelte b/frontend/src/routes/(root)/(logged)/amqp_triggers/+page.svelte new file mode 100644 index 0000000000..8d7316d6ed --- /dev/null +++ b/frontend/src/routes/(root)/(logged)/amqp_triggers/+page.svelte @@ -0,0 +1,550 @@ + + + + + + (x.summary ?? '') + ' ' + x.path + ' (' + x.script_path + ')'} +/> + + + + + + + {#if isCloudHosted()} + + AMQP triggers are disabled in the multi-tenant cloud. + +
+ {/if} +
+
+ +
+
Filter by path of
+ + {#snippet children({ item })} + + + {/snippet} + +
+ + +
+ {#if $userStore?.is_super_admin && $userStore.username.includes('@')} + + {:else if $userStore?.is_admin || $userStore?.is_super_admin} + + {/if} +
+
+ {#if loading} + {#each new Array(6) as _} + + {/each} + {:else if !triggers?.length} +
No AMQP triggers
+ {:else if items?.length} +
+ {#each items.slice(0, nbDisplayed) as { path, edited_by, edited_at, script_path, is_flow, extra_perms, canWrite, error, last_server_ping, server_id, mode, retry, error_handler_path, error_handler_args, labels, draft_only, is_draft } (path)} + {@const href = `${is_flow ? '/flows/get' : '/scripts/get'}/${script_path}`} + {@const ping = last_server_ping ? new Date(last_server_ping) : undefined} + {@const pinging = ping && ping.getTime() > new Date().getTime() - 15 * 1000} + {@const enabled = mode === 'enabled' || mode === 'suspended'} + +
+
+ + + amqpTriggerEditor?.openEdit(path, is_flow)} + class="min-w-0 grow hover:underline decoration-gray-400" + > +
+ {path}{(getLocalDraftHint($workspaceStore, 'trigger_amqp', path) ?? is_draft) ? '*' : ''} +
+
+ runnable: {script_path} +
+ + + + +
+ {#if (enabled && (!pinging || error)) || (!enabled && error) || (enabled && !server_id)} + + + + + + {#snippet text()} +
+ {#if enabled} + {#if !server_id} + AMQP trigger is starting... + {:else} + AMQP trigger is not connected{error ? ': ' + error : ''} + {/if} + {:else} + AMQP trigger was disabled because of an error: {error} + {/if} +
+ {/snippet} + + {:else if enabled} + + + + + {#snippet text()} +
+ AMQP trigger is connected{!server_id ? ' (shutting down...)' : ''} +
+ {/snippet} + + {/if} +
+ + onToggleMode(path, newMode)} + triggerMode={mode} + includeModalConfig={{ + triggerPath: path, + triggerKind: 'amqp', + runnableConfig: { + path: script_path, + kind: is_flow ? 'flow' : 'script', + retry, + errorHandlerPath: error_handler_path, + errorHandlerArgs: error_handler_args + } + }} + {canWrite} + hideToggleLabels + hideDropdown + /> + +
+ + { + goto(href) + } + }, + ...(canWrite && mode !== 'suspended' + ? [ + { + displayName: 'Suspend job execution', + icon: Pause, + action: () => { + onToggleMode(path, 'suspended') + } + } + ] + : []), + { + displayName: canWrite ? 'Edit' : 'View', + icon: canWrite ? Pen : Eye, + action: () => { + amqpTriggerEditor?.openEdit(path, is_flow) + } + }, + ...(isDeployable('trigger', path, deployUiSettings) + ? [ + { + displayName: 'Deploy to prod/staging', + icon: FileUp, + action: () => { + deploymentDrawer?.openDrawer(path, 'trigger', { + triggers: { + kind: 'amqp' + } + }) + } + } + ] + : []), + { + displayName: 'Audit logs', + icon: Eye, + href: `${base}/audit_logs?resource=${path}` + }, + { + displayName: 'Permissions', + icon: Shield, + action: () => { + shareModal?.openDrawer(path, 'amqp_trigger') + } + }, + { + displayName: 'Delete', + type: 'delete', + icon: Trash, + disabled: !canWrite, + action: async () => { + await AmqpTriggerService.deleteAmqpTrigger({ + workspace: $workspaceStore ?? '', + path + }) + loadTriggers() + } + } + ]} + /> +
+
+
+
edited by {edited_by}
the {displayDate(edited_at)}
+
+ {/each} +
+ {:else} + + {/if} +
+ {#if items && items?.length > 15 && nbDisplayed < items.length} + {nbDisplayed} items out of {items.length} + + {/if} + + + { + loadTriggers() + }} +/> diff --git a/system_prompts/auto-generated/cli/cli-commands.md b/system_prompts/auto-generated/cli/cli-commands.md index 40909bbf3c..8d04a720db 100644 --- a/system_prompts/auto-generated/cli/cli-commands.md +++ b/system_prompts/auto-generated/cli/cli-commands.md @@ -675,12 +675,12 @@ trigger related commands - `--json` - Output as JSON (for piping to jq) - `trigger get ` - get a trigger's details - `--json` - Output as JSON (for piping to jq) - - `--kind ` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email). Recommended for faster lookup + - `--kind ` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email). Recommended for faster lookup - `trigger new ` - create a new trigger locally - - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) - `trigger push ` - push a local trigger spec. This overrides any remote versions. - `trigger set-permissioned-as ` - Set the email (run-as user) for a trigger (requires admin or wm_deployers group) - - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) ### user diff --git a/system_prompts/auto-generated/prompts.ts b/system_prompts/auto-generated/prompts.ts index 8e82c36b74..fb2e40e6ec 100644 --- a/system_prompts/auto-generated/prompts.ts +++ b/system_prompts/auto-generated/prompts.ts @@ -21,7 +21,7 @@ export const SCRIPT_BASE = `# Windmill Script Writing Guide ## Preprocessor Scripts -Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. +Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. The returned object determines the parameter values passed to the flow. e.g., \`{ b: 1, a: 2 }\` calls the flow with \`a = 2\` and \`b = 1\`, assuming the flow has two inputs called \`a\` and \`b\`. @@ -761,7 +761,7 @@ A script joins the pipeline when its source begins with the \`pipeline\` annotat - \`// on \` — declares an execution-DAG **input** (what triggers/feeds this node). \`\` is either: - an **asset URI** (the node runs when that asset is produced upstream): \`ducklake://main/orders\`, \`datatable://main/users\`, \`s3://\`, \`$res:f/folder/my_resource\`, \`volume://name/path\`. - - a **native trigger kind**: \`schedule\`, \`webhook\`, \`email\`, \`kafka\`, \`mqtt\`, \`nats\`, \`postgres\`, \`sqs\`, \`gcp\`, or \`data_upload\` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. + - a **native trigger kind**: \`schedule\`, \`webhook\`, \`email\`, \`kafka\`, \`mqtt\`, \`amqp\`, \`nats\`, \`postgres\`, \`sqs\`, \`gcp\`, or \`data_upload\` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. - **Outputs** are inferred from what the body writes — a \`CREATE TABLE\`, a \`wmill.writeS3File(...)\`, a DuckLake/datatable write. To declare a managed output explicitly, use \`// materialize \`. - Optional badges: \`// partitioned \`, \`// freshness \` (e.g. \`1h\`), \`// tag \`, \`// retry [delay]\`, \`// data_test ...\`. @@ -3394,12 +3394,12 @@ trigger related commands - \`--json\` - Output as JSON (for piping to jq) - \`trigger get \` - get a trigger's details - \`--json\` - Output as JSON (for piping to jq) - - \`--kind \` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email). Recommended for faster lookup + - \`--kind \` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email). Recommended for faster lookup - \`trigger new \` - create a new trigger locally - - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) - \`trigger push \` - push a local trigger spec. This overrides any remote versions. - \`trigger set-permissioned-as \` - Set the email (run-as user) for a trigger (requires admin or wm_deployers group) - - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - \`--kind \` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) ### user diff --git a/system_prompts/auto-generated/schemas/amqp_trigger.schema.yaml b/system_prompts/auto-generated/schemas/amqp_trigger.schema.yaml new file mode 100644 index 0000000000..6d8f661652 --- /dev/null +++ b/system_prompts/auto-generated/schemas/amqp_trigger.schema.yaml @@ -0,0 +1,113 @@ +type: object +properties: + script_path: + type: string + description: Path to the script or flow to execute when triggered + permissioned_as: + type: string + description: The user or group this trigger runs as (permissioned_as) + is_flow: + type: boolean + description: True if script_path points to a flow, false if it points to a script + labels: + type: array + items: + type: string + draft_only: + type: boolean + description: 'True when this row is a per-user draft with no deployed + + trigger at the same path. Set by list endpoints when + + `include_draft_only=true` synthesizes the row from the + + draft. Frontend renders a "Draft" badge. + + ' + is_draft: + type: boolean + description: 'True when the authed user has a per-user draft at this path + + (over a deployed row or a synthesized draft-only row). + + Frontend appends a `*` to the displayed name. + + ' + amqp_resource_path: + type: string + description: Path to the AMQP resource containing broker connection configuration + queue_name: + type: string + description: Name of the queue to consume messages from + exchange: + type: object + properties: + exchange_name: + type: string + description: Name of the exchange to bind the consumed queue to + routing_keys: + type: array + items: + type: string + description: Routing keys used to bind the queue to the exchange + options: + type: object + properties: + declare_queue: + type: boolean + description: Declare the queue (durable) before consuming; when false the + queue is declared passively and must already exist + prefetch_count: + type: integer + format: int32 + minimum: 1 + maximum: 65535 + description: Maximum number of unacknowledged messages the broker delivers + at once (1-65535) + error_handler_path: + type: string + description: Path to a script or flow to run when the triggered job fails + error_handler_args: + type: object + description: The arguments to pass to the script or flow + retry: + type: object + properties: + constant: + type: object + description: Retry with constant delay between attempts + properties: + attempts: + type: integer + description: Number of retry attempts + seconds: + type: integer + description: Seconds to wait between retries + exponential: + type: object + description: Retry with exponential backoff (delay doubles each time) + properties: + attempts: + type: integer + description: Number of retry attempts + multiplier: + type: integer + description: Multiplier for exponential backoff + seconds: + type: integer + minimum: 1 + description: Initial delay in seconds + random_factor: + type: integer + minimum: 0 + maximum: 100 + description: Random jitter percentage (0-100) to avoid thundering herd + retry_if: + $ref: '#/components/schemas/RetryIf' + description: Retry configuration for failed module executions +required: +- script_path +- permissioned_as +- is_flow +- amqp_resource_path +- queue_name diff --git a/system_prompts/auto-generated/script.md b/system_prompts/auto-generated/script.md index 408f1354b7..9d393d2c7f 100644 --- a/system_prompts/auto-generated/script.md +++ b/system_prompts/auto-generated/script.md @@ -19,7 +19,7 @@ ## Preprocessor Scripts -Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. +Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. The returned object determines the parameter values passed to the flow. e.g., `{ b: 1, a: 2 }` calls the flow with `a = 2` and `b = 1`, assuming the flow has two inputs called `a` and `b`. diff --git a/system_prompts/auto-generated/skills/cli-commands/SKILL.md b/system_prompts/auto-generated/skills/cli-commands/SKILL.md index c2d7fe27ef..cba3a1da40 100644 --- a/system_prompts/auto-generated/skills/cli-commands/SKILL.md +++ b/system_prompts/auto-generated/skills/cli-commands/SKILL.md @@ -680,12 +680,12 @@ trigger related commands - `--json` - Output as JSON (for piping to jq) - `trigger get ` - get a trigger's details - `--json` - Output as JSON (for piping to jq) - - `--kind ` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email). Recommended for faster lookup + - `--kind ` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email). Recommended for faster lookup - `trigger new ` - create a new trigger locally - - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) - `trigger push ` - push a local trigger spec. This overrides any remote versions. - `trigger set-permissioned-as ` - Set the email (run-as user) for a trigger (requires admin or wm_deployers group) - - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, sqs, gcp, azure, email) + - `--kind ` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email) ### user diff --git a/system_prompts/base/pipeline-base.md b/system_prompts/base/pipeline-base.md index a0f2e2492a..55c1c53169 100644 --- a/system_prompts/base/pipeline-base.md +++ b/system_prompts/base/pipeline-base.md @@ -8,7 +8,7 @@ A script joins the pipeline when its source begins with the `pipeline` annotatio - `// on ` — declares an execution-DAG **input** (what triggers/feeds this node). `` is either: - an **asset URI** (the node runs when that asset is produced upstream): `ducklake://main/orders`, `datatable://main/users`, `s3://`, `$res:f/folder/my_resource`, `volume://name/path`. - - a **native trigger kind**: `schedule`, `webhook`, `email`, `kafka`, `mqtt`, `nats`, `postgres`, `sqs`, `gcp`, or `data_upload` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. + - a **native trigger kind**: `schedule`, `webhook`, `email`, `kafka`, `mqtt`, `amqp`, `nats`, `postgres`, `sqs`, `gcp`, or `data_upload` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. - **Outputs** are inferred from what the body writes — a `CREATE TABLE`, a `wmill.writeS3File(...)`, a DuckLake/datatable write. To declare a managed output explicitly, use `// materialize `. - Optional badges: `// partitioned `, `// freshness ` (e.g. `1h`), `// tag `, `// retry [delay]`, `// data_test ...`. diff --git a/system_prompts/base/script-base.md b/system_prompts/base/script-base.md index 842667928a..71ffada86e 100644 --- a/system_prompts/base/script-base.md +++ b/system_prompts/base/script-base.md @@ -19,7 +19,7 @@ ## Preprocessor Scripts -Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. +Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean. The returned object determines the parameter values passed to the flow. e.g., `{ b: 1, a: 2 }` calls the flow with `a = 2` and `b = 1`, assuming the flow has two inputs called `a` and `b`. diff --git a/system_prompts/generate.py b/system_prompts/generate.py index beac340499..d315e3a97b 100644 --- a/system_prompts/generate.py +++ b/system_prompts/generate.py @@ -828,6 +828,7 @@ WORKSPACE_TOOL_ZOD_SCHEMAS = [ ('NewNatsTrigger', 'natsTriggerRequestSchema'), ('NewPostgresTrigger', 'postgresTriggerRequestSchema'), ('NewMqttTrigger', 'mqttTriggerRequestSchema'), + ('NewAmqpTrigger', 'amqpTriggerRequestSchema'), ('NewSqsTrigger', 'sqsTriggerRequestSchema'), ('GcpTriggerData', 'gcpTriggerRequestSchema'), ('AzureTriggerData', 'azureTriggerRequestSchema'), @@ -842,6 +843,7 @@ WORKSPACE_TOOL_TRIGGER_SCHEMAS = [ ('nats', 'natsTriggerRequestSchema'), ('postgres', 'postgresTriggerRequestSchema'), ('mqtt', 'mqttTriggerRequestSchema'), + ('amqp', 'amqpTriggerRequestSchema'), ('sqs', 'sqsTriggerRequestSchema'), ('gcp', 'gcpTriggerRequestSchema'), ('azure', 'azureTriggerRequestSchema'), @@ -1582,6 +1584,7 @@ SKILL_DEFINITIONS = [ ('NatsTrigger', 'nats_trigger'), ('PostgresTrigger', 'postgres_trigger'), ('MqttTrigger', 'mqtt_trigger'), + ('AmqpTrigger', 'amqp_trigger'), ('SqsTrigger', 'sqs_trigger'), ('GcpTrigger', 'gcp_trigger'), ('AzureTrigger', 'azure_trigger'), @@ -2478,6 +2481,7 @@ def main(): 'NatsTrigger', 'NewNatsTrigger', 'PostgresTrigger', 'NewPostgresTrigger', 'MqttTrigger', 'NewMqttTrigger', + 'AmqpTrigger', 'NewAmqpTrigger', 'SqsTrigger', 'NewSqsTrigger', 'GcpTrigger', 'AzureTrigger', diff --git a/system_prompts/utils.py b/system_prompts/utils.py index 8ae4df291c..9666b6cf0c 100644 --- a/system_prompts/utils.py +++ b/system_prompts/utils.py @@ -46,6 +46,7 @@ SCHEMA_MAPPINGS = { ('NatsTrigger', 'nats_trigger'), ('PostgresTrigger', 'postgres_trigger'), ('MqttTrigger', 'mqtt_trigger'), + ('AmqpTrigger', 'amqp_trigger'), ('SqsTrigger', 'sqs_trigger'), ('GcpTrigger', 'gcp_trigger'), ('AzureTrigger', 'azure_trigger'),