mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-10 08:07:03 +00:00
Ruben FiszelandClaude Opus 4.8
65e504146d
* feat(frontend): add data-pipeline run recorder and interactive player Adds a recorder/player for data pipelines, mirroring the existing flow and script recorders. Arm "Record" on a pipeline, run it, and the resulting cascade is captured into a downloadable JSON that the /replay player can rerun fully offline. Because a pipeline run is a cascade of independent jobs (not a single root SSE job like flows), the recording captures three things: the resolved asset graph, the per-node cascade status timeline (from the orchestrator's onUpdate), and each node's job stream (opened via getupdate_sse on launch). The player renders the graph read-only, animates the recorded node transitions in real time, and lets you click any node to inspect its recorded args, logs and result — reusing the same JobLoader replay path the flow/script players use (setActiveReplay + isReplay gating), so no network calls are made during replay. - recording/types.ts: PipelineRecording, PipelineTimelineFrame, RecordedNodeState - recording/pipelineRecording.svelte.ts: createPipelineRecording() store - recording/PipelineRecordingReplay.svelte: the player component - replay/+page.svelte: dispatch type === 'pipeline' - pipeline/[folder]/+page.svelte: Record toggle + Download recording; capture the whole-pipeline / bounded cascade run Fixes WIN-2156 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): capture DuckLake/datatable data samples in pipeline recordings Follow-up to the pipeline recorder/player: asset nodes are now inspectable offline in the player, showing what each table held after the recorded run. At record finalization, for each ducklake/datatable asset in the pipeline the recorder samples the table (up to 100 rows + columns + row count) reusing the exact live-preview query path (loadAllTablesMetaData + getRows), so a replayed sample matches what the asset-detail pane would have shown. Captures are best-effort and per-asset — a missing/unconfigured table is stored as an error marker, never thrown, so the recording still completes. The player renders the sample as a read-only typed grid when an asset node is clicked (script nodes keep their logs/result/args detail). - recording/types.ts: PipelineAssetSample + assetSamples on PipelineRecording - recording/pipelineAssetSample.ts: capturePipelineAssetSample() helper - recording/pipelineRecording.svelte.ts: recordAssetSample() + assetSamples - recording/PipelineRecordingReplay.svelte: asset-node data-sample panel - pipeline/[folder]/+page.svelte: sample each asset in finalizePipelineRecording Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * recorder * feat(hub): record data pipelines in deploy-to-hub with interactive player Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hub): match editor cascade timeout, warn on cycles, reset badge on re-run Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): address review — finalize race, stale replay timers, /replay redirect, bounded sampling, jobs validation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): structural recording validation, guard-clear + SSE cleanup on throw paths Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): validate nested graph arrays and timeline frame statuses Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hub): scope recording to bundle membership, fail cyclic runs, validate recording elements Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hub): prune recorded graph + asset samples to bundle membership Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): validate graph.triggers array and per-job initial_job/events shapes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): guard non-object payloads, event elements, and asset-sample/code maps Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): render error boundary + validate trigger_kind and non-empty sample error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): validate event.data and recorded-job shapes for all replay types Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): make the replay event timer crash-proof against malformed events Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): await replay completion and boundary-wrap all three players Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(recording): guard flow Play handler, cap ?src= download size, trim comment Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windmill Backend
This folder holds all backend components, the src/ folder only contains files used to build the "root" binary.
Components
| name | description |
|---|---|
| windmill-api | The API server, exposing functionality to other components and the frontend |
| windmill-audit | Contains audit functionality, allowing different components to record important actions |
| windmill-common | Common code shared by all crates |
| windmill-queue | Contains job & flow queuing functionality, commonly written to by the API server and read from by workers |
| windmill-worker | The worker. Used to process and execute flows & jobs. |
| parsers | Contains code to parse signatures in different langauges. |
Compile sqlx for offline ci
cargo sqlx prepare --workspace -- --bin windmill --features enterprise