starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-09-05 08:02:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

perf(backend): gate asset producer-change event on write-set changes (#9672)

Browse Source 
* perf(backend): only emit asset producer-change event on write-set changes

Data Pipelines (#9193) made every script deploy emit a
`notify_asset_producer_change` event: `clear_static_asset_usage` inserted
into `notify_event` unconditionally on every clear, and the per-asset
insert path emitted nothing. So a plain script with no assets — the
overwhelming majority of deploys — wrote a `notify_event` row that made
every worker drop its `ASSET_PRODUCER_WRITES_CACHE` entry instance-wide,
needlessly thrashing the cache the feature added.

That cache only tracks script rows with write access (`usage_access_type
IN ('w','rw')`), so a deploy changes it only when the script gains or
loses a write producer. Gate the event on exactly that:

- `clear_static_asset_usage` / `clear_static_asset_usage_by_script_hash`
  emit only when the delete removed a 'w'/'rw' row (via `RETURNING`).
- `insert_static_asset_usage` emits only when it actually inserts a
  'w'/'rw' script row (no-op `ON CONFLICT`, read-only, and flow usage
  stay silent).

Plain, read-only, and flow deploys now emit nothing; producer-changing
deploys still invalidate, atomically and visible-only-on-commit as
before. Adds a test asserting the emit/no-emit matrix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(backend): dedup producer-change notify on write-asset redeploys

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(backend): derive replace write-set from persisted rows; document auth contract

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(backend): correct replace_static_asset_usage call-site comment

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ruben Fiszel
2026-06-19 13:49:32 +00:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 6833a554ae
commit 5970510cf6
9 changed files with 759 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/windmill-api-scripts/src/scripts.rs
+12 -7
View File
@@ -45,8 +45,8 @@ use windmill_dep_map::scoped_dependency_map::ScopedDependencyMap;
use windmill_common::{
assets::{
clear_script_triggers, clear_static_asset_usage, clear_static_asset_usage_by_script_hash,
insert_script_trigger, insert_static_asset_usage, parse_duration_secs,
parse_pipeline_annotations, trigger_spec_to_row, AssetUsageKind, TriggerSpec,
insert_script_trigger, parse_duration_secs, parse_pipeline_annotations,
replace_static_asset_usage, trigger_spec_to_row, AssetUsageKind, TriggerSpec,
},
error::{self, to_anyhow},
min_version::{MIN_VERSION_SUPPORTS_DEBOUNCING, MIN_VERSION_SUPPORTS_DEBOUNCING_V2},
@@ -1609,11 +1609,16 @@ async fn create_script_internal<'c>(
);
}
clear_static_asset_usage(&mut *tx, &w_id, &script_path, AssetUsageKind::Script).await?;
for asset in effective_assets.as_ref().into_iter().flatten() {
insert_static_asset_usage(&mut *tx, &w_id, &asset, &ns.path, AssetUsageKind::Script)
.await?;
}
// Clear + reinsert this script's producer rows at script_path (== ns.path),
// invalidating the producer-writes cache once iff the write-producer set
// changed (see replace_static_asset_usage).
replace_static_asset_usage(
&mut tx,
&w_id,
&script_path,
effective_assets.as_deref().unwrap_or(&[]),
)
.await?;
// Pipeline trigger edges: wipe-and-reinsert per deploy so removing an
// `// on ...` annotation drops the edge. Only Asset / Schedule produce