mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-07 00:01:49 +00:00
* feat(fork): merge a fork deletion on evidence, not on the counters `workspace_diff.ahead`/`.behind` record that a write happened on a side, not what it was or who made it. That leaves one row shape undecidable: an item the parent has and the fork does not can mean the parent added it, the fork deleted it, or a git-sync pull reverted a deploy that had just brought it in. #10467 kept every such row out of the merge direction, which killed the phantom but also dropped the only way to propagate a fork-side deletion and left a rename's old path behind in the parent. Record the evidence instead: - `workspace_diff` gains, per side, the last event's kind (`write` / `delete` / `rename_from`) and origin (`authored` / `sync`). Rows written before the migration have neither and keep #10467's behavior. - The kind is probed from whether the path still holds an item once the write has committed; an item kind the probe doesn't map records no evidence rather than a deletion. Create and update are not split — nothing at that point tells them apart for every kind, and the comparison already recomputes existence per side. - The origin comes from an `X-Windmill-Deploy-Origin` header the API scopes into a task-local for the request. It is the load-bearing half: recording `delete` alone would read a git-sync revert as a fork deletion and reproduce the original bug. Two clients set it — `wmill sync push` (which the git-sync auto-pull runs inside a job) and the compare page's parent→fork "Update fork". Merging the other way stays authored so a deletion keeps propagating up a fork chain. - The merge direction admits a parent-only row only when the fork's last event was an authored delete or rename-away. Such a row stays opt-in, never bulk-selected, and reads "Removes in <parent>"; the update direction keeps offering it back as "New". A fork deletion and a rename now merge into the parent, a rename leaves no duplicate behind, and a fork the parent also edited surfaces in both directions instead of the parent silently winning. Fixes WIN-2289 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): address review — detached tallies, enum wire values, doc duplication Codex P1: a dependency job tallies its deploy whenever it happens to finish, and the event kind is probed from the state at that moment. If anything removed the path in between (a git-sync revert), the stale tally read that deletion as its own and filed it as authored — handing the merge exactly the removal this is meant to withhold. `tally_deployed_object_changes` now takes `Option<DeployOrigin>`; `None` bumps the counter and leaves the evidence columns as the last vouching tally left them, and the worker path passes it. Covered by extending the removal-origin test: a detached tally after the sync archive must not disturb `(delete, sync)`. Also from review: - `fork_removed_it` compares through `DeployOrigin::as_str()` / `DeployEventKind::as_str()` rather than repeating their wire values, so a renamed variant can't silently make the predicate always false. - `deploy_origin`'s module doc no longer claims `sync` is inert: it cannot make the merge propose a removal, but it does drop a row out of both sides of the `all_ahead_items_visible` comparison. - `WorkspaceDiffRow` says why only the fork half of the evidence is consumed. - The delete-vs-revert rationale is stated once (the migration) instead of restated in eight files. - `PATH_KEYED_TABLES` is swept by a test: its query is built at runtime, so a wrong table name is not a compile error and would only surface as a failed tally for that trigger kind in a fork. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): let only a request task vouch for a deploy event Round 2 found the first fix incomplete. Detaching only the failed/cancelled dependency path left the common route untouched: a dependency job that succeeds calls `handle_deployment_metadata` from the worker, where `deploy_origin::current()` read as `Authored`. A sync archiving the script while its lock generation was pending then had its deletion probed on completion and refiled as authored — the same fabricated removal, on the path most deploys actually take. `current()` now returns `Option`, `Some` only inside the request scope the API always enters. Having no scope means "not the task that served this write", which is true of every worker-side call and needs no marking at the call site. The integration test drives the real `handle_deployment_metadata` off a request task instead of the tally directly, and fails without this. Two more from the same round: - The script dependency handler passed no `renamed_from`, unlike the flow and app handlers next to it. A lock-generating create has no earlier tally, so that was the only chance for the path a rename vacated to be recorded at all — renames of Python/TS scripts left the old path in the parent, which the bash-only manual check missed. - The tally now drops a `renamed_from` equal to the path itself. Callers pass the previous path whether or not the deploy moved the item, so an unfiltered one both counted the path twice and stamped it `rename_from` when nothing was renamed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): carry a deploy's origin into the dependency job it queues Round 3 caught the previous fix cutting too deep. Refusing a detached tally any claim also refused its rename evidence, and a lock-generating deploy has no other tally — so the `renamed_from` added alongside it was inert, and a renamed flow, app or Python script still left its old path in the parent with nothing to merge. Flows and apps always generate, so renames worked essentially nowhere. The two capabilities are now separate. `TallyEvidence` says whether the tallying task served the write (`Served`, may probe what the path holds now) or is reporting one that committed earlier (`Deferred`, may not), and each column is written only from a source that answers for it. The origin itself is a fact of the deploy either way, so the request stamps it into the dependency job's args and the worker re-enters the scope with it — the last place that knows it handing it to the only tally that will run. Also from round 3: `WorkspaceDiffRow`'s event fields skip serializing `None` rather than emitting `null`, matching what the schema declares (OpenAPI 3.0.3 ignores a `description` sibling of `$ref`, so those moved onto the shared schemas). Verified against a live worker: renaming a flow in a fork records `(rename_from, authored)` on the vacated path and the merge offers its removal, while the deployed path claims nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): mark the CLI's parent-to-fork merge as sync `wmill workspace merge --direction to-fork` is the CLI's "Update fork" and deletes items in the fork, but without the marker the compare page sets. Its deletions were recorded as authored fork decisions, so once the parent recreated such a path the merge would offer deleting it there. Also from review: an unrecognized deploy-origin arg now reads as no evidence rather than as authored — strict where a request header is lenient, since an unmarked request really is authored but an unreadable stored value is skew. Reading the arg moved next to `stamp_origin_arg`, the half that writes it, so the round trip a lock-generating deploy depends on is covered by one test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the imports the shared arg reader made unused CI compiles with `-D warnings`, so this was four red Backend jobs rather than a lint. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): stop a stale deferred rename from restating a removed path Nothing orders these events. A tally that served the write made its claim inside its own commit, but a deferred one reports a write that landed at an unknown remove. So a lock-generating rename whose dependency job finished after a sync had removed the vacated path could overwrite `(delete, sync)` with `(rename_from, authored)` — the path is gone either way, so the merge would then offer removing it from the parent on the strength of the older event. A deferred claim now only writes where the side has none, which is the case it exists for: a vacated path that nothing else has spoken for. The regression asserts the ordering directly, and fails without the guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): record a rename's vacated path from the request that made it The deferred mechanism could not be made correct, as round 7 showed: its guard protected an existing row, but that row is deleted as soon as the two workspaces agree on the path — so a rename job finishing after the reconciliation inserted fresh, and the stale claim reappeared against whatever the parent later recreated there. Ordering cannot be recovered outside the row, because the row is disposable. So the vacated path is now recorded by the request, which is inside its own commit and whose row shares the counter's lifetime. A deploy that hands its metadata to a dependency job — every flow and app, and any script needing a lock — calls `tally_rename_vacated_path` once its transaction has committed; scripts reach it through the post-commit hook they already had, which grew a second variant rather than new plumbing. That lets the whole deferred apparatus go: `TallyEvidence`, the origin job arg and its round trip. `deploy_origin::current` is `Some` only inside a request scope again, and `handle_deployment_metadata` hands `renamed_from` to the tally only when it can answer for it — git-sync still gets it either way, so the rename keeps naming itself in the commit message. The vacated path's kind now reads `delete` rather than `rename_from` for these deploys, since it is probed rather than declared. The merge treats the two alike; only the row's tooltip is less specific. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(fork): cover raw-app renames, and stop firing CI before the lock exists Two things the vacated-path call broke or missed: - `create_script` reads its third return value as "no lock generation needed" to decide whether the script is runnable now, and the new `VacatedPath` variant made that true for renames that do generate. Those fired dependent CI tests from the API against a version with no lockfile, and again from the dependency job. The variant now decides it explicitly. - Raw apps rename through `update_app_raw`, a separate route into `update_app_internal`, which the new call had not been attached to. Both routes now go through one helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(fork): assert the kind only an inline rename can record `rename_from` is what a deploy says when it knows it moved the item, which only the path that reports both halves from its own request can. Nothing pinned it, and that is the side the vacated-path change touched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to a45bec03922d305aad5893ed354dc029c7f97bb4 This commit updates the EE repository reference after PR #709 was merged in windmill-ee-private. Previous ee-repo-ref: 62f494b2a51de0dfc0cfa0c3530ff19a1d32667c New ee-repo-ref: a45bec03922d305aad5893ed354dc029c7f97bb4 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
131 lines
5.1 KiB
Rust
131 lines
5.1 KiB
Rust
/*
|
|
* Author: Ruben Fiszel
|
|
* Copyright: Windmill Labs, Inc 2022
|
|
* This file and its contents are licensed under the AGPLv3 License.
|
|
* Please see the included NOTICE for copyright information and
|
|
* LICENSE-AGPL for a copy of the license.
|
|
*/
|
|
|
|
//! Where a deploy event came from, for the fork tally (`workspace_diff` — see the
|
|
//! `workspace_diff_last_event` migration for what the tally does with it).
|
|
//!
|
|
//! A client that applies a state computed elsewhere (a git-sync pull, a
|
|
//! workspace-to-workspace deploy) says so with [`DEPLOY_ORIGIN_HEADER`]; the API
|
|
//! scopes it for the request and the tally records it alongside the counters.
|
|
//! Anything unmarked is [`DeployOrigin::Authored`].
|
|
//!
|
|
//! The flag is evidence for a UI offer, never authority. Claiming `sync` cannot
|
|
//! make the merge propose a removal — that needs `authored` — but it is not inert
|
|
//! either: `compare_workspaces` excludes a parent-only `sync` row from both sides
|
|
//! of its `all_ahead_items_visible` comparison, so stamping `sync` on a row the
|
|
//! caller cannot see suppresses the "changes not visible to your user" warning
|
|
//! that `authored` would raise. That flag is a visibility guarantee rather than an
|
|
//! authorization boundary (each item is re-authorized at deploy time), and this
|
|
//! matches how an untallied row already behaves, so it is bounded — but do not
|
|
//! read the header as unable to affect anything.
|
|
|
|
/// Request header a sync client sets to mark its writes as applied rather than
|
|
/// authored. Only `sync` is meaningful; any other value reads as authored.
|
|
pub const DEPLOY_ORIGIN_HEADER: &str = "x-windmill-deploy-origin";
|
|
|
|
/// Who caused a deploy event at a path.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum DeployOrigin {
|
|
/// Written in this workspace by whoever made the request.
|
|
Authored,
|
|
/// Applied to this workspace by a sync (git-sync pull, cross-workspace deploy).
|
|
Sync,
|
|
}
|
|
|
|
impl DeployOrigin {
|
|
pub fn as_str(&self) -> &'static str {
|
|
match self {
|
|
DeployOrigin::Authored => "authored",
|
|
DeployOrigin::Sync => "sync",
|
|
}
|
|
}
|
|
|
|
pub fn from_header_value(value: &str) -> Self {
|
|
if value.trim().eq_ignore_ascii_case("sync") {
|
|
DeployOrigin::Sync
|
|
} else {
|
|
DeployOrigin::Authored
|
|
}
|
|
}
|
|
}
|
|
|
|
/// What a deploy event did to the path it is recorded against.
|
|
///
|
|
/// Create and update are one value: at the point the tally runs, the write has
|
|
/// already committed and no item kind carries a signal that separates them.
|
|
/// Existence *after* the event is what the merge direction needs, and the
|
|
/// comparison recomputes that per side (`exists_in_source` / `exists_in_fork`).
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum DeployEventKind {
|
|
/// The path holds an item after the event.
|
|
Write,
|
|
/// The path holds no item after the event.
|
|
Delete,
|
|
/// The path was vacated by a rename to another path.
|
|
RenameFrom,
|
|
}
|
|
|
|
impl DeployEventKind {
|
|
pub fn as_str(&self) -> &'static str {
|
|
match self {
|
|
DeployEventKind::Write => "write",
|
|
DeployEventKind::Delete => "delete",
|
|
DeployEventKind::RenameFrom => "rename_from",
|
|
}
|
|
}
|
|
}
|
|
|
|
tokio::task_local! {
|
|
static REQUEST_DEPLOY_ORIGIN: DeployOrigin;
|
|
}
|
|
|
|
/// Run `f` with `origin` as the origin of every deploy event it causes. Entered
|
|
/// for every request, marked or not, so that being in scope at all means "this
|
|
/// task is serving the write it is about to report" — see [`current`].
|
|
///
|
|
/// Task-locals do not cross `tokio::spawn`, and the tally is spawned: read
|
|
/// [`current`] on the request task and carry the value into the spawned future.
|
|
pub async fn scope<F: std::future::Future>(origin: DeployOrigin, f: F) -> F::Output {
|
|
REQUEST_DEPLOY_ORIGIN.scope(origin, f).await
|
|
}
|
|
|
|
/// The origin in scope, or `None` where there is no scope to read.
|
|
///
|
|
/// `None` is what a worker gets. A dependency job reports a deploy that
|
|
/// committed before the job was even queued and finishes whenever lock
|
|
/// generation finishes, so its reading of the workspace describes whoever wrote
|
|
/// last, not its own event. Nothing outside a request can answer for that, so
|
|
/// nothing outside a request is asked to.
|
|
pub fn current() -> Option<DeployOrigin> {
|
|
REQUEST_DEPLOY_ORIGIN.try_with(|origin| *origin).ok()
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// The whole worker side of the tally rests on this: a deploy reported from
|
|
/// anywhere but the task that served it records no evidence, without each
|
|
/// call site remembering to say so.
|
|
#[tokio::test]
|
|
async fn origin_is_absent_outside_a_request() {
|
|
assert_eq!(current(), None);
|
|
assert_eq!(
|
|
scope(DeployOrigin::Authored, async { current() }).await,
|
|
Some(DeployOrigin::Authored)
|
|
);
|
|
// Not through a spawn, which is why `handle_deployment_metadata` reads it
|
|
// before spawning the tally rather than inside.
|
|
let spawned = scope(DeployOrigin::Sync, async {
|
|
tokio::spawn(async { current() }).await.unwrap()
|
|
})
|
|
.await;
|
|
assert_eq!(spawned, None);
|
|
}
|
|
}
|