Files
windmill/cli/test/case_insensitive_collisions_unit.test.ts
Ruben Fiszel c258928ab6 fix(cli): reconcile case-only path drift during sync on case-insensitive filesystems (WIN-2020) (#9485)
* fix(cli): reconcile case-only path drift during sync on case-insensitive filesystems

Windmill paths are case-sensitive, but Windows (and the default macOS
setup) use case-insensitive filesystems. The real-world failure behind
WIN-2020 is not a user authoring both f/Caps and f/caps — it is a single
capitalized folder whose on-disk casing silently drifts (Windows stores
and reports whatever case the directory was first created with,
regardless of the server's path). The diff then sees the drifted local
path as a brand-new item and emits a destructive "delete f/Caps +
add f/caps" pair, so a capitalized folder appears to vanish and a
lowercase clone shows up out of nowhere — and a push can clobber the
real server item.

Fix: on a case-insensitive filesystem, reconcile case-only drift before
diffing. The server's path casing is authoritative, so compareDynFSElement
now rewrites local keys that differ from a remote key only by case to the
server's casing (canonicalizeCaseInsensitiveKeys), making the diff treat
them as the same item. Case-insensitivity is auto-detected by probing the
sync directory, with a WMILL_CASE_INSENSITIVE_FS=true/false override to
force Windows behaviour (or emulate it for tests / cross-platform repos)
on any host. Reconciled paths are summarized in a single info line.

Genuinely unrepresentable collisions — two DISTINCT server paths that
differ only by case — cannot be canonicalized to one target; those are
detected and warned about on every platform so a case-sensitive-Linux
author learns their tree won't round-trip for a Windows/macOS teammate.

Tests:
- Pure unit tests for findCaseInsensitiveCollisions,
  canonicalizeCaseInsensitiveKeys and summarizeCaseRewrites (platform
  independent).
- An end-to-end drift test that runs on BOTH CI jobs: on the Windows
  runner it exercises the real case-insensitive NTFS + auto-probe; on
  Linux it reproduces the drift via rename, asserts the destructive
  phantom appears without the fix, and asserts a clean no-op push with
  the fix forced on.

Fixes WIN-2020

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): canonicalize local-only descendants of drifted folders; dedupe nested case collisions

Address two review findings on the WIN-2020 case-insensitive sync fix:

P1 (correctness): canonicalizeCaseInsensitiveKeys previously only rewrote
local keys with an exact full-path remote match. A brand-new local file
under a drifted folder (e.g. adding f/caps/New.ts when the server has
f/Caps but no f/caps/New.ts) had no exact match, so it kept its lowercase
casing and push uploaded it as-is — recreating f/caps beside f/Caps and
reintroducing the very collision the fix prevents. Canonicalization is now
segment-by-segment against a trie of remote paths, so local-only
descendants inherit the longest unambiguous server folder casing. A segment
is only adopted when the server casing is unambiguous; at the first
ambiguous/unknown segment the remainder keeps local casing. The original
key's separator style is preserved so rewritten keys still round-trip.

P2 (nit): findCaseInsensitiveCollisions reported the folder group AND a
nested per-file group when case-variant folders held same-named files,
inflating the "Found N path(s)" count. It now reports only the shallowest
clash (drops a group whose ancestor prefix is itself a collision).

Tests: add unit coverage for the new-file-under-drifted-folder rewrite, the
stop-at-first-unguided-segment behavior, and shallowest-only collision
reporting; extend the e2e drift test to assert a new item added under the
drifted folder is pushed under the server's folder casing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 08:09:04 +00:00

233 lines
8.8 KiB
TypeScript

import { expect, test } from "bun:test";
import {
findCaseInsensitiveCollisions,
canonicalizeCaseInsensitiveKeys,
summarizeCaseRewrites,
} from "../src/commands/sync/sync.ts";
// =============================================================================
// Case-insensitive sync handling (WIN-2020)
//
// Windmill paths are case-sensitive, but Windows and the default macOS setup
// use case-insensitive filesystems. The real-world failure is NOT a user
// deliberately authoring both f/Caps and f/caps — it is a single capitalized
// folder whose on-disk casing drifts (Windows reports the case the directory
// was first created with), so the diff sees a brand-new lowercase path and a
// destructive delete of the real one. These tests pin:
// 1. findCaseInsensitiveCollisions — warn about genuinely unrepresentable
// server-side collisions (two distinct remote paths differing only by
// case).
// 2. canonicalizeCaseInsensitiveKeys — the fix: rewrite drifted local keys
// to the server's casing so no phantom delete+add is produced.
// =============================================================================
function normalize(groups: string[][]): string[][] {
return groups
.map((g) => [...g])
.sort((a, b) => a.join().localeCompare(b.join()));
}
// ---------------------------------------------------------------------------
// findCaseInsensitiveCollisions
// ---------------------------------------------------------------------------
test("collision detection: sibling folders differing only by case", () => {
const collisions = findCaseInsensitiveCollisions([
"f/Caps/a.script.ts",
"f/caps/b.script.ts",
]);
expect(normalize(collisions)).toEqual([["f/Caps", "f/caps"]]);
});
test("collision detection: leaf files differing only by case", () => {
const collisions = findCaseInsensitiveCollisions([
"f/team/Report.script.ts",
"f/team/report.script.ts",
]);
expect(normalize(collisions)).toEqual([
["f/team/Report.script.ts", "f/team/report.script.ts"],
]);
});
test("collision detection: none for case-consistent distinct paths", () => {
const collisions = findCaseInsensitiveCollisions([
"f/caps/foo.script.ts",
"f/caps/bar.script.ts",
"f/other/baz.script.ts",
]);
expect(collisions).toEqual([]);
});
test("collision detection: normalizes mixed forward/back slashes", () => {
const collisions = findCaseInsensitiveCollisions([
"f\\Caps\\a.script.ts",
"f/caps/b.script.ts",
]);
expect(normalize(collisions)).toEqual([["f/Caps", "f/caps"]]);
});
test("collision detection: reports only the shallowest clash for same-named leaves", () => {
// Two case-variant folders that ALSO hold a same-named file must report the
// single folder clash, not the folder group plus a nested per-file group.
const collisions = findCaseInsensitiveCollisions([
"f/Caps/main.script.ts",
"f/caps/main.script.ts",
]);
expect(normalize(collisions)).toEqual([["f/Caps", "f/caps"]]);
});
test("collision detection: groups three distinct casings together", () => {
const collisions = findCaseInsensitiveCollisions([
"f/caps/a.script.ts",
"f/CAPS/b.script.ts",
"f/Caps/c.script.ts",
]);
expect(normalize(collisions)).toEqual([["f/CAPS", "f/Caps", "f/caps"]]);
});
// ---------------------------------------------------------------------------
// canonicalizeCaseInsensitiveKeys (the WIN-2020 fix)
// ---------------------------------------------------------------------------
test("canonicalize: drifted local folder casing adopts the server casing", () => {
// Server (remote) is authoritative: f/Caps. The local tree drifted to
// f/caps on a case-insensitive FS.
const remote = {
"f/Caps/x.script.ts": "content",
"f/Caps/x.script.yaml": "meta",
};
const local = {
"f/caps/x.script.ts": "content",
"f/caps/x.script.yaml": "meta",
};
const { map, ambiguous, rewritten } = canonicalizeCaseInsensitiveKeys(
local,
remote,
);
// Local keys are rewritten to the server casing, so a subsequent exact-key
// diff sees identical paths — no phantom delete+add.
expect(Object.keys(map).sort()).toEqual([
"f/Caps/x.script.ts",
"f/Caps/x.script.yaml",
]);
expect(ambiguous).toEqual([]);
expect(rewritten).toEqual([
{ from: "f/caps/x.script.ts", to: "f/Caps/x.script.ts" },
{ from: "f/caps/x.script.yaml", to: "f/Caps/x.script.yaml" },
]);
});
test("canonicalize: preserves content values while rewriting keys", () => {
const remote = { "f/MyFolder/Script.script.ts": "remote" };
const local = { "f/myfolder/Script.script.ts": "LOCAL EDIT" };
const { map } = canonicalizeCaseInsensitiveKeys(local, remote);
expect(map["f/MyFolder/Script.script.ts"]).toEqual("LOCAL EDIT");
expect(map["f/myfolder/Script.script.ts"]).toBeUndefined();
});
test("canonicalize: leaves keys with no case-insensitive remote match", () => {
const remote = { "f/Caps/x.script.ts": "a" };
const local = {
"f/Caps/x.script.ts": "a",
"f/brand_new/y.script.ts": "b", // genuinely local-only add
};
const { map, rewritten } = canonicalizeCaseInsensitiveKeys(local, remote);
expect(rewritten).toEqual([]);
expect(Object.keys(map).sort()).toEqual([
"f/Caps/x.script.ts",
"f/brand_new/y.script.ts",
]);
});
test("canonicalize: does NOT rewrite when the server casing is ambiguous", () => {
// The server itself holds two paths differing only by case — we must not
// silently pick one. Leave the local key untouched and report the ambiguity.
const remote = {
"f/Caps/x.script.ts": "a",
"f/caps/x.script.ts": "b",
};
const local = { "f/CAPS/x.script.ts": "local" };
const { map, ambiguous, rewritten } = canonicalizeCaseInsensitiveKeys(
local,
remote,
);
expect(rewritten).toEqual([]);
expect(Object.keys(map)).toEqual(["f/CAPS/x.script.ts"]);
// Reported as the shallowest (folder) clash, not the nested per-file group.
expect(normalize(ambiguous)).toEqual([["f/Caps", "f/caps"]]);
});
test("canonicalize: new local file under a drifted folder adopts the server folder casing", () => {
// Regression for the P1 review finding: a brand-new local file has no exact
// remote match, but it still lives under a folder whose casing drifted. It
// must inherit the server's folder casing (f/Caps), otherwise push would
// create f/caps/New beside f/Caps/* and reintroduce the case-only collision.
const remote = { "f/Caps/Existing.script.ts": "remote" };
const local = {
"f/caps/Existing.script.ts": "remote", // drifted, existing
"f/caps/New.script.ts": "brand new", // drifted folder, local-only file
};
const { map, rewritten } = canonicalizeCaseInsensitiveKeys(local, remote);
expect(Object.keys(map).sort()).toEqual([
"f/Caps/Existing.script.ts",
"f/Caps/New.script.ts",
]);
expect(map["f/Caps/New.script.ts"]).toEqual("brand new");
expect(rewritten).toContainEqual({
from: "f/caps/New.script.ts",
to: "f/Caps/New.script.ts",
});
});
test("canonicalize: stops at the first segment with no server guidance", () => {
// Only the folder prefix that exists on the server is canonicalized; deeper
// local-only directories keep their own casing.
const remote = { "f/Caps/x.script.ts": "a" };
const local = { "f/caps/SubDir/y.script.ts": "b" };
const { map } = canonicalizeCaseInsensitiveKeys(local, remote);
expect(Object.keys(map)).toEqual(["f/Caps/SubDir/y.script.ts"]);
});
test("canonicalize: identical casing is a no-op", () => {
const remote = { "f/Caps/x.script.ts": "a" };
const local = { "f/Caps/x.script.ts": "a" };
const { map, rewritten, ambiguous } = canonicalizeCaseInsensitiveKeys(
local,
remote,
);
expect(rewritten).toEqual([]);
expect(ambiguous).toEqual([]);
expect(map).toEqual({ "f/Caps/x.script.ts": "a" });
});
// ---------------------------------------------------------------------------
// summarizeCaseRewrites
// ---------------------------------------------------------------------------
test("summarize: collapses per-file rewrites into one folder entry", () => {
const summary = summarizeCaseRewrites([
{ from: "f/caps/x.script.ts", to: "f/Caps/x.script.ts" },
{ from: "f/caps/x.script.yaml", to: "f/Caps/x.script.yaml" },
{ from: "f/caps/y.script.ts", to: "f/Caps/y.script.ts" },
]);
expect(summary).toEqual(["f/caps -> f/Caps"]);
});
test("summarize: reports a leaf-file casing change at file granularity", () => {
const summary = summarizeCaseRewrites([
{ from: "f/team/report.script.ts", to: "f/team/Report.script.ts" },
]);
expect(summary).toEqual([
"f/team/report.script.ts -> f/team/Report.script.ts",
]);
});
test("summarize: reports independent folder drifts separately", () => {
const summary = summarizeCaseRewrites([
{ from: "f/caps/x.script.ts", to: "f/Caps/x.script.ts" },
{ from: "u/alice/y.script.ts", to: "u/Alice/y.script.ts" },
]);
expect(summary.sort()).toEqual(["f/caps -> f/Caps", "u/alice -> u/Alice"]);
});