mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 00:02:31 +00:00
build(xterm): restore the patch regeneration harness and gate it in CI (#15223)
* build(xterm): restore the patch regeneration harness and gate it in CI docs/reference/ime-architecture.md says "Never hand-edit the bundles in the patch" and links to docs/reference/xterm-patch-regeneration.md. That doc does not exist, and neither does the harness it describes. Both landed in29117bf776and were deleted by17cfc968cf, a revert of the composition-ownership change, which swept up a build tool and a CI gate as collateral. The rule survived; its enforcement did not. Every xterm patch since has had to hand-edit minified bundles to comply with the surrounding architecture, because everything resolves to lib/xterm.mjs at runtime and under vitest, so a src-only edit is inert. The shipped bundles were therefore not the output of any build, and this restores them to build output. Comparing identifier multisets against a pristine build of the pinned commit finds hand-written names a minifier never emits ($rl, $hp, $tid), const in an otherwise let-only esbuild bundle, !! where the source reads Boolean(), an escaped LRM where esbuild emits the literal, and a return block esbuild collapses to void(...). Every remaining token difference is a minifier local reallocating. The old source patch could not be reused. It described the reverted composition-ownership architecture, so restoring it would have re-applied an abandoned design on top of dropping three accumulated fixes. It is re-derived from the shipped patch instead, and the derivation is a fixed point. Two deliberate departures from the deleted version. Sourcemaps are included rather than deleted, because a live test reads lib/*.map and asserts the mapped version matches the runtime version. The source-patch superset carve-out is gone, so a source hunk the shipped patch cannot name now fails loudly instead of being carved out silently. The doc's claim that the webgl and serialize addons reproduce byte for byte was half wrong. Their ESM output does reproduce at the pinned commit, but both also publish CJS that the root package script never builds, so folding either in needs a build step this harness lacks. Recorded as a blocker rather than a confident sentence. xterm_patch_sync runs the regenerator in --check mode, so a patch that does not match a rebuild of the pinned upstream now fails PR CI. The -diff -text attribute is required, not cosmetic: pnpm hashes the patch byte-for-byte, so a CRLF checkout breaks the install outright. Not verified: the CI job has not run on a real runner, the addon CJS bundles are unreproduced, and the generator is untested on Windows and Linux. * build(xterm): make the regenerator runnable on Windows and drop dead paths Readiness review on the restore found one blocking gap and two cheap cleanups. None of them change the emitted patch, which is byte-identical before and after. The generator could not run on Windows at all. Three sites called npm through execFileSync with shell:false, but npm ships as npm.cmd there, execFile applies no PATHEXT, and since CVE-2024-27980 it refuses a .cmd target without a shell. That matters because this harness arms a blocking gate whose documented remedy is --write, so a Windows contributor who tripped the gate had no remedy except hand-editing a 7MB minified bundle, which is the practice the gate exists to abolish. Four sibling scripts in config/scripts already handle this; the fix follows them and lands in run(), so the manifest-driven build step is covered too. git and tar are real executables in System32 and keep resolving without a shell, which avoids quoting exposure on paths with spaces. deleteGeneratedSourcemaps was unreachable, since the policy is include. Deleting it left "delete" as a legal policy value that nothing honoured, so a manifest asking for it would have silently shipped sourcemaps that do not match the bundle. The enum is narrowed and an unrecognised policy now throws rather than falling through. generatedHunks moved into the test file rather than being dropped; its partition assertion, that generated and source hunks reconstruct the whole patch, is worth keeping. The -text attribute now covers all five patch files. pnpm hashes each of them byte-for-byte, so the CRLF hazard the xterm patch was protected from applies equally to node-pty and the three addons. All five were already LF in the object DB, so this pins existing behaviour. -diff stays scoped to the xterm patch, since the others are readable. The doc's claim that the addons reproduce byte for byte is now dated and marked a one-off measurement rather than an invariant, because nothing re-runs it. Effective lines fall from 591 to 568 against the 600 budget. Still the largest file in config/scripts, and adding a second package to the manifest would need a split first.
This commit is contained in:
@@ -12,3 +12,9 @@
|
||||
/src/cli/bundled-skill-guides.ts text eol=lf
|
||||
# Bundled plugin trees are byte-hashed; CRLF checkout would break the pinned hash.
|
||||
/resources/plugins/** text eol=lf
|
||||
# pnpm hashes every patch byte-for-byte, so a CRLF checkout breaks the install.
|
||||
/config/patches/*.patch -text
|
||||
# The xterm bundle hunks also make a diff nobody can read; review the hand-written
|
||||
# source patch under xterm-src/ instead. The sibling patches stay diffable.
|
||||
/config/patches/@xterm__xterm@*.patch -diff
|
||||
/config/patches/xterm-src/*.patch text eol=lf
|
||||
|
||||
@@ -229,6 +229,34 @@ jobs:
|
||||
done
|
||||
exit "$status"
|
||||
|
||||
xterm_patch_sync:
|
||||
name: xterm patch sync
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- uses: ./.github/actions/install-node-dependencies
|
||||
|
||||
# Why: the check rebuilds xterm.js from a pinned upstream commit. Caching the
|
||||
# npm metadata and the shallow clone turns a ~4 min cold run into well under a
|
||||
# minute; the key is the manifest, so a commit or toolchain bump invalidates it.
|
||||
- name: Restore upstream xterm build inputs
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.npm
|
||||
${{ runner.temp }}/xterm-patch-build/upstream/.git
|
||||
key: xterm-upstream-${{ hashFiles('config/patches/xterm-upstream.json') }}
|
||||
|
||||
- name: Verify xterm patches match the pinned upstream build
|
||||
env:
|
||||
WORK_DIR: ${{ runner.temp }}/xterm-patch-build
|
||||
run: node config/scripts/regenerate-xterm-patches.mjs --check --work-dir="$WORK_DIR"
|
||||
|
||||
shell_contracts:
|
||||
name: shell contracts
|
||||
runs-on: ubuntu-latest
|
||||
@@ -605,6 +633,7 @@ jobs:
|
||||
- root_directory_guard
|
||||
- typecheck
|
||||
- git_compatibility
|
||||
- xterm_patch_sync
|
||||
- shell_contracts
|
||||
- test
|
||||
- managed_hook_node18
|
||||
@@ -627,6 +656,7 @@ jobs:
|
||||
ROOT_DIRECTORY_GUARD: ${{ needs.root_directory_guard.result }}
|
||||
TYPECHECK: ${{ needs.typecheck.result }}
|
||||
GIT_COMPATIBILITY: ${{ needs.git_compatibility.result }}
|
||||
XTERM_PATCH_SYNC: ${{ needs.xterm_patch_sync.result }}
|
||||
SHELL_CONTRACTS: ${{ needs.shell_contracts.result }}
|
||||
TEST: ${{ needs.test.result }}
|
||||
MANAGED_HOOK_NODE18: ${{ needs.managed_hook_node18.result }}
|
||||
@@ -638,6 +668,7 @@ jobs:
|
||||
"$ROOT_DIRECTORY_GUARD" \
|
||||
"$TYPECHECK" \
|
||||
"$GIT_COMPATIBILITY" \
|
||||
"$XTERM_PATCH_SYNC" \
|
||||
"$SHELL_CONTRACTS" \
|
||||
"$TEST" \
|
||||
"$MANAGED_HOOK_NODE18" \
|
||||
|
||||
@@ -109,6 +109,7 @@ docs/**
|
||||
!docs/reference/ssh-reconnect-source-recovery.md
|
||||
!docs/reference/windows-setup-shell.md
|
||||
!docs/reference/worktree-scan-fingerprint.md
|
||||
!docs/reference/xterm-patch-regeneration.md
|
||||
|
||||
# Stably CLI (only docs/ are tracked)
|
||||
.stably/*
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"$schemaNote": "Consumed by config/scripts/regenerate-xterm-patches.mjs. See docs/reference/xterm-patch-regeneration.md.",
|
||||
"upstream": {
|
||||
"repository": "https://github.com/xtermjs/xterm.js.git",
|
||||
"commit": "53a98a720ae4a973e384fa2440880d09537132f3",
|
||||
"commitSource": "bin/publish.js stamps package.json.commit before npm publish, so the published tarball names its own commit. The generator asserts the two agree."
|
||||
},
|
||||
"sourcemaps": {
|
||||
"policy": "include",
|
||||
"why": "terminal-ime-xterm-transaction-events.test.ts reads lib/*.map and asserts the mapped Version.ts matches the runtime version, so the maps have to move with the bundles rather than be dropped. Patching them costs ~5.8MB of the emitted patch; the alternative, deleting them, is only available again once nothing reads them."
|
||||
},
|
||||
"toolchain": {
|
||||
"why": "Pinned by the upstream package-lock at the commit above. The generator asserts these resolve as expected so a silent upstream resolution change surfaces as a toolchain error rather than a mystery patch diff.",
|
||||
"esbuild": "0.28.1",
|
||||
"webpack": "5.107.0",
|
||||
"terser": "5.47.1",
|
||||
"@typescript/native-preview": "7.0.0-dev.20260521.1"
|
||||
},
|
||||
"packages": [
|
||||
{
|
||||
"name": "@xterm/xterm",
|
||||
"version": "6.1.0-beta.287",
|
||||
"packageDir": ".",
|
||||
"versionStampFile": "src/common/Version.ts",
|
||||
"sourcePatch": "config/patches/xterm-src/@xterm__xterm@6.1.0-beta.287.src.patch",
|
||||
"patch": "config/patches/@xterm__xterm@6.1.0-beta.287.patch",
|
||||
"generatedPaths": ["lib/"],
|
||||
"build": [{ "cwd": ".", "command": "npm", "args": ["run", "package"] }]
|
||||
}
|
||||
],
|
||||
"forbiddenBuildScripts": {
|
||||
"why": "`npm run setup` runs a development esbuild (minify:false), so calling it after the packaging build overwrites lib/*.mjs with an unminified bundle and a mismatched map. Publish order is: stamp Version.ts, then `npm run package` only.",
|
||||
"scripts": ["setup", "presetup", "postsetup", "esbuild", "esbuild-watch", "dev"]
|
||||
}
|
||||
}
|
||||
@@ -202,7 +202,12 @@ describe('PR workflow parallelism', () => {
|
||||
(step) => step.uses === './.github/actions/install-node-dependencies'
|
||||
)
|
||||
|
||||
for (const jobName of ['static_analysis', 'typecheck', 'git_compatibility']) {
|
||||
for (const jobName of [
|
||||
'static_analysis',
|
||||
'typecheck',
|
||||
'git_compatibility',
|
||||
'xterm_patch_sync'
|
||||
]) {
|
||||
expect(installFor(jobName).with, jobName).toBeUndefined()
|
||||
}
|
||||
expect(installFor('shell_contracts').with['native-runtime']).toBe('node')
|
||||
@@ -303,6 +308,7 @@ describe('PR workflow parallelism', () => {
|
||||
'root_directory_guard',
|
||||
'typecheck',
|
||||
'git_compatibility',
|
||||
'xterm_patch_sync',
|
||||
'shell_contracts',
|
||||
'test',
|
||||
'managed_hook_node18',
|
||||
|
||||
@@ -0,0 +1,690 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import { createHash } from 'node:crypto'
|
||||
import {
|
||||
copyFileSync,
|
||||
cpSync,
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
realpathSync,
|
||||
rmSync,
|
||||
statSync,
|
||||
writeFileSync
|
||||
} from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
|
||||
const DEFAULT_REPO_ROOT = path.resolve(import.meta.dirname, '..', '..')
|
||||
const MANIFEST_RELATIVE_PATH = path.join('config', 'patches', 'xterm-upstream.json')
|
||||
|
||||
/**
|
||||
* Flags pnpm@10 passes to `git diff` in its own `diffFolders()`. A patch built
|
||||
* with anything else is a patch pnpm may re-diff differently on the next
|
||||
* `pnpm patch-commit`, so the byte-comparison gate would never settle.
|
||||
*/
|
||||
export const PNPM_DIFF_FLAGS = [
|
||||
'-c',
|
||||
'core.safecrlf=false',
|
||||
'diff',
|
||||
'--src-prefix=a/',
|
||||
'--dst-prefix=b/',
|
||||
'--ignore-cr-at-eol',
|
||||
'--irreversible-delete',
|
||||
'--full-index',
|
||||
'--no-index',
|
||||
'--text',
|
||||
'--no-ext-diff',
|
||||
'--no-color'
|
||||
]
|
||||
|
||||
/**
|
||||
* The same formatting as PNPM_DIFF_FLAGS minus `--no-index`, so a diff taken
|
||||
* inside the upstream checkout is byte-comparable with the emitted patch.
|
||||
*/
|
||||
export const CHECKOUT_DIFF_FLAGS = PNPM_DIFF_FLAGS.filter((flag) => flag !== '--no-index')
|
||||
|
||||
/** Blanks the vars pnpm blanks so user and system git config cannot reach the diff. */
|
||||
export function pnpmDiffEnvironment(baseEnvironment = process.env) {
|
||||
return {
|
||||
...baseEnvironment,
|
||||
GIT_CONFIG_NOSYSTEM: '1',
|
||||
HOME: '',
|
||||
XDG_CONFIG_HOME: '',
|
||||
USERPROFILE: ''
|
||||
}
|
||||
}
|
||||
|
||||
function escapeRegExp(value) {
|
||||
return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
||||
}
|
||||
|
||||
function trimSurroundingSlashes(value) {
|
||||
return value[0] === '/' || value.endsWith('/') ? value.replace(/^\/|\/$/g, '') : value
|
||||
}
|
||||
|
||||
/**
|
||||
* Reproduces pnpm's post-processing of the raw `git diff` output: strip the two
|
||||
* scratch folder prefixes, drop a trailing no-newline marker, and remove
|
||||
* .DS_Store entries a macOS run would otherwise smuggle in.
|
||||
*/
|
||||
export function normalizePnpmDiff(stdout, folderA, folderB) {
|
||||
const a = folderA.replace(/\\/g, '/')
|
||||
const b = folderB.replace(/\\/g, '/')
|
||||
return stdout
|
||||
.replace(new RegExp(`(a|b)(${escapeRegExp(`/${trimSurroundingSlashes(a)}/`)})`, 'g'), '$1/')
|
||||
.replace(new RegExp(`(a|b)${escapeRegExp(`/${trimSurroundingSlashes(b)}/`)}`, 'g'), '$1/')
|
||||
.replace(new RegExp(escapeRegExp(`${a}/`), 'g'), '')
|
||||
.replace(new RegExp(escapeRegExp(`${b}/`), 'g'), '')
|
||||
.replace(/\n\\ No newline at end of file\n$/, '\n')
|
||||
.replace(/^diff --git a\/.*\.DS_Store b\/.*\.DS_Store[\s\S]+?(?=^diff --git)/gm, '')
|
||||
.replace(/^diff --git a\/.*\.DS_Store b\/.*\.DS_Store[\s\S]*$/gm, '')
|
||||
}
|
||||
|
||||
/** Splits a patch into one entry per `diff --git` stanza, keeping the raw text. */
|
||||
export function splitPatchEntries(patchText) {
|
||||
return patchText
|
||||
.split(/^(?=diff --git )/m)
|
||||
.filter((entry) => entry.startsWith('diff --git '))
|
||||
.map((text) => {
|
||||
const header = text.slice(0, text.indexOf('\n'))
|
||||
const match = /^diff --git a\/(.+) b\/\1$/.exec(header)
|
||||
if (!match) {
|
||||
throw new Error(`Unsupported diff header (renames are not supported): ${header}`)
|
||||
}
|
||||
return { path: match[1], text }
|
||||
})
|
||||
}
|
||||
|
||||
export function selectPatchEntries(patchText, matches) {
|
||||
return splitPatchEntries(patchText)
|
||||
.filter((entry) => matches(entry.path))
|
||||
.map((entry) => entry.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
/** The hand-editable half of a patch: everything under `src/`. */
|
||||
export function sourceHunks(patchText) {
|
||||
return selectPatchEntries(patchText, (file) => file.startsWith('src/'))
|
||||
}
|
||||
|
||||
/**
|
||||
* The source patch and the emitted patch are the same edits diffed two ways, so
|
||||
* they must produce the same bytes. A source hunk the emitted patch cannot carry
|
||||
* would be deleted by the next `--write`, so this fails instead of shipping one.
|
||||
*/
|
||||
export function assertSourceDerivationsAgree(checkoutSource, patchText) {
|
||||
const checkout = sourceHunks(checkoutSource)
|
||||
const emitted = sourceHunks(patchText)
|
||||
if (checkout === emitted) {
|
||||
return
|
||||
}
|
||||
throw new Error(
|
||||
[
|
||||
'The checkout diff and the emitted patch disagree on a source file.',
|
||||
` from checkout: [${splitPatchEntries(checkout)
|
||||
.map((e) => e.path)
|
||||
.join(', ')}]`,
|
||||
` from patch: [${splitPatchEntries(emitted)
|
||||
.map((e) => e.path)
|
||||
.join(', ')}]`,
|
||||
` first difference at character ${firstDifferenceIndex(checkout, emitted)}`,
|
||||
'',
|
||||
'A hunk the emitted patch cannot name is never installed, so it cannot ship',
|
||||
'here; upstream .npmignore strips `src/**/*.test.ts`.'
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
export function stampVersionSource(source, version) {
|
||||
const stamped = source.replace(
|
||||
/export const XTERM_VERSION = '[^']+';/,
|
||||
`export const XTERM_VERSION = '${version}';`
|
||||
)
|
||||
if (stamped === source && !source.includes(`'${version}'`)) {
|
||||
throw new Error('Version stamp file does not declare XTERM_VERSION')
|
||||
}
|
||||
return stamped
|
||||
}
|
||||
|
||||
/**
|
||||
* The published tarball names the commit it was built from, so a version bump
|
||||
* that forgets the manifest fails here instead of producing a patch against the
|
||||
* wrong tree.
|
||||
*/
|
||||
export function assertPublishedCommit(publishedPackageJson, packageEntry, upstreamCommit) {
|
||||
if (publishedPackageJson.version !== packageEntry.version) {
|
||||
throw new Error(
|
||||
`${packageEntry.name}: registry served ${publishedPackageJson.version}, manifest pins ${packageEntry.version}`
|
||||
)
|
||||
}
|
||||
if (publishedPackageJson.commit !== upstreamCommit) {
|
||||
throw new Error(
|
||||
[
|
||||
`${packageEntry.name}@${packageEntry.version} was published from commit`,
|
||||
` ${publishedPackageJson.commit ?? '(absent)'}`,
|
||||
`but ${MANIFEST_RELATIVE_PATH} pins`,
|
||||
` ${upstreamCommit}`,
|
||||
'Update upstream.commit in the manifest to the published commit, then rerun with --write.'
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/** Guards the publish-order trap: a dev esbuild pass would silently de-minify lib/*.mjs. */
|
||||
export function assertBuildStepsAllowed(manifest) {
|
||||
const forbidden = new Set(manifest.forbiddenBuildScripts?.scripts ?? [])
|
||||
for (const packageEntry of manifest.packages) {
|
||||
for (const step of packageEntry.build) {
|
||||
const script = step.command === 'npm' && step.args[0] === 'run' ? step.args[1] : undefined
|
||||
if (script !== undefined && forbidden.has(script)) {
|
||||
throw new Error(
|
||||
`${packageEntry.name}: build step \`npm run ${script}\` is forbidden. ${manifest.forbiddenBuildScripts.why}`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// `delete` was dropped with the code that implemented it: accepting a policy
|
||||
// nothing honours would silently ship maps that do not match the bundle.
|
||||
export const SOURCEMAP_POLICIES = new Set(['include'])
|
||||
|
||||
/** An unrecognised policy is a manifest bug, not a default to fall through to. */
|
||||
export function assertSourcemapPolicy(manifest) {
|
||||
const policy = manifest.sourcemaps?.policy
|
||||
if (!SOURCEMAP_POLICIES.has(policy)) {
|
||||
throw new Error(
|
||||
`sourcemaps.policy must be one of ${[...SOURCEMAP_POLICIES].join(', ')}, got ${JSON.stringify(policy)}`
|
||||
)
|
||||
}
|
||||
return policy
|
||||
}
|
||||
|
||||
/**
|
||||
* pnpm keys the patched package directory and the lockfile entry by the
|
||||
* sha256 of the patch file itself, so a regenerated patch that leaves
|
||||
* pnpm-lock.yaml alone fails `--frozen-lockfile` on every machine but the
|
||||
* author's.
|
||||
*/
|
||||
export function patchHash(patchText) {
|
||||
return createHash('sha256').update(patchText, 'utf8').digest('hex')
|
||||
}
|
||||
|
||||
function lockfilePatchHashPattern(packageKey) {
|
||||
// Unscoped keys such as `node-pty@1.1.0` are emitted unquoted.
|
||||
return new RegExp(`(^ '?${escapeRegExp(packageKey)}'?:\\n hash: )([0-9a-f]{64})$`, 'm')
|
||||
}
|
||||
|
||||
export function readLockfilePatchHash(lockfileText, packageKey) {
|
||||
const match = lockfilePatchHashPattern(packageKey).exec(lockfileText)
|
||||
if (!match) {
|
||||
throw new Error(`pnpm-lock.yaml has no patchedDependencies entry for '${packageKey}'`)
|
||||
}
|
||||
return match[2]
|
||||
}
|
||||
|
||||
function lockfileResolutionHashPattern(packageKey) {
|
||||
const separator = packageKey.lastIndexOf('@')
|
||||
const name = escapeRegExp(packageKey.slice(0, separator))
|
||||
const version = escapeRegExp(packageKey.slice(separator + 1))
|
||||
// Two spellings: `name@version(patch_hash=…)` in dependency keys, and a bare
|
||||
// `: version(patch_hash=…)` under `version:` and in resolved dependency maps.
|
||||
return new RegExp(`(?:${name}@|: )${version}\\(patch_hash=([0-9a-f]{64})\\)`, 'g')
|
||||
}
|
||||
|
||||
/**
|
||||
* pnpm repeats the hash inside every resolution key that depends on the patched
|
||||
* package, not just in `patchedDependencies`. Updating one and not the other leaves
|
||||
* a lockfile that installs on a warm store and drifts on a cold one, which is CI.
|
||||
*/
|
||||
export function readLockfileResolutionHashes(lockfileText, packageKey) {
|
||||
return Array.from(
|
||||
lockfileText.matchAll(lockfileResolutionHashPattern(packageKey)),
|
||||
(match) => match[1]
|
||||
)
|
||||
}
|
||||
|
||||
export function lockfilePatchHashIsStale(lockfileText, packageKey, hash) {
|
||||
return (
|
||||
readLockfilePatchHash(lockfileText, packageKey) !== hash ||
|
||||
readLockfileResolutionHashes(lockfileText, packageKey).some((value) => value !== hash)
|
||||
)
|
||||
}
|
||||
|
||||
export function updateLockfilePatchHash(lockfileText, packageKey, hash) {
|
||||
readLockfilePatchHash(lockfileText, packageKey)
|
||||
return lockfileText
|
||||
.replace(lockfilePatchHashPattern(packageKey), `$1${hash}`)
|
||||
.replace(lockfileResolutionHashPattern(packageKey), (match, current) =>
|
||||
match.replace(`patch_hash=${current}`, `patch_hash=${hash}`)
|
||||
)
|
||||
}
|
||||
|
||||
export function firstDifferenceIndex(left, right) {
|
||||
const limit = Math.min(left.length, right.length)
|
||||
for (let index = 0; index < limit; index += 1) {
|
||||
if (left[index] !== right[index]) {
|
||||
return index
|
||||
}
|
||||
}
|
||||
return left.length === right.length ? -1 : limit
|
||||
}
|
||||
|
||||
export function formatCheckFailure({ name, patchPath, committed, regenerated }) {
|
||||
const index = firstDifferenceIndex(committed, regenerated)
|
||||
const committedFiles = splitPatchEntries(committed).map((entry) => entry.path)
|
||||
const regeneratedFiles = splitPatchEntries(regenerated).map((entry) => entry.path)
|
||||
return [
|
||||
`${name}: ${patchPath} is not what the pinned upstream build produces.`,
|
||||
` committed: ${Buffer.byteLength(committed)} bytes, files [${committedFiles.join(', ')}]`,
|
||||
` regenerated: ${Buffer.byteLength(regenerated)} bytes, files [${regeneratedFiles.join(', ')}]`,
|
||||
` first difference at character ${index}`,
|
||||
'',
|
||||
'The bundle hunks are generated. Do not edit them. Change the source patch',
|
||||
'instead and regenerate both files:',
|
||||
'',
|
||||
' node config/scripts/regenerate-xterm-patches.mjs --write',
|
||||
'',
|
||||
'See docs/reference/xterm-patch-regeneration.md.'
|
||||
].join('\n')
|
||||
}
|
||||
|
||||
// npm ships as `npm.cmd` on Windows, and execFile applies no PATHEXT and refuses
|
||||
// a `.cmd` target without a shell (CVE-2024-27980). git and tar are real .exe.
|
||||
const WINDOWS_SHIM_COMMANDS = new Set(['npm', 'npx', 'pnpm', 'yarn'])
|
||||
|
||||
function run(command, args, options = {}) {
|
||||
const shim = process.platform === 'win32' && WINDOWS_SHIM_COMMANDS.has(command)
|
||||
return execFileSync(shim ? `${command}.cmd` : command, args, {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 256 * 1024 * 1024,
|
||||
stdio: ['ignore', 'pipe', 'inherit'],
|
||||
shell: shim,
|
||||
...options
|
||||
})
|
||||
}
|
||||
|
||||
function listFilesRelative(root, base = root) {
|
||||
const files = []
|
||||
for (const entry of readdirSync(base, { withFileTypes: true })) {
|
||||
const absolute = path.join(base, entry.name)
|
||||
if (entry.isDirectory()) {
|
||||
files.push(...listFilesRelative(root, absolute))
|
||||
} else if (entry.isFile()) {
|
||||
files.push(path.relative(root, absolute))
|
||||
}
|
||||
}
|
||||
return files.sort()
|
||||
}
|
||||
|
||||
function sameBytes(left, right) {
|
||||
return (
|
||||
statSync(left).size === statSync(right).size && readFileSync(left).equals(readFileSync(right))
|
||||
)
|
||||
}
|
||||
|
||||
function fetchPristinePackage(packageEntry, workDir) {
|
||||
const target = path.join(workDir, 'pristine', packageEntry.name.replace(/[@/]/g, '_'))
|
||||
rmSync(target, { recursive: true, force: true })
|
||||
mkdirSync(target, { recursive: true })
|
||||
const spec = `${packageEntry.name}@${packageEntry.version}`
|
||||
const output = run('npm', ['pack', spec, '--pack-destination', target, '--silent'], {
|
||||
cwd: workDir
|
||||
})
|
||||
const tarball = path.join(target, output.trim().split('\n').at(-1).trim())
|
||||
run('tar', ['xzf', tarball, '-C', target])
|
||||
return path.join(target, 'package')
|
||||
}
|
||||
|
||||
function hasCommit(root, commit) {
|
||||
try {
|
||||
return run('git', ['cat-file', '-t', commit], { cwd: root, stdio: 'pipe' }).trim() === 'commit'
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function ensureUpstreamCheckout(manifest, workDir) {
|
||||
const root = path.join(workDir, 'upstream')
|
||||
const { repository, commit } = manifest.upstream
|
||||
if (!existsSync(path.join(root, '.git'))) {
|
||||
mkdirSync(root, { recursive: true })
|
||||
run('git', ['init', '--quiet'], { cwd: root })
|
||||
run('git', ['remote', 'add', 'origin', repository], { cwd: root })
|
||||
}
|
||||
if (!hasCommit(root, commit)) {
|
||||
run('git', ['fetch', '--depth=1', 'origin', commit], { cwd: root, stdio: 'inherit' })
|
||||
}
|
||||
run('git', ['checkout', '--quiet', '--detach', commit], { cwd: root })
|
||||
run('git', ['reset', '--quiet', '--hard', commit], { cwd: root })
|
||||
return root
|
||||
}
|
||||
|
||||
function ensureDependencies(upstreamRoot, manifest) {
|
||||
const lockfile = path.join(upstreamRoot, 'package-lock.json')
|
||||
const stamp = path.join(upstreamRoot, 'node_modules', '.orca-xterm-install-stamp')
|
||||
const want = `${manifest.upstream.commit}\n${statSync(lockfile).size}\n`
|
||||
if (existsSync(stamp) && readFileSync(stamp, 'utf8') === want) {
|
||||
return
|
||||
}
|
||||
run('npm', ['ci'], {
|
||||
cwd: upstreamRoot,
|
||||
env: { ...process.env, PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1', PUPPETEER_SKIP_DOWNLOAD: '1' }
|
||||
})
|
||||
assertToolchain(upstreamRoot, manifest)
|
||||
writeFileSync(stamp, want)
|
||||
}
|
||||
|
||||
function assertToolchain(upstreamRoot, manifest) {
|
||||
const expected = manifest.toolchain
|
||||
for (const [name, version] of Object.entries(expected)) {
|
||||
if (name === 'why') {
|
||||
continue
|
||||
}
|
||||
const installed = path.join(upstreamRoot, 'node_modules', name, 'package.json')
|
||||
if (!existsSync(installed)) {
|
||||
throw new Error(
|
||||
`Upstream install is missing ${name}. The pinned toolchain is no longer resolvable; see the tsgo note in docs/reference/xterm-patch-regeneration.md.`
|
||||
)
|
||||
}
|
||||
const actual = JSON.parse(readFileSync(installed, 'utf8')).version
|
||||
if (actual !== version) {
|
||||
throw new Error(
|
||||
`Upstream ${name} resolved to ${actual}, manifest expects ${version}. Update the toolchain block only together with a verified rebuild.`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The published src/ must equal the pinned commit's src/ apart from the version
|
||||
* stamp publish.js rewrites. If it does not, the manifest points at the wrong
|
||||
* commit and every hunk below would be nonsense.
|
||||
*/
|
||||
function assertPristineSourceMatches(pristineDir, upstreamRoot, packageEntry) {
|
||||
const stampFile = packageEntry.versionStampFile
|
||||
const sourceRoot = path.join(pristineDir, 'src')
|
||||
const drifted = listFilesRelative(sourceRoot)
|
||||
.map((relative) => path.join('src', relative))
|
||||
.filter((relative) => relative !== stampFile)
|
||||
.filter(
|
||||
(relative) =>
|
||||
!sameBytes(
|
||||
path.join(pristineDir, relative),
|
||||
path.join(upstreamRoot, packageEntry.packageDir, relative)
|
||||
)
|
||||
)
|
||||
if (drifted.length > 0) {
|
||||
throw new Error(
|
||||
`Published src/ does not match ${packageEntry.packageDir} at the pinned commit: ${drifted.join(', ')}`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
function buildPackage(upstreamRoot, packageEntry, manifest) {
|
||||
const packageRoot = path.join(upstreamRoot, packageEntry.packageDir)
|
||||
for (const directory of ['lib', 'out', 'out-esbuild']) {
|
||||
rmSync(path.join(packageRoot, directory), { recursive: true, force: true })
|
||||
}
|
||||
const stampPath = path.join(packageRoot, packageEntry.versionStampFile)
|
||||
writeFileSync(
|
||||
stampPath,
|
||||
stampVersionSource(readFileSync(stampPath, 'utf8'), packageEntry.version)
|
||||
)
|
||||
assertBuildStepsAllowed(manifest)
|
||||
for (const step of packageEntry.build) {
|
||||
run(step.command, step.args, { cwd: path.join(packageRoot, step.cwd), stdio: 'inherit' })
|
||||
}
|
||||
}
|
||||
|
||||
/** Proves the pinned toolchain still reproduces the untouched published bundles. */
|
||||
function assertReproducesPristineBundles(pristineDir, upstreamRoot, packageEntry) {
|
||||
const packageRoot = path.join(upstreamRoot, packageEntry.packageDir)
|
||||
const drifted = listFilesRelative(pristineDir)
|
||||
.filter((relative) =>
|
||||
packageEntry.generatedPaths.some((prefix) => toPosix(relative).startsWith(prefix))
|
||||
)
|
||||
.filter(
|
||||
(relative) => !sameBytes(path.join(pristineDir, relative), path.join(packageRoot, relative))
|
||||
)
|
||||
if (drifted.length > 0) {
|
||||
throw new Error(
|
||||
[
|
||||
`Rebuilding ${packageEntry.name}@${packageEntry.version} from the pinned commit did not reproduce the published bundles:`,
|
||||
...drifted.map((relative) => ` ${relative}`),
|
||||
'',
|
||||
'Refusing to emit a patch. Either the toolchain drifted or the build ran in the',
|
||||
'wrong order (a dev `npm run setup` pass de-minifies lib/*.mjs).'
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
function toPosix(value) {
|
||||
return value.split(path.sep).join('/')
|
||||
}
|
||||
|
||||
function overlayBuildOutput(pristineDir, upstreamRoot, packageEntry, destination) {
|
||||
rmSync(destination, { recursive: true, force: true })
|
||||
cpSync(pristineDir, destination, { recursive: true })
|
||||
const packageRoot = path.join(upstreamRoot, packageEntry.packageDir)
|
||||
for (const relative of listFilesRelative(pristineDir)) {
|
||||
// package.json carries the registry's version/commit stamp, which the build
|
||||
// tree has no way to reproduce and which we never want to patch.
|
||||
if (relative === 'package.json') {
|
||||
continue
|
||||
}
|
||||
const built = path.join(packageRoot, relative)
|
||||
if (!existsSync(built)) {
|
||||
throw new Error(`Published file has no build-tree counterpart: ${relative}`)
|
||||
}
|
||||
copyFileSync(built, path.join(destination, relative))
|
||||
}
|
||||
}
|
||||
|
||||
function diffFolders(folderA, folderB) {
|
||||
let stdout
|
||||
try {
|
||||
stdout = execFileSync('git', [...PNPM_DIFF_FLAGS, folderA, folderB], {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 512 * 1024 * 1024,
|
||||
env: pnpmDiffEnvironment(),
|
||||
stdio: ['ignore', 'pipe', 'pipe']
|
||||
})
|
||||
} catch (error) {
|
||||
// `git diff --no-index` exits 1 whenever it finds differences.
|
||||
if (error.status !== 1 || error.stderr?.length > 0) {
|
||||
throw error
|
||||
}
|
||||
stdout = error.stdout
|
||||
}
|
||||
return normalizePnpmDiff(stdout, folderA, folderB)
|
||||
}
|
||||
|
||||
/** The source of truth for the hand-written half: what the checkout itself holds. */
|
||||
function diffCheckoutSource(packageRoot) {
|
||||
return run('git', [...CHECKOUT_DIFF_FLAGS, '--', 'src/'], {
|
||||
cwd: packageRoot,
|
||||
env: pnpmDiffEnvironment(),
|
||||
maxBuffer: 64 * 1024 * 1024
|
||||
})
|
||||
}
|
||||
|
||||
function regeneratePackage(packageEntry, manifest, context) {
|
||||
const { workDir, repoRoot } = context
|
||||
const pristineDir = fetchPristinePackage(packageEntry, workDir)
|
||||
const published = JSON.parse(readFileSync(path.join(pristineDir, 'package.json'), 'utf8'))
|
||||
assertPublishedCommit(published, packageEntry, manifest.upstream.commit)
|
||||
|
||||
const upstreamRoot = ensureUpstreamCheckout(manifest, workDir)
|
||||
ensureDependencies(upstreamRoot, manifest)
|
||||
assertPristineSourceMatches(pristineDir, upstreamRoot, packageEntry)
|
||||
|
||||
buildPackage(upstreamRoot, packageEntry, manifest)
|
||||
assertReproducesPristineBundles(pristineDir, upstreamRoot, packageEntry)
|
||||
|
||||
run('git', ['reset', '--quiet', '--hard', manifest.upstream.commit], { cwd: upstreamRoot })
|
||||
run('git', ['apply', '--whitespace=nowarn', path.join(repoRoot, packageEntry.sourcePatch)], {
|
||||
cwd: path.join(upstreamRoot, packageEntry.packageDir)
|
||||
})
|
||||
buildPackage(upstreamRoot, packageEntry, manifest)
|
||||
|
||||
const patchedDir = path.join(workDir, 'patched', packageEntry.name.replace(/[@/]/g, '_'))
|
||||
overlayBuildOutput(pristineDir, upstreamRoot, packageEntry, patchedDir)
|
||||
|
||||
// Leave the checkout diffable: the pinned commit plus the source patch, with
|
||||
// no publish-time version stamp mixed in, so `git diff` there is the source
|
||||
// patch and nothing else.
|
||||
run('git', ['checkout', '--', packageEntry.versionStampFile], {
|
||||
cwd: path.join(upstreamRoot, packageEntry.packageDir)
|
||||
})
|
||||
|
||||
const source = diffCheckoutSource(path.join(upstreamRoot, packageEntry.packageDir))
|
||||
const patch = diffFolders(pristineDir, patchedDir)
|
||||
assertSourceDerivationsAgree(source, patch)
|
||||
return { patch, source }
|
||||
}
|
||||
|
||||
export function regenerateXtermPatches({
|
||||
mode,
|
||||
repoRoot = DEFAULT_REPO_ROOT,
|
||||
workDir = path.join(tmpdir(), 'orca-xterm-patch-build'),
|
||||
log = console.info
|
||||
} = {}) {
|
||||
const manifest = JSON.parse(readFileSync(path.join(repoRoot, MANIFEST_RELATIVE_PATH), 'utf8'))
|
||||
assertBuildStepsAllowed(manifest)
|
||||
assertSourcemapPolicy(manifest)
|
||||
mkdirSync(workDir, { recursive: true })
|
||||
|
||||
const lockfilePath = path.join(repoRoot, 'pnpm-lock.yaml')
|
||||
let lockfile = readFileSync(lockfilePath, 'utf8')
|
||||
let lockfileChanged = false
|
||||
|
||||
const failures = []
|
||||
for (const packageEntry of manifest.packages) {
|
||||
const shortCommit = manifest.upstream.commit.slice(0, 12)
|
||||
log(`${packageEntry.name}@${packageEntry.version}: regenerating from ${shortCommit}`)
|
||||
const { patch: regenerated, source: canonicalSource } = regeneratePackage(
|
||||
packageEntry,
|
||||
manifest,
|
||||
{ workDir, repoRoot }
|
||||
)
|
||||
const patchPath = path.join(repoRoot, packageEntry.patch)
|
||||
const sourcePatchPath = path.join(repoRoot, packageEntry.sourcePatch)
|
||||
const packageKey = `${packageEntry.name}@${packageEntry.version}`
|
||||
const hash = patchHash(regenerated)
|
||||
|
||||
if (mode === 'write') {
|
||||
writeFileSync(patchPath, regenerated)
|
||||
writeFileSync(sourcePatchPath, canonicalSource)
|
||||
log(` wrote ${packageEntry.patch} (${Buffer.byteLength(regenerated)} bytes)`)
|
||||
log(` wrote ${packageEntry.sourcePatch} (${Buffer.byteLength(canonicalSource)} bytes)`)
|
||||
if (lockfilePatchHashIsStale(lockfile, packageKey, hash)) {
|
||||
lockfile = updateLockfilePatchHash(lockfile, packageKey, hash)
|
||||
lockfileChanged = true
|
||||
log(` updated pnpm-lock.yaml patch hash to ${hash}`)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (lockfilePatchHashIsStale(lockfile, packageKey, hash)) {
|
||||
const stale = Array.from(
|
||||
new Set(readLockfileResolutionHashes(lockfile, packageKey).filter((v) => v !== hash))
|
||||
)
|
||||
failures.push(
|
||||
[
|
||||
`${packageKey}: pnpm-lock.yaml records a stale patch hash.`,
|
||||
` patchedDependencies: ${readLockfilePatchHash(lockfile, packageKey)}`,
|
||||
` resolution keys: ${stale.length > 0 ? stale.join(', ') : 'in sync'}`,
|
||||
` patch: ${hash}`,
|
||||
'',
|
||||
'pnpm keys the patched package by the sha256 of the patch file, so',
|
||||
'`pnpm install --frozen-lockfile` will fail. Rerun with --write.'
|
||||
].join('\n')
|
||||
)
|
||||
}
|
||||
|
||||
const committed = readFileSync(patchPath, 'utf8')
|
||||
if (committed !== regenerated) {
|
||||
failures.push(
|
||||
formatCheckFailure({
|
||||
name: packageEntry.name,
|
||||
patchPath: packageEntry.patch,
|
||||
committed,
|
||||
regenerated
|
||||
})
|
||||
)
|
||||
continue
|
||||
}
|
||||
const committedSource = readFileSync(sourcePatchPath, 'utf8')
|
||||
if (committedSource !== canonicalSource) {
|
||||
failures.push(
|
||||
formatCheckFailure({
|
||||
name: packageEntry.name,
|
||||
patchPath: packageEntry.sourcePatch,
|
||||
committed: committedSource,
|
||||
regenerated: canonicalSource
|
||||
})
|
||||
)
|
||||
continue
|
||||
}
|
||||
log(` in sync (${Buffer.byteLength(regenerated)} bytes)`)
|
||||
}
|
||||
|
||||
if (lockfileChanged) {
|
||||
writeFileSync(lockfilePath, lockfile)
|
||||
}
|
||||
if (failures.length > 0) {
|
||||
throw new Error(failures.join('\n\n'))
|
||||
}
|
||||
}
|
||||
|
||||
const USAGE =
|
||||
'Usage: regenerate-xterm-patches.mjs [--check | --write] [--work-dir=<path>]\n' +
|
||||
' --check (default) verifies the shipped patches match the pinned upstream build;\n' +
|
||||
' --write regenerates them from config/patches/xterm-src/. Build outside this repo:\n' +
|
||||
' tsc otherwise walks up into our node_modules. See\n' +
|
||||
' docs/reference/xterm-patch-regeneration.md.'
|
||||
|
||||
function main(argv) {
|
||||
if (argv.includes('--help') || argv.includes('-h')) {
|
||||
console.info(USAGE)
|
||||
return
|
||||
}
|
||||
// --check is the default, so an unrecognised flag would otherwise silently run a full
|
||||
// upstream build instead of whatever the caller meant.
|
||||
const known = (v) =>
|
||||
!v.startsWith('-') || ['--write', '--check'].includes(v) || v.startsWith('--work-dir=')
|
||||
const unknown = argv.filter((value) => !known(value))
|
||||
if (unknown.length > 0) {
|
||||
throw new Error(`Unknown option: ${unknown.join(', ')}\n${USAGE}`)
|
||||
}
|
||||
const write = argv.includes('--write')
|
||||
const check = argv.includes('--check') || !write
|
||||
if (write && argv.includes('--check')) {
|
||||
throw new Error('Pass either --write or --check, not both')
|
||||
}
|
||||
const workDirArgument = argv.find((value) => value.startsWith('--work-dir='))
|
||||
regenerateXtermPatches({
|
||||
mode: write ? 'write' : 'check',
|
||||
workDir: workDirArgument ? path.resolve(workDirArgument.slice('--work-dir='.length)) : undefined
|
||||
})
|
||||
if (check) {
|
||||
console.info('xterm patches are in sync with the pinned upstream build.')
|
||||
}
|
||||
}
|
||||
|
||||
// realpathSync so a symlinked checkout path still registers as a direct run.
|
||||
const invokedPath = process.argv[1] ? pathToFileURL(realpathSync(process.argv[1])).href : null
|
||||
if (invokedPath === import.meta.url) {
|
||||
try {
|
||||
main(process.argv.slice(2))
|
||||
} catch (error) {
|
||||
console.error(`\n${error.message}\n`)
|
||||
process.exit(1)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,463 @@
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import { createHash } from 'node:crypto'
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import {
|
||||
CHECKOUT_DIFF_FLAGS,
|
||||
PNPM_DIFF_FLAGS,
|
||||
assertBuildStepsAllowed,
|
||||
assertPublishedCommit,
|
||||
assertSourceDerivationsAgree,
|
||||
assertSourcemapPolicy,
|
||||
firstDifferenceIndex,
|
||||
formatCheckFailure,
|
||||
lockfilePatchHashIsStale,
|
||||
normalizePnpmDiff,
|
||||
patchHash,
|
||||
pnpmDiffEnvironment,
|
||||
readLockfilePatchHash,
|
||||
readLockfileResolutionHashes,
|
||||
sourceHunks,
|
||||
splitPatchEntries,
|
||||
stampVersionSource,
|
||||
updateLockfilePatchHash
|
||||
} from './regenerate-xterm-patches.mjs'
|
||||
|
||||
// Only the tests need to slice the generated half out of a patch; the generator
|
||||
// reads `generatedPaths` directly where it compares against the pristine build.
|
||||
function generatedHunks(patchText, generatedPaths) {
|
||||
return splitPatchEntries(patchText)
|
||||
.filter((entry) => generatedPaths.some((prefix) => entry.path.startsWith(prefix)))
|
||||
.map((entry) => entry.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
const REPO_ROOT = path.resolve(import.meta.dirname, '..', '..')
|
||||
const MANIFEST_PATH = path.join(REPO_ROOT, 'config', 'patches', 'xterm-upstream.json')
|
||||
const temporaryDirectories = []
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(
|
||||
temporaryDirectories
|
||||
.splice(0)
|
||||
.map((directory) => rm(directory, { recursive: true, force: true }))
|
||||
)
|
||||
})
|
||||
|
||||
async function createDirectory() {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), 'orca-xterm-patch-'))
|
||||
temporaryDirectories.push(directory)
|
||||
return directory
|
||||
}
|
||||
|
||||
async function writeTree(root, files) {
|
||||
for (const [relative, contents] of Object.entries(files)) {
|
||||
const target = path.join(root, relative)
|
||||
await mkdir(path.dirname(target), { recursive: true })
|
||||
await writeFile(target, contents)
|
||||
}
|
||||
}
|
||||
|
||||
/** The three exported diff pieces, composed the way the generator composes them. */
|
||||
function diffFolders(folderA, folderB) {
|
||||
let stdout
|
||||
try {
|
||||
stdout = execFileSync('git', [...PNPM_DIFF_FLAGS, folderA, folderB], {
|
||||
encoding: 'utf8',
|
||||
env: pnpmDiffEnvironment(),
|
||||
stdio: ['ignore', 'pipe', 'pipe']
|
||||
})
|
||||
} catch (error) {
|
||||
if (error.status !== 1) {
|
||||
throw error
|
||||
}
|
||||
stdout = error.stdout
|
||||
}
|
||||
return normalizePnpmDiff(stdout, folderA, folderB)
|
||||
}
|
||||
|
||||
const PRISTINE = {
|
||||
'src/Widget.ts': 'export function widget(): number {\n return 1\n}\n',
|
||||
'src/Other.ts': 'export const other = 0\n',
|
||||
'lib/widget.js': 'function widget(){return 1}\n',
|
||||
'lib/widget.js.map': '{"version":3,"sources":["../src/Widget.ts"],"mappings":"AAAA"}\n',
|
||||
'package.json': '{\n "name": "@scope/widget"\n}\n'
|
||||
}
|
||||
|
||||
const PATCHED = {
|
||||
...PRISTINE,
|
||||
'src/Widget.ts': 'export function widget(): number {\n return 2\n}\n',
|
||||
'lib/widget.js': 'function widget(){return 2}\n',
|
||||
'lib/widget.js.map': '{"version":3,"sources":["../src/Widget.ts"],"mappings":"AAAC"}\n'
|
||||
}
|
||||
|
||||
describe('pnpm diff format', () => {
|
||||
it('keeps the exact git flags pnpm uses, so patches survive `pnpm patch-commit`', () => {
|
||||
expect(PNPM_DIFF_FLAGS).toEqual([
|
||||
'-c',
|
||||
'core.safecrlf=false',
|
||||
'diff',
|
||||
'--src-prefix=a/',
|
||||
'--dst-prefix=b/',
|
||||
'--ignore-cr-at-eol',
|
||||
'--irreversible-delete',
|
||||
'--full-index',
|
||||
'--no-index',
|
||||
'--text',
|
||||
'--no-ext-diff',
|
||||
'--no-color'
|
||||
])
|
||||
})
|
||||
|
||||
it('blanks the config-bearing environment variables', () => {
|
||||
const environment = pnpmDiffEnvironment({ PATH: '/usr/bin', HOME: '/Users/someone' })
|
||||
expect(environment).toMatchObject({
|
||||
PATH: '/usr/bin',
|
||||
GIT_CONFIG_NOSYSTEM: '1',
|
||||
HOME: '',
|
||||
XDG_CONFIG_HOME: '',
|
||||
USERPROFILE: ''
|
||||
})
|
||||
})
|
||||
|
||||
it('strips both scratch folder prefixes from headers and index lines', async () => {
|
||||
const root = await createDirectory()
|
||||
const folderA = path.join(root, 'pristine')
|
||||
const folderB = path.join(root, 'patched')
|
||||
await writeTree(folderA, PRISTINE)
|
||||
await writeTree(folderB, PATCHED)
|
||||
|
||||
const patch = diffFolders(folderA, folderB)
|
||||
|
||||
expect(patch).not.toContain(root)
|
||||
expect(patch).toContain('diff --git a/lib/widget.js b/lib/widget.js')
|
||||
expect(patch).toContain('--- a/src/Widget.ts')
|
||||
expect(patch).toContain('+++ b/src/Widget.ts')
|
||||
})
|
||||
|
||||
it('drops a trailing no-newline marker and .DS_Store entries', () => {
|
||||
const withMarker = 'diff --git a/x b/x\n@@ -1 +1 @@\n-a\n+b\n\\ No newline at end of file\n'
|
||||
expect(normalizePnpmDiff(withMarker, '/a', '/b')).toBe(
|
||||
'diff --git a/x b/x\n@@ -1 +1 @@\n-a\n+b\n'
|
||||
)
|
||||
|
||||
const withJunk = [
|
||||
'diff --git a/.DS_Store b/.DS_Store\n',
|
||||
'index 000..111\n',
|
||||
'Binary files differ\n',
|
||||
'diff --git a/lib/x.js b/lib/x.js\n',
|
||||
'@@ -1 +1 @@\n-a\n+b\n'
|
||||
].join('')
|
||||
expect(normalizePnpmDiff(withJunk, '/a', '/b')).toBe(
|
||||
'diff --git a/lib/x.js b/lib/x.js\n@@ -1 +1 @@\n-a\n+b\n'
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('patch entry splitting', () => {
|
||||
it('separates hand-edited source hunks from generated bundle hunks', async () => {
|
||||
const root = await createDirectory()
|
||||
const folderA = path.join(root, 'pristine')
|
||||
const folderB = path.join(root, 'patched')
|
||||
await writeTree(folderA, PRISTINE)
|
||||
await writeTree(folderB, PATCHED)
|
||||
const patch = diffFolders(folderA, folderB)
|
||||
|
||||
expect(splitPatchEntries(patch).map((entry) => entry.path)).toEqual([
|
||||
'lib/widget.js',
|
||||
'lib/widget.js.map',
|
||||
'src/Widget.ts'
|
||||
])
|
||||
expect(splitPatchEntries(sourceHunks(patch)).map((entry) => entry.path)).toEqual([
|
||||
'src/Widget.ts'
|
||||
])
|
||||
expect(splitPatchEntries(generatedHunks(patch, ['lib/'])).map((entry) => entry.path)).toEqual([
|
||||
'lib/widget.js',
|
||||
'lib/widget.js.map'
|
||||
])
|
||||
})
|
||||
|
||||
it('rejects renames rather than emitting a header it cannot round-trip', () => {
|
||||
expect(() => splitPatchEntries('diff --git a/old.ts b/new.ts\n')).toThrow(
|
||||
/renames are not supported/
|
||||
)
|
||||
})
|
||||
|
||||
it('concatenating the two halves reproduces the whole patch', async () => {
|
||||
const root = await createDirectory()
|
||||
const folderA = path.join(root, 'pristine')
|
||||
const folderB = path.join(root, 'patched')
|
||||
await writeTree(folderA, PRISTINE)
|
||||
await writeTree(folderB, PATCHED)
|
||||
const patch = diffFolders(folderA, folderB)
|
||||
|
||||
expect(generatedHunks(patch, ['lib/']) + sourceHunks(patch)).toBe(patch)
|
||||
})
|
||||
})
|
||||
|
||||
describe('round-trip stability', () => {
|
||||
it('re-diffing an applied patch yields the identical patch', async () => {
|
||||
const root = await createDirectory()
|
||||
const folderA = path.join(root, 'pristine')
|
||||
const folderB = path.join(root, 'patched')
|
||||
await writeTree(folderA, PRISTINE)
|
||||
await writeTree(folderB, PATCHED)
|
||||
const patch = diffFolders(folderA, folderB)
|
||||
|
||||
const replay = path.join(root, 'replay')
|
||||
await writeTree(replay, PRISTINE)
|
||||
const patchFile = path.join(root, 'round-trip.patch')
|
||||
await writeFile(patchFile, patch)
|
||||
execFileSync('git', ['apply', '-p1', '--whitespace=nowarn', patchFile], { cwd: replay })
|
||||
|
||||
expect(await readFile(path.join(replay, 'lib/widget.js'), 'utf8')).toBe(
|
||||
PATCHED['lib/widget.js']
|
||||
)
|
||||
expect(diffFolders(folderA, replay)).toBe(patch)
|
||||
})
|
||||
|
||||
it('applying only the source half leaves the bundle untouched', async () => {
|
||||
const root = await createDirectory()
|
||||
const folderA = path.join(root, 'pristine')
|
||||
const folderB = path.join(root, 'patched')
|
||||
await writeTree(folderA, PRISTINE)
|
||||
await writeTree(folderB, PATCHED)
|
||||
const patchFile = path.join(root, 'src.patch')
|
||||
await writeFile(patchFile, sourceHunks(diffFolders(folderA, folderB)))
|
||||
|
||||
const replay = path.join(root, 'replay')
|
||||
await writeTree(replay, PRISTINE)
|
||||
execFileSync('git', ['apply', '-p1', '--whitespace=nowarn', patchFile], { cwd: replay })
|
||||
|
||||
expect(await readFile(path.join(replay, 'src/Widget.ts'), 'utf8')).toBe(
|
||||
PATCHED['src/Widget.ts']
|
||||
)
|
||||
expect(await readFile(path.join(replay, 'lib/widget.js'), 'utf8')).toBe(
|
||||
PRISTINE['lib/widget.js']
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
// `--write` rewrites the source patch from the emitted patch, so a hunk the
|
||||
// emitted patch cannot name would delete itself on the next run.
|
||||
describe('source derivation agreement', () => {
|
||||
const publishedEntry = [
|
||||
'diff --git a/src/browser/Types.ts b/src/browser/Types.ts',
|
||||
'index 1111111..2222222 100644',
|
||||
'--- a/src/browser/Types.ts',
|
||||
'+++ b/src/browser/Types.ts',
|
||||
'@@ -1 +1,2 @@',
|
||||
' interface ICompositionHelper {',
|
||||
'+ handleCompositionInput(data: string): boolean;',
|
||||
''
|
||||
].join('\n')
|
||||
const unpublishedEntry = [
|
||||
'diff --git a/src/browser/TestUtils.test.ts b/src/browser/TestUtils.test.ts',
|
||||
'index 3333333..4444444 100644',
|
||||
'--- a/src/browser/TestUtils.test.ts',
|
||||
'+++ b/src/browser/TestUtils.test.ts',
|
||||
'@@ -1 +1,2 @@',
|
||||
' class MockCompositionHelper {',
|
||||
'+ public handleCompositionInput(): boolean { return false; }',
|
||||
''
|
||||
].join('\n')
|
||||
it('fails when the source patch carries a file the emitted patch cannot', () => {
|
||||
expect(() =>
|
||||
assertSourceDerivationsAgree(publishedEntry + unpublishedEntry, publishedEntry)
|
||||
).toThrow(/disagree on a source file/)
|
||||
})
|
||||
|
||||
it('fails when the two derivations disagree on a file', () => {
|
||||
expect(() =>
|
||||
assertSourceDerivationsAgree(publishedEntry.replace('boolean;', 'void;'), publishedEntry)
|
||||
).toThrow(/disagree on a source file/)
|
||||
})
|
||||
|
||||
it('diffs the checkout with pnpm formatting so the two halves stay comparable', () => {
|
||||
expect(CHECKOUT_DIFF_FLAGS).toEqual(PNPM_DIFF_FLAGS.filter((flag) => flag !== '--no-index'))
|
||||
expect(CHECKOUT_DIFF_FLAGS).toContain('--full-index')
|
||||
expect(CHECKOUT_DIFF_FLAGS).not.toContain('--no-index')
|
||||
})
|
||||
})
|
||||
|
||||
describe('manifest guards', () => {
|
||||
const packageEntry = { name: '@xterm/xterm', version: '6.1.0-beta.287' }
|
||||
const commit = '53a98a720ae4a973e384fa2440880d09537132f3'
|
||||
|
||||
it('accepts a tarball that names the pinned commit', () => {
|
||||
const published = { version: '6.1.0-beta.287', commit }
|
||||
expect(() => assertPublishedCommit(published, packageEntry, commit)).not.toThrow()
|
||||
})
|
||||
|
||||
it('fails when a version bump moved the upstream commit', () => {
|
||||
const published = { version: '6.1.0-beta.287', commit: 'f'.repeat(40) }
|
||||
expect(() => assertPublishedCommit(published, packageEntry, commit)).toThrow(
|
||||
/was published from commit[\s\S]*Update upstream\.commit/
|
||||
)
|
||||
})
|
||||
|
||||
it('fails when the registry serves a different version than the manifest pins', () => {
|
||||
const published = { version: '6.1.0-beta.288', commit }
|
||||
expect(() => assertPublishedCommit(published, packageEntry, commit)).toThrow(/registry served/)
|
||||
})
|
||||
|
||||
it('fails when the tarball carries no commit stamp at all', () => {
|
||||
expect(() =>
|
||||
assertPublishedCommit({ version: '6.1.0-beta.287' }, packageEntry, commit)
|
||||
).toThrow(/\(absent\)/)
|
||||
})
|
||||
|
||||
it('refuses a build step that would de-minify the bundle', () => {
|
||||
const manifest = {
|
||||
forbiddenBuildScripts: { why: 'dev esbuild', scripts: ['setup'] },
|
||||
packages: [
|
||||
{
|
||||
name: '@xterm/xterm',
|
||||
build: [
|
||||
{ cwd: '.', command: 'npm', args: ['run', 'setup'] },
|
||||
{ cwd: '.', command: 'npm', args: ['run', 'package'] }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
expect(() => assertBuildStepsAllowed(manifest)).toThrow(/`npm run setup` is forbidden/)
|
||||
})
|
||||
|
||||
it('refuses a sourcemap policy it does not implement', () => {
|
||||
expect(assertSourcemapPolicy({ sourcemaps: { policy: 'include' } })).toBe('include')
|
||||
// `delete` named a code path that no longer exists; accepting it would ship
|
||||
// maps that do not match the bundle.
|
||||
expect(() => assertSourcemapPolicy({ sourcemaps: { policy: 'delete' } })).toThrow(
|
||||
/must be one of include, got "delete"/
|
||||
)
|
||||
expect(() => assertSourcemapPolicy({ sourcemaps: { policy: 'exclude' } })).toThrow(
|
||||
/must be one of include, got "exclude"/
|
||||
)
|
||||
expect(() => assertSourcemapPolicy({})).toThrow(/got undefined/)
|
||||
})
|
||||
|
||||
it('stamps the published version into the version source', () => {
|
||||
const source = "export const XTERM_VERSION = '6.0.0';\n"
|
||||
expect(stampVersionSource(source, '6.1.0-beta.287')).toBe(
|
||||
"export const XTERM_VERSION = '6.1.0-beta.287';\n"
|
||||
)
|
||||
expect(() => stampVersionSource('export const OTHER = 1\n', '6.1.0')).toThrow(/XTERM_VERSION/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('lockfile coupling', () => {
|
||||
const lockfile = [
|
||||
'patchedDependencies:',
|
||||
" '@xterm/xterm@6.1.0-beta.287':",
|
||||
` hash: ${'0'.repeat(64)}`,
|
||||
' path: config/patches/@xterm__xterm@6.1.0-beta.287.patch',
|
||||
' node-pty@1.1.0:',
|
||||
` hash: ${'1'.repeat(64)}`,
|
||||
' path: config/patches/node-pty@1.1.0.patch',
|
||||
'snapshots:',
|
||||
` '@xterm/addon-fit@0.12.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=${'0'.repeat(64)}))':`,
|
||||
` '@xterm/xterm': 6.1.0-beta.287(patch_hash=${'0'.repeat(64)})`,
|
||||
` node-pty@1.1.0(patch_hash=${'1'.repeat(64)}):`,
|
||||
''
|
||||
].join('\n')
|
||||
|
||||
it('hashes the patch the way pnpm keys the store directory', () => {
|
||||
expect(patchHash('diff --git a/x b/x\n')).toBe(
|
||||
createHash('sha256').update('diff --git a/x b/x\n').digest('hex')
|
||||
)
|
||||
})
|
||||
|
||||
it('reads quoted and unquoted package keys', () => {
|
||||
expect(readLockfilePatchHash(lockfile, '@xterm/xterm@6.1.0-beta.287')).toBe('0'.repeat(64))
|
||||
expect(readLockfilePatchHash(lockfile, 'node-pty@1.1.0')).toBe('1'.repeat(64))
|
||||
})
|
||||
|
||||
it('rewrites only the targeted entry', () => {
|
||||
const updated = updateLockfilePatchHash(lockfile, '@xterm/xterm@6.1.0-beta.287', 'a'.repeat(64))
|
||||
expect(readLockfilePatchHash(updated, '@xterm/xterm@6.1.0-beta.287')).toBe('a'.repeat(64))
|
||||
expect(readLockfilePatchHash(updated, 'node-pty@1.1.0')).toBe('1'.repeat(64))
|
||||
expect(updated.split('\n')).toHaveLength(lockfile.split('\n').length)
|
||||
})
|
||||
|
||||
// pnpm repeats the hash in every resolution key. Rewriting only patchedDependencies
|
||||
// installs fine on a warm store and drifts on a cold one, so it fails in CI only.
|
||||
it('rewrites the resolution keys as well as patchedDependencies', () => {
|
||||
const key = '@xterm/xterm@6.1.0-beta.287'
|
||||
expect(readLockfileResolutionHashes(lockfile, key)).toEqual(['0'.repeat(64), '0'.repeat(64)])
|
||||
|
||||
const updated = updateLockfilePatchHash(lockfile, key, 'a'.repeat(64))
|
||||
|
||||
expect(readLockfileResolutionHashes(updated, key)).toEqual(['a'.repeat(64), 'a'.repeat(64)])
|
||||
expect(readLockfileResolutionHashes(updated, 'node-pty@1.1.0')).toEqual(['1'.repeat(64)])
|
||||
expect(updated).not.toContain('0'.repeat(64))
|
||||
})
|
||||
|
||||
it('reports a lockfile stale in its resolution keys alone', () => {
|
||||
const key = '@xterm/xterm@6.1.0-beta.287'
|
||||
const halfUpdated = lockfile.replace(`hash: ${'0'.repeat(64)}`, `hash: ${'a'.repeat(64)}`)
|
||||
|
||||
expect(readLockfilePatchHash(halfUpdated, key)).toBe('a'.repeat(64))
|
||||
expect(lockfilePatchHashIsStale(halfUpdated, key, 'a'.repeat(64))).toBe(true)
|
||||
expect(lockfilePatchHashIsStale(lockfile, key, '0'.repeat(64))).toBe(false)
|
||||
})
|
||||
|
||||
it('fails loudly when the package is not patched at all', () => {
|
||||
expect(() => readLockfilePatchHash(lockfile, '@xterm/addon-webgl@0.20.0-beta.286')).toThrow(
|
||||
/no patchedDependencies entry/
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('check-mode reporting', () => {
|
||||
it('points at the source patch instead of the bundle', () => {
|
||||
const message = formatCheckFailure({
|
||||
name: '@xterm/xterm',
|
||||
patchPath: 'config/patches/@xterm__xterm@6.1.0-beta.287.patch',
|
||||
committed: 'diff --git a/lib/x.js b/lib/x.js\n@@ -1 +1 @@\n-a\n+b\n',
|
||||
regenerated: 'diff --git a/lib/x.js b/lib/x.js\n@@ -1 +1 @@\n-a\n+c\n'
|
||||
})
|
||||
expect(message).toContain('Do not edit them')
|
||||
expect(message).toContain('--write')
|
||||
expect(message).toContain('docs/reference/xterm-patch-regeneration.md')
|
||||
expect(message).toContain('files [lib/x.js]')
|
||||
})
|
||||
|
||||
it('locates the first differing character', () => {
|
||||
expect(firstDifferenceIndex('abc', 'abd')).toBe(2)
|
||||
expect(firstDifferenceIndex('abc', 'abc')).toBe(-1)
|
||||
expect(firstDifferenceIndex('abc', 'abcd')).toBe(3)
|
||||
})
|
||||
})
|
||||
|
||||
// These run without network or a build, so ordinary `pnpm test` catches the two
|
||||
// desyncs that would otherwise only surface in the heavy xterm_patch_sync job.
|
||||
describe('committed xterm patch artifacts', () => {
|
||||
it('records the lockfile hash pnpm derives from the patch file', async () => {
|
||||
const manifest = JSON.parse(await readFile(MANIFEST_PATH, 'utf8'))
|
||||
const lockfile = await readFile(path.join(REPO_ROOT, 'pnpm-lock.yaml'), 'utf8')
|
||||
for (const packageEntry of manifest.packages) {
|
||||
const patch = await readFile(path.join(REPO_ROOT, packageEntry.patch), 'utf8')
|
||||
const key = `${packageEntry.name}@${packageEntry.version}`
|
||||
expect(readLockfilePatchHash(lockfile, key)).toBe(patchHash(patch))
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps the source patch and the full patch equal on every source file', async () => {
|
||||
const manifest = JSON.parse(await readFile(MANIFEST_PATH, 'utf8'))
|
||||
for (const packageEntry of manifest.packages) {
|
||||
const patch = await readFile(path.join(REPO_ROOT, packageEntry.patch), 'utf8')
|
||||
const source = await readFile(path.join(REPO_ROOT, packageEntry.sourcePatch), 'utf8')
|
||||
expect(sourceHunks(source)).toBe(sourceHunks(patch))
|
||||
expect(generatedHunks(patch, packageEntry.generatedPaths)).not.toBe('')
|
||||
}
|
||||
})
|
||||
|
||||
it('pins a full upstream commit and a buildable package entry', async () => {
|
||||
const manifest = JSON.parse(await readFile(MANIFEST_PATH, 'utf8'))
|
||||
expect(manifest.upstream.commit).toMatch(/^[0-9a-f]{40}$/)
|
||||
expect(manifest.packages.length).toBeGreaterThan(0)
|
||||
expect(() => assertBuildStepsAllowed(manifest)).not.toThrow()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,244 @@
|
||||
# xterm Patch Regeneration
|
||||
|
||||
## Scope
|
||||
|
||||
Orca ships `@xterm/xterm` with four source changes it needs and upstream has
|
||||
not taken: the IME composition hooks, the `xterm-composition-*` custom events
|
||||
they raise, the `ICompositionHelper` surface those hooks widen, and a `SortedList`
|
||||
fix. pnpm applies them through `config/patches/@xterm__xterm@<version>.patch`.
|
||||
|
||||
That patch touches eight files. Four are hand-authored source
|
||||
(`src/browser/CoreBrowserTerminal.ts`, `src/browser/Types.ts`,
|
||||
`src/browser/input/CompositionHelper.ts`, `src/common/SortedList.ts`) and four
|
||||
are the build output those sources produce (`lib/xterm.js`, `lib/xterm.mjs`,
|
||||
and both sourcemaps). The bundle half is 7.3 MB of minified code. It is
|
||||
generated, and this document exists so nobody edits it by hand.
|
||||
|
||||
The two halves are the same edits diffed two ways, so the generator requires
|
||||
them to match byte for byte on every source file. A hunk the shipped patch
|
||||
cannot name — upstream's `.npmignore` strips `src/**/*.test.ts` — would be
|
||||
dropped by the next `--write`, so it fails the run instead.
|
||||
|
||||
`config/patches/xterm-src/@xterm__xterm@<version>.src.patch` is the source of
|
||||
truth. Everything else is derived from it by
|
||||
`config/scripts/regenerate-xterm-patches.mjs`, which is pinned to the exact
|
||||
upstream commit the published tarball was built from.
|
||||
|
||||
This policy covers `@xterm/xterm` only. The addon patches
|
||||
(`@xterm/addon-webgl`, `@xterm/addon-serialize`) are still hand-edited bundles
|
||||
and are tracked separately; see [Known Gaps](#known-gaps).
|
||||
|
||||
## Rules
|
||||
|
||||
1. Never edit `config/patches/@xterm__xterm@<version>.patch`. Edit the source
|
||||
patch and regenerate.
|
||||
2. Never edit `lib/` inside a patched `node_modules` tree and re-run
|
||||
`pnpm patch-commit`. That is how bundle hunks stop matching their sources.
|
||||
3. Every source change must land together with the regenerated bundle hunks and
|
||||
the `pnpm-lock.yaml` hash bump, in one commit.
|
||||
4. The upstream commit lives in `config/patches/xterm-upstream.json`, not in a
|
||||
comment. A version bump that leaves it stale fails the generator, it does not
|
||||
silently patch the wrong tree.
|
||||
5. Sourcemaps move with the bundle, and are never silently omitted. The patch
|
||||
moves the code, so dropping only the map hunks would ship offsets pointing at
|
||||
the wrong lines. `sourcemaps.policy` accepts `include` and nothing else: it
|
||||
costs about 5.8 MB of the emitted patch and is required because
|
||||
`src/renderer/src/components/terminal-pane/terminal-ime-xterm-transaction-events.test.ts`
|
||||
reads `lib/*.map` and asserts the mapped `Version.ts` matches the runtime
|
||||
version. Deleting the maps was once an option; the code that did it was
|
||||
removed as unreachable, so re-adding the policy means re-adding that code.
|
||||
6. `--check` is the authority on the lockfile, not `pnpm install`. pnpm writes the
|
||||
patch hash in two places — `patchedDependencies` and every resolution key that
|
||||
depends on the patched package — and on a warm store it will leave the
|
||||
resolution keys at their previous value while reporting success. That installs
|
||||
locally and drifts on CI's cold store. Always finish on step 4, and if it
|
||||
reports a stale hash after an install, rerun `--write`.
|
||||
|
||||
## Workflow
|
||||
|
||||
```sh
|
||||
# 1. Edit the source hunks.
|
||||
$EDITOR config/patches/xterm-src/@xterm__xterm@6.1.0-beta.287.src.patch
|
||||
|
||||
# 2. Rebuild the bundle hunks, the full patch, and the lockfile hash.
|
||||
node config/scripts/regenerate-xterm-patches.mjs --write
|
||||
|
||||
# 3. Reinstall so node_modules picks up the new patch hash.
|
||||
pnpm install
|
||||
|
||||
# 4. Confirm the tree is self-consistent.
|
||||
node config/scripts/regenerate-xterm-patches.mjs --check
|
||||
```
|
||||
|
||||
Editing a patch file by hand is awkward for anything larger than a one-liner.
|
||||
For a substantial change, work in the generator's own checkout instead — after
|
||||
any run it is left at the pinned commit with the source patch applied:
|
||||
|
||||
```sh
|
||||
node config/scripts/regenerate-xterm-patches.mjs --check --work-dir=/tmp/xterm
|
||||
$EDITOR /tmp/xterm/upstream/src/browser/input/CompositionHelper.ts
|
||||
git -C /tmp/xterm/upstream diff -- src/ > config/patches/xterm-src/@xterm__xterm@6.1.0-beta.287.src.patch
|
||||
node config/scripts/regenerate-xterm-patches.mjs --write --work-dir=/tmp/xterm
|
||||
```
|
||||
|
||||
`--write` rewrites the source patch into the canonical form it would emit on a
|
||||
re-diff, so a hand-produced `git diff` gets normalized on the first run rather
|
||||
than fighting `--check` forever.
|
||||
|
||||
Run the checkout outside this repository. A build tree underneath it makes
|
||||
`tsgo` walk up into Orca's own `node_modules` and fail with `TS2300: Duplicate
|
||||
identifier`, which is a symptom of where the tree sits and not of the patch.
|
||||
|
||||
## How the Commit Is Known
|
||||
|
||||
Upstream `bin/publish.js` sets `packageJson.commit` before `npm publish`, so
|
||||
each published tarball names the commit that built it. The generator asserts
|
||||
that stamp against `xterm-upstream.json` and then compares the tarball's `src/`
|
||||
against the checkout file by file. Only `src/common/Version.ts` may differ,
|
||||
because `publish.js` rewrites the version immediately before packaging; the
|
||||
generator applies the same stamp.
|
||||
|
||||
That pair of checks is what makes the rebuild trustworthy. Without them a wrong
|
||||
commit would still produce a plausible-looking 7 MB patch.
|
||||
|
||||
## Build Order
|
||||
|
||||
Upstream's publish path is `npm ci` → stamp `Version.ts` → `npm run package`.
|
||||
`npm run package` runs webpack for `lib/xterm.js` and then, via `postpackage`,
|
||||
`bin/esbuild_all.mjs --prod` for `lib/xterm.mjs`.
|
||||
|
||||
**Do not run `npm run setup` after the packaging build.** `setup` is the
|
||||
development esbuild pass with `minify: false`. Running it afterwards overwrites
|
||||
`lib/xterm.mjs` with an unminified bundle and a map that no longer matches, and
|
||||
the resulting patch is silently wrong — the failure mode is a `.mjs` that is
|
||||
50% larger than the published one, which is easy to miss inside a 7 MB diff.
|
||||
`forbiddenBuildScripts` in the manifest encodes this and the generator refuses
|
||||
to run a build step that names one of those scripts.
|
||||
|
||||
The generator also builds the *unmodified* commit first and asserts that it
|
||||
reproduces the published `lib/` byte for byte before it emits anything. A
|
||||
toolchain or build-order problem therefore surfaces as an explicit "did not
|
||||
reproduce the published bundles" error rather than as 7 MB of mystery diff.
|
||||
|
||||
## Recovering From Hand-Edited Bundles
|
||||
|
||||
Between 2026-08-09 and 2026-08-17 this harness did not exist, and four fixes
|
||||
landed by editing the minified bundles directly. The tell is code no minifier
|
||||
emits: `const` in an otherwise `let`-only bundle, and identifiers like `$rl`,
|
||||
`$hp`, `$tid`.
|
||||
|
||||
Recovery is not a rewrite. The hand-edits were applied to `src/` as well, so the
|
||||
source hunks in the shipped patch were already correct and `--write` re-derives
|
||||
the bundles from them. What changes is cosmetic and expected:
|
||||
|
||||
- Hand-written locals collapse back into minifier names, which shifts esbuild's
|
||||
frequency-ordered allocation and can swap two short names bundle-wide (`i`↔`t`
|
||||
in the `.mjs`, `w`↔`y` in the `.js`). Most differing lines are the same length.
|
||||
- Hand-written equivalents normalize to what the toolchain actually emits
|
||||
(`!!x` back to `Boolean(x)`, an escaped `\u200E` back to the literal
|
||||
character).
|
||||
|
||||
To confirm a regeneration is semantically a no-op rather than a revert, compare
|
||||
identifier multisets between the old and new bundle instead of reading the diff:
|
||||
every name that is not a single-letter minifier local should appear the same
|
||||
number of times in both. Anything else is a real change and needs explaining.
|
||||
|
||||
## The Lockfile Moves With the Patch
|
||||
|
||||
pnpm derives the `patchedDependencies` hash in `pnpm-lock.yaml` — and the
|
||||
`.pnpm/@xterm+xterm@<version>_patch_hash=<hash>/` store directory name — from
|
||||
the sha256 of the patch file itself. A regenerated patch without the lockfile
|
||||
bump fails `pnpm install --frozen-lockfile` on every machine except the
|
||||
author's. `--write` makes that edit; `--check` fails if it is missing.
|
||||
|
||||
`config/scripts/regenerate-xterm-patches.test.mjs` asserts the same thing
|
||||
without a network or a build, so the ordinary test job catches lockfile drift
|
||||
in milliseconds even though the full rebuild runs in its own CI lane.
|
||||
|
||||
## Toolchain Pin
|
||||
|
||||
`toolchain` in the manifest records what upstream's `package-lock.json` resolves
|
||||
at the pinned commit, and the generator fails if `npm ci` produces something
|
||||
else. The entry that matters is `@typescript/native-preview`
|
||||
(`tsgo`), which upstream pins to a **dated development build** —
|
||||
`7.0.0-dev.20260521.1` at the time of writing. It is a real published version
|
||||
and npm does not prune old releases, but it is the one dependency of this scheme
|
||||
that is not a stable release.
|
||||
|
||||
If that version ever becomes unresolvable the generator fails with a toolchain
|
||||
error naming it. Recovery is to move the pin to the next upstream commit whose
|
||||
`package-lock.json` resolves, re-verify that the rebuild still reproduces the
|
||||
published bundles, and regenerate. The committed patch keeps working the whole
|
||||
time — only regeneration is blocked, so this is never an outage.
|
||||
|
||||
## Version Bumps
|
||||
|
||||
Bumping `@xterm/xterm` is:
|
||||
|
||||
1. Update the version in `package.json` and run `pnpm install`.
|
||||
2. Rename both patch files to the new version and update `patch`,
|
||||
`sourcePatch`, and `version` in `xterm-upstream.json`.
|
||||
3. Update `upstream.commit` to the `commit` field of the new tarball's
|
||||
`package.json`, and `toolchain` to whatever the new `package-lock.json`
|
||||
resolves.
|
||||
4. `node config/scripts/regenerate-xterm-patches.mjs --write`.
|
||||
|
||||
Step 4 is where a real upstream conflict shows up: `git apply` of the source
|
||||
patch fails against the new tree. Resolve it in the checkout, re-diff, and
|
||||
rerun. The bundle hunks need no attention at any point.
|
||||
|
||||
## Why Not Vendor a Fork
|
||||
|
||||
A vendored `@xterm/xterm` fork removes the patch entirely, but it moves Orca off
|
||||
the published package, so every upstream beta becomes a merge rather than a
|
||||
version bump, and Orca inherits responsibility for building and publishing a
|
||||
package it does not own. The patch is four small source hunks against a commit
|
||||
that reproduces byte for byte; a fork is a much larger standing cost for the
|
||||
same result.
|
||||
|
||||
## Why Not Handle Composition at Runtime
|
||||
|
||||
`CompositionHelper` hooks four private call sites upstream of `onData`, and
|
||||
`SortedList` has no public surface at all. There is no supported extension point
|
||||
that reaches either, so a runtime shim would mean reaching into `_core`
|
||||
internals that upstream renames freely between betas. The patch is the smaller
|
||||
risk.
|
||||
|
||||
## CI Contract
|
||||
|
||||
`xterm_patch_sync` in `.github/workflows/pr.yml` runs
|
||||
`regenerate-xterm-patches.mjs --check` on every PR and is part of the `verify`
|
||||
aggregate. It clones the pinned commit, installs upstream's toolchain, builds
|
||||
twice, and byte-compares the result against the committed patch. Both builds and
|
||||
the diff together are about eight seconds; `npm ci` for upstream's toolchain is
|
||||
what the job actually spends its minutes on, and the cache key is the manifest.
|
||||
|
||||
`config/scripts/regenerate-xterm-patches.test.mjs` covers the pure pieces —
|
||||
pnpm's diff flags and normalization, hunk splitting, round-trip stability, the
|
||||
commit and build-order assertions, and lockfile coupling — with no network and
|
||||
no build, so they run in the ordinary test shards.
|
||||
|
||||
## Known Gaps
|
||||
|
||||
`@xterm/addon-webgl` and `@xterm/addon-serialize` are still hand-edited minified
|
||||
bundles. Their patches carry a literal `/* PATCH(orca): ... */` comment inside
|
||||
minified code and parser round-trip artifacts, and neither patch touches its
|
||||
`.map` file, so both addons currently ship sourcemaps whose offsets do not match
|
||||
the shipped bundle — the defect `sourcemaps.policy` rules out for `@xterm/xterm`
|
||||
and which folding them into this manifest would also fix.
|
||||
|
||||
Both addons do build from the pinned commit: the registry stamps
|
||||
`53a98a720ae4a973e384fa2440880d09537132f3` on `addon-webgl@0.20.0-beta.286` and
|
||||
`addon-serialize@0.15.0-beta.287` alike, despite the mismatched version numbers.
|
||||
On 2026-08-17 their published `lib/*.mjs` and `lib/*.mjs.map` reproduced byte for
|
||||
byte from that commit. That was a one-off measurement, not an invariant: no check
|
||||
in this repo re-runs it, so treat it as a starting point to re-measure rather than
|
||||
as something the harness holds true.
|
||||
|
||||
What blocks folding them in is the other half of their published output. Both
|
||||
also ship a CJS `lib/addon-*.js`, and the root `package` script does not build
|
||||
it — upstream's webpack entry is core's `Terminal.js` only, and `postpackage`
|
||||
emits ESM. So a manifest entry for either addon needs a build step this harness
|
||||
does not have, and the CJS halves are **unverified**: nothing here has yet
|
||||
reproduced them from source. Until that exists, folding them in would emit a
|
||||
patch whose CJS stanza came from the current hand-edited bundle.
|
||||
Generated
+24
-24
@@ -18,7 +18,7 @@ patchedDependencies:
|
||||
hash: 6da7d7770b6427246f2a0d057d97da418040e498068b41d0c2d3c6b20bf49258
|
||||
path: config/patches/@xterm__addon-webgl@0.20.0-beta.286.patch
|
||||
'@xterm/xterm@6.1.0-beta.287':
|
||||
hash: 0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067
|
||||
hash: 7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4
|
||||
path: config/patches/@xterm__xterm@6.1.0-beta.287.patch
|
||||
node-pty@1.1.0:
|
||||
hash: 8fc49f17011b6611a5b8c00e83a6f12e14e75aada2b0ef26dc5393f8376d20e8
|
||||
@@ -45,7 +45,7 @@ importers:
|
||||
version: 2.5.6
|
||||
'@xterm/addon-serialize':
|
||||
specifier: 0.15.0-beta.287
|
||||
version: 0.15.0-beta.287(patch_hash=af3b156143ed2ee9903b753145b63dd4a2ee72cadd7ef333ab0f4d5d3ebfc32c)(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.15.0-beta.287(patch_hash=af3b156143ed2ee9903b753145b63dd4a2ee72cadd7ef333ab0f4d5d3ebfc32c)(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/headless':
|
||||
specifier: 6.1.0-beta.287
|
||||
version: 6.1.0-beta.287
|
||||
@@ -208,25 +208,25 @@ importers:
|
||||
version: 5.2.0(rolldown-vite@7.3.1(@emnapi/core@1.11.2)(@emnapi/runtime@1.11.2)(@types/node@25.9.5)(jiti@2.7.0)(yaml@2.8.4))
|
||||
'@xterm/addon-fit':
|
||||
specifier: 0.12.0-beta.287
|
||||
version: 0.12.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.12.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/addon-ligatures':
|
||||
specifier: 0.11.0-beta.287
|
||||
version: 0.11.0-beta.287(patch_hash=47405b9994b5acf1b4e90b49250358c1ca03649854d59560e7732b72fe336920)(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.11.0-beta.287(patch_hash=47405b9994b5acf1b4e90b49250358c1ca03649854d59560e7732b72fe336920)(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/addon-search':
|
||||
specifier: 0.17.0-beta.287
|
||||
version: 0.17.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.17.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/addon-unicode11':
|
||||
specifier: 0.10.0-beta.287
|
||||
version: 0.10.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.10.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/addon-web-links':
|
||||
specifier: 0.13.0-beta.287
|
||||
version: 0.13.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.13.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/addon-webgl':
|
||||
specifier: 0.20.0-beta.286
|
||||
version: 0.20.0-beta.286(patch_hash=6da7d7770b6427246f2a0d057d97da418040e498068b41d0c2d3c6b20bf49258)(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))
|
||||
version: 0.20.0-beta.286(patch_hash=6da7d7770b6427246f2a0d057d97da418040e498068b41d0c2d3c6b20bf49258)(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))
|
||||
'@xterm/xterm':
|
||||
specifier: 6.1.0-beta.287
|
||||
version: 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
version: 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
class-variance-authority:
|
||||
specifier: ^0.7.1
|
||||
version: 0.7.1
|
||||
@@ -9591,39 +9591,39 @@ snapshots:
|
||||
|
||||
'@xmldom/xmldom@0.8.13': {}
|
||||
|
||||
'@xterm/addon-fit@0.12.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-fit@0.12.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
|
||||
'@xterm/addon-ligatures@0.11.0-beta.287(patch_hash=47405b9994b5acf1b4e90b49250358c1ca03649854d59560e7732b72fe336920)(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-ligatures@0.11.0-beta.287(patch_hash=47405b9994b5acf1b4e90b49250358c1ca03649854d59560e7732b72fe336920)(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
lru-cache: 11.5.1
|
||||
opentype.js: 2.0.0
|
||||
|
||||
'@xterm/addon-search@0.17.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-search@0.17.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
|
||||
'@xterm/addon-serialize@0.15.0-beta.287(patch_hash=af3b156143ed2ee9903b753145b63dd4a2ee72cadd7ef333ab0f4d5d3ebfc32c)(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-serialize@0.15.0-beta.287(patch_hash=af3b156143ed2ee9903b753145b63dd4a2ee72cadd7ef333ab0f4d5d3ebfc32c)(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
|
||||
'@xterm/addon-unicode11@0.10.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-unicode11@0.10.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
|
||||
'@xterm/addon-web-links@0.13.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-web-links@0.13.0-beta.287(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
|
||||
'@xterm/addon-webgl@0.20.0-beta.286(patch_hash=6da7d7770b6427246f2a0d057d97da418040e498068b41d0c2d3c6b20bf49258)(@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067))':
|
||||
'@xterm/addon-webgl@0.20.0-beta.286(patch_hash=6da7d7770b6427246f2a0d057d97da418040e498068b41d0c2d3c6b20bf49258)(@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4))':
|
||||
dependencies:
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)
|
||||
'@xterm/xterm': 6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)
|
||||
|
||||
'@xterm/headless@6.1.0-beta.287': {}
|
||||
|
||||
'@xterm/xterm@6.1.0-beta.287(patch_hash=0793ce045b9ec6e19c5802658e0b28aec89517e6986c496515c4d8def083e067)': {}
|
||||
'@xterm/xterm@6.1.0-beta.287(patch_hash=7fd86b33ead4457937d9ba9a406a8f0c26de4d69c4dc545c010c7c42ed6c0ba4)': {}
|
||||
|
||||
abbrev@4.0.0: {}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user