mirror of
https://github.com/stablyai/orca.git
synced 2026-10-18 08:03:22 +00:00
feat(ssh): compose bounded Windows relay tree transfer
Co-authored-by: Orca <help@stably.ai>
This commit is contained in:
@@ -290,6 +290,7 @@ jobs:
|
||||
src/main/ssh/ssh-relay-runtime-posix-file-destination.test.ts \
|
||||
src/main/ssh/ssh-relay-runtime-windows-file-destination.test.ts \
|
||||
src/main/ssh/ssh-relay-runtime-windows-staging-control.test.ts \
|
||||
src/main/ssh/ssh-relay-runtime-windows-tree-transfer.test.ts \
|
||||
src/main/ssh/ssh-relay-runtime-posix-tree-transfer.test.ts \
|
||||
src/main/ssh/ssh-relay-runtime-system-ssh-file-channel.test.ts \
|
||||
src/main/ssh/ssh-relay-runtime-sftp-file-destination.test.ts \
|
||||
@@ -1031,6 +1032,7 @@ jobs:
|
||||
src/main/ssh/ssh-relay-runtime-posix-file-destination.test.ts `
|
||||
src/main/ssh/ssh-relay-runtime-windows-file-destination.test.ts `
|
||||
src/main/ssh/ssh-relay-runtime-windows-staging-control.test.ts `
|
||||
src/main/ssh/ssh-relay-runtime-windows-tree-transfer.test.ts `
|
||||
src/main/ssh/ssh-relay-runtime-posix-tree-transfer.test.ts `
|
||||
src/main/ssh/ssh-relay-runtime-system-ssh-file-channel.test.ts `
|
||||
src/main/ssh/ssh-relay-runtime-sftp-file-destination.test.ts `
|
||||
|
||||
@@ -204,6 +204,7 @@ describe('SSH relay runtime artifact workflow', () => {
|
||||
'ssh-relay-runtime-posix-file-destination.test.ts',
|
||||
'ssh-relay-runtime-windows-file-destination.test.ts',
|
||||
'ssh-relay-runtime-windows-staging-control.test.ts',
|
||||
'ssh-relay-runtime-windows-tree-transfer.test.ts',
|
||||
'ssh-relay-runtime-posix-tree-transfer.test.ts',
|
||||
'ssh-relay-runtime-system-ssh-file-channel.test.ts',
|
||||
'ssh-relay-runtime-sftp-file-destination.test.ts',
|
||||
|
||||
+11
-6
@@ -10,12 +10,17 @@ A checked box means the work has evidence in the detailed ledger. Design approva
|
||||
complete a box.
|
||||
|
||||
Active checkpoint: **Milestone 6 / Work Package 5 bounded runtime transfer — disconnected Windows
|
||||
system-SSH tree-composition contract audit, 2026-07-15, Codex implementation owner.**
|
||||
The next slice is limited to composing the proven staging control, per-file receiver, source stream,
|
||||
and existing system-SSH channel adapter into one exclusive Windows staging tree with parent-first
|
||||
directories, bounded concurrency/progress, and joined owned-root cleanup. Live OpenSSH, product
|
||||
callers, legacy upload/fallback/default behavior, tuple enablement, publication, and SignPath remain
|
||||
out of scope.
|
||||
system-SSH tree-composition native CI proof, 2026-07-15, Codex implementation owner.**
|
||||
The local package is green; push its isolated commit and require exact-head all-six native artifact
|
||||
and adjacent workflow evidence before auditing any live/full-size Windows OpenSSH package. Live
|
||||
OpenSSH, product callers, legacy upload/fallback/default behavior, tuple enablement, publication,
|
||||
and SignPath remain out of scope.
|
||||
`E-M6-WINDOWS-SYSTEM-SSH-TREE-LOCAL-RED-001` proves the native workflow oracle passes 9/9 and the
|
||||
purpose suite fails solely because the audited production tree-composition module is absent.
|
||||
`E-M6-WINDOWS-SYSTEM-SSH-TREE-LOCAL-001` records 8/8 purpose and 9/9 workflow-oracle cases, 258
|
||||
focused passes with three declared skips, 694 broad relay passes with nine declared skips, 282/282
|
||||
release contracts, typecheck, full lint, formatting, max-lines, diff, protected-resolver, and
|
||||
no-product-consumer gates. Exact-head native CI remains required.
|
||||
`E-M6-WINDOWS-SYSTEM-SSH-CONTROL-LOCAL-RED-001` proves the native workflow oracle passes 9/9 and
|
||||
the purpose suite fails solely because the audited production staging-control module is absent.
|
||||
`E-M6-WINDOWS-SYSTEM-SSH-CONTROL-LOCAL-001` records 16/17 purpose cases locally with the real
|
||||
|
||||
+75
-2
@@ -8,8 +8,8 @@ work; keep exact commands, runner identities, hashes, metrics, and residual gaps
|
||||
|
||||
Date created: 2026-07-14<br>
|
||||
Last updated: 2026-07-15<br>
|
||||
Current phase: Milestone 6 / Work Package 5 bounded runtime transfer — **In progress — 2026-07-15, Codex implementation owner: disconnected Windows system-SSH tree-composition contract audit**. Keep every capability disconnected from every production/default SSH path. Real Apple/SignPath rehearsals remain an explicit late gate and are not on this package's critical path.<br>
|
||||
Session checkpoint: **WINDOWS STAGING CONTROL NATIVE PROOF CLOSED / TREE COMPOSITION AUDIT ACTIVE — 2026-07-15, Codex implementation owner** — `E-M6-WINDOWS-SYSTEM-SSH-CONTROL-CI-001` proves the real PowerShell staging-control contract on native Windows x64/arm64 and all adjacent exact-head gates, with only the declared hosted Windows arm64 floor gap. Audit the next disconnected tree owner before writing RED tests. Do not connect a product caller, change legacy upload/fallback/default behavior, enable a tuple, publish an artifact, or perform SignPath work. Merging to `main` remains prohibited, SignPath is deferred, and legacy remains the production default.<br>
|
||||
Current phase: Milestone 6 / Work Package 5 bounded runtime transfer — **In progress — 2026-07-15, Codex implementation owner: disconnected Windows system-SSH tree-composition native CI proof**. Keep every capability disconnected from every production/default SSH path. Real Apple/SignPath rehearsals remain an explicit late gate and are not on this package's critical path.<br>
|
||||
Session checkpoint: **WINDOWS TREE COMPOSITION LOCAL GREEN / EXACT-HEAD NATIVE CI NEXT — 2026-07-15, Codex implementation owner** — `E-M6-WINDOWS-SYSTEM-SSH-TREE-LOCAL-001` proves the disconnected composer and every audited local lifecycle gate. Commit and push the isolated package, then require all-six native artifact and adjacent exact-head workflows before auditing live/full-size Windows OpenSSH. Do not connect a product caller, change legacy upload/fallback/default behavior, enable a tuple, publish an artifact, or perform SignPath work. Merging to `main` remains prohibited, SignPath is deferred, and legacy remains the production default.<br>
|
||||
Primary design: [SSH relay GitHub Release plan](./2026-07-14-ssh-relay-github-release-plan.html)<br>
|
||||
Motivating issues: [#8450](https://github.com/stablyai/orca/issues/8450), [#1693](https://github.com/stablyai/orca/issues/1693)
|
||||
|
||||
@@ -18411,6 +18411,79 @@ src/main/ssh/ssh-remote-node-resolution.test.ts` is empty. A non-test reference
|
||||
publication/default behavior, or SignPath. Legacy remains the sole production/default path and
|
||||
every tuple stays disabled.
|
||||
|
||||
### E-M6-WINDOWS-SYSTEM-SSH-TREE-LOCAL-RED-001 — audited tree composer is absent
|
||||
|
||||
- Date/owner/base: 2026-07-15, Codex implementation owner; uncommitted test/workflow package atop
|
||||
`c39e20cc6ccdfc66dbac25bd17d86fd4e53fdacd`.
|
||||
- Format command:
|
||||
`pnpm exec oxfmt --check .github/workflows/ssh-relay-runtime-artifacts.yml config/scripts/ssh-relay-runtime-workflow.test.mjs src/main/ssh/ssh-relay-runtime-windows-tree-transfer.test.ts docs/reference/plans/2026-07-14-ssh-relay-github-release-implementation-checklist.md docs/reference/plans/2026-07-14-ssh-relay-github-release-implementation-checklist-summary.md`
|
||||
passes all five files in 9.616 seconds.
|
||||
- RED command:
|
||||
`pnpm exec vitest run --config config/vitest.config.ts --maxWorkers=1 src/main/ssh/ssh-relay-runtime-windows-tree-transfer.test.ts config/scripts/ssh-relay-runtime-workflow.test.mjs`.
|
||||
The workflow oracle passes 9/9 in 215 ms. The purpose suite collects zero tests and fails at its
|
||||
production import only with `Cannot find module '/src/main/ssh/ssh-relay-runtime-windows-tree-transfer'`;
|
||||
there is no assertion, fixture, workflow-oracle, or adjacent failure.
|
||||
- Disposition: the RED state matches `E-M6-WINDOWS-SYSTEM-SSH-TREE-AUDIT-001`. Implement only the
|
||||
disconnected composer and expand the green suite to every audited lifecycle case. No product
|
||||
consumer, live Windows OpenSSH claim, default/fallback change, tuple enablement, publication, or
|
||||
SignPath work is authorized by this evidence.
|
||||
|
||||
### E-M6-WINDOWS-SYSTEM-SSH-TREE-LOCAL-001 — disconnected tree composer is locally green
|
||||
|
||||
- Date/owner/base/runner: 2026-07-15, Codex implementation owner; uncommitted implementation,
|
||||
purpose-test, native-workflow, and checklist package atop exact base
|
||||
`c39e20cc6ccdfc66dbac25bd17d86fd4e53fdacd` on Apple arm64, macOS 26.2 build 25C56, Node
|
||||
26.5.0, pnpm 10.24.0. Local Node is newer than the repository's Node 24 contract, so exact-head
|
||||
all-six native proof remains mandatory.
|
||||
- Implementation: add one disconnected Windows tree owner over the proven staging-control,
|
||||
per-file receiver, source-stream scheduler, and authenticated system-SSH channel adapter. It
|
||||
validates a Windows scanned tree, rooted drive/UNC staging path, system transport, exact signal,
|
||||
and one-to-four concurrency before I/O; defaults to one file; creates one exclusive root and every
|
||||
declared directory parent-first; streams exact descendant paths/sizes; and returns only frozen
|
||||
aggregate progress. Root ownership begins only after successful exclusive creation. Later failure
|
||||
or cancellation joins all source workers before a separate five-second owned-root cleanup, and
|
||||
joins primary plus cleanup errors.
|
||||
- Purpose command: `/usr/bin/time -l pnpm exec vitest run --config config/vitest.config.ts
|
||||
--maxWorkers=1 src/main/ssh/ssh-relay-runtime-windows-tree-transfer.test.ts
|
||||
config/scripts/ssh-relay-runtime-workflow.test.mjs`. Result: two files and 17/17 cases pass: 8/8
|
||||
composer lifecycle cases plus the 9/9 workflow oracle; 2.17 seconds real, 139,853,824-byte maximum
|
||||
RSS, 98,866,936-byte peak footprint, zero swaps and block I/O. Proof covers hostile/pre-aborted
|
||||
input before I/O, exact root/parent-first directories/files/bytes/counts/path-free progress,
|
||||
default-one and maximum-four destinations, collision preservation, directory/file failure,
|
||||
retained-write cancellation before cleanup, joined cleanup failure, five-second cleanup timeout,
|
||||
and no later progress.
|
||||
- Focused command: `/usr/bin/time -l pnpm exec vitest run --config config/vitest.config.ts
|
||||
--maxWorkers=1` with the Windows tree/control/file, POSIX tree/control/file, system-channel,
|
||||
source-tree/scan/stream, system-fallback, connection, and workflow-oracle suites. Result: 13 files,
|
||||
258 passed and three declared platform skips out of 261; 20.40 seconds real, 186,826,752-byte
|
||||
maximum RSS, 97,179,360-byte peak footprint, zero swaps and block I/O.
|
||||
- Broad relay command: `/usr/bin/time -l pnpm exec vitest run --config config/vitest.config.ts
|
||||
--maxWorkers=1 src/main/ssh/ssh-relay-*.test.ts`. Result: 53 files passed, five files skipped; 694
|
||||
passed and nine declared skips out of 703; 46.75 seconds real, 324,173,824-byte maximum RSS,
|
||||
97,146,592-byte peak footprint, zero swaps and block I/O.
|
||||
- Release-contract command: `/usr/bin/time -l pnpm exec vitest run --config
|
||||
config/vitest.config.ts --maxWorkers=1 config/scripts/ssh-relay-runtime-*.test.mjs`. Result: 50
|
||||
files and 282/282 cases pass; 19.62 seconds real, 195,706,880-byte maximum RSS, 97,900,328-byte
|
||||
peak footprint, zero swaps and block I/O.
|
||||
- Static evidence: final-code `pnpm typecheck` passes. `/usr/bin/time -l pnpm lint` passes in 23.95
|
||||
seconds real with 2,073,870,336-byte maximum RSS and 97,949,456-byte peak footprint; its 26
|
||||
warnings pre-exist outside this diff. Full lint proves switch exhaustiveness, 41 reliability gates,
|
||||
the 355-entry max-lines ratchet with no new bypass, bundled guides, and localization checks.
|
||||
Targeted `oxlint` and six-file `oxfmt --check` pass.
|
||||
- Isolation evidence: `git diff --check` passes; `git diff --exit-code HEAD --
|
||||
src/main/ssh/ssh-remote-node-resolution.ts src/main/ssh/ssh-remote-node-resolution.test.ts` is
|
||||
empty. An asserted non-test reference search finds the new export only in its owning production
|
||||
module. The intended diff is limited to the disconnected composer/purpose test, its one entry in
|
||||
each native workflow command family and oracle, and both checklist documents. No legacy uploader,
|
||||
Electron/startup/product importer, settings, Beta mode, fallback, tuple, publication, default-path,
|
||||
or signing behavior changes.
|
||||
- Residual boundary: this is adapter-backed composition, not live Windows OpenSSH, a full-size
|
||||
runtime transfer/measurement, long-path or endpoint-protection proof, primitive classification/
|
||||
cache, remote bundled-Node verification/install/publish/launch, product/Beta/fallback/default
|
||||
behavior, tuple enablement, publication, Apple signing, or SignPath. Exact-head all-six native and
|
||||
adjacent workflow proof remains required before this package closes. Legacy remains the sole
|
||||
production/default path and every tuple stays disabled.
|
||||
|
||||
## Accepted Gaps
|
||||
|
||||
No product gap is accepted merely because it appears in this list. Each entry requires explicit
|
||||
|
||||
@@ -0,0 +1,390 @@
|
||||
import { mkdir, mkdtemp, readFile, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
import { publishSshRelayArtifactCacheEntry } from './ssh-relay-artifact-cache-entry'
|
||||
import { createSshRelayArtifactCacheEntryFixture } from './ssh-relay-artifact-cache-entry-fixture'
|
||||
import {
|
||||
acquireSshRelayArtifactCacheInUseLease,
|
||||
type SshRelayArtifactCacheInUseLease
|
||||
} from './ssh-relay-artifact-cache-in-use-lease'
|
||||
import { scanSshRelayRuntimeSourceTree } from './ssh-relay-runtime-source-scan'
|
||||
import { createSshRelayRuntimeSourceTree } from './ssh-relay-runtime-source-tree'
|
||||
import type { SshRelayRuntimeSystemSshConnection } from './ssh-relay-runtime-system-ssh-file-channel'
|
||||
import {
|
||||
transferSshRelayRuntimeTreeViaWindowsSystemSsh,
|
||||
SSH_RELAY_RUNTIME_WINDOWS_TREE_TRANSFER_LIMITS
|
||||
} from './ssh-relay-runtime-windows-tree-transfer'
|
||||
|
||||
const stagingControl = vi.hoisted(() => vi.fn())
|
||||
const openFileDestination = vi.hoisted(() => vi.fn())
|
||||
|
||||
vi.mock('./ssh-relay-runtime-windows-staging-control', () => ({
|
||||
runSshRelayRuntimeWindowsStagingControl: stagingControl
|
||||
}))
|
||||
vi.mock('./ssh-relay-runtime-windows-file-destination', () => ({
|
||||
openSshRelayRuntimeWindowsFileDestination: openFileDestination
|
||||
}))
|
||||
|
||||
const cleanupRoots = new Set<string>()
|
||||
const cleanupLeases = new Set<SshRelayArtifactCacheInUseLease>()
|
||||
|
||||
async function treeFixture(os: 'win32' | 'linux' = 'win32') {
|
||||
const root = await mkdtemp(join(tmpdir(), 'orca-relay-windows-tree-'))
|
||||
cleanupRoots.add(root)
|
||||
const inputRoot = join(root, 'input')
|
||||
await mkdir(inputRoot)
|
||||
const fixture = await createSshRelayArtifactCacheEntryFixture({ root: inputRoot, os })
|
||||
const cacheRoot = join(root, 'cache')
|
||||
const entry = await publishSshRelayArtifactCacheEntry({
|
||||
cacheRoot,
|
||||
artifact: fixture.artifact,
|
||||
archivePath: fixture.archivePath
|
||||
})
|
||||
const lease = await acquireSshRelayArtifactCacheInUseLease({ cacheRoot, entry })
|
||||
cleanupLeases.add(lease)
|
||||
const source = createSshRelayRuntimeSourceTree({
|
||||
kind: 'ready',
|
||||
source: 'cache',
|
||||
artifact: fixture.artifact,
|
||||
entry,
|
||||
lease
|
||||
})
|
||||
return scanSshRelayRuntimeSourceTree(source, new AbortController().signal)
|
||||
}
|
||||
|
||||
function connection(system = true): SshRelayRuntimeSystemSshConnection {
|
||||
return { usesSystemSshTransport: () => system, exec: vi.fn() }
|
||||
}
|
||||
|
||||
function retainDestinationWrites() {
|
||||
let active = 0
|
||||
let peak = 0
|
||||
let holding = true
|
||||
const pending = new Set<() => void>()
|
||||
openFileDestination.mockImplementation(async ({ signal }: { signal: AbortSignal }) => {
|
||||
active += 1
|
||||
peak = Math.max(peak, active)
|
||||
let owned = true
|
||||
const settle = (): void => {
|
||||
if (owned) {
|
||||
owned = false
|
||||
active -= 1
|
||||
}
|
||||
}
|
||||
return {
|
||||
write: vi.fn(() => {
|
||||
if (!holding) {
|
||||
return Promise.resolve()
|
||||
}
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
const release = (): void => {
|
||||
signal.removeEventListener('abort', onAbort)
|
||||
pending.delete(release)
|
||||
resolve()
|
||||
}
|
||||
const onAbort = (): void => {
|
||||
pending.delete(release)
|
||||
reject(signal.reason)
|
||||
}
|
||||
pending.add(release)
|
||||
signal.addEventListener('abort', onAbort, { once: true })
|
||||
})
|
||||
}),
|
||||
close: vi.fn(async () => settle()),
|
||||
abort: vi.fn(async () => settle())
|
||||
}
|
||||
})
|
||||
return {
|
||||
peak: () => peak,
|
||||
release: () => {
|
||||
holding = false
|
||||
for (const release of pending) {
|
||||
release()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
stagingControl.mockReset().mockResolvedValue(undefined)
|
||||
openFileDestination.mockReset().mockImplementation(async () => ({
|
||||
write: vi.fn(async () => {}),
|
||||
close: vi.fn(async () => {}),
|
||||
abort: vi.fn(async () => {})
|
||||
}))
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
vi.useRealTimers()
|
||||
await Promise.all([...cleanupLeases].map((lease) => lease.release().catch(() => {})))
|
||||
cleanupLeases.clear()
|
||||
await Promise.all([...cleanupRoots].map((root) => rm(root, { recursive: true, force: true })))
|
||||
cleanupRoots.clear()
|
||||
})
|
||||
|
||||
describe('SSH relay runtime Windows system-SSH tree transfer', () => {
|
||||
it('rejects POSIX trees, unsafe roots, transport, concurrency, and pre-abort before I/O', async () => {
|
||||
const windows = await treeFixture()
|
||||
const linux = await treeFixture('linux')
|
||||
const signal = new AbortController().signal
|
||||
for (const options of [
|
||||
{ tree: linux, connection: connection(), remoteStagingRoot: 'C:/stage', signal },
|
||||
{ tree: windows, connection: connection(), remoteStagingRoot: '../stage', signal },
|
||||
{ tree: windows, connection: connection(), remoteStagingRoot: 'C:/CON/stage', signal },
|
||||
{ tree: windows, connection: connection(), remoteStagingRoot: '//server/share', signal },
|
||||
{ tree: windows, connection: connection(false), remoteStagingRoot: 'C:/stage', signal },
|
||||
{
|
||||
tree: windows,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/stage',
|
||||
maximumConcurrency: 5,
|
||||
signal
|
||||
}
|
||||
]) {
|
||||
await expect(transferSshRelayRuntimeTreeViaWindowsSystemSsh(options)).rejects.toThrow()
|
||||
}
|
||||
const controller = new AbortController()
|
||||
const reason = new Error('pre-aborted')
|
||||
controller.abort(reason)
|
||||
await expect(
|
||||
transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree: windows,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/stage',
|
||||
signal: controller.signal
|
||||
})
|
||||
).rejects.toBe(reason)
|
||||
expect(stagingControl).not.toHaveBeenCalled()
|
||||
expect(openFileDestination).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('creates parent-first directories and streams every exact file before returning progress', async () => {
|
||||
const tree = await treeFixture()
|
||||
const remoteRoot = 'C:/Users/测试/.orca/stage'
|
||||
const written = new Map<string, Buffer>()
|
||||
const events: string[] = []
|
||||
stagingControl.mockImplementation(async ({ operation, remotePath }) => {
|
||||
events.push(operation === 'create-directory' ? `directory:${remotePath}` : operation)
|
||||
})
|
||||
openFileDestination.mockImplementation(
|
||||
async ({ remotePath, expectedSize, signal, openChannel }) => {
|
||||
events.push(`file:${remotePath}`)
|
||||
expect(expectedSize).toBe(tree.files.find((file) => remotePath.endsWith(file.path))?.size)
|
||||
expect(signal).toBeInstanceOf(AbortSignal)
|
||||
expect(openChannel).toBeTypeOf('function')
|
||||
return {
|
||||
write: vi.fn(async (chunk: Buffer) => {
|
||||
written.set(
|
||||
remotePath,
|
||||
Buffer.concat([written.get(remotePath) ?? Buffer.alloc(0), chunk])
|
||||
)
|
||||
}),
|
||||
close: vi.fn(async () => {}),
|
||||
abort: vi.fn(async () => {})
|
||||
}
|
||||
}
|
||||
)
|
||||
const progress: unknown[] = []
|
||||
const result = await transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: remoteRoot,
|
||||
maximumConcurrency: 2,
|
||||
signal: new AbortController().signal,
|
||||
onProgress: (value) => progress.push(value)
|
||||
})
|
||||
|
||||
expect(result).toMatchObject({
|
||||
remoteStagingRoot: remoteRoot,
|
||||
filesCompleted: tree.fileCount,
|
||||
totalFiles: tree.fileCount,
|
||||
bytesTransferred: tree.expandedBytes,
|
||||
totalBytes: tree.expandedBytes
|
||||
})
|
||||
expect(stagingControl.mock.calls[0]?.[0]).toMatchObject({
|
||||
operation: 'create-root',
|
||||
remoteRoot
|
||||
})
|
||||
const directoryCalls = stagingControl.mock.calls.slice(1).map(([value]) => value)
|
||||
const expectedDirectories = [...tree.directories]
|
||||
.sort(
|
||||
(left, right) =>
|
||||
left.path.split('/').length - right.path.split('/').length ||
|
||||
(left.path < right.path ? -1 : left.path > right.path ? 1 : 0)
|
||||
)
|
||||
.map((directory) => ({
|
||||
operation: 'create-directory',
|
||||
remoteRoot,
|
||||
remotePath: `${remoteRoot}/${directory.path}`
|
||||
}))
|
||||
expect(directoryCalls).toEqual(
|
||||
expectedDirectories.map((value) => expect.objectContaining(value))
|
||||
)
|
||||
expect(events.findIndex((event) => event.startsWith('file:'))).toBe(tree.directories.length + 1)
|
||||
for (const file of tree.files) {
|
||||
const remotePath = `${remoteRoot}/${file.path}`
|
||||
expect(written.get(remotePath)).toEqual(await readFile(file.localPath))
|
||||
expect(openFileDestination).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ remotePath, expectedSize: file.size })
|
||||
)
|
||||
}
|
||||
expect(progress.length).toBeGreaterThan(0)
|
||||
expect(progress.at(-1)).toMatchObject({
|
||||
filesCompleted: tree.fileCount,
|
||||
bytesTransferred: tree.expandedBytes,
|
||||
activeFiles: 0
|
||||
})
|
||||
expect(progress.some((value) => 'path' in (value as object))).toBe(false)
|
||||
})
|
||||
|
||||
it('does not clean a pre-existing root when exclusive creation fails', async () => {
|
||||
const tree = await treeFixture()
|
||||
const collision = new Error('root exists')
|
||||
stagingControl.mockRejectedValueOnce(collision)
|
||||
await expect(
|
||||
transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/existing/stage',
|
||||
signal: new AbortController().signal
|
||||
})
|
||||
).rejects.toBe(collision)
|
||||
expect(stagingControl).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('defaults to one active destination and permits no more than four', async () => {
|
||||
const tree = await treeFixture()
|
||||
for (const maximumConcurrency of [undefined, 4]) {
|
||||
const retained = retainDestinationWrites()
|
||||
const transfer = transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/private/stage',
|
||||
maximumConcurrency,
|
||||
signal: new AbortController().signal
|
||||
})
|
||||
const expected = maximumConcurrency ?? 1
|
||||
await vi.waitFor(() => expect(retained.peak()).toBe(expected))
|
||||
retained.release()
|
||||
await transfer
|
||||
expect(retained.peak()).toBeLessThanOrEqual(expected)
|
||||
openFileDestination.mockReset()
|
||||
}
|
||||
})
|
||||
|
||||
it('cleans an owned root after a directory failure', async () => {
|
||||
const tree = await treeFixture()
|
||||
const primary = new Error('directory denied')
|
||||
stagingControl.mockImplementation(async ({ operation }) => {
|
||||
if (operation === 'create-directory') {
|
||||
throw primary
|
||||
}
|
||||
})
|
||||
await expect(
|
||||
transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/private/stage',
|
||||
signal: new AbortController().signal
|
||||
})
|
||||
).rejects.toBe(primary)
|
||||
expect(stagingControl.mock.calls.map(([value]) => value.operation)).toEqual([
|
||||
'create-root',
|
||||
'create-directory',
|
||||
'remove-root'
|
||||
])
|
||||
})
|
||||
|
||||
it('joins file and owned-root cleanup failures', async () => {
|
||||
const tree = await treeFixture()
|
||||
const primary = new Error('remote disk full')
|
||||
const cleanup = new Error('endpoint protection lock')
|
||||
openFileDestination.mockRejectedValueOnce(primary)
|
||||
stagingControl.mockImplementation(async ({ operation }: { operation: string }) => {
|
||||
if (operation === 'remove-root') {
|
||||
throw cleanup
|
||||
}
|
||||
})
|
||||
await expect(
|
||||
transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/private/stage',
|
||||
signal: new AbortController().signal
|
||||
})
|
||||
).rejects.toMatchObject({ errors: expect.arrayContaining([primary, cleanup]) })
|
||||
expect(stagingControl.mock.calls.at(-1)?.[0]).toMatchObject({ operation: 'remove-root' })
|
||||
})
|
||||
|
||||
it('settles a retained write before cleanup on cancellation and emits no later progress', async () => {
|
||||
const tree = await treeFixture()
|
||||
const controller = new AbortController()
|
||||
const reason = new Error('cancelled Windows tree transfer')
|
||||
const events: string[] = []
|
||||
const progress: unknown[] = []
|
||||
stagingControl.mockImplementation(async ({ operation }) => events.push(operation))
|
||||
openFileDestination.mockImplementation(async ({ signal }: { signal: AbortSignal }) => ({
|
||||
write: vi.fn(
|
||||
() =>
|
||||
new Promise<void>((_resolve, reject) => {
|
||||
events.push('write-start')
|
||||
signal.addEventListener('abort', () => reject(signal.reason), { once: true })
|
||||
})
|
||||
),
|
||||
close: vi.fn(async () => events.push('file-close')),
|
||||
abort: vi.fn(async () => events.push('file-abort'))
|
||||
}))
|
||||
const transfer = transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/private/stage',
|
||||
signal: controller.signal,
|
||||
onProgress: (value) => progress.push(value)
|
||||
})
|
||||
await vi.waitFor(() => expect(events).toContain('write-start'))
|
||||
controller.abort(reason)
|
||||
await expect(transfer).rejects.toBe(reason)
|
||||
expect(events.indexOf('file-abort')).toBeLessThan(events.indexOf('remove-root'))
|
||||
const settledProgress = progress.length
|
||||
await new Promise((resolve) => setTimeout(resolve, 0))
|
||||
expect(progress).toHaveLength(settledProgress)
|
||||
})
|
||||
|
||||
it('bounds unresponsive owned-root cleanup below ten seconds', async () => {
|
||||
const tree = await treeFixture()
|
||||
vi.useFakeTimers()
|
||||
const primary = new Error('file open failed')
|
||||
openFileDestination.mockRejectedValueOnce(primary)
|
||||
stagingControl.mockImplementation(({ operation, signal }) => {
|
||||
if (operation !== 'remove-root') {
|
||||
return Promise.resolve()
|
||||
}
|
||||
return new Promise((_resolve, reject) => {
|
||||
signal.addEventListener('abort', () => reject(signal.reason), { once: true })
|
||||
})
|
||||
})
|
||||
const startedAt = Date.now()
|
||||
const transfer = transferSshRelayRuntimeTreeViaWindowsSystemSsh({
|
||||
tree,
|
||||
connection: connection(),
|
||||
remoteStagingRoot: 'C:/private/stage',
|
||||
signal: new AbortController().signal
|
||||
})
|
||||
const rejection = expect(transfer).rejects.toMatchObject({
|
||||
errors: expect.arrayContaining([
|
||||
primary,
|
||||
expect.objectContaining({ message: expect.stringMatching(/cleanup timed out/i) })
|
||||
])
|
||||
})
|
||||
await vi.waitFor(() =>
|
||||
expect(stagingControl.mock.calls.at(-1)?.[0]).toMatchObject({ operation: 'remove-root' })
|
||||
)
|
||||
await vi.advanceTimersByTimeAsync(
|
||||
SSH_RELAY_RUNTIME_WINDOWS_TREE_TRANSFER_LIMITS.cleanupTimeoutMs
|
||||
)
|
||||
await rejection
|
||||
expect(Date.now() - startedAt).toBeLessThan(10_000)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,192 @@
|
||||
import { assertSafeRemotePathSegment, normalizeWindowsRemotePath } from './ssh-remote-platform'
|
||||
import type { SshRelayRuntimeScannedSourceTree } from './ssh-relay-runtime-source-scan'
|
||||
import {
|
||||
streamSshRelayRuntimeSourceTree,
|
||||
type SshRelayRuntimeSourceStreamOptions,
|
||||
type SshRelayRuntimeSourceStreamResult
|
||||
} from './ssh-relay-runtime-source-stream'
|
||||
import {
|
||||
openSshRelayRuntimeSystemSshFileChannel,
|
||||
type SshRelayRuntimeSystemSshConnection
|
||||
} from './ssh-relay-runtime-system-ssh-file-channel'
|
||||
import { openSshRelayRuntimeWindowsFileDestination } from './ssh-relay-runtime-windows-file-destination'
|
||||
import {
|
||||
runSshRelayRuntimeWindowsStagingControl,
|
||||
type RunSshRelayRuntimeWindowsStagingControlOptions
|
||||
} from './ssh-relay-runtime-windows-staging-control'
|
||||
|
||||
export type SshRelayRuntimeWindowsTreeTransferOptions = Readonly<{
|
||||
tree: SshRelayRuntimeScannedSourceTree
|
||||
connection: SshRelayRuntimeSystemSshConnection
|
||||
remoteStagingRoot: string
|
||||
signal: AbortSignal
|
||||
maximumConcurrency?: number
|
||||
onProgress?: SshRelayRuntimeSourceStreamOptions['onProgress']
|
||||
}>
|
||||
|
||||
export type SshRelayRuntimeWindowsTreeTransferResult = Readonly<
|
||||
SshRelayRuntimeSourceStreamResult & { remoteStagingRoot: string }
|
||||
>
|
||||
|
||||
const MAXIMUM_CONCURRENT_FILES = 4
|
||||
const CLEANUP_TIMEOUT_MS = 5_000
|
||||
|
||||
type OpenChannel = RunSshRelayRuntimeWindowsStagingControlOptions['openChannel']
|
||||
|
||||
function normalizeStagingRoot(value: string): string {
|
||||
if (
|
||||
typeof value !== 'string' ||
|
||||
value === '' ||
|
||||
value.includes('\0') ||
|
||||
value.includes('\r') ||
|
||||
value.includes('\n')
|
||||
) {
|
||||
throw new Error('SSH relay runtime Windows staging root is invalid')
|
||||
}
|
||||
const normalized = normalizeWindowsRemotePath(value)
|
||||
const driveRooted = /^[A-Za-z]:\//u.test(normalized)
|
||||
const uncRooted = /^\/\/[^/]+\/[^/]+\//u.test(normalized)
|
||||
if ((!driveRooted && !uncRooted) || normalized.endsWith('/')) {
|
||||
throw new Error('SSH relay runtime Windows staging root is invalid')
|
||||
}
|
||||
const segments = driveRooted ? normalized.slice(3).split('/') : normalized.slice(2).split('/')
|
||||
try {
|
||||
for (const segment of segments) {
|
||||
assertSafeRemotePathSegment(segment, 'windows')
|
||||
}
|
||||
} catch {
|
||||
throw new Error('SSH relay runtime Windows staging root is invalid')
|
||||
}
|
||||
return normalized
|
||||
}
|
||||
|
||||
function validateOptions(options: SshRelayRuntimeWindowsTreeTransferOptions): number {
|
||||
const concurrency = options?.maximumConcurrency ?? 1
|
||||
if (
|
||||
!options?.tree ||
|
||||
!options.connection ||
|
||||
typeof options.connection.usesSystemSshTransport !== 'function' ||
|
||||
typeof options.connection.exec !== 'function' ||
|
||||
!options.signal ||
|
||||
typeof options.signal.throwIfAborted !== 'function' ||
|
||||
!Number.isInteger(concurrency) ||
|
||||
concurrency < 1 ||
|
||||
concurrency > MAXIMUM_CONCURRENT_FILES
|
||||
) {
|
||||
throw new Error('SSH relay runtime Windows tree transfer input or concurrency is invalid')
|
||||
}
|
||||
if (options.tree.os !== 'win32') {
|
||||
throw new Error('SSH relay runtime Windows tree transfer requires a Windows tree')
|
||||
}
|
||||
if (!options.connection.usesSystemSshTransport()) {
|
||||
throw new Error('SSH relay runtime Windows tree transfer requires system SSH transport')
|
||||
}
|
||||
return concurrency
|
||||
}
|
||||
|
||||
function remoteManifestPath(remoteStagingRoot: string, manifestPath: string): string {
|
||||
const segments = manifestPath.split('/')
|
||||
for (const segment of segments) {
|
||||
assertSafeRemotePathSegment(segment, 'windows')
|
||||
}
|
||||
// Why: manifest paths are remote slash paths; client-native path utilities would corrupt them.
|
||||
return `${remoteStagingRoot}/${segments.join('/')}`
|
||||
}
|
||||
|
||||
function joinedFailure(primary: unknown, cleanupFailure: unknown | undefined): unknown {
|
||||
return cleanupFailure === undefined
|
||||
? primary
|
||||
: new AggregateError(
|
||||
[primary, cleanupFailure],
|
||||
'SSH relay runtime Windows tree transfer cleanup failed'
|
||||
)
|
||||
}
|
||||
|
||||
async function cleanupOwnedRoot(
|
||||
remoteStagingRoot: string,
|
||||
openChannel: OpenChannel
|
||||
): Promise<void> {
|
||||
const controller = new AbortController()
|
||||
// Why: caller cancellation initiates cleanup; a separate bounded signal lets cleanup still run.
|
||||
const timeout = setTimeout(
|
||||
() => controller.abort(new Error('SSH relay runtime Windows tree cleanup timed out')),
|
||||
CLEANUP_TIMEOUT_MS
|
||||
)
|
||||
try {
|
||||
await runSshRelayRuntimeWindowsStagingControl({
|
||||
operation: 'remove-root',
|
||||
remoteRoot: remoteStagingRoot,
|
||||
signal: controller.signal,
|
||||
openChannel
|
||||
})
|
||||
} finally {
|
||||
clearTimeout(timeout)
|
||||
}
|
||||
}
|
||||
|
||||
export async function transferSshRelayRuntimeTreeViaWindowsSystemSsh(
|
||||
options: SshRelayRuntimeWindowsTreeTransferOptions
|
||||
): Promise<SshRelayRuntimeWindowsTreeTransferResult> {
|
||||
const maximumConcurrency = validateOptions(options)
|
||||
const { tree, connection, signal, onProgress } = options
|
||||
const remoteStagingRoot = normalizeStagingRoot(options.remoteStagingRoot)
|
||||
const openChannel: OpenChannel = (command, exactSignal) =>
|
||||
openSshRelayRuntimeSystemSshFileChannel(connection, command, exactSignal)
|
||||
const runControl = (
|
||||
control: Omit<RunSshRelayRuntimeWindowsStagingControlOptions, 'signal' | 'openChannel'>
|
||||
): Promise<void> => runSshRelayRuntimeWindowsStagingControl({ ...control, signal, openChannel })
|
||||
signal.throwIfAborted()
|
||||
let rootOwned = false
|
||||
|
||||
try {
|
||||
await runControl({ operation: 'create-root', remoteRoot: remoteStagingRoot })
|
||||
rootOwned = true
|
||||
signal.throwIfAborted()
|
||||
|
||||
const directories = [...tree.directories].sort(
|
||||
(left, right) =>
|
||||
left.path.split('/').length - right.path.split('/').length ||
|
||||
(left.path < right.path ? -1 : left.path > right.path ? 1 : 0)
|
||||
)
|
||||
for (const directory of directories) {
|
||||
signal.throwIfAborted()
|
||||
await runControl({
|
||||
operation: 'create-directory',
|
||||
remoteRoot: remoteStagingRoot,
|
||||
remotePath: remoteManifestPath(remoteStagingRoot, directory.path)
|
||||
})
|
||||
signal.throwIfAborted()
|
||||
}
|
||||
|
||||
const result = await streamSshRelayRuntimeSourceTree({
|
||||
tree,
|
||||
signal,
|
||||
maximumConcurrency,
|
||||
onProgress,
|
||||
openDestination: (file, exactSignal) =>
|
||||
openSshRelayRuntimeWindowsFileDestination({
|
||||
remotePath: remoteManifestPath(remoteStagingRoot, file.path),
|
||||
expectedSize: file.size,
|
||||
signal: exactSignal,
|
||||
openChannel
|
||||
})
|
||||
})
|
||||
signal.throwIfAborted()
|
||||
return Object.freeze({ remoteStagingRoot, ...result })
|
||||
} catch (error) {
|
||||
let cleanupFailure: unknown
|
||||
if (rootOwned) {
|
||||
try {
|
||||
await cleanupOwnedRoot(remoteStagingRoot, openChannel)
|
||||
} catch (cleanupError) {
|
||||
cleanupFailure = cleanupError
|
||||
}
|
||||
}
|
||||
throw joinedFailure(error, cleanupFailure)
|
||||
}
|
||||
}
|
||||
|
||||
export const SSH_RELAY_RUNTIME_WINDOWS_TREE_TRANSFER_LIMITS = Object.freeze({
|
||||
maximumConcurrentFiles: MAXIMUM_CONCURRENT_FILES,
|
||||
cleanupTimeoutMs: CLEANUP_TIMEOUT_MS
|
||||
})
|
||||
Reference in New Issue
Block a user