mirror of
https://github.com/stablyai/orca.git
synced 2026-10-20 16:03:20 +00:00
* fix(ssh): enable TCP_NODELAY on ssh2 client to eliminate per-keystroke typing lag (#1660) ssh2 leaves Nagle's algorithm on by default. For single-byte keystrokes through a remote PTY, Nagle interacts with the kernel's delayed-ACK timer and adds up to ~40 ms per keystroke — visible as the typing lag reported in #1660. OpenSSH's `ssh` client sets TCP_NODELAY whenever a PTY is allocated; this change mirrors that on the ssh2 client right after the `ready` event in doSsh2Connect, covering both initial connect and auto-reconnect. Proxy-command / proxy-jump connections (where ssh2's underlying socket is a custom Duplex over a child-process pipe) are a no-op by design, gated by the public Client.setNoDelay()'s own type guard. A discriminating log line records which path each connect took. Tests cover initial connect and a full reconnect cycle to guard against the regression class "Nagle is re-enabled because someone refactored only the initial connect path." Co-authored-by: Orca <help@stably.ai> * fix(ssh): bound relay-lost reconnect with exponential backoff When the relay exec channel keeps dying (e.g. a remote-side bug closes every fresh --connect channel right after handshake, or a stale bridge keeps being replaced), the unguarded _onRelayLost handler reconnects as fast as the network allows — spawning relay deploy attempts in a tight loop until the user force-quits. Each iteration spawns a fresh ssh2 exec channel, hammers sshd's MaxSessions counter, and floods the renderer with state churn. Add per-target exponential backoff (500ms → 15s, capped at 6 attempts) so the loop terminates instead of running forever. After the cap the session goes to 'error' state with a 'Relay channel kept dropping. Please reconnect.' message — visible in the renderer instead of an invisible failure where typing in remote terminals just stops working. Successful 'ready' resets the attempt counter only if the session stabilized for >= 5s; faster flaps preserve the counter so a flaky remote backs off rather than retrying indefinitely on every brief ready→lost cycle. Backoff state is cleared on explicit disconnect, on session replacement during reconnect, and on connect failures, so a real reconnect attempt after backoff exhaustion always starts from zero. Co-authored-by: Orca <help@stably.ai> * fix(ssh): detect stale relay daemons via running-version marker The on-disk relay version check compares local .version against the remote .version file in the relay dir. A daemon launched by an earlier deploy keeps running its in-memory copy of the OLD relay code, so when the client later rewrites relay.js + .version on disk and bridges in via --connect, the new bridge process drives a stale daemon. Protocol or behavior changes between the two versions then tear down the channel in a tight reconnect loop (observed against PR #1672 on a daemon predating that change). The daemon now writes its running version into a .running-version sidecar at startup, anchored to the relay-script directory rather than process.cwd() so test spawns cannot pollute the repo root. Before attaching to an existing socket, the client probes that marker and, on mismatch with the locally-deployed .version, kills the stale daemon (TERM only, never KILL) and falls through to a fresh launch. Conservative defaults: when either marker is unreadable, attach so older builds keep their live PTYs. Co-authored-by: Orca <help@stably.ai> * Revert "fix(ssh): detect stale relay daemons via running-version marker" This reverts commite58acf07c0. * fix(ssh): isolate relay versions via per-version install dirs and wire handshake The relay's previous single-dir layout (~/.orca-remote/relay-v0.1.0/) let the deploy step rewrite relay.js in place while a daemon was still loaded in memory at the previous version. New clients then drove that stale daemon, surfacing as a reconnect loop (issue #1660 follow-up) and the field failure observed against an 8-day-old daemon on openclaw. Switch to a VS Code-style versioned layout where each (RELAY_VERSION + content-hash) bundle installs into its own directory and is never mutated after install. A v2 client's --connect socket path is rooted in relay-${v2-hash}/ and structurally cannot reach a v1 daemon's socket. Defense-in-depth: the daemon now reads exactly one Handshake-typed frame on each newly-accepted Unix socket before attaching the JSON-RPC dispatcher (mirrors VS Code's remoteExtensionHostAgentServer.ts:340). Mismatch closes the socket; the bridge exits with code 42; client maps that to a typed RelayVersionMismatchError and skips the relay-lost backoff loop instead of retrying through 6 attempts. Other deploy hardening: - atomic mkdir-based install lock with stale-lock recovery serialises concurrent first-installs of the same version - .install-complete sentinel distinguishes a finished install from a crashed-mid-install partial that should be retried - gcOldRelayVersions removes unreferenced sibling dirs (allowlist regex, skips locked or incomplete dirs, skips dirs with a live socket) - readLocalFullVersion fails fast on a missing/empty local .version rather than silently falling back to a path where a daemon from a different code generation may already be running Includes a cross-version isolation test that fails any future refactor which collapses the per-version layout back to a shared dir. Co-authored-by: Orca <help@stably.ai> * fix(ssh): harden relay versioning per review feedback Address must-fix and should-fix findings from the parallel triple review of26d1666e: - Surface RelayVersionMismatchError to ssh.ts on initial establish() (not just reconnect), so the user sees the typed terminal error instead of silent retry on first connect (#13). - Give the sentinel timeout a 500ms grace window for the close handler to deliver exit-42, so a slow remote does not misclassify a wire-handshake mismatch as a generic timeout (#D11). - Drain the handshake decoder's residue at the handshake -> dispatcher transition on both daemon and --connect sides; pipelined frames that were coalesced with the handshake are now forwarded into the dispatcher / stdout instead of silently dropped (#A1, #A2). - Reset the install-lock acquire timer after a stale-lock recovery so a single post-recovery race does not immediately exhaust the budget (#E14). - Treat a stale install-lock as recoverable in the GC pass when .install-complete is present (covers an interrupted finalize where the rm-lock failed) (#E15). - GC legacy relay-v\d+\.\d+\.\d+ install dirs whose daemons have died, now that .install-complete is no longer required for them (#12). - Resolve symlinks in readLaunchVersion() so a daemon launched via a symlinked entry script still reads .version next to the real file (#G21). - Flush stderr before exit-42 in --connect handshake mismatch path so the diagnostic line reaches the client before the process tears down (#C8). Tests: - Round-trip handshake over a real Socket pair: matching version, mismatch exit-42, leftover bytes preserved on both sides when frames are coalesced with the handshake. - waitForSentinel exit-42 -> RelayVersionMismatchError, exit-1 -> generic. - SshRelaySession terminal-error callback fires on both establish() and reconnect() when deployAndLaunchRelay throws RelayVersionMismatchError. - acquireInstallLock concurrent BUSY -> OK polling, stale-lock recovery with reset timeout window, and fresh-lock timeout failure path. - gcOldRelayVersions stale-lock-with-complete branch, legacy-dead path, legacy-alive path; existing locked-test asserts fresh-lock now keeps. - Cross-version isolation test now asserts a blanket invariant that every v1-referencing command from a v2 deploy is a read-only liveness probe. Lint and typecheck clean across all 3 tsconfigs; 426 SSH/relay tests pass. Co-authored-by: Orca <help@stably.ai> * fix(ssh): bypass npm init for content-hashed relay dirs and harden install probe The versioned-install dirs land at `relay-${version}+${hash}/` (e.g. `relay-0.1.0+07994a7870e1`). npm 11 / Node 26 reject the `+` in derived package names and `npm init -y` exits 1 — silently, since both stderr and the failure landed inside the `2>/dev/null && ...` chain. The catch swallowed the throw, `.install-complete` was written anyway, and every reconnect surfaced 'node-pty is not available' at first pty.spawn. Sidestep `npm init` entirely: SFTP-write a hardcoded minimal package.json (`name: orca-relay`, `type: commonjs`) and run `npm install node-pty` directly. `type: commonjs` pins the module system against future Node default flips or remote-side .npmrc overrides. Also harden the install path against the same class of silent failure: - npm install errors now propagate (no more `.install-complete` on hard fail; future reconnects retry instead of stranding the user) - Replace the weak `test -d node-pty` post-install probe with `node -e 'require("node-pty")'` so built-but-unloadable installs (missing prebuild, wrong arch, broken native binding) surface clearly - Add a session-level error handler on the SFTP write so a torn-down session rejects the promise instead of hanging until enclosing timeout Separate fix: add `for-each-ref` to the relay's git subcommand allowlist. Client code (`src/main/git/repo.ts` ref-search and worktree-listing) calls `git for-each-ref` over SSH; the relay was rejecting it. The `--shell`/`--python`/`--perl`/`--tcl` format flags only control output quoting (no eval) and the relay invokes git via execFileAsync (no shell), so the read-only allowlist treatment matches `rev-parse`, `log`, etc. Co-authored-by: Orca <help@stably.ai> * comment(ssh-relay): TODO link to #1693 for VS Code-style pre-bundled node-pty Co-authored-by: Orca <help@stably.ai> * fix(ssh): harden node-pty install probe and tighten review-fix tests Round-3 review fixes on top of963f56d7. deploy.ts: - Replace endsWith('OK') with includes('ORCA-NPTY-PROBE-OK'). Node can emit deprecation/experimental warnings to stderr after our stdout 'OK' write, and 2>&1 would push them past 'OK' producing false NPTY-MISSING warnings. A unique sentinel survives any trailing stderr noise. - Switch sftpPkg/ws .on -> .once for error/close. A late session 'error' after the promise had already settled would otherwise become an unhandled EventEmitter error and crash main. - Trim per-block comments to 1-2 lines per AGENTS.md (was 7-9). Tests: - Pin the BEFORE-ordering contract: SftpWriteCapture now records the count of execCommand calls observed at the moment ws.end() ran for each path, and the test asserts that count <= the index of npm install. Catches a future Promise.all-style refactor that would still pass final-state checks. - Strengthen the SSH-channel-failure test: assert the rejection actually came from the probe call (not an earlier exec) by finding the probe invocation in mock.calls. Also assert NPTY-INSTALL-FAIL is NOT logged (channel failure must not be conflated with install failure) and that abandonInstall was called so the lock is released. - Fix misleading clearAllMocks comment: it claims to wipe mockReturnValue, but actually clearAllMocks only resets .mock.calls. Re-priming was defense-in-depth, not a correctness requirement. Validator: - Add for-each-ref negative cases (--git-dir, --output, --work-tree) to the global-denied-flags it.each. The first round of for-each-ref enablement trusted that the post-subcommand GLOBAL_DENIED_FLAGS check applied; this pins it so a future allowlist refactor that bypasses the global check fails loudly. Co-authored-by: Orca <help@stably.ai> * fix(ssh): split node-pty probe into test-d guard + load-test Round-4 review fixes for the install probe in installNativeDeps: (1) test -d guard runs before the load-test. If the install dir vanished between npm install and probe (concurrent rm, fs unmount, permission flip), the deploy now throws and the next reconnect retries fresh — previously the cd failure flowed into '|| echo MISSING' and we'd write .install-complete, stranding the user in degraded mode. (2) Load-test discards stderr (2>/dev/null) so customized .bashrc output (NVM init, conda greetings, etc.) can't pollute the sentinel match. The shell-level '|| echo MISSING' is preserved so SSH-channel rejections still propagate as exec errors, distinct from require failures which exit the node process nonzero. (3) PROBE_OK is passed via process.argv[1] so the JS literal stays trivial regardless of future sentinel characters. Test changes: - New 'dir-gone' probe mode in makeExecResponses - New test pinning that vanished-dir throws (not silent MISSING) - SSH-channel test now asserts probeCallIdx > npmInstallIdx - cross-version-isolation feeds an extra '' for the test -d slot Co-authored-by: Orca <help@stably.ai> * fix(ssh): simplify node-pty probe and harden test ordering pins Round-5 review fixes for installNativeDeps: Production: - Drop redundant test -d guard. `cd ${dir} && (...)` short-circuits on cd-failure (dir-vanished) and propagates as exec reject already; the separate guard added a round trip without preventing anything. - Capture probe stderr to a per-deploy file rather than 2>&1 or 2>/dev/null. .bashrc noise can't pollute the sentinel match, but the require() error message is preserved in the [NPTY-MISSING] log breadcrumb so bug reports point at the real cause (e.g. GLIBC version mismatch). - Mirror the install command's PATH (export PATH=${binDir}:$PATH) so any future require-time child_process call resolves the same node binary used during install. - Add platform tuple to [NPTY-MISSING] and [NPTY-INSTALL-FAIL] logs for triageable bug reports without asking users to dig out their arch. - Trim probe comment per AGENTS.md (why-only, no mechanism narration). Tests: - Pin full installNativeDeps ordering: npm install < chmod prebuilds < probe. Catches refactors that probe before install or move chmod after. - Pressure-test .includes(PROBE_OK) survives bashrc/MOTD noise prefixed to probe stdout (corporate banner / NVM init / conda greeting case). - Pressure-test MISSING detection survives Node deprecation warnings prepended to the MISSING token. - Pin platform tuple appears in [NPTY-MISSING] log. - Pin finalizeInstall called exactly once + abandonInstall not called on happy paths; reverse on failure paths. - Strengthen dir-gone test: assert probeIdx > npmInstallIdx so a refactor that swaps order doesn't silently let the test pass on its own injected error string. - New probeStdoutOverride option in makeExecResponses for shell-noise injection tests. Cross-version-isolation: dropped obsolete test -d slot, added rm-stderr cleanup slot to match the new probe shape. eslint-disable max-lines on both files with rationale (pattern used widely in this repo for cohesive single-responsibility modules). 441/441 tests pass; lint clean; typecheck clean. Probe shape verified end-to-end on real remote. Co-authored-by: Orca <help@stably.ai> --------- Co-authored-by: Orca <help@stably.ai>
396 lines
13 KiB
TypeScript
396 lines
13 KiB
TypeScript
/* eslint-disable max-lines -- Why: SSH connection lifecycle, credential retries, reconnect policy, and transport fallback are intentionally co-located so state transitions stay auditable in one file. */
|
|
import * as net from 'net'
|
|
import { Client as SshClient } from 'ssh2'
|
|
import type { ChildProcess } from 'child_process'
|
|
import type { ClientChannel, ConnectConfig, SFTPWrapper } from 'ssh2'
|
|
import type { SshTarget, SshConnectionState, SshConnectionStatus } from '../../shared/ssh-types'
|
|
import { spawnSystemSsh, type SystemSshProcess } from './ssh-system-fallback'
|
|
import { resolveWithSshG } from './ssh-config-parser'
|
|
import {
|
|
INITIAL_RETRY_ATTEMPTS,
|
|
INITIAL_RETRY_DELAY_MS,
|
|
RECONNECT_BACKOFF_MS,
|
|
CONNECT_TIMEOUT_MS,
|
|
isTransientError,
|
|
isAuthError,
|
|
isPassphraseError,
|
|
sleep,
|
|
buildConnectConfig,
|
|
resolveEffectiveProxy,
|
|
spawnProxyCommand,
|
|
type SshConnectionCallbacks
|
|
} from './ssh-connection-utils'
|
|
export type { SshConnectionCallbacks } from './ssh-connection-utils'
|
|
|
|
export class SshConnection {
|
|
private client: SshClient | null = null
|
|
private proxyProcess: ChildProcess | null = null
|
|
private systemSsh: SystemSshProcess | null = null
|
|
private state: SshConnectionState
|
|
private callbacks: SshConnectionCallbacks
|
|
private target: SshTarget
|
|
private reconnectTimer: ReturnType<typeof setTimeout> | null = null
|
|
private disposed = false
|
|
private cachedPassphrase: string | null = null
|
|
private cachedPassword: string | null = null
|
|
private connectGeneration = 0
|
|
|
|
constructor(target: SshTarget, callbacks: SshConnectionCallbacks) {
|
|
this.target = target
|
|
this.callbacks = callbacks
|
|
this.state = {
|
|
targetId: target.id,
|
|
status: 'disconnected',
|
|
error: null,
|
|
reconnectAttempt: 0
|
|
}
|
|
}
|
|
|
|
getState(): SshConnectionState {
|
|
return { ...this.state }
|
|
}
|
|
getClient(): SshClient | null {
|
|
return this.client
|
|
}
|
|
getTarget(): SshTarget {
|
|
return { ...this.target }
|
|
}
|
|
|
|
// Why: exposes whether a passphrase/password is already cached in-memory for
|
|
// this connection. Used by ssh:needsPassphrasePrompt so callers can decide
|
|
// whether a manual-reconnect will prompt or go through silently. Without this,
|
|
// lastRequiredPassphrase stays true across the session even after the user
|
|
// has entered the credential once, causing redundant "enter passphrase"
|
|
// prompts on disconnect→reconnect cycles within a single app session.
|
|
hasCachedCredential(): boolean {
|
|
return this.cachedPassphrase != null || this.cachedPassword != null
|
|
}
|
|
|
|
async exec(cmd: string): Promise<ClientChannel> {
|
|
if (!this.client) {
|
|
throw new Error('Not connected')
|
|
}
|
|
return new Promise((res, rej) => this.client!.exec(cmd, (e, ch) => (e ? rej(e) : res(ch))))
|
|
}
|
|
|
|
async sftp(): Promise<SFTPWrapper> {
|
|
if (!this.client) {
|
|
throw new Error('Not connected')
|
|
}
|
|
return new Promise((res, rej) => this.client!.sftp((e, s) => (e ? rej(e) : res(s))))
|
|
}
|
|
|
|
async connect(): Promise<void> {
|
|
if (this.disposed) {
|
|
throw new Error('Connection disposed')
|
|
}
|
|
|
|
let lastError: Error | null = null
|
|
|
|
for (let attempt = 0; attempt < INITIAL_RETRY_ATTEMPTS; attempt++) {
|
|
try {
|
|
await this.attemptConnect()
|
|
return
|
|
} catch (err) {
|
|
lastError = err instanceof Error ? err : new Error(String(err))
|
|
|
|
if (isAuthError(lastError) || isPassphraseError(lastError)) {
|
|
this.setState('auth-failed', lastError.message)
|
|
throw lastError
|
|
}
|
|
|
|
if (!isTransientError(lastError)) {
|
|
this.setState('error', lastError.message)
|
|
throw lastError
|
|
}
|
|
|
|
if (attempt < INITIAL_RETRY_ATTEMPTS - 1) {
|
|
await sleep(INITIAL_RETRY_DELAY_MS)
|
|
}
|
|
}
|
|
}
|
|
|
|
const finalError = lastError ?? new Error('Connection failed')
|
|
this.setState('error', finalError.message)
|
|
throw finalError
|
|
}
|
|
|
|
private async attemptConnect(): Promise<void> {
|
|
this.setState('connecting')
|
|
this.proxyProcess?.kill()
|
|
this.proxyProcess = null
|
|
const connectGeneration = ++this.connectGeneration
|
|
|
|
const resolved = await resolveWithSshG(this.target.configHost || this.target.label).catch(
|
|
() => null
|
|
)
|
|
const config = buildConnectConfig(this.target, resolved)
|
|
|
|
// Why: ssh2 doesn't support ProxyCommand/ProxyJump natively. Spawn the
|
|
// resolved proxy and pipe its stdin/stdout as config.sock.
|
|
const effectiveProxy = resolveEffectiveProxy(this.target, resolved)
|
|
if (effectiveProxy) {
|
|
const proxy = spawnProxyCommand(effectiveProxy, config.host!, config.port!, config.username!)
|
|
this.proxyProcess = proxy.process
|
|
config.sock = proxy.sock
|
|
}
|
|
|
|
if (this.cachedPassphrase) {
|
|
config.passphrase = this.cachedPassphrase
|
|
}
|
|
if (this.cachedPassword) {
|
|
config.password = this.cachedPassword
|
|
}
|
|
|
|
try {
|
|
await this.doSsh2Connect(config, connectGeneration)
|
|
} catch (err) {
|
|
if (!(err instanceof Error) || !this.callbacks.onCredentialRequest) {
|
|
this.proxyProcess?.kill()
|
|
this.proxyProcess = null
|
|
throw err
|
|
}
|
|
// Why: prompt for passphrase on encrypted-key error, then retry with
|
|
// a fresh proxy socket (ssh2 may have destroyed the original).
|
|
if (isPassphraseError(err) && !this.cachedPassphrase) {
|
|
const detail = this.target.identityFile || resolved?.identityFile?.[0] || '(unknown)'
|
|
const val = await this.callbacks.onCredentialRequest(this.target.id, 'passphrase', detail)
|
|
if (val) {
|
|
this.cachedPassphrase = val
|
|
config.passphrase = val
|
|
this.respawnProxy(config, effectiveProxy)
|
|
await this.doSsh2Connect(config, connectGeneration)
|
|
return
|
|
}
|
|
}
|
|
// Why: prompt for password on auth failure. Check the original error
|
|
// (not a retry error) to avoid conflating passphrase vs password failures.
|
|
if (isAuthError(err) && !this.cachedPassword) {
|
|
const val = await this.callbacks.onCredentialRequest(
|
|
this.target.id,
|
|
'password',
|
|
config.host || this.target.label
|
|
)
|
|
if (val) {
|
|
this.cachedPassword = val
|
|
config.password = val
|
|
this.respawnProxy(config, effectiveProxy)
|
|
await this.doSsh2Connect(config, connectGeneration)
|
|
return
|
|
}
|
|
}
|
|
this.proxyProcess?.kill()
|
|
this.proxyProcess = null
|
|
throw err
|
|
}
|
|
}
|
|
|
|
// Why: ssh2 may destroy the proxy socket on auth failure, so credential
|
|
// retries need a fresh proxy process and Duplex stream.
|
|
private respawnProxy(
|
|
config: ConnectConfig,
|
|
proxy: ReturnType<typeof resolveEffectiveProxy> | null | undefined
|
|
): void {
|
|
if (!proxy) {
|
|
return
|
|
}
|
|
this.proxyProcess?.kill()
|
|
const p = spawnProxyCommand(proxy, config.host!, config.port!, config.username!)
|
|
this.proxyProcess = p.process
|
|
config.sock = p.sock
|
|
}
|
|
|
|
private doSsh2Connect(config: ConnectConfig, connectGeneration: number): Promise<void> {
|
|
return new Promise<void>((resolve, reject) => {
|
|
const client = new SshClient()
|
|
let settled = false
|
|
client.on('ready', () => {
|
|
if (settled) {
|
|
return
|
|
}
|
|
// Why: connect() completion races with explicit disconnect(). Once a
|
|
// newer connect attempt or disconnect bumps the generation/disposed
|
|
// state, this late ready event must not resurrect the torn-down client.
|
|
if (this.disposed || connectGeneration !== this.connectGeneration) {
|
|
settled = true
|
|
client.end()
|
|
client.destroy()
|
|
reject(new Error('SSH connection attempt was cancelled'))
|
|
return
|
|
}
|
|
settled = true
|
|
this.client = client
|
|
this.proxyProcess = null
|
|
// Why: ssh2 leaves Nagle's algorithm on by default. For single-byte
|
|
// keystrokes through a remote PTY this stacks with the kernel's
|
|
// delayed-ACK timer and adds up to ~40 ms per keystroke. OpenSSH's
|
|
// `ssh` sets TCP_NODELAY whenever a PTY is allocated; we mirror that
|
|
// because every channel we open over this connection (PTY data,
|
|
// JSON-RPC requests, port-scan probes) is latency-sensitive. No-op
|
|
// for proxy-command / proxy-jump connections where _sock is a custom
|
|
// Duplex; that case relies on the proxy program's own TCP behavior,
|
|
// same as native ssh.
|
|
const sock = (client as unknown as { _sock?: { setNoDelay?: unknown } })._sock
|
|
if (sock instanceof net.Socket) {
|
|
console.warn(`[ssh] TCP_NODELAY enabled for ${this.target.label}`)
|
|
} else {
|
|
console.warn(`[ssh] TCP_NODELAY skipped for ${this.target.label} (proxy socket)`)
|
|
}
|
|
client.setNoDelay(true)
|
|
this.setState('connected')
|
|
this.setupDisconnectHandler(client)
|
|
resolve()
|
|
})
|
|
client.on('error', (err) => {
|
|
if (settled) {
|
|
return
|
|
}
|
|
settled = true
|
|
client.destroy()
|
|
reject(err)
|
|
})
|
|
client.connect(config)
|
|
})
|
|
}
|
|
|
|
// Why: guard on identity so a late event from the old client doesn't
|
|
// null out a successful reconnect.
|
|
private setupDisconnectHandler(client: SshClient): void {
|
|
const onDrop = () => {
|
|
if (this.disposed || this.client !== client) {
|
|
return
|
|
}
|
|
this.client = null
|
|
this.scheduleReconnect()
|
|
}
|
|
client.on('end', onDrop)
|
|
client.on('close', onDrop)
|
|
client.on('error', (err) => {
|
|
if (this.disposed || this.client !== client) {
|
|
return
|
|
}
|
|
console.warn(`[ssh] Connection error for ${this.target.label}: ${err.message}`)
|
|
this.client = null
|
|
this.scheduleReconnect()
|
|
})
|
|
}
|
|
|
|
private scheduleReconnect(): void {
|
|
if (this.disposed || this.reconnectTimer) {
|
|
return
|
|
}
|
|
const attempt = this.state.reconnectAttempt
|
|
if (attempt >= RECONNECT_BACKOFF_MS.length) {
|
|
this.setState('reconnection-failed', 'Max reconnection attempts reached')
|
|
return
|
|
}
|
|
this.setState('reconnecting')
|
|
this.reconnectTimer = setTimeout(async () => {
|
|
this.reconnectTimer = null
|
|
if (this.disposed) {
|
|
return
|
|
}
|
|
try {
|
|
// Why: reset reconnectAttempt before attemptConnect so setState('connected')
|
|
// broadcasts reconnectAttempt=0, which ssh.ts uses to trigger relay re-establishment.
|
|
this.state.reconnectAttempt = 0
|
|
await this.attemptConnect()
|
|
} catch (err) {
|
|
if (this.disposed) {
|
|
return
|
|
}
|
|
const error = err instanceof Error ? err : new Error(String(err))
|
|
if (isAuthError(error) || isPassphraseError(error)) {
|
|
this.setState('auth-failed', error.message)
|
|
return
|
|
}
|
|
if (!isTransientError(error)) {
|
|
this.setState('error', error.message)
|
|
return
|
|
}
|
|
this.state.reconnectAttempt = attempt + 1
|
|
this.scheduleReconnect()
|
|
}
|
|
}, RECONNECT_BACKOFF_MS[attempt])
|
|
}
|
|
|
|
async connectViaSystemSsh(): Promise<SystemSshProcess> {
|
|
if (this.disposed) {
|
|
throw new Error('Connection disposed')
|
|
}
|
|
this.systemSsh?.kill()
|
|
this.systemSsh = null
|
|
this.setState('connecting')
|
|
try {
|
|
const proc = spawnSystemSsh(this.target)
|
|
this.systemSsh = proc
|
|
let settled = false
|
|
await new Promise<void>((resolve, reject) => {
|
|
const timeout = setTimeout(() => {
|
|
settled = true
|
|
proc.kill()
|
|
reject(new Error('System SSH connection timed out'))
|
|
}, CONNECT_TIMEOUT_MS)
|
|
proc.stdout.once('data', () => {
|
|
settled = true
|
|
clearTimeout(timeout)
|
|
resolve()
|
|
})
|
|
proc.onExit((code) => {
|
|
if (settled) {
|
|
return
|
|
}
|
|
settled = true
|
|
clearTimeout(timeout)
|
|
reject(
|
|
new Error(
|
|
code !== 0
|
|
? `System SSH exited with code ${code}`
|
|
: 'System SSH exited before producing output'
|
|
)
|
|
)
|
|
})
|
|
})
|
|
this.setState('connected')
|
|
// Why: register reconnection handler only after the initial handshake
|
|
// succeeds. The onExit registered above guards with `settled` so it
|
|
// won't fire a duplicate for exits during the handshake phase.
|
|
proc.onExit(() => {
|
|
if (!this.disposed && this.systemSsh === proc) {
|
|
this.systemSsh = null
|
|
this.scheduleReconnect()
|
|
}
|
|
})
|
|
return proc
|
|
} catch (err) {
|
|
this.setState('error', err instanceof Error ? err.message : String(err))
|
|
throw err
|
|
}
|
|
}
|
|
|
|
async disconnect(): Promise<void> {
|
|
this.disposed = true
|
|
this.connectGeneration += 1
|
|
if (this.reconnectTimer) {
|
|
clearTimeout(this.reconnectTimer)
|
|
}
|
|
this.reconnectTimer = null
|
|
this.cachedPassphrase = null
|
|
this.cachedPassword = null
|
|
this.client?.end()
|
|
this.client = null
|
|
this.proxyProcess?.kill()
|
|
this.proxyProcess = null
|
|
this.systemSsh?.kill()
|
|
this.systemSsh = null
|
|
this.setState('disconnected')
|
|
}
|
|
|
|
private setState(status: SshConnectionStatus, error?: string): void {
|
|
this.state = { ...this.state, status, error: error ?? null }
|
|
this.callbacks.onStateChange(this.target.id, { ...this.state })
|
|
}
|
|
}
|
|
|
|
export { SshConnectionManager } from './ssh-connection-manager'
|