mirror of
https://github.com/stablyai/orca.git
synced 2026-11-15 02:17:31 +00:00
* fix(terminal): bound the in-process macOS process table's blocking calls and ps fallbacks (#31783) Follow-up to #35987 from its post-merge reviews. - cwd and executable path: node-pty's readProcessCwd/readProcessPath now run PROC_PIDVNODEPATHINFO and proc_pidpath on a libuv worker (the synchronous exports are gone, so an older pty.node falls back to ps/lsof). JS caps them at two outstanding lookups and 250ms; a wedged mount or a full cap answers from lsof -b, which never stats, so a dead NFS/SMB/FUSE mount cannot freeze the daemon's event loop or take the whole pool. - Unreadable argv prints "(comm)", as ps does, instead of a proc_pidpath lookup under Orca's credentials on the table worker. - The table exports each row's euid. Argv recovery runs setuid ps only for another user's processes, never for our own mid-exec or exiting ones (or SIDL/exiting rows), which ps cannot read either: a pane looping short commands forked ps on ~15% of full reads, now none. - Recovery is coalesced per process across concurrent readers and a failed recovery waits 5s before costing another ps. The cache keys on comm and start time. - Full-table and shell-foreground captures, which feed strict evidence with the capture's age, take only argv ps read after their request; display readers keep the 30s cache. - While the reader is down (stalled-pool breaker open, or a rejecting addon, which now backs off 1s..30s), runPsInProcess runs /bin/ps itself through one shared budget: identical invocations share a run, a burst of 4 forks then one a second, and a run spawned before a request never answers it. Descendant teardown and foreground-group signalling keep their own full-timeout ps (callerRunsPs). 13 pollers at 750ms: 1,040 ps/min before, 63 after. No daemon protocol bump: the wire format is unchanged, and each daemon loads JS and pty.node from the same runtime copy. * fix(terminal): make the libproc reader survive worker teardown, fail closed and share one fair fork budget (#31783) Review follow-ups to the previous commit. - Native: every libproc AsyncWorker derives from OrcaPromiseWorker, which skips JS completion once an env cleanup hook marks the env closing and swallows a Napi::Error thrown mid-teardown. Terminating a worker_thread with cwd or table reads in flight aborted the process; 200 terminates each now survive. - (comm) in a terminal's foreground group (argv neither reader recovered, e.g. a root `su`) judges 'unknown', never 'agent', so the launch line is not typed into it. - Fresh argv only for getFreshSnapshot captures; the 500ms TTL readers keep the 30s cache. Argv recovery runs through its own fork budget (2, then one per 2s). Root `su` pane, 13 pollers at 750ms: TTL 79 -> 2 ps/min, fresh 158 -> 31. - The fork budget serves invocations in FIFO turns, grouping each one's waiters onto one run. 13 streams: served 80,42,1,1,2,0x8 -> 10-12 each. - Forced subprocess captures (a held naming pass) go through the fallback budget: 5 readers, 400 direct ps/min -> 63, 395 of 400 captures served. - A failed recovery is retried 5s after ps gave up, not after it started. - Tests pin Orca's euid instead of reading the runner's and assuming root is 0. - process-cwd keeps one lsof per pid until it exits, so a dead mount no longer leaks one per poll; shell readiness reads the exe path through runProcess, which gives up on an lsof that ignores SIGTERM. - A #35987 pty.node (table reader, no async path reads) keeps the in-process table; cwd/path use lsof, with a one-time warning. - captureProcessTable moves to process-table-ps-capture.ts. * fix(terminal): make withheld argv unverifiable and reuse recovered argv for at most 2s (#31783) Round-2 review follow-ups. - Worker teardown: drop OrcaPromiseWorker and its env flag (Node drains the pool before cleanup hooks, so it never fired). node-pty's mac pty target now builds with NODE_API_SWALLOW_UNTHROWABLE_EXCEPTIONS, which covers both the call-in and completion paths. The smoke test now calls the reads from a timer while terminating: the previous build aborted 2 of 3 runs, this one survives 3/3 x 100. pty.cc is back to the first follow-up's exact content. - A foreground process whose argv ps withheld (`(comm)`: another user's, exiting, or a recovery the budget skipped) makes the batch resolver, and so host-stamped evidence, `argv_unreadable`/unverifiable rather than live with no agent; agent-foreground resolution reports it unavailable. - judgeTerminalForeground fails closed only for an unreadable shell (`(bash)` from a root `su`), not our own exiting `(git)`. - Recovered argv is reused for at most 2s per identity (pid, start, comm) for every reader (the next commit ages evidence by it), and fresh reads no longer queue behind the recovery budget. Root `su` pane, 13 fresh pollers: p50 2,989ms -> 53ms, 31 -> 26 ps/min (TTL reader 1 -> 26). - The fork budgets serve interactive and fresh readers before display polls (cheap and TTL snapshot captures), FIFO within each. 20 display streams plus a 3s launch-paste read every 5s: 0/12 -> 12/12 served, polls 4-7 each. - On a selected terminal, the foreground group's argv is recovered even with no process of ours left (`exec su`); the root login is not. - Budgeted captures keep the 32 MiB output ceiling. * fix(terminal): age strict evidence by its oldest recovered argv (#31783) Round-3 review follow-ups. - Recovered argv carried no read time, so a capture reusing it (cached, or a recovery still in flight) stamped its own capture time over argv up to 2s older, and the remote write guard could take a same-comm re-exec for the agent. Rows now carry when ps read their recovered argv; runPsInProcess reports the oldest through onArgvReadAt, and the snapshot reader stamps capturedAtMs at that input, so capturedAgeMs is the age of the oldest input and consumers' own freshness checks (the write guard's 500ms) decide. A recovery begun more than 2s before a reader asked no longer answers it, and an older recovery finishing late cannot replace a newer one. - resolveSpawnFileForegroundFromRows reports a withheld `(node)` foreground unavailable. - One budget queue entry per invocation: an interactive request promotes a queued poll's, a poll joins an interactive one. - parseShellForegroundRows gets its JSDoc back. * fix(terminal): keep a cached name from authorizing a refused write; integer, under-2s argv ages (#31783) Round-4 review follow-ups. - When the relay's evidence declines a write (stale, unverifiable, another agent), its foreground name may still prove the shell but no longer proves the agent: that name can be the tracker's 1s display cache, e.g. `claude` cached just before Claude exited to a root shell under su. Relays that send no evidence keep the name as before. - Snapshot ages round up to whole milliseconds; the wire rejects fractions from the monotonic-to-wall conversion. - Recovered argv is reused for at most 1.5s, so reuse plus the 500ms snapshot cache stays under the 2s freshness gate; reuse ages from when that ps spawned, not from the wait for its fork token, and only an answer is remembered (a timed-out or withheld run proves nothing). Fork budget unchanged. Root su pane, 13 pollers: 30 ps/min; fresh p50 51ms; strict evidence age p50 752ms, max 1,508ms. * fix(terminal): keep the relay's agent name behind structurally unverifiable evidence (#31783) The previous commit refused the name fallback whenever the relay sent any evidence, so `unverifiable` evidence for structural reasons also blocked writes: npm Codex (node codex.js spawning native codex) and Gemini (relaunching itself) put two agent rows in the foreground group (ambiguous_foreground_group), and remote tmux reads multiplexer_boundary. On SSH POSIX hosts that dropped launch prompts and failed worker briefs, conversation send/cancel and automations with agent_not_in_foreground. The name now cannot authorize only when the evidence saw the group (`live`: stale, or no agent in it, as in the cached-`claude`-over-root-shell repro) or withheld a foreground argv (`argv_unreadable`, possibly a root `su` shell). Other unverifiable reasons keep the name fallback, as before this branch. * fix(terminal): let a relay name complete only fresh agent-positive evidence (#31783) The SSH name fallback still let a stale or cached name authorize a write under any unverifiable reason but argv_unreadable: ps EAGAIN (process_table_unreadable) after the tracker cached `claude` over a root bash, anchor_missing, tty_boundary (`sudo -i` with use_pty, a root shell on a new pty), or a 1.5s-old ambiguous capture. - The name now completes evidence only when it is fresh (within the guard's 500ms bound), its reason is one that saw an agent row (ambiguous_foreground_group, multiplexer_boundary, candidate_start_time_missing), and the name is the launched agent. Every other reason, including ones this build does not know, reads unknown. Relays that send no evidence keep the name, for mixed versions. - The relay's evidence names one agent that holds two rows of the group (npm Codex: node codex.js spawning native codex; Gemini relaunching itself), so a new relay proves them live even when its name is `node`. Two different agents stay ambiguous. No wire change. * fix(terminal): judge a relay name by the host's capture age; keep old relays' npm Codex (#31783) - Name completion compared the capture's start with the read's start, so a slow response made old evidence look fresh: a capture 1,500ms old arriving after 1,100ms passed. It now uses the host-reported capturedAgeMs against the 500ms bound, which latency cannot shrink. Live evidence keeps its capture-start rule. - Relays before same-agent dedupe report npm Codex (node wrapper plus native binary) as ambiguous_foreground_group and name it `node`. For that reason only, `node` now completes the evidence when the launched agent is npm-installed per the node entrypoint tables; other agents and reasons stay fail-closed. * fix(terminal): bound relay-name completion by twice the snapshot TTL (#31783) The 500ms bound refused ~9.5% of tmux name completions on a host with 13 pollers (cached snapshot up to one 500ms TTL old plus ~30ms capture, max 528ms), and a refusal drops the launch prompt or throws with no retry. The bound is now 2 x PROCESS_TABLE_SNAPSHOT_MAX_STALENESS_MS (1s), still on the host-reported age: 0 of 767 reads exceed it, the delayed 1,500ms repro still refuses, and the cached-name hazard (the tracker's ~1s display cache) is no wider. * fix(terminal): complete relay evidence with a name read now, not the display cache (#31783) With the 1s bound, a capture at 0-528ms, `codex` cached at 100ms, Codex exiting at 101ms and the guard at 999ms still let the cached name authorize a write over the bash that replaced it. - Name completion (structural reasons, host age within 2x the snapshot TTL) now asks confirmForegroundProcess, which reads the foreground now (a fresh scan, or node-pty's raw process name) rather than the tracker's 1s display cache; a null answer reads unknown. No new wire surface: the local daemon answers confirmForegroundProcess, paired runtimes evaluate guards locally, and the production controller returns null (read as unknown, failing safe) when a provider lacks it. Controllers without the method, and relays sending no evidence, keep the cached name as the legacy path. - Live evidence also accepts a host-reported age within 2x the TTL, so a real agent behind a 500-528ms cached snapshot on a fast link is no longer refused; a capture begun after the read was asked still counts, and 5,000ms-old evidence still refuses. * fix(terminal): confirm live evidence older than one snapshot TTL with a fresh read (#31783) Accepting live evidence up to 2x the TTL reopened the window the previous commit closed for name completion: a capture at 0-528ms, the agent exiting at ~529ms, and at 999ms `live claude` still read as agent over a bash. Live evidence begun no more than one TTL before the read was asked (so any host age within one TTL, however long ps took) still answers alone. Between that and 2x the TTL, confirmForegroundProcess reads the foreground now: the launched agent proves it, a shell reads shell, anything else unknown. Controllers without that read fall through and refuse, as before. Reachability: the remote (host.remote) write-guard path has no production caller since the SSH provider's removal (#27499); the registry dispatches only local providers and paired runtimes evaluate guards locally. This is defence in depth for any future remote provider.
712 lines
26 KiB
TypeScript
712 lines
26 KiB
TypeScript
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||
|
||
const { runProcess } = vi.hoisted(() => ({ runProcess: vi.fn() }))
|
||
vi.mock('@orca/process-host', () => ({ runProcess }))
|
||
vi.mock('./darwin-terminal-names', () => ({
|
||
nameDarwinTerminalDevices: async (devices: ReadonlySet<string>) =>
|
||
new Map([...devices].filter((device) => device === '16/4').map((device) => [device, 'ttys004']))
|
||
}))
|
||
|
||
import {
|
||
setLibprocAddonForTests,
|
||
type LibprocAddon,
|
||
type LibprocProcessRow,
|
||
type LibprocQuery
|
||
} from './libproc-addon'
|
||
import { nativeReadWaitersForTests, resetNativeReadsForTests } from './libproc-native-reads'
|
||
import { runPsInProcess } from './libproc-process-table'
|
||
import { resetPsFallbackForTests } from './libproc-ps-fallback'
|
||
import { createProcessTableSnapshotReader } from './process-table-snapshot-reader'
|
||
import { parseCheapProcessTableRows, parseStrictProcessTableRows } from './process-table-snapshot'
|
||
|
||
const TTYS004 = (16 << 24) | 4
|
||
// 2026-09-03T16:02:01Z, a single-digit day so lstart pads it.
|
||
const START = Date.UTC(2026, 8, 3, 16, 2, 1) / 1000
|
||
// Pin the locale so lstart formatting doesn't depend on the runner's LANG.
|
||
const BUDGET = { timeoutMs: 1000, env: { LC_ALL: 'C' } }
|
||
|
||
function row(overrides: Partial<LibprocProcessRow>): LibprocProcessRow {
|
||
return {
|
||
pid: 1,
|
||
ppid: 0,
|
||
pgid: 1,
|
||
tpgid: 0,
|
||
tdev: -1,
|
||
status: 2,
|
||
flags: 0,
|
||
sessionLeader: false,
|
||
startSec: START,
|
||
comm: 'launchd',
|
||
command: '/sbin/launchd',
|
||
commandIncomplete: true,
|
||
...overrides
|
||
}
|
||
}
|
||
|
||
const TABLE = [
|
||
row({
|
||
pid: 4300,
|
||
ppid: 4242,
|
||
pgid: 4300,
|
||
tpgid: 4300,
|
||
tdev: TTYS004,
|
||
flags: 0x2,
|
||
comm: 'node',
|
||
command: 'node /usr/local/bin/claude --resume',
|
||
commandIncomplete: false
|
||
}),
|
||
row({
|
||
pid: 4242,
|
||
ppid: 4200,
|
||
pgid: 4242,
|
||
tpgid: 4300,
|
||
tdev: TTYS004,
|
||
flags: 0x2,
|
||
sessionLeader: true,
|
||
comm: 'zsh',
|
||
command: '-zsh',
|
||
commandIncomplete: false
|
||
}),
|
||
// A pane's root login is root-owned and sits under the terminal host.
|
||
row({
|
||
pid: 4200,
|
||
ppid: 1,
|
||
pgid: 4200,
|
||
tdev: TTYS004,
|
||
flags: 0x2,
|
||
comm: 'login',
|
||
command: '/usr/bin/login'
|
||
}),
|
||
row({
|
||
pid: 4400,
|
||
ppid: 4242,
|
||
pgid: 4400,
|
||
tpgid: 4300,
|
||
status: 4,
|
||
comm: 'vim',
|
||
command: 'vim',
|
||
commandIncomplete: false
|
||
}),
|
||
row({ pid: 4401, ppid: 4242, pgid: 4401, status: 5, comm: 'sh', command: '<defunct>' }),
|
||
row({})
|
||
]
|
||
|
||
function tableFor(
|
||
query: LibprocQuery,
|
||
table: readonly LibprocProcessRow[] = TABLE
|
||
): LibprocProcessRow[] | null {
|
||
if (query.tty !== undefined && !/^(?:\/dev\/)?ttys004$/.test(query.tty)) {
|
||
return null
|
||
}
|
||
return table
|
||
.filter((entry) => !query.pids || query.pids.includes(entry.pid))
|
||
.filter((entry) => query.tty === undefined || entry.tdev === TTYS004)
|
||
.map((entry) =>
|
||
query.command ? entry : { ...entry, command: undefined, commandIncomplete: undefined }
|
||
)
|
||
}
|
||
|
||
function installAddon(overrides: Partial<LibprocAddon> = {}): LibprocAddon {
|
||
const addon: LibprocAddon = {
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query)),
|
||
readProcessCwd: vi.fn(async () => null),
|
||
readProcessPath: vi.fn(async () => null),
|
||
...overrides
|
||
}
|
||
setLibprocAddonForTests(addon)
|
||
return addon
|
||
}
|
||
|
||
// Orca's euid is pinned, so neither a root runner nor Windows (no geteuid) changes whose argv is "own".
|
||
const OWN_UID = 501
|
||
const OTHER_UID = 502
|
||
const realGeteuid = process.geteuid
|
||
|
||
beforeEach(() => {
|
||
process.geteuid = () => OWN_UID
|
||
})
|
||
|
||
const PS_RESULT = {
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 1\n',
|
||
stderr: '',
|
||
timedOut: false,
|
||
outputTruncated: false
|
||
}
|
||
|
||
afterEach(() => {
|
||
process.geteuid = realGeteuid
|
||
setLibprocAddonForTests(undefined)
|
||
resetNativeReadsForTests()
|
||
resetPsFallbackForTests()
|
||
runProcess.mockReset()
|
||
vi.useRealTimers()
|
||
})
|
||
|
||
describe('runPsInProcess', () => {
|
||
it('prints a whole-table capture the strict and cheap parsers read like ps output', async () => {
|
||
installAddon()
|
||
const utc = { env: { TZ: 'UTC0', LC_ALL: 'C' }, timeoutMs: 1000 }
|
||
|
||
const full = await runPsInProcess(
|
||
['-axo', 'pid=,ppid=,pgid=,tpgid=,stat=,tty=,lstart=,command='],
|
||
utc
|
||
)
|
||
expect(full?.code).toBe(0)
|
||
// ps sorts by terminal, then pid.
|
||
expect(parseStrictProcessTableRows(full!.stdout)).toEqual([
|
||
expect.objectContaining({ pid: 1, tty: '??', command: '/sbin/launchd' }),
|
||
expect.objectContaining({ pid: 4400, stat: 'T', tty: '??', command: 'vim' }),
|
||
expect.objectContaining({ pid: 4401, stat: 'Z', command: '<defunct>' }),
|
||
expect.objectContaining({ pid: 4200, stat: 'S', tty: 'ttys004', command: '/usr/bin/login' }),
|
||
expect.objectContaining({ pid: 4242, stat: 'Ss', tty: 'ttys004', command: '-zsh' }),
|
||
expect.objectContaining({
|
||
pid: 4300,
|
||
stat: 'S+',
|
||
tty: 'ttys004',
|
||
startTime: 'Thu Sep 3 16:02:01 2026',
|
||
command: 'node /usr/local/bin/claude --resume'
|
||
})
|
||
])
|
||
// Neither the root daemon nor the pane's root login is a pane descendant worth a setuid ps.
|
||
expect(runProcess).not.toHaveBeenCalled()
|
||
|
||
const cheap = await runPsInProcess(['-axo', 'pid=,ppid=,pgid=,tpgid=,stat=,lstart='], utc)
|
||
expect(parseCheapProcessTableRows(cheap!.stdout).at(-1)).toEqual({
|
||
pid: 4300,
|
||
ppid: 4242,
|
||
pgid: 4300,
|
||
tpgid: 4300,
|
||
stat: 'S+',
|
||
startTime: 'Thu Sep 3 16:02:01 2026'
|
||
})
|
||
})
|
||
|
||
it('selects by pid or terminal and exits 1 like ps when nothing matches', async () => {
|
||
const addon = installAddon()
|
||
|
||
expect(await runPsInProcess(['-p', '4300,4300', '-o', 'pid=,tdev='], BUDGET)).toMatchObject({
|
||
code: 0,
|
||
stdout: '4300 16/4\n'
|
||
})
|
||
expect(addon.readProcessTable).toHaveBeenLastCalledWith({ command: false, pids: [4300] })
|
||
expect(await runPsInProcess(['-p', '9999', '-o', 'stat=,lstart='], BUDGET)).toMatchObject({
|
||
code: 1,
|
||
stdout: ''
|
||
})
|
||
const tty = await runPsInProcess(['-o', 'pid=,stat=', '-t', '/dev/ttys004'], BUDGET)
|
||
expect(tty?.stdout).toBe('4200 S\n4242 Ss\n4300 S+\n')
|
||
expect(addon.readProcessTable).toHaveBeenLastCalledWith({ command: false, tty: '/dev/ttys004' })
|
||
expect(await runPsInProcess(['-o', 'pid=', '-t', 'ttys999'], BUDGET)).toMatchObject({
|
||
code: 1,
|
||
timedOut: false
|
||
})
|
||
})
|
||
|
||
it('keeps the real ps for invocations it cannot reproduce, and answers a failing addon with /bin/ps', async () => {
|
||
installAddon()
|
||
expect(await runPsInProcess(['-axo', 'pid=PROCESS_ID'], BUDGET)).toBeNull()
|
||
expect(await runPsInProcess(['-axo', 'pid=,uid='], BUDGET)).toBeNull()
|
||
expect(await runPsInProcess(['-e', '-o', 'pid='], BUDGET)).toBeNull()
|
||
expect(await runPsInProcess(['-p', '1', '-t', 'ttys004', '-o', 'pid='], BUDGET)).toBeNull()
|
||
expect(
|
||
await runPsInProcess(['-axo', 'lstart='], { env: { TZ: 'America/New_York' }, timeoutMs: 1 })
|
||
).toBeNull()
|
||
expect(
|
||
await runPsInProcess(['-axo', 'lstart='], { env: { LC_ALL: 'de_DE.UTF-8' }, timeoutMs: 1 })
|
||
).toBeNull()
|
||
|
||
installAddon({
|
||
readProcessTable: async () => {
|
||
throw new Error('sysctl failed')
|
||
}
|
||
})
|
||
runProcess.mockResolvedValue(PS_RESULT)
|
||
expect(await runPsInProcess(['-axo', 'pid='], BUDGET)).toEqual(PS_RESULT)
|
||
expect(runProcess).toHaveBeenCalledWith(
|
||
expect.objectContaining({ program: '/bin/ps', args: ['-axo', 'pid='] })
|
||
)
|
||
|
||
setLibprocAddonForTests(null)
|
||
expect(await runPsInProcess(['-axo', 'pid='], BUDGET)).toBeNull()
|
||
})
|
||
|
||
it('times out each caller of a stalled shared read like a killed ps', async () => {
|
||
vi.useFakeTimers()
|
||
const addon = installAddon({ readProcessTable: vi.fn(() => new Promise<never>(() => {})) })
|
||
|
||
const short = runPsInProcess(['-p', '4300', '-o', 'pid=,tpgid=,tty='], { timeoutMs: 125 })
|
||
const long = runPsInProcess(['-p', '4300', '-o', 'pid=,tpgid=,tty='], { timeoutMs: 1000 })
|
||
await vi.advanceTimersByTimeAsync(125)
|
||
await expect(short).resolves.toMatchObject({ code: null, timedOut: true, stdout: '' })
|
||
await vi.advanceTimersByTimeAsync(875)
|
||
await expect(long).resolves.toMatchObject({ timedOut: true })
|
||
expect(addon.readProcessTable).toHaveBeenCalledTimes(1)
|
||
})
|
||
|
||
it('asks ps once for argv only it can read, for another user’s process under a pane shell', async () => {
|
||
const sudo = row({
|
||
pid: 4500,
|
||
ppid: 4242,
|
||
pgid: 4500,
|
||
tpgid: 4300,
|
||
tdev: TTYS004,
|
||
flags: 0x2,
|
||
comm: 'node',
|
||
command: '/usr/local/bin/node'
|
||
})
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, sudo]))
|
||
})
|
||
runProcess.mockResolvedValue({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 4500 node /usr/local/lib/claude-code/cli.js\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
})
|
||
const args = ['-o', 'pid=,ppid=,pgid=,tpgid=,stat=,command=', '-t', 'ttys004']
|
||
const options = { env: { LC_ALL: 'en_US.UTF-8' }, timeoutMs: 1000 }
|
||
|
||
for (let poll = 0; poll < 2; poll += 1) {
|
||
const result = await runPsInProcess(args, options)
|
||
expect(result?.stdout).toContain(
|
||
'4500 4242 4500 4300 S node /usr/local/lib/claude-code/cli.js'
|
||
)
|
||
expect(result?.stdout).toContain('4200 1 4200 0 S /usr/bin/login')
|
||
}
|
||
expect(runProcess).toHaveBeenCalledTimes(1)
|
||
expect(runProcess).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
program: '/bin/ps',
|
||
args: ['-ww', '-o', 'pid=,command=', '-p', '4500']
|
||
})
|
||
)
|
||
})
|
||
|
||
it('never joins a read submitted before the request, so stale rows never reach a caller', async () => {
|
||
type Rows = LibprocProcessRow[] | null
|
||
let finishStale: (rows: Rows) => void = () => {}
|
||
const stale = new Promise<Rows>((resolve) => {
|
||
finishStale = resolve
|
||
})
|
||
const readProcessTable = vi
|
||
.fn<(query: LibprocQuery) => Promise<Rows>>()
|
||
.mockReturnValueOnce(stale)
|
||
.mockImplementation(async (query) => tableFor(query))
|
||
installAddon({ readProcessTable })
|
||
const args = ['-axo', 'pid=,ppid=']
|
||
|
||
expect(await runPsInProcess(args, { timeoutMs: 10 })).toMatchObject({ timedOut: true })
|
||
const fresh = runPsInProcess(args, BUDGET)
|
||
finishStale([row({ pid: 9999, ppid: 4242 })])
|
||
expect((await fresh)?.stdout).not.toContain('9999')
|
||
expect(readProcessTable).toHaveBeenCalledTimes(2)
|
||
})
|
||
|
||
it('queues past the read cap without retaining timed-out waiters, then hands a stalled pool to budgeted ps', async () => {
|
||
vi.useFakeTimers()
|
||
const addon = installAddon({ readProcessTable: vi.fn(() => new Promise<never>(() => {})) })
|
||
runProcess.mockResolvedValue(PS_RESULT)
|
||
const read = (pid: number, timeoutMs = 50) =>
|
||
runPsInProcess(['-p', String(pid), '-o', 'pid='], { timeoutMs })
|
||
|
||
// Eight distinct reads fill the cap and stall; overflow callers wait on one slot notifier.
|
||
const capped = Array.from({ length: 8 }, (_, index) => read(5000 + index, 10_000))
|
||
const overflow = Array.from({ length: 300 }, (_, index) => read(6000 + index))
|
||
expect(nativeReadWaitersForTests()).toBe(300)
|
||
await vi.advanceTimersByTimeAsync(50)
|
||
expect(nativeReadWaitersForTests()).toBe(0)
|
||
// The stalled burst costs the fork budget, not one ps per caller; the rest time out like a killed ps.
|
||
await vi.advanceTimersByTimeAsync(50)
|
||
const results = await Promise.all(overflow)
|
||
expect(results.filter((result) => result?.timedOut === false)).toHaveLength(4)
|
||
expect(results.filter((result) => result?.timedOut === true)).toHaveLength(296)
|
||
expect(runProcess).toHaveBeenCalledTimes(4)
|
||
|
||
// The breaker stays open for its cool-down: callers go to ps without queueing on the pool.
|
||
await vi.advanceTimersByTimeAsync(1_000)
|
||
expect(await read(7000)).toEqual(PS_RESULT)
|
||
expect(nativeReadWaitersForTests()).toBe(0)
|
||
expect(addon.readProcessTable).toHaveBeenCalledTimes(8)
|
||
await vi.advanceTimersByTimeAsync(10_000)
|
||
expect(await Promise.all(capped)).toEqual(
|
||
Array(8).fill(expect.objectContaining({ timedOut: true }))
|
||
)
|
||
|
||
// Past the cool-down with the cap still held, callers learn it at once instead of after their timeout.
|
||
await vi.advanceTimersByTimeAsync(30_000)
|
||
let settled: unknown = 'pending'
|
||
void read(8000, 5_000).then((result) => {
|
||
settled = result
|
||
})
|
||
await vi.advanceTimersByTimeAsync(0)
|
||
expect(settled).toEqual(PS_RESULT)
|
||
})
|
||
|
||
it('recovers argv for a sudo use_pty tree on its own pty, but not for root daemons or GUI helpers', async () => {
|
||
const TTYS005 = (16 << 24) | 5
|
||
const root = { commandIncomplete: true, flags: 0x2 }
|
||
const table = [
|
||
...TABLE,
|
||
row({
|
||
...root,
|
||
pid: 4600,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
comm: 'sudo',
|
||
command: '/usr/bin/sudo'
|
||
}),
|
||
row({
|
||
...root,
|
||
pid: 4601,
|
||
ppid: 4600,
|
||
tdev: TTYS005,
|
||
comm: 'sudo',
|
||
command: '/usr/bin/sudo'
|
||
}),
|
||
row({ ...root, pid: 4602, ppid: 4601, tdev: TTYS005, comm: 'zsh', command: '/bin/zsh' }),
|
||
row({ ...root, pid: 4603, ppid: 4602, tdev: TTYS005, comm: 'node', command: '/opt/node' }),
|
||
row({ pid: 300, ppid: 1, comm: 'syslogd', command: '/usr/sbin/syslogd' }),
|
||
row({
|
||
pid: 700,
|
||
ppid: 1,
|
||
comm: 'Orca',
|
||
command: '/Applications/Orca',
|
||
commandIncomplete: false
|
||
}),
|
||
row({ pid: 701, ppid: 700, comm: 'helper', command: '/usr/libexec/helper' })
|
||
]
|
||
installAddon({ readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, table)) })
|
||
runProcess.mockResolvedValue({ code: 0, signal: null, stdout: '', stderr: '', timedOut: false })
|
||
|
||
await runPsInProcess(['-axo', 'pid=,command='], { env: { LC_ALL: 'C' }, timeoutMs: 1000 })
|
||
const pids = runProcess.mock.calls[0]?.[0].args.at(-1).split(',').map(Number).sort()
|
||
expect(pids).toEqual([4600, 4601, 4602, 4603])
|
||
})
|
||
|
||
it('asks ps again once exec changes the program behind a cached pid and start time', async () => {
|
||
const shell = row({ pid: 4700, ppid: 4242, tdev: TTYS004, comm: 'zsh', command: '/bin/zsh' })
|
||
let current = shell
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, current]))
|
||
})
|
||
runProcess.mockImplementation(async () => ({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: current === shell ? ' 4700 -zsh\n' : ' 4700 node /opt/claude/cli.js\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
}))
|
||
const args = ['-p', '4700', '-o', 'command=']
|
||
const options = { env: { LC_ALL: 'C' }, timeoutMs: 1000 }
|
||
|
||
expect((await runPsInProcess(args, options))?.stdout).toBe('-zsh\n')
|
||
expect((await runPsInProcess(args, options))?.stdout).toBe('-zsh\n')
|
||
current = { ...shell, comm: 'node', command: '/opt/node' }
|
||
expect((await runPsInProcess(args, options))?.stdout).toBe('node /opt/claude/cli.js\n')
|
||
expect(runProcess).toHaveBeenCalledTimes(2)
|
||
})
|
||
|
||
it('lets concurrent tty= readers wait out one naming pass instead of running ps', async () => {
|
||
installAddon()
|
||
// 4300's read is still stalled from the timeout case above.
|
||
const args = ['-o', 'pid=,tty=', '-p', '4242']
|
||
const results = await Promise.all([1, 2, 3].map(() => runPsInProcess(args, BUDGET)))
|
||
expect(results.map((result) => result?.stdout)).toEqual(Array(3).fill('4242 ttys004\n'))
|
||
expect(runProcess).not.toHaveBeenCalled()
|
||
})
|
||
|
||
it('leaves argv of the user’s own mid-exec processes and of exiting ones to (comm), asking ps only for other users', async () => {
|
||
const foreign = { ppid: 4242, tdev: TTYS004, uid: OTHER_UID, flags: 0x2 }
|
||
const table = [
|
||
...TABLE,
|
||
// A pane looping short commands: our own child between fork and exec reads like another user's.
|
||
row({ pid: 4800, ppid: 4242, tdev: TTYS004, uid: OWN_UID, comm: 'git', command: '(git)' }),
|
||
row({ ...foreign, pid: 4801, flags: 0x2 | 0x2000, comm: 'sudo', command: '(sudo)' }),
|
||
row({ ...foreign, pid: 4802, status: 1, comm: 'sudo', command: '(sudo)' }),
|
||
row({ ...foreign, pid: 4803, comm: 'node', command: '(node)' })
|
||
]
|
||
installAddon({ readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, table)) })
|
||
runProcess.mockResolvedValue({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 4803 node /opt/claude/cli.js\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
})
|
||
|
||
const result = await runPsInProcess(['-axo', 'pid=,command='], BUDGET)
|
||
expect(result?.stdout).toContain('4800 (git)\n')
|
||
expect(result?.stdout).toContain('4803 node /opt/claude/cli.js\n')
|
||
expect(runProcess).toHaveBeenCalledTimes(1)
|
||
expect(runProcess.mock.calls[0]?.[0].args.at(-1)).toBe('4803')
|
||
})
|
||
|
||
it('shares one argv ps among concurrent readers and waits out a failed recovery before asking again', async () => {
|
||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'performance'] })
|
||
const sudo = row({
|
||
pid: 4900,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
uid: OTHER_UID,
|
||
comm: 'node',
|
||
command: '(node)'
|
||
})
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, sudo]))
|
||
})
|
||
// ps omits a process that exited, or one it cannot read either.
|
||
runProcess.mockResolvedValue({ code: 1, signal: null, stdout: '', stderr: '', timedOut: false })
|
||
const read = (): Promise<unknown> =>
|
||
runPsInProcess(['-o', 'pid=,command=', '-t', 'ttys004'], BUDGET)
|
||
|
||
const concurrent = await Promise.all(Array.from({ length: 24 }, read))
|
||
expect(concurrent).toEqual(Array(24).fill(expect.objectContaining({ timedOut: false })))
|
||
expect(runProcess).toHaveBeenCalledTimes(1)
|
||
|
||
await read()
|
||
expect(runProcess).toHaveBeenCalledTimes(1)
|
||
await vi.advanceTimersByTimeAsync(5_000)
|
||
await read()
|
||
expect(runProcess).toHaveBeenCalledTimes(2)
|
||
})
|
||
|
||
it('reuses recovered argv for at most 1.5s, so a same-comm re-exec is seen', async () => {
|
||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'performance'] })
|
||
const sudo = row({
|
||
pid: 5100,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
uid: OTHER_UID,
|
||
comm: 'node',
|
||
command: '(node)'
|
||
})
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, sudo]))
|
||
})
|
||
// Same pid, start time and comm: node re-exec'd from Claude into an unrelated server.
|
||
let argv = 'node /opt/claude/cli.js'
|
||
runProcess.mockImplementation(async () => ({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ` 5100 ${argv}\n`,
|
||
stderr: '',
|
||
timedOut: false
|
||
}))
|
||
const args = ['-p', '5100', '-o', 'command=']
|
||
|
||
expect((await runPsInProcess(args, BUDGET))?.stdout).toBe(`node /opt/claude/cli.js\n`)
|
||
argv = 'node /srv/server.js'
|
||
expect((await runPsInProcess(args, BUDGET))?.stdout).toBe(`node /opt/claude/cli.js\n`)
|
||
await vi.advanceTimersByTimeAsync(1_500)
|
||
expect((await runPsInProcess(args, BUDGET))?.stdout).toBe('node /srv/server.js\n')
|
||
expect(runProcess).toHaveBeenCalledTimes(2)
|
||
})
|
||
|
||
it('waits out a failed recovery from when ps gave up, however long it ran', async () => {
|
||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'performance'] })
|
||
const sudo = row({
|
||
pid: 5200,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
uid: OTHER_UID,
|
||
comm: 'su',
|
||
command: '(su)'
|
||
})
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, sudo]))
|
||
})
|
||
// A loaded host: ps takes 6s, then omits the process.
|
||
let finish: () => void = () => {}
|
||
runProcess.mockImplementation(
|
||
() =>
|
||
new Promise((resolve) => {
|
||
finish = () => resolve({ code: 1, signal: null, stdout: '', stderr: '', timedOut: false })
|
||
})
|
||
)
|
||
const read = () => runPsInProcess(['-p', '5200', '-o', 'command='], { timeoutMs: 10_000 })
|
||
|
||
const slow = read()
|
||
await vi.advanceTimersByTimeAsync(6_000)
|
||
finish()
|
||
await slow
|
||
await read()
|
||
expect(runProcess).toHaveBeenCalledTimes(1)
|
||
})
|
||
|
||
it('recovers a su pane’s argv within the recovery budget however often it is read', async () => {
|
||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'performance'] })
|
||
const su = row({
|
||
pid: 5300,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
uid: OTHER_UID,
|
||
comm: 'su',
|
||
command: '(su)'
|
||
})
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, su]))
|
||
})
|
||
runProcess.mockResolvedValue({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 5300 su -l\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
})
|
||
const args = ['-o', 'pid=,command=', '-t', 'ttys004']
|
||
|
||
const reads: ReturnType<typeof runPsInProcess>[] = []
|
||
for (let poll = 0; poll < 20; poll += 1) {
|
||
// readTerminalProcessRows' budget: a read may wait for the next recovery token.
|
||
reads.push(runPsInProcess(args, { ...BUDGET, timeoutMs: 3_000 }))
|
||
await vi.advanceTimersByTimeAsync(750)
|
||
}
|
||
for (const read of await Promise.all(reads)) {
|
||
expect(read?.stdout).toContain('5300 su -l\n')
|
||
}
|
||
// The recovery budget: a burst of 2, then one per 2s over the ~16s the reads span.
|
||
expect(runProcess.mock.calls.length).toBeLessThanOrEqual(10)
|
||
})
|
||
|
||
it('asks again at once after a recovery ps that timed out, which proved nothing', async () => {
|
||
const su = row({
|
||
pid: 5350,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
uid: OTHER_UID,
|
||
comm: 'su',
|
||
command: '(su)'
|
||
})
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, su]))
|
||
})
|
||
runProcess
|
||
.mockResolvedValueOnce({
|
||
code: null,
|
||
signal: 'SIGTERM',
|
||
stdout: '',
|
||
stderr: '',
|
||
timedOut: true
|
||
})
|
||
.mockResolvedValue({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 5350 su -l\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
})
|
||
const read = () => runPsInProcess(['-p', '5350', '-o', 'command='], BUDGET)
|
||
|
||
expect((await read())?.stdout).toBe('(su)\n')
|
||
expect((await read())?.stdout).toBe('su -l\n')
|
||
})
|
||
|
||
it('recovers the foreground of a terminal where no process of ours is left (exec su)', async () => {
|
||
const root = { tdev: TTYS004, uid: OTHER_UID, flags: 0x2, tpgid: 5400 }
|
||
const table = [
|
||
row({ ...root, pid: 4200, ppid: 1, pgid: 4200, comm: 'login', command: '(login)' }),
|
||
row({ ...root, pid: 5400, ppid: 4200, pgid: 5400, comm: 'sh', command: '(sh)' })
|
||
]
|
||
installAddon({ readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, table)) })
|
||
runProcess.mockResolvedValue({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 5400 -sh\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
})
|
||
|
||
const result = await runPsInProcess(['-o', 'pid=,command=', '-t', 'ttys004'], BUDGET)
|
||
expect(result?.stdout).toContain('5400 -sh\n')
|
||
// The pane's root login is not in the foreground group, so it never costs a ps.
|
||
expect(runProcess.mock.calls.map(([spec]) => spec.args.at(-1))).toEqual(['5400'])
|
||
})
|
||
|
||
describe('recovered argv age in strict evidence', () => {
|
||
// Each case its own pid: the argv cache is process-wide.
|
||
let pid = 5500
|
||
const agent = () =>
|
||
row({
|
||
pid,
|
||
ppid: 4242,
|
||
tdev: TTYS004,
|
||
uid: OTHER_UID,
|
||
comm: 'node',
|
||
command: '(node)'
|
||
})
|
||
const args = ['-o', 'pid=,command=', '-t', 'ttys004']
|
||
// The production reader, stamping on the same monotonic clock the fake timers drive.
|
||
const strictReader = () =>
|
||
createProcessTableSnapshotReader({
|
||
runPs: (_fresh, inputAt) =>
|
||
runPsInProcess(args, { ...BUDGET, timeoutMs: 10_000, onArgvReadAt: inputAt }).then(
|
||
(result) => result?.stdout
|
||
),
|
||
now: () => performance.now(),
|
||
ttlMs: 0
|
||
})
|
||
const psAnswers = (argv: () => string) =>
|
||
runProcess.mockImplementation(async () => ({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ` ${pid} ${argv()}\n`,
|
||
stderr: '',
|
||
timedOut: false
|
||
}))
|
||
|
||
beforeEach(() => {
|
||
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'performance'] })
|
||
installAddon({
|
||
readProcessTable: vi.fn(async (query: LibprocQuery) => tableFor(query, [...TABLE, agent()]))
|
||
})
|
||
})
|
||
|
||
it('ages a capture by the cached argv it reused, so a same-comm re-exec cannot read as fresh', async () => {
|
||
let argv = 'node /opt/claude/cli.js'
|
||
psAnswers(() => argv)
|
||
const reader = strictReader()
|
||
await reader.getSnapshotWithAge()
|
||
// node re-execs into a server; same pid, start time and comm.
|
||
argv = 'node /srv/server.js'
|
||
await vi.advanceTimersByTimeAsync(1_400)
|
||
|
||
const snapshot = await reader.getSnapshotWithAge()
|
||
expect(snapshot.value).toContain('claude')
|
||
expect(snapshot.capturedAgeMs).toBeGreaterThanOrEqual(1_400)
|
||
})
|
||
|
||
it('never answers a reader with a recovery begun more than 1.5s before it asked', async () => {
|
||
pid = 5600
|
||
let finishSlow: (stdout: string) => void = () => {}
|
||
runProcess
|
||
.mockImplementationOnce(
|
||
() =>
|
||
new Promise((resolve) => {
|
||
finishSlow = (stdout) =>
|
||
resolve({ code: 0, signal: null, stdout, stderr: '', timedOut: false })
|
||
})
|
||
)
|
||
.mockResolvedValue({
|
||
code: 0,
|
||
signal: null,
|
||
stdout: ' 5600 node /srv/server.js\n',
|
||
stderr: '',
|
||
timedOut: false
|
||
})
|
||
// Two captures (say the daemon's and main's), sharing one recovery that takes 6s.
|
||
void strictReader().getSnapshotWithAge()
|
||
await vi.advanceTimersByTimeAsync(5_900)
|
||
|
||
const late = strictReader().getSnapshotWithAge()
|
||
await vi.advanceTimersByTimeAsync(100)
|
||
finishSlow(' 5600 node /opt/claude/cli.js\n')
|
||
const snapshot = await late
|
||
expect(snapshot.value).toContain('/srv/server.js')
|
||
expect(runProcess).toHaveBeenCalledTimes(2)
|
||
})
|
||
})
|
||
})
|