Files
orca/src/main/daemon/daemon-init.ts
T
Jinwoo HongandOrca daaf2aa73d feat(settings): manage sessions panel + pid-recycling daemon guard (#1343)
* fix(daemon): guard killStaleDaemon against pid recycling

The daemon's pid file carried only a bare integer, so killStaleDaemon had
no way to verify the current owner of that pid was still the process it
forked. Unix pids recycle — on macOS the default ceiling is ~2048 and
wraps in minutes under load — so blind SIGTERM/SIGKILL on a recycled pid
could hit an unrelated user process (editor, language server, background
task) silently.

Three moving pieces:

1. Pid file now carries startedAtMs. daemon-spawner exports DaemonPidFile
   and serializeDaemonPidFile; daemon-init fills startedAtMs from
   getProcessStartedAtMs(child.pid) right after the daemon signals ready.
   parseDaemonPidFile tries JSON first and falls back to bare-integer for
   backward compatibility (startedAtMs: null on legacy files).

2. isDaemonProcess takes expected startedAtMs. In addition to the cmdline
   match, it consults getProcessStartedAtMs(pid) and compares with a
   ±1.5s tolerance. Null on either side is fail-open so the guard never
   strengthens existing behavior negatively (Windows, legacy pid files,
   kernel-thread /proc failures).

3. SIGKILL re-check. The SIGTERM-then-wait window is up to 3s — long
   enough for the pid to be recycled if the original daemon dies during
   the wait. Before escalating to SIGKILL, isDaemonProcess runs again;
   on mismatch we log reason=pid_recycled and skip SIGKILL.

Carried forward from #1323 as a standalone safety fix — independent of
that PR's router/drain machinery, which is not shipping.

Co-authored-by: Orca <help@stably.ai>

* test(daemon): cover pid-recycling guard surface in daemon-health

Adds unit coverage for the new Phase 0 surface:

- parseDaemonPidFile: JSON round-trip, JSON without startedAtMs,
  bare-integer fallback for legacy pid files, malformed-input rejection.
- startTimeMatches: null-expected fail-open, null-actual fail-open,
  within-tolerance match, outside-tolerance rejection.
- killStaleDaemon: with a mismatched startedAtMs in the pid file,
  assert that no SIGTERM/SIGKILL is sent even though the liveness probe
  (process.kill(pid, 0)) runs.

startTimeMatches was promoted from module-private to exported so it can
be exercised directly — it's a pure function with no internal state.

Co-authored-by: Orca <help@stably.ai>

* feat(settings): manage sessions panel for daemon staleness UX

Add a Manage Sessions settings panel with list/kill-all/kill-one/restart
backed by a new pty:management IPC surface. Rows are hover-highlighted and
click to reveal the corresponding terminal pane, mirroring the bottom
status-bar sessions popover. Kill-all and restart-daemon are icon buttons
(Trash2, RotateCw) with tooltips so the restart action doesn't collide with
the row-refresh RefreshCw icon.

Co-authored-by: Orca <help@stably.ai>

* fix(settings): honest killAll counts + suppress post-kill spawn toast

killAll now snapshots the initial session IDs and polls listSessions every
100ms for up to 6.5s — past the daemon's 5s SIGTERM→SIGKILL ladder — so
well-behaved shells hosting long-running agents finish their SIGTERM
handlers before we classify them as "refused to exit." Shutdowns fire once
per initial session (no retry spam), and fresh session IDs that appear
mid-poll (renderer remounts) don't inflate remainingCount.

pty-transport's connect() catch now detects the adapter's
TerminalKilledError tombstone rejection ("...was explicitly killed") and
suppresses the red "file an issue" toast. After Kill All, a pane remount
would call pty:spawn on the dead session ID; surfacing the tombstone as
a scary error misrepresented an intentional user action. The pane still
renders "Process exited" via the normal lifecycle.

Co-authored-by: Orca <help@stably.ai>

* feat(daemon): foundation for pty:management IPC surface

Adds the plumbing the Manage Sessions settings panel depends on:

- daemon-init exports getDaemonProvider / replaceDaemonProvider /
  restartDaemon / cleanupDaemonForProtocol so the pty:management
  handlers can access the current provider and coordinate a clean
  restart without importing window-services internals.
- daemon-pty-router exports getAllAdapters so the killAll / listSessions
  handlers can fan across the current adapter plus any legacy-protocol
  adapters still attached for in-flight sessions.
- daemon-pty-adapter gains a listSessions RPC and readonly
  protocolVersion so the handlers can annotate each session with the
  adapter it belongs to and route killOne back to the right adapter.
- types.ts exports DaemonSessionInfo (SessionInfo + protocolVersion)
  as the shared shape the preload API surface mirrors.
- ipc/pty.ts, attach-main-window-services, TerminalPane and
  terminal-search pick up the small bindings required to wire the
  router through existing code paths without regressions.

Co-authored-by: Orca <help@stably.ai>

* fix(settings): remove high-session-count warning banner

The banner nagged at 20 sessions, which is well within normal use for
users with many open worktrees. Count is already visible in the header
bar, and the table supports per-row and bulk kills, so the banner added
noise without actionable value.

Co-authored-by: Orca <help@stably.ai>

---------

Co-authored-by: Orca <help@stably.ai>
2026-05-02 23:05:44 -07:00

490 lines
18 KiB
TypeScript

/* eslint-disable max-lines -- Why: this module owns the complete daemon
lifecycle for the Electron main process — init, out-of-process launch,
current+legacy adapter wiring, restart orchestration (the 7-step sequence
from docs/daemon-staleness-ux.md §Phase 1), and teardown on app quit. Splitting
it would scatter the "swap the running provider atomically" invariant across
files with no cleaner ownership seam: restart, replaceDaemonProvider, and the
module-level spawner/adapter singletons must stay co-located so a future
change cannot leave them drifting out of sync. */
import { join } from 'path'
import { app } from 'electron'
import { mkdirSync, existsSync, unlinkSync, writeFileSync } from 'fs'
import { fork } from 'child_process'
import { connect } from 'net'
import {
DaemonSpawner,
getDaemonPidPath,
getDaemonSocketPath,
getDaemonTokenPath,
serializeDaemonPidFile,
type DaemonLauncher
} from './daemon-spawner'
import { DaemonPtyAdapter } from './daemon-pty-adapter'
import { DaemonPtyRouter } from './daemon-pty-router'
import { DaemonClient } from './client'
import {
PREVIOUS_DAEMON_PROTOCOL_VERSIONS,
PROTOCOL_VERSION,
type ListSessionsResult
} from './types'
import { getProcessStartedAtMs, healthCheckDaemon, killStaleDaemon } from './daemon-health'
import {
setLocalPtyProvider,
unbindLocalProviderListeners,
rebindLocalProviderListeners
} from '../ipc/pty'
let spawner: DaemonSpawner | null = null
let adapter: DaemonPtyRouter | DaemonPtyAdapter | null = null
// Why: coalesce concurrent restartDaemon() calls so two clicks (or a UI
// click racing an internal caller) can't both enter the 7-step sequence —
// the second entry would read the already-disposed current adapter and
// race cleanupDaemonForProtocol against a half-spawned replacement.
let restartInFlight: Promise<RestartDaemonResult> | null = null
function getRuntimeDir(): string {
const dir = join(app.getPath('userData'), 'daemon')
mkdirSync(dir, { recursive: true })
return dir
}
function getHistoryDir(): string {
const dir = join(app.getPath('userData'), 'terminal-history')
mkdirSync(dir, { recursive: true })
return dir
}
function getDaemonEntryPath(): string {
const appPath = app.getAppPath()
// Why: electron-builder unpacks daemon-entry.js so child_process.fork() can
// execute it from disk. In packaged apps app.getAppPath() points at
// app.asar, so redirect to the unpacked sibling before joining the script.
const basePath = app.isPackaged ? appPath.replace('app.asar', 'app.asar.unpacked') : appPath
return join(basePath, 'out', 'main', 'daemon-entry.js')
}
// Why: before spawning a new daemon, check if an existing one is alive by
// attempting a TCP connection to the socket. If it connects, the daemon
// survived from a previous app session — reuse it instead of spawning.
function probeSocket(socketPath: string): Promise<boolean> {
return new Promise((resolve) => {
if (process.platform !== 'win32' && !existsSync(socketPath)) {
resolve(false)
return
}
const sock = connect({ path: socketPath })
const timer = setTimeout(() => {
sock.destroy()
resolve(false)
}, 1000)
sock.on('connect', () => {
clearTimeout(timer)
sock.destroy()
resolve(true)
})
sock.on('error', () => {
clearTimeout(timer)
resolve(false)
})
})
}
function createOutOfProcessLauncher(runtimeDir: string): DaemonLauncher {
return async (socketPath, tokenPath) => {
const healthy = await healthCheckDaemon(socketPath, tokenPath)
if (healthy) {
// Why: daemon is already running from a previous app session and
// responded to a protocol-level ping. Safe to reuse.
return {
shutdown: async () => {
await cleanupDaemonForProtocol(runtimeDir, PROTOCOL_VERSION)
}
}
}
// Why: a raw socket can outlive a broken or wedged daemon. Kill by PID
// before respawn so the new daemon does not race the stale process.
await killStaleDaemon(runtimeDir, socketPath, tokenPath)
const entryPath = getDaemonEntryPath()
const userDataPath = app.getPath('userData')
const child = fork(entryPath, ['--socket', socketPath, '--token', tokenPath], {
// Why: detached + unref lets the daemon outlive the Electron process.
// stdio 'ignore' prevents the child from holding the parent's stdout
// open, which would prevent Electron from exiting cleanly.
detached: true,
stdio: ['ignore', 'ignore', 'ignore', 'ipc'],
// Why: ELECTRON_RUN_AS_NODE makes the forked process run as a plain
// Node.js process instead of an Electron renderer/main process. Without
// it, Electron's GPU/display initialization can interfere with native
// module operations like node-pty's posix_spawn of the spawn-helper.
env: {
...process.env,
ELECTRON_RUN_AS_NODE: '1',
// Why: the detached daemon is plain Node and cannot call Electron's
// app.getPath(), but shell-ready rcfiles must live outside swept tmp.
ORCA_USER_DATA_PATH: userDataPath
}
})
// Wait for the daemon to signal readiness via IPC
await new Promise<void>((resolve, reject) => {
const fail = (error: Error): void => {
clearTimeout(timer)
if (child.pid) {
try {
process.kill(child.pid, 'SIGTERM')
} catch {
// Already dead
}
}
reject(error)
}
const timer = setTimeout(() => {
fail(new Error('Daemon startup timed out'))
}, 10000)
child.on('message', (msg: unknown) => {
if (msg && typeof msg === 'object' && (msg as { type?: string }).type === 'ready') {
clearTimeout(timer)
if (child.pid) {
// Why: JSON pid file carries pid + process start time so later
// killStaleDaemon() can verify the pid still belongs to the daemon
// we forked before SIGTERMing it. Prevents pid-recycling hazard
// where the OS hands the daemon's old pid to an unrelated process.
writeFileSync(
getDaemonPidPath(runtimeDir),
serializeDaemonPidFile({
pid: child.pid,
startedAtMs: getProcessStartedAtMs(child.pid)
}),
{ mode: 0o600 }
)
}
// Why: disconnect IPC channel and unref so Electron can exit
// without waiting for the daemon. The daemon keeps running.
child.disconnect()
child.unref()
resolve()
}
})
child.on('error', (err) => {
fail(err)
})
child.on('exit', (code) => {
fail(new Error(`Daemon exited during startup with code ${code}`))
})
})
return {
shutdown: async () => {
if (child.pid) {
try {
process.kill(child.pid, 'SIGTERM')
} catch {
// Already dead
}
}
}
}
}
}
export async function initDaemonPtyProvider(): Promise<void> {
const runtimeDir = getRuntimeDir()
const newSpawner = new DaemonSpawner({
runtimeDir,
launcher: createOutOfProcessLauncher(runtimeDir)
})
// Why: assign spawner/adapter only after both succeed. If ensureRunning()
// throws, a stale spawner would prevent shutdownDaemon() from cleaning up
// correctly on retry.
const info = await newSpawner.ensureRunning()
const newAdapter = new DaemonPtyAdapter({
socketPath: info.socketPath,
tokenPath: info.tokenPath,
historyPath: getHistoryDir(),
// Why: when the daemon process dies (e.g. killed by a signal, OOM, or
// cascading from a force-quit of child processes), the adapter's
// ensureConnected() detects the dead socket and calls this to fork a
// replacement daemon before retrying the connection.
respawn: async () => {
console.warn('[daemon] Daemon process died — respawning')
newSpawner.resetHandle()
await newSpawner.ensureRunning()
}
})
const legacyAdapters = await createLegacyDaemonAdapters(runtimeDir)
const routedAdapter =
legacyAdapters.length > 0
? new DaemonPtyRouter({
current: newAdapter,
legacy: legacyAdapters
})
: newAdapter
if (routedAdapter instanceof DaemonPtyRouter) {
await routedAdapter.discoverLegacySessions()
}
spawner = newSpawner
adapter = routedAdapter
setLocalPtyProvider(routedAdapter)
}
// Why: the Manage Sessions IPC handlers need read access to the current
// adapter/router to list sessions, kill them, etc. Exposed as a narrow getter
// rather than exporting the module-level variable to keep the "swap on
// restart" invariant in one place (replaceDaemonProvider).
export function getDaemonProvider(): DaemonPtyRouter | DaemonPtyAdapter | null {
return adapter
}
// Why: the "Restart daemon" flow rebuilds the current-protocol adapter and
// must update both the module-level `adapter` singleton here and the
// `localProvider` reference inside ipc/pty.ts. Without this helper they could
// drift — app-quit would dispose a stale adapter reference.
export function replaceDaemonProvider(newAdapter: DaemonPtyAdapter | DaemonPtyRouter): void {
adapter = newAdapter
setLocalPtyProvider(newAdapter)
}
export type RestartDaemonResult = {
killedCount: number
}
// Why: the 7-step sequence from docs/daemon-staleness-ux.md §Phase 1 restart.
// Current-protocol only — legacy adapters are preserved and route to their
// original daemons with no respawn path. See the design doc for rationale on
// each step, notably why synthetic exits must fan out *before* the listener
// unsubscribe.
export async function restartDaemon(): Promise<RestartDaemonResult> {
if (restartInFlight) {
return restartInFlight
}
restartInFlight = runRestartDaemon().finally(() => {
restartInFlight = null
})
return restartInFlight
}
async function runRestartDaemon(): Promise<RestartDaemonResult> {
const currentSpawner = spawner
const currentAdapter = adapter
if (!currentSpawner || !currentAdapter) {
throw new Error('restartDaemon called before initDaemonPtyProvider')
}
const runtimeDir = getRuntimeDir()
const currentOnly =
currentAdapter instanceof DaemonPtyRouter ? currentAdapter.getCurrentAdapter() : currentAdapter
const legacyAdapters =
currentAdapter instanceof DaemonPtyRouter ? [...currentAdapter.getLegacyAdapters()] : []
// Step 1: synthesize pty:exit for every active session on the current
// adapter BEFORE any teardown. The daemon's kill-all-and-shutdown path
// explicitly does not fan onExit to clients (session.ts:246-252), so
// without this the renderer would never see exits and would black-hole
// writes against the disposed adapter.
const killedCount = currentOnly.getActiveSessionIds().length
currentOnly.fanoutSyntheticExits(-1)
// Step 2: detach renderer listeners from the current adapter. Must happen
// AFTER step 1 so the synthesized exits actually reach the renderer, and
// BEFORE step 6 so the new provider isn't bound with stale listeners.
unbindLocalProviderListeners()
// Step 3: kill the current-protocol daemon process (shutdown RPC → fallback
// killStaleDaemon → socket/pid unlink). Legacy adapters untouched.
await cleanupDaemonForProtocol(runtimeDir, PROTOCOL_VERSION)
// Step 4: reuse the existing spawner so the respawn closure baked into
// long-lived adapters stays valid. Do NOT construct a new DaemonSpawner.
currentSpawner.resetHandle()
const info = await currentSpawner.ensureRunning()
// Step 5: build a fresh current adapter against the respawned daemon. Its
// respawn callback closes over the same spawner instance (identical to the
// crash-respawn closure in initDaemonPtyProvider).
const newCurrent = new DaemonPtyAdapter({
socketPath: info.socketPath,
tokenPath: info.tokenPath,
historyPath: getHistoryDir(),
respawn: async () => {
console.warn('[daemon] Daemon process died — respawning')
currentSpawner.resetHandle()
await currentSpawner.ensureRunning()
}
})
// Re-wrap in router if there were legacy adapters at startup; otherwise
// point straight at the new adapter. Legacy instances are preserved by
// reference — they still route to the same pre-upgrade daemons.
const newProvider =
legacyAdapters.length > 0
? new DaemonPtyRouter({ current: newCurrent, legacy: legacyAdapters })
: newCurrent
if (newProvider instanceof DaemonPtyRouter) {
await newProvider.discoverLegacySessions()
}
// Why: drain the outgoing router's subscriptions from the shared legacy
// adapters before installing the new router (which subscribes fresh). Must
// run *after* the new provider exists so no adapter event is unhandled in
// the narrow window, and *before* replaceDaemonProvider so the swap is
// atomic from the renderer's perspective. Plain dispose() would also tear
// down the legacy adapters themselves — use the router-only variant.
if (currentAdapter instanceof DaemonPtyRouter) {
currentAdapter.disposeRouterOnly()
}
// Step 6: swap module state (adapter + localProvider) atomically.
replaceDaemonProvider(newProvider)
// Step 7: rebind renderer listeners against the new provider.
rebindLocalProviderListeners()
return { killedCount }
}
// Why: disconnect from the daemon without killing it. The daemon runs as a
// separate process and survives app quit — sessions stay alive for warm
// reattach on next launch. Leave history sessions marked "unclean" here so a
// later daemon crash while Orca is closed is still recoverable on next launch.
export async function disconnectDaemon(): Promise<void> {
await adapter?.disconnectOnly()
adapter = null
}
/** Kill the daemon and all its sessions. Use for full cleanup only. */
export async function shutdownDaemon(): Promise<void> {
adapter?.dispose()
adapter = null
await spawner?.shutdown()
spawner = null
try {
unlinkSync(getDaemonPidPath(getRuntimeDir()))
} catch {
// Best-effort
}
}
export type OrphanedDaemonCleanupResult = {
/** True when we detected a live daemon socket and connected to tear it down.
* False when no daemon was running (fresh install or clean previous quit). */
cleaned: boolean
/** Number of live PTY sessions killed during cleanup. The caller surfaces this
* to the user so they know what background work was stopped. */
killedCount: number
}
export async function cleanupDaemonForProtocol(
runtimeDir: string,
protocolVersion: number
): Promise<OrphanedDaemonCleanupResult> {
const socketPath = getDaemonSocketPath(runtimeDir, protocolVersion)
const tokenPath = getDaemonTokenPath(runtimeDir, protocolVersion)
const pidPath = getDaemonPidPath(runtimeDir, protocolVersion)
const alive = await probeSocket(socketPath)
if (!alive) {
// Why: still best-effort remove a stale socket file so a future opt-in
// launch doesn't hit EADDRINUSE when the daemon tries to bind.
if (process.platform !== 'win32' && existsSync(socketPath)) {
try {
unlinkSync(socketPath)
} catch {
// Best-effort
}
}
try {
unlinkSync(pidPath)
} catch {
// Best-effort
}
return { cleaned: false, killedCount: 0 }
}
const client = new DaemonClient({ socketPath, tokenPath, protocolVersion })
let killedCount = 0
let didRequestShutdown = false
let didKillStaleDaemon = false
try {
await client.ensureConnected()
const sessions = await client
.request<ListSessionsResult>('listSessions', undefined)
.catch(() => ({ sessions: [] }))
killedCount = sessions.sessions.filter((s) => s.isAlive).length
// Why: the daemon exposes a single-shot `shutdown` RPC (daemon-server.ts)
// that kills every session and then terminates its own process. Using it
// avoids the race between per-session `kill` calls and the daemon exiting.
await client.request('shutdown', { killSessions: true }).catch(() => {
// Daemon exits immediately after handling the RPC — the socket may close
// before the reply round-trips. Treat that as success.
})
didRequestShutdown = true
} catch {
// Why: previous-protocol daemons may be wedged or too old to complete the
// RPC cleanup path. Fall back to PID cleanup, but daemon-health only
// unlinks a live socket after proving it killed the matching process.
didKillStaleDaemon = await killStaleDaemon(runtimeDir, socketPath, tokenPath, protocolVersion)
} finally {
client.disconnect()
}
// Why: after `shutdown`, the daemon unlinks its socket itself — but on some
// crash paths the file lingers. Clean up defensively so a later opt-in
// relaunch can bind cleanly.
if (didRequestShutdown && process.platform !== 'win32' && existsSync(socketPath)) {
try {
unlinkSync(socketPath)
} catch {
// Best-effort
}
}
try {
unlinkSync(pidPath)
} catch {
// Best-effort
}
return { cleaned: didRequestShutdown || didKillStaleDaemon, killedCount }
}
async function createLegacyDaemonAdapters(runtimeDir: string): Promise<DaemonPtyAdapter[]> {
const adapters: DaemonPtyAdapter[] = []
for (const protocolVersion of PREVIOUS_DAEMON_PROTOCOL_VERSIONS) {
const socketPath = getDaemonSocketPath(runtimeDir, protocolVersion)
const tokenPath = getDaemonTokenPath(runtimeDir, protocolVersion)
if (!(await probeSocket(socketPath))) {
continue
}
// Why: old daemon PTYs can be running long-lived agents during an app
// upgrade. Keep those sessions routed to their original daemon while new
// terminals use the current protocol, instead of killing background work.
// Legacy adapters intentionally do not respawn: respawning an old protocol
// daemon from new code would recreate stale env semantics and can be less
// predictable than letting the session fail if that old daemon dies.
// Why historyPath is still passed: checkpoint writes will fail silently
// (pre-v4 daemons don't support getSnapshot), but the HistoryManager is
// still needed for cleanup — close/exit events must remove history dirs
// and mark meta.json as ended. Without it, a later v4 session reusing
// the same ID could false-restore stale scrollback.bin.
adapters.push(
new DaemonPtyAdapter({
socketPath,
tokenPath,
protocolVersion,
historyPath: getHistoryDir()
})
)
}
return adapters
}