From 27dabcbe9097376bc4ecaa2501571de716d673ea Mon Sep 17 00:00:00 2001 From: Jinwoo-H Date: Mon, 21 Sep 2026 01:25:33 -0400 Subject: [PATCH] docs(macos): fold PostHog field data into the STA-7948 plan and drop the helper probe daemon_pty_cwd_denied fired for 1,438 users in 21 days (2.8% of users adopting a different-version daemon, 96% on Documents/Desktop/Downloads), so the daemon's existing verdict already observes the failure and the helper probe for old daemons has no population left to cover. A fresh same-version daemon ended the denial for about 69% of observable users; the notice copy now tells the rest to re-allow the folder. --- .../macos-tcc-explicit-recovery-design.md | 208 ++++++++---------- docs/plans/validation/README.md | 33 +++ 2 files changed, 129 insertions(+), 112 deletions(-) diff --git a/docs/plans/macos-tcc-explicit-recovery-design.md b/docs/plans/macos-tcc-explicit-recovery-design.md index 1e55e425ac6..7163f91b02e 100644 --- a/docs/plans/macos-tcc-explicit-recovery-design.md +++ b/docs/plans/macos-tcc-explicit-recovery-design.md @@ -1,138 +1,122 @@ # STA-7948: macOS daemon folder-access mismatch — finalized plan -Status: finalized 2026-09-21 after live evidence on a production adopted daemon. Supersedes the -2026-09-14 helper-first design (kept below as Layer B, now evidence-gated). No production code yet. -Evidence: [validation/README.md](validation/README.md) (section "2026-09-21 live evidence"). +Status: finalized 2026-09-21 after live evidence on a production adopted daemon and PostHog field +data. Supersedes the 2026-09-14 helper-first design. No production code yet. +Evidence: [validation/README.md](validation/README.md) (sections "2026-09-21 live evidence" and +"PostHog field data"). ## 1. What is settled -1. **macOS TCC lets `access(2)` and `stat` succeed on Documents/Desktop/Downloads while - `opendir`/`readdir` fails with EPERM.** Measured in a grant-less launchd process on - `~/Documents`: `access(R_OK|X_OK)=True`, `stat=ok`, `scandir=EPERM`. Full Disk Access folders - behave differently (`access()` is denied too). Consequence: the daemon's existing - `cwdReadableByDaemon` verdict (`accessSync` in `terminal-host-session-create.ts`) is **blind to - the incident's folder class**, and the `daemon_pty_cwd_denied` telemetry has a structural zero - for it. The original report's `ls -lde .` success is consistent with this (it only stats). -2. **Daemon-spawned shells carry the daemon's effective grant.** On the production adopted daemon - (spawned by 1.4.207-adhoc, app now 1.4.206), the daemon's verdict, a login-wrapped shell's - `scandir`, and a control shell agreed on `~/Documents`, `/tmp`, and Full-Disk-Access-gated - `~/Library/Safari`. `responsibility_get_pid_responsible_for_pid` reports the daemon and each - shell as its own responsible pid, never Orca main, yet the shells read Safari, which only - `com.stablyai.orca` holds. `com.stablyai.orca.helper` has no TCC row of its own. -3. **The login wrapper is unconditional in production**: 1734 of 1734 recorded spawns were - `wrapped`. -4. **The incident itself has never been reproduced** (2026-09-01 matrix, 2026-09-21 run). Field - recovery for the one confirmed case reportedly required daemon restart **plus** - `tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca` and re-allowing. +1. **The failure is real, common, and tied to app updates.** PostHog `daemon_pty_cwd_denied` + (shipped in #18043, 2026-09-01) fires only when the daemon reports the cwd unreadable and the + app can read it. In the 21 days to 2026-09-21 it fired for 1,438 users (5,674 events on + Documents, 2,379 on Desktop, 369 on Downloads). 1,374 of those users were in a protected folder + class, and 96% carried `app_version_match=different`, i.e. an adopted daemon forked by an + older app build. That is 2.8% of the 50,876 users who adopted a different-version daemon. + Denials recur: 528 users hit it on two or more days. +2. **The daemon's existing verdict detects it.** The field data above is produced by the + current `accessSync(R_OK|X_OK)` check in `terminal-host-session-create.ts`, so no new daemon + code is required to observe the failure on daemons that already ship that field. A separate + grant-less launchd probe on this machine showed a different TCC mode on `~/Documents` + (`access()` passes, `opendir` fails), so enumeration is the more faithful check and Layer A + switches to it, but detection does not wait on that. +3. **A fresh daemon from the current app fixes most cases, not all.** Among denied users whose + later history is visible, 151 stopped being denied after a same-version daemon appeared and + 68 were denied again even with a same-version daemon (about 69% versus 31%). The 31% need the + folder grant itself repaired (`tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca` + and re-allow), which matches the one hands-on recovery on record. +4. **Daemon-spawned shells carry the daemon's effective grant.** On the production adopted + daemon here (healthy), the daemon's verdict, a login-wrapped shell's `scandir`, and a control + shell agreed on `~/Documents`, `/tmp`, and Full-Disk-Access-gated `~/Library/Safari`. The + shells report themselves as their own responsible pid yet read Safari, which only + `com.stablyai.orca` holds; `com.stablyai.orca.helper` has no TCC row. The login wrapper is + unconditional in production (1,734 of 1,734 spawns `wrapped`). +5. **The user-visible symptom** (Slack, 2026-09-20): inside a workspace under `~/Documents`, + `ls -lde .` works (stat only), `os.scandir('.')` and opening a file both fail with EPERM, and + Codex starts then dies with "Operation not permitted" reading its cwd. Commands that do not + touch the folder work. macOS 26.5.1. ## 2. Design -Two layers. Layer A ships first and is small. Layer B is the previous helper design, built only -if evidence says the population it covers matters. +One layer. The previous helper probe is dropped (section 4). -### Layer A — enumeration verdict in the daemon, notice in the app (ship now) - -**Daemon.** Replace `isCwdReadableByThisProcess` in -`src/main/daemon/terminal-host-session-create.ts` with real enumeration: `fs.opendirSync(cwd)`, -one `dir.readSync()`, `dir.closeSync()`. `EPERM`/`EACCES` → `false`. `ENOENT`/`ENOTDIR`/anything -else → `true`, exactly as today, so a non-permission failure never masquerades as denial. Keep the -wire field name and type (`cwdReadableByDaemon?: boolean`); its meaning becomes "enumerable by -the daemon". Old clients read it unchanged. This runs where `accessSync` already runs (before the -fork, synchronous, local paths only, skipped for WSL). +**Daemon (hardening, not a prerequisite).** Replace `isCwdReadableByThisProcess` with real +enumeration: `fs.opendirSync(cwd)`, one `dir.readSync()`, `dir.closeSync()`. `EPERM`/`EACCES` → +`false`; `ENOENT`/`ENOTDIR`/anything else → `true`, exactly as today. Keep the wire field name and +type (`cwdReadableByDaemon?: boolean`). Runs where `accessSync` already runs. **Main: evidence.** In `src/main/daemon/daemon-pty-session-spawn.ts`, next to the existing `trackDaemonPtyCwdDeniedIfDiverged` call, record proven divergence in a new -`src/main/daemon/daemon-folder-access-mismatch.ts`: `{ canonicalPath, daemonIdentity: { pid, -startedAtMs, launchNonce }, appSessionId, observedAtMs }`. The app-side comparison uses the same -`opendir`/one-`readdir`/`closedir` sequence (switch `trackDaemonPtyCwdDeniedIfDiverged` from -`accessSync` to it as well). Keep at most one entry per daemon identity; a later spawn that -enumerates successfully on that identity clears it. `canonicalPath` is the cwd actually sent to the -daemon; no `realpath`, no case folding. Local current-protocol adapter only; SSH, WSL, relay, -legacy adapters, and local fallback providers never write evidence. +`src/main/daemon/daemon-folder-access-mismatch.ts`: `{ canonicalPath, cwdClass, daemonIdentity: +{ pid, startedAtMs, launchNonce }, appSessionId, observedAtMs }`. The app-side comparison uses +the same `opendir`/one-`readdir`/`closedir` sequence (switch the telemetry emitter to it too). +Keep at most one entry per daemon identity; a later spawn that succeeds on that identity clears +it. `canonicalPath` is the cwd sent to the daemon; no `realpath`, no case folding. Local +current-protocol adapter only. Daemons that omit the field (pre-#18043) produce no evidence. -**Main: IPC.** Extend the existing `pty:management:macTccAttribution` handler in -`src/main/ipc/pty-management.ts` to return -`{ health, folderAccessMismatch: { daemonScope: string } | null }` where `daemonScope` is a -stable hash of the daemon identity. Mirror the type in `src/preload/api/pty-management-api.ts` -and return `null` from `src/renderer/src/web/preload-api/web-terminal-api.ts`. No new channel, -no push event. +**Main: IPC.** Extend the existing `pty:management:macTccAttribution` handler to return +`{ health, folderAccessMismatch: { daemonScope: string, cwdClass } | null }`. Mirror the type in +`src/preload/api/pty-management-api.ts`; the web fallback returns `null`. No new channel. -**Renderer.** In `src/renderer/src/hooks/useMacTccAttributionSeveredNotice.ts` add a second toast -(`mac-daemon-folder-access-mismatch`) driven by the same focus-time poll, latched per -`daemonScope` per app session, dismissable per scope. Copy (final wording depends on gate G1): +**Renderer.** In `useMacTccAttributionSeveredNotice.ts` add a second toast +(`mac-daemon-folder-access-mismatch`) on the same focus-time poll, latched per `daemonScope` per +app session, dismissable per scope. Copy: -> **Orca's terminal service can't read a folder Orca can.** -> The terminal service was denied access to a workspace folder that Orca itself can read. -> Restart the daemon from Manage Sessions. If macOS asks again, allow the folder. This closes all -> running Orca terminals and agents. +> **Orca's terminal service can't read your Documents folder.** +> Terminals opened in this folder will fail with "Operation not permitted" even though Orca +> itself can read it. Restart the daemon from Manage Sessions; this closes all running Orca +> terminals and agents. If it still fails afterwards, re-allow the folder for Orca in System +> Settings → Privacy & Security → Files and Folders. -Action: the existing **Open Manage Sessions** target and the existing restart confirmation in -`useDaemonActions`. Keep the severed-attribution toast separate. +Substitute Desktop/Downloads from `cwdClass`. Action: the existing **Open Manage Sessions** target +and the existing restart confirmation in `useDaemonActions`. Keep the severed-attribution toast +separate. -**Recovery and clearing.** Use `restartDaemon()` unchanged. Evidence is keyed by daemon identity -and a restart always replaces the identity, so the next poll returns `null` and the hook dismisses -the toast. If the replacement daemon is also denied, the next spawn re-records and re-toasts. No -post-restart probe: the next terminal spawn is the probe. +**Recovery and clearing.** Use `restartDaemon()` unchanged. Evidence is keyed by daemon +identity; a restart replaces the identity, so the next poll returns `null` and the hook dismisses +the toast. If the replacement daemon is also denied (the 31% case), the next spawn re-records and +re-toasts, now with the re-allow sentence doing the work. No post-restart probe. -**Telemetry.** `daemon_pty_cwd_denied` becomes meaningful for the Documents class. Add -`daemon_folder_access_notice` with `{ action: shown | dismissed | restart_clicked }` and the -existing `cwd_class` enum; no paths. +**Telemetry.** Add `daemon_folder_access_notice` with `{ action: shown | dismissed | +restart_clicked, cwd_class }`. Keep `daemon_pty_cwd_denied` as the denominator. -**Tests.** Extend `terminal-host-cwd-readability.test.ts` for opendir mapping (EPERM, EACCES, -ENOENT, ENOTDIR, readable, empty directory). Unit-test the evidence store (records only on -divergence, one per identity, clears on success or identity change). IPC handler shape. Hook: -toast once per scope, dismissal latch, clears when the poll returns `null`, no toast when both -sides fail or the path is missing. No Electron perf gate is needed: nothing new runs on the -spawn path beyond one `opendir`. +**Tests.** `terminal-host-cwd-readability.test.ts` for opendir mapping (EPERM, EACCES, ENOENT, +ENOTDIR, readable, empty). Evidence store: records only on divergence, one per identity, clears +on success or identity change, ignores daemons without the field. IPC shape. Hook: toast once per +scope, dismissal latch, clears when the poll returns `null`, no toast when both sides fail or the +path is missing. No Electron perf gate: nothing new runs on the spawn path beyond one `opendir`. -**Accepted gap.** Daemons already running pre-change code (every adopted daemon deployed today) -never produce the enumeration verdict. They are covered after their next restart or reboot, so -each user is blind for at most one update cycle. Layer B exists for that population. +## 3. Open gate -### Layer B — helper probe for already-deployed old daemons (evidence-gated, unchanged design) - -The 2026-09-14 design (signed helper launched through `createOrAttach` with `shellOverride`, -neutral cwd, nonce-framed single record, direct and login-wrapped strategies, lease and cleanup -contract, one active plus one queued per daemon, Electron A/B performance gate) is the only path -that observes an old daemon's view without new daemon code. Build it only if, after Layer A ships: - -- PostHog `daemon_adopted` shows the adopted-old-daemon population is large enough that a - one-cycle blind spot matters, or -- gate G1 shows the daemon's in-process verdict disagrees with its shells (the daemon enumerates - but the terminal is denied), which would mean Layer A's oracle is wrong for the broken state. - -If Layer B is built, do not route it through the plan's createOrAttach-plus-lease machinery for -new daemons; new daemons get a single capability-gated `directoryAccessProbe` request built on the -existing `ptySpawnHealth` pattern (`runPtySpawnHealthProbe` in -`src/main/daemon/pty-subprocess/spawn-preflight.ts`) with cleanup on `onClientDisconnected`. - -## 3. Open gates - -- **G1 — affected machine, remedy and oracle.** On a machine showing the failure, inside the - affected Orca terminal: +- **G1 — one affected machine, oracle check.** Field data proves the daemon is denied; it does + not prove the daemon's verdict and the terminal's experience always agree. The one hands-on + case (Slack) agrees. Ask the affected user, inside the affected terminal: 1. `python3 -c "import os; list(os.scandir(os.path.expanduser('~/Documents')))"` → expect EPERM. - 2. Settings → Terminal → Manage Sessions → Restart daemon. Open a new terminal, rerun step 1. - 3. If still denied: `tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca`, re-allow - when prompted, rerun step 1. - Decides whether the notice says "restart" or "restart, then re-allow", and whether the daemon - identity is what breaks (step 2 fixes it) or the app's grant (step 3 fixes it). -- **G2 — population.** PostHog: count of `daemon_adopted` per week, split by - `app_version_match`. Ignore `daemon_pty_cwd_denied` for now: it is structurally zero for the - Documents class until Layer A ships. -- **G3 — Documents prompt behaviour on a signed build.** Confirm the daemon's `opendir` on a - never-granted Documents folder does not raise a consent prompt attributed to the helper bundle. - On this machine the grant-less probe was denied silently; verify once on a signed build with a - fresh TCC profile. + 2. Settings → Terminal → Manage Sessions → Restart daemon. New terminal, rerun step 1. + 3. If still denied: `tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca`, re-allow when + prompted, rerun step 1. + This confirms the copy, not the code. -## 4. Action items +Closed: G2 (population, above). G3 (prompt behaviour of the daemon's `opendir` on a signed build) +is folded into the Layer A PR's manual check. -1. Send G1 to the affected user (draft below). Owner: Jinwoo. -2. Run G2 in PostHog. Owner: whoever holds PostHog access. -3. Land this doc and the 2026-09-21 evidence on PR #21740; keep the durable-relocation design - separate and unblocked. -4. Implement Layer A in one PR against `main` (daemon verdict, evidence store, IPC field, toast, - tests). Do not wait on G1 for the code; G1 only changes the copy. -5. Decide Layer B after two release cycles of Layer A telemetry. +## 4. Why the helper probe is dropped + +It existed to observe daemons running code that predates the verdict. Every daemon that has +produced the field data above already ships the verdict, and daemons older than #18043 age out +with the next restart or reboot. A signed helper, PTY-launched probe, lease registry, and +second daemon client would buy coverage of a population that is already shrinking to zero. + +## 5. Action items + +1. Ship the design above in one PR against `main`. Owner: engineering. +2. Send G1 to the affected user (draft below). Owner: Jinwoo. Changes copy only. +3. After one release: read `daemon_folder_access_notice` against `daemon_pty_cwd_denied` to + confirm the toast reaches the denied population, and watch whether the 31% re-toast rate + holds. +4. Separately, reopen the prevention question with the field numbers: 2.8% of updaters losing + folder access to their terminals is not a two-report curiosity. The durable-relocation design + stays a separate doc. Draft for G1: @@ -144,7 +128,7 @@ Draft for G1: > allow the folder when macOS asks, run (1) once more. Which step made it work tells us where the > permission is breaking. Thanks! -## 5. Non-goals +## 6. Non-goals Automatic restart, continuous polling, parsing arbitrary terminal output, Python or shell-command fallbacks, failed-admission and restore triggers (the verdict exists only on a completed spawn; diff --git a/docs/plans/validation/README.md b/docs/plans/validation/README.md index 6cd11901c7d..ae6d2f59dd3 100644 --- a/docs/plans/validation/README.md +++ b/docs/plans/validation/README.md @@ -168,3 +168,36 @@ away and never appears. - Whether the daemon's in-process `opendir` and its shells' reads diverge when the lineage is broken (gate G1 in the design). - Whether restart alone recovers, or restart plus `tccutil reset` and re-allow is required (G1). + +--- + +# PostHog field data (queried 2026-09-21, 21-day window) + +`daemon_pty_cwd_denied` fires only on proven divergence: daemon `accessSync` denied, app +`accessSync` succeeded, macOS only (#18043, shipped 2026-09-01). + +| cwd_class | app_version_match | events | users | +| --- | --- | ---: | ---: | +| documents | different | 5,674 | 863 | +| desktop | different | 2,379 | 404 | +| outside-home | different | 480 | 58 | +| downloads | different | 369 | 79 | +| documents | same | 113 | 33 | +| desktop | same | 113 | 25 | +| other-home | different | 25 | 8 | + +- Distinct denied users: 1,438 (1,374 in documents/desktop/downloads). Users who adopted a + different-version daemon in the window: 50,876. Denial rate among them: 2.8%. +- Persistence: 910 users denied on one day, 413 on 2–3 days, 115 on 4 or more days. +- Daily denied users ramped from single digits on 2026-09-07 to about 300/day by 2026-09-16 as + the telemetry release rolled out, then about 110–135/day over the weekend. +- Remedy signal: among denied users with a later same-version `daemon_adopted` (a daemon forked + by the current app), 151 had no further denial and 68 were denied again (about 69% / 31%). + 1,219 had no same-version adoption inside the window. +- `daemon_adopted` with `tcc_attribution=severed` (the existing notice's trigger): 45 users. The + cwd-denial population is about 30 times larger. + +Consequence: the daemon's existing `accessSync` verdict does observe the failure in the field. +The grant-less launchd probe (section A above) shows a second TCC mode on `~/Documents` where +`access()` passes and `opendir` fails; the two modes are different TCC states, and enumeration +covers both.