mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 00:02:31 +00:00
docs(macos): fold PostHog field data into the STA-7948 plan and drop the helper probe
daemon_pty_cwd_denied fired for 1,438 users in 21 days (2.8% of users adopting a different-version daemon, 96% on Documents/Desktop/Downloads), so the daemon's existing verdict already observes the failure and the helper probe for old daemons has no population left to cover. A fresh same-version daemon ended the denial for about 69% of observable users; the notice copy now tells the rest to re-allow the folder.
This commit is contained in:
Jinwoo-H
@@ -1,138 +1,122 @@
|
||||
# STA-7948: macOS daemon folder-access mismatch — finalized plan
|
||||
|
||||
Status: finalized 2026-09-21 after live evidence on a production adopted daemon. Supersedes the
|
||||
2026-09-14 helper-first design (kept below as Layer B, now evidence-gated). No production code yet.
|
||||
Evidence: [validation/README.md](validation/README.md) (section "2026-09-21 live evidence").
|
||||
Status: finalized 2026-09-21 after live evidence on a production adopted daemon and PostHog field
|
||||
data. Supersedes the 2026-09-14 helper-first design. No production code yet.
|
||||
Evidence: [validation/README.md](validation/README.md) (sections "2026-09-21 live evidence" and
|
||||
"PostHog field data").
|
||||
|
||||
## 1. What is settled
|
||||
|
||||
1. **macOS TCC lets `access(2)` and `stat` succeed on Documents/Desktop/Downloads while
|
||||
`opendir`/`readdir` fails with EPERM.** Measured in a grant-less launchd process on
|
||||
`~/Documents`: `access(R_OK|X_OK)=True`, `stat=ok`, `scandir=EPERM`. Full Disk Access folders
|
||||
behave differently (`access()` is denied too). Consequence: the daemon's existing
|
||||
`cwdReadableByDaemon` verdict (`accessSync` in `terminal-host-session-create.ts`) is **blind to
|
||||
the incident's folder class**, and the `daemon_pty_cwd_denied` telemetry has a structural zero
|
||||
for it. The original report's `ls -lde .` success is consistent with this (it only stats).
|
||||
2. **Daemon-spawned shells carry the daemon's effective grant.** On the production adopted daemon
|
||||
(spawned by 1.4.207-adhoc, app now 1.4.206), the daemon's verdict, a login-wrapped shell's
|
||||
`scandir`, and a control shell agreed on `~/Documents`, `/tmp`, and Full-Disk-Access-gated
|
||||
`~/Library/Safari`. `responsibility_get_pid_responsible_for_pid` reports the daemon and each
|
||||
shell as its own responsible pid, never Orca main, yet the shells read Safari, which only
|
||||
`com.stablyai.orca` holds. `com.stablyai.orca.helper` has no TCC row of its own.
|
||||
3. **The login wrapper is unconditional in production**: 1734 of 1734 recorded spawns were
|
||||
`wrapped`.
|
||||
4. **The incident itself has never been reproduced** (2026-09-01 matrix, 2026-09-21 run). Field
|
||||
recovery for the one confirmed case reportedly required daemon restart **plus**
|
||||
`tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca` and re-allowing.
|
||||
1. **The failure is real, common, and tied to app updates.** PostHog `daemon_pty_cwd_denied`
|
||||
(shipped in #18043, 2026-09-01) fires only when the daemon reports the cwd unreadable and the
|
||||
app can read it. In the 21 days to 2026-09-21 it fired for 1,438 users (5,674 events on
|
||||
Documents, 2,379 on Desktop, 369 on Downloads). 1,374 of those users were in a protected folder
|
||||
class, and 96% carried `app_version_match=different`, i.e. an adopted daemon forked by an
|
||||
older app build. That is 2.8% of the 50,876 users who adopted a different-version daemon.
|
||||
Denials recur: 528 users hit it on two or more days.
|
||||
2. **The daemon's existing verdict detects it.** The field data above is produced by the
|
||||
current `accessSync(R_OK|X_OK)` check in `terminal-host-session-create.ts`, so no new daemon
|
||||
code is required to observe the failure on daemons that already ship that field. A separate
|
||||
grant-less launchd probe on this machine showed a different TCC mode on `~/Documents`
|
||||
(`access()` passes, `opendir` fails), so enumeration is the more faithful check and Layer A
|
||||
switches to it, but detection does not wait on that.
|
||||
3. **A fresh daemon from the current app fixes most cases, not all.** Among denied users whose
|
||||
later history is visible, 151 stopped being denied after a same-version daemon appeared and
|
||||
68 were denied again even with a same-version daemon (about 69% versus 31%). The 31% need the
|
||||
folder grant itself repaired (`tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca`
|
||||
and re-allow), which matches the one hands-on recovery on record.
|
||||
4. **Daemon-spawned shells carry the daemon's effective grant.** On the production adopted
|
||||
daemon here (healthy), the daemon's verdict, a login-wrapped shell's `scandir`, and a control
|
||||
shell agreed on `~/Documents`, `/tmp`, and Full-Disk-Access-gated `~/Library/Safari`. The
|
||||
shells report themselves as their own responsible pid yet read Safari, which only
|
||||
`com.stablyai.orca` holds; `com.stablyai.orca.helper` has no TCC row. The login wrapper is
|
||||
unconditional in production (1,734 of 1,734 spawns `wrapped`).
|
||||
5. **The user-visible symptom** (Slack, 2026-09-20): inside a workspace under `~/Documents`,
|
||||
`ls -lde .` works (stat only), `os.scandir('.')` and opening a file both fail with EPERM, and
|
||||
Codex starts then dies with "Operation not permitted" reading its cwd. Commands that do not
|
||||
touch the folder work. macOS 26.5.1.
|
||||
|
||||
## 2. Design
|
||||
|
||||
Two layers. Layer A ships first and is small. Layer B is the previous helper design, built only
|
||||
if evidence says the population it covers matters.
|
||||
One layer. The previous helper probe is dropped (section 4).
|
||||
|
||||
### Layer A — enumeration verdict in the daemon, notice in the app (ship now)
|
||||
|
||||
**Daemon.** Replace `isCwdReadableByThisProcess` in
|
||||
`src/main/daemon/terminal-host-session-create.ts` with real enumeration: `fs.opendirSync(cwd)`,
|
||||
one `dir.readSync()`, `dir.closeSync()`. `EPERM`/`EACCES` → `false`. `ENOENT`/`ENOTDIR`/anything
|
||||
else → `true`, exactly as today, so a non-permission failure never masquerades as denial. Keep the
|
||||
wire field name and type (`cwdReadableByDaemon?: boolean`); its meaning becomes "enumerable by
|
||||
the daemon". Old clients read it unchanged. This runs where `accessSync` already runs (before the
|
||||
fork, synchronous, local paths only, skipped for WSL).
|
||||
**Daemon (hardening, not a prerequisite).** Replace `isCwdReadableByThisProcess` with real
|
||||
enumeration: `fs.opendirSync(cwd)`, one `dir.readSync()`, `dir.closeSync()`. `EPERM`/`EACCES` →
|
||||
`false`; `ENOENT`/`ENOTDIR`/anything else → `true`, exactly as today. Keep the wire field name and
|
||||
type (`cwdReadableByDaemon?: boolean`). Runs where `accessSync` already runs.
|
||||
|
||||
**Main: evidence.** In `src/main/daemon/daemon-pty-session-spawn.ts`, next to the existing
|
||||
`trackDaemonPtyCwdDeniedIfDiverged` call, record proven divergence in a new
|
||||
`src/main/daemon/daemon-folder-access-mismatch.ts`: `{ canonicalPath, daemonIdentity: { pid,
|
||||
startedAtMs, launchNonce }, appSessionId, observedAtMs }`. The app-side comparison uses the same
|
||||
`opendir`/one-`readdir`/`closedir` sequence (switch `trackDaemonPtyCwdDeniedIfDiverged` from
|
||||
`accessSync` to it as well). Keep at most one entry per daemon identity; a later spawn that
|
||||
enumerates successfully on that identity clears it. `canonicalPath` is the cwd actually sent to the
|
||||
daemon; no `realpath`, no case folding. Local current-protocol adapter only; SSH, WSL, relay,
|
||||
legacy adapters, and local fallback providers never write evidence.
|
||||
`src/main/daemon/daemon-folder-access-mismatch.ts`: `{ canonicalPath, cwdClass, daemonIdentity:
|
||||
{ pid, startedAtMs, launchNonce }, appSessionId, observedAtMs }`. The app-side comparison uses
|
||||
the same `opendir`/one-`readdir`/`closedir` sequence (switch the telemetry emitter to it too).
|
||||
Keep at most one entry per daemon identity; a later spawn that succeeds on that identity clears
|
||||
it. `canonicalPath` is the cwd sent to the daemon; no `realpath`, no case folding. Local
|
||||
current-protocol adapter only. Daemons that omit the field (pre-#18043) produce no evidence.
|
||||
|
||||
**Main: IPC.** Extend the existing `pty:management:macTccAttribution` handler in
|
||||
`src/main/ipc/pty-management.ts` to return
|
||||
`{ health, folderAccessMismatch: { daemonScope: string } | null }` where `daemonScope` is a
|
||||
stable hash of the daemon identity. Mirror the type in `src/preload/api/pty-management-api.ts`
|
||||
and return `null` from `src/renderer/src/web/preload-api/web-terminal-api.ts`. No new channel,
|
||||
no push event.
|
||||
**Main: IPC.** Extend the existing `pty:management:macTccAttribution` handler to return
|
||||
`{ health, folderAccessMismatch: { daemonScope: string, cwdClass } | null }`. Mirror the type in
|
||||
`src/preload/api/pty-management-api.ts`; the web fallback returns `null`. No new channel.
|
||||
|
||||
**Renderer.** In `src/renderer/src/hooks/useMacTccAttributionSeveredNotice.ts` add a second toast
|
||||
(`mac-daemon-folder-access-mismatch`) driven by the same focus-time poll, latched per
|
||||
`daemonScope` per app session, dismissable per scope. Copy (final wording depends on gate G1):
|
||||
**Renderer.** In `useMacTccAttributionSeveredNotice.ts` add a second toast
|
||||
(`mac-daemon-folder-access-mismatch`) on the same focus-time poll, latched per `daemonScope` per
|
||||
app session, dismissable per scope. Copy:
|
||||
|
||||
> **Orca's terminal service can't read a folder Orca can.**
|
||||
> The terminal service was denied access to a workspace folder that Orca itself can read.
|
||||
> Restart the daemon from Manage Sessions. If macOS asks again, allow the folder. This closes all
|
||||
> running Orca terminals and agents.
|
||||
> **Orca's terminal service can't read your Documents folder.**
|
||||
> Terminals opened in this folder will fail with "Operation not permitted" even though Orca
|
||||
> itself can read it. Restart the daemon from Manage Sessions; this closes all running Orca
|
||||
> terminals and agents. If it still fails afterwards, re-allow the folder for Orca in System
|
||||
> Settings → Privacy & Security → Files and Folders.
|
||||
|
||||
Action: the existing **Open Manage Sessions** target and the existing restart confirmation in
|
||||
`useDaemonActions`. Keep the severed-attribution toast separate.
|
||||
Substitute Desktop/Downloads from `cwdClass`. Action: the existing **Open Manage Sessions** target
|
||||
and the existing restart confirmation in `useDaemonActions`. Keep the severed-attribution toast
|
||||
separate.
|
||||
|
||||
**Recovery and clearing.** Use `restartDaemon()` unchanged. Evidence is keyed by daemon identity
|
||||
and a restart always replaces the identity, so the next poll returns `null` and the hook dismisses
|
||||
the toast. If the replacement daemon is also denied, the next spawn re-records and re-toasts. No
|
||||
post-restart probe: the next terminal spawn is the probe.
|
||||
**Recovery and clearing.** Use `restartDaemon()` unchanged. Evidence is keyed by daemon
|
||||
identity; a restart replaces the identity, so the next poll returns `null` and the hook dismisses
|
||||
the toast. If the replacement daemon is also denied (the 31% case), the next spawn re-records and
|
||||
re-toasts, now with the re-allow sentence doing the work. No post-restart probe.
|
||||
|
||||
**Telemetry.** `daemon_pty_cwd_denied` becomes meaningful for the Documents class. Add
|
||||
`daemon_folder_access_notice` with `{ action: shown | dismissed | restart_clicked }` and the
|
||||
existing `cwd_class` enum; no paths.
|
||||
**Telemetry.** Add `daemon_folder_access_notice` with `{ action: shown | dismissed |
|
||||
restart_clicked, cwd_class }`. Keep `daemon_pty_cwd_denied` as the denominator.
|
||||
|
||||
**Tests.** Extend `terminal-host-cwd-readability.test.ts` for opendir mapping (EPERM, EACCES,
|
||||
ENOENT, ENOTDIR, readable, empty directory). Unit-test the evidence store (records only on
|
||||
divergence, one per identity, clears on success or identity change). IPC handler shape. Hook:
|
||||
toast once per scope, dismissal latch, clears when the poll returns `null`, no toast when both
|
||||
sides fail or the path is missing. No Electron perf gate is needed: nothing new runs on the
|
||||
spawn path beyond one `opendir`.
|
||||
**Tests.** `terminal-host-cwd-readability.test.ts` for opendir mapping (EPERM, EACCES, ENOENT,
|
||||
ENOTDIR, readable, empty). Evidence store: records only on divergence, one per identity, clears
|
||||
on success or identity change, ignores daemons without the field. IPC shape. Hook: toast once per
|
||||
scope, dismissal latch, clears when the poll returns `null`, no toast when both sides fail or the
|
||||
path is missing. No Electron perf gate: nothing new runs on the spawn path beyond one `opendir`.
|
||||
|
||||
**Accepted gap.** Daemons already running pre-change code (every adopted daemon deployed today)
|
||||
never produce the enumeration verdict. They are covered after their next restart or reboot, so
|
||||
each user is blind for at most one update cycle. Layer B exists for that population.
|
||||
## 3. Open gate
|
||||
|
||||
### Layer B — helper probe for already-deployed old daemons (evidence-gated, unchanged design)
|
||||
|
||||
The 2026-09-14 design (signed helper launched through `createOrAttach` with `shellOverride`,
|
||||
neutral cwd, nonce-framed single record, direct and login-wrapped strategies, lease and cleanup
|
||||
contract, one active plus one queued per daemon, Electron A/B performance gate) is the only path
|
||||
that observes an old daemon's view without new daemon code. Build it only if, after Layer A ships:
|
||||
|
||||
- PostHog `daemon_adopted` shows the adopted-old-daemon population is large enough that a
|
||||
one-cycle blind spot matters, or
|
||||
- gate G1 shows the daemon's in-process verdict disagrees with its shells (the daemon enumerates
|
||||
but the terminal is denied), which would mean Layer A's oracle is wrong for the broken state.
|
||||
|
||||
If Layer B is built, do not route it through the plan's createOrAttach-plus-lease machinery for
|
||||
new daemons; new daemons get a single capability-gated `directoryAccessProbe` request built on the
|
||||
existing `ptySpawnHealth` pattern (`runPtySpawnHealthProbe` in
|
||||
`src/main/daemon/pty-subprocess/spawn-preflight.ts`) with cleanup on `onClientDisconnected`.
|
||||
|
||||
## 3. Open gates
|
||||
|
||||
- **G1 — affected machine, remedy and oracle.** On a machine showing the failure, inside the
|
||||
affected Orca terminal:
|
||||
- **G1 — one affected machine, oracle check.** Field data proves the daemon is denied; it does
|
||||
not prove the daemon's verdict and the terminal's experience always agree. The one hands-on
|
||||
case (Slack) agrees. Ask the affected user, inside the affected terminal:
|
||||
1. `python3 -c "import os; list(os.scandir(os.path.expanduser('~/Documents')))"` → expect EPERM.
|
||||
2. Settings → Terminal → Manage Sessions → Restart daemon. Open a new terminal, rerun step 1.
|
||||
3. If still denied: `tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca`, re-allow
|
||||
when prompted, rerun step 1.
|
||||
Decides whether the notice says "restart" or "restart, then re-allow", and whether the daemon
|
||||
identity is what breaks (step 2 fixes it) or the app's grant (step 3 fixes it).
|
||||
- **G2 — population.** PostHog: count of `daemon_adopted` per week, split by
|
||||
`app_version_match`. Ignore `daemon_pty_cwd_denied` for now: it is structurally zero for the
|
||||
Documents class until Layer A ships.
|
||||
- **G3 — Documents prompt behaviour on a signed build.** Confirm the daemon's `opendir` on a
|
||||
never-granted Documents folder does not raise a consent prompt attributed to the helper bundle.
|
||||
On this machine the grant-less probe was denied silently; verify once on a signed build with a
|
||||
fresh TCC profile.
|
||||
2. Settings → Terminal → Manage Sessions → Restart daemon. New terminal, rerun step 1.
|
||||
3. If still denied: `tccutil reset SystemPolicyDocumentsFolder com.stablyai.orca`, re-allow when
|
||||
prompted, rerun step 1.
|
||||
This confirms the copy, not the code.
|
||||
|
||||
## 4. Action items
|
||||
Closed: G2 (population, above). G3 (prompt behaviour of the daemon's `opendir` on a signed build)
|
||||
is folded into the Layer A PR's manual check.
|
||||
|
||||
1. Send G1 to the affected user (draft below). Owner: Jinwoo.
|
||||
2. Run G2 in PostHog. Owner: whoever holds PostHog access.
|
||||
3. Land this doc and the 2026-09-21 evidence on PR #21740; keep the durable-relocation design
|
||||
separate and unblocked.
|
||||
4. Implement Layer A in one PR against `main` (daemon verdict, evidence store, IPC field, toast,
|
||||
tests). Do not wait on G1 for the code; G1 only changes the copy.
|
||||
5. Decide Layer B after two release cycles of Layer A telemetry.
|
||||
## 4. Why the helper probe is dropped
|
||||
|
||||
It existed to observe daemons running code that predates the verdict. Every daemon that has
|
||||
produced the field data above already ships the verdict, and daemons older than #18043 age out
|
||||
with the next restart or reboot. A signed helper, PTY-launched probe, lease registry, and
|
||||
second daemon client would buy coverage of a population that is already shrinking to zero.
|
||||
|
||||
## 5. Action items
|
||||
|
||||
1. Ship the design above in one PR against `main`. Owner: engineering.
|
||||
2. Send G1 to the affected user (draft below). Owner: Jinwoo. Changes copy only.
|
||||
3. After one release: read `daemon_folder_access_notice` against `daemon_pty_cwd_denied` to
|
||||
confirm the toast reaches the denied population, and watch whether the 31% re-toast rate
|
||||
holds.
|
||||
4. Separately, reopen the prevention question with the field numbers: 2.8% of updaters losing
|
||||
folder access to their terminals is not a two-report curiosity. The durable-relocation design
|
||||
stays a separate doc.
|
||||
|
||||
Draft for G1:
|
||||
|
||||
@@ -144,7 +128,7 @@ Draft for G1:
|
||||
> allow the folder when macOS asks, run (1) once more. Which step made it work tells us where the
|
||||
> permission is breaking. Thanks!
|
||||
|
||||
## 5. Non-goals
|
||||
## 6. Non-goals
|
||||
|
||||
Automatic restart, continuous polling, parsing arbitrary terminal output, Python or shell-command
|
||||
fallbacks, failed-admission and restore triggers (the verdict exists only on a completed spawn;
|
||||
|
||||
@@ -168,3 +168,36 @@ away and never appears.
|
||||
- Whether the daemon's in-process `opendir` and its shells' reads diverge when the lineage is
|
||||
broken (gate G1 in the design).
|
||||
- Whether restart alone recovers, or restart plus `tccutil reset` and re-allow is required (G1).
|
||||
|
||||
---
|
||||
|
||||
# PostHog field data (queried 2026-09-21, 21-day window)
|
||||
|
||||
`daemon_pty_cwd_denied` fires only on proven divergence: daemon `accessSync` denied, app
|
||||
`accessSync` succeeded, macOS only (#18043, shipped 2026-09-01).
|
||||
|
||||
| cwd_class | app_version_match | events | users |
|
||||
| --- | --- | ---: | ---: |
|
||||
| documents | different | 5,674 | 863 |
|
||||
| desktop | different | 2,379 | 404 |
|
||||
| outside-home | different | 480 | 58 |
|
||||
| downloads | different | 369 | 79 |
|
||||
| documents | same | 113 | 33 |
|
||||
| desktop | same | 113 | 25 |
|
||||
| other-home | different | 25 | 8 |
|
||||
|
||||
- Distinct denied users: 1,438 (1,374 in documents/desktop/downloads). Users who adopted a
|
||||
different-version daemon in the window: 50,876. Denial rate among them: 2.8%.
|
||||
- Persistence: 910 users denied on one day, 413 on 2–3 days, 115 on 4 or more days.
|
||||
- Daily denied users ramped from single digits on 2026-09-07 to about 300/day by 2026-09-16 as
|
||||
the telemetry release rolled out, then about 110–135/day over the weekend.
|
||||
- Remedy signal: among denied users with a later same-version `daemon_adopted` (a daemon forked
|
||||
by the current app), 151 had no further denial and 68 were denied again (about 69% / 31%).
|
||||
1,219 had no same-version adoption inside the window.
|
||||
- `daemon_adopted` with `tcc_attribution=severed` (the existing notice's trigger): 45 users. The
|
||||
cwd-denial population is about 30 times larger.
|
||||
|
||||
Consequence: the daemon's existing `accessSync` verdict does observe the failure in the field.
|
||||
The grant-less launchd probe (section A above) shows a second TCC mode on `~/Documents` where
|
||||
`access()` passes and `opendir` fails; the two modes are different TCC states, and enumeration
|
||||
covers both.
|
||||
|
||||
Reference in New Issue
Block a user