mirror of
https://github.com/stablyai/orca.git
synced 2026-09-21 16:02:20 +00:00
main
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Neil
|
5127d1eb3b |
refactor(windows): vendor the registry addon as @orca/windows-registry (#20438)
* refactor(windows): vendor the registry addon as @orca/windows-registry windows-native-registry@3.2.2 was last published in 2023 by a single maintainer. Orca called two of its exports, both read-only, so the whole dependency is replaced by a local N-API addon under native/. The vendored addon is read-only by construction: setValue, createKey and deleteKey are gone, so RegDeleteTreeW no longer ships in the app. Two upstream defects are also fixed rather than carried over — the name/data scratch buffers were file-scope statics that concurrent reads would scribble over, and createKey/deleteKey called .c_str() on a temporary. Build wiring keeps the existing shape: still an optionalDependency gated to win32, still excluded from pnpm's allowBuilds so only Orca's own Windows rebuild runs node-gyp for it, still copied into the packaged resources. The CI native caches now key on the vendored sources so an addon.cc edit cannot restore a stale .node. * test(windows): check the vendored registry addon against reg.exe The addon is vendored source, so no upstream release proves it still decodes values the way Orca's PATH readers expect. reg.exe is the only independent oracle on the box. * ci(windows): register the registry addon test on the Windows runner A Windows-gated file self-skips on ubuntu, so without both registrations it reports success while running on no machine at all. * fix(build): link the registry addon as a workspace package, not file: As a `file:` dependency pnpm re-resolved and re-linked the package on every install, including `--frozen-lockfile` (measured: "added 1" on a repeat no-op install). That virtual-store churn ran concurrently with node-gyp reading the same tree and cost @vscode/windows-process-tree its binding.gyp mid-rebuild, failing package (windows) whenever the native cache hit and only that module needed building. The linux packaging job hit the same race from the other side, as a pnpm staging move failure. A workspace link resolves once and leaves the store alone; repeat installs are now 55ms no-ops. native/windows-registry is listed explicitly so `packages:` still does not auto-discover mobile/. * fix(build): stop tracking node-gyp output for the vendored addon The build/ tree is generated per host and ABI; the committed copy was macOS-specific gyp scaffolding from a local build and would have shipped stale Makefiles to every checkout. * chore: ignore the vendored addon's node-gyp bin output too node-gyp also emits bin/<platform>-<abi>/ beside build/; both are per-host generated output that must never be committed. |
||
|
Neil
|
56fcb544e0 |
fix(browser): move cookie scoping off psl's stale suffix list (#20421)
* fix(browser): move cookie scoping off psl's stale suffix list psl@1.15.0 is its latest release and ships a Dec-2024 snapshot of the public suffix list. Measured against the current upstream list, it fails to recognise 600 of 10,030 suffixes; tldts misses 2. That gap is a cookie-isolation bug. psl does not know `api.br` is a suffix, so it falls back to the `br` rule and maps foo.api.br, bar.api.br and example.api.br all onto the single family `api.br`. Unrelated registrants then share a removal scope, and a replace-mode import for one clears the others' cookies. The same holds for seg.ar, co.az, gov.cz and ~597 more. tldts is called with allowPrivateDomains, without which the PSL's PRIVATE section is ignored and every *.github.io / *.s3.amazonaws.com / *.vercel.app tenant collapses into one family — 21 of 49 probed hosts changed family under the default. The new test pins that boundary. One deliberate behaviour change: hosts under `.local` (not in the PSL) were their own family under psl, which returned an all-null parse for them; they now resolve to the two-label boundary (app.orca.local -> orca.local), matching what Chromium treats as the registrable domain. * fix(build): bundle tldts into the main process like psl was psl sat in BUNDLED_MAIN_DEPENDENCIES, so it was inlined into the main bundle rather than externalized and copied into resources/node_modules. Swapping the dependency without moving that entry left a bare tldts import that afterPack's runtime-closure check rejects. * fix(build): point the output contract at tldts and drop the psl shim The contract test still asserted psl was in BUNDLED_MAIN_DEPENDENCIES, so it failed once the entry became tldts. src/types/psl.ts declared a module that no longer resolves; tldts ships its own types. * test(browser): pin the suffix boundaries the tldts swap moved Three semantic changes shipped untested: - `.local` is unlisted, and the libraries disagreed on what that means. psl returned an all-null parse so every `*.orca.local` host was its own family; tldts stops at `orca.local`. The consequence is wider than the family name — importDomainAncestors now yields the shared parent, so a replace-mode import of one host clears non-host-only cookies every sibling shares. - psl's snapshot had `compute.amazonaws.com` as a literal PRIVATE suffix; the current list only carries the wildcard, so the bare host is ICANN now. - The renderer's `psl.isValid` gate had no direct test at all — nothing imported the module from a test. Also drops comments that explained a boundary in terms of psl's internals. One was wrong under tldts: bracketed IPv6 does not reach an error branch, it parses with the brackets stripped and falls through the unlisted path. |
||
 Jinjing
|
9fed61e5c2 |
Persist agents sidebar search visibility as pairing-local preference (#19313)
* Persist agents sidebar search field visibility as pairing-local preferen - Add `agentsShowSearch` to workspace UI state with default on - Include in pairing-local fields so preference syncs across clients - Convert search from menu action to checkbox menu item for explicit toggle - Update activity thread options menu to reflect checkbox state - Add localization strings across all supported languages - Update RPC schemas and preference persistence layer - Includes readiness validation reports confirming feature is clean * rm review * fix documentation |
||
|
Neil
|
314506003a |
fix: retain MSYS shell descendants in their terminal job (#19068)
* fix: retain MSYS shell descendants in their terminal job * test: complete MSYS regression CI registration and teardown contract * fix(windows): deny job breakaway for the whole Cygwin/MSYS shell family The per-PTY job probed only msys-2.0.dll, and only for bash.exe/sh.exe. Cygwin ships the same spawn.cc breakaway logic under cygwin1.dll, and an MSYS2 zsh escapes exactly like its bash does, so both kept the orphan bug. Probe the runtime DLL on the shell's own search path instead of matching shell names: that is the property that decides whether the runtime will ask for CREATE_BREAKAWAY_FROM_JOB, and it drops the name special-casing. * chore(patch): restore the conpty.cc index line The earlier hand-edit dropped it while every sibling section kept one. Recomputed against the real blobs: applying this patch to 7b286d3d yields exactly 4b06d185, so git apply -3 has its fallback back. |
||
  Brennan BensonandMerge Sim
|
1478101342 |
fix(windows): unblock structured native chat by exposing process creation time (#18986)
* fix(windows): guard process creation times
* fix(windows): ask the relay's bare addon for creation times too
The relay addon build now emits creationTimeMs, but the runtime binding
for the bare addon still declared only CommandLine, so a Windows relay
host requested flag 2 and every row came back without a creation time.
That leaves captureWindowsDescendantSnapshot returning null and
verifyWindowsProcessIdentity false forever on those hosts -- the relay
half of the patch was unreachable.
Naming CreationTime in the adapter is safe because the bare addon is a
content-hashed relay artifact: it ships in the same immutable relay
directory as the bundle reading it, so it can never be older than the
code asking for the bit.
Also bound the win32 guard test on our own row, which the addon can
never fail to answer, so an unconverted FILETIME or a 1601-epoch stamp
fails instead of satisfying a bare count.
* fix(windows): make the compiled addon prove its own CreationTime support
CI caught the real defect: the win32 guard test read
isWindowsProcessStartTimeAvailable() as true and then found 0 rows
carrying creationTimeMs. Unlike node-pty, this package publishes a
prebuilt .node at the same build/Release path node-gyp writes to, so
pnpm patches the source tree and leaves that binary alone. A host then
holds a patched lib/index.js -- ProcessDataFlag.CreationTime and all --
over a binary that ignores flag 4, and neither a load check nor a path
check can see the difference.
So the binary now says so itself: addon.cc exports
supportedProcessDataFlags, lib/index.js re-exports it, and
- windows-process-tree-creation-time.cjs asserts it during install,
which is what forces a from-source rebuild. It is shared by the Node
probe in ensure-native-runtime.mjs and the Electron probe in
rebuild-native-deps.mjs, exactly as node-pty-job-ownership.cjs is --
the Electron half matters because that probe decides onlyModules, so
without it the packaged app would ship the stale prebuilt.
- isWindowsProcessStartTimeAvailable() gates on the reported bit, not
the enum. Believing the enum is worse than reporting false: the
descendant snapshot returns null forever and the exit proof latches
unverifiable while structured chat believes it has a reaper.
rebuildNodeRuntimeModules could not actually have rebuilt this package:
the patched binding.gyp includes deps/node-addon-api, which the tarball
does not ship, and node-gyp must run from the physical dir.
Also closes the relay repair path's divergence: repairCreationTimeSources
wrote the C++ but not the buildNode splat or the tree-node typing, and
assertPatchApplied checked neither, so a repaired tree passed as patched
with buildProcessTree silently dropping the field.
The guard test is unchanged.
* fix(windows): keep the process-tree patch LF-only
windows-process-tree-patch-contract.test.mjs requires the patch file to
carry no CR bytes. Regenerating through pnpm patch-commit emitted 199 of
them, because the creation-time change is the first to touch files the
package ships as CRLF (src/process.h, src/process_worker.cc,
src/addon.cc, lib/index.js, lib/index.ts, the typings) -- and #17886's
own hunks over binding.gyp and src/process_commandline.cc carry the rest.
Stripping them is safe and changes nothing the lockfile records: pnpm
hashes patches CRLF-normalized, so the digest stays
e66202cc623996d02040c93449eb9ae353fddadf426cb53202a59ee710ee6fe7 and now
equals the file's plain sha256 too. It also still applies -- verified
against a deleted store entry, not a warm one -- and the precedent was
already there: the previous patch was LF-only and had been patching
those same CRLF files all along.
ensure-native-runtime.test.mjs stages the siblings the script loads at
module scope into its temp project. The import walk added by #17886 sees
`from './x.mjs'` only, so the createRequire'd .cjs siblings still have to
be named, and this PR adds a second one.
---------
Co-authored-by: Merge Sim <sim@local>
|
||
  OrcaWinandOrca Worker
|
fc5fa16870 |
perf(windows): split the process table into two flag sets (#17866)
* perf(windows): split the process table into two flag sets
MDE flags "suspicious memory activity" on the process-table reader: it
opened a handle into every process on the box and read each one's PEB on
a repeating cadence. Two changes narrow that.
Drop `Memory` outright. It cost a second
OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ) plus
GetProcessMemoryInfo per process, and nothing reads a working set off
this table -- the Resource Manager runs its own sweep, and the addon
stores WorkingSetSize into a DWORD so anything above 4 GB wraps.
Split the rest in two. `readWindowsProcessIdentityTable[Fresh]` is a
bare Toolhelp32 walk with zero per-process handles, and returns
`WindowsProcessIdentityRow`, which has no `command` to read.
`readWindowsProcessTable[Fresh]` keeps the command line for the callers
that match on it. PTY root identity and the owner start-time probe move
to the cheap reader; agent recognition, port attribution, codex turn
processes and structured-TUI matching all genuinely need the command
line and stay.
Two independently single-flighted caches, never one per caller: the
fan-out this module prevents is one scan per caller, and each reader
still serves every caller wanting its flag set. The wedge gate and the
3s deadline stay shared, because both readers call the same addon and
one wedged read latches its one `requestInProgress`. With no binding
there is only the 1.4s PowerShell scan to run, so the identity view
rides the detailed snapshot rather than forking a second one.
Measured on Windows 11, 492 processes (p50/p95): identity 6.3/7.0 ms,
detailed 12.3/13.4 ms, previous memory+commandLine 13.1/14.1 ms.
* fix(windows): serialize native process-table reads across flag sets
The two flag-set readers could both be in flight at once, and the
vendored wrapper does not tolerate that. `getRawProcessList` pushes the
callback onto one list and calls the addon only when no request is in
progress, so a second concurrent caller's `flags` are DISCARDED and it
is handed the first caller's rows. Measured against the real addon:
identity issued first, both callers got the same array, 0 of 541 rows
with a command line. A detailed read overlapping an identity read
therefore returned a table with every command line empty, which agent
recognition reads as "no agent" -- silently, and only under concurrency.
Nothing already here excluded that. Each snapshot cache single-flights
only within itself, and the wedge set latches only after a read misses
its 3s deadline, so through the healthy ~12ms of a scan neither reader
excluded the other. Overlap is the normal state: panes poll detailed at
750ms while a teardown takes identity snapshots.
`nativeReadGate` admits one native read at a time across both flag sets.
It also fixes the relay path, where `adaptAddon` has no queue at all and
two simultaneous CreateToolhelp32Snapshot calls are the crash the
vendor's queue exists to prevent. Every link settles, so a wedged read
never strands a waiter; the waiter re-checks the wedge and rejects. With
one call outstanding, retention stays bounded at one callback rather
than one per reader.
Also from review:
- The CIM fallback now belongs to the detailed flag set alone, and the
identity view projects that snapshot through `toIdentityRow`, so an
identity row carries no command line on a no-binding host either.
- The concurrency test modelled the wrapper's coalescing queue, which
the previous synchronous mock could not express; verified failing
without the gate and passing with it.
- `agent-session-process-identity-probe` early-returns when the
creation-time flag is unavailable, which no shipped addon build
provides, instead of scanning the table to produce null.
- Corrected the cost framing: Memory took an OpenProcess(...|VM_READ)
it never read through, so dropping it halves per-process handle opens
and leaves the PEB/ReadProcessMemory telemetry unchanged.
* test(windows): keep read exclusion across resets and flag each field
Two review follow-ups, both about tests passing for the wrong reason.
`resetNativeReaderState` replaced the read gate with a resolved promise,
so waiters still holding the old chain ran beside reads queued on the
new one. Reachable only from the `__set*ForTests` hooks, which is what
makes it worth fixing: it hands a suite two concurrent calls into its
own mock addon -- the exact condition the concurrency tests exist to
detect. Chain onto the gate instead; every link settles within the
deadline, so the bounded wait that costs is the right trade.
The coalescing mock shaped every field off the CommandLine bit, so an
identity read that did request CreationTime got `creationTimeMs`
stripped. The identity-side assertion was then only `!('command' in
row)`, which a correctly flagged read and a coalesced one satisfy
equally: a future regression losing identity flags under concurrency
would have kept the case green. Gate each field on its own bit and
assert `creationTimeMs` positively, inside the helper both orderings
share.
Concurrency assertions move to a new bare-addon mock. The coalescing
mock's own latch means it can never report more than one call in
flight, so measuring exclusion there proved nothing; the bare addon has
no queue -- like `adaptAddon` on a relay, where re-entering
CreateToolhelp32Snapshot is a real crash -- and makes re-entry visible.
Verified by deletion: restoring `nativeReadGate = Promise.resolve()`
fails the reset case with `expected 2 to be 1`, and restoring the
single-bit mock fails both overlap orderings on `creationTimeMs`.
* docs(windows): count the third test defect in the list that names them
The section opened "Two defects have now shipped", numbered two, then
described the third in its closing paragraph -- a list that reads as a
complete account while quietly omitting one, which is the exact failure
the section exists to warn about. Say three and number it, and note that
the third arrived inside the fix for the first two.
Also record why the creationTimeMs and flags-array assertions are not
redundant, in the doc and beside the assertions: the flags array catches
a read served another flag set's rows, the positional creationTimeMs
check catches field shaping (identity dropping CreationTime, or
toIdentityRow not forwarding it). Neither sees the other's failure.
* docs(windows): stop describing a PEB read this release removed
Every comment here that justified the flag split in terms of PEB reads became
false when the command-line reader moved to the kernel. Left alone, the
enumeration doc contradicted itself inside one file: the flag-set section
described three chained `ReadProcessMemory` calls per process while the
sections below it explained that the addon contains no such primitive and has
no PEB fallback.
The measurement is now attributed rather than merged. Dropping `Memory` halved
the per-process handle opens and nothing else -- both handles carried
`PROCESS_VM_READ` at the time -- and it was replacing the PEB walk that took
`PROCESS_VM_READ` and `ReadProcessMemory` out of the addon. Neither change
substitutes for the other, which is worth keeping straight: the split's
remaining value is the handle itself, not the memory access.
Also adds `relay/windows-port-scan.ts` to the caller table, the one caller this
effort introduced, and records that it reads only pid/name through the detailed
reader -- free while a pane is polling, not free on a headless relay.
* test(windows): pin the fresh links path against the identity TTL cache
The identity and detailed tables are separate snapshot readers with
independent TTLs, so the detailed path's existing freshness guard says
nothing about the ancestry walk's. Cover the identity reader on its own.
---------
Co-authored-by: Orca Worker <orca-worker@localhost>
|
||
|
|
bfc6a262a7 |
fix(windows): read command lines from the kernel, not each process's PEB (#17886)
* fix(windows): read command lines from the kernel, not each process's PEB MDE incident D scored Orca for suspicious memory activity: the vendored `@vscode/windows-process-tree` recovered every process's command line by opening it with `PROCESS_QUERY_INFORMATION | PROCESS_VM_READ` and chaining three `ReadProcessMemory` calls through the PEB and `RTL_USER_PROCESS_PARAMETERS`. On a 750ms/2s cadence over the whole table that is the credential-dumping primitive, whatever the intent. Windows 8.1 added `NtQueryInformationProcess`'s `ProcessCommandLineInformation` class (60), which returns the same string as a kernel-built `UNICODE_STRING` under `PROCESS_QUERY_LIMITED_INFORMATION` alone. Electron's floor is Windows 10, so every supported OS has it. The PEB reader stays behind a process-wide latch that only `STATUS_INVALID_INFO_CLASS`/`NOT_SUPPORTED`/`NOT_IMPLEMENTED` can set; a pid that merely denied a handle does not re-arm it, because `PROCESS_QUERY_INFORMATION` implicitly grants the limited right and so cannot be obtained where the weaker open already failed. The same hunk drops `PROCESS_VM_READ` from `GetProcessMemoryUsage` and `GetCpuUsage`, which acquired it and never read an address space. Measured on Windows 11 (514 processes), counted in-process by swapping the addon's import table entries for counting stubs, per CommandLine scan: `ReadProcessMemory` 1128 -> 0, desired access 0x0410 -> 0x1000, p50 12.7ms -> 9.3ms. Command lines were byte-identical on every process both readers recovered (376/376, 379/379 across runs), including a 24,068-character argv with quotes, non-ASCII and trailing whitespace, and a WOW64 target. Three processes that refused the old rights granted the new one; none went the other way. * chore(deps): refresh the windows-process-tree patch hash in the lockfile * fix(windows): drop the PEB fallback and detect the unpatched prebuilt Review of #17886 found three ways the reader could still perform, or silently resume, the primitive it exists to remove. The class-missing latch was a permanent, process-wide, one-way downgrade back to the PEB read, and any single target returning STATUS_INVALID_INFO_CLASS / NOT_SUPPORTED / NOT_IMPLEMENTED could trip it. On an EDR-hooked ntdll -- the entire premise of this change -- a hook that does not recognise class 60 would have restored PROCESS_VM_READ plus three ReadProcessMemory per pid per scan for the life of the process, unobservably, on precisely the machines this was written for. The fallback is deleted rather than guarded: GetProcessCommandLine now returns false and leaves the command line empty, which callers already handle, so the addon imports no ReadProcessMemory at all. That absence is what makes the property checkable on the artifact. The published 0.8.0 tarball ships a loadable prebuilt built from unpatched source; it is node-addon-api, so a bare require() accepts it, allowBuilds is false and CI installs with --ignore-scripts, and a rebuild that soft-exits on a Windows file lock leaves it in place. Source-text guards could never see it. windowsProcessTreeAddonReadsProcessMemory() checks the compiled binary instead, and is wired into the install check, the rebuild, and the relay build. The repair itself never worked: `git apply` run inside a work tree prefixes patch paths with the cwd-relative prefix, skips what does not match, and exits 0, so the branch always fell through to its own post-check throw. The package dir is always under the project root, while the fixture that covered it was in %TEMP%, outside any repo. Blinding git with GIT_DIR fixes it, and the test now runs inside a real work tree. Also from review: bounds-check the returned UNICODE_STRING against the allocation (not the size the second query clobbers) and cap the probe so a bogus length cannot bad_alloc a whole scan; test NT_SUCCESS explicitly; value- initialize ProcessInfo, which left `memory` as stack garbage -- measured, 82 processes reported the same bogus working set; and correct a comment in windows-process-table.ts that still described the command line as a PEB read. Re-measured on Windows 11 (543 processes): ReadProcessMemory 1128 -> 0, with the symbol absent from the import table so the IAT hook finds no slot to count; desired access 0x0410 -> 0x1000 on all 543 opens; p50 13.5 -> 12.3ms; 405/405 command lines byte-identical including a 24,087-character quoted non-ASCII argv and a WOW64 target; 3 processes recovered only by the new path, 0 only by the old. * chore(deps): refresh the windows-process-tree patch hash in the lockfile * test(scripts): stage a script's local imports into the native-runtime fixture ensure-native-runtime.mjs gained an import of windows-process-tree-gyp-rebuild.mjs, but the fixture copied only the script itself, so every case in the suite died with ERR_MODULE_NOT_FOUND before reaching its own assertions. copyScriptWithLocalModules already walks a script's co-located imports for exactly this reason -- its own doc comment names this failure -- so use it rather than listing files by hand. The two Windows cases still fail here, on a missing node-pty ConPTY runtime that also fails on main; this only stops a resolution error from standing in front of whatever they were meant to catch. * fix(windows): route a locked stale addon to the Windows file-lock message `pnpm install` with Orca running aborted with a raw EPERM stack. The stale-binary guard -- which deletes an addon that still imports ReadProcessMemory so a skipped rebuild cannot use it -- ran outside the try whose catch classifies Windows file locks, and whose message is literally "Close running Orca/Electron/dev processes for this worktree": exactly this situation. Measured rather than assumed: rmSync against a loaded (memory-mapped) addon throws EPERM, and `force: true` does not help, since it only swallows ENOENT. Cold copies of the same file delete fine. So the delete threw a page before the handler that knows what it means. Moving the guard inside the try is the whole fix; the classifier already matches the EPERM text. The new case runs the real script against a temp project whose stale addon is held open by a live child process, and fails against the old placement with the raw `syscall: 'rm'` stack the report described. * feat(windows): warn once when command-line recovery is refused host-wide Removing the PEB fallback removed a total-defeat vector, but it left a cliff: if NtQueryInformationProcess(ProcessCommandLineInformation) is refused -- a hooked ntdll that does not know class 60 -- every command line comes back empty and agent identity matching silently degrades to image names. The addon still loads and still enumerates, so every health check the app has stays green. A cliff nobody can see is the failure mode this area keeps producing. The querying process is the unambiguous probe. A process can always open itself with PROCESS_QUERY_LIMITED_INFORMATION, so its own command line coming back empty means the query is refused for every process -- not that some target denied a handle, which is normal for roughly a quarter of the table. Keying on our own row rather than a fraction means no threshold to tune and no false positive on a hardened box where most processes deny. One warning per session, gated on the CommandLine flag actually being requested so a future identity-only reader cannot trip it. The suite's own SELF fixture gains a command line for the same reason: a self row without one is the alarm, not a detail. * fix(windows): check the relay's staged addon at load, and answer tri-state Two gaps in the ReadProcessMemory check, both about what it does not see. It only ever looked at node_modules/@vscode/windows-process-tree. A relay host has no node_modules of ours: it loads ./windows-process-tree.node staged beside the bundle. The relay build asserts the symbol on the artifact it produces, but a bundle and the addon beside it redeploy independently, so a host that has not taken a new bundle keeps whatever binary is already there -- and the published prebuilt is node-addon-api, so it binds cleanly and then walks every process's address space. loadWindowsProcessTree now checks that file too and refuses it, falling back to the CIM scan: slower, but not the thing an EDR quarantines a host for. The predicate is duplicated rather than imported, because the config-script copy is install-time tooling that drags in node-gyp and child_process, and this module is bundled into the app and the relay. And it returned false for a binary that is not there. All three callers happened to be safe, but the name read as a safety predicate, so a future caller would take a missing binary as verified. inspectWindowsProcessTreeAddon() now answers clean/unpatched/missing over an explicit binary path -- which is also what lets the relay's staged addon be checked at all -- and each caller states which state it acts on. Both are covered by cases that fail against the old code: without the load-time check the unpatched staged addon is bound and the CIM fallback never runs, and with 'missing' folded back into 'clean' the absence case fails outright. * test(windows): load the addon in beforeAll, not at collection time loadAddon() ran while the file was being collected, so on a Windows checkout with no built addon the require threw before any case existed and took the seven patch-text cases down with it -- cases that read only the patch file and need no binary at all. Verified both ways against a deliberately unresolvable addon path: at collection time vitest reports "no tests" for the file; from beforeAll the seven text cases pass and only the three addon cases go. * fix(deps): normalize the windows-process-tree patch to LF and let pnpm own its hash `pnpm install --frozen-lockfile` failed on this branch on every platform with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH, which breaks CI and the release build. Two coupled defects. The patch file was committed with CRLF -- 174 CR bytes, against zero on main -- and `.gitattributes` pins `/config/patches/*.patch -text` precisely so checkout cannot convert it, so those bytes reached every runner. And pnpm hashes a patch **LF-normalized**, so the raw sha256 of a CRLF file is a value pnpm never computes: raw sha256 322965470c05f63d8527f7d8e892ee26ee444136b66b57fd64c362a9f2ff05d1 LF-normalized f8ea245391c94da5770045aeea01fa6de466c2199c6ef46b5b769b398aa9823e The lockfile carried the raw one, at all three sites. It is the only one of the seven patches where the two digests differ, which is why the other six passed. Normalized the patch to LF and took pnpm's own value from `pnpm install --no-frozen-lockfile`; nothing here is hand-computed. With the file LF-only the two interpretations coincide, so the lockfile, the contract test's no-CR assertion and its hash assertion all agree at one number -- and `config/scripts/windows-process-tree-patch-contract.test.mjs`, which was red on this branch for the same reason, is green again. The lockfile diff is exactly the three hash lines. The regression check is the installer, not a digest. Two separate reviews "verified" the shipped hash by recomputing sha256(patchBytes) and matching the lockfile; both were wrong, because both repeated the same wrong assumption about which bytes pnpm hashes. A check that reproduces the original mistake is not independent. So the new case runs `pnpm install --frozen-lockfile --lockfile-only --ignore-scripts` against a copy of the manifest, lockfile and patches, and asserts exit 0 -- verified by deletion: restoring the shipped hash fails it with the exact ERR_PNPM_LOCKFILE_CONFIG_MISMATCH from the branch's package (windows) job. Also corrected the `.gitattributes` comment claiming pnpm hashes patches byte-for-byte. The `-text` setting is right -- `git apply` needs the exact bytes -- but that sentence is the claim that produced the wrong hash twice. * ci(windows): run the process-tree patch suites in CI Both suites only self-skip off Windows, so the binary-level check that the addon carries no ReadProcessMemory passed vacuously in every lane. * fix(windows): force core.autocrlf=input for the patch repair My LF normalization of the windows-process-tree patch broke the `git apply` repair path introduced in this PR. The two are coupled and I checked only one. Those 174 CR bytes were not editor noise. They sat on exactly the pre-image lines and nowhere else -- 107/107 in src/process.cc, 67/67 in src/process_commandline.cc, 0 on every added or context line -- because @vscode/windows-process-tree@0.8.0 ships those two sources as CRLF. Normalizing the patch made its pre-image stop matching the file it is applied against. Measured, reconstructing the true CRLF pre-image from the pre-normalization blob and applying the current LF patch: core.autocrlf plain -c core.autocrlf=input true exit 0 exit 0 input exit 0 exit 0 false exit 1 exit 0 `false` is Git's own built-in default and what "checkout as-is" selects in the Git for Windows installer -- on this box the `true` that hides it comes from the installer's system gitconfig, not from anything in the repo. There the repair throws, ensureWindowsProcessTreeCommandLinePatch reports "still reads the PEB, and repairing it ... failed", isWindowsNativeLockError does not match that text, and `pnpm install` dies with no path forward. Forcing the mode rather than `--ignore-whitespace`: both fix every cell and both leave the applied file fully LF, but `input` relaxes line endings only, so a hunk whose real content drifted is still rejected. The repair rewrites a security-relevant source file; it should stay strict about everything except the thing that is legitimately ambiguous. Not reverting the patch to CRLF: windows-process-tree-patch-contract.test.mjs (pre-existing on main) forbids CR bytes in it, and pnpm computes the same hash either way. LF plus the forced mode is the end state. The suite could not have caught this. The fixture built its pre-image from the patch itself and joined with '\n', so fixture and patch agreed by construction on any encoding -- once again a test that passes without its fix. It now emits the CRLF the real package ships, and the case runs under both autocrlf modes pinned through a temp HOME gitconfig, because the repair blinds git to the repo and so reads global config. Verified by deletion in both directions: with the flag removed the autocrlf=false case fails with the exact "still reads the PEB" dead end while autocrlf=true still passes, and with the fixture back on LF all eight cases pass with no fix present at all. Also corrected the .gitattributes comment I added last commit. It said `git apply` needs the bytes the patch was written against, which is now false -- the pinned bytes are LF and the bytes it was written against are CRLF. That is the same class of confident-and-wrong claim that produced the bad hash twice. * fix(windows): assert the rebuilt addon, and install the patch for real in tests Three follow-ups from review. **The packaged binary had no check.** The relay build asserts its own artifact and ensure-native-runtime asserts what it loads, but nothing looked at the addon copied into the packaged app -- so a rebuild that silently produced the upstream reader shipped. `rebuild-native-deps.mjs` now asserts `clean` on it after `rebuild()`. This is also the caller D4's tri-state was missing: every existing site branches on `=== 'unpatched'`, so `missing` still behaved exactly like `clean` everywhere, which was the thing making it a state rather than a boolean. Here both non-clean states fail, and they fail differently: after a rebuild that reported success, an absent binary is a broken build, not an absence to shrug at. The fake `rebuild()` had to start producing a binary for that to mean anything, so it now emits stand-in bytes and takes `addon: 'clean' | 'unpatched' | 'none'`. Verified by deletion: with the assertion removed both new cases pass. **The frozen-install case could not see a patch at all.** `--lockfile-only` resolves and never applies one, so its coverage stops at hash consistency. Added a case that installs `@vscode/windows-process-tree@0.8.0` for real with the patch and asserts the materialized `src/process_commandline.cc` carries the marker and no longer carries `ReadProcessMemory` -- about 1.5s for the pair. Correcting the brief on that one: it does **not** catch the `git apply` breakage from the previous commit. Measured -- with `-c core.autocrlf=input` removed it passes cleanly, because `pnpm install` uses pnpm's own patch applier and never runs our repair script. What it does catch is a patch pnpm can no longer apply: corrupting one pre-image line fails both cases. The repair path stays covered by the CRLF fixture in rebuild-native-deps-node-pty.test.mjs. Worth recording, since it decides whether the LF normalization was safe at all: pnpm applies the LF patch to the CRLF tarball sources without complaint, and materializes them as LF with the marker present and `ReadProcessMemory` absent. The primary install path was never affected -- only the `git apply` fallback was. **Dead timeout.** The frozen-install case passed `timeoutMs: 300_000` to the spawn while vitest capped the case itself at 30s, so on a cold runner vitest would have killed it first. Both cases now declare the budget they use. * test(windows): route the frozen-install check through the pnpm invocation owner The new patched-dependencies check hand-rolled a PATH walk naming 'pnpm.cmd', which the windows batch shim spawn boundary ratchet rejects: pnpm-cli-invocation already owns that decision for every other script, and its allowlist only shrinks. Reuse resolvePnpmCliInvocation for the command and prefixArgs, and the shared resolveCliCommand for the presence check, so no shim name is spelled here. Its `shell` flag is dropped because runProcessSync refuses it and already drives a shim through the interpreter itself. --------- Co-authored-by: Orca Worker <orca-worker@localhost> Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com> |
||
|
Neil
|
53f105827b |
perf(windows): stop asking the process table for memory, and share one projection per snapshot (#18151)
Two costs on the Windows process-table hot path, plus the EDR doc that described neither of them accurately. 1. The snapshot set `ProcessDataFlag.Memory` and surfaced `memoryBytes`, which nothing read. The addon serves that flag with a second `OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ)` and a `GetProcessMemoryInfo` per process (process.cc:47-63), so the flag was one wasted handle per process per snapshot. 2. The shared TTL cache gave every pane the same native rows array, but each pane still ran `native.map(toProcessRow)` over the whole table, rebuilt a `childrenByPpid` Map from scratch, and did two linear scans. The `.map()` also handed `getProcessTableIndex` a new array each call, defeating the POSIX memo by construction. Both now cache per snapshot identity, and the POSIX resolver drops its duplicate descendant walk. `getProcessTableIndex` / `buildProcessTableIndex` are generic over the row shape so the Windows rows reuse the existing pass instead of a parallel one. No behavior change: same rows in, same rows out, same descendant ordering and same has-children answers. |
||
|
Neil
|
b17f60d744 | build: upgrade to pnpm 12 (#17156) | ||
|
Jinjing
|
c4b39295c1 |
style: format codebase (#16935)
* style: format codebase * style: format codebase * refactor: extract skill install dialog footer and content Extract footer and content sections from SkillInstallDialog and SkillInstallManagementDialog into separate components for improved maintainability and clarity of component responsibilities. |
||
|
Neil
|
350423b7cb |
Speed up PR CI with path skips, native caches, and fewer shards (#16863)
* Speed up PR CI with per-job path skips and native caches Skip git-compat, xterm, packaging, and shell jobs when their inputs are unchanged, reuse the composite install action (including Windows node-pty cache), skip compiling the Windows CLI launcher on a cache hit, and cut the test matrix from 16x2 to 8x2 shards without dropping coverage. * Widen PR job skip prefixes for orcad browser and live shells Chrome session/tab modules and zsh/fish wrapper templates are inputs to required jobs the classifier previously skipped. Include that implementation graph so those jobs still run when the files they load change. * Fix CI cache safety and required gates * Build scriptless Windows addons explicitly * Preserve node-pty Windows support prebuild * Remove duplicated Windows launcher unit lane |
||
|
Brennan Benson
|
7c3bfe72d7 |
fix(windows): stop a wedged process-table reader retaining a callback per cooldown (#16696)
* fix(windows): stop a wedged process-table reader retaining a callback per cooldown The vendored reader pushes every callback onto a module-global queue and drains it only when the request holding its `requestInProgress` latch completes. When a Toolhelp32 snapshot never comes back, that latch is stuck for the life of the process, so the 30 s cooldown -- which let one probe through per window -- bounded the rate of new callbacks but not the total: one more closure retained every 30 s, forever, plus a full 3 s deadline block on whichever caller drew the probe. Gate on the outstanding read instead. Once a read misses its deadline and has not called back, every further read is refused until that read's callback fires, which bounds retention at exactly one callback. Nothing is given up on recovery: a probe queued behind the latch could never have observed the drain anyway, whereas the stuck callback firing IS the drain, so the reader now resumes the instant it recovers rather than up to 30 s later. It matters more on a relay, which binds the bare addon with no JS queue to absorb the retries. Each read there is a `Napi::AsyncWorker`, so a wedged one holds a libuv threadpool slot for good and one probe per window would have pinned all four default threads inside ~2 minutes -- hanging every async `fs` and DNS call in that process, not just the process table. A wedge still does not engage the PowerShell fallback, and a wedged read still rejects rather than resolving empty, so "unavailable" stays distinguishable from "nothing is running" on every host. Fixes STA-5499. * fix(windows): invalidate stale reader deadlines on reset |
||
|
Neil
|
0096e47850 |
fix(windows): keep windows-process-tree gyp paths absolute under pnpm (#16688)
* fix(windows): keep windows-process-tree gyp paths absolute under pnpm Hourly Windows builds have failed since #16598 at `build-windows-process-tree-relay-addon`: `require('node-addon-api').targets` is cwd-relative, so node-gyp evaluates it from the pnpm store realpath and then loads it from the `node_modules` symlink. That resolves `node_addon_api.gyp` outside the repo. Use `require.resolve` for an absolute path, matching the node-pty patch. * i18n: keep ja skill-filter labels on the catalog's Agent brand #16682 merged with a failing localization catalog: ja used エージェント in three new skill-filter strings, and repair-locale-catalog rewrites those to Agent. Match the rest of ja.json so static analysis can pass. |
||
|
Neil
|
64c992cd56 |
fix(memory): report the Windows number that predicts paging, not just resident pages (#16211) (#16589)
* fix(memory): report Windows commit charge, not just working set (#16211) On Windows the per-process figure was working set — resident pages only. An agent whose pages Windows has trimmed to the pagefile shrinks its working set while still holding the commit that pushes the host into paging, so Resource Manager and `orca diagnostics memory` understated an owned tree by 10-40x (9 codex.exe: 1.4 GB working set, 13.4 GB private) and could not warn before the host was already thrashing. Add committed private bytes as a second, separately-labelled quantity rather than redefining the existing one: - CIM sweep gains one property (PageFileUsage, UInt32 KB); the typeperf fallback gains one counter (\Process(*)\Private Bytes). Both ride the sweep that already runs. - MemorySnapshot gains optional `privateMemory` per app/worktree/session plus `processCommitMetric` and `totalPrivateMemory`. Rule 1 additive optional fields: old clients ignore them, and absence reads as "not measured", never as zero — Unix hosts and older hosts send nothing. - `totalMemory` and `processMemoryMetric` keep their exact meaning, so the "shared pages may repeat" copy stays true; the working-set copy now also says paged-out memory is not counted. - Resource Manager shows "Σ Private" beside "Σ WS", and tints the badge yellow/red once tracked commit passes 60/80% of physical RAM — the same thresholds `usageTextColorClass` already uses for host usage. Tint and tooltip only; no toast, and the badge number is unchanged. The parsers move to windows-process-sample-parsing.ts and the Windows sweep tests to their own file to stay under max-lines. Not migrating the collector to windows-process-table.ts: the native snapshot exposes no commit figure and no CPU times, and truncates WorkingSetSize through a DWORD. Documented in the enumeration reference. * fix(memory): derive the typeperf field cap from the counter list The fallback parser's 8192-field cap was sized for three `\Process(*)` counters. Adding `Private Bytes` cut the parsable process count from ~2730 to ~2047, and overrun is a blackout (`parseTypeperfCsvLine` returns `[]`, so the whole sweep reports nothing) rather than a truncation. The counter list now lives beside the decoder that reads those names back out of the PDH header, and the cap is derived from it. Also collapses the four spellings of "omit privateMemory when unmeasured" in collector.ts onto one `commitField` helper, drops the unread parameter and the never-rendered `columnLabel` from `getResourceCommitMetricCopy`, folds `getCommitPressurePercent` into the only function that called it, and reverts unrelated Prettier churn in the Windows enumeration doc. The commit tint's doc comment no longer claims to predict host paging: it measures Orca's own share of physical RAM. Host commit charge / commit limit stays a follow-up (#16211). |
||
|
Neil
|
19e9ec695b |
perf(windows): ship the native process table to Windows relay hosts (#16598)
* feat(windows): let a relay host bind the native process table directly The CIM fallback from #16550 answers on relay hosts, but it costs a powershell.exe and ~1.4s per scan where the native reader costs ~57ms. It is a parachute, not the destination. Teach the loader a second source: the desktop app keeps resolving the npm package, and a relay host -- which has none of our node_modules -- binds a bare `windows-process-tree.node` staged beside the bundle. The CIM scan stays as the last resort, so a host with neither is unchanged. Bind the addon directly rather than its package wrapper. lib/index.js adds only a queue over getProcessList, and that queue is the wedge this module already defends against: it latches a module-global requestInProgress with no try/catch. We hold our own single-flight and deadline, so going straight to the addon drops the duplicate. Measured on a Windows 11 SSH host with ~1490 processes, running the relay-externals bundle from the deployed relay directory: no addon staged nativeAvailable=false 1247ms (CIM) addon staged nativeAvailable=true 57ms memory restored Degradation was exercised on that host, not just in fakes: a truncated upload, a text file, and a foreign-arch ELF each fall through to the scan rather than throwing, and restoring a good addon recovers. A file that loads but lacks getProcessList is rejected by shape, because binding to it would reject every read forever where falling through still answers. No artifact is staged yet, so this is inert until the packaging change lands: today every relay takes the same CIM path it does now. * build(relay): ship the Windows process-table addon to relay hosts The CIM scan restored correctness on Windows SSH hosts, but it costs a powershell.exe and ~1.4s per read where the native addon costs ~57ms. It was always the floor, not the destination. The addon cannot be npm-installed on a relay host: it carries a binding.gyp, so npm rebuilds from source and the build wants Spectre-mitigated libraries even where MSVC is already present. The binary inside the published tarball loads, but predates our patch and still caps enumeration at 1024 processes -- on a 1486-process host it returned exactly 1024 rows with the querying process among the missing, which reads as unavailable only under load. No published alternative clears the bar either; the one fork with a working prebuild story still carries the same cap. So build it where a compiler exists and ship the result. The build script refuses unpatched source -- checking the source rather than trusting the install, because the Spectre hunk fails loudly while the 1024 hunk fails silently -- and verifies the PE machine field so a cross-build cannot emit host arch for another target. The artifact is optional: hashed when present so a relay carrying it never shares an immutable directory with one that does not, and never probed, since requiring a file only a Windows build machine can produce would make a correct relay read as MISSING and redeploy forever. Builds on any other OS keep using the scan, unchanged. arm64 cross-compiles from the x64 runner but needs the optional MSVC ARM64 toolset, so it stays best-effort: a runner image without that component should cost arm64 relays the fast path, not fail the release the x64 relay is riding on. ORCA_REQUIRE_RELAY_NATIVE_ADDONS is a per-arch list rather than a flag for exactly that reason. * build(relay): require the arm64 process-table addon too The arm64 cross-compile is no longer unproven. On a Windows x64 machine with the MSVC v143 ARM64 build tools component installed, node-gyp --arch=arm64 produces a genuine ARM64 image: x64 machine=0x8664 152064 bytes arm64 machine=0xaa64 139776 bytes So arm64 stops being best-effort and joins x64 in the required list. It was only best-effort because the component is optional and I had not seen it succeed; a runner image without it now fails the build with MSB8020 naming the missing component, and that step runs before the long packaging step so the failure costs seconds rather than twenty minutes. The env var stays a per-arch list rather than reverting to a flag, so a future arch can land best-effort before being promoted the same way. |
||
|
Neil
|
7f034a182f |
docs(windows): correct why the process-tree addon is not installed on relay hosts (#16565)
The note said the package "ships no prebuilds". It does: the published 0.8.0 tarball carries build/Release/windows_process_tree.node, apparently an accidentally published MSVC build directory (.obj and .tlog files ship with it). The conclusion was right and the reason was wrong, so record what was actually measured on a Windows SSH host with 1486 processes. Installing it normally rebuilds from source, because the tarball carries a binding.gyp and npm runs node-gyp regardless of what is already compiled inside. That build fails with MSB8040 (Spectre-mitigated libraries) even on a host that already has MSVC Build Tools 2022 -- the requirement our binding.gyp patch deletes, and patches do not cross SSH. Skipping the build keeps the tarball binary, which loads (it is N-API) but predates the src/process.cc patch and still caps enumeration at 1024. On that host it returned exactly 1024 rows with the querying process among the missing, which the self-presence guard rejects -- so it would work on a quiet machine and fail only under load, the shape of bug that survives testing. Also records the measured cost of the fallback, since the table's 706ms figure is from a 1050-process host and reads as more headroom than there is, and names the fix for the tracked gap: ship our own patched .node as a relay asset, as config/relay-assets already does for node-pty. |
||
|
JinjingandNeil
|
e4d95e032d |
fix(windows): restore a CIM fallback for relay hosts with no native binding (#16550)
Co-authored-by: Neil <4138956+nwparker@users.noreply.github.com> |
||
|
Neil
|
2b1254d681 |
fix(windows): own PTY process trees with job objects (#15755)
* fix(windows): own PTY process trees with job objects Teardown used to answer 'is this tree mine, and how do I kill it?' by scraping the process table, walking parent pids back to Orca, and running taskkill /T /F only if the walk said yes. Every step is a guess, and the code said so itself: windows-pty-root-identity.ts:35 already named the fix -- 'an inherited handle / Job Object'. The guesses fail in the ways users report. A pid walk cannot survive pid reuse, so teardown refused whenever it could not prove ownership, and a refused kill is an orphaned agent tree holding the worktree directory open (#9045, #10475, #10087). A descendant that reparented is invisible to the walk. The scrape itself could be blocked by policy, which read as 'no evidence'. node-pty now creates a job object per ConPTY and assigns the shell under CREATE_SUSPENDED, before it can spawn anything -- assigning afterwards leaves a window in which a fast child escapes. Termination is one TerminateJobObject; liveness is QueryInformationJobObject. Verified on Windows 11 against a shell whose grandchild was spawned detached: job membership came back [shell, grandchild] and one call killed both. Neither a parent-pid walk nor GetConsoleProcessList sees that grandchild -- it leaves the console and reparents, which is exactly the claude.exe/node.exe/cmd.exe orphan in #9045. KILL_ON_JOB_CLOSE means a daemon that dies without unwinding no longer strands shells (#9195, #10415). The job is the daemon's, not the app's, so an app-main crash still leaves sessions alive -- the guarantee win-crash-survival-e2e asserts. Both entry points report unavailable rather than a false success when a pty has no job: an outer job without BREAKAWAY_OK can refuse the assignment, and a pty from an older build has none. Reading 'we could not tell' as 'already dead' is the original bug, so the old probe stays as the fallback. * test(windows): pin job ownership against a real detached grandchild The unit tests pin the contract; this pins what the contract is for. A grandchild spawned detached leaves the pane's console and reparents, so GetConsoleProcessList and a parent-pid walk both miss it -- that is the process that outlived its pane and held the worktree directory open. Includes a guard that this build actually has job support, so a node-pty rebuilt from unpatched sources fails loudly instead of letting every assertion pass vacuously. * fix(windows): correct the job liveness contract to what Windows actually does I claimed an emptied tree would report [] and that this was the evidence a stale registry entry lacks (#15549). Running it on Windows 11 showed otherwise: node-pty drops its handle record and closes the job when the shell exits, so a dead tree reports null. Null therefore means unverifiable in the sense of docs/reference/ssh-execution-boundary.md -- no job support, not a ConPTY, or no longer tracked -- and is never evidence that processes died. A caller reading it as proof of death would have been right by accident after a normal exit and wrong on a host that refused the assignment. What the API does add is descendant liveness for a tree that is still tracked, including children that detached from the console. * fix(windows): stop a clean shell exit from reaping backgrounded processes Measured on Windows 11: with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE on the per-PTY job, releasing the handle when the shell exits also killed whatever the user had backgrounded. Typing 'exit' in a pane reaped a detached server that survived before this patch. That is a behaviour change nobody asked for. The approved change was that killing the terminal daemon reaps its shells -- not that a clean exit reaps your background job. The job's purpose is to make an EXPLICIT teardown exact, which TerminateJobObject still does. Reaping a dead daemon's shells now needs the daemon-level job the design called for: the daemon assigns itself, children inherit membership, and its closure on daemon death reaps them without touching clean-exit semantics. Not in this PR; noted in the reference doc. * test(windows): pin that a clean exit leaves backgrounded work alone The counterpart to the tree-kill test. Without it, re-adding JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE would look like a tightening rather than the regression it is. * fix(windows): stop a winpty pty id from matching a ConPTY job winpty.cc and conpty.cc each mint their 'pty' id from an independent counter, and windowsPtyAgent stores both in the same _pty field. So a winpty-backed terminal's id can collide with a live ConPTY baton -- and closing that pane would have terminated an unrelated pane's entire process tree. Both job entry points now take the shell pid and the native side refuses unless GetProcessId(hShell) matches, which makes the id unforgeable. Two more from the same read-through: - ResumeThread's failure was ignored. A shell left suspended is a pane that never prints and never exits, which is far harder to diagnose than a failed spawn; it now cleans up and throws. - handle->hJob was assigned before LoadConptyDll, which can throw. A baton carrying a job but never reaching SetupExitCallback has nothing left to close it, so the assignment moved down beside hShell. * docs(windows): record the unsynchronised node-pty baton table Pre-existing upstream -- the exit thread erases while the main thread reads -- but terminatePtyJob adds an instance of it, so it belongs in writing rather than in someone's head. * fix(windows): close four gaps found in review BREAKAWAY. The per-PTY job set no limits, so a child asking for CREATE_BREAKAWAY_FROM_JOB was refused with ERROR_ACCESS_DENIED. Installers, msiexec and some updater and service-control paths spawn that way deliberately -- they worked before this patch and would have failed only inside an Orca terminal, which is the worst shape a bug report can take. JOB_OBJECT_LIMIT_BREAKAWAY_OK restores it; a child still has to ask, so ordinary descendants stay owned. EMPTY IS NOT UNAVAILABLE. The native reader returns an empty list -- not an error -- when CreateToolhelp32Snapshot fails, which is what an EDR hook or a restricted token produces. Callers read that as 'nothing is running' and teardown concludes a live PTY root is already gone. The snapshot must contain the querying process; nothing else is unfalsifiable, and one predicate catches empty, truncated and permission-filtered tables alike. NO DEADLINE. Replacing execFile dropped its 3s timeout. The vendored reader latches a module-global while a request is in flight and clears it only after draining its callbacks, with no try/catch -- so one wedge leaves every later call queued behind a promise that never settles, and the process table is dead for the life of the app. The bound is back. GUESSED IMAGE PATH. executablePath was derived from the first space-delimited token, which reads 'C:\Program' out of an unquoted 'C:\Program Files\nodejs\node.exe ...'. Wrong evidence is worse than none, and the only consumer already had the full path in , so the field is gone rather than repaired. Also: remove_pty_baton no longer sits inside assert(), which NDEBUG would compile away along with the call, and the job accessors hold a lock across lookup and use -- handle values are recycled, so an unguarded read could pass the shell-pid check against an unrelated process and terminate the wrong job. * fix(windows): apply the job lock once per accessor The patch script matched a string its own replacement still contained, so PtyTerminateJob got two lock_guards named guard and PtyListJobProcessIds got none. MSVC caught it: error C2374 redefinition. * test(windows): pin that a child can still break away from the job Verified on Windows 11: 'start /b' writes its marker and no access-denied appears. Without JOB_OBJECT_LIMIT_BREAKAWAY_OK this fails, and it fails only inside an Orca terminal -- so the failure would look like Orca corrupting unrelated software rather than like a job-object change. * fix(windows): stop the ownership guard from reading a closing handle The guard called GetProcessId(hShell) to prove identity, but the exit watcher closes hShell on another thread -- so the guard could read a closed handle, and under strict handle checks that is fatal rather than merely wrong. Worse, it widened the gap between validating hJob and using it from two instructions to a kernel round-trip, and handle values recycle: the likeliest occupant of a freshly recycled value in this process is another pane's job. The pid never needed a handle. It is captured at spawn and compared as a DWORD, so the guard touches no handle at all, and hShell is now closed inside the same lock as hJob. Also from review: - reject CR/LF in a cmd argument. cmd ends the command at a raw line break whatever the quote state, so there is no escape for it; encoding one anyway truncates the argument and can leave the remainder to run as a command. Agent prompts are this encoder's motivating input. - ask the process table only for the fields a caller needs. Memory and CommandLine each cost an OpenProcess per process, inline, for every process on the box -- and the 1024 bound is patched out. Ancestry reads now skip both. - corpus gains the degenerate quote-only and two-quote arguments. - PtyListJobProcessIds' docblock still taught the empty-list contract that was corrected on the TS side, and now records that the ConPTY console host is never a job member. - drop a write to NumberOfAssignedProcesses, which is output-only. - pty_baton::hShell is initialised; ownsShell was only safe because && short-circuited ahead of it. The backgrounded-child test is rescoped: 'start /b' uses CREATE_NEW_CONSOLE, not CREATE_BREAKAWAY_FROM_JOB, so it proves job membership does not block backgrounding -- not that BREAKAWAY_OK works. That flag rests on the Win32 contract, and I have said so rather than letting the test imply coverage it does not have. * fix(windows): bound retries after the process table wedges The 3s deadline stops a caller hanging, but the timed-out call leaves its callback in the vendored module's queue -- and that queue drains only when the latched request completes, which in this wedge never happens. Retrying at the caller's poll rate would add a closure per tick forever. A 30s cooldown bounds it to one probe, and a late callback clears the cooldown because it proves the reader recovered. Also pins the deadlock invariant in the patch: the exit thread's lock must close before tsfn.BlockingCall, because that waits on the JS thread and the JS thread can be waiting on the same mutex inside PtyTerminateJob. Correct today by scoping; a comment so a later refactor does not widen it. * revert(windows): drop the field-selection API, which cannot pay off I added it for a real perf finding -- Memory and CommandLine each cost an OpenProcess per process -- and then never wired a caller, so the claim that ancestry reads skip them was wrong. Wiring it would have been worse than leaving it dead. The only ancestry consumer is the teardown identity probe, which needs a snapshot that started AFTER it asked, for pid-recycle detection. Bypassing the shared reader to get narrow fields would let that request join a scan already in flight -- trading a correctness guarantee for milliseconds. Field selection only pays off if callers can ask for less, and they cannot: one shared snapshot serves every caller so a 32-wide teardown collapses into a single scan, which means it has to carry every field. The reasoning now lives next to the flags instead of in a dead export. * fix(process): three P1s from review — a crash vector and two wedge bugs STDIN EPIPE COULD TAKE DOWN THE MAIN PROCESS. A child that exits without reading makes the queued write fail with EPIPE, and an unhandled error on a stream is an uncaught exception. The child's own error listener does not cover its stdin stream, so runProcess({ input }) against a short-lived child was a crash, not a failed call. THE COOLDOWN LEAKED A BATCH PER CYCLE INSTEAD OF BOUNDING IT. At expiry every concurrent caller passed the check before any of them re-armed it, so each enqueued a callback into the still-latched native queue and each cycle leaked another batch. The cooldown is now re-armed BEFORE probing, so exactly one caller gets through. A SYNCHRONOUS THROW LEFT ITS DEADLINE RUNNING. The timer was declared inside the try, so catch could not clear it; it fired later and wedged a reader that had already recovered. Hoisted and cleared, and wedge state now carries a generation so a request that lost its deadline cannot mutate it on behalf of the one that replaced it. Found by review once the prompts were short enough for the reviewer to finish -- the previous two rounds died on prompt length. * fix(process): stop a stream error from crashing the main process Same class as the stdin EPIPE finding, two instances further on: stdout and stderr had data listeners and no error listeners, and an unhandled error on a stream is an uncaught exception. Scoped to runProcess, which owns the child outright. spawnProcess hands the streams to its caller, and a blanket handler there defeats callers that track and remove their own listeners -- the SSH ProxyCommand transport does exactly that, and its cleanup test caught the attempt. Documented on spawnProcess so the boundary is explicit rather than inferred. * fix(windows): validate the ConPTY DLL before creating the process LoadConptyDll throws when conpty.dll is missing -- a real state, and one this branch hit during development. It ran after CreateProcessW and ResumeThread but before the baton and the exit watcher were installed, so a throw leaked the job, process and thread handles and left an untracked shell tree running. Once per attempt, so a broken install accumulates orphan shells on every retry. Resolving the DLL first costs nothing and leaves exactly two throws after creation: the CreateProcessW failure, where nothing exists yet, and the resume failure, which already cleans up after itself. This also closes the same leak for hProcess and hThread, which predates the job work. * feat(windows): add the daemon-level job the design called for The plan specified two nested jobs and I built one. That gap is why dropping KILL_ON_JOB_CLOSE from the per-PTY job cost the approved guarantee that a dead daemon reaps its shells -- I had one job trying to answer two questions, and the two answers conflict. They are separate jobs. The per-PTY job answers 'kill exactly this pane's tree, now', and cannot be kill-on-close because its handle is released when the shell exits, which would reap whatever the user backgrounded. The daemon assigns itself to a second job that IS kill-on-close; its handle is released only when the daemon dies. Children inherit membership, so every pty is covered and the per-PTY jobs nest inside it. Daemon, never app: an app-main crash must still leave sessions alive, which win-crash-survival-e2e asserts. Both jobs carry BREAKAWAY_OK, or a child asking to break away is refused at whichever level lacks it. Restores #9195 and #10415, which I withdrew from this PR earlier. * docs(windows): record what the host job does not cover An app-hosted PTY gets a per-PTY job but no crash reaping, because the alternative is a kill-on-close job on the app -- which is precisely what the crash-survival guarantee forbids. * ci(windows): run the win32 suites in the PR windows job Both were skip-on-non-win32 and had only ever run on one machine I drive by hand -- which went unreachable at exactly the moment I needed to verify the percent-escaping fix. Verification that depends on one box is not verification. The job already builds node-pty from patched source and already runs a useConptyDll test, so the ConPTY runtime files are in place by this step. This also makes the encoder a gate: the corpus is the only thing standing between an agent prompt and a mangled argv, and it now runs against real cmd.exe on every PR. * fix(deps): refresh the lockfile for the current patch hashes pnpm records a hash per patched dependency, and I regenerated both patches repeatedly across the review rounds without refreshing the lockfile. Every local run used --frozen-lockfile's looser sibling, so nothing caught it until CI did: ERR_PNPM_LOCKFILE_CONFIG_MISMATCH Cannot proceed with the frozen installation. The current "patchedDependencies" configuration doesn't match the value found in the lockfile Verified with pnpm install --frozen-lockfile locally this time. * ci(windows): build node-pty from source before the win32 suites CI proved the encoder fix on real cmd.exe -- 26/26 -- and in the same run proved the job suite had been testing an unpatched binary. node-pty prefers its upstream prebuild, which does not contain this patch, so every job-object export was absent and isPtyJobOwnershipAvailable() was false. That guard is why the failure was loud rather than a vacuous pass, and it is the reason the assertion exists. Packaging was never affected: rebuild-native-deps.mjs already builds node-pty from source for Electron and restores the ConPTY runtime files. The gap was the node-runtime test environment only. Not changing requiresPatchedNodePtySourceBuild's win32 exemption here. Its premise -- that the patch is Unix-only -- is now false, but lifting it also needs pnpm rebuild to force a source build, and I cannot validate that on macOS and Linux from here. Recorded as a follow-up instead of changed blind. * test(windows): gate the host-job guarantee in CI The daemon-level job had one hand-run proof and no automated coverage -- the same shape of gap that let an unpatched node-pty go unnoticed until CI caught it. It needs a real second process, because the assertion is about what happens when that process is force-killed: a host in a kill-on-close job must strand neither its pty nor a grandchild spawned detached, which is the process a parent-pid walk cannot see. Runs in the Windows PR job alongside the per-pty and encoder suites, so both halves of the two-job design are now gated rather than asserted. * fix(windows): serialise host-job creation Two callers racing PtyAssignCurrentProcessToJob would each create a job, put the process in both, and leak the first handle -- and the handle is what keeps a kill-on-close job alive, so a leaked one is never released. 'Only JS calls it' is not a guarantee: a worker thread with its own N-API env shares these statics. Also records the ordering requirement it depends on. AssignProcessToJobObject adds only the named process; children inherit membership, but a pty that already exists does not join retroactively and would not be reaped. The daemon assigns at startup, before the ConPTY warmup and before any session, which is correct today and now stated rather than implied. * fix(daemon): keep the host job off the startup path Assigning the host job at daemon startup resolves the node-pty native module, which loads the ConPTY addon -- and paying that before the endpoint is published delayed readiness enough that daemon-boot-smoke failed on windows-latest, deterministically. windows-conpty-warmup already carries the comment for this exact hazard ('setImmediate keeps the ready/handshake path ahead of the warm-up') and I put an eager load in front of it anyway. Moved to the pty spawn path, which already pays ConPTY cost, and memoised. Children inherit job membership, so assigning immediately before the first spawn still covers every pty -- and nothing can spawn one before the endpoint exists. |
||
|
Neil
|
057fbfcffc |
perf(windows): read the process table natively instead of forking PowerShell (#15749)
* perf(windows): read the process table natively instead of forking PowerShell Seven independent readers each forked powershell.exe to run Get-CimInstance Win32_Process, with a wmic fallback that Windows 11 24H2 has removed. On a domain-joined host with PowerShell Transcription enabled by policy, one of them running every ~2s recorded ~289GB across 1.4 million files (#15209). The same scan cost ~700ms and ran per pane (#15036), and a Group Policy or AV block turned it into 'unavailable', which callers read as 'no evidence' -- which is how a PTY tree survives its own teardown (#9045, #10475). A Toolhelp32 snapshot answers the same question with no child process. Measured on Windows 11 with 1050 processes, p50/p95: pid+ppid+name 15.9 / 17.5 ms +memory +command line 30.6 / 33.7 ms Get-CimInstance 706 / 723 ms Two upstream defects needed patching, both found by running it on real hardware. The binding requires Spectre-mitigated libraries our agents do not carry (node-pty is patched the same way). And enumeration stopped after 1024 processes: on a host with 1051 the module returned exactly 1024, and the querying process was itself among the 27 missing -- a truncated snapshot silently hides the descendants teardown is looking for, which is the failure this whole change exists to remove. Migrated: the foreground/descendant reader (the #15209 scraper and the teardown identity gate) and the port scanner's PID attribution. NOT migrated: the memory collector and three identity probes, which need Win32_Process.CreationDate and have no native equivalent. Start time is a proxy for identity anyway; an inherited job handle is the real answer, so those belong with the job-object work rather than here. Packaging follows the windows-native-registry contract exactly: optional, absent from onlyBuiltDependencies so macOS/Linux never run node-gyp, win32-only in the packaged runtime. Asserted by the existing contract test, which also stops pinning a whole source literal that only tested its own formatting. * chore(process): ratchet the child_process allowlist down windows-foreground-process-rows.ts no longer spawns anything, so its allowlist line is stale. The guard fails on a stale entry as well as a new one, precisely so a migrated file cannot keep a slot open and hide the next regression in the same path. * fix(ports): import the process-table reader the scanner uses Missing import: the migration replaced the PowerShell call but the new symbol was never imported, so tsc failed. Vitest transpiles without typechecking, which is why the port-scanner suite stayed green. * fix(deps): sync this branch's lockfile with its patch set Same class as the fix on the tip branch: pnpm records a hash per patched dependency, and this branch introduces the windows-process-tree patch without its lockfile entry matching. Every job here failed at install with ERR_PNPM_LOCKFILE_CONFIG_MISMATCH. Verified with --frozen-lockfile, which is what CI runs and what my local runs were not. * test(relay): drive the relay's Windows fixtures from the native snapshot Two relay cases fed a PowerShell CIM payload through a mocked execFile. That reader is gone, so both failed -- deterministically, on every PR run for this branch and the one above it. I did not catch it because my own verification sweep was 'src/main src/shared config/scripts' and never included src/relay. The relay is a first-class consumer of the process table; leaving it out of the sweep is how a deterministic failure survived six review rounds. |