Files
tty7/crates/tty7-server/tests/handoff.rs
T
l0ng-aiandl0ng-ai 88bf9a5da5 feat(daemon): upgrade in place, keep pane screens across a crash, and give panes their own history (#449)
* feat(daemon): keep a pane's screen across a death nobody chose

A daemon that crashes, is `kill -9`'d, or goes down with the machine takes
every pane's replay ring with it, and the window comes back to a row of
blank shells. The processes cannot be saved that way — nothing written to
a file brings a process back — but the picture can.

The daemon now keeps a capped tail of each pane's ring under the config
directory, and a client whose `Attach` found nothing can ask, on the
`Spawn` that replaces it, for the dead pane's screen. The new pane opens
showing it, under a rule that says the shell below is new.

- periodic and dirty-only: a ring that has not moved is not rewritten, so
  an idle machine does no IO at all. Write-through would be an enormous
  amount of write amplification for a few seconds of freshness.
- capped at 256 KiB per pane, far below the ring's 8 MiB: the value of
  scrollback decays with distance from the bottom, and every byte here is
  a byte of someone's terminal on disk.
- off by default. The ring holds whatever the pane printed, including
  echoed tokens, `env` output and agent transcripts; in memory that dies
  with the daemon, and writing it down is the whole feature and the whole
  cost. Files are 0600, and turning the setting off deletes what was kept.
- dropped by relevance, not by calendar: a pane the user closed, or one no
  workspace names any more, has its file removed on the next sweep.

Restored bytes are replayed at the geometry they were written at, and are
preceded by resets — leave the alternate screen, show the cursor, restore
autowrap, clear SGR — because a snapshot is cut at the front and can begin
in the middle of any of them.

* feat(daemon): upgrade the daemon in place instead of killing every shell

Picking up a new build meant stopping the daemon, and stopping the daemon
means every pane dies: the pty master is a descriptor this process holds,
so when the process goes the slave side raises SIGHUP and takes the shell,
the agent and the half-finished command with it. That is why the update
path leaves the old daemon serving and Settings has to offer the restart
as a thing you schedule for a quiet moment.

`execve` does not have that problem. It replaces the image and keeps the
process: same pid, same children, same descriptors, same file locks. The
daemon now rewrites itself that way on `ClientMsg::Handoff` — it writes
what it knows about each pane into a blob, clears FD_CLOEXEC on the pty
masters, the blob and the singleton lock, and execs the new binary, which
picks the panes back up on the other side.

- **the seat travels on the command line, not in the blob.** The lock is
  still held by this process, so the new image must adopt the descriptor
  rather than ask for the lock again — asking would be refused by its own
  lock and it would stand down in favour of itself. A daemon that loses
  its panes is a bad afternoon; a daemon that exits leaves the machine
  with nothing serving, so that one fact has to survive an unreadable blob.
- **the blob is unlinked before it is written.** It holds every pane's
  ring, which is the output `scrollback` makes people opt into storing;
  a handoff must not be a back door for writing it to disk.
- **the exec is the last step.** Everything is staged first, so any
  failure before it costs a log line and the daemon carries on serving —
  which is what lets callers treat a failed handoff as "fall back to a
  restart" without having lost anything on the way.

Native SSH panes cannot cross — their session is cipher state in memory,
not a descriptor — so they are hung up first and the far end sees a clean
close. Windows has neither execve nor a transferable ConPTY handle, so it
keeps the stop/start path; the dialogs there still promise what they
always did, and the new copy is shown only where it is true.

Also retries flock on EINTR: a signal landing mid-call said nothing about
the lock, but was reported as "could not be evaluated", which starts a
second daemon beside the first — the split machine singleton exists to
prevent.

The end-to-end test sets a variable in the shell, hands over, and reads it
back. Nothing but the original process can answer that, and the daemon's
instance id changing while its pid does not is what says an exec really
happened.

* feat(shell): give each pane its own history when asked

Two panes running zsh with `share_history` are appending to one file and
reading each other's lines back, which is either the feature or the
problem depending on what the panes are for. Someone with a pane per task
wants Up to walk that task's commands, not an interleaving of four.

Each pane can now have its own history file instead. It is seeded from
the shell's real history, so a new pane is not blank, and what the pane
added is appended back when it closes, so nothing typed is lost — a
per-pane history that evaporated would be a way of losing commands, not
of organising them.

The seeding is done by the shell, not the daemon, and that is the only
reason it works: `HISTFILE` belongs to the user's rc file and can point
anywhere, long after the pane's environment was decided. tty7's snippet
is appended to the rc it wraps, so it runs after that decision and is the
one place the real path is known — it copies the tail, records how much it
copied, and repoints. Both shells load history after their startup files,
so the switch lands before the first line is read.

The daemon's half is a filename, a rename when a restored pane inherits
its predecessor's file, a merge on close, and a sweep for the panes a
killed daemon never got to retire.

Off by default: shared history is what a terminal has always done, and
someone who did not ask for the change would experience it as their
history mysteriously forgetting the other window. bash and zsh only —
fish and PowerShell do not keep a HISTFILE, and a shell launched with the
user's own arguments gets no snippet to repoint anything in.

* fix(daemon): store pane screens on the shutdown a restart actually uses

The periodic writer covers a death nobody prepares for and the SIGTERM
path covers a signal, but the restart the app itself performs goes through
ClientMsg::Shutdown — which killed every pty without taking a copy first.
That is the one shutdown where the panes are expected back.

* fix(daemon): leave nothing dangerous behind when a handoff fails or lands

Review findings on the in-place upgrade and per-pane history:

- A failed exec now puts back everything it had staged: FD_CLOEXEC on the
  seat and every pty master (a child inheriting the seat keeps the flock
  held past the daemon's death, so no future daemon could seat itself),
  and the SIGPIPE disposition plus this thread's signal mask, both of
  which Command::exec resets on its way to the attempt — without this,
  the still-serving daemon dies on the first client that hangs up
  mid-write.
- The adopting image restores close-on-exec on the seat and on every
  adopted master, so children it spawns later cannot hold a pty open
  past its pane, or the seat past the daemon.
- The target binary is checked before the handoff gives anything up:
  native-SSH panes are hung up on the promise that this process is about
  to be replaced, and an exec that was never going to work must not
  collect on it.
- The integration snippets raise HISTSIZE/HISTFILESIZE (bash) and
  SAVEHIST/HISTSIZE (zsh) for the pane's private history file. At their
  defaults the exit rewrite truncates the file below its own seed mark,
  which the merge-back rightly reads as "replaced under us" — silently
  losing the pane's commands for anyone with more history than the caps.
- The restart dialog's promise now binds the action: where the copy said
  "nothing is interrupted", a failed handoff is reported instead of
  silently traded for the restart that kills every pane.
- The scrollback writer checks the ring's mark before cloning it, so an
  idle pane no longer costs a full ring copy under the state lock every
  tick.

Each behavioural fix carries a test that fails without it; the history
truncation one was verified to fail with the snippet change removed.

---------

Co-authored-by: l0ng-ai <24760907+l0ng-ai@users.noreply.github.com>
2026-08-10 10:59:02 +08:00

259 lines
8.5 KiB
Rust

//! The daemon replaces its own binary and the shells never notice.
//!
//! Everything else about a handoff can be checked in a unit test — the blob
//! round-trips, the flags parse, the ids keep climbing. None of that answers
//! the only question that matters, which is whether the process on the other
//! end of the pty is still the same process afterwards. So this runs a real
//! daemon, puts a real shell in a real pty, and asks the shell.
//!
//! The proof is a variable. `tty7_kept=…` lives in that shell's memory and
//! nowhere else: no file has it, the daemon has never seen it, and a shell
//! started after the handoff would answer with an empty string. Getting the
//! value back is something only the original process can do.
#![cfg(unix)]
use std::path::{Path, PathBuf};
use std::process::{Child, Command, Stdio};
use std::time::{Duration, Instant};
use tty7_core::client::PaneClient;
use tty7_core::daemon::protocol::{DaemonMsg, ShellSpec, WinSize};
const READY_WITHIN: Duration = Duration::from_secs(30);
const STREAM_WITHIN: Duration = Duration::from_secs(30);
struct Daemon {
child: Child,
dir: tempfile::TempDir,
}
impl Daemon {
fn start() -> Daemon {
let dir = tempfile::TempDir::new().unwrap();
let child = Command::new(Self::binary())
.arg("--daemon")
.arg("--config-dir")
.arg(dir.path())
.env("TTY7_DATA_DIR", dir.path())
.env("TTY7_CONTROL_SOCK", dir.path().join("control.sock"))
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.spawn()
.expect("start tty7-server --daemon");
let daemon = Daemon { child, dir };
daemon.await_ready();
daemon
}
fn binary() -> PathBuf {
PathBuf::from(env!("CARGO_BIN_EXE_tty7-server"))
}
fn panes(&self) -> PaneClient {
PaneClient::at(self.dir.path().join("daemon.sock"))
}
/// The pid the daemon records for itself. An `exec` keeps it; stopping and
/// starting cannot.
fn recorded_pid(&self) -> Option<u32> {
std::fs::read_to_string(self.dir.path().join("daemon.pid"))
.ok()?
.trim()
.parse()
.ok()
}
/// A fresh uuid per program image: it is minted on first use and lives in
/// memory, so it survives anything except being replaced. Together with the
/// pid it pins down exactly what happened — same pid and a new instance is
/// an `exec` and nothing else.
fn instance(&self) -> String {
self.panes()
.version()
.expect("the daemon answers its version")
.instance
}
fn await_ready(&self) {
let deadline = Instant::now() + READY_WITHIN;
loop {
if self.panes().version().is_ok() {
return;
}
assert!(
Instant::now() < deadline,
"the daemon did not open its pane endpoint within {READY_WITHIN:?}"
);
std::thread::sleep(Duration::from_millis(50));
}
}
}
impl Drop for Daemon {
fn drop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
}
}
fn size() -> WinSize {
WinSize {
cols: 100,
rows: 30,
cell_w: 8,
cell_h: 16,
}
}
fn interactive_shell() -> ShellSpec {
ShellSpec {
program: "/bin/sh".into(),
args: Vec::new(),
args_are_tty7_defaults: false,
}
}
fn windows_contain(haystack: &[u8], needle: &[u8]) -> bool {
haystack.windows(needle.len()).any(|w| w == needle)
}
fn collect_until(session: &mut tty7_core::client::PaneSession, marker: &[u8]) -> Vec<u8> {
let mut seen: Vec<u8> = Vec::new();
loop {
match session.recv() {
Ok(DaemonMsg::Output(bytes)) | Ok(DaemonMsg::Snapshot(bytes)) => {
seen.extend_from_slice(&bytes);
if windows_contain(&seen, marker) {
return seen;
}
}
Ok(DaemonMsg::Exited { code }) => panic!(
"the pane exited ({code:?}) before {:?} appeared; saw {:?}",
String::from_utf8_lossy(marker),
String::from_utf8_lossy(&seen)
),
Ok(_) => {}
Err(e) => panic!(
"the pane stream ended early: {e}; saw {:?}",
String::from_utf8_lossy(&seen)
),
}
}
}
#[test]
fn a_handoff_keeps_the_process_the_pty_and_the_shell_that_is_on_it() {
let daemon = Daemon::start();
let panes = daemon.panes();
let before_pid = daemon.recorded_pid().expect("the daemon records its pid");
let before_instance = daemon.instance();
let mut session = panes
.spawn(None, size(), Some(interactive_shell()), None, None)
.expect("spawn an interactive pane");
let pane_id = session.pane_id();
session
.set_recv_timeout(Some(STREAM_WITHIN))
.expect("bound the stream reads");
// Put something in this shell's memory that exists nowhere else, and print
// a marker so we know the shell has read its input before we hand over.
session
.input(b"tty7_kept=survivor; echo tty7_before_$tty7_kept\r")
.expect("the shell takes input");
collect_until(&mut session, b"tty7_before_survivor");
drop(session);
panes
.hand_off(&Daemon::binary())
.expect("the daemon hands over");
daemon.await_ready();
assert_eq!(
daemon.recorded_pid(),
Some(before_pid),
"an exec keeps the process; a different pid here would mean the daemon stopped and \
started, which is the thing this is supposed to avoid"
);
assert_ne!(
daemon.instance(),
before_instance,
"and a *new image* has to be what is answering — without this the test would pass just \
as well if the handoff had quietly done nothing at all"
);
let mut session = panes
.attach(pane_id, size())
.expect("the pane is still there under the same id");
session
.set_recv_timeout(Some(STREAM_WITHIN))
.expect("bound the stream reads");
// The replay is the ring the previous image was holding.
let replayed = collect_until(&mut session, b"tty7_before_survivor");
assert!(
windows_contain(&replayed, b"tty7_before_survivor"),
"the pane came back without the output it had before the handoff"
);
// And the variable. Only the shell that ran the first line can answer this;
// the command line echoes back unexpanded, so the expansion in the output
// is unambiguous.
session
.input(b"echo tty7_after_$tty7_kept\r")
.expect("the shell still takes input");
collect_until(&mut session, b"tty7_after_survivor");
session.kill().expect("kill the pane");
}
#[test]
fn a_handoff_to_something_that_will_not_exec_leaves_the_daemon_serving() {
let daemon = Daemon::start();
let panes = daemon.panes();
let before_pid = daemon.recorded_pid().expect("the daemon records its pid");
let before_instance = daemon.instance();
let mut session = panes
.spawn(None, size(), Some(interactive_shell()), None, None)
.expect("spawn an interactive pane");
let pane_id = session.pane_id();
session
.set_recv_timeout(Some(STREAM_WITHIN))
.expect("bound the stream reads");
session
.input(b"echo tty7_still_here\r")
.expect("the shell takes input");
collect_until(&mut session, b"tty7_still_here");
let err = panes
.hand_off(Path::new("/nonexistent/tty7-that-is-not-there"))
.expect_err("a binary that cannot be executed must be reported, not assumed");
assert!(
err.to_string().contains("cannot become"),
"a missing binary is refused before anything is given up; the refusal was {err}"
);
// The state is staged before the exec and the exec is the last step, so a
// failure at that step has to cost nothing at all.
assert_eq!(daemon.recorded_pid(), Some(before_pid));
assert_eq!(
daemon.instance(),
before_instance,
"nothing was replaced, so the same image has to still be answering"
);
session
.input(b"echo tty7_unharmed\r")
.expect("the pane still has its shell");
collect_until(&mut session, b"tty7_unharmed");
assert_eq!(
session.pane_id(),
pane_id,
"the pane the caller was holding is the pane it still holds"
);
session.kill().expect("kill the pane");
}