mirror of
https://github.com/neondatabase/neon.git
synced 2026-06-05 06:20:37 +00:00
7b22b5c433
Refactors Compute::prepare_and_run. It's split into subroutines differently, to make it easier to attach tracing spans to the different stages. The high-level logic for waiting for Postgres to exit is moved to the caller. Replace 'env_logger' with 'tracing', and add `#instrument` directives to different stages fo the startup process. This is a fairly mechanical change, except for the changes in 'spec.rs'. 'spec.rs' contained some complicated formatting, where parts of log messages were printed directly to stdout with `print`s. That was a bit messed up because the log normally goes to stderr, but those lines were printed to stdout. In our docker images, stderr and stdout both go to the same place so you wouldn't notice, but I don't think it was intentional. This changes the log format to the default 'tracing_subscriber::format' format. It's different from the Postgres log format, however, and because both compute_tools and Postgres print to the same log, it's now a mix of two different formats. I'm not sure how the Grafana log parsing pipeline can handle that. If it's a problem, we can build custom formatter to change the compute_tools log format to be the same as Postgres's, like it was before this commit, or we can change the Postgres log format to match tracing_formatter's, or we can start printing compute_tool's log output to a different destination than Postgres
51 lines
1.4 KiB
Rust
51 lines
1.4 KiB
Rust
use std::path::Path;
|
|
use std::process;
|
|
use std::thread;
|
|
use std::time::Duration;
|
|
use tracing::{info, warn};
|
|
|
|
use anyhow::{Context, Result};
|
|
|
|
const VM_INFORMANT_PATH: &str = "/bin/vm-informant";
|
|
const RESTART_INFORMANT_AFTER_MILLIS: u64 = 5000;
|
|
|
|
/// Launch a thread to start the VM informant if it's present (and restart, on failure)
|
|
pub fn spawn_vm_informant_if_present() -> Result<Option<thread::JoinHandle<()>>> {
|
|
let exists = Path::new(VM_INFORMANT_PATH)
|
|
.try_exists()
|
|
.context("could not check if path exists")?;
|
|
|
|
if !exists {
|
|
return Ok(None);
|
|
}
|
|
|
|
Ok(Some(
|
|
thread::Builder::new()
|
|
.name("run-vm-informant".into())
|
|
.spawn(move || run_informant())?,
|
|
))
|
|
}
|
|
|
|
fn run_informant() -> ! {
|
|
let restart_wait = Duration::from_millis(RESTART_INFORMANT_AFTER_MILLIS);
|
|
|
|
info!("starting VM informant");
|
|
|
|
loop {
|
|
let mut cmd = process::Command::new(VM_INFORMANT_PATH);
|
|
// Block on subprocess:
|
|
let result = cmd.status();
|
|
|
|
match result {
|
|
Err(e) => warn!("failed to run VM informant at {VM_INFORMANT_PATH:?}: {e}"),
|
|
Ok(status) if !status.success() => {
|
|
warn!("{VM_INFORMANT_PATH} exited with code {status:?}, retrying")
|
|
}
|
|
Ok(_) => info!("{VM_INFORMANT_PATH} ended gracefully (unexpectedly). Retrying"),
|
|
}
|
|
|
|
// Wait before retrying
|
|
thread::sleep(restart_wait);
|
|
}
|
|
}
|