mirror of
https://github.com/neondatabase/neon.git
synced 2026-10-28 14:36:51 +00:00
Christian Schwarz
c7481402a0
# Motivation In https://github.com/neondatabase/neon/pull/8832 I get tokio runtime worker stack overflow errors in debug builds. In a similar vein, I had tokio runtimer worker stack overflow when trying to eliminate `async_trait` (https://github.com/neondatabase/neon/pull/8296). The 2MiB default is kind of arbitrary - so this PR bumps it to 4MiB. It also adds an env var to control it. # Risk Assessment With our 4 runtimes, the worst case stack memory usage is `4 (runtimes) * ($num_cpus (executor threads) + 512 (blocking pool threads)) * 4MiB`. On i3en.3xlarge, that's `8384 MiB`. On im4gn.2xlarge, that's `8320 MiB`. Before this change, it was half that. Looking at production metrics, we _do_ have the headroom to accomodate this worst case case. # Alternatives The problems only occur with debug builds, so technically we could only raise the stack size for debug builds. However, it would be another configuration where `debug != release`. # Future Work If we ever enable single runtime mode in prod (=> https://github.com/neondatabase/neon/issues/7312 ) then the worst case will drop to 25% of its current value. Eliminating the use of `tokio::spawn_blocking` / `tokio::fs` in favor of `tokio-epoll-uring` (=> https://github.com/neondatabase/neon/issues/7370 ) would reduce the worst case to `4 (runtimes) * $num_cpus (executor threads) * 4 MiB`.