I'd carried over behavior from wezterm where we only considered
tracked markdown files, which could be confusing for newly
authored files: they wouldn't get considered or updated until
they had been committed.
Several crates depend on the config crate, which means that
several things needed to be rebuilt after committing/amending/syncing.
Separating it out reduces that impact.
the pages workflow silently corrects these so they go unnoticed,
leaving the corrections to show up for the next person to run
`docs/build.sh`, polluting their later commit with unrelated
changes.
Augment the verify-pages workflow to also run on `main` and
surface the corrections with a failed workflow run.
This commit introduces awareness of memory limits that may
have been established for the process.
The idea is that the hard/soft limits are read from the active
cgroup, falling back to classic ulimit hard/soft limits, falling
back to the system RAM size.
In the absence of an explicitly configured hard limit, the RAM size
is used for the hard limit.
In the absence of an explicitly configured soft limit, 75% of the
hard limit is used for the soft limit.
A background thread monitors the memory usage and (potentially
adjusted) memory limits.
A low memory state is when the usage is within 10% of the soft limit.
In this state, we start to trim back memory usage, shrinking loaded
data when messages are placed into the ready queue.
A more severe memory state is when the usage exceeds the soft limit.
In this state, new message reception is rejected until the usage
recovers.
kumod doesn't do anything directly with the hard limit, but assumes
that the system will terminate the process with no opportunity to
safely clean up if that value is reached.
In order to reduce the memory usage, the allocator has been switched
to jemalloc which has superior management of fragmentation and
a control interface to request release of various memory caches.
We make use of jemalloc specific functions in the tokio thread pools;
when they park (idle), we release the thread local cache, and when
we reach the soft limit we aggressively flush the caches until
memory usage falls below the limit.
There's like some room for tuning of the thresholds around this,
but this is a reasonable start.
Note that there are no explicit config options in kumod to set
the soft and hard limits: those are taken from the process
environment and can be simply set either via ulimit in a calling
shell script, or via systemd service configuration.
We may introduce some knobs for the low memory state threshold
in the future.
I broke spool enumeration for local disk when I adjusted
the path hashing a little while back, and didn't notice
because I was using rocksdb more than regular disk.
This gives a higher quality panic stack trace at the cost of inflating
the executable size from ~50MB -> ~500MB.
Full debug info is a possibility, but the executable size is ~800MB,
which is high, but not completely crazy. I don't know if we'll really
need that, so going with line debug for now.
Wez Furlong
Mike Hillyer
Tom Mairs