These allow optionally reducing how aggressively the dispatcher and
maintainers will be awoken when message(s) are submitted to the ready
queue.
The default behavior remains the same; the new thing here is the
ability to make it more relaxed, which should reduce some CPU
overheads for very busy systems with many queues.
Making things more relaxed does introduce a possibility for higher
outbound latency in some edge cases with low or bursty traffic.
This commit does two things:
* Migrate the limit of 128 concurrent lookups from the shaping code
and into the general mx lookup code
* Adjust the mx lookup implementation to run inside the cache
getter, which applies thundering herd protection to the lookup.
These together will constrain the amount of queries being sent
to the upstream dns resolver and reduce the chances of it being
overwhelmed, especially if we are doing duplicate queries from
multiple concurrent contexts simultaneously.
Occasionally we'll have someone report that systemd timed out
and sigkill'd their kumo on shutdown.
One possible scenario for this is a lua delivery handler that
is taking too long, presumably because the other end of it
(eg: webhook or other custom endpoint) is not responding in
a timely fashion.
The way that we handle shutdown is that we compute a maximum
theoretical timeout value by summing up all of the smtp client
timeout values. Some of those can be several minutes in
duration because the are using default values derived from
a very conservative set of values suggested by the SMTP
RFCs from the '70s.
Those obviously should not apply to a custom delivery handler,
but also, in the context of an established SMTP session, we
should not add in the connection-establishment-specific values
when we're just waiting for a per-message send.
This commit addresses this situation on two fronts:
* Introduce a new system_shutdown_timeout value that allows the
user to conveniently express their desired timeout value
in a single option. This is *not* set by default!
* The default value for system_shutdown_timeout is computed by
summing the per-message-delivery smtp timeout options, which
is a much more reasonable, and more importantly, shorter than
our 300s TimeoutStopSec value in kumomta.service
Previously, we'd use the qmaint pool to spawn both the scheduled
and ready queue maintenance tasks.
This commit splits them apart in order to avoid the potential for
cross-task contention on the same threads if a scheduled queue
and ready queue pair are communicating with each other.
We recently observed a system running on an over-committed VM that
reported 4x the actually available parallelism.
Since we scale our thread pool sizes from this value, it resulted
in an extra-over-committed configuration for kumod.
You may now set KUMO_AVAILABLE_PARALLELISM in the environment to
override the value that we see both interally and expose via
the lua function with the same name.
This allows pre-defining connection metadata values. When coupled with
`peer` and/or `via`, these can be done based on the corresponding
addresses associated with the session.
closes: https://github.com/KumoCorp/kumomta/issues/355
The motivation here is to remove tls_config from EsmtpListenerParams
to make some future configuration changes easier, so this commit
moves that simple cache out to an explicit lru ttl cache.
This has the welcome side effect of enabling periodic reloading
of the tls parameters, which in turn makes it a hands-off process
for updating certificates: we no longer require the service to
be restarted for that.
These are hooked up only for memoize at this time. No default
behavior is changed by this commit, but you can optionally
specify these parameters in order to change the behavior.
We'll wait up to 3s at a time for however many mesages are available
to extract from the tsa daemon websocket, then process the results
in batches.
This avoids the potential for geometric complexity if there is a run of
subscription updates happening around the same time.
This commit allows setting a per-message `expires` timestamp
via msg:set_scheduling (and thus msg:import_scheduling_header).
The expiration takes precedence over max_age; max_age will be
ignored for messages that have configured and expiration time.
The expiration time is independent of the other scheduling
restrictions.
I worry that 1 minute might be too short to be universally good,
particularly as a change in behavior from an earlier build: going from
no timeout to 1 minute could be super surprising and un-welcome.
Let's bump this up to 5 minutes which more closely matches the
default for DATA in the base SMTP RFC.
This is why slow DNS often expanded to 3-4x as long as the expected
timeout value; we make a handful of calls in succession, assuming
that the cache was effective in an earlier stage.
We now cache errors for 5 minutes by default, and it can be configured
as desired.
refs: https://github.com/KumoCorp/kumomta/issues/325
If you have a lot of RAM, using only default percentages could
potentially leave some RAM unused. These lua functions
allow more precise control over the limits.
This was always present, it just wasn't something we provided a syntax
to specify.
max_burst allows you to control how quickly your throttle budget will
be consumed over the throttle time period.
The default max_burst is equal to the limit you specify, allowing
the full budget to be instantaneously consumed.
refs: https://github.com/KumoCorp/kumomta/issues/326
Wez Furlong
Edgar Vaitkevicius
Ryan Bonnell