Avoid client connection stalls and timeouts during connection bursts by
allowing configuration of the listen backlog, replacing the
rust-internal hardcoded 128 entry limit.
Prevent transient accept() errors from permanently stopping listeners
while the process stays alive and appears healthy. Trivial peer resets
that happen between kernel queue and accept are logged only at debug
level to close remote log spam vectors.
A message received via inter-node transfer, on systems hosted on AWS, could
end up with a wildly incorrect far-future timestamp. The underlying mac_address
crate would pick a NIC with the same MAC address as other AWS instances in the
cluster, and that triggered a code path where the collision resolving logic
misinterpreted the timestamp portion of the incoming spool id.
The thing that made this painful was the logic in spool_id.rs: it
misinterpreted the subsecond portion of the timestamp extracted from
the uuid, and due to the way that that field wraps, could produce wildly
inaccurate deltas with a huge multiplier.
As a belt and suspenders treatment, allow the user to influence which
MAC address is selected via the new KUMO_MAC_INTERFACE and
KUMO_MAC_ADDRESS environment variables.
Some upstream peers (e.g. QQ Mail's rate-limiter) silently hold a
proxied TCP connection open indefinitely — sending no data, FIN, or RST
— rather than cleanly refusing. Without some kind of timeout
management, the two file descriptors for such a session remain open for
the lifetime of the process, slowly exhausting the kernel's
file-descriptor table and occupying proxy-server worker slots.
This commit configures kernel level keepalive options with reasonable
defaults to detect and close out this class of connection.
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Closes: https://github.com/KumoCorp/kumomta/pull/509
We're keeping a reference to it via kumo.start_proxy_listener briefly,
just in case someone is using that live somewhere.
We don't need to changelog this, as this whole feature hasn't
been in a stable tag yet.
refs: https://github.com/KumoCorp/kumomta/pull/472
This option was named `--no-splice` on the command line because that
was the easiest UX there. The negative logic makes it harder to
understand how to enable/disable, so let's flip it and default
use_splice=true.
refs: #459
Add TLS encryption and username/password authentication support for
the KumoProxy SOCKS5 server, with full Lua configuration capabilities.
Proxy Server Changes:
- Add kumo.start_proxy_listener() Lua function with TLS support
- Add proxy_server_auth_rfc1929 event for Lua-based auth validation
- Return AuthInfo from auth for ACL system integration
- Support optional and required authentication modes
- Maintain backwards-compatible legacy CLI mode (--listen, --timeout-seconds)
Breaking Changes:
- Cache renamed from rfc5321_rustls_config to rustls_client_config
Refs: #451
Closes: 459
This makes the startup behavior consistent with kumod and tsa-daemon,
which can help with "Too many open files" errors in systems without
any explicit tunings for the proxy server.
It is still a good idea to explicitly tune that value up, and it is
something we should put into a systemd unit when we get around to
shipping one of those for the proxy server (#188)
We've been trying to run down an issue where a user has reported that
some sessions that are running via the proxy seem to hang waiting for a
response from the peer. The common theme is that the size of the
payload is approximately 1MB in size, and that the proxy is in use.
I haven't been able to get it to reproduce at all, but in looking
carefully at the code here, my splice(2) implementation used a pipe
buffer that was 1MB in size, and doing non-blocking IO outside of
tokio's internals is a bit of a black art, so I'd buy the theory
that we might be getting stuck somewhere if we did fill up that
pipe buffer.
Since I couldn't catch it in the act, I've opted to go for the simple
and safer route here, as a speculative remediation:
* Added a `--no-splice` command line parameter to opt out of using
`splice(2)` completely on Linux so that we can rule out weirdness
with splice completely. The result will have lower theoretical
max throughput, but a simpler internal implementation.
* There now exists a `tokio-splice` crate that has the same
functionality as my own splice_copy code does, but a different
implementation. Adopt that for the `splice(2)` mode.
* Switch away from splitting the stream into read/write halves: there
are now utility functions available in tokio and tokio_splice that
don't require splitting the streams, which further simplifies
the implementation.
Previously we were very tight-lipped. We now will log some
context to the journal for the proxy service for issues that
we couldn't propagate back to the client.
This is a pretty tight, lightweight implementation of SOCKS5
intended to be used to facilitate an MTA asking it to bind and
connect.
It uses the splice(2) syscall to avoid kernel/userspace copies
when passing data through, so it should be quite efficient.
refs: https://github.com/KumoCorp/kumomta/issues/45