Some upstream peers (e.g. QQ Mail's rate-limiter) silently hold a
proxied TCP connection open indefinitely — sending no data, FIN, or RST
— rather than cleanly refusing. Without some kind of timeout
management, the two file descriptors for such a session remain open for
the lifetime of the process, slowly exhausting the kernel's
file-descriptor table and occupying proxy-server worker slots.
This commit configures kernel level keepalive options with reasonable
defaults to detect and close out this class of connection.
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Closes: https://github.com/KumoCorp/kumomta/pull/509
We're keeping a reference to it via kumo.start_proxy_listener briefly,
just in case someone is using that live somewhere.
We don't need to changelog this, as this whole feature hasn't
been in a stable tag yet.
refs: https://github.com/KumoCorp/kumomta/pull/472
This option was named `--no-splice` on the command line because that
was the easiest UX there. The negative logic makes it harder to
understand how to enable/disable, so let's flip it and default
use_splice=true.
refs: #459
Add TLS encryption and username/password authentication support for
the KumoProxy SOCKS5 server, with full Lua configuration capabilities.
Proxy Server Changes:
- Add kumo.start_proxy_listener() Lua function with TLS support
- Add proxy_server_auth_rfc1929 event for Lua-based auth validation
- Return AuthInfo from auth for ACL system integration
- Support optional and required authentication modes
- Maintain backwards-compatible legacy CLI mode (--listen, --timeout-seconds)
Breaking Changes:
- Cache renamed from rfc5321_rustls_config to rustls_client_config
Refs: #451
Closes: 459
This makes the startup behavior consistent with kumod and tsa-daemon,
which can help with "Too many open files" errors in systems without
any explicit tunings for the proxy server.
It is still a good idea to explicitly tune that value up, and it is
something we should put into a systemd unit when we get around to
shipping one of those for the proxy server (#188)
We've been trying to run down an issue where a user has reported that
some sessions that are running via the proxy seem to hang waiting for a
response from the peer. The common theme is that the size of the
payload is approximately 1MB in size, and that the proxy is in use.
I haven't been able to get it to reproduce at all, but in looking
carefully at the code here, my splice(2) implementation used a pipe
buffer that was 1MB in size, and doing non-blocking IO outside of
tokio's internals is a bit of a black art, so I'd buy the theory
that we might be getting stuck somewhere if we did fill up that
pipe buffer.
Since I couldn't catch it in the act, I've opted to go for the simple
and safer route here, as a speculative remediation:
* Added a `--no-splice` command line parameter to opt out of using
`splice(2)` completely on Linux so that we can rule out weirdness
with splice completely. The result will have lower theoretical
max throughput, but a simpler internal implementation.
* There now exists a `tokio-splice` crate that has the same
functionality as my own splice_copy code does, but a different
implementation. Adopt that for the `splice(2)` mode.
* Switch away from splitting the stream into read/write halves: there
are now utility functions available in tokio and tokio_splice that
don't require splitting the streams, which further simplifies
the implementation.
Previously we were very tight-lipped. We now will log some
context to the journal for the proxy service for issues that
we couldn't propagate back to the client.
This is a pretty tight, lightweight implementation of SOCKS5
intended to be used to facilitate an MTA asking it to bind and
connect.
It uses the splice(2) syscall to avoid kernel/userspace copies
when passing data through, so it should be quite efficient.
refs: https://github.com/KumoCorp/kumomta/issues/45
Wez Furlong
Jack
Aditya Ganti
Kevin Vu
Dirkjan Ochtman