This was deliberately adjusted to only change when a queue
was reapead in a prior release, in the interests of raw performance.
While that might work for most domains, it's not ideal for busy
domains that have continual traffic.
This change adopts an ArcSwap to atomically swap out the underlying
ArrayQueue. This introduces a small amount of overhead in the hot
path to atomically acquire the current version of the queue, but
it shouldn't be enough to be noticeable in practice.
We need to explicitly launch the connection attempt into a separate
task, otherwise the select! macro won't run it in parallel with
awaiting the shutdown notification.
Remove the r2d2 dep (which is synchronous only) and replace with
mobc which is a bit easier to use, async, and seems more developed.
Update the interface to expose more of mobc's connection pool
options.
Add explicit redis cluster integration tests.
We've been trying to run down an issue where a user has reported that
some sessions that are running via the proxy seem to hang waiting for a
response from the peer. The common theme is that the size of the
payload is approximately 1MB in size, and that the proxy is in use.
I haven't been able to get it to reproduce at all, but in looking
carefully at the code here, my splice(2) implementation used a pipe
buffer that was 1MB in size, and doing non-blocking IO outside of
tokio's internals is a bit of a black art, so I'd buy the theory
that we might be getting stuck somewhere if we did fill up that
pipe buffer.
Since I couldn't catch it in the act, I've opted to go for the simple
and safer route here, as a speculative remediation:
* Added a `--no-splice` command line parameter to opt out of using
`splice(2)` completely on Linux so that we can rule out weirdness
with splice completely. The result will have lower theoretical
max throughput, but a simpler internal implementation.
* There now exists a `tokio-splice` crate that has the same
functionality as my own splice_copy code does, but a different
implementation. Adopt that for the `splice(2)` mode.
* Switch away from splitting the stream into read/write halves: there
are now utility functions available in tokio and tokio_splice that
don't require splitting the streams, which further simplifies
the implementation.
try to nudge folks away from adopting copypasta of the advanced
section; we've seen more than few people using this when they
should just use the sources policy helper.
Add the note about the weighted robin implementation to make_egress_pool
as well.
Some http servers would canonicalize the trailing dot from the policy
domain by issuing a redirect. Since we don't follow redirects for
MTA-STS policies (per the RFC: redirects MUST NOT be followed), this
would result in no policy being loaded for that domain.
Correct this by stripping off that trailing dot.
To facilitate this, some adjustments needed to be made to the time
calculation in the maintainer because we previously didn't consider it
to be valid to have a retry_interval below 1 minute, but in order for
this integration test to be viable to run as part of the CI it needs to
run in significantly less time than 1 minute.
The approach taken here is to avoid considering 1 minute as the
baseline, but rather take 1/20th of the retry_interval. In the default
configuration, the numbers work out the same as previously, but they
will scale down as the retry_interval is reduced.
Care is taken to avoid a couple of borderline busy wait scenarios where
we might otherwise have woken up at unrealistically small intervals:
timeq can suggest 1ms in a few scenarios, and we just round those up to
the next second to avoid that.
It's worth noting that we do not consider the scheduled queue to be a
realtime, high granularity queue: anything that lands there is
considered to be bulk/batch and will be handled later: it isn't worth
prioritizing with high granularity because messages that land there are
generally not going to be delivered quickly.
Previously we were very tight-lipped. We now will log some
context to the journal for the proxy service for issues that
we couldn't propagate back to the client.
Change the field from a SocketAddr to a struct with distinct fields:
```json
// For SMTP delivery, the source address (and port) that was used.
// (*Since: Dev Builds Only*)
"source_address": {
// The source address. The port number may be unknown and reported
// as zero when using a proxy protocol.
"address": "10.0.0.1:53210",
// If a proxy protocol was used, this field will be
// set to its name. It may be null/not set for no proxy,
// "haproxy" or "socks5".
"protocol": "socks5",
// If a proxy protocol was used, this field will be
// set to the proxy server address. It will be null/not set
// when no proxy was used.
"server": "192.168.1.1:5000"
},
```
In #154, the request was to log configuration information here, but I
opted against this as there can be a number of different configuration
fields and the combinatorics for future changes make me uncomfortable
from a code maintainance perspective--it will already be heavy to
try to pass thu all of the existing config information, and as we
add more options in the future it will be awful not just to look at,
but also from a memory and storage overhead.
The approach taken here is to make a little struct that is flexible
enough to convey the desired information without it being too much of a
burden.
closes: #40closes: #154
The motivation for this is:
My test environment is not permitted to reach outbound port 25.
If I run an ad-hoc test without setting up an explicit sink,
I end up with messages that try to reach the public internet.
Since they are blocked at a firewall, each of the MX hosts in
the connection plan is subject to a 60s wait before trying the next
thing.
In addition, this can cause the shutdown to take longer while
we wait for the in-flight delivery attempts to complete.
Making a separate configuration option allows the local administrator
to decide how to split the time waiting for a connection from
the time waiting for the banner.
refs: https://github.com/KumoCorp/kumomta/issues/196
It is now possible to trace outbound SMTP sessions, filtering
by a variety of properties.
Details are in `kcli trace-smtp-client --help` and also in
the docs at /reference/kcli/trace-smtp-client.md
refs: #87
This is to facilitate more precise control over the lifetime of
connection objects.
You can now define `function sender:close()` on the connection
object that is returned from your lua protocol constructor.
This method will be called when the underlying
QueueDispatcher::close_connection method is called.
Adjust the logic in ready_queue to explicitly call
QueueDispatcher::close_connection in the cases where we're closing the
connection; previously we'd just leave it to the Drop handler to take
care of this.
Adjust the amqp docs to show how to bubble the close call through
to the associated client.
This macro allows embedding TOML data into the docs,
and showing it in a tab that has both the TOML and JSON
representation of that data.
It works by executing the toml2jsonc helper that was added
in an earlier commit.
There's some machinery here to compile that utility to run
in the context of the mkdocs docker image; that works
locally, let's see how well it works in CI!
Usage is simple; before:
```toml
["something"]
foo = "bar"
```
after:
{% call toml_data() %}
["something"]
foo = "bar"
{% endcall %}
The first page to get switched over to this is https://docs.kumomta.com/tutorial/configuring_kumomta/
refs: https://github.com/KumoCorp/kumomta/issues/212
Wez Furlong
Mike Hillyer
Daniel Schaaff
Tom Mairs