It passes, and I'm not sure that I want to commit it as-is, because
it takes ~17 seconds to run due to the way that config refreshing
works. The latency is much lower when using epoch based testing,
but we're not sure if either is significant at this time.
It was reported that AWS SES could mark the DKIM signature as failed
when using a specific combination of headers in the header list,
with specific lengths of the other fields.
The reason for this is that we had two passes of wrapping applied
to the header, and they might not agree on the formatting of
the header.
The solution is to remove the second pass and just take a bit more
care to emit the header in a wrapped form in the first instance,
that way there can be no discrepancy or conflict.
closes: https://github.com/KumoCorp/kumomta/pull/483
As part of this, we move template compilation to happen before
we queue up deferred generation; the compilation step should
be plenty fast enough that we can reasonably do that synchronously
and report any compilation errors back to the peer.
The goal is to treat peer-initiated-unilateral-disconnects as being
somewhat equivalent to the way that we would idle out the connection if
we had no messages for it, on the supposition that the most likely
cause of a disconnect in between command verbs that we send is that
the peer decided that we were idle too long. That isn't the only
possible reason, but it is the motivating example.
For the idled-out case we should close the current session and
have our message(s) go out on a separate session. We don't want
to try the next host in the connection plan in case we're talking
to some kind of honey pit configuration where only the first
MX in the plan is valid, and even talking to a secondary can
harm your ability to send to the first in the future.
Handling this is a little tricky because we need to take care of
the distinction between getting a unilateral before or after
we've attempted to send a message.
In the before-case we don't want to blindly assume that we should start
a new session because that would mean that a persistent issue on the
first MX would have us spinning our wheels trying new connections only
to the first MX over and over.
This commit adds a couple of integration tests that contrive situations
where we exercise some pertinent cases.
refs: https://github.com/KumoCorp/kumomta/pull/482
This crate queries machine/system information which is intended
to be used in system monitoring.
This information is not centrally collected or retained by kumocorp,
merely reported via an API endpoint which is accessible only to trusted
IPs.
Machine info tries to interrogate information about the running/hosting
cloud platform for the major cloud providers, as well as indicate
whether it is running in a container of some kind.
The full cloud information is not re-exported via the API endpoint at
this time, but a fingerprint that encodes things like the instance-id is
included.
We'd sometimes get a spurious timeout because we'd wait for
the SMTP message to hit the sink before shutting it down,
but not the AMQP messages to be delivered.
This is an alternative implementation of
https://github.com/KumoCorp/kumomta/pull/482 that checks for a
unilateral response as part of deciding whether we can re-use
the current connection.
By detecting this condition before we've popped a message, we avoid
classifying that message attempt as a transient connection failure, and
can send it with lower latency overall.
closes: https://github.com/KumoCorp/kumomta/pull/482
I believe this to be more of a micro optimization, because the
heavy lifting was already being done in the should_enq function,
which filters out before we commit the event to the spool.
Thanks to @smsvip for noticing that there was a discrepancy
between the UNINTERESTING_LOG_RECORD_TYPES and the per-record
configuration; we now use the former to derive the latter.
I've add more obviously irrelevant to TSA types to the config
as part of this commit.
refs: https://github.com/KumoCorp/kumomta/pull/481
refs: https://github.com/KumoCorp/kumomta/issues/478
I noticed that we weren't grabbing the Activity handle for injection
requests. Doing so allows us to reject a request that comes in
while we are shutting down the service, rather than accept it to
have it potentially dropped as we shut down.
I also realized that we need to make the xfer injection handler
match the same set of rules for the http injection handler, so
this commit refactors that logic to reuse it in both places.
If you start up while injectors are actively trying to send,
you might trigger a race condition where the spool isn't fully
assigned by the time that an injection request wants to store
a message to the spool.
Add a similar check for spool readiness to what we have in
the smtp server path.
We're keeping a reference to it via kumo.start_proxy_listener briefly,
just in case someone is using that live somewhere.
We don't need to changelog this, as this whole feature hasn't
been in a stable tag yet.
refs: https://github.com/KumoCorp/kumomta/pull/472
This is a conceptually breaking change, but shouldn't be harmful
in practice.
Recently, Microsoft hosted domains have had a number of incidences where
the DNS stopped returning IPv4 addresses, which lead to systems that
employ skip_hosts to disable their IPv6 addresses (presumably due to
difficulties in warmup and reputation) to trigger the terminal case
where we would sweep the ready queue away as a permanent routing
failure.
In the context of a transient failure to route ipv4 the permanent
disposition was surprising and unwanted so it feels better to make
the effect of this situation more closely map to how we handle the
case where no MX records are returned.
This reduces the boilerplate around declaring metrics
(counters, histograms, gauges) in their various forms,
and more or less standardizes them, making the syntax
more regular regardless of how the metrics are actually
stored.
We move the help out to doc comments, making it easier
to write a multi-line exposition on a given metric (in
the future; we're not doing that yet).
linkme is used to form a registry that can be used to
eagerly collect metadata from the various metrics. This
will be used to drive some automated documentation
extraction for the various metrics in a future commit.
Ordinarily, the logic that calls the requeue_message event is
responsible for increasing the number of attempts, computing the revised
due time, and processing message expiration.
When using XFER to move a message to another node, we capture the
scheduling state at the point that the xfer was initiated.
If we do this inside the requeue_message event callback then we will
capture the scheduling information *prior* to the normal adjustments
that would happen later.
This commit adds kumo.xfer.xfer_in_requeue to handle this case; you pass
through the scheduling information parameters that were added to the
requeue_message event in the prior commit, and it will compute any
revised scheduling for the message before wrapping the message up
in the XFER encapsulation and moving the message to the xfer queue.
This adjusted process ignores message expiration, so if the message
is on its last retry it will be xfer'd to the target host which then
might choose to perform the expiration. We can't handle the expiration
in the context of `requeue_message` because the event handler doesn't
own the message and has no way to signal that it has reached the end of
its life.
Folks doing xfer in requeue_message will generally be moving the message
well in advance of it being expired, so this minor limitation is not
expected to be a practical problem.
Wez Furlong
Aditya Ganti
Tobias Horst
Daniel Smith
Edgar Vaitkevicius
kay ozaki