No spf record should use the set of search domains in a local resolver;
all names are implicitly FQDN. Let's explicitly mark up the names that
way so that we can avoid both potentially matching a surprising local
record and also excess DNS lookups for records that might not exist.
I noticed a test failure where we hadn't yet received all of the
trace events that we wanted to inspect, so this commit introduces
the ability to wait for matching records to appear in the collected
set of data.
Similar to the general purpose resolve function, we should map
NoRecordsFound to empty list(s) of results in the specialized
ip, mx and ptr lookup functions, otherwise we can cause rule
evaluation to terminate too soon.
There's some kind of bad linker issue in CI that will take a little
bit to run down, and the update wasn't critical, so let's rever it
for the time being.
This never worked as well as the other amqp option that we support,
and upgrading it to the current version is a bit of a chore.
I'd rather just drop this dep and keep things more trim.
Since none of our sponsors are using the lapin implementation, let's
mark it as deprecated and we can take it out after we cut the next
stable release.
A "casualty" of this is that the client-ip crate, which we were using
for its InsecureClientIp type for informational purposes, has retired
the InsecureClientIp type because it wasn't trustworthy overall
(https://github.com/imbolc/axum-client-ip/issues/32).
What we do for the time being is configure that crate to use the direct
peer IP. There isn't a way to automagically select the "best"
information available without building in some kind of middleware, and
we probably should make that configurable in order to be fully
trustworthy.
So for now, we're punting on that until someone shows up with some
requirements and sponsorship and we'll make it happen then.
This commit implements a kumomta-specific message transfer
protocol that is intended to be used to migrate messages
from one kumomta node to another.
The transfer is carried out using an HTTP POST request
to the destination node's http listener.
The request includes the full message metadata and body,
in a compressed form.
An xfer request can be made via `kcli xfer` (and thus also via an HTTP API
endpoint). It works similarly to a rebind operation; you specify the
criteria to be used to match scheduled queues, along with the target
node for the xfer, and kumomta will find matching queues, drain out the
messages, make an adjustment to the metadata to capture current
scheduling information, and then place the messages into an xfer queue.
The xfer queue has hard-coded scheduling queue configuration of its own,
with the base retry interval set to 10 seconds, which should be suitably
aggressive for the intended use case.
You may apply shaping to affect the number of concurrent requests in a
similar way to how TSA shaping is configured.
On the receiving side, the incoming xfer sanity check to prohibit
trying to xfer to itself.
The spool id of the Message is not suitable to be reused verbatim on
another node (spool ids include the local mac address and creation
timestamp information, as well as a random component), so the receiving
side will derive an id that should be suitable for use on that node.
The originating node id and spool id will be preserved in metadata to
aid in tracing.
It is possible for an xfer request to target an existing xfer queue, so
that you can correct/update the target in various circumstances. In that
situation the messages will be "simply" moved from the source queue to
the destination queue.
It is possible to cancel an xfer request via `kcli xfer-cancel` (and
thus also via an HTTP API endpoint). You specify the target queue,
which must be an xfer queue, and it will have its messages drained and
the metadata changes that were applied when the xfer was initiated will
be reversed, allowing the messages to then be reinserted into their
originating queue.
refs: https://github.com/KumoCorp/kumomta/issues/311
These changes have been amended and rebased over the past few
weeks, so the version numbers for the json file no longer
correlate to real commits or dates.
If we already saved one or more messages that were split out
of a multi-recipient transaction, and then the subsequent save
fails, we would previously leave those earlier messages saved
in the spool and deliver them despite reporting a failure back
to the client.
This commit removes them from the spool and logs a Bounce record
for them.
This is still not an ideal outcome, but it will at least not
send a message for which we haven't claimed responsibility.
message:recipient() may now return an array style table
holding the recipient list, if there is more than a single
recipient on the message.
Since this can be somewhat ambiguous/frustrating to work with,
there is now also a message:recipient_list() that will always
return an array style table, even if it holds just a single
element.
The included helpers have been updated to use `message:recipient_list`.
message:set_recipient() will now optionally accept an array
style table holding the recipient list to be set.
This commit adds a test for exceeding the number of recipients
allowed in a transaction and fleshes out our handling of that
status so that we will immediately try the remaining recipients
of the batch, without delaying the message by eg: 20 minutes
in the default retry parameters.
As part of this, I noticed that we were returning the wrong
smtp status code in our server implementation, so this
commit fixes that up.
This makes a first pass over relaying a batch, with a test
to prove that it worked.
The logic in the smtp client likely needs to be enhanced
to handle hitting recipient-per-transaction limits; tests
for various scenarios need to be added in follow-on commits.
This adds a `batch_handling` parameter that optionally allows splitting
up a recipient batch based on the destination domain.
The system does not yet have any support for correctly delivering such a
batch.
This commit updates the type signatures of some key structures
to allow for the possibility of a Message having multiple recipients:
* Message (continuing from previous commit)
* JsonLogRecord
* MessageInformation
Some logic has been updated to account for multiple recipients,
but critically, nothing in the core will generate them, and none
of the queue management or delivery infrastructure is aware
of recipients beyond the first.
Use the more relaxed form of Name parsing so that domains
with underscores (which are in common use with email) do
not generate an error at runtime when processing spf
include rules.
Wez Furlong
Sophia J. TurnerandSophia Turner
kay ozaki