This allows a (more or less) arbitrary lua function to be used to load
data.
The primary advantage of this is that the size of the resulting
KeySource struct is smaller than it would be to hold the actual data
inline. It also enables the surrounding code to be slightly better
factored.
Add CBC and ecb Block mode aes encryption/decryption.
To facilitate testing this better, and just to be nicer overall, adjust
data-loader's KeySource::key_data variant to support loading raw binary
bytes.
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Closes: https://github.com/KumoCorp/kumomta/pull/395
Add a convenience function that is intended to be used around the
time that you might be building up the overall set of
Authentication-Results.
Populate missing smtp.mailfrom in SPF authentication result.
We were including the CRLF in the measurement that we were making,
while the default value was the value from the spec excluding the CRLF.
Let's increase the default value to 1000 to make things match up.
This commit does some plumbing work to allow defining alternative
resolvers with different names that can be used to query different
upstream DNS servers.
This is most useful when implement DNSBLs where you might have a custom
DNS server configured with one or more RBL zones that is reserved purely
for RBL lookups.
The plumbing introduces a more regular syntax for defining one of the
various implementations of the Resolver trait, as well as a new
Aggregate resolver impl that can query across multiple Resolvers until a
query is satisfied.
This allows some interesting and powerful configurations, such as
loading a static zone file into memory to query it directly, and/or
blending that together with querying either the system or some other
upstream DNS server as a fallback.
The various lookup functions (except for MX!) have been updated to
accept an optional alternate resolver name, so that they work together
with the above.
A new rbl_lookup function is also provided as a convenience for querying
the most common form of RBLS.
ptr_host and reverse_ip are two string utility functions that are likely
not going to be widely used, but are very convenient to have when you
do have a usecase that requires it!
closes: https://github.com/KumoCorp/kumomta/issues/269
This never worked as well as the other amqp option that we support,
and upgrading it to the current version is a bit of a chore.
I'd rather just drop this dep and keep things more trim.
Since none of our sponsors are using the lapin implementation, let's
mark it as deprecated and we can take it out after we cut the next
stable release.
A "casualty" of this is that the client-ip crate, which we were using
for its InsecureClientIp type for informational purposes, has retired
the InsecureClientIp type because it wasn't trustworthy overall
(https://github.com/imbolc/axum-client-ip/issues/32).
What we do for the time being is configure that crate to use the direct
peer IP. There isn't a way to automagically select the "best"
information available without building in some kind of middleware, and
we probably should make that configurable in order to be fully
trustworthy.
So for now, we're punting on that until someone shows up with some
requirements and sponsorship and we'll make it happen then.
This commit implements a kumomta-specific message transfer
protocol that is intended to be used to migrate messages
from one kumomta node to another.
The transfer is carried out using an HTTP POST request
to the destination node's http listener.
The request includes the full message metadata and body,
in a compressed form.
An xfer request can be made via `kcli xfer` (and thus also via an HTTP API
endpoint). It works similarly to a rebind operation; you specify the
criteria to be used to match scheduled queues, along with the target
node for the xfer, and kumomta will find matching queues, drain out the
messages, make an adjustment to the metadata to capture current
scheduling information, and then place the messages into an xfer queue.
The xfer queue has hard-coded scheduling queue configuration of its own,
with the base retry interval set to 10 seconds, which should be suitably
aggressive for the intended use case.
You may apply shaping to affect the number of concurrent requests in a
similar way to how TSA shaping is configured.
On the receiving side, the incoming xfer sanity check to prohibit
trying to xfer to itself.
The spool id of the Message is not suitable to be reused verbatim on
another node (spool ids include the local mac address and creation
timestamp information, as well as a random component), so the receiving
side will derive an id that should be suitable for use on that node.
The originating node id and spool id will be preserved in metadata to
aid in tracing.
It is possible for an xfer request to target an existing xfer queue, so
that you can correct/update the target in various circumstances. In that
situation the messages will be "simply" moved from the source queue to
the destination queue.
It is possible to cancel an xfer request via `kcli xfer-cancel` (and
thus also via an HTTP API endpoint). You specify the target queue,
which must be an xfer queue, and it will have its messages drained and
the metadata changes that were applied when the xfer was initiated will
be reversed, allowing the messages to then be reinserted into their
originating queue.
refs: https://github.com/KumoCorp/kumomta/issues/311
These changes have been amended and rebased over the past few
weeks, so the version numbers for the json file no longer
correlate to real commits or dates.
message:recipient() may now return an array style table
holding the recipient list, if there is more than a single
recipient on the message.
Since this can be somewhat ambiguous/frustrating to work with,
there is now also a message:recipient_list() that will always
return an array style table, even if it holds just a single
element.
The included helpers have been updated to use `message:recipient_list`.
message:set_recipient() will now optionally accept an array
style table holding the recipient list to be set.
This commit updates the type signatures of some key structures
to allow for the possibility of a Message having multiple recipients:
* Message (continuing from previous commit)
* JsonLogRecord
* MessageInformation
Some logic has been updated to account for multiple recipients,
but critically, nothing in the core will generate them, and none
of the queue management or delivery infrastructure is aware
of recipients beyond the first.
I can see someone wanting to know the raw physical connection
info at some point, regardless of the adjusted xclient values,
or in the future, the proxy protocol adjusted values.
Hooks up the parsed XCLIENT command to the smtp server.
We support just the address changing attributes; we don't
do anything special with the NAME attribute, and I'd like
to think a lot more about authentication before allowing
messing with the LOGIN attribute.
After applying changes to the metadata in the connection,
take care to re-evaluate the listener parameters and
dynamic parameters so that `via` and `peer` blocks can
match the new configuration.
Finishes up support for these extensions by adding a send-time
check and error for the case where we have 8 bit content and
the appropriate extension is not advertised by the next hop.
A shaping option is provided to bypass this, both to facilitate
testing and as a way to bypass it for situations where the
operator knows best.
closes: https://github.com/KumoCorp/kumomta/issues/327
To facilitate this we need to feed the `AppState` down through
the Router more generally, and expand it to grab a copy of the
http listener parameters.
closes: https://github.com/KumoCorp/kumomta/issues/417
We made the unfortunate choice to emit the `timestamp` and `created`
fields as unix timestamps with no sub-second precision.
This commit causes us to emit the full-resolution timestamps that we
already have as additional event_time and created_time fields
in the JsonLogRecord, without internally duplicating those fields.
refs: https://github.com/KumoCorp/kumomta/issues/405
Adjusts how we resolve the attachment name when processing mime
attachment parameters:
* First check Content-Disposition::filename
* Then check Content-Type::name
De-couple parsing out the `Content-Id` header from having
`Content-Disposition`.
In the lua bindings; remove the synthesized filename fallback;
we now just leave it nil to match the rust logic.
This commit adds some plumbing to facilitate generation of RFC 3464
non-delivery reports and some corresponding glue to enable calling
it from lua.
Examples in the docs that are also added.
It was possible to deadlock your policy by triggering queue
length/emptiness checks conurrently with the queue consumer thread.
Let's make it more explicitly clear which methods cannot be used that
way by making their mutex acquisition non-blocking and raise an error.
In addition, let's improve the close method so that it doesn't require
exclusive access to the consumer side of the queue.
Provide a shutdown_logging event that can be used to explicitly close
queues on shutdown, if that is appropriate for your use case.
This enables configuring mTLS support in the listener.
Adjust TLS client tests to use this on the sink side and verify
that mTLS works for both inbound and outbound SMTP.
refs: #100
refs: #391
This commit extracts the TLS version, cipher and subject name from
the TLS state and captures it:
* In the trace headers
* In the connection metadata
* In the Reception log record
In order to capture the info in trace headers, this commit will
now change the reception protocol to ESMTPS (for SSL) or ESMTPSA (for
authenticated SSL), and emit the version and cipher information as
a comment like: `with ESMTPS (TLSv1_3:TLS13_AES_256_GCM_SHA384)`.
closes: #100
The read_dir and glob functions have been logically moved into that new
namespace, leaving deprecated versions of them in the `kumo` module.
A new `kumo.fs.open` function that works similarly to `io.open` is
provided. This function cooperates with the kumo async io scheduler
and won't block it if the filesystem is under pressure.
It returns file handles that are simlar to the builtin lua file handle
objects, but do not support formatting or parsing of writes or reads
respectively: the calling code is responsible for that. The rationale
for this difference is that is that lua's semantics for those functions
are frankly a bit weird and are hard to replicate precisely.
This function allows defining a non-durable, non-persistent, in-memory
queue that is intended to be used as glue when building up more advanced
processing flows within kumomta.
The intent is that the queue is processed from within a task spawned via
`spawn_task`.
Wez Furlong
Dario Maiocchi
Tom Mairs
ncai
kay ozaki
smsvip
Pankaj Rathi