Surface the underlying DKIM signature tags and the published DMARC
policy tags as auth-result props, so callers (and downstream
Authentication-Results headers) can see what was actually checked.
DKIM (crates/dkim):
* Factor populate_props() over the parsed tagged-header and emit
header.d, header.i, header.a, header.s, header.c, header.t and
header.x. The previous code only emitted d/i/a/s on the success
path.
* On DKIMHeader::parse failure, fall back to a generic TaggedHeader
parse so the resulting permerror AuthenticationResult still carries
whatever tags were extractable (e.g. for expired signatures we now
surface header.d and header.x). Covered by a new roundtrip test.
DMARC (crates/kumo-dmarc, crates/kumod):
* Record now retains the raw key=value tags it parsed, exposed via
Record::tags().
* DispositionWithContext carries a new props map. Record::evaluate
initialises it empty; drop the no-longer-relevant ToXml derive.
* DmarcContext::check copies the matched record's tags into the
result props as policy.<tag> via a new policy_tags() helper.
* kumod's dmarc.rs threads result.props through into the
AuthenticationResult returned to Lua (rather than starting empty),
preserving the existing policy.published-domain-policy insertion
for Quarantine/Reject.
This is ostensibly "just" making the nom parsing functions return
BString instead of String, but the fan-out makes the commit much larger,
as a number of surfaces (eg: as_unstructured) have now changed types.
Removes the deref to str as this is now a fallible operation that
can panic.
Process the fanout of this change, which is dealing with more
possibility that things might be binary in more places.
There are some transitional APIs introduced as deprecated so that I'm
nagged to remove them already: I want to get the bulk of the code aware
of the possibility of binary sorted before I tackle adjusting the
message parser for that, so that that particular commit is more
self-contained.
It was pointed out that we had this policy and that it wasn't part of
the spec.
That behavior was imported along with the crate when we forked it from
the cloudflare implementation.
Let's remove it; folks that need it can use a couple of lines of lua to
recreate it if they need it.
It was reported that AWS SES could mark the DKIM signature as failed
when using a specific combination of headers in the header list,
with specific lengths of the other fields.
The reason for this is that we had two passes of wrapping applied
to the header, and they might not agree on the formatting of
the header.
The solution is to remove the second pass and just take a bit more
care to emit the header in a wrapped form in the first instance,
that way there can be no discrepancy or conflict.
closes: https://github.com/KumoCorp/kumomta/pull/483
This commit does some plumbing work to allow defining alternative
resolvers with different names that can be used to query different
upstream DNS servers.
This is most useful when implement DNSBLs where you might have a custom
DNS server configured with one or more RBL zones that is reserved purely
for RBL lookups.
The plumbing introduces a more regular syntax for defining one of the
various implementations of the Resolver trait, as well as a new
Aggregate resolver impl that can query across multiple Resolvers until a
query is satisfied.
This allows some interesting and powerful configurations, such as
loading a static zone file into memory to query it directly, and/or
blending that together with querying either the system or some other
upstream DNS server as a fallback.
The various lookup functions (except for MX!) have been updated to
accept an optional alternate resolver name, so that they work together
with the above.
A new rbl_lookup function is also provided as a convenience for querying
the most common form of RBLS.
ptr_host and reverse_ip are two string utility functions that are likely
not going to be widely used, but are very convenient to have when you
do have a usecase that requires it!
closes: https://github.com/KumoCorp/kumomta/issues/269
This was probably the casualty of some earlier refactoring
that has gone unnoticed until now.
Since we don't have explicit context on which key format to
parse in this helper function that is used in multiple places,
let's just make it try to parse both rsa and ed25519.
closes: #368
While auditing Answer::as_txt usage as a follow up from the recent
SPF fix, I noticed a TODO in the dkim code (which we forked from
another implementation) to support processing multiple TXT
records.
This commit implements the necessary tweaks to extract multiple
signatures and attempt to verify them against the incoming message.
I've been having trouble with moka's eviction policy leading
to some unwanted duplicate work, especially when the system
is under load and the latency is higher.
This commit swaps it out in favor of a relatively simple
implementation on top of dashmap; that gives us a decent
concurrent base and better respects the concurrency
limit and LRU around the contentious insertion case.
For LRU, since dashmap is a concurrent data structure, it
is ~impossible to use the classic doubly linked LRU approach
safely.
The strategy used here is a proabalistic sampling LRU using a technique
similar to that used in redis.
We use atomics to tag read and write entries with a monotonic counter.
Entries with smaller counter numbers are least-recent than entries with
a larger number.
During eviction we take a random sample of 10 entries from the cache
map, remove any that have expired due to TTL, and if we still need space
after that, we'll pick up to half of those to evict. That guarantees
that we don't pick the most recent entry of the sample, and on aggregate
this should be a reasonable approximation of the "true" LRU.
In the background, every 30s, a maintenance task will remove expired
entries from the caches. This is primarily to reduce memory utilization
when the cache is otherwise idle. When a cache is hot, expiration is
performed as part of the eviction logic described above.
The signer cache maps the signer parameters to a pre-made
signing context.
It is essentially a map of (domain, key) -> signer.
Assuming that the (domain,key) tuple is unique, then this is
a good, effective use of that cache.
However, if multiple domains can share the same key then we
can end up re-parsing the same key data for each of them,
which is moderately expensive and a waste of CPU.
This commit introduces an additional cache for the key source to
the resultant compiled key.
This allows sharing of the same compiled key across signing
parameters that otherwise vary, and should help to shave off
some latency.
We keep getting asked about
https://rustsec.org/advisories/RUSTSEC-2023-0071.html and how it impacts
kumomta.
The answer to that question is: in the default build configuration, we
use openssl's RSA signing implementation rather than that of the rsa
crate. The reason for this is that OpenSSL's RSA implementation is due
to the performance gap between the two implementations
(https://github.com/RustCrypto/RSA/issues/339). The result of this is
that the problematic code and attack vector described in the security
advisory does not apply to KumoMTA, because it is not used to compute
any signatures.
In the interest of not raising any false alarms as more and more people
perform security analyses on kumomta, this commit removes the `rsa`
crate from the build graph. In order to do so, we need to port
verification over to the openssl RSA implementation which is what this
commit does.
I look forward to a future version of the `rsa` crate being published
that has this issue resolved, and that closes the performance gap!
refs: https://github.com/RustCrypto/RSA/issues/390
dependabot wanted to upgrade to base64 0.21, but I resent that
the API for the common case of base64 encoding got more difficult
in the move from 0.13.
So, I'm switching us to data_encoding's implementation instead.
Note that the overall rust ecosystem on which we depend here,
transitively depends on 3 different versions of the base64 crate, so
even though we've removed direct deps on base64 from kumomta, the lock
file still references base64 0.10, 0.13 and 0.21, so that kinda sucks.
My reading of the spec says that we can swap `:` for `:FWS` when
wrapping, so let's try that.
Previously, we'd wrap on the hyphens in header names and that
seems to cause some problems for some verifiers.
Replaces the dkim result types from this crate with
AuthenticationResults which contain the additional
data required to construct high-quality Authentication-Result
headers.
refs: https://github.com/KumoCorp/kumomta/issues/82
kay ozaki
Wez Furlong
Aditya Ganti
Sophia Turner
Dirkjan Ochtman