We had a sponsor report an issue where we'd log an error during
processing a log hook. It transpired that the recipient list we were
attempting to log was `["metadata is not loaded"]`, which doesn't parse
as a valid EnvelopeAddress (see prior commit:
52e2f95e9f)
This commit fixes up the recipient_list logic in the logging function to
avoid taking the error message as the recipient list, and moves the
metadata loading to happen just before we try to read the recipient
list.
This should work around the issue, but a consequence is that we might
be holding on to the metadata for that message in RAM for longer than
we should; will need to do some thinking about managing the resident
status in this situation better.
This aggregates the various authentication-results producing auth checks
into a more convenient function.
Part of this change improves some plumbing in the dkim checking (we no
longer raise an error for a missing From, but instead indicate a failed
dkim result), and we now support passing down a resolver name to the
various checking functions, to facilitate testing and other more
advanced use cases.
refs: https://github.com/KumoCorp/kumomta/issues/16
refs: https://github.com/KumoCorp/kumomta/issues/84
This commit adds some plumbing to allow us to collect metadata
about the caches and spit it out into a json file that we can
then use in the docs to show the predefined caches, capacities
and brief comments about their purpose.
We put that info on the set_lruttl_cache_capacity doc page.
Augment the docs to clarify that msg:append_header and
msg:prepend_header do no modification of the value by default.
Adjust the wrapping used by Header:new_unstructured so that it
hard wraps before the SMTP line length limit.
Add optional encode parameter to msg:append_header and
msg:prepend_header so that those methods can opt-in to the
Header:new_unstructured formatting.
Add HeaderMap:append method to mirror the existing HeaderMap:prepend
method.
Add unit tests that demonstrate the effect of wrapping/encoding
for both the "dumb" msg:append_header and msg:prepend_header functions,
and the "smarter" HeaderMap:prepend and HeaderMap:append methods that
always encode their parameters.
Update the docs for the headermap flavor of these methods to indicate
that they will always encode the value.
This allows a (more or less) arbitrary lua function to be used to load
data.
The primary advantage of this is that the size of the resulting
KeySource struct is smaller than it would be to hold the actual data
inline. It also enables the surrounding code to be slightly better
factored.
Add CBC and ecb Block mode aes encryption/decryption.
To facilitate testing this better, and just to be nicer overall, adjust
data-loader's KeySource::key_data variant to support loading raw binary
bytes.
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Closes: https://github.com/KumoCorp/kumomta/pull/395
Add a convenience function that is intended to be used around the
time that you might be building up the overall set of
Authentication-Results.
Populate missing smtp.mailfrom in SPF authentication result.
This is incremented each time we log a Rejection record from the smtp
server.
The service label identifies the listener address, and there is a
special `total` that is incremented as a rollup across all listeners.
```
# HELP smtp_server_rejections number of Rejection records logged by the smtp server
# TYPE smtp_server_rejections counter
smtp_server_rejections{service="127.0.0.1:2025"} 1
smtp_server_rejections{service="total"} 1
```
closes: https://github.com/KumoCorp/kumomta/issues/429
Let's base our quoting decisions based on whether the string is fully
atext compatible, and once quoting, on whether the character requires
quoting within the quoted string.
refs: https://github.com/KumoCorp/kumomta/issues/428
Partially reverts aac842439f and fixes the
underlying issue that led me to throw that together: the issue was that
the check_line_lengths function needed to add 2 to the index otherwise
iterations after the first would over-estimate the length of the
currently found line by 2, because the prior index included the CRLF
that it found in the input.
While poking at this, I found that there was a fairly harmless edge case
around non-DATA line handling: we used a 1024 stack buffer to read the
command in, and would accept anything that fit into that as valid, even
though the limit is set to a slightly smaller value.
We were including the CRLF in the measurement that we were making,
while the default value was the value from the spec excluding the CRLF.
Let's increase the default value to 1000 to make things match up.
I saw this flake out on the last push. This commit adjusts the
expectations to allow 10% of slack either side of the theoretical
50% target specified by this test.
* Add exp= test based on an example in the rfc
* Discovered that we were a bit too strict and didn't allow spaces
as shown in that example
* Add integration test for the ehlo_domain string quoting issue
resolved in 236afa417f
* lua spf check: Assume ehlo_domain = domain when sender is none
When populating ehlo_domain in particular, we were using the json
Value::to_string() impl which produces a json-serialized string
which includes quotes.
This is a bit of a foot-gun, so let's add a get_meta_string method
to ConnectionMetaData to parallel the equivalent
Message::get_meta_string method.
This commit does some plumbing work to allow defining alternative
resolvers with different names that can be used to query different
upstream DNS servers.
This is most useful when implement DNSBLs where you might have a custom
DNS server configured with one or more RBL zones that is reserved purely
for RBL lookups.
The plumbing introduces a more regular syntax for defining one of the
various implementations of the Resolver trait, as well as a new
Aggregate resolver impl that can query across multiple Resolvers until a
query is satisfied.
This allows some interesting and powerful configurations, such as
loading a static zone file into memory to query it directly, and/or
blending that together with querying either the system or some other
upstream DNS server as a fallback.
The various lookup functions (except for MX!) have been updated to
accept an optional alternate resolver name, so that they work together
with the above.
A new rbl_lookup function is also provided as a convenience for querying
the most common form of RBLS.
ptr_host and reverse_ip are two string utility functions that are likely
not going to be widely used, but are very convenient to have when you
do have a usecase that requires it!
closes: https://github.com/KumoCorp/kumomta/issues/269
In 34622edbd2 we made a similar change to
enable querying domain names with underscores. Let's preemptively make
that same change to all places where we construct a Name.
No spf record should use the set of search domains in a local resolver;
all names are implicitly FQDN. Let's explicitly mark up the names that
way so that we can avoid both potentially matching a surprising local
record and also excess DNS lookups for records that might not exist.
I noticed a test failure where we hadn't yet received all of the
trace events that we wanted to inspect, so this commit introduces
the ability to wait for matching records to appear in the collected
set of data.
Similar to the general purpose resolve function, we should map
NoRecordsFound to empty list(s) of results in the specialized
ip, mx and ptr lookup functions, otherwise we can cause rule
evaluation to terminate too soon.
Wez Furlong
crx
Dario MaiocchiandWez Furlong