The sampled region needs to be clamped to fit the length of the string;
we were previously assuming that there could be 8 additional bytes after
the error position, which is not guaranteed.
We've been hoping that mkdocs-material will ship the much anticipated
search enhancements for some time, but it's time to recognize that
we need to do something to improve the search results with how
things work right now.
This is a big commit that changes the titles of the various pages
from the code-annotated synopsis to just the name of the function.
This makes it much easier now to match things like `kumo.reject`
directly, but `reject` remains awkward to find.
I think this is the best that we can do at this time.
A few functions have been annotated with the `status: deprecated` to
show as deprecated in the toc/nav (shows with a little trash can next
to the name).
Some sites are non-conforming in their behavior when they have a policy
that issues a transient failure and then snips the connection without
explicitly issuing a 421.
This can cause "splash damage" where a message is transiently failed
and the connection snipped "disgracefully" and a subsequent message
that is due to go out on that same established connection encounters
a transport error when the RSET part of the pipeline is sent.
This commit provides methods on the ClientError and Response types
to help reason about whether the response might be specifically
about the message being delivered, vs. more of a transport error.
The gist of this check is that timeouts and transport level stuff
where there is no protocol level response are not considered to
be due to the message. Protocol level responses prior to MAIL FROM
are similarly considered to be more like transport errors.
The functional change in this commit is:
* If we get a transport-ish error and we have more candidate hosts
available in the connection plan, then after logging a
TransientFailure for the error we just encountered, we'll push
back the message to be tried again on the next connection that
we open in the current session.
* For timeout errors specifically, this behavior is gated behind
a try_next_host_on_timeout boolean option.
The rfc includes a bogus date in one of its examples, which the
underlying chrono library produces an error for.
Since we don't loudly report failures in parsing incoming reports,
this sort of error is confusing for the user.
These date fields are optional, so we can convert the error to
None without harming the overall outcome.
It is common to want to express that incoming reports should
be logged, but not relayed.
This commit facilitates that by expanding `log_arf` and `log_oob`
to allow more than just a simple boolean value.
We now support both LogThenRelay and LogThenDrop as dispositions
for an incoming report, as well as Ignore to not care about the
report content.
For the LogThenDrop case we'll log only the incoming OOB or Feedback
record to the logs and then discard the incoming message.
For LogThenRelay we'll log the incoming OOB/Feedback report as well
as enqueue the message and log a Reception and, subsequently, records
for its delivery attempts.
The previously supported boolean values are mapped to `Ignore` (false)
and `LogThenRelay` (true), respectively: their behavior is the same as
in prior releases.
The integration test and smtp server tracing infra has been enhanced a
bit to support extracting these new disposition states from the tracer
during the integration test.
When using throttles with a long period, a porting issue from the JS
based throttle code meant that we'd compute a TTL in *seconds* but tell
redis that its value is in *milliseconds*, resulting in a much shorter
expiration time than desired, allowing more messages to pass the
throttling condition.
This issue doesn't apply to the redis-cell based throttle
implementation.
The fix here is simply to switch the `PX` (TTL in milliseconds) to
`EX` (TTL in seconds).
Possibly this was the source of the flakeyness that we could never
run down in https://github.com/KumoCorp/kumomta/issues/297
TL;DR: you can easily halve your system performance by logging headers
vs. logging meta.
This is one of those things that is easy to overlook or forget,
but: whenever you need to operate on the message data, rather
than its metadata, the aggregate cost is high.
In this case, we were recently troubleshooting a system where
the CPU was bogged down and we traced it to the logging configuration: a
number of message headers were being logged in a configuration that
made heavy use of throttles and limits in its traffic shaping, and
thus had a large number of Delayed and TransientFailure events being
written to the logs.
When logging headers, each one of those events requires loading
the message from the spool and parsing out the headers. When the
average message size is ~100KB this imposes a notable overhead
on the CPU and IO utilization of the system.
What we recommend instead of logging headers directly is capturing
the information that you want to log into the message metadata
at the time that the message is received.
The message meta is usually already loaded, but is also typically
much smaller and easier to decode than the full message content
in the cases where it is not loaded.
As a result, it is much cheaper to log meta than to log headers.
This commit adds some warnings and cross links to help folks
be aware of this, and to generally navigate related meta and logging
topics more easily via tags.
Tom Mairs
Mike Hillyer
Wez Furlong
Mike HillyerandWez Furlong