I believe this to be a relatively recent regression since we started
to preserve the incoming domain name in RCPT TO, rather than
normalizing it. (2026.04.09-ea3b2a9b)
refs: https://github.com/KumoCorp/kumomta/issues/533
Two main cases:
* If the user accidentally places files in the directory that
are not kumo-jsonl compatible segments, then we won't trip
over them and stop processing.
* If the log segments are incomplete (eg: kumod was SIGKILL'd)
then we log the issue and avance to the next segment
Some upstream peers (e.g. QQ Mail's rate-limiter) silently hold a
proxied TCP connection open indefinitely — sending no data, FIN, or RST
— rather than cleanly refusing. Without some kind of timeout
management, the two file descriptors for such a session remain open for
the lifetime of the process, slowly exhausting the kernel's
file-descriptor table and occupying proxy-server worker slots.
This commit configures kernel level keepalive options with reasonable
defaults to detect and close out this class of connection.
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Closes: https://github.com/KumoCorp/kumomta/pull/509
import_headers takes an array of per-spec option tables, each describing
how a single header name or pattern should be imported into the message
metadata. Compared to import_x_headers it adds:
* Trailing-`*` wildcard patterns (e.g. `X-*`) alongside exact names.
Bare/leading/interior wildcards are rejected at compile time.
* `match` of `first`, `last` (default), or `all`. `all` captures every
matching header instance as an array of strings; the others capture
a string. Specs that produce no matches write nothing.
* `transform` selects the metadata key style: `snake_case` (default,
matches the existing import_x_headers behavior), `kebab_case`,
`camel_case`, or `pascal_case`. Header matching itself is always
case-insensitive.
* `target` overrides the metadata key for exact-name specs.
* `remove` strips the matched headers from the message body in a
single follow-up pass.
When more than one spec could match a header, the first matching spec
wins, so callers can place specific rules ahead of a wildcard catch-all.
import_x_headers now delegates to import_headers, so its behavior is
unchanged and the two share a single implementation.
retain_headers now passes the header index alongside the &Header to its
closure, which import_headers uses for its post-pass removal step
instead of tracking a parallel counter. Existing callers that don't
need the index ignore it with `_`.
Closes: #515
This was previously just returning everything before the final at-sign.
Now we use the same parsing approach and return the normalized local
part.
This is technically a breaking change, but it is minor and improves
the overall state of things.
This got lost during merge conflict resolution, as well as
incorrectly indicated that it applied to rustls in the original
draft, when it was in fact only applicable to openssl.
Pre-populate a `to_header` template substitution with the default
formatted `To` header for each recipient. Users can reference it via
`{{ to_header }}` and override it per-recipient in substitutions.
closes: https://github.com/KumoCorp/kumomta/pull/501
This commit refactors the EnvelopeAddress types(!) so that the brains
of them are centralized in the rfc5321 crate, removes the one from
the log-types crate, and replaces the internals of the one in the
message crate with the rfc5321 implementation.
This revised implementation accommodates quoted local parts more
consistently and exposes the local part via a normalizing accessor which
is in turn used when comparing addresses for equality.
This means that `"foo"@` and `foo@` now compare the same, and if you
have setup a maildir that generates per-user maildirs, we'll use the
normalized form of the local part rather than whatever is on the
incoming address.
Another side effect of this commit is that we can now accept exotic
quoted addresses like `"info@"@example.com` without falsely complaining
about having too many `@` signs in the address.
closes: https://github.com/KumoCorp/kumomta/issues/495
We were propagating the parse error as a server error and mapping it
to the 421 technical difficulties response instead of returning
the appropriate syntax error response defined by SMTP.
refs: https://github.com/KumoCorp/kumomta/issues/495
These are more immediately useful to me for writing tests, but are
generally useful when you have a policy that might need to to deal
with legacy encoding schemes more directly.
It was pointed out that we had this policy and that it wasn't part of
the spec.
That behavior was imported along with the crate when we forked it from
the cloudflare implementation.
Let's remove it; folks that need it can use a couple of lines of lua to
recreate it if they need it.
This is a follow on from the previous commit; apparently some sites
will enclose addresses in <> even though this is not covered in the
RFC.
Let's cut the address parsing over to the SMTP parser rather than
the mail header parser, as they have rather different semantics
and both ARF and OOB are SMTP-centric.
We had a report that the enclosing <> were being passed through
to the JsonLogRecord in some log hook flows, which meant that
we'd try to parse them as EnvelopeAddresses and fail.
This dumps out a trace of all tokio tasks. It is quite expensive,
and currently unsettles the tokio runtime such that you need to
repeatedly call this endpoint in order for a subsequent graceful
shutdown to clock through and complete.
In our thundering herd protection we intentionally limit the number of
pending lookups for a given key to just 1. That is controlled via
a semaphore.
We have pre-existing logic that checks to see if the semaphore has been
closed (eg: by a task being cancelled), but despite this, we do
occasionally have reports from users with `timed out after 120s on
semaphore acquire while waiting for cache to populate` for the shaping
data cache.
This typically correlates with eg: long IO waits due to eg: a spike in
transient failures and usually some sort of logging of headers, but not
always.
The not always situation bothers me as we don't currently have a good
explanation of what might be causing the excessive delay.
This commit adds an additional sanity check: when a new lookup is
initiated, if the semaphore is still open (not cancelled), but has been
open for longer than the populate timeout, then we treat this similarly
to the semaphore being closed: we'll create a new semaphore, reset the
expiration and the caller will typically then promote itself to the task
that will satisfy the lookup.
The side effect of this all waiters should become unblocked and awake
either with an error status or with the result of the new lookup, which
should help to clear any persistent/recurring blocking state associated
with this.
This change doesn't provide more insight (I have something in mind for a
follow-on commit for that), but should help make forward progress.
Mike Hillyer
Wez Furlong
Harsh Jha
Aditya Ganti
kay ozaki
Oleg Orekhov
Jack
Raphael Sprenger