1529 Commits
Author SHA1 Message Date
Wez Furlong dfb17abd0d dane: engage for secure CNAME into unsigned zone
Per RFC 7672 section 2.2.2, an MX host that is a securely published
CNAME remains DANE-eligible at its original name even when the alias
target lands in an unsigned zone: it is the secure TLSA RRset, not the
address records, that authenticates the peer. When the address chain is
insecure but MX selection was secure, an explicit CNAME query isolates
the alias's own DNSSEC status; a secure alias engages DANE, an
indeterminate status defers for downgrade resistance.

refs: https://github.com/KumoCorp/kumomta/pull/545#discussion_r3472353021
2026-07-06 11:46:51 +01:00
Wez Furlong 4fc3cbca92 add treat_mx_list_as_secure to SmtpProtocol
This allows you to explicit indicate that an mx_list should
be treated as DNSSEC secure.

We cannot simply assume that lua code setting mx_list be treated
as secure, because the lua code may be simply passing through
the result of some other DNS lookup--we need an explicit way
to thread through the overall secure flag in order for the
secure property to be correctly upheld.
2026-07-06 11:46:51 +01:00
Wez Furlong e13734d186 address review feedback on https://github.com/KumoCorp/kumomta/pull/545
Address most of the easy points from the review, others will land
separately.
2026-07-06 11:46:51 +01:00
Wez Furlong 3b61117145 dns: add RFC 5011 managed trust anchor option for unbound
trust_anchor_file now accepts `{ managed = "<path>" }` for an RFC 5011
auto-maintained anchor file (unbound only; hickory errors). Unbound seeds
the file from the bundled root anchors when the file is absent.
2026-07-06 11:46:51 +01:00
Wez Furlong 1874dc09f7 enable DNSSEC when using hickory, allows enable_dane w/o unbound
When we first built out DANE, the hickory implementation was
all or nothing when DNSSEC validation was enabled, making it unsuitable
for use in an MTA.

Since that time hickory has matured; this commit wires up the
DNSSEC signals and that enables its use with DANE.

We do still require openssl for establishing the connection when
DANE is enabled; that's a different constraint.
2026-07-06 11:46:50 +01:00
Wez Furlong 7331cb4ec7 new allow_smtp_auth_plain_without_valid_certificate option
This is technically a breaking change, but in a good way.
We will no longer send SMTP AUTH PLAIN creds when we didn't
validate the peer certificate.  This is what you want in
all production deployments.  The new option facilitates
test setups or other unusual configurations where the certs
cannot be verified but you have contrived to trust the
remote system through some other means.
2026-07-06 11:46:50 +01:00
Wez Furlong a4c4b40fbe add dane_result_count metric
This surfaces aggregate dane results
2026-07-06 11:46:50 +01:00
Wez Furlong d920ffb5c4 fix: DANE downgrade resistance
We weren't checking the DNSSEC status for A/AAAA when considering
DANE, and were overly permissive in a couple of related cases.

This diff ended up fairly large because the bulk of it is to
facilitate testing.

refs: https://github.com/KumoCorp/kumomta/issues/543
refs: https://github.com/KumoCorp/kumomta/pull/517
2026-07-06 11:46:49 +01:00
Wez Furlong 3f11963acd docs: fix broken link 2026-06-27 18:58:12 +01:00
Wez Furlong 407379dd36 track sched->readyq fan in, clean up some internals
This commit removes the queue_name_for_config_change_purposes_only wart
from inside the readyq machinery.  This field had a bunch of caveats
and could in some cases become stale if the originating scheduled
queue had aged out.

It's been replaced by a HashSet of Weak refs back to the associated
scheduled queues so that it is now possible to walk the reverse of
the fan-in graph when we need to.
2026-06-25 13:41:01 +01:00
Mike Hillyer db31223667 Add changelog entry for community shaping file pull request. 2026-06-23 17:24:54 -04:00
Wez Furlong cf5db1a796 fixup doc build 2026-06-23 15:54:06 +01:00
Wez Furlong 4d576922f8 deps: hickory 0.25 -> 0.26
This is fairly big diff because a number of hickory types moved around
or changed signature.

In particular, the config structs changed in a way that would fan out
and require attention by our users.  So the approach I opted to take
here was to define our own types that look like the hickory 0.25
shape and then we have explicit logic to map those to hickory and
also to our unbound resolver crate.

This commit also bumps up the unbound resolver crate because it also
uses hickory's types.  I took the opportunity to upgrade the version
of the embedded unbound resolver as part of that work.
2026-06-23 15:47:08 +01:00
Wez Furlong a1a7e16759 docs: fixup links 2026-06-23 10:13:11 +01:00
Wez Furlong 2a657f68fb included scheduled queue config in resolve-egress-path output
and also in resolve-shaping-domain, for consistency.

We can use the message rate from the scheduled queue config to refine
the computed ceilings/constraints too, which is nice.
2026-06-23 10:00:07 +01:00
Wez Furlong 49e1a62733 add resolve-egress-path API endpoint
This is conceptually equivalent to resolve-shaping-domain but operates
on a live instance, rather than loading up the policy offline.
2026-06-23 10:00:05 +01:00
Wez Furlong 61bb95a558 resolve-shaping-domain: expand output
This changes the output to be a bit more human friendly by default
by rendering as toml.

It also includes the rate ceilings added in the prior commit.
2026-06-23 09:59:32 +01:00
Wez Furlong 0a72ce7f02 introduce EgressPathConfigConstraints concept
This can be computed for an EgressPathConfig and is useful to indicate
what the effective ceilings are for this egress path.

We return this from inspect-ready-q
2026-06-23 09:58:38 +01:00
Wez Furlong b9e677dd16 add abort-ready-q-conn and inspect-readyq endpoints/commands
These build on the infrastructure from the prior commit and improve
the observability of the system.

refs: https://github.com/KumoCorp/kumomta/issues/539
2026-06-23 09:58:36 +01:00
Wez Furlong 423abc4704 add per-dispatcher state tracking and watchdog
The ready queue maintainer now tracks for each dispatcher whether it
is making progress.  If it exceeds dispatcher_progress_watchdog_timeout
then the task will be aborted and any message(s) it held onto will
be returned to the scheduled queue.  This will free up the connection
slot and allow another connection to be attempted, potentially
making further progress.

refs: https://github.com/KumoCorp/kumomta/issues/539
2026-06-23 09:56:45 +01:00
Wez Furlong 002d6593f0 new: suspend_when_unplumbed and suspend_when_proxy_unhealthy source options
Egress sources can now be configured to auto-suspend when their local
bind address appears unplumbed or when their configured proxy server
appears unreachable.  A suspended source is skipped during pool selection
until the configured duration elapses.
2026-06-23 09:22:27 +01:00
Wez Furlong e21bb1c672 ha_proxy_server and socks5_proxy_server now also resolve host names
We'll expand these to a list of candidate addresses, trying each in
turn.  We use the same overall connect timeout regardless of how
many candidates are present, to respect that timeout bound.

closes: https://github.com/KumoCorp/kumomta/pull/402
2026-06-23 09:11:53 +01:00
Wez Furlong 9987248313 spool: surface and handle rocksdb background errors
We recently investigated an issue where a rocksdb had been damaged by
corrupting/removing SST files (it sounded like this was accidentally
self-inflicted by some backup/orchestration infrastructure) leaving the
system in a silently-broken state: writes just wouldn't make progress
and there were no error messages.

Inspecting the `/var/spool/kumomta/data/LOG` log file (which is a
readable text file) revealed messages like:

```
2026/06/12-14:55:31.884227 2875746 [ERROR] [db/compaction/compaction.cc:262] Unable to load table properties for file 29704 --- IO error: No such file or directory: While open a file for random read: /var/spool/kumomta/data/029704.sst: No such file or directory
2026/06/12-14:55:31.884311 2875746 [ERROR] [db/db_impl/db_impl_compaction_flush.cc:3385] Waiting after background compaction error: IO error: No such file or directory: While open a file for random read: /var/spool/kumomta/data/029704.sst: No such file or directory, Accumulated background error counts: 6363
```

This commit improves the observability in this situation by proactively
checking for error conditions:

1. The store() and remove() operations now use our own polling within
   a deadline loop rather than spawning a blocking task and delegating
   to rocksdb's blocking interface.  This allows us to inspect the
   background error count and be cancellable, safely respecting and
   caller provided smtp max transaction duration.

2. All read and write operations check for IO and Corruption errors
   and immediately latch an error state

3. The metrics monitoring task inspects and track background error
   counts and latch us into an unhealthy state when the background
   error state appears unhealthy and persistent.

4. Additional metrics are exposed to help monitoring and alerting

While adding integration test coverage for this, I found a typo that
meant that spool errors were ignored in the message crate; they got
silently converted to `true` in all cases rather than just mapping
the success case to a `true`.

Integration tests handle the case where an SST file is corrupted
(truncated) during runtime, as well as starting up when an SST file
is missing.   These excercise both the foreground and background
error detection paths.
2026-06-23 09:11:48 +01:00
Wez Furlong 9c8183a433 docs: update for release 2026.06.23-f3af1cd0 2026-06-23 09:02:10 +01:00
Wez Furlong d88fee2e0d docs: manually run cargo run -p jsonschematodocs
This somehow got missed in an earlier commit
2026-06-17 22:52:45 +01:00
Mike Hillyer d3d4a4d7c2 Final shuffle and updates prior to publish. 2026-06-11 15:25:53 -04:00
Mike Hillyer 1999ba0251 Continue traffic shaping chapter refactor. 2026-06-09 13:23:56 -04:00
Mike Hillyer 164f109326 Traffic Shaping Refactor First Pass 2026-06-09 13:23:56 -04:00
Wez Furlong 4eedf9361e fix punycode normalization for STARTTLS
I believe this to be a relatively recent regression since we started
to preserve the incoming domain name in RCPT TO, rather than
normalizing it. (2026.04.09-ea3b2a9b)

refs: https://github.com/KumoCorp/kumomta/issues/533
2026-05-30 17:35:28 +01:00
Harsh JhaandWez Furlong 00f3131f7d Allow configuring which records TSA skips
Closes: https://github.com/KumoCorp/kumomta/pull/525

Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2026-05-29 15:02:31 +01:00
Aditya GantiandWez Furlong 7bfbbe720a fix(mod-aws-sigv4): apply Trimall, require host header, fix S3 content-sha256
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Closes: https://github.com/KumoCorp/kumomta/pull/522
2026-05-29 10:16:06 +01:00
Wez Furlong 5fb9dd104d fix regression in serializing original_message in ARF
This got overlooked around the time that 7518ca6de was pushed.
The intent was to serialize as a string when possible, bytes otherwise.
2026-05-29 09:45:22 +01:00
Wez Furlong 06a26f4ba6 kumo-jsonl: make tailer more tolerant of errors
Two main cases:

 * If the user accidentally places files in the directory that
   are not kumo-jsonl compatible segments, then we won't trip
   over them and stop processing.

 * If the log segments are incomplete (eg: kumod was SIGKILL'd)
   then we log the issue and avance to the next segment
2026-05-29 08:57:47 +01:00
Wez Furlong 5a7aa96cf1 docs: update for 2026.05.12-a6845223 release 2026-05-27 06:47:55 +01:00
Wez Furlong 62de291279 docs: suggest folks prefix their metadata names
Should help to avoid possible conflicts with the core metadata names.
2026-05-12 14:35:16 +01:00
Harsh JhaandWez Furlong 7cba56bc06 http injection: allow static per-recipient metadata
That metadata is accessible in to via msg:get_meta('extra')

closes: https://github.com/KumoCorp/kumomta/pull/516
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2026-05-12 14:25:46 +01:00
kay ozakiandWez Furlong 1febfcc55e lua counter series using kumo-counter-series
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
closes: https://github.com/KumoCorp/kumomta/pull/507
2026-05-07 15:35:17 +01:00
Wez Furlong 2c0e9982e1 docs: fmt examples in import_headers.md 2026-05-07 14:25:34 +01:00
Wez Furlong 421bff1a41 docs: changelog for #504 2026-05-07 14:21:01 +01:00
Oleg Orekhov c237e16c5f mod-http: allow disabling certificate verification on the client
closes: https://github.com/KumoCorp/kumomta/pull/504
2026-05-07 14:18:09 +01:00
Wez Furlong 421b8a289e proxy: keepalive will work on macos and other unix too
So remove the linux only gate
2026-05-06 22:38:50 +01:00
JackandWez Furlong c4dbaa743b proxy-server: add tcp_keepalive option
Some upstream peers (e.g. QQ Mail's rate-limiter) silently hold a
proxied TCP connection open indefinitely — sending no data, FIN, or RST
— rather than cleanly refusing.  Without some kind of timeout
management, the two file descriptors for such a session remain open for
the lifetime of the process, slowly exhausting the kernel's
file-descriptor table and occupying proxy-server worker slots.

This commit configures kernel level keepalive options with reasonable
defaults to detect and close out this class of connection.

Co-authored-by: Wez Furlong <wez@wezfurlong.org>

Closes: https://github.com/KumoCorp/kumomta/pull/509
2026-05-06 21:09:10 +01:00
Wez Furlong ee1e5b980e message: add msg:import_headers, a flexible alternative to import_x_headers
import_headers takes an array of per-spec option tables, each describing
how a single header name or pattern should be imported into the message
metadata. Compared to import_x_headers it adds:

 * Trailing-`*` wildcard patterns (e.g. `X-*`) alongside exact names.
   Bare/leading/interior wildcards are rejected at compile time.
 * `match` of `first`, `last` (default), or `all`. `all` captures every
   matching header instance as an array of strings; the others capture
   a string. Specs that produce no matches write nothing.
 * `transform` selects the metadata key style: `snake_case` (default,
   matches the existing import_x_headers behavior), `kebab_case`,
   `camel_case`, or `pascal_case`. Header matching itself is always
   case-insensitive.
 * `target` overrides the metadata key for exact-name specs.
 * `remove` strips the matched headers from the message body in a
   single follow-up pass.

When more than one spec could match a header, the first matching spec
wins, so callers can place specific rules ahead of a wildcard catch-all.

import_x_headers now delegates to import_headers, so its behavior is
unchanged and the two share a single implementation.

retain_headers now passes the header index alongside the &Header to its
closure, which import_headers uses for its post-pass removal step
instead of tracking a parallel counter. Existing callers that don't
need the index ignore it with `_`.

Closes: #515
2026-05-06 16:30:36 +01:00
Wez Furlong 4ab07b7f1c Fix EnvelopeAddress::parse("")
This is similar to https://github.com/KumoCorp/kumomta/pull/512, but
whereas that one was a parser precedence issue, this one is due to
how Null is represented outside of the context of rfc5321.

refs: https://github.com/KumoCorp/kumomta/issues/511
2026-05-05 19:39:27 +01:00
kay ozaki a997a6664b make_message fails for from address which starts with postmaster
closes: https://github.com/KumoCorp/kumomta/pull/512
2026-05-05 19:30:26 +01:00
kay ozaki 7921fa5a19 typing.lua boolean-true default prevents mail_auth modules from being disabled
refs: https://github.com/KumoCorp/kumomta/pull/505
2026-04-29 12:06:35 +01:00
kay ozaki de95e63b2a multipart/mixed > multipart/related was not parsing
closes: https://github.com/KumoCorp/kumomta/pull/506
2026-04-29 11:58:03 +01:00
Wez Furlong 37236c40b1 docs: update for the 2026.04.09-ea3b2a9b release 2026-04-29 08:07:13 +01:00
Wez Furlong ea87ea6ab3 make HeaderAddress localpart consistent with EnvelopeAddress
This was previously just returning everything before the final at-sign.
Now we use the same parsing approach and return the normalized local
part.

This is technically a breaking change, but it is minor and improves
the overall state of things.
2026-04-09 15:42:40 +01:00
Wez Furlong a800f49d7d docs: changelog for https://github.com/KumoCorp/kumomta/pull/497 2026-04-04 19:55:43 +01:00