When an egress path has enable_dane set true but the global DANE
isolation switch is off, defer messages with a 451 transient failure
instead of risking silent DANE downgrades on shared ready queues.
Reset a message's delivery attempt count when it moves to an xfer queue,
preventing unsafe backoff calculation overflow that caused panics, premature
expiry, or endless immediate retries.
Reported-by: @edgarsendernet
Closes: #644
handlebars::Context::wraps serializes its argument with
serde_json::to_value, so passing it an existing serde_json::Value
deep-copies the whole substitution context a second time.
Both Handlebars paths already hold an owned serde_json::Value at that
point (render_to_write, and prepare_context which batch injection uses
once per recipient), so take ownership of it with Context::from
instead. The rendered output is unchanged, and the error case that
wraps reported (a serialization failure) cannot occur for a value that
is already JSON.
---------
Co-authored-by: Akshit Sethi <akshits@adobe.com>
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
Fixes the timeout deadlock where calling a memoized function while loading
the policy would hang waiting for cache population. This also fixes the
same deadlock when cache population triggers a policy reload, and retries
population correctly when an in-progress attempt exits early.
When enabled (default is disabled), domains with insecure MX selection
but valid DANE TLSA records use a distinct site name. This prevents an
insecure domain from denying DANE authentication to DNSSEC secure
sibling domains on the same MX hosts.
When multiple recipient domains share a ready queue, a message for a domain
that publishes MTA-STS enforce will be deferred rather than delivered over
a cleartext connection established by another domain.
MTA-STS and DANE policies can only raise, never lower, the configured
enable_tls level for a destination.
Integration tests and documentation adapted from @terrorobe's PR #616.
closes: https://github.com/KumoCorp/kumomta/pull/616
https://github.com/hickory-dns/hickory-dns/issues/3974 poses problems
for users that need dnssec validation
This commit points to the current ref from the `release/0.26` branch
for the moment; when hickory makes a new release we'll re-point.
This commit also enables live-dns-tests in the main `make test` flow
to make it more likely to detect issues of this sort in future,
at the cost of potentially more flakes in CI.
Per-thread-pool tokio runtime metrics are now exported for every named
runtime, not just the main runtime. An lruttl_populate_duration histogram
records cache populate time. An integration test verifies both metrics are
exported.
Add kumo.serde.load_files which loads and parses a list of JSON and
TOML files with only one await point, which is more efficient than
the individual calls for folks using the sources helper with large
numbers of source toml files.
RFC 3464 section 2.1.2 makes address types case-insensitive, but the
recipient parser only normalizes addresses for an exact lowercase rfc822
match. This leaves angle brackets intact in OOB recipients for RFC822
reports.
Trim the type before comparing it case-insensitively, reuse the existing
SMTP address parser, and cover original and final recipients with tests.
Preserve the reported type spelling and other address types.
closes: https://github.com/KumoCorp/kumomta/pull/640
Add kumo.string.psl_suffix_is_known and kumo.string.psl_domain_is_known,
which report whether the resolved public suffix is explicitly listed in
Mozilla's Public Suffix List.
The psl crate applies a "fallback rule": a domain whose public suffix is
not in the list (e.g. `domain.invalid`) still resolves to a suffix, but
that suffix is not explicitly known. Existing kumo.string.psl_suffix /
psl_domain discard this information, so fallback-resolved suffixes are
indistinguishable from explicitly listed ones.
The new functions return `true` when the suffix is explicitly listed,
or `false` otherwise (fallback-resolved, or no public suffix at all).
closes: https://github.com/KumoCorp/kumomta/pull/592
Redact credentials from both inbound rejection logs and outbound
delivery failure records. Credentials are replaced with [REDACTED] in all
log and diagnostic output.
closes: https://github.com/KumoCorp/kumomta/pull/636
Matches the existing max_connections behavior from SMTP and proxy listeners.
Operators can monitor and alert on endpoint saturation.
refs: https://github.com/KumoCorp/kumomta/issues/631
Precompile static headers once at batch compile time and clone them per
recipient. Only headers that actually use template variables are evaluated
and constructed separately for each recipient.
closes: https://github.com/KumoCorp/kumomta/pull/620
Fixes two concrete bugs: empty message bodies leaked the resident count
upward, and node-to-node transferred messages were never counted, driving
the gauge downward. All modifications are in crates/message, with an
accompanying changelog entry.
Fixes: #588
Reported-by: @kayozaki
Skip failing entries and proceed with healthy alternatives. When all entries
fail, defer delivery with a transient failure listing the lookup errors.
Previously failures were only logged at debug level with no delivery failure
recorded.
Closes#628
When one address family lookup fails in a dual-stack query, limit the
combined cache entry expiry to 60 seconds instead of using the full TTL
of the surviving family. This avoids pinning the failed family out of
use for up to a day. #630
Treat SERVFAIL, REFUSED, and DNSSEC-bogus MX answers as temporary errors
instead of misinterpreting them as NODATA. This prevents such answers from
incorrectly triggering fallback to the domain's A/AAAA records.
Refs: #629
Nobody runs it and it is not as good as RocksDB. I want to remove it
in the future so that users don't unwittingly deploy a poor configuration.
We now log errors on startup to help surface configuration that will
need attention, but we don't block startup.
Since hickory can now be used to enable DANE, the only remaining
use case for the unbound embedded resolver is disolved.
I want to remove it from the tree because:
* I don't want to own the rust crate wrapping libunbound
* Reduces complexity
* Users have reported issues with unbound that simply disappear
by switching back to hickory
* Essentially nobody is running kumomta with unbound enabled,
so it is has no real production coverage: you're a snowflake
if you're running it
Removing it will reduce the potential for a user unwittingly enabling
something that isn't known to work as well as the golden path.
The deprecation manifests as an error printed on startup when
enabling unbound, and adjustments in the docs.
It will continue to work as it does today.
Cache population now runs detached by default for top-level memoized
functions. Other callers waiting on the same key now receive the result
even if the original triggering caller is cancelled mid-populate;
previously that case could impact the other yet-to-be-timed-out
waiters and cause cache population to restart or otherwise fail.
refs: #602
Lets operators tell local concurrency permit wait timeouts apart from real
remote resolver query timeouts, to avoid misdiagnosing overload as remote
deliverability failure.
Adds metrics dns_mx_resolve_permit_timeout, dns_mx_resolve_query_timeout
and dns_mx_resolve_permit_waiters.
Indeterminate MX lookup failures now use a short 30s transient negative TTL,
while NXDOMAIN and policy rejections retain the 5 minute TTL. Lookups blocked
by the concurrency limit are not cached at all. You can configure the transient
TTL with kumo.dns.set_mx_transient_negative_cache_ttl.
This prevents transient failures from causing domains to drop out of shaping
rollups and incorrectly flipping the shaping hash.
Explain the factoring phases and input edge cases, and make the DNS
fixture and session-reuse timeout intent easier to read.
Move site-name guidance to the queues reference with an inline release
marker, and link to it from lookup_mx and the traffic-shaping guide.
Add a breaking-change note with an observed collision pair.
Normalize, deduplicate and sort MX destinations before deriving site
names. Use per-label alternatives only when their expansion matches
the destination set; otherwise factor complete hostname branches.
Keep MX preferences in connection plans rather than site identity.
Keep asterisks literal and percent-encode other special label bytes
so names remain distinct without introducing metric-label escapes.
Cover collisions, normalization, ports, shaping and queue/session
sharing, and document the naming and preference semantics.
Cap iprev at 10 PTR names for forward lookups, as required by RFC 8601.
Prevents malicious reverse DNS zones from causing unbounded DNS lookups.
closes: https://github.com/KumoCorp/kumomta/issues/623
A message with 8-bit headers or body could not be bounced at all: building the
returned-content part failed and aborted the whole report. It now produces a
7-bit clean report, returning as much of the original as can be represented and
omitting the rest.
Prevent transient accept() errors from permanently stopping listeners
while the process stays alive and appears healthy. Trivial peer resets
that happen between kernel queue and accept are logged only at debug
level to close remote log spam vectors.
On Linux, we didn't relinquish our root-ness to the full extent
that we intended, which meant that it was technically possible
to regain those privileges via syscall.
That meant that, if an attacker found a way to perform remote code
execution in kumod, they could potentially do so with root privileges.
(Note that there are no known RCE vulnerabilities in kumod).
This commit closes that gap.
Publish metrics showing how far each out-of-process log consumer trails
the newest data in the directories this node logs to, so that operators
can tell whether a consumer is keeping up. Covers both the
configure_local_logs and kumo.jsonl.new_writer directories.
The documentation generators now stamp each page they emit with an
@generated comment so that tooling can tell generated pages apart from
hand-written ones. Update the table-of-contents generator to skip YAML
front matter when picking a page title, since the marker is now stored
there.