Commit Graph
3976 Commits
Author SHA1 Message Date
dependabot[bot] b51b7bf8e8 build(deps): bump docker/login-action
Bumps the github-actions group with 1 update: [docker/login-action](https://github.com/docker/login-action).


Updates `docker/login-action` from 4.7.0 to 4.8.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v4.7.0...v4.8.0)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-12 09:53:45 +00:00
Wez Furlong 81f191dff6 config: improve typing.lua performance for queues.lua
Add some internal functions that return the set of valid field
names for the various make_XXX config building lua functions
so that we can change the cost of validating the helper config
types via typing.lua from O(N) relatively expensive individual
calls through lua/mlua/rust into O(1) lua table lookups.

This takes the cost of loading a 2000 entry queues.toml down
from an uncomfortable 300ms to below 50ms and the cost of
the get_queue_config event handler down from 240us to 10us.

The same pattern is also applied to listener_domains.lua
and sources.lua.
2026-11-11 07:44:55 +00:00
Wez Furlong 5ae9ba0777 map: make DomainMap, CidrMap and RegexSetMap clones cheap
These types now use Arc shared storage with copy-on-write for inserts.
Clone is now a cheap pointer increment instead of a full deep copy,
removing the per-message cost that scaled linearly with queues.toml size.
2026-11-10 21:07:39 +00:00
Wez Furlong 8f9f63b078 egress_source: add sticky source assignment for egress pools
Add a `StickyWeightedRoundRobin` selection strategy for egress pools that
retains the same assigned source for a message across all retries, while
still using weighted round robin for the first attempt.

Implements #670
2026-11-10 16:14:08 +00:00
Wez Furlong 1d3f34574f queue: check message expiry before queuing
Check expiry whenever a message is placed on a delivery queue, not only
after failed delivery attempts.

Fixes #671
2026-11-10 09:16:28 +00:00
Wez Furlong 88432e2238 shaping: SetConfig from provider rules now uses mx_rollup=true
Provider-scoped automation rules using SetConfig now apply their override
to the full MX cluster for the triggering domain, rather than just that
single domain.
2026-11-10 08:35:51 +00:00
Wez Furlong f98d18c853 shaping: clean up rule match scope types
Split static configuration from runtime match state and resolve a
type name collision between match scope and event key types.
2026-11-10 08:35:51 +00:00
Wez Furlong 8ee5e392ca shaping: add SetProviderConfig automation action
Add an automation action that applies config overrides scoped to an entire
matched provider, rather than only the triggering domain.
2026-11-10 08:35:51 +00:00
Wez Furlong 0a8515194e listener: make listen backlog configurable
Avoid client connection stalls and timeouts during connection bursts by
allowing configuration of the listen backlog, replacing the
rust-internal hardcoded 128 entry limit.
2026-11-09 16:35:07 +00:00
Wez Furlong c2d3d39e7d memory: preserve callstack attribution on failed realloc
When realloc fails, retain the original allocation's existing callstack
record instead of capturing a new one. This keeps memory diagnostics
accurate during allocation failures.
2026-11-09 15:52:20 +00:00
Wez Furlong a8c1e43f58 memory: fix race erasing callstack records for new allocations
When callstack tracking is enabled, the prior deallocation cleanup was
running after the memory had been returned to the allocator, allowing
a new allocation to reuse the address before its record was cleared.
2026-11-09 14:54:41 +00:00
Wez Furlong 0a80ed8ca4 memory: reduce contention on allocation tracking counters
Use sharded counters keyed by allocation address to reduce global atomic
contention, while preserving separate accounting for Rust allocations.

With callstack tracking disabled, the 256-shard prototype of this change
improved 16-thread allocation/free throughput by roughly 10x for
same-thread frees and 3x for cross-thread frees, using 32 KiB of counter
storage.  Single-thread throughput was comparable. These are
microbenchmarks: the gain fell below 1% with roughly 3.5 us of other
work per pair.
2026-11-09 11:31:02 +00:00
Wez Furlong 958e4668f1 acct: make max_file_size work for accounting logs
The `max_file_size` option of `kumo.aaa.configure_acct_log` had no
effect because the written byte counter was never incremented.
2026-11-08 07:56:56 +00:00
Wez Furlong adbd37063e acct: fix missing flush on shutdown
Ensure that we explicitly flush and wait for the log to finish
writing on shutdown.
2026-11-08 07:21:00 +00:00
Wez Furlong 5814ed7ec1 queue: fix indefinite graceful shutdown hang
Prevent graceful shutdown from hanging indefinitely waiting for
the queue meta maintainer when shutting down while messages are
in flight.

Closes: #654
2026-11-06 13:33:42 +00:00
Wez Furlong 2d1966d2e2 kumo-template: cache compiled templates across requests
Add a centralized LRU cache for compiled templates, avoiding repeated
compilation per HTTP injection request. This yields large performance
gains for Handlebars workloads.

Closes: https://github.com/KumoCorp/kumomta/pull/638
2026-11-06 07:05:23 +00:00
Wez Furlong 50c53b5266 message: fix flakey test 2026-11-06 07:05:23 +00:00
Wez Furlong e61bc0844a docs: fixup ci build
Recent github ubuntu image changes broke the rocksdb build
in our docs CI flow, so switch it to our builder image where
rocksdb is pre-built.

Revise the build to use a pre-installed mkdocs so that we don't
have to docker it.
2026-11-05 11:30:52 +00:00
Wez Furlong 980521eb67 docs: format this page 2026-11-05 11:29:55 +00:00
Wez Furlong b70672f52d ci: add docs toolchain to ubuntu 24.04 builder image
Add mkdocs, material theme, required plugins and zensical to the builder
image to run documentation builds directly there later.
2026-11-05 10:57:44 +00:00
dependabot[bot] ddc3054981 build(deps): bump the github-actions group across 1 directory with 2 updates
Bumps the github-actions group with 2 updates in the / directory: [docker/login-action](https://github.com/docker/login-action) and [docker/build-push-action](https://github.com/docker/build-push-action).


Updates `docker/login-action` from 4.6.0 to 4.7.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v4.6.0...v4.7.0)

Updates `docker/build-push-action` from 7.3.0 to 7.4.0
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v7.3.0...v7.4.0)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
- dependency-name: docker/build-push-action
  dependency-version: 7.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-05 10:57:17 +00:00
Wez Furlong 7a32d83ec6 smtp_dispatcher: add send-time DANE consistency guard
When an egress path has enable_dane set true but the global DANE
isolation switch is off, defer messages with a 451 transient failure
instead of risking silent DANE downgrades on shared ready queues.
2026-11-05 08:10:32 +00:00
Wez Furlong 79c1b80174 xfer: reset attempt count when moving messages to xfer queues
Reset a message's delivery attempt count when it moves to an xfer queue,
preventing unsafe backoff calculation overflow that caused panics, premature
expiry, or endless immediate retries.

Reported-by: @edgarsendernet
Closes: #644
2026-11-05 06:20:20 +00:00
3abb6bb615 kumo-template: avoid copying the Handlebars context (#637)
handlebars::Context::wraps serializes its argument with
serde_json::to_value, so passing it an existing serde_json::Value
deep-copies the whole substitution context a second time.

Both Handlebars paths already hold an owned serde_json::Value at that
point (render_to_write, and prepare_context which batch injection uses
once per recipient), so take ownership of it with Context::from
instead. The rendered output is unchanged, and the error case that
wraps reported (a serialization failure) cannot occur for a value that
is already JSON.

---------

Co-authored-by: Akshit Sethi <akshits@adobe.com>
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2026-11-04 16:42:56 +00:00
Wez Furlong 507781eaa1 memoize: fix deadlock on file-scope memoized calls during policy load
Fixes the timeout deadlock where calling a memoized function while loading
the policy would hang waiting for cache population. This also fixes the
same deadlock when cache population triggers a policy reload, and retries
population correctly when an in-progress attempt exits early.
2026-11-04 14:37:14 +00:00
Wez Furlong 06b6a09752 next: reduce verbosity by default 2026-11-04 09:03:39 +00:00
Wez Furlong 6b11b9070e http: Disable bundled webpki roots for all reqwest clients
All HTTP clients use only the runtime OS trust store, with no compiled-in
root CA bundle. Existing TLS and HTTP behaviour is otherwise unchanged.

This commit was brought to you by the number 4.  May your private
keys be sufficiently long, now and into the future.
2026-11-04 08:50:13 +00:00
Wez Furlong ba510cea03 ci: install cargo cooldown with stable rust toolchain
its using the repo-level toolchain, which has lower rust requirements
than cooldown itself.
2026-11-04 07:34:45 +00:00
Wez Furlong 0b64c9a682 dns: isolate DANE-capable insecure MX domains
When enabled (default is disabled), domains with insecure MX selection
but valid DANE TLSA records use a distinct site name. This prevents an
insecure domain from denying DANE authentication to DNSSEC secure
sibling domains on the same MX hosts.
2026-11-04 07:21:38 +00:00
Wez Furlong 2aa4706f16 test: assert MTA-STS deferral responses by exact equality
Replace substring checks with exact full-string assertions for MTA-STS deferral
responses. Only the peer's ephemeral port is masked out. No other normalization
is performed. This does not change tested behavior.
2026-11-04 07:21:37 +00:00
Wez Furlong b0f0148c26 smtp_dispatcher: defer MTA-STS "enforce" mail on shared cleartext connection
When multiple recipient domains share a ready queue, a message for a domain
that publishes MTA-STS enforce will be deferred rather than delivered over
a cleartext connection established by another domain.
2026-11-04 07:21:37 +00:00
Wez Furlong a6bf765101 smtp: only allow MTA-STS and DANE to raise TLS policy
MTA-STS and DANE policies can only raise, never lower, the configured
enable_tls level for a destination.

Integration tests and documentation adapted from @terrorobe's PR #616.

closes: https://github.com/KumoCorp/kumomta/pull/616
2026-11-04 07:21:35 +00:00
Wez Furlong 9075eb4415 smtp_dispatcher: clear tls_info between connections 2026-11-04 07:20:56 +00:00
denny.weinert 02b5412cfd fix: returns the tsa endpoint ip instead of an index number 2026-11-03 11:19:18 +00:00
cargo-dependency-cooldown[bot] 058a4deca5 deps: update Cargo.lock via cooldown 2026-10-26 08:02:43 +00:00
Wez Furlong 6b517bccee Revert "Update get_queue_config docs (#651)"
This reverts commit 45b3b455fd,
which was merged in error.
2026-10-23 06:40:03 +01:00
Tom Mairs 45b3b455fd Update get_queue_config docs (#651)
* Modify kumo-proxy.md to reflect latest functions.

* Correct queue handling process with error and nil.

* Merge branch 'main' into tm221020262135
2026-10-22 21:47:52 -06:00
Wez Furlong 28cacd46be dns: fixup and enable spf live tests too
These were failing in my sandbox environment because the live
DNS zone is too big for UDP responses, and my sandbox's DNS
doesn't handle TCP fallback.

Let's adopt the same fixed upstream pattern that we use for the
DANE/DNSSEC tests so that the live tests stand a better chance
at passing despite the local DNS infra configuration.  They are still
prone to flakes if the zone content changes.
2026-10-22 16:24:06 +01:00
Wez Furlong e99031cbbe dns-resolver: mitigate hickory 0.26 dnssec issue
https://github.com/hickory-dns/hickory-dns/issues/3974 poses problems
for users that need dnssec validation

This commit points to the current ref from the `release/0.26` branch
for the moment; when hickory makes a new release we'll re-point.

This commit also enables live-dns-tests in the main `make test` flow
to make it more likely to detect issues of this sort in future,
at the cost of potentially more flakes in CI.
2026-10-22 16:03:28 +01:00
Wez Furlong f701e209cd rfc5321: fix Response::was_due_to_message() classification
Correctly classify errors for partially parsed commands, including the
redacted AUTH case broken by AUTH command redaction.

https://github.com/KumoCorp/kumomta/pull/636#issuecomment-6314956974
2026-10-21 18:37:03 +01:00
Wez Furlong 685ccdc3ec policy-extras: add cache tuning options for sources.lua
Add the same set of memoize tuning parameters to the sources helper
that are already available on the shaping helper, fully backwards
compatible.
2026-10-21 18:00:06 +01:00
Wez Furlong 95d80d96d3 metrics: export per-thread-pool runtime metrics and cache populate duration
Per-thread-pool tokio runtime metrics are now exported for every named
runtime, not just the main runtime. An lruttl_populate_duration histogram
records cache populate time. An integration test verifies both metrics are
exported.
2026-10-21 18:00:06 +01:00
Wez Furlong 7f57cf07fa config: move Lua VM teardown off pool mutex and worker threads
Use a dedicated background dropper thread for stale Lua VMs. Avoid running
lua_close while holding the pool lock or on tokio worker threads.
2026-10-21 18:00:06 +01:00
Wez Furlong 83fe178032 kumo.serde: add kumo.serde.load_files
Add kumo.serde.load_files which loads and parses a list of JSON and
TOML files with only one await point, which is more efficient than
the individual calls for folks using the sources helper with large
numbers of source toml files.
2026-10-21 18:00:05 +01:00
Wez Furlong 3f026ba2dc lruttl: avoid starving memoized cache population tasks
Spawn cache populate tasks on the main runtime instead of the caller's
runtime to avoid potential contention with the calling runtime.

For example, if DKIM signing is running in the calling context and
we're trying to run an async heavy workload (serially reading
data from numerous sources) in the populate function, the DKIM
compute heavy work could starve the populate function and prevent
it from making progress.
2026-10-21 18:00:05 +01:00
Harsh Jha 1540e6a51d rfc3464: recognize recipient address types case-insensitively
RFC 3464 section 2.1.2 makes address types case-insensitive, but the
recipient parser only normalizes addresses for an exact lowercase rfc822
match. This leaves angle brackets intact in OOB recipients for RFC822
reports.

Trim the type before comparing it case-insensitively, reuse the existing
SMTP address parser, and cover original and final recipients with tests.
Preserve the reported type spelling and other address types.

closes: https://github.com/KumoCorp/kumomta/pull/640
2026-10-21 10:39:23 +01:00
Wez Furlong 91ac946abb docs: changelog for #615
closes: https://github.com/KumoCorp/kumomta/pull/615
2026-10-21 09:04:45 +01:00
Wez Furlong 99dcdb8ec7 tls-helper: remove dead try_dup trait method
Removes the method that existed solely to support plaintext stream
recovery after failed TLS handshakes, which is no longer used.
2026-10-21 09:04:44 +01:00
Wez Furlong 9fd9b38c1b rfc5321 client: remove plaintext fallback on TLS handshake failure
Drop the logic that recovered the plaintext stream after handshake failure;
the connection is always discarded on handshake failure now.
2026-10-21 09:04:44 +01:00
Wez Furlong 1c1e6c36ff smtp_server: stop trying to recover socket after failed TLS handshake
Since the handshake failure path now drops the connection, recovering the
plaintext stream doesn't serve a purpose. Use plain accept() instead of
into_fallible(). The socket is simply consumed on failure.
2026-10-21 09:04:44 +01:00