Commit Graph

1296 Commits

Author SHA1 Message Date
Wez Furlong 407d2bc332 docs: update for multi-recipient changes 2025-10-07 08:35:59 +01:00
Wez Furlong 794113f0b0 smtp_dispatcher: add max_recipients_per_batch
This sets a client side shaping control to manage the maximum number of
RCPT TO commands issued in a single transaction.
2025-10-07 07:53:15 +01:00
Wez Furlong f968935315 add smtp_server_split_transaction event
This provides full control over how an incoming smtp transaction
will be split into batches.
2025-10-07 07:53:15 +01:00
Wez Furlong b157e2f617 add smtp_server_data event for per-txn message validation
Great for doing content scanning and recipient list modification!
2025-10-07 07:53:13 +01:00
Wez Furlong db5ebb88a7 message: expose recipient list to lua api
message:recipient() may now return an array style table
holding the recipient list, if there is more than a single
recipient on the message.

Since this can be somewhat ambiguous/frustrating to work with,
there is now also a message:recipient_list() that will always
return an array style table, even if it holds just a single
element.

The included helpers have been updated to use `message:recipient_list`.

message:set_recipient() will now optionally accept an array
style table holding the recipient list to be set.
2025-10-07 07:52:58 +01:00
Wez Furlong 18e55a6e91 smtp: handle "too many recipients" status more appropriately
This commit adds a test for exceeding the number of recipients
allowed in a transaction and fleshes out our handling of that
status so that we will immediately try the remaining recipients
of the batch, without delaying the message by eg: 20 minutes
in the default retry parameters.

As part of this, I noticed that we were returning the wrong
smtp status code in our server implementation, so this
commit fixes that up.
2025-10-07 07:52:58 +01:00
Wez Furlong 9079a9e569 WIP: more multi-recipient type changes
This commit updates the type signatures of some key structures
to allow for the possibility of a Message having multiple recipients:

 * Message (continuing from previous commit)
 * JsonLogRecord
 * MessageInformation

Some logic has been updated to account for multiple recipients,
but critically, nothing in the core will generate them, and none
of the queue management or delivery infrastructure is aware
of recipients beyond the first.
2025-10-07 07:52:57 +01:00
Wez Furlong 206dc7c7fa optionally detect legacy 8-bit during check-fix 2025-10-07 07:50:46 +01:00
Wez Furlong 981de67f1c docs: update for 2025.10.06-5ec871ab release 2025-10-07 07:38:10 +01:00
Wez Furlong 307d0b3093 docs: changelog for #423
closes: https://github.com/KumoCorp/kumomta/issues/423
2025-10-06 19:41:53 +01:00
Wez Furlong 4ba41290a6 docs: format and fixup whitespace 2025-10-06 07:02:18 +01:00
Wez Furlong dba45cfa28 logging: allow * to match meta names
This uses the same scheme as allowed for matching header names
in logging.
2025-10-06 07:02:18 +01:00
Tom Mairs 873f9e524d Update sample code 2025-10-03 20:27:23 +01:00
Wez Furlong 5228f794bb xclient: capture original via/from address in context
I can see someone wanting to know the raw physical connection
info at some point, regardless of the adjusted xclient values,
or in the future, the proxy protocol adjusted values.
2025-10-03 20:03:14 +01:00
ncai ce743133b0 fix typo in doc 2025-09-30 07:35:32 +01:00
Wez Furlong d9d3d14e3e enable XCLIENT support
Hooks up the parsed XCLIENT command to the smtp server.
We support just the address changing attributes; we don't
do anything special with the NAME attribute, and I'd like
to think a lot more about authentication before allowing
messing with the LOGIN attribute.

After applying changes to the metadata in the connection,
take care to re-evaluate the listener parameters and
dynamic parameters so that `via` and `peer` blocks can
match the new configuration.
2025-09-26 09:36:15 +01:00
Wez Furlong 90dd172f6c wrap up 8BITMIME and SMTPUTF8 support
Finishes up support for these extensions by adding a send-time
check and error for the case where we have 8 bit content and
the appropriate extension is not advertised by the next hop.

A shaping option is provided to bypass this, both to facilitate
testing and as a way to bypass it for situations where the
operator knows best.

closes: https://github.com/KumoCorp/kumomta/issues/327
2025-09-25 12:37:11 +01:00
Wez Furlong 33e1883d79 add kumo.apply_supplemental_trace_header
This allows manual and conditional control over whether and how this
header is added to a message.
2025-09-25 09:02:41 +01:00
Wez Furlong 6fdffa53e3 expose hostname and received_via for http injections
To facilitate this we need to feed the `AppState` down through
the Router more generally, and expand it to grab a copy of the
http listener parameters.

closes: https://github.com/KumoCorp/kumomta/issues/417
2025-09-25 08:37:06 +01:00
Wez Furlong eb54cc74d9 proxy-server: raise NOFILE soft limit to match hard limit
This makes the startup behavior consistent with kumod and tsa-daemon,
which can help with "Too many open files" errors in systems without
any explicit tunings for the proxy server.

It is still a good idea to explicitly tune that value up, and it is
something we should put into a systemd unit when we get around to
shipping one of those for the proxy server (#188)
2025-09-18 15:54:18 +01:00
Wez Furlong 220edf24b4 fix doc link 2025-09-18 15:27:05 +01:00
Wez Furlong b615646e84 mod-encode: be more relaxed about excess baseXX padding
The implementation of these encodings in the data-encoding crate is
overly strict when extraneous/excess padding is used, presenting
problems from the perspective of Postel's law especially in the context
of email: there are base64 encoders widely deployed that can produce
extraneous padding that is otherwise harmless but which will raise an
error from this crate.

This commit works around this by manipulating the encoding context
to remove the padding requirement, as well as the padding, and
decodes the input without padding.
2025-09-18 15:22:04 +01:00
Wez Furlong f4b2ff4b4e logging: emit additional event_time and created_time fields
We made the unfortunate choice to emit the `timestamp` and `created`
fields as unix timestamps with no sub-second precision.

This commit causes us to emit the full-resolution timestamps that we
already have as additional event_time and created_time fields
in the JsonLogRecord, without internally duplicating those fields.

refs: https://github.com/KumoCorp/kumomta/issues/405
2025-09-12 15:28:22 +01:00
Wez Furlong 91c0c2ac57 mailparsing: fallback to Content-Type::name for attachment name
Adjusts how we resolve the attachment name when processing mime
attachment parameters:

* First check Content-Disposition::filename
* Then check Content-Type::name

De-couple parsing out the `Content-Id` header from having
`Content-Disposition`.

In the lua bindings; remove the synthesized filename fallback;
we now just leave it nil to match the rust logic.
2025-09-10 10:42:48 +01:00
kay ozaki 934a8eebab adding crc32
closes: https://github.com/KumoCorp/kumomta/pull/404
2025-09-09 14:35:48 +01:00
Wez Furlong 7fffccf410 Add support for generating NDRs
This commit adds some plumbing to facilitate generation of RFC 3464
non-delivery reports and some corresponding glue to enable calling
it from lua.

Examples in the docs that are also added.
2025-09-09 14:12:52 +01:00
Wez Furlong c7cbd99a9f add kumo.inject_message 2025-09-09 10:38:36 +01:00
Wez Furlong 1bf9d138ce extended lua mime parsing API
refs: https://github.com/KumoCorp/kumomta/issues/117

Could also make it possible to do eg:
https://github.com/KumoCorp/kumomta/issues/142
without any explicit direct support in the product.
2025-09-09 09:15:58 +01:00
Mike Hillyer 9c16e45a92 Working on clarifying the scopes used in Kumo's shaping.lua. 2025-08-28 11:46:12 -04:00
Daniel Schaaff 3a42354289 fix prettier removing indentation on note section
Signed-off-by: Daniel Schaaff <daniel@danielschaaff.com>
2025-08-28 10:21:44 -04:00
Daniel Schaaff 1bcb2c31ac fix minor typo in logging doc 2025-08-28 10:21:44 -04:00
smsvip 05d4a23b60 Update set_smtpsrv_threads.md
Repair set_smtpsrv_threads documents
2025-08-25 22:20:23 -04:00
Tom Mairs a1a5f71852 Update note about minimum supported instance 2025-08-08 14:47:18 +00:00
Wez Furlong 6a05a65f73 mpsc: provide shutdown event and clarify single-consumer parts
It was possible to deadlock your policy by triggering queue
length/emptiness checks conurrently with the queue consumer thread.

Let's make it more explicitly clear which methods cannot be used that
way by making their mutex acquisition non-blocking and raise an error.

In addition, let's improve the close method so that it doesn't require
exclusive access to the consumer side of the queue.

Provide a shutdown_logging event that can be used to explicitly close
queues on shutdown, if that is appropriate for your use case.
2025-07-29 15:22:51 +01:00
Tom Mairs 634a00ec6f Add new tested cloud environments 2025-07-27 22:50:10 +00:00
Wez Furlong 91884d3892 docs: format 2025-07-26 13:30:00 +01:00
Wez Furlong 33b109c792 docs: changelog for #399 2025-07-25 06:05:53 +01:00
Wez Furlong bc7c95b42a docs: changelog for #394 2025-07-25 06:02:57 +01:00
Pankaj Rathi eae61a90fe Add support for custom key names in HashiCorp Vault secrets (#399)
* Add support for custom key names in HashiCorp Vault secrets

Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-07-25 06:02:21 +01:00
Wez Furlong 9e62a275c5 add tls_required_client_ca esmtp listener parameter
This enables configuring mTLS support in the listener.

Adjust TLS client tests to use this on the sink side and verify
that mTLS works for both inbound and outbound SMTP.

refs: #100
refs: #391
2025-07-24 15:40:49 +01:00
Wez Furlong c8027effb6 smtp_server: collect incoming tls information for logs/meta
This commit extracts the TLS version, cipher and subject name from
the TLS state and captures it:

* In the trace headers
* In the connection metadata
* In the Reception log record

In order to capture the info in trace headers, this commit will
now change the reception protocol to ESMTPS (for SSL) or ESMTPSA (for
authenticated SSL), and emit the version and cipher information as
a comment like: `with ESMTPS (TLSv1_3:TLS13_AES_256_GCM_SHA384)`.

closes: #100
2025-07-24 13:01:13 +01:00
kayozaki 76add2ced5 Adding support for presenting Client cert as part of make_egress_path (#391) 2025-07-24 13:00:27 +01:00
Wez Furlong 7e6ca34e2d new: kumo.fs lua module
The read_dir and glob functions have been logically moved into that new
namespace, leaving deprecated versions of them in the `kumo` module.

A new `kumo.fs.open` function that works similarly to `io.open` is
provided.  This function cooperates with the kumo async io scheduler
and won't block it if the filesystem is under pressure.

It returns file handles that are simlar to the builtin lua file handle
objects, but do not support formatting or parsing of writes or reads
respectively: the calling code is responsible for that.  The rationale
for this difference is that is that lua's semantics for those functions
are frankly a bit weird and are hard to replicate precisely.
2025-07-23 16:51:06 +01:00
Wez Furlong 7d617bb270 add kumo.mpsc.define
This function allows defining a non-durable, non-persistent, in-memory
queue that is intended to be used as glue when building up more advanced
processing flows within kumomta.

The intent is that the queue is processed from within a task spawned via
`spawn_task`.
2025-07-23 12:44:10 +01:00
Tom Mairs 88b182d3c3 removed duplication 2025-07-22 23:08:46 +00:00
Tom Mairs 7064344b04 Update list of command line tools 2025-07-22 22:49:12 +00:00
Wez Furlong 7903dff857 docs: update json file versions
these were lagging behind due to a local configuration issue
on my build system in an earlier commit.
2025-07-22 09:08:40 +01:00
Wez Furlong 5a3c942675 docs: update for #390 2025-07-22 09:08:18 +01:00
kayozaki a7315633de add kumo.dns.lookup_ptr (#390)
* add lookup_ptr

Co-authored-by: Wez Furlong <wez@wezfurlong.org>
2025-07-22 08:45:27 +01:00
Wez Furlong fa5d6a6e7a allow specify port number in queue and routing_domain meta
The motivating use case here is to allow using a rebind command to
specify `smarthost:port`.  You can achieve that via an invocation
like:

```
kcli rebind --everything --set queue=smarthost:port
```

or:

```
kcli rebind --everything --set routing_domain=smarthost:port
```

refs: https://github.com/KumoCorp/kumomta/issues/352
2025-07-21 15:38:28 +01:00