For eg: the Sender and From headers.
This commit introduces a parser for RFC 5322 message header syntax,
as modified by RFC 2047 for encoded header fields.
This allows tracing incoming SMTP sessions.
Sessions are traced through to the cli in realtime.
The implementation is via a websocket, which makes this also potentially
pluggable into a webui in the future, although care must be taken to
ensure that only authorized system operators can enable this, as it can
reveal authentication secrets in the traced dialog, as well as the
content of messages.
This commit doesn't include documentation on the shape of the websocket
JSON at this time.
refs: https://github.com/KumoCorp/kumomta/issues/44
```
; ./target/debug/kcli trace-smtp-server
[127.0.0.1:51778->127.0.0.1:2025] === Connected {"received_from":"127.0.0.1:51778","received_via":"127.0.0.1:2025","reception_protocol":"ESMTP"}
[127.0.0.1:51778->127.0.0.1:2025] <- 220 foo Welcome to KumoMTA!
[127.0.0.1:51778->127.0.0.1:2025] -> EHLO foo.lan
[127.0.0.1:51778->127.0.0.1:2025] === smtp_server_ehlo: Ok
[127.0.0.1:51778->127.0.0.1:2025] <- 250-foo Aloha foo.lan
[127.0.0.1:51778->127.0.0.1:2025] <- 250-PIPELINING
[127.0.0.1:51778->127.0.0.1:2025] <- 250-ENHANCEDSTATUSCODES
[127.0.0.1:51778->127.0.0.1:2025] <- 250 STARTTLS
[127.0.0.1:51778->127.0.0.1:2025] -> STARTTLS
[127.0.0.1:51778->127.0.0.1:2025] <- 220 Ready to Start TLS
[127.0.0.1:51778->127.0.0.1:2025] -> EHLO foo.lan
[127.0.0.1:51778->127.0.0.1:2025] === smtp_server_ehlo: Ok
[127.0.0.1:51778->127.0.0.1:2025] <- 250-foo Aloha foo.lan
[127.0.0.1:51778->127.0.0.1:2025] <- 250-PIPELINING
[127.0.0.1:51778->127.0.0.1:2025] <- 250-ENHANCEDSTATUSCODES
[127.0.0.1:51778->127.0.0.1:2025] <- 250 AUTH PLAIN
[127.0.0.1:51778->127.0.0.1:2025] -> AUTH PLAIN AHNjb3R0AHRpZ2Vy
[127.0.0.1:51778->127.0.0.1:2025] === smtp_server_auth_plain: Ok: Bool(true)
[127.0.0.1:51778->127.0.0.1:2025] <- 235 2.7.0 AUTH OK!
[127.0.0.1:51778->127.0.0.1:2025] === conn_meta updated to {"authn_id":"scott","authz_id":"scott","received_from":"127.0.0.1:51778","received_via":"127.0.0.1:2025","reception_protocol":"ESMTP"}
[127.0.0.1:51778->127.0.0.1:2025] -> MAIL FROM:<wez@example.com>
[127.0.0.1:51778->127.0.0.1:2025] === smtp_server_mail_from: Ok
[127.0.0.1:51778->127.0.0.1:2025] <- 250 OK EnvelopeAddress(\"wez@example.com\")
[127.0.0.1:51778->127.0.0.1:2025] -> RCPT TO:<wez@wezfurlong.org>
[127.0.0.1:51778->127.0.0.1:2025] === smtp_server_rcpt_to: Ok
[127.0.0.1:51778->127.0.0.1:2025] <- 250 OK EnvelopeAddress(\"wez@wezfurlong.org\")
[127.0.0.1:51778->127.0.0.1:2025] -> DATA
[127.0.0.1:51778->127.0.0.1:2025] <- 354 Send body; end with CRLF.CRLF
[127.0.0.1:51778->127.0.0.1:2025] -> Date: Sun, 13 Aug 2023 10:10:40 -0700
[127.0.0.1:51778->127.0.0.1:2025] -> To: wez@wezfurlong.org
[127.0.0.1:51778->127.0.0.1:2025] -> From: wez@example.com
[127.0.0.1:51778->127.0.0.1:2025] -> Subject: test Sun, 13 Aug 2023 10:10:40 -0700
[127.0.0.1:51778->127.0.0.1:2025] -> Message-Id: <20230813101040.1426417@foo>
[127.0.0.1:51778->127.0.0.1:2025] -> X-Mailer: swaks v20201014.0 jetmore.org/john/code/swaks/
[127.0.0.1:51778->127.0.0.1:2025] ->
[127.0.0.1:51778->127.0.0.1:2025] -> This is a test mailing
[127.0.0.1:51778->127.0.0.1:2025] ->
[127.0.0.1:51778->127.0.0.1:2025] ->
[127.0.0.1:51778->127.0.0.1:2025] -> .
[127.0.0.1:51778->127.0.0.1:2025] === smtp_server_message_received: Ok
[127.0.0.1:51778->127.0.0.1:2025] === Message from=wez@example.comto=wez@wezfurlong.org id=5465de7739fc11ee8af250ebf67f93bd
[127.0.0.1:51778->127.0.0.1:2025] === Message queue=wezfurlong.org relay=true log_arf=false log_oob=false
[127.0.0.1:51778->127.0.0.1:2025] === Message meta: {"authn_id":"scott","authz_id":"scott","received_from":"127.0.0.1:51778","received_via":"127.0.0.1:2025","reception_protocol":"ESMTP"}
[127.0.0.1:51778->127.0.0.1:2025] <- 250 OK ids=5465de7739fc11ee8af250ebf67f93bd
[127.0.0.1:51778->127.0.0.1:2025] -> QUIT
[127.0.0.1:51778->127.0.0.1:2025] <- 221 So long, and thanks for all the fish!
[127.0.0.1:51778->127.0.0.1:2025] === Closed
```
This commit causes the scheduled queue maintainer to refresh
the queue config by calling the get_queue_config event approximately
every minute while the queue is alive.
In addition, we now thread the routing_domain through to get_queue_config
You may now remove or replace the test domain suffix from the set
of senders/recipients by usingn `--domain-suffix ''` to remove
it, or some other value to set it to something else.
`--domain` can now be used multiple times to build out the list
of test domains.
A couple of scenarios we're shooting for here:
* Startup ordering races, or other "burps" due to eg: restarting the
tsa daemon to update its config
* For "replication" scenarios, you may be running multiple instances
of tsa-daemon on multiple nodes, and list them all in both the
publish and subscribe lists on the clients.
In that situation the client will try to read from each node;
the data its reads should be essentially the same from each of them,
and it is fine if only one out of the set responds, or honestly,
even if none of them respond in the moment, as we'll eventually
be able to read that data.
Errors making those http requests are logged to the diagnostic log,
but the shaping data is otherwise allowed to load.
We now generate more context-rich error messages to help pinpoint
the problem:
```
caused by: failed to parse '10.0.0.1/24' as CIDR notation: host part of address was not zero
```
It's easy to typo this and get surprised by the results, so explicitly
check for it and provide a more actionable message than the DNS
resolution error for `cox.` would imply:
```
Entry for domain 'cox' consists of a single DNS label. Domain names in TOML sections need to be quoted like '["cox.com"]` otherwise the '.' will create a nested table rather than being added to the domain name.
error resolving MX for cox: no record found for Query { name: Name("cox."), query_type: A, query_class: IN }. Ignoring the shaping config for that domain.
```
Augments our queue name format to be
`campaign:tenant@domain!routing_domain`.
The routing_domain is optional. If the routing_domain is not set, its
effective value is that of the recipient domain.
You can `msg:set_meta('routing_domain', 'bar.com')` to set the
routing_domain for a message, so if the original recipient was
`user@foo.com`, that would cause the computed queue name for it to be
`foo.com!bar.com`.
The routing_domain is used when deciding on the ready_queue name
and destination MXs, so continuing our example, instead of resolving
`foo.com` MX records we'd resolve `bar.com` and deliver to that site.
The `get_egress_path_config` event `domain` parameter is redefined to be
the effective `routing_domain`.
The `get_queue_config` event `domain` parameter is the regular recipient
domain. The `routing_domain` is not currently made available to
`get_queue_config`. If/when we expose it, it will likely be via a
queue name object instead of adding an additional parameter. That would
be a breaking change.
The consequence of not exposing this parameter is that per-message
routing scenarios for the same domain (but different routing domains)
cannot vary the scheduled queue parmeters (eg: retry intervals). Even
though they would have separate scheduled queue instances, those
instances would have the same scheduled queue parameters. If you need
to be able to do that, then explicitly setting the domain portion of the
queue name would be a way to do that: `msg:set_meta('queue',
'foo.com-via-bar.com!bar.com')`. `get_queue_config` would then be
called with `domain='foo.com-via-bar.com'` and your policy could then
respond accordingly.
Previously, you would do either:
`msg:set_meta('queue', 'smart.host.domain')`
or
`msg:set_meta('queue', '[10.0.0.1]')`
to override the effective domain for a message and cause it to be routed
to somewhere other than the recipient domain.
That was OK for basic smart hosting, but limiting when you wanted to use
multiple candidate hosts.
This commit expands the queue config `protocol` field to support
specifying an explicit list of MX hosts that should be used instead.
The integration tests have been migrated away from the old style to this
new style.
While adding plumbing for this, I uncovered an inconsistency between the
queue name generated for the ready queue and the name used by suspension
handling. The inconsistency was introduced in
0842a0fc8b and related work. This commit
resolves it.
Use the pause emoji for suspensions, and the wastebasket emoji for
bounces. These are shown in the final column of the respective
sections.
Note that for bounces there will only be a short time window where you
will see a bounced domain show up in the list because the bounce will
remove it from the system fairly quickly.
There were a few cases where we'd drain the fifo and then explicitly
set the metrics counter to 0. Critically, the drain and the counter
update were not atomic wrt. other actors that might be inserting
data into the queue, so there was potential to race and perturb
the ready queue count.
This commit avoids every unilaterally setting the new value in
the ready queue module, and instead factors out the drain operation
to collect the messages and then adjust the count by the number
removed.
Allows filtering the results to just those queues associated
with the requested domain. Uses site names to map domains
to their associated ready queues.
Since the refactoring that made ready_queue more abstract,
I noticed that every dispatcher implementation was reported
as `smtp_client`, which is a bit non-sensical for lua and maildir.
This commit tweaks that so that we roll those up as `smtp_client`,
`lua` and `maildir` in the metrics.
To facilitate more dynamically updating the configuration, this commit:
* Introduces a ConfigHandle type to aid in building shared configuration
objects that don't require full mutex interlock
* Switches ReadyQueue and Dispatcher to hold egress path config in a ConfigHandle
* ReadyQueue maintainer will now refresh, by calling
get_egress_path_config, the value in the config handle
* shaping.lua now uses a ttl of 1 minute (which is the same as the
ReadyQueue maintainer interval), so that the ready queues should
reflect egress path configuration changes approximately every minute.
The simplest way to convey suspension status is via a config override.
So let's allow configuring a path as suspended and deliver that
alongside the other configuration shared by the tsa-daemon.
Some additional work is needed in kumod to ensure that we reload
the path config while the ready queue is in existence; that will
occur in a follow up commit.
This produces commented shaping.toml output corresponding to any config
overrides set by the automation.
For example, this non-sensical rule that applies to every domain:
```toml
[["default".automation]]
regex = "250 2\\.0\\.0 Ok"
action = {SetConfig={name="max_connection_rate", value="100/s"}}
trigger = {Threshold="2/hr"}
duration = "30 secs"
```
when triggered for messages sent to my own domain:
```console
$ curl -s 'http://localhost:8008/get_config_v1'
# Generated by tsa-daemon
# Number of entries: 1
["wezfurlong.org"]
["wezfurlong.org".sources]
["wezfurlong.org".sources.unspecified]
mx_rollup = false
# reason: automation rule: 250 2\.0\.0 Ok
# expires: 2023-08-03T01:47:37+00:00
max_connection_rate = "100/s"
```
This is using an in-memory sqlite db to keep track of events
and actions that we trigger.
In the future, the db will be persisted at a configurable location
on local storage.
Next step is to add endpoints for both config overrides and suspensions
that can be consumed by the mta nodes.
Include `@protocol-info` on the end of the ready queue names.
This prevents surprising false sharing of custom delivery protocols
when returning different protocols based on the tenant or campaign
metadata.
This is visible in the stats:
```json
"total_connection_count": {
"help": "total number of active connections ever made",
"type": "counter",
"value": {
"service": {
"smtp_client": 1.0,
"smtp_client:source2->(in1-smtp|in2-smtp).messagingengine.com@smtp": 1.0
}
}
},
```
and perhaps unexpectedly in the `site_name` field of the json log
records. That will need to get tidied up in a separate commit.
allows stuff like:
```toml
[["default".automation]]
regex = "250 2\\.0\\.0 boop"
action = {SetConfig={name="max_connection_rate", value="100/s"}}
trigger = {Threshold="2/hr"}
duration = "2 hours"
```
This commit only enables parsing this information; no action is
taken on it at this time.
I think something got screwed up somewhere, because I'm sure this
used to work, but: the value wasn't being parsed out of the loaded
data.
Add an integration test to assert that we can load things.
This test may need some auto-detection to run successfully on CI.
Let's see what happens.
This allows us to avoid rebuilding the map on each message reception
when using the dkim helpers, which should improve the performance
for sites with large numbers of signing domains.
Wez Furlong