This commit causes the scheduled queue maintainer to refresh
the queue config by calling the get_queue_config event approximately
every minute while the queue is alive.
In addition, we now thread the routing_domain through to get_queue_config
You may now remove or replace the test domain suffix from the set
of senders/recipients by usingn `--domain-suffix ''` to remove
it, or some other value to set it to something else.
`--domain` can now be used multiple times to build out the list
of test domains.
We had a user get tripped up by the incorrect example on this page.
I think they probably should have been using the helper, but the
way this page was constructed, it was easy to keep reading past
the bit about the helper and get bogged down by the lua examples.
Rearrange this page to try to avoid that.
I've seen this trip up at least two people so far.
While we're in there, update the example to show how to memoize and make
it both easier to write and more efficient at runtime.
This allows listing out authorization identities.
If the incoming client is authorized to act as one of the listed
`relay_from_authz` authorization identities, then the incoming
session will be added to the relay_from CIDR for the scope of
that connected session, and the result is that they will be allowed
to relay from the associated domain.
This is useful if you want to give a tenant SMTP auth credentials
but only want to allow them to send from the domain(s) that
they are permitted to send as/from.
A couple of scenarios we're shooting for here:
* Startup ordering races, or other "burps" due to eg: restarting the
tsa daemon to update its config
* For "replication" scenarios, you may be running multiple instances
of tsa-daemon on multiple nodes, and list them all in both the
publish and subscribe lists on the clients.
In that situation the client will try to read from each node;
the data its reads should be essentially the same from each of them,
and it is fine if only one out of the set responds, or honestly,
even if none of them respond in the moment, as we'll eventually
be able to read that data.
Errors making those http requests are logged to the diagnostic log,
but the shaping data is otherwise allowed to load.
We now generate more context-rich error messages to help pinpoint
the problem:
```
caused by: failed to parse '10.0.0.1/24' as CIDR notation: host part of address was not zero
```
This is a prototype of a means for dealing with rolling up microsoft's
domains so that shaping can be applied holistically over both eg:
hotmail.com and outlook.com.
Those domains don't work with our default automatic MX based rollup
because they don't return the same MX hostnames.
Checking this in to make it easier for others to test/evaluate.
In the future, we may find a way to integrate this with our main
shaping helpers.
It's easy to typo this and get surprised by the results, so explicitly
check for it and provide a more actionable message than the DNS
resolution error for `cox.` would imply:
```
Entry for domain 'cox' consists of a single DNS label. Domain names in TOML sections need to be quoted like '["cox.com"]` otherwise the '.' will create a nested table rather than being added to the domain name.
error resolving MX for cox: no record found for Query { name: Name("cox."), query_type: A, query_class: IN }. Ignoring the shaping config for that domain.
```
Augments our queue name format to be
`campaign:tenant@domain!routing_domain`.
The routing_domain is optional. If the routing_domain is not set, its
effective value is that of the recipient domain.
You can `msg:set_meta('routing_domain', 'bar.com')` to set the
routing_domain for a message, so if the original recipient was
`user@foo.com`, that would cause the computed queue name for it to be
`foo.com!bar.com`.
The routing_domain is used when deciding on the ready_queue name
and destination MXs, so continuing our example, instead of resolving
`foo.com` MX records we'd resolve `bar.com` and deliver to that site.
The `get_egress_path_config` event `domain` parameter is redefined to be
the effective `routing_domain`.
The `get_queue_config` event `domain` parameter is the regular recipient
domain. The `routing_domain` is not currently made available to
`get_queue_config`. If/when we expose it, it will likely be via a
queue name object instead of adding an additional parameter. That would
be a breaking change.
The consequence of not exposing this parameter is that per-message
routing scenarios for the same domain (but different routing domains)
cannot vary the scheduled queue parmeters (eg: retry intervals). Even
though they would have separate scheduled queue instances, those
instances would have the same scheduled queue parameters. If you need
to be able to do that, then explicitly setting the domain portion of the
queue name would be a way to do that: `msg:set_meta('queue',
'foo.com-via-bar.com!bar.com')`. `get_queue_config` would then be
called with `domain='foo.com-via-bar.com'` and your policy could then
respond accordingly.
Previously, you would do either:
`msg:set_meta('queue', 'smart.host.domain')`
or
`msg:set_meta('queue', '[10.0.0.1]')`
to override the effective domain for a message and cause it to be routed
to somewhere other than the recipient domain.
That was OK for basic smart hosting, but limiting when you wanted to use
multiple candidate hosts.
This commit expands the queue config `protocol` field to support
specifying an explicit list of MX hosts that should be used instead.
The integration tests have been migrated away from the old style to this
new style.
While adding plumbing for this, I uncovered an inconsistency between the
queue name generated for the ready queue and the name used by suspension
handling. The inconsistency was introduced in
0842a0fc8b and related work. This commit
resolves it.
This should create a release from a tag and populate it with
a basic changelog.
We'll see if it actually works when we cut the next release!
refs: https://github.com/KumoCorp/kumomta/issues/71
Use the pause emoji for suspensions, and the wastebasket emoji for
bounces. These are shown in the final column of the respective
sections.
Note that for bounces there will only be a short time window where you
will see a bounced domain show up in the list because the bounce will
remove it from the system fairly quickly.
There were a few cases where we'd drain the fifo and then explicitly
set the metrics counter to 0. Critically, the drain and the counter
update were not atomic wrt. other actors that might be inserting
data into the queue, so there was potential to race and perturb
the ready queue count.
This commit avoids every unilaterally setting the new value in
the ready queue module, and instead factors out the drain operation
to collect the messages and then adjust the count by the number
removed.
Allows filtering the results to just those queues associated
with the requested domain. Uses site names to map domains
to their associated ready queues.
Wez Furlong
Mike Hillyer