mirror of
https://github.com/mailscope/kumomta.git
synced 2026-08-21 11:58:17 +00:00
Wez Furlong
73d811b8ef
The back story here is that we were investigating a report of get_egress_path_config sometimes taking a long time (30-60 seconds!) to complete. The issue correlates with a large number of TSA entries that disable the use of TLS for broken sites (>30,000 entries), and some kind of DNS resolution. Running the sample of the TSA output through validate-shaping on my local system can take 30+ seconds to resolve, with a handful of the entries timing out after 15 seconds (which is 5 seconds timeout on a query, plus 2 retries of 5 seconds each). In this sort of situation with a persistently slow resolve on specific domains, the best case result is therefore 15 seconds, but it could be longer depending on the ordering of the results and prevailing winds on the upstream dns server. What's tricky in this situation is that we have to resolve the site name for each of the entries that have mx_rollup=true enabled in order to correctly match any given domain when satisfying the get_egress_path_config event callout. Being lazy with resolution doesn't help at load time because we're only loaded immediately prior to invoking this event. The next logical question to ask is: why do we need to have 30,000+ entries to disable TLS for these broken sites--can't we just retry in clear text? Well, it's a bit tricky: we're not guaranteed to be able to continue in clear text after STARTLS fails. If we were to disconnect and reconnect to the same host immediately after, that's technically 2 connections being made in quick succession to the remote site, and that might have reputation influencing consequences. I'd like to avoid adding that sort of reconnection logic without providing a way to control that behavior, and that is something for a separate change from this one. So, looking at the nature of the problematic rule, it doesn't actually need to be keyed by the site name because there are not shared limits or constraints that it might make sense to apply, and in the context of this issue, it's not necessary to use rollup. This commit introduces a `SetDomainConfig` action that will emit a config entry that explicitly sets `mx_rollup=false` regardless of the value of `mx_rollup` for the defining rule condition. When mx_rollup=false, we do not need to resolve the MX record for the entry when we load the shaping rules, which eliminates the DNS latency for the aggregate set of domains with broken TLS. It would be great if we could automagically fixup the TSA config database when updating to this version, but since the rule is an optional part of the suggested shaping configuration, which may not even be loaded by a given overall configuration, it's not appropriate to assume that we should do any fixup. There's likely a one liner that can be run in sqlite to remove or force the expiration of the old rules.