I kinda lobbed things over the wall in df08d6bd15.
You can't strip a shell script.
And we need to explicitly add files to the list in the rpm spec file
Adds expanded constructor method that allows bypassing the hook.
Exposes the config resolver method on the returned queue helper object.
Co-authored-by: ncai <ncai@chapsvision.com>
Co-authored-by: Wez Furlong <wez@wezfurlong.org>
I didn't register this correct for multi-event setup,
so no pre_init event was ever triggered, causing the log_hooks
stuff to not register the log hook.
The way we test whether logs should continue is by looking for
an empty return statement, so we shouldn't `return nil` for these
otherwise we'll indicate that we definitively can't resolve the
queue configuration.
Allow multiple should_enqueue_log_record hooks to be registered,
and take advantage of that inside the shaping helper.
`shaper.should_enqueue_log_record` no longer needs to be explicitly
plumbed from the config, but we allow calling it still for compatibility
reasons.
We will remove that compatibility after the next stable release.
Take advantage of the ability to define multiple get_queue_config
callbacks by internalizing the TSA webhook queue configuration.
`shaper.get_queue_config` no longer needs to be called, and
that fragment of code will continue to "work" by simply not
having an effect so that we don't break folks on upgrade.
We'll remove this compatibility in the release following
the next stable release.
Avoid referencing implementation details that we may change
(install.sh).
Document what goes where with a bit of explanation.
Remove examples that show launching `kumod &` as we do not recommend
running the service in the background without some kind of supervision
(such as systemd). Instead, show how how to run the process and leave
it to the reader to decide how they with to deploy it in their
environment.
closes: https://github.com/KumoCorp/kumomta/issues/72
New users trip over the very conservative defaults on log flushing,
so let's make the `init.lua` that we populate for new installs
set the duration to 10 seconds by default, and adjust the language
in the docs to match that.
Clarify the logging documentation about how the logs are
chunked/flushed.
use the drone cli to check that the starlark is well-formed,
and to try to run a build.
Useful for sanity checking a pending CI commit.
Slim down the deps very slight on ubuntu systems.
The drone plugin does stuff very strangely and is incompatible
with our approach.
Do it ourselves (with blackjack and ...), so that we can avoid
cloning and bloating the repo.
These were for working around constraints in GH actions and since
we aren't using it anymore for the main portion of our CI/CD,
we don't need them anymore.
How it works:
* When the lowest preference MX host names match a pattern like
`.mail.protection.outlook.com`, the message has its routing_domain
set to a placeholder domain whose name ends with `.ip_rollup`.
* That results in a scheduled queue name like
`foo.com!outlook.ip_rollup`, which makes it possible to know both the
original domain and the fact that rollup is in use.
* `get_queue_config` can check to see if the routing_domain is set to
something that ends with `.ip_rollup` to override the `mx_list`
in the queue configuration with just the lowest preference IP
addresses from the original domain.
* Now, instead of computing a site_name base around
`foo-com.mail.protection.outlook.com`, which includes the individual
original recipient domain, and would cause there to be a separate
ready queue for each domain, the overridden mx_list causes
the site_name to be eg: `mx_list:[104.47.24.36],[104.47.25.36]`.
That same site_name will be used for every domain that shares those
same IP addresses
* When `get_egress_path_config` is called to get shaping parameters,
it is passed the routing domain `outlook.ip_rollup`. You can use that
name with mx_rollup=false as the key in your shaping.toml, of if you
are directly implementing `get_egress_path_config`, you can use that
name to determine the appropriate configuration.
```lua
kumo.on('get_queue_config', function(domain, tenant, campaign, routing_domain)
local params = {}
rollup.apply_ip_rollup_to_queue_config(domain, routing_domain, params
return kumo.make_queue_config(params)
end)
kumo.on('smtp_server_message_received', function(msg)
rollup.reroute_using_ip_rollup(msg, {
['.mail.protection.outlook.com.'] = 'outlook.ip_rollup',
})
end)
```
In your shaping.toml:
```toml
["outlook.ip_rollup"]
mx_rollup = false
# shaping parameters here
```
Caveats:
* With this technique, we'll never try to use any of the lower
priority/higher preference value MX records for any of the matching
domains.
* The IP addresses to which the MX host names resolve can vary over time.
We'll still queue the mail to the same scheduled queue (eg:
`foo.com!outlook.ip_rollup`), but it's possible for there to be
multiple ready queues with different names based on those changed
IPs. This is actually a feature: if the destination domain has
an outage and are now publishing different IPs, we'll pick those up
and use them.
* Since the ready queue names look like `mx_list:[104.47.24.36],[104.47.25.36]`
it can be hard to intuit just from glancing at that name where those queues go.
This allows listing out authorization identities.
If the incoming client is authorized to act as one of the listed
`relay_from_authz` authorization identities, then the incoming
session will be added to the relay_from CIDR for the scope of
that connected session, and the result is that they will be allowed
to relay from the associated domain.
This is useful if you want to give a tenant SMTP auth credentials
but only want to allow them to send from the domain(s) that
they are permitted to send as/from.
This is a prototype of a means for dealing with rolling up microsoft's
domains so that shaping can be applied holistically over both eg:
hotmail.com and outlook.com.
Those domains don't work with our default automatic MX based rollup
because they don't return the same MX hostnames.
Checking this in to make it easier for others to test/evaluate.
In the future, we may find a way to integrate this with our main
shaping helpers.
To facilitate more dynamically updating the configuration, this commit:
* Introduces a ConfigHandle type to aid in building shared configuration
objects that don't require full mutex interlock
* Switches ReadyQueue and Dispatcher to hold egress path config in a ConfigHandle
* ReadyQueue maintainer will now refresh, by calling
get_egress_path_config, the value in the config handle
* shaping.lua now uses a ttl of 1 minute (which is the same as the
ReadyQueue maintainer interval), so that the ready queues should
reflect egress path configuration changes approximately every minute.
Make use of our new ability to memoize the compiled domain map
so that we can avoid rebuilding it on each lua hook invocation.
This should improve performance for sites with large numbers of
listener domains.
Wez Furlong
Mike Hillyer
drone