When we're running tests, we're generally running the debug build, so it
doesn't make sense to run the (potentially cached and not yet up to
date) release build.
These are primarily for my benefit; they make it a bit easier
to run kumod for one-off scripts without running the full service,
with less "noise" than you'd get for hacking around it as I was
doing previously in the resolve-site-name script and in the lua
test runner.
The idea is that you can now define a script like:
```lua
local kumo = require 'kumo'
kumo.on('main', function(domain)
local mx = kumo.dns.lookup_mx(domain)
print(mx.site_name)
end)
```
And (more) conveniently run it like this:
```
$ kumod --policy /path/to/script --script --script-args outlook.com
outlook-com.olc.protection.outlook.com
```
This will perform more explicit checks to make sure that the name
is defined and so on. It also now will check for conflicting
log hook names as well.
refs: #211
I regret optimizing the structure for toml when this module was
created, as it makes it difficult to define a regular schema.
This commit introduces typed records for the queue helper
data structure, and, unfortunately, a parsing layer to adapt
between the toml data and the well-defined types.
It also adjusts how lua tests are run; previously we'd co-opt the main
flow of parsing by looking at an env var, but since the modules can now
potentially require each other (especially the typing module) we need to
de-couple from that, otherwise what happens is that we'd only ever run
the typing module tests and exit. So we now have a script that imports
all the modules and runs their respective `:test()` methods.
refs: #211
The primary purpose here is to allow defining typed records
for use in our helpers.
The dkim_sign module has been updated to make use of this
to ensure that the correct shape of data has been loaded.
The purpose is to provide a deeper, offline validation pass
of the policy configuration, prior to deploying and making it
live.
The system behavior changes when in `--validate` mode:
* Listeners, spool and spawned tasks will be silently skipped;
the parameters will be validated but the primary functions
of those things will be skipped silently.
* After triggering the `init` event, an additional new `validate_config`
event (which can be registered multiple times) will be triggered
to allow lua modules to perform extended validation.
* A module can either raise an error via `error` to immediately report
a problem, or instead call a new, preferred, `kumo.validation_failed()`
function to flag validation as failed but allow additional validation
to be performed and summarized all together.
* Once the `validate_config` event returns, the process will terminate
with either exit code 0 for a successful validation, or non-zero
to indicate that something failed.
Validation errors are reported in a human readable form.
This commit adds validate_config event handlers for the following
helper modules:
* `shaping` - any warnings reported by the underlying rust code
will be reported here and cause validation to fail. This is
functionally equivalent to using the `validate-shaping` binary,
except that it will automatically be passed the set of shaping
files defined by your `init.lua`
If the `sources` helper is also configured, the list of sources
referenced by the shaping config will be cross-checked against
the sources data to confirm that all possible sources are defined.
* `sources` - each listed source and pool will be validated by
calling `kumo.make_egress_source` or `kumo.make_egress_pool`
respectively.
Pool membership will be validated to confirm that every
listed pool is defined in the sources data.
* `queues` - each domain and tenant that references an egress_pool
will be cross-checked with the `sources` helper, if the sources
helper has been configured.
It is now an error to attempt to setup any of the above helpers
more than once.
refs: https://github.com/KumoCorp/kumomta/issues/211
The guts of this hvae been shared on discord a couple of times.
Let's put this in a convenient script.
We should arrange to install this in our official packages,
but for now, let's get it in the repo.
We need to avoid recursively copying any symlinks-to-dirs as
directories. We can do that with cp, but the parallel xcp utility
doesn't support the --no-dereference flag at this time.
refs: https://github.com/tarka/xcp/issues/52
This is essentially a refactoring of the stuff that we've been running
on GH, but broken out to run on our woodpecker instance.
I've removed sccache from the image; it's actually been failing the
build on amazonlinux:2, but the GH action version of the build has
been showing up as green(!)
Adds a couple of options that provide more control over the default
timeouts and logging.
These are exposed to the shaping helper as `publish_timeout`,
`publish_pool_idle_timeout` and `publish_connection_verbose`.
while troubleshooting a situation where the
http://127.0.0.1:8008.tsa.kumomta queue is backing up, I noticed that
we're using the default queue configuration for this queue.
Let's make it more inline with the defaults for webhooks; give it a
1 minute base retry with a 20 minute max.
These parameters are configurable; you can pass in `tsa_queue_config` to
the setup_with_automation call to specify your preferred values for the
scheduled queue config.
We skip logging the 421 we generate while shutting down because
it feels a bit redundant; you'll see the server shutting down
in the journal anyway.
refs: https://github.com/KumoCorp/kumomta/issues/88
This commit connects the new websocket based suspension feed
up to shaping.lua. This allows ready-q suspensions to be
enacted in realtime, as well as sets things up to support
scheduled queue suspensions in a later commit.
refs: https://github.com/KumoCorp/kumomta/issues/113
This commit refactors listener_domains.lua to facilitate unit testing
and adds a couple of basic test cases.
The functional change here is that we were missing an explicit
fallback step in the case where no listener or domain matches
the provided values directly; we need to explicitly add a check
against the `*` listener AND `*` domain for the final step.
Previously, we would only look at the `*` listener for the final
step.
closes: #128
This commit allows:
```toml
[queue.'my.own.hostname']
routing_domain = '[10.0.0.1]'
```
The effect of this is that, if the recipient domain matches a `queue`
block with a `routing_domain`, the value of that entry will be assigned
as the `routing_domain` meta field of the message.
The intent is for this to be used when DNS routes mail to the current
instance, but for when it is desirable to relay the mail onto some
location that is not visible in DNS.
refs: https://github.com/KumoCorp/kumomta/issues/141
Wez Furlong
Mike Hillyer