Commit Graph
29 Commits
Author SHA1 Message Date
Wez Furlong a412757513 Add smart-sink example docker compose
Adds an example docker compose file that can be used to quickly
stand up an SMTP sink service that is useful when load testing
an SMTP server.

The sink has configurable and "realistic" bounce response rates,
so that you can excercise queue growth scenarios.
2024-09-10 08:00:06 -07:00
Wez Furlong 50681b149c spool-util: add initial version
This isn't shipped in the packages at this time.
It's a utility for post-mortem or offline analysis
of kumomta spools.

This version assumes rocksdb, but could be tweaked to work
with a plain filesystem spool.

We could add functions in the future to migrate from one format
to another, or perform other sorts of maintenance/admin operations.
2024-08-08 07:02:23 -07:00
Wez Furlong 435b6cb476 add tls-probe utility
This is conceptually similar in purpose to running `openssl s_client`
to figure out the capabilities of a remote mx and its TLS
implementation.

This commit also adds a currently-deliberately-undocumented
tls_prefer_openssl option to the egress path config that forces
the selection of an openssl based client connector.

I want to try this out a bit before fleshing it out into a
larger product feature and its additional configuration.

RUST_LOG=trace ./tls-probe example.com:25
2024-07-19 05:52:36 -07:00
Wez Furlong 007e234d9e Add toml2jsonc utility to convert toml to json with comments
The intended purpose is to use to auto-generate json representations
of example toml data files for the documentation.

refs: https://github.com/KumoCorp/kumomta/issues/212
2024-07-03 11:53:59 -07:00
Wez Furlong f0b91dd652 queue.lua: add typing
I regret optimizing the structure for toml when this module was
created, as it makes it difficult to define a regular schema.

This commit introduces typed records for the queue helper
data structure, and, unfortunately, a parsing layer to adapt
between the toml data and the well-defined types.

It also adjusts how lua tests are run; previously we'd co-opt the main
flow of parsing by looking at an env var, but since the modules can now
potentially require each other (especially the typing module) we need to
de-couple from that, otherwise what happens is that we'd only ever run
the typing module tests and exit. So we now have a script that imports
all the modules and runs their respective `:test()` methods.

refs: #211
2024-06-28 08:54:46 -07:00
Wez Furlong af99c0bb75 run-lua-test: factor out of Makefile
This makes the Makefile less verbose/boilerplate-y
2024-06-27 05:51:11 -07:00
Wez Furlong b9dccf1cc3 add policy-extras.typing lua module
The primary purpose here is to allow defining typed records
for use in our helpers.

The dkim_sign module has been updated to make use of this
to ensure that the correct shape of data has been loaded.
2024-06-26 16:38:22 -07:00
Wez Furlong c9b98adc10 ci: cache target dir directly 2024-06-12 12:19:29 -07:00
Wez Furlong c5428dd5c8 ci: add xcp (parallel cp) to builder image 2024-06-12 09:15:22 -07:00
Wez Furlong 3e97cd3106 optimize the fast path for queue insertion
Avoid doing some work in the fast path, and accept that we might do
some redundant extra work in the slow path, in order to make the
fast path faster.
2024-05-01 22:36:26 -07:00
Wez Furlong a230e78ce6 add helper script for analyzing stack traces 2024-04-03 07:54:09 -07:00
Wez Furlong e94506fd96 Add new Rejection log event
We skip logging the 421 we generate while shutting down because
it feels a bit redundant; you'll see the server shutting down
in the journal anyway.

refs: https://github.com/KumoCorp/kumomta/issues/88
2024-03-29 16:37:37 -07:00
Wez Furlong 15b1fab110 checkpoint some local testing/hacking in my test configs 2024-03-27 16:33:17 -07:00
Wez Furlong f6f8c55ea4 listener_domains: fix fallback issue
This commit refactors listener_domains.lua to facilitate unit testing
and adds a couple of basic test cases.

The functional change here is that we were missing an explicit
fallback step in the case where no listener or domain matches
the provided values directly; we need to explicitly add a check
against the `*` listener AND `*` domain for the final step.
Previously, we would only look at the `*` listener for the final
step.

closes: #128
2024-03-22 09:48:18 -07:00
Wez Furlong a075e97e8d fixup make test to pick up username even if $USER is unset 2024-03-21 16:23:26 -07:00
Wez Furlong a145100105 fix running new lua unit tests when run as root 2024-03-21 12:19:25 -07:00
Wez Furlong 8ef3af530a add overall_max_message_rate option to queue helper
The intent is to limit the overall rate at which a tenant
is able to send messages out from the server. It is NOT related
to the rate at which messages can be injected to ths server.

This is allowed in two locations:

* Directly in the tenant block
* In a new campaigns table located under a tenant

```toml
[tenant.'mytenant']
egress_pool = 'tpool'
overall_max_message_rate = "100/s"

[tenant.'mytenant'.campaigns.'mycampaign']
overall_max_message_rate = "50/s"
```

The tenant level throttle is checked first, and if it permits
the message, then the campaign specific throttle is checked.

refs: https://github.com/KumoCorp/kumomta/issues/143
2024-03-21 09:16:40 -07:00
Wez Furlong e6973c2a04 slim down some deps
This helps to remove "false" deps on things like openssl; now that
is linked in only to those components that need it (eg: things that
use unbound for dnssec, or the smtp client for DANE).

This makes the binaries other than kumod (which still has max deps)
a few MB smaller; not a massive win, but nice all the same.
2023-12-22 10:24:30 -07:00
Wez Furlong 837f9e5c0e automate updating the openapi spec files
I definitely forgot to do this on the very first change after
adding them, so I want to make it less likely in the future.

This causes the specs to get updated in the docs if they have
changed (other than the version field) since the last time they
were updated.

We check this during test and doc building.  That should be
sufficient to catch this during my dev loop.

refs: https://github.com/KumoCorp/kumomta/issues/96
2023-11-23 09:02:00 -07:00
Wez Furlong 9c0041b051 remove drone bits from makefile 2023-10-04 10:51:51 -07:00
drone f8674c3d63 Add woodpecker CI pipeline 2023-09-30 15:57:48 -07:00
Wez Furlong 620470f5aa ci: add local checker
use the drone cli to check that the starlark is well-formed,
and to try to run a build.

Useful for sanity checking a pending CI commit.

Slim down the deps very slight on ubuntu systems.
2023-09-19 09:54:54 -07:00
Wez Furlong 5d882096a9 return array of pipelines 2023-09-16 11:30:50 -07:00
Wez Furlong 257dc4a25a flesh out integration testing some more
This has our first end-to-end integration test that validates
smtp -> source mta -> sink mta -> maildir
and confirms that the message has the right bits inside it.

The end to end test will get refactored into more easily usable
pieces in follow-up commit(s).
2023-03-07 18:42:14 -07:00
Wez Furlong a05724e40c auto-format generate-toc.py too 2023-03-07 07:36:50 -07:00
Wez Furlong 79afc79794 add lua code formatting 2023-02-15 06:51:59 -07:00
Wez Furlong e388a2cc4f refine smtp sink targets 2023-02-14 17:13:32 -07:00
Wez Furlong 5422a5fa76 can now deliver mail 2023-02-14 16:11:24 -07:00
Wez Furlong 3287523d84 Skeleton tokio server for kumod 2023-02-10 16:44:10 -07:00