Commit Graph
782 Commits
Author SHA1 Message Date
Wez Furlong fd4aeb00a2 dkim_sign.lua: fixup missing vault_path_prefix usage 2023-06-09 11:47:27 -07:00
Wez Furlong 5001095044 dkim: sketch out an entirely untested dkim signing helper 2023-06-09 11:43:05 -07:00
Wez Furlong bcfba07dad dkim: switch to my fork of cfdkim for RSA signatures
That fork removes the lifetime from the Signer type which makes
it easier to cache more efficiently.
2023-06-09 06:46:05 -07:00
Tom Mairs 756405d133 update dkim docs 2023-06-09 06:07:47 +00:00
Tom Mairs 130b419168 clean up files and add notes 2023-06-08 22:09:40 +00:00
Wez Furlong e9e158039f ci: trigger pkg build for policy changes 2023-06-08 14:48:19 -07:00
Wez Furlong b930ee5ec1 shaping.toml: remove unresolvable protection.outlook.com domain
entries in this file must be resolvable domains, and this one
has neither A nor MX records
2023-06-08 14:39:27 -07:00
Wez Furlong 88fd825be4 dkim: add cfdkim as a runtime alternative RSA signer
The mail-auth crate doesn't support 1024 bit RSA keys, which are
currently the dominant size of key used by senders at scale.

It is not possible to even configure a 2048 bit RSA key in some DNS
providers, which blocks widespread adoption of larger key sizes.

This commit pulls in the cfdkim crate as an alternative implementation.

I'm not happy with this implementation because the API of the crate
defeats some caching optimizations, doesn't support certain signing
attributes that are supported by mail-auth and the generated header isn't
nicely wrapped.

We'll likely "do something" to resolve this in a future commit.

For now, this implementation is enabled by setting `use_cf = true`
in the signing parameters.
2023-06-08 14:29:19 -07:00
Wez Furlong 887bb7e9e4 update rocksdb dep to fix build with gcc 13
refs: https://github.com/facebook/rocksdb/pull/11118
2023-06-08 13:29:53 -07:00
Wez Furlong ad70b6ac33 cargo update 2023-06-08 13:29:52 -07:00
Tom Mairs 6516f895e3 fix keygen details 2023-06-08 20:06:57 +00:00
Wez Furlong f11511031a dkim: refine error message
I couldn't find definitive evidence that rsa-1024 has been broken,
but I did find some cases where eg: GPG had a side-channel attack
that could break PGP keys of that size, so it's easy to imagine
that other similar approaches might work for other software.

Regardless, the actual limitation comes from the Rust ring crate:

https://docs.rs/ring/latest/ring/signature/struct.RsaKeyPair.html#method.from_pkcs8

> Only two-prime (not multi-prime) keys are supported. The public modulus
> (n) must be at least 2047 bits. The public modulus must be no larger
> than 4096 bits. It is recommended that the public modulus be exactly
> 2048 or 3072 bits. The public exponent must be at least 65537.

Refine the error message to be more factual.
2023-06-08 10:32:42 -07:00
Wez Furlong ac913fad29 dkim: improve error messaging when loading a key fails 2023-06-08 10:13:58 -07:00
Tom Mairs 13ab3593ec edit DKIM generation details 2023-06-08 16:06:48 +00:00
Wez Furlong 8625d9d182 ship proxy-server in the kumo rpms and docker image
In the future I want to add separate packages for just the proxy
server, but for now this is the most expedient way to deliver
the binary.

refs: https://github.com/KumoCorp/kumomta/issues/45
2023-06-07 15:24:06 -07:00
Wez Furlong 22451330f4 CLI: add kcli command for interacting with admin http endpoints
```console
$ kcli --help
KumoMTA CLI.

Full docs available at: <https://docs.kumomta.com>

Usage: kcli --endpoint <ENDPOINT> <COMMAND>

Commands:
  bounce
          Administratively bounce messages in matching queues
  set-log-filter
          Changes the diagnostic log filter
  help
          Print this message or the help of the given subcommand(s)

Options:
      --endpoint <ENDPOINT>
          URL to reach the KumoMTA HTTP API

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version
```

```console
$ kcli bounce --help
Administratively bounce messages in matching queues.

Each individual message that is bounced will generate a log record capturing the event and then be removed from the spool.

Make sure that you mean it, as there is no going back!

The bounce will be applied immediately to queued messages, and the directive will remain in effect for the duration specified, causing newly received messages or messages that were in a transient state at the time the directive was received, to also be bounced as they are placed back into the matching queue(s).

Usage: kcli --endpoint <ENDPOINT> bounce [OPTIONS] --reason <REASON>

Options:
      --domain <DOMAIN>
          The domain name to match. If omitted, any domains will match!

      --campaign <CAMPAIGN>
          The campaign name to match. If omitted, any campaigns will match!

      --tenant <TENANT>
          The tenant name to match. If omitted, any tenant will match!

      --reason <REASON>
          The reason to log in the delivery logs

      --everything
          Purge all queues

      --duration <DURATION>
          The duration over which matching messages will continue to bounce. The default is '5m'

  -h, --help
          Print help (see a summary with '-h')
```

```console
$ kcli set-log-filter --help
Changes the diagnostic log filter

See <https://docs.kumomta.com/reference/kumo/set_diagnostic_log_filter/> for more information about the log filter syntax.

Usage: kcli --endpoint <ENDPOINT> set-log-filter <FILTER>

Arguments:
  <FILTER>

Options:
  -h, --help  Print help
```

refs: https://github.com/KumoCorp/kumomta/issues/55
2023-06-07 12:09:07 -07:00
Mike Hillyer 2c5f87ad97 Updated changelog. 2023-06-01 17:26:57 -04:00
Mike Hillyer d99217a8d3 Automated doc formatting fix 2023-06-01 21:19:52 +00:00
Mike Hillyer 63af89ed47 Update example policy and sections on sending IPs and Listeners to use the new policy helpers. 2023-06-01 17:18:57 -04:00
Wez Furlong 7db243025e add policy helper for sources and pools 2023-06-01 13:08:05 -07:00
Tom Mairs 092c972b42 requires sudo 2023-06-01 19:48:39 +00:00
Tom Mairs ba2661e70c fix case in docker command 2023-06-01 19:42:55 +00:00
Wez Furlong e8a049cc83 document listener parameter to get_listener_domain update helpers
Allow per-listener listener-domain configuration
2023-05-31 15:46:12 -07:00
Wez Furlong ec85c2da6f gracefully shutdown for SIGHUP, SIGTERM as well as SIGINT
Systemd doesn't terminate processes using SIGINT, so we didn't
get a chance to flush logs or messages to spool.

This should address that.

refs: https://github.com/KumoCorp/kumomta/issues/46
2023-05-31 12:45:36 -07:00
Wez Furlong 9708bbb1de fix more typos 2023-05-31 12:44:46 -07:00
Wez Furlong 97f5abfcf0 fix typo 2023-05-31 12:34:02 -07:00
Wez Furlong ce7dece927 move domains out of listener, replace with get_listener_domain event
This commit also includes a policy helper
`policy_extras.listener_domains` to make it convenient to define
listener domains in toml and/or json files.

To facilitate this, the DomainMap rust structure has been exposed
to lua code via the new `kumo.domain_map.new` function.
2023-05-31 12:29:44 -07:00
Wez Furlong b7c557c31a define_egress_(pool|source) -> make_egress_(pool|source)
Switch the configuration plumbing for pools and sources to be pull-based
rather than push based.

In other words, rather than defining them in the `init` event,
you now need to supply them to the new `get_egress_pool` and
`get_egress_source` events.

Data is cached by default for 1 minute. This allows for new sources
to come into being on-demand, and for data to age out and change
over time, without requiring that the server be restarted.

This commit updates the reference section, but there is some content
in the user guide that refers to the old style of configuration that
will need to be updated.

refs: https://github.com/KumoCorp/kumomta/issues/13
2023-05-30 14:08:07 -07:00
Mike Hillyer 38290015a6 Update changelog. 2023-05-30 14:23:08 -04:00
Wez Furlong 81a4eba5a2 logging: Explicitly clear and flush the file map before completing
Otherwise we can race on shutdown and the process may exit without
fully flushing the logs!

refs: #46
2023-05-30 11:01:14 -07:00
Wez Furlong 9b1d82275c add spool_message_enumerated event
closes: #42
2023-05-30 10:14:37 -07:00
Wez Furlong ec6a7d83c9 dns: use fully qualified name when resolving A/AAAA records
We were so close, yet so far!

closes: https://github.com/KumoCorp/kumomta/issues/49
2023-05-30 09:57:01 -07:00
Mike Hillyer 0eefbccd5d Typo fix. 2023-05-18 16:44:32 -04:00
Tom Mairs 10521350c7 fix typos 2023-05-18 05:05:12 +00:00
Tom Mairs f9afc272d4 Update shaping.lua (#47)
Fix shaping.toml location
2023-05-17 22:50:08 -06:00
Mike Hillyer 1e05c7bb31 Automated doc formatting fix 2023-05-17 21:40:38 +00:00
Mike Hillyer 0d85aea133 Adding SMTP AUTH to the user guide. 2023-05-17 17:39:23 -04:00
Mike Hillyer 842c971811 Formatting fix. 2023-05-17 17:39:23 -04:00
Mike Hillyer 7cbf93e316 Automated doc formatting fix 2023-05-17 19:58:41 +00:00
Mike Hillyer c03cf00408 Initial build out of SOCKS5 docs in userguide, converting HAProxy section into general proxy section. 2023-05-17 15:57:22 -04:00
Mike Hillyer 9780e3f3a5 Updating logic per Wez. 2023-05-16 16:43:40 -04:00
Mike Hillyer cd7dbe7aec Automated doc formatting fix 2023-05-16 19:12:58 +00:00
Mike Hillyer bd1d7a1d03 This timne for sure. 2023-05-16 15:11:39 -04:00
Mike Hillyer f26551a205 Too many typos. 2023-05-16 15:10:17 -04:00
Mike Hillyer b741a1ab06 Typo 2023-05-16 15:06:38 -04:00
Mike Hillyer 297895e85d Adding AMQP to the User Guide. 2023-05-16 15:05:09 -04:00
Wez Furlong 536827db99 make it an error to redefine an event handler
Produces an error like:

```
; KUMOD_LOG=kumod=info ./target/debug/kumod --policy simple_policy.lua
2023-05-16T18:16:02.112017Z  INFO main kumod::memory: using limits: soft=Some("101.22 GB"), hard=Some("134.96 GB")
Error: callback error
stack traceback:
        [C]: in function 'kumo.on'
        [string "simple_policy.lua"]:232: in main chunk
caused by: smtp_server_auth_plain event already has a handler defined at simple_policy.lua:226
```
2023-05-16 11:16:37 -07:00
Wez Furlong acef2ecab9 Add timeouts to socks server handshake/connection initiation
Let's avoid things hanging forever while setting up a connection.

refs: https://github.com/KumoCorp/kumomta/issues/45
2023-05-15 17:55:28 -07:00
Wez Furlong f21b7c53aa Add socks5 client support
Enable it via:

```lua
kumo.define_egress_source {
  name = "my source",
  socks5_proxy_server = '127.0.0.1:1080',
  socks5_proxy_source_address = '192.168.1.92',
}
```

couple it with:

```console
; RUST_LOG=proxy_server=trace cargo run -p proxy-server -- --listen 127.0.0.1:1080
   Compiling proxy-server v0.1.0 (/home/wez/wez-personal/kumomta/crates/proxy-server)
    Finished dev [unoptimized + debuginfo] target(s) in 1.30s
     Running `target/debug/proxy-server --listen '127.0.0.1:1080'`
[2023-05-11T23:54:56Z INFO  proxy_server] proxy listener on 127.0.0.1:1080
[2023-05-11T23:57:53Z TRACE proxy_server::proxy_handler] peer=127.0.0.1:49824 request: SocksV5Request { command: Bind, host: Ipv4([192, 168, 1, 92]), port: 0 }
[2023-05-11T23:57:53Z TRACE proxy_server::proxy_handler] peer=127.0.0.1:49824: status -> RequestStatus { status: Success, host: Ipv4([0, 0, 0, 0]), port: 0 }
[2023-05-11T23:57:53Z TRACE proxy_server::proxy_handler] peer=127.0.0.1:49824 request: SocksV5Request { command: Connect, host: Ipv4([103, 168, 172, 220]), port: 25 }
[2023-05-11T23:57:53Z TRACE proxy_server::proxy_handler] peer=127.0.0.1:49824: status -> RequestStatus { status: Success, host: Ipv4([192, 168, 1, 92]), port: 54052 }
[2023-05-11T23:57:53Z TRACE proxy_server::proxy_handler] peer=127.0.0.1:49824 -> going to passthru mode
```

refs: https://github.com/KumoCorp/kumomta/issues/45
2023-05-15 17:38:17 -07:00
Wez Furlong beb120fda5 Add a SOCKS5 proxy server implementation
This is a pretty tight, lightweight implementation of SOCKS5
intended to be used to facilitate an MTA asking it to bind and
connect.

It uses the splice(2) syscall to avoid kernel/userspace copies
when passing data through, so it should be quite efficient.

refs: https://github.com/KumoCorp/kumomta/issues/45
2023-05-15 17:38:17 -07:00