Commit Graph
880 Commits
Author SHA1 Message Date
Wez Furlong 201e3e26c1 include peer address in TLS handshake error messages
closes: #64
2023-06-19 07:45:29 -07:00
Wez Furlong 81e64c65aa dkim: enable openssl rsa signing by default
This requires a fix to the docker build to avoid broken ssl
linkage.
2023-06-16 12:01:35 -07:00
Wez Furlong d60c961479 increase lua pool limits, allow configuring it 2023-06-16 10:03:49 -07:00
Wez Furlong cf2cccd953 get-deps.sh: fix install-from-source URL 2023-06-16 08:50:40 -07:00
Wez Furlong f7f4016a72 maildir-sink.lua: fix up max_segment_duration typo 2023-06-16 08:49:25 -07:00
Wez Furlong 45c5d011d1 simple_policy.lua: s/max_connections/connection_limit/
use the correct name for this field
2023-06-16 08:47:33 -07:00
Wez Furlong 1d5180a221 sources.lua: tolerate empty pool or source sections in a file
We can compose multiple files, so it is OK for there to not
be any sources or pools in any given file.
2023-06-16 08:46:35 -07:00
Wez Furlong a2feac004f actually add ehlo_domain to make_egress_source
Enable that field and add it to the fallback used by the
smtp_dispatcher.  The ehlo_domain configured for the pathway
takes precedence, then we'll take the value from the source,
then fallback to the local hostname.

refs: https://github.com/KumoCorp/kumomta/issues/59
2023-06-16 08:43:54 -07:00
Wez Furlong 21fc29bba8 make it a deserialize error when invalid fields are present
It turns out that ehlo_domain isn't valid for make_egress_source,
even though that is what we'd documented.

This commit surfaces this class of error visibly.

refs: https://github.com/KumoCorp/kumomta/issues/59
2023-06-16 08:33:39 -07:00
Wez Furlong 09895219e0 rfc5321 client: allow tracing SMTP dialog
If you set `rfc5321=trace` in the diagnostic log filter, we will
now trace the commands we send and their responses, making it
easier to diagnose the outbound SMTP state especially when TLS
is enabled.

refs: #59
2023-06-16 08:15:30 -07:00
Wez Furlong fb1df13447 shaping.lua: make DNS resolution errors non-blocking
We'll just ignore the associated rules instead.

refs: #57
2023-06-16 07:49:15 -07:00
Wez Furlong 1085e4bb19 dkim: move rsa key loading into DkimPrivateKey type
This makes it easy for the message crate to use the faster openssl
functionality if that has been enabled for the dkim crate.
2023-06-16 07:49:15 -07:00
Wez Furlong ffe2fd08e1 dkim: add option to use openssl for RSA signing
This is not on by default, and is not currently enabled
for KumoMTA.

Using openssl is currently about 3x faster than the pure rust
RSA crate, and inline with the perf of the ring crate, which
we could use here instead, but which doesn't support keys of
size 1024 which are currently widely deployed by the industry,
and for which larger sizes do not fit in DNS records.
2023-06-16 07:49:15 -07:00
Wez Furlong 86d4d22519 dkim: tweaks to make it easier to run a benchmark under perf 2023-06-16 07:49:15 -07:00
Ben Dyer 97d67fbb7f Resend EHLO after STARTTLS
Fixes https://github.com/KumoCorp/kumomta/issues/61

RFC 3207 suggests clients MUST forget the EHLO capabilities obtained prior to STARTTLS, and SHOULD re-send an EHLO after TLS negotiation completes (https://datatracker.ietf.org/doc/html/rfc3207#section-4.2). Some SMTP servers seem to _require_ an EHLO after STARTTLS so this ensures we send one.
2023-06-16 07:49:05 -07:00
Wez Furlong 4cf2e65c94 dkim: add comparitive benchmark against mail-auth
```
signing/sign Simple     time:   [1.2990 ms 1.3002 ms 1.3015 ms]
                        thrpt:  [2.3581 MiB/s 2.3603 MiB/s 2.3626 MiB/s]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

signing/sign Relaxed    time:   [1.2942 ms 1.2955 ms 1.2968 ms]
                        thrpt:  [2.3665 MiB/s 2.3689 MiB/s 2.3712 MiB/s]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

mail-auth signing/sign Simple
                        time:   [487.97 µs 488.22 µs 488.48 µs]
                        thrpt:  [6.2826 MiB/s 6.2860 MiB/s 6.2892 MiB/s]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

mail-auth signing/sign Relaxed
                        time:   [494.68 µs 495.68 µs 496.68 µs]
                        thrpt:  [6.1789 MiB/s 6.1913 MiB/s 6.2039 MiB/s]
```
2023-06-15 15:41:18 -07:00
Wez Furlong 30b8522873 dkim: precompile list of headers to sign during signing
We can exploit eg: knowing that the list of headers is a unique
list (no repeated headers) and save some overhead when using the
same signer multiple times.
2023-06-15 15:18:17 -07:00
Wez Furlong 4805f60837 dkim: avoid allocating a vec when signing headers 2023-06-15 14:45:40 -07:00
Wez Furlong 02d1378495 dkim: avoid re-parsing header list during signing 2023-06-15 14:28:11 -07:00
Wez Furlong e5d03052be dkim: tidy up handling of l= body length tag
We should parse at the edge and pass down the parsed number
2023-06-15 13:23:44 -07:00
Wez Furlong cf11540f84 dkim: remove another extraneous allocation, tidy up canon interface 2023-06-15 13:13:28 -07:00
Wez Furlong 28f0786e42 dkim: update crate deps 2023-06-15 11:37:59 -07:00
Wez Furlong 19c9201e44 dkim: remove some redundant clone calls 2023-06-15 11:30:44 -07:00
Wez Furlong 2e40c00121 dkim: introduce ParsedEmail type
This help transitions to a cheaper email parser, that doesn't need to
parse the entire MIME structure

Note that as part of this, I found that several tests were running
against email messages using unix line endings instead of canonical
CRLF.

This was silently being masked and an empty or otherwise incorrect byte
slice was being hashed; I've updated the affected tests and their test
expectations accordingly, as well as made this error bubble up at the
time of parsing the email, rather than silently squashing it in the
depths of the hasher.
2023-06-15 11:27:03 -07:00
Wez Furlong 3af3f52f66 dkim: pick some lint various bits 2023-06-15 10:27:33 -07:00
Wez Furlong 19aca29007 dkim: don't require moving data into verify_signature 2023-06-15 10:07:33 -07:00
Wez Furlong 424192eb5f dkim: tidy up DKIMHeader construction 2023-06-15 10:07:26 -07:00
Wez Furlong aecd187c21 dkim: remove some string copies 2023-06-15 09:53:53 -07:00
Wez Furlong ed8ccbc810 dkim: remove slog dependency 2023-06-15 09:27:28 -07:00
Wez Furlong af3236a72e dkim: remove Cargo.lock 2023-06-15 09:17:43 -07:00
Wez Furlong eb44e0780e Use vendored in dkim fork 2023-06-15 09:11:34 -07:00
Wez Furlong 49c1533435 Merge in my DKIM crate fork 2023-06-15 09:10:39 -07:00
Wez Furlong 261f68078f sources.lua make_egress_poll typo -> make_egress_pool 2023-06-15 08:19:16 -07:00
Wez Furlong 907e85053a kumod: improve shutdown diagnostics
Log something every 15 seconds to demonstrate that we didn't get
hung up
2023-06-15 07:47:26 -07:00
Wez Furlong 4f56314f48 kumomta.service: increase shutdown timeout
The default of 90 seconds is too short for a busy server.
Increase to 5 minutes.
2023-06-15 07:36:52 -07:00
Wez Furlong 38ae7cf20e smtp_server: improve debug/trace output
Show the buffer contents as a quoted string rather than an array
of the decimal bytes
2023-06-15 07:32:50 -07:00
Wez Furlong 40ce3cf22b break out benchmark by canon type 2023-06-14 20:24:34 -07:00
Wez Furlong 0c70ab2c52 improve error message when converting lua -> rust types
refs: https://github.com/KumoCorp/kumomta/issues/56
2023-06-14 20:14:14 -07:00
Wez Furlong 420dade8fb sources.lua: fixup initialization of name
refs: https://github.com/KumoCorp/kumomta/issues/56
2023-06-14 19:56:16 -07:00
Wez Furlong 8ff2ea950f dkim: update to pick up some perf improvements from my fork 2023-06-14 15:39:24 -07:00
Wez Furlong bde03d2036 reduce heap allocations during hashing
This doesn't totally eliminate them, but for the body at least,
we can stream and hash the data and avoid making potentially
very many very large allocations in a busy signing application.
2023-06-14 15:35:39 -07:00
Wez Furlong 5b3311dd8e hash: avoid making a copy of the body bytes
avoid thrashing the heap by returning the computed slice instead.
2023-06-14 10:02:48 -07:00
Wez Furlong 0fa0de10ce replace bytes::find with memchr::memmem::find
This is a much more efficient implementation with ~20% improved
throughput and latency with the simple signing benchmark.

signing/sign            time:   [1.3024 ms 1.3050 ms 1.3076 ms]
                        thrpt:  [2.2886 MiB/s 2.2933 MiB/s 2.2978 MiB/s]
                 change:
                        time:   [-19.016% -18.855% -18.691%] (p = 0.00 < 0.05)
                        thrpt:  [+22.987% +23.237% +23.482%]
                        Performance has improved.
2023-06-14 10:02:13 -07:00
Wez Furlong fb16925f99 add criterion driven benchmarking 2023-06-14 09:56:23 -07:00
Wez Furlong 2992e42138 add received_from and received_via meta values
We don't set received_via for http because axum/hyper make it a
bit weird to access that information and it will require some
effort to figure out how to get that without breaking the
existing proxy-aware client IP determination.
2023-06-13 15:18:06 -07:00
Wez Furlong 1e3e208405 traffic-gen: fix usage with starttls
We were passing `hostname:port` as the server name used to verify
the certificate name, but it should just be `hostname`.
2023-06-13 10:29:21 -07:00
Wez Furlong 628cda23ce shaping.lua: table_keys lives in utils 2023-06-12 16:07:47 -07:00
Wez Furlong c31b7e8191 dkim_sign: wrap an around the additional_signatures bit over here too 2023-06-12 14:56:02 -07:00
Wez Furlong 9c0ca973c2 replace ouroborous crate with self_cell
I picked ouroborous originally because the API was a bit nicer than that
in self_cell (we get to pick our own field names with ouroborous, but
don't in self_cell). However, ouroboros is now unmaintained and has a
soundness issue. The recommendation is to migrate, so that's what this
commit is.

refs: https://rustsec.org/advisories/RUSTSEC-2023-0042.html
refs: https://github.com/joshua-maros/ouroboros/issues/88

closes: https://github.com/KumoCorp/kumomta/security/dependabot/8
2023-06-12 14:52:16 -07:00
Wez Furlong b37c75e5e4 integration tests: explain this panic away better
Seemed to be an intermittent burp, but would benefit
from a better explanation anyway.
2023-06-12 14:48:11 -07:00