Commit Graph
861 Commits
Author SHA1 Message Date
Wez Furlong e5d03052be dkim: tidy up handling of l= body length tag
We should parse at the edge and pass down the parsed number
2023-06-15 13:23:44 -07:00
Wez Furlong cf11540f84 dkim: remove another extraneous allocation, tidy up canon interface 2023-06-15 13:13:28 -07:00
Wez Furlong 28f0786e42 dkim: update crate deps 2023-06-15 11:37:59 -07:00
Wez Furlong 19c9201e44 dkim: remove some redundant clone calls 2023-06-15 11:30:44 -07:00
Wez Furlong 2e40c00121 dkim: introduce ParsedEmail type
This help transitions to a cheaper email parser, that doesn't need to
parse the entire MIME structure

Note that as part of this, I found that several tests were running
against email messages using unix line endings instead of canonical
CRLF.

This was silently being masked and an empty or otherwise incorrect byte
slice was being hashed; I've updated the affected tests and their test
expectations accordingly, as well as made this error bubble up at the
time of parsing the email, rather than silently squashing it in the
depths of the hasher.
2023-06-15 11:27:03 -07:00
Wez Furlong 3af3f52f66 dkim: pick some lint various bits 2023-06-15 10:27:33 -07:00
Wez Furlong 19aca29007 dkim: don't require moving data into verify_signature 2023-06-15 10:07:33 -07:00
Wez Furlong 424192eb5f dkim: tidy up DKIMHeader construction 2023-06-15 10:07:26 -07:00
Wez Furlong aecd187c21 dkim: remove some string copies 2023-06-15 09:53:53 -07:00
Wez Furlong ed8ccbc810 dkim: remove slog dependency 2023-06-15 09:27:28 -07:00
Wez Furlong af3236a72e dkim: remove Cargo.lock 2023-06-15 09:17:43 -07:00
Wez Furlong eb44e0780e Use vendored in dkim fork 2023-06-15 09:11:34 -07:00
Wez Furlong 49c1533435 Merge in my DKIM crate fork 2023-06-15 09:10:39 -07:00
Wez Furlong 261f68078f sources.lua make_egress_poll typo -> make_egress_pool 2023-06-15 08:19:16 -07:00
Wez Furlong 907e85053a kumod: improve shutdown diagnostics
Log something every 15 seconds to demonstrate that we didn't get
hung up
2023-06-15 07:47:26 -07:00
Wez Furlong 4f56314f48 kumomta.service: increase shutdown timeout
The default of 90 seconds is too short for a busy server.
Increase to 5 minutes.
2023-06-15 07:36:52 -07:00
Wez Furlong 38ae7cf20e smtp_server: improve debug/trace output
Show the buffer contents as a quoted string rather than an array
of the decimal bytes
2023-06-15 07:32:50 -07:00
Wez Furlong 40ce3cf22b break out benchmark by canon type 2023-06-14 20:24:34 -07:00
Wez Furlong 0c70ab2c52 improve error message when converting lua -> rust types
refs: https://github.com/KumoCorp/kumomta/issues/56
2023-06-14 20:14:14 -07:00
Wez Furlong 420dade8fb sources.lua: fixup initialization of name
refs: https://github.com/KumoCorp/kumomta/issues/56
2023-06-14 19:56:16 -07:00
Wez Furlong 8ff2ea950f dkim: update to pick up some perf improvements from my fork 2023-06-14 15:39:24 -07:00
Wez Furlong bde03d2036 reduce heap allocations during hashing
This doesn't totally eliminate them, but for the body at least,
we can stream and hash the data and avoid making potentially
very many very large allocations in a busy signing application.
2023-06-14 15:35:39 -07:00
Wez Furlong 5b3311dd8e hash: avoid making a copy of the body bytes
avoid thrashing the heap by returning the computed slice instead.
2023-06-14 10:02:48 -07:00
Wez Furlong 0fa0de10ce replace bytes::find with memchr::memmem::find
This is a much more efficient implementation with ~20% improved
throughput and latency with the simple signing benchmark.

signing/sign            time:   [1.3024 ms 1.3050 ms 1.3076 ms]
                        thrpt:  [2.2886 MiB/s 2.2933 MiB/s 2.2978 MiB/s]
                 change:
                        time:   [-19.016% -18.855% -18.691%] (p = 0.00 < 0.05)
                        thrpt:  [+22.987% +23.237% +23.482%]
                        Performance has improved.
2023-06-14 10:02:13 -07:00
Wez Furlong fb16925f99 add criterion driven benchmarking 2023-06-14 09:56:23 -07:00
Wez Furlong 2992e42138 add received_from and received_via meta values
We don't set received_via for http because axum/hyper make it a
bit weird to access that information and it will require some
effort to figure out how to get that without breaking the
existing proxy-aware client IP determination.
2023-06-13 15:18:06 -07:00
Wez Furlong 1e3e208405 traffic-gen: fix usage with starttls
We were passing `hostname:port` as the server name used to verify
the certificate name, but it should just be `hostname`.
2023-06-13 10:29:21 -07:00
Wez Furlong 628cda23ce shaping.lua: table_keys lives in utils 2023-06-12 16:07:47 -07:00
Wez Furlong c31b7e8191 dkim_sign: wrap an around the additional_signatures bit over here too 2023-06-12 14:56:02 -07:00
Wez Furlong 9c0ca973c2 replace ouroborous crate with self_cell
I picked ouroborous originally because the API was a bit nicer than that
in self_cell (we get to pick our own field names with ouroborous, but
don't in self_cell). However, ouroboros is now unmaintained and has a
soundness issue. The recommendation is to migrate, so that's what this
commit is.

refs: https://rustsec.org/advisories/RUSTSEC-2023-0042.html
refs: https://github.com/joshua-maros/ouroboros/issues/88

closes: https://github.com/KumoCorp/kumomta/security/dependabot/8
2023-06-12 14:52:16 -07:00
Wez Furlong b37c75e5e4 integration tests: explain this panic away better
Seemed to be an intermittent burp, but would benefit
from a better explanation anyway.
2023-06-12 14:48:11 -07:00
Wez Furlong fe2f674180 listener_domains: defer building domain_map
Ideally we'd memoize the compiled map, but right now it isn't
compatible with being serialized, so we have to do it after
we call memoize
2023-06-12 14:26:11 -07:00
Wez Furlong 79f45005d5 dkim: defer building domain map
Ideally we'd memoize the compiled map, but right now it isn't
compatible with being serialized, so we have to do it after
we call memoize
2023-06-12 14:19:19 -07:00
Wez Furlong 0249bd0d4d kcli: you may now list and cancel admin bounce entries
Entries now have a uuid that identifies them within a given instance
(NOT within a cluster).  The id is returned when the entry is
established:

```console
$ kcli --endpoint http://127.0.0.1:8000 bounce --everything --reason boop
{
  "id": "9a946198-9351-4e0e-b460-fc84e066878a",
  "bounced": {},
  "total_bounced": 0
}
```

You can list entries with their remaining duration:

```console
$ kcli --endpoint http://127.0.0.1:8000 bounce-list
[
  {
    "id": "9a946198-9351-4e0e-b460-fc84e066878a",
    "campaign": null,
    "tenant": null,
    "domain": null,
    "reason": "boop",
    "duration": "4m 57s 313ms 891us 882ns"
  }
]
```

and cancel them:

```console
$ kcli --endpoint http://127.0.0.1:8000 bounce-cancel --id 9a946198-9351-4e0e-b460-fc84e066878a
removed 9a946198-9351-4e0e-b460-fc84e066878a
```

and the error case if you try it again:

```console
$ kcli --endpoint http://127.0.0.1:8000 bounce-cancel --id 9a946198-9351-4e0e-b460-fc84e066878a
Error: bounce entry 9a946198-9351-4e0e-b460-fc84e066878a not found
```
2023-06-12 10:11:53 -07:00
Mike Hillyer e12bf25d60 Move the listener domain information to the right page. 2023-06-12 12:31:36 -04:00
Wez Furlong d6d2064564 admin: bounce: add id field to internals and to response
This doesn't do anything in the current milestone, but I wanted
to get it in there ready for work for the next milestone,
where I plan to make it possible to list and remove/cancel
a bounce, as well as persist them across a restart.
2023-06-12 07:11:43 -07:00
Wez Furlong 22404630aa dkim_sign.lua: fix another data.base vs. base 2023-06-09 14:15:19 -07:00
Wez Furlong 87cffb945e dkim_sign.lua: avoid nil reference for additional_signatures 2023-06-09 13:06:27 -07:00
Wez Furlong 30bcf40497 dkim_sign.lua: fix array syntax for headers 2023-06-09 11:55:36 -07:00
Wez Furlong 643eecc9cc dkim_sign.lua: fixup base 2023-06-09 11:48:19 -07:00
Wez Furlong fd4aeb00a2 dkim_sign.lua: fixup missing vault_path_prefix usage 2023-06-09 11:47:27 -07:00
Wez Furlong 5001095044 dkim: sketch out an entirely untested dkim signing helper 2023-06-09 11:43:05 -07:00
Wez Furlong bcfba07dad dkim: switch to my fork of cfdkim for RSA signatures
That fork removes the lifetime from the Signer type which makes
it easier to cache more efficiently.
2023-06-09 06:46:05 -07:00
Tom Mairs 756405d133 update dkim docs 2023-06-09 06:07:47 +00:00
Wez Furlong d97b403bd8 remove lifetime from the Signer struct
It is rather difficult to reuse the Signer in a dynamic application with
the lifetime present in the struct.

This commit removes the lifetime to make it easier to cache and reuse.

The logger has also been removed from the API for the signer. It was
used only in one place to log a single debug line, and it is a bit of an
imposition on the embedding application to require the use of the `slog`
crate just for that, and its removal helps with the goal of removing
lifetimes from the struct as well.

Some builder methods have been made a little more ergonomic by accepting
values that are convertible to String rather than requiring str
references.

Since this commit changes the API signature, it also bumps the version
from 0.2 to 0.3.
2023-06-08 17:18:15 -07:00
Tom Mairs 130b419168 clean up files and add notes 2023-06-08 22:09:40 +00:00
Wez Furlong e9e158039f ci: trigger pkg build for policy changes 2023-06-08 14:48:19 -07:00
Wez Furlong b930ee5ec1 shaping.toml: remove unresolvable protection.outlook.com domain
entries in this file must be resolvable domains, and this one
has neither A nor MX records
2023-06-08 14:39:27 -07:00
Wez Furlong 88fd825be4 dkim: add cfdkim as a runtime alternative RSA signer
The mail-auth crate doesn't support 1024 bit RSA keys, which are
currently the dominant size of key used by senders at scale.

It is not possible to even configure a 2048 bit RSA key in some DNS
providers, which blocks widespread adoption of larger key sizes.

This commit pulls in the cfdkim crate as an alternative implementation.

I'm not happy with this implementation because the API of the crate
defeats some caching optimizations, doesn't support certain signing
attributes that are supported by mail-auth and the generated header isn't
nicely wrapped.

We'll likely "do something" to resolve this in a future commit.

For now, this implementation is enabled by setting `use_cf = true`
in the signing parameters.
2023-06-08 14:29:19 -07:00
Wez Furlong 887bb7e9e4 update rocksdb dep to fix build with gcc 13
refs: https://github.com/facebook/rocksdb/pull/11118
2023-06-08 13:29:53 -07:00