Commit Graph
865 Commits
Author SHA1 Message Date
Wez Furlong 6645763606 docs: document HTTP APIs 2023-02-25 22:42:08 -07:00
Wez Furlong 51d9928ead docs: add more reference content 2023-02-25 20:41:50 -07:00
Mike HillyerandMike Hillyer bedf423d18 Mike reference manual (#2)
* Remove reference to shell variables.

* Lose the variable sample.

* Whoops, somehow I brought back index.markdown

---------

Co-authored-by: Mike Hillyer <mike@kumomta.com>
2023-02-25 18:28:06 -05:00
Wez Furlong b869d10e17 docs: add define_spool 2023-02-25 16:14:12 -07:00
Wez Furlong 82dc9c7821 docs: add configure_local_logs, start_esmtp_listener 2023-02-25 15:55:14 -07:00
Wez Furlong d994f0242e docs: apply admonish to the actual note on that page 2023-02-25 15:11:35 -07:00
Wez Furlong a0c0107a54 docs: install mdbook-admonish
See: https://tommilligan.github.io/mdbook-admonish/overview.html
https://tommilligan.github.io/mdbook-admonish/reference.html
2023-02-25 15:08:54 -07:00
Wez Furlong bfe1d5f94b docs: add some basic reference info
The content here needs to be fleshed out some more, but this commit
is intended to get the skeleton in place.
2023-02-25 14:59:20 -07:00
Wez Furlong 6acae33725 ci: build docs on PRs that touch docs 2023-02-25 12:28:49 -07:00
Wez Furlong c24ec1fb19 ci: urgh, remove anchors 2023-02-25 12:09:15 -07:00
Wez Furlong a8b6302e3b ci: refactor some path ignores 2023-02-25 12:06:26 -07:00
Mike Hillyer 098592c10e Merge branch 'main' of https://github.com/kumomta/kumomta 2023-02-25 14:05:57 -05:00
Mike Hillyer a147033987 Introduce initial skeleton content to the docs. 2023-02-25 14:05:50 -05:00
Wez Furlong 31a5af901c ci: fix typo 2023-02-25 12:04:02 -07:00
Wez Furlong 3af3a1199d ci: avoid building code when only docs changed 2023-02-25 11:54:42 -07:00
Wez Furlong e3146f0c7a ci: another run at pages deploy 2023-02-25 11:51:12 -07:00
Wez Furlong c8287d9c20 ci: maybe fix permissions for pages build 2023-02-25 11:28:27 -07:00
Wez Furlong c8a8d7da00 docs: install gelatyx earlier 2023-02-25 11:24:37 -07:00
Wez Furlong b8573f8059 ci: fix typo in doc builder 2023-02-25 11:10:21 -07:00
Wez Furlong bb328ca000 ci: automate doc build and publish to gh_pages branch 2023-02-25 11:00:18 -07:00
Wez Furlong 831bfe9aab docs: skeleton for building the docs
Rather than edit SUMMARY.md, you edit docs/generate-toc.py
and it will flesh out sections based on the list of files
in a directory.
2023-02-25 10:50:58 -07:00
Wez Furlong c1849551ed http injector: make a first pass at it
Still needs to call out to lua for routing and meta updates.
May need to parallelize the generation stage if throughput
isn't where we want it to be.
2023-02-25 10:12:07 -07:00
Wez Furlong 282dae670d refactor: move http auth to its own module 2023-02-24 11:17:21 -07:00
Wez Furlong 79ff6354c7 format 2023-02-24 10:53:17 -07:00
Wez Furlong 12f1637087 http: add https and some plumbing for auth
* Reuse the tls acceptor stuff from the smtp_server.
  We'll automatically generate self-signed certs when
  none are provided.
* Add trusted_hosts setting for the http server
* Add an auth middleware that will accept connections from
  trusted hosts as pre-authenticated. Other hosts must provide
  http authorization.
* Add lua hooks for the policy script to validate either http
  basic auth (user+pass), or bearer auth (token).

There's not currently an authentication storage subsystem, but
the lua hook allows for very basic stuff to be configured, and
could also be used to employ eg: looking things up from a database
once we add some lua utility functions for that purpose.

The `/metrics` endpoint will only respond to trusted IPs.

Other endpoints all require some kind of successful authentication.
2023-02-24 10:47:06 -07:00
Wez Furlong c9ad8db439 note about graceful shutdown of the http server 2023-02-24 06:30:14 -07:00
Wez Furlong 995e066572 queue: fix priority inversion issue with QueueManager
QueueManager::insert could acquire MANAGER lock then a queue handle
lock.

Meanwhile, the queue maintainer would hold a queue handle lock
and then attempt to acquire MANAGER lock when idling out and removing
itself, leading to a livelock because MANAGER is protected by a
tokio Mutex which ping pongs in and out of the event loop.
2023-02-23 20:35:16 -07:00
Wez Furlong af653a489f make docker build work with older versions of docker 2023-02-23 10:16:48 -07:00
Wez Furlong 2cba4196c5 revise docker build; move away from musl/alpine
A downside of pulling in rocksdb is that it is a pretty big
C++ library and musl/alpine don't really get on with it.

Switch to using the latest ubuntu LTS image as our base.
2023-02-23 10:07:29 -07:00
Wez Furlong 602e9721c9 CI: fixup path for rename from smtpd -> kumod 2023-02-23 07:57:08 -07:00
Wez Furlong 6b85cfb2ee CI: don't require a tty to build docker image 2023-02-23 07:38:12 -07:00
Wez Furlong 42b173eac5 CI: take a run at building the docker image 2023-02-23 07:36:48 -07:00
Wez Furlong df232cbc48 refactor: rename shutdown/lifetime -> lifecycle 2023-02-23 07:30:32 -07:00
Wez Furlong 229a093560 shutdown if we have errors setting up the spool
Adds some plumbing that allows the application to request
a shutdown that is channelled along the same path as ctrl-c
handling.

If we have errors defining spools or during spool start,
then we request a shutdown on the basis that we can't store
anything and it would be unsafe to continue.
2023-02-23 07:23:06 -07:00
Wez Furlong a2fa65a5cc spool: improve some startup error messages 2023-02-23 07:08:10 -07:00
Wez Furlong 6067de108c spool: use bytes rather than strings for rocksdb keys
Reduces space of id keys by half
2023-02-22 23:28:38 -07:00
Wez Furlong 175dbb46e0 spool: break out of startup enumeration early if ctrl-c is pressed 2023-02-22 23:23:42 -07:00
Wez Furlong 5438824ef3 smtp_server: add reception limits
allow limiting the number of messages and the number of recipients
that come in on a single connection
2023-02-22 23:12:39 -07:00
Wez Furlong 029b922e68 smtp_server: avoid logging verbose errors when client disconnects
This transforms any read error into a Disconnect that we process
silently.

The error is logged at trace level.
2023-02-22 20:26:28 -07:00
Wez Furlong dd7c7a6292 graceful shutdown part 2
This adds some logic to save any messages that had spooling
deferred and release them once shutdown is requested.
2023-02-22 18:50:19 -07:00
Wez Furlong 85f4ab69b9 add graceful shutdown, part 1
This teaches the smtp listener and delivery sides
how to shutdown when a shutdown is requested, as well
as how to indicate when they are still doing important
work.

This doesn't guarantee that any deferred spooling is
done; that will be part 2.
2023-02-22 17:00:00 -07:00
Wez Furlong 3c5959aa9d fixup build, add compression level as an option 2023-02-22 12:52:44 -07:00
Wez Furlong daf3b87753 add num_attempts field, and zstd compress the logs
I don't plan to remove compression or to allow it to be disabled: it
makes a massive difference to the size of the log files and saves IO and
storage space.
2023-02-22 12:33:58 -07:00
Wez Furlong bcd4a6fc6d take a crack at logging receptions and deliveries 2023-02-22 11:57:26 -07:00
Wez Furlong b03f74f2c0 kumod: remove unused parameter 2023-02-22 07:47:58 -07:00
Wez Furlong 681784f3e1 allow "mail from:user@host" with no <>
Same for "rcpt to". The <> are required in the ABNF in the spec,
but they are widely considered to be optional by major implementations,
which makes it convenient when ad-hoc testing via telnet.

We require the <> when ESMTP mail from or rcpt to parameters are
used.
2023-02-22 07:20:14 -07:00
Wez Furlong aa508b012a smtpclient: include command in Response
This will make the logs more useful when we start writing those
2023-02-22 06:59:13 -07:00
Wez Furlong 1cd30161e7 Allow starting as root in order to bind to port 25
We require that `--user username` be passed and specify
the user id that we should assume instead of running as
root.

We do the magic setup needed to preserve the ability
to bind to port 25 while dropping the other privs.
2023-02-21 22:38:51 -07:00
Wez Furlong 5cfb086fed spool: remove sled
It's beta and rocksdb performs better; there's not a compelling
reason to retain it until it stabilizes.
2023-02-21 21:28:36 -07:00
Wez Furlong b25ff76118 add dkim signing
A fairly sprawling commit:

* Move config to its own crate to facilitate making things more modular
* Crates/modules can now provide a registration function that can be
  used to setup functions in the lua environment
* Message crate now has a dkim module with a signer type that can
  be loaded from lua; signers are cached (with ttl) and shareable
* Message now has a dkim_sign method that does the signing, as well
  as methods for appending and prepending headers that are necessary
  to support signing.
2023-02-21 21:18:49 -07:00