Wez Furlong
092aacebb5
fix scheduler deadlock when core count is low
...
The GH actions CI hangs were due to a deadlock in rt_spawn.
The GH runners have only two cores available which meant
that it would only take one call to send_blocking or recv_blocking
to tie up the localset threads and deadlock.
This commit adjusts that logic to make rt_spawn async and to
provide a non-blocking fire and forget variant for the couple
of situations that need it.
Also adds a bit of tracing!
2023-03-08 07:38:35 -07:00
Wez Furlong
70ab30eb5c
ci: keep cache even if the build or tests fail
2023-03-07 23:18:10 -07:00
Wez Furlong
c7aa190315
ci: revert splitting of stages
2023-03-07 23:07:21 -07:00
Wez Furlong
e65aa45740
turn up tracing to try to get more insight into hang on GH actions
2023-03-07 22:46:11 -07:00
Wez Furlong
0c8adcf68e
ci: split rust into build and test stages
2023-03-07 22:36:23 -07:00
Wez Furlong
1488c2dfd7
add timeout when stopping daemon in integration tests
2023-03-07 22:26:14 -07:00
Wez Furlong
6e15132be0
more tweaks to integration tests
...
trying to figure out why they hang on GH actions
2023-03-07 22:07:28 -07:00
Wez Furlong
70e6c21839
capture egress path resolution failures
...
Log them as transient failures and also debug log them
2023-03-07 21:44:25 -07:00
Wez Furlong
25c229fae3
rustfmt
2023-03-07 20:22:04 -07:00
Wez Furlong
1d485d9fba
dump delivery logs in integration test
2023-03-07 20:20:13 -07:00
Wez Furlong
f4370faeac
ci: make it easier to debug the new integration test
2023-03-07 19:37:57 -07:00
Wez Furlong
257dc4a25a
flesh out integration testing some more
...
This has our first end-to-end integration test that validates
smtp -> source mta -> sink mta -> maildir
and confirms that the message has the right bits inside it.
The end to end test will get refactored into more easily usable
pieces in follow-up commit(s).
2023-03-07 18:42:14 -07:00
Wez Furlong
51d1408aa6
scaffolding for integration tests
2023-03-07 14:54:19 -07:00
Wez Furlong
92a5509af5
update simple_policy for changes to the dkim vault parameters
2023-03-07 14:52:36 -07:00
Wez Furlong
fc17efb7c9
ci: use cargo nextest for running tests
2023-03-07 11:35:58 -07:00
Wez Furlong
3e97cea1f6
docs build: simplify dep logic a little
2023-03-07 11:31:01 -07:00
Wez Furlong
8d529d1692
ci: ensure we use the cargo.lock locked deps for cargo installed stuff
2023-03-07 11:30:24 -07:00
Wez Furlong
7df50feb64
ci: streamline workflows a bit
2023-03-07 11:25:40 -07:00
Wez Furlong
258d9f8f64
docs: add a bit of an explanation about queueing
2023-03-07 08:27:44 -07:00
Wez Furlong
a05724e40c
auto-format generate-toc.py too
2023-03-07 07:36:50 -07:00
Wez Furlong
6d23055570
deps: update to revised maildir PR
2023-03-07 07:20:57 -07:00
Wez Furlong
3e203eb0d1
switch to my redis-cell fork to avoid shared object in the build
2023-03-07 06:47:17 -07:00
Wez Furlong
96350031dc
add index pages for redis and sqlite
2023-03-06 20:19:16 -07:00
Wez Furlong
7054fb31c9
switch to using human time for the queue related time intervals
2023-03-06 19:33:05 -07:00
Wez Furlong
5bb9363aaa
docs: format
2023-03-06 19:16:35 -07:00
Wez Furlong
2ec5f49ea0
docs: fixup dkim page
...
mdbook's mermaid support doesn't like the embedded titles
2023-03-06 19:15:13 -07:00
Wez Furlong
6dfbaf1113
docs: tweak user guide
2023-03-06 19:09:05 -07:00
Wez Furlong
00c253a9dd
add maildir support
...
This is to facilitate end-to-end testing to validate that the
right bits come out at the end.
2023-03-06 19:05:49 -07:00
Tom Mairs
fad4786ac9
fixing dkim diagram
2023-03-07 00:41:30 +00:00
Tom Mairs
63f9aea845
fixing dkim diagram
2023-03-07 00:38:25 +00:00
Tom Mairs
8186354766
fix relative links in files
2023-03-07 00:23:04 +00:00
Tom Mairs
295a33dce3
bulk updated to user guide
2023-03-07 00:11:12 +00:00
Wez Furlong
30e64c577f
tweak simple policy to show sqlite usage for auth
2023-03-06 12:35:25 -07:00
Wez Furlong
265698e4c3
refactor: make task spawning less boilerplatey
2023-03-06 11:01:31 -07:00
Wez Furlong
43155c809b
notes on how to start with tokio-console
2023-03-06 09:48:44 -07:00
Wez Furlong
8e0583b6cf
adopt tokio::task::Builder to label tasks
2023-03-06 09:38:18 -07:00
Wez Furlong
af327d7e21
add tokio-console support
...
We don't show much of anything at the moment; will need to make
a pass through and instrument a bunch of things
2023-03-06 07:53:27 -07:00
Wez Furlong
4ae2c1abca
allow redirecting diagnostic logs to a log directory
...
While in there, expose a few options for the logging format
2023-03-06 07:41:43 -07:00
Wez Furlong
cd5f7efbb0
docs: link to ARF rfc, add note about extensions:
2023-03-05 20:37:45 -07:00
Wez Furlong
c6292440d0
docs: fix a typo and add a missing paragraph
2023-03-05 20:34:29 -07:00
Wez Furlong
1bfa3a42af
Add custom logging
2023-03-05 20:29:14 -07:00
Wez Furlong
a0e73b72b1
allow logging selected meta and headers to the logs
2023-03-05 15:58:19 -07:00
Wez Furlong
1ac929e6b2
add max_segment_duration option to the logger
2023-03-05 15:25:08 -07:00
Wez Furlong
56135506f2
refactor: use any_err in more places
2023-03-05 10:44:54 -07:00
Wez Furlong
0ca585bd5a
Optionally use redis for throttles
2023-03-05 10:39:56 -07:00
Wez Furlong
ca78839a59
docs: add missing require
2023-03-05 09:59:06 -07:00
Wez Furlong
eaaddbb6b6
add redis client
2023-03-05 09:57:03 -07:00
Wez Furlong
139975e886
queue: remove maintainer joinhandle
...
While the logic compiled, a runtim assertion complained about
using blocking locks in a localset.
The lifecycle management stuff made this maintainer handle obsolete,
so we can simply delete at no cost.
2023-03-04 18:14:38 -07:00
Wez Furlong
2cf3d2387c
add sqlite data access
...
A bit of a big commit because I wanted sqlite queries to run
async and there isn't a sane way to tell in mlua whether a
given context is safe to yield for an await, which meant making
every case where we call into lua be fully async.
This adds a simple but powerful binding to sqlite that performs
its IO in a blocking tokio thread pool, and returns the results
back to lua.
There is no implicit caching of queries, and no implicit connection
pooling.
2023-03-04 18:07:31 -07:00
Wez Furlong
beb70061c8
ci: rename some workflows, disable full docker build
...
We're not doing anything with the output, so let's just save
the compute for something else.
2023-03-04 11:07:18 -07:00
Wez Furlong
Tom Mairs