kumomta

mirror of https://github.com/mailscope/kumomta.git synced 2026-09-10 04:22:13 +00:00

Files

T

Wez Furlong 7fbec87587 spool enumeration no longer blocks incoming traffic

The original design of the spooling layer didn't require that
the SpoolId have the creation time encoded within it, which
meant that spool enumeration could consider any and all messages
found in the spool and import them into the queue subsystem.

If we allowed reception of new messages and wrote them to the spool
concurrently with the enumeration process, it would be possible
for the enumerator to observe the newly received messages and
import them into the queue subsystem, even though we had already
placed those newly received messages into the queue subsystem.
The result would be that we might send an additional copy of
each of the messages observed in this way.

To defend against that, the system refused to accept new messages
until spool enumeration was complete.

However, for sites with large spools, the enumeration process could
take some time to complete which could present challenges for
deploying updated configurations without an impact to their
service uptime.

This commit tackles that issue:

* Enumeration now filters out any messages that we created at or
  after the start of the enumeration process, so it is not possible
  for the duplicate scenario to occur.
* We no longer keep global track of whether spool enumeration is
  in progress, but will still log that progress to the diagnostic
  log.
* The liveness checks no longer check whether spool enumeration
  is in progress.

A potential consequence of this change is that the concurrent writes
to the spool may further reduce the speed at which enumeration
operates, but that's a reasonable trade.

2024-10-09 09:33:53 -07:00