This commit addresses a couple of related issues around scheduled qeueue
suspensions:
1. There was no check in the ready queue logic to confirm that a
given message was not part of a suspension. Ideally, it wouldn't
land in the ready queue if it is suspended, but if you have a large
ready queue and one of the messages generates a suspension, then
the remainder would get attempted, oblivious to the new suspension.
The resolution here is to add a check for that case, log a transfail
and requeue the message.
2. We only checked whether the scheduled queue was suspended in the
case where a message was being newly inserted into the queuing
system. Importantly, messages being promoted from the scheduled
queue didn't use this code path. This commit fixes this up by
relocating the check to the appropriate location. In addition,
we now will log a transfail for this case and delay the message
according to its retry schedule.
3. Since we're in here changing the retry schedule for suspensions,
take the opportunity to take care of #293 which applies to the
more general logic around all sources being suspended.
The upshot of this is that we're now logging transfails in a number
of suspension cases where we weren't previously, and using the normal
retry schedule for those cases where we weren't previously doing
that either.
refs: https://github.com/KumoCorp/kumomta/issues/290
refs: https://github.com/KumoCorp/kumomta/issues/293
Rather than queueing up a complete copy of all logs to send
to the TSA daemon, we now perform a match on the client side.
If the record doesn't match any rules then we won't bother
sending it to the daemon.
Otherwise, we'll queue and send.
This reduces the amount of data that we'd need to queue, thus
reducing IO pressure, and reducing inter-node bandwidth.
This new behavior is currently opt-in, but will likely
be the default by the time we cut the next release.
To opt-in, add `pre_filter = true` to the `setup_with_automation`
parameters.
This commit connects the new websocket based suspension feed
up to shaping.lua. This allows ready-q suspensions to be
enacted in realtime, as well as sets things up to support
scheduled queue suspensions in a later commit.
refs: https://github.com/KumoCorp/kumomta/issues/113
Wez Furlong