mirror of
https://github.com/mailscope/kumomta.git
synced 2026-09-08 03:28:57 +00:00
Wez Furlong
e8e0f208ee
Previously we were using only our timeq module, which is built on top of hashed hierarchical timer wheels. Timer wheels have O(1) insertion and removal which are excellent properties for larger delayed queues. However, they do not know how to answer the question "when is the next item due", but only "what is due in the next tick". The underlying timer wheel implementation assumes a 1ms granularity which is a little bit at-odds with our pragmatic view of the scheduled queue, which is "if it's in there, precision timing isn't important, and it's generally fine to consider once per minute", requiring that we either aggressively scheduled a maintainer task to wake up every 1ms per scheduled queue (untenable!) or have it wake up no more than once per second but ideally closer to once per minute to then tick however milliseconds are necessary to advance the wheel to the current slot. For small numbers of scheduled queues with sufficiently large retry intervals this hasn't bubbled up as an issue so far, but it bothers me that it isn't as efficient as it could be because we have to wake up reasonably frequently to keep things ticking over, and that introduces higher continual CPU utilization. It's small, but I worry about the aggregate cost spread over very large numbers of scheduled queues. What I really want here is a a timer wheel that I can tick with arbitrary granularity and with that in mind I took a look at adapting the handful of existing implementations and found that we're already using the cleanest implementation, and it would take some effort that I didn't really want to spend right now. I opted for a reasonably simple alternative option, which is to adopt a skiplist for the queue. This has O(log n) insertion to maintain ordering with O(1) removal and can answer "when is the next item due". What this means is that we pay a slightly higher insertion cost one-time in exchange for being able to put the maintainer for the queue asleep until we need it, and not have to keep waking up between times, which should scale better. What this means in practice is that we now wake up the maintainer either when the next message is due, or once per minute to re-evaluate the queue configuration hook, so we're slightly better off, but totally where I'd like to be. I've introduced a reap_interval (default 10 minutes) and a refresh_interval (default 1 minute) as parameters in get_queue_config so that you can increase that 1m interval for reloading. What I'd like to do in a follow up commit is introduce a way to define the refresh policy. For example, it would be neat to say "watch my policy directory and refresh when it changes", which would make things the most efficient for many users. For those that are loading their config from a remote datasource, we'd need to consider some other mechanism for this; maybe some kind of long-poll or pubsub, but will obviously still be able to support the current interval based polling. Now, with all of that said: I didn't want to switch the product default over and hope for best, so what I did was add a strategy option to allow this to be adopted on a per-queue basis. Since I was in here adding some options, I also added an option that allows explicitly setting the interval used for timerwheel ticks, so you now have a lot more opportunities for tuning this stuff.