mirror of
https://github.com/mailscope/kumomta.git
synced 2026-08-24 13:18:18 +00:00
Wez Furlong
12d1569973
For large classifier rulesets (more than 3,000 rules), the cost of classifying any single response can be as high as 2ms. This is significant; given that the loggers are each single threaded a 2ms overhead limits the maximum throughput to 500 messages per second. This commit improves the structure of the code in order to mitigate the potential for a bottleneck: * Don't bother categorizing Reception records. They are not bounces and are generated by the local machine. It's a waste of CPU and introduces the potential to put back pressure on the injector. * Introduce a cache for classification results. The cache is split into two parts so that unclassified results don't churn out the successfully classified results. * Introduce a bounce-classification thread pool. If we are unlucky and encounter a long series of "random" responses with no cache hits, and have up to 2ms per classification constraining us to around 500 msgs/s, then we need to apply more CPU cores to the classifier to achieve multiples of that throughput, and that is what we have here. We spawn 1/4 the number of cores threads into this thread pool.