mirror of
https://github.com/KumoCorp/kumomta.git
synced 2026-08-19 16:01:08 +00:00
Wez Furlong
d3ba5ccd2c
To support distributed connection limits, adjust the way that connection limits are handled. Previously, we'd do a simple in-memory comparison with the limit to guide whether we had reached the limit. In a distributed/clustered scenario we need a shared understanding of the state of the connections, so we're moving to a lease-based model. Each connection is associated with a UUID. Each egress path maintains a set of connections and their UUIDs. When attempting to make a new connection, we can only proceed if the number of connections associated with an egress path is below the limit. When a connection is closed, its UUID is removed from the associated set. To handle crashes, kills and netsplits, the path -> uuid association is also accompanied by an expiration time beyond which it can be assumed that the lease is no longer valid. We compute that time based on the worst case timeout value for a single message send on the given pathway. Each time a connection is ready to obtain a new message, it will try to extend its lease by that same duration. This commit introduces these concepts and models them using an in-memory store for the single node case, and using redis for the cluster case. When your policy enables `kumo.configure_redis_throttles`, that same redis configuration is used to back to the connection limiting functionality. refs: https://github.com/KumoCorp/kumomta/issues/41