Wez Furlong c95fd24d9e memoize/lruttl: improve robustness ...

This commit fixes up a few edge cases:

* The recent switch to moka for lruttl adopted an LFU eviction
  policy which could choose not to take a recently updated value
  in some situations, which is undesirable for expensive cache
  population functions. Explicitly configure it to use classic
  LRU for its eviction policy.
* The semaphore based thundering herd control in memoize had
  a very hard to pin down edge case where we could end up
  blocking for an extended period of time.  Switch to the
  `.get_or_try_insert_with_freshness_status` method that moka
  enabled in our lruttl module
* That didn't eliminate the edge case, but rather limited the
  scope of its negative effects. There is still an edge case,
  but now it results in serial duplicate calls to the cache
  population function, which is wasteful but not as egregious
  as the extended period of blocking.
  I've opened https://github.com/moka-rs/moka/discussions/502
  with moka to see what is going wrong there
* Adopt dashmap for the map of caches, to reduce hypothetical
  contention across multiple caches. This is a micro optimization.
* Improved tracing of lruttl and memoize

2025-03-04 16:23:14 -07:00

..

src

memoize/lruttl: improve robustness

2025-03-04 16:23:14 -07:00

Cargo.toml

roll our own timeq

2025-02-06 12:38:08 -07:00