Arthur Petukhovsky f25dd75be9 Fix deadlock in safekeeper metrics (#2566) ...

We had a problem where almost all of the threads were waiting on a futex syscall. More specifically:
- `/metrics` handler was inside `TimelineCollector::collect()`, waiting on a mutex for a single Timeline
- This exact timeline was inside `control_file::FileStorage::persist()`, waiting on a mutex for Lazy initialization of `PERSIST_CONTROL_FILE_SECONDS`
- `PERSIST_CONTROL_FILE_SECONDS: Lazy<Histogram>` was blocked on `prometheus::register`
- `prometheus::register` calls `DEFAULT_REGISTRY.write().register()` to take a write lock on Registry and add a new metric
- `DEFAULT_REGISTRY` lock was already taken inside `DEFAULT_REGISTRY.gather()`, which was called by `/metrics` handler to collect all metrics

This commit creates another Registry with a separate lock, to avoid deadlock in a case where `TimelineCollector` triggers registration of new metrics inside default registry.

2022-10-06 01:07:02 +03:00

..

spec

Fix typos

2022-05-28 14:02:05 +03:00

src

Fix deadlock in safekeeper metrics (#2566)

2022-10-06 01:07:02 +03:00

Cargo.toml

Remove control_plane deps on pageserver/safekeeper (#2513)

2022-10-04 11:14:45 -07:00