mirror of
https://github.com/neondatabase/neon.git
synced 2026-01-13 16:32:56 +00:00
a2d170b6d0
When `NeonEnv.from_repo_dir` was introduced, storage controller stored its state exclusively `attachments.json`. Since then, it has moved to using Postgres, which stores its state in `storage_controller_db`. But `NeonEnv.from_repo_dir` wasn't adjusted to do this. This PR rectifies the situation. Context for this is failures in `test_pageserver_characterize_throughput_with_n_tenants` CF: https://neondb.slack.com/archives/C033RQ5SPDH/p1721035799502239?thread_ts=1720901332.293769&cid=C033RQ5SPDH Notably, `from_repo_dir` is also used by the backwards- and forwards-compatibility. Thus, the changes in this PR affect those tests as well. However, it turns out that the compatibility snapshot already contains the `storage_controller_db`. Thus, it should just work and in fact we can remove hacks like `fixup_storage_controller`. Follow-ups created as part of this work: * https://github.com/neondatabase/neon/issues/8399 * https://github.com/neondatabase/neon/issues/8400
How to reproduce benchmark results / run these benchmarks interactively.
- Get an EC2 instance with Instance Store. Use the same instance type as used for the benchmark run.
- Mount the Instance Store =>
neon.git/scripts/ps_ec2_setup_instance_store - Use a pytest command line (see other READMEs further up in the pytest hierarchy).
For tests that take a long time to set up / consume a lot of storage space,
we use the test suite's repo_dir snapshotting functionality (from_repo_dir).
It supports mounting snapshots using overlayfs, which improves iteration time.
Here's a full command line.
RUST_BACKTRACE=1 NEON_ENV_BUILDER_USE_OVERLAYFS_FOR_SNAPSHOTS=1 DEFAULT_PG_VERSION=15 BUILD_TYPE=release \
./scripts/pytest test_runner/performance/pageserver/pagebench/test_pageserver_max_throughput_getpage_at_latest_lsn.py