rust/neon
1
0
Fork 0
mirror of https://github.com/neondatabase/neon.git synced 2026-01-08 14:02:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
06ce7040413234f4460f89fbca666b3a19f86446
neon/test_runner/regress
History
Dmitrii Kovalkov 136eaeb74a pageserver: basebackup cache (hackathon project) (#11989) 
## Problem
Basebackup cache is on the hot path of compute startup and is generated
on every request (may be slow).

- Issue: https://github.com/neondatabase/cloud/issues/29353

## Summary of changes
- Add `BasebackupCache` which stores basebackups on local disk.
- Basebackup prepare requests are triggered by
`XLOG_CHECKPOINT_SHUTDOWN` records in the log.
- Limit the size of the cache by number of entries.
- Add `basebackup_cache_enabled` feature flag to TenantConfig.
- Write tests for the cache

## Not implemented yet
- Limit the size of the cache by total size in bytes

---------

Co-authored-by: Aleksandr Sarantsev <aleksandr@neon.tech>
2025-05-22 12:45:00 +00:00
..
data
Add remote extension test with library component (#11301)
2025-04-24 22:33:46 +00:00
test_ancestor_branch.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_attach_tenant_config.py
Add rel_size_replica_cache (#11889)
2025-05-20 15:38:27 +00:00
test_auth_broker.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_auth.py
Enable all pyupgrade checks in ruff
2024-10-08 14:32:26 -05:00
test_backpressure.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_bad_connection.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_basebackup_error.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_basebackup.py
pageserver: basebackup cache (hackathon project) (#11989)
2025-05-22 12:45:00 +00:00
test_branch_and_gc.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_branch_behind.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_branching.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_broken_timeline.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_build_info_metric.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_change_pageserver.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_clog_truncate.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_close_fds.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_combocid.py
test_runner: use LFC by default (#8613)
2024-11-25 09:01:05 +00:00
test_compaction.py
fix(test): allow flush error in gc-compaction tests (#11822)
2025-05-05 12:15:22 +00:00
test_compatibility.py
tests: remove mentions of ALLOW_*_COMPATIBILITY_BREAKAGE (#11618)
2025-04-17 10:03:21 +00:00
test_compute_catalog.py
fix(compute_ctl): Dollar escaping and tests (#11969)
2025-05-20 09:03:36 +00:00
test_compute_http.py
Make the audience claim in compute JWTs a vector (#11845)
2025-05-06 22:19:15 +00:00
test_compute_locales.py
Switch compute-related locales to C.UTF-8 by default
2024-11-08 12:19:18 -06:00
test_compute_metrics.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_compute_migrations.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_compute_monitor.py
feat(compute): Introduce Postgres downtime metrics (#11346)
2025-04-24 13:51:09 +00:00
test_compute_reconfigure.py
Consolidate compute_ctl configuration structures (#11514)
2025-04-11 15:06:29 +00:00
test_config.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_crafted_wal_end.py
safekeeper: use protobuf for sending compressed records to pageserver (#9821)
2024-11-27 12:12:21 +00:00
test_createdropdb.py
Enable all pyupgrade checks in ruff
2024-10-08 14:32:26 -05:00
test_createuser.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_ddl_forwarding.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_disk_usage_eviction.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_download_extensions.py
fix(compute) Correct the PGXN s3 gateway URL. (#11796)
2025-05-07 16:34:08 +00:00
test_endpoint_crash.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_endpoint_storage.py
endpoint_storage compute_ctl integration (#11550)
2025-05-06 22:02:12 +00:00
test_explain_with_lfc_stats.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_extensions.py
compute_ctl: Add endpoint that allows extensions to be installed (#9344)
2024-10-18 15:07:36 +03:00
test_fsm_truncate.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_fullbackup.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_gc_aggressive.py
improve ondemand-download latency observability (#11421)
2025-04-04 18:04:39 +00:00
test_gin_redo.py
Enable all pyupgrade checks in ruff
2024-10-08 14:32:26 -05:00
test_gist.py
Use unlogged build for gist_indexsortbuild_flush_ready_pages (#11753)
2025-05-06 07:24:27 +00:00
test_hot_standby.py
test_runner: improve wait_until (#9936)
2024-12-02 10:26:15 +00:00
test_import_pgdata.py
pageserver: make imports work with tenant shut downs (#11855)
2025-05-13 17:49:49 +00:00
test_import.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_ingestion_layer_size.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_large_schema.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_layer_bloating.py
Fix logical_replication_sync test fixture (#10531)
2025-02-03 12:44:47 +00:00
test_layer_eviction.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_layer_writers_fail.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_layers_from_future.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_lfc_prefetch.py
Remove loop from pageserver_try_receive (#11387)
2025-03-31 19:49:32 +00:00
test_lfc_prewarm.py
endpoint_storage compute_ctl integration (#11550)
2025-05-06 22:02:12 +00:00
test_lfc_resize.py
Skip hole tags in local_cache view (#11454)
2025-04-08 03:52:50 +00:00
test_lfc_working_set_approximation.py
Disable autovacuum and increase limit for WS approximation (#11583)
2025-04-17 05:07:45 +00:00
test_local_file_cache.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_logging.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_logical_replication.py
Fix logical_replication_sync test fixture (#10531)
2025-02-03 12:44:47 +00:00
test_lsn_mapping.py
pageserver: return 412 PreconditionFailed in get_timestamp_of_lsn if timestamp is not found (#11491)
2025-04-09 13:16:15 +00:00
test_multixact.py
Enable all pyupgrade checks in ruff
2024-10-08 14:32:26 -05:00
test_nbtree_pagesplit_cycleid.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_neon_cli.py
Rename object_storage->endpoint_storage (#11678)
2025-04-23 14:03:19 +00:00
test_neon_extension.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_neon_local_cli.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_neon_superuser.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_next_xid.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_normal_work.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_oid_overflow.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_old_request_lsn.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_ondemand_download.py
improve ondemand-download latency observability (#11421)
2025-04-04 18:04:39 +00:00
test_ondemand_slru_download.py
Python 3.11 (#9515)
2024-11-21 16:25:31 +00:00
test_ondemand_wal_download.py
Register custom xlog reader callbacks for on-demand WAL download in StartupDecodingContext (#9007)
2024-11-19 22:29:57 +02:00
test_page_service_batching_regressions.py
pageserver: support keys at different LSNs in one get page batch (#11494)
2025-04-14 09:05:29 +00:00
test_pageserver_api.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_catchup.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_config.py
pageserver config: ignore+warn about unknown fields (instead of deny_unknown_fields) (#11275)
2025-04-04 17:30:58 +00:00
test_pageserver_crash_consistency.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_generations.py
pageserver: make control_plane_api & generations fully mandatory (#10715)
2025-04-28 17:24:55 +00:00
test_pageserver_getpage_throttle.py
tests: make test_pageserver_getpage_throttle less flaky (#11482)
2025-04-11 09:38:05 +00:00
test_pageserver_layer_rolling.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_metric_collection.py
pageserver: add branch-local consumption metrics (#11852)
2025-05-22 12:26:32 +00:00
test_pageserver_reconnect.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_restart.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_restarts_under_workload.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pageserver_secondary.py
Migrate to correct logger interface (#11956)
2025-05-17 21:12:01 +00:00
test_pg_query_cancellation.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pg_regress.py
fix(test): increase timeouts for some tests (#11781)
2025-05-01 18:36:26 +00:00
test_pg_waldump.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_pgstat.py
Move neon_pgstat_file_size_limit to the extension (#10959)
2025-02-25 12:23:04 +00:00
test_physical_and_logical_replicaiton.py
Fix logical_replication_sync test fixture (#10531)
2025-02-03 12:44:47 +00:00
test_physical_replication.py
use a prod-like shared_buffers size for some perf unit tests (#11373)
2025-04-02 10:43:05 +00:00
test_pitr_gc.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_postgres_version.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_prefetch_buffer_resize.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_proxy_allowed_ips.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_proxy_metric_collection.py
merge pg-sni-router into proxy (#11882)
2025-05-12 15:48:48 +00:00
test_proxy_websockets.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_proxy.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_read_validation.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_readonly_node.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_recovery.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_relations.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_remote_storage.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_replica_start.py
Add rel_size_replica_cache (#11889)
2025-05-20 15:38:27 +00:00
test_role_grants.py
fix(neon-rls): optimistically assume role grants are already assigned for replicas (#11811)
2025-05-09 07:48:30 +00:00
test_s3_restore.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_safekeeper_deletion.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_setup.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_sharding.py
storcon: fix split aborts removing other tenants (#11837)
2025-05-06 13:57:34 +00:00
test_signed_char.py
Add -fsigned-char for cross platform signed chars (#10852)
2025-02-28 21:07:21 +00:00
test_sni_router.py
merge pg-sni-router into proxy (#11882)
2025-05-12 15:48:48 +00:00
test_ssl.py
storage_broker: https handler (#11603)
2025-04-25 14:28:56 +00:00
test_storage_controller.py
storage_controller: make leadership protocol more robust (#11703)
2025-04-24 16:59:56 +00:00
test_storage_scrubber.py
fix(tests): improve test_scrubber_tenant_snapshot stability (#11471)
2025-04-08 10:03:38 +00:00
test_subscriber_branching.py
Consolidate compute_ctl configuration structures (#11514)
2025-04-11 15:06:29 +00:00
test_subscriber_restart.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_subxacts.py
safekeeper: use protobuf for sending compressed records to pageserver (#9821)
2024-11-27 12:12:21 +00:00
test_tenant_conf.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_tenant_delete.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_tenant_detach.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_tenant_relocation.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_tenant_size.py
fix(test): wait for shard split complete for test_lsn_lease_storcon (#11436)
2025-04-03 17:49:45 +00:00
test_tenant_tasks.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_tenants_with_remote_storage.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_tenants.py
fix(test): allow stale generation warnings in storcon (#11624)
2025-04-17 16:12:24 +00:00
test_threshold_based_eviction.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_timeline_archive.py
Add metric for number of offloaded timelines (#11976)
2025-05-21 11:28:22 +00:00
test_timeline_delete.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_timeline_detach_ancestor.py
tests: flush wal before waiting for last record lsn (#11726)
2025-05-08 10:00:45 +00:00
test_timeline_gc_blocking.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_timeline_size.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_truncate.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_twophase.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_unlogged.py
Enable all pyupgrade checks in ruff
2024-10-08 14:32:26 -05:00
test_unstable_extensions.py
Create the notion of unstable extensions
2024-10-28 17:47:15 -05:00
test_vm_bits.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_vm_truncate.py
Correctly truncate VM (#9342)
2024-11-14 17:19:13 +02:00
test_wal_acceptor_async.py
walproposer: make it aware of membership (#11407)
2025-04-10 09:55:37 +00:00
test_wal_acceptor.py
walproposer: make it aware of membership (#11407)
2025-04-10 09:55:37 +00:00
test_wal_receiver.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_wal_restore.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00
test_walredo_not_left_behind_on_detach.py
ruff: enable TC — flake8-type-checking (#11368)
2025-03-30 18:58:33 +00:00