Christian Schwarz a0e61145c8 fix: cleanup of layers from the future can race with their re-creation (#5890) ...

fixes https://github.com/neondatabase/neon/issues/5878
obsoletes https://github.com/neondatabase/neon/issues/5879

Before this PR, it could happen that `load_layer_map` schedules removal
of the future
image layer. Then a later compaction run could re-create the same image
layer, scheduling a PUT.
Due to lack of an upload queue barrier, the PUT and DELETE could be
re-ordered.
The result was IndexPart referencing a non-existent object.

## Summary of changes

* Add support to `pagectl` / Python tests to decode `IndexPart`
  * Rust
    * new `pagectl` Subcommand
* `IndexPart::{from,to}_s3_bytes()` methods to internalize knowledge
about encoding of `IndexPart`
  * Python
    * new `NeonCli` subclass
* Add regression test
  * Rust
* Ability to force repartitioning; required to ensure image layer
creation at last_record_lsn
  * Python
    * The regression test.
* Fix the issue
  * Insert an `UploadOp::Barrier` after scheduling the deletions.

2023-11-23 13:33:41 +00:00

..

draw_timeline_dir.rs

pagectl: skip metadata file in pagectl draw-timeline (#4872)

2023-08-07 08:24:50 +01:00

index_part.rs

fix: cleanup of layers from the future can race with their re-creation (#5890)

2023-11-23 13:33:41 +00:00

layer_map_analyzer.rs

fix: replace all std::PathBufs with camino::Utf8PathBuf (#5352)

2023-10-04 17:52:23 +03:00

layers.rs

fix: replace all std::PathBufs with camino::Utf8PathBuf (#5352)

2023-10-04 17:52:23 +03:00

main.rs

fix: cleanup of layers from the future can race with their re-creation (#5890)

2023-11-23 13:33:41 +00:00