Arpad Müller 48b05b7c50 Add a time_travel_remote_storage http endpoint (#6533) ...

Adds an endpoint to the pageserver to S3-recover an entire tenant to a
specific given timestamp.

Required input parameters:
* `travel_to`: the target timestamp to recover the S3 state to
* `done_if_after`: a timestamp that marks the beginning of the recovery
process. retries of the query should keep this value constant. it *must*
be after `travel_to`, and also after any changes we want to revert, and
must represent a point in time before the endpoint is being called, all
of these time points in terms of the time source used by S3. these
criteria need to hold even in the face of clock differences, so I
recommend waiting a specific amount of time, then taking
`done_if_after`, then waiting some amount of time again, and only then
issuing the request.

Also important to note: the timestamps in S3 work at second accuracy, so
one needs to add generous waits before and after for the process to work
smoothly (at least 2-3 seconds).

We ignore the added test for the mocked S3 for now due to a limitation
in moto: https://github.com/getmoto/moto/issues/7300 .

Part of https://github.com/neondatabase/cloud/issues/8233

2024-02-02 14:52:12 +01:00

..

src

Add a time_travel_remote_storage http endpoint (#6533)

2024-02-02 14:52:12 +01:00

tests

Do backoff::retry in s3 timetravel test (#6493)

2024-01-26 16:43:56 +00:00

Cargo.toml

fix(remote_storage): buffering, by using streams for upload and download (#5446)

2023-12-07 15:52:22 +00:00