Make schedule_layer_file_deletion atomic.

Make sure that we don't mutate the upload queue, until we're sure that
we will complete all the changse. Otherwise, we could bail out after
removing some files from upload_queue.latest_files, but not all.

This is purely theoretical: there were only two ? calls in the
function, and neither of them should actually return an error:

1. `RelativePath::from_local_path` only returns error if the argument
  path is not in the base directory. I.e. in this case, if the argument
  path was outside the timeline directory. Shouldn't happen.

2. latest_metadata.to_bytes() only returns an error if the serde
   serialization fails. It really shouldn't fail.

I considered turning those into panics instead, but as long as the
function as whole can return an Err, the callers need to be prepared
for that anyway, so there's little difference. Nevertheless, I
refactored the code a little to make it atomic even one of those
can't-happen errors happen after all. And I used a closure to make it
harder to introduce new dangerous ?-operators in the future.

test_runner/regress/test_tenants_with_remote_storage.py

@@ -164,7 +164,10 @@ def test_tenants_attached_after_download(
     detail_before = client.timeline_detail(
         tenant_id, timeline_id, include_non_incremental_physical_size=True
     )
-    assert detail_before["current_physical_size_non_incremental"] == detail_before["current_physical_size"]
+    assert (
+        detail_before["current_physical_size_non_incremental"]
+        == detail_before["current_physical_size"]
+    )
 
     env.pageserver.stop()