Joonas Koivunen
3295d6245a
fix: provide better context for the other test
2023-08-30 14:19:54 +03:00
Joonas Koivunen
a4bfdd8013
test: actually duplicate L1 layer in test
2023-08-30 14:19:54 +03:00
Joonas Koivunen
cd869e5737
handle duplicate l1s almost safely
...
there is still the possibility of the old layer not really being
readable or returning bogus values if the internal representation has
shifted. cannot do anything for that... unless we rename them away, or
something like that.
2023-08-30 14:19:54 +03:00
Joonas Koivunen
544136f13f
fix: require ResidentLayer to keep resident
...
it only matters when calling this, it can get "evicted" after, even
though, it's file will probably not be readable.
2023-08-30 10:31:56 +03:00
Joonas Koivunen
1cbb7fccaa
allow keeping a duplicate resident
...
except that the residentness is not enforced right now.
2023-08-30 10:02:08 +03:00
Joonas Koivunen
9ab35cc5d2
fix: upload compacted layers
2023-08-30 09:45:37 +03:00
Joonas Koivunen
bb03c74217
refactor: rename for_written_tempfile -> finish_creating
2023-08-29 15:57:34 +03:00
Joonas Koivunen
774f34d778
refactor: move all metrics updates to layer
...
this fixes some missing increments for num_persistent_files_created,
persistent_bytes_written and removes double entries for residence
events.
2023-08-29 15:45:03 +03:00
Joonas Koivunen
f5a171076b
doc: comment about chance of both evictions selecting same layer
2023-08-29 15:20:25 +03:00
Joonas Koivunen
881fdbc04c
doc: fix broken link
2023-08-29 14:34:06 +03:00
Joonas Koivunen
8a27e58894
doc: fix outdated commit
2023-08-29 14:19:53 +03:00
Joonas Koivunen
b4c81f7dff
refactor: rename LayerInner::on_drop to on_downloaded_layer_drop
2023-08-29 14:13:09 +03:00
Joonas Koivunen
9151b71b19
doc: fix link
2023-08-29 13:25:36 +03:00
Joonas Koivunen
6e47438f43
fixup residentlayer comment
2023-08-29 12:16:45 +03:00
Joonas Koivunen
579e85e92d
doc: assert &Arc<LayerInner> and DownloadedLayer::owner
2023-08-29 12:16:27 +03:00
Joonas Koivunen
53d2b48ea2
doc: adjust more LayerInner::on_drop
2023-08-29 12:15:46 +03:00
Joonas Koivunen
ac6604b6ed
doc: adjust while in queue
2023-08-29 12:06:47 +03:00
Joonas Koivunen
91b64427ed
doc: remove comment about backoff
2023-08-29 12:04:34 +03:00
Joonas Koivunen
ffe0f90083
doc: when => while
2023-08-29 11:59:24 +03:00
Joonas Koivunen
59b5a55dbf
doc: create guard => new download has been started
2023-08-29 11:57:03 +03:00
Joonas Koivunen
b9290c7005
doc: simplify comment
2023-08-29 11:55:53 +03:00
Joonas Koivunen
302a58e8ea
image/deltalayer: shuffle comments around
2023-08-29 11:55:08 +03:00
Joonas Koivunen
def51361ae
doc: drop comment in favor of drop_eviction_guard
2023-08-29 11:14:12 +03:00
Joonas Koivunen
d67d4b3eee
doc: add validation
2023-08-29 11:11:27 +03:00
Joonas Koivunen
cd1b548a8f
doc: explain DownloadedLayer::get owner param
2023-08-29 11:07:14 +03:00
Joonas Koivunen
282372aa5a
reorder: 1. DownloadedLayer, 2. ResidentLayer
2023-08-29 11:06:51 +03:00
Joonas Koivunen
1f0cd3b50e
doc: note running without remote storage again
2023-08-29 11:06:29 +03:00
Joonas Koivunen
4973419a38
doc: cancellation safety with evict_and_wait
2023-08-29 11:06:12 +03:00
Joonas Koivunen
44ef584842
doc: residentlayer vs. downloadedlayer and eviction
2023-08-29 10:51:08 +03:00
Joonas Koivunen
55c42da91b
info: stop using stat
...
we no longer need to use it because in the latter versions we initialize
to correct on-filesystem state with Layer::for_{resident,evicted}.
2023-08-29 10:49:41 +03:00
Joonas Koivunen
5c343af807
doc: check_expected_download
2023-08-29 10:49:25 +03:00
Joonas Koivunen
87ecb2e6ca
reorder: get and get_or_apply_evictedness
2023-08-29 10:49:13 +03:00
Joonas Koivunen
c659d0f218
fix: subscribe before evicting
2023-08-29 10:48:55 +03:00
Joonas Koivunen
9f7688b1d2
doc: another pass on LayerInner
2023-08-29 10:48:36 +03:00
Joonas Koivunen
3edff352b5
doc: explain what the consecutive failures are for
2023-08-29 10:08:53 +03:00
Joonas Koivunen
08680f6591
doc: typo
2023-08-28 16:55:28 +03:00
Joonas Koivunen
55105ad1c3
refactor: Result<(), NeedsDownload>
2023-08-28 16:53:32 +03:00
Joonas Koivunen
df328758f0
refactor: simplify schedule upload and tests
2023-08-28 16:33:01 +03:00
Joonas Koivunen
d5ac61d566
doc: add cancellation safe comment
2023-08-28 14:41:43 +03:00
Joonas Koivunen
355ea43ac7
eviction_task: remove confusing drop(candidates)
2023-08-28 14:36:34 +03:00
Joonas Koivunen
6f0ab326b4
doc: inmemlayer: cleanup comments
2023-08-28 14:24:38 +03:00
Joonas Koivunen
c06a4fb511
doc: delete fixme about gentlemans agreements and strings
2023-08-28 14:19:11 +03:00
Joonas Koivunen
9a714ac6b8
doc: link to LayerMap::search
2023-08-28 14:10:49 +03:00
Joonas Koivunen
8c21edc9c5
doc: cleanup, add missing "the"
2023-08-28 14:10:49 +03:00
Joonas Koivunen
6ff324a12d
doc: link to inmemorylayer
2023-08-28 14:10:49 +03:00
Joonas Koivunen
090f9a5a80
doc: remove obsolete comment
2023-08-28 14:01:56 +03:00
Joonas Koivunen
53eacacb6b
botched rebase: lost impl AsRe<DeltaLayerInner>
2023-08-28 13:47:03 +03:00
Joonas Koivunen
d40b9a515a
refactor: split guard_against_eviction into three
...
- download
- keep_resident
- download_and_keep_resident
No need to bool enum.
2023-08-28 13:35:53 +03:00
Joonas Koivunen
7c2f687bd6
rename: garbage_collect => &_on_drop
2023-08-28 13:35:53 +03:00
Joonas Koivunen
effc151244
fix: allow dropping from UploadQueue by spawn_blocking
2023-08-28 13:35:53 +03:00
