mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-09-24 22:25:34 +00:00
jeremyhi
9c2933de0b
* feat(log-store): add the object store WAL batch, catalog and I/O Add the three modules between the object format and the store of the object store WAL: - batch: the open batch that accumulates admitted entries into the next object and assigns entry ids at admission. Ids are object-sequence-major, `(object_seq << 20) | position`, with positions starting at one per region per object, so a batch that is rolled back and admitted again under the same sequence hands out the same ids. The time the first entry was admitted is kept for the store's age-based sealing; an empty admission does not start it. - catalog: the in-memory index over object footers, by sequence and per region. Insertion is atomic and rejects an empty footer, duplicate region segments, invalid entry ranges, an already indexed sequence and entry ranges that are not strictly increasing across objects. The next object sequence continues after the largest indexed one and is raised above the largest entry id of any region, and is rejected once it no longer fits an entry id. - io: object store access under `<prefix>/objects/`: a conditional create whose retry with identical content is a no-op and whose conflicting content is rejected, whole and range reads, and a listing of well-formed keys in sequence order. A prefix with a `.` or `..` component is rejected. When the store reports that the object exists but the read that compares its content fails, the read failure is returned with its retry hint instead of the create failure. The modules have no callers until the store lands, so they are declared with `#[allow(dead_code)]`. Only `entry_id` is exported. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * test(log-store): drop comments that restate the batch test assertions Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor(log-store): use pub(crate) in the object store WAL batch, catalog and I/O Signed-off-by: jeremyhi <fengjiachun@gmail.com> * fix(log-store): keep entry_id crate-private and narrow the create collision check Re-export `entry_id` as `pub(crate)`: nothing outside `log-store` uses it yet, and exporting it would freeze the raw `(object_seq, position)` encoding before the store owns id allocation. Treat only `ConditionNotMatch` as the sign that a conditional create collided with an existing object. A store may report `AlreadyExists` for an unrelated path, for example when a parent of the object is a file, and that failure must come back as the write failure rather than as the error of the read that would compare content. Signed-off-by: jeremyhi <fengjiachun@gmail.com> * refactor(log-store): rename the catalog's out_of_order helper to out_of_order_reason Signed-off-by: jeremyhi <fengjiachun@gmail.com> --------- Signed-off-by: jeremyhi <fengjiachun@gmail.com>