A full reconcile that ran out of time resumes after the last object it
processed, in the sorted order of every listed id. It listed the
collections selected at the time it resumed and marked them all loaded
at its end, so a book turned on in between lost the objects that sorted
before that point, and was never loaded again. The reconcile now keeps
the collections it started with; one selected since is loaded after it
like any newly selected collection.
The SyncState's `selected` said which collections were on the device,
and changed only when a load or a removal had finished. A book turned
off and on again while the device was still removing it looked loaded,
so the contacts removed so far never came back; a calendar came back
empty. A book turned on and off again while it was still loading never
looked loaded, so the contacts loaded so far were never removed.
`selected` now names only collections whose rows are all on the device:
a collection leaves it before its first row is removed, and the
contacts SyncState lists collections that are loaded or removed only in
part, so the next run loads them again when they are selected, or
removes the rest when they are not. A book that still holds a change
waiting for its upload stays in that list instead of `selected`.
After an upload the stored state stays where the download left it, and
the engine relied on /changes to report its own creates back. /changes
leaves out an object that was created and destroyed after the state it
is asked from (Stalwart does, RFC 8620 allows it), so an event or
contact created on the device and deleted on the server before the next
download stayed on the device for good.
The SyncState now lists the objects our uploads created, written with
their identity, and the next run fetches them first: one that is gone
loses its rows, an unchanged one writes nothing. An upload-only sync
runs in full while the list is not empty.
The contacts SyncState lists the group rows on the device, so a group
an app hard-deleted (Fossify does) is found by its absence and deleted
on the server. The list changed only with a later state op, not with
the batch that inserted or deleted the rows. When a run stopped in
between (cancelled, out of time, a crash, or turning sync off while a
deselected book was being dropped), a group the engine had removed was
still listed, taken for a hard delete by the next run or by the
teardown's upload, and its card destroyed on the server; a group it had
inserted was never listed, so a later hard delete never went up.
A batch that inserts or deletes a group row now ends with a state op
holding that change, retries included, so the list and the rows never
disagree. A group a download finds unchanged is listed too, for rows
whose batch lost its state op at a provider yield point. And an absent
group whose card is no longer a group or is in no synced book is only
forgotten, never destroyed.
Checked with a crash at every checkpoint of a deselection, a teardown
and a deadline stop during one, a server-side move in a page of several
chunks, and a group written before an interruption and then deleted in
Fossify.
The sync adapters from ed2525e start a headless task for every sync, but
the task only reported "disabled". This adds the engine it runs
(`src/device-sync/engine`, `src/device-sync/jmap`): for one Android
account and authority it downloads the server's changes, uploads what
the device changed, and hands a report back to SyncManager.
- Downloads follow `/changes` from the state kept in the provider's
SyncState. Rows and the state they reach are written in one batch, so
a killed run resumes where it stopped. Without a state, after
`cannotCalculateChanges` or after the user discarded deletions, a full
reconcile compares ids instead.
- Uploads go in three phases across accounts (creates, updates,
deletions), so moving an item to another account never destroys it
before the copy exists. A create carries a uid written to the device
first, and the server is asked for that uid before creating, so a lost
response or a killed run never makes a duplicate. Updates are patches
guarded by `ifInState`; nothing is replaced whole.
- More than 50 deletions that are also more than a fifth of the synced
items hold every deletion back and report `tooManyDeletions`, so
Android asks the user; "undo" restores the items from the server.
- An item the server refuses is marked and not sent again until it
changes or its back-off (1 hour, doubling to a day) ends. Such
refusals and planner errors stay with their item; the rest of the run
goes on.
- Runs of one account and authority take a lock, stop at checkpoints
when cancelled or close to the deadline, and never throw: every
failure becomes an outcome, and a rejected sign-in posts one "Sign in
again" notification.
- The JMAP side uses a client of its own per app account, never the
UI's, since a run can share the JS runtime with the app.
Turning sync off or signing out runs the same upload first
(`teardownAuthority`) and deletes the rows only once nothing is waiting,
or when the user agreed to lose it.
The planners that map items to rows land in the next commits; until
then a run fails before it touches a row.
Checked: 94 engine tests against the fake provider and JMAP server
(sync flows, failures, crashes and retries, teardown), and `npm run
typecheck`, `npm test` and `npm run i18n:check` on this commit.
Linus Rath