A contact or event deleted on the phone that another client meanwhile
filed in an address book or calendar the phone doesn't sync was
destroyed on the server by the phone's next sync, and the other
client's move with it. 0faee56 made the sync decide such a deletion on
the server's version, but the planners still destroyed it: the
contacts planner saw the card in no synced book and destroyed it, and
the calendar planner could not take the new version at all (it placed
the event before looking at the deletion), so the old one decided.
The deletion is now moot when the server's version holds none of the
collections the rows were written for and none the phone syncs: only
the rows go, nothing is uploaded, and the object stays where the other
client put it. A deletion still goes up when the book or calendar was
deselected on the phone before the upload, and when the object was
moved into another collection the phone syncs, since the phone would
otherwise miss an object it should show. The calendar planner now lets
a deleted row's shadow follow the server before it places the event.
A contact added on the phone to a group in a read-only address book
(one shared with the user) could never reach the server, yet it stayed
waiting on every run: the planner puts such a membership back, but the
engine never told it which groups are read-only, so only groups of
another account were recognised. Turning sync off or signing out then
always asked about a change that could never go up.
The engine now tells the planner which synced groups are in read-only
books only, from the group cards it holds and the books' rights. The
membership is put back and reported, and nothing is left waiting.
A change the server cannot take was put back on the phone without a
word: an edit in a read-only calendar or address book, an answer
where answers are not allowed, an event moved to another account's
calendar, an occurrence the server keeps without its series, a
contact added to a group this device can't change. The change just
vanished.
The planners now say why they put a change back, and the sync report
lists the item with that reason (`readOnly`, `rsvpRefused`,
`instanceOnly`, `crossAccountMove`, `groupNotWritable`). A contact
whose only change still uploads through a group card is not listed.
A refused edit was also reported as "invitations not sent", as if it
had been stored without them; the note now appears only when the
change was stored without its scheduling messages.
A contact created on the device and put into a group before its first
sync lost that group: the write after its create took the rows' own
memberships as what the server already had, found them missing there
and deleted the rows, so the group card was never patched.
The create's write now starts from no memberships, as its fallback
already did: the device's memberships count as additions, stay on the
device, keep the contact dirty until the group cards show them, and
upload as patches of the group cards.
Checked with a planner test and end to end through the engine.
ContactsProvider keeps a photo within its 96 px thumbnail as the
thumbnail alone, without a PHOTO_FILE_ID, and the thumbnail does not
read back. Such a Photo row looked empty to the planner. Every
download of the card inserted the photo again (two, then three
rows), and any edit of the contact on the device deleted the photo on
the server.
A Photo row now always counts as the contact's photo. Photo rows that
earlier downloads added again are deleted by the contact's next clean
write, one row is kept, and nothing reaches the server for them. A
photo set on the device is still uploaded.
Checked with planner tests and end to end through the engine, with a
provider that stores small photos as thumbnails as the device does.
A contact's rows are written in one provider batch, and one batch is
one Binder transaction of at most 1 MB. A contact with a larger photo
failed with `tooLarge` on every run, stayed on `stale`, never showed
on the device, and its device edits never uploaded.
Photos above about 512 KiB are now kept on the server only: the
contact is written without the photo. The shadow records that the
device holds no row for the card's photo, so the missing row is not
read as a deletion. The same applies to a photo whose bytes could not
be fetched, which an unrelated edit of the contact used to delete from
the server. A photo set on the device still replaces the server's, and
one deleted there is still deleted on the server.
Checked with planner tests for the insert, the size limit, a photo
that could not be fetched and a merge, and end to end through the
engine with the probe's 900 KB photo.
Types Android has but RFC 9553 lacks uploaded as a label naming them,
yet every label read back as a custom type, and MMS and company main
read back as Other and Main. A number typed Car showed up as
"Custom: car" right after the sync, an MMS number as Other, a mobile
email as "Custom: mobile" and a blog link as "Custom: blog".
Such a label now names its type again, for phones only next to the
feature the type uploads with, so a custom label stays custom. MMS
uploads its `mms` label next to `text`, which keeps a text-capable
number from reading as MMS, and `main-number` with `work` reads as
company main.
Checked with a round trip of every phone, email and website type,
both for a contact created on the device and for a type changed on
a synced entry.
Accepting a membership patch of a group card wrote the server's title
over the device's and cleared DIRTY. A rename that had not reached the
server in the same run, because the server refused it or the user made
it after the group's own upload was planned, was silently undone.
The accepted write now keeps a pending rename that the accepted card
does not show: it stores the new shadow but leaves the title and
DIRTY, so the rename uploads against it. A card that shows the
device's title, or a different one from a concurrent server rename,
is taken as before.
Checked with a planner test and end to end through the engine with a
refused rename and a membership change in the same run.
AOSP Contacts edits a postal address as one formatted line; the
provider then copies it into the street and empties the other parts.
The upload sent only the new `full`, so the components on the server
kept the old values: CardDAV clients and the app's editor, which show
the parts, still showed the old city.
A changed line now uploads with the parts it implies. They are read
back along the old line: each of its segments must be some of the old
parts, and a segment whose change is confined to one part rewrites
that part like an in-place edit of its field. When the layout changed
or a change spans two parts, the address keeps the whole line as its
street, which is what the device holds, rather than an address
without parts that the app's editor would show empty and drop on its
next save. An unchanged line still uploads nothing, and editors that
write the parts themselves are handled as before.
Checked with planner tests for the edit from the device test, a
street-only edit, the fallback, a re-inserted line and an editor
that keeps the parts, and with unit tests of the read-back.
A contact added to or removed from a group this device can't change
stayed dirty on every run: the membership upload skips such groups,
while the contact's clean write kept the change pending until the
server showed it. Turning sync off and signing out then always warned
about changes waiting for the server.
The clean write now puts such a membership back to what the server
has, as it does for any edit of a read-only contact, and clears DIRTY;
when that was the contact's only change the upload counts it as
skipped. Memberships of writable groups in the contact's own account
still wait until the group card shows them.
This covers groups of another JMAP account. For groups in read-only
address books the planner is ready, but the engine does not tell it a
group's rights yet; that follows separately.
Checked with planner tests for additions, removals, both kinds of
group and a change next to an edit that uploads, and end to end
through the engine for a group of a shared account, where turning
sync off afterwards finds nothing waiting.
A contact or group deleted on the device was uploaded against the
shadow of its last sync: a download into a deleted item wrote nothing.
When the server had meanwhile also filed the card in an address book
this device does not sync, the deletion still saw only the synced book
and destroyed the card, its copy in the other book included.
A download into a deleted contact or group now writes the new server
card into its shadow, and takes over an identity that was never
written, so deleting our own lost create goes by id rather than by a
uid lookup. The rows and the group title stay as they are: the device
delete still wins. The upload then removes only the synced memberships
of a card that is also in other books, as the calendar planner already
does for events.
Checked with planner tests for contacts and groups and end to end
through the engine with the real planner: the card stays in the
unsynced book, the device rows are purged, and an echo writes nothing.
The contacts planner read RAW_CONTACT_IS_READ_ONLY with every raw
contact, but ContactsProvider takes that column only on writes: every
projection and selection that names it fails with "Invalid column", on
API 32 and API 35 alike. Every contacts sync on a device therefore
failed before it wrote a row. The fake provider accepted the column, so
no test noticed.
The flag is now only written, along with any raw contact write that
happens anyway, so echoes stay write-free. Whether a contact's device
edits are put back is decided from the server's current rights on the
card, which is also right when only the rights changed. The fake now
refuses the column in queries and selections like the real providers,
and every other column the planners query was checked against the real
providers on both emulators.
Checked: the contacts, engine and fake provider tests, `npm run
typecheck` and `npm test` on this commit, and a contacts sync of usera
on the API 32 emulator, which now completes.
This is the planner for contacts (`src/device-sync/contacts`), the last
piece the sync task was waiting for: it turns JSContact cards into raw
contacts and data rows, and device edits back into JMAP writes.
- Names, nicknames, organizations and titles, emails, phones, postal
addresses, websites, birthdays and anniversaries (also without a
year), relations, notes and photos map both ways, with their types and
custom labels. Group cards become groups, and their members become
group memberships.
- Device edits upload as patches of the fields whose rows changed; a
whole map goes up only where Stalwart needs one. Whatever a card holds
that Android can't show (pronouns, online services, keywords, a second
note, an address's coordinates) is never touched, so an edit on the
phone leaves it intact on the server.
- Rows are matched to their card's entries by the keys they carry and,
where an editor re-inserted them without (Fossify saves that way), by
their values. An entry the editor didn't show is never read as
deleted, and the keys Stalwart assigns by position to cards from other
clients are followed when they shift.
- Contacts in read-only address books are put back when the device
changes them. Text beyond what the provider stores is cut on the device
and never uploaded.
Checked: 151 contacts tests (mappings both ways, one test per edited
field, Fossify and AOSP editing, re-keyed cards, merges, groups,
read-only books, a full lifecycle), the engine's tests with both real
planners, and `npm run typecheck`, `npm test` and `npm run i18n:check`
on this commit.
Linus Rath