The app now has two product flavors. `github` is the sideloaded APK as
before, with the same package name and its in-app updater. `play` is a
bundle for Google Play as `org.bulwarkmail.mobile`, without the updater
or the installer permission, since Play forbids apps updating themselves.
- `release-android.yml` builds the Play bundle in a second job, checks
its signer, package name and permissions, and keeps it as an artifact.
With `PLAY_SERVICE_ACCOUNT_JSON` set it uploads it to internal testing.
- The Play flavor uses the new Firebase project; the GitHub flavor keeps
the old one until the app re-registers its token when it changes.
- Widget clicks and the device-sync account type follow the package name.
- Photos come from the system picker without asking for media access,
and the unused microphone permission from expo-camera is removed.
- A manual workflow run without a tag no longer replaces the APK of the
latest release.
Calendars were listed in the order the server returns them (creation
order on Stalwart) and sortOrder was never read. Sort them like the
webmail now does and as JMAP Calendars asks: lower sortOrder first,
equal sortOrder by name (case- and accent-insensitive, numbers by
value), each account's calendars kept together. The store keeps the list
sorted, so the drawer, the event editor and the other pickers agree.
The drawer's long-press panel gains Move up / Move down for the user's
own calendars. A move renumbers that account's calendars 1, 2, 3... and
writes sortOrder only where it changes; the panel stays open so a
calendar can be moved several places.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The settings list now has the account card on top, title-case group
labels and 56 dp rows that show each pane's current value. About and
"About & Data" were one placeholder page with a letter logo, a fixed
name and a store link that pointed at GitHub releases.
- About: the real logo, the app name from app.config.js, the update
state, a source link that a build can set with SOURCE_CODE_URL, and
rows for the server, update checks (Android), open-source licences,
privacy and a prefilled GitHub issue. iPhone updates link TestFlight.
- Storage & backup: usage card, offline mail, settings export/import
and reset, and a Troubleshooting page for refresh and debug logging.
- Sender favicons moved to Appearance, like the webmail.
- The S/MIME and Plugins stub pages are gone; both are webmail plugins.
The licence list is generated by `scripts/generate-licences.mjs`.
The app was locked to portrait in app.config.js and the hand-kept
AndroidManifest.xml. Android 16 ignores that lock on screens 600 dp and
wider, so tablets and foldables rotated into layouts that were never
tested. This removes the lock in both places, keeps React state across
rotation and split-screen resizes (`smallestScreenSize` joins
`configChanges`), and fixes what broke in landscape:
- the thread pager sizes its pages from its own width and snaps back to
the open message after a rotation instead of showing half a neighbour
- compose and the thread view share one keyboard tracker; a rotation
with the keyboard open closes it, since Android sends no new keyboard
frame and the old portrait one blanked the whole landscape composer
- in a short window with the keyboard up the composer hides the format
bar and tightens its header so a few lines of text stay visible
- inputs with suggestions or context around them opt out of Android's
full-screen landscape keyboard
- month views fit whole rows to the real height, keep the month across a
remount, and the month dropdown re-measures and caps its height
- the week grid's header follows the remounted list after a rotation
- the event sheet's actions stay on screen in landscape
- every RN Modal allows all orientations, or iOS forces portrait
Android's WebView shrinks the editor page's visual viewport to the part of
the view that is on screen and pans it to reveal the caret while typing.
The editor is an auto-height WebView inside the composer's ScrollView, so
nothing ever resets that pan: scrolled back to the top, the body is still
drawn shifted up by the pan offset and its first lines sit hidden under the
header fields until a scroll to the very bottom clamps it away.
The page now undoes every pan as it happens (scrolling the page top into
view, since window.scrollTo only moves the layout viewport) and reports the
offset as a `viewportShift` message; the composer scrolls its ScrollView by
the same amount so the caret stays visible. Android only; iOS and the other
RichTextEditor hosts keep their behaviour (hosts without the callback still
get the reset).
Reproduced and verified on a Pixel 4 (Android 13, WebView 153): before,
visualViewport.offsetTop stayed at ~18 px at the top of the composer and
cut the first line; after, it stays 0 and the caret still follows typing
above the keyboard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A draft's attachments and inline images are MIME parts of that draft, and
their blobIds die with the version that holds them. Each save creates the
next version and then destroys the previous one (#849), but the composer kept
sending the blobIds it was seeded with. The first save worked, every later
save failed with "blobId ... does not exist on this server", and so did the
send. This is the second half of webmail #849, which the webmail fixed in
5d2b937a but which was never ported here.
- saveDraftVersion() reads the new version's attachment parts back in the
same request (Email/get on the creation id "#draft"; Stalwart 0.16 does not
resolve a result reference into Email/set "created"). createDraft() keeps
its signature on top of it.
- The composer maps each held blobId to the matching part of the latest
version (inline by Content-ID, the rest by name and size) and sends that.
Uploads are re-pointed too, since an unreferenced upload expires. The map
lives in a ref, so the dirty snapshot does not change on every save.
- Opening a draft with inline images no longer looks like an edit. The
hydration swapped the editor body for its normalised DOM HTML but compared
it with the raw server HTML, so the autosave replaced the draft about two
seconds after it opened. That destroyed the opened version, and a plain
"open, then Send" failed with the same blobNotFound.
Tests: unit tests for the matching and rebasing, a fake-server test that
reproduces the failure without the rebase, and an opt-in live Stalwart test
(verified against Stalwart 0.16.22).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In the day, 3 days and week grids an event could only be started with a
long press, which few people find. Tapping an empty slot now marks it
with a "New event" block, and tapping that block opens the editor at
that hour; a second tap elsewhere moves the block. Long press still
opens the editor straight away. Tapping a day's header in the week or
3 days grid opens that day.
The editor also started events from a slot half an hour late, because
it rounded a time already on the hour up to the next half hour.
The agenda showed each day as a grey header bar above its events. It now
reads like a schedule: the weekday and date sit in a column on the left
of the day's first event (today in a filled circle), a heading marks
where each month starts and a quiet line names each week's range. An
empty today says "Nothing planned. Tap to create." and tapping it opens
a new event. In the drawer the view is now called Schedule.
Events that fill a whole day are listed as all day instead of
"00:00 – 00:00".
The month view used to be a small grid of dots with the selected day's
events listed underneath, so you only ever saw one day's events at a
time. It now fills the screen: each day is a cell with its events as
solid bars in the calendar colour and "+N" for the ones that don't fit.
Tapping a day opens it in the day view, and long-pressing still creates
an event there. Events that fill the whole day no longer show 00:00.
The dotted month stays for the month that drops down under the title.
The calendar header was crowded: four icon toggles, a separate row of
arrows and a Today button, and a small create button squeezed in next
to them. Navigation now follows Google Calendar's layout while keeping
the app's own look:
- the header shows the month; tapping it drops down a month you can
swipe through and pick a day from
- a calendar icon with today's date jumps to today
- Schedule, Day, 3 days, Week and Month are listed at the top of the
drawer, styled like the mail drawer's folders, with Refresh below
- a round create button sits in the corner like the mail compose
button; with tasks on it offers Event or Task
- a new 3 days view shows three columns of the time grid
The arrows stay in the header when free scrolling is off, since the
views can't be swiped then.
With no Trash mailbox, or when Mailbox/get came back as an error, the
Delete action fell through to destroying the email. A shared account
without its own Trash lost mail permanently on one tap.
- only move to Trash; do nothing when there is no Trash or the lookup
failed
- check that Email/set actually updated the email
- show "(No Subject)" in the collapsed text instead of a blank line
- require unlock for Delete (Android 12+)
The tap fix makes a notification open a single page, so the viewer no
longer needs to render up to the opened index. That setting mounted
about N mail panes on every list open, on iOS too. Also drop the
onScrollToIndexFailed handler, which FlatList never calls when
getItemLayout is set.
The drawer hid the whole Unified section unless a second account (with
cross-account on) or a merged group inbox fed it (#843), so a single
account never reached the cross-folder views - although they span the
inbox and every custom folder and repeat no single folder. Sieve-sorted
mail could only be read folder by folder.
Port the webmail's per-view toggles (enableCrossUnreadView,
enableCrossStarredView, enableCrossAllView; off by default) to Settings >
Layout. A turned-on view shows in the Unified section even when the
section would otherwise stay hidden; "All inboxes" and the per-role rows
keep the #843 rule. Alongside the section all three still show, as
before.
The toggles are labelled with the drawer rows' names: their
`label_mobile` in each RN overlay is that language's
`sidebar.unified_all_*` from the webmail catalog, so settings search
lands on the row. Descriptions are the webmail's own.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Uploads only reached internal testers. The public link's "external" group
still served 0.1.66, because nothing added later builds to it or
submitted them for Beta App Review.
A new `public-testflight` job runs after the upload on a Linux runner.
It waits for App Store Connect to process the build, adds it to the
external group, sets "What to Test" to the GitHub release and submits
it for review (`scripts/testflight-public.mjs`, safe to re-run). The
README links the TestFlight beta and the latest APK.
Since releases ship an `.apk.sha256` file, every in-app update checks
the downloaded APK against it. The check read the 40 MB file in base64
chunks and hashed it in JS, which took longer than the download, and it
reset the progress bar to 0 under the "Downloading" label, so the update
looked like it downloaded twice, the second time slowly.
The file is now read and hashed natively through expo-crypto (the JS
hasher remains as a fallback), and the bar stays full under a
"Verifying…" label while the check runs.
The "Sync with address book" toggle under Content & Senders was off until
the user turned it on, so senders trusted on the phone stayed in the
local allow-list and never reached the "Trusted Senders" address book the
other devices read. A setting the user never chose now counts as on;
only an explicit opt-out, or a server without contacts, turns it off.
Adds the phase 2 and 3 results to the verification checklist: the
corrections to phase 1 (two link failures were the test rig), every
action and link re-run on both servers, which commit fixes which cause,
the layout review and what is still open.
Widgets had 22dp corners, rounder than the launcher's own widgets and
the app's cards. They now use 16dp, the Pixel launcher's widget radius,
on the surface, its frame and the picker previews.
The widgets drew every sender as initials, while the app and the
webmail show the contact's photo or the sender's logo first. The
widget avatars now follow the same order: an embedded contact photo,
then the logo of a non-personal sender domain when sender favicons are
on, then the initials.
A widget draws from the stored snapshot and cannot load anything while
it draws, so the pictures are fetched when the snapshot is built and
kept in it as small data URIs. Logos have a cache of their own (a week
for a logo, a day for a domain without one) that survives being
offline, and pictures over 40 KB or in a format Android cannot decode
stay out.
The app had six ways to turn a name into avatar initials, and none of
them was the webmail's. The webmail's Avatar skips punctuation and
symbols in front of each name part ("Sofia" Russo is SR, not "S), takes
two letters of a single name and the first letter of the address
without one. Senders, accounts (sidebar and sign-in), contacts, compose
suggestions and the home-screen widgets all use that now.
The picker previews still showed the layouts from before the fixes in
this series (Month's arrows, the Vacation switch, the old task rows and
triage header). Redrawn from the sample data at each widget's default
size, light theme, with the rounded corners and frame the launcher
draws.
The triage card kept its preview to two lines however tall it was
resized, leaving most of a large card empty, and Latest message showed
no preview at all. Both now use the room they have for the start of
the message.
Folder counts showed only its "Folders" heading when there was no
folder to list, which looks broken; it now says there are no folders to
show. A finished task past its due date still read "Overdue" on the
Tasks widget; it shows its date instead.
The Month widget drew back and forward arrows, and the Vacation widget
an on/off switch. A widget cannot page through months or switch the
vacation reply, so both only opened the app, which the rest of the
widget already does. The arrows are gone; the switch becomes a chevron,
and the Active/Inactive pill under the title still says which state
the reply is in.
Most widget buttons were 32dp tall (Archive, Delete, Reply and Next on
the triage card, Accept, Maybe and Decline, Send wishes, Mail my free
times), Up next's were 36dp and the round compose/new buttons in the
headers 28-32dp, all under the 48dp a finger needs, so taps landed
beside them and did nothing.
Button and Fab take a transparent tap area around what they draw
(`slop`), and the layouts give up the same amount of padding or gap
next to them, so the widgets look the same but the buttons take a 48dp
tap.
Checked every widget at its smallest size with long names, big counts
and German labels. What spilled over:
- the Inbox header at two cells: the count pushed the compose button
off the edge (search now goes at that width, the title truncates
first, and only the unread count stays);
- a long sender pushed the triage card's time and position out;
- German button labels were cut ("Archivie", "Lösc"); the triage
buttons now drop to icons, Archive and Delete first, when the labels
do not fit;
- five-digit counts overflowed Unread count, Tag and Mail and next, and
"/ 23000" wrapped a character per line; counts from 10,000 up are
shortened (10K, 12,3K) and the total goes where it does not fit;
- "ungelesen" was cut beside the number in Mail and next and moves
under it instead.
Every widget header now lets its title give way before the buttons on
its right.
The previous fix started the refresh as the app went to the background,
but it still stopped after its first request: fetch hands every response
over through a JS timer, and React Native pauses timers while the app's
activity is in the background, so the answer was only read once the app
came back. The widgets kept the previous account after a switch and
Home, and changes made in the app reached them late.
The refresh now runs as a headless task of its own,
BulwarkWidgetRefresh, started through BulwarkWidgetRefreshService. A
running headless task keeps the timers going, and the service keeps the
process from being frozen before the widgets are redrawn. Where Android
refuses to start the service the app refreshes in place as before. A
refresh started for a change also no longer joins one that began before
it.
The calendar, tasks, files and scheduled-send parts of the widgets only
refreshed while the app's UI was running, because they reuse the app's
stores, which are bound to its JMAP client. A widget event or push that
woke the app without its UI refreshed mail only, so invitations
answered elsewhere, tasks ticked on another device and new events
stayed as the app last left them. A tick or RSVP the server refused in
that state was also never undone, since nothing reloaded those parts.
When no UI has started in the runtime, nothing else uses that client,
so the widget refresh now signs it in to the active account and loads
calendar, tasks, files and scheduled sends the way the app does. Once
the UI starts it owns the client again.
Event links from the widgets named only the event (and the JMAP account
of a shared calendar), never the signed-in account it was shown for.
With another account active by then, the Calendar tab opened without
the event, or with an id both accounts use, on the other account's
event. The links now carry `appAccount`, and the app switches to that
account first, as it already did for message links.
An RSVP or task tick named only the item's id and looked it up in the
stored snapshot when the tap arrived, then used the snapshot's active
account. Between a refresh and the redraw that follows, a tap made on
the old drawing could reach a snapshot of another account, and JMAP ids
are short per-account sequences, so the lookup could match that
account's task or event with the same id. A tick also flipped whatever
the stored state was, so a quick second tap on a drawing that had not
caught up yet undid the first.
The tap now carries what the call needs: the item's account, its server
ids and, for a tick, the state the user asked for. The change laid over
the widget data only touches items of that account. Drawings made
before this keep working through the old lookup.
Leaving the app scheduled the widget refresh with a zero-delay timer.
React Native pauses JS timers while the activity is in the background
(timers in the bridgeless runtime have no immediate path for a zero
delay), so the refresh only ran once the app came back or a widget task
happened to start. Switching accounts and pressing Home left the
widgets on the previous account's mail. The refresh now starts directly
from the app state change.
A widget button whose JMAP call failed (offline, refused, the account's
credentials unavailable) only logged a console warning, so to the user
the button had simply done nothing, or had worked. The change is now
undone on the widget and a notice says what failed: "Couldn't archive.
Tap to try again." on the triage card, which goes back to the message,
and the same for RSVPs on the invitation card and task ticks on Tasks
and Today overview. Tapping the notice repeats the tap; any new tap or
ten minutes clear it.
The tick on the Tasks widget was a 22dp circle, and the rest of the row
did nothing: a tap on the title or just beside the circle was lost. It
now takes the row's full height and 50dp of its width, and the rest of
the row opens the calendar. Today overview's task rows grow from 32dp
to 48dp for the same reason; the widget fits fewer of them where space
is short.
Every widget tap runs as its own headless task in the same JS runtime,
and the tasks interleave at every await. Each one loaded the stored
snapshot, edited it and saved it back, then refreshed from the server.
That went wrong in three ways:
- two taps loaded the same snapshot and the second save dropped the
first change;
- the refresh of the first tap read the server before the second tap's
archive landed and was stored anyway, so the message came back (seen
on the emulator with a slow `Email/set`);
- a redraw of an older snapshot could land after a newer one, leaving
buttons that act on a message the widget no longer shows.
The snapshot now only holds what the server last said. A button's
change is kept as a pending op next to it and laid over it when a
widget draws; it stays until a refresh that started after the server
took it has been stored (calendar and task changes only count as caught
up by a refresh that loaded calendar and tasks). A call that fails
drops its op, which also undoes an archive made offline: before, the
failed refresh kept the edited snapshot and the message stayed hidden
although it was never archived. Reads, writes and redraws of the
widget data run one at a time, and every redraw draws the latest view.
react-native-android-widget registers every button's PendingIntent with
the current time in milliseconds as its request code and
FLAG_CANCEL_CURRENT. All buttons of a widget share the same action and
provider, so the request code is the only thing telling them apart: two
buttons registered within one millisecond got the same intent, and the
second registration cancelled the first. That button then did nothing
when tapped. The emulator takes 3-10 ms per button, a current phone well
under one, so which button died depended on timing.
The library is patched with patch-package (run on postinstall) to hand
out request codes that count up from the time, unique in the process
and clear of the codes an earlier process left on the launcher. The
patch keeps the library's CRLF line endings, hence `-text` for
`patches/*.patch`.
App store terms put limits on recipients, such as device caps and DRM,
that section 10 of the AGPL does not allow, so the app could not be
shipped through the Apple App Store or Google Play without breaking the
licence.
Add a "Bulwark Mobile" header and a section 7 additional permission to
LICENSE that allows app store distribution as long as the source of
every shipped version is public, free, linked from the listing or the
app, and not further restricted.
The permission is not final yet: not every earlier contributor has
answered the consent request. A reservation limits it to contributions
made on or after 30 September 2026 and to older parts whose authors
have agreed. The README licence line says the same.
Brings feat/device-sync into main: on Android, each account's address
books and calendars can sync both ways with the phone's Contacts and
Calendar apps (Settings -> Contacts / Calendar -> Sync to this
device), through a "Bulwark" Android account with a contacts and a
calendar sync adapter.
- native side: the account, both sync adapters, provider access scoped
to the account, push routing to the syncs, sync problem notices
- the sync engine in `src/device-sync`, run as a headless task
- the settings sections, sign-out and removal handling, translations
- design, limitations and verification in `docs/device-sync.md`
The fixes since the first device runs were checked on the emulators
too: moves that stay the same event, statuses, server deletions, items
moved away before a deletion, quiet runs, the permission flow,
uninstalling and reinstalling, and the settings' collection count. The
list of what only tests cover shrinks accordingly.
The device runs also showed that Stalwart stores a calendar's default
alerts on the series, so removing every reminder of an occurrence is
put back and reported for such a series as well, and that Google
Calendar's first edit of an occurrence brings its reminders back
without a report.
The planner tests that build meetings of 25 attendees on 52 overrides
take 0.3-1.8 s alone, but one of them passed the default 5 s limit in
a full run next to a release build. The two slowest tests of the file
already had 60 s; the other heavy ones get the same.
The account row in the sync settings counts the synced address books
or calendars from its own copy of the list, loaded when the row
appears. The chooser loads a fresh list when it opens, so a calendar
created meanwhile showed there but not in the count ("4 of 5 sync"
stayed after a sixth appeared, and turning it on or off changed
nothing) until the settings page was opened again.
Every fresh list now reaches every view of that account's collections,
so the count follows the chooser.
Brings the doc up to the code after the second round of fixes and the
code review: moves of meetings, deletions of items another client
moved away, big meetings written in several transactions, occurrence
colours and reminders, the stored status of runs that did nothing,
push types by server capability, reminders while the phone's calendar
can't sync, and sign-out waiting for a turn-on in progress.
The known limitations gain what the fixers and the device runs found
but left alone: pairs whose patch fails, a field the server changes
during a pending move, Google Calendar's copy-and-delete moves and its
copied reminders, group cards deleted after they moved away, and a
status line that shows item errors but not conflicts.
A contact or event deleted on the phone that another client meanwhile
filed in an address book or calendar the phone doesn't sync was
destroyed on the server by the phone's next sync, and the other
client's move with it. 0faee56 made the sync decide such a deletion on
the server's version, but the planners still destroyed it: the
contacts planner saw the card in no synced book and destroyed it, and
the calendar planner could not take the new version at all (it placed
the event before looking at the deletion), so the old one decided.
The deletion is now moot when the server's version holds none of the
collections the rows were written for and none the phone syncs: only
the rows go, nothing is uploaded, and the object stays where the other
client put it. A deletion still goes up when the book or calendar was
deselected on the phone before the upload, and when the object was
moved into another collection the phone syncs, since the phone would
otherwise miss an object it should show. The calendar planner now lets
a deleted row's shadow follow the server before it places the event.
A sync that had to start React Native left a listener on the React
host until a context came up. When the start failed (a debug build
without Metro) or never finished, every retried sync added another
one, each holding its run, account and extras, for the life of the
process, and they all fired at once when a context finally came up.
The wait now ends once, and removes its listener, for whatever
reason: a ready context, a start task that faulted, or 10 minutes
(no sync or push job waits longer; JobScheduler stops them). A
context that comes up later runs nothing of it. The adapter's own
check of its run in the callback still covers a run it gave up
sooner.
Turning device sync on retried with the app account's registry id as
the Android account name whenever adding the account failed. That
fallback is for one case only: the login name turned out to belong to
another app account's Android account, which the native module
reports with the `conflict` code. A passing AccountManager failure or
a permission error instead created a second account named after the
opaque id, which then showed in the Contacts and Calendar apps.
Only a `conflict` rejection takes the fallback now; any other failure
fails the turn-on as before.
Turning device sync on or off runs one step after the other per
account, but signing out did not wait its turn. A sign-out that
started while a turn-on was still setting up (or queued behind a slow
turn-off) found no Android account yet and went ahead; the turn-on
then created the Android account, stored it and requested a sync for
the signed-out account, and the next check posted "Sign in to keep
syncing". A turn-off still tearing down could likewise store its
preferences after the sign-out had forgotten the account.
The release before a sign-out is now a step in the queue of every
account it releases, taken in a fixed order: it waits for the steps
under way and releases what they set up. A turn-on or turn-off asked
for while the sign-out runs comes after it, and does nothing once the
account was released; if the user stays signed in it goes ahead.
With calendar sync on and the calendar app reminding, Bulwark skipped
its own reminders for the synced calendars whenever sync was on in the
app, even when the phone's calendar received nothing: its calendar sync
paused in Android's account settings, Android's auto-sync off, or the
calendar permission revoked. Events created on the server meanwhile
never reached the phone, and nobody reminded of them.
The reminder filter now keeps what Android last said about each
account (the permission, `getMasterSyncAutomatically` and the
calendar's `getSyncAutomatically`) and stays quiet only while the
phone's calendar receives events. Auto-sync off counts as not
receiving: Android then runs manual syncs only, and the periodic,
pushed and app-triggered syncs are not manual. Android's side is read
when the scheduler subscribes and again whenever the filter is asked
for; a changed answer makes the scheduler reschedule. Until it was
read, Bulwark reminds: a double reminder is better than a missed one.
ContactsProvider soft-deletes every raw contact and group of a sync
account, also one that never reached the server (DELETED=1, SOURCE_ID
NULL). When a contact or group was created and deleted on the phone
before a run claimed it, no upload phase took the row: without an
identity or a pending create it belongs to no JMAP account, and the
claim step skipped deleted rows. The row stayed for good, and the
upload-only check and the teardown counted it as a waiting change:
every upload sync the providers schedule became a full run, and
turning sync off or signing out always asked about a change that would
be lost.
The claim step now hands such a row to the planner, whose plan for it
is a purge, and `hasLocalWork` and `countPending` leave out deleted
rows that have neither an identity nor a pending create. Nothing is
uploaded for them. Calendar has no such gap: CalendarProvider
hard-deletes an app-deleted event without `_SYNC_ID`, and a claimed
event carries `~pending/<uid>`, so it is soft-deleted with its pending
create and destroyed by uid as before.
The native push router and the push task decided separately whether a
push may carry mail, and disagreed. For an account entry that is not
an object, or a map that names no type, the router started the mail
task as possible mail, but the task's guard read the same payload as
"no mail" and returned, so new mail it announced was never looked for.
A push without a kind was judged by its map in the router but taken
for mail by the task.
Both now apply the same rules: anything but a StateChange, a map that
is missing, unreadable or names no type, an unreadable account entry,
or a mail type means the mail task runs. Only a readable StateChange
naming contact, calendar or other non-mail types skips it.
Checked with one table of payloads run against the router's decision
and the task's guard alike.
On main every push ran the mail task, whose legacy path notifies the
newest unread inbox mail. Device sync narrowed that to pushes naming
EmailDelivery, so contact and calendar changes never notify. But a
subscription made by an older build listens to Email and Mailbox until
the app is opened again, and a server without EmailDelivery reports
new mail only as those changes. Their new mail no longer showed a
notification while the app was closed.
Email and Mailbox now count as possible mail again, in the native push
router and in the push task alike. A push that names only contact or
calendar types still never takes the mail path.
Checked with the push router and push task tests: an Email or Mailbox
change starts the task and the legacy path notifies the message,
next to the syncs a mixed map asks for.
The push subscription got the contact and calendar types whenever the
account had device sync on, without asking whether its server has
contacts or calendars. Sync can be on for such an account: the settings
list every signed-in account, and a server can drop a capability later.
A server that refuses unknown types then refused the update and the
replacement, and push setup failed: a new account got no mail push, an
existing one lost it when its subscription expired.
The types are now added only for a capability the account's session
advertises, and a write the server refuses while it carries device
sync's types is retried with EmailDelivery alone, so mail push never
depends on device sync.
Checked with the push setup tests against a fake server that refuses
unknown types: a new subscription is created mail-only, an existing
one is kept and refreshed, and another account's subscription is left
alone when its server lacks the capability.
The settings show the stored status of the last sync. A sync that
finds nothing to do still replaced it, so a change the server refused
and the phone put back, or a conflict, was shown for a second or two
and then gone. The previous fix covered only the upload sync Android
starts right after a manual one. When changes arrive during a sync,
the adapter runs a full sync right after it, which finds nothing new
on the server and replaced the status 1-2 s after the real one; a
manual sync right after another does the same.
Now what counts is what a run did, not how it started. A run that
wrote nothing either way and reported no conflict, item error, note or
failure keeps the details of the run before it and moves only the time
on. After a failed run it shows the sync as working again, without the
failure's message. Any other run replaces the status, so a problem
stays only until the next run that does something.
Etar moves an event by inserting a copy from its editor and deleting
the old row. Its editor keeps the user's own attendee row out of the
attendee list, so the copy of a meeting that lists the user has no row
for the user, and the copy of one that does not gets the user as its
organizer. The pairing compared the attendee addresses of both rows
exactly, so a moved meeting never paired: it was deleted on the server
and created again under a new uid, the other attendees' answers were
lost, and everyone got a cancellation and a new invitation.
The pairing now leaves the user's own addresses out of that
comparison. Anyone else still has to be on both rows, so a copy that
lost or gained another attendee is not taken for the same event.
Checked with planner tests for both kinds of meeting and a run through
the engine with a fake Etar move.
The fake providers and the fake server were more lenient than the
real ones, so three bugs showed only on a device: an event inserted
without a status, small contact photos, and meetings too big for one
provider transaction.
The fake CalendarProvider leaves STATUS NULL when an app writes none.
The fake ContactsProvider keeps a photo within the 96 px thumbnail as
the thumbnail alone, without a PHOTO_FILE_ID, and serves it to
readPhoto from there. A batch fails with tooLarge when it weighs more
than one Binder transaction, a photo counted by its bytes. The fake
server drops empty maps inside recurrence overrides, as Stalwart does.
Tests now read photos the way the engine does, and the helpers that
made up for the old fakes are gone.
Android starts an upload-only sync right after a manual sync, and 30 s
after any app write. When nothing waited on the device it ended at
once, but it still stored its status, so the settings showed a plain
"synced" and lost the conflicts and item errors of the run just
before it.
Such a run did nothing and now leaves the stored status alone.
Clearing the colour of one occurrence of a coloured series removed the
occurrence's own colour, so it showed the series' colour again after
the sync. The occurrence now gets an empty colour, which Stalwart keeps
and which shows as the calendar's colour, as on the phone.
Removing every reminder of one occurrence sent an empty set of alerts,
which Stalwart does not store: the occurrence then showed the series'
reminders again, without a word. When the series uses the calendar's
default reminders, the occurrence leaves the defaults and has none, as
before. When the series has reminders of its own, the occurrence would
show them whatever is sent, so the change is put back and the report
says why (`remindersNotRepresentable`).
Checked against calcard 0.3.14, with planner tests and runs through
the engine and the fake server.
A recurring meeting with many attendees on many occurrences (25 on 52,
100 on 12) is about 2 MB of provider operations, and one Android
provider transaction carries 1 MB. Every event was written as one
group that the engine never splits, so such a meeting was refused every
run and never reached a phone that synced it for the first time.
An item too big for one group now comes as a chain of groups that the
engine applies one by one, each in a transaction of its own. A new
event is written master first, with its identity, then its occurrences
in groups that fit, then its shadow; a write into an existing event
writes the occurrences first and the series with its new shadow last.
When a run stops between two groups, the next one finds the event by
its identity and writes only what is missing, planned from the old
shadow, so no row is written twice and no server change is lost.
Checked with planner tests for both meeting sizes and every cut point,
and runs through the engine with a provider that refuses transactions
over 1 MB, a crash after every batch included.
When another client moved an event into a calendar this device does
not sync while the device had an unsent edit of it, the calendar
planner refused to merge the server's version, since the event is in
no synced calendar any more. The edit then waited on every run as a
planner error, and the event never left the device.
The server's version now merges into the rows where they are, so the
edit uploads against it without undoing the other client's changes,
and the rows go once the server has taken the edit.
Checked with a planner test and a run through the engine and the fake
server: the series rescheduled on the device keeps the occurrence the
other client added, and no rows are left.
The device sync doc still described the feature as first built. The
review fixes since then changed pairs, merges of items outside the
synced collections, the order of uploads, group handling, photos, the
provider-op scope checks, the settings actions and what the report
says about changes put back on the phone.
Each fix's doc notes are applied against the code as it is now, the
known limitations gain what the device runs found (an identity as
organizer, Stalwart refusing answers in read-only calendars, Fossify
Calendar around DST changes, AOSP's one-line address edits), and a new
Verification section lists what was checked on emulators and the scale
run's numbers.
The strings of device sync were English only: the "Sync to this
device" sections in the Contacts and Calendar settings with their
dialogs, and the notifications about signing in again and about sync
problems. Each language keeps the form of address its file already
uses, the app's own terms, and Android's names for its settings,
auto-sync and the Contacts and Calendar apps.
Checked: every message parses as ICU with the same arguments as the
English one and the plural categories of its language,
`node scripts/rn-missing-translations.mjs <lang>` lists no device sync
key for any language, `npm run i18n:check` and the i18n tests pass.
A contact added on the phone to a group in a read-only address book
(one shared with the user) could never reach the server, yet it stayed
waiting on every run: the planner puts such a membership back, but the
engine never told it which groups are read-only, so only groups of
another account were recognised. Turning sync off or signing out then
always asked about a change that could never go up.
The engine now tells the planner which synced groups are in read-only
books only, from the group cards it holds and the books' rights. The
membership is put back and reported, and nothing is left waiting.
A change the server cannot take was put back on the phone without a
word: an edit in a read-only calendar or address book, an answer
where answers are not allowed, an event moved to another account's
calendar, an occurrence the server keeps without its series, a
contact added to a group this device can't change. The change just
vanished.
The planners now say why they put a change back, and the sync report
lists the item with that reason (`readOnly`, `rsvpRefused`,
`instanceOnly`, `crossAccountMove`, `groupNotWritable`). A contact
whose only change still uploads through a group card is not listed.
A refused edit was also reported as "invitations not sent", as if it
had been stored without them; the note now appears only when the
change was stored without its scheduling messages.
An event created on the phone in a calendar the user may only read was
meant to be removed, since the server takes no changes there, but the
claim step ignored that answer: the event stayed on the phone, was
never sent or reported, and turning sync off always warned that a
change had not reached the server.
The claim step now applies it: the row goes, the report lists it as
read-only, and nothing is left waiting.
Etar moves an event by deleting the row and inserting a copy. When the
event was deleted on the server meanwhile, the sync removed the deleted
row but kept the copy, which then uploaded as a new event: the event
the server had deleted came back in the target calendar, for everyone.
That happened whether the download found the event gone first, which
is the usual case, or the patch of the move did.
A move is an edit, and the server's deletion wins over an edit: the
copy that pairs with the deleted row now goes with it, and the report
counts the conflict. A copy edited beyond the move stays a new event.
Checked through the engine with the event destroyed before the run,
while the move was sent, and with the patch answered notFound.
Etar moves an event, or turns a series into a single event, by deleting
the row and inserting a copy. The pair uploaded the move as a patch of
the event, but then took the copy as it stood for the new baseline and
wrote the server's version over it, so everything else saved with the
move was dropped: a changed reminder came back, and an edit made before
the move never reached the server. The synced occurrences, which Etar
does not copy, were deleted with the old row, losing an occurrence edit
that was still waiting.
The copy now starts from the old row's baseline with only what the move
changed, so any other difference stays dirty and uploads right after
the move as an ordinary update. The old row's occurrences move to the
new row with what they wait to upload. Attendee rows of the copy are
the app's re-insert (Etar writes everyone as required, without a
status) and are not read as edits.
Checked with planner tests and runs through the engine and the fake
server: a reminder saved with the move and a pending occurrence edit
both reach the server, the event keeps its uid, and the echo writes
nothing.
When another client moved a card out of every address book the device
syncs while the device deleted the contact, the engine noted the card
for dropping but never showed its new version to the planner, so the
deletion was decided on the old copy and destroyed the card. A deleted
item is now merged like any changed one first; the contacts planner
then leaves a card that is in no synced book. When the planner cannot
place the object, the deletion stays as it was.
Turning sync off uploads what the device changed without downloading
first; every upload carries the state the device last synced, so the
server refuses it when something changed meanwhile and the engine
downloads first. Before the first sync of an account had ended there
was no such state, and edits and deletions went up unguarded, able to
overwrite what another client had changed. They now wait, and the app
asks before they are dropped; new items still go up.
Turning sync off first uploads what the device changed, then removes
the synced rows. Started from the settings screen, that work yielded
the JS thread with timers between its steps, and React Native pauses
timers while the app is in the background and no headless task runs:
leaving the app stopped it halfway, holding the account's sync lock,
until the app came back. The teardown now waits without timers.
The batch budget counted JSON characters, but a Parcel carries strings
as UTF-16 plus a few hundred bytes per operation, so a full batch was
about 700 KB of a 1 MB Binder buffer the whole process shares. Batches
are now weighed at two bytes per character plus 300 bytes per op and
kept under 500 KB of that. The state op no longer joins a batch whose
last item already fills a provider yield window of 499 operations,
which ContactsProvider refused.
When a group's members change on the server, the download fetches the
contacts whose membership flipped in a chunk after the group's. The
group's chunk already stored its new version, so when the sync stopped
before the members' chunk, the next sync found no flip any more and
the phone kept the old members. Those contacts are now listed with the
group's rows, as objects to fetch first, until their own chunk ran.
A full reconcile that ran out of time resumes after the last object it
processed, in the sorted order of every listed id. It listed the
collections selected at the time it resumed and marked them all loaded
at its end, so a book turned on in between lost the objects that sorted
before that point, and was never loaded again. The reconcile now keeps
the collections it started with; one selected since is loaded after it
like any newly selected collection.
A sync with nothing to download, and the upload before sync is turned
off, used to plan uploads with an empty group index, so the write after
the upload deleted the uploaded contact's membership rows on the phone.
The previous commit's refresh of the index before planning uploads
fixes both; these tests cover the two paths so it stays fixed.
Checked: both tests fail without that refresh.
The SyncState's `selected` said which collections were on the device,
and changed only when a load or a removal had finished. A book turned
off and on again while the device was still removing it looked loaded,
so the contacts removed so far never came back; a calendar came back
empty. A book turned on and off again while it was still loading never
looked loaded, so the contacts loaded so far were never removed.
`selected` now names only collections whose rows are all on the device:
a collection leaves it before its first row is removed, and the
contacts SyncState lists collections that are loaded or removed only in
part, so the next run loads them again when they are selected, or
removes the rest when they are not. A book that still holds a change
waiting for its upload stays in that list instead of `selected`.
The settings chooser and the reminder filter treat only the capability's
primary account as the user's own, while the engine also counted any
account the server marks as personal. On a server that lists a second
such account, its address books and calendars synced although the
chooser showed them off. The engine now uses the same rule as the app.
A provider that is busy may commit a batch up to a yield point and then
fail it. The engine retried every group of such a batch and relied on
the groups' asserts to refuse one that was already committed, but a
download into an unchanged event asserts only the event's own columns:
when it had only added a reminder or an attendee, the retry added it a
second time.
A download is now planned again from a new read after a failed batch
instead of being sent again, which turns a committed group into an echo.
The fake provider can commit a batch up to a yield point to test it.
When the server refused an edit made on the device (a book or calendar
that turned read-only, an event in a calendar where only one's own
events may be changed), the engine put the item back by deleting its
rows and inserting a fresh copy. A contact lost its star, ringtone and
the link to the contacts it was joined with; an event got a new row id.
The rows are now rewritten in place from the server's version, as after
an accepted upload. A refused deletion still brings the item back as a
new copy.
After an upload the stored state stays where the download left it, and
the engine relied on /changes to report its own creates back. /changes
leaves out an object that was created and destroyed after the state it
is asked from (Stalwart does, RFC 8620 allows it), so an event or
contact created on the device and deleted on the server before the next
download stayed on the device for good.
The SyncState now lists the objects our uploads created, written with
their identity, and the next run fetches them first: one that is gone
loses its rows, an unchanged one writes nothing. An upload-only sync
runs in full while the list is not empty.
A new contact is claimed for an address book before its create is
sent, so a retry after a lost response finds it again. When that book
was deleted on the server before the create went up, every attempt
failed and was retried after up to a day, forever. A book that is gone
can hold nothing of an earlier attempt, so the contact is now claimed
again for the book the settings choose.
An event cannot move on its own to another calendar: a create into a
calendar that is gone is no longer sent, and the report says why, until
the user moves the event in the calendar app.
A group whose latest server version the device could not store waits
on the stale list, and nothing uploads for it until it has been fetched
again. Membership edits bypassed that check: adding a contact to such a
group sent the whole member list as the device last knew it, which
removed members another client had added meanwhile. Membership edits
for a stale group now wait for that fetch, and the report lists them.
Whether a membership edit still has to go up is decided against the
group cards on the server, which the engine indexes from their shadows.
The index was only read before planning a download that held contacts,
so in a run that downloaded no contact, such as the upload sync after
an edit, it was empty: removing a contact from a group looked already
done on the server, the contact was marked clean and the removal was
never sent. The engine now reads the index before planning uploads too.
A contact stays marked as changed until the group cards on the server
show its membership edits. The engine uploaded those edits as patches
of the group cards after the contacts, but never looked at the contact
again in that run, so it stayed marked until the next sync. Turning
contacts sync off or signing out right after adding a contact to a
group therefore asked whether to drop one waiting change, although
nothing was waiting.
Once the membership patches are accepted, the contacts that have
nothing else to upload are now cleared in the same run, behind the
same version check as any other clean write.
The contacts SyncState lists the group rows on the device, so a group
an app hard-deleted (Fossify does) is found by its absence and deleted
on the server. The list changed only with a later state op, not with
the batch that inserted or deleted the rows. When a run stopped in
between (cancelled, out of time, a crash, or turning sync off while a
deselected book was being dropped), a group the engine had removed was
still listed, taken for a hard delete by the next run or by the
teardown's upload, and its card destroyed on the server; a group it had
inserted was never listed, so a later hard delete never went up.
A batch that inserts or deletes a group row now ends with a state op
holding that change, retries included, so the list and the rows never
disagree. A group a download finds unchanged is listed too, for rows
whose batch lost its state op at a provider yield point. And an absent
group whose card is no longer a group or is in no synced book is only
forgotten, never destroyed.
Checked with a crash at every checkpoint of a deselection, a teardown
and a deadline stop during one, a server-side move in a page of several
chunks, and a group written before an interruption and then deleted in
Fossify.
When another client moved a contact or an event out of every address
book or calendar the device syncs while the device had an unsent edit
of it, the download only noted it for dropping after the upload. Its
shadow stayed at the old version while the stored state moved past the
change, so the upload passed `ifInState` and resent whole maps built
from the old shadow: a number or an occurrence added on the server
meanwhile was lost. A full reconcile that found such an object did the
same.
The download now merges the new version into the dirty rows like any
other change, so the upload builds on it, and the rows still go once
it is accepted. A contact's collections are read from its shadow:
SYNC1 lists only the selected books and is empty for a card that left
them all, so such a contact was never dropped after its upload. The
calendar planner cannot merge an event that is in no synced calendar
yet; until it can, such an event waits on the device and is reported
instead of overwriting the server.
Checked with the real planners through `/changes` and through a full
reconcile.
Etar moves an event to another calendar, and turns a series into a
single event, by deleting the row and inserting a copy. The engine was
meant to pair the two rows and upload a patch of the existing event,
but it claimed every new row (a uid and a pending create) before it
asked the planner for pairs, and the planner pairs only unclaimed rows.
No pair ever formed: the event was destroyed and created again under a
new uid, which lost what the device cannot show (keywords, virtual
locations, links, the other attendees' answers, alerts relative to the
end) and sent every attendee a cancellation and a new invitation.
Pairs now upload before new rows are claimed, and a paired deletion no
longer counts toward the deletion threshold, since it is a move. When
an app changes the rows while the patch is on its way, they are read
and paired again; rows that no longer pair are left without the
identity instead of having the server's version written into the new
row, which overwrote the edit and left two rows with one identity.
Checked with the real planners: a move and a series turned single keep
their uid, keywords and attendee answers and send nothing; an edit made
during the upload is kept; an unchanged rewrite still pairs.
"Android account settings" opened Android's page for the Bulwark
account through `ACCOUNT_SYNC_SETTINGS` with the account as the
`account` extra. The Settings app of Android 12L and later passes only
`:settings:show_fragment_args` on to that page, which then found no
account and closed at once ("Account provided does not exist: null").
Since starting it succeeded, the fallback to the list of synced
accounts never ran, and the button seemed to do nothing.
The account now also goes in the fragment arguments, where those
Settings apps read it; older ones still read the plain extra.
Checked on the API 32 and API 34 emulators: the button opens the
account's "Account sync" page with its Contacts and Calendar switches.
A contact created on the device and put into a group before its first
sync lost that group: the write after its create took the rows' own
memberships as what the server already had, found them missing there
and deleted the rows, so the group card was never patched.
The create's write now starts from no memberships, as its fallback
already did: the device's memberships count as additions, stay on the
device, keep the contact dirty until the group cards show them, and
upload as patches of the group cards.
Checked with a planner test and end to end through the engine.
ContactsProvider keeps a photo within its 96 px thumbnail as the
thumbnail alone, without a PHOTO_FILE_ID, and the thumbnail does not
read back. Such a Photo row looked empty to the planner. Every
download of the card inserted the photo again (two, then three
rows), and any edit of the contact on the device deleted the photo on
the server.
A Photo row now always counts as the contact's photo. Photo rows that
earlier downloads added again are deleted by the contact's next clean
write, one row is kept, and nothing reaches the server for them. A
photo set on the device is still uploaded.
Checked with planner tests and end to end through the engine, with a
provider that stores small photos as thumbnails as the device does.
A contact's rows are written in one provider batch, and one batch is
one Binder transaction of at most 1 MB. A contact with a larger photo
failed with `tooLarge` on every run, stayed on `stale`, never showed
on the device, and its device edits never uploaded.
Photos above about 512 KiB are now kept on the server only: the
contact is written without the photo. The shadow records that the
device holds no row for the card's photo, so the missing row is not
read as a deletion. The same applies to a photo whose bytes could not
be fetched, which an unrelated edit of the contact used to delete from
the server. A photo set on the device still replaces the server's, and
one deleted there is still deleted on the server.
Checked with planner tests for the insert, the size limit, a photo
that could not be fetched and a merge, and end to end through the
engine with the probe's 900 KB photo.
Types Android has but RFC 9553 lacks uploaded as a label naming them,
yet every label read back as a custom type, and MMS and company main
read back as Other and Main. A number typed Car showed up as
"Custom: car" right after the sync, an MMS number as Other, a mobile
email as "Custom: mobile" and a blog link as "Custom: blog".
Such a label now names its type again, for phones only next to the
feature the type uploads with, so a custom label stays custom. MMS
uploads its `mms` label next to `text`, which keeps a text-capable
number from reading as MMS, and `main-number` with `work` reads as
company main.
Checked with a round trip of every phone, email and website type,
both for a contact created on the device and for a type changed on
a synced entry.
Accepting a membership patch of a group card wrote the server's title
over the device's and cleared DIRTY. A rename that had not reached the
server in the same run, because the server refused it or the user made
it after the group's own upload was planned, was silently undone.
The accepted write now keeps a pending rename that the accepted card
does not show: it stores the new shadow but leaves the title and
DIRTY, so the rename uploads against it. A card that shows the
device's title, or a different one from a concurrent server rename,
is taken as before.
Checked with a planner test and end to end through the engine with a
refused rename and a membership change in the same run.
AOSP Contacts edits a postal address as one formatted line; the
provider then copies it into the street and empties the other parts.
The upload sent only the new `full`, so the components on the server
kept the old values: CardDAV clients and the app's editor, which show
the parts, still showed the old city.
A changed line now uploads with the parts it implies. They are read
back along the old line: each of its segments must be some of the old
parts, and a segment whose change is confined to one part rewrites
that part like an in-place edit of its field. When the layout changed
or a change spans two parts, the address keeps the whole line as its
street, which is what the device holds, rather than an address
without parts that the app's editor would show empty and drop on its
next save. An unchanged line still uploads nothing, and editors that
write the parts themselves are handled as before.
Checked with planner tests for the edit from the device test, a
street-only edit, the fallback, a re-inserted line and an editor
that keeps the parts, and with unit tests of the read-back.
A contact added to or removed from a group this device can't change
stayed dirty on every run: the membership upload skips such groups,
while the contact's clean write kept the change pending until the
server showed it. Turning sync off and signing out then always warned
about changes waiting for the server.
The clean write now puts such a membership back to what the server
has, as it does for any edit of a read-only contact, and clears DIRTY;
when that was the contact's only change the upload counts it as
skipped. Memberships of writable groups in the contact's own account
still wait until the group card shows them.
This covers groups of another JMAP account. For groups in read-only
address books the planner is ready, but the engine does not tell it a
group's rights yet; that follows separately.
Checked with planner tests for additions, removals, both kinds of
group and a change next to an edit that uploads, and end to end
through the engine for a group of a shared account, where turning
sync off afterwards finds nothing waiting.
A contact or group deleted on the device was uploaded against the
shadow of its last sync: a download into a deleted item wrote nothing.
When the server had meanwhile also filed the card in an address book
this device does not sync, the deletion still saw only the synced book
and destroyed the card, its copy in the other book included.
A download into a deleted contact or group now writes the new server
card into its shadow, and takes over an identity that was never
written, so deleting our own lost create goes by id rather than by a
uid lookup. The rows and the group title stay as they are: the device
delete still wins. The upload then removes only the synced memberships
of a card that is also in other books, as the calendar planner already
does for events.
Checked with planner tests for contacts and groups and end to end
through the engine with the real planner: the card stays in the
unsynced book, the device rows are purged, and an echo writes nothing.
After an edit on the phone, every write into a recurring event asserted
each attendee and reminder row of every occurrence. For a meeting with
25 attendees and 52 occurrences with details of their own that alone
made the write 1.6 MB, over Android's 1 MB transaction limit: the event
could no longer be written, went stale in every run and kept its edit.
Only rows that are dirty themselves now assert their attendees and
reminders. A clean row's DIRTY flag already catches an app edit of
them, since CalendarProvider sets it on every app write to an event,
its attendees or reminders, so such a write still fails and is planned
again. The same meeting's writes now take about 0.65 MB.
Checked with the meeting above through the planner, and with an app
edit of a clean occurrence's attendees between planning and writing.
An app that moves an event by changing its calendar in place can move
it into a calendar of another account on the device. A patch cannot
move an event between accounts, so the move was dropped; with nothing
else changed, the row was then marked clean where it stood. The phone
showed the event in the other account's calendar while the server kept
it in the old one, until the next server change moved it back.
The row now goes back to its calendar at once, rewritten from the last
server version. Changes made together with the move still upload, and
their accepted write puts the row back as before.
Checked with a run through the engine and the fake server with a shared
account: the event is back in its calendar and clean, and the server
and the other account are untouched.
In a calendar the user may only answer (mayRSVP without write rights),
an answer to the whole series was uploaded, but an answer to a single
occurrence goes into the occurrence's override and counted as an edit:
it was undone on the next sync without a word.
A change that holds nothing but the user's own answer, to the series
or to occurrences, now counts as an RSVP and is uploaded. Stalwart
0.16.23 refuses every update in a shared calendar without write rights,
answers included; the engine then puts the rows back from the server
and reports the refusal, as it already did for an answer to the series,
so the answer is no longer lost silently.
Checked with a planner test (a new and an existing override, and an
extra edit that is still reverted) and a run through the engine and
the fake server.
In a series that uses the calendar's default reminders, a reminder set
on one occurrence went into the occurrence's override, but the
occurrence still used the defaults it inherits from the series, which
ignore an event's own alerts. The phone showed the default reminder
again after the sync while the server held the new one.
The override now also leaves the defaults (`useDefaultAlerts: false`),
so only that occurrence uses its own alerts and the series keeps the
calendar's defaults. Stalwart stores the flag of an override with the
override (as a JSPROP) and serves it back.
Checked with a planner test (an existing override and a new "this
event" edit) and a run through the engine and the fake server.
Google Calendar leaves STATUS empty on the events and exceptions it
creates, and CalendarProvider has no default for it. Device sync read
the empty value as 0, STATUS_TENTATIVE, so every event and every "this
event" edit made there uploaded as "tentative" and came back to the
phone that way; a copy made with "Copy to" also never paired with the
deleted original.
An empty STATUS now sets no status: a create leaves `status` out, so
the server keeps its default of confirmed, an override gets none of its
own, an edited event never uploads one, a merge takes the server's
without counting a conflict, and a pair ignores it. After the upload
the rows hold the server's status.
A new series from Etar's "this and following" also no longer uploads
the old series' exclusions from before its start, which Etar copies
into the new row's EXDATE.
Checked with planner tests for each path and a run through the engine
and the fake server.
Clearing the description or location of one occurrence of a series in a
calendar app removed the value from the occurrence's override. An
override without a value of its own shows the series' one, so the write
after the upload put "Daily agenda" or "Room 1" back into the occurrence
and made it clean: the user's edit was gone without a trace.
An existing override now gets an explicit empty value while the series
has one to fall back to: the empty text for the description, and a
location without a name, the only empty forms Stalwart keeps (calcard
writes no LOCATION line for an empty map, which then reads back as the
series' locations). Without a series value the override's own is removed
as before. New exception rows keep treating an empty value as "inherit".
Checked with a planner test and a run through the engine and the fake
server: the row stays cleared and clean, and the echo writes nothing.
The native side ANDs the account's scope onto each selection the engine
sends, as `(where) AND (scope)`. A selection such as `1) OR (1` closed
the scope's parenthesis itself: on the calendar tables, where
CalendarProvider adds no account filter of its own, an update or delete
built that way would have reached the events, attendees and calendars
of other accounts, e.g. the user's Google calendar. The engine only
sends constant selections, so this guards against bugs rather than
anything an outside app could send, which is what the scope is for.
- Selections and sort orders must balance their parentheses outside
quoted literals, and may not end the statement, carry comments or
query other tables (`SELECT`, `UNION`, `ATTACH`, `PRAGMA`).
- An update of a data row by id that writes `data1` without restating
the mimetype has the row's mimetype looked up: on a membership row
`data1` names a group, which must be the account's, as on insert.
Such writes by selection are refused.
- `data14`, a photo row's file id, is never written: another account's
file id would have let `readPhoto` read that account's photo.
Checked: 58 JVM tests (`gradlew :app:testDebugUnitTest`, three new), the
debug build, and every selection the engine and planners send against
the new rules.
The design in f666699 was written before the code, and building it
changed a number of details, several of them after runs against the
real providers on the emulators. The doc now says what the code does:
- a sync run's steps in the order they run, the stale list, the
reconcile cursor, the owner check, the time margins, and accounts that
are left out when they refuse to list their collections;
- uploads in phases across accounts, the `ifInState` catch-up, account
refusals, series splits and copied events, creates adopting only in
their target collection, and poison markers;
- the contacts and calendar mappings as built (membership rows, text
limits, postal and relation rules, read-only rows and calendars,
exceptions, reminders under the old owner), and the columns the
providers refuse to return;
- the native module's scoping, the app's store, lifecycle and triggers,
and the failure matrix;
- decisions and limitations, with the questions still open.
Checked: every statement against the code of this branch.
The sync adapter logged only the SyncResult counters of a run, so a
failed sync showed up in logcat as `ioErrors=1` and nothing else. The
reason was in the run's report, and JS warnings from a headless run
don't reach logcat; finding out that a run failed on "Invalid column
raw_contact_is_read_only" took reading the app's storage. The log line
now carries the report's outcome, its message (up to 300 characters)
and the number of failed items.
Checked: 55 JVM tests (`gradlew :app:testDebugUnitTest`, one new) and
the debug build.
On Android, Settings > Contacts and Settings > Calendar get a "Sync to
this device" section with a row per signed-in account. This makes device
sync reachable:
- The switch asks for the contacts or calendar permission after a short
explanation (denied: sync stays off and says why, with a link to the
app's system settings once Android stops asking), creates the
account's Bulwark account in Android and syncs right away. Turning it
off uploads what the device changed first and asks before anything
that could not go up is lost. The switch shows Android's own state, so
sync turned off in Android Settings shows as paused.
- Which address books or calendars sync: the personal account's are on
by default, shared ones and the Trusted Senders book off; read-only
ones are marked, and a note says when another signed-in account
already syncs the same shared one.
- The address book new contacts from other apps go to, who reminds of
synced events (the calendar app or Bulwark, asked when calendar sync
is turned on while Bulwark's reminders are on), and how often Android
syncs on its own (every 15 minutes to every 6 hours, or not at all).
- "Sync now", the last sync with its result, or the error with what to
do about it ("Sign in again", "Grant access", "Review deletions"), and
a link to the account in Android Settings.
- Accounts the app signed out while they synced are listed with "Sign
in again" and "Remove from this device".
The settings search finds both sections. iOS, and Android builds without
the native module, show nothing new.
Checked: the collection listing and status tests, the settings search
and i18n tests, `npm run typecheck`, `npm test` and `npm run i18n:check`
on this commit, and on the API 32 emulator with usera on the local
server: the section, turning contacts sync on (Android account created,
152 contacts on the device, one per card) and the status after a run.
434c554 left an account that refuses to list its address books or
calendars out of the run, but reported it as an item error. The sync
settings then said "1 item could not be synced" after every run for an
account that merely shares a mailbox: nothing the user can fix, and no
item behind it. Such an account is now only logged.
Checked: the engine tests, `npm run typecheck` and `npm test` on this
commit, and the sync settings of usera on the API 32 emulator after a
new run.
The contacts planner read RAW_CONTACT_IS_READ_ONLY with every raw
contact, but ContactsProvider takes that column only on writes: every
projection and selection that names it fails with "Invalid column", on
API 32 and API 35 alike. Every contacts sync on a device therefore
failed before it wrote a row. The fake provider accepted the column, so
no test noticed.
The flag is now only written, along with any raw contact write that
happens anyway, so echoes stay write-free. Whether a contact's device
edits are put back is decided from the server's current rights on the
card, which is also right when only the rights changed. The fake now
refuses the column in queries and selections like the real providers,
and every other column the planners query was checked against the real
providers on both emulators.
Checked: the contacts, engine and fake provider tests, `npm run
typecheck` and `npm test` on this commit, and a contacts sync of usera
on the API 32 emulator, which now completes.
Stalwart lists an account shared with the user for mail only with every
capability, and answers `forbidden` when the user asks for its address
books or calendars. The engine treated that like any failed request, so
on the local test server every sync of usera ended with an I/O error and
never reached the device, only because userb shares a mailbox.
An account that refuses to list its collections (`forbidden`,
`accountNotFound`, `accountNotSupportedByMethod`) is now left out of the
run and reported as an item error; the other accounts sync as before.
The rows written for such an account stay on the device: a refusal can
pass, and dropping the account's contacts and events for one refused
request would throw away device edits that are still waiting.
Checked: three new engine tests (a shared account that never listed,
one that stops listing its address books, one that stops listing its
calendars), which fail without the fix, `npm run typecheck` and
`npm test` on this commit, and a contacts sync of usera on the API 32
emulator, which now ends `ok` with one item error.
When an account's calendars sync to the device and the user chose the
device's calendar app to remind them, that app gets the events'
reminders. Bulwark's own reminder scheduler now skips the events of
those calendars, which would otherwise remind the user twice. Tasks
never sync and keep Bulwark's reminders, and a change of who reminds, or
of which calendars sync, reschedules right away.
Checked: the reminder filter and scheduler tests, `npm run typecheck`
and `npm test` on this commit.
Signing out of an account (or of all of them, or removing one) now first
uploads what the device changed in its synced contacts and calendars,
then deletes them and removes its Android account, while the credentials
still work. When changes could not be uploaded (offline, server errors)
the app asks before signing out and losing them; staying signed in
keeps the account syncing. Several accounts are released in parallel,
and an internal error never keeps anyone from signing out: the next
check against Android's accounts stops what was left from syncing.
A successful sign-in clears the account's "Sign in again" notification
and resumes an account that stopped syncing when it was signed out.
Checked: `npm run typecheck` and `npm test` on this commit (the release
and resume paths are covered by the lifecycle tests of e67e240).
With device sync on, Android should hear about a change as soon as the
app does, not only at the next periodic sync. This adds what tells it:
- The app's own contact and calendar edits, StateChanges on the live
stream for `AddressBook`/`ContactCard` and `Calendar`/`CalendarEvent`,
and the app coming back to the foreground request a sync of that
account and authority. Requests within 5 s fold into one, and a
StateChange the engine caused itself (its own upload echoing back) is
skipped.
- While the app is closed, the account's push subscription carries
those types too. The FCM and UnifiedPush services hand every push to
the native router from f743ab0, which requests the syncs, and start
the mail push task only when a push has `EmailDelivery`; the task also
returns early for such pushes, so a contact change can never show the
"newest unread mail" notification.
- At launch, on foreground and when Android's accounts change, the app
checks Android's accounts against its own: an account removed in
Android Settings shows sync off and is not recreated, and one whose app
account is gone stops syncing and posts "Sign in again", keeping its
data.
The functions that turn sync on and off (`src/device-sync/app/lifecycle.ts`)
come with this; the settings that call them land in a later commit, so
nothing here runs before an account has device sync on.
Checked: the new store, lifecycle, trigger and push tests, `npm run
typecheck`, `npm test` and `npm run i18n:check` on this commit, and the
debug build.
This is the planner for contacts (`src/device-sync/contacts`), the last
piece the sync task was waiting for: it turns JSContact cards into raw
contacts and data rows, and device edits back into JMAP writes.
- Names, nicknames, organizations and titles, emails, phones, postal
addresses, websites, birthdays and anniversaries (also without a
year), relations, notes and photos map both ways, with their types and
custom labels. Group cards become groups, and their members become
group memberships.
- Device edits upload as patches of the fields whose rows changed; a
whole map goes up only where Stalwart needs one. Whatever a card holds
that Android can't show (pronouns, online services, keywords, a second
note, an address's coordinates) is never touched, so an edit on the
phone leaves it intact on the server.
- Rows are matched to their card's entries by the keys they carry and,
where an editor re-inserted them without (Fossify saves that way), by
their values. An entry the editor didn't show is never read as
deleted, and the keys Stalwart assigns by position to cards from other
clients are followed when they shift.
- Contacts in read-only address books are put back when the device
changes them. Text beyond what the provider stores is cut on the device
and never uploaded.
Checked: 151 contacts tests (mappings both ways, one test per edited
field, Fossify and AOSP editing, re-keyed cards, merges, groups,
read-only books, a full lifecycle), the engine's tests with both real
planners, and `npm run typecheck`, `npm test` and `npm run i18n:check`
on this commit.
The engine hands every event it syncs to a planner that turns it into
CalendarContract rows, and device edits back into JMAP writes. This is
that planner for calendars (`src/device-sync/calendar`), now used by the
sync task:
- Calendars become calendar rows with what the apps need (SYNC_EVENTS=1,
an owner, colours, the reminders allowed); calendars of shared accounts
carry the account's name.
- Events keep their time zone, floating and all-day times, recurrence
rules, excluded and changed instances, reminders, attendees and the
user's own reply. A rule Android can't hold (RSCALE, SKIP, leap months)
stays on the server untouched, and local times Stalwart can't resolve
(a start in a repeated hour, an UNTIL in a skipped one) are written in
a form it keeps.
- Device edits upload as patches of what differs from the last synced
state, never a whole event, and only what the device can show is ever
sent. A single-instance edit or deletion becomes an override of its
series; "this and following" caps the series and creates a new one.
How Etar and Fossify save (a cancelled instance instead of an
exclusion, attendee rows deleted and re-inserted) is read as the edit
it stands for, not as a deletion. Invitations go out on the changes a
calendar app would notify about.
- Events in read-only calendars and subscribed feeds are put back when
the device changes them; only the user's own reply goes up, where the
server allows it. Tasks, task-only calendars and the birthday calendar
never reach the device.
- Reminders are written as Reminders rows when the calendar app is to
remind the user, and left out when Bulwark does.
Checked: 179 calendar tests (download, upload, exceptions, the edits
Etar, Fossify and CONTENT_EXCEPTION_URI make, echoes that write nothing,
time zone and DST edges), and `npm run typecheck`, `npm test` and `npm
run i18n:check` on this commit.
The sync adapters from ed2525e start a headless task for every sync, but
the task only reported "disabled". This adds the engine it runs
(`src/device-sync/engine`, `src/device-sync/jmap`): for one Android
account and authority it downloads the server's changes, uploads what
the device changed, and hands a report back to SyncManager.
- Downloads follow `/changes` from the state kept in the provider's
SyncState. Rows and the state they reach are written in one batch, so
a killed run resumes where it stopped. Without a state, after
`cannotCalculateChanges` or after the user discarded deletions, a full
reconcile compares ids instead.
- Uploads go in three phases across accounts (creates, updates,
deletions), so moving an item to another account never destroys it
before the copy exists. A create carries a uid written to the device
first, and the server is asked for that uid before creating, so a lost
response or a killed run never makes a duplicate. Updates are patches
guarded by `ifInState`; nothing is replaced whole.
- More than 50 deletions that are also more than a fifth of the synced
items hold every deletion back and report `tooManyDeletions`, so
Android asks the user; "undo" restores the items from the server.
- An item the server refuses is marked and not sent again until it
changes or its back-off (1 hour, doubling to a day) ends. Such
refusals and planner errors stay with their item; the rest of the run
goes on.
- Runs of one account and authority take a lock, stop at checkpoints
when cancelled or close to the deadline, and never throw: every
failure becomes an outcome, and a rejected sign-in posts one "Sign in
again" notification.
- The JMAP side uses a client of its own per app account, never the
UI's, since a run can share the JS runtime with the app.
Turning sync off or signing out runs the same upload first
(`teardownAuthority`) and deletes the rows only once nothing is waiting,
or when the user agreed to lose it.
The planners that map items to rows land in the next commits; until
then a run fails before it touches a row.
Checked: 94 engine tests against the fake provider and JMAP server
(sync flows, failures, crashes and retries, teardown), and `npm run
typecheck`, `npm test` and `npm run i18n:check` on this commit.
The engine plans its provider reads and writes in JS, but only native
code can reach ContactsProvider and CalendarProvider. This adds that
bridge to the BulwarkDeviceSync module, with the rest of what the engine
and the app need from it:
- `query`, `applyBatch`, `readSyncState` and `readPhoto` take the
`ProviderQuery`/`ProviderOp[]` JSON of `src/device-sync/types.ts`,
check it (`sync/ProviderOps.kt`) and run it for one Android account of
our type (`sync/ProviderScope.kt`, `sync/ProviderIo.kt`). Every
selection gets the account ANDed in, every row an op names by id or
points to as a parent must be the account's, and every URI carries
`caller_is_syncadapter`, so nothing the sync writes marks rows dirty or
schedules an upload. The calls run one at a time on the module's own
thread through an unstable provider client, so a long batch never
stalls other module calls and a provider that dies fails the call
instead of the app.
- `showSyncProblem` and `clearSyncProblem` post and clear the one
notification per account that needs the user, e.g. "Sign in again".
- `setPushRoutes` stores which Android accounts each JMAP account feeds,
for the push router in `sync/PushRouting.kt`; the push services start
using it in a later commit.
- A run now ends exactly once and refuses a report that comes after
that. A sync requested while a run is open makes the run ask for
another sync when it ends, because SyncManager drops such requests as
duplicates. `onPerformSync` lets nothing escape any more: an exception
there killed the whole process, UI included.
- `ensureAccount` no longer hands an existing Android account to another
app account (`conflict`): its rows and SyncState belong to the first.
Payloads cross as JSON strings, which the bridge moves much faster than
nested maps. `src/device-sync/native.ts` is the typed JS side. The fake
provider follows the rules the emulator runs turned up: contacts
`data1`..`data14` read back as text, events and calendars the sync
adapter inserts have DIRTY NULL, attendee rows are addressed by id, an
op by id expects one row, and a vanished parent fails with `assert`
while another account's fails with `scope`.
Checked: 54 JVM tests (`gradlew :app:testDebugUnitTest`), debug and
release (R8) builds, `npm run typecheck` and `npm test` on this commit,
and on the API 32 emulator through the debug receiver: inserts with
back-references and group memberships, SyncState, photo writes and
scaled reads, asserts and expected counts, both scope refusals, the
499-op yield boundary, `tooLarge` for a 1.3 MB batch, and the
notification.
AOSP Contacts, and the apps built on its account-type parser, check the
EditSchema in `xml/contacts.xml` and drop the whole account type when it
breaks one of their rules: every data kind with a type column needs
`<Type>` children, the other kinds must have none, and name, photo,
nickname, organization, note and group membership need maxOccurs="1".
The schema from ed2525e listed phone, email, postal, event and
relationship without types, so the editors would not have offered the
Bulwark account at all.
The kinds now list the types device sync maps to JSContact, and the
rules are written down in the file so the next edit keeps them.
Checked on the API 32 emulator (AOSP Contacts): with a Bulwark account
present, "Create new contact" offers it and opens the editor saving to
it.
The device sync engine reaches Android's providers through a native
bridge and the server through a detached JMAP client, and neither runs
in vitest. This adds the stand-ins the engine and the mappers are tested
against, and the contracts they implement:
- `src/device-sync/types.ts`, `planner.ts`, `wire.ts`,
`android-columns.ts`: the bridge, the provider op model, the JMAP
port, and the interface between the engine and the contacts and
calendar planners
- a fake ContactsProvider and CalendarProvider with the behaviour the
design relies on: the DIRTY and VERSION rules, soft and hard deletes,
exception linking, normalisation, asserts, the yield-point limit, and
a "user" face that edits the way AOSP Contacts, Fossify and Etar do
- a fake Stalwart with `/changes` paging and compaction,
`cannotCalculateChanges`, `ifInState`, uid checks behind a lagging
search index, positional re-keying and fault injection (lost
responses, 429, auth, SetErrors)
- shared pure helpers (canonical JSON, PatchObjects with Stalwart's
missing-parent rule, key and uid minting) and a test that walks the
import graph, so no mapper or engine module pulls in React Native
- `src/stores/device-sync-store.ts`: the per-account preferences and
last run status the engine and the settings will share
Checked: `npx vitest run src/device-sync` (91 tests), the full suite
(2209 tests) and `npm run typecheck`.
Issue #34 asks for contacts and calendars that show up in Android's own
apps and whose edits there reach the server. docs/device-sync.md is the
design the implementation follows: Android's sync framework drives the
runs, a thin Kotlin shell starts a headless JS task, and the engine is
TypeScript on a detached JMAP client, so token refresh, mTLS and session
discovery keep a single owner. A pure-Kotlin engine was rejected: it
would have had to redeem rotating refresh tokens outside that owner and
re-implement the client.
It pins down what every provider column holds, the JSContact and
JSCalendar mapping tables, how device edits are told apart from provider
normalisation (a baseline per row), how rows are matched when an editor
re-inserts them, the merge and conflict rules, uploads as patches with
`ifInState`, the native API, the failure matrix and the limitations.
The platform and server facts it relies on come from AOSP's providers
and calendar/contacts apps, Fossify, Etar and Stalwart v0.16.23's
source. These were checked on the local test server:
- map keys written through JMAP survive patches; `uid` duplicates are
refused; unknown and plural recurrence properties are rejected
- a start or `until` in a DST gap or overlap is silently dropped (a
daily rule came back unbounded); `Etc/GMT-2` keeps both
- `ifInState` yields `stateMismatch`; `relatedTo/<key>: null` does not
delete; result references to `/set` are refused
An adversarial review of the first draft found 20 paths that could lose
data, duplicate items or loop (own creates duplicated after a lost
response, unsynced edits dropped on deselection or eviction, Fossify
rewrites uploaded as edits, Etar moves turned into destroy + create,
and more); the design was changed for each before any engine code.
Android syncs the contacts and calendars of other services through
accounts and sync adapters. This adds the Android side of device sync:
a "Bulwark" account type (`${applicationId}.account`, set in
build.gradle), an authenticator that sends Settings → Add account to
the app instead of asking for credentials, and one sync adapter each
for the contacts and the calendar provider.
An adapter does no syncing itself. It starts the BulwarkDeviceSync
headless JS task from the main thread, through the helper the push job
uses, waits for the task's report and hands it to SyncManager as the
SyncResult. The engine that will write the report runs in JS, so the
existing client stays the only owner of token refresh, mTLS and
session discovery; for now the task only reports that device sync is
off. Nothing in the app creates an account yet, so the feature stays
unreachable until its settings land. Debug builds get an adb hook for
the device tests (DeviceSyncDebugReceiver, shell only).
Hard errors are reported without progress: SyncManager retries a
failed sync at once when it made progress, which would loop an auth
failure.
Checked on the API 34 and API 32 emulators with the debug build, from
logcat and `dumpsys content`:
- app in the foreground: report after 319-334 ms
- app in the background: 320-327 ms
- process killed: SyncManager starts it for the sync service, React
Native starts headless, and the report arrives after 16.2 s (API 34)
and 8.5 s (API 32), most of it the debug bundle download from Metro
- an io report is retried with back-off and an auth report is not
retried; too many deletions shows the system's "Attempted to delete
too many Contacts." notification
- with Metro unreachable the adapter gives up after 2 s instead of
waiting for its deadline
- `gradlew :app:testDebugUnitTest`: 10 JVM tests for the report mapping
The code that starts React Native for a headless JS task, when the
process was woken without any UI, lived inside BulwarkPushJobService.
The device sync adapters need the same start, so it moves to a shared
HeadlessJs object, with a small startTask helper next to it.
Push behaviour does not change:
- onStartJob still calls the start code on the main thread, and the
listener-then-ready-check order is the same code, moved verbatim
- the task still starts from `UiThreadUtil.runOnUiThread`, its id is
still recorded in that same UI-thread turn, and a refusal because the
app came to the foreground still logs and finishes the job
- the start code now returns React Native's start task, so a caller can
notice a start that fails (a debug build with no Metro); the push job
ignores it
The helper documents that it must run on the main thread: the ReactHost
is created on first use without a lock, so a background caller racing
MainActivity could create a second one.
Checked with `gradlew assembleDebug -PreactNativeArchitectures=x86_64`.
Changing every occurrence of a series takes its changed and cancelled
occurrences along, and two can still meet on one occurrence (bda118b).
A series on Mondays and Tuesdays that starts a day and an hour later
takes its first Monday's change onto the Tuesday, at the new time,
where the Tuesday's own override lands as well. Only an override that
stayed exactly where it was won such a meeting. Here neither did, so
the order the server listed the overrides in decided, and it could
keep the Monday's change and lose the Tuesday's cancellation.
The override that stays where it is now wins, then the one that stays
on its own day, then the first; the others are left out and logged, as
in the web app. The rest of a series split "this and following" keys
its overrides the same way.
The app records which server filled each account's caches and checks it
with every new session (a330d77). A read of that record that failed -
a storage error, such as a busy database - answered "none recorded",
so the session counted as the first sight of the account: its queued
offline edits were dropped, with a notice that the mail server had
changed when it had not, and the record was written anew. Had the
server been replaced, that record hid the change from every later
check.
A read that fails now fails the check: nothing is dropped or written,
and the next session checks again. As with any failed check, signing
in goes on.
A switch that began while an account was being added, or while a cold
start still brought the last account up, made that sign-in or restore
give up (efd31de). When the switch then failed as well, what it put
back was no working account:
- during an add-account sign-in, the switch fell back on a snapshot of
the client, which served no account before the new session was up
and the new account after. The app stayed on the account in use while
every call failed with "Not connected", the session retry saw a
session and returned, and tapping the account did nothing; or the
client served the new account under the one in use, whose calendars,
contacts and identities then came from it, and whose sends went
through it.
- a cold start shows the last account's cached mail, drawer and all,
while its session loads. A switch from there that failed left the
app on the sign-in screen, with every account still registered.
A switch now falls back on the connection an add-account sign-in under
way saved, and resumes the queue of offline edits the sign-in held, as
a failed sign-in does itself. It waits for a restore under way and
switches from the account restored. A sign-in waits for switches under
way and a restore for neither, so none of them waits on another.
Two faults in the new series "This and following events" makes, found
on 0.16.23 with the app's own code against the test server (1.0 did
not show them):
- The new series named its organizer only when the editor wrote one,
which it does only for an event that had none. Created without it,
0.16.23 makes the user who creates the event its organizer, whatever
the owner participant says: a sharee who split the owner's meeting
became the organizer of the rest (organizerCalendarAddress
mailto:usera, isOrigin false, the invitation sent from the sharee),
and a series organized under an alias would have moved to the user's
main address. 1.0 takes the owner participant. The new series now
carries the series' organizer address whenever it has participants.
- The overrides the new series takes over kept the old series' SEQUENCE
while the new series starts at its own, and 0.16.23 listed each such
occurrence twice, as changed and as the series has it, for the
organizer and the attendee alike. The overrides now leave the old
number behind.
Three faults in how a series' changed and cancelled occurrences follow a
change of the whole series (found in review, live on 1.0 and 0.16.23 for
the web's version of the same code):
- A rule that names its days - a series on Mondays and Tuesdays, one on
the 5th and the 6th of each month - keeps its occurrences on those days
when its start moves. Moving every occurrence by a day moved each
override along anyway, so the Monday's landed on the Tuesday's key and
one silently overwrote the other: the cancelled Tuesday came back with
the Monday's title, and the Monday lost its change. Under such a rule
every override but the first occurrence's now stays on its day, at
the series' new time; the same holds for the new series of "this and
following". Two overrides that still meet on one key (an hourly series
turned all-day) no longer overwrite each other: the one that stays
where it is, else the first, is kept, and the others are left out and
logged.
- "Does not repeat" for every occurrence kept the first occurrence's
override on the single event, because a series' start counts as an
occurrence of any rule: cancelled, 1.0 left the event out of every
range query (still stored); retitled, 0.16.23 listed it twice. A series
made single now keeps no overrides.
- The SEQUENCE a change of every occurrence raises (0.16.6 to 0.16.23
refuse a lower one) came from the series as the app held it. A series
opened from a reminder or a link was that copy, possibly older than the
server's. The store now reads a series from the server every time a
scope edit asks for it, so the raise, the overrides a split or a move
rewrites and the rule all start from what the server holds.
More edits of a recurring meeting went wrong at the edges of a series
(0.16 and 1.0 alike, attendees included):
- A series the app shows as itself - opened from a reminder or an
event link, or one an edit just made recurring (0.16 before 0.16.20
showed it until the next reload) - took "This event only" as a change
of the whole series: `CalendarEvent/set` on the series' id with the
editor's title, start and rule. "Delete this event only" deleted the
series. The series now stands for its first occurrence: a change
becomes an override of it, in two writes when the user organizes the
meeting, sends invitations and an attendee has answered (Stalwart
sends a new override only to attendees who have not), and a delete
excludes it. "This and following" from the first occurrence changes,
or deletes, the whole series instead of ending it before its first
occurrence. A series opened from a reminder on a calendar shared with
the user is now known to the store, so the change goes to the owner's
account under the series' own id.
- An edit that makes an event recurring, or single again, now loads the
range again, so the grid shows the occurrences instead of the series
itself.
- 0.16 (0.16.23 checked) reads a rule's UNTIL as the series' wall clock
taken for UTC, so in a zone behind UTC a split or "delete this and
following" kept the occurrence it was to end before. The UNTIL is now
earlier by the zone's offset (the larger of the split's day and the
day before), which ends the series there on 0.16 and 1.0 alike.
- 0.16.6 to 0.16.23 send a change of one occurrence with its SEQUENCE
raised but do not keep the raise, so a later change of every
occurrence - a new title - went out with a lower number for that
occurrence and the attendee's server refused it. A change of every
occurrence that the user as organizer sends, of something the
attendees see, on a series with changed occurrences, now raises the
SEQUENCE of the series and of each changed occurrence by one. 1.0
raises and keeps the number itself; the result there is the same. The
app's update sends `sequence` as it is, so nothing has to keep it.
- An existing override written on the base event (0.16 before 0.16.20,
and ranges the device expands) gets the series' organizer again when
it has none: 0.16.19 refuses an answer to such an occurrence. 1.0
refuses the key inside an override once, and the store writes the
override again without it.
A series keeps its changed and cancelled occurrences as overrides keyed
by recurrence id, where the rule puts each occurrence. The app's series
edits left them behind and made more of a series than was asked (0.16
and 1.0 alike, attendees included):
- "All events" that moved the series kept the overrides under the old
ids: a cancelled occurrence came back, a retitled one showed twice.
It also wrote everything the editor showed to the series, so a new
room on an occurrence with a title of its own renamed the series,
and it measured a move on the calendar's clock, so across a week in
which only one of two zones changes to or from summer time the series
moved an hour more or less than picked.
- "This and following events" ended the rule but not the overrides
after the split, which showed on next to the new series; the new
series got the whole COUNT again and started at the opened
occurrence's own time in the calendar's zone. Deleting "this and
following" left the later overrides too.
- A failed create of the rest of the series put back only the rule, and
without telling the attendees the truncation had told.
Now, as on the web (`lib/recurrence-scope-edit.ts`, a port of the
webmail module of the same name):
- "All" moves each override with its occurrence in the same update: by
the series' move, onto the same day at the new time when the series
turns all-day or timed. An occurrence moved on its own keeps its time,
except the one the editor was opened on; a new rule drops the
overrides it no longer has, and overrides that move ask their
attendees again when the series does. Only the fields the user
changed go to the series, and the move is measured on the series'
own clock.
- The split removes the overrides from the split on together with the
rule and gives them to the new series, re-keyed and with its
participants (0.16 keeps an override as an event of its own). The new
series goes on from the split's recurrence id in the series' zone,
moved only by the editor's move; its COUNT is what is left, an UNTIL
stays, and a rule the user changed is taken as it is. It takes the
old series' details, not the opened occurrence's own ones.
- A failed create puts back the rule and the whole overrides map (a
pointer fails on a series left without overrides), with the
truncation's scheduling, and fails the save; a store that answers
with nothing counts as failed.
- The truncation tells the attendees only when the user organizes the
series and sends invitations; an attendee ends their own copy
quietly.
The scope handling moved from CalendarScreen into the new module, where
it is tested. The editor hands its dates to the save. An invitation to
one occurrence alone, which has no series, is changed or deleted as
itself for every scope instead of making a second event.
Someone who writes to a calendar shared with them (read & write) changes
the owner's meeting there on the owner's behalf: Stalwart sends the
change from the addresses of the account the event is in. The app
decided who organizes by the user's own addresses only, so it took the
sharee for an attendee (0.16 and 1.0 alike):
- the editor wrote no participants, hid "Send invitations" and sent
nothing, so the attendees kept the old time and their answers
- "All events" asked nobody again, and "This and following events"
made the rest a new series that kept the old answers, which Stalwart
sends to nobody who had answered
The app now reads `isOrigin`, which both versions compute from the
addresses of the account the event is in: true for the owner's meeting
and every expanded occurrence of it, false for an invitation the owner
got. `userOrganizes` counts such a copy as the organizer's; an
invitation is still changed without messages. "This and following"
asks by the old series, since the new one carries no `isOrigin` yet.
The editor no longer leaves the user's own main address off the
attendees it opens with, only the organizer's (or the user's, for an
event that names none yet): a user who invited their main address to a
meeting organized under an alias lost it on the next save.
"Send now" and "Reschedule" create a replacement submission and then
cancel the original. The server checks the recipients again for the
replacement, and a throttle used up since the original was taken ("452
4.4.5 Rate limit exceeded") can refuse some of them. The app kept the
original only when every recipient was refused; on a partial refusal it
cancelled it, and a recipient the original had taken lost the message.
Port of the webmail's aa99f038.
- the original's delivery status is read with its envelope, in the same
request; when the replacement refuses a recipient the original took,
the replacement is withdrawn (cancelled, or only its record destroyed
when it took nobody) and the original goes at its old time to
everyone it took
- a recipient the original was refused for as well loses nothing, so
the reschedule goes ahead and names it; without a delivery status of
the original, every recipient counts as taken
- a replacement that can't be withdrawn lets the reschedule go ahead,
naming the refused recipients: "Send now" holds it for a second only,
so it may be out already, and keeping both would send the others two
copies; a withdraw whose answer got lost is read back, and when it
went through the original is kept, since going ahead would reach
nobody
- the alert in Scheduled and the undo bar's toast lead with "The
message is still scheduled for ..." and say "Nothing was changed: the
server refused ..."; they used to open with "Not sent", which invited
sending the message a second time
Checked live on 0.16.23 and 1.0 with a relay-refused recipient: the
lookup, going ahead, and withdrawing a partly refused replacement while
the original stays at its old time. One new string, with the webmail's
translations in all 27 languages.
- The app crashed on every cold start whose cached folders held a
shared or group folder, which on Stalwart 1.0 every group member has:
the sidebar works out the accounts to span while it renders, before
the session is up, and the JMAP client's account id throws until one
is. The client now also names its account without throwing, for code
that runs while rendering; the mail list's account helpers (the
sidebar's accounts, a row's account, the viewer's pages) use it, and
a read receipt sent by itself waits for the session. API calls still
throw without a session.
- Signing out of the last account left the app on an empty mail view
("No mailboxes found") with nothing to retry: the account list was
read before the account was removed, so the app switched to the
account it had just signed out of, which did nothing. It is read
again now, and the sign-in screen shows. A failed switch to the next
account no longer puts the signed-out one back: when its server does
not answer, that account shows offline, as on a cold start, and one
whose sign-in is refused is skipped.
- After signing in with the same address on another server (another
port) or on a server reinstalled at the address (0.16 to 1.0), Files
kept showing the old server's drive without asking the new one, and
actions on its rows would have used the old server's ids. The local
account id leaves the port out and stays the same there, so the
drive now also starts over when the server behind the session
changes, and is read again with every new session. The account id
itself is unchanged: it keys the stored credentials and every
per-account cache, and changing it would sign such accounts out and
orphan their data.
A first change to one occurrence that keeps the attendees' answers goes
out in two writes: the override as the occurrence stands, quietly, then
the change as an update of it. The occurrence is looked up again in
between, because the new override can give it another synthetic id.
When that lookup failed, the change went to the old id. Up to Stalwart
0.16.20 synthetic ids are positions in the series' expansion
(calendar_event/mod.rs), so after the new override the old id can name
the next occurrence, which then got the change and its scheduling
messages. 0.16.21 and 1.0 key the ids by recurrence id, and the id
stays (checked live on 1.0 and 0.16.23).
Now, when the occurrence is not found again, found twice, or the lookup
fails, nothing more is sent and the editor says the change was not
saved. The quiet override changes nothing that shows; the calendar is
read again so a retry uses the ids the series has now, and goes out as
an update of that override, which reaches every attendee. The message
is translated into all 26 languages.
The app compares the server behind an account with the one it recorded
and drops what came from a replaced one. The released version recorded
none, so on the first start of this one there is nothing to compare
with, and a server replaced before the update (Stalwart 1.0 is a fresh
install) went unnoticed:
- the push subscription expiry the released version recorded made the
subscription the new server lacks look revoked, and push was turned
off for good; it is now forgotten on the first sight of an account, so
the subscription is re-created
- the queue of offline edits the released version left was replayed
against the new server, where its ids name other messages, so a
queued delete deleted another message; it is now discarded on the
first sight, with the notice a replaced server shows. Nothing proves
a queued edit belongs to the server: the queue records none, a
reinstalled Stalwart may accept old JMAP states, and a queued move or
delete took its message out of the cached list, so there is no copy
to compare the server's message with
iCal subscriptions recognise their mirror calendar by name, but a sync
still imported the feed into whatever calendar the recorded id named:
- the feed now goes only into a calendar that carries the
subscription's name, and otherwise into a new mirror; when the name
can't be read, the sync fails and is tried again
- a rename the server refused renamed the subscription anyway, which
then no longer recognised its mirror: pruning stopped and removing the
subscription kept the calendar. Now nothing changes and the error is
shown
- a replaced server claimed subscriptions made before they recorded
their account for every account with the same JMAP id, another
server's account included. They now go only when no other signed-in
account has that id, or may have it, and a sync that finds their
mirror records the account on them
Adding an account put the new account's session in the JMAP client as
soon as it was up, while the queue of offline edits (outbox) stayed on
the account in use, and ready to replay, until the new account's server
was checked. A retry, a network flip or the return from the OAuth
browser in that window replayed the queue through the new account, and
edits to the user's own mail took its JMAP account id. Stalwart's ids
repeat across accounts, so a queued move or delete of one account's
message "x" hit the other account's "x". An edit made in that window ran
the same way, and so did the rest of a replay already under way.
Account switches could overlap too: tapping B, then C before B's session
was up, left the mail list on C while B's switch, finishing last, put B
in the client and its queue in the outbox, so edits in C's list went to
B under C's ids. An older switch that failed last put back the account
it had left, under the newer one.
- the outbox replays and runs edits only while the client serves the
account the queue belongs to, checked before every op, and holds the
queue while another account signs in; a failed sign-in gives the
client and the queue back to the account in use
- the outbox resumes only for the account the mail list shows
- every account change (a switch, a sign-in, a sign-out) takes a
generation, and one a newer change overtook touches neither the client
nor the stores; the client no longer installs the session of a load or
sign-in that a newer one overtook
- a failed switch puts back the account the first of overlapping
switches left, and tapping that account while they are under way goes
straight back to it
- a session retry gives way to a switch or sign-in under way
- tokens refreshed for an account being left stay with that account;
they were saved into the credentials of the account loaded meanwhile,
which then signed in as the other one
Two places took a narrow refusal from the server for a complete one.
Port of the webmail's 7977a6ed.
- Sending at a full submission quota (Stalwart 1.0): when making room
found nothing, no search ran for five minutes, for any send. Scheduled
mail and undo-send holds can go once their message is out, and until
the next search every send failed with "ask your administrator". A
send the user starts, a read receipt included, now always looks again;
the pause stays for automatic read receipts.
- vCard export: Stalwart answers Blob/get only in accounts the user is a
member of, so for an address book shared with the user it came back
`forbidden` ("You are not an owner of account ...") and the export fell
back to one photo download after the other. It checks the access to
each blob by itself, whichever account the request names, so
`getBlobsAsBase64` now asks again in the user's own account, on 0.16
and 1.0 alike.
Stalwart runs the SMTP RCPT stage while it creates a submission and
records a refused recipient as `delivered: "no"` with its reply: a relay
it doesn't allow ("550 5.1.2 Relay not allowed.") or a used-up throttle
("452 4.4.5 Rate limit exceeded, try again later."), on 0.16 and 1.0
alike. The create still succeeds and onSuccessUpdateEmail files the
message in Sent. The app read only the new record's id and sendAt, said
"Email sent", and on Stalwart destroyed the record right after, the only
trace of the refusal. The message was lost without a bounce. Port of
the webmail's e0b0dfe0.
- the send reads the new submission's delivery status in the same
request (`EmailSubmission/get` of `#sub-1`, its creation id; the
create's own answer has none), also with the retry after a full
submission quota, before the record is destroyed
- every recipient refused: the send fails with the server's reply per
address, the copy filed in Sent is removed and the previous draft
kept, so the composer or the quick reply keeps the text and the
address can be fixed; held sends (undo send, scheduled) are refused
when they are created, so this covers them too
- some recipients refused: the others get the message, and a warning
names the refused ones with their replies
- "Send now" and "Reschedule" check the replacement the same way and
keep the original, which still goes out at its old time, when every
recipient of the replacement is refused; a partial refusal there gets
the warning as well
- the submission prune leaves a shared account that refuses its records
with `forbidden` alone while the app runs, for that login, with a
debug log instead of a warning, and the scheduled list skips it:
0.16.23 lists an account that only shared an address book or a
calendar with the submission capability
`RecipientsRefusedError` and `RefusedRecipient` are added to
`jmap-result.ts`. Five new strings in all 27 languages, four of them
with the webmail's translations.
"Send now" and "Reschedule" at a full submission quota (Stalwart 1.0)
made room and then created the replacement again. Making room can take
a while - a prune, a walk through the change log - and the original
could go out meanwhile, so the replacement sent it a second time, with
a HOLDFOR that still counted from before the wait. The original is now
read again first: nothing is resubmitted unless it is still pending
("This message was already sent" or "no longer scheduled" otherwise,
and the undo bar lets go of it), a failed read changes nothing, and the
hold is worked out again from the time the user picked. Port of the
webmail's 65eb4b10.
- the Stalwart explanation of a full quota (Bulwark removed what it
could, an administrator has to raise the limit) needs
`urn:stalwart:jmap` on the submission account; another server's
`overQuota` gets a generic sentence with its own description
- when making room finds nothing that can go, the account is not
searched again for five minutes; each send, reschedule and automatic
read receipt at a full quota used to run the prune, the change-log
walk and the gets again
- unsubscribing by mail at a full quota explains the quota instead of
only saying "Unable to unsubscribe"
- push on Stalwart 1.0 (`principals:owner` on the account) takes the
emailpush capability as the answer to which accounts notify: a group
with only mail no longer counts as unreachable, and a refusal
remembered under 0.16 no longer keeps a group out of the map
- a push re-registration the user asked for at the server's
subscription limit removes the old record first and tries once more;
the old record counted against the limit, so it could never be
replaced
- "Mark folder as read" on a folder and its subfolders goes on past a
folder the server refuses and reports the refusal at the end; the
folders and the list refresh either way
- marking, starring or tagging a selection writes every account before
reporting a refusal and shows only the messages the server updated;
one refusing account or message used to leave the rest unwritten,
and what was written looked unchanged
- a new photo for a contact opened from the cache is not saved when the
card's media can't be read from the server either; it would have
replaced the card's other media (a logo, more photos)
Every save in the event editor rebuilt the participants with new ids
and put every attendee back to needs-action, the user's own answer on
an invitation included. Saves now keep each attendee's id and answer;
new attendees are asked. As on the web, only an organizer who moves an
event (its start, end, all-day or recurrence rule; not a time zone
rewrite, and not a zero-length event) asks the others again.
The series scopes did not fit that:
- "All events" wrote the opened occurrence's start, length, zone and
participants onto the series: a stretched occurrence stretched every
one, and one day's decline spread to the whole series. The series now
keeps its own list and takes only the attendees the user added or
removed, and its times change only when the occurrence moved (by the
same delta), its rule only when it changed.
- "This and following" makes a new series, which Stalwart sends only to
attendees who have not answered: its attendees are asked again.
- Stalwart sends a new occurrence override, the first change to one
occurrence, only to attendees who have not answered (0.16.6 to 1.0).
A change that keeps answers now goes out in two writes: the override
as the occurrence stands without scheduling, then the change as an
update of it, to the occurrence looked up again. Checked live on 1.0
and 0.16.23.
- A save of an invitation someone else organizes writes no participants
while the user left them alone, and never asks for scheduling, so the
organizer gets no reply for a changed alert. The switch to send
invitations is hidden there.
Also:
- a reminder change on one occurrence of a series that uses the
calendar's default alerts turns the defaults off for the series first
(keeping what it shows under ids of its own); 1.0 kept showing the
defaults
- all-day and floating occurrences expanded on the device no longer
carry the master's utcStart/utcEnd, so their reminders are scheduled
for each day; master instants that are not the master's own (0.16.23's
year 32548, or the next occurrence's) are ignored
- the editor warns when the attendees and the organizer go over the
calendars capability's maxParticipantsPerEvent, and a refusal for it
(Stalwart 1.0 enforces it, answers included) is explained in the
editor, the series dialogs, the detail sheet and the invitation
banner, in all languages; a failed save used to fail silently
- refused event creates, updates and answers keep their SetError type
- Calendar/get asks for its properties by name
When the server behind an account is replaced (a fresh Stalwart 1.0
install at the address 0.16 ran on), its ids start over. The app
already dropped its caches then, but kept what the user set up with
server ids inside:
- iCal subscriptions kept the ids of their mirror calendars. The next
sync deleted the "stale" events of whatever calendar now had the id
and filled it with the feed, and "Remove subscription" destroyed it.
Now the account's subscriptions keep their feed, name, colour and
interval but lose their mirror, and the next sync makes a new one as
adding a subscription does. Renaming or removing an unlinked one
touches no calendar. Events are only pruned from, and remove only
destroys, a calendar that still carries the subscription's name.
Subscriptions now record the signed-in account, so accounts on two
servers that got the same JMAP id no longer share them.
- the preferred sender identity kept its id, which a new server can
give to another address (live: identity "b" is usera@ on 0.16.23 and
team@ on 1.0). It is forgotten for the account's old and new JMAP
id, which the app now records with the server fingerprint.
- hidden calendars kept their ids. The list is shared by all accounts,
so it is cleared.
Offline changes the drop discards are now announced once ("3 offline
changes were discarded because the mail server changed"), in all
languages.
Revoking this device from another one turns push off here again: a
subscription gone before its recorded expiry counts as revoked, as it
did before the server check came in. A replaced server forgets the
recorded expiry, so there the subscription is re-created instead.
The queue of offline edits (outbox) and the offline mail cache were
attached to an account only when the email store switched to it. A cold
start restores the view of the account it showed last, so there was no
switch: the queue from the last session was never loaded or sent, an
edit made meanwhile was dropped with a console warning, and the offline
cache neither stored nor served bodies.
On an account switch the queue was loaded and flushed right away,
before the new account's session was up and before the check for a
replaced server. The JMAP client still served the previous account
then, so queued edits could reach it.
- the email store attaches the outbox and the offline cache whenever it
is pointed at an account, also the one it already shows
- the outbox holds its ops until the auth store resumes it, once the
account's session is up and its server was checked; edits made before
that are queued, and edits made while the queue loads are kept
- the JMAP client serves no account while it loads or signs in to
another one; a failed switch puts the previous account and its queue
back
- dropping an account's queue or cache for a replaced server cancels a
read of it still under way
- the offline sync stops when the cache or the client moves to another
account, instead of filing one account's mail under another
Stalwart 1.0 refuses keywords over 128 bytes and more than 100
keywords on one message, per message in Email/set `notUpdated`; a
read-only shared folder refuses any keyword on every version. The app
stopped a large selection at the first refused batch, "mark folder as
read" ignored refusals, and the list's swipe, bulk and tag actions let
the rejection go unhandled, so nothing was said. The keyword sort
polarity verdict was stored per server address for a week, so a server
reinstalled as 1.0 (which sorts as the RFC says, where 0.16.8-0.16.22
invert) kept the old reading.
- Email/set writes send every batch before reporting the first refused
message, and "mark folder as read" marks the rest, then reports the
refusal. The list shows "Tagging failed" or "Failed to update email"
for swipe, bulk and tag actions, and the viewer's automatic
mark-as-read puts the message back to unread when it is refused.
- The sort polarity verdict is keyed by the server fingerprint as well
(stored as `keyword-sort-polarity:v2`), so a replaced or upgraded
server is probed afresh. An upgrade that leaves the capabilities
alone, such as 0.16.22 to 0.16.23, still waits out the week.
- Tests pin that Stalwart 1.0's `$hasattachment` / `$hasnoattachment`
never show as tags or as tags to recover; the app lists no other
keywords, and tag ids stay within 128 bytes by construction.
Stalwart 1.0 advertises `urn:ietf:params:jmap:metadata` by default, but
its handlers are still `todo!()`: a get that selects `metadata` or
`privateMetadata`, or a /changes call with `ignoreMetadataOnlyChanges`,
panics the request on the server. The app never sends either, but its
Mailbox/get and SieveScript/get calls asked for the server's default
properties, which a later release could grow to include metadata.
- Mailbox/get (own folders, shared accounts' folders, by id, and the
polling fallback) and SieveScript/get ask for explicit RFC property
lists, the same as today's defaults on 0.16 and 1.0 (checked live).
- A test checks the requests the folder, filter, delta-sync and polling
paths build, on 0.16.23 and 1.0 sessions, and scans the app's sources
for metadata properties, the /changes argument and the capability.
Stalwart 1.0 has no migration from 0.16, so the way there is a fresh
install at the same address, where ids start over ("a", "b", ...) and
name other objects. The app persists a lot that only means something on
the server it came from: JMAP states and query states, offline bodies
keyed by message id, an outbox that replays stored ids (a `destroy`
included), the calendar cache and push de-dupe lists. It also turned
push off for good when the server no longer had the subscription, and
would have taken another device's subscription for its own once the
reinstalled server handed out the recorded id again.
- Email/changes, Email/queryChanges and Mailbox/changes answers whose
Stalwart state went below the one asked from count as not computable
and lead to a full fetch (`stalwartChangeId`, a port of the webmail
guard). It only reads accounts that list `urn:stalwart:jmap` and
states in the exact form Stalwart writes, so it never trips while
states grow or on other servers.
- Every time a session is established, a fingerprint of the server (API
URL, primary account id, the session's and the primary account's
capability names) is compared with the one recorded for the account.
On a change, before anything reads them, the account's folders,
lists, states and query states, offline bodies, queued and failed
outbox ops, the calendar and contacts caches and the push de-dupe
lists are dropped. Preferences, templates, tags and iCal
subscriptions stay. The first sight of an account only records it.
- The whole outbox goes: every op names a message by its server id.
Offline edits still waiting are lost, but never a message the user
wrote, because the outbox holds no sends.
- A subscription the server no longer has counts as expired and is
re-created, and the recorded one is only ours when its device client
id matches, on setup and on teardown. Revoking this device from
another one therefore no longer keeps push off here; turning it off
on the device does.
Stalwart refuses a push subscription's whole emailPush map as a
`forbidden` SetError when it names a mailbox shared by ACL (0.16.16 and
later, and 1.0), and 1.0 stops listing emailpush on such accounts. The
app narrowed the map on any `forbidden`, including a refused method
call, and put every account with folders in the first attempt, so 1.0
cost one refused call per install. Worse, a failed listing counted as
"the server has no subscription" and dropped the key of a working
registration, and any failed refresh - a network error, a timeout, rate
limiting - started a replacement.
- `jmap-result` gains `JMAPSetError` (a `JMAPMethodError` with the same
name, carrying `properties`) and `isJMAPSetError`. `assertSetResult`
throws it, so every refused object can be told apart from a refused
call; existing `instanceof` checks and the outbox's name-based error
classification are unchanged.
- The emailPush map holds the primary account and the accounts whose
`accountCapabilities` list `urn:ietf:params:jmap:emailpush`, which
leaves ACL shares out up front on 1.0. On 0.16.16-0.16.23 the refused
map still narrows to the accounts that look owned, then to the
primary account, but only on a `forbidden` SetError.
- A subscription ends only with a verified replacement, a user disable
or the server's verdict: a failed listing aborts the run, and only a
SetError on the refresh starts a replacement. Other failures leave
the subscription and its key alone and the next run tries again.
- `forbidden` and `overQuota` read as plain text in settings and the
onboarding prompt, and settings says that mailboxes others share
don't send background notifications. New strings in all 27 languages.
Files in the app assumed Stalwart 0.16 in several places:
- Stalwart 1.0 lists the owner's folders above a node shared with the
user, without dates and with every right false. The app took the
owner's top folder for the shared root, so the folder actually shared
sat several levels down, under folders where rename, delete and move
could only fail. Those placeholders are now left out of the root and
of folder listings.
- A node counted as a folder when it had no blobId, so 1.0's symlinks
showed up as empty folders that could be opened. Listings ask for
`nodeType` (returned since 0.16.6, never sent in FileNode/set), and
a symlink is an inert row with a link icon: it can be renamed, moved
and deleted, but not opened, downloaded, duplicated or shared.
- Only "/" was checked in names, and uploads were not checked at all,
so `<>:"\|?*` already failed on 0.16. New folder and rename now apply
the naming rules from the filenode capability plus 1.0's stricter
ones (control characters, 255 bytes, reserved stems such as
`con.txt`) and say what is wrong. Uploaded names are made acceptable
instead: `con.txt` becomes `con_.txt`, also on 0.16.6 to 0.16.23.
- One refused file stopped a whole upload. The batch now goes on and
reports the files that failed at the end.
- Every media type over 30 characters, which is every Office document,
was stored as application/octet-stream. New files get the bare
lowercase media type 1.0 requires, and no `size`, which 1.0 refuses
when it differs from the blob.
- File errors keep their JMAP type, and the alerts say why an action
failed: too many files or folders (`overQuota`), folders nested too
deeply, a refused name, missing rights, or a file too large.
The app stores its filter rules as JSON inside a `/* */` comment at the
top of the Sieve script. A `*/` anywhere in a rule name, a value or the
auto-reply text ended that comment early and turned the rest of the
rules into script, and a CR or LF in a rule name did the same to its
`# Rule:` comment. Stalwart (sieve-rs, on 0.16 and 1.0) also accepts
only the letters A-Z, digits, `-` and `_` in a header name. Any of these
made the server refuse the whole script, so saving any filter, and the
auto-reply with it, failed.
- `*/` in the embedded rules is written as `*\/`, which JSON reads back
as `*/`, so every client parses the rules as before
- the `# Rule:` comment carries the rule name without line breaks, and
the parser matches the comment back to its rule by that form
- the filter editor trims header names, turns autocorrect off for them
and refuses to save one the server would reject, with a translated
message; the generator leaves a rule with such a header name out of
the script, since dropping only the condition could widen an "all of"
rule, and keeps it in the stored rules for editing
When Mailbox/changes answered with `hasMoreChanges` (Stalwart caps one
answer at 5000 changes, other servers earlier), the folder sync applied
that page but kept the old state and ran again, which asked for and got
the same first page back, without end.
The sync now follows `newState` page by page, storing each page's state
as it applies it. After five pages, or when a page does not move the
state on or can't be calculated, it reads the whole folder list instead,
which takes one request whatever changed.
Four smaller calendar changes that Stalwart 1.0 needs.
- Time zone: 1.0 fails every CalendarEvent get and query whose
timeZone its database lacks, with invalidArguments "Invalid timeZone".
It matches IANA names case-sensitively and knows no offsets, so a
"+01:00" or "europe/berlin" that Intl accepts empties the calendar.
0.16 reads such a zone as UTC. The zone is now checked with a get of
no events that rides along with the first calendar list; once refused,
reads ask in UTC and log one warning, and a read refused before the
check answered is repeated once in UTC. The comment that said Stalwart
ignores bad values is corrected.
- Default alerts: 1.0 fills a calendar's default alerts into the
alerts of an event that uses them. The editor wrote them back with
useDefaultAlerts false on every save, freezing today's defaults into
the event. It now writes nothing while the reminders are unchanged; a
change turns the defaults off and saves the alerts under ids of their
own, since 1.0 drops alerts saved under a default alert's id.
- Calendar colour: renaming sent the colour along, and 1.0 accepts only
#rgb, #rrggbb or a CSS name, so a calendar another client coloured
with an 8-digit hex could not be renamed. The edit sheet and the iCal
subscription settings now send only the fields that changed.
- Reply stubs: 0.16 stored an attendee's answer to one occurrence
without a title, place or description, and 1.0 builds that occurrence
from the override alone. Hydration now takes a missing title from the
series, and the place and description when the override removes the
title.
Stalwart 1.0 refuses Principal/query with `forbidden` unless the
administrator allows directory queries, which is off by default. The
app took the missing Principal/get answer for an empty directory: the
calendar share sheet said "Sharing is not available on this server.",
the file share sheet said "No other users found", and existing shares
showed bare ids.
- `listPrincipals` says whether the listing is complete, `restricted`
or failed. On `forbidden` it falls back to Principal/get without ids,
which 1.0 still answers with the user's own account, their groups and
the accounts that shared something with them.
- Principal/get asks for `id`, `type`, `name`, `description` and `email`
only; 1.0's default list adds `capabilities` and `accounts`.
- Both share sheets explain a restricted directory under the search
box. Only 1.0's own refusal ("The administrator has disabled directory
queries.") gets the wording that points at Stalwart's sharing
settings; any other refusal gets a generic one, and a failed listing
says so.
- A sharee the app can't name shows as "Unknown user" with its id, and
can still be changed or removed.
On 0.16 with the default roles the listing is complete and the sheets
behave as before.
Stalwart 1.0 reads a text filter value that ends in `*` as one prefix
over the whole string, not split into words and capped at 16 bytes. The
app added `*` to every word it searched for, so on 1.0 `project budget`
became the prefix "project* budget", an address was no single word and
`Geschäftsführung` (18 bytes) was over the cap: all three found
nothing. Quoted phrases broke too, since `"annual report" 2024` went out
as one value ending in `*`.
The search text now goes out as typed, split the way the webmail does
it (5967a1c4): each quoted phrase and each `word*` becomes a condition
of its own, ANDed with one condition that holds the other words, so a
query without quotes or stars is sent unchanged. The folder, all-folders
and tag-view searches, the unified inbox and the from, to, subject and
body filters all use it, and `searchEmails` puts `inMailbox` next to
the text conditions in one AND instead of on an operator.
0.16's built-in index dropped the `*` anyway, so its results stay the
same; there a quoted phrase is an AND of its words. On 1.0 a typed
`word*` is now a real prefix.
The fake server of the store tests now matches text the way Stalwart's
built-in index does, which the old `*` after every word fails.
Stalwart 1.0 returns recurrence overrides as RFC 8984 PatchObjects,
with keys like participants/p1/participationStatus and nulls that
remove. Expanding a series on the device spread them onto the
occurrence, so a declined occurrence stayed accepted, a changed room
stayed the same and the pointer keys became junk properties. Moved or
resized occurrences also kept the time and length the rule gave them,
and were placed in the range by their original date.
- apply overrides as patches: pointer paths decoded per RFC 6901, null
deletes, shorter paths first, a whole participants map completed
from the series
- time an occurrence from its patched start, time zone and duration,
and range-check a moved occurrence by where it is now
- fall back to the JSCalendar duration when the master's utcStart or
utcEnd don't parse (0.16.23 returned the year 32548 for a counted
series), and never keep the master's instants on another occurrence
- merge a change into an existing override without keys that are
prefixes of one another, and match an occurrence listed twice
against the members its override patches
The device expands series on 0.16.6 to 0.16.19, after a created series
and when a range can't be expanded on the server. 0.16 returns whole
override values, which apply the same way.
Stalwart 1.0 refuses series properties such as the organizer on a
single occurrence unless they repeat the series' value, and refuses
them inside a recurrence override on the base event. The event editor
wrote the user's first address as organizerCalendarAddress on every
save of an event with attendees, so editing one occurrence failed with
"This property cannot be modified on a single occurrence." whenever the
series was organized by someone else, under another alias or in
another letter case. Removing every attendee from one occurrence failed
the same way.
- leave series keys and per-event metadata out of changes to one
occurrence, as Stalwart's validate.rs lists them, and pointers at a
participant's calendarAddress
- write overrides on the base event as before (0.16.19 needs the
organizer there) and, when the server refuses series keys inside
them, repeat the write once without them and remember that
- answer one occurrence through its synthetic id without the organizer
- let the editor keep the organizer of an event: an invitation the
user received is no longer rewritten with the user as organizer, and
organizerCalendarAddress is only written for an event that has none
- keep the SetError type on a refused event update
0.16.20 to 0.16.23 dropped these keys on synthetic ids anyway; the
base-event override payload for older servers is unchanged.
Stalwart 1.0 refuses an expanded CalendarEvent/query that spans more
than the account's maxExpandedQueryDuration (P52W1D by default) with
expandDurationTooLarge, and one that expands to more than 3000
occurrences with cannotCalculateOccurrences. The scrolling views load
windows of up to about 460 days, and refresh() reloads the whole
window on every calendar push, after series edits and on pull to
refresh, so the calendar showed a raw error over stale events from
then on. The fallback to expanding on the device only knew 0.16's
invalidArguments.
Expanded range queries are now cut into chunks a day shorter than the
advertised limit, with overlapping borders, sent together as far as
maxCallsInRequest allows, and their ids de-duplicated. A chunk the
server still refuses is halved; if a week still expands to too many
occurrences, the whole range is expanded on the device as before.
The expanded query keeps a filter of the two bounds only: 1.0 refuses
operators there, so filtering several calendars would take a query per
calendar, and the store loads every calendar anyway. 0.16 advertises
the same limit without enforcing it and returns the same events.
Stalwart 1.0 returns every photo, logo and sound in a contact card as a
blobId without a uri, including photos saved as data: URIs. The app read
only `uri`, so sender avatars, the contact list, the detail screen, the
form preview and vCard export showed nothing. Worse, the form rebuilt
`media` from the card on every save and sent it without the photo, or
as `null`, so editing a phone number deleted the photo on the server.
- `ContactMedia` carries `blobId`, `label`, `pref` and `contexts`, and
`uri` is optional. The photo shown is the photo entry with the lowest
`pref`, as a URI (0.16) or a blob (1.0).
- Blob photos load from the download URL with the Authorization
header, like file thumbnails. At most half of the server's
`maxConcurrentRequests` load at once, a failed one is tried once more
and then left alone for two minutes, and one that loaded skips the
queue from then on.
- The form leaves `media` out of the save unless the photo changed. A
changed photo is built on the card as the server has it now, so a
stale blobId can't make 1.0 refuse the update, and a new photo keeps
the key, `pref`, `label` and `contexts` of the one it replaces.
- A new photo is uploaded into the user's own account and sent as a
blobId where the blob capability lists ContactCard (1.0). Otherwise
it goes as a data: URI, and a refusal like 0.16's "blobIds in media
is not supported." retries once with the data: URI and is remembered
for the account.
- vCard export fetches blob photos with batched Blob/get requests and
writes them inline. It also writes photos whose data: URI names no
media type, which it dropped before. Duplicate and moving a card to
another account send the bytes instead of the source's blobIds.
- An update never clears `id`, `uid` or `addressBookIds`, and vCard
export writes MEMBER values that are already URIs as they are instead
of `urn:uuid:urn:uuid:…`.
The form's state and patch building moved to `src/lib/contact-form.ts`
so the save path can be tested.
Stalwart 1.0 counts every EmailSubmission record of an account against
`maxSubmissions` (500 by default), whatever its status or age, and its
own purge of old records never runs. The app never removed a record, so
after about 500 sends from an account every send, quick reply, read
receipt and "Send now" failed with Stalwart's raw "There are too many
email submissions, please delete some" error. This ports the webmail
fix (48382170).
The app now removes the records that can no longer be cancelled.
Destroying a record never stops its delivery, but a pending one could
no longer be cancelled, so a record EmailSubmission/get reports as
pending is never touched:
- an immediate send (no undo delay, no schedule) and a read receipt
destroy their own record right after the send, riding along with the
old-draft cleanup when there is one; only where `urn:stalwart:jmap`
is advertised, and never decided by the created undoStatus, which
0.16.10-0.16.16 report as final for held sends
- a throttled background prune (`src/api/submissions.ts`) destroys
canceled records and final records whose sendAt has passed, going by
EmailSubmission/get, in every submission account; it runs at app
start, on return to the foreground and after sends and undo
- a confirmed cancel (undo, cancel, reschedule, "Send now") destroys
the canceled record in a request of its own
- an `overQuota` on a submission create prunes at once, on Stalwart
walks `EmailSubmission/changes` from "n" for records older than the
query's three-day window, retries once, and otherwise throws
`SubmissionQuotaError`, shown as a translated explanation in the
composer, quick reply, read receipts, the undo bar and the scheduled
list; the copy the failed attempt made is removed as before
0.16 never enforces the limit, and destroy never cancels on any
version, so the same cleanup runs there unchanged.
The undo bar now also says when an undo or "Send now" failed instead
of only logging it. The scheduled list reads every page of
EmailSubmission/query with calculateTotal; it asked for 200 records,
oldest first, so the mail scheduled furthest ahead dropped off once the
three-day window held more than that.
`SubmissionQuotaError` is added to `jmap-result.ts` as a subclass of
`JMAPMethodError`.
Stalwart 1.0 refuses a CalendarEvent/get that asks for
recurrenceOverrides together with utcStart or utcEnd, and every event
read of the app did. The calendar showed a raw error over stale events,
and tasks, reminders, "all events" edits, contact activity, the
invitation banner's UID look-up and iCal subscriptions stopped working.
Subscriptions and imports also lost their UID check, so every sync tried
to create the whole feed again and ended in refusals.
- read a stored event's overrides with a second get in the same
request; server-expanded occurrences get them from their base event
as before
- read only ids, UIDs and calendars for the import and subscription
UID look-ups
- fail a read when either get answers with an error, so the calendar
keeps the events it has and shows the error
- keep the JMAP error type on calendar method errors (JMAPMethodError,
plus a shared jmapErrorType helper in jmap-result)
0.16 accepts the split reads and returns the same events.
A virtual location in an app scheme (msteams:, zoommtg:) is kept as a last
resort instead of hiding the meeting row; script schemes never are. A
location only joins the meeting when the whole label is a service name, so
"Salle Teams" still opens the maps. Recognise Zoom's /wc/ join path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Invitations - Teams above all - leave virtualLocations empty: LOCATION reads
"Microsoft Teams Meeting" and the join URL sits in the description between
help and dial-in links, so the sheet showed no meeting button. Find the
join URL of known conferencing services (Teams, Zoom, Meet, Webex, Jitsi,
Whereby, GoTo) in the location or description, unwrapping Outlook Safe
Links, and offer it as the meeting link. Nothing is written to the event.
The location was plain text. Tapping it now opens it: a URL opens as a
link, a location that only names the service joins the meeting, anything
else is searched in the maps app (geo: intent on Android, Google Maps
universal link elsewhere). Long-press copies it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Allow webmail-configured OAuth providers on separate hosts without trusting callback-supplied endpoints or broadly allowing sibling domains. Bind the client, JMAP server, issuer and token endpoint to HTTPS metadata from the selected webmail for both handoff and pairing.
2026-09-18 04:01:58 -07:00
1497 changed files with 380051 additions and 104731 deletions
# The notification service extension (plugins/ios-push) is its
# own App ID and needs its own App Store profile.
echo "::error::IOS_NSE_PROVISIONING_PROFILE_BASE64 is not set: the notification extension (org.bulwarkmail.mobile.NotificationService) can't be signed."
exit 1
fi
echo "signed=true" >> "$GITHUB_OUTPUT"
else
echo "signed=false" >> "$GITHUB_OUTPUT"
echo "::notice::Signing secrets are not configured - compiling without signing. No IPA will be produced."
| #45 error when re-registering to relay | [08](docs/parity/08-settings-push-i18n-ui.md) push section |
| #34 Native contacts/calendar sync adapters | feature request, out of parity scope; noted in [06](docs/parity/06-contacts.md) |
| #34 Native contacts/calendar sync adapters | done on Android: Settings → Contacts / Calendar → "Sync to this device" syncs both ways with the device's Contacts and Calendar apps ([design](docs/device-sync.md)); iOS has no API for it |
| #5 Sort ascending/descending | in progress in the working tree; ordering presets in [02](docs/parity/02-mail-list-folders.md) |
| #3 TLS client auth | Android-only today; gaps in [01](docs/parity/01-auth-accounts.md) and [09](docs/parity/09-jmap-core-sync-security.md) |
| #1 Shared settings between webmail and native | [08](docs/parity/08-settings-push-i18n-ui.md) — webmail sync is server-side/cookie-bound; a JMAP-blob design is sketched there |
React Native (Expo SDK 54) client for [Bulwark Webmail](https://github.com/bulwarkmail/webmail) - a JMAP-based mail, calendar, and contacts app.
## Try it
- **iPhone and iPad:** join the beta on TestFlight: https://testflight.apple.com/join/rumcjNSp. A new release shows up there once Apple has reviewed it, usually within a day.
- **Android:** download the APK from the [latest release](https://github.com/bulwarkmail/native/releases/latest).
- Push notifications via the Bulwark relay - FCM by default, or [UnifiedPush](https://unifiedpush.org) (e.g. ntfy) for devices without Google Play services
- Android: contacts and calendars sync both ways with the phone's Contacts and Calendar apps (Settings → Contacts / Calendar → Sync to this device)
- In-app sideload updates from GitHub Releases
## What's missing or rough
- iOS builds, but push notifications and client certificates are Android-only so far
- iOS builds, but push notifications, client certificates and syncing with the phone's contacts and calendars are Android-only so far
- S/MIME, plugins, themes - UI stubs only (filters & rules, the vacation responder and file storage are real implementations)
- Calendar editing is partial; contacts editing is basic
- No Play Store distribution yet (sideload APK from Releases); iOS ships via TestFlight
- No Play Store or App Store listing yet: Android is a sideloaded APK, iOS a TestFlight beta (see [Try it](#try-it))
## Run locally
@@ -45,4 +51,4 @@ For iOS builds and TestFlight distribution see [docs/ios-release.md](docs/ios-re
## License
AGPL-3.0-only. See [LICENSE](LICENSE).
AGPL-3.0-only, with an additional permission to distribute the app through app stores such as the Apple App Store and Google Play. See [LICENSE](LICENSE). Contributions are accepted under the same terms. The permission is provisional until every earlier contributor has agreed to it ([consent request](https://github.com/orgs/bulwarkmail/discussions/1113)); contributions made since 30 September 2026 are already covered.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Linus Rath
lucletoffeandClaude Opus 5.5
arnauda-gh
Michael Brusegard