Compare commits

...
255 Commits
Author SHA1 Message Date
Linus Rath e0f18713aa chore: update version number to 0.1.81 2026-10-16 19:27:04 +02:00
Linus Rath 56d826c508 test: cover how the PDF preview reports protected and unreadable files 2026-10-16 19:24:19 +02:00
Linus Rath e5492b89c8 feat: show PDFs in the app on Android 2026-10-16 19:23:17 +02:00
Linus Rath b50436b1ba chore: update version number to 0.1.80 2026-10-16 19:12:43 +02:00
Linus Rath 11e74042e2 fix: hand files with a generic type to other apps by the type their name implies 2026-10-16 19:11:50 +02:00
Linus Rath 95580b84ea feat: play video and audio, and show SVG, HTML and Markdown files in the preview 2026-10-16 19:11:24 +02:00
Linus Rath 9a15334bef chore: update version number to 0.1.79 2026-10-16 18:49:43 +02:00
Linus Rath 3aca2d80c0 fix: preview .eml attachments that carry attachments of their own 2026-10-16 18:48:26 +02:00
Linus Rath 7c534301ee refactor: open files from mail, the list, Files and compose through one preview 2026-10-16 18:48:26 +02:00
Linus Rath 14dc5ff2d4 Merge pull request #76 from lucletoffe/feat/calendar-sort-order
feat(calendar): order calendars by sortOrder and move them from the drawer
2026-10-15 23:37:37 +02:00
Linus Rath 14ba28c79c Merge pull request #74 from lucletoffe/fix/compose-visual-viewport
fix(composer): stop the body's first lines hiding under To/Cc/Subject on Android
2026-10-15 23:23:40 +02:00
Linus Rath 598adc154d Merge pull request #72 from lucletoffe/fix/draft-part-blobids
fix(drafts): keep a re-opened draft's attachments on live blobs across saves and sends
2026-10-15 23:23:26 +02:00
lucletoffeandClaude Opus 5.5 3c0af52c4e feat(calendar): order calendars by sortOrder and move them from the drawer
Calendars were listed in the order the server returns them (creation
order on Stalwart) and sortOrder was never read. Sort them like the
webmail now does and as JMAP Calendars asks: lower sortOrder first,
equal sortOrder by name (case- and accent-insensitive, numbers by
value), each account's calendars kept together. The store keeps the list
sorted, so the drawer, the event editor and the other pickers agree.

The drawer's long-press panel gains Move up / Move down for the user's
own calendars. A move renumbers that account's calendars 1, 2, 3... and
writes sortOrder only where it changes; the panel stays open so a
calendar can be moved several places.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 22:07:33 +02:00
Linus Rath 7108431dbb chore: update version number to 0.1.78 2026-10-15 16:58:08 +02:00
Linus Rath 744b17908d fix: open a settings page at its top instead of the list's scroll position 2026-10-15 15:36:59 +02:00
Linus Rath c7edae9ed3 fix: move a setting's control under its label when a long label leaves no room 2026-10-15 15:29:13 +02:00
Linus Rath c07b39a070 fix: keep device sync free of expo by registering device locales from i18n #75 2026-10-15 15:19:18 +02:00
Linus Rath 7e5e296271 fix: use the app switch for the sender override in compose 2026-10-15 15:16:32 +02:00
Linus Rath ad6dff79ba fix: draw contact form fields like the webmail and quiet the letter headers in the contact list 2026-10-15 15:16:32 +02:00
Linus Rath a4ca72fb3d fix: draw event and task fields like the webmail and use the app switch in the event form 2026-10-15 15:16:32 +02:00
Linus Rath 107eb807e3 chore: update version number to 0.1.77 2026-10-15 15:15:29 +02:00
Linus Rath e731357b89 fix: format dates in the device region for the app language #75 2026-10-15 15:15:19 +02:00
Linus Rath 9cf1d3e849 fix: draw the text field in dialogs like the other inputs 2026-10-15 15:04:01 +02:00
Linus Rath d3dcc4e830 fix: show folder roles, filter origins and app modes as outline badges 2026-10-15 15:04:01 +02:00
Linus Rath e0d2b190d3 fix: give the Files header the same title size as Mail and Contacts 2026-10-15 15:04:00 +02:00
Linus Rath b048eaca03 fix: tint unread mail and draw the mail search, subject, quick reply and thread buttons like the webmail 2026-10-15 15:04:00 +02:00
Linus Rath 6f515e15f7 feat: rework settings into the picked list, About and Storage & backup pages
The settings list now has the account card on top, title-case group
labels and 56 dp rows that show each pane's current value. About and
"About & Data" were one placeholder page with a letter logo, a fixed
name and a store link that pointed at GitHub releases.

- About: the real logo, the app name from app.config.js, the update
  state, a source link that a build can set with SOURCE_CODE_URL, and
  rows for the server, update checks (Android), open-source licences,
  privacy and a prefilled GitHub issue. iPhone updates link TestFlight.
- Storage & backup: usage card, offline mail, settings export/import
  and reset, and a Troubleshooting page for refresh and debug logging.
- Sender favicons moved to Appearance, like the webmail.
- The S/MIME and Plugins stub pages are gone; both are webmail plugins.

The licence list is generated by `scripts/generate-licences.mjs`.
2026-10-15 14:49:37 +02:00
Linus Rath 0a176ab51c feat: draw buttons, switches, selects, inputs and badges like the webmail 2026-10-15 14:49:37 +02:00
Linus Rath 75299244cf refactor: draw the webmail's Tabler icons instead of Lucide 2026-10-15 14:13:02 +02:00
Linus Rath 6aaa479875 chore: update version number to 0.1.76 2026-10-15 08:32:45 +02:00
Linus Rath 124c0258ad docs: add the final report to the native ui and ux audit 2026-10-15 04:19:41 +02:00
Linus Rath ccffd6b10d docs: tick the latest audit findings and note the deferred ones 2026-10-15 04:09:03 +02:00
Linus Rath 0d96d8d044 fix: fit every selection action and the count in the mail list header on a phone 2026-10-15 04:08:33 +02:00
Linus Rath 95c31631fb fix: show event titles instead of bare start times in a narrow month view 2026-10-15 04:05:55 +02:00
Linus Rath cf7275c9c3 fix: write calendar dates in the order of the app language 2026-10-15 04:05:55 +02:00
Linus Rath e452f72a8a fix: keep the all-day label in the day and week grid on one line 2026-10-15 04:05:49 +02:00
Linus Rath 776447dd41 fix: give sheet close buttons, chips and small text buttons a full-size touch target 2026-10-15 04:05:40 +02:00
Linus Rath 52eb18dc33 fix: show unified inbox rows with the list's text size, density and preview settings 2026-10-15 03:37:58 +02:00
Linus Rath 33dc3cd0dc fix: move through the contact name fields with the return key 2026-10-15 03:37:58 +02:00
Linus Rath 72b4f4abac fix: keep the edits when the recurring event question is cancelled 2026-10-15 03:37:58 +02:00
Linus Rath de1b955bfd docs: tick more fixed findings in the ux audit 2026-10-15 03:30:37 +02:00
Linus Rath c8434b0610 fix: make small buttons easier to hit 2026-10-15 03:30:00 +02:00
Linus Rath e19229a5c6 fix: keep screens clear of the camera cutout and side navigation bar in landscape 2026-10-15 03:29:59 +02:00
Linus Rath d7cee7ca78 fix: label the star button Unstar on a starred message 2026-10-15 03:29:27 +02:00
Linus Rath b59d00f0b7 fix: let the tag sheet scroll 2026-10-15 03:29:26 +02:00
Linus Rath 975ed08c82 fix: keep every thread toolbar button on screen, icon-only in landscape 2026-10-15 03:29:26 +02:00
Linus Rath 76170648f5 fix: open a long conversation at its first unread message 2026-10-15 03:29:25 +02:00
Linus Rath 509583e729 fix: widen the composer's field labels with the font size 2026-10-15 03:21:18 +02:00
Linus Rath 7b93265b4f fix: stop text fields cutting off their text at a larger font size 2026-10-15 03:21:18 +02:00
Linus Rath b5c4d62bde fix: put the mail search in the header row on a phone in landscape 2026-10-15 03:20:48 +02:00
Linus Rath c6cb44db61 fix: apply the text filters once typing pauses instead of querying per keystroke 2026-10-15 03:20:48 +02:00
Linus Rath 0db4bf00aa fix: say when a swipe, bulk action or import fails instead of failing silently 2026-10-15 03:20:48 +02:00
Linus Rath e94b782f8e fix: keep mail lines a readable length on wide screens 2026-10-15 03:18:58 +02:00
Linus Rath ae1ef5ba5e fix: move from username to password to code with the return key when signing in 2026-10-15 03:18:24 +02:00
Linus Rath f891bf3abb fix: announce buttons as buttons with their disabled and busy state 2026-10-15 03:17:08 +02:00
Linus Rath af55e0acef fix: name every format button for screen readers 2026-10-15 03:17:07 +02:00
Linus Rath 1ebc5c66f7 fix: ask about unsaved contact changes on every way out of the form 2026-10-15 03:16:46 +02:00
Linus Rath 0b3710265b fix: keep the address book chips whole in landscape 2026-10-15 03:16:46 +02:00
Linus Rath 56e1593646 fix: move from recipients to subject to body with the return key 2026-10-15 03:00:06 +02:00
Linus Rath e7857ef806 fix: close an untouched reply without asking to save it 2026-10-15 03:00:05 +02:00
Linus Rath b3290c3202 fix: use the webmail's dark theme colours: white primary on neutral greys 2026-10-15 02:59:45 +02:00
Linus Rath f231d1c5c6 docs: tick the fixed findings in the ux audit 2026-10-15 02:54:38 +02:00
Linus Rath 984c359bb4 fix: end the unified inbox and scheduled lists above the gesture bar 2026-10-15 02:53:07 +02:00
Linus Rath 50ab7145c4 fix: keep the compose button clear of the undo bar 2026-10-15 02:53:07 +02:00
Linus Rath 8f68d24bdb fix: make the filter warnings readable on the light theme 2026-10-15 02:51:56 +02:00
Linus Rath f22e851c82 fix: give the vacation subject the full width instead of squeezing its label 2026-10-15 02:51:55 +02:00
Linus Rath d4982f6932 fix: ask before removing someone's access to a shared file 2026-10-15 02:51:55 +02:00
Linus Rath 275d674577 fix: ask before deleting a tag 2026-10-15 02:51:54 +02:00
Linus Rath 0460da51e5 fix: ask before deleting an event, a task or a calendar feed 2026-10-15 02:51:54 +02:00
Linus Rath 90170ea0e7 fix: show one signature separator when the signature brings its own 2026-10-15 02:36:11 +02:00
Linus Rath cbc41c963b fix: fit the files header under the status bar and clear of landscape insets 2026-10-15 02:36:00 +02:00
Linus Rath 63eefd7700 fix: show file dates in the app's language and time format 2026-10-15 02:36:00 +02:00
Linus Rath d97571cf0f fix: open folder settings from the drawer's folders icon 2026-10-15 02:35:31 +02:00
Linus Rath 8403843e76 fix: ask before signing out 2026-10-15 02:35:30 +02:00
Linus Rath 756ceaa436 fix: leave contact selection and search with back before leaving the tab 2026-10-15 02:35:21 +02:00
Linus Rath 99de74de93 fix: show the logo in the mail header on the light theme 2026-10-15 02:35:21 +02:00
Linus Rath 46d30169b2 fix: clear the mail search with back instead of leaving the app 2026-10-15 02:35:21 +02:00
Linus Rath 512a28c494 fix: show the app in the chosen language right after a cold start 2026-10-15 02:35:11 +02:00
Linus Rath 29c172f2af fix: use the theme's tag colours in the tag sheet 2026-10-15 02:35:11 +02:00
Linus Rath 9e463d3a66 fix: show the whole section labels on the contact page 2026-10-15 02:35:10 +02:00
Linus Rath ab712f11ba fix: say why an event couldn't be saved instead of failing silently 2026-10-15 02:35:10 +02:00
Linus Rath 053fde8d56 fix: keep the text typed in quick reply when opening the full composer 2026-10-15 02:35:09 +02:00
Linus Rath 250c0a9c26 feat: centre the sign-in steps on tablets 2026-10-15 02:35:09 +02:00
Linus Rath d62805f805 docs: tick the rotation and tablet layout findings 2026-10-15 02:11:08 +02:00
Linus Rath b9e6ba417d fix: keep the compose button and list ends above the tablet taskbar 2026-10-15 02:11:08 +02:00
Linus Rath 0087e4c8ff feat: open the composer as a centred panel on tablets 2026-10-15 02:08:18 +02:00
Linus Rath 2470e99dc8 feat: fit the files grid to the width and preview files beside the list on tablets 2026-10-15 02:07:49 +02:00
Linus Rath 8ab82cdeb7 feat: show settings sections and the open page side by side on tablets 2026-10-15 02:07:21 +02:00
Linus Rath 64a62b6306 feat: show contacts as list and details side by side on tablets 2026-10-15 02:06:50 +02:00
Linus Rath f4ff07da7a feat: centre sheets and float toasts bottom right on tablets 2026-10-15 02:06:19 +02:00
Linus Rath 2288d28e46 feat: show five or seven days in the multi-day calendar view on wider screens 2026-10-15 02:05:31 +02:00
Linus Rath 295e292e47 fix: stop the contact and group screens crashing after a delete 2026-10-15 01:49:49 +02:00
Linus Rath 802474d3f2 feat: show mail as folders, list and message side by side on tablets 2026-10-15 01:45:57 +02:00
Linus Rath bfa02f1177 feat: let the app rotate and work in landscape
The app was locked to portrait in app.config.js and the hand-kept
AndroidManifest.xml. Android 16 ignores that lock on screens 600 dp and
wider, so tablets and foldables rotated into layouts that were never
tested. This removes the lock in both places, keeps React state across
rotation and split-screen resizes (`smallestScreenSize` joins
`configChanges`), and fixes what broke in landscape:

- the thread pager sizes its pages from its own width and snaps back to
  the open message after a rotation instead of showing half a neighbour
- compose and the thread view share one keyboard tracker; a rotation
  with the keyboard open closes it, since Android sends no new keyboard
  frame and the old portrait one blanked the whole landscape composer
- in a short window with the keyboard up the composer hides the format
  bar and tightens its header so a few lines of text stay visible
- inputs with suggestions or context around them opt out of Android's
  full-screen landscape keyboard
- month views fit whole rows to the real height, keep the month across a
  remount, and the month dropdown re-measures and caps its height
- the week grid's header follows the remounted list after a rotation
- the event sheet's actions stay on screen in landscape
- every RN Modal allows all orientations, or iOS forces portrait
2026-10-15 01:45:23 +02:00
Linus Rath 4b29cdfd3c feat: add window size classes for tablet and landscape layouts 2026-10-15 01:44:45 +02:00
Linus Rath 2a47913d70 docs: add native ui and ux audit 2026-10-15 01:19:00 +02:00
lucletoffeandClaude Opus 5.5 8ce53816af fix(composer): stop the body's first lines hiding under To/Cc/Subject on Android
Android's WebView shrinks the editor page's visual viewport to the part of
the view that is on screen and pans it to reveal the caret while typing.
The editor is an auto-height WebView inside the composer's ScrollView, so
nothing ever resets that pan: scrolled back to the top, the body is still
drawn shifted up by the pan offset and its first lines sit hidden under the
header fields until a scroll to the very bottom clamps it away.

The page now undoes every pan as it happens (scrolling the page top into
view, since window.scrollTo only moves the layout viewport) and reports the
offset as a `viewportShift` message; the composer scrolls its ScrollView by
the same amount so the caret stays visible. Android only; iOS and the other
RichTextEditor hosts keep their behaviour (hosts without the callback still
get the reset).

Reproduced and verified on a Pixel 4 (Android 13, WebView 153): before,
visualViewport.offsetTop stayed at ~18 px at the top of the composer and
cut the first line; after, it stays 0 and the caret still follows typing
above the keyboard.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:49:57 +02:00
lucletoffeandClaude Opus 5.5 767d4b9f9f fix: keep a re-opened draft's attachments on live blobs across saves and sends
A draft's attachments and inline images are MIME parts of that draft, and
their blobIds die with the version that holds them. Each save creates the
next version and then destroys the previous one (#849), but the composer kept
sending the blobIds it was seeded with. The first save worked, every later
save failed with "blobId ... does not exist on this server", and so did the
send. This is the second half of webmail #849, which the webmail fixed in
5d2b937a but which was never ported here.

- saveDraftVersion() reads the new version's attachment parts back in the
  same request (Email/get on the creation id "#draft"; Stalwart 0.16 does not
  resolve a result reference into Email/set "created"). createDraft() keeps
  its signature on top of it.
- The composer maps each held blobId to the matching part of the latest
  version (inline by Content-ID, the rest by name and size) and sends that.
  Uploads are re-pointed too, since an unreferenced upload expires. The map
  lives in a ref, so the dirty snapshot does not change on every save.
- Opening a draft with inline images no longer looks like an edit. The
  hydration swapped the editor body for its normalised DOM HTML but compared
  it with the raw server HTML, so the autosave replaced the draft about two
  seconds after it opened. That destroyed the opened version, and a plain
  "open, then Send" failed with the same blobNotFound.

Tests: unit tests for the matching and rebasing, a fake-server test that
reproduces the failure without the rebase, and an opt-in live Stalwart test
(verified against Stalwart 0.16.22).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:47:21 +02:00
Linus Rath e19885dc72 chore: update version number to 0.1.75 2026-10-13 18:52:35 +02:00
Linus Rath 4816f0e08b chore: update version number to 0.1.74 2026-10-13 18:07:46 +02:00
Linus Rath ef1d4bc387 fix: stop the app crashing when a notification opens a mail 2026-10-13 18:07:46 +02:00
Linus Rath 61690603a2 chore: update version number to 0.1.73 2026-10-10 23:54:40 +02:00
Linus Rath 765ff26aa5 Merge remote-tracking branch 'origin/main' 2026-10-10 23:20:25 +02:00
Linus Rath b832ac0c49 feat(calendar): tap an empty time slot to start a new event
In the day, 3 days and week grids an event could only be started with a
long press, which few people find. Tapping an empty slot now marks it
with a "New event" block, and tapping that block opens the editor at
that hour; a second tap elsewhere moves the block. Long press still
opens the editor straight away. Tapping a day's header in the week or
3 days grid opens that day.

The editor also started events from a slot half an hour late, because
it rounded a time already on the hour up to the next half hour.
2026-10-10 22:59:35 +02:00
Linus Rath f53cee7ec9 feat(calendar): lay out the agenda as a schedule with dates on the left
The agenda showed each day as a grey header bar above its events. It now
reads like a schedule: the weekday and date sit in a column on the left
of the day's first event (today in a filled circle), a heading marks
where each month starts and a quiet line names each week's range. An
empty today says "Nothing planned. Tap to create." and tapping it opens
a new event. In the drawer the view is now called Schedule.

Events that fill a whole day are listed as all day instead of
"00:00 – 00:00".
2026-10-10 22:53:40 +02:00
Linus Rath 21d469f7e8 feat(calendar): show the month as a full-screen grid of events
The month view used to be a small grid of dots with the selected day's
events listed underneath, so you only ever saw one day's events at a
time. It now fills the screen: each day is a cell with its events as
solid bars in the calendar colour and "+N" for the ones that don't fit.
Tapping a day opens it in the day view, and long-pressing still creates
an event there. Events that fill the whole day no longer show 00:00.

The dotted month stays for the month that drops down under the title.
2026-10-10 22:47:13 +02:00
Linus Rath c1e0cb36c9 feat(calendar): move views into the drawer and add a create button
The calendar header was crowded: four icon toggles, a separate row of
arrows and a Today button, and a small create button squeezed in next
to them. Navigation now follows Google Calendar's layout while keeping
the app's own look:

- the header shows the month; tapping it drops down a month you can
  swipe through and pick a day from
- a calendar icon with today's date jumps to today
- Schedule, Day, 3 days, Week and Month are listed at the top of the
  drawer, styled like the mail drawer's folders, with Refresh below
- a round create button sits in the corner like the mail compose
  button; with tasks on it offers Event or Task
- a new 3 days view shows three columns of the time grid

The arrows stay in the header when free scrolling is off, since the
views can't be swiped then.
2026-10-10 22:41:46 +02:00
Linus Rath d8879617ed fix: never destroy mail from the notification Delete action #68
With no Trash mailbox, or when Mailbox/get came back as an error, the
Delete action fell through to destroying the email. A shared account
without its own Trash lost mail permanently on one tap.

- only move to Trash; do nothing when there is no Trash or the lookup
  failed
- check that Email/set actually updated the email
- show "(No Subject)" in the collapsed text instead of a blank line
- require unlock for Delete (Android 12+)
2026-10-10 19:20:53 +02:00
Linus Rath 7b62755aee Merge pull request #68 from arnauda-gh/feat/notification-preview-and-actions
feat(notifications): add rich preview snippet and quick actions (mark read, delete, reply)
2026-10-10 19:20:46 +02:00
Linus Rath 96e1cc1c3f fix: open a list mail without mounting every pane before it #67
The tap fix makes a notification open a single page, so the viewer no
longer needs to render up to the opened index. That setting mounted
about N mail panes on every list open, on iOS too. Also drop the
onScrollToIndexFailed handler, which FlatList never calls when
getItemLayout is set.
2026-10-10 19:20:46 +02:00
Linus Rath ec137e931b Merge pull request #67 from arnauda-gh/fix/notification-open-target-email
fix(notifications): ensure notification tap always opens target email
2026-10-10 19:20:40 +02:00
Linus Rath cf78b07939 Merge pull request #69 from arnauda-gh/fix/quick-reply-keyboard-avoidance
fix(viewer): adapt screen and auto-scroll quick reply box when keyboard appears
2026-10-10 19:20:39 +02:00
arnauda-gh 5237d9004d fix(viewer): adapt screen and auto-scroll quick reply box when keyboard appears 2026-10-09 15:00:51 +02:00
arnauda-gh be15af5d4c fix(notifications): execute background actions directly via HeadlessJs without service 2026-10-09 12:41:53 +02:00
arnauda-gh 90fe43882a feat(notifications): add email preview snippet and quick actions (mark read, delete, reply) 2026-10-09 09:16:53 +02:00
arnauda-gh 465eeffd1e fix(viewer): use info.index * windowWidth for onScrollToIndexFailed offset 2026-10-09 09:05:27 +02:00
arnauda-gh 200404483c fix(notifications): ensure notification tap always opens target email 2026-10-09 09:01:28 +02:00
Linus Rath 28f6787279 fix: confirm a copied event location on iOS too #59 2026-10-06 23:32:04 +02:00
Linus Rath 89f0524ad4 fix: only offer meeting links the app can open #59 2026-10-06 23:30:57 +02:00
Linus Rath 172110f7b0 Merge pull request #59 from lucletoffe/fix/calendar-location-links
fix(calendar): join the meeting and open the location from the event sheet
2026-10-06 23:30:07 +02:00
Linus Rath aa612e6f0e Merge pull request #55 from michaelbrusegard/fix/external-idp-handoff
fix(auth): verify external identity providers during handoff
2026-10-06 23:29:29 +02:00
Linus Rath 76180b332d ci: put every iOS release on the public TestFlight link
Uploads only reached internal testers. The public link's "external" group
still served 0.1.66, because nothing added later builds to it or
submitted them for Beta App Review.

A new `public-testflight` job runs after the upload on a Linux runner.
It waits for App Store Connect to process the build, adds it to the
external group, sets "What to Test" to the GitHub release and submits
it for review (`scripts/testflight-public.mjs`, safe to re-run). The
README links the TestFlight beta and the latest APK.
2026-10-02 16:33:07 +02:00
Linus Rath f5c491f6d7 chore: update version number to 0.1.72 2026-10-02 15:02:12 +02:00
Linus Rath 23af324d79 fix: verify app updates without a second slow progress bar
Since releases ship an `.apk.sha256` file, every in-app update checks
the downloaded APK against it. The check read the 40 MB file in base64
chunks and hashed it in JS, which took longer than the download, and it
reset the progress bar to 0 under the "Downloading" label, so the update
looked like it downloaded twice, the second time slowly.

The file is now read and hashed natively through expo-crypto (the JS
hasher remains as a fallback), and the bar stays full under a
"Verifying…" label while the check runs.
2026-10-02 15:02:10 +02:00
Linus Rath 761c35918f chore: update version number to 0.1.71 2026-10-02 14:09:36 +02:00
Linus Rath 86e33b9afe fix: sync trusted senders to the address book by default
The "Sync with address book" toggle under Content & Senders was off until
the user turned it on, so senders trusted on the phone stayed in the
local allow-list and never reached the "Trusted Senders" address book the
other devices read. A setting the user never chose now counts as on;
only an explicit opt-out, or a server without contacts, turns it off.
2026-10-02 14:09:34 +02:00
Linus Rath 2a1265a9ac chore: update version number to 0.1.70 2026-10-02 13:50:49 +02:00
Linus Rath aba5c70b3e docs: record the widget checks after the fixes on Stalwart 0.16 and 1.0 #52
Adds the phase 2 and 3 results to the verification checklist: the
corrections to phase 1 (two link failures were the test rig), every
action and link re-run on both servers, which commit fixes which cause,
the layout review and what is still open.
2026-10-02 13:45:57 +02:00
Linus Rath ccd8030af0 fix: round the widget corners less #52
Widgets had 22dp corners, rounder than the launcher's own widgets and
the app's cards. They now use 16dp, the Pixel launcher's widget radius,
on the surface, its frame and the picker previews.
2026-10-02 13:35:47 +02:00
Linus Rath ad10d4a8aa feat: show contact photos and sender logos in the widget avatars #52
The widgets drew every sender as initials, while the app and the
webmail show the contact's photo or the sender's logo first. The
widget avatars now follow the same order: an embedded contact photo,
then the logo of a non-personal sender domain when sender favicons are
on, then the initials.

A widget draws from the stored snapshot and cannot load anything while
it draws, so the pictures are fetched when the snapshot is built and
kept in it as small data URIs. Logos have a cache of their own (a week
for a logo, a day for a domain without one) that survives being
offline, and pictures over 40 KB or in a format Android cannot decode
stay out.
2026-10-02 13:34:30 +02:00
Linus Rath 9de8c56743 fix: work out avatar initials the way the webmail does, everywhere in the app
The app had six ways to turn a name into avatar initials, and none of
them was the webmail's. The webmail's Avatar skips punctuation and
symbols in front of each name part ("Sofia" Russo is SR, not "S), takes
two letters of a single name and the first letter of the address
without one. Senders, accounts (sidebar and sign-in), contacts, compose
suggestions and the home-screen widgets all use that now.
2026-10-02 13:18:29 +02:00
Linus Rath 5fcf9367a6 fix: show the widgets' current layouts in the widget picker #52
The picker previews still showed the layouts from before the fixes in
this series (Month's arrows, the Vacation switch, the old task rows and
triage header). Redrawn from the sample data at each widget's default
size, light theme, with the rounded corners and frame the launcher
draws.
2026-10-02 13:11:29 +02:00
Linus Rath bba0274a40 feat: translate the home-screen widgets into German #52 2026-10-02 13:05:14 +02:00
Linus Rath d179e1de82 fix: keep "Tap to try again" visible on a widget notice about a long task #52 2026-10-02 13:05:07 +02:00
Linus Rath 63f2e07254 feat: show more of the message on taller triage and latest-message widgets #52
The triage card kept its preview to two lines however tall it was
resized, leaving most of a large card empty, and Latest message showed
no preview at all. Both now use the room they have for the start of
the message.
2026-10-02 13:04:38 +02:00
Linus Rath 80b57161ca fix: say what an empty folder list or a finished task is on the widgets #52
Folder counts showed only its "Folders" heading when there was no
folder to list, which looks broken; it now says there are no folders to
show. A finished task past its due date still read "Overdue" on the
Tasks widget; it shows its date instead.
2026-10-02 13:04:01 +02:00
Linus Rath ee20f9a06d fix: drop widget controls that look like they do something they do not #52
The Month widget drew back and forward arrows, and the Vacation widget
an on/off switch. A widget cannot page through months or switch the
vacation reply, so both only opened the app, which the rest of the
widget already does. The arrows are gone; the switch becomes a chevron,
and the Active/Inactive pill under the title still says which state
the reply is in.
2026-10-02 13:03:19 +02:00
Linus Rath 60b92e6ef4 fix: give widget buttons a 48dp tap area #52
Most widget buttons were 32dp tall (Archive, Delete, Reply and Next on
the triage card, Accept, Maybe and Decline, Send wishes, Mail my free
times), Up next's were 36dp and the round compose/new buttons in the
headers 28-32dp, all under the 48dp a finger needs, so taps landed
beside them and did nothing.

Button and Fab take a transparent tap area around what they draw
(`slop`), and the layouts give up the same amount of padding or gap
next to them, so the widgets look the same but the buttons take a 48dp
tap.
2026-10-02 13:01:57 +02:00
Linus Rath c520e0a125 fix: keep widget text and counts inside small and translated widgets #52
Checked every widget at its smallest size with long names, big counts
and German labels. What spilled over:

- the Inbox header at two cells: the count pushed the compose button
  off the edge (search now goes at that width, the title truncates
  first, and only the unread count stays);
- a long sender pushed the triage card's time and position out;
- German button labels were cut ("Archivie", "Lösc"); the triage
  buttons now drop to icons, Archive and Delete first, when the labels
  do not fit;
- five-digit counts overflowed Unread count, Tag and Mail and next, and
  "/ 23000" wrapped a character per line; counts from 10,000 up are
  shortened (10K, 12,3K) and the total goes where it does not fit;
- "ungelesen" was cut beside the number in Mail and next and moves
  under it instead.

Every widget header now lets its title give way before the buttons on
its right.
2026-10-02 13:01:05 +02:00
Linus Rath c9a0718acf fix: keep the app from closing when a files widget opens it offline #52 2026-10-02 12:57:43 +02:00
Linus Rath c5143975db fix: finish the widget refresh after leaving the app #52
The previous fix started the refresh as the app went to the background,
but it still stopped after its first request: fetch hands every response
over through a JS timer, and React Native pauses timers while the app's
activity is in the background, so the answer was only read once the app
came back. The widgets kept the previous account after a switch and
Home, and changes made in the app reached them late.

The refresh now runs as a headless task of its own,
BulwarkWidgetRefresh, started through BulwarkWidgetRefreshService. A
running headless task keeps the timers going, and the service keeps the
process from being frozen before the widgets are redrawn. Where Android
refuses to start the service the app refreshes in place as before. A
refresh started for a change also no longer joins one that began before
it.
2026-10-02 12:01:31 +02:00
Linus Rath ae3121a746 fix: load calendar and tasks into the widgets when the app is not running #52
The calendar, tasks, files and scheduled-send parts of the widgets only
refreshed while the app's UI was running, because they reuse the app's
stores, which are bound to its JMAP client. A widget event or push that
woke the app without its UI refreshed mail only, so invitations
answered elsewhere, tasks ticked on another device and new events
stayed as the app last left them. A tick or RSVP the server refused in
that state was also never undone, since nothing reloaded those parts.

When no UI has started in the runtime, nothing else uses that client,
so the widget refresh now signs it in to the active account and loads
calendar, tasks, files and scheduled sends the way the app does. Once
the UI starts it owns the client again.
2026-10-02 12:01:20 +02:00
Linus Rath 2cec096040 fix: open a widget's event in the account it belongs to #52
Event links from the widgets named only the event (and the JMAP account
of a shared calendar), never the signed-in account it was shown for.
With another account active by then, the Calendar tab opened without
the event, or with an id both accounts use, on the other account's
event. The links now carry `appAccount`, and the app switches to that
account first, as it already did for message links.
2026-10-02 11:46:53 +02:00
Linus Rath a694751944 fix: send widget RSVPs and task ticks to the item the tap was made on #52
An RSVP or task tick named only the item's id and looked it up in the
stored snapshot when the tap arrived, then used the snapshot's active
account. Between a refresh and the redraw that follows, a tap made on
the old drawing could reach a snapshot of another account, and JMAP ids
are short per-account sequences, so the lookup could match that
account's task or event with the same id. A tick also flipped whatever
the stored state was, so a quick second tap on a drawing that had not
caught up yet undid the first.

The tap now carries what the call needs: the item's account, its server
ids and, for a tick, the state the user asked for. The change laid over
the widget data only touches items of that account. Drawings made
before this keep working through the old lookup.
2026-10-02 11:45:48 +02:00
Linus Rath 6c797f6e2c fix: refresh the widgets when the app goes to the background #52
Leaving the app scheduled the widget refresh with a zero-delay timer.
React Native pauses JS timers while the activity is in the background
(timers in the bridgeless runtime have no immediate path for a zero
delay), so the refresh only ran once the app came back or a widget task
happened to start. Switching accounts and pressing Home left the
widgets on the previous account's mail. The refresh now starts directly
from the app state change.
2026-10-02 11:42:50 +02:00
Linus Rath 7fe4a1ba27 feat: show on the widget when an action could not reach the server #52
A widget button whose JMAP call failed (offline, refused, the account's
credentials unavailable) only logged a console warning, so to the user
the button had simply done nothing, or had worked. The change is now
undone on the widget and a notice says what failed: "Couldn't archive.
Tap to try again." on the triage card, which goes back to the message,
and the same for RSVPs on the invitation card and task ticks on Tasks
and Today overview. Tapping the notice repeats the tap; any new tap or
ten minutes clear it.
2026-10-02 11:41:22 +02:00
Linus Rath 79c53e713c fix: make widget task ticks easy to hit #52
The tick on the Tasks widget was a 22dp circle, and the rest of the row
did nothing: a tap on the title or just beside the circle was lost. It
now takes the row's full height and 50dp of its width, and the rest of
the row opens the calendar. Today overview's task rows grow from 32dp
to 48dp for the same reason; the widget fits fewer of them where space
is short.
2026-10-02 11:32:50 +02:00
Linus Rath 26d0cf3ccc fix: keep a widget archive when a second tap's refresh lands before it #52
Every widget tap runs as its own headless task in the same JS runtime,
and the tasks interleave at every await. Each one loaded the stored
snapshot, edited it and saved it back, then refreshed from the server.
That went wrong in three ways:

- two taps loaded the same snapshot and the second save dropped the
  first change;
- the refresh of the first tap read the server before the second tap's
  archive landed and was stored anyway, so the message came back (seen
  on the emulator with a slow `Email/set`);
- a redraw of an older snapshot could land after a newer one, leaving
  buttons that act on a message the widget no longer shows.

The snapshot now only holds what the server last said. A button's
change is kept as a pending op next to it and laid over it when a
widget draws; it stays until a refresh that started after the server
took it has been stored (calendar and task changes only count as caught
up by a refresh that loaded calendar and tasks). A call that fails
drops its op, which also undoes an archive made offline: before, the
failed refresh kept the edited snapshot and the message stayed hidden
although it was never archived. Reads, writes and redraws of the
widget data run one at a time, and every redraw draws the latest view.
2026-10-02 11:32:38 +02:00
Linus Rath bc74d3bbac fix: keep the app from closing when a calendar widget opens it offline #52 2026-10-02 11:22:10 +02:00
Linus Rath 80f7278a6f fix: stop widget buttons from going dead when two are drawn in the same millisecond #52
react-native-android-widget registers every button's PendingIntent with
the current time in milliseconds as its request code and
FLAG_CANCEL_CURRENT. All buttons of a widget share the same action and
provider, so the request code is the only thing telling them apart: two
buttons registered within one millisecond got the same intent, and the
second registration cancelled the first. That button then did nothing
when tapped. The emulator takes 3-10 ms per button, a current phone well
under one, so which button died depended on timing.

The library is patched with patch-package (run on postinstall) to hand
out request codes that count up from the time, unique in the process
and clear of the codes an earlier process left on the launcher. The
patch keeps the library's CRLF line endings, hence `-text` for
`patches/*.patch`.
2026-10-02 11:14:40 +02:00
Linus Rath 604843a408 docs: list every widget tap target and how each one fails today #52 2026-10-02 11:08:05 +02:00
Linus Rath 1b4b185ac8 chore: allow app store distribution under a provisional AGPL permission
App store terms put limits on recipients, such as device caps and DRM,
that section 10 of the AGPL does not allow, so the app could not be
shipped through the Apple App Store or Google Play without breaking the
licence.

Add a "Bulwark Mobile" header and a section 7 additional permission to
LICENSE that allows app store distribution as long as the source of
every shipped version is public, free, linked from the listing or the
app, and not further restricted.

The permission is not final yet: not every earlier contributor has
answered the consent request. A reservation limits it to contributions
made on or after 30 September 2026 and to older parts whose authors
have agreed. The README licence line says the same.
2026-09-30 12:13:28 +02:00
Linus Rath dfaf1f3e5e chore: update version number to 0.1.69 2026-09-27 18:30:20 +02:00
Linus Rath 97c4f58379 feat: change the app icon to the white mark on raspberry 2026-09-27 18:29:50 +02:00
Linus Rath 5dfe869acb ci: publish a checksum and signed build provenance with the release apk 2026-09-27 15:23:51 +02:00
Linus Rath ab8a911863 chore: update version number to 0.1.68 2026-09-27 12:47:00 +02:00
Linus Rath dca9417aee feat: restyle calendar events and toasts to the app design 2026-09-27 12:34:58 +02:00
Linus Rath 8f604bf02e feat: merge contacts and calendar sync with the phone on Android #34
Brings feat/device-sync into main: on Android, each account's address
books and calendars can sync both ways with the phone's Contacts and
Calendar apps (Settings -> Contacts / Calendar -> Sync to this
device), through a "Bulwark" Android account with a contacts and a
calendar sync adapter.

- native side: the account, both sync adapters, provider access scoped
  to the account, push routing to the syncs, sync problem notices
- the sync engine in `src/device-sync`, run as a headless task
- the settings sections, sign-out and removal handling, translations
- design, limitations and verification in `docs/device-sync.md`
2026-09-27 12:14:37 +02:00
Linus Rath cf93f68fd3 docs: add the device checks of the review fixes to device sync's verification #34
The fixes since the first device runs were checked on the emulators
too: moves that stay the same event, statuses, server deletions, items
moved away before a deletion, quiet runs, the permission flow,
uninstalling and reinstalling, and the settings' collection count. The
list of what only tests cover shrinks accordingly.

The device runs also showed that Stalwart stores a calendar's default
alerts on the series, so removing every reminder of an occurrence is
put back and reported for such a series as well, and that Google
Calendar's first edit of an occurrence brings its reminders back
without a report.
2026-09-27 11:59:20 +02:00
Linus Rath 2ee056749c test: give the big meeting tests time on a busy machine #34
The planner tests that build meetings of 25 attendees on 52 overrides
take 0.3-1.8 s alone, but one of them passed the default 5 s limit in
a full run next to a release build. The two slowest tests of the file
already had 60 s; the other heavy ones get the same.
2026-09-27 11:57:56 +02:00
Linus Rath 10edc331a5 fix: keep device sync's notes out of the warning toast of debug builds #34 2026-09-27 11:23:44 +02:00
Linus Rath 157ea398b8 fix: update the sync settings' collection count after the chooser listed new ones #34
The account row in the sync settings counts the synced address books
or calendars from its own copy of the list, loaded when the row
appears. The chooser loads a fresh list when it opens, so a calendar
created meanwhile showed there but not in the count ("4 of 5 sync"
stayed after a sixth appeared, and turning it on or off changed
nothing) until the settings page was opened again.

Every fresh list now reaches every view of that account's collections,
so the count follows the chooser.
2026-09-27 11:23:44 +02:00
Linus Rath e09640cfc2 docs: describe device sync after the last round of fixes #34
Brings the doc up to the code after the second round of fixes and the
code review: moves of meetings, deletions of items another client
moved away, big meetings written in several transactions, occurrence
colours and reminders, the stored status of runs that did nothing,
push types by server capability, reminders while the phone's calendar
can't sync, and sign-out waiting for a turn-on in progress.

The known limitations gain what the fixers and the device runs found
but left alone: pairs whose patch fails, a field the server changes
during a pending move, Google Calendar's copy-and-delete moves and its
copied reminders, group cards deleted after they moved away, and a
status line that shows item errors but not conflicts.
2026-09-27 10:32:25 +02:00
Linus Rath ae679a7d3d docs: list contacts and calendar sync with the phone as done on Android #34 2026-09-27 10:24:06 +02:00
Linus Rath 156e79b348 fix: keep a card or event another app moved away when it is deleted on the phone #34
A contact or event deleted on the phone that another client meanwhile
filed in an address book or calendar the phone doesn't sync was
destroyed on the server by the phone's next sync, and the other
client's move with it. 0faee56 made the sync decide such a deletion on
the server's version, but the planners still destroyed it: the
contacts planner saw the card in no synced book and destroyed it, and
the calendar planner could not take the new version at all (it placed
the event before looking at the deletion), so the old one decided.

The deletion is now moot when the server's version holds none of the
collections the rows were written for and none the phone syncs: only
the rows go, nothing is uploaded, and the object stays where the other
client put it. A deletion still goes up when the book or calendar was
deselected on the phone before the upload, and when the object was
moved into another collection the phone syncs, since the phone would
otherwise miss an object it should show. The calendar planner now lets
a deleted row's shadow follow the server before it places the event.
2026-09-27 10:20:56 +02:00
Linus Rath df3175ff72 refactor: decide in one place whether a card is in read-only books only #34 2026-09-27 10:20:55 +02:00
Linus Rath 800c4e1442 fix: stop waiting for React Native once a sync gave up on it #34
A sync that had to start React Native left a listener on the React
host until a context came up. When the start failed (a debug build
without Metro) or never finished, every retried sync added another
one, each holding its run, account and extras, for the life of the
process, and they all fired at once when a context finally came up.

The wait now ends once, and removes its listener, for whatever
reason: a ready context, a start task that faulted, or 10 minutes
(no sync or push job waits longer; JobScheduler stops them). A
context that comes up later runs nothing of it. The adapter's own
check of its run in the callback still covers a run it gave up
sooner.
2026-09-27 10:20:55 +02:00
Linus Rath 5e62cf71f4 fix: add no extra account named after its id when Android refuses one #34
Turning device sync on retried with the app account's registry id as
the Android account name whenever adding the account failed. That
fallback is for one case only: the login name turned out to belong to
another app account's Android account, which the native module
reports with the `conflict` code. A passing AccountManager failure or
a permission error instead created a second account named after the
opaque id, which then showed in the Contacts and Calendar apps.

Only a `conflict` rejection takes the fallback now; any other failure
fails the turn-on as before.
2026-09-27 10:20:55 +02:00
Linus Rath ef96acd05f fix: finish turning sync on or off before signing an account out #34
Turning device sync on or off runs one step after the other per
account, but signing out did not wait its turn. A sign-out that
started while a turn-on was still setting up (or queued behind a slow
turn-off) found no Android account yet and went ahead; the turn-on
then created the Android account, stored it and requested a sync for
the signed-out account, and the next check posted "Sign in to keep
syncing". A turn-off still tearing down could likewise store its
preferences after the sign-out had forgotten the account.

The release before a sign-out is now a step in the queue of every
account it releases, taken in a fixed order: it waits for the steps
under way and releases what they set up. A turn-on or turn-off asked
for while the sign-out runs comes after it, and does nothing once the
account was released; if the user stays signed in it goes ahead.
2026-09-27 10:20:55 +02:00
Linus Rath 0664a31f8b fix: keep reminding of synced events while the phone's calendar can't sync #34
With calendar sync on and the calendar app reminding, Bulwark skipped
its own reminders for the synced calendars whenever sync was on in the
app, even when the phone's calendar received nothing: its calendar sync
paused in Android's account settings, Android's auto-sync off, or the
calendar permission revoked. Events created on the server meanwhile
never reached the phone, and nobody reminded of them.

The reminder filter now keeps what Android last said about each
account (the permission, `getMasterSyncAutomatically` and the
calendar's `getSyncAutomatically`) and stays quiet only while the
phone's calendar receives events. Auto-sync off counts as not
receiving: Android then runs manual syncs only, and the periodic,
pushed and app-triggered syncs are not manual. Android's side is read
when the scheduler subscribes and again whenever the filter is asked
for; a changed answer makes the scheduler reschedule. Until it was
read, Bulwark reminds: a double reminder is better than a missed one.
2026-09-27 10:20:55 +02:00
Linus Rath 217bb59d57 fix: forget a contact created and deleted on the phone between two syncs #34
ContactsProvider soft-deletes every raw contact and group of a sync
account, also one that never reached the server (DELETED=1, SOURCE_ID
NULL). When a contact or group was created and deleted on the phone
before a run claimed it, no upload phase took the row: without an
identity or a pending create it belongs to no JMAP account, and the
claim step skipped deleted rows. The row stayed for good, and the
upload-only check and the teardown counted it as a waiting change:
every upload sync the providers schedule became a full run, and
turning sync off or signing out always asked about a change that would
be lost.

The claim step now hands such a row to the planner, whose plan for it
is a purge, and `hasLocalWork` and `countPending` leave out deleted
rows that have neither an identity nor a pending create. Nothing is
uploaded for them. Calendar has no such gap: CalendarProvider
hard-deletes an app-deleted event without `_SYNC_ID`, and a claimed
event carries `~pending/<uid>`, so it is soft-deleted with its pending
create and destroyed by uid as before.
2026-09-27 10:20:55 +02:00
Linus Rath eebb00238c fix: look for new mail when a push names changes it can't read #34
The native push router and the push task decided separately whether a
push may carry mail, and disagreed. For an account entry that is not
an object, or a map that names no type, the router started the mail
task as possible mail, but the task's guard read the same payload as
"no mail" and returned, so new mail it announced was never looked for.
A push without a kind was judged by its map in the router but taken
for mail by the task.

Both now apply the same rules: anything but a StateChange, a map that
is missing, unreadable or names no type, an unreadable account entry,
or a mail type means the mail task runs. Only a readable StateChange
naming contact, calendar or other non-mail types skips it.

Checked with one table of payloads run against the router's decision
and the task's guard alike.
2026-09-27 10:20:55 +02:00
Linus Rath 41e2bdf922 fix: notify new mail again when a push names Email but not EmailDelivery #34
On main every push ran the mail task, whose legacy path notifies the
newest unread inbox mail. Device sync narrowed that to pushes naming
EmailDelivery, so contact and calendar changes never notify. But a
subscription made by an older build listens to Email and Mailbox until
the app is opened again, and a server without EmailDelivery reports
new mail only as those changes. Their new mail no longer showed a
notification while the app was closed.

Email and Mailbox now count as possible mail again, in the native push
router and in the push task alike. A push that names only contact or
calendar types still never takes the mail path.

Checked with the push router and push task tests: an Email or Mailbox
change starts the task and the legacy path notifies the message,
next to the syncs a mixed map asks for.
2026-09-27 10:20:55 +02:00
Linus Rath 11b897177d fix: keep mail push on servers without contacts or calendars #34
The push subscription got the contact and calendar types whenever the
account had device sync on, without asking whether its server has
contacts or calendars. Sync can be on for such an account: the settings
list every signed-in account, and a server can drop a capability later.
A server that refuses unknown types then refused the update and the
replacement, and push setup failed: a new account got no mail push, an
existing one lost it when its subscription expired.

The types are now added only for a capability the account's session
advertises, and a write the server refuses while it carries device
sync's types is retried with EmailDelivery alone, so mail push never
depends on device sync.

Checked with the push setup tests against a fake server that refuses
unknown types: a new subscription is created mail-only, an existing
one is kept and refreshed, and another account's subscription is left
alone when its server lacks the capability.
2026-09-27 10:20:54 +02:00
Linus Rath 029eba4562 fix: keep the last sync's result when a sync after it finds nothing to do #34
The settings show the stored status of the last sync. A sync that
finds nothing to do still replaced it, so a change the server refused
and the phone put back, or a conflict, was shown for a second or two
and then gone. The previous fix covered only the upload sync Android
starts right after a manual one. When changes arrive during a sync,
the adapter runs a full sync right after it, which finds nothing new
on the server and replaced the status 1-2 s after the real one; a
manual sync right after another does the same.

Now what counts is what a run did, not how it started. A run that
wrote nothing either way and reported no conflict, item error, note or
failure keeps the details of the run before it and moves only the time
on. After a failed run it shows the sync as working again, without the
failure's message. Any other run replaces the status, so a problem
stays only until the next run that does something.
2026-09-27 10:20:54 +02:00
Linus Rath 66789251e3 fix: keep a meeting moved in Etar the same event instead of cancelling and re-inviting #34
Etar moves an event by inserting a copy from its editor and deleting
the old row. Its editor keeps the user's own attendee row out of the
attendee list, so the copy of a meeting that lists the user has no row
for the user, and the copy of one that does not gets the user as its
organizer. The pairing compared the attendee addresses of both rows
exactly, so a moved meeting never paired: it was deleted on the server
and created again under a new uid, the other attendees' answers were
lost, and everyone got a cancellation and a new invitation.

The pairing now leaves the user's own addresses out of that
comparison. Anyone else still has to be on both rows, so a copy that
lost or gained another attendee is not taken for the same event.

Checked with planner tests for both kinds of meeting and a run through
the engine with a fake Etar move.
2026-09-27 10:20:54 +02:00
Linus Rath 117823f746 test: make the device sync fakes as strict as Android and Stalwart #34
The fake providers and the fake server were more lenient than the
real ones, so three bugs showed only on a device: an event inserted
without a status, small contact photos, and meetings too big for one
provider transaction.

The fake CalendarProvider leaves STATUS NULL when an app writes none.
The fake ContactsProvider keeps a photo within the 96 px thumbnail as
the thumbnail alone, without a PHOTO_FILE_ID, and serves it to
readPhoto from there. A batch fails with tooLarge when it weighs more
than one Binder transaction, a photo counted by its bytes. The fake
server drops empty maps inside recurrence overrides, as Stalwart does.
Tests now read photos the way the engine does, and the helpers that
made up for the old fakes are gone.
2026-09-27 10:20:54 +02:00
Linus Rath 3ab131dbe8 fix: keep the last sync's result when an upload sync finds nothing to do #34
Android starts an upload-only sync right after a manual sync, and 30 s
after any app write. When nothing waited on the device it ended at
once, but it still stored its status, so the settings showed a plain
"synced" and lost the conflicts and item errors of the run just
before it.

Such a run did nothing and now leaves the stored status alone.
2026-09-27 10:20:54 +02:00
Linus Rath 9439a4ffc0 fix: keep an occurrence's colour cleared, and report reminders that can't go #34
Clearing the colour of one occurrence of a coloured series removed the
occurrence's own colour, so it showed the series' colour again after
the sync. The occurrence now gets an empty colour, which Stalwart keeps
and which shows as the calendar's colour, as on the phone.

Removing every reminder of one occurrence sent an empty set of alerts,
which Stalwart does not store: the occurrence then showed the series'
reminders again, without a word. When the series uses the calendar's
default reminders, the occurrence leaves the defaults and has none, as
before. When the series has reminders of its own, the occurrence would
show them whatever is sent, so the change is put back and the report
says why (`remindersNotRepresentable`).

Checked against calcard 0.3.14, with planner tests and runs through
the engine and the fake server.
2026-09-27 10:20:54 +02:00
Linus Rath 4bcca70548 fix: write big meetings in several provider transactions #34
A recurring meeting with many attendees on many occurrences (25 on 52,
100 on 12) is about 2 MB of provider operations, and one Android
provider transaction carries 1 MB. Every event was written as one
group that the engine never splits, so such a meeting was refused every
run and never reached a phone that synced it for the first time.

An item too big for one group now comes as a chain of groups that the
engine applies one by one, each in a transaction of its own. A new
event is written master first, with its identity, then its occurrences
in groups that fit, then its shadow; a write into an existing event
writes the occurrences first and the series with its new shadow last.
When a run stops between two groups, the next one finds the event by
its identity and writes only what is missing, planned from the old
shadow, so no row is written twice and no server change is lost.

Checked with planner tests for both meeting sizes and every cut point,
and runs through the engine with a provider that refuses transactions
over 1 MB, a crash after every batch included.
2026-09-27 10:20:54 +02:00
Linus Rath 0110eebb78 fix: upload an edited event another client moved out of synced calendars #34
When another client moved an event into a calendar this device does
not sync while the device had an unsent edit of it, the calendar
planner refused to merge the server's version, since the event is in
no synced calendar any more. The edit then waited on every run as a
planner error, and the event never left the device.

The server's version now merges into the rows where they are, so the
edit uploads against it without undoing the other client's changes,
and the rows go once the server has taken the edit.

Checked with a planner test and a run through the engine and the fake
server: the series rescheduled on the device keeps the occurrence the
other client added, and no rows are left.
2026-09-27 10:20:54 +02:00
Linus Rath 724c3b80ee chore: update version number to 0.1.67 2026-09-27 10:15:08 +02:00
Linus Rath f24d919fdb docs: describe device sync after the review fixes and how it was verified #34
The device sync doc still described the feature as first built. The
review fixes since then changed pairs, merges of items outside the
synced collections, the order of uploads, group handling, photos, the
provider-op scope checks, the settings actions and what the report
says about changes put back on the phone.

Each fix's doc notes are applied against the code as it is now, the
known limitations gain what the device runs found (an identity as
organizer, Stalwart refusing answers in read-only calendars, Fossify
Calendar around DST changes, AOSP's one-line address edits), and a new
Verification section lists what was checked on emulators and the scale
run's numbers.
2026-09-27 09:15:53 +02:00
Linus Rath 13d5d1733b fix: say "1 of 3 syncs" when one collection of several syncs #34 2026-09-27 08:50:59 +02:00
Linus Rath 84f6b5777e feat: translate device sync into all 26 languages #34
The strings of device sync were English only: the "Sync to this
device" sections in the Contacts and Calendar settings with their
dialogs, and the notifications about signing in again and about sync
problems. Each language keeps the form of address its file already
uses, the app's own terms, and Android's names for its settings,
auto-sync and the Contacts and Calendar apps.

Checked: every message parses as ICU with the same arguments as the
English one and the plural categories of its language,
`node scripts/rn-missing-translations.mjs <lang>` lists no device sync
key for any language, `npm run i18n:check` and the i18n tests pass.
2026-09-27 08:50:15 +02:00
Linus Rath 7453513e08 fix: stop waiting for group memberships in read-only address books #34
A contact added on the phone to a group in a read-only address book
(one shared with the user) could never reach the server, yet it stayed
waiting on every run: the planner puts such a membership back, but the
engine never told it which groups are read-only, so only groups of
another account were recognised. Turning sync off or signing out then
always asked about a change that could never go up.

The engine now tells the planner which synced groups are in read-only
books only, from the group cards it holds and the books' rights. The
membership is put back and reported, and nothing is left waiting.
2026-09-27 08:44:58 +02:00
Linus Rath d1ae713b1f fix: report changes the phone puts back because the server can't take them #34
A change the server cannot take was put back on the phone without a
word: an edit in a read-only calendar or address book, an answer
where answers are not allowed, an event moved to another account's
calendar, an occurrence the server keeps without its series, a
contact added to a group this device can't change. The change just
vanished.

The planners now say why they put a change back, and the sync report
lists the item with that reason (`readOnly`, `rsvpRefused`,
`instanceOnly`, `crossAccountMove`, `groupNotWritable`). A contact
whose only change still uploads through a group card is not listed.

A refused edit was also reported as "invitations not sent", as if it
had been stored without them; the note now appears only when the
change was stored without its scheduling messages.
2026-09-27 08:44:58 +02:00
Linus Rath 1c75da8733 fix: remove an event created in a read-only calendar and say so #34
An event created on the phone in a calendar the user may only read was
meant to be removed, since the server takes no changes there, but the
claim step ignored that answer: the event stayed on the phone, was
never sent or reported, and turning sync off always warned that a
change had not reached the server.

The claim step now applies it: the row goes, the report lists it as
read-only, and nothing is left waiting.
2026-09-27 08:44:58 +02:00
Linus Rath 1a348792c0 fix: let the server's deletion win over an event moved on the phone #34
Etar moves an event by deleting the row and inserting a copy. When the
event was deleted on the server meanwhile, the sync removed the deleted
row but kept the copy, which then uploaded as a new event: the event
the server had deleted came back in the target calendar, for everyone.
That happened whether the download found the event gone first, which
is the usual case, or the patch of the move did.

A move is an edit, and the server's deletion wins over an edit: the
copy that pairs with the deleted row now goes with it, and the report
counts the conflict. A copy edited beyond the move stays a new event.

Checked through the engine with the event destroyed before the run,
while the move was sent, and with the patch answered notFound.
2026-09-27 08:44:58 +02:00
Linus Rath f24ff7da9e fix: keep the other edits of an event moved to another calendar #34
Etar moves an event, or turns a series into a single event, by deleting
the row and inserting a copy. The pair uploaded the move as a patch of
the event, but then took the copy as it stood for the new baseline and
wrote the server's version over it, so everything else saved with the
move was dropped: a changed reminder came back, and an edit made before
the move never reached the server. The synced occurrences, which Etar
does not copy, were deleted with the old row, losing an occurrence edit
that was still waiting.

The copy now starts from the old row's baseline with only what the move
changed, so any other difference stays dirty and uploads right after
the move as an ordinary update. The old row's occurrences move to the
new row with what they wait to upload. Attendee rows of the copy are
the app's re-insert (Etar writes everyone as required, without a
status) and are not read as edits.

Checked with planner tests and runs through the engine and the fake
server: a reminder saved with the move and a pending occurrence edit
both reach the server, the event keeps its uid, and the echo writes
nothing.
2026-09-27 08:44:58 +02:00
Linus Rath 0faee568cc fix: let a contact deleted on the phone and moved away meanwhile survive #34
When another client moved a card out of every address book the device
syncs while the device deleted the contact, the engine noted the card
for dropping but never showed its new version to the planner, so the
deletion was decided on the old copy and destroyed the card. A deleted
item is now merged like any changed one first; the contacts planner
then leaves a card that is in no synced book. When the planner cannot
place the object, the deletion stays as it was.
2026-09-27 07:53:37 +02:00
Linus Rath 654675d90b fix: keep unsynced changes waiting when sync is turned off before the first sync ended #34
Turning sync off uploads what the device changed without downloading
first; every upload carries the state the device last synced, so the
server refuses it when something changed meanwhile and the engine
downloads first. Before the first sync of an account had ended there
was no such state, and edits and deletions went up unguarded, able to
overwrite what another client had changed. They now wait, and the app
asks before they are dropped; new items still go up.
2026-09-27 07:53:37 +02:00
Linus Rath 153e85cb80 fix: finish turning sync off when the app goes to the background #34
Turning sync off first uploads what the device changed, then removes
the synced rows. Started from the settings screen, that work yielded
the JS thread with timers between its steps, and React Native pauses
timers while the app is in the background and no headless task runs:
leaving the app stopped it halfway, holding the account's sync lock,
until the app came back. The teardown now waits without timers.
2026-09-27 07:53:37 +02:00
Linus Rath b86e0a118e fix: size provider batches by what the Binder transaction carries #34
The batch budget counted JSON characters, but a Parcel carries strings
as UTF-16 plus a few hundred bytes per operation, so a full batch was
about 700 KB of a 1 MB Binder buffer the whole process shares. Batches
are now weighed at two bytes per character plus 300 bytes per op and
kept under 500 KB of that. The state op no longer joins a batch whose
last item already fills a provider yield window of 499 operations,
which ContactsProvider refused.
2026-09-27 07:53:37 +02:00
Linus Rath 1ee6bc459d fix: move group members on the phone after an interrupted sync #34
When a group's members change on the server, the download fetches the
contacts whose membership flipped in a chunk after the group's. The
group's chunk already stored its new version, so when the sync stopped
before the members' chunk, the next sync found no flip any more and
the phone kept the old members. Those contacts are now listed with the
group's rows, as objects to fetch first, until their own chunk ran.
2026-09-27 07:53:36 +02:00
Linus Rath fe0aaa1247 fix: load a book selected while the first sync was cut short #34
A full reconcile that ran out of time resumes after the last object it
processed, in the sorted order of every listed id. It listed the
collections selected at the time it resumed and marked them all loaded
at its end, so a book turned on in between lost the objects that sorted
before that point, and was never loaded again. The reconcile now keeps
the collections it started with; one selected since is loaded after it
like any newly selected collection.
2026-09-27 07:53:36 +02:00
Linus Rath 78700401a2 test: keep a contact's groups when an upload runs without a download #34
A sync with nothing to download, and the upload before sync is turned
off, used to plan uploads with an empty group index, so the write after
the upload deleted the uploaded contact's membership rows on the phone.
The previous commit's refresh of the index before planning uploads
fixes both; these tests cover the two paths so it stays fixed.

Checked: both tests fail without that refresh.
2026-09-27 07:53:36 +02:00
Linus Rath 2e1cb58548 fix: finish loading or removing a book after an interrupted sync #34
The SyncState's `selected` said which collections were on the device,
and changed only when a load or a removal had finished. A book turned
off and on again while the device was still removing it looked loaded,
so the contacts removed so far never came back; a calendar came back
empty. A book turned on and off again while it was still loading never
looked loaded, so the contacts loaded so far were never removed.

`selected` now names only collections whose rows are all on the device:
a collection leaves it before its first row is removed, and the
contacts SyncState lists collections that are loaded or removed only in
part, so the next run loads them again when they are selected, or
removes the rest when they are not. A book that still holds a change
waiting for its upload stays in that list instead of `selected`.
2026-09-27 07:53:36 +02:00
Linus Rath ecbd218c34 fix: sync only the primary account's collections by default #34
The settings chooser and the reminder filter treat only the capability's
primary account as the user's own, while the engine also counted any
account the server marks as personal. On a server that lists a second
such account, its address books and calendars synced although the
chooser showed them off. The engine now uses the same rule as the app.
2026-09-27 07:53:36 +02:00
Linus Rath 474eae9992 fix: never duplicate an event's reminders after a busy calendar database #34
A provider that is busy may commit a batch up to a yield point and then
fail it. The engine retried every group of such a batch and relied on
the groups' asserts to refuse one that was already committed, but a
download into an unchanged event asserts only the event's own columns:
when it had only added a reminder or an attendee, the retry added it a
second time.

A download is now planned again from a new read after a failed batch
instead of being sent again, which turns a committed group into an echo.
The fake provider can commit a batch up to a yield point to test it.
2026-09-27 07:53:36 +02:00
Linus Rath 7701a67dff fix: keep favourites and joins when the server refuses an edit #34
When the server refused an edit made on the device (a book or calendar
that turned read-only, an event in a calendar where only one's own
events may be changed), the engine put the item back by deleting its
rows and inserting a fresh copy. A contact lost its star, ringtone and
the link to the contacts it was joined with; an event got a new row id.

The rows are now rewritten in place from the server's version, as after
an accepted upload. A refused deletion still brings the item back as a
new copy.
2026-09-27 07:53:36 +02:00
Linus Rath 2e9e5bb785 fix: remove items the server deleted right after this device created them #34
After an upload the stored state stays where the download left it, and
the engine relied on /changes to report its own creates back. /changes
leaves out an object that was created and destroyed after the state it
is asked from (Stalwart does, RFC 8620 allows it), so an event or
contact created on the device and deleted on the server before the next
download stayed on the device for good.

The SyncState now lists the objects our uploads created, written with
their identity, and the next run fetches them first: one that is gone
loses its rows, an unchanged one writes nothing. An upload-only sync
runs in full while the list is not empty.
2026-09-27 07:53:36 +02:00
Linus Rath fad7b30dbf fix: upload new contacts whose address book was deleted meanwhile #34
A new contact is claimed for an address book before its create is
sent, so a retry after a lost response finds it again. When that book
was deleted on the server before the create went up, every attempt
failed and was retried after up to a day, forever. A book that is gone
can hold nothing of an earlier attempt, so the contact is now claimed
again for the book the settings choose.

An event cannot move on its own to another calendar: a create into a
calendar that is gone is no longer sent, and the report says why, until
the user moves the event in the calendar app.
2026-09-27 07:53:36 +02:00
Linus Rath 34236f677b fix: never resend a group's members from an outdated copy #34
A group whose latest server version the device could not store waits
on the stale list, and nothing uploads for it until it has been fetched
again. Membership edits bypassed that check: adding a contact to such a
group sent the whole member list as the device last knew it, which
removed members another client had added meanwhile. Membership edits
for a stale group now wait for that fetch, and the report lists them.
2026-09-27 07:53:36 +02:00
Linus Rath 09c2ff1f67 fix: upload a contact's removal from a group made on the phone #34
Whether a membership edit still has to go up is decided against the
group cards on the server, which the engine indexes from their shadows.
The index was only read before planning a download that held contacts,
so in a run that downloaded no contact, such as the upload sync after
an edit, it was empty: removing a contact from a group looked already
done on the server, the contact was marked clean and the removal was
never sent. The engine now reads the index before planning uploads too.
2026-09-27 07:53:35 +02:00
Linus Rath d088dd14e9 fix: stop asking about waiting changes after a group edit reached the server #34
A contact stays marked as changed until the group cards on the server
show its membership edits. The engine uploaded those edits as patches
of the group cards after the contacts, but never looked at the contact
again in that run, so it stayed marked until the next sync. Turning
contacts sync off or signing out right after adding a contact to a
group therefore asked whether to drop one waiting change, although
nothing was waiting.

Once the membership patches are accepted, the contacts that have
nothing else to upload are now cleared in the same run, behind the
same version check as any other clean write.
2026-09-27 07:53:35 +02:00
Linus Rath 58b74527b2 fix: keep group cards on the server when a sync stops while dropping them #34
The contacts SyncState lists the group rows on the device, so a group
an app hard-deleted (Fossify does) is found by its absence and deleted
on the server. The list changed only with a later state op, not with
the batch that inserted or deleted the rows. When a run stopped in
between (cancelled, out of time, a crash, or turning sync off while a
deselected book was being dropped), a group the engine had removed was
still listed, taken for a hard delete by the next run or by the
teardown's upload, and its card destroyed on the server; a group it had
inserted was never listed, so a later hard delete never went up.

A batch that inserts or deletes a group row now ends with a state op
holding that change, retries included, so the list and the rows never
disagree. A group a download finds unchanged is listed too, for rows
whose batch lost its state op at a provider yield point. And an absent
group whose card is no longer a group or is in no synced book is only
forgotten, never destroyed.

Checked with a crash at every checkpoint of a deselection, a teardown
and a deadline stop during one, a server-side move in a page of several
chunks, and a group written before an interruption and then deleted in
Fossify.
2026-09-27 07:53:35 +02:00
Linus Rath 7187cab904 fix: never undo server changes to a card moved out of synced books #34
When another client moved a contact or an event out of every address
book or calendar the device syncs while the device had an unsent edit
of it, the download only noted it for dropping after the upload. Its
shadow stayed at the old version while the stored state moved past the
change, so the upload passed `ifInState` and resent whole maps built
from the old shadow: a number or an occurrence added on the server
meanwhile was lost. A full reconcile that found such an object did the
same.

The download now merges the new version into the dirty rows like any
other change, so the upload builds on it, and the rows still go once
it is accepted. A contact's collections are read from its shadow:
SYNC1 lists only the selected books and is empty for a card that left
them all, so such a contact was never dropped after its upload. The
calendar planner cannot merge an event that is in no synced calendar
yet; until it can, such an event waits on the device and is reported
instead of overwriting the server.

Checked with the real planners through `/changes` and through a full
reconcile.
2026-09-27 07:53:35 +02:00
Linus Rath 7dbbd1ce25 fix: move events between calendars without recreating them on the server #34
Etar moves an event to another calendar, and turns a series into a
single event, by deleting the row and inserting a copy. The engine was
meant to pair the two rows and upload a patch of the existing event,
but it claimed every new row (a uid and a pending create) before it
asked the planner for pairs, and the planner pairs only unclaimed rows.
No pair ever formed: the event was destroyed and created again under a
new uid, which lost what the device cannot show (keywords, virtual
locations, links, the other attendees' answers, alerts relative to the
end) and sent every attendee a cancellation and a new invitation.

Pairs now upload before new rows are claimed, and a paired deletion no
longer counts toward the deletion threshold, since it is a move. When
an app changes the rows while the patch is on its way, they are read
and paired again; rows that no longer pair are left without the
identity instead of having the server's version written into the new
row, which overwrote the edit and left two rows with one identity.

Checked with the real planners: a move and a series turned single keep
their uid, keywords and attendee answers and send nothing; an edit made
during the upload is kept; an unchanged rewrite still pairs.
2026-09-27 07:53:35 +02:00
Linus Rath 78684e2492 fix: keep the last sync time current while the sync settings stay open #34 2026-09-27 07:52:12 +02:00
Linus Rath c635ccd586 fix: open the account's sync page from the sync settings #34
"Android account settings" opened Android's page for the Bulwark
account through `ACCOUNT_SYNC_SETTINGS` with the account as the
`account` extra. The Settings app of Android 12L and later passes only
`:settings:show_fragment_args` on to that page, which then found no
account and closed at once ("Account provided does not exist: null").
Since starting it succeeded, the fallback to the list of synced
accounts never ran, and the button seemed to do nothing.

The account now also goes in the fragment arguments, where those
Settings apps read it; older ones still read the plain extra.

Checked on the API 32 and API 34 emulators: the button opens the
account's "Account sync" page with its Contacts and Calendar switches.
2026-09-27 07:49:53 +02:00
Linus Rath 534cb8239e fix: keep the groups of a contact created on the device #34
A contact created on the device and put into a group before its first
sync lost that group: the write after its create took the rows' own
memberships as what the server already had, found them missing there
and deleted the rows, so the group card was never patched.

The create's write now starts from no memberships, as its fallback
already did: the device's memberships count as additions, stay on the
device, keep the contact dirty until the group cards show them, and
upload as patches of the group cards.

Checked with a planner test and end to end through the engine.
2026-09-27 07:19:27 +02:00
Linus Rath 6f574de0b9 fix: keep small contact photos on the device and on the server #34
ContactsProvider keeps a photo within its 96 px thumbnail as the
thumbnail alone, without a PHOTO_FILE_ID, and the thumbnail does not
read back. Such a Photo row looked empty to the planner. Every
download of the card inserted the photo again (two, then three
rows), and any edit of the contact on the device deleted the photo on
the server.

A Photo row now always counts as the contact's photo. Photo rows that
earlier downloads added again are deleted by the contact's next clean
write, one row is kept, and nothing reaches the server for them. A
photo set on the device is still uploaded.

Checked with planner tests and end to end through the engine, with a
provider that stores small photos as thumbnails as the device does.
2026-09-27 07:19:27 +02:00
Linus Rath d34641d1f9 fix: sync contacts whose photo is too large for the device #34
A contact's rows are written in one provider batch, and one batch is
one Binder transaction of at most 1 MB. A contact with a larger photo
failed with `tooLarge` on every run, stayed on `stale`, never showed
on the device, and its device edits never uploaded.

Photos above about 512 KiB are now kept on the server only: the
contact is written without the photo. The shadow records that the
device holds no row for the card's photo, so the missing row is not
read as a deletion. The same applies to a photo whose bytes could not
be fetched, which an unrelated edit of the contact used to delete from
the server. A photo set on the device still replaces the server's, and
one deleted there is still deleted on the server.

Checked with planner tests for the insert, the size limit, a photo
that could not be fetched and a merge, and end to end through the
engine with the probe's 900 KB photo.
2026-09-27 07:19:27 +02:00
Linus Rath 00f1a66095 fix: keep phone, email and website types as picked on the device #34
Types Android has but RFC 9553 lacks uploaded as a label naming them,
yet every label read back as a custom type, and MMS and company main
read back as Other and Main. A number typed Car showed up as
"Custom: car" right after the sync, an MMS number as Other, a mobile
email as "Custom: mobile" and a blog link as "Custom: blog".

Such a label now names its type again, for phones only next to the
feature the type uploads with, so a custom label stays custom. MMS
uploads its `mms` label next to `text`, which keeps a text-capable
number from reading as MMS, and `main-number` with `work` reads as
company main.

Checked with a round trip of every phone, email and website type,
both for a contact created on the device and for a type changed on
a synced entry.
2026-09-27 07:19:27 +02:00
Linus Rath 05e3605ec7 fix: keep a group rename when a membership change of the group uploads first #34
Accepting a membership patch of a group card wrote the server's title
over the device's and cleared DIRTY. A rename that had not reached the
server in the same run, because the server refused it or the user made
it after the group's own upload was planned, was silently undone.

The accepted write now keeps a pending rename that the accepted card
does not show: it stores the new shadow but leaves the title and
DIRTY, so the rename uploads against it. A card that shows the
device's title, or a different one from a concurrent server rename,
is taken as before.

Checked with a planner test and end to end through the engine with a
refused rename and a membership change in the same run.
2026-09-27 07:19:27 +02:00
Linus Rath 20effab062 fix: keep an address's parts in step with a one-line edit on the device #34
AOSP Contacts edits a postal address as one formatted line; the
provider then copies it into the street and empties the other parts.
The upload sent only the new `full`, so the components on the server
kept the old values: CardDAV clients and the app's editor, which show
the parts, still showed the old city.

A changed line now uploads with the parts it implies. They are read
back along the old line: each of its segments must be some of the old
parts, and a segment whose change is confined to one part rewrites
that part like an in-place edit of its field. When the layout changed
or a change spans two parts, the address keeps the whole line as its
street, which is what the device holds, rather than an address
without parts that the app's editor would show empty and drop on its
next save. An unchanged line still uploads nothing, and editors that
write the parts themselves are handled as before.

Checked with planner tests for the edit from the device test, a
street-only edit, the fallback, a re-inserted line and an editor
that keeps the parts, and with unit tests of the read-back.
2026-09-27 07:19:27 +02:00
Linus Rath c30b4a23d2 fix: stop contacts waiting forever on group memberships the server can't take #34
A contact added to or removed from a group this device can't change
stayed dirty on every run: the membership upload skips such groups,
while the contact's clean write kept the change pending until the
server showed it. Turning sync off and signing out then always warned
about changes waiting for the server.

The clean write now puts such a membership back to what the server
has, as it does for any edit of a read-only contact, and clears DIRTY;
when that was the contact's only change the upload counts it as
skipped. Memberships of writable groups in the contact's own account
still wait until the group card shows them.

This covers groups of another JMAP account. For groups in read-only
address books the planner is ready, but the engine does not tell it a
group's rights yet; that follows separately.

Checked with planner tests for additions, removals, both kinds of
group and a change next to an edit that uploads, and end to end
through the engine for a group of a shared account, where turning
sync off afterwards finds nothing waiting.
2026-09-27 07:19:26 +02:00
Linus Rath 13264d6479 fix: keep contacts the server also filed in unsynced books when deleted on the device #34
A contact or group deleted on the device was uploaded against the
shadow of its last sync: a download into a deleted item wrote nothing.
When the server had meanwhile also filed the card in an address book
this device does not sync, the deletion still saw only the synced book
and destroyed the card, its copy in the other book included.

A download into a deleted contact or group now writes the new server
card into its shadow, and takes over an identity that was never
written, so deleting our own lost create goes by id rather than by a
uid lookup. The rows and the group title stay as they are: the device
delete still wins. The upload then removes only the synced memberships
of a card that is also in other books, as the calendar planner already
does for events.

Checked with planner tests for contacts and groups and end to end
through the engine with the real planner: the card stays in the
unsynced book, the device rows are purged, and an echo writes nothing.
2026-09-27 07:19:26 +02:00
Linus Rath 7a53cd5d84 fix: let the user delete or restore held-back deletions from the sync settings #34
When a sync held back many deletions made on the device (Android's
2026-09-27 07:19:26 +02:00
Linus Rath 2b9d19521f fix: keep sync on when it is turned back on during a turn-off #34
Turning device sync off uploads what the device changed and then deletes
2026-09-27 07:19:26 +02:00
Linus Rath c777dc4089 fix: keep syncing a big meeting after it was edited on the phone #34
After an edit on the phone, every write into a recurring event asserted
each attendee and reminder row of every occurrence. For a meeting with
25 attendees and 52 occurrences with details of their own that alone
made the write 1.6 MB, over Android's 1 MB transaction limit: the event
could no longer be written, went stale in every run and kept its edit.

Only rows that are dirty themselves now assert their attendees and
reminders. A clean row's DIRTY flag already catches an app edit of
them, since CalendarProvider sets it on every app write to an event,
its attendees or reminders, so such a write still fails and is planned
again. The same meeting's writes now take about 0.65 MB.

Checked with the meeting above through the planner, and with an app
edit of a clean occurrence's attendees between planning and writing.
2026-09-27 07:10:47 +02:00
Linus Rath 6785276377 fix: put back an event moved in place to another account's calendar #34
An app that moves an event by changing its calendar in place can move
it into a calendar of another account on the device. A patch cannot
move an event between accounts, so the move was dropped; with nothing
else changed, the row was then marked clean where it stood. The phone
showed the event in the other account's calendar while the server kept
it in the old one, until the next server change moved it back.

The row now goes back to its calendar at once, rewritten from the last
server version. Changes made together with the move still upload, and
their accepted write puts the row back as before.

Checked with a run through the engine and the fake server with a shared
account: the event is back in its calendar and clean, and the server
and the other account are untouched.
2026-09-27 07:10:47 +02:00
Linus Rath d3262ff387 fix: send an answer to one occurrence in an RSVP-only calendar #34
In a calendar the user may only answer (mayRSVP without write rights),
an answer to the whole series was uploaded, but an answer to a single
occurrence goes into the occurrence's override and counted as an edit:
it was undone on the next sync without a word.

A change that holds nothing but the user's own answer, to the series
or to occurrences, now counts as an RSVP and is uploaded. Stalwart
0.16.23 refuses every update in a shared calendar without write rights,
answers included; the engine then puts the rows back from the server
and reports the refusal, as it already did for an answer to the series,
so the answer is no longer lost silently.

Checked with a planner test (a new and an existing override, and an
extra edit that is still reverted) and a run through the engine and
the fake server.
2026-09-27 07:10:47 +02:00
Linus Rath 7736b3b551 fix: keep a reminder changed on one occurrence of a series #34
In a series that uses the calendar's default reminders, a reminder set
on one occurrence went into the occurrence's override, but the
occurrence still used the defaults it inherits from the series, which
ignore an event's own alerts. The phone showed the default reminder
again after the sync while the server held the new one.

The override now also leaves the defaults (`useDefaultAlerts: false`),
so only that occurrence uses its own alerts and the series keeps the
calendar's defaults. Stalwart stores the flag of an override with the
override (as a JSPROP) and serves it back.

Checked with a planner test (an existing override and a new "this
event" edit) and a run through the engine and the fake server.
2026-09-27 07:10:47 +02:00
Linus Rath 2870a59971 fix: stop events from Google Calendar arriving as tentative #34
Google Calendar leaves STATUS empty on the events and exceptions it
creates, and CalendarProvider has no default for it. Device sync read
the empty value as 0, STATUS_TENTATIVE, so every event and every "this
event" edit made there uploaded as "tentative" and came back to the
phone that way; a copy made with "Copy to" also never paired with the
deleted original.

An empty STATUS now sets no status: a create leaves `status` out, so
the server keeps its default of confirmed, an override gets none of its
own, an edited event never uploads one, a merge takes the server's
without counting a conflict, and a pair ignores it. After the upload
the rows hold the server's status.

A new series from Etar's "this and following" also no longer uploads
the old series' exclusions from before its start, which Etar copies
into the new row's EXDATE.

Checked with planner tests for each path and a run through the engine
and the fake server.
2026-09-27 07:10:47 +02:00
Linus Rath 232e892328 fix: keep an occurrence's description or location empty once cleared #34
Clearing the description or location of one occurrence of a series in a
calendar app removed the value from the occurrence's override. An
override without a value of its own shows the series' one, so the write
after the upload put "Daily agenda" or "Room 1" back into the occurrence
and made it clean: the user's edit was gone without a trace.

An existing override now gets an explicit empty value while the series
has one to fall back to: the empty text for the description, and a
location without a name, the only empty forms Stalwart keeps (calcard
writes no LOCATION line for an empty map, which then reads back as the
series' locations). Without a series value the override's own is removed
as before. New exception rows keep treating an empty value as "inherit".

Checked with a planner test and a run through the engine and the fake
server: the row stays cleared and clean, and the echo writes nothing.
2026-09-27 07:10:47 +02:00
Linus Rath 49c5d05c30 fix: keep every provider write of device sync inside its own account #34
The native side ANDs the account's scope onto each selection the engine
sends, as `(where) AND (scope)`. A selection such as `1) OR (1` closed
the scope's parenthesis itself: on the calendar tables, where
CalendarProvider adds no account filter of its own, an update or delete
built that way would have reached the events, attendees and calendars
of other accounts, e.g. the user's Google calendar. The engine only
sends constant selections, so this guards against bugs rather than
anything an outside app could send, which is what the scope is for.

- Selections and sort orders must balance their parentheses outside
  quoted literals, and may not end the statement, carry comments or
  query other tables (`SELECT`, `UNION`, `ATTACH`, `PRAGMA`).
- An update of a data row by id that writes `data1` without restating
  the mimetype has the row's mimetype looked up: on a membership row
  `data1` names a group, which must be the account's, as on insert.
  Such writes by selection are refused.
- `data14`, a photo row's file id, is never written: another account's
  file id would have let `readPhoto` read that account's photo.

Checked: 58 JVM tests (`gradlew :app:testDebugUnitTest`, three new), the
debug build, and every selection the engine and planners send against
the new rules.
2026-09-27 06:12:08 +02:00
Linus Rath d135bb2449 docs: describe device sync as it was built #34
The design in f666699 was written before the code, and building it
changed a number of details, several of them after runs against the
real providers on the emulators. The doc now says what the code does:

- a sync run's steps in the order they run, the stale list, the
  reconcile cursor, the owner check, the time margins, and accounts that
  are left out when they refuse to list their collections;
- uploads in phases across accounts, the `ifInState` catch-up, account
  refusals, series splits and copied events, creates adopting only in
  their target collection, and poison markers;
- the contacts and calendar mappings as built (membership rows, text
  limits, postal and relation rules, read-only rows and calendars,
  exceptions, reminders under the old owner), and the columns the
  providers refuse to return;
- the native module's scoping, the app's store, lifecycle and triggers,
  and the failure matrix;
- decisions and limitations, with the questions still open.

Checked: every statement against the code of this branch.
2026-09-27 05:41:26 +02:00
Linus Rath 417ed5a326 fix: log why a device sync run failed #34
The sync adapter logged only the SyncResult counters of a run, so a
failed sync showed up in logcat as `ioErrors=1` and nothing else. The
reason was in the run's report, and JS warnings from a headless run
don't reach logcat; finding out that a run failed on "Invalid column
raw_contact_is_read_only" took reading the app's storage. The log line
now carries the report's outcome, its message (up to 300 characters)
and the number of failed items.

Checked: 55 JVM tests (`gradlew :app:testDebugUnitTest`, one new) and
the debug build.
2026-09-27 05:27:13 +02:00
Linus Rath 77fd494633 feat: turn on contacts and calendar sync to this device in settings #34
On Android, Settings > Contacts and Settings > Calendar get a "Sync to
this device" section with a row per signed-in account. This makes device
sync reachable:

- The switch asks for the contacts or calendar permission after a short
  explanation (denied: sync stays off and says why, with a link to the
  app's system settings once Android stops asking), creates the
  account's Bulwark account in Android and syncs right away. Turning it
  off uploads what the device changed first and asks before anything
  that could not go up is lost. The switch shows Android's own state, so
  sync turned off in Android Settings shows as paused.
- Which address books or calendars sync: the personal account's are on
  by default, shared ones and the Trusted Senders book off; read-only
  ones are marked, and a note says when another signed-in account
  already syncs the same shared one.
- The address book new contacts from other apps go to, who reminds of
  synced events (the calendar app or Bulwark, asked when calendar sync
  is turned on while Bulwark's reminders are on), and how often Android
  syncs on its own (every 15 minutes to every 6 hours, or not at all).
- "Sync now", the last sync with its result, or the error with what to
  do about it ("Sign in again", "Grant access", "Review deletions"), and
  a link to the account in Android Settings.
- Accounts the app signed out while they synced are listed with "Sign
  in again" and "Remove from this device".

The settings search finds both sections. iOS, and Android builds without
the native module, show nothing new.

Checked: the collection listing and status tests, the settings search
and i18n tests, `npm run typecheck`, `npm test` and `npm run i18n:check`
on this commit, and on the API 32 emulator with usera on the local
server: the section, turning contacts sync on (Android account created,
152 contacts on the device, one per card) and the status after a run.
2026-09-27 05:15:32 +02:00
Linus Rath 9d83b2fb62 fix: stop warning about shared accounts that share only mail #34
434c554 left an account that refuses to list its address books or
calendars out of the run, but reported it as an item error. The sync
settings then said "1 item could not be synced" after every run for an
account that merely shares a mailbox: nothing the user can fix, and no
item behind it. Such an account is now only logged.

Checked: the engine tests, `npm run typecheck` and `npm test` on this
commit, and the sync settings of usera on the API 32 emulator after a
new run.
2026-09-27 05:14:04 +02:00
Linus Rath a27d014c59 fix: ask before a sign-in link from another app signs in
Any web page (one tap) or installed app (no tap) could fire a
`bulwarkmail://pair` link at the app, which then silently added the
sender's mailbox and switched to it, so the user might send or save mail
into an attacker's account. A `connect` link likewise opened any
server's sign-in page.

- Links that arrive from outside the app now ask first, naming the
  webmail's host and whether the link adds an account. Cancel discards
  the link for good; a link delivered again while the question is open
  is dropped. Codes scanned or pasted inside the app still go straight
  through.
- With the account registry full, a link shows the limit instead of
  opening "Add account".
- A pairing link for a plain-http webmail says the connection would be
  unencrypted instead of "That code isn't a Bulwark sign-in code", or
  nothing at all for a tapped link.
2026-09-27 05:11:07 +02:00
Linus Rath 26bb947940 fix: keep the account limit from leaving the app signed in as the wrong account
Signing in connects the shared client to the new account and stores its
credentials before the account registry takes it. With ten accounts the
registry refused, but nothing undid the connect: every request then went
out as the new account while the app still showed the old one. A
sign-in link made this easy to reach, since it opened "Add account"
without the limit check the menus have.

- A failed registration now restores the previous connection and drops
  credentials the registry never took.
- Adding an account to a full registry is refused before a sign-in code
  is spent or a browser sign-in starts, with "Remove an account in
  Settings to add another one."
2026-09-27 05:10:50 +02:00
Linus Rath ad0f83c72f fix: stop reading the read-only flag the contacts provider never returns #34
The contacts planner read RAW_CONTACT_IS_READ_ONLY with every raw
contact, but ContactsProvider takes that column only on writes: every
projection and selection that names it fails with "Invalid column", on
API 32 and API 35 alike. Every contacts sync on a device therefore
failed before it wrote a row. The fake provider accepted the column, so
no test noticed.

The flag is now only written, along with any raw contact write that
happens anyway, so echoes stay write-free. Whether a contact's device
edits are put back is decided from the server's current rights on the
card, which is also right when only the rights changed. The fake now
refuses the column in queries and selections like the real providers,
and every other column the planners query was checked against the real
providers on both emulators.

Checked: the contacts, engine and fake provider tests, `npm run
typecheck` and `npm test` on this commit, and a contacts sync of usera
on the API 32 emulator, which now completes.
2026-09-27 05:07:58 +02:00
Linus Rath 434c554038 fix: keep syncing when a shared account refuses access to its address books or calendars #34
Stalwart lists an account shared with the user for mail only with every
capability, and answers `forbidden` when the user asks for its address
books or calendars. The engine treated that like any failed request, so
on the local test server every sync of usera ended with an I/O error and
never reached the device, only because userb shares a mailbox.

An account that refuses to list its collections (`forbidden`,
`accountNotFound`, `accountNotSupportedByMethod`) is now left out of the
run and reported as an item error; the other accounts sync as before.
The rows written for such an account stay on the device: a refusal can
pass, and dropping the account's contacts and events for one refused
request would throw away device edits that are still waiting.

Checked: three new engine tests (a shared account that never listed,
one that stops listing its address books, one that stops listing its
calendars), which fail without the fix, `npm run typecheck` and
`npm test` on this commit, and a contacts sync of usera on the API 32
emulator, which now ends `ok` with one item error.
2026-09-27 05:06:36 +02:00
Linus Rath 1c574a5a93 feat: leave reminders of synced events to the device's calendar app when asked #34
When an account's calendars sync to the device and the user chose the
device's calendar app to remind them, that app gets the events'
reminders. Bulwark's own reminder scheduler now skips the events of
those calendars, which would otherwise remind the user twice. Tasks
never sync and keep Bulwark's reminders, and a change of who reminds, or
of which calendars sync, reschedules right away.

Checked: the reminder filter and scheduler tests, `npm run typecheck`
and `npm test` on this commit.
2026-09-27 04:39:53 +02:00
Linus Rath eb8b33160c feat: remove synced contacts and calendars from the device on sign-out #34
Signing out of an account (or of all of them, or removing one) now first
uploads what the device changed in its synced contacts and calendars,
then deletes them and removes its Android account, while the credentials
still work. When changes could not be uploaded (offline, server errors)
the app asks before signing out and losing them; staying signed in
keeps the account syncing. Several accounts are released in parallel,
and an internal error never keeps anyone from signing out: the next
check against Android's accounts stops what was left from syncing.

A successful sign-in clears the account's "Sign in again" notification
and resumes an account that stopped syncing when it was signed out.

Checked: `npm run typecheck` and `npm test` on this commit (the release
and resume paths are covered by the lifecycle tests of e67e240).
2026-09-27 04:38:34 +02:00
Linus Rath e67e240840 feat: ask android to sync when contacts or calendars change #34
With device sync on, Android should hear about a change as soon as the
app does, not only at the next periodic sync. This adds what tells it:

- The app's own contact and calendar edits, StateChanges on the live
  stream for `AddressBook`/`ContactCard` and `Calendar`/`CalendarEvent`,
  and the app coming back to the foreground request a sync of that
  account and authority. Requests within 5 s fold into one, and a
  StateChange the engine caused itself (its own upload echoing back) is
  skipped.
- While the app is closed, the account's push subscription carries
  those types too. The FCM and UnifiedPush services hand every push to
  the native router from f743ab0, which requests the syncs, and start
  the mail push task only when a push has `EmailDelivery`; the task also
  returns early for such pushes, so a contact change can never show the
  "newest unread mail" notification.
- At launch, on foreground and when Android's accounts change, the app
  checks Android's accounts against its own: an account removed in
  Android Settings shows sync off and is not recreated, and one whose app
  account is gone stops syncing and posts "Sign in again", keeping its
  data.

The functions that turn sync on and off (`src/device-sync/app/lifecycle.ts`)
come with this; the settings that call them land in a later commit, so
nothing here runs before an account has device sync on.

Checked: the new store, lifecycle, trigger and push tests, `npm run
typecheck`, `npm test` and `npm run i18n:check` on this commit, and the
debug build.
2026-09-27 04:36:51 +02:00
Linus Rath a948738506 feat: map contacts and groups to android's contacts provider #34
This is the planner for contacts (`src/device-sync/contacts`), the last
piece the sync task was waiting for: it turns JSContact cards into raw
contacts and data rows, and device edits back into JMAP writes.

- Names, nicknames, organizations and titles, emails, phones, postal
  addresses, websites, birthdays and anniversaries (also without a
  year), relations, notes and photos map both ways, with their types and
  custom labels. Group cards become groups, and their members become
  group memberships.
- Device edits upload as patches of the fields whose rows changed; a
  whole map goes up only where Stalwart needs one. Whatever a card holds
  that Android can't show (pronouns, online services, keywords, a second
  note, an address's coordinates) is never touched, so an edit on the
  phone leaves it intact on the server.
- Rows are matched to their card's entries by the keys they carry and,
  where an editor re-inserted them without (Fossify saves that way), by
  their values. An entry the editor didn't show is never read as
  deleted, and the keys Stalwart assigns by position to cards from other
  clients are followed when they shift.
- Contacts in read-only address books are put back when the device
  changes them. Text beyond what the provider stores is cut on the device
  and never uploaded.

Checked: 151 contacts tests (mappings both ways, one test per edited
field, Fossify and AOSP editing, re-keyed cards, merges, groups,
read-only books, a full lifecycle), the engine's tests with both real
planners, and `npm run typecheck`, `npm test` and `npm run i18n:check`
on this commit.
2026-09-27 04:23:22 +02:00
Linus Rath 1131073323 feat: open sign-in links from the camera app and paste them into the app
The webmail's QR encodes a `bulwarkmail://pair?...` link, but nothing
registered that scheme: scanning it with the system camera or Google
Lens did nothing, and without a camera there was no way in at all.

- `bulwarkmail` is registered in `app.config.js` (iOS) and the committed
  AndroidManifest. Links are parked in a small store and picked up by
  the sign-in screen, or by "Add account" when already signed in, once
  per launch.
- "Paste sign-in link" on the sign-in screen (and next to the server
  step's scan hint) takes a link copied from the webmail.
- While pairing, the waiting screen names the webmail being paired with.
2026-09-27 04:23:01 +02:00
Linus Rath 59cfe50793 fix: say why a sign-in code failed and accept password pairing from the webmail
Every failed redeem read "That code has expired": a code already used,
a webmail without pairing (404), a rate limit, a server error and a
network failure all looked the same, and a non-JSON answer surfaced as a
raw parse error. The app now reads the webmail's `invalid_code`,
`expired_code` and `used_code` answers (webmail 1.11's catch-all 400
reads as "expired or already used"), gives up after 15 s, and names the
host involved; a failure after the code was redeemed says the phone was
signed in but could not reach the mail server.

- `loginViaPairing` accepts `flow: 'password'` bundles (an app password,
  or the account password from servers without OAuth) in normal and
  add-account mode, and never redeems the same code twice.
- A `pair` link must point at an https webmail (loopback http for
  development); `pair/?` is accepted too.
- A token refresh that answers 200 with a non-JSON body keeps the
  account instead of signing it out.
2026-09-27 04:22:33 +02:00
Linus Rath 3ed5fa53af feat: map calendars and events to android's calendar provider #34
The engine hands every event it syncs to a planner that turns it into
CalendarContract rows, and device edits back into JMAP writes. This is
that planner for calendars (`src/device-sync/calendar`), now used by the
sync task:

- Calendars become calendar rows with what the apps need (SYNC_EVENTS=1,
  an owner, colours, the reminders allowed); calendars of shared accounts
  carry the account's name.
- Events keep their time zone, floating and all-day times, recurrence
  rules, excluded and changed instances, reminders, attendees and the
  user's own reply. A rule Android can't hold (RSCALE, SKIP, leap months)
  stays on the server untouched, and local times Stalwart can't resolve
  (a start in a repeated hour, an UNTIL in a skipped one) are written in
  a form it keeps.
- Device edits upload as patches of what differs from the last synced
  state, never a whole event, and only what the device can show is ever
  sent. A single-instance edit or deletion becomes an override of its
  series; "this and following" caps the series and creates a new one.
  How Etar and Fossify save (a cancelled instance instead of an
  exclusion, attendee rows deleted and re-inserted) is read as the edit
  it stands for, not as a deletion. Invitations go out on the changes a
  calendar app would notify about.
- Events in read-only calendars and subscribed feeds are put back when
  the device changes them; only the user's own reply goes up, where the
  server allows it. Tasks, task-only calendars and the birthday calendar
  never reach the device.
- Reminders are written as Reminders rows when the calendar app is to
  remind the user, and left out when Bulwark does.

Checked: 179 calendar tests (download, upload, exceptions, the edits
Etar, Fossify and CONTENT_EXCEPTION_URI make, echoes that write nothing,
time zone and DST edges), and `npm run typecheck`, `npm test` and `npm
run i18n:check` on this commit.
2026-09-27 04:21:30 +02:00
Linus Rath 96a6b38321 feat: sync the device's contacts and calendars with the server in the background #34
The sync adapters from ed2525e start a headless task for every sync, but
the task only reported "disabled". This adds the engine it runs
(`src/device-sync/engine`, `src/device-sync/jmap`): for one Android
account and authority it downloads the server's changes, uploads what
the device changed, and hands a report back to SyncManager.

- Downloads follow `/changes` from the state kept in the provider's
  SyncState. Rows and the state they reach are written in one batch, so
  a killed run resumes where it stopped. Without a state, after
  `cannotCalculateChanges` or after the user discarded deletions, a full
  reconcile compares ids instead.
- Uploads go in three phases across accounts (creates, updates,
  deletions), so moving an item to another account never destroys it
  before the copy exists. A create carries a uid written to the device
  first, and the server is asked for that uid before creating, so a lost
  response or a killed run never makes a duplicate. Updates are patches
  guarded by `ifInState`; nothing is replaced whole.
- More than 50 deletions that are also more than a fifth of the synced
  items hold every deletion back and report `tooManyDeletions`, so
  Android asks the user; "undo" restores the items from the server.
- An item the server refuses is marked and not sent again until it
  changes or its back-off (1 hour, doubling to a day) ends. Such
  refusals and planner errors stay with their item; the rest of the run
  goes on.
- Runs of one account and authority take a lock, stop at checkpoints
  when cancelled or close to the deadline, and never throw: every
  failure becomes an outcome, and a rejected sign-in posts one "Sign in
  again" notification.
- The JMAP side uses a client of its own per app account, never the
  UI's, since a run can share the JS runtime with the app.

Turning sync off or signing out runs the same upload first
(`teardownAuthority`) and deletes the rows only once nothing is waiting,
or when the user agreed to lose it.

The planners that map items to rows land in the next commits; until
then a run fails before it touches a row.

Checked: 94 engine tests against the fake provider and JMAP server
(sync flows, failures, crashes and retries, teardown), and `npm run
typecheck`, `npm test` and `npm run i18n:check` on this commit.
2026-09-27 04:19:21 +02:00
Linus Rath f743ab0f7d feat: give device sync scoped access to the contacts and calendar providers #34
The engine plans its provider reads and writes in JS, but only native
code can reach ContactsProvider and CalendarProvider. This adds that
bridge to the BulwarkDeviceSync module, with the rest of what the engine
and the app need from it:

- `query`, `applyBatch`, `readSyncState` and `readPhoto` take the
  `ProviderQuery`/`ProviderOp[]` JSON of `src/device-sync/types.ts`,
  check it (`sync/ProviderOps.kt`) and run it for one Android account of
  our type (`sync/ProviderScope.kt`, `sync/ProviderIo.kt`). Every
  selection gets the account ANDed in, every row an op names by id or
  points to as a parent must be the account's, and every URI carries
  `caller_is_syncadapter`, so nothing the sync writes marks rows dirty or
  schedules an upload. The calls run one at a time on the module's own
  thread through an unstable provider client, so a long batch never
  stalls other module calls and a provider that dies fails the call
  instead of the app.
- `showSyncProblem` and `clearSyncProblem` post and clear the one
  notification per account that needs the user, e.g. "Sign in again".
- `setPushRoutes` stores which Android accounts each JMAP account feeds,
  for the push router in `sync/PushRouting.kt`; the push services start
  using it in a later commit.
- A run now ends exactly once and refuses a report that comes after
  that. A sync requested while a run is open makes the run ask for
  another sync when it ends, because SyncManager drops such requests as
  duplicates. `onPerformSync` lets nothing escape any more: an exception
  there killed the whole process, UI included.
- `ensureAccount` no longer hands an existing Android account to another
  app account (`conflict`): its rows and SyncState belong to the first.

Payloads cross as JSON strings, which the bridge moves much faster than
nested maps. `src/device-sync/native.ts` is the typed JS side. The fake
provider follows the rules the emulator runs turned up: contacts
`data1`..`data14` read back as text, events and calendars the sync
adapter inserts have DIRTY NULL, attendee rows are addressed by id, an
op by id expects one row, and a vanished parent fails with `assert`
while another account's fails with `scope`.

Checked: 54 JVM tests (`gradlew :app:testDebugUnitTest`), debug and
release (R8) builds, `npm run typecheck` and `npm test` on this commit,
and on the API 32 emulator through the debug receiver: inserts with
back-references and group memberships, SyncState, photo writes and
scaled reads, asserts and expected counts, both scope refusals, the
499-op yield boundary, `tooLarge` for a 1.3 MB batch, and the
notification.
2026-09-27 04:04:00 +02:00
Linus Rath 95c8324f12 fix: let contacts apps edit contacts of the bulwark account #34
AOSP Contacts, and the apps built on its account-type parser, check the
EditSchema in `xml/contacts.xml` and drop the whole account type when it
breaks one of their rules: every data kind with a type column needs
`<Type>` children, the other kinds must have none, and name, photo,
nickname, organization, note and group membership need maxOccurs="1".
The schema from ed2525e listed phone, email, postal, event and
relationship without types, so the editors would not have offered the
Bulwark account at all.

The kinds now list the types device sync maps to JSContact, and the
rules are written down in the file so the next edit keeps them.

Checked on the API 32 emulator (AOSP Contacts): with a Bulwark account
present, "Create new contact" offers it and opens the editor saving to
it.
2026-09-27 04:03:16 +02:00
Linus Rath 59fcf386b9 test: add a fake device provider and jmap server for the sync tests #34
The device sync engine reaches Android's providers through a native
bridge and the server through a detached JMAP client, and neither runs
in vitest. This adds the stand-ins the engine and the mappers are tested
against, and the contracts they implement:
- `src/device-sync/types.ts`, `planner.ts`, `wire.ts`,
  `android-columns.ts`: the bridge, the provider op model, the JMAP
  port, and the interface between the engine and the contacts and
  calendar planners
- a fake ContactsProvider and CalendarProvider with the behaviour the
  design relies on: the DIRTY and VERSION rules, soft and hard deletes,
  exception linking, normalisation, asserts, the yield-point limit, and
  a "user" face that edits the way AOSP Contacts, Fossify and Etar do
- a fake Stalwart with `/changes` paging and compaction,
  `cannotCalculateChanges`, `ifInState`, uid checks behind a lagging
  search index, positional re-keying and fault injection (lost
  responses, 429, auth, SetErrors)
- shared pure helpers (canonical JSON, PatchObjects with Stalwart's
  missing-parent rule, key and uid minting) and a test that walks the
  import graph, so no mapper or engine module pulls in React Native
- `src/stores/device-sync-store.ts`: the per-account preferences and
  last run status the engine and the settings will share

Checked: `npx vitest run src/device-sync` (91 tests), the full suite
(2209 tests) and `npm run typecheck`.
2026-09-27 02:53:22 +02:00
Linus Rath f66669964a docs: describe how contacts and calendars will sync with the device #34
Issue #34 asks for contacts and calendars that show up in Android's own
apps and whose edits there reach the server. docs/device-sync.md is the
design the implementation follows: Android's sync framework drives the
runs, a thin Kotlin shell starts a headless JS task, and the engine is
TypeScript on a detached JMAP client, so token refresh, mTLS and session
discovery keep a single owner. A pure-Kotlin engine was rejected: it
would have had to redeem rotating refresh tokens outside that owner and
re-implement the client.

It pins down what every provider column holds, the JSContact and
JSCalendar mapping tables, how device edits are told apart from provider
normalisation (a baseline per row), how rows are matched when an editor
re-inserts them, the merge and conflict rules, uploads as patches with
`ifInState`, the native API, the failure matrix and the limitations.

The platform and server facts it relies on come from AOSP's providers
and calendar/contacts apps, Fossify, Etar and Stalwart v0.16.23's
source. These were checked on the local test server:
- map keys written through JMAP survive patches; `uid` duplicates are
  refused; unknown and plural recurrence properties are rejected
- a start or `until` in a DST gap or overlap is silently dropped (a
  daily rule came back unbounded); `Etc/GMT-2` keeps both
- `ifInState` yields `stateMismatch`; `relatedTo/<key>: null` does not
  delete; result references to `/set` are refused

An adversarial review of the first draft found 20 paths that could lose
data, duplicate items or loop (own creates duplicated after a lost
response, unsynced edits dropped on deselection or eviction, Fossify
rewrites uploaded as edits, Etar moves turned into destroy + create,
and more); the design was changed for each before any engine code.
2026-09-27 02:49:46 +02:00
Linus Rath ed2525e5cb feat: register a bulwark account with android's contacts and calendar sync #34
Android syncs the contacts and calendars of other services through
accounts and sync adapters. This adds the Android side of device sync:
a "Bulwark" account type (`${applicationId}.account`, set in
build.gradle), an authenticator that sends Settings → Add account to
the app instead of asking for credentials, and one sync adapter each
for the contacts and the calendar provider.

An adapter does no syncing itself. It starts the BulwarkDeviceSync
headless JS task from the main thread, through the helper the push job
uses, waits for the task's report and hands it to SyncManager as the
SyncResult. The engine that will write the report runs in JS, so the
existing client stays the only owner of token refresh, mTLS and
session discovery; for now the task only reports that device sync is
off. Nothing in the app creates an account yet, so the feature stays
unreachable until its settings land. Debug builds get an adb hook for
the device tests (DeviceSyncDebugReceiver, shell only).

Hard errors are reported without progress: SyncManager retries a
failed sync at once when it made progress, which would loop an auth
failure.

Checked on the API 34 and API 32 emulators with the debug build, from
logcat and `dumpsys content`:
- app in the foreground: report after 319-334 ms
- app in the background: 320-327 ms
- process killed: SyncManager starts it for the sync service, React
  Native starts headless, and the report arrives after 16.2 s (API 34)
  and 8.5 s (API 32), most of it the debug bundle download from Metro
- an io report is retried with back-off and an auth report is not
  retried; too many deletions shows the system's "Attempted to delete
  too many Contacts." notification
- with Metro unreachable the adapter gives up after 2 s instead of
  waiting for its deadline
- `gradlew :app:testDebugUnitTest`: 10 JVM tests for the report mapping
2026-09-27 01:53:40 +02:00
Linus Rath 1f7acaf08a refactor: start headless tasks for push and device sync the same way
The code that starts React Native for a headless JS task, when the
process was woken without any UI, lived inside BulwarkPushJobService.
The device sync adapters need the same start, so it moves to a shared
HeadlessJs object, with a small startTask helper next to it.

Push behaviour does not change:
- onStartJob still calls the start code on the main thread, and the
  listener-then-ready-check order is the same code, moved verbatim
- the task still starts from `UiThreadUtil.runOnUiThread`, its id is
  still recorded in that same UI-thread turn, and a refusal because the
  app came to the foreground still logs and finishes the job
- the start code now returns React Native's start task, so a caller can
  notice a start that fails (a debug build with no Metro); the push job
  ignores it

The helper documents that it must run on the main thread: the ReactHost
is created on first use without a lock, so a background caller racing
MainActivity could create a second one.

Checked with `gradlew assembleDebug -PreactNativeArchitectures=x86_64`.
2026-09-27 01:19:17 +02:00
Linus Rath db07acc37c chore: update version number to 0.1.66 2026-09-27 00:34:53 +02:00
Linus Rath 39628d84dc fix: keep widget corners and border even when the launcher crops the widget 2026-09-27 00:34:29 +02:00
lucletoffeandClaude Opus 5.5 bd1b2f3fd3 fix(calendar): keep app-scheme meeting links and match service labels exactly
A virtual location in an app scheme (msteams:, zoommtg:) is kept as a last
resort instead of hiding the meeting row; script schemes never are. A
location only joins the meeting when the whole label is a service name, so
"Salle Teams" still opens the maps. Recognise Zoom's /wc/ join path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 09:59:58 +02:00
lucletoffeandClaude Opus 5.5 7c4b584d9c fix(calendar): join the meeting and open the location from the event sheet
Invitations - Teams above all - leave virtualLocations empty: LOCATION reads
"Microsoft Teams Meeting" and the join URL sits in the description between
help and dial-in links, so the sheet showed no meeting button. Find the
join URL of known conferencing services (Teams, Zoom, Meet, Webex, Jitsi,
Whereby, GoTo) in the location or description, unwrapping Outlook Safe
Links, and offer it as the meeting link. Nothing is written to the event.

The location was plain text. Tapping it now opens it: a URL opens as a
link, a location that only names the service joins the meeting, anything
else is searched in the maps app (geo: intent on Android, Google Maps
universal link elsewhere). Long-press copies it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 09:47:15 +02:00
Michael Brusegard e35cd82f5f fix(auth): verify external identity providers during handoff
Allow webmail-configured OAuth providers on separate hosts without trusting callback-supplied endpoints or broadly allowing sibling domains. Bind the client, JMAP server, issuer and token endpoint to HTTPS metadata from the selected webmail for both handoff and pairing.
2026-09-18 04:01:58 -07:00
586 changed files with 67328 additions and 4325 deletions
+3
View File
@@ -0,0 +1,3 @@
# patch-package patches must keep the patched files' line endings byte for byte
# (react-native-android-widget ships CRLF sources).
patches/*.patch -text
+11
View File
@@ -23,6 +23,8 @@ jobs:
timeout-minutes: 45
permissions:
contents: write
id-token: write
attestations: write
steps:
# Fail in seconds instead of after a 20-minute build. There is no
# unsigned or debug-signed fallback: anything signed with the debug key
@@ -249,6 +251,7 @@ jobs:
SRC=android/app/build/outputs/apk/release/app-release.apk
DEST="bulwark-mobile-${VERSION}-${GITHUB_SHA::7}.apk"
cp "$SRC" "$DEST"
sha256sum "$DEST" > "$DEST.sha256"
MAPPING_SRC=android/app/build/outputs/mapping/release/mapping.txt
MAPPING=""
@@ -266,10 +269,18 @@ jobs:
{
echo "files<<FILES_EOF"
echo "$DEST"
echo "$DEST.sha256"
if [ -n "$MAPPING" ]; then echo "$MAPPING"; fi
echo "FILES_EOF"
} >> "$GITHUB_OUTPUT"
# Ties the APK to this commit and workflow run, which the signing key
# alone does not: `gh attestation verify <apk> --repo bulwarkmail/native`.
- name: Attest build provenance
uses: actions/attest@v4
with:
subject-path: ${{ steps.artifact.outputs.path }}
- uses: actions/upload-artifact@v4
with:
name: ${{ steps.artifact.outputs.path }}
+35
View File
@@ -25,6 +25,9 @@ jobs:
timeout-minutes: 90
permissions:
contents: write
outputs:
version: ${{ steps.export.outputs.version }}
uploaded: ${{ steps.upload.outputs.done }}
env:
EXPO_NO_TELEMETRY: "1"
# App Store Connect refuses a CFBundleVersion it has already seen for this
@@ -254,6 +257,7 @@ jobs:
if-no-files-found: error
- name: Upload to TestFlight
id: upload
# The IPA is not attached to the GitHub release: without an Apple device
# provisioned for it, it is not installable, and TestFlight is the only
# distribution channel for it.
@@ -271,4 +275,35 @@ jobs:
--apiKey "$ASC_KEY_ID" --apiIssuer "$ASC_ISSUER_ID"
xcrun altool --upload-app --type ios --file "$IPA" \
--apiKey "$ASC_KEY_ID" --apiIssuer "$ASC_ISSUER_ID"
echo "done=true" >> "$GITHUB_OUTPUT"
echo "::notice::Uploaded build ${IOS_BUILD_NUMBER} to App Store Connect. Processing usually takes 5-30 minutes before it appears in TestFlight."
# Puts the upload on the public TestFlight link (README): waits for App Store
# Connect to process it, adds it to the external group and submits it for
# beta review. On Linux, because most of this job is waiting.
public-testflight:
needs: build
if: needs.build.outputs.uploaded == 'true'
runs-on: ubuntu-latest
timeout-minutes: 75
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Install the App Store Connect API key
env:
ASC_KEY_BASE64: ${{ secrets.APP_STORE_CONNECT_PRIVATE_KEY }}
ASC_KEY_ID: ${{ secrets.APP_STORE_CONNECT_KEY_ID }}
run: |
mkdir -p ~/private_keys
echo "$ASC_KEY_BASE64" | base64 -d > ~/private_keys/AuthKey_${ASC_KEY_ID}.p8
chmod 600 ~/private_keys/AuthKey_${ASC_KEY_ID}.p8
- name: Add the build to the public group and submit it for beta review
env:
ASC_KEY_ID: ${{ secrets.APP_STORE_CONNECT_KEY_ID }}
ASC_ISSUER_ID: ${{ secrets.APP_STORE_CONNECT_ISSUER_ID }}
run: node scripts/testflight-public.mjs "${{ needs.build.outputs.version }}" "${{ github.run_number }}"
+100 -37
View File
@@ -1,13 +1,14 @@
import React from 'react';
import { ActivityIndicator, AppState, Linking, StyleSheet, Text, View } from 'react-native';
import { StatusBar } from 'expo-status-bar';
import { useSafeAreaInsets } from 'react-native-safe-area-context';
import { useColorScheme } from 'react-native';
import {
DarkTheme, DefaultTheme, NavigationContainer, createNavigationContainerRef, useIsFocused,
} from '@react-navigation/native';
import { createBottomTabNavigator } from '@react-navigation/bottom-tabs';
import { createNativeStackNavigator, type NativeStackScreenProps } from '@react-navigation/native-stack';
import { Mail, Calendar, BookUser, HardDrive, Settings } from 'lucide-react-native';
import { Mail, Calendar, BookUser, HardDrive, Settings } from './src/components/icons';
import { startLiveUpdates, type LiveUpdatesHandle } from './src/api/push-stream';
import { jmapClient } from './src/api/jmap-client';
@@ -18,6 +19,7 @@ import {
startCalendarReminderTapHandling,
} from './src/lib/calendar-notifications';
import { startLivenessMonitor } from './src/lib/connection-liveness';
import { startDeviceSyncTriggers } from './src/device-sync/app/triggers';
import {
setPendingCalendarOpen,
type CalendarReminderTarget,
@@ -42,10 +44,10 @@ import ComposeScreen from './src/screens/ComposeScreen';
import EmailThreadScreen from './src/screens/EmailThreadScreen';
import EmailSourceScreen from './src/screens/EmailSourceScreen';
import LoginScreen from './src/screens/LoginScreen';
import EmailListScreen from './src/screens/EmailListScreen';
import MailTab from './src/screens/MailTab';
import FilesScreen from './src/screens/FilesScreen';
import CalendarScreen from './src/screens/CalendarScreen';
import ContactsScreen from './src/screens/ContactsScreen';
import ContactsTab from './src/screens/ContactsTab';
import ContactDetailScreen from './src/screens/ContactDetailScreen';
import ContactFormScreen from './src/screens/ContactFormScreen';
import GroupDetailScreen from './src/screens/GroupDetailScreen';
@@ -56,12 +58,12 @@ import { useAccountStore } from './src/stores/account-store';
import { useAuthStore } from './src/stores/auth-store';
import { useCalendarStore } from './src/stores/calendar-store';
import { useContactsStore } from './src/stores/contacts-store';
import { useEmailStore, viewerParamsForRow } from './src/stores/email-store';
import { mailboxAccountId } from './src/lib/mailbox-tree';
import { useEmailStore } from './src/stores/email-store';
import { loadDetail, prefetchMessage } from './src/lib/email-detail-cache';
import { useHasCalendar, useHasContacts, useHasFiles } from './src/lib/capabilities';
import { useSettingsStore } from './src/stores/settings-store';
import { useLocaleStore } from './src/stores/locale-store';
import { useLayout } from './src/lib/use-layout';
import { toast } from './src/stores/toast-store';
import { useNetworkStore } from './src/stores/network-store';
import { useUpdatesStore } from './src/stores/updates-store';
@@ -73,7 +75,16 @@ import { AppIconBadge } from './src/components/AppIconBadge';
import { getEmails } from './src/api/email';
import { signOutWidgets, startWidgetSync } from './src/widgets/sync';
import { draftContextFromEmail } from './src/lib/draft-context';
import { handleDeepLink, parseDeepLink, shareToDeepLink, type DeepLink } from './src/navigation/linking';
import {
acceptSignInLink,
handleDeepLink,
parseDeepLink,
routeParkedSignInLink,
shareToDeepLink,
type DeepLink,
} from './src/navigation/linking';
import { usePendingSignInLinkStore } from './src/navigation/pending-sign-in-link';
import { MAX_ACCOUNTS } from './src/lib/account-utils';
import { addShareListener, getInitialShare, shareAttachments } from './src/lib/share-intent';
import { OfflineCacheBanner } from './src/components/OfflineCacheBanner';
import { useOfflineCacheStore } from './src/stores/offline-cache-store';
@@ -86,6 +97,10 @@ const Stack = createNativeStackNavigator<RootStackParamList>();
const Tab = createBottomTabNavigator<MainTabsParamList>();
const navigationRef = createNavigationContainerRef<RootStackParamList>();
// The launch URL is read once per process: Android hands the same intent back
// on every getInitialURL(), and a sign-in link must not run twice.
let initialSignInLinkRead = false;
async function navigateToNotificationTap(payload: NotificationTapPayload): Promise<void> {
if (!navigationRef.isReady()) return;
@@ -105,9 +120,11 @@ async function navigateToNotificationTap(payload: NotificationTapPayload): Promi
navigationRef.navigate('EmailThread', {
emailId: payload.emailId,
threadId: payload.threadId,
emailIds: [payload.emailId],
subject: payload.subject,
// A group mailbox's message lives under another JMAP account (#839).
jmapAccountId: notificationTapJmapAccountId(payload),
...(payload.action === 'reply' ? { action: 'reply' as const } : {}),
});
}
@@ -227,9 +244,15 @@ function MainTabsNavigator({ navigation }: NativeStackScreenProps<RootStackParam
settings: t('sidebar.settings', 'Settings'),
};
const unavailable = (name: string) => t('sidebar.tab_unavailable', '{name} (unavailable)', { name });
// Side by side layouts move the tabs into a rail on the left, like the
// webmail's navigation rail.
const { isSplit } = useLayout();
// A bottom tab bar keeps the screens clear of the gesture bar or taskbar;
// with the tabs in a side rail the whole tab area has to.
const bottomInset = useSafeAreaInsets().bottom;
return (
<View style={{ flex: 1, backgroundColor: c.background }}>
<View style={{ flex: 1, backgroundColor: c.background, paddingBottom: isSplit ? bottomInset : 0 }}>
<UpdateBanner />
<OfflineCacheBanner />
<PushOnboardingPrompt />
@@ -239,9 +262,21 @@ function MainTabsNavigator({ navigation }: NativeStackScreenProps<RootStackParam
headerShown: false,
// Tabs out of view stop re-rendering until they are shown again.
freezeOnBlur: true,
tabBarPosition: isSplit ? 'left' : 'bottom',
tabBarVariant: isSplit ? 'material' : 'uikit',
// A narrow rail (icon over label), not a wide sidebar with labels
// beside the icons.
...(isSplit ? { tabBarLabelPosition: 'below-icon' as const } : null),
tabBarActiveTintColor: c.text,
tabBarInactiveTintColor: c.textSecondary,
tabBarStyle: {
tabBarStyle: isSplit ? {
backgroundColor: c.secondary,
borderRightColor: c.border,
borderRightWidth: 1,
elevation: 0,
shadowOpacity: 0,
shadowColor: 'transparent',
} : {
backgroundColor: c.background,
borderTopColor: c.border,
borderTopWidth: 1,
@@ -275,30 +310,7 @@ function MainTabsNavigator({ navigation }: NativeStackScreenProps<RootStackParam
},
}}
>
{() => (
<EmailListScreen
onComposePress={() => navigation.navigate('Compose')}
onEmailPress={(email) => {
// The list holds the open folder's mail: name its account so the
// viewer never has to guess it from whatever folder is open.
const { mailboxes: all, currentMailboxId } = useEmailStore.getState();
const target = {
jmapAccountId: mailboxAccountId(all, currentMailboxId),
// A row of an "All folders" list or a tag view names its own
// account instead, and pages over that account's rows.
...viewerParamsForRow(email),
};
// Start on the body now: mounting the viewer takes a while.
prefetchMessage(email, target.jmapAccountId);
navigation.navigate('EmailThread', {
emailId: email.id,
threadId: email.threadId,
subject: email.subject,
...target,
});
}}
/>
)}
{() => <MailTab navigation={navigation} />}
</Tab.Screen>
<Tab.Screen
name="Calendar"
@@ -317,7 +329,7 @@ function MainTabsNavigator({ navigation }: NativeStackScreenProps<RootStackParam
/>
<Tab.Screen
name="Contacts"
component={ContactsScreen}
component={ContactsTab}
options={{
title: tabLabels.contacts,
tabBarIcon: ({ color, size }) => <BookUser size={size} color={color} />,
@@ -386,6 +398,7 @@ export default function App() {
// cached mail list instead of the "Restoring session" spinner; the real
// JMAP session comes up in the background.
const hasPersistedAccount = useAccountStore((state) => state.activeAccountId != null);
const { isSplit } = useLayout();
React.useEffect(() => {
if (!hasRestoredSession) {
@@ -399,6 +412,9 @@ export default function App() {
// Calendar reminders are kept scheduled from launch, not only once the
// Calendar tab has been opened.
startCalendarNotificationSync();
// Device sync (Android, #34): live changes, the app's own edits and the
// foreground ask Android to sync; accounts are checked against Android's.
startDeviceSyncTriggers();
return useNetworkStore.getState().init();
}, []);
@@ -566,6 +582,52 @@ export default function App() {
};
}, [isAuthenticated]);
// Sign-in links (`bulwarkmail://pair?…` from the webmail's Link Mobile App,
// `bulwarkmail://connect?…`) work signed out too, so unlike the links above
// they are listened for from launch. They are parked for the login screen
// (the signed-out one takes them as it mounts), marked as coming from
// outside the app: any page or app can fire one, so the login screen asks
// before it redeems or opens anything.
React.useEffect(() => {
if (!initialSignInLinkRead) {
initialSignInLinkRead = true;
void Linking.getInitialURL().then((url) => { acceptSignInLink(url); }).catch(() => undefined);
}
const subscription = Linking.addEventListener('url', ({ url }) => { acceptSignInLink(url); });
return () => subscription.remove();
}, []);
// Signed in, a sign-in link adds an account: open Add account, whose login
// screen takes the parked link (or says why a refused one won't be used).
// On a cold start the navigator mounts right after the auth gate flips, so
// wait for it like a reminder tap does.
const signInLinkPending = usePendingSignInLinkStore((s) => s.pending !== null || s.refusal !== null);
const addAccountForLink = isAuthenticated && signInLinkPending;
React.useEffect(() => {
if (!addAccountForLink) return;
let cancelled = false;
void (async () => {
for (let attempt = 0; attempt < 50 && !navigationRef.isReady(); attempt++) {
await new Promise((resolve) => setTimeout(resolve, 100));
if (cancelled) return;
}
if (cancelled || !navigationRef.isReady()) return;
const route = routeParkedSignInLink(useAccountStore.getState().accounts.length);
if (route === 'account-limit') {
// Said like the Add account buttons say it, instead of opening a
// screen whose sign-in can only fail.
const t = useLocaleStore.getState().t;
toast.error(
t('settings.account.accounts.limit', 'Maximum of {count} accounts reached.', { count: MAX_ACCOUNTS }),
t('login.mobile.err_account_limit_detail', 'Remove an account in Settings to add another one.'),
);
return;
}
if (route === 'add-account') navigationRef.navigate('AddAccount');
})();
return () => { cancelled = true; };
}, [addAccountForLink]);
// Re-register the device with the configured relay once authenticated,
// and whenever the FCM token rotates. Honours the user's notification
// preference - flipping it off tears down THIS account's subscription so
@@ -767,10 +829,11 @@ export default function App() {
<Stack.Screen
name="Compose"
component={ComposeScreen}
options={{
presentation: 'modal',
animation: 'slide_from_bottom',
}}
// A tablet shows the composer as a panel over the screen it came
// from, so that screen stays drawn behind the dimmed backdrop.
options={isSplit
? { presentation: 'transparentModal', animation: 'fade' }
: { presentation: 'modal', animation: 'slide_from_bottom' }}
/>
<Stack.Screen name="ContactDetail" component={ContactDetailScreen} />
<Stack.Screen
+61
View File
@@ -1,3 +1,64 @@
Bulwark Mobile
Copyright (C) 2026 Bulwark Project Authors
This project is licensed under the GNU Affero General Public License,
version 3 only, with the additional permission for app store
distribution below. That permission is provisional; see the
reservation at its end.
Additional permission under GNU AGPL version 3 section 7
The copyright holders of this Program give you additional permission to
convey a covered work (this Program, a modified version of it, or a work
that contains parts of it) through an app store, even though the app
store's terms of service, usage rules or technical measures place
restrictions on recipients that section 10 of this License would otherwise
not allow, such as a limit on the number of devices a copy may be installed
on, or digital rights management applied to the copy.
This permission applies only if you:
a) make the Corresponding Source of each version you convey through the
app store available, at no charge and under the terms of this
License, from a network server that anyone can access;
b) state, in the app store listing or inside the application, where
that Corresponding Source can be obtained; and
c) do not impose any further restrictions of your own on the exercise
of the rights granted under this License.
"App store" means a digital distribution service for applications that is
operated by a third party, such as the Apple App Store and TestFlight,
Google Play, the Microsoft Store, the Samsung Galaxy Store or the Huawei
AppGallery.
This permission covers only the parts of a covered work that its copyright
holders license under this License; it does not extend to parts licensed
under other terms. As section 7 of this License allows, you may remove this
permission from copies you convey and from your modified versions.
Reservation
This additional permission has not been finalized. The contributors of
earlier parts of this Program have been asked whether they agree to it:
https://github.com/orgs/bulwarkmail/discussions/1113
Until that request is settled, the permission is granted subject to this
reservation:
- It applies to all contributions made on or after 30 September 2026.
These contributions are accepted under this License together with
this permission.
- It applies to parts of this Program written before that date only
where their copyright holders have agreed to it. Parts whose
copyright holders do not agree will be rewritten or removed.
This reservation will be removed once every earlier contributor has
agreed or their parts have been replaced.
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
+1 -1
View File
@@ -153,7 +153,7 @@ Open split them by tick; the P columns count the priority tags on those items
| #47 Account security not accessible | [01](docs/parity/01-auth-accounts.md) capability map bug (P1) |
| #46 email preview shows css | [03](docs/parity/03-email-viewer.md) HTML-only body detection (P1) |
| #45 error when re-registering to relay | [08](docs/parity/08-settings-push-i18n-ui.md) push section |
| #34 Native contacts/calendar sync adapters | feature request, out of parity scope; noted in [06](docs/parity/06-contacts.md) |
| #34 Native contacts/calendar sync adapters | done on Android: Settings → Contacts / Calendar → "Sync to this device" syncs both ways with the device's Contacts and Calendar apps ([design](docs/device-sync.md)); iOS has no API for it |
| #5 Sort ascending/descending | in progress in the working tree; ordering presets in [02](docs/parity/02-mail-list-folders.md) |
| #3 TLS client auth | Android-only today; gaps in [01](docs/parity/01-auth-accounts.md) and [09](docs/parity/09-jmap-core-sync-security.md) |
| #1 Shared settings between webmail and native | [08](docs/parity/08-settings-push-i18n-ui.md) — webmail sync is server-side/cookie-bound; a JMAP-blob design is sketched there |
+9 -3
View File
@@ -14,6 +14,11 @@
React Native (Expo SDK 54) client for [Bulwark Webmail](https://github.com/bulwarkmail/webmail) - a JMAP-based mail, calendar, and contacts app.
## Try it
- **iPhone and iPad:** join the beta on TestFlight: https://testflight.apple.com/join/rumcjNSp. A new release shows up there once Apple has reviewed it, usually within a day.
- **Android:** download the APK from the [latest release](https://github.com/bulwarkmail/native/releases/latest).
## What works today
- Sign in to any JMAP server (e.g. Stalwart)
@@ -22,14 +27,15 @@ React Native (Expo SDK 54) client for [Bulwark Webmail](https://github.com/bulwa
- Calendar (basic)
- Contacts (basic)
- Push notifications via the Bulwark relay - FCM by default, or [UnifiedPush](https://unifiedpush.org) (e.g. ntfy) for devices without Google Play services
- Android: contacts and calendars sync both ways with the phone's Contacts and Calendar apps (Settings → Contacts / Calendar → Sync to this device)
- In-app sideload updates from GitHub Releases
## What's missing or rough
- iOS builds, but push notifications and client certificates are Android-only so far
- iOS builds, but push notifications, client certificates and syncing with the phone's contacts and calendars are Android-only so far
- S/MIME, plugins, themes - UI stubs only (filters & rules, the vacation responder and file storage are real implementations)
- Calendar editing is partial; contacts editing is basic
- No Play Store distribution yet (sideload APK from Releases); iOS ships via TestFlight
- No Play Store or App Store listing yet: Android is a sideloaded APK, iOS a TestFlight beta (see [Try it](#try-it))
## Run locally
@@ -45,4 +51,4 @@ For iOS builds and TestFlight distribution see [docs/ios-release.md](docs/ios-re
## License
AGPL-3.0-only. See [LICENSE](LICENSE).
AGPL-3.0-only, with an additional permission to distribute the app through app stores such as the Apple App Store and Google Play. See [LICENSE](LICENSE). Contributions are accepted under the same terms. The permission is provisional until every earlier contributor has agreed to it ([consent request](https://github.com/orgs/bulwarkmail/discussions/1113)); contributions made since 30 September 2026 are already covered.
+1 -1
View File
@@ -1 +1 @@
0.1.65
0.1.81
+10
View File
@@ -154,6 +154,11 @@ android {
versionCode appVersionCode.toInteger()
versionName appVersionName
// Android account type of device sync (#34), derived from the package
// name so a later rename needs no other change. The XML in res/xml and
// the Kotlin in sync/ read it as @string/device_sync_account_type.
resValue "string", "device_sync_account_type", "${applicationId}.account"
buildConfigField "String", "REACT_NATIVE_RELEASE_LEVEL", "\"${findProperty('reactNativeReleaseLevel') ?: 'stable'}\""
}
signingConfigs {
@@ -280,6 +285,11 @@ dependencies {
// Kotlin 2.1 toolchain; 3.3.5 went back to 2.2 which it can read.
implementation("org.unifiedpush.android:connector:3.3.5")
// JVM unit tests for the logic in the Kotlin shell of device sync (src/test).
// android.jar only stubs org.json, so the tests use the real one.
testImplementation("junit:junit:4.13.2")
testImplementation("org.json:json:20240303")
def isGifEnabled = (findProperty('expo.gif.enabled') ?: "") == "true";
def isWebpEnabled = (findProperty('expo.webp.enabled') ?: "") == "true";
def isWebpAnimatedEnabled = (findProperty('expo.webp.animated') ?: "") == "true";
+4 -1
View File
@@ -3,5 +3,8 @@
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>
<application android:usesCleartextTraffic="true" tools:targetApi="28" tools:ignore="GoogleAppIndexingWarning" tools:replace="android:usesCleartextTraffic" />
<application android:usesCleartextTraffic="true" tools:targetApi="28" tools:ignore="GoogleAppIndexingWarning" tools:replace="android:usesCleartextTraffic">
<!-- adb hooks for the device sync tests (#34). DUMP: only the shell may send them. -->
<receiver android:name=".sync.DeviceSyncDebugReceiver" android:exported="true" android:permission="android.permission.DUMP" />
</application>
</manifest>
@@ -0,0 +1,108 @@
package com.anonymous.bulwarkmobile.sync
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.util.Base64
import android.util.Log
import java.io.File
import kotlin.concurrent.thread
/**
* Debug builds only: lets adb set up device sync accounts and drive the
* provider bridge without the settings UI or JS, for the device tests in
* docs/device-sync.md ("Testing"). Protected by android.permission.DUMP,
* which only the shell and the system hold. Syncs themselves are triggered
* with `adb shell requestsync`.
*
* adb shell am broadcast -n com.anonymous.bulwarkmobile/.sync.DeviceSyncDebugReceiver \
* --es cmd ensure --es name usera@example.org --es registry usera@example.org@10.0.2.2 \
* --es enable contacts,calendar
*
* Commands: `ensure` (name, registry, optional enable), `enable` (name,
* authority, optional --ez on false), `remove` (name), `list`, and the
* provider calls `query` and `batch` (name, authority, and the
* `ProviderQuery`/`ProviderOp[]` JSON base64-encoded in `b64`, which spares
* the shell's quoting, or pushed to the app's external files directory and
* named by `file` when it is too long for a command line), `syncstate`
* (name, authority) and `photo` (name, `--el id`, optional `--ei max`); and
* the sync-problem notification, `problem` (name, title, text, uri, optional
* channel) and `clearproblem` (name). Results go to logcat (tag
* BulwarkDeviceSync) and the broadcast's result data.
*/
class DeviceSyncDebugReceiver : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
val pending = goAsync()
// Provider calls block; keep them off the main thread.
thread(name = "DeviceSyncDebugReceiver") {
val result = try {
run(context, intent)
} catch (e: Exception) {
"error: ${e.javaClass.simpleName}: ${e.message}"
}
Log.i(TAG, "debug command: $result")
pending.resultData = result
pending.finish()
}
}
private fun run(context: Context, intent: Intent): String {
val name = intent.getStringExtra("name")
return when (intent.getStringExtra("cmd")) {
"ensure" -> {
val created = DeviceSyncAccounts.ensure(context, name!!, intent.getStringExtra("registry")!!)
for (which in intent.getStringExtra("enable").orEmpty().split(',').filter { it.isNotBlank() }) {
DeviceSyncAccounts.setSyncEnabled(context, name, authority(which), true)
}
"created=$created"
}
"enable" -> {
val on = intent.getBooleanExtra("on", true)
DeviceSyncAccounts.setSyncEnabled(context, name!!, authority(intent.getStringExtra("authority")!!), on)
"enabled=$on"
}
"remove" -> "removed=${DeviceSyncAccounts.remove(context, name!!)}"
"list" -> DeviceSyncAccounts.list(context).joinToString(";") { (account, registryId) -> "${account.name}=$registryId" }
"query" -> io(context, intent) { it.query(json(context, intent)) }
"batch" -> io(context, intent) { it.applyBatch(json(context, intent)) }
"syncstate" -> io(context, intent) { it.readSyncState().toString() }
"photo" -> ProviderIo(context, name!!, DeviceSyncAccounts.CONTACTS_AUTHORITY).use {
it.readPhoto(intent.getLongExtra("id", -1), intent.getIntExtra("max", 512)).toString()
}
"problem" -> {
SyncProblemNotifier.show(
context,
name!!,
intent.getStringExtra("title").orEmpty(),
intent.getStringExtra("text").orEmpty(),
intent.getStringExtra("uri").orEmpty(),
intent.getStringExtra("channel"),
)
"shown"
}
"clearproblem" -> {
SyncProblemNotifier.clear(context, name!!)
"cleared"
}
else -> "unknown cmd"
}
}
private fun io(context: Context, intent: Intent, call: (ProviderIo) -> String): String =
ProviderIo(context, intent.getStringExtra("name")!!, authority(intent.getStringExtra("authority")!!)).use(call)
private fun json(context: Context, intent: Intent): String {
intent.getStringExtra("file")?.let { return File(context.getExternalFilesDir(null), it).readText() }
return String(Base64.decode(intent.getStringExtra("b64")!!, Base64.DEFAULT), Charsets.UTF_8)
}
private fun authority(which: String) = when (which) {
"contacts" -> DeviceSyncAccounts.CONTACTS_AUTHORITY
"calendar" -> DeviceSyncAccounts.CALENDAR_AUTHORITY
else -> which
}
companion object {
private const val TAG = "BulwarkDeviceSync"
}
}
+40 -1
View File
@@ -6,6 +6,14 @@
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" android:maxSdkVersion="32"/>
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES"/>
<uses-permission android:name="android.permission.VIBRATE"/>
<!-- Device sync (#34): contacts and calendars in Android's providers, driven by the sync framework. -->
<uses-permission android:name="android.permission.READ_CONTACTS"/>
<uses-permission android:name="android.permission.WRITE_CONTACTS"/>
<uses-permission android:name="android.permission.READ_CALENDAR"/>
<uses-permission android:name="android.permission.WRITE_CALENDAR"/>
<uses-permission android:name="android.permission.READ_SYNC_SETTINGS"/>
<uses-permission android:name="android.permission.WRITE_SYNC_SETTINGS"/>
<uses-permission android:name="android.permission.READ_SYNC_STATS"/>
<queries>
<intent>
<action android:name="android.intent.action.VIEW"/>
@@ -17,7 +25,7 @@
<meta-data android:name="expo.modules.updates.ENABLED" android:value="false"/>
<meta-data android:name="expo.modules.updates.EXPO_UPDATES_CHECK_ON_LAUNCH" android:value="ALWAYS"/>
<meta-data android:name="expo.modules.updates.EXPO_UPDATES_LAUNCH_WAIT_MS" android:value="0"/>
<activity android:name=".MainActivity" android:configChanges="keyboard|keyboardHidden|orientation|screenSize|screenLayout|uiMode" android:launchMode="singleTask" android:windowSoftInputMode="adjustResize" android:theme="@style/Theme.App.SplashScreen" android:exported="true" android:screenOrientation="portrait">
<activity android:name=".MainActivity" android:configChanges="keyboard|keyboardHidden|orientation|screenSize|smallestScreenSize|screenLayout|uiMode" android:launchMode="singleTask" android:windowSoftInputMode="adjustResize" android:theme="@style/Theme.App.SplashScreen" android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
@@ -28,6 +36,13 @@
<category android:name="android.intent.category.BROWSABLE"/>
<data android:scheme="bulwarkmobile"/>
</intent-filter>
<!-- Sign-in links from the webmail (bulwarkmail://pair?..., bulwarkmail://connect?...). -->
<intent-filter>
<action android:name="android.intent.action.VIEW"/>
<category android:name="android.intent.category.DEFAULT"/>
<category android:name="android.intent.category.BROWSABLE"/>
<data android:scheme="bulwarkmail"/>
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.VIEW"/>
<action android:name="android.intent.action.SENDTO"/>
@@ -59,13 +74,37 @@
</intent-filter>
</service>
<service android:name=".BulwarkPushTaskService" android:exported="false"/>
<service android:name=".BulwarkWidgetRefreshService" android:exported="false"/>
<service android:name=".BulwarkPushJobService" android:permission="android.permission.BIND_JOB_SERVICE" android:exported="false"/>
<receiver android:name=".BulwarkNotificationActionReceiver" android:exported="false"/>
<service android:name=".BulwarkUnifiedPushService" android:exported="false">
<intent-filter>
<action android:name="org.unifiedpush.android.connector.PUSH_EVENT"/>
</intent-filter>
</service>
<meta-data android:name="com.google.firebase.messaging.default_notification_channel_id" android:value="bulwark_mail"/>
<!-- Device sync (#34), see docs/device-sync.md. The system binds these, so they are exported.
They run in the app's main process on purpose: the adapters start the sync engine as a
headless JS task in the same React Native runtime as the UI. -->
<service android:name=".sync.BulwarkAuthenticatorService" android:exported="true">
<intent-filter>
<action android:name="android.accounts.AccountAuthenticator"/>
</intent-filter>
<meta-data android:name="android.accounts.AccountAuthenticator" android:resource="@xml/authenticator"/>
</service>
<service android:name=".sync.ContactsSyncService" android:exported="true">
<intent-filter>
<action android:name="android.content.SyncAdapter"/>
</intent-filter>
<meta-data android:name="android.content.SyncAdapter" android:resource="@xml/sync_contacts"/>
<meta-data android:name="android.provider.CONTACTS_STRUCTURE" android:resource="@xml/contacts"/>
</service>
<service android:name=".sync.CalendarSyncService" android:exported="true">
<intent-filter>
<action android:name="android.content.SyncAdapter"/>
</intent-filter>
<meta-data android:name="android.content.SyncAdapter" android:resource="@xml/sync_calendar"/>
</service>
<!-- BEGIN WIDGETS: generated by scripts/sync-android-widgets.mjs, do not edit by hand -->
<service android:name="com.reactnativeandroidwidget.RNWidgetCollectionService" android:permission="android.permission.BIND_REMOTEVIEWS"/>
<receiver android:name=".widget.InboxWidget" android:exported="false" android:label="@string/widgetprovider_inboxwidget_label">
@@ -4,6 +4,7 @@ import android.app.NotificationManager
import android.app.PendingIntent
import android.content.Context
import android.content.Intent
import android.net.Uri
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.Canvas
@@ -72,14 +73,20 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
val groupTitle = options.takeIf { it.hasKey("groupTitle") }?.getString("groupTitle")
?: accountId ?: "Bulwark Mail"
val preview = options.takeIf { it.hasKey("preview") }?.getString("preview")
val markReadLabel = options.takeIf { it.hasKey("markReadLabel") }?.getString("markReadLabel") ?: "Mark as read"
val deleteLabel = options.takeIf { it.hasKey("deleteLabel") }?.getString("deleteLabel") ?: "Delete"
val replyLabel = options.takeIf { it.hasKey("replyLabel") }?.getString("replyLabel") ?: "Reply"
// Bitmap fetch + draw off the bridge thread so the caller doesn't
// block waiting for the favicon request.
thread(name = "bulwark-notification") {
val largeIcon = iconUrl?.let { fetchBitmap(it) }
?: makeLetterAvatar(initials, bgColorHex)
postNotification(
notificationId, title, body, largeIcon, bgColorHex,
notificationId, title, body, preview, largeIcon, bgColorHex,
emailId, threadId, subject, accountId, jmapAccountId, groupKey,
groupTitle, markReadLabel, deleteLabel, replyLabel,
)
if (groupKey != null) postGroupSummary(groupKey, groupTitle, bgColorHex, accountId)
promise.resolve(null)
@@ -102,6 +109,7 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
notificationId: String,
title: String,
body: String,
preview: String?,
largeIcon: Bitmap,
colorHex: String,
emailId: String?,
@@ -110,10 +118,15 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
accountId: String?,
jmapAccountId: String?,
groupKey: String?,
groupTitle: String?,
markReadLabel: String,
deleteLabel: String,
replyLabel: String,
) {
val ctx = reactApplicationContext
val intent = Intent(ctx, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP
data = Uri.parse("bulwark-notification://$notificationId")
if (emailId != null) putExtra(NotificationTapStore.EXTRA_EMAIL_ID, emailId)
if (threadId != null) putExtra(NotificationTapStore.EXTRA_THREAD_ID, threadId)
if (subject != null) putExtra(NotificationTapStore.EXTRA_SUBJECT, subject)
@@ -127,12 +140,27 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
val bigText = buildString {
if (!subject.isNullOrBlank()) append(subject)
if (!preview.isNullOrBlank()) {
if (isNotEmpty()) append("\n")
append(preview)
}
}
val builder = NotificationCompat.Builder(ctx, BulwarkMessagingService.CHANNEL_ID)
.setSmallIcon(R.drawable.ic_notification)
.setLargeIcon(largeIcon)
.setContentTitle(title)
// body is the subject with the "(No subject)" fallback; an empty
// subject is "" here, not null, so `subject ?: ...` showed a blank line.
.setContentText(body)
.setStyle(NotificationCompat.BigTextStyle().bigText(body))
.setStyle(
NotificationCompat.BigTextStyle()
.bigText(if (bigText.isNotBlank()) bigText else body)
.setBigContentTitle(title)
.setSummaryText(groupTitle)
)
.setColor(parseColor(colorHex, fallback = Color.parseColor("#2563eb")))
.setAutoCancel(true)
.setPriority(NotificationCompat.PRIORITY_HIGH)
@@ -142,6 +170,72 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
builder.setGroupAlertBehavior(NotificationCompat.GROUP_ALERT_CHILDREN)
}
if (emailId != null && accountId != null) {
// Action 1: Mark as read
val markReadIntent = Intent(ctx, BulwarkNotificationActionReceiver::class.java).apply {
action = ACTION_MARK_READ
data = Uri.parse("bulwark-action-read://$notificationId")
putExtra(EXTRA_NOTIFICATION_ID, notificationId)
putExtra(EXTRA_ACTION, "markRead")
putExtra(NotificationTapStore.EXTRA_EMAIL_ID, emailId)
putExtra(NotificationTapStore.EXTRA_ACCOUNT_ID, accountId)
if (jmapAccountId != null) putExtra(NotificationTapStore.EXTRA_JMAP_ACCOUNT_ID, jmapAccountId)
}
val markReadPending = PendingIntent.getBroadcast(
ctx,
(notificationId + ":read").hashCode(),
markReadIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
builder.addAction(
NotificationCompat.Action.Builder(0, markReadLabel, markReadPending).build()
)
// Action 2: Delete
val deleteIntent = Intent(ctx, BulwarkNotificationActionReceiver::class.java).apply {
action = ACTION_DELETE
data = Uri.parse("bulwark-action-delete://$notificationId")
putExtra(EXTRA_NOTIFICATION_ID, notificationId)
putExtra(EXTRA_ACTION, "delete")
putExtra(NotificationTapStore.EXTRA_EMAIL_ID, emailId)
putExtra(NotificationTapStore.EXTRA_ACCOUNT_ID, accountId)
if (jmapAccountId != null) putExtra(NotificationTapStore.EXTRA_JMAP_ACCOUNT_ID, jmapAccountId)
}
val deletePending = PendingIntent.getBroadcast(
ctx,
(notificationId + ":delete").hashCode(),
deleteIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
builder.addAction(
NotificationCompat.Action.Builder(0, deleteLabel, deletePending)
// Android 12+: unlock first, so a locked phone can't delete mail.
.setAuthenticationRequired(true)
.build()
)
// Action 3: Reply
val replyIntent = Intent(ctx, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP
data = Uri.parse("bulwark-reply://$notificationId")
putExtra(NotificationTapStore.EXTRA_EMAIL_ID, emailId)
if (threadId != null) putExtra(NotificationTapStore.EXTRA_THREAD_ID, threadId)
if (subject != null) putExtra(NotificationTapStore.EXTRA_SUBJECT, subject)
putExtra(NotificationTapStore.EXTRA_ACCOUNT_ID, accountId)
if (jmapAccountId != null) putExtra(NotificationTapStore.EXTRA_JMAP_ACCOUNT_ID, jmapAccountId)
putExtra(EXTRA_ACTION, "reply")
}
val replyPending = PendingIntent.getActivity(
ctx,
(notificationId + ":reply").hashCode(),
replyIntent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
builder.addAction(
NotificationCompat.Action.Builder(0, replyLabel, replyPending).build()
)
}
val manager = ctx.getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
manager.notify(notificationId, notificationId.hashCode(), builder.build())
}
@@ -157,7 +251,11 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
it.groupKey?.endsWith(groupKey) == true &&
(it.notification.flags and android.app.Notification.FLAG_GROUP_SUMMARY) == 0
}
val count = maxOf(children.size, 1)
if (children.size < 2) {
manager.cancel(groupKey, groupKey.hashCode())
return
}
val count = children.size
val inbox = NotificationCompat.InboxStyle()
children.sortedByDescending { it.postTime }.take(5).forEach { sbn ->
val extras = sbn.notification.extras
@@ -172,6 +270,7 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
val intent = Intent(ctx, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP
data = Uri.parse("bulwark-group-summary://$groupKey")
if (accountId != null) putExtra(NotificationTapStore.EXTRA_ACCOUNT_ID, accountId)
}
val pending = PendingIntent.getActivity(
@@ -273,6 +372,11 @@ class BulwarkFcmModule(reactContext: ReactApplicationContext)
@Volatile private var currentInstance: BulwarkFcmModule? = null
const val EXTRA_ACTION = "bulwark.notification.action"
const val EXTRA_NOTIFICATION_ID = "bulwark.notification.notificationId"
const val ACTION_MARK_READ = "com.anonymous.bulwarkmobile.ACTION_MARK_READ"
const val ACTION_DELETE = "com.anonymous.bulwarkmobile.ACTION_DELETE"
fun emit(eventName: String, params: WritableMap?) {
val module = currentInstance ?: return
val ctx = module.reactApplicationContext
@@ -1,5 +1,6 @@
package com.anonymous.bulwarkmobile
import com.anonymous.bulwarkmobile.sync.BulwarkDeviceSyncModule
import com.facebook.react.ReactPackage
import com.facebook.react.bridge.NativeModule
import com.facebook.react.bridge.ReactApplicationContext
@@ -12,6 +13,9 @@ class BulwarkFcmPackage : ReactPackage {
add(BulwarkFcmModule(reactContext))
add(BulwarkUnifiedPushModule(reactContext))
add(BulwarkClientCertModule(reactContext))
add(BulwarkDeviceSyncModule(reactContext))
add(BulwarkWidgetsModule(reactContext))
add(BulwarkPdfModule(reactContext))
// The bridge registers React Native's own module under this name.
if (ReactNativeNewArchitectureFeatureFlags.enableBridgelessArchitecture()) {
add(BulwarkHeadlessJsTaskSupportModule(reactContext))
@@ -11,6 +11,7 @@ import android.os.Bundle
import android.os.PowerManager
import android.util.Log
import androidx.core.app.NotificationCompat
import com.anonymous.bulwarkmobile.sync.DeviceSyncPushRouter
import com.facebook.react.bridge.Arguments
import com.google.firebase.messaging.FirebaseMessagingService
import com.google.firebase.messaging.RemoteMessage
@@ -26,12 +27,16 @@ class BulwarkMessagingService : FirebaseMessagingService() {
val data = message.data
// Contact and calendar changes become syncs of the Android accounts
// they feed (device sync, #34); only mail needs the push task.
val mail = DeviceSyncPushRouter.route(applicationContext, data)
// Hand off to JS via a headless task only when the app isn't already
// running in the foreground - HeadlessJsTaskContext throws if started
// while foreground. When the app is open, the main JS instance already
// receives JMAP push directly, so the fcm:message event below is
// sufficient for it to refresh state.
if (!isAppInForeground()) {
if (mail && !isAppInForeground()) {
startHeadlessTask(applicationContext, data)
}
@@ -0,0 +1,124 @@
package com.anonymous.bulwarkmobile
import android.app.Application
import android.app.NotificationManager
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.os.Bundle
import android.os.Handler
import android.os.Looper
import android.util.Log
import com.facebook.react.bridge.Arguments
import com.facebook.react.jstasks.HeadlessJsTaskConfig
import com.facebook.react.jstasks.HeadlessJsTaskContext
import com.facebook.react.jstasks.HeadlessJsTaskEventListener
class BulwarkNotificationActionReceiver : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
val notificationId = intent.getStringExtra(BulwarkFcmModule.EXTRA_NOTIFICATION_ID)
val action = intent.getStringExtra(BulwarkFcmModule.EXTRA_ACTION)
val emailId = intent.getStringExtra(NotificationTapStore.EXTRA_EMAIL_ID)
val accountId = intent.getStringExtra(NotificationTapStore.EXTRA_ACCOUNT_ID)
val jmapAccountId = intent.getStringExtra(NotificationTapStore.EXTRA_JMAP_ACCOUNT_ID)
// Dismiss the notification immediately from the tray
val manager = context.getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
if (notificationId != null) {
manager.cancel(notificationId, notificationId.hashCode())
}
if (emailId.isNullOrBlank() || action.isNullOrBlank() || accountId.isNullOrBlank()) {
Log.w(TAG, "Missing parameters for notification action: action=$action, emailId=$emailId, accountId=$accountId")
return
}
// On Android 8+, a broadcast receiver in background cannot start a Service
// (BackgroundServiceStartNotAllowedException). Instead, we use goAsync() and
// HeadlessJs.withReadyReactContext to execute the headless JS task directly.
val pendingResult = goAsync()
val app = context.applicationContext as Application
val mainHandler = Handler(Looper.getMainLooper())
// Safety timeout to ensure pendingResult.finish() is always called
val timeoutRunnable = Runnable {
Log.w(TAG, "Notification action task timed out or failed to initialize ($action, $emailId)")
try {
pendingResult.finish()
} catch (_: Exception) {}
}
mainHandler.postDelayed(timeoutRunnable, 20_000L)
try {
HeadlessJs.withReadyReactContext(app) { reactContext ->
if (!reactContext.hasActiveReactInstance()) {
Log.w(TAG, "React context does not have active instance")
mainHandler.removeCallbacks(timeoutRunnable)
try {
pendingResult.finish()
} catch (_: Exception) {}
return@withReadyReactContext
}
val tasks = HeadlessJsTaskContext.getInstance(reactContext)
val taskBundle = Bundle().apply {
putString("action", action)
putString("emailId", emailId)
putString("accountId", accountId)
if (jmapAccountId != null) putString("jmapAccountId", jmapAccountId)
// Keep raw extras as well
putString(BulwarkFcmModule.EXTRA_ACTION, action)
putString(NotificationTapStore.EXTRA_EMAIL_ID, emailId)
putString(NotificationTapStore.EXTRA_ACCOUNT_ID, accountId)
if (jmapAccountId != null) putString(NotificationTapStore.EXTRA_JMAP_ACCOUNT_ID, jmapAccountId)
}
val config = HeadlessJsTaskConfig(
"BulwarkNotificationAction",
Arguments.fromBundle(taskBundle),
15_000L,
true // allowExecutionInForeground
)
val listener = object : HeadlessJsTaskEventListener {
override fun onHeadlessJsTaskStart(taskId: Int) = Unit
override fun onHeadlessJsTaskFinish(taskId: Int) {
Log.i(TAG, "Notification action task $taskId finished for action $action")
tasks.removeTaskEventListener(this)
mainHandler.removeCallbacks(timeoutRunnable)
try {
pendingResult.finish()
} catch (_: Exception) {}
}
}
tasks.addTaskEventListener(listener)
HeadlessJs.startTask(
tasks,
config,
onStarted = { taskId ->
Log.i(TAG, "Started notification action task $taskId for action $action ($emailId)")
},
onRefused = { e ->
Log.w(TAG, "Notification action task refused: ${e.message}")
tasks.removeTaskEventListener(listener)
mainHandler.removeCallbacks(timeoutRunnable)
try {
pendingResult.finish()
} catch (_: Exception) {}
}
)
}
} catch (e: Exception) {
Log.e(TAG, "Failed to dispatch headless task from action receiver", e)
mainHandler.removeCallbacks(timeoutRunnable)
try {
pendingResult.finish()
} catch (_: Exception) {}
}
}
companion object {
private const val TAG = "BulwarkNotifAction"
}
}
@@ -0,0 +1,141 @@
package com.anonymous.bulwarkmobile
import android.graphics.Bitmap
import android.graphics.Color
import android.graphics.pdf.PdfRenderer
import android.net.Uri
import android.os.ParcelFileDescriptor
import com.facebook.react.bridge.Arguments
import com.facebook.react.bridge.Promise
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
import java.io.File
import java.io.FileOutputStream
import java.util.concurrent.Executors
/**
* Renders PDF pages to PNG files for the in-app file preview, with the
* platform's own PdfRenderer: Android has no PDF view, and its WebView
* doesn't show PDFs either. The preview draws the pages in a WebView of
* its own so they zoom, asking for each page as it scrolls into view.
*
* PdfRenderer is not thread-safe and allows one open page at a time, so
* every call runs on one background thread, and the open document is kept
* between calls for the file the preview shows.
*/
class BulwarkPdfModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaModule(reactContext) {
private val worker = Executors.newSingleThreadExecutor { r -> Thread(r, "BulwarkPdf").apply { isDaemon = true } }
private var openPath: String? = null
private var descriptor: ParcelFileDescriptor? = null
private var renderer: PdfRenderer? = null
override fun getName(): String = "BulwarkPdf"
override fun invalidate() {
worker.execute { closeDocument() }
worker.shutdown()
super.invalidate()
}
/** Page count and each page's size in points: `{ pages: [{ width, height }] }`. */
@ReactMethod
fun open(fileUri: String, promise: Promise) {
worker.execute {
try {
val doc = document(fileUri)
val pages = Arguments.createArray()
for (i in 0 until doc.pageCount) {
doc.openPage(i).use { page ->
pages.pushMap(Arguments.createMap().apply {
putInt("width", page.width)
putInt("height", page.height)
})
}
}
promise.resolve(Arguments.createMap().apply { putArray("pages", pages) })
} catch (e: SecurityException) {
// Password-protected; PdfRenderer can't open those.
closeDocument()
promise.reject("E_PDF_PROTECTED", e.message, e)
} catch (e: Throwable) {
closeDocument()
promise.reject("E_PDF_OPEN", e.message, e)
}
}
}
/** Render one page `width` pixels wide into `outUri` (a PNG). Resolves with the file URI. */
@ReactMethod
fun renderPage(fileUri: String, index: Int, width: Int, outUri: String, promise: Promise) {
worker.execute {
try {
val doc = document(fileUri)
if (index < 0 || index >= doc.pageCount) throw IndexOutOfBoundsException("page $index")
doc.openPage(index).use { page ->
val w = width.coerceIn(1, MAX_PAGE_PX)
val h = (w.toLong() * page.height / page.width.coerceAtLeast(1)).toInt().coerceIn(1, MAX_PAGE_PX * 2)
val bitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888)
try {
// Pages are transparent where nothing is drawn.
bitmap.eraseColor(Color.WHITE)
page.render(bitmap, null, null, PdfRenderer.Page.RENDER_MODE_FOR_DISPLAY)
val out = fileOf(outUri)
out.parentFile?.mkdirs()
FileOutputStream(out).use { stream -> bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream) }
} finally {
bitmap.recycle()
}
}
promise.resolve(outUri)
} catch (e: Throwable) {
promise.reject("E_PDF_RENDER", e.message, e)
}
}
}
/** Close the document the preview showed. */
@ReactMethod
fun close(fileUri: String, promise: Promise) {
worker.execute {
if (openPath == fileOf(fileUri).absolutePath) closeDocument()
promise.resolve(null)
}
}
private fun document(fileUri: String): PdfRenderer {
val path = fileOf(fileUri).absolutePath
renderer?.let { if (openPath == path) return it }
closeDocument()
val fd = ParcelFileDescriptor.open(File(path), ParcelFileDescriptor.MODE_READ_ONLY)
try {
val doc = PdfRenderer(fd)
descriptor = fd
renderer = doc
openPath = path
return doc
} catch (e: Throwable) {
fd.close()
throw e
}
}
private fun closeDocument() {
try { renderer?.close() } catch (_: Throwable) {}
try { descriptor?.close() } catch (_: Throwable) {}
renderer = null
descriptor = null
openPath = null
}
private fun fileOf(uri: String): File {
val path = if (uri.startsWith("file:")) Uri.parse(uri).path else uri
return File(path ?: throw IllegalArgumentException("not a file: $uri"))
}
companion object {
// A 4096 px page bitmap is 64 MB; keep well below.
private const val MAX_PAGE_PX = 2480
}
}
@@ -11,16 +11,11 @@ import android.os.Bundle
import android.os.PersistableBundle
import android.os.SystemClock
import android.util.Log
import com.facebook.react.ReactApplication
import com.facebook.react.ReactInstanceEventListener
import com.facebook.react.bridge.ReactContext
import com.facebook.react.bridge.UiThreadUtil
import com.facebook.react.internal.featureflags.ReactNativeNewArchitectureFeatureFlags
import com.facebook.react.jstasks.HeadlessJsTaskConfig
import com.facebook.react.jstasks.HeadlessJsTaskContext
import com.facebook.react.jstasks.HeadlessJsTaskEventListener
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.atomic.AtomicBoolean
/**
* Runs the BulwarkPushTask headless JS task from a JobScheduler job.
@@ -44,7 +39,8 @@ class BulwarkPushJobService : JobService(), HeadlessJsTaskEventListener {
override fun onStartJob(params: JobParameters): Boolean {
val config = BulwarkPushTaskService.taskConfig(Bundle(params.extras))
withReadyReactContext { startTask(it, config, params) }
// JobService callbacks run on the main thread, where HeadlessJs wants to be called.
HeadlessJs.withReadyReactContext(application) { startTask(it, config, params) }
return true
}
@@ -76,68 +72,17 @@ class BulwarkPushJobService : JobService(), HeadlessJsTaskEventListener {
tasks.addTaskEventListener(this)
taskContext = tasks
}
UiThreadUtil.runOnUiThread {
try {
val taskId = tasks.startTask(config)
running[taskId] = RunningTask(params, SystemClock.elapsedRealtime())
} catch (e: IllegalStateException) {
HeadlessJs.startTask(
tasks,
config,
onStarted = { taskId -> running[taskId] = RunningTask(params, SystemClock.elapsedRealtime()) },
onRefused = { e ->
// The app came to the foreground since the push arrived and
// syncs on its own there.
Log.i(TAG, "Push job ${params.jobId} skipped: ${e.message}")
jobFinished(params, false)
}
}
}
/**
* Calls [onReady] once with a React context whose JS instance is up,
* starting React Native if the process was woken for this job.
*
* A context exists before its instance is ready, and a task started on
* it then never reaches JS (HeadlessJsTaskContext only logs "CatalystInstance
* not available" and waits for the timeout), which happened when several
* pushes arrived while the app was starting. So the listener goes in
* first and the ready check after it; whichever sees the ready instance
* first runs the task.
*/
private fun withReadyReactContext(onReady: (ReactContext) -> Unit) {
val done = AtomicBoolean(false)
val runOnce = { context: ReactContext -> if (done.compareAndSet(false, true)) onReady(context) }
val app = application as ReactApplication
if (ReactNativeNewArchitectureFeatureFlags.enableBridgelessArchitecture()) {
val host = checkNotNull(app.reactHost) { "ReactHost is not initialized in New Architecture" }
val listener = object : ReactInstanceEventListener {
override fun onReactContextInitialized(context: ReactContext) {
host.removeReactInstanceEventListener(this)
runOnce(context)
}
}
host.addReactInstanceEventListener(listener)
val current = host.currentReactContext
if (current != null && current.hasActiveReactInstance()) {
host.removeReactInstanceEventListener(listener)
runOnce(current)
} else {
host.start()
}
} else {
@Suppress("DEPRECATION")
val manager = app.reactNativeHost.reactInstanceManager
val listener = object : ReactInstanceEventListener {
override fun onReactContextInitialized(context: ReactContext) {
manager.removeReactInstanceEventListener(this)
runOnce(context)
}
}
manager.addReactInstanceEventListener(listener)
val current = manager.currentReactContext
if (current != null && current.hasActiveReactInstance()) {
manager.removeReactInstanceEventListener(listener)
runOnce(current)
} else if (!manager.hasStartedCreatingInitialContext()) {
manager.createReactContextInBackground()
}
}
},
)
}
companion object {
@@ -1,6 +1,7 @@
package com.anonymous.bulwarkmobile
import android.content.Context
import com.anonymous.bulwarkmobile.sync.DeviceSyncPushRouter
import com.facebook.react.bridge.Arguments
import org.json.JSONArray
import org.json.JSONObject
@@ -31,7 +32,10 @@ class BulwarkUnifiedPushService : PushService() {
BulwarkMessagingService.ensureChannel(this)
val data = parsePayload(message.content) ?: return
if (!BulwarkMessagingService.isAppInForeground()) {
// Same routing as FCM: contact and calendar changes request device
// syncs, and only mail starts the push task.
val mail = DeviceSyncPushRouter.route(applicationContext, data)
if (mail && !BulwarkMessagingService.isAppInForeground()) {
BulwarkMessagingService.startHeadlessTask(applicationContext, data)
}
@@ -0,0 +1,20 @@
package com.anonymous.bulwarkmobile
import android.content.Intent
import com.facebook.react.HeadlessJsTaskService
import com.facebook.react.bridge.Arguments
import com.facebook.react.jstasks.HeadlessJsTaskConfig
/**
* Runs the widgets' refresh (BulwarkWidgetRefresh, index.ts) as a headless
* task when the app goes to the background. Run from the app itself the
* refresh stalled: React Native pauses JS timers while the activity is in
* the background, and fetch resolves every response through a timer, so the
* first request went out and its answer was never read. A running headless
* task keeps timers going, and the service keeps the process from being
* frozen before the widgets are redrawn.
*/
class BulwarkWidgetRefreshService : HeadlessJsTaskService() {
override fun getTaskConfig(intent: Intent?): HeadlessJsTaskConfig =
HeadlessJsTaskConfig("BulwarkWidgetRefresh", Arguments.createMap(), 30000L, true)
}
@@ -0,0 +1,28 @@
package com.anonymous.bulwarkmobile
import android.content.Intent
import com.facebook.react.bridge.Promise
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
/** JS side: src/widgets/sync.ts. */
class BulwarkWidgetsModule(reactContext: ReactApplicationContext) :
ReactContextBaseJavaModule(reactContext) {
override fun getName(): String = "BulwarkWidgets"
/** Starts BulwarkWidgetRefreshService; resolves false when Android refuses the start. */
@ReactMethod
fun refreshInBackground(promise: Promise) {
try {
reactApplicationContext.startService(Intent(reactApplicationContext, BulwarkWidgetRefreshService::class.java))
promise.resolve(true)
} catch (e: IllegalStateException) {
// Background start limits (the app left the foreground too long ago).
promise.resolve(false)
} catch (e: SecurityException) {
promise.resolve(false)
}
}
}
@@ -0,0 +1,146 @@
package com.anonymous.bulwarkmobile
import android.app.Application
import android.os.Handler
import android.os.Looper
import android.os.SystemClock
import android.util.Log
import androidx.annotation.MainThread
import com.facebook.react.ReactApplication
import com.facebook.react.ReactInstanceEventListener
import com.facebook.react.bridge.ReactContext
import com.facebook.react.bridge.UiThreadUtil
import com.facebook.react.interfaces.TaskInterface
import com.facebook.react.internal.featureflags.ReactNativeNewArchitectureFeatureFlags
import com.facebook.react.jstasks.HeadlessJsTaskConfig
import com.facebook.react.jstasks.HeadlessJsTaskContext
import java.util.concurrent.atomic.AtomicBoolean
/**
* Running a headless JS task without HeadlessJsTaskService: starting React
* Native when the process was woken without any UI, and starting the task on
* its context. Shared by the push job (BulwarkPushJobService) and the device
* sync adapters.
*/
object HeadlessJs {
private const val TAG = "HeadlessJs"
/** No caller waits longer for React Native: JobScheduler stops sync and push jobs after 10 minutes. */
private const val MAX_WAIT_MS = 600_000L
private const val WATCH_MS = 1_000L
/**
* Calls [onReady] once with a React context whose JS instance is up,
* starting React Native if the process was woken for this work.
*
* A context exists before its instance is ready, and a task started on
* it then never reaches JS (HeadlessJsTaskContext only logs "CatalystInstance
* not available" and waits for the timeout), which happened when several
* pushes arrived while the app was starting. So the listener goes in
* first and the ready check after it; whichever sees the ready instance
* first runs the task.
*
* The wait ends once, and its listener goes with it: with a ready context,
* or without one when React fails to start (the start task faults) or
* after [MAX_WAIT_MS], when no caller can still be waiting. A context that
* comes up later runs nothing of it, so waits given up on don't pile up on
* the host with everything their [onReady] holds. (A caller that gives up
* sooner, like the sync adapter at its deadline, checks that in [onReady].)
*
* Call it on the main thread; [onReady] runs there too. MainApplication
* creates the ReactHost on first use without a lock (ExpoReactHostFactory),
* so a background caller racing MainActivity could create a second one,
* with a second JS runtime.
*
* Returns React Native's start task when it had to be started, so a caller
* can notice a start that fails (a debug build with no Metro to load the
* bundle from), or null when the context was already up.
*/
@MainThread
fun withReadyReactContext(app: Application, onReady: (ReactContext) -> Unit): TaskInterface<Void>? {
val reactApp = app as ReactApplication
if (ReactNativeNewArchitectureFeatureFlags.enableBridgelessArchitecture()) {
val host = checkNotNull(reactApp.reactHost) { "ReactHost is not initialized in New Architecture" }
val wait = ContextWait(onReady) { host.removeReactInstanceEventListener(it) }
host.addReactInstanceEventListener(wait)
val current = host.currentReactContext
if (current != null && current.hasActiveReactInstance()) {
wait.onReactContextInitialized(current)
return null
}
return host.start().also { wait.watch(it) }
}
@Suppress("DEPRECATION")
val manager = reactApp.reactNativeHost.reactInstanceManager
val wait = ContextWait(onReady) { manager.removeReactInstanceEventListener(it) }
manager.addReactInstanceEventListener(wait)
val current = manager.currentReactContext
if (current != null && current.hasActiveReactInstance()) {
wait.onReactContextInitialized(current)
return null
}
if (!manager.hasStartedCreatingInitialContext()) manager.createReactContextInBackground()
wait.watch(null)
return null
}
/** One caller's wait for a ready context: a listener on the host that ends once and then removes itself. */
private class ContextWait(
private val onReady: (ReactContext) -> Unit,
private val removeListener: (ReactInstanceEventListener) -> Unit,
) : ReactInstanceEventListener {
private val ended = AtomicBoolean(false)
override fun onReactContextInitialized(context: ReactContext) {
if (end()) onReady(context)
}
/** Ends the wait without a context once [boot] faulted or [MAX_WAIT_MS] passed; checked on the main thread. */
fun watch(boot: TaskInterface<Void>?) {
val main = Handler(Looper.getMainLooper())
val giveUpAt = SystemClock.elapsedRealtime() + MAX_WAIT_MS
main.postDelayed(object : Runnable {
override fun run() {
if (ended.get()) return
val faulted = boot?.isFaulted() == true
if (!faulted && SystemClock.elapsedRealtime() < giveUpAt) {
main.postDelayed(this, WATCH_MS)
} else if (end()) {
val why = if (faulted) "React Native did not start" else "no React context after $MAX_WAIT_MS ms"
Log.w(TAG, "$why; no longer waiting")
}
}
}, WATCH_MS)
}
private fun end(): Boolean {
if (!ended.compareAndSet(false, true)) return false
removeListener(this)
return true
}
}
/**
* Starts [config] on the UI thread, where HeadlessJsTaskContext wants it.
* [onStarted] gets the task id in that same UI-thread turn, so before any
* finish callback (HeadlessJsTaskContext posts those to the UI thread).
* [onRefused] gets the refusal when the context is resumed and [config]
* isn't allowed in the foreground.
*/
fun startTask(
tasks: HeadlessJsTaskContext,
config: HeadlessJsTaskConfig,
onStarted: (taskId: Int) -> Unit,
onRefused: (IllegalStateException) -> Unit,
) {
UiThreadUtil.runOnUiThread {
val taskId = try {
tasks.startTask(config)
} catch (e: IllegalStateException) {
onRefused(e)
return@runOnUiThread
}
onStarted(taskId)
}
}
}
@@ -43,7 +43,8 @@ object NotificationTapStore {
// The JMAP account a group/shared mailbox's message lives in - a JMAP
// id like the email id. Optional: older notifications don't carry it.
val jmapAccountId = extras.getString(EXTRA_JMAP_ACCOUNT_ID)?.takeIf { ID_PATTERN.matches(it) }
val payload = TapPayload(emailId, threadId, subject, accountId, jmapAccountId)
val action = extras.getString(EXTRA_ACTION)?.takeIf { it == "reply" }
val payload = TapPayload(emailId, threadId, subject, accountId, jmapAccountId, action)
pending = payload
// Clear so a subsequent activity lifecycle event doesn't replay this.
extras.remove(EXTRA_EMAIL_ID)
@@ -51,6 +52,7 @@ object NotificationTapStore {
extras.remove(EXTRA_SUBJECT)
extras.remove(EXTRA_ACCOUNT_ID)
extras.remove(EXTRA_JMAP_ACCOUNT_ID)
extras.remove(EXTRA_ACTION)
return payload
}
@@ -60,6 +62,7 @@ object NotificationTapStore {
val subject: String?,
val accountId: String?,
val jmapAccountId: String?,
val action: String? = null,
) {
fun toMap(): WritableMap = Arguments.createMap().apply {
putString("emailId", emailId)
@@ -67,6 +70,7 @@ object NotificationTapStore {
if (subject != null) putString("subject", subject)
if (accountId != null) putString("accountId", accountId)
if (jmapAccountId != null) putString("jmapAccountId", jmapAccountId)
if (action != null) putString("action", action)
}
}
@@ -75,4 +79,5 @@ object NotificationTapStore {
const val EXTRA_SUBJECT = "bulwark.notification.subject"
const val EXTRA_ACCOUNT_ID = "bulwark.notification.accountId"
const val EXTRA_JMAP_ACCOUNT_ID = "bulwark.notification.jmapAccountId"
const val EXTRA_ACTION = "bulwark.notification.action"
}
@@ -0,0 +1,79 @@
package com.anonymous.bulwarkmobile.sync
import android.accounts.AbstractAccountAuthenticator
import android.accounts.Account
import android.accounts.AccountAuthenticatorResponse
import android.accounts.AccountManager
import android.app.Service
import android.content.Context
import android.content.Intent
import android.net.Uri
import android.os.Bundle
import android.os.IBinder
import com.anonymous.bulwarkmobile.MainActivity
/**
* The authenticator of the "Bulwark" account type. The app holds the
* credentials, not the account: accounts are only ever created by the app,
* from Settings → Contacts or Calendar, for an account that is signed in.
*/
class BulwarkAuthenticator(private val context: Context) : AbstractAccountAuthenticator(context) {
/** Settings → Accounts → Add account opens the app's device sync settings instead. */
override fun addAccount(
response: AccountAuthenticatorResponse?,
accountType: String?,
authTokenType: String?,
requiredFeatures: Array<out String>?,
options: Bundle?,
): Bundle {
val intent = Intent(context, MainActivity::class.java)
.setAction(Intent.ACTION_VIEW)
.setData(Uri.parse(SETTINGS_URI))
return Bundle().apply { putParcelable(AccountManager.KEY_INTENT, intent) }
}
/** Removal from Android Settings is allowed; the app notices and turns device sync off. */
override fun getAccountRemovalAllowed(response: AccountAuthenticatorResponse?, account: Account?): Bundle =
Bundle().apply { putBoolean(AccountManager.KEY_BOOLEAN_RESULT, true) }
override fun getAuthToken(
response: AccountAuthenticatorResponse?,
account: Account?,
authTokenType: String?,
options: Bundle?,
): Bundle = unsupported()
override fun confirmCredentials(response: AccountAuthenticatorResponse?, account: Account?, options: Bundle?): Bundle =
unsupported()
override fun updateCredentials(
response: AccountAuthenticatorResponse?,
account: Account?,
authTokenType: String?,
options: Bundle?,
): Bundle = unsupported()
override fun editProperties(response: AccountAuthenticatorResponse?, accountType: String?): Bundle = unsupported()
override fun getAuthTokenLabel(authTokenType: String?): String? = null
override fun hasFeatures(response: AccountAuthenticatorResponse?, account: Account?, features: Array<out String>?): Bundle =
Bundle().apply { putBoolean(AccountManager.KEY_BOOLEAN_RESULT, false) }
private fun unsupported() = Bundle().apply {
putInt(AccountManager.KEY_ERROR_CODE, AccountManager.ERROR_CODE_UNSUPPORTED_OPERATION)
putString(AccountManager.KEY_ERROR_MESSAGE, "Bulwark signs in from the app")
}
companion object {
/** Handled by the app's deep links: Settings → Contacts, with its "Sync to this device" section. */
const val SETTINGS_URI = "bulwarkmobile://settings/contacts"
}
}
class BulwarkAuthenticatorService : Service() {
private val authenticator by lazy { BulwarkAuthenticator(this) }
override fun onBind(intent: Intent): IBinder? = authenticator.iBinder
}
@@ -0,0 +1,331 @@
package com.anonymous.bulwarkmobile.sync
import android.accounts.Account
import android.accounts.AccountManager
import android.accounts.OnAccountsUpdateListener
import android.content.ActivityNotFoundException
import android.content.ContentResolver
import android.content.Intent
import android.os.Build
import android.os.Bundle
import android.os.Handler
import android.os.Looper
import android.provider.Settings
import android.util.Log
import com.facebook.react.bridge.Arguments
import com.facebook.react.bridge.Promise
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
import com.facebook.react.bridge.WritableMap
import com.facebook.react.modules.core.DeviceEventManagerModule.RCTDeviceEventEmitter
import org.json.JSONObject
import java.util.concurrent.ExecutorService
import java.util.concurrent.Executors
import java.util.concurrent.RejectedExecutionException
/**
* `NativeModules.BulwarkDeviceSync`: Android accounts, sync settings, the run
* handshake, provider I/O, push routes and the sync-problem notification of
* device sync (#34). The contract is `DeviceSyncNativeModule` in
* src/device-sync/types.ts; docs/device-sync.md explains it.
*/
class BulwarkDeviceSyncModule(reactContext: ReactApplicationContext) :
ReactContextBaseJavaModule(reactContext) {
private val accountsListener = OnAccountsUpdateListener {
emit(ACCOUNTS_CHANGED_EVENT, null)
}
/**
* Provider I/O runs here, one call at a time: a batch can hold the
* provider for seconds (it sleeps at a contended yield point), which would
* stall every other native module call on React's native-modules thread.
*/
private val io: ExecutorService = Executors.newSingleThreadExecutor { task ->
Thread(task, "BulwarkDeviceSync-io").apply { isDaemon = true }
}
init {
synchronized(BulwarkDeviceSyncModule::class.java) { currentInstance = this }
}
override fun getName(): String = NAME
override fun initialize() {
super.initialize()
val manager = AccountManager.get(reactApplicationContext)
val handler = Handler(Looper.getMainLooper())
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
manager.addOnAccountsUpdatedListener(accountsListener, handler, false, arrayOf(accountType()))
} else {
manager.addOnAccountsUpdatedListener(accountsListener, handler, false)
}
} catch (e: RuntimeException) {
Log.w(TAG, "Account change listener not registered", e)
}
}
override fun invalidate() {
try {
AccountManager.get(reactApplicationContext).removeOnAccountsUpdatedListener(accountsListener)
} catch (e: RuntimeException) {
// Not registered.
}
io.shutdown()
super.invalidate()
}
private fun accountType() = DeviceSyncAccounts.accountType(reactApplicationContext)
// ── accounts ────────────────────────────────────────────────
@ReactMethod
fun getInfo(promise: Promise) {
promise.resolve(Arguments.createMap().apply {
putString("accountType", accountType())
putInt("sdkInt", Build.VERSION.SDK_INT)
})
}
@ReactMethod
fun listAccounts(promise: Promise) = settle(promise) {
Arguments.createArray().apply {
for ((account, registryId) in DeviceSyncAccounts.list(reactApplicationContext)) {
pushMap(Arguments.createMap().apply {
putString("name", account.name)
putString("registryId", registryId)
})
}
}
}
@ReactMethod
fun ensureAccount(name: String, registryId: String, promise: Promise) = settle(promise) {
DeviceSyncAccounts.ensure(reactApplicationContext, name, registryId)
}
@ReactMethod
fun removeAccount(name: String, promise: Promise) = settle(promise) {
DeviceSyncAccounts.remove(reactApplicationContext, name)
}
// ── sync settings ───────────────────────────────────────────
@ReactMethod
fun getSyncSettings(name: String, promise: Promise) = settle(promise) {
val account = DeviceSyncAccounts.account(reactApplicationContext, name)
Arguments.createMap().apply {
putBoolean("masterAutomatic", ContentResolver.getMasterSyncAutomatically())
putMap("authorities", Arguments.createMap().apply {
for (authority in DeviceSyncAccounts.AUTHORITIES) putMap(authority, authoritySettings(account, authority))
})
}
}
private fun authoritySettings(account: Account, authority: String): WritableMap = Arguments.createMap().apply {
putInt("syncable", ContentResolver.getIsSyncable(account, authority))
putBoolean("automatic", ContentResolver.getSyncAutomatically(account, authority))
val periodic = ContentResolver.getPeriodicSyncs(account, authority).minOfOrNull { it.period } ?: 0L
putDouble("periodicSeconds", periodic.toDouble())
putBoolean("active", ContentResolver.isSyncActive(account, authority))
putBoolean("pending", ContentResolver.isSyncPending(account, authority))
}
@ReactMethod
fun setSyncEnabled(name: String, authority: String, enabled: Boolean, promise: Promise) = settle(promise) {
DeviceSyncAccounts.setSyncEnabled(reactApplicationContext, name, authority, enabled)
null
}
@ReactMethod
fun setPeriodicSync(name: String, authority: String, seconds: Double, promise: Promise) = settle(promise) {
DeviceSyncAccounts.setPeriodicSync(reactApplicationContext, name, authority, seconds.toLong())
null
}
@ReactMethod
fun requestSync(name: String, authority: String, optionsJson: String, promise: Promise) = settle(promise) {
val options = JSONObject(optionsJson)
val flags = SyncRequestFlags(
manual = options.optBoolean("manual"),
expedited = options.optBoolean("expedited"),
upload = options.optBoolean("upload"),
overrideTooManyDeletions = options.optBoolean("overrideTooManyDeletions"),
discardLocalDeletions = options.optBoolean("discardLocalDeletions"),
)
DeviceSyncAccounts.requestSync(reactApplicationContext, name, authority, flags)
null
}
/** The account's own sync screen where the platform has one, else the list of synced accounts. */
@ReactMethod
fun openAccountSettings(name: String, promise: Promise) = settle(promise) {
val account = DeviceSyncAccounts.account(reactApplicationContext, name)
val perAccount = Intent(ACTION_ACCOUNT_SYNC_SETTINGS)
.putExtra(EXTRA_ACCOUNT, account)
// Newer Settings apps pass only these arguments on to the account's
// screen; without them it closes at once ("Account provided does
// not exist: null", Android 12L), and no exception tells us so.
.putExtra(EXTRA_SHOW_FRAGMENT_ARGUMENTS, Bundle().apply { putParcelable(EXTRA_ACCOUNT, account) })
val overview = Intent(Settings.ACTION_SYNC_SETTINGS)
.putExtra(Settings.EXTRA_ACCOUNT_TYPES, arrayOf(account.type))
val activity = reactApplicationContext.currentActivity
for (intent in listOf(perAccount, overview)) {
try {
if (activity != null) {
activity.startActivity(intent)
} else {
reactApplicationContext.startActivity(intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
}
return@settle null
} catch (e: ActivityNotFoundException) {
// Try the next one.
}
}
throw IllegalStateException("No settings screen for sync accounts")
}
// ── run handshake ───────────────────────────────────────────
@ReactMethod
fun finishRun(runId: String, reportJson: String, promise: Promise) {
val accepted = DeviceSyncRuns.finish(runId, reportJson)
if (!accepted) Log.i(TAG, "Report for run $runId ignored: nothing waits for it")
promise.resolve(accepted)
}
@ReactMethod
fun isRunCancelled(runId: String, promise: Promise) {
promise.resolve(DeviceSyncRuns.isCancelled(runId))
}
// ── provider I/O ────────────────────────────────────────────
/** `ProviderQuery` JSON in, `ProviderRows` JSON out; rejects with `scope`, `permission` or `provider`. */
@ReactMethod
fun query(name: String, authority: String, queryJson: String, promise: Promise) = onIo(promise) {
ProviderIo(reactApplicationContext, name, authority).use { it.query(queryJson) }
}
/** `ProviderOp[]` JSON in, `BatchResult` JSON out. Never rejects: a failed batch is a result. */
@ReactMethod
fun applyBatch(name: String, authority: String, opsJson: String, promise: Promise) {
val failed = { e: Throwable -> ProviderResults.failed(ProviderIo.failureOf(e), e.message ?: e.javaClass.simpleName) }
try {
io.execute {
val result = try {
ProviderIo(reactApplicationContext, name, authority).use { it.applyBatch(opsJson) }
} catch (e: Throwable) {
failed(e)
}
promise.resolve(result)
}
} catch (e: RejectedExecutionException) {
promise.resolve(failed(e))
}
}
@ReactMethod
fun readSyncState(name: String, authority: String, promise: Promise) = onIo(promise) {
ProviderIo(reactApplicationContext, name, authority).use { it.readSyncState() }
}
/** `PhotoData` JSON, or null when the raw contact has no photo; rejects with `scope` for another account's. */
@ReactMethod
fun readPhoto(name: String, rawContactId: Double, maxPx: Double, promise: Promise) = onIo(promise) {
ProviderIo(reactApplicationContext, name, DeviceSyncAccounts.CONTACTS_AUTHORITY)
.use { it.readPhoto(rawContactId.toLong(), maxPx.toInt()) }
}
/** Settles [promise] with [block]'s result on the I/O thread; rejects with a `BatchFailure` code. */
private fun onIo(promise: Promise, block: () -> Any?) {
try {
io.execute {
try {
promise.resolve(block())
} catch (e: Throwable) {
promise.reject(ProviderIo.failureOf(e).wire, e.message ?: e.javaClass.simpleName, e)
}
}
} catch (e: RejectedExecutionException) {
promise.reject("provider", "Device sync is shutting down", e)
}
}
// ── push routes and notifications ───────────────────────────
/** `{ jmapAccountId: [{ accountName, authorities }] }` for the push router (DeviceSyncPushRouter). */
@ReactMethod
fun setPushRoutes(routesJson: String, promise: Promise) = settle(promise) {
DeviceSyncPushRouter.saveRoutes(reactApplicationContext, routesJson)
null
}
/**
* Posts (or replaces) the account's sync-problem notification. [channelName]
* names the notification channel in the app's language; null keeps English.
*/
@ReactMethod
fun showSyncProblem(
accountName: String,
title: String,
text: String,
uri: String,
channelName: String?,
promise: Promise,
) = settle(promise) {
SyncProblemNotifier.show(reactApplicationContext, accountName, title, text, uri, channelName)
null
}
@ReactMethod
fun clearSyncProblem(accountName: String, promise: Promise) = settle(promise) {
SyncProblemNotifier.clear(reactApplicationContext, accountName)
null
}
// NativeEventEmitter bookkeeping.
@ReactMethod
fun addListener(eventName: String) {}
@ReactMethod
fun removeListeners(count: Int) {}
private inline fun settle(promise: Promise, block: () -> Any?) {
try {
promise.resolve(block())
} catch (e: AccountConflictException) {
promise.reject("conflict", e.message, e)
} catch (e: SecurityException) {
promise.reject("permission", e.message, e)
} catch (e: IllegalArgumentException) {
promise.reject("bad_args", e.message, e)
} catch (e: Exception) {
promise.reject("failed", e.message, e)
}
}
companion object {
const val NAME = "BulwarkDeviceSync"
private const val TAG = "BulwarkDeviceSync"
const val ACCOUNTS_CHANGED_EVENT = "BulwarkDeviceSync:accountsChanged"
/** Settings' per-account sync screen; public on AOSP-based settings apps, but not in the SDK. */
private const val ACTION_ACCOUNT_SYNC_SETTINGS = "android.settings.ACCOUNT_SYNC_SETTINGS"
private const val EXTRA_ACCOUNT = "account"
/** SettingsActivity.EXTRA_SHOW_FRAGMENT_ARGUMENTS: the arguments of the screen it shows. */
private const val EXTRA_SHOW_FRAGMENT_ARGUMENTS = ":settings:show_fragment_args"
@Volatile private var currentInstance: BulwarkDeviceSyncModule? = null
/** Dropped when no React instance or no JS listener is there, like BulwarkFcmModule.emit. */
fun emit(eventName: String, params: WritableMap?) {
val module = currentInstance ?: return
val ctx = module.reactApplicationContext
if (!ctx.hasActiveReactInstance()) return
ctx.getJSModule(RCTDeviceEventEmitter::class.java).emit(eventName, params)
}
}
}
@@ -0,0 +1,144 @@
package com.anonymous.bulwarkmobile.sync
import android.accounts.Account
import android.accounts.AccountManager
import android.content.ContentResolver
import android.content.Context
import android.os.Bundle
import android.provider.CalendarContract
import android.provider.ContactsContract
import com.anonymous.bulwarkmobile.R
/**
* The Android accounts of device sync (#34) and their sync settings. One
* account per app account that has device sync on; see docs/device-sync.md.
*
* Android's own sync settings are the source of truth for on/off: the
* Contacts and Calendar toggles in Settings → Accounts are
* `getSyncAutomatically`, and the app reads them back instead of keeping a
* copy.
*/
object DeviceSyncAccounts {
const val CONTACTS_AUTHORITY: String = ContactsContract.AUTHORITY
const val CALENDAR_AUTHORITY: String = CalendarContract.AUTHORITY
val AUTHORITIES = listOf(CONTACTS_AUTHORITY, CALENDAR_AUTHORITY)
/** userData key holding the app account (`AccountEntry.id`) an Android account belongs to. */
const val USER_DATA_REGISTRY_ID = "registryId"
/** `<applicationId>.account`, set in build.gradle so the package name is never hard-coded. */
fun accountType(context: Context): String = context.getString(R.string.device_sync_account_type)
fun account(context: Context, name: String) = Account(name, accountType(context))
fun list(context: Context): List<Pair<Account, String?>> {
val manager = AccountManager.get(context)
return manager.getAccountsByType(accountType(context)).map { it to manager.getUserData(it, USER_DATA_REGISTRY_ID) }
}
fun exists(context: Context, name: String): Boolean =
AccountManager.get(context).getAccountsByType(accountType(context)).any { it.name == name }
/**
* Adds the account when it is missing. Both authorities start syncable
* with automatic sync off; the app turns on the ones the user picked.
* Returns true when the account was created, false when it exists for
* [registryId] already. An account that belongs to another registry id
* keeps it ([AccountConflictException]): its rows and SyncState were
* written for that app account.
*/
fun ensure(context: Context, name: String, registryId: String): Boolean {
val manager = AccountManager.get(context)
val account = account(context, name)
val exists = manager.getAccountsByType(account.type).any { it.name == name }
val current = if (exists) manager.getUserData(account, USER_DATA_REGISTRY_ID) else null
when (ensureAction(exists, current, registryId)) {
EnsureAction.KEEP -> return false
EnsureAction.ADOPT -> {
manager.setUserData(account, USER_DATA_REGISTRY_ID, registryId)
return false
}
EnsureAction.CONFLICT -> throw AccountConflictException(
"The Android account $name belongs to another app account ($current)",
)
EnsureAction.CREATE -> Unit
}
val userData = Bundle().apply { putString(USER_DATA_REGISTRY_ID, registryId) }
check(manager.addAccountExplicitly(account, null, userData)) { "Android refused to add the account" }
for (authority in AUTHORITIES) {
ContentResolver.setIsSyncable(account, authority, 1)
ContentResolver.setSyncAutomatically(account, authority, false)
}
return true
}
enum class EnsureAction { CREATE, KEEP, ADOPT, CONFLICT }
/** An existing account without a registry id (never completed) is adopted; one with another id is not touched. */
fun ensureAction(exists: Boolean, currentRegistryId: String?, registryId: String): EnsureAction = when {
!exists -> EnsureAction.CREATE
currentRegistryId == registryId -> EnsureAction.KEEP
currentRegistryId.isNullOrEmpty() -> EnsureAction.ADOPT
else -> EnsureAction.CONFLICT
}
/** Removes the account; the Contacts and Calendar providers drop all of its rows. */
fun remove(context: Context, name: String): Boolean {
val manager = AccountManager.get(context)
val account = account(context, name)
if (manager.getAccountsByType(account.type).none { it.name == name }) return false
return manager.removeAccountExplicitly(account)
}
fun setSyncEnabled(context: Context, name: String, authority: String, enabled: Boolean) {
requireAuthority(authority)
val account = account(context, name)
ContentResolver.setIsSyncable(account, authority, 1)
ContentResolver.setSyncAutomatically(account, authority, enabled)
}
/** Replaces our periodic sync for the authority; 0 removes it. */
fun setPeriodicSync(context: Context, name: String, authority: String, seconds: Long) {
requireAuthority(authority)
val account = account(context, name)
for (periodic in ContentResolver.getPeriodicSyncs(account, authority)) {
ContentResolver.removePeriodicSync(account, authority, periodic.extras)
}
if (seconds > 0) ContentResolver.addPeriodicSync(account, authority, Bundle.EMPTY, seconds)
}
/**
* Requests a sync. SyncManager drops a request that matches a sync it is
* running, so a run of that account and authority is also flagged to
* ask for another sync when it ends (see [DeviceSyncRuns.flagSyncAgain]).
*/
fun requestSync(context: Context, name: String, authority: String, flags: SyncRequestFlags) {
requireAuthority(authority)
DeviceSyncRuns.flagSyncAgain(name, authority)
ContentResolver.requestSync(account(context, name), authority, flags.toExtras())
}
fun requireAuthority(authority: String) {
require(authority in AUTHORITIES) { "Unknown authority $authority" }
}
}
/** [DeviceSyncAccounts.ensure] refused to hand an account of one app account to another. */
class AccountConflictException(message: String) : Exception(message)
/** The SYNC_EXTRAS_* flags a request carries. Constant extras let repeated requests coalesce. */
data class SyncRequestFlags(
val manual: Boolean = false,
val expedited: Boolean = false,
val upload: Boolean = false,
val overrideTooManyDeletions: Boolean = false,
val discardLocalDeletions: Boolean = false,
) {
fun toExtras(): Bundle = Bundle().apply {
if (manual) putBoolean(ContentResolver.SYNC_EXTRAS_MANUAL, true)
if (expedited) putBoolean(ContentResolver.SYNC_EXTRAS_EXPEDITED, true)
if (upload) putBoolean(ContentResolver.SYNC_EXTRAS_UPLOAD, true)
if (overrideTooManyDeletions) putBoolean(ContentResolver.SYNC_EXTRAS_OVERRIDE_TOO_MANY_DELETIONS, true)
if (discardLocalDeletions) putBoolean(ContentResolver.SYNC_EXTRAS_DISCARD_LOCAL_DELETIONS, true)
}
}
@@ -0,0 +1,213 @@
package com.anonymous.bulwarkmobile.sync
import android.accounts.Account
import android.accounts.AccountManager
import android.app.Application
import android.app.Service
import android.content.AbstractThreadedSyncAdapter
import android.content.ContentProviderClient
import android.content.ContentResolver
import android.content.Context
import android.content.Intent
import android.content.SyncResult
import android.os.Bundle
import android.os.Handler
import android.os.IBinder
import android.os.Looper
import android.os.SystemClock
import android.util.Log
import androidx.annotation.MainThread
import com.anonymous.bulwarkmobile.HeadlessJs
import com.facebook.react.bridge.Arguments
import com.facebook.react.bridge.WritableMap
import com.facebook.react.jstasks.HeadlessJsTaskConfig
import com.facebook.react.jstasks.HeadlessJsTaskContext
import com.facebook.react.jstasks.HeadlessJsTaskEventListener
import java.util.concurrent.TimeUnit
/**
* The sync adapter behind both authorities (#34). It does no syncing itself:
* it starts the BulwarkDeviceSync headless JS task, where the engine lives
* (src/device-sync), waits for its report and hands that to SyncManager. See
* docs/device-sync.md.
*
* onPerformSync runs on its own SyncAdapterThread and may block; React Native
* is started from the main thread (see HeadlessJs), which stays free.
* SyncManager holds a wake lock for as long as onPerformSync runs.
*/
class DeviceSyncAdapter(context: Context) :
AbstractThreadedSyncAdapter(context, /* autoInitialize = */ true, /* allowParallelSyncs = */ false) {
private val main = Handler(Looper.getMainLooper())
/**
* Anything thrown out of here is rethrown on the sync thread
* (AbstractThreadedSyncAdapter's SyncThread) and kills the process, the UI
* included. So every failure becomes a result: a soft error SyncManager
* retries with back-off, or a database error for a missing permission,
* as the framework would report it.
*/
override fun onPerformSync(
account: Account,
extras: Bundle,
authority: String,
provider: ContentProviderClient,
syncResult: SyncResult,
) {
try {
performSync(account, extras, authority, syncResult)
} catch (e: SecurityException) {
Log.w(TAG, "Sync of $authority refused by a permission check", e)
syncResult.databaseError = true
} catch (t: Throwable) {
Log.e(TAG, "Sync of $authority failed", t)
syncResult.stats.numIoExceptions++
}
}
private fun performSync(account: Account, extras: Bundle, authority: String, syncResult: SyncResult) {
val registryId = AccountManager.get(context).getUserData(account, DeviceSyncAccounts.USER_DATA_REGISTRY_ID)
if (registryId.isNullOrEmpty()) {
Log.w(TAG, "Sync of $authority skipped: the account has no app account")
syncResult.databaseError = true
return
}
val startedAt = SystemClock.elapsedRealtime()
// A sync scheduled as an expedited job (Android 12+) gets 3 minutes
// from JobScheduler instead of 10; stay inside either.
val expedited = extras.getBoolean(SYNC_EXTRAS_SCHEDULE_AS_EXPEDITED_JOB, false)
val deadline = startedAt + if (expedited) EXPEDITED_BUDGET_MS else BUDGET_MS
val run = DeviceSyncRuns.open(account.name, authority)
main.post { startRun(run, account, registryId, authority, extras, deadline) }
try {
while (!run.done.await(POLL_MS, TimeUnit.MILLISECONDS)) {
val boot = run.boot
if (boot != null && boot.isFaulted()) {
run.end("React Native did not start: ${boot.getError()?.message}")
} else if (SystemClock.elapsedRealtime() >= deadline) {
run.end("no report before the deadline")
}
}
val outcome = SyncReports.outcome(run.report, run.endReason)
SyncReports.apply(outcome, syncResult)
Log.i(TAG, "Sync ${run.runId} of $authority ended (${run.endReason}) after " +
"${SystemClock.elapsedRealtime() - startedAt} ms: ${SyncReports.summary(run.report)}; $outcome")
} catch (e: InterruptedException) {
// onSyncCanceled interrupts this thread. JS sees the run as
// cancelled at its next checkpoint (isRunCancelled).
run.end("cancelled")
Log.i(TAG, "Sync ${run.runId} of $authority cancelled")
Thread.currentThread().interrupt()
} finally {
DeviceSyncRuns.close(run)
// After startRun, which was posted first, so a listener it added is removed.
main.post { run.detach() }
}
// A change pushed while the run was open: SyncManager dropped its
// request as a duplicate of this sync, so ask for the next one here.
if (run.syncAgainRequested) {
Log.i(TAG, "Sync ${run.runId} of $authority: changes arrived meanwhile, syncing again")
syncResult.fullSyncRequested = true
}
}
@MainThread
private fun startRun(
run: DeviceSyncRuns.Run,
account: Account,
registryId: String,
authority: String,
extras: Bundle,
deadline: Long,
) {
if (run.cancelled) return
try {
run.boot = HeadlessJs.withReadyReactContext(context.applicationContext as Application) { reactContext ->
if (run.cancelled) return@withReadyReactContext
if (!reactContext.hasActiveReactInstance()) {
run.end("the React Native instance is gone")
return@withReadyReactContext
}
val tasks = HeadlessJsTaskContext.getInstance(reactContext)
run.attach(tasks, object : HeadlessJsTaskEventListener {
override fun onHeadlessJsTaskStart(taskId: Int) = Unit
// JS settled without a report, timed out, or the context was reloaded.
override fun onHeadlessJsTaskFinish(taskId: Int) {
if (taskId == run.taskId) run.end("task finished")
}
})
val taskTimeout = (deadline - SystemClock.elapsedRealtime() - LATCH_SLACK_MS).coerceAtLeast(MIN_TASK_MS)
val data = Arguments.createMap().apply {
putString("runId", run.runId)
putString("accountName", account.name)
putString("registryId", registryId)
putString("authority", authority)
putMap("extras", runExtras(extras))
putDouble("deadline", (System.currentTimeMillis() + taskTimeout - JS_SLACK_MS).toDouble())
}
HeadlessJs.startTask(
tasks,
HeadlessJsTaskConfig(TASK_KEY, data, taskTimeout, /* isAllowedInForeground = */ true),
onStarted = { run.taskId = it },
onRefused = { run.end("refused: ${it.message}") },
)
}
} catch (e: RuntimeException) {
run.end("could not start: ${e.message}")
}
}
private fun runExtras(extras: Bundle): WritableMap = Arguments.createMap().apply {
fun flag(key: String, name: String) {
if (extras.getBoolean(key, false)) putBoolean(name, true)
}
flag(ContentResolver.SYNC_EXTRAS_MANUAL, "manual")
flag(ContentResolver.SYNC_EXTRAS_UPLOAD, "upload")
flag(ContentResolver.SYNC_EXTRAS_EXPEDITED, "expedited")
flag(ContentResolver.SYNC_EXTRAS_IGNORE_BACKOFF, "ignoreBackoff")
flag(ContentResolver.SYNC_EXTRAS_OVERRIDE_TOO_MANY_DELETIONS, "overrideTooManyDeletions")
flag(ContentResolver.SYNC_EXTRAS_DISCARD_LOCAL_DELETIONS, "discardLocalDeletions")
}
companion object {
private const val TAG = "BulwarkDeviceSync"
const val TASK_KEY = "BulwarkDeviceSync"
/** ContentResolver.SYNC_EXTRAS_SCHEDULE_AS_EXPEDITED_JOB (API 31). */
private const val SYNC_EXTRAS_SCHEDULE_AS_EXPEDITED_JOB = "schedule_as_expedited_job"
private const val BUDGET_MS = 570_000L // 9.5 min of JobScheduler's 10
private const val EXPEDITED_BUDGET_MS = 150_000L // 2.5 min of 3
private const val LATCH_SLACK_MS = 20_000L // the task times out this long before our deadline
private const val JS_SLACK_MS = 30_000L // JS aims to be done this long before the task timeout
private const val MIN_TASK_MS = 30_000L
private const val POLL_MS = 500L
}
}
/** Binds the contacts authority's adapter (xml/sync_contacts.xml). */
class ContactsSyncService : Service() {
override fun onBind(intent: Intent): IBinder = adapter(this).syncAdapterBinder
companion object {
@Volatile private var adapter: DeviceSyncAdapter? = null
private fun adapter(context: Context): DeviceSyncAdapter = adapter ?: synchronized(this) {
adapter ?: DeviceSyncAdapter(context.applicationContext).also { adapter = it }
}
}
}
/** Binds the calendar authority's adapter (xml/sync_calendar.xml). */
class CalendarSyncService : Service() {
override fun onBind(intent: Intent): IBinder = adapter(this).syncAdapterBinder
companion object {
@Volatile private var adapter: DeviceSyncAdapter? = null
private fun adapter(context: Context): DeviceSyncAdapter = adapter ?: synchronized(this) {
adapter ?: DeviceSyncAdapter(context.applicationContext).also { adapter = it }
}
}
}
@@ -0,0 +1,125 @@
package com.anonymous.bulwarkmobile.sync
import com.facebook.react.interfaces.TaskInterface
import com.facebook.react.jstasks.HeadlessJsTaskContext
import com.facebook.react.jstasks.HeadlessJsTaskEventListener
import java.util.UUID
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.CountDownLatch
import java.util.concurrent.atomic.AtomicBoolean
import java.util.concurrent.atomic.AtomicReference
/**
* The device sync runs an adapter is waiting for, keyed by a random run id.
*
* A run ends exactly once, on the first of: JS calling finishRun, the
* headless task ending (JS settled, timed out or was reloaded away), React
* refusing or failing to start, the adapter's deadline, or a cancel. A report
* that arrives after that is refused whole rather than half-applied. Headless
* task ids restart at 1 with every React context, so they never identify a
* run; the run id does.
*/
object DeviceSyncRuns {
const val NO_TASK = -1
enum class State { WAITING, ENDED }
class Run(val runId: String, val accountName: String, val authority: String) {
private val state = AtomicReference(State.WAITING)
private val syncAgain = AtomicBoolean(false)
val done = CountDownLatch(1)
/** Written before [done] opens, read after it: the latch orders them. */
@Volatile var report: String? = null
private set
@Volatile var endReason: String? = null
private set
@Volatile var cancelled = false
@Volatile var boot: TaskInterface<Void>? = null
/** Written and compared on the UI thread only. */
@Volatile var taskId = NO_TASK
@Volatile private var tasks: HeadlessJsTaskContext? = null
@Volatile private var listener: HeadlessJsTaskEventListener? = null
val isWaiting: Boolean get() = state.get() == State.WAITING
/** True once a sync was requested for this account and authority while the run was open. */
val syncAgainRequested: Boolean get() = syncAgain.get()
/** Releases the waiting adapter. Only the first end counts; returns whether this was it. */
fun end(reason: String): Boolean = end(reason, null)
internal fun end(reason: String, reportJson: String?): Boolean {
if (!state.compareAndSet(State.WAITING, State.ENDED)) return false
report = reportJson
endReason = reason
done.countDown()
return true
}
internal fun requestSyncAgain() {
syncAgain.set(true)
}
fun attach(tasks: HeadlessJsTaskContext, listener: HeadlessJsTaskEventListener) {
this.tasks = tasks
this.listener = listener
tasks.addTaskEventListener(listener)
}
fun detach() {
val tasks = tasks ?: return
listener?.let { tasks.removeTaskEventListener(it) }
this.tasks = null
this.listener = null
}
}
private val runs = ConcurrentHashMap<String, Run>()
fun open(accountName: String, authority: String): Run =
Run(UUID.randomUUID().toString(), accountName, authority).also { runs[it.runId] = it }
/**
* Forgets the run: a late finishRun is then refused and isCancelled
* answers true. Read [Run.syncAgainRequested] after this, so a request
* can no longer slip in between reading the flag and closing.
*/
fun close(run: Run) {
run.end("closed")
run.cancelled = true
runs.remove(run.runId)
}
/** From JS. Accepted only while the adapter still waits for this run: false when it ended, timed out or is unknown. */
fun finish(runId: String, reportJson: String): Boolean =
runs[runId]?.end("reported", reportJson) ?: false
/** Runs nobody waits for, or that already ended, count as cancelled: JS should stop. */
fun isCancelled(runId: String): Boolean {
val run = runs[runId] ?: return true
return run.cancelled || !run.isWaiting
}
fun cancel(runId: String) {
runs[runId]?.cancelled = true
}
/**
* Flags the open runs of [accountName] and [authority] to request another
* sync when they end (`SyncResult.fullSyncRequested`): SyncManager drops a
* `requestSync` that matches a sync it is running, so a change pushed
* mid-run would otherwise wait for the next trigger. Returns whether a
* run was flagged.
*/
fun flagSyncAgain(accountName: String, authority: String): Boolean {
var flagged = false
for (run in runs.values) {
if (run.accountName == accountName && run.authority == authority) {
run.requestSyncAgain()
flagged = true
}
}
return flagged
}
}
@@ -0,0 +1,376 @@
package com.anonymous.bulwarkmobile.sync
import android.accounts.Account
import android.content.ContentProviderClient
import android.content.ContentProviderOperation
import android.content.ContentUris
import android.content.ContentValues
import android.content.Context
import android.content.OperationApplicationException
import android.database.Cursor
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.net.Uri
import android.os.TransactionTooLargeException
import android.provider.CalendarContract
import android.provider.ContactsContract
import android.provider.SyncStateContract
import android.util.Base64
import org.json.JSONObject
import java.io.ByteArrayOutputStream
import java.io.FileNotFoundException
/**
* Runs device sync's provider calls for one Android account of our type and
* one authority: `query`, `applyBatch`, `readSyncState` and `readPhoto` of
* `NativeModules.BulwarkDeviceSync`. [ScopePlanner] decides what may run; this
* class only builds the calls and reads the answers.
*
* Every URI carries `caller_is_syncadapter=true` and the account
* (CalendarProvider accepts no other query parameter), so writes neither set
* DIRTY nor schedule upload syncs, and ContactsProvider filters data rows by
* the account. The scope lookups are the exception: they read ids and owners
* of rows of any account.
*
* The client is unstable: a provider process that dies fails the call instead
* of taking this process (and the UI) with it. Call from a background thread;
* close when done.
*/
class ProviderIo(context: Context, accountName: String, private val authority: String) : ScopeLookup, AutoCloseable {
private val appContext: Context = context.applicationContext
private val account: AccountRef
private val client: ContentProviderClient
init {
if (authority !in DeviceSyncAccounts.AUTHORITIES) {
throw ProviderRefusal(BatchFailure.SCOPE, "$authority is not a device sync authority")
}
account = AccountRef(accountName, DeviceSyncAccounts.accountType(appContext))
client = appContext.contentResolver.acquireUnstableContentProviderClient(authority)
?: throw IllegalStateException("No content provider for $authority")
}
override fun close() {
client.close()
}
// ── the bridge calls ────────────────────────────────────────
/** `ProviderQuery` JSON in, `ProviderRows` JSON out. Throws [ProviderRefusal] or what the provider threw. */
fun query(queryJson: String): String {
val plan = ScopePlanner(account, this).planQuery(ProviderOpParser.parseQuery(authority, queryJson))
val writer = ProviderRowsWriter(plan.columns)
val projection = plan.columns.toTypedArray()
for (selection in plan.selections) {
query(uri(plan.table), projection, selection, plan.orderBy).use { cursor ->
while (cursor.moveToNext()) {
writer.beginRow()
for (i in projection.indices) {
when (cursor.getType(i)) {
Cursor.FIELD_TYPE_INTEGER -> writer.longCell(cursor.getLong(i))
Cursor.FIELD_TYPE_FLOAT -> writer.doubleCell(cursor.getDouble(i))
Cursor.FIELD_TYPE_STRING -> writer.textCell(cursor.getString(i))
// Blobs (photo thumbnails) never cross the bridge; readPhoto reads them.
else -> writer.nullCell()
}
}
writer.endRow()
}
}
}
return writer.finish()
}
/** `ProviderOp[]` JSON in, `BatchResult` JSON out. Never throws: failures are results. */
fun applyBatch(opsJson: String): String = try {
val ops = ProviderOpParser.parseBatch(authority, opsJson)
// Rows written for an account that is gone would outlive it: the
// providers purge an account's rows only when accounts change.
if (!DeviceSyncAccounts.exists(appContext, account.name)) {
throw ProviderRefusal(BatchFailure.SCOPE, "The account ${account.name} no longer exists")
}
if (ops.isEmpty()) {
ProviderResults.ok(emptyList())
} else {
val planned = ScopePlanner(account, this).planBatch(ops)
val results = client.applyBatch(ArrayList(planned.map(::build)))
ProviderResults.ok(planned.zip(results) { op, result -> outcome(op, result.uri, result.count) })
}
} catch (e: Throwable) {
ProviderResults.failed(failureOf(e), e.message ?: e.javaClass.simpleName)
}
/** The account's SyncState blob as text, or null when there is none. */
fun readSyncState(): String? =
SyncStateContract.Helpers.get(client, syncStateUri(), Account(account.name, account.type))
?.toString(Charsets.UTF_8)
/**
* The raw contact's display photo, or its thumbnail when it has none,
* scaled so the longer side is at most [maxPx], as `PhotoData` JSON; null
* when the raw contact has no photo.
*/
fun readPhoto(rawContactId: Long, maxPx: Int): String? {
require(authority == DeviceSyncAccounts.CONTACTS_AUTHORITY) { "Photos are contacts data" }
require(maxPx > 0) { "maxPx must be positive" }
if (rootAccounts(ProviderTable.RAW_CONTACTS, setOf(rawContactId))[rawContactId] != account) {
throw ProviderRefusal(BatchFailure.SCOPE, "Raw contact $rawContactId is not part of the account")
}
var fileId: Long? = null
var thumbnail: ByteArray? = null
val photoRow = Selection(
"raw_contact_id = ? AND mimetype = ?",
listOf(rawContactId.toString(), ContactsContract.CommonDataKinds.Photo.CONTENT_ITEM_TYPE),
)
query(
uri(ProviderTable.DATA),
arrayOf(ContactsContract.CommonDataKinds.Photo.PHOTO_FILE_ID, ContactsContract.CommonDataKinds.Photo.PHOTO),
photoRow,
"${ContactsContract.Data.IS_PRIMARY} DESC",
).use { cursor ->
if (cursor.moveToFirst()) {
if (!cursor.isNull(0)) fileId = cursor.getLong(0)
if (cursor.getType(1) == Cursor.FIELD_TYPE_BLOB) thumbnail = cursor.getBlob(1)
}
}
val bytes = readDisplayPhoto(rawContactId) ?: thumbnail ?: return null
val jpeg = scaledJpeg(bytes, maxPx) ?: return null
return JSONObject()
.put("jpegBase64", Base64.encodeToString(jpeg, Base64.NO_WRAP))
.put("fileId", fileId ?: JSONObject.NULL)
.toString()
}
private fun readDisplayPhoto(rawContactId: Long): ByteArray? {
val uri = ContentUris.withAppendedId(ContactsContract.RawContacts.CONTENT_URI, rawContactId).buildUpon()
.appendPath(ContactsContract.RawContacts.DisplayPhoto.CONTENT_DIRECTORY)
.scoped()
.build()
return try {
client.openAssetFile(uri, "r")?.use { fd -> fd.createInputStream().use { it.readBytes() } }
} catch (e: FileNotFoundException) {
// No display photo stored (small photos only have the thumbnail).
null
}
}
private fun scaledJpeg(bytes: ByteArray, maxPx: Int): ByteArray? {
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
BitmapFactory.decodeByteArray(bytes, 0, bytes.size, bounds)
if (bounds.outWidth <= 0 || bounds.outHeight <= 0) return null
val options = BitmapFactory.Options().apply {
inSampleSize = PhotoScaling.sampleSize(bounds.outWidth, bounds.outHeight, maxPx)
}
val decoded = BitmapFactory.decodeByteArray(bytes, 0, bytes.size, options) ?: return null
val (width, height) = PhotoScaling.targetSize(decoded.width, decoded.height, maxPx)
val scaled = if (width == decoded.width && height == decoded.height) {
decoded
} else {
Bitmap.createScaledBitmap(decoded, width, height, true)
}
return ByteArrayOutputStream().use { out ->
scaled.compress(Bitmap.CompressFormat.JPEG, JPEG_QUALITY, out)
out.toByteArray()
}
}
// ── ScopeLookup ─────────────────────────────────────────────
override fun ownCalendarIds(): Set<Long> {
val ids = HashSet<Long>()
val selection = Selection(
"${ScopePlanner.ACCOUNT_NAME} = ? AND ${ScopePlanner.ACCOUNT_TYPE} = ?",
listOf(account.name, account.type),
)
query(uri(ProviderTable.CALENDARS), arrayOf(ID), selection, null).use { c ->
while (c.moveToNext()) ids += c.getLong(0)
}
return ids
}
override fun rootAccounts(table: ProviderTable, ids: Set<Long>): Map<Long, AccountRef> {
val owners = HashMap<Long, AccountRef>()
forEachRow(table, arrayOf(ID, ScopePlanner.ACCOUNT_NAME, ScopePlanner.ACCOUNT_TYPE), ID, ids) { c ->
owners[c.getLong(0)] = AccountRef(c.getString(1).orEmpty(), c.getString(2).orEmpty())
}
return owners
}
override fun dataRawContacts(ids: Set<Long>): Map<Long, Long> =
idPairs(ProviderTable.DATA, "raw_contact_id", ids)
override fun dataMimetypes(ids: Set<Long>): Map<Long, String> {
val mimetypes = HashMap<Long, String>()
forEachRow(ProviderTable.DATA, arrayOf(ID, "mimetype"), ID, ids) { c ->
if (!c.isNull(1)) mimetypes[c.getLong(0)] = c.getString(1)
}
return mimetypes
}
override fun eventCalendars(ids: Set<Long>): Map<Long, Long> =
idPairs(ProviderTable.EVENTS, "calendar_id", ids)
override fun childEvents(table: ProviderTable, ids: Set<Long>): Map<Long, Long> =
idPairs(table, "event_id", ids)
override fun childEventIdsMatching(table: ProviderTable, where: String?, args: List<String>): Set<Long> {
val events = HashSet<Long>()
query(uri(table, scoped = false), arrayOf("event_id"), Selection(where, args), null).use { c ->
while (c.moveToNext()) if (!c.isNull(0)) events += c.getLong(0)
}
return events
}
/** Row id → [column] for the rows of [ids] that exist, in any account. */
private fun idPairs(table: ProviderTable, column: String, ids: Set<Long>): Map<Long, Long> {
val pairs = HashMap<Long, Long>()
forEachRow(table, arrayOf(ID, column), table.idColumn!!, ids) { c ->
if (!c.isNull(1)) pairs[c.getLong(0)] = c.getLong(1)
}
return pairs
}
private fun forEachRow(
table: ProviderTable,
projection: Array<String>,
idColumn: String,
ids: Set<Long>,
read: (Cursor) -> Unit,
) {
for (chunk in ids.chunked(ScopePlanner.ID_LIST_CHUNK)) {
val selection = Selection(ScopePlanner.idList(idColumn, chunk), emptyList())
query(uri(table, scoped = false), projection, selection, null).use { c ->
while (c.moveToNext()) read(c)
}
}
}
private fun query(uri: Uri, projection: Array<String>, selection: Selection, orderBy: String?): Cursor =
client.query(uri, projection, selection.sql, selection.args.toTypedArray(), orderBy)
?: throw IllegalStateException("$authority returned no cursor for $uri")
// ── operations ──────────────────────────────────────────────
private fun build(op: PlannedOp): ContentProviderOperation = when (op) {
is PlannedOp.Insert -> ContentProviderOperation.newInsert(uri(op.table))
.withValues(contentValues(op.values))
.apply { for ((column, index) in op.backReferences) withValueBackReference(column, index) }
.withYieldAllowed(op.yieldAllowed)
.build()
is PlannedOp.Update -> ContentProviderOperation
.newUpdate(op.itemId?.let { uri(op.table, itemId = it) } ?: uri(op.table))
.withValues(contentValues(op.values))
.selectRows(op.selection)
.expectRows(op.expectedCount)
.withYieldAllowed(op.yieldAllowed)
.build()
is PlannedOp.Delete -> ContentProviderOperation.newDelete(uri(op.table))
.selectRows(op.selection)
.expectRows(op.expectedCount)
.withYieldAllowed(op.yieldAllowed)
.build()
is PlannedOp.Assert -> ContentProviderOperation.newAssertQuery(uri(op.table))
.selectRows(op.selection)
.apply {
op.values?.let { values ->
withValues(ContentValues().apply { for ((column, text) in values) put(column, text) })
}
}
.expectRows(op.expectedCount)
.withYieldAllowed(op.yieldAllowed)
.build()
// An insert that replaces the account's row; never a yield point.
is PlannedOp.SetSyncState -> SyncStateContract.Helpers.newSetOperation(
syncStateUri(),
Account(account.name, account.type),
op.value.toByteArray(Charsets.UTF_8),
)
}
private fun ContentProviderOperation.Builder.selectRows(selection: Selection) = apply {
// An item URI takes no selection at all (CalendarProvider refuses one).
if (selection.sql != null) withSelection(selection.sql, selection.args.toTypedArray())
}
private fun ContentProviderOperation.Builder.expectRows(count: Int?) = apply {
if (count != null) withExpectedCount(count)
}
private fun outcome(op: PlannedOp, uri: Uri?, count: Int?): OpOutcome = when (op) {
// Settings inserts answer with the account's URI, which has no id.
is PlannedOp.Insert -> OpOutcome(id = uri?.lastPathSegment?.toLongOrNull())
is PlannedOp.SetSyncState -> OpOutcome()
else -> OpOutcome(count = count)
}
private fun contentValues(values: Map<String, Cell>) = ContentValues(values.size).apply {
for ((column, cell) in values) {
when (cell) {
is Cell.Text -> put(column, cell.value)
is Cell.Integer -> put(column, cell.value)
is Cell.Real -> put(column, cell.value)
is Cell.Blob -> put(column, decodeBlob(column, cell.base64))
Cell.Null -> putNull(column)
}
}
}
private fun decodeBlob(column: String, base64: String): ByteArray = try {
Base64.decode(base64, Base64.DEFAULT)
} catch (e: IllegalArgumentException) {
throw ProviderRefusal(BatchFailure.PROVIDER, "$column is not base64")
}
// ── URIs ────────────────────────────────────────────────────
private fun uri(table: ProviderTable, itemId: Long? = null, scoped: Boolean = true): Uri {
val base = when (table) {
ProviderTable.RAW_CONTACTS -> ContactsContract.RawContacts.CONTENT_URI
ProviderTable.DATA -> ContactsContract.Data.CONTENT_URI
ProviderTable.GROUPS -> ContactsContract.Groups.CONTENT_URI
ProviderTable.SETTINGS -> ContactsContract.Settings.CONTENT_URI
ProviderTable.CALENDARS -> CalendarContract.Calendars.CONTENT_URI
ProviderTable.EVENTS -> CalendarContract.Events.CONTENT_URI
ProviderTable.ATTENDEES -> CalendarContract.Attendees.CONTENT_URI
ProviderTable.REMINDERS -> CalendarContract.Reminders.CONTENT_URI
ProviderTable.EXTENDED_PROPERTIES -> CalendarContract.ExtendedProperties.CONTENT_URI
ProviderTable.COLORS -> CalendarContract.Colors.CONTENT_URI
}
val builder = (itemId?.let { ContentUris.withAppendedId(base, it) } ?: base).buildUpon()
return if (scoped) builder.scoped().build() else builder.syncAdapter().build()
}
private fun syncStateUri(): Uri = when (authority) {
DeviceSyncAccounts.CONTACTS_AUTHORITY -> ContactsContract.SyncState.CONTENT_URI
else -> CalendarContract.SyncState.CONTENT_URI
}.buildUpon().scoped().build()
private fun Uri.Builder.syncAdapter() = appendQueryParameter(CALLER_IS_SYNCADAPTER, "true")
private fun Uri.Builder.scoped() = syncAdapter()
.appendQueryParameter(ScopePlanner.ACCOUNT_NAME, account.name)
.appendQueryParameter(ScopePlanner.ACCOUNT_TYPE, account.type)
companion object {
private const val ID = "_id"
private const val JPEG_QUALITY = 85
/** Same parameter name in ContactsContract and CalendarContract. */
private const val CALLER_IS_SYNCADAPTER = ContactsContract.CALLER_IS_SYNCADAPTER
/** The `BatchFailure` of an exception from planning or from the provider (possibly wrapped). */
fun failureOf(e: Throwable): BatchFailure {
var t: Throwable? = e
while (t != null) {
when (t) {
is ProviderRefusal -> return t.reason
is TransactionTooLargeException -> return BatchFailure.TOO_LARGE
is SecurityException -> return BatchFailure.PERMISSION
is OperationApplicationException -> return ProviderResults.failureOfOperationApplication(t.message)
}
t = t.cause?.takeIf { it !== t }
}
return BatchFailure.PROVIDER
}
}
}
@@ -0,0 +1,466 @@
package com.anonymous.bulwarkmobile.sync
import org.json.JSONArray
import org.json.JSONException
import org.json.JSONObject
import java.math.BigDecimal
import java.math.BigInteger
import kotlin.math.abs
/**
* The provider operations of device sync (#34) as the engine sends them: the
* `ProviderQuery` and `ProviderOp[]` JSON of src/device-sync/types.ts, parsed
* into a validated model. [ScopePlanner] turns the model into provider calls
* scoped to one account and [ProviderIo] runs them. Nothing here touches
* Android types, so JVM tests cover all of it.
*/
/** `BatchFailure` of types.ts: why a batch was refused or failed. */
enum class BatchFailure(val wire: String) {
ASSERT("assert"),
TOO_LARGE("tooLarge"),
SCOPE("scope"),
PERMISSION("permission"),
PROVIDER("provider"),
}
/** A query or batch refused before it reached the provider. */
class ProviderRefusal(val reason: BatchFailure, message: String) : Exception(message)
/** How the rows of a table are tied to an account; see [ScopePlanner]. */
enum class TableScope { ACCOUNT_COLUMNS, URI_PARAMETERS, CALENDAR_IDS, EVENT_IDS }
/** The tables the engine may touch, per authority. Any other table is refused. */
enum class ProviderTable(
val wire: String,
val authority: String,
val scope: TableScope,
/** The table whose row each row here belongs to, named by [parentColumn]. */
val parent: ProviderTable? = null,
val parentColumn: String? = null,
/**
* How a selection names the row id; null when rows have none (Settings
* rows are keyed by their account). CalendarProvider queries attendees
* joined with events and calendars, where a bare `_id` is ambiguous.
*/
val idColumn: String? = "_id",
) {
RAW_CONTACTS("raw_contacts", DeviceSyncAccounts.CONTACTS_AUTHORITY, TableScope.ACCOUNT_COLUMNS),
DATA("data", DeviceSyncAccounts.CONTACTS_AUTHORITY, TableScope.URI_PARAMETERS, RAW_CONTACTS, "raw_contact_id"),
GROUPS("groups", DeviceSyncAccounts.CONTACTS_AUTHORITY, TableScope.ACCOUNT_COLUMNS),
SETTINGS("settings", DeviceSyncAccounts.CONTACTS_AUTHORITY, TableScope.ACCOUNT_COLUMNS, idColumn = null),
CALENDARS("calendars", DeviceSyncAccounts.CALENDAR_AUTHORITY, TableScope.ACCOUNT_COLUMNS),
EVENTS("events", DeviceSyncAccounts.CALENDAR_AUTHORITY, TableScope.CALENDAR_IDS, CALENDARS, "calendar_id"),
ATTENDEES(
"attendees", DeviceSyncAccounts.CALENDAR_AUTHORITY, TableScope.EVENT_IDS, EVENTS, "event_id", "Attendees._id",
),
REMINDERS(
"reminders", DeviceSyncAccounts.CALENDAR_AUTHORITY, TableScope.EVENT_IDS, EVENTS, "event_id", "Reminders._id",
),
EXTENDED_PROPERTIES(
"extended_properties",
DeviceSyncAccounts.CALENDAR_AUTHORITY,
TableScope.EVENT_IDS,
EVENTS,
"event_id",
"ExtendedProperties._id",
),
COLORS("colors", DeviceSyncAccounts.CALENDAR_AUTHORITY, TableScope.ACCOUNT_COLUMNS),
;
/** An insert returns the new row id, which later ops can back-reference. */
val insertReturnsId: Boolean get() = idColumn != null
companion object {
fun of(authority: String, wire: String): ProviderTable =
entries.firstOrNull { it.wire == wire && it.authority == authority }
?: throw ProviderRefusal(BatchFailure.SCOPE, "Table $wire is not part of $authority")
}
}
/**
* A cell as it crosses the bridge: text, a number, null, or (written to
* `data.data15` only) photo bytes as base64. Numbers that are integers become
* [Integer], so an assert compares `5` with the provider's `5`, not `5.0`.
*/
sealed interface Cell {
data class Text(val value: String) : Cell
data class Integer(val value: Long) : Cell
data class Real(val value: Double) : Cell
data class Blob(val base64: String) : Cell
data object Null : Cell
/** The text ContentProviderOperation compares an assert with; blobs never reach an assert. */
fun text(): String? = when (this) {
is Text -> value
is Integer -> value.toString()
is Real -> value.toString()
is Blob -> throw IllegalStateException("Blobs have no text form")
Null -> null
}
/** The row id this cell names, if it names one (`12` or `"12"`). */
fun rowId(): Long? = when (this) {
is Integer -> value
is Text -> value.toLongOrNull()
else -> null
}
}
/** Which rows an update, delete or assert addresses: one row by id, a `where` set, or the id row if it matches. */
data class RowTarget(val id: Long?, val where: String?, val args: List<String>)
data class ProviderQuery(
val table: ProviderTable,
val columns: List<String>,
val where: String?,
val args: List<String>,
val orderBy: String?,
)
/** One `ProviderOp` of types.ts, validated. */
sealed interface ProviderOp {
val yieldAllowed: Boolean
data class Insert(
val table: ProviderTable,
val values: Map<String, Cell>,
/** Column → index of an earlier insert of the batch whose new row id fills it. */
val refs: Map<String, Int>,
override val yieldAllowed: Boolean,
) : ProviderOp
data class Update(
val table: ProviderTable,
val target: RowTarget,
val values: Map<String, Cell>,
val expectCount: Int?,
override val yieldAllowed: Boolean,
) : ProviderOp
data class Delete(
val table: ProviderTable,
val target: RowTarget,
val expectCount: Int?,
override val yieldAllowed: Boolean,
) : ProviderOp
data class Assert(
val table: ProviderTable,
val target: RowTarget,
val values: Map<String, Cell>?,
val expectCount: Int?,
override val yieldAllowed: Boolean,
) : ProviderOp
data class SetSyncState(val value: String, override val yieldAllowed: Boolean) : ProviderOp
}
const val GROUP_MEMBERSHIP_MIMETYPE = "vnd.android.cursor.item/group_membership"
/**
* Checks a selection or sort order before the account's scope is ANDed onto
* it as `(text) AND (scope)`: its parentheses must balance outside quoted
* literals, or `1) OR (1` would close the scope's own parenthesis. It may
* not end or comment out the statement, nor query other tables. The engine's
* selections are constants with `?` arguments; this guards against bugs.
*/
object SqlFragments {
private val FORBIDDEN_WORDS = Regex("\\b(SELECT|UNION|ATTACH|DETACH|PRAGMA)\\b", RegexOption.IGNORE_CASE)
fun check(text: String, what: String) {
var depth = 0
var quote: Char? = null
val outside = StringBuilder(text.length)
var i = 0
while (i < text.length) {
val c = text[i]
if (quote != null) {
if (c == quote) {
// A doubled quote is an escaped one inside the literal.
if (i + 1 < text.length && text[i + 1] == quote) i++ else quote = null
}
outside.append(' ')
} else {
when (c) {
'\'', '"' -> {
quote = c
outside.append(' ')
}
'(' -> {
depth++
outside.append(c)
}
')' -> {
if (--depth < 0) refuse(what, "closes a parenthesis it did not open")
outside.append(c)
}
';' -> refuse(what, "ends the statement")
'\u0000' -> refuse(what, "contains a NUL character")
else -> outside.append(c)
}
}
i++
}
if (quote != null) refuse(what, "leaves a quote open")
if (depth != 0) refuse(what, "leaves a parenthesis open")
val bare = outside.toString()
if ("--" in bare || "/*" in bare) refuse(what, "contains a comment")
FORBIDDEN_WORDS.find(bare)?.let { refuse(what, "uses ${it.value.uppercase()}") }
}
private fun refuse(what: String, why: String): Nothing =
throw ProviderRefusal(BatchFailure.SCOPE, "$what $why")
}
/**
* The table a written column points into when the row it names must belong to
* the account as well: the parent of a row, the master of an exception event,
* the group of a membership row.
*/
internal fun referencedTable(table: ProviderTable, column: String, values: Map<String, Cell>): ProviderTable? = when {
column == table.parentColumn -> table.parent
table == ProviderTable.EVENTS && column == "original_id" -> ProviderTable.EVENTS
table == ProviderTable.DATA && column == "data1" &&
values["mimetype"] == Cell.Text(GROUP_MEMBERSHIP_MIMETYPE) -> ProviderTable.GROUPS
else -> null
}
object ProviderOpParser {
/**
* ContactsProvider throws when its op counter reaches this. It counts an
* op before resetting at a yield point, so at most 499 ops may follow a
* yield point up to and including the next one.
*/
const val CONTACTS_OPS_PER_YIELD_POINT = 500
/** JS numbers beyond this are not exact integers any more. */
private const val MAX_SAFE_INTEGER = 9_007_199_254_740_991.0
fun parseQuery(authority: String, json: String): ProviderQuery {
val o = parseObject(json, "query")
val table = ProviderTable.of(authority, o.requireString("table"))
val rawColumns = o.opt("columns") as? JSONArray ?: throw invalid("query needs a columns array")
val columns = (0 until rawColumns.length()).map { i ->
rawColumns.opt(i) as? String ?: throw invalid("columns[$i] must be a string")
}
if (columns.isEmpty()) throw invalid("query needs at least one column")
val (where, args) = o.whereArgs("query")
val orderBy = o.optionalString("orderBy")?.also { SqlFragments.check(it, "query: orderBy") }
return ProviderQuery(table, columns, where, args, orderBy)
}
fun parseBatch(authority: String, json: String): List<ProviderOp> {
val array = try {
JSONArray(json)
} catch (e: JSONException) {
throw invalid("ops are not a JSON array: ${e.message}")
}
val ops = (0 until array.length()).map { i ->
val o = array.opt(i) as? JSONObject ?: throw invalid("op $i is not an object")
parseOp(authority, i, o)
}
checkSyncState(ops)
checkBackReferences(ops)
if (authority == DeviceSyncAccounts.CONTACTS_AUTHORITY) checkYieldPoints(ops)
return ops
}
private fun parseOp(authority: String, index: Int, o: JSONObject): ProviderOp {
val where = "op $index"
val kind = o.opt("op") as? String ?: throw invalid("$where has no op")
val yieldAllowed = o.optBoolean("yieldAllowed", false)
if (kind == "syncState") {
val value = o.opt("value") as? String ?: throw invalid("$where: syncState needs a string value")
return ProviderOp.SetSyncState(value, yieldAllowed)
}
val table = ProviderTable.of(authority, o.requireString("table"))
return when (kind) {
"insert" -> ProviderOp.Insert(table, o.writeValues(table, where), o.refs(where), yieldAllowed)
"update" -> {
val values = o.writeValues(table, where)
if (values.isEmpty()) throw invalid("$where: an update needs values")
ProviderOp.Update(table, o.target(table, where), values, o.expectCount(where), yieldAllowed)
}
"delete" -> ProviderOp.Delete(table, o.target(table, where), o.expectCount(where), yieldAllowed)
"assert" -> {
val target = o.target(table, where)
val values = o.assertValues(where)
val expectCount = o.expectCount(where)
if (values.isNullOrEmpty() && expectCount == null && target.id == null) {
throw invalid("$where: an assert needs values, an expected count or an id")
}
ProviderOp.Assert(table, target, values?.takeIf { it.isNotEmpty() }, expectCount, yieldAllowed)
}
else -> throw invalid("$where: unknown op $kind")
}
}
/**
* The state describes the rows of its batch, so it must commit with them:
* as the last op, and not at a yield point, where the provider may commit
* everything before it on its own.
*/
private fun checkSyncState(ops: List<ProviderOp>) {
ops.forEachIndexed { i, op ->
if (op !is ProviderOp.SetSyncState) return@forEachIndexed
if (i != ops.lastIndex) {
throw ProviderRefusal(BatchFailure.SCOPE, "op $i: syncState must be the last op of its batch")
}
if (op.yieldAllowed) throw ProviderRefusal(BatchFailure.SCOPE, "op $i: syncState must not be a yield point")
}
}
private fun checkBackReferences(ops: List<ProviderOp>) {
ops.forEachIndexed { i, op ->
if (op !is ProviderOp.Insert) return@forEachIndexed
for ((column, ref) in op.refs) {
val target = ops.getOrNull(ref)
if (ref >= i || target !is ProviderOp.Insert || !target.table.insertReturnsId) {
throw invalid("op $i: $column refers to op $ref, which is not an earlier insert with a row id")
}
val expected = referencedTable(op.table, column, op.values)
if (expected != null && target.table != expected) {
throw ProviderRefusal(
BatchFailure.SCOPE,
"op $i: $column must refer to a ${expected.wire} insert, not ${target.table.wire}",
)
}
}
val parentColumn = op.table.parentColumn ?: return@forEachIndexed
val named = parentColumn in op.refs || (op.values[parentColumn] ?: Cell.Null) != Cell.Null
if (!named) {
throw ProviderRefusal(BatchFailure.SCOPE, "op $i: a ${op.table.wire} insert must name its $parentColumn")
}
}
}
private fun checkYieldPoints(ops: List<ProviderOp>) {
var count = 0
ops.forEachIndexed { i, op ->
if (++count >= CONTACTS_OPS_PER_YIELD_POINT) {
throw invalid(
"op $i: more than ${CONTACTS_OPS_PER_YIELD_POINT - 1} ops without a yield point " +
"(ContactsProvider refuses the batch)",
)
}
if (i > 0 && op.yieldAllowed) count = 0
}
}
// ── JSON helpers ────────────────────────────────────────────
private fun invalid(message: String) = ProviderRefusal(BatchFailure.PROVIDER, message)
private fun parseObject(json: String, what: String): JSONObject = try {
JSONObject(json)
} catch (e: JSONException) {
throw invalid("$what is not a JSON object: ${e.message}")
}
private fun JSONObject.requireString(key: String): String =
opt(key) as? String ?: throw invalid("$key must be a string")
private fun JSONObject.optionalString(key: String): String? = when (val v = opt(key)) {
null, JSONObject.NULL -> null
is String -> v.takeIf { it.isNotBlank() }
else -> throw invalid("$key must be a string")
}
private fun JSONObject.target(table: ProviderTable, where: String): RowTarget {
val id = when (val raw = opt("id")) {
null, JSONObject.NULL -> null
else -> (scalar(raw, "$where: id") as? Cell.Integer)?.value?.takeIf { it >= 0 }
?: throw invalid("$where: id must be a row id")
}
if (id != null && table.idColumn == null) throw invalid("$where: ${table.wire} rows have no id")
val (sql, args) = whereArgs(where)
return RowTarget(id, sql, args)
}
private fun JSONObject.whereArgs(where: String): Pair<String?, List<String>> {
val sql = optionalString("where")?.also { SqlFragments.check(it, "$where: where") }
val args = when (val raw = opt("args")) {
null, JSONObject.NULL -> emptyList()
is JSONArray -> (0 until raw.length()).map { i ->
scalar(raw.opt(i), "$where: args[$i]").text()
?: throw invalid("$where: args[$i] must be a string or a number")
}
else -> throw invalid("$where: args must be an array")
}
if (sql == null && args.isNotEmpty()) throw invalid("$where: args without a where")
return sql to args
}
private fun JSONObject.expectCount(where: String): Int? = when (val raw = opt("expectCount")) {
null, JSONObject.NULL -> null
else -> (scalar(raw, "$where: expectCount") as? Cell.Integer)?.value
?.takeIf { it in 0..Int.MAX_VALUE }?.toInt()
?: throw invalid("$where: expectCount must be a count")
}
private fun JSONObject.refs(where: String): Map<String, Int> {
val raw = opt("refs") ?: return emptyMap()
if (raw == JSONObject.NULL) return emptyMap()
if (raw !is JSONObject) throw invalid("$where: refs must be an object")
val out = LinkedHashMap<String, Int>()
for (column in raw.keys()) {
val index = (scalar(raw.opt(column), "$where: refs.$column") as? Cell.Integer)?.value
?.takeIf { it in 0..Int.MAX_VALUE }
?: throw invalid("$where: refs.$column must be an op index")
out[column] = index.toInt()
}
return out
}
private fun JSONObject.writeValues(table: ProviderTable, where: String): Map<String, Cell> {
val raw = opt("values") as? JSONObject ?: throw invalid("$where: values must be an object")
val out = LinkedHashMap<String, Cell>()
for (column in raw.keys()) {
val v = raw.opt(column)
out[column] = if (v is JSONObject) blob(table, column, v, where) else scalar(v, "$where: $column")
}
return out
}
private fun JSONObject.assertValues(where: String): Map<String, Cell>? {
val raw = opt("values") ?: return null
if (raw == JSONObject.NULL) return null
if (raw !is JSONObject) throw invalid("$where: values must be an object")
val out = LinkedHashMap<String, Cell>()
for (column in raw.keys()) out[column] = scalar(raw.opt(column), "$where: $column")
return out
}
/** Only photo rows carry bytes (`data.data15`); the provider stores and scales them itself. */
private fun blob(table: ProviderTable, column: String, v: JSONObject, where: String): Cell.Blob {
if (table != ProviderTable.DATA || column != "data15") {
throw ProviderRefusal(BatchFailure.SCOPE, "$where: bytes are only accepted for data.data15, not ${table.wire}.$column")
}
val base64 = v.opt("b64") as? String ?: throw invalid("$where: $column needs { b64 }")
return Cell.Blob(base64)
}
private fun scalar(v: Any?, what: String): Cell = when (v) {
null, JSONObject.NULL -> Cell.Null
is String -> Cell.Text(v)
is Boolean -> Cell.Integer(if (v) 1 else 0)
is Number -> numberCell(v)
else -> throw invalid("$what must be text, a number or null")
}
/** org.json yields Integer, Long, Double, BigInteger or BigDecimal depending on the platform and the value. */
internal fun numberCell(n: Number): Cell = when (n) {
is Int, is Long, is Short, is Byte -> Cell.Integer(n.toLong())
is BigInteger -> if (n.bitLength() < 64) Cell.Integer(n.toLong()) else Cell.Real(n.toDouble())
is BigDecimal -> try {
Cell.Integer(n.longValueExact())
} catch (e: ArithmeticException) {
Cell.Real(n.toDouble())
}
else -> {
val d = n.toDouble()
if (d.isFinite() && d == Math.rint(d) && abs(d) <= MAX_SAFE_INTEGER) Cell.Integer(d.toLong()) else Cell.Real(d)
}
}
}
@@ -0,0 +1,549 @@
package com.anonymous.bulwarkmobile.sync
import org.json.JSONArray
import org.json.JSONObject
/**
* Plans provider calls for one account: every selection gets the account
* condition ANDed in, and every row an operation names by id, or points to
* from a written column, is checked to belong to the account first
* (docs/device-sync.md, "Native module API", Scoping).
*
* How a table is tied to the account ([TableScope]):
* - raw contacts, groups, settings, calendars and colors carry it in their
* `account_name`/`account_type` columns;
* - contacts data rows are filtered by ContactsProvider itself through the
* `account_name`/`account_type` URI parameters. Their account columns are
* never selected on: Android 17 drops them from the Data view for apps
* targeting more than API 36;
* - events get `calendar_id IN (…)` of the account's calendars, not the
* account columns: CalendarProvider updates events on the Events table,
* which has none;
* - attendees, reminders and extended properties get `event_id IN (…)` of the
* account's events among those the operation touches. CalendarProvider
* applies no account filter to them.
*
* Selections never use subqueries: the providers may compile them strictly.
*/
class ScopePlanner(private val account: AccountRef, private val lookup: ScopeLookup) {
private val calendarIds: Set<Long> by lazy { lookup.ownCalendarIds() }
fun planQuery(query: ProviderQuery): PlannedQuery {
val target = RowTarget(null, query.where, query.args)
val selections = if (query.table.scope == TableScope.EVENT_IDS) {
// Mostly a handful of events (the engine reads children per chunk
// of events); a query over all of them is split so no selection
// outgrows a Binder transaction. Rows are then ordered per chunk.
ownEventsMatching(query.table, target)
.sorted()
.chunked(ID_LIST_CHUNK)
.map { chunk -> selection(query.table, target, chunk) }
} else {
listOf(selection(query.table, target, null))
}
return PlannedQuery(query.table, query.columns, selections, query.orderBy)
}
fun planBatch(ops: List<ProviderOp>): List<PlannedOp> {
val memberships = membershipUpdates(ops)
val owners = Owners(ownershipRequests(ops, memberships))
return ops.mapIndexed { i, op -> plan(i, op, owners, memberships) }
}
/**
* Membership rows among the data rows that updates by id write `data1`
* of without restating the mimetype: their `data1` names a group, which
* must be the account's like on insert.
*/
private fun membershipUpdates(ops: List<ProviderOp>): Set<Long> {
val ids = ops.asSequence()
.filterIsInstance<ProviderOp.Update>()
.filter { it.table == ProviderTable.DATA && writesUntypedData1(it.values) }
.mapNotNull { it.target.id }
.toSet()
if (ids.isEmpty()) return emptySet()
return lookup.dataMimetypes(ids).filterValues { it == GROUP_MEMBERSHIP_MIMETYPE }.keys
}
private fun writesUntypedData1(values: Map<String, Cell>) =
MIMETYPE !in values && (values[DATA1] ?: Cell.Null) != Cell.Null
private fun plan(i: Int, op: ProviderOp, owners: Owners, memberships: Set<Long>): PlannedOp = when (op) {
is ProviderOp.SetSyncState -> PlannedOp.SetSyncState(op.value)
is ProviderOp.Insert -> {
checkAccountColumns(i, op.table, op.values)
checkPhotoFile(i, op.table, op.values)
checkReferences(i, op.table, op.values - op.refs.keys, owners)
val values = if (op.table.scope == TableScope.ACCOUNT_COLUMNS) op.values + accountCells() else op.values
PlannedOp.Insert(op.table, values, op.refs, op.yieldAllowed)
}
is ProviderOp.Update -> {
checkAccountColumns(i, op.table, op.values)
checkPhotoFile(i, op.table, op.values)
checkReferences(i, op.table, op.values, owners)
if (op.table == ProviderTable.DATA && writesUntypedData1(op.values)) {
checkMembershipUpdate(i, op, owners, memberships)
}
checkTarget(i, op.table, op.target, owners)
if (op.table == ProviderTable.EXTENDED_PROPERTIES) {
// CalendarProvider updates extended properties only through
// their item URI, which takes no selection: the id check above
// is the scope.
val id = op.target.id
if (id == null || op.target.where != null) {
throw ProviderRefusal(BatchFailure.PROVIDER, "op $i: extended properties can only be updated by id")
}
PlannedOp.Update(op.table, Selection.NONE, id, op.values, op.expectCount ?: 1, op.yieldAllowed)
} else {
val selection = selection(op.table, op.target, eventScope(op.table, op.target, owners))
PlannedOp.Update(op.table, selection, null, op.values, expectedCount(op.target, op.expectCount), op.yieldAllowed)
}
}
is ProviderOp.Delete -> {
checkTarget(i, op.table, op.target, owners)
val selection = selection(op.table, op.target, eventScope(op.table, op.target, owners))
PlannedOp.Delete(op.table, selection, expectedCount(op.target, op.expectCount), op.yieldAllowed)
}
is ProviderOp.Assert -> {
checkTarget(i, op.table, op.target, owners)
val selection = selection(op.table, op.target, eventScope(op.table, op.target, owners))
val values = op.values?.mapValues { (_, cell) -> cell.text() }
PlannedOp.Assert(op.table, selection, values, expectedCount(op.target, op.expectCount), op.yieldAllowed)
}
}
/** An op that names one row by id expects exactly that row: the engine relies on it to notice a vanished row. */
private fun expectedCount(target: RowTarget, expectCount: Int?): Int? =
expectCount ?: if (target.id != null) 1 else null
private fun accountCells() = mapOf(
ACCOUNT_NAME to Cell.Text(account.name),
ACCOUNT_TYPE to Cell.Text(account.type),
)
/**
* Account columns may only restate the account, on the tables that have
* them; a data row, event or attendee follows its parent's account.
*/
private fun checkAccountColumns(i: Int, table: ProviderTable, values: Map<String, Cell>) {
for (column in listOf(ACCOUNT_NAME, ACCOUNT_TYPE, DATA_SET)) {
val cell = values[column] ?: continue
val restatesAccount = table.scope == TableScope.ACCOUNT_COLUMNS && when (column) {
ACCOUNT_NAME -> cell == Cell.Text(account.name)
ACCOUNT_TYPE -> cell == Cell.Text(account.type)
else -> cell == Cell.Null
}
if (!restatesAccount) {
throw ProviderRefusal(BatchFailure.SCOPE, "op $i: ${table.wire}.$column would leave the account")
}
}
}
/**
* Parents, masters and groups a written column names must be rows of the
* account that still exist. (Back-references were checked by the parser:
* they name inserts of the same batch.)
*/
private fun checkReferences(i: Int, table: ProviderTable, values: Map<String, Cell>, owners: Owners) {
for ((column, cell) in values) {
val referenced = referencedTable(table, column, values) ?: continue
if (cell == Cell.Null) {
if (column == table.parentColumn) {
throw ProviderRefusal(BatchFailure.SCOPE, "op $i: ${table.wire}.$column must name a ${referenced.wire} row")
}
continue
}
val id = cell.rowId()
?: throw ProviderRefusal(BatchFailure.PROVIDER, "op $i: ${table.wire}.$column must be a row id")
when (owners.of(referenced, id)) {
Owner.OURS -> Unit
Owner.FOREIGN -> throw ProviderRefusal(
BatchFailure.SCOPE,
"op $i: ${table.wire}.$column names ${referenced.wire} $id of another account",
)
// Deleted meanwhile, as far as the engine can tell: re-read and re-plan.
Owner.MISSING -> throw ProviderRefusal(
BatchFailure.ASSERT,
"op $i: ${table.wire}.$column names ${referenced.wire} $id, which no longer exists",
)
}
}
}
/**
* `data1` written by an update that doesn't restate the mimetype: on a
* membership row it names a group. Rows chosen by a selection could be
* memberships the lookup didn't see, so only updates by id may do it.
*/
private fun checkMembershipUpdate(i: Int, op: ProviderOp.Update, owners: Owners, memberships: Set<Long>) {
val id = op.target.id
?: throw ProviderRefusal(BatchFailure.SCOPE, "op $i: data1 may only be written to a data row by id")
if (id !in memberships) return
val group = op.values[DATA1]?.rowId()
?: throw ProviderRefusal(BatchFailure.PROVIDER, "op $i: a membership's data1 must be a group id")
when (owners.of(ProviderTable.GROUPS, group)) {
Owner.OURS -> Unit
Owner.FOREIGN -> throw ProviderRefusal(BatchFailure.SCOPE, "op $i: data.data1 names group $group of another account")
Owner.MISSING -> throw ProviderRefusal(BatchFailure.ASSERT, "op $i: data.data1 names group $group, which no longer exists")
}
}
/**
* A photo row's file id (`data14`) is the provider's: it points into the
* photo store, where another account's file id would let `readPhoto`
* read that account's photo. Photos are written as bytes (`data15`).
*/
private fun checkPhotoFile(i: Int, table: ProviderTable, values: Map<String, Cell>) {
if (table == ProviderTable.DATA && (values[DATA14] ?: Cell.Null) != Cell.Null) {
throw ProviderRefusal(BatchFailure.SCOPE, "op $i: data.data14 (a photo file id) is written by the provider only")
}
}
/** A row named by id must not belong to another account; a missing one fails on the expected count instead. */
private fun checkTarget(i: Int, table: ProviderTable, target: RowTarget, owners: Owners) {
val id = target.id ?: return
if (owners.of(table, id) == Owner.FOREIGN) {
throw ProviderRefusal(BatchFailure.SCOPE, "op $i: ${table.wire} $id belongs to another account")
}
}
/** For attendees, reminders and extended properties: the account's events the op may touch. */
private fun eventScope(table: ProviderTable, target: RowTarget, owners: Owners): Collection<Long>? {
if (table.scope != TableScope.EVENT_IDS) return null
val id = target.id ?: return ownEventsMatching(table, target)
return listOfNotNull(owners.eventOf(table, id))
}
private fun ownEventsMatching(table: ProviderTable, target: RowTarget): Set<Long> {
val candidates = lookup.childEventIdsMatching(table, target.where, target.args)
if (candidates.isEmpty()) return emptySet()
return lookup.eventCalendars(candidates).filterValues { it in calendarIds }.keys
}
private fun selection(table: ProviderTable, target: RowTarget, eventIds: Collection<Long>?): Selection {
val parts = ArrayList<String>(3)
val args = ArrayList<String>()
target.id?.let {
parts += "${table.idColumn} = ?"
args += it.toString()
}
target.where?.let {
parts += it
args += target.args
}
when (table.scope) {
TableScope.ACCOUNT_COLUMNS -> {
parts += "$ACCOUNT_NAME = ? AND $ACCOUNT_TYPE = ?"
args += account.name
args += account.type
}
TableScope.URI_PARAMETERS -> Unit
TableScope.CALENDAR_IDS -> parts += idList("calendar_id", calendarIds)
TableScope.EVENT_IDS -> parts += idList("event_id", eventIds.orEmpty())
}
return Selection(parts.takeIf { it.isNotEmpty() }?.joinToString(" AND ") { "($it)" }, args)
}
/** Every row id the batch names, by table: op targets and referenced rows. */
private fun ownershipRequests(ops: List<ProviderOp>, memberships: Set<Long>): Map<ProviderTable, Set<Long>> {
val requests = HashMap<ProviderTable, MutableSet<Long>>()
fun request(table: ProviderTable, id: Long) {
requests.getOrPut(table) { HashSet() } += id
}
fun requestReferences(table: ProviderTable, values: Map<String, Cell>) {
for ((column, cell) in values) {
val referenced = referencedTable(table, column, values) ?: continue
cell.rowId()?.let { request(referenced, it) }
}
}
for (op in ops) {
when (op) {
is ProviderOp.Insert -> requestReferences(op.table, op.values)
is ProviderOp.Update -> {
requestReferences(op.table, op.values)
op.target.id?.let { request(op.table, it) }
if (op.target.id in memberships) op.values[DATA1]?.rowId()?.let { request(ProviderTable.GROUPS, it) }
}
is ProviderOp.Delete -> op.target.id?.let { request(op.table, it) }
is ProviderOp.Assert -> op.target.id?.let { request(op.table, it) }
is ProviderOp.SetSyncState -> Unit
}
}
return requests
}
private enum class Owner { OURS, FOREIGN, MISSING }
/** Who owns the rows a batch names, resolved up front with as few lookups as the tables allow. */
private inner class Owners(requests: Map<ProviderTable, Set<Long>>) {
private val owners = HashMap<ProviderTable, Map<Long, Owner>>()
private val childEvents = HashMap<ProviderTable, Map<Long, Long>>()
init {
// Children lead to events, data rows to raw contacts; both are
// added to their parent's lookup.
val eventIds = HashSet(requests[ProviderTable.EVENTS].orEmpty())
for (table in CHILD_TABLES) {
val ids = requests[table] ?: continue
val events = lookup.childEvents(table, ids)
childEvents[table] = events
eventIds += events.values
}
val rawContactIds = HashSet(requests[ProviderTable.RAW_CONTACTS].orEmpty())
val dataParents = requests[ProviderTable.DATA]?.let { lookup.dataRawContacts(it) }.orEmpty()
rawContactIds += dataParents.values
if (eventIds.isNotEmpty()) {
owners[ProviderTable.EVENTS] = lookup.eventCalendars(eventIds)
.mapValues { (_, calendarId) -> if (calendarId in calendarIds) Owner.OURS else Owner.FOREIGN }
}
for ((table, events) in childEvents) {
// A child whose event is gone is an orphan, not ours.
owners[table] = events.mapValues { (_, eventId) -> ownerOrForeign(ProviderTable.EVENTS, eventId) }
}
val roots = HashMap<ProviderTable, Set<Long>>()
for (table in ROOT_TABLES) requests[table]?.let { roots[table] = it }
if (rawContactIds.isNotEmpty()) roots[ProviderTable.RAW_CONTACTS] = rawContactIds
for ((table, ids) in roots) {
owners[table] = lookup.rootAccounts(table, ids)
.mapValues { (_, owner) -> if (owner == account) Owner.OURS else Owner.FOREIGN }
}
if (dataParents.isNotEmpty()) {
owners[ProviderTable.DATA] = dataParents.mapValues { (_, rawContactId) ->
ownerOrForeign(ProviderTable.RAW_CONTACTS, rawContactId)
}
}
}
private fun ownerOrForeign(table: ProviderTable, id: Long): Owner =
owners[table]?.get(id)?.takeIf { it != Owner.MISSING } ?: Owner.FOREIGN
fun of(table: ProviderTable, id: Long): Owner = owners[table]?.get(id) ?: Owner.MISSING
/** The event of a child row, when both exist and are the account's. */
fun eventOf(table: ProviderTable, id: Long): Long? =
childEvents[table]?.get(id)?.takeIf { of(table, id) == Owner.OURS }
}
companion object {
const val ACCOUNT_NAME = "account_name"
const val ACCOUNT_TYPE = "account_type"
const val DATA_SET = "data_set"
private const val MIMETYPE = "mimetype"
private const val DATA1 = "data1"
private const val DATA14 = "data14"
/** Ids per `IN (…)` list: a few KB of selection, far from SQLite's and Binder's limits. */
const val ID_LIST_CHUNK = 500
private val CHILD_TABLES = ProviderTable.entries.filter { it.scope == TableScope.EVENT_IDS }
private val ROOT_TABLES = ProviderTable.entries.filter { it.scope == TableScope.ACCOUNT_COLUMNS }
/** `column IN (1,2,3)`, or a condition no row meets. The ids are ours to inline: numbers we read. */
fun idList(column: String, ids: Collection<Long>): String =
if (ids.isEmpty()) "0" else ids.sorted().joinToString(",", "$column IN (", ")")
}
}
/** An Android account of our type: the scope of every provider call. */
data class AccountRef(val name: String, val type: String)
/**
* What [ScopePlanner] needs to know about rows beyond the operations
* themselves. [ProviderIo] answers with provider queries: except for
* [ownCalendarIds], these read rows of any account, but only their ids and
* owners, to tell rows of another account from rows that are gone.
*/
interface ScopeLookup {
/** `_id`s of the account's calendars. */
fun ownCalendarIds(): Set<Long>
/** The account of each row of a root table that exists. */
fun rootAccounts(table: ProviderTable, ids: Set<Long>): Map<Long, AccountRef>
/** The raw contact of each data row that exists. */
fun dataRawContacts(ids: Set<Long>): Map<Long, Long>
/** The mimetype of each data row that exists. */
fun dataMimetypes(ids: Set<Long>): Map<Long, String>
/** The calendar of each event that exists. */
fun eventCalendars(ids: Set<Long>): Map<Long, Long>
/** The event of each attendee, reminder or extended property that exists. */
fun childEvents(table: ProviderTable, ids: Set<Long>): Map<Long, Long>
/** The events of the rows of a child table that match [where]. */
fun childEventIdsMatching(table: ProviderTable, where: String?, args: List<String>): Set<Long>
}
/** A selection with its `?` arguments; a null [sql] selects every row the URI allows. */
data class Selection(val sql: String?, val args: List<String>) {
companion object {
val NONE = Selection(null, emptyList())
}
}
/** A provider query, split into [selections] when its scope list is long; no selection means no rows. */
data class PlannedQuery(
val table: ProviderTable,
val columns: List<String>,
val selections: List<Selection>,
val orderBy: String?,
)
/** One ContentProviderOperation, ready to build on the account's sync-adapter URI. */
sealed interface PlannedOp {
val yieldAllowed: Boolean
data class Insert(
val table: ProviderTable,
val values: Map<String, Cell>,
val backReferences: Map<String, Int>,
override val yieldAllowed: Boolean,
) : PlannedOp
/** [itemId] set: an update through the row's own URI, without a selection. */
data class Update(
val table: ProviderTable,
val selection: Selection,
val itemId: Long?,
val values: Map<String, Cell>,
val expectedCount: Int?,
override val yieldAllowed: Boolean,
) : PlannedOp
data class Delete(
val table: ProviderTable,
val selection: Selection,
val expectedCount: Int?,
override val yieldAllowed: Boolean,
) : PlannedOp
/** [values] compared as text, as ContentProviderOperation does. */
data class Assert(
val table: ProviderTable,
val selection: Selection,
val values: Map<String, String?>?,
val expectedCount: Int?,
override val yieldAllowed: Boolean,
) : PlannedOp
data class SetSyncState(val value: String) : PlannedOp {
override val yieldAllowed: Boolean get() = false
}
}
/** The result of one op, as `OpResult` of types.ts. */
data class OpOutcome(val id: Long? = null, val count: Int? = null)
object ProviderResults {
fun ok(results: List<OpOutcome>): String {
val array = JSONArray()
for (result in results) {
array.put(JSONObject().apply {
result.id?.let { put("id", it) }
result.count?.let { put("count", it) }
})
}
return JSONObject().put("ok", true).put("results", array).toString()
}
fun failed(reason: BatchFailure, message: String?): String = JSONObject()
.put("ok", false)
.put("reason", reason.wire)
.put("message", message ?: reason.wire)
.toString()
/**
* An OperationApplicationException is an assert that did not hold or an
* expected count that did not match (`assert`), except when an insert got
* no row back or ContactsProvider refused the batch's yield points.
*/
fun failureOfOperationApplication(message: String?): BatchFailure = when {
message == null -> BatchFailure.ASSERT
message.startsWith("Insert into ") -> BatchFailure.PROVIDER
message.startsWith("Too many content provider operations") -> BatchFailure.PROVIDER
else -> BatchFailure.ASSERT
}
}
/**
* Query results as `ProviderRows` JSON, built by hand: a chunk of contacts
* can carry megabytes of shadows, and this is the hot path of every run.
*/
class ProviderRowsWriter(columns: List<String>) {
private val out = StringBuilder(4096)
private var rows = 0
private var cells = 0
init {
out.append("{\"columns\":[")
columns.forEachIndexed { i, column ->
if (i > 0) out.append(',')
appendString(column)
}
out.append("],\"rows\":[")
}
fun beginRow() {
if (rows++ > 0) out.append(',')
out.append('[')
cells = 0
}
fun endRow() {
out.append(']')
}
fun nullCell() = cell { out.append("null") }
fun longCell(value: Long) = cell { out.append(value) }
fun doubleCell(value: Double) = cell { if (value.isFinite()) out.append(value) else out.append("null") }
fun textCell(value: String) = cell { appendString(value) }
fun finish(): String = out.append("]}").toString()
private inline fun cell(write: () -> Unit) {
if (cells++ > 0) out.append(',')
write()
}
private fun appendString(s: String) {
out.append('"')
for (c in s) {
when {
c == '"' -> out.append("\\\"")
c == '\\' -> out.append("\\\\")
c == '\n' -> out.append("\\n")
c == '\r' -> out.append("\\r")
c == '\t' -> out.append("\\t")
c < ' ' || c.code == 0x2028 || c.code == 0x2029 ->
out.append("\\u").append(Integer.toHexString(c.code).padStart(4, '0'))
else -> out.append(c)
}
}
out.append('"')
}
}
/** Sizes for [ProviderIo.readPhoto]: decode close to the target, then scale the longer side down to it. */
object PhotoScaling {
/** The largest power of two that keeps the decoded image at least [maxPx] on its longer side. */
fun sampleSize(width: Int, height: Int, maxPx: Int): Int {
require(maxPx > 0) { "maxPx must be positive" }
var sample = 1
val longer = maxOf(width, height)
while (longer / (sample * 2) >= maxPx) sample *= 2
return sample
}
fun targetSize(width: Int, height: Int, maxPx: Int): Pair<Int, Int> {
val longer = maxOf(width, height)
if (longer <= maxPx) return width to height
val scale = maxPx.toDouble() / longer
return maxOf(1, Math.round(width * scale).toInt()) to maxOf(1, Math.round(height * scale).toInt())
}
}
@@ -0,0 +1,173 @@
package com.anonymous.bulwarkmobile.sync
import android.content.Context
import android.util.Log
import org.json.JSONArray
import org.json.JSONException
import org.json.JSONObject
/**
* What a push asks for (#34): syncs of the Android accounts that contacts and
* calendar changes feed, and whether the mail push task should run at all.
*
* The relay forwards the JMAP StateChange's `changed` map verbatim (FCM data
* key `changed`: JMAP account id → type → state; only its first account is
* repeated as `accountId`), so every account key is looked at. The mail task
* runs only for mail: when some account's map has a mail type, for anything
* but a StateChange (an EmailPush), when `changed` is missing, unreadable or
* names no type (older relays), or when an account's entry is unreadable, so
* a contact change never ends in a mail notification. The push task's guard
* (`carriesNoMail` in src/lib/push-background-task.ts) applies the same rules.
*/
object PushRouting {
/** An Android account and the authorities it syncs, fed by one JMAP account. */
data class Route(val accountName: String, val authorities: Set<String>)
data class Decision(
/** (Android account name, authority) pairs to request a sync for. */
val syncs: Set<Pair<String, String>>,
val startMailTask: Boolean,
)
private const val STATE_CHANGE_KIND = "jmap-state-change"
/**
* Types that may mean new mail: `EmailDelivery`, and `Email`/`Mailbox`,
* which new mail changes too. A subscription made by an older build
* listens to them until the launch-time resync replaces its types, and a
* server without `EmailDelivery` has only them. Keep in sync with
* MAIL_TYPES in src/lib/push-background-task.ts.
*/
private val MAIL_TYPES = setOf("EmailDelivery", "Email", "Mailbox")
private val CONTACT_TYPES = setOf("ContactCard", "AddressBook")
private val CALENDAR_TYPES = setOf("CalendarEvent", "Calendar")
fun decide(data: Map<String, String>, routes: Map<String, List<Route>>): Decision {
val changed = data["changed"]?.let(::parseChanged)
// Only a StateChange can say it carries no mail.
var mail = data["kind"] != STATE_CHANGE_KIND || changed == null
val syncs = LinkedHashSet<Pair<String, String>>()
for ((jmapAccountId, types) in changed.orEmpty()) {
if (types == null) {
// One account's entry is unreadable: it may be mail.
mail = true
continue
}
if (types.any { it in MAIL_TYPES }) mail = true
val contacts = types.any { it in CONTACT_TYPES }
val calendar = types.any { it in CALENDAR_TYPES }
for (route in routes[jmapAccountId].orEmpty()) {
if (contacts && DeviceSyncAccounts.CONTACTS_AUTHORITY in route.authorities) {
syncs += route.accountName to DeviceSyncAccounts.CONTACTS_AUTHORITY
}
if (calendar && DeviceSyncAccounts.CALENDAR_AUTHORITY in route.authorities) {
syncs += route.accountName to DeviceSyncAccounts.CALENDAR_AUTHORITY
}
}
}
return Decision(syncs, mail)
}
/**
* JMAP account id → its changed types (null for an entry that is not an
* object). Null for a map that is unreadable or names no type at all,
* which the relay sends as `{}` when it had no map.
*/
private fun parseChanged(json: String): Map<String, Set<String>?>? {
val o = try {
JSONObject(json)
} catch (e: JSONException) {
return null
}
val out = LinkedHashMap<String, Set<String>?>()
for (jmapAccountId in o.keys()) {
val types = o.opt(jmapAccountId) as? JSONObject
out[jmapAccountId] = types?.keys()?.asSequence()?.toSet()
}
return out.takeIf { entries -> entries.values.any { it == null || it.isNotEmpty() } }
}
/**
* `{ jmapAccountId: [{ accountName, authorities }] }` as JS stores it.
* [strict] refuses anything malformed (IllegalArgumentException); otherwise
* malformed entries are skipped, so a bad route never stops mail.
*/
fun parseRoutes(json: String?, strict: Boolean = false): Map<String, List<Route>> {
if (json.isNullOrEmpty()) return emptyMap()
fun bad(message: String): Nothing = throw IllegalArgumentException("Push routes: $message")
val o = try {
JSONObject(json)
} catch (e: JSONException) {
if (strict) bad("not a JSON object") else return emptyMap()
}
val out = LinkedHashMap<String, List<Route>>()
for (jmapAccountId in o.keys()) {
val entries = o.opt(jmapAccountId) as? JSONArray
?: if (strict) bad("$jmapAccountId must map to an array") else continue
val routes = ArrayList<Route>()
for (i in 0 until entries.length()) {
val entry = entries.opt(i) as? JSONObject
val name = entry?.opt("accountName") as? String
val authorities = entry?.opt("authorities") as? JSONArray
if (name.isNullOrEmpty() || authorities == null) {
if (strict) bad("$jmapAccountId[$i] needs accountName and authorities") else continue
}
val known = (0 until authorities.length()).mapNotNull { j ->
val authority = authorities.opt(j) as? String
if (authority !in DeviceSyncAccounts.AUTHORITIES) {
if (strict) bad("$jmapAccountId[$i]: unknown authority $authority") else null
} else {
authority
}
}
routes += Route(name, known.toSet())
}
out[jmapAccountId] = routes
}
return out
}
}
/**
* Applies [PushRouting] to incoming pushes (BulwarkMessagingService,
* BulwarkUnifiedPushService) with the routes JS stored through
* `setPushRoutes`.
*/
object DeviceSyncPushRouter {
private const val TAG = "BulwarkDeviceSync"
private const val PREFS = "bulwark_device_sync"
private const val KEY_ROUTES = "pushRoutes"
/** Validates and stores the routes; they must survive the process, pushes arrive without JS. */
fun saveRoutes(context: Context, json: String) {
PushRouting.parseRoutes(json, strict = true)
val saved = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit()
.putString(KEY_ROUTES, json)
.commit()
check(saved) { "Push routes could not be saved" }
}
/**
* Requests the syncs the push asks for, with constant (empty) extras so
* repeated pushes coalesce into one pending sync. Returns whether the mail
* push task should run; any failure here says yes, so mail never depends
* on device sync.
*/
fun route(context: Context, data: Map<String, String>): Boolean = try {
val routes = PushRouting.parseRoutes(
context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getString(KEY_ROUTES, null),
)
val decision = PushRouting.decide(data, routes)
for ((accountName, authority) in decision.syncs) {
try {
DeviceSyncAccounts.requestSync(context, accountName, authority, SyncRequestFlags())
} catch (e: RuntimeException) {
Log.w(TAG, "Push: no sync requested for $authority", e)
}
}
decision.startMailTask
} catch (e: Exception) {
Log.w(TAG, "Push not routed", e)
true
}
}
@@ -0,0 +1,75 @@
package com.anonymous.bulwarkmobile.sync
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.content.Context
import android.content.Intent
import android.net.Uri
import android.os.Build
import androidx.core.app.NotificationCompat
import com.anonymous.bulwarkmobile.MainActivity
import com.anonymous.bulwarkmobile.R
/**
* The one notification per Android account that device sync needs the user
* for (#34), e.g. "Sign in again" after the server rejected the credentials.
* The engine posts it through `showSyncProblem`; the app clears it once the
* problem is solved. Tapping it opens a `bulwarkmobile://` deep link in the
* app.
*/
object SyncProblemNotifier {
const val CHANNEL_ID = "bulwark_sync"
private const val DEFAULT_CHANNEL_NAME = "Sync problems"
private const val TAG_PREFIX = "bulwark-sync:"
private const val NOTIFICATION_ID = 1
private const val DEEP_LINK_SCHEME = "bulwarkmobile"
/**
* Creates the channel, or renames it: JS passes the name in the app's
* language. Nothing urgent, so default importance and no vibration.
*/
fun ensureChannel(context: Context, name: String?) {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) return
val manager = context.getSystemService(NotificationManager::class.java)
val channel = NotificationChannel(
CHANNEL_ID,
name?.takeIf { it.isNotBlank() } ?: DEFAULT_CHANNEL_NAME,
NotificationManager.IMPORTANCE_DEFAULT,
).apply { enableVibration(false) }
manager.createNotificationChannel(channel)
}
fun show(context: Context, accountName: String, title: String, text: String, uri: String, channelName: String?) {
val link = Uri.parse(uri)
require(link.scheme == DEEP_LINK_SCHEME) { "Only $DEEP_LINK_SCHEME:// links open from a sync notification" }
ensureChannel(context, channelName)
val tag = TAG_PREFIX + accountName
// Explicit, so no other app can answer the tap.
val intent = Intent(Intent.ACTION_VIEW, link, context, MainActivity::class.java)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_SINGLE_TOP or Intent.FLAG_ACTIVITY_CLEAR_TOP)
val pending = PendingIntent.getActivity(
context,
tag.hashCode(),
intent,
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
)
val notification = NotificationCompat.Builder(context, CHANNEL_ID)
.setSmallIcon(R.drawable.ic_notification)
.setContentTitle(title)
.setContentText(text)
.setStyle(NotificationCompat.BigTextStyle().bigText(text))
.setCategory(NotificationCompat.CATEGORY_ERROR)
.setPriority(NotificationCompat.PRIORITY_DEFAULT)
// Runs repeat the same problem until it is solved; say it once.
.setOnlyAlertOnce(true)
.setAutoCancel(true)
.setContentIntent(pending)
.build()
context.getSystemService(NotificationManager::class.java).notify(tag, NOTIFICATION_ID, notification)
}
fun clear(context: Context, accountName: String) {
context.getSystemService(NotificationManager::class.java).cancel(TAG_PREFIX + accountName, NOTIFICATION_ID)
}
}
@@ -0,0 +1,112 @@
package com.anonymous.bulwarkmobile.sync
import android.content.SyncResult
import org.json.JSONException
import org.json.JSONObject
/**
* What a run's report (src/device-sync/types.ts `RunReport`) asks of the
* framework's SyncResult. Kept free of Android classes so JVM tests cover it;
* [SyncReports.apply] copies it onto the real SyncResult.
*
* SyncManager treats io errors as soft (retry with back-off), auth errors,
* database errors and too many deletions as hard (no retry until something
* changes), and `fullSyncRequested` as "schedule another sync now".
*/
data class SyncOutcome(
val ioErrors: Long = 0,
val authErrors: Long = 0,
val databaseError: Boolean = false,
val tooManyDeletions: Boolean = false,
val inserts: Long = 0,
val updates: Long = 0,
val deletes: Long = 0,
val entries: Long = 0,
val skipped: Long = 0,
/** Epoch seconds; 0 when unset. */
val delayUntil: Long = 0,
val fullSyncRequested: Boolean = false,
)
object SyncReports {
/**
* Translates a report. [reportJson] is null when the run ended without
* one ([endReason] says why: JS never answered, React failed to start, …),
* which is a soft error so the framework retries.
*/
fun outcome(reportJson: String?, endReason: String?): SyncOutcome {
if (reportJson == null) return SyncOutcome(ioErrors = 1)
val report = try {
JSONObject(reportJson)
} catch (e: JSONException) {
return SyncOutcome(ioErrors = 1)
}
val stats = report.optJSONObject("stats")
val down = stats?.optJSONObject("downloaded")
val up = stats?.optJSONObject("uploaded")
fun count(side: JSONObject?, key: String) = side?.optLong(key, 0) ?: 0
val base = SyncOutcome(
inserts = count(down, "created") + count(up, "created"),
updates = count(down, "updated") + count(up, "updated"),
deletes = count(down, "deleted") + count(up, "deleted"),
entries = stats?.optLong("entries", 0) ?: 0,
skipped = stats?.optLong("skipped", 0) ?: 0,
delayUntil = report.optLong("delayUntil", 0),
fullSyncRequested = report.optBoolean("moreRecordsToGet", false),
)
// SyncManager retries a failed sync at once, without back-off, when it
// made progress (inserts, updates, or deletes without
// tooManyDeletions). A hard error must therefore report none, or an
// auth failure after some work would loop.
val hard = base.copy(inserts = 0, updates = 0, deletes = 0)
return when (report.optString("outcome")) {
"ok", "cancelled", "disabled" -> base
"auth" -> hard.copy(authErrors = 1)
"permission", "unsupported", "safetyAbort" -> hard.copy(databaseError = true)
"tooManyDeletions" -> hard.copy(
tooManyDeletions = true,
// Shown in the system's "too many deletions" notification.
deletes = report.optJSONObject("tooManyDeletions")?.optLong("count", 0) ?: 0,
)
// "io", "internal" and anything a newer JS might send: retry, at
// once if some work got done, else with back-off.
else -> base.copy(ioErrors = 1)
}
}
/**
* One line for logcat: the report's outcome, its message and how many
* items failed. The reason a run failed is otherwise only in the report,
* and JS warnings from a headless run don't reach logcat.
*/
fun summary(reportJson: String?): String {
if (reportJson == null) return "no report"
val report = try {
JSONObject(reportJson)
} catch (e: JSONException) {
return "unreadable report"
}
val parts = mutableListOf("outcome=" + report.optString("outcome", "?"))
val items = report.optJSONArray("itemErrors")?.length() ?: 0
if (items > 0) parts += "itemErrors=$items"
val message = report.optString("message", "")
if (message.isNotEmpty()) parts += "message=" + message.take(MAX_LOGGED_MESSAGE)
return parts.joinToString(" ")
}
private const val MAX_LOGGED_MESSAGE = 300
fun apply(outcome: SyncOutcome, result: SyncResult) {
result.stats.numIoExceptions += outcome.ioErrors
result.stats.numAuthExceptions += outcome.authErrors
result.stats.numInserts += outcome.inserts
result.stats.numUpdates += outcome.updates
result.stats.numDeletes += outcome.deletes
result.stats.numEntries += outcome.entries
result.stats.numSkippedEntries += outcome.skipped
if (outcome.databaseError) result.databaseError = true
if (outcome.tooManyDeletions) result.tooManyDeletions = true
if (outcome.delayUntil > 0) result.delayUntil = outcome.delayUntil
if (outcome.fullSyncRequested) result.fullSyncRequested = true
}
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.6 KiB

After

Width:  |  Height:  |  Size: 8.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.8 KiB

After

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.9 KiB

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.5 KiB

After

Width:  |  Height:  |  Size: 7.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.3 KiB

After

Width:  |  Height:  |  Size: 7.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 41 KiB

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.2 KiB

After

Width:  |  Height:  |  Size: 9.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 13 KiB

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Home-screen widget edge, drawn over the widget by layout/rn_widget.xml. -->
<shape xmlns:android="http://schemas.android.com/apk/res/android"
android:shape="rectangle">
<stroke
android:width="1dp"
android:color="@color/widget_frame" />
<corners android:radius="@dimen/widget_radius" />
</shape>
@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Home-screen widget root (layout/rn_widget.xml): its rounded outline is what the widget clips to. -->
<shape xmlns:android="http://schemas.android.com/apk/res/android"
android:shape="rectangle">
<solid android:color="@color/widget_surface" />
<corners android:radius="@dimen/widget_radius" />
</shape>
@@ -0,0 +1,38 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Night twin of layout/rn_widget.xml, which explains the override. -->
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:id="@android:id/background"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@drawable/widget_surface"
android:clipToOutline="true"
android:foreground="@drawable/widget_frame"
tools:targetApi="31">
<ImageView
android:id="@+id/rn_widget_image_light"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@android:color/transparent"
android:scaleType="matrix"
android:visibility="gone" />
<ImageView
android:id="@+id/rn_widget_image_dark"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@android:color/transparent"
android:scaleType="matrix"
android:visibility="visible" />
<FrameLayout
android:id="@+id/rn_widget_clickable_container"
android:layout_width="match_parent"
android:layout_height="match_parent" />
<FrameLayout
android:id="@+id/rn_widget_collection_container"
android:layout_width="match_parent"
android:layout_height="match_parent" />
</FrameLayout>
@@ -0,0 +1,50 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Overrides react-native-android-widget's rn_widget.xml (same ids and children;
layout-night/rn_widget.xml is its night twin).
The library draws each widget into one bitmap sized from the size the
launcher reports, and pins it to the top left without scaling. When the real
frame is smaller (after a rotation, or on a launcher that over-reports), the
bitmap's bottom edge is cut off and takes the rounded corners and the border
with it, so the bottom corners fall back to whatever the launcher clips to.
This root always has the real size, so it owns the shape instead: it clips to
a rounded outline (Android 12+; the launcher then leaves the corners to us),
fills in where the bitmap falls short, and draws the frame on top.
-->
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:id="@android:id/background"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@drawable/widget_surface"
android:clipToOutline="true"
android:foreground="@drawable/widget_frame"
tools:targetApi="31">
<ImageView
android:id="@+id/rn_widget_image_light"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@android:color/transparent"
android:scaleType="matrix"
android:visibility="visible" />
<ImageView
android:id="@+id/rn_widget_image_dark"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@android:color/transparent"
android:scaleType="matrix"
android:visibility="gone" />
<FrameLayout
android:id="@+id/rn_widget_clickable_container"
android:layout_width="match_parent"
android:layout_height="match_parent" />
<FrameLayout
android:id="@+id/rn_widget_collection_container"
android:layout_width="match_parent"
android:layout_height="match_parent" />
</FrameLayout>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 482 B

After

Width:  |  Height:  |  Size: 1.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 KiB

After

Width:  |  Height:  |  Size: 2.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 482 B

After

Width:  |  Height:  |  Size: 2.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 350 B

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 KiB

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 350 B

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 634 B

After

Width:  |  Height:  |  Size: 2.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.1 KiB

After

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 634 B

After

Width:  |  Height:  |  Size: 3.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 964 B

After

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.3 KiB

After

Width:  |  Height:  |  Size: 5.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 964 B

After

Width:  |  Height:  |  Size: 4.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 KiB

After

Width:  |  Height:  |  Size: 4.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.8 KiB

After

Width:  |  Height:  |  Size: 7.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 KiB

After

Width:  |  Height:  |  Size: 6.5 KiB

@@ -3,4 +3,6 @@
<color name="splashscreen_background">#09090b</color>
<!-- Matches the dark palette background (src/theme/tokens.ts). -->
<color name="window_background">#09090b</color>
<!-- DARK.bg in src/widgets/theme.ts. -->
<color name="widget_surface">#0a0a0a</color>
</resources>
+5 -1
View File
@@ -1,9 +1,13 @@
<resources>
<!-- Splash background, as `splash.backgroundColor` in app.config.js; the dark one is in values-night. -->
<color name="splashscreen_background">#ffffff</color>
<color name="iconBackground">#09090b</color>
<color name="iconBackground">#db2d54</color>
<color name="colorPrimary">#023c69</color>
<color name="colorPrimaryDark">#09090b</color>
<!-- Behind the React root until the first frame; matches the light palette background (src/theme/tokens.ts). -->
<color name="window_background">#ffffff</color>
<!-- Home-screen widget surface (LIGHT.bg in src/widgets/theme.ts); it only shows where the drawn widget falls short of the launcher's frame. -->
<color name="widget_surface">#ffffff</color>
<!-- Home-screen widget edge, one colour for both schemes: the app can force a theme the launcher's night mode doesn't know about. -->
<color name="widget_frame">#4d808080</color>
</resources>
@@ -0,0 +1,4 @@
<resources>
<!-- Home-screen widget corners; WIDGET_RADIUS in src/widgets/primitives.tsx. -->
<dimen name="widget_radius">16dp</dimen>
</resources>
@@ -2,4 +2,6 @@
<string name="app_name">Bulwark Mobile</string>
<string name="expo_splash_screen_resize_mode" translatable="false">contain</string>
<string name="expo_splash_screen_status_bar_translucent" translatable="false">false</string>
<!-- Account type label of device sync (#34), shown in Settings → Accounts. -->
<string name="device_sync_account_label" translatable="false">Bulwark</string>
</resources>
@@ -0,0 +1,7 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- The "Bulwark" account type of device sync (#34); see docs/device-sync.md. -->
<account-authenticator xmlns:android="http://schemas.android.com/apk/res/android"
android:accountType="@string/device_sync_account_type"
android:icon="@mipmap/ic_launcher"
android:smallIcon="@mipmap/ic_launcher"
android:label="@string/device_sync_account_label" />
+83
View File
@@ -0,0 +1,83 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
How Contacts apps show and edit contacts of the Bulwark account (#34). The
EditSchema lists exactly the data kinds device sync maps to JSContact
(docs/device-sync.md, "Contacts mapping"); apps offer nothing else.
AOSP Contacts drops the whole account type when this schema breaks its rules
(BaseAccountType): every kind with a type column needs <Type> children and
the others must have none, name needs maxOccurs="1" and every supports*
attribute true (supportsDisplayName for older releases), and photo,
nickname, organization, note and group_membership need maxOccurs="1".
A Type's maxOccurs limits that type only; birthday gets none unless given.
-->
<ContactsAccountType xmlns:android="http://schemas.android.com/apk/res/android">
<EditSchema>
<DataKind kind="name" maxOccurs="1"
supportsDisplayName="true"
supportsPrefix="true"
supportsMiddleName="true"
supportsSuffix="true"
supportsPhoneticFamilyName="true"
supportsPhoneticMiddleName="true"
supportsPhoneticGivenName="true" />
<DataKind kind="photo" maxOccurs="1" />
<DataKind kind="phone">
<Type type="mobile" />
<Type type="home" />
<Type type="work" />
<Type type="main" />
<Type type="fax_work" />
<Type type="fax_home" />
<Type type="other_fax" />
<Type type="pager" />
<Type type="work_mobile" />
<Type type="work_pager" />
<Type type="tty_tdd" />
<Type type="other" />
<Type type="custom" />
</DataKind>
<DataKind kind="email">
<Type type="home" />
<Type type="work" />
<Type type="other" />
<Type type="mobile" />
<Type type="custom" />
</DataKind>
<DataKind kind="nickname" maxOccurs="1" />
<DataKind kind="postal" needsStructured="true">
<Type type="home" />
<Type type="work" />
<Type type="other" />
<Type type="custom" />
</DataKind>
<DataKind kind="organization" maxOccurs="1" />
<!-- No <Type> allowed: websites are always OTHER in the editor. -->
<DataKind kind="website" />
<DataKind kind="event" dateWithTime="false">
<Type type="birthday" maxOccurs="1" yearOptional="true" />
<Type type="anniversary" yearOptional="true" />
<Type type="other" yearOptional="true" />
<Type type="custom" yearOptional="true" />
</DataKind>
<DataKind kind="relationship">
<Type type="assistant" />
<Type type="brother" />
<Type type="child" />
<Type type="domestic_partner" />
<Type type="father" />
<Type type="friend" />
<Type type="manager" />
<Type type="mother" />
<Type type="parent" />
<Type type="partner" />
<Type type="referred_by" />
<Type type="relative" />
<Type type="sister" />
<Type type="spouse" />
<Type type="custom" />
</DataKind>
<DataKind kind="note" maxOccurs="1" />
<DataKind kind="group_membership" maxOccurs="1" />
</EditSchema>
</ContactsAccountType>
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Calendar sync adapter of device sync (#34); see docs/device-sync.md. -->
<sync-adapter xmlns:android="http://schemas.android.com/apk/res/android"
android:accountType="@string/device_sync_account_type"
android:contentAuthority="com.android.calendar"
android:supportsUploading="true"
android:userVisible="true"
android:allowParallelSyncs="false"
android:isAlwaysSyncable="false" />
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Contacts sync adapter of device sync (#34); see docs/device-sync.md. -->
<sync-adapter xmlns:android="http://schemas.android.com/apk/res/android"
android:accountType="@string/device_sync_account_type"
android:contentAuthority="com.android.contacts"
android:supportsUploading="true"
android:userVisible="true"
android:allowParallelSyncs="false"
android:isAlwaysSyncable="false" />
@@ -0,0 +1,18 @@
package com.anonymous.bulwarkmobile.sync
import com.anonymous.bulwarkmobile.sync.DeviceSyncAccounts.EnsureAction
import org.junit.Assert.assertEquals
import org.junit.Test
class DeviceSyncAccountsTest {
@Test
fun `ensureAccount creates, keeps, adopts or refuses`() {
assertEquals(EnsureAction.CREATE, DeviceSyncAccounts.ensureAction(false, null, "alice@example.org@mail"))
assertEquals(EnsureAction.KEEP, DeviceSyncAccounts.ensureAction(true, "alice@example.org@mail", "alice@example.org@mail"))
// An account whose setup never wrote a registry id.
assertEquals(EnsureAction.ADOPT, DeviceSyncAccounts.ensureAction(true, null, "alice@example.org@mail"))
assertEquals(EnsureAction.ADOPT, DeviceSyncAccounts.ensureAction(true, "", "alice@example.org@mail"))
// Its rows and SyncState belong to another app account: never rewritten.
assertEquals(EnsureAction.CONFLICT, DeviceSyncAccounts.ensureAction(true, "alice@example.org@old", "alice@example.org@mail"))
}
}
@@ -0,0 +1,62 @@
package com.anonymous.bulwarkmobile.sync
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
class DeviceSyncRunsTest {
private val contacts = "com.android.contacts"
private val calendar = "com.android.calendar"
@Test
fun `a report is accepted once, while the adapter waits`() {
val run = DeviceSyncRuns.open("alice", contacts)
assertTrue(DeviceSyncRuns.finish(run.runId, "{\"outcome\":\"ok\"}"))
assertFalse(DeviceSyncRuns.finish(run.runId, "{\"outcome\":\"io\"}"))
assertFalse(run.end("no report before the deadline"))
assertEquals(0L, run.done.count)
assertEquals("{\"outcome\":\"ok\"}", run.report)
assertEquals("reported", run.endReason)
DeviceSyncRuns.close(run)
}
@Test
fun `a report after the run ended is refused whole`() {
val run = DeviceSyncRuns.open("alice", contacts)
assertTrue(run.end("no report before the deadline"))
assertFalse(DeviceSyncRuns.finish(run.runId, "{\"outcome\":\"ok\"}"))
assertNull(run.report)
assertEquals("no report before the deadline", run.endReason)
DeviceSyncRuns.close(run)
assertFalse(DeviceSyncRuns.finish(run.runId, "{\"outcome\":\"ok\"}"))
}
@Test
fun `runs nobody waits for, or that ended, count as cancelled`() {
assertTrue(DeviceSyncRuns.isCancelled("unknown"))
val run = DeviceSyncRuns.open("alice", calendar)
assertFalse(DeviceSyncRuns.isCancelled(run.runId))
run.end("task finished")
assertTrue(DeviceSyncRuns.isCancelled(run.runId))
DeviceSyncRuns.close(run)
assertTrue(DeviceSyncRuns.isCancelled(run.runId))
}
@Test
fun `a sync requested during a run flags that run only, and the flag outlives the close`() {
val mine = DeviceSyncRuns.open("alice", contacts)
val otherAuthority = DeviceSyncRuns.open("alice", calendar)
val otherAccount = DeviceSyncRuns.open("bob", contacts)
assertTrue(DeviceSyncRuns.flagSyncAgain("alice", contacts))
DeviceSyncRuns.close(mine)
assertTrue(mine.syncAgainRequested)
assertFalse(otherAuthority.syncAgainRequested)
assertFalse(otherAccount.syncAgainRequested)
// Closed runs are not flagged any more.
assertFalse(DeviceSyncRuns.flagSyncAgain("alice", contacts))
DeviceSyncRuns.close(otherAuthority)
DeviceSyncRuns.close(otherAccount)
}
}
@@ -0,0 +1,562 @@
package com.anonymous.bulwarkmobile.sync
import org.json.JSONArray
import org.json.JSONObject
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Assert.fail
import org.junit.Test
import java.math.BigDecimal
import java.math.BigInteger
private const val CONTACTS = "com.android.contacts"
private const val CALENDAR = "com.android.calendar"
private val ME = AccountRef("alice@example.org", "com.anonymous.bulwarkmobile.account")
private val OTHER = AccountRef("bob@gmail.com", "com.google")
/** The provider as far as the planner asks: who owns which row; [me] is the account the lookup is bound to. */
private class FakeLookup(private val me: AccountRef = ME) : ScopeLookup {
val rawContacts = HashMap<Long, AccountRef>()
val groups = HashMap<Long, AccountRef>()
val calendars = HashMap<Long, AccountRef>()
val colors = HashMap<Long, AccountRef>()
/** data id → raw contact */
val data = HashMap<Long, Long>()
/** data id → mimetype */
val mimetypes = HashMap<Long, String>()
/** event id → calendar */
val events = HashMap<Long, Long>()
/** child table → child id → event */
val children = HashMap<ProviderTable, HashMap<Long, Long>>()
/** (child table, where) → events of the matching rows; a null where matches every row. */
val matching = HashMap<Pair<ProviderTable, String?>, Set<Long>>()
override fun ownCalendarIds() = calendars.filterValues { it == me }.keys
override fun rootAccounts(table: ProviderTable, ids: Set<Long>): Map<Long, AccountRef> = when (table) {
ProviderTable.RAW_CONTACTS -> rawContacts
ProviderTable.GROUPS -> groups
ProviderTable.CALENDARS -> calendars
ProviderTable.COLORS -> colors
else -> throw AssertionError("no root lookup for $table")
}.filterKeys { it in ids }
override fun dataRawContacts(ids: Set<Long>) = data.filterKeys { it in ids }
override fun dataMimetypes(ids: Set<Long>) = mimetypes.filterKeys { it in ids }
override fun eventCalendars(ids: Set<Long>) = events.filterKeys { it in ids }
override fun childEvents(table: ProviderTable, ids: Set<Long>): Map<Long, Long> =
children[table].orEmpty().filterKeys { it in ids }
override fun childEventIdsMatching(table: ProviderTable, where: String?, args: List<String>): Set<Long> =
matching[table to where] ?: if (where == null) children[table].orEmpty().values.toSet() else emptySet()
fun child(table: ProviderTable, id: Long, event: Long) {
children.getOrPut(table) { HashMap() }[id] = event
}
}
class ProviderOpsTest {
private val lookup = FakeLookup().apply {
rawContacts[1] = ME
rawContacts[2] = OTHER
groups[10] = ME
groups[11] = OTHER
data[100] = 1
data[101] = 2
calendars[20] = ME
calendars[21] = ME
calendars[22] = OTHER
colors[40] = ME
events[30] = 20
events[31] = 21
events[32] = 22
child(ProviderTable.ATTENDEES, 50, 30)
child(ProviderTable.ATTENDEES, 51, 32)
child(ProviderTable.REMINDERS, 60, 31)
child(ProviderTable.EXTENDED_PROPERTIES, 70, 30)
child(ProviderTable.EXTENDED_PROPERTIES, 71, 32)
}
private val planner = ScopePlanner(ME, lookup)
private fun ops(vararg op: String) = "[${op.joinToString(",")}]"
private fun plan(authority: String, vararg op: String) =
planner.planBatch(ProviderOpParser.parseBatch(authority, ops(*op)))
private fun refused(reason: BatchFailure, block: () -> Unit): String {
try {
block()
} catch (e: ProviderRefusal) {
assertEquals(e.message, reason, e.reason)
return e.message.orEmpty()
}
fail("expected a $reason refusal")
throw AssertionError()
}
// ── tables and parsing ──────────────────────────────────────
@Test
fun `tables are enumerated per authority and anything else is out of scope`() {
assertEquals(ProviderTable.EXTENDED_PROPERTIES, ProviderTable.of(CALENDAR, "extended_properties"))
assertEquals(ProviderTable.SETTINGS, ProviderTable.of(CONTACTS, "settings"))
refused(BatchFailure.SCOPE) { ProviderTable.of(CONTACTS, "events") }
refused(BatchFailure.SCOPE) { ProviderTable.of(CALENDAR, "raw_contacts") }
refused(BatchFailure.SCOPE) { ProviderTable.of(CONTACTS, "contacts") }
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"delete","table":"instances","where":"1"}""") }
}
@Test
fun `numbers keep integers exact whatever org_json parsed them into`() {
assertEquals(Cell.Integer(5), ProviderOpParser.numberCell(5))
assertEquals(Cell.Integer(5), ProviderOpParser.numberCell(5.0))
assertEquals(Cell.Integer(5), ProviderOpParser.numberCell(BigDecimal("5.0")))
assertEquals(Cell.Integer(1L shl 60), ProviderOpParser.numberCell(BigInteger.ONE.shiftLeft(60)))
assertEquals(Cell.Integer(1_700_000_000_000), ProviderOpParser.numberCell(1_700_000_000_000L))
assertEquals(Cell.Real(1.5), ProviderOpParser.numberCell(BigDecimal("1.5")))
assertEquals(Cell.Real(1.5), ProviderOpParser.numberCell(1.5))
}
@Test
fun `values keep text, numbers, booleans as 0 and 1, and nulls`() {
val insert = ProviderOpParser.parseBatch(
CONTACTS,
ops("""{"op":"insert","table":"raw_contacts","values":{"sync1":"a","dirty":0,"starred":true,"sync2":null}}"""),
).single() as ProviderOp.Insert
assertEquals(
mapOf("sync1" to Cell.Text("a"), "dirty" to Cell.Integer(0), "starred" to Cell.Integer(1), "sync2" to Cell.Null),
insert.values,
)
}
@Test
fun `malformed ops are provider failures`() {
refused(BatchFailure.PROVIDER) { ProviderOpParser.parseBatch(CONTACTS, "{}") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"upsert","table":"groups","values":{}}""") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"update","table":"groups","id":10,"values":{}}""") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"assert","table":"groups","where":"dirty = 1"}""") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"delete","table":"settings","id":1}""") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"delete","table":"groups","args":["x"]}""") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"delete","table":"groups","id":1.5}""") }
refused(BatchFailure.PROVIDER) { plan(CONTACTS, """{"op":"delete","table":"groups","where":"a = ?","args":[null]}""") }
}
@Test
fun `bytes are only accepted for data_data15`() {
val planned = plan(
CONTACTS,
"""{"op":"insert","table":"data","values":{"raw_contact_id":1,"mimetype":"vnd.android.cursor.item/photo","data15":{"b64":"AAEC"}}}""",
).single() as PlannedOp.Insert
assertEquals(Cell.Blob("AAEC"), planned.values["data15"])
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"update","table":"raw_contacts","id":1,"values":{"sync2":{"b64":"AAEC"}}}""")
}
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,"data1":{"b64":"AA"}}}""")
}
refused(BatchFailure.PROVIDER) {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,"data15":{"bytes":"AA"}}}""")
}
}
// ── batch rules ─────────────────────────────────────────────
@Test
fun `a syncState op must be the last op and not a yield point`() {
val planned = plan(
CONTACTS,
"""{"op":"update","table":"groups","id":10,"values":{"sync2":"x"},"yieldAllowed":true}""",
"""{"op":"syncState","value":"{\"v\":1}"}""",
)
assertEquals(PlannedOp.SetSyncState("{\"v\":1}"), planned.last())
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"syncState","value":"{}"}""", """{"op":"delete","table":"groups","id":10}""")
}
refused(BatchFailure.SCOPE) {
plan(
CONTACTS,
"""{"op":"delete","table":"groups","id":10}""",
"""{"op":"syncState","value":"{}","yieldAllowed":true}""",
)
}
}
private fun deletes(count: Int, yieldAt: Set<Int> = emptySet()) = (0 until count).joinToString(",", "[", "]") { i ->
"""{"op":"delete","table":"groups","where":"sync1 = ?","args":["$i"]${if (i in yieldAt) ""","yieldAllowed":true""" else ""}}"""
}
@Test
fun `contacts batches stop at 499 ops between yield points, counted as ContactsProvider does`() {
assertEquals(499, ProviderOpParser.parseBatch(CONTACTS, deletes(499)).size)
refused(BatchFailure.PROVIDER) { ProviderOpParser.parseBatch(CONTACTS, deletes(500)) }
// The yielding op still counts toward the ops before it.
assertEquals(998, ProviderOpParser.parseBatch(CONTACTS, deletes(998, setOf(498))).size)
refused(BatchFailure.PROVIDER) { ProviderOpParser.parseBatch(CONTACTS, deletes(999, setOf(498))) }
refused(BatchFailure.PROVIDER) { ProviderOpParser.parseBatch(CONTACTS, deletes(600, setOf(499))) }
// A yield on the first op is ignored, as the provider ignores it.
refused(BatchFailure.PROVIDER) { ProviderOpParser.parseBatch(CONTACTS, deletes(500, setOf(0))) }
}
@Test
fun `calendar batches have no yield point limit`() {
val many = (0 until 600).joinToString(",", "[", "]") { """{"op":"delete","table":"colors","where":"color_index = '$it'"}""" }
assertEquals(600, ProviderOpParser.parseBatch(CALENDAR, many).size)
}
@Test
fun `back-references name earlier inserts, and parents name their parent table`() {
val planned = plan(
CONTACTS,
"""{"op":"insert","table":"raw_contacts","values":{"sourceid":"c/1"}}""",
"""{"op":"insert","table":"data","values":{"mimetype":"vnd.android.cursor.item/name","data1":"A"},"refs":{"raw_contact_id":0}}""",
)
assertEquals(mapOf("raw_contact_id" to 0), (planned[1] as PlannedOp.Insert).backReferences)
refused(BatchFailure.PROVIDER) {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"mimetype":"x"},"refs":{"raw_contact_id":0}}""")
}
refused(BatchFailure.PROVIDER) {
plan(
CONTACTS,
"""{"op":"delete","table":"groups","id":10}""",
"""{"op":"insert","table":"data","values":{"mimetype":"x"},"refs":{"raw_contact_id":0}}""",
)
}
// Settings rows have no id to refer to.
refused(BatchFailure.PROVIDER) {
plan(
CONTACTS,
"""{"op":"insert","table":"settings","values":{"ungrouped_visible":1}}""",
"""{"op":"insert","table":"data","values":{"mimetype":"x"},"refs":{"raw_contact_id":0}}""",
)
}
refused(BatchFailure.SCOPE) {
plan(
CONTACTS,
"""{"op":"insert","table":"groups","values":{"title":"g"}}""",
"""{"op":"insert","table":"data","values":{"mimetype":"x"},"refs":{"raw_contact_id":0}}""",
)
}
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"insert","table":"data","values":{"mimetype":"x"}}""") }
refused(BatchFailure.SCOPE) {
plan(CALENDAR, """{"op":"insert","table":"attendees","values":{"event_id":null,"attendeeEmail":"a@b"}}""")
}
}
// ── scope: inserts ──────────────────────────────────────────
@Test
fun `root inserts are written into the account, whatever the op says`() {
val planned = plan(
CONTACTS,
"""{"op":"insert","table":"raw_contacts","values":{"sourceid":"c/1","account_name":"alice@example.org"}}""",
).single() as PlannedOp.Insert
assertEquals(Cell.Text(ME.name), planned.values["account_name"])
assertEquals(Cell.Text(ME.type), planned.values["account_type"])
val calendar = plan(CALENDAR, """{"op":"insert","table":"calendars","values":{"name":"c"}}""").single()
assertEquals(Cell.Text(ME.type), (calendar as PlannedOp.Insert).values["account_type"])
}
@Test
fun `account columns never move a row out of the account`() {
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"insert","table":"raw_contacts","values":{"account_name":"bob@gmail.com"}}""")
}
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"update","table":"groups","id":10,"values":{"account_type":"com.google"}}""")
}
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"insert","table":"groups","values":{"title":"g","data_set":"plugin"}}""")
}
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,"account_name":"alice@example.org"}}""")
}
refused(BatchFailure.SCOPE) {
plan(CALENDAR, """{"op":"update","table":"events","id":30,"values":{"account_name":"alice@example.org"}}""")
}
}
@Test
fun `inserts must name a parent of the account that still exists`() {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,"mimetype":"x"}}""")
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":"1","mimetype":"x"}}""")
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":2,"mimetype":"x"}}""") }
refused(BatchFailure.ASSERT) { plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":9,"mimetype":"x"}}""") }
refused(BatchFailure.PROVIDER) {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":"me","mimetype":"x"}}""")
}
plan(CALENDAR, """{"op":"insert","table":"events","values":{"calendar_id":21,"dtstart":0}}""")
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"insert","table":"events","values":{"calendar_id":22,"dtstart":0}}""") }
refused(BatchFailure.SCOPE) {
plan(CALENDAR, """{"op":"insert","table":"events","values":{"calendar_id":20,"original_id":32}}""")
}
plan(CALENDAR, """{"op":"insert","table":"reminders","values":{"event_id":31,"minutes":10}}""")
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"insert","table":"reminders","values":{"event_id":32,"minutes":10}}""") }
refused(BatchFailure.ASSERT) { plan(CALENDAR, """{"op":"insert","table":"attendees","values":{"event_id":99}}""") }
}
@Test
fun `a membership row may only name a group of the account`() {
val membership = """"mimetype":"vnd.android.cursor.item/group_membership""""
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,$membership,"data1":10}}""")
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,$membership,"group_sourceid":"c/g"}}""")
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,$membership,"data1":11}}""")
}
// data1 of other kinds is plain data.
plan(CONTACTS, """{"op":"insert","table":"data","values":{"raw_contact_id":1,"mimetype":"x","data1":11}}""")
}
@Test
fun `a membership row's group is checked on updates by id too`() {
lookup.mimetypes[100] = GROUP_MEMBERSHIP_MIMETYPE
plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"data1":10}}""")
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"data1":11}}""") }
refused(BatchFailure.ASSERT) { plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"data1":12}}""") }
// data1 by a selection could reach memberships the lookup never saw.
refused(BatchFailure.SCOPE) {
plan(CONTACTS, """{"op":"update","table":"data","where":"raw_contact_id = ?","args":[1],"values":{"data1":11}}""")
}
// data1 of other kinds is plain data.
lookup.mimetypes[100] = "vnd.android.cursor.item/phone_v2"
plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"data1":11}}""")
}
@Test
fun `photo file ids are never written`() {
val photo = """"raw_contact_id":1,"mimetype":"vnd.android.cursor.item/photo""""
plan(CONTACTS, """{"op":"insert","table":"data","values":{$photo,"data14":null,"data15":{"b64":"AAAA"}}}""")
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"insert","table":"data","values":{$photo,"data14":5}}""") }
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"data14":5}}""") }
}
@Test
fun `a selection can't close the scope's parentheses or query elsewhere`() {
for (where in listOf("1) OR (1", "dirty = 1) OR (1 = 1", "(1", "1; DELETE FROM events", "1 -- x", "1 /* x */",
"_id IN (SELECT _id FROM events)", "1 UNION ALL x", "title = 'open")) {
val quoted = JSONObject.quote(where)
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"update","table":"events","where":$quoted,"values":{"dirty":1}}""") }
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"delete","table":"attendees","where":$quoted}""") }
refused(BatchFailure.SCOPE) {
ProviderOpParser.parseQuery(CALENDAR, """{"table":"attendees","columns":["_id"],"where":$quoted}""")
}
}
refused(BatchFailure.SCOPE) {
ProviderOpParser.parseQuery(CONTACTS, """{"table":"raw_contacts","columns":["_id"],"orderBy":"_id); DROP TABLE x; --"}""")
}
// Parentheses and keywords inside quoted literals are data.
plan(CALENDAR, """{"op":"update","table":"events","where":"title = '(x' OR title = 'select'","values":{"dirty":1}}""")
plan(CALENDAR, """{"op":"assert","table":"events","where":"(dirty = 1 OR deleted = 1) AND _sync_id LIKE '~pending/%'","expectCount":0}""")
ProviderOpParser.parseQuery(CONTACTS, """{"table":"raw_contacts","columns":["_id"],"orderBy":"_id DESC"}""")
}
@Test
fun `moving a row to another parent is checked like an insert`() {
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"update","table":"events","id":30,"values":{"calendar_id":22}}""") }
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"raw_contact_id":null}}""") }
plan(CALENDAR, """{"op":"update","table":"events","id":30,"values":{"calendar_id":21}}""")
}
// ── scope: selections ───────────────────────────────────────
@Test
fun `root tables are selected by their account columns`() {
val op = plan(
CONTACTS,
"""{"op":"update","table":"raw_contacts","where":"dirty = ? AND deleted = 0","args":[1],"values":{"dirty":0}}""",
).single() as PlannedOp.Update
assertEquals(
Selection("(dirty = ? AND deleted = 0) AND (account_name = ? AND account_type = ?)", listOf("1", ME.name, ME.type)),
op.selection,
)
assertNull(op.expectedCount)
}
@Test
fun `an op by id selects that row of the account and expects exactly one row`() {
val op = plan(CONTACTS, """{"op":"delete","table":"groups","id":10}""").single() as PlannedOp.Delete
assertEquals(Selection("(_id = ?) AND (account_name = ? AND account_type = ?)", listOf("10", ME.name, ME.type)), op.selection)
assertEquals(1, op.expectedCount)
val counted = plan(CONTACTS, """{"op":"assert","table":"raw_contacts","id":1,"expectCount":0}""").single()
assertEquals(0, (counted as PlannedOp.Assert).expectedCount)
}
@Test
fun `an id of another account is refused, a vanished one is left to the expected count`() {
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"delete","table":"raw_contacts","id":2}""") }
refused(BatchFailure.SCOPE) { plan(CONTACTS, """{"op":"update","table":"data","id":101,"values":{"data1":"x"}}""") }
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"assert","table":"events","id":32,"values":{"dirty":0}}""") }
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"delete","table":"attendees","id":51}""") }
refused(BatchFailure.SCOPE) { plan(CALENDAR, """{"op":"update","table":"calendars","id":22,"values":{"name":"x"}}""") }
val gone = plan(CONTACTS, """{"op":"delete","table":"raw_contacts","id":9}""").single() as PlannedOp.Delete
assertEquals(1, gone.expectedCount)
}
@Test
fun `data rows rely on the URI parameters and never select on the Data view's account columns`() {
val op = plan(CONTACTS, """{"op":"update","table":"data","id":100,"values":{"data1":"x"}}""").single() as PlannedOp.Update
assertEquals(Selection("(_id = ?)", listOf("100")), op.selection)
val query = planner.planQuery(ProviderOpParser.parseQuery(CONTACTS, """{"table":"data","columns":["_id"],"where":"raw_contact_id IN (1)"}"""))
assertEquals(listOf(Selection("(raw_contact_id IN (1))", emptyList())), query.selections)
val all = planner.planQuery(ProviderOpParser.parseQuery(CONTACTS, """{"table":"data","columns":["_id"]}"""))
assertEquals(listOf(Selection.NONE), all.selections)
}
@Test
fun `events are selected by the account's calendars`() {
val query = planner.planQuery(
ProviderOpParser.parseQuery(CALENDAR, """{"table":"events","columns":["_id","title"],"where":"dirty = 1","orderBy":"_id"}"""),
)
assertEquals(listOf(Selection("(dirty = 1) AND (calendar_id IN (20,21))", emptyList())), query.selections)
assertEquals("_id", query.orderBy)
val update = plan(CALENDAR, """{"op":"update","table":"events","id":30,"values":{"dirty":0}}""").single() as PlannedOp.Update
assertEquals(Selection("(_id = ?) AND (calendar_id IN (20,21))", listOf("30")), update.selection)
}
@Test
fun `an account without calendars selects no events`() {
val carol = AccountRef("carol", ME.type)
val empty = ScopePlanner(carol, FakeLookup(carol).apply { calendars.putAll(lookup.calendars) })
val query = empty.planQuery(ProviderOpParser.parseQuery(CALENDAR, """{"table":"events","columns":["_id"]}"""))
assertEquals(listOf(Selection("(0)", emptyList())), query.selections)
}
@Test
fun `children of events are selected by the account's events among those they touch`() {
lookup.matching[ProviderTable.ATTENDEES to "event_id IN (30,32)"] = setOf(30, 32)
val query = planner.planQuery(
ProviderOpParser.parseQuery(CALENDAR, """{"table":"attendees","columns":["attendeeEmail"],"where":"event_id IN (30,32)"}"""),
)
assertEquals(listOf(Selection("(event_id IN (30,32)) AND (event_id IN (30))", emptyList())), query.selections)
val byId = plan(CALENDAR, """{"op":"delete","table":"attendees","id":50}""").single() as PlannedOp.Delete
assertEquals(Selection("(Attendees._id = ?) AND (event_id IN (30))", listOf("50")), byId.selection)
assertEquals(1, byId.expectedCount)
val gone = plan(CALENDAR, """{"op":"delete","table":"reminders","id":69}""").single() as PlannedOp.Delete
assertEquals(Selection("(Reminders._id = ?) AND (0)", listOf("69")), gone.selection)
lookup.matching[ProviderTable.REMINDERS to "event_id = ?"] = setOf(31)
val byWhere = plan(CALENDAR, """{"op":"delete","table":"reminders","where":"event_id = ?","args":[31]}""").single()
assertEquals(Selection("(event_id = ?) AND (event_id IN (31))", listOf("31")), (byWhere as PlannedOp.Delete).selection)
}
@Test
fun `a child query that touches none of the account's events reads nothing`() {
lookup.matching[ProviderTable.ATTENDEES to "event_id = 32"] = setOf(32)
val query = planner.planQuery(
ProviderOpParser.parseQuery(CALENDAR, """{"table":"attendees","columns":["_id"],"where":"event_id = 32"}"""),
)
assertTrue(query.selections.isEmpty())
}
@Test
fun `long event lists are split into chunks`() {
val many = (1000L until 1000L + 1200).toSet()
for (id in many) lookup.events[id] = 20
lookup.matching[ProviderTable.REMINDERS to null] = many
val query = planner.planQuery(ProviderOpParser.parseQuery(CALENDAR, """{"table":"reminders","columns":["minutes"]}"""))
assertEquals(3, query.selections.size)
assertTrue(query.selections.all { it.sql!!.startsWith("(event_id IN (") })
}
@Test
fun `extended properties are updated through their item URI, so only by id`() {
val op = plan(CALENDAR, """{"op":"update","table":"extended_properties","id":70,"values":{"value":"v"}}""").single()
assertEquals(PlannedOp.Update(ProviderTable.EXTENDED_PROPERTIES, Selection.NONE, 70, mapOf("value" to Cell.Text("v")), 1, false), op)
refused(BatchFailure.SCOPE) {
plan(CALENDAR, """{"op":"update","table":"extended_properties","id":71,"values":{"value":"v"}}""")
}
refused(BatchFailure.PROVIDER) {
plan(CALENDAR, """{"op":"update","table":"extended_properties","where":"name = 'x'","values":{"value":"v"}}""")
}
}
@Test
fun `asserts compare text and keep nulls`() {
val op = plan(
CONTACTS,
"""{"op":"assert","table":"raw_contacts","id":1,"values":{"version":3,"dirty":0,"sync3":null,"sourceid":"c/1"}}""",
).single() as PlannedOp.Assert
assertEquals(mapOf("version" to "3", "dirty" to "0", "sync3" to null, "sourceid" to "c/1"), op.values)
assertEquals(1, op.expectedCount)
}
@Test
fun `settings are written by account`() {
val insert = plan(CONTACTS, """{"op":"insert","table":"settings","values":{"ungrouped_visible":1,"should_sync":1}}""").single()
assertEquals(Cell.Text(ME.name), (insert as PlannedOp.Insert).values["account_name"])
val update = plan(CONTACTS, """{"op":"update","table":"settings","values":{"ungrouped_visible":1}}""").single()
assertEquals(
Selection("(account_name = ? AND account_type = ?)", listOf(ME.name, ME.type)),
(update as PlannedOp.Update).selection,
)
}
// ── results ─────────────────────────────────────────────────
@Test
fun `batch results are BatchResult JSON`() {
val ok = JSONObject(ProviderResults.ok(listOf(OpOutcome(id = 7), OpOutcome(count = 1), OpOutcome())))
assertEquals(true, ok.getBoolean("ok"))
val results = ok.getJSONArray("results")
assertEquals(7L, results.getJSONObject(0).getLong("id"))
assertEquals(1, results.getJSONObject(1).getInt("count"))
assertEquals(0, results.getJSONObject(2).length())
val failed = JSONObject(ProviderResults.failed(BatchFailure.TOO_LARGE, "TransactionTooLargeException"))
assertEquals(false, failed.getBoolean("ok"))
assertEquals("tooLarge", failed.getString("reason"))
assertEquals("TransactionTooLargeException", failed.getString("message"))
}
@Test
fun `an OperationApplicationException is an assert unless an insert got no row`() {
assertEquals(BatchFailure.ASSERT, ProviderResults.failureOfOperationApplication("Found value 4 when expected 3 for column version"))
assertEquals(BatchFailure.ASSERT, ProviderResults.failureOfOperationApplication("Expected 1 rows but actual 0"))
assertEquals(BatchFailure.ASSERT, ProviderResults.failureOfOperationApplication("wrong number of rows: 0"))
assertEquals(
BatchFailure.PROVIDER,
ProviderResults.failureOfOperationApplication("Insert into content://com.android.contacts/settings returned no result"),
)
assertEquals(
BatchFailure.PROVIDER,
ProviderResults.failureOfOperationApplication("Too many content provider operations between yield points."),
)
}
@Test
fun `query rows are ProviderRows JSON with escaped text`() {
val writer = ProviderRowsWriter(listOf("_id", "title", "x"))
writer.beginRow()
writer.longCell(9_007_199_254_740_991)
writer.textCell("a \"quoted\"\\ line\nbreak\u0001 and \u2028")
writer.nullCell()
writer.endRow()
writer.beginRow()
writer.longCell(2)
writer.doubleCell(1.5)
writer.doubleCell(Double.NaN)
writer.endRow()
val parsed = JSONObject(writer.finish())
assertEquals(JSONArray(listOf("_id", "title", "x")).toString(), parsed.getJSONArray("columns").toString())
val rows = parsed.getJSONArray("rows")
assertEquals(9_007_199_254_740_991L, rows.getJSONArray(0).getLong(0))
assertEquals("a \"quoted\"\\ line\nbreak\u0001 and \u2028", rows.getJSONArray(0).getString(1))
assertTrue(rows.getJSONArray(0).isNull(2))
assertEquals(1.5, rows.getJSONArray(1).getDouble(1), 0.0)
assertTrue(rows.getJSONArray(1).isNull(2))
assertEquals("{\"columns\":[],\"rows\":[]}", ProviderRowsWriter(emptyList()).finish())
}
@Test
fun `photos decode close to the target size and scale their longer side down to it`() {
assertEquals(1, PhotoScaling.sampleSize(720, 720, 512))
assertEquals(4, PhotoScaling.sampleSize(4000, 3000, 512))
assertEquals(512 to 384, PhotoScaling.targetSize(1000, 750, 512))
assertEquals(96 to 96, PhotoScaling.targetSize(96, 96, 512))
assertEquals(1 to 512, PhotoScaling.targetSize(3, 4000, 512))
}
}
@@ -0,0 +1,192 @@
package com.anonymous.bulwarkmobile.sync
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Test
class PushRoutingTest {
private val contacts = "com.android.contacts"
private val calendar = "com.android.calendar"
private val routes = PushRouting.parseRoutes(
"""
{
"c": [{ "accountName": "alice@example.org", "authorities": ["$contacts", "$calendar"] }],
"team": [
{ "accountName": "alice@example.org", "authorities": ["$contacts"] },
{ "accountName": "alice-work", "authorities": ["$calendar"] }
]
}
""".trimIndent(),
strict = true,
)
private fun stateChange(changed: String) = mapOf(
"kind" to "jmap-state-change",
"accountLabel" to "alice",
"accountId" to "c",
"emailIds" to "[]",
"changed" to changed,
)
@Test
fun `a contact change requests a contacts sync and no mail task`() {
val decision = PushRouting.decide(stateChange("""{"c":{"ContactCard":"s1"}}"""), routes)
assertEquals(setOf("alice@example.org" to contacts), decision.syncs)
assertFalse(decision.startMailTask)
}
@Test
fun `calendar types request calendar syncs of the routed accounts only`() {
val decision = PushRouting.decide(stateChange("""{"team":{"Calendar":"s1","CalendarEvent":"s2"}}"""), routes)
assertEquals(setOf("alice-work" to calendar), decision.syncs)
assertFalse(decision.startMailTask)
}
@Test
fun `every account key is routed, not only the one repeated as accountId`() {
val decision = PushRouting.decide(
stateChange("""{"c":{"AddressBook":"s1"},"team":{"ContactCard":"s2","CalendarEvent":"s3"}}"""),
routes,
)
// alice@example.org's contacts sync is requested once.
assertEquals(setOf("alice@example.org" to contacts, "alice-work" to calendar), decision.syncs)
}
@Test
fun `mail anywhere in the map starts the mail task, next to the syncs`() {
val decision = PushRouting.decide(
stateChange("""{"team":{"CalendarEvent":"s1"},"c":{"EmailDelivery":"s2","Email":"s3"}}"""),
routes,
)
assertTrue(decision.startMailTask)
assertEquals(setOf("alice-work" to calendar), decision.syncs)
}
@Test
fun `an Email or Mailbox change is possible mail, as a subscription from an older build sends it`() {
for (changed in listOf(
"""{"c":{"Email":"s1","Mailbox":"s2"}}""",
"""{"c":{"Email":"s1"}}""",
"""{"c":{"Mailbox":"s1"}}""",
)) {
val decision = PushRouting.decide(stateChange(changed), routes)
assertTrue("changed=$changed", decision.startMailTask)
assertTrue("changed=$changed", decision.syncs.isEmpty())
}
val mixed = PushRouting.decide(stateChange("""{"team":{"CalendarEvent":"e1"},"c":{"Email":"s1"}}"""), routes)
assertTrue(mixed.startMailTask)
assertEquals(setOf("alice-work" to calendar), mixed.syncs)
// Types that are neither mail nor device sync say nothing about mail.
val other = PushRouting.decide(stateChange("""{"c":{"Thread":"t1","ContactCard":"s1"}}"""), routes)
assertFalse(other.startMailTask)
assertEquals(setOf("alice@example.org" to contacts), other.syncs)
}
@Test
fun `an email push always starts the mail task`() {
val data = mapOf("kind" to "jmap-email-push", "accountId" to "c", "changed" to """{"c":{"EmailDelivery":"s"}}""")
assertTrue(PushRouting.decide(data, routes).startMailTask)
assertTrue(PushRouting.decide(mapOf("kind" to "jmap-email-push"), routes).startMailTask)
}
@Test
fun `payloads without a readable map are legacy mail pushes`() {
for (changed in listOf(null, "", "{}", "not json", "[1]", """{"c":{}}""")) {
val data = stateChange(changed ?: "").let { if (changed == null) it - "changed" else it }
val decision = PushRouting.decide(data, routes)
assertTrue("changed=$changed", decision.startMailTask)
assertTrue("changed=$changed", decision.syncs.isEmpty())
}
}
@Test
fun `an unreadable account entry may be mail`() {
val decision = PushRouting.decide(stateChange("""{"c":"garbled","team":{"ContactCard":"s"}}"""), routes)
assertTrue(decision.startMailTask)
assertEquals(setOf("alice@example.org" to contacts), decision.syncs)
}
@Test
fun `reads a push the same way as the push task`() {
// kind (null: none), changed (null: none) → whether the mail task starts. The
// same table as MAIL_CASES in src/lib/__tests__/push-background-task.test.ts,
// whose carriesNoMail must agree.
val state = "jmap-state-change"
val emailPush = "jmap-email-push"
val cases: List<Triple<String?, String?, Boolean>> = listOf(
// No readable map: as before device sync.
Triple(state, null, true),
Triple(state, "", true),
Triple(state, "not json", true),
Triple(state, "[1]", true),
Triple(state, "{}", true),
// A map that names no type is read like a missing one.
Triple(state, """{"c":{}}""", true),
Triple(state, """{"c":{},"team":{}}""", true),
// An account entry that can't be read may be mail.
Triple(state, """{"c":null}""", true),
Triple(state, """{"c":"garbled","team":{"ContactCard":"s"}}""", true),
Triple(state, """{"c":[],"team":{"ContactCard":"s"}}""", true),
Triple(state, """{"c":5}""", true),
// Mail types anywhere in the map.
Triple(state, """{"c":{"EmailDelivery":"s"}}""", true),
Triple(state, """{"c":{"Email":"s"}}""", true),
Triple(state, """{"c":{"Mailbox":"s"}}""", true),
Triple(state, """{"c":{"ContactCard":"s1"},"team":{"EmailDelivery":"s2"}}""", true),
// Only a StateChange can say it carries no mail.
Triple(null, """{"c":{"ContactCard":"s"}}""", true),
Triple(emailPush, """{"c":{"EmailDelivery":"s"}}""", true),
Triple(emailPush, null, true),
// Contact and calendar changes, and types that say nothing about mail.
Triple(state, """{"c":{"ContactCard":"s1","AddressBook":"s2"}}""", false),
Triple(state, """{"c":{"CalendarEvent":"e1"},"team":{"Calendar":"c1"}}""", false),
Triple(state, """{"c":{"ContactCard":"s"},"team":{}}""", false),
Triple(state, """{"elsewhere":{"ContactCard":"s"}}""", false),
Triple(state, """{"c":{"Thread":"t1","ContactCard":"s1"}}""", false),
)
for ((kind, changed, mail) in cases) {
val data = buildMap<String, String> {
if (kind != null) put("kind", kind)
put("accountLabel", "alice")
put("accountId", "c")
put("emailIds", "[]")
if (changed != null) put("changed", changed)
}
assertEquals("kind=$kind changed=$changed", mail, PushRouting.decide(data, routes).startMailTask)
}
}
@Test
fun `accounts without routes are ignored`() {
val decision = PushRouting.decide(stateChange("""{"elsewhere":{"ContactCard":"s"}}"""), routes)
assertTrue(decision.syncs.isEmpty())
assertFalse(decision.startMailTask)
assertTrue(PushRouting.decide(stateChange("""{"c":{"ContactCard":"s"}}"""), emptyMap()).syncs.isEmpty())
}
@Test
fun `stored routes are validated strictly and read leniently`() {
val bad = listOf(
"[]",
"""{"c":{}}""",
"""{"c":[{"authorities":["$contacts"]}]}""",
"""{"c":[{"accountName":"a","authorities":["com.example.tasks"]}]}""",
)
for (json in bad) {
try {
PushRouting.parseRoutes(json, strict = true)
throw AssertionError("accepted $json")
} catch (e: IllegalArgumentException) {
// expected
}
}
assertEquals(emptyMap<String, List<PushRouting.Route>>(), PushRouting.parseRoutes("[]"))
assertEquals(
mapOf("c" to listOf(PushRouting.Route("a", setOf(contacts)))),
PushRouting.parseRoutes("""{"c":[{"accountName":"a","authorities":["$contacts","com.example.tasks"]},{"x":1}]}"""),
)
assertEquals(emptyMap<String, List<PushRouting.Route>>(), PushRouting.parseRoutes(null))
}
}
@@ -0,0 +1,99 @@
package com.anonymous.bulwarkmobile.sync
import org.junit.Assert.assertEquals
import org.junit.Test
class SyncReportsTest {
private fun report(outcome: String, extra: String = "") = """
{"v":1,"runId":"r","authority":"com.android.contacts","outcome":"$outcome",
"startedAt":0,"durationMs":5,"conflicts":0,"itemErrors":[],
"stats":{"downloaded":{"created":3,"updated":2,"deleted":1},
"uploaded":{"created":1,"updated":4,"deleted":2},"entries":13,"skipped":1}$extra}
""".trimIndent()
@Test
fun `a run without a report is a soft error`() {
assertEquals(SyncOutcome(ioErrors = 1), SyncReports.outcome(null, "task finished"))
}
@Test
fun `an unreadable report is a soft error`() {
assertEquals(SyncOutcome(ioErrors = 1), SyncReports.outcome("{not json", "reported"))
}
@Test
fun `ok sums both directions into the framework's stats`() {
assertEquals(
SyncOutcome(inserts = 4, updates = 6, deletes = 3, entries = 13, skipped = 1),
SyncReports.outcome(report("ok"), "reported"),
)
}
@Test
fun `io keeps the progress so the framework retries at once`() {
val outcome = SyncReports.outcome(report("io"), "reported")
assertEquals(1L, outcome.ioErrors)
assertEquals(4L, outcome.inserts)
}
@Test
fun `auth is hard and reports no progress, or the framework would retry it at once`() {
assertEquals(
SyncOutcome(authErrors = 1, entries = 13, skipped = 1),
SyncReports.outcome(report("auth"), "reported"),
)
}
@Test
fun `permission, unsupported and safety aborts are database errors without progress`() {
for (outcome in listOf("permission", "unsupported", "safetyAbort")) {
assertEquals(
outcome,
SyncOutcome(databaseError = true, entries = 13, skipped = 1),
SyncReports.outcome(report(outcome), "reported"),
)
}
}
@Test
fun `too many deletions reports the pending deletions for the system notification`() {
assertEquals(
SyncOutcome(tooManyDeletions = true, deletes = 42, entries = 13, skipped = 1),
SyncReports.outcome(report("tooManyDeletions", ""","tooManyDeletions":{"count":42,"threshold":20}"""), "reported"),
)
}
@Test
fun `delayUntil and moreRecordsToGet reach the framework`() {
val outcome = SyncReports.outcome(report("ok", ""","delayUntil":1790000000,"moreRecordsToGet":true"""), "reported")
assertEquals(1790000000L, outcome.delayUntil)
assertEquals(true, outcome.fullSyncRequested)
}
@Test
fun `cancelled and disabled runs are not errors`() {
for (outcome in listOf("cancelled", "disabled")) {
assertEquals(0L, SyncReports.outcome(report(outcome), "reported").ioErrors)
}
}
@Test
fun `an outcome this build does not know is retried`() {
assertEquals(1L, SyncReports.outcome(report("somethingNew"), "reported").ioErrors)
}
@Test
fun `the log summary names the outcome, failed items and the message`() {
assertEquals("outcome=ok", SyncReports.summary(report("ok")))
assertEquals(
"outcome=internal message=Invalid column x",
SyncReports.summary(report("internal", ",\"message\":\"Invalid column x\"")),
)
val withItems = report("ok").replace("\"itemErrors\":[]", "\"itemErrors\":[{\"ref\":\"a\"},{\"ref\":\"b\"}]")
assertEquals("outcome=ok itemErrors=2", SyncReports.summary(withItems))
assertEquals("no report", SyncReports.summary(null))
assertEquals("unreadable report", SyncReports.summary("{"))
val long = SyncReports.summary(report("io", ",\"message\":\"" + "m".repeat(400) + "\""))
assertEquals("outcome=io message=".length + 300, long.length)
}
}
+9 -4
View File
@@ -26,10 +26,12 @@ module.exports = {
expo: {
name: 'Bulwark Mobile',
slug: 'bulwark-mobile',
// mailto: lets Android/iOS offer the app for mail links in other apps.
scheme: ['bulwarkmobile', 'mailto'],
// mailto: lets Android/iOS offer the app for mail links in other apps;
// bulwarkmail: is the webmail's sign-in link (bulwarkmail://pair?...).
// Android's manifest is maintained by hand and lists the same schemes.
scheme: ['bulwarkmobile', 'bulwarkmail', 'mailto'],
version: VERSION,
orientation: 'portrait',
orientation: 'default',
icon: './assets/icon.png',
userInterfaceStyle: 'automatic',
newArchEnabled: true,
@@ -56,7 +58,7 @@ module.exports = {
android: {
adaptiveIcon: {
foregroundImage: './assets/adaptive-icon.png',
backgroundColor: '#09090b',
backgroundColor: '#db2d54',
},
edgeToEdgeEnabled: true,
predictiveBackGestureEnabled: false,
@@ -97,6 +99,9 @@ module.exports = {
],
extra: {
commit: COMMIT,
// Settings → About → Source code. A modified build sets SOURCE_CODE_URL
// to where its own source is (AGPL §13); the default is upstream.
sourceCodeUrl: process.env.SOURCE_CODE_URL || 'https://github.com/bulwarkmail/native',
},
},
};
Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 13 KiB

Some files were not shown because too many files have changed in this diff Show More